Vacancy expired!
- 5+ years of hands-on experience with big data
- Knowledge of Programming/Scripting Languages including experience in Python, Java or Scala – Core; Must have experience in object-oriented programming concepts and Shell Scripting;
- Knowledge of RDBMS/Databases including experience in ANSI SQL;
- Must have Hadoop/Big Data including HDFS and Spark.
- Experience in any Cloud technologies (Azure is preferred, but AWS or Google Cloud are OK as well)
- Knowledge of Skills and Design Patterns including Statistics, Data Visualizations and Microservices preferred; Must have experience in REST API, Data Modelling and Performance Tuning
- Experience in Oracle/MySQL/Sybase; Knowledge of MongoDB (Object Stores) and Snowflake
- Experience in Job Scheduling including ControlM, Crontab, Airflow or Autosys
- Experience with Libraries including Python Pandas, Python ETL, Python or Java
- Experience in Hive/Spark SQL, Sqoop and Data warehousing; Knowledge of HBase, Phoenix, Datameer (or other analytical tools), PowerBI (or other reporting tool i.e., Tableau) and SAS/Dataflex preferred
- Experience in Data Factory or other Cloud ETL tools and Cloud Data Lake (Storage); Knowledge of Databricks; Knowledge of Azure Batch, Delta Lake, Azure HDInsight, Cosmos DB, Azure EventHub and Synapse preferred.
- Knowledge of Backoffice Full Stack including Servlet, Spring Framework, HTML, Javascript, CSS, React and HighCharts preferred
- Experience in Project Management/Agile including SVN/GIT/GIThub; Knowledge of Azure DevOps and CICD preferred
- Knowledge of Tools including Experience in IDE – IntelliJ/Eclipse/PyCharm/Visual Studio and DBeaver/SQL developer or other SQL development tools
- Knowledge of Hortonworks/Cloudera – Ambari and Trifacta
- 5+ years of hands-on experience with big data
- Responsible for big data engineering, data wrangling, data analysis and user support primarily focused on the “Cloudera Hadoop” platform, currently extending to the cloud.
- Strong hands-on technical skills as well as being able to mentor and train other Engineers on conventional ETL and SQL skills with programming as well as data science languages such as Python and R, using big data techniques.
- Proactively analyze the business needs, profile large data sets and build custom data models and applications to drive business decision making and customers experience
- Build workflows that empower analysts to efficiently use data components for various data engineering functions areas.
- Perform requirements analysis, planning and forecasting for Hadoop data engineering/ingestion projects
- Design optimized Hadoop and big data solutions for data ingestion, data processing, data wrangling, and data delivery
- Design, develop tune data products, streaming applications, and integrations on large-scale data platforms (Hadoop, Kafka Streaming, Hana, SQL server, Data warehousing, big data) with an emphasis on performance, reliability and scalability, and most of all quality.
- Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability
- Build the infrastructure required for efficient extraction, transformation, and loading of data from a wide variety of data sources
- Build data tools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader.
- Develop custom data models and algorithms
- Identify opportunities for data acquisition
- Peer review of the code developed by team members
- Propose recommendations to streamline processes for efficiency and effectiveness.
- Work in multi-functional agile teams to continuously experiment, iterate and execute on data-driven product objectives.
- Identify and resolve day-to-day issues to ensure continuous improvement.
- Network with colleagues to share knowledge and gain new perspectives (Mgmnt/IC Track).
Vacancy expired!