Vacancy expired!
- This position requires a bachelor's degree in Computer Science or a related technical field, and 5+ years of relevant employment experience.Experience with Big Data technologies such as Hadoop/Hive/Spark specific to AWS related services is a plus.Expertise in ETL optimization, designing, coding, and tuning big data processes using Apache Spark or similar technologies.Sound knowledge of distributed systems and data architecture - design and implement batch and stream data processing pipelines, knows how to optimize the distribution, partitioning, and MPP of high-level data structures.5+ years of work experience with building Data Pipelines, Data Processing, Data Modeling, and Data Architecture.Experience operating very large data warehouses or data lakes. (Snowflake, etc)Excellent skills in writing and optimizing SQL.Knowledge of Engineering and Operational Excellence using standard methodologies.
- Knowledge of IT, service-oriented architectures, software development life cycles, or information security platforms and applications.
- Minimum 5+ years of experience in software development.
- 3+ years of related industry experience in an enterprise environment.
- 5+ years of data engineering experience.
- Scala / Python, PySpark(Boto, Boto3, etc.)/ Spark experience.
- Delta lake, delta table and lakehouse architecture
- Datawarehouse Experience (Snowflake)
- Experience with lambda, EMR, SQS, DynamoDB, Glue, Stepfunctions, etc.
- Linux and shell scripting.
- Kubernetes.
- Formal design patterns and industry best-practices.
- 2+ years of experience with requirements, design, implementation, integration, and testing for data and analytics integration.
- 2+ years of experience across a variety of technologies such databases, directory services, application servers, network infrastructures, Linux operating systems, and an understanding of fundamental security and data flows within these components.
- Excellent verbal and written communication skills.
- Self-motivated, driven, and creative individual.
- Scaling systems and microservices.
- Familiarity with CI/CD processes
- Code coverage analysis / static analysis tools.
- Agile programming processes and methodologies such as Scrum.
- Scheduling tools like Autosys, ControlM.
- Informatica IICS, Talend
Vacancy expired!