Vacancy expired!
Role: Sr Data Engineer (SQL, Python, Airflow) AWS Location: Santa Monica CA/ Cal/Seattle/NYC Hybrid - expected to be onsite 3-4 days a week Duration: Fulltime What you will do:
- Think and communicate critically about architecture, design, and best practices and guide your team to adopting them.
- Design data systems that allow managed growth of the data model to minimize risk and cost of change.
- Write transformation and validation code that applies complex data aggregation and calculation using SQL and Python
- Drive implementation of automated testing for data pipelines within a CI environment
- Create new pipelines or rewrite existing pipelines and build reusable components at scale to support accounting functions, as well as reporting & analytics.
- Collaborate with other Disney and Hulu teams to identify and document shifting data requirements while also advocating for a minimal change set for your team.
- Solve complex data issues and perform root cause analysis to proactively resolve product and operational issues.
- Collaborate with leadership and other engineers to develop technical story backlog derived from high level business requirements and design collaboration and estimating story points.
- BS or MS in Computer Science, a related field, or equivalent industry experience
- 3 years of professional experience engineering complex, high-volume data pipelines using SQL, Python, and Airflow
- 3 years of experience building cloud scalable and high-performance data lake / data warehouse solutions using AWS products - S3, Athena, Glue, and EMR
- Experience with binary data serialization formats such as Parquet
- Deep understanding of data structures and algorithms
- Understanding of code versioning tools such as GIT
- Have a passion for data solutions
- Exposure to AWS cloud data pipeline tools such as Managed Airflow and Glue
- Experience integrating with Ad Tech platforms such as Operative and STAQ
- Exposure and opinions regarding alternate orchestration tooling beyond Airflow
- Understanding of SOX compliance needs and how they affect system design.
- Have worked with a variety of Airflow Operator types, including REST, Lambda, ECS
- Can flex between Python and Javascript/Typescript.
- Aurora/Hive (databases)
- Spark (large-scale data processing)
- Airflow (workflow management)
- Docker (software packaging and delivery)
- AWS (development and hosting)
Vacancy expired!