Vacancy expired!
Job Summary:
- A client of ours in Seattle WA is looking for a Sr. Data Pipeline Engineer for a Fulltime opportunity.
- AURORA
- SPARK
- AIRFLOW
- DOCKER
- AWS
- Our combined Disney and Hulu team is looking for hardworking team players to join the Ad
- Engineering team, who will thrive upon the legacy of Disney and Hulu, embrace unconventional
- thinking, and who are passionate about contributing to The Walt Disney Company's direct-to-consumer (DTC) experience through strategic hard work and determination.
- We are looking for a Senior Data Pipeline Engineer with experience in building data pipeline
- solutions integrating components built on top of an AWS technology stack. If you have
- experience building financial applications, take ownership in the technical direction of your
- team's product we'd like to talk with you.
- The Data Pipeline Engineer will be responsible for building efficient data pipelines that populate
- our data lake, apply calculations and aggregations across the data set and load the results into
- SQL databases that serve both analytical and operational use cases. This role will be working
- closely with different engineering teams and product managers to meet data requirements of
- various initiatives in Ad Engineering.
- Think and communicate critically about architecture, design, and best practices and guide your team to adopting them.
- Design data systems that allow managed growth of the data model to minimize risk and cost of change.
- Write transformation and validation code that applies complex data aggregation and calculation using SQL and Python
- Drive implementation of automated testing for data pipelines within a CI environment
- Create new pipelines or rewrite existing pipelines and build reusable components at scale to support accounting functions, as well as reporting & analytics.
- Collaborate with other Disney and Hulu teams to identify and document shifting data requirements while also advocating for a minimal change set for your team.
- Solve complex data issues and perform root cause analysis to proactively resolve product and operational issues.
- Collaborate with leadership and other engineers to develop technical story backlog derived from high level business requirements and design collaboration and estimating story points.
- BS or MS in Computer Science, a related field, or equivalent industry experience
- 3 years of professional experience engineering complex, high-volume data pipelines using SQL, Python, and Airflow
- 3 years of experience building cloud scalable and high-performance data lake / data warehouse solutions using AWS products - S3, Athena, Glue, and EMR
- Experience with binary data serialization formats such as Parquet
- Deep understanding of data structures and algorithms
- Understanding of code versioning tools such as GIT
- Have a passion for data solutions
- Exposure to AWS cloud data pipeline tools such as Managed Airflow and Glue
- Experience integrating with Ad Tech platforms such as Operative and STAQ
- Exposure and opinions regarding alternate orchestration tooling beyond Airflow
- Understanding of SOX compliance needs and how they affect system design.
- Have worked with a variety of Airflow Operator types, including REST, Lambda, ECS
- Can flex between Python and Javascript/Typescript.
- Aurora/Hive (databases)
- Spark (large-scale data processing)
- Airflow (workflow management)
- Docker (software packaging and delivery)
- AWS (development and hosting)
- BS/MS in Computer Science or sim. field
- This team is considered Hybrid so looking for someone that's close to a Hulu Office. So.Cal/Seattle/NYC - Can be remote to start but expectation is in office part time if needed.
Vacancy expired!