Vacancy expired!
Location: New York City, NY Description: Our client is currently seeking a Sr. Data Software Engineer
This job will have the following responsibilities:- Design and implement data pipelines, building scalable and optimized enterprise data systems. We use Airflow and the tasks are written in Python.
- Responsible for architecting, building, launching and maintaining scalable data pipelines to support our data processing, analytics and product needs.
- Acts as a critical member of a team that has ownership of the core data systems that handle all of our incoming events and transform then in both stream and batch processes
- Experience working with streaming systems such as Kafka or Kinesis. This team owns a large Java application built on top of Kinesis.
- Grounded understanding of SQL and deeper understanding of at least one commonly used DB. We use Snowflake heavily as a datastore.
- Design and implement data pipelines, building scalable and optimized enterprise level data systems
- Working with our SRE teams to properly manage and scale the architecture in a cost-efficient way
- Collaborate with project stakeholders to identify product and technical requirements. Conduct analysis to determine integration needs
- Work with and mentor junior Software Development Engineers.
- Evolve data model and data schema based on business and engineering needs focusing on DRY principles and an easy of maintainability.
- Develop tools supporting self-service data pipeline management (ETL)
Vacancy expired!