Vacancy expired!
- Experience with Apache Airflow, including installing, configuring and monitoring Airflow cluster.
- Develop guidelines for Airflow cluster and DAG (Direct Acyclic Graph).
- Build DAG to orchestrate augmented pipeline implementing models in pipeline, running reconciliation, DQ, etc.
- Develop data pipeline to on-board and change management of datasets
- Implement CI/CD for Airflow deployment and test automation frameworks.
- Understanding of airflow rest services and integration of airflow of platform eco-system
- Orchestrating the Airflow / workflow in hybrid cloud environment is big plus
- Experience working in an Agile or iterative approach to delivery preferred.
- Should be hands on with: Python & Pandas, with demonstrated experience in ETL.
- Ability to maintain clean and secure data environments.
- Working in a hybrid cloud environment.
- Develop and contribute across the stack (Javascript/React, Java backend)
- Experience in setting and Schedulers: Developing hooks, templates, Alerts and scheduling for pipelines in Airflow
Vacancy expired!