Vacancy expired!
In this role, you will be responsible for expanding and evolving the data pipeline framework of a cloud based data platform. The ideal candidate will have experience moving and processing large volume of data across on premise and cloud environments in a secure manner. The framework engineer will support software developers, data architects, data analysts and business users to get the most value out of the platform with minimal friction. The candidate must be self-directed with ability to own and execute the platform improvement activities while collaborating with other team members and stakeholders.
- 5-7 years of core Python development experience including setting up development environments, using code repository such as Bitbucket, requesting and performing pull requests, configuring Python libraries and more required.
- 5-7 years of experience in developing, deploying and running data pipelines using Python, and SQL.
- 4-5 years of experience in architecture, designing and operationalizing including data lakes, data warehouses and data marts, data layering, data virtualization/physicalization, data normalization/denormalization, data storage and movement patterns, data supply chains, and data catalogs.
- 5-7 years of experience in building and maintaining data platforms, ETL processes and connected sources and targets required.
- 5-7 years of experience with Data Management.
- 3-5 years of performance measurement and tuning experience preferred.
- AWS Cloud Services: S3, CloudWatch, RDS, EMR (Hive), Redshift Spectrum, EC2, VPC, and Step Functions.
- Experience implementing enterprise systems with security best practices and site reliability engineering principles.
- Deep understanding of the design and development considerations around data partitioning, job scheduling, data versioning, data import/export, archival, and schema management.
- Experience creating data warehouses that process large volume of data in a specific time window.
- Experience merging, separating, and sunsetting data warehouses.
- Implement the best practices and design patterns for the data lake, enterprise data warehouse, and domain specific data marts.
- Experience in the financial services or banking industry required.
Vacancy expired!