Vacancy expired!
Position: DATA ENGINEER WITH PYTHON AND PYSPARK Location: Austin, TX Onsite (Max 3-4 Week remote allowed) Duration: Long Term Exp Level: Need 10+ Years Job Description
- Design, development and testing of data ingestion pipelines, perform end to end automation of ETL process and migration of various datasets that are being ingested into the platform.
- Participate in discussion for technical architecture, Migration activities, collaborate with Product Managers and Architects to establish the physical application framework (e.g. libraries, modules, execution environments)
- Develop transformation logic, interfaces and reports as need to meet project requirements.
- 10+ years of experience building and managing complex products/solutions.
- 5+ years of experience supporting data heavy applications and databases leveraging Python and PySpark
- Strong data handling and text log analysis skills
- Expert level experience in Python, PySpark and SQL
- Good to have Scala, distributed computing, Kafka and REST based API skills.
Vacancy expired!