Vacancy expired!
Lead Data Engineer with Data Pipeline experience Location: OAKS, PA Fulltime Permanent Role Mandatory skills required:
PySpark KAFKA Python Oracle Experience on Building Data Lake on Azure cloud Snowflake knowledge & experience would be a plus. Job Description Strong hands-on experience for developing ETL data pipeline implementation in cloud native infrastructure Strong experience in PySpark and Python for building data pipelines activities Experience in data ingestion programs to process large data sets in real-time and batch data pipelines that bring together data from disparate sources using PySpark, SparkSQL, Kafka, and Spark Streaming API etc. Experience on various Spark transformations, actions and data frames for cleansing, preparing the input data before loading into target systems and databases Experience on building Data Lake in On-premise and cloud will be a plus Hands-on experience in SQL, PL/SQL and query tuning in Oracle Database Excellent knowledge of performance tuning in SQL and data partitioning Strong expertise in implementing Golden Gate Replication, Bi-directional replication using Golden Gate. Experience in design technical solutions in the above areas Experience in team management and guiding developers on the technical front 8+ years of professional experiences in the area of software development and system analysis and design. Strong experience in PySpark and Python for building data pipelines activities Experience in data ingestion programs to process large data sets in real-time and batch data pipelines that bring together data from disparate sources using PySpark, SparkSQL, Kafka and Spark Streaming API etc. Experience on various Spark transformations, actions and data frames for cleansing, preparing the input data before loading into target systems and databases Experience on building Data Lake in On-premise and cloud will be a plus Hands-on experience in SQL, PL/SQL and query tuning in Oracle Database Excellent knowledge of performance tuning in SQL and data partitioning Vacancy expired!
