Vacancy expired!
- Selecting and integrating any Big Data tools and frameworks required to provide requested capabilities.
- Implementing ETL process
- Monitoring performance and advising any necessary infrastructure changes
- Knowledge of various persistence (RDBMS, NoSQL, HDFS, Cassandra, Redis)
- Understanding Data Catalog, Data Governance, Data Lineage
- Experience with security, authentication in data platform
- Bachelor' s Degree or above (Computer Science, Bio Engineering, Electronics and Electrical Engineering or any related field)
- 5+ years of experience in data engineering
- Proficient understanding and experience in one or more programming languages (Python or Scala.)
- Experience with building stream-processing systems, using solutions such as Spark-Streaming or Flink
- Experience with integration of data from multiple data sources
- Experience with building data lakes and data warehouses by leveraging any of the major cloud providers (Azure)
- Familiar with Hadoop ecosystem (HDFS, HBase etc.), especially Spark
- Good knowledge of Big Data querying tools
- Knowledge of various ETL techniques
- Knowledge of Spark MLlib
- Knowledge of messaging systems, such as Kafka or RabbitMQ
Vacancy expired!