Vacancy expired!
- We are looking for a qualified Hadoop developer to join our project team and help drive build out of our initial pilot implementation.
- We will be focusing on the data model, data loads, and native PySpark API and insuring highly scalable architecture adhering to the industry best practices and utilizing structured and unstructured storage tuned for very large data volumes and performance
- Candidate will play a key role of assisting and guiding larger project team during the ramp up phase with
- Big Data technologies and establishing a larger team expertise
- Phase 1 will focus on building incremental data loading and ETL pipeline using Hadoop tools and Spark integrated into existing production data flow designed to allow both appends and updates.
- You'll be securing the data by designing and implementing highly performance mechanism that would support both record and columnar data entitlements, and expanding and tuning initial API design utilizing PySpark runtime to support Big Data analytics use cases for native Hadoop/Spark API clients
- Over time this platform may need to be extended web interface built on top of it for downstream systems and end user applications to connect
- Ideal candidate would be a strong problem-solver able to operate independently with minimal supervision, team player, quick learner and passionate about Big Data and new technologies, work well under pressure meeting aggressive deadlines while delivering quality software
- In-depth understanding of Hadoop architecture, data modeling, data security/authentication, and performance tuning to support very large data volumes using HDFS and Hive, solid experience with parallel processing using Spark, MapReduce, and Hadoop Yarn
- Strong software engineering background with the working knowledge of Python, background in Java, Scala, C, C# would be considered if willing to learn Python
- Experience with real-time messaging technologies such as Kafka or AMPS, RESTful API and Web server technologies such as Tornado or Apache
- Prior banking industry experience with large scale big data projects supporting trading risk functions very helpful
- Able to work as a W2 employee of Genesis10 (no Corp-to-Corp)
Vacancy expired!