Vacancy expired!
- Build unique high-impact business solutions utilizing advanced technologies for use by world class clients.
- Create and maintain the underlying data pipeline architecture for the solution offerings from raw client data tofinal solution output.
- Create, populate, and maintain data structures for machine learning and other analytics.
- Use quantitative and statistical methods to derive insights from data.
- Guide the data technology stack used to build our solution offerings.
- Combine machine learning, artificial intelligence (ontologies, inference engines and rules) and natural language processing under a holistic vision to scale and transform businesses.
- Create and maintain optimal data pipeline architecture, incorporating data wrangling and ETL flows.
- Assemble large, complex data sets to meet analytical requirements - analytics tables, feature-engineering.
- Build the infrastructure required for optimal, automated extraction, transformation, and loading of data
- from a wide variety of data sources using SQL and other 'big data' technologies such as Databricks.
- Build automated analytics tools that utilize the data pipeline to derive actionable insights.
- Identify, design, and implement internal process improvements: automating manual processes,
- optimizing data delivery, re-designing infrastructure for greater scalability, etc.
- Design and develop data integrations and data quality framework.
- Develop appropriate testing strategies and reports for the solution as well as data from external sources.
- Evaluate new technology for use within the organization.
- Work with other client personnel to administer and operate client-specific instances of our solution offerings:
- Configure data pipelines to accommodate client-specific requirements to onboard new clients.
- Perform regular operations tasks to ingest new and changing data - implement automation where possible.
- Implement processes and tools to monitor data quality - investigate and remedy any data-related issues in daily solution operations.
- Minimum of a bachelor's degree in Computer Science or related field
- 3+ years hands on experience as a data engineer or similar position
- 3+ years of commercial experience with Python or Scala Programming Language
- 3+ years of SQL and experience working with relational databases
- Knowledge of at least one of the following - Databricks, Spark, Hadoop or Kafka
- Demonstratable knowledge and experience developing data pipelines to automate data processing work flows
- Demonstratable experience in data modelling
- Demonstratable knowledge of data warehousing, business intelligence, and application data integration solutions
- Demonstratable experience in developing applications and services that run on a cloud infrastructure (Azurepreferred)
- Excellent problem-solving and communication skills
Vacancy expired!