Vacancy expired!
TITLE: Data Engineer
Location: Remote Position SummaryOur Global BI and Analytics team is growing and we're looking to hire a Data Engineer. In this role, you will demonstrate broad and deep knowledge of ETL development in both on-prem SQL Server dimensional and relational databases and Azure Cloud environments. The Data Engineer will support our data analysts and data scientists on data initiatives and ensure optimal data delivery architecture is consistent throughout ongoing projects. You will engage in supporting the data needs of multiple teams, systems and products. Do you find the prospect of optimizing or even re-designing our company's data architecture to support our next generation of products and data initiatives most exciting? We really should explore together.This is a hybrid position, so there may be a need to be in the office here and there but for the most part you can work nearly entirely remote/virtual/work from home.Essential Duties And Responsibilities- Design ETL jobs and reusable components to implement specified business requirements Troubleshoot and optimize ETL code; interpret ETL logs, perform data validation, understand the benefits and drawbacks of parallelism, proper use of expressions, scoping of variables, commonly used transforms, event handlers and logging providers, understand and optimize the surrogate key generation and inconsistent data type handling
- Solidify and extend existing ETL Framework using SSIS, Python and T-SQL.
- Build the infrastructure required for optimal ETL/ELT pipelines to ingest data from a wide variety of data sources using Microsoft Azure technologies such as Azure Data Factory and Databricks.
- Create data tools for data analytics and data science team members to deliver actionable insights into customer acquisition, operational efficiency and other key business performance metrics.
- Conduct code reviews and participate in technical design
- Conduct performance analysis and optimize ETL/ELT jobs, custom SQL and T-SQL jobs
- Orchestrate large, complex data sets that meet functional/non-functional business requirements.
- Seek out, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability.
- Partner with data and analytics talent to strive for greater functionality in our data systems.
- A relevant technical BS Degree in Information Technology and 8 years of experience, or 12 years of relevant professional experience implementing well-architected data pipelines that are dynamically scalable, highly available, fault-tolerant, and reliable for analytics and platform solutions
- 6+ years writing SQL queries against any RDBMS with query optimization and performance tuning
- 5+ years of experience implementing ETL for Data Warehouse and Business intelligence solutions
- Deep familiarity with MS SQL Server 2012 - 2019 and Azure database technologies
- Working experience with Python; familiarity with .NET and C# framework is a plus
- 3+ years with SSIS and T-SQL/ stored procedures
- Skills to read and write effective, modular, dynamic, parameterized and robust code, establish and follow already established code standards, and ETL framework
- Experience in performance tuning of the SSIS packages, deploying and administering SSIS packages, proficient in utilizing scripting in SSIS
- Understanding SQL Server / External scheduling: ability to select the most optimal scheduling solution and call SSIS jobs from the command line or from the Scheduling tool locally / remotely
- Strong analytical, problem solving, and troubleshooting abilities, experience performing root cause analysis
- Good understanding of unit testing, software change management, and software release management
- Knowledge of Dev-Ops processes (including CI/CD) and Infrastructure as Code fundamentals
- 3+ years of data engineering experience leveraging technologies such as Azure Data Factory, ADLS Gen 2, Logic Apps, Azure Functions, Databricks, Apache Spark, SQL DW
- Understanding the pros and cons, and best practices of implementing Data Lake, using Microsoft Azure Data Lake Storage
- Experience structuring Data Lake for the reliability, security and performance
- Experience working within an agile team; in-depth knowledge of agile process and principles
- Excellent communication skills
Vacancy expired!