Vacancy expired!
Azure Data Engineer - multiple roles Platform: SQL Databases, ELT and ETL tools, Microsoft Azure: Azure SQL Database, Data Factory, SQL Data Warehouse, App Services (Web, API, Logic apps), Azure Functions, Log Analytics, networking & security, Azure storage, Service Bus, ExpressRoute/VPN, Azure Active Directory, Azure Dev Ops, Azure ARM templates Technologies: SQL, PolyBase, Python, PySpark, Spark internals, PowerShell, Bash, C#, JavaScript, Entity Framework, RESTful services, Microservices, Visual Studio Qualifications 12+ years of professional solution delivery experience and a Bachelor of Computer Science, MIS or equivalent degree 10+ years of experience as an ETL developer in 1 or more: Informatica, SSIS, Ab Initio, SAP Data Services 3-5 years of designing and building modern data pipelines and data streams in the Azure/Databricks environment and deployed the solution in Production 3-5 years of designing Azure data storage solutions (SQL Database, SQL Data Warehouse, Data Lake Storage) 3-5 years of developing data ingestion, data processing and data optimization (Databricks, Data Factory, Informatica, PolyBase) 3-5 years of developing complex scripting leveraging Python, Pyspark and SQL Experience in delivering solutions within a Massively Parallel Processing (MPP) environment Experience in solving complex ASDW query performance and defining execution rules Experience in curation process and leveraged Data Flows, Data Bricks, Delta lake for Slowly Changed dimensions Experience in handling various formats viz. Avro, Parquet, Delta Azure Certification a plus Experience in designing and developing User interfaces using .NET technologies Understating of Master Data Management (MDM) and data quality tools and processes Understanding of Lambda and Kappa architecture patterns Understanding of DevOps including CI/CD Experience developing software architectures and key software components High-level understanding of common authentication patterns and flow including single sign-on and OAuth Experience with Agile/Scrum methodology Ability to apply technology and consulting to solve a client business problem Sufficient depth and breadth of technical knowledge to be individually responsible for the design and scope of deliverables of a data warehouse solution Able to communicate and present complex issues with preciseness, assurance, and confidence A disciplined approach to software development and problem solving Passion for technology and a high technical aptitude Insightful and always looking for breakthrough ideas Self-sufficient, high integrity, more than just competent Responsibilities Collaborate with data engineers, data architect, business analysts to build a modern data warehouse application on Azure Perform POCs on various concepts using Azure services as required Ingest, curate data using Data Factory and Data Bricks Design, develop dimensional model in MPP environment using Azure Synapse Analytics Understand the data architecture design, solution structures and component design Ensure that technical software development process is followed on the project, be familiar with industry best practices for software development and agile practices Understand a broad spectrum of technology in order to provide part or all of a detailed technical design which meets customer requirements Act as senior data engineer on customer projects to deliver a wide-range of PaaS-focused solutions Participate in standups, agile breakout sessions and independently review the deliverables in a timely manner Communicate across the client's community consistently viewed as adding value Contribute knowledge, tools, and positivity to the rest of internal Data Engineer community
Vacancy expired!