Vacancy expired!
- Discovery and Documentation
- Analyze and document the ingestion tables and SAP Data Services Pipelines
- Analyze and document the SQL Stored procedures for the Right-Angle tables
- Document the .Net Process that makes updates into the Azure SQL DB
- Document the business processes that happen in the DataBricks
- Prepare the user stories for Data Ingestion and Preparation
- Prepare a detailed schedule of activities for the next phases based on the information gathered in the Discovery phase
- Data Preparation (Phase-1)
- 40 Tables (approx.) needs to be ingested from Right Angle DB
- SAP Data Services Transformation logics needs to be re-implemented in Azure
- Remove the SAP Data Services dependency for Risk Warehouse
- Utilize Azure Copy to move the data from On-Prem to the Azure Storage Account.
- Adjusting the .Net app to utilize Azure Synapse Tables
- Three-stage Code Migration (Phase-2)
- Redevelop the Synapse pipelines with Spark for the Ontime load process as required
- Create the Logic App for the alerts to be sent
- Validate codes and data sets against the current production system
- Continue working on the Pipelines in Synapse with Spark for the Daily process
- Create the Snapshot process using Spark.
- Convert the functions that are utilized in DataBricks to Spark
- Adjustment Process – Synapse Pipeline Generation (including Adjusting the .Net app to utilize Azure Synapse data)
- Implement the .Net Application to allow the data writes to Synapse
- Create the Utilities pipelines that are required from ADF to Synapse
- Validate all the mappings and process that are migrated to the Synapse pipelines
- Production Readiness
- Validate the code against the Synapse platform.
- Work with Business Users to test the application and data
- Validate .Net User input and update and validation process
- Validate the Data Snapshot process
Vacancy expired!