Description

Key Responsibilities

• Design, develop, and maintain data pipelines using Azure Data Factory.
• Build and manage enterprise-scale data lakes using Azure Data Lake Storage.
• Develop ETL/ELT workflows to ingest data from multiple sources, including databases, APIs, flat files, and cloud applications.
• Implement data transformation processes using Mapping Data Flows, SQL, and Spark technologies.
• Monitor, troubleshoot, and optimize data pipelines for performance and reliability.
• Configure incremental data loads, Change Data Capture (CDC), and data validation processes.
• Collaborate with data analysts, business stakeholders, and development teams to understand data requirements.
• Design data models and support data warehousing solutions.
• Implement security, governance, and access controls for cloud data platforms.
• Create technical documentation and maintain data architecture standards.
• Support the migration of on-premises data solutions to Azure cloud environments.

Required Qualifications

• Expertise in Azure Data Factory for data pipeline development and maintenance.
• Experience with Azure Data Lake Storage and enterprise-scale data lakes.
• Experience developing ETL/ELT workflows for data ingestion from multiple sources.
• Knowledge of Mapping Data Flows, SQL, and Spark technologies for data transformation.
• Experience with incremental data loads, Change Data Capture (CDC), and data validation.
• Ability to troubleshoot and optimize data pipelines for performance and reliability.
• Knowledge of data modeling, data warehousing, and cloud data platform security and governance.
• Experience supporting on-premises data migration to Azure cloud environments

Key Skills

Education

Bachelor's degree