Description

Design, develop, and maintain data pipelines and ETL workflows using Azure Data Factory and Azure Databricks.

This role is remote.

Responsibilities

• Build and optimize data ingestion, transformation, and orchestration across Azure Data Lake Storage and Azure Synapse.
• Develop data transformation logic in Python and PySpark notebooks, applying Delta Lake concepts.
• Support data migration from on-premise sources to Azure cloud platforms and modernize existing pipelines.
• Implement pipeline reliability with validation, error handling, logging, and security best practices (IAM/Key Vault).

Required Skills

• 7-12 years of experience designing and delivering Azure-based data pipelines and ETL workflows.
• Hands-on expertise with Azure Data Factory (pipelines, triggers, linked services) and Azure Data Lake Storage.
• Proficiency in Azure Databricks (notebooks, PySpark, Delta Lake basics) and Azure Synapse Analytics.
• Strong SQL and Python skills for data extraction, transformation, and ETL/ELT pipeline design.
• Experience with basic DevOps/CI-CD, Git version control, and performance tuning in Azure.
• Bachelor's Degree in Computer Science or related field.

Preferred Skills

• Power BI for reporting and data visualization.
• Informatica or ADF Mapping Data Flows for ETL development.
• Familiarity with Azure DevOps, JIRA, Agile methodologies, Docker, or Kubernetes.

Key Skills

Education

Any Graduate