Description

Key Skills: Azure Data lake, Azure Databricks, Azure ADF, Azure Synapse, ETL, SQL, Python, Power BI, Jira, Kubernetes, Agile Methods, Data quality, Azure Devops, Informatica

Roles and Responsibilities:

• Design, develop, and maintain data pipelines and ETL workflows using Azure Data Factory and Azure Databricks.
• Build and optimize data ingestion, transformation, and orchestration across Azure Data Lake Storage and Azure Synapse environments.
• Develop data transformation logic in Python and/or PySpark notebooks, including basic Delta Lake concepts.
• Support data migration from on-premise sources to Azure cloud data platforms and modernize existing pipelines.
• Implement pipeline reliability with validation, error handling, and logging, and apply security best practices (IAM/Key Vault) for Azure data services.

Skills Required:

• 7-12 years of relevant experience designing and delivering Azure-based data pipelines and ETL workflows.
• Azure Data Factory (pipelines, triggers, linked services) and Azure Data Lake Storage (file system hierarchy and permissions).
• Azure Databricks (notebooks, PySpark, Delta Lake basics) and Azure Synapse Analytics and/or Azure SQL Database.
• Strong SQL and Python skills for data extraction and transformation, including ETL/ELT pipeline design and data modeling fundamentals.
• Basic DevOps/CI-CD and version control using Git, with exposure to performance tuning, monitoring, and logging in Azure data pipelines.

Good to Have:

• Power BI for reporting and data visualization.
• Informatica and/or ADF Mapping Data Flows for ETL development.
• Azure DevOps, Azure DevOps pipelines, and JIRA; familiarity with Agile methodologies.
• Knowledge of data quality and data governance concepts; exposure to Docker/Kubernetes.

Education: Bachelor's Degree in Computer Science or related field

Key Skills

Education

Any Graduate