Description

Key Skills: Azure Devops, Pyspark, Azure Databricks, Azure Data lake, ETL, Azure SQL

Roles and Responsibilities:

• Design and develop scalable data ingestion and transformation pipelines using Azure Data Factory and Azure Databricks.
• Build and optimize ETL/ELT processes for various data types.
• Implement data quality checks and monitoring mechanisms.
• Design and implement Lakehouse architectures using Azure Databricks.
• Support CI/CD implementation using Azure DevOps.

Skills Required:

• 4 - 6 years of relevant experience in data engineering.
• Strong hands-on experience in Azure Databricks development using PySpark and SQL.
• Experience building enterprise data pipelines using Azure Data Factory (ADF).
• Strong understanding of Data Lake and Lakehouse concepts.
• Experience with CI/CD implementation using Azure DevOps.

Good to Have:

• ETL
• Azure SQL

Education: Bachelor's degree in Computer Science or related field

Key Skills

Education

Any Graduate