Description

Key Skills: Azure Devops, Pyspark, Azure Databricks, Azure Data lake, ETL

Roles and Responsibilities:

• Design and develop scalable data ingestion and transformation pipelines using Azure Data Factory and Azure Databricks.
• Build and optimize ETL/ELT processes for various data types.
• Implement data quality checks and monitoring mechanisms.
• Collaborate with stakeholders to understand data requirements and translate them into technical solutions.
• Support CI/CD implementation using Azure DevOps.

Skills Required:

• Strong hands-on experience with PySpark.
• Experience with Azure Databricks.
• Good knowledge of Azure Data Factory (ADF).
• Experience working with Azure Data Lake.
• Experience with Azure DevOps and CI/CD pipelines.
• Good understanding of ETL/ELT concepts.
• Experience working with large datasets and data transformation.
• Good troubleshooting and problem-solving skills.
• Good communication and collaboration skills.

Good to Have

• Experience with SQL and relational databases.
• Knowledge of data warehousing concepts.
• Experience with cloud-based data engineering solutions.
• Knowledge of data quality and monitoring tools.

Education: Bachelor's degree in a relevant field

Key Skills

Education

Any Graduate