Description

Build and maintain data pipelines using Azure Databricks and PySpark.

This role is on-site.

Responsibilities

• Develop and optimize data processing solutions using Azure Databricks and PySpark.
• Orchestrate data workflows and ETL processes with Azure Data Factory.
• Write efficient code in Python and Scala for data engineering tasks.
• Maintain CI/CD pipelines for data environments.
• Collaborate with stakeholders to define data requirements and deliverables.

Required Skills

• 6+ years of experience in data engineering roles.
• Strong proficiency in Azure Databricks and PySpark.
• Hands-on experience with Azure Data Factory.
• Proficiency in Python and Scala programming languages.
• Experience with CI/CD pipeline implementation.

Preferred Skills

• Experience with Azure SQL Database, Azure SQL Warehouse, or Synapse Analytics.
• Knowledge of Hive Programming.

Key Skills

Education

Any Graduate