Description

You will build and maintain scalable data pipelines and ETL processes within the Azure ecosystem.

This role is remote.

Responsibilities

• Design and develop data pipelines using Azure Databricks and Azure Data Factory.
• Implement and optimize Spark jobs, data transformations, and processing workflows.
• Integrate data from databases, APIs, and streaming sources into Databricks.
• Optimize data processing performance by managing cluster resources and Spark job efficiency.
• Automate pipeline and infrastructure deployment using Azure DevOps and CI/CD practices.

Required Skills

• 5+ years of experience in data engineering roles.
• Expertise in Azure Databricks.
• Proficiency in Azure Data Factory.
• Strong experience with Spark.
• Advanced Python programming skills.
• Expert-level SQL.
• Experience with Azure DevOps and CI/CD workflows.
• Ability to implement data governance and security policies.

Preferred Skills

• Degree in any graduate field.

Key Skills

Education

Any Graduate