Description

You will design, build, and optimize large-scale data solutions on the Databricks Unified Analytics Platform.

This role is remote.

Responsibilities

• Design and develop scalable data pipelines using Apache Spark on Databricks.
• Build and optimize ETL/ELT workflows for structured and unstructured data.
• Integrate Databricks with Azure Data Factory and other Azure services.
• Automate deployments through CI/CD and DevOps pipelines.
• Implement data validation, governance, and security using Unity Catalog and Delta Lake.

Required Skills

• 12+ years of experience in data engineering or analytics.
• Advanced proficiency in Databricks, Apache Spark, and Delta Lake.
• Strong coding skills in Python, R, and SQL.
• Deep understanding of Spark architecture, including RDDs, DataFrames, and Spark SQL.
• Hands-on experience with Azure Cloud, Data Factory, DevOps, and CI/CD.
• Experience working in agile, multicultural environments.

Key Skills

Education

Any Graduate