Description

You build and maintain data products using Databricks in an Azure environment.

Responsibilities

• Build data products in Databricks using Scala/Spark.
• Manage operations for developed and deployed data products in production.
• Test data products against specifications and perform end-to-end validation.
• Set up monitoring, logging, and alerting for Spark jobs and data pipelines using Azure Monitor/Log Analytics or similar tools.
• Coordinate work with the offshore team in India.

Required Skills

• 5+ years of experience in data engineering on cloud data platforms.
• Hands-on experience building Spark jobs with Scala/Spark and/or PySpark on Databricks.
• Experience ingesting data from batch and streaming sources into ADLS Gen2 using Delta or Apache Iceberg tables.
• Strong SQL skills for joins, aggregations, and data quality checks.
• Understanding of core Azure data services (Event Hubs/Kafka, Data Factory/Databricks Workflows, Key Vault).
• Experience with Git-based workflows and CI/CD in Azure DevOps or GitHub.
• Proficiency with Databricks, Spark, Azure, Python, Cosmos DB, Azure DevOps, Docker, JUnit, Postman, and SonarQube.

Key Skills

Education

Any Graduate