Description

Key Skills: Pyspark, ETL, Azure Databricks, Databricks, MySQL, Python

Roles and Responsibilities:

• Design, develop, and maintain scalable ELT pipelines using Databricks, Delta Live Tables, PySpark, SQL, and Python across Bronze/Silver/Gold lakehouse layers.
• Build and optimize Delta Live Tables (DLT) pipelines with data quality expectations, lineage visibility, and auto-healing behavior.
• Ingest and integrate data from sources including databases and connectors into the lakehouse, ensuring consistent transformations and reusable shared code packages.
• Implement data quality frameworks (schema checks, freshness/accuracy SLAs, completeness monitoring) and maintain table-level documentation for data contracts and lineage.
• Operate and monitor end-to-end pipelines, track job execution and SLA adherence, perform RCA for pipeline incidents, and support change requests and controlled releases across environments.

Skills Required:

• 1.5 - 3 years of hands-on data engineering experience.
• ETL development experience with ELT patterns on Databricks lakehouse workflows.
• Hands-on PySpark and Databricks development (jobs, notebooks, DLT pipelines, Delta Lake).
• Azure Databricks experience and ability to work within Azure cloud environments.
• Strong SQL and Python coding skills for transformations, data quality checks, and pipeline logic.

Good to Have:

• MySQL and additional database integration experience.

Education: Any Graduation or Any Post Graduation

Key Skills

Education

Any Graduate