Description

You design, build, and optimize scalable data pipelines using Databricks, Python, and database engineering principles to support analytics and business intelligence.

This role is on-site.

Responsibilities

• Design, build, and maintain ETL/ELT workflows on Databricks using Delta Lake and Spark SQL.
• Develop scalable data pipelines and optimize transformation logic for performance and cost efficiency.
• Manage data models, schemas, and tables across SQL and NoSQL databases.
• Implement data quality, validation, and monitoring frameworks to ensure reliability.
• Troubleshoot pipeline failures and integrate data from APIs, streaming sources, and cloud storage.

Required Skills

• Databricks (mandatory) – Delta Lake, Notebooks, Spark SQL, Jobs, Workflows
• Python – Data processing, Spark APIs, automation, optimization
• Database Engineering – Strong SQL, query tuning, schema design, stored procedures
• Apache Spark – RDDs/DataFrames, optimization, partitioning
• Cloud Platforms – AWS, Azure, or GCP
• ETL/ELT tools – ADF, Glue, Dataflow, or equivalent
• CI/CD for data pipelines – Git, DevOps, Jenkins
• Experience with large-scale datasets

Preferred Skills

• Experience collaborating with Data Scientists and Cloud Engineers to support ML workloads
• Knowledge of data security, governance, and compliance standards

Key Skills

Education

Any Graduate