Description

You will design, develop, and maintain scalable data pipelines using Databricks for structured and unstructured data.

This role is hybrid.

Responsibilities

• Design and maintain scalable data pipelines using Databricks.
• Develop data transformations using PySpark and Spark SQL.
• Optimize ETL/ELT workflows and perform performance tuning for large-scale datasets.
• Monitor workflows, troubleshoot production issues, and ensure data quality.
• Collaborate with data scientists and stakeholders on data requirements.

Required Skills

• 10+ years of experience in data engineering roles.
• Strong hands-on expertise with Databricks and Apache Spark (PySpark / Spark SQL).
• Solid experience building data pipelines, data lakes, and data modeling.
• Proficient in SQL and workflow orchestration tools (Airflow, ADF).
• Experience with cloud platforms: AWS, Azure, or GCP.
• Strong debugging and performance tuning skills for large-scale datasets.
• BE / BTech / MCA / MSc in Computer Science, Data Engineering, or related field.

Preferred Skills

• Experience with Delta Lake and Databricks optimization techniques.
• Knowledge of CI/CD for data pipelines.
• Exposure to real-time or streaming data processing.

Key Skills

Education

Any Gradute