Description

Design and maintain scalable ETL/ELT pipelines on Databricks using Delta Lake, Auto Loader, and Databricks SQL.

This role is hybrid.

Responsibilities

• Build batch, streaming, and CDC ingestion pipelines for near real-time data processing.
• Migrate data from Redshift, DB2, Oracle, MySQL, and MSSQL into Databricks.
• Implement Medallion architecture (Bronze, Silver, Gold layers) and ensure data quality via reusable DBX functions.
• Manage secure data governance and sharing using Unity Catalog and Delta Sharing.
• Deploy pipelines using Databricks Asset Bundles (DAB) with automated CI/CD via GitHub or GitLab.

Required Skills

• 13+ years of experience in data engineering and distributed systems.
• Expert-level SQL and data modeling skills.
• Proficiency in Databricks, Delta Lake, Auto Loader, and Databricks SQL.
• Experience with Kafka and Structured Streaming for real-time data ingestion.
• Hands-on experience with CI/CD tools and Databricks Asset Bundles (DAB).
• Familiarity with data source systems including Redshift, DB2, Oracle, MySQL, and PL/SQL.
• Experience integrating BI tools like Power BI, Tableau, or Looker with Databricks via ODBC/JDBC.

Preferred Skills

• Bachelor's degree in Computer Science or related field.
• Experience with cross-functional collaboration for secure data sharing across domains.

Referral Bonus: ₹50,000 INR

Key Skills

Education

Bachelor's degree