Description

Lead technical efforts in data engineering, designing and optimizing ETL/ELT pipelines on Databricks and AWS. Build and manage data lakes using Delta Lake and implement CI/CD workflows with GitLab.

This role is on-site.

Responsibilities

• Design and optimize ETL/ELT data pipelines using Databricks on AWS.
• Build and manage data lakes leveraging Delta Lake for scalable storage.
• Query databases and process data using strong SQL and PySpark capabilities.
• Implement and maintain CI/CD pipelines using GitLab.
• Manage data workflows using Jira and Agile methodologies.

Required Skills

• 8+ years of experience in data engineering.
• Expertise with Databricks architecture, tools, and capabilities.
• Strong command of SQL and proficiency in PySpark.
• Experience with AWS native services (S3, DataSync, DMS, Lambda, SNS, SQS).
• Experience with Delta Lake and Spark.
• Familiarity with CI/CD pipelines using GitLab.
• Knowledge of Jira and Agile methodologies.

Preferred Skills

• Knowledge in Data Warehousing concepts.
• Experience in Retail & Supply Chain domain.

Key Skills

Education

Any Gradute