Description

You will design and maintain scalable data pipelines using Databricks and AWS.

This role is on-site.

Responsibilities

• Design and develop scalable ETL/ELT pipelines using Databricks and Apache Spark (PySpark).
• Lead migration of data pipelines and applications from on-premises environments to AWS.
• Build data ingestion frameworks for structured, semi-structured, and unstructured sources.
• Review existing data solutions and recommend architectural improvements for performance.
• Implement CI/CD pipelines and enforce data quality and governance standards.

Required Skills

• 12+ years of overall IT experience with a focus on data engineering.
• Hands-on expertise with Databricks SaaS, Python, and PySpark.
• Expert-level SQL skills, including writing and optimizing complex queries.
• Strong experience with AWS-based ETL services (AWS Glue, EC2, EMR).
• Solid understanding of Data Lake and Lakehouse architectures.
• Experience working with RDBMS and large-scale analytical datasets.
• Proven experience deploying production code using Git and CI/CD pipelines.

Key Skills

Education

Bachelor's degree