Description

Design and optimize production-grade ETL pipelines using PySpark and Databricks to integrate data from diverse sources.

This role is on-site.

Responsibilities

• Build and maintain scalable ETL processes to extract, transform, and load data from multiple sources.
• Develop PySpark-based data transformations and manage Databricks workflows for efficient processing.
• Ensure data integrity through rigorous testing, validation, and monitoring of ETL outputs.
• Optimize existing ETL workflows for improved performance, efficiency, and reliability.
• Prepare technical documentation and support production-ready data integration solutions.

Required Skills

• 7+ years of data engineering or ETL experience.
• Strong proficiency in Python and PySpark for building scalable data transformations.
• Hands-on experience with Databricks for production ETL and transformation workflows.
• Solid understanding of data engineering fundamentals, including SQL-based query optimization and performance tuning.
• Experience with Informatica and Informatica MDM.

Preferred Skills

• Azure and cloud exposure.
• Teradata and relational database development experience.
• Amazon Connect (AWS Contact Center) exposure.

Key Skills

Education

Any Graduate