Description

You will lead the design, development, and optimization of large-scale distributed data pipelines using PySpark. This role is on-site.

Responsibilities

• Lead the design and development of large-scale distributed data pipelines using PySpark.
• Provide technical leadership across data engineering teams, reviewing code and guiding best practices.
• Build high-performance ETL/ELT solutions optimized for large-volume processing.
• Optimize PySpark jobs, improving performance, tuning clusters, and ensuring pipeline reliability.
• Collaborate with business, analytics, platform, and architecture teams to define data solutions.

Required Skills

• 10+ years of experience in software or data engineering.
• Deep expertise in PySpark and distributed computing.
• Strong experience in data transformation and Spark optimization techniques.
• Proficiency in ETL design and scalable ETL patterns.
• Strong understanding of Data Modeling and Processing Frameworks.
• Experience with cloud-based big-data ecosystems (AWS, Azure, or GCP).
• Proven ability to lead development teams and deliver enterprise-grade data solutions.
• Excellent communication and analytical problem-solving skills.

Preferred Skills

• Experience leading technical initiatives in a collaborative cross-functional environment.

Key Skills

Education

Any Graduate