Description

You will lead the design and development of large-scale distributed data pipelines using PySpark.

Responsibilities

• Lead the design and development of large-scale distributed data pipelines using PySpark.
• Provide technical leadership across data engineering teams, reviewing code and guiding best practices.
• Build high-performance ETL/ELT solutions optimized for large-volume processing.
• Collaborate with business, analytics, platform, and architecture teams to define data solutions.
• Optimize PySpark jobs, improving performance, tuning clusters, and ensuring pipeline reliability.

Required Skills

• 10+ years of experience in software/data engineering with deep expertise in PySpark.
• Strong experience in distributed computing, data transformation, and Spark optimization techniques.
• Familiarity with cloud-based big-data ecosystems (AWS, Azure, or GCP).
• Strong understanding of Data Modeling and Processing Frameworks.
• Experience in Etl Design and scalable ETL design.
• Proven ability to lead development teams and deliver enterprise-grade data solutions.
• Excellent communication and analytical problem-solving skills.

Key Skills

Education

Any Graduate