Description

Lead data engineering efforts by designing big data solutions and mentoring the team.

Responsibilities

• Lead the design and implementation of big data solutions using PySpark.
• Build and implement scalable data pipelines.
• Collaborate with cross-functional teams to develop data-driven solutions based on business needs.
• Implement data validation and quality checks to ensure accuracy and integrity.
• Mentor and coach team members on PySpark and big data best practices.

Required Skills

• 8+ years of experience in PySpark and data engineering development.
• Proven experience in a PySpark Lead or similar leadership role.
• Expertise in PySpark and Spark.
• Experience with Hadoop and Hive.
• Ability to design scalable data architectures.
• Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.

Key Skills

Education

Bachelor’s or Master’s degree