Description

You will lead the design and development of scalable data processing systems and end-to-end technical solutions.

Responsibilities

• Design and develop scalable data processing systems using Python and PySpark.
• Gather requirements and translate complex business needs into technical architectures.
• Manage data projects from initial conception through to production deployment.
• Mentor team members to enhance their technical capabilities and engineering standards.
• Oversee the delivery of high-quality results within established project timelines.

Required Skills

• 5+ years of experience in data engineering.
• Expertise in Python and PySpark, including Spark architecture and performance tuning.
• Proficiency in SQL with experience in writing and optimizing complex queries.
• Hands-on experience with AWS services including Glue, Lambda, API Gateway, and S3.
• Experience building and managing CI/CD pipelines.
• Strong analytical and problem-solving capabilities for technical troubleshooting.
• Effective communication skills for conveying technical concepts to stakeholders.
• Any Graduate degree.

Preferred Skills

• Experience with AWS CloudFormation.
• Working knowledge of Unix environments.

Key Skills

Education

Any Graduate