Description

You will design, develop, and optimize scalable data pipelines using Python, SQL, PySpark, and AWS services. You will manage distributed computing systems, handle large datasets, and ensure high performance and reliability of data platforms.

This role is hybrid.

Responsibilities

• Design and build scalable data pipelines and distributed systems.
• Perform data modeling, schema design, and management.
• Conduct performance tuning and cost optimization for AWS resources.
• Troubleshoot complex data and system issues in production.
• Collaborate with cross-functional teams to support analytics needs.

Required Skills

• 10+ years of experience in data engineering.
• Strong proficiency in Python and SQL.
• Expertise in PySpark and Spark SQL.
• Hands-on experience with AWS data services: EMR, Glue, Lambda, S3, RDS/MySQL, Redshift.
• Experience with CloudWatch and Secrets Manager.
• Proven track record in distributed data processing and data modeling.
• Strong debugging and problem-solving skills.

Preferred Skills

• Experience in schema evolution and management.
• Familiarity with production-grade, fast-paced environments.

Key Skills

Education

Any Graduate