Description

Job Overview
We are seeking an experienced Data Engineer to design, build, and optimize scalable data platforms and pipelines. The ideal candidate will bring strong expertise in Python, SQL, PySpark, and AWS data services, with hands-on experience handling large-scale distributed systems and performance tuning.

Key Responsibilities:
Design, develop, and maintain scalable ETL/ELT pipelines
Perform large-scale data processing using PySpark and Spark SQL
Build and manage data solutions on AWS (EMR, Glue, Lambda, S3)
Develop and optimize data warehouses using Redshift, RDS, and MySQL
Implement robust data modeling, schema design, and validation
Monitor, log, and troubleshoot pipelines using AWS CloudWatch
Secure credentials and secrets using AWS Secrets Manager
Optimize performance and ensure high availability of data systems
Collaborate with analytics, data science, and business teams
Debug complex production issues and drive long-term data reliability

Required Skills & Qualifications:
10–12 years of experience as a Data Engineer or similar role
Strong proficiency in Python and SQL
Extensive hands-on experience with PySpark and Spark SQL
Deep experience with AWS Data Services, including:
EMR/ Glue/ Lambda
S3/ RDS/ MySQL
Redshift
CloudWatch, Secrets Manager
Strong understanding of distributed computing architectures
Proven expertise in data modeling, schema handling, and performance tuning
Excellent debugging and problem-solving skills

Nice to Have
Experience with streaming technologies (Kafka, Kinesis, etc.)
Exposure to CI/CD pipelines for data platforms
Knowledge of data governance, security, and compliance best practices

Key Skills

Education

Any Graduate