Description

Design, build, and lead scalable data engineering solutions, real-time streaming pipelines, and distributed processing frameworks.

This role is on-site.

Responsibilities

• Build and optimize data pipelines using Python, PySpark, AWS Lambda, AWS Glue, S3, Kinesis, Apache Flink, and Confluent Kafka.
• Architect large-scale cloud-based data platforms ensuring high performance, reliability, and cost optimization.
• Tune complex SQL queries, data models, and integration processes to support analytics and reporting.
• Establish engineering best practices, conduct code reviews, and mentor team members.
• Implement monitoring, automation, governance, and security controls for production pipelines.

Required Skills

• 10+ years of experience in Data Engineering, Big Data, and cloud-based data platform development.
• Strong hands-on expertise with Python and PySpark.
• Deep experience with AWS services: Lambda, Glue, S3, and Kinesis.
• Proficiency in Apache Flink and Confluent Kafka for real-time streaming.
• Advanced SQL skills for complex queries and data modeling.
• Bachelor’s degree in Computer Science, IT, Data Engineering, Software Engineering, or a related field.
• Proven track record designing scalable architectures and distributed systems.

Preferred Skills

• Experience with data governance and security controls in cloud environments.
• Background in mentoring and providing technical leadership.

Key Skills

Education

Any Graduate