Description

You will build and maintain scalable data pipelines using Python, PySpark, and AWS Glue, while leading technical initiatives and mentoring junior developers.

This role is on-site.

Responsibilities

• Design and optimize distributed data processing systems using Spark and AWS services (S3, EMR, Lambda, Step Functions, Athena).
• Develop and maintain RAG pipelines, embeddings, and basic LLM integrations.
• Implement CI/CD pipelines with Git, ensuring code quality and automated testing for data workflows.
• Lead collaborative assignments, guide team members, and set performance objectives.
• Analyze complex data from multiple sources to solve problems and support business strategy.

Required Skills

• Expert-level Python programming.
• Hands-on experience with PySpark and Spark optimization.
• Proficiency in AWS Glue and AWS data engineering services.
• Strong SQL skills for analytical modeling and reporting.
• Experience with Kafka or Kinesis for streaming analytics.
• Familiarity with CI/CD, Git, and automated testing frameworks.
• Understanding of vectorization, RAG pipelines, and embeddings.
• Bachelor’s degree in Computer Science, IT, or Engineering.

Preferred Skills

• 8+ years of experience in data engineering roles.
• Experience building dashboards and collaborating with BI teams.
• Ability to mentor junior developers and lead technical teams.

Key Skills

Education

Any Graduate