Description

You will build and maintain data pipelines to process large datasets using Spark and Python.

Responsibilities

• Build and manage data pipelines using Spark and Python technologies.
• Own the end-to-end SDLC, including implementation, JUnit testing, functional testing, deployment, and troubleshooting.
• Implement software based on technical specifications and design requirements.
• Collaborate with tech leads and stakeholders on system design.
• Support development teams by debugging, testing, and troubleshooting use cases.

Required Skills

• 5+ years of experience in data engineering.
• Proficiency with Spark and Python.
• Strong SQL knowledge and database experience.
• Experience working with datasets from Amazon S3, Data Lakes, or Snowflake.
• Ability to build and consume APIs.
• Experience with the full software development life cycle.
• Bachelor's degree or equivalent graduate qualification.

Key Skills

Education

Any Graduate