Description

You will build and maintain data pipelines to process large datasets using Spark and Python.

Responsibilities

• Build and manage data pipelines using Spark and Python.
• Own the end-to-end SDLC, including implementation, testing, deployment, and troubleshooting.
• Implement software based on technical specifications and design requirements.
• Collaborate with tech leads and stakeholders on system design.
• Support development teams by debugging and troubleshooting use cases.

Required Skills

• 5+ years of experience in data engineering.
• Proficiency with Apache Spark.
• Strong Python programming skills.
• Advanced SQL knowledge and database experience.
• Experience working with datasets from Amazon S3, Data Lakes, or Snowflake.
• Ability to build and consume APIs.
• Experience with the full software development life cycle.

Preferred Skills

• Bachelor's degree or equivalent graduate qualification.

Key Skills

Education

Any Graduate