Description

You will build and maintain large-scale data processing applications using the Spark ecosystem.

Responsibilities

• Develop Spark applications using Scala and Spark SQL.
• Build and manage data processing pipelines within distributed computing frameworks.
• Collaborate with cross-functional teams in an agile environment.
• Manage code and deployment workflows through CI/CD pipelines.

Required Skills

• 6+ years of experience in data engineering.
• Hands-on development with Apache Spark and Scala.
• Proficiency with Spark SQL and the Spark ecosystem.
• Experience with Hadoop and HDFS.
• Working knowledge of distributed computing frameworks for large-scale processing.
• Experience using Git or Bitbucket for version control.
• Experience with build and CI/CD tools including Gradle and Jenkins.
• Proficiency with Jira and Confluence for project tracking.
• Technical experience working within agile methodologies.

Key Skills

Education

Any Graduate