Description

You will build and maintain data pipelines to process, clean, and transform large-scale raw datasets.

Responsibilities

• Clean and transform vast amounts of raw data from multiple systems using Spark.
• Develop production-embedded data pipelines and handle ad hoc data requests.
• Optimize Spark cluster utilization to ensure efficient data processing.
• Deliver ready-to-use data sets for feature developers and business analysts.

Required Skills

• 5+ years of experience in software development.
• Proficiency in Java and the Spark ecosystem.
• Strong knowledge of Big Data technologies and distributed programming concepts.
• Ability to write complex SQL queries and design SQL data structures.
• Solid understanding of algorithms and data structures.
• Proven problem-solving skills and ability to learn new technologies.
• Effective communication skills for collaborating with technical and business teams.
• Bachelor's degree in any field.

Preferred Skills

• Experience with Python.
• Knowledge of DevOps practices.

Key Skills

Education

Any Graduate