Description

You will build and maintain large-scale data pipelines using Spark and the Hadoop ecosystem.

Responsibilities

• Create, deploy, and maintain data pipelines for big data processing.
• Debug and troubleshoot data workflows within the Hadoop ecosystem.
• Schedule and monitor data jobs to ensure pipeline stability.
• Develop automation tools for data processing and job management.
• Collaborate with internal and external teams to meet technical deadlines.

Required Skills

• 5+ years of experience with PySpark and Spark.
• 4+ years of professional experience using Scala and Python.
• Extensive experience with Hadoop distributed frameworks.
• Proficient knowledge of SQL and any RDBMS.
• Bachelor's degree or equivalent experience (3 years of progressive experience in lieu of each year of education).
• Proven ability to prioritize tasks and meet technical deadlines.

Preferred Skills

• 1+ years of AWS development experience.
• Knowledge of DevOps and drive automation.

Key Skills

Education

Bachelor's Degree