Description

You will lead the design and implementation of big data solutions using Hadoop ecosystem components and cloud platforms.

Responsibilities

• Build and optimize data pipelines using Spark DataFrames and PySpark (SparkSQL).
• Develop and maintain shell scripts and Python applications for data processing.
• Manage data infrastructure on AWS, Azure, or GCP.
• Support migration projects from legacy Hadoop environments.
• Apply Agile methodologies and automation practices to improve workflow efficiency.

Required Skills

• 5+ years of hands-on experience with Hadoop ecosystem components.
• Strong proficiency in PySpark and SparkSQL.
• Experience with at least one major cloud provider: AWS, Azure, or GCP.
• Proficiency in Python and Shell scripting.
• Good knowledge of SQL.
• Familiarity with workflow engines like Oozie or Autosys.

Preferred Skills

• Experience with Hadoop migration projects.
• Background in Agile development.

Key Skills

Education

Any Gradute