Description

You will design and develop big data solutions using Spark and Python.

This role is on-site.

Responsibilities

• Develop and optimize Spark and Hive queries to process large datasets.
• Migrate existing Hive queries to Impala.
• Build batch analysis job prototypes using Hadoop, Pig, Oozie, Hue, and Hive.
• Collaborate with application teams to manage Hadoop updates, patches, and version upgrades.
• Perform data analysis to determine attribute relevance and variable attribution.

Required Skills

• 6+ years of experience in Big Data design and coding with Hadoop and Spark/Scala.
• 5+ years of recent experience in Python and Spark code development.
• Expertise in Python, including data science packages like Pandas and NumPy.
• Proficiency in Java or other object-oriented programming languages.
• Strong knowledge of Oracle SQL.
• Solid understanding of Linux environments.
• Deep knowledge of Computer Science fundamentals, including data structures, algorithms, and object-oriented design.
• Experience with Spark and Hive query optimization.

Preferred Skills

• Experience with R.

Key Skills

Education

Any Graduate