Description

You will develop and deliver code within a Hadoop Distributed File System environment to meet specific quality and timing standards.

Responsibilities

• Translate complex functional and technical requirements into detailed designs.
• Ingest complex datasets into Hadoop using Spark, Hive, and Sqoop techniques.
• Transform data using Spark with Scala and manage Hive objects.
• Perform impact assessments on large data stores to protect existing pipelines and uncover insights.
• Execute unit and system testing to ensure code quality before handing over to operational teams.

Required Skills

• 5+ years of experience in Hadoop development and implementation.
• Proficiency with Scala and Spark for data transformation.
• Hands-on experience with Hive, Sqoop, and Hadoop ecosystems.
• Working knowledge of Unix, WinSCP, and Putty.
• Experience with IntelliJ IDEA and Autosys job scheduling.
• Proficiency using GitHub and CI/CD pipelines such as TeamCity or Jenkins.
• Ability to maintain security and data privacy protocols.
• Experience in designing and supporting Hadoop-based environments.

Preferred Skills

• Knowledge of Teradata.

Key Skills

Education

Any Graduate