Description

You will design, build, and support Hadoop-based environments, focusing on data ingestion and transformation. You own the end-to-end development lifecycle from requirement analysis to production deployment, ensuring data integrity and performance.

Responsibilities

• Interact with stakeholders to translate functional and technical requirements into detailed system designs.
• Perform impact assessments on large data stores to prevent pipeline disruptions and uncover insights.
• Ingest complex datasets into Hadoop using Spark, Hive, and Sqoop techniques.
• Transform data using Spark with Scala and manage Hive objects.
• Execute unit and system testing to ensure code quality and oversee handover to operational teams.

Required Skills

• 5+ years of experience in Hadoop Distributed File System (HDFS) development and MapReduce.
• Strong proficiency in Scala for Spark-based data transformation.
• Hands-on experience with Hive for data warehousing and object management.
• Working knowledge of Unix/Linux environments, including WinSCP and Putty.
• Experience with IntelliJ IDEA, Autosys job scheduling, and version control via GitHub.
• Familiarity with CI/CD pipelines (TeamCity or Jenkins) for productionizing code.
• Ability to create scalable web services for data tracking and high-speed querying.

Preferred Skills

• Knowledge of Teradata.

Key Skills

Education

Any Graduate