Description

You will own the development and maintenance of Big Data architectures on the Hadoop stack, focusing on data ingestion, transformation, and performance optimization.

This role is on-site.

Responsibilities

• Write Spark and Scala code to ingest data from SFTP, mainframe, RDBMS, and Kafka sources.
• Develop data ingestion processes using Ni-Fi and Spark to transform data according to business rules.
• Load results into Hive, HBase, and RDBMS tables while executing migrations from relational databases to HDFS clusters.
• Perform application performance tuning, troubleshooting, and data store analysis.
• Translate functional and technical requirements into detailed designs ensuring security and data entitlement compliance.

Required Skills

• 5+ years of experience in Big Data development.
• Proficiency with the Hadoop stack, including HDFS clusters and Hive.
• Strong experience with Spark and Scala.
• Hands-on experience with Ni-Fi for data ingestion.
• Experience working with Kafka, Mainframe, and RDBMS sources.
• Ability to work with JSON and disparate datasets in multiple formats.
• Bachelor’s degree in Computer Science, Information Technology, or a closely related field.

Key Skills

Education

Any Graduate