Description

You will design, build, and maintain security-enabled Big Data workflows and pipelines to process billions of records within a Hadoop Distributed File System (HDFS).

Responsibilities

• Design and implement data ingestion and consumption pipelines for enterprise-scale datasets.
• Build and maintain workflows to move data into and out of HDFS.
• Participate in sprint requirements and design discussions.
• Troubleshoot and resolve data processing issues proactively.
• Work autonomously to deliver results with limited direction.

Required Skills

• 7+ years of experience in Big Data development.
• Extensive experience with Spark and Kafka.
• Proficiency in Scala, Python, and Java.
• Hands-on experience with Hive, MapReduce, and Pig.
• Experience using Oozie and Flume for pipeline management.
• Bachelor’s Degree in a related field or equivalent experience.

Key Skills

Education

Bachelor’s Degree