Description

Build and maintain scalable data architectures and processing pipelines within an on-premises environment.

Responsibilities

• Implement scalable data architectures using Hadoop, Impala, Hive, Spark, and NiFi.
• Develop performant Spark jobs using Java.
• Build data pipelines and compute tiers operating on Hadoop, Impala, and Spark.
• Review code and mentor junior engineers to improve the quality of Hadoop and Spark batch jobs.
• Evaluate new technologies to improve system performance, scalability, and reliability.

Required Skills

• 6+ years of experience in data engineering or big data development.
• Strong proficiency with Hadoop and Spark.
• Experience with Impala and Hive.
• Hands-on experience with NiFi.
• Proficiency in Java for developing Spark jobs.
• Experience building and managing on-premises data architectures.
• Background in designing data pipelines and compute tiers.
• Ability to conduct critical code reviews for batch processing workflows.
• Degree in any graduate field.

Key Skills

Education

Any graduate