Description

You will design and implement end-to-end distributed data processing pipelines and large-scale software solutions.

Responsibilities

• Design and implement distributed data processing pipelines using Spark, Hive, and Python within the Hadoop ecosystem.
• Build utilities, user-defined functions, and frameworks to optimize data flow patterns.
• Publish RESTful APIs using OpenAPI specifications to enable real-time data consumption.
• Collaborate with architecture and engineering leads to define and adhere to engineering best practices.

Required Skills

• 6+ years of experience in large-scale software development.
• 3+ years of experience working with Big Data technologies.
• Strong programming proficiency in Java, Scala, Python, and SQL.
• Hands-on experience with Spark, MapReduce, and Hive.
• Experience developing and publishing RESTful APIs.
• Proficiency in Shell scripting.
• Experience with NoSQL technologies such as HBase, DynamoDB, or Cassandra.

Preferred Skills

• Familiarity with distributed stream processing frameworks like Apache Spark, Flink, or Kafka.
• Experience with Business Rule management systems like Drools.
• Bachelor's degree in Computer Science or a related discipline.

Key Skills

Education

Bachelor's degree in Computer Science