Description

You will design and implement large-scale data processing solutions using the Hadoop ecosystem, owning the full lifecycle from pipeline development to deployment.

This role is on-site.

Responsibilities

• Develop batch and real-time streaming data pipelines using Apache Kafka and Spark.
• Implement processing logic for batch and streaming workloads in Python or Scala.
• Manage data orchestration using Oozie and Airflow schedulers.
• Implement CI/CD workflows for data engineering deployments.
• Maintain data storage and query layers using Hive, Impala, Ozone, and Iceberg.

Required Skills

• 6+ years of experience in the Hadoop Big Data ecosystem.
• Expertise in Apache Spark using Python or Scala.
• Hands-on experience with Apache Kafka.
• Practical experience with GCP cloud environments.
• Proficiency in SQL and Python or Shell scripting.
• Experience with Cloudera ecosystem tools.
• Working knowledge of Hadoop fundamental concepts.
• Experience with Hive, Impala, Ozone, and Iceberg.

Preferred Skills

• Degree in any field of study.

Key Skills

Education

Any graduate