Description

You will build and maintain large-scale data pipelines using Hadoop, Spark, and Kafka. This role is on-site.

Responsibilities

• Develop and optimize data processing logic in Python and SQL.
• Implement structured streaming solutions and manage data storage in Hive and HBase.
• Handle data ingestion and transformation for JSON, CSV, and XML formats.
• Manage deployments and version control using Git and Jenkins in an Agile environment.

Required Skills

• 6+ years of experience with Hadoop and Big Data ecosystems.
• Strong proficiency in Python and SQL (mandatory).
• Hands-on experience with Apache Spark, specifically Structured Streaming.
• Experience with Apache Kafka, Hive, and HBase.
• Proficiency with data formats including JSON, CSV, and XML.
• Working knowledge of Linux, Git, and Jenkins.
• Experience working in Agile methodologies.

Preferred Skills

• Graduate degree in any field.

Key Skills

Education

Any Gradute