Description

You will design, build, and maintain near-real-time and batch data pipelines for distributed systems.

This role is on-site.

Responsibilities

• Build and optimize data ingestion pipelines from message queues, file shares, REST APIs, and relational databases.
• Develop streaming applications using Spark Structured Streaming and performance-tune Spark, MapReduce, or SQL jobs.
• Implement end-to-end data solutions using Hadoop, NoSQL (HBase, SOLR), and Elastic Stack (Elastic Search, Kibana).
• Manage source code, version control (Git, Bitbucket), and CI/CD processes using Jenkins.
• Resolve complex analytical problems and adapt to new technologies within an Agile SDLC.

Required Skills

• 5+ years of experience with Big Data ecosystems (Hadoop, HDP/Cloudera) and distributed systems.
• Strong SQL proficiency (MySQL, HIVE, Impala, SPARK SQL).
• Hands-on programming in Python, Scala, PHP, or Shell Scripting.
• Experience with data formats (JSON, CSV, XML) and ingestion from various sources.
• Proficiency with Linux operating systems.
• Experience with Kafka, Sqoop, and MapReduce.
• Strong analytical mindset with the ability to work with minimal supervision.

Preferred Skills

• Experience with PHP or Shell Scripting beyond basic usage.
• Deep understanding of various phases of the Software Development Life Cycle.

Any Graduate

Key Skills

Education

Any Gradute