Description

You will develop and maintain big data pipelines using Hadoop and Spark.

Responsibilities

• Design and build end-to-end Near-Real Time and Batch Data Pipelines.
• Ingest data from message queues, file shares, REST APIs, and relational databases.
• Tune performance for Spark, MapReduce, and SQL jobs.
• Develop solutions to comprehend customer requests and resolve technical issues.

Required Skills

• 7+ years of professional experience in data engineering.
• Expertise in Hadoop and distributed systems.
• Proficiency in Spark, with experience in Spark Structured Streaming.
• Strong SQL skills using Hive, Impala, or MySQL.
• Hands-on programming experience in Scala or Python.
• Experience with data ingestion and formats (JSON, CSV, XML).
• Working knowledge of Linux operating system.
• Experience with version control systems like Git or SVN.
• Familiarity with Agile development and SDLC phases.

Key Skills

Education

Any Graduate