Description

You will design and build data pipelines using Hadoop ecosystem tools.

Responsibilities

• Develop and maintain data processing jobs using PySpark.
• Manage and optimize data storage and retrieval across HDFS.
• Implement orchestration workflows using tools like Oozie.
• Write complex SQL queries for data extraction and transformation.

Required Skills

• 7+ years of experience in data engineering.
• Expertise with Hadoop Platform utilities: Hive, YARN, Impala, HDFS, Oozie.
• Proficiency in Python (2.7/3.0) and Shell scripting.
• Strong command of SQL Queries.
• Experience with data streaming platforms like Kafka.
• Familiarity with Core Java and Spark.
• Proficiency with version control using Git.
• Experience with CI/CD tools such as Jenkins.

Preferred Skills

• Bachelor's degree in Computer Science or related field.

Key Skills

Education

Bachelor's degree