Description

You will build and manage data processing pipelines using PySpark and Python.

Responsibilities

• Develop APIs and microservices using Python and Object Oriented Programming principles.
• Extract and process data from HDFS using Spark.
• Analyze migration plans for Cloudera distributions of Apache Hadoop to perform impact analysis and propose mitigation strategies.
• Manage end-to-end project lifecycles, including technical issue resolution and team support.
• Participate in SIT, UAT, and business calls to finalize logic for new rules.

Required Skills

• 5+ years of professional experience in data engineering roles.
• Expertise in Python and PySpark.
• Strong proficiency in Spark, Scala, and Hadoop.
• Experience with Hive and Impala.
• Advanced SQL skills.
• Solid understanding of Object Oriented Programming.
• Experience working with HDFS.
• Any Graduate degree.

Key Skills

Education

Any Graduate