Description

You will design, develop, and maintain scalable data pipelines using streaming technologies and automation tools. You will identify system enhancements, analyze complex business issues, and implement solutions for data systems. Your work involves conducting testing, debugging, and writing code for design specifications while adhering to compliance and data governance standards.

Responsibilities

• Design and maintain robust data pipelines using Spark, Hadoop, and Kafka.
• Develop and optimize data processing scripts using Python and PySpark.
• Analyze system issues and implement solutions for complex business problems.
• Conduct testing, debugging, and code reviews for data applications.
• Ensure data governance, standardization, and compliance in data strategies.

Required Skills

• 5+ years of experience in programming and debugging business applications.
• Strong proficiency in Spark, Hadoop, Hive, and Kafka.
• Expertise in Python, PySpark, and SQL.
• Experience with Oracle, MongoDB, and PostgreSQL databases.
• Working knowledge of data governance and metadata management frameworks.
• Ability to translate regulatory requirements into actionable data strategies.

Preferred Skills

• Experience with Impala, Scala, or Cloudera.
• Proficiency in shell scripting for data processing automation.

Key Skills

Education

Any Graduate