Description

Build and maintain scalable data pipelines using Hadoop ecosystem tools.

This role is on-site.

Responsibilities

• Design, develop, and maintain scalable data pipelines for extracting, transforming, and loading data from various sources.
• Collaborate with cross-functional teams to integrate data from multiple disparate sources, ensuring consistency and accuracy.
• Optimize data processing workflows and storage solutions for performance, scalability, and cost-efficiency.
• Implement data quality checks and validation processes to ensure accuracy and completeness throughout the data lifecycle.
• Monitor data pipeline performance, identifying and resolving issues to maintain system stability.

Required Skills

• 5+ years of experience with Hadoop ecosystem components: HDFS, Map Reduce, Hive, Spark, and Kafka.
• Strong proficiency in UNIX/Linux command-line tools and scripting.
• Experience with debugging and troubleshooting complex data processing issues.
• Familiarity with scheduling tools such as Autosys.
• Understanding of Agile methodology for project execution.
• Bachelor's or Master's degree in Engineering or equivalent.

Preferred Skills

• Knowledge of Banking Operations or financial data systems.
• Experience with data architecture and governance practices.

Key Skills

Education

Any Graduate