Description

You will develop and manage data workflows and sanitization processes within a Hadoop ecosystem.

Responsibilities

• Build workflow configurations and execute data sanitization validations for multiple lines of business.
• Perform file extraction and data access processes, including row and column level file-splits.
• Monitor Hadoop clusters and provide regular progress updates to the leadership team.
• Translate business requirements into technical data workflows.

Required Skills

• 6+ years of IT experience.
• Hands-on experience with Hadoop architecture, ecosystem, Hive, and HDFS (AVRO, Parquet, QRC formats).
• Working knowledge of MapReduce and Spark concepts.
• Strong UNIX programming and scripting skills.
• Experience with ETL processes using tools like DataStage, Informatica, or Teradata+Unix.
• Exposure to scheduling and automation using AutoSys.
• Proficiency with data connection methods via JDBC and ODBC.
• Database management experience with RDBMS such as DB2, SQL, Oracle, Netezza, Sybase, Siebel, Teradata, or MySQL.
• Any Graduate degree.

Preferred Skills

• Knowledge of Kafka.
• Experience in banking environments.
• Data discovery knowledge regarding confidential data.

Key Skills

Education

Any Graduate