Description

You will build and maintain a reporting platform to extract Capital Markets Surveillance reports from an enterprise data lake.

Responsibilities

• Develop and optimize data extraction processes from enterprise data lakes.
• Build and maintain reporting platforms for Capital Markets Surveillance.
• Implement data processing logic using Spark and PySpark.
• Manage data workflows within Big Data environments.

Required Skills

• 8+ years of experience in data engineering or related roles.
• Proficiency with PySpark and SparkSQL.
• Experience with Spark and Scala.
• Hands-on work with HDFS.
• Strong T-SQL skills.
• Experience with Python, Batch Script, and Shell Script.
• Knowledge of Git, SVN, and Liquibase.
• Experience working in Agile and Scrum environments using Jira.

Preferred Skills

• Experience with SSIS and PowerShell scripting.

Key Skills

Education

Any Gradute