Description

You will own the development and maintenance of Big Data processing solutions.

Responsibilities

• Design and create low-level and technical design documents for parallel, batch, and sequence jobs.
• Extract client data from Azure Blob storage, Flat files, MS SQL server, and Oracle legacy systems into the business warehouse.
• Monitor errors and warnings using Azure event manager and MS SQL Server job agents.
• Coordinate knowledge transfer, communicate timelines, and conduct daily review meetings between client and development teams.
• Plan and deploy new Hadoop environments within an AWS environment.

Required Skills

• 5+ years of professional experience in data development.
• Proficiency in Big Data Analytics and Hadoop ecosystems.
• Strong programming skills in Scala and PySpark.
• Experience with Hive and SQL for data manipulation.
• Familiarity with Teradata environments.
• Experience designing and creating User-Defined SQL queries for source system extraction.
• Ability to develop automated load processes using PowerShell scripting.
• Experience with Azure Data Factory (ADF) error handling.

Key Skills

Education

Any Graduate