Description

Key Responsibilities

Design, develop, maintain and support scalable data pipelines using Big Data and AWS technologies

Lead and support data platform migration initiatives (On-Prem to AWS Cloud), ideally with Netezza background.

Develop and manage ETL/ELT processes using tools like SSIS, Pentaho, or similar

Implement and manage AWS services such as EMR, S3, EC2, Redshift, Glue, and Airflow

Build and optimize data workflows and orchestration pipelines using Airflow

Work with real-time streaming technologies such as Kafka and Spark Streaming

Perform data ingestion, transformation, and validation from multiple data sources

Optimize SQL queries for performance and scalability

Monitor system performance, troubleshoot issues, and ensure system reliability

Collaborate with cross-functional teams including developers, architects, and infrastructure teams

Required Technical Skills

Cloud Technologies

Strong experience in AWS Cloud services:

EMR, EC2, S3, VPC, RDS, Redshift

AWS Glue, IAM, CloudWatch, CloudFormation

Airflow (or AWS Managed Workflows)

Databases

Experience working with:

Netezza is mandatory

SQL-based systems: SQL Server and PostgreSQL

Data warehouses: Teradata, Redshift, Netezza

ETL Tools

Hands-on experience with:

SSIS

Programming & Scripting

Strong proficiency in:

SQL

Shell scripting

Good to have:

Python

Operating Systems

Strong experience in Unix/Linux environments

Key Skills

Education

Bachelor's degree