Description

You will lead the end-to-end design and implementation of ETL workflows for structured and semi-structured data from SFTP, DB2, Oracle, and APIs.

Responsibilities

• Architect and maintain data ingestion and transformation pipelines using IBM DataStage (CP4D) and AWS Glue/Lambda functions.
• Optimize data load performance and manage large data volumes using partitioning, incremental loads, and error handling.
• Ensure data lineage, metadata management, and data quality aligning with enterprise data governance standards.
• Support CI/CD integration, parameterization, and version control of ETL code via Git and DevOps pipelines.
• Lead and mentor ETL developers; establish best practices and conduct code reviews.

Required Skills

• 8+ years of enterprise data integration experience, with 3+ years as a technical lead.
• Strong expertise in IBM DataStage, including deployment on Cloud Pak for Data (CP4D).
• Solid experience with AWS Glue (PySpark) and AWS Lambda for serverless data transformation.
• Proven knowledge of Snowflake data warehousing, including streams, tasks, and performance tuning.
• Experience with complex ETL orchestration involving parallel processing and dynamic parameterization.
• Strong SQL and Python skills.
• Understanding of data lakehouse architectures, S3, and ingestion patterns.
• Familiarity with data governance and lineage tools (Collibra, BigID, Manta).
• Knowledge of CI/CD, Git, and agile/scrum delivery.

Key Skills

Education

Any Gradute