Description

You will design, develop, and maintain scalable data pipelines and integration solutions.

Responsibilities

• Design and implement ETL/ELT pipelines using IICS, Python, and PySpark to process large datasets.
• Integrate data from multiple systems while ensuring quality, consistency, and reliability.
• Develop shell scripts to automate routine data processing tasks and ETL workflows.
• Build and manage cloud-based data solutions using AWS services including S3, Glue, Lambda, Redshift, and EMR.
• Manage CI/CD pipelines using Jenkins or GitHub Actions to automate testing, deployment, and monitoring.
• Monitor and optimize the performance of data pipelines for efficiency and scalability.

Required Skills

• 5+ years of experience in designing and developing ETL/ELT pipelines.
• Proficiency with Informatica Intelligent Cloud Services (IICS).
• Strong experience with Python and PySpark.
• Hands-on experience with AWS services: S3, Glue, Lambda, Redshift, and EMR.
• Proficiency in shell scripting for task automation.
• Experience with CI/CD tools such as Jenkins or GitHub Actions.
• Bachelor's or Master's degree in Computer Science, Information Technology, or a related field.

Preferred Skills

• Experience with AWS Athena, Step Functions, and CloudWatch.
• Knowledge of containerization and orchestration using Docker and Kubernetes.
• Familiarity with data security and privacy best practices.

Key Skills

Education

Any Graduate