Description

You will build and maintain scalable data processing solutions and ETL workflows using Ab Initio, Python, and PySpark.

This role is remote.

Responsibilities

• Design and maintain ETL workflows using Ab Initio components including GDG, Co>Operating System, and EME.
• Develop scalable data processing solutions using Python and PySpark for batch and real-time requirements.
• Execute ETL activities including data extraction, transformation, and loading.
• Optimize data pipeline performance and troubleshoot production issues.
• Implement data quality checks and validation processes.

Required Skills

• 5+ years of experience in Ab Initio development.
• Strong hands-on experience in Python scripting.
• Solid experience in PySpark and the Spark framework.
• Knowledge of the Hadoop ecosystem (HDFS, Hive).
• Strong SQL knowledge.
• Experience working with Unix/Linux environments.
• Experience with data warehousing concepts.
• Proficiency in ETL and data integration best practices.

Preferred Skills

• 10+ years of total working experience in data engineering.
• Graduate degree in a technical field.

Key Skills

Education

Any Gradute