Description

You will build and maintain data pipelines for big data processing.

This role is remote.

Responsibilities

• Build and manage ETL processes using Informatica.
• Develop and automate workflows using Airflow.
• Maintain data lakes utilizing the Iceberg format.
• Ensure data quality across all processing stages.
• Manage data storage and compute using AWS services.

Required Skills

• 5+ years of experience in data engineering.
• Proficiency in Python and PySpark.
• Strong SQL skills for data manipulation.
• Hands-on experience with AWS services (S3, Glue, Redshift, Lambda, EMR).
• Experience with Informatica for ETL.
• Knowledge of Postgres and Airflow.
• Ability to write Bash/Shell scripts.
• Experience working with Iceberg data lake formats.

Key Skills

Education

Any Graduate