Description

Build and maintain scalable data pipelines using Python, PySpark, and AWS services.

This role is remote.

Responsibilities

• Develop and optimize data processing workflows using Python and PySpark.
• Manage data lake architectures, specifically in Iceberg format.
• Orchestrate ETL jobs using AWS Glue, Airflow, and Lambda.
• Write and maintain Bash/Shell scripts for automation and infrastructure tasks.

Required Skills

• 8+ years of overall IT experience.
• 5+ years of hands-on experience with Python and PySpark.
• Proficiency in SQL and big data processing.
• Strong knowledge of AWS data services (S3, Redshift, Glue, Lambda, EMR).
• Experience with Airflow, Postgres, SNS, and Event Bridge.
• Expertise in Bash/Shell scripting.
• Understanding of healthcare or finance systems.

Key Skills

Education

Any Graduate