Description

You build, deploy, and maintain scalable data pipelines using Python, PySpark, and Airflow.

This role is on-site.

Responsibilities

• Design and optimize batch and streaming data pipelines using Python and PySpark.
• Build, schedule, and monitor complex workflows using Apache Airflow.
• Implement CI/CD pipelines for data engineering projects using GitHub and Docker.
• Apply test-driven development (TDD) and automate unit/integration tests.
• Create technical documentation, including process and data flow diagrams.

Required Skills

• 5+ years of experience building production-grade data pipelines with Python and PySpark.
• Proven experience designing and deploying Airflow DAGs in enterprise environments.
• Ability to build and maintain CI/CD pipelines for data engineering workflows.
• Experience with Docker containerization and cloud platforms (GCP).
• Proficiency in Python, including object-oriented programming and dependency management.
• Strong command-line skills in Unix/Linux environments.
• Solid understanding of SQL for data ingestion and analysis.
• Proficiency with Git for source code management.

Preferred Skills

• Bachelor’s or graduate degree in Computer Science, Data Analytics, or related field.

Key Skills

Education

Any Graduate