Description

You will lead the delivery of major data engineering initiatives.

This role is remote.

Responsibilities

• Design, develop, and optimize scalable data pipelines using Python and PySpark for batch and streaming workloads.
• Build, schedule, and monitor complex workflows using Airflow.
• Architect and implement CI/CD pipelines for data engineering projects using GitHub and Docker.
• Apply test-driven development practices and automate unit and integration tests for data pipelines.
• Mentor junior engineers on coding, testing, and deployment best practices.

Required Skills

• Minimum 7+ years overall IT experience.
• Minimum 5+ years building production-grade data pipelines with Python and PySpark.
• Proven experience designing and managing Airflow DAGs in enterprise environments.
• Ability to build and maintain CI/CD pipelines for data engineering workflows.
• Experience with containerization (Docker) and GCP for data engineering workloads.
• Fluency in writing object-oriented Python code and managing dependencies.
• Proficiency with Git for source code management and collaboration.
• Solid understanding of SQL for data ingestion and analysis.
• Experience with waterfall, iterative, and agile methodologies.

Key Skills

Education

Any Graduate