Description

Design and optimize scalable data pipelines using Apache Airflow and Python on Kubernetes or OpenShift.

This role is hybrid.

Responsibilities

• Build and maintain DAGs in Apache Airflow, focusing on scheduling and performance tuning.
• Develop complex data models and macros using Core technologies.
• Write and optimize Python code for data engineering tasks and automation.
• Manage distributed workloads and ensure platform stability in production environments.
• Execute complex SQL transformations and analytics queries.

Required Skills

• Expert-level Apache Airflow (DAG design, scheduling, performance tuning).
• Strong proficiency in Python for data engineering and automation.
• Deep understanding of Kubernetes and/or OpenShift in production.
• Extensive experience with distributed workload management and performance optimization.
• Strong SQL skills for complex transformations and analytics.
• Familiarity with containerized deployments, CI/CD pipelines, and Git-based workflows.
• 5+ years of relevant experience.
• Any Graduate degree.

Preferred Skills

• Experience supporting financial services or accounting platforms.
• Exposure to enterprise system migrations (e.g., legacy platform to modern data stack).

Key Skills

Education

Any Graduate