Description

You will build and optimize scalable data pipelines for financial and accounting platforms, managing complex orchestration, modeling, and containerized deployments.

This role is hybrid.

Responsibilities

• Design, develop, and maintain complex Apache Airflow DAGs for batch and event-driven pipelines, optimizing scheduler and worker configurations for high concurrency.
• Lead dbt Core implementation, including project structure, environments, CI/CD integration, and the creation of staging, intermediate, and mart models.
• Deploy and manage data workloads on Kubernetes and OpenShift, handling resource optimization, autoscaling, and pod scheduling.
• Troubleshoot container-level performance issues and tune end-to-end pipeline performance across Airflow, dbt, and the underlying data platforms.
• Implement observability solutions and enforce data engineering standards, security best practices, and governance policies.

Required Skills

• 10+ years of professional experience in data engineering, analytics engineering, or platform engineering.
• Expert-level proficiency in Apache Airflow, including DAG design, scheduling, performance tuning, and dependency management.
• Expert-level dbt Core experience, covering data modeling, testing, macros, and incremental models.
• Strong proficiency in Python for data engineering and automation tasks.
• Deep understanding of Kubernetes and/or OpenShift in production environments.
• Extensive experience with distributed workload management, performance optimization, and SQL for complex transformations.
• Familiarity with containerized deployments, CI/CD pipelines, and Git-based workflows.

Preferred Skills

• Experience supporting financial services or accounting platforms.
• Exposure to enterprise system migrations from legacy platforms to modern data stacks.

Key Skills

Education

Any Graduate