Description

You will own the design, development, and reliability of enterprise-scale data platforms and pipelines.

Responsibilities

• Design and maintain complex Airflow DAGs for batch and event-driven pipelines, optimizing scheduler, executor, and worker configurations for high-concurrency workloads.
• Implement dbt Core projects, including project structure, CI/CD integration, and robust data modeling (staging, intermediate, marts) with comprehensive testing and documentation.
• Deploy and manage data workloads on Kubernetes and OpenShift, configuring resource limits, autoscaling, and pod scheduling to ensure horizontal scaling and fault tolerance.
• Monitor and tune end-to-end pipeline performance across Airflow, dbt, and infrastructure, implementing observability solutions for proactive issue detection and resolution.
• Enforce data engineering standards, security best practices, and governance policies while supporting financial reporting and regulatory data use cases.

Required Skills

• 10+ years of professional experience in data engineering, analytics engineering, or platform engineering.
• Expert-level proficiency with Apache Airflow, including DAG design, performance tuning, and SLA monitoring.
• Expert-level proficiency with dbt Core, including data modeling, macros, incremental models, and query optimization.
• Strong proficiency in Python for data engineering and automation.
• Deep understanding of Kubernetes and/or OpenShift in production environments.
• Extensive experience with distributed workload management, SQL for complex transformations, and performance optimization.
• Experience running data platforms on cloud environments with familiarity in containerized deployments and Git-based CI/CD workflows.
• Any Graduate degree.

Preferred Skills

• Experience supporting financial services or accounting platforms.
• Exposure to enterprise system migrations from legacy platforms to modern data stacks.
• Experience with Oracle data warehouses.

Key Skills

Education

Any Gradute