Description

You will manage and optimize the lifecycle of data pipelines and infrastructure. You own the design, development, and implementation of data pipelines using Apache Airflow and dbt.

Responsibilities

• Design, develop, and implement data pipelines using Apache Airflow.
• Monitor and optimize ELT pipelines for performance, reliability, and scalability.
• Troubleshoot data pipeline issues to ensure data integrity and quality.
• Utilize dbt for data transformation and establish operational best practices.
• Implement data governance practices including data lineage and metadata management.

Required Skills

• Minimum 10 years of hands-on experience in DataOps or a similar role.
• Proficiency in Apache Airflow for data orchestration.
• Strong experience with Python, SQL, and Git.
• Hands-on experience with dbt for data transformation.
• Experience with AWS services including Redshift, MWAA, Glue, Lambda, and S3.
• Knowledge of DevOps concepts such as CI/CD and containerization.
• Familiarity with GitLab and data observability tools.
• Proficiency in TypeScript and AWS cloud platforms.

Preferred Skills

• Bachelor's degree in Computer Science.

Key Skills

Education

Bachelor's degree in Computer Science