Description

Key Responsibilities:

- Design and build scalable data pipelines for ingestion, transformation, and delivery across source and target systems to ensure high data availability for downstream consumers.
- Implement and maintain Data Vault 2.0 models, including Hubs, Links, Satellites, and PIT/Bridge tables on Snowflake to support complex analytical requirements.
- Author dbt models and Airflow DAGs for pipeline orchestration, scheduling, and dependency management to maintain efficient and reliable data workflows.
- Develop and enforce comprehensive data quality frameworks, including test cases, validation SQL, and monitoring, to ensure data integrity across the entire pipeline.
- Manage CI/CD for data pipeline deployments, maintaining rigorous code repository and version control standards to ensure stable and reproducible releases.
- Build and maintain metadata stores, data catalogs, and lineage tracking to provide transparency and governance across the data platform.
- Collaborate with AI/ML engineers to validate and productionise machine-generated code and pipeline artefacts, ensuring seamless integration between data engineering and data science workflows.

Required Skillset:

- Demonstrated expertise in designing and managing data architectures on Snowflake, with a deep understanding of Data Vault 2.0 methodologies.
- Proficiency in Python for data manipulation and automation, combined with hands-on experience in orchestrating complex workflows using Apache Airflow.
- Advanced skills in dbt for modular data transformation and building maintainable SQL-based data models.
- Ability to translate complex business requirements into technical specifications while communicating effectively with stakeholders and technical team members.
- Strong analytical mindset with a focus on building automated testing and monitoring frameworks to ensure high data reliability.
- Proven experience in managing CI/CD pipelines and version control systems to support agile development cycles

Key Skills

Education

Bachelor's degree