Description

You will design, develop, and maintain data pipelines to support data flow into customer-facing digital applications.

Responsibilities

• Build ETL processes to extract, transform, and load data from various sources into data stores.
• Develop and manage data pipelines using Python and Astronomer Airflow.
• Implement data validation and quality checks to ensure accuracy and integrity.
• Monitor pipelines to proactively identify and resolve issues, minimizing latency.
• Document pipeline designs, data flows, and operational procedures.

Required Skills

• 5+ years of experience in data engineering roles.
• Proficiency in Python programming.
• Hands-on experience with Apache Airflow or Astronomer Airflow, including DAG creation and scheduling.
• Strong knowledge of ETL processes and data integration techniques.
• Expertise in SQL and relational databases.
• Experience with Microsoft Azure and cloud platforms.
• Practical knowledge of Data Modelling and Docker.
• Ability to work within the UK time zone.

Key Skills

Education

Any Graduate