Description

You own the design and development of data pipelines that extract, transform, and load data from various sources into appropriate storage systems.

Responsibilities

• Design and develop data pipelines to extract, transform, and load data from diverse sources.
• Integrate data from databases, data warehouses, APIs, and external systems while ensuring data consistency and integrity.
• Transform raw data using cleansing, aggregation, filtering, and enrichment techniques.
• Optimize data processing workflows for performance, scalability, and efficiency.
• Implement data quality checks and validations within pipelines to maintain data accuracy and completeness.

Required Skills

• 8+ years of overall experience in data engineering, including ETL/ELT and data warehousing.
• Proficiency in programming languages: Python, SQL, Java, and C#.
• Experience with Microsoft technologies: Synapse, CosmosDB, HDInsight, Azure Data Factory, Azure DataBricks, and Power BI.
• Experience with data modelling, master data management, and data governance.
• Ability to translate complex technical information for technical and non-technical audiences.
• Critical and creative thinking skills applied to complex data analysis.
• Experience advising decision-makers and influencing outcomes.

Key Skills

Education

Bachelor's degree