Description

You design and develop data pipelines that extract, transform, and load data from various sources into appropriate storage systems.

Responsibilities

• Integrate data from databases, data warehouses, APIs, and external systems.
• Ensure data consistency and integrity by performing validation and cleaning during integration.
• Transform raw data using cleansing, aggregation, filtering, and enrichment techniques.
• Optimize data pipelines and processing workflows for performance and scalability.
• Monitor and tune data systems, resolving bottlenecks and implementing indexing strategies.

Required Skills

• 8+ years of overall experience in data engineering, including data modelling and ETL/ELT.
• Proficiency in programming languages such as Python, SQL, Java, and C#.
• Experience with Data Integration and ETL/ELT processes.
• Familiarity with SQL Server.
• Working knowledge of Azure Data Factory and Azure Databricks.
• Experience with data quality checks and governance practices.
• Ability to translate complex technical information for various audiences.

Key Skills

Education

Bachelor's degree