Description

Design and maintain data pipeline architectures to handle large, complex datasets.

This role is on-site.

Responsibilities

• Build and modify Azure Data Pipelines, Spark Python/SQL jobs, and ETL processes for the Enterprise Data Warehouse.
• Own the infrastructure for extracting, transforming, and loading data from various sources using SQL and Azure big data technologies.
• Automate manual processes and optimize data delivery to improve scalability.
• Create analytics tools and data resources to support data science teams and business performance metrics.
• Troubleshoot Azure cloud services and resolve production issues through performance tuning.

Required Skills

• 3+ years of experience in data engineering.
• Strong experience with Azure Data Factory, Databricks, and Azure Data Lake.
• Proficiency in Spark, PySpark, Python, and SQL.
• Expertise in ETL and Data Warehouse (DW) concepts.
• Hands-on experience managing MLOps pipelines.
• Understanding of data governance, master data management, and data catalog solutions.
• Practical knowledge of Azure Cloud services, specifically regarding error handling and performance tuning.
• Strong analytical skills with a solid grasp of data structures and algorithms.
• Bachelor's degree.

Preferred Skills

• Exposure to Trino/Presto, Docker, and Kubernetes.

Key Skills

Education

Bachelor's degree