Description

You will design and maintain data pipeline architectures to handle large, complex datasets. You will own the infrastructure for extracting, transforming, and loading data from various sources using SQL and Azure big data technologies.

Responsibilities

• Build and modify Azure Data Pipelines, Spark Python/SQL jobs, and ETL processes for the Enterprise Data Warehouse.
• Automate manual processes and optimize data delivery to improve scalability.
• Create analytics tools and data resources to support data science teams and business performance metrics.
• Troubleshoot Azure cloud services and resolve production issues through performance tuning.

Required Skills

• 3+ years of experience in data engineering.
• Strong experience with Azure Data Factory, Databricks, and Azure Data Lake.
• Proficiency in Spark, PySpark, Python, and SQL.
• Expertise in ETL and Data Warehouse (DW) concepts.
• Hands-on experience managing MLOps pipelines.
• Understanding of data governance, master data management, and data catalog solutions.
• Practical knowledge of Azure Cloud services, specifically regarding error handling and performance tuning.
• Strong analytical skills with a solid grasp of data structures and algorithms.
• Bachelor's degree.

Preferred Skills

• Exposure to Trino/Presto, Docker, and Kubernetes.

Key Skills

Education

Bachelor's degree