Description

You will architect and develop end-to-end data pipelines across multiple data sources and systems of record.

Responsibilities

• Design and develop data models, structures, and ETL jobs for acquisition and transformation.
• Manage and maintain cloud-based data and analytics platforms, ensuring data quality and security throughout the lifecycle.
• Develop data services and APIs to expose RESTful data services using Azure API Management Service.
• Translate business stakeholder requirements into technical solutions and prototype solutions through proofs of concept.
• Create, schedule, and monitor Azure Data Factory pipelines and implement master data management standards.

Required Skills

• 10+ years of relevant experience with a Bachelor’s degree in Computer Science, Computer Engineering, or a related field.
• Expertise in Azure Cloud platform service offerings and Azure SQL, including complex queries, stored procedures, and query optimization.
• Hands-on experience with real-time data capture and processing using Azure Event Hubs and Azure Service Bus.
• Proficiency in building pipelines and complex transformations using Databricks/Spark and one of: .Net/C#, Python, Java, R, or Scala.
• Experience with NoSQL databases such as Azure CosmosDB or other document stores.
• Experience using Azure Purview for data cataloging, classification, lineage, and metadata management.
• Experience configuring data quality management workflows using Profisee.
• Working knowledge of ADLS Gen2, Azure Blob storage, and data formats like Parquet, Avro, CSV, and ORC.
• Experience creating, scheduling, and monitoring Azure Data Factory (ADF) pipelines.

Preferred Skills

• Experience with Azure Synapse Analytics, Azure Analysis Services, and Power BI.
• Understanding of MPP databases and large-scale data processing.

Key Skills

Education

Bachelor’s Degree