Description

You will lead the engineering of scalable end-to-end data pipelines and digital data management solutions.

Responsibilities

• Architect and develop end-to-end data pipelines across multiple data sources and systems of record.
• Design data models, structures, and ETL jobs for data acquisition and transformation.
• Develop data services and APIs to expose RESTful data services.
• Manage and maintain cloud-based data and analytics platforms, including proof of concept prototyping.
• Ensure data quality, integrity, security, and completeness throughout the entire data lifecycle.

Required Skills

• 10+ years of relevant experience in data engineering.
• Bachelor’s degree in computer science, computer engineering, or a related field.
• Expertise in Azure Cloud platform service offerings and Azure SQL, including complex queries and performance tuning.
• Experience with Azure Functions and Azure API Management Service (APIMS).
• Hands-on experience with real-time data processing using Azure Event Hubs, Azure Service Bus, and Azure Stream Analytics.
• Proficiency in creating pipelines and complex transformations using .Net/C#, Python, Java, R, or Scala with Databricks/Spark.
• Experience with NoSQL databases, specifically Azure CosmosDB or other document stores.
• Experience with Azure Data Factory (ADF) for pipeline creation, scheduling, and monitoring.
• Experience using Azure Purview for data cataloging, lineage, and metadata management.

Preferred Skills

• Experience with Azure Synapse Analytics, Azure Analysis Services, and Power BI.
• Understanding of MPP databases and large-scale data processing.

Key Skills

Education

Bachelor’s degree in computer science