Description

• Design and implement scalable, secure, and high-performance data architecture on Microsoft Azure, supporting both cloud-native and hybrid environments.
• Lead the development of data ingestion, transformation, and integration pipelines using Azure Data Factory, Azure Databricks, and Azure Synapse Analytics.
• Work with the Data Architect and manage data lakes and structured storage solutions using Azure Data Lake Storage Gen2, ensuring efficient access and governance.
• Integrate data from diverse source systems including ServiceNow, and geospatial systems, using APIs, connectors, and custom scripts.
• Develop and maintain robust data models and semantic layers to support operational reporting, analytics, and machine learning use cases.
• Build and optimize data workflows using Python and SQL for data cleansing, enrichment, and advanced analytics within Azure Databricks.
• Design and expose secure data services and APIs using Azure API Management for downstream systems.
• Implement data governance practices, including metadata management, data classification, and lineage tracking.
• Ensure compliance with privacy and regulatory standards (e.g., FOIP, GDPR) through role-based access controls, encryption, and data masking.
• Monitor and troubleshoot data pipelines and integrations, ensuring reliability, scalability, and performance across the platform.
• Utilize AI and automation tools to streamline data engineering workflows, including pipeline development, testing, monitoring, and documentation.
• Leverage AI-assisted tools for code generation, optimization, and review to improve development efficiency and code quality.
• Design and curate standardized, high-quality datasets that are suitable for advanced analytics and future AI use cases.
• Other duties as needed

Key Skills

Education

Bachelor's degree