Description

Design and implement enterprise data solutions for state-level systems using Microsoft Fabric and Azure Databricks.

Responsibilities

• Design end-to-end data architecture leveraging Microsoft Fabric, including OneLake storage strategies and Synapse Analytics workspaces.
• Architect data integration patterns between IES Gateway and analytics platforms using Delta Lake and medallion architecture.
• Implement modern lakehouse architectures and scalable ETL/ELT pipelines using Azure Databricks.
• Establish data governance frameworks using Microsoft Purview and Unity Catalog for lineage, compliance, and row-level security.
• Optimize large-scale data processing through Spark configuration tuning, partitioning strategies, and caching mechanisms.

Required Skills

• 6+ years of experience in data architecture and engineering.
• 2+ years of hands-on experience with Azure Databricks and Apache Spark.
• Recent professional experience with the Microsoft Fabric platform.
• Proficiency in SQL, including complex JOINs, subqueries, and window functions.
• Strong Python and PySpark skills for large-scale data processing.
• Experience implementing Delta Lake for ACID transactions and schema enforcement.
• Knowledge of designing security patterns for sensitive data handling and audit logging.
• Experience with Terraform for infrastructure management.

Preferred Skills

• Bachelor's degree in Computer Science or a related field.

Key Skills

Education

Bachelor's Degree