Description

Design and build enterprise data warehouse solutions using Spark and cloud technologies. Develop frameworks to load and transform data using data vault modeling for large-scale historical data consumption. Build and optimize big data pipelines, architectures, and datasets. Implement performance tuning and quality improvements for large-scale systems. Apply DevOps engineering practices including micro-services, CI/CD pipelines, and test automation.

Responsibilities

• Design next-generation data warehouse solutions using cloud technology and industry standards.
• Develop frameworks for data loading and transformation using data vault modeling.
• Build and optimize big data pipelines and architectures.
• Implement performance tuning and quality improvements for large-scale systems.
• Apply DevOps practices including CI/CD pipelines and test automation.

Required Skills

• 15+ years of industry experience in enterprise data warehouse solutions.
• 8-10+ years working in large global teams on high-performance, large-scale systems.
• Expertise in Databricks and Spark using Python or Scala.
• Experience with Spark/Databricks batch and streaming solutions, including Delta Lake and Lakehouse.
• Proficiency with Microsoft Azure, Azure Data Factory, ADLS Gen2, Kafka, and Event Hubs.
• Strong foundation in Algorithms and Data Structures.
• Experience with Infrastructure-as-Code using Terraform, ARM, Bicep, or CloudFormation.
• Ability to use Azure CLI and set up ADO pipelines and Terraform Enterprise.
• Bachelor’s or Master’s degree in Computer Science, Computer Engineering, or a related technical discipline.

Key Skills

Education

Bachelor's degree in Computer Science