Description

You will design and build large-scale data architectures for energy systems.

This role is on-site.

Responsibilities

• Build petabyte-scale data lakes using Azure ADLS Gen2 and Delta Lake.
• Develop real-time data pipelines utilizing Kafka or Event Hubs and Spark Structured Streaming.
• Design data models and databases for energy systems, including meter data and SCADA telemetry integration.
• Implement ETL processes using DBT, Airflow, and APIs.
• Optimize Databricks workflows using Unity Catalog and Delta Live Tables.

Required Skills

• 5+ years experience building petabyte-scale data lakes on Azure (ADLS Gen2, Delta Lake).
• Expert level proficiency with Databricks, including Unity Catalog and Delta Live Tables.
• Deep experience optimizing Spark jobs.
• Experience designing time-series databases like InfluxDB.
• Hands-on experience with ETL tools such as DBT and Airflow.
• Proficiency with Kafka and Event Hubs for real-time data ingestion.
• Strong knowledge of API integration patterns.
• Experience in database design specific to energy systems (EMS/BMS integration).

Preferred Skills

• Experience with Azure cloud infrastructure.
• Background in energy sector data systems.

Key Skills

Education

Any Graduate