Description

Manage and optimize ETL pipelines within a large-scale, petabyte-level data lake environment.

Responsibilities

• Design and implement Azure data pipelines for scalable data ingestion.
• Develop and optimize Kusto Query Language (KQL) queries for telemetry analysis, alerting, and dashboarding.
• Build and maintain custom dashboards and alerts for real-time system insights.
• Collaborate with DevOps, SRE, and application teams to ensure end-to-end observability and performance tracking.

Required Skills

• 5+ years of experience in data engineering.
• Strong hands-on experience with Azure Data Factory, Databricks, and Azure Functions.
• Proficiency in Python, PySpark, or Scala.
• Experience with Kusto Query Language (KQL) and Azure Data Explorer (ADX).
• Hands-on experience with Azure Event Hub, Event Grid, and Blob Storage.
• Solid understanding of Azure cloud resources, pipelines, and data lake environments.
• Experience working in high-scale environments managing terabytes to petabytes of data.
• Knowledge of automation tools including PowerShell, ARM templates, or Terraform.

Preferred Skills

• Microsoft certifications such as AZ-104, AZ-400, or DP-500.

Key Skills

Education

Any Gradute