Description

You will design, build, and maintain scalable data pipelines and solutions on the Azure cloud platform.

This role is on-site.

Responsibilities

• Design and develop efficient ETL/ELT pipelines using PySpark, SQL, and Azure Data Factory.
• Build and optimize data models, partitioning strategies, and performance tuning for large datasets using Azure SQL Database and Azure Synapse Analytics.
• Implement data solutions using Azure Data Lake Storage (ADLS), Azure Databricks, Azure Functions, and Synapse Pipelines.
• Develop and maintain infrastructure as code using Terraform, ARM templates, or Bicep.
• Provide technical leadership and mentorship to junior engineers.

Required Skills

• 6+ years of hands-on experience in data engineering roles.
• Strong expertise with Azure Databricks, Azure SQL, Azure Data Factory, and ADLS.
• Proficiency in Python for data manipulation and workflow scripting.
• Advanced SQL skills including performance tuning, data modeling, and partitioning strategies.
• Hands-on experience with Apache Spark (PySpark) and distributed computing.
• Experience with infrastructure as code (Terraform, Bicep, or ARM templates).
• Knowledge of Azure security practices, including RBAC, Key Vault, and Purview.
• Experience with Shell Scripting.

Preferred Skills

• Experience with Kafka on Azure.
• Integration experience with Power BI or Azure Logic Apps.

Key Skills

Education

Any Graduate