Description

Design and optimize large-scale data pipelines and analytics solutions within the Microsoft Azure ecosystem.

Responsibilities

• Build and maintain ETL/ELT frameworks using Azure Databricks, PySpark, and Delta Lake.
• Develop automated workflows using Azure Data Factory (ADF) pipelines and Databricks jobs.
• Process structured, semi-structured, and unstructured data using Spark DataFrames and Delta Lake architecture.
• Manage and optimize Databricks clusters and workspaces for performance and cost efficiency.
• Design analytical data models (star/snowflake schemas) and enforce data governance standards.

Required Skills

• Azure Databricks (Spark) and PySpark
• Azure Data Factory (ADF)
• Azure Data Lake Storage (ADLS)
• Delta Lake (ACID transactions, time travel)
• Azure Key Vault for secrets handling
• Data Modeling (Star / Snowflake schemas)
• Distributed Data Processing
• Cloud Data Architecture
• 10+ years of relevant experience

Preferred Skills

• Azure Synapse Analytics
• Cost optimization and performance tuning
• Data governance and security implementation

Key Skills

Education

Any Graduate