Description

• Design and develop scalable, reliable data pipelines on Azure to collect, process, and store large datasets
• Implement efficient data models and data warehousing solutions aligning with business requirements
• Collaborate with data scientists, analysts, and application teams to integrate data workflows
• Optimize data pipeline performance, reliability, and security
• Support data governance, compliance, and data privacy standards
• Monitor operational health of data solutions and troubleshoot issues proactively
• Document architecture, workflows, and best practices for data ingestion and transformation
• Stay updated with the latest Azure data platform innovations and incorporate improvements
  
   

Software Requirements

 

• Proficiency in SQL and Python for data processing and scripting
• Hands-on experience with Azure Data Services such as Azure Data Factory, Azure Data Lake, Azure Databricks, and Azure Synapse Analytics
• Understanding of data warehousing, data modeling, and ETL/ELT processes
• Experience implementing data pipelines for ingestion, transformation, and storage
• Knowledge of Azure Blob Storage, Cosmos DB, and relational databases (Azure SQL Database, SQL Server)
• Familiarity with DevOps practices for deployment and continuous integration in the cloud
• Experience with Spark, Scala, or Java in Azure Databricks
• Knowledge of Power BI or other data visualization tools
• Exposure to DataOps, automation, and monitoring tools within Azure
  
   

Category-wise Technical Skills

Data Processing & Frameworks:

 

• Azure Data Factory, Azure Databricks, Azure Data Lake, Synapse Analytics
• Python, PySpark, Scala, or Java for data processing
• SQL, T-SQL, and experience with relational and non-relational data stores
  
   

Cloud & Infrastructure:

 

• Azure AI & Machine Learning services (optional)
• Azure Storage options: Blob, Data Lake, Cosmos DB
• Azure DevOps, CI/CD pipelines, Infrastructure as Code (ARM templates, Terraform)
  
   

Data Modeling & Warehousing:

 

• Star schema, data marts, normalization/denormalization principles
• Data governance, lineage, and security standards
  
   

Tools & Monitoring:

 

• Power BI, Azure Monitor, Log Analytics for data visualization and monitoring
• Version control systems like Git
  
   

Experience

 

• 5+ years of experience in data engineering, data architecture, or related roles
• Proven track record of designing and deploying data pipelines using Azure services
• Hands-on experience with cloud data solutions in a commercial or enterprise environment
• Experience working with cross-functional teams on large-scale data projects
• Familiarity with big data processing frameworks and emerging data technologies

Key Skills

Education

Any Gradute