Description

Lead technical architecture and data engineering efforts within the Azure ecosystem.

Responsibilities

• Build and manage Azure Data Factory pipelines using ARM templates and PowerShell.
• Write and implement Spark and Scala programs to process large datasets.
• Optimize and tune Hive and Spark queries for efficiency.
• Perform data analysis and root cause analysis for production issues.
• Manage cloud resources including Azure Data Lake, Storage Accounts, and HD Insights.

Required Skills

• 10+ years of professional experience in data engineering.
• Hands-on expertise with Azure Data Factory and Azure Data Lake.
• Proficiency in PySpark, Spark, and Scala programming.
• Experience with Azure Analysis Services and Azure SQL.
• Working knowledge of Azure Automation Accounts, Batch Accounts, and Azure Databricks.
• Ability to use PowerShell for pipeline deployment and automation.
• Experience tuning Hive and Spark queries.
• Any Graduate degree.

Key Skills

Education

Any Graduate