Description

You will design and maintain data pipelines and analytics solutions within the Azure ecosystem.

Responsibilities

• Build and manage data pipelines using Azure Data Factory and ARM templates.
• Develop and execute Spark and Scala programs to process large datasets.
• Optimize and tune Hive and Spark queries for performance.
• Automate workflows using PowerShell and Azure Automation Accounts.
• Perform data analysis and conduct root cause analysis for production issues.

Required Skills

• 10 to 18 years of experience in data engineering.
• Hands-on experience with Azure Data Factory and Azure Data Lake.
• Proficiency in PySpark, Spark, and Scala programming.
• Experience with Azure SQL and Azure Analysis Services.
• Knowledge of Azure Storage Accounts, HD Insights, and Azure Databricks.
• Ability to use PowerShell for pipeline deployment and automation.
• Experience managing Batch Accounts and Azure Automation.
• Any Graduate degree.

Key Skills

Education

Any Graduate