Description

You will build and optimize big data pipelines, architectures, and datasets within the Azure ecosystem.

Responsibilities

• Develop ETL pipelines using Azure Databricks and Spark.
• Build and optimize big data architectures and datasets.
• Perform data analysis, manipulation, and statistical modeling on structured and unstructured datasets.
• Deploy data pipelines through CI/CD using Azure DevOps.

Required Skills

• 5+ years of experience with Azure Data Lake Storage Gen2, Azure Data Factory, Databricks, Event Hub, and Azure Synapse.
• Proficiency in Spark, Python, Scala, and SQL.
• Hands-on experience with Azure Databricks for ETL development.
• Experience managing structured and unstructured datasets.
• Practical knowledge of Azure DevOps for pipeline deployment.
• Familiarity with Docker, Azure Kubernetes Service, Azure DWH, API Azure, Azure Function, or Azure Cognitive Services.

Preferred Skills

• Experience with Power BI for data visualization.

Key Skills

Education

Any Graduate