Description

You will design, develop, and maintain data pipelines, models, and storage solutions within the Azure ecosystem.

Responsibilities

• Build data pipelines using Azure Data Factory to ingest data from SQL Server, MongoDB, Blob Storage, and Kafka into Azure SQL Database and Azure Cosmos DB.
• Develop data transformation logic using Databricks and Spark for processing and analytics.
• Design and implement data models for data warehousing on Azure SQL DB.
• Configure and manage Hadoop and Spark clusters in Azure HDInsight for big data processing.
• Implement data governance and security measures to protect sensitive information.

Required Skills

• 5+ years of experience in data engineering.
• Proficiency in Python and SQL.
• Experience with C#.
• Hands-on expertise with Azure Data Factory.
• Experience managing Azure SQL Database and Azure Cosmos DB.
• Direct experience tuning Databricks environments.
• Knowledge of Spark and Hadoop cluster management.
• Bachelor’s Degree in a relevant field.

Key Skills

Education

Bachelor’s Degree