Description

Design, develop, and maintain data pipelines, models, and storage solutions within the Azure ecosystem.

Responsibilities

• Design and maintain data pipelines using Azure Data Factory to ingest data from SQL Server, MongoDB, Blob Storage, Azure SQL Database, Cosmos DB, and Kafka.
• Build transformation logic using Databricks and Spark, including tuning Databricks environments.
• Implement data models for warehousing on Azure SQL DB and manage storage via Azure Blob and Data Lake.
• Configure and manage Hadoop and Spark clusters in Azure HDInsight for big data processing.
• Enforce data governance and security measures to protect sensitive information.

Required Skills

• 5+ years of experience managing data platforms and architecture using Azure services.
• Proficiency in Python, SQL, and C#.
• Hands-on experience with Azure Data Factory, Azure SQL Database, Azure Cosmos DB, and Azure HDInsight.
• Strong background in data warehousing, data integration, and ETL tools.
• Experience with NoSQL databases including MongoDB and Cassandra.
• Working knowledge of SQL Server, Kafka, and Spark.
• Proficiency with version control tools such as Git and Bitbucket.
• Degree in Computer Science or a related field.

Preferred Skills

• Familiarity with Aveva/OSISoft PI systems and SCADA systems.

Key Skills

Education

Any Graduate