Description

Design and build scalable data pipelines and ETL processes within the Azure ecosystem.

Responsibilities

• Design and develop scalable data pipelines and ETL processes using Azure Data Factory and Azure Databricks.
• Implement data storage solutions using Azure SQL Database, Azure Data Lake, and Azure Cosmos DB.
• Create and maintain data models and schemas for efficient storage and retrieval.
• Optimize and troubleshoot data pipelines to ensure performance and reliability.
• Ensure data quality and integrity through validation and cleansing techniques.

Required Skills

• Minimum of 6 years of experience in data engineering.
• At least 3 years of experience specifically working with Azure data services.
• Strong expertise in Azure Data Factory and Azure Databricks.
• Proficiency in SQL and Python.
• Hands-on experience with Azure Data Lake, Azure SQL Database, and Azure Cosmos DB.
• Solid understanding of data modeling, ETL processes, and data integration.
• Experience with data warehousing and big data technologies.

Preferred Skills

• Experience with data security and compliance including encryption, access controls, and monitoring.

Key Skills

Education

Bachelor's degree in Computer Science