Description

You will design and maintain scalable data storage solutions and pipelines on the Azure platform. You own the end-to-end lifecycle of data ingestion, processing, and transformation, ensuring data quality and security.

Responsibilities

• Build and maintain data storage architectures using Azure SQL Database, Azure Data Lake, and Azure Blob Storage.
• Design and implement ETL operations and data pipelines using Azure Data Factory.
• Assemble and process large, complex datasets using Azure Databricks and Apache Spark.
• Implement data validation and cleansing procedures to ensure integrity and reliability.
• Collaborate with stakeholders to translate requirements into scalable data platform architectures.

Required Skills

• 5+ years of experience in data engineering.
• Strong proficiency in Python and SQL.
• Hands-on experience with Azure Databricks and Apache Spark for big data processing.
• Expertise in Azure Data Factory for ETL orchestration.
• Experience with Azure Data Lake and Blob Storage.
• Familiarity with Azure DevOps for CI/CD practices.
• Undergraduate degree or equivalent experience.

Preferred Skills

• Experience ensuring data security and compliance within cloud environments.

Key Skills

Education

Any Gradute