Description

You will build and maintain data integration solutions using Azure services, migrating legacy ETL processes to modern cloud architectures.

This role is hybrid.

Responsibilities

• Develop and maintain ETL pipelines to extract, transform, and load data from various sources into Databricks.
• Migrate ETL processes from DataStage to Azure Data Factory.
• Implement data engineering best practices, including data quality checks, data lineage, and data governance.
• Automate the deployment and management of data pipelines and infrastructure using Azure DevOps and CI/CD practices.
• Architect and design Databricks deployment strategies for data processing, transformation, and integration.

Required Skills

• 6+ years of experience in data engineering, with 2+ years in a technical lead role.
• 3+ years of hands-on experience with Azure Data Factory, Azure SQL Database, and other Azure data services.
• Proficiency in Python and SQL for data manipulation and scripting.
• Experience working with SQL and NoSQL databases within Azure.
• Deep understanding of Azure Cloud Platform and Engineering Infrastructure setup.
• Familiarity with data warehousing concepts and tools like Azure Synapse Analytics.
• Experience implementing data governance, security, and compliance best practices in Azure.

Preferred Skills

• Hands-on experience with PySpark using Databricks Notebooks and Workspace.
• Strong understanding of distributed computing principles and data processing frameworks.

Key Skills

Education

Any Graduate