Description

You will build and maintain scalable data pipelines and warehousing solutions on the Azure cloud platform.

This role is hybrid.

Responsibilities

• Design and implement ETL/ELT processes using Databricks, PySpark, and Spark SQL.
• Develop data transformation models and manage schema using DBT.
• Orchestrate data workflows and pipelines with Prefect and Fivetran.
• Manage CI/CD pipelines and version control via Azure DevOps.
• Optimize data storage and ingestion within Azure Data Lake and Azure Data Factory.

Required Skills

• 10+ years of hands-on experience as a Data Engineer.
• Proficiency in Python and SQL for data manipulation and analysis.
• Deep expertise with Databricks, including PySpark and Spark SQL.
• Strong experience with DBT for data transformation.
• Solid background in Azure Data Lake, Azure Data Factory, and Azure Data Services.
• Experience with data integration tools such as Fivetran.
• Familiarity with workflow orchestration tools like Prefect.
• Knowledge of CI/CD practices using Azure DevOps.

Preferred Skills

• Experience with modern data platform architectures and data warehousing best practices.

Key Skills

Education

Any Graduate