Description

You will design and maintain data load processes using Azure Data Factory and Azure Databricks. This role owns the full lifecycle of ETL/ELT workflows, from ingestion to storage, ensuring reliable incremental processing and schema management.

This role is on-site.

Responsibilities

• Build and optimize data transformations using Python and PySpark in Azure Databricks.
• Implement Delta Lake-based storage patterns for scalable and reliable data processing.
• Design database schemas, queries, and stored procedures in SQL Server for data warehouse needs.
• Handle semi-structured data formats like JSON, ensuring correct parsing and validation.
• Integrate backend services using .NET Core and Web APIs, with frontend support in React or Angular.

Required Skills

• 5+ years of data engineering experience.
• Expertise in Azure Data Factory for orchestrating data ingestion workflows.
• Proficiency in Azure Databricks and Delta Lake for scalable processing.
• Strong skills in Python and PySpark for data transformation.
• Experience with SQL Server for schema design and query optimization.
• Ability to work with JSON and XML data formats.

Preferred Skills

• Experience with ASP.NET Core and Web APIs for backend integration.
• Familiarity with Angular or React JS for frontend application development.

Key Skills

Education

Any Graduate