Description

You will design and maintain scalable data pipelines for large-scale cloud datasets.

This role is remote.

Responsibilities

• Design and maintain ETL pipelines using Azure Data Factory and Databricks for ingestion and integration.
• Build and optimize Databricks notebooks with PySpark and SparkSQL for batch and streaming workloads.
• Migrate legacy ETL workflows into scalable Databricks-based processes.
• Integrate data from APIs, relational databases, and cloud stores into Azure Data Lake and Synapse.
• Implement data validation, monitoring, and quality checks across all data layers.

Required Skills

• 5+ years of experience in data engineering or ETL development within a cloud ecosystem.
• Strong hands-on experience with Azure Databricks (PySpark, SparkSQL, Delta Lake).
• Proficiency with Azure Data Factory (ADF) pipelines and orchestration.
• Experience utilizing Azure Synapse, Data Lake, and SQL Database.
• Solid SQL development skills, including writing optimized queries and DDL scripts.
• Experience integrating data from APIs (REST/JSON) and third-party platforms.
• Familiarity with source control (Git) and DevOps CI/CD pipelines.

Key Skills

Education

Any Graduate