Description
Responsibilities:
- Develop and maintain ETL/ELT pipelines using Azure Data Factory (ADF).
- Work with Databricks for data processing and transformation.
- Write and optimize SQL queries for data extraction and transformation.
- Work with PySpark or Python for large-scale data processing.
- Integrate data from multiple sources into data warehouses or data lakes.
- Monitor, troubleshoot, and optimize data pipelines.
- Collaborate with teams to ensure data quality and reliability.
Required Skills:
- Experience with Databricks and Azure Data Factory (ADF).
- Strong understanding of ETL/ELT concepts.
- Good knowledge of SQL and data transformations.
- Basic experience with Python or PySpark.
- Understanding of Azure cloud services and data lake concepts
Education
Bachelor's degree