Description

You will build and manage data pipelines within the Azure ecosystem.

Responsibilities

• Design and develop data pipelines using Azure Databricks and PySpark.
• Implement data transformation logic to process large datasets efficiently.
• Create, configure, and monitor Apache Spark clusters within Azure Databricks.
• Manage ETL processes to ensure data integrity and flow.

Required Skills

• 5+ years of experience in data engineering roles.
• Proficiency in PySpark for large-scale data processing.
• Hands-on experience with Azure Databricks.
• Experience building and managing Azure Data Factory pipelines.
• Strong understanding of ETL processes.
• Experience with Big Data processing techniques.
• Any Graduate degree.

Key Skills

Education

Any Graduate