Description

You will build and maintain data engineering pipelines using Azure Databricks and Apache Spark.

This role is on-site.

Responsibilities

• Develop and optimize big data processing workflows using Apache Spark.
• Implement real-time data streaming solutions for immediate data ingestion and analysis.
• Write efficient SQL queries and Python scripts to transform and manage data assets.
• Collaborate with stakeholders to define data requirements and ensure delivery.
• Maintain and improve existing data infrastructure for reliability and performance.

Required Skills

• 5+ years of experience in data engineering or related fields.
• Proficiency in Databricks and Apache Spark.
• Strong command of Python for data processing and scripting.
• Advanced SQL skills for query optimization and data manipulation.
• Experience with streaming or real-time data processing frameworks.
• Any Graduate degree.

Preferred Skills

• Experience with Azure cloud services beyond Databricks.
• Familiarity with CI/CD pipelines for data engineering tasks.

Key Skills

Education

Any Graduate