Description

Design, build, and maintain scalable data pipelines using Databricks, Spark, and Azure technologies. You will own the ETL lifecycle, optimize performance, and ensure data reliability across cloud environments.

This role is hybrid.

Responsibilities

• Design and maintain data pipelines using Databricks, Spark, and Azure cloud services.
• Implement ETL processes, data warehouse solutions, and robust data validation methods.
• Optimize pipeline performance, scalability, and reliability for large-scale data workloads.
• Troubleshoot data-related issues and perform root cause analysis to prevent recurrence.
• Collaborate with data scientists and analysts to meet technical data requirements.

Required Skills

• 5+ years of data engineering experience with Databricks and Spark.
• Proficiency in Python, PySpark, and SQL.
• Hands-on experience with Azure Data Factory (ADF).
• Experience with Azure Databricks Medallion Architecture, DLT, and Iceberg.
• Knowledge of Azure cloud platform and services.
• Understanding of data warehouse and data lake architectures.
• Bachelor's degree in Computer Science, Engineering, or equivalent work experience.

Preferred Skills

• Experience in a Financial or Corporate Banking context.
• Exposure to big data technologies such as Kafka, Hadoop, or Hive.

Key Skills

Education

Any Graduate