Description

You will design and deliver scalable data and AI solutions on Databricks, balancing architectural direction with hands-on implementation.

Responsibilities

• Design and implement end-to-end data solutions using Spark, Delta Lake, notebooks, jobs, and clusters within a Lakehouse architecture.
• Define reference architectures, best practices, and technical standards for data and AI workloads.
• Implement CI/CD pipelines, DevSecOps practices, and Infrastructure as Code for data platform deployments.
• Ensure strong data governance, security, and cost optimisation across cloud environments.
• Translate business requirements into scalable technical designs through close collaboration with stakeholders and engineering teams.

Required Skills

• 7+ years of data architecture and engineering experience.
• 3–4+ years of hands-on experience with Databricks in production environments.
• Strong expertise with Spark, Delta Lake, Databricks jobs, notebooks, and cluster management.
• Proven experience defining architectural standards and best practices.
• Strong knowledge of at least one cloud platform: Azure, AWS, or GCP.
• Experience with CI/CD, DevSecOps, and infrastructure as code (e.g., Terraform).
• Solid understanding of security, governance, and cost management on cloud data platforms.

Preferred Skills

• Experience integrating Databricks with dbt Cloud, Snowflake, Fivetran, or BI tools.
• Background in financial services or capital markets.
• Experience with Machine Learning or AI workloads on Databricks.

Key Skills

Education

Any Graduate