Description

Key Responsibilities
Architecture & Platform Strategy

• Design and implement an enterprise-grade Databricks Lakehouse architecture leveraging Delta Lake, Unity Catalog, and Databricks SQL tailored for financial data models.
• Establish architectural patterns for high-throughput batch and real-time streaming data pipelines (e.g., Structured Streaming, Auto Loader) integrating core banking systems, payment gateways, and third-party data providers.
• Drive the migration strategy from legacy data warehouses (e.g., Teradata, Oracle, Snowflake, mainframe environments) to Databricks.

Security, Compliance & Governance

• Implement strict data security, enterprise access controls, and fine-grained permissions using Databricks Unity Catalog to comply with financial regulations (e.g., GDPR, SOX, PCI-DSS).
• Architect data masking, tokenization, and encryption strategies for Personally Identifiable Information (PII) and Sensitive Personal Data (SPD).
• Ensure full data lineage, auditing, and metadata management across multi-cloud or hybrid-cloud deployment environments (Azure).

Data Engineering & Performance Optimization

• Define best practices for PySpark, Delta Live Tables (DLT), and SQL optimization to maximize compute efficiency and reduce total cost of ownership (TCO).
• Establish CI/CD, DevOps, and MLOps automation frameworks for code deployment, data quality validation (e.g., Great Expectations), and model lifecycle management.
• Partner with Data Science, Quantitative Analytics, and BI teams to support advanced financial modeling, predictive risk analysis, and customer 360 initiatives.

Stakeholder & Domain Leadership

• Collaborate with business leaders, Chief Risk Officers (CROs), and compliance teams to align data platform capabilities with core banking business objectives.
• Provide technical leadership, mentorship, and architecture governance to data engineering and analytics teams.

Qualifications & Skills
Required Technical Skills

• Databricks Expertise: 5+ years of hands-on experience designing and running large-scale Databricks environments in production.
• Cloud Architecture: Deep knowledge of AWS, Azure, or GCP data services (e.g., Azure Data Lake Storage/Data Factory, AWS S3/Glue, or GCP GCS/BigQuery).
• Data Processing & Storage: Mastery of Delta Lake, PySpark, Databricks SQL, Structured Streaming, and Delta Live Tables.
• Governance & Security: Strong hands-on experience configuring Databricks Unity Catalog, RBAC, ABAC, and data governance frameworks.
• DevOps & Infrastructure as Code: Proficiency with Terraform, CI/CD pipelines (GitHub Actions, Azure DevOps, Jenkins), and Databricks Asset Bundles (DABs).

Required Banking Domain Knowledge

• Domain Experience: 3+ years working within a retail bank, investment bank, or financial services institution.
• Financial Data Models: Familiarity with core banking operations, payments, lending, credit risk, anti-money laundering (AML), or fraud data structures.
• Regulatory Landscape: Strong understanding of banking data compliance requirements, auditability, and data sovereignty standards.

Education & Certifications

• Education: Bachelor’s or Master’s degree in Computer Science, Data Engineering, Information Systems, or a related quantitative field (or equivalent practical experience).
• Certifications (Preferred):
  Databricks Certified Data Engineer Professional
  Cloud Solutions Architect Certification (AWS Certified Solutions Architect / Azure Solutions Architect Expert)

Key Skills

Education

Bachelor's or Master's degrees