Description

Lead the modernization of legacy Cloudera platforms into Databricks Lakehouse architectures and design scalable data models for finance and risk domains.

This role is on-site.

Responsibilities

• Lead modernization of legacy Cloudera platforms (CDH, CDP, Hive, HBase, Impala, Spark) into Databricks Lakehouse.
• Redesign ingestion and transformation patterns from HDFS-centric architectures to cloud object storage and Delta Lake.
• Refactor legacy Hive/Impala logic into PySpark and Spark SQL based ELT pipelines ensuring data parity and audit integrity.
• Design and govern enterprise Data Warehouse and Data Lake architectures with layered structures for raw, curated, and semantic consumption.
• Implement semantic consumption layers using Databricks SQL, Delta tables, and dbt to support BI, finance, and risk analytics.

Required Skills

• 12+ years of experience in data engineering and architecture.
• Deep expertise with Databricks ecosystem, including Delta Lake and Databricks SQL.
• Strong proficiency in PySpark and Spark SQL for large-scale pipeline engineering.
• Experience with dbt or equivalent transformation frameworks.
• Proven track record designing Data Warehouse and Data Lake architectures.
• Ability to modernize legacy Cloudera/CDH environments.
• Understanding of finance and risk data modeling (GL, sub-ledger, risk exposure).

Preferred Skills

• Experience with General Ledger and sub-ledger data structures.
• Knowledge of regulatory reporting and management reporting use cases.

Key Skills

Education

Any Gradute