Description

Key Responsibilities

• Pipeline Architecture & Strategy: Architect, build, and optimize high-throughput, fault-tolerant batch and streaming data pipelines capable of handling mission-critical financial transactions and accounting workflows.
• Technical Leadership & Delivery: Lead a pod of data engineers, conduct rigorous code reviews, enforce design best practices, and ensure timely delivery aligned with banking release cycles.
• Data Quality & Reconciliation: Implement automated data validation, integrity checks, and ledger reconciliation frameworks to adhere to strict banking accuracy standards.
• ETL Frameworks & Optimization: Build reusable, modular ETL/ELT frameworks; tune distributed cluster compute, query execution plans, and storage layout to meet aggressive SLAs.
• Stakeholder Management: Work closely with delivery heads, enterprise architects, product owners, and risk/compliance officers to translate banking business requirements into scalable technical implementations.

Mandatory Skills & Qualifications

• Experience: Minimum 8+ years of hands-on data engineering experience, including at least 2 years in a technical leadership or module lead capacity.
• Databricks: Deep expertise in Databricks Lakehouse architecture, Delta Lake features (Time Travel, ACID transactions, vacuuming, compaction), and Unity Catalog governance.
• PySpark & Distributed Computing: Advanced proficiency with Apache Spark / PySpark (dataframes, RDDs, partitioning strategies, memory management, shuffle tuning).
• Python & SQL: Strong software engineering fundamentals in Python (OOP, error handling, unit testing) and expert-level SQL (window functions, query tuning, complex aggregations).
• ETL & Data Integration: Proven track record designing enterprise-scale ETL pipelines integrating diverse legacy systems, core banking databases, and modern data warehouses.
• BFSI Domain Knowledge: Solid familiarity with banking data models (e.g., core transactions, cards, loans, AML, or risk reporting)

Key Skills

Education

Any Graduate