Description

You will design, optimize, and maintain data pipelines and database systems processing millions of financial transactions daily for the Banking Foundation Platform.

Responsibilities

• Design, build, and maintain scalable ETL data pipelines using Azure Databricks, Python, Azure SQL Database, and Azure Cosmos DB.
• Perform deep database performance tuning, query optimization, and indexing across Azure SQL Database and Cosmos DB.
• Lead database migration initiatives, specifically migrating from Azure SQL Database and CosmosDB to PostgreSQL.
• Implement efficient data models and manage data storage using ADLS Gen2, integrating real-time streaming via ADF, Spark, and Confluent Kafka.
• Develop and maintain data quality checks, monitoring, and alerting systems, ensuring compliance with standards like PCI DSS.

Required Skills

• 10+ years of hands-on experience in data engineering with strong database expertise.
• 5+ years implementing an Operational Data Store (ODS) for retail banking, handling high throughput and low latency using RDBMS or NoSQL.
• Hands-on experience with ADLS Gen2, Azure Blob storage, Azure Databricks, Azure SQL Database, and Azure Cosmos DB.
• Expert-level skills in database performance tuning, query optimization, and execution plan analysis.
• Proven experience with large-scale database migrations, preferably Azure SQL Database to PostgreSQL.
• Strong programming skills in Python for data processing and automation.
• Experience with ETL/ELT processes, data modeling, and data warehouse concepts.
• Familiarity with Azure platform engineering using PowerShell and ARM Templates.

Key Skills

Education

Any Graduate