Description

Key Responsibilities

• Design, develop, and maintain scalable ETL pipelines and data processing frameworks.
• Build and support Data Mart solutions to meet business and analytical requirements.
• Develop high-quality, maintainable, and efficient code using Python and PySpark.
• Participate in the complete Software Development Life Cycle (SDLC), including:
• Requirement Analysis
• Development
• Unit Testing
• UAT Support
• Defect Fixing
• Production Deployment
• Post-Production Support
• Perform complex data analysis and troubleshoot data-related issues.
• Debug and optimize PySpark applications for performance and reliability.
• Write and optimize complex Oracle SQL queries.
• Work with structured, semi-structured, and unstructured datasets.
• Implement data quality, validation, and monitoring processes.
• Collaborate with cross-functional teams to resolve dependencies and ensure timely project delivery.
• Follow software engineering best practices, coding standards, testing methodologies, and CI/CD processes.

Required Skills & Experience

Data Engineering & Big Data

• PySpark
• Apache Spark
• Hadoop Ecosystem
• MapReduce
• Hive
• ETL Development
• Data Pipeline Development
• Data Mart Development
• Data Warehousing

Programming

• Python
• Pandas
• Jupyter Notebook

Database Technologies

• Oracle SQL
• SQL
• NoSQL Databases
• Query Optimization
• Data Analysis

Workflow & Automation

• Apache Airflow
• Oozie
• Jenkins
• Git
• CI/CD Pipelines

Data Engineering Techniques

• Data Cleansing
• Data Linking
• Feature Engineering
• Data Imputation Techniques
• Data Validation and Quality Checks

Preferred Experience

• Banking or Financial Services domain experience.
• Experience supporting enterprise-scale data platforms.
• Exposure to production support and deployment activities.
• Experience working in Agile environments

Key Skills

Education

Any Gradute