Description

Key Responsibilities

• Design, develop, and maintain scalable ETL/ELT data pipelines using Databricks, PySpark, and Python.
• Build and optimize large-scale data processing solutions on the AWS Cloud Platform.
• Develop and manage data ingestion frameworks for structured and unstructured data.
• Implement data transformation, cleansing, validation, and aggregation processes.
• Work closely with Data Architects, Business Analysts, and Banking stakeholders to understand data requirements.
• Optimize Spark jobs and Databricks workloads for performance, scalability, and cost efficiency.
• Develop solutions using Delta Lake, Databricks Workflows, and Unity Catalog.
• Implement data quality checks, monitoring, and troubleshooting mechanisms.
• Support CI/CD deployment processes and automation of data engineering workflows.
• Ensure compliance with banking data governance, security, and regulatory standards.
• Participate in code reviews, performance tuning, and production support activities

Key Skills

Education

Bachelor's degree