Description

You will build and maintain scalable data processing solutions on Azure Cloud using PySpark.

This role is on-site.

Responsibilities

• Develop and optimize data processing pipelines using PySpark on Azure Cloud.
• Design and implement scalable data models and architecture for analytical needs.
• Cleanse, transform, and enrich data to ensure high quality and availability.
• Monitor and troubleshoot data processing jobs to ensure performance and reliability.
• Collaborate with cross-functional teams to translate business requirements into technical specifications.

Required Skills

• 10+ years of experience in PySpark development.
• Proven experience with Azure Data Factory and Azure Databricks.
• Proficiency in PySpark for ETL tasks.
• Experience with SQL and NoSQL databases.
• Knowledge of Azure services for big data and analytics.
• Prior experience in banking, investment banking, or financial services.

Preferred Skills

• Experience with Docker and Kubernetes.

Key Skills

Education

Any Graduate