Description

You will build data processing solutions on Azure Cloud using PySpark.

Responsibilities

• Develop, maintain, and optimize data processing pipelines using PySpark on Azure Cloud.
• Collaborate with cross-functional teams to translate business needs into technical specifications.
• Design and implement scalable data models and data architecture for analytical needs.
• Perform data cleansing, transformation, and enrichment processes to enhance data quality.
• Monitor and troubleshoot data processing jobs, ensuring high availability and performance.

Required Skills

• 10+ years of experience in PySpark development.
• Proven experience with Azure Data Factory and Azure Databricks.
• Proficiency in PySpark for ETL tasks.
• Experience with SQL and NoSQL databases.
• Knowledge of Azure services for big data and analytics.
• Prior experience in banking, investment banking, or financial services.

Preferred Skills

• Experience with Docker and Kubernetes.

Key Skills

Education

Any Graduate