Description

You will optimize PySpark code for performance and infrastructure cost reduction while deploying jobs on Kubernetes.

Responsibilities

• Optimize PySpark code to improve performance and reduce infrastructure costs.
• Deploy and run PySpark jobs within Kubernetes container services.
• Prepare realistic test data using Automation Testing Frameworks and BDD concepts.
• Collaborate with stakeholders to ensure accurate test data alignment.

Required Skills

• 5+ years of experience in data analytics and code optimization.
• Strong proficiency in PySpark for performance tuning.
• Experience deploying PySpark jobs on Kubernetes.
• Familiarity with Automation Testing Frameworks.
• Knowledge of Behavior-Driven Development (BDD) concepts.
• Ability to collaborate effectively with stakeholders for test data preparation.

Key Skills

Education

Any Gradutate