Description

You will develop and maintain data pipelines and infrastructure using Python, PySpark, and Terraform.

This role is hybrid.

Responsibilities

• Develop and optimize data processing workflows using PySpark and Pandas/Polars.
• Implement infrastructure as code using Terraform to manage cloud resources.
• Build and manage data workflows within Databricks environments.
• Write and execute tests using appropriate testing frameworks to ensure code quality.
• Collaborate in an Agile methodology to deliver features on schedule.

Required Skills

• 5+ years of professional experience in software development.
• Expert proficiency in Python.
• Hands-on experience with PySpark for distributed data processing.
• Experience with Pandas or Polars for data manipulation.
• Familiarity with Databricks platforms and tools.
• Working knowledge of AWS services.
• Proficiency with Terraform for infrastructure management.
• Experience implementing and maintaining testing frameworks.
• Experience operating within an Agile methodology.

Key Skills

Education

Any Graduate