Description

You will develop and maintain data pipelines using Python and PySpark.

This role is on-site.

Responsibilities

• Develop and maintain data pipelines using Python and PySpark.
• Perform data transformations using Spark SQL and DataFrames.
• Optimize Spark jobs for performance and handle large-scale datasets efficiently.
• Debug and support production data pipelines.

Required Skills

• 6+ years of professional experience.
• Strong experience in Python.
• Hands-on experience with PySpark / Apache Spark.
• Good knowledge of SQL.
• Understanding of ETL concepts.
• Experience with data formats like Parquet, JSON, CSV.
• Familiarity with Linux and Git.

Key Skills

Education

Any Graduate