Description

You will build and maintain data pipelines using Spark and Python.

Responsibilities

• Develop and optimize data processing tasks using Apache Spark.
• Write efficient code using PySpark or SparkSQL.
• Manage data workflows within cloud environments.
• Utilize Jupyter Notebooks and Apache Superset for data analysis and visualization.

Required Skills

• 8+ years of professional experience.
• Proficiency in Python and PySpark.
• Hands-on experience with Apache Spark.
• Experience with SparkSQL.
• Working knowledge of Jupyter Notebooks.
• Experience with Apache Superset or equivalent analytic tools.
• Cloud experience with AWS and Azure or GCP.
• Bachelor's degree.

Preferred Skills

• Experience with Scala.
• Knowledge of Bash or Shell scripting.

Key Skills

Education

Bachelor's degree