Description

You will develop and manage big data processing pipelines using Apache Spark and Scala.

Responsibilities

• Develop and maintain data processing pipelines using Apache Spark and Scala.
• Implement data transformations and ETL processes across big data platforms.
• Apply distributed computing principles to handle large datasets.
• Work with data serialization formats such as Parquet, Avro, or ORC.

Required Skills

• 4+ years of experience in Information Technology.
• Bachelor's degree or foreign equivalent from an accredited institution.
• 2+ years of experience with Apache Spark, Scala, Spark SQL, and Starburst.
• Strong understanding of distributed computing principles and big data technologies.
• Experience with ETL processes and Data Warehouses.
• Knowledge of data serialization formats (Parquet, Avro, ORC).
• Familiarity with data processing and transformation techniques.

Preferred Skills

• Experience with data lakes.
• Understanding of Agile software development frameworks.

Key Skills

Education

Bachelor's degree