Description

You will build and maintain backend systems focusing on data engineering tasks.

Responsibilities

  • Design, develop, and maintain data pipelines using Spark, Python, Scala, and Java.
  • Write optimized SQL queries for ETL processes.
  • Manipulate and analyze large datasets using DataFrame operations.
  • Implement data storage and processing solutions leveraging AWS or GCP.
  • Build and maintain real-time data streaming pipelines with MSK/Kafka.
  • Ensure data quality, integrity, and security across pipelines.
  • Troubleshoot and resolve issues in data pipelines and back-end systems.

Required Skills

  • 5+ years of experience in back-end development with a focus on data engineering.
  • Strong proficiency in Spark, Python, Scala, and Java.
  • Expertise in SQL and working with relational databases.
  • Experience with cloud technologies, specifically AWS or GCP.
  • Experience with message streaming platforms like MSK/Kafka.
  • Experience with S3 or similar object storage.
  • Experience with data lake technologies like Iceberg.
  • Solid understanding of data warehousing concepts.
  • Excellent communication and collaboration skills.

Education

Bachelor's degree