Description

You will design and optimize real-time data processing pipelines for high-volume streams in auto-scalable environments.

Responsibilities

• Build and fine-tune complex stream processing systems for high-throughput data.
• Design and manage data storage solutions across SQL and NoSQL databases.
• Develop scalable services to handle large-scale data streams.
• Implement and maintain data processing workflows in cloud environments.

Required Skills

• 3+ years of experience in data engineering.
• Proficiency in Java, Python, and Go.
• Hands-on experience with Spark, Spark Streaming, or Kafka Streams.
• Experience with Kafka and Redis for streaming.
• Working knowledge of Hadoop and Big Data ecosystems.
• Experience with PostgreSQL, Elasticsearch, MongoDB, or Cassandra.
• Proficiency in Linux shell scripting and Git.
• Familiarity with cloud data technologies in Azure or AWS.
• Bachelor's degree in Computer Science or related field.

Preferred Skills

• Experience with ClickHouse.
• Knowledge of data lakes and warehouses such as Snowflake, Amazon Redshift, or S3.
• Experience with Docker, Kubernetes, Airflow, Luigi, or Azkaban.

Key Skills

Education

Bachelor's degree in Computer Science