Description

You will lead data engineering projects for the Data Lake team, focusing on building scalable systems for massive data volumes.

Responsibilities

• Build and maintain event streaming applications and data pipelines to make data consumable for users and data scientists.
• Design and manage data schema life-cycles.
• Proactively improve process efficiency and recommend system improvements.
• Develop systems capable of handling large-scale data ingestion and processing.

Required Skills

• 5+ years of experience in data engineering.
• Proficiency in Python, Java, Scala, R, or SQL.
• Experience with big data tools including Hadoop, Spark, and Kafka.
• Hands-on knowledge of relational and NoSQL databases, specifically Postgres and Cassandra.
• Understanding of in-memory processing and data formats such as Avro, Parquet, and JSON.
• Experience with stream-processing systems like Storm, Spark-Structured-Streaming, or Kafka consumers.
• Knowledge of AWS services including EC2, MSK, S3, RDS, SNS, and SQS.
• Experience with workflow management tools such as Apache Airflow, Argo, or AWS Data Pipeline.
• Proficiency with software engineering tools including Github, VSCode, and CI/CD.

Key Skills

Education

Any Gradute