Description

You will build and maintain data pipelines using big data technologies and cloud services.

Responsibilities

• Develop data processing workflows using Spark, Flink, and Kafka.
• Implement data modeling techniques to structure large-scale datasets.
• Manage data storage and processing within the AWS ecosystem, including Glue and EMR.
• Apply DataOps practices to streamline data engineering lifecycles.
• Utilize Hadoop ecosystem tools like Hive and HDFS to manage big data workloads.

Required Skills

• 4+ years of experience in data engineering.
• Proficiency in Python, PySpark, or Scala.
• Hands-on experience with Snowflake.
• Experience with AWS data services, specifically Glue and EMR.
• Working knowledge of Hadoop, Spark, Hive, Kafka, and Flink.
• Strong understanding of HDFS and the 5Vs of big data.
• Practical application of Data Modeling techniques.
• Degree in any graduate field.

Key Skills

Education

Any Graduate