Description

Data Engineer responsible for building and maintaining big data infrastructure and ETL pipelines in Minneapolis, MN.

Responsibilities

• Develop and integrate software applications using standard architectural patterns and development methodologies.
• Build libraries, user-defined functions, and frameworks around Hadoop to extend core capabilities.
• Perform end-to-end automation of ETL processes for datasets ingested into the big data platform.
• Develop user-defined functions to provide custom capabilities for Hive, HDFS, Kafka, and Spark.
• Collaborate with Business Analysts, Architects, and Senior Developers to establish physical application frameworks.

Required Skills

• 5+ years of experience with Core Java.
• Strong proficiency in Spark or Scala.
• Expert-level SQL skills.
• Deep understanding of Hadoop internals.
• Experience with Hadoop ecosystem tools: Hive, Impala, HBase.
• Proficiency in Unix/Linux Shell Scripting.
• Proficiency in Python.
• Bachelor's degree in Computer Science or related field.

Preferred Skills

• Knowledge of cloud platforms.
• Ability to develop and execute test scripts independently.

Key Skills

Education

Bachelor's degree