Description

You will build and optimize data processing applications using the Spark and Hadoop ecosystems.

Responsibilities

• Develop programs using Spark Java APIs over Hadoop Yarn for data analytics.
• Design Spark models for applications in collaboration with Technology Architects.
• Implement Spark Core, Spark Streaming, and Spark SQL for data processing and testing.
• Build Spark jobs to parse XML and JSON data formats.
• Integrate HDFS applications with legacy systems and validate datasets against business requirements.

Required Skills

• 5+ years of professional experience in software engineering.
• Proficiency with Spark Java APIs, Spark SQL, and Spark Core.
• Experience with Hadoop Yarn and HDFS.
• Strong background in Clojure and functional programming.
• Experience working with Apache Parquet and various flat file formats.
• Ability to design solutions using Spark Datasets and RDDs for in-memory computations.
• Experience parsing XML and JSON data.
• Ability to analyze and optimize SQL queries and scripts.
• Basic understanding of Python and Scala scripting.

Preferred Skills

• Any graduate degree.

Key Skills

Education

Any gradudate