Description

You will build and maintain scalable data processing applications.

Responsibilities

• Design, develop, and maintain highly scalable data processing applications.
• Deliver big data projects using Spark, Scala, Python, and SQL.
• Maintain and tune existing Spark applications, identifying optimization opportunities.
• Translate complex technical and functional requirements into detailed designs.
• Serve as a mentor, conducting technical training and reviewing junior staff work.

Required Skills

• 8+ years of software development experience with Hadoop framework components (HDFS, Spark, Scala, PySpark).
• Strong data engineering skills with at least 6+ years using Python/Scala, Spark, and SQL.
• Excellent understanding of Spark architecture, DataFrames, and tuning techniques.
• Strong knowledge of database concepts, systems architecture, and data structures.
• Proficiency in Spark, Scala, Python, SQL, Core Java, Linux, and HDFS.
• Experience in writing Python standalone applications interacting with the PySpark API.
• Ability to present and defend solutions using technical facts and business proficiency.

Preferred Skills

• Knowledge of DELTA.IO package.
• Knowledge of shell scripting.

Key Skills

Education

Any Graduate