Description

You will build and maintain a scoring engine for merchant data.

Responsibilities

• Process large volumes of structured and unstructured data from various sources.
• Triage and resolve performance and scaling issues when handling data at scale.
• Optimize data readability using appropriate file formats, partitioning, and bucketing strategies.
• Develop and maintain testable data jobs using Spark or equivalent frameworks.

Required Skills

• 3+ years of experience in data engineering.
• Extensive experience writing testable jobs using Spark.
• Proficiency in Scala and Java EE.
• Strong SQL and Bash scripting skills.
• Experience with micro-services and event-driven architectures.
• Knowledge of asynchronous Pub-Sub and Point-to-Point messaging systems.
• Understanding of application security principles.
• Experience managing data at scale across diverse sources.

Preferred Skills

• Experience with streaming within the Hadoop ecosystem.

Key Skills

Education

Any Graduate