Description

You will manage data lineage, quality, and dataset onboarding within a distributed computing environment.

Responsibilities

• Document data lineage using existing templates.
• Translate data science requirements into functional data quality rules.
• Onboard new incremental datasets and configure associated quality rules.
• Execute data validation checks and report on quality issues.
• Manage the transfer of production data into test environments.

Required Skills

• 6+ years of experience in data engineering.
• Cloudera (Hadoop) ecosystem.
• Apache Spark with Scala or Java.
• Strong SQL proficiency.
• Experience with Hive.
• Experience with Aerospike.
• Strong analytical skills.
• Bachelor's degree or equivalent graduate qualification.

Key Skills

Education

Any Graduate