Description

You will own the design and implementation of data monitoring pipelines to identify and resolve quality issues affecting downstream products.

Responsibilities

• Design and implement monitoring pipelines to proactively resolve data quality issues.
• Collaborate with stakeholders to define requirements and develop metrics for pipeline quality.
• Negotiate data quality SLAs on behalf of downstream data product owners.
• Build monitoring solutions using Python, Spark, and Airflow.
• Develop new methodologies to improve access to trustworthy data.

Required Skills

• 8+ years of experience in Data Engineering or similar roles working with big data pipelines.
• 5+ years of hands-on experience with SQL in scalable data warehouses like BigQuery or Snowflake.
• 8+ years of coding experience in Python, Java, or equivalent languages.
• Expertise with Apache Airflow.
• Proficiency in cloud technologies, specifically GCP and/or AWS.
• Experience with CI/CD tools such as Jenkins.
• Deep understanding of distributed systems and effective data management.

Preferred Skills

• Bachelor’s Degree in Computer Science, Engineering, or a related STEM field.

Key Skills

Education

ANY GRADUATE