Description

You will build and maintain the data foundations that power machine learning capabilities.

Responsibilities

• Design and implement scalable data pipelines for ML training and validation.
• Build and manage feature stores supporting batch and real-time use cases.
• Develop frameworks for data quality monitoring, validation, and testing.
• Create tools for dataset versioning, lineage tracking, and reproducibility.
• Optimize data storage and access patterns for machine learning workflows.

Required Skills

• 7+ years of software engineering experience, with at least 3 years in data infrastructure.
• Strong experience with Google Cloud Platform (GCP) services: BigQuery, Dataflow, Cloud Storage, Vertex AI Feature Store, Cloud Composer (Airflow), and Dataproc.
• Expertise in data processing frameworks including Spark, Beam, or Flink.
• Experience with feature stores such as Feast or Tecton.
• Proficiency in Python and SQL.
• Knowledge of data versioning tools and systems.
• Experience with data quality frameworks and testing.
• Familiarity with orchestration tools like Airflow or Dagster.
• Bachelor's or Master's degree in Computer Science, Data Engineering, or a related field.

Preferred Skills

• Experience with streaming systems such as Kafka, Kinesis, or Pub/Sub.
• Familiarity with GCP-specific IAM, security best practices, Cloud Logging, and Monitoring.
• Understanding of CI/CD tools like Cloud Build and Cloud Deploy.

Key Skills

Education

Bachelor's or Master's degrees