Description

You will build and maintain the ML data infrastructure platform, enabling efficient data preparation, feature engineering, and dataset management.

This role is remote.

Responsibilities

• Design and implement scalable data processing pipelines for ML training and validation.
• Build and maintain feature stores supporting both batch and real-time features.
• Develop data quality monitoring, validation, and testing frameworks.
• Create systems for dataset versioning, lineage tracking, and reproducibility.
• Design efficient data storage and access patterns for ML workloads.

Required Skills

• 7+ years of software engineering experience, with 3+ years in data infrastructure.
• Expertise in GCP data and ML infrastructure: BigQuery, Dataflow, Cloud Storage, Vertex AI Feature Store, Cloud Composer, and Dataproc.
• Deep expertise in data processing frameworks: Spark, Beam, or Flink.
• Proficiency in Python and SQL.
• Experience with feature stores (Feast, Tecton) and data versioning tools.
• Knowledge of pipeline orchestration using Airflow or Dagster.
• Experience with data quality and testing frameworks.

Preferred Skills

• Experience with streaming systems such as Kafka, Kinesis, or Pub/Sub.
• Familiarity with dbt or similar data transformation tools.
• Knowledge of GCP security, IAM, Cloud Logging, and Cloud Monitoring.

Key Skills

Education

Any Gradute