Description

You will own the AIOps machine learning lifecycle, from building petabyte-scale Lakehouse data pipelines to deploying real-time anomaly detection models. You will design schema contracts, manage feature stores, and integrate ML insights into incident management platforms to reduce alert noise and quantify business impact.

This role is on-site.

Responsibilities

• Design and evolve Delta Lake/Iceberg schemas for multi-domain observability data at petabyte scale.
• Build ingestion pipelines from OTel Collector through Kafka to the Lakehouse with exactly-once semantics.
• Implement dbt transformation models to generate mart-ready fact and dimension tables.
• Train and deploy streaming ML models for multivariate anomaly detection, root-cause analysis, and incident forecasting.
• Build low-latency inference pipelines using Flink or Spark Streaming for real-time anomaly scoring.

Required Skills

• 5+ years of experience in machine learning engineering or data engineering.
• Strong proficiency in Python and SQL.
• Experience with streaming data processing using Kafka and Spark.
• Knowledge of MLOps practices, including model serving, drift detection, and feature store management.
• Familiarity with Delta Lake or Apache Iceberg for data lakehouse architectures.
• Experience with dbt for data transformation and modeling.
• Understanding of time-series data patterns and query optimization techniques.

Preferred Skills

• Experience with log intelligence models (e.g., LogBERT, DRAIN3) and NLP classification.
• Knowledge of incident management platforms like PagerDuty or Opsgenie.
• Experience with behavioral analytics and unsupervised learning methods.

Key Skills

Education

Bachelor's degree