Description

Lead the design and maintenance of scalable data pipelines and machine learning workflows.

This role is remote.

Responsibilities

• Build and maintain scalable data pipelines for ingestion, transformation, and integration using Kafka and Databricks.
• Develop feature engineering pipelines for machine learning workflows using Vertex AI and BigQuery ML.
• Modernize data assets across SQL, NoSQL, cloud-based, and real-time streaming platforms.
• Implement automated unit, integration, and performance testing frameworks to ensure data quality.
• Conduct cost/benefit analyses and draft architectural documentation to guide scalable data solutions.

Required Skills

• 5+ years of experience in data engineering.
• Proficiency in Python and SQL.
• Hands-on experience with Google Cloud Platform (GCP).
• Experience with Vertex AI.
• Experience building pipelines with Kafka and Databricks.
• Knowledge of feature engineering techniques for ML workflows.
• Experience with both SQL and NoSQL data platforms.
• Ability to design real-time streaming data platforms.

Preferred Skills

• Experience with BigQuery ML.

Key Skills

Education

Any Gradute