Description

You will build and support data flows for a machine learning platform on GCP, working closely with data science teams to meet ML-specific data requirements.

Responsibilities

• Design, build, and maintain scalable data pipelines for ingestion, transformation, and integration using Kafka and Databricks.
• Develop feature engineering pipelines for machine learning workflows using Vertex AI, BigQuery ML, and Python.
• Implement automated unit, integration, and performance testing frameworks to ensure data quality and compliance.
• Optimize data workflows for performance, cost efficiency, and scalability across large datasets.
• Provide technical leadership, mentor team members, and draft architectural documentation and interface specifications.

Required Skills

• 4+ years of professional data development experience.
• 4+ years of experience with SQL and NoSQL technologies.
• 3+ years of experience building and maintaining data pipelines and workflows.
• Proficiency in Python and Spark/PySpark development.
• Experience with GCP services including Vertex AI Platform, Cloud Storage, AutoMLOps, and Dataflow.
• Hands-on experience with Kafka, Eventhub, and CI/CD pipelines.
• Experience with version control using Git.
• Strong understanding of Agile/Scrum principles.
• Ability to perform cost/benefit analysis for architectural decisions.

Key Skills

Education

Any Gradute