Description

You will design and maintain scalable data pipelines and machine learning workflows within a GCP environment.

Responsibilities

• Build and maintain scalable data pipelines for ingestion, transformation, and integration using Kafka and Databricks.
• Develop feature engineering pipelines for machine learning workflows using Vertex AI and BigQuery ML.
• Implement automated unit, integration, and performance testing frameworks to ensure data quality.
• Optimize data workflows for performance, cost efficiency, and scalability across large datasets.
• Provide technical leadership and mentor team members on data principles, patterns, and practices.
• Collaborate with data science teams to support ML platform flows and translate ML concepts into data requirements.

Required Skills

• 4+ years of professional data development experience.
• 4+ years of experience with SQL and NoSQL technologies.
• 3+ years of experience building and maintaining data pipelines and workflows.
• Proficiency with Python and Spark/PySpark development.
• Hands-on experience with GCP services including Vertex AI, Cloud Storage, AutoMLOps, and Dataflow.
• Experience with Kafka and Databricks.
• Experience implementing CI/CD pipelines and processes.
• Experience with Git version control.
• Experience with automated testing (unit, integration, and performance).
• Strong understanding of Agile/Scrum principles.

Preferred Skills

• Experience with BigQuery ML.

Key Skills

Education

Any Graduate