Description

You will design, train, and deploy Vision-Language Models (VLMs) for multimodal applications and autonomous use cases.

Responsibilities

• Design and deploy efficient VLMs for multimodal applications, including video-based implementations for autonomous robotics or navigation.
• Optimize training and inference using knowledge distillation, modal-adaptive pruning, and LoRA fine-tuning.
• Build scalable training and testing pipelines on AWS SageMaker or Azure ML.
• Develop multimodal solutions for image-text matching, visual question answering (VQA), and document data extraction.
• Apply VLMs to healthcare use cases such as medical imaging analysis, position detection, and motion measurement.
• Benchmark models like GPT-4V and Claude 3.5 for accuracy, speed, and cost-effectiveness.

Required Skills

• 10+ years of experience in machine learning or data science with a focus on vision-language models.
• Proven expertise in deploying production-grade multimodal AI solutions.
• Proficiency in Python and ML frameworks including PyTorch and TensorFlow.
• Hands-on experience with VLMs such as VILA, Isaac Sim, or VSS.
• Experience with cloud platforms including AWS SageMaker or Azure ML Studio.
• Experience implementing video-based VLMs in autonomous use cases.
• Knowledge of medical datasets and healthcare regulations.
• Ability to optimize models for real-world constraints using lightweight architectures.

Preferred Skills

• Experience with Docker and MLflow.
• Expertise in LoRA fine-tuning and modal-adaptive pruning.
• Familiarity with interleaved image-text datasets and cross-attention layers.

Key Skills

Education

Any Gradute