Description

Build and optimize scalable machine learning pipelines for distributed training and inference.

Responsibilities

• Design and implement solutions to optimize distributed training, hyperparameter optimization, and inference latency.
• Research and implement state-of-the-art LLM models, including fine-tuning and serving.
• Build deep learning models with optimal parallelism on CPUs and GPUs.
• Ensure ML model performance, uptime, and scale while maintaining high code quality and monitoring.
• Optimize integration between machine learning libraries and cloud data processing frameworks.

Required Skills

• 3+ years of industry experience with Python in a programming-intensive role.
• 3+ years of experience with distributed computing frameworks such as Spark or the Kubernetes ecosystem.
• 3+ years of experience with ML frameworks including PyTorch, TensorFlow, Keras, HuggingFace Transformers, or Spark MLlib.
• 3+ years of experience with major cloud computing services.
• 2+ years of experience in machine learning topics such as deep learning, classification, clustering, optimization, or recommendation systems.
• Experience building and scaling Generative AI applications using Langchain, PGVector, or Pinecone.
• MS or PhD in Computer Science, Software Engineering, Electrical Engineering, or a related field.

Preferred Skills

• Proficiency in Python, PySpark, and Azure ML for model deployment.
• Experience designing scalable services and controller architecture using FastAPI.
• Experience with containerization services and CI/CD frameworks.

Key Skills

Education

Bachelor's degree