Description

Key Responsibilities · Design and develop data engineering and machine learning solutions using Databricks. · Build and optimize scalable ETL/ELT pipelines using PySpark and Delta Lake. · Develop, train, evaluate, and deploy machine learning models on Databricks. · Implement MLOps practices using MLflow, Databricks Model Registry, and CI/CD pipelines. · Build Generative AI solutions using LLMs, RAG architectures, vector databases, and prompt engineering. · Integrate AI/ML solutions with cloud platforms such as AWS, Azure, or GCP. · Collaborate with business stakeholders, data scientists, and architects to translate business requirements into technical solutions. · Monitor model performance, data quality, and pipeline reliability in production environments. · Ensure security, governance, and compliance standards are followed across data and AI platforms. Required Skills · Strong experience with Databricks Lakehouse Platform · Expertise in PySpark, Python, SQL · Hands-on experience with Delta Lake, Unity Catalog · Experience building ML models using Scikit-Learn, TensorFlow, PyTorch, XGBoost · Strong knowledge of MLflow, MLOps, CI/CD · Experience with Generative AI, LLMs, RAG, Vector Databases · Knowledge of Azure Databricks, AWS Databricks, or GCP Databricks · Experience with Apache Spark, Data Engineering, Data Pipelines · Familiarity with REST APIs and microservices deployment Preferred Skills · Experience with OpenAI, Azure OpenAI, LangChain, LlamaIndex · Knowledge of MCP (Model Context Protocol) and Agentic AI frameworks · Experience with Snowflake, Kafka, Airflow · Healthcare or insurance domain experience · Databricks Certified Data Engineer or Machine Learning Professional certification

Key Skills

Education

Any Gradute