Description

You will design and build scalable data pipelines, AI models, and generative AI applications using the Databricks Lakehouse platform on Azure, AWS, or GCP.

This role is on-site.

Responsibilities

• Design and develop scalable data pipelines using Databricks, PySpark, Delta Lake, and Spark SQL, implementing Medallion Architecture.
• Build ELT/ETL solutions for structured and unstructured data, managing ingestion from cloud and on-premise sources.
• Develop and deploy Machine Learning models using Databricks ML, implementing MLOps practices for training, deployment, and monitoring.
• Create Generative AI applications using LLMs, RAG, Vector Databases, and frameworks like LangChain and LangGraph.
• Implement secure, scalable Lakehouse architectures using Unity Catalog and Delta Sharing on Azure, AWS, or GCP.

Required Skills

• 6+ years of experience in Data Engineering.
• 3+ years of experience with Databricks Lakehouse Platform.
• Strong proficiency in Python, PySpark, and Spark SQL.
• Hands-on experience with Delta Lake and Databricks Workflows.
• Experience delivering AI/ML or Generative AI solutions in production environments.
• Knowledge of Azure Data Factory or AWS Glue, SQL, and Data Modeling.
• Familiarity with Git, CI/CD, and DevOps practices.
• Understanding of Machine Learning, Deep Learning, and Vector Databases (FAISS, Pinecone, ChromaDB).

Preferred Skills

• Databricks Certified Data Engineer or Machine Learning certification.
• Experience with Azure OpenAI Service and Agentic AI.
• Domain experience in Healthcare, Life Sciences, Banking, Retail, or Manufacturing.

Key Skills

Education

Bachelor's or Master's degrees