Description

Key Responsibilities

Data Engineering
· Design and develop scalable data pipelines using Databricks, PySpark, Delta Lake, and Spark SQL.
· Build ELT/ETL solutions for structured and unstructured data.
· Implement Medallion Architecture (Bronze, Silver, Gold layers).
· Optimize Databricks jobs, clusters, and workflows for performance and cost efficiency.
· Manage data ingestion from multiple cloud and on-premises sources.

AI & Machine Learning
· Develop and deploy Machine Learning models using Databricks ML and Python.
· Implement MLOps practices for model training, deployment, monitoring, and governance.
· Build predictive analytics solutions and recommendation engines.
· Create feature stores and reusable ML pipelines.

Generative AI
· Design and develop GenAI applications using:
o Large Language Models (LLMs)
o Retrieval-Augmented Generation (RAG)
o Vector Databases
o AI Agents and Multi-Agent Frameworks

Key Skills

Education

Bachelor's degree