Description

Key Responsibilities Data Engineering · Design and develop scalable data pipelines using Databricks, PySpark, Delta Lake, and Spark SQL. · Build ELT/ETL solutions for structured and unstructured data. · Implement Medallion Architecture (Bronze, Silver, Gold layers). · Optimize Databricks jobs, clusters, and workflows for performance and cost efficiency. · Manage data ingestion from multiple cloud and on-premise sources. AI & Machine Learning · Develop and deploy Machine Learning models using Databricks ML and Python. · Implement MLOps practices for model training, deployment, monitoring, and governance. · Build predictive analytics solutions and recommendation engines. · Create feature stores and reusable ML pipelines. Generative AI · Design and develop GenAI applications using: o Large Language Models (LLMs) o Retrieval Augmented Generation (RAG) o Vector Databases o AI Agents and Multi-Agent Frameworks

Key Skills

Education

Bachelor's degree