Description

Lead the design and development of scalable data pipelines for AI and analytics use cases.

This role is hybrid.

Responsibilities

• Build and maintain ETL/ELT processes for structured and unstructured data.
• Design and implement Generative AI solutions using LLMs (OpenAI, Azure OpenAI) and optimize RAG architectures.
• Collaborate with AI/ML teams to integrate models into production systems and work with stakeholders to translate business requirements.
• Implement data governance, security, and compliance best practices while mentoring junior data engineers.
• Collaborate with DevOps teams for CI/CD and MLOps implementation.

Required Skills

• 5+ years of experience in Data Engineering and Big Data technologies.
• Proficiency in Python and SQL.
• Experience with big data tools such as Spark, Hadoop, and Kafka.
• Strong understanding of data modeling, data warehousing, and ETL/ELT frameworks.
• Hands-on experience with cloud platforms such as AWS, Azure, or GCP.
• Experience with vector databases such as Pinecone, FAISS, or Weaviate.
• Strong knowledge of MLOps and data pipeline orchestration tools.

Preferred Skills

• Experience with Azure OpenAI, AWS Bedrock, or Google Vertex AI.
• Knowledge of LangChain, LlamaIndex, or similar frameworks.
• Hands-on experience with Docker, Kubernetes, and Terraform.

Key Skills

Education

Bachelor's degree