Description

Key Responsibilities

Workflow Auditing & AI Opportunity Mapping

• Audit existing SDLC workflows—planning, development, code review, testing, and deployment—to identify high-ROI AI integration opportunities
• Produce prioritised findings and recommendations for the AI CoE and engineering leadership
• Build a repeatable AI audit framework applicable across engineering teams

Hands-on AI Utility Development

• Design and develop internal tools, scripts, and LLM-powered integrations using Python
• Containerize and deploy workloads using Docker to support long-running workflows and agentic orchestrations
• Maintain a shared internal library of reusable AI components

Agentic AI Development

• Build and deploy autonomous and multi-agent AI workflows using frameworks such as LangGraph, LangChain, AutoGen, or similar
• Apply spec-driven development by defining task specifications and validating AI outputs systematically
• Integrate agentic workflows into CI/CD pipelines (GitHub, Azure DevOps, GitLab)

LLMOps & AgentOps

• Manage the operational lifecycle of LLM integrations including prompt versioning, output monitoring, cost tracking, and model updates
• Ensure agent observability by tracing sessions, monitoring tool usage, and documenting runbooks
• Define and enforce AI quality checkpoints within CI/CD pipelines

AI Evaluations & Guardrails

• Design and implement guardrails for AI-generated outputs in collaboration with the CoE team
• Run AI evaluations to measure quality, reliability, and safety
• Act as the engineering authority on hallucination detection, edge cases, and AI quality standards

AI-First Engineering Practice

• Adopt AI coding assistants such as GitHub Copilot and Cursor as daily productivity tools
• Conduct lightweight enablement sessions to upskill engineering teams
• Demonstrate high-performance AI-augmented engineering practices

Required Qualifications

• 6–8 years of hands-on software development experience (mindset and learning velocity prioritized)
• Strong SDLC fundamentals with experience applying AI across the development lifecycle
• Proficiency in Python; familiarity with .Net or TypeScript is a plus
• Hands-on experience with LLM APIs (OpenAI, Azure OpenAI, Anthropic, or equivalent)
• Practical prompt engineering experience including structured outputs and prompt versioning
• Exposure to agentic AI frameworks such as LangGraph, LangChain, AutoGen, or CrewAI
• Familiarity with LLMOps and AgentOps practices
• Practical knowledge of AI evaluations and guardrail design
• DevOps awareness including CI/CD, Docker, and enterprise Git workflows
• Hands-on use of AI coding assistants like GitHub Copilot or Cursor

Good to Have

• Experience building internal AI-powered developer tools
• Exposure to AI observability platforms such as LangSmith, AgentOps, Arize, Helicone, or Braintrust
• Prior experience in AI CoE, platform engineering, or internal developer tooling
• Exposure to Azure AI Foundry, Azure AI Search, or Azure Document Intelligence

Key Skills

Education

Bachelor's degree