Description

· Build and operate agentic applications that interpret user intent, manage conversation context, call tools and APIs, and carry out multi-step analysis with appropriate user review.

· Develop the Angular user experience, .NET APIs, and Python agent services, integrating with existing reporting systems, authentication, and data access controls.

· Own the evaluation pipeline: curate representative datasets with domain experts, define scoring rubrics and acceptance criteria, and combine automated checks with human review.

· Evaluate answer correctness, supporting evidence, tool selection and execution, and end-to-end task success, including ambiguous requests, unsupported questions, and failure scenarios.

· Integrate evaluations and regression tests into CI/CD. Version prompts, evaluation datasets, and configuration; compare changes against baselines and enforce quality gates before release.

· Monitor production quality, latency, cost, and failures through traces and user feedback. Turn observed failures into reproducible test cases and measurable improvements.

· Implement reliable execution with timeouts, retries, cancellation, durable state, and recovery, while protecting sensitive data and enforcing authorization at tool and data boundaries.

· Work with product owners and domain experts to validate analytical findings, preserve source traceability, and communicate uncertainty and application limitations clearly

Mandatory skills

· Professional software development experience, including delivery and support of secure, scalable enterprise applications.

· Hands-on experience building agentic applications using large language models, tool calling, structured outputs, prompt and context design, and multi-step workflows.

· Practical experience building evaluation and quality pipelines for AI applications, including test datasets, scoring criteria, regression analysis, and human review. Ability to explain how evaluation results informed a release or quality improvement.

· Strong full stack development skills with Angular, .NET Web APIs, and Python, including integration with existing enterprise services.

· Strong SQL, relational data modelling, and data analysis skills, with attention to numerical accuracy, missing data, and traceability of results.

· Experience with cloud-native application development on Azure, CI/CD, automated unit and integration testing, and end-to-end testing.

· Experience with authentication, authorization, secure API design, observability, and reliable asynchronous processing.

· Strong problem-solving and communication skills, with the ability to work independently and collaborate with product, engineering, and domain experts.

Preferred Qualifications: -

· Experience with retrieval-augmented generation (RAG), including document ingestion, retrieval, grounding, and retrieval quality evaluation.

· Experience with healthcare analytics, enterprise reporting, or other applications that require careful handling of sensitive data and verifiable analytical results.

· Experience taking AI prototypes into production and improving quality, latency, and operating cost using evaluation and production evidence. Proven experience architecting and delivering systems using agentic IDEs Ability to:

· Define architectural intent that agents can follow • Break features into agent executable tasks

· Govern AI autonomy (guardrails, permissions, reviews) • Integrate agentic workflows into CI/CD pipelines Experience supervising AI agents across: • Multi service systems • Legacy modernization

· Large codebases / monorepos Strong understanding of: • Security implications of autonomous code execution • Compliance, auditability, and traceability • AI assisted SDLC operating models

Core Responsibility:

· Guide effective use of agentic IDEs for complex, multi-module or cross-service changes

· Establish review practices and quality checks for AI-generated code

· Mentor team members on balancing autonomy, correctness, and maintainability in AI-assisted development

· Design system architectures that support AI-augmented and agentic development workflows

· Define guardrails, standards, and governance for the use of autonomous coding agents

· Evaluate impact of agentic IDEs on SDLC, CI/CD pipelines, security posture, and technical debt

Key Skills

Education

Bachelor's degree