Description

Responsibilities

• Design, build, and refine test strategies for agentic AI systems tasked with automating legacy application modernization
• Develop and execute test plans that validate AI agent performance in autonomous upgrade scenarios, with a target of 90% successful autonomous completion
• Work without comprehensive documentation by reverse-engineering application behavior, designing workarounds, and automating tests for undocumented systems
• Combine expertise in Java/Selenium migration to Playwright and desktop automation to test AI agents across legacy and modernized environments
• Validate end-to-end modernization workflows, including infrastructure upgrades (e.g., JDK version migrations) and security transitions (e.g., legacy authentication to Okta integration)
• Identify edge cases, failure modes, and downstream system impacts that AI agents must handle or flag for human review
• Collaborate with agentic engineers to translate test findings into agent skill refinements and prompt improvements
• Establish quality gates and validation frameworks for autonomous upgrade execution across enterprise-scale systems

Qualifications

• Required
  • Strong, hands-on experience with agentic AI systems, prompt engineering, and AI agent orchestration
  • Proven expertise transforming test automation from Java/Selenium to Playwright-based frameworks
  • Deep proficiency in desktop automation testing
  • Demonstrated ability to test and validate systems with minimal or no documentation, using reverse-engineering and exploratory techniques
  • Systems-level thinking: ability to map downstream impacts of infrastructure and security changes across interconnected enterprise applications
  • Experience navigating high-stakes production incidents and applying lessons learned to system design and testing strategy
• Preferred
  • Experience modernizing legacy Java applications or managing large-scale dependency upgrades
  • Familiarity with identity and access management transitions (e.g., LDAP, ISD to modern OAuth/Okta)
  • Background in infrastructure or platform engineering testing
  • Knowledge of Claude Code or similar AI-driven code generation and automation tools

Key Skills

Education

Bachelor's degree