Software Engineer in Test (Agentic QA, Reliability)

This Software Engineer in Test (Agentic QA, Reliability) role exists to replace manual QA with autonomous AI agents that continuously test, validate, and ensure reliability across complex enterprise ERP workflows at scale. Our client is rebuilding industry‑specific ERP software from the ground up using AI‑first, agentic systems, not bolting AI onto legacy platforms. As a Software Engineer in Test (Agentic QA, Reliability) on a small Modernization Strike Team, you’ll design AI‑driven QA automation and reliability systems that explore, test, and stress‑test enterprise software automatically. This is not traditional QA. You won’t write manual test cases or maintain brittle scripts; you’ll build self‑testing platforms powered by autonomous AI agents. What You’ll Do Build autonomous QA agents that test ERP workflows through UI, APIs, and data validation Design AI-driven verification systems to ensure financial and business logic accuracy Develop self-healing test suites that adapt to UI and workflow changes Create synthetic data pipelines to stress-test agent workflows and retrieval systems Build evaluation and “judge” agents that detect hallucinations, logic errors, and regressions Use AI to reverse-engineer legacy business logic and wrap it in automated tests What We’re Looking For Strong experience with Python, TypeScript, .NET, or SQL Background in QA automation, SDET, or reliability engineering Experience with Playwright, API testing, or automated integration testing Familiarity with LLMs, agent frameworks, or AI-driven automation Comfort working with complex legacy systems and business logic A mindset focused on automation, reliability, and eliminating manual work Tech Stack Python, TypeScript, .NET, SQL, Playwright, LangChain/LangGraph, DSPy, OpenAI/Anthropic, Docker, GitHub Actions, AgentOps, OpenTelemetry, Weights & Biases. Why Join Work on one of the hardest problems in software: automating QA for complex ERP systems Build AI-powered testing agents instead of traditional test scripts Small, high-impact team with high ownership and minimal bureaucracy Access to frontier AI models and modern agent frameworks Opportunity to build a fully autonomous QA system If you want to help build the autonomous QA systems that power next-generation enterprise software, we’d love to talk. Show more Show less