> faq

An AI tester is a software tool that uses LLMs, vision models, or reinforcement-learning agents to generate, run, maintain, or fix software tests. The category is distinct from the job title 'AI test engineer' (the human role) and broader than self-healing test automation (which was the 2024 term for a subset). Read more at /category-overview.

Agentic testing is LLM-driven test design, autonomous execution, and self-repair without human supervision during a run. We define it on a five-level capability ladder: Level 0 is traditional scripts; Level 3 is what QA Wolf and Momentic ship today; Level 5 is still a research problem. Read the full definition at /llm-test-automation.

Self-healing means tests that automatically repair broken locators when the UI changes. The classic three-identifier model uses visual appearance, DOM locator, and AI text description for each element -- when one fails, the others recover it. Rainforest QA, Mabl, and Testim are the main self-healing tools. Read more at /self-healing-tests.

Mutation testing evaluates test suite quality by artificially seeding small code bugs (mutations) and checking whether the tests catch them. A mutation score of 90% means 9 out of 10 bugs are caught. Line coverage tells you which code was run; mutation score tells you whether tests can detect bugs. Read more at /glossary#mutation-testing.

A flaky test produces different results on identical code without any change to the test or source. Flake rate is measured by running the suite 100 times on stable code -- any test that fails at least once is flaky. Good E2E suites target below 2% flake rate. Read more at /glossary#flaky-test.

Playwright MCP is a Model Context Protocol server published by Microsoft that lets LLMs (Copilot, Claude) drive a real Chromium browser. The LLM navigates the app, observes DOM state, and generates Playwright test code from actual browser interactions. It is the most practical Level 2 agentic bridge available as of April 2026. Read more at /playwright-ai.

There is no single best tool. JVM shops: Diffblue Cover for unit tests. Playwright-first: QA Wolf or Copilot+MCP. QA-led orgs: testRigor. Visual regression: Meticulous. Startup velocity: Momentic. Enterprise with existing suite: Mabl. See the full matrix at / and the detailed comparisons at /tool-comparison.

Yes, if you were planning to hire 2-3 QA engineers. QA Wolf is $50-150k/year as a managed service. It delivers a growing Playwright test suite with 24/7 monitoring. The tests are real Playwright code you own. ROI calculation: compare to QA headcount cost, not to Mabl's subscription cost. They are different products.

For enterprise teams with 50+ existing tests and high maintenance burden: Mabl. For teams starting from zero, with QA-led authoring, or with limited budget: testRigor. testRigor has a free tier and transparent pricing. Mabl has better enterprise governance and more sophisticated self-healing. Read the direct comparison at /compare/testrigor-vs-mabl.

JVM shop (Java/Kotlin): Diffblue Cover -- 91% mutation score vs Copilot's 74% on our Java benchmark. Non-JVM or multi-language: Copilot or Qodo. Both have no lock-in (standard JUnit/pytest output). See the direct comparison at /compare/diffblue-vs-copilot.

Diffblue Cover's free IntelliJ plugin for JVM unit tests. Qodo's free developer tier for multi-language unit tests. testRigor's free plan for E2E testing. GitHub Copilot's free tier (limited) for authoring assistance. GitHub Copilot + Playwright MCP (free, requires Copilot subscription) for browser-observed E2E generation.

Not for new evaluations. Functionize has been overtaken by QA Wolf, Momentic, and Mabl in the agentic and auto-healing categories. Enterprise-only pricing and a slower innovation rate make it a poor choice for new adopters. The only justification is an existing enterprise contract with high switching costs.

Ranges from free (Diffblue IntelliJ, Qodo, testRigor free tier) to $150k+/year (QA Wolf managed service). Most enterprise self-healing tools (Mabl, Momentic, Meticulous) are custom-priced at $30-80k/year. See our normalised comparison at /pricing-comparison.

Mabl targets enterprise procurement, not SMB self-serve. Enterprise software buyers are not price-sensitive in the same way as self-serve buyers -- the deal is negotiated around contract terms, SLAs, and implementation support, not a published price. The opacity is frustrating during evaluation; budget 4-8 weeks for procurement.

Zero-budget: Diffblue Cover free IntelliJ plugin (JVM) + Qodo free tier (Python/JS) + GitHub Copilot + Playwright MCP ($10/user/mo) for E2E. Under $500/month for a 20-person team: testRigor ($200-400/mo) + Qase Community (free) for test management. These combinations cover unit generation, E2E generation, and test case management.

testRigor: parallelization fees scale with test volume. Diffblue: per-LoC fees grow with codebase size. QA Wolf: managed service markup on human QA layer. BrowserStack AI: requires BrowserStack Automate base subscription. Meticulous: SDK injection implementation cost. See our full hidden-cost flags at /pricing-comparison.

Three paradigms: RL-based (Diffblue) -- explores bytecode via reinforcement learning, optimises for mutation killing; LLM-based (Qodo, Copilot) -- generates test code from source via language model; Trace-based (Meticulous) -- captures real user interactions and replays them for visual comparison. Each produces different quality of tests for different purposes. Read more at /unit-test-generation.

LLM-based tools (Qodo, Copilot) work across Python, JavaScript, TypeScript, Java, Go, C#, and more. RL-based tools (Diffblue) are JVM-only. E2E tools (QA Wolf, Momentic, testRigor) work against any web app regardless of backend language -- they test the UI, not the source.

For Playwright tests (from QA Wolf, Copilot+MCP): standard Playwright GitHub Actions workflow. For Diffblue: Maven or Gradle build step with Cover plugin in GitHub Actions. For testRigor: testRigor provides a GitHub Actions integration and API-triggered run. For Mabl: Mabl provides a GitHub Actions integration. See our CI integration guide at /ci-integration.

Syntactically correct tests most of the time. Quality varies: simple pure functions are well-covered, complex stateful code and edge cases are weaker. Main failure mode: tests that always pass because assertions are too weak to catch mutations. Our benchmark found 74% mutation score on express-auth-api -- solid but not best-in-class. Always evaluate generated tests with mutation testing.

Playwright Healer is an experimental agent that repairs failing Playwright tests with locator errors automatically. When a test fails because a selector broke, Healer uses an LLM to inspect the current DOM and patch the test. It is distinct from Playwright MCP (which generates new tests). Read more at /playwright-ai.

No -- but the role changes. Capgemini's 2025 survey found 63% enterprise AI QA adoption. AI replaces repetitive regression execution and maintenance. AI cannot replace exploratory testing, usability judgment, business-logic intuition, or release-readiness calls. The QA role in 2026 is less test maintenance and more test strategy. Read more at /ai-qa.

Playwright or Selenium (E2E authoring), mutation testing literacy (understanding mutation score vs coverage), LLM prompt engineering for test generation, CI/CD pipeline basics (GitHub Actions at minimum), and test management platforms (Qase, Xray, Zephyr). The 2020 skill set of manual test case writing is being automated; the 2026 skill set is test strategy, AI supervision, and flake analysis.

Yes, and it is more relevant than ever. SDETs who can build and maintain agentic test infrastructure, configure mutation testing pipelines, evaluate AI-generated test quality, and debug complex flake patterns are in high demand. The role has become more senior and more strategic. Manual QA engineers who do not develop SDET skills will face automation of their core tasks.

No AI-testing-specific certification is yet widely recognised (April 2026). The closest relevant credentials: ISTQB Advanced Level (Test Automation Engineering), Playwright's official training, and AWS/GCP/Azure certifications for CI/CD pipeline context. Practical portfolio work (a public GitHub repo with mutation testing setup, Playwright MCP experiments) is more valuable than any certification for demonstrating AI testing skills.

No. This is an independent technical reference site. We are not affiliated with, endorsed by, or paid by any vendor covered here. Some pricing pages carry affiliate links to TestRigor, Qase, BrowserStack, LambdaTest, and Testsigma -- these are disclosed inline with {affiliate} tags. Affiliate status does not influence verdicts, rankings, or benchmark methodology.

Pricing pages are re-verified monthly. Benchmark data is re-run quarterly. Comparison verdicts are reviewed quarterly. The verification log at /log shows the specific dates each page was last checked and what (if anything) changed.

The verification log at /log is the correction feed. We do not have a public submission form yet. If you are a vendor and believe we have published incorrect pricing or feature data, our correction policy is: we publish your response verbatim alongside our original data. We do not remove unfavourable results -- we add context.