The QA Engineer's Guide to LLM Testing: From Test Cases to Golden Sets

Every QA methodology you've used shares one assumption: the same input produces the same output. Press B4, get Doritos. LLM features break that assumption on purpose — the variability is the product. So "expected result equals actual result" stops working as a test design, and a lot of experienced QA engineers conclude that AI features can't be tested. They can. The discipline just changes shape.

## What replaces the test case? The golden set.

A golden set is a curated collection of inputs with expected *behaviors* — not expected strings. For a dispute-classification feature, a golden case isn't "input X produces output Y verbatim"; it's "input X is classified as card-not-present fraud, with confidence above the auto-route threshold." You're testing the decision, not the words.

Coverage is where QA instincts transfer directly. A credible golden set has four case types, and the mix matters more than the count: **typical** cases that represent everyday traffic, **edge** cases at the boundaries (ambiguous phrasing, borderline amounts, mixed intents), **adversarial** cases deliberately designed to break guardrails (prompt injection, requests for out-of-policy actions, inputs crafted to trigger forbidden behavior), and **out-of-scope** cases the system must refuse or escalate rather than attempt. Start at 15–20 cases to make your first go/no-go call; grow toward 50+ before production. Every production incident becomes a new case — your golden set is a living regression suite.

## How do you score outputs that are never identical?

Three judges, used together. **Rule-based checks** handle structure: valid JSON, required fields, no forbidden strings — fast and free. **LLM-as-judge** handles quality: a second model scores each output against a rubric you write ("Is the classification supported by the dispute text? Is the confidence calibrated to the ambiguity?"). Validate the judge against human scores on a sample before trusting it. **Human review** handles the sample you can't automate — and calibrates the other two.

The output of an eval run is a score distribution, not a pass/fail. This is the biggest mental shift: a 91% aggregate with three failures concentrated in adversarial cases is *worse* than an 87% spread evenly across typical cases, because guardrail failures ship as incidents. Read the distribution by case type before you read the headline number.

## The part QA should refuse to give up

Set the passing threshold *before* the eval runs, and write it down where stakeholders can see it. If the threshold is negotiated after the results come in, you don't have a quality gate — you have a ritual. The NO-GO verdict is QA's entire authority in an AI project, and it only means something if it can actually block a launch.

None of this requires reading model code, understanding transformers, or writing Python. It requires exactly what made you good at QA: adversarial imagination, systematic coverage, and the stubbornness to hold a bar someone wants lowered.