Iterating and Testing Prompts

Why gut feeling is not enough

Mei Chen, service desk lead at Kestrel Energy, writes a prompt that sorts incoming tickets into four queues. She tries it on a ticket, the label is right, she adjusts a sentence, tries another ticket, and after half an hour it "works". Three things make that conclusion shaky:

• Answers vary. The same prompt can give different answers on different runs; Microsoft's Copilot guidance tells users to expect this. One good answer does not show that the next one will be good.
• Fixes do not generalize by themselves. The sentence that fixed ticket 2 may have broken ticket 1, which she did not try again. Microsoft's prompt engineering documentation warns that a prompt working in one scenario does not necessarily work in others.
• Memory is selective. The striking failure gets remembered, the quiet ones do not.

Prompt evaluation is the practice of measuring how well a prompt performs by running it on a fixed set of test cases and grading the results against criteria decided in advance. Anthropic puts it before any prompt engineering at all: first define success criteria, then a way to test them, then write the first draft. It is the same discipline as automated testing in software, adapted to output that is not always the same.

Success criteria you can measure

"Good summaries" cannot be tested. "Every summary mentions the ticket number, the affected service and the current status, in at most 50 words" can. Anthropic's guidance asks for criteria that are specific, measurable, achievable and relevant, and notes that most uses need several dimensions at once, among them task fidelity, consistency, tone, privacy, latency and price. (The idea of a success criterion as part of every prompt is introduced on Anatomy of a Good Prompt.)

Note the safety row: an average hides the case that must never go wrong. Give such cases their own threshold instead of folding them into the overall score.

Building an eval set

An eval set is a fixed collection of realistic inputs, each with the expected result or grading rule, that is used to measure a prompt and to compare versions of it. It plays the role of a test set in machine learning, but for a prompt rather than for a trained model.

• Take real cases, with personal data removed or replaced (see Privacy and Data Protection). Invented test tickets tend to be cleaner than the real ones.
• Cover the edge cases on purpose: tickets in Dutch, tickets that mention two problems, tickets with only a screenshot reference, the "portal slow, but only on VPN" case. Anthropic's advice is explicit: do not forget the edge cases.
• Prefer more cases over perfect grading. Anthropic recommends many automatically graded questions over a few carefully hand-graded ones. Twenty cases is a start; a hundred tells you much more.
• Keep it apart from your examples. A ticket that appears as an example in the prompt (see Zero-Shot, One-Shot and Few-Shot Prompting) will be answered correctly for the wrong reason. Test cases and prompt examples must not overlap.
• Keep it under version control next to the prompt, so you always know which prompt was measured on which set (see Prompt Templates and Reuse).

Golden answers

A golden answer is the reference result for a test case, agreed on by people in advance, against which a model's answer is graded. For a classification it is simply the correct label. For free text it is usually not a full reference text but a set of facts or properties the answer must have ("mentions INC-40311", "says that someone will call back", "does not promise a time"). Agreeing on golden answers is often where the real work is: if two service desk colleagues disagree on the right queue for a ticket, the prompt cannot be graded on it until they settle it.

Grading: code, a model, people

Exact-match grading

Exact-match grading compares a model's answer character for character with the golden answer, usually after a small, fixed normalization such as ignoring letter case and surrounding spaces. It is fast, free and repeatable, and it fits labels, numbers, yes/no answers and IDs. The example grades five recorded answers (real answers differ from run to run and model to model).

# Golden answers (labels a person agreed on) and recorded model answers
cases = [
    ("INC-40311", "Account", "Account"),
    ("INC-40312", "Network", "network"),
    ("INC-40313", "Portal", "Portal."),
    ("INC-40314", "Billing", "Label: Billing"),
    ("INC-40315", "Network", "Portal"),
]


def normalize(answer):
    """Forgive case, spaces and a final period; nothing else."""
    return answer.strip().rstrip(".").strip().lower()


strict = sum(answer == golden for _, golden, answer in cases)
lenient = sum(normalize(answer) == normalize(golden) for _, golden, answer in cases)
print(f"strict exact match:     {strict}/{len(cases)}")
print(f"normalized exact match: {lenient}/{len(cases)}")
for ticket, golden, answer in cases:
    if normalize(answer) != normalize(golden):
        print(f"  {ticket}: expected {golden!r}, got {answer!r}")

strict exact match:     1/5
normalized exact match: 3/5
  INC-40314: expected 'Billing', got 'Label: Billing'
  INC-40315: expected 'Network', got 'Portal'

Normalization rescued "network" and "Portal." but rightly not "Label: Billing": the prompt asked for the label only, and a program that consumes the answer would choke on it. That failure is not a grading problem but a prompt problem, and it shows up only because the grading is strict. Decide the normalization once and keep it fixed; loosening it every time a case fails makes the score meaningless.

Rule-based checks for free text

A reply to a customer has no single correct text, but it has properties code can check: length, required elements, forbidden phrases, ticket numbers that must come from the input.

import re

# Checks for a free-text answer: a first reply to a customer ticket
ticket = "INC-40311: customer locked out of the portal after a password change."
reply = (
    "Thank you for reporting this (INC-40311). Your account was locked after "
    "the password change. A colleague will call you today to verify your identity "
    "and unlock it; this will be fixed within 15 minutes. Reference: INC-40377."
)


def check_reply(ticket, reply, max_words=60):
    problems = []
    words = len(reply.split())
    if words > max_words:
        problems.append(f"too long: {words} words (max {max_words})")
    invented = set(re.findall(r"INC-\d{5}", reply)) - set(re.findall(r"INC-\d{5}", ticket))
    for number in sorted(invented):
        problems.append(f"ticket number not in the input: {number}")
    if re.search(r"\bwithin \d+ (minutes|hours|days)\b", reply):
        problems.append("promises a resolution time")
    return problems


for problem in check_reply(ticket, reply) or ["all checks passed"]:
    print(problem)

ticket number not in the input: INC-40377
promises a resolution time

Both problems are typical of free text: a reference number that appears nowhere in the input (see fabricated references) and a promise nobody made. Checks like these are crude, but they run on every case, every time, at no cost.

LLM-as-a-judge

Some qualities need judgment: is the tone friendly, is the summary faithful to the ticket, does the answer actually address the question? LLM-as-a-judge is grading in which a second language model scores or classifies an answer against explicit criteria, so that judgment-based checks can run on many cases automatically. The judge prompt works best when it asks for a narrow, fixed verdict rather than a general impression:

You are grading a draft reply to a customer ticket.

<ticket>…</ticket>
<reply>…</reply>

Criterion: every factual statement in the reply is supported by the ticket text.
First list each factual statement in the reply. Then mark each one "supported" or "not supported". End with one line: VERDICT: PASS if all are supported, otherwise VERDICT: FAIL.

A judge is a model too, with its own errors and biases, as Evaluating Models explains for model-as-judge evaluation in general. Before trusting it, have people grade twenty or thirty cases and check that the judge agrees with them. Anthropic's evaluation guide calls it best practice to use a different model as judge than the one that produced the answers, so the two do not share the same blind spots.

Human review

People remain the reference: they set the golden answers, check the judge, and review the cases that automated grading flags or cannot decide. Their time is the scarcest resource, so spend it on the cases that matter most and on calibrating the automated checks, not on rereading answers that code can grade.

The iteration loop

The loop is simple; the discipline is in two rules. Change one thing at a time, or you cannot tell which change helped. And look at the cases, not only the score: a version that scores higher may have fixed three easy cases and broken the one that matters. The example compares two versions of Mei's prompt on ten recorded results.

import csv
import io

# One row per test case: golden label, then recorded answers of two prompt versions
results = """ticket,golden,v1,v2
INC-40401,Network,Network,Network
INC-40402,Account,Account,Account
INC-40403,Portal,Network,Portal
INC-40404,Billing,Billing,Billing
INC-40405,Network,Portal,Network
INC-40406,Account,Account,Portal
INC-40407,Portal,Portal,Portal
INC-40408,Billing,Account,Account
INC-40409,Portal,Portal,Portal
INC-40410,Network,Network,Network
"""

rows = list(csv.DictReader(io.StringIO(results)))
for version in ("v1", "v2"):
    correct = sum(row[version] == row["golden"] for row in rows)
    print(f"{version}: {correct}/{len(rows)} correct ({correct / len(rows):.0%})")

fixed = [r["ticket"] for r in rows if r["v1"] != r["golden"] and r["v2"] == r["golden"]]
broken = [r["ticket"] for r in rows if r["v1"] == r["golden"] and r["v2"] != r["golden"]]
still = [r["ticket"] for r in rows if r["v1"] != r["golden"] and r["v2"] != r["golden"]]
print("fixed by v2:  ", ", ".join(fixed))
print("broken by v2: ", ", ".join(broken) or "none")
print("wrong in both:", ", ".join(still) or "none")

v1: 7/10 correct (70%)
v2: 8/10 correct (80%)
fixed by v2:   INC-40403, INC-40405
broken by v2:  INC-40406
wrong in both: INC-40408

Version 2 is better on average, and it broke INC-40406, which version 1 got right. Before keeping it, Mei reads that ticket: if the new wording pushed account problems into the portal queue, other account tickets will follow. A regression in one case is a question to answer, not a footnote. INC-40408, wrong in both versions, is the next thing to work on, or a sign that its golden answer needs another look.

Keep a short log per round: the change, the score, what broke. It takes a minute and saves you from trying the same idea twice.

Run each case more than once

Because answers vary between runs, one run per case measures the prompt and luck together. For tasks where the answer should not change, run each case several times and look at the spread:

from collections import Counter

# The same prompt run three times per test case (recorded answers)
runs = {
    "INC-40401": ["Network", "Network", "Network"],
    "INC-40403": ["Portal", "Network", "Portal"],
    "INC-40406": ["Account", "Account", "Account"],
    "INC-40408": ["Billing", "Account", "Account"],
}

stable = 0
for ticket, answers in runs.items():
    counts = Counter(answers)
    label, votes = counts.most_common(1)[0]
    if votes == len(answers):
        stable += 1
        print(f"{ticket}: stable   {label}")
    else:
        spread = ", ".join(f"{k} x{v}" for k, v in sorted(counts.items()))
        print(f"{ticket}: UNSTABLE {spread}")
print(f"{stable} of {len(runs)} cases gave the same answer in every run")

INC-40401: stable   Network
INC-40403: UNSTABLE Network x1, Portal x2
INC-40406: stable   Account
INC-40408: UNSTABLE Account x2, Billing x1
2 of 4 cases gave the same answer in every run

An unstable case is a borderline case: the prompt does not settle it, so the model lands on either side. That is useful information. Either the ticket is ambiguous (and maybe both labels are acceptable), or the prompt needs an example or a rule for this kind of ticket. Settings that make output more predictable help, but as of October 2026 several current models no longer accept a custom temperature setting at all, and even temperature 0 never guaranteed identical output (see temperature). Plan for variation instead of trying to switch it off.

Classify each Kestrel Energy ticket as exactly one of: Network, Account, Portal, Billing. Answer with one line per ticket: the number, a colon, the label. Nothing else.

1: Portal is very slow, but only when I'm on the company VPN.
2: My bill shows two meter readings for the same day.
3: I'm locked out after changing my password.
4: The Wi-Fi in the Zwolle office drops every few minutes.
5: The meter-reading form gives an error when I press submit.
6: I want to change the bank account for my direct debit.
7: The portal says my session expired right after I log in.
8: Can't reach the portal from home, other websites work fine.

Prompt regressions and model changes

A prompt regression is a test case that a prompt handled correctly before and handles incorrectly after a change, either to the prompt itself or to the model, settings or surrounding application. The second kind is easy to miss, because nobody touched the prompt.

• Pin the model version. OpenAI advises pinning production applications to specific model snapshots and building evaluation suites; every Claude model ID refers to a fixed snapshot (as of October 2026). An alias that silently moves to a newer model also moves your prompt's behavior.
• Expect forced upgrades. Models are retired on a schedule. Anthropic, for example, lists Claude Haiku 4.5 for retirement not sooner than 15 October 2026. Before a switch, run the eval set on the new model and compare, exactly as for a prompt change.
• Re-test old tricks. Advice that helped one model can hurt the next. Anthropic's model-specific guidance repeatedly tells developers to check techniques against their own evals rather than carrying them over, and to test settings such as reasoning effort again on each model.
• Version prompts like code. Store each prompt with its eval results, review changes and re-run the set before release (see Prompt Templates and Reuse). Then "the answers got worse last week" becomes a diff you can read.