Breaking Down Tasks: Prompt Chaining

Why one big prompt struggles

Sanne de Vries, incident manager at Kestrel Energy, has forty ticket notes from a morning of portal complaints. She wants the common symptoms, the likely cause, a problem record and a customer update. One prompt can ask for all four, and a capable model will produce all four. But:

• Errors carry forward invisibly. If the model merges two different symptoms in its head, the cause, the problem record and the update are all built on that mistake, and nothing in the final text shows where it happened.
• Attention is shared. Every instruction competes with the others for the model's attention; tasks in the middle of a long list are the easiest to skip or skimp on (see How a Model Reads Your Prompt).
• You cannot reuse or test the parts. When the customer update is wrong, you have to rerun and reread everything.

Task decomposition is the first move: breaking a job into smaller subtasks that each have one clear goal and a result you can check. Microsoft's prompt engineering guidance puts it simply: models often perform better when a task is broken down into smaller steps. It is the same thing you would do when handing the work to a new colleague, and the same skill as in a structured troubleshooting method.

Prompt chaining

Prompt chaining carries out a decomposed task as a sequence of separate prompts, in which the output of one step is passed as input to the next. Each step is its own request, so between two steps there is a moment where you can look at the result, check it, fix it, or stop.

For Sanne's forty tickets, a chain could look like this:

1. Extract: "List the distinct symptoms in these ticket notes, with the ticket numbers that report each one."
2. Analyze: "Given this list of symptoms, which single change or failure could explain all of them? Name the evidence for and against."
3. Write: "Draft a customer update based only on this conclusion."

Sanne reads the symptom list after step 1. If it merged "login fails" and "login slow", she fixes the list before step 2 sees it. That single correction is something a one-shot prompt never gives her the chance to make.

Designing a chain

One job per step

Each step should have one goal, a clear input and an output you can judge without reading the rest of the chain. Good boundaries are the points where you would naturally check the work: after the facts are collected, after a decision is made, before anything is sent to a customer. Steps that need different settings are also good boundaries: extraction may need a fast model, analysis a reasoning model with more reasoning effort.

Intermediate output

An intermediate output is the result of one step in a chain that is used as input for a later step rather than shown as the final answer. Design it for the next step, not for a human reader: a fixed format (a list, a table, JSON with known fields) is easier to check and easier to pass on than free prose. Asking for that format is covered on Structured Output.

When you pass it on, mark it off clearly so the next prompt knows which part is data from the previous step. XML-style tags work well for that and are recommended by Anthropic, OpenAI and Google alike (see Giving Context and Reference Material):

Below is a list of symptoms extracted from today's portal tickets, with the ticket numbers that report each one.

<symptoms>
- Login fails with "session expired" right after entering the code: INC-40501, INC-40507, INC-40512, INC-40519
- Login page loads slowly (over 10 seconds): INC-40503, INC-40511
- Meter-reading form shows an error after submit: INC-40508
</symptoms>

Which single change or failure could explain all of these symptoms? Give your best explanation, the evidence for it and against it, and say which symptoms it does not explain.

Notice that step 2 does not get the forty raw tickets. It gets the checked list. Each step sees only what it needs, which keeps the prompts short and the model's attention on the task.

Gate checks between steps

A gate check is a test applied to an intermediate output before the next step may use it: valid format, required fields present, values from the allowed set, nothing obviously invented. If the check fails, the chain stops, retries the step, or hands the case to a person. It is the same idea as a quality gate in a CI/CD pipeline: nothing passes to the next stage without meeting the bar.

Most gate checks are ordinary code, cheap and deterministic. The example below checks three recorded outputs of an extraction step that was asked to return JSON with a ticket number, a category from a fixed list and a symptom. (Recorded answers keep the example runnable without a model; real answers differ from run to run.)

import json

ALLOWED = {"Network", "Account", "Portal", "Billing"}


def gate_check(raw):
    """Check step 1's output before step 2 may use it. Returns a list of problems."""
    try:
        data = json.loads(raw)
    except json.JSONDecodeError as err:
        return [f"not valid JSON ({err.msg})"]
    problems = []
    for key in ("ticket", "category", "symptom"):
        if key not in data:
            problems.append(f"missing field '{key}'")
    if data.get("category") not in ALLOWED:
        problems.append(f"unknown category {data.get('category')!r}")
    if not str(data.get("ticket", "")).startswith("INC-"):
        problems.append("ticket number does not start with INC-")
    return problems


# Three recorded outputs of step 1 ("extract the facts as JSON")
recorded = [
    '{"ticket": "INC-40311", "category": "Account", "symptom": "account locked"}',
    '{"ticket": "INC-40312", "category": "Hardware", "symptom": "laptop fan"}',
    'Sure! Here is the JSON: {"ticket": "INC-40313"}',
]

for raw in recorded:
    problems = gate_check(raw)
    verdict = "pass" if not problems else "STOP: " + "; ".join(problems)
    print(verdict)

pass
STOP: unknown category 'Hardware'
STOP: not valid JSON (Expecting value)

The second output is valid JSON with a perfectly reasonable category, just not one of yours; without the check it would quietly create a fifth queue. The third is the classic chatty answer with text around the JSON. Both are stopped before they reach the next step.

The next example strings three steps together (summarize, classify, draft a reply) with a check after each one. Each step's prompt is a template that receives only the previous step's result. Two sets of recorded answers show a run that completes and one that stops.

from string import Template

# One prompt template per step. Each step gets only what it needs.
STEPS = {
    "summarize": Template(
        "Summarize this ticket in one sentence.\n<ticket>\n$ticket\n</ticket>"
    ),
    "classify": Template(
        "Classify the summary as Network, Account, Portal or Billing. "
        "Answer with the label only.\n<summary>\n$summary\n</summary>"
    ),
    "draft": Template(
        "Write a two-sentence first reply for this $label ticket.\n"
        "<summary>\n$summary\n</summary>"
    ),
}
LABELS = {"Network", "Account", "Portal", "Billing"}


def run_chain(ticket, call_model):
    summary = call_model("summarize", STEPS["summarize"].substitute(ticket=ticket))
    print(f"  1 summary: {summary}")
    if not summary or len(summary.split()) > 30:  # gate check 1
        return "  stopped after step 1: summary empty or too long"

    label = call_model("classify", STEPS["classify"].substitute(summary=summary))
    print(f"  2 label:   {label!r}")
    if label not in LABELS:  # gate check 2
        return "  stopped after step 2: label not in the allowed set"

    reply = call_model("draft", STEPS["draft"].substitute(label=label, summary=summary))
    return f"  3 reply:   {reply}"


# Stand-in for the model: two sets of recorded answers, so the example runs offline.
recorded_runs = {
    "run A": {
        "summarize": "User is locked out of the portal after a password change.",
        "classify": "Account",
        "draft": "Thanks for your report. We will call you to verify your identity and unlock your account.",
    },
    "run B": {
        "summarize": "User is locked out of the portal after a password change.",
        "classify": "Account issue (probably)",
        "draft": "(never requested)",
    },
}

ticket = "INC-40311: since this morning's password change the portal says 'account locked'."
for name, answers in recorded_runs.items():
    print(name)
    print(run_chain(ticket, lambda step, prompt: answers[step]))

run A
  1 summary: User is locked out of the portal after a password change.
  2 label:   'Account'
  3 reply:   Thanks for your report. We will call you to verify your identity and unlock your account.
run B
  1 summary: User is locked out of the portal after a password change.
  2 label:   'Account issue (probably)'
  stopped after step 2: label not in the allowed set

In run B the model hedged ("Account issue (probably)"), which a human reads easily but a program should not guess about. The chain stops before writing a reply for the wrong queue. In production, the stop would typically retry once with a reminder ("Answer with the label only") and then route the ticket to a person.

Common chain patterns

Draft, review, refine

Anthropic describes self-correction as the most common chain: the model drafts, a second call reviews the draft against explicit criteria, and a third revises it using the review. Because each is a separate call, you can log, evaluate or branch at every point.

Review the draft customer update below against these criteria:
1. Every fact appears in the incident notes.
2. No promise of a resolution time.
3. At most 80 words, no technical terms.

<notes>…</notes>
<draft>…</draft>

List each criterion with "met" or "not met" and, if not met, the sentence that breaks it.

The review works better as a separate request than as "check your own work" at the end of the drafting prompt, because the reviewing call reads the draft as finished text against fixed criteria instead of continuing its own line of thought.

Map-reduce for long input

Map-reduce summarization handles input that is too long, or too noisy, for one prompt: the input is split into chunks, each chunk is processed with the same prompt (the map step), and a final prompt combines the short results (the reduce step). The name comes from the distributed data-processing pattern of the same name. Splitting works like the chunking used for retrieval, but here every chunk is read, not only the most similar ones.

Tomasz Nowak wants to know why the VPN tunnel to a partner dropped at night. The firewall log is long, so the program below splits it into chunks, builds one map prompt per chunk and then a reduce prompt from the recorded findings.

# A long log (here 10 lines; in practice thousands) split into chunks.
log = [
    "02:00:01 fw-edge-02 IKE SA established peer 203.0.113.10",
    "02:03:12 fw-edge-02 interface ge-0/0/1 up",
    "02:06:40 fw-edge-02 NTP sync ok",
    "02:08:30 fw-edge-02 config commit by admin (change CHG-2207)",
    "02:09:55 fw-edge-02 IKE rekey started peer 203.0.113.10",
    "02:10:02 fw-edge-02 IKE rekey failed: no proposal chosen",
    "02:14:07 fw-edge-02 tunnel st0.1 down peer 203.0.113.10",
    "02:14:09 fw-edge-02 BGP neighbor 198.51.100.1 down",
    "02:20:00 fw-edge-02 NTP sync ok",
    "02:25:41 fw-edge-02 IKE SA establishment failed peer 203.0.113.10",
]


def chunks(lines, size):
    for start in range(0, len(lines), size):
        yield start + 1, lines[start:start + size]


# Map step: one prompt per chunk, each small enough to read carefully.
map_prompts = []
for first, part in chunks(log, 4):
    text = "\n".join(part)
    map_prompts.append(
        f"List the lines in this log excerpt (lines {first}-{first + len(part) - 1}) "
        f"that point to a VPN problem. Quote them exactly.\n<log>\n{text}\n</log>"
    )
print(f"{len(log)} lines -> {len(map_prompts)} map prompts. The first one:")
print(map_prompts[0])

# Recorded answers of the map step (in practice: one model call per prompt).
map_answers = [
    "02:00:01 fw-edge-02 IKE SA established peer 203.0.113.10",
    "02:10:02 fw-edge-02 IKE rekey failed: no proposal chosen\n"
    "02:14:07 fw-edge-02 tunnel st0.1 down peer 203.0.113.10",
    "02:25:41 fw-edge-02 IKE SA establishment failed peer 203.0.113.10",
]

# Reduce step: one prompt that only sees the short findings, not the full log.
findings = "\n".join(f"<finding chunk=\"{i}\">{a}</finding>"
                     for i, a in enumerate(map_answers, start=1))
reduce_prompt = (
    "These findings come from consecutive parts of one firewall log.\n"
    f"{findings}\n"
    "Give the most likely cause of the VPN outage in two sentences."
)
print("\nThe reduce prompt:")
print(reduce_prompt)

10 lines -> 3 map prompts. The first one:
List the lines in this log excerpt (lines 1-4) that point to a VPN problem. Quote them exactly.
<log>
02:00:01 fw-edge-02 IKE SA established peer 203.0.113.10
02:03:12 fw-edge-02 interface ge-0/0/1 up
02:06:40 fw-edge-02 NTP sync ok
02:08:30 fw-edge-02 config commit by admin (change CHG-2207)
</log>

The reduce prompt:
These findings come from consecutive parts of one firewall log.
<finding chunk="1">02:00:01 fw-edge-02 IKE SA established peer 203.0.113.10</finding>
<finding chunk="2">02:10:02 fw-edge-02 IKE rekey failed: no proposal chosen
02:14:07 fw-edge-02 tunnel st0.1 down peer 203.0.113.10</finding>
<finding chunk="3">02:25:41 fw-edge-02 IKE SA establishment failed peer 203.0.113.10</finding>
Give the most likely cause of the VPN outage in two sentences.

Look closely at what the reduce prompt does not contain: line 4, the configuration commit for change CHG-2207 at 02:08:30, two minutes before the rekey failed. In chunk 1 it did not look like a VPN problem, so the map step left it out, and the reduce step can no longer connect the failure to the change, which is the most likely cause. This is the weak spot of map-reduce: each map call judges its chunk in isolation. Two remedies: make the map prompt broader ("list every error, change or state transition"), and keep the line numbers so a person can go back to the full log.

Extract, then answer

A simple two-step chain for questions about long documents: first ask for the passages that are relevant to the question, quoted exactly; check that the quotes exist; then ask the question with only those quotes as context. It is one of the most effective habits against invented answers, covered on Reducing Hallucinations.

Chains, reasoning models and agents

A chain is a workflow: you decide the steps and their order in advance. An AI agent decides its next step itself, in a loop of reasoning and actions (the ReAct pattern from Yao et al., 2022, which interleaves reasoning with actions such as searches). Agents handle problems whose steps you cannot predict; chains are cheaper, faster, easier to test and easier to explain to an auditor.

Reasoning models have also taken over part of what chains used to do: Anthropic notes that with built-in thinking a model handles most multi-step reasoning internally. Explicit chaining remains useful, in Anthropic's words, when you need to inspect intermediate outputs or enforce a specific pipeline structure. In practice that means:

• a person or a rule must approve something between steps;
• steps need different inputs, models, settings or permissions (the drafting step should not see customer data that the extraction step needed);
• the input does not fit in one prompt (map-reduce);
• you want to test and improve one step without rerunning the others (see Iterating and Testing Prompts).

If none of these apply, one well-written prompt to a reasoning model is often the simpler choice.

Chaining by hand in a chat

You do not need code to chain prompts. In a chat assistant, ask for step 1, read the result, correct it if needed, and paste the corrected version into the next prompt. Starting a new chat for each step keeps the earlier conversation from influencing the later one, and makes you pass on exactly what you checked instead of everything that was said.

Here are notes from six Kestrel Energy tickets. Find the root cause and write a 60-word update for customers.

INC-40501: login fails with "session expired" right after entering the code (09:12)
INC-40503: login page takes 15 seconds to load (09:20)
INC-40507: "session expired" after entering the code, on mobile (09:25)
INC-40508: meter-reading form shows an error after submit (09:31)
INC-40511: login page slow, then works (09:40)
INC-40512: code accepted, then "session expired" (09:44)

Here are notes from six Kestrel Energy tickets. List the distinct symptoms, each with the ticket numbers that report it. Do not guess at causes yet.

INC-40501: login fails with "session expired" right after entering the code (09:12)
INC-40503: login page takes 15 seconds to load (09:20)
INC-40507: "session expired" after entering the code, on mobile (09:25)
INC-40508: meter-reading form shows an error after submit (09:31)
INC-40511: login page slow, then works (09:40)
INC-40512: code accepted, then "session expired" (09:44)

These are the symptoms reported in today's Kestrel Energy portal tickets. Which explanations would fit all of them, and which symptoms would each explanation leave unexplained? Do not write a customer update yet.

<symptoms>
(paste the list from the previous answer here)
</symptoms>

Costs and pitfalls

• More calls, more waiting. Three steps are three requests. Usually each is shorter, so the total cost need not be higher, but the steps run one after another.
• Lost context. A step only knows what you pass it. If the drafting step needs the customer's language or the ticket number, pass it explicitly.
• Errors still propagate when there is no check between steps. A chain without gate checks is one big prompt with extra latency.
• Over-splitting. Ten tiny steps for a task one prompt handles well add complexity without benefit. Split where a check adds value, not everywhere.
• Untrusted data travels along. Text from a customer email that contains instructions is still in step 3 if steps 1 and 2 copied it. Treat every intermediate output that came from outside data as data, not as instructions (see Prompt Security and Responsible Use).