Reasoning: Step by Step and Thinking Models

Why working it out helps

A language model produces its answer one token at a time, and it spends roughly the same amount of computation on every token (see How a Model Reads Your Prompt). If you ask "Which change window conflicts with the backup job?" and the very first token of the answer must be "A" or "B", the model has to settle the whole question in that single step.

When the model first writes "The backup runs from 01:00 to 03:30. Window A starts at 02:00, which is inside that period…", every one of those tokens becomes part of the text the model reads for the next token. Intermediate results are now on the page instead of having to be worked out implicitly, in one go. A common way to put it: the written steps give the model room to compute. That is why asking for reasoning helps most on problems with several dependent steps (arithmetic, time zones, subnet calculations, logic puzzles, planning) and hardly at all on questions that need one fact.

Two things it does not do. It does not give the model knowledge it lacks: a model that does not know your network will reason its way to a confident guess. And the written steps are still generated text, which can contain errors. More on both below.

Chain-of-thought prompting

Chain-of-thought prompting asks a model to write out intermediate reasoning steps before its final answer, either by showing worked examples that include the steps or by instructing it to reason first. The name comes from a 2022 paper by Wei et al. at Google, which gave a large model a few worked examples (a form of few-shot prompting) in which each answer was preceded by its reasoning. On math word problems, commonsense and symbolic reasoning tasks this beat examples with the answer alone; with eight such examples their largest model (PaLM, 540 billion parameters) reached the best result published at the time on GSM8K, a benchmark of grade-school math word problems. The authors noted that the ability appeared only in sufficiently large models.

Q: A maintenance window runs from 22:00 to 01:30. The backup starts at 00:45 and takes 90 minutes. Does the backup finish inside the window?
A: The backup ends at 00:45 + 1:30 = 02:15. The window ends at 01:30. 02:15 is later than 01:30, so no.

Q: A maintenance window runs from 23:00 to 02:00. The patch job starts at 23:40 and takes 70 minutes. Does it finish inside the window?
A:

The worked example teaches two things at once: the format of the answer and the habit of computing the end time before deciding.

"Let's think step by step"

A few months later Kojima et al. showed that the examples were not even necessary. Adding one sentence, "Let's think step by step", before the answer was enough to make a model of that time write out its reasoning, and its accuracy jumped: on MultiArith (arithmetic word problems) from 17.7% to 78.7%, on GSM8K from 10.4% to 40.7%. This is called zero-shot chain of thought: reasoning triggered by an instruction alone, without examples.

Those numbers are why "think step by step" became the most copied line in prompting advice. They were measured on 2022 models that answered immediately unless told otherwise. Most models you use in 2026 are different, as the next section explains, and much older material has not caught up.

Reasoning models and built-in thinking

A reasoning model is trained to reason before it answers, without being asked. It does in a separate phase what chain-of-thought prompting asked for in the visible answer. As of October 2026 the flagship models of Anthropic, OpenAI and Google all work this way, and so do the assistants built on them; some assistants also switch between a fast mode and a thinking mode, automatically or with a button.

Thinking tokens

Thinking tokens (OpenAI calls them reasoning tokens) are the tokens a reasoning model generates while it reasons, before the answer you see. Three practical consequences:

• You pay for them. All three large providers bill thinking as output tokens, the expensive kind, even when you never see the text (as of October 2026).
• They take space. Thinking tokens occupy the context window while the answer is generated. On Claude, the maximum output length you set in an API call covers thinking and answer together, so a low limit can cut the answer off before it starts (see Prompts in Applications: the API).
• You usually see a summary, not the raw text. Providers return a summary of the reasoning, or nothing, rather than the full thinking (at all three providers the summary is an opt-in setting on current models).

Extended thinking and adaptive thinking

Extended thinking is Anthropic's name for Claude's built-in reasoning mode, in which the model produces thinking content before its answer. Earlier Claude models let you switch it on with a fixed token budget for thinking. As of October 2026 that budget setting is deprecated on Claude Opus 4.6 and Sonnet 4.6, and on later models it is rejected with an error. These models use adaptive thinking: the model decides per request how much to think, and on the newest models (Claude Fable 5.1 and Opus 5.5, for example) thinking is always on. Claude Haiku 4.5, the smallest current model, still uses the budget setting.

Reasoning effort

Reasoning effort is a setting that tells a reasoning model roughly how much thinking to do before it answers: low for speed and cost, high for hard problems. Each provider names and scales it differently. Anthropic describes it as "a behavioral signal, not a strict token budget": the model treats it as guidance, not as a hard limit.

In a chat assistant you rarely see this setting by name; a "fast" versus "thinking" choice in the interface is the same idea. In an application, it is the first thing to tune: a lower effort on a simple task can cut cost and waiting time sharply with no loss in quality, and only your own tests tell you where that point is (see Iterating and Testing Prompts).

Prompting a thinking model

Built-in thinking changes the advice, and the vendors agree on the main point: do not tell a reasoning model how to think.

Research points the same way. A 2025 study from the Wharton School (Prompting Science Report 2) tested chain-of-thought instructions on several models: for reasoning models they gave marginal gains in accuracy, if any, while clearly increasing time and tokens; for non-reasoning models a small average improvement, with more variation between questions.

What does help with a thinking model:

• State the problem completely. All the facts, constraints and the exact question. Thinking cannot replace missing information; it can only reason about what is there.
• Say what a good answer looks like, not how to get there: "Give the answer and one sentence why" or "List the conflicts as a table".
• Ask for a check when it matters. "Before you finish, verify that every time in your answer appears in the schedule above" names a concrete test. Anthropic notes that this kind of self-check instruction helps on most of its models but made one model over-verify: a reminder to re-test such tricks on each new model.
• Examples still work. Few-shot examples combine with thinking; show each example as the problem, the method and the expected answer (see Zero-Shot, One-Shot and Few-Shot Prompting).

Our backup job runs daily from 01:00 to 03:30 CET. Change window A is 02:00–02:45 CET, change window B is 04:00–05:00 CET, change window C is 03:15–04:15 CET.

Which windows overlap the backup job? Answer with the window letters, then one sentence per window explaining why.

When thinking is off: reasoning in the prompt

Models without built-in thinking, or with thinking switched off where the API allows it, still benefit from writing out steps on multi-step problems. Anthropic's advice for that case is manual chain of thought: ask for the reasoning first and put the final answer in its own tags, so a program (or a busy reader) can find it without reading the rest. On models where thinking is always on, lower the effort setting instead; asking them to write out their reasoning may be declined.

Our backup job runs daily from 01:00 to 03:30 CET. Change window A is 02:00–02:45 CET, change window B is 04:00–05:00 CET.

Work out which window overlaps the backup job. Reason first, then give only the letter of the overlapping window inside <answer></answer> tags.

The tags matter when the answer is used by code: the reasoning varies from run to run, the tags give you a stable place to read the result. The example in the next section extracts it.

Self-consistency: ask more than once

Because answers vary from run to run, one run can take a wrong turn that another avoids. Self-consistency is a technique that runs the same reasoning prompt several times and takes the final answer that comes up most often, on the idea that correct reasoning paths tend to agree while errors scatter. Wang et al. (2022) introduced it on top of chain-of-thought prompting and improved accuracy considerably, for example by 17.9 percentage points on GSM8K.

The program below takes five recorded answers to the backup question above, as if the same prompt had been sent five times (real answers differ from run to run and model to model; these are made up for the example). It extracts what is between the <answer> tags and counts the votes.

import re
from collections import Counter

# Five recorded answers to the same question, asked five times in five
# separate requests. Each answer reasons first and ends with <answer> tags.
recorded = [
    "Backup runs 01:00-03:30. Window A is 02:00-02:45, inside it. <answer>A</answer>",
    "Window B starts at 04:00, after the backup ends. <answer>A</answer>",
    "02:00 lies between 01:00 and 03:30, so A overlaps. <answer>A</answer>",
    "Reading 03:30 as 3:30 PM, window B would be free... <answer>B</answer>",
    "Only window A overlaps the backup job.",
]


def final_answer(text):
    """Return what is inside <answer>...</answer>, or None if missing."""
    match = re.search(r"<answer>\s*(.*?)\s*</answer>", text, re.DOTALL)
    return match.group(1) if match else None


answers = [final_answer(text) for text in recorded]
print("Extracted:", answers)

votes = Counter(a for a in answers if a is not None)
best, count = max(sorted(votes.items()), key=lambda item: item[1])
print(f"Votes: {dict(sorted(votes.items()))}")
print(f"Majority answer: {best} ({count} of {len(recorded)} runs)")
print(f"Runs without an <answer> tag: {answers.count(None)}")

Extracted: ['A', 'A', 'A', 'B', None]
Votes: {'A': 3, 'B': 1}
Majority answer: A (3 of 5 runs)
Runs without an <answer> tag: 1

Three details carry over to real use. The program never trusts one run. It counts a missing tag separately instead of guessing what the model meant. And it breaks ties in a fixed way (sorting first), so the same input always gives the same result. The price is obvious: five runs cost five times as much. Self-consistency is worth it for a few important, checkable answers, not for every request. A disagreement between runs is useful information in itself; see Reducing Hallucinations.

When extra reasoning does not help

• Simple tasks. Classifying a ticket, extracting a field or rewording a sentence does not get better with more thought. Anthropic recommends low effort for speed-sensitive or simple tasks; higher settings only add latency and cost.
• Missing facts. No amount of reasoning tells a model which firewall rule changed last night. Give it the data (see Giving Context and Reference Material).
• A wrong premise. If the prompt says the backup starts at 02:00 when it really starts at 01:00, careful reasoning produces a carefully wrong answer.
• Large jobs with many parts. Reasoning helps within one problem; a job with separate stages (collect symptoms, find the cause, write the update) is often better split into steps you can check, see Breaking Down Tasks: Prompt Chaining.
• Precise arithmetic at scale. Reasoning reduces arithmetic slips but does not make a model a calculator. For exact sums, subnet math or date arithmetic on many rows, ask for code or a formula you run yourself (see Limits and Myths).

Think step by step very carefully before you answer and write out every step of your reasoning. Kestrel Energy's backup job runs daily from 01:00 to 03:30 CET. Change window A is 02:00–02:45 CET. Change window B is 04:00–05:00 CET. Which window overlaps the backup job?

Kestrel Energy's backup job runs daily from 01:00 to 03:30 CET. Change window A is 02:00–02:45 CET. Change window B is 04:00–05:00 CET. Which window overlaps the backup job? Give the answer and one sentence why.