Giving Context and Reference Material

Why context matters

A language model has read a great deal of public text during training, but it has never seen your knowledge base, your network, your change calendar or the ticket in front of you. It also does not know today's date unless the application tells it. When a prompt asks about something the model has no information on, it does not stop; it continues the text with whatever is most plausible. That is how invented procedures, ticket numbers and settings appear in otherwise fluent answers (see hallucination).

The remedy is to put the facts in the prompt. Reference material is the text you add to a prompt for the model to work from: a knowledge article, a runbook section, a policy, a log excerpt, a ticket history. Microsoft's prompt engineering guidance calls this grounding data and describes it as one of the most effective ways to get reliable answers. It adds a useful observation: the closer your source material is to the final form of the answer you want, the less work the model has to do, and the less room it has to go wrong.

Do not assume an assistant "already knows" your environment because it runs inside a company tool. As of October 2026, Microsoft Copilot Chat with a work account answers from the web by default; your organization's documents only come into play when you paste or upload them, when a file is open in Copilot's Word, Excel or PowerPoint agent, when you use it in Outlook, or through an agent that is connected to them. Other assistants work the same way: they see what you give them and what they are connected to, nothing else.

What to include

Think of the briefing you would give a capable colleague who joined this morning. Useful context falls into a few groups:

• Background facts: who you are, who the answer is for, which system and version, what has already been tried.
• Reference documents: the knowledge article, procedure, policy or specification the answer must follow.
• The data to work on: the ticket text, the log lines, the configuration section, the customer's email.
• Decisions and constraints: "we cannot restart the firewall during business hours", "the customer has already received a credit".

The same request with and without context shows the difference. The first prompt leaves everything to the model's imagination:

Write a reply to a customer whose smart meter reading is missing.

Write a reply to a Kestrel Energy customer. Their smart meter has sent no readings since 29 September. I checked the metering dashboard: the meter status is "no communication". Our procedure says we create a ticket for the Metering team, who reply within two working days, and that missing readings are estimated, so the customer does not need to do anything.

The reply is for a customer without technical knowledge: at most 80 words, friendly, no promises beyond what the procedure says.

The second prompt contains the facts the answer needs, so the model can write it without inventing a cause, a team or a deadline. Whatever it adds beyond those facts is now easy to spot.

Marking material off with delimiters

Once a prompt holds both instructions and material, the model has to work out which is which. A sentence in a pasted email such as "please reply before Friday" could be read as an instruction to the model. A prompt delimiter is a marker that shows where a piece of content starts and ends, so the boundaries between instructions, examples and data are unambiguous. Common delimiters are XML-style tags, Markdown headings, lines of three dashes and triple quotes.

An XML tag in a prompt is a label in angle brackets, such as <email> … </email>, that wraps and names one part of the prompt. It borrows the syntax of XML, but nobody parses it strictly: the tag names are your choice, and they work because they are clear to the model. Use names that describe the content (<policy>, <ticket>, <log>), use the same names every time, and refer to them in your instructions ("using the policy in the <policy> tags").

<policy>
Account unlocks: the service desk unlocks an account only after verifying the caller's identity by calling back the phone number in the HR system.
</policy>

<ticket>
INC-40311 · Field engineer reports "account locked" after changing his password this morning. Calling from a number not in the HR system.
</ticket>

Using only the policy, list the steps Amara must take for this ticket, in order.

The vendors agree on the idea and differ in their preferred notation (as of October 2026):

None of these is magic syntax; consistency matters more than the choice. Pick one style for your team and keep it.

Several documents

When you give several documents, wrap each one separately and give it a source and a number, so the model can say where a statement comes from and you can check it. Anthropic's documentation shows this structure: a <documents> element holding one <document> per source, each with a <source> and a <document_content>. The example builds that structure in Python. It escapes <, > and & inside the content (raw syslog lines start with a priority such as <134>), so text inside a document can never look like a closing tag. Models read the escaped form without trouble.

from xml.sax.saxutils import escape

# Reference material for one question: (source, content) pairs.
documents = [
    ("KB-0217 VPN error 809",
     "Error 809 means the local network blocks VPN traffic.\n"
     "Fix: connect through a phone hotspot, or ask the venue to allow VPN."),
    ("Syslog excerpt, fw-edge-02",
     "<134>Oct  3 02:14:07 fw-edge-02 ike: tunnel to 203.0.113.10 down\n"
     "<134>Oct  3 02:14:09 fw-edge-02 ike: retry 1 failed (peer timeout)"),
]


def as_documents(docs):
    """Wrap each document in tags; escape <, > and & inside the content."""
    lines = ["<documents>"]
    for index, (source, content) in enumerate(docs, start=1):
        lines.append(f'<document index="{index}">')
        lines.append(f"<source>{escape(source)}</source>")
        lines.append(f"<document_content>\n{escape(content)}\n</document_content>")
        lines.append("</document>")
    lines.append("</documents>")
    return "\n".join(lines)


question = ("Using only the documents above, explain in two sentences "
            "why the tunnel went down. Cite the document index.")

print(as_documents(documents))
print()
print(question)

<documents>
<document index="1">
<source>KB-0217 VPN error 809</source>
<document_content>
Error 809 means the local network blocks VPN traffic.
Fix: connect through a phone hotspot, or ask the venue to allow VPN.
</document_content>
</document>
<document index="2">
<source>Syslog excerpt, fw-edge-02</source>
<document_content>
&lt;134&gt;Oct  3 02:14:07 fw-edge-02 ike: tunnel to 203.0.113.10 down
&lt;134&gt;Oct  3 02:14:09 fw-edge-02 ike: retry 1 failed (peer timeout)
</document_content>
</document>
</documents>

Using only the documents above, explain in two sentences why the tunnel went down. Cite the document index.

Building the prompt in code has a second advantage: every request has exactly the same structure, which makes results comparable when you test the prompt (see Iterating and Testing Prompts).

Long documents: material first, question last

Long-context prompting means writing prompts that contain a large amount of material, often tens of thousands of tokens: a full runbook, months of change records, a long contract. Current models accept a lot (as of October 2026 the larger models from Anthropic, OpenAI and Google take around one million tokens; see context window), but how you lay the material out still matters.

Anthropic advises putting long documents at the top of the prompt, above the instructions, examples and question, and reports that in its tests a question at the end improved response quality by up to 30%, especially with complex inputs made of several documents. Google's Gemini guidance says the same: supply all the context first and place the specific instructions or question at the very end. Microsoft adds that the end of a prompt can have more influence than the beginning, and suggests repeating the key instruction at the end of a long prompt.

Position matters because models do not attend to every part of a long input equally well; research found that information in the middle of a long context is the most likely to be missed (see How a Model Reads Your Prompt). That also means a bigger context window is not an invitation to paste everything. Each extra page dilutes the parts that matter, costs tokens on every request and gives the model more material to confuse. When the material is too large to select by hand, let software pick the relevant passages: retrieval-augmented generation does exactly that, and Context Engineering and Agents covers managing everything a model sees.

Answering from the material only

Pasting a document does not stop the model from also using what it learned in training. Sometimes that is what you want; for a procedure question it is not, because general knowledge about "how energy companies usually do it" is not your procedure. Say explicitly that the answer must come from the material, and give the model a way out for questions the material does not answer. Without that way out, the most plausible continuation is still an answer, invented if necessary.

<runbook>
(the runbook section for VPN outages)
</runbook>

Answer the question using only the runbook above, not general knowledge. If the runbook does not answer the question, reply exactly: "The runbook does not cover this."

Question: who must be informed when more than 50 home workers lose VPN access?

This lowers the rate of invented answers; it does not remove it. Reducing Hallucinations goes further into giving the model permission to say "I don't know" and checking its claims.

Ask for quotes first

Quote extraction is the technique of asking the model to first copy, word for word, the passages of the material that are relevant to the question, and then to answer based on those passages. Anthropic recommends it for long documents. It helps in two ways: the model narrows its attention to the right parts before answering, and you get quotes you can search for in the source. A quote that is not in the document is a clear warning sign.

<change_records>
(change records for the customer portal, June to September)
</change_records>

First, copy the sentences from the change records that mention the login service, word for word, inside <quotes> tags, each with its change number. Then, inside <answer> tags, say which change most likely relates to the login errors that started on 12 September, based only on those quotes.

What is Kestrel Energy's procedure when a customer reports that their smart meter has shown no readings for three days? Who should the service desk contact first, and does the customer get compensation?

<procedure>
Kestrel Energy · Procedure SM-04: smart meter without readings
1. Ask the customer for the meter number and the date of the last reading shown in the customer portal.
2. Check the meter status in the metering dashboard.
3. If the status has been "no communication" for more than 48 hours, create a ticket in queue MTR-2 for the Metering team.
4. Tell the customer that the Metering team replies within two working days.
5. Readings for the missing days are estimated; the customer does not need to do anything.
</procedure>

Answer only from the procedure above, not from general knowledge. For each question, quote the step you used. If the procedure does not answer a question, write "The procedure does not cover this."

1. Who should the service desk contact first?
2. Does the customer get compensation for the missing readings?

Answers differ from run to run and from assistant to assistant, so try each prompt twice. The pattern is usually stable even when the wording is not.

Give the minimum that answers the question

More context is not automatically better context. A 5,000-line log with one relevant error is mostly noise to the model, and every line you paste is a line that leaves your organization if the assistant is an external service. Data protection law and most company AI policies ask for the same habit: share only the data the task needs (data minimization, see the GDPR principles and what never to paste).

The example keeps only the lines around each error in a log, with their original line numbers, and marks the gaps between the kept blocks. The model gets the evidence, you get line numbers to check its answer against.

log = """02:13:40 ike: keepalive to 203.0.113.10 ok
02:13:50 ike: keepalive to 203.0.113.10 ok
02:13:55 dhcp: lease renewed for 192.0.2.41
02:14:00 ike: keepalive to 203.0.113.10 ok
02:14:05 ike: keepalive to 203.0.113.10 no reply
02:14:07 ike: ERROR tunnel to 203.0.113.10 down (peer timeout)
02:14:09 ike: retry 1 failed
02:14:15 ntp: clock synchronized
02:14:20 dhcp: lease renewed for 192.0.2.57
02:14:30 ike: keepalive to 198.51.100.7 ok
02:14:40 ike: keepalive to 198.51.100.7 ok
02:14:52 ike: ERROR tunnel to 203.0.113.10 down (peer timeout)
02:14:55 ike: retry 2 failed
02:15:00 ike: keepalive to 198.51.100.7 ok"""


def around(lines, word, before=2, after=1):
    """Keep only the lines near a match, with their original line numbers."""
    keep = set()
    for i, line in enumerate(lines):
        if word in line:
            keep.update(range(max(0, i - before), min(len(lines), i + after + 1)))
    excerpt, previous = [], None
    for i in sorted(keep):
        if previous is not None and i > previous + 1:
            excerpt.append("    [lines omitted]")
        excerpt.append(f"{i + 1:>2}: {lines[i]}")
        previous = i
    return excerpt, len(keep)


lines = log.splitlines()
excerpt, kept = around(lines, "ERROR")
print(f"Kept {kept} of {len(lines)} lines:")
print("\n".join(excerpt))

Kept 8 of 14 lines:
 4: 02:14:00 ike: keepalive to 203.0.113.10 ok
 5: 02:14:05 ike: keepalive to 203.0.113.10 no reply
 6: 02:14:07 ike: ERROR tunnel to 203.0.113.10 down (peer timeout)
 7: 02:14:09 ike: retry 1 failed
    [lines omitted]
10: 02:14:30 ike: keepalive to 198.51.100.7 ok
11: 02:14:40 ike: keepalive to 198.51.100.7 ok
12: 02:14:52 ike: ERROR tunnel to 203.0.113.10 down (peer timeout)
13: 02:14:55 ike: retry 2 failed

An excerpt like this is still raw data: remove passwords, tokens, customer names and anything else your policy forbids before it goes into a prompt. Prompts for IT Work shows more ways to prepare logs and tickets.

Delimiters are not a security boundary

Marking material off helps against a specific risk: text inside a document, email or web page that tries to give the model instructions (indirect prompt injection). Telling the model that the content of the <email> tags is data from a customer, never instructions to follow, is one of the mitigations OWASP lists: segregate and clearly identify external content.

It reduces the risk; it does not remove it. A model can still be swayed by well-crafted text inside the tags. Anything that matters (sending email, changing a ticket, running a command) needs controls outside the prompt: limited permissions, human approval, and treating the model's output as untrusted. See AI Security and Prompt Security and Responsible Use.