HTTP Caching

Why caching

An HTTP cache keeps copies of responses so that a later request for the same resource can be answered from the copy instead of from the server. The rules are defined in RFC 9111 (2022). Caching pays off in three ways:

• Latency: a copy on the user's own disk or in a CDN edge server nearby arrives in milliseconds instead of a round trip across the world (see latency).
• Load: every request answered by a cache is one the application servers and database do not see. A traffic spike on a popular page becomes harmless.
• Cost: less outgoing data from your servers or cloud account, and fewer servers needed.

The price is the risk of serving something old. The whole art of HTTP caching is telling each cache exactly how long a copy may be used, and how to check whether it is still current.

Private and shared caches

The difference matters for anything personal. A user's own bank statement may sit in that user's browser cache; it must never sit in a CDN cache, where the next visitor could receive it. Shared caches are also the ones you control least directly: you can clear your browser in a second, but a copy at hundreds of CDN edge locations stays there until it expires or you purge it.

Cache-Control

Cache-Control is the HTTP header in which the server tells caches whether a response may be stored, by whom, for how long, and what to do once it is old. It holds a comma-separated list of directives:

Cache-Control: public, max-age=3600, stale-while-revalidate=60

Without any Cache-Control or Expires header, caches may still store a response and guess a freshness lifetime — typically a fraction (often 10%) of the time since its Last-Modified date. That guessing is a frequent cause of "random" stale content, so set Cache-Control explicitly on everything you serve. The older Expires header (an absolute date) is ignored when max-age is present.

no-cache versus no-store

The two names suggest the opposite of what one of them does, and older material often mixes them up:

• no-cache does not forbid caching. The copy is stored, but each time it is needed the cache asks the server "is this still current?" Usually the answer is a tiny 304 Not Modified, so the body is not transferred again. Use it for HTML pages and API responses that must always be current.
• no-store forbids storing anything. Every request fetches the full response again. Use it for sensitive data — account pages, bank statements, medical records — that should not stay on disk, for example on a shared computer.

Freshness and validation

A cached response is fresh while its age is below its freshness lifetime (max-age), and stale after that. A fresh copy is used without contacting the server. A stale copy is not necessarily wrong — it may simply not have been checked recently. Cache validation is that check: the cache asks the server whether its stored copy is still current, and the server either confirms it (without resending the body) or sends the new version.

ETag and Last-Modified

To make validation possible, the server attaches a validator to each response:

• An ETag (entity tag) is an opaque identifier for one version of a resource, for example a hash of its content: ETag: "7d3dc5d8f9e8d5c2". Any change to the content gives a new ETag. A tag starting with W/ is a weak ETag: it promises the content is equivalent, not byte-for-byte identical.
• Last-Modified is the date and time the resource last changed, for example Last-Modified: Wed, 30 Sep 2026 08:00:00 GMT. It works with one-second precision, so ETags are more reliable.

Conditional requests and 304 Not Modified

A conditional request is a request that the server should carry out only if a condition about the resource's current version holds. For revalidation, the cache sends the stored validator back: If-None-Match with the ETag, or If-Modified-Since with the Last-Modified date. If the resource has not changed, the server answers 304 Not Modified with headers but no body, and the cache keeps using its copy as fresh again. If it has changed, the server sends a normal 200 OK with the new version.

Conditional requests have a second use in APIs. A client that wants to update a resource sends If-Match with the ETag it last read; if someone else changed the resource in the meantime, the ETag no longer matches and the server refuses with 412 Precondition Failed instead of silently overwriting the other change (see REST APIs).

An ETag and a 304, from a local server

This script starts a small local server (Python's http.server, meant for learning, not for production) that serves a maintenance notice with an ETag and Cache-Control: no-cache. The client plays the cache: it fetches the notice, revalidates it, and revalidates again after the notice has changed.

import hashlib
import http.client
import threading
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer

notice = {"text": "No planned maintenance this week.\n"}


class Notices(BaseHTTPRequestHandler):
    def do_GET(self):
        body = notice["text"].encode()
        etag = '"' + hashlib.sha256(body).hexdigest()[:16] + '"'
        # Simplified: real servers also accept lists and weak (W/) ETags
        if self.headers.get("If-None-Match") == etag:
            self.send_response(304)          # your copy is still good
            self.send_header("ETag", etag)
            self.send_header("Cache-Control", "no-cache")
            self.end_headers()
            return
        self.send_response(200)
        self.send_header("ETag", etag)
        self.send_header("Cache-Control", "no-cache")
        self.send_header("Content-Type", "text/plain; charset=utf-8")
        self.send_header("Content-Length", str(len(body)))
        self.end_headers()
        self.wfile.write(body)

    def log_message(self, *args):
        pass


server = ThreadingHTTPServer(("127.0.0.1", 0), Notices)
threading.Thread(target=server.serve_forever, daemon=True).start()


def get(etag=None):
    conn = http.client.HTTPConnection("127.0.0.1", server.server_port)
    headers = {"If-None-Match": etag} if etag else {}
    conn.request("GET", "/notices", headers=headers)
    resp = conn.getresponse()
    body = resp.read()
    print(resp.status, resp.reason)
    for name in ("ETag", "Cache-Control"):
        print(f"  {name}: {resp.getheader(name)}")
    print(f"  body: {len(body)} bytes {body.decode()!r}")
    conn.close()
    return resp.getheader("ETag")


print("1. First visit, nothing cached")
etag = get()
print("2. Revalidate with If-None-Match")
get(etag)
print("3. The notice changes, then revalidate again")
notice["text"] = "Planned maintenance: Saturday 02:00-04:00 UTC.\n"
get(etag)

server.shutdown()

1. First visit, nothing cached
200 OK
  ETag: "7d3dc5d8f9e8d5c2"
  Cache-Control: no-cache
  body: 34 bytes 'No planned maintenance this week.\n'
2. Revalidate with If-None-Match
304 Not Modified
  ETag: "7d3dc5d8f9e8d5c2"
  Cache-Control: no-cache
  body: 0 bytes ''
3. The notice changes, then revalidate again
200 OK
  ETag: "699e46494df99c3b"
  Cache-Control: no-cache
  body: 47 bytes 'Planned maintenance: Saturday 02:00-04:00 UTC.\n'

Request 2 gets 304 Not Modified and zero bytes of body: the cache may keep using the 34 bytes it already has. Request 3 sends the same old ETag, but the content has changed, so the server answers 200 with the new body and a new ETag, which the cache stores for next time. With a large page or image, that difference between a 304 and a 200 is most of the transfer.

The Vary header

A cache normally stores one response per URL. But some servers send different responses for the same URL depending on the request: compressed or not, in Dutch or in English, with or without CORS permission for a particular origin. The Vary header lists the request headers a response depends on, so that a cache stores a separate copy for each combination and reuses a copy only for a request with the same values:

Content-Encoding: br
Vary: Accept-Encoding

• Vary: Accept-Encoding: a brotli-compressed copy must not go to a client that cannot decompress it. Make sure your web server adds it when it compresses: Apache's mod_deflate does so automatically, while Nginx needs gzip_vary on;.
• Vary: Origin: needed when Access-Control-Allow-Origin depends on the request's origin, so that one site's CORS permission is not served to another (see CORS response headers).
• Vary: Accept-Language, Vary: Cookie: technically possible, but every distinct value creates another copy, so caches holding these are often nearly useless. Many CDNs simply do not cache such responses. Separate URLs (such as /nl/ and /en/) cache much better.

Cache busting

Long cache lifetimes and fast updates seem to contradict each other. Cache busting resolves that by changing a file's URL whenever its content changes, so that caches treat the new version as a different resource and never serve the old copy for it. The common form is a fingerprint — a short hash of the content — in the file name, added by the build tool:

/assets/app.3f9a1c7e.css     Cache-Control: public, max-age=31536000, immutable
/assets/app.b81d04aa.js      Cache-Control: public, max-age=31536000, immutable
/index.html                  Cache-Control: no-cache

The fingerprinted files can be cached for a year with immutable, because that exact name will never have other content. The HTML that refers to them uses no-cache: it is revalidated on every visit, and after a release it points to the new file names, which no cache has seen yet. Users get the new version on their next page load, while unchanged files still come from cache.

A query string such as app.css?v=42 also works with browsers and most CDNs, but some caches ignore or strip query strings, and someone must remember to change the number. Fingerprints in the file name are more reliable.

CDN purging

When content without a fingerprint changes — a press release that contained an error, an image replaced under the same name — you can ask the CDN to purge (invalidate) its copies: by URL, by path prefix, by tag on some CDNs, or everything. CDNs offer this in their console and API, so a deployment pipeline can purge automatically.

"Why do I still see the old version?"

After a release, someone reports the old page or an old script. Work from the user toward the server:

1. The browser. Reload while bypassing the cache (Ctrl+Shift+R, Cmd+Shift+R on a Mac), or open a private window. In the Network panel of the developer tools, the size column shows "(disk cache)" or "(memory cache)" for responses that never left the browser, and status 304 for revalidations. "Disable cache" helps while testing.
2. A service worker. A site with offline support may have a service worker that serves its own cached copies, even when you bypass the HTTP cache. Check the Application panel.
3. The CDN or proxy. Look at the response headers: Age (how many seconds a shared cache has held the copy) and the CDN's cache status header (such as X-Cache: HIT; names differ per CDN). Compare with a request that goes directly to the origin.
4. The servers. Behind a load balancer, did the release reach every server? One server still on the old version gives "sometimes old, sometimes new". Also check caches inside the application itself.
5. DNS. After a move to new servers, some resolvers may still point to the old ones until the record's TTL runs out (see TTL and caching).

$ curl -sI https://portal.example.com/assets/app.css

HTTP/2 200
content-type: text/css
cache-control: public, max-age=86400
etag: "5e1b0c9a"
last-modified: Mon, 28 Sep 2026 14:02:11 GMT
age: 51230
x-cache: HIT

Here a CDN has held this copy for about 14 hours (age: 51230) and may keep serving it for the rest of its 24-hour lifetime. The file has no fingerprint, so a release that changed app.css will not be visible until the copy expires or is purged: a reason to switch to fingerprinted names.

Caching and private data

The worst caching failure is not stale content but someone else's content: a shared cache stores a page made for one logged-in user and serves it to the next visitor — with their name, address or account balance. It happens when personalized responses carry cache headers meant for public pages, or when a CDN is configured to "cache everything" on a path that turns out to include account pages.

• Mark every personalized response Cache-Control: private (the user's browser may keep it) or no-store (nobody keeps it). Use no-store for sensitive data.
• Shared caches must not store responses to requests with an Authorization header unless the response explicitly allows it (for example with public or s-maxage). Do not add those directives to API responses that contain user data.
• Be careful with responses that set a cookie (Set-Cookie): a cached copy hands that cookie to everyone who receives the copy. Configure the CDN not to cache them.
• Keep public and personal content on clearly separated paths or host names, so that CDN rules cannot accidentally match account pages. Attacks that trick a cache into storing private pages are covered in Web Application Security.