In this article
OpenAI prompt caching reuses processing state for unchanged tokens at the beginning of a request. It does not reuse a previous answer: the model still processes new input and generates a new response (OpenAI prompt-caching guide).
For applications that repeatedly send instructions, conversation history, tools, schemas, or images, the key boundary is the shared request prefix. Stable content at the beginning may be reusable. A changed timestamp, reordered tool, revised schema, or rewritten message can shorten the match from the first changed token onward.
This is a documentation-based explanation; no independent API testing was performed.
How prefix matching works
Consider two requests:
Request 1:
[developer instructions]
[reference material]
[tool definitions]
[user question A]
Request 2:
[developer instructions]
[reference material]
[tool definitions]
[user question B]
The instructions, reference material, and tool definitions form an unchanged prefix. The user questions differ only after that shared content.
Putting a dynamic value first changes the structure:
Request 1:
[timestamp A]
[developer instructions]
...
Request 2:
[timestamp B]
[developer instructions]
...
These requests diverge immediately. OpenAI recommends placing stable developer instructions and shared reference material first, with timestamps, user-specific values, and other dynamic content later (current guide).
A cache breakpoint marks the end of a prefix that can be saved. Later requests search eligible breakpoints for the longest available matching prefix, working backward until they find a match. An early change can therefore prevent reuse after that point without necessarily eliminating every shorter eligible match (current guide).
Matching structure does not guarantee a cache hit. A matching cache entry must also be available.
What participates in the prefix?
| Request element | What should remain stable | What can shorten or prevent a match |
|---|---|---|
| Instructions and reference material | Shared text and its position near the beginning | Editing early text or inserting dynamic values before it |
| Conversation history | Earlier messages, tool calls, and results | Rewriting, summarizing, compacting, or truncating prior turns |
| Tools | Names, descriptions, schemas, instructions, and ordering | Removing, reordering, renaming, or redefining tools |
| Structured Outputs | text.format instructions and the requested schema |
Changing the format configuration or schema |
| Images | In the older example, the images, their ordering, and detail setting |
Changing an image or its detail value within the shared prefix |
| Request configuration | Settings that affect instructions or context | Changes to model, parallel tool calls, reasoning effort, verbosity, or compaction |
The Prompt Caching 201 cookbook describes messages, images, audio, tool definitions, and structured-output schemas as content that can participate in the request prefix (Prompt Caching 201). That older cookbook is not a current compatibility matrix for every model and endpoint.
Append conversation turns
In a multi-turn application, retain the existing conversation and append each new message, tool call, or tool result:
Turn 1: instructions + user message
Turn 2: previous content + assistant reply + new user message
Turn 3: previous content + tool call + tool result + new message
This preserves the earlier sequence and allows the reusable prefix to grow. Both the current guide and the older Prompt Caching 101 example recommend appending conversation elements rather than modifying earlier ones (current guide; Prompt Caching 101).
Summarization, compaction, and truncation may still serve context-management goals, but they can replace earlier content and reset prefix reuse from the changed point (current guide).
Keep tools and schemas stable
Tool definitions participate in the prefix, including their order. Requests containing the same tools in different orders no longer have identical content at that location (Prompt Caching 101).
Where supported, the current guide suggests changing tool availability without changing the supplied tool list:
- Set
tool_choiceto"none"instead of removing tool definitions. - Use
allowed_toolsto restrict which supplied tools are callable. - Preserve tool names, descriptions, parameter schemas, instructions, and ordering.
These options and their effects are described in the current guide.
Structured Outputs have the same structural concern. text.format adds output-format instructions and the requested schema to the request, so changing either can alter the prefix (current guide).
The supplied documentation does not define canonicalization rules. Do not assume that semantically equivalent tools or schemas will match when serialized differently.
Treat image guidance as an older example
Prompt Caching 101 says that linked and base64-encoded images in user messages can qualify for caching. It also says the image detail parameter must remain consistent because it affects tokenization (Prompt Caching 101).
That example reflects GPT-4o-mini-era behavior. It illustrates the importance of consistent image details but does not establish image support for every current model or API.
Separate GPT-5.6 controls from earlier behavior
For GPT-5.6 and later, the current guide documents:
- A minimum cacheable prefix of 1,024 tokens; hidden OpenAI system content does not count toward that minimum.
prompt_cache_options.modeandprompt_cache_breakpointfor controlling breakpoints.- An optional
prompt_cache_keyfor separate customer cache accounting.
These controls are specific to the current GPT-5.6-and-later guidance (current guide). An explicit breakpoint can be placed after stable content so that a changing tail does not become part of that saved prefix.
For models before GPT-5.6, minimum cacheable length varies by request settings. The current guide recommends a stable prompt_cache_key as routing guidance for requests that share a reusable prefix (current guide).
A key does not guarantee routing stickiness or a cache hit. Prompt Caching 201 describes routing as load-balanced and caching as best-effort (Prompt Caching 201).
Check usage rather than assuming reuse
The documented cache-read indicator is cached_tokens. In the older Chat Completions example, it appears at:
usage.prompt_tokens_details.cached_tokens
That published GPT-4o-mini example reported zero cached tokens on its first request and 1,024 on the next after extending the conversation history (Prompt Caching 101). This is an illustrative historical result, not a benchmark or promised outcome.
The current guide also names cache_write_tokens for comparing cache writes, although the supplied material does not establish its complete response-object path across every API.
The practical rule is straightforward: keep reusable content unchanged at the beginning, append changing content afterward, use controls appropriate to the model generation, and inspect usage fields instead of assuming a hit.
No comments yet