TL;DRContext (prompt) caching reuses repeated prompt prefixes to cut cost and latency.
If many requests share a long prefix (a big system prompt, a document), recomputing it every time is wasteful. Prompt/context caching stores the processed prefix so repeated calls skip that work — major cost and latency savings for chatbots and RAG with stable instructions. Structure prompts with the stable, cacheable part first and the variable part last to maximize cache hits.
Key points
Reuse processed repeated prompt prefixes
Big savings for stable system prompts/docs
Put stable content first, variable last
Helps chatbots and RAG
Common mistakes
Putting variable content before the stable prefix
Assuming caching with constantly-changing prompts
Ignoring provider cache rules/TTL
Try it: Explain how prompt structure affects cache hit rate.
Example code
<!doctype html><html><head><meta charset="utf-8"></head>
<body style="background:#06040d;color:#e6e0ff;font-family:monospace;padding:20px"><pre>[ stable system prompt + docs ] ← cache this
[ variable user question ] ← changes
stable-first → high cache hits → cheaper/faster</pre></body></html>