Prompt caching vs semantic caching: what each one saves
Prompt caching makes a repeated prefix cheaper. Semantic caching removes the model call. How each works, what we measured and which to use when.
Tag
Prompt caching
Engineering.Measured and explained.
Provider prompt caching and response caching for LLM apps, what each one saves on tokens and cost, and where each one stops helping.
2 articles in Prompt caching
Articles
Ideas. Experiments. Updates.
HotKV cut Claude token use by 70% in our Banking77 benchmark
Four Claude models, 3,000 support requests each: HotKV caching cut model calls, tokens and API cost by about 70%. Results and answer-quality trade-offs.
Follow your curiosity