AI and LLMLLM applicationsPerformance and Scale plans

Exact Prompt Caching

Never pay twice for the same prompt. An exact-match LLM cache that counts the tokens and dollars it saved you.

match, zero false hits: Exact
lookup latency in STATS: p50
savings tracked per namespace: $

Step 1 of 4

Create a namespace

Give it a TTL so answers refresh, and what a hit saves on average.

Example sessionplaying

# Create a namespace

hotkv> PCACHE.CREATE reports TTL 86400 COST_PER_HIT 0.012

An animated walkthrough with real HotKV commands and example replies. Click any step to jump to it.

The problem

Batch jobs, retries, scheduled reports and popular questions send the same prompt to the model again and again. Each one is billed in full, and nobody can say how much a cache would have saved because nothing measures it.

How HotKV handles it

The prompt cache stores the response for each exact prompt, together with the model and token counts of the call that produced it. Every later hit is counted, so HotKV can report the requests, tokens and dollars you did not spend.

Step by step

What happens, command by command

01
Create a namespace
Give it a TTL so answers refresh, and what a hit saves on average.
PCACHE.CREATE reports TTL 86400 COST_PER_HIT 0.012
02
Miss, then store with usage
The first call goes to the model. The answer is stored with the model and token counts that produced it.
PCACHE.SET reports "Summarize Q3 sales for EMEA" "EMEA grew 14% ..." MODEL claude-sonnet-5 INPUT 820 OUTPUT 140
03
Every repeat is free
The nightly job and every dashboard view ask the same thing. All of them are hits.
PCACHE.GETEX reports "Summarize Q3 sales for EMEA"
04
See what you saved
Statistics include the hit rate, lookup latency and the usage you did not pay for.
PCACHE.STATS reports

Where it fits

Real-world scenarios

Scheduled reporting

The same summaries are generated for every viewer of a dashboard. One model call per report per day instead of one per view.

Translation and localisation

UI strings and product descriptions are translated once per language and served from the cache on every request after that.

Classification pipelines

Tickets, reviews and emails with identical text get the same label without another model call.

Built in

What you get

Exact match

A hit only for the identical prompt, so a cached answer is never served for a different question.

PCACHE.GETPCACHE.SET

Usage-aware

Store the model and token counts with each answer so savings are computed from real usage.

MODELINPUTOUTPUT

Metadata and hit counts

PCACHE.GETEX returns the answer with its metadata and how often it has been served.

PCACHE.GETEX

Hit rate and latency

PCACHE.STATS reports hits, misses, evictions, memory, lookup latency and savings.

PCACHE.STATS

Reference:PCACHE.CREATE PCACHE.SET PCACHE.STATS

Common questions

When should I use the prompt cache rather than the semantic cache?

Use the prompt cache when prompts repeat exactly, such as batch jobs and fixed templates. Use the semantic cache when people word the same question differently. Many applications check both.

Is this the same as the provider's prompt caching?

No. Provider prompt caching discounts repeated prefixes on their side. A HotKV hit never reaches the provider at all, so the two add up.

Which plans include it?

The prompt cache is part of the Performance and Scale plans.