AI SaaS products
Each plan includes a monthly allowance of tokens. The budget is enforced at the moment of each call, not discovered on the invoice.
AI and LLMLLM operationsPerformance and Scale plans
LLM Cost Governance
Enforce a token and spend budget per user or tenant, see real spend per bucket, and embed identical text only once.
- charge or refuse
- Atomic
- limits per window
- Tokens + $
- embedding per unique text
- 1x
Step 1 of 6
Set a budget
2 million tokens and 50 USD over 30 days for one tenant.
Example sessionplaying
# Set a budget
hotkv> LLMBUDGET.SET tenant:acme 2000000 50 2592000
OK
An animated walkthrough with real HotKV commands and example replies. Click any step to jump to it.
The problem
LLM costs are usage-based, and one loop in an agent can spend a month of budget in an afternoon. Checking usage after the fact only tells you what you already lost, and a budget check that is not atomic can be overrun by concurrent requests.
How HotKV handles it
HotKV keeps a budget for each user or tenant and charges it atomically, so a request is either allowed and charged or refused before the model is called. Separate statistics record real tokens, cost and latency per bucket, and embeddings are stored by content hash so identical text is embedded once.
Step by step
What happens, command by command
- 01
Set a budget
2 million tokens and 50 USD over 30 days for one tenant.
LLMBUDGET.SET tenant:acme 2000000 50 2592000 - 02
Charge before you call
Allowed and charged in one step. Then the model is called.
LLMBUDGET.CONSUME tenant:acme 1850 0.004 - 03
Record the real usage
Input, output and cached tokens, cost and latency of the call, per bucket.
LLMSTATS.RECORD tenant:acme 1200 650 0 0.004 840 - 04
A runaway loop hits the cap
The tenant's agent keeps going. Once the next call would go over the limit, it is refused and not charged.
LLMBUDGET.CONSUME tenant:acme 90000 4.90 - 05
Other tenants are unaffected
Every tenant has its own budget.
LLMBUDGET.CONSUME tenant:beta 900 0.002 - 06
Check where a tenant stands
Usage, limits, time left in the window and how many requests were refused.
LLMBUDGET.STATUS tenant:acme
Where it fits
Real-world scenarios
Internal AI platforms
Each team has a budget. A forgotten notebook that loops overnight stops at the cap.
Document processing
Contract chunks repeat across versions. Deduplicated embeddings mean a revised document only pays for the paragraphs that changed.
Built in
What you get
Atomic budget checks
A request is charged or refused in one step, so concurrent agents cannot overrun the cap together.
LLMBUDGET.CONSUMETokens and dollars
Limit by token count, by cost, or both, over a window you set.
LLMBUDGET.SETLLMBUDGET.STATUSReal spend per bucket
Record input, output and cached tokens, cost and latency, and read them back.
LLMSTATS.RECORDLLMSTATS.GETEmbeddings stored once
Content-hash deduplication means identical text is only embedded once.
EMBED.DEDUP.PUTEMBED.DEDUP.GETCommon questions
What happens when a tenant is over budget?
LLMBUDGET.CONSUME returns 0 and does not charge. Your application then declines the request or falls back to a cheaper path.
Can I limit by cost as well as tokens?
Yes. A budget has a token limit and a cost limit in US dollars, over the window you choose.
Which plans include it?
LLM budgets, statistics and embedding deduplication are part of the Performance and Scale plans.
Related use cases
Try it on your own workload
HotKV speaks the Redis protocol, so you can point an existing client at it and run these same commands.