AI and LLMLLM applicationsPerformance and Scale plans

AI Customer Support Assistant

A support bot that answers repeats instantly, matches paraphrases by meaning, grounds new answers in your help center, and never overspends.

AI engines in one server: 5
model call per new question: 1
extra databases to run: 0

Step 1 of 6

Check the tenant's budget

Each tenant's budget was set once with LLMBUDGET.SET. Before any model work, charge it atomically. A tenant over its cap is refused here, before money is spent.

Example sessionplaying

# Check the tenant's budget

hotkv> LLMBUDGET.CONSUME tenant:acme 900 0.002

An animated walkthrough with real HotKV commands and example replies. Click any step to jump to it.

The problem

A real support assistant is more than a prompt. It needs a cache so the same question is not paid for twice, retrieval over the help center, memory of the conversation, and a budget so one customer cannot run up the bill. That usually means four or five services, and more time on glue than on the product.

How HotKV handles it

HotKV runs every one of those pieces as commands in one server. The assistant checks the tenant's budget, answers exact repeats from the prompt cache and paraphrases from the semantic cache, retrieves help center passages for anything new, keeps the conversation, and caches each new answer for the next customer who asks.

Step by step

What happens, command by command

01
Check the tenant's budget
Each tenant's budget was set once with LLMBUDGET.SET. Before any model work, charge it atomically. A tenant over its cap is refused here, before money is spent.
LLMBUDGET.CONSUME tenant:acme 900 0.002
02
Exact repeat, answered at once
The same question asked word for word is a prompt cache hit. No embedding, no model call.
PCACHE.GET support "How do I reset my password?"
03
A paraphrase, matched by meaning
"I forgot my login password" is worded differently. Its embedding is close enough to a question in the semantic cache, created once with SCACHE.CREATE, so the stored answer comes back.
SCACHE.GET support 0.013 -0.21 0.88 ...
04
Something new, so retrieve context
A question nobody has asked yet. Hybrid search over the ingested help center returns the passages the model needs.
RAG.SEARCH helpdesk "refund for an annual plan" K 3 VECTOR 0.41,0.07,0.66
05
One model call, remembered
The model answers with the passages as context. The turn is appended to the conversation, opened with AMEM.CONV.CREATE when the chat started, so the next message has its history.
AMEM.CONV.APPEND chat:7781 assistant "Annual plans can be refunded within 30 days."
06
Cache it for the next customer
The new answer goes into the semantic cache, so every later paraphrase of this question is a hit.
SCACHE.SET support "Can I get a refund on the yearly plan?" "Annual plans can be refunded within 30 days." 0.41 0.07 ...

Where it fits

Real-world scenarios

SaaS help desk

Thousands of customers ask about passwords, invoices and plan limits. Many questions are repeats or paraphrases and are answered from cache. Only genuinely new ones reach the model.

E-commerce returns bot

Return and delivery policies are ingested once as a RAG pipeline. Answers are grounded in the current policy text and every shop on the platform has its own budget.

Internal IT assistant

Employees ask the same VPN and laptop questions every week. The assistant keeps each conversation, recalls earlier turns and stops a runaway loop before it burns the monthly allowance.

Built in