AI and LLMAgentsPerformance and Scale plans

Agent Tool Call Caching

Agents call the same tools with the same arguments over and over. Memoize the results and stop paying for the same API call twice.

lookup before each tool call: 1
per namespace or entry: TTL
saved, reported by STATS: $

Step 1 of 5

Create the namespace

Weather changes slowly, so results live for 15 minutes.

Example sessionplaying

# Create the namespace

hotkv> TOOL.CACHE.CREATE tools MAX_ENTRIES 100000 TTL 900

An animated walkthrough with real HotKV commands and example replies. Click any step to jump to it.

The problem

Agents loop. They look up the same weather, the same exchange rate, the same SQL result and the same web page several times in one run and again in the next. Each call costs latency, rate limit and often money, and slow tools make every agent step slow.

How HotKV handles it

The tool cache memoizes deterministic tool calls under a key built from the tool name and its arguments. Before calling a tool, the agent asks the cache. A hit returns the stored result immediately, a miss calls the tool once and stores the result with a TTL that matches how fresh it needs to be.

Step by step

What happens, command by command

01
Create the namespace
Weather changes slowly, so results live for 15 minutes.
TOOL.CACHE.CREATE tools MAX_ENTRIES 100000 TTL 900
02
First call, a miss
Nothing cached for these arguments yet.
TOOL.CACHE.GET tools "get_weather(city=Lisbon)"
03
Call the tool once and store it
The real API is called, and the result is cached for the next 15 minutes.
TOOL.CACHE.SET tools "get_weather(city=Lisbon)" "{\"temp_c\":21,\"sky\":\"clear\"}" TTL 900
04
Every repeat is a hit
The agent plans a second day trip and asks again. So do other agents for other travellers.
TOOL.CACHE.GET tools "get_weather(city=Lisbon)"
05
Count the savings
Hits, misses and the usage saved are reported per namespace.
TOOL.CACHE.STATS tools

Where it fits

Real-world scenarios

Travel and booking agents

Weather, currency and availability lookups repeat across travellers and planning steps. Each result is fetched once per TTL.

Data agents

Read-only SQL queries and dashboard metrics are memoized, so an agent exploring the data does not hammer the warehouse.

Browsing agents

Fetched and parsed web pages are cached, so the same page is not downloaded again in every run.

Built in

What you get

Memoize by arguments

Keys are built from the tool name and its arguments, so each distinct call is cached separately.

TOOL.CACHE.GETTOOL.CACHE.SET

Freshness control

A default TTL per namespace and an optional TTL per entry.

TTL

Bounded memory

An entry limit per namespace, with eviction when it is full.

MAX_ENTRIES

Savings report

Hits, misses, evictions, memory and usage saved.

TOOL.CACHE.STATS

Reference:TOOL.CACHE.CREATE TOOL.CACHE.GET TOOL.CACHE.SET TOOL.CACHE.STATS

Common questions

Which tools should I cache?

Deterministic, read-only calls, such as lookups, searches and queries whose result only depends on the arguments. Never cache tools that change something, such as sending an email.

Can a cached result be removed early?

Yes. TOOL.CACHE.DEL removes one entry and TOOL.CACHE.FLUSH empties a namespace.

Which plans include it?

The tool cache is part of the Performance and Scale plans.