Web and app analytics
Daily and monthly unique users per page or feature, merged across days with PFMERGE.
Core dataAnalytics
Counting at Scale with Probabilistic Structures
Unique visitors, "have we seen this?", top items and frequencies over billions of events, each in a small, fixed amount of memory.
- probabilistic structures
- 4
- memory per structure
- Fixed
- error bounds
- Known
Step 1 of 5
Unique visitors
Add visitor ids to a HyperLogLog. Memory stays small however many arrive.
Example sessionplaying
# Unique visitors
hotkv> PFADD visitors:2026-10-16 user:42 user:77 user:91
(integer) 1
An animated walkthrough with real HotKV commands and example replies. Click any step to jump to it.
The problem
Exact answers to "how many unique users", "have we sent this email before" or "what are the top searches" need memory that grows with every distinct item. At billions of events that becomes gigabytes of sets and counters per question.
How HotKV handles it
Probabilistic structures trade a small, known error for constant memory. HyperLogLog counts distinct items, Bloom filters answer membership, Top-K tracks the most frequent items and Count-min sketches estimate how often any item occurred. All four are built into HotKV.
Step by step
What happens, command by command
- 01
Unique visitors
Add visitor ids to a HyperLogLog. Memory stays small however many arrive.
PFADD visitors:2026-10-16 user:42 user:77 user:91 - 02
How many?
An estimate of distinct visitors with a small standard error.
PFCOUNT visitors:2026-10-16 - 03
Never send twice
Before sending a campaign email, check whether this address already got it.
BF.EXISTS sent:campaign:fall ada@example.com - 04
Top searches
Top-K keeps the most frequent items as events stream in.
TOPK.LIST searches:today WITHCOUNT - 05
How often for any item
A Count-min sketch estimates the count of any item, listed or not.
CMS.QUERY pageviews /home /pricing /blog
Where it fits
Real-world scenarios
Email and notifications
Bloom filters stop duplicate sends across retries and campaigns.
Search and commerce
Top queries and most-viewed products are tracked live for merchandising.
Built in
What you get
HyperLogLog
Distinct counts, mergeable across buckets.
PFADDPFCOUNTPFMERGEBloom filters
Membership with a configurable false positive rate.
BF.RESERVEBF.ADDBF.EXISTSTop-K
The most frequent items in a stream.
TOPK.ADDTOPK.LISTCount-min sketch
Frequency estimates for any item.
CMS.INCRBYCMS.QUERYCommon questions
How accurate are they?
Bloom filters let you set the error rate, and Count-min sketches the width and depth. HyperLogLog has a small, fixed standard error.
Can a Bloom filter miss something it has seen?
No. It can report a false positive, never a false negative.
Do I need a module?
No. They are part of HotKV.
Related use cases
Try it on your own workload
HotKV speaks the Redis protocol, so you can point an existing client at it and run these same commands.