Transactional email and SMS
Sign-up, receipt and reset messages are queued by the web app and sent by a pool of workers.
Core dataBackground work
Job Queues
FIFO queues that do not lose jobs when a worker dies, with producers and consumers running in parallel.
- LPUSH/s pipelined, measured
- 31.6M
- jobs lost on a crash
- 0
- polling
- 0
Step 1 of 5
Producers push jobs
Any service adds work without waiting for consumers.
Example sessionplaying
# Producers push jobs
hotkv> LPUSH queue:emails "{\"to\":\"ada@example.com\",\"tpl\":\"welcome\"}"
(integer) 18
An animated walkthrough with real HotKV commands and example replies. Click any step to jump to it.
The problem
Background work such as emails, image resizing and webhooks needs a queue between the code that creates it and the code that does it. A queue that loses a job when a worker crashes mid-task is worse than no queue, because nobody notices the job is gone.
How HotKV handles it
Lists give you a simple FIFO queue. BLMOVE moves each job onto a processing list as it is handed to a worker, so an in-progress job is always stored somewhere if that worker disappears. Producers and consumers are served by parallel worker threads.
Step by step
What happens, command by command
- 01
Producers push jobs
Any service adds work without waiting for consumers.
LPUSH queue:emails "{\"to\":\"ada@example.com\",\"tpl\":\"welcome\"}" - 02
A worker claims a job safely
BLMOVE blocks until a job is available, then moves it to a processing list in one step.
BLMOVE queue:emails queue:emails:processing RIGHT LEFT 0 - 03
Acknowledge when done
The email went out. The job is removed from the processing list.
LREM queue:emails:processing 1 "{\"to\":\"ada@example.com\",\"tpl\":\"welcome\"}" - 04
A worker crashes
Worker 2 dies mid-job. Its job is still in the processing list, not lost.
LRANGE queue:emails:processing 0 -1 - 05
Requeue and carry on
A monitor moves the abandoned job back to the queue and another worker picks it up.
LMOVE queue:emails:processing queue:emails LEFT RIGHT
Where it fits
Real-world scenarios
Media processing
Uploads are queued for thumbnailing and transcoding, and a crashed transcoder never drops a file.
Webhook delivery
Outgoing webhooks are queued and retried without blocking the request that triggered them.
Built in
What you get
FIFO with blocking reads
LPUSH with BRPOP, and workers sleep until work arrives.
LPUSHBRPOPNo lost jobs
BLMOVE hands each job to a processing list atomically.
BLMOVELREMParallel producers and consumers
Producers and consumers are served by parallel worker threads.
Need more?
For acknowledgements, consumer groups and replay, use streams.
XADDXREADGROUPCommon questions
What happens to a job if a worker crashes?
If you claim jobs with BLMOVE, the job sits in the processing list. A monitor can move anything that has been there too long back onto the queue.
Do workers have to poll?
No. BRPOP and BLMOVE block until a job arrives, with an optional timeout.
Lists or streams?
Lists are the simplest queue. Streams add consumer groups, per-message acknowledgement and history. See the event streaming use case.
Related use cases
Try it on your own workload
HotKV speaks the Redis protocol, so you can point an existing client at it and run these same commands.