Redis Crash & Cache Stampede
Simulate what happens when Redis crashes: cache timeouts, database fallback, cache stampede pressure, rising latency, 5xx errors, and recovery after Redis restarts cold.
Continue with the guided workspace
Use the paired workspace to connect this guide's mental model with live component state, visible signals, and the outcome each step or scenario is meant to prove.
When a caching layer fails or restarts cold, the requests it used to absorb fall straight through to the backing database. If thousands of clients query for the same expired or missing data at once, they trigger a cache stampede — identical queries overwhelm the database, connection pools fill, latency spikes, and the whole system can degrade.
Mental Model
Imagine a busy e-commerce site where the homepage product list is cached in Redis. Under normal operation, Redis handles 99% of requests instantly. If Redis crashes, the application falls back to querying the database directly. If thousands of clients query the database at the same time for the same expired or missing data, they trigger a Cache Stampede (or “thundering herd”). A single-flight or request-coalescing guard can let one request refill the key while peers wait. The database is overwhelmed with identical queries, connection pools fill up, request latency skyrockets, and the entire system can enter a degraded state where even simple operations fail.
Components and Why They Exist
Clients
Represent users generating application requests. Generates traffic load and measures client-side request latencies. In the demo, watch how client traffic is routed and how their experienced latency increases when the caching layer goes offline.
Application Server
The compute layer that reads from the cache and falls back to the database on cache misses. Coordinates cache-aside retrieval logic and handles database queries on cache misses. In the demo, observe the application server waiting for database connections and queries to return under a stampede.
Redis Cache
Provides low-latency in-memory query interception. Stores active cache entries and handles the majority of read queries when healthy. In the demo, watch it intercept traffic when healthy (status UP) and force requests to fall through to the database when crashed (status DOWN).
Database
Persistent database source of truth. Handles query requests on cache misses and records writes. In the demo, watch the Database Load and Queue Size metrics rise when Redis crashes or restarts cold.
Edge Cases and Misconceptions
Cold Restart vs. Simple Recovery
Restarting Redis resolves the “crashed” state but does not immediately fix latency. Because the cache starts completely empty (“cold”), the system is still vulnerable to a stampede until popular keys are re-written to the cache.
Increasing Database Capacity
A common misconception is that scaling the database is the best solution for cache stampedes. However, scaling database resources is expensive and does not prevent the underlying thundering herd behavior; it only raises the threshold before the database crashes.
Throttling vs. Load Shedding
During recovery, reducing client request rates (throttling) can help the database catch up and allow the cache to warm up. In the simulator, check how the “Recovery Throttling” scenario lowers the request rate to stabilize the database during warming.
Common Questions
What is a Cache Stampede?
A cache stampede (also called dog-piling or thundering herd) occurs when a popular cache key expires or is invalidated, and multiple concurrent application requests simultaneously realize the cache is empty. They all fall through to query the database and attempt to rebuild the cache, causing a sudden, massive spike in database load and latency.
How does a cold Redis restart aggravate database latency?
When Redis restarts, its memory is empty (a cold cache). As traffic resumes, every single incoming query results in a cache miss. The application servers are forced to route all requests to the database, creating a cache stampede across many keys at once and leading to connection pool exhaustion, elevated query latency, and potentially cascading 5xx server errors.
What is the difference between Cache-Aside and Cache stampede dynamics?
In a Cache-Aside pattern, the application checks the cache, queries the database on a miss, and writes the value back to the cache. While simple, it has no built-in concurrency controls; if a key with high request volume expires, many threads will concurrently miss the cache and hit the database before the first thread can write the calculated value back, resulting in a stampede.
How do Mutex locks prevent cache stampedes?
A Mutex lock forces only the first thread that misses the cache to acquire a distributed lock, query the database, and write the value back. All other threads waiting for the same key either block, poll the cache periodically, or return a slightly stale/default value until the lock is released and the cache is repopulated.
What is Probabilistic Early Expiration (XFetch)?
XFetch is an algorithm that probabilistically recomputes and updates a cache entry before it actually expires. As a key nears its TTL, the probability of an application thread triggering an early background update increases based on the request rate and calculation time. This ensures the cache is continuously refreshed by a single background request, preventing expirations that trigger stampedes.