When the Cache Goes Down

but let’s say due to some reason redis is down.

Now all our calls will go to the DB?

How can we mitigate this, and what are the different ways ot handle this kind of scenario

When a distributed cache like Redis goes down in a high-read system, the resulting flood of traffic directly to the database is called a Cache Stampede (or Thundering Herd). Without safeguards, this sudden spike in active connections will immediately exhaust the database’s CPU and memory, causing a cascading system failure where the entire platform goes offline.

To mitigate this, you must implement defense mechanisms across multiple layers of your architecture.

1. Multi-Tier Caching (L1/L2)

Do not rely solely on a remote cache. Implement a localized, in-memory cache (L1) directly on your application servers (e.g., Caffeine in Java, LRU-cache in Node.js, or sync.Map in Go).

How it works: The application checks L1 first, then Redis (L2), then the database.

The mitigation: L1 caches usually have a very short Time-To-Live (TTL) of just 1 to 5 minutes. If Redis goes down, the application continues to serve reads directly from its local RAM. While the DB load will increase when L1 keys expire, the L1 cache absorbs the initial shock and prevents a simultaneous stampede.

2. Request Coalescing (Single-Flight Pattern)

If Redis fails and 10,000 users request the exact same missing piece of data (like a viral product page) at the exact same millisecond, the application should not execute 10,000 database queries.

How it works: You implement a concurrency lock at the application level. The system allows exactly one thread to execute the database query.

The mitigation: The other 9,999 concurrent requests are paused. Once the single database query returns, the application shares that single result with all 10,000 waiting requests simultaneously.

3. Circuit Breakers

Your application must recognize when the database is struggling and stop hitting it before it crashes entirely.

How it works: Wrap your database queries in a circuit breaker pattern (using libraries like Resilience4j or Polly).

The mitigation: If the database starts timing out or throwing connection errors due to the increased load, the circuit “opens” and blocks further database calls. The application instantly returns a 503 HTTP error or a safe fallback response (like a static default page) until the database has recovered enough to handle traffic again.

4. Strict Connection Pooling and Fail-Fast

A common mistake during a cache failure is allowing the application to queue up thousands of database requests indefinitely while waiting for an open connection.

How it works: Cap your database connection pool to a strict maximum limit that your database hardware can safely handle (e.g., 200 connections).

The mitigation: If all connections are busy, configure the application to fail fast. It should instantly reject new requests rather than putting them in a queue. Queued requests are dangerous because users will often refresh the page while waiting, doubling the pending load on the database for requests they have already abandoned.

5. Graceful Degradation (Load Shedding)

If the cache is down and the database is absorbing all the impact, you must dynamically reduce the complexity of the work the application is asking the database to do.

How it works: Identify core features (e.g., processing payments, logging in) versus non-core features (e.g., personalized recommendations, live view counters, complex filtering).

The mitigation: When the system detects high latency or a cache failure, it automatically disables the non-essential DB queries. The UI degrades gracefully (hiding the “recommended items” section), reserving all remaining database capacity to keep the critical paths functional.

**
**