Quqo
How Can Cache Systems Fail?
The following diagram shows four typical cases where caches can fail and their solutions:
1. Cache Stampede
What is it?
● This occurs when a large number of cache keys expire simultaneously. As a result, all requests
go directly to the database, overwhelming it.
Solutions:
● Random expiration times: Avoid setting the same expiration time for all keys by adding a
random factor to the expiration configuration.
● Restrict database access: Only allow critical business data to reach the database, and block
non-essential data from hitting the database until the cache is restored.
2. Cache Penetration
What is it?
● This happens when a key doesn’t exist in either the cache or the database. In this case, the
system cannot retrieve the data to update the cache, leading to continuous hits to the database,
which increases load on both the cache and the database.
Solutions:
● Cache null values: Cache a null value for non-existent keys, avoiding repeated hits to the
database.
● Use a Bloom filter: A Bloom filter can check if a key exists before querying the database. If the
key doesn’t exist, the database won’t be queried.
3. Cache Breakdown
What is it?
● Similar to the cache stampede, this occurs when a frequently accessed (hot) key expires. Many
requests then hit the database, which can overwhelm it, as hot keys often handle a large portion
of requests.
Solutions:
● Avoid expiration for hot keys: Do not set an expiration time for hot keys that handle the majority
of queries, so they stay available in the cache.
4. Cache Failure
What is it?
● This occurs when the cache goes down entirely, forcing all requests to go directly to the
database, leading to an overload.
Solutions:
● Use a circuit breaker: Set up a circuit breaker to prevent requests from reaching the database
when the cache is down. This way, the application can temporarily halt until the cache is restored.
● Use a cache cluster: Implement a cache cluster to improve availability. If one cache instance
fails, another can take over, maintaining cache functionality.
In summary, cache systems can fail due to expiration overload, non-existent keys, expired hot keys, or
complete cache outages. Solutions include spreading out expiration times, caching null values, using
Bloom filters, avoiding expiration for critical keys, and improving reliability with cache clusters and circuit
breakers.
