Caching · Commitwise

A product page needs four database queries and takes 120 ms. The same page, from a cache, takes 2 ms, and the database doesn’t notice it was asked. Caching is the cheapest performance win in system design, and the source of its most confusing bugs: prices that don’t update, one user seeing another’s account, and a database that falls over the moment the cache empties.

New to some of these? Open what you need, or skip ahead.

Basics:What is a cache?

A cache keeps a copy of data somewhere faster to reach than its original source. Instead of asking the database for the same popular product 1,000 times a second, the server keeps it in memory (or in a cache server like Redis) and answers from there.

Two questions come with every cache: how long may a copy be used before it might be out of date (its time to live, or TTL), and what happens when the original changes (the copy must be removed or updated).

Basics:How do HTTP requests work?

Apps and servers talk with HTTP. A request has a method that says what to do (GET to read, POST to create, PUT to replace, DELETE to remove), a URL (https://api.example.com/books/42), headers (extra information, like who you are) and sometimes a body (the data, usually JSON).

The server sends back a response with a status code that says how it went (200 OK, 201 created, 404 not found, 500 server error), headers, and usually a body. Codes starting with 4 mean the client asked for something wrong; codes starting with 5 mean the server failed.

Part Not started

Where caches live

Think of how you get milk. There’s a carton in your fridge (fastest), a corner shop down the street, a supermarket across town, and the dairy farm (the original source, slow to reach). Most days the fridge is enough; you only go further when it’s empty.

A request for data works the same way. Each layer between the user and the database can keep a copy, and the closer the copy, the faster the answer.

Each layer trades freshness and control for latency and offload. Client and CDN caches remove requests entirely (no server cost at all) but are the hardest to invalidate, so they’re controlled by HTTP semantics (Cache-Control, ETag, Vary). Server-side caches (in-process, then distributed) reduce database load under your direct control. The design question is which data can tolerate which staleness at which layer, and which data must never be shared.

Layers of caching

Tap a box to see what it does.

HTTP caching is how the server tells the app and the CDN what they may keep:

Cache-Control: max-age=300 means “you may reuse this response for 300 seconds without asking me”.
Cache-Control: private means “only the user’s own app or browser may keep it, never a shared cache like a CDN”. public allows shared caches.
Cache-Control: no-store means “don’t keep this anywhere” (for sensitive data).
An ETag is a fingerprint of the response. Next time, the client sends If-None-Match with it, and if nothing changed, the server answers 304 Not Modified with no body, saving bandwidth.

Spot the bugA profile that belongs to someone else

Tap the line with the bug.

@GetMapping("/me")
fun me(@AuthenticationPrincipal user: User): ResponseEntity<Profile> =
    ResponseEntity.ok()
        .cacheControl(CacheControl.maxAge(Duration.ofMinutes(5)).cachePublic())
        .body(profiles.load(user.id))

@GetMapping("/me")
ResponseEntity<Profile> me(@AuthenticationPrincipal User user) {
    return ResponseEntity.ok()
            .cacheControl(CacheControl.maxAge(Duration.ofMinutes(5)).cachePublic())
            .body(profiles.load(user.id()));
}

app.get("/me", requireAuth, async (req, res) => {
  res.set("Cache-Control", "public, max-age=300");
  res.json(await profiles.load(req.user.id));
});

/me returns a different response for every user, but it’s marked public, so the CDN stores Ana’s profile under the URL /me and serves it to everyone who asks for /me in the next five minutes, Ben included. That’s a data leak caused by a header.

Personalized responses must never be stored by shared caches:

.cacheControl(CacheControl.noStore())          // or maxAge(...).cachePrivate() for the user's own device

.cacheControl(CacheControl.noStore())          // or maxAge(...).cachePrivate() for the user's own device

res.set("Cache-Control", "private, no-store");

Configure the CDN to bypass the cache for authenticated requests by default, and to cache only paths you explicitly mark as public (/products/*, /images/*). When a public response varies by a header, like language, say so with Vary: Accept-Language, or each language overwrites the other.

Predict the output

The product list is served with Cache-Control: public, max-age=300. At 10:00:00 a shop owner changes a price. At 10:02:00 a customer who loaded the list at 09:59:00 refreshes. Which price can they see?

HTTP caching that scales

Immutable assets: content-hashed URLs (app.3f9a2c.js, /images/{hash}.jpg) with Cache-Control: public, max-age=31536000, immutable. “Invalidation” is a new URL.
Public API data: short max-age plus stale-while-revalidate lets clients and CDNs serve slightly stale data instantly while refreshing in the background; s-maxage sets a different TTL for shared caches than for browsers.
Revalidation: strong ETags (or Last-Modified) turn repeat loads into 304s; on mobile this saves battery and data for large JSON payloads.
CDN purge exists but is eventually consistent across edge locations; design so correctness never depends on a purge arriving.
Mobile apps cache too: URLCache on iOS and OkHttp’s Cache on Android honor these headers if you configure a disk cache, which is a cheap way to make cold starts and offline reads faster.

Layers: client (HTTP cache, app memory/disk) → CDN → in-process → shared (Redis) → database buffer pool. Earlier layers save more and are harder to invalidate.
Cache-Control: private/public, max-age, s-maxage, no-store, stale-while-revalidate, immutable; ETag + If-None-Match → 304; Vary for header-dependent responses.
Personalized responses are private/no-store; CDNs bypass authenticated traffic by default. Version URLs instead of relying on purges.

Check this part

What does Cache-Control: private mean?
Shared caches must never keep personalized responses.
Why can a user still see an old price after it changed?
The server can't recall copies already given out.
How should static assets be cached and "invalidated"?
Correctness shouldn't depend on purges reaching every edge.

Caches sit at every layer: the app on the phone, a CDN near users, the server's memory, a shared Redis, the database's own memory. HTTP headers (Cache-Control, ETag) control the first two. Anything cached in a shared layer must be the same for everyone who receives it.

Part Not started

Keeping cached data fresh enough

The most common pattern is cache-aside: the application checks the cache first and fills it itself on a miss.

Cache-aside

fun product(id: String): Product {
    redis.get("product:$id")?.let { return json.decode(it) }    // 1. hit: done
    val product = db.findProduct(id)                             // 2. miss: load from the database
    redis.setex("product:$id", 600, json.encode(product))        // 3. keep it for 10 minutes
    return product
}

fun updatePrice(id: String, cents: Int) {
    db.updatePrice(id, cents)                                    // 1. change the source of truth
    redis.del("product:$id")                                     // 2. delete the copy; next read reloads it
}

Product product(String id) {
    String cached = redis.get("product:" + id);
    if (cached != null) return json.decode(cached);              // 1. hit: done
    Product product = db.findProduct(id);                        // 2. miss: load from the database
    redis.setex("product:" + id, 600, json.encode(product));     // 3. keep it for 10 minutes
    return product;
}

void updatePrice(String id, int cents) {
    db.updatePrice(id, cents);                                   // 1. change the source of truth
    redis.del("product:" + id);                                  // 2. delete the copy; next read reloads it
}

async function product(id: string): Promise<Product> {
  const cached = await redis.get(`product:${id}`);
  if (cached) return JSON.parse(cached);                         // 1. hit: done
  const product = await db.findProduct(id);                      // 2. miss: load from the database
  await redis.set(`product:${id}`, JSON.stringify(product), { EX: 600 });   // 3. keep 10 min
  return product;
}

async function updatePrice(id: string, cents: number): Promise<void> {
  await db.updatePrice(id, cents);                               // 1. change the source of truth
  await redis.del(`product:${id}`);                              // 2. delete the copy; next read reloads it
}

On read: cache first; on a miss, load from the database and save it with a TTL (time to live). After the TTL, the copy disappears and the next read loads a fresh one.
On write: update the database, then delete the cached copy (rather than writing the new value into it). The next read loads the new value. Deleting is simpler and avoids storing a value that the database didn’t actually save.
The TTL is your safety net: if an update ever forgets to delete the copy, it’s still wrong for at most 10 minutes.
When the cache is full, it removes something to make room, usually the least recently used item (LRU).

How the same idea looks with Spring’s @Cacheable is in Caching with Redis.

Strategy	Read path	Write path	Use when
Cache-aside	App checks cache, loads on miss	App writes DB, deletes key	Default; cache failures degrade to DB load
Read-through	Cache library loads on miss	—	Same, with loading logic centralized
Write-through	Reads hit warm cache	Write cache and DB synchronously	Read-after-write must hit cache; write latency acceptable
Write-behind	Reads hit cache	Write cache, flush to DB asynchronously	Write-heavy counters; risk of data loss on cache failure
Refresh-ahead	Background refresh before expiry	—	Hot keys with predictable access

Staleness is bounded by the TTL and the invalidation path. Choose TTLs per data type by asking “what does a wrong answer cost”: a stale product description for a minute is harmless; a stale balance or permission is not, and usually shouldn’t be cached without versioning.

Exerciseeasy

Choose cache settings

For a shopping app, decide for each item: cache it or not, where (CDN, Redis, nowhere), and for how long. Give one sentence of reasoning each.

Product photos
Product names and descriptions
Product prices
Stock count (“only 3 left”)
The user’s order history
The user’s account balance (store credit)

›

Ask two questions for each: is it the same for everyone (CDN is possible) or personal (it isn’t)? And what does showing an old value cost?

SolutionTry it yourself first

Data	Where	TTL	Why
Product photos	CDN + app	1 year, versioned URL	Never change at a given URL; a new photo gets a new URL
Names, descriptions	CDN or Redis	Minutes to hours	Same for everyone; a stale description is harmless, delete on edit
Prices	Redis, short CDN TTL	~1 minute, delete on change	Must update soon; checkout re-reads the price from the database anyway
Stock count	Redis, very short, or not at all	Seconds	“Only 3 left” may be slightly off for display; the order itself checks real stock
Order history	Per-user cache in Redis, or none	Minutes, delete on new order	Personal: never CDN; cheap enough to load directly for most apps
Account balance	Not cached	—	A wrong balance costs money and trust; read it from the source of truth

The pattern: display can be slightly stale; decisions (checkout, payments, permissions) read the source of truth.

Hit rate is everything

Predict the output

A cache hit costs 1 ms, a miss costs 50 ms (cache check plus database). What's the average latency at a 95% hit rate, and at 80%?

Ways to raise it: cache the right granularity (whole rendered fragments vs individual rows), normalize keys (no tracking parameters in URL-based keys), size memory for the working set, and use LFU for skewed popularity where LRU churns.

Cache-aside by default: read cache → load on miss → set with TTL; write DB → delete key. TTL bounds staleness when invalidation misses.
Read-through, write-through, write-behind (data-loss risk), refresh-ahead (hot keys).
Decide TTL by the cost of a wrong answer; decisions (checkout, permissions, balances) read the source of truth.
Average latency = hit × t_hit + miss × t_miss; small hit-rate drops multiply database load. Eviction: LRU default, LFU for skewed popularity.

Check this part

In cache-aside, what happens on a cache miss?
The next read for the same key is then a hit.
After updating a price in the database, what should the app do with the cached copy?
The TTL is the safety net if a delete is ever missed.
Hit rate drops from 95% to 80%. What happens to database load?
Misses are what reach the database.
Which strategy risks losing writes if the cache fails?
Writes sit in the cache until they're flushed to the database.

Cache-aside reads from the cache, loads from the database on a miss and stores the result with a TTL; writes update the database and delete the cached copy. The TTL caps how stale data can get. Eviction (LRU) decides what goes when memory is full.

Part Not started

How caches fail

Problem	What happens	Defense
Stampede	A popular key expires; 5,000 requests miss at once and all query the database	Only one request rebuilds (a lock or “single flight”); refresh before expiry; add random jitter to TTLs
Hot key	One key gets most of the traffic and overloads one cache server	A short in-process cache on each app server; copy the key across nodes
Missing data	Requests for IDs that don’t exist always miss and always hit the database	Cache “not found” briefly too; validate IDs first
Cold cache	After a restart or flush, every request misses at once	Warm the cache before taking traffic; restore gradually
Cache down	Redis is unreachable	Fall back to the database with timeouts and a limit, or serve stale data

A cache stampede is the one to remember. Imagine a shop that opens its doors at exactly 9:00 and the whole crowd rushes in at once. The cache entry for the home page expires at the same moment for everyone, and thousands of requests ask the database for the same thing at the same time, which is exactly the load the cache was protecting it from.

Most cache incidents are correlated misses: synchronized expiry (same TTL set at the same time, e.g. after a deploy warm-up), a flush, a node loss, or a key-format change. Defenses work at three levels: coalescing (one loader per key per process or cluster), decorrelation (TTL jitter, probabilistic early refresh), and load shedding at the database (pool limits, circuit breakers) so a cache failure degrades latency instead of taking the database down.

One rebuild per key, with jittered TTLs

private val inFlight = ConcurrentHashMap<String, Deferred<Product>>()

suspend fun product(id: String): Product = coroutineScope {
    redis.get("product:$id")?.let { return@coroutineScope json.decode<Product>(it) }
    inFlight.computeIfAbsent(id) {                       // only one rebuild per key in this server
        async {
            try {
                db.findProduct(id).also { p ->
                    val ttl = 600 + Random.nextInt(120)      // jitter: keys don't expire together
                    redis.setex("product:$id", ttl, json.encode(p))
                }
            } finally { inFlight.remove(id) }
        }
    }.await()
}

private final ConcurrentHashMap<String, CompletableFuture<Product>> inFlight = new ConcurrentHashMap<>();

CompletableFuture<Product> product(String id) {
    String cached = redis.get("product:" + id);
    if (cached != null) return CompletableFuture.completedFuture(json.decode(cached));
    return inFlight.computeIfAbsent(id, key -> CompletableFuture
            .supplyAsync(() -> {
                Product p = db.findProduct(key);
                int ttl = 600 + ThreadLocalRandom.current().nextInt(120);   // jitter
                redis.setex("product:" + key, ttl, json.encode(p));
                return p;
            }, loaderPool)
            .whenComplete((p, e) -> inFlight.remove(key)));     // only one rebuild per key
}

const inFlight = new Map<string, Promise<Product>>();

async function product(id: string): Promise<Product> {
  const cached = await redis.get(`product:${id}`);
  if (cached) return JSON.parse(cached);
  let pending = inFlight.get(id);                       // only one rebuild per key in this server
  if (!pending) {
    pending = (async () => {
      try {
        const p = await db.findProduct(id);
        const ttl = 600 + Math.floor(Math.random() * 120);   // jitter: keys don't expire together
        await redis.set(`product:${id}`, JSON.stringify(p), { EX: ttl });
        return p;
      } finally {
        inFlight.delete(id);
      }
    })();
    inFlight.set(id, pending);
  }
  return pending;
}

Two small changes: concurrent misses for the same key in one server share one database query, and each entry gets a slightly random TTL so popular keys don’t all expire in the same second. With 20 servers, a stampede of 5,000 queries becomes at most 20.

Production puzzles

Spot the bugA price that stays wrong for ten minutes

Tap the line with the bug.

fun product(id: String): Product {
    redis.get("product:$id")?.let { return json.decode(it) }
    val product = db.findProduct(id)
    redis.setex("product:$id", 600, json.encode(product))
    return product
}

fun updatePrice(id: String, cents: Int) {
    db.updatePrice(id, cents)
    redis.del("product:$id")
}

Product product(String id) {
    String cached = redis.get("product:" + id);
    if (cached != null) return json.decode(cached);
    Product product = db.findProduct(id);
    redis.setex("product:" + id, 600, json.encode(product));
    return product;
}

void updatePrice(String id, int cents) {
    db.updatePrice(id, cents);
    redis.del("product:" + id);
}

async function product(id: string): Promise<Product> {
  const cached = await redis.get(`product:${id}`);
  if (cached) return JSON.parse(cached);
  const product = await db.findProduct(id);
  await redis.set(`product:${id}`, JSON.stringify(product), { EX: 600 });
  return product;
}

async function updatePrice(id: string, cents: number): Promise<void> {
  await db.updatePrice(id, cents);
  await redis.del(`product:${id}`);
}

Exercisemedium

A flash sale page

At 12:00 a limited sneaker drop goes live. Expect 200,000 users refreshing one product page in the first minute; the page shows details, price and “in stock / sold out”. The product row lives in PostgreSQL; there’s a Redis cluster and a CDN. Design the caching so the database survives and nobody can buy more than the stock.

›

Separate what can be stale (details, images) from what drives decisions (stock). One key will be extremely hot. What protects Redis, and what protects the database?

SolutionTry it yourself first

Static parts at the edge: product details and images are public: pre-render, put on the CDN with a short s-maxage (e.g. 5 s) plus stale-while-revalidate, and warm it before 12:00. Most of the 200,000 requests never reach the servers.
The hot key: the dynamic part (price, stock status) comes from a small endpoint cached in each app server’s memory for 1–2 seconds, in front of Redis. 50 servers × 1 request/second to Redis instead of thousands; no single Redis node gets the whole stream.
No stampede: single-flight per key in each server, values refreshed ahead of expiry by a background task during the sale, never expiring all at once.
Stock isn’t a cache question: display “in stock” from the cached value, but the purchase decrements stock atomically at the source of truth (UPDATE … SET stock = stock - 1 WHERE id = ? AND stock > 0, or an atomic counter in Redis with persistence, reconciled with the database), and the response decides success. A slightly stale “in stock” badge is fine; overselling is not.
Protect the database anyway: strict pool limits, a queue or token-based “waiting room” if checkout demand exceeds capacity, and a circuit breaker that serves the cached page (with “checking availability…”) if the database slows down.

Stampede: single-flight per key (per process, or a distributed lock), refresh-ahead for hot keys, TTL jitter.
Hot keys: in-process micro-caches (1–2 s), key replication across nodes.
Penetration: negative caching with short TTLs, input validation, Bloom filters for huge ID spaces.
Fill/invalidate race in cache-aside can pin stale data for a full TTL: short TTLs, versioned writes, delayed double delete, leases, or CDC-driven invalidation.
Cache down: timeouts and fallback with load shedding; never let a cache outage become a database outage.

Check this part

What is a cache stampede?
Let one request rebuild it, and add jitter so keys don't expire together.
Requests for products that don't exist always reach the database. A simple fix?
Negative caching stops repeated misses for the same missing ID.
In cache-aside, how can a stale value get into the cache after an update deleted the key?
Bound it with short TTLs, versioned writes, a delayed second delete or leases.
During a flash sale, where must "can this user buy it?" be decided?
Display can be stale; decisions can't.

Caches fail in predictable ways: many requests rebuilding the same expired key at once (stampede), one key too hot for one node, lookups for missing data that always reach the database, and races that store stale values. Each has a standard defense.

Key takeaways

Caches sit at every layer, from the user’s phone to the database; the closer, the faster and the harder to update.
Never let shared caches (CDNs) store personal responses: use private or no-store.
Cache-aside: read from the cache, load on a miss, store with a TTL; on writes, update the database and delete the copy.
Show slightly stale data if you must, but make decisions (payments, stock, permissions) from the source of truth.

Pick the layer by who shares the data and how stale it may be; control client and CDN layers with HTTP semantics and versioned URLs.
Hit rate drives latency and database load; monitor it per key family.
Correlated misses (stampedes, flushes, cold starts) are the main failure mode: coalesce, jitter, refresh ahead, shed load.
Cache-aside has a fill/invalidate race; bound it with TTLs, versioning or leases.