Interview Notes Search

Hello InterviewQuick Reference

Full Article

Caching

When to Use Caching

1

Read-heavy workloads

The same data is accessed frequently.

2

Expensive computations or queries

Results can be stored and reused.

3

Data that doesn't change often

Or where slight staleness is acceptable.

4

Reducing load on downstream services

Take pressure off databases and APIs.

Cache Architectures

The four cache architectures: Cache-Aside (app reads cache, on miss reads database then populates), Write-Through (write to cache and database at the same time), Write-Behind (write to cache first, later flush to database), and Read-Through (cache fetches from database on miss for the app)

  • Cache-Aside (Lazy Loading) : App checks cache first, on miss reads from DB and populates cache. Most common pattern.
  • Write-Through : Every write goes to cache AND DB simultaneously. Ensures consistency but adds write latency.
  • Write-Behind (Write-Back) : Writes go to cache first, async flush to DB. Fast writes but risk of data loss.
  • Read-Through : Cache itself handles fetching from DB on miss. Simplifies app logic.

Eviction Policies

The four eviction policies: LRU evicts the least recently accessed item (most common default), LFU evicts the least frequently accessed item (good for skew), TTL expires items after a set duration (simple, predictable), and FIFO is first in first out (rarely used in practice)

  • LRU (Least Recently Used) : Evicts the least recently accessed item. Most common default.
  • LFU (Least Frequently Used) : Evicts the least frequently accessed item. Good for skewed access patterns.
  • TTL (Time-To-Live) : Items expire after a set duration. Simple and predictable.
  • FIFO : First in, first out. Rarely used in practice.

Cache Types

The four cache types: In-Process Cache lives in app memory (fastest, not shared across instances), External Cache is shared across instances and adds a network hop (~1ms), CDN Cache has edge servers caching static assets close to users, and Client-Side Cache is the browser cache using HTTP headers like Cache-Control and ETag

  • In-Process Cache : Lives in app memory (HashMap). Fastest but not shared across instances.
  • External Cache (Redis/Memcached) : Shared across instances. Adds network hop (~1ms).
  • CDN Cache : Edge servers cache static assets close to users. Great for images, JS, CSS.
  • Client-Side Cache : Browser cache, HTTP cache headers (Cache-Control, ETag, Last-Modified).

Key Numbers

The key caching numbers compared as bars, fastest to slowest: an in-process cache responds in nanoseconds, Redis handles 100k+ ops/sec with sub-millisecond latency, Memcached has similar throughput but is simpler with no persistence, a CDN hit takes ~5-20ms, and origin ~100-500ms

Redis

100k+ ops/sec, sub-millisecond latency

Memcached

Similar throughput, simpler (no persistence)

CDN hit

~5-20ms vs origin ~100-500ms

In-process cache

Nanoseconds

Common Pitfalls

The four common caching pitfalls: Cache Stampede (many requests miss at once, overwhelming the backend), Stale Data (cache serves outdated values instead of fresh data), Cold Start (empty cache after restart overloads the database), and Over-caching (caching everything wastes memory, cache only hot data)

  • Cache Stampede : Many requests hit cache miss simultaneously. Fix: locking, request coalescing.
  • Stale Data : Cache returns outdated data. Fix: TTL, cache invalidation on writes.
  • Cold Start : Empty cache after restart causes DB overload. Fix: cache warming.
  • Over-caching : Caching everything wastes memory. Cache only hot data.