AdultInfra Get a test plan

Scaling the thumbnail and image plane

Images dominate request count on adult portals and exhaust inode and object counts before bandwidth. How to scale thumbnails, sprites, and image delivery without breaking caching.

THE PROBLEM The image path is slow and the cache is thrashing, even though image bytes are a small share of traffic.

LAST REVIEWED

On adult portals, images are the traffic nobody plans for. Video gets the architecture review; images quietly become the majority of requests, the majority of cache objects, and the first thing to break under a catalogue migration. The failure is rarely bandwidth — it is object count, cache churn, and a shared cache path that lets tiny images evict expensive video.

The mechanism

A catalogue of a million titles produces several images each: posters, grid thumbnails, hero images, hover sprites, and preview clips. Public traces of a large adult portal show images were roughly 80% of requests against roughly 37% of bytes, while video was roughly 20% of requests against roughly 63% of bytes. Images are cheap per byte and expensive per object. Every derived size is a distinct cache object, so a single source image can become ten objects across form factors and breakpoints. The result is enormous object cardinality with tiny average size.

Cause 1: inode and object-count exhaustion

Storage backends fail on object count before capacity. A filesystem can run out of inodes (df -i shows 100% used) while df -h shows free space; object storage has per-bucket request-rate and key-count considerations; and a cache tier has a fixed maximum number of entries regardless of their size. A catalogue of many small derivatives is exactly the workload that hits these limits. Track inode/object count and cache entry count as first-class capacity metrics, not afterthoughts.

Mitigations:

  • Shard keys across multiple buckets or prefixes to spread request rate.
  • Generate sizes on demand from a small set of source images instead of storing every variant.
  • Consolidate sprites so a grid of previews is one object rather than twenty.
  • Prune unused derivatives with the same lineage discipline used for takedowns.

Cause 2: hover sprites and previews

Hover previews and animated thumbnails are the highest-cardinality, highest-request image objects on many sites. A preview clip per title can double or triple the object count. Treat them as a distinct class with their own cache policy and, where possible, serve them from a sprite sheet or a short low-bitrate video rather than N separate images. Because they are requested on hover, they amplify cache churn disproportionately to their byte size.

Cause 3: image optimization done badly

Naive image optimization re-encodes on every request or stores a combinatorial matrix of size × format × quality. Both scale badly. Good practice:

  • Negotiate format at the edge from Accept, serving modern formats where supported and falling back otherwise.
  • Cap the variant matrix — a defined set of sizes and one or two formats, not the cross-product of every device hint.
  • Cache transformed output keyed by source plus transform parameters, so a transform is computed once.
  • Strip metadata that is not needed for delivery, and never expose original uploads.

For adult platforms there is an additional consideration: originals can carry sensitive metadata (location, device, and sometimes identity). Strip it on ingest, and never make the original publicly addressable.

Cause 4: image and video share a cache path

If thumbnails and video segments share one cache tier and key space, a burst of image requests evicts video bytes that are expensive to refill, and vice versa. Separate the planes:

  • Different cache paths or resources for image and video, so eviction is isolated.
  • Different freshness: images are immutable once published, so they can be cached very aggressively; manifests and API responses are not. See Cache control and cache keys.
  • Different cache keys: images keyed by source and transform, video by object and range.

Immutable image URLs (content-addressed or versioned) let you cache images for a long time and change them by changing the URL, which also makes takedowns and corrections clean.

What good looks like

  • Object/inode count and cache entry count are monitored capacity metrics.
  • A bounded variant matrix with edge format negotiation.
  • Hover previews consolidated into sprites or short clips.
  • Image and video caches separated, with image freshness long and immutable.
  • Originals stripped of metadata and never public.

How to prove it

Split image and video cache metrics on one hostname, then route the image plane through a tuned path and compare image byte offload, origin requests, cache entry churn, and p95 image latency against the incumbent. This is a controlled test that leaves video delivery untouched.