Media hotlinking and catalogue scraping are two faces of the same problem: requests that consume your delivery capacity but produce none of your revenue. Both are easy to detect in aggregate and hard to stop per request, because the naive defences either fail outright or destroy caching. The goal is enforcement that scales with your traffic, not with your abuse.
The mechanism
A hotlinker embeds your media URL on their page; their visitors’ browsers fetch bytes from your edge. A scraper pulls manifests, segments, and metadata to rebuild or mirror your catalogue. In both cases the requests are technically valid HTTP — correct paths, often correct headers — so the only things that distinguish them from legitimate traffic are who is asking, why, and under what entitlement. Defence is therefore about binding delivery to entitlement and about making scraping economically unattractive.
Why referer checks fail
Referer checking is the classic hotlink defence and the weakest one:
- It is trivially spoofed by any client that sets a header, so it stops only the laziest embedding.
- Legitimate clients strip or omit it (privacy modes, native players, some browsers), causing false positives.
- It cannot express “this viewer is entitled” — only “this page looks like ours”.
- It does nothing against scrapers that set the right referer deliberately.
Use Referer at most as a weak signal in a scoring system, never as the enforcement boundary.
Token-bound URLs
Bind delivery to a short-lived signed token issued after your own authorisation step. The token should encode the content ID, an expiry, and enough scope to limit what it grants (a single rendition, a byte budget, a session). Validate it at the edge before serving, so an unauthenticated request never reaches origin. Key design matters: validate the token but keep it out of the media cache key, or every viewer becomes a unique cache entry and the cache becomes a pass-through. Derive the cache key from the object identity, not the entitlement. See Cache control and cache keys.
For scraping specifically, bind tokens to a session and a rate ceiling, so a single token cannot be replayed across a whole catalogue. Rotate signing keys and keep expiry short enough that a leaked URL has little value.
Bot management and WAF
A web application and API protection layer (WAF with bot management) handles the requests that get past static rules: credential stuffing against the token issuer, headless browsers, distributed scraping, and API enumeration. Effective signals include request-rate anomalies per ASN, TLS and header fingerprints, navigation patterns (a scraper requests many manifests and few images), and absence of normal player telemetry such as CMCD. Challenge suspicious clients rather than blocking outright, and keep a path to distinguish a legitimate partner integration from a scraper.
Scrape-to-rehost economics
Scraping is an economic decision. Make it expensive: signed, short-lived URLs raise the cost of maintaining a mirror; per-session rate limits cap throughput; watermarking or per-user manifests make a stolen copy traceable; and detecting bulk sequential access lets you cut a scraper’s token early. You will not eliminate determined scraping, but you can make it cheaper to license your content than to steal it.
Cache-key design under enforcement
The tension is real: enforcement wants per-request context, caching wants object identity. Resolve it by separating the two planes:
- Entitlement plane: token, session, geo, and policy checks — per request, cheap, never cached.
- Object plane: media cache key from content ID, rendition, and byte range only — cacheable across all entitled viewers.
Enforce at request time, cache at object level. This keeps byte offload high while still refusing unentitled traffic.
What good looks like
- Signed, short-lived, scoped tokens validated at the edge.
- Cache key derived from the object, not the token.
- Bot management scoring on fingerprints, rates, and player telemetry.
- Per-session rate ceilings that cap scraping throughput.
How to prove it
Instrument hotlink and scrape signals on one hostname, then route a slice through token validation plus bot management and compare origin traffic from unentitled sources, byte offload, and false-positive rate against the incumbent. Contact us to run it as a controlled test without a migration.
Related reading
- Cache control and cache keys for key hygiene.
- Adult video CDN for the delivery context.
- Making takedowns actually disappear for the removal side.