Implement referer blocklist to prevent hotlinking #90

New Issue

clawbot · 2026-08-09T03:51:22+02:00

clawbot commented

2026-08-09 03:51:22 +02:00

Verified against main at 61f42e6. This is the "referer blacklist" P2 item in TODO.md; confirmed missing — the Referer header is read only for logging (internal/middleware/middleware.go:97) and never used for any decision.

The use case is hotlink protection: a third-party site embedding pixa-served images makes pixa pay the bandwidth and the origin pay the fetches, on someone else's traffic. Since signed URLs are shareable by design and long-lived until exp, a leaked URL embedded elsewhere is a realistic drain — and with q/fit currently unsigned (filed separately), the amplification is worse.

Naming: TODO.md says "blacklist". Per the repo's inclusive-terminology convention (established when whitelist became allowlist in #41), this should be referer_blocklist or similar. Update the TODO.md wording in the same change.

Definition of done

A referer_blocklist config key (list of host patterns, supporting the same exact/suffix matching semantics the allowlist already uses — reuse internal/allowlist's matcher rather than writing a second one). Strict parsing: an invalid entry aborts startup naming the key and the value, per repo policy.
A request whose Referer matches is refused with 403 before any upstream fetch or cache write, so a blocked request costs nothing.
Absent or unparseable Referer is treated as not-blocked (a blocklist cannot be a de facto allowlist — many legitimate clients send no referer). Document this explicitly; it is the main limitation of the feature and operators should know it is trivially bypassed by omitting the header.
Decide and document whether a blocked request should still be served from cache if already present — recommend no, refuse uniformly, so behavior does not depend on cache state.
Failing tests first: a blocked referer returns 403 and triggers no fetch; an unlisted referer succeeds; an absent referer succeeds; suffix matching behaves like the allowlist's.
README and config.example.yml document the key and its limitations. make check green.

Note

If an allowlist model would suit better than a blocklist (refuse everything except named referers), say so before implementing — for hotlink protection an allowlist is usually the more effective shape, and the blocklist is the weaker default. Flagging the choice rather than assuming TODO.md's wording settles it.

Verified against `main` at `61f42e6`. This is the "referer blacklist" P2 item in `TODO.md`; confirmed missing — the `Referer` header is read only for logging (`internal/middleware/middleware.go:97`) and never used for any decision. The use case is hotlink protection: a third-party site embedding pixa-served images makes pixa pay the bandwidth and the origin pay the fetches, on someone else's traffic. Since signed URLs are shareable by design and long-lived until `exp`, a leaked URL embedded elsewhere is a realistic drain — and with `q`/`fit` currently unsigned (filed separately), the amplification is worse. Naming: `TODO.md` says "blacklist". Per the repo's inclusive-terminology convention (established when `whitelist` became `allowlist` in #41), this should be `referer_blocklist` or similar. Update the `TODO.md` wording in the same change. ## Definition of done 1. A `referer_blocklist` config key (list of host patterns, supporting the same exact/suffix matching semantics the allowlist already uses — reuse `internal/allowlist`'s matcher rather than writing a second one). Strict parsing: an invalid entry aborts startup naming the key and the value, per repo policy. 2. A request whose `Referer` matches is refused with 403 before any upstream fetch or cache write, so a blocked request costs nothing. 3. Absent or unparseable `Referer` is treated as not-blocked (a blocklist cannot be a de facto allowlist — many legitimate clients send no referer). Document this explicitly; it is the main limitation of the feature and operators should know it is trivially bypassed by omitting the header. 4. Decide and document whether a blocked request should still be served from cache if already present — recommend no, refuse uniformly, so behavior does not depend on cache state. 5. Failing tests first: a blocked referer returns 403 and triggers no fetch; an unlisted referer succeeds; an absent referer succeeds; suffix matching behaves like the allowlist's. 6. README and `config.example.yml` document the key and its limitations. `make check` green. ## Note If an allowlist model would suit better than a blocklist (refuse everything except named referers), say so before implementing — for hotlink protection an allowlist is usually the more effective shape, and the blocklist is the weaker default. Flagging the choice rather than assuming `TODO.md`'s wording settles it.

clawbot added this to the 1.0.0 milestone 2026-08-09 03:51:22 +02:00

clawbot referenced this issue

2026-08-09 16:10:39 +02:00

Tracking: 1.0.0 sequencing and critical path #103

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: sneak/pixa#90