Refuse image requests whose Referer is on referer_blocklist (closes #90)

A new setting, referer_blocklist (PIXA_REFERER_BLOCKLIST), lists hosts
whose pages may not show pixa's images. Entries are written and matched
as allowlist_hosts are, with the same matcher. Both image routes check
the Referer before the signature, the cache and the upstream fetch, and
answer 403 with the JSON error, so a blocked request costs nothing and
is refused whether or not the image is cached. No Referer, or one that
does not parse, is served; README.md says this makes the list easy to
get around. An entry of either host list that is not a host name
(letters, digits, hyphens, underscores, dots, at most one leading dot)
or an IP address now aborts startup naming the setting and the entry.

Model: opus-5-5

.gitattributes

@@ -0,0 +1,4 @@
+# Every PR adds an entry at the top of TODO.md's Completed Steps; union keeps
+# both sides instead of conflicting. Git never reports a conflict here: read
+# the merged entries after every merge or rebase.
+TODO.md merge=union

.golangci.yml

@@ -10,14 +10,20 @@ run:
 
 linters:
   default: all
+  enable:
+    # Successor to the deprecated gomodguard. Named explicitly, rather than
+    # left to `default: all`, because it carries the module policy below.
+    - gomodguard_v2
   disable:
     # Genuinely incompatible with project patterns
     - exhaustruct      # Requires all struct fields
-    - depguard         # Dependency allow/block lists
     - godot            # Requires comments to end with periods
-    - wsl              # Deprecated, replaced by wsl_v5
     - wrapcheck        # Too verbose for internal packages
     - varnamelen       # Short names like db, id are idiomatic Go
+    # Deprecated: the warning is attached to the old name, so it is
+    # silenced by disabling that name, not by enabling the successor.
+    - wsl              # Deprecated, replaced by wsl_v5
+    - gomodguard       # Deprecated, replaced by gomodguard_v2
   settings:
     lll:
       line-length: 88
@@ -28,6 +34,64 @@ linters:
       max-complexity: 15
     dupl:
       threshold: 100
+    depguard:
+      # Test-support code must not be compiled into the shipped binary. A
+      # test-support package exists to hand a test privileges the program
+      # itself must never have, so a file that is not a test must not import
+      # one. Test files, and the files inside a package whose directory name
+      # ends in `test`, are where that code belongs, and are exempt.
+      #
+      # The deny list below is the one part of this file a repository is
+      # expected to extend, and the only part it may. depguard matches an
+      # import path against a list of prefixes, so it cannot be told "any path
+      # whose last segment ends in test"; a repository's own test-support
+      # packages have to be named here one at a time, by full import path,
+      # under a module path that differs from repository to repository. Add
+      # them; change nothing else.
+      rules:
+        test-support:
+          list-mode: lax
+          files:
+            - "$all"
+            - "!$test"
+            - "!**/*test/**"
+          deny:
+            - pkg: net/http/httptest
+              desc: >-
+                Test-support code belongs in test files and in packages whose
+                directory name ends in test, not in the shipped binary.
+    # Only decisions already recorded in the Go package defaults are
+    # listed here. Every entry matches the module path exactly.
+    gomodguard_v2:
+      blocked:
+        - module: github.com/rs/zerolog
+          recommendations:
+            - log/slog
+          reason: "Structured logging is stdlib log/slog."
+        # One entry per pre-fork module path, because the later releases
+        # are separate paths. A prefix match would be shorter but would
+        # also reach github.com/go-redis/redismock, the test double for
+        # the successor these entries recommend.
+        - module: github.com/go-redis/redis
+          recommendations:
+            - github.com/redis/go-redis/v9
+          reason: "Pre-fork module; use the maintained go-redis v9."
+        - module: github.com/go-redis/redis/v7
+          recommendations:
+            - github.com/redis/go-redis/v9
+          reason: "Pre-fork module; use the maintained go-redis v9."
+        - module: github.com/go-redis/redis/v8
+          recommendations:
+            - github.com/redis/go-redis/v9
+          reason: "Pre-fork module; use the maintained go-redis v9."
+        - module: github.com/sergi/go-diff
+          recommendations:
+            - github.com/aymanbagabas/go-udiff
+          reason: "No unified diff output; use go-udiff."
+        - module: github.com/hexops/gotextdiff
+          recommendations:
+            - github.com/aymanbagabas/go-udiff
+          reason: "Unmaintained fork; use go-udiff."
 
 issues:
   max-issues-per-linter: 0

Dockerfile

@@ -18,9 +18,14 @@ COPY . .
 # Tells script/lint it is inside a container, so it runs the linter.
 ENV container=docker
 
-# Run formatting check and linter
+# Run formatting check and linter. script/cibuild and script/docker pass
-RUN make fmt-check
+# a new CHECK_EPOCH on every run, and each check step names it in its
-RUN make lint
+# command, so a new value reruns the step instead of reusing a cached
+# success that checked nothing. A plain `docker build .` leaves it empty
+# and reuses the check steps only for an identical build context.
+ARG CHECK_EPOCH
+RUN echo "check epoch: ${CHECK_EPOCH}" && make fmt-check
+RUN echo "check epoch: ${CHECK_EPOCH}" && make lint
 
 # Build stage
 # golang:1.25.4-alpine, 2026-02-25
@@ -39,8 +44,9 @@ RUN script/bootstrap
 # Copy source code
 COPY . .
 
-# Run tests
+# Run tests; a new CHECK_EPOCH reruns them, as in the lint stage.
-RUN make test
+ARG CHECK_EPOCH
+RUN echo "check epoch: ${CHECK_EPOCH}" && make test
 
 # VERSION is declared here, not earlier: a new value reruns only the
 # build, not script/bootstrap or the tests. Given none, the version is

README.md

@@ -18,7 +18,7 @@ make build
 # run with a config file: copy the example and set a real signing key
 # (the example placeholder is refused at startup), e.g. with
 # openssl rand -base64 32
-cp config.example.yml config.yml
+cp configs/config.example.yml config.yml
 $EDITOR config.yml   # replace the signing_key placeholder
 ./bin/pixad --config config.yml
 
@@ -34,6 +34,67 @@ else has a built-in default. A config file mounted at `/etc/pixa/config.yml`
 is optional: it is read when present, and an environment variable wins over
 the same setting in it.
 
+## Deployment
+
+pixa listens on plain HTTP and runs behind a reverse proxy that terminates TLS.
+[`configs/Caddyfile`](configs/Caddyfile) is an example for Caddy, chosen because
+it is the smallest correct one: Caddy gets the TLS certificate itself and does
+everything in this list without further settings. The reverse proxy must:
+
+- terminate TLS, as the login and generator pages work only over HTTPS (see
+  Routes);
+- pass the `Host`, `Origin` and `Referer` headers on unchanged, as pixa refuses
+  a form from those pages unless `Origin` or `Referer` names the host in `Host`,
+  builds encrypted URLs from `Host`, and checks `Referer` against
+  `referer_blocklist`;
+- set `X-Forwarded-For` to the client's address, with `trusted_proxies` set to
+  the address pixa sees the proxy's requests come from, so the login limit
+  counts each client by its own address (see `trusted_proxies` under
+  Configuration);
+- wait for pixa's answer for at least `downstream_timeout` (default `60s`), the
+  longest pixa takes to fetch, convert and send an image.
+
+It may also refuse `/metrics`, as the example does, so that only a scraper that
+reaches pixa directly can read it; pixa itself asks for the metrics username and
+password there.
+
+pixa does the rest itself: it checks signatures and encrypted URLs, applies the
+allowlist, refuses upstream hosts with private or local addresses, limits login
+attempts, upstream response size and image dimensions, and sends the security
+headers, `Strict-Transport-Security` included, with every response.
+
+The state directory (`state_dir`, `/var/lib/pixa` in the container) holds the
+database and the disk cache:
+
+- It needs a persistent volume: without one, every restart starts with an empty
+  cache. In the container, the startup script gives the directory to the user
+  pixa runs as (uid 65532) and sets its mode to `750`; outside it, that user
+  must be able to write the directory.
+- `cache_max_bytes` limits the source and transformed images together. The
+  database, the metadata files, the `.meta` file beside each transformed image
+  and files still being written come on top, and eviction runs in the
+  background, so the cache can pass the limit for a while: leave room on the
+  volume beyond it.
+- Set `cache_max_bytes` for a lasting deployment. Its default, worked out each
+  time pixa starts, is 75% of the sum of the space free on the volume and the
+  space the cached images already take, so a restart keeps the limit the cache
+  had, but anything else that fills or frees space on the volume moves it.
+
+A load balancer's health check can request `/.well-known/healthcheck.json`,
+which answers 200 whenever pixa is running, in maintenance mode too (see
+`maintenance_mode`).
+
+On SIGTERM or SIGINT pixa stops accepting connections, gives the requests in
+progress and the images being processed 5 seconds to finish, and exits: with 0,
+or with 1 when images were still being processed after those 5 seconds or
+another part of pixa failed to stop. A request not finished by then is cut off.
+`docker stop` waits 10 seconds before it kills the container.
+
+Outside Docker, pixa needs libvips (the image has 8.15) and libheif to run, as
+it uses libvips through CGO; building it also needs their development files,
+`pkg-config` and a C compiler. `script/bootstrap` installs all of these with
+nix, apt, brew or apk.
+
 ## Running under upaas
 
 What the [upaas](https://git.eeqj.de/sneak/upaas) app for pixa needs:
@@ -49,7 +110,7 @@ What the [upaas](https://git.eeqj.de/sneak/upaas) app for pixa needs:
     - `PIXA_ALLOWLIST_HOSTS`: upstream hosts served without a signature,
       comma-separated
     - `PIXA_CACHE_MAX_BYTES`: disk cache limit in bytes; `0` disables it;
-      default 75% of free space
+      default 75% of (free space + what the cache holds)
     - the rest are in the table under Configuration below
 - **Health check:** the image's `HEALTHCHECK` requests
   `/.well-known/healthcheck.json`. upaas reads the container's health 60
@@ -71,13 +132,17 @@ prevent abuse, and allowlisted source hosts for open access.
 ### Storage
 
 - **Source content**:
-  `<statedir>/cache/src-content/<ab>/<cd>/<sha256 of source content>`
+  `<state_dir>/cache/sources/<ab>/<cd>/<sha256 of source content>`
 - **Source metadata**:
-  `<statedir>/cache/src-metadata/<hostname>/<sha256 of path>.json`
+  `<state_dir>/cache/metadata/<hostname>/<sha256 of path and query>.json`
-  (fetch time, original headers, request, content hash)
+  (host, path and query, content hash, upstream status and headers, fetch time)
-- **Database**: `<statedir>/state.sqlite3` (SQLite)
+- **Database**: `<state_dir>/state.sqlite3` (SQLite)
-- **Output documents**:
+- **Transformed images**:
-  `<statedir>/cache/dst-content/<ab>/<cd>/<sha256 of output content>`
+  `<state_dir>/cache/variants/<ab>/<cd>/<sha256 of host, path, query, size, format, quality and fit>`,
+  each with a `.meta` file beside it holding its content type
+
+`<ab>` and `<cd>` are the first and second pairs of characters of the file's
+name.
 
 Multiple source paths may reference the same content blob; the
 database tracks references rather than using filesystem refcounting.
@@ -88,17 +153,99 @@ the metadata file stored beside it.
 
 ### Routes
 
+pixa answers these routes; any other path answers 404. A path in this list asked
+with a method the list does not give answers 405, except `/static/<file>`, which
+answers any method as it answers `GET`. A browser's CORS preflight request
+(`OPTIONS` with `Origin` and `Access-Control-Request-Method` headers) to any
+path under `/v1/` answers 200, in maintenance mode too.
+
+- `GET /` — the login page, or the URL generator page with a login session
+  (see Encrypted URLs). Needs: nothing. Answers: 200.
+- `POST /` — log in with the signing key typed into the login page. Needs: the
+  login page's form (below). Answers: 303 to `/` with a login session cookie
+  that lasts 30 days for the right key; 200 with the login page and an error for
+  a wrong key; 429 over the login limit (below).
+- `POST /generate` — make an encrypted URL from the generator page's form.
+  Needs: a login session and the generator page's form (below); without a login
+  session it answers 303 to `/`. Answers: 200 with the page showing the URL; 400
+  with the page naming a field that is not valid; 500 when the URL cannot be
+  made.
+- `GET /logout` — end the login session. Needs: nothing. Answers: 303 to `/`.
+- `GET` or `HEAD` `/v1/image/<host>/<path>/<size>.<format>` — an image, fetched,
+  resized and converted (below). Needs: a signature, unless the host is
+  allowlisted (see Source Hosts). Answers: 200; 304 when `If-None-Match` matches
+  the image's `ETag`; 400 for a URL or parameter that is not valid; 401 for a
+  missing or wrong signature, a missing `exp` or an `exp` in the past; 403 when
+  the request's `Referer` names a host in `referer_blocklist`, checked before
+  the signature, the cache and the upstream fetch; 403 when the upstream host,
+  or a host it redirects to, is `localhost`, ends in `.localhost` or `.local`,
+  or has an address in a blocked network (see `blocked_networks`); 502 when the
+  upstream answered with an error status, and for 5 minutes after that for the
+  same source URL; 503 when pixa is busy or in maintenance mode; 500 for any
+  other failure.
+- `GET` or `HEAD` `/v1/e/<token>/<name>` — an image through an encrypted URL
+  (see Encrypted URLs). Needs: nothing but the URL. Answers: 200; 304 when
+  `If-None-Match` matches the image's `ETag`; 400 for a token that does not
+  decrypt, or that asks for a size or fit that is not valid; 410 once it has
+  expired; 504 when the upstream has not sent its response headers within
+  `upstream_fetch_timeout`, but 500 when that time runs out while the image
+  itself is still arriving; 403, 502, 503 and 500 as for `/v1/image/`.
+- `GET /robots.txt` — asks every crawler to stay away (`Disallow: /`). Needs:
+  nothing. Answers: 200.
+- `GET /.well-known/healthcheck.json` — JSON with `status` (`ok`), `now`,
+  `uptime_seconds`, `uptime_human`, `version`, `appname` and
+  `maintenance_mode`. Needs: nothing. Answers: 200, always.
+- `GET /static/<file>` — the stylesheet and script the login and generator
+  pages load. Needs: nothing. Answers: 200, or 404 for a file that does not
+  exist.
+- `GET /metrics` — Prometheus metrics (see Architecture). Needs: HTTP basic
+  authentication with `metrics.username` and `metrics.password`. Answers: 200;
+  401 without them; 404 when they are not set, as the route then does not exist.
+
+Every response carries an `X-Request-ID` header holding the request's ID, which
+a client can quote when reporting a problem: the request's own `X-Request-ID`,
+as a reverse proxy in front of pixa may send, when it is at most 64 letters,
+digits, `-`, `_` or `.`; otherwise a random one pixa makes for the request,
+which tells nothing about the machine or the other requests. pixa's log line for
+the request carries the same ID as `request_id`, and so do the lines it logs
+when it fetches, converts and serves an image; the fetch sends it to the
+upstream host as `X-Request-ID`.
+
+Both `POST` routes accept only a form that pixa's own page served: the page puts
+a token in the form and sets a cookie to match, and a request without both is
+refused with 403, so another site cannot submit the form from a visitor's
+browser. The login and generator pages are meant to be opened over HTTPS: while
+`debug` is off, a form sent from a page opened over plain HTTP is refused with
+403, and while it is on, so is one sent from a page opened over HTTPS. Plain
+HTTP is for development on the browser's own machine: the login session cookie
+is always marked `Secure`, and over plain HTTP a browser keeps such a cookie
+only for its own machine (`localhost`), if at all. A form is also refused with
+403 when the page's host is not the `Host` header pixa receives, so a reverse
+proxy in front of pixa must pass that header on unchanged. A form body over
+1 MiB is refused with 413. The image routes answer the errors listed for them
+with JSON holding `error`, `status` and `timestamp`.
+
+An image URL has this form:
+
 ```
-/v1/image/<host>/<path>/<size>.<format>?sig=<signature>&exp=<expiration>
+/v1/image/<host>/<path>/<size>.<format>?sig=<signature>&exp=<expiration>&q=<quality>&fit=<fit>
 ```
 
-Images are only fetched from origins using TLS with valid certificates.
+Images are only fetched from origins using TLS with valid certificates, unless
+`allow_http` is set: then pixa fetches every image over plain HTTP, which is for
+testing only.
 
 A request whose query string cannot be decoded, or gives any parameter more
 than once, is refused with 400.
 
-- `<format>`: one of `orig`, `png`, `jpeg`, `webp`
+- `<format>`: one of `orig` (or `original`), `jpeg` (or `jpg`), `png`, `webp`,
+  `avif`, `gif`
 - `<size>`: `orig` or `<width>x<height>` (e.g. `800x600`)
+- `sig` and `exp`: the signature and its expiry, needed unless the host is
+  allowlisted (see Signature Specification)
+- `q` and `fit`: the output quality and how the image is fitted to `<size>`,
+  both optional (values under Signature Specification). Both are part of what
+  is cached, so each value of either is a separate cached image.
 
 An image is served with `Cache-Control: public, max-age=<seconds>, immutable`.
 When the URL has an expiry (an `exp`, or the TTL of an encrypted URL),
@@ -107,6 +254,16 @@ or proxy cache keeps the image after pixa would refuse the URL. A URL with no
 expiry gets one year. `immutable` only stops a client revalidating while its
 copy is fresh.
 
+When several requests for the same image, size, format, quality and fit miss
+the cache at once, they share one upstream fetch (or one read of the cached
+source) and one transcode: the first request does the work, and the others wait
+for its image or its error, holding no upstream connection or processing slot
+of their own. A waiting request stops waiting when its own client goes away.
+The work goes on for the others even if the first request's client goes away,
+until that request's `downstream_timeout` ends. The shared fetch sends the first
+request's ID upstream, and the lines logged for the fetch and the transcode
+carry that ID.
+
 The login form (`POST /`) is limited to 5 attempts per minute per client
 address, counting an IPv6 client by its /64; an attempt over the limit is
 refused with 429 and a `Retry-After` header. Behind a reverse proxy the client
@@ -124,6 +281,39 @@ its own `X-Forwarded-For`, whether it connects directly or through the proxy,
 because its own address is trusted too. Setting `trusted_proxies` to only the
 address pixa sees for requests that come through the proxy closes this.
 
+### Encrypted URLs
+
+An encrypted URL is an image URL made on pixa's own web page by someone who
+knows the signing key. It works for any upstream host, allowlisted or not,
+without a signature, and whoever gets it can neither read the source URL from it
+nor change what it asks for.
+
+1. Open `/` in a browser over HTTPS (or over plain HTTP while `debug` is on, see
+   Routes) and log in with the signing key (`signing_key`). The login session
+   lasts 30 days, or until `/logout`.
+2. On the generator page, give the source image's URL, the width and height, the
+   format, quality and fit, and how long the URL lasts, then submit the form
+   (`POST /generate`). Width and height both empty or `0` keep the original
+   size; if only one of them is empty or `0`, that side is scaled to keep the
+   image's proportions.
+3. The page shows the URL, `https://<host>/v1/e/<token>/img.<format>`, and when
+   it expires. `<host>` is the host the page was opened on, and the URL starts
+   with `http` instead while `debug` is on. The name after the token is ignored
+   and only gives the URL a file extension, `jpg` for `orig`.
+
+The token holds the source's host, path and query and the size, format,
+quality, fit and expiry, encrypted with a key derived from `signing_key`. The
+source URL's scheme is not kept: the image is fetched like any other (see
+Routes), and the blocked networks still apply.
+
+How long the URL lasts is chosen on the page, from 1 minute to 1 year, or
+never. The expiry is fixed in the token when the URL is made and cannot be
+changed or revoked afterwards. Until then the image is served with a `max-age`
+that ends at the expiry (see Routes); after it the URL answers 410
+`URL has expired`. A URL made to last forever stops working only when
+`signing_key` changes: changing it makes every encrypted URL already handed out
+answer 400, and ends every login session.
+
 ### Image Metadata
 
 pixa decodes and re-encodes every image it serves, and removes all metadata from
@@ -165,7 +355,8 @@ Where:
 - `query` — source query string, empty string if none
 - `width` — requested width in pixels, `0` for original
 - `height` — requested height in pixels, `0` for original
-- `format` — output format (jpeg, png, webp, avif, gif, orig)
+- `format` — output format, one of those listed under Routes, with `original`
+  signed as `orig` and `jpg` as `jpeg`
 - `expiration` — the URL's `exp` query parameter, the Unix timestamp when
   the signature expires; a request whose `exp` is not a whole number, an
   empty `exp=` included, is refused with 400
@@ -204,6 +395,11 @@ and the URL is
 - **Suffix match**: `.example.com` — matches `cdn.example.com`,
   `images.example.com`, and `example.com`
 
+An IP address is matched exactly; write an IPv6 address without brackets. An
+entry that is neither a host name (letters, digits, hyphens, underscores and
+dots, with at most one leading dot) nor an IP address, such as one with a port
+or a `*.` wildcard, aborts startup.
+
 ### Configuration
 
 Every setting can be given as an environment variable, in a YAML config
@@ -221,14 +417,26 @@ startup naming it, as an unknown config key does. The one other accepted
 name is `PIXA_CONFIG_PATH`, the config file's path (like `--config`). The
 variables set by the file's `env:` section are checked the same way.
 
+pixa reads at most one config file: the one given with `--config` (or `-c`),
+otherwise the one `PIXA_CONFIG_PATH` names, otherwise the first of these that
+pixa finds: `/etc/pixa/config.yml`, `/etc/pixa/config.yaml`,
+`~/.config/pixa/config.yml`, `~/.config/pixa/config.yaml`, then `config.yml`
+and `config.yaml` in the working directory. A named file that does not exist,
+cannot be read or does not parse aborts startup. Of the files pixa looks for on
+its own, only one that does not exist is passed over, without a message. One
+that pixa cannot read or parse aborts startup, naming the file. So does one in a
+directory pixa may not enter, whether or not it is there, since pixa cannot
+tell. With no file, pixa uses the environment and the defaults.
+
 | Variable                             | Config key                      | Meaning                                                                      |
 | ------------------------------------ | ------------------------------- | ---------------------------------------------------------------------------- |
 | `PIXA_SIGNING_KEY`                   | `signing_key`                   | Required: secret for signed and encrypted URLs and login, 32+ characters     |
 | `PORT`                               | `port`                          | Port to listen on; default `8080`                                            |
 | `PIXA_STATE_DIR`                     | `state_dir`                     | Directory for the database and the disk cache; default `/var/lib/pixa`       |
 | `PIXA_DB_URL`                        | `db_url`                        | SQLite database URL; default `state.sqlite3` in the state directory          |
-| `PIXA_CACHE_MAX_BYTES`               | `cache_max_bytes`               | Disk cache limit in bytes; `0` disables it; default 75% of free space        |
+| `PIXA_CACHE_MAX_BYTES`               | `cache_max_bytes`               | Disk cache limit in bytes; `0` disables it; default 75% of (free + cached)   |
 | `PIXA_ALLOWLIST_HOSTS`               | `allowlist_hosts`               | Upstream hosts served without a signature                                    |
+| `PIXA_REFERER_BLOCKLIST`             | `referer_blocklist`             | Hosts whose pages the image routes refuse with 403, by `Referer`             |
 | `PIXA_BLOCKED_NETWORKS`              | `blocked_networks`              | CIDR ranges never fetched from, on top of the built-in ones                  |
 | `PIXA_TRUSTED_PROXIES`               | `trusted_proxies`               | CIDR ranges of proxies whose `X-Forwarded-For` is believed; default RFC 1918 |
 | `PIXA_ALLOW_HTTP`                    | `allow_http`                    | Allow plain-HTTP upstreams, for testing only; default `false`                |
@@ -238,7 +446,7 @@ variables set by the file's `env:` section are checked the same way.
 | `PIXA_UPSTREAM_FETCH_TIMEOUT`        | `upstream_fetch_timeout`        | Time allowed for one fetch from an upstream host; default `30s`              |
 | `PIXA_UPSTREAM_MAX_RESPONSE_SIZE`    | `upstream_max_response_size`    | Largest upstream response accepted, in bytes; default 50 MiB                 |
 | `PIXA_DOWNSTREAM_TIMEOUT`            | `downstream_timeout`            | Time allowed for answering one client request; default `60s`                 |
-| `PIXA_ACCESS_CONTROL_ALLOW_ORIGIN`   | `access_control_allow_origin`   | CORS origin allowed to read responses: `*` or one origin; default `*`        |
+| `PIXA_ACCESS_CONTROL_ALLOW_ORIGIN`   | `access_control_allow_origin`   | CORS origin allowed to read image responses: `*` or one origin; default `*`  |
 | `PIXA_METRICS_USERNAME`              | `metrics.username`              | Username for `/metrics`, which is served only when both are set              |
 | `PIXA_METRICS_PASSWORD`              | `metrics.password`              | Password for `/metrics`; set together with the username                      |
 | `PIXA_SENTRY_DSN`                    | `sentry_dsn`                    | Sentry DSN for error reporting; empty disables it                            |
@@ -247,8 +455,9 @@ variables set by the file's `env:` section are checked the same way.
 
 Key settings in more detail:
 
-- `access_control_allow_origin` — the origin a browser lets read pixa's
+- `access_control_allow_origin` — the origin a browser lets read the responses
-  responses, sent as the CORS `Access-Control-Allow-Origin` header: `*`, the
+  of the image routes, `/v1/image/` and `/v1/e/`, sent as the CORS
+  `Access-Control-Allow-Origin` header; no other route sends it. `*`, the
   default, is any site; otherwise one `http` or `https` origin such as
   `https://example.com`, whose host is a lowercase host name (letters,
   digits, hyphens and dots, with a letter in its last part) or an IP address
@@ -256,6 +465,18 @@ Key settings in more detail:
   that has no leading zero and is not the scheme's default. Any other value,
   including another scheme such as a browser extension's, aborts startup
 - `allowlist_hosts` — list of allowed upstream hosts
+- `referer_blocklist` — list of hosts whose pages may not show pixa's images, to
+  stop other sites hotlinking them. Entries are written and matched as for
+  `allowlist_hosts` (see Allowlist patterns), and an entry that is neither a
+  host name nor an IP address aborts startup. A request to `/v1/image/` or
+  `/v1/e/` whose `Referer` header names a listed host is refused with 403 before
+  its signature or token is checked and before the cache or the upstream host is
+  used, so it fetches nothing, and it is refused even when the image is cached.
+  A request with no `Referer`, or one that does not parse as a URL
+  with a host, is served, as many clients send none. So this is easily got
+  around: a site whose pages send no `Referer` (for example with
+  `Referrer-Policy: no-referrer`) is not stopped. It does not apply to the login
+  and generator pages. Default: empty
 - `blocked_networks` — list of CIDR ranges to refuse for SSRF protection,
   added to the always-enforced built-in ranges (loopback, private,
   link-local, CGNAT, benchmark, NAT64, and the like); an invalid CIDR
@@ -290,9 +511,17 @@ Key settings in more detail:
   waits for an upstream connection and for a processing slot (up to 10 seconds
   each), so keep it longer than `upstream_fetch_timeout` plus 20 seconds
 - `signing_key` — HMAC secret for URL signatures
+- `db_url` — the SQLite database to open; omitted, it is
+  `file:<state_dir>/state.sqlite3?_pragma=journal_mode(WAL)`, which keeps the
+  database in WAL mode. pixa adds `_pragma=busy_timeout(5000)` to any `db_url`,
+  so a write that finds another in progress waits up to five seconds for it
+  instead of failing. WAL mode comes only from the URL: keep
+  `_pragma=journal_mode(WAL)` in one you set
 - `cache_max_bytes` — disk cache size limit in bytes; `0` disables the
-  disk cache entirely; omitted defaults to 75% of the free space on
+  disk cache entirely; omitted defaults to 75% of the sum of the free space on
-  the filesystem containing `<state_dir>/cache/` (minimum 500 MiB)
+  the filesystem containing `<state_dir>/cache/` and the bytes of source and
+  transformed images the cache already holds, worked out at startup (minimum
+  500 MiB)
 - `upstream_connections` — the most connections to upstream hosts at once, all
   hosts together, on top of `upstream_connections_per_host`; default `64`. A
   fetch holds its connection until its image has been processed. A fetch that
@@ -305,14 +534,14 @@ Key settings in more detail:
   seconds for one to free up; if none does, and `downstream_timeout` has not
   ended first, it is answered 503 the same way
 - `maintenance_mode` — while `true`, the image routes (`/v1/image/` and
-  `/v1/e/`) answer every request with 503, a `Retry-After` header and a JSON
+  `/v1/e/`) answer every request for an image with 503, a `Retry-After` header
-  error body. The health check (`/.well-known/healthcheck.json`) still answers
+  and a JSON error body. The health check (`/.well-known/healthcheck.json`)
-  200 and reports `"maintenance_mode": true`. It stays 200 because the image's
+  still answers 200 and reports `"maintenance_mode": true`. It stays 200
-  Docker `HEALTHCHECK` requests it: a 503 there would make the container
+  because the image's Docker `HEALTHCHECK` requests it: a 503 there would make
-  unhealthy, and upaas marks a deploy failed when its container is unhealthy.
+  the container unhealthy, and upaas marks a deploy failed when its container
-  The login and URL generator pages and `/metrics` keep working
+  is unhealthy. The login and URL generator pages and `/metrics` keep working
 
-See `config.example.yml` for all options with defaults.
+See `configs/config.example.yml` for all options with defaults.
 
 ### Architecture
 
@@ -321,7 +550,10 @@ See `config.example.yml` for all options with defaults.
 - **Image processing**: govips (CGO wrapper for libvips)
 - **Database**: SQLite via modernc.org/sqlite
 - **Static assets**: embedded via `//go:embed`
-- **Metrics**: Prometheus
+- **Metrics**: Prometheus, at `/metrics`: generic HTTP request metrics
+  (duration, response size, requests in flight) and the Go runtime and process
+  metrics; requests are measured and `/metrics` is served only when
+  `metrics.username` and `metrics.password` are set
 - **Logging**: stdlib slog
 
 ## Entrypoints
@@ -344,8 +576,9 @@ them. We provide:
 - `script/check` — run test, lint, and fmt-check
 - `script/docker` — build the Docker image tagged via `script/projectname`
 - `script/docker-smoke` — build the image, start it, wait for it to be healthy
-- `script/cibuild` — CI entrypoint: `docker build .` (the Dockerfile
+- `script/cibuild` — CI entrypoint: `docker build .` with a new
-  runs the checks, so a green build implies a green repo)
+  `CHECK_EPOCH` on every run, so the Dockerfile's checks run instead of
+  coming from the build cache, and a green run implies a green repo
 - `script/precommit` — pre-commit checks (`go mod tidy` guard, then
   `script/check`)
 - `script/install-precommit` — install the git pre-commit hook that

TODO.md

@@ -3,6 +3,8 @@
 * branch per issue from `next`
 * do the work in Next Step
 * move Next Step to the top of Completed Steps
+* `TODO.md` merges with git's union merge (`.gitattributes`), which never
+  reports a conflict: read the merged entries after every merge or rebase
 * move the top item of Future Steps into Next Step
 * commit (`TODO.md` changes in the same commit as the work)
 * open a PR based on `next`
@@ -25,10 +27,223 @@ The disk cache is now size-bounded with LRU eviction
 
 # Next Step
 
-P2: security: referer blacklist
+P2: security: per-IP rate limiting on the image routes
 
 # Completed Steps
 
+- 2026-10-04 referer blocklist (closes #90): `referer_blocklist`
+  (`PIXA_REFERER_BLOCKLIST`) lists hosts, written and matched as for
+  `allowlist_hosts` with the same matcher; an entry of either list that is
+  neither a host name (letters, digits, hyphens, underscores and dots, with at
+  most one leading dot) nor an IP address, such as one with a port or a `*.`
+  wildcard, aborts startup naming the setting and the entry. Both image routes
+  refuse a request whose `Referer` names a listed host with 403 and a JSON error
+  before the signature, the cache and the upstream fetch, so it fetches nothing
+  and is refused whether or not the image is cached. A request with no
+  `Referer`, or one that does not parse as a URL with a host, is served, so the
+  list is easily got around; `README.md` and `configs/config.example.yml` say
+  so. It does not apply to the login and generator pages.
+- 2026-10-04 fewer files in the repository root (closes #97):
+  `config.example.yml` moved unchanged to `configs/config.example.yml`, and
+  `README.md`, the comments in `internal/config/config.go` and the startup error
+  for the placeholder signing key name the new path; `scripts/manual-test.sh`
+  and its directory are deleted, as the handler tests in `internal/handlers`
+  cover every check it made except two: fetching a real image from the
+  internet, and a URL made on the generator page with a `ttl` answering 410 once
+  the `ttl` has passed (https://git.eeqj.de/sneak/pixa/issues/199);
+  `CONVENTIONS.md` is deleted, as `REPO_POLICIES.md` links the canonical Go HTTP
+  server conventions.
+- 2026-10-04 SQLite writes no longer fail with "database is locked" (closes
+  #198): pixa adds `_pragma=busy_timeout(5000)` to every `db_url`, so a write
+  that finds another in progress on another connection waits up to five seconds
+  for it, and the default `db_url` turns on WAL mode with
+  `_pragma=journal_mode(WAL)`. The old default's `_journal_mode=WAL` is not a
+  parameter the driver reads, so the database was never in WAL mode.
+- 2026-10-04 `TestPeriodicReconciliationAdoptsFileThatAppearsAfterStartup`
+  only passes through a periodic pass (closes #189): it slept for three
+  eviction intervals before writing its file, and a startup pass still running
+  then could adopt the file itself. It now holds the test database's only
+  connection until the startup pass waits for it after walking the empty
+  variant directory, writes the file and lets the connection go, as
+  `TestEvictionRunsOnPeriodicSchedule` does, so only a periodic reconciliation
+  pass can adopt the file. Test only.
+- 2026-10-04 logging in, logging out, the URL generator and `/v1/e/` have
+  handler tests (closes #77): new tests in `internal/handlers`, with no
+  network, check that `GET /` without a login session shows the login form; a
+  wrong key shows it again with an error and sets no session cookie; the right
+  key answers 303 to `/` with a session cookie marked `Secure`, `HttpOnly` and
+  `SameSite=Strict`, with which `GET /` shows the generator page; `GET /logout`
+  answers 303 to `/` with an empty session cookie sent with `Max-Age=0`;
+  `POST /generate` without a login session answers 303 to `/`; `/v1/e/` serves
+  the image for a valid token, answers 410 for an expired one and 400 for one
+  with a character changed, cut short or made with another signing key; and a
+  URL made on the generator page is served by `/v1/e/`. No code changes.
+- 2026-10-04 `TODO.md` merges with git's union merge (closes #190): a root
+  `.gitattributes`, copied from `sneak/prompts`, marks it `merge=union`, so two
+  branches that each add an entry at the top of Completed Steps merge without a
+  conflict and keep both entries. Git now never reports a conflict in
+  `TODO.md`: a real one keeps both versions of the lines, and two entries that
+  share an identical line can end up one inside the other, which a rebase can
+  do to an entry already on `next`. The Workflow above says to read the merged
+  entries after every merge or rebase.
+- 2026-10-04 the default `cache_max_bytes` no longer shrinks as the cache fills
+  (closes #184): for an omitted key, the cache works out the limit when it
+  opens, after the database is open, as 75% of the sum of the free space on the
+  filesystem containing `<state_dir>/cache/` and what the cache already holds by
+  its own size accounting, at least 500 MiB, so a cache filled to its limit
+  keeps that limit across a restart. The computation and its tests moved from
+  `internal/config` to `internal/imgcache`; the config only records whether the
+  key was set.
+- 2026-10-04 `TestEvictionRunsOnPeriodicSchedule` no longer races the evictor
+  (closes #183): it wrote each variant file and then inserted its accounting row
+  by hand, and a reconciliation pass between the two adopted the file first, so
+  the insert failed. It now writes the files only, while holding the test
+  database's only connection so the evictor's startup pass waits after walking
+  the empty variant directory; a periodic reconciliation pass then adopts the
+  files and the eviction pass after it evicts them. No other test in
+  `internal/imgcache` inserts a row by hand after starting the evictor. Test
+  only.
+- 2026-10-04 a config file pixa cannot read aborts startup (closes #176): of the
+  places pixa looks for its config file on its own, only one where the file does
+  not exist is passed over; any other error, such as a directory on the path
+  that pixa may not enter, aborts startup naming the file, as a file that does
+  not parse already did.
+- 2026-10-04 `.golangci.yml` re-vendored from the canonical copy (closes #57):
+  the deprecated `gomodguard` is switched off, so lint runs print no
+  deprecation warning; its successor `gomodguard_v2` runs with the shared
+  module block list, and `depguard` keeps `net/http/httptest` out of files that
+  are not tests. The tree needed no code changes.
+- 2026-10-04 the Content-Security-Policy allows no inline script or style
+  (closes #125): `script-src` and `style-src` are `'self'` only. The generator
+  page's two inline `onclick` handlers moved into
+  `internal/static/generator.js`, attached with `addEventListener`; the bundled
+  Tailwind script, which built styles in the browser, is replaced by a small
+  hand-written `internal/static/style.css` with only the rules the login and
+  generator pages use, the templates carrying a few plain class names in place
+  of Tailwind's. No build step. The pages keep their layout, not every pixel of
+  it.
+- 2026-10-04 deployment guide and example Caddy config (closes #89):
+  "Deployment" in `README.md` says what the reverse proxy in front of pixa must
+  do (terminate TLS; pass `Host`, `Origin` and `Referer` on unchanged; set
+  `X-Forwarded-For`, with `trusted_proxies` to match; wait at least
+  `downstream_timeout`; optionally refuse `/metrics`) and what pixa does itself,
+  that the state directory needs a persistent volume and what `cache_max_bytes`
+  counts, the health check for a load balancer, what a stop does and its exit
+  codes, and what running outside Docker needs; `configs/Caddyfile` is the
+  example, checked with `caddy validate`.
+- 2026-10-04 the metrics basic auth, CORS preflight, request logging and
+  metrics recording have tests (closes #79): `MetricsAuth` on its own answers
+  401 with a challenge without credentials or with a wrong username or password
+  and lets the configured ones through; a preflight request gets `*` for any
+  origin when `access_control_allow_origin` is `*` and no
+  `Access-Control-Allow-Origin` from another origin than the configured one; a
+  `POST /` carrying the signing key leaves no trace of it in the request log
+  line, and the login handler's own log lines leave out the submitted key; the
+  metrics middleware on its own records a request it served, and the router
+  records nothing while no metrics username is set. Not tested: that the router
+  puts the basic auth in front of `/metrics` and records requests when a
+  metrics username is set. Only one test per package can set up `/metrics`, and
+  in `internal/server` that is `TestMaintenanceModeKeepsOtherRoutes`, which
+  needs the owner's approval to change; #180 holds it. Tests only; the basic
+  auth library already compares the password in constant time.
+- 2026-10-04 the image route's signature check and error answers are tested
+  (closes #76): new tests in `internal/handlers`, with no network, check the
+  status and JSON error body for a missing, wrong, unpadded, upper-case or
+  expired signature on a host not on the allowlist, or a valid one sent for
+  its parent domain, a sibling host, a subdomain or the host with another
+  domain appended (401), an unparseable path (400), `localhost` as the
+  upstream host (403) and an upstream error (502); that an allowlisted host is
+  served without a signature, another host only with a valid one; and the
+  answers of `/robots.txt` and the health check. No code changes.
+- 2026-10-04 request IDs returned and passed on, and `/v1/e/` revalidates
+  (closes #84): pixa's own `RequestID` middleware, in place of chi's, gives each
+  request an ID, its own `X-Request-ID` when that is at most 64 letters, digits,
+  `-`, `_` or `.` and a random one otherwise, stores it where chi's did and
+  sends it back as `X-Request-ID` on every response; the upstream fetch sends
+  that ID, and the "upstream fetched", "image converted" and "image served" log
+  lines carry it as `request_id`, a fetch shared by several requests carrying
+  the first request's; `/v1/e/` sets `ETag`, answers a matching `If-None-Match`
+  with 304 and is routed for `HEAD`, the `ETag` and 304 code being
+  `notModified`, which `/v1/image/` calls too; its token checks moved unchanged
+  into `parseImageEncRequest` to keep `HandleImageEnc` within the line limit; no
+  `Vary` is added, as no response depends on a request header except the image
+  routes' CORS headers, for which `go-chi/cors` already sends `Vary: Origin`;
+  `Vary: Accept` is left to #88.
+- 2026-10-04 routes, encrypted URLs and config file documented (closes #75):
+  "Routes" in `README.md` lists every route with its method, purpose, what it
+  needs and the status codes it answers with, and says `q` and `fit` are part
+  of what is cached; "Encrypted URLs" covers logging in, making one on the
+  generator page, how long it lasts and the 410 once it has expired;
+  "Configuration" gives the order in which pixa looks for its config file;
+  `config.example.yml` lists `db_url` and `env` and gives every key's default;
+  `scripts/manual-test.sh` is left to #97.
+- 2026-10-04 shutdown stops cache eviction in progress (closes #102):
+  `StartEviction` runs the eviction goroutine with its own context, which
+  `StopEviction` cancels, so a pass in progress stops at its next database
+  call, file, row or eviction candidate instead of running to completion, and
+  no pass starts after it, so a stop logs at most one warning;
+  `StopEviction` takes a context and, when that context ends before the
+  goroutine exits, stops waiting and returns its error; the handlers' stop hook
+  passes fx's stop context, so an eviction still running when fx's stop
+  deadline ends fails the stop and makes the exit code 1.
+- 2026-10-04 dead code in `internal/imgcache` is gone (closes #73): `Purge`,
+  which only returned an error and which nothing called, is no longer part of
+  the `ImageCache` interface or `Service`; the `SignatureValidator`,
+  `Allowlist` and `Storage` interfaces, which nothing implemented or used, are
+  deleted. Nothing else changes.
+- 2026-10-04 upstream host semaphores and variant `.meta` files no longer
+  outlive their use (closes #87): the fetcher counts the fetches holding or
+  waiting for a slot of each upstream host's semaphore and removes the host's
+  semaphore once none is left, so fetches from many hosts no longer leave one
+  semaphore each until restart; `VariantStorage.Delete` removes the variant's
+  `.meta` file along with it, a missing `.meta` file not being an error, and
+  `DeleteWithMeta`, which eviction called for that, is gone.
+- 2026-10-04 `README.md` matches the code (closes #74): "Storage" names the
+  cache directories pixa uses (`cache/sources`, `cache/metadata`,
+  `cache/variants`) and how files are named in each, and the comments in
+  `001_schema.sql` name the same paths; the routes and the signature section
+  list the same output formats, `jpg` and `original` included; the TLS
+  sentence names `allow_http` as its exception; "Metrics" says only generic
+  HTTP and Go runtime metrics exist, measured and served only when the metrics
+  username and password are set.
+- 2026-10-03 shutdown sets the exit code and waits for image processing
+  (closes #86): fx alone handles SIGINT and SIGTERM, and the server's own
+  signal handler is gone; fx's `Run` in `cmd/pixad` exits with the shutdown's
+  code: 0 for a signal, 1 when the HTTP server cannot listen or the app fails
+  to start or to stop; the server's stop hook, which fx waits for, stops the
+  HTTP server, waits for the images still being processed, both within 5
+  seconds, then flushes Sentry; images still being processed after that are
+  logged with their count and make the exit code 1; a Sentry DSN that cannot be
+  used fails startup, so the stop hooks of what had already started run,
+  instead of exiting the process from a goroutine; the eviction loop is left to
+  #102.
+- 2026-10-03 every `script/cibuild` and `script/docker` run executes the checks
+  (closes #101): the `Dockerfile` declares `CHECK_EPOCH` above `make fmt-check`
+  and `make lint` in the lint stage and above `make test` in the build stage,
+  and each of those steps names it in its command; both scripts pass a new value
+  on every run, so Docker runs the checks instead of reusing cached results,
+  while the `script/bootstrap` steps stay cached; a plain `docker build .` still
+  works, leaves it empty, and reuses the check steps only for an identical build
+  context; the `script/cibuild` comment and `README.md` no longer say that any
+  successful build implies a green repo.
+- 2026-09-29 share concurrent misses (closes #65): requests that miss the same
+  variant at once (the same cache key, so quality and fit included) share one
+  upstream fetch or cached source read and one transcode through
+  `golang.org/x/sync/singleflight`; the first request's processing ignores its
+  cancellation but keeps its deadline, and the others wait for its image or
+  error holding no upstream connection or processing slot, and stop waiting
+  when their own context ends; the request doing the processing waits for it
+  even then, up to its deadline; a request whose context has already ended
+  starts nothing; each request counts one miss, and the processing counts its
+  fetch and transcode once; a panic while processing is reported to Sentry when
+  `sentry_dsn` is set and becomes an error for every waiting request instead of
+  stopping pixad; documented in `README.md`.
+- 2026-09-29 only the image routes send CORS headers (closes #98): the CORS
+  middleware, with the `access_control_allow_origin` origin, moved from the
+  router root onto a `/v1` subrouter holding `/v1/image/` and `/v1/e/`, where it
+  still answers a preflight `OPTIONS` request; the login and URL generator
+  pages, `/metrics` and the other routes send no `Access-Control-Allow-Origin`;
+  documented in `README.md` and `config.example.yml`.
 - 2026-10-02 a plain `docker build .` stamps the tag or short commit, not
   `dev` (closes #166): `.dockerignore` lets `.git` into the build context,
   without `.git/config`; with no `VERSION` build argument the `Dockerfile`
@@ -358,12 +573,10 @@ P2: security: referer blacklist
 # Future Steps
 
 - P2: security
-    - per-IP rate limiting on the image routes
     - per-origin rate limiting
 - P2: HTTP response handling
     - Last-Modified headers
     - Vary header for content negotiation
-    - X-Request-ID propagation
 - P2: auto format selection (format=auto based on Accept header)
 - P2: configuration
     - YAML config file support
@@ -373,8 +586,3 @@ P2: security: referer blacklist
     - Prometheus performance metrics
     - integration tests for the image proxy flow
     - load tests to verify the 1k to 5k req/s target
-- P2: documentation
-    - configuration options
-    - API endpoints
-    - deployment guide
-    - example nginx or caddy reverse proxy config

cmd/pixad/main.go

@@ -4,6 +4,8 @@ package main
 import (
 	"fmt"
 	"os"
+	"os/signal"
+	"syscall"
 
 	"github.com/spf13/cobra"
 	"go.uber.org/fx"
@@ -45,6 +47,9 @@ func run(_ *cobra.Command, _ []string) {
 		_ = os.Setenv("PIXA_CONFIG_PATH", configPath)
 	}
 
+	// A write to a closed stdout or stderr must not end the process.
+	signal.Ignore(syscall.SIGPIPE)
+
 	fx.New(
 		fx.Provide(
 			config.New,

configs/Caddyfile

@@ -0,0 +1,17 @@
+# Example Caddy config for running pixa behind Caddy; see "Deployment" in
+# README.md. Replace images.example.com with pixa's public host name, and
+# 127.0.0.1:8080 with the address Caddy reaches pixa on.
+#
+# Caddy gets and renews the TLS certificate for the host name, passes the
+# Host, Origin and Referer headers on unchanged, sets X-Forwarded-For to the
+# client's address, and waits for pixa's answer with no time limit of its
+# own, so pixa's downstream_timeout is what ends a slow request.
+
+images.example.com
+
+# pixa asks for metrics.username and metrics.password on /metrics. This
+# line also keeps it off the public address, for a scraper that reaches
+# pixa directly; remove it to read /metrics through Caddy.
+respond /metrics 404
+
+reverse_proxy 127.0.0.1:8080

configs/config.example.yml

@@ -12,28 +12,42 @@
 # Durations are Go duration strings such as 30s or 2m and must be
 # positive; a bare number has no unit and aborts startup. Sizes are a
 # whole number of bytes.
+#
+# A key left out takes the default its comment gives.
 
-# Server settings
+# Port to listen on (default: 8080)
 port: 8080
+
+# Debug logging and plain-HTTP local development (default: false)
 debug: false
 
 # While true, the image routes (/v1/image/ and /v1/e/) answer every request
-# with 503 and a Retry-After header. The health check keeps answering 200 and
+# for an image with 503 and a Retry-After header. The health check keeps
-# reports maintenance_mode as true. It stays 200 because the image's Docker
+# answering 200 and reports maintenance_mode as true. It stays 200 because
-# HEALTHCHECK requests it: a 503 there would make the container unhealthy, and
+# the image's Docker HEALTHCHECK requests it: a 503 there would make the
-# upaas marks a deploy failed when its container is unhealthy.
+# container unhealthy, and upaas marks a deploy failed when its container is
+# unhealthy. (default: false)
 maintenance_mode: false
 
 # Data directory for SQLite database and cache files
+# (default: /var/lib/pixa)
 state_dir: ./data
 
+# SQLite database URL (default:
+# file:<state_dir>/state.sqlite3?_pragma=journal_mode(WAL)). pixa adds
+# _pragma=busy_timeout(5000) to it. An empty value aborts startup; leave the
+# key out to use the default.
+# db_url: "file:./data/state.sqlite3?_pragma=journal_mode(WAL)"
+
 # Image proxy settings
 # HMAC signing key for URL signatures (required, at least 32 characters)
 # Generate with: openssl rand -base64 32
 signing_key: "CHANGE_ME_generate_with_openssl_rand_base64_32"
 
-# Hosts that don't require signatures
+# Hosts that don't require signatures (default: none)
 # Use "." prefix for wildcard subdomain matching (e.g., ".example.com" matches "cdn.example.com")
+# An entry that is neither a host name nor an IP address (IPv6 without
+# brackets), such as one with a port or a "*." wildcard, aborts startup.
 allowlist_hosts:
   - s3.sneak.cloud
   - static.sneak.cloud
@@ -41,11 +55,21 @@ allowlist_hosts:
   - github.com
   - user-images.githubusercontent.com
 
+# Hosts whose pages may not show pixa's images, written as for
+# allowlist_hosts. A request to /v1/image/ or /v1/e/ whose Referer header
+# names one of them is answered 403 before anything is fetched, even when
+# the image is cached. A request with no Referer, or one that does not
+# parse, is served, so a site whose pages send no Referer is not stopped.
+# The login and generator pages are not covered. (default: none)
+# referer_blocklist:
+#   - leech.example
+#   - .hotlinker.example
+
 # Additional CIDR ranges to refuse when fetching upstream, extending the
 # SSRF protection. These are added to the always-enforced built-in ranges
 # (loopback, RFC 1918 private, link-local, CGNAT, benchmark, NAT64, and
 # similar), never replacing them. Each entry must be a valid CIDR in IPv4
-# or IPv6 form; an invalid entry aborts startup.
+# or IPv6 form; an invalid entry aborts startup. (default: none)
 # blocked_networks:
 #   - 100.64.0.0/10
 #   - 2001:db8::/32
@@ -72,6 +96,7 @@ allowlist_hosts:
 #   - 2001:db8::/32
 
 # Allow HTTP upstream (only for testing, always use HTTPS in production)
+# (default: false)
 allow_http: false
 
 # Maximum concurrent connections per upstream host (default: 20)
@@ -103,8 +128,9 @@ upstream_max_response_size: 52428800
 # longer than upstream_fetch_timeout plus 20 seconds.
 downstream_timeout: 60s
 
-# The origin a browser lets read pixa's responses, sent as the CORS
+# The origin a browser lets read the responses of the image routes,
-# Access-Control-Allow-Origin header: "*" (the default) is any site;
+# /v1/image/ and /v1/e/, sent as the CORS Access-Control-Allow-Origin
+# header; no other route sends it. "*" (the default) is any site;
 # otherwise one http or https origin such as https://example.com, whose
 # host is a lowercase host name (letters, digits, hyphens and dots, with a
 # letter in its last part) or an IP address (IPv6 in brackets, in its
@@ -115,15 +141,22 @@ access_control_allow_origin: "*"
 
 # Maximum disk cache size in bytes. Explicit values are used exactly as
 # given; 0 disables the disk cache entirely (every request fetches and
-# processes uncached). When omitted, the default is 75% of the free
+# processes uncached). When omitted, the default is 75% of the sum of
-# space on the filesystem containing <state_dir>/cache/ at startup,
+# the free space on the filesystem containing <state_dir>/cache/ and
+# the bytes of images the cache already holds, worked out at startup,
 # with a minimum of 500 MiB.
 # cache_max_bytes: 10737418240
 
-# Sentry error reporting (optional)
+# Sentry DSN for error reporting (default: empty, which turns it off)
 sentry_dsn: ""
 
-# Metrics endpoint authentication (optional)
+# Username and password for /metrics, set together (default: unset). Metrics
+# are measured and /metrics is served only when both are set.
 # metrics:
 #   username: "admin"
 #   password: "secret"
+
+# Environment variables set while this file loads, as described at the top
+# (default: none)
+# env:
+#   PIXA_DEBUG: "true"

go.mod

@@ -20,6 +20,7 @@ require (
 	github.com/spf13/cobra v1.10.2
 	go.uber.org/fx v1.24.0
 	golang.org/x/crypto v0.41.0
+	golang.org/x/sync v0.19.0
 	modernc.org/sqlite v1.42.2
 )
 
@@ -134,7 +135,6 @@ require (
 	golang.org/x/image v0.34.0 // indirect
 	golang.org/x/net v0.43.0 // indirect
 	golang.org/x/oauth2 v0.30.0 // indirect
-	golang.org/x/sync v0.19.0 // indirect
 	golang.org/x/sys v0.36.0 // indirect
 	golang.org/x/term v0.34.0 // indirect
 	golang.org/x/text v0.32.0 // indirect

internal/config/cache_max_bytes_internal_test.go

@@ -1,26 +1,10 @@
 package config
 
 import (
-	"errors"
-	"log/slog"
-	"os"
-	"path/filepath"
 	"strings"
 	"testing"
 )
 
-// Static errors returned by the stub free-space probes below.
-var (
-	errTestStatfsFailed     = errors.New("statfs failed")
-	errTestProbeNotExpected = errors.New("probe must not be called")
-)
-
-// discardLogger returns a logger that swallows all output, for tests
-// that exercise code paths which log.
-func discardLogger() *slog.Logger {
-	return slog.New(slog.DiscardHandler)
-}
-
 // TestCacheMaxBytesExplicitValueUsedWithoutFloor verifies that an
 // explicitly configured cache_max_bytes value is used exactly as
 // given: the 500 MiB floor applies only to the computed default, never
@@ -151,155 +135,30 @@ func TestCacheMaxBytesInvalidValuesAbortStartup(t *testing.T) {
 	}
 }
 
-// TestComputeDefaultCacheMaxBytesUses75PercentOfFreeSpace verifies the
+// TestCacheMaxBytesExplicitIsRecorded verifies that an omitted
-// computed default is 75% of the probed free space when that exceeds
+// cache_max_bytes is recorded as not explicit, so the cache works out
-// the floor.
+// the default when it opens, and that an explicit zero is recorded as
-func TestComputeDefaultCacheMaxBytesUses75PercentOfFreeSpace(t *testing.T) {
+// explicit, so it disables the disk cache instead.
+func TestCacheMaxBytesExplicitIsRecorded(t *testing.T) {
 	t.Parallel()
 
-	// 4 GiB free -> 3 GiB default.
+	signingKeyLine := "signing_key: " + validTestSigningKey + "\n"
-	probe := func(string) (uint64, error) { return 4294967296, nil }
 
-	got, err := ComputeDefaultCacheMaxBytes(t.TempDir(), probe)
+	omitted, err := configFromYAML(t, signingKeyLine)
-	if err != nil {
-		t.Fatalf("ComputeDefaultCacheMaxBytes returned error: %v", err)
-	}
-
-	if got != 3221225472 {
-		t.Errorf("ComputeDefaultCacheMaxBytes = %d, want 3221225472 (75%% of 4 GiB)",
-			got)
-	}
-}
-
-// TestComputeDefaultCacheMaxBytesAppliesFloorToComputedDefault
-// verifies that when 75% of free space is below 500 MiB, the computed
-// default is floored at DefaultCacheMaxBytesFloor.
-func TestComputeDefaultCacheMaxBytesAppliesFloorToComputedDefault(t *testing.T) {
-	t.Parallel()
-
-	cases := []struct {
-		name      string
-		freeBytes uint64
-	}{
-		{name: "100 MiB free", freeBytes: 104857600},
-		{name: "zero free", freeBytes: 0},
-		{name: "just below floor threshold", freeBytes: 699050665},
-	}
-
-	for _, tc := range cases {
-		t.Run(tc.name, func(t *testing.T) {
-			t.Parallel()
-
-			probe := func(string) (uint64, error) { return tc.freeBytes, nil }
-
-			got, err := ComputeDefaultCacheMaxBytes(t.TempDir(), probe)
-			if err != nil {
-				t.Fatalf("ComputeDefaultCacheMaxBytes returned error: %v", err)
-			}
-
-			if got != DefaultCacheMaxBytesFloor {
-				t.Errorf("ComputeDefaultCacheMaxBytes = %d, want floor %d",
-					got, DefaultCacheMaxBytesFloor)
-			}
-		})
-	}
-}
-
-// TestComputeDefaultCacheMaxBytesPropagatesProbeError verifies that a
-// failing free-space probe produces an error naming the config key,
-// instead of a silently wrong default.
-func TestComputeDefaultCacheMaxBytesPropagatesProbeError(t *testing.T) {
-	t.Parallel()
-
-	probe := func(string) (uint64, error) { return 0, errTestStatfsFailed }
-
-	_, err := ComputeDefaultCacheMaxBytes(t.TempDir(), probe)
-	if err == nil {
-		t.Fatal("probe failure must produce an error, got nil")
-	}
-
-	t.Logf("got expected error: %v", err)
-
-	if !strings.Contains(err.Error(), keyCacheMaxBytes) {
-		t.Errorf("error %q does not name the config key cache_max_bytes", err.Error())
-	}
-}
-
-// TestResolveCacheMaxBytesComputesDefaultWhenOmitted verifies that an
-// omitted cache_max_bytes key resolves to the computed default, that
-// the probe is pointed at <state_dir>/cache/ (which must be created
-// first so statfs measures the right filesystem), and that the result
-// lands on the Config.
-func TestResolveCacheMaxBytesComputesDefaultWhenOmitted(t *testing.T) {
-	t.Parallel()
-
-	c, err := configFromYAML(t, "signing_key: "+validTestSigningKey+"\n")
 	if err != nil {
 		t.Fatalf("minimal config should be valid, got error: %v", err)
 	}
 
-	c.StateDir = t.TempDir()
+	if omitted.CacheMaxBytesExplicit {
-	wantCacheDir := filepath.Join(c.StateDir, "cache")
+		t.Error("omitted cache_max_bytes recorded as explicit")
-
-	var probedPath string
-
-	// 4 GiB free -> 3 GiB default.
-	probe := func(path string) (uint64, error) {
-		probedPath = path
-
-		return 4294967296, nil
 	}
 
-	err = c.resolveCacheMaxBytes(discardLogger(), probe)
+	zero, err := configFromYAML(t, signingKeyLine+"cache_max_bytes: 0\n")
 	if err != nil {
-		t.Fatalf("resolveCacheMaxBytes returned error: %v", err)
+		t.Fatalf("cache_max_bytes: 0 must be accepted, got error: %v", err)
 	}
 
-	if c.CacheMaxBytes != 3221225472 {
+	if !zero.CacheMaxBytesExplicit {
-		t.Errorf("CacheMaxBytes = %d, want computed default 3221225472",
+		t.Error("cache_max_bytes: 0 not recorded as explicit")
-			c.CacheMaxBytes)
-	}
-
-	if probedPath != wantCacheDir {
-		t.Errorf("free space probed at %q, want cache directory %q",
-			probedPath, wantCacheDir)
-	}
-
-	info, err := os.Stat(wantCacheDir)
-	if err != nil || !info.IsDir() {
-		t.Errorf("cache directory %q was not created before probing: info=%v err=%v",
-			wantCacheDir, info, err)
-	}
-}
-
-// TestResolveCacheMaxBytesDoesNotOverrideExplicitValue verifies that
-// an explicitly configured value survives resolution untouched and
-// that the free-space probe is never consulted for it.
-func TestResolveCacheMaxBytesDoesNotOverrideExplicitValue(t *testing.T) {
-	t.Parallel()
-
-	yamlContent := "signing_key: " + validTestSigningKey + "\ncache_max_bytes: 1024\n"
-
-	c, err := configFromYAML(t, yamlContent)
-	if err != nil {
-		t.Fatalf("explicit cache_max_bytes must be accepted, got error: %v", err)
-	}
-
-	c.StateDir = t.TempDir()
-
-	probe := func(string) (uint64, error) {
-		t.Error("free-space probe must not be consulted for explicit values")
-
-		return 0, errTestProbeNotExpected
-	}
-
-	err = c.resolveCacheMaxBytes(discardLogger(), probe)
-	if err != nil {
-		t.Fatalf("resolveCacheMaxBytes returned error: %v", err)
-	}
-
-	if c.CacheMaxBytes != 1024 {
-		t.Errorf("CacheMaxBytes = %d, want explicit 1024 (no floor, no recompute)",
-			c.CacheMaxBytes)
 	}
 }

internal/config/cachesize.go

@@ -1,116 +0,0 @@
-package config
-
-import (
-	"fmt"
-	"log/slog"
-	"math"
-	"os"
-	"path/filepath"
-	"syscall"
-)
-
-// DefaultCacheMaxBytesFloor is the minimum computed default for the
-// cache_max_bytes setting: 500 MiB. The floor applies only to the
-// computed default (when the key is omitted from the configuration),
-// never to explicitly configured values.
-const DefaultCacheMaxBytesFloor int64 = 524288000
-
-// cacheDirPerms is the permission mode for the cache directory created
-// before probing free space, matching the state directory permissions.
-const cacheDirPerms = 0o750
-
-// freeSpaceFractionNumerator and freeSpaceFractionDenominator express
-// the 75% share of free space used for the computed default limit as
-// integer arithmetic (dividing before multiplying avoids overflow).
-const (
-	freeSpaceFractionNumerator   uint64 = 3
-	freeSpaceFractionDenominator uint64 = 4
-)
-
-// FreeSpaceProbeFunc reports the number of free bytes available on the
-// filesystem containing path. It is a function type so tests can
-// inject a fake probe instead of depending on the host disk.
-type FreeSpaceProbeFunc func(path string) (uint64, error)
-
-// defaultFreeSpaceProbe reports free filesystem bytes via statfs on
-// the given path, as available to unprivileged processes.
-func defaultFreeSpaceProbe(path string) (uint64, error) {
-	var stat syscall.Statfs_t
-
-	err := syscall.Statfs(path, &stat)
-	if err != nil {
-		return 0, err
-	}
-
-	if stat.Bsize < 0 {
-		return 0, fmt.Errorf("%w %d for %q", errNegativeBlockSize, stat.Bsize, path)
-	}
-
-	blockSize := uint64(stat.Bsize)
-
-	return stat.Bavail * blockSize, nil
-}
-
-// ComputeDefaultCacheMaxBytes returns the default cache size limit for
-// the filesystem containing cacheDir: 75% of the free bytes reported
-// by probe, with a floor of DefaultCacheMaxBytesFloor.
-func ComputeDefaultCacheMaxBytes(
-	cacheDir string, probe FreeSpaceProbeFunc,
-) (int64, error) {
-	freeBytes, err := probe(cacheDir)
-	if err != nil {
-		return 0, fmt.Errorf("config key %q: cannot determine free space for %q: %w",
-			"cache_max_bytes", cacheDir, err)
-	}
-
-	computed := freeBytes / freeSpaceFractionDenominator * freeSpaceFractionNumerator
-	computed = min(computed, math.MaxInt64)
-
-	// gosec cannot see that min() above bounds computed, so it reads
-	// this conversion as potentially overflowing. It cannot: computed is
-	// at most math.MaxInt64 on every path here.
-	//nolint:gosec // G115: clamped to MaxInt64 by min above
-	limit := int64(computed)
-	limit = max(limit, DefaultCacheMaxBytesFloor)
-
-	return limit, nil
-}
-
-// resolveCacheMaxBytes finalizes CacheMaxBytes after state_dir
-// validation: an explicitly configured value is kept as-is (no floor
-// applies), while an omitted key receives the computed default based
-// on free space in <state_dir>/cache/. The cache directory is created
-// first so statfs measures the filesystem that will actually hold the
-// cache. The effective limit is logged either way.
-func (c *Config) resolveCacheMaxBytes(
-	log *slog.Logger, probe FreeSpaceProbeFunc,
-) error {
-	if !c.cacheMaxBytesExplicit {
-		cacheDir := filepath.Join(c.StateDir, "cache")
-
-		err := os.MkdirAll(cacheDir, cacheDirPerms)
-		if err != nil {
-			return fmt.Errorf("config key %q: cannot create cache directory %q: %w",
-				keyCacheMaxBytes, cacheDir, err)
-		}
-
-		limit, err := ComputeDefaultCacheMaxBytes(cacheDir, probe)
-		if err != nil {
-			return err
-		}
-
-		c.CacheMaxBytes = limit
-
-		log.Info("computed default cache size limit from free space",
-			"cache_max_bytes", limit,
-			"cache_dir", cacheDir,
-		)
-	}
-
-	log.Info("effective cache size limit",
-		"cache_max_bytes", c.CacheMaxBytes,
-		"cache_disabled", c.CacheMaxBytes == 0,
-	)
-
-	return nil
-}

internal/config/config.go

@@ -4,16 +4,19 @@ package config
 import (
 	"errors"
 	"fmt"
+	"io/fs"
 	"log/slog"
 	"math"
 	"net/netip"
 	"net/url"
 	"os"
 	"path/filepath"
+	"regexp"
 	"runtime"
 	"sort"
 	"strconv"
 	"strings"
+	"syscall"
 	"time"
 
 	"git.eeqj.de/sneak/smartconfig"
@@ -46,6 +49,7 @@ const (
 	keyMetricsPassword            = "metrics.password"
 	keySigningKey                 = "signing_key"
 	keyAllowlistHosts             = "allowlist_hosts"
+	keyRefererBlocklist           = "referer_blocklist"
 	keyAllowHTTP                  = "allow_http"
 	keyUpstreamConnectionsPerHost = "upstream_connections_per_host"
 	keyUpstreamConnections        = "upstream_connections"
@@ -60,7 +64,7 @@ const (
 )
 
 // placeholderSigningKey is the dummy signing_key shipped in
-// config.example.yml. It is 45 characters, so it passes the length
+// configs/config.example.yml. It is 45 characters, so it passes the length
 // check, but it is public in this repository and must be rejected at
 // startup so no deployment ever signs URLs with it.
 const placeholderSigningKey = "CHANGE_ME_generate_with_openssl_rand_base64_32"
@@ -86,23 +90,20 @@ var (
 	errMustBeAtLeastOne  = errors.New("must be at least 1")
 	errValueTooShort     = errors.New("value too short")
 	errPlaceholderKey    = errors.New(
-		"is the placeholder from config.example.yml; " +
+		"is the placeholder from configs/config.example.yml; " +
 			"generate a real key with: openssl rand -base64 32")
 	errMustBeSetTogether = errors.New("must be set together")
 	errMustNotBeNegative = errors.New("must not be negative")
 	errOverflowsInt64    = errors.New("overflows a 64-bit integer")
-	errNegativeBlockSize = errors.New(
+	errValueNull         = errors.New(
-		"statfs reported negative block size")
-	errValueNull = errors.New(
 		"value is null; omit the key entirely to use the default")
 	errValuesNull = errors.New(
 		"value is null; omit a key entirely to use its default")
-	errNotBareHostname = errors.New(
+	errNotAHost = errors.New("must be a host name such as " +
-		"must be a bare hostname without scheme, path, or whitespace")
+		"cdn.example.com or .example.com, or an IP address")
-	errNoHostnameLabels = errors.New("contains no hostname labels")
+	errNotADuration   = errors.New("not a duration such as 30s or 2m")
-	errNotADuration     = errors.New("not a duration such as 30s or 2m")
+	errMustBePositive = errors.New("must be positive")
-	errMustBePositive   = errors.New("must be positive")
+	errNotAnOrigin    = errors.New(
-	errNotAnOrigin      = errors.New(
 		`not "*" or an origin such as https://example.com`)
 )
 
@@ -130,6 +131,10 @@ type Config struct {
 	AllowHTTP                  bool     // Allow non-TLS upstream (testing only)
 	UpstreamConnectionsPerHost int      // Max concurrent connections per upstream host
 
+	// RefererBlocklist holds host patterns, matched as AllowlistHosts is: the
+	// image routes refuse a request whose Referer names a matching host.
+	RefererBlocklist []string
+
 	// UpstreamConnections is the most concurrent connections to all
 	// upstream hosts together, on top of the per-host limit.
 	// MaxConcurrentProcessing is the most images processed at once.
@@ -169,18 +174,19 @@ type Config struct {
 	// address, and an explicit list replaces the default.
 	TrustedProxies []netip.Prefix
 
-	// CacheMaxBytes is the disk cache size limit in bytes. Zero
+	// CacheMaxBytes is the disk cache size limit in bytes. Only an
-	// disables the disk cache entirely. When cache_max_bytes is
+	// explicit zero (CacheMaxBytesExplicit true) disables the disk
-	// omitted from the configuration, this holds the computed default
+	// cache. Zero with CacheMaxBytesExplicit false means
-	// (75% of free space on the filesystem containing
+	// cache_max_bytes was omitted, and the cache works out the default
-	// <state_dir>/cache/, floored at DefaultCacheMaxBytesFloor).
+	// limit when it opens.
 	CacheMaxBytes int64
 
-	// cacheMaxBytesExplicit records whether cache_max_bytes was
+	// CacheMaxBytesExplicit records whether cache_max_bytes was
 	// explicitly set, in the environment or the configuration file.
-	// Explicit values are used exactly as given; only an omitted key
+	// Explicit values are used exactly as given; for an omitted key the
-	// gets the computed default (and its floor) in resolveCacheMaxBytes.
+	// cache works out the default limit when it opens (see
-	cacheMaxBytesExplicit bool
+	// imgcache.CacheConfig.UseDefaultMaxBytes).
+	CacheMaxBytesExplicit bool
 }
 
 // New creates a new Config instance from the environment and the
@@ -217,9 +223,13 @@ func New(_ fx.Lifecycle, params Params) (*Config, error) {
 		return nil, err
 	}
 
-	err = c.resolveCacheMaxBytes(log, defaultFreeSpaceProbe)
+	// An omitted cache_max_bytes is worked out and logged when the
-	if err != nil {
+	// cache opens.
-		return nil, err
+	if c.CacheMaxBytesExplicit {
+		log.Info("effective cache size limit",
+			"cache_max_bytes", c.CacheMaxBytes,
+			"cache_disabled", c.CacheMaxBytes == 0,
+		)
 	}
 
 	if c.Debug {
@@ -265,7 +275,6 @@ func newFromSmartConfig(sc *smartconfig.Config) (*Config, error) {
 	}
 
 	loader := &strictLoader{sc: sc}
-
 	c := &Config{
 		Debug:           loader.boolVal(keyDebug, false),
 		MaintenanceMode: loader.boolVal(keyMaintenanceMode, false),
@@ -293,16 +302,17 @@ func newFromSmartConfig(sc *smartconfig.Config) (*Config, error) {
 			keyAccessControlAllowOrigin, DefaultAccessControlAllowOrigin),
 		DownstreamTimeout: loader.durationVal(
 			keyDownstreamTimeout, DefaultDownstreamTimeout),
-		CacheMaxBytes:   loader.int64Val(keyCacheMaxBytes, 0),
+		CacheMaxBytes:    loader.int64Val(keyCacheMaxBytes, 0),
-		BlockedNetworks: blockedNetworks,
+		BlockedNetworks:  blockedNetworks,
-		TrustedProxies:  trustedProxies,
+		TrustedProxies:   trustedProxies,
+		RefererBlocklist: loader.hostListVal(keyRefererBlocklist),
 	}
 
-	// The computed default for cache_max_bytes needs a validated
+	// The default for an omitted cache_max_bytes is worked out when
-	// state_dir, so it is resolved later (resolveCacheMaxBytes); here
+	// the cache opens; here we only record whether the operator set
-	// we only record whether the operator set the key explicitly.
+	// the key explicitly.
 	if _, present := lookupValue(sc, keyCacheMaxBytes); present {
-		c.cacheMaxBytesExplicit = true
+		c.CacheMaxBytesExplicit = true
 	}
 
 	// Build DBURL from StateDir if not explicitly set. The derived URL
@@ -315,7 +325,8 @@ func newFromSmartConfig(sc *smartconfig.Config) (*Config, error) {
 				settingName(keyDBURL), errValueEmpty)
 		}
 
-		c.DBURL = fmt.Sprintf("file:%s/state.sqlite3?_journal_mode=WAL", c.StateDir)
+		// The driver sets the journal mode only through a _pragma parameter.
+		c.DBURL = fmt.Sprintf("file:%s/state.sqlite3?_pragma=journal_mode(WAL)", c.StateDir)
 	}
 
 	if loader.err != nil {
@@ -414,7 +425,8 @@ func isKnownConfigKey(key string) bool {
 		keyUpstreamConnectionsPerHost, keyUpstreamConnections,
 		keyMaxConcurrentProcessing, keyCacheMaxBytes, keyBlockedNetworks,
 		keyTrustedProxies, keyAccessControlAllowOrigin, keyUpstreamFetchTimeout,
-		keyUpstreamMaxResponseSize, keyDownstreamTimeout, "env":
+		keyUpstreamMaxResponseSize, keyDownstreamTimeout, keyRefererBlocklist,
+		"env":
 		return true
 	}
 
@@ -437,6 +449,7 @@ func envVarNames() map[string]string {
 		keyMetricsPassword:            "PIXA_METRICS_PASSWORD",
 		keySigningKey:                 "PIXA_SIGNING_KEY",
 		keyAllowlistHosts:             "PIXA_ALLOWLIST_HOSTS",
+		keyRefererBlocklist:           "PIXA_REFERER_BLOCKLIST",
 		keyAllowHTTP:                  "PIXA_ALLOW_HTTP",
 		keyUpstreamConnectionsPerHost: "PIXA_UPSTREAM_CONNECTIONS_PER_HOST",
 		keyUpstreamConnections:        "PIXA_UPSTREAM_CONNECTIONS",
@@ -548,8 +561,8 @@ func (c *Config) ensureStateDirWritable() error {
 }
 
 // validateSigningKey checks that the signing key is present, long
-// enough, and not the public placeholder from config.example.yml. The
+// enough, and not the public placeholder from configs/config.example.yml.
-// key value itself is never echoed in error messages.
+// The key value itself is never echoed in error messages.
 func (c *Config) validateSigningKey() error {
 	if c.SigningKey == "" {
 		return fmt.Errorf("%s: %w", settingName(keySigningKey), errValueRequired)
@@ -606,7 +619,7 @@ func (c *Config) validate() error {
 	}
 
 	for _, host := range c.AllowlistHosts {
-		err := validateAllowlistHost(host)
+		err := validateHostPattern(keyAllowlistHosts, host)
 		if err != nil {
 			return err
 		}
@@ -729,25 +742,24 @@ func (c *Config) validateConcurrencyLimits() error {
 	return nil
 }
 
-// validateAllowlistHost checks that an allowlist_hosts entry is a bare
+// hostNamePattern matches a host name: letters, digits, hyphens, underscores
-// hostname, optionally with a leading dot for suffix matching. URLs,
+// and dots, optionally after one leading dot.
-// paths, and whitespace indicate a misconfigured entry. An entry with
+var hostNamePattern = regexp.MustCompile(`^\.?[A-Za-z0-9_-][A-Za-z0-9_.-]*$`)
-// no hostname labels (such as ".") is rejected: the allowlist matcher
+
-// treats a leading dot as a suffix pattern, so a bare "." would match
+// validateHostPattern checks that an entry of the named key, allowlist_hosts
-// any upstream host written in FQDN trailing-dot form and effectively
+// or referer_blocklist, is an IP address or a host name, the host name
-// disable URL signing.
+// optionally with one leading dot for suffix matching. Anything else, such as
-func validateAllowlistHost(host string) error {
+// a URL, a port or a "*." wildcard, can never match a host name that resolves,
-	if strings.Contains(host, "://") || strings.ContainsAny(host, "/ \t") {
+// so it is refused.
-		return fmt.Errorf("%s: entry %q %w",
+// So is "." alone: the allowlist matcher would match it against any host
-			settingName(keyAllowlistHosts), host, errNotBareHostname)
+// written with a trailing dot, which in allowlist_hosts disables URL signing.
+func validateHostPattern(key, host string) error {
+	_, err := netip.ParseAddr(host)
+	if err == nil || hostNamePattern.MatchString(host) {
+		return nil
 	}
 
-	if strings.Trim(host, ".") == "" {
+	return fmt.Errorf("%s: entry %q %w", settingName(key), host, errNotAHost)
-		return fmt.Errorf("%s: entry %q %w",
-			settingName(keyAllowlistHosts), host, errNoHostnameLabels)
-	}
-
-	return nil
 }
 
 // loadConfigFile loads configuration from the PIXA_CONFIG_PATH env var
@@ -778,19 +790,27 @@ func loadConfigFile(log *slog.Logger, appName string) (*smartconfig.Config, erro
 	for _, path := range configPaths {
 		cleanPath := filepath.Clean(path)
 
+		// Only a config file that does not exist is skipped, including
+		// one whose path runs through a file, such as under a HOME of
+		// /dev/null. One that cannot be read or does not parse is a
+		// fatal startup error.
 		_, statErr := os.Stat(cleanPath)
-		if statErr == nil {
+		if errors.Is(statErr, fs.ErrNotExist) || errors.Is(statErr, syscall.ENOTDIR) {
-			// A config file that exists but does not parse is a fatal
+			continue
-			// startup error, never something to skip over.
-			sc, err := smartconfig.NewFromConfigPath(path)
-			if err != nil {
-				return nil, fmt.Errorf("failed to parse config file %s: %w", path, err)
-			}
-
-			log.Info("loaded config file", "path", path)
-
-			return sc, nil
 		}
+
+		if statErr != nil {
+			return nil, fmt.Errorf("failed to read config file %s: %w", path, statErr)
+		}
+
+		sc, err := smartconfig.NewFromConfigPath(path)
+		if err != nil {
+			return nil, fmt.Errorf("failed to parse config file %s: %w", path, err)
+		}
+
+		log.Info("loaded config file", "path", path)
+
+		return sc, nil
 	}
 
 	return nil, nil //nolint:nilnil // nil config is valid (use defaults)
@@ -869,6 +889,19 @@ func (l *strictLoader) boolVal(key string, defaultVal bool) bool {
 	return val
 }
 
+func (l *strictLoader) hostListVal(key string) []string {
+	if l.err != nil {
+		return nil
+	}
+
+	val, err := parseHostList(l.sc, key)
+	if err != nil {
+		l.err = err
+	}
+
+	return val
+}
+
 // getString returns the string value for key, or defaultVal if the key
 // is omitted. A present value that is not a string, or is explicitly
 // null, is an error.
@@ -1166,7 +1199,7 @@ func parseCIDRList(sc *smartconfig.Config, key string) ([]netip.Prefix, error) {
 		return nil, errNullConfigValue(key)
 	}
 
-	entries, err := cidrListEntries(raw, key)
+	entries, err := listEntries(raw, key)
 	if err != nil {
 		return nil, err
 	}
@@ -1186,11 +1219,41 @@ func parseCIDRList(sc *smartconfig.Config, key string) ([]netip.Prefix, error) {
 	return prefixes, nil
 }
 
-// cidrListEntries extracts the raw entries of the named CIDR-list key as
+// parseHostList parses the value of the named config key into host patterns,
-// trimmed, non-empty strings, from either a YAML list of strings or a
+// or returns nil if the key is omitted. It accepts a YAML list of strings or a
-// comma-separated string; an empty string is an empty list, as for
+// comma-separated string. An explicitly null value, a wrong type, an empty
-// allowlist_hosts. Any other shape is a configuration error.
+// entry, a non-string entry, or an entry validateHostPattern rejects aborts
-func cidrListEntries(raw any, key string) ([]string, error) {
+// startup naming the key and the offending value.
+func parseHostList(sc *smartconfig.Config, key string) ([]string, error) {
+	raw, ok := lookupValue(sc, key)
+	if !ok {
+		return nil, nil
+	}
+
+	if raw == nil {
+		return nil, errNullConfigValue(key)
+	}
+
+	entries, err := listEntries(raw, key)
+	if err != nil {
+		return nil, err
+	}
+
+	for _, entry := range entries {
+		err := validateHostPattern(key, entry)
+		if err != nil {
+			return nil, err
+		}
+	}
+
+	return entries, nil
+}
+
+// listEntries extracts the raw entries of the named list key as trimmed,
+// non-empty strings, from either a YAML list of strings or a comma-separated
+// string; an empty string is an empty list, as for allowlist_hosts. Any other
+// shape is a configuration error.
+func listEntries(raw any, key string) ([]string, error) {
 	switch val := raw.(type) {
 	case []any:
 		entries := make([]string, 0, len(val))

internal/config/config_validation_internal_test.go

@@ -1,14 +1,18 @@
 package config
 
 import (
+	"database/sql"
 	"log/slog"
 	"os"
 	"path/filepath"
+	"slices"
 	"strings"
 	"testing"
 	"time"
 
 	"git.eeqj.de/sneak/smartconfig"
+
+	_ "modernc.org/sqlite" // SQLite driver registration
 )
 
 // validTestSigningKey is a 32-character signing key that satisfies the
@@ -94,12 +98,43 @@ func TestOmittedValuesUseDefaults(t *testing.T) {
 		t.Errorf("AllowlistHosts = %v, want empty", c.AllowlistHosts)
 	}
 
-	wantDBURL := "file:" + DefaultStateDir + "/state.sqlite3?_journal_mode=WAL"
+	wantDBURL := "file:" + DefaultStateDir +
+		"/state.sqlite3?_pragma=journal_mode(WAL)"
 	if c.DBURL != wantDBURL {
 		t.Errorf("DBURL = %q, want derived default %q", c.DBURL, wantDBURL)
 	}
 }
 
+// TestDefaultDBURLOpensTheDatabaseInWALMode opens the db_url derived from
+// state_dir with the SQLite driver pixad uses and checks that the database
+// is in WAL mode: the driver ignores any parameter it does not know.
+func TestDefaultDBURLOpensTheDatabaseInWALMode(t *testing.T) {
+	t.Parallel()
+
+	c, err := configFromYAML(t, signingKeyLine+"state_dir: "+t.TempDir()+"\n")
+	if err != nil {
+		t.Fatalf("config with only state_dir set should be valid, got: %v", err)
+	}
+
+	db, err := sql.Open("sqlite", c.DBURL)
+	if err != nil {
+		t.Fatalf("failed to open %q: %v", c.DBURL, err)
+	}
+
+	t.Cleanup(func() { _ = db.Close() })
+
+	var journalMode string
+
+	err = db.QueryRowContext(t.Context(), "PRAGMA journal_mode").Scan(&journalMode)
+	if err != nil {
+		t.Fatalf("failed to read the journal mode of %q: %v", c.DBURL, err)
+	}
+
+	if journalMode != "wal" {
+		t.Errorf("journal mode of %q = %q, want wal", c.DBURL, journalMode)
+	}
+}
+
 func TestExplicitValidValuesAreUsed(t *testing.T) {
 	t.Parallel()
 
@@ -182,6 +217,25 @@ func TestCommaSeparatedAllowlistStillSupported(t *testing.T) {
 	}
 }
 
+// TestAllowlistHostsAcceptsUnderscore checks that an upstream host name with
+// an underscore, which pixa can fetch from, is accepted as an entry.
+func TestAllowlistHostsAcceptsUnderscore(t *testing.T) {
+	t.Parallel()
+
+	c, err := configFromYAML(t, signingKeyLine+`allowlist_hosts:
+  - my_bucket.example.com
+  - .my_bucket.example.org
+`)
+	if err != nil {
+		t.Fatalf("host names with an underscore should load, got error: %v", err)
+	}
+
+	want := []string{"my_bucket.example.com", ".my_bucket.example.org"}
+	if !slices.Equal(c.AllowlistHosts, want) {
+		t.Errorf("AllowlistHosts = %v, want %v", c.AllowlistHosts, want)
+	}
+}
+
 // runAbortCases asserts that each case's config aborts startup with an
 // error message mentioning every expected substring.
 func runAbortCases(t *testing.T, cases []abortCase) {
@@ -284,6 +338,20 @@ func invalidHostAndCredentialCases() []abortCase {
 				keyAllowlistHosts, "example.com/images",
 			},
 		},
+		{
+			name: "allowlist host with wildcard",
+			yaml: signingKeyLine + "allowlist_hosts:\n  - \"*.example.com\"\n",
+			wantErrSubstrings: []string{
+				keyAllowlistHosts, "*.example.com",
+			},
+		},
+		{
+			name: "allowlist host with port",
+			yaml: signingKeyLine + "allowlist_hosts:\n  - example.com:8443\n",
+			wantErrSubstrings: []string{
+				keyAllowlistHosts, "example.com:8443",
+			},
+		},
 		{
 			name:              "allowlist host with whitespace",
 			yaml:              signingKeyLine + "allowlist_hosts:\n  - \"exa mple.com\"\n",
@@ -564,6 +632,116 @@ func TestMalformedConfigFileAbortsStartup(t *testing.T) {
 	t.Logf("got expected error: %v", err)
 }
 
+// TestConfigFileInDirectoryPixaMayNotEnterAbortsStartup checks that a
+// config file pixa cannot read because it may not enter its directory
+// aborts startup instead of being passed over.
+func TestConfigFileInDirectoryPixaMayNotEnterAbortsStartup(t *testing.T) {
+	if os.Geteuid() == 0 {
+		t.Skip("root may enter any directory")
+	}
+
+	home := t.TempDir()
+	configDir := filepath.Join(home, ".config", "pixa-test-nonexistent-app")
+	configPath := filepath.Join(configDir, "config.yml")
+
+	err := os.MkdirAll(configDir, 0o700)
+	if err != nil {
+		t.Fatalf("failed to create config directory: %v", err)
+	}
+
+	err = os.WriteFile(configPath, []byte(signingKeyLine), 0o600)
+	if err != nil {
+		t.Fatalf("failed to write config: %v", err)
+	}
+
+	err = os.Chmod(configDir, 0)
+	if err != nil {
+		t.Fatalf("failed to remove the config directory's permissions: %v", err)
+	}
+
+	// Give the directory back its permissions so t.TempDir can remove it.
+	t.Cleanup(func() {
+		//nolint:gosec // G302: a directory needs its execute bit to be removed
+		_ = os.Chmod(configDir, 0o700)
+	})
+
+	// The ~/.config candidate is the only one that exists: the appname
+	// rules out /etc, and the working directory is empty.
+	t.Setenv("PIXA_CONFIG_PATH", "")
+	t.Setenv("HOME", home)
+	t.Chdir(t.TempDir())
+
+	log := slog.New(slog.DiscardHandler)
+
+	sc, err := loadConfigFile(log, "pixa-test-nonexistent-app")
+	if err == nil {
+		t.Fatalf("config file pixa cannot read must abort startup, got config: %v",
+			sc)
+	}
+
+	t.Logf("got expected error: %v", err)
+
+	if !strings.Contains(err.Error(), configPath) {
+		t.Errorf("error %q does not name the config file %s", err.Error(), configPath)
+	}
+}
+
+// TestConfigFileLinkingToItselfAbortsStartup checks that a config file
+// pixa cannot read for a reason other than not existing aborts startup,
+// as root too: a symbolic link to itself fails with "too many levels of
+// symbolic links".
+func TestConfigFileLinkingToItselfAbortsStartup(t *testing.T) {
+	workDir := t.TempDir()
+
+	err := os.Symlink("config.yml", filepath.Join(workDir, "config.yml"))
+	if err != nil {
+		t.Fatalf("failed to create symbolic link: %v", err)
+	}
+
+	// Only the working directory's config.yml is there: the appname rules
+	// out /etc, and HOME is empty.
+	t.Setenv("PIXA_CONFIG_PATH", "")
+	t.Setenv("HOME", t.TempDir())
+	t.Chdir(workDir)
+
+	log := slog.New(slog.DiscardHandler)
+
+	sc, err := loadConfigFile(log, "pixa-test-nonexistent-app")
+	if err == nil {
+		t.Fatalf("config file pixa cannot read must abort startup, got config: %v",
+			sc)
+	}
+
+	t.Logf("got expected error: %v", err)
+
+	if !strings.Contains(err.Error(), "config.yml") {
+		t.Errorf("error %q does not name the config file config.yml", err.Error())
+	}
+}
+
+// TestConfigPathThroughFileIsPassedOver checks that a config file path
+// that runs through a file, such as one under a HOME of /dev/null, is
+// passed over like one that does not exist, since no file can be there.
+func TestConfigPathThroughFileIsPassedOver(t *testing.T) {
+	// No config file is there: the appname rules out /etc, HOME is
+	// /dev/null, and the working directory is empty.
+	t.Setenv("PIXA_CONFIG_PATH", "")
+	t.Setenv("HOME", os.DevNull)
+	t.Chdir(t.TempDir())
+
+	log := slog.New(slog.DiscardHandler)
+
+	sc, err := loadConfigFile(log, "pixa-test-nonexistent-app")
+	if err != nil {
+		t.Fatalf("a config path through a file must be passed over, got error: %v",
+			err)
+	}
+
+	if sc != nil {
+		t.Errorf("expected no config file, got config: %v", sc)
+	}
+}
+
 func TestEnsureStateDirCreatesDirectory(t *testing.T) {
 	t.Parallel()
 

internal/config/env_internal_test.go

@@ -65,6 +65,7 @@ func TestEnvironmentSetsEveryKey(t *testing.T) {
 	t.Setenv("PIXA_METRICS_PASSWORD", "metricspass")
 	t.Setenv("PIXA_SIGNING_KEY", validTestSigningKey)
 	t.Setenv("PIXA_ALLOWLIST_HOSTS", "s3.sneak.cloud,.example.com")
+	t.Setenv("PIXA_REFERER_BLOCKLIST", "hotlinker.example,.leech.example")
 	t.Setenv("PIXA_ALLOW_HTTP", "true")
 	t.Setenv("PIXA_UPSTREAM_CONNECTIONS_PER_HOST", "5")
 	t.Setenv("PIXA_UPSTREAM_CONNECTIONS", "10")
@@ -93,12 +94,13 @@ func TestEnvironmentSetsEveryKey(t *testing.T) {
 		MetricsPassword:            "metricspass",
 		SigningKey:                 validTestSigningKey,
 		AllowlistHosts:             []string{testHostS3, ".example.com"},
+		RefererBlocklist:           []string{"hotlinker.example", ".leech.example"},
 		AllowHTTP:                  true,
 		UpstreamConnectionsPerHost: 5,
 		UpstreamConnections:        10,
 		MaxConcurrentProcessing:    3,
 		CacheMaxBytes:              1024,
-		cacheMaxBytesExplicit:      true,
+		CacheMaxBytesExplicit:      true,
 		BlockedNetworks:            []netip.Prefix{netip.MustParsePrefix("203.0.113.0/24")},
 		TrustedProxies:             []netip.Prefix{netip.MustParsePrefix("192.0.2.0/24")},
 		AccessControlAllowOrigin:   "https://app.example.com",

internal/config/referer_blocklist_internal_test.go

@@ -0,0 +1,166 @@
+package config
+
+import (
+	"slices"
+	"testing"
+)
+
+// TestRefererBlocklistParsed loads a referer_blocklist with a host and a
+// pattern starting with "." and checks both are kept in order.
+func TestRefererBlocklistParsed(t *testing.T) {
+	t.Parallel()
+
+	c, err := configFromYAML(t, signingKeyLine+`referer_blocklist:
+  - leech.example
+  - .hotlinker.example
+`)
+	if err != nil {
+		t.Fatalf("valid referer_blocklist should load, got error: %v", err)
+	}
+
+	want := []string{"leech.example", ".hotlinker.example"}
+	if !slices.Equal(c.RefererBlocklist, want) {
+		t.Errorf("RefererBlocklist = %v, want %v", c.RefererBlocklist, want)
+	}
+}
+
+// TestRefererBlocklistAcceptsIPAddresses checks that IPv4 and IPv6 addresses,
+// the IPv6 one written without brackets, are accepted as entries.
+func TestRefererBlocklistAcceptsIPAddresses(t *testing.T) {
+	t.Parallel()
+
+	c, err := configFromYAML(t, signingKeyLine+`referer_blocklist:
+  - 192.0.2.7
+  - "2001:db8::7"
+`)
+	if err != nil {
+		t.Fatalf("IP address entries should load, got error: %v", err)
+	}
+
+	want := []string{"192.0.2.7", "2001:db8::7"}
+	if !slices.Equal(c.RefererBlocklist, want) {
+		t.Errorf("RefererBlocklist = %v, want %v", c.RefererBlocklist, want)
+	}
+}
+
+// TestRefererBlocklistAcceptsUnderscore checks that a host name with an
+// underscore, which a page can be served from, is accepted as an entry.
+func TestRefererBlocklistAcceptsUnderscore(t *testing.T) {
+	t.Parallel()
+
+	c, err := configFromYAML(t, signingKeyLine+`referer_blocklist:
+  - my_site.leech.example
+  - .my_site.hotlinker.example
+`)
+	if err != nil {
+		t.Fatalf("host names with an underscore should load, got error: %v", err)
+	}
+
+	want := []string{"my_site.leech.example", ".my_site.hotlinker.example"}
+	if !slices.Equal(c.RefererBlocklist, want) {
+		t.Errorf("RefererBlocklist = %v, want %v", c.RefererBlocklist, want)
+	}
+}
+
+// TestRefererBlocklistOmittedIsEmpty checks that an omitted key blocks no
+// referer.
+func TestRefererBlocklistOmittedIsEmpty(t *testing.T) {
+	t.Parallel()
+
+	c, err := configFromYAML(t, signingKeyLine)
+	if err != nil {
+		t.Fatalf("minimal config should be valid, got error: %v", err)
+	}
+
+	if len(c.RefererBlocklist) != 0 {
+		t.Errorf("RefererBlocklist = %v, want empty", c.RefererBlocklist)
+	}
+}
+
+// TestRefererBlocklistInvalidAbortsStartup checks that an entry that is not a
+// host, or a value that is not a list of them, aborts startup with an error
+// naming the key and the entry.
+func TestRefererBlocklistInvalidAbortsStartup(t *testing.T) {
+	t.Parallel()
+
+	runAbortCases(t, []abortCase{
+		{
+			name: "entry with a scheme",
+			yaml: signingKeyLine + "referer_blocklist:\n  - https://leech.example\n",
+			wantErrSubstrings: []string{
+				keyRefererBlocklist, "https://leech.example",
+			},
+		},
+		{
+			name: "entry with a path",
+			yaml: signingKeyLine + "referer_blocklist:\n  - leech.example/page\n",
+			wantErrSubstrings: []string{
+				keyRefererBlocklist, "leech.example/page",
+			},
+		},
+		{
+			name: "wildcard entry",
+			yaml: signingKeyLine + "referer_blocklist:\n  - \"*.leech.example\"\n",
+			wantErrSubstrings: []string{
+				keyRefererBlocklist, "*.leech.example",
+			},
+		},
+		{
+			name: "entry with a port",
+			yaml: signingKeyLine + "referer_blocklist:\n  - leech.example:8080\n",
+			wantErrSubstrings: []string{
+				keyRefererBlocklist, "leech.example:8080",
+			},
+		},
+		{
+			name: "two leading dots",
+			yaml: signingKeyLine + "referer_blocklist:\n  - ..leech.example\n",
+			wantErrSubstrings: []string{
+				keyRefererBlocklist, "..leech.example",
+			},
+		},
+		{
+			name:              "dot only",
+			yaml:              signingKeyLine + "referer_blocklist:\n  - \".\"\n",
+			wantErrSubstrings: []string{keyRefererBlocklist, `"."`},
+		},
+		{
+			name:              "empty entry",
+			yaml:              signingKeyLine + "referer_blocklist:\n  - \"\"\n",
+			wantErrSubstrings: []string{keyRefererBlocklist},
+		},
+		{
+			name:              "entry not a string",
+			yaml:              signingKeyLine + "referer_blocklist:\n  - 42\n",
+			wantErrSubstrings: []string{keyRefererBlocklist, "42"},
+		},
+		{
+			name:              "null value",
+			yaml:              signingKeyLine + "referer_blocklist:\n",
+			wantErrSubstrings: []string{keyRefererBlocklist, nullValueText},
+		},
+	})
+}
+
+// TestRefererBlocklistFromEnvironment checks that PIXA_REFERER_BLOCKLIST
+// takes comma-separated entries, and that an entry in it that is not a host
+// aborts startup naming the variable and the entry.
+func TestRefererBlocklistFromEnvironment(t *testing.T) {
+	t.Setenv("PIXA_SIGNING_KEY", validTestSigningKey)
+	t.Setenv("PIXA_REFERER_BLOCKLIST", " leech.example , .hotlinker.example ")
+
+	c, err := newFromSmartConfig(nil)
+	if err != nil {
+		t.Fatalf("valid PIXA_REFERER_BLOCKLIST should load, got error: %v", err)
+	}
+
+	want := []string{"leech.example", ".hotlinker.example"}
+	if !slices.Equal(c.RefererBlocklist, want) {
+		t.Errorf("RefererBlocklist = %v, want %v", c.RefererBlocklist, want)
+	}
+
+	t.Setenv("PIXA_REFERER_BLOCKLIST", "leech.example,https://hotlinker.example")
+
+	_, err = newFromSmartConfig(nil)
+	wantStartupError(t, err, "PIXA_REFERER_BLOCKLIST", "https://hotlinker.example")
+}

internal/database/concurrent_writes_internal_test.go

@@ -0,0 +1,153 @@
+package database
+
+import (
+	"context"
+	"database/sql"
+	"fmt"
+	"log/slog"
+	"path/filepath"
+	"sync"
+	"testing"
+
+	"sneak.berlin/go/pixa/internal/config"
+)
+
+// TestConcurrentWritesAllSucceed opens a database the way pixad does and
+// writes to it from several goroutines at once, so the writes run on
+// separate connections, as one request's writes and the background eviction
+// pass do. Every write must succeed, none failing with "database is locked",
+// whether or not db_url already has parameters, and the parameters it has
+// must still apply.
+func TestConcurrentWritesAllSucceed(t *testing.T) {
+	t.Parallel()
+
+	tests := []struct {
+		name            string
+		query           string
+		wantJournalMode string
+	}{
+		{
+			name:            "db_url without parameters",
+			query:           "",
+			wantJournalMode: "delete",
+		},
+		{
+			name:            "db_url with the WAL parameter",
+			query:           "?_pragma=journal_mode(WAL)",
+			wantJournalMode: "wal",
+		},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			t.Parallel()
+
+			dbURL := "file:" + filepath.Join(t.TempDir(), "state.sqlite3") + tt.query
+
+			d := &Database{
+				log:    slog.New(slog.DiscardHandler),
+				config: &config.Config{DBURL: dbURL},
+			}
+
+			err := d.connect(t.Context())
+			if err != nil {
+				t.Fatalf("failed to connect to %q: %v", dbURL, err)
+			}
+
+			t.Cleanup(func() { _ = d.db.Close() })
+
+			writeConcurrently(t, d.db)
+
+			var journalMode string
+
+			err = d.db.QueryRowContext(t.Context(), "PRAGMA journal_mode").
+				Scan(&journalMode)
+			if err != nil {
+				t.Fatalf("failed to read the journal mode: %v", err)
+			}
+
+			if journalMode != tt.wantJournalMode {
+				t.Errorf("journal mode = %q, want %q", journalMode, tt.wantJournalMode)
+			}
+		})
+	}
+}
+
+// writeConcurrently runs writeLikeOneRequest from several goroutines at once
+// and checks that every write was made.
+func writeConcurrently(t *testing.T, db *sql.DB) {
+	t.Helper()
+
+	const (
+		writers       = 4
+		requestsEach  = 20
+		totalRequests = writers * requestsEach
+	)
+
+	ctx := t.Context()
+
+	var wg sync.WaitGroup
+
+	for writer := range writers {
+		wg.Go(func() {
+			for request := range requestsEach {
+				key := fmt.Sprintf("%d-%d", writer, request)
+
+				err := writeLikeOneRequest(ctx, db, key)
+				if err != nil {
+					t.Errorf("writer %d: %v", writer, err)
+
+					return
+				}
+			}
+		})
+	}
+
+	wg.Wait()
+
+	var hits, sources int
+
+	err := db.QueryRowContext(ctx, `
+		SELECT hit_count, (SELECT COUNT(*) FROM source_content)
+		FROM cache_stats WHERE id = 1
+	`).Scan(&hits, &sources)
+	if err != nil {
+		t.Fatalf("failed to count the writes: %v", err)
+	}
+
+	if hits != totalRequests || sources != totalRequests {
+		t.Errorf("hit_count = %d and %d source_content rows, want %d of each",
+			hits, sources, totalRequests)
+	}
+}
+
+// writeLikeOneRequest makes the writes one request and the eviction pass
+// make: it counts a cache hit, stores a source, records a transformed image
+// and deletes that record again.
+func writeLikeOneRequest(ctx context.Context, db *sql.DB, key string) error {
+	_, err := db.ExecContext(ctx,
+		`UPDATE cache_stats SET hit_count = hit_count + 1 WHERE id = 1`)
+	if err != nil {
+		return fmt.Errorf("counting a cache hit: %w", err)
+	}
+
+	_, err = db.ExecContext(ctx, `INSERT INTO source_content
+		(content_hash, content_type, size_bytes) VALUES (?, 'image/png', 1)`, key)
+	if err != nil {
+		return fmt.Errorf("storing a source: %w", err)
+	}
+
+	_, err = db.ExecContext(ctx, `INSERT INTO variant_content
+		(cache_key, size_bytes, content_type) VALUES (?, 1, 'image/png')`, key)
+	if err != nil {
+		return fmt.Errorf("recording a transformed image: %w", err)
+	}
+
+	_, err = db.ExecContext(ctx,
+		`DELETE FROM variant_content WHERE cache_key = ?`, key)
+	if err != nil {
+		return fmt.Errorf("evicting a transformed image: %w", err)
+	}
+
+	return nil
+}

internal/database/database.go

@@ -243,7 +243,17 @@ func (s *Database) DB() *sql.DB {
 }
 
 func (s *Database) connect(ctx context.Context) error {
-	dbURL := s.config.DBURL
+	// Requests and the eviction pass write on separate connections. With
+	// a busy timeout, a write that finds another one in progress waits up
+	// to five seconds for it instead of failing at once with "database is
+	// locked". The driver runs each _pragma parameter on every connection
+	// it opens.
+	separator := "?"
+	if strings.Contains(s.config.DBURL, "?") {
+		separator = "&"
+	}
+
+	dbURL := s.config.DBURL + separator + "_pragma=busy_timeout(5000)"
 
 	s.log.Info("connecting to database", "url", dbURL)
 

internal/db/migrations/001_schema.sql

@@ -2,7 +2,7 @@
 -- Creates all tables for the pixa caching image proxy
 
 -- Source content blobs
--- Files stored at: cache/src-content/<ab>/<cd>/<sha256>
+-- Files stored at: cache/sources/<ab>/<cd>/<sha256>
 -- last_accessed_at is NULL until the first LRU touch; eviction falls
 -- back to fetched_at for rows that have never been touched.
 CREATE TABLE IF NOT EXISTS source_content (
@@ -16,7 +16,7 @@ CREATE INDEX IF NOT EXISTS idx_source_content_last_accessed
     ON source_content(last_accessed_at);
 
 -- Source URL metadata - maps URLs to content hashes
--- JSON stored at: cache/src-metadata/<hostname>/<path_hash>.json
+-- JSON stored at: cache/metadata/<hostname>/<path_hash>.json
 CREATE TABLE IF NOT EXISTS source_metadata (
     id INTEGER PRIMARY KEY AUTOINCREMENT,
     source_host TEXT NOT NULL,
@@ -56,7 +56,8 @@ CREATE INDEX IF NOT EXISTS idx_variant_content_last_accessed
     ON variant_content(last_accessed_at);
 
 -- Output/transformed content blobs
--- Files stored at: cache/dst-content/<ab>/<cd>/<sha256>
+-- Not written: transformed images are stored in cache/variants and
+-- tracked in variant_content above.
 CREATE TABLE IF NOT EXISTS output_content (
     content_hash TEXT PRIMARY KEY,
     content_type TEXT NOT NULL,

internal/handlers/auth_login_log_internal_test.go

@@ -0,0 +1,59 @@
+package handlers
+
+import (
+	"bytes"
+	"log/slog"
+	"net/http"
+	"net/http/httptest"
+	"net/url"
+	"strings"
+	"testing"
+
+	"sneak.berlin/go/pixa/internal/config"
+	"sneak.berlin/go/pixa/internal/session"
+)
+
+// TestLoginLogLeavesOutSubmittedKey verifies that the log lines for a
+// failed and for a successful login do not contain the submitted key.
+func TestLoginLogLeavesOutSubmittedKey(t *testing.T) {
+	t.Parallel()
+
+	const wrongKey = "wrong-signing-key-fedcba9876543210"
+
+	var buf bytes.Buffer
+
+	sessMgr, err := session.NewManager(testSigningKey)
+	if err != nil {
+		t.Fatalf("session.NewManager() error = %v", err)
+	}
+
+	h := &Handlers{
+		log:     slog.New(slog.NewJSONHandler(&buf, nil)),
+		config:  &config.Config{SigningKey: testSigningKey},
+		sessMgr: sessMgr,
+	}
+
+	submittedKeys := []string{wrongKey, testSigningKey}
+
+	for _, key := range submittedKeys {
+		form := url.Values{loginKeyField: {key}}
+		req := httptest.NewRequestWithContext(
+			t.Context(), http.MethodPost, "/",
+			strings.NewReader(form.Encode()))
+		req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
+
+		h.handleLoginPost(httptest.NewRecorder(), req)
+	}
+
+	for _, msg := range []string{"failed login attempt", "successful login"} {
+		if !strings.Contains(buf.String(), msg) {
+			t.Fatalf("log missing %q; got %q", msg, buf.String())
+		}
+	}
+
+	for _, key := range submittedKeys {
+		if strings.Contains(buf.String(), key) {
+			t.Errorf("log contains submitted key %q; got %q", key, buf.String())
+		}
+	}
+}

internal/handlers/auth_session_internal_test.go

@@ -0,0 +1,243 @@
+package handlers
+
+import (
+	"log/slog"
+	"net/http"
+	"net/http/httptest"
+	"net/url"
+	"regexp"
+	"strings"
+	"testing"
+
+	"sneak.berlin/go/pixa/internal/imgcache"
+	"sneak.berlin/go/pixa/internal/session"
+)
+
+// formatField is the generator form's format field name.
+const formatField = "format"
+
+// Markers telling the login page from the generator page.
+const (
+	loginForm     = `action="/"`
+	loginKeyInput = `name="key"`
+	generatorForm = `action="/generate"`
+)
+
+// generatedURLPattern extracts the path of the URL the generator page shows.
+// The test router runs with debug on, so the URL starts with http, and its
+// host is httptest's default request host.
+var generatedURLPattern = regexp.MustCompile(
+	`value="http://example\.com(/v1/e/[^"]+)"`)
+
+// findSessionCookie returns the session cookie rec sets, or nil if it sets
+// none.
+func findSessionCookie(rec *httptest.ResponseRecorder) *http.Cookie {
+	for _, c := range rec.Result().Cookies() {
+		if c.Name == session.CookieName {
+			return c
+		}
+	}
+
+	return nil
+}
+
+// TestHandleRoot_NoSession_ShowsLoginForm verifies that GET / without a
+// login session shows the login form.
+func TestHandleRoot_NoSession_ShowsLoginForm(t *testing.T) {
+	t.Parallel()
+
+	_, srv := newCSRFTestRouter(t)
+
+	rec := httptest.NewRecorder()
+	srv.ServeHTTP(rec, httptest.NewRequestWithContext(
+		t.Context(), http.MethodGet, "/", nil))
+
+	if rec.Code != http.StatusOK {
+		t.Fatalf("status = %d, want %d", rec.Code, http.StatusOK)
+	}
+
+	body := rec.Body.String()
+	if !strings.Contains(body, loginForm) || !strings.Contains(body, loginKeyInput) {
+		t.Errorf("page is not the login form: %s", body)
+	}
+}
+
+// TestLoginPost_WrongKey_ShowsErrorWithoutSession verifies that a wrong key
+// shows the login form again with an error, and sets no session cookie.
+func TestLoginPost_WrongKey_ShowsErrorWithoutSession(t *testing.T) {
+	t.Parallel()
+
+	_, srv := newCSRFTestRouter(t)
+
+	cookies, token := csrfCredentials(t, srv, nil)
+
+	rec := postForm(srv, "/", cookies, url.Values{
+		loginKeyField:  {"wrong-signing-key-fedcba9876543210"},
+		csrfTokenField: {token},
+	})
+
+	if rec.Code != http.StatusOK {
+		t.Fatalf("status = %d, want %d", rec.Code, http.StatusOK)
+	}
+
+	body := rec.Body.String()
+	if !strings.Contains(body, loginForm) || !strings.Contains(body, loginKeyInput) {
+		t.Errorf("page is not the login form: %s", body)
+	}
+
+	if !strings.Contains(body, "Invalid signing key") {
+		t.Error("login form does not show the error")
+	}
+
+	if c := findSessionCookie(rec); c != nil {
+		t.Errorf("wrong key set a session cookie: %s", c)
+	}
+}
+
+// TestLoginPost_RightKey_SetsSessionCookie verifies that the right key answers
+// 303 to / with a session cookie marked Secure, HttpOnly and SameSite=Strict,
+// and that GET / with that cookie shows the generator page.
+func TestLoginPost_RightKey_SetsSessionCookie(t *testing.T) {
+	t.Parallel()
+
+	_, srv := newCSRFTestRouter(t)
+
+	cookies, token := csrfCredentials(t, srv, nil)
+
+	rec := postForm(srv, "/", cookies, url.Values{
+		loginKeyField:  {testSigningKey},
+		csrfTokenField: {token},
+	})
+
+	if rec.Code != http.StatusSeeOther || rec.Header().Get("Location") != "/" {
+		t.Fatalf("status = %d, Location = %q, want %d to /",
+			rec.Code, rec.Header().Get("Location"), http.StatusSeeOther)
+	}
+
+	sessionCookie := findSessionCookie(rec)
+	if sessionCookie == nil {
+		t.Fatal("right key set no session cookie")
+	}
+
+	t.Logf("Set-Cookie: %s", sessionCookie)
+
+	if !sessionCookie.Secure {
+		t.Error("session cookie is not Secure")
+	}
+
+	if !sessionCookie.HttpOnly {
+		t.Error("session cookie is not HttpOnly")
+	}
+
+	if sessionCookie.SameSite != http.SameSiteStrictMode {
+		t.Errorf("session cookie SameSite = %v, want Strict", sessionCookie.SameSite)
+	}
+
+	req := httptest.NewRequestWithContext(t.Context(), http.MethodGet, "/", nil)
+	req.AddCookie(sessionCookie)
+
+	rec = httptest.NewRecorder()
+	srv.ServeHTTP(rec, req)
+
+	if rec.Code != http.StatusOK ||
+		!strings.Contains(rec.Body.String(), generatorForm) {
+		t.Errorf("GET / with the session cookie: status = %d, "+
+			"want %d and the generator page", rec.Code, http.StatusOK)
+	}
+}
+
+// TestHandleLogout_ClearsSessionCookie verifies that GET /logout answers 303
+// to / and replaces the session cookie with an empty one sent with
+// Max-Age=0, which makes the browser delete it.
+func TestHandleLogout_ClearsSessionCookie(t *testing.T) {
+	t.Parallel()
+
+	h, _ := newCSRFTestRouter(t)
+
+	req := httptest.NewRequestWithContext(
+		t.Context(), http.MethodGet, "/logout", nil)
+	req.AddCookie(newSessionCookie(t, h))
+
+	rec := httptest.NewRecorder()
+	h.HandleLogout().ServeHTTP(rec, req)
+
+	if rec.Code != http.StatusSeeOther || rec.Header().Get("Location") != "/" {
+		t.Fatalf("status = %d, Location = %q, want %d to /",
+			rec.Code, rec.Header().Get("Location"), http.StatusSeeOther)
+	}
+
+	t.Logf("Set-Cookie: %s", rec.Header().Get("Set-Cookie"))
+
+	sessionCookie := findSessionCookie(rec)
+	if sessionCookie == nil {
+		t.Fatal("logout did not set the session cookie")
+	}
+
+	if sessionCookie.Value != "" {
+		t.Errorf("session cookie value = %q, want empty", sessionCookie.Value)
+	}
+
+	// net/http reads a Max-Age=0 attribute back as MaxAge -1.
+	if sessionCookie.MaxAge != -1 {
+		t.Errorf("session cookie MaxAge = %d, want -1 (Max-Age=0)",
+			sessionCookie.MaxAge)
+	}
+}
+
+// TestGeneratePost_NoSession_RedirectsToLogin verifies that POST /generate
+// with a valid CSRF token but no login session answers 303 to / and makes no
+// URL.
+func TestGeneratePost_NoSession_RedirectsToLogin(t *testing.T) {
+	t.Parallel()
+
+	_, srv := newCSRFTestRouter(t)
+
+	cookies, token := csrfCredentials(t, srv, nil)
+
+	rec := postForm(srv, "/generate", cookies, url.Values{
+		sourceURLField: {testSourceURL},
+		csrfTokenField: {token},
+	})
+
+	if rec.Code != http.StatusSeeOther || rec.Header().Get("Location") != "/" {
+		t.Fatalf("status = %d, Location = %q, want %d to /",
+			rec.Code, rec.Header().Get("Location"), http.StatusSeeOther)
+	}
+
+	if strings.Contains(rec.Body.String(), "/v1/e/") {
+		t.Error("a URL was made without a login session")
+	}
+}
+
+// TestGeneratePost_URLServesImage verifies that the URL the generator page
+// makes is served by /v1/e/. The image route runs on handlers of its own,
+// made with the same signing key.
+func TestGeneratePost_URLServesImage(t *testing.T) {
+	t.Parallel()
+
+	_, imageSrv := newSignedHostServer(t, slog.New(slog.DiscardHandler))
+
+	rec := generatePost(t, url.Values{
+		sourceURLField: {"https://" + signedHost + photoPath},
+		widthField:     {"50"},
+		heightField:    {"50"},
+		formatField:    {string(imgcache.FormatJPEG)},
+	})
+
+	if rec.Code != http.StatusOK {
+		t.Fatalf("POST /generate status = %d, want %d", rec.Code, http.StatusOK)
+	}
+
+	match := generatedURLPattern.FindStringSubmatch(rec.Body.String())
+	if match == nil {
+		t.Fatalf("generator page shows no URL: %s", rec.Body.String())
+	}
+
+	t.Logf("generated URL path: %s", match[1])
+
+	imageRec := httptest.NewRecorder()
+	imageSrv.ServeHTTP(imageRec, httptest.NewRequestWithContext(
+		t.Context(), http.MethodGet, match[1], nil))
+
+	requireServedPhoto(t, imageRec)
+}

internal/handlers/cache_max_bytes_internal_test.go

@@ -0,0 +1,147 @@
+package handlers
+
+import (
+	"os"
+	"path/filepath"
+	"testing"
+
+	"go.uber.org/fx/fxtest"
+
+	"sneak.berlin/go/pixa/internal/config"
+	"sneak.berlin/go/pixa/internal/database"
+	"sneak.berlin/go/pixa/internal/globals"
+	"sneak.berlin/go/pixa/internal/logger"
+)
+
+// TestNewCacheConfigFromCacheMaxBytes checks the cache configuration
+// built from cache_max_bytes: omitted, the cache works out the default
+// limit; 0 turns the disk cache off; a positive value is the limit,
+// unchanged.
+func TestNewCacheConfigFromCacheMaxBytes(t *testing.T) {
+	t.Parallel()
+
+	const oneGiB = 1 << 30
+
+	cases := []struct {
+		name                   string
+		cacheMaxBytes          int64
+		cacheMaxBytesExplicit  bool
+		wantMaxBytes           int64
+		wantUseDefaultMaxBytes bool
+		wantDisableDiskCache   bool
+	}{
+		{
+			name:                   "cache_max_bytes omitted",
+			cacheMaxBytes:          0,
+			cacheMaxBytesExplicit:  false,
+			wantMaxBytes:           0,
+			wantUseDefaultMaxBytes: true,
+			wantDisableDiskCache:   false,
+		},
+		{
+			name:                   "cache_max_bytes: 0",
+			cacheMaxBytes:          0,
+			cacheMaxBytesExplicit:  true,
+			wantMaxBytes:           0,
+			wantUseDefaultMaxBytes: false,
+			wantDisableDiskCache:   true,
+		},
+		{
+			name:                   "cache_max_bytes: 1 GiB",
+			cacheMaxBytes:          oneGiB,
+			cacheMaxBytesExplicit:  true,
+			wantMaxBytes:           oneGiB,
+			wantUseDefaultMaxBytes: false,
+			wantDisableDiskCache:   false,
+		},
+	}
+
+	for _, tc := range cases {
+		t.Run(tc.name, func(t *testing.T) {
+			t.Parallel()
+
+			cfg := &config.Config{
+				CacheMaxBytes:         tc.cacheMaxBytes,
+				CacheMaxBytesExplicit: tc.cacheMaxBytesExplicit,
+			}
+
+			got := newCacheConfig(cfg, nil)
+			t.Logf("MaxBytes = %d, UseDefaultMaxBytes = %v, DisableDiskCache = %v",
+				got.MaxBytes, got.UseDefaultMaxBytes, got.DisableDiskCache)
+
+			if got.MaxBytes != tc.wantMaxBytes {
+				t.Errorf("MaxBytes = %d, want %d", got.MaxBytes, tc.wantMaxBytes)
+			}
+
+			if got.UseDefaultMaxBytes != tc.wantUseDefaultMaxBytes {
+				t.Errorf("UseDefaultMaxBytes = %v, want %v",
+					got.UseDefaultMaxBytes, tc.wantUseDefaultMaxBytes)
+			}
+
+			if got.DisableDiskCache != tc.wantDisableDiskCache {
+				t.Errorf("DisableDiskCache = %v, want %v",
+					got.DisableDiskCache, tc.wantDisableDiskCache)
+			}
+		})
+	}
+}
+
+// TestDiskCacheOffOnlyForExplicitZeroCacheMaxBytes starts the handlers
+// once with cache_max_bytes omitted and once with cache_max_bytes: 0,
+// and checks by whether the cache directories were created that the
+// disk cache is on in the first case and off in the second.
+func TestDiskCacheOffOnlyForExplicitZeroCacheMaxBytes(t *testing.T) {
+	t.Parallel()
+
+	cases := []struct {
+		name                  string
+		cacheMaxBytesExplicit bool
+		wantDiskCache         bool
+	}{
+		{name: "cache_max_bytes omitted", cacheMaxBytesExplicit: false, wantDiskCache: true},
+		{name: "cache_max_bytes: 0", cacheMaxBytesExplicit: true, wantDiskCache: false},
+	}
+
+	for _, tc := range cases {
+		t.Run(tc.name, func(t *testing.T) {
+			t.Parallel()
+
+			stateDir := t.TempDir()
+			cfg := &config.Config{
+				SigningKey:            testSigningKey,
+				StateDir:              stateDir,
+				DBURL:                 "file:" + filepath.Join(stateDir, "state.sqlite3"),
+				CacheMaxBytes:         0,
+				CacheMaxBytesExplicit: tc.cacheMaxBytesExplicit,
+			}
+
+			lc := fxtest.NewLifecycle(t)
+
+			log, err := logger.New(lc, logger.Params{Globals: &globals.Globals{}})
+			if err != nil {
+				t.Fatalf("logger.New() error = %v", err)
+			}
+
+			db, err := database.New(lc, database.Params{Logger: log, Config: cfg})
+			if err != nil {
+				t.Fatalf("database.New() error = %v", err)
+			}
+
+			_, err = New(lc, Params{Logger: log, Database: db, Config: cfg})
+			if err != nil {
+				t.Fatalf("New() error = %v", err)
+			}
+
+			lc.RequireStart()
+			t.Cleanup(lc.RequireStop)
+
+			_, err = os.Stat(filepath.Join(stateDir, "cache", "variants"))
+			gotDiskCache := err == nil
+
+			if gotDiskCache != tc.wantDiskCache {
+				t.Errorf("cache directories created = %v, want %v",
+					gotDiskCache, tc.wantDiskCache)
+			}
+		})
+	}
+}

internal/handlers/handlers.go

@@ -9,6 +9,7 @@ import (
 	"time"
 
 	"go.uber.org/fx"
+	"sneak.berlin/go/pixa/internal/allowlist"
 	"sneak.berlin/go/pixa/internal/config"
 	"sneak.berlin/go/pixa/internal/database"
 	"sneak.berlin/go/pixa/internal/encurl"
@@ -40,6 +41,10 @@ type Handlers struct {
 	sessMgr     *session.Manager
 	encGen      *encurl.Generator
 	csrfProtect func(http.Handler) http.Handler
+
+	// refererBlocklist matches the hosts of referer_blocklist; its IsAllowed
+	// reports whether a URL's host is on that list.
+	refererBlocklist *allowlist.HostAllowList
 }
 
 // New creates a new Handlers instance.
@@ -50,49 +55,56 @@ func New(lc fx.Lifecycle, params Params) (*Handlers, error) {
 	}
 
 	s := &Handlers{
-		log:         params.Logger.Get(),
+		log:              params.Logger.Get(),
-		hc:          params.Healthcheck,
+		hc:               params.Healthcheck,
-		db:          params.Database,
+		db:               params.Database,
-		config:      params.Config,
+		config:           params.Config,
-		csrfProtect: csrfProtect,
+		csrfProtect:      csrfProtect,
+		refererBlocklist: allowlist.New(params.Config.RefererBlocklist),
 	}
 
 	lc.Append(fx.Hook{
-		// The eviction goroutine must outlive OnStart, so it cannot
+		//nolint:contextcheck // the eviction loop outlives OnStart; OnStop cancels it
-		// inherit this hook's context. It makes its own instead, which
-		// leaves it uncancellable: an in-flight pass runs to completion
-		// during OnStop regardless of the shutdown deadline. Making the
-		// loop cancellable changes shutdown semantics and is tracked
-		// separately in issue #102, rather than being folded into the
-		// lint-conformance change that surfaced it.
-		//nolint:contextcheck // see issue #102
 		OnStart: func(_ context.Context) error {
 			return s.initImageService()
 		},
-		OnStop: func(_ context.Context) error {
+		OnStop: func(ctx context.Context) error {
-			if s.imgCache != nil {
+			if s.imgCache == nil {
-				s.imgCache.StopEviction()
+				return nil
 			}
 
-			return nil
+			return s.imgCache.StopEviction(ctx)
 		},
 	})
 
 	return s, nil
 }
 
+// WaitForProcessing waits until no image is being processed, or until ctx
+// ends, and returns how many images were still being processed then.
+func (s *Handlers) WaitForProcessing(ctx context.Context) int {
+	return s.imgSvc.WaitForProcessing(ctx)
+}
+
+// newCacheConfig builds the image cache's configuration from cfg.
+// cache_max_bytes: 0 disables the disk cache entirely; any other value
+// is the eviction limit in bytes; when it is omitted, the cache works
+// out the default limit itself.
+func newCacheConfig(cfg *config.Config, log *slog.Logger) imgcache.CacheConfig {
+	return imgcache.CacheConfig{
+		StateDir:           cfg.StateDir,
+		CacheTTL:           imgcache.DefaultCacheTTL,
+		NegativeTTL:        imgcache.DefaultNegativeTTL,
+		MaxBytes:           cfg.CacheMaxBytes,
+		UseDefaultMaxBytes: !cfg.CacheMaxBytesExplicit,
+		DisableDiskCache:   cfg.CacheMaxBytesExplicit && cfg.CacheMaxBytes == 0,
+		Logger:             log,
+	}
+}
+
 // initImageService initializes the image cache and service.
 func (s *Handlers) initImageService() error {
-	// Create the cache. cache_max_bytes: 0 disables the disk cache
+	cache, err := imgcache.NewCache(s.db.DB(), newCacheConfig(s.config, s.log))
-	// entirely; any other value is the eviction limit in bytes.
-	cache, err := imgcache.NewCache(s.db.DB(), imgcache.CacheConfig{
-		StateDir:         s.config.StateDir,
-		CacheTTL:         imgcache.DefaultCacheTTL,
-		NegativeTTL:      imgcache.DefaultNegativeTTL,
-		MaxBytes:         s.config.CacheMaxBytes,
-		DisableDiskCache: s.config.CacheMaxBytes == 0,
-		Logger:           s.log,
-	})
 	if err != nil {
 		return err
 	}

internal/handlers/image.go

@@ -10,6 +10,7 @@ import (
 	"time"
 
 	"github.com/go-chi/chi/v5"
+	"github.com/go-chi/chi/v5/middleware"
 	"sneak.berlin/go/pixa/internal/encurl"
 	"sneak.berlin/go/pixa/internal/httpfetcher"
 	"sneak.berlin/go/pixa/internal/imageprocessor"
@@ -20,6 +21,10 @@ import (
 // /v1/image/<host>/<path>/<width>x<height>.<format>
 func (s *Handlers) HandleImage() http.HandlerFunc {
 	return func(w http.ResponseWriter, r *http.Request) {
+		if s.refuseBlockedReferer(w, r) {
+			return
+		}
+
 		req, ok := s.parseImageRequest(w, r)
 		if !ok {
 			return
@@ -247,6 +252,42 @@ func cacheControl(expires time.Time) string {
 	return fmt.Sprintf("public, max-age=%d, immutable", int64(maxAge/time.Second))
 }
 
+// refuseBlockedReferer answers 403 with a JSON error when the request's Referer
+// names a host on referer_blocklist, and reports whether it answered. A request
+// with no Referer, or one that does not parse as a URL with a host, is not
+// refused.
+func (s *Handlers) refuseBlockedReferer(
+	w http.ResponseWriter, r *http.Request,
+) bool {
+	referer, err := url.Parse(r.Referer())
+	if err != nil || !s.refererBlocklist.IsAllowed(referer) {
+		return false
+	}
+
+	s.respondError(w, "referer blocked", http.StatusForbidden)
+
+	return true
+}
+
+// notModified sets the ETag header to etag and, when the request's
+// If-None-Match is that ETag, answers 304 Not Modified. It reports whether it
+// answered. An empty etag sets no header and never answers.
+func notModified(w http.ResponseWriter, r *http.Request, etag string) bool {
+	if etag == "" {
+		return false
+	}
+
+	w.Header().Set("ETag", etag)
+
+	if r.Header.Get("If-None-Match") != etag {
+		return false
+	}
+
+	w.WriteHeader(http.StatusNotModified)
+
+	return true
+}
+
 // writeImageResponse writes headers and streams the image content,
 // handling conditional and HEAD requests.
 func (s *Handlers) writeImageResponse(
@@ -265,17 +306,8 @@ func (s *Handlers) writeImageResponse(
 	w.Header().Set("Cache-Control", cacheControl(req.Expires))
 	w.Header().Set("X-Pixa-Cache", string(resp.CacheStatus))
 
-	if resp.ETag != "" {
+	if notModified(w, r, resp.ETag) {
-		w.Header().Set("ETag", resp.ETag)
+		return
-
-		// Check for conditional request (If-None-Match)
-		if ifNoneMatch := r.Header.Get("If-None-Match"); ifNoneMatch != "" {
-			if ifNoneMatch == resp.ETag {
-				w.WriteHeader(http.StatusNotModified)
-
-				return
-			}
-		}
 	}
 
 	// Handle HEAD request - return headers only
@@ -298,6 +330,7 @@ func (s *Handlers) writeImageResponse(
 	// Log cache status and timing after serving
 	duration := time.Since(startTime)
 	s.log.Info("image served",
+		"request_id", middleware.GetReqID(r.Context()),
 		"cache_key", cacheKey,
 		"cache_status", resp.CacheStatus,
 		"duration_ms", duration.Milliseconds(),

internal/handlers/image_cache_control_internal_test.go

@@ -23,8 +23,10 @@ const photoPath = "/images/photo.jpg"
 // newSignedHostServer returns a router for both image routes, and the Handlers
 // behind it, whose fetcher serves a JPEG at photoPath on signedHost. signedHost
 // is not on the allowlist, so a /v1/image/ URL for it is served only with a
-// valid signature.
+// valid signature. The handlers and the image service log to log.
-func newSignedHostServer(t *testing.T) (*Handlers, http.Handler) {
+func newSignedHostServer(
+	t *testing.T, log *slog.Logger,
+) (*Handlers, http.Handler) {
 	t.Helper()
 
 	cache, err := imgcache.NewCache(setupTestDB(t), imgcache.CacheConfig{
@@ -44,6 +46,7 @@ func newSignedHostServer(t *testing.T) (*Handlers, http.Handler) {
 			signedHost + photoPath: &fstest.MapFile{Data: jpegData},
 		}),
 		SigningKey: testSigningKey,
+		Logger:     log,
 	})
 	if err != nil {
 		t.Fatalf("imgcache.NewService() error = %v", err)
@@ -55,7 +58,7 @@ func newSignedHostServer(t *testing.T) (*Handlers, http.Handler) {
 	}
 
 	h := &Handlers{
-		log:    slog.New(slog.DiscardHandler),
+		log:    log,
 		imgSvc: svc,
 		encGen: encGen,
 	}
@@ -103,7 +106,7 @@ func getMaxAge(t *testing.T, srv http.Handler, target string) int {
 func TestHandleImage_SignedURL_MaxAgeEndsAtExp(t *testing.T) {
 	t.Parallel()
 
-	h, srv := newSignedHostServer(t)
+	h, srv := newSignedHostServer(t, slog.New(slog.DiscardHandler))
 
 	signedURL, err := h.imgSvc.GenerateSignedURL("", &imgcache.ImageRequest{
 		SourceHost: signedHost,
@@ -179,7 +182,7 @@ func TestHandleImageEnc_MaxAge(t *testing.T) {
 		t.Run(tt.name, func(t *testing.T) {
 			t.Parallel()
 
-			h, srv := newSignedHostServer(t)
+			h, srv := newSignedHostServer(t, slog.New(slog.DiscardHandler))
 
 			token, err := h.encGen.Generate(&encurl.Payload{
 				SourceHost: signedHost,

internal/handlers/image_errors_internal_test.go

@@ -0,0 +1,267 @@
+package handlers
+
+import (
+	"encoding/json"
+	"fmt"
+	"image/color"
+	"log/slog"
+	"net/http"
+	"net/http/httptest"
+	"strings"
+	"testing"
+	"testing/fstest"
+	"time"
+
+	"github.com/go-chi/chi/v5"
+	"sneak.berlin/go/pixa/internal/httpfetcher"
+	"sneak.berlin/go/pixa/internal/imgcache"
+	"sneak.berlin/go/pixa/internal/signature"
+)
+
+// allowlistedHost is the only host on the allowlist of the image route
+// newImageRoute builds.
+const allowlistedHost = "allowed.example.com"
+
+// newImageRoute returns the image route of a Handlers whose service fetches
+// with fetcher and checks signatures with testSigningKey.
+func newImageRoute(t *testing.T, fetcher httpfetcher.Fetcher) http.Handler {
+	t.Helper()
+
+	cache, err := imgcache.NewCache(setupTestDB(t), imgcache.CacheConfig{
+		StateDir:    t.TempDir(),
+		CacheTTL:    time.Hour,
+		NegativeTTL: 5 * time.Minute,
+	})
+	if err != nil {
+		t.Fatalf("failed to create cache: %v", err)
+	}
+
+	svc, err := imgcache.NewService(&imgcache.ServiceConfig{
+		Cache:      cache,
+		Fetcher:    fetcher,
+		SigningKey: testSigningKey,
+		Allowlist:  []string{allowlistedHost},
+	})
+	if err != nil {
+		t.Fatalf("failed to create service: %v", err)
+	}
+
+	h := &Handlers{imgSvc: svc, log: slog.New(slog.DiscardHandler)}
+
+	r := chi.NewRouter()
+	r.Get("/v1/image/*", h.HandleImage())
+
+	return r
+}
+
+// newPhotoFetcher returns a mock fetcher that serves a JPEG at photoPath on
+// each of hosts, and answers any other URL with an upstream error.
+func newPhotoFetcher(t *testing.T, hosts ...string) *httpfetcher.MockFetcher {
+	t.Helper()
+
+	photo := &fstest.MapFile{
+		Data: generateTestJPEG(t, 100, 100, color.RGBA{255, 0, 0, 255}),
+	}
+
+	files := fstest.MapFS{}
+	for _, host := range hosts {
+		files[host+photoPath] = photo
+	}
+
+	return httpfetcher.NewMock(files)
+}
+
+// photoURL returns the image route URL of photoPath on host, as a 50x50 JPEG.
+func photoURL(host string) string {
+	return "/v1/image/" + host + photoPath + "/50x50.jpeg"
+}
+
+// photoURLWithSig returns photoURL(host) with sig and expires as its sig and
+// exp.
+func photoURLWithSig(host, sig string, expires time.Time) string {
+	return fmt.Sprintf("%s?sig=%s&exp=%d", photoURL(host), sig, expires.Unix())
+}
+
+// photoSignature returns the signature of photoURL(host) at the default
+// quality and fit, made with key and expiring at expires.
+func photoSignature(key, host string, expires time.Time) string {
+	return signature.New(key).Sign(&signature.Request{
+		SourceHost: host,
+		SourcePath: photoPath,
+		Width:      50,
+		Height:     50,
+		Format:     string(imgcache.FormatJPEG),
+		Quality:    85,
+		FitMode:    string(imgcache.FitCover),
+		Expires:    expires,
+	})
+}
+
+// sendGet sends a GET for target to route and returns the response.
+func sendGet(
+	t *testing.T, route http.Handler, target string,
+) *httptest.ResponseRecorder {
+	t.Helper()
+
+	req := httptest.NewRequestWithContext(t.Context(), http.MethodGet, target, nil)
+	rec := httptest.NewRecorder()
+
+	route.ServeHTTP(rec, req)
+	t.Logf("GET %s: %d", target, rec.Code)
+
+	return rec
+}
+
+// checkErrorBody checks that rec has status wantStatus and the JSON error body
+// the image route sends: wantError, wantStatus and the time in RFC 3339.
+func checkErrorBody(
+	t *testing.T, rec *httptest.ResponseRecorder, wantStatus int, wantError string,
+) {
+	t.Helper()
+
+	if rec.Code != wantStatus {
+		t.Errorf("status = %d, want %d", rec.Code, wantStatus)
+	}
+
+	if ct := rec.Header().Get("Content-Type"); ct != "application/json" {
+		t.Errorf("Content-Type = %q, want application/json", ct)
+	}
+
+	var body struct {
+		Error     string `json:"error"`
+		Status    int    `json:"status"`
+		Timestamp string `json:"timestamp"`
+	}
+
+	err := json.NewDecoder(rec.Body).Decode(&body)
+	if err != nil {
+		t.Fatalf("decoding response body: %v", err)
+	}
+
+	if body.Error != wantError || body.Status != wantStatus {
+		t.Errorf("body error and status = %q %d, want %q %d",
+			body.Error, body.Status, wantError, wantStatus)
+	}
+
+	_, err = time.Parse(time.RFC3339, body.Timestamp)
+	if err != nil {
+		t.Errorf("body timestamp: %v", err)
+	}
+}
+
+// TestHandleImage_ErrorAnswers checks the status and the JSON error body the
+// image route answers each request below with. The JPEG at photoPath exists on
+// signedHost and on each host below that differs from it, so a request refused
+// with 401 would otherwise be served.
+func TestHandleImage_ErrorAnswers(t *testing.T) {
+	t.Parallel()
+
+	// A signature for signedHost must not verify for any of these.
+	parentHost := "example.com"
+	siblingHost := "other.example.com"
+	subdomainHost := "img." + signedHost
+	appendedHost := signedHost + ".example.net"
+
+	photos := newPhotoFetcher(t,
+		signedHost, parentHost, siblingHost, subdomainHost, appendedHost)
+	// The real fetcher refuses localhost before any lookup or connection.
+	realFetcher := httpfetcher.New(httpfetcher.DefaultConfig())
+
+	exp := time.Now().Add(time.Hour)
+	expired := time.Now().Add(-time.Hour)
+	sig := photoSignature(testSigningKey, signedHost, exp)
+	otherKeySig := photoSignature("another-signing-key", signedHost, exp)
+	expiredSig := photoSignature(testSigningKey, signedHost, expired)
+	localhostSig := photoSignature(testSigningKey, "localhost", exp)
+
+	// The error every request refused for its signature gets.
+	const unauthorized = "unauthorized"
+
+	tests := []struct {
+		name       string
+		fetcher    httpfetcher.Fetcher
+		target     string
+		wantStatus int
+		wantError  string
+	}{
+		{"no sig or exp", photos, photoURL(signedHost),
+			http.StatusUnauthorized, unauthorized},
+		{"exp but no sig", photos,
+			fmt.Sprintf("%s?exp=%d", photoURL(signedHost), exp.Unix()),
+			http.StatusUnauthorized, unauthorized},
+		{"sig made with another key", photos,
+			photoURLWithSig(signedHost, otherKeySig, exp),
+			http.StatusUnauthorized, unauthorized},
+		{"sig without its = padding", photos,
+			photoURLWithSig(signedHost, strings.TrimRight(sig, "="), exp),
+			http.StatusUnauthorized, unauthorized},
+		{"sig in upper case", photos,
+			photoURLWithSig(signedHost, strings.ToUpper(sig), exp),
+			http.StatusUnauthorized, unauthorized},
+		{"expired sig", photos, photoURLWithSig(signedHost, expiredSig, expired),
+			http.StatusUnauthorized, unauthorized},
+		{"sig sent for the parent domain", photos,
+			photoURLWithSig(parentHost, sig, exp),
+			http.StatusUnauthorized, unauthorized},
+		{"sig sent for a sibling host", photos,
+			photoURLWithSig(siblingHost, sig, exp),
+			http.StatusUnauthorized, unauthorized},
+		{"sig sent for a subdomain", photos,
+			photoURLWithSig(subdomainHost, sig, exp),
+			http.StatusUnauthorized, unauthorized},
+		{"sig sent with another domain appended", photos,
+			photoURLWithSig(appendedHost, sig, exp),
+			http.StatusUnauthorized, unauthorized},
+		{"unparseable path", photos,
+			"/v1/image/" + allowlistedHost + photoPath + "/big.jpeg",
+			http.StatusBadRequest, "invalid image URL: invalid size format"},
+		{"blocked upstream address", realFetcher,
+			photoURLWithSig("localhost", localhostSig, exp),
+			http.StatusForbidden, "forbidden"},
+		{"upstream error", photos,
+			"/v1/image/" + allowlistedHost + "/images/missing.jpg/50x50.jpeg",
+			http.StatusBadGateway, "upstream error"},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			t.Parallel()
+
+			rec := sendGet(t, newImageRoute(t, tt.fetcher), tt.target)
+			checkErrorBody(t, rec, tt.wantStatus, tt.wantError)
+		})
+	}
+}
+
+// TestHandleImage_AllowlistOrSignature checks that the image route serves an
+// image without a signature for a host on the allowlist only, and for another
+// host only with a valid signature.
+func TestHandleImage_AllowlistOrSignature(t *testing.T) {
+	t.Parallel()
+
+	photos := newPhotoFetcher(t, allowlistedHost, signedHost)
+	exp := time.Now().Add(time.Hour)
+	sig := photoSignature(testSigningKey, signedHost, exp)
+
+	tests := []struct {
+		name       string
+		target     string
+		wantStatus int
+	}{
+		{"allowlisted host, no sig", photoURL(allowlistedHost), http.StatusOK},
+		{"other host, no sig", photoURL(signedHost), http.StatusUnauthorized},
+		{"other host, valid sig", photoURLWithSig(signedHost, sig, exp),
+			http.StatusOK},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			t.Parallel()
+
+			rec := sendGet(t, newImageRoute(t, photos), tt.target)
+			if rec.Code != tt.wantStatus {
+				t.Errorf("status = %d, want %d", rec.Code, tt.wantStatus)
+			}
+		})
+	}
+}

internal/handlers/imageenc.go

@@ -9,6 +9,7 @@ import (
 	"time"
 
 	"github.com/go-chi/chi/v5"
+	"github.com/go-chi/chi/v5/middleware"
 
 	"sneak.berlin/go/pixa/internal/encurl"
 	"sneak.berlin/go/pixa/internal/httpfetcher"
@@ -21,46 +22,15 @@ import (
 // browsers identify the content type.
 func (s *Handlers) HandleImageEnc() http.HandlerFunc {
 	return func(w http.ResponseWriter, r *http.Request) {
+		if s.refuseBlockedReferer(w, r) {
+			return
+		}
+
 		ctx := r.Context()
 		start := time.Now()
 
-		// Extract token from URL
+		req, ok := s.parseImageEncRequest(w, r)
-		token := chi.URLParam(r, "token")
+		if !ok {
-		if token == "" {
-			s.respondError(w, "missing token", http.StatusBadRequest)
-
-			return
-		}
-
-		// Decrypt and validate the payload
-		payload, err := s.encGen.Parse(token)
-		if err != nil {
-			if errors.Is(err, encurl.ErrExpired) {
-				s.log.Debug("encrypted URL expired", "error", err)
-				s.respondError(w, "URL has expired", http.StatusGone)
-
-				return
-			}
-
-			s.log.Debug("failed to decrypt URL", "error", err)
-			s.respondError(w, "invalid encrypted URL", http.StatusBadRequest)
-
-			return
-		}
-
-		// Convert payload to ImageRequest
-		req := payload.ToImageRequest()
-
-		// Apply the same dimension and fit-mode bounds as the plain image
-		// route: a sealed payload is trusted for its origin, not for staying
-		// within limits, so an over-limit size or unknown fit mode is a 400
-		// here rather than an out-of-memory or a 500 from the processor.
-		err = imgcache.ValidateImageRequest(req)
-		if err != nil {
-			s.log.Debug("encrypted URL failed validation", "error", err)
-			s.respondError(w, "invalid encrypted URL: "+err.Error(),
-				http.StatusBadRequest)
-
 			return
 		}
 
@@ -94,6 +64,17 @@ func (s *Handlers) HandleImageEnc() http.HandlerFunc {
 		w.Header().Set("Cache-Control", cacheControl(req.Expires))
 		w.Header().Set("X-Pixa-Cache", string(resp.CacheStatus))
 
+		if notModified(w, r, resp.ETag) {
+			return
+		}
+
+		// A HEAD request gets the headers only
+		if r.Method == http.MethodHead {
+			w.WriteHeader(http.StatusOK)
+
+			return
+		}
+
 		// Stream the response
 		written, err := io.Copy(w, resp.Content)
 		if err != nil {
@@ -105,6 +86,7 @@ func (s *Handlers) HandleImageEnc() http.HandlerFunc {
 		// Log completion
 		duration := time.Since(start)
 		s.log.Info("image served",
+			"request_id", middleware.GetReqID(ctx),
 			"cache_key", imgcache.CacheKey(req),
 			"host", req.SourceHost,
 			"path", req.SourcePath,
@@ -116,6 +98,56 @@ func (s *Handlers) HandleImageEnc() http.HandlerFunc {
 	}
 }
 
+// parseImageEncRequest decrypts the token of an encrypted image URL into an
+// ImageRequest and checks it. On a token that is missing, does not decrypt,
+// has expired or asks for something not valid, it writes an error response
+// and returns false.
+func (s *Handlers) parseImageEncRequest(
+	w http.ResponseWriter, r *http.Request,
+) (*imgcache.ImageRequest, bool) {
+	// Extract token from URL
+	token := chi.URLParam(r, "token")
+	if token == "" {
+		s.respondError(w, "missing token", http.StatusBadRequest)
+
+		return nil, false
+	}
+
+	// Decrypt and validate the payload
+	payload, err := s.encGen.Parse(token)
+	if err != nil {
+		if errors.Is(err, encurl.ErrExpired) {
+			s.log.Debug("encrypted URL expired", "error", err)
+			s.respondError(w, "URL has expired", http.StatusGone)
+
+			return nil, false
+		}
+
+		s.log.Debug("failed to decrypt URL", "error", err)
+		s.respondError(w, "invalid encrypted URL", http.StatusBadRequest)
+
+		return nil, false
+	}
+
+	// Convert payload to ImageRequest
+	req := payload.ToImageRequest()
+
+	// Apply the same dimension and fit-mode bounds as the plain image
+	// route: a sealed payload is trusted for its origin, not for staying
+	// within limits, so an over-limit size or unknown fit mode is a 400
+	// here rather than an out-of-memory or a 500 from the processor.
+	err = imgcache.ValidateImageRequest(req)
+	if err != nil {
+		s.log.Debug("encrypted URL failed validation", "error", err)
+		s.respondError(w, "invalid encrypted URL: "+err.Error(),
+			http.StatusBadRequest)
+
+		return nil, false
+	}
+
+	return req, true
+}
+
 // handleImageError converts image service errors to HTTP responses.
 func (s *Handlers) handleImageError(w http.ResponseWriter, err error) {
 	switch {

internal/handlers/imageenc_internal_test.go

@@ -96,3 +96,70 @@ func TestHandleImageEnc_InvalidFitMode_Returns400(t *testing.T) {
 		t.Fatalf("status = %d, want %d", rec.Code, http.StatusBadRequest)
 	}
 }
+
+// TestHandleImageEnc_IfNoneMatch_Returns304 verifies that an image served
+// through an encrypted URL carries an ETag, and that a request whose
+// If-None-Match is that ETag is answered 304 Not Modified with no body.
+func TestHandleImageEnc_IfNoneMatch_Returns304(t *testing.T) {
+	t.Parallel()
+
+	h, srv := newSignedHostServer(t, slog.New(slog.DiscardHandler))
+	target := encPhotoURL(t, h)
+
+	rec := httptest.NewRecorder()
+	srv.ServeHTTP(rec, httptest.NewRequestWithContext(
+		t.Context(), http.MethodGet, target, nil))
+
+	etag := rec.Header().Get("ETag")
+	t.Logf("GET: %d, ETag %q", rec.Code, etag)
+
+	if rec.Code != http.StatusOK || etag == "" {
+		t.Fatalf("GET: status = %d, ETag = %q, want %d and an ETag",
+			rec.Code, etag, http.StatusOK)
+	}
+
+	req := httptest.NewRequestWithContext(t.Context(), http.MethodGet, target, nil)
+	req.Header.Set("If-None-Match", etag)
+
+	rec = httptest.NewRecorder()
+	srv.ServeHTTP(rec, req)
+	t.Logf("GET with If-None-Match: %d, %d body bytes", rec.Code, rec.Body.Len())
+
+	if rec.Code != http.StatusNotModified || rec.Body.Len() != 0 {
+		t.Errorf("status = %d with %d body bytes, want %d with none",
+			rec.Code, rec.Body.Len(), http.StatusNotModified)
+	}
+}
+
+// TestHandleImageEnc_HEAD_ReturnsHeadersOnly verifies that HEAD on an
+// encrypted URL is answered 200 with the headers GET sends and no body.
+func TestHandleImageEnc_HEAD_ReturnsHeadersOnly(t *testing.T) {
+	t.Parallel()
+
+	h, _ := newSignedHostServer(t, slog.New(slog.DiscardHandler))
+
+	r := chi.NewRouter()
+	r.Head("/v1/e/{token}/*", h.HandleImageEnc())
+
+	rec := httptest.NewRecorder()
+	r.ServeHTTP(rec, httptest.NewRequestWithContext(
+		t.Context(), http.MethodHead, encPhotoURL(t, h), nil))
+	t.Logf("HEAD: %d, headers %v, %d body bytes",
+		rec.Code, rec.Header(), rec.Body.Len())
+
+	if rec.Code != http.StatusOK {
+		t.Fatalf("status = %d, want %d", rec.Code, http.StatusOK)
+	}
+
+	for _, name := range []string{
+		"Content-Type", "Content-Length", "Cache-Control", "ETag",
+	} {
+		if rec.Header().Get(name) == "" {
+			t.Errorf("HEAD response has no %s", name)
+		}
+	}
+
+	if rec.Body.Len() != 0 {
+		t.Errorf("HEAD response body has %d bytes, want none", rec.Body.Len())
+	}
+}

internal/handlers/imageenc_token_internal_test.go

@@ -0,0 +1,126 @@
+package handlers
+
+import (
+	"image/jpeg"
+	"log/slog"
+	"net/http"
+	"net/http/httptest"
+	"testing"
+	"time"
+
+	"sneak.berlin/go/pixa/internal/encurl"
+)
+
+// requireServedPhoto requires that rec answers 200 with the JPEG at photoPath
+// on signedHost at the 50x50 that encPhotoURL and the generator tests ask for.
+func requireServedPhoto(t *testing.T, rec *httptest.ResponseRecorder) {
+	t.Helper()
+
+	if rec.Code != http.StatusOK {
+		t.Fatalf("status = %d, want %d; body %q",
+			rec.Code, http.StatusOK, rec.Body.String())
+	}
+
+	contentType := rec.Header().Get("Content-Type")
+	if contentType != "image/jpeg" {
+		t.Errorf("Content-Type = %q, want image/jpeg", contentType)
+	}
+
+	img, err := jpeg.DecodeConfig(rec.Body)
+	if err != nil {
+		t.Fatalf("body is not a JPEG: %v", err)
+	}
+
+	if img.Width != 50 || img.Height != 50 {
+		t.Errorf("image is %dx%d, want 50x50", img.Width, img.Height)
+	}
+}
+
+// TestHandleImageEnc_ValidToken_ServesImage verifies that a token made with
+// the signing key serves the image it asks for.
+func TestHandleImageEnc_ValidToken_ServesImage(t *testing.T) {
+	t.Parallel()
+
+	h, srv := newSignedHostServer(t, slog.New(slog.DiscardHandler))
+
+	rec := httptest.NewRecorder()
+	srv.ServeHTTP(rec, httptest.NewRequestWithContext(
+		t.Context(), http.MethodGet, encPhotoURL(t, h), nil))
+
+	requireServedPhoto(t, rec)
+}
+
+// TestHandleImageEnc_RejectedToken verifies that a token that has expired
+// answers 410, and that a token with one character changed, a token cut
+// short, and a token made with another signing key answer 400. The server
+// would serve the photo for a token it accepted.
+func TestHandleImageEnc_RejectedToken(t *testing.T) {
+	t.Parallel()
+
+	h, srv := newSignedHostServer(t, slog.New(slog.DiscardHandler))
+
+	photo := encurl.Payload{
+		SourceHost: signedHost,
+		SourcePath: photoPath,
+		Width:      50,
+		Height:     50,
+	}
+
+	valid, err := h.encGen.Generate(&photo)
+	if err != nil {
+		t.Fatalf("Generate() error = %v", err)
+	}
+
+	expiredPhoto := photo
+	expiredPhoto.ExpiresAt = time.Now().Add(-time.Minute).Unix()
+
+	expired, err := h.encGen.Generate(&expiredPhoto)
+	if err != nil {
+		t.Fatalf("Generate() error = %v", err)
+	}
+
+	otherGen, err := encurl.NewGenerator("another-signing-key-fedcba9876543210")
+	if err != nil {
+		t.Fatalf("encurl.NewGenerator() error = %v", err)
+	}
+
+	otherKey, err := otherGen.Generate(&photo)
+	if err != nil {
+		t.Fatalf("Generate() error = %v", err)
+	}
+
+	// Changing a character in the middle always changes the decoded bytes;
+	// the last character of unpadded base64 can carry unused bits.
+	middle := len(valid) / 2
+
+	replacement := "A"
+	if valid[middle] == 'A' {
+		replacement = "B"
+	}
+
+	changed := valid[:middle] + replacement + valid[middle+1:]
+
+	tests := []struct {
+		name       string
+		token      string
+		wantStatus int
+	}{
+		{"expired", expired, http.StatusGone},
+		{"one character changed", changed, http.StatusBadRequest},
+		{"cut short", valid[:middle], http.StatusBadRequest},
+		{"another signing key", otherKey, http.StatusBadRequest},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			t.Parallel()
+
+			rec := getEncToken(srv, tt.token)
+			t.Logf("GET /v1/e/%s/img.jpg: %d %s", tt.token, rec.Code, rec.Body)
+
+			if rec.Code != tt.wantStatus {
+				t.Errorf("status = %d, want %d", rec.Code, tt.wantStatus)
+			}
+		})
+	}
+}

internal/handlers/referer_blocklist_internal_test.go

@@ -0,0 +1,185 @@
+package handlers
+
+import (
+	"context"
+	"log/slog"
+	"net/http"
+	"net/http/httptest"
+	"sync/atomic"
+	"testing"
+	"time"
+
+	"github.com/go-chi/chi/v5"
+	"sneak.berlin/go/pixa/internal/allowlist"
+	"sneak.berlin/go/pixa/internal/encurl"
+	"sneak.berlin/go/pixa/internal/httpfetcher"
+	"sneak.berlin/go/pixa/internal/imgcache"
+)
+
+// blockedReferer is a page on leech.example, which newRefererRoutes puts on
+// referer_blocklist.
+const blockedReferer = "https://leech.example/page.html"
+
+// countingFetcher passes each fetch on to the fetcher it holds and counts it.
+type countingFetcher struct {
+	httpfetcher.Fetcher
+
+	fetches atomic.Int32
+}
+
+// Fetch counts the fetch and passes it on.
+func (f *countingFetcher) Fetch(
+	ctx context.Context, url string,
+) (*httpfetcher.FetchResult, error) {
+	f.fetches.Add(1)
+
+	return f.Fetcher.Fetch(ctx, url)
+}
+
+// newRefererRoutes returns both image routes of a Handlers whose
+// referer_blocklist is "leech.example" and ".hotlinker.example", the
+// Handlers, and the fetcher the routes fetch through. The JPEG at photoPath
+// exists on allowlistedHost and on signedHost.
+func newRefererRoutes(t *testing.T) (http.Handler, *Handlers, *countingFetcher) {
+	t.Helper()
+
+	fetcher := &countingFetcher{
+		Fetcher: newPhotoFetcher(t, allowlistedHost, signedHost),
+	}
+
+	cache, err := imgcache.NewCache(setupTestDB(t), imgcache.CacheConfig{
+		StateDir:    t.TempDir(),
+		CacheTTL:    time.Hour,
+		NegativeTTL: 5 * time.Minute,
+	})
+	if err != nil {
+		t.Fatalf("imgcache.NewCache() error = %v", err)
+	}
+
+	svc, err := imgcache.NewService(&imgcache.ServiceConfig{
+		Cache:      cache,
+		Fetcher:    fetcher,
+		SigningKey: testSigningKey,
+		Allowlist:  []string{allowlistedHost},
+	})
+	if err != nil {
+		t.Fatalf("imgcache.NewService() error = %v", err)
+	}
+
+	encGen, err := encurl.NewGenerator(testSigningKey)
+	if err != nil {
+		t.Fatalf("encurl.NewGenerator() error = %v", err)
+	}
+
+	h := &Handlers{
+		log:    slog.New(slog.DiscardHandler),
+		imgSvc: svc,
+		encGen: encGen,
+		refererBlocklist: allowlist.New(
+			[]string{"leech.example", ".hotlinker.example"}),
+	}
+
+	r := chi.NewRouter()
+	r.Get("/v1/image/*", h.HandleImage())
+	r.Get("/v1/e/{token}/*", h.HandleImageEnc())
+
+	return r, h, fetcher
+}
+
+// getWithReferer sends a GET for target to routes with referer as its
+// Referer header, or with none when referer is empty, and returns the
+// response.
+func getWithReferer(
+	t *testing.T, routes http.Handler, target, referer string,
+) *httptest.ResponseRecorder {
+	t.Helper()
+
+	req := httptest.NewRequestWithContext(t.Context(), http.MethodGet, target, nil)
+	if referer != "" {
+		req.Header.Set("Referer", referer)
+	}
+
+	rec := httptest.NewRecorder()
+
+	routes.ServeHTTP(rec, req)
+	t.Logf("GET %s with Referer %q: %d", target, referer, rec.Code)
+
+	return rec
+}
+
+// TestRefererBlocklist verifies that both image routes refuse a request whose
+// Referer names a host on referer_blocklist with 403 and the JSON error,
+// without fetching from the upstream host, and serve a request with no
+// Referer, one that does not parse, or one naming any other host. Hosts are
+// matched as allowlist_hosts matches them.
+func TestRefererBlocklist(t *testing.T) {
+	t.Parallel()
+
+	cases := []struct {
+		name    string
+		referer string
+		want    int
+	}{
+		{"no referer", "", http.StatusOK},
+		{"unlisted host", "https://unlisted.example/page.html", http.StatusOK},
+		{"unparseable", "%zz", http.StatusOK},
+		{"listed host", blockedReferer, http.StatusForbidden},
+		{"subdomain of listed host", "https://www.leech.example/", http.StatusOK},
+		{"subdomain of dot pattern", "https://www.hotlinker.example/a.html",
+			http.StatusForbidden},
+		{"dot pattern without its dot", "https://hotlinker.example/",
+			http.StatusForbidden},
+		{"host continuing past dot pattern",
+			"https://hotlinker.example.evil.example/", http.StatusOK},
+	}
+
+	// The photo's URL on each image route.
+	photoURLs := map[string]func(t *testing.T, h *Handlers) string{
+		"plain URL": func(t *testing.T, _ *Handlers) string {
+			t.Helper()
+
+			return photoURL(allowlistedHost)
+		},
+		"encrypted URL": encPhotoURL,
+	}
+
+	for urlName, photoURLFor := range photoURLs {
+		for _, tc := range cases {
+			t.Run(urlName+", "+tc.name, func(t *testing.T) {
+				t.Parallel()
+
+				routes, h, fetcher := newRefererRoutes(t)
+
+				rec := getWithReferer(t, routes, photoURLFor(t, h), tc.referer)
+
+				if tc.want == http.StatusOK {
+					requireServedPhoto(t, rec)
+
+					return
+				}
+
+				checkErrorBody(t, rec, http.StatusForbidden, "referer blocked")
+
+				if n := fetcher.fetches.Load(); n != 0 {
+					t.Errorf("upstream fetched %d times, want 0", n)
+				}
+			})
+		}
+	}
+}
+
+// TestBlockedRefererRefusedWhenImageIsCached verifies that a request whose
+// Referer is on referer_blocklist is refused even when the image it asks for
+// is already cached, so the answer does not depend on the cache.
+func TestBlockedRefererRefusedWhenImageIsCached(t *testing.T) {
+	t.Parallel()
+
+	routes, h, _ := newRefererRoutes(t)
+
+	for _, target := range []string{photoURL(allowlistedHost), encPhotoURL(t, h)} {
+		requireServedPhoto(t, getWithReferer(t, routes, target, ""))
+
+		rec := getWithReferer(t, routes, target, blockedReferer)
+		checkErrorBody(t, rec, http.StatusForbidden, "referer blocked")
+	}
+}

internal/handlers/request_id_internal_test.go

@@ -0,0 +1,131 @@
+package handlers
+
+import (
+	"bytes"
+	"context"
+	"encoding/json"
+	"io"
+	"log/slog"
+	"net/http"
+	"net/http/httptest"
+	"testing"
+	"time"
+
+	"github.com/go-chi/chi/v5/middleware"
+
+	"sneak.berlin/go/pixa/internal/encurl"
+	"sneak.berlin/go/pixa/internal/imgcache"
+)
+
+// signedPhotoURL returns a signed /v1/image/ URL, valid for a minute, for the
+// JPEG at photoPath on signedHost at 50x50, made with h's image service.
+func signedPhotoURL(t *testing.T, h *Handlers) string {
+	t.Helper()
+
+	signedURL, err := h.imgSvc.GenerateSignedURL("", &imgcache.ImageRequest{
+		SourceHost: signedHost,
+		SourcePath: photoPath,
+		Size:       imgcache.Size{Width: 50, Height: 50},
+		Format:     imgcache.FormatJPEG,
+	}, time.Minute)
+	if err != nil {
+		t.Fatalf("GenerateSignedURL() error = %v", err)
+	}
+
+	return signedURL
+}
+
+// encPhotoURL returns an encrypted /v1/e/ URL, which never expires, for the
+// JPEG at photoPath on signedHost at 50x50, made with h's generator.
+func encPhotoURL(t *testing.T, h *Handlers) string {
+	t.Helper()
+
+	token, err := h.encGen.Generate(&encurl.Payload{
+		SourceHost: signedHost,
+		SourcePath: photoPath,
+		Width:      50,
+		Height:     50,
+		Format:     imgcache.FormatJPEG,
+	})
+	if err != nil {
+		t.Fatalf("Generate() error = %v", err)
+	}
+
+	return "/v1/e/" + token + "/img.jpg"
+}
+
+// requestIDByMessage reads the JSON log lines in logs and returns the
+// request_id of each line, by its message.
+func requestIDByMessage(t *testing.T, logs io.Reader) map[string]string {
+	t.Helper()
+
+	logged := make(map[string]string)
+
+	dec := json.NewDecoder(logs)
+	for dec.More() {
+		var line map[string]any
+
+		err := dec.Decode(&line)
+		if err != nil {
+			t.Fatalf("decoding log line: %v", err)
+		}
+
+		msg, _ := line["msg"].(string)
+		requestID, _ := line["request_id"].(string)
+		logged[msg] = requestID
+	}
+
+	return logged
+}
+
+// TestImageLogLinesCarryRequestID verifies that the lines logged when an image
+// is fetched, converted and served through either image route carry the
+// request's ID as request_id, as the request log line does, so they can be
+// found from it.
+func TestImageLogLinesCarryRequestID(t *testing.T) {
+	t.Parallel()
+
+	const requestID = "test-request-id"
+
+	imageURLs := map[string]func(*testing.T, *Handlers) string{
+		"/v1/image/": signedPhotoURL,
+		"/v1/e/":     encPhotoURL,
+	}
+
+	for route, imageURL := range imageURLs {
+		t.Run(route, func(t *testing.T) {
+			t.Parallel()
+
+			var logs bytes.Buffer
+
+			h, srv := newSignedHostServer(t,
+				slog.New(slog.NewJSONHandler(&logs, nil)))
+
+			ctx := context.WithValue(t.Context(),
+				middleware.RequestIDKey, requestID)
+			rec := httptest.NewRecorder()
+			srv.ServeHTTP(rec, httptest.NewRequestWithContext(
+				ctx, http.MethodGet, imageURL(t, h), nil))
+
+			if rec.Code != http.StatusOK {
+				t.Fatalf("status = %d, want %d", rec.Code, http.StatusOK)
+			}
+
+			t.Logf("logged:\n%s", logs.String())
+
+			logged := requestIDByMessage(t, &logs)
+
+			for _, msg := range []string{
+				"upstream fetched", "image converted", "image served",
+			} {
+				got, ok := logged[msg]
+				if !ok {
+					t.Errorf("no %q line logged", msg)
+				} else if got != requestID {
+					t.Errorf("%q line has request_id %q, want %q",
+						msg, got, requestID)
+				}
+			}
+		})
+	}
+}

internal/handlers/robots_healthcheck_internal_test.go

@@ -0,0 +1,90 @@
+package handlers
+
+import (
+	"encoding/json"
+	"log/slog"
+	"net/http"
+	"testing"
+
+	"go.uber.org/fx/fxtest"
+	"sneak.berlin/go/pixa/internal/config"
+	"sneak.berlin/go/pixa/internal/globals"
+	"sneak.berlin/go/pixa/internal/healthcheck"
+	"sneak.berlin/go/pixa/internal/logger"
+)
+
+// TestHandleRobotsTxt checks that /robots.txt asks every crawler to stay off
+// the whole site.
+func TestHandleRobotsTxt(t *testing.T) {
+	t.Parallel()
+
+	h := &Handlers{log: slog.New(slog.DiscardHandler)}
+	rec := sendGet(t, h.HandleRobotsTxt(), "/robots.txt")
+
+	if rec.Code != http.StatusOK {
+		t.Errorf("status = %d, want %d", rec.Code, http.StatusOK)
+	}
+
+	if ct := rec.Header().Get("Content-Type"); ct != "text/plain" {
+		t.Errorf("Content-Type = %q, want text/plain", ct)
+	}
+
+	want := "User-agent: *\nDisallow: /\n"
+	if rec.Body.String() != want {
+		t.Errorf("body = %q, want %q", rec.Body.String(), want)
+	}
+}
+
+// TestHandleHealthCheck checks that the health check answers 200 with status
+// ok, the app's name and version, now, uptime_seconds, uptime_human and
+// maintenance_mode, which is true here: the health check stays 200 while
+// maintenance mode is on.
+func TestHandleHealthCheck(t *testing.T) {
+	t.Parallel()
+
+	lc := fxtest.NewLifecycle(t)
+
+	log, err := logger.New(lc, logger.Params{Globals: &globals.Globals{}})
+	if err != nil {
+		t.Fatalf("logger.New() error = %v", err)
+	}
+
+	hc, err := healthcheck.New(lc, healthcheck.Params{
+		Globals: &globals.Globals{Appname: "pixad", Version: "v1.2.3"},
+		Config:  &config.Config{MaintenanceMode: true},
+		Logger:  log,
+	})
+	if err != nil {
+		t.Fatalf("healthcheck.New() error = %v", err)
+	}
+
+	h := &Handlers{hc: hc, log: slog.New(slog.DiscardHandler)}
+	rec := sendGet(t, h.HandleHealthCheck(), "/.well-known/healthcheck.json")
+
+	if rec.Code != http.StatusOK {
+		t.Errorf("status = %d, want %d", rec.Code, http.StatusOK)
+	}
+
+	if ct := rec.Header().Get("Content-Type"); ct != "application/json" {
+		t.Errorf("Content-Type = %q, want application/json", ct)
+	}
+
+	var body map[string]any
+
+	err = json.NewDecoder(rec.Body).Decode(&body)
+	if err != nil {
+		t.Fatalf("decoding response body: %v", err)
+	}
+
+	if body["status"] != "ok" || body["appname"] != "pixad" ||
+		body["version"] != "v1.2.3" || body["maintenance_mode"] != true {
+		t.Errorf("body = %v, want status ok, appname pixad, version v1.2.3 "+
+			"and maintenance_mode true", body)
+	}
+
+	for _, key := range []string{"now", "uptime_seconds", "uptime_human"} {
+		if _, ok := body[key]; !ok {
+			t.Errorf("body = %v, has no %s", body, key)
+		}
+	}
+}

internal/httpfetcher/fetch_internal_test.go

@@ -191,7 +191,23 @@ func fetchBody(t *testing.T, f *HTTPFetcher, path string) string {
 
 // semLen reports how many per-host semaphore slots are currently held.
 func semLen(f *HTTPFetcher, host string) int {
-	return len(f.getHostSemaphore(host))
+	f.hostSemMu.Lock()
+	defer f.hostSemMu.Unlock()
+
+	sem, ok := f.hostSems[host]
+	if !ok {
+		return 0
+	}
+
+	return len(sem.slots)
+}
+
+// hostSemCount reports how many hosts have a semaphore in hostSems.
+func hostSemCount(f *HTTPFetcher) int {
+	f.hostSemMu.Lock()
+	defer f.hostSemMu.Unlock()
+
+	return len(f.hostSems)
 }
 
 func TestFetchRedirectToPrivateIPBlocked(t *testing.T) {

internal/httpfetcher/httpfetcher.go

@@ -18,6 +18,8 @@ import (
 	"strings"
 	"sync"
 	"time"
+
+	"github.com/go-chi/chi/v5/middleware"
 )
 
 // Fetcher configuration constants.
@@ -160,10 +162,13 @@ func DefaultConfig() *Config {
 // HTTPFetcher implements Fetcher with SSRF protection and connection limits
 // per host and for all hosts together.
 type HTTPFetcher struct {
-	client    *http.Client
+	client *http.Client
-	config    *Config
+	config *Config
-	hostSems  map[string]chan struct{} // per-host semaphores
+	// hostSems holds the semaphore of each host with a fetch holding or
-	hostSemMu sync.Mutex               // protects hostSems map
+	// waiting for one of its slots; the entry is removed when the host's
+	// last such fetch gives its slot back or stops waiting.
+	hostSems  map[string]*hostSemaphore
+	hostSemMu sync.Mutex // protects hostSems and each entry's count
 	// allHostsSemaphore has one slot per connection allowed to all hosts
 	// together (config.MaxConnections).
 	allHostsSemaphore chan struct{}
@@ -171,6 +176,14 @@ type HTTPFetcher struct {
 	connectionWaitTimeout time.Duration
 }
 
+// hostSemaphore is one host's connection slots
+// (config.MaxConnectionsPerHost) and the number of fetches holding or
+// waiting for one of them.
+type hostSemaphore struct {
+	slots chan struct{}
+	count int
+}
+
 // New creates a new HTTPFetcher with SSRF protection.
 func New(config *Config) *HTTPFetcher {
 	if config == nil {
@@ -211,7 +224,7 @@ func New(config *Config) *HTTPFetcher {
 	return &HTTPFetcher{
 		client:                client,
 		config:                config,
-		hostSems:              make(map[string]chan struct{}),
+		hostSems:              make(map[string]*hostSemaphore),
 		allHostsSemaphore:     make(chan struct{}, config.MaxConnections),
 		connectionWaitTimeout: ConnectionWaitTimeout,
 	}
@@ -256,6 +269,13 @@ func (f *HTTPFetcher) Fetch(ctx context.Context, url string) (*FetchResult, erro
 	req.Header.Set("User-Agent", f.config.UserAgent)
 	req.Header.Set("Accept", strings.Join(f.config.AllowedContentTypes, ", "))
 
+	// The ID of the request this fetch serves, so the fetch can be found in
+	// the upstream host's logs
+	requestID := middleware.GetReqID(ctx)
+	if requestID != "" {
+		req.Header.Set(middleware.RequestIDHeader, requestID)
+	}
+
 	// Use httptrace to capture connection details
 	var remoteAddr string
 
@@ -307,6 +327,8 @@ func (f *HTTPFetcher) acquireConnection(
 	select {
 	case hostSem <- struct{}{}:
 	case <-ctx.Done():
+		f.putHostSemaphore(host)
+
 		return nil, ctx.Err()
 	}
 
@@ -314,32 +336,55 @@ func (f *HTTPFetcher) acquireConnection(
 	case f.allHostsSemaphore <- struct{}{}:
 	case <-time.After(f.connectionWaitTimeout):
 		<-hostSem
+		f.putHostSemaphore(host)
 
 		return nil, ErrTooManyConnections
 	case <-ctx.Done():
 		<-hostSem
+		f.putHostSemaphore(host)
 
 		return nil, ctx.Err()
 	}
 
 	return func() {
 		<-hostSem
+		f.putHostSemaphore(host)
 		<-f.allHostsSemaphore
 	}, nil
 }
 
-// getHostSemaphore returns the semaphore for a host, creating it if necessary.
+// getHostSemaphore returns the semaphore for a host, creating it if
+// necessary, and counts the caller among the fetches using it. The caller
+// calls putHostSemaphore once it holds no slot and waits for none.
 func (f *HTTPFetcher) getHostSemaphore(host string) chan struct{} {
 	f.hostSemMu.Lock()
 	defer f.hostSemMu.Unlock()
 
 	sem, ok := f.hostSems[host]
 	if !ok {
-		sem = make(chan struct{}, f.config.MaxConnectionsPerHost)
+		sem = &hostSemaphore{
+			slots: make(chan struct{}, f.config.MaxConnectionsPerHost),
+		}
 		f.hostSems[host] = sem
 	}
 
-	return sem
+	sem.count++
+
+	return sem.slots
+}
+
+// putHostSemaphore stops counting the caller among the fetches using the
+// host's semaphore, and removes the semaphore when no fetch uses it.
+func (f *HTTPFetcher) putHostSemaphore(host string) {
+	f.hostSemMu.Lock()
+	defer f.hostSemMu.Unlock()
+
+	sem := f.hostSems[host]
+
+	sem.count--
+	if sem.count == 0 {
+		delete(f.hostSems, host)
+	}
 }
 
 // buildResult validates the upstream response and assembles a FetchResult

internal/httpfetcher/max_connections_internal_test.go

@@ -5,6 +5,7 @@ import (
 	"errors"
 	"net"
 	"strconv"
+	"sync"
 	"testing"
 	"time"
 )
@@ -119,6 +120,98 @@ func TestFetchFreesHostSlotWhenContextEndsWaitingForConnection(t *testing.T) {
 	}
 }
 
+// TestFetchRemovesIdleHostSemaphores checks that a host's semaphore is
+// removed once no fetch holds or waits for one of its slots: after 100
+// concurrent fetches from 50 hosts have all finished, no semaphore is left.
+func TestFetchRemovesIdleHostSemaphores(t *testing.T) {
+	t.Parallel()
+
+	srv := startUpstream(t)
+	f, _ := newServerFetcher(t, srv, nil)
+	ctx := testContext(t)
+
+	var wg sync.WaitGroup
+
+	for i := range 100 {
+		wg.Go(func() {
+			res, err := f.Fetch(ctx, imageURLOnPort(1+i%50))
+			if err != nil {
+				t.Errorf("Fetch() error = %v", err)
+
+				return
+			}
+
+			_ = res.Content.Close()
+		})
+	}
+
+	wg.Wait()
+
+	if n := hostSemCount(f); n != 0 {
+		t.Errorf("%d host semaphores left after every fetch finished, want 0", n)
+	}
+}
+
+// TestFetchRemovesHostSemaphoreWhenNoConnection checks that a fetch that
+// ends without a connection leaves no semaphore behind: when it is refused
+// after waiting for a connection shared by all hosts, when its context ends
+// while it waits for its host's slot, and when its context ends while it
+// waits for a connection shared by all hosts, long before the 10 second
+// wait timeout.
+func TestFetchRemovesHostSemaphoreWhenNoConnection(t *testing.T) {
+	t.Parallel()
+
+	srv := startUpstream(t)
+
+	cfg := DefaultConfig()
+	cfg.MaxConnections = 1
+	cfg.MaxConnectionsPerHost = 1
+
+	f, _ := newServerFetcher(t, srv, cfg)
+	f.connectionWaitTimeout = 100 * time.Millisecond
+
+	open, err := f.Fetch(testContext(t), imageURLOnPort(81))
+	if err != nil {
+		t.Fatalf("first Fetch() error = %v", err)
+	}
+
+	_, err = f.Fetch(testContext(t), imageURLOnPort(82))
+	if !errors.Is(err, ErrTooManyConnections) {
+		t.Fatalf("Fetch() from another host: error = %v, "+
+			"want ErrTooManyConnections", err)
+	}
+
+	ctx, cancel := context.WithTimeout(t.Context(), 100*time.Millisecond)
+	defer cancel()
+
+	_, err = f.Fetch(ctx, imageURLOnPort(81))
+	if !errors.Is(err, context.DeadlineExceeded) {
+		t.Fatalf("Fetch() from the busy host: error = %v, "+
+			"want context.DeadlineExceeded", err)
+	}
+
+	// Back to the 10 second wait, so the next fetch's context ends first.
+	f.connectionWaitTimeout = ConnectionWaitTimeout
+
+	ctx, cancel = context.WithTimeout(t.Context(), 100*time.Millisecond)
+	defer cancel()
+
+	_, err = f.Fetch(ctx, imageURLOnPort(83))
+	if !errors.Is(err, context.DeadlineExceeded) {
+		t.Fatalf("Fetch() from a host with nothing open: error = %v, "+
+			"want context.DeadlineExceeded", err)
+	}
+
+	err = open.Content.Close()
+	if err != nil {
+		t.Fatalf("close first body: %v", err)
+	}
+
+	if n := hostSemCount(f); n != 0 {
+		t.Errorf("%d host semaphores left after every fetch finished, want 0", n)
+	}
+}
+
 // TestFetchReleasesConnectionOnError checks that a fetch that fails after
 // taking its connection gives it back: with MaxConnections at 1, the slot
 // must be free after the failure and the next fetch must succeed.

internal/httpfetcher/request_id_internal_test.go

@@ -0,0 +1,50 @@
+package httpfetcher
+
+import (
+	"context"
+	"io"
+	"net/http"
+	"net/http/httptest"
+	"testing"
+
+	"github.com/go-chi/chi/v5/middleware"
+)
+
+// TestFetchSendsRequestID verifies that a fetch sends the ID of the request
+// it serves, which the RequestID middleware stores in the request context,
+// to the upstream host as X-Request-Id, so the fetch can be found in that
+// host's logs.
+func TestFetchSendsRequestID(t *testing.T) {
+	t.Parallel()
+
+	const requestID = "test-request-id"
+
+	received := make(chan string, 1)
+
+	srv := httptest.NewServer(http.HandlerFunc(
+		func(w http.ResponseWriter, r *http.Request) {
+			received <- r.Header.Get("X-Request-Id")
+
+			w.Header().Set("Content-Type", contentTypeJPEG)
+			_, _ = io.WriteString(w, imagePayload)
+		}))
+	t.Cleanup(srv.Close)
+
+	f, _ := newServerFetcher(t, srv, nil)
+
+	ctx := context.WithValue(testContext(t), middleware.RequestIDKey, requestID)
+
+	res, err := f.Fetch(ctx, upstreamURL("/image"))
+	if err != nil {
+		t.Fatalf("Fetch() error = %v", err)
+	}
+
+	_ = res.Content.Close()
+
+	got := <-received
+	t.Logf("upstream received X-Request-Id %q", got)
+
+	if got != requestID {
+		t.Errorf("upstream X-Request-Id = %q, want %q", got, requestID)
+	}
+}

internal/imageprocessor/imageprocessor.go

@@ -331,6 +331,31 @@ func FormatToMIME(format Format) string {
 	}
 }
 
+// WaitForProcessing waits until no image is being processed, or until ctx
+// ends, and returns how many images were still being processed then. It
+// waits by taking each slot in processingSemaphore as it frees up until it
+// holds them all, or until ctx ends, then gives back the slots it took.
+func (p *ImageProcessor) WaitForProcessing(ctx context.Context) int {
+	taken := 0
+
+	defer func() {
+		for range taken {
+			<-p.processingSemaphore
+		}
+	}()
+
+	for taken < cap(p.processingSemaphore) {
+		select {
+		case p.processingSemaphore <- struct{}{}:
+			taken++
+		case <-ctx.Done():
+			return len(p.processingSemaphore) - taken
+		}
+	}
+
+	return 0
+}
+
 // acquireSlot takes a slot in processingSemaphore, waiting at most
 // processingWaitTimeout for one to free up, and returns the func that gives
 // it back. A free slot is taken even when ctx has ended; only the wait for

internal/imageprocessor/max_concurrent_processing_internal_test.go

@@ -300,3 +300,69 @@ func TestProcessReleasesSlotOnError(t *testing.T) {
 		})
 	}
 }
+
+// TestWaitForProcessing holds a processing slot with a Process call that
+// cannot finish reading its input. WaitForProcessing must report that image
+// when its context ends first, wait for it otherwise, return 0 once it has
+// finished, and give back the slots it took while waiting.
+func TestWaitForProcessing(t *testing.T) {
+	t.Parallel()
+
+	proc := New(Params{MaxConcurrentProcessing: 2})
+
+	gate := make(chan struct{})
+	entered := make(chan struct{}, 1)
+	results := make(chan error, 1)
+
+	openGate := sync.OnceFunc(func() { close(gate) })
+	t.Cleanup(openGate)
+
+	processInBackground(proc, &gatedReader{
+		data: bytes.NewReader(createTestJPEG(t, 10, 10)), gate: gate,
+		entered: entered, counter: &readingCounter{},
+	}, results)
+	waitForEntries(t, entered, 1)
+
+	ctx, cancel := context.WithTimeout(t.Context(), 100*time.Millisecond)
+	defer cancel()
+
+	stillProcessing := proc.WaitForProcessing(ctx)
+	t.Logf("WaitForProcessing() after its context ended: %d", stillProcessing)
+
+	if stillProcessing != 1 {
+		t.Errorf("WaitForProcessing() after its context ended = %d, want 1",
+			stillProcessing)
+	}
+
+	waited := make(chan int, 1)
+
+	go func() { waited <- proc.WaitForProcessing(t.Context()) }()
+
+	select {
+	case got := <-waited:
+		t.Fatalf("WaitForProcessing() = %d while an image was being processed",
+			got)
+	case <-time.After(100 * time.Millisecond):
+	}
+
+	openGate()
+
+	err := <-results
+	if err != nil {
+		t.Errorf("Process() error = %v, want nil", err)
+	}
+
+	select {
+	case got := <-waited:
+		if got != 0 {
+			t.Errorf("WaitForProcessing() once processing finished = %d, want 0",
+				got)
+		}
+	case <-time.After(5 * time.Second):
+		t.Fatal("WaitForProcessing() did not return once processing finished")
+	}
+
+	if held := len(proc.processingSemaphore); held != 0 {
+		t.Errorf("%d slots still held after WaitForProcessing() returned", held)
+	}
+}

internal/imgcache/cache.go

@@ -11,7 +11,6 @@ import (
 	"io"
 	"log/slog"
 	"path/filepath"
-	"sync"
 	"time"
 
 	lru "github.com/hashicorp/golang-lru/v2"
@@ -38,11 +37,16 @@ type CacheConfig struct {
 	NegativeTTL time.Duration
 
 	// MaxBytes is the disk cache size limit in bytes that eviction
-	// enforces. Zero means no limit is enforced (no eviction). The
+	// enforces. Zero means no limit is enforced (no eviction).
-	// config layer supplies the computed default when the operator
-	// omits cache_max_bytes.
 	MaxBytes int64
 
+	// UseDefaultMaxBytes makes NewCache replace MaxBytes with the
+	// default limit: 75% of the sum of the space free on the filesystem
+	// holding the cache and the bytes the cache already holds, at least
+	// DefaultCacheMaxBytesFloor. The config layer sets this when the
+	// operator omits cache_max_bytes.
+	UseDefaultMaxBytes bool
+
 	// DisableDiskCache turns the disk cache off entirely: no cache
 	// directories are created, lookups always miss, stores are
 	// no-ops, and no eviction machinery runs. The config layer sets
@@ -69,11 +73,11 @@ type Cache struct {
 
 	// Eviction machinery. The channels are created in NewCache so
 	// stores can signal write pressure without racing StartEviction.
+	// evictionCancel, set by StartEviction, cancels the eviction
+	// goroutine's context.
 	evictionPressure chan struct{}
-	evictionStop     chan struct{}
 	evictionDone     chan struct{}
-	evictionStarted  bool
+	evictionCancel   context.CancelFunc
-	evictionStopOnce sync.Once
 
 	// metaCache holds the content types of the variants most recently
 	// stored or served, so a hit does not read the variant's .meta file.
@@ -96,6 +100,14 @@ type Cache struct {
 
 // NewCache creates a new cache instance.
 func NewCache(db *sql.DB, config CacheConfig) (*Cache, error) {
+	return newCache(db, config, defaultFreeSpaceProbe)
+}
+
+// newCache is NewCache with the free-space probe passed in, so tests
+// can fake the free space the default limit is worked out from.
+func newCache(
+	db *sql.DB, config CacheConfig, probe FreeSpaceProbeFunc,
+) (*Cache, error) {
 	log := config.Logger
 	if log == nil {
 		log = slog.Default()
@@ -112,7 +124,6 @@ func NewCache(db *sql.DB, config CacheConfig) (*Cache, error) {
 		log:              log,
 		disabled:         config.DisableDiskCache,
 		evictionPressure: make(chan struct{}, 1),
-		evictionStop:     make(chan struct{}),
 		evictionDone:     make(chan struct{}),
 		metaCache:        metaCache,
 		contentLocks:     newContentLock(),
@@ -147,6 +158,20 @@ func NewCache(db *sql.DB, config CacheConfig) (*Cache, error) {
 	c.variants = variants
 	c.srcMetadata = srcMetadata
 
+	if config.UseDefaultMaxBytes {
+		limit, err := c.computeDefaultMaxBytes(context.Background(), probe)
+		if err != nil {
+			return nil, err
+		}
+
+		c.config.MaxBytes = limit
+
+		log.Info("computed default cache size limit from free space and cache contents",
+			"cache_max_bytes", limit,
+			"cache_dir", filepath.Join(config.StateDir, "cache"),
+		)
+	}
+
 	return c, nil
 }
 
@@ -471,7 +496,8 @@ func (c *Cache) Stats(ctx context.Context) (*CacheStats, error) {
 	return &stats, nil
 }
 
-// IncrementStats increments cache statistics.
+// IncrementStats counts a cache hit or miss, and an upstream fetch that read
+// fetchBytes bytes, as IncrementUpstreamFetch does.
 func (c *Cache) IncrementStats(ctx context.Context, hit bool, fetchBytes int64) {
 	var err error
 
@@ -495,18 +521,26 @@ func (c *Cache) IncrementStats(ctx context.Context, hit bool, fetchBytes int64)
 		c.log.Warn("failed to count cache hit or miss", "hit", hit, "error", err)
 	}
 
-	if fetchBytes > 0 {
+	c.IncrementUpstreamFetch(ctx, fetchBytes)
-		_, err = c.db.ExecContext(ctx, `
+}
-			UPDATE cache_stats
+
-			SET upstream_fetch_count = upstream_fetch_count + 1,
+// IncrementUpstreamFetch counts one upstream fetch that read fetchBytes bytes.
-			    upstream_fetch_bytes = upstream_fetch_bytes + ?,
+// A fetch that read no bytes is not counted.
-			    last_updated_at = CURRENT_TIMESTAMP
+func (c *Cache) IncrementUpstreamFetch(ctx context.Context, fetchBytes int64) {
-			WHERE id = 1
+	if fetchBytes <= 0 {
-		`, fetchBytes)
+		return
-		if err != nil {
+	}
-			c.log.Warn("failed to count upstream fetch",
+
-				"fetch_bytes", fetchBytes, "error", err)
+	_, err := c.db.ExecContext(ctx, `
-		}
+		UPDATE cache_stats
+		SET upstream_fetch_count = upstream_fetch_count + 1,
+		    upstream_fetch_bytes = upstream_fetch_bytes + ?,
+		    last_updated_at = CURRENT_TIMESTAMP
+		WHERE id = 1
+	`, fetchBytes)
+	if err != nil {
+		c.log.Warn("failed to count upstream fetch",
+			"fetch_bytes", fetchBytes, "error", err)
 	}
 }
 

internal/imgcache/cachesize.go

@@ -0,0 +1,89 @@
+package imgcache
+
+import (
+	"context"
+	"errors"
+	"fmt"
+	"math"
+	"path/filepath"
+	"syscall"
+)
+
+// DefaultCacheMaxBytesFloor is the minimum computed default for the
+// cache_max_bytes setting: 500 MiB. The floor applies only to the
+// computed default (when the key is omitted from the configuration),
+// never to explicitly configured values.
+const DefaultCacheMaxBytesFloor int64 = 524288000
+
+// freeSpaceFractionNumerator and freeSpaceFractionDenominator express
+// the 75% share used for the computed default limit as integer
+// arithmetic (dividing before multiplying avoids overflow).
+const (
+	freeSpaceFractionNumerator   uint64 = 3
+	freeSpaceFractionDenominator uint64 = 4
+)
+
+var errNegativeBlockSize = errors.New("statfs reported negative block size")
+
+// FreeSpaceProbeFunc reports the number of free bytes available on the
+// filesystem containing path. It is a function type so tests can
+// inject a fake probe instead of depending on the host disk.
+type FreeSpaceProbeFunc func(path string) (uint64, error)
+
+// defaultFreeSpaceProbe reports free filesystem bytes via statfs on
+// the given path, as available to unprivileged processes.
+func defaultFreeSpaceProbe(path string) (uint64, error) {
+	var stat syscall.Statfs_t
+
+	err := syscall.Statfs(path, &stat)
+	if err != nil {
+		return 0, err
+	}
+
+	if stat.Bsize < 0 {
+		return 0, fmt.Errorf("%w %d for %q", errNegativeBlockSize, stat.Bsize, path)
+	}
+
+	blockSize := uint64(stat.Bsize)
+
+	return stat.Bavail * blockSize, nil
+}
+
+// computeDefaultMaxBytes returns the default cache size limit: 75% of
+// the sum of the free bytes probe reports for <state_dir>/cache/ and
+// the bytes the cache already holds, with a floor of
+// DefaultCacheMaxBytesFloor. Counting what the cache holds keeps the
+// limit from shrinking as the cache fills.
+func (c *Cache) computeDefaultMaxBytes(
+	ctx context.Context, probe FreeSpaceProbeFunc,
+) (int64, error) {
+	cacheDir := filepath.Join(c.config.StateDir, "cache")
+
+	freeBytes, err := probe(cacheDir)
+	if err != nil {
+		return 0, fmt.Errorf(
+			"default cache_max_bytes: cannot determine free space for %q: %w",
+			cacheDir, err)
+	}
+
+	usedBytes, err := c.UsageBytes(ctx)
+	if err != nil {
+		return 0, err
+	}
+
+	// Both terms are at most math.MaxInt64, so the sum cannot overflow.
+	//nolint:gosec // G115: UsageBytes sums file sizes, never negative
+	spaceBytes := min(freeBytes, math.MaxInt64) + uint64(usedBytes)
+
+	computed := spaceBytes / freeSpaceFractionDenominator * freeSpaceFractionNumerator
+	computed = min(computed, math.MaxInt64)
+
+	// gosec cannot see that min() above bounds computed, so it reads
+	// this conversion as potentially overflowing. It cannot: computed is
+	// at most math.MaxInt64 on every path here.
+	//nolint:gosec // G115: clamped to MaxInt64 by min above
+	limit := int64(computed)
+	limit = max(limit, DefaultCacheMaxBytesFloor)
+
+	return limit, nil
+}

internal/imgcache/cachesize_internal_test.go

@@ -0,0 +1,188 @@
+package imgcache
+
+import (
+	"errors"
+	"os"
+	"path/filepath"
+	"strings"
+	"testing"
+)
+
+// Static errors returned by the stub free-space probes below.
+var (
+	errTestStatfsFailed     = errors.New("statfs failed")
+	errTestProbeNotExpected = errors.New("probe must not be called")
+)
+
+// TestComputeDefaultMaxBytesCountsWhatTheCacheHolds verifies that the
+// default limit is 75% of the free space plus what the cache already
+// holds, so a cache filled to its limit keeps that limit across a
+// restart instead of shrinking to 75% of the space left free.
+func TestComputeDefaultMaxBytesCountsWhatTheCacheHolds(t *testing.T) {
+	t.Parallel()
+
+	cache, _ := newEvictionTestCache(t, 1<<30)
+
+	// Empty cache, 4 GiB free -> 3 GiB default.
+	got, err := cache.computeDefaultMaxBytes(t.Context(),
+		func(string) (uint64, error) { return 4294967296, nil })
+	if err != nil {
+		t.Fatalf("computeDefaultMaxBytes returned error: %v", err)
+	}
+
+	t.Logf("default for an empty cache with 4 GiB free: %d", got)
+
+	if got != 3221225472 {
+		t.Errorf("default for an empty cache = %d, want 3221225472 (75%% of 4 GiB)",
+			got)
+	}
+
+	// The cache now holds those 3 GiB, which leaves 1 GiB free.
+	_, err = cache.db.ExecContext(t.Context(),
+		`INSERT INTO variant_content (cache_key, size_bytes, content_type)
+		 VALUES (?, ?, ?)`,
+		string(testVariantKeyOne), 3221225472, testContentTypeWebP,
+	)
+	if err != nil {
+		t.Fatalf("failed to insert variant accounting row: %v", err)
+	}
+
+	got, err = cache.computeDefaultMaxBytes(t.Context(),
+		func(string) (uint64, error) { return 1073741824, nil })
+	if err != nil {
+		t.Fatalf("computeDefaultMaxBytes returned error: %v", err)
+	}
+
+	t.Logf("default for a cache holding 3 GiB with 1 GiB free: %d", got)
+
+	if got != 3221225472 {
+		t.Errorf("default for a cache holding 3 GiB with 1 GiB free = %d, "+
+			"want 3221225472 (75%% of 1 GiB + 3 GiB)", got)
+	}
+}
+
+// TestComputeDefaultMaxBytesAppliesFloor verifies that when 75% of the
+// free space plus what the cache holds is below 500 MiB, the default
+// is floored at DefaultCacheMaxBytesFloor.
+func TestComputeDefaultMaxBytesAppliesFloor(t *testing.T) {
+	t.Parallel()
+
+	cases := []struct {
+		name      string
+		freeBytes uint64
+	}{
+		{name: "100 MiB free", freeBytes: 104857600},
+		{name: "zero free", freeBytes: 0},
+		{name: "just below floor threshold", freeBytes: 699050665},
+	}
+
+	for _, tc := range cases {
+		t.Run(tc.name, func(t *testing.T) {
+			t.Parallel()
+
+			cache, _ := newEvictionTestCache(t, 1<<30)
+
+			got, err := cache.computeDefaultMaxBytes(t.Context(),
+				func(string) (uint64, error) { return tc.freeBytes, nil })
+			if err != nil {
+				t.Fatalf("computeDefaultMaxBytes returned error: %v", err)
+			}
+
+			if got != DefaultCacheMaxBytesFloor {
+				t.Errorf("computeDefaultMaxBytes = %d, want floor %d",
+					got, DefaultCacheMaxBytesFloor)
+			}
+		})
+	}
+}
+
+// TestComputeDefaultMaxBytesPropagatesProbeError verifies that a
+// failing free-space probe produces an error naming cache_max_bytes,
+// instead of a silently wrong default.
+func TestComputeDefaultMaxBytesPropagatesProbeError(t *testing.T) {
+	t.Parallel()
+
+	cache, _ := newEvictionTestCache(t, 1<<30)
+
+	_, err := cache.computeDefaultMaxBytes(t.Context(),
+		func(string) (uint64, error) { return 0, errTestStatfsFailed })
+	if err == nil {
+		t.Fatal("probe failure must produce an error, got nil")
+	}
+
+	t.Logf("got expected error: %v", err)
+
+	if !strings.Contains(err.Error(), "cache_max_bytes") {
+		t.Errorf("error %q does not name cache_max_bytes", err.Error())
+	}
+}
+
+// TestNewCacheComputesDefaultMaxBytesWhenAsked verifies that with
+// UseDefaultMaxBytes set, the cache's limit becomes the computed
+// default, and that the probe is pointed at <state_dir>/cache/, which
+// must be created first so statfs measures the right filesystem.
+func TestNewCacheComputesDefaultMaxBytesWhenAsked(t *testing.T) {
+	t.Parallel()
+
+	stateDir := t.TempDir()
+	wantCacheDir := filepath.Join(stateDir, "cache")
+
+	var probedPath string
+
+	// 4 GiB free -> 3 GiB default.
+	probe := func(path string) (uint64, error) {
+		probedPath = path
+
+		info, err := os.Stat(path)
+		if err != nil || !info.IsDir() {
+			t.Errorf("cache directory %q was not created before probing: info=%v err=%v",
+				path, info, err)
+		}
+
+		return 4294967296, nil
+	}
+
+	cache, err := newCache(evictionTestDB(t), CacheConfig{
+		StateDir:           stateDir,
+		UseDefaultMaxBytes: true,
+	}, probe)
+	if err != nil {
+		t.Fatalf("newCache returned error: %v", err)
+	}
+
+	if cache.config.MaxBytes != 3221225472 {
+		t.Errorf("MaxBytes = %d, want computed default 3221225472",
+			cache.config.MaxBytes)
+	}
+
+	if probedPath != wantCacheDir {
+		t.Errorf("free space probed at %q, want cache directory %q",
+			probedPath, wantCacheDir)
+	}
+}
+
+// TestNewCacheKeepsExplicitMaxBytes verifies that without
+// UseDefaultMaxBytes the cache keeps MaxBytes exactly as given and
+// never consults the free-space probe.
+func TestNewCacheKeepsExplicitMaxBytes(t *testing.T) {
+	t.Parallel()
+
+	probe := func(string) (uint64, error) {
+		t.Error("free-space probe must not be consulted for explicit values")
+
+		return 0, errTestProbeNotExpected
+	}
+
+	cache, err := newCache(evictionTestDB(t), CacheConfig{
+		StateDir: t.TempDir(),
+		MaxBytes: 1024,
+	}, probe)
+	if err != nil {
+		t.Fatalf("newCache returned error: %v", err)
+	}
+
+	if cache.config.MaxBytes != 1024 {
+		t.Errorf("MaxBytes = %d, want explicit 1024 (no floor, no recompute)",
+			cache.config.MaxBytes)
+	}
+}

internal/imgcache/concurrent_misses_internal_test.go

@@ -0,0 +1,509 @@
+package imgcache
+
+import (
+	"bytes"
+	"context"
+	"errors"
+	"image/jpeg"
+	"io"
+	"io/fs"
+	"strings"
+	"sync"
+	"sync/atomic"
+	"testing"
+	"time"
+
+	"github.com/getsentry/sentry-go"
+	"sneak.berlin/go/pixa/internal/httpfetcher"
+	"sneak.berlin/go/pixa/internal/magic"
+)
+
+// arrivalWait is how long a test gives requests it has started to reach the
+// point where they wait for a held fetch.
+const arrivalWait = 100 * time.Millisecond
+
+// heldFetcher counts the fetches it is asked for and holds each one until
+// releaseFetches is called, so that a test can have requests arrive while a
+// fetch is in progress. started receives once for every fetch.
+type heldFetcher struct {
+	upstream    httpfetcher.Fetcher
+	fetches     atomic.Int32
+	started     chan struct{}
+	release     chan struct{}
+	releaseOnce sync.Once
+}
+
+func (f *heldFetcher) Fetch(
+	ctx context.Context, url string,
+) (*httpfetcher.FetchResult, error) {
+	f.fetches.Add(1)
+
+	f.started <- struct{}{}
+
+	select {
+	case <-f.release:
+	case <-ctx.Done():
+		return nil, ctx.Err()
+	}
+
+	return f.upstream.Fetch(ctx, url)
+}
+
+// releaseFetches lets every held fetch, and every later one, go on.
+func (f *heldFetcher) releaseFetches() {
+	f.releaseOnce.Do(func() { close(f.release) })
+}
+
+// setupHeldFetchService returns a test service whose fetches go through a
+// heldFetcher. Its database is limited to one connection: each connection to
+// an in-memory SQLite database opens a new, empty one, so requests running at
+// once must share the connection that holds the schema.
+func setupHeldFetchService(t *testing.T) (*Service, *TestFixtures, *heldFetcher) {
+	t.Helper()
+
+	svc, fixtures := SetupTestService(t)
+	svc.cache.db.SetMaxOpenConns(1)
+
+	fetcher := &heldFetcher{
+		upstream: svc.fetcher,
+		started:  make(chan struct{}, 100),
+		release:  make(chan struct{}),
+	}
+	svc.fetcher = fetcher
+
+	t.Cleanup(fetcher.releaseFetches)
+
+	return svc, fixtures, fetcher
+}
+
+// photoVariant asks for the test photo, 100x100, at 50x25 as a JPEG of the
+// given quality and fit mode. Each call returns a new request, as Get writes
+// to the request it is given.
+func photoVariant(fixtures *TestFixtures, quality int, fit FitMode) *ImageRequest {
+	return &ImageRequest{
+		SourceHost: fixtures.GoodHost,
+		SourcePath: testPathPhoto,
+		Size:       Size{Width: 50, Height: 25},
+		Format:     FormatJPEG,
+		Quality:    quality,
+		FitMode:    fit,
+	}
+}
+
+// getResult is what one Get call returned, with the image read out.
+type getResult struct {
+	image []byte
+	err   error
+}
+
+// startGet calls Get in a goroutine of its own and delivers what it returned
+// on the channel.
+func startGet(
+	ctx context.Context, svc *Service, req *ImageRequest,
+) <-chan getResult {
+	results := make(chan getResult, 1)
+
+	go func() {
+		resp, err := svc.Get(ctx, req)
+		if err != nil {
+			results <- getResult{err: err}
+
+			return
+		}
+
+		defer func() { _ = resp.Content.Close() }()
+
+		image, err := io.ReadAll(resp.Content)
+		results <- getResult{image: image, err: err}
+	}()
+
+	return results
+}
+
+// jpegSize returns the width and height of the JPEG image in data, or 0 and 0
+// if data is not one.
+func jpegSize(data []byte) (int, int) {
+	config, err := jpeg.DecodeConfig(bytes.NewReader(data))
+	if err != nil {
+		return 0, 0
+	}
+
+	return config.Width, config.Height
+}
+
+// TestService_Get_ConcurrentMissesShareOneFetch starts several requests for
+// one uncached variant while the first one's fetch is held. Between them they
+// must fetch the source once and transcode it once, every one must be answered
+// with the same 50x25 JPEG, and each must count one miss.
+func TestService_Get_ConcurrentMissesShareOneFetch(t *testing.T) {
+	t.Parallel()
+
+	svc, fixtures, fetcher := setupHeldFetchService(t)
+
+	const requests = 8
+
+	pending := make([]<-chan getResult, 0, requests)
+	for range requests {
+		pending = append(pending,
+			startGet(t.Context(), svc, photoVariant(fixtures, 85, FitCover)))
+	}
+
+	<-fetcher.started
+	time.Sleep(arrivalWait)
+	fetcher.releaseFetches()
+
+	var first []byte
+
+	for i, results := range pending {
+		got := <-results
+		if got.err != nil {
+			t.Fatalf("request %d: Get() error = %v", i, got.err)
+		}
+
+		if width, height := jpegSize(got.image); width != 50 || height != 25 {
+			t.Errorf("request %d: image is %dx%d, want a 50x25 JPEG", i, width, height)
+		}
+
+		if first == nil {
+			first = got.image
+		} else if !bytes.Equal(got.image, first) {
+			t.Errorf("request %d: image differs from request 0's", i)
+		}
+	}
+
+	if fetches := fetcher.fetches.Load(); fetches != 1 {
+		t.Errorf("%d requests made %d upstream fetches, want 1", requests, fetches)
+	}
+
+	// NewTestFS builds the same files the test service's fetcher serves.
+	testFS, _ := NewTestFS(t)
+
+	photo, err := fs.ReadFile(testFS, fixtures.GoodHostJPEG)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	want := cacheStatsCounters{0, requests, 1, int64(len(photo)), 1}
+
+	if got := readCacheStatsCounters(t, svc.cache); got != want {
+		t.Errorf("counters = %+v, want %+v", got, want)
+	}
+}
+
+// TestService_Get_ConcurrentVariantsStayApart requests three variants of the
+// test photo at once that differ only in quality or fit. Each must be made by
+// a fetch and a transcode of its own, and each answer must be its own variant.
+func TestService_Get_ConcurrentVariantsStayApart(t *testing.T) {
+	t.Parallel()
+
+	svc, fixtures, fetcher := setupHeldFetchService(t)
+
+	cover := startGet(t.Context(), svc, photoVariant(fixtures, 85, FitCover))
+	lowQuality := startGet(t.Context(), svc, photoVariant(fixtures, 40, FitCover))
+	contain := startGet(t.Context(), svc, photoVariant(fixtures, 85, FitContain))
+
+	for range 3 {
+		select {
+		case <-fetcher.started:
+		case <-time.After(5 * time.Second):
+			t.Fatal("fewer fetches started than variants requested: " +
+				"variants differing in quality or fit were merged")
+		}
+	}
+
+	fetcher.releaseFetches()
+
+	images := make(map[string][]byte)
+
+	for _, variant := range []struct {
+		name          string
+		results       <-chan getResult
+		width, height int
+	}{
+		{"q=85 fit=cover", cover, 50, 25},
+		{"q=40 fit=cover", lowQuality, 50, 25},
+		{"q=85 fit=contain", contain, 25, 25},
+	} {
+		got := <-variant.results
+		if got.err != nil {
+			t.Fatalf("%s: Get() error = %v", variant.name, got.err)
+		}
+
+		width, height := jpegSize(got.image)
+		if width != variant.width || height != variant.height {
+			t.Errorf("%s: image is %dx%d, want a %dx%d JPEG", variant.name,
+				width, height, variant.width, variant.height)
+		}
+
+		images[variant.name] = got.image
+	}
+
+	if bytes.Equal(images["q=85 fit=cover"], images["q=40 fit=cover"]) {
+		t.Error("q=40 was answered with the q=85 image")
+	}
+}
+
+// TestService_Get_WaiterStopsWhenItsContextEnds has a second request for a
+// variant join the first one's held fetch, then ends the second request's
+// context. The second request must return at once with the context's error,
+// while the fetch is still held, and the first must still be answered.
+func TestService_Get_WaiterStopsWhenItsContextEnds(t *testing.T) {
+	t.Parallel()
+
+	svc, fixtures, fetcher := setupHeldFetchService(t)
+
+	first := startGet(t.Context(), svc, photoVariant(fixtures, 85, FitCover))
+
+	<-fetcher.started
+
+	waiterCtx, cancelWaiter := context.WithCancel(t.Context())
+	waiter := startGet(waiterCtx, svc, photoVariant(fixtures, 85, FitCover))
+
+	time.Sleep(arrivalWait)
+	cancelWaiter()
+
+	select {
+	case got := <-waiter:
+		if !errors.Is(got.err, context.Canceled) {
+			t.Errorf("waiting request: Get() error = %v, want %v",
+				got.err, context.Canceled)
+		}
+	case <-time.After(time.Second):
+		t.Fatal("waiting request did not return when its context ended")
+	}
+
+	fetcher.releaseFetches()
+
+	got := <-first
+	if got.err != nil {
+		t.Fatalf("first request: Get() error = %v", got.err)
+	}
+
+	if width, height := jpegSize(got.image); width != 50 || height != 25 {
+		t.Errorf("first request: image is %dx%d, want a 50x25 JPEG", width, height)
+	}
+
+	if fetches := fetcher.fetches.Load(); fetches != 1 {
+		t.Errorf("upstream fetches = %d, want 1", fetches)
+	}
+}
+
+// TestService_Get_FirstRequestLeavingKeepsTheWork ends the context of the
+// request whose fetch is held, after a second request has joined it. The fetch
+// and transcode must go on and answer the second request. The first request
+// waits for its own work, as every request did before misses were shared, and
+// is answered too.
+func TestService_Get_FirstRequestLeavingKeepsTheWork(t *testing.T) {
+	t.Parallel()
+
+	svc, fixtures, fetcher := setupHeldFetchService(t)
+
+	firstCtx, cancelFirst := context.WithCancel(t.Context())
+	first := startGet(firstCtx, svc, photoVariant(fixtures, 85, FitCover))
+
+	<-fetcher.started
+
+	second := startGet(t.Context(), svc, photoVariant(fixtures, 85, FitCover))
+
+	time.Sleep(arrivalWait)
+	cancelFirst()
+	fetcher.releaseFetches()
+
+	for name, results := range map[string]<-chan getResult{
+		"first request": first, "second request": second,
+	} {
+		got := <-results
+		if got.err != nil {
+			t.Fatalf("%s: Get() error = %v", name, got.err)
+		}
+
+		if width, height := jpegSize(got.image); width != 50 || height != 25 {
+			t.Errorf("%s: image is %dx%d, want a 50x25 JPEG", name, width, height)
+		}
+	}
+
+	if fetches := fetcher.fetches.Load(); fetches != 1 {
+		t.Errorf("upstream fetches = %d, want 1", fetches)
+	}
+}
+
+// TestService_Get_ConcurrentMissesShareAFailure has several requests for an
+// image that cannot be served arrive while its fetch is held: the one fetch
+// answers all of them with its error. The request after them is answered from
+// the negative cache when the failure is kept there, and fetches again when it
+// is not.
+func TestService_Get_ConcurrentMissesShareAFailure(t *testing.T) {
+	t.Parallel()
+
+	tests := []struct {
+		name        string
+		path        string
+		wantErr     error // what every request at once gets
+		wantNextErr error // what the request after them gets
+		wantFetches int32 // fetches once the request after them is answered
+	}{
+		{"upstream answers 404, kept in the negative cache",
+			"/images/missing.jpg", httpfetcher.ErrUpstreamError,
+			ErrNegativeCached, 1},
+		{"source fails the magic byte check, not kept",
+			"/images/text.png", magic.ErrUnknownFormat, magic.ErrUnknownFormat, 2},
+	}
+
+	for _, tc := range tests {
+		t.Run(tc.name, func(t *testing.T) {
+			t.Parallel()
+
+			svc, fixtures, fetcher := setupHeldFetchService(t)
+
+			request := func() *ImageRequest {
+				req := photoVariant(fixtures, 85, FitCover)
+				req.SourcePath = tc.path
+
+				return req
+			}
+
+			pending := make([]<-chan getResult, 0, 4)
+			for range 4 {
+				pending = append(pending, startGet(t.Context(), svc, request()))
+			}
+
+			<-fetcher.started
+			time.Sleep(arrivalWait)
+			fetcher.releaseFetches()
+
+			for i, results := range pending {
+				if got := <-results; !errors.Is(got.err, tc.wantErr) {
+					t.Errorf("request %d: Get() error = %v, want %v", i, got.err, tc.wantErr)
+				}
+			}
+
+			_, err := svc.Get(t.Context(), request())
+			if !errors.Is(err, tc.wantNextErr) {
+				t.Errorf("next request: Get() error = %v, want %v", err, tc.wantNextErr)
+			}
+
+			if fetches := fetcher.fetches.Load(); fetches != tc.wantFetches {
+				t.Errorf("upstream fetches = %d, want %d", fetches, tc.wantFetches)
+			}
+		})
+	}
+}
+
+// TestService_Get_EndedRequestFetchesNothing checks that a request whose
+// context has already ended when it misses the cache starts no fetch.
+func TestService_Get_EndedRequestFetchesNothing(t *testing.T) {
+	t.Parallel()
+
+	svc, fixtures, fetcher := setupHeldFetchService(t)
+
+	ctx, cancel := context.WithCancel(t.Context())
+	cancel()
+
+	_, err := svc.Get(ctx, photoVariant(fixtures, 85, FitCover))
+	if !errors.Is(err, context.Canceled) {
+		t.Errorf("Get() error = %v, want %v", err, context.Canceled)
+	}
+
+	if fetches := fetcher.fetches.Load(); fetches != 0 {
+		t.Errorf("upstream fetches = %d, want 0", fetches)
+	}
+}
+
+// TestService_Get_ReturnsByItsDeadline gives a request a deadline and holds
+// its fetch until the fetch's context ends, as the fetcher does while it waits
+// for a free connection to the host. The request must return by its deadline
+// with the deadline's error.
+func TestService_Get_ReturnsByItsDeadline(t *testing.T) {
+	t.Parallel()
+
+	svc, fixtures, _ := setupHeldFetchService(t)
+
+	const timeout = 200 * time.Millisecond
+
+	ctx, cancel := context.WithTimeout(t.Context(), timeout)
+	defer cancel()
+
+	results := startGet(ctx, svc, photoVariant(fixtures, 85, FitCover))
+
+	select {
+	case got := <-results:
+		if !errors.Is(got.err, context.DeadlineExceeded) {
+			t.Errorf("Get() error = %v, want %v", got.err, context.DeadlineExceeded)
+		}
+	case <-time.After(timeout + time.Second):
+		t.Fatal("request did not return by its deadline")
+	}
+}
+
+// panickingFetcher panics on every fetch.
+type panickingFetcher struct{}
+
+func (panickingFetcher) Fetch(
+	context.Context, string,
+) (*httpfetcher.FetchResult, error) {
+	panic("upstream fetcher panicked")
+}
+
+// TestService_Get_PanicBecomesAnError checks that a panic while a variant is
+// being made reaches its request as an error naming the panic, instead of
+// being raised again.
+func TestService_Get_PanicBecomesAnError(t *testing.T) {
+	t.Parallel()
+
+	svc, fixtures := SetupTestService(t)
+	svc.fetcher = panickingFetcher{}
+
+	var err error
+
+	func() {
+		defer func() {
+			if recovered := recover(); recovered != nil {
+				t.Fatalf("Get() panicked: %v", recovered)
+			}
+		}()
+
+		_, err = svc.Get(t.Context(), photoVariant(fixtures, 85, FitCover))
+	}()
+
+	if err == nil || !strings.Contains(err.Error(), "upstream fetcher panicked") {
+		t.Errorf("Get() error = %v, want one naming the panic", err)
+	}
+}
+
+// TestService_Get_PanicIsReportedToSentry checks that a panic while a variant
+// is being made is reported through the Sentry hub on the request's context,
+// where the Sentry middleware puts one when sentry_dsn is set.
+func TestService_Get_PanicIsReportedToSentry(t *testing.T) {
+	t.Parallel()
+
+	svc, fixtures := SetupTestService(t)
+	svc.fetcher = panickingFetcher{}
+
+	transport := &sentry.MockTransport{}
+
+	client, err := sentry.NewClient(sentry.ClientOptions{
+		Dsn:       "https://abc123@sentry.example.com/42",
+		Transport: transport,
+	})
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	ctx := sentry.SetHubOnContext(t.Context(),
+		sentry.NewHub(client, sentry.NewScope()))
+
+	func() {
+		defer func() {
+			if recovered := recover(); recovered != nil {
+				t.Fatalf("Get() panicked: %v", recovered)
+			}
+		}()
+
+		_, _ = svc.Get(ctx, photoVariant(fixtures, 85, FitCover))
+	}()
+
+	events := transport.Events()
+	if len(events) != 1 || events[0].Message != "upstream fetcher panicked" {
+		t.Errorf("Sentry events = %+v, want one for the panic", events)
+	}
+}

internal/imgcache/eviction.go

@@ -117,7 +117,10 @@ func (c *Cache) EvictToLimit(ctx context.Context) error {
 
 // evictBatch fetches one batch of LRU candidates across variants and
 // source blobs and evicts them oldest-first until excessBytes are
-// freed or the batch is exhausted. It returns the bytes freed.
+// freed or the batch is exhausted. It returns the bytes freed. A
+// candidate that fails once ctx is cancelled (every one started after
+// that fails at its first database call) ends the batch with ctx's
+// error, without a warning.
 func (c *Cache) evictBatch(ctx context.Context, excessBytes int64) (int64, error) {
 	candidates, err := c.evictionCandidates(ctx)
 	if err != nil {
@@ -133,6 +136,10 @@ func (c *Cache) evictBatch(ctx context.Context, excessBytes int64) (int64, error
 
 		err := c.evictCandidate(ctx, candidate)
 		if err != nil {
+			if ctx.Err() != nil {
+				return freed, ctx.Err()
+			}
+
 			c.log.Warn("failed to evict cache entry",
 				"cache_key", candidate.cacheKey,
 				"content_hash", candidate.contentHash,
@@ -283,7 +290,7 @@ func (c *Cache) evictVariant(ctx context.Context, cacheKey VariantKey) error {
 
 	c.metaCache.Remove(cacheKey)
 
-	err = c.variants.DeleteWithMeta(cacheKey)
+	err = c.variants.Delete(cacheKey)
 	if err != nil {
 		return err
 	}
@@ -424,37 +431,44 @@ func (c *Cache) notifyWritePressure() {
 // startup and again on every periodic tick thereafter, and evicts to
 // the configured limit on the given periodic interval and on
 // write-pressure notifications. It is a no-op on a disabled cache or
-// when already started.
+// when already started. The goroutine outlives the caller, so it runs
+// with its own context, which StopEviction cancels.
 func (c *Cache) StartEviction(interval time.Duration) {
-	if c.disabled || c.evictionStarted {
+	if c.disabled || c.evictionCancel != nil {
 		return
 	}
 
-	c.evictionStarted = true
+	ctx, cancel := context.WithCancel(context.Background())
+	c.evictionCancel = cancel
 
-	go c.evictionLoop(interval)
+	go c.evictionLoop(ctx, interval)
 }
 
-// StopEviction stops the background eviction goroutine and waits for
+// StopEviction cancels the background eviction goroutine, which
-// it to exit. It is safe to call when eviction was never started, and
+// interrupts a pass in progress, and waits for it to exit or for ctx to
-// safe to call more than once.
+// end, whichever comes first. In the second case it returns an error
-func (c *Cache) StopEviction() {
+// wrapping ctx's error. It is safe to call when eviction was never
-	if !c.evictionStarted {
+// started, and safe to call more than once.
-		return
+func (c *Cache) StopEviction(ctx context.Context) error {
+	if c.evictionCancel == nil {
+		return nil
 	}
 
-	c.evictionStopOnce.Do(func() {
+	c.evictionCancel()
-		close(c.evictionStop)
+
-		<-c.evictionDone
+	select {
-	})
+	case <-c.evictionDone:
+		return nil
+	case <-ctx.Done():
+		return fmt.Errorf("cache eviction still running: %w", ctx.Err())
+	}
 }
 
-// evictionLoop is the body of the background eviction goroutine.
+// evictionLoop is the body of the background eviction goroutine. It
-func (c *Cache) evictionLoop(interval time.Duration) {
+// returns when ctx is cancelled, and starts no pass after that.
+func (c *Cache) evictionLoop(ctx context.Context, interval time.Duration) {
 	defer close(c.evictionDone)
 
-	ctx := context.Background()
-
 	c.runReconciliationPass(ctx)
 	c.runEvictionPass(ctx)
 
@@ -463,7 +477,7 @@ func (c *Cache) evictionLoop(interval time.Duration) {
 
 	for {
 		select {
-		case <-c.evictionStop:
+		case <-ctx.Done():
 			return
 		case <-ticker.C:
 			// Reconciliation walks the cache directories, so it only
@@ -484,8 +498,13 @@ func (c *Cache) evictionLoop(interval time.Duration) {
 }
 
 // runEvictionPass runs one eviction pass, logging failures instead of
-// propagating them (the loop must keep running).
+// propagating them (the loop must keep running). It does nothing once
+// ctx is cancelled.
 func (c *Cache) runEvictionPass(ctx context.Context) {
+	if ctx.Err() != nil {
+		return
+	}
+
 	err := c.EvictToLimit(ctx)
 	if err != nil {
 		c.log.Warn("cache eviction pass failed", "error", err)
@@ -493,8 +512,13 @@ func (c *Cache) runEvictionPass(ctx context.Context) {
 }
 
 // runReconciliationPass runs one reconciliation pass, logging failures
-// instead of propagating them (the loop must keep running).
+// instead of propagating them (the loop must keep running). It does
+// nothing once ctx is cancelled.
 func (c *Cache) runReconciliationPass(ctx context.Context) {
+	if ctx.Err() != nil {
+		return
+	}
+
 	err := c.reconcileAccounting(ctx)
 	if err != nil {
 		c.log.Warn("cache accounting reconciliation failed", "error", err)
@@ -511,7 +535,8 @@ func (c *Cache) runReconciliationPass(ctx context.Context) {
 // know (and rows whose files are gone), and sweeps stale temp files
 // left behind by crashed writes. Running it periodically, not just
 // once, bounds how long such drift can accumulate unaccounted for on a
-// long-running process to one eviction interval.
+// long-running process to one eviction interval. Once ctx is cancelled,
+// it stops at the next file or row and returns ctx's error.
 func (c *Cache) reconcileAccounting(ctx context.Context) error {
 	if c.disabled {
 		return nil
@@ -546,6 +571,10 @@ func (c *Cache) reconcileVariantFiles(ctx context.Context) error {
 	return filepath.WalkDir(
 		c.variants.baseDir,
 		func(path string, entry fs.DirEntry, err error) error {
+			if ctx.Err() != nil {
+				return ctx.Err()
+			}
+
 			if err != nil || entry.IsDir() {
 				return err
 			}
@@ -634,6 +663,10 @@ func (c *Cache) reconcileVariantRows(ctx context.Context) error {
 	}
 
 	for _, key := range keys {
+		if ctx.Err() != nil {
+			return ctx.Err()
+		}
+
 		if c.variants.Exists(key) {
 			continue
 		}
@@ -699,6 +732,10 @@ func (c *Cache) reconcileSourceFiles(ctx context.Context) error {
 	return filepath.WalkDir(
 		c.srcContent.baseDir,
 		func(path string, entry fs.DirEntry, err error) error {
+			if ctx.Err() != nil {
+				return ctx.Err()
+			}
+
 			if err != nil || entry.IsDir() {
 				return err
 			}
@@ -760,6 +797,10 @@ func (c *Cache) reconcileSourceRows(ctx context.Context) error {
 	}
 
 	for _, hash := range hashes {
+		if ctx.Err() != nil {
+			return ctx.Err()
+		}
+
 		if c.srcContent.Exists(hash) {
 			continue
 		}

internal/imgcache/eviction_internal_test.go

@@ -4,9 +4,12 @@ import (
 	"bytes"
 	"context"
 	"database/sql"
+	"errors"
 	"io/fs"
+	"log/slog"
 	"os"
 	"path/filepath"
+	"strings"
 	"testing"
 	"time"
 
@@ -657,7 +660,7 @@ func TestEvictionRunsUnderWritePressure(t *testing.T) {
 	// An interval far longer than the test ensures only write
 	// pressure can trigger eviction here.
 	cache.StartEviction(time.Hour)
-	defer cache.StopEviction()
+	defer func() { _ = cache.StopEviction(t.Context()) }()
 
 	keys := []VariantKey{
 		testVariantKeyOne, testVariantKeyTwo, testVariantKeyThree,
@@ -678,6 +681,11 @@ func TestEvictionRunsUnderWritePressure(t *testing.T) {
 	assertNoDanglingReferences(t, cache)
 }
 
+// TestEvictionRunsOnPeriodicSchedule writes three variant files straight
+// to disk, bypassing StoreVariant, so they have no accounting rows and no
+// write-pressure notification fires. Only a periodic reconciliation pass
+// can then adopt them, and only the eviction pass that follows it can
+// evict them.
 func TestEvictionRunsOnPeriodicSchedule(t *testing.T) {
 	t.Parallel()
 
@@ -685,12 +693,28 @@ func TestEvictionRunsOnPeriodicSchedule(t *testing.T) {
 
 	cache, _ := newEvictionTestCache(t, limit)
 
-	// Start the evictor while the cache is empty, then create tracked
+	// Hold the test database's only connection, so the startup pass
-	// over-limit state WITHOUT going through the store methods, so no
+	// waits for it after walking the still empty variant directory: the
-	// write-pressure notification fires and only the periodic ticker
+	// files written while it waits are first seen by a periodic pass.
-	// can trigger eviction.
+	conn, err := cache.db.Conn(t.Context())
+	if err != nil {
+		t.Fatalf("failed to take the database connection: %v", err)
+	}
+
+	defer func() { _ = conn.Close() }()
+
 	cache.StartEviction(100 * time.Millisecond)
-	defer cache.StopEviction()
+	defer func() { _ = cache.StopEviction(t.Context()) }()
+
+	deadline := time.Now().Add(5 * time.Second)
+
+	for cache.db.Stats().WaitCount == 0 {
+		if time.Now().After(deadline) {
+			t.Fatal("the startup pass never waited for the database")
+		}
+
+		time.Sleep(10 * time.Millisecond)
+	}
 
 	keys := []VariantKey{
 		testVariantKeyOne, testVariantKeyTwo, testVariantKeyThree,
@@ -700,25 +724,43 @@ func TestEvictionRunsOnPeriodicSchedule(t *testing.T) {
 	for i, key := range keys {
 		content := bytes.Repeat([]byte{fills[i]}, 1000)
 
-		_, err := cache.variants.Store(key, bytes.NewReader(content), "image/webp")
+		_, err = cache.variants.Store(key, bytes.NewReader(content), "image/webp")
 		if err != nil {
 			t.Fatalf("failed to store variant file: %v", err)
 		}
+	}
 
-		_, err = cache.db.ExecContext(t.Context(),
+	_ = conn.Close()
-			`INSERT INTO variant_content (cache_key, size_bytes, content_type)
+
-			 VALUES (?, ?, ?)`,
+	// Only one of the 1000-byte files fits under the limit: wait until
-			string(key), len(content), "image/webp",
+	// the evictor has removed the other two.
-		)
+	stored := len(keys)
-		if err != nil {
+	deadline = time.Now().Add(5 * time.Second)
-			t.Fatalf("failed to insert variant accounting row: %v", err)
+
+	for stored > 1 && time.Now().Before(deadline) {
+		time.Sleep(25 * time.Millisecond)
+
+		stored = 0
+
+		for _, key := range keys {
+			if cache.variants.Exists(key) {
+				stored++
+			}
 		}
 	}
 
-	usage := waitForUsageAtOrBelow(t, cache, limit, 5*time.Second)
+	if stored > 1 {
+		t.Fatalf("periodic schedule did not trigger eviction: %d of %d "+
+			"variant files still on disk, want at most 1", stored, len(keys))
+	}
+
+	usage, err := cache.UsageBytes(t.Context())
+	if err != nil {
+		t.Fatalf("UsageBytes failed: %v", err)
+	}
+
 	if usage > limit {
-		t.Errorf("periodic schedule did not trigger eviction: usage = %d, want <= %d",
+		t.Errorf("usage after eviction = %d, want <= %d", usage, limit)
-			usage, limit)
 	}
 
 	assertNoDanglingReferences(t, cache)
@@ -751,7 +793,7 @@ func TestStartEvictionReconcilesAccountingWithDisk(t *testing.T) {
 	}
 
 	cache.StartEviction(time.Hour)
-	defer cache.StopEviction()
+	defer func() { _ = cache.StopEviction(t.Context()) }()
 
 	deadline := time.Now().Add(5 * time.Second)
 
@@ -805,15 +847,28 @@ func TestPeriodicReconciliationAdoptsFileThatAppearsAfterStartup(t *testing.T) {
 
 	cache, _ := newEvictionTestCache(t, 1<<30)
 
-	const interval = 100 * time.Millisecond
+	// Hold the test database's only connection, so the startup pass
+	// waits for it after walking the still empty variant directory: the
+	// file written while it waits is first seen by a periodic pass.
+	conn, err := cache.db.Conn(t.Context())
+	if err != nil {
+		t.Fatalf("failed to take the database connection: %v", err)
+	}
 
-	cache.StartEviction(interval)
+	defer func() { _ = conn.Close() }()
-	defer cache.StopEviction()
 
-	// Let startup reconciliation run and settle on an empty cache
+	cache.StartEviction(100 * time.Millisecond)
-	// before introducing the untracked file, so the adoption we assert
+	defer func() { _ = cache.StopEviction(t.Context()) }()
-	// below can only be the work of a later, periodic pass.
+
-	time.Sleep(3 * interval)
+	deadline := time.Now().Add(5 * time.Second)
+
+	for cache.db.Stats().WaitCount == 0 {
+		if time.Now().After(deadline) {
+			t.Fatal("the startup pass never waited for the database")
+		}
+
+		time.Sleep(10 * time.Millisecond)
+	}
 
 	// Simulate a variant whose accounting insert failed after the
 	// process was already running and serving requests: the content
@@ -822,14 +877,16 @@ func TestPeriodicReconciliationAdoptsFileThatAppearsAfterStartup(t *testing.T) {
 	// insert had failed and only the file write had succeeded.
 	untracked := bytes.Repeat([]byte{0x41}, 900)
 
-	_, err := cache.variants.Store(
+	_, err = cache.variants.Store(
 		"aabbccdd0099", bytes.NewReader(untracked), "image/webp",
 	)
 	if err != nil {
 		t.Fatalf("failed to store untracked variant file: %v", err)
 	}
 
-	deadline := time.Now().Add(5 * time.Second)
+	_ = conn.Close()
+
+	deadline = time.Now().Add(5 * time.Second)
 
 	var usage int64
 
@@ -862,6 +919,242 @@ func TestPeriodicReconciliationAdoptsFileThatAppearsAfterStartup(t *testing.T) {
 	}
 }
 
+// TestStopEvictionInterruptsPassInProgress holds the test database's
+// only connection, so the startup reconciliation pass waits for it, and
+// checks that StopEviction stops that pass instead of waiting for the
+// connection to come free, and that the stop logs one warning: the
+// interrupted reconciliation's, with no eviction pass started after it.
+func TestStopEvictionInterruptsPassInProgress(t *testing.T) {
+	t.Parallel()
+
+	cache, _ := newEvictionTestCache(t, 1<<30)
+
+	var logBuf bytes.Buffer
+
+	cache.log = slog.New(slog.NewJSONHandler(&logBuf, nil))
+
+	conn, err := cache.db.Conn(t.Context())
+	if err != nil {
+		t.Fatalf("failed to take the database connection: %v", err)
+	}
+
+	defer func() { _ = conn.Close() }()
+
+	cache.StartEviction(time.Hour)
+
+	// The pass is in progress once it waits for the connection.
+	deadline := time.Now().Add(5 * time.Second)
+
+	for cache.db.Stats().WaitCount == 0 {
+		if time.Now().After(deadline) {
+			t.Fatal("the reconciliation pass never waited for the database")
+		}
+
+		time.Sleep(10 * time.Millisecond)
+	}
+
+	ctx, cancel := context.WithTimeout(t.Context(), 5*time.Second)
+	defer cancel()
+
+	err = cache.StopEviction(ctx)
+	t.Logf("StopEviction() error = %v", err)
+
+	if err != nil {
+		t.Fatalf("StopEviction() error = %v, want nil: the pass waiting for "+
+			"the database did not stop", err)
+	}
+
+	t.Logf("log output: %s", logBuf.String())
+
+	warnings := strings.Count(logBuf.String(), `"level":"WARN"`)
+	if warnings != 1 {
+		t.Errorf("the stop logged %d warnings, want 1", warnings)
+	}
+}
+
+// TestEvictToLimitStopsAtNextCandidateOnceCancelled cancels the context
+// while the oldest of three source blobs is being evicted, and checks that
+// EvictToLimit then returns context.Canceled without evicting the other
+// two or logging a warning for either of them.
+func TestEvictToLimitStopsAtNextCandidateOnceCancelled(t *testing.T) {
+	t.Parallel()
+
+	cache, _ := newEvictionTestCache(t, 1)
+
+	var logBuf bytes.Buffer
+
+	cache.log = slog.New(slog.NewJSONHandler(&logBuf, nil))
+
+	hashes := []ContentHash{
+		storeEvictionTestSource(t, cache, "cancel.example.com", "/a.jpg",
+			bytes.Repeat([]byte{0x61}, 1000)),
+		storeEvictionTestSource(t, cache, "cancel.example.com", "/b.jpg",
+			bytes.Repeat([]byte{0x62}, 1000)),
+		storeEvictionTestSource(t, cache, "cancel.example.com", "/c.jpg",
+			bytes.Repeat([]byte{0x63}, 1000)),
+	}
+
+	base := time.Now().Add(-time.Hour)
+
+	for i, hash := range hashes {
+		setSourceLastAccessed(t, cache, hash, base.Add(time.Duration(i)*time.Minute))
+	}
+
+	ctx, cancel := context.WithCancel(t.Context())
+	defer cancel()
+
+	cache.evictSourceBlobTestHook = func(ContentHash) { cancel() }
+
+	err := cache.EvictToLimit(ctx)
+	t.Logf("EvictToLimit() error = %v", err)
+	t.Logf("log output: %s", logBuf.String())
+
+	if !errors.Is(err, context.Canceled) {
+		t.Errorf("EvictToLimit() error = %v, want context.Canceled", err)
+	}
+
+	if cache.srcContent.Exists(hashes[0]) {
+		t.Errorf("source blob %s, evicted when the context was cancelled, "+
+			"is still on disk", hashes[0])
+	}
+
+	for _, hash := range hashes[1:] {
+		if !cache.srcContent.Exists(hash) {
+			t.Errorf("source blob %s was evicted after the context was cancelled", hash)
+		}
+	}
+
+	if strings.Contains(logBuf.String(), `"level":"WARN"`) {
+		t.Errorf("EvictToLimit logged a warning after the context was cancelled")
+	}
+
+	assertNoDanglingReferences(t, cache)
+}
+
+// TestStopEvictionReturnsWhenItsContextEnds pauses an eviction pass where
+// cancellation cannot reach it, after a source blob's rows are deleted and
+// before its file is removed, and checks that StopEviction returns its
+// context's error when that context ends instead of waiting for the pass.
+// Once the pass goes on, the goroutine exits and no row points at a
+// missing file.
+func TestStopEvictionReturnsWhenItsContextEnds(t *testing.T) {
+	t.Parallel()
+
+	cache, _ := newEvictionTestCache(t, 1)
+
+	paused := make(chan struct{})
+	resume := make(chan struct{})
+
+	cache.evictSourceBlobTestHook = func(ContentHash) {
+		close(paused)
+		<-resume
+	}
+
+	hash := storeEvictionTestSource(t, cache, "stop.example.com", "/a.jpg",
+		bytes.Repeat([]byte{0x61}, 1000))
+
+	cache.StartEviction(time.Hour)
+
+	select {
+	case <-paused:
+	case <-time.After(5 * time.Second):
+		t.Fatal("the eviction pass never reached the source blob")
+	}
+
+	ctx, cancel := context.WithTimeout(t.Context(), 50*time.Millisecond)
+	defer cancel()
+
+	err := cache.StopEviction(ctx)
+	t.Logf("StopEviction() error = %v", err)
+
+	if !errors.Is(err, context.DeadlineExceeded) {
+		t.Errorf("StopEviction() error = %v, want context.DeadlineExceeded", err)
+	}
+
+	close(resume)
+
+	err = cache.StopEviction(t.Context())
+	if err != nil {
+		t.Fatalf("second StopEviction() error = %v, want nil", err)
+	}
+
+	assertNoDanglingReferences(t, cache)
+
+	if cache.srcContent.Exists(hash) {
+		t.Errorf("source blob %s is still on disk after its rows were deleted", hash)
+	}
+}
+
+// TestReconciliationWalksStopOnceCancelled checks that both directory
+// walks of a reconciliation pass return the context's error once it is
+// cancelled, leaving in place a stale temp file they would otherwise
+// remove.
+func TestReconciliationWalksStopOnceCancelled(t *testing.T) {
+	t.Parallel()
+
+	cache, _ := newEvictionTestCache(t, 1<<30)
+
+	ctx, cancel := context.WithCancel(t.Context())
+	cancel()
+
+	staleTime := time.Now().Add(-2 * staleTempFileAge)
+
+	walks := map[string]func(context.Context) error{
+		cache.variants.baseDir:   cache.reconcileVariantFiles,
+		cache.srcContent.baseDir: cache.reconcileSourceFiles,
+	}
+
+	for dir, walk := range walks {
+		tempFile := filepath.Join(dir, tempFilePrefix+"stale")
+
+		err := os.WriteFile(tempFile, []byte("partial"), 0o600)
+		if err != nil {
+			t.Fatalf("failed to write temp file: %v", err)
+		}
+
+		err = os.Chtimes(tempFile, staleTime, staleTime)
+		if err != nil {
+			t.Fatalf("failed to backdate temp file: %v", err)
+		}
+
+		err = walk(ctx)
+		t.Logf("walk of %s: error = %v", dir, err)
+
+		if !errors.Is(err, context.Canceled) {
+			t.Errorf("walk of %s: error = %v, want context.Canceled", dir, err)
+		}
+
+		_, err = os.Stat(tempFile)
+		if err != nil {
+			t.Errorf("walk of %s went on after cancellation: %v", dir, err)
+		}
+	}
+}
+
+// TestReconciliationPassLogsNoWarningOnceCancelled checks that a
+// reconciliation pass run with an already cancelled context logs no
+// warning, so a periodic tick the loop takes after a stop adds no
+// warning to the one from the pass the stop interrupted.
+func TestReconciliationPassLogsNoWarningOnceCancelled(t *testing.T) {
+	t.Parallel()
+
+	cache, _ := newEvictionTestCache(t, 1<<30)
+
+	var logBuf bytes.Buffer
+
+	cache.log = slog.New(slog.NewJSONHandler(&logBuf, nil))
+
+	ctx, cancel := context.WithCancel(t.Context())
+	cancel()
+
+	cache.runReconciliationPass(ctx)
+	t.Logf("log output: %s", logBuf.String())
+
+	if strings.Contains(logBuf.String(), `"level":"WARN"`) {
+		t.Errorf("runReconciliationPass logged a warning with a cancelled context")
+	}
+}
+
 // TestEvictSourceBlobExcludesConcurrentStoreOfIdenticalContent exercises
 // the exact TOCTOU window between evictSourceBlob's row-deletion
 // transaction commit and its content file unlink: a concurrent

internal/imgcache/imgcache.go

@@ -5,7 +5,6 @@ import (
 	"context"
 	"errors"
 	"io"
-	"net/url"
 	"time"
 )
 
@@ -154,9 +153,6 @@ type ImageCache interface {
 	// Warm pre-fetches and caches an image without returning it
 	Warm(ctx context.Context, req *ImageRequest) error
 
-	// Purge removes a cached image
-	Purge(ctx context.Context, req *ImageRequest) error
-
 	// Stats returns cache statistics
 	Stats(ctx context.Context) (*CacheStats, error)
 }
@@ -176,29 +172,3 @@ type CacheStats struct {
 	// HitRate is HitCount / (HitCount + MissCount)
 	HitRate float64
 }
-
-// SignatureValidator validates request signatures
-type SignatureValidator interface {
-	// Validate checks if the signature is valid for the request
-	Validate(req *ImageRequest) error
-	// Generate creates a signature for a request
-	Generate(req *ImageRequest) string
-}
-
-// Allowlist checks if a URL is allowlisted (no signature required)
-type Allowlist interface {
-	// IsAllowlisted returns true if the URL doesn't require a signature
-	IsAllowlisted(u *url.URL) bool
-}
-
-// Storage handles persistent storage of cached content
-type Storage interface {
-	// Store saves content and returns its hash
-	Store(ctx context.Context, content io.Reader) (hash string, err error)
-	// Load retrieves content by hash
-	Load(ctx context.Context, hash string) (io.ReadCloser, error)
-	// Delete removes content by hash
-	Delete(ctx context.Context, hash string) error
-	// Exists checks if content exists
-	Exists(ctx context.Context, hash string) (bool, error)
-}

internal/imgcache/service.go

@@ -8,9 +8,13 @@ import (
 	"io"
 	"log/slog"
 	"net/url"
+	"runtime/debug"
 	"time"
 
 	"github.com/dustin/go-humanize"
+	"github.com/getsentry/sentry-go"
+	"github.com/go-chi/chi/v5/middleware"
+	"golang.org/x/sync/singleflight"
 	"sneak.berlin/go/pixa/internal/allowlist"
 	"sneak.berlin/go/pixa/internal/httpfetcher"
 	"sneak.berlin/go/pixa/internal/imageprocessor"
@@ -29,6 +33,9 @@ type Service struct {
 	log             *slog.Logger
 	allowHTTP       bool
 	maxResponseSize int64
+	// variantsInProgress lets the requests that miss the same variant at the
+	// same time share one fetch and one transcode.
+	variantsInProgress singleflight.Group
 }
 
 // ServiceConfig holds configuration for the image service.
@@ -50,11 +57,10 @@ type ServiceConfig struct {
 	Logger *slog.Logger
 }
 
-// Static errors for service construction and unimplemented operations.
+// Static errors for service construction.
 var (
-	errCacheRequired       = errors.New("cache is required")
+	errCacheRequired      = errors.New("cache is required")
-	errSigningKeyRequired  = errors.New("signing key is required")
+	errSigningKeyRequired = errors.New("signing key is required")
-	errPurgeNotImplemented = errors.New("purge not implemented")
 )
 
 // NewService creates a new image service.
@@ -160,14 +166,12 @@ func (s *Service) Get(ctx context.Context, req *ImageRequest) (*ImageResponse, e
 		}
 	}
 
-	// Cache miss - process the cached source or fetch it, then count the
+	// Cache miss - get the variant, processed once for all the requests that
-	// miss with the bytes it fetched from upstream, also when it failed or
+	// miss it at the same time, then count this request's miss, also when it
-	// the request context has ended meanwhile
+	// failed or the request context has ended meanwhile
-	cacheKey := CacheKey(req)
+	response, err := s.processOrWait(ctx, req)
 
-	response, fetchedBytes, err := s.processFromSourceOrFetch(ctx, req, cacheKey)
+	s.cache.IncrementStats(context.WithoutCancel(ctx), false, 0)
-
-	s.cache.IncrementStats(context.WithoutCancel(ctx), false, fetchedBytes)
 
 	if err != nil {
 		return nil, err
@@ -185,16 +189,17 @@ func (s *Service) Warm(ctx context.Context, req *ImageRequest) error {
 	return err
 }
 
-// Purge removes a cached image. Purging is not implemented yet.
-func (s *Service) Purge(_ context.Context, _ *ImageRequest) error {
-	return errPurgeNotImplemented
-}
-
 // Stats returns cache statistics.
 func (s *Service) Stats(ctx context.Context) (*CacheStats, error) {
 	return s.cache.Stats(ctx)
 }
 
+// WaitForProcessing waits until no image is being processed, or until ctx
+// ends, and returns how many images were still being processed then.
+func (s *Service) WaitForProcessing(ctx context.Context) int {
+	return s.processor.WaitForProcessing(ctx)
+}
+
 // ValidateRequest validates the request signature if required.
 func (s *Service) ValidateRequest(req *ImageRequest) error {
 	// Check if host is allowed (no signature required)
@@ -241,6 +246,96 @@ func (s *Service) GenerateSignedURL(
 		baseURL, path, sig, exp, req.Quality, req.FitMode), nil
 }
 
+// errPanicked is returned when processing a variant panicked.
+var errPanicked = errors.New("panic while processing image")
+
+// processOrWait returns the variant req asks for. The first of the requests
+// that miss a variant at the same time processes it, and singleflight hands
+// its result, or its error, to the others: they fetch nothing, read no source
+// and take no upstream connection or processing slot. The processing ignores
+// the first request's cancellation, so the others are still served if that
+// client goes away, but keeps its deadline.
+func (s *Service) processOrWait(
+	ctx context.Context, req *ImageRequest,
+) (*ImageResponse, error) {
+	// A request that has already ended starts no processing
+	if ctx.Err() != nil {
+		return nil, ctx.Err()
+	}
+
+	cacheKey := CacheKey(req)
+
+	// Closed when this request's own function runs, which singleflight does
+	// only when no other request is processing the variant
+	processing := make(chan struct{})
+
+	results := s.variantsInProgress.DoChan(string(cacheKey),
+		func() (_ any, err error) {
+			close(processing)
+
+			// singleflight would raise a panic again in a goroutine of its
+			// own, where no handler recovers it, and stop pixad. It is
+			// reported through the Sentry hub that the Sentry middleware puts
+			// on the request's context when sentry_dsn is set.
+			defer func() {
+				recovered := recover()
+				if recovered != nil {
+					s.log.Error("panic while processing image",
+						"host", req.SourceHost, "path", req.SourcePath,
+						"panic", recovered, "stack", string(debug.Stack()))
+
+					if hub := sentry.GetHubFromContext(ctx); hub != nil {
+						hub.RecoverWithContext(ctx, recovered)
+					}
+
+					err = fmt.Errorf("%w: %v", errPanicked, recovered)
+				}
+			}()
+
+			processingCtx := context.WithoutCancel(ctx)
+
+			if deadline, ok := ctx.Deadline(); ok {
+				var cancel context.CancelFunc
+
+				processingCtx, cancel = context.WithDeadline(processingCtx, deadline)
+				defer cancel()
+			}
+
+			return s.processFromSourceOrFetch(processingCtx, req, cacheKey)
+		})
+
+	var result singleflight.Result
+
+	select {
+	case result = <-results:
+	case <-ctx.Done():
+		select {
+		case <-processing:
+			// This request is processing the variant: it waits for the
+			// result, as every request did before misses were shared
+			result = <-results
+		default:
+			// Another request is processing the variant, or this request's
+			// function has not started yet; the processing goes on without it
+			return nil, ctx.Err()
+		}
+	}
+
+	if result.Err != nil {
+		return nil, result.Err
+	}
+
+	variant, _ := result.Val.(*processedVariant)
+
+	return &ImageResponse{
+		Content:       io.NopCloser(bytes.NewReader(variant.data)),
+		ContentLength: int64(len(variant.data)),
+		ContentType:   variant.contentType,
+		FetchedBytes:  variant.fetchedBytes,
+		ETag:          formatETag(cacheKey),
+	}, nil
+}
+
 // loadCachedSource opens source content from cache, without reading it, and
 // returns it with its size; nil if the cached data is unavailable, empty or
 // exceeds maxResponseSize.
@@ -275,13 +370,12 @@ func (s *Service) loadCachedSource(
 }
 
 // processFromSourceOrFetch processes an image, using cached source content
-// if available. It also returns the number of bytes fetched from upstream,
+// if available.
-// as fetchAndProcess does, or 0 when the cached source was used.
 func (s *Service) processFromSourceOrFetch(
 	ctx context.Context,
 	req *ImageRequest,
 	cacheKey VariantKey,
-) (*ImageResponse, int64, error) {
+) (*processedVariant, error) {
 	// Check if we have cached source content
 	contentHash, _, err := s.cache.LookupSource(ctx, req)
 	if err != nil {
@@ -308,19 +402,17 @@ func (s *Service) processFromSourceOrFetch(
 	// Process using cached source; nothing was fetched from upstream. The
 	// image processor reads the source only once it has a processing slot,
 	// so a request waiting for one holds none of it in memory.
-	resp, err := s.processAndStore(ctx, req, cacheKey, source, sourceSize)
+	return s.processAndStore(ctx, req, cacheKey, source, sourceSize)
-
-	return resp, 0, err
 }
 
 // fetchAndProcess fetches from upstream, processes, and caches the result.
-// It also returns the number of bytes read from upstream, including when
+// It counts the fetch with the bytes read from upstream, including when
 // reading the response or a later step fails.
 func (s *Service) fetchAndProcess(
 	ctx context.Context,
 	req *ImageRequest,
 	cacheKey VariantKey,
-) (*ImageResponse, int64, error) {
+) (*processedVariant, error) {
 	// Fetch from upstream
 	sourceURL := req.SourceURL()
 
@@ -339,7 +431,7 @@ func (s *Service) fetchAndProcess(
 			}
 		}
 
-		return nil, 0, fmt.Errorf("upstream fetch failed: %w", err)
+		return nil, fmt.Errorf("upstream fetch failed: %w", err)
 	}
 
 	// Closing the body frees the upstream connection. It is closed only
@@ -352,8 +444,11 @@ func (s *Service) fetchAndProcess(
 	sourceData, err := io.ReadAll(fetchResult.Content)
 	fetchBytes := int64(len(sourceData))
 
+	// Counted also when the request context has ended meanwhile
+	s.cache.IncrementUpstreamFetch(context.WithoutCancel(ctx), fetchBytes)
+
 	if err != nil {
-		return nil, fetchBytes, fmt.Errorf("failed to read upstream response: %w", err)
+		return nil, fmt.Errorf("failed to read upstream response: %w", err)
 	}
 
 	// Calculate download bitrate
@@ -367,6 +462,7 @@ func (s *Service) fetchAndProcess(
 
 	// Log upstream fetch details
 	s.log.Info("upstream fetched",
+		"request_id", middleware.GetReqID(ctx),
 		"host", req.SourceHost,
 		"path", req.SourcePath,
 		"bytes", fetchBytes,
@@ -381,7 +477,7 @@ func (s *Service) fetchAndProcess(
 	// Validate magic bytes match content type
 	err = magic.ValidateMagicBytes(sourceData, fetchResult.ContentType)
 	if err != nil {
-		return nil, fetchBytes, fmt.Errorf("content validation failed: %w", err)
+		return nil, fmt.Errorf("content validation failed: %w", err)
 	}
 
 	// Store source content
@@ -391,11 +487,17 @@ func (s *Service) fetchAndProcess(
 		// Continue even if caching fails
 	}
 
-	resp, err := s.processAndStore(
+	return s.processAndStore(
 		ctx, req, cacheKey, bytes.NewReader(sourceData), fetchBytes,
 	)
+}
 
-	return resp, fetchBytes, err
+// processedVariant is a variant as processAndStore made it. Each request that
+// shared its processing serves it through a reader of its own.
+type processedVariant struct {
+	data         []byte
+	contentType  string
+	fetchedBytes int64
 }
 
 // processAndStore processes the image read from source and stores the
@@ -406,7 +508,7 @@ func (s *Service) processAndStore(
 	cacheKey VariantKey,
 	source io.Reader,
 	fetchBytes int64,
-) (*ImageResponse, error) {
+) (*processedVariant, error) {
 	// Process the image
 	processStart := time.Now()
 
@@ -445,6 +547,7 @@ func (s *Service) processAndStore(
 	}
 
 	s.log.Info("image converted",
+		"request_id", middleware.GetReqID(ctx),
 		"host", req.SourceHost,
 		"path", req.SourcePath,
 		"src_format", processResult.InputFormat,
@@ -470,12 +573,10 @@ func (s *Service) processAndStore(
 		// Continue even if caching fails
 	}
 
-	return &ImageResponse{
+	return &processedVariant{
-		Content:       io.NopCloser(bytes.NewReader(processedData)),
+		data:         processedData,
-		ContentLength: outputSize,
+		contentType:  processResult.ContentType,
-		ContentType:   processResult.ContentType,
+		fetchedBytes: fetchBytes,
-		FetchedBytes:  fetchBytes,
-		ETag:          formatETag(cacheKey),
 	}, nil
 }
 

internal/imgcache/storage.go

@@ -564,7 +564,8 @@ func (s *VariantStorage) Exists(key VariantKey) bool {
 	return err == nil
 }
 
-// Delete removes content at the given key.
+// Delete removes the content at the given key together with its .meta
+// sidecar file. A missing file is not an error.
 func (s *VariantStorage) Delete(key VariantKey) error {
 	path := s.keyToPath(key)
 
@@ -573,18 +574,7 @@ func (s *VariantStorage) Delete(key VariantKey) error {
 		return fmt.Errorf("failed to delete content: %w", err)
 	}
 
-	return nil
+	metaPath := path + ".meta"
-}
-
-// DeleteWithMeta removes the content at the given key together with
-// its .meta sidecar file. A missing file is not an error.
-func (s *VariantStorage) DeleteWithMeta(key VariantKey) error {
-	err := s.Delete(key)
-	if err != nil {
-		return err
-	}
-
-	metaPath := s.keyToPath(key) + ".meta"
 
 	err = os.Remove(metaPath)
 	if err != nil && !os.IsNotExist(err) {

internal/imgcache/storage_internal_test.go

@@ -438,3 +438,73 @@ func TestVariantStorage_StoreLogsFailedMetaWrite(t *testing.T) {
 		t.Errorf("log missing %s; got %q", want, logBuf.String())
 	}
 }
+
+// storeTestVariant stores one variant, with its .meta file, in a new
+// VariantStorage and returns the storage and the variant's key.
+func storeTestVariant(t *testing.T) (*VariantStorage, VariantKey) {
+	t.Helper()
+
+	storage, err := NewVariantStorage(t.TempDir(), slog.New(slog.DiscardHandler))
+	if err != nil {
+		t.Fatalf("NewVariantStorage() error = %v", err)
+	}
+
+	key := CacheKey(&ImageRequest{SourceHost: testHostCDN, SourcePath: testPathCat})
+
+	_, err = storage.Store(key, bytes.NewReader([]byte("variant data")), "image/webp")
+	if err != nil {
+		t.Fatalf("Store() error = %v", err)
+	}
+
+	return storage, key
+}
+
+// TestVariantStorage_DeleteRemovesMeta verifies that Delete removes the
+// variant's .meta file along with the variant file.
+func TestVariantStorage_DeleteRemovesMeta(t *testing.T) {
+	t.Parallel()
+
+	storage, key := storeTestVariant(t)
+	metaPath := storage.keyToPath(key) + ".meta"
+
+	_, err := os.Stat(metaPath)
+	if err != nil {
+		t.Fatalf("Store() wrote no .meta file: %v", err)
+	}
+
+	err = storage.Delete(key)
+	if err != nil {
+		t.Fatalf("Delete() error = %v", err)
+	}
+
+	if storage.Exists(key) {
+		t.Error("Exists() = true after delete, want false")
+	}
+
+	_, err = os.Stat(metaPath)
+	if !os.IsNotExist(err) {
+		t.Errorf(".meta file left after Delete() (stat err=%v)", err)
+	}
+}
+
+// TestVariantStorage_DeleteWithoutMeta verifies that Delete succeeds for a
+// variant whose .meta file is missing.
+func TestVariantStorage_DeleteWithoutMeta(t *testing.T) {
+	t.Parallel()
+
+	storage, key := storeTestVariant(t)
+
+	err := os.Remove(storage.keyToPath(key) + ".meta")
+	if err != nil {
+		t.Fatalf("removing .meta file: %v", err)
+	}
+
+	err = storage.Delete(key)
+	if err != nil {
+		t.Fatalf("Delete() error = %v, want nil", err)
+	}
+
+	if storage.Exists(key) {
+		t.Error("Exists() = true after delete, want false")
+	}
+}

internal/middleware/middleware.go

@@ -2,9 +2,12 @@
 package middleware
 
 import (
+	"context"
+	"crypto/rand"
 	"log/slog"
 	"net/http"
 	"net/netip"
+	"regexp"
 	"time"
 
 	basicauth "github.com/99designs/basicauth-go"
@@ -32,13 +35,10 @@ const HSTSValue = "max-age=31536000; includeSubDomains"
 
 // ContentSecurityPolicyValue is the Content-Security-Policy header value.
 // default-src 'self' is the baseline and frame-ancestors 'none' is the primary
-// clickjacking control. 'unsafe-inline' is required in script-src and style-src
+// clickjacking control.
-// because the served templates carry inline onclick handlers (generator page)
-// and the bundled Tailwind asset injects a runtime <style> element; dropping it
-// needs template changes outside this issue's scope.
 const ContentSecurityPolicyValue = "default-src 'self'; " +
-	"script-src 'self' 'unsafe-inline'; " +
+	"script-src 'self'; " +
-	"style-src 'self' 'unsafe-inline'; " +
+	"style-src 'self'; " +
 	"object-src 'none'; " +
 	"base-uri 'self'; " +
 	"form-action 'self'; " +
@@ -115,6 +115,32 @@ func (s *Middleware) RateLimit(
 		})
 }
 
+// requestIDPattern is what a request's own X-Request-Id must look like to be
+// kept as its ID: 1 to 64 letters, digits, '-', '_' or '.'.
+var requestIDPattern = regexp.MustCompile(`^[A-Za-z0-9._-]{1,64}$`)
+
+// RequestID returns a middleware that gives each request an ID and sends it as
+// the X-Request-Id response header, so a client can quote it when reporting a
+// problem. The ID is the request's own X-Request-Id when that matches
+// requestIDPattern, and otherwise a random one, which tells nothing about the
+// machine or the traffic. It is stored in the request context under chi's
+// RequestIDKey, where the logging middleware, the handlers and the upstream
+// fetch read it.
+func (s *Middleware) RequestID() func(http.Handler) http.Handler {
+	return func(next http.Handler) http.Handler {
+		return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+			id := r.Header.Get(middleware.RequestIDHeader)
+			if !requestIDPattern.MatchString(id) {
+				id = rand.Text()
+			}
+
+			w.Header().Set(middleware.RequestIDHeader, id)
+			ctx := context.WithValue(r.Context(), middleware.RequestIDKey, id)
+			next.ServeHTTP(w, r.WithContext(ctx))
+		})
+	}
+}
+
 type loggingResponseWriter struct {
 	http.ResponseWriter
 

internal/middleware/middleware_internal_test.go

@@ -1,11 +1,16 @@
 package middleware
 
 import (
+	"bytes"
 	"log/slog"
 	"net/http"
 	"net/http/httptest"
+	"net/url"
+	"strings"
 	"testing"
 
+	"github.com/prometheus/client_golang/prometheus/promhttp"
+
 	"sneak.berlin/go/pixa/internal/config"
 )
 
@@ -56,6 +61,203 @@ func TestCORSAnswersWithConfiguredOrigin(t *testing.T) {
 	}
 }
 
+// TestCORSAnswersPreflightWithConfiguredOrigin checks that the CORS
+// middleware answers a preflight request, which the CORS library handles
+// apart from other requests, the same way: "*" lets any origin read
+// responses and a single origin lets only that origin read them.
+func TestCORSAnswersPreflightWithConfiguredOrigin(t *testing.T) {
+	t.Parallel()
+
+	const appOrigin = "https://app.example.com"
+
+	cases := []struct {
+		configured    string
+		requestOrigin string
+		want          string
+	}{
+		{"*", "https://any.example.com", "*"},
+		{appOrigin, appOrigin, appOrigin},
+		{appOrigin, "https://other.example.com", ""},
+	}
+
+	testHandler := http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
+		w.WriteHeader(http.StatusOK)
+	})
+
+	for _, tc := range cases {
+		mw := &Middleware{
+			log:    slog.Default(),
+			config: &config.Config{AccessControlAllowOrigin: tc.configured},
+		}
+
+		handler := mw.CORS()(testHandler)
+
+		// An OPTIONS request naming the method it asks about is the
+		// preflight a browser sends before some cross-origin requests.
+		req := httptest.NewRequestWithContext(
+			t.Context(), http.MethodOptions, "/v1/image/example.com/a.jpg/1x1.png", nil)
+		req.Header.Set("Origin", tc.requestOrigin)
+		req.Header.Set("Access-Control-Request-Method", http.MethodGet)
+
+		rec := httptest.NewRecorder()
+
+		handler.ServeHTTP(rec, req)
+
+		got := rec.Header().Get("Access-Control-Allow-Origin")
+		if got != tc.want {
+			t.Errorf("configured %q, preflight from %q: "+
+				"Access-Control-Allow-Origin = %q, want %q",
+				tc.configured, tc.requestOrigin, got, tc.want)
+		}
+	}
+}
+
+// TestMetricsAuthRequiresConfiguredCredentials checks that MetricsAuth on
+// its own answers 401 with a challenge to a request without credentials or
+// with a wrong username or password, and lets a request with the configured
+// username and password through. That the router puts it in front of
+// /metrics is not tested.
+func TestMetricsAuthRequiresConfiguredCredentials(t *testing.T) {
+	t.Parallel()
+
+	const (
+		username  = "metricsuser"
+		password  = "metricspass"
+		challenge = `Basic realm="metrics"`
+	)
+
+	// An empty username stands for a request sent without credentials.
+	cases := []struct {
+		name        string
+		username    string
+		password    string
+		wantReached bool
+	}{
+		{"no credentials", "", "", false},
+		{"wrong username", "someone", password, false},
+		{"wrong password", username, "wrongpass", false},
+		{"configured credentials", username, password, true},
+	}
+
+	for _, tc := range cases {
+		t.Run(tc.name, func(t *testing.T) {
+			t.Parallel()
+
+			mw := &Middleware{
+				log: slog.Default(),
+				config: &config.Config{
+					MetricsUsername: username,
+					MetricsPassword: password,
+				},
+			}
+
+			reached := false
+			handler := mw.MetricsAuth()(http.HandlerFunc(
+				func(http.ResponseWriter, *http.Request) {
+					reached = true
+				}))
+
+			req := httptest.NewRequestWithContext(
+				t.Context(), http.MethodGet, "/metrics", nil)
+
+			if tc.username != "" {
+				req.SetBasicAuth(tc.username, tc.password)
+			}
+
+			rec := httptest.NewRecorder()
+
+			handler.ServeHTTP(rec, req)
+
+			if reached != tc.wantReached {
+				t.Fatalf("request reached /metrics = %v, want %v",
+					reached, tc.wantReached)
+			}
+
+			if tc.wantReached {
+				return
+			}
+
+			if rec.Code != http.StatusUnauthorized {
+				t.Errorf("status = %d, want %d",
+					rec.Code, http.StatusUnauthorized)
+			}
+
+			if got := rec.Header().Get("WWW-Authenticate"); got != challenge {
+				t.Errorf("WWW-Authenticate = %q, want %q", got, challenge)
+			}
+		})
+	}
+}
+
+// TestMetricsRecordsServedRequest checks that the metrics middleware
+// records a request it served, so /metrics reports it. It is the only test
+// in this package that sets up the metrics middleware, which registers with
+// the process-wide Prometheus registry and can do so only once.
+func TestMetricsRecordsServedRequest(t *testing.T) {
+	t.Parallel()
+
+	// The line /metrics shows once one GET /test has been served.
+	const want = `http_request_duration_seconds_count{` +
+		`code="200",handler="/test",method="GET",service=""} 1`
+
+	mw := &Middleware{log: slog.Default(), config: &config.Config{}}
+
+	handler := mw.Metrics()(http.HandlerFunc(
+		func(w http.ResponseWriter, _ *http.Request) {
+			w.WriteHeader(http.StatusOK)
+		}))
+
+	handler.ServeHTTP(httptest.NewRecorder(), httptest.NewRequestWithContext(
+		t.Context(), http.MethodGet, "/test", nil))
+
+	rec := httptest.NewRecorder()
+	promhttp.Handler().ServeHTTP(rec, httptest.NewRequestWithContext(
+		t.Context(), http.MethodGet, "/metrics", nil))
+
+	if !strings.Contains(rec.Body.String(), want) {
+		t.Errorf("/metrics does not report the GET /test served; "+
+			"want the line %q in:\n%s", want, rec.Body.String())
+	}
+}
+
+// TestLoggingLeavesOutSubmittedSigningKey checks that a login, a POST /
+// whose form carries the signing key, leaves no trace of the key in the
+// request's log line.
+func TestLoggingLeavesOutSubmittedSigningKey(t *testing.T) {
+	t.Parallel()
+
+	const signingKey = "test-signing-key-0123456789abcdef"
+
+	var buf bytes.Buffer
+
+	mw := newTestMiddleware(t, &buf)
+
+	// The handler reads the key from the form, as the login handler does.
+	handler := mw.Logging()(http.HandlerFunc(
+		func(w http.ResponseWriter, r *http.Request) {
+			if got := r.FormValue("key"); got != signingKey {
+				t.Errorf("key in form = %q, want %q", got, signingKey)
+			}
+
+			w.WriteHeader(http.StatusSeeOther)
+		}))
+
+	form := url.Values{"key": {signingKey}}
+	req := httptest.NewRequestWithContext(t.Context(), http.MethodPost, "/",
+		strings.NewReader(form.Encode()))
+	req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
+
+	handler.ServeHTTP(httptest.NewRecorder(), req)
+
+	if !strings.Contains(buf.String(), `"method":"POST"`) {
+		t.Fatalf("no log line for the request; got %q", buf.String())
+	}
+
+	if strings.Contains(buf.String(), signingKey) {
+		t.Errorf("log output contains the signing key; got %q", buf.String())
+	}
+}
+
 func TestSecurityHeaders(t *testing.T) {
 	t.Parallel()
 
@@ -123,6 +325,13 @@ func TestSecurityHeaders_PolicyHeaders(t *testing.T) {
 
 	handler.ServeHTTP(rec, req)
 
+	// The login and generator pages load their script and stylesheet from
+	// /static, so the policy allows no inline script or style.
+	csp := rec.Header().Get("Content-Security-Policy")
+	if strings.Contains(csp, "unsafe-inline") {
+		t.Errorf("Content-Security-Policy allows unsafe-inline: %q", csp)
+	}
+
 	tests := []struct {
 		header string
 		want   string
@@ -131,8 +340,8 @@ func TestSecurityHeaders_PolicyHeaders(t *testing.T) {
 		{
 			"Content-Security-Policy",
 			"default-src 'self'; " +
-				"script-src 'self' 'unsafe-inline'; " +
+				"script-src 'self'; " +
-				"style-src 'self' 'unsafe-inline'; " +
+				"style-src 'self'; " +
 				"object-src 'none'; " +
 				"base-uri 'self'; " +
 				"form-action 'self'; " +

internal/middleware/request_id_internal_test.go

@@ -0,0 +1,118 @@
+package middleware
+
+import (
+	"log/slog"
+	"net/http"
+	"net/http/httptest"
+	"os"
+	"strings"
+	"testing"
+
+	"github.com/go-chi/chi/v5/middleware"
+
+	"sneak.berlin/go/pixa/internal/config"
+)
+
+// sendRequestID sends a request through the RequestID middleware, carrying
+// incoming as its X-Request-Id unless that is empty. It returns the ID the
+// next handler found in the request context, which the upstream fetch sends
+// and the log lines carry, and the X-Request-Id of the response.
+func sendRequestID(t *testing.T, incoming string) (string, string) {
+	t.Helper()
+
+	mw := &Middleware{log: slog.Default(), config: &config.Config{}}
+
+	var inContext string
+
+	handler := mw.RequestID()(http.HandlerFunc(
+		func(_ http.ResponseWriter, r *http.Request) {
+			inContext = middleware.GetReqID(r.Context())
+		}))
+
+	req := httptest.NewRequestWithContext(t.Context(), http.MethodGet, "/", nil)
+	if incoming != "" {
+		req.Header.Set("X-Request-Id", incoming)
+	}
+
+	rec := httptest.NewRecorder()
+	handler.ServeHTTP(rec, req)
+
+	return inContext, rec.Header().Get("X-Request-Id")
+}
+
+// TestRequestIDKeepsShortPlainID verifies that a request's own X-Request-Id of
+// at most 64 letters, digits, '-', '_' or '.' is kept as its ID.
+func TestRequestIDKeepsShortPlainID(t *testing.T) {
+	t.Parallel()
+
+	for _, incoming := range []string{
+		"client-request-id",
+		"A1_b2.c3-d4",
+		strings.Repeat("a", 64),
+	} {
+		inContext, inResponse := sendRequestID(t, incoming)
+
+		if inContext != incoming || inResponse != incoming {
+			t.Errorf("incoming %q: context has %q, response %q, want both %q",
+				incoming, inContext, inResponse, incoming)
+		}
+	}
+}
+
+// TestRequestIDReplacesLongOrUnusualID verifies that a request's own
+// X-Request-Id that is over 64 characters or holds anything but letters,
+// digits, '-', '_' or '.' is neither sent back nor sent upstream: the request
+// gets a fresh ID instead.
+func TestRequestIDReplacesLongOrUnusualID(t *testing.T) {
+	t.Parallel()
+
+	for _, incoming := range []string{
+		strings.Repeat("a", 65),
+		strings.Repeat("a", 9000),
+		"has space",
+		"a/b",
+		"a,b",
+		"<script>",
+		"ünicode",
+	} {
+		inContext, inResponse := sendRequestID(t, incoming)
+		t.Logf("incoming %.20q: made up %q", incoming, inResponse)
+
+		if inResponse == "" || inResponse == incoming {
+			t.Errorf("incoming %.20q: response has %q, want a fresh ID",
+				incoming, inResponse)
+		}
+
+		if inContext != inResponse {
+			t.Errorf("incoming %.20q: context has %q, want the response's %q",
+				incoming, inContext, inResponse)
+		}
+	}
+}
+
+// TestRequestIDMadeUpTellsNothing verifies that the ID made up for a request
+// that sent none differs for every request and does not hold the host name.
+func TestRequestIDMadeUpTellsNothing(t *testing.T) {
+	t.Parallel()
+
+	hostname, err := os.Hostname()
+	if err != nil {
+		t.Fatalf("os.Hostname() error = %v", err)
+	}
+
+	firstInContext, first := sendRequestID(t, "")
+	_, second := sendRequestID(t, "")
+	t.Logf("host %q, made up %q and %q", hostname, first, second)
+
+	if first == "" || first == second {
+		t.Errorf("made up %q and %q, want two different IDs", first, second)
+	}
+
+	if firstInContext != first {
+		t.Errorf("context has %q, want the response's %q", firstInContext, first)
+	}
+
+	if strings.Contains(first, hostname) {
+		t.Errorf("made-up ID %q holds the host name %q", first, hostname)
+	}
+}

internal/server/cors_internal_test.go

@@ -0,0 +1,64 @@
+package server
+
+import (
+	"net/http"
+	"net/http/httptest"
+	"testing"
+)
+
+// TestCORSOnlyOnImageRoutes verifies that the image routes answer with the
+// configured access_control_allow_origin, a preflight request included, and
+// that the login and URL generator pages send no Access-Control-Allow-Origin,
+// so no other site can read them. /metrics is left out: its middleware
+// registers with the process-wide Prometheus registry, which only one test
+// in this package can do.
+func TestCORSOnlyOnImageRoutes(t *testing.T) {
+	t.Parallel()
+
+	const appOrigin = "https://app.example.com"
+
+	s := newTestServer(t)
+	s.config.AccessControlAllowOrigin = appOrigin
+	s.SetupRoutes()
+
+	requests := []struct {
+		method string
+		path   string
+		want   string
+	}{
+		{http.MethodGet, unsignedImagePath, appOrigin},
+		{http.MethodHead, unsignedImagePath, appOrigin},
+		{http.MethodOptions, unsignedImagePath, appOrigin},
+		{http.MethodGet, encryptedImagePath, appOrigin},
+		{http.MethodGet, "/", ""},
+		{http.MethodOptions, "/", ""},
+		{http.MethodPost, "/generate", ""},
+		{http.MethodGet, "/logout", ""},
+	}
+
+	for _, tc := range requests {
+		t.Run(tc.method+" "+tc.path, func(t *testing.T) {
+			t.Parallel()
+
+			req := httptest.NewRequestWithContext(
+				t.Context(), tc.method, tc.path, nil)
+			req.Header.Set("Origin", appOrigin)
+
+			// An OPTIONS request naming the method it asks about is the
+			// preflight a browser sends before some cross-origin requests.
+			if tc.method == http.MethodOptions {
+				req.Header.Set("Access-Control-Request-Method", http.MethodGet)
+			}
+
+			rec := httptest.NewRecorder()
+			s.ServeHTTP(rec, req)
+			t.Logf("status %d", rec.Code)
+
+			got := rec.Header().Get("Access-Control-Allow-Origin")
+			if got != tc.want {
+				t.Errorf("Access-Control-Allow-Origin = %q, want %q",
+					got, tc.want)
+			}
+		})
+	}
+}

internal/server/http.go

@@ -5,6 +5,8 @@ import (
 	"fmt"
 	"net/http"
 	"time"
+
+	"go.uber.org/fx"
 )
 
 // HTTP server configuration constants.
@@ -36,19 +38,19 @@ func (s *Server) newHTTPServer() *http.Server {
 	}
 }
 
+// serveUntilShutdown serves on s.httpServer until it is shut down. When it
+// stops for any other reason, such as its port being in use, it asks fx to
+// shut down with exit code 1.
 func (s *Server) serveUntilShutdown() {
-	s.httpServer = s.newHTTPServer()
-
-	s.SetupRoutes()
-
 	s.log.Info("http begin listen", "listenaddr", s.httpServer.Addr)
 
 	err := s.httpServer.ListenAndServe()
 	if err != nil && !errors.Is(err, http.ErrServerClosed) {
 		s.log.Error("listen error", "error", err)
 
-		if s.cancelFunc != nil {
+		err = s.shutdowner.Shutdown(fx.ExitCode(1))
-			s.cancelFunc()
+		if err != nil {
+			s.log.Error("shutdown request failed", "error", err)
 		}
 	}
 }

internal/server/maintenance_internal_test.go

@@ -13,6 +13,10 @@ import (
 // unsignedImagePath is an image URL that carries no signature.
 const unsignedImagePath = "/v1/image/cdn.example.com/cat.jpg/100x100.jpeg"
 
+// encryptedImagePath is an encrypted image URL whose token cannot be
+// decrypted.
+const encryptedImagePath = "/v1/e/token/cat.jpg"
+
 // TestMaintenanceModeRefusesImageRequests verifies that while maintenance
 // mode is on, both image routes answer 503 Service Unavailable with a
 // Retry-After header and the JSON error body the image handlers send.
@@ -28,7 +32,7 @@ func TestMaintenanceModeRefusesImageRequests(t *testing.T) {
 	}{
 		{http.MethodGet, unsignedImagePath},
 		{http.MethodHead, unsignedImagePath},
-		{http.MethodGet, "/v1/e/token/cat.jpg"},
+		{http.MethodGet, encryptedImagePath},
 	}
 
 	for _, tc := range requests {
@@ -95,7 +99,7 @@ func TestImageRequestsServedWithoutMaintenanceMode(t *testing.T) {
 	}{
 		{http.MethodGet, unsignedImagePath, http.StatusUnauthorized},
 		{http.MethodHead, unsignedImagePath, http.StatusUnauthorized},
-		{http.MethodGet, "/v1/e/token/cat.jpg", http.StatusBadRequest},
+		{http.MethodGet, encryptedImagePath, http.StatusBadRequest},
 	}
 
 	for _, tc := range requests {

internal/server/metrics_internal_test.go

@@ -0,0 +1,32 @@
+package server
+
+import (
+	"net/http"
+	"net/http/httptest"
+	"strings"
+	"testing"
+
+	"github.com/prometheus/client_golang/prometheus/promhttp"
+)
+
+// TestNoMetricsRecordedWithoutMetricsUsername checks that with no metrics
+// username set the router records nothing about the requests it serves.
+// /metrics is not served then, so the process-wide Prometheus registry is
+// read directly. TestMaintenanceModeKeepsOtherRoutes records into the same
+// registry, but never a GET /robots.txt.
+func TestNoMetricsRecordedWithoutMetricsUsername(t *testing.T) {
+	t.Parallel()
+
+	s := newTestServer(t)
+	s.ServeHTTP(httptest.NewRecorder(), httptest.NewRequestWithContext(
+		t.Context(), http.MethodGet, "/robots.txt", nil))
+
+	rec := httptest.NewRecorder()
+	promhttp.Handler().ServeHTTP(rec, httptest.NewRequestWithContext(
+		t.Context(), http.MethodGet, "/metrics", nil))
+
+	if strings.Contains(rec.Body.String(), `handler="/robots.txt"`) {
+		t.Errorf("with no metrics username GET /robots.txt was recorded:\n%s",
+			rec.Body.String())
+	}
+}

internal/server/request_id_internal_test.go

@@ -0,0 +1,58 @@
+package server
+
+import (
+	"net/http"
+	"net/http/httptest"
+	"testing"
+)
+
+// requestIDHeader is the header that carries a request's ID.
+const requestIDHeader = "X-Request-Id"
+
+// TestResponsesCarryRequestID verifies that every response, whatever its route
+// and status, carries the request's ID as X-Request-Id, so a client can quote
+// it when reporting a problem: one pixa made up when the request brought none,
+// and the request's own X-Request-Id when it brought one.
+func TestResponsesCarryRequestID(t *testing.T) {
+	t.Parallel()
+
+	const clientRequestID = "client-request-id"
+
+	s := newTestServer(t)
+
+	paths := []string{
+		"/robots.txt",
+		"/no-such-path",
+		unsignedImagePath,
+		encryptedImagePath,
+	}
+
+	for _, path := range paths {
+		t.Run(path, func(t *testing.T) {
+			t.Parallel()
+
+			rec := httptest.NewRecorder()
+			s.ServeHTTP(rec, httptest.NewRequestWithContext(
+				t.Context(), http.MethodGet, path, nil))
+			t.Logf("status %d, %s %q",
+				rec.Code, requestIDHeader, rec.Header().Get(requestIDHeader))
+
+			if rec.Header().Get(requestIDHeader) == "" {
+				t.Errorf("response has no %s", requestIDHeader)
+			}
+
+			req := httptest.NewRequestWithContext(
+				t.Context(), http.MethodGet, path, nil)
+			req.Header.Set(requestIDHeader, clientRequestID)
+
+			rec = httptest.NewRecorder()
+			s.ServeHTTP(rec, req)
+
+			got := rec.Header().Get(requestIDHeader)
+			if got != clientRequestID {
+				t.Errorf("%s = %q, want the request's own %q",
+					requestIDHeader, got, clientRequestID)
+			}
+		})
+	}
+}

internal/server/routes.go

@@ -28,7 +28,7 @@ func (s *Server) SetupRoutes() {
 	s.router = chi.NewRouter()
 
 	s.router.Use(middleware.Recoverer)
-	s.router.Use(middleware.RequestID)
+	s.router.Use(s.mw.RequestID())
 	s.router.Use(s.mw.ClientIP())
 	s.router.Use(s.mw.SecurityHeaders())
 	s.router.Use(s.mw.Logging())
@@ -38,7 +38,6 @@ func (s *Server) SetupRoutes() {
 		s.router.Use(s.mw.Metrics())
 	}
 
-	s.router.Use(s.mw.CORS())
 	s.router.Use(middleware.Timeout(s.config.DownstreamTimeout))
 
 	if s.sentryEnabled {
@@ -54,7 +53,7 @@ func (s *Server) SetupRoutes() {
 	// Robots.txt
 	s.router.Get("/robots.txt", s.h.HandleRobotsTxt())
 
-	// Static files (Tailwind CSS, etc.)
+	// The login and generator pages' stylesheet and script
 	s.router.Handle("/static/*", http.StripPrefix("/static/", static.Handler()))
 
 	// Login/generator UI. The form routes carry CSRF protection; the
@@ -74,22 +73,32 @@ func (s *Server) SetupRoutes() {
 
 	s.router.Get("/logout", s.h.HandleLogout())
 
-	// Image routes, refused while maintenance mode is on. Only these: the
+	// Image routes, the only ones that send CORS headers, as pages on other
-	// image's Docker HEALTHCHECK requests the health check, a 503 there
+	// sites read them. They are a subrouter rather than a group: a group's
-	// would make the container unhealthy, and upaas marks a deploy failed
+	// middleware runs only for a request that matches one of its routes,
-	// when its container is unhealthy.
+	// and a browser's preflight OPTIONS request matches none, so the CORS
-	s.router.Group(func(r chi.Router) {
+	// middleware could not answer it.
-		r.Use(s.refuseDuringMaintenance)
+	s.router.Route("/v1", func(r chi.Router) {
+		r.Use(s.mw.CORS())
 
-		// Main image proxy route
+		// Refused while maintenance mode is on. Only these: the image's
-		// /v1/image/<host>/<path>/<width>x<height>.<format>
+		// Docker HEALTHCHECK requests the health check, a 503 there would
-		r.Get("/v1/image/*", s.h.HandleImage())
+		// make the container unhealthy, and upaas marks a deploy failed
-		r.Head("/v1/image/*", s.h.HandleImage())
+		// when its container is unhealthy.
+		r.Group(func(r chi.Router) {
+			r.Use(s.refuseDuringMaintenance)
 
-		// Encrypted image URL route
+			// Main image proxy route
-		// The trailing filename (e.g., /img.jpg) is ignored but helps
+			// /v1/image/<host>/<path>/<width>x<height>.<format>
-		// browsers with content type
+			r.Get("/image/*", s.h.HandleImage())
-		r.Get("/v1/e/{token}/*", s.h.HandleImageEnc())
+			r.Head("/image/*", s.h.HandleImage())
+
+			// Encrypted image URL route
+			// The trailing filename (e.g., /img.jpg) is ignored but helps
+			// browsers with content type
+			r.Get("/e/{token}/*", s.h.HandleImageEnc())
+			r.Head("/e/{token}/*", s.h.HandleImageEnc())
+		})
 	})
 
 	// Metrics endpoint with auth

internal/server/routes_internal_test.go

@@ -0,0 +1,27 @@
+package server
+
+import (
+	"net/http"
+	"net/http/httptest"
+	"testing"
+)
+
+// TestEncryptedImageRouteAnswersHEAD verifies that HEAD on the encrypted image
+// route reaches its handler, as GET does, instead of being answered 405 Method
+// Not Allowed. The handler refuses a token it cannot decrypt with 400, so that
+// status shows the request got through.
+func TestEncryptedImageRouteAnswersHEAD(t *testing.T) {
+	t.Parallel()
+
+	s := newTestServer(t)
+
+	rec := httptest.NewRecorder()
+	s.ServeHTTP(rec, httptest.NewRequestWithContext(
+		t.Context(), http.MethodHead, encryptedImagePath, nil))
+	t.Logf("status %d", rec.Code)
+
+	if rec.Code != http.StatusBadRequest {
+		t.Errorf("status = %d, want %d from the encrypted image handler",
+			rec.Code, http.StatusBadRequest)
+	}
+}

internal/server/server.go

@@ -3,12 +3,10 @@ package server
 
 import (
 	"context"
+	"errors"
 	"fmt"
 	"log/slog"
 	"net/http"
-	"os"
-	"os/signal"
-	"syscall"
 	"time"
 
 	"github.com/getsentry/sentry-go"
@@ -27,6 +25,10 @@ const (
 	SentryFlushTimeout = 2 * time.Second
 )
 
+// errStillProcessing is returned by the server's stop hook when images are
+// still being processed once ShutdownTimeout has passed.
+var errStillProcessing = errors.New("images still being processed at shutdown")
+
 // Params defines dependencies for Server.
 type Params struct {
 	fx.In
@@ -36,6 +38,7 @@ type Params struct {
 	Config     *config.Config
 	Middleware *middleware.Middleware
 	Handlers   *handlers.Handlers
+	Shutdowner fx.Shutdowner
 }
 
 // Server is the main HTTP server.
@@ -45,59 +48,58 @@ type Server struct {
 	globals       *globals.Globals
 	mw            *middleware.Middleware
 	h             *handlers.Handlers
+	shutdowner    fx.Shutdowner
 	startupTime   time.Time
-	exitCode      int
 	sentryEnabled bool
-	cancelFunc    context.CancelFunc
 	httpServer    *http.Server
 	router        *chi.Mux
 }
 
-// New creates a new Server instance.
+// New creates a new Server instance. Its start hook starts Sentry and the
+// HTTP server; its stop hook, which fx runs on SIGINT, SIGTERM or a
+// shutdown request, shuts them down.
 func New(lc fx.Lifecycle, params Params) (*Server, error) {
 	s := &Server{
-		log:     params.Logger.Get(),
+		log:        params.Logger.Get(),
-		config:  params.Config,
+		config:     params.Config,
-		globals: params.Globals,
+		globals:    params.Globals,
-		mw:      params.Middleware,
+		mw:         params.Middleware,
-		h:       params.Handlers,
+		h:          params.Handlers,
+		shutdowner: params.Shutdowner,
 	}
 
 	lc.Append(fx.Hook{
-		OnStart: func(ctx context.Context) error {
+		OnStart: func(_ context.Context) error {
 			s.startupTime = time.Now()
-			go s.Run(context.WithoutCancel(ctx))
 
-			return nil
+			err := s.enableSentry()
-		},
+			if err != nil {
-		OnStop: func(_ context.Context) error {
+				return err
-			if s.cancelFunc != nil {
-				s.cancelFunc()
 			}
 
+			s.SetupRoutes()
+			s.httpServer = s.newHTTPServer()
+
+			go s.serveUntilShutdown()
+
 			return nil
 		},
+		OnStop: s.cleanShutdown,
 	})
 
 	return s, nil
 }
 
-// Run starts the server.
-func (s *Server) Run(ctx context.Context) {
-	s.enableSentry()
-	s.serve(ctx)
-}
-
 // MaintenanceMode returns whether maintenance mode is enabled.
 func (s *Server) MaintenanceMode() bool {
 	return s.config.MaintenanceMode
 }
 
-func (s *Server) enableSentry() {
+func (s *Server) enableSentry() error {
 	s.sentryEnabled = false
 
 	if s.config.SentryDSN == "" {
-		return
+		return nil
 	}
 
 	err := sentry.Init(sentry.ClientOptions{
@@ -105,55 +107,42 @@ func (s *Server) enableSentry() {
 		Release: fmt.Sprintf("%s-%s", s.globals.Appname, s.globals.Version),
 	})
 	if err != nil {
-		s.log.Error("sentry init failure", "error", err)
+		return fmt.Errorf("sentry init failure: %w", err)
-		os.Exit(1)
 	}
 
 	s.log.Info("sentry error reporting activated")
 	s.sentryEnabled = true
+
+	return nil
 }
 
-func (s *Server) serve(ctx context.Context) int {
+// cleanShutdown stops the HTTP server, waits for the images still being
-	ctx, cancelFunc := context.WithCancel(ctx)
+// processed, then flushes Sentry. The first two share ShutdownTimeout. It
-	s.cancelFunc = cancelFunc
+// returns errStillProcessing when images are still being processed after
+// that, as their work is abandoned.
+func (s *Server) cleanShutdown(ctx context.Context) error {
+	s.log.Info("shutting down")
 
-	go func() {
+	ctxShutdown, shutdownCancel := context.WithTimeout(ctx, ShutdownTimeout)
-		c := make(chan os.Signal, 1)
-
-		signal.Ignore(syscall.SIGPIPE)
-		signal.Notify(c, os.Interrupt, syscall.SIGTERM)
-
-		sig := <-c
-		s.log.Info("signal received", "signal", sig)
-
-		if s.cancelFunc != nil {
-			s.cancelFunc()
-		}
-	}()
-
-	go s.serveUntilShutdown()
-
-	<-ctx.Done()
-	s.cleanShutdown(ctx)
-
-	return s.exitCode
-}
-
-func (s *Server) cleanShutdown(ctx context.Context) {
-	s.exitCode = 0
-
-	ctxShutdown, shutdownCancel := context.WithTimeout(
-		context.WithoutCancel(ctx), ShutdownTimeout)
 	defer shutdownCancel()
 
-	if s.httpServer != nil {
+	err := s.httpServer.Shutdown(ctxShutdown)
-		err := s.httpServer.Shutdown(ctxShutdown)
+	if err != nil {
-		if err != nil {
+		s.log.Error("server clean shutdown failed", "error", err)
-			s.log.Error("server clean shutdown failed", "error", err)
-		}
 	}
 
+	stillProcessing := s.h.WaitForProcessing(ctxShutdown)
+
 	if s.sentryEnabled {
 		sentry.Flush(SentryFlushTimeout)
 	}
+
+	if stillProcessing > 0 {
+		s.log.Error("images still being processed at shutdown",
+			"count", stillProcessing)
+
+		return errStillProcessing
+	}
+
+	return nil
 }

internal/server/shutdown_internal_test.go

@@ -0,0 +1,96 @@
+package server
+
+import (
+	"log/slog"
+	"net"
+	"testing"
+	"time"
+
+	"go.uber.org/fx"
+	"go.uber.org/fx/fxtest"
+
+	"sneak.berlin/go/pixa/internal/config"
+	"sneak.berlin/go/pixa/internal/globals"
+	"sneak.berlin/go/pixa/internal/logger"
+)
+
+// shutdownRecorder is an fx.Shutdowner that sends the options of each
+// shutdown request on requests.
+type shutdownRecorder struct {
+	requests chan []fx.ShutdownOption
+}
+
+func (r shutdownRecorder) Shutdown(opts ...fx.ShutdownOption) error {
+	r.requests <- opts
+
+	return nil
+}
+
+// TestSentryInitFailureFailsStartup checks that a Sentry DSN that cannot be
+// used makes the server's start hook fail, so fx stops what has already
+// started, instead of the process exiting from a goroutine.
+func TestSentryInitFailureFailsStartup(t *testing.T) {
+	t.Parallel()
+
+	lc := fxtest.NewLifecycle(t)
+
+	log, err := logger.New(lc, logger.Params{Globals: &globals.Globals{}})
+	if err != nil {
+		t.Fatalf("logger.New() error = %v", err)
+	}
+
+	_, err = New(lc, Params{
+		Logger:  log,
+		Globals: &globals.Globals{Appname: "pixad"},
+		Config:  &config.Config{SentryDSN: "not-a-dsn"},
+	})
+	if err != nil {
+		t.Fatalf("New() error = %v", err)
+	}
+
+	err = lc.Start(t.Context())
+	t.Logf("Start() error = %v", err)
+
+	if err == nil {
+		t.Fatal("Start() error = nil, want the Sentry initialization error")
+	}
+}
+
+// TestListenErrorRequestsShutdownWithExitCode1 occupies the server's port
+// and checks that the listen error asks fx to shut down with exit code 1.
+func TestListenErrorRequestsShutdownWithExitCode1(t *testing.T) {
+	t.Parallel()
+
+	busy, err := (&net.ListenConfig{}).Listen(t.Context(), "tcp", ":0")
+	if err != nil {
+		t.Fatalf("Listen() error = %v", err)
+	}
+
+	t.Cleanup(func() { _ = busy.Close() })
+
+	addr, ok := busy.Addr().(*net.TCPAddr)
+	if !ok {
+		t.Fatalf("listener address %v is not a TCP address", busy.Addr())
+	}
+
+	requests := make(chan []fx.ShutdownOption, 1)
+	s := &Server{
+		log:        slog.New(slog.DiscardHandler),
+		config:     &config.Config{Port: addr.Port},
+		shutdowner: shutdownRecorder{requests: requests},
+	}
+	s.httpServer = s.newHTTPServer()
+
+	go s.serveUntilShutdown()
+
+	select {
+	case opts := <-requests:
+		t.Logf("shutdown options = %v", opts)
+
+		if len(opts) != 1 || opts[0] != fx.ExitCode(1) {
+			t.Errorf("shutdown options = %v, want [fx.ExitCode(1)]", opts)
+		}
+	case <-time.After(5 * time.Second):
+		t.Fatal("no shutdown was requested after the listen error")
+	}
+}

internal/static/generator.js

@@ -0,0 +1,10 @@
+// Generator page: a click on the generated URL selects it, and the Copy
+// button copies it. Both are on the page only once a URL has been generated.
+const generatedURL = document.getElementById("generated-url");
+
+if (generatedURL) {
+    generatedURL.addEventListener("click", () => generatedURL.select());
+    document.getElementById("copy-url").addEventListener("click", () => {
+        navigator.clipboard.writeText(generatedURL.value);
+    });
+}

internal/static/static.go

@@ -7,7 +7,7 @@ import (
 	"net/http"
 )
 
-//go:embed *.js
+//go:embed *.css *.js
 var files embed.FS
 
 // FS returns the embedded filesystem containing static files.

internal/static/style.css

@@ -0,0 +1,190 @@
+/* The login and generator pages. */
+
+* {
+    box-sizing: border-box;
+}
+
+body {
+    margin: 0;
+    min-height: 100vh;
+    background: #f3f4f6;
+    font-family: system-ui, sans-serif;
+    line-height: 1.5;
+}
+
+h1 {
+    margin: 0;
+    font-size: 1.5rem;
+    line-height: 2rem;
+    font-weight: 700;
+    color: #1f2937;
+}
+
+label {
+    display: block;
+    margin-bottom: 0.25rem;
+    font-size: 0.875rem;
+    font-weight: 500;
+    color: #374151;
+}
+
+input,
+select {
+    width: 100%;
+    padding: 0.5rem 0.75rem;
+    border: 1px solid #d1d5db;
+    border-radius: 0.375rem;
+    box-shadow: 0 1px 2px rgb(0 0 0 / 5%);
+    font: inherit;
+}
+
+input:focus,
+select:focus {
+    outline: none;
+    border-color: #3b82f6;
+    box-shadow: 0 0 0 2px #3b82f6;
+}
+
+button {
+    width: 100%;
+    padding: 0.5rem 1rem;
+    border: none;
+    border-radius: 0.375rem;
+    background: #2563eb;
+    color: #fff;
+    font: inherit;
+    cursor: pointer;
+    transition: background-color 0.15s;
+}
+
+button:hover {
+    background: #1d4ed8;
+}
+
+button:focus {
+    outline: 2px solid #3b82f6;
+    outline-offset: 2px;
+}
+
+form > * + * {
+    margin-top: 1rem;
+}
+
+.card {
+    padding: 1.5rem;
+    border-radius: 0.5rem;
+    background: #fff;
+    box-shadow:
+        0 4px 6px -1px rgb(0 0 0 / 10%),
+        0 2px 4px -2px rgb(0 0 0 / 10%);
+}
+
+.error {
+    margin-bottom: 1rem;
+    padding: 0.75rem 1rem;
+    border: 1px solid #f87171;
+    border-radius: 0.25rem;
+    background: #fee2e2;
+    color: #b91c1c;
+}
+
+/* Login page: the card centred on the screen. */
+
+.login {
+    display: flex;
+    align-items: center;
+    justify-content: center;
+}
+
+.login .card {
+    width: 100%;
+    max-width: 28rem;
+    padding: 2rem;
+}
+
+.login h1 {
+    margin-bottom: 1.5rem;
+    text-align: center;
+}
+
+/* Generator page. */
+
+.page {
+    max-width: 42rem;
+    margin: 0 auto;
+    padding: 2rem 1rem;
+}
+
+header {
+    display: flex;
+    justify-content: space-between;
+    align-items: center;
+    margin-bottom: 2rem;
+}
+
+header a {
+    font-size: 0.875rem;
+    color: #4b5563;
+}
+
+header a:hover {
+    color: #1f2937;
+}
+
+.result {
+    margin-bottom: 1.5rem;
+    padding: 1rem;
+    border: 1px solid #bbf7d0;
+    border-radius: 0.5rem;
+    background: #f0fdf4;
+}
+
+.result h2 {
+    margin: 0 0 0.5rem;
+    font-size: 0.875rem;
+    font-weight: 500;
+    color: #166534;
+}
+
+.result div {
+    display: flex;
+    gap: 0.5rem;
+}
+
+.result input {
+    flex: 1;
+    border-color: #86efac;
+    box-shadow: none;
+    font-family: ui-monospace, monospace;
+    font-size: 0.875rem;
+}
+
+.result button {
+    width: auto;
+    padding: 0.5rem 0.75rem;
+    background: #16a34a;
+    font-size: 0.875rem;
+}
+
+.result button:hover {
+    background: #15803d;
+}
+
+.result p {
+    margin: 0.5rem 0 0;
+    font-size: 0.75rem;
+    color: #16a34a;
+}
+
+.columns {
+    display: grid;
+    grid-template-columns: repeat(2, minmax(0, 1fr));
+    gap: 1rem;
+}
+
+.note {
+    margin-top: 1rem;
+    font-size: 0.75rem;
+    color: #6b7280;
+    text-align: center;
+}

internal/templates/generator.html

@@ -4,52 +4,47 @@
     <meta charset="UTF-8">
     <meta name="viewport" content="width=device-width, initial-scale=1.0">
     <title>Pixa - URL Generator</title>
-    <script src="/static/tailwind.js"></script>
+    <link rel="stylesheet" href="/static/style.css">
 </head>
-<body class="bg-gray-100 min-h-screen">
+<body>
-    <div class="max-w-2xl mx-auto py-8 px-4">
+    <div class="page">
-        <div class="flex justify-between items-center mb-8">
+        <header>
-            <h1 class="text-2xl font-bold text-gray-800">Pixa URL Generator</h1>
+            <h1>Pixa URL Generator</h1>
-            <a href="/logout" class="text-sm text-gray-600 hover:text-gray-800 underline">
+            <a href="/logout">
                 Logout
             </a>
-        </div>
+        </header>
 
         {{if .GeneratedURL}}
-        <div class="bg-green-50 border border-green-200 rounded-lg p-4 mb-6">
+        <div class="result">
-            <h2 class="text-sm font-medium text-green-800 mb-2">Generated URL</h2>
+            <h2>Generated URL</h2>
-            <div class="flex gap-2">
+            <div>
                 <input
                     type="text"
                     readonly
                     value="{{.GeneratedURL}}"
                     id="generated-url"
-                    class="flex-1 px-3 py-2 bg-white border border-green-300 rounded-md text-sm font-mono"
-                    onclick="this.select()"
-                >
-                <button
-                    onclick="navigator.clipboard.writeText(document.getElementById('generated-url').value)"
-                    class="px-3 py-2 bg-green-600 text-white rounded-md hover:bg-green-700 text-sm"
                 >
+                <button id="copy-url">
                     Copy
                 </button>
             </div>
-            <p class="text-xs text-green-600 mt-2">
+            <p>
                 Expires: {{.ExpiresAt}}
             </p>
         </div>
         {{end}}
 
         {{if .Error}}
-        <div class="bg-red-100 border border-red-400 text-red-700 px-4 py-3 rounded mb-6">
+        <div class="error">
             {{.Error}}
         </div>
         {{end}}
 
-        <form method="POST" action="/generate" class="bg-white rounded-lg shadow-md p-6 space-y-4">
+        <form method="POST" action="/generate" class="card">
             {{ .CSRFField }}
             <div>
-                <label for="url" class="block text-sm font-medium text-gray-700 mb-1">
+                <label for="url">
                     Source URL
                 </label>
                 <input
@@ -59,13 +54,12 @@
                     required
                     placeholder="https://example.com/image.jpg"
                     value="{{.FormURL}}"
-                    class="w-full px-3 py-2 border border-gray-300 rounded-md shadow-sm focus:outline-none focus:ring-2 focus:ring-blue-500 focus:border-blue-500"
                 >
             </div>
 
-            <div class="grid grid-cols-2 gap-4">
+            <div class="columns">
                 <div>
-                    <label for="width" class="block text-sm font-medium text-gray-700 mb-1">
+                    <label for="width">
                         Width
                     </label>
                     <input
@@ -76,11 +70,10 @@
                         max="8192"
                         value="{{if .FormWidth}}{{.FormWidth}}{{else}}0{{end}}"
                         placeholder="0 = original"
-                        class="w-full px-3 py-2 border border-gray-300 rounded-md shadow-sm focus:outline-none focus:ring-2 focus:ring-blue-500 focus:border-blue-500"
                     >
                 </div>
                 <div>
-                    <label for="height" class="block text-sm font-medium text-gray-700 mb-1">
+                    <label for="height">
                         Height
                     </label>
                     <input
@@ -91,21 +84,16 @@
                         max="8192"
                         value="{{if .FormHeight}}{{.FormHeight}}{{else}}0{{end}}"
                         placeholder="0 = original"
-                        class="w-full px-3 py-2 border border-gray-300 rounded-md shadow-sm focus:outline-none focus:ring-2 focus:ring-blue-500 focus:border-blue-500"
                     >
                 </div>
             </div>
 
-            <div class="grid grid-cols-2 gap-4">
+            <div class="columns">
                 <div>
-                    <label for="format" class="block text-sm font-medium text-gray-700 mb-1">
+                    <label for="format">
                         Format
                     </label>
-                    <select
+                    <select id="format" name="format">
-                        id="format"
-                        name="format"
-                        class="w-full px-3 py-2 border border-gray-300 rounded-md shadow-sm focus:outline-none focus:ring-2 focus:ring-blue-500 focus:border-blue-500"
-                    >
                         <option value="orig" {{if eq .FormFormat "orig"}}selected{{end}}>Original</option>
                         <option value="jpeg" {{if eq .FormFormat "jpeg"}}selected{{end}}>JPEG</option>
                         <option value="png" {{if eq .FormFormat "png"}}selected{{end}}>PNG</option>
@@ -115,14 +103,10 @@
                     </select>
                 </div>
                 <div>
-                    <label for="quality" class="block text-sm font-medium text-gray-700 mb-1">
+                    <label for="quality">
                         Quality
                     </label>
-                    <select
+                    <select id="quality" name="quality">
-                        id="quality"
-                        name="quality"
-                        class="w-full px-3 py-2 border border-gray-300 rounded-md shadow-sm focus:outline-none focus:ring-2 focus:ring-blue-500 focus:border-blue-500"
-                    >
                         <option value="25" {{if eq .FormQuality "25"}}selected{{end}}>Potato</option>
                         <option value="50" {{if eq .FormQuality "50"}}selected{{end}}>Low</option>
                         <option value="70" {{if eq .FormQuality "70"}}selected{{end}}>Medium</option>
@@ -132,16 +116,12 @@
                 </div>
             </div>
 
-            <div class="grid grid-cols-2 gap-4">
+            <div class="columns">
                 <div>
-                    <label for="fit" class="block text-sm font-medium text-gray-700 mb-1">
+                    <label for="fit">
                         Fit Mode
                     </label>
-                    <select
+                    <select id="fit" name="fit">
-                        id="fit"
-                        name="fit"
-                        class="w-full px-3 py-2 border border-gray-300 rounded-md shadow-sm focus:outline-none focus:ring-2 focus:ring-blue-500 focus:border-blue-500"
-                    >
                         <option value="cover" {{if eq .FormFit "cover"}}selected{{end}}>Cover</option>
                         <option value="contain" {{if eq .FormFit "contain"}}selected{{end}}>Contain</option>
                         <option value="fill" {{if eq .FormFit "fill"}}selected{{end}}>Fill</option>
@@ -150,14 +130,10 @@
                     </select>
                 </div>
                 <div>
-                    <label for="ttl" class="block text-sm font-medium text-gray-700 mb-1">
+                    <label for="ttl">
                         Expires In
                     </label>
-                    <select
+                    <select id="ttl" name="ttl">
-                        id="ttl"
-                        name="ttl"
-                        class="w-full px-3 py-2 border border-gray-300 rounded-md shadow-sm focus:outline-none focus:ring-2 focus:ring-blue-500 focus:border-blue-500"
-                    >
                         <option value="0" {{if or (eq .FormTTL "0") (eq .FormTTL "")}}selected{{end}}>Never</option>
                         <option value="60" {{if eq .FormTTL "60"}}selected{{end}}>1 minute</option>
                         <option value="3600" {{if eq .FormTTL "3600"}}selected{{end}}>1 hour</option>
@@ -169,17 +145,15 @@
                 </div>
             </div>
 
-            <button
+            <button type="submit">
-                type="submit"
-                class="w-full bg-blue-600 text-white py-2 px-4 rounded-md hover:bg-blue-700 focus:outline-none focus:ring-2 focus:ring-blue-500 focus:ring-offset-2 transition-colors"
-            >
                 Generate Encrypted URL
             </button>
         </form>
 
-        <p class="text-xs text-gray-500 mt-4 text-center">
+        <p class="note">
             Generated URLs are encrypted and cannot be modified. They will expire at the specified time.
         </p>
     </div>
+    <script src="/static/generator.js"></script>
 </body>
 </html>

internal/templates/login.html

@@ -4,22 +4,22 @@
     <meta charset="UTF-8">
     <meta name="viewport" content="width=device-width, initial-scale=1.0">
     <title>Pixa - Login</title>
-    <script src="/static/tailwind.js"></script>
+    <link rel="stylesheet" href="/static/style.css">
 </head>
-<body class="bg-gray-100 min-h-screen flex items-center justify-center">
+<body class="login">
-    <div class="bg-white p-8 rounded-lg shadow-md w-full max-w-md">
+    <div class="card">
-        <h1 class="text-2xl font-bold text-gray-800 mb-6 text-center">Pixa Image Proxy</h1>
+        <h1>Pixa Image Proxy</h1>
 
         {{if .Error}}
-        <div class="bg-red-100 border border-red-400 text-red-700 px-4 py-3 rounded mb-4">
+        <div class="error">
             {{.Error}}
         </div>
         {{end}}
 
-        <form method="POST" action="/" class="space-y-4">
+        <form method="POST" action="/">
             {{ .CSRFField }}
             <div>
-                <label for="key" class="block text-sm font-medium text-gray-700 mb-1">
+                <label for="key">
                     Signing Key
                 </label>
                 <input
@@ -28,15 +28,11 @@
                     name="key"
                     required
                     autocomplete="current-password"
-                    class="w-full px-3 py-2 border border-gray-300 rounded-md shadow-sm focus:outline-none focus:ring-2 focus:ring-blue-500 focus:border-blue-500"
                     placeholder="Enter your signing key"
                 >
             </div>
 
-            <button
+            <button type="submit">
-                type="submit"
-                class="w-full bg-blue-600 text-white py-2 px-4 rounded-md hover:bg-blue-700 focus:outline-none focus:ring-2 focus:ring-blue-500 focus:ring-offset-2 transition-colors"
-            >
                 Login
             </button>
         </form>

script/cibuild

@@ -1,15 +1,18 @@
 #!/bin/sh
 # script/cibuild: run the CI build. The Dockerfile runs the checks
-# (make fmt-check, lint, test), so a successful build implies a green
+# (make fmt-check, lint, test) as build steps. This script passes a new
-# repo. Generic: needs no adaptation. The Gitea workflow runs this on
+# CHECK_EPOCH on every run, so Docker runs those steps instead of
-# push.
+# reusing cached results: a successful run means the checks ran and
+# passed on this tree. Generic: needs no adaptation. The Gitea workflow
+# runs this on push.
 set -eu
 
 ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
 
 main() {
     cd "$ROOT"
-    docker build .
+    epoch="$(date +%s)$$"
+    docker build --build-arg CHECK_EPOCH="$epoch" .
 }
 
 main "$@"

script/docker

@@ -1,7 +1,9 @@
 #!/bin/sh
 # script/docker: build the Docker image tagged with the project name.
-# Identical in all repos; the tag comes from script/projectname.
+# Identical in all repos; the tag comes from script/projectname. Like
-# Generic: needs no adaptation.
+# script/cibuild, it passes a new CHECK_EPOCH, so the build runs the
+# checks instead of reusing cached results. Generic: needs no
+# adaptation.
 set -eu
 
 SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
@@ -9,7 +11,9 @@ ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
 
 main() {
     cd "$ROOT"
-    docker build -t "$("$SCRIPT_DIR/projectname")" .
+    epoch="$(date +%s)$$"
+    docker build --build-arg CHECK_EPOCH="$epoch" \
+        -t "$("$SCRIPT_DIR/projectname")" .
 }
 
 main "$@"

scripts/manual-test.sh

@@ -1,147 +0,0 @@
-#!/bin/bash
-#
-# Manual test script for pixa server
-# Requires: server running on localhost:8080
-#
-set -e
-
-BASE_URL="${BASE_URL:-http://localhost:8080}"
-SIGNING_KEY="${SIGNING_KEY:-test-signing-key-for-development-only}"
-TEST_IMAGE_URL="https://s3.sneak.cloud/sneak-public/2021/2021-04-18.untitled.a7r4.07723.jpg"
-COOKIE_JAR=$(mktemp)
-
-cleanup() {
-    rm -f "$COOKIE_JAR"
-}
-trap cleanup EXIT
-
-pass() {
-    echo "✓ PASS: $1"
-}
-
-fail() {
-    echo "✗ FAIL: $1"
-    exit 1
-}
-
-echo "=== Pixa Manual Test Suite ==="
-echo "Base URL: $BASE_URL"
-echo ""
-
-# Test 1: Healthcheck
-echo "--- Test 1: Healthcheck endpoint ---"
-HEALTH=$(curl -sf "$BASE_URL/.well-known/healthcheck.json")
-if echo "$HEALTH" | grep -q '"status"'; then
-    pass "Healthcheck returns status"
-else
-    fail "Healthcheck did not return expected response"
-fi
-
-# Test 2: Login page displays
-echo "--- Test 2: Login page (GET /) ---"
-LOGIN_PAGE=$(curl -sf "$BASE_URL/")
-if echo "$LOGIN_PAGE" | grep -qi "password\|login\|sign"; then
-    pass "Login page displays password form"
-else
-    fail "Login page did not display expected content"
-fi
-
-# Test 3: Wrong password shows error
-echo "--- Test 3: Login with wrong password ---"
-WRONG_LOGIN=$(curl -sf -X POST "$BASE_URL/" -d "key=wrong-key" -c "$COOKIE_JAR")
-if echo "$WRONG_LOGIN" | grep -qi "invalid\|error\|incorrect\|wrong"; then
-    pass "Wrong password shows error message"
-else
-    fail "Wrong password did not show error"
-fi
-
-# Test 4: Correct password redirects to generator
-echo "--- Test 4: Login with correct signing key ---"
-curl -sf -X POST "$BASE_URL/" -d "key=$SIGNING_KEY" -c "$COOKIE_JAR" -b "$COOKIE_JAR" -L -o /dev/null
-GENERATOR_PAGE=$(curl -sf "$BASE_URL/" -b "$COOKIE_JAR")
-if echo "$GENERATOR_PAGE" | grep -qi "generate\|url\|source\|logout"; then
-    pass "Correct password shows generator page"
-else
-    fail "Generator page not displayed after login"
-fi
-
-# Test 5: Generate encrypted URL
-echo "--- Test 5: Generate encrypted URL ---"
-GEN_RESULT=$(curl -sf -X POST "$BASE_URL/generate" -b "$COOKIE_JAR" \
-    -d "url=$TEST_IMAGE_URL" \
-    -d "width=800" \
-    -d "height=600" \
-    -d "format=jpeg" \
-    -d "quality=85" \
-    -d "fit=cover" \
-    -d "ttl=3600")
-if echo "$GEN_RESULT" | grep -q "/v1/e/"; then
-    pass "Encrypted URL generated"
-    # Extract the encrypted URL
-    ENC_URL=$(echo "$GEN_RESULT" | grep -o '/v1/e/[^"<]*' | head -1)
-    echo "  Generated URL: $ENC_URL"
-else
-    fail "Failed to generate encrypted URL"
-fi
-
-# Test 6: Fetch image via encrypted URL
-echo "--- Test 6: Fetch image via encrypted URL ---"
-if [ -n "$ENC_URL" ]; then
-    HTTP_CODE=$(curl -sf -o /dev/null -w "%{http_code}" "$BASE_URL$ENC_URL")
-    if [ "$HTTP_CODE" = "200" ]; then
-        pass "Encrypted URL returns image (HTTP 200)"
-    else
-        fail "Encrypted URL returned HTTP $HTTP_CODE"
-    fi
-else
-    fail "No encrypted URL to test"
-fi
-
-# Test 7: Fetch image via allowlisted host (direct proxy)
-echo "--- Test 7: Fetch image via direct proxy (allowlisted host) ---"
-# URL format: /v1/image/<host>/<path>/<WxH>.<format>
-PROXY_PATH="/v1/image/s3.sneak.cloud/sneak-public/2021/2021-04-18.untitled.a7r4.07723.jpg/400x300.jpeg"
-HTTP_CODE=$(curl -sf -o /dev/null -w "%{http_code}" "$BASE_URL$PROXY_PATH")
-if [ "$HTTP_CODE" = "200" ]; then
-    pass "Direct proxy returns image (HTTP 200)"
-else
-    fail "Direct proxy returned HTTP $HTTP_CODE"
-fi
-
-# Test 8: Logout
-echo "--- Test 8: Logout ---"
-curl -sf "$BASE_URL/logout" -b "$COOKIE_JAR" -c "$COOKIE_JAR" -L -o /dev/null
-AFTER_LOGOUT=$(curl -sf "$BASE_URL/" -b "$COOKIE_JAR")
-if echo "$AFTER_LOGOUT" | grep -qi "password\|login"; then
-    pass "Logout redirects to login page"
-else
-    fail "Logout did not redirect to login"
-fi
-
-# Test 9: Generate short-TTL URL and verify expiration
-echo "--- Test 9: Expired URL returns 410 ---"
-# Login again
-curl -sf -X POST "$BASE_URL/" -d "key=$SIGNING_KEY" -c "$COOKIE_JAR" -b "$COOKIE_JAR" -L -o /dev/null
-# Generate URL with 1 second TTL
-GEN_RESULT=$(curl -sf -X POST "$BASE_URL/generate" -b "$COOKIE_JAR" \
-    -d "url=$TEST_IMAGE_URL" \
-    -d "width=100" \
-    -d "height=100" \
-    -d "format=jpeg" \
-    -d "ttl=1")
-SHORT_URL=$(echo "$GEN_RESULT" | grep -o '/v1/e/[^"<]*' | head -1)
-if [ -n "$SHORT_URL" ]; then
-    echo "  Waiting 2 seconds for URL to expire..."
-    sleep 2
-    HTTP_CODE=$(curl -s -o /dev/null -w "%{http_code}" "$BASE_URL$SHORT_URL")
-    if [ "$HTTP_CODE" = "410" ]; then
-        pass "Expired URL returns 410 Gone"
-    else
-        fail "Expired URL returned HTTP $HTTP_CODE (expected 410)"
-    fi
-else
-    fail "Could not generate short-TTL URL"
-fi
-
-echo ""
-echo "=== All tests passed! ==="