One image request type instead of five (closes #307)

The new package internal/imagerequest holds the request type, its two
parts Source and Transform, the Format and Fit types, the dimension and
fit checks, the /v1/image/ path parsing and the variant key; it imports
no other package of pixa. imgcache.ParsedURL, imgcache.ImageRequest,
signature.Request, imageprocessor.Request, the second Size, format and
fit types and the conversions between them are gone. The image
processor takes a Transform only. encurl keeps a private struct with the
token's CBOR field names, so tokens made before still decode. VariantKey
moves with the function that makes it. Tests follow the moved code; the
tests of the removed conversions go with them.

Model: opus-5-5

Makefile

@@ -1,4 +1,4 @@
-.PHONY: bootstrap setup check lint test fmt fmt-check build clean docker docker-smoke docker-versioned docker-test devserver devserver-stop hooks loadtest bench
+.PHONY: bootstrap setup check lint test fmt fmt-check build clean docker docker-smoke docker-versioned docker-test devserver devserver-stop hooks loadtest
 
 VERSION := $(shell git describe --tags --always --dirty 2>/dev/null || echo "dev")
 LDFLAGS := -X main.Version=$(VERSION)
@@ -64,12 +64,6 @@ docker-smoke:
 loadtest:
 	@script/loadtest
 
-# Serve cache hits in parallel through the image handler and report hits per
-# second (needs Go on the host after `script/bootstrap --cgo`, not Docker; a
-# benchmark, not part of check)
-bench:
-	@script/bench
-
 # Build Docker image as `make docker` does, and also tag it pixa:$(VERSION)
 docker-versioned:
 	@script/docker

README.md

@@ -76,7 +76,8 @@ database and the disk cache:
   pixa runs as (uid 65532) and sets its mode to `750`; outside it, that user
   must be able to write the directory.
 - `cache_max_bytes` limits the source and transformed images together. The
-  database and files still being written come on top, and eviction runs in the
+  database, the metadata files, the `.meta` file beside each transformed image
+  and files still being written come on top, and eviction runs in the
   background, so the cache can pass the limit for a while: leave room on the
   volume beyond it.
 - Set `cache_max_bytes` for a lasting deployment. Its default, worked out each
@@ -88,17 +89,13 @@ A load balancer's health check can request `/.well-known/healthcheck.json`,
 which answers 200 whenever pixa is running, in maintenance mode too (see
 `maintenance_mode`).
 
-On SIGTERM or SIGINT pixa stops accepting connections. The requests in progress
+On SIGTERM or SIGINT pixa stops accepting connections, gives the requests in
-and the images being processed get 5 seconds to finish. A request not finished
+progress and the images being processed 5 seconds to finish, then writes to the
-by then is cut off. pixa then writes the waiting last-used times of cached
+database the counts of cache hits, misses, fetches and conversions that requests
-images to the database (see Storage) and exits. It exits with 1 when:
+could not write by their deadline, and exits: with 0, or with 1 when images were
-
+still being processed after those 5 seconds, some of those counts could not be
-- images were still being processed after those 5 seconds,
+written, or another part of pixa failed to stop. A request not finished by then
-- some of those times could not be written, or
+is cut off. `docker stop` waits 10 seconds before it kills the container.
-- another part of pixa failed to stop.
-
-Otherwise it exits with 0. `docker stop` waits 10 seconds before it kills the
-container.
 
 Outside Docker, pixa needs libvips (the image has 8.16) and libheif to run, as
 it uses libvips through CGO. pixad does not start unless libvips has its JPEG XL
@@ -180,46 +177,38 @@ allowlisted source hosts for open access.
 
 ### Storage
 
-- **Source images**:
+- **Source content**:
-  `<state_dir>/cache/sources/<ab>/<cd>/<sha256 of host, path and query>`, one
+  `<state_dir>/cache/sources/<ab>/<cd>/<sha256 of source content>`
-  file for each source URL
+- **Source metadata**:
+  `<state_dir>/cache/metadata/<hostname>/<sha256 of path and query>.json` (host,
+  path and query, content hash, upstream status and headers, fetch time)
 - **Database**: `<state_dir>/state.sqlite3` (SQLite)
 - **Transformed images**:
-  `<state_dir>/cache/variants/<ab>/<cd>/<sha256 of host, path, query, size, format, quality and fit>`
+  `<state_dir>/cache/variants/<ab>/<cd>/<sha256 of host, path, query, size, format, quality and fit>`,
+  each with a `.meta` file beside it holding its content type
 
 `<ab>` and `<cd>` are the first and second pairs of characters of the file's
 name.
 
-Each source image and each transformed image has one row in the database,
+Multiple source paths may reference the same content blob; the database tracks
-holding its size and when it was last used. Two source URLs that serve the same
+references rather than using filesystem refcounting.
-bytes are stored twice, once under each URL. A source image is stored by writing
-its file and then its row; when the row cannot be written, the file is removed
-again.
 
-Source and transformed image files are written under a temporary name, synced to
+Source and transformed image files, and the `.meta` file beside each transformed
-disk, and only then renamed into place, so a crash leaves no file under its own
+image, are written under a temporary name, synced to disk, and only then renamed
-name with part of its data. When pixa starts, and every five minutes after, it
+into place, so a crash leaves no file under its own name with part of its data.
-removes each cached source or transformed image whose file is empty, or not the
+When pixa starts, and every five minutes after, it removes each cached source or
-size the database records for it, with a warning naming the file; the next
+transformed image whose file is empty, or not the size the database records for
-request for it fetches the source or converts the image again. Looking for those
+it, with a warning naming the file; the next request for it fetches the source
-files reads the database once for every cached file, so pixa skips each
+or converts the image again. Looking for those files reads the database once for
-five-minute look that would start less than ten times as long after the last
+every cached file, so pixa skips each five-minute look that would start less
-look ended as that look took. After a look that took one minute, the next one
+than ten times as long after the last look ended as that look took. After a look
-starts 10 to 15 minutes after that look ended.
+that took one minute, the next one starts 10 to 15 minutes after that look
+ended.
 
 pixa's target is 1-5k r/s, which has not been measured at that rate (see Load
-Test). Toward it, a cache hit opens the transformed image's file and nothing
+Test). Toward it, pixa keeps in memory the content types of the 10,000
-else, and makes no database call. The content type comes from the file's first
+transformed images most recently cached or served, so a cache hit on one of them
-bytes, which are those of the format pixa wrote it in; a file whose first bytes
+reads only the image file from disk and not the metadata file stored beside it.
-are not those of a format pixa writes is not served, and the image is converted
-again. A hit hands the time it was served to a background writer and does not
-wait for the writer. Eviction orders by that time. The hit is counted in memory.
-A miss that uses a cached source image also hands that writer the time it used
-the image. Once a second that writer writes the waiting times in one
-transaction. While 10,000 times of transformed images, or 10,000 of source
-images, wait to be written, a further one is dropped, and that image only looks
-less recently used to eviction. `script/bench` measures how many hits pixa
-serves per second (see Load Test).
 
 ### Routes
 
@@ -248,58 +237,32 @@ path under `/v1/` answers 200, in maintenance mode too.
 - `GET` or `HEAD` `/v1/image/<host>/<path>/<size>.<format>`, or
   `/v1/image/<host>/<path>/<size>` with no format — an image, fetched, resized
   and converted (below). Needs: a signature, unless the host is allowlisted (see
-  Source Hosts). Answers:
+  Source Hosts). Answers: 200; 304 when `If-None-Match` matches the image's
-    - 200 with the image.
+  `ETag`; 400 for a URL or parameter that is not valid, or for the format `auto`
-    - 304 when `If-None-Match` matches the image's `ETag`.
+  an `Accept` header that is not valid; 406 for the format `auto` when `Accept`
-    - 400 for a URL or parameter that is not valid.
+  allows none of the formats it chooses from; 401 for a missing or wrong
-    - 400 when the format is `auto` and the `Accept` header is not valid.
+  signature, a missing `exp` or an `exp` in the past; 403 when the request's
-    - 406 when the format is `auto` and `Accept` allows none of the formats it
+  `Referer` names a host in `referer_blocklist`, checked before the signature,
-      chooses from.
+  the cache and the upstream fetch; 403 when the upstream host, or a host it
-    - 401 for a missing or wrong signature.
+  redirects to, is `localhost`, ends in `.localhost` or `.local`, or has an
-    - 401 for a missing `exp`.
+  address in a blocked network (see `blocked_networks`); 429 over
-    - 401 for an `exp` in the past.
+  `image_requests_per_minute` (below); 415 when the upstream answers with a
-    - 403 when the request's `Referer` names a host in `referer_blocklist`,
+  `Content-Type` that is not one of the source formats (below), with a body
-      checked before the signature, the cache and the upstream fetch.
+  whose first bytes are not those of the format its `Content-Type` names, or
-    - 403 when the upstream host, or a host it redirects to, is `localhost` or
+  with an image whose first bytes libvips reads as another format; 502 when the
-      ends in `.localhost` or `.local`.
+  upstream answered with an error status, and for 5 minutes after that, or after
-    - 403 when the upstream host, or a host it redirects to, has an address in a
+  a body whose first bytes are not those of its `Content-Type`, for an image of
-      blocked network (see `blocked_networks`).
+  the same source URL that pixa can neither serve from its cache nor make from
-    - 429 over `image_requests_per_minute` (below).
+  the source image still in its cache; 503 when pixa is busy or in maintenance
-    - 415 when the upstream answers with a `Content-Type` that is not one of the
+  mode; 500 for any other failure.
-      source formats (below).
-    - 415 when the upstream sends a body whose first bytes are not those of the
-      format its `Content-Type` names.
-    - 415 when the upstream sends an image whose first bytes libvips reads as
-      another format.
-    - 502 when the upstream answers with an error status.
-    - 502 for 5 minutes after that for the same source URL, when pixa has
-      neither the image nor its source image in its cache.
-    - 502 likewise for 5 minutes after a body whose first bytes are not those of
-      its `Content-Type`.
-    - 503 when pixa is busy.
-    - 503 when pixa is in maintenance mode.
-    - 504 when the upstream has not sent its response headers within
-      `upstream_fetch_timeout`.
-    - 500 when that time runs out while the image itself is still arriving.
-    - 500 for any other failure.
 - `GET` or `HEAD` `/v1/e/<token>/<name>` — an image through an encrypted URL
-  (see Encrypted URLs). Needs: nothing but the URL. Answers:
+  (see Encrypted URLs). Needs: nothing but the URL. Answers: 200; 304 when
-    - 200 with the image.
+  `If-None-Match` matches the image's `ETag`; 400 for a token that does not
-    - 304 when `If-None-Match` matches the image's `ETag`.
+  decrypt, or that asks for a size or fit that is not valid; 410 once it has
-    - 400 for a token that does not decrypt.
+  expired; 504 when the upstream has not sent its response headers within
-    - 400 for a token that asks for a size or fit that is not valid.
+  `upstream_fetch_timeout`, but 500 when that time runs out while the image
-    - 410 once the token has expired.
+  itself is still arriving; 400 for an `Accept` header that is not valid, and
-    - 400 when the token's format is `auto` and the `Accept` header is not
+  406, 403, 415, 429, 502, 503 and 500, as for `/v1/image/`.
-      valid.
-    - These answers, for the same causes as on `/v1/image/`:
-        - 406
-        - 403
-        - 415
-        - 429
-        - 502
-        - 503
-        - 504
-        - 500
 - `GET /robots.txt` — asks every crawler to stay away (`Disallow: /`). Needs:
   nothing. Answers: 200.
 - `GET /.well-known/healthcheck.json` — JSON with `status` (`ok`), `now`,
@@ -791,17 +754,7 @@ Key settings in more detail:
   duration; default `60s`. The upstream fetch counts toward it, and so do the
   waits for an upstream connection and for a processing slot (up to 10 seconds
   each), so keep it longer than `upstream_fetch_timeout` plus 20 seconds
-- `signing_key` — the secret that protects signed URLs, encrypted URLs and
+- `signing_key` — HMAC secret for URL signatures
-  logins. pixa uses it as it is for two things:
-    - the HMAC key of signed URLs (see Signature Specification)
-    - the password of the login page
-
-    pixa derives every other key it uses from it at startup, with HKDF-SHA256:
-    - a key that signs the session cookie
-    - a key that encrypts the session cookie
-    - a key that encrypts the token of an encrypted URL
-    - a key that signs the CSRF token cookie of the login and generator pages
-
 - `db_url` — the SQLite database to open; omitted, it is
   `file:<state_dir>/state.sqlite3?_pragma=journal_mode(WAL)`, which keeps the
   database in WAL mode. `state_dir` is percent-encoded in it, so a `%`, `?` or
@@ -848,34 +801,20 @@ See `configs/config.example.yml` for all options with defaults.
 
 ### Architecture
 
-- **Dependency injection**: Uber fx. `internal/app` builds every part of pixad
+- **Dependency injection**: Uber fx
-  from the config in one fx graph. Each setting's default is declared once, in
-  `internal/config`.
 - **HTTP router**: go-chi
 - **Image processing**: govips (CGO wrapper for libvips)
 - **Database**: SQLite via modernc.org/sqlite
 - **Static assets**: embedded via `//go:embed`
-- **Metrics**: Prometheus, at `/metrics`. pixa serves `/metrics` and measures
+- **Metrics**: Prometheus, at `/metrics`: generic HTTP request metrics
-  HTTP requests only when `metrics.username` and `metrics.password` are set.
+  (duration, response size, requests in flight) and the Go runtime and process
-  `/metrics` holds:
+  metrics; requests are measured and `/metrics` is served only when
-    - the duration of HTTP requests
+  `metrics.username` and `metrics.password` are set. A request's `handler` label
-    - the size of HTTP responses
+  is the pattern of the route it matches, such as `/v1/image/*`, never its path.
-    - the HTTP requests in flight
+  Requests that match no route share the label `unmatched`: those the router
-    - the Go runtime and process metrics
+  answers 404 or 405, and CORS preflights. A request's `method` label is its
-    - `pixa_cache_hits_total`: image requests served from the cache
+  method when that is `GET`, `HEAD`, `POST`, `PUT`, `PATCH`, `DELETE`,
-    - `pixa_cache_misses_total`: image requests not served from the cache
+  `OPTIONS`, `CONNECT` or `TRACE`, and `other` for any other method
-    - `pixa_upstream_fetches_total`: fetches of a source image that read any
-      bytes
-    - `pixa_upstream_fetch_bytes_total`: the bytes those fetches read
-    - `pixa_transforms_total`: images converted
-
-    A request's `handler` label is the pattern of the route it matches, such as
-    `/v1/image/*`, never its path. Requests that match no route share the label
-    `unmatched`: those the router answers 404 or 405, and CORS preflights. A
-    request's `method` label is its method when that is `GET`, `HEAD`, `POST`,
-    `PUT`, `PATCH`, `DELETE`, `OPTIONS`, `CONNECT` or `TRACE`, and `other` for
-    any other method.
-
 - **Logging**: stdlib slog
 
 ## Entrypoints
@@ -907,9 +846,6 @@ provide:
 - `script/docker-smoke` — build the image, start it, wait for it to be healthy
 - `script/loadtest` — measure pixad's throughput, latency and peak memory; a
   benchmark, not part of `script/check` (see Load Test)
-- `script/bench` — run the Go benchmark that serves cache hits in parallel
-  through the image handler, with Go on the host after `script/bootstrap --cgo`;
-  a benchmark, not part of `script/check` (see Load Test)
 - `script/cibuild` — CI entrypoint: run `script/bootstrap` (without `--cgo`),
   then `script/check`, then build the image as `script/docker` does
 - `script/precommit` — pre-commit checks (`go mod tidy` guard, then
@@ -990,17 +926,6 @@ The numbers depend on the machine and on whatever else runs on it. The first
 measurement, made on a shared machine with few clients, is in `TODO.md`; it says
 nothing about the target.
 
-`script/bench` (or `make bench`) measures cache hits alone, without the network
-and with no pixad process: a Go benchmark in `internal/handlers` serves one
-cached image through the `/v1/image/` handler from as many goroutines at once as
-the machine has CPUs, and reports the hits served per second as `hits/s`. Each
-run starts with an empty cache and an empty database, a SQLite file of its own,
-and puts the image in the cache before it measures. The benchmark fails if
-serving the hits ran any statement on the database. Like the load test it is not
-part of `script/check`. It runs Go on the host, as the load test does, and
-compiles pixa there, so it needs the C compiler and libraries that
-`script/bootstrap --cgo` installs, but not Docker.
-
 ## TODO
 
 See [TODO.md](TODO.md) for the full prioritized task list.

TODO.md

@@ -30,154 +30,6 @@ P2: security: per-origin rate limiting
 
 # Completed Steps
 
-- 2026-10-26 both image routes serve an image with one function, `serveImage`,
-  and answer its errors with one, `respondImageError` (closes #316).
-  `HandleImage` parses the path and verifies the signature, and `HandleImageEnc`
-  decrypts the token. Each then calls `serveImage`.
-    - `handleImageError`, the encrypted route's own mapping, is gone.
-    - `/v1/image/` answers 504 when the upstream has not sent its response
-      headers within `upstream_fetch_timeout`, as `/v1/e/` did. It answered 500.
-    - `respondImageError` logs the image service's errors as "failed to get
-      image". Each answer gets the same level on `/v1/image/` and on `/v1/e/`:
-        - 500: error, as pixa's own failure
-        - 400: warn
-        - 403: warn
-        - 415: warn
-        - 502: warn
-        - 503: warn
-        - 504: warn
-    - Before, `/v1/image/` logged all of these answers at error level. `/v1/e/`
-      logged only its 500, as "image request failed", without the host and the
-      path.
-    - `respondImageError` does not log the 415 for a body refused for its first
-      bytes, on either route. The image service logs it as "source refused", at
-      warn level.
-    - The "image served" line carries the host, the path and `fetched_bytes` on
-      both routes. The debug line "encrypted image request" is gone.
-    - The debug line "failed to decrypt URL" is now "invalid encrypted URL". A
-      token that decrypts to a payload that does not decode also gets it.
-    - A failed write is logged once, as "failed to write response" at warn
-      level, since most often the client has gone away. It carries the
-      `request_id`, the host, the path, `served_bytes` and the error, and is not
-      also logged as "image served".
-- 2026-10-26 every key derived from the signing key is derived in one place
-  (closes #315): `seal.DeriveKeys` returns them as one `seal.Keys`, with their
-  salts declared beside it. `internal/providers` calls it once at startup. Each
-  key is derived as before, with `seal.DeriveKey` and the same salt.
-    - The session manager, the encrypted URL generator and the CSRF protection
-      each take their own key instead of the signing key.
-    - `session.NewManager`, `encurl.NewGenerator`, `newCSRFProtect` and
-      `handlers.New` return no error, as nothing in them can fail now.
-    - The signing key itself stays the HMAC key of signed URLs and the login
-      password.
-    - `README.md` says under `signing_key` what it protects.
-- 2026-10-26 the image request holds only what names the cached object (closes
-  #314): its source and its transform, which the variant key covers. The signed
-  URL format, the signing algorithm and every route's answers are unchanged.
-    - The `/v1/image/` handler checks `allowlist_hosts` and verifies the
-      signature itself, with `Signer.Verify(req, sig, expires)`.
-    - Each image handler keeps its URL's expiry as a value of its own, for
-      `Cache-Control`. `encurl`'s `Generate` takes the expiry and its `Parse`
-      returns it.
-    - The fetcher takes the source and builds its URL with `Source.URL`, over
-      http only when the fetcher's own `AllowHTTP` is set. `validateURL` takes
-      that URL, so no URL string is built and parsed again.
-    - `chooseAutoFormat` returns the format it chooses, and the handler puts it
-      into the request it parsed.
-    - The image service serves images through `Get`, and has `WaitForProcessing`
-      for shutdown.
-    - Removed:
-        - `Signature`, `Expires`, `AllowHTTP` and `SourceURL` of the image
-          request
-        - `Service.ValidateRequest`, and the service's signer, allowlist and
-          `AllowHTTP`
-        - `extractHost` of the fetcher and `urlToFSPath` of the mock fetcher
-- 2026-10-26 pixad's fx graph builds the image pipeline (closes #313).
-  `database.New` now opens the database, so the parts below can read it while
-  they are built; its stop hook still runs after the cache's. The cache's
-  `MaxBytes` is the one value for the disk cache limit, and 0 turns the disk
-  cache off. For an omitted `cache_max_bytes`, `internal/providers` works out
-  `config.DefaultCacheMaxBytes` before it builds the cache. `internal/app` and
-  the server tests build the parts below with `providers.Constructors()`; the
-  image proxy test replaces only the fetcher's dial function.
-    - Built by the providers in `internal/providers`, and given to
-      `handlers.New`:
-        - the cache
-        - the fetcher
-        - the processor
-        - the image service
-        - the session manager
-        - the encrypted URL generator
-    - Removed:
-        - `initImageService` and `handlers.Params.Fetcher`
-        - `CacheConfig.UseDefaultMaxBytes`, `CacheConfig.DisableDiskCache` and
-          `Cache.disabled`
-        - `httpfetcher.DefaultConfig()` and the fetcher's copies of setting
-          defaults
-        - `imageprocessor.DefaultMaxInputBytes`, and the processor's defaults
-          for a zero `MaxInputBytes` or `MaxConcurrentProcessing`
-- 2026-10-26 the code nothing in production called is gone (closes #312). The
-  handlers tests sign their image URLs with `generateSignedURL`, a test helper
-  that writes them as the removed generator did. Removed:
-    - `Service.Warm` and the `ImageCache` interface
-    - `Service.GenerateSignedURL` and `Signer.GenerateSignedURL`
-    - `signature.ParseParams`
-    - `imagerequest.ParseImagePath`
-    - `PeekAndValidate`, `IsSupportedMIMEType`, `ImageFormatToMIME` and
-      `FormatOriginal` of `internal/magic`
-    - `SupportedInputFormats` and `SupportedOutputFormats` of the image
-      processor
-    - `IsEmpty` and `Count` of the allowlist
-    - `ssrfSafeDialer`
-    - `CacheStale`, `ErrCacheMiss`, `ErrNegativeCache` and
-      `ImageResponse.LastModified`
-    - `ContentStorage.Load` and `VariantStorage.Load`
-- 2026-10-26 cache hits, misses, upstream fetches, the bytes those fetches read
-  and transforms are counted as Prometheus counters in memory (closes #311).
-  They are `pixa_cache_hits_total`, `pixa_cache_misses_total`,
-  `pixa_upstream_fetches_total`, `pixa_upstream_fetch_bytes_total` and
-  `pixa_transforms_total`, fields of the cache. `NewCache` makes them and
-  registers them on the registry it is given (`internal/imgcache/counters.go`).
-  pixad gives it Prometheus' default registry, which `/metrics` serves; each
-  test gives it a new one. The `Increment` methods add to their cache's
-  counters. `cache_stats`, `Cache.Stats`, `Service.Stats`, `CacheStats` and the
-  pending counts are gone. The background goroutine writes only access times
-  now: `StartPendingAccessTimeWrites` starts it and
-  `StopPendingAccessTimeWrites` stops it, in `pendingaccesstimes.go`.
-- 2026-10-21 the schema file declares only the tables and columns pixa writes
-  and reads (closes #310): `output_content` and `request_cache` are gone, and so
-  are the columns nothing read: the host, path, query, `content_type` and
-  `fetched_at` of `source_content`, the `content_type` and `created_at` of
-  `variant_content`, and the `status_code`, `error_message` and `fetched_at` of
-  `negative_cache`. `StoreSource` no longer takes the fetch result,
-  `LookupSource` returns only the source key, `StoreVariant` no longer takes a
-  content type, `StoreNegative` no longer takes a status code or message, and
-  reconciliation adopts a variant file without reading its first bytes.
-- 2026-10-21 a cache hit makes no database call (closes #309): `Service.Get`
-  opens the variant file with `Cache.GetVariant` and nothing else, with no
-  existence check first. The content type comes from the file's first bytes; a
-  file whose first bytes are not those of an output format, SVG included, is a
-  miss, and the variant is made again. A hit hands its access time to a channel
-  that holds up to 10,000 and drops the rest, and its hit count to the pending
-  counts, without waiting; every count is now a pending count. A miss that finds
-  a cached source hands that source's access time to a second such channel. The
-  goroutine `StartPendingCountWrites` starts writes, once a second, the access
-  times in one transaction and the counts in one `UPDATE`;
-  `StopPendingCountWrites` writes what is left once at shutdown and logs
-  separately the access times and the counts it could not write.
-  `checkNegativeCache` no longer deletes an expired row; `CleanExpired` in the
-  eviction goroutine does. The `.meta` files, `VariantMeta`, `LoadWithMeta`,
-  `variantContentTypeFromSidecar`, `metaCache`, `Lookup` and `golang-lru` are
-  gone. `make bench` (`script/bench`) runs `BenchmarkHandleImage_CacheHits` with
-  Go on the host after `script/bootstrap --cgo`, outside `make check`.
-- 2026-10-21 a source is stored the way a variant is (closes #308): one
-  `source_content` row and one file per source URL, the file named by the
-  source's `SourceKey`, the SHA-256 of its host, path and query made as the
-  variant key is, under `cache/sources/`. A store writes the file, then the row,
-  and removes the file when the row cannot be written; an eviction deletes the
-  row, then the file, with no lock and no transaction. `source_metadata`, the
-  JSON files under `cache/metadata/`, `MetadataStorage` and the content lock are
-  gone. Two source URLs that serve the same bytes now store them twice.
 - 2026-10-21 one image request type instead of five (closes #307): the new
   package `internal/imagerequest`, which imports no other package of pixa, holds
   `imagerequest.Request` with its two parts, `Source` (host, path and query) and

go.mod

@@ -14,6 +14,7 @@ require (
 	github.com/go-chi/httprate v0.16.0
 	github.com/gorilla/csrf v1.7.3
 	github.com/gorilla/securecookie v1.1.2
+	github.com/hashicorp/golang-lru/v2 v2.0.7
 	github.com/prometheus/client_golang v1.23.2
 	github.com/slok/go-http-metrics v0.13.0
 	github.com/spf13/cobra v1.10.2

go.sum

@@ -228,6 +228,8 @@ github.com/hashicorp/go-version v1.2.1/go.mod h1:fltr4n8CU8Ke44wwGCBoEymUuxUHl09
 github.com/hashicorp/golang-lru v0.5.0/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8=
 github.com/hashicorp/golang-lru v0.5.4 h1:YDjusn29QI/Das2iO9M0BHnIbxPeyuCHsjMW+lJfyTc=
 github.com/hashicorp/golang-lru v0.5.4/go.mod h1:iADmTwqILo4mZ8BN3D2Q6+9jd8WM5uGBxy+E8yxSoD4=
+github.com/hashicorp/golang-lru/v2 v2.0.7 h1:a+bsQ5rvGLjzHuww6tVxozPZFVghXaHOwFs4luLUK2k=
+github.com/hashicorp/golang-lru/v2 v2.0.7/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM=
 github.com/hashicorp/hcl v1.0.1-vault-7 h1:ag5OxFVy3QYTFTJODRzTKVZ6xvdfLLCA1cy/Y6xGI0I=
 github.com/hashicorp/hcl v1.0.1-vault-7/go.mod h1:XYhtn6ijBSAj6n4YqAaf7RBPS4I06AItNorpy+MoQNM=
 github.com/hashicorp/logutils v1.0.0/go.mod h1:QIAnNjmIWmVIIkWDTG1z5v++HQmx9WQRO+LraFDTW64=

internal/allowlist/allowlist.go

@@ -72,3 +72,13 @@ func (w *HostAllowList) IsAllowed(u *url.URL) bool {
 
 	return false
 }
+
+// IsEmpty returns true if the allow list has no entries.
+func (w *HostAllowList) IsEmpty() bool {
+	return len(w.exactHosts) == 0 && len(w.suffixHosts) == 0
+}
+
+// Count returns the total number of allow list entries.
+func (w *HostAllowList) Count() int {
+	return len(w.exactHosts) + len(w.suffixHosts)
+}

internal/allowlist/allowlist_test.go

@@ -139,3 +139,87 @@ func TestHostAllowList_IsAllowed_SuffixMatch(t *testing.T) {
 		},
 	})
 }
+
+func TestHostAllowList_IsEmpty(t *testing.T) {
+	t.Parallel()
+
+	tests := []struct {
+		name     string
+		patterns []string
+		want     bool
+	}{
+		{
+			name:     "empty",
+			patterns: []string{},
+			want:     true,
+		},
+		{
+			name:     "nil",
+			patterns: nil,
+			want:     true,
+		},
+		{
+			name:     "whitespace only",
+			patterns: []string{"  ", ""},
+			want:     true,
+		},
+		{
+			name:     "has entries",
+			patterns: []string{"example.com"},
+			want:     false,
+		},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			t.Parallel()
+
+			w := allowlist.New(tt.patterns)
+			if got := w.IsEmpty(); got != tt.want {
+				t.Errorf("IsEmpty() = %v, want %v", got, tt.want)
+			}
+		})
+	}
+}
+
+func TestHostAllowList_Count(t *testing.T) {
+	t.Parallel()
+
+	tests := []struct {
+		name     string
+		patterns []string
+		want     int
+	}{
+		{
+			name:     "empty",
+			patterns: []string{},
+			want:     0,
+		},
+		{
+			name:     "exact hosts only",
+			patterns: []string{"a.com", "b.com", "c.com"},
+			want:     3,
+		},
+		{
+			name:     "suffix hosts only",
+			patterns: []string{".a.com", ".b.com"},
+			want:     2,
+		},
+		{
+			name:     "mixed",
+			patterns: []string{"exact.com", ".suffix.com"},
+			want:     2,
+		},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			t.Parallel()
+
+			w := allowlist.New(tt.patterns)
+			if got := w.Count(); got != tt.want {
+				t.Errorf("Count() = %v, want %v", got, tt.want)
+			}
+		})
+	}
+}

internal/app/app.go

@@ -7,7 +7,6 @@ import (
 	"os/signal"
 	"syscall"
 
-	"github.com/prometheus/client_golang/prometheus"
 	"github.com/spf13/cobra"
 	"go.uber.org/fx"
 	"sneak.berlin/go/pixa/internal/config"
@@ -17,7 +16,6 @@ import (
 	"sneak.berlin/go/pixa/internal/healthcheck"
 	"sneak.berlin/go/pixa/internal/logger"
 	"sneak.berlin/go/pixa/internal/middleware"
-	"sneak.berlin/go/pixa/internal/providers"
 	"sneak.berlin/go/pixa/internal/server"
 )
 
@@ -63,10 +61,7 @@ func run(_ *cobra.Command, _ []string) {
 			server.New,
 			middleware.New,
 			healthcheck.New,
-			// The registry /metrics serves
-			func() prometheus.Registerer { return prometheus.DefaultRegisterer },
 		),
-		fx.Provide(providers.Constructors()...),
 		fx.Invoke(
 			func(log *logger.Logger) { log.Identify() },
 			func(*server.Server) {},

internal/config/cache_max_bytes_internal_test.go

@@ -135,38 +135,10 @@ func TestCacheMaxBytesInvalidValuesAbortStartup(t *testing.T) {
 	}
 }
 
-// TestDefaultCacheMaxBytesAppliesFloor verifies that when 75% of the
-// free space plus what the cache holds is below 500 MiB, the default is
-// floored at DefaultCacheMaxBytesFloor.
-func TestDefaultCacheMaxBytesAppliesFloor(t *testing.T) {
-	t.Parallel()
-
-	cases := []struct {
-		name      string
-		freeBytes uint64
-	}{
-		{name: "100 MiB free", freeBytes: 104857600},
-		{name: "zero free", freeBytes: 0},
-		{name: "just below floor threshold", freeBytes: 699050665},
-	}
-
-	for _, tc := range cases {
-		t.Run(tc.name, func(t *testing.T) {
-			t.Parallel()
-
-			got := DefaultCacheMaxBytes(tc.freeBytes, 0)
-			if got != DefaultCacheMaxBytesFloor {
-				t.Errorf("DefaultCacheMaxBytes = %d, want floor %d",
-					got, DefaultCacheMaxBytesFloor)
-			}
-		})
-	}
-}
-
 // TestCacheMaxBytesExplicitIsRecorded verifies that an omitted
-// cache_max_bytes is recorded as not explicit, so pixad works out the
+// cache_max_bytes is recorded as not explicit, so the cache works out
-// default when it builds the cache, and that an explicit zero is recorded
+// the default when it opens, and that an explicit zero is recorded as
-// as explicit, so it disables the disk cache instead.
+// explicit, so it disables the disk cache instead.
 func TestCacheMaxBytesExplicitIsRecorded(t *testing.T) {
 	t.Parallel()
 

internal/config/config.go

@@ -45,37 +45,6 @@ func DefaultMaxConcurrentProcessing() int {
 	return max(runtime.GOMAXPROCS(0)/2, 1) //nolint:mnd // half the CPUs
 }
 
-// DefaultCacheMaxBytesFloor is the least DefaultCacheMaxBytes returns:
-// 500 MiB. It applies only to the default, never to an explicit
-// cache_max_bytes.
-const DefaultCacheMaxBytesFloor int64 = 500 << 20
-
-// freeSpaceFractionNumerator and freeSpaceFractionDenominator express
-// the 75% share of DefaultCacheMaxBytes as integer arithmetic (dividing
-// before multiplying avoids overflow).
-const (
-	freeSpaceFractionNumerator   uint64 = 3
-	freeSpaceFractionDenominator uint64 = 4
-)
-
-// DefaultCacheMaxBytes returns the disk cache limit when cache_max_bytes
-// is omitted: 75% of the sum of freeBytes, the space free on the
-// filesystem holding the cache, and heldBytes, the bytes the cache
-// already holds, and at least DefaultCacheMaxBytesFloor. Counting what
-// the cache holds keeps the limit from shrinking as the cache fills.
-// pixad works it out when it builds the cache, as both numbers are known
-// only then.
-func DefaultCacheMaxBytes(freeBytes uint64, heldBytes int64) int64 {
-	// Both terms are at most math.MaxInt64, so the sum cannot overflow.
-	//nolint:gosec // G115: heldBytes is the cache's total, never negative
-	spaceBytes := min(freeBytes, math.MaxInt64) + uint64(heldBytes)
-
-	computed := spaceBytes / freeSpaceFractionDenominator * freeSpaceFractionNumerator
-	computed = min(computed, math.MaxInt64)
-
-	return max(int64(computed), DefaultCacheMaxBytesFloor)
-}
-
 // Configuration key names.
 const (
 	keyDebug                      = "debug"
@@ -221,14 +190,18 @@ type Config struct {
 	// address, and an explicit list replaces the default.
 	TrustedProxies []netip.Prefix
 
-	// CacheMaxBytes is the disk cache size limit in bytes, as set; 0
+	// CacheMaxBytes is the disk cache size limit in bytes. Only an
-	// turns the disk cache off.
+	// explicit zero (CacheMaxBytesExplicit true) disables the disk
+	// cache. Zero with CacheMaxBytesExplicit false means
+	// cache_max_bytes was omitted, and the cache works out the default
+	// limit when it opens.
 	CacheMaxBytes int64
 
 	// CacheMaxBytesExplicit records whether cache_max_bytes was
 	// explicitly set, in the environment or the configuration file.
-	// Explicit values are used exactly as given. For an omitted key
+	// Explicit values are used exactly as given; for an omitted key the
-	// pixad works out DefaultCacheMaxBytes when it builds the cache.
+	// cache works out the default limit when it opens (see
+	// imgcache.CacheConfig.UseDefaultMaxBytes).
 	CacheMaxBytesExplicit bool
 }
 
@@ -266,8 +239,8 @@ func New(_ fx.Lifecycle, params Params) (*Config, error) {
 		return nil, err
 	}
 
-	// An omitted cache_max_bytes is worked out and logged when pixad
+	// An omitted cache_max_bytes is worked out and logged when the
-	// builds the cache.
+	// cache opens.
 	if c.CacheMaxBytesExplicit {
 		log.Info("effective cache size limit",
 			"cache_max_bytes", c.CacheMaxBytes,
@@ -352,8 +325,8 @@ func newFromSmartConfig(sc *smartconfig.Config) (*Config, error) {
 	}
 
 	// The default for an omitted cache_max_bytes is worked out when
-	// pixad builds the cache; here we only record whether the operator
+	// the cache opens; here we only record whether the operator set
-	// set the key explicitly.
+	// the key explicitly.
 	_, c.CacheMaxBytesExplicit = lookupValue(sc, keyCacheMaxBytes)
 
 	// Build DBURL from StateDir if not explicitly set. The derived URL

internal/database/concurrent_writes_internal_test.go

@@ -104,31 +104,40 @@ func writeConcurrently(t *testing.T, db *sql.DB) {
 
 	wg.Wait()
 
-	var sources int
+	var hits, sources int
 
-	err := db.QueryRowContext(ctx, `SELECT COUNT(*) FROM source_content`).
+	err := db.QueryRowContext(ctx, `
-		Scan(&sources)
+		SELECT hit_count, (SELECT COUNT(*) FROM source_content)
+		FROM cache_stats WHERE id = 1
+	`).Scan(&hits, &sources)
 	if err != nil {
 		t.Fatalf("failed to count the writes: %v", err)
 	}
 
-	if sources != totalRequests {
+	if hits != totalRequests || sources != totalRequests {
-		t.Errorf("%d source_content rows, want %d", sources, totalRequests)
+		t.Errorf("hit_count = %d and %d source_content rows, want %d of each",
+			hits, sources, totalRequests)
 	}
 }
 
 // writeLikeOneRequest makes the writes one request and the eviction pass
-// make: it stores a source, records a transformed image and deletes that
+// make: it counts a cache hit, stores a source, records a transformed image
-// record again.
+// and deletes that record again.
 func writeLikeOneRequest(ctx context.Context, db *sql.DB, key string) error {
-	_, err := db.ExecContext(ctx, `INSERT INTO source_content
+	_, err := db.ExecContext(ctx,
-		(source_key, size_bytes) VALUES (?, 1)`, key)
+		`UPDATE cache_stats SET hit_count = hit_count + 1 WHERE id = 1`)
+	if err != nil {
+		return fmt.Errorf("counting a cache hit: %w", err)
+	}
+
+	_, err = db.ExecContext(ctx, `INSERT INTO source_content
+		(content_hash, content_type, size_bytes) VALUES (?, 'image/png', 1)`, key)
 	if err != nil {
 		return fmt.Errorf("storing a source: %w", err)
 	}
 
 	_, err = db.ExecContext(ctx, `INSERT INTO variant_content
-		(cache_key, size_bytes) VALUES (?, 1)`, key)
+		(cache_key, size_bytes, content_type) VALUES (?, 1, 'image/png')`, key)
 	if err != nil {
 		return fmt.Errorf("recording a transformed image: %w", err)
 	}

internal/database/database.go

@@ -82,11 +82,7 @@ func ParseMigrationVersion(filename string) (int, error) {
 	return version, nil
 }
 
-// New opens the database and applies the migrations, so that the parts
+// New creates a new Database instance.
-// fx builds after it can read the database while they are built. Its
-// stop hook closes the database. fx runs stop hooks in the reverse of the
-// order it built their parts in, so the stop hooks of those parts run
-// before this one.
 func New(lc fx.Lifecycle, params Params) (*Database, error) {
 	s := &Database{
 		log:    params.Logger.Get(),
@@ -95,16 +91,20 @@ func New(lc fx.Lifecycle, params Params) (*Database, error) {
 
 	s.log.Info("Database instantiated")
 
-	err := s.connect(context.Background())
-	if err != nil {
-		return nil, err
-	}
-
 	lc.Append(fx.Hook{
+		OnStart: func(ctx context.Context) error {
+			s.log.Info("Database OnStart Hook")
+
+			return s.connect(ctx)
+		},
 		OnStop: func(_ context.Context) error {
 			s.log.Info("Database OnStop Hook")
 
-			return s.db.Close()
+			if s.db != nil {
+				return s.db.Close()
+			}
+
+			return nil
 		},
 	})
 

internal/database/database_internal_test.go

@@ -159,7 +159,10 @@ func TestApplyMigrations_CreatesSchemaAndTables(t *testing.T) {
 	}
 
 	// Verify that the application tables created by 001.sql exist.
-	tables := []string{"source_content", "negative_cache"}
+	tables := []string{
+		"source_content", "source_metadata", "output_content",
+		"request_cache", "negative_cache", "cache_stats",
+	}
 	for _, table := range tables {
 		var count int
 

internal/db/migrations/001_schema.sql

@@ -1,27 +1,52 @@
 -- Migration 001: Initial schema
 -- Creates all tables for the pixa caching image proxy
 
--- Source images, one row per source URL
+-- Source content blobs
--- Files stored at: cache/sources/<ab>/<cd>/<source_key>, where source_key
+-- Files stored at: cache/sources/<ab>/<cd>/<sha256>
--- is the SHA-256 of the source's host, path and query
 CREATE TABLE IF NOT EXISTS source_content (
-    source_key TEXT PRIMARY KEY,
+    content_hash TEXT PRIMARY KEY,
+    content_type TEXT NOT NULL,
     size_bytes INTEGER NOT NULL,
+    fetched_at DATETIME DEFAULT CURRENT_TIMESTAMP,
     last_accessed_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP
 );
 CREATE INDEX IF NOT EXISTS idx_source_content_last_accessed
     ON source_content(last_accessed_at);
 
+-- Source URL metadata - maps URLs to content hashes
+-- JSON stored at: cache/metadata/<hostname>/<path_hash>.json
+CREATE TABLE IF NOT EXISTS source_metadata (
+    id INTEGER PRIMARY KEY AUTOINCREMENT,
+    source_host TEXT NOT NULL,
+    source_path TEXT NOT NULL,
+    source_query TEXT NOT NULL DEFAULT '',
+    path_hash TEXT NOT NULL,
+    content_hash TEXT,
+    status_code INTEGER NOT NULL,
+    content_type TEXT,
+    response_headers TEXT,
+    fetched_at DATETIME DEFAULT CURRENT_TIMESTAMP,
+    expires_at DATETIME,
+    etag TEXT,
+    last_modified TEXT,
+    UNIQUE(source_host, source_path, source_query),
+    FOREIGN KEY (content_hash) REFERENCES source_content(content_hash)
+);
+CREATE INDEX IF NOT EXISTS idx_source_meta_host ON source_metadata(source_host);
+CREATE INDEX IF NOT EXISTS idx_source_meta_path_hash ON source_metadata(path_hash);
+CREATE INDEX IF NOT EXISTS idx_source_meta_expires ON source_metadata(expires_at);
+CREATE INDEX IF NOT EXISTS idx_source_meta_content_hash ON source_metadata(content_hash);
+
 -- Processed variant blobs
--- Files stored at: cache/variants/<ab>/<cd>/<cache_key>. Tracked here
+-- Files stored at: cache/variants/<ab>/<cd>/<cache_key> (plus a .meta
--- (like source content blobs above) so total cache usage is known
+-- sidecar with the content type). Tracked here (like source content
--- without a directory scan, and so LRU eviction has a timestamp to order
+-- blobs above) so total cache usage is known without a directory scan,
--- on. A cache hit does not read this table: it takes the content type
+-- and so LRU eviction has a timestamp to order on.
--- from the file's first bytes, and its last_accessed_at is written
--- later, in a batch.
 CREATE TABLE IF NOT EXISTS variant_content (
     cache_key TEXT PRIMARY KEY,
     size_bytes INTEGER NOT NULL,
+    content_type TEXT NOT NULL,
+    created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
     last_accessed_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP
 );
 CREATE INDEX IF NOT EXISTS idx_variant_content_last_accessed
@@ -95,19 +120,63 @@ BEGIN
     WHERE id = 1;
 END;
 
--- Negative cache: source URLs whose fetch failed with an upstream error
+-- Output/transformed content blobs
--- status, or whose first bytes did not match their Content-Type. pixa does
+-- Not written: transformed images are stored in cache/variants and
--- not fetch such a URL again until its row expires.
+-- tracked in variant_content above.
+CREATE TABLE IF NOT EXISTS output_content (
+    content_hash TEXT PRIMARY KEY,
+    content_type TEXT NOT NULL,
+    size_bytes INTEGER NOT NULL,
+    fetched_at DATETIME DEFAULT CURRENT_TIMESTAMP
+);
+
+-- Request cache - maps full request params to output content
+CREATE TABLE IF NOT EXISTS request_cache (
+    id INTEGER PRIMARY KEY AUTOINCREMENT,
+    cache_key TEXT NOT NULL UNIQUE,
+    source_metadata_id INTEGER NOT NULL,
+    output_hash TEXT NOT NULL,
+    width INTEGER NOT NULL,
+    height INTEGER NOT NULL,
+    format TEXT NOT NULL,
+    quality INTEGER NOT NULL DEFAULT 85,
+    fit_mode TEXT NOT NULL DEFAULT 'cover',
+    fetched_at DATETIME DEFAULT CURRENT_TIMESTAMP,
+    access_count INTEGER NOT NULL DEFAULT 1,
+    FOREIGN KEY (source_metadata_id) REFERENCES source_metadata(id),
+    FOREIGN KEY (output_hash) REFERENCES output_content(content_hash)
+);
+CREATE INDEX IF NOT EXISTS idx_request_cache_key ON request_cache(cache_key);
+CREATE INDEX IF NOT EXISTS idx_request_cache_source ON request_cache(source_metadata_id);
+CREATE INDEX IF NOT EXISTS idx_request_cache_output ON request_cache(output_hash);
+CREATE INDEX IF NOT EXISTS idx_request_cache_fetched ON request_cache(fetched_at);
+
+-- Negative cache for failed fetches (404s, timeouts, etc.)
 CREATE TABLE IF NOT EXISTS negative_cache (
     id INTEGER PRIMARY KEY AUTOINCREMENT,
     source_host TEXT NOT NULL,
     source_path TEXT NOT NULL,
     source_query TEXT NOT NULL DEFAULT '',
+    status_code INTEGER NOT NULL,
+    error_message TEXT,
+    fetched_at DATETIME DEFAULT CURRENT_TIMESTAMP,
     expires_at DATETIME NOT NULL,
     UNIQUE(source_host, source_path, source_query)
 );
 CREATE INDEX IF NOT EXISTS idx_negative_cache_expires ON negative_cache(expires_at);
 
+-- Cache statistics for monitoring
+CREATE TABLE IF NOT EXISTS cache_stats (
+    id INTEGER PRIMARY KEY CHECK (id = 1),
+    hit_count INTEGER NOT NULL DEFAULT 0,
+    miss_count INTEGER NOT NULL DEFAULT 0,
+    upstream_fetch_count INTEGER NOT NULL DEFAULT 0,
+    upstream_fetch_bytes INTEGER NOT NULL DEFAULT 0,
+    transform_count INTEGER NOT NULL DEFAULT 0,
+    last_updated_at DATETIME DEFAULT CURRENT_TIMESTAMP
+);
+INSERT OR IGNORE INTO cache_stats (id) VALUES (1);
+
 -- A random value that a login session carries from its login. pixa makes it
 -- when the table has no row, and each logout replaces it; a login session
 -- carrying another value is refused, so logging out ends every login session.

internal/encurl/encurl.go

@@ -16,6 +16,9 @@ const (
 	DefaultQuality = 85
 	DefaultFormat  = imagerequest.FormatJXL
 	DefaultFitMode = imagerequest.FitCover
+
+	// HKDF salt for URL encryption key derivation
+	urlKeySalt = "pixa-urlenc-v1"
 )
 
 // Errors returned by encurl operations.
@@ -45,20 +48,23 @@ type Generator struct {
 	key [seal.KeySize]byte
 }
 
-// NewGenerator creates an encrypted URL generator that encrypts its tokens
+// NewGenerator creates an encrypted URL generator with a key derived
-// with key.
+// from the signing key.
-func NewGenerator(key [seal.KeySize]byte) *Generator {
+func NewGenerator(signingKey string) (*Generator, error) {
-	return &Generator{key: key}
+	key, err := seal.DeriveKey([]byte(signingKey), urlKeySalt)
+	if err != nil {
+		return nil, err
+	}
+
+	return &Generator{key: key}, nil
 }
 
-// Generate creates an encrypted URL token from the request's source and
+// Generate creates an encrypted URL token from the request's source,
-// transform, each as the request holds it, and from expires; an empty format,
+// transform and expiry, each as the request holds it; an empty format, a
-// a quality of 0 or an empty fit mode is left out of the token, and a zero
+// quality of 0 or an empty fit mode is left out of the token, and a zero
-// expires makes a token that never expires. The token is CBOR-encoded,
+// Expires makes a token that never expires. The token is CBOR-encoded,
 // encrypted with NaCl secretbox, and base64url-encoded.
-func (g *Generator) Generate(
+func (g *Generator) Generate(req *imagerequest.Request) (string, error) {
-	req *imagerequest.Request, expires time.Time,
-) (string, error) {
 	p := &payload{
 		SourceHost:  req.Source.Host,
 		SourcePath:  req.Source.Path,
@@ -70,8 +76,8 @@ func (g *Generator) Generate(
 		FitMode:     req.Transform.FitMode,
 	}
 
-	if !expires.IsZero() {
+	if !req.Expires.IsZero() {
-		p.ExpiresAt = expires.Unix()
+		p.ExpiresAt = req.Expires.Unix()
 	}
 
 	// CBOR encode the payload
@@ -85,19 +91,19 @@ func (g *Generator) Generate(
 }
 
 // Parse decrypts and validates an encrypted URL token and returns the
-// request it holds, with default values for omitted optional fields, and its
+// request it holds, with default values for omitted optional fields. A token
-// expiry. A token that never expires gives the zero time.
+// that never expires gives the zero Expires.
 // Returns ErrExpired if the token has expired, or other errors for invalid tokens.
-func (g *Generator) Parse(token string) (*imagerequest.Request, time.Time, error) {
+func (g *Generator) Parse(token string) (*imagerequest.Request, error) {
 	// Decrypt
 	data, err := seal.Decrypt(g.key, token)
 	if err != nil {
 		if errors.Is(err, seal.ErrDecryptionFailed) ||
 			errors.Is(err, seal.ErrInvalidPayload) {
-			return nil, time.Time{}, ErrDecryptFailed
+			return nil, ErrDecryptFailed
 		}
 
-		return nil, time.Time{}, err
+		return nil, err
 	}
 
 	// CBOR decode
@@ -105,12 +111,12 @@ func (g *Generator) Parse(token string) (*imagerequest.Request, time.Time, error
 
 	err = cbor.Unmarshal(data, &p)
 	if err != nil {
-		return nil, time.Time{}, ErrInvalidFormat
+		return nil, ErrInvalidFormat
 	}
 
 	// Check expiration (0 = never expires)
 	if p.ExpiresAt != 0 && time.Now().Unix() > p.ExpiresAt {
-		return nil, time.Time{}, ErrExpired
+		return nil, ErrExpired
 	}
 
 	format := p.Format
@@ -145,5 +151,6 @@ func (g *Generator) Parse(token string) (*imagerequest.Request, time.Time, error
 			Quality: quality,
 			FitMode: fitMode,
 		},
-	}, expires, nil
+		Expires: expires,
+	}, nil
 }

internal/encurl/encurl_test.go

@@ -7,7 +7,6 @@ import (
 
 	"sneak.berlin/go/pixa/internal/encurl"
 	"sneak.berlin/go/pixa/internal/imagerequest"
-	"sneak.berlin/go/pixa/internal/seal"
 )
 
 // Shared test fixture strings.
@@ -20,13 +19,11 @@ const (
 func TestGenerator_GenerateAndParse(t *testing.T) {
 	t.Parallel()
 
-	keys, err := seal.DeriveKeys("test-signing-key-12345")
+	gen, err := encurl.NewGenerator("test-signing-key-12345")
 	if err != nil {
-		t.Fatalf("seal.DeriveKeys() error = %v", err)
+		t.Fatalf("NewGenerator() error = %v", err)
 	}
 
-	gen := encurl.NewGenerator(keys.EncryptedURLKey)
-
 	req := &imagerequest.Request{
 		Source: imagerequest.Source{
 			Host:  testSourceHost,
@@ -39,10 +36,10 @@ func TestGenerator_GenerateAndParse(t *testing.T) {
 			Quality: 90,
 			FitMode: imagerequest.FitContain,
 		},
+		Expires: time.Now().Add(time.Hour),
 	}
-	expires := time.Now().Add(time.Hour)
 
-	token, err := gen.Generate(req, expires)
+	token, err := gen.Generate(req)
 	if err != nil {
 		t.Fatalf("Generate() error = %v", err)
 	}
@@ -51,7 +48,7 @@ func TestGenerator_GenerateAndParse(t *testing.T) {
 		t.Fatal("Generate() returned empty token")
 	}
 
-	parsed, parsedExpires, err := gen.Parse(token)
+	parsed, err := gen.Parse(token)
 	if err != nil {
 		t.Fatalf("Parse() error = %v", err)
 	}
@@ -94,30 +91,30 @@ func TestGenerator_GenerateAndParse(t *testing.T) {
 			parsed.Transform.FitMode, req.Transform.FitMode)
 	}
 
-	if parsedExpires.Unix() != expires.Unix() {
+	if parsed.Expires.Unix() != req.Expires.Unix() {
-		t.Errorf("ExpiresAt = %d, want %d", parsedExpires.Unix(), expires.Unix())
+		t.Errorf("ExpiresAt = %d, want %d", parsed.Expires.Unix(), req.Expires.Unix())
 	}
 }
 
 func TestGenerator_Parse_Expired(t *testing.T) {
 	t.Parallel()
 
-	keys, _ := seal.DeriveKeys("test-signing-key-12345")
+	gen, _ := encurl.NewGenerator("test-signing-key-12345")
-	gen := encurl.NewGenerator(keys.EncryptedURLKey)
 
 	req := &imagerequest.Request{
 		Source: imagerequest.Source{
 			Host: testSourceHost,
 			Path: testSourcePath,
 		},
+		Expires: time.Now().Add(-time.Hour), // Already expired
 	}
 
-	token, err := gen.Generate(req, time.Now().Add(-time.Hour)) // Already expired
+	token, err := gen.Generate(req)
 	if err != nil {
 		t.Fatalf("Generate() error = %v", err)
 	}
 
-	_, _, err = gen.Parse(token)
+	_, err = gen.Parse(token)
 	if err == nil {
 		t.Error("Parse() should fail for expired token")
 	}
@@ -130,10 +127,9 @@ func TestGenerator_Parse_Expired(t *testing.T) {
 func TestGenerator_Parse_InvalidToken(t *testing.T) {
 	t.Parallel()
 
-	keys, _ := seal.DeriveKeys("test-signing-key-12345")
+	gen, _ := encurl.NewGenerator("test-signing-key-12345")
-	gen := encurl.NewGenerator(keys.EncryptedURLKey)
 
-	_, _, err := gen.Parse("not-a-valid-token")
+	_, err := gen.Parse("not-a-valid-token")
 	if err == nil {
 		t.Error("Parse() should fail for invalid token")
 	}
@@ -142,17 +138,17 @@ func TestGenerator_Parse_InvalidToken(t *testing.T) {
 func TestGenerator_Parse_TamperedToken(t *testing.T) {
 	t.Parallel()
 
-	keys, _ := seal.DeriveKeys("test-signing-key-12345")
+	gen, _ := encurl.NewGenerator("test-signing-key-12345")
-	gen := encurl.NewGenerator(keys.EncryptedURLKey)
 
 	req := &imagerequest.Request{
 		Source: imagerequest.Source{
 			Host: testSourceHost,
 			Path: testSourcePath,
 		},
+		Expires: time.Now().Add(time.Hour),
 	}
 
-	token, _ := gen.Generate(req, time.Now().Add(time.Hour))
+	token, _ := gen.Generate(req)
 
 	// Tamper with the token
 	tampered := []byte(token)
@@ -160,7 +156,7 @@ func TestGenerator_Parse_TamperedToken(t *testing.T) {
 		tampered[10] ^= 0x01
 	}
 
-	_, _, err := gen.Parse(string(tampered))
+	_, err := gen.Parse(string(tampered))
 	if err == nil {
 		t.Error("Parse() should fail for tampered token")
 	}
@@ -169,21 +165,20 @@ func TestGenerator_Parse_TamperedToken(t *testing.T) {
 func TestGenerator_Parse_WrongKey(t *testing.T) {
 	t.Parallel()
 
-	keys1, _ := seal.DeriveKeys("signing-key-1")
+	gen1, _ := encurl.NewGenerator("signing-key-1")
-	keys2, _ := seal.DeriveKeys("signing-key-2")
+	gen2, _ := encurl.NewGenerator("signing-key-2")
-	gen1 := encurl.NewGenerator(keys1.EncryptedURLKey)
-	gen2 := encurl.NewGenerator(keys2.EncryptedURLKey)
 
 	req := &imagerequest.Request{
 		Source: imagerequest.Source{
 			Host: testSourceHost,
 			Path: testSourcePath,
 		},
+		Expires: time.Now().Add(time.Hour),
 	}
 
-	token, _ := gen1.Generate(req, time.Now().Add(time.Hour))
+	token, _ := gen1.Generate(req)
 
-	_, _, err := gen2.Parse(token)
+	_, err := gen2.Parse(token)
 	if err == nil {
 		t.Error("Parse() should fail with different signing key")
 	}
@@ -192,8 +187,7 @@ func TestGenerator_Parse_WrongKey(t *testing.T) {
 func TestGenerator_Parse_Defaults(t *testing.T) {
 	t.Parallel()
 
-	keys, _ := seal.DeriveKeys("test-signing-key-12345")
+	gen, _ := encurl.NewGenerator("test-signing-key-12345")
-	gen := encurl.NewGenerator(keys.EncryptedURLKey)
 
 	// Request with only required fields - should get defaults
 	token, err := gen.Generate(&imagerequest.Request{
@@ -201,12 +195,13 @@ func TestGenerator_Parse_Defaults(t *testing.T) {
 			Host: testSourceHost,
 			Path: testSourcePath,
 		},
-	}, time.Now().Add(time.Hour))
+		Expires: time.Now().Add(time.Hour),
+	})
 	if err != nil {
 		t.Fatalf("Generate() error = %v", err)
 	}
 
-	req, _, err := gen.Parse(token)
+	req, err := gen.Parse(token)
 	if err != nil {
 		t.Fatalf("Parse() error = %v", err)
 	}
@@ -230,17 +225,17 @@ func TestGenerator_Parse_Defaults(t *testing.T) {
 func TestGenerator_TokenIsURLSafe(t *testing.T) {
 	t.Parallel()
 
-	keys, _ := seal.DeriveKeys("test-signing-key-12345")
+	gen, _ := encurl.NewGenerator("test-signing-key-12345")
-	gen := encurl.NewGenerator(keys.EncryptedURLKey)
 
 	req := &imagerequest.Request{
 		Source: imagerequest.Source{
 			Host: testSourceHost,
 			Path: testSourcePath,
 		},
+		Expires: time.Now().Add(time.Hour),
 	}
 
-	token, _ := gen.Generate(req, time.Now().Add(time.Hour))
+	token, _ := gen.Generate(req)
 
 	// Check that token only contains URL-safe characters
 	for _, c := range token {

internal/handlers/auth.go

@@ -153,20 +153,20 @@ func (s *Handlers) HandleGenerateURL() http.HandlerFunc {
 			return
 		}
 
-		// The image URL must fetch the source URL as it was written. The
+		// The image URL must fetch the source URL as it was written
-		// fetcher fetches over http only when allow_http is set.
+		req.AllowHTTP = s.config.AllowHTTP
-		fetchURL := req.Source.URL(s.config.AllowHTTP)
+		fetchURL := req.SourceURL()
 
 		if !fetchedAsWritten(parsed, fetchURL) {
 			s.renderGeneratorWithForm(w, r, "Source URL cannot be fetched as "+
-				"written: pixa would fetch "+fetchURL.String(),
+				"written: pixa would fetch "+fetchURL,
 				r.Form, http.StatusBadRequest)
 
 			return
 		}
 
 		// Generate encrypted token
-		token, err := s.encGen.Generate(req, expiresAt)
+		token, err := s.encGen.Generate(req)
 		if err != nil {
 			s.log.Error("failed to generate encrypted URL", "error", err)
 			s.renderGeneratorWithForm(w, r, "Failed to generate URL", r.Form,
@@ -226,23 +226,25 @@ func parseSourceURL(sourceURL string) (*url.URL, error) {
 // in RawPath when it is not the decoded path percent-encoded again.
 // EscapedPath ignores RawPath when it also holds a character net/url encodes,
 // such as a space, so RawPath is compared itself.
-func fetchedAsWritten(source, fetchURL *url.URL) bool {
+func fetchedAsWritten(source *url.URL, fetchURL string) bool {
 	writtenPath := source.RawPath
 	if writtenPath == "" {
 		writtenPath = source.EscapedPath()
 	}
 
-	return source.User == nil && !source.ForceQuery &&
+	fetched, err := url.Parse(fetchURL)
-		fetchURL.EscapedPath() == writtenPath &&
+
-		fetchURL.RawQuery == source.RawQuery
+	return err == nil && source.User == nil && !source.ForceQuery &&
+		fetched.EscapedPath() == writtenPath &&
+		fetched.RawQuery == source.RawQuery
 }
 
 // buildGenerateRequest parses the numeric form fields and assembles the
-// image request the encrypted URL's token is made from, and its expiry.
+// image request the encrypted URL's token is made from. ttl=0 means never
-// ttl=0 means never expires (the expiry is the zero time). A non-numeric or
+// expires (Expires stays the zero time). A non-numeric or out-of-range
-// out-of-range width, height, quality or ttl, or an unrecognized fit mode, is
+// width, height, quality or ttl, or an unrecognized fit mode, is a client
-// a client error naming the offending field. The format field is passed
+// error naming the offending field. The format field is passed through
-// through unchecked.
+// unchecked.
 func buildGenerateRequest(
 	parsed *url.URL, form url.Values,
 ) (*imagerequest.Request, time.Time, int, error) {
@@ -292,6 +294,7 @@ func buildGenerateRequest(
 			Quality: quality,
 			FitMode: fitMode,
 		},
+		Expires: expiresAt,
 	}
 
 	return req, expiresAt, ttl, nil

internal/handlers/auth_csrf_internal_test.go

@@ -14,7 +14,6 @@ import (
 
 	"sneak.berlin/go/pixa/internal/config"
 	"sneak.berlin/go/pixa/internal/encurl"
-	"sneak.berlin/go/pixa/internal/seal"
 	"sneak.berlin/go/pixa/internal/session"
 )
 
@@ -45,17 +44,27 @@ func newCSRFTestRouter(t *testing.T) (*Handlers, http.Handler) {
 
 	cfg := &config.Config{SigningKey: testSigningKey, Debug: true}
 
-	keys, err := seal.DeriveKeys(testSigningKey)
+	sessMgr, err := session.NewManager(testSigningKey)
 	if err != nil {
-		t.Fatalf("seal.DeriveKeys() error = %v", err)
+		t.Fatalf("session.NewManager() error = %v", err)
+	}
+
+	encGen, err := encurl.NewGenerator(testSigningKey)
+	if err != nil {
+		t.Fatalf("encurl.NewGenerator() error = %v", err)
+	}
+
+	protect, err := newCSRFProtect(testSigningKey, cfg.Debug)
+	if err != nil {
+		t.Fatalf("newCSRFProtect() error = %v", err)
 	}
 
 	h := &Handlers{
 		log:         slog.New(slog.DiscardHandler),
 		config:      cfg,
-		sessMgr:     session.NewManager(keys.SessionHashKey, keys.SessionBlockKey),
+		sessMgr:     sessMgr,
-		encGen:      encurl.NewGenerator(keys.EncryptedURLKey),
+		encGen:      encGen,
-		csrfProtect: newCSRFProtect(keys.CSRFKey, cfg.Debug),
+		csrfProtect: protect,
 	}
 
 	r := chi.NewRouter()

internal/handlers/auth_login_log_internal_test.go

@@ -10,7 +10,6 @@ import (
 	"testing"
 
 	"sneak.berlin/go/pixa/internal/config"
-	"sneak.berlin/go/pixa/internal/seal"
 	"sneak.berlin/go/pixa/internal/session"
 )
 
@@ -23,15 +22,15 @@ func TestLoginLogLeavesOutSubmittedKey(t *testing.T) {
 
 	var buf bytes.Buffer
 
-	keys, err := seal.DeriveKeys(testSigningKey)
+	sessMgr, err := session.NewManager(testSigningKey)
 	if err != nil {
-		t.Fatalf("seal.DeriveKeys() error = %v", err)
+		t.Fatalf("session.NewManager() error = %v", err)
 	}
 
 	h := &Handlers{
 		log:     slog.New(slog.NewJSONHandler(&buf, nil)),
 		config:  &config.Config{SigningKey: testSigningKey},
-		sessMgr: session.NewManager(keys.SessionHashKey, keys.SessionBlockKey),
+		sessMgr: sessMgr,
 	}
 
 	submittedKeys := []string{wrongKey, testSigningKey}

internal/handlers/auth_session_internal_test.go

@@ -11,7 +11,6 @@ import (
 	"time"
 
 	"sneak.berlin/go/pixa/internal/imagerequest"
-	"sneak.berlin/go/pixa/internal/seal"
 	"sneak.berlin/go/pixa/internal/session"
 )
 
@@ -233,13 +232,8 @@ func TestHandleLogout_SaveFails_Answers500(t *testing.T) {
 
 	db := setupTestDB(t)
 
-	keys, err := seal.DeriveKeys(testSigningKey)
-	if err != nil {
-		t.Fatalf("seal.DeriveKeys() error = %v", err)
-	}
-
 	sessMgr, err := session.NewManagerWithDatabase(
-		t.Context(), keys.SessionHashKey, keys.SessionBlockKey, db)
+		t.Context(), testSigningKey, db)
 	if err != nil {
 		t.Fatalf("session.NewManagerWithDatabase() error = %v", err)
 	}

internal/handlers/avif_too_large_internal_test.go

@@ -12,11 +12,11 @@ import (
 	"sneak.berlin/go/pixa/internal/imagerequest"
 )
 
-// TestAVIFOutputTooLargeAnswers400 checks that respondImageError, which
+// TestAVIFOutputTooLargeAnswers400 checks that respondImageError and
-// answers the errors of both image routes, answers 400 naming the AVIF pixel
+// handleImageError, which answer the errors of the two image routes, answer
-// limit when the image processor refuses an AVIF output above it, wrapped as
+// 400 naming the AVIF pixel limit when the image processor refuses an AVIF
-// the service wraps it. The request's source only goes into a log line, so it
+// output above it, wrapped as the service wraps it. The request's source only
-// is left empty.
+// goes into a log line, so it is left empty.
 func TestAVIFOutputTooLargeAnswers400(t *testing.T) {
 	t.Parallel()
 
@@ -25,15 +25,25 @@ func TestAVIFOutputTooLargeAnswers400(t *testing.T) {
 		imageprocessor.ErrAVIFOutputTooLarge)
 	wantBody := fmt.Sprintf("above %d pixels", imageprocessor.MaxAVIFOutputPixels)
 
-	rec := httptest.NewRecorder()
+	plain := httptest.NewRecorder()
-	h.respondImageError(rec, &imagerequest.Request{}, err)
+	h.respondImageError(plain, &imagerequest.Request{}, err)
-	t.Logf("status = %d, body = %q", rec.Code, rec.Body.String())
 
-	if rec.Code != http.StatusBadRequest {
+	encrypted := httptest.NewRecorder()
-		t.Errorf("status = %d, want %d", rec.Code, http.StatusBadRequest)
+	h.handleImageError(encrypted, err)
-	}
 
-	if !strings.Contains(rec.Body.String(), wantBody) {
+	for name, rec := range map[string]*httptest.ResponseRecorder{
-		t.Errorf("body = %q, want it to say %q", rec.Body.String(), wantBody)
+		"respondImageError": plain, "handleImageError": encrypted,
+	} {
+		t.Logf("%s: status = %d, body = %q", name, rec.Code, rec.Body.String())
+
+		if rec.Code != http.StatusBadRequest {
+			t.Errorf("%s: status = %d, want %d",
+				name, rec.Code, http.StatusBadRequest)
+		}
+
+		if !strings.Contains(rec.Body.String(), wantBody) {
+			t.Errorf("%s: body = %q, want it to say %q",
+				name, rec.Body.String(), wantBody)
+		}
 	}
 }

internal/handlers/bodylimit_internal_test.go

@@ -11,7 +11,6 @@ import (
 
 	"sneak.berlin/go/pixa/internal/config"
 	"sneak.berlin/go/pixa/internal/encurl"
-	"sneak.berlin/go/pixa/internal/seal"
 	"sneak.berlin/go/pixa/internal/session"
 )
 
@@ -33,17 +32,27 @@ func newBodyLimitTestRouter(
 
 	cfg := &config.Config{SigningKey: testSigningKey, Debug: true}
 
-	keys, err := seal.DeriveKeys(testSigningKey)
+	sessMgr, err := session.NewManager(testSigningKey)
 	if err != nil {
-		t.Fatalf("seal.DeriveKeys() error = %v", err)
+		t.Fatalf("session.NewManager() error = %v", err)
+	}
+
+	encGen, err := encurl.NewGenerator(testSigningKey)
+	if err != nil {
+		t.Fatalf("encurl.NewGenerator() error = %v", err)
+	}
+
+	protect, err := newCSRFProtect(testSigningKey, cfg.Debug)
+	if err != nil {
+		t.Fatalf("newCSRFProtect() error = %v", err)
 	}
 
 	h := &Handlers{
 		log:         slog.New(slog.DiscardHandler),
 		config:      cfg,
-		sessMgr:     session.NewManager(keys.SessionHashKey, keys.SessionBlockKey),
+		sessMgr:     sessMgr,
-		encGen:      encurl.NewGenerator(keys.EncryptedURLKey),
+		encGen:      encGen,
-		csrfProtect: newCSRFProtect(keys.CSRFKey, cfg.Debug),
+		csrfProtect: protect,
 	}
 
 	r := chi.NewRouter()

internal/handlers/cache_hit_internal_test.go

@@ -1,277 +0,0 @@
-package handlers
-
-import (
-	"bytes"
-	"context"
-	"database/sql"
-	"database/sql/driver"
-	"image"
-	"image/jpeg"
-	"log/slog"
-	"net/http"
-	"net/http/httptest"
-	"path/filepath"
-	"sync/atomic"
-	"testing"
-	"testing/fstest"
-	"time"
-
-	"github.com/go-chi/chi/v5"
-	"github.com/prometheus/client_golang/prometheus"
-
-	"sneak.berlin/go/pixa/internal/allowlist"
-	"sneak.berlin/go/pixa/internal/database"
-	"sneak.berlin/go/pixa/internal/encurl"
-	"sneak.berlin/go/pixa/internal/imageprocessor"
-	"sneak.berlin/go/pixa/internal/imagerequest"
-	"sneak.berlin/go/pixa/internal/imgcache"
-	"sneak.berlin/go/pixa/internal/seal"
-	"sneak.berlin/go/pixa/internal/signature"
-)
-
-// countingHost is the allowlisted host whose photoPath the server of
-// newCountingDatabaseServer serves.
-const countingHost = "counting.example.com"
-
-// countingConnector opens connections to the SQLite database file name with
-// sqliteDriver, each one counting in statements the statements run on it.
-type countingConnector struct {
-	sqliteDriver driver.Driver
-	name         string
-	statements   *atomic.Int64
-}
-
-func (c countingConnector) Connect(context.Context) (driver.Conn, error) {
-	conn, err := c.sqliteDriver.Open(c.name)
-	if err != nil {
-		return nil, err
-	}
-
-	return countingConn{Conn: conn, statements: c.statements}, nil
-}
-
-func (c countingConnector) Driver() driver.Driver {
-	return c.sqliteDriver
-}
-
-// countingConn counts the statements run on its connection. It has only the
-// methods of driver.Conn, so database/sql prepares every query and every
-// other statement with Prepare, which counts it.
-type countingConn struct {
-	driver.Conn
-
-	statements *atomic.Int64
-}
-
-func (c countingConn) Prepare(query string) (driver.Stmt, error) {
-	c.statements.Add(1)
-
-	return c.Conn.Prepare(query)
-}
-
-// newCountingDatabase returns a database with pixa's schema, in a file of
-// its own and with one connection, and the count of the statements run on
-// it once the schema is in place.
-func newCountingDatabase(tb testing.TB) (*sql.DB, *atomic.Int64) {
-	tb.Helper()
-
-	path := filepath.Join(tb.TempDir(), "state.sqlite3")
-
-	schemaDB, err := sql.Open("sqlite", path)
-	if err != nil {
-		tb.Fatalf("opening the database: %v", err)
-	}
-
-	defer func() { _ = schemaDB.Close() }()
-
-	err = database.ApplyMigrations(tb.Context(), schemaDB, nil)
-	if err != nil {
-		tb.Fatalf("applying the migrations: %v", err)
-	}
-
-	statements := new(atomic.Int64)
-
-	db := sql.OpenDB(countingConnector{
-		sqliteDriver: schemaDB.Driver(),
-		name:         path,
-		statements:   statements,
-	})
-	db.SetMaxOpenConns(1)
-
-	tb.Cleanup(func() { _ = db.Close() })
-
-	return db, statements
-}
-
-// newCountingDatabaseServer returns a router for both image routes whose
-// image cache uses newCountingDatabase, the count of the statements run on
-// that database, and a URL on each route for a different image of
-// photoPath on countingHost. Nothing writes the cache's access times in the
-// background, so every statement counted comes from a request.
-func newCountingDatabaseServer(
-	tb testing.TB,
-) (http.Handler, *atomic.Int64, []string) {
-	tb.Helper()
-
-	db, statements := newCountingDatabase(tb)
-
-	var photo bytes.Buffer
-
-	err := jpeg.Encode(&photo, image.NewRGBA(image.Rect(0, 0, 100, 100)), nil)
-	if err != nil {
-		tb.Fatalf("encoding the photo: %v", err)
-	}
-
-	cache, err := imgcache.NewCache(db, imgcache.CacheConfig{
-		StateDir:    tb.TempDir(),
-		NegativeTTL: 5 * time.Minute,
-		MaxBytes:    1 << 30,
-	}, prometheus.NewRegistry())
-	if err != nil {
-		tb.Fatalf("imgcache.NewCache() error = %v", err)
-	}
-
-	svc, err := imgcache.NewService(&imgcache.ServiceConfig{
-		Cache: cache,
-		Fetcher: newMockFetcher(fstest.MapFS{
-			countingHost + photoPath: &fstest.MapFile{Data: photo.Bytes()},
-		}),
-		Processor: imageprocessor.New(imageprocessor.Params{
-			MaxInputBytes:           50 << 20,
-			MaxConcurrentProcessing: 2,
-		}),
-		MaxResponseSize: 50 << 20,
-		Logger:          slog.New(slog.DiscardHandler),
-	})
-	if err != nil {
-		tb.Fatalf("imgcache.NewService() error = %v", err)
-	}
-
-	keys, err := seal.DeriveKeys(testSigningKey)
-	if err != nil {
-		tb.Fatalf("seal.DeriveKeys() error = %v", err)
-	}
-
-	encGen := encurl.NewGenerator(keys.EncryptedURLKey)
-
-	token, err := encGen.Generate(&imagerequest.Request{
-		Source: imagerequest.Source{Host: countingHost, Path: photoPath},
-		Transform: imagerequest.Transform{
-			Size:    imagerequest.Size{Width: 40, Height: 40},
-			Format:  imagerequest.FormatJPEG,
-			Quality: 85,
-			FitMode: imagerequest.FitCover,
-		},
-	}, time.Time{})
-	if err != nil {
-		tb.Fatalf("encGen.Generate() error = %v", err)
-	}
-
-	h := &Handlers{
-		log:       slog.New(slog.DiscardHandler),
-		imgSvc:    svc,
-		encGen:    encGen,
-		signer:    signature.New(testSigningKey),
-		allowlist: allowlist.New([]string{countingHost}),
-	}
-
-	r := chi.NewRouter()
-	r.Get("/v1/image/*", h.HandleImage())
-	r.Get("/v1/e/{token}/*", h.HandleImageEnc())
-
-	return r, statements, []string{
-		"/v1/image/" + countingHost + photoPath + "/50x50.jpeg",
-		"/v1/e/" + token + "/img.jpg",
-	}
-}
-
-// serveImage sends srv a GET for target and returns the response, failing
-// the test unless it is a 200 with an image.
-func serveImage(
-	tb testing.TB, srv http.Handler, target string,
-) *httptest.ResponseRecorder {
-	tb.Helper()
-
-	rec := httptest.NewRecorder()
-	srv.ServeHTTP(rec,
-		httptest.NewRequestWithContext(tb.Context(), http.MethodGet, target, nil))
-
-	if rec.Code != http.StatusOK || rec.Body.Len() == 0 {
-		tb.Errorf("GET %s: %d with %d bytes, want 200 with an image",
-			target, rec.Code, rec.Body.Len())
-	}
-
-	return rec
-}
-
-// TestImageRoutes_CacheHitsMakeNoDatabaseCall serves an image on each image
-// route once, which makes and caches it, then serves it again from the cache.
-// Serving those cache hits must run no statement on the database.
-func TestImageRoutes_CacheHitsMakeNoDatabaseCall(t *testing.T) {
-	t.Parallel()
-
-	srv, statements, targets := newCountingDatabaseServer(t)
-
-	for _, target := range targets {
-		before := statements.Load()
-
-		serveImage(t, srv, target)
-
-		afterMiss := statements.Load()
-		t.Logf("GET %s, a miss: %d statements", target, afterMiss-before)
-
-		if afterMiss == before {
-			t.Fatalf("GET %s, a miss: no statement counted, want some", target)
-		}
-
-		for range 3 {
-			rec := serveImage(t, srv, target)
-
-			cacheStatus := rec.Header().Get("X-Pixa-Cache")
-			if cacheStatus != string(imgcache.CacheHit) {
-				t.Errorf("GET %s: X-Pixa-Cache = %q, want %s",
-					target, cacheStatus, imgcache.CacheHit)
-			}
-		}
-
-		if hitStatements := statements.Load() - afterMiss; hitStatements != 0 {
-			t.Errorf("GET %s: 3 cache hits ran %d statements, want 0",
-				target, hitStatements)
-		}
-	}
-}
-
-// BenchmarkHandleImage_CacheHits serves one cached image through the
-// /v1/image/ route from b.RunParallel's goroutines, and reports the cache
-// hits served per second. It fails if serving them ran any statement on the
-// database. script/bench runs it.
-func BenchmarkHandleImage_CacheHits(b *testing.B) {
-	srv, statements, targets := newCountingDatabaseServer(b)
-	target := targets[0]
-
-	serveImage(b, srv, target)
-
-	before := statements.Load()
-
-	b.ResetTimer()
-
-	b.RunParallel(func(pb *testing.PB) {
-		for pb.Next() {
-			rec := serveImage(b, srv, target)
-
-			cacheStatus := rec.Header().Get("X-Pixa-Cache")
-			if cacheStatus != string(imgcache.CacheHit) {
-				b.Errorf("GET %s: X-Pixa-Cache = %q, want %s",
-					target, cacheStatus, imgcache.CacheHit)
-
-				return
-			}
-		}
-	})
-
-	b.ReportMetric(float64(b.N)/b.Elapsed().Seconds(), "hits/s")
-
-	if hitStatements := statements.Load() - before; hitStatements != 0 {
-		b.Fatalf("%d cache hits ran %d statements, want 0", b.N, hitStatements)
-	}
-}

internal/handlers/cache_max_bytes_internal_test.go

@@ -0,0 +1,147 @@
+package handlers
+
+import (
+	"os"
+	"path/filepath"
+	"testing"
+
+	"go.uber.org/fx/fxtest"
+
+	"sneak.berlin/go/pixa/internal/config"
+	"sneak.berlin/go/pixa/internal/database"
+	"sneak.berlin/go/pixa/internal/globals"
+	"sneak.berlin/go/pixa/internal/logger"
+)
+
+// TestNewCacheConfigFromCacheMaxBytes checks the cache configuration
+// built from cache_max_bytes: omitted, the cache works out the default
+// limit; 0 turns the disk cache off; a positive value is the limit,
+// unchanged.
+func TestNewCacheConfigFromCacheMaxBytes(t *testing.T) {
+	t.Parallel()
+
+	const oneGiB = 1 << 30
+
+	cases := []struct {
+		name                   string
+		cacheMaxBytes          int64
+		cacheMaxBytesExplicit  bool
+		wantMaxBytes           int64
+		wantUseDefaultMaxBytes bool
+		wantDisableDiskCache   bool
+	}{
+		{
+			name:                   "cache_max_bytes omitted",
+			cacheMaxBytes:          0,
+			cacheMaxBytesExplicit:  false,
+			wantMaxBytes:           0,
+			wantUseDefaultMaxBytes: true,
+			wantDisableDiskCache:   false,
+		},
+		{
+			name:                   "cache_max_bytes: 0",
+			cacheMaxBytes:          0,
+			cacheMaxBytesExplicit:  true,
+			wantMaxBytes:           0,
+			wantUseDefaultMaxBytes: false,
+			wantDisableDiskCache:   true,
+		},
+		{
+			name:                   "cache_max_bytes: 1 GiB",
+			cacheMaxBytes:          oneGiB,
+			cacheMaxBytesExplicit:  true,
+			wantMaxBytes:           oneGiB,
+			wantUseDefaultMaxBytes: false,
+			wantDisableDiskCache:   false,
+		},
+	}
+
+	for _, tc := range cases {
+		t.Run(tc.name, func(t *testing.T) {
+			t.Parallel()
+
+			cfg := &config.Config{
+				CacheMaxBytes:         tc.cacheMaxBytes,
+				CacheMaxBytesExplicit: tc.cacheMaxBytesExplicit,
+			}
+
+			got := newCacheConfig(cfg, nil)
+			t.Logf("MaxBytes = %d, UseDefaultMaxBytes = %v, DisableDiskCache = %v",
+				got.MaxBytes, got.UseDefaultMaxBytes, got.DisableDiskCache)
+
+			if got.MaxBytes != tc.wantMaxBytes {
+				t.Errorf("MaxBytes = %d, want %d", got.MaxBytes, tc.wantMaxBytes)
+			}
+
+			if got.UseDefaultMaxBytes != tc.wantUseDefaultMaxBytes {
+				t.Errorf("UseDefaultMaxBytes = %v, want %v",
+					got.UseDefaultMaxBytes, tc.wantUseDefaultMaxBytes)
+			}
+
+			if got.DisableDiskCache != tc.wantDisableDiskCache {
+				t.Errorf("DisableDiskCache = %v, want %v",
+					got.DisableDiskCache, tc.wantDisableDiskCache)
+			}
+		})
+	}
+}
+
+// TestDiskCacheOffOnlyForExplicitZeroCacheMaxBytes starts the handlers
+// once with cache_max_bytes omitted and once with cache_max_bytes: 0,
+// and checks by whether the cache directories were created that the
+// disk cache is on in the first case and off in the second.
+func TestDiskCacheOffOnlyForExplicitZeroCacheMaxBytes(t *testing.T) {
+	t.Parallel()
+
+	cases := []struct {
+		name                  string
+		cacheMaxBytesExplicit bool
+		wantDiskCache         bool
+	}{
+		{name: "cache_max_bytes omitted", cacheMaxBytesExplicit: false, wantDiskCache: true},
+		{name: "cache_max_bytes: 0", cacheMaxBytesExplicit: true, wantDiskCache: false},
+	}
+
+	for _, tc := range cases {
+		t.Run(tc.name, func(t *testing.T) {
+			t.Parallel()
+
+			stateDir := t.TempDir()
+			cfg := &config.Config{
+				SigningKey:            testSigningKey,
+				StateDir:              stateDir,
+				DBURL:                 "file:" + filepath.Join(stateDir, "state.sqlite3"),
+				CacheMaxBytes:         0,
+				CacheMaxBytesExplicit: tc.cacheMaxBytesExplicit,
+			}
+
+			lc := fxtest.NewLifecycle(t)
+
+			log, err := logger.New(lc, logger.Params{Globals: &globals.Globals{}})
+			if err != nil {
+				t.Fatalf("logger.New() error = %v", err)
+			}
+
+			db, err := database.New(lc, database.Params{Logger: log, Config: cfg})
+			if err != nil {
+				t.Fatalf("database.New() error = %v", err)
+			}
+
+			_, err = New(lc, Params{Logger: log, Database: db, Config: cfg})
+			if err != nil {
+				t.Fatalf("New() error = %v", err)
+			}
+
+			lc.RequireStart()
+			t.Cleanup(lc.RequireStop)
+
+			_, err = os.Stat(filepath.Join(stateDir, "cache", "variants"))
+			gotDiskCache := err == nil
+
+			if gotDiskCache != tc.wantDiskCache {
+				t.Errorf("cache directories created = %v, want %v",
+					gotDiskCache, tc.wantDiskCache)
+			}
+		})
+	}
+}

internal/handlers/csrf.go

@@ -9,10 +9,15 @@ import (
 	"sneak.berlin/go/pixa/internal/seal"
 )
 
+// csrfKeySalt provides domain separation for the CSRF authentication key,
+// derived from the signing key so tokens survive restarts without extra
+// configuration and never reuse the session or encrypted-URL key material.
+const csrfKeySalt = "pixa-csrf-v1"
+
 // newCSRFProtect builds the CSRF-protection middleware for the
 // state-mutating HTML form routes. The token lives in its own cookie,
 // independent of the session cookie, so it also protects the login POST
-// where no session exists yet (login CSRF). key signs that cookie.
+// where no session exists yet (login CSRF).
 //
 // When plaintext is true (local HTTP development), requests are marked
 // plaintext so the library neither demands an https Referer nor sets a
@@ -20,8 +25,13 @@ import (
 // service runs behind a TLS-terminating proxy, so plaintext is false and
 // the library enforces its https Referer origin check.
 func newCSRFProtect(
-	key [seal.KeySize]byte, plaintext bool,
+	signingKey string, plaintext bool,
-) func(http.Handler) http.Handler {
+) (func(http.Handler) http.Handler, error) {
+	key, err := seal.DeriveKey([]byte(signingKey), csrfKeySalt)
+	if err != nil {
+		return nil, err
+	}
+
 	protect := csrf.Protect(
 		key[:],
 		csrf.Path("/"),
@@ -30,7 +40,7 @@ func newCSRFProtect(
 	)
 
 	if !plaintext {
-		return protect
+		return protect, nil
 	}
 
 	return func(next http.Handler) http.Handler {
@@ -39,7 +49,7 @@ func newCSRFProtect(
 		return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
 			protected.ServeHTTP(w, csrf.PlaintextHTTPRequest(r))
 		})
-	}
+	}, nil
 }
 
 // CSRF returns the CSRF-protection middleware for the login and generator

internal/handlers/fetcher_internal_test.go

@@ -0,0 +1,61 @@
+package handlers
+
+import (
+	"net/http"
+	"net/netip"
+	"path/filepath"
+	"testing"
+	"time"
+
+	"github.com/go-chi/chi/v5"
+	"go.uber.org/fx"
+	"go.uber.org/fx/fxtest"
+
+	"sneak.berlin/go/pixa/internal/config"
+	"sneak.berlin/go/pixa/internal/database"
+	"sneak.berlin/go/pixa/internal/globals"
+	"sneak.berlin/go/pixa/internal/healthcheck"
+	"sneak.berlin/go/pixa/internal/logger"
+)
+
+// TestHandlersBuildTheirOwnFetcherWhenNoneIsProvided builds the handlers as
+// pixad does, in an fx app that provides no fetcher, and requests an image
+// from 192.0.2.10, which is on the allowlist and in blocked_networks. The URL
+// check accepts that address; only the dialer that refuses internal
+// addresses checks blocked_networks, so the answer is 403 only if the
+// fetcher the handlers build from the config connects with that dialer. Any
+// other dialer would try to connect until the upstream fetch timeout, which
+// is short so that the test then fails quickly.
+func TestHandlersBuildTheirOwnFetcherWhenNoneIsProvided(t *testing.T) {
+	t.Parallel()
+
+	const host = "192.0.2.10"
+
+	stateDir := t.TempDir()
+	cfg := &config.Config{
+		SigningKey:           testSigningKey,
+		StateDir:             stateDir,
+		DBURL:                "file:" + filepath.Join(stateDir, "state.sqlite3"),
+		AllowlistHosts:       []string{host},
+		BlockedNetworks:      []netip.Prefix{netip.MustParsePrefix("192.0.2.0/24")},
+		UpstreamFetchTimeout: 2 * time.Second,
+		// With no connection slots, the fetch would fail before dialing.
+		UpstreamConnections: config.DefaultUpstreamConnections,
+	}
+
+	var h *Handlers
+
+	app := fxtest.New(t,
+		fx.Supply(cfg),
+		fx.Provide(globals.New, logger.New, database.New, healthcheck.New, New),
+		fx.Populate(&h),
+	)
+	app.RequireStart()
+	t.Cleanup(app.RequireStop)
+
+	r := chi.NewRouter()
+	r.Get("/v1/image/*", h.HandleImage())
+
+	rec := sendGet(t, r, photoURL(host))
+	checkErrorBody(t, rec, http.StatusForbidden, "forbidden")
+}

internal/handlers/first_bytes_internal_test.go

@@ -12,15 +12,10 @@ import (
 	"time"
 
 	"github.com/go-chi/chi/v5"
-	"github.com/prometheus/client_golang/prometheus"
-	"sneak.berlin/go/pixa/internal/allowlist"
 	"sneak.berlin/go/pixa/internal/encurl"
 	"sneak.berlin/go/pixa/internal/httpfetcher"
-	"sneak.berlin/go/pixa/internal/imageprocessor"
 	"sneak.berlin/go/pixa/internal/imagerequest"
 	"sneak.berlin/go/pixa/internal/imgcache"
-	"sneak.berlin/go/pixa/internal/seal"
-	"sneak.berlin/go/pixa/internal/signature"
 )
 
 // mismatchedSourcePath is the path of the source that
@@ -30,9 +25,8 @@ import (
 const mismatchedSourcePath = "/images/source.png"
 
 // newRoutesFetchingThrough returns both image routes, and the Handlers behind
-// them, whose image service fetches through fetcher, and which have
+// them, whose image service fetches through fetcher and has allowlistedHost on
-// allowlistedHost on their allowlist. The handlers and the image service log
+// its allowlist. The handlers and the image service log to log.
-// to log.
 func newRoutesFetchingThrough(
 	t *testing.T, fetcher httpfetcher.Fetcher, log *slog.Logger,
 ) (*Handlers, http.Handler) {
@@ -41,40 +35,28 @@ func newRoutesFetchingThrough(
 	cache, err := imgcache.NewCache(setupTestDB(t), imgcache.CacheConfig{
 		StateDir:    t.TempDir(),
 		NegativeTTL: 5 * time.Minute,
-		MaxBytes:    1 << 30,
+	})
-	}, prometheus.NewRegistry())
 	if err != nil {
 		t.Fatalf("imgcache.NewCache() error = %v", err)
 	}
 
 	svc, err := imgcache.NewService(&imgcache.ServiceConfig{
-		Cache:   cache,
+		Cache:      cache,
-		Fetcher: fetcher,
+		Fetcher:    fetcher,
-		Processor: imageprocessor.New(imageprocessor.Params{
+		SigningKey: testSigningKey,
-			MaxInputBytes:           50 << 20,
+		Allowlist:  []string{allowlistedHost},
-			MaxConcurrentProcessing: 2,
+		Logger:     log,
-		}),
-		MaxResponseSize: 50 << 20,
-		Logger:          log,
 	})
 	if err != nil {
 		t.Fatalf("imgcache.NewService() error = %v", err)
 	}
 
-	keys, err := seal.DeriveKeys(testSigningKey)
+	encGen, err := encurl.NewGenerator(testSigningKey)
 	if err != nil {
-		t.Fatalf("seal.DeriveKeys() error = %v", err)
+		t.Fatalf("encurl.NewGenerator() error = %v", err)
 	}
 
-	encGen := encurl.NewGenerator(keys.EncryptedURLKey)
+	h := &Handlers{log: log, imgSvc: svc, encGen: encGen}
-
-	h := &Handlers{
-		log:       log,
-		imgSvc:    svc,
-		encGen:    encGen,
-		signer:    signature.New(testSigningKey),
-		allowlist: allowlist.New([]string{allowlistedHost}),
-	}
 
 	r := chi.NewRouter()
 	r.Get("/v1/image/*", h.HandleImage())
@@ -103,7 +85,7 @@ func mismatchedSourceEncURL(t *testing.T, h *Handlers) string {
 			Size:   imagerequest.Size{Width: 50, Height: 50},
 			Format: imagerequest.FormatJPEG,
 		},
-	}, time.Time{})
+	})
 	if err != nil {
 		t.Fatalf("Generate() error = %v", err)
 	}

internal/handlers/format_auto.go

@@ -18,17 +18,16 @@ var (
 		"image/jxl, image/avif, image/webp or image/jpeg")
 )
 
-// chooseAutoFormat returns the format to serve for the requested format: for
+// chooseAutoFormat replaces the format auto in req with the format
-// the format auto, the format formatForAccept chooses from r's Accept header,
+// formatForAccept chooses from r's Accept header, and adds Vary: Accept to the
-// and any other format as it is. For auto it adds Vary: Accept to the
 // response, which then depends on that header. It answers 400 for an Accept
 // header that is not valid and 406 for one that allows none of the formats,
-// and reports whether the image can be served.
+// and reports whether req can be served. Any other format is left as it is.
 func (s *Handlers) chooseAutoFormat(
-	w http.ResponseWriter, r *http.Request, requested imagerequest.Format,
+	w http.ResponseWriter, r *http.Request, req *imagerequest.Request,
-) (imagerequest.Format, bool) {
+) bool {
-	if requested != imagerequest.FormatAuto {
+	if req.Transform.Format != imagerequest.FormatAuto {
-		return requested, true
+		return true
 	}
 
 	w.Header().Add("Vary", "Accept")
@@ -37,16 +36,18 @@ func (s *Handlers) chooseAutoFormat(
 	if errors.Is(err, errNotAcceptable) {
 		s.respondError(w, err.Error(), http.StatusNotAcceptable)
 
-		return "", false
+		return false
 	}
 
 	if err != nil {
 		s.respondError(w, err.Error(), http.StatusBadRequest)
 
-		return "", false
+		return false
 	}
 
-	return format, true
+	req.Transform.Format = format
+
+	return true
 }
 
 // formatForAccept returns the format an auto URL is served in for the Accept

internal/handlers/format_auto_internal_test.go

@@ -75,13 +75,11 @@ func TestFormatForAccept(t *testing.T) {
 
 // autoPhotoURLs returns a signed /v1/image/ URL and an encrypted /v1/e/ URL,
 // both valid for a minute, for the JPEG at photoPath on signedHost at 50x50 in
-// the format auto. The first is signed with testSigningKey, the key the
+// the format auto, made with h's image service and generator.
-// handlers that newSignedHostServer builds check signatures with. The second
-// is made with h's generator.
 func autoPhotoURLs(t *testing.T, h *Handlers) (string, string) {
 	t.Helper()
 
-	signedURL := generateSignedURL(testSigningKey, &imagerequest.Request{
+	signedURL, err := h.imgSvc.GenerateSignedURL("", &imagerequest.Request{
 		Source: imagerequest.Source{
 			Host: signedHost,
 			Path: photoPath,
@@ -91,6 +89,9 @@ func autoPhotoURLs(t *testing.T, h *Handlers) (string, string) {
 			Format: imagerequest.FormatAuto,
 		},
 	}, time.Minute)
+	if err != nil {
+		t.Fatalf("GenerateSignedURL() error = %v", err)
+	}
 
 	token, err := h.encGen.Generate(&imagerequest.Request{
 		Source: imagerequest.Source{
@@ -101,7 +102,8 @@ func autoPhotoURLs(t *testing.T, h *Handlers) (string, string) {
 			Size:   imagerequest.Size{Width: 50, Height: 50},
 			Format: imagerequest.FormatAuto,
 		},
-	}, time.Now().Add(time.Minute))
+		Expires: time.Now().Add(time.Minute),
+	})
 	if err != nil {
 		t.Fatalf("Generate() error = %v", err)
 	}
@@ -186,9 +188,9 @@ func TestFormatAuto_ChosenFromAccept(t *testing.T) {
 func TestFormatAuto_SignatureCoversAuto(t *testing.T) {
 	t.Parallel()
 
-	_, srv := newSignedHostServer(t, slog.New(slog.DiscardHandler))
+	h, srv := newSignedHostServer(t, slog.New(slog.DiscardHandler))
 
-	signedForAVIF := generateSignedURL(testSigningKey, &imagerequest.Request{
+	signedForAVIF, err := h.imgSvc.GenerateSignedURL("", &imagerequest.Request{
 		Source: imagerequest.Source{
 			Host: signedHost,
 			Path: photoPath,
@@ -198,6 +200,9 @@ func TestFormatAuto_SignatureCoversAuto(t *testing.T) {
 			Format: imagerequest.FormatAVIF,
 		},
 	}, time.Minute)
+	if err != nil {
+		t.Fatalf("GenerateSignedURL() error = %v", err)
+	}
 
 	target := strings.Replace(signedForAVIF, "/50x50.avif?", "/50x50.auto?", 1)
 	if target == signedForAVIF {

internal/handlers/handlers.go

@@ -4,6 +4,7 @@ package handlers
 import (
 	"context"
 	"encoding/json"
+	"errors"
 	"log/slog"
 	"net/http"
 	"time"
@@ -11,13 +12,13 @@ import (
 	"go.uber.org/fx"
 	"sneak.berlin/go/pixa/internal/allowlist"
 	"sneak.berlin/go/pixa/internal/config"
+	"sneak.berlin/go/pixa/internal/database"
 	"sneak.berlin/go/pixa/internal/encurl"
 	"sneak.berlin/go/pixa/internal/healthcheck"
+	"sneak.berlin/go/pixa/internal/httpfetcher"
 	"sneak.berlin/go/pixa/internal/imgcache"
 	"sneak.berlin/go/pixa/internal/logger"
-	"sneak.berlin/go/pixa/internal/seal"
 	"sneak.berlin/go/pixa/internal/session"
-	"sneak.berlin/go/pixa/internal/signature"
 )
 
 // Params defines dependencies for Handlers.
@@ -26,48 +27,70 @@ type Params struct {
 
 	Logger      *logger.Logger
 	Healthcheck *healthcheck.Healthcheck
+	Database    *database.Database
 	Config      *config.Config
-	Keys        seal.Keys
 
-	ImageService          *imgcache.Service
+	// Fetcher, when provided, fetches upstream images in place of the
-	SessionManager        *session.Manager
+	// fetcher the handlers build from the config. Only tests provide one;
-	EncryptedURLGenerator *encurl.Generator
+	// pixad does not.
+	Fetcher httpfetcher.Fetcher `optional:"true"`
 }
 
 // Handlers provides HTTP request handlers.
 type Handlers struct {
 	log         *slog.Logger
 	hc          *healthcheck.Healthcheck
+	db          *database.Database
 	config      *config.Config
+	fetcher     httpfetcher.Fetcher
 	imgSvc      *imgcache.Service
+	imgCache    *imgcache.Cache
 	sessMgr     *session.Manager
 	encGen      *encurl.Generator
 	csrfProtect func(http.Handler) http.Handler
 
-	// signer checks the signature of a /v1/image/ URL with signing_key.
-	signer *signature.Signer
-	// allowlist matches the hosts of allowlist_hosts, which a /v1/image/
-	// URL may name without a signature.
-	allowlist *allowlist.HostAllowList
 	// refererBlocklist matches the hosts of referer_blocklist; its IsAllowed
 	// reports whether a URL's host is on that list.
 	refererBlocklist *allowlist.HostAllowList
 }
 
 // New creates a new Handlers instance.
-func New(_ fx.Lifecycle, params Params) *Handlers {
+func New(lc fx.Lifecycle, params Params) (*Handlers, error) {
-	return &Handlers{
+	csrfProtect, err := newCSRFProtect(params.Config.SigningKey, params.Config.Debug)
+	if err != nil {
+		return nil, err
+	}
+
+	s := &Handlers{
 		log:              params.Logger.Get(),
 		hc:               params.Healthcheck,
+		db:               params.Database,
 		config:           params.Config,
-		imgSvc:           params.ImageService,
+		fetcher:          params.Fetcher,
-		sessMgr:          params.SessionManager,
+		csrfProtect:      csrfProtect,
-		encGen:           params.EncryptedURLGenerator,
-		csrfProtect:      newCSRFProtect(params.Keys.CSRFKey, params.Config.Debug),
-		signer:           signature.New(params.Config.SigningKey),
-		allowlist:        allowlist.New(params.Config.AllowlistHosts),
 		refererBlocklist: allowlist.New(params.Config.RefererBlocklist),
 	}
+
+	lc.Append(fx.Hook{
+		//nolint:contextcheck // the cache's goroutines outlive OnStart; OnStop stops them
+		OnStart: func(_ context.Context) error {
+			return s.initImageService()
+		},
+		// The pending counts are written here, before the database's stop
+		// hook closes it.
+		OnStop: func(ctx context.Context) error {
+			if s.imgCache == nil {
+				return nil
+			}
+
+			return errors.Join(
+				s.imgCache.StopEviction(ctx),
+				s.imgCache.StopPendingCountWrites(ctx),
+			)
+		},
+	})
+
+	return s, nil
 }
 
 // WaitForProcessing waits until no image is being processed, or until ctx
@@ -76,6 +99,94 @@ func (s *Handlers) WaitForProcessing(ctx context.Context) int {
 	return s.imgSvc.WaitForProcessing(ctx)
 }
 
+// newCacheConfig builds the image cache's configuration from cfg.
+// cache_max_bytes: 0 disables the disk cache entirely; any other value
+// is the eviction limit in bytes; when it is omitted, the cache works
+// out the default limit itself.
+func newCacheConfig(cfg *config.Config, log *slog.Logger) imgcache.CacheConfig {
+	return imgcache.CacheConfig{
+		StateDir:           cfg.StateDir,
+		NegativeTTL:        imgcache.DefaultNegativeTTL,
+		MaxBytes:           cfg.CacheMaxBytes,
+		UseDefaultMaxBytes: !cfg.CacheMaxBytesExplicit,
+		DisableDiskCache:   cfg.CacheMaxBytesExplicit && cfg.CacheMaxBytes == 0,
+		Logger:             log,
+	}
+}
+
+// initImageService initializes the image cache and service.
+func (s *Handlers) initImageService() error {
+	cache, err := imgcache.NewCache(s.db.DB(), newCacheConfig(s.config, s.log))
+	if err != nil {
+		return err
+	}
+
+	s.imgCache = cache
+
+	// Background eviction: startup reconciliation, then periodic and
+	// write-pressure passes, and the removal of expired negative cache
+	// rows, which is all it does when the disk cache is disabled.
+	cache.StartEviction(imgcache.DefaultEvictionInterval)
+
+	// Writes the counts requests could not write by their deadline
+	cache.StartPendingCountWrites()
+
+	// Create the fetcher config
+	fetcherCfg := httpfetcher.DefaultConfig()
+	fetcherCfg.AllowHTTP = s.config.AllowHTTP
+	fetcherCfg.Timeout = s.config.UpstreamFetchTimeout
+	fetcherCfg.MaxResponseSize = s.config.UpstreamMaxResponseSize
+
+	if s.config.UpstreamConnectionsPerHost > 0 {
+		fetcherCfg.MaxConnectionsPerHost = s.config.UpstreamConnectionsPerHost
+	}
+
+	fetcherCfg.MaxConnections = s.config.UpstreamConnections
+	fetcherCfg.BlockedNetworks = s.config.BlockedNetworks
+
+	// Create the service. With no fetcher provided, it builds its own from
+	// fetcherCfg.
+	svc, err := imgcache.NewService(&imgcache.ServiceConfig{
+		Cache:                   cache,
+		FetcherConfig:           fetcherCfg,
+		Fetcher:                 s.fetcher,
+		SigningKey:              s.config.SigningKey,
+		Allowlist:               s.config.AllowlistHosts,
+		MaxConcurrentProcessing: s.config.MaxConcurrentProcessing,
+		Logger:                  s.log,
+	})
+	if err != nil {
+		return err
+	}
+
+	s.imgSvc = svc
+	s.log.Info("image service initialized")
+
+	// Initialize session manager (signing key is validated at config load
+	// time). Session cookies are always Secure/HttpOnly/SameSite=Strict.
+	// The random value that each logout replaces is saved in the database,
+	// so it outlasts a restart.
+	sessMgr, err := session.NewManagerWithDatabase(
+		context.Background(), s.config.SigningKey, s.db.DB())
+	if err != nil {
+		return err
+	}
+
+	s.sessMgr = sessMgr
+
+	// Initialize encrypted URL generator
+	encGen, err := encurl.NewGenerator(s.config.SigningKey)
+	if err != nil {
+		return err
+	}
+
+	s.encGen = encGen
+
+	s.log.Info("session manager and URL generator initialized")
+
+	return nil
+}
+
 func (s *Handlers) respondJSON(w http.ResponseWriter, data any, status int) {
 	w.Header().Set("Content-Type", "application/json")
 	w.WriteHeader(status)

internal/handlers/handlers_internal_test.go

@@ -18,19 +18,15 @@ import (
 	"time"
 
 	"github.com/go-chi/chi/v5"
-	"github.com/prometheus/client_golang/prometheus"
-	"sneak.berlin/go/pixa/internal/allowlist"
 	"sneak.berlin/go/pixa/internal/database"
 	"sneak.berlin/go/pixa/internal/httpfetcher"
-	"sneak.berlin/go/pixa/internal/imageprocessor"
-	"sneak.berlin/go/pixa/internal/imagerequest"
 	"sneak.berlin/go/pixa/internal/imgcache"
-	"sneak.berlin/go/pixa/internal/signature"
 )
 
 // testFixtures contains test data.
 type testFixtures struct {
 	handler  *Handlers
+	service  *imgcache.Service
 	goodHost string
 }
 
@@ -52,34 +48,29 @@ func setupTestHandler(t *testing.T) *testFixtures {
 	cache, err := imgcache.NewCache(db, imgcache.CacheConfig{
 		StateDir:    tmpDir,
 		NegativeTTL: 5 * time.Minute,
-		MaxBytes:    1 << 30,
+	})
-	}, prometheus.NewRegistry())
 	if err != nil {
 		t.Fatalf("failed to create cache: %v", err)
 	}
 
 	svc, err := imgcache.NewService(&imgcache.ServiceConfig{
-		Cache:   cache,
+		Cache:      cache,
-		Fetcher: newMockFetcher(mockFS),
+		Fetcher:    newMockFetcher(mockFS),
-		Processor: imageprocessor.New(imageprocessor.Params{
+		SigningKey: "test-signing-key-must-be-32-chars",
-			MaxInputBytes:           50 << 20,
+		Allowlist:  []string{goodHost},
-			MaxConcurrentProcessing: 2,
-		}),
-		MaxResponseSize: 50 << 20,
 	})
 	if err != nil {
 		t.Fatalf("failed to create service: %v", err)
 	}
 
 	h := &Handlers{
-		imgSvc:    svc,
+		imgSvc: svc,
-		log:       slog.Default(),
+		log:    slog.Default(),
-		signer:    signature.New("test-signing-key-must-be-32-chars"),
-		allowlist: allowlist.New([]string{goodHost}),
 	}
 
 	return &testFixtures{
 		handler:  h,
+		service:  svc,
 		goodHost: goodHost,
 	}
 }
@@ -130,9 +121,10 @@ func newMockFetcher(fs fs.FS) *mockFetcher {
 }
 
 func (f *mockFetcher) Fetch(
-	_ context.Context, source imagerequest.Source,
+	_ context.Context, url string,
 ) (*httpfetcher.FetchResult, error) {
-	path := source.Host + source.Path
+	// Remove https:// prefix
+	path := url[8:] // Remove "https://"
 
 	data, err := fs.ReadFile(f.fs, path)
 	if err != nil {

internal/handlers/image.go

@@ -7,7 +7,6 @@ import (
 	"net/http"
 	"net/url"
 	"strconv"
-	"strings"
 	"time"
 
 	"github.com/go-chi/chi/v5/middleware"
@@ -28,13 +27,13 @@ func (s *Handlers) HandleImage() http.HandlerFunc {
 			return
 		}
 
-		req, sig, expires, ok := s.parseImageRequest(w, r)
+		req, ok := s.parseImageRequest(w, r)
 		if !ok {
 			return
 		}
 
 		// Validate signature if required
-		err := s.validateSignature(req, sig, expires)
+		err := s.imgSvc.ValidateRequest(req)
 		if err != nil {
 			s.log.Warn("signature validation failed",
 				"host", req.Source.Host,
@@ -46,7 +45,27 @@ func (s *Handlers) HandleImage() http.HandlerFunc {
 			return
 		}
 
-		s.serveImage(w, r, req, expires)
+		// The signature covers the format auto, not the format chosen
+		if !s.chooseAutoFormat(w, r, req) {
+			return
+		}
+
+		// Get cache key for logging
+		cacheKey := imagerequest.CacheKey(req)
+
+		// Get the image (from cache or fetch/process)
+		startTime := time.Now()
+
+		resp, err := s.imgSvc.Get(r.Context(), req)
+		if err != nil {
+			s.respondImageError(w, req, err)
+
+			return
+		}
+
+		defer func() { _ = resp.Content.Close() }()
+
+		s.writeImageResponse(w, r, req, resp, cacheKey, startTime)
 	}
 }
 
@@ -63,11 +82,11 @@ func (s *Handlers) HandleRobotsTxt() http.HandlerFunc {
 }
 
 // parseImageRequest parses the image URL's path and query parameters into
-// an image request, and returns it with the URL's signature and expiry. On
+// an image request. On invalid input it writes an error response and
-// invalid input it writes an error response and returns false.
+// returns false.
 func (s *Handlers) parseImageRequest(
 	w http.ResponseWriter, r *http.Request,
-) (*imagerequest.Request, string, time.Time, bool) {
+) (*imagerequest.Request, bool) {
 	// The path is parsed decoded once, as r.URL.Path holds it, and not from
 	// chi's wildcard: chi routes on the path as the client wrote it whenever
 	// that is not Go's own encoding of it, so its wildcard would still hold
@@ -81,7 +100,7 @@ func (s *Handlers) parseImageRequest(
 		)
 		s.respondError(w, "invalid image URL: "+err.Error(), http.StatusBadRequest)
 
-		return nil, "", time.Time{}, false
+		return nil, false
 	}
 
 	// Parse signature params from query string. r.URL.Query() would silently
@@ -94,7 +113,7 @@ func (s *Handlers) parseImageRequest(
 		s.respondError(w, fmt.Sprintf("invalid query string %q: %v",
 			r.URL.RawQuery, err), http.StatusBadRequest)
 
-		return nil, "", time.Time{}, false
+		return nil, false
 	}
 
 	for name, values := range query {
@@ -102,15 +121,17 @@ func (s *Handlers) parseImageRequest(
 			s.respondError(w, fmt.Sprintf("invalid %s: given more than once",
 				name), http.StatusBadRequest)
 
-			return nil, "", time.Time{}, false
+			return nil, false
 		}
 	}
 
-	expires, err := parseExpires(query)
+	req.Signature = query.Get("sig")
+
+	req.Expires, err = parseExpires(query)
 	if err != nil {
 		s.respondError(w, err.Error(), http.StatusBadRequest)
 
-		return nil, "", time.Time{}, false
+		return nil, false
 	}
 
 	// Parse optional quality and fit params. Only a q missing from the URL is
@@ -122,7 +143,7 @@ func (s *Handlers) parseImageRequest(
 		s.respondError(w, `invalid q: not a number, got ""`,
 			http.StatusBadRequest)
 
-		return nil, "", time.Time{}, false
+		return nil, false
 	}
 
 	req.Transform.Quality, err = parseFormInt(query, "q",
@@ -131,7 +152,7 @@ func (s *Handlers) parseImageRequest(
 		s.respondError(w, fmt.Sprintf("%v, got %q", err, qStr),
 			http.StatusBadRequest)
 
-		return nil, "", time.Time{}, false
+		return nil, false
 	}
 
 	// Only a fit missing from the URL is cover. A fit in the URL that is not a
@@ -141,7 +162,7 @@ func (s *Handlers) parseImageRequest(
 	if query.Has("fit") && req.Transform.FitMode == "" {
 		s.respondError(w, `invalid fit: not a fit mode, got ""`, http.StatusBadRequest)
 
-		return nil, "", time.Time{}, false
+		return nil, false
 	}
 
 	// Default fit mode if not set
@@ -158,40 +179,15 @@ func (s *Handlers) parseImageRequest(
 		s.respondError(w, "invalid image request: "+err.Error(),
 			http.StatusBadRequest)
 
-		return nil, "", time.Time{}, false
+		return nil, false
 	}
 
-	return req, query.Get("sig"), expires, true
+	return req, true
-}
-
-// validateSignature returns nil when the source host of req is on
-// allowlist_hosts and names no port. Otherwise it verifies sig, the URL's
-// signature of req, and expires, the URL's expiry.
-func (s *Handlers) validateSignature(
-	req *imagerequest.Request, sig string, expires time.Time,
-) error {
-	// An allowlist entry names no port, so a host followed by ":", with or
-	// without a port, needs a signature. Go's URL type reads the port as the
-	// fetcher does: in an IPv6 address written without brackets, such as
-	// 2001:db8::1:8443, it is what follows the last ":". This is checked here
-	// and not in allowlist.IsAllowed, which also matches referer_blocklist,
-	// where a Referer must be blocked whatever its port. Port and IsAllowed
-	// read only the host of a URL.
-	sourceURL := &url.URL{Host: req.Source.Host}
-	hostHasPort := sourceURL.Port() != "" ||
-		strings.HasSuffix(sourceURL.Host, ":")
-
-	if !hostHasPort && s.allowlist.IsAllowed(sourceURL) {
-		return nil
-	}
-
-	// Signature required for non-allowed hosts
-	return s.signer.Verify(req, sig, expires)
 }
 
 // parseExpires reads the exp query parameter, a Unix time in seconds. An exp
-// missing from the URL gives the zero time, which the signature check refuses
+// missing from the URL gives the zero time, which the signature check takes
-// as a missing exp. An exp in the URL that is not a whole number, an empty
+// as no expiration. An exp in the URL that is not a whole number, an empty
 // one included, is an error naming exp and the value.
 func parseExpires(query url.Values) (time.Time, error) {
 	if !query.Has("exp") {
@@ -219,11 +215,7 @@ func (s *Handlers) HandleTooManyRequests() http.HandlerFunc {
 	}
 }
 
-// respondImageError answers err, the image service's error for req, on both
+// respondImageError maps image retrieval errors to HTTP responses.
-// image routes. It logs a 500 at error level, as pixa's own failure. It logs
-// the other answers at warn level, since the client, the upstream or a full
-// limit caused them. The 415 for a source refused for its first bytes is not
-// logged here, because the image service logs it.
 func (s *Handlers) respondImageError(
 	w http.ResponseWriter, req *imagerequest.Request, err error,
 ) {
@@ -239,53 +231,59 @@ func (s *Handlers) respondImageError(
 		return
 	}
 
-	var (
+	s.log.Error("failed to get image",
-		status  int
-		message string
-	)
-
-	switch {
-	case errors.Is(err, httpfetcher.ErrSSRFBlocked):
-		status, message = http.StatusForbidden, "forbidden"
-	case errors.Is(err, httpfetcher.ErrUpstreamError):
-		status, message = http.StatusBadGateway, "upstream error"
-	case errors.Is(err, httpfetcher.ErrUpstreamTimeout):
-		status, message = http.StatusGatewayTimeout, "upstream timeout"
-	case errors.Is(err, httpfetcher.ErrTooManyConnections),
-		errors.Is(err, imageprocessor.ErrTooManyImages),
-		errors.Is(err, imgcache.ErrProcessingTimedOut):
-		status = http.StatusServiceUnavailable
-		message = "server busy, try again later"
-	case errors.Is(err, httpfetcher.ErrInvalidContentType),
-		errors.Is(err, imageprocessor.ErrUnsupportedInputFormat):
-		status = http.StatusUnsupportedMediaType
-		message = "unsupported source format"
-	case errors.Is(err, imageprocessor.ErrOutputTooLarge):
-		status = http.StatusBadRequest
-		message = fmt.Sprintf("size too large for the source: the side "+
-			"computed from its proportions is above %d",
-			imageprocessor.MaxOutputDimension)
-	case errors.Is(err, imageprocessor.ErrAVIFOutputTooLarge):
-		status = http.StatusBadRequest
-		message = fmt.Sprintf("size too large for AVIF: the output "+
-			"would be above %d pixels", imageprocessor.MaxAVIFOutputPixels)
-	default:
-		s.log.Error("failed to get image",
-			"host", req.Source.Host,
-			"path", req.Source.Path,
-			"error", err,
-		)
-		s.respondError(w, "internal error", http.StatusInternalServerError)
-
-		return
-	}
-
-	s.log.Warn("failed to get image",
 		"host", req.Source.Host,
 		"path", req.Source.Path,
 		"error", err,
 	)
-	s.respondError(w, message, status)
+
+	// Check for specific error types
+	if errors.Is(err, httpfetcher.ErrSSRFBlocked) {
+		s.respondError(w, "forbidden", http.StatusForbidden)
+
+		return
+	}
+
+	if errors.Is(err, httpfetcher.ErrUpstreamError) {
+		s.respondError(w, "upstream error", http.StatusBadGateway)
+
+		return
+	}
+
+	if errors.Is(err, httpfetcher.ErrTooManyConnections) ||
+		errors.Is(err, imageprocessor.ErrTooManyImages) ||
+		errors.Is(err, imgcache.ErrProcessingTimedOut) {
+		s.respondError(w, "server busy, try again later",
+			http.StatusServiceUnavailable)
+
+		return
+	}
+
+	if errors.Is(err, httpfetcher.ErrInvalidContentType) ||
+		errors.Is(err, imageprocessor.ErrUnsupportedInputFormat) {
+		s.respondError(w, "unsupported source format",
+			http.StatusUnsupportedMediaType)
+
+		return
+	}
+
+	if errors.Is(err, imageprocessor.ErrOutputTooLarge) {
+		s.respondError(w, fmt.Sprintf("size too large for the source: the side "+
+			"computed from its proportions is above %d",
+			imageprocessor.MaxOutputDimension), http.StatusBadRequest)
+
+		return
+	}
+
+	if errors.Is(err, imageprocessor.ErrAVIFOutputTooLarge) {
+		s.respondError(w, fmt.Sprintf("size too large for AVIF: the output "+
+			"would be above %d pixels", imageprocessor.MaxAVIFOutputPixels),
+			http.StatusBadRequest)
+
+		return
+	}
+
+	s.respondError(w, "internal error", http.StatusInternalServerError)
 }
 
 // cacheControl returns the Cache-Control header for an image served through a
@@ -342,34 +340,13 @@ func notModified(w http.ResponseWriter, r *http.Request, etag string) bool {
 	return true
 }
 
-// serveImage serves the image that req asks for. Both image routes call it
+// writeImageResponse writes headers and streams the image content,
-// once they have the request and its URL's expiry, which ends the
+// handling conditional and HEAD requests.
-// Cache-Control max-age. It chooses the format for the format auto, answers
+func (s *Handlers) writeImageResponse(
-// If-None-Match with 304 and HEAD with the headers only, and answers an error
-// of the image service with respondImageError.
-func (s *Handlers) serveImage(
 	w http.ResponseWriter, r *http.Request,
-	req *imagerequest.Request, expires time.Time,
+	req *imagerequest.Request, resp *imgcache.ImageResponse,
+	cacheKey imagerequest.VariantKey, startTime time.Time,
 ) {
-	// The signature or the token covers the format auto, not the format chosen
-	format, ok := s.chooseAutoFormat(w, r, req.Transform.Format)
-	if !ok {
-		return
-	}
-
-	req.Transform.Format = format
-
-	startTime := time.Now()
-
-	resp, err := s.imgSvc.Get(r.Context(), req)
-	if err != nil {
-		s.respondImageError(w, req, err)
-
-		return
-	}
-
-	defer func() { _ = resp.Content.Close() }()
-
 	// Set response headers
 	w.Header().Set("Content-Type", resp.ContentType)
 
@@ -378,7 +355,7 @@ func (s *Handlers) serveImage(
 	}
 
 	// Cache control headers
-	w.Header().Set("Cache-Control", cacheControl(expires))
+	w.Header().Set("Cache-Control", cacheControl(req.Expires))
 	w.Header().Set("X-Pixa-Cache", string(resp.CacheStatus))
 
 	if notModified(w, r, resp.ETag) {
@@ -395,28 +372,18 @@ func (s *Handlers) serveImage(
 	// Stream the response
 	w.WriteHeader(http.StatusOK)
 
-	// A failed write is most often a client that has gone away, not pixa's
-	// failure, so it is logged at warn level
 	servedBytes, err := io.Copy(w, resp.Content)
 	if err != nil {
-		s.log.Warn("failed to write response",
+		s.log.Error("failed to write response",
-			"request_id", middleware.GetReqID(r.Context()),
-			"host", req.Source.Host,
-			"path", req.Source.Path,
-			"served_bytes", servedBytes,
 			"error", err,
 		)
-
-		return
 	}
 
 	// Log cache status and timing after serving
 	duration := time.Since(startTime)
 	s.log.Info("image served",
 		"request_id", middleware.GetReqID(r.Context()),
-		"cache_key", imagerequest.CacheKey(req),
+		"cache_key", cacheKey,
-		"host", req.Source.Host,
-		"path", req.Source.Path,
 		"cache_status", resp.CacheStatus,
 		"duration_ms", duration.Milliseconds(),
 		"format", req.Transform.Format,

internal/handlers/image_cache_control_internal_test.go

@@ -12,15 +12,10 @@ import (
 	"time"
 
 	"github.com/go-chi/chi/v5"
-	"github.com/prometheus/client_golang/prometheus"
 
-	"sneak.berlin/go/pixa/internal/allowlist"
 	"sneak.berlin/go/pixa/internal/encurl"
-	"sneak.berlin/go/pixa/internal/imageprocessor"
 	"sneak.berlin/go/pixa/internal/imagerequest"
 	"sneak.berlin/go/pixa/internal/imgcache"
-	"sneak.berlin/go/pixa/internal/seal"
-	"sneak.berlin/go/pixa/internal/signature"
 )
 
 // photoPath is the path of the JPEG that newSignedHostServer serves.
@@ -38,8 +33,7 @@ func newSignedHostServer(
 	cache, err := imgcache.NewCache(setupTestDB(t), imgcache.CacheConfig{
 		StateDir:    t.TempDir(),
 		NegativeTTL: 5 * time.Minute,
-		MaxBytes:    1 << 30,
+	})
-	}, prometheus.NewRegistry())
 	if err != nil {
 		t.Fatalf("imgcache.NewCache() error = %v", err)
 	}
@@ -51,30 +45,22 @@ func newSignedHostServer(
 		Fetcher: newMockFetcher(fstest.MapFS{
 			signedHost + photoPath: &fstest.MapFile{Data: jpegData},
 		}),
-		Processor: imageprocessor.New(imageprocessor.Params{
+		SigningKey: testSigningKey,
-			MaxInputBytes:           50 << 20,
+		Logger:     log,
-			MaxConcurrentProcessing: 2,
-		}),
-		MaxResponseSize: 50 << 20,
-		Logger:          log,
 	})
 	if err != nil {
 		t.Fatalf("imgcache.NewService() error = %v", err)
 	}
 
-	keys, err := seal.DeriveKeys(testSigningKey)
+	encGen, err := encurl.NewGenerator(testSigningKey)
 	if err != nil {
-		t.Fatalf("seal.DeriveKeys() error = %v", err)
+		t.Fatalf("encurl.NewGenerator() error = %v", err)
 	}
 
-	encGen := encurl.NewGenerator(keys.EncryptedURLKey)
-
 	h := &Handlers{
-		log:       log,
+		log:    log,
-		imgSvc:    svc,
+		imgSvc: svc,
-		encGen:    encGen,
+		encGen: encGen,
-		signer:    signature.New(testSigningKey),
-		allowlist: allowlist.New(nil),
 	}
 
 	r := chi.NewRouter()
@@ -120,9 +106,9 @@ func getMaxAge(t *testing.T, srv http.Handler, target string) int {
 func TestHandleImage_SignedURL_MaxAgeEndsAtExp(t *testing.T) {
 	t.Parallel()
 
-	_, srv := newSignedHostServer(t, slog.New(slog.DiscardHandler))
+	h, srv := newSignedHostServer(t, slog.New(slog.DiscardHandler))
 
-	signedURL := generateSignedURL(testSigningKey, &imagerequest.Request{
+	signedURL, err := h.imgSvc.GenerateSignedURL("", &imagerequest.Request{
 		Source: imagerequest.Source{
 			Host: signedHost,
 			Path: photoPath,
@@ -132,6 +118,9 @@ func TestHandleImage_SignedURL_MaxAgeEndsAtExp(t *testing.T) {
 			Format: imagerequest.FormatJPEG,
 		},
 	}, time.Minute)
+	if err != nil {
+		t.Fatalf("GenerateSignedURL() error = %v", err)
+	}
 
 	maxAge := getMaxAge(t, srv, signedURL)
 	if maxAge < 50 || maxAge > 60 {
@@ -208,7 +197,8 @@ func TestHandleImageEnc_MaxAge(t *testing.T) {
 					Size:   imagerequest.Size{Width: 50, Height: 50},
 					Format: imagerequest.FormatJPEG,
 				},
-			}, tt.expires)
+				Expires: tt.expires,
+			})
 			if err != nil {
 				t.Fatalf("Generate() error = %v", err)
 			}

internal/handlers/image_errors_internal_test.go

@@ -1,27 +1,19 @@
 package handlers
 
 import (
-	"bytes"
-	"context"
 	"encoding/json"
 	"fmt"
 	"image/color"
-	"io"
 	"log/slog"
 	"net/http"
 	"net/http/httptest"
-	"slices"
 	"strings"
 	"testing"
 	"testing/fstest"
 	"time"
 
 	"github.com/go-chi/chi/v5"
-	"github.com/go-chi/chi/v5/middleware"
-	"github.com/prometheus/client_golang/prometheus"
-	"sneak.berlin/go/pixa/internal/allowlist"
 	"sneak.berlin/go/pixa/internal/httpfetcher"
-	"sneak.berlin/go/pixa/internal/imageprocessor"
 	"sneak.berlin/go/pixa/internal/imagerequest"
 	"sneak.berlin/go/pixa/internal/imgcache"
 	"sneak.berlin/go/pixa/internal/signature"
@@ -32,38 +24,29 @@ import (
 const allowlistedHost = "allowed.example.com"
 
 // newImageRoute returns the image route of a Handlers whose service fetches
-// with fetcher, and which checks signatures with testSigningKey.
+// with fetcher and checks signatures with testSigningKey.
 func newImageRoute(t *testing.T, fetcher httpfetcher.Fetcher) http.Handler {
 	t.Helper()
 
 	cache, err := imgcache.NewCache(setupTestDB(t), imgcache.CacheConfig{
 		StateDir:    t.TempDir(),
 		NegativeTTL: 5 * time.Minute,
-		MaxBytes:    1 << 30,
+	})
-	}, prometheus.NewRegistry())
 	if err != nil {
 		t.Fatalf("failed to create cache: %v", err)
 	}
 
 	svc, err := imgcache.NewService(&imgcache.ServiceConfig{
-		Cache:   cache,
+		Cache:      cache,
-		Fetcher: fetcher,
+		Fetcher:    fetcher,
-		Processor: imageprocessor.New(imageprocessor.Params{
+		SigningKey: testSigningKey,
-			MaxInputBytes:           50 << 20,
+		Allowlist:  []string{allowlistedHost},
-			MaxConcurrentProcessing: 2,
-		}),
-		MaxResponseSize: 50 << 20,
 	})
 	if err != nil {
 		t.Fatalf("failed to create service: %v", err)
 	}
 
-	h := &Handlers{
+	h := &Handlers{imgSvc: svc, log: slog.New(slog.DiscardHandler)}
-		imgSvc:    svc,
-		log:       slog.New(slog.DiscardHandler),
-		signer:    signature.New(testSigningKey),
-		allowlist: allowlist.New([]string{allowlistedHost}),
-	}
 
 	r := chi.NewRouter()
 	r.Get("/v1/image/*", h.HandleImage())
@@ -72,7 +55,7 @@ func newImageRoute(t *testing.T, fetcher httpfetcher.Fetcher) http.Handler {
 }
 
 // newPhotoFetcher returns a mock fetcher that serves a JPEG at photoPath on
-// each of hosts, and answers any other source with an upstream error.
+// each of hosts, and answers any other URL with an upstream error.
 func newPhotoFetcher(t *testing.T, hosts ...string) *httpfetcher.MockFetcher {
 	t.Helper()
 
@@ -99,10 +82,10 @@ func photoURLWithSig(host, sig string, expires time.Time) string {
 	return fmt.Sprintf("%s?sig=%s&exp=%d", photoURL(host), sig, expires.Unix())
 }
 
-// photoRequest returns the image request of photoURL(host) at the default
+// photoSignature returns the signature of photoURL(host) at the default
-// quality and fit.
+// quality and fit, made with key and expiring at expires.
-func photoRequest(host string) *imagerequest.Request {
+func photoSignature(key, host string, expires time.Time) string {
-	return &imagerequest.Request{
+	return signature.New(key).Sign(&imagerequest.Request{
 		Source: imagerequest.Source{
 			Host: host,
 			Path: photoPath,
@@ -113,13 +96,8 @@ func photoRequest(host string) *imagerequest.Request {
 			Quality: 85,
 			FitMode: imagerequest.FitCover,
 		},
-	}
+		Expires: expires,
-}
+	})
-
-// photoSignature returns the signature of photoURL(host) at the default
-// quality and fit, made with key and expiring at expires.
-func photoSignature(key, host string, expires time.Time) string {
-	return signature.New(key).Sign(photoRequest(host), expires)
 }
 
 // sendGet sends a GET for target to route and returns the response.
@@ -190,12 +168,7 @@ func TestHandleImage_ErrorAnswers(t *testing.T) {
 	photos := newPhotoFetcher(t,
 		signedHost, parentHost, siblingHost, subdomainHost, appendedHost)
 	// The real fetcher refuses localhost before any lookup or connection.
-	realFetcher := httpfetcher.New(&httpfetcher.Config{
+	realFetcher := httpfetcher.New(httpfetcher.DefaultConfig())
-		Timeout:               30 * time.Second,
-		MaxResponseSize:       50 << 20,
-		MaxConnectionsPerHost: 20,
-		MaxConnections:        64,
-	})
 
 	exp := time.Now().Add(time.Hour)
 	expired := time.Now().Add(-time.Hour)
@@ -300,9 +273,10 @@ func TestHandleImage_AllowlistOrSignature(t *testing.T) {
 // host can be, and checks that each is refused with 400 naming the host as
 // decoded: allowlistedHost with a character after it that a URL host cannot
 // hold, percent-encoded, an IPv6 address without brackets, and brackets
-// around something that is not an IPv6 address. Later steps put the host
+// around something that is not an IPv6 address. A "?" or "#" after
-// unparsed into the allowlist check, the cache key and the upstream URL, so
+// allowlistedHost would otherwise end the host when the source URL is read,
-// such a host is refused first.
+// so the request would pass the allowlist and be fetched and cached under a
+// host that is not the one fetched.
 func TestHandleImage_InvalidHost_Returns400(t *testing.T) {
 	t.Parallel()
 
@@ -373,9 +347,10 @@ func TestHandleImage_HostForms_Served(t *testing.T) {
 
 // TestHandleImage_HashInSourcePathOrQuery_Returns400 sends image URLs whose
 // source path or source query holds a "#", written as %23, and checks that
-// each is refused with 400, the one signed for that source path included.
+// each is refused with 400, the one signed for that source path included. The
-// A "#" starts a URL's fragment, so neither the source path nor the source
+// source URL would otherwise end at the "#" when it is fetched, so each text
-// query may hold one.
+// after it would fetch the image at photoPath and cache it under keys of its
+// own.
 func TestHandleImage_HashInSourcePathOrQuery_Returns400(t *testing.T) {
 	t.Parallel()
 
@@ -393,7 +368,8 @@ func TestHandleImage_HashInSourcePathOrQuery_Returns400(t *testing.T) {
 			Quality: 85,
 			FitMode: imagerequest.FitCover,
 		},
-	}, exp)
+		Expires: exp,
+	})
 
 	tests := []struct {
 		name   string
@@ -419,168 +395,3 @@ func TestHandleImage_HashInSourcePathOrQuery_Returns400(t *testing.T) {
 		})
 	}
 }
-
-// failingFetcher answers every fetch with its error.
-type failingFetcher struct {
-	err error
-}
-
-// Fetch returns f's error.
-func (f failingFetcher) Fetch(
-	context.Context, imagerequest.Source,
-) (*httpfetcher.FetchResult, error) {
-	return nil, f.err
-}
-
-// photoURLsByRoute returns, for each image route, the function that makes the
-// URL of the JPEG at photoPath on signedHost on that route.
-func photoURLsByRoute() map[string]func(*testing.T, *Handlers) string {
-	return map[string]func(*testing.T, *Handlers) string{
-		"/v1/image/": signedPhotoURL,
-		"/v1/e/":     encPhotoURL,
-	}
-}
-
-// TestImageRoutes_SameErrorAnswers checks that both image routes answer each
-// upstream fetch error below with the same status. They log it as "failed to
-// get image" at the same level: error for the 500 and warn for the others.
-// Each request gets routes of its own, so no request finds another's failed
-// fetch in the negative cache.
-func TestImageRoutes_SameErrorAnswers(t *testing.T) {
-	t.Parallel()
-
-	tests := []struct {
-		name       string
-		err        error
-		wantStatus int
-		wantLevel  slog.Level
-	}{
-		{"blocked upstream address", httpfetcher.ErrSSRFBlocked,
-			http.StatusForbidden, slog.LevelWarn},
-		{"upstream error status",
-			fmt.Errorf("%w: status 404", httpfetcher.ErrUpstreamError),
-			http.StatusBadGateway, slog.LevelWarn},
-		{"no response headers within the fetch timeout",
-			httpfetcher.ErrUpstreamTimeout, http.StatusGatewayTimeout,
-			slog.LevelWarn},
-		{"no free upstream connection", httpfetcher.ErrTooManyConnections,
-			http.StatusServiceUnavailable, slog.LevelWarn},
-		{"Content-Type not a source format",
-			fmt.Errorf("%w: image/svg+xml", httpfetcher.ErrInvalidContentType),
-			http.StatusUnsupportedMediaType, slog.LevelWarn},
-		{"any other failure", io.ErrUnexpectedEOF,
-			http.StatusInternalServerError, slog.LevelError},
-	}
-
-	for _, tt := range tests {
-		for route, imageURL := range photoURLsByRoute() {
-			t.Run(route+" "+tt.name, func(t *testing.T) {
-				t.Parallel()
-
-				var logs bytes.Buffer
-
-				h, routes := newRoutesFetchingThrough(t,
-					failingFetcher{err: tt.err},
-					slog.New(slog.NewJSONHandler(&logs, nil)))
-
-				rec := sendGet(t, routes, imageURL(t, h))
-				if rec.Code != tt.wantStatus {
-					t.Errorf("status = %d, want %d: %s",
-						rec.Code, tt.wantStatus, rec.Body)
-				}
-
-				levels := loggedLevels(t, &logs)
-				if !slices.Contains(levels[tt.wantLevel.String()],
-					"failed to get image") {
-					t.Errorf(`no "failed to get image" line logged at level %s`,
-						tt.wantLevel)
-				}
-
-				if tt.wantLevel != slog.LevelError &&
-					len(levels[slog.LevelError.String()]) != 0 {
-					t.Errorf("logged at error level: %q",
-						levels[slog.LevelError.String()])
-				}
-			})
-		}
-	}
-}
-
-// failingResponseWriter is a response writer whose Write always fails, as when
-// the client has gone away.
-type failingResponseWriter struct {
-	*httptest.ResponseRecorder
-}
-
-// Write writes nothing and returns io.ErrClosedPipe.
-func (failingResponseWriter) Write([]byte) (int, error) {
-	return 0, io.ErrClosedPipe
-}
-
-// TestImageRoutes_FailedWriteLoggedOnce checks that both image routes log a
-// failed write of the image once, at warn level, with the request's ID, the
-// source host and path, the bytes served and the error. Neither may also log
-// the image as served.
-func TestImageRoutes_FailedWriteLoggedOnce(t *testing.T) {
-	t.Parallel()
-
-	const requestID = "test-request-id"
-
-	for route, imageURL := range photoURLsByRoute() {
-		t.Run(route, func(t *testing.T) {
-			t.Parallel()
-
-			var logs bytes.Buffer
-
-			h, srv := newSignedHostServer(t,
-				slog.New(slog.NewJSONHandler(&logs, nil)))
-
-			ctx := context.WithValue(t.Context(),
-				middleware.RequestIDKey, requestID)
-			srv.ServeHTTP(failingResponseWriter{httptest.NewRecorder()},
-				httptest.NewRequestWithContext(
-					ctx, http.MethodGet, imageURL(t, h), nil))
-
-			t.Logf("logged:\n%s", logs.String())
-
-			var failedWrites []map[string]any
-
-			dec := json.NewDecoder(&logs)
-			for dec.More() {
-				var line map[string]any
-
-				err := dec.Decode(&line)
-				if err != nil {
-					t.Fatalf("decoding log line: %v", err)
-				}
-
-				switch line["msg"] {
-				case "image served":
-					t.Error(`"image served" logged after a failed write`)
-				case "failed to write response":
-					failedWrites = append(failedWrites, line)
-				}
-			}
-
-			if len(failedWrites) != 1 {
-				t.Fatalf(`"failed to write response" logged %d times, want 1`,
-					len(failedWrites))
-			}
-
-			want := map[string]any{
-				"level":        slog.LevelWarn.String(),
-				"request_id":   requestID,
-				"host":         signedHost,
-				"path":         photoPath,
-				"served_bytes": float64(0),
-				"error":        io.ErrClosedPipe.Error(),
-			}
-
-			for key, wantValue := range want {
-				if got := failedWrites[0][key]; got != wantValue {
-					t.Errorf("%s = %v, want %v", key, got, wantValue)
-				}
-			}
-		})
-	}
-}

internal/handlers/image_signature_internal_test.go

@@ -5,7 +5,6 @@ import (
 	"fmt"
 	"net/http"
 	"net/http/httptest"
-	"net/url"
 	"testing"
 	"time"
 
@@ -37,47 +36,6 @@ func getImage(t *testing.T, fix *testFixtures, target string) int {
 	return rec.Code
 }
 
-// generateSignedURL returns the /v1/image/ URL of req, signed with key and
-// expiring after ttl. It signs a copy of req, so req itself is not changed.
-// An unset quality or fit in the copy is set to 85 or cover, the values the
-// route uses when a URL has no q or fit. The URL carries q and fit next to sig
-// and exp, so the route verifies it for the quality and fit it was signed
-// with.
-func generateSignedURL(
-	key string, req *imagerequest.Request, ttl time.Duration,
-) string {
-	signed := *req
-
-	if signed.Transform.Quality == 0 {
-		signed.Transform.Quality = 85
-	}
-
-	if signed.Transform.FitMode == "" {
-		signed.Transform.FitMode = imagerequest.FitCover
-	}
-
-	expires := time.Now().Add(ttl)
-	sig := signature.New(key).Sign(&signed, expires)
-
-	// A source query follows the source path after a "?". The image route
-	// reads the path decoded once, so the whole path is written in Go's own
-	// percent-encoding of a URL path, which writes "?" as %3F and "%" as %25,
-	// and leaves "/", ";", ",", "=" and "&" as they are.
-	sourcePathAndQuery := signed.Source.Path
-	if signed.Source.Query != "" {
-		sourcePathAndQuery += "?" + signed.Source.Query
-	}
-
-	unescapedPath := fmt.Sprintf("/v1/image/%s%s/%dx%d.%s",
-		signed.Source.Host, sourcePathAndQuery,
-		signed.Transform.Size.Width, signed.Transform.Size.Height,
-		signed.Transform.Format)
-
-	return fmt.Sprintf("%s?sig=%s&exp=%d&q=%d&fit=%s",
-		(&url.URL{Path: unescapedPath}).EscapedPath(), sig, expires.Unix(),
-		signed.Transform.Quality, signed.Transform.FitMode)
-}
-
 // TestHandleImage_SignatureCoversQualityAndFit signs a URL for quality 85
 // and fit cover, the values the route uses when a URL has no q or fit, and
 // sends that signature with each q and fit below.
@@ -97,7 +55,8 @@ func TestHandleImage_SignatureCoversQualityAndFit(t *testing.T) {
 			Quality: 85,
 			FitMode: imagerequest.FitCover,
 		},
-	}, expires)
+		Expires: expires,
+	})
 	signedURL := fmt.Sprintf("/v1/image/%s/images/photo.jpg/50x50.jpeg?sig=%s&exp=%d",
 		signedHost, sig, expires.Unix())
 
@@ -124,8 +83,8 @@ func TestHandleImage_SignatureCoversQualityAndFit(t *testing.T) {
 	}
 }
 
-// TestHandleImage_GeneratedSignedURLVerifies sends URLs built by
+// TestHandleImage_GeneratedSignedURLVerifies sends URLs built by the
-// generateSignedURL to the route.
+// service's signed-URL generator to the route.
 func TestHandleImage_GeneratedSignedURLVerifies(t *testing.T) {
 	t.Parallel()
 
@@ -142,21 +101,25 @@ func TestHandleImage_GeneratedSignedURLVerifies(t *testing.T) {
 		t.Run(tt.name, func(t *testing.T) {
 			t.Parallel()
 
-			signedURL := generateSignedURL("test-signing-key-must-be-32-chars",
+			fix := setupTestHandler(t)
-				&imagerequest.Request{
-					Source: imagerequest.Source{
-						Host: signedHost,
-						Path: "/images/photo.jpg",
-					},
-					Transform: imagerequest.Transform{
-						Size:    imagerequest.Size{Width: 50, Height: 50},
-						Format:  imagerequest.FormatJPEG,
-						Quality: tt.quality,
-						FitMode: tt.fitMode,
-					},
-				}, time.Hour)
 
-			status := getImage(t, setupTestHandler(t), signedURL)
+			signedURL, err := fix.service.GenerateSignedURL("", &imagerequest.Request{
+				Source: imagerequest.Source{
+					Host: signedHost,
+					Path: "/images/photo.jpg",
+				},
+				Transform: imagerequest.Transform{
+					Size:    imagerequest.Size{Width: 50, Height: 50},
+					Format:  imagerequest.FormatJPEG,
+					Quality: tt.quality,
+					FitMode: tt.fitMode,
+				},
+			}, time.Hour)
+			if err != nil {
+				t.Fatalf("GenerateSignedURL() error = %v", err)
+			}
+
+			status := getImage(t, fix, signedURL)
 			if status != http.StatusBadGateway {
 				t.Errorf("status = %d, want %d", status, http.StatusBadGateway)
 			}
@@ -215,10 +178,10 @@ func TestHandleImage_InvalidExp_Returns400(t *testing.T) {
 }
 
 // TestHandleImage_GeneratedSignedURLWithSourceQueryVerifies sends URLs built
-// by generateSignedURL for source URLs with a query to the route, through a
+// by the service's signed-URL generator for source URLs with a query to the
-// chi router: chi routes on the path as the client wrote it whenever that is
+// route, through a chi router: chi routes on the path as the client wrote it
-// not Go's own encoding of it. A 502 means the signature passed, so the route
+// whenever that is not Go's own encoding of it. A 502 means the signature
-// read the source path and query that were signed.
+// passed, so the route read the source path and query that were signed.
 func TestHandleImage_GeneratedSignedURLWithSourceQueryVerifies(t *testing.T) {
 	t.Parallel()
 
@@ -238,20 +201,24 @@ func TestHandleImage_GeneratedSignedURLWithSourceQueryVerifies(t *testing.T) {
 		t.Run(tt.name, func(t *testing.T) {
 			t.Parallel()
 
-			signedURL := generateSignedURL("test-signing-key-must-be-32-chars",
+			fix := setupTestHandler(t)
-				&imagerequest.Request{
-					Source: imagerequest.Source{
-						Host:  signedHost,
-						Path:  photoPath,
-						Query: tt.query,
-					},
-					Transform: imagerequest.Transform{
-						Size:   imagerequest.Size{Width: 50, Height: 50},
-						Format: imagerequest.FormatJPEG,
-					},
-				}, time.Hour)
 
-			status := getImage(t, setupTestHandler(t), signedURL)
+			signedURL, err := fix.service.GenerateSignedURL("", &imagerequest.Request{
+				Source: imagerequest.Source{
+					Host:  signedHost,
+					Path:  photoPath,
+					Query: tt.query,
+				},
+				Transform: imagerequest.Transform{
+					Size:   imagerequest.Size{Width: 50, Height: 50},
+					Format: imagerequest.FormatJPEG,
+				},
+			}, time.Hour)
+			if err != nil {
+				t.Fatalf("GenerateSignedURL() error = %v", err)
+			}
+
+			status := getImage(t, fix, signedURL)
 			if status != http.StatusBadGateway {
 				t.Errorf("status = %d, want %d", status, http.StatusBadGateway)
 			}
@@ -280,7 +247,8 @@ func TestHandleImage_SourceQueryEncodedAnyWay(t *testing.T) {
 			Quality: 85,
 			FitMode: imagerequest.FitCover,
 		},
-	}, expires)
+		Expires: expires,
+	})
 
 	tests := []struct {
 		name         string
@@ -312,7 +280,7 @@ func TestHandleImage_SourceQueryEncodedAnyWay(t *testing.T) {
 // TestHandleImage_SourceURLHoldingPercent25Verifies signs a source path and a
 // source query that each hold %25, the percent-escape of a literal "%", and
 // sends each to the route written as the README says, with that "%" written as
-// %25, and as generateSignedURL writes it. A 502 means the signature passed.
+// %25, and as GenerateSignedURL writes it. A 502 means the signature passed.
 func TestHandleImage_SourceURLHoldingPercent25Verifies(t *testing.T) {
 	t.Parallel()
 
@@ -346,7 +314,8 @@ func TestHandleImage_SourceURLHoldingPercent25Verifies(t *testing.T) {
 					Quality: 85,
 					FitMode: imagerequest.FitCover,
 				},
-			}, expires)
+				Expires: expires,
+			})
 
 			target := fmt.Sprintf("/v1/image/%s%s/50x50.jpeg?sig=%s&exp=%d",
 				signedHost, tt.writtenPathAndQuery, sig, expires.Unix())
@@ -357,23 +326,27 @@ func TestHandleImage_SourceURLHoldingPercent25Verifies(t *testing.T) {
 			}
 		})
 
-		t.Run(tt.name+" written by generateSignedURL", func(t *testing.T) {
+		t.Run(tt.name+" written by GenerateSignedURL", func(t *testing.T) {
 			t.Parallel()
 
-			signedURL := generateSignedURL("test-signing-key-must-be-32-chars",
+			fix := setupTestHandler(t)
-				&imagerequest.Request{
-					Source: imagerequest.Source{
-						Host:  signedHost,
-						Path:  tt.sourcePath,
-						Query: tt.sourceQuery,
-					},
-					Transform: imagerequest.Transform{
-						Size:   imagerequest.Size{Width: 50, Height: 50},
-						Format: imagerequest.FormatJPEG,
-					},
-				}, time.Hour)
 
-			status := getImage(t, setupTestHandler(t), signedURL)
+			signedURL, err := fix.service.GenerateSignedURL("", &imagerequest.Request{
+				Source: imagerequest.Source{
+					Host:  signedHost,
+					Path:  tt.sourcePath,
+					Query: tt.sourceQuery,
+				},
+				Transform: imagerequest.Transform{
+					Size:   imagerequest.Size{Width: 50, Height: 50},
+					Format: imagerequest.FormatJPEG,
+				},
+			}, time.Hour)
+			if err != nil {
+				t.Fatalf("GenerateSignedURL() error = %v", err)
+			}
+
+			status := getImage(t, fix, signedURL)
 			if status != http.StatusBadGateway {
 				t.Errorf("status = %d, want %d", status, http.StatusBadGateway)
 			}

internal/handlers/imageenc.go

@@ -2,13 +2,21 @@ package handlers
 
 import (
 	"errors"
+	"fmt"
+	"io"
 	"net/http"
+	"strconv"
 	"time"
 
 	"github.com/go-chi/chi/v5"
+	"github.com/go-chi/chi/v5/middleware"
 
 	"sneak.berlin/go/pixa/internal/encurl"
+	"sneak.berlin/go/pixa/internal/httpfetcher"
+	"sneak.berlin/go/pixa/internal/imageprocessor"
 	"sneak.berlin/go/pixa/internal/imagerequest"
+	"sneak.berlin/go/pixa/internal/imgcache"
+	"sneak.berlin/go/pixa/internal/magic"
 )
 
 // HandleImageEnc handles requests to /v1/e/{token}/* for encrypted
@@ -20,59 +28,157 @@ func (s *Handlers) HandleImageEnc() http.HandlerFunc {
 			return
 		}
 
-		req, expires, ok := s.parseImageEncRequest(w, r)
+		ctx := r.Context()
-		if !ok {
+		start := time.Now()
+
+		req, ok := s.parseImageEncRequest(w, r)
+		if !ok || !s.chooseAutoFormat(w, r, req) {
 			return
 		}
 
-		s.serveImage(w, r, req, expires)
+		// Log the request
+		s.log.Debug("encrypted image request",
+			"host", req.Source.Host,
+			"path", req.Source.Path,
+			"dimensions", fmt.Sprintf("%dx%d",
+				req.Transform.Size.Width, req.Transform.Size.Height),
+			"format", req.Transform.Format,
+		)
+
+		// Fetch and process the image (no signature validation
+		// needed - encrypted URL is trusted)
+		resp, err := s.imgSvc.Get(ctx, req)
+		if err != nil {
+			s.handleImageError(w, err)
+
+			return
+		}
+
+		defer func() { _ = resp.Content.Close() }()
+
+		// Set response headers
+		w.Header().Set("Content-Type", resp.ContentType)
+
+		if resp.ContentLength > 0 {
+			w.Header().Set("Content-Length", strconv.FormatInt(resp.ContentLength, 10))
+		}
+
+		// Cache headers: max-age ends at the URL's expiry
+		w.Header().Set("Cache-Control", cacheControl(req.Expires))
+		w.Header().Set("X-Pixa-Cache", string(resp.CacheStatus))
+
+		if notModified(w, r, resp.ETag) {
+			return
+		}
+
+		// A HEAD request gets the headers only
+		if r.Method == http.MethodHead {
+			w.WriteHeader(http.StatusOK)
+
+			return
+		}
+
+		// Stream the response
+		written, err := io.Copy(w, resp.Content)
+		if err != nil {
+			s.log.Error("failed to write response", "error", err)
+
+			return
+		}
+
+		// Log completion
+		duration := time.Since(start)
+		s.log.Info("image served",
+			"request_id", middleware.GetReqID(ctx),
+			"cache_key", imagerequest.CacheKey(req),
+			"host", req.Source.Host,
+			"path", req.Source.Path,
+			"format", req.Transform.Format,
+			"cache_status", resp.CacheStatus,
+			"served_bytes", written,
+			"duration_ms", duration.Milliseconds(),
+		)
 	}
 }
 
 // parseImageEncRequest decrypts the token of an encrypted image URL into an
-// image request and checks it, and returns it with the token's expiry. On a
+// image request and checks it. On a token that is missing, does not decrypt,
-// token that is missing, does not decrypt, has expired or asks for something
+// has expired or asks for something not valid, it writes an error response
-// not valid, it writes an error response and returns false.
+// and returns false.
 func (s *Handlers) parseImageEncRequest(
 	w http.ResponseWriter, r *http.Request,
-) (*imagerequest.Request, time.Time, bool) {
+) (*imagerequest.Request, bool) {
 	// Extract token from URL
 	token := chi.URLParam(r, "token")
 	if token == "" {
 		s.respondError(w, "missing token", http.StatusBadRequest)
 
-		return nil, time.Time{}, false
+		return nil, false
 	}
 
 	// Decrypt and validate the payload
-	req, expires, err := s.encGen.Parse(token)
+	req, err := s.encGen.Parse(token)
 	if err != nil {
 		if errors.Is(err, encurl.ErrExpired) {
 			s.log.Debug("encrypted URL expired", "error", err)
 			s.respondError(w, "URL has expired", http.StatusGone)
 
-			return nil, time.Time{}, false
+			return nil, false
 		}
 
-		s.log.Debug("invalid encrypted URL", "error", err)
+		s.log.Debug("failed to decrypt URL", "error", err)
 		s.respondError(w, "invalid encrypted URL", http.StatusBadRequest)
 
-		return nil, time.Time{}, false
+		return nil, false
 	}
 
 	// Apply the same dimension and fit-mode bounds as the plain image
 	// route: a sealed payload is trusted for its origin, not for staying
-	// within limits. This refuses an over-limit size before the source is
+	// within limits, so an over-limit size or unknown fit mode is a 400
-	// fetched. An unknown fit mode is a 400 here rather than a 500 from the
+	// here rather than an out-of-memory or a 500 from the processor.
-	// processor.
 	err = imagerequest.ValidateImageRequest(req)
 	if err != nil {
 		s.log.Debug("encrypted URL failed validation", "error", err)
 		s.respondError(w, "invalid encrypted URL: "+err.Error(),
 			http.StatusBadRequest)
 
-		return nil, time.Time{}, false
+		return nil, false
 	}
 
-	return req, expires, true
+	return req, true
+}
+
+// handleImageError converts image service errors to HTTP responses.
+func (s *Handlers) handleImageError(w http.ResponseWriter, err error) {
+	switch {
+	case errors.Is(err, httpfetcher.ErrSSRFBlocked):
+		s.respondError(w, "forbidden", http.StatusForbidden)
+	case errors.Is(err, httpfetcher.ErrUpstreamError):
+		s.respondError(w, "upstream error", http.StatusBadGateway)
+	case errors.Is(err, httpfetcher.ErrUpstreamTimeout):
+		s.respondError(w, "upstream timeout", http.StatusGatewayTimeout)
+	case errors.Is(err, httpfetcher.ErrTooManyConnections),
+		errors.Is(err, imageprocessor.ErrTooManyImages),
+		errors.Is(err, imgcache.ErrProcessingTimedOut):
+		s.respondError(w, "server busy, try again later",
+			http.StatusServiceUnavailable)
+	case errors.Is(err, httpfetcher.ErrInvalidContentType),
+		errors.Is(err, imageprocessor.ErrUnsupportedInputFormat),
+		errors.Is(err, magic.ErrMagicByteMismatch),
+		errors.Is(err, magic.ErrUnknownFormat),
+		errors.Is(err, magic.ErrNotEnoughData):
+		s.respondError(w, "unsupported source format",
+			http.StatusUnsupportedMediaType)
+	case errors.Is(err, imageprocessor.ErrOutputTooLarge):
+		s.respondError(w, fmt.Sprintf("size too large for the source: the side "+
+			"computed from its proportions is above %d",
+			imageprocessor.MaxOutputDimension), http.StatusBadRequest)
+	case errors.Is(err, imageprocessor.ErrAVIFOutputTooLarge):
+		s.respondError(w, fmt.Sprintf("size too large for AVIF: the output "+
+			"would be above %d pixels", imageprocessor.MaxAVIFOutputPixels),
+			http.StatusBadRequest)
+	default:
+		s.log.Error("image request failed", "error", err)
+		s.respondError(w, "internal error", http.StatusInternalServerError)
+	}
 }

internal/handlers/imageenc_internal_test.go

@@ -6,13 +6,11 @@ import (
 	"net/http"
 	"net/http/httptest"
 	"testing"
-	"time"
 
 	"github.com/go-chi/chi/v5"
 
 	"sneak.berlin/go/pixa/internal/encurl"
 	"sneak.berlin/go/pixa/internal/imagerequest"
-	"sneak.berlin/go/pixa/internal/seal"
 )
 
 // newEncTestServer builds a router serving the encrypted-URL route with a
@@ -22,13 +20,11 @@ import (
 func newEncTestServer(t *testing.T) (*encurl.Generator, http.Handler) {
 	t.Helper()
 
-	keys, err := seal.DeriveKeys(testSigningKey)
+	encGen, err := encurl.NewGenerator(testSigningKey)
 	if err != nil {
-		t.Fatalf("seal.DeriveKeys() error = %v", err)
+		t.Fatalf("encurl.NewGenerator() error = %v", err)
 	}
 
-	encGen := encurl.NewGenerator(keys.EncryptedURLKey)
-
 	h := &Handlers{
 		log:    slog.New(slog.DiscardHandler),
 		encGen: encGen,
@@ -66,7 +62,7 @@ func TestHandleImageEnc_OverLimitDimension_Returns400(t *testing.T) {
 		Transform: imagerequest.Transform{
 			Size: imagerequest.Size{Width: 100000, Height: 100000},
 		},
-	}, time.Time{})
+	})
 	if err != nil {
 		t.Fatalf("Generate() error = %v", err)
 	}
@@ -95,7 +91,7 @@ func TestHandleImageEnc_InvalidFitMode_Returns400(t *testing.T) {
 			Size:    imagerequest.Size{Width: 800, Height: 600},
 			FitMode: imagerequest.Fit("bogus"),
 		},
-	}, time.Time{})
+	})
 	if err != nil {
 		t.Fatalf("Generate() error = %v", err)
 	}

internal/handlers/imageenc_token_internal_test.go

@@ -10,7 +10,6 @@ import (
 
 	"sneak.berlin/go/pixa/internal/encurl"
 	"sneak.berlin/go/pixa/internal/imagerequest"
-	"sneak.berlin/go/pixa/internal/seal"
 )
 
 // requireServedPhoto requires that rec answers 200 with the JPEG at photoPath
@@ -71,24 +70,25 @@ func TestHandleImageEnc_RejectedToken(t *testing.T) {
 		},
 	}
 
-	valid, err := h.encGen.Generate(&photo, time.Time{})
+	valid, err := h.encGen.Generate(&photo)
 	if err != nil {
 		t.Fatalf("Generate() error = %v", err)
 	}
 
-	expired, err := h.encGen.Generate(&photo, time.Now().Add(-time.Minute))
+	expiredPhoto := photo
+	expiredPhoto.Expires = time.Now().Add(-time.Minute)
+
+	expired, err := h.encGen.Generate(&expiredPhoto)
 	if err != nil {
 		t.Fatalf("Generate() error = %v", err)
 	}
 
-	keys, err := seal.DeriveKeys("another-signing-key-fedcba9876543210")
+	otherGen, err := encurl.NewGenerator("another-signing-key-fedcba9876543210")
 	if err != nil {
-		t.Fatalf("seal.DeriveKeys() error = %v", err)
+		t.Fatalf("encurl.NewGenerator() error = %v", err)
 	}
 
-	otherGen := encurl.NewGenerator(keys.EncryptedURLKey)
+	otherKey, err := otherGen.Generate(&photo)
-
-	otherKey, err := otherGen.Generate(&photo, time.Time{})
 	if err != nil {
 		t.Fatalf("Generate() error = %v", err)
 	}

internal/handlers/jpegxl_default_internal_test.go

@@ -69,7 +69,8 @@ func TestImageWithoutFormat_SignedAsJXL(t *testing.T) {
 			Quality: encurl.DefaultQuality,
 			FitMode: imagerequest.FitCover,
 		},
-	}, expires)
+		Expires: expires,
+	})
 	query := fmt.Sprintf("?sig=%s&exp=%d", sig, expires.Unix())
 
 	for _, size := range []string{"50x50.jxl", "50x50"} {
@@ -93,7 +94,7 @@ func TestImageEncWithoutFormat_ServesJPEGXL(t *testing.T) {
 		Transform: imagerequest.Transform{
 			Size: imagerequest.Size{Width: 50, Height: 50},
 		},
-	}, time.Time{})
+	})
 	if err != nil {
 		t.Fatalf("Generate() error = %v", err)
 	}

internal/handlers/referer_blocklist_internal_test.go

@@ -10,15 +10,10 @@ import (
 	"time"
 
 	"github.com/go-chi/chi/v5"
-	"github.com/prometheus/client_golang/prometheus"
 	"sneak.berlin/go/pixa/internal/allowlist"
 	"sneak.berlin/go/pixa/internal/encurl"
 	"sneak.berlin/go/pixa/internal/httpfetcher"
-	"sneak.berlin/go/pixa/internal/imageprocessor"
-	"sneak.berlin/go/pixa/internal/imagerequest"
 	"sneak.berlin/go/pixa/internal/imgcache"
-	"sneak.berlin/go/pixa/internal/seal"
-	"sneak.berlin/go/pixa/internal/signature"
 )
 
 // blockedReferer is a page on leech.example, which newRefererRoutes puts on
@@ -34,11 +29,11 @@ type countingFetcher struct {
 
 // Fetch counts the fetch and passes it on.
 func (f *countingFetcher) Fetch(
-	ctx context.Context, source imagerequest.Source,
+	ctx context.Context, url string,
 ) (*httpfetcher.FetchResult, error) {
 	f.fetches.Add(1)
 
-	return f.Fetcher.Fetch(ctx, source)
+	return f.Fetcher.Fetch(ctx, url)
 }
 
 // newRefererRoutes returns both image routes of a Handlers whose
@@ -55,38 +50,30 @@ func newRefererRoutes(t *testing.T) (http.Handler, *Handlers, *countingFetcher)
 	cache, err := imgcache.NewCache(setupTestDB(t), imgcache.CacheConfig{
 		StateDir:    t.TempDir(),
 		NegativeTTL: 5 * time.Minute,
-		MaxBytes:    1 << 30,
+	})
-	}, prometheus.NewRegistry())
 	if err != nil {
 		t.Fatalf("imgcache.NewCache() error = %v", err)
 	}
 
 	svc, err := imgcache.NewService(&imgcache.ServiceConfig{
-		Cache:   cache,
+		Cache:      cache,
-		Fetcher: fetcher,
+		Fetcher:    fetcher,
-		Processor: imageprocessor.New(imageprocessor.Params{
+		SigningKey: testSigningKey,
-			MaxInputBytes:           50 << 20,
+		Allowlist:  []string{allowlistedHost},
-			MaxConcurrentProcessing: 2,
-		}),
-		MaxResponseSize: 50 << 20,
 	})
 	if err != nil {
 		t.Fatalf("imgcache.NewService() error = %v", err)
 	}
 
-	keys, err := seal.DeriveKeys(testSigningKey)
+	encGen, err := encurl.NewGenerator(testSigningKey)
 	if err != nil {
-		t.Fatalf("seal.DeriveKeys() error = %v", err)
+		t.Fatalf("encurl.NewGenerator() error = %v", err)
 	}
 
-	encGen := encurl.NewGenerator(keys.EncryptedURLKey)
-
 	h := &Handlers{
-		log:       slog.New(slog.DiscardHandler),
+		log:    slog.New(slog.DiscardHandler),
-		imgSvc:    svc,
+		imgSvc: svc,
-		encGen:    encGen,
+		encGen: encGen,
-		signer:    signature.New(testSigningKey),
-		allowlist: allowlist.New([]string{allowlistedHost}),
 		refererBlocklist: allowlist.New(
 			[]string{"leech.example", ".hotlinker.example"}),
 	}

internal/handlers/request_id_internal_test.go

@@ -17,13 +17,11 @@ import (
 )
 
 // signedPhotoURL returns a signed /v1/image/ URL, valid for a minute, for the
-// JPEG at photoPath on signedHost at 50x50. It is signed with testSigningKey,
+// JPEG at photoPath on signedHost at 50x50, made with h's image service.
-// the key the handlers that newSignedHostServer builds check signatures with.
+func signedPhotoURL(t *testing.T, h *Handlers) string {
-// It takes a Handlers it does not use, so that it has the type of encPhotoURL.
-func signedPhotoURL(t *testing.T, _ *Handlers) string {
 	t.Helper()
 
-	return generateSignedURL(testSigningKey, &imagerequest.Request{
+	signedURL, err := h.imgSvc.GenerateSignedURL("", &imagerequest.Request{
 		Source: imagerequest.Source{
 			Host: signedHost,
 			Path: photoPath,
@@ -33,6 +31,11 @@ func signedPhotoURL(t *testing.T, _ *Handlers) string {
 			Format: imagerequest.FormatJPEG,
 		},
 	}, time.Minute)
+	if err != nil {
+		t.Fatalf("GenerateSignedURL() error = %v", err)
+	}
+
+	return signedURL
 }
 
 // encPhotoURL returns an encrypted /v1/e/ URL, which never expires, for the
@@ -49,7 +52,7 @@ func encPhotoURL(t *testing.T, h *Handlers) string {
 			Size:   imagerequest.Size{Width: 50, Height: 50},
 			Format: imagerequest.FormatJPEG,
 		},
-	}, time.Time{})
+	})
 	if err != nil {
 		t.Fatalf("Generate() error = %v", err)
 	}

internal/handlers/server_busy_internal_test.go

@@ -15,10 +15,9 @@ import (
 	"sneak.berlin/go/pixa/internal/imgcache"
 )
 
-// TestServerBusyAnswers503 checks that respondImageError, which answers the
+// TestServerBusyAnswers503 checks that both image routes answer 503 with
-// errors of both image routes, answers 503 with a clear error when the image
+// a clear error when the image service gives up waiting for a free upstream
-// service gives up waiting for a free upstream connection or processing slot,
+// connection or processing slot, wrapped as the service wraps them.
-// wrapped as the service wraps them.
 func TestServerBusyAnswers503(t *testing.T) {
 	t.Parallel()
 
@@ -31,28 +30,35 @@ func TestServerBusyAnswers503(t *testing.T) {
 		fmt.Errorf("upstream fetch failed: %w", httpfetcher.ErrTooManyConnections),
 		fmt.Errorf("image processing failed: %w", imageprocessor.ErrTooManyImages),
 	} {
-		rec := httptest.NewRecorder()
+		plain := httptest.NewRecorder()
-		h.respondImageError(rec, req, err)
+		h.respondImageError(plain, req, err)
 
-		if rec.Code != http.StatusServiceUnavailable {
+		encrypted := httptest.NewRecorder()
-			t.Errorf("for %v: status = %d, want %d",
+		h.handleImageError(encrypted, err)
-				err, rec.Code, http.StatusServiceUnavailable)
-		}
 
-		if !strings.Contains(rec.Body.String(), "server busy, try again later") {
+		for route, rec := range map[string]*httptest.ResponseRecorder{
-			t.Errorf("for %v: body = %q, want the server busy error",
+			"/v1/image/": plain, "/v1/e/": encrypted,
-				err, rec.Body.String())
+		} {
+			if rec.Code != http.StatusServiceUnavailable {
+				t.Errorf("%s for %v: status = %d, want %d",
+					route, err, rec.Code, http.StatusServiceUnavailable)
+			}
+
+			if !strings.Contains(rec.Body.String(), "server busy, try again later") {
+				t.Errorf("%s for %v: body = %q, want the server busy error",
+					route, err, rec.Body.String())
+			}
 		}
 	}
 }
 
-// TestProcessingTimedOutAnswers503 checks that respondImageError, which
+// TestProcessingTimedOutAnswers503 checks that respondImageError and
-// answers the errors of both image routes, answers 503 with the server busy
+// handleImageError, which answer the errors of the two image routes, answer
-// error when the image service reports that processing stopped at the
+// 503 with the server busy error when the image service reports that
-// deadline of the request doing it, wrapped as the service wraps it. A request
+// processing stopped at the deadline of the request doing it, wrapped as the
-// that waited for that processing gets this error while its own deadline may
+// service wraps it. A request that waited for that processing gets this error
-// have time left. The request's source only goes into a log line, so it is
+// while its own deadline may have time left. The request's source only goes
-// left empty.
+// into a log line, so it is left empty.
 func TestProcessingTimedOutAnswers503(t *testing.T) {
 	t.Parallel()
 
@@ -60,15 +66,25 @@ func TestProcessingTimedOutAnswers503(t *testing.T) {
 	err := fmt.Errorf("%w: failed to encode: %w",
 		imgcache.ErrProcessingTimedOut, context.DeadlineExceeded)
 
-	rec := httptest.NewRecorder()
+	plain := httptest.NewRecorder()
-	h.respondImageError(rec, &imagerequest.Request{}, err)
+	h.respondImageError(plain, &imagerequest.Request{}, err)
-	t.Logf("status = %d, body = %q", rec.Code, rec.Body.String())
 
-	if rec.Code != http.StatusServiceUnavailable {
+	encrypted := httptest.NewRecorder()
-		t.Errorf("status = %d, want %d", rec.Code, http.StatusServiceUnavailable)
+	h.handleImageError(encrypted, err)
-	}
 
-	if !strings.Contains(rec.Body.String(), "server busy, try again later") {
+	for name, rec := range map[string]*httptest.ResponseRecorder{
-		t.Errorf("body = %q, want the server busy error", rec.Body.String())
+		"respondImageError": plain, "handleImageError": encrypted,
+	} {
+		t.Logf("%s: status = %d, body = %q", name, rec.Code, rec.Body.String())
+
+		if rec.Code != http.StatusServiceUnavailable {
+			t.Errorf("%s: status = %d, want %d",
+				name, rec.Code, http.StatusServiceUnavailable)
+		}
+
+		if !strings.Contains(rec.Body.String(), "server busy, try again later") {
+			t.Errorf("%s: body = %q, want the server busy error",
+				name, rec.Body.String())
+		}
 	}
 }

internal/handlers/unsupported_source_internal_test.go

@@ -13,11 +13,10 @@ import (
 	"sneak.berlin/go/pixa/internal/imagerequest"
 )
 
-// TestUnsupportedSourceFormatAnswers415 checks that respondImageError, which
+// TestUnsupportedSourceFormatAnswers415 checks that both image routes answer
-// answers the errors of both image routes, answers 415 with a clear error when
+// 415 with a clear error when the upstream answers with a content type pixa
-// the upstream answers with a content type pixa does not accept, such as SVG,
+// does not accept, such as SVG, and when orig is asked of a source in a
-// and when orig is asked of a source in a format the processor cannot name,
+// format the processor cannot name, wrapped as the service wraps them.
-// wrapped as the service wraps them.
 func TestUnsupportedSourceFormatAnswers415(t *testing.T) {
 	t.Parallel()
 
@@ -32,18 +31,26 @@ func TestUnsupportedSourceFormatAnswers415(t *testing.T) {
 		fmt.Errorf("image processing failed: %w: unknown",
 			imageprocessor.ErrUnsupportedInputFormat),
 	} {
-		rec := httptest.NewRecorder()
+		plain := httptest.NewRecorder()
-		h.respondImageError(rec, req, err)
+		h.respondImageError(plain, req, err)
-		t.Logf("for %v: %d %s", err, rec.Code, rec.Body.String())
 
-		if rec.Code != http.StatusUnsupportedMediaType {
+		encrypted := httptest.NewRecorder()
-			t.Errorf("for %v: status = %d, want %d",
+		h.handleImageError(encrypted, err)
-				err, rec.Code, http.StatusUnsupportedMediaType)
-		}
 
-		if !strings.Contains(rec.Body.String(), "unsupported source format") {
+		for route, rec := range map[string]*httptest.ResponseRecorder{
-			t.Errorf("for %v: body = %q, want the unsupported source format error",
+			"HandleImage": plain, "HandleImageEnc": encrypted,
-				err, rec.Body.String())
+		} {
+			t.Logf("%s for %v: %d %s", route, err, rec.Code, rec.Body.String())
+
+			if rec.Code != http.StatusUnsupportedMediaType {
+				t.Errorf("%s for %v: status = %d, want %d",
+					route, err, rec.Code, http.StatusUnsupportedMediaType)
+			}
+
+			if !strings.Contains(rec.Body.String(), "unsupported source format") {
+				t.Errorf("%s for %v: body = %q, want the unsupported source format error",
+					route, err, rec.Body.String())
+			}
 		}
 	}
 }

internal/handlers/validate_signature_internal_test.go

@@ -1,211 +0,0 @@
-package handlers
-
-import (
-	"testing"
-	"time"
-
-	"sneak.berlin/go/pixa/internal/allowlist"
-	"sneak.berlin/go/pixa/internal/signature"
-)
-
-// Addresses the port tests below put on the allowlist.
-const (
-	allowlistedIPv4 = "203.0.113.5"
-	allowlistedIPv6 = "2001:db8::1"
-)
-
-// newSignatureChecker returns Handlers that check signatures with
-// testSigningKey and have allowlistHosts on their allowlist, as New builds
-// them from signing_key and allowlist_hosts.
-func newSignatureChecker(allowlistHosts ...string) *Handlers {
-	return &Handlers{
-		signer:    signature.New(testSigningKey),
-		allowlist: allowlist.New(allowlistHosts),
-	}
-}
-
-func TestValidateSignature_NonAllowlistedHost_NoSignature(t *testing.T) {
-	t.Parallel()
-
-	h := newSignatureChecker(allowlistedHost)
-
-	// Should fail validation - not allowlisted and no signature
-	err := h.validateSignature(photoRequest(signedHost), "", time.Time{})
-	if err == nil {
-		t.Error("validateSignature() expected error for non-allowlisted " +
-			"host without signature")
-	}
-}
-
-func TestValidateSignature_NonAllowlistedHost_ValidSignature(t *testing.T) {
-	t.Parallel()
-
-	h := newSignatureChecker(allowlistedHost)
-	req := photoRequest(signedHost)
-
-	// Generate a valid signature
-	expires := time.Now().Add(time.Hour)
-	sig := signature.New(testSigningKey).Sign(req, expires)
-
-	// Should pass validation
-	err := h.validateSignature(req, sig, expires)
-	if err != nil {
-		t.Errorf("validateSignature() error = %v", err)
-	}
-}
-
-func TestValidateSignature_NonAllowlistedHost_ExpiredSignature(t *testing.T) {
-	t.Parallel()
-
-	h := newSignatureChecker(allowlistedHost)
-	req := photoRequest(signedHost)
-
-	// Generate an expired signature
-	expires := time.Now().Add(-time.Hour)
-	sig := signature.New(testSigningKey).Sign(req, expires)
-
-	// Should fail validation
-	err := h.validateSignature(req, sig, expires)
-	if err == nil {
-		t.Error("validateSignature() expected error for expired signature")
-	}
-}
-
-func TestValidateSignature_NonAllowlistedHost_InvalidSignature(t *testing.T) {
-	t.Parallel()
-
-	h := newSignatureChecker(allowlistedHost)
-
-	// Should fail validation
-	err := h.validateSignature(photoRequest(signedHost), "invalid-signature",
-		time.Now().Add(time.Hour))
-	if err == nil {
-		t.Error("validateSignature() expected error for invalid signature")
-	}
-}
-
-// TestValidateSignature_NoAllowlist checks that with no host on the
-// allowlist, a request without a signature fails validation.
-func TestValidateSignature_NoAllowlist(t *testing.T) {
-	t.Parallel()
-
-	h := newSignatureChecker()
-
-	err := h.validateSignature(photoRequest(signedHost), "", time.Time{})
-	if err == nil {
-		t.Error("validateSignature() expected error when no host is allowlisted")
-	}
-}
-
-// TestValidateSignature_SignatureExactHostMatch verifies that
-// validateSignature enforces exact host matching for signatures. A
-// signature for one host must not verify for a different host, even
-// if they share a domain suffix.
-func TestValidateSignature_SignatureExactHostMatch(t *testing.T) {
-	t.Parallel()
-
-	h := newSignatureChecker()
-
-	// Sign a request for signedHost
-	signedReq := photoRequest(signedHost)
-	expires := time.Now().Add(time.Hour)
-	sig := signature.New(testSigningKey).Sign(signedReq, expires)
-
-	// The original request should pass validation
-	t.Run("exact host passes", func(t *testing.T) {
-		t.Parallel()
-
-		err := h.validateSignature(signedReq, sig, expires)
-		if err != nil {
-			t.Errorf("validateSignature() exact host failed: %v", err)
-		}
-	})
-
-	// Try to reuse the signature with different hosts
-	tests := []struct {
-		name string
-		host string
-	}{
-		{"parent domain", "example.com"},
-		{"sibling subdomain", "images.example.com"},
-		{"deeper subdomain", "a." + signedHost},
-		{"evil suffix domain", signedHost + ".evil.com"},
-		{"prefixed host", "evil" + signedHost},
-	}
-
-	for _, tt := range tests {
-		t.Run(tt.name+" rejected", func(t *testing.T) {
-			t.Parallel()
-
-			err := h.validateSignature(photoRequest(tt.host), sig, expires)
-			if err == nil {
-				t.Errorf("validateSignature() should reject signature for "+
-					"host %q (signed for %q)", tt.host, signedHost)
-			}
-		})
-	}
-}
-
-// TestValidateSignature_HostWithPortNeedsSignature verifies that a host
-// followed by a port, or by a colon with no port, matches no allowlist entry,
-// so that a request for it without a signature is refused, while the same
-// host with no port is allowlisted.
-func TestValidateSignature_HostWithPortNeedsSignature(t *testing.T) {
-	t.Parallel()
-
-	h := newSignatureChecker(allowlistedHost, allowlistedIPv4, allowlistedIPv6)
-
-	tests := []struct {
-		host        string
-		allowlisted bool
-	}{
-		{allowlistedHost, true},
-		{allowlistedHost + ":8443", false},
-		{allowlistedHost + ":443", false},
-		{allowlistedHost + ":", false},
-		{allowlistedIPv4, true},
-		{allowlistedIPv4 + ":8080", false},
-		{"[" + allowlistedIPv6 + "]", true},
-		{"[" + allowlistedIPv6 + "]:8443", false},
-	}
-
-	for _, tt := range tests {
-		t.Run(tt.host, func(t *testing.T) {
-			t.Parallel()
-
-			err := h.validateSignature(photoRequest(tt.host), "", time.Time{})
-			t.Logf("validateSignature() for host %q with no signature: %v",
-				tt.host, err)
-
-			if allowlisted := err == nil; allowlisted != tt.allowlisted {
-				t.Errorf("host %q served without a signature = %v, want %v",
-					tt.host, allowlisted, tt.allowlisted)
-			}
-		})
-	}
-}
-
-// TestValidateSignature_UnbracketedIPv6PortNeedsSignature verifies that an
-// allowlisted IPv6 address written without brackets and followed by a port,
-// or by a colon with no port, needs a signature. Go's URL type reads
-// 2001:db8::1:8443 as the host 2001:db8::1 with the port 8443.
-func TestValidateSignature_UnbracketedIPv6PortNeedsSignature(t *testing.T) {
-	t.Parallel()
-
-	h := newSignatureChecker(allowlistedIPv6)
-
-	for _, host := range []string{allowlistedIPv6 + ":8443", allowlistedIPv6 + ":"} {
-		t.Run(host, func(t *testing.T) {
-			t.Parallel()
-
-			err := h.validateSignature(photoRequest(host), "", time.Time{})
-			t.Logf("validateSignature() for host %q with no signature: %v",
-				host, err)
-
-			if err == nil {
-				t.Errorf("host %q served without a signature, want refused",
-					host)
-			}
-		})
-	}
-}

internal/httpfetcher/blocked_networks_internal_test.go

@@ -7,7 +7,6 @@ import (
 	"net/http"
 	"net/netip"
 	"testing"
-	"time"
 )
 
 // TestIsPrivateIPBlocksSpecialRanges covers the internal and special-use
@@ -71,15 +70,12 @@ func transportOf(t *testing.T, f *HTTPFetcher) *http.Transport {
 func TestDialerEnforcesBlockedNetworks(t *testing.T) {
 	t.Parallel()
 
-	transport := transportOf(t, New(&Config{
+	cfg := DefaultConfig()
-		Timeout:               30 * time.Second,
+	// TEST-NET-2 (198.51.100.0/24) is public to the built-in check, so
-		MaxResponseSize:       50 << 20,
+	// blocking it can only come from the operator-supplied list.
-		MaxConnectionsPerHost: 20,
+	cfg.BlockedNetworks = []netip.Prefix{netip.MustParsePrefix("198.51.100.0/24")}
-		MaxConnections:        64,
+
-		// TEST-NET-2 (198.51.100.0/24) is public to the built-in check, so
+	transport := transportOf(t, New(cfg))
-		// blocking it can only come from the operator-supplied list.
-		BlockedNetworks: []netip.Prefix{netip.MustParsePrefix("198.51.100.0/24")},
-	}))
 
 	blocked := []string{
 		"198.51.100.5:80", // operator-supplied range

internal/httpfetcher/dial_context_internal_test.go

@@ -4,7 +4,6 @@ import (
 	"errors"
 	"net"
 	"testing"
-	"time"
 )
 
 // TestNewUsesCheckedDialerWithoutDialContext checks that a fetcher built
@@ -14,12 +13,7 @@ func TestNewUsesCheckedDialerWithoutDialContext(t *testing.T) {
 	t.Parallel()
 
 	srv := startUpstream(t)
-	transport := transportOf(t, New(&Config{
+	transport := transportOf(t, New(DefaultConfig()))
-		Timeout:               30 * time.Second,
-		MaxResponseSize:       50 << 20,
-		MaxConnectionsPerHost: 20,
-		MaxConnections:        64,
-	}))
 
 	addr := srv.Listener.Addr().String()
 
@@ -38,25 +32,21 @@ func TestDialContextReplacesOnlyTheDialer(t *testing.T) {
 	srv := startUpstream(t)
 	dialer := &recordingDialer{target: srv.Listener.Addr().String()}
 
-	f := New(&Config{
+	cfg := DefaultConfig()
-		Timeout:               30 * time.Second,
+	cfg.AllowHTTP = true
-		MaxResponseSize:       50 << 20,
+	cfg.DialContext = dialer.dialContext
-		AllowHTTP:             true,
+	f := New(cfg)
-		MaxConnectionsPerHost: 20,
-		MaxConnections:        64,
-		DialContext:           dialer.dialContext,
-	})
 
 	if body := fetchBody(t, f, "/image"); body != imagePayload {
 		t.Errorf("body = %q, want %q", body, imagePayload)
 	}
 
-	_, err := f.Fetch(testContext(t), imageSourceOn("127.0.0.1"))
+	_, err := f.Fetch(testContext(t), "http://127.0.0.1/image")
 	if !errors.Is(err, ErrSSRFBlocked) {
 		t.Errorf("Fetch(loopback URL) error = %v, want ErrSSRFBlocked", err)
 	}
 
-	_, err = f.Fetch(testContext(t), upstreamSource("/redirect/private"))
+	_, err = f.Fetch(testContext(t), upstreamURL("/redirect/private"))
 	if !errors.Is(err, ErrSSRFBlocked) {
 		t.Errorf("Fetch(/redirect/private) error = %v, want ErrSSRFBlocked", err)
 	}

internal/httpfetcher/fetch_internal_test.go

@@ -14,8 +14,6 @@ import (
 	"sync"
 	"testing"
 	"time"
-
-	"sneak.berlin/go/pixa/internal/imagerequest"
 )
 
 // testPublicHost is a TEST-NET-1 (RFC 5737) literal. isPrivateIP treats it as
@@ -31,9 +29,9 @@ const imagePayload = "fake-jpeg-bytes"
 // would mean SSRF protection let a forbidden target reach the transport.
 var errUnexpectedDial = errors.New("unexpected dial target")
 
-// upstreamSource returns the source at path on the fake public host.
+// upstreamURL builds a fetch URL on the fake public host for the given path.
-func upstreamSource(path string) imagerequest.Source {
+func upstreamURL(path string) string {
-	return imagerequest.Source{Host: testPublicHost, Path: path}
+	return "http://" + testPublicHost + path
 }
 
 // recordingDialer records every address the transport asks it to dial and
@@ -122,12 +120,7 @@ func newServerFetcher(
 	t.Helper()
 
 	if cfg == nil {
-		cfg = &Config{
+		cfg = DefaultConfig()
-			Timeout:               30 * time.Second,
-			MaxResponseSize:       50 << 20,
-			MaxConnectionsPerHost: 20,
-			MaxConnections:        64,
-		}
 	}
 
 	cfg.AllowHTTP = true
@@ -160,7 +153,7 @@ func testContext(t *testing.T) context.Context {
 func fetchImage(t *testing.T, f *HTTPFetcher, path string) *FetchResult {
 	t.Helper()
 
-	res, err := f.Fetch(testContext(t), upstreamSource(path))
+	res, err := f.Fetch(testContext(t), upstreamURL(path))
 	if err != nil {
 		t.Fatalf("Fetch(%s) error = %v", path, err)
 	}
@@ -172,7 +165,7 @@ func fetchImage(t *testing.T, f *HTTPFetcher, path string) *FetchResult {
 func fetchExpectError(t *testing.T, f *HTTPFetcher, path string) error {
 	t.Helper()
 
-	res, err := f.Fetch(testContext(t), upstreamSource(path))
+	res, err := f.Fetch(testContext(t), upstreamURL(path))
 	if err == nil {
 		_ = res.Content.Close()
 
@@ -224,7 +217,7 @@ func TestFetchRedirectToPrivateIPBlocked(t *testing.T) {
 	srv := startUpstream(t)
 	f, dialer := newServerFetcher(t, srv, nil)
 
-	_, err := f.Fetch(testContext(t), upstreamSource("/redirect/private"))
+	_, err := f.Fetch(testContext(t), upstreamURL("/redirect/private"))
 	if !errors.Is(err, ErrSSRFBlocked) {
 		t.Fatalf("Fetch() error = %v, want ErrSSRFBlocked", err)
 	}
@@ -287,12 +280,8 @@ func TestFetchMaxResponseSizeEnforced(t *testing.T) {
 
 	srv := startUpstream(t)
 
-	cfg := &Config{
+	cfg := DefaultConfig()
-		Timeout:               30 * time.Second,
+	cfg.MaxResponseSize = 8
-		MaxResponseSize:       8,
-		MaxConnectionsPerHost: 20,
-		MaxConnections:        64,
-	}
 
 	f, _ := newServerFetcher(t, srv, cfg)
 
@@ -321,12 +310,8 @@ func fetchWithMaxResponseSize(
 	srv := httptest.NewServer(handler)
 	t.Cleanup(srv.Close)
 
-	cfg := &Config{
+	cfg := DefaultConfig()
-		Timeout:               30 * time.Second,
+	cfg.MaxResponseSize = maxResponseSize
-		MaxResponseSize:       maxResponseSize,
-		MaxConnectionsPerHost: 20,
-		MaxConnections:        64,
-	}
 
 	f, _ := newServerFetcher(t, srv, cfg)
 
@@ -451,12 +436,8 @@ func TestFetchSemaphoreReleasedOnError(t *testing.T) {
 
 	srv := startUpstream(t)
 
-	cfg := &Config{
+	cfg := DefaultConfig()
-		Timeout:               30 * time.Second,
+	cfg.MaxConnectionsPerHost = 1
-		MaxResponseSize:       50 << 20,
-		MaxConnectionsPerHost: 1,
-		MaxConnections:        64,
-	}
 
 	f, _ := newServerFetcher(t, srv, cfg)
 
@@ -485,12 +466,8 @@ func assertSlotReleasedByClose(
 
 	srv := startUpstream(t)
 
-	cfg := &Config{
+	cfg := DefaultConfig()
-		Timeout:               30 * time.Second,
+	cfg.MaxConnectionsPerHost = 1
-		MaxResponseSize:       50 << 20,
-		MaxConnectionsPerHost: 1,
-		MaxConnections:        64,
-	}
 
 	f, _ := newServerFetcher(t, srv, cfg)
 
@@ -547,7 +524,7 @@ func TestFetchSemaphoreReleasedOnPartialReadClose(t *testing.T) {
 	})
 }
 
-// The dial-time re-resolution in dialSSRFSafe is what closes the DNS
+// The dial-time re-resolution in ssrfSafeDialer is what closes the DNS
 // rebinding window: even if validateURL saw a public answer earlier, the
 // dialer independently re-checks the address it is about to connect to. A full
 // rebinding simulation (a resolver returning public, then private) would mean
@@ -567,9 +544,9 @@ func TestSSRFSafeDialerBlocksPrivateTarget(t *testing.T) {
 		t.Run(addr, func(t *testing.T) {
 			t.Parallel()
 
-			_, err := dialSSRFSafe(context.Background(), "tcp", addr, nil)
+			_, err := ssrfSafeDialer(context.Background(), "tcp", addr)
 			if !errors.Is(err, ErrSSRFBlocked) {
-				t.Errorf("dialSSRFSafe(%q) = %v, want ErrSSRFBlocked", addr, err)
+				t.Errorf("ssrfSafeDialer(%q) = %v, want ErrSSRFBlocked", addr, err)
 			}
 		})
 	}
@@ -583,7 +560,7 @@ func TestSSRFSafeDialerAllowsPublicTarget(t *testing.T) {
 	ctx, cancel := context.WithCancel(context.Background())
 	cancel()
 
-	_, err := dialSSRFSafe(ctx, "tcp", testPublicHost+":80", nil)
+	_, err := ssrfSafeDialer(ctx, "tcp", testPublicHost+":80")
 	if err == nil {
 		t.Fatal("expected a dial error for an unreachable public target")
 	}
@@ -592,53 +569,3 @@ func TestSSRFSafeDialerAllowsPublicTarget(t *testing.T) {
 		t.Errorf("public target was SSRF-blocked: %v", err)
 	}
 }
-
-// TestFetchSchemeFollowsAllowHTTP checks that the fetcher fetches a source
-// over https, on port 443, unless AllowHTTP is set, and then over http, on
-// port 80. The test server speaks plain HTTP, so only the fetch over http
-// succeeds.
-func TestFetchSchemeFollowsAllowHTTP(t *testing.T) {
-	t.Parallel()
-
-	tests := []struct {
-		allowHTTP bool
-		wantAddr  string
-	}{
-		{false, net.JoinHostPort(testPublicHost, "443")},
-		{true, net.JoinHostPort(testPublicHost, "80")},
-	}
-
-	for _, tt := range tests {
-		t.Run(fmt.Sprintf("AllowHTTP %v", tt.allowHTTP), func(t *testing.T) {
-			t.Parallel()
-
-			srv := startUpstream(t)
-			dialer := &recordingDialer{target: srv.Listener.Addr().String()}
-
-			f := New(&Config{
-				Timeout:               30 * time.Second,
-				MaxResponseSize:       50 << 20,
-				AllowHTTP:             tt.allowHTTP,
-				MaxConnectionsPerHost: 20,
-				MaxConnections:        64,
-				DialContext:           dialer.dialContext,
-			})
-
-			res, err := f.Fetch(testContext(t), upstreamSource("/image"))
-			if err == nil {
-				_ = res.Content.Close()
-			}
-
-			dialed := dialer.dialedAddrs()
-			t.Logf("Fetch(): error %v, dialed %v", err, dialed)
-
-			if served := err == nil; served != tt.allowHTTP {
-				t.Errorf("served = %v, want %v", served, tt.allowHTTP)
-			}
-
-			if len(dialed) != 1 || dialed[0] != tt.wantAddr {
-				t.Errorf("dialed %v, want only %s", dialed, tt.wantAddr)
-			}
-		})
-	}
-}

internal/httpfetcher/httpfetcher.go

@@ -20,20 +20,20 @@ import (
 	"time"
 
 	"github.com/go-chi/chi/v5/middleware"
-	"sneak.berlin/go/pixa/internal/imagerequest"
 )
 
 // Fetcher configuration constants.
 const (
-	DefaultTLSTimeout      = 10 * time.Second
+	DefaultFetchTimeout          = 30 * time.Second
-	DefaultMaxIdleConns    = 100
+	DefaultMaxResponseSize       = 50 << 20 // 50MB
-	DefaultIdleConnTimeout = 90 * time.Second
+	DefaultTLSTimeout            = 10 * time.Second
-	DefaultMaxRedirects    = 10
+	DefaultMaxIdleConns          = 100
+	DefaultIdleConnTimeout       = 90 * time.Second
+	DefaultMaxRedirects          = 10
+	DefaultMaxConnectionsPerHost = 20
+	DefaultMaxConnections        = 64
 )
 
-// userAgent is the User-Agent header sent to upstream servers.
-const userAgent = "pixa/1.0"
-
 // ConnectionWaitTimeout is how long Fetch waits for a free connection when
 // MaxConnections fetches are already in progress.
 const ConnectionWaitTimeout = 10 * time.Second
@@ -92,8 +92,8 @@ var (
 
 // Fetcher retrieves content from upstream origins.
 type Fetcher interface {
-	// Fetch retrieves the content of the source image.
+	// Fetch retrieves content from the given URL.
-	Fetch(ctx context.Context, source imagerequest.Source) (*FetchResult, error)
+	Fetch(ctx context.Context, url string) (*FetchResult, error)
 }
 
 // FetchResult contains the result of fetching from upstream.
@@ -120,28 +120,17 @@ type FetchResult struct {
 	TLSCipherSuite string
 }
 
-// allowedContentTypes returns the content types an upstream response may
+// Config holds configuration for the upstream fetcher.
-// have, which Fetch also sends as its Accept header.
-func allowedContentTypes() []string {
-	return []string{
-		contentTypeJPEG,
-		contentTypePNG,
-		contentTypeGIF,
-		contentTypeWebP,
-		contentTypeAVIF,
-		contentTypeJXL,
-	}
-}
-
-// Config holds configuration for the upstream fetcher. Every field but
-// AllowHTTP, BlockedNetworks and DialContext must be set.
 type Config struct {
 	// Timeout for upstream requests.
 	Timeout time.Duration
 	// MaxResponseSize is the maximum allowed response body size.
 	MaxResponseSize int64
-	// AllowHTTP makes the fetcher fetch sources over http instead of https,
+	// UserAgent to send to upstream servers.
-	// and follow redirects to http (for testing only).
+	UserAgent string
+	// AllowedContentTypes is an allow list of MIME types to accept.
+	AllowedContentTypes []string
+	// AllowHTTP allows non-TLS connections (for testing only).
 	AllowHTTP bool
 	// MaxConnectionsPerHost limits concurrent connections to each upstream host.
 	MaxConnectionsPerHost int
@@ -158,6 +147,26 @@ type Config struct {
 	DialContext func(ctx context.Context, network, addr string) (net.Conn, error)
 }
 
+// DefaultConfig returns a Config with sensible defaults.
+func DefaultConfig() *Config {
+	return &Config{
+		Timeout:         DefaultFetchTimeout,
+		MaxResponseSize: DefaultMaxResponseSize,
+		UserAgent:       "pixa/1.0",
+		AllowedContentTypes: []string{
+			contentTypeJPEG,
+			contentTypePNG,
+			contentTypeGIF,
+			contentTypeWebP,
+			contentTypeAVIF,
+			contentTypeJXL,
+		},
+		AllowHTTP:             false,
+		MaxConnectionsPerHost: DefaultMaxConnectionsPerHost,
+		MaxConnections:        DefaultMaxConnections,
+	}
+}
+
 // HTTPFetcher implements Fetcher with SSRF protection and connection limits
 // per host and for all hosts together.
 type HTTPFetcher struct {
@@ -185,6 +194,10 @@ type hostSemaphore struct {
 
 // New creates a new HTTPFetcher with SSRF protection.
 func New(config *Config) *HTTPFetcher {
+	if config == nil {
+		config = DefaultConfig()
+	}
+
 	// Unless config.DialContext replaces it, the transport connects with
 	// the SSRF-safe dialer, which re-resolves and re-checks at connect time
 	// (closing the DNS-rebinding window) against both the built-in ranges
@@ -213,7 +226,7 @@ func New(config *Config) *HTTPFetcher {
 			}
 
 			// Validate the redirect target
-			err := validateURL(req.Context(), req.URL, config.AllowHTTP)
+			err := validateURL(req.Context(), req.URL.String(), config.AllowHTTP)
 			if err != nil {
 				return fmt.Errorf("redirect blocked: %w", err)
 			}
@@ -231,22 +244,18 @@ func New(config *Config) *HTTPFetcher {
 	}
 }
 
-// Fetch retrieves the source image with SSRF protection, over https, or over
+// Fetch retrieves content from the given URL with SSRF protection. When
-// http when AllowHTTP is set. When MaxConnections fetches are already in
+// MaxConnections fetches are already in progress, it waits up to
-// progress, it waits up to ConnectionWaitTimeout for one to finish, then
+// ConnectionWaitTimeout for one to finish, then fails with
-// fails with ErrTooManyConnections.
+// ErrTooManyConnections.
-func (f *HTTPFetcher) Fetch(
+func (f *HTTPFetcher) Fetch(ctx context.Context, url string) (*FetchResult, error) {
-	ctx context.Context, source imagerequest.Source,
-) (*FetchResult, error) {
-	sourceURL := source.URL(f.config.AllowHTTP)
-
 	// Validate URL before making request
-	err := validateURL(ctx, sourceURL, f.config.AllowHTTP)
+	err := validateURL(ctx, url, f.config.AllowHTTP)
 	if err != nil {
 		return nil, err
 	}
 
-	release, err := f.acquireConnection(ctx, source.Host)
+	release, err := f.acquireConnection(ctx, extractHost(url))
 	if err != nil {
 		return nil, err
 	}
@@ -260,14 +269,19 @@ func (f *HTTPFetcher) Fetch(
 		}
 	}()
 
+	parsedURL, err := neturl.Parse(url)
+	if err != nil {
+		return nil, fmt.Errorf("failed to parse URL: %w", err)
+	}
+
 	req := &http.Request{
 		Method: http.MethodGet,
-		URL:    sourceURL,
+		URL:    parsedURL,
 		Header: make(http.Header),
 	}
 
-	req.Header.Set("User-Agent", userAgent)
+	req.Header.Set("User-Agent", f.config.UserAgent)
-	req.Header.Set("Accept", strings.Join(allowedContentTypes(), ", "))
+	req.Header.Set("Accept", strings.Join(f.config.AllowedContentTypes, ", "))
 
 	// The ID of the request this fetch serves, so the fetch can be found in
 	// the upstream host's logs
@@ -446,7 +460,7 @@ func (f *HTTPFetcher) isAllowedContentType(contentType string) bool {
 	// Extract the MIME type without parameters
 	mediaType := strings.TrimSpace(strings.Split(contentType, ";")[0])
 
-	for _, allowed := range allowedContentTypes() {
+	for _, allowed := range f.config.AllowedContentTypes {
 		if strings.EqualFold(mediaType, allowed) {
 			return true
 		}
@@ -456,14 +470,19 @@ func (f *HTTPFetcher) isAllowedContentType(contentType string) bool {
 }
 
 // validateURL checks if a URL is safe to fetch (not internal/private).
-func validateURL(ctx context.Context, fetchURL *neturl.URL, allowHTTP bool) error {
+func validateURL(ctx context.Context, rawURL string, allowHTTP bool) error {
-	if !allowHTTP && fetchURL.Scheme != "https" {
+	if !allowHTTP && !strings.HasPrefix(rawURL, "https://") {
 		return ErrUnsupportedScheme
 	}
 
+	parsedURL, err := neturl.Parse(rawURL)
+	if err != nil {
+		return fmt.Errorf("%w: %w", ErrInvalidHost, err)
+	}
+
 	// Hostname drops any port, and the brackets of an IPv6 address whether
 	// or not a port follows them
-	host := fetchURL.Hostname()
+	host := parsedURL.Hostname()
 	if host == "" {
 		return ErrInvalidHost
 	}
@@ -489,6 +508,25 @@ func validateURL(ctx context.Context, fetchURL *neturl.URL, allowHTTP bool) erro
 	return nil
 }
 
+// extractHost extracts the host from a URL string.
+func extractHost(rawURL string) string {
+	// Simple extraction without full URL parsing
+	url := rawURL
+	if idx := strings.Index(url, "://"); idx != -1 {
+		url = url[idx+3:]
+	}
+
+	if idx := strings.Index(url, "/"); idx != -1 {
+		url = url[:idx]
+	}
+
+	if idx := strings.Index(url, "?"); idx != -1 {
+		url = url[:idx]
+	}
+
+	return url
+}
+
 // isLocalhost checks if the host is localhost.
 func isLocalhost(host string) bool {
 	host = strings.ToLower(host)
@@ -601,6 +639,13 @@ func ipv4CompatibleToIPv4(ip net.IP) net.IP {
 	return ip
 }
 
+// ssrfSafeDialer validates IP addresses against the built-in blocked ranges
+// before connecting. New wraps dialSSRFSafe with the operator-supplied
+// blocklist; this entry point enforces the built-in ranges alone.
+func ssrfSafeDialer(ctx context.Context, network, addr string) (net.Conn, error) {
+	return dialSSRFSafe(ctx, network, addr, nil)
+}
+
 // dialSSRFSafe re-resolves addr and refuses to connect to any built-in
 // internal range or operator-supplied blocked prefix, closing the
 // DNS-rebinding window at connect time.

internal/httpfetcher/httpfetcher_internal_test.go

@@ -5,26 +5,62 @@ import (
 	"errors"
 	"io"
 	"net"
-	"net/url"
 	"testing"
 	"testing/fstest"
-	"time"
-
-	"sneak.berlin/go/pixa/internal/imagerequest"
 )
 
 // testHost is the hostname used by mock fetch tests.
 const testHost = "example.com"
 
+func TestDefaultConfig(t *testing.T) {
+	t.Parallel()
+
+	cfg := DefaultConfig()
+
+	if cfg.Timeout != DefaultFetchTimeout {
+		t.Errorf("Timeout = %v, want %v", cfg.Timeout, DefaultFetchTimeout)
+	}
+
+	if cfg.MaxResponseSize != DefaultMaxResponseSize {
+		t.Errorf("MaxResponseSize = %d, want %d", cfg.MaxResponseSize, DefaultMaxResponseSize)
+	}
+
+	if cfg.MaxConnectionsPerHost != DefaultMaxConnectionsPerHost {
+		t.Errorf("MaxConnectionsPerHost = %d, want %d",
+			cfg.MaxConnectionsPerHost, DefaultMaxConnectionsPerHost)
+	}
+
+	if cfg.AllowHTTP {
+		t.Error("AllowHTTP should default to false")
+	}
+
+	if len(cfg.AllowedContentTypes) == 0 {
+		t.Error("AllowedContentTypes should not be empty")
+	}
+}
+
+func TestNewWithNilConfigUsesDefaults(t *testing.T) {
+	t.Parallel()
+
+	f := New(nil)
+
+	if f == nil {
+		t.Fatal("New(nil) returned nil")
+	}
+
+	if f.config == nil {
+		t.Fatal("config should be populated from DefaultConfig")
+	}
+
+	if f.config.Timeout != DefaultFetchTimeout {
+		t.Errorf("Timeout = %v, want %v", f.config.Timeout, DefaultFetchTimeout)
+	}
+}
+
 func TestIsAllowedContentType(t *testing.T) {
 	t.Parallel()
 
-	f := New(&Config{
+	f := New(DefaultConfig())
-		Timeout:               30 * time.Second,
-		MaxResponseSize:       50 << 20,
-		MaxConnectionsPerHost: 20,
-		MaxConnections:        64,
-	})
 
 	tests := []struct {
 		contentType string
@@ -52,6 +88,33 @@ func TestIsAllowedContentType(t *testing.T) {
 	}
 }
 
+func TestExtractHost(t *testing.T) {
+	t.Parallel()
+
+	tests := []struct {
+		url  string
+		want string
+	}{
+		{"https://example.com/path", testHost},
+		{"http://example.com:8080/path", "example.com:8080"},
+		{"https://example.com", testHost},
+		{"https://example.com?q=1", testHost},
+		{"example.com/path", testHost},
+		{"", ""},
+	}
+
+	for _, tc := range tests {
+		t.Run(tc.url, func(t *testing.T) {
+			t.Parallel()
+
+			got := extractHost(tc.url)
+			if got != tc.want {
+				t.Errorf("extractHost(%q) = %q, want %q", tc.url, got, tc.want)
+			}
+		})
+	}
+}
+
 func TestIsLocalhost(t *testing.T) {
 	t.Parallel()
 
@@ -164,7 +227,7 @@ func TestIsPrivateIPJudgesEmbeddedIPv4Address(t *testing.T) {
 func TestValidateURL_RejectsNonHTTPS(t *testing.T) {
 	t.Parallel()
 
-	err := validateURL(t.Context(), &url.URL{Scheme: "http", Host: testHost}, false)
+	err := validateURL(t.Context(), "http://example.com/path", false)
 	if !errors.Is(err, ErrUnsupportedScheme) {
 		t.Errorf("validateURL http = %v, want ErrUnsupportedScheme", err)
 	}
@@ -174,8 +237,7 @@ func TestValidateURL_AllowsHTTPWhenConfigured(t *testing.T) {
 	t.Parallel()
 
 	// Use a host that won't resolve (explicit .invalid TLD) so we don't hit DNS.
-	err := validateURL(t.Context(),
+	err := validateURL(t.Context(), "http://nonexistent.invalid/path", true)
-		&url.URL{Scheme: "http", Host: "nonexistent.invalid"}, true)
 	// We expect a host resolution error, not ErrUnsupportedScheme.
 	if errors.Is(err, ErrUnsupportedScheme) {
 		t.Error("validateURL with AllowHTTP should not return ErrUnsupportedScheme")
@@ -185,7 +247,7 @@ func TestValidateURL_AllowsHTTPWhenConfigured(t *testing.T) {
 func TestValidateURL_RejectsLocalhost(t *testing.T) {
 	t.Parallel()
 
-	err := validateURL(t.Context(), &url.URL{Scheme: "https", Host: "localhost"}, false)
+	err := validateURL(t.Context(), "https://localhost/path", false)
 	if !errors.Is(err, ErrSSRFBlocked) {
 		t.Errorf("validateURL localhost = %v, want ErrSSRFBlocked", err)
 	}
@@ -194,7 +256,7 @@ func TestValidateURL_RejectsLocalhost(t *testing.T) {
 func TestValidateURL_EmptyHost(t *testing.T) {
 	t.Parallel()
 
-	err := validateURL(t.Context(), &url.URL{Scheme: "https"}, false)
+	err := validateURL(t.Context(), "https:///path", false)
 	if !errors.Is(err, ErrInvalidHost) {
 		t.Errorf("validateURL empty host = %v, want ErrInvalidHost", err)
 	}
@@ -209,8 +271,7 @@ func TestMockFetcher_FetchesFile(t *testing.T) {
 
 	m := NewMock(mockFS)
 
-	result, err := m.Fetch(context.Background(),
+	result, err := m.Fetch(context.Background(), "https://example.com/images/photo.jpg")
-		imagerequest.Source{Host: testHost, Path: "/images/photo.jpg"})
 	if err != nil {
 		t.Fatalf("Fetch() error = %v", err)
 	}
@@ -241,8 +302,7 @@ func TestMockFetcher_MissingFileReturnsUpstreamError(t *testing.T) {
 	mockFS := fstest.MapFS{}
 	m := NewMock(mockFS)
 
-	_, err := m.Fetch(context.Background(),
+	_, err := m.Fetch(context.Background(), "https://example.com/missing.jpg")
-		imagerequest.Source{Host: testHost, Path: "/missing.jpg"})
 	if !errors.Is(err, ErrUpstreamError) {
 		t.Errorf("Fetch() error = %v, want ErrUpstreamError", err)
 	}
@@ -259,7 +319,7 @@ func TestMockFetcher_RespectsContextCancellation(t *testing.T) {
 	ctx, cancel := context.WithCancel(context.Background())
 	cancel()
 
-	_, err := m.Fetch(ctx, imagerequest.Source{Host: testHost, Path: "/photo.jpg"})
+	_, err := m.Fetch(ctx, "https://example.com/photo.jpg")
 	if !errors.Is(err, context.Canceled) {
 		t.Errorf("Fetch() error = %v, want context.Canceled", err)
 	}

internal/httpfetcher/jpegxl_internal_test.go

@@ -1,26 +1,16 @@
 package httpfetcher
 
-import (
+import "testing"
-	"testing"
-	"time"
-)
 
 // TestJPEGXLContentType verifies that the fetcher accepts an upstream answer
-// of type image/jxl, and that the mock fetcher serves a .jxl file as
+// of type image/jxl by default, and that the mock fetcher serves a .jxl file
-// image/jxl.
+// as image/jxl.
 func TestJPEGXLContentType(t *testing.T) {
 	t.Parallel()
 
 	const jxlType = "image/jxl"
 
-	f := New(&Config{
+	if !New(DefaultConfig()).isAllowedContentType(jxlType) {
-		Timeout:               30 * time.Second,
-		MaxResponseSize:       50 << 20,
-		MaxConnectionsPerHost: 20,
-		MaxConnections:        64,
-	})
-
-	if !f.isAllowedContentType(jxlType) {
 		t.Errorf("isAllowedContentType(%q) = false, want true", jxlType)
 	}
 

internal/httpfetcher/max_connections_internal_test.go

@@ -8,20 +8,22 @@ import (
 	"sync"
 	"testing"
 	"time"
-
-	"sneak.berlin/go/pixa/internal/imagerequest"
 )
 
-// imageSourceOn is the fake upstream's image route on host.
+// imageURLOnPort is the fake upstream's image route on testPublicHost at
-func imageSourceOn(host string) imagerequest.Source {
-	return imagerequest.Source{Host: host, Path: "/image"}
-}
-
-// imageSourceOnPort is the fake upstream's image route on testPublicHost at
 // port. Each port is a different host to the per-host limit, while the test
 // dialer sends every port to the one test server.
-func imageSourceOnPort(port int) imagerequest.Source {
+func imageURLOnPort(port int) string {
-	return imageSourceOn(net.JoinHostPort(testPublicHost, strconv.Itoa(port)))
+	return "http://" + net.JoinHostPort(testPublicHost, strconv.Itoa(port)) +
+		"/image"
+}
+
+func TestDefaultConfigMaxConnections(t *testing.T) {
+	t.Parallel()
+
+	if got := DefaultConfig().MaxConnections; got != DefaultMaxConnections {
+		t.Errorf("MaxConnections = %d, want %d", got, DefaultMaxConnections)
+	}
 }
 
 // TestFetchLimitsConnectionsToAllHostsTogether checks that MaxConnections
@@ -35,22 +37,18 @@ func TestFetchLimitsConnectionsToAllHostsTogether(t *testing.T) {
 
 	srv := startUpstream(t)
 
-	cfg := &Config{
+	cfg := DefaultConfig()
-		Timeout:               30 * time.Second,
+	cfg.MaxConnections = 2
-		MaxResponseSize:       50 << 20,
-		MaxConnectionsPerHost: 20,
-		MaxConnections:        2,
-	}
 
 	f, _ := newServerFetcher(t, srv, cfg)
 	f.connectionWaitTimeout = 100 * time.Millisecond
 
-	first, err := f.Fetch(testContext(t), imageSourceOnPort(81))
+	first, err := f.Fetch(testContext(t), imageURLOnPort(81))
 	if err != nil {
 		t.Fatalf("first Fetch() error = %v", err)
 	}
 
-	second, err := f.Fetch(testContext(t), imageSourceOnPort(82))
+	second, err := f.Fetch(testContext(t), imageURLOnPort(82))
 	if err != nil {
 		t.Fatalf("second Fetch() error = %v", err)
 	}
@@ -59,7 +57,7 @@ func TestFetchLimitsConnectionsToAllHostsTogether(t *testing.T) {
 
 	start := time.Now()
 
-	_, err = f.Fetch(testContext(t), imageSourceOnPort(83))
+	_, err = f.Fetch(testContext(t), imageURLOnPort(83))
 	if !errors.Is(err, ErrTooManyConnections) {
 		t.Fatalf("third Fetch() error = %v, want ErrTooManyConnections", err)
 	}
@@ -78,7 +76,7 @@ func TestFetchLimitsConnectionsToAllHostsTogether(t *testing.T) {
 		t.Fatalf("close first body: %v", err)
 	}
 
-	third, err := f.Fetch(testContext(t), imageSourceOnPort(83))
+	third, err := f.Fetch(testContext(t), imageURLOnPort(83))
 	if err != nil {
 		t.Fatalf("Fetch() after a response was closed: error = %v", err)
 	}
@@ -96,16 +94,12 @@ func TestFetchFreesHostSlotWhenContextEndsWaitingForConnection(t *testing.T) {
 
 	srv := startUpstream(t)
 
-	cfg := &Config{
+	cfg := DefaultConfig()
-		Timeout:               30 * time.Second,
+	cfg.MaxConnections = 1
-		MaxResponseSize:       50 << 20,
-		MaxConnectionsPerHost: 20,
-		MaxConnections:        1,
-	}
 
 	f, _ := newServerFetcher(t, srv, cfg)
 
-	first, err := f.Fetch(testContext(t), imageSourceOnPort(81))
+	first, err := f.Fetch(testContext(t), imageURLOnPort(81))
 	if err != nil {
 		t.Fatalf("first Fetch() error = %v", err)
 	}
@@ -115,7 +109,7 @@ func TestFetchFreesHostSlotWhenContextEndsWaitingForConnection(t *testing.T) {
 	ctx, cancel := context.WithTimeout(t.Context(), 100*time.Millisecond)
 	defer cancel()
 
-	_, err = f.Fetch(ctx, imageSourceOnPort(82))
+	_, err = f.Fetch(ctx, imageURLOnPort(82))
 	if !errors.Is(err, context.DeadlineExceeded) {
 		t.Fatalf("second Fetch() error = %v, want context.DeadlineExceeded", err)
 	}
@@ -140,7 +134,7 @@ func TestFetchRemovesIdleHostSemaphores(t *testing.T) {
 
 	for i := range 100 {
 		wg.Go(func() {
-			res, err := f.Fetch(ctx, imageSourceOnPort(1+i%50))
+			res, err := f.Fetch(ctx, imageURLOnPort(1+i%50))
 			if err != nil {
 				t.Errorf("Fetch() error = %v", err)
 
@@ -169,22 +163,19 @@ func TestFetchRemovesHostSemaphoreWhenNoConnection(t *testing.T) {
 
 	srv := startUpstream(t)
 
-	cfg := &Config{
+	cfg := DefaultConfig()
-		Timeout:               30 * time.Second,
+	cfg.MaxConnections = 1
-		MaxResponseSize:       50 << 20,
+	cfg.MaxConnectionsPerHost = 1
-		MaxConnectionsPerHost: 1,
-		MaxConnections:        1,
-	}
 
 	f, _ := newServerFetcher(t, srv, cfg)
 	f.connectionWaitTimeout = 100 * time.Millisecond
 
-	open, err := f.Fetch(testContext(t), imageSourceOnPort(81))
+	open, err := f.Fetch(testContext(t), imageURLOnPort(81))
 	if err != nil {
 		t.Fatalf("first Fetch() error = %v", err)
 	}
 
-	_, err = f.Fetch(testContext(t), imageSourceOnPort(82))
+	_, err = f.Fetch(testContext(t), imageURLOnPort(82))
 	if !errors.Is(err, ErrTooManyConnections) {
 		t.Fatalf("Fetch() from another host: error = %v, "+
 			"want ErrTooManyConnections", err)
@@ -193,7 +184,7 @@ func TestFetchRemovesHostSemaphoreWhenNoConnection(t *testing.T) {
 	ctx, cancel := context.WithTimeout(t.Context(), 100*time.Millisecond)
 	defer cancel()
 
-	_, err = f.Fetch(ctx, imageSourceOnPort(81))
+	_, err = f.Fetch(ctx, imageURLOnPort(81))
 	if !errors.Is(err, context.DeadlineExceeded) {
 		t.Fatalf("Fetch() from the busy host: error = %v, "+
 			"want context.DeadlineExceeded", err)
@@ -205,7 +196,7 @@ func TestFetchRemovesHostSemaphoreWhenNoConnection(t *testing.T) {
 	ctx, cancel = context.WithTimeout(t.Context(), 100*time.Millisecond)
 	defer cancel()
 
-	_, err = f.Fetch(ctx, imageSourceOnPort(83))
+	_, err = f.Fetch(ctx, imageURLOnPort(83))
 	if !errors.Is(err, context.DeadlineExceeded) {
 		t.Fatalf("Fetch() from a host with nothing open: error = %v, "+
 			"want context.DeadlineExceeded", err)
@@ -228,15 +219,15 @@ func TestFetchReleasesConnectionOnError(t *testing.T) {
 	t.Parallel()
 
 	cases := []struct {
-		name   string
+		name string
-		source imagerequest.Source
+		url  string
-		want   error
+		want error
 	}{
-		{"upstream answers 500", upstreamSource("/status/500"), ErrUpstreamError},
+		{"upstream answers 500", upstreamURL("/status/500"), ErrUpstreamError},
-		{"upstream sends HTML", upstreamSource("/html"), ErrInvalidContentType},
+		{"upstream sends HTML", upstreamURL("/html"), ErrInvalidContentType},
 		// 198.51.100.7 (TEST-NET-2) passes the SSRF checks, and the test
 		// dialer refuses every host but testPublicHost.
-		{"connecting fails", imageSourceOn("198.51.100.7"), errUnexpectedDial},
+		{"connecting fails", "http://198.51.100.7/image", errUnexpectedDial},
 	}
 
 	for _, tc := range cases {
@@ -245,17 +236,13 @@ func TestFetchReleasesConnectionOnError(t *testing.T) {
 
 			srv := startUpstream(t)
 
-			cfg := &Config{
+			cfg := DefaultConfig()
-				Timeout:               30 * time.Second,
+			cfg.MaxConnections = 1
-				MaxResponseSize:       50 << 20,
-				MaxConnectionsPerHost: 20,
-				MaxConnections:        1,
-			}
 
 			f, _ := newServerFetcher(t, srv, cfg)
 			f.connectionWaitTimeout = 100 * time.Millisecond
 
-			_, err := f.Fetch(testContext(t), tc.source)
+			_, err := f.Fetch(testContext(t), tc.url)
 			if !errors.Is(err, tc.want) {
 				t.Fatalf("Fetch() error = %v, want %v", err, tc.want)
 			}

internal/httpfetcher/mock.go

@@ -7,14 +7,14 @@ import (
 	"io/fs"
 	"net/http"
 	"strings"
-
-	"sneak.berlin/go/pixa/internal/imagerequest"
 )
 
+// errEmptyURLPath is returned when a mock URL has no usable path.
+var errEmptyURLPath = errors.New("empty URL path")
+
 // MockFetcher implements Fetcher using an embedded filesystem.
 // Files are organized as: hostname/path/to/file.ext
-// The source with host example.com and path /images/photo.jpg maps to
+// URLs like https://example.com/images/photo.jpg map to example.com/images/photo.jpg.
-// example.com/images/photo.jpg; the source query is ignored.
 type MockFetcher struct {
 	fs fs.FS
 }
@@ -24,10 +24,8 @@ func NewMock(fsys fs.FS) *MockFetcher {
 	return &MockFetcher{fs: fsys}
 }
 
-// Fetch retrieves the source's content from the mock filesystem.
+// Fetch retrieves content from the mock filesystem.
-func (m *MockFetcher) Fetch(
+func (m *MockFetcher) Fetch(ctx context.Context, url string) (*FetchResult, error) {
-	ctx context.Context, source imagerequest.Source,
-) (*FetchResult, error) {
 	// Check context cancellation
 	select {
 	case <-ctx.Done():
@@ -35,7 +33,11 @@ func (m *MockFetcher) Fetch(
 	default:
 	}
 
-	path := source.Host + source.Path
+	// Parse URL to get filesystem path
+	path, err := urlToFSPath(url)
+	if err != nil {
+		return nil, err
+	}
 
 	// Open the file
 	f, err := m.fs.Open(path)
@@ -66,6 +68,32 @@ func (m *MockFetcher) Fetch(
 	}, nil
 }
 
+// urlToFSPath converts a URL to a filesystem path.
+// https://example.com/images/photo.jpg -> example.com/images/photo.jpg
+func urlToFSPath(rawURL string) (string, error) {
+	// Strip scheme
+	url := rawURL
+	if idx := strings.Index(url, "://"); idx != -1 {
+		url = url[idx+3:]
+	}
+
+	// Remove query string
+	if idx := strings.Index(url, "?"); idx != -1 {
+		url = url[:idx]
+	}
+
+	// Remove fragment
+	if idx := strings.Index(url, "#"); idx != -1 {
+		url = url[:idx]
+	}
+
+	if url == "" {
+		return "", errEmptyURLPath
+	}
+
+	return url, nil
+}
+
 // detectContentTypeFromPath returns the MIME type based on file extension.
 func detectContentTypeFromPath(path string) string {
 	path = strings.ToLower(path)

internal/httpfetcher/request_id_internal_test.go

@@ -34,7 +34,7 @@ func TestFetchSendsRequestID(t *testing.T) {
 
 	ctx := context.WithValue(testContext(t), middleware.RequestIDKey, requestID)
 
-	res, err := f.Fetch(ctx, upstreamSource("/image"))
+	res, err := f.Fetch(ctx, upstreamURL("/image"))
 	if err != nil {
 		t.Fatalf("Fetch() error = %v", err)
 	}

internal/imageprocessor/avif_too_large_internal_test.go

@@ -69,10 +69,7 @@ func TestImageProcessor_RefusesAVIFAboveMaxPixels(t *testing.T) {
 		t.Run(tt.name, func(t *testing.T) {
 			t.Parallel()
 
-			_, err := New(Params{
+			_, err := New(Params{}).Process(t.Context(), bytes.NewReader(tt.input),
-				MaxInputBytes:           50 << 20,
-				MaxConcurrentProcessing: 1,
-			}).Process(t.Context(), bytes.NewReader(tt.input),
 				&imagerequest.Transform{
 					Size:    imagerequest.Size{Width: tt.width, Height: tt.height},
 					Format:  tt.format,

internal/imageprocessor/computed_side_internal_test.go

@@ -38,10 +38,7 @@ func TestImageProcessor_RefusesComputedSideAboveMaximum(t *testing.T) {
 
 			input := createTestJPEG(t, tt.inputWidth, tt.inputHeight)
 
-			result, err := New(Params{
+			result, err := New(Params{}).Process(t.Context(),
-				MaxInputBytes:           50 << 20,
-				MaxConcurrentProcessing: 1,
-			}).Process(t.Context(),
 				bytes.NewReader(input),
 				&imagerequest.Transform{
 					Size:    tt.size,
@@ -107,10 +104,7 @@ func TestImageProcessor_ComputedSideIsAtLeastOne(t *testing.T) {
 
 				input := createTestJPEG(t, tt.inputWidth, tt.inputHeight)
 
-				result, err := New(Params{
+				result, err := New(Params{}).Process(t.Context(),
-					MaxInputBytes:           50 << 20,
-					MaxConcurrentProcessing: 1,
-				}).Process(t.Context(),
 					bytes.NewReader(input),
 					&imagerequest.Transform{
 						Size:    tt.size,

internal/imageprocessor/deadline_internal_test.go

@@ -84,9 +84,9 @@ func checkProcessStopsAtDeadline(
 ) {
 	t.Helper()
 
-	proc := New(Params{MaxInputBytes: 50 << 20, MaxConcurrentProcessing: 1})
+	proc := New(Params{MaxConcurrentProcessing: 1})
 
-	result, err := proc.Process(ctx, input, smallJPEGTransform())
+	result, err := proc.Process(ctx, input, smallJPEGRequest())
 	if result != nil {
 		t.Logf("Process() made a %dx%d image after its deadline passed",
 			result.Width, result.Height)

internal/imageprocessor/empty_format_internal_test.go

@@ -14,10 +14,7 @@ import (
 func TestImageProcessor_EmptyFormatRefused(t *testing.T) {
 	t.Parallel()
 
-	_, err := New(Params{
+	_, err := New(Params{}).Process(
-		MaxInputBytes:           50 << 20,
-		MaxConcurrentProcessing: 1,
-	}).Process(
 		t.Context(), bytes.NewReader(createTestJPEG(t, 20, 20)),
 		&imagerequest.Transform{},
 	)

internal/imageprocessor/encoder_settings_internal_test.go

@@ -14,14 +14,11 @@ import (
 // processedSize runs input through Process and returns the output's size in
 // bytes.
 func processedSize(
-	t *testing.T, input []byte, transform *imagerequest.Transform,
+	t *testing.T, input []byte, req *imagerequest.Transform,
 ) int64 {
 	t.Helper()
 
-	result, err := New(Params{
+	result, err := New(Params{}).Process(t.Context(), bytes.NewReader(input), req)
-		MaxInputBytes:           50 << 20,
-		MaxConcurrentProcessing: 1,
-	}).Process(t.Context(), bytes.NewReader(input), transform)
 	if err != nil {
 		t.Fatalf("Process() error = %v", err)
 	}

internal/imageprocessor/fit_cover_internal_test.go

@@ -36,10 +36,7 @@ func TestImageProcessor_FitCoverServesThinSource(t *testing.T) {
 
 			input := createTestJPEG(t, tt.inputWidth, tt.inputHeight)
 
-			result, err := New(Params{
+			result, err := New(Params{}).Process(t.Context(),
-				MaxInputBytes:           50 << 20,
-				MaxConcurrentProcessing: 1,
-			}).Process(t.Context(),
 				bytes.NewReader(input),
 				&imagerequest.Transform{
 					Size:    tt.size,
@@ -109,10 +106,7 @@ func TestImageProcessor_FitCoverCropsTheCentre(t *testing.T) {
 
 			input := createBandsPNG(t, tt.inputWidth, tt.inputHeight)
 
-			result, err := New(Params{
+			result, err := New(Params{}).Process(t.Context(),
-				MaxInputBytes:           50 << 20,
-				MaxConcurrentProcessing: 1,
-			}).Process(t.Context(),
 				bytes.NewReader(input),
 				&imagerequest.Transform{
 					Size:    imagerequest.Size{Width: 50, Height: 50},
@@ -236,10 +230,7 @@ func TestImageProcessor_FitCoverKeepsProportions(t *testing.T) {
 
 			input := createSquarePNG(t, tt.side, tt.squareSide)
 
-			result, err := New(Params{
+			result, err := New(Params{}).Process(t.Context(),
-				MaxInputBytes:           50 << 20,
-				MaxConcurrentProcessing: 1,
-			}).Process(t.Context(),
 				bytes.NewReader(input),
 				&imagerequest.Transform{
 					Size:    tt.size,

internal/imageprocessor/fit_outside_internal_test.go

@@ -41,10 +41,7 @@ func TestImageProcessor_RefusesFitOutsideAboveMaximum(t *testing.T) {
 
 			input := createTestJPEG(t, tt.inputWidth, tt.inputHeight)
 
-			result, err := New(Params{
+			result, err := New(Params{}).Process(t.Context(),
-				MaxInputBytes:           50 << 20,
-				MaxConcurrentProcessing: 1,
-			}).Process(t.Context(),
 				bytes.NewReader(input),
 				&imagerequest.Transform{
 					Size:    tt.size,
@@ -98,10 +95,7 @@ func TestImageProcessor_FitOutsideSideIsAtLeastOne(t *testing.T) {
 
 			input := createTestJPEG(t, tt.inputWidth, tt.inputHeight)
 
-			result, err := New(Params{
+			result, err := New(Params{}).Process(t.Context(),
-				MaxInputBytes:           50 << 20,
-				MaxConcurrentProcessing: 1,
-			}).Process(t.Context(),
 				bytes.NewReader(input),
 				&imagerequest.Transform{
 					Size:    tt.size,

internal/imageprocessor/imageprocessor.go

@@ -11,6 +11,7 @@ import (
 	"time"
 
 	"github.com/davidbyttow/govips/v2/vips"
+	"sneak.berlin/go/pixa/internal/config"
 	"sneak.berlin/go/pixa/internal/imagerequest"
 )
 
@@ -78,6 +79,10 @@ type Result struct {
 // Images larger than this are rejected to prevent DoS via decompression bombs.
 const MaxInputDimension = 8192
 
+// DefaultMaxInputBytes is the default maximum input size in bytes (50 MiB).
+// This matches the default upstream fetcher limit.
+const DefaultMaxInputBytes = 50 << 20
+
 // ErrInputTooLarge is returned when input image dimensions exceed MaxInputDimension.
 var ErrInputTooLarge = errors.New("input image dimensions exceed maximum")
 
@@ -137,22 +142,35 @@ type ImageProcessor struct {
 	processingWaitTimeout time.Duration
 }
 
-// Params holds configuration for creating an ImageProcessor. Both fields
+// Params holds configuration for creating an ImageProcessor.
-// must be at least 1.
+// Zero values use sensible defaults (MaxInputBytes defaults to DefaultMaxInputBytes).
 type Params struct {
 	// MaxInputBytes is the maximum allowed input size in bytes.
+	// If <= 0, DefaultMaxInputBytes is used.
 	MaxInputBytes int64
 	// MaxConcurrentProcessing is the most images processed at once.
+	// If <= 0, config.DefaultMaxConcurrentProcessing() is used.
 	MaxConcurrentProcessing int
 }
 
 // New creates a new image processor with the given parameters.
+// A zero-value Params{} uses sensible defaults.
 func New(params Params) *ImageProcessor {
 	initVips()
 
+	maxInputBytes := params.MaxInputBytes
+	if maxInputBytes <= 0 {
+		maxInputBytes = DefaultMaxInputBytes
+	}
+
+	maxConcurrentProcessing := params.MaxConcurrentProcessing
+	if maxConcurrentProcessing <= 0 {
+		maxConcurrentProcessing = config.DefaultMaxConcurrentProcessing()
+	}
+
 	return &ImageProcessor{
-		maxInputBytes:         params.MaxInputBytes,
+		maxInputBytes:         maxInputBytes,
-		processingSemaphore:   make(chan struct{}, params.MaxConcurrentProcessing),
+		processingSemaphore:   make(chan struct{}, maxConcurrentProcessing),
 		processingWaitTimeout: ProcessingWaitTimeout,
 	}
 }
@@ -304,6 +322,30 @@ const (
 	mimeJXL  = "image/jxl"
 )
 
+// SupportedInputFormats returns MIME types this processor can read.
+func (p *ImageProcessor) SupportedInputFormats() []string {
+	return []string{
+		mimeJPEG,
+		mimePNG,
+		mimeGIF,
+		mimeWebP,
+		mimeAVIF,
+		mimeJXL,
+	}
+}
+
+// SupportedOutputFormats returns formats this processor can write.
+func (p *ImageProcessor) SupportedOutputFormats() []imagerequest.Format {
+	return []imagerequest.Format{
+		imagerequest.FormatJPEG,
+		imagerequest.FormatPNG,
+		imagerequest.FormatGIF,
+		imagerequest.FormatWebP,
+		imagerequest.FormatAVIF,
+		imagerequest.FormatJXL,
+	}
+}
+
 // FormatToMIME converts a Format to its MIME type string.
 func FormatToMIME(format imagerequest.Format) string {
 	switch format {

internal/imageprocessor/imageprocessor_internal_test.go

@@ -119,19 +119,19 @@ func detectMIME(data []byte) string {
 func TestImageProcessor_ResizeJPEG(t *testing.T) {
 	t.Parallel()
 
-	proc := New(Params{MaxInputBytes: 50 << 20, MaxConcurrentProcessing: 1})
+	proc := New(Params{})
 	ctx := context.Background()
 
 	input := createTestJPEG(t, 800, 600)
 
-	transform := &imagerequest.Transform{
+	req := &imagerequest.Transform{
 		Size:    imagerequest.Size{Width: 400, Height: 300},
 		Format:  imagerequest.FormatJPEG,
 		Quality: 85,
 		FitMode: imagerequest.FitCover,
 	}
 
-	result, err := proc.Process(ctx, bytes.NewReader(input), transform)
+	result, err := proc.Process(ctx, bytes.NewReader(input), req)
 	if err != nil {
 		t.Fatalf("Process() error = %v", err)
 	}
@@ -165,18 +165,18 @@ func TestImageProcessor_ResizeJPEG(t *testing.T) {
 func TestImageProcessor_ConvertToPNG(t *testing.T) {
 	t.Parallel()
 
-	proc := New(Params{MaxInputBytes: 50 << 20, MaxConcurrentProcessing: 1})
+	proc := New(Params{})
 	ctx := context.Background()
 
 	input := createTestJPEG(t, 200, 150)
 
-	transform := &imagerequest.Transform{
+	req := &imagerequest.Transform{
 		Size:    imagerequest.Size{Width: 200, Height: 150},
 		Format:  imagerequest.FormatPNG,
 		FitMode: imagerequest.FitCover,
 	}
 
-	result, err := proc.Process(ctx, bytes.NewReader(input), transform)
+	result, err := proc.Process(ctx, bytes.NewReader(input), req)
 	if err != nil {
 		t.Fatalf("Process() error = %v", err)
 	}
@@ -201,19 +201,19 @@ func processAndCheckSize(
 ) {
 	t.Helper()
 
-	proc := New(Params{MaxInputBytes: 50 << 20, MaxConcurrentProcessing: 1})
+	proc := New(Params{})
 	ctx := context.Background()
 
 	input := createTestJPEG(t, inputW, inputH)
 
-	transform := &imagerequest.Transform{
+	req := &imagerequest.Transform{
 		Size:    size,
 		Format:  imagerequest.FormatJPEG,
 		Quality: 85,
 		FitMode: imagerequest.FitCover,
 	}
 
-	result, err := proc.Process(ctx, bytes.NewReader(input), transform)
+	result, err := proc.Process(ctx, bytes.NewReader(input), req)
 	if err != nil {
 		t.Fatalf("Process() error = %v", err)
 	}
@@ -239,21 +239,21 @@ func TestImageProcessor_OriginalSize(t *testing.T) {
 func TestImageProcessor_FitContain(t *testing.T) {
 	t.Parallel()
 
-	proc := New(Params{MaxInputBytes: 50 << 20, MaxConcurrentProcessing: 1})
+	proc := New(Params{})
 	ctx := context.Background()
 
 	// 800x400 image (2:1 aspect) into 400x400 box with contain
 	// Should result in 400x200 (maintaining aspect ratio)
 	input := createTestJPEG(t, 800, 400)
 
-	transform := &imagerequest.Transform{
+	req := &imagerequest.Transform{
 		Size:    imagerequest.Size{Width: 400, Height: 400},
 		Format:  imagerequest.FormatJPEG,
 		Quality: 85,
 		FitMode: imagerequest.FitContain,
 	}
 
-	result, err := proc.Process(ctx, bytes.NewReader(input), transform)
+	result, err := proc.Process(ctx, bytes.NewReader(input), req)
 	if err != nil {
 		t.Fatalf("Process() error = %v", err)
 	}
@@ -285,18 +285,18 @@ func TestImageProcessor_ProportionalScale_HeightOnly(t *testing.T) {
 func TestImageProcessor_ProcessPNG(t *testing.T) {
 	t.Parallel()
 
-	proc := New(Params{MaxInputBytes: 50 << 20, MaxConcurrentProcessing: 1})
+	proc := New(Params{})
 	ctx := context.Background()
 
 	input := createTestPNG(t, 400, 300)
 
-	transform := &imagerequest.Transform{
+	req := &imagerequest.Transform{
 		Size:    imagerequest.Size{Width: 200, Height: 150},
 		Format:  imagerequest.FormatPNG,
 		FitMode: imagerequest.FitCover,
 	}
 
-	result, err := proc.Process(ctx, bytes.NewReader(input), transform)
+	result, err := proc.Process(ctx, bytes.NewReader(input), req)
 	if err != nil {
 		t.Fatalf("Process() error = %v", err)
 	}
@@ -312,6 +312,22 @@ func TestImageProcessor_ProcessPNG(t *testing.T) {
 	}
 }
 
+func TestImageProcessor_SupportedFormats(t *testing.T) {
+	t.Parallel()
+
+	proc := New(Params{})
+
+	inputFormats := proc.SupportedInputFormats()
+	if len(inputFormats) == 0 {
+		t.Error("SupportedInputFormats() returned empty slice")
+	}
+
+	outputFormats := proc.SupportedOutputFormats()
+	if len(outputFormats) == 0 {
+		t.Error("SupportedOutputFormats() returned empty slice")
+	}
+}
+
 func TestImageProcessor_RejectsOversizedInput(t *testing.T) {
 	t.Parallel()
 
@@ -330,18 +346,18 @@ func TestImageProcessor_RejectsOversizedInput(t *testing.T) {
 		t.Run(tt.name, func(t *testing.T) {
 			t.Parallel()
 
-			proc := New(Params{MaxInputBytes: 50 << 20, MaxConcurrentProcessing: 1})
+			proc := New(Params{})
 			ctx := context.Background()
 			input := createTestJPEG(t, tt.width, tt.height)
 
-			transform := &imagerequest.Transform{
+			req := &imagerequest.Transform{
 				Size:    imagerequest.Size{Width: 100, Height: 100},
 				Format:  imagerequest.FormatJPEG,
 				Quality: 85,
 				FitMode: imagerequest.FitCover,
 			}
 
-			_, err := proc.Process(ctx, bytes.NewReader(input), transform)
+			_, err := proc.Process(ctx, bytes.NewReader(input), req)
 			if err == nil {
 				t.Error("Process() should reject oversized input images")
 			}
@@ -356,20 +372,20 @@ func TestImageProcessor_RejectsOversizedInput(t *testing.T) {
 func TestImageProcessor_AcceptsMaxDimensionInput(t *testing.T) {
 	t.Parallel()
 
-	proc := New(Params{MaxInputBytes: 50 << 20, MaxConcurrentProcessing: 1})
+	proc := New(Params{})
 	ctx := context.Background()
 
 	// Create an image at exactly MaxInputDimension - should be accepted
 	input := createTestJPEG(t, MaxInputDimension, 100)
 
-	transform := &imagerequest.Transform{
+	req := &imagerequest.Transform{
 		Size:    imagerequest.Size{Width: 100, Height: 100},
 		Format:  imagerequest.FormatJPEG,
 		Quality: 85,
 		FitMode: imagerequest.FitCover,
 	}
 
-	result, err := proc.Process(ctx, bytes.NewReader(input), transform)
+	result, err := proc.Process(ctx, bytes.NewReader(input), req)
 	if err != nil {
 		t.Fatalf(
 			"Process() should accept images at MaxInputDimension, got error: %v",
@@ -387,19 +403,19 @@ func encodeAndCheck(
 ) {
 	t.Helper()
 
-	proc := New(Params{MaxInputBytes: 50 << 20, MaxConcurrentProcessing: 1})
+	proc := New(Params{})
 	ctx := context.Background()
 
 	input := createTestJPEG(t, 200, 150)
 
-	transform := &imagerequest.Transform{
+	req := &imagerequest.Transform{
 		Size:    imagerequest.Size{Width: 100, Height: 75},
 		Format:  format,
 		Quality: quality,
 		FitMode: imagerequest.FitCover,
 	}
 
-	result, err := proc.Process(ctx, bytes.NewReader(input), transform)
+	result, err := proc.Process(ctx, bytes.NewReader(input), req)
 	if err != nil {
 		t.Fatalf("Process() error = %v, want nil", err)
 	}
@@ -436,7 +452,7 @@ func TestImageProcessor_EncodeWebP(t *testing.T) {
 func TestImageProcessor_DecodeAVIF(t *testing.T) {
 	t.Parallel()
 
-	proc := New(Params{MaxInputBytes: 50 << 20, MaxConcurrentProcessing: 1})
+	proc := New(Params{})
 	ctx := context.Background()
 
 	// Load test AVIF file
@@ -446,14 +462,14 @@ func TestImageProcessor_DecodeAVIF(t *testing.T) {
 	}
 
 	// Request resize and convert to JPEG
-	transform := &imagerequest.Transform{
+	req := &imagerequest.Transform{
 		Size:    imagerequest.Size{Width: 2, Height: 2},
 		Format:  imagerequest.FormatJPEG,
 		Quality: 85,
 		FitMode: imagerequest.FitCover,
 	}
 
-	result, err := proc.Process(ctx, bytes.NewReader(input), transform)
+	result, err := proc.Process(ctx, bytes.NewReader(input), req)
 	if err != nil {
 		t.Fatalf("Process() error = %v, want nil (AVIF decoding should work)", err)
 	}
@@ -478,7 +494,7 @@ func TestImageProcessor_RejectsOversizedInputData(t *testing.T) {
 	// Create a processor with a very small byte limit
 	const limit = 1024
 
-	proc := New(Params{MaxInputBytes: limit, MaxConcurrentProcessing: 1})
+	proc := New(Params{MaxInputBytes: limit})
 	ctx := context.Background()
 
 	// Create a valid JPEG that exceeds the byte limit
@@ -487,14 +503,14 @@ func TestImageProcessor_RejectsOversizedInputData(t *testing.T) {
 		t.Fatalf("test JPEG must exceed %d bytes, got %d", limit, len(input))
 	}
 
-	transform := &imagerequest.Transform{
+	req := &imagerequest.Transform{
 		Size:    imagerequest.Size{Width: 100, Height: 75},
 		Format:  imagerequest.FormatJPEG,
 		Quality: 85,
 		FitMode: imagerequest.FitCover,
 	}
 
-	_, err := proc.Process(ctx, bytes.NewReader(input), transform)
+	_, err := proc.Process(ctx, bytes.NewReader(input), req)
 	if err == nil {
 		t.Fatal("Process() should reject input exceeding maxInputBytes")
 	}
@@ -511,17 +527,17 @@ func TestImageProcessor_AcceptsInputWithinLimit(t *testing.T) {
 	input := createTestJPEG(t, 10, 10)
 	limit := int64(len(input)) * 10 // 10× headroom
 
-	proc := New(Params{MaxInputBytes: limit, MaxConcurrentProcessing: 1})
+	proc := New(Params{MaxInputBytes: limit})
 	ctx := context.Background()
 
-	transform := &imagerequest.Transform{
+	req := &imagerequest.Transform{
 		Size:    imagerequest.Size{Width: 10, Height: 10},
 		Format:  imagerequest.FormatJPEG,
 		Quality: 85,
 		FitMode: imagerequest.FitCover,
 	}
 
-	result, err := proc.Process(ctx, bytes.NewReader(input), transform)
+	result, err := proc.Process(ctx, bytes.NewReader(input), req)
 	if err != nil {
 		t.Fatalf("Process() error = %v, want nil", err)
 	}
@@ -529,6 +545,22 @@ func TestImageProcessor_AcceptsInputWithinLimit(t *testing.T) {
 	defer func() { _ = result.Content.Close() }()
 }
 
+func TestImageProcessor_DefaultMaxInputBytes(t *testing.T) {
+	t.Parallel()
+
+	// Passing 0 should use the default
+	proc := New(Params{})
+	if proc.maxInputBytes != DefaultMaxInputBytes {
+		t.Errorf("maxInputBytes = %d, want %d", proc.maxInputBytes, DefaultMaxInputBytes)
+	}
+
+	// Passing negative should also use the default
+	proc = New(Params{MaxInputBytes: -1})
+	if proc.maxInputBytes != DefaultMaxInputBytes {
+		t.Errorf("maxInputBytes = %d, want %d", proc.maxInputBytes, DefaultMaxInputBytes)
+	}
+}
+
 func TestImageProcessor_EncodeAVIF(t *testing.T) {
 	t.Parallel()
 
@@ -538,15 +570,12 @@ func TestImageProcessor_EncodeAVIF(t *testing.T) {
 // processAndDecode runs input through Process and decodes the output with
 // vips, so a test can inspect the image a client would receive.
 func processAndDecode(
-	t *testing.T, input []byte, transform *imagerequest.Transform,
+	t *testing.T, input []byte, req *imagerequest.Transform,
 ) *vips.ImageRef {
 	t.Helper()
 
-	result, err := New(Params{
+	result, err := New(Params{}).Process(
-		MaxInputBytes:           50 << 20,
+		context.Background(), bytes.NewReader(input), req,
-		MaxConcurrentProcessing: 1,
-	}).Process(
-		context.Background(), bytes.NewReader(input), transform,
 	)
 	if err != nil {
 		t.Fatalf("Process() error = %v", err)

internal/imageprocessor/jpegxl_output_internal_test.go

@@ -16,16 +16,13 @@ import (
 func TestImageProcessor_EncodeJPEGXL(t *testing.T) {
 	t.Parallel()
 
-	transform := &imagerequest.Transform{
+	req := &imagerequest.Transform{
 		Size:   imagerequest.Size{Width: 100, Height: 75},
 		Format: imagerequest.FormatJXL,
 	}
 
-	result, err := New(Params{
+	result, err := New(Params{}).Process(
-		MaxInputBytes:           50 << 20,
+		t.Context(), bytes.NewReader(createTestJPEG(t, 200, 150)), req,
-		MaxConcurrentProcessing: 1,
-	}).Process(
-		t.Context(), bytes.NewReader(createTestJPEG(t, 200, 150)), transform,
 	)
 	if err != nil {
 		t.Fatalf("Process() error = %v", err)
@@ -67,10 +64,7 @@ func TestImageProcessor_JPEGXLQuality(t *testing.T) {
 	outputBytes := make(map[int]int64)
 
 	for _, quality := range []int{30, 90} {
-		result, err := New(Params{
+		result, err := New(Params{}).Process(t.Context(),
-			MaxInputBytes:           50 << 20,
-			MaxConcurrentProcessing: 1,
-		}).Process(t.Context(),
 			bytes.NewReader(input),
 			&imagerequest.Transform{Format: imagerequest.FormatJXL, Quality: quality})
 		if err != nil {

internal/imageprocessor/max_concurrent_processing_internal_test.go

@@ -5,6 +5,7 @@ import (
 	"context"
 	"errors"
 	"io"
+	"runtime"
 	"strings"
 	"sync"
 	"testing"
@@ -77,8 +78,8 @@ func (r *gatedReader) Read(p []byte) (int, error) {
 	return n, err
 }
 
-// smallJPEGTransform asks for a 5x5 JPEG.
+// smallJPEGRequest asks for a 5x5 JPEG.
-func smallJPEGTransform() *imagerequest.Transform {
+func smallJPEGRequest() *imagerequest.Transform {
 	return &imagerequest.Transform{
 		Size:    imagerequest.Size{Width: 5, Height: 5},
 		Format:  imagerequest.FormatJPEG,
@@ -93,7 +94,7 @@ func processInBackground(
 	proc *ImageProcessor, reader *gatedReader, results chan<- error,
 ) {
 	go func() {
-		result, err := proc.Process(context.Background(), reader, smallJPEGTransform())
+		result, err := proc.Process(context.Background(), reader, smallJPEGRequest())
 		if err == nil {
 			_ = result.Content.Close()
 		}
@@ -116,15 +117,47 @@ func waitForEntries(t *testing.T, entered <-chan struct{}, count int) {
 	}
 }
 
-func TestNewMakesMaxConcurrentProcessingSlots(t *testing.T) {
+func TestNewDefaultsMaxConcurrentProcessingToHalfTheCPUs(t *testing.T) {
 	t.Parallel()
 
-	proc := New(Params{MaxInputBytes: 50 << 20, MaxConcurrentProcessing: 3})
+	for _, limit := range []int{0, -1} {
+		proc := New(Params{MaxConcurrentProcessing: limit})
+		if got := cap(proc.processingSemaphore); got != max(runtime.GOMAXPROCS(0)/2, 1) {
+			t.Errorf("MaxConcurrentProcessing %d: %d slots, want %d, half the CPUs",
+				limit, got, max(runtime.GOMAXPROCS(0)/2, 1))
+		}
+	}
+
+	proc := New(Params{MaxConcurrentProcessing: 3})
 	if got := cap(proc.processingSemaphore); got != 3 {
 		t.Errorf("MaxConcurrentProcessing 3: %d slots, want 3", got)
 	}
 }
 
+// TestNewDefaultIsHalfTheCPUsAndAtLeastOne checks that New, given no
+// MaxConcurrentProcessing, makes one slot for every two CPUs Go uses,
+// rounded down, and one slot when Go uses one CPU.
+//
+//nolint:paralleltest // it sets GOMAXPROCS, which other tests read
+func TestNewDefaultIsHalfTheCPUsAndAtLeastOne(t *testing.T) {
+	cpusBefore := runtime.GOMAXPROCS(0)
+
+	t.Cleanup(func() { runtime.GOMAXPROCS(cpusBefore) })
+
+	for _, tc := range []struct{ cpus, want int }{
+		{1, 1}, {2, 1}, {3, 1}, {8, 4},
+	} {
+		runtime.GOMAXPROCS(tc.cpus)
+
+		got := cap(New(Params{}).processingSemaphore)
+		t.Logf("GOMAXPROCS %d: %d slots", tc.cpus, got)
+
+		if got != tc.want {
+			t.Errorf("GOMAXPROCS %d: %d slots, want %d", tc.cpus, got, tc.want)
+		}
+	}
+}
+
 // TestProcessNeverExceedsMaxConcurrentProcessing starts more Process calls
 // than MaxConcurrentProcessing allows and holds the first ones inside
 // Process until the test lets them go. No more than the limit may be
@@ -138,7 +171,7 @@ func TestProcessNeverExceedsMaxConcurrentProcessing(t *testing.T) {
 		calls = 6
 	)
 
-	proc := New(Params{MaxInputBytes: 50 << 20, MaxConcurrentProcessing: limit})
+	proc := New(Params{MaxConcurrentProcessing: limit})
 	input := createTestJPEG(t, 50, 50)
 
 	counter := &readingCounter{}
@@ -185,7 +218,7 @@ func TestProcessNeverExceedsMaxConcurrentProcessing(t *testing.T) {
 func TestProcessWaitsThenFailsWhenNoSlotFrees(t *testing.T) {
 	t.Parallel()
 
-	proc := New(Params{MaxInputBytes: 50 << 20, MaxConcurrentProcessing: 1})
+	proc := New(Params{MaxConcurrentProcessing: 1})
 	proc.processingWaitTimeout = 100 * time.Millisecond
 
 	input := createTestJPEG(t, 10, 10)
@@ -206,7 +239,7 @@ func TestProcessWaitsThenFailsWhenNoSlotFrees(t *testing.T) {
 	start := time.Now()
 
 	_, err := proc.Process(context.Background(), bytes.NewReader(input),
-		smallJPEGTransform())
+		smallJPEGRequest())
 	if !errors.Is(err, ErrTooManyImages) {
 		t.Fatalf("Process() error = %v, want ErrTooManyImages", err)
 	}
@@ -232,38 +265,38 @@ func TestProcessReleasesSlotOnError(t *testing.T) {
 
 	valid := createTestJPEG(t, 10, 10)
 
-	unsupported := smallJPEGTransform()
+	unsupported := smallJPEGRequest()
 	unsupported.Format = "bmp"
 
 	cases := []struct {
-		name      string
+		name  string
-		input     io.Reader
+		input io.Reader
-		transform *imagerequest.Transform
+		req   *imagerequest.Transform
 		// want is the error Process must return; nil means any error.
 		want error
 	}{
 		{
-			name:      "input cannot be read",
+			name:  "input cannot be read",
-			input:     iotest.ErrReader(errTestReadFailed),
+			input: iotest.ErrReader(errTestReadFailed),
-			transform: smallJPEGTransform(),
+			req:   smallJPEGRequest(),
-			want:      errTestReadFailed,
+			want:  errTestReadFailed,
 		},
 		{
-			name:      "input over the byte limit",
+			name:  "input over the byte limit",
-			input:     bytes.NewReader(createTestJPEG(t, 800, 600)),
+			input: bytes.NewReader(createTestJPEG(t, 800, 600)),
-			transform: smallJPEGTransform(),
+			req:   smallJPEGRequest(),
-			want:      ErrInputDataTooLarge,
+			want:  ErrInputDataTooLarge,
 		},
 		{
-			name:      "input not an image",
+			name:  "input not an image",
-			input:     strings.NewReader("not an image"),
+			input: strings.NewReader("not an image"),
-			transform: smallJPEGTransform(),
+			req:   smallJPEGRequest(),
 		},
 		{
-			name:      "output format not supported",
+			name:  "output format not supported",
-			input:     bytes.NewReader(valid),
+			input: bytes.NewReader(valid),
-			transform: unsupported,
+			req:   unsupported,
-			want:      ErrUnsupportedOutputFormat,
+			want:  ErrUnsupportedOutputFormat,
 		},
 	}
 
@@ -274,7 +307,7 @@ func TestProcessReleasesSlotOnError(t *testing.T) {
 			proc := New(Params{MaxInputBytes: 4096, MaxConcurrentProcessing: 1})
 			proc.processingWaitTimeout = 100 * time.Millisecond
 
-			_, err := proc.Process(context.Background(), tc.input, tc.transform)
+			_, err := proc.Process(context.Background(), tc.input, tc.req)
 			if err == nil || (tc.want != nil && !errors.Is(err, tc.want)) {
 				t.Fatalf("Process() error = %v, want %v", err, tc.want)
 			}
@@ -284,7 +317,7 @@ func TestProcessReleasesSlotOnError(t *testing.T) {
 			}
 
 			result, err := proc.Process(context.Background(), bytes.NewReader(valid),
-				smallJPEGTransform())
+				smallJPEGRequest())
 			if err != nil {
 				t.Fatalf("Process() after the error = %v, want nil", err)
 			}
@@ -301,7 +334,7 @@ func TestProcessReleasesSlotOnError(t *testing.T) {
 func TestWaitForProcessing(t *testing.T) {
 	t.Parallel()
 
-	proc := New(Params{MaxInputBytes: 50 << 20, MaxConcurrentProcessing: 2})
+	proc := New(Params{MaxConcurrentProcessing: 2})
 
 	gate := make(chan struct{})
 	entered := make(chan struct{}, 1)

internal/imageprocessor/orig_format_internal_test.go

@@ -27,10 +27,7 @@ func TestImageProcessor_OrigOfUnnamedFormatRefused(t *testing.T) {
 		t.Fatalf("failed to save test TIFF: %v", err)
 	}
 
-	result, err := New(Params{
+	result, err := New(Params{}).Process(
-		MaxInputBytes:           50 << 20,
-		MaxConcurrentProcessing: 1,
-	}).Process(
 		t.Context(), bytes.NewReader(tiff),
 		&imagerequest.Transform{Format: imagerequest.FormatOriginal},
 	)

internal/imageprocessor/source_formats_internal_test.go

@@ -14,7 +14,7 @@ import (
 func TestImageProcessor_LoadsEachSourceFormat(t *testing.T) {
 	t.Parallel()
 
-	proc := New(Params{MaxInputBytes: 50 << 20, MaxConcurrentProcessing: 1})
+	proc := New(Params{})
 
 	for _, format := range []imagerequest.Format{
 		imagerequest.FormatJPEG, imagerequest.FormatPNG, imagerequest.FormatGIF,

internal/imagerequest/imagerequest.go

@@ -1,12 +1,13 @@
 // Package imagerequest holds the image request type, its parts, its checks,
-// the parsing of the /v1/image/ path, the variant key, the source key and the
+// the parsing of the /v1/image/ path and the variant key. It imports no other
-// source's upstream URL. It imports no other package of this repo, so every
+// package of this repo, so every package that handles an image request can
-// package that handles an image request can import it.
+// import it.
 package imagerequest
 
 import (
 	"errors"
 	"net/url"
+	"time"
 )
 
 // ErrInvalidFitMode is returned when an invalid fit mode is provided.
@@ -84,13 +85,19 @@ func ValidateImageRequest(req *Request) error {
 }
 
 // Request represents a request for a processed image: the source to fetch
-// and the transform to apply to it. It holds exactly what the variant key
+// and the transform to apply to it.
-// covers.
 type Request struct {
 	// Source is what to fetch
 	Source Source
 	// Transform is what to do to it
 	Transform Transform
+	// Signature is the HMAC signature for non-allowlisted hosts
+	Signature string
+	// Expires is when the URL expires: the exp of a signed URL, or the expiry
+	// of an encrypted URL; the zero time if it has none
+	Expires time.Time
+	// AllowHTTP indicates whether HTTP (non-TLS) is allowed for this request
+	AllowHTTP bool
 }
 
 // Source is the image a request fetches from the origin.
@@ -118,20 +125,23 @@ type Transform struct {
 	FitMode Fit
 }
 
-// URL returns the source's upstream URL, with the scheme http when allowHTTP
+// SourceURL returns the full upstream URL to fetch: the source path
-// is true and https otherwise. net/url writes its path percent-encoded again,
+// percent-encoded again, as net/url writes a path, so each source path is one
-// so each source path is one upstream path. Its query is the source query as
+// upstream path, and the source query as it is, with the characters
-// it is, with the characters escapeQuery encodes percent-encoded.
+// escapeQuery encodes percent-encoded. Uses http:// scheme when AllowHTTP is
-func (s Source) URL(allowHTTP bool) *url.URL {
+// true, otherwise https://.
+func (r *Request) SourceURL() string {
 	scheme := "https"
-	if allowHTTP {
+	if r.AllowHTTP {
 		scheme = "http"
 	}
 
-	return &url.URL{
+	sourceURL := url.URL{
 		Scheme:   scheme,
-		Host:     s.Host,
+		Host:     r.Source.Host,
-		Path:     s.Path,
+		Path:     r.Source.Path,
-		RawQuery: escapeQuery(s.Query),
+		RawQuery: escapeQuery(r.Source.Query),
 	}
+
+	return sourceURL.String()
 }

internal/imagerequest/sourcekey.go

@@ -1,26 +0,0 @@
-package imagerequest
-
-import (
-	"crypto/sha256"
-	"encoding/hex"
-	"fmt"
-)
-
-// SourceKey is a SHA256 hash identifying a cached source image (hex-encoded).
-type SourceKey string
-
-// Key returns the key the source is cached under, made as CacheKey makes a
-// variant key.
-// Format: sha256(host:path:query)
-// The path and query go through escapePercentAndColon, so that a ":" in
-// either cannot be taken for the one between them.
-func (s Source) Key() SourceKey {
-	data := fmt.Sprintf("%s:%s:%s",
-		s.Host,
-		escapePercentAndColon(s.Path),
-		escapePercentAndColon(s.Query),
-	)
-	h := sha256.Sum256([]byte(data))
-
-	return SourceKey(hex.EncodeToString(h[:]))
-}

internal/imagerequest/sourcekey_internal_test.go

@@ -1,40 +0,0 @@
-package imagerequest
-
-import "testing"
-
-// TestSourceKey_DiffersForEachSource checks that each source gets a key of
-// 64 hex characters, the same each time, and that no two of these sources
-// share one, also when they differ only by a ":" or "%" in the path or query.
-func TestSourceKey_DiffersForEachSource(t *testing.T) {
-	t.Parallel()
-
-	sources := []Source{
-		{Host: testHostCDN, Path: testPathCat},
-		{Host: testHostCDN, Path: testPathCat, Query: "v=2"},
-		{Host: "other.example.com", Path: testPathCat},
-		{Host: testHostCDN, Path: "/c:d"},
-		{Host: testHostCDN, Path: "/c", Query: "d:"},
-		{Host: testHostCDN, Path: "/c%3Ad"},
-	}
-
-	seen := make(map[SourceKey]Source)
-
-	for _, source := range sources {
-		key := source.Key()
-		t.Logf("Key() of %+v = %s", source, key)
-
-		if len(key) != 64 {
-			t.Errorf("Key() of %+v length = %d, want 64", source, len(key))
-		}
-
-		if again := source.Key(); again != key {
-			t.Errorf("Key() of %+v not deterministic: %s vs %s", source, key, again)
-		}
-
-		if earlier, found := seen[key]; found {
-			t.Errorf("Key() gave %+v and %+v the same key", earlier, source)
-		}
-
-		seen[key] = source
-	}
-}

internal/imagerequest/sourceurl_internal_test.go

@@ -2,49 +2,57 @@ package imagerequest
 
 import "testing"
 
-func TestSource_URL_DefaultHTTPS(t *testing.T) {
+func TestImageRequest_SourceURL_DefaultHTTPS(t *testing.T) {
 	t.Parallel()
 
-	source := Source{
+	req := &Request{
-		Host:  testHostCDN,
+		Source: Source{
-		Path:  testPathCat,
+			Host:  testHostCDN,
-		Query: "v=2",
+			Path:  testPathCat,
+			Query: "v=2",
+		},
 	}
 
-	got := source.URL(false).String()
+	got := req.SourceURL()
 
 	want := "https://cdn.example.com/photos/cat.jpg?v=2"
 	if got != want {
-		t.Errorf("URL() = %q, want %q", got, want)
+		t.Errorf("SourceURL() = %q, want %q", got, want)
 	}
 }
 
-func TestSource_URL_AllowHTTP(t *testing.T) {
+func TestImageRequest_SourceURL_AllowHTTP(t *testing.T) {
 	t.Parallel()
 
-	source := Source{
+	req := &Request{
-		Host: "localhost:8080",
+		Source: Source{
-		Path: testPathCat,
+			Host: "localhost:8080",
+			Path: testPathCat,
+		},
+		AllowHTTP: true,
 	}
 
-	got := source.URL(true).String()
+	got := req.SourceURL()
 
 	want := "http://localhost:8080/photos/cat.jpg"
 	if got != want {
-		t.Errorf("URL() = %q, want %q", got, want)
+		t.Errorf("SourceURL() = %q, want %q", got, want)
 	}
 }
 
-func TestSource_URL_AllowHTTPFalse(t *testing.T) {
+func TestImageRequest_SourceURL_AllowHTTPFalse(t *testing.T) {
 	t.Parallel()
 
-	source := Source{
+	req := &Request{
-		Host: testHostCDN,
+		Source: Source{
-		Path: "/img.jpg",
+			Host: testHostCDN,
+			Path: "/img.jpg",
+		},
+		AllowHTTP: false,
 	}
 
-	got := source.URL(false).String()
+	got := req.SourceURL()
 	if got != "https://cdn.example.com/img.jpg" {
-		t.Errorf("URL() = %q, want https scheme", got)
+		t.Errorf("SourceURL() = %q, want https scheme", got)
 	}
 }

internal/imagerequest/urlparser.go

@@ -66,6 +66,23 @@ var queryEscapeRegex = regexp.MustCompile(`%[0-9A-Fa-f]{2}|[^!-~]|["#'<>%]`)
 var hostRegex = regexp.MustCompile(
 	`^(?:[A-Za-z0-9_.-]+|\[([0-9A-Fa-f:.]+)\])(?::[0-9]*)?$`)
 
+// ParseImagePath parses an image URL's path after /v1/image/, decoded once:
+// <host>/<path>/<size>.<format>, or <host>/<path>/<size> for JPEG XL.
+// Examples:
+//   - cdn.example.com/photos/cat.jpg/800x600.webp
+//   - cdn.example.com/photos/cat.jpg/0x0.jpeg
+//   - cdn.example.com/photos/cat.jpg/orig.png
+//   - cdn.example.com/photos/cat.jpg/800x600
+func ParseImagePath(path string) (*Request, error) {
+	// Strip leading slash if present (chi may include it)
+	path = strings.TrimPrefix(path, "/")
+	if path == "" {
+		return nil, ErrMissingHost
+	}
+
+	return parseImageComponents(path)
+}
+
 // ParseImageURL parses an image URL's path, decoded once as r.URL.Path holds
 // it: /v1/image/<host>/<path>/<size>.<format>, or /v1/image/<host>/<path>/<size>
 // for JPEG XL. A source query follows the source path after a "?", which the
@@ -126,15 +143,15 @@ func parseImageComponents(remainder string) (*Request, error) {
 		return nil, ErrMissingHost
 	}
 
-	// Later steps put the host unparsed into the allowlist check, the cache key
+	// Only the forms hostRegex describes are accepted. A "?", "#" or "@" in
-	// and the upstream URL, so only the forms hostRegex describes are accepted
+	// the host would make Go's URL parser, which the allowlist check and the
+	// fetcher use, read a different host from the one allowlisted and cached
 	hostMatch := hostRegex.FindStringSubmatch(host)
 	if hostMatch == nil {
 		return nil, fmt.Errorf("%w: %q", ErrInvalidHost, host)
 	}
 
-	// The allowlist check and the fetcher drop the brackets, so brackets
+	// Go's URL parser refuses brackets around anything but an IPv6 address
-	// around anything but an IPv6 address would give a host a second spelling
 	if inBrackets := hostMatch[1]; inBrackets != "" {
 		addr, err := netip.ParseAddr(inBrackets)
 		if err != nil || !addr.Is6() {
@@ -147,8 +164,9 @@ func parseImageComponents(remainder string) (*Request, error) {
 	// host
 	host = strings.ToLower(host)
 
-	// A "#" starts a URL's fragment, so neither the source path nor the source
+	// Go's URL parser, which the fetcher uses, reads a "#" as the start of a
-	// query may hold one
+	// fragment and drops the rest, so each text after it would be another
+	// spelling, cached under its own keys, of one upstream request
 	if strings.Contains(path, "#") {
 		return nil, fmt.Errorf(
 			`%w: the source path or query holds a "#"`, ErrInvalidPath)

internal/imagerequest/urlparser_internal_test.go

@@ -12,9 +12,9 @@ const (
 	testPathCat     = "/photos/cat.jpg"
 )
 
-// assertRequest compares the source host, path and query and the size and
+// assertParsedURL compares all fields of a parsed URL against the
-// format of got against want.
+// expected value.
-func assertRequest(t *testing.T, got, want *Request) {
+func assertParsedURL(t *testing.T, got, want *Request) {
 	t.Helper()
 
 	if got.Source.Host != want.Source.Host {
@@ -116,7 +116,7 @@ func TestParseImageURL(t *testing.T) {
 				t.Fatalf("ParseImageURL() unexpected error = %v", err)
 			}
 
-			assertRequest(t, got, tt.want)
+			assertParsedURL(t, got, tt.want)
 		})
 	}
 }
@@ -192,16 +192,76 @@ func TestParseImageURL_Errors(t *testing.T) {
 	}
 }
 
-// TestParseImageURL_LowerCasesHost verifies that the host is lower-cased, so
+func TestParseImagePath(t *testing.T) {
+	t.Parallel()
+
+	// ParseImagePath is for chi wildcard capture (no /v1/image/ prefix)
+	tests := []struct {
+		name    string
+		input   string
+		want    *Request
+		wantErr bool
+	}{
+		{
+			name:  "chi wildcard capture",
+			input: "cdn.example.com/photos/cat.jpg/800x600.webp",
+			want: &Request{
+				Source: Source{
+					Host: testHostCDN,
+					Path: testPathCat,
+				},
+				Transform: Transform{
+					Size:   Size{Width: 800, Height: 600},
+					Format: FormatWebP,
+				},
+			},
+		},
+		{
+			name:  "with leading slash from chi",
+			input: "/cdn.example.com/photos/cat.jpg/800x600.webp",
+			want: &Request{
+				Source: Source{
+					Host: testHostCDN,
+					Path: testPathCat,
+				},
+				Transform: Transform{
+					Size:   Size{Width: 800, Height: 600},
+					Format: FormatWebP,
+				},
+			},
+		},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			t.Parallel()
+
+			got, err := ParseImagePath(tt.input)
+			if (err != nil) != tt.wantErr {
+				t.Errorf("ParseImagePath() error = %v, wantErr %v", err, tt.wantErr)
+
+				return
+			}
+
+			if err != nil {
+				return
+			}
+
+			assertParsedURL(t, got, tt.want)
+		})
+	}
+}
+
+// TestParseImagePath_LowerCasesHost verifies that the host is lower-cased, so
 // that every spelling of its letter case is the same host to the allowlist,
 // the upstream connection limits and the cache, and that the path keeps its
 // case.
-func TestParseImageURL_LowerCasesHost(t *testing.T) {
+func TestParseImagePath_LowerCasesHost(t *testing.T) {
 	t.Parallel()
 
-	got, err := ParseImageURL("/v1/image/CDN.Example.COM/Photos/Cat.jpg/800x600.webp")
+	got, err := ParseImagePath("CDN.Example.COM/Photos/Cat.jpg/800x600.webp")
 	if err != nil {
-		t.Fatalf("ParseImageURL() error = %v", err)
+		t.Fatalf("ParseImagePath() error = %v", err)
 	}
 
 	t.Logf("parsed host %q, path %q", got.Source.Host, got.Source.Path)
@@ -215,21 +275,21 @@ func TestParseImageURL_LowerCasesHost(t *testing.T) {
 	}
 }
 
-// TestParseImageURL_RefusesUserinfo verifies that a host holding "@" is
+// TestParseImagePath_RefusesUserinfo verifies that a host holding "@" is
 // refused: a URL reads what comes before the "@" as a user name and password
 // and the rest as the host.
-func TestParseImageURL_RefusesUserinfo(t *testing.T) {
+func TestParseImagePath_RefusesUserinfo(t *testing.T) {
 	t.Parallel()
 
 	for _, input := range []string{
-		"/v1/image/user@cdn.example.com/photos/cat.jpg/800x600.webp",
+		"user@cdn.example.com/photos/cat.jpg/800x600.webp",
-		"/v1/image/user:password@cdn.example.com/photos/cat.jpg/800x600.webp",
+		"user:password@cdn.example.com/photos/cat.jpg/800x600.webp",
 	} {
-		got, err := ParseImageURL(input)
+		got, err := ParseImagePath(input)
-		t.Logf("ParseImageURL(%q) = %+v, error = %v", input, got, err)
+		t.Logf("ParseImagePath(%q) = %+v, error = %v", input, got, err)
 
 		if !errors.Is(err, ErrInvalidHost) {
-			t.Errorf("ParseImageURL(%q) error = %v, want %v",
+			t.Errorf("ParseImagePath(%q) error = %v, want %v",
 				input, err, ErrInvalidHost)
 		}
 	}
@@ -255,10 +315,6 @@ func TestParseImageURL_PathTraversal(t *testing.T) {
 			name:  "parent in middle of path",
 			input: "/v1/image/cdn.example.com/photos/../../../etc/passwd/800x600.jpeg",
 		},
-		{
-			name:  "parent directory in path",
-			input: "/v1/image/cdn.example.com/photos/../secret/800x600.jpeg",
-		},
 		{
 			name:  "encoded parent directory",
 			input: "/v1/image/cdn.example.com/photos/%2e%2e/secret/800x600.jpeg",
@@ -301,6 +357,40 @@ func TestParseImageURL_PathTraversal(t *testing.T) {
 	}
 }
 
+func TestParseImagePath_PathTraversal(t *testing.T) {
+	t.Parallel()
+
+	// Test path traversal via ParseImagePath (chi wildcard)
+	tests := []struct {
+		name  string
+		input string
+	}{
+		{
+			name:  "parent directory in path",
+			input: "cdn.example.com/photos/../secret/800x600.jpeg",
+		},
+		{
+			name:  "encoded traversal",
+			input: "cdn.example.com/photos/%2e%2e/secret/800x600.jpeg",
+		},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			t.Parallel()
+
+			_, err := ParseImagePath(tt.input)
+			if err == nil {
+				t.Error("ParseImagePath() should reject path traversal attempts")
+			}
+
+			if !errors.Is(err, ErrPathTraversal) {
+				t.Errorf("ParseImagePath() error = %v, want ErrPathTraversal", err)
+			}
+		})
+	}
+}
+
 // TestParseImageURL_SourceURLHoldingPercent25 verifies that a source path and
 // a source query holding %25, the percent-escape of a literal "%", are
 // accepted as they are, although decoding them once more leaves a "%" that
@@ -342,7 +432,7 @@ func TestParseImageURL_SourceURLHoldingPercent25(t *testing.T) {
 				t.Fatalf("ParseImageURL() unexpected error = %v", err)
 			}
 
-			assertRequest(t, got, tt.want)
+			assertParsedURL(t, got, tt.want)
 		})
 	}
 }

internal/imgcache/cache.go

@@ -2,23 +2,36 @@ package imgcache
 
 import (
 	"context"
+	"crypto/sha256"
 	"database/sql"
+	"encoding/hex"
+	"encoding/json"
 	"errors"
 	"fmt"
 	"io"
 	"log/slog"
 	"path/filepath"
+	"sync"
+	"sync/atomic"
 	"time"
 
-	"github.com/prometheus/client_golang/prometheus"
+	lru "github.com/hashicorp/golang-lru/v2"
+	"sneak.berlin/go/pixa/internal/httpfetcher"
 	"sneak.berlin/go/pixa/internal/imagerequest"
 )
 
-// maxPendingAccessTimes is how many access times of variants served from
+// Cache errors.
-// the cache, and how many of cached sources a miss used, may each wait to be
+var (
-// written, at about 80 bytes each. An access time handed over while that
+	ErrCacheMiss     = errors.New("cache miss")
-// many of its kind wait is dropped.
+	ErrNegativeCache = errors.New("negative cache hit")
-const maxPendingAccessTimes = 10000
+)
+
+// HTTP status code for successful fetch.
+const httpStatusOK = 200
+
+// metaCacheSize is how many variants' content types metaCache holds. A
+// variant not among them is served as before, reading its .meta file.
+const metaCacheSize = 10000
 
 // cleanExpiredBatchSize is the most expired negative cache rows one
 // statement of CleanExpired deletes. Each batch is a statement of its
@@ -32,12 +45,23 @@ type CacheConfig struct {
 	NegativeTTL time.Duration
 
 	// MaxBytes is the disk cache size limit in bytes that eviction
-	// enforces. Zero turns the disk cache off entirely: no cache
+	// enforces. Zero means no limit is enforced (no eviction).
-	// directories are created, lookups always miss, stores are no-ops,
-	// and the eviction goroutine only removes the expired negative cache
-	// rows.
 	MaxBytes int64
 
+	// UseDefaultMaxBytes makes NewCache replace MaxBytes with the
+	// default limit: 75% of the sum of the space free on the filesystem
+	// holding the cache and the bytes the cache already holds, at least
+	// DefaultCacheMaxBytesFloor. The config layer sets this when the
+	// operator omits cache_max_bytes.
+	UseDefaultMaxBytes bool
+
+	// DisableDiskCache turns the disk cache off entirely: no cache
+	// directories are created, lookups always miss, stores are
+	// no-ops, and the eviction goroutine only removes the expired
+	// negative cache rows. The config layer sets this when the
+	// operator configures cache_max_bytes: 0.
+	DisableDiskCache bool
+
 	// Logger receives accounting and eviction log output. A nil
 	// Logger means slog.Default().
 	Logger *slog.Logger
@@ -45,11 +69,16 @@ type CacheConfig struct {
 
 // Cache implements the caching layer for the image proxy.
 type Cache struct {
-	db         *sql.DB
+	db          *sql.DB
-	srcContent *ContentStorage // source images by source key
+	srcContent  *ContentStorage  // source images by content hash
-	variants   *VariantStorage // processed variants by cache key
+	variants    *VariantStorage  // processed variants by cache key
-	config     CacheConfig
+	srcMetadata *MetadataStorage // source metadata by host/path
-	log        *slog.Logger
+	config      CacheConfig
+	log         *slog.Logger
+
+	// disabled means the disk cache is turned off entirely: lookups
+	// always miss, stores are no-ops, and no eviction runs.
+	disabled bool
 
 	// Eviction machinery. The channels are created in NewCache so
 	// stores can signal write pressure without racing StartEviction.
@@ -59,30 +88,47 @@ type Cache struct {
 	evictionDone     chan struct{}
 	evictionCancel   context.CancelFunc
 
-	// pendingVariantAccessTimes holds the variants served from the cache,
+	// The pending counts: hits, misses, upstream fetches and transforms
-	// and pendingSourceAccessTimes the cached sources a miss used, whose
+	// whose write to the database missed its request's deadline, kept
-	// access time is still to be written by the goroutine that
+	// here until they are written by the goroutine that
-	// StartPendingAccessTimeWrites starts, so that no request waits for the
+	// StartPendingCountWrites starts. As for eviction, the channels are
-	// database to record it. As for eviction, the channels are created in
+	// created in NewCache: pendingCountsAdded wakes that goroutine, and
-	// NewCache, so that requests can hand over access times before that
+	// pendingCountsDone is closed when it returns. pendingCountsCancel,
-	// goroutine starts; pendingAccessTimesDone is closed when it returns.
+	// set by StartPendingCountWrites, cancels its context, which tells it
-	// pendingAccessTimesCancel, set by StartPendingAccessTimeWrites,
+	// to finish.
-	// cancels its context, which tells it to finish.
+	pendingHits               atomic.Int64
-	pendingVariantAccessTimes chan imagerequest.VariantKey
+	pendingMisses             atomic.Int64
-	pendingSourceAccessTimes  chan imagerequest.SourceKey
+	pendingUpstreamFetches    atomic.Int64
-	pendingAccessTimesDone    chan struct{}
+	pendingUpstreamFetchBytes atomic.Int64
-	pendingAccessTimesCancel  context.CancelFunc
+	pendingTransforms         atomic.Int64
+	pendingCountsAdded        chan struct{}
+	pendingCountsDone         chan struct{}
+	pendingCountsCancel       context.CancelFunc
 
-	// The counts of what the cache does, made and registered by
+	// Held through each write of the pending counts, and by
-	// registerCounters (see counters.go)
+	// readHitAndMissCounts to read them
-	cacheHits          prometheus.Counter
+	pendingCountsWriteMutex sync.Mutex
-	cacheMisses        prometheus.Counter
+
-	upstreamFetches    prometheus.Counter
+	// metaCache holds the content types of the variants most recently
-	upstreamFetchBytes prometheus.Counter
+	// stored or served, so a hit does not read the variant's .meta file.
-	transforms         prometheus.Counter
+	// It never stands in for the variant file, which is always opened.
+	metaCache *lru.Cache[imagerequest.VariantKey, string]
+
+	// contentLocks serializes StoreSource and evictSourceBlob per
+	// content hash, closing the race window between an eviction's row
+	// deletion and its file unlink against a concurrent store of
+	// identical content.
+	contentLocks *contentLock
+
+	// evictSourceBlobTestHook, when set, is invoked by evictSourceBlob
+	// after its row-deletion transaction commits and before the
+	// content file is unlinked. It exists solely so tests can
+	// deterministically pause inside that window to exercise
+	// concurrent stores against it; production code leaves it nil.
+	evictSourceBlobTestHook func(ContentHash)
 
 	// reconciliationPageSize is the most rows one read of a content table
-	// returns in the reconciliation pass. NewCache sets it to
+	// returns in the reconciliation pass and in Stats. newCache sets it to
 	// defaultReconciliationPageSize; tests set it smaller.
 	reconciliationPageSize int
 
@@ -93,37 +139,43 @@ type Cache struct {
 	reconciliationReadTestHook func(rows int)
 }
 
-// NewCache creates a new cache instance and registers its counters on
+// NewCache creates a new cache instance.
-// metricsRegistry. pixad passes Prometheus' default registry, the one
+func NewCache(db *sql.DB, config CacheConfig) (*Cache, error) {
-// /metrics serves. It makes no database call.
+	return newCache(db, config, defaultFreeSpaceProbe)
-func NewCache(
+}
-	db *sql.DB, config CacheConfig, metricsRegistry prometheus.Registerer,
+
+// newCache is NewCache with the free-space probe passed in, so tests
+// can fake the free space the default limit is worked out from.
+func newCache(
+	db *sql.DB, config CacheConfig, probe FreeSpaceProbeFunc,
 ) (*Cache, error) {
 	log := config.Logger
 	if log == nil {
 		log = slog.Default()
 	}
 
+	metaCache, err := lru.New[imagerequest.VariantKey, string](metaCacheSize)
+	if err != nil {
+		return nil, fmt.Errorf("failed to create variant content type cache: %w", err)
+	}
+
 	c := &Cache{
 		db:               db,
 		config:           config,
 		log:              log,
+		disabled:         config.DisableDiskCache,
 		evictionPressure: make(chan struct{}, 1),
 		evictionDone:     make(chan struct{}),
+		metaCache:        metaCache,
+		contentLocks:     newContentLock(),
 
-		pendingVariantAccessTimes: make(chan imagerequest.VariantKey, maxPendingAccessTimes),
+		pendingCountsAdded: make(chan struct{}, 1),
-		pendingSourceAccessTimes:  make(chan imagerequest.SourceKey, maxPendingAccessTimes),
+		pendingCountsDone:  make(chan struct{}),
-		pendingAccessTimesDone:    make(chan struct{}),
 
 		reconciliationPageSize: defaultReconciliationPageSize,
 	}
 
-	err := c.registerCounters(metricsRegistry)
+	if c.disabled {
-	if err != nil {
-		return nil, err
-	}
-
-	if config.MaxBytes == 0 {
 		return c, nil
 	}
 
@@ -135,80 +187,184 @@ func NewCache(
 	}
 
 	variants, err := NewVariantStorage(
-		filepath.Join(config.StateDir, "cache", "variants"),
+		filepath.Join(config.StateDir, "cache", "variants"), log,
 	)
 	if err != nil {
 		return nil, fmt.Errorf("failed to create variant storage: %w", err)
 	}
 
+	srcMetadata, err := NewMetadataStorage(
+		filepath.Join(config.StateDir, "cache", "metadata"),
+	)
+	if err != nil {
+		return nil, fmt.Errorf("failed to create source metadata storage: %w", err)
+	}
+
 	c.srcContent = srcContent
 	c.variants = variants
+	c.srcMetadata = srcMetadata
+
+	if config.UseDefaultMaxBytes {
+		limit, err := c.computeDefaultMaxBytes(context.Background(), probe)
+		if err != nil {
+			return nil, err
+		}
+
+		c.config.MaxBytes = limit
+
+		log.Info("computed default cache size limit from free space and cache contents",
+			"cache_max_bytes", limit,
+			"cache_dir", filepath.Join(config.StateDir, "cache"),
+		)
+	}
 
 	return c, nil
 }
 
-// GetVariant opens a cached variant and returns a reader, its size and its
+// LookupResult contains the result of a cache lookup.
-// content type, read from the variant file's first bytes. It opens that one
+type LookupResult struct {
-// file and makes no database call. A variant not in the cache, or any
+	Hit         bool
-// variant of a disabled cache, is ErrNotFound; a variant file whose first
+	CacheKey    imagerequest.VariantKey
-// bytes are not those of an output format is another error.
+	ContentType string
+	SizeBytes   int64
+	CacheStatus CacheStatus
+}
+
+// Lookup checks if a processed variant exists on disk. Hits touch the
+// variant's LRU timestamp; a disabled cache always misses.
+func (c *Cache) Lookup(
+	ctx context.Context, req *imagerequest.Request,
+) (*LookupResult, error) {
+	cacheKey := imagerequest.CacheKey(req)
+
+	// Check variant storage directly - no DB needed for cache hits
+	if !c.disabled && c.variants.Exists(cacheKey) {
+		c.touchVariant(ctx, cacheKey)
+
+		return &LookupResult{
+			Hit:         true,
+			CacheKey:    cacheKey,
+			CacheStatus: CacheHit,
+		}, nil
+	}
+
+	return &LookupResult{
+		Hit:         false,
+		CacheKey:    cacheKey,
+		CacheStatus: CacheMiss,
+	}, nil
+}
+
+// GetVariant returns a reader, size, and content type for a cached
+// variant. The content type comes from metaCache, or else from the
+// variant's .meta file and is then kept in metaCache. A variant with
+// no .meta file is served as application/octet-stream, which is not
+// kept.
 func (c *Cache) GetVariant(
 	cacheKey imagerequest.VariantKey,
 ) (io.ReadCloser, int64, string, error) {
-	if c.config.MaxBytes == 0 {
+	if c.disabled {
 		return nil, 0, "", ErrNotFound
 	}
 
-	return c.variants.LoadWithContentType(cacheKey)
+	contentType, known := c.metaCache.Get(cacheKey)
+	if !known {
+		return c.loadVariantWithMeta(cacheKey)
+	}
+
+	reader, size, err := c.variants.LoadWithSize(cacheKey)
+	if err != nil {
+		// The file is gone, e.g. deleted outside pixa
+		c.metaCache.Remove(cacheKey)
+
+		return nil, 0, "", err
+	}
+
+	return reader, size, contentType, nil
 }
 
-// StoreSource stores fetched source content: it writes the file named by
+// StoreSource stores fetched source content and metadata. On a
-// the source's key, then the source's row. When the row cannot be written
+// disabled cache it is a no-op returning an empty hash.
-// the file is removed and the store fails, as a file with no row is never
-// served or evicted; if that removal fails too, the next reconciliation
-// pass removes the file. On a disabled cache it is a no-op returning an
-// empty key.
 func (c *Cache) StoreSource(
 	ctx context.Context,
 	req *imagerequest.Request,
 	content io.Reader,
-) (imagerequest.SourceKey, error) {
+	result *httpfetcher.FetchResult,
-	if c.config.MaxBytes == 0 {
+) (ContentHash, error) {
+	if c.disabled {
 		return "", nil
 	}
 
+	// Hash the content ourselves (rather than via srcContent.Store,
+	// which would hash internally) so the content hash is known before
+	// any file or database work happens: that lets the entire store be
+	// serialized, per hash, against a concurrent eviction of the same
+	// content below.
 	data, err := io.ReadAll(content)
 	if err != nil {
 		return "", fmt.Errorf("failed to read source content: %w", err)
 	}
 
-	key := req.Source.Key()
+	sum := sha256.Sum256(data)
+	contentHash := ContentHash(hex.EncodeToString(sum[:]))
 
-	size, err := c.srcContent.Store(key, data)
+	// Hold the content hash's lock for the whole store operation. A
+	// concurrent eviction of this exact hash (the real SHA-256 dedup
+	// case: a different source path whose bytes hash identically)
+	// deletes the accounting rows and unlinks the file inside the same
+	// lock, so the two can never interleave: either this store
+	// completes first (and a subsequent eviction removes it together
+	// with its rows and file, correctly), or eviction completes first
+	// (and this store finds the file already gone and recreates it
+	// fresh) — never a fresh row left pointing at a file eviction is
+	// mid-unlink on.
+	unlock := c.contentLocks.Lock(string(contentHash))
+	defer unlock()
+
+	size, err := c.srcContent.StoreHashed(contentHash, data)
 	if err != nil {
 		return "", fmt.Errorf("failed to store source content: %w", err)
 	}
 
-	_, err = c.db.ExecContext(ctx, `
+	// Store in database
-		INSERT INTO source_content (source_key, size_bytes)
+	pathHash := HashPath(req.Source.Path + "?" + req.Source.Query)
-		VALUES (?, ?)
-		ON CONFLICT(source_key) DO UPDATE SET
-			size_bytes = excluded.size_bytes,
-			last_accessed_at = CURRENT_TIMESTAMP
-	`, string(key), size)
-	if err != nil {
-		deleteErr := c.srcContent.Delete(key)
-		if deleteErr != nil {
-			c.log.Warn("failed to remove source file whose row was not written",
-				"source_key", key, "error", deleteErr)
-		}
 
+	headersJSON, err := json.Marshal(result.Headers)
+	if err != nil {
+		return "", fmt.Errorf("failed to marshal response headers: %w", err)
+	}
+
+	_, err = c.db.ExecContext(ctx, `
+		INSERT INTO source_content (content_hash, content_type, size_bytes)
+		VALUES (?, ?, ?)
+		ON CONFLICT(content_hash) DO NOTHING
+	`, contentHash, result.ContentType, size)
+	if err != nil {
 		return "", fmt.Errorf("failed to insert source content: %w", err)
 	}
 
+	_, err = c.db.ExecContext(ctx, `
+		INSERT INTO source_metadata
+			(source_host, source_path, source_query, path_hash,
+			 content_hash, status_code, content_type, response_headers)
+		VALUES (?, ?, ?, ?, ?, ?, ?, ?)
+		ON CONFLICT(source_host, source_path, source_query) DO UPDATE SET
+			content_hash = excluded.content_hash,
+			status_code = excluded.status_code,
+			content_type = excluded.content_type,
+			response_headers = excluded.response_headers,
+			fetched_at = CURRENT_TIMESTAMP
+	`, req.Source.Host, req.Source.Path, req.Source.Query, pathHash,
+		contentHash, httpStatusOK, result.ContentType, string(headersJSON))
+	if err != nil {
+		return "", fmt.Errorf("failed to insert source metadata: %w", err)
+	}
+
+	c.writeMetadataSidecar(req, pathHash, contentHash, result)
+
 	c.notifyWritePressure()
 
-	return key, nil
+	return contentHash, nil
 }
 
 // StoreVariant stores a processed variant by its cache key and records
@@ -216,24 +372,28 @@ func (c *Cache) StoreSource(
 // accounting insert is best-effort (the startup reconciliation pass
 // adopts any variant file that misses its accounting row).
 func (c *Cache) StoreVariant(
-	ctx context.Context, cacheKey imagerequest.VariantKey, content io.Reader,
+	ctx context.Context, cacheKey imagerequest.VariantKey,
+	content io.Reader, contentType string,
 ) error {
-	if c.config.MaxBytes == 0 {
+	if c.disabled {
 		return nil
 	}
 
-	size, err := c.variants.Store(cacheKey, content)
+	size, err := c.variants.Store(cacheKey, content, contentType)
 	if err != nil {
 		return err
 	}
 
+	c.metaCache.Add(cacheKey, contentType)
+
 	_, err = c.db.ExecContext(ctx, `
-		INSERT INTO variant_content (cache_key, size_bytes)
+		INSERT INTO variant_content (cache_key, size_bytes, content_type)
-		VALUES (?, ?)
+		VALUES (?, ?, ?)
 		ON CONFLICT(cache_key) DO UPDATE SET
 			size_bytes = excluded.size_bytes,
+			content_type = excluded.content_type,
 			last_accessed_at = CURRENT_TIMESTAMP
-	`, string(cacheKey), size)
+	`, string(cacheKey), size, contentType)
 	if err != nil {
 		c.log.Warn("failed to record variant in size accounting",
 			"cache_key", cacheKey, "error", err)
@@ -245,53 +405,60 @@ func (c *Cache) StoreVariant(
 }
 
 // LookupSource checks if we have cached source content for a request.
-// Returns the source key if found, or an empty key if not; a row whose
+// Returns the content hash and content type if found, or empty values
-// file is missing is not found. A source found has its access time
+// if not. Hits touch the blob's LRU timestamp; a disabled cache always
-// handed to the background writer without waiting (see
+// reports no cached source.
-// touchSourceContent); a disabled cache always reports no cached source.
 func (c *Cache) LookupSource(
 	ctx context.Context, req *imagerequest.Request,
-) (imagerequest.SourceKey, error) {
+) (ContentHash, string, error) {
-	if c.config.MaxBytes == 0 {
+	if c.disabled {
-		return "", nil
+		return "", "", nil
 	}
 
-	key := req.Source.Key()
+	var hashStr, contentType string
-
-	var rowExists bool
 
 	err := c.db.QueryRowContext(ctx, `
-		SELECT EXISTS(SELECT 1 FROM source_content WHERE source_key = ?)
+		SELECT content_hash, content_type FROM source_metadata
-	`, string(key)).Scan(&rowExists)
+		WHERE source_host = ? AND source_path = ? AND source_query = ?
-	if err != nil {
+	`, req.Source.Host, req.Source.Path, req.Source.Query).Scan(&hashStr, &contentType)
-		return "", fmt.Errorf("failed to lookup source: %w", err)
+
+	if errors.Is(err, sql.ErrNoRows) {
+		return "", "", nil
 	}
 
-	if !rowExists {
+	if err != nil {
-		return "", nil
+		return "", "", fmt.Errorf("failed to lookup source: %w", err)
 	}
 
+	contentHash := ContentHash(hashStr)
+
 	// Verify the content file exists
-	if !c.srcContent.Exists(key) {
+	if !c.srcContent.Exists(contentHash) {
-		return "", nil
+		return "", "", nil
 	}
 
-	c.touchSourceContent(key)
+	c.touchSourceContent(ctx, contentHash)
 
-	return key, nil
+	return contentHash, contentType, nil
 }
 
 // StoreNegative stores a negative cache entry for a failed fetch.
-func (c *Cache) StoreNegative(ctx context.Context, req *imagerequest.Request) error {
+func (c *Cache) StoreNegative(
+	ctx context.Context, req *imagerequest.Request, statusCode int, errMsg string,
+) error {
 	expiresAt := time.Now().UTC().Add(c.config.NegativeTTL)
 
 	_, err := c.db.ExecContext(ctx, `
 		INSERT INTO negative_cache
-			(source_host, source_path, source_query, expires_at)
+			(source_host, source_path, source_query, status_code,
-		VALUES (?, ?, ?, ?)
+			 error_message, expires_at)
+		VALUES (?, ?, ?, ?, ?, ?)
 		ON CONFLICT(source_host, source_path, source_query) DO UPDATE SET
+			status_code = excluded.status_code,
+			error_message = excluded.error_message,
+			fetched_at = CURRENT_TIMESTAMP,
 			expires_at = excluded.expires_at
-	`, req.Source.Host, req.Source.Path, req.Source.Query, expiresAt)
+	`, req.Source.Host, req.Source.Path, req.Source.Query, statusCode, errMsg, expiresAt)
 	if err != nil {
 		return fmt.Errorf("failed to insert negative cache: %w", err)
 	}
@@ -299,16 +466,33 @@ func (c *Cache) StoreNegative(ctx context.Context, req *imagerequest.Request) er
 	return nil
 }
 
-// GetSourceContent returns a reader for cached source content by its key,
+// GetSourceMetadataID returns the source metadata ID for a request.
+func (c *Cache) GetSourceMetadataID(
+	ctx context.Context, req *imagerequest.Request,
+) (int64, error) {
+	var id int64
+
+	err := c.db.QueryRowContext(ctx, `
+		SELECT id FROM source_metadata
+		WHERE source_host = ? AND source_path = ? AND source_query = ?
+	`, req.Source.Host, req.Source.Path, req.Source.Query).Scan(&id)
+	if err != nil {
+		return 0, fmt.Errorf("failed to get source metadata ID: %w", err)
+	}
+
+	return id, nil
+}
+
+// GetSourceContent returns a reader for cached source content by its hash,
 // and the content's size in bytes.
 func (c *Cache) GetSourceContent(
-	key imagerequest.SourceKey,
+	contentHash ContentHash,
 ) (io.ReadCloser, int64, error) {
-	if c.config.MaxBytes == 0 {
+	if c.disabled {
 		return nil, 0, ErrNotFound
 	}
 
-	return c.srcContent.LoadWithSize(key)
+	return c.srcContent.LoadWithSize(contentHash)
 }
 
 // CleanExpired removes the negative cache rows past their expiry, one
@@ -334,63 +518,232 @@ func (c *Cache) CleanExpired(ctx context.Context) error {
 	}
 }
 
-// IncrementStats counts a cache hit or miss, and an upstream fetch that read
+// Stats returns cache statistics.
-// fetchBytes bytes, as IncrementUpstreamFetch does. Like the other Increment
+func (c *Cache) Stats(ctx context.Context) (*CacheStats, error) {
-// methods, it only adds to the cache's Prometheus counters in memory (see
+	var stats CacheStats
-// counters.go), so it never waits for the database or fails.
+
-func (c *Cache) IncrementStats(hit bool, fetchBytes int64) {
+	err := c.readHitAndMissCounts(ctx, &stats)
-	if hit {
+	if err != nil && !errors.Is(err, sql.ErrNoRows) {
-		c.cacheHits.Inc()
+		return nil, fmt.Errorf("failed to get cache stats: %w", err)
-	} else {
-		c.cacheMisses.Inc()
 	}
 
-	c.IncrementUpstreamFetch(fetchBytes)
+	// Count and size the cached source images and processed variants from
+	// their tables. A disabled cache holds none, whatever rows an earlier
+	// run left.
+	if !c.disabled {
+		err = c.db.QueryRowContext(ctx, `
+			SELECT (SELECT COUNT(*) FROM source_content)
+			     + (SELECT COUNT(*) FROM variant_content)
+		`).Scan(&stats.TotalItems)
+		if err != nil {
+			c.log.Warn("failed to count cache items for stats", "error", err)
+		}
+
+		stats.TotalSizeBytes, err = c.sumContentSizeBytes(ctx)
+		if err != nil {
+			c.log.Warn("failed to sum cache size for stats", "error", err)
+		}
+	}
+
+	// Compute hit rate as a ratio
+	if stats.HitCount+stats.MissCount > 0 {
+		stats.HitRate = float64(stats.HitCount) / float64(stats.HitCount+stats.MissCount)
+	}
+
+	return &stats, nil
+}
+
+// IncrementStats counts a cache hit or miss, and an upstream fetch that read
+// fetchBytes bytes, as IncrementUpstreamFetch does. Like the other Increment
+// methods, it writes to the database with ctx's deadline but not its
+// cancellation, so an ended request is still counted but does not wait for
+// the database past its deadline; a count not written by then becomes a
+// pending count (see StartPendingCountWrites).
+func (c *Cache) IncrementStats(ctx context.Context, hit bool, fetchBytes int64) {
+	countCtx, cancel := withoutCancelKeepingDeadline(ctx)
+	defer cancel()
+
+	var err error
+
+	pendingCount := &c.pendingMisses
+
+	if hit {
+		pendingCount = &c.pendingHits
+
+		_, err = c.db.ExecContext(countCtx, `
+			UPDATE cache_stats
+			SET hit_count = hit_count + 1,
+			    last_updated_at = CURRENT_TIMESTAMP
+			WHERE id = 1
+		`)
+	} else {
+		_, err = c.db.ExecContext(countCtx, `
+			UPDATE cache_stats
+			SET miss_count = miss_count + 1,
+			    last_updated_at = CURRENT_TIMESTAMP
+			WHERE id = 1
+		`)
+	}
+
+	switch {
+	case errors.Is(err, context.DeadlineExceeded):
+		c.addPendingCount(pendingCount, 1)
+	case err != nil:
+		c.log.Warn("failed to count cache hit or miss", "hit", hit, "error", err)
+	}
+
+	c.IncrementUpstreamFetch(ctx, fetchBytes)
 }
 
 // IncrementUpstreamFetch counts one upstream fetch that read fetchBytes bytes.
 // A fetch that read no bytes is not counted.
-func (c *Cache) IncrementUpstreamFetch(fetchBytes int64) {
+func (c *Cache) IncrementUpstreamFetch(ctx context.Context, fetchBytes int64) {
 	if fetchBytes <= 0 {
 		return
 	}
 
-	c.upstreamFetches.Inc()
+	countCtx, cancel := withoutCancelKeepingDeadline(ctx)
-	c.upstreamFetchBytes.Add(float64(fetchBytes))
+	defer cancel()
+
+	_, err := c.db.ExecContext(countCtx, `
+		UPDATE cache_stats
+		SET upstream_fetch_count = upstream_fetch_count + 1,
+		    upstream_fetch_bytes = upstream_fetch_bytes + ?,
+		    last_updated_at = CURRENT_TIMESTAMP
+		WHERE id = 1
+	`, fetchBytes)
+
+	switch {
+	case errors.Is(err, context.DeadlineExceeded):
+		c.addPendingCount(&c.pendingUpstreamFetches, 1)
+		c.addPendingCount(&c.pendingUpstreamFetchBytes, fetchBytes)
+	case err != nil:
+		c.log.Warn("failed to count upstream fetch",
+			"fetch_bytes", fetchBytes, "error", err)
+	}
 }
 
 // IncrementTransformCount counts one image transcoded by the image processor.
-func (c *Cache) IncrementTransformCount() {
+func (c *Cache) IncrementTransformCount(ctx context.Context) {
-	c.transforms.Inc()
+	countCtx, cancel := withoutCancelKeepingDeadline(ctx)
-}
+	defer cancel()
 
-// touchVariant hands the access time of a variant served from the cache to
+	_, err := c.db.ExecContext(countCtx, `
-// the goroutine StartPendingAccessTimeWrites starts, which writes it in its
+		UPDATE cache_stats
-// next batch. It never waits: while maxPendingAccessTimes access times of
+		SET transform_count = transform_count + 1,
-// variants wait to be written, this one is dropped, and the variant only
+		    last_updated_at = CURRENT_TIMESTAMP
-// looks colder to eviction.
+		WHERE id = 1
-func (c *Cache) touchVariant(cacheKey imagerequest.VariantKey) {
+	`)
-	select {
+
-	case c.pendingVariantAccessTimes <- cacheKey:
+	switch {
-	default:
+	case errors.Is(err, context.DeadlineExceeded):
+		c.addPendingCount(&c.pendingTransforms, 1)
+	case err != nil:
+		c.log.Warn("failed to count transform", "error", err)
 	}
 }
 
-// touchSourceContent hands the access time of a cached source a miss used to
+// readHitAndMissCounts sets the hit and miss counts of stats to those of the
-// the goroutine StartPendingAccessTimeWrites starts, which writes it in its
+// cache_stats row plus the pending hits and misses (see
-// next batch. It never waits: while maxPendingAccessTimes access times of
+// StartPendingCountWrites). It holds pendingCountsWriteMutex, so that no
-// sources wait to be written, this one is dropped, and the source only looks
+// write of the pending counts falls between the two reads.
-// colder to eviction.
+func (c *Cache) readHitAndMissCounts(ctx context.Context, stats *CacheStats) error {
-func (c *Cache) touchSourceContent(key imagerequest.SourceKey) {
+	c.pendingCountsWriteMutex.Lock()
-	select {
+	defer c.pendingCountsWriteMutex.Unlock()
-	case c.pendingSourceAccessTimes <- key:
+
-	default:
+	err := c.db.QueryRowContext(ctx, `
+		SELECT hit_count, miss_count
+		FROM cache_stats WHERE id = 1
+	`).Scan(&stats.HitCount, &stats.MissCount)
+
+	stats.HitCount += c.pendingHits.Load()
+	stats.MissCount += c.pendingMisses.Load()
+
+	return err
+}
+
+// loadVariantWithMeta is GetVariant for a variant metaCache does not
+// hold: it reads the content type from the variant's .meta file and
+// keeps it in metaCache, unless a StoreVariant has put one there
+// meanwhile, as the store's is newer. A read that finds no .meta file,
+// as one can when the store's .meta write failed or an eviction removed
+// the file after the read opened the variant, serves
+// application/octet-stream and keeps nothing, so metaCache only ever
+// holds a type read from a .meta file or passed to StoreVariant.
+func (c *Cache) loadVariantWithMeta(
+	cacheKey imagerequest.VariantKey,
+) (io.ReadCloser, int64, string, error) {
+	reader, size, contentType, err := c.variants.LoadWithMeta(cacheKey)
+	if err != nil {
+		return nil, 0, "", err
+	}
+
+	if contentType == "" {
+		return reader, size, fallbackContentType, nil
+	}
+
+	c.metaCache.ContainsOrAdd(cacheKey, contentType)
+
+	return reader, size, contentType, nil
+}
+
+// writeMetadataSidecar writes the JSON metadata sidecar of a stored source.
+// A failure is logged and is otherwise non-fatal; the metadata is in the
+// database.
+func (c *Cache) writeMetadataSidecar(
+	req *imagerequest.Request,
+	pathHash PathHash,
+	contentHash ContentHash,
+	result *httpfetcher.FetchResult,
+) {
+	meta := &SourceMetadata{
+		Host:            req.Source.Host,
+		Path:            req.Source.Path,
+		Query:           req.Source.Query,
+		ContentHash:     string(contentHash),
+		StatusCode:      result.StatusCode,
+		ContentType:     result.ContentType,
+		ContentLength:   result.ContentLength,
+		ResponseHeaders: result.Headers,
+		FetchedAt:       time.Now().UTC().Unix(),
+		FetchDurationMs: result.FetchDurationMs,
+		RemoteAddr:      result.RemoteAddr,
+	}
+
+	err := c.srcMetadata.Store(req.Source.Host, pathHash, meta)
+	if err != nil {
+		c.log.Warn("failed to write metadata sidecar",
+			"host", req.Source.Host, "path_hash", pathHash, "error", err)
 	}
 }
 
-// checkNegativeCache checks if a request is in the negative cache. It only
+// touchVariant updates the LRU timestamp of a variant, best-effort:
-// reads: a row past its expiry is not a hit, and the eviction goroutine
+// a failed touch only makes the entry look colder to eviction.
-// deletes it with the other expired rows (see CleanExpired).
+func (c *Cache) touchVariant(ctx context.Context, cacheKey imagerequest.VariantKey) {
+	_, err := c.db.ExecContext(ctx, `
+		UPDATE variant_content SET last_accessed_at = CURRENT_TIMESTAMP
+		WHERE cache_key = ?
+	`, string(cacheKey))
+	if err != nil {
+		c.log.Debug("failed to touch variant LRU timestamp",
+			"cache_key", cacheKey, "error", err)
+	}
+}
+
+// touchSourceContent updates the LRU timestamp of a source content
+// blob, best-effort: a failed touch only makes the blob look colder.
+func (c *Cache) touchSourceContent(ctx context.Context, contentHash ContentHash) {
+	_, err := c.db.ExecContext(ctx, `
+		UPDATE source_content SET last_accessed_at = CURRENT_TIMESTAMP
+		WHERE content_hash = ?
+	`, string(contentHash))
+	if err != nil {
+		c.log.Debug("failed to touch source content LRU timestamp",
+			"content_hash", contentHash, "error", err)
+	}
+}
+
+// checkNegativeCache checks if a request is in the negative cache.
 func (c *Cache) checkNegativeCache(
 	ctx context.Context, req *imagerequest.Request,
 ) (bool, error) {
@@ -409,9 +762,22 @@ func (c *Cache) checkNegativeCache(
 		return false, fmt.Errorf("failed to check negative cache: %w", err)
 	}
 
-	expired := time.Now().After(expiresAt)
+	// Check if expired
+	if time.Now().After(expiresAt) {
+		// Clean up expired entry
+		_, err = c.db.ExecContext(ctx, `
+			DELETE FROM negative_cache
+			WHERE source_host = ? AND source_path = ? AND source_query = ?
+		`, req.Source.Host, req.Source.Path, req.Source.Query)
+		if err != nil {
+			c.log.Warn("failed to delete expired negative cache entry",
+				"host", req.Source.Host, "path", req.Source.Path, "error", err)
+		}
 
-	return !expired, nil
+		return false, nil
+	}
+
+	return true, nil
 }
 
 // cleanExpiredBatch deletes up to cleanExpiredBatchSize negative cache

internal/imgcache/cache_hit_internal_test.go

@@ -1,303 +0,0 @@
-package imgcache
-
-import (
-	"bytes"
-	"errors"
-	"fmt"
-	"image/color"
-	"io"
-	"os"
-	"testing"
-	"time"
-
-	"sneak.berlin/go/pixa/internal/imageprocessor"
-	"sneak.berlin/go/pixa/internal/imagerequest"
-	"sneak.berlin/go/pixa/internal/magic"
-)
-
-// TestGetVariant_ContentTypeFromFirstBytes stores an image in each output
-// format as a variant and checks that GetVariant serves it whole with the
-// content type of its format, read from the file.
-func TestGetVariant_ContentTypeFromFirstBytes(t *testing.T) {
-	t.Parallel()
-
-	cache, _ := newEvictionTestCache(t, 1<<30)
-	processor := imageprocessor.New(imageprocessor.Params{
-		MaxInputBytes:           50 << 20,
-		MaxConcurrentProcessing: 1,
-	})
-	source := generateTestJPEG(t, 40, 30, color.RGBA{200, 40, 40, 255})
-
-	for _, format := range []imagerequest.Format{
-		imagerequest.FormatJPEG, imagerequest.FormatPNG, imagerequest.FormatGIF,
-		imagerequest.FormatWebP, imagerequest.FormatAVIF, imagerequest.FormatJXL,
-	} {
-		result, err := processor.Process(t.Context(), bytes.NewReader(source),
-			&imagerequest.Transform{
-				Size:    imagerequest.Size{Width: 20, Height: 15},
-				Format:  format,
-				Quality: 85,
-				FitMode: imagerequest.FitCover,
-			})
-		if err != nil {
-			t.Fatalf("making a %s image: %v", format, err)
-		}
-
-		image, err := io.ReadAll(result.Content)
-		if err != nil {
-			t.Fatalf("reading the %s image: %v", format, err)
-		}
-
-		key := imagerequest.VariantKey("aabbccdd10" + string(format))
-
-		err = cache.StoreVariant(t.Context(), key, bytes.NewReader(image))
-		if err != nil {
-			t.Fatalf("StoreVariant(%s) error = %v", format, err)
-		}
-
-		reader, size, contentType, err := cache.GetVariant(key)
-		if err != nil {
-			t.Fatalf("GetVariant(%s) error = %v", format, err)
-		}
-
-		served, err := io.ReadAll(reader)
-		_ = reader.Close()
-
-		if err != nil {
-			t.Fatalf("reading the %s variant: %v", format, err)
-		}
-
-		t.Logf("%s: %d bytes served as %s", format, size, contentType)
-
-		if want := imageprocessor.FormatToMIME(format); contentType != want {
-			t.Errorf("%s variant content type = %q, want %q", format, contentType, want)
-		}
-
-		if size != int64(len(image)) || !bytes.Equal(served, image) {
-			t.Errorf("%s variant served %d bytes, size %d, want the %d bytes stored",
-				format, len(served), size, len(image))
-		}
-	}
-}
-
-// TestGetVariant_RefusesFileOfNoOutputFormat checks that GetVariant refuses
-// a variant file whose first bytes are not those of an output format, with
-// an error other than ErrNotFound, and that a variant with no file, or any
-// variant of a disabled cache, is ErrNotFound.
-func TestGetVariant_RefusesFileOfNoOutputFormat(t *testing.T) {
-	t.Parallel()
-
-	cache, _ := newEvictionTestCache(t, 1<<30)
-	jpegImage := generateTestJPEG(t, 10, 10, color.RGBA{0, 0, 255, 255})
-
-	tests := []struct {
-		name    string
-		content []byte
-		wantErr error
-	}{
-		{"text", []byte("this is text, not an image"), magic.ErrUnknownFormat},
-		{"SVG", []byte(`<svg xmlns="http://www.w3.org/2000/svg"></svg>`),
-			errNotAnOutputFormat},
-		{"JPEG cut to 10 bytes", jpegImage[:10], io.EOF},
-		{"empty", nil, io.EOF},
-	}
-
-	for i, tt := range tests {
-		key := imagerequest.VariantKey(fmt.Sprintf("aabbccdd20%02d", i))
-
-		_, err := cache.variants.Store(key, bytes.NewReader(tt.content))
-		if err != nil {
-			t.Fatalf("%s: storing the variant file: %v", tt.name, err)
-		}
-
-		reader, _, contentType, err := cache.GetVariant(key)
-		t.Logf("%s: GetVariant() error = %v", tt.name, err)
-
-		if reader != nil || contentType != "" || !errors.Is(err, tt.wantErr) {
-			t.Errorf("%s: GetVariant() = reader %v, content type %q, error %v; "+
-				"want no reader and %v", tt.name, reader, contentType, err, tt.wantErr)
-		}
-	}
-
-	reader, _, _, err := cache.GetVariant(testVariantKeyFour)
-	if reader != nil || !errors.Is(err, ErrNotFound) {
-		t.Errorf("GetVariant() of a variant with no file = reader %v, error %v; "+
-			"want no reader and %v", reader, err, ErrNotFound)
-	}
-
-	disabled, _ := newEvictionTestCache(t, 0)
-
-	reader, _, _, err = disabled.GetVariant(testVariantKeyOne)
-	if reader != nil || !errors.Is(err, ErrNotFound) {
-		t.Errorf("GetVariant() on a disabled cache = reader %v, error %v; "+
-			"want no reader and %v", reader, err, ErrNotFound)
-	}
-}
-
-// TestService_Get_MakesAgainVariantOfNoOutputFormat replaces a cached
-// variant's file with text, and checks that the next request is a miss
-// answered with the variant made again, and the one after a hit.
-func TestService_Get_MakesAgainVariantOfNoOutputFormat(t *testing.T) {
-	t.Parallel()
-
-	svc, fixtures := SetupTestService(t)
-	req := photoVariant(fixtures, 85, imagerequest.FitCover)
-
-	want, _ := getImage(t, svc, req)
-
-	path := svc.cache.variants.keyToPath(imagerequest.CacheKey(req))
-
-	err := os.WriteFile(path, []byte("this is text, not an image"), StorageFilePerm)
-	if err != nil {
-		t.Fatalf("replacing the variant file: %v", err)
-	}
-
-	got, status := getImage(t, svc, req)
-	t.Logf("after the replacement: %d bytes, %s", len(got), status)
-
-	if status != CacheMiss || !bytes.Equal(got, want) {
-		t.Errorf("after the replacement: Get() = %d bytes, %s; want the %d bytes "+
-			"made again, %s", len(got), status, len(want), CacheMiss)
-	}
-
-	got, status = getImage(t, svc, req)
-
-	if status != CacheHit || !bytes.Equal(got, want) {
-		t.Errorf("next request: Get() = %d bytes, %s; want the %d bytes, %s",
-			len(got), status, len(want), CacheHit)
-	}
-}
-
-// TestService_Get_HitWaitsForNoDatabaseWrite holds the database's one
-// connection while a cached variant is requested. The hit must be answered
-// while the connection is still held, and its access time must reach the
-// database once it is free.
-func TestService_Get_HitWaitsForNoDatabaseWrite(t *testing.T) {
-	t.Parallel()
-
-	svc, fixtures := SetupTestService(t)
-	req := photoVariant(fixtures, 85, imagerequest.FitCover)
-	key := imagerequest.CacheKey(req)
-
-	getImage(t, svc, req)
-
-	// Each connection to an in-memory SQLite database opens a new, empty one,
-	// so the goroutine that writes the access times must share the
-	// connection that holds the schema.
-	svc.cache.db.SetMaxOpenConns(1)
-	svc.cache.StartPendingAccessTimeWrites()
-
-	defer func() { _ = svc.cache.StopPendingAccessTimeWrites(t.Context()) }()
-
-	anHourAgo := time.Now().Add(-time.Hour)
-	setVariantLastAccessed(t, svc.cache, key, anHourAgo)
-
-	releaseDatabase := holdDatabase(t, svc.cache)
-	defer releaseDatabase()
-
-	results := startGet(t.Context(), svc, req)
-
-	select {
-	case got := <-results:
-		if got.err != nil {
-			t.Fatalf("Get() of the cached variant error = %v", got.err)
-		}
-	case <-time.After(pendingAccessTimesWait):
-		t.Fatal("the cache hit waited for the database")
-	}
-
-	releaseDatabase()
-
-	deadline := time.Now().Add(pendingAccessTimesWait)
-
-	for countRows(t, svc.cache, `
-		SELECT COUNT(*) FROM variant_content
-		WHERE cache_key = ? AND last_accessed_at > ?
-	`, string(key), anHourAgo.UTC().Format(sqliteTimestampFormat)) == 0 {
-		if time.Now().After(deadline) {
-			t.Fatal("the hit's access time was not written")
-		}
-
-		time.Sleep(10 * time.Millisecond)
-	}
-}
-
-// TestTouchVariantAndTouchSourceContent_DropAccessTimesPastTheLimit hands one
-// more access time of a variant, and one more of a source, than may wait to
-// be written. touchVariant and touchSourceContent must return each time
-// without waiting, and the last of each must be dropped.
-func TestTouchVariantAndTouchSourceContent_DropAccessTimesPastTheLimit(t *testing.T) {
-	t.Parallel()
-
-	cache, _ := newEvictionTestCache(t, 1<<30)
-
-	done := make(chan struct{})
-
-	go func() {
-		defer close(done)
-
-		for range maxPendingAccessTimes + 1 {
-			cache.touchVariant(testVariantKeyOne)
-			cache.touchSourceContent("aabbccdd3001")
-		}
-	}()
-
-	select {
-	case <-done:
-	case <-time.After(pendingAccessTimesWait):
-		t.Fatal("touchVariant or touchSourceContent waited once the access " +
-			"times reached the limit")
-	}
-
-	if waiting := len(cache.pendingVariantAccessTimes); waiting != maxPendingAccessTimes {
-		t.Errorf("%d access times of variants wait to be written, want %d",
-			waiting, maxPendingAccessTimes)
-	}
-
-	if waiting := len(cache.pendingSourceAccessTimes); waiting != maxPendingAccessTimes {
-		t.Errorf("%d access times of sources wait to be written, want %d",
-			waiting, maxPendingAccessTimes)
-	}
-}
-
-// TestStopPendingAccessTimeWrites_WritesPendingAccessTimes hands the access
-// time of one of two variants, three times, then stops the goroutine that
-// writes them, which never started. The stop must write that access time and
-// leave the other variant's.
-func TestStopPendingAccessTimeWrites_WritesPendingAccessTimes(t *testing.T) {
-	t.Parallel()
-
-	cache, _ := newEvictionTestCache(t, 1<<30)
-	anHourAgo := time.Now().Add(-time.Hour)
-
-	for _, key := range []imagerequest.VariantKey{testVariantKeyOne, testVariantKeyTwo} {
-		storeEvictionTestVariant(t, cache, key, []byte("variant bytes"))
-		setVariantLastAccessed(t, cache, key, anHourAgo)
-	}
-
-	for range 3 {
-		cache.touchVariant(testVariantKeyOne)
-	}
-
-	err := cache.StopPendingAccessTimeWrites(t.Context())
-	if err != nil {
-		t.Fatalf("StopPendingAccessTimeWrites() error = %v", err)
-	}
-
-	for key, wantTouched := range map[imagerequest.VariantKey]int{
-		testVariantKeyOne: 1, testVariantKeyTwo: 0,
-	} {
-		touched := countRows(t, cache, `
-			SELECT COUNT(*) FROM variant_content
-			WHERE cache_key = ? AND last_accessed_at > ?
-		`, string(key), anHourAgo.UTC().Format(sqliteTimestampFormat))
-		if touched != wantTouched {
-			t.Errorf("variant %s: %d rows with a new access time, want %d",
-				key, touched, wantTouched)
-		}
-	}
-
-	if waiting := len(cache.pendingVariantAccessTimes); waiting != 0 {
-		t.Errorf("%d access times still wait to be written, want 0", waiting)
-	}
-}

internal/imgcache/cache_internal_test.go

@@ -1,14 +1,16 @@
 package imgcache
 
 import (
+	"bytes"
 	"context"
 	"database/sql"
 	"os"
 	"testing"
 	"time"
 
-	"github.com/prometheus/client_golang/prometheus"
 	_ "modernc.org/sqlite"
+	"sneak.berlin/go/pixa/internal/httpfetcher"
+	"sneak.berlin/go/pixa/internal/imagerequest"
 )
 
 func setupTestDB(t *testing.T) *sql.DB {
@@ -22,18 +24,67 @@ func setupTestDB(t *testing.T) *sql.DB {
 	// Create schema
 	schema := `
 		CREATE TABLE source_content (
-			source_key TEXT PRIMARY KEY,
+			content_hash TEXT PRIMARY KEY,
+			content_type TEXT NOT NULL,
 			size_bytes INTEGER NOT NULL,
-			last_accessed_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP
+			fetched_at DATETIME DEFAULT CURRENT_TIMESTAMP
+		);
+		CREATE TABLE source_metadata (
+			id INTEGER PRIMARY KEY AUTOINCREMENT,
+			source_host TEXT NOT NULL,
+			source_path TEXT NOT NULL,
+			source_query TEXT NOT NULL DEFAULT '',
+			path_hash TEXT NOT NULL,
+			content_hash TEXT,
+			status_code INTEGER NOT NULL,
+			content_type TEXT,
+			response_headers TEXT,
+			fetched_at DATETIME DEFAULT CURRENT_TIMESTAMP,
+			expires_at DATETIME,
+			etag TEXT,
+			last_modified TEXT,
+			UNIQUE(source_host, source_path, source_query)
+		);
+		CREATE TABLE output_content (
+			content_hash TEXT PRIMARY KEY,
+			content_type TEXT NOT NULL,
+			size_bytes INTEGER NOT NULL,
+			fetched_at DATETIME DEFAULT CURRENT_TIMESTAMP
+		);
+		CREATE TABLE request_cache (
+			id INTEGER PRIMARY KEY AUTOINCREMENT,
+			cache_key TEXT NOT NULL UNIQUE,
+			source_metadata_id INTEGER NOT NULL,
+			output_hash TEXT NOT NULL,
+			width INTEGER NOT NULL,
+			height INTEGER NOT NULL,
+			format TEXT NOT NULL,
+			quality INTEGER NOT NULL DEFAULT 85,
+			fit_mode TEXT NOT NULL DEFAULT 'cover',
+			fetched_at DATETIME DEFAULT CURRENT_TIMESTAMP,
+			access_count INTEGER NOT NULL DEFAULT 1
 		);
 		CREATE TABLE negative_cache (
 			id INTEGER PRIMARY KEY AUTOINCREMENT,
 			source_host TEXT NOT NULL,
 			source_path TEXT NOT NULL,
 			source_query TEXT NOT NULL DEFAULT '',
+			status_code INTEGER NOT NULL,
+			error_message TEXT,
+			fetched_at DATETIME DEFAULT CURRENT_TIMESTAMP,
 			expires_at DATETIME NOT NULL,
 			UNIQUE(source_host, source_path, source_query)
 		);
+		CREATE TABLE cache_stats (
+			id INTEGER PRIMARY KEY CHECK (id = 1),
+			hit_count INTEGER NOT NULL DEFAULT 0,
+			miss_count INTEGER NOT NULL DEFAULT 0,
+			upstream_fetch_count INTEGER NOT NULL DEFAULT 0,
+			upstream_fetch_bytes INTEGER NOT NULL DEFAULT 0,
+			transform_count INTEGER NOT NULL DEFAULT 0,
+			last_updated_at DATETIME DEFAULT CURRENT_TIMESTAMP
+		);
+		INSERT INTO cache_stats (id) VALUES (1);
 	`
 
 	_, err = db.ExecContext(t.Context(), schema)
@@ -44,6 +95,389 @@ func setupTestDB(t *testing.T) *sql.DB {
 	return db
 }
 
+func setupTestCache(t *testing.T) *Cache {
+	t.Helper()
+
+	tmpDir := t.TempDir()
+	db := setupTestDB(t)
+
+	cache, err := NewCache(db, CacheConfig{
+		StateDir:    tmpDir,
+		NegativeTTL: 5 * time.Minute,
+	})
+	if err != nil {
+		t.Fatalf("failed to create cache: %v", err)
+	}
+
+	return cache
+}
+
+func TestCache_LookupMiss(t *testing.T) {
+	t.Parallel()
+
+	cache := setupTestCache(t)
+	ctx := context.Background()
+
+	req := &imagerequest.Request{
+		Source: imagerequest.Source{
+			Host: testHostCDN,
+			Path: testPathCat,
+		},
+		Transform: imagerequest.Transform{
+			Size:    imagerequest.Size{Width: 800, Height: 600},
+			Format:  imagerequest.FormatWebP,
+			Quality: 85,
+			FitMode: imagerequest.FitCover,
+		},
+	}
+
+	result, err := cache.Lookup(ctx, req)
+	if err != nil {
+		t.Fatalf("Lookup() error = %v", err)
+	}
+
+	if result.Hit {
+		t.Error("Lookup() hit = true, want false for cache miss")
+	}
+
+	if result.CacheStatus != CacheMiss {
+		t.Errorf("Lookup() status = %v, want %v", result.CacheStatus, CacheMiss)
+	}
+}
+
+func TestCache_StoreAndLookup(t *testing.T) {
+	t.Parallel()
+
+	cache := setupTestCache(t)
+	ctx := context.Background()
+
+	req := &imagerequest.Request{
+		Source: imagerequest.Source{
+			Host: testHostCDN,
+			Path: testPathCat,
+		},
+		Transform: imagerequest.Transform{
+			Size:    imagerequest.Size{Width: 800, Height: 600},
+			Format:  imagerequest.FormatWebP,
+			Quality: 85,
+			FitMode: imagerequest.FitCover,
+		},
+	}
+
+	// Store source content
+	sourceContent := []byte("fake jpeg data")
+	fetchResult := &httpfetcher.FetchResult{
+		ContentType: testContentTypeJPEG,
+		Headers:     map[string][]string{testHeaderContentType: {testContentTypeJPEG}},
+	}
+
+	contentHash, err := cache.StoreSource(
+		ctx, req, bytes.NewReader(sourceContent), fetchResult)
+	if err != nil {
+		t.Fatalf("StoreSource() error = %v", err)
+	}
+
+	if contentHash == "" {
+		t.Error("StoreSource() returned empty hash")
+	}
+
+	// Store variant
+	cacheKey := imagerequest.CacheKey(req)
+	outputContent := []byte("fake webp data")
+
+	err = cache.StoreVariant(
+		t.Context(), cacheKey, bytes.NewReader(outputContent), "image/webp")
+	if err != nil {
+		t.Fatalf("StoreVariant() error = %v", err)
+	}
+
+	// Now lookup should hit
+	result, err := cache.Lookup(ctx, req)
+	if err != nil {
+		t.Fatalf("Lookup() error = %v", err)
+	}
+
+	if !result.Hit {
+		t.Error("Lookup() hit = false, want true after store")
+	}
+
+	if result.CacheStatus != CacheHit {
+		t.Errorf("Lookup() status = %v, want %v", result.CacheStatus, CacheHit)
+	}
+
+	if result.CacheKey == "" {
+		t.Error("Lookup() returned empty cache key")
+	}
+}
+
+func TestCache_NegativeCache(t *testing.T) {
+	t.Parallel()
+
+	cache := setupTestCache(t)
+	ctx := context.Background()
+
+	req := &imagerequest.Request{
+		Source: imagerequest.Source{
+			Host: testHostCDN,
+			Path: "/photos/notfound.jpg",
+		},
+		Transform: imagerequest.Transform{
+			Size:   imagerequest.Size{Width: 800, Height: 600},
+			Format: imagerequest.FormatWebP,
+		},
+	}
+
+	// Store negative cache entry
+	err := cache.StoreNegative(ctx, req, 404, "not found")
+	if err != nil {
+		t.Fatalf("StoreNegative() error = %v", err)
+	}
+
+	// Lookup should return miss (negative cache is checked at service level)
+	result, err := cache.Lookup(ctx, req)
+	if err != nil {
+		t.Fatalf("Lookup() error = %v", err)
+	}
+
+	if result.Hit {
+		t.Error("Lookup() hit = true, want false for negative cache entry")
+	}
+}
+
+func TestCache_NegativeCacheExpiry(t *testing.T) {
+	t.Parallel()
+
+	tmpDir := t.TempDir()
+	db := setupTestDB(t)
+
+	// Very short negative TTL for testing
+	cache, err := NewCache(db, CacheConfig{
+		StateDir:    tmpDir,
+		NegativeTTL: 1 * time.Millisecond,
+	})
+	if err != nil {
+		t.Fatalf("failed to create cache: %v", err)
+	}
+
+	ctx := context.Background()
+
+	req := &imagerequest.Request{
+		Source: imagerequest.Source{
+			Host: testHostCDN,
+			Path: "/photos/expired.jpg",
+		},
+		Transform: imagerequest.Transform{
+			Size:   imagerequest.Size{Width: 800, Height: 600},
+			Format: imagerequest.FormatWebP,
+		},
+	}
+
+	// Store negative cache entry
+	err = cache.StoreNegative(ctx, req, 404, "not found")
+	if err != nil {
+		t.Fatalf("StoreNegative() error = %v", err)
+	}
+
+	// Wait for expiry
+	time.Sleep(10 * time.Millisecond)
+
+	// Lookup should return miss after expiry
+	result, err := cache.Lookup(ctx, req)
+	if err != nil {
+		t.Fatalf("Lookup() error = %v, want nil after expiry", err)
+	}
+
+	if result.Hit {
+		t.Error("Lookup() hit = true, want false after negative cache expiry")
+	}
+}
+
+func TestCache_VariantLookup(t *testing.T) {
+	t.Parallel()
+
+	cache := setupTestCache(t)
+	ctx := context.Background()
+
+	req := &imagerequest.Request{
+		Source: imagerequest.Source{
+			Host: testHostCDN,
+			Path: "/photos/variant.jpg",
+		},
+		Transform: imagerequest.Transform{
+			Size:    imagerequest.Size{Width: 800, Height: 600},
+			Format:  imagerequest.FormatWebP,
+			Quality: 85,
+			FitMode: imagerequest.FitCover,
+		},
+	}
+
+	// Store variant
+	cacheKey := imagerequest.CacheKey(req)
+	outputContent := []byte("output data")
+
+	err := cache.StoreVariant(
+		t.Context(), cacheKey, bytes.NewReader(outputContent), "image/webp")
+	if err != nil {
+		t.Fatalf("StoreVariant() error = %v", err)
+	}
+
+	// First lookup
+	result1, err := cache.Lookup(ctx, req)
+	if err != nil {
+		t.Fatalf("Lookup() first error = %v", err)
+	}
+
+	if !result1.Hit {
+		t.Error("Lookup() first hit = false")
+	}
+
+	// Second lookup should also hit (from disk)
+	result2, err := cache.Lookup(ctx, req)
+	if err != nil {
+		t.Fatalf("Lookup() second error = %v", err)
+	}
+
+	if !result2.Hit {
+		t.Error("Lookup() second hit = false")
+	}
+
+	if result1.CacheKey != result2.CacheKey {
+		t.Error("Lookup() returned different cache keys")
+	}
+}
+
+func TestCache_GetVariant_ReturnsContentType(t *testing.T) {
+	t.Parallel()
+
+	cache := setupTestCache(t)
+	ctx := context.Background()
+
+	req := &imagerequest.Request{
+		Source: imagerequest.Source{
+			Host: testHostCDN,
+			Path: "/photos/variantct.jpg",
+		},
+		Transform: imagerequest.Transform{
+			Size:    imagerequest.Size{Width: 800, Height: 600},
+			Format:  imagerequest.FormatWebP,
+			Quality: 85,
+			FitMode: imagerequest.FitCover,
+		},
+	}
+
+	// Store variant
+	cacheKey := imagerequest.CacheKey(req)
+	outputContent := []byte("output webp data")
+
+	err := cache.StoreVariant(
+		t.Context(), cacheKey, bytes.NewReader(outputContent), "image/webp")
+	if err != nil {
+		t.Fatalf("StoreVariant() error = %v", err)
+	}
+
+	// Lookup
+	result, err := cache.Lookup(ctx, req)
+	if err != nil {
+		t.Fatalf("Lookup() error = %v", err)
+	}
+
+	if !result.Hit {
+		t.Fatal("Lookup() hit = false, want true")
+	}
+
+	// GetVariant should return content type
+	reader, size, contentType, err := cache.GetVariant(result.CacheKey)
+	if err != nil {
+		t.Fatalf("GetVariant() error = %v", err)
+	}
+
+	defer func() { _ = reader.Close() }()
+
+	if contentType != "image/webp" {
+		t.Errorf("GetVariant() ContentType = %q, want %q", contentType, "image/webp")
+	}
+
+	if size != int64(len(outputContent)) {
+		t.Errorf("GetVariant() size = %d, want %d", size, len(outputContent))
+	}
+}
+
+func TestCache_GetVariant(t *testing.T) {
+	t.Parallel()
+
+	cache := setupTestCache(t)
+	ctx := context.Background()
+
+	req := &imagerequest.Request{
+		Source: imagerequest.Source{
+			Host: testHostCDN,
+			Path: "/photos/output.jpg",
+		},
+		Transform: imagerequest.Transform{
+			Size:    imagerequest.Size{Width: 800, Height: 600},
+			Format:  imagerequest.FormatWebP,
+			Quality: 85,
+			FitMode: imagerequest.FitCover,
+		},
+	}
+
+	// Store variant
+	cacheKey := imagerequest.CacheKey(req)
+	outputContent := []byte("the actual output content")
+
+	err := cache.StoreVariant(
+		t.Context(), cacheKey, bytes.NewReader(outputContent), "image/webp")
+	if err != nil {
+		t.Fatalf("StoreVariant() error = %v", err)
+	}
+
+	// Lookup to get cache key
+	result, err := cache.Lookup(ctx, req)
+	if err != nil {
+		t.Fatalf("Lookup() error = %v", err)
+	}
+
+	// Get variant content
+	reader, _, _, err := cache.GetVariant(result.CacheKey)
+	if err != nil {
+		t.Fatalf("GetVariant() error = %v", err)
+	}
+
+	defer func() { _ = reader.Close() }()
+
+	buf := make([]byte, 100)
+	n, _ := reader.Read(buf)
+
+	if !bytes.Equal(buf[:n], outputContent) {
+		t.Errorf("GetVariant() content = %q, want %q", buf[:n], outputContent)
+	}
+}
+
+func TestCache_Stats(t *testing.T) {
+	t.Parallel()
+
+	cache := setupTestCache(t)
+	ctx := context.Background()
+
+	// Increment some stats
+	cache.IncrementStats(ctx, true, 0)
+	cache.IncrementStats(ctx, true, 0)
+	cache.IncrementStats(ctx, false, 1024)
+
+	stats, err := cache.Stats(ctx)
+	if err != nil {
+		t.Fatalf("Stats() error = %v", err)
+	}
+
+	if stats.HitCount != 2 {
+		t.Errorf("Stats() HitCount = %d, want 2", stats.HitCount)
+	}
+
+	if stats.MissCount != 1 {
+		t.Errorf("Stats() MissCount = %d, want 1", stats.MissCount)
+	}
+}
+
 func TestCache_CleanExpired(t *testing.T) {
 	t.Parallel()
 
@@ -53,16 +487,15 @@ func TestCache_CleanExpired(t *testing.T) {
 	cache, _ := NewCache(db, CacheConfig{
 		StateDir:    tmpDir,
 		NegativeTTL: 1 * time.Millisecond,
-		MaxBytes:    1 << 30,
+	})
-	}, prometheus.NewRegistry())
 
 	ctx := context.Background()
 
 	// Insert expired negative cache entry directly
 	_, err := db.ExecContext(ctx, `
 		INSERT INTO negative_cache
-			(source_host, source_path, source_query, expires_at)
+			(source_host, source_path, source_query, status_code, expires_at)
-		VALUES ('example.com', '/old.jpg', '', datetime('now', '-1 hour'))
+		VALUES ('example.com', '/old.jpg', '', 404, datetime('now', '-1 hour'))
 	`)
 	if err != nil {
 		t.Fatalf("failed to insert test data: %v", err)
@@ -105,8 +538,7 @@ func TestCache_StorageDirectoriesCreated(t *testing.T) {
 
 	_, err := NewCache(db, CacheConfig{
 		StateDir: tmpDir,
-		MaxBytes: 1 << 30,
+	})
-	}, prometheus.NewRegistry())
 	if err != nil {
 		t.Fatalf("NewCache() error = %v", err)
 	}
@@ -115,6 +547,7 @@ func TestCache_StorageDirectoriesCreated(t *testing.T) {
 	dirs := []string{
 		"cache/sources",
 		"cache/variants",
+		"cache/metadata",
 	}
 
 	for _, dir := range dirs {

internal/imgcache/cache_usage_internal_test.go

@@ -31,10 +31,10 @@ func TestEvictionReadsTheUsageTotal(t *testing.T) {
 	storeEvictionTestVariant(t, cache, testVariantKeyTwo,
 		bytes.Repeat([]byte{0x75}, 200))
 
-	// Storing a source URL again changes its size, so the total must
+	// pixa never changes a source's size, but a statement that does must
-	// follow a statement that changes it.
+	// change the total too.
 	_, err := cache.db.ExecContext(ctx,
-		`UPDATE source_content SET size_bytes = 900 WHERE source_key = ?`,
+		`UPDATE source_content SET size_bytes = 900 WHERE content_hash = ?`,
 		string(kept))
 	if err != nil {
 		t.Fatalf("failed to change the source's size: %v", err)
@@ -155,9 +155,8 @@ func TestUsageTotalNotCorrectedFromAnOlderSum(t *testing.T) {
 // pass. No read the pass makes of a content table, to check its rows or
 // to sum them, may cover more than two rows, so a request's query waits
 // for one page at most. Between them the reads must still cover every
-// variant row twice, once to check it and once to sum it, and every
+// row of both tables twice, once to check it and once to sum it, and the
-// source row once, as source rows are read only to sum them, and the sum
+// sum must put a wrong total right.
-// must put a wrong total right.
 func TestReconciliationReadsAPageAtATime(t *testing.T) {
 	t.Parallel()
 
@@ -208,10 +207,9 @@ func TestReconciliationReadsAPageAtATime(t *testing.T) {
 		coveredRows += rows
 	}
 
-	// Five variant rows, each read once to check it and once to sum it,
+	// Ten rows, each read once to check it and once to sum it.
-	// and five source rows, read only to sum them.
+	if coveredRows != 20 {
-	if coveredRows != 15 {
+		t.Errorf("the reads covered %d rows in all, want 20", coveredRows)
-		t.Errorf("the reads covered %d rows in all, want 15", coveredRows)
 	}
 
 	usage, err := cache.UsageBytes(ctx)

internal/imgcache/cachesize.go

@@ -0,0 +1,85 @@
+package imgcache
+
+import (
+	"context"
+	"errors"
+	"fmt"
+	"math"
+	"path/filepath"
+	"syscall"
+)
+
+// DefaultCacheMaxBytesFloor is the minimum computed default for the
+// cache_max_bytes setting: 500 MiB. The floor applies only to the
+// computed default (when the key is omitted from the configuration),
+// never to explicitly configured values.
+const DefaultCacheMaxBytesFloor int64 = 524288000
+
+// freeSpaceFractionNumerator and freeSpaceFractionDenominator express
+// the 75% share used for the computed default limit as integer
+// arithmetic (dividing before multiplying avoids overflow).
+const (
+	freeSpaceFractionNumerator   uint64 = 3
+	freeSpaceFractionDenominator uint64 = 4
+)
+
+var errNegativeBlockSize = errors.New("statfs reported negative block size")
+
+// FreeSpaceProbeFunc reports the number of free bytes available on the
+// filesystem containing path. It is a function type so tests can
+// inject a fake probe instead of depending on the host disk.
+type FreeSpaceProbeFunc func(path string) (uint64, error)
+
+// defaultFreeSpaceProbe reports free filesystem bytes via statfs on
+// the given path, as available to unprivileged processes.
+func defaultFreeSpaceProbe(path string) (uint64, error) {
+	var stat syscall.Statfs_t
+
+	err := syscall.Statfs(path, &stat)
+	if err != nil {
+		return 0, err
+	}
+
+	if stat.Bsize < 0 {
+		return 0, fmt.Errorf("%w %d for %q", errNegativeBlockSize, stat.Bsize, path)
+	}
+
+	blockSize := uint64(stat.Bsize)
+
+	return stat.Bavail * blockSize, nil
+}
+
+// computeDefaultMaxBytes returns the default cache size limit: 75% of
+// the sum of the free bytes probe reports for <state_dir>/cache/ and
+// the bytes the cache already holds, with a floor of
+// DefaultCacheMaxBytesFloor. Counting what the cache holds keeps the
+// limit from shrinking as the cache fills.
+func (c *Cache) computeDefaultMaxBytes(
+	ctx context.Context, probe FreeSpaceProbeFunc,
+) (int64, error) {
+	cacheDir := filepath.Join(c.config.StateDir, "cache")
+
+	freeBytes, err := probe(cacheDir)
+	if err != nil {
+		return 0, fmt.Errorf(
+			"default cache_max_bytes: cannot determine free space for %q: %w",
+			cacheDir, err)
+	}
+
+	usedBytes, err := c.UsageBytes(ctx)
+	if err != nil {
+		return 0, err
+	}
+
+	// Both terms are at most math.MaxInt64, so the sum cannot overflow.
+	//nolint:gosec // G115: UsageBytes returns the total cache usage, never negative
+	spaceBytes := min(freeBytes, math.MaxInt64) + uint64(usedBytes)
+
+	computed := spaceBytes / freeSpaceFractionDenominator * freeSpaceFractionNumerator
+	computed = min(computed, math.MaxInt64)
+
+	limit := int64(computed)
+	limit = max(limit, DefaultCacheMaxBytesFloor)
+
+	return limit, nil
+}

internal/imgcache/cachesize_internal_test.go

@@ -0,0 +1,188 @@
+package imgcache
+
+import (
+	"errors"
+	"os"
+	"path/filepath"
+	"strings"
+	"testing"
+)
+
+// Static errors returned by the stub free-space probes below.
+var (
+	errTestStatfsFailed     = errors.New("statfs failed")
+	errTestProbeNotExpected = errors.New("probe must not be called")
+)
+
+// TestComputeDefaultMaxBytesCountsWhatTheCacheHolds verifies that the
+// default limit is 75% of the free space plus what the cache already
+// holds, so a cache filled to its limit keeps that limit across a
+// restart instead of shrinking to 75% of the space left free.
+func TestComputeDefaultMaxBytesCountsWhatTheCacheHolds(t *testing.T) {
+	t.Parallel()
+
+	cache, _ := newEvictionTestCache(t, 1<<30)
+
+	// Empty cache, 4 GiB free -> 3 GiB default.
+	got, err := cache.computeDefaultMaxBytes(t.Context(),
+		func(string) (uint64, error) { return 4294967296, nil })
+	if err != nil {
+		t.Fatalf("computeDefaultMaxBytes returned error: %v", err)
+	}
+
+	t.Logf("default for an empty cache with 4 GiB free: %d", got)
+
+	if got != 3221225472 {
+		t.Errorf("default for an empty cache = %d, want 3221225472 (75%% of 4 GiB)",
+			got)
+	}
+
+	// The cache now holds those 3 GiB, which leaves 1 GiB free.
+	_, err = cache.db.ExecContext(t.Context(),
+		`INSERT INTO variant_content (cache_key, size_bytes, content_type)
+		 VALUES (?, ?, ?)`,
+		string(testVariantKeyOne), 3221225472, testContentTypeWebP,
+	)
+	if err != nil {
+		t.Fatalf("failed to insert variant accounting row: %v", err)
+	}
+
+	got, err = cache.computeDefaultMaxBytes(t.Context(),
+		func(string) (uint64, error) { return 1073741824, nil })
+	if err != nil {
+		t.Fatalf("computeDefaultMaxBytes returned error: %v", err)
+	}
+
+	t.Logf("default for a cache holding 3 GiB with 1 GiB free: %d", got)
+
+	if got != 3221225472 {
+		t.Errorf("default for a cache holding 3 GiB with 1 GiB free = %d, "+
+			"want 3221225472 (75%% of 1 GiB + 3 GiB)", got)
+	}
+}
+
+// TestComputeDefaultMaxBytesAppliesFloor verifies that when 75% of the
+// free space plus what the cache holds is below 500 MiB, the default
+// is floored at DefaultCacheMaxBytesFloor.
+func TestComputeDefaultMaxBytesAppliesFloor(t *testing.T) {
+	t.Parallel()
+
+	cases := []struct {
+		name      string
+		freeBytes uint64
+	}{
+		{name: "100 MiB free", freeBytes: 104857600},
+		{name: "zero free", freeBytes: 0},
+		{name: "just below floor threshold", freeBytes: 699050665},
+	}
+
+	for _, tc := range cases {
+		t.Run(tc.name, func(t *testing.T) {
+			t.Parallel()
+
+			cache, _ := newEvictionTestCache(t, 1<<30)
+
+			got, err := cache.computeDefaultMaxBytes(t.Context(),
+				func(string) (uint64, error) { return tc.freeBytes, nil })
+			if err != nil {
+				t.Fatalf("computeDefaultMaxBytes returned error: %v", err)
+			}
+
+			if got != DefaultCacheMaxBytesFloor {
+				t.Errorf("computeDefaultMaxBytes = %d, want floor %d",
+					got, DefaultCacheMaxBytesFloor)
+			}
+		})
+	}
+}
+
+// TestComputeDefaultMaxBytesPropagatesProbeError verifies that a
+// failing free-space probe produces an error naming cache_max_bytes,
+// instead of a silently wrong default.
+func TestComputeDefaultMaxBytesPropagatesProbeError(t *testing.T) {
+	t.Parallel()
+
+	cache, _ := newEvictionTestCache(t, 1<<30)
+
+	_, err := cache.computeDefaultMaxBytes(t.Context(),
+		func(string) (uint64, error) { return 0, errTestStatfsFailed })
+	if err == nil {
+		t.Fatal("probe failure must produce an error, got nil")
+	}
+
+	t.Logf("got expected error: %v", err)
+
+	if !strings.Contains(err.Error(), "cache_max_bytes") {
+		t.Errorf("error %q does not name cache_max_bytes", err.Error())
+	}
+}
+
+// TestNewCacheComputesDefaultMaxBytesWhenAsked verifies that with
+// UseDefaultMaxBytes set, the cache's limit becomes the computed
+// default, and that the probe is pointed at <state_dir>/cache/, which
+// must be created first so statfs measures the right filesystem.
+func TestNewCacheComputesDefaultMaxBytesWhenAsked(t *testing.T) {
+	t.Parallel()
+
+	stateDir := t.TempDir()
+	wantCacheDir := filepath.Join(stateDir, "cache")
+
+	var probedPath string
+
+	// 4 GiB free -> 3 GiB default.
+	probe := func(path string) (uint64, error) {
+		probedPath = path
+
+		info, err := os.Stat(path)
+		if err != nil || !info.IsDir() {
+			t.Errorf("cache directory %q was not created before probing: info=%v err=%v",
+				path, info, err)
+		}
+
+		return 4294967296, nil
+	}
+
+	cache, err := newCache(evictionTestDB(t), CacheConfig{
+		StateDir:           stateDir,
+		UseDefaultMaxBytes: true,
+	}, probe)
+	if err != nil {
+		t.Fatalf("newCache returned error: %v", err)
+	}
+
+	if cache.config.MaxBytes != 3221225472 {
+		t.Errorf("MaxBytes = %d, want computed default 3221225472",
+			cache.config.MaxBytes)
+	}
+
+	if probedPath != wantCacheDir {
+		t.Errorf("free space probed at %q, want cache directory %q",
+			probedPath, wantCacheDir)
+	}
+}
+
+// TestNewCacheKeepsExplicitMaxBytes verifies that without
+// UseDefaultMaxBytes the cache keeps MaxBytes exactly as given and
+// never consults the free-space probe.
+func TestNewCacheKeepsExplicitMaxBytes(t *testing.T) {
+	t.Parallel()
+
+	probe := func(string) (uint64, error) {
+		t.Error("free-space probe must not be consulted for explicit values")
+
+		return 0, errTestProbeNotExpected
+	}
+
+	cache, err := newCache(evictionTestDB(t), CacheConfig{
+		StateDir: t.TempDir(),
+		MaxBytes: 1024,
+	}, probe)
+	if err != nil {
+		t.Fatalf("newCache returned error: %v", err)
+	}
+
+	if cache.config.MaxBytes != 1024 {
+		t.Errorf("MaxBytes = %d, want explicit 1024 (no floor, no recompute)",
+			cache.config.MaxBytes)
+	}
+}

internal/imgcache/concurrent_misses_internal_test.go

@@ -36,7 +36,7 @@ type heldFetcher struct {
 }
 
 func (f *heldFetcher) Fetch(
-	ctx context.Context, source imagerequest.Source,
+	ctx context.Context, url string,
 ) (*httpfetcher.FetchResult, error) {
 	f.fetches.Add(1)
 
@@ -48,7 +48,7 @@ func (f *heldFetcher) Fetch(
 		return nil, ctx.Err()
 	}
 
-	return f.upstream.Fetch(ctx, source)
+	return f.upstream.Fetch(ctx, url)
 }
 
 // releaseFetches lets every held fetch, and every later one, go on.
@@ -79,8 +79,8 @@ func setupHeldFetchService(t *testing.T) (*Service, *TestFixtures, *heldFetcher)
 }
 
 // photoVariant asks for the test photo, 100x100, at 50x25 as a JPEG of the
-// given quality and fit mode. Each call returns a new request, which a test
+// given quality and fit mode. Each call returns a new request, as Get writes
-// may change.
+// to the request it is given.
 func photoVariant(
 	fixtures *TestFixtures, quality int, fit imagerequest.Fit,
 ) *imagerequest.Request {
@@ -191,9 +191,9 @@ func TestService_Get_ConcurrentMissesShareOneFetch(t *testing.T) {
 		t.Fatal(err)
 	}
 
-	want := counterValues{0, requests, 1, int64(len(photo)), 1}
+	want := cacheStatsCounters{0, requests, 1, int64(len(photo)), 1}
 
-	if got := readCounters(svc.cache); got != want {
+	if got := readCacheStatsCounters(t, svc.cache); got != want {
 		t.Errorf("counters = %+v, want %+v", got, want)
 	}
 }
@@ -455,7 +455,7 @@ func TestService_Get_WaiterGetsProcessingTimedOut(t *testing.T) {
 
 	svc, fixtures, fetcher := setupHeldFetchService(t)
 	svc.processor = imageprocessor.New(
-		imageprocessor.Params{MaxInputBytes: 50 << 20, MaxConcurrentProcessing: 1},
+		imageprocessor.Params{MaxConcurrentProcessing: 1},
 	)
 
 	holdProcessingSlot(t, svc.processor)
@@ -489,7 +489,7 @@ func TestService_Get_WaiterGetsProcessingTimedOut(t *testing.T) {
 type panickingFetcher struct{}
 
 func (panickingFetcher) Fetch(
-	context.Context, imagerequest.Source,
+	context.Context, string,
 ) (*httpfetcher.FetchResult, error) {
 	panic("upstream fetcher panicked")
 }

internal/imgcache/contentlock.go

@@ -0,0 +1,64 @@
+package imgcache
+
+import "sync"
+
+// contentLock provides per-key mutual exclusion for content-hash keyed
+// operations. StoreSource and evictSourceBlob each hold a content
+// hash's lock for the full duration of their file-plus-accounting-row
+// work, so a store and an eviction racing on identical content bytes
+// (the real SHA-256 content-addressed dedup case, not a contrived one)
+// can never interleave: the unlink of an evicted blob's file can never
+// race the creation of a fresh database row for a concurrently
+// re-stored copy of the same content. Entries are removed once no
+// goroutine holds or is waiting for them, so a long-running process
+// does not accumulate memory proportional to the number of distinct
+// content hashes it has ever seen.
+type contentLock struct {
+	mu      sync.Mutex
+	entries map[string]*contentLockEntry
+}
+
+// contentLockEntry is one key's exclusion lock plus a count of
+// goroutines currently holding or waiting to acquire it, used to know
+// when it is safe to remove the entry from the map.
+type contentLockEntry struct {
+	mu    sync.Mutex
+	count int
+}
+
+// newContentLock creates an empty contentLock.
+func newContentLock() *contentLock {
+	return &contentLock{entries: make(map[string]*contentLockEntry)}
+}
+
+// Lock acquires exclusive access for key, blocking until it is
+// available, and returns a function that releases it. The caller must
+// invoke the returned function exactly once to release the lock.
+func (c *contentLock) Lock(key string) func() {
+	c.mu.Lock()
+
+	entry, ok := c.entries[key]
+	if !ok {
+		entry = &contentLockEntry{}
+		c.entries[key] = entry
+	}
+
+	entry.count++
+
+	c.mu.Unlock()
+
+	entry.mu.Lock()
+
+	return func() {
+		entry.mu.Unlock()
+
+		c.mu.Lock()
+
+		entry.count--
+		if entry.count == 0 {
+			delete(c.entries, key)
+		}
+
+		c.mu.Unlock()
+	}
+}

internal/imgcache/contentlock_internal_test.go

@@ -0,0 +1,142 @@
+package imgcache
+
+import (
+	"sync"
+	"sync/atomic"
+	"testing"
+	"time"
+)
+
+// TestContentLockExcludesSameKey verifies that two goroutines locking
+// the same key never run their critical sections concurrently.
+func TestContentLockExcludesSameKey(t *testing.T) {
+	t.Parallel()
+
+	lock := newContentLock()
+
+	var (
+		active  atomic.Int32
+		maxSeen int32
+		wg      sync.WaitGroup
+	)
+
+	const goroutines = 20
+
+	wg.Add(goroutines)
+
+	for range goroutines {
+		go func() {
+			defer wg.Done()
+
+			unlock := lock.Lock("same-key")
+			defer unlock()
+
+			n := active.Add(1)
+
+			for {
+				seen := atomic.LoadInt32(&maxSeen)
+				if n <= seen || atomic.CompareAndSwapInt32(&maxSeen, seen, n) {
+					break
+				}
+			}
+
+			time.Sleep(time.Millisecond)
+
+			active.Add(-1)
+		}()
+	}
+
+	wg.Wait()
+
+	if maxSeen != 1 {
+		t.Errorf("max concurrent holders of the same key = %d, want 1", maxSeen)
+	}
+}
+
+// TestContentLockAllowsDifferentKeys verifies that locking distinct
+// keys does not serialize unrelated work: all goroutines must be able
+// to enter their critical sections at once, proven by every one of
+// them reaching the rendezvous point before any is allowed to
+// proceed.
+func TestContentLockAllowsDifferentKeys(t *testing.T) {
+	t.Parallel()
+
+	lock := newContentLock()
+
+	const goroutines = 20
+
+	var (
+		wg      sync.WaitGroup
+		inside  atomic.Int32
+		reached = make(chan struct{}, goroutines)
+	)
+
+	wg.Add(goroutines)
+
+	release := make(chan struct{})
+
+	for i := range goroutines {
+		key := string(rune('a' + i))
+
+		go func() {
+			defer wg.Done()
+
+			unlock := lock.Lock(key)
+			defer unlock()
+
+			inside.Add(1)
+
+			reached <- struct{}{}
+
+			<-release
+		}()
+	}
+
+	// Every goroutine must reach the rendezvous point (i.e. acquire its
+	// own key's lock) without needing any other to release first. If
+	// keys were incorrectly serialized onto one underlying lock, only
+	// one would get here and this would time out.
+	for i := range goroutines {
+		select {
+		case <-reached:
+		case <-time.After(2 * time.Second):
+			t.Fatalf("only %d/%d goroutines locking distinct keys made progress; "+
+				"keys may be incorrectly serialized", i, goroutines)
+		}
+	}
+
+	if n := inside.Load(); n != goroutines {
+		t.Errorf("goroutines inside their critical section = %d, want %d",
+			n, goroutines)
+	}
+
+	close(release)
+	wg.Wait()
+}
+
+// TestContentLockRemovesEntryAfterUnlock verifies that the internal
+// entries map does not grow without bound: once no goroutine holds or
+// awaits a key, its entry is removed.
+func TestContentLockRemovesEntryAfterUnlock(t *testing.T) {
+	t.Parallel()
+
+	lock := newContentLock()
+
+	unlock := lock.Lock("k")
+
+	lock.mu.Lock()
+	if _, ok := lock.entries["k"]; !ok {
+		lock.mu.Unlock()
+		t.Fatal("entry missing while lock is held")
+	}
+	lock.mu.Unlock()
+
+	unlock()
+
+	lock.mu.Lock()
+	defer lock.mu.Unlock()
+
+	if _, ok := lock.entries["k"]; ok {
+		t.Error("entry for key still present after the last holder unlocked")
+	}
+}

internal/imgcache/counters.go

@@ -1,45 +0,0 @@
-package imgcache
-
-import (
-	"fmt"
-
-	"github.com/prometheus/client_golang/prometheus"
-)
-
-// registerCounters makes the cache's counters and registers them on
-// metricsRegistry. The counters are kept in memory, so adding to one waits
-// for nothing and makes no database call. It fails when metricsRegistry
-// already holds counters of the same names.
-func (c *Cache) registerCounters(metricsRegistry prometheus.Registerer) error {
-	c.cacheHits = prometheus.NewCounter(prometheus.CounterOpts{
-		Name: "pixa_cache_hits_total",
-		Help: "Image requests served from the cache.",
-	})
-	c.cacheMisses = prometheus.NewCounter(prometheus.CounterOpts{
-		Name: "pixa_cache_misses_total",
-		Help: "Image requests not served from the cache.",
-	})
-	c.upstreamFetches = prometheus.NewCounter(prometheus.CounterOpts{
-		Name: "pixa_upstream_fetches_total",
-		Help: "Fetches of a source image from upstream that read any bytes.",
-	})
-	c.upstreamFetchBytes = prometheus.NewCounter(prometheus.CounterOpts{
-		Name: "pixa_upstream_fetch_bytes_total",
-		Help: "Bytes read from upstream by fetches of source images.",
-	})
-	c.transforms = prometheus.NewCounter(prometheus.CounterOpts{
-		Name: "pixa_transforms_total",
-		Help: "Images the image processor transformed.",
-	})
-
-	for _, counter := range []prometheus.Counter{
-		c.cacheHits, c.cacheMisses, c.upstreamFetches, c.upstreamFetchBytes, c.transforms,
-	} {
-		err := metricsRegistry.Register(counter)
-		if err != nil {
-			return fmt.Errorf("failed to register the cache's counters: %w", err)
-		}
-	}
-
-	return nil
-}

internal/imgcache/counters_internal_test.go

@@ -1,87 +0,0 @@
-package imgcache
-
-import (
-	"errors"
-	"testing"
-
-	"github.com/prometheus/client_golang/prometheus"
-	"github.com/prometheus/client_golang/prometheus/testutil"
-)
-
-// counterValues holds the values of a cache's counters.
-type counterValues struct {
-	hitCount           int64
-	missCount          int64
-	upstreamFetchCount int64
-	upstreamFetchBytes int64
-	transformCount     int64
-}
-
-// readCounters reads cache's counters.
-func readCounters(cache *Cache) counterValues {
-	return counterValues{
-		hitCount:           int64(testutil.ToFloat64(cache.cacheHits)),
-		missCount:          int64(testutil.ToFloat64(cache.cacheMisses)),
-		upstreamFetchCount: int64(testutil.ToFloat64(cache.upstreamFetches)),
-		upstreamFetchBytes: int64(testutil.ToFloat64(cache.upstreamFetchBytes)),
-		transformCount:     int64(testutil.ToFloat64(cache.transforms)),
-	}
-}
-
-// TestNewCache_RegistersItsCountersOnItsRegistry checks that NewCache
-// registers the cache's counters on the registry it is given, under the
-// names README.md gives them.
-func TestNewCache_RegistersItsCountersOnItsRegistry(t *testing.T) {
-	t.Parallel()
-
-	registry := prometheus.NewRegistry()
-
-	_, err := NewCache(evictionTestDB(t),
-		CacheConfig{StateDir: t.TempDir(), MaxBytes: 1 << 30}, registry)
-	if err != nil {
-		t.Fatalf("NewCache() error = %v", err)
-	}
-
-	names := []string{
-		"pixa_cache_hits_total",
-		"pixa_cache_misses_total",
-		"pixa_upstream_fetches_total",
-		"pixa_upstream_fetch_bytes_total",
-		"pixa_transforms_total",
-	}
-
-	count, err := testutil.GatherAndCount(registry, names...)
-	if err != nil {
-		t.Fatalf("testutil.GatherAndCount() error = %v", err)
-	}
-
-	if count != len(names) {
-		t.Errorf("%d of the counters %v are on the registry, want %d",
-			count, names, len(names))
-	}
-}
-
-// TestNewCache_FailsWhenItsCountersCannotBeRegistered makes a second cache
-// on the registry of a first. Its counters have the same names, so the
-// registry refuses them, and NewCache must return that error.
-func TestNewCache_FailsWhenItsCountersCannotBeRegistered(t *testing.T) {
-	t.Parallel()
-
-	registry := prometheus.NewRegistry()
-
-	_, err := NewCache(evictionTestDB(t),
-		CacheConfig{StateDir: t.TempDir(), MaxBytes: 1 << 30}, registry)
-	if err != nil {
-		t.Fatalf("NewCache() of the first cache error = %v", err)
-	}
-
-	_, err = NewCache(evictionTestDB(t),
-		CacheConfig{StateDir: t.TempDir(), MaxBytes: 1 << 30}, registry)
-	t.Logf("NewCache() of the second cache error = %v", err)
-
-	var alreadyRegistered prometheus.AlreadyRegisteredError
-	if !errors.As(err, &alreadyRegistered) {
-		t.Errorf("NewCache() of the second cache error = %v, want %T",
-			err, alreadyRegistered)
-	}
-}

internal/imgcache/eviction.go

@@ -3,6 +3,7 @@ package imgcache
 import (
 	"context"
 	"database/sql"
+	"encoding/json"
 	"errors"
 	"fmt"
 	"io/fs"
@@ -51,26 +52,27 @@ const sqliteTimestampLayout = "2006-01-02 15:04:05"
 // writes (".tmp-*" patterns in the storage layer).
 const tempFilePrefix = ".tmp-"
 
+// variantMetaSuffix is the sidecar suffix VariantStorage writes next
+// to each variant file.
+const variantMetaSuffix = ".meta"
+
+// fallbackContentType is the content type given to a variant file that
+// has no readable .meta sidecar, when it is served or reconciled.
+const fallbackContentType = "application/octet-stream"
+
 // UsageBytes returns the total number of bytes of cache content
-// tracked in the database, as the package function UsageBytes does, or 0
+// tracked in the database (source content blobs plus processed
-// when the disk cache is off.
+// variants). It reads the total the database keeps up to date as rows
+// are added and removed, so it neither scans the cache directories nor
+// sums the tables.
 func (c *Cache) UsageBytes(ctx context.Context) (int64, error) {
-	if c.config.MaxBytes == 0 {
+	if c.disabled {
 		return 0, nil
 	}
 
-	return UsageBytes(ctx, c.db)
-}
-
-// UsageBytes returns the total number of bytes of cache content tracked
-// in db (source content blobs plus processed variants). It reads the
-// total the database keeps up to date as rows are added and removed, so
-// it neither scans the cache directories nor sums the tables. pixad reads
-// it before it builds the cache, to work out the default cache_max_bytes.
-func UsageBytes(ctx context.Context, db *sql.DB) (int64, error) {
 	var total int64
 
-	err := db.QueryRowContext(ctx,
+	err := c.db.QueryRowContext(ctx,
 		`SELECT total_size_bytes FROM cache_usage WHERE id = 1`,
 	).Scan(&total)
 	if err != nil {
@@ -82,20 +84,20 @@ func UsageBytes(ctx context.Context, db *sql.DB) (int64, error) {
 
 // evictionCandidate is one LRU eviction victim candidate: either a
 // processed variant (isVariant true, identified by cacheKey) or a
-// source content blob (identified by sourceKey).
+// source content blob (identified by contentHash).
 type evictionCandidate struct {
 	isVariant      bool
 	cacheKey       imagerequest.VariantKey
-	sourceKey      imagerequest.SourceKey
+	contentHash    ContentHash
 	sizeBytes      int64
 	lastAccessedAt string
 }
 
 // EvictToLimit evicts least-recently-used cache entries until total
 // tracked usage is at or below the configured MaxBytes limit. It is a
-// no-op when the cache is disabled.
+// no-op when the cache is disabled or no limit is configured.
 func (c *Cache) EvictToLimit(ctx context.Context) error {
-	if c.config.MaxBytes == 0 {
+	if c.disabled || c.config.MaxBytes <= 0 {
 		return nil
 	}
 
@@ -158,7 +160,7 @@ func (c *Cache) evictBatch(ctx context.Context, excessBytes int64) (int64, error
 
 			c.log.Warn("failed to evict cache entry",
 				"cache_key", candidate.cacheKey,
-				"source_key", candidate.sourceKey,
+				"content_hash", candidate.contentHash,
 				"error", err,
 			)
 
@@ -177,7 +179,7 @@ func (c *Cache) evictCandidate(ctx context.Context, candidate evictionCandidate)
 		return c.evictVariant(ctx, candidate.cacheKey)
 	}
 
-	return c.evictSourceBlob(ctx, candidate.sourceKey)
+	return c.evictSourceBlob(ctx, candidate.contentHash)
 }
 
 // evictionCandidates returns up to evictionBatchSize variants and
@@ -257,9 +259,9 @@ func (c *Cache) variantCandidates(ctx context.Context) ([]evictionCandidate, err
 // SQLite reads the rows in order from that column's index instead of
 // sorting the whole table.
 const sourceCandidatesQuery = `
-	SELECT source_key, size_bytes, last_accessed_at
+	SELECT content_hash, size_bytes, last_accessed_at
 	FROM source_content
-	ORDER BY last_accessed_at ASC, source_key ASC
+	ORDER BY last_accessed_at ASC, content_hash ASC
 	LIMIT ?`
 
 // sourceCandidates returns the least recently used source blobs.
@@ -276,14 +278,14 @@ func (c *Cache) sourceCandidates(ctx context.Context) ([]evictionCandidate, erro
 	for rows.Next() {
 		var candidate evictionCandidate
 
-		var key string
+		var hash string
 
-		err := rows.Scan(&key, &candidate.sizeBytes, &candidate.lastAccessedAt)
+		err := rows.Scan(&hash, &candidate.sizeBytes, &candidate.lastAccessedAt)
 		if err != nil {
 			return nil, fmt.Errorf("failed to scan source candidate: %w", err)
 		}
 
-		candidate.sourceKey = imagerequest.SourceKey(key)
+		candidate.contentHash = ContentHash(hash)
 		candidates = append(candidates, candidate)
 	}
 
@@ -296,7 +298,10 @@ func (c *Cache) sourceCandidates(ctx context.Context) ([]evictionCandidate, erro
 }
 
 // evictVariant removes one variant: accounting row first, then the
-// file, so the database never references a deleted file.
+// content and .meta files, so the database never references a deleted
+// file. The metaCache entry goes before the files; a GetVariant that
+// read them just before may put it back, and the next GetVariant then
+// fails to open the file and removes it again.
 func (c *Cache) evictVariant(
 	ctx context.Context, cacheKey imagerequest.VariantKey,
 ) error {
@@ -306,6 +311,8 @@ func (c *Cache) evictVariant(
 		return fmt.Errorf("failed to delete variant accounting row: %w", err)
 	}
 
+	c.metaCache.Remove(cacheKey)
+
 	err = c.variants.Delete(cacheKey)
 	if err != nil {
 		return err
@@ -314,18 +321,117 @@ func (c *Cache) evictVariant(
 	return nil
 }
 
-// evictSourceBlob removes one source: its row first, then its file, as
+// sourceReference identifies one source_metadata row's JSON sidecar.
-// evictVariant does for a variant. It holds no lock: a store of the same
+type sourceReference struct {
-// source in between can leave the source's row with no file, which
+	host     string
-// LookupSource treats as a miss.
+	pathHash PathHash
-func (c *Cache) evictSourceBlob(ctx context.Context, key imagerequest.SourceKey) error {
+}
-	_, err := c.db.ExecContext(ctx,
+
-		`DELETE FROM source_content WHERE source_key = ?`, string(key))
+// evictSourceBlob removes one source content blob. All source_metadata
+// rows referencing the blob are deleted together with its
+// source_content row in a single transaction BEFORE the file is
+// unlinked: a blob referenced by multiple source paths is only ever
+// removed together with all of its references, and database rows never
+// point at deleted files. The JSON metadata sidecars for the removed
+// rows are deleted afterwards.
+//
+// The whole operation holds the content hash's lock (the same one
+// StoreSource holds for its full store), so a concurrent store of
+// identical content bytes can never observe the file gone but a row
+// still present, or insert a fresh row between this transaction's
+// commit and the file unlink below: it either runs entirely before
+// this eviction starts, or is blocked until this eviction (row
+// deletion and unlink together) has fully completed.
+func (c *Cache) evictSourceBlob(ctx context.Context, contentHash ContentHash) error {
+	unlock := c.contentLocks.Lock(string(contentHash))
+	defer unlock()
+
+	references, err := c.sourceReferences(ctx, contentHash)
+	if err != nil {
+		return err
+	}
+
+	tx, err := c.db.BeginTx(ctx, nil)
+	if err != nil {
+		return fmt.Errorf("failed to begin eviction transaction: %w", err)
+	}
+
+	defer func() { _ = tx.Rollback() }()
+
+	_, err = tx.ExecContext(ctx,
+		`DELETE FROM source_metadata WHERE content_hash = ?`, string(contentHash))
+	if err != nil {
+		return fmt.Errorf("failed to delete source metadata rows: %w", err)
+	}
+
+	_, err = tx.ExecContext(ctx,
+		`DELETE FROM source_content WHERE content_hash = ?`, string(contentHash))
 	if err != nil {
 		return fmt.Errorf("failed to delete source content row: %w", err)
 	}
 
-	return c.srcContent.Delete(key)
+	err = tx.Commit()
+	if err != nil {
+		return fmt.Errorf("failed to commit eviction transaction: %w", err)
+	}
+
+	if c.evictSourceBlobTestHook != nil {
+		c.evictSourceBlobTestHook(contentHash)
+	}
+
+	// Only after the rows are gone may the files be removed.
+	for _, reference := range references {
+		err := c.srcMetadata.Delete(reference.host, reference.pathHash)
+		if err != nil {
+			c.log.Warn("failed to delete metadata sidecar",
+				"host", reference.host, "path_hash", reference.pathHash, "error", err)
+		}
+	}
+
+	err = c.srcContent.Delete(contentHash)
+	if err != nil {
+		return err
+	}
+
+	return nil
+}
+
+// sourceReferences lists the metadata sidecar locations of every
+// source_metadata row referencing the given blob.
+func (c *Cache) sourceReferences(
+	ctx context.Context, contentHash ContentHash,
+) ([]sourceReference, error) {
+	rows, err := c.db.QueryContext(ctx, `
+		SELECT source_host, path_hash FROM source_metadata WHERE content_hash = ?
+	`, string(contentHash))
+	if err != nil {
+		return nil, fmt.Errorf("failed to query source references: %w", err)
+	}
+
+	defer func() { _ = rows.Close() }()
+
+	var references []sourceReference
+
+	for rows.Next() {
+		var reference sourceReference
+
+		var pathHash string
+
+		err := rows.Scan(&reference.host, &pathHash)
+		if err != nil {
+			return nil, fmt.Errorf("failed to scan source reference: %w", err)
+		}
+
+		reference.pathHash = PathHash(pathHash)
+		references = append(references, reference)
+	}
+
+	err = rows.Err()
+	if err != nil {
+		return nil, fmt.Errorf("source reference iteration failed: %w", err)
+	}
+
+	return references, nil
 }
 
 // notifyWritePressure wakes the background evictor after a store, so
@@ -333,7 +439,7 @@ func (c *Cache) evictSourceBlob(ctx context.Context, key imagerequest.SourceKey)
 // storing request. The notification channel has capacity one and drops
 // when a wakeup is already pending.
 func (c *Cache) notifyWritePressure() {
-	if c.config.MaxBytes == 0 {
+	if c.disabled || c.config.MaxBytes <= 0 {
 		return
 	}
 
@@ -480,19 +586,19 @@ func (c *Cache) runCleanExpired(ctx context.Context) {
 // as long after the last pass ended as that pass took, off the request
 // hot path: it adopts variant files that predate the accounting table
 // (or whose accounting insert failed, e.g. StoreVariant's best-effort
-// insert under transient DB contention), drops variant accounting rows
+// insert under transient DB contention), drops accounting rows whose
-// whose files are missing, removes source blob files the database does
+// files are missing, removes source blob files the database does not
-// not know, removes source and variant files that are empty or not the
+// know (and rows whose files are gone), removes source and variant files
-// size their rows record, sweeps stale temp files left behind by crashed
+// that are empty or not the size their rows record, sweeps stale temp
-// writes, and last checks the total cache usage against the tables. It
+// files left behind by crashed writes, and last checks the total cache
-// reads the tables a page at a time.
+// usage against the tables. It reads the tables a page at a time.
 // Running it periodically, not just once, bounds how long such drift can
 // accumulate unaccounted for on a long-running process to about one
 // eviction interval plus ten times the last pass's time.
 // Once ctx is cancelled, it stops at the next file or row and returns
 // ctx's error.
 func (c *Cache) reconcileAccounting(ctx context.Context) error {
-	if c.config.MaxBytes == 0 {
+	if c.disabled {
 		return nil
 	}
 
@@ -511,19 +617,20 @@ func (c *Cache) reconcileAccounting(ctx context.Context) error {
 		return err
 	}
 
+	err = c.reconcileSourceRows(ctx)
+	if err != nil {
+		return err
+	}
+
 	return c.reconcileUsageTotal(ctx)
 }
 
-// reconcileFiles walks baseDir, one of the cache's storage directories,
+// reconcileVariantFiles walks the variant storage directory, checking
-// sweeping stale temp files and passing each other file, with its name, to
+// each variant file against its accounting row (reconcileVariantFile)
-// reconcileFile. Once ctx is cancelled it stops at the next file and
+// and sweeping stale temp files.
-// returns ctx's error.
+func (c *Cache) reconcileVariantFiles(ctx context.Context) error {
-func (c *Cache) reconcileFiles(
-	ctx context.Context, baseDir string,
-	reconcileFile func(path string, entry fs.DirEntry, name string) error,
-) error {
 	return filepath.WalkDir(
-		baseDir,
+		c.variants.baseDir,
 		func(path string, entry fs.DirEntry, err error) error {
 			if ctx.Err() != nil {
 				return ctx.Err()
@@ -541,21 +648,15 @@ func (c *Cache) reconcileFiles(
 				return nil
 			}
 
-			return reconcileFile(path, entry, name)
+			if strings.HasSuffix(name, variantMetaSuffix) {
+				return nil
+			}
+
+			return c.reconcileVariantFile(ctx, path, entry, imagerequest.VariantKey(name))
 		},
 	)
 }
 
-// reconcileVariantFiles walks the variant storage directory, checking
-// each variant file against its accounting row (reconcileVariantFile)
-// and sweeping stale temp files.
-func (c *Cache) reconcileVariantFiles(ctx context.Context) error {
-	return c.reconcileFiles(ctx, c.variants.baseDir,
-		func(path string, entry fs.DirEntry, name string) error {
-			return c.reconcileVariantFile(ctx, path, entry, imagerequest.VariantKey(name))
-		})
-}
-
 // reconcileVariantFile checks one variant file against its accounting
 // row. A file without a row is adopted into the accounting, unless it is
 // empty. A file that is empty, or not the size its row records, as a
@@ -577,7 +678,7 @@ func (c *Cache) reconcileVariantFile(
 
 	switch {
 	case errors.Is(err, sql.ErrNoRows) && info.Size() > 0:
-		return c.adoptVariantFile(ctx, info, cacheKey)
+		return c.adoptVariantFile(ctx, path, info, cacheKey)
 	case err != nil && !errors.Is(err, sql.ErrNoRows):
 		return fmt.Errorf("failed to check variant accounting row: %w", err)
 	case info.Size() > 0 && info.Size() == recordedSize:
@@ -594,14 +695,16 @@ func (c *Cache) reconcileVariantFile(
 // adoptVariantFile inserts an accounting row for a variant file that
 // has none, using the file's size and modification time.
 func (c *Cache) adoptVariantFile(
-	ctx context.Context, info fs.FileInfo, cacheKey imagerequest.VariantKey,
+	ctx context.Context, path string, info fs.FileInfo, cacheKey imagerequest.VariantKey,
 ) error {
 	modTime := info.ModTime().UTC().Format(sqliteTimestampLayout)
+	contentType := c.variantContentTypeFromSidecar(path)
 
 	_, err := c.db.ExecContext(ctx, `
-		INSERT INTO variant_content (cache_key, size_bytes, last_accessed_at)
+		INSERT INTO variant_content
-		VALUES (?, ?, ?)
+			(cache_key, size_bytes, content_type, created_at, last_accessed_at)
-	`, string(cacheKey), info.Size(), modTime)
+		VALUES (?, ?, ?, ?, ?)
+	`, string(cacheKey), info.Size(), contentType, modTime, modTime)
 	if err != nil {
 		return fmt.Errorf("failed to adopt variant file into accounting: %w", err)
 	}
@@ -612,6 +715,23 @@ func (c *Cache) adoptVariantFile(
 	return nil
 }
 
+// variantContentTypeFromSidecar reads the content type from a variant
+// .meta sidecar, falling back to application/octet-stream.
+func (c *Cache) variantContentTypeFromSidecar(variantPath string) string {
+	//nolint:gosec // path from cache walk
+	metaData, err := os.ReadFile(variantPath + variantMetaSuffix)
+	if err != nil {
+		return fallbackContentType
+	}
+
+	var meta VariantMeta
+	if json.Unmarshal(metaData, &meta) != nil || meta.ContentType == "" {
+		return fallbackContentType
+	}
+
+	return meta.ContentType
+}
+
 // reconcileVariantRows drops accounting rows whose variant files are
 // missing, so the database never references deleted content.
 func (c *Cache) reconcileVariantRows(ctx context.Context) error {
@@ -703,20 +823,38 @@ func queryStringColumn[T ~string](
 // each blob file against its source_content row (reconcileSourceFile)
 // and sweeping stale temp files.
 func (c *Cache) reconcileSourceFiles(ctx context.Context) error {
-	return c.reconcileFiles(ctx, c.srcContent.baseDir,
+	return filepath.WalkDir(
-		func(path string, entry fs.DirEntry, name string) error {
+		c.srcContent.baseDir,
-			return c.reconcileSourceFile(ctx, path, entry, imagerequest.SourceKey(name))
+		func(path string, entry fs.DirEntry, err error) error {
-		})
+			if ctx.Err() != nil {
+				return ctx.Err()
+			}
+
+			if err != nil || entry.IsDir() {
+				return err
+			}
+
+			name := entry.Name()
+
+			if strings.HasPrefix(name, tempFilePrefix) {
+				c.sweepStaleTempFile(path, entry)
+
+				return nil
+			}
+
+			return c.reconcileSourceFile(ctx, path, entry, ContentHash(name))
+		},
+	)
 }
 
 // reconcileSourceFile checks one source blob file against its
-// source_content row. A file without a row is removed, as no lookup
+// source_content row. A file without a row is removed: it is
-// serves it and no eviction removes it. A file that is empty, or not the
+// unreachable, as source lookups always go through source_metadata. A
-// size its row records, as a crash or a restored snapshot can leave it,
+// file that is empty, or not the size its row records, as a crash or a
-// is removed with its row and a warning naming it, so the source is
+// restored snapshot can leave it, is removed with its rows and a warning
-// fetched again.
+// naming it, so the source is fetched again.
 func (c *Cache) reconcileSourceFile(
-	ctx context.Context, path string, entry fs.DirEntry, key imagerequest.SourceKey,
+	ctx context.Context, path string, entry fs.DirEntry, contentHash ContentHash,
 ) error {
 	info, err := entry.Info()
 	if err != nil {
@@ -726,13 +864,13 @@ func (c *Cache) reconcileSourceFile(
 	var recordedSize int64
 
 	err = c.db.QueryRowContext(ctx,
-		`SELECT size_bytes FROM source_content WHERE source_key = ?`,
+		`SELECT size_bytes FROM source_content WHERE content_hash = ?`,
-		string(key),
+		string(contentHash),
 	).Scan(&recordedSize)
 
 	switch {
 	case errors.Is(err, sql.ErrNoRows):
-		return c.removeUntrackedSourceFile(path, key)
+		return c.removeUntrackedSourceFile(ctx, path, contentHash)
 	case err != nil:
 		return fmt.Errorf("failed to check source content row: %w", err)
 	case info.Size() > 0 && info.Size() == recordedSize:
@@ -743,34 +881,96 @@ func (c *Cache) reconcileSourceFile(
 	c.log.Warn("removing damaged cached source file", "path", path,
 		"file_bytes", info.Size(), "recorded_bytes", recordedSize)
 
-	return c.evictSourceBlob(ctx, key)
+	return c.evictSourceBlob(ctx, contentHash)
 }
 
 // removeUntrackedSourceFile deletes a source blob file that has no
-// source_content row.
+// source_content row. Any source_metadata rows referencing the hash
+// are removed first so no row ever points at a deleted file.
 func (c *Cache) removeUntrackedSourceFile(
-	path string, key imagerequest.SourceKey,
+	ctx context.Context, path string, contentHash ContentHash,
 ) error {
-	err := os.Remove(path)
+	_, err := c.db.ExecContext(ctx,
+		`DELETE FROM source_metadata WHERE content_hash = ?`, string(contentHash))
+	if err != nil {
+		return fmt.Errorf("failed to delete metadata rows for untracked blob: %w", err)
+	}
+
+	err = os.Remove(path)
 	if err != nil && !os.IsNotExist(err) {
 		return fmt.Errorf("failed to remove untracked source file: %w", err)
 	}
 
-	c.log.Info("removed untracked source content file", "source_key", key)
+	c.log.Info("removed untracked source content file", "content_hash", contentHash)
 
 	return nil
 }
 
+// reconcileSourceRows removes source_content rows (and their metadata
+// references and sidecars) whose blob files are missing on disk.
+func (c *Cache) reconcileSourceRows(ctx context.Context) error {
+	var after ContentHash
+
+	for {
+		hashes, err := c.sourceContentHashesAfter(ctx, after)
+		if err != nil {
+			return err
+		}
+
+		if c.reconciliationReadTestHook != nil {
+			c.reconciliationReadTestHook(len(hashes))
+		}
+
+		if len(hashes) == 0 {
+			return nil
+		}
+
+		for _, hash := range hashes {
+			if ctx.Err() != nil {
+				return ctx.Err()
+			}
+
+			if c.srcContent.Exists(hash) {
+				continue
+			}
+
+			// The blob file is already gone; evictSourceBlob removes the
+			// rows and sidecars and tolerates the missing file.
+			err := c.evictSourceBlob(ctx, hash)
+			if err != nil {
+				return err
+			}
+
+			c.log.Info("dropped rows for missing source content file", "content_hash", hash)
+		}
+
+		after = hashes[len(hashes)-1]
+	}
+}
+
+// sourceContentHashesAfter returns, in order, up to
+// c.reconciliationPageSize tracked source content hashes that sort after
+// the given one.
+func (c *Cache) sourceContentHashesAfter(
+	ctx context.Context, after ContentHash,
+) ([]ContentHash, error) {
+	return queryStringColumn[ContentHash](ctx, c.db, `
+		SELECT content_hash FROM source_content
+		WHERE content_hash > ? ORDER BY content_hash LIMIT ?
+	`, "source content hashes", "content hash", string(after),
+		c.reconciliationPageSize)
+}
+
 // Each of these queries sums size_bytes over the next page of rows of
 // one content table, the rows that sort after a key, and returns the
 // page's last key, the sum and the number of rows in the page. Past the
 // last row the page has no rows and the key is NULL.
 const (
 	sourceSizePageQuery = `
-		SELECT MAX(source_key), COALESCE(SUM(size_bytes), 0), COUNT(*)
+		SELECT MAX(content_hash), COALESCE(SUM(size_bytes), 0), COUNT(*)
 		FROM (
-			SELECT source_key, size_bytes FROM source_content
+			SELECT content_hash, size_bytes FROM source_content
-			WHERE source_key > ? ORDER BY source_key LIMIT ?
+			WHERE content_hash > ? ORDER BY content_hash LIMIT ?
 		)`
 	variantSizePageQuery = `
 		SELECT MAX(cache_key), COALESCE(SUM(size_bytes), 0), COUNT(*)

internal/imgcache/eviction_internal_test.go

@@ -10,13 +10,12 @@ import (
 	"os"
 	"path/filepath"
 	"strings"
-	"sync"
 	"testing"
 	"time"
 
-	"github.com/prometheus/client_golang/prometheus"
+	_ "modernc.org/sqlite"
-	"modernc.org/sqlite"
 	"sneak.berlin/go/pixa/internal/database"
+	"sneak.berlin/go/pixa/internal/httpfetcher"
 	"sneak.berlin/go/pixa/internal/imagerequest"
 )
 
@@ -66,12 +65,16 @@ func newEvictionTestCache(t *testing.T, maxBytes int64) (*Cache, string) {
 	tmpDir := t.TempDir()
 	db := evictionTestDB(t)
 
-	// maxBytes zero turns the disk cache off, as cache_max_bytes: 0 does.
+	// maxBytes zero mirrors the production mapping of
+	// cache_max_bytes: 0 (handlers sets DisableDiskCache); at the
+	// CacheConfig layer itself a zero MaxBytes means "no limit" for
+	// backwards compatibility with existing fixtures.
 	cache, err := NewCache(db, CacheConfig{
-		StateDir:    tmpDir,
+		StateDir:         tmpDir,
-		NegativeTTL: 5 * time.Minute,
+		NegativeTTL:      5 * time.Minute,
-		MaxBytes:    maxBytes,
+		MaxBytes:         maxBytes,
-	}, prometheus.NewRegistry())
+		DisableDiskCache: maxBytes == 0,
+	})
 	if err != nil {
 		t.Fatalf("failed to create cache: %v", err)
 	}
@@ -80,10 +83,10 @@ func newEvictionTestCache(t *testing.T, maxBytes int64) (*Cache, string) {
 }
 
 // storeEvictionTestSource stores content as a fetched source for
-// host/path and returns the resulting source key.
+// host/path and returns the resulting content hash.
 func storeEvictionTestSource(
 	t *testing.T, cache *Cache, host, path string, content []byte,
-) imagerequest.SourceKey {
+) ContentHash {
 	t.Helper()
 
 	req := &imagerequest.Request{
@@ -98,14 +101,23 @@ func storeEvictionTestSource(
 		},
 	}
 
-	key, err := cache.StoreSource(
+	result := &httpfetcher.FetchResult{
-		context.Background(), req, bytes.NewReader(content),
+		StatusCode:    200,
+		ContentType:   testContentTypeJPEG,
+		ContentLength: int64(len(content)),
+		Headers: map[string][]string{
+			testHeaderContentType: {testContentTypeJPEG},
+		},
+	}
+
+	hash, err := cache.StoreSource(
+		context.Background(), req, bytes.NewReader(content), result,
 	)
 	if err != nil {
 		t.Fatalf("StoreSource(%s%s) failed: %v", host, path, err)
 	}
 
-	return key
+	return hash
 }
 
 // storeEvictionTestVariant stores content as a processed variant under
@@ -115,7 +127,7 @@ func storeEvictionTestVariant(
 ) {
 	t.Helper()
 
-	err := cache.StoreVariant(t.Context(), key, bytes.NewReader(content))
+	err := cache.StoreVariant(t.Context(), key, bytes.NewReader(content), "image/webp")
 	if err != nil {
 		t.Fatalf("StoreVariant(%s) failed: %v", key, err)
 	}
@@ -150,13 +162,13 @@ func setVariantLastAccessed(
 // setSourceLastAccessed backdates the last access time of a tracked
 // source content blob.
 func setSourceLastAccessed(
-	t *testing.T, cache *Cache, key imagerequest.SourceKey, when time.Time,
+	t *testing.T, cache *Cache, hash ContentHash, when time.Time,
 ) {
 	t.Helper()
 
 	res, err := cache.db.ExecContext(t.Context(),
-		`UPDATE source_content SET last_accessed_at = ? WHERE source_key = ?`,
+		`UPDATE source_content SET last_accessed_at = ? WHERE content_hash = ?`,
-		when.UTC().Format(sqliteTimestampFormat), string(key),
+		when.UTC().Format(sqliteTimestampFormat), string(hash),
 	)
 	if err != nil {
 		t.Fatalf("failed to set source last_accessed_at: %v", err)
@@ -168,7 +180,7 @@ func setSourceLastAccessed(
 	}
 
 	if affected != 1 {
-		t.Fatalf("source %s has no accounting row (affected=%d)", key, affected)
+		t.Fatalf("source %s has no accounting row (affected=%d)", hash, affected)
 	}
 }
 
@@ -193,31 +205,32 @@ func assertNoDanglingReferences(t *testing.T, cache *Cache) {
 	t.Helper()
 
 	rows, err := cache.db.QueryContext(t.Context(),
-		`SELECT source_key FROM source_content`,
+		`SELECT content_hash FROM source_metadata
+		 WHERE content_hash IS NOT NULL AND content_hash != ''`,
 	)
 	if err != nil {
-		t.Fatalf("failed to query source_content: %v", err)
+		t.Fatalf("failed to query source_metadata: %v", err)
 	}
 
 	defer func() { _ = rows.Close() }()
 
 	for rows.Next() {
-		var key string
+		var hash string
 
-		err := rows.Scan(&key)
+		err := rows.Scan(&hash)
 		if err != nil {
-			t.Fatalf("failed to scan source_key: %v", err)
+			t.Fatalf("failed to scan content_hash: %v", err)
 		}
 
-		if !cache.srcContent.Exists(imagerequest.SourceKey(key)) {
+		if !cache.srcContent.Exists(ContentHash(hash)) {
-			t.Errorf("source_content references content %s but the file is missing",
+			t.Errorf("source_metadata references content %s but the file is missing",
-				key)
+				hash)
 		}
 	}
 
 	err = rows.Err()
 	if err != nil {
-		t.Fatalf("source_content iteration failed: %v", err)
+		t.Fatalf("source_metadata iteration failed: %v", err)
 	}
 
 	variantRows, err := cache.db.QueryContext(t.Context(),
@@ -300,6 +313,33 @@ func TestUsageBytesAccountsSourceAndVariantBytes(t *testing.T) {
 	}
 }
 
+func TestUsageBytesCountsMultiReferencedBlobOnce(t *testing.T) {
+	t.Parallel()
+
+	cache, _ := newEvictionTestCache(t, 1<<30)
+
+	content := bytes.Repeat([]byte{0xCC}, 1200)
+
+	hashOne := storeEvictionTestSource(t, cache,
+		"src.example.com", "/one.jpg", content)
+	hashTwo := storeEvictionTestSource(t, cache,
+		"src.example.com", "/two.jpg", content)
+
+	if hashOne != hashTwo {
+		t.Fatalf("identical content produced different hashes: %s vs %s",
+			hashOne, hashTwo)
+	}
+
+	usage, err := cache.UsageBytes(context.Background())
+	if err != nil {
+		t.Fatalf("UsageBytes failed: %v", err)
+	}
+
+	if usage != 1200 {
+		t.Errorf("UsageBytes = %d, want 1200 (deduplicated blob counted once)", usage)
+	}
+}
+
 func TestEvictToLimitEvictsLeastRecentlyUsedFirst(t *testing.T) {
 	t.Parallel()
 
@@ -353,15 +393,102 @@ func TestEvictToLimitEvictsLeastRecentlyUsedFirst(t *testing.T) {
 	assertNoDanglingReferences(t, cache)
 }
 
+func TestEvictionRemovesMultiReferencedBlobTogetherWithAllReferences(t *testing.T) {
+	t.Parallel()
+
+	const limit = 1000
+
+	cache, _ := newEvictionTestCache(t, limit)
+
+	now := time.Now()
+
+	// One 800-byte blob referenced by two source paths.
+	sharedContent := bytes.Repeat([]byte{0xDD}, 800)
+	sharedHash := storeEvictionTestSource(t, cache,
+		"src.example.com", "/a.jpg", sharedContent)
+
+	h := storeEvictionTestSource(t, cache,
+		"src.example.com", "/b.jpg", sharedContent)
+	if h != sharedHash {
+		t.Fatalf("identical content produced different hashes: %s vs %s",
+			h, sharedHash)
+	}
+
+	// A newer 600-byte blob referenced by one source path.
+	recentHash := storeEvictionTestSource(t, cache, "src.example.com", "/c.jpg",
+		bytes.Repeat([]byte{0xEE}, 600))
+
+	setSourceLastAccessed(t, cache, sharedHash, now.Add(-2*time.Hour))
+	setSourceLastAccessed(t, cache, recentHash, now.Add(-time.Minute))
+
+	err := cache.EvictToLimit(context.Background())
+	if err != nil {
+		t.Fatalf("EvictToLimit failed: %v", err)
+	}
+
+	usage, err := cache.UsageBytes(context.Background())
+	if err != nil {
+		t.Fatalf("UsageBytes failed: %v", err)
+	}
+
+	if usage > limit {
+		t.Errorf("usage after eviction = %d, want <= %d", usage, limit)
+	}
+
+	// The multi-referenced blob must be gone from disk, from
+	// source_content, and from BOTH source_metadata rows: references
+	// are removed together with the blob, never left dangling.
+	if cache.srcContent.Exists(sharedHash) {
+		t.Errorf("evicted blob %s still exists on disk", sharedHash)
+	}
+
+	if n := countRows(t, cache,
+		`SELECT COUNT(*) FROM source_content WHERE content_hash = ?`, string(sharedHash),
+	); n != 0 {
+		t.Errorf("evicted blob %s still has %d source_content rows", sharedHash, n)
+	}
+
+	if n := countRows(t, cache,
+		`SELECT COUNT(*) FROM source_metadata WHERE content_hash = ?`, string(sharedHash),
+	); n != 0 {
+		t.Errorf("evicted blob %s still has %d source_metadata references", sharedHash, n)
+	}
+
+	// The JSON metadata sidecars for both referencing paths must be
+	// removed along with the rows.
+	for _, path := range []string{"/a.jpg", "/b.jpg"} {
+		pathHash := HashPath(path + "?")
+		if cache.srcMetadata.Exists("src.example.com", pathHash) {
+			t.Errorf("metadata sidecar for %s must be removed with its row", path)
+		}
+	}
+
+	// The more recently used blob survives fully intact.
+	if !cache.srcContent.Exists(recentHash) {
+		t.Errorf("recently used blob %s must survive eviction", recentHash)
+	}
+
+	if n := countRows(t, cache,
+		`SELECT COUNT(*) FROM source_metadata WHERE content_hash = ?`, string(recentHash),
+	); n != 1 {
+		t.Errorf("recently used blob %s has %d source_metadata rows, want 1",
+			recentHash, n)
+	}
+
+	assertNoDanglingReferences(t, cache)
+}
+
 func TestEvictionKeepsEverythingWhenUnderLimit(t *testing.T) {
 	t.Parallel()
 
 	cache, _ := newEvictionTestCache(t, 1<<30)
 
 	content := bytes.Repeat([]byte{0xDF}, 800)
-	keys := []imagerequest.SourceKey{
+	hash := storeEvictionTestSource(t, cache, "src.example.com", "/a.jpg", content)
-		storeEvictionTestSource(t, cache, "src.example.com", "/a.jpg", content),
+
-		storeEvictionTestSource(t, cache, "src.example.com", "/b.jpg", content),
+	h := storeEvictionTestSource(t, cache, "src.example.com", "/b.jpg", content)
+	if h != hash {
+		t.Fatalf("identical content produced different hashes: %s vs %s", h, hash)
 	}
 
 	storeEvictionTestVariant(t, cache, testVariantKeyOne,
@@ -372,16 +499,14 @@ func TestEvictionKeepsEverythingWhenUnderLimit(t *testing.T) {
 		t.Fatalf("EvictToLimit failed: %v", err)
 	}
 
-	for _, key := range keys {
+	if !cache.srcContent.Exists(hash) {
-		if !cache.srcContent.Exists(key) {
+		t.Errorf("blob %s must not be evicted while usage is under the limit", hash)
-			t.Errorf("source %s must not be evicted while usage is under the limit", key)
+	}
-		}
 
-		if n := countRows(t, cache,
+	if n := countRows(t, cache,
-			`SELECT COUNT(*) FROM source_content WHERE source_key = ?`, string(key),
+		`SELECT COUNT(*) FROM source_metadata WHERE content_hash = ?`, string(hash),
-		); n != 1 {
+	); n != 2 {
-			t.Errorf("source %s has %d source_content rows, want 1", key, n)
+		t.Errorf("blob %s has %d source_metadata rows, want 2", hash, n)
-		}
 	}
 
 	if !cache.variants.Exists(testVariantKeyOne) {
@@ -424,19 +549,26 @@ func assertDisabledCacheWritesAreNoOps(
 	ctx := t.Context()
 
 	err := cache.StoreVariant(
-		ctx, imagerequest.CacheKey(req), bytes.NewReader([]byte("data")),
+		ctx, imagerequest.CacheKey(req), bytes.NewReader([]byte("data")), "image/webp",
 	)
 	if err != nil {
 		t.Fatalf("StoreVariant on disabled cache must be a no-op, got error: %v", err)
 	}
 
-	key, err := cache.StoreSource(ctx, req, bytes.NewReader([]byte("data")))
+	result := &httpfetcher.FetchResult{
+		StatusCode:    200,
+		ContentType:   testContentTypeJPEG,
+		ContentLength: 4,
+		Headers:       map[string][]string{},
+	}
+
+	hash, err := cache.StoreSource(ctx, req, bytes.NewReader([]byte("data")), result)
 	if err != nil {
 		t.Fatalf("StoreSource on disabled cache must be a no-op, got error: %v", err)
 	}
 
-	if key != "" {
+	if hash != "" {
-		t.Errorf("StoreSource on disabled cache returned key %q, want empty", key)
+		t.Errorf("StoreSource on disabled cache returned hash %q, want empty", hash)
 	}
 }
 
@@ -449,13 +581,23 @@ func assertDisabledCacheReadsAlwaysMiss(
 
 	ctx := t.Context()
 
-	key, err := cache.LookupSource(ctx, req)
+	lookup, err := cache.Lookup(ctx, req)
+	if err != nil {
+		t.Fatalf("Lookup on disabled cache failed: %v", err)
+	}
+
+	if lookup.Hit {
+		t.Error("Lookup on disabled cache must always miss")
+	}
+
+	srcHash, srcType, err := cache.LookupSource(ctx, req)
 	if err != nil {
 		t.Fatalf("LookupSource on disabled cache failed: %v", err)
 	}
 
-	if key != "" {
+	if srcHash != "" || srcType != "" {
-		t.Errorf("LookupSource on disabled cache = %q, want empty", key)
+		t.Errorf("LookupSource on disabled cache = (%q, %q), want empty",
+			srcHash, srcType)
 	}
 }
 
@@ -476,6 +618,10 @@ func assertDisabledCacheTracksNothing(t *testing.T, cache *Cache) {
 	if n := countRows(t, cache, `SELECT COUNT(*) FROM source_content`); n != 0 {
 		t.Errorf("disabled cache wrote %d source_content rows, want 0", n)
 	}
+
+	if n := countRows(t, cache, `SELECT COUNT(*) FROM source_metadata`); n != 0 {
+		t.Errorf("disabled cache wrote %d source_metadata rows, want 0", n)
+	}
 }
 
 // assertDisabledCacheWroteNothingToDisk verifies that a disabled cache
@@ -586,7 +732,7 @@ func TestEvictionRunsOnPeriodicSchedule(t *testing.T) {
 	for i, key := range keys {
 		content := bytes.Repeat([]byte{fills[i]}, 1000)
 
-		_, err = cache.variants.Store(key, bytes.NewReader(content))
+		_, err = cache.variants.Store(key, bytes.NewReader(content), "image/webp")
 		if err != nil {
 			t.Fatalf("failed to store variant file: %v", err)
 		}
@@ -638,7 +784,7 @@ func TestStartEvictionReconcilesAccountingWithDisk(t *testing.T) {
 	untracked := bytes.Repeat([]byte{0x31}, 1000)
 
 	_, err := cache.variants.Store(
-		testVariantKeyOne, bytes.NewReader(untracked),
+		testVariantKeyOne, bytes.NewReader(untracked), "image/webp",
 	)
 	if err != nil {
 		t.Fatalf("failed to store untracked variant file: %v", err)
@@ -646,8 +792,9 @@ func TestStartEvictionReconcilesAccountingWithDisk(t *testing.T) {
 
 	// An accounting row whose file is missing must be dropped.
 	_, err = cache.db.ExecContext(t.Context(),
-		`INSERT INTO variant_content (cache_key, size_bytes) VALUES (?, ?)`,
+		`INSERT INTO variant_content (cache_key, size_bytes, content_type)
-		"deadbeef0001", 700,
+		 VALUES (?, ?, ?)`,
+		"deadbeef0001", 700, "image/webp",
 	)
 	if err != nil {
 		t.Fatalf("failed to insert stale variant accounting row: %v", err)
@@ -739,7 +886,7 @@ func TestPeriodicReconciliationAdoptsFileThatAppearsAfterStartup(t *testing.T) {
 	untracked := bytes.Repeat([]byte{0x41}, 900)
 
 	_, err = cache.variants.Store(
-		"aabbccdd0099", bytes.NewReader(untracked),
+		"aabbccdd0099", bytes.NewReader(untracked), "image/webp",
 	)
 	if err != nil {
 		t.Fatalf("failed to store untracked variant file: %v", err)
@@ -917,15 +1064,9 @@ func TestStopEvictionInterruptsPassInProgress(t *testing.T) {
 }
 
 // TestEvictToLimitStopsAtNextCandidateOnceCancelled cancels the context
-// once the oldest of three sources is evicted, while the next one waits
+// while the oldest of three source blobs is being evicted, and checks that
-// for the database, and checks that EvictToLimit then returns
+// EvictToLimit then returns context.Canceled without evicting the other
-// context.Canceled, with that source gone and the other two kept, and
+// two or logging a warning for either of them.
-// logs no warning. The test database has one connection. When the
-// deletion of the oldest source's row commits, a commit hook holds the
-// deletion until the test waits for the connection, so the test takes it
-// as soon as the deletion is done. The eviction of the next source then
-// waits for the connection, and the test cancels the context while it
-// waits.
 func TestEvictToLimitStopsAtNextCandidateOnceCancelled(t *testing.T) {
 	t.Parallel()
 
@@ -935,7 +1076,7 @@ func TestEvictToLimitStopsAtNextCandidateOnceCancelled(t *testing.T) {
 
 	cache.log = slog.New(slog.NewJSONHandler(&logBuf, nil))
 
-	keys := []imagerequest.SourceKey{
+	hashes := []ContentHash{
 		storeEvictionTestSource(t, cache, "cancel.example.com", "/a.jpg",
 			bytes.Repeat([]byte{0x61}, 1000)),
 		storeEvictionTestSource(t, cache, "cancel.example.com", "/b.jpg",
@@ -946,155 +1087,69 @@ func TestEvictToLimitStopsAtNextCandidateOnceCancelled(t *testing.T) {
 
 	base := time.Now().Add(-time.Hour)
 
-	for i, key := range keys {
+	for i, hash := range hashes {
-		setSourceLastAccessed(t, cache, key, base.Add(time.Duration(i)*time.Minute))
+		setSourceLastAccessed(t, cache, hash, base.Add(time.Duration(i)*time.Minute))
 	}
 
-	waitsBefore := cache.db.Stats().WaitCount
-	oldestRowDeleted := make(chan struct{})
-
-	var oldestRowDeletedOnce sync.Once
-
-	setCommitHook(t, cache, func() int32 {
-		oldestRowDeletedOnce.Do(func() {
-			close(oldestRowDeleted)
-
-			deadline := time.Now().Add(5 * time.Second)
-
-			for cache.db.Stats().WaitCount == waitsBefore && time.Now().Before(deadline) {
-				time.Sleep(time.Millisecond)
-			}
-		})
-
-		return 0
-	})
-
 	ctx, cancel := context.WithCancel(t.Context())
 	defer cancel()
 
-	evicted := make(chan error, 1)
+	cache.evictSourceBlobTestHook = func(ContentHash) { cancel() }
 
-	go func() { evicted <- cache.EvictToLimit(ctx) }()
+	err := cache.EvictToLimit(ctx)
-
-	select {
-	case <-oldestRowDeleted:
-	case err := <-evicted:
-		t.Fatalf("EvictToLimit() returned %v without deleting a row", err)
-	case <-time.After(5 * time.Second):
-		t.Fatal("EvictToLimit never deleted a row")
-	}
-
-	releaseDatabase := holdDatabase(t, cache)
-	defer releaseDatabase()
-
-	waitForConnectionWaits(t, cache.db, waitsBefore+2)
-	cancel()
-
-	err := <-evicted
 	t.Logf("EvictToLimit() error = %v", err)
 	t.Logf("log output: %s", logBuf.String())
 
-	// The checks below read the database, so the connection goes back.
-	releaseDatabase()
-
-	setCommitHook(t, cache, nil)
-
 	if !errors.Is(err, context.Canceled) {
 		t.Errorf("EvictToLimit() error = %v, want context.Canceled", err)
 	}
 
-	assertOnlyOldestSourceEvicted(t, cache, keys)
+	if cache.srcContent.Exists(hashes[0]) {
+		t.Errorf("source blob %s, evicted when the context was cancelled, "+
+			"is still on disk", hashes[0])
+	}
+
+	for _, hash := range hashes[1:] {
+		if !cache.srcContent.Exists(hash) {
+			t.Errorf("source blob %s was evicted after the context was cancelled", hash)
+		}
+	}
 
 	if strings.Contains(logBuf.String(), `"level":"WARN"`) {
 		t.Errorf("EvictToLimit logged a warning after the context was cancelled")
 	}
-}
-
-// setCommitHook makes the SQLite driver call hook each time a write on
-// the test database's one connection commits, or stops it when hook is
-// nil.
-func setCommitHook(t *testing.T, cache *Cache, hook sqlite.CommitHookFn) {
-	t.Helper()
-
-	conn, err := cache.db.Conn(t.Context())
-	if err != nil {
-		t.Fatalf("failed to take the database connection: %v", err)
-	}
-
-	defer func() { _ = conn.Close() }()
-
-	err = conn.Raw(func(driverConn any) error {
-		hooks, ok := driverConn.(sqlite.HookRegisterer)
-		if !ok {
-			t.Fatalf("the database connection is a %T, which takes no hooks",
-				driverConn)
-		}
-
-		hooks.RegisterCommitHook(hook)
-
-		return nil
-	})
-	if err != nil {
-		t.Fatalf("failed to set the commit hook: %v", err)
-	}
-}
-
-// assertOnlyOldestSourceEvicted checks that the first of keys has neither
-// its file nor its row, and that every other one has both.
-func assertOnlyOldestSourceEvicted(
-	t *testing.T, cache *Cache, keys []imagerequest.SourceKey,
-) {
-	t.Helper()
-
-	if cache.srcContent.Exists(keys[0]) {
-		t.Errorf("source %s, evicted when the context was cancelled, "+
-			"is still on disk", keys[0])
-	}
-
-	for _, key := range keys[1:] {
-		if !cache.srcContent.Exists(key) {
-			t.Errorf("source %s was evicted after the context was cancelled", key)
-		}
-
-		if n := countRows(t, cache,
-			`SELECT COUNT(*) FROM source_content WHERE source_key = ?`, string(key),
-		); n != 1 {
-			t.Errorf("source %s, kept, has %d source_content rows, want 1", key, n)
-		}
-	}
 
 	assertNoDanglingReferences(t, cache)
 }
 
-// TestStopEvictionReturnsWhenItsContextEnds pauses the startup
+// TestStopEvictionReturnsWhenItsContextEnds pauses an eviction pass where
-// reconciliation pass in the hook called after each of its reads, where
+// cancellation cannot reach it, after a source blob's rows are deleted and
-// cancellation cannot reach it, and checks that StopEviction returns an
+// before its file is removed, and checks that StopEviction returns its
-// error wrapping its context's error when that context ends, instead of
+// context's error when that context ends instead of waiting for the pass.
-// waiting for the pass. Once the pass goes on, the goroutine exits, and a
+// Once the pass goes on, the goroutine exits and no row points at a
-// second StopEviction returns nil.
+// missing file.
 func TestStopEvictionReturnsWhenItsContextEnds(t *testing.T) {
 	t.Parallel()
 
-	cache, _ := newEvictionTestCache(t, 1<<30)
+	cache, _ := newEvictionTestCache(t, 1)
 
 	paused := make(chan struct{})
 	resume := make(chan struct{})
 
-	var pauseOnce sync.Once
+	cache.evictSourceBlobTestHook = func(ContentHash) {
-
+		close(paused)
-	cache.reconciliationReadTestHook = func(int) {
+		<-resume
-		pauseOnce.Do(func() {
-			close(paused)
-			<-resume
-		})
 	}
 
+	hash := storeEvictionTestSource(t, cache, "stop.example.com", "/a.jpg",
+		bytes.Repeat([]byte{0x61}, 1000))
+
 	cache.StartEviction(time.Hour)
 
 	select {
 	case <-paused:
 	case <-time.After(5 * time.Second):
-		t.Fatal("the reconciliation pass never made a read")
+		t.Fatal("the eviction pass never reached the source blob")
 	}
 
 	ctx, cancel := context.WithTimeout(t.Context(), 50*time.Millisecond)
@@ -1113,6 +1168,12 @@ func TestStopEvictionReturnsWhenItsContextEnds(t *testing.T) {
 	if err != nil {
 		t.Fatalf("second StopEviction() error = %v, want nil", err)
 	}
+
+	assertNoDanglingReferences(t, cache)
+
+	if cache.srcContent.Exists(hash) {
+		t.Errorf("source blob %s is still on disk after its rows were deleted", hash)
+	}
 }
 
 // TestReconciliationWalksStopOnceCancelled checks that both directory
@@ -1184,3 +1245,131 @@ func TestReconciliationPassLogsNoWarningOnceCancelled(t *testing.T) {
 		t.Errorf("runReconciliationPass logged a warning with a cancelled context")
 	}
 }
+
+// TestEvictSourceBlobExcludesConcurrentStoreOfIdenticalContent exercises
+// the exact TOCTOU window between evictSourceBlob's row-deletion
+// transaction commit and its content file unlink: a concurrent
+// StoreSource for a different source path whose content hashes to the
+// same value (real SHA-256 dedup, not a contrived case) must not be
+// able to insert a fresh row referencing the file while eviction is
+// mid-unlink, and must not lose its own store once eviction has fully
+// released the content hash.
+func TestEvictSourceBlobExcludesConcurrentStoreOfIdenticalContent(t *testing.T) {
+	t.Parallel()
+
+	cache, _ := newEvictionTestCache(t, 1<<30)
+	ctx := context.Background()
+
+	content := bytes.Repeat([]byte{0x55}, 400)
+
+	hash := storeEvictionTestSource(t, cache,
+		"race.example.com", "/first.jpg", content)
+
+	proceed := make(chan struct{})
+	storeAttempted := make(chan struct{})
+
+	cache.evictSourceBlobTestHook = func(gotHash ContentHash) {
+		if gotHash != hash {
+			t.Errorf("test hook invoked for hash %s, want %s", gotHash, hash)
+		}
+
+		close(storeAttempted)
+		<-proceed
+	}
+
+	evictDone := make(chan error, 1)
+
+	go func() {
+		evictDone <- cache.evictSourceBlob(ctx, hash)
+	}()
+
+	// Wait until eviction has committed its delete transaction and is
+	// paused (inside the test hook) immediately before unlinking the
+	// content file: exactly the window the review flagged.
+	<-storeAttempted
+
+	storeDone := storeIdenticalContentConcurrently(ctx, cache, content)
+
+	// The concurrent store must not be able to complete while eviction
+	// still holds the content hash (i.e. before the file is unlinked):
+	// if it could, it would insert a row referencing a file about to be
+	// removed out from under it.
+	select {
+	case err := <-storeDone:
+		t.Fatalf("StoreSource for identical content completed (err=%v) while eviction "+
+			"still held the content hash open between commit and unlink; the store and "+
+			"the evict of identical content are not mutually exclusive", err)
+	case <-time.After(200 * time.Millisecond):
+		// Expected: the store is blocked behind eviction's exclusion.
+	}
+
+	close(proceed)
+
+	err := <-evictDone
+	if err != nil {
+		t.Fatalf("evictSourceBlob failed: %v", err)
+	}
+
+	err = <-storeDone
+	if err != nil {
+		t.Fatalf("StoreSource failed: %v", err)
+	}
+
+	assertNoDanglingReferences(t, cache)
+
+	dupHash, _, err := cache.LookupSource(ctx, &imagerequest.Request{
+		Source: imagerequest.Source{
+			Host: "race.example.com",
+			Path: "/dup.jpg",
+		},
+	})
+	if err != nil {
+		t.Fatalf("LookupSource failed: %v", err)
+	}
+
+	if dupHash == "" {
+		t.Fatal("re-stored blob was lost: the store legitimately ran after eviction " +
+			"released the content hash and must have recreated the file and row")
+	}
+
+	if !cache.srcContent.Exists(dupHash) {
+		t.Errorf("source_content/source_metadata references %s but its file is missing",
+			dupHash)
+	}
+}
+
+// storeIdenticalContentConcurrently starts a StoreSource for a second
+// source path whose body hashes to the same content hash, returning the
+// channel its error is delivered on.
+func storeIdenticalContentConcurrently(
+	ctx context.Context, cache *Cache, content []byte,
+) chan error {
+	storeDone := make(chan error, 1)
+
+	go func() {
+		req := &imagerequest.Request{
+			Source: imagerequest.Source{
+				Host: "race.example.com",
+				Path: "/dup.jpg",
+			},
+			Transform: imagerequest.Transform{
+				Format:  imagerequest.FormatJPEG,
+				Quality: 85,
+				FitMode: imagerequest.FitCover,
+			},
+		}
+		result := &httpfetcher.FetchResult{
+			StatusCode:    200,
+			ContentType:   testContentTypeJPEG,
+			ContentLength: int64(len(content)),
+			Headers: map[string][]string{
+				testHeaderContentType: {testContentTypeJPEG},
+			},
+		}
+
+		_, err := cache.StoreSource(ctx, req, bytes.NewReader(content), result)
+		storeDone <- err
+	}()
+
+	return storeDone
+}

internal/imgcache/imgcache.go

@@ -1,7 +1,13 @@
-// Package imgcache provides types for the image caching proxy.
+// Package imgcache provides interfaces and types for the image caching proxy.
 package imgcache
 
-import "io"
+import (
+	"context"
+	"io"
+	"time"
+
+	"sneak.berlin/go/pixa/internal/imagerequest"
+)
 
 // ImageResponse represents a processed image ready to serve
 type ImageResponse struct {
@@ -13,7 +19,9 @@ type ImageResponse struct {
 	ContentType string
 	// ETag is the entity tag for caching
 	ETag string
-	// CacheStatus indicates HIT or MISS
+	// LastModified is when the content was last modified
+	LastModified time.Time
+	// CacheStatus indicates HIT, MISS, or STALE
 	CacheStatus CacheStatus
 	// FetchedBytes is the number of bytes fetched from upstream (0 if cache hit)
 	FetchedBytes int64
@@ -24,6 +32,35 @@ type CacheStatus string
 
 // Cache status values for response headers.
 const (
-	CacheHit  CacheStatus = "HIT"
+	CacheHit   CacheStatus = "HIT"
-	CacheMiss CacheStatus = "MISS"
+	CacheMiss  CacheStatus = "MISS"
+	CacheStale CacheStatus = "STALE"
 )
+
+// ImageCache is the main interface for the image caching proxy
+type ImageCache interface {
+	// Get retrieves a processed image, fetching and processing if necessary
+	Get(ctx context.Context, req *imagerequest.Request) (*ImageResponse, error)
+
+	// Warm pre-fetches and caches an image without returning it
+	Warm(ctx context.Context, req *imagerequest.Request) error
+
+	// Stats returns cache statistics
+	Stats(ctx context.Context) (*CacheStats, error)
+}
+
+// CacheStats contains cache statistics
+type CacheStats struct {
+	// TotalItems is the number of cached source images plus processed
+	// variants
+	TotalItems int64
+	// TotalSizeBytes is the total size of cached source images and
+	// processed variants
+	TotalSizeBytes int64
+	// HitCount is the number of cache hits
+	HitCount int64
+	// MissCount is the number of cache misses
+	MissCount int64
+	// HitRate is HitCount / (HitCount + MissCount)
+	HitRate float64
+}

internal/imgcache/max_concurrent_processing_internal_test.go

@@ -66,7 +66,7 @@ func TestService_Get_WaitsForSlotBeforeReadingCachedSource(t *testing.T) {
 
 	svc, fixtures := SetupTestService(t)
 	svc.processor = imageprocessor.New(
-		imageprocessor.Params{MaxInputBytes: 50 << 20, MaxConcurrentProcessing: 1},
+		imageprocessor.Params{MaxConcurrentProcessing: 1},
 	)
 
 	// A first request caches the photo as a source.
@@ -77,11 +77,11 @@ func TestService_Get_WaitsForSlotBeforeReadingCachedSource(t *testing.T) {
 
 	_ = resp.Content.Close()
 
-	sourceKey, err := svc.cache.LookupSource(t.Context(),
+	contentHash, _, err := svc.cache.LookupSource(t.Context(),
 		widthOnlyRequest(fixtures, 50))
-	if err != nil || sourceKey == "" {
+	if err != nil || contentHash == "" {
 		t.Fatalf("LookupSource() = %q, %v; want the cached source",
-			sourceKey, err)
+			contentHash, err)
 	}
 
 	release := holdProcessingSlot(t, svc.processor)
@@ -103,7 +103,7 @@ func TestService_Get_WaitsForSlotBeforeReadingCachedSource(t *testing.T) {
 	// before waiting, it would have read it by now.
 	time.Sleep(100 * time.Millisecond)
 
-	err = os.WriteFile(svc.cache.srcContent.keyToPath(sourceKey),
+	err = os.WriteFile(svc.cache.srcContent.hashToPath(contentHash),
 		generateTestJPEG(t, 100, 50, color.RGBA{0, 0, 255, 255}), 0o600)
 	if err != nil {
 		t.Fatalf("failed to rewrite the cached source: %v", err)

internal/imgcache/metacache_internal_test.go

@@ -0,0 +1,361 @@
+package imgcache
+
+import (
+	"bytes"
+	"encoding/json"
+	"errors"
+	"fmt"
+	"io"
+	"os"
+	"sync"
+	"testing"
+	"time"
+
+	"sneak.berlin/go/pixa/internal/imagerequest"
+)
+
+// webpRequest returns a request for a 100x100 WebP variant of path.
+func webpRequest(path string) *imagerequest.Request {
+	return &imagerequest.Request{
+		Source: imagerequest.Source{
+			Host: testHostCDN,
+			Path: path,
+		},
+		Transform: imagerequest.Transform{
+			Size:    imagerequest.Size{Width: 100, Height: 100},
+			Format:  imagerequest.FormatWebP,
+			Quality: 85,
+			FitMode: imagerequest.FitCover,
+		},
+	}
+}
+
+// assertVariantServed checks that GetVariant serves key with the given
+// content and the image/webp content type storeEvictionTestVariant stores.
+func assertVariantServed(
+	t *testing.T, cache *Cache, key imagerequest.VariantKey, content []byte,
+) {
+	t.Helper()
+
+	reader, size, contentType, err := cache.GetVariant(key)
+	if err != nil {
+		t.Fatalf("GetVariant(%s) error = %v", key, err)
+	}
+
+	defer func() { _ = reader.Close() }()
+
+	got, err := io.ReadAll(reader)
+	if err != nil {
+		t.Fatalf("reading variant %s: %v", key, err)
+	}
+
+	if !bytes.Equal(got, content) {
+		t.Errorf("GetVariant(%s) content = %q, want %q", key, got, content)
+	}
+
+	if size != int64(len(content)) {
+		t.Errorf("GetVariant(%s) size = %d, want %d", key, size, len(content))
+	}
+
+	if contentType != testContentTypeWebP {
+		t.Errorf("GetVariant(%s) content type = %q, want %q",
+			key, contentType, testContentTypeWebP)
+	}
+}
+
+// assertVariantNotFound checks that GetVariant refuses key with
+// ErrNotFound.
+func assertVariantNotFound(
+	t *testing.T, cache *Cache, key imagerequest.VariantKey,
+) {
+	t.Helper()
+
+	reader, _, _, err := cache.GetVariant(key)
+	if err == nil {
+		_ = reader.Close()
+	}
+
+	if !errors.Is(err, ErrNotFound) {
+		t.Errorf("GetVariant(%s) error = %v, want ErrNotFound", key, err)
+	}
+}
+
+// assertLookupMisses checks that Lookup reports request as a miss.
+func assertLookupMisses(
+	t *testing.T, cache *Cache, request *imagerequest.Request,
+) {
+	t.Helper()
+
+	lookup, err := cache.Lookup(t.Context(), request)
+	if err != nil {
+		t.Fatalf("Lookup(%s) error = %v", request.Source.Path, err)
+	}
+
+	if lookup.Hit {
+		t.Errorf("Lookup(%s) is a hit, want a miss", request.Source.Path)
+	}
+}
+
+// renameFile renames the file at from to to.
+func renameFile(t *testing.T, from, to string) {
+	t.Helper()
+
+	err := os.Rename(from, to)
+	if err != nil {
+		t.Fatalf("renaming %s: %v", from, err)
+	}
+}
+
+// TestSecondHitDoesNotReadMetaFile checks that once a variant has been
+// stored or read, a hit takes its content type from memory: with the
+// .meta file deleted, GetVariant must still return the stored content
+// type rather than the application/octet-stream it uses without one.
+func TestSecondHitDoesNotReadMetaFile(t *testing.T) {
+	t.Parallel()
+
+	cache, _ := newEvictionTestCache(t, 1<<20)
+	content := []byte("webp variant bytes")
+
+	storeEvictionTestVariant(t, cache, testVariantKeyOne, content)
+
+	// A second Cache on the same state directory starts with nothing in
+	// memory, as pixad does after a restart, so its first read uses the
+	// .meta file.
+	restarted, err := NewCache(cache.db, cache.config)
+	if err != nil {
+		t.Fatalf("NewCache() error = %v", err)
+	}
+
+	assertVariantServed(t, restarted, testVariantKeyOne, content)
+
+	err = os.Remove(cache.variants.keyToPath(testVariantKeyOne) + ".meta")
+	if err != nil {
+		t.Fatalf("removing .meta file: %v", err)
+	}
+
+	assertVariantServed(t, cache, testVariantKeyOne, content)
+	assertVariantServed(t, restarted, testVariantKeyOne, content)
+}
+
+// TestReadDuringStoreKeepsStoredContentType checks that a GetVariant
+// which began before StoreVariant finished cannot replace the content
+// type the store kept in memory. Such a read can find the variant file
+// but not yet its .meta file, and so gets application/octet-stream. The
+// test deletes the .meta file after the store, then runs the part of
+// GetVariant that comes after its check of memory.
+func TestReadDuringStoreKeepsStoredContentType(t *testing.T) {
+	t.Parallel()
+
+	cache, _ := newEvictionTestCache(t, 1<<20)
+	content := []byte("webp variant bytes")
+
+	storeEvictionTestVariant(t, cache, testVariantKeyOne, content)
+
+	err := os.Remove(cache.variants.keyToPath(testVariantKeyOne) + ".meta")
+	if err != nil {
+		t.Fatalf("removing .meta file: %v", err)
+	}
+
+	reader, _, contentType, err := cache.loadVariantWithMeta(testVariantKeyOne)
+	if err != nil {
+		t.Fatalf("loadVariantWithMeta(%s) error = %v", testVariantKeyOne, err)
+	}
+
+	_ = reader.Close()
+
+	t.Logf("the read without a .meta file got content type %q", contentType)
+
+	assertVariantServed(t, cache, testVariantKeyOne, content)
+}
+
+// TestReadOfOlderMetaFileKeepsStoredContentType checks that a read
+// which got its content type from a .meta file that StoreVariant had
+// not yet rewritten cannot replace the type the store kept in memory.
+// The test writes such a .meta file, with a different content type,
+// after the store, then runs the part of GetVariant that comes after
+// its check of memory.
+func TestReadOfOlderMetaFileKeepsStoredContentType(t *testing.T) {
+	t.Parallel()
+
+	cache, _ := newEvictionTestCache(t, 1<<20)
+	content := []byte("webp variant bytes")
+
+	storeEvictionTestVariant(t, cache, testVariantKeyOne, content)
+
+	olderMeta, err := json.Marshal(VariantMeta{
+		ContentType: testContentTypeJPEG,
+		Size:        int64(len(content)),
+	})
+	if err != nil {
+		t.Fatalf("encoding .meta file: %v", err)
+	}
+
+	metaPath := cache.variants.keyToPath(testVariantKeyOne) + ".meta"
+
+	err = os.WriteFile(metaPath, olderMeta, StorageFilePerm)
+	if err != nil {
+		t.Fatalf("writing .meta file: %v", err)
+	}
+
+	reader, _, contentType, err := cache.loadVariantWithMeta(testVariantKeyOne)
+	if err != nil {
+		t.Fatalf("loadVariantWithMeta(%s) error = %v", testVariantKeyOne, err)
+	}
+
+	_ = reader.Close()
+
+	if contentType != testContentTypeJPEG {
+		t.Fatalf("loadVariantWithMeta(%s) content type = %q, want %q from the .meta file",
+			testVariantKeyOne, contentType, testContentTypeJPEG)
+	}
+
+	kept, _ := cache.metaCache.Get(testVariantKeyOne)
+	if kept != testContentTypeWebP {
+		t.Errorf("content type in memory = %q, want the stored %q",
+			kept, testContentTypeWebP)
+	}
+}
+
+// TestFailedReadDuringStoreKeepsStoredContentType checks that a read
+// which found no .meta file cannot leave application/octet-stream in
+// memory, even when another read has removed the content type
+// StoreVariant kept there. In this order: the store; a read that finds
+// the variant in memory but cannot open its file, and so removes it
+// from memory; a read that opened the variant file before the store
+// wrote its .meta file. Later hits must get the stored content type.
+func TestFailedReadDuringStoreKeepsStoredContentType(t *testing.T) {
+	t.Parallel()
+
+	cache, _ := newEvictionTestCache(t, 1<<20)
+	content := []byte("webp variant bytes")
+	variantPath := cache.variants.keyToPath(testVariantKeyOne)
+	metaPath := variantPath + ".meta"
+
+	storeEvictionTestVariant(t, cache, testVariantKeyOne, content)
+
+	renameFile(t, variantPath, variantPath+".hidden")
+	assertVariantNotFound(t, cache, testVariantKeyOne)
+	renameFile(t, variantPath+".hidden", variantPath)
+
+	renameFile(t, metaPath, metaPath+".hidden")
+
+	reader, _, contentType, err := cache.GetVariant(testVariantKeyOne)
+	if err != nil {
+		t.Fatalf("GetVariant(%s) error = %v", testVariantKeyOne, err)
+	}
+
+	_ = reader.Close()
+
+	t.Logf("the read without a .meta file got content type %q", contentType)
+
+	renameFile(t, metaPath+".hidden", metaPath)
+
+	assertVariantServed(t, cache, testVariantKeyOne, content)
+}
+
+// TestEvictedVariantIsNotServed checks that a variant the evictor
+// removed is a miss and cannot be read, although it had been stored
+// and served before.
+func TestEvictedVariantIsNotServed(t *testing.T) {
+	t.Parallel()
+
+	cache, _ := newEvictionTestCache(t, 1500)
+
+	oldRequest := webpRequest("/old.jpg")
+	newRequest := webpRequest("/new.jpg")
+	oldKey := imagerequest.CacheKey(oldRequest)
+	newKey := imagerequest.CacheKey(newRequest)
+	oldContent := bytes.Repeat([]byte{0x01}, 1000)
+	newContent := bytes.Repeat([]byte{0x02}, 1000)
+
+	storeEvictionTestVariant(t, cache, oldKey, oldContent)
+	storeEvictionTestVariant(t, cache, newKey, newContent)
+	assertVariantServed(t, cache, oldKey, oldContent)
+	setVariantLastAccessed(t, cache, oldKey, time.Now().Add(-time.Hour))
+
+	err := cache.EvictToLimit(t.Context())
+	if err != nil {
+		t.Fatalf("EvictToLimit() error = %v", err)
+	}
+
+	assertLookupMisses(t, cache, oldRequest)
+	assertVariantNotFound(t, cache, oldKey)
+	assertVariantServed(t, cache, newKey, newContent)
+}
+
+// TestVariantDeletedFromDiskIsNotServed checks that a variant whose
+// file was deleted by something other than the evictor cannot be read,
+// and is a miss afterwards.
+func TestVariantDeletedFromDiskIsNotServed(t *testing.T) {
+	t.Parallel()
+
+	cache, _ := newEvictionTestCache(t, 1<<20)
+
+	request := webpRequest("/deleted.jpg")
+	key := imagerequest.CacheKey(request)
+	content := []byte("webp variant bytes")
+
+	storeEvictionTestVariant(t, cache, key, content)
+	assertVariantServed(t, cache, key, content)
+
+	err := os.Remove(cache.variants.keyToPath(key))
+	if err != nil {
+		t.Fatalf("removing variant file: %v", err)
+	}
+
+	assertVariantNotFound(t, cache, key)
+	assertLookupMisses(t, cache, request)
+}
+
+// TestConcurrentVariantStoreReadAndEvict stores, reads and evicts
+// variants from several goroutines at once, for the race detector.
+func TestConcurrentVariantStoreReadAndEvict(t *testing.T) {
+	t.Parallel()
+
+	cache, _ := newEvictionTestCache(t, 1<<20)
+	ctx := t.Context()
+
+	var wg sync.WaitGroup
+
+	for goroutine := range 8 {
+		wg.Go(func() {
+			key := imagerequest.VariantKey(fmt.Sprintf("aabbccdd01%02d", goroutine))
+			content := []byte(key)
+
+			for range 20 {
+				err := cache.StoreVariant(
+					ctx, key, bytes.NewReader(content), testContentTypeWebP)
+				if err != nil {
+					t.Errorf("StoreVariant(%s) error = %v", key, err)
+
+					return
+				}
+
+				reader, _, contentType, err := cache.GetVariant(key)
+				if err != nil {
+					t.Errorf("GetVariant(%s) error = %v", key, err)
+
+					return
+				}
+
+				_ = reader.Close()
+
+				if contentType != testContentTypeWebP {
+					t.Errorf("GetVariant(%s) content type = %q, want %q",
+						key, contentType, testContentTypeWebP)
+				}
+
+				err = cache.evictVariant(ctx, key)
+				if err != nil {
+					t.Errorf("evictVariant(%s) error = %v", key, err)
+
+					return
+				}
+
+				assertVariantNotFound(t, cache, key)
+			}
+		})
+	}
+
+	wg.Wait()
+}

internal/imgcache/negative_cache_internal_test.go

@@ -7,7 +7,6 @@ import (
 	"testing"
 	"time"
 
-	"github.com/prometheus/client_golang/prometheus"
 	"sneak.berlin/go/pixa/internal/imagerequest"
 )
 
@@ -20,8 +19,7 @@ func TestNegativeCache_StoreAndCheck(t *testing.T) {
 	cache, err := NewCache(db, CacheConfig{
 		StateDir:    dir,
 		NegativeTTL: 5 * time.Minute,
-		MaxBytes:    1 << 30,
+	})
-	}, prometheus.NewRegistry())
 	if err != nil {
 		t.Fatal(err)
 	}
@@ -45,7 +43,7 @@ func TestNegativeCache_StoreAndCheck(t *testing.T) {
 	}
 
 	// Store a negative entry
-	err = cache.StoreNegative(ctx, req)
+	err = cache.StoreNegative(ctx, req, 404, "not found")
 	if err != nil {
 		t.Fatal(err)
 	}
@@ -70,8 +68,7 @@ func TestNegativeCache_Expired(t *testing.T) {
 	cache, err := NewCache(db, CacheConfig{
 		StateDir:    dir,
 		NegativeTTL: 1 * time.Millisecond, // very short TTL
-		MaxBytes:    1 << 30,
+	})
-	}, prometheus.NewRegistry())
 	if err != nil {
 		t.Fatal(err)
 	}
@@ -85,7 +82,7 @@ func TestNegativeCache_Expired(t *testing.T) {
 	}
 
 	// Store a negative entry with very short TTL
-	err = cache.StoreNegative(ctx, req)
+	err = cache.StoreNegative(ctx, req, 500, "server error")
 	if err != nil {
 		t.Fatal(err)
 	}
@@ -123,7 +120,8 @@ func TestService_Get_ServesCachedVariantOfNegativeCachedURL(t *testing.T) {
 	_ = resp.Content.Close()
 
 	err = svc.cache.StoreNegative(t.Context(),
-		photoVariant(fixtures, 85, imagerequest.FitCover))
+		photoVariant(fixtures, 85, imagerequest.FitCover),
+		httpStatusBadGateway, "upstream answered 503")
 	if err != nil {
 		t.Fatalf("StoreNegative() error = %v", err)
 	}
@@ -149,13 +147,13 @@ func TestService_Get_ServesCachedVariantOfNegativeCachedURL(t *testing.T) {
 
 	_ = resp.Content.Close()
 
-	sourceKey, err := svc.cache.LookupSource(t.Context(),
+	contentHash, _, err := svc.cache.LookupSource(t.Context(),
 		photoVariant(fixtures, 85, imagerequest.FitCover))
-	if err != nil || sourceKey == "" {
+	if err != nil || contentHash == "" {
-		t.Fatalf("LookupSource() = %q, %v, want the cached source", sourceKey, err)
+		t.Fatalf("LookupSource() = %q, %v, want the cached source", contentHash, err)
 	}
 
-	err = svc.cache.evictSourceBlob(t.Context(), sourceKey)
+	err = svc.cache.evictSourceBlob(t.Context(), contentHash)
 	if err != nil {
 		t.Fatalf("evictSourceBlob() error = %v", err)
 	}
@@ -179,11 +177,11 @@ func TestEvictionLoopRemovesExpiredNegativeCacheRows(t *testing.T) {
 	t.Parallel()
 
 	tests := []struct {
-		name     string
+		name             string
-		maxBytes int64
+		disableDiskCache bool
 	}{
-		{name: "disk cache enabled", maxBytes: 1 << 30},
+		{name: "disk cache enabled", disableDiskCache: false},
-		{name: "disk cache disabled", maxBytes: 0},
+		{name: "disk cache disabled", disableDiskCache: true},
 	}
 
 	for _, tt := range tests {
@@ -191,10 +189,10 @@ func TestEvictionLoopRemovesExpiredNegativeCacheRows(t *testing.T) {
 			t.Parallel()
 
 			cache, err := NewCache(evictionTestDB(t), CacheConfig{
-				StateDir:    t.TempDir(),
+				StateDir:         t.TempDir(),
-				NegativeTTL: time.Second,
+				NegativeTTL:      time.Second,
-				MaxBytes:    tt.maxBytes,
+				DisableDiskCache: tt.disableDiskCache,
-			}, prometheus.NewRegistry())
+			})
 			if err != nil {
 				t.Fatalf("failed to create cache: %v", err)
 			}
@@ -203,7 +201,7 @@ func TestEvictionLoopRemovesExpiredNegativeCacheRows(t *testing.T) {
 				Source: imagerequest.Source{Host: testHostExample, Path: "/gone.jpg"},
 			}
 
-			err = cache.StoreNegative(t.Context(), expired)
+			err = cache.StoreNegative(t.Context(), expired, 404, "not found")
 			if err != nil {
 				t.Fatalf("StoreNegative failed: %v", err)
 			}
@@ -212,8 +210,8 @@ func TestEvictionLoopRemovesExpiredNegativeCacheRows(t *testing.T) {
 
 			_, err = cache.db.ExecContext(t.Context(), `
 				INSERT INTO negative_cache
-					(source_host, source_path, expires_at)
+					(source_host, source_path, status_code, expires_at)
-				VALUES (?, ?, datetime('now', '+1 hour'))
+				VALUES (?, ?, 404, datetime('now', '+1 hour'))
 			`, testHostExample, notExpiredPath)
 			if err != nil {
 				t.Fatalf("failed to insert a row that has not expired: %v", err)
@@ -268,9 +266,7 @@ func negativeCacheRowCount(tb testing.TB, cache *Cache, sourcePath string) int {
 func TestCleanExpiredDeletesInBatches(t *testing.T) {
 	t.Parallel()
 
-	cache, err := NewCache(evictionTestDB(t),
+	cache, err := NewCache(evictionTestDB(t), CacheConfig{StateDir: t.TempDir()})
-		CacheConfig{StateDir: t.TempDir(), MaxBytes: 1 << 30},
-		prometheus.NewRegistry())
 	if err != nil {
 		t.Fatalf("failed to create cache: %v", err)
 	}
@@ -283,8 +279,8 @@ func TestCleanExpiredDeletesInBatches(t *testing.T) {
 
 	_, err = cache.db.ExecContext(ctx, `
 		INSERT INTO negative_cache
-			(source_host, source_path, expires_at)
+			(source_host, source_path, status_code, expires_at)
-		VALUES (?, ?, datetime('now', '+1 hour'))
+		VALUES (?, ?, 404, datetime('now', '+1 hour'))
 	`, testHostExample, notExpiredPath)
 	if err != nil {
 		t.Fatalf("failed to insert a row that has not expired: %v", err)
@@ -324,9 +320,7 @@ func TestCleanExpiredDeletesInBatches(t *testing.T) {
 func TestCleanExpiredRemovesMoreThanOneBatch(t *testing.T) {
 	t.Parallel()
 
-	cache, err := NewCache(evictionTestDB(t),
+	cache, err := NewCache(evictionTestDB(t), CacheConfig{StateDir: t.TempDir()})
-		CacheConfig{StateDir: t.TempDir(), MaxBytes: 1 << 30},
-		prometheus.NewRegistry())
 	if err != nil {
 		t.Fatalf("failed to create cache: %v", err)
 	}
@@ -356,9 +350,7 @@ func TestCleanExpiredRemovesMoreThanOneBatch(t *testing.T) {
 func TestCleanExpiredStopsOnceContextEnds(t *testing.T) {
 	t.Parallel()
 
-	cache, err := NewCache(evictionTestDB(t),
+	cache, err := NewCache(evictionTestDB(t), CacheConfig{StateDir: t.TempDir()})
-		CacheConfig{StateDir: t.TempDir(), MaxBytes: 1 << 30},
-		prometheus.NewRegistry())
 	if err != nil {
 		t.Fatalf("failed to create cache: %v", err)
 	}
@@ -386,8 +378,8 @@ func insertExpiredNegativeCacheRows(t *testing.T, cache *Cache, count int) {
 	for i := range count {
 		_, err := cache.db.ExecContext(t.Context(), `
 			INSERT INTO negative_cache
-				(source_host, source_path, expires_at)
+				(source_host, source_path, status_code, expires_at)
-			VALUES (?, ?, datetime('now', '-1 hour'))
+			VALUES (?, ?, 404, datetime('now', '-1 hour'))
 		`, testHostExample, "/expired/"+strconv.Itoa(i)+".jpg")
 		if err != nil {
 			t.Fatalf("failed to insert an expired row: %v", err)

internal/imgcache/partial_cache_files_internal_test.go

@@ -2,6 +2,8 @@ package imgcache
 
 import (
 	"bytes"
+	"crypto/sha256"
+	"encoding/hex"
 	"io"
 	"log/slog"
 	"os"
@@ -65,12 +67,12 @@ func cachedSourcePath(
 ) string {
 	t.Helper()
 
-	sourceKey, err := svc.cache.LookupSource(t.Context(), req)
+	contentHash, _, err := svc.cache.LookupSource(t.Context(), req)
-	if err != nil || sourceKey == "" {
+	if err != nil || contentHash == "" {
-		t.Fatalf("LookupSource() = %q, %v; want the cached source", sourceKey, err)
+		t.Fatalf("LookupSource() = %q, %v; want the cached source", contentHash, err)
 	}
 
-	return svc.cache.srcContent.keyToPath(sourceKey)
+	return svc.cache.srcContent.hashToPath(contentHash)
 }
 
 // reconcileAsAtStartup runs the cache's reconciliation pass, as pixa does
@@ -128,7 +130,7 @@ func TestService_Get_FetchesAgainForEmptyCachedSource(t *testing.T) {
 	getImage(t, svc, photoVariant(fixtures, 40, imagerequest.FitCover))
 	getImage(t, svc, photoVariant(fixtures, 30, imagerequest.FitCover))
 
-	fetches := readCounters(svc.cache).upstreamFetchCount
+	fetches := readCacheStatsCounters(t, svc.cache).upstreamFetchCount
 	t.Logf("upstream fetches = %d", fetches)
 
 	if fetches != 2 {
@@ -164,7 +166,7 @@ func TestService_Get_FetchesAgainForPartialCachedSource(t *testing.T) {
 	getImage(t, svc, photoVariant(fixtures, 40, imagerequest.FitCover))
 	getImage(t, svc, photoVariant(fixtures, 30, imagerequest.FitCover))
 
-	fetches := readCounters(svc.cache).upstreamFetchCount
+	fetches := readCacheStatsCounters(t, svc.cache).upstreamFetchCount
 	t.Logf("upstream fetches = %d", fetches)
 
 	if fetches != 2 {
@@ -239,7 +241,8 @@ func TestReconciliationRemovesEmptyVariantFileWithoutRow(t *testing.T) {
 
 	cache, _ := newEvictionTestCache(t, 1<<30)
 
-	_, err := cache.variants.Store(testVariantKeyOne, bytes.NewReader(nil))
+	_, err := cache.variants.Store(testVariantKeyOne, bytes.NewReader(nil),
+		testContentTypeWebP)
 	if err != nil {
 		t.Fatalf("failed to store an empty variant file: %v", err)
 	}
@@ -260,10 +263,63 @@ func TestReconciliationRemovesEmptyVariantFileWithoutRow(t *testing.T) {
 	}
 }
 
-// TestContentStorage_StoreReplacesFileOfAnotherSize checks that a
+// TestVariantStorage_StoreReplacesMetaFile stores a variant over one
-// store over a file of another size at the source's path, such as a
+// already stored and checks that Store puts a new .meta file in place of
+// the old one, as it does the image, rather than writing into the old
+// file, which a crash could leave empty. The old .meta file is given a
+// second name before the store, so that it can be told apart from the new
+// one after it.
+func TestVariantStorage_StoreReplacesMetaFile(t *testing.T) {
+	t.Parallel()
+
+	storage, key := storeTestVariant(t)
+	metaPath := storage.keyToPath(key) + variantMetaSuffix
+	oldMetaPath := metaPath + ".old"
+
+	err := os.Link(metaPath, oldMetaPath)
+	if err != nil {
+		t.Fatalf("giving the .meta file a second name: %v", err)
+	}
+
+	_, err = storage.Store(key, bytes.NewReader([]byte("new variant data")),
+		testContentTypeJPEG)
+	if err != nil {
+		t.Fatalf("Store() error = %v", err)
+	}
+
+	oldMeta, err := os.Stat(oldMetaPath)
+	if err != nil {
+		t.Fatalf("stat of the old .meta file: %v", err)
+	}
+
+	newMeta, err := os.Stat(metaPath)
+	if err != nil {
+		t.Fatalf("stat of the new .meta file: %v", err)
+	}
+
+	if os.SameFile(oldMeta, newMeta) {
+		t.Error("Store() wrote into the old .meta file instead of putting " +
+			"a new one in its place")
+	}
+
+	reader, _, contentType, err := storage.LoadWithMeta(key)
+	if err != nil {
+		t.Fatalf("LoadWithMeta() error = %v", err)
+	}
+
+	_ = reader.Close()
+
+	t.Logf("content type after the store: %s", contentType)
+
+	if contentType != testContentTypeJPEG {
+		t.Errorf("content type = %q, want %q", contentType, testContentTypeJPEG)
+	}
+}
+
+// TestContentStorage_StoreHashedReplacesFileOfAnotherSize checks that a
+// store over a file of another size at the content's path, such as a
 // partial file a crash left, replaces it with the content.
-func TestContentStorage_StoreReplacesFileOfAnotherSize(t *testing.T) {
+func TestContentStorage_StoreHashedReplacesFileOfAnotherSize(t *testing.T) {
 	t.Parallel()
 
 	storage, err := NewContentStorage(t.TempDir())
@@ -272,12 +328,13 @@ func TestContentStorage_StoreReplacesFileOfAnotherSize(t *testing.T) {
 	}
 
 	content := []byte("the whole source image")
-	key := imagerequest.Source{Host: testHostCDN, Path: testPathCat}.Key()
+	sum := sha256.Sum256(content)
-	path := storage.keyToPath(key)
+	hash := ContentHash(hex.EncodeToString(sum[:]))
+	path := storage.hashToPath(hash)
 
 	err = os.MkdirAll(filepath.Dir(path), StorageDirPerm)
 	if err != nil {
-		t.Fatalf("creating the source's directory: %v", err)
+		t.Fatalf("creating the content's directory: %v", err)
 	}
 
 	err = os.WriteFile(path, content[:5], StorageFilePerm)
@@ -285,14 +342,14 @@ func TestContentStorage_StoreReplacesFileOfAnotherSize(t *testing.T) {
 		t.Fatalf("writing a partial file: %v", err)
 	}
 
-	_, err = storage.Store(key, content)
+	_, err = storage.StoreHashed(hash, content)
 	if err != nil {
-		t.Fatalf("Store() error = %v", err)
+		t.Fatalf("StoreHashed() error = %v", err)
 	}
 
-	stored, _, err := storage.LoadWithSize(key)
+	stored, err := storage.Load(hash)
 	if err != nil {
-		t.Fatalf("LoadWithSize() error = %v", err)
+		t.Fatalf("Load() error = %v", err)
 	}
 
 	defer func() { _ = stored.Close() }()

internal/imgcache/pendingaccesstimes.go

@@ -1,160 +0,0 @@
-package imgcache
-
-import (
-	"context"
-	"fmt"
-	"time"
-
-	"sneak.berlin/go/pixa/internal/imagerequest"
-)
-
-// pendingWriteInterval is how often the goroutine
-// StartPendingAccessTimeWrites starts writes the pending access times.
-// SQLite's CURRENT_TIMESTAMP, which an access time is written as, counts
-// whole seconds.
-const pendingWriteInterval = time.Second
-
-// StartPendingAccessTimeWrites starts the goroutine that writes, every
-// pendingWriteInterval, the pending access times in one transaction.
-// Requests only hand these to it, so none waits for the database to record
-// them, and only this goroutine's write ever waits for it. It is a no-op
-// when already started. The goroutine outlives the caller, so it runs with
-// its own context, which StopPendingAccessTimeWrites cancels to tell it to
-// finish.
-func (c *Cache) StartPendingAccessTimeWrites() {
-	if c.pendingAccessTimesCancel != nil {
-		return
-	}
-
-	ctx, cancel := context.WithCancel(context.Background())
-	c.pendingAccessTimesCancel = cancel
-
-	go func() {
-		c.pendingAccessTimeWriteLoop(ctx)
-	}()
-}
-
-// StopPendingAccessTimeWrites tells the goroutine
-// StartPendingAccessTimeWrites started, if it did, to finish, and waits for
-// its write under way to end and for it to return. It then writes the
-// pending access times left, once, so that they are written before the
-// database closes. It waits for all of this at most until ctx ends. When
-// access times are not written, it logs that and returns an error.
-func (c *Cache) StopPendingAccessTimeWrites(ctx context.Context) error {
-	var err error
-
-	if c.pendingAccessTimesCancel != nil {
-		c.pendingAccessTimesCancel()
-
-		select {
-		case <-c.pendingAccessTimesDone:
-		case <-ctx.Done():
-			// The write given up on here may not reach the database:
-			// database/sql closes without waiting for a query, and a write
-			// still waiting for the connection fails when the database
-			// closes. Nothing is written here.
-			err = fmt.Errorf("access times still being written: %w", ctx.Err())
-		}
-	}
-
-	if err == nil {
-		err = c.writePendingAccessTimes(ctx)
-	}
-
-	if err != nil {
-		c.log.Error("access times not written at shutdown", "error", err)
-	}
-
-	return err
-}
-
-// pendingAccessTimeWriteLoop is the body of the goroutine
-// StartPendingAccessTimeWrites starts. Once ctx is cancelled it starts no
-// write and returns, but does not cut off a write under way then. Access
-// times whose write fails are dropped.
-func (c *Cache) pendingAccessTimeWriteLoop(ctx context.Context) {
-	defer close(c.pendingAccessTimesDone)
-
-	ticker := time.NewTicker(pendingWriteInterval)
-	defer ticker.Stop()
-
-	for {
-		select {
-		case <-ctx.Done():
-			return
-		case <-ticker.C:
-		}
-
-		// select picks at random when ctx is cancelled and a tick waits
-		// too.
-		if ctx.Err() != nil {
-			return
-		}
-
-		err := c.writePendingAccessTimes(context.WithoutCancel(ctx))
-		if err != nil {
-			c.log.Warn("failed to write access times", "error", err)
-		}
-	}
-}
-
-// writePendingAccessTimes takes out the variants waiting in
-// pendingVariantAccessTimes and the sources waiting in
-// pendingSourceAccessTimes, and sets their last_accessed_at to now in one
-// transaction, once for each however often it was used. Nothing else takes
-// from those channels while it runs: only the goroutine
-// StartPendingAccessTimeWrites starts calls it, or
-// StopPendingAccessTimeWrites once that goroutine has returned.
-func (c *Cache) writePendingAccessTimes(ctx context.Context) error {
-	variants := make(map[imagerequest.VariantKey]struct{})
-
-	for range len(c.pendingVariantAccessTimes) {
-		variants[<-c.pendingVariantAccessTimes] = struct{}{}
-	}
-
-	sources := make(map[imagerequest.SourceKey]struct{})
-
-	for range len(c.pendingSourceAccessTimes) {
-		sources[<-c.pendingSourceAccessTimes] = struct{}{}
-	}
-
-	accessTimes := len(variants) + len(sources)
-	if accessTimes == 0 {
-		return nil
-	}
-
-	tx, err := c.db.BeginTx(ctx, nil)
-	if err != nil {
-		return fmt.Errorf("failed to write %d access times: %w", accessTimes, err)
-	}
-
-	// Does nothing once the transaction is committed
-	defer func() { _ = tx.Rollback() }()
-
-	for cacheKey := range variants {
-		_, err = tx.ExecContext(ctx, `
-			UPDATE variant_content SET last_accessed_at = CURRENT_TIMESTAMP
-			WHERE cache_key = ?
-		`, string(cacheKey))
-		if err != nil {
-			return fmt.Errorf("failed to write %d access times: %w", accessTimes, err)
-		}
-	}
-
-	for sourceKey := range sources {
-		_, err = tx.ExecContext(ctx, `
-			UPDATE source_content SET last_accessed_at = CURRENT_TIMESTAMP
-			WHERE source_key = ?
-		`, string(sourceKey))
-		if err != nil {
-			return fmt.Errorf("failed to write %d access times: %w", accessTimes, err)
-		}
-	}
-
-	err = tx.Commit()
-	if err != nil {
-		return fmt.Errorf("failed to write %d access times: %w", accessTimes, err)
-	}
-
-	return nil
-}

internal/imgcache/pendingaccesstimes_internal_test.go

@@ -1,319 +0,0 @@
-package imgcache
-
-import (
-	"bytes"
-	"context"
-	"database/sql"
-	"errors"
-	"log/slog"
-	"strings"
-	"testing"
-	"time"
-
-	"sneak.berlin/go/pixa/internal/imagerequest"
-)
-
-// pendingAccessTimesWait is how long a test waits for the database's
-// connection, for an access time to reach the database, for a write to start
-// waiting for the connection, for a request to return past its deadline, or
-// for StopPendingAccessTimeWrites to return.
-const pendingAccessTimesWait = 5 * time.Second
-
-// otherWritesWait is how long a test gives other writes, once one write waits
-// for the database's connection, to start waiting for it too.
-const otherWritesWait = 100 * time.Millisecond
-
-// holdDatabase takes the one connection of cache's database, so that every
-// other query waits for it, and returns the func that frees it.
-func holdDatabase(t *testing.T, cache *Cache) func() {
-	t.Helper()
-
-	ctx, cancel := context.WithTimeout(t.Context(), pendingAccessTimesWait)
-	defer cancel()
-
-	conn, err := cache.db.Conn(ctx)
-	if err != nil {
-		t.Fatalf("failed to take the database connection: %v", err)
-	}
-
-	return func() { _ = conn.Close() }
-}
-
-// waitForConnectionWaits waits until db.Stats().WaitCount, the number of
-// times a caller has waited for a connection, reaches waits.
-func waitForConnectionWaits(t *testing.T, db *sql.DB, waits int64) {
-	t.Helper()
-
-	deadline := time.Now().Add(pendingAccessTimesWait)
-
-	for db.Stats().WaitCount < waits {
-		if time.Now().After(deadline) {
-			t.Fatalf("callers waited for the database connection %d times, want %d",
-				db.Stats().WaitCount, waits)
-		}
-
-		time.Sleep(10 * time.Millisecond)
-	}
-}
-
-// TestService_Get_ReturnsByItsDeadlineWhileTheDatabaseIsBusy holds the
-// database's one connection while a request whose fetch is held reaches its
-// deadline. The request must return with the deadline's error while the
-// connection is still held, as it waits for no database write past its
-// deadline.
-func TestService_Get_ReturnsByItsDeadlineWhileTheDatabaseIsBusy(t *testing.T) {
-	t.Parallel()
-
-	svc, fixtures, fetcher := setupHeldFetchService(t)
-
-	svc.cache.StartPendingAccessTimeWrites()
-	defer func() { _ = svc.cache.StopPendingAccessTimeWrites(t.Context()) }()
-
-	// Room for the request to reach its fetch on a busy host
-	const timeout = 2 * time.Second
-
-	ctx, cancel := context.WithTimeout(t.Context(), timeout)
-	defer cancel()
-
-	deadline, _ := ctx.Deadline()
-
-	results := startGet(ctx, svc, photoVariant(fixtures, 85, imagerequest.FitCover))
-
-	// The request has made its database reads by the time it fetches.
-	select {
-	case <-fetcher.started:
-	case <-time.After(timeout):
-		t.Fatal("request did not reach its fetch by its deadline")
-	}
-
-	releaseDatabase := holdDatabase(t, svc.cache)
-	defer releaseDatabase()
-
-	select {
-	case got := <-results:
-		t.Logf("Get() returned %v after its deadline, error = %v",
-			time.Since(deadline), got.err)
-
-		if !errors.Is(got.err, context.DeadlineExceeded) {
-			t.Errorf("Get() error = %v, want %v", got.err, context.DeadlineExceeded)
-		}
-	case <-time.After(time.Until(deadline) + pendingAccessTimesWait):
-		t.Fatal("request did not return while the database was busy")
-	}
-}
-
-// TestStopPendingAccessTimeWrites_LetsTheWriteUnderWayFinish holds the
-// database's one connection while the goroutine that writes the pending
-// access times waits for it, hands over another access time, then stops that
-// goroutine. The stop must leave the write under way to finish rather than
-// cut it off, and once the connection is free it must write the access time
-// that write left.
-func TestStopPendingAccessTimeWrites_LetsTheWriteUnderWayFinish(t *testing.T) {
-	t.Parallel()
-
-	cache, _ := newEvictionTestCache(t, 1<<30)
-	anHourAgo := time.Now().Add(-time.Hour)
-
-	for _, key := range []imagerequest.VariantKey{testVariantKeyOne, testVariantKeyTwo} {
-		storeEvictionTestVariant(t, cache, key, []byte("variant bytes"))
-		setVariantLastAccessed(t, cache, key, anHourAgo)
-	}
-
-	cache.StartPendingAccessTimeWrites()
-	defer func() { _ = cache.StopPendingAccessTimeWrites(t.Context()) }()
-
-	releaseDatabase := holdDatabase(t, cache)
-	defer releaseDatabase()
-
-	waitsBefore := cache.db.Stats().WaitCount
-
-	cache.touchVariant(testVariantKeyOne)
-
-	// The goroutine's write waits for the connection.
-	waitForConnectionWaits(t, cache.db, waitsBefore+1)
-
-	// Handed over after that write took the pending access times
-	cache.touchVariant(testVariantKeyTwo)
-
-	// The goroutine is told to finish here as well as by the stop, so that it
-	// is told before its write under way ends, however late the stop runs.
-	// The access time above then reaches the database only through the
-	// stop's own write.
-	cache.pendingAccessTimesCancel()
-
-	stopped := make(chan error, 1)
-
-	go func() {
-		stopped <- cache.StopPendingAccessTimeWrites(t.Context())
-	}()
-
-	// Had the stop cut off the write under way, its own write would wait for
-	// the connection too.
-	time.Sleep(otherWritesWait)
-
-	waits := cache.db.Stats().WaitCount - waitsBefore
-	t.Logf("%d writes waited for the database", waits)
-
-	if waits > 1 {
-		t.Errorf("%d writes waited for the database, want 1: "+
-			"the stop cut off the write under way", waits)
-	}
-
-	releaseDatabase()
-
-	select {
-	case err := <-stopped:
-		if err != nil {
-			t.Fatalf("StopPendingAccessTimeWrites() error = %v", err)
-		}
-	case <-time.After(pendingAccessTimesWait):
-		t.Fatal("StopPendingAccessTimeWrites() did not return once the database was free")
-	}
-
-	for _, key := range []imagerequest.VariantKey{testVariantKeyOne, testVariantKeyTwo} {
-		if countRows(t, cache, `
-			SELECT COUNT(*) FROM variant_content
-			WHERE cache_key = ? AND last_accessed_at > ?
-		`, string(key), anHourAgo.UTC().Format(sqliteTimestampFormat)) != 1 {
-			t.Errorf("the access time of variant %s was not written", key)
-		}
-	}
-}
-
-// TestStopPendingAccessTimeWrites_ReturnsWhenItsContextEnds holds the
-// database's one connection while the goroutine that writes the pending
-// access times waits for it, then stops that goroutine with a context that
-// ends first. The stop must return the context's error, and the write under
-// way must still reach the database once the connection is free.
-func TestStopPendingAccessTimeWrites_ReturnsWhenItsContextEnds(t *testing.T) {
-	t.Parallel()
-
-	cache, _ := newEvictionTestCache(t, 1<<30)
-	storeEvictionTestVariant(t, cache, testVariantKeyOne, []byte("variant bytes"))
-
-	anHourAgo := time.Now().Add(-time.Hour)
-	setVariantLastAccessed(t, cache, testVariantKeyOne, anHourAgo)
-
-	cache.StartPendingAccessTimeWrites()
-	defer func() { _ = cache.StopPendingAccessTimeWrites(t.Context()) }()
-
-	releaseDatabase := holdDatabase(t, cache)
-	defer releaseDatabase()
-
-	waitsBefore := cache.db.Stats().WaitCount
-
-	cache.touchVariant(testVariantKeyOne)
-
-	// The goroutine's write waits for the connection.
-	waitForConnectionWaits(t, cache.db, waitsBefore+1)
-
-	stopCtx, cancelStop := context.WithCancel(t.Context())
-	cancelStop()
-
-	stopped := make(chan error, 1)
-
-	go func() {
-		stopped <- cache.StopPendingAccessTimeWrites(stopCtx)
-	}()
-
-	select {
-	case err := <-stopped:
-		if !errors.Is(err, context.Canceled) {
-			t.Errorf("StopPendingAccessTimeWrites() error = %v, want %v",
-				err, context.Canceled)
-		}
-	case <-time.After(pendingAccessTimesWait):
-		t.Fatal("StopPendingAccessTimeWrites() did not return once its context ended")
-	}
-
-	releaseDatabase()
-
-	deadline := time.Now().Add(pendingAccessTimesWait)
-
-	for countRows(t, cache, `
-		SELECT COUNT(*) FROM variant_content
-		WHERE cache_key = ? AND last_accessed_at > ?
-	`, string(testVariantKeyOne), anHourAgo.UTC().Format(sqliteTimestampFormat)) == 0 {
-		if time.Now().After(deadline) {
-			t.Fatal("the access time being written was not written")
-		}
-
-		time.Sleep(10 * time.Millisecond)
-	}
-}
-
-// TestLookupSource_HandsAccessTimeToTheBackgroundWriter finds a cached source
-// whose access time is an hour old. LookupSource must not write the access
-// time itself but hand it over to be written, and StopPendingAccessTimeWrites
-// must then write it.
-func TestLookupSource_HandsAccessTimeToTheBackgroundWriter(t *testing.T) {
-	t.Parallel()
-
-	cache, _ := newEvictionTestCache(t, 1<<30)
-	key := storeEvictionTestSource(t, cache, "example.com", "/photo.jpg",
-		[]byte("source bytes"))
-	anHourAgo := time.Now().Add(-time.Hour)
-	setSourceLastAccessed(t, cache, key, anHourAgo)
-
-	touchedQuery := `
-		SELECT COUNT(*) FROM source_content
-		WHERE source_key = ? AND last_accessed_at > ?
-	`
-	anHourAgoInDatabase := anHourAgo.UTC().Format(sqliteTimestampFormat)
-
-	found, err := cache.LookupSource(t.Context(), &imagerequest.Request{
-		Source: imagerequest.Source{Host: "example.com", Path: "/photo.jpg"},
-	})
-	if err != nil || found != key {
-		t.Fatalf("LookupSource() = %q, %v; want %q", found, err, key)
-	}
-
-	if countRows(t, cache, touchedQuery, string(key), anHourAgoInDatabase) != 0 {
-		t.Error("LookupSource wrote the source's access time itself")
-	}
-
-	if waiting := len(cache.pendingSourceAccessTimes); waiting != 1 {
-		t.Errorf("%d access times of sources wait to be written, want 1", waiting)
-	}
-
-	err = cache.StopPendingAccessTimeWrites(t.Context())
-	if err != nil {
-		t.Fatalf("StopPendingAccessTimeWrites() error = %v", err)
-	}
-
-	if countRows(t, cache, touchedQuery, string(key), anHourAgoInDatabase) != 1 {
-		t.Error("the source's access time was not written")
-	}
-}
-
-// TestStopPendingAccessTimeWrites_LogsWhatWasNotWritten makes the write of
-// the access times fail at shutdown, by dropping the table they are written
-// to. The stop must return an error and log that access times were not
-// written.
-func TestStopPendingAccessTimeWrites_LogsWhatWasNotWritten(t *testing.T) {
-	t.Parallel()
-
-	cache, _ := newEvictionTestCache(t, 1<<30)
-
-	var logs bytes.Buffer
-
-	cache.log = slog.New(slog.NewJSONHandler(&logs, nil))
-
-	cache.touchVariant(testVariantKeyOne)
-
-	_, err := cache.db.ExecContext(t.Context(), `DROP TABLE variant_content`)
-	if err != nil {
-		t.Fatalf("dropping variant_content: %v", err)
-	}
-
-	err = cache.StopPendingAccessTimeWrites(t.Context())
-	t.Logf("StopPendingAccessTimeWrites() error = %v; logged:\n%s", err, logs.String())
-
-	if err == nil {
-		t.Error("StopPendingAccessTimeWrites() error = nil, want the access times' error")
-	}
-
-	if !strings.Contains(logs.String(), "access times not written at shutdown") {
-		t.Error("the access times not written were not logged")
-	}
-}

internal/imgcache/pendingcounts.go

@@ -0,0 +1,149 @@
+package imgcache
+
+import (
+	"context"
+	"fmt"
+	"sync/atomic"
+)
+
+// StartPendingCountWrites starts the goroutine that writes the pending
+// counts to the database whenever there are some, one UPDATE at a time:
+// however many requests pass their deadline before their counts are
+// written, at most this one write waits for the database. It is a no-op
+// when already started. The goroutine outlives the caller, so it runs with
+// its own context, which StopPendingCountWrites cancels to tell it to
+// finish.
+func (c *Cache) StartPendingCountWrites() {
+	if c.pendingCountsCancel != nil {
+		return
+	}
+
+	ctx, cancel := context.WithCancel(context.Background())
+	c.pendingCountsCancel = cancel
+
+	go func() {
+		c.pendingCountWriteLoop(ctx)
+	}()
+}
+
+// StopPendingCountWrites tells the goroutine StartPendingCountWrites
+// started, if it did, to finish, and waits for its write under way to end
+// and for it to return. It then writes the pending counts left, so that
+// they are written before the database closes. It waits for all of this at
+// most until ctx ends, then logs the counts not written and returns an
+// error.
+func (c *Cache) StopPendingCountWrites(ctx context.Context) error {
+	var err error
+
+	if c.pendingCountsCancel != nil {
+		c.pendingCountsCancel()
+
+		select {
+		case <-c.pendingCountsDone:
+		case <-ctx.Done():
+			// The write given up on here may not reach the database:
+			// database/sql closes without waiting for a query, and a
+			// write still waiting for the connection fails when the
+			// database closes.
+			err = fmt.Errorf("pending counts still being written: %w", ctx.Err())
+		}
+	}
+
+	if err == nil {
+		err = c.writePendingCounts(ctx)
+	}
+
+	if err != nil {
+		c.log.Error("counts not written at shutdown",
+			"hits", c.pendingHits.Load(),
+			"misses", c.pendingMisses.Load(),
+			"upstream_fetches", c.pendingUpstreamFetches.Load(),
+			"upstream_fetch_bytes", c.pendingUpstreamFetchBytes.Load(),
+			"transforms", c.pendingTransforms.Load(),
+			"error", err,
+		)
+
+		return err
+	}
+
+	return nil
+}
+
+// pendingCountWriteLoop is the body of the goroutine
+// StartPendingCountWrites starts. Once ctx is cancelled it starts no write
+// and returns, but does not cut off a write under way then. A write that
+// fails leaves the counts pending, for the next write.
+func (c *Cache) pendingCountWriteLoop(ctx context.Context) {
+	defer close(c.pendingCountsDone)
+
+	for {
+		select {
+		case <-ctx.Done():
+			return
+		case <-c.pendingCountsAdded:
+		}
+
+		// select picks at random when ctx is cancelled and a wakeup
+		// waits too.
+		if ctx.Err() != nil {
+			return
+		}
+
+		err := c.writePendingCounts(context.WithoutCancel(ctx))
+		if err != nil {
+			c.log.Warn("failed to write pending counts", "error", err)
+		}
+	}
+}
+
+// addPendingCount adds n to pendingCount, one of the pending counts, and
+// wakes the goroutine that writes them. pendingCountsAdded has capacity one,
+// so a wakeup already waiting is enough.
+func (c *Cache) addPendingCount(pendingCount *atomic.Int64, n int64) {
+	pendingCount.Add(n)
+
+	select {
+	case c.pendingCountsAdded <- struct{}{}:
+	default:
+	}
+}
+
+// writePendingCounts adds the pending counts to the cache_stats row in one
+// UPDATE, then takes what it wrote out of them, leaving any added
+// meanwhile. When the UPDATE fails, they all stay pending.
+func (c *Cache) writePendingCounts(ctx context.Context) error {
+	c.pendingCountsWriteMutex.Lock()
+	defer c.pendingCountsWriteMutex.Unlock()
+
+	hits := c.pendingHits.Load()
+	misses := c.pendingMisses.Load()
+	upstreamFetches := c.pendingUpstreamFetches.Load()
+	upstreamFetchBytes := c.pendingUpstreamFetchBytes.Load()
+	transforms := c.pendingTransforms.Load()
+
+	if hits+misses+upstreamFetches+upstreamFetchBytes+transforms == 0 {
+		return nil
+	}
+
+	_, err := c.db.ExecContext(ctx, `
+		UPDATE cache_stats
+		SET hit_count = hit_count + ?,
+		    miss_count = miss_count + ?,
+		    upstream_fetch_count = upstream_fetch_count + ?,
+		    upstream_fetch_bytes = upstream_fetch_bytes + ?,
+		    transform_count = transform_count + ?,
+		    last_updated_at = CURRENT_TIMESTAMP
+		WHERE id = 1
+	`, hits, misses, upstreamFetches, upstreamFetchBytes, transforms)
+	if err != nil {
+		return fmt.Errorf("failed to write pending counts: %w", err)
+	}
+
+	c.pendingHits.Add(-hits)
+	c.pendingMisses.Add(-misses)
+	c.pendingUpstreamFetches.Add(-upstreamFetches)
+	c.pendingUpstreamFetchBytes.Add(-upstreamFetchBytes)
+	c.pendingTransforms.Add(-transforms)
+
+	return nil
+}

internal/imgcache/pendingcounts_internal_test.go

@@ -0,0 +1,329 @@
+package imgcache
+
+import (
+	"context"
+	"database/sql"
+	"errors"
+	"testing"
+	"time"
+
+	"sneak.berlin/go/pixa/internal/imagerequest"
+)
+
+// pendingCountsWait is how long a test waits for the database's connection,
+// for a count to reach the database, for a write to start waiting for the
+// connection, for a request to return past its deadline, or for
+// StopPendingCountWrites to return.
+const pendingCountsWait = 5 * time.Second
+
+// otherWritesWait is how long a test gives other writes, once one write waits
+// for the database's connection, to start waiting for it too.
+const otherWritesWait = 100 * time.Millisecond
+
+// holdDatabase takes the one connection of cache's database, so that every
+// other query waits for it, and returns the func that frees it.
+func holdDatabase(t *testing.T, cache *Cache) func() {
+	t.Helper()
+
+	ctx, cancel := context.WithTimeout(t.Context(), pendingCountsWait)
+	defer cancel()
+
+	conn, err := cache.db.Conn(ctx)
+	if err != nil {
+		t.Fatalf("failed to take the database connection: %v", err)
+	}
+
+	return func() { _ = conn.Close() }
+}
+
+// waitForConnectionWaits waits until db.Stats().WaitCount, the number of
+// times a caller has waited for a connection, reaches waits.
+func waitForConnectionWaits(t *testing.T, db *sql.DB, waits int64) {
+	t.Helper()
+
+	deadline := time.Now().Add(pendingCountsWait)
+
+	for db.Stats().WaitCount < waits {
+		if time.Now().After(deadline) {
+			t.Fatalf("callers waited for the database connection %d times, want %d",
+				db.Stats().WaitCount, waits)
+		}
+
+		time.Sleep(10 * time.Millisecond)
+	}
+}
+
+// waitForCounters waits until the cache_stats row holds want.
+func waitForCounters(t *testing.T, cache *Cache, want cacheStatsCounters) {
+	t.Helper()
+
+	deadline := time.Now().Add(pendingCountsWait)
+
+	for {
+		got := readCacheStatsCounters(t, cache)
+		if got == want {
+			return
+		}
+
+		if time.Now().After(deadline) {
+			t.Fatalf("counters = %+v, want %+v", got, want)
+		}
+
+		time.Sleep(10 * time.Millisecond)
+	}
+}
+
+// TestService_Get_ReturnsByItsDeadlineWhileTheDatabaseIsBusy holds the
+// database's one connection while a request whose fetch is held reaches its
+// deadline. The request must return with the deadline's error while the
+// connection is still held, as it waits for no database write past its
+// deadline, and its miss must reach the database once the connection is free.
+func TestService_Get_ReturnsByItsDeadlineWhileTheDatabaseIsBusy(t *testing.T) {
+	t.Parallel()
+
+	svc, fixtures, fetcher := setupHeldFetchService(t)
+
+	svc.cache.StartPendingCountWrites()
+	defer func() { _ = svc.cache.StopPendingCountWrites(t.Context()) }()
+
+	// Room for the request to reach its fetch on a busy host
+	const timeout = 2 * time.Second
+
+	ctx, cancel := context.WithTimeout(t.Context(), timeout)
+	defer cancel()
+
+	deadline, _ := ctx.Deadline()
+
+	results := startGet(ctx, svc, photoVariant(fixtures, 85, imagerequest.FitCover))
+
+	// The request has made its database reads by the time it fetches.
+	select {
+	case <-fetcher.started:
+	case <-time.After(timeout):
+		t.Fatal("request did not reach its fetch by its deadline")
+	}
+
+	releaseDatabase := holdDatabase(t, svc.cache)
+	defer releaseDatabase()
+
+	select {
+	case got := <-results:
+		t.Logf("Get() returned %v after its deadline, error = %v",
+			time.Since(deadline), got.err)
+
+		if !errors.Is(got.err, context.DeadlineExceeded) {
+			t.Errorf("Get() error = %v, want %v", got.err, context.DeadlineExceeded)
+		}
+	case <-time.After(time.Until(deadline) + pendingCountsWait):
+		t.Fatal("request did not return while the database was busy")
+	}
+
+	releaseDatabase()
+
+	waitForCounters(t, svc.cache, cacheStatsCounters{missCount: 1})
+}
+
+// TestIncrementStats_ManyCountsPastTheirDeadlineLeaveOneWriteWaiting holds the
+// database's one connection while many misses are counted past their
+// deadline. At most one write may then wait for the connection, and every
+// miss must reach the database once it is free.
+func TestIncrementStats_ManyCountsPastTheirDeadlineLeaveOneWriteWaiting(t *testing.T) {
+	t.Parallel()
+
+	cache, _ := newEvictionTestCache(t, 1<<30)
+
+	cache.StartPendingCountWrites()
+	defer func() { _ = cache.StopPendingCountWrites(t.Context()) }()
+
+	releaseDatabase := holdDatabase(t, cache)
+	defer releaseDatabase()
+
+	waitsBefore := cache.db.Stats().WaitCount
+
+	// Writes given a context past its deadline fail at once.
+	ended, cancel := context.WithDeadline(t.Context(), time.Now())
+	defer cancel()
+
+	const misses = 20
+
+	for range misses {
+		cache.IncrementStats(ended, false, 0)
+	}
+
+	// Once one write waits for the connection, any others start waiting too.
+	waitForConnectionWaits(t, cache.db, waitsBefore+1)
+	time.Sleep(otherWritesWait)
+
+	waits := cache.db.Stats().WaitCount - waitsBefore
+	t.Logf("%d writes waited for the database", waits)
+
+	if waits > 1 {
+		t.Errorf("%d writes waited for the database, want at most 1", waits)
+	}
+
+	releaseDatabase()
+
+	waitForCounters(t, cache, cacheStatsCounters{missCount: misses})
+}
+
+// TestStats_IncludesPendingCounts counts hits and misses whose writes miss
+// their deadline, and checks that Stats adds them to the database's counts
+// before they are written.
+func TestStats_IncludesPendingCounts(t *testing.T) {
+	t.Parallel()
+
+	cache, _ := newEvictionTestCache(t, 1<<30)
+
+	_, err := cache.db.ExecContext(t.Context(),
+		`UPDATE cache_stats SET hit_count = 75, miss_count = 25 WHERE id = 1`)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	// Writes given a context past its deadline fail at once.
+	ended, cancel := context.WithDeadline(t.Context(), time.Now())
+	defer cancel()
+
+	cache.IncrementStats(ended, true, 0)
+	cache.IncrementStats(ended, true, 0)
+	cache.IncrementStats(ended, false, 0)
+
+	stats, err := cache.Stats(t.Context())
+	if err != nil {
+		t.Fatalf("Stats() error = %v", err)
+	}
+
+	if stats.HitCount != 77 || stats.MissCount != 26 {
+		t.Errorf("HitCount = %d, MissCount = %d, want 77 and 26",
+			stats.HitCount, stats.MissCount)
+	}
+
+	want := cacheStatsCounters{hitCount: 75, missCount: 25}
+	if got := readCacheStatsCounters(t, cache); got != want {
+		t.Errorf("counters in the database = %+v, want %+v", got, want)
+	}
+}
+
+// TestStopPendingCountWrites_WritesPendingCounts holds the database's one
+// connection while the goroutine that writes the pending counts waits for it,
+// counts more, then stops that goroutine. The stop must leave the write under
+// way to finish rather than cut it off, and once the connection is free it
+// must write the counts that write left.
+func TestStopPendingCountWrites_WritesPendingCounts(t *testing.T) {
+	t.Parallel()
+
+	cache, _ := newEvictionTestCache(t, 1<<30)
+
+	cache.StartPendingCountWrites()
+	defer func() { _ = cache.StopPendingCountWrites(t.Context()) }()
+
+	releaseDatabase := holdDatabase(t, cache)
+	defer releaseDatabase()
+
+	waitsBefore := cache.db.Stats().WaitCount
+
+	// Writes given a context past its deadline fail at once.
+	ended, cancel := context.WithDeadline(t.Context(), time.Now())
+	defer cancel()
+
+	cache.IncrementStats(ended, true, 0)
+
+	// The goroutine's write waits for the connection.
+	waitForConnectionWaits(t, cache.db, waitsBefore+1)
+
+	// Counted after that write read the pending counts
+	cache.IncrementStats(ended, false, 0)
+	cache.IncrementUpstreamFetch(ended, 1024)
+	cache.IncrementTransformCount(ended)
+
+	// The goroutine is told to finish here as well as by the stop, so that it
+	// is told before its write under way ends, however late the stop runs.
+	// The counts above then reach the database only through the stop's own
+	// write.
+	cache.pendingCountsCancel()
+
+	stopped := make(chan error, 1)
+
+	go func() {
+		stopped <- cache.StopPendingCountWrites(t.Context())
+	}()
+
+	// Had the stop cut off the write under way, its own write would wait for
+	// the connection too.
+	time.Sleep(otherWritesWait)
+
+	waits := cache.db.Stats().WaitCount - waitsBefore
+	t.Logf("%d writes waited for the database", waits)
+
+	if waits > 1 {
+		t.Errorf("%d writes waited for the database, want 1: "+
+			"the stop cut off the write under way", waits)
+	}
+
+	releaseDatabase()
+
+	select {
+	case err := <-stopped:
+		if err != nil {
+			t.Fatalf("StopPendingCountWrites() error = %v", err)
+		}
+	case <-time.After(pendingCountsWait):
+		t.Fatal("StopPendingCountWrites() did not return once the database was free")
+	}
+
+	want := cacheStatsCounters{1, 1, 1, 1024, 1}
+	if got := readCacheStatsCounters(t, cache); got != want {
+		t.Errorf("counters = %+v, want %+v", got, want)
+	}
+}
+
+// TestStopPendingCountWrites_ReturnsWhenItsContextEnds holds the database's
+// one connection while the goroutine that writes the pending counts waits for
+// it, then stops that goroutine with a context that ends first. The stop must
+// return the context's error, and the write under way must still reach the
+// database once the connection is free.
+func TestStopPendingCountWrites_ReturnsWhenItsContextEnds(t *testing.T) {
+	t.Parallel()
+
+	cache, _ := newEvictionTestCache(t, 1<<30)
+
+	cache.StartPendingCountWrites()
+	defer func() { _ = cache.StopPendingCountWrites(t.Context()) }()
+
+	releaseDatabase := holdDatabase(t, cache)
+	defer releaseDatabase()
+
+	waitsBefore := cache.db.Stats().WaitCount
+
+	// Writes given a context past its deadline fail at once.
+	ended, cancel := context.WithDeadline(t.Context(), time.Now())
+	defer cancel()
+
+	cache.IncrementStats(ended, true, 0)
+
+	// The goroutine's write waits for the connection.
+	waitForConnectionWaits(t, cache.db, waitsBefore+1)
+
+	stopCtx, cancelStop := context.WithCancel(t.Context())
+	cancelStop()
+
+	stopped := make(chan error, 1)
+
+	go func() {
+		stopped <- cache.StopPendingCountWrites(stopCtx)
+	}()
+
+	select {
+	case err := <-stopped:
+		if !errors.Is(err, context.Canceled) {
+			t.Errorf("StopPendingCountWrites() error = %v, want %v",
+				err, context.Canceled)
+		}
+	case <-time.After(pendingCountsWait):
+		t.Fatal("StopPendingCountWrites() did not return once its context ended")
+	}
+
+	releaseDatabase()
+
+	waitForCounters(t, cache, cacheStatsCounters{hitCount: 1})
+}

internal/imgcache/service.go

@@ -7,25 +7,33 @@ import (
 	"fmt"
 	"io"
 	"log/slog"
+	"net/url"
 	"runtime/debug"
+	"strings"
 	"time"
 
 	"github.com/dustin/go-humanize"
 	"github.com/getsentry/sentry-go"
 	"github.com/go-chi/chi/v5/middleware"
 	"golang.org/x/sync/singleflight"
+	"sneak.berlin/go/pixa/internal/allowlist"
 	"sneak.berlin/go/pixa/internal/httpfetcher"
 	"sneak.berlin/go/pixa/internal/imageprocessor"
 	"sneak.berlin/go/pixa/internal/imagerequest"
 	"sneak.berlin/go/pixa/internal/magic"
+	"sneak.berlin/go/pixa/internal/signature"
 )
 
-// Service orchestrates the cache, fetcher, and processor.
+// Service implements the ImageCache interface, orchestrating cache,
+// fetcher, and processor.
 type Service struct {
 	cache           *Cache
 	fetcher         httpfetcher.Fetcher
 	processor       *imageprocessor.ImageProcessor
+	signer          *signature.Signer
+	allowlist       *allowlist.HostAllowList
 	log             *slog.Logger
+	allowHTTP       bool
 	maxResponseSize int64
 	// variantsInProgress lets the requests that miss the same variant at the
 	// same time share one fetch and one transcode.
@@ -36,19 +44,27 @@ type Service struct {
 type ServiceConfig struct {
 	// Cache is the cache instance
 	Cache *Cache
-	// Fetcher fetches source images from upstream hosts
+	// FetcherConfig configures the upstream fetcher built when Fetcher is
+	// not set. Its AllowHTTP and MaxResponseSize are used either way.
+	FetcherConfig *httpfetcher.Config
+	// Fetcher is an optional custom fetcher (for testing)
 	Fetcher httpfetcher.Fetcher
-	// Processor converts and resizes the images
+	// SigningKey is the HMAC signing key (empty disables signing)
-	Processor *imageprocessor.ImageProcessor
+	SigningKey string
-	// MaxResponseSize is the largest source accepted, in bytes; a cached
+	// Allowlist is the list of hosts that don't require signatures
-	// source larger than this is fetched again
+	Allowlist []string
-	MaxResponseSize int64
+	// MaxConcurrentProcessing is the most images processed at once; zero
+	// uses the image processor's default, half the CPUs
+	MaxConcurrentProcessing int
 	// Logger for logging
 	Logger *slog.Logger
 }
 
-// errCacheRequired is returned when the service is built without a cache.
+// Static errors for service construction.
-var errCacheRequired = errors.New("cache is required")
+var (
+	errCacheRequired      = errors.New("cache is required")
+	errSigningKeyRequired = errors.New("signing key is required")
+)
 
 // NewService creates a new image service.
 func NewService(cfg *ServiceConfig) (*Service, error) {
@@ -56,11 +72,36 @@ func NewService(cfg *ServiceConfig) (*Service, error) {
 		return nil, errCacheRequired
 	}
 
+	if cfg.SigningKey == "" {
+		return nil, errSigningKeyRequired
+	}
+
+	// Resolve fetcher config for defaults
+	fetcherCfg := cfg.FetcherConfig
+	if fetcherCfg == nil {
+		fetcherCfg = httpfetcher.DefaultConfig()
+	}
+
+	// Use custom fetcher if provided, otherwise create HTTP fetcher
+	var fetcher httpfetcher.Fetcher
+	if cfg.Fetcher != nil {
+		fetcher = cfg.Fetcher
+	} else {
+		fetcher = httpfetcher.New(fetcherCfg)
+	}
+
+	signer := signature.New(cfg.SigningKey)
+
 	log := cfg.Logger
 	if log == nil {
 		log = slog.Default()
 	}
 
+	allowHTTP := false
+	if cfg.FetcherConfig != nil {
+		allowHTTP = cfg.FetcherConfig.AllowHTTP
+	}
+
 	// JPEG XL is the default output format, so pixad does not start
 	// without it.
 	err := imageprocessor.CheckJPEGXLSupport()
@@ -68,12 +109,21 @@ func NewService(cfg *ServiceConfig) (*Service, error) {
 		return nil, err
 	}
 
+	maxResponseSize := fetcherCfg.MaxResponseSize
+	processor := imageprocessor.New(imageprocessor.Params{
+		MaxInputBytes:           maxResponseSize,
+		MaxConcurrentProcessing: cfg.MaxConcurrentProcessing,
+	})
+
 	return &Service{
 		cache:           cfg.Cache,
-		fetcher:         cfg.Fetcher,
+		fetcher:         fetcher,
-		processor:       cfg.Processor,
+		processor:       processor,
+		signer:          signer,
+		allowlist:       allowlist.New(cfg.Allowlist),
 		log:             log,
-		maxResponseSize: cfg.MaxResponseSize,
+		allowHTTP:       allowHTTP,
+		maxResponseSize: maxResponseSize,
 	}, nil
 }
 
@@ -90,34 +140,36 @@ var ErrProcessingTimedOut = errors.New(
 // While its source URL is in the negative cache, a cached variant is still
 // served and a variant not in the cache is still made from a cached source;
 // only a request that needs the source fetched is refused.
-//
-// A cache hit opens the variant file and makes no database call: its access
-// time is handed to the goroutine that StartPendingAccessTimeWrites starts,
-// without waiting for it, and its hit is counted in memory. A variant file
-// that cannot be served, such as one whose first bytes are not those of an
-// output format, is a miss, and the variant is made again.
 func (s *Service) Get(
 	ctx context.Context, req *imagerequest.Request,
 ) (*ImageResponse, error) {
-	cacheKey := imagerequest.CacheKey(req)
+	// Propagate AllowHTTP setting to the request
+	req.AllowHTTP = s.allowHTTP
 
-	reader, size, contentType, err := s.cache.GetVariant(cacheKey)
+	// Check variant cache first (disk only, no DB)
-	if err == nil {
+	result, err := s.cache.Lookup(ctx, req)
-		s.cache.touchVariant(cacheKey)
+	if err != nil {
-		s.cache.IncrementStats(true, 0)
+		s.log.Warn("cache lookup failed", "error", err)
-
-		return &ImageResponse{
-			Content:       reader,
-			ContentLength: size,
-			ContentType:   contentType,
-			CacheStatus:   CacheHit,
-			ETag:          formatETag(cacheKey),
-		}, nil
 	}
 
-	if !errors.Is(err, ErrNotFound) {
+	// Cache hit - serve directly from disk
-		s.log.Warn("cached variant not served, making it again",
+	if result != nil && result.Hit {
-			"cache_key", cacheKey, "error", err)
+		reader, size, contentType, err := s.cache.GetVariant(result.CacheKey)
+		if err != nil {
+			s.log.Error("failed to get cached variant", "key", result.CacheKey, "error", err)
+			// Fall through to re-process
+		} else {
+			// Counted also when the request context has ended meanwhile
+			s.cache.IncrementStats(ctx, true, 0)
+
+			return &ImageResponse{
+				Content:       reader,
+				ContentLength: size,
+				ContentType:   contentType,
+				CacheStatus:   CacheHit,
+				ETag:          formatETag(result.CacheKey),
+			}, nil
+		}
 	}
 
 	// Cache miss - get the variant, processed once for all the requests that
@@ -125,7 +177,7 @@ func (s *Service) Get(
 	// failed or the request context has ended meanwhile
 	response, err := s.processOrWait(ctx, req)
 
-	s.cache.IncrementStats(false, 0)
+	s.cache.IncrementStats(ctx, false, 0)
 
 	if err != nil {
 		return nil, err
@@ -136,12 +188,75 @@ func (s *Service) Get(
 	return response, nil
 }
 
+// Warm pre-fetches and caches an image without returning it.
+func (s *Service) Warm(ctx context.Context, req *imagerequest.Request) error {
+	_, err := s.Get(ctx, req)
+
+	return err
+}
+
+// Stats returns cache statistics.
+func (s *Service) Stats(ctx context.Context) (*CacheStats, error) {
+	return s.cache.Stats(ctx)
+}
+
 // WaitForProcessing waits until no image is being processed, or until ctx
 // ends, and returns how many images were still being processed then.
 func (s *Service) WaitForProcessing(ctx context.Context) int {
 	return s.processor.WaitForProcessing(ctx)
 }
 
+// ValidateRequest validates the request signature if required.
+func (s *Service) ValidateRequest(req *imagerequest.Request) error {
+	// Check if host is allowed (no signature required)
+	sourceURL := req.SourceURL()
+
+	parsedURL, err := url.Parse(sourceURL)
+	if err != nil {
+		return fmt.Errorf("invalid source URL: %w", err)
+	}
+
+	// An allowlist entry names no port, so a host followed by ":", with or
+	// without a port, needs a signature. The port is read from the parsed
+	// URL, as the fetcher reads it: in an IPv6 address written without
+	// brackets, such as 2001:db8::1:8443, it is what follows the last ":".
+	// This is checked here and not in allowlist.IsAllowed, which also matches
+	// referer_blocklist, where a Referer must be blocked whatever its port.
+	hostHasPort := parsedURL.Port() != "" ||
+		strings.HasSuffix(parsedURL.Host, ":")
+
+	if !hostHasPort && s.allowlist.IsAllowed(parsedURL) {
+		return nil
+	}
+
+	// Signature required for non-allowed hosts
+	return s.signer.Verify(req)
+}
+
+// GenerateSignedURL generates a signed URL for the given request. The URL
+// carries q and fit next to sig and exp, so the image route verifies it for
+// the quality and fit it was signed with. An unset quality or fit is first
+// set to 85 or cover, the values the route uses when a URL has no q or fit.
+func (s *Service) GenerateSignedURL(
+	baseURL string,
+	req *imagerequest.Request,
+	ttl time.Duration,
+) (string, error) {
+	if req.Transform.Quality == 0 {
+		req.Transform.Quality = 85
+	}
+
+	if req.Transform.FitMode == "" {
+		req.Transform.FitMode = imagerequest.FitCover
+	}
+
+	// Sets the request's Expires and Signature
+	path, sig, exp := s.signer.GenerateSignedURL(req, ttl)
+
+	return fmt.Sprintf("%s%s?sig=%s&exp=%d&q=%d&fit=%s",
+		baseURL, path, sig, exp, req.Transform.Quality, req.Transform.FitMode), nil
+}
+
 // errPanicked is returned when processing a variant panicked.
 var errPanicked = errors.New("panic while processing image")
 
@@ -244,9 +359,9 @@ func withoutCancelKeepingDeadline(
 // returns it with its size; nil if the cached data is unavailable, empty or
 // exceeds maxResponseSize.
 func (s *Service) loadCachedSource(
-	sourceKey imagerequest.SourceKey,
+	contentHash ContentHash,
 ) (io.ReadCloser, int64) {
-	reader, size, err := s.cache.GetSourceContent(sourceKey)
+	reader, size, err := s.cache.GetSourceContent(contentHash)
 	if err != nil {
 		s.log.Warn("failed to load cached source, fetching", "error", err)
 
@@ -257,7 +372,7 @@ func (s *Service) loadCachedSource(
 		_ = reader.Close()
 
 		s.log.Warn("cached source exceeds max response size, discarding",
-			"source_key", sourceKey,
+			"hash", contentHash,
 			"max_bytes", s.maxResponseSize,
 		)
 
@@ -281,7 +396,7 @@ func (s *Service) processFromSourceOrFetch(
 	cacheKey imagerequest.VariantKey,
 ) (*processedVariant, error) {
 	// Check if we have cached source content
-	sourceKey, err := s.cache.LookupSource(ctx, req)
+	contentHash, _, err := s.cache.LookupSource(ctx, req)
 	if err != nil {
 		s.log.Warn("source lookup failed", "error", err)
 	}
@@ -291,9 +406,9 @@ func (s *Service) processFromSourceOrFetch(
 		sourceSize int64
 	)
 
-	if sourceKey != "" {
+	if contentHash != "" {
-		s.log.Debug("using cached source", "source_key", sourceKey)
+		s.log.Debug("using cached source", "hash", contentHash)
-		source, sourceSize = s.loadCachedSource(sourceKey)
+		source, sourceSize = s.loadCachedSource(contentHash)
 	}
 
 	// Fetch from upstream if we don't have source data or it's empty
@@ -334,14 +449,15 @@ func (s *Service) fetchAndProcess(
 	}
 
 	// Fetch from upstream
-	s.log.Debug("fetching from upstream", "host", req.Source.Host,
+	sourceURL := req.SourceURL()
-		"path", req.Source.Path, "query", req.Source.Query)
 
-	fetchResult, err := s.fetcher.Fetch(ctx, req.Source)
+	s.log.Debug("fetching from upstream", "url", sourceURL)
+
+	fetchResult, err := s.fetcher.Fetch(ctx, sourceURL)
 	if err != nil {
 		// Store negative cache for certain errors
 		if isNegativeCacheable(err) {
-			s.storeNegative(ctx, req)
+			s.storeNegative(ctx, req, extractStatusCode(err), err)
 		}
 
 		return nil, fmt.Errorf("upstream fetch failed: %w", err)
@@ -358,7 +474,7 @@ func (s *Service) fetchAndProcess(
 	fetchBytes := int64(len(sourceData))
 
 	// Counted also when the request context has ended meanwhile
-	s.cache.IncrementUpstreamFetch(fetchBytes)
+	s.cache.IncrementUpstreamFetch(ctx, fetchBytes)
 
 	if err != nil {
 		return nil, fmt.Errorf("failed to read upstream response: %w", err)
@@ -395,13 +511,13 @@ func (s *Service) fetchAndProcess(
 		s.log.Warn("source refused", "request_id", middleware.GetReqID(ctx),
 			"host", req.Source.Host, "path", req.Source.Path,
 			"content_type", fetchResult.ContentType, "error", err)
-		s.storeNegative(ctx, req)
+		s.storeNegative(ctx, req, httpStatusUnsupportedMediaType, err)
 
 		return nil, fmt.Errorf("content validation failed: %w", err)
 	}
 
 	// Store source content
-	_, err = s.cache.StoreSource(ctx, req, bytes.NewReader(sourceData))
+	_, err = s.cache.StoreSource(ctx, req, bytes.NewReader(sourceData), fetchResult)
 	if err != nil {
 		s.log.Warn("failed to store source content", "error", err)
 		// Continue even if caching fails
@@ -413,13 +529,15 @@ func (s *Service) fetchAndProcess(
 }
 
 // storeNegative stores a negative cache entry for req's source, whose fetch
-// failed, so the source is not fetched again until the entry expires. A
+// failed with err, so the source is not fetched again until the entry
-// failure to store the entry is only logged.
+// expires. A failure to store the entry is only logged.
-func (s *Service) storeNegative(ctx context.Context, req *imagerequest.Request) {
+func (s *Service) storeNegative(
-	err := s.cache.StoreNegative(ctx, req)
+	ctx context.Context, req *imagerequest.Request, statusCode int, err error,
-	if err != nil {
+) {
+	storeErr := s.cache.StoreNegative(ctx, req, statusCode, err.Error())
+	if storeErr != nil {
 		s.log.Warn("failed to store negative cache entry",
-			"host", req.Source.Host, "path", req.Source.Path, "error", err)
+			"host", req.Source.Host, "path", req.Source.Path, "error", storeErr)
 	}
 }
 
@@ -457,7 +575,7 @@ func (s *Service) processAndStore(
 	processDuration := time.Since(processStart)
 
 	// Counted also when the request context has ended meanwhile
-	s.cache.IncrementTransformCount()
+	s.cache.IncrementTransformCount(ctx)
 
 	// Read processed content
 	processedData, err := io.ReadAll(processResult.Content)
@@ -495,7 +613,9 @@ func (s *Service) processAndStore(
 	)
 
 	// Store variant to cache
-	err = s.cache.StoreVariant(ctx, cacheKey, bytes.NewReader(processedData))
+	err = s.cache.StoreVariant(
+		ctx, cacheKey, bytes.NewReader(processedData), processResult.ContentType,
+	)
 	if err != nil {
 		s.log.Warn("failed to store variant", "error", err)
 		// Continue even if caching fails
@@ -508,11 +628,28 @@ func (s *Service) processAndStore(
 	}, nil
 }
 
+// HTTP status codes for error responses.
+const (
+	httpStatusBadGateway           = 502
+	httpStatusInternalError        = 500
+	httpStatusUnsupportedMediaType = 415
+)
+
 // isNegativeCacheable returns true if the error should be cached.
 func isNegativeCacheable(err error) bool {
 	return errors.Is(err, httpfetcher.ErrUpstreamError)
 }
 
+// extractStatusCode extracts HTTP status code from error message.
+func extractStatusCode(err error) int {
+	// Default to 502 Bad Gateway for upstream errors
+	if errors.Is(err, httpfetcher.ErrUpstreamError) {
+		return httpStatusBadGateway
+	}
+
+	return httpStatusInternalError
+}
+
 // etagHashLength is the number of hash characters to use for ETags.
 const etagHashLength = 16
 

internal/imgcache/service_internal_test.go

@@ -2,6 +2,7 @@ package imgcache
 
 import (
 	"context"
+	"encoding/json"
 	"io"
 	"os"
 	"testing"
@@ -9,15 +10,17 @@ import (
 
 	"sneak.berlin/go/pixa/internal/imagerequest"
 	"sneak.berlin/go/pixa/internal/magic"
+	"sneak.berlin/go/pixa/internal/signature"
 )
 
 // Test data literals used repeatedly in this file (goconst).
 const (
 	testPathPhoto  = "/images/photo.jpg"
 	testPathUpload = "/uploads/image.jpg"
+	testSigningKey = "test-signing-key-12345"
 )
 
-func TestService_Get_GoodHost(t *testing.T) {
+func TestService_Get_AllowlistedHost(t *testing.T) {
 	t.Parallel()
 
 	svc, fixtures := SetupTestService(t)
@@ -58,12 +61,36 @@ func TestService_Get_GoodHost(t *testing.T) {
 	}
 }
 
-// TestService_Get_OtherHost checks that the image service serves a second host
+func TestService_Get_NonAllowlistedHost_NoSignature(t *testing.T) {
-// as it serves GoodHost.
-func TestService_Get_OtherHost(t *testing.T) {
 	t.Parallel()
 
-	svc, fixtures := SetupTestService(t)
+	svc, fixtures := SetupTestService(t, WithSigningKey("test-key"))
+
+	req := &imagerequest.Request{
+		Source: imagerequest.Source{
+			Host: fixtures.OtherHost,
+			Path: testPathUpload,
+		},
+		Transform: imagerequest.Transform{
+			Size:    imagerequest.Size{Width: 50, Height: 50},
+			Format:  imagerequest.FormatJPEG,
+			Quality: 85,
+			FitMode: imagerequest.FitCover,
+		},
+	}
+
+	// Should fail validation - not allowlisted and no signature
+	err := svc.ValidateRequest(req)
+	if err == nil {
+		t.Error("ValidateRequest() expected error for non-allowlisted host without signature")
+	}
+}
+
+func TestService_Get_NonAllowlistedHost_ValidSignature(t *testing.T) {
+	t.Parallel()
+
+	signingKey := testSigningKey
+	svc, fixtures := SetupTestService(t, WithSigningKey(signingKey))
 	ctx := context.Background()
 
 	req := &imagerequest.Request{
@@ -79,6 +106,17 @@ func TestService_Get_OtherHost(t *testing.T) {
 		},
 	}
 
+	// Generate a valid signature
+	signer := signature.New(signingKey)
+	req.Expires = time.Now().Add(time.Hour)
+	req.Signature = signer.Sign(req)
+
+	// Should pass validation
+	err := svc.ValidateRequest(req)
+	if err != nil {
+		t.Errorf("ValidateRequest() error = %v", err)
+	}
+
 	// Should fetch successfully
 	resp, err := svc.Get(ctx, req)
 	if err != nil {
@@ -97,6 +135,244 @@ func TestService_Get_OtherHost(t *testing.T) {
 	}
 }
 
+func TestService_Get_NonAllowlistedHost_ExpiredSignature(t *testing.T) {
+	t.Parallel()
+
+	signingKey := testSigningKey
+	svc, fixtures := SetupTestService(t, WithSigningKey(signingKey))
+
+	req := &imagerequest.Request{
+		Source: imagerequest.Source{
+			Host: fixtures.OtherHost,
+			Path: testPathUpload,
+		},
+		Transform: imagerequest.Transform{
+			Size:    imagerequest.Size{Width: 50, Height: 50},
+			Format:  imagerequest.FormatJPEG,
+			Quality: 85,
+			FitMode: imagerequest.FitCover,
+		},
+	}
+
+	// Generate an expired signature
+	signer := signature.New(signingKey)
+	req.Expires = time.Now().Add(-time.Hour) // Already expired
+	req.Signature = signer.Sign(req)
+
+	// Should fail validation
+	err := svc.ValidateRequest(req)
+	if err == nil {
+		t.Error("ValidateRequest() expected error for expired signature")
+	}
+}
+
+func TestService_Get_NonAllowlistedHost_InvalidSignature(t *testing.T) {
+	t.Parallel()
+
+	signingKey := testSigningKey
+	svc, fixtures := SetupTestService(t, WithSigningKey(signingKey))
+
+	req := &imagerequest.Request{
+		Source: imagerequest.Source{
+			Host: fixtures.OtherHost,
+			Path: testPathUpload,
+		},
+		Transform: imagerequest.Transform{
+			Size:    imagerequest.Size{Width: 50, Height: 50},
+			Format:  imagerequest.FormatJPEG,
+			Quality: 85,
+			FitMode: imagerequest.FitCover,
+		},
+	}
+
+	// Set an invalid signature
+	req.Signature = "invalid-signature"
+	req.Expires = time.Now().Add(time.Hour)
+
+	// Should fail validation
+	err := svc.ValidateRequest(req)
+	if err == nil {
+		t.Error("ValidateRequest() expected error for invalid signature")
+	}
+}
+
+// TestService_ValidateRequest_SignatureExactHostMatch verifies that
+// ValidateRequest enforces exact host matching for signatures. A
+// signature for one host must not verify for a different host, even
+// if they share a domain suffix.
+func TestService_ValidateRequest_SignatureExactHostMatch(t *testing.T) {
+	t.Parallel()
+
+	signingKey := "test-signing-key-must-be-32-chars"
+	svc, _ := SetupTestService(t,
+		WithSigningKey(signingKey),
+		WithNoAllowlist(),
+	)
+
+	signer := signature.New(signingKey)
+
+	// Sign a request for "cdn.example.com"
+	signedReq := &imagerequest.Request{
+		Source: imagerequest.Source{
+			Host: testHostCDN,
+			Path: testPathCat,
+		},
+		Transform: imagerequest.Transform{
+			Size:    imagerequest.Size{Width: 50, Height: 50},
+			Format:  imagerequest.FormatJPEG,
+			Quality: 85,
+			FitMode: imagerequest.FitCover,
+		},
+		Expires: time.Now().Add(time.Hour),
+	}
+	signedReq.Signature = signer.Sign(signedReq)
+
+	// The original request should pass validation
+	t.Run("exact host passes", func(t *testing.T) {
+		t.Parallel()
+
+		err := svc.ValidateRequest(signedReq)
+		if err != nil {
+			t.Errorf("ValidateRequest() exact host failed: %v", err)
+		}
+	})
+
+	// Try to reuse the signature with different hosts
+	tests := []struct {
+		name string
+		host string
+	}{
+		{"parent domain", testHostExample},
+		{"sibling subdomain", "images.example.com"},
+		{"deeper subdomain", "a.cdn.example.com"},
+		{"evil suffix domain", "cdn.example.com.evil.com"},
+		{"prefixed host", "evilcdn.example.com"},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name+" rejected", func(t *testing.T) {
+			t.Parallel()
+
+			req := &imagerequest.Request{
+				Source: imagerequest.Source{
+					Host:  tt.host,
+					Path:  signedReq.Source.Path,
+					Query: signedReq.Source.Query,
+				},
+				Transform: signedReq.Transform,
+				Expires:   signedReq.Expires,
+				Signature: signedReq.Signature,
+			}
+
+			err := svc.ValidateRequest(req)
+			if err == nil {
+				t.Errorf(
+					"ValidateRequest() should reject signature for host %q (signed for %q)",
+					tt.host, signedReq.Source.Host)
+			}
+		})
+	}
+}
+
+// TestService_ValidateRequest_HostWithPortNeedsSignature verifies that a host
+// followed by a port, or by a colon with no port, matches no allowlist entry,
+// so that a request for it without a signature is refused, while the same
+// host with no port is allowlisted.
+func TestService_ValidateRequest_HostWithPortNeedsSignature(t *testing.T) {
+	t.Parallel()
+
+	svc, _ := SetupTestService(t,
+		WithSigningKey(testSigningKey),
+		WithAllowlist(testHostCDN, "203.0.113.5", "2001:db8::1"),
+	)
+
+	tests := []struct {
+		host        string
+		allowlisted bool
+	}{
+		{testHostCDN, true},
+		{"cdn.example.com:8443", false},
+		{"cdn.example.com:443", false},
+		{"cdn.example.com:", false},
+		{"203.0.113.5", true},
+		{"203.0.113.5:8080", false},
+		{"[2001:db8::1]", true},
+		{"[2001:db8::1]:8443", false},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.host, func(t *testing.T) {
+			t.Parallel()
+
+			req := &imagerequest.Request{
+				Source: imagerequest.Source{
+					Host: tt.host,
+					Path: testPathCat,
+				},
+				Transform: imagerequest.Transform{
+					Size:    imagerequest.Size{Width: 50, Height: 50},
+					Format:  imagerequest.FormatJPEG,
+					Quality: 85,
+					FitMode: imagerequest.FitCover,
+				},
+			}
+
+			err := svc.ValidateRequest(req)
+			t.Logf("ValidateRequest() for host %q with no signature: %v",
+				tt.host, err)
+
+			if allowlisted := err == nil; allowlisted != tt.allowlisted {
+				t.Errorf("host %q served without a signature = %v, want %v",
+					tt.host, allowlisted, tt.allowlisted)
+			}
+		})
+	}
+}
+
+// TestService_ValidateRequest_UnbracketedIPv6PortNeedsSignature verifies that
+// an allowlisted IPv6 address written without brackets and followed by a
+// port, or by a colon with no port, needs a signature. Go's URL parser, which
+// the fetcher uses, reads 2001:db8::1:8443 as the host 2001:db8::1 with the
+// port 8443.
+func TestService_ValidateRequest_UnbracketedIPv6PortNeedsSignature(
+	t *testing.T,
+) {
+	t.Parallel()
+
+	svc, _ := SetupTestService(t,
+		WithSigningKey(testSigningKey),
+		WithAllowlist("2001:db8::1"),
+	)
+
+	for _, host := range []string{"2001:db8::1:8443", "2001:db8::1:"} {
+		t.Run(host, func(t *testing.T) {
+			t.Parallel()
+
+			req := &imagerequest.Request{
+				Source: imagerequest.Source{
+					Host: host,
+					Path: testPathCat,
+				},
+				Transform: imagerequest.Transform{
+					Size:    imagerequest.Size{Width: 50, Height: 50},
+					Format:  imagerequest.FormatJPEG,
+					Quality: 85,
+					FitMode: imagerequest.FitCover,
+				},
+			}
+
+			err := svc.ValidateRequest(req)
+			t.Logf("ValidateRequest() for host %q with no signature: %v",
+				host, err)
+
+			if err == nil {
+				t.Errorf("host %q served without a signature, want refused",
+					host)
+			}
+		})
+	}
+}
+
 func TestService_Get_InvalidFile(t *testing.T) {
 	t.Parallel()
 
@@ -274,15 +550,21 @@ func TestService_Get_Caching(t *testing.T) {
 
 // TestService_Get_ServesSourceAndVariantCachedTwoDaysAgo caches a source and
 // a variant of it and sets back two days every time pixa stored for them.
-// For the source: last_accessed_at in source_content, and the access and
+// For the source: fetched_at in source_metadata, fetched_at and
-// modification times of its file. For the variant: last_accessed_at in
+// last_accessed_at in source_content, the fetched_at in its metadata file,
-// variant_content, and the access and modification times of its file. It
+// and the access and modification times of its file and of its metadata
-// then checks that the variant is still served from the cache and that a new
+// file. For the variant: created_at and last_accessed_at in variant_content,
-// variant is made from the cached source without fetching it again.
+// the created_at in its .meta file, and the access and modification times of
+// its file and of its .meta file. It then checks that the variant is still
+// served from the cache and that a new variant is made from the cached source
+// without fetching it again.
 //
 // Left at the present: the files' status change and birth times, which a
 // program cannot set, and the times of the directories holding the files,
-// which other entries share.
+// which other entries share. The source's stored response headers are empty,
+// as the mock fetcher sends none. expires_at, etag and last_modified in
+// source_metadata and in the metadata file are never written, nor are the
+// request_cache and output_content tables.
 func TestService_Get_ServesSourceAndVariantCachedTwoDaysAgo(t *testing.T) {
 	t.Parallel()
 
@@ -320,7 +602,7 @@ func TestService_Get_ServesSourceAndVariantCachedTwoDaysAgo(t *testing.T) {
 
 	_ = resp.Content.Close()
 
-	fetches := readCounters(svc.cache).upstreamFetchCount
+	fetches := readCacheStatsCounters(t, svc.cache).upstreamFetchCount
 	if fetches != 1 {
 		t.Errorf("upstream fetches = %d, want 1", fetches)
 	}
@@ -333,14 +615,39 @@ func backdateSource(
 ) {
 	t.Helper()
 
-	key := req.Source.Key()
+	pathHash := HashPath(req.Source.Path + "?" + req.Source.Query)
-	setSourceLastAccessed(t, cache, key, when)
 
-	path := cache.srcContent.keyToPath(key)
+	meta, err := cache.srcMetadata.Load(req.Source.Host, pathHash)
-
-	err := os.Chtimes(path, when, when)
 	if err != nil {
-		t.Fatalf("setting back the times of %s: %v", path, err)
+		t.Fatalf("reading the source's metadata file: %v", err)
+	}
+
+	meta.FetchedAt = when.Unix()
+
+	err = cache.srcMetadata.Store(req.Source.Host, pathHash, meta)
+	if err != nil {
+		t.Fatalf("writing the source's metadata file: %v", err)
+	}
+
+	dbTime := when.UTC().Format(sqliteTimestampFormat)
+
+	updateOneRow(t, cache, `
+		UPDATE source_metadata SET fetched_at = ?
+		WHERE source_host = ? AND source_path = ? AND source_query = ?
+	`, dbTime, req.Source.Host, req.Source.Path, req.Source.Query)
+	updateOneRow(t, cache,
+		`UPDATE source_content SET fetched_at = ? WHERE content_hash = ?`,
+		dbTime, meta.ContentHash)
+	setSourceLastAccessed(t, cache, ContentHash(meta.ContentHash), when)
+
+	for _, path := range []string{
+		cache.srcContent.hashToPath(ContentHash(meta.ContentHash)),
+		cache.srcMetadata.metaPath(req.Source.Host, pathHash),
+	} {
+		err = os.Chtimes(path, when, when)
+		if err != nil {
+			t.Fatalf("setting back the times of %s: %v", path, err)
+		}
 	}
 }
 
@@ -353,13 +660,57 @@ func backdateVariant(
 	t.Helper()
 
 	key := imagerequest.CacheKey(req)
+
+	content, size, contentType, err := cache.variants.LoadWithMeta(key)
+	if err != nil || contentType == "" {
+		t.Fatalf("reading the variant's .meta file: content type %q, error %v",
+			contentType, err)
+	}
+
+	_ = content.Close()
+
+	meta, err := json.Marshal(VariantMeta{
+		ContentType: contentType,
+		Size:        size,
+		CreatedAt:   when.Unix(),
+	})
+	if err != nil {
+		t.Fatalf("encoding the variant's .meta file: %v", err)
+	}
+
 	path := cache.variants.keyToPath(key)
 
+	err = os.WriteFile(path+variantMetaSuffix, meta, StorageFilePerm)
+	if err != nil {
+		t.Fatalf("writing the variant's .meta file: %v", err)
+	}
+
+	updateOneRow(t, cache,
+		`UPDATE variant_content SET created_at = ? WHERE cache_key = ?`,
+		when.UTC().Format(sqliteTimestampFormat), string(key))
 	setVariantLastAccessed(t, cache, key, when)
 
-	err := os.Chtimes(path, when, when)
+	for _, file := range []string{path, path + variantMetaSuffix} {
+		err = os.Chtimes(file, when, when)
+		if err != nil {
+			t.Fatalf("setting back the times of %s: %v", file, err)
+		}
+	}
+}
+
+// updateOneRow runs the UPDATE statement query and fails the test unless it
+// changed exactly one row.
+func updateOneRow(t *testing.T, cache *Cache, query string, args ...any) {
+	t.Helper()
+
+	result, err := cache.db.ExecContext(t.Context(), query, args...)
 	if err != nil {
-		t.Fatalf("setting back the times of %s: %v", path, err)
+		t.Fatalf("%s: %v", query, err)
+	}
+
+	rows, err := result.RowsAffected()
+	if err != nil || rows != 1 {
+		t.Fatalf("%s: rows = %d, error = %v, want 1 row", query, rows, err)
 	}
 }
 
@@ -417,6 +768,33 @@ func TestService_Get_DifferentSizes(t *testing.T) {
 	}
 }
 
+func TestService_ValidateRequest_NoSigningKey(t *testing.T) {
+	t.Parallel()
+
+	// Service with no signing key - all non-allowlisted requests should fail
+	svc, fixtures := SetupTestService(t, WithNoAllowlist())
+
+	req := &imagerequest.Request{
+		Source: imagerequest.Source{
+			Host: fixtures.OtherHost,
+			Path: testPathUpload,
+		},
+		Transform: imagerequest.Transform{
+			Size:    imagerequest.Size{Width: 50, Height: 50},
+			Format:  imagerequest.FormatJPEG,
+			Quality: 85,
+			FitMode: imagerequest.FitCover,
+		},
+	}
+
+	err := svc.ValidateRequest(req)
+	if err == nil {
+		t.Error(
+			"ValidateRequest() expected error when no signing key and host not allowlisted",
+		)
+	}
+}
+
 func TestService_Get_ContextCancellation(t *testing.T) {
 	t.Parallel()
 

internal/imgcache/source_content_internal_test.go

@@ -1,141 +0,0 @@
-package imgcache
-
-import (
-	"bytes"
-	"io"
-	"os"
-	"path/filepath"
-	"testing"
-
-	"sneak.berlin/go/pixa/internal/imagerequest"
-)
-
-// TestStoreSource_StoresIdenticalBytesOfTwoURLsTwice stores the same bytes
-// for two source URLs and checks that each URL has a source_content row of
-// its own and a file of its own, named by its source key under
-// cache/sources/<ab>/<cd>/.
-func TestStoreSource_StoresIdenticalBytesOfTwoURLsTwice(t *testing.T) {
-	t.Parallel()
-
-	cache, stateDir := newEvictionTestCache(t, 1<<30)
-	content := bytes.Repeat([]byte{0x5A}, 400)
-
-	keys := map[string]imagerequest.SourceKey{
-		"/a.jpg": storeEvictionTestSource(t, cache, testHostCDN, "/a.jpg", content),
-		"/b.jpg": storeEvictionTestSource(t, cache, testHostCDN, "/b.jpg", content),
-	}
-	t.Logf("source keys: %v", keys)
-
-	if keys["/a.jpg"] == keys["/b.jpg"] {
-		t.Fatalf("both source URLs got the key %s", keys["/a.jpg"])
-	}
-
-	for sourcePath, key := range keys {
-		rows := countRows(t, cache,
-			`SELECT COUNT(*) FROM source_content WHERE source_key = ?`, string(key))
-		if rows != 1 {
-			t.Errorf("%s has %d source_content rows, want 1", sourcePath, rows)
-		}
-
-		name := string(key)
-		file := filepath.Join(stateDir, "cache", "sources", name[0:2], name[2:4], name)
-
-		stored, err := os.ReadFile(file) //nolint:gosec // path in the test's state dir
-		if err != nil || !bytes.Equal(stored, content) {
-			t.Errorf("file of %s at %s holds %d bytes (error %v), want the %d stored",
-				sourcePath, file, len(stored), err, len(content))
-		}
-	}
-
-	usage, err := cache.UsageBytes(t.Context())
-	if err != nil {
-		t.Fatalf("UsageBytes() error = %v", err)
-	}
-
-	if usage != 800 {
-		t.Errorf("UsageBytes() = %d, want 800, the 400 bytes once for each URL", usage)
-	}
-}
-
-// TestEvictSourceBlob_LeavesSourceWithIdenticalBytesServed stores the same
-// bytes for two source URLs and evicts the first. Its row and file must be
-// gone, and the second must still be found and read from the cache.
-func TestEvictSourceBlob_LeavesSourceWithIdenticalBytesServed(t *testing.T) {
-	t.Parallel()
-
-	cache, _ := newEvictionTestCache(t, 1<<30)
-	content := bytes.Repeat([]byte{0x5B}, 400)
-
-	evicted := storeEvictionTestSource(t, cache, testHostCDN, "/evicted.jpg", content)
-	storeEvictionTestSource(t, cache, testHostCDN, "/kept.jpg", content)
-
-	err := cache.evictSourceBlob(t.Context(), evicted)
-	if err != nil {
-		t.Fatalf("evictSourceBlob() error = %v", err)
-	}
-
-	if cache.srcContent.Exists(evicted) {
-		t.Error("the evicted source's file is still on disk")
-	}
-
-	if n := countRows(t, cache,
-		`SELECT COUNT(*) FROM source_content WHERE source_key = ?`, string(evicted),
-	); n != 0 {
-		t.Errorf("the evicted source still has %d source_content rows", n)
-	}
-
-	kept, err := cache.LookupSource(t.Context(), &imagerequest.Request{
-		Source: imagerequest.Source{Host: testHostCDN, Path: "/kept.jpg"},
-	})
-	t.Logf("LookupSource() of the kept source = %q, error %v", kept, err)
-
-	if err != nil || kept == "" {
-		t.Fatalf("LookupSource() of the kept source = %q, %v; want it cached", kept, err)
-	}
-
-	reader, _, err := cache.GetSourceContent(kept)
-	if err != nil {
-		t.Fatalf("GetSourceContent() of the kept source error = %v", err)
-	}
-
-	defer func() { _ = reader.Close() }()
-
-	served, err := io.ReadAll(reader)
-	if err != nil {
-		t.Fatalf("reading the kept source: %v", err)
-	}
-
-	if !bytes.Equal(served, content) {
-		t.Errorf("the kept source reads as %d bytes, want the %d stored",
-			len(served), len(content))
-	}
-}
-
-// TestStoreSource_FailedRowWriteLeavesNoFile drops source_content, so that
-// StoreSource cannot write the source's row, and checks that the store fails
-// and leaves no file under the source's key.
-func TestStoreSource_FailedRowWriteLeavesNoFile(t *testing.T) {
-	t.Parallel()
-
-	cache, _ := newEvictionTestCache(t, 1<<30)
-
-	_, err := cache.db.ExecContext(t.Context(), `DROP TABLE source_content`)
-	if err != nil {
-		t.Fatalf("failed to drop source_content: %v", err)
-	}
-
-	req := &imagerequest.Request{
-		Source: imagerequest.Source{Host: testHostCDN, Path: testPathCat},
-	}
-	key, err := cache.StoreSource(t.Context(), req,
-		bytes.NewReader([]byte("source bytes")))
-	t.Logf("StoreSource() = %q, error = %v", key, err)
-
-	if err == nil {
-		t.Error("StoreSource() error = nil, want the failed row write")
-	}
-
-	if cache.srcContent.Exists(req.Source.Key()) {
-		t.Error("the source's file is on disk although its row was not written")
-	}
-}

internal/imgcache/stats_internal_test.go

@@ -3,18 +3,282 @@ package imgcache
 import (
 	"bytes"
 	"context"
+	"database/sql"
 	"image/color"
 	"io"
 	"io/fs"
+	"log/slog"
+	"math"
+	"strings"
 	"testing"
+	"time"
 
+	"sneak.berlin/go/pixa/internal/database"
 	"sneak.berlin/go/pixa/internal/httpfetcher"
 	"sneak.berlin/go/pixa/internal/imagerequest"
 )
 
+func setupStatsTestDB(t *testing.T) *sql.DB {
+	t.Helper()
+
+	db, err := sql.Open("sqlite", ":memory:")
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	err = database.ApplyMigrations(context.Background(), db, nil)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	t.Cleanup(func() { _ = db.Close() })
+
+	return db
+}
+
+func TestStats_HitRateIsRatio(t *testing.T) {
+	t.Parallel()
+
+	db := setupStatsTestDB(t)
+	dir := t.TempDir()
+
+	cache, err := NewCache(db, CacheConfig{
+		StateDir:    dir,
+		NegativeTTL: 5 * time.Minute,
+	})
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	ctx := context.Background()
+
+	// Set some hit/miss counts and a transform_count
+	_, err = db.ExecContext(ctx, `
+		UPDATE cache_stats
+		SET hit_count = 75, miss_count = 25, transform_count = 9999
+		WHERE id = 1
+	`)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	stats, err := cache.Stats(ctx)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	if stats.HitCount != 75 {
+		t.Errorf("HitCount = %d, want 75", stats.HitCount)
+	}
+
+	if stats.MissCount != 25 {
+		t.Errorf("MissCount = %d, want 25", stats.MissCount)
+	}
+
+	// HitRate should be 0.75, NOT 9999 (transform_count)
+	expectedRate := 0.75
+	if math.Abs(stats.HitRate-expectedRate) > 0.001 {
+		t.Errorf("HitRate = %f, want %f (was it scanning transform_count?)",
+			stats.HitRate, expectedRate)
+	}
+}
+
+func TestStats_ZeroCounts(t *testing.T) {
+	t.Parallel()
+
+	db := setupStatsTestDB(t)
+	dir := t.TempDir()
+
+	cache, err := NewCache(db, CacheConfig{
+		StateDir:    dir,
+		NegativeTTL: 5 * time.Minute,
+	})
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	stats, err := cache.Stats(context.Background())
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	// With zero hits and misses, HitRate should be 0, not some garbage value
+	if stats.HitRate != 0.0 {
+		t.Errorf("HitRate = %f, want 0.0 for zero counts", stats.HitRate)
+	}
+}
+
+// TestStats_LogsFailedCountQueries verifies that a failed item count query
+// and a failed size query are each logged at warn and Stats still succeeds.
+func TestStats_LogsFailedCountQueries(t *testing.T) {
+	t.Parallel()
+
+	db := setupStatsTestDB(t)
+
+	var logBuf bytes.Buffer
+
+	cache, err := NewCache(db, CacheConfig{
+		StateDir:    t.TempDir(),
+		NegativeTTL: 5 * time.Minute,
+		Logger:      slog.New(slog.NewJSONHandler(&logBuf, nil)),
+	})
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	_, err = db.ExecContext(t.Context(),
+		`DROP TABLE source_content; DROP TABLE variant_content`)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	_, err = cache.Stats(t.Context())
+	if err != nil {
+		t.Fatalf("Stats() error = %v, want nil", err)
+	}
+
+	for _, msg := range []string{
+		"failed to count cache items for stats",
+		"failed to sum cache size for stats",
+	} {
+		want := `"level":"WARN","msg":"` + msg + `"`
+		if !strings.Contains(logBuf.String(), want) {
+			t.Errorf("log missing %s; got %q", want, logBuf.String())
+		}
+	}
+}
+
+// TestIncrementStats_LogsFailedUpdates verifies that a failed hit or miss
+// count update and a failed upstream fetch count update are each logged at
+// warn.
+func TestIncrementStats_LogsFailedUpdates(t *testing.T) {
+	t.Parallel()
+
+	db := setupStatsTestDB(t)
+
+	var logBuf bytes.Buffer
+
+	cache, err := NewCache(db, CacheConfig{
+		StateDir:    t.TempDir(),
+		NegativeTTL: 5 * time.Minute,
+		Logger:      slog.New(slog.NewJSONHandler(&logBuf, nil)),
+	})
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	_, err = db.ExecContext(t.Context(), `DROP TABLE cache_stats`)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	cache.IncrementStats(t.Context(), false, 1024)
+
+	for _, msg := range []string{
+		"failed to count cache hit or miss",
+		"failed to count upstream fetch",
+	} {
+		want := `"level":"WARN","msg":"` + msg + `"`
+		if !strings.Contains(logBuf.String(), want) {
+			t.Errorf("log missing %s; got %q", want, logBuf.String())
+		}
+	}
+}
+
+// TestStats_TotalsCountSourcesAndVariants verifies that TotalItems and
+// TotalSizeBytes cover the stored source images and processed variants.
+func TestStats_TotalsCountSourcesAndVariants(t *testing.T) {
+	t.Parallel()
+
+	cache, _ := newEvictionTestCache(t, 1<<30)
+
+	storeEvictionTestSource(t, cache, testHostCDN, testPathCat,
+		bytes.Repeat([]byte{0xAA}, 1000))
+	storeEvictionTestVariant(t, cache, testVariantKeyOne,
+		bytes.Repeat([]byte{0xAB}, 500))
+	storeEvictionTestVariant(t, cache, testVariantKeyTwo,
+		bytes.Repeat([]byte{0xAC}, 250))
+
+	stats, err := cache.Stats(t.Context())
+	if err != nil {
+		t.Fatalf("Stats() error = %v", err)
+	}
+
+	if stats.TotalItems != 3 {
+		t.Errorf("TotalItems = %d, want 3 (1 source, 2 variants)", stats.TotalItems)
+	}
+
+	if stats.TotalSizeBytes != 1750 {
+		t.Errorf("TotalSizeBytes = %d, want 1750 (1000+500+250)",
+			stats.TotalSizeBytes)
+	}
+}
+
+// TestStats_DisabledCacheReportsNoItems verifies that a disabled disk cache
+// reports no items and no size, even when its database still holds the
+// rows of an earlier run with the disk cache enabled.
+func TestStats_DisabledCacheReportsNoItems(t *testing.T) {
+	t.Parallel()
+
+	enabled, _ := newEvictionTestCache(t, 1<<30)
+
+	storeEvictionTestSource(t, enabled, testHostCDN, testPathCat,
+		bytes.Repeat([]byte{0xAA}, 1000))
+	storeEvictionTestVariant(t, enabled, testVariantKeyOne,
+		bytes.Repeat([]byte{0xAB}, 500))
+
+	disabled, err := NewCache(enabled.db, CacheConfig{
+		StateDir:         t.TempDir(),
+		NegativeTTL:      5 * time.Minute,
+		DisableDiskCache: true,
+	})
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	stats, err := disabled.Stats(t.Context())
+	if err != nil {
+		t.Fatalf("Stats() error = %v", err)
+	}
+
+	if stats.TotalItems != 0 || stats.TotalSizeBytes != 0 {
+		t.Errorf("TotalItems = %d, TotalSizeBytes = %d, want 0 and 0",
+			stats.TotalItems, stats.TotalSizeBytes)
+	}
+}
+
+// cacheStatsCounters holds the counters of the cache_stats row, in column
+// order.
+type cacheStatsCounters struct {
+	hitCount           int64
+	missCount          int64
+	upstreamFetchCount int64
+	upstreamFetchBytes int64
+	transformCount     int64
+}
+
+// readCacheStatsCounters reads the counters of the cache_stats row.
+func readCacheStatsCounters(t *testing.T, cache *Cache) cacheStatsCounters {
+	t.Helper()
+
+	var got cacheStatsCounters
+
+	err := cache.db.QueryRowContext(t.Context(), `
+		SELECT hit_count, miss_count, upstream_fetch_count,
+		       upstream_fetch_bytes, transform_count
+		FROM cache_stats WHERE id = 1
+	`).Scan(&got.hitCount, &got.missCount, &got.upstreamFetchCount,
+		&got.upstreamFetchBytes, &got.transformCount)
+	if err != nil {
+		t.Fatalf("failed to read cache_stats: %v", err)
+	}
+
+	return got
+}
+
 // TestService_Get_CountsStats walks Get through a miss that fetches the
 // source, a hit, a miss that reuses the cached source, and two misses whose
-// source cannot be used, checking every counter of the cache after each.
+// source cannot be used, checking every cache_stats counter after each.
 func TestService_Get_CountsStats(t *testing.T) {
 	t.Parallel()
 
@@ -41,18 +305,18 @@ func TestService_Get_CountsStats(t *testing.T) {
 		path    string
 		size    int
 		wantErr bool
-		want    counterValues
+		want    cacheStatsCounters
 	}{
 		{"miss that fetches the source", testPathPhoto, 50, false,
-			counterValues{0, 1, 1, photoBytes, 1}},
+			cacheStatsCounters{0, 1, 1, photoBytes, 1}},
 		{"hit", testPathPhoto, 50, false,
-			counterValues{1, 1, 1, photoBytes, 1}},
+			cacheStatsCounters{1, 1, 1, photoBytes, 1}},
 		{"miss that reuses the cached source", testPathPhoto, 25, false,
-			counterValues{1, 2, 1, photoBytes, 2}},
+			cacheStatsCounters{1, 2, 1, photoBytes, 2}},
 		{"miss whose source fails the magic byte check", "/images/fake.jpg", 50, true,
-			counterValues{1, 3, 2, photoBytes + fakeBytes, 2}},
+			cacheStatsCounters{1, 3, 2, photoBytes + fakeBytes, 2}},
 		{"miss whose source is not found", "/images/nonexistent.jpg", 50, true,
-			counterValues{1, 4, 2, photoBytes + fakeBytes, 2}},
+			cacheStatsCounters{1, 4, 2, photoBytes + fakeBytes, 2}},
 	}
 
 	for _, step := range steps {
@@ -76,7 +340,7 @@ func TestService_Get_CountsStats(t *testing.T) {
 			_ = resp.Content.Close()
 		}
 
-		got := readCounters(svc.cache)
+		got := readCacheStatsCounters(t, svc.cache)
 		if got != step.want {
 			t.Fatalf("after the %s: counters = %+v, want %+v", step.name, got, step.want)
 		}
@@ -93,7 +357,7 @@ type fakeUpstream struct {
 }
 
 func (u *fakeUpstream) Fetch(
-	context.Context, imagerequest.Source,
+	context.Context, string,
 ) (*httpfetcher.FetchResult, error) {
 	return &httpfetcher.FetchResult{
 		Content:       io.NopCloser(u),
@@ -114,7 +378,7 @@ func (u *fakeUpstream) Read(p []byte) (int, error) {
 	return 0, u.err
 }
 
-// TestService_Get_CountsInterruptedMisses checks every counter of the cache
+// TestService_Get_CountsInterruptedMisses checks every cache_stats counter
 // after a miss whose request context ends during or after the upstream
 // fetch, and after a miss whose upstream body is over the size limit.
 func TestService_Get_CountsInterruptedMisses(t *testing.T) {
@@ -130,15 +394,15 @@ func TestService_Get_CountsInterruptedMisses(t *testing.T) {
 		cancel  bool  // whether the body then ends the request context
 		readErr error // what the body then returns
 		wantErr bool
-		want    counterValues
+		want    cacheStatsCounters
 	}{
 		{"request context ends during the fetch", half, true, context.Canceled, true,
-			counterValues{0, 1, 1, int64(half), 0}},
+			cacheStatsCounters{0, 1, 1, int64(half), 0}},
 		{"request context ends after the fetch", len(photo), true, io.EOF, false,
-			counterValues{0, 1, 1, int64(len(photo)), 1}},
+			cacheStatsCounters{0, 1, 1, int64(len(photo)), 1}},
 		{"upstream body over the size limit", half, false,
 			httpfetcher.ErrResponseTooLarge, true,
-			counterValues{0, 1, 1, int64(half), 0}},
+			cacheStatsCounters{0, 1, 1, int64(half), 0}},
 	}
 
 	for _, tc := range tests {
@@ -182,7 +446,7 @@ func TestService_Get_CountsInterruptedMisses(t *testing.T) {
 				_ = resp.Content.Close()
 			}
 
-			got := readCounters(svc.cache)
+			got := readCacheStatsCounters(t, svc.cache)
 			if got != tc.want {
 				t.Errorf("counters = %+v, want %+v", got, tc.want)
 			}
@@ -190,9 +454,9 @@ func TestService_Get_CountsInterruptedMisses(t *testing.T) {
 	}
 }
 
-// TestService_Get_CountsHitAfterRequestEnds checks every counter of the
+// TestService_Get_CountsHitAfterRequestEnds checks every cache_stats counter
-// cache after a hit served with a request context that has already ended:
+// after a hit served with a request context that has already ended: only
-// only the hit count moves.
+// the hit count moves.
 func TestService_Get_CountsHitAfterRequestEnds(t *testing.T) {
 	t.Parallel()
 
@@ -219,7 +483,7 @@ func TestService_Get_CountsHitAfterRequestEnds(t *testing.T) {
 
 	_ = resp.Content.Close()
 
-	want := readCounters(svc.cache)
+	want := readCacheStatsCounters(t, svc.cache)
 	want.hitCount++
 
 	ctx, cancel := context.WithCancel(t.Context())
@@ -236,7 +500,7 @@ func TestService_Get_CountsHitAfterRequestEnds(t *testing.T) {
 		t.Fatalf("CacheStatus = %v, want %v", resp.CacheStatus, CacheHit)
 	}
 
-	got := readCounters(svc.cache)
+	got := readCacheStatsCounters(t, svc.cache)
 	if got != want {
 		t.Errorf("counters = %+v, want %+v", got, want)
 	}

internal/imgcache/storage.go

@@ -1,14 +1,18 @@
 package imgcache
 
 import (
+	"crypto/sha256"
+	"encoding/hex"
+	"encoding/json"
 	"errors"
 	"fmt"
 	"io"
+	"log/slog"
 	"os"
 	"path/filepath"
+	"time"
 
 	"sneak.berlin/go/pixa/internal/imagerequest"
-	"sneak.berlin/go/pixa/internal/magic"
 )
 
 // Storage constants.
@@ -26,8 +30,14 @@ var (
 	ErrNotFound = errors.New("content not found")
 )
 
-// ContentStorage handles source image file storage, one file per source
+// ContentHash is a SHA256 hash of file content (hex-encoded).
-// URL. Files are stored at: <basedir>/<ab>/<cd>/<source key>
+type ContentHash string
+
+// PathHash is a SHA256 hash of a URL path (hex-encoded).
+type PathHash string
+
+// ContentStorage handles content-addressable file storage.
+// Files are stored at: <basedir>/<ab>/<cd>/<abcdef...sha256>
 type ContentStorage struct {
 	baseDir string
 }
@@ -42,19 +52,38 @@ func NewContentStorage(baseDir string) (*ContentStorage, error) {
 	return &ContentStorage{baseDir: baseDir}, nil
 }
 
-// Store writes data with writeFileSynced to the path derived from key,
+// Store writes content to storage and returns its SHA256 hash.
-// replacing any file already there, and returns its size.
+// The content is read fully into memory to compute the hash before writing.
-func (s *ContentStorage) Store(
+func (s *ContentStorage) Store(r io.Reader) (ContentHash, int64, error) {
-	key imagerequest.SourceKey, data []byte,
+	// Read all content to compute hash
-) (int64, error) {
+	data, err := io.ReadAll(r)
-	path := s.keyToPath(key)
-
-	err := os.MkdirAll(filepath.Dir(path), StorageDirPerm)
 	if err != nil {
-		return 0, fmt.Errorf("failed to create directory: %w", err)
+		return "", 0, fmt.Errorf("failed to read content: %w", err)
 	}
 
-	err = writeFileSynced(path, data)
+	// Compute hash
+	h := sha256.Sum256(data)
+	hash := ContentHash(hex.EncodeToString(h[:]))
+	size := int64(len(data))
+
+	err = s.writeIfAbsent(hash, data)
+	if err != nil {
+		return "", 0, err
+	}
+
+	return hash, size, nil
+}
+
+// StoreHashed writes pre-hashed content to storage at the path derived
+// from hash, without recomputing it. Callers that already know the
+// hash before writing (e.g. because they must hold a hash-keyed lock
+// across the whole store operation) use this instead of Store. Like
+// Store, it is idempotent: a file of the content's size already at that
+// path is left untouched, and a file of another size is replaced.
+func (s *ContentStorage) StoreHashed(
+	hash ContentHash, data []byte,
+) (int64, error) {
+	err := s.writeIfAbsent(hash, data)
 	if err != nil {
 		return 0, err
 	}
@@ -62,13 +91,27 @@ func (s *ContentStorage) Store(
 	return int64(len(data)), nil
 }
 
-// LoadWithSize returns a reader and file size for the content with the given key.
+// Load returns a reader for the content with the given hash.
-func (s *ContentStorage) LoadWithSize(
+func (s *ContentStorage) Load(hash ContentHash) (io.ReadCloser, error) {
-	key imagerequest.SourceKey,
+	path := s.hashToPath(hash)
-) (io.ReadCloser, int64, error) {
-	path := s.keyToPath(key)
 
-	f, err := os.Open(path) //nolint:gosec // path derived from source key
+	f, err := os.Open(path) //nolint:gosec // path derived from content hash
+	if err != nil {
+		if os.IsNotExist(err) {
+			return nil, ErrNotFound
+		}
+
+		return nil, fmt.Errorf("failed to open content: %w", err)
+	}
+
+	return f, nil
+}
+
+// LoadWithSize returns a reader and file size for the content with the given hash.
+func (s *ContentStorage) LoadWithSize(hash ContentHash) (io.ReadCloser, int64, error) {
+	path := s.hashToPath(hash)
+
+	f, err := os.Open(path) //nolint:gosec // path derived from content hash
 	if err != nil {
 		if os.IsNotExist(err) {
 			return nil, 0, ErrNotFound
@@ -87,9 +130,9 @@ func (s *ContentStorage) LoadWithSize(
 	return f, stat.Size(), nil
 }
 
-// Delete removes content with the given key.
+// Delete removes content with the given hash.
-func (s *ContentStorage) Delete(key imagerequest.SourceKey) error {
+func (s *ContentStorage) Delete(hash ContentHash) error {
-	path := s.keyToPath(key)
+	path := s.hashToPath(hash)
 
 	err := os.Remove(path)
 	if err != nil && !os.IsNotExist(err) {
@@ -99,14 +142,34 @@ func (s *ContentStorage) Delete(key imagerequest.SourceKey) error {
 	return nil
 }
 
-// Exists checks if content with the given key exists.
+// Exists checks if content with the given hash exists.
-func (s *ContentStorage) Exists(key imagerequest.SourceKey) bool {
+func (s *ContentStorage) Exists(hash ContentHash) bool {
-	path := s.keyToPath(key)
+	path := s.hashToPath(hash)
 	_, err := os.Stat(path)
 
 	return err == nil
 }
 
+// writeIfAbsent writes data with writeFileSynced to the path derived
+// from hash, unless a file of the data's size is already there.
+func (s *ContentStorage) writeIfAbsent(hash ContentHash, data []byte) error {
+	// Build path: <basedir>/<ab>/<cd>/<hash>
+	path := s.hashToPath(hash)
+
+	// A file of another size there is damaged, and is replaced
+	info, statErr := os.Stat(path)
+	if statErr == nil && info.Size() == int64(len(data)) {
+		return nil
+	}
+
+	err := os.MkdirAll(filepath.Dir(path), StorageDirPerm)
+	if err != nil {
+		return fmt.Errorf("failed to create directory: %w", err)
+	}
+
+	return writeFileSynced(path, data)
+}
+
 // writeFileSynced writes data to path through a temp file in the same
 // directory: the temp file is synced to disk, renamed to path, and the
 // directory synced after, so that neither a concurrent reader nor a
@@ -181,117 +244,305 @@ func syncDirectory(dir string) error {
 	return nil
 }
 
-// keyToPath converts a key to a file path: <basedir>/<ab>/<cd>/<key>
+// hashToPath converts a hash to a file path: <basedir>/<ab>/<cd>/<hash>
-func (s *ContentStorage) keyToPath(key imagerequest.SourceKey) string {
+func (s *ContentStorage) hashToPath(hash ContentHash) string {
-	k := string(key)
+	h := string(hash)
-	if len(k) < MinHashLength {
+	if len(h) < MinHashLength {
-		return filepath.Clean(filepath.Join(s.baseDir, k))
+		return filepath.Clean(filepath.Join(s.baseDir, h))
 	}
 
-	return filepath.Clean(filepath.Join(s.baseDir, k[0:2], k[2:4], k))
+	return filepath.Clean(filepath.Join(s.baseDir, h[0:2], h[2:4], h))
+}
+
+// MetadataStorage handles JSON metadata file storage.
+// Files are stored at: <basedir>/<hostname>/<path_hash>.json
+type MetadataStorage struct {
+	baseDir string
+}
+
+// NewMetadataStorage creates a new metadata storage at the given base directory.
+func NewMetadataStorage(baseDir string) (*MetadataStorage, error) {
+	err := os.MkdirAll(baseDir, StorageDirPerm)
+	if err != nil {
+		return nil, fmt.Errorf("failed to create metadata directory: %w", err)
+	}
+
+	return &MetadataStorage{baseDir: baseDir}, nil
+}
+
+// SourceMetadata represents cached metadata about a source URL.
+//
+//nolint:tagliatelle // stored metadata format uses snake_case
+type SourceMetadata struct {
+	Host            string              `json:"host"`
+	Path            string              `json:"path"`
+	Query           string              `json:"query,omitempty"`
+	ContentHash     string              `json:"content_hash,omitempty"`
+	StatusCode      int                 `json:"status_code"`
+	ContentType     string              `json:"content_type,omitempty"`
+	ContentLength   int64               `json:"content_length,omitempty"`
+	ResponseHeaders map[string][]string `json:"response_headers,omitempty"`
+	FetchedAt       int64               `json:"fetched_at"`
+	FetchDurationMs int64               `json:"fetch_duration_ms,omitempty"`
+	ExpiresAt       int64               `json:"expires_at,omitempty"`
+	ETag            string              `json:"etag,omitempty"`
+	LastModified    string              `json:"last_modified,omitempty"`
+	RemoteAddr      string              `json:"remote_addr,omitempty"`
+}
+
+// Store writes metadata to storage.
+func (s *MetadataStorage) Store(
+	host string, pathHash PathHash, meta *SourceMetadata,
+) error {
+	path := s.metaPath(host, pathHash)
+
+	// Create directory structure
+	dir := filepath.Dir(path)
+
+	err := os.MkdirAll(dir, StorageDirPerm)
+	if err != nil {
+		return fmt.Errorf("failed to create directory: %w", err)
+	}
+
+	// Marshal to JSON
+	data, err := json.MarshalIndent(meta, "", "  ")
+	if err != nil {
+		return fmt.Errorf("failed to marshal metadata: %w", err)
+	}
+
+	// Write to temp file first, then rename for atomicity
+	tmpFile, err := os.CreateTemp(dir, ".tmp-*.json")
+	if err != nil {
+		return fmt.Errorf("failed to create temp file: %w", err)
+	}
+
+	tmpPath := tmpFile.Name()
+
+	_, err = tmpFile.Write(data)
+	if err != nil {
+		_ = tmpFile.Close()
+		_ = os.Remove(tmpPath)
+
+		return fmt.Errorf("failed to write metadata: %w", err)
+	}
+
+	err = tmpFile.Close()
+	if err != nil {
+		_ = os.Remove(tmpPath)
+
+		return fmt.Errorf("failed to close temp file: %w", err)
+	}
+
+	// Atomic rename
+	err = os.Rename(filepath.Clean(tmpPath), filepath.Clean(path))
+	if err != nil {
+		_ = os.Remove(tmpPath)
+
+		return fmt.Errorf("failed to rename temp file: %w", err)
+	}
+
+	return nil
+}
+
+// Load reads metadata from storage.
+func (s *MetadataStorage) Load(
+	host string, pathHash PathHash,
+) (*SourceMetadata, error) {
+	path := s.metaPath(host, pathHash)
+
+	data, err := os.ReadFile(path) //nolint:gosec // path derived from host+hash
+	if err != nil {
+		if os.IsNotExist(err) {
+			return nil, ErrNotFound
+		}
+
+		return nil, fmt.Errorf("failed to read metadata: %w", err)
+	}
+
+	var meta SourceMetadata
+
+	err = json.Unmarshal(data, &meta)
+	if err != nil {
+		return nil, fmt.Errorf("failed to unmarshal metadata: %w", err)
+	}
+
+	return &meta, nil
+}
+
+// Delete removes metadata for the given host and path hash.
+func (s *MetadataStorage) Delete(host string, pathHash PathHash) error {
+	path := s.metaPath(host, pathHash)
+
+	err := os.Remove(path)
+	if err != nil && !os.IsNotExist(err) {
+		return fmt.Errorf("failed to delete metadata: %w", err)
+	}
+
+	return nil
+}
+
+// Exists checks if metadata exists for the given host and path hash.
+func (s *MetadataStorage) Exists(host string, pathHash PathHash) bool {
+	path := s.metaPath(host, pathHash)
+	_, err := os.Stat(path)
+
+	return err == nil
+}
+
+// metaPath returns the file path for metadata: <basedir>/<host>/<path_hash>.json
+func (s *MetadataStorage) metaPath(host string, pathHash PathHash) string {
+	return filepath.Clean(filepath.Join(s.baseDir, host, string(pathHash)+".json"))
+}
+
+// HashPath computes the SHA256 hash of a path string.
+func HashPath(path string) PathHash {
+	h := sha256.Sum256([]byte(path))
+
+	return PathHash(hex.EncodeToString(h[:]))
 }
 
 // VariantStorage handles key-based file storage for processed image variants.
 // Files are stored at: <basedir>/<ab>/<cd>/<cache_key>
+// Metadata is stored at: <basedir>/<ab>/<cd>/<cache_key>.meta
+// Unlike ContentStorage, the key is provided by the caller (not computed from content).
 type VariantStorage struct {
 	baseDir string
+	log     *slog.Logger
 }
 
-// errNotAnOutputFormat is returned for a variant file whose first bytes are
+// VariantMeta contains metadata about a cached variant.
-// those of a format pixa reads but never writes, such as SVG.
+//
-var errNotAnOutputFormat = errors.New("variant file is not in an output format")
+//nolint:tagliatelle // stored metadata format uses snake_case
+type VariantMeta struct {
+	ContentType string `json:"content_type"`
+	Size        int64  `json:"size"`
+	CreatedAt   int64  `json:"created_at"`
+}
 
 // NewVariantStorage creates a new variant storage at the given base directory.
-func NewVariantStorage(baseDir string) (*VariantStorage, error) {
+// A failed .meta write is logged to log.
+func NewVariantStorage(baseDir string, log *slog.Logger) (*VariantStorage, error) {
 	err := os.MkdirAll(baseDir, StorageDirPerm)
 	if err != nil {
 		return nil, fmt.Errorf("failed to create variant storage directory: %w", err)
 	}
 
-	return &VariantStorage{baseDir: baseDir}, nil
+	return &VariantStorage{baseDir: baseDir, log: log}, nil
 }
 
-// Store writes content to storage at the given key with writeFileSynced.
+// Store writes content and metadata to storage at the given key, each
+// with writeFileSynced. The .meta file is written first, so that a crash
+// cannot leave the content in place without it.
 func (s *VariantStorage) Store(
-	key imagerequest.VariantKey, r io.Reader,
+	key imagerequest.VariantKey, r io.Reader, contentType string,
 ) (int64, error) {
 	data, err := io.ReadAll(r)
 	if err != nil {
 		return 0, fmt.Errorf("failed to read content: %w", err)
 	}
 
+	size := int64(len(data))
 	path := s.keyToPath(key)
+	metaPath := path + ".meta"
 
 	err = os.MkdirAll(filepath.Dir(path), StorageDirPerm)
 	if err != nil {
 		return 0, fmt.Errorf("failed to create directory: %w", err)
 	}
 
+	meta := VariantMeta{
+		ContentType: contentType,
+		Size:        size,
+		CreatedAt:   time.Now().UTC().Unix(),
+	}
+
+	metaData, err := json.Marshal(meta)
+	if err != nil {
+		return 0, fmt.Errorf("failed to marshal metadata: %w", err)
+	}
+
+	// Metadata write failure is non-fatal: the content is then served as
+	// application/octet-stream.
+	err = writeFileSynced(metaPath, metaData)
+	if err != nil {
+		s.log.Warn("failed to write variant metadata sidecar",
+			"path", metaPath, "error", err)
+	}
+
 	err = writeFileSynced(path, data)
 	if err != nil {
 		return 0, err
 	}
 
-	return int64(len(data)), nil
+	return size, nil
 }
 
-// LoadWithContentType opens the content at the given key and returns a
+// Load returns a reader for the content at the given key.
-// reader, the file size and the content type, which is read from the
+func (s *VariantStorage) Load(key imagerequest.VariantKey) (io.ReadCloser, error) {
-// file's first bytes. A missing file is ErrNotFound. A file whose first
-// bytes are not those of an output format, an empty one included, is
-// closed again and the reason returned.
-func (s *VariantStorage) LoadWithContentType(
-	key imagerequest.VariantKey,
-) (io.ReadCloser, int64, string, error) {
 	path := s.keyToPath(key)
 
 	f, err := os.Open(path) //nolint:gosec // path derived from cache key
 	if err != nil {
 		if os.IsNotExist(err) {
-			return nil, 0, "", ErrNotFound
+			return nil, ErrNotFound
 		}
 
-		return nil, 0, "", fmt.Errorf("failed to open content: %w", err)
+		return nil, fmt.Errorf("failed to open content: %w", err)
+	}
+
+	return f, nil
+}
+
+// LoadWithSize returns a reader and file size for the content at the
+// given key.
+func (s *VariantStorage) LoadWithSize(
+	key imagerequest.VariantKey,
+) (io.ReadCloser, int64, error) {
+	path := s.keyToPath(key)
+
+	f, err := os.Open(path) //nolint:gosec // path derived from cache key
+	if err != nil {
+		if os.IsNotExist(err) {
+			return nil, 0, ErrNotFound
+		}
+
+		return nil, 0, fmt.Errorf("failed to open content: %w", err)
 	}
 
 	stat, err := f.Stat()
 	if err != nil {
 		_ = f.Close()
 
-		return nil, 0, "", fmt.Errorf("failed to stat content: %w", err)
+		return nil, 0, fmt.Errorf("failed to stat content: %w", err)
 	}
 
-	contentType, err := outputContentType(f)
+	return f, stat.Size(), nil
-	if err != nil {
+}
-		_ = f.Close()
 
+// LoadWithMeta returns a reader, size, and content type for the content at
+// the given key. The content type is read from the .meta file, and is
+// empty when that file is missing or unreadable.
+func (s *VariantStorage) LoadWithMeta(
+	key imagerequest.VariantKey,
+) (io.ReadCloser, int64, string, error) {
+	f, size, err := s.LoadWithSize(key)
+	if err != nil {
 		return nil, 0, "", err
 	}
 
-	return f, stat.Size(), contentType, nil
+	var contentType string
-}
 
-// outputContentType reads the first bytes of content, without moving its
+	metaPath := s.keyToPath(key) + ".meta"
-// offset, and returns the content type of the output format they show.
-func outputContentType(content io.ReaderAt) (string, error) {
-	firstBytes := make([]byte, magic.MinMagicBytes)
 
-	_, err := content.ReadAt(firstBytes, 0)
+	metaData, err := os.ReadFile(metaPath) //nolint:gosec // path derived from cache key
-	if err != nil {
+	if err == nil {
-		return "", fmt.Errorf("failed to read the first bytes: %w", err)
+		var meta VariantMeta
+		if json.Unmarshal(metaData, &meta) == nil && meta.ContentType != "" {
+			contentType = meta.ContentType
+		}
 	}
 
-	detected, err := magic.DetectFormat(firstBytes)
+	return f, size, contentType, nil
-	if err != nil {
-		return "", err
-	}
-
-	_, isOutputFormat := magic.MIMEToImageFormat(string(detected))
-	if !isOutputFormat {
-		return "", fmt.Errorf("%w: %s", errNotAnOutputFormat, detected)
-	}
-
-	return string(detected), nil
 }
 
 // Exists checks if content exists at the given key.
@@ -302,8 +553,8 @@ func (s *VariantStorage) Exists(key imagerequest.VariantKey) bool {
 	return err == nil
 }
 
-// Delete removes the content at the given key. A missing file is not an
+// Delete removes the content at the given key together with its .meta
-// error.
+// sidecar file. A missing file is not an error.
 func (s *VariantStorage) Delete(key imagerequest.VariantKey) error {
 	path := s.keyToPath(key)
 
@@ -312,6 +563,13 @@ func (s *VariantStorage) Delete(key imagerequest.VariantKey) error {
 		return fmt.Errorf("failed to delete content: %w", err)
 	}
 
+	metaPath := path + ".meta"
+
+	err = os.Remove(metaPath)
+	if err != nil && !os.IsNotExist(err) {
+		return fmt.Errorf("failed to delete variant metadata: %w", err)
+	}
+
 	return nil
 }
 

internal/imgcache/storage_internal_test.go

@@ -4,14 +4,16 @@ import (
 	"bytes"
 	"errors"
 	"io"
+	"log/slog"
 	"os"
 	"path/filepath"
+	"strings"
 	"testing"
 
 	"sneak.berlin/go/pixa/internal/imagerequest"
 )
 
-func TestContentStorage_StoreAndLoadWithSize(t *testing.T) {
+func TestContentStorage_StoreAndLoad(t *testing.T) {
 	t.Parallel()
 
 	tmpDir := t.TempDir()
@@ -22,9 +24,8 @@ func TestContentStorage_StoreAndLoadWithSize(t *testing.T) {
 	}
 
 	content := []byte("hello world")
-	key := imagerequest.Source{Host: testHostCDN, Path: testPathCat}.Key()
 
-	size, err := storage.Store(key, content)
+	hash, size, err := storage.Store(bytes.NewReader(content))
 	if err != nil {
 		t.Fatalf("Store() error = %v", err)
 	}
@@ -33,10 +34,14 @@ func TestContentStorage_StoreAndLoadWithSize(t *testing.T) {
 		t.Errorf("Store() size = %d, want %d", size, len(content))
 	}
 
-	// Verify file exists at expected path
+	if hash == "" {
-	keyStr := string(key)
+		t.Error("Store() returned empty hash")
+	}
 
-	expectedPath := filepath.Join(tmpDir, keyStr[0:2], keyStr[2:4], keyStr)
+	// Verify file exists at expected path
+	hashStr := string(hash)
+
+	expectedPath := filepath.Join(tmpDir, hashStr[0:2], hashStr[2:4], hashStr)
 
 	_, err = os.Stat(expectedPath)
 	if err != nil {
@@ -44,9 +49,9 @@ func TestContentStorage_StoreAndLoadWithSize(t *testing.T) {
 	}
 
 	// Load and verify content
-	r, _, err := storage.LoadWithSize(key)
+	r, err := storage.Load(hash)
 	if err != nil {
-		t.Fatalf("LoadWithSize() error = %v", err)
+		t.Fatalf("Load() error = %v", err)
 	}
 
 	defer func() { _ = r.Close() }()
@@ -57,11 +62,11 @@ func TestContentStorage_StoreAndLoadWithSize(t *testing.T) {
 	}
 
 	if !bytes.Equal(loaded, content) {
-		t.Errorf("LoadWithSize() content = %q, want %q", loaded, content)
+		t.Errorf("Load() content = %q, want %q", loaded, content)
 	}
 }
 
-func TestContentStorage_LoadWithSizeNotFound(t *testing.T) {
+func TestContentStorage_StoreIdempotent(t *testing.T) {
 	t.Parallel()
 
 	tmpDir := t.TempDir()
@@ -71,9 +76,36 @@ func TestContentStorage_LoadWithSizeNotFound(t *testing.T) {
 		t.Fatalf("NewContentStorage() error = %v", err)
 	}
 
-	_, _, err = storage.LoadWithSize(imagerequest.SourceKey("nonexistent"))
+	content := []byte("same content")
+
+	hash1, _, err := storage.Store(bytes.NewReader(content))
+	if err != nil {
+		t.Fatalf("Store() first error = %v", err)
+	}
+
+	hash2, _, err := storage.Store(bytes.NewReader(content))
+	if err != nil {
+		t.Fatalf("Store() second error = %v", err)
+	}
+
+	if hash1 != hash2 {
+		t.Errorf("Store() hashes differ: %s vs %s", hash1, hash2)
+	}
+}
+
+func TestContentStorage_LoadNotFound(t *testing.T) {
+	t.Parallel()
+
+	tmpDir := t.TempDir()
+
+	storage, err := NewContentStorage(tmpDir)
+	if err != nil {
+		t.Fatalf("NewContentStorage() error = %v", err)
+	}
+
+	_, err = storage.Load(ContentHash("nonexistent"))
 	if !errors.Is(err, ErrNotFound) {
-		t.Errorf("LoadWithSize() error = %v, want ErrNotFound", err)
+		t.Errorf("Load() error = %v, want ErrNotFound", err)
 	}
 }
 
@@ -88,23 +120,22 @@ func TestContentStorage_Delete(t *testing.T) {
 	}
 
 	content := []byte("to be deleted")
-	key := imagerequest.Source{Host: testHostCDN, Path: testPathCat}.Key()
 
-	_, err = storage.Store(key, content)
+	hash, _, err := storage.Store(bytes.NewReader(content))
 	if err != nil {
 		t.Fatalf("Store() error = %v", err)
 	}
 
-	if !storage.Exists(key) {
+	if !storage.Exists(hash) {
 		t.Error("Exists() = false, want true")
 	}
 
-	err = storage.Delete(key)
+	err = storage.Delete(hash)
 	if err != nil {
 		t.Fatalf("Delete() error = %v", err)
 	}
 
-	if storage.Exists(key) {
+	if storage.Exists(hash) {
 		t.Error("Exists() = true after delete, want false")
 	}
 }
@@ -120,13 +151,13 @@ func TestContentStorage_DeleteNonexistent(t *testing.T) {
 	}
 
 	// Should not error
-	err = storage.Delete(imagerequest.SourceKey("nonexistent"))
+	err = storage.Delete(ContentHash("nonexistent"))
 	if err != nil {
 		t.Errorf("Delete() error = %v, want nil", err)
 	}
 }
 
-func TestContentStorage_KeyToPath(t *testing.T) {
+func TestContentStorage_HashToPath(t *testing.T) {
 	t.Parallel()
 
 	tmpDir := t.TempDir()
@@ -138,19 +169,263 @@ func TestContentStorage_KeyToPath(t *testing.T) {
 
 	// Test by storing and verifying the resulting path structure
 	content := []byte("test content for path verification")
-	key := imagerequest.Source{Host: testHostCDN, Path: testPathCat}.Key()
 
-	_, err = storage.Store(key, content)
+	hash, _, err := storage.Store(bytes.NewReader(content))
 	if err != nil {
 		t.Fatalf("Store() error = %v", err)
 	}
 
-	keyStr := string(key)
+	hashStr := string(hash)
 
-	expectedPath := filepath.Join(tmpDir, keyStr[0:2], keyStr[2:4], keyStr)
+	expectedPath := filepath.Join(tmpDir, hashStr[0:2], hashStr[2:4], hashStr)
 
 	_, err = os.Stat(expectedPath)
 	if err != nil {
 		t.Errorf("File not at expected path %s: %v", expectedPath, err)
 	}
 }
+
+func TestMetadataStorage_StoreAndLoad(t *testing.T) {
+	t.Parallel()
+
+	tmpDir := t.TempDir()
+
+	storage, err := NewMetadataStorage(tmpDir)
+	if err != nil {
+		t.Fatalf("NewMetadataStorage() error = %v", err)
+	}
+
+	meta := &SourceMetadata{
+		Host:        testHostCDN,
+		Path:        testPathCat,
+		ContentHash: "abc123",
+		StatusCode:  200,
+		ContentType: testContentTypeJPEG,
+		FetchedAt:   1704067200,
+		ETag:        `"etag123"`,
+	}
+
+	pathHash := HashPath(testPathCat)
+
+	err = storage.Store(testHostCDN, pathHash, meta)
+	if err != nil {
+		t.Fatalf("Store() error = %v", err)
+	}
+
+	// Verify file exists at expected path
+	expectedPath := filepath.Join(tmpDir, testHostCDN, string(pathHash)+".json")
+
+	_, err = os.Stat(expectedPath)
+	if err != nil {
+		t.Errorf("File not at expected path %s: %v", expectedPath, err)
+	}
+
+	// Load and verify
+	loaded, err := storage.Load(testHostCDN, pathHash)
+	if err != nil {
+		t.Fatalf("Load() error = %v", err)
+	}
+
+	if loaded.Host != meta.Host {
+		t.Errorf("Host = %q, want %q", loaded.Host, meta.Host)
+	}
+
+	if loaded.Path != meta.Path {
+		t.Errorf("Path = %q, want %q", loaded.Path, meta.Path)
+	}
+
+	if loaded.ContentHash != meta.ContentHash {
+		t.Errorf("ContentHash = %q, want %q", loaded.ContentHash, meta.ContentHash)
+	}
+
+	if loaded.StatusCode != meta.StatusCode {
+		t.Errorf("StatusCode = %d, want %d", loaded.StatusCode, meta.StatusCode)
+	}
+
+	if loaded.ETag != meta.ETag {
+		t.Errorf("ETag = %q, want %q", loaded.ETag, meta.ETag)
+	}
+}
+
+func TestMetadataStorage_LoadNotFound(t *testing.T) {
+	t.Parallel()
+
+	tmpDir := t.TempDir()
+
+	storage, err := NewMetadataStorage(tmpDir)
+	if err != nil {
+		t.Fatalf("NewMetadataStorage() error = %v", err)
+	}
+
+	_, err = storage.Load(testHostExample, PathHash("nonexistent"))
+	if !errors.Is(err, ErrNotFound) {
+		t.Errorf("Load() error = %v, want ErrNotFound", err)
+	}
+}
+
+func TestMetadataStorage_Delete(t *testing.T) {
+	t.Parallel()
+
+	tmpDir := t.TempDir()
+
+	storage, err := NewMetadataStorage(tmpDir)
+	if err != nil {
+		t.Fatalf("NewMetadataStorage() error = %v", err)
+	}
+
+	meta := &SourceMetadata{
+		Host:       testHostExample,
+		Path:       "/test.jpg",
+		StatusCode: 200,
+	}
+
+	pathHash := HashPath("/test.jpg")
+
+	err = storage.Store(testHostExample, pathHash, meta)
+	if err != nil {
+		t.Fatalf("Store() error = %v", err)
+	}
+
+	if !storage.Exists(testHostExample, pathHash) {
+		t.Error("Exists() = false, want true")
+	}
+
+	err = storage.Delete(testHostExample, pathHash)
+	if err != nil {
+		t.Fatalf("Delete() error = %v", err)
+	}
+
+	if storage.Exists(testHostExample, pathHash) {
+		t.Error("Exists() = true after delete, want false")
+	}
+}
+
+func TestHashPath(t *testing.T) {
+	t.Parallel()
+
+	// Same input should produce same hash
+	hash1 := HashPath(testPathCat)
+	hash2 := HashPath(testPathCat)
+
+	if hash1 != hash2 {
+		t.Errorf("HashPath() not deterministic: %s vs %s", hash1, hash2)
+	}
+
+	// Different input should produce different hash
+	hash3 := HashPath("/photos/dog.jpg")
+
+	if hash1 == hash3 {
+		t.Error("HashPath() produced same hash for different inputs")
+	}
+
+	// Hash should be 64 hex chars (256 bits)
+	if len(string(hash1)) != 64 {
+		t.Errorf("HashPath() length = %d, want 64", len(string(hash1)))
+	}
+}
+
+// TestVariantStorage_StoreLogsFailedMetaWrite verifies that a .meta write
+// that fails is logged at warn and the store still succeeds.
+func TestVariantStorage_StoreLogsFailedMetaWrite(t *testing.T) {
+	t.Parallel()
+
+	var logBuf bytes.Buffer
+
+	storage, err := NewVariantStorage(
+		t.TempDir(), slog.New(slog.NewJSONHandler(&logBuf, nil)))
+	if err != nil {
+		t.Fatalf("NewVariantStorage() error = %v", err)
+	}
+
+	key := imagerequest.CacheKey(&imagerequest.Request{
+		Source: imagerequest.Source{Host: testHostCDN, Path: testPathCat},
+	})
+
+	// A directory where the .meta file goes makes the .meta write fail.
+	err = os.MkdirAll(storage.keyToPath(key)+".meta", StorageDirPerm)
+	if err != nil {
+		t.Fatalf("failed to create directory: %v", err)
+	}
+
+	_, err = storage.Store(key, bytes.NewReader([]byte("variant data")), "image/webp")
+	if err != nil {
+		t.Fatalf("Store() error = %v, want nil", err)
+	}
+
+	want := `"level":"WARN","msg":"failed to write variant metadata sidecar"`
+	if !strings.Contains(logBuf.String(), want) {
+		t.Errorf("log missing %s; got %q", want, logBuf.String())
+	}
+}
+
+// storeTestVariant stores one variant, with its .meta file, in a new
+// VariantStorage and returns the storage and the variant's key.
+func storeTestVariant(t *testing.T) (*VariantStorage, imagerequest.VariantKey) {
+	t.Helper()
+
+	storage, err := NewVariantStorage(t.TempDir(), slog.New(slog.DiscardHandler))
+	if err != nil {
+		t.Fatalf("NewVariantStorage() error = %v", err)
+	}
+
+	key := imagerequest.CacheKey(&imagerequest.Request{
+		Source: imagerequest.Source{Host: testHostCDN, Path: testPathCat},
+	})
+
+	_, err = storage.Store(key, bytes.NewReader([]byte("variant data")), "image/webp")
+	if err != nil {
+		t.Fatalf("Store() error = %v", err)
+	}
+
+	return storage, key
+}
+
+// TestVariantStorage_DeleteRemovesMeta verifies that Delete removes the
+// variant's .meta file along with the variant file.
+func TestVariantStorage_DeleteRemovesMeta(t *testing.T) {
+	t.Parallel()
+
+	storage, key := storeTestVariant(t)
+	metaPath := storage.keyToPath(key) + ".meta"
+
+	_, err := os.Stat(metaPath)
+	if err != nil {
+		t.Fatalf("Store() wrote no .meta file: %v", err)
+	}
+
+	err = storage.Delete(key)
+	if err != nil {
+		t.Fatalf("Delete() error = %v", err)
+	}
+
+	if storage.Exists(key) {
+		t.Error("Exists() = true after delete, want false")
+	}
+
+	_, err = os.Stat(metaPath)
+	if !os.IsNotExist(err) {
+		t.Errorf(".meta file left after Delete() (stat err=%v)", err)
+	}
+}
+
+// TestVariantStorage_DeleteWithoutMeta verifies that Delete succeeds for a
+// variant whose .meta file is missing.
+func TestVariantStorage_DeleteWithoutMeta(t *testing.T) {
+	t.Parallel()
+
+	storage, key := storeTestVariant(t)
+
+	err := os.Remove(storage.keyToPath(key) + ".meta")
+	if err != nil {
+		t.Fatalf("removing .meta file: %v", err)
+	}
+
+	err = storage.Delete(key)
+	if err != nil {
+		t.Fatalf("Delete() error = %v, want nil", err)
+	}
+
+	if storage.Exists(key) {
+		t.Error("Exists() = true after delete, want false")
+	}
+}