Split internal/imgcache into the image proxy and the cache (closes #317)

The new package internal/imageproxy holds Proxy, which was imgcache.Service,
with its Config, ImageResponse, CacheStatus and errors. internal/imgcache holds
the cache only, and imgcache.CacheConfig is imgcache.Config. The cache reads
and writes each kind it keeps with one pair of methods: Variant and PutVariant,
Source and PutSource, Failure and PutFailure. Source replaces LookupSource
followed by GetSourceContent. imgcache.go and module.go are gone. The proxy's
tests moved with it. They reach the cache through its database, its state
directory and its metrics registry, and run its reconciliation pass with a
second cache, as pixad does at startup.

Model: opus-5-5

Makefile

@@ -1,4 +1,4 @@
-.PHONY: bootstrap setup check lint test fmt fmt-check build clean docker docker-smoke docker-versioned docker-test devserver devserver-stop hooks loadtest
+.PHONY: bootstrap setup check lint test fmt fmt-check build clean docker docker-smoke docker-versioned docker-test devserver devserver-stop hooks loadtest bench
 
 VERSION := $(shell git describe --tags --always --dirty 2>/dev/null || echo "dev")
 LDFLAGS := -X main.Version=$(VERSION)
@@ -64,6 +64,12 @@ docker-smoke:
 loadtest:
 	@script/loadtest
 
+# Serve cache hits in parallel through the image handler and report hits per
+# second (needs Go on the host after `script/bootstrap --cgo`, not Docker; a
+# benchmark, not part of check)
+bench:
+	@script/bench
+
 # Build Docker image as `make docker` does, and also tag it pixa:$(VERSION)
 docker-versioned:
 	@script/docker

README.md

@@ -76,9 +76,9 @@ database and the disk cache:
   pixa runs as (uid 65532) and sets its mode to `750`; outside it, that user
   must be able to write the directory.
 - `cache_max_bytes` limits the source and transformed images together. The
-  database, the `.meta` file beside each transformed image and files still being
-  written come on top, and eviction runs in the background, so the cache can
-  pass the limit for a while: leave room on the volume beyond it.
+  database and files still being written come on top, and eviction runs in the
+  background, so the cache can pass the limit for a while: leave room on the
+  volume beyond it.
 - Set `cache_max_bytes` for a lasting deployment. Its default, worked out each
   time pixa starts, is 75% of the sum of the space free on the volume and the
   space the cached images already take, so a restart keeps the limit the cache
@@ -88,13 +88,17 @@ A load balancer's health check can request `/.well-known/healthcheck.json`,
 which answers 200 whenever pixa is running, in maintenance mode too (see
 `maintenance_mode`).
 
-On SIGTERM or SIGINT pixa stops accepting connections, gives the requests in
-progress and the images being processed 5 seconds to finish, then writes to the
-database the counts of cache hits, misses, fetches and conversions that requests
-could not write by their deadline, and exits: with 0, or with 1 when images were
-still being processed after those 5 seconds, some of those counts could not be
-written, or another part of pixa failed to stop. A request not finished by then
-is cut off. `docker stop` waits 10 seconds before it kills the container.
+On SIGTERM or SIGINT pixa stops accepting connections. The requests in progress
+and the images being processed get 5 seconds to finish. A request not finished
+by then is cut off. pixa then writes the waiting last-used times of cached
+images to the database (see Storage) and exits. It exits with 1 when:
+
+- images were still being processed after those 5 seconds,
+- some of those times could not be written, or
+- another part of pixa failed to stop.
+
+Otherwise it exits with 0. `docker stop` waits 10 seconds before it kills the
+container.
 
 Outside Docker, pixa needs libvips (the image has 8.16) and libheif to run, as
 it uses libvips through CGO. pixad does not start unless libvips has its JPEG XL
@@ -181,8 +185,7 @@ allowlisted source hosts for open access.
   file for each source URL
 - **Database**: `<state_dir>/state.sqlite3` (SQLite)
 - **Transformed images**:
-  `<state_dir>/cache/variants/<ab>/<cd>/<sha256 of host, path, query, size, format, quality and fit>`,
-  each with a `.meta` file beside it holding its content type
+  `<state_dir>/cache/variants/<ab>/<cd>/<sha256 of host, path, query, size, format, quality and fit>`
 
 `<ab>` and `<cd>` are the first and second pairs of characters of the file's
 name.
@@ -193,22 +196,30 @@ bytes are stored twice, once under each URL. A source image is stored by writing
 its file and then its row; when the row cannot be written, the file is removed
 again.
 
-Source and transformed image files, and the `.meta` file beside each transformed
-image, are written under a temporary name, synced to disk, and only then renamed
-into place, so a crash leaves no file under its own name with part of its data.
-When pixa starts, and every five minutes after, it removes each cached source or
-transformed image whose file is empty, or not the size the database records for
-it, with a warning naming the file; the next request for it fetches the source
-or converts the image again. Looking for those files reads the database once for
-every cached file, so pixa skips each five-minute look that would start less
-than ten times as long after the last look ended as that look took. After a look
-that took one minute, the next one starts 10 to 15 minutes after that look
-ended.
+Source and transformed image files are written under a temporary name, synced to
+disk, and only then renamed into place, so a crash leaves no file under its own
+name with part of its data. When pixa starts, and every five minutes after, it
+removes each cached source or transformed image whose file is empty, or not the
+size the database records for it, with a warning naming the file; the next
+request for it fetches the source or converts the image again. Looking for those
+files reads the database once for every cached file, so pixa skips each
+five-minute look that would start less than ten times as long after the last
+look ended as that look took. After a look that took one minute, the next one
+starts 10 to 15 minutes after that look ended.
 
 pixa's target is 1-5k r/s, which has not been measured at that rate (see Load
-Test). Toward it, pixa keeps in memory the content types of the 10,000
-transformed images most recently cached or served, so a cache hit on one of them
-reads only the image file from disk and not the metadata file stored beside it.
+Test). Toward it, a cache hit opens the transformed image's file and nothing
+else, and makes no database call. The content type comes from the file's first
+bytes, which are those of the format pixa wrote it in; a file whose first bytes
+are not those of a format pixa writes is not served, and the image is converted
+again. A hit hands the time it was served to a background writer and does not
+wait for the writer. Eviction orders by that time. The hit is counted in memory.
+A miss that uses a cached source image also hands that writer the time it used
+the image. Once a second that writer writes the waiting times in one
+transaction. While 10,000 times of transformed images, or 10,000 of source
+images, wait to be written, a further one is dropped, and that image only looks
+less recently used to eviction. `script/bench` measures how many hits pixa
+serves per second (see Load Test).
 
 ### Routes
 
@@ -237,32 +248,58 @@ path under `/v1/` answers 200, in maintenance mode too.
 - `GET` or `HEAD` `/v1/image/<host>/<path>/<size>.<format>`, or
   `/v1/image/<host>/<path>/<size>` with no format — an image, fetched, resized
   and converted (below). Needs: a signature, unless the host is allowlisted (see
-  Source Hosts). Answers: 200; 304 when `If-None-Match` matches the image's
-  `ETag`; 400 for a URL or parameter that is not valid, or for the format `auto`
-  an `Accept` header that is not valid; 406 for the format `auto` when `Accept`
-  allows none of the formats it chooses from; 401 for a missing or wrong
-  signature, a missing `exp` or an `exp` in the past; 403 when the request's
-  `Referer` names a host in `referer_blocklist`, checked before the signature,
-  the cache and the upstream fetch; 403 when the upstream host, or a host it
-  redirects to, is `localhost`, ends in `.localhost` or `.local`, or has an
-  address in a blocked network (see `blocked_networks`); 429 over
-  `image_requests_per_minute` (below); 415 when the upstream answers with a
-  `Content-Type` that is not one of the source formats (below), with a body
-  whose first bytes are not those of the format its `Content-Type` names, or
-  with an image whose first bytes libvips reads as another format; 502 when the
-  upstream answered with an error status, and for 5 minutes after that, or after
-  a body whose first bytes are not those of its `Content-Type`, for an image of
-  the same source URL that pixa can neither serve from its cache nor make from
-  the source image still in its cache; 503 when pixa is busy or in maintenance
-  mode; 500 for any other failure.
+  Source Hosts). Answers:
+    - 200 with the image.
+    - 304 when `If-None-Match` matches the image's `ETag`.
+    - 400 for a URL or parameter that is not valid.
+    - 400 when the format is `auto` and the `Accept` header is not valid.
+    - 406 when the format is `auto` and `Accept` allows none of the formats it
+      chooses from.
+    - 401 for a missing or wrong signature.
+    - 401 for a missing `exp`.
+    - 401 for an `exp` in the past.
+    - 403 when the request's `Referer` names a host in `referer_blocklist`,
+      checked before the signature, the cache and the upstream fetch.
+    - 403 when the upstream host, or a host it redirects to, is `localhost` or
+      ends in `.localhost` or `.local`.
+    - 403 when the upstream host, or a host it redirects to, has an address in a
+      blocked network (see `blocked_networks`).
+    - 429 over `image_requests_per_minute` (below).
+    - 415 when the upstream answers with a `Content-Type` that is not one of the
+      source formats (below).
+    - 415 when the upstream sends a body whose first bytes are not those of the
+      format its `Content-Type` names.
+    - 415 when the upstream sends an image whose first bytes libvips reads as
+      another format.
+    - 502 when the upstream answers with an error status.
+    - 502 for 5 minutes after that for the same source URL, when pixa has
+      neither the image nor its source image in its cache.
+    - 502 likewise for 5 minutes after a body whose first bytes are not those of
+      its `Content-Type`.
+    - 503 when pixa is busy.
+    - 503 when pixa is in maintenance mode.
+    - 504 when the upstream has not sent its response headers within
+      `upstream_fetch_timeout`.
+    - 500 when that time runs out while the image itself is still arriving.
+    - 500 for any other failure.
 - `GET` or `HEAD` `/v1/e/<token>/<name>` — an image through an encrypted URL
-  (see Encrypted URLs). Needs: nothing but the URL. Answers: 200; 304 when
-  `If-None-Match` matches the image's `ETag`; 400 for a token that does not
-  decrypt, or that asks for a size or fit that is not valid; 410 once it has
-  expired; 504 when the upstream has not sent its response headers within
-  `upstream_fetch_timeout`, but 500 when that time runs out while the image
-  itself is still arriving; 400 for an `Accept` header that is not valid, and
-  406, 403, 415, 429, 502, 503 and 500, as for `/v1/image/`.
+  (see Encrypted URLs). Needs: nothing but the URL. Answers:
+    - 200 with the image.
+    - 304 when `If-None-Match` matches the image's `ETag`.
+    - 400 for a token that does not decrypt.
+    - 400 for a token that asks for a size or fit that is not valid.
+    - 410 once the token has expired.
+    - 400 when the token's format is `auto` and the `Accept` header is not
+      valid.
+    - These answers, for the same causes as on `/v1/image/`:
+        - 406
+        - 403
+        - 415
+        - 429
+        - 502
+        - 503
+        - 504
+        - 500
 - `GET /robots.txt` — asks every crawler to stay away (`Disallow: /`). Needs:
   nothing. Answers: 200.
 - `GET /.well-known/healthcheck.json` — JSON with `status` (`ok`), `now`,
@@ -754,7 +791,17 @@ Key settings in more detail:
   duration; default `60s`. The upstream fetch counts toward it, and so do the
   waits for an upstream connection and for a processing slot (up to 10 seconds
   each), so keep it longer than `upstream_fetch_timeout` plus 20 seconds
-- `signing_key` — HMAC secret for URL signatures
+- `signing_key` — the secret that protects signed URLs, encrypted URLs and
+  logins. pixa uses it as it is for two things:
+    - the HMAC key of signed URLs (see Signature Specification)
+    - the password of the login page
+
+    pixa derives every other key it uses from it at startup, with HKDF-SHA256:
+    - a key that signs the session cookie
+    - a key that encrypts the session cookie
+    - a key that encrypts the token of an encrypted URL
+    - a key that signs the CSRF token cookie of the login and generator pages
+
 - `db_url` — the SQLite database to open; omitted, it is
   `file:<state_dir>/state.sqlite3?_pragma=journal_mode(WAL)`, which keeps the
   database in WAL mode. `state_dir` is percent-encoded in it, so a `%`, `?` or
@@ -801,20 +848,34 @@ See `configs/config.example.yml` for all options with defaults.
 
 ### Architecture
 
-- **Dependency injection**: Uber fx
+- **Dependency injection**: Uber fx. `internal/app` builds every part of pixad
+  from the config in one fx graph. Each setting's default is declared once, in
+  `internal/config`.
 - **HTTP router**: go-chi
 - **Image processing**: govips (CGO wrapper for libvips)
 - **Database**: SQLite via modernc.org/sqlite
 - **Static assets**: embedded via `//go:embed`
-- **Metrics**: Prometheus, at `/metrics`: generic HTTP request metrics
-  (duration, response size, requests in flight) and the Go runtime and process
-  metrics; requests are measured and `/metrics` is served only when
-  `metrics.username` and `metrics.password` are set. A request's `handler` label
-  is the pattern of the route it matches, such as `/v1/image/*`, never its path.
-  Requests that match no route share the label `unmatched`: those the router
-  answers 404 or 405, and CORS preflights. A request's `method` label is its
-  method when that is `GET`, `HEAD`, `POST`, `PUT`, `PATCH`, `DELETE`,
-  `OPTIONS`, `CONNECT` or `TRACE`, and `other` for any other method
+- **Metrics**: Prometheus, at `/metrics`. pixa serves `/metrics` and measures
+  HTTP requests only when `metrics.username` and `metrics.password` are set.
+  `/metrics` holds:
+    - the duration of HTTP requests
+    - the size of HTTP responses
+    - the HTTP requests in flight
+    - the Go runtime and process metrics
+    - `pixa_cache_hits_total`: image requests served from the cache
+    - `pixa_cache_misses_total`: image requests not served from the cache
+    - `pixa_upstream_fetches_total`: fetches of a source image that read any
+      bytes
+    - `pixa_upstream_fetch_bytes_total`: the bytes those fetches read
+    - `pixa_transforms_total`: images converted
+
+    A request's `handler` label is the pattern of the route it matches, such as
+    `/v1/image/*`, never its path. Requests that match no route share the label
+    `unmatched`: those the router answers 404 or 405, and CORS preflights. A
+    request's `method` label is its method when that is `GET`, `HEAD`, `POST`,
+    `PUT`, `PATCH`, `DELETE`, `OPTIONS`, `CONNECT` or `TRACE`, and `other` for
+    any other method.
+
 - **Logging**: stdlib slog
 
 ## Entrypoints
@@ -846,6 +907,9 @@ provide:
 - `script/docker-smoke` — build the image, start it, wait for it to be healthy
 - `script/loadtest` — measure pixad's throughput, latency and peak memory; a
   benchmark, not part of `script/check` (see Load Test)
+- `script/bench` — run the Go benchmark that serves cache hits in parallel
+  through the image handler, with Go on the host after `script/bootstrap --cgo`;
+  a benchmark, not part of `script/check` (see Load Test)
 - `script/cibuild` — CI entrypoint: run `script/bootstrap` (without `--cgo`),
   then `script/check`, then build the image as `script/docker` does
 - `script/precommit` — pre-commit checks (`go mod tidy` guard, then
@@ -926,6 +990,17 @@ The numbers depend on the machine and on whatever else runs on it. The first
 measurement, made on a shared machine with few clients, is in `TODO.md`; it says
 nothing about the target.
 
+`script/bench` (or `make bench`) measures cache hits alone, without the network
+and with no pixad process: a Go benchmark in `internal/handlers` serves one
+cached image through the `/v1/image/` handler from as many goroutines at once as
+the machine has CPUs, and reports the hits served per second as `hits/s`. Each
+run starts with an empty cache and an empty database, a SQLite file of its own,
+and puts the image in the cache before it measures. The benchmark fails if
+serving the hits ran any statement on the database. Like the load test it is not
+part of `script/check`. It runs Go on the host, as the load test does, and
+compiles pixa there, so it needs the C compiler and libraries that
+`script/bootstrap --cgo` installs, but not Docker.
+
 ## TODO
 
 See [TODO.md](TODO.md) for the full prioritized task list.

TODO.md

@@ -30,6 +30,171 @@ P2: security: per-origin rate limiting
 
 # Completed Steps
 
+- 2026-10-26 `internal/imgcache` holds the cache only (closes #317). The new
+  package `internal/imageproxy` fetches, transforms, caches and serves images.
+  It imports `imgcache`, and `imgcache` does not import it.
+    - `imgcache.Service` is `imageproxy.Proxy`, built by `imageproxy.NewProxy`
+      from an `imageproxy.Config`.
+    - `ImageResponse`, `CacheStatus`, `ErrNegativeCached` and
+      `ErrProcessingTimedOut` moved with it, under their names.
+    - `imgcache.CacheConfig` is `imgcache.Config`.
+    - The cache has one read and one write for each kind it keeps. Each takes
+      the place of an earlier method:
+        - `Variant`, of `GetVariant`
+        - `PutVariant`, of `StoreVariant`
+        - `Source`, of `LookupSource` followed by `GetSourceContent`
+        - `PutSource`, of `StoreSource`
+        - `Failure`, of `checkNegativeCache`
+        - `PutFailure`, of `StoreNegative`
+    - `Variant` and `Source` each open the file and hand its access time to the
+      background writer. A cache hit still makes no database call.
+    - `imgcache.go` and `module.go` are gone.
+    - A cached source whose row cannot be read is logged as "failed to load
+      cached source, fetching". It was logged as "source lookup failed".
+    - The proxy's tests moved to `internal/imageproxy`. They reach the cache
+      through its database, its state directory and its metrics registry.
+    - Those tests run the reconciliation pass as pixad does when it starts: with
+      a second cache over the same database and directory.
+- 2026-10-26 both image routes serve an image with one function, `serveImage`,
+  and answer its errors with one, `respondImageError` (closes #316).
+  `HandleImage` parses the path and verifies the signature, and `HandleImageEnc`
+  decrypts the token. Each then calls `serveImage`.
+    - `handleImageError`, the encrypted route's own mapping, is gone.
+    - `/v1/image/` answers 504 when the upstream has not sent its response
+      headers within `upstream_fetch_timeout`, as `/v1/e/` did. It answered 500.
+    - `respondImageError` logs the image service's errors as "failed to get
+      image". Each answer gets the same level on `/v1/image/` and on `/v1/e/`:
+        - 500: error, as pixa's own failure
+        - 400: warn
+        - 403: warn
+        - 415: warn
+        - 502: warn
+        - 503: warn
+        - 504: warn
+    - Before, `/v1/image/` logged all of these answers at error level. `/v1/e/`
+      logged only its 500, as "image request failed", without the host and the
+      path.
+    - `respondImageError` does not log the 415 for a body refused for its first
+      bytes, on either route. The image service logs it as "source refused", at
+      warn level.
+    - The "image served" line carries the host, the path and `fetched_bytes` on
+      both routes. The debug line "encrypted image request" is gone.
+    - The debug line "failed to decrypt URL" is now "invalid encrypted URL". A
+      token that decrypts to a payload that does not decode also gets it.
+    - A failed write is logged once, as "failed to write response" at warn
+      level, since most often the client has gone away. It carries the
+      `request_id`, the host, the path, `served_bytes` and the error, and is not
+      also logged as "image served".
+- 2026-10-26 every key derived from the signing key is derived in one place
+  (closes #315): `seal.DeriveKeys` returns them as one `seal.Keys`, with their
+  salts declared beside it. `internal/providers` calls it once at startup. Each
+  key is derived as before, with `seal.DeriveKey` and the same salt.
+    - The session manager, the encrypted URL generator and the CSRF protection
+      each take their own key instead of the signing key.
+    - `session.NewManager`, `encurl.NewGenerator`, `newCSRFProtect` and
+      `handlers.New` return no error, as nothing in them can fail now.
+    - The signing key itself stays the HMAC key of signed URLs and the login
+      password.
+    - `README.md` says under `signing_key` what it protects.
+- 2026-10-26 the image request holds only what names the cached object (closes
+  #314): its source and its transform, which the variant key covers. The signed
+  URL format, the signing algorithm and every route's answers are unchanged.
+    - The `/v1/image/` handler checks `allowlist_hosts` and verifies the
+      signature itself, with `Signer.Verify(req, sig, expires)`.
+    - Each image handler keeps its URL's expiry as a value of its own, for
+      `Cache-Control`. `encurl`'s `Generate` takes the expiry and its `Parse`
+      returns it.
+    - The fetcher takes the source and builds its URL with `Source.URL`, over
+      http only when the fetcher's own `AllowHTTP` is set. `validateURL` takes
+      that URL, so no URL string is built and parsed again.
+    - `chooseAutoFormat` returns the format it chooses, and the handler puts it
+      into the request it parsed.
+    - The image service serves images through `Get`, and has `WaitForProcessing`
+      for shutdown.
+    - Removed:
+        - `Signature`, `Expires`, `AllowHTTP` and `SourceURL` of the image
+          request
+        - `Service.ValidateRequest`, and the service's signer, allowlist and
+          `AllowHTTP`
+        - `extractHost` of the fetcher and `urlToFSPath` of the mock fetcher
+- 2026-10-26 pixad's fx graph builds the image pipeline (closes #313).
+  `database.New` now opens the database, so the parts below can read it while
+  they are built; its stop hook still runs after the cache's. The cache's
+  `MaxBytes` is the one value for the disk cache limit, and 0 turns the disk
+  cache off. For an omitted `cache_max_bytes`, `internal/providers` works out
+  `config.DefaultCacheMaxBytes` before it builds the cache. `internal/app` and
+  the server tests build the parts below with `providers.Constructors()`; the
+  image proxy test replaces only the fetcher's dial function.
+    - Built by the providers in `internal/providers`, and given to
+      `handlers.New`:
+        - the cache
+        - the fetcher
+        - the processor
+        - the image service
+        - the session manager
+        - the encrypted URL generator
+    - Removed:
+        - `initImageService` and `handlers.Params.Fetcher`
+        - `CacheConfig.UseDefaultMaxBytes`, `CacheConfig.DisableDiskCache` and
+          `Cache.disabled`
+        - `httpfetcher.DefaultConfig()` and the fetcher's copies of setting
+          defaults
+        - `imageprocessor.DefaultMaxInputBytes`, and the processor's defaults
+          for a zero `MaxInputBytes` or `MaxConcurrentProcessing`
+- 2026-10-26 the code nothing in production called is gone (closes #312). The
+  handlers tests sign their image URLs with `generateSignedURL`, a test helper
+  that writes them as the removed generator did. Removed:
+    - `Service.Warm` and the `ImageCache` interface
+    - `Service.GenerateSignedURL` and `Signer.GenerateSignedURL`
+    - `signature.ParseParams`
+    - `imagerequest.ParseImagePath`
+    - `PeekAndValidate`, `IsSupportedMIMEType`, `ImageFormatToMIME` and
+      `FormatOriginal` of `internal/magic`
+    - `SupportedInputFormats` and `SupportedOutputFormats` of the image
+      processor
+    - `IsEmpty` and `Count` of the allowlist
+    - `ssrfSafeDialer`
+    - `CacheStale`, `ErrCacheMiss`, `ErrNegativeCache` and
+      `ImageResponse.LastModified`
+    - `ContentStorage.Load` and `VariantStorage.Load`
+- 2026-10-26 cache hits, misses, upstream fetches, the bytes those fetches read
+  and transforms are counted as Prometheus counters in memory (closes #311).
+  They are `pixa_cache_hits_total`, `pixa_cache_misses_total`,
+  `pixa_upstream_fetches_total`, `pixa_upstream_fetch_bytes_total` and
+  `pixa_transforms_total`, fields of the cache. `NewCache` makes them and
+  registers them on the registry it is given (`internal/imgcache/counters.go`).
+  pixad gives it Prometheus' default registry, which `/metrics` serves; each
+  test gives it a new one. The `Increment` methods add to their cache's
+  counters. `cache_stats`, `Cache.Stats`, `Service.Stats`, `CacheStats` and the
+  pending counts are gone. The background goroutine writes only access times
+  now: `StartPendingAccessTimeWrites` starts it and
+  `StopPendingAccessTimeWrites` stops it, in `pendingaccesstimes.go`.
+- 2026-10-21 the schema file declares only the tables and columns pixa writes
+  and reads (closes #310): `output_content` and `request_cache` are gone, and so
+  are the columns nothing read: the host, path, query, `content_type` and
+  `fetched_at` of `source_content`, the `content_type` and `created_at` of
+  `variant_content`, and the `status_code`, `error_message` and `fetched_at` of
+  `negative_cache`. `StoreSource` no longer takes the fetch result,
+  `LookupSource` returns only the source key, `StoreVariant` no longer takes a
+  content type, `StoreNegative` no longer takes a status code or message, and
+  reconciliation adopts a variant file without reading its first bytes.
+- 2026-10-21 a cache hit makes no database call (closes #309): `Service.Get`
+  opens the variant file with `Cache.GetVariant` and nothing else, with no
+  existence check first. The content type comes from the file's first bytes; a
+  file whose first bytes are not those of an output format, SVG included, is a
+  miss, and the variant is made again. A hit hands its access time to a channel
+  that holds up to 10,000 and drops the rest, and its hit count to the pending
+  counts, without waiting; every count is now a pending count. A miss that finds
+  a cached source hands that source's access time to a second such channel. The
+  goroutine `StartPendingCountWrites` starts writes, once a second, the access
+  times in one transaction and the counts in one `UPDATE`;
+  `StopPendingCountWrites` writes what is left once at shutdown and logs
+  separately the access times and the counts it could not write.
+  `checkNegativeCache` no longer deletes an expired row; `CleanExpired` in the
+  eviction goroutine does. The `.meta` files, `VariantMeta`, `LoadWithMeta`,
+  `variantContentTypeFromSidecar`, `metaCache`, `Lookup` and `golang-lru` are
+  gone. `make bench` (`script/bench`) runs `BenchmarkHandleImage_CacheHits` with
+  Go on the host after `script/bootstrap --cgo`, outside `make check`.
 - 2026-10-21 a source is stored the way a variant is (closes #308): one
   `source_content` row and one file per source URL, the file named by the
   source's `SourceKey`, the SHA-256 of its host, path and query made as the

go.mod

@@ -14,7 +14,6 @@ require (
 	github.com/go-chi/httprate v0.16.0
 	github.com/gorilla/csrf v1.7.3
 	github.com/gorilla/securecookie v1.1.2
-	github.com/hashicorp/golang-lru/v2 v2.0.7
 	github.com/prometheus/client_golang v1.23.2
 	github.com/slok/go-http-metrics v0.13.0
 	github.com/spf13/cobra v1.10.2

go.sum

@@ -228,8 +228,6 @@ github.com/hashicorp/go-version v1.2.1/go.mod h1:fltr4n8CU8Ke44wwGCBoEymUuxUHl09
 github.com/hashicorp/golang-lru v0.5.0/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8=
 github.com/hashicorp/golang-lru v0.5.4 h1:YDjusn29QI/Das2iO9M0BHnIbxPeyuCHsjMW+lJfyTc=
 github.com/hashicorp/golang-lru v0.5.4/go.mod h1:iADmTwqILo4mZ8BN3D2Q6+9jd8WM5uGBxy+E8yxSoD4=
-github.com/hashicorp/golang-lru/v2 v2.0.7 h1:a+bsQ5rvGLjzHuww6tVxozPZFVghXaHOwFs4luLUK2k=
-github.com/hashicorp/golang-lru/v2 v2.0.7/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM=
 github.com/hashicorp/hcl v1.0.1-vault-7 h1:ag5OxFVy3QYTFTJODRzTKVZ6xvdfLLCA1cy/Y6xGI0I=
 github.com/hashicorp/hcl v1.0.1-vault-7/go.mod h1:XYhtn6ijBSAj6n4YqAaf7RBPS4I06AItNorpy+MoQNM=
 github.com/hashicorp/logutils v1.0.0/go.mod h1:QIAnNjmIWmVIIkWDTG1z5v++HQmx9WQRO+LraFDTW64=

internal/allowlist/allowlist.go

@@ -72,13 +72,3 @@ func (w *HostAllowList) IsAllowed(u *url.URL) bool {
 
 	return false
 }
-
-// IsEmpty returns true if the allow list has no entries.
-func (w *HostAllowList) IsEmpty() bool {
-	return len(w.exactHosts) == 0 && len(w.suffixHosts) == 0
-}
-
-// Count returns the total number of allow list entries.
-func (w *HostAllowList) Count() int {
-	return len(w.exactHosts) + len(w.suffixHosts)
-}

internal/allowlist/allowlist_test.go

@@ -139,87 +139,3 @@ func TestHostAllowList_IsAllowed_SuffixMatch(t *testing.T) {
 		},
 	})
 }
-
-func TestHostAllowList_IsEmpty(t *testing.T) {
-	t.Parallel()
-
-	tests := []struct {
-		name     string
-		patterns []string
-		want     bool
-	}{
-		{
-			name:     "empty",
-			patterns: []string{},
-			want:     true,
-		},
-		{
-			name:     "nil",
-			patterns: nil,
-			want:     true,
-		},
-		{
-			name:     "whitespace only",
-			patterns: []string{"  ", ""},
-			want:     true,
-		},
-		{
-			name:     "has entries",
-			patterns: []string{"example.com"},
-			want:     false,
-		},
-	}
-
-	for _, tt := range tests {
-		t.Run(tt.name, func(t *testing.T) {
-			t.Parallel()
-
-			w := allowlist.New(tt.patterns)
-			if got := w.IsEmpty(); got != tt.want {
-				t.Errorf("IsEmpty() = %v, want %v", got, tt.want)
-			}
-		})
-	}
-}
-
-func TestHostAllowList_Count(t *testing.T) {
-	t.Parallel()
-
-	tests := []struct {
-		name     string
-		patterns []string
-		want     int
-	}{
-		{
-			name:     "empty",
-			patterns: []string{},
-			want:     0,
-		},
-		{
-			name:     "exact hosts only",
-			patterns: []string{"a.com", "b.com", "c.com"},
-			want:     3,
-		},
-		{
-			name:     "suffix hosts only",
-			patterns: []string{".a.com", ".b.com"},
-			want:     2,
-		},
-		{
-			name:     "mixed",
-			patterns: []string{"exact.com", ".suffix.com"},
-			want:     2,
-		},
-	}
-
-	for _, tt := range tests {
-		t.Run(tt.name, func(t *testing.T) {
-			t.Parallel()
-
-			w := allowlist.New(tt.patterns)
-			if got := w.Count(); got != tt.want {
-				t.Errorf("Count() = %v, want %v", got, tt.want)
-			}
-		})
-	}
-}

internal/app/app.go

@@ -7,6 +7,7 @@ import (
 	"os/signal"
 	"syscall"
 
+	"github.com/prometheus/client_golang/prometheus"
 	"github.com/spf13/cobra"
 	"go.uber.org/fx"
 	"sneak.berlin/go/pixa/internal/config"
@@ -16,6 +17,7 @@ import (
 	"sneak.berlin/go/pixa/internal/healthcheck"
 	"sneak.berlin/go/pixa/internal/logger"
 	"sneak.berlin/go/pixa/internal/middleware"
+	"sneak.berlin/go/pixa/internal/providers"
 	"sneak.berlin/go/pixa/internal/server"
 )
 
@@ -61,7 +63,10 @@ func run(_ *cobra.Command, _ []string) {
 			server.New,
 			middleware.New,
 			healthcheck.New,
+			// The registry /metrics serves
+			func() prometheus.Registerer { return prometheus.DefaultRegisterer },
 		),
+		fx.Provide(providers.Constructors()...),
 		fx.Invoke(
 			func(log *logger.Logger) { log.Identify() },
 			func(*server.Server) {},

internal/config/cache_max_bytes_internal_test.go

@@ -135,10 +135,38 @@ func TestCacheMaxBytesInvalidValuesAbortStartup(t *testing.T) {
 	}
 }
 
+// TestDefaultCacheMaxBytesAppliesFloor verifies that when 75% of the
+// free space plus what the cache holds is below 500 MiB, the default is
+// floored at DefaultCacheMaxBytesFloor.
+func TestDefaultCacheMaxBytesAppliesFloor(t *testing.T) {
+	t.Parallel()
+
+	cases := []struct {
+		name      string
+		freeBytes uint64
+	}{
+		{name: "100 MiB free", freeBytes: 104857600},
+		{name: "zero free", freeBytes: 0},
+		{name: "just below floor threshold", freeBytes: 699050665},
+	}
+
+	for _, tc := range cases {
+		t.Run(tc.name, func(t *testing.T) {
+			t.Parallel()
+
+			got := DefaultCacheMaxBytes(tc.freeBytes, 0)
+			if got != DefaultCacheMaxBytesFloor {
+				t.Errorf("DefaultCacheMaxBytes = %d, want floor %d",
+					got, DefaultCacheMaxBytesFloor)
+			}
+		})
+	}
+}
+
 // TestCacheMaxBytesExplicitIsRecorded verifies that an omitted
-// cache_max_bytes is recorded as not explicit, so the cache works out
-// the default when it opens, and that an explicit zero is recorded as
-// explicit, so it disables the disk cache instead.
+// cache_max_bytes is recorded as not explicit, so pixad works out the
+// default when it builds the cache, and that an explicit zero is recorded
+// as explicit, so it disables the disk cache instead.
 func TestCacheMaxBytesExplicitIsRecorded(t *testing.T) {
 	t.Parallel()
 

internal/config/config.go

@@ -45,6 +45,37 @@ func DefaultMaxConcurrentProcessing() int {
 	return max(runtime.GOMAXPROCS(0)/2, 1) //nolint:mnd // half the CPUs
 }
 
+// DefaultCacheMaxBytesFloor is the least DefaultCacheMaxBytes returns:
+// 500 MiB. It applies only to the default, never to an explicit
+// cache_max_bytes.
+const DefaultCacheMaxBytesFloor int64 = 500 << 20
+
+// freeSpaceFractionNumerator and freeSpaceFractionDenominator express
+// the 75% share of DefaultCacheMaxBytes as integer arithmetic (dividing
+// before multiplying avoids overflow).
+const (
+	freeSpaceFractionNumerator   uint64 = 3
+	freeSpaceFractionDenominator uint64 = 4
+)
+
+// DefaultCacheMaxBytes returns the disk cache limit when cache_max_bytes
+// is omitted: 75% of the sum of freeBytes, the space free on the
+// filesystem holding the cache, and heldBytes, the bytes the cache
+// already holds, and at least DefaultCacheMaxBytesFloor. Counting what
+// the cache holds keeps the limit from shrinking as the cache fills.
+// pixad works it out when it builds the cache, as both numbers are known
+// only then.
+func DefaultCacheMaxBytes(freeBytes uint64, heldBytes int64) int64 {
+	// Both terms are at most math.MaxInt64, so the sum cannot overflow.
+	//nolint:gosec // G115: heldBytes is the cache's total, never negative
+	spaceBytes := min(freeBytes, math.MaxInt64) + uint64(heldBytes)
+
+	computed := spaceBytes / freeSpaceFractionDenominator * freeSpaceFractionNumerator
+	computed = min(computed, math.MaxInt64)
+
+	return max(int64(computed), DefaultCacheMaxBytesFloor)
+}
+
 // Configuration key names.
 const (
 	keyDebug                      = "debug"
@@ -190,18 +221,14 @@ type Config struct {
 	// address, and an explicit list replaces the default.
 	TrustedProxies []netip.Prefix
 
-	// CacheMaxBytes is the disk cache size limit in bytes. Only an
-	// explicit zero (CacheMaxBytesExplicit true) disables the disk
-	// cache. Zero with CacheMaxBytesExplicit false means
-	// cache_max_bytes was omitted, and the cache works out the default
-	// limit when it opens.
+	// CacheMaxBytes is the disk cache size limit in bytes, as set; 0
+	// turns the disk cache off.
 	CacheMaxBytes int64
 
 	// CacheMaxBytesExplicit records whether cache_max_bytes was
 	// explicitly set, in the environment or the configuration file.
-	// Explicit values are used exactly as given; for an omitted key the
-	// cache works out the default limit when it opens (see
-	// imgcache.CacheConfig.UseDefaultMaxBytes).
+	// Explicit values are used exactly as given. For an omitted key
+	// pixad works out DefaultCacheMaxBytes when it builds the cache.
 	CacheMaxBytesExplicit bool
 }
 
@@ -239,8 +266,8 @@ func New(_ fx.Lifecycle, params Params) (*Config, error) {
 		return nil, err
 	}
 
-	// An omitted cache_max_bytes is worked out and logged when the
-	// cache opens.
+	// An omitted cache_max_bytes is worked out and logged when pixad
+	// builds the cache.
 	if c.CacheMaxBytesExplicit {
 		log.Info("effective cache size limit",
 			"cache_max_bytes", c.CacheMaxBytes,
@@ -325,8 +352,8 @@ func newFromSmartConfig(sc *smartconfig.Config) (*Config, error) {
 	}
 
 	// The default for an omitted cache_max_bytes is worked out when
-	// the cache opens; here we only record whether the operator set
-	// the key explicitly.
+	// pixad builds the cache; here we only record whether the operator
+	// set the key explicitly.
 	_, c.CacheMaxBytesExplicit = lookupValue(sc, keyCacheMaxBytes)
 
 	// Build DBURL from StateDir if not explicitly set. The derived URL

internal/database/concurrent_writes_internal_test.go

@@ -104,41 +104,31 @@ func writeConcurrently(t *testing.T, db *sql.DB) {
 
 	wg.Wait()
 
-	var hits, sources int
+	var sources int
 
-	err := db.QueryRowContext(ctx, `
-		SELECT hit_count, (SELECT COUNT(*) FROM source_content)
-		FROM cache_stats WHERE id = 1
-	`).Scan(&hits, &sources)
+	err := db.QueryRowContext(ctx, `SELECT COUNT(*) FROM source_content`).
+		Scan(&sources)
 	if err != nil {
 		t.Fatalf("failed to count the writes: %v", err)
 	}
 
-	if hits != totalRequests || sources != totalRequests {
-		t.Errorf("hit_count = %d and %d source_content rows, want %d of each",
-			hits, sources, totalRequests)
+	if sources != totalRequests {
+		t.Errorf("%d source_content rows, want %d", sources, totalRequests)
 	}
 }
 
 // writeLikeOneRequest makes the writes one request and the eviction pass
-// make: it counts a cache hit, stores a source, records a transformed image
-// and deletes that record again.
+// make: it stores a source, records a transformed image and deletes that
+// record again.
 func writeLikeOneRequest(ctx context.Context, db *sql.DB, key string) error {
-	_, err := db.ExecContext(ctx,
-		`UPDATE cache_stats SET hit_count = hit_count + 1 WHERE id = 1`)
-	if err != nil {
-		return fmt.Errorf("counting a cache hit: %w", err)
-	}
-
-	_, err = db.ExecContext(ctx, `INSERT INTO source_content
-		(source_key, source_host, source_path, content_type, size_bytes)
-		VALUES (?, 'example.com', '/image.png', 'image/png', 1)`, key)
+	_, err := db.ExecContext(ctx, `INSERT INTO source_content
+		(source_key, size_bytes) VALUES (?, 1)`, key)
 	if err != nil {
 		return fmt.Errorf("storing a source: %w", err)
 	}
 
 	_, err = db.ExecContext(ctx, `INSERT INTO variant_content
-		(cache_key, size_bytes, content_type) VALUES (?, 1, 'image/png')`, key)
+		(cache_key, size_bytes) VALUES (?, 1)`, key)
 	if err != nil {
 		return fmt.Errorf("recording a transformed image: %w", err)
 	}

internal/database/database.go

@@ -82,7 +82,11 @@ func ParseMigrationVersion(filename string) (int, error) {
 	return version, nil
 }
 
-// New creates a new Database instance.
+// New opens the database and applies the migrations, so that the parts
+// fx builds after it can read the database while they are built. Its
+// stop hook closes the database. fx runs stop hooks in the reverse of the
+// order it built their parts in, so the stop hooks of those parts run
+// before this one.
 func New(lc fx.Lifecycle, params Params) (*Database, error) {
 	s := &Database{
 		log:    params.Logger.Get(),
@@ -91,20 +95,16 @@ func New(lc fx.Lifecycle, params Params) (*Database, error) {
 
 	s.log.Info("Database instantiated")
 
-	lc.Append(fx.Hook{
-		OnStart: func(ctx context.Context) error {
-			s.log.Info("Database OnStart Hook")
+	err := s.connect(context.Background())
+	if err != nil {
+		return nil, err
+	}
 
-			return s.connect(ctx)
-		},
+	lc.Append(fx.Hook{
 		OnStop: func(_ context.Context) error {
 			s.log.Info("Database OnStop Hook")
 
-			if s.db != nil {
-				return s.db.Close()
-			}
-
-			return nil
+			return s.db.Close()
 		},
 	})
 

internal/database/database_internal_test.go

@@ -159,10 +159,7 @@ func TestApplyMigrations_CreatesSchemaAndTables(t *testing.T) {
 	}
 
 	// Verify that the application tables created by 001.sql exist.
-	tables := []string{
-		"source_content", "output_content",
-		"request_cache", "negative_cache", "cache_stats",
-	}
+	tables := []string{"source_content", "negative_cache"}
 	for _, table := range tables {
 		var count int
 

internal/db/migrations/001_schema.sql

@@ -6,27 +6,22 @@
 -- is the SHA-256 of the source's host, path and query
 CREATE TABLE IF NOT EXISTS source_content (
     source_key TEXT PRIMARY KEY,
-    source_host TEXT NOT NULL,
-    source_path TEXT NOT NULL,
-    source_query TEXT NOT NULL DEFAULT '',
-    content_type TEXT NOT NULL,
     size_bytes INTEGER NOT NULL,
-    fetched_at DATETIME DEFAULT CURRENT_TIMESTAMP,
     last_accessed_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP
 );
 CREATE INDEX IF NOT EXISTS idx_source_content_last_accessed
     ON source_content(last_accessed_at);
 
 -- Processed variant blobs
--- Files stored at: cache/variants/<ab>/<cd>/<cache_key> (plus a .meta
--- sidecar with the content type). Tracked here (like source content
--- blobs above) so total cache usage is known without a directory scan,
--- and so LRU eviction has a timestamp to order on.
+-- Files stored at: cache/variants/<ab>/<cd>/<cache_key>. Tracked here
+-- (like source content blobs above) so total cache usage is known
+-- without a directory scan, and so LRU eviction has a timestamp to order
+-- on. A cache hit does not read this table: it takes the content type
+-- from the file's first bytes, and its last_accessed_at is written
+-- later, in a batch.
 CREATE TABLE IF NOT EXISTS variant_content (
     cache_key TEXT PRIMARY KEY,
     size_bytes INTEGER NOT NULL,
-    content_type TEXT NOT NULL,
-    created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
     last_accessed_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP
 );
 CREATE INDEX IF NOT EXISTS idx_variant_content_last_accessed
@@ -100,63 +95,19 @@ BEGIN
     WHERE id = 1;
 END;
 
--- Output/transformed content blobs
--- Not written: transformed images are stored in cache/variants and
--- tracked in variant_content above.
-CREATE TABLE IF NOT EXISTS output_content (
-    content_hash TEXT PRIMARY KEY,
-    content_type TEXT NOT NULL,
-    size_bytes INTEGER NOT NULL,
-    fetched_at DATETIME DEFAULT CURRENT_TIMESTAMP
-);
-
--- Request cache - maps full request params to output content
-CREATE TABLE IF NOT EXISTS request_cache (
-    id INTEGER PRIMARY KEY AUTOINCREMENT,
-    cache_key TEXT NOT NULL UNIQUE,
-    source_metadata_id INTEGER NOT NULL,
-    output_hash TEXT NOT NULL,
-    width INTEGER NOT NULL,
-    height INTEGER NOT NULL,
-    format TEXT NOT NULL,
-    quality INTEGER NOT NULL DEFAULT 85,
-    fit_mode TEXT NOT NULL DEFAULT 'cover',
-    fetched_at DATETIME DEFAULT CURRENT_TIMESTAMP,
-    access_count INTEGER NOT NULL DEFAULT 1,
-    FOREIGN KEY (source_metadata_id) REFERENCES source_metadata(id),
-    FOREIGN KEY (output_hash) REFERENCES output_content(content_hash)
-);
-CREATE INDEX IF NOT EXISTS idx_request_cache_key ON request_cache(cache_key);
-CREATE INDEX IF NOT EXISTS idx_request_cache_source ON request_cache(source_metadata_id);
-CREATE INDEX IF NOT EXISTS idx_request_cache_output ON request_cache(output_hash);
-CREATE INDEX IF NOT EXISTS idx_request_cache_fetched ON request_cache(fetched_at);
-
--- Negative cache for failed fetches (404s, timeouts, etc.)
+-- Negative cache: source URLs whose fetch failed with an upstream error
+-- status, or whose first bytes did not match their Content-Type. pixa does
+-- not fetch such a URL again until its row expires.
 CREATE TABLE IF NOT EXISTS negative_cache (
     id INTEGER PRIMARY KEY AUTOINCREMENT,
     source_host TEXT NOT NULL,
     source_path TEXT NOT NULL,
     source_query TEXT NOT NULL DEFAULT '',
-    status_code INTEGER NOT NULL,
-    error_message TEXT,
-    fetched_at DATETIME DEFAULT CURRENT_TIMESTAMP,
     expires_at DATETIME NOT NULL,
     UNIQUE(source_host, source_path, source_query)
 );
 CREATE INDEX IF NOT EXISTS idx_negative_cache_expires ON negative_cache(expires_at);
 
--- Cache statistics for monitoring
-CREATE TABLE IF NOT EXISTS cache_stats (
-    id INTEGER PRIMARY KEY CHECK (id = 1),
-    hit_count INTEGER NOT NULL DEFAULT 0,
-    miss_count INTEGER NOT NULL DEFAULT 0,
-    upstream_fetch_count INTEGER NOT NULL DEFAULT 0,
-    upstream_fetch_bytes INTEGER NOT NULL DEFAULT 0,
-    transform_count INTEGER NOT NULL DEFAULT 0,
-    last_updated_at DATETIME DEFAULT CURRENT_TIMESTAMP
-);
-INSERT OR IGNORE INTO cache_stats (id) VALUES (1);
-
 -- A random value that a login session carries from its login. pixa makes it
 -- when the table has no row, and each logout replaces it; a login session
 -- carrying another value is refused, so logging out ends every login session.

internal/encurl/encurl.go

@@ -16,9 +16,6 @@ const (
 	DefaultQuality = 85
 	DefaultFormat  = imagerequest.FormatJXL
 	DefaultFitMode = imagerequest.FitCover
-
-	// HKDF salt for URL encryption key derivation
-	urlKeySalt = "pixa-urlenc-v1"
 )
 
 // Errors returned by encurl operations.
@@ -48,23 +45,20 @@ type Generator struct {
 	key [seal.KeySize]byte
 }
 
-// NewGenerator creates an encrypted URL generator with a key derived
-// from the signing key.
-func NewGenerator(signingKey string) (*Generator, error) {
-	key, err := seal.DeriveKey([]byte(signingKey), urlKeySalt)
-	if err != nil {
-		return nil, err
-	}
-
-	return &Generator{key: key}, nil
+// NewGenerator creates an encrypted URL generator that encrypts its tokens
+// with key.
+func NewGenerator(key [seal.KeySize]byte) *Generator {
+	return &Generator{key: key}
 }
 
-// Generate creates an encrypted URL token from the request's source,
-// transform and expiry, each as the request holds it; an empty format, a
-// quality of 0 or an empty fit mode is left out of the token, and a zero
-// Expires makes a token that never expires. The token is CBOR-encoded,
+// Generate creates an encrypted URL token from the request's source and
+// transform, each as the request holds it, and from expires; an empty format,
+// a quality of 0 or an empty fit mode is left out of the token, and a zero
+// expires makes a token that never expires. The token is CBOR-encoded,
 // encrypted with NaCl secretbox, and base64url-encoded.
-func (g *Generator) Generate(req *imagerequest.Request) (string, error) {
+func (g *Generator) Generate(
+	req *imagerequest.Request, expires time.Time,
+) (string, error) {
 	p := &payload{
 		SourceHost:  req.Source.Host,
 		SourcePath:  req.Source.Path,
@@ -76,8 +70,8 @@ func (g *Generator) Generate(req *imagerequest.Request) (string, error) {
 		FitMode:     req.Transform.FitMode,
 	}
 
-	if !req.Expires.IsZero() {
-		p.ExpiresAt = req.Expires.Unix()
+	if !expires.IsZero() {
+		p.ExpiresAt = expires.Unix()
 	}
 
 	// CBOR encode the payload
@@ -91,19 +85,19 @@ func (g *Generator) Generate(req *imagerequest.Request) (string, error) {
 }
 
 // Parse decrypts and validates an encrypted URL token and returns the
-// request it holds, with default values for omitted optional fields. A token
-// that never expires gives the zero Expires.
+// request it holds, with default values for omitted optional fields, and its
+// expiry. A token that never expires gives the zero time.
 // Returns ErrExpired if the token has expired, or other errors for invalid tokens.
-func (g *Generator) Parse(token string) (*imagerequest.Request, error) {
+func (g *Generator) Parse(token string) (*imagerequest.Request, time.Time, error) {
 	// Decrypt
 	data, err := seal.Decrypt(g.key, token)
 	if err != nil {
 		if errors.Is(err, seal.ErrDecryptionFailed) ||
 			errors.Is(err, seal.ErrInvalidPayload) {
-			return nil, ErrDecryptFailed
+			return nil, time.Time{}, ErrDecryptFailed
 		}
 
-		return nil, err
+		return nil, time.Time{}, err
 	}
 
 	// CBOR decode
@@ -111,12 +105,12 @@ func (g *Generator) Parse(token string) (*imagerequest.Request, error) {
 
 	err = cbor.Unmarshal(data, &p)
 	if err != nil {
-		return nil, ErrInvalidFormat
+		return nil, time.Time{}, ErrInvalidFormat
 	}
 
 	// Check expiration (0 = never expires)
 	if p.ExpiresAt != 0 && time.Now().Unix() > p.ExpiresAt {
-		return nil, ErrExpired
+		return nil, time.Time{}, ErrExpired
 	}
 
 	format := p.Format
@@ -151,6 +145,5 @@ func (g *Generator) Parse(token string) (*imagerequest.Request, error) {
 			Quality: quality,
 			FitMode: fitMode,
 		},
-		Expires: expires,
-	}, nil
+	}, expires, nil
 }

internal/encurl/encurl_test.go

@@ -7,6 +7,7 @@ import (
 
 	"sneak.berlin/go/pixa/internal/encurl"
 	"sneak.berlin/go/pixa/internal/imagerequest"
+	"sneak.berlin/go/pixa/internal/seal"
 )
 
 // Shared test fixture strings.
@@ -19,11 +20,13 @@ const (
 func TestGenerator_GenerateAndParse(t *testing.T) {
 	t.Parallel()
 
-	gen, err := encurl.NewGenerator("test-signing-key-12345")
+	keys, err := seal.DeriveKeys("test-signing-key-12345")
 	if err != nil {
-		t.Fatalf("NewGenerator() error = %v", err)
+		t.Fatalf("seal.DeriveKeys() error = %v", err)
 	}
 
+	gen := encurl.NewGenerator(keys.EncryptedURLKey)
+
 	req := &imagerequest.Request{
 		Source: imagerequest.Source{
 			Host:  testSourceHost,
@@ -36,10 +39,10 @@ func TestGenerator_GenerateAndParse(t *testing.T) {
 			Quality: 90,
 			FitMode: imagerequest.FitContain,
 		},
-		Expires: time.Now().Add(time.Hour),
 	}
+	expires := time.Now().Add(time.Hour)
 
-	token, err := gen.Generate(req)
+	token, err := gen.Generate(req, expires)
 	if err != nil {
 		t.Fatalf("Generate() error = %v", err)
 	}
@@ -48,7 +51,7 @@ func TestGenerator_GenerateAndParse(t *testing.T) {
 		t.Fatal("Generate() returned empty token")
 	}
 
-	parsed, err := gen.Parse(token)
+	parsed, parsedExpires, err := gen.Parse(token)
 	if err != nil {
 		t.Fatalf("Parse() error = %v", err)
 	}
@@ -91,30 +94,30 @@ func TestGenerator_GenerateAndParse(t *testing.T) {
 			parsed.Transform.FitMode, req.Transform.FitMode)
 	}
 
-	if parsed.Expires.Unix() != req.Expires.Unix() {
-		t.Errorf("ExpiresAt = %d, want %d", parsed.Expires.Unix(), req.Expires.Unix())
+	if parsedExpires.Unix() != expires.Unix() {
+		t.Errorf("ExpiresAt = %d, want %d", parsedExpires.Unix(), expires.Unix())
 	}
 }
 
 func TestGenerator_Parse_Expired(t *testing.T) {
 	t.Parallel()
 
-	gen, _ := encurl.NewGenerator("test-signing-key-12345")
+	keys, _ := seal.DeriveKeys("test-signing-key-12345")
+	gen := encurl.NewGenerator(keys.EncryptedURLKey)
 
 	req := &imagerequest.Request{
 		Source: imagerequest.Source{
 			Host: testSourceHost,
 			Path: testSourcePath,
 		},
-		Expires: time.Now().Add(-time.Hour), // Already expired
 	}
 
-	token, err := gen.Generate(req)
+	token, err := gen.Generate(req, time.Now().Add(-time.Hour)) // Already expired
 	if err != nil {
 		t.Fatalf("Generate() error = %v", err)
 	}
 
-	_, err = gen.Parse(token)
+	_, _, err = gen.Parse(token)
 	if err == nil {
 		t.Error("Parse() should fail for expired token")
 	}
@@ -127,9 +130,10 @@ func TestGenerator_Parse_Expired(t *testing.T) {
 func TestGenerator_Parse_InvalidToken(t *testing.T) {
 	t.Parallel()
 
-	gen, _ := encurl.NewGenerator("test-signing-key-12345")
+	keys, _ := seal.DeriveKeys("test-signing-key-12345")
+	gen := encurl.NewGenerator(keys.EncryptedURLKey)
 
-	_, err := gen.Parse("not-a-valid-token")
+	_, _, err := gen.Parse("not-a-valid-token")
 	if err == nil {
 		t.Error("Parse() should fail for invalid token")
 	}
@@ -138,17 +142,17 @@ func TestGenerator_Parse_InvalidToken(t *testing.T) {
 func TestGenerator_Parse_TamperedToken(t *testing.T) {
 	t.Parallel()
 
-	gen, _ := encurl.NewGenerator("test-signing-key-12345")
+	keys, _ := seal.DeriveKeys("test-signing-key-12345")
+	gen := encurl.NewGenerator(keys.EncryptedURLKey)
 
 	req := &imagerequest.Request{
 		Source: imagerequest.Source{
 			Host: testSourceHost,
 			Path: testSourcePath,
 		},
-		Expires: time.Now().Add(time.Hour),
 	}
 
-	token, _ := gen.Generate(req)
+	token, _ := gen.Generate(req, time.Now().Add(time.Hour))
 
 	// Tamper with the token
 	tampered := []byte(token)
@@ -156,7 +160,7 @@ func TestGenerator_Parse_TamperedToken(t *testing.T) {
 		tampered[10] ^= 0x01
 	}
 
-	_, err := gen.Parse(string(tampered))
+	_, _, err := gen.Parse(string(tampered))
 	if err == nil {
 		t.Error("Parse() should fail for tampered token")
 	}
@@ -165,20 +169,21 @@ func TestGenerator_Parse_TamperedToken(t *testing.T) {
 func TestGenerator_Parse_WrongKey(t *testing.T) {
 	t.Parallel()
 
-	gen1, _ := encurl.NewGenerator("signing-key-1")
-	gen2, _ := encurl.NewGenerator("signing-key-2")
+	keys1, _ := seal.DeriveKeys("signing-key-1")
+	keys2, _ := seal.DeriveKeys("signing-key-2")
+	gen1 := encurl.NewGenerator(keys1.EncryptedURLKey)
+	gen2 := encurl.NewGenerator(keys2.EncryptedURLKey)
 
 	req := &imagerequest.Request{
 		Source: imagerequest.Source{
 			Host: testSourceHost,
 			Path: testSourcePath,
 		},
-		Expires: time.Now().Add(time.Hour),
 	}
 
-	token, _ := gen1.Generate(req)
+	token, _ := gen1.Generate(req, time.Now().Add(time.Hour))
 
-	_, err := gen2.Parse(token)
+	_, _, err := gen2.Parse(token)
 	if err == nil {
 		t.Error("Parse() should fail with different signing key")
 	}
@@ -187,7 +192,8 @@ func TestGenerator_Parse_WrongKey(t *testing.T) {
 func TestGenerator_Parse_Defaults(t *testing.T) {
 	t.Parallel()
 
-	gen, _ := encurl.NewGenerator("test-signing-key-12345")
+	keys, _ := seal.DeriveKeys("test-signing-key-12345")
+	gen := encurl.NewGenerator(keys.EncryptedURLKey)
 
 	// Request with only required fields - should get defaults
 	token, err := gen.Generate(&imagerequest.Request{
@@ -195,13 +201,12 @@ func TestGenerator_Parse_Defaults(t *testing.T) {
 			Host: testSourceHost,
 			Path: testSourcePath,
 		},
-		Expires: time.Now().Add(time.Hour),
-	})
+	}, time.Now().Add(time.Hour))
 	if err != nil {
 		t.Fatalf("Generate() error = %v", err)
 	}
 
-	req, err := gen.Parse(token)
+	req, _, err := gen.Parse(token)
 	if err != nil {
 		t.Fatalf("Parse() error = %v", err)
 	}
@@ -225,17 +230,17 @@ func TestGenerator_Parse_Defaults(t *testing.T) {
 func TestGenerator_TokenIsURLSafe(t *testing.T) {
 	t.Parallel()
 
-	gen, _ := encurl.NewGenerator("test-signing-key-12345")
+	keys, _ := seal.DeriveKeys("test-signing-key-12345")
+	gen := encurl.NewGenerator(keys.EncryptedURLKey)
 
 	req := &imagerequest.Request{
 		Source: imagerequest.Source{
 			Host: testSourceHost,
 			Path: testSourcePath,
 		},
-		Expires: time.Now().Add(time.Hour),
 	}
 
-	token, _ := gen.Generate(req)
+	token, _ := gen.Generate(req, time.Now().Add(time.Hour))
 
 	// Check that token only contains URL-safe characters
 	for _, c := range token {

internal/handlers/auth.go

@@ -153,20 +153,20 @@ func (s *Handlers) HandleGenerateURL() http.HandlerFunc {
 			return
 		}
 
-		// The image URL must fetch the source URL as it was written
-		req.AllowHTTP = s.config.AllowHTTP
-		fetchURL := req.SourceURL()
+		// The image URL must fetch the source URL as it was written. The
+		// fetcher fetches over http only when allow_http is set.
+		fetchURL := req.Source.URL(s.config.AllowHTTP)
 
 		if !fetchedAsWritten(parsed, fetchURL) {
 			s.renderGeneratorWithForm(w, r, "Source URL cannot be fetched as "+
-				"written: pixa would fetch "+fetchURL,
+				"written: pixa would fetch "+fetchURL.String(),
 				r.Form, http.StatusBadRequest)
 
 			return
 		}
 
 		// Generate encrypted token
-		token, err := s.encGen.Generate(req)
+		token, err := s.encGen.Generate(req, expiresAt)
 		if err != nil {
 			s.log.Error("failed to generate encrypted URL", "error", err)
 			s.renderGeneratorWithForm(w, r, "Failed to generate URL", r.Form,
@@ -226,25 +226,23 @@ func parseSourceURL(sourceURL string) (*url.URL, error) {
 // in RawPath when it is not the decoded path percent-encoded again.
 // EscapedPath ignores RawPath when it also holds a character net/url encodes,
 // such as a space, so RawPath is compared itself.
-func fetchedAsWritten(source *url.URL, fetchURL string) bool {
+func fetchedAsWritten(source, fetchURL *url.URL) bool {
 	writtenPath := source.RawPath
 	if writtenPath == "" {
 		writtenPath = source.EscapedPath()
 	}
 
-	fetched, err := url.Parse(fetchURL)
-
-	return err == nil && source.User == nil && !source.ForceQuery &&
-		fetched.EscapedPath() == writtenPath &&
-		fetched.RawQuery == source.RawQuery
+	return source.User == nil && !source.ForceQuery &&
+		fetchURL.EscapedPath() == writtenPath &&
+		fetchURL.RawQuery == source.RawQuery
 }
 
 // buildGenerateRequest parses the numeric form fields and assembles the
-// image request the encrypted URL's token is made from. ttl=0 means never
-// expires (Expires stays the zero time). A non-numeric or out-of-range
-// width, height, quality or ttl, or an unrecognized fit mode, is a client
-// error naming the offending field. The format field is passed through
-// unchecked.
+// image request the encrypted URL's token is made from, and its expiry.
+// ttl=0 means never expires (the expiry is the zero time). A non-numeric or
+// out-of-range width, height, quality or ttl, or an unrecognized fit mode, is
+// a client error naming the offending field. The format field is passed
+// through unchecked.
 func buildGenerateRequest(
 	parsed *url.URL, form url.Values,
 ) (*imagerequest.Request, time.Time, int, error) {
@@ -294,7 +292,6 @@ func buildGenerateRequest(
 			Quality: quality,
 			FitMode: fitMode,
 		},
-		Expires: expiresAt,
 	}
 
 	return req, expiresAt, ttl, nil

internal/handlers/auth_csrf_internal_test.go

@@ -14,6 +14,7 @@ import (
 
 	"sneak.berlin/go/pixa/internal/config"
 	"sneak.berlin/go/pixa/internal/encurl"
+	"sneak.berlin/go/pixa/internal/seal"
 	"sneak.berlin/go/pixa/internal/session"
 )
 
@@ -44,27 +45,17 @@ func newCSRFTestRouter(t *testing.T) (*Handlers, http.Handler) {
 
 	cfg := &config.Config{SigningKey: testSigningKey, Debug: true}
 
-	sessMgr, err := session.NewManager(testSigningKey)
+	keys, err := seal.DeriveKeys(testSigningKey)
 	if err != nil {
-		t.Fatalf("session.NewManager() error = %v", err)
-	}
-
-	encGen, err := encurl.NewGenerator(testSigningKey)
-	if err != nil {
-		t.Fatalf("encurl.NewGenerator() error = %v", err)
-	}
-
-	protect, err := newCSRFProtect(testSigningKey, cfg.Debug)
-	if err != nil {
-		t.Fatalf("newCSRFProtect() error = %v", err)
+		t.Fatalf("seal.DeriveKeys() error = %v", err)
 	}
 
 	h := &Handlers{
 		log:         slog.New(slog.DiscardHandler),
 		config:      cfg,
-		sessMgr:     sessMgr,
-		encGen:      encGen,
-		csrfProtect: protect,
+		sessMgr:     session.NewManager(keys.SessionHashKey, keys.SessionBlockKey),
+		encGen:      encurl.NewGenerator(keys.EncryptedURLKey),
+		csrfProtect: newCSRFProtect(keys.CSRFKey, cfg.Debug),
 	}
 
 	r := chi.NewRouter()

internal/handlers/auth_login_log_internal_test.go

@@ -10,6 +10,7 @@ import (
 	"testing"
 
 	"sneak.berlin/go/pixa/internal/config"
+	"sneak.berlin/go/pixa/internal/seal"
 	"sneak.berlin/go/pixa/internal/session"
 )
 
@@ -22,15 +23,15 @@ func TestLoginLogLeavesOutSubmittedKey(t *testing.T) {
 
 	var buf bytes.Buffer
 
-	sessMgr, err := session.NewManager(testSigningKey)
+	keys, err := seal.DeriveKeys(testSigningKey)
 	if err != nil {
-		t.Fatalf("session.NewManager() error = %v", err)
+		t.Fatalf("seal.DeriveKeys() error = %v", err)
 	}
 
 	h := &Handlers{
 		log:     slog.New(slog.NewJSONHandler(&buf, nil)),
 		config:  &config.Config{SigningKey: testSigningKey},
-		sessMgr: sessMgr,
+		sessMgr: session.NewManager(keys.SessionHashKey, keys.SessionBlockKey),
 	}
 
 	submittedKeys := []string{wrongKey, testSigningKey}

internal/handlers/auth_session_internal_test.go

@@ -11,6 +11,7 @@ import (
 	"time"
 
 	"sneak.berlin/go/pixa/internal/imagerequest"
+	"sneak.berlin/go/pixa/internal/seal"
 	"sneak.berlin/go/pixa/internal/session"
 )
 
@@ -232,8 +233,13 @@ func TestHandleLogout_SaveFails_Answers500(t *testing.T) {
 
 	db := setupTestDB(t)
 
+	keys, err := seal.DeriveKeys(testSigningKey)
+	if err != nil {
+		t.Fatalf("seal.DeriveKeys() error = %v", err)
+	}
+
 	sessMgr, err := session.NewManagerWithDatabase(
-		t.Context(), testSigningKey, db)
+		t.Context(), keys.SessionHashKey, keys.SessionBlockKey, db)
 	if err != nil {
 		t.Fatalf("session.NewManagerWithDatabase() error = %v", err)
 	}

internal/handlers/avif_too_large_internal_test.go

@@ -12,11 +12,11 @@ import (
 	"sneak.berlin/go/pixa/internal/imagerequest"
 )
 
-// TestAVIFOutputTooLargeAnswers400 checks that respondImageError and
-// handleImageError, which answer the errors of the two image routes, answer
-// 400 naming the AVIF pixel limit when the image processor refuses an AVIF
-// output above it, wrapped as the service wraps it. The request's source only
-// goes into a log line, so it is left empty.
+// TestAVIFOutputTooLargeAnswers400 checks that respondImageError, which
+// answers the errors of both image routes, answers 400 naming the AVIF pixel
+// limit when the image processor refuses an AVIF output above it, wrapped as
+// the service wraps it. The request's source only goes into a log line, so it
+// is left empty.
 func TestAVIFOutputTooLargeAnswers400(t *testing.T) {
 	t.Parallel()
 
@@ -25,25 +25,15 @@ func TestAVIFOutputTooLargeAnswers400(t *testing.T) {
 		imageprocessor.ErrAVIFOutputTooLarge)
 	wantBody := fmt.Sprintf("above %d pixels", imageprocessor.MaxAVIFOutputPixels)
 
-	plain := httptest.NewRecorder()
-	h.respondImageError(plain, &imagerequest.Request{}, err)
+	rec := httptest.NewRecorder()
+	h.respondImageError(rec, &imagerequest.Request{}, err)
+	t.Logf("status = %d, body = %q", rec.Code, rec.Body.String())
 
-	encrypted := httptest.NewRecorder()
-	h.handleImageError(encrypted, err)
+	if rec.Code != http.StatusBadRequest {
+		t.Errorf("status = %d, want %d", rec.Code, http.StatusBadRequest)
+	}
 
-	for name, rec := range map[string]*httptest.ResponseRecorder{
-		"respondImageError": plain, "handleImageError": encrypted,
-	} {
-		t.Logf("%s: status = %d, body = %q", name, rec.Code, rec.Body.String())
-
-		if rec.Code != http.StatusBadRequest {
-			t.Errorf("%s: status = %d, want %d",
-				name, rec.Code, http.StatusBadRequest)
-		}
-
-		if !strings.Contains(rec.Body.String(), wantBody) {
-			t.Errorf("%s: body = %q, want it to say %q",
-				name, rec.Body.String(), wantBody)
-		}
+	if !strings.Contains(rec.Body.String(), wantBody) {
+		t.Errorf("body = %q, want it to say %q", rec.Body.String(), wantBody)
 	}
 }

internal/handlers/bodylimit_internal_test.go

@@ -11,6 +11,7 @@ import (
 
 	"sneak.berlin/go/pixa/internal/config"
 	"sneak.berlin/go/pixa/internal/encurl"
+	"sneak.berlin/go/pixa/internal/seal"
 	"sneak.berlin/go/pixa/internal/session"
 )
 
@@ -32,27 +33,17 @@ func newBodyLimitTestRouter(
 
 	cfg := &config.Config{SigningKey: testSigningKey, Debug: true}
 
-	sessMgr, err := session.NewManager(testSigningKey)
+	keys, err := seal.DeriveKeys(testSigningKey)
 	if err != nil {
-		t.Fatalf("session.NewManager() error = %v", err)
-	}
-
-	encGen, err := encurl.NewGenerator(testSigningKey)
-	if err != nil {
-		t.Fatalf("encurl.NewGenerator() error = %v", err)
-	}
-
-	protect, err := newCSRFProtect(testSigningKey, cfg.Debug)
-	if err != nil {
-		t.Fatalf("newCSRFProtect() error = %v", err)
+		t.Fatalf("seal.DeriveKeys() error = %v", err)
 	}
 
 	h := &Handlers{
 		log:         slog.New(slog.DiscardHandler),
 		config:      cfg,
-		sessMgr:     sessMgr,
-		encGen:      encGen,
-		csrfProtect: protect,
+		sessMgr:     session.NewManager(keys.SessionHashKey, keys.SessionBlockKey),
+		encGen:      encurl.NewGenerator(keys.EncryptedURLKey),
+		csrfProtect: newCSRFProtect(keys.CSRFKey, cfg.Debug),
 	}
 
 	r := chi.NewRouter()

internal/handlers/cache_hit_internal_test.go

@@ -0,0 +1,278 @@
+package handlers
+
+import (
+	"bytes"
+	"context"
+	"database/sql"
+	"database/sql/driver"
+	"image"
+	"image/jpeg"
+	"log/slog"
+	"net/http"
+	"net/http/httptest"
+	"path/filepath"
+	"sync/atomic"
+	"testing"
+	"testing/fstest"
+	"time"
+
+	"github.com/go-chi/chi/v5"
+	"github.com/prometheus/client_golang/prometheus"
+
+	"sneak.berlin/go/pixa/internal/allowlist"
+	"sneak.berlin/go/pixa/internal/database"
+	"sneak.berlin/go/pixa/internal/encurl"
+	"sneak.berlin/go/pixa/internal/imageprocessor"
+	"sneak.berlin/go/pixa/internal/imageproxy"
+	"sneak.berlin/go/pixa/internal/imagerequest"
+	"sneak.berlin/go/pixa/internal/imgcache"
+	"sneak.berlin/go/pixa/internal/seal"
+	"sneak.berlin/go/pixa/internal/signature"
+)
+
+// countingHost is the allowlisted host whose photoPath the server of
+// newCountingDatabaseServer serves.
+const countingHost = "counting.example.com"
+
+// countingConnector opens connections to the SQLite database file name with
+// sqliteDriver, each one counting in statements the statements run on it.
+type countingConnector struct {
+	sqliteDriver driver.Driver
+	name         string
+	statements   *atomic.Int64
+}
+
+func (c countingConnector) Connect(context.Context) (driver.Conn, error) {
+	conn, err := c.sqliteDriver.Open(c.name)
+	if err != nil {
+		return nil, err
+	}
+
+	return countingConn{Conn: conn, statements: c.statements}, nil
+}
+
+func (c countingConnector) Driver() driver.Driver {
+	return c.sqliteDriver
+}
+
+// countingConn counts the statements run on its connection. It has only the
+// methods of driver.Conn, so database/sql prepares every query and every
+// other statement with Prepare, which counts it.
+type countingConn struct {
+	driver.Conn
+
+	statements *atomic.Int64
+}
+
+func (c countingConn) Prepare(query string) (driver.Stmt, error) {
+	c.statements.Add(1)
+
+	return c.Conn.Prepare(query)
+}
+
+// newCountingDatabase returns a database with pixa's schema, in a file of
+// its own and with one connection, and the count of the statements run on
+// it once the schema is in place.
+func newCountingDatabase(tb testing.TB) (*sql.DB, *atomic.Int64) {
+	tb.Helper()
+
+	path := filepath.Join(tb.TempDir(), "state.sqlite3")
+
+	schemaDB, err := sql.Open("sqlite", path)
+	if err != nil {
+		tb.Fatalf("opening the database: %v", err)
+	}
+
+	defer func() { _ = schemaDB.Close() }()
+
+	err = database.ApplyMigrations(tb.Context(), schemaDB, nil)
+	if err != nil {
+		tb.Fatalf("applying the migrations: %v", err)
+	}
+
+	statements := new(atomic.Int64)
+
+	db := sql.OpenDB(countingConnector{
+		sqliteDriver: schemaDB.Driver(),
+		name:         path,
+		statements:   statements,
+	})
+	db.SetMaxOpenConns(1)
+
+	tb.Cleanup(func() { _ = db.Close() })
+
+	return db, statements
+}
+
+// newCountingDatabaseServer returns a router for both image routes whose
+// image cache uses newCountingDatabase, the count of the statements run on
+// that database, and a URL on each route for a different image of
+// photoPath on countingHost. Nothing writes the cache's access times in the
+// background, so every statement counted comes from a request.
+func newCountingDatabaseServer(
+	tb testing.TB,
+) (http.Handler, *atomic.Int64, []string) {
+	tb.Helper()
+
+	db, statements := newCountingDatabase(tb)
+
+	var photo bytes.Buffer
+
+	err := jpeg.Encode(&photo, image.NewRGBA(image.Rect(0, 0, 100, 100)), nil)
+	if err != nil {
+		tb.Fatalf("encoding the photo: %v", err)
+	}
+
+	cache, err := imgcache.NewCache(db, imgcache.Config{
+		StateDir:    tb.TempDir(),
+		NegativeTTL: 5 * time.Minute,
+		MaxBytes:    1 << 30,
+	}, prometheus.NewRegistry())
+	if err != nil {
+		tb.Fatalf("imgcache.NewCache() error = %v", err)
+	}
+
+	svc, err := imageproxy.NewProxy(&imageproxy.Config{
+		Cache: cache,
+		Fetcher: newMockFetcher(fstest.MapFS{
+			countingHost + photoPath: &fstest.MapFile{Data: photo.Bytes()},
+		}),
+		Processor: imageprocessor.New(imageprocessor.Params{
+			MaxInputBytes:           50 << 20,
+			MaxConcurrentProcessing: 2,
+		}),
+		MaxResponseSize: 50 << 20,
+		Logger:          slog.New(slog.DiscardHandler),
+	})
+	if err != nil {
+		tb.Fatalf("imageproxy.NewProxy() error = %v", err)
+	}
+
+	keys, err := seal.DeriveKeys(testSigningKey)
+	if err != nil {
+		tb.Fatalf("seal.DeriveKeys() error = %v", err)
+	}
+
+	encGen := encurl.NewGenerator(keys.EncryptedURLKey)
+
+	token, err := encGen.Generate(&imagerequest.Request{
+		Source: imagerequest.Source{Host: countingHost, Path: photoPath},
+		Transform: imagerequest.Transform{
+			Size:    imagerequest.Size{Width: 40, Height: 40},
+			Format:  imagerequest.FormatJPEG,
+			Quality: 85,
+			FitMode: imagerequest.FitCover,
+		},
+	}, time.Time{})
+	if err != nil {
+		tb.Fatalf("encGen.Generate() error = %v", err)
+	}
+
+	h := &Handlers{
+		log:       slog.New(slog.DiscardHandler),
+		imgSvc:    svc,
+		encGen:    encGen,
+		signer:    signature.New(testSigningKey),
+		allowlist: allowlist.New([]string{countingHost}),
+	}
+
+	r := chi.NewRouter()
+	r.Get("/v1/image/*", h.HandleImage())
+	r.Get("/v1/e/{token}/*", h.HandleImageEnc())
+
+	return r, statements, []string{
+		"/v1/image/" + countingHost + photoPath + "/50x50.jpeg",
+		"/v1/e/" + token + "/img.jpg",
+	}
+}
+
+// serveImage sends srv a GET for target and returns the response, failing
+// the test unless it is a 200 with an image.
+func serveImage(
+	tb testing.TB, srv http.Handler, target string,
+) *httptest.ResponseRecorder {
+	tb.Helper()
+
+	rec := httptest.NewRecorder()
+	srv.ServeHTTP(rec,
+		httptest.NewRequestWithContext(tb.Context(), http.MethodGet, target, nil))
+
+	if rec.Code != http.StatusOK || rec.Body.Len() == 0 {
+		tb.Errorf("GET %s: %d with %d bytes, want 200 with an image",
+			target, rec.Code, rec.Body.Len())
+	}
+
+	return rec
+}
+
+// TestImageRoutes_CacheHitsMakeNoDatabaseCall serves an image on each image
+// route once, which makes and caches it, then serves it again from the cache.
+// Serving those cache hits must run no statement on the database.
+func TestImageRoutes_CacheHitsMakeNoDatabaseCall(t *testing.T) {
+	t.Parallel()
+
+	srv, statements, targets := newCountingDatabaseServer(t)
+
+	for _, target := range targets {
+		before := statements.Load()
+
+		serveImage(t, srv, target)
+
+		afterMiss := statements.Load()
+		t.Logf("GET %s, a miss: %d statements", target, afterMiss-before)
+
+		if afterMiss == before {
+			t.Fatalf("GET %s, a miss: no statement counted, want some", target)
+		}
+
+		for range 3 {
+			rec := serveImage(t, srv, target)
+
+			cacheStatus := rec.Header().Get("X-Pixa-Cache")
+			if cacheStatus != string(imageproxy.CacheHit) {
+				t.Errorf("GET %s: X-Pixa-Cache = %q, want %s",
+					target, cacheStatus, imageproxy.CacheHit)
+			}
+		}
+
+		if hitStatements := statements.Load() - afterMiss; hitStatements != 0 {
+			t.Errorf("GET %s: 3 cache hits ran %d statements, want 0",
+				target, hitStatements)
+		}
+	}
+}
+
+// BenchmarkHandleImage_CacheHits serves one cached image through the
+// /v1/image/ route from b.RunParallel's goroutines, and reports the cache
+// hits served per second. It fails if serving them ran any statement on the
+// database. script/bench runs it.
+func BenchmarkHandleImage_CacheHits(b *testing.B) {
+	srv, statements, targets := newCountingDatabaseServer(b)
+	target := targets[0]
+
+	serveImage(b, srv, target)
+
+	before := statements.Load()
+
+	b.ResetTimer()
+
+	b.RunParallel(func(pb *testing.PB) {
+		for pb.Next() {
+			rec := serveImage(b, srv, target)
+
+			cacheStatus := rec.Header().Get("X-Pixa-Cache")
+			if cacheStatus != string(imageproxy.CacheHit) {
+				b.Errorf("GET %s: X-Pixa-Cache = %q, want %s",
+					target, cacheStatus, imageproxy.CacheHit)
+
+				return
+			}
+		}
+	})
+
+	b.ReportMetric(float64(b.N)/b.Elapsed().Seconds(), "hits/s")
+
+	if hitStatements := statements.Load() - before; hitStatements != 0 {
+		b.Fatalf("%d cache hits ran %d statements, want 0", b.N, hitStatements)
+	}
+}

internal/handlers/cache_max_bytes_internal_test.go

@@ -1,147 +0,0 @@
-package handlers
-
-import (
-	"os"
-	"path/filepath"
-	"testing"
-
-	"go.uber.org/fx/fxtest"
-
-	"sneak.berlin/go/pixa/internal/config"
-	"sneak.berlin/go/pixa/internal/database"
-	"sneak.berlin/go/pixa/internal/globals"
-	"sneak.berlin/go/pixa/internal/logger"
-)
-
-// TestNewCacheConfigFromCacheMaxBytes checks the cache configuration
-// built from cache_max_bytes: omitted, the cache works out the default
-// limit; 0 turns the disk cache off; a positive value is the limit,
-// unchanged.
-func TestNewCacheConfigFromCacheMaxBytes(t *testing.T) {
-	t.Parallel()
-
-	const oneGiB = 1 << 30
-
-	cases := []struct {
-		name                   string
-		cacheMaxBytes          int64
-		cacheMaxBytesExplicit  bool
-		wantMaxBytes           int64
-		wantUseDefaultMaxBytes bool
-		wantDisableDiskCache   bool
-	}{
-		{
-			name:                   "cache_max_bytes omitted",
-			cacheMaxBytes:          0,
-			cacheMaxBytesExplicit:  false,
-			wantMaxBytes:           0,
-			wantUseDefaultMaxBytes: true,
-			wantDisableDiskCache:   false,
-		},
-		{
-			name:                   "cache_max_bytes: 0",
-			cacheMaxBytes:          0,
-			cacheMaxBytesExplicit:  true,
-			wantMaxBytes:           0,
-			wantUseDefaultMaxBytes: false,
-			wantDisableDiskCache:   true,
-		},
-		{
-			name:                   "cache_max_bytes: 1 GiB",
-			cacheMaxBytes:          oneGiB,
-			cacheMaxBytesExplicit:  true,
-			wantMaxBytes:           oneGiB,
-			wantUseDefaultMaxBytes: false,
-			wantDisableDiskCache:   false,
-		},
-	}
-
-	for _, tc := range cases {
-		t.Run(tc.name, func(t *testing.T) {
-			t.Parallel()
-
-			cfg := &config.Config{
-				CacheMaxBytes:         tc.cacheMaxBytes,
-				CacheMaxBytesExplicit: tc.cacheMaxBytesExplicit,
-			}
-
-			got := newCacheConfig(cfg, nil)
-			t.Logf("MaxBytes = %d, UseDefaultMaxBytes = %v, DisableDiskCache = %v",
-				got.MaxBytes, got.UseDefaultMaxBytes, got.DisableDiskCache)
-
-			if got.MaxBytes != tc.wantMaxBytes {
-				t.Errorf("MaxBytes = %d, want %d", got.MaxBytes, tc.wantMaxBytes)
-			}
-
-			if got.UseDefaultMaxBytes != tc.wantUseDefaultMaxBytes {
-				t.Errorf("UseDefaultMaxBytes = %v, want %v",
-					got.UseDefaultMaxBytes, tc.wantUseDefaultMaxBytes)
-			}
-
-			if got.DisableDiskCache != tc.wantDisableDiskCache {
-				t.Errorf("DisableDiskCache = %v, want %v",
-					got.DisableDiskCache, tc.wantDisableDiskCache)
-			}
-		})
-	}
-}
-
-// TestDiskCacheOffOnlyForExplicitZeroCacheMaxBytes starts the handlers
-// once with cache_max_bytes omitted and once with cache_max_bytes: 0,
-// and checks by whether the cache directories were created that the
-// disk cache is on in the first case and off in the second.
-func TestDiskCacheOffOnlyForExplicitZeroCacheMaxBytes(t *testing.T) {
-	t.Parallel()
-
-	cases := []struct {
-		name                  string
-		cacheMaxBytesExplicit bool
-		wantDiskCache         bool
-	}{
-		{name: "cache_max_bytes omitted", cacheMaxBytesExplicit: false, wantDiskCache: true},
-		{name: "cache_max_bytes: 0", cacheMaxBytesExplicit: true, wantDiskCache: false},
-	}
-
-	for _, tc := range cases {
-		t.Run(tc.name, func(t *testing.T) {
-			t.Parallel()
-
-			stateDir := t.TempDir()
-			cfg := &config.Config{
-				SigningKey:            testSigningKey,
-				StateDir:              stateDir,
-				DBURL:                 "file:" + filepath.Join(stateDir, "state.sqlite3"),
-				CacheMaxBytes:         0,
-				CacheMaxBytesExplicit: tc.cacheMaxBytesExplicit,
-			}
-
-			lc := fxtest.NewLifecycle(t)
-
-			log, err := logger.New(lc, logger.Params{Globals: &globals.Globals{}})
-			if err != nil {
-				t.Fatalf("logger.New() error = %v", err)
-			}
-
-			db, err := database.New(lc, database.Params{Logger: log, Config: cfg})
-			if err != nil {
-				t.Fatalf("database.New() error = %v", err)
-			}
-
-			_, err = New(lc, Params{Logger: log, Database: db, Config: cfg})
-			if err != nil {
-				t.Fatalf("New() error = %v", err)
-			}
-
-			lc.RequireStart()
-			t.Cleanup(lc.RequireStop)
-
-			_, err = os.Stat(filepath.Join(stateDir, "cache", "variants"))
-			gotDiskCache := err == nil
-
-			if gotDiskCache != tc.wantDiskCache {
-				t.Errorf("cache directories created = %v, want %v",
-					gotDiskCache, tc.wantDiskCache)
-			}
-		})
-	}
-}

internal/handlers/csrf.go

@@ -9,15 +9,10 @@ import (
 	"sneak.berlin/go/pixa/internal/seal"
 )
 
-// csrfKeySalt provides domain separation for the CSRF authentication key,
-// derived from the signing key so tokens survive restarts without extra
-// configuration and never reuse the session or encrypted-URL key material.
-const csrfKeySalt = "pixa-csrf-v1"
-
 // newCSRFProtect builds the CSRF-protection middleware for the
 // state-mutating HTML form routes. The token lives in its own cookie,
 // independent of the session cookie, so it also protects the login POST
-// where no session exists yet (login CSRF).
+// where no session exists yet (login CSRF). key signs that cookie.
 //
 // When plaintext is true (local HTTP development), requests are marked
 // plaintext so the library neither demands an https Referer nor sets a
@@ -25,13 +20,8 @@ const csrfKeySalt = "pixa-csrf-v1"
 // service runs behind a TLS-terminating proxy, so plaintext is false and
 // the library enforces its https Referer origin check.
 func newCSRFProtect(
-	signingKey string, plaintext bool,
-) (func(http.Handler) http.Handler, error) {
-	key, err := seal.DeriveKey([]byte(signingKey), csrfKeySalt)
-	if err != nil {
-		return nil, err
-	}
-
+	key [seal.KeySize]byte, plaintext bool,
+) func(http.Handler) http.Handler {
 	protect := csrf.Protect(
 		key[:],
 		csrf.Path("/"),
@@ -40,7 +30,7 @@ func newCSRFProtect(
 	)
 
 	if !plaintext {
-		return protect, nil
+		return protect
 	}
 
 	return func(next http.Handler) http.Handler {
@@ -49,7 +39,7 @@ func newCSRFProtect(
 		return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
 			protected.ServeHTTP(w, csrf.PlaintextHTTPRequest(r))
 		})
-	}, nil
+	}
 }
 
 // CSRF returns the CSRF-protection middleware for the login and generator

internal/handlers/fetcher_internal_test.go

@@ -1,61 +0,0 @@
-package handlers
-
-import (
-	"net/http"
-	"net/netip"
-	"path/filepath"
-	"testing"
-	"time"
-
-	"github.com/go-chi/chi/v5"
-	"go.uber.org/fx"
-	"go.uber.org/fx/fxtest"
-
-	"sneak.berlin/go/pixa/internal/config"
-	"sneak.berlin/go/pixa/internal/database"
-	"sneak.berlin/go/pixa/internal/globals"
-	"sneak.berlin/go/pixa/internal/healthcheck"
-	"sneak.berlin/go/pixa/internal/logger"
-)
-
-// TestHandlersBuildTheirOwnFetcherWhenNoneIsProvided builds the handlers as
-// pixad does, in an fx app that provides no fetcher, and requests an image
-// from 192.0.2.10, which is on the allowlist and in blocked_networks. The URL
-// check accepts that address; only the dialer that refuses internal
-// addresses checks blocked_networks, so the answer is 403 only if the
-// fetcher the handlers build from the config connects with that dialer. Any
-// other dialer would try to connect until the upstream fetch timeout, which
-// is short so that the test then fails quickly.
-func TestHandlersBuildTheirOwnFetcherWhenNoneIsProvided(t *testing.T) {
-	t.Parallel()
-
-	const host = "192.0.2.10"
-
-	stateDir := t.TempDir()
-	cfg := &config.Config{
-		SigningKey:           testSigningKey,
-		StateDir:             stateDir,
-		DBURL:                "file:" + filepath.Join(stateDir, "state.sqlite3"),
-		AllowlistHosts:       []string{host},
-		BlockedNetworks:      []netip.Prefix{netip.MustParsePrefix("192.0.2.0/24")},
-		UpstreamFetchTimeout: 2 * time.Second,
-		// With no connection slots, the fetch would fail before dialing.
-		UpstreamConnections: config.DefaultUpstreamConnections,
-	}
-
-	var h *Handlers
-
-	app := fxtest.New(t,
-		fx.Supply(cfg),
-		fx.Provide(globals.New, logger.New, database.New, healthcheck.New, New),
-		fx.Populate(&h),
-	)
-	app.RequireStart()
-	t.Cleanup(app.RequireStop)
-
-	r := chi.NewRouter()
-	r.Get("/v1/image/*", h.HandleImage())
-
-	rec := sendGet(t, r, photoURL(host))
-	checkErrorBody(t, rec, http.StatusForbidden, "forbidden")
-}

internal/handlers/first_bytes_internal_test.go

@@ -12,10 +12,16 @@ import (
 	"time"
 
 	"github.com/go-chi/chi/v5"
+	"github.com/prometheus/client_golang/prometheus"
+	"sneak.berlin/go/pixa/internal/allowlist"
 	"sneak.berlin/go/pixa/internal/encurl"
 	"sneak.berlin/go/pixa/internal/httpfetcher"
+	"sneak.berlin/go/pixa/internal/imageprocessor"
+	"sneak.berlin/go/pixa/internal/imageproxy"
 	"sneak.berlin/go/pixa/internal/imagerequest"
 	"sneak.berlin/go/pixa/internal/imgcache"
+	"sneak.berlin/go/pixa/internal/seal"
+	"sneak.berlin/go/pixa/internal/signature"
 )
 
 // mismatchedSourcePath is the path of the source that
@@ -25,38 +31,51 @@ import (
 const mismatchedSourcePath = "/images/source.png"
 
 // newRoutesFetchingThrough returns both image routes, and the Handlers behind
-// them, whose image service fetches through fetcher and has allowlistedHost on
-// its allowlist. The handlers and the image service log to log.
+// them, whose image service fetches through fetcher, and which have
+// allowlistedHost on their allowlist. The handlers and the image service log
+// to log.
 func newRoutesFetchingThrough(
 	t *testing.T, fetcher httpfetcher.Fetcher, log *slog.Logger,
 ) (*Handlers, http.Handler) {
 	t.Helper()
 
-	cache, err := imgcache.NewCache(setupTestDB(t), imgcache.CacheConfig{
+	cache, err := imgcache.NewCache(setupTestDB(t), imgcache.Config{
 		StateDir:    t.TempDir(),
 		NegativeTTL: 5 * time.Minute,
-	})
+		MaxBytes:    1 << 30,
+	}, prometheus.NewRegistry())
 	if err != nil {
 		t.Fatalf("imgcache.NewCache() error = %v", err)
 	}
 
-	svc, err := imgcache.NewService(&imgcache.ServiceConfig{
-		Cache:      cache,
-		Fetcher:    fetcher,
-		SigningKey: testSigningKey,
-		Allowlist:  []string{allowlistedHost},
-		Logger:     log,
+	svc, err := imageproxy.NewProxy(&imageproxy.Config{
+		Cache:   cache,
+		Fetcher: fetcher,
+		Processor: imageprocessor.New(imageprocessor.Params{
+			MaxInputBytes:           50 << 20,
+			MaxConcurrentProcessing: 2,
+		}),
+		MaxResponseSize: 50 << 20,
+		Logger:          log,
 	})
 	if err != nil {
-		t.Fatalf("imgcache.NewService() error = %v", err)
+		t.Fatalf("imageproxy.NewProxy() error = %v", err)
 	}
 
-	encGen, err := encurl.NewGenerator(testSigningKey)
+	keys, err := seal.DeriveKeys(testSigningKey)
 	if err != nil {
-		t.Fatalf("encurl.NewGenerator() error = %v", err)
+		t.Fatalf("seal.DeriveKeys() error = %v", err)
 	}
 
-	h := &Handlers{log: log, imgSvc: svc, encGen: encGen}
+	encGen := encurl.NewGenerator(keys.EncryptedURLKey)
+
+	h := &Handlers{
+		log:       log,
+		imgSvc:    svc,
+		encGen:    encGen,
+		signer:    signature.New(testSigningKey),
+		allowlist: allowlist.New([]string{allowlistedHost}),
+	}
 
 	r := chi.NewRouter()
 	r.Get("/v1/image/*", h.HandleImage())
@@ -85,7 +104,7 @@ func mismatchedSourceEncURL(t *testing.T, h *Handlers) string {
 			Size:   imagerequest.Size{Width: 50, Height: 50},
 			Format: imagerequest.FormatJPEG,
 		},
-	})
+	}, time.Time{})
 	if err != nil {
 		t.Fatalf("Generate() error = %v", err)
 	}

internal/handlers/format_auto.go

@@ -18,16 +18,17 @@ var (
 		"image/jxl, image/avif, image/webp or image/jpeg")
 )
 
-// chooseAutoFormat replaces the format auto in req with the format
-// formatForAccept chooses from r's Accept header, and adds Vary: Accept to the
+// chooseAutoFormat returns the format to serve for the requested format: for
+// the format auto, the format formatForAccept chooses from r's Accept header,
+// and any other format as it is. For auto it adds Vary: Accept to the
 // response, which then depends on that header. It answers 400 for an Accept
 // header that is not valid and 406 for one that allows none of the formats,
-// and reports whether req can be served. Any other format is left as it is.
+// and reports whether the image can be served.
 func (s *Handlers) chooseAutoFormat(
-	w http.ResponseWriter, r *http.Request, req *imagerequest.Request,
-) bool {
-	if req.Transform.Format != imagerequest.FormatAuto {
-		return true
+	w http.ResponseWriter, r *http.Request, requested imagerequest.Format,
+) (imagerequest.Format, bool) {
+	if requested != imagerequest.FormatAuto {
+		return requested, true
 	}
 
 	w.Header().Add("Vary", "Accept")
@@ -36,18 +37,16 @@ func (s *Handlers) chooseAutoFormat(
 	if errors.Is(err, errNotAcceptable) {
 		s.respondError(w, err.Error(), http.StatusNotAcceptable)
 
-		return false
+		return "", false
 	}
 
 	if err != nil {
 		s.respondError(w, err.Error(), http.StatusBadRequest)
 
-		return false
+		return "", false
 	}
 
-	req.Transform.Format = format
-
-	return true
+	return format, true
 }
 
 // formatForAccept returns the format an auto URL is served in for the Accept

internal/handlers/format_auto_internal_test.go

@@ -75,11 +75,13 @@ func TestFormatForAccept(t *testing.T) {
 
 // autoPhotoURLs returns a signed /v1/image/ URL and an encrypted /v1/e/ URL,
 // both valid for a minute, for the JPEG at photoPath on signedHost at 50x50 in
-// the format auto, made with h's image service and generator.
+// the format auto. The first is signed with testSigningKey, the key the
+// handlers that newSignedHostServer builds check signatures with. The second
+// is made with h's generator.
 func autoPhotoURLs(t *testing.T, h *Handlers) (string, string) {
 	t.Helper()
 
-	signedURL, err := h.imgSvc.GenerateSignedURL("", &imagerequest.Request{
+	signedURL := generateSignedURL(testSigningKey, &imagerequest.Request{
 		Source: imagerequest.Source{
 			Host: signedHost,
 			Path: photoPath,
@@ -89,9 +91,6 @@ func autoPhotoURLs(t *testing.T, h *Handlers) (string, string) {
 			Format: imagerequest.FormatAuto,
 		},
 	}, time.Minute)
-	if err != nil {
-		t.Fatalf("GenerateSignedURL() error = %v", err)
-	}
 
 	token, err := h.encGen.Generate(&imagerequest.Request{
 		Source: imagerequest.Source{
@@ -102,8 +101,7 @@ func autoPhotoURLs(t *testing.T, h *Handlers) (string, string) {
 			Size:   imagerequest.Size{Width: 50, Height: 50},
 			Format: imagerequest.FormatAuto,
 		},
-		Expires: time.Now().Add(time.Minute),
-	})
+	}, time.Now().Add(time.Minute))
 	if err != nil {
 		t.Fatalf("Generate() error = %v", err)
 	}
@@ -188,9 +186,9 @@ func TestFormatAuto_ChosenFromAccept(t *testing.T) {
 func TestFormatAuto_SignatureCoversAuto(t *testing.T) {
 	t.Parallel()
 
-	h, srv := newSignedHostServer(t, slog.New(slog.DiscardHandler))
+	_, srv := newSignedHostServer(t, slog.New(slog.DiscardHandler))
 
-	signedForAVIF, err := h.imgSvc.GenerateSignedURL("", &imagerequest.Request{
+	signedForAVIF := generateSignedURL(testSigningKey, &imagerequest.Request{
 		Source: imagerequest.Source{
 			Host: signedHost,
 			Path: photoPath,
@@ -200,9 +198,6 @@ func TestFormatAuto_SignatureCoversAuto(t *testing.T) {
 			Format: imagerequest.FormatAVIF,
 		},
 	}, time.Minute)
-	if err != nil {
-		t.Fatalf("GenerateSignedURL() error = %v", err)
-	}
 
 	target := strings.Replace(signedForAVIF, "/50x50.avif?", "/50x50.auto?", 1)
 	if target == signedForAVIF {

internal/handlers/handlers.go

@@ -4,7 +4,6 @@ package handlers
 import (
 	"context"
 	"encoding/json"
-	"errors"
 	"log/slog"
 	"net/http"
 	"time"
@@ -12,13 +11,13 @@ import (
 	"go.uber.org/fx"
 	"sneak.berlin/go/pixa/internal/allowlist"
 	"sneak.berlin/go/pixa/internal/config"
-	"sneak.berlin/go/pixa/internal/database"
 	"sneak.berlin/go/pixa/internal/encurl"
 	"sneak.berlin/go/pixa/internal/healthcheck"
-	"sneak.berlin/go/pixa/internal/httpfetcher"
-	"sneak.berlin/go/pixa/internal/imgcache"
+	"sneak.berlin/go/pixa/internal/imageproxy"
 	"sneak.berlin/go/pixa/internal/logger"
+	"sneak.berlin/go/pixa/internal/seal"
 	"sneak.berlin/go/pixa/internal/session"
+	"sneak.berlin/go/pixa/internal/signature"
 )
 
 // Params defines dependencies for Handlers.
@@ -27,70 +26,48 @@ type Params struct {
 
 	Logger      *logger.Logger
 	Healthcheck *healthcheck.Healthcheck
-	Database    *database.Database
 	Config      *config.Config
+	Keys        seal.Keys
 
-	// Fetcher, when provided, fetches upstream images in place of the
-	// fetcher the handlers build from the config. Only tests provide one;
-	// pixad does not.
-	Fetcher httpfetcher.Fetcher `optional:"true"`
+	ImageService          *imageproxy.Proxy
+	SessionManager        *session.Manager
+	EncryptedURLGenerator *encurl.Generator
 }
 
 // Handlers provides HTTP request handlers.
 type Handlers struct {
 	log         *slog.Logger
 	hc          *healthcheck.Healthcheck
-	db          *database.Database
 	config      *config.Config
-	fetcher     httpfetcher.Fetcher
-	imgSvc      *imgcache.Service
-	imgCache    *imgcache.Cache
+	imgSvc      *imageproxy.Proxy
 	sessMgr     *session.Manager
 	encGen      *encurl.Generator
 	csrfProtect func(http.Handler) http.Handler
 
+	// signer checks the signature of a /v1/image/ URL with signing_key.
+	signer *signature.Signer
+	// allowlist matches the hosts of allowlist_hosts, which a /v1/image/
+	// URL may name without a signature.
+	allowlist *allowlist.HostAllowList
 	// refererBlocklist matches the hosts of referer_blocklist; its IsAllowed
 	// reports whether a URL's host is on that list.
 	refererBlocklist *allowlist.HostAllowList
 }
 
 // New creates a new Handlers instance.
-func New(lc fx.Lifecycle, params Params) (*Handlers, error) {
-	csrfProtect, err := newCSRFProtect(params.Config.SigningKey, params.Config.Debug)
-	if err != nil {
-		return nil, err
-	}
-
-	s := &Handlers{
+func New(_ fx.Lifecycle, params Params) *Handlers {
+	return &Handlers{
 		log:              params.Logger.Get(),
 		hc:               params.Healthcheck,
-		db:               params.Database,
 		config:           params.Config,
-		fetcher:          params.Fetcher,
-		csrfProtect:      csrfProtect,
+		imgSvc:           params.ImageService,
+		sessMgr:          params.SessionManager,
+		encGen:           params.EncryptedURLGenerator,
+		csrfProtect:      newCSRFProtect(params.Keys.CSRFKey, params.Config.Debug),
+		signer:           signature.New(params.Config.SigningKey),
+		allowlist:        allowlist.New(params.Config.AllowlistHosts),
 		refererBlocklist: allowlist.New(params.Config.RefererBlocklist),
 	}
-
-	lc.Append(fx.Hook{
-		//nolint:contextcheck // the cache's goroutines outlive OnStart; OnStop stops them
-		OnStart: func(_ context.Context) error {
-			return s.initImageService()
-		},
-		// The pending counts are written here, before the database's stop
-		// hook closes it.
-		OnStop: func(ctx context.Context) error {
-			if s.imgCache == nil {
-				return nil
-			}
-
-			return errors.Join(
-				s.imgCache.StopEviction(ctx),
-				s.imgCache.StopPendingCountWrites(ctx),
-			)
-		},
-	})
-
-	return s, nil
 }
 
 // WaitForProcessing waits until no image is being processed, or until ctx
@@ -99,94 +76,6 @@ func (s *Handlers) WaitForProcessing(ctx context.Context) int {
 	return s.imgSvc.WaitForProcessing(ctx)
 }
 
-// newCacheConfig builds the image cache's configuration from cfg.
-// cache_max_bytes: 0 disables the disk cache entirely; any other value
-// is the eviction limit in bytes; when it is omitted, the cache works
-// out the default limit itself.
-func newCacheConfig(cfg *config.Config, log *slog.Logger) imgcache.CacheConfig {
-	return imgcache.CacheConfig{
-		StateDir:           cfg.StateDir,
-		NegativeTTL:        imgcache.DefaultNegativeTTL,
-		MaxBytes:           cfg.CacheMaxBytes,
-		UseDefaultMaxBytes: !cfg.CacheMaxBytesExplicit,
-		DisableDiskCache:   cfg.CacheMaxBytesExplicit && cfg.CacheMaxBytes == 0,
-		Logger:             log,
-	}
-}
-
-// initImageService initializes the image cache and service.
-func (s *Handlers) initImageService() error {
-	cache, err := imgcache.NewCache(s.db.DB(), newCacheConfig(s.config, s.log))
-	if err != nil {
-		return err
-	}
-
-	s.imgCache = cache
-
-	// Background eviction: startup reconciliation, then periodic and
-	// write-pressure passes, and the removal of expired negative cache
-	// rows, which is all it does when the disk cache is disabled.
-	cache.StartEviction(imgcache.DefaultEvictionInterval)
-
-	// Writes the counts requests could not write by their deadline
-	cache.StartPendingCountWrites()
-
-	// Create the fetcher config
-	fetcherCfg := httpfetcher.DefaultConfig()
-	fetcherCfg.AllowHTTP = s.config.AllowHTTP
-	fetcherCfg.Timeout = s.config.UpstreamFetchTimeout
-	fetcherCfg.MaxResponseSize = s.config.UpstreamMaxResponseSize
-
-	if s.config.UpstreamConnectionsPerHost > 0 {
-		fetcherCfg.MaxConnectionsPerHost = s.config.UpstreamConnectionsPerHost
-	}
-
-	fetcherCfg.MaxConnections = s.config.UpstreamConnections
-	fetcherCfg.BlockedNetworks = s.config.BlockedNetworks
-
-	// Create the service. With no fetcher provided, it builds its own from
-	// fetcherCfg.
-	svc, err := imgcache.NewService(&imgcache.ServiceConfig{
-		Cache:                   cache,
-		FetcherConfig:           fetcherCfg,
-		Fetcher:                 s.fetcher,
-		SigningKey:              s.config.SigningKey,
-		Allowlist:               s.config.AllowlistHosts,
-		MaxConcurrentProcessing: s.config.MaxConcurrentProcessing,
-		Logger:                  s.log,
-	})
-	if err != nil {
-		return err
-	}
-
-	s.imgSvc = svc
-	s.log.Info("image service initialized")
-
-	// Initialize session manager (signing key is validated at config load
-	// time). Session cookies are always Secure/HttpOnly/SameSite=Strict.
-	// The random value that each logout replaces is saved in the database,
-	// so it outlasts a restart.
-	sessMgr, err := session.NewManagerWithDatabase(
-		context.Background(), s.config.SigningKey, s.db.DB())
-	if err != nil {
-		return err
-	}
-
-	s.sessMgr = sessMgr
-
-	// Initialize encrypted URL generator
-	encGen, err := encurl.NewGenerator(s.config.SigningKey)
-	if err != nil {
-		return err
-	}
-
-	s.encGen = encGen
-
-	s.log.Info("session manager and URL generator initialized")
-
-	return nil
-}
-
 func (s *Handlers) respondJSON(w http.ResponseWriter, data any, status int) {
 	w.Header().Set("Content-Type", "application/json")
 	w.WriteHeader(status)

internal/handlers/handlers_internal_test.go

@@ -18,15 +18,20 @@ import (
 	"time"
 
 	"github.com/go-chi/chi/v5"
+	"github.com/prometheus/client_golang/prometheus"
+	"sneak.berlin/go/pixa/internal/allowlist"
 	"sneak.berlin/go/pixa/internal/database"
 	"sneak.berlin/go/pixa/internal/httpfetcher"
+	"sneak.berlin/go/pixa/internal/imageprocessor"
+	"sneak.berlin/go/pixa/internal/imageproxy"
+	"sneak.berlin/go/pixa/internal/imagerequest"
 	"sneak.berlin/go/pixa/internal/imgcache"
+	"sneak.berlin/go/pixa/internal/signature"
 )
 
 // testFixtures contains test data.
 type testFixtures struct {
 	handler  *Handlers
-	service  *imgcache.Service
 	goodHost string
 }
 
@@ -45,32 +50,37 @@ func setupTestHandler(t *testing.T) *testFixtures {
 	tmpDir := t.TempDir()
 	db := setupTestDB(t)
 
-	cache, err := imgcache.NewCache(db, imgcache.CacheConfig{
+	cache, err := imgcache.NewCache(db, imgcache.Config{
 		StateDir:    tmpDir,
 		NegativeTTL: 5 * time.Minute,
-	})
+		MaxBytes:    1 << 30,
+	}, prometheus.NewRegistry())
 	if err != nil {
 		t.Fatalf("failed to create cache: %v", err)
 	}
 
-	svc, err := imgcache.NewService(&imgcache.ServiceConfig{
-		Cache:      cache,
-		Fetcher:    newMockFetcher(mockFS),
-		SigningKey: "test-signing-key-must-be-32-chars",
-		Allowlist:  []string{goodHost},
+	svc, err := imageproxy.NewProxy(&imageproxy.Config{
+		Cache:   cache,
+		Fetcher: newMockFetcher(mockFS),
+		Processor: imageprocessor.New(imageprocessor.Params{
+			MaxInputBytes:           50 << 20,
+			MaxConcurrentProcessing: 2,
+		}),
+		MaxResponseSize: 50 << 20,
 	})
 	if err != nil {
-		t.Fatalf("failed to create service: %v", err)
+		t.Fatalf("failed to create proxy: %v", err)
 	}
 
 	h := &Handlers{
-		imgSvc: svc,
-		log:    slog.Default(),
+		imgSvc:    svc,
+		log:       slog.Default(),
+		signer:    signature.New("test-signing-key-must-be-32-chars"),
+		allowlist: allowlist.New([]string{goodHost}),
 	}
 
 	return &testFixtures{
 		handler:  h,
-		service:  svc,
 		goodHost: goodHost,
 	}
 }
@@ -121,10 +131,9 @@ func newMockFetcher(fs fs.FS) *mockFetcher {
 }
 
 func (f *mockFetcher) Fetch(
-	_ context.Context, url string,
+	_ context.Context, source imagerequest.Source,
 ) (*httpfetcher.FetchResult, error) {
-	// Remove https:// prefix
-	path := url[8:] // Remove "https://"
+	path := source.Host + source.Path
 
 	data, err := fs.ReadFile(f.fs, path)
 	if err != nil {

internal/handlers/image.go

@@ -7,14 +7,15 @@ import (
 	"net/http"
 	"net/url"
 	"strconv"
+	"strings"
 	"time"
 
 	"github.com/go-chi/chi/v5/middleware"
 	"sneak.berlin/go/pixa/internal/encurl"
 	"sneak.berlin/go/pixa/internal/httpfetcher"
 	"sneak.berlin/go/pixa/internal/imageprocessor"
+	"sneak.berlin/go/pixa/internal/imageproxy"
 	"sneak.berlin/go/pixa/internal/imagerequest"
-	"sneak.berlin/go/pixa/internal/imgcache"
 	"sneak.berlin/go/pixa/internal/magic"
 )
 
@@ -27,13 +28,13 @@ func (s *Handlers) HandleImage() http.HandlerFunc {
 			return
 		}
 
-		req, ok := s.parseImageRequest(w, r)
+		req, sig, expires, ok := s.parseImageRequest(w, r)
 		if !ok {
 			return
 		}
 
 		// Validate signature if required
-		err := s.imgSvc.ValidateRequest(req)
+		err := s.validateSignature(req, sig, expires)
 		if err != nil {
 			s.log.Warn("signature validation failed",
 				"host", req.Source.Host,
@@ -45,27 +46,7 @@ func (s *Handlers) HandleImage() http.HandlerFunc {
 			return
 		}
 
-		// The signature covers the format auto, not the format chosen
-		if !s.chooseAutoFormat(w, r, req) {
-			return
-		}
-
-		// Get cache key for logging
-		cacheKey := imagerequest.CacheKey(req)
-
-		// Get the image (from cache or fetch/process)
-		startTime := time.Now()
-
-		resp, err := s.imgSvc.Get(r.Context(), req)
-		if err != nil {
-			s.respondImageError(w, req, err)
-
-			return
-		}
-
-		defer func() { _ = resp.Content.Close() }()
-
-		s.writeImageResponse(w, r, req, resp, cacheKey, startTime)
+		s.serveImage(w, r, req, expires)
 	}
 }
 
@@ -82,11 +63,11 @@ func (s *Handlers) HandleRobotsTxt() http.HandlerFunc {
 }
 
 // parseImageRequest parses the image URL's path and query parameters into
-// an image request. On invalid input it writes an error response and
-// returns false.
+// an image request, and returns it with the URL's signature and expiry. On
+// invalid input it writes an error response and returns false.
 func (s *Handlers) parseImageRequest(
 	w http.ResponseWriter, r *http.Request,
-) (*imagerequest.Request, bool) {
+) (*imagerequest.Request, string, time.Time, bool) {
 	// The path is parsed decoded once, as r.URL.Path holds it, and not from
 	// chi's wildcard: chi routes on the path as the client wrote it whenever
 	// that is not Go's own encoding of it, so its wildcard would still hold
@@ -100,7 +81,7 @@ func (s *Handlers) parseImageRequest(
 		)
 		s.respondError(w, "invalid image URL: "+err.Error(), http.StatusBadRequest)
 
-		return nil, false
+		return nil, "", time.Time{}, false
 	}
 
 	// Parse signature params from query string. r.URL.Query() would silently
@@ -113,7 +94,7 @@ func (s *Handlers) parseImageRequest(
 		s.respondError(w, fmt.Sprintf("invalid query string %q: %v",
 			r.URL.RawQuery, err), http.StatusBadRequest)
 
-		return nil, false
+		return nil, "", time.Time{}, false
 	}
 
 	for name, values := range query {
@@ -121,17 +102,15 @@ func (s *Handlers) parseImageRequest(
 			s.respondError(w, fmt.Sprintf("invalid %s: given more than once",
 				name), http.StatusBadRequest)
 
-			return nil, false
+			return nil, "", time.Time{}, false
 		}
 	}
 
-	req.Signature = query.Get("sig")
-
-	req.Expires, err = parseExpires(query)
+	expires, err := parseExpires(query)
 	if err != nil {
 		s.respondError(w, err.Error(), http.StatusBadRequest)
 
-		return nil, false
+		return nil, "", time.Time{}, false
 	}
 
 	// Parse optional quality and fit params. Only a q missing from the URL is
@@ -143,7 +122,7 @@ func (s *Handlers) parseImageRequest(
 		s.respondError(w, `invalid q: not a number, got ""`,
 			http.StatusBadRequest)
 
-		return nil, false
+		return nil, "", time.Time{}, false
 	}
 
 	req.Transform.Quality, err = parseFormInt(query, "q",
@@ -152,7 +131,7 @@ func (s *Handlers) parseImageRequest(
 		s.respondError(w, fmt.Sprintf("%v, got %q", err, qStr),
 			http.StatusBadRequest)
 
-		return nil, false
+		return nil, "", time.Time{}, false
 	}
 
 	// Only a fit missing from the URL is cover. A fit in the URL that is not a
@@ -162,7 +141,7 @@ func (s *Handlers) parseImageRequest(
 	if query.Has("fit") && req.Transform.FitMode == "" {
 		s.respondError(w, `invalid fit: not a fit mode, got ""`, http.StatusBadRequest)
 
-		return nil, false
+		return nil, "", time.Time{}, false
 	}
 
 	// Default fit mode if not set
@@ -179,15 +158,40 @@ func (s *Handlers) parseImageRequest(
 		s.respondError(w, "invalid image request: "+err.Error(),
 			http.StatusBadRequest)
 
-		return nil, false
+		return nil, "", time.Time{}, false
 	}
 
-	return req, true
+	return req, query.Get("sig"), expires, true
+}
+
+// validateSignature returns nil when the source host of req is on
+// allowlist_hosts and names no port. Otherwise it verifies sig, the URL's
+// signature of req, and expires, the URL's expiry.
+func (s *Handlers) validateSignature(
+	req *imagerequest.Request, sig string, expires time.Time,
+) error {
+	// An allowlist entry names no port, so a host followed by ":", with or
+	// without a port, needs a signature. Go's URL type reads the port as the
+	// fetcher does: in an IPv6 address written without brackets, such as
+	// 2001:db8::1:8443, it is what follows the last ":". This is checked here
+	// and not in allowlist.IsAllowed, which also matches referer_blocklist,
+	// where a Referer must be blocked whatever its port. Port and IsAllowed
+	// read only the host of a URL.
+	sourceURL := &url.URL{Host: req.Source.Host}
+	hostHasPort := sourceURL.Port() != "" ||
+		strings.HasSuffix(sourceURL.Host, ":")
+
+	if !hostHasPort && s.allowlist.IsAllowed(sourceURL) {
+		return nil
+	}
+
+	// Signature required for non-allowed hosts
+	return s.signer.Verify(req, sig, expires)
 }
 
 // parseExpires reads the exp query parameter, a Unix time in seconds. An exp
-// missing from the URL gives the zero time, which the signature check takes
-// as no expiration. An exp in the URL that is not a whole number, an empty
+// missing from the URL gives the zero time, which the signature check refuses
+// as a missing exp. An exp in the URL that is not a whole number, an empty
 // one included, is an error naming exp and the value.
 func parseExpires(query url.Values) (time.Time, error) {
 	if !query.Has("exp") {
@@ -215,7 +219,11 @@ func (s *Handlers) HandleTooManyRequests() http.HandlerFunc {
 	}
 }
 
-// respondImageError maps image retrieval errors to HTTP responses.
+// respondImageError answers err, the image service's error for req, on both
+// image routes. It logs a 500 at error level, as pixa's own failure. It logs
+// the other answers at warn level, since the client, the upstream or a full
+// limit caused them. The 415 for a source refused for its first bytes is not
+// logged here, because the image service logs it.
 func (s *Handlers) respondImageError(
 	w http.ResponseWriter, req *imagerequest.Request, err error,
 ) {
@@ -231,59 +239,53 @@ func (s *Handlers) respondImageError(
 		return
 	}
 
-	s.log.Error("failed to get image",
+	var (
+		status  int
+		message string
+	)
+
+	switch {
+	case errors.Is(err, httpfetcher.ErrSSRFBlocked):
+		status, message = http.StatusForbidden, "forbidden"
+	case errors.Is(err, httpfetcher.ErrUpstreamError):
+		status, message = http.StatusBadGateway, "upstream error"
+	case errors.Is(err, httpfetcher.ErrUpstreamTimeout):
+		status, message = http.StatusGatewayTimeout, "upstream timeout"
+	case errors.Is(err, httpfetcher.ErrTooManyConnections),
+		errors.Is(err, imageprocessor.ErrTooManyImages),
+		errors.Is(err, imageproxy.ErrProcessingTimedOut):
+		status = http.StatusServiceUnavailable
+		message = "server busy, try again later"
+	case errors.Is(err, httpfetcher.ErrInvalidContentType),
+		errors.Is(err, imageprocessor.ErrUnsupportedInputFormat):
+		status = http.StatusUnsupportedMediaType
+		message = "unsupported source format"
+	case errors.Is(err, imageprocessor.ErrOutputTooLarge):
+		status = http.StatusBadRequest
+		message = fmt.Sprintf("size too large for the source: the side "+
+			"computed from its proportions is above %d",
+			imageprocessor.MaxOutputDimension)
+	case errors.Is(err, imageprocessor.ErrAVIFOutputTooLarge):
+		status = http.StatusBadRequest
+		message = fmt.Sprintf("size too large for AVIF: the output "+
+			"would be above %d pixels", imageprocessor.MaxAVIFOutputPixels)
+	default:
+		s.log.Error("failed to get image",
+			"host", req.Source.Host,
+			"path", req.Source.Path,
+			"error", err,
+		)
+		s.respondError(w, "internal error", http.StatusInternalServerError)
+
+		return
+	}
+
+	s.log.Warn("failed to get image",
 		"host", req.Source.Host,
 		"path", req.Source.Path,
 		"error", err,
 	)
-
-	// Check for specific error types
-	if errors.Is(err, httpfetcher.ErrSSRFBlocked) {
-		s.respondError(w, "forbidden", http.StatusForbidden)
-
-		return
-	}
-
-	if errors.Is(err, httpfetcher.ErrUpstreamError) {
-		s.respondError(w, "upstream error", http.StatusBadGateway)
-
-		return
-	}
-
-	if errors.Is(err, httpfetcher.ErrTooManyConnections) ||
-		errors.Is(err, imageprocessor.ErrTooManyImages) ||
-		errors.Is(err, imgcache.ErrProcessingTimedOut) {
-		s.respondError(w, "server busy, try again later",
-			http.StatusServiceUnavailable)
-
-		return
-	}
-
-	if errors.Is(err, httpfetcher.ErrInvalidContentType) ||
-		errors.Is(err, imageprocessor.ErrUnsupportedInputFormat) {
-		s.respondError(w, "unsupported source format",
-			http.StatusUnsupportedMediaType)
-
-		return
-	}
-
-	if errors.Is(err, imageprocessor.ErrOutputTooLarge) {
-		s.respondError(w, fmt.Sprintf("size too large for the source: the side "+
-			"computed from its proportions is above %d",
-			imageprocessor.MaxOutputDimension), http.StatusBadRequest)
-
-		return
-	}
-
-	if errors.Is(err, imageprocessor.ErrAVIFOutputTooLarge) {
-		s.respondError(w, fmt.Sprintf("size too large for AVIF: the output "+
-			"would be above %d pixels", imageprocessor.MaxAVIFOutputPixels),
-			http.StatusBadRequest)
-
-		return
-	}
-
-	s.respondError(w, "internal error", http.StatusInternalServerError)
+	s.respondError(w, message, status)
 }
 
 // cacheControl returns the Cache-Control header for an image served through a
@@ -340,13 +342,34 @@ func notModified(w http.ResponseWriter, r *http.Request, etag string) bool {
 	return true
 }
 
-// writeImageResponse writes headers and streams the image content,
-// handling conditional and HEAD requests.
-func (s *Handlers) writeImageResponse(
+// serveImage serves the image that req asks for. Both image routes call it
+// once they have the request and its URL's expiry, which ends the
+// Cache-Control max-age. It chooses the format for the format auto, answers
+// If-None-Match with 304 and HEAD with the headers only, and answers an error
+// of the image service with respondImageError.
+func (s *Handlers) serveImage(
 	w http.ResponseWriter, r *http.Request,
-	req *imagerequest.Request, resp *imgcache.ImageResponse,
-	cacheKey imagerequest.VariantKey, startTime time.Time,
+	req *imagerequest.Request, expires time.Time,
 ) {
+	// The signature or the token covers the format auto, not the format chosen
+	format, ok := s.chooseAutoFormat(w, r, req.Transform.Format)
+	if !ok {
+		return
+	}
+
+	req.Transform.Format = format
+
+	startTime := time.Now()
+
+	resp, err := s.imgSvc.Get(r.Context(), req)
+	if err != nil {
+		s.respondImageError(w, req, err)
+
+		return
+	}
+
+	defer func() { _ = resp.Content.Close() }()
+
 	// Set response headers
 	w.Header().Set("Content-Type", resp.ContentType)
 
@@ -355,7 +378,7 @@ func (s *Handlers) writeImageResponse(
 	}
 
 	// Cache control headers
-	w.Header().Set("Cache-Control", cacheControl(req.Expires))
+	w.Header().Set("Cache-Control", cacheControl(expires))
 	w.Header().Set("X-Pixa-Cache", string(resp.CacheStatus))
 
 	if notModified(w, r, resp.ETag) {
@@ -372,18 +395,28 @@ func (s *Handlers) writeImageResponse(
 	// Stream the response
 	w.WriteHeader(http.StatusOK)
 
+	// A failed write is most often a client that has gone away, not pixa's
+	// failure, so it is logged at warn level
 	servedBytes, err := io.Copy(w, resp.Content)
 	if err != nil {
-		s.log.Error("failed to write response",
+		s.log.Warn("failed to write response",
+			"request_id", middleware.GetReqID(r.Context()),
+			"host", req.Source.Host,
+			"path", req.Source.Path,
+			"served_bytes", servedBytes,
 			"error", err,
 		)
+
+		return
 	}
 
 	// Log cache status and timing after serving
 	duration := time.Since(startTime)
 	s.log.Info("image served",
 		"request_id", middleware.GetReqID(r.Context()),
-		"cache_key", cacheKey,
+		"cache_key", imagerequest.CacheKey(req),
+		"host", req.Source.Host,
+		"path", req.Source.Path,
 		"cache_status", resp.CacheStatus,
 		"duration_ms", duration.Milliseconds(),
 		"format", req.Transform.Format,

internal/handlers/image_cache_control_internal_test.go

@@ -12,10 +12,16 @@ import (
 	"time"
 
 	"github.com/go-chi/chi/v5"
+	"github.com/prometheus/client_golang/prometheus"
 
+	"sneak.berlin/go/pixa/internal/allowlist"
 	"sneak.berlin/go/pixa/internal/encurl"
+	"sneak.berlin/go/pixa/internal/imageprocessor"
+	"sneak.berlin/go/pixa/internal/imageproxy"
 	"sneak.berlin/go/pixa/internal/imagerequest"
 	"sneak.berlin/go/pixa/internal/imgcache"
+	"sneak.berlin/go/pixa/internal/seal"
+	"sneak.berlin/go/pixa/internal/signature"
 )
 
 // photoPath is the path of the JPEG that newSignedHostServer serves.
@@ -30,37 +36,46 @@ func newSignedHostServer(
 ) (*Handlers, http.Handler) {
 	t.Helper()
 
-	cache, err := imgcache.NewCache(setupTestDB(t), imgcache.CacheConfig{
+	cache, err := imgcache.NewCache(setupTestDB(t), imgcache.Config{
 		StateDir:    t.TempDir(),
 		NegativeTTL: 5 * time.Minute,
-	})
+		MaxBytes:    1 << 30,
+	}, prometheus.NewRegistry())
 	if err != nil {
 		t.Fatalf("imgcache.NewCache() error = %v", err)
 	}
 
 	jpegData := generateTestJPEG(t, 100, 100, color.RGBA{255, 0, 0, 255})
 
-	svc, err := imgcache.NewService(&imgcache.ServiceConfig{
+	svc, err := imageproxy.NewProxy(&imageproxy.Config{
 		Cache: cache,
 		Fetcher: newMockFetcher(fstest.MapFS{
 			signedHost + photoPath: &fstest.MapFile{Data: jpegData},
 		}),
-		SigningKey: testSigningKey,
-		Logger:     log,
+		Processor: imageprocessor.New(imageprocessor.Params{
+			MaxInputBytes:           50 << 20,
+			MaxConcurrentProcessing: 2,
+		}),
+		MaxResponseSize: 50 << 20,
+		Logger:          log,
 	})
 	if err != nil {
-		t.Fatalf("imgcache.NewService() error = %v", err)
+		t.Fatalf("imageproxy.NewProxy() error = %v", err)
 	}
 
-	encGen, err := encurl.NewGenerator(testSigningKey)
+	keys, err := seal.DeriveKeys(testSigningKey)
 	if err != nil {
-		t.Fatalf("encurl.NewGenerator() error = %v", err)
+		t.Fatalf("seal.DeriveKeys() error = %v", err)
 	}
 
+	encGen := encurl.NewGenerator(keys.EncryptedURLKey)
+
 	h := &Handlers{
-		log:    log,
-		imgSvc: svc,
-		encGen: encGen,
+		log:       log,
+		imgSvc:    svc,
+		encGen:    encGen,
+		signer:    signature.New(testSigningKey),
+		allowlist: allowlist.New(nil),
 	}
 
 	r := chi.NewRouter()
@@ -106,9 +121,9 @@ func getMaxAge(t *testing.T, srv http.Handler, target string) int {
 func TestHandleImage_SignedURL_MaxAgeEndsAtExp(t *testing.T) {
 	t.Parallel()
 
-	h, srv := newSignedHostServer(t, slog.New(slog.DiscardHandler))
+	_, srv := newSignedHostServer(t, slog.New(slog.DiscardHandler))
 
-	signedURL, err := h.imgSvc.GenerateSignedURL("", &imagerequest.Request{
+	signedURL := generateSignedURL(testSigningKey, &imagerequest.Request{
 		Source: imagerequest.Source{
 			Host: signedHost,
 			Path: photoPath,
@@ -118,9 +133,6 @@ func TestHandleImage_SignedURL_MaxAgeEndsAtExp(t *testing.T) {
 			Format: imagerequest.FormatJPEG,
 		},
 	}, time.Minute)
-	if err != nil {
-		t.Fatalf("GenerateSignedURL() error = %v", err)
-	}
 
 	maxAge := getMaxAge(t, srv, signedURL)
 	if maxAge < 50 || maxAge > 60 {
@@ -197,8 +209,7 @@ func TestHandleImageEnc_MaxAge(t *testing.T) {
 					Size:   imagerequest.Size{Width: 50, Height: 50},
 					Format: imagerequest.FormatJPEG,
 				},
-				Expires: tt.expires,
-			})
+			}, tt.expires)
 			if err != nil {
 				t.Fatalf("Generate() error = %v", err)
 			}

internal/handlers/image_errors_internal_test.go

@@ -1,19 +1,28 @@
 package handlers
 
 import (
+	"bytes"
+	"context"
 	"encoding/json"
 	"fmt"
 	"image/color"
+	"io"
 	"log/slog"
 	"net/http"
 	"net/http/httptest"
+	"slices"
 	"strings"
 	"testing"
 	"testing/fstest"
 	"time"
 
 	"github.com/go-chi/chi/v5"
+	"github.com/go-chi/chi/v5/middleware"
+	"github.com/prometheus/client_golang/prometheus"
+	"sneak.berlin/go/pixa/internal/allowlist"
 	"sneak.berlin/go/pixa/internal/httpfetcher"
+	"sneak.berlin/go/pixa/internal/imageprocessor"
+	"sneak.berlin/go/pixa/internal/imageproxy"
 	"sneak.berlin/go/pixa/internal/imagerequest"
 	"sneak.berlin/go/pixa/internal/imgcache"
 	"sneak.berlin/go/pixa/internal/signature"
@@ -24,29 +33,38 @@ import (
 const allowlistedHost = "allowed.example.com"
 
 // newImageRoute returns the image route of a Handlers whose service fetches
-// with fetcher and checks signatures with testSigningKey.
+// with fetcher, and which checks signatures with testSigningKey.
 func newImageRoute(t *testing.T, fetcher httpfetcher.Fetcher) http.Handler {
 	t.Helper()
 
-	cache, err := imgcache.NewCache(setupTestDB(t), imgcache.CacheConfig{
+	cache, err := imgcache.NewCache(setupTestDB(t), imgcache.Config{
 		StateDir:    t.TempDir(),
 		NegativeTTL: 5 * time.Minute,
-	})
+		MaxBytes:    1 << 30,
+	}, prometheus.NewRegistry())
 	if err != nil {
 		t.Fatalf("failed to create cache: %v", err)
 	}
 
-	svc, err := imgcache.NewService(&imgcache.ServiceConfig{
-		Cache:      cache,
-		Fetcher:    fetcher,
-		SigningKey: testSigningKey,
-		Allowlist:  []string{allowlistedHost},
+	svc, err := imageproxy.NewProxy(&imageproxy.Config{
+		Cache:   cache,
+		Fetcher: fetcher,
+		Processor: imageprocessor.New(imageprocessor.Params{
+			MaxInputBytes:           50 << 20,
+			MaxConcurrentProcessing: 2,
+		}),
+		MaxResponseSize: 50 << 20,
 	})
 	if err != nil {
-		t.Fatalf("failed to create service: %v", err)
+		t.Fatalf("failed to create proxy: %v", err)
 	}
 
-	h := &Handlers{imgSvc: svc, log: slog.New(slog.DiscardHandler)}
+	h := &Handlers{
+		imgSvc:    svc,
+		log:       slog.New(slog.DiscardHandler),
+		signer:    signature.New(testSigningKey),
+		allowlist: allowlist.New([]string{allowlistedHost}),
+	}
 
 	r := chi.NewRouter()
 	r.Get("/v1/image/*", h.HandleImage())
@@ -55,7 +73,7 @@ func newImageRoute(t *testing.T, fetcher httpfetcher.Fetcher) http.Handler {
 }
 
 // newPhotoFetcher returns a mock fetcher that serves a JPEG at photoPath on
-// each of hosts, and answers any other URL with an upstream error.
+// each of hosts, and answers any other source with an upstream error.
 func newPhotoFetcher(t *testing.T, hosts ...string) *httpfetcher.MockFetcher {
 	t.Helper()
 
@@ -82,10 +100,10 @@ func photoURLWithSig(host, sig string, expires time.Time) string {
 	return fmt.Sprintf("%s?sig=%s&exp=%d", photoURL(host), sig, expires.Unix())
 }
 
-// photoSignature returns the signature of photoURL(host) at the default
-// quality and fit, made with key and expiring at expires.
-func photoSignature(key, host string, expires time.Time) string {
-	return signature.New(key).Sign(&imagerequest.Request{
+// photoRequest returns the image request of photoURL(host) at the default
+// quality and fit.
+func photoRequest(host string) *imagerequest.Request {
+	return &imagerequest.Request{
 		Source: imagerequest.Source{
 			Host: host,
 			Path: photoPath,
@@ -96,8 +114,13 @@ func photoSignature(key, host string, expires time.Time) string {
 			Quality: 85,
 			FitMode: imagerequest.FitCover,
 		},
-		Expires: expires,
-	})
+	}
+}
+
+// photoSignature returns the signature of photoURL(host) at the default
+// quality and fit, made with key and expiring at expires.
+func photoSignature(key, host string, expires time.Time) string {
+	return signature.New(key).Sign(photoRequest(host), expires)
 }
 
 // sendGet sends a GET for target to route and returns the response.
@@ -168,7 +191,12 @@ func TestHandleImage_ErrorAnswers(t *testing.T) {
 	photos := newPhotoFetcher(t,
 		signedHost, parentHost, siblingHost, subdomainHost, appendedHost)
 	// The real fetcher refuses localhost before any lookup or connection.
-	realFetcher := httpfetcher.New(httpfetcher.DefaultConfig())
+	realFetcher := httpfetcher.New(&httpfetcher.Config{
+		Timeout:               30 * time.Second,
+		MaxResponseSize:       50 << 20,
+		MaxConnectionsPerHost: 20,
+		MaxConnections:        64,
+	})
 
 	exp := time.Now().Add(time.Hour)
 	expired := time.Now().Add(-time.Hour)
@@ -273,10 +301,9 @@ func TestHandleImage_AllowlistOrSignature(t *testing.T) {
 // host can be, and checks that each is refused with 400 naming the host as
 // decoded: allowlistedHost with a character after it that a URL host cannot
 // hold, percent-encoded, an IPv6 address without brackets, and brackets
-// around something that is not an IPv6 address. A "?" or "#" after
-// allowlistedHost would otherwise end the host when the source URL is read,
-// so the request would pass the allowlist and be fetched and cached under a
-// host that is not the one fetched.
+// around something that is not an IPv6 address. Later steps put the host
+// unparsed into the allowlist check, the cache key and the upstream URL, so
+// such a host is refused first.
 func TestHandleImage_InvalidHost_Returns400(t *testing.T) {
 	t.Parallel()
 
@@ -347,10 +374,9 @@ func TestHandleImage_HostForms_Served(t *testing.T) {
 
 // TestHandleImage_HashInSourcePathOrQuery_Returns400 sends image URLs whose
 // source path or source query holds a "#", written as %23, and checks that
-// each is refused with 400, the one signed for that source path included. The
-// source URL would otherwise end at the "#" when it is fetched, so each text
-// after it would fetch the image at photoPath and cache it under keys of its
-// own.
+// each is refused with 400, the one signed for that source path included.
+// A "#" starts a URL's fragment, so neither the source path nor the source
+// query may hold one.
 func TestHandleImage_HashInSourcePathOrQuery_Returns400(t *testing.T) {
 	t.Parallel()
 
@@ -368,8 +394,7 @@ func TestHandleImage_HashInSourcePathOrQuery_Returns400(t *testing.T) {
 			Quality: 85,
 			FitMode: imagerequest.FitCover,
 		},
-		Expires: exp,
-	})
+	}, exp)
 
 	tests := []struct {
 		name   string
@@ -395,3 +420,168 @@ func TestHandleImage_HashInSourcePathOrQuery_Returns400(t *testing.T) {
 		})
 	}
 }
+
+// failingFetcher answers every fetch with its error.
+type failingFetcher struct {
+	err error
+}
+
+// Fetch returns f's error.
+func (f failingFetcher) Fetch(
+	context.Context, imagerequest.Source,
+) (*httpfetcher.FetchResult, error) {
+	return nil, f.err
+}
+
+// photoURLsByRoute returns, for each image route, the function that makes the
+// URL of the JPEG at photoPath on signedHost on that route.
+func photoURLsByRoute() map[string]func(*testing.T, *Handlers) string {
+	return map[string]func(*testing.T, *Handlers) string{
+		"/v1/image/": signedPhotoURL,
+		"/v1/e/":     encPhotoURL,
+	}
+}
+
+// TestImageRoutes_SameErrorAnswers checks that both image routes answer each
+// upstream fetch error below with the same status. They log it as "failed to
+// get image" at the same level: error for the 500 and warn for the others.
+// Each request gets routes of its own, so no request finds another's failed
+// fetch in the negative cache.
+func TestImageRoutes_SameErrorAnswers(t *testing.T) {
+	t.Parallel()
+
+	tests := []struct {
+		name       string
+		err        error
+		wantStatus int
+		wantLevel  slog.Level
+	}{
+		{"blocked upstream address", httpfetcher.ErrSSRFBlocked,
+			http.StatusForbidden, slog.LevelWarn},
+		{"upstream error status",
+			fmt.Errorf("%w: status 404", httpfetcher.ErrUpstreamError),
+			http.StatusBadGateway, slog.LevelWarn},
+		{"no response headers within the fetch timeout",
+			httpfetcher.ErrUpstreamTimeout, http.StatusGatewayTimeout,
+			slog.LevelWarn},
+		{"no free upstream connection", httpfetcher.ErrTooManyConnections,
+			http.StatusServiceUnavailable, slog.LevelWarn},
+		{"Content-Type not a source format",
+			fmt.Errorf("%w: image/svg+xml", httpfetcher.ErrInvalidContentType),
+			http.StatusUnsupportedMediaType, slog.LevelWarn},
+		{"any other failure", io.ErrUnexpectedEOF,
+			http.StatusInternalServerError, slog.LevelError},
+	}
+
+	for _, tt := range tests {
+		for route, imageURL := range photoURLsByRoute() {
+			t.Run(route+" "+tt.name, func(t *testing.T) {
+				t.Parallel()
+
+				var logs bytes.Buffer
+
+				h, routes := newRoutesFetchingThrough(t,
+					failingFetcher{err: tt.err},
+					slog.New(slog.NewJSONHandler(&logs, nil)))
+
+				rec := sendGet(t, routes, imageURL(t, h))
+				if rec.Code != tt.wantStatus {
+					t.Errorf("status = %d, want %d: %s",
+						rec.Code, tt.wantStatus, rec.Body)
+				}
+
+				levels := loggedLevels(t, &logs)
+				if !slices.Contains(levels[tt.wantLevel.String()],
+					"failed to get image") {
+					t.Errorf(`no "failed to get image" line logged at level %s`,
+						tt.wantLevel)
+				}
+
+				if tt.wantLevel != slog.LevelError &&
+					len(levels[slog.LevelError.String()]) != 0 {
+					t.Errorf("logged at error level: %q",
+						levels[slog.LevelError.String()])
+				}
+			})
+		}
+	}
+}
+
+// failingResponseWriter is a response writer whose Write always fails, as when
+// the client has gone away.
+type failingResponseWriter struct {
+	*httptest.ResponseRecorder
+}
+
+// Write writes nothing and returns io.ErrClosedPipe.
+func (failingResponseWriter) Write([]byte) (int, error) {
+	return 0, io.ErrClosedPipe
+}
+
+// TestImageRoutes_FailedWriteLoggedOnce checks that both image routes log a
+// failed write of the image once, at warn level, with the request's ID, the
+// source host and path, the bytes served and the error. Neither may also log
+// the image as served.
+func TestImageRoutes_FailedWriteLoggedOnce(t *testing.T) {
+	t.Parallel()
+
+	const requestID = "test-request-id"
+
+	for route, imageURL := range photoURLsByRoute() {
+		t.Run(route, func(t *testing.T) {
+			t.Parallel()
+
+			var logs bytes.Buffer
+
+			h, srv := newSignedHostServer(t,
+				slog.New(slog.NewJSONHandler(&logs, nil)))
+
+			ctx := context.WithValue(t.Context(),
+				middleware.RequestIDKey, requestID)
+			srv.ServeHTTP(failingResponseWriter{httptest.NewRecorder()},
+				httptest.NewRequestWithContext(
+					ctx, http.MethodGet, imageURL(t, h), nil))
+
+			t.Logf("logged:\n%s", logs.String())
+
+			var failedWrites []map[string]any
+
+			dec := json.NewDecoder(&logs)
+			for dec.More() {
+				var line map[string]any
+
+				err := dec.Decode(&line)
+				if err != nil {
+					t.Fatalf("decoding log line: %v", err)
+				}
+
+				switch line["msg"] {
+				case "image served":
+					t.Error(`"image served" logged after a failed write`)
+				case "failed to write response":
+					failedWrites = append(failedWrites, line)
+				}
+			}
+
+			if len(failedWrites) != 1 {
+				t.Fatalf(`"failed to write response" logged %d times, want 1`,
+					len(failedWrites))
+			}
+
+			want := map[string]any{
+				"level":        slog.LevelWarn.String(),
+				"request_id":   requestID,
+				"host":         signedHost,
+				"path":         photoPath,
+				"served_bytes": float64(0),
+				"error":        io.ErrClosedPipe.Error(),
+			}
+
+			for key, wantValue := range want {
+				if got := failedWrites[0][key]; got != wantValue {
+					t.Errorf("%s = %v, want %v", key, got, wantValue)
+				}
+			}
+		})
+	}
+}

internal/handlers/image_signature_internal_test.go

@@ -5,6 +5,7 @@ import (
 	"fmt"
 	"net/http"
 	"net/http/httptest"
+	"net/url"
 	"testing"
 	"time"
 
@@ -36,6 +37,47 @@ func getImage(t *testing.T, fix *testFixtures, target string) int {
 	return rec.Code
 }
 
+// generateSignedURL returns the /v1/image/ URL of req, signed with key and
+// expiring after ttl. It signs a copy of req, so req itself is not changed.
+// An unset quality or fit in the copy is set to 85 or cover, the values the
+// route uses when a URL has no q or fit. The URL carries q and fit next to sig
+// and exp, so the route verifies it for the quality and fit it was signed
+// with.
+func generateSignedURL(
+	key string, req *imagerequest.Request, ttl time.Duration,
+) string {
+	signed := *req
+
+	if signed.Transform.Quality == 0 {
+		signed.Transform.Quality = 85
+	}
+
+	if signed.Transform.FitMode == "" {
+		signed.Transform.FitMode = imagerequest.FitCover
+	}
+
+	expires := time.Now().Add(ttl)
+	sig := signature.New(key).Sign(&signed, expires)
+
+	// A source query follows the source path after a "?". The image route
+	// reads the path decoded once, so the whole path is written in Go's own
+	// percent-encoding of a URL path, which writes "?" as %3F and "%" as %25,
+	// and leaves "/", ";", ",", "=" and "&" as they are.
+	sourcePathAndQuery := signed.Source.Path
+	if signed.Source.Query != "" {
+		sourcePathAndQuery += "?" + signed.Source.Query
+	}
+
+	unescapedPath := fmt.Sprintf("/v1/image/%s%s/%dx%d.%s",
+		signed.Source.Host, sourcePathAndQuery,
+		signed.Transform.Size.Width, signed.Transform.Size.Height,
+		signed.Transform.Format)
+
+	return fmt.Sprintf("%s?sig=%s&exp=%d&q=%d&fit=%s",
+		(&url.URL{Path: unescapedPath}).EscapedPath(), sig, expires.Unix(),
+		signed.Transform.Quality, signed.Transform.FitMode)
+}
+
 // TestHandleImage_SignatureCoversQualityAndFit signs a URL for quality 85
 // and fit cover, the values the route uses when a URL has no q or fit, and
 // sends that signature with each q and fit below.
@@ -55,8 +97,7 @@ func TestHandleImage_SignatureCoversQualityAndFit(t *testing.T) {
 			Quality: 85,
 			FitMode: imagerequest.FitCover,
 		},
-		Expires: expires,
-	})
+	}, expires)
 	signedURL := fmt.Sprintf("/v1/image/%s/images/photo.jpg/50x50.jpeg?sig=%s&exp=%d",
 		signedHost, sig, expires.Unix())
 
@@ -83,8 +124,8 @@ func TestHandleImage_SignatureCoversQualityAndFit(t *testing.T) {
 	}
 }
 
-// TestHandleImage_GeneratedSignedURLVerifies sends URLs built by the
-// service's signed-URL generator to the route.
+// TestHandleImage_GeneratedSignedURLVerifies sends URLs built by
+// generateSignedURL to the route.
 func TestHandleImage_GeneratedSignedURLVerifies(t *testing.T) {
 	t.Parallel()
 
@@ -101,25 +142,21 @@ func TestHandleImage_GeneratedSignedURLVerifies(t *testing.T) {
 		t.Run(tt.name, func(t *testing.T) {
 			t.Parallel()
 
-			fix := setupTestHandler(t)
+			signedURL := generateSignedURL("test-signing-key-must-be-32-chars",
+				&imagerequest.Request{
+					Source: imagerequest.Source{
+						Host: signedHost,
+						Path: "/images/photo.jpg",
+					},
+					Transform: imagerequest.Transform{
+						Size:    imagerequest.Size{Width: 50, Height: 50},
+						Format:  imagerequest.FormatJPEG,
+						Quality: tt.quality,
+						FitMode: tt.fitMode,
+					},
+				}, time.Hour)
 
-			signedURL, err := fix.service.GenerateSignedURL("", &imagerequest.Request{
-				Source: imagerequest.Source{
-					Host: signedHost,
-					Path: "/images/photo.jpg",
-				},
-				Transform: imagerequest.Transform{
-					Size:    imagerequest.Size{Width: 50, Height: 50},
-					Format:  imagerequest.FormatJPEG,
-					Quality: tt.quality,
-					FitMode: tt.fitMode,
-				},
-			}, time.Hour)
-			if err != nil {
-				t.Fatalf("GenerateSignedURL() error = %v", err)
-			}
-
-			status := getImage(t, fix, signedURL)
+			status := getImage(t, setupTestHandler(t), signedURL)
 			if status != http.StatusBadGateway {
 				t.Errorf("status = %d, want %d", status, http.StatusBadGateway)
 			}
@@ -178,10 +215,10 @@ func TestHandleImage_InvalidExp_Returns400(t *testing.T) {
 }
 
 // TestHandleImage_GeneratedSignedURLWithSourceQueryVerifies sends URLs built
-// by the service's signed-URL generator for source URLs with a query to the
-// route, through a chi router: chi routes on the path as the client wrote it
-// whenever that is not Go's own encoding of it. A 502 means the signature
-// passed, so the route read the source path and query that were signed.
+// by generateSignedURL for source URLs with a query to the route, through a
+// chi router: chi routes on the path as the client wrote it whenever that is
+// not Go's own encoding of it. A 502 means the signature passed, so the route
+// read the source path and query that were signed.
 func TestHandleImage_GeneratedSignedURLWithSourceQueryVerifies(t *testing.T) {
 	t.Parallel()
 
@@ -201,24 +238,20 @@ func TestHandleImage_GeneratedSignedURLWithSourceQueryVerifies(t *testing.T) {
 		t.Run(tt.name, func(t *testing.T) {
 			t.Parallel()
 
-			fix := setupTestHandler(t)
+			signedURL := generateSignedURL("test-signing-key-must-be-32-chars",
+				&imagerequest.Request{
+					Source: imagerequest.Source{
+						Host:  signedHost,
+						Path:  photoPath,
+						Query: tt.query,
+					},
+					Transform: imagerequest.Transform{
+						Size:   imagerequest.Size{Width: 50, Height: 50},
+						Format: imagerequest.FormatJPEG,
+					},
+				}, time.Hour)
 
-			signedURL, err := fix.service.GenerateSignedURL("", &imagerequest.Request{
-				Source: imagerequest.Source{
-					Host:  signedHost,
-					Path:  photoPath,
-					Query: tt.query,
-				},
-				Transform: imagerequest.Transform{
-					Size:   imagerequest.Size{Width: 50, Height: 50},
-					Format: imagerequest.FormatJPEG,
-				},
-			}, time.Hour)
-			if err != nil {
-				t.Fatalf("GenerateSignedURL() error = %v", err)
-			}
-
-			status := getImage(t, fix, signedURL)
+			status := getImage(t, setupTestHandler(t), signedURL)
 			if status != http.StatusBadGateway {
 				t.Errorf("status = %d, want %d", status, http.StatusBadGateway)
 			}
@@ -247,8 +280,7 @@ func TestHandleImage_SourceQueryEncodedAnyWay(t *testing.T) {
 			Quality: 85,
 			FitMode: imagerequest.FitCover,
 		},
-		Expires: expires,
-	})
+	}, expires)
 
 	tests := []struct {
 		name         string
@@ -280,7 +312,7 @@ func TestHandleImage_SourceQueryEncodedAnyWay(t *testing.T) {
 // TestHandleImage_SourceURLHoldingPercent25Verifies signs a source path and a
 // source query that each hold %25, the percent-escape of a literal "%", and
 // sends each to the route written as the README says, with that "%" written as
-// %25, and as GenerateSignedURL writes it. A 502 means the signature passed.
+// %25, and as generateSignedURL writes it. A 502 means the signature passed.
 func TestHandleImage_SourceURLHoldingPercent25Verifies(t *testing.T) {
 	t.Parallel()
 
@@ -314,8 +346,7 @@ func TestHandleImage_SourceURLHoldingPercent25Verifies(t *testing.T) {
 					Quality: 85,
 					FitMode: imagerequest.FitCover,
 				},
-				Expires: expires,
-			})
+			}, expires)
 
 			target := fmt.Sprintf("/v1/image/%s%s/50x50.jpeg?sig=%s&exp=%d",
 				signedHost, tt.writtenPathAndQuery, sig, expires.Unix())
@@ -326,27 +357,23 @@ func TestHandleImage_SourceURLHoldingPercent25Verifies(t *testing.T) {
 			}
 		})
 
-		t.Run(tt.name+" written by GenerateSignedURL", func(t *testing.T) {
+		t.Run(tt.name+" written by generateSignedURL", func(t *testing.T) {
 			t.Parallel()
 
-			fix := setupTestHandler(t)
+			signedURL := generateSignedURL("test-signing-key-must-be-32-chars",
+				&imagerequest.Request{
+					Source: imagerequest.Source{
+						Host:  signedHost,
+						Path:  tt.sourcePath,
+						Query: tt.sourceQuery,
+					},
+					Transform: imagerequest.Transform{
+						Size:   imagerequest.Size{Width: 50, Height: 50},
+						Format: imagerequest.FormatJPEG,
+					},
+				}, time.Hour)
 
-			signedURL, err := fix.service.GenerateSignedURL("", &imagerequest.Request{
-				Source: imagerequest.Source{
-					Host:  signedHost,
-					Path:  tt.sourcePath,
-					Query: tt.sourceQuery,
-				},
-				Transform: imagerequest.Transform{
-					Size:   imagerequest.Size{Width: 50, Height: 50},
-					Format: imagerequest.FormatJPEG,
-				},
-			}, time.Hour)
-			if err != nil {
-				t.Fatalf("GenerateSignedURL() error = %v", err)
-			}
-
-			status := getImage(t, fix, signedURL)
+			status := getImage(t, setupTestHandler(t), signedURL)
 			if status != http.StatusBadGateway {
 				t.Errorf("status = %d, want %d", status, http.StatusBadGateway)
 			}

internal/handlers/imageenc.go

@@ -2,21 +2,13 @@ package handlers
 
 import (
 	"errors"
-	"fmt"
-	"io"
 	"net/http"
-	"strconv"
 	"time"
 
 	"github.com/go-chi/chi/v5"
-	"github.com/go-chi/chi/v5/middleware"
 
 	"sneak.berlin/go/pixa/internal/encurl"
-	"sneak.berlin/go/pixa/internal/httpfetcher"
-	"sneak.berlin/go/pixa/internal/imageprocessor"
 	"sneak.berlin/go/pixa/internal/imagerequest"
-	"sneak.berlin/go/pixa/internal/imgcache"
-	"sneak.berlin/go/pixa/internal/magic"
 )
 
 // HandleImageEnc handles requests to /v1/e/{token}/* for encrypted
@@ -28,157 +20,59 @@ func (s *Handlers) HandleImageEnc() http.HandlerFunc {
 			return
 		}
 
-		ctx := r.Context()
-		start := time.Now()
-
-		req, ok := s.parseImageEncRequest(w, r)
-		if !ok || !s.chooseAutoFormat(w, r, req) {
+		req, expires, ok := s.parseImageEncRequest(w, r)
+		if !ok {
 			return
 		}
 
-		// Log the request
-		s.log.Debug("encrypted image request",
-			"host", req.Source.Host,
-			"path", req.Source.Path,
-			"dimensions", fmt.Sprintf("%dx%d",
-				req.Transform.Size.Width, req.Transform.Size.Height),
-			"format", req.Transform.Format,
-		)
-
-		// Fetch and process the image (no signature validation
-		// needed - encrypted URL is trusted)
-		resp, err := s.imgSvc.Get(ctx, req)
-		if err != nil {
-			s.handleImageError(w, err)
-
-			return
-		}
-
-		defer func() { _ = resp.Content.Close() }()
-
-		// Set response headers
-		w.Header().Set("Content-Type", resp.ContentType)
-
-		if resp.ContentLength > 0 {
-			w.Header().Set("Content-Length", strconv.FormatInt(resp.ContentLength, 10))
-		}
-
-		// Cache headers: max-age ends at the URL's expiry
-		w.Header().Set("Cache-Control", cacheControl(req.Expires))
-		w.Header().Set("X-Pixa-Cache", string(resp.CacheStatus))
-
-		if notModified(w, r, resp.ETag) {
-			return
-		}
-
-		// A HEAD request gets the headers only
-		if r.Method == http.MethodHead {
-			w.WriteHeader(http.StatusOK)
-
-			return
-		}
-
-		// Stream the response
-		written, err := io.Copy(w, resp.Content)
-		if err != nil {
-			s.log.Error("failed to write response", "error", err)
-
-			return
-		}
-
-		// Log completion
-		duration := time.Since(start)
-		s.log.Info("image served",
-			"request_id", middleware.GetReqID(ctx),
-			"cache_key", imagerequest.CacheKey(req),
-			"host", req.Source.Host,
-			"path", req.Source.Path,
-			"format", req.Transform.Format,
-			"cache_status", resp.CacheStatus,
-			"served_bytes", written,
-			"duration_ms", duration.Milliseconds(),
-		)
+		s.serveImage(w, r, req, expires)
 	}
 }
 
 // parseImageEncRequest decrypts the token of an encrypted image URL into an
-// image request and checks it. On a token that is missing, does not decrypt,
-// has expired or asks for something not valid, it writes an error response
-// and returns false.
+// image request and checks it, and returns it with the token's expiry. On a
+// token that is missing, does not decrypt, has expired or asks for something
+// not valid, it writes an error response and returns false.
 func (s *Handlers) parseImageEncRequest(
 	w http.ResponseWriter, r *http.Request,
-) (*imagerequest.Request, bool) {
+) (*imagerequest.Request, time.Time, bool) {
 	// Extract token from URL
 	token := chi.URLParam(r, "token")
 	if token == "" {
 		s.respondError(w, "missing token", http.StatusBadRequest)
 
-		return nil, false
+		return nil, time.Time{}, false
 	}
 
 	// Decrypt and validate the payload
-	req, err := s.encGen.Parse(token)
+	req, expires, err := s.encGen.Parse(token)
 	if err != nil {
 		if errors.Is(err, encurl.ErrExpired) {
 			s.log.Debug("encrypted URL expired", "error", err)
 			s.respondError(w, "URL has expired", http.StatusGone)
 
-			return nil, false
+			return nil, time.Time{}, false
 		}
 
-		s.log.Debug("failed to decrypt URL", "error", err)
+		s.log.Debug("invalid encrypted URL", "error", err)
 		s.respondError(w, "invalid encrypted URL", http.StatusBadRequest)
 
-		return nil, false
+		return nil, time.Time{}, false
 	}
 
 	// Apply the same dimension and fit-mode bounds as the plain image
 	// route: a sealed payload is trusted for its origin, not for staying
-	// within limits, so an over-limit size or unknown fit mode is a 400
-	// here rather than an out-of-memory or a 500 from the processor.
+	// within limits. This refuses an over-limit size before the source is
+	// fetched. An unknown fit mode is a 400 here rather than a 500 from the
+	// processor.
 	err = imagerequest.ValidateImageRequest(req)
 	if err != nil {
 		s.log.Debug("encrypted URL failed validation", "error", err)
 		s.respondError(w, "invalid encrypted URL: "+err.Error(),
 			http.StatusBadRequest)
 
-		return nil, false
+		return nil, time.Time{}, false
 	}
 
-	return req, true
-}
-
-// handleImageError converts image service errors to HTTP responses.
-func (s *Handlers) handleImageError(w http.ResponseWriter, err error) {
-	switch {
-	case errors.Is(err, httpfetcher.ErrSSRFBlocked):
-		s.respondError(w, "forbidden", http.StatusForbidden)
-	case errors.Is(err, httpfetcher.ErrUpstreamError):
-		s.respondError(w, "upstream error", http.StatusBadGateway)
-	case errors.Is(err, httpfetcher.ErrUpstreamTimeout):
-		s.respondError(w, "upstream timeout", http.StatusGatewayTimeout)
-	case errors.Is(err, httpfetcher.ErrTooManyConnections),
-		errors.Is(err, imageprocessor.ErrTooManyImages),
-		errors.Is(err, imgcache.ErrProcessingTimedOut):
-		s.respondError(w, "server busy, try again later",
-			http.StatusServiceUnavailable)
-	case errors.Is(err, httpfetcher.ErrInvalidContentType),
-		errors.Is(err, imageprocessor.ErrUnsupportedInputFormat),
-		errors.Is(err, magic.ErrMagicByteMismatch),
-		errors.Is(err, magic.ErrUnknownFormat),
-		errors.Is(err, magic.ErrNotEnoughData):
-		s.respondError(w, "unsupported source format",
-			http.StatusUnsupportedMediaType)
-	case errors.Is(err, imageprocessor.ErrOutputTooLarge):
-		s.respondError(w, fmt.Sprintf("size too large for the source: the side "+
-			"computed from its proportions is above %d",
-			imageprocessor.MaxOutputDimension), http.StatusBadRequest)
-	case errors.Is(err, imageprocessor.ErrAVIFOutputTooLarge):
-		s.respondError(w, fmt.Sprintf("size too large for AVIF: the output "+
-			"would be above %d pixels", imageprocessor.MaxAVIFOutputPixels),
-			http.StatusBadRequest)
-	default:
-		s.log.Error("image request failed", "error", err)
-		s.respondError(w, "internal error", http.StatusInternalServerError)
-	}
+	return req, expires, true
 }

internal/handlers/imageenc_internal_test.go

@@ -6,11 +6,13 @@ import (
 	"net/http"
 	"net/http/httptest"
 	"testing"
+	"time"
 
 	"github.com/go-chi/chi/v5"
 
 	"sneak.berlin/go/pixa/internal/encurl"
 	"sneak.berlin/go/pixa/internal/imagerequest"
+	"sneak.berlin/go/pixa/internal/seal"
 )
 
 // newEncTestServer builds a router serving the encrypted-URL route with a
@@ -20,11 +22,13 @@ import (
 func newEncTestServer(t *testing.T) (*encurl.Generator, http.Handler) {
 	t.Helper()
 
-	encGen, err := encurl.NewGenerator(testSigningKey)
+	keys, err := seal.DeriveKeys(testSigningKey)
 	if err != nil {
-		t.Fatalf("encurl.NewGenerator() error = %v", err)
+		t.Fatalf("seal.DeriveKeys() error = %v", err)
 	}
 
+	encGen := encurl.NewGenerator(keys.EncryptedURLKey)
+
 	h := &Handlers{
 		log:    slog.New(slog.DiscardHandler),
 		encGen: encGen,
@@ -62,7 +66,7 @@ func TestHandleImageEnc_OverLimitDimension_Returns400(t *testing.T) {
 		Transform: imagerequest.Transform{
 			Size: imagerequest.Size{Width: 100000, Height: 100000},
 		},
-	})
+	}, time.Time{})
 	if err != nil {
 		t.Fatalf("Generate() error = %v", err)
 	}
@@ -91,7 +95,7 @@ func TestHandleImageEnc_InvalidFitMode_Returns400(t *testing.T) {
 			Size:    imagerequest.Size{Width: 800, Height: 600},
 			FitMode: imagerequest.Fit("bogus"),
 		},
-	})
+	}, time.Time{})
 	if err != nil {
 		t.Fatalf("Generate() error = %v", err)
 	}

internal/handlers/imageenc_token_internal_test.go

@@ -10,6 +10,7 @@ import (
 
 	"sneak.berlin/go/pixa/internal/encurl"
 	"sneak.berlin/go/pixa/internal/imagerequest"
+	"sneak.berlin/go/pixa/internal/seal"
 )
 
 // requireServedPhoto requires that rec answers 200 with the JPEG at photoPath
@@ -70,25 +71,24 @@ func TestHandleImageEnc_RejectedToken(t *testing.T) {
 		},
 	}
 
-	valid, err := h.encGen.Generate(&photo)
+	valid, err := h.encGen.Generate(&photo, time.Time{})
 	if err != nil {
 		t.Fatalf("Generate() error = %v", err)
 	}
 
-	expiredPhoto := photo
-	expiredPhoto.Expires = time.Now().Add(-time.Minute)
-
-	expired, err := h.encGen.Generate(&expiredPhoto)
+	expired, err := h.encGen.Generate(&photo, time.Now().Add(-time.Minute))
 	if err != nil {
 		t.Fatalf("Generate() error = %v", err)
 	}
 
-	otherGen, err := encurl.NewGenerator("another-signing-key-fedcba9876543210")
+	keys, err := seal.DeriveKeys("another-signing-key-fedcba9876543210")
 	if err != nil {
-		t.Fatalf("encurl.NewGenerator() error = %v", err)
+		t.Fatalf("seal.DeriveKeys() error = %v", err)
 	}
 
-	otherKey, err := otherGen.Generate(&photo)
+	otherGen := encurl.NewGenerator(keys.EncryptedURLKey)
+
+	otherKey, err := otherGen.Generate(&photo, time.Time{})
 	if err != nil {
 		t.Fatalf("Generate() error = %v", err)
 	}

internal/handlers/jpegxl_default_internal_test.go

@@ -69,8 +69,7 @@ func TestImageWithoutFormat_SignedAsJXL(t *testing.T) {
 			Quality: encurl.DefaultQuality,
 			FitMode: imagerequest.FitCover,
 		},
-		Expires: expires,
-	})
+	}, expires)
 	query := fmt.Sprintf("?sig=%s&exp=%d", sig, expires.Unix())
 
 	for _, size := range []string{"50x50.jxl", "50x50"} {
@@ -94,7 +93,7 @@ func TestImageEncWithoutFormat_ServesJPEGXL(t *testing.T) {
 		Transform: imagerequest.Transform{
 			Size: imagerequest.Size{Width: 50, Height: 50},
 		},
-	})
+	}, time.Time{})
 	if err != nil {
 		t.Fatalf("Generate() error = %v", err)
 	}

internal/handlers/referer_blocklist_internal_test.go

@@ -10,10 +10,16 @@ import (
 	"time"
 
 	"github.com/go-chi/chi/v5"
+	"github.com/prometheus/client_golang/prometheus"
 	"sneak.berlin/go/pixa/internal/allowlist"
 	"sneak.berlin/go/pixa/internal/encurl"
 	"sneak.berlin/go/pixa/internal/httpfetcher"
+	"sneak.berlin/go/pixa/internal/imageprocessor"
+	"sneak.berlin/go/pixa/internal/imageproxy"
+	"sneak.berlin/go/pixa/internal/imagerequest"
 	"sneak.berlin/go/pixa/internal/imgcache"
+	"sneak.berlin/go/pixa/internal/seal"
+	"sneak.berlin/go/pixa/internal/signature"
 )
 
 // blockedReferer is a page on leech.example, which newRefererRoutes puts on
@@ -29,11 +35,11 @@ type countingFetcher struct {
 
 // Fetch counts the fetch and passes it on.
 func (f *countingFetcher) Fetch(
-	ctx context.Context, url string,
+	ctx context.Context, source imagerequest.Source,
 ) (*httpfetcher.FetchResult, error) {
 	f.fetches.Add(1)
 
-	return f.Fetcher.Fetch(ctx, url)
+	return f.Fetcher.Fetch(ctx, source)
 }
 
 // newRefererRoutes returns both image routes of a Handlers whose
@@ -47,33 +53,41 @@ func newRefererRoutes(t *testing.T) (http.Handler, *Handlers, *countingFetcher)
 		Fetcher: newPhotoFetcher(t, allowlistedHost, signedHost),
 	}
 
-	cache, err := imgcache.NewCache(setupTestDB(t), imgcache.CacheConfig{
+	cache, err := imgcache.NewCache(setupTestDB(t), imgcache.Config{
 		StateDir:    t.TempDir(),
 		NegativeTTL: 5 * time.Minute,
-	})
+		MaxBytes:    1 << 30,
+	}, prometheus.NewRegistry())
 	if err != nil {
 		t.Fatalf("imgcache.NewCache() error = %v", err)
 	}
 
-	svc, err := imgcache.NewService(&imgcache.ServiceConfig{
-		Cache:      cache,
-		Fetcher:    fetcher,
-		SigningKey: testSigningKey,
-		Allowlist:  []string{allowlistedHost},
+	svc, err := imageproxy.NewProxy(&imageproxy.Config{
+		Cache:   cache,
+		Fetcher: fetcher,
+		Processor: imageprocessor.New(imageprocessor.Params{
+			MaxInputBytes:           50 << 20,
+			MaxConcurrentProcessing: 2,
+		}),
+		MaxResponseSize: 50 << 20,
 	})
 	if err != nil {
-		t.Fatalf("imgcache.NewService() error = %v", err)
+		t.Fatalf("imageproxy.NewProxy() error = %v", err)
 	}
 
-	encGen, err := encurl.NewGenerator(testSigningKey)
+	keys, err := seal.DeriveKeys(testSigningKey)
 	if err != nil {
-		t.Fatalf("encurl.NewGenerator() error = %v", err)
+		t.Fatalf("seal.DeriveKeys() error = %v", err)
 	}
 
+	encGen := encurl.NewGenerator(keys.EncryptedURLKey)
+
 	h := &Handlers{
-		log:    slog.New(slog.DiscardHandler),
-		imgSvc: svc,
-		encGen: encGen,
+		log:       slog.New(slog.DiscardHandler),
+		imgSvc:    svc,
+		encGen:    encGen,
+		signer:    signature.New(testSigningKey),
+		allowlist: allowlist.New([]string{allowlistedHost}),
 		refererBlocklist: allowlist.New(
 			[]string{"leech.example", ".hotlinker.example"}),
 	}

internal/handlers/request_id_internal_test.go

@@ -17,11 +17,13 @@ import (
 )
 
 // signedPhotoURL returns a signed /v1/image/ URL, valid for a minute, for the
-// JPEG at photoPath on signedHost at 50x50, made with h's image service.
-func signedPhotoURL(t *testing.T, h *Handlers) string {
+// JPEG at photoPath on signedHost at 50x50. It is signed with testSigningKey,
+// the key the handlers that newSignedHostServer builds check signatures with.
+// It takes a Handlers it does not use, so that it has the type of encPhotoURL.
+func signedPhotoURL(t *testing.T, _ *Handlers) string {
 	t.Helper()
 
-	signedURL, err := h.imgSvc.GenerateSignedURL("", &imagerequest.Request{
+	return generateSignedURL(testSigningKey, &imagerequest.Request{
 		Source: imagerequest.Source{
 			Host: signedHost,
 			Path: photoPath,
@@ -31,11 +33,6 @@ func signedPhotoURL(t *testing.T, h *Handlers) string {
 			Format: imagerequest.FormatJPEG,
 		},
 	}, time.Minute)
-	if err != nil {
-		t.Fatalf("GenerateSignedURL() error = %v", err)
-	}
-
-	return signedURL
 }
 
 // encPhotoURL returns an encrypted /v1/e/ URL, which never expires, for the
@@ -52,7 +49,7 @@ func encPhotoURL(t *testing.T, h *Handlers) string {
 			Size:   imagerequest.Size{Width: 50, Height: 50},
 			Format: imagerequest.FormatJPEG,
 		},
-	})
+	}, time.Time{})
 	if err != nil {
 		t.Fatalf("Generate() error = %v", err)
 	}

internal/handlers/server_busy_internal_test.go

@@ -11,13 +11,14 @@ import (
 
 	"sneak.berlin/go/pixa/internal/httpfetcher"
 	"sneak.berlin/go/pixa/internal/imageprocessor"
+	"sneak.berlin/go/pixa/internal/imageproxy"
 	"sneak.berlin/go/pixa/internal/imagerequest"
-	"sneak.berlin/go/pixa/internal/imgcache"
 )
 
-// TestServerBusyAnswers503 checks that both image routes answer 503 with
-// a clear error when the image service gives up waiting for a free upstream
-// connection or processing slot, wrapped as the service wraps them.
+// TestServerBusyAnswers503 checks that respondImageError, which answers the
+// errors of both image routes, answers 503 with a clear error when the image
+// service gives up waiting for a free upstream connection or processing slot,
+// wrapped as the service wraps them.
 func TestServerBusyAnswers503(t *testing.T) {
 	t.Parallel()
 
@@ -30,61 +31,44 @@ func TestServerBusyAnswers503(t *testing.T) {
 		fmt.Errorf("upstream fetch failed: %w", httpfetcher.ErrTooManyConnections),
 		fmt.Errorf("image processing failed: %w", imageprocessor.ErrTooManyImages),
 	} {
-		plain := httptest.NewRecorder()
-		h.respondImageError(plain, req, err)
+		rec := httptest.NewRecorder()
+		h.respondImageError(rec, req, err)
 
-		encrypted := httptest.NewRecorder()
-		h.handleImageError(encrypted, err)
+		if rec.Code != http.StatusServiceUnavailable {
+			t.Errorf("for %v: status = %d, want %d",
+				err, rec.Code, http.StatusServiceUnavailable)
+		}
 
-		for route, rec := range map[string]*httptest.ResponseRecorder{
-			"/v1/image/": plain, "/v1/e/": encrypted,
-		} {
-			if rec.Code != http.StatusServiceUnavailable {
-				t.Errorf("%s for %v: status = %d, want %d",
-					route, err, rec.Code, http.StatusServiceUnavailable)
-			}
-
-			if !strings.Contains(rec.Body.String(), "server busy, try again later") {
-				t.Errorf("%s for %v: body = %q, want the server busy error",
-					route, err, rec.Body.String())
-			}
+		if !strings.Contains(rec.Body.String(), "server busy, try again later") {
+			t.Errorf("for %v: body = %q, want the server busy error",
+				err, rec.Body.String())
 		}
 	}
 }
 
-// TestProcessingTimedOutAnswers503 checks that respondImageError and
-// handleImageError, which answer the errors of the two image routes, answer
-// 503 with the server busy error when the image service reports that
-// processing stopped at the deadline of the request doing it, wrapped as the
-// service wraps it. A request that waited for that processing gets this error
-// while its own deadline may have time left. The request's source only goes
-// into a log line, so it is left empty.
+// TestProcessingTimedOutAnswers503 checks that respondImageError, which
+// answers the errors of both image routes, answers 503 with the server busy
+// error when the image service reports that processing stopped at the
+// deadline of the request doing it, wrapped as the service wraps it. A request
+// that waited for that processing gets this error while its own deadline may
+// have time left. The request's source only goes into a log line, so it is
+// left empty.
 func TestProcessingTimedOutAnswers503(t *testing.T) {
 	t.Parallel()
 
 	h := &Handlers{log: slog.New(slog.DiscardHandler)}
 	err := fmt.Errorf("%w: failed to encode: %w",
-		imgcache.ErrProcessingTimedOut, context.DeadlineExceeded)
+		imageproxy.ErrProcessingTimedOut, context.DeadlineExceeded)
 
-	plain := httptest.NewRecorder()
-	h.respondImageError(plain, &imagerequest.Request{}, err)
+	rec := httptest.NewRecorder()
+	h.respondImageError(rec, &imagerequest.Request{}, err)
+	t.Logf("status = %d, body = %q", rec.Code, rec.Body.String())
 
-	encrypted := httptest.NewRecorder()
-	h.handleImageError(encrypted, err)
+	if rec.Code != http.StatusServiceUnavailable {
+		t.Errorf("status = %d, want %d", rec.Code, http.StatusServiceUnavailable)
+	}
 
-	for name, rec := range map[string]*httptest.ResponseRecorder{
-		"respondImageError": plain, "handleImageError": encrypted,
-	} {
-		t.Logf("%s: status = %d, body = %q", name, rec.Code, rec.Body.String())
-
-		if rec.Code != http.StatusServiceUnavailable {
-			t.Errorf("%s: status = %d, want %d",
-				name, rec.Code, http.StatusServiceUnavailable)
-		}
-
-		if !strings.Contains(rec.Body.String(), "server busy, try again later") {
-			t.Errorf("%s: body = %q, want the server busy error",
-				name, rec.Body.String())
-		}
+	if !strings.Contains(rec.Body.String(), "server busy, try again later") {
+		t.Errorf("body = %q, want the server busy error", rec.Body.String())
 	}
 }

internal/handlers/unsupported_source_internal_test.go

@@ -13,10 +13,11 @@ import (
 	"sneak.berlin/go/pixa/internal/imagerequest"
 )
 
-// TestUnsupportedSourceFormatAnswers415 checks that both image routes answer
-// 415 with a clear error when the upstream answers with a content type pixa
-// does not accept, such as SVG, and when orig is asked of a source in a
-// format the processor cannot name, wrapped as the service wraps them.
+// TestUnsupportedSourceFormatAnswers415 checks that respondImageError, which
+// answers the errors of both image routes, answers 415 with a clear error when
+// the upstream answers with a content type pixa does not accept, such as SVG,
+// and when orig is asked of a source in a format the processor cannot name,
+// wrapped as the service wraps them.
 func TestUnsupportedSourceFormatAnswers415(t *testing.T) {
 	t.Parallel()
 
@@ -31,26 +32,18 @@ func TestUnsupportedSourceFormatAnswers415(t *testing.T) {
 		fmt.Errorf("image processing failed: %w: unknown",
 			imageprocessor.ErrUnsupportedInputFormat),
 	} {
-		plain := httptest.NewRecorder()
-		h.respondImageError(plain, req, err)
+		rec := httptest.NewRecorder()
+		h.respondImageError(rec, req, err)
+		t.Logf("for %v: %d %s", err, rec.Code, rec.Body.String())
 
-		encrypted := httptest.NewRecorder()
-		h.handleImageError(encrypted, err)
+		if rec.Code != http.StatusUnsupportedMediaType {
+			t.Errorf("for %v: status = %d, want %d",
+				err, rec.Code, http.StatusUnsupportedMediaType)
+		}
 
-		for route, rec := range map[string]*httptest.ResponseRecorder{
-			"HandleImage": plain, "HandleImageEnc": encrypted,
-		} {
-			t.Logf("%s for %v: %d %s", route, err, rec.Code, rec.Body.String())
-
-			if rec.Code != http.StatusUnsupportedMediaType {
-				t.Errorf("%s for %v: status = %d, want %d",
-					route, err, rec.Code, http.StatusUnsupportedMediaType)
-			}
-
-			if !strings.Contains(rec.Body.String(), "unsupported source format") {
-				t.Errorf("%s for %v: body = %q, want the unsupported source format error",
-					route, err, rec.Body.String())
-			}
+		if !strings.Contains(rec.Body.String(), "unsupported source format") {
+			t.Errorf("for %v: body = %q, want the unsupported source format error",
+				err, rec.Body.String())
 		}
 	}
 }

internal/handlers/validate_signature_internal_test.go

@@ -0,0 +1,211 @@
+package handlers
+
+import (
+	"testing"
+	"time"
+
+	"sneak.berlin/go/pixa/internal/allowlist"
+	"sneak.berlin/go/pixa/internal/signature"
+)
+
+// Addresses the port tests below put on the allowlist.
+const (
+	allowlistedIPv4 = "203.0.113.5"
+	allowlistedIPv6 = "2001:db8::1"
+)
+
+// newSignatureChecker returns Handlers that check signatures with
+// testSigningKey and have allowlistHosts on their allowlist, as New builds
+// them from signing_key and allowlist_hosts.
+func newSignatureChecker(allowlistHosts ...string) *Handlers {
+	return &Handlers{
+		signer:    signature.New(testSigningKey),
+		allowlist: allowlist.New(allowlistHosts),
+	}
+}
+
+func TestValidateSignature_NonAllowlistedHost_NoSignature(t *testing.T) {
+	t.Parallel()
+
+	h := newSignatureChecker(allowlistedHost)
+
+	// Should fail validation - not allowlisted and no signature
+	err := h.validateSignature(photoRequest(signedHost), "", time.Time{})
+	if err == nil {
+		t.Error("validateSignature() expected error for non-allowlisted " +
+			"host without signature")
+	}
+}
+
+func TestValidateSignature_NonAllowlistedHost_ValidSignature(t *testing.T) {
+	t.Parallel()
+
+	h := newSignatureChecker(allowlistedHost)
+	req := photoRequest(signedHost)
+
+	// Generate a valid signature
+	expires := time.Now().Add(time.Hour)
+	sig := signature.New(testSigningKey).Sign(req, expires)
+
+	// Should pass validation
+	err := h.validateSignature(req, sig, expires)
+	if err != nil {
+		t.Errorf("validateSignature() error = %v", err)
+	}
+}
+
+func TestValidateSignature_NonAllowlistedHost_ExpiredSignature(t *testing.T) {
+	t.Parallel()
+
+	h := newSignatureChecker(allowlistedHost)
+	req := photoRequest(signedHost)
+
+	// Generate an expired signature
+	expires := time.Now().Add(-time.Hour)
+	sig := signature.New(testSigningKey).Sign(req, expires)
+
+	// Should fail validation
+	err := h.validateSignature(req, sig, expires)
+	if err == nil {
+		t.Error("validateSignature() expected error for expired signature")
+	}
+}
+
+func TestValidateSignature_NonAllowlistedHost_InvalidSignature(t *testing.T) {
+	t.Parallel()
+
+	h := newSignatureChecker(allowlistedHost)
+
+	// Should fail validation
+	err := h.validateSignature(photoRequest(signedHost), "invalid-signature",
+		time.Now().Add(time.Hour))
+	if err == nil {
+		t.Error("validateSignature() expected error for invalid signature")
+	}
+}
+
+// TestValidateSignature_NoAllowlist checks that with no host on the
+// allowlist, a request without a signature fails validation.
+func TestValidateSignature_NoAllowlist(t *testing.T) {
+	t.Parallel()
+
+	h := newSignatureChecker()
+
+	err := h.validateSignature(photoRequest(signedHost), "", time.Time{})
+	if err == nil {
+		t.Error("validateSignature() expected error when no host is allowlisted")
+	}
+}
+
+// TestValidateSignature_SignatureExactHostMatch verifies that
+// validateSignature enforces exact host matching for signatures. A
+// signature for one host must not verify for a different host, even
+// if they share a domain suffix.
+func TestValidateSignature_SignatureExactHostMatch(t *testing.T) {
+	t.Parallel()
+
+	h := newSignatureChecker()
+
+	// Sign a request for signedHost
+	signedReq := photoRequest(signedHost)
+	expires := time.Now().Add(time.Hour)
+	sig := signature.New(testSigningKey).Sign(signedReq, expires)
+
+	// The original request should pass validation
+	t.Run("exact host passes", func(t *testing.T) {
+		t.Parallel()
+
+		err := h.validateSignature(signedReq, sig, expires)
+		if err != nil {
+			t.Errorf("validateSignature() exact host failed: %v", err)
+		}
+	})
+
+	// Try to reuse the signature with different hosts
+	tests := []struct {
+		name string
+		host string
+	}{
+		{"parent domain", "example.com"},
+		{"sibling subdomain", "images.example.com"},
+		{"deeper subdomain", "a." + signedHost},
+		{"evil suffix domain", signedHost + ".evil.com"},
+		{"prefixed host", "evil" + signedHost},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name+" rejected", func(t *testing.T) {
+			t.Parallel()
+
+			err := h.validateSignature(photoRequest(tt.host), sig, expires)
+			if err == nil {
+				t.Errorf("validateSignature() should reject signature for "+
+					"host %q (signed for %q)", tt.host, signedHost)
+			}
+		})
+	}
+}
+
+// TestValidateSignature_HostWithPortNeedsSignature verifies that a host
+// followed by a port, or by a colon with no port, matches no allowlist entry,
+// so that a request for it without a signature is refused, while the same
+// host with no port is allowlisted.
+func TestValidateSignature_HostWithPortNeedsSignature(t *testing.T) {
+	t.Parallel()
+
+	h := newSignatureChecker(allowlistedHost, allowlistedIPv4, allowlistedIPv6)
+
+	tests := []struct {
+		host        string
+		allowlisted bool
+	}{
+		{allowlistedHost, true},
+		{allowlistedHost + ":8443", false},
+		{allowlistedHost + ":443", false},
+		{allowlistedHost + ":", false},
+		{allowlistedIPv4, true},
+		{allowlistedIPv4 + ":8080", false},
+		{"[" + allowlistedIPv6 + "]", true},
+		{"[" + allowlistedIPv6 + "]:8443", false},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.host, func(t *testing.T) {
+			t.Parallel()
+
+			err := h.validateSignature(photoRequest(tt.host), "", time.Time{})
+			t.Logf("validateSignature() for host %q with no signature: %v",
+				tt.host, err)
+
+			if allowlisted := err == nil; allowlisted != tt.allowlisted {
+				t.Errorf("host %q served without a signature = %v, want %v",
+					tt.host, allowlisted, tt.allowlisted)
+			}
+		})
+	}
+}
+
+// TestValidateSignature_UnbracketedIPv6PortNeedsSignature verifies that an
+// allowlisted IPv6 address written without brackets and followed by a port,
+// or by a colon with no port, needs a signature. Go's URL type reads
+// 2001:db8::1:8443 as the host 2001:db8::1 with the port 8443.
+func TestValidateSignature_UnbracketedIPv6PortNeedsSignature(t *testing.T) {
+	t.Parallel()
+
+	h := newSignatureChecker(allowlistedIPv6)
+
+	for _, host := range []string{allowlistedIPv6 + ":8443", allowlistedIPv6 + ":"} {
+		t.Run(host, func(t *testing.T) {
+			t.Parallel()
+
+			err := h.validateSignature(photoRequest(host), "", time.Time{})
+			t.Logf("validateSignature() for host %q with no signature: %v",
+				host, err)
+
+			if err == nil {
+				t.Errorf("host %q served without a signature, want refused",
+					host)
+			}
+		})
+	}
+}

internal/httpfetcher/blocked_networks_internal_test.go

@@ -7,6 +7,7 @@ import (
 	"net/http"
 	"net/netip"
 	"testing"
+	"time"
 )
 
 // TestIsPrivateIPBlocksSpecialRanges covers the internal and special-use
@@ -70,12 +71,15 @@ func transportOf(t *testing.T, f *HTTPFetcher) *http.Transport {
 func TestDialerEnforcesBlockedNetworks(t *testing.T) {
 	t.Parallel()
 
-	cfg := DefaultConfig()
-	// TEST-NET-2 (198.51.100.0/24) is public to the built-in check, so
-	// blocking it can only come from the operator-supplied list.
-	cfg.BlockedNetworks = []netip.Prefix{netip.MustParsePrefix("198.51.100.0/24")}
-
-	transport := transportOf(t, New(cfg))
+	transport := transportOf(t, New(&Config{
+		Timeout:               30 * time.Second,
+		MaxResponseSize:       50 << 20,
+		MaxConnectionsPerHost: 20,
+		MaxConnections:        64,
+		// TEST-NET-2 (198.51.100.0/24) is public to the built-in check, so
+		// blocking it can only come from the operator-supplied list.
+		BlockedNetworks: []netip.Prefix{netip.MustParsePrefix("198.51.100.0/24")},
+	}))
 
 	blocked := []string{
 		"198.51.100.5:80", // operator-supplied range

internal/httpfetcher/dial_context_internal_test.go

@@ -4,6 +4,7 @@ import (
 	"errors"
 	"net"
 	"testing"
+	"time"
 )
 
 // TestNewUsesCheckedDialerWithoutDialContext checks that a fetcher built
@@ -13,7 +14,12 @@ func TestNewUsesCheckedDialerWithoutDialContext(t *testing.T) {
 	t.Parallel()
 
 	srv := startUpstream(t)
-	transport := transportOf(t, New(DefaultConfig()))
+	transport := transportOf(t, New(&Config{
+		Timeout:               30 * time.Second,
+		MaxResponseSize:       50 << 20,
+		MaxConnectionsPerHost: 20,
+		MaxConnections:        64,
+	}))
 
 	addr := srv.Listener.Addr().String()
 
@@ -32,21 +38,25 @@ func TestDialContextReplacesOnlyTheDialer(t *testing.T) {
 	srv := startUpstream(t)
 	dialer := &recordingDialer{target: srv.Listener.Addr().String()}
 
-	cfg := DefaultConfig()
-	cfg.AllowHTTP = true
-	cfg.DialContext = dialer.dialContext
-	f := New(cfg)
+	f := New(&Config{
+		Timeout:               30 * time.Second,
+		MaxResponseSize:       50 << 20,
+		AllowHTTP:             true,
+		MaxConnectionsPerHost: 20,
+		MaxConnections:        64,
+		DialContext:           dialer.dialContext,
+	})
 
 	if body := fetchBody(t, f, "/image"); body != imagePayload {
 		t.Errorf("body = %q, want %q", body, imagePayload)
 	}
 
-	_, err := f.Fetch(testContext(t), "http://127.0.0.1/image")
+	_, err := f.Fetch(testContext(t), imageSourceOn("127.0.0.1"))
 	if !errors.Is(err, ErrSSRFBlocked) {
 		t.Errorf("Fetch(loopback URL) error = %v, want ErrSSRFBlocked", err)
 	}
 
-	_, err = f.Fetch(testContext(t), upstreamURL("/redirect/private"))
+	_, err = f.Fetch(testContext(t), upstreamSource("/redirect/private"))
 	if !errors.Is(err, ErrSSRFBlocked) {
 		t.Errorf("Fetch(/redirect/private) error = %v, want ErrSSRFBlocked", err)
 	}

internal/httpfetcher/fetch_internal_test.go

@@ -14,6 +14,8 @@ import (
 	"sync"
 	"testing"
 	"time"
+
+	"sneak.berlin/go/pixa/internal/imagerequest"
 )
 
 // testPublicHost is a TEST-NET-1 (RFC 5737) literal. isPrivateIP treats it as
@@ -29,9 +31,9 @@ const imagePayload = "fake-jpeg-bytes"
 // would mean SSRF protection let a forbidden target reach the transport.
 var errUnexpectedDial = errors.New("unexpected dial target")
 
-// upstreamURL builds a fetch URL on the fake public host for the given path.
-func upstreamURL(path string) string {
-	return "http://" + testPublicHost + path
+// upstreamSource returns the source at path on the fake public host.
+func upstreamSource(path string) imagerequest.Source {
+	return imagerequest.Source{Host: testPublicHost, Path: path}
 }
 
 // recordingDialer records every address the transport asks it to dial and
@@ -120,7 +122,12 @@ func newServerFetcher(
 	t.Helper()
 
 	if cfg == nil {
-		cfg = DefaultConfig()
+		cfg = &Config{
+			Timeout:               30 * time.Second,
+			MaxResponseSize:       50 << 20,
+			MaxConnectionsPerHost: 20,
+			MaxConnections:        64,
+		}
 	}
 
 	cfg.AllowHTTP = true
@@ -153,7 +160,7 @@ func testContext(t *testing.T) context.Context {
 func fetchImage(t *testing.T, f *HTTPFetcher, path string) *FetchResult {
 	t.Helper()
 
-	res, err := f.Fetch(testContext(t), upstreamURL(path))
+	res, err := f.Fetch(testContext(t), upstreamSource(path))
 	if err != nil {
 		t.Fatalf("Fetch(%s) error = %v", path, err)
 	}
@@ -165,7 +172,7 @@ func fetchImage(t *testing.T, f *HTTPFetcher, path string) *FetchResult {
 func fetchExpectError(t *testing.T, f *HTTPFetcher, path string) error {
 	t.Helper()
 
-	res, err := f.Fetch(testContext(t), upstreamURL(path))
+	res, err := f.Fetch(testContext(t), upstreamSource(path))
 	if err == nil {
 		_ = res.Content.Close()
 
@@ -217,7 +224,7 @@ func TestFetchRedirectToPrivateIPBlocked(t *testing.T) {
 	srv := startUpstream(t)
 	f, dialer := newServerFetcher(t, srv, nil)
 
-	_, err := f.Fetch(testContext(t), upstreamURL("/redirect/private"))
+	_, err := f.Fetch(testContext(t), upstreamSource("/redirect/private"))
 	if !errors.Is(err, ErrSSRFBlocked) {
 		t.Fatalf("Fetch() error = %v, want ErrSSRFBlocked", err)
 	}
@@ -280,8 +287,12 @@ func TestFetchMaxResponseSizeEnforced(t *testing.T) {
 
 	srv := startUpstream(t)
 
-	cfg := DefaultConfig()
-	cfg.MaxResponseSize = 8
+	cfg := &Config{
+		Timeout:               30 * time.Second,
+		MaxResponseSize:       8,
+		MaxConnectionsPerHost: 20,
+		MaxConnections:        64,
+	}
 
 	f, _ := newServerFetcher(t, srv, cfg)
 
@@ -310,8 +321,12 @@ func fetchWithMaxResponseSize(
 	srv := httptest.NewServer(handler)
 	t.Cleanup(srv.Close)
 
-	cfg := DefaultConfig()
-	cfg.MaxResponseSize = maxResponseSize
+	cfg := &Config{
+		Timeout:               30 * time.Second,
+		MaxResponseSize:       maxResponseSize,
+		MaxConnectionsPerHost: 20,
+		MaxConnections:        64,
+	}
 
 	f, _ := newServerFetcher(t, srv, cfg)
 
@@ -436,8 +451,12 @@ func TestFetchSemaphoreReleasedOnError(t *testing.T) {
 
 	srv := startUpstream(t)
 
-	cfg := DefaultConfig()
-	cfg.MaxConnectionsPerHost = 1
+	cfg := &Config{
+		Timeout:               30 * time.Second,
+		MaxResponseSize:       50 << 20,
+		MaxConnectionsPerHost: 1,
+		MaxConnections:        64,
+	}
 
 	f, _ := newServerFetcher(t, srv, cfg)
 
@@ -466,8 +485,12 @@ func assertSlotReleasedByClose(
 
 	srv := startUpstream(t)
 
-	cfg := DefaultConfig()
-	cfg.MaxConnectionsPerHost = 1
+	cfg := &Config{
+		Timeout:               30 * time.Second,
+		MaxResponseSize:       50 << 20,
+		MaxConnectionsPerHost: 1,
+		MaxConnections:        64,
+	}
 
 	f, _ := newServerFetcher(t, srv, cfg)
 
@@ -524,7 +547,7 @@ func TestFetchSemaphoreReleasedOnPartialReadClose(t *testing.T) {
 	})
 }
 
-// The dial-time re-resolution in ssrfSafeDialer is what closes the DNS
+// The dial-time re-resolution in dialSSRFSafe is what closes the DNS
 // rebinding window: even if validateURL saw a public answer earlier, the
 // dialer independently re-checks the address it is about to connect to. A full
 // rebinding simulation (a resolver returning public, then private) would mean
@@ -544,9 +567,9 @@ func TestSSRFSafeDialerBlocksPrivateTarget(t *testing.T) {
 		t.Run(addr, func(t *testing.T) {
 			t.Parallel()
 
-			_, err := ssrfSafeDialer(context.Background(), "tcp", addr)
+			_, err := dialSSRFSafe(context.Background(), "tcp", addr, nil)
 			if !errors.Is(err, ErrSSRFBlocked) {
-				t.Errorf("ssrfSafeDialer(%q) = %v, want ErrSSRFBlocked", addr, err)
+				t.Errorf("dialSSRFSafe(%q) = %v, want ErrSSRFBlocked", addr, err)
 			}
 		})
 	}
@@ -560,7 +583,7 @@ func TestSSRFSafeDialerAllowsPublicTarget(t *testing.T) {
 	ctx, cancel := context.WithCancel(context.Background())
 	cancel()
 
-	_, err := ssrfSafeDialer(ctx, "tcp", testPublicHost+":80")
+	_, err := dialSSRFSafe(ctx, "tcp", testPublicHost+":80", nil)
 	if err == nil {
 		t.Fatal("expected a dial error for an unreachable public target")
 	}
@@ -569,3 +592,53 @@ func TestSSRFSafeDialerAllowsPublicTarget(t *testing.T) {
 		t.Errorf("public target was SSRF-blocked: %v", err)
 	}
 }
+
+// TestFetchSchemeFollowsAllowHTTP checks that the fetcher fetches a source
+// over https, on port 443, unless AllowHTTP is set, and then over http, on
+// port 80. The test server speaks plain HTTP, so only the fetch over http
+// succeeds.
+func TestFetchSchemeFollowsAllowHTTP(t *testing.T) {
+	t.Parallel()
+
+	tests := []struct {
+		allowHTTP bool
+		wantAddr  string
+	}{
+		{false, net.JoinHostPort(testPublicHost, "443")},
+		{true, net.JoinHostPort(testPublicHost, "80")},
+	}
+
+	for _, tt := range tests {
+		t.Run(fmt.Sprintf("AllowHTTP %v", tt.allowHTTP), func(t *testing.T) {
+			t.Parallel()
+
+			srv := startUpstream(t)
+			dialer := &recordingDialer{target: srv.Listener.Addr().String()}
+
+			f := New(&Config{
+				Timeout:               30 * time.Second,
+				MaxResponseSize:       50 << 20,
+				AllowHTTP:             tt.allowHTTP,
+				MaxConnectionsPerHost: 20,
+				MaxConnections:        64,
+				DialContext:           dialer.dialContext,
+			})
+
+			res, err := f.Fetch(testContext(t), upstreamSource("/image"))
+			if err == nil {
+				_ = res.Content.Close()
+			}
+
+			dialed := dialer.dialedAddrs()
+			t.Logf("Fetch(): error %v, dialed %v", err, dialed)
+
+			if served := err == nil; served != tt.allowHTTP {
+				t.Errorf("served = %v, want %v", served, tt.allowHTTP)
+			}
+
+			if len(dialed) != 1 || dialed[0] != tt.wantAddr {
+				t.Errorf("dialed %v, want only %s", dialed, tt.wantAddr)
+			}
+		})
+	}
+}

internal/httpfetcher/httpfetcher.go

@@ -20,20 +20,20 @@ import (
 	"time"
 
 	"github.com/go-chi/chi/v5/middleware"
+	"sneak.berlin/go/pixa/internal/imagerequest"
 )
 
 // Fetcher configuration constants.
 const (
-	DefaultFetchTimeout          = 30 * time.Second
-	DefaultMaxResponseSize       = 50 << 20 // 50MB
-	DefaultTLSTimeout            = 10 * time.Second
-	DefaultMaxIdleConns          = 100
-	DefaultIdleConnTimeout       = 90 * time.Second
-	DefaultMaxRedirects          = 10
-	DefaultMaxConnectionsPerHost = 20
-	DefaultMaxConnections        = 64
+	DefaultTLSTimeout      = 10 * time.Second
+	DefaultMaxIdleConns    = 100
+	DefaultIdleConnTimeout = 90 * time.Second
+	DefaultMaxRedirects    = 10
 )
 
+// userAgent is the User-Agent header sent to upstream servers.
+const userAgent = "pixa/1.0"
+
 // ConnectionWaitTimeout is how long Fetch waits for a free connection when
 // MaxConnections fetches are already in progress.
 const ConnectionWaitTimeout = 10 * time.Second
@@ -92,8 +92,8 @@ var (
 
 // Fetcher retrieves content from upstream origins.
 type Fetcher interface {
-	// Fetch retrieves content from the given URL.
-	Fetch(ctx context.Context, url string) (*FetchResult, error)
+	// Fetch retrieves the content of the source image.
+	Fetch(ctx context.Context, source imagerequest.Source) (*FetchResult, error)
 }
 
 // FetchResult contains the result of fetching from upstream.
@@ -120,17 +120,28 @@ type FetchResult struct {
 	TLSCipherSuite string
 }
 
-// Config holds configuration for the upstream fetcher.
+// allowedContentTypes returns the content types an upstream response may
+// have, which Fetch also sends as its Accept header.
+func allowedContentTypes() []string {
+	return []string{
+		contentTypeJPEG,
+		contentTypePNG,
+		contentTypeGIF,
+		contentTypeWebP,
+		contentTypeAVIF,
+		contentTypeJXL,
+	}
+}
+
+// Config holds configuration for the upstream fetcher. Every field but
+// AllowHTTP, BlockedNetworks and DialContext must be set.
 type Config struct {
 	// Timeout for upstream requests.
 	Timeout time.Duration
 	// MaxResponseSize is the maximum allowed response body size.
 	MaxResponseSize int64
-	// UserAgent to send to upstream servers.
-	UserAgent string
-	// AllowedContentTypes is an allow list of MIME types to accept.
-	AllowedContentTypes []string
-	// AllowHTTP allows non-TLS connections (for testing only).
+	// AllowHTTP makes the fetcher fetch sources over http instead of https,
+	// and follow redirects to http (for testing only).
 	AllowHTTP bool
 	// MaxConnectionsPerHost limits concurrent connections to each upstream host.
 	MaxConnectionsPerHost int
@@ -147,26 +158,6 @@ type Config struct {
 	DialContext func(ctx context.Context, network, addr string) (net.Conn, error)
 }
 
-// DefaultConfig returns a Config with sensible defaults.
-func DefaultConfig() *Config {
-	return &Config{
-		Timeout:         DefaultFetchTimeout,
-		MaxResponseSize: DefaultMaxResponseSize,
-		UserAgent:       "pixa/1.0",
-		AllowedContentTypes: []string{
-			contentTypeJPEG,
-			contentTypePNG,
-			contentTypeGIF,
-			contentTypeWebP,
-			contentTypeAVIF,
-			contentTypeJXL,
-		},
-		AllowHTTP:             false,
-		MaxConnectionsPerHost: DefaultMaxConnectionsPerHost,
-		MaxConnections:        DefaultMaxConnections,
-	}
-}
-
 // HTTPFetcher implements Fetcher with SSRF protection and connection limits
 // per host and for all hosts together.
 type HTTPFetcher struct {
@@ -194,10 +185,6 @@ type hostSemaphore struct {
 
 // New creates a new HTTPFetcher with SSRF protection.
 func New(config *Config) *HTTPFetcher {
-	if config == nil {
-		config = DefaultConfig()
-	}
-
 	// Unless config.DialContext replaces it, the transport connects with
 	// the SSRF-safe dialer, which re-resolves and re-checks at connect time
 	// (closing the DNS-rebinding window) against both the built-in ranges
@@ -226,7 +213,7 @@ func New(config *Config) *HTTPFetcher {
 			}
 
 			// Validate the redirect target
-			err := validateURL(req.Context(), req.URL.String(), config.AllowHTTP)
+			err := validateURL(req.Context(), req.URL, config.AllowHTTP)
 			if err != nil {
 				return fmt.Errorf("redirect blocked: %w", err)
 			}
@@ -244,18 +231,22 @@ func New(config *Config) *HTTPFetcher {
 	}
 }
 
-// Fetch retrieves content from the given URL with SSRF protection. When
-// MaxConnections fetches are already in progress, it waits up to
-// ConnectionWaitTimeout for one to finish, then fails with
-// ErrTooManyConnections.
-func (f *HTTPFetcher) Fetch(ctx context.Context, url string) (*FetchResult, error) {
+// Fetch retrieves the source image with SSRF protection, over https, or over
+// http when AllowHTTP is set. When MaxConnections fetches are already in
+// progress, it waits up to ConnectionWaitTimeout for one to finish, then
+// fails with ErrTooManyConnections.
+func (f *HTTPFetcher) Fetch(
+	ctx context.Context, source imagerequest.Source,
+) (*FetchResult, error) {
+	sourceURL := source.URL(f.config.AllowHTTP)
+
 	// Validate URL before making request
-	err := validateURL(ctx, url, f.config.AllowHTTP)
+	err := validateURL(ctx, sourceURL, f.config.AllowHTTP)
 	if err != nil {
 		return nil, err
 	}
 
-	release, err := f.acquireConnection(ctx, extractHost(url))
+	release, err := f.acquireConnection(ctx, source.Host)
 	if err != nil {
 		return nil, err
 	}
@@ -269,19 +260,14 @@ func (f *HTTPFetcher) Fetch(ctx context.Context, url string) (*FetchResult, erro
 		}
 	}()
 
-	parsedURL, err := neturl.Parse(url)
-	if err != nil {
-		return nil, fmt.Errorf("failed to parse URL: %w", err)
-	}
-
 	req := &http.Request{
 		Method: http.MethodGet,
-		URL:    parsedURL,
+		URL:    sourceURL,
 		Header: make(http.Header),
 	}
 
-	req.Header.Set("User-Agent", f.config.UserAgent)
-	req.Header.Set("Accept", strings.Join(f.config.AllowedContentTypes, ", "))
+	req.Header.Set("User-Agent", userAgent)
+	req.Header.Set("Accept", strings.Join(allowedContentTypes(), ", "))
 
 	// The ID of the request this fetch serves, so the fetch can be found in
 	// the upstream host's logs
@@ -460,7 +446,7 @@ func (f *HTTPFetcher) isAllowedContentType(contentType string) bool {
 	// Extract the MIME type without parameters
 	mediaType := strings.TrimSpace(strings.Split(contentType, ";")[0])
 
-	for _, allowed := range f.config.AllowedContentTypes {
+	for _, allowed := range allowedContentTypes() {
 		if strings.EqualFold(mediaType, allowed) {
 			return true
 		}
@@ -470,19 +456,14 @@ func (f *HTTPFetcher) isAllowedContentType(contentType string) bool {
 }
 
 // validateURL checks if a URL is safe to fetch (not internal/private).
-func validateURL(ctx context.Context, rawURL string, allowHTTP bool) error {
-	if !allowHTTP && !strings.HasPrefix(rawURL, "https://") {
+func validateURL(ctx context.Context, fetchURL *neturl.URL, allowHTTP bool) error {
+	if !allowHTTP && fetchURL.Scheme != "https" {
 		return ErrUnsupportedScheme
 	}
 
-	parsedURL, err := neturl.Parse(rawURL)
-	if err != nil {
-		return fmt.Errorf("%w: %w", ErrInvalidHost, err)
-	}
-
 	// Hostname drops any port, and the brackets of an IPv6 address whether
 	// or not a port follows them
-	host := parsedURL.Hostname()
+	host := fetchURL.Hostname()
 	if host == "" {
 		return ErrInvalidHost
 	}
@@ -508,25 +489,6 @@ func validateURL(ctx context.Context, rawURL string, allowHTTP bool) error {
 	return nil
 }
 
-// extractHost extracts the host from a URL string.
-func extractHost(rawURL string) string {
-	// Simple extraction without full URL parsing
-	url := rawURL
-	if idx := strings.Index(url, "://"); idx != -1 {
-		url = url[idx+3:]
-	}
-
-	if idx := strings.Index(url, "/"); idx != -1 {
-		url = url[:idx]
-	}
-
-	if idx := strings.Index(url, "?"); idx != -1 {
-		url = url[:idx]
-	}
-
-	return url
-}
-
 // isLocalhost checks if the host is localhost.
 func isLocalhost(host string) bool {
 	host = strings.ToLower(host)
@@ -639,13 +601,6 @@ func ipv4CompatibleToIPv4(ip net.IP) net.IP {
 	return ip
 }
 
-// ssrfSafeDialer validates IP addresses against the built-in blocked ranges
-// before connecting. New wraps dialSSRFSafe with the operator-supplied
-// blocklist; this entry point enforces the built-in ranges alone.
-func ssrfSafeDialer(ctx context.Context, network, addr string) (net.Conn, error) {
-	return dialSSRFSafe(ctx, network, addr, nil)
-}
-
 // dialSSRFSafe re-resolves addr and refuses to connect to any built-in
 // internal range or operator-supplied blocked prefix, closing the
 // DNS-rebinding window at connect time.

internal/httpfetcher/httpfetcher_internal_test.go

@@ -5,62 +5,26 @@ import (
 	"errors"
 	"io"
 	"net"
+	"net/url"
 	"testing"
 	"testing/fstest"
+	"time"
+
+	"sneak.berlin/go/pixa/internal/imagerequest"
 )
 
 // testHost is the hostname used by mock fetch tests.
 const testHost = "example.com"
 
-func TestDefaultConfig(t *testing.T) {
-	t.Parallel()
-
-	cfg := DefaultConfig()
-
-	if cfg.Timeout != DefaultFetchTimeout {
-		t.Errorf("Timeout = %v, want %v", cfg.Timeout, DefaultFetchTimeout)
-	}
-
-	if cfg.MaxResponseSize != DefaultMaxResponseSize {
-		t.Errorf("MaxResponseSize = %d, want %d", cfg.MaxResponseSize, DefaultMaxResponseSize)
-	}
-
-	if cfg.MaxConnectionsPerHost != DefaultMaxConnectionsPerHost {
-		t.Errorf("MaxConnectionsPerHost = %d, want %d",
-			cfg.MaxConnectionsPerHost, DefaultMaxConnectionsPerHost)
-	}
-
-	if cfg.AllowHTTP {
-		t.Error("AllowHTTP should default to false")
-	}
-
-	if len(cfg.AllowedContentTypes) == 0 {
-		t.Error("AllowedContentTypes should not be empty")
-	}
-}
-
-func TestNewWithNilConfigUsesDefaults(t *testing.T) {
-	t.Parallel()
-
-	f := New(nil)
-
-	if f == nil {
-		t.Fatal("New(nil) returned nil")
-	}
-
-	if f.config == nil {
-		t.Fatal("config should be populated from DefaultConfig")
-	}
-
-	if f.config.Timeout != DefaultFetchTimeout {
-		t.Errorf("Timeout = %v, want %v", f.config.Timeout, DefaultFetchTimeout)
-	}
-}
-
 func TestIsAllowedContentType(t *testing.T) {
 	t.Parallel()
 
-	f := New(DefaultConfig())
+	f := New(&Config{
+		Timeout:               30 * time.Second,
+		MaxResponseSize:       50 << 20,
+		MaxConnectionsPerHost: 20,
+		MaxConnections:        64,
+	})
 
 	tests := []struct {
 		contentType string
@@ -88,33 +52,6 @@ func TestIsAllowedContentType(t *testing.T) {
 	}
 }
 
-func TestExtractHost(t *testing.T) {
-	t.Parallel()
-
-	tests := []struct {
-		url  string
-		want string
-	}{
-		{"https://example.com/path", testHost},
-		{"http://example.com:8080/path", "example.com:8080"},
-		{"https://example.com", testHost},
-		{"https://example.com?q=1", testHost},
-		{"example.com/path", testHost},
-		{"", ""},
-	}
-
-	for _, tc := range tests {
-		t.Run(tc.url, func(t *testing.T) {
-			t.Parallel()
-
-			got := extractHost(tc.url)
-			if got != tc.want {
-				t.Errorf("extractHost(%q) = %q, want %q", tc.url, got, tc.want)
-			}
-		})
-	}
-}
-
 func TestIsLocalhost(t *testing.T) {
 	t.Parallel()
 
@@ -227,7 +164,7 @@ func TestIsPrivateIPJudgesEmbeddedIPv4Address(t *testing.T) {
 func TestValidateURL_RejectsNonHTTPS(t *testing.T) {
 	t.Parallel()
 
-	err := validateURL(t.Context(), "http://example.com/path", false)
+	err := validateURL(t.Context(), &url.URL{Scheme: "http", Host: testHost}, false)
 	if !errors.Is(err, ErrUnsupportedScheme) {
 		t.Errorf("validateURL http = %v, want ErrUnsupportedScheme", err)
 	}
@@ -237,7 +174,8 @@ func TestValidateURL_AllowsHTTPWhenConfigured(t *testing.T) {
 	t.Parallel()
 
 	// Use a host that won't resolve (explicit .invalid TLD) so we don't hit DNS.
-	err := validateURL(t.Context(), "http://nonexistent.invalid/path", true)
+	err := validateURL(t.Context(),
+		&url.URL{Scheme: "http", Host: "nonexistent.invalid"}, true)
 	// We expect a host resolution error, not ErrUnsupportedScheme.
 	if errors.Is(err, ErrUnsupportedScheme) {
 		t.Error("validateURL with AllowHTTP should not return ErrUnsupportedScheme")
@@ -247,7 +185,7 @@ func TestValidateURL_AllowsHTTPWhenConfigured(t *testing.T) {
 func TestValidateURL_RejectsLocalhost(t *testing.T) {
 	t.Parallel()
 
-	err := validateURL(t.Context(), "https://localhost/path", false)
+	err := validateURL(t.Context(), &url.URL{Scheme: "https", Host: "localhost"}, false)
 	if !errors.Is(err, ErrSSRFBlocked) {
 		t.Errorf("validateURL localhost = %v, want ErrSSRFBlocked", err)
 	}
@@ -256,7 +194,7 @@ func TestValidateURL_RejectsLocalhost(t *testing.T) {
 func TestValidateURL_EmptyHost(t *testing.T) {
 	t.Parallel()
 
-	err := validateURL(t.Context(), "https:///path", false)
+	err := validateURL(t.Context(), &url.URL{Scheme: "https"}, false)
 	if !errors.Is(err, ErrInvalidHost) {
 		t.Errorf("validateURL empty host = %v, want ErrInvalidHost", err)
 	}
@@ -271,7 +209,8 @@ func TestMockFetcher_FetchesFile(t *testing.T) {
 
 	m := NewMock(mockFS)
 
-	result, err := m.Fetch(context.Background(), "https://example.com/images/photo.jpg")
+	result, err := m.Fetch(context.Background(),
+		imagerequest.Source{Host: testHost, Path: "/images/photo.jpg"})
 	if err != nil {
 		t.Fatalf("Fetch() error = %v", err)
 	}
@@ -302,7 +241,8 @@ func TestMockFetcher_MissingFileReturnsUpstreamError(t *testing.T) {
 	mockFS := fstest.MapFS{}
 	m := NewMock(mockFS)
 
-	_, err := m.Fetch(context.Background(), "https://example.com/missing.jpg")
+	_, err := m.Fetch(context.Background(),
+		imagerequest.Source{Host: testHost, Path: "/missing.jpg"})
 	if !errors.Is(err, ErrUpstreamError) {
 		t.Errorf("Fetch() error = %v, want ErrUpstreamError", err)
 	}
@@ -319,7 +259,7 @@ func TestMockFetcher_RespectsContextCancellation(t *testing.T) {
 	ctx, cancel := context.WithCancel(context.Background())
 	cancel()
 
-	_, err := m.Fetch(ctx, "https://example.com/photo.jpg")
+	_, err := m.Fetch(ctx, imagerequest.Source{Host: testHost, Path: "/photo.jpg"})
 	if !errors.Is(err, context.Canceled) {
 		t.Errorf("Fetch() error = %v, want context.Canceled", err)
 	}

internal/httpfetcher/jpegxl_internal_test.go

@@ -1,16 +1,26 @@
 package httpfetcher
 
-import "testing"
+import (
+	"testing"
+	"time"
+)
 
 // TestJPEGXLContentType verifies that the fetcher accepts an upstream answer
-// of type image/jxl by default, and that the mock fetcher serves a .jxl file
-// as image/jxl.
+// of type image/jxl, and that the mock fetcher serves a .jxl file as
+// image/jxl.
 func TestJPEGXLContentType(t *testing.T) {
 	t.Parallel()
 
 	const jxlType = "image/jxl"
 
-	if !New(DefaultConfig()).isAllowedContentType(jxlType) {
+	f := New(&Config{
+		Timeout:               30 * time.Second,
+		MaxResponseSize:       50 << 20,
+		MaxConnectionsPerHost: 20,
+		MaxConnections:        64,
+	})
+
+	if !f.isAllowedContentType(jxlType) {
 		t.Errorf("isAllowedContentType(%q) = false, want true", jxlType)
 	}
 

internal/httpfetcher/max_connections_internal_test.go

@@ -8,22 +8,20 @@ import (
 	"sync"
 	"testing"
 	"time"
+
+	"sneak.berlin/go/pixa/internal/imagerequest"
 )
 
-// imageURLOnPort is the fake upstream's image route on testPublicHost at
-// port. Each port is a different host to the per-host limit, while the test
-// dialer sends every port to the one test server.
-func imageURLOnPort(port int) string {
-	return "http://" + net.JoinHostPort(testPublicHost, strconv.Itoa(port)) +
-		"/image"
+// imageSourceOn is the fake upstream's image route on host.
+func imageSourceOn(host string) imagerequest.Source {
+	return imagerequest.Source{Host: host, Path: "/image"}
 }
 
-func TestDefaultConfigMaxConnections(t *testing.T) {
-	t.Parallel()
-
-	if got := DefaultConfig().MaxConnections; got != DefaultMaxConnections {
-		t.Errorf("MaxConnections = %d, want %d", got, DefaultMaxConnections)
-	}
+// imageSourceOnPort is the fake upstream's image route on testPublicHost at
+// port. Each port is a different host to the per-host limit, while the test
+// dialer sends every port to the one test server.
+func imageSourceOnPort(port int) imagerequest.Source {
+	return imageSourceOn(net.JoinHostPort(testPublicHost, strconv.Itoa(port)))
 }
 
 // TestFetchLimitsConnectionsToAllHostsTogether checks that MaxConnections
@@ -37,18 +35,22 @@ func TestFetchLimitsConnectionsToAllHostsTogether(t *testing.T) {
 
 	srv := startUpstream(t)
 
-	cfg := DefaultConfig()
-	cfg.MaxConnections = 2
+	cfg := &Config{
+		Timeout:               30 * time.Second,
+		MaxResponseSize:       50 << 20,
+		MaxConnectionsPerHost: 20,
+		MaxConnections:        2,
+	}
 
 	f, _ := newServerFetcher(t, srv, cfg)
 	f.connectionWaitTimeout = 100 * time.Millisecond
 
-	first, err := f.Fetch(testContext(t), imageURLOnPort(81))
+	first, err := f.Fetch(testContext(t), imageSourceOnPort(81))
 	if err != nil {
 		t.Fatalf("first Fetch() error = %v", err)
 	}
 
-	second, err := f.Fetch(testContext(t), imageURLOnPort(82))
+	second, err := f.Fetch(testContext(t), imageSourceOnPort(82))
 	if err != nil {
 		t.Fatalf("second Fetch() error = %v", err)
 	}
@@ -57,7 +59,7 @@ func TestFetchLimitsConnectionsToAllHostsTogether(t *testing.T) {
 
 	start := time.Now()
 
-	_, err = f.Fetch(testContext(t), imageURLOnPort(83))
+	_, err = f.Fetch(testContext(t), imageSourceOnPort(83))
 	if !errors.Is(err, ErrTooManyConnections) {
 		t.Fatalf("third Fetch() error = %v, want ErrTooManyConnections", err)
 	}
@@ -76,7 +78,7 @@ func TestFetchLimitsConnectionsToAllHostsTogether(t *testing.T) {
 		t.Fatalf("close first body: %v", err)
 	}
 
-	third, err := f.Fetch(testContext(t), imageURLOnPort(83))
+	third, err := f.Fetch(testContext(t), imageSourceOnPort(83))
 	if err != nil {
 		t.Fatalf("Fetch() after a response was closed: error = %v", err)
 	}
@@ -94,12 +96,16 @@ func TestFetchFreesHostSlotWhenContextEndsWaitingForConnection(t *testing.T) {
 
 	srv := startUpstream(t)
 
-	cfg := DefaultConfig()
-	cfg.MaxConnections = 1
+	cfg := &Config{
+		Timeout:               30 * time.Second,
+		MaxResponseSize:       50 << 20,
+		MaxConnectionsPerHost: 20,
+		MaxConnections:        1,
+	}
 
 	f, _ := newServerFetcher(t, srv, cfg)
 
-	first, err := f.Fetch(testContext(t), imageURLOnPort(81))
+	first, err := f.Fetch(testContext(t), imageSourceOnPort(81))
 	if err != nil {
 		t.Fatalf("first Fetch() error = %v", err)
 	}
@@ -109,7 +115,7 @@ func TestFetchFreesHostSlotWhenContextEndsWaitingForConnection(t *testing.T) {
 	ctx, cancel := context.WithTimeout(t.Context(), 100*time.Millisecond)
 	defer cancel()
 
-	_, err = f.Fetch(ctx, imageURLOnPort(82))
+	_, err = f.Fetch(ctx, imageSourceOnPort(82))
 	if !errors.Is(err, context.DeadlineExceeded) {
 		t.Fatalf("second Fetch() error = %v, want context.DeadlineExceeded", err)
 	}
@@ -134,7 +140,7 @@ func TestFetchRemovesIdleHostSemaphores(t *testing.T) {
 
 	for i := range 100 {
 		wg.Go(func() {
-			res, err := f.Fetch(ctx, imageURLOnPort(1+i%50))
+			res, err := f.Fetch(ctx, imageSourceOnPort(1+i%50))
 			if err != nil {
 				t.Errorf("Fetch() error = %v", err)
 
@@ -163,19 +169,22 @@ func TestFetchRemovesHostSemaphoreWhenNoConnection(t *testing.T) {
 
 	srv := startUpstream(t)
 
-	cfg := DefaultConfig()
-	cfg.MaxConnections = 1
-	cfg.MaxConnectionsPerHost = 1
+	cfg := &Config{
+		Timeout:               30 * time.Second,
+		MaxResponseSize:       50 << 20,
+		MaxConnectionsPerHost: 1,
+		MaxConnections:        1,
+	}
 
 	f, _ := newServerFetcher(t, srv, cfg)
 	f.connectionWaitTimeout = 100 * time.Millisecond
 
-	open, err := f.Fetch(testContext(t), imageURLOnPort(81))
+	open, err := f.Fetch(testContext(t), imageSourceOnPort(81))
 	if err != nil {
 		t.Fatalf("first Fetch() error = %v", err)
 	}
 
-	_, err = f.Fetch(testContext(t), imageURLOnPort(82))
+	_, err = f.Fetch(testContext(t), imageSourceOnPort(82))
 	if !errors.Is(err, ErrTooManyConnections) {
 		t.Fatalf("Fetch() from another host: error = %v, "+
 			"want ErrTooManyConnections", err)
@@ -184,7 +193,7 @@ func TestFetchRemovesHostSemaphoreWhenNoConnection(t *testing.T) {
 	ctx, cancel := context.WithTimeout(t.Context(), 100*time.Millisecond)
 	defer cancel()
 
-	_, err = f.Fetch(ctx, imageURLOnPort(81))
+	_, err = f.Fetch(ctx, imageSourceOnPort(81))
 	if !errors.Is(err, context.DeadlineExceeded) {
 		t.Fatalf("Fetch() from the busy host: error = %v, "+
 			"want context.DeadlineExceeded", err)
@@ -196,7 +205,7 @@ func TestFetchRemovesHostSemaphoreWhenNoConnection(t *testing.T) {
 	ctx, cancel = context.WithTimeout(t.Context(), 100*time.Millisecond)
 	defer cancel()
 
-	_, err = f.Fetch(ctx, imageURLOnPort(83))
+	_, err = f.Fetch(ctx, imageSourceOnPort(83))
 	if !errors.Is(err, context.DeadlineExceeded) {
 		t.Fatalf("Fetch() from a host with nothing open: error = %v, "+
 			"want context.DeadlineExceeded", err)
@@ -219,15 +228,15 @@ func TestFetchReleasesConnectionOnError(t *testing.T) {
 	t.Parallel()
 
 	cases := []struct {
-		name string
-		url  string
-		want error
+		name   string
+		source imagerequest.Source
+		want   error
 	}{
-		{"upstream answers 500", upstreamURL("/status/500"), ErrUpstreamError},
-		{"upstream sends HTML", upstreamURL("/html"), ErrInvalidContentType},
+		{"upstream answers 500", upstreamSource("/status/500"), ErrUpstreamError},
+		{"upstream sends HTML", upstreamSource("/html"), ErrInvalidContentType},
 		// 198.51.100.7 (TEST-NET-2) passes the SSRF checks, and the test
 		// dialer refuses every host but testPublicHost.
-		{"connecting fails", "http://198.51.100.7/image", errUnexpectedDial},
+		{"connecting fails", imageSourceOn("198.51.100.7"), errUnexpectedDial},
 	}
 
 	for _, tc := range cases {
@@ -236,13 +245,17 @@ func TestFetchReleasesConnectionOnError(t *testing.T) {
 
 			srv := startUpstream(t)
 
-			cfg := DefaultConfig()
-			cfg.MaxConnections = 1
+			cfg := &Config{
+				Timeout:               30 * time.Second,
+				MaxResponseSize:       50 << 20,
+				MaxConnectionsPerHost: 20,
+				MaxConnections:        1,
+			}
 
 			f, _ := newServerFetcher(t, srv, cfg)
 			f.connectionWaitTimeout = 100 * time.Millisecond
 
-			_, err := f.Fetch(testContext(t), tc.url)
+			_, err := f.Fetch(testContext(t), tc.source)
 			if !errors.Is(err, tc.want) {
 				t.Fatalf("Fetch() error = %v, want %v", err, tc.want)
 			}

internal/httpfetcher/mock.go

@@ -7,14 +7,14 @@ import (
 	"io/fs"
 	"net/http"
 	"strings"
-)
 
-// errEmptyURLPath is returned when a mock URL has no usable path.
-var errEmptyURLPath = errors.New("empty URL path")
+	"sneak.berlin/go/pixa/internal/imagerequest"
+)
 
 // MockFetcher implements Fetcher using an embedded filesystem.
 // Files are organized as: hostname/path/to/file.ext
-// URLs like https://example.com/images/photo.jpg map to example.com/images/photo.jpg.
+// The source with host example.com and path /images/photo.jpg maps to
+// example.com/images/photo.jpg; the source query is ignored.
 type MockFetcher struct {
 	fs fs.FS
 }
@@ -24,8 +24,10 @@ func NewMock(fsys fs.FS) *MockFetcher {
 	return &MockFetcher{fs: fsys}
 }
 
-// Fetch retrieves content from the mock filesystem.
-func (m *MockFetcher) Fetch(ctx context.Context, url string) (*FetchResult, error) {
+// Fetch retrieves the source's content from the mock filesystem.
+func (m *MockFetcher) Fetch(
+	ctx context.Context, source imagerequest.Source,
+) (*FetchResult, error) {
 	// Check context cancellation
 	select {
 	case <-ctx.Done():
@@ -33,11 +35,7 @@ func (m *MockFetcher) Fetch(ctx context.Context, url string) (*FetchResult, erro
 	default:
 	}
 
-	// Parse URL to get filesystem path
-	path, err := urlToFSPath(url)
-	if err != nil {
-		return nil, err
-	}
+	path := source.Host + source.Path
 
 	// Open the file
 	f, err := m.fs.Open(path)
@@ -68,32 +66,6 @@ func (m *MockFetcher) Fetch(ctx context.Context, url string) (*FetchResult, erro
 	}, nil
 }
 
-// urlToFSPath converts a URL to a filesystem path.
-// https://example.com/images/photo.jpg -> example.com/images/photo.jpg
-func urlToFSPath(rawURL string) (string, error) {
-	// Strip scheme
-	url := rawURL
-	if idx := strings.Index(url, "://"); idx != -1 {
-		url = url[idx+3:]
-	}
-
-	// Remove query string
-	if idx := strings.Index(url, "?"); idx != -1 {
-		url = url[:idx]
-	}
-
-	// Remove fragment
-	if idx := strings.Index(url, "#"); idx != -1 {
-		url = url[:idx]
-	}
-
-	if url == "" {
-		return "", errEmptyURLPath
-	}
-
-	return url, nil
-}
-
 // detectContentTypeFromPath returns the MIME type based on file extension.
 func detectContentTypeFromPath(path string) string {
 	path = strings.ToLower(path)

internal/httpfetcher/request_id_internal_test.go

@@ -34,7 +34,7 @@ func TestFetchSendsRequestID(t *testing.T) {
 
 	ctx := context.WithValue(testContext(t), middleware.RequestIDKey, requestID)
 
-	res, err := f.Fetch(ctx, upstreamURL("/image"))
+	res, err := f.Fetch(ctx, upstreamSource("/image"))
 	if err != nil {
 		t.Fatalf("Fetch() error = %v", err)
 	}

internal/imageprocessor/avif_too_large_internal_test.go

@@ -69,7 +69,10 @@ func TestImageProcessor_RefusesAVIFAboveMaxPixels(t *testing.T) {
 		t.Run(tt.name, func(t *testing.T) {
 			t.Parallel()
 
-			_, err := New(Params{}).Process(t.Context(), bytes.NewReader(tt.input),
+			_, err := New(Params{
+				MaxInputBytes:           50 << 20,
+				MaxConcurrentProcessing: 1,
+			}).Process(t.Context(), bytes.NewReader(tt.input),
 				&imagerequest.Transform{
 					Size:    imagerequest.Size{Width: tt.width, Height: tt.height},
 					Format:  tt.format,

internal/imageprocessor/computed_side_internal_test.go

@@ -38,7 +38,10 @@ func TestImageProcessor_RefusesComputedSideAboveMaximum(t *testing.T) {
 
 			input := createTestJPEG(t, tt.inputWidth, tt.inputHeight)
 
-			result, err := New(Params{}).Process(t.Context(),
+			result, err := New(Params{
+				MaxInputBytes:           50 << 20,
+				MaxConcurrentProcessing: 1,
+			}).Process(t.Context(),
 				bytes.NewReader(input),
 				&imagerequest.Transform{
 					Size:    tt.size,
@@ -104,7 +107,10 @@ func TestImageProcessor_ComputedSideIsAtLeastOne(t *testing.T) {
 
 				input := createTestJPEG(t, tt.inputWidth, tt.inputHeight)
 
-				result, err := New(Params{}).Process(t.Context(),
+				result, err := New(Params{
+					MaxInputBytes:           50 << 20,
+					MaxConcurrentProcessing: 1,
+				}).Process(t.Context(),
 					bytes.NewReader(input),
 					&imagerequest.Transform{
 						Size:    tt.size,

internal/imageprocessor/deadline_internal_test.go

@@ -84,7 +84,7 @@ func checkProcessStopsAtDeadline(
 ) {
 	t.Helper()
 
-	proc := New(Params{MaxConcurrentProcessing: 1})
+	proc := New(Params{MaxInputBytes: 50 << 20, MaxConcurrentProcessing: 1})
 
 	result, err := proc.Process(ctx, input, smallJPEGTransform())
 	if result != nil {

internal/imageprocessor/empty_format_internal_test.go

@@ -14,7 +14,10 @@ import (
 func TestImageProcessor_EmptyFormatRefused(t *testing.T) {
 	t.Parallel()
 
-	_, err := New(Params{}).Process(
+	_, err := New(Params{
+		MaxInputBytes:           50 << 20,
+		MaxConcurrentProcessing: 1,
+	}).Process(
 		t.Context(), bytes.NewReader(createTestJPEG(t, 20, 20)),
 		&imagerequest.Transform{},
 	)

internal/imageprocessor/encoder_settings_internal_test.go

@@ -18,7 +18,10 @@ func processedSize(
 ) int64 {
 	t.Helper()
 
-	result, err := New(Params{}).Process(t.Context(), bytes.NewReader(input), transform)
+	result, err := New(Params{
+		MaxInputBytes:           50 << 20,
+		MaxConcurrentProcessing: 1,
+	}).Process(t.Context(), bytes.NewReader(input), transform)
 	if err != nil {
 		t.Fatalf("Process() error = %v", err)
 	}

internal/imageprocessor/fit_cover_internal_test.go

@@ -36,7 +36,10 @@ func TestImageProcessor_FitCoverServesThinSource(t *testing.T) {
 
 			input := createTestJPEG(t, tt.inputWidth, tt.inputHeight)
 
-			result, err := New(Params{}).Process(t.Context(),
+			result, err := New(Params{
+				MaxInputBytes:           50 << 20,
+				MaxConcurrentProcessing: 1,
+			}).Process(t.Context(),
 				bytes.NewReader(input),
 				&imagerequest.Transform{
 					Size:    tt.size,
@@ -106,7 +109,10 @@ func TestImageProcessor_FitCoverCropsTheCentre(t *testing.T) {
 
 			input := createBandsPNG(t, tt.inputWidth, tt.inputHeight)
 
-			result, err := New(Params{}).Process(t.Context(),
+			result, err := New(Params{
+				MaxInputBytes:           50 << 20,
+				MaxConcurrentProcessing: 1,
+			}).Process(t.Context(),
 				bytes.NewReader(input),
 				&imagerequest.Transform{
 					Size:    imagerequest.Size{Width: 50, Height: 50},
@@ -230,7 +236,10 @@ func TestImageProcessor_FitCoverKeepsProportions(t *testing.T) {
 
 			input := createSquarePNG(t, tt.side, tt.squareSide)
 
-			result, err := New(Params{}).Process(t.Context(),
+			result, err := New(Params{
+				MaxInputBytes:           50 << 20,
+				MaxConcurrentProcessing: 1,
+			}).Process(t.Context(),
 				bytes.NewReader(input),
 				&imagerequest.Transform{
 					Size:    tt.size,

internal/imageprocessor/fit_outside_internal_test.go

@@ -41,7 +41,10 @@ func TestImageProcessor_RefusesFitOutsideAboveMaximum(t *testing.T) {
 
 			input := createTestJPEG(t, tt.inputWidth, tt.inputHeight)
 
-			result, err := New(Params{}).Process(t.Context(),
+			result, err := New(Params{
+				MaxInputBytes:           50 << 20,
+				MaxConcurrentProcessing: 1,
+			}).Process(t.Context(),
 				bytes.NewReader(input),
 				&imagerequest.Transform{
 					Size:    tt.size,
@@ -95,7 +98,10 @@ func TestImageProcessor_FitOutsideSideIsAtLeastOne(t *testing.T) {
 
 			input := createTestJPEG(t, tt.inputWidth, tt.inputHeight)
 
-			result, err := New(Params{}).Process(t.Context(),
+			result, err := New(Params{
+				MaxInputBytes:           50 << 20,
+				MaxConcurrentProcessing: 1,
+			}).Process(t.Context(),
 				bytes.NewReader(input),
 				&imagerequest.Transform{
 					Size:    tt.size,

internal/imageprocessor/imageprocessor.go

@@ -11,7 +11,6 @@ import (
 	"time"
 
 	"github.com/davidbyttow/govips/v2/vips"
-	"sneak.berlin/go/pixa/internal/config"
 	"sneak.berlin/go/pixa/internal/imagerequest"
 )
 
@@ -79,10 +78,6 @@ type Result struct {
 // Images larger than this are rejected to prevent DoS via decompression bombs.
 const MaxInputDimension = 8192
 
-// DefaultMaxInputBytes is the default maximum input size in bytes (50 MiB).
-// This matches the default upstream fetcher limit.
-const DefaultMaxInputBytes = 50 << 20
-
 // ErrInputTooLarge is returned when input image dimensions exceed MaxInputDimension.
 var ErrInputTooLarge = errors.New("input image dimensions exceed maximum")
 
@@ -142,35 +137,22 @@ type ImageProcessor struct {
 	processingWaitTimeout time.Duration
 }
 
-// Params holds configuration for creating an ImageProcessor.
-// Zero values use sensible defaults (MaxInputBytes defaults to DefaultMaxInputBytes).
+// Params holds configuration for creating an ImageProcessor. Both fields
+// must be at least 1.
 type Params struct {
 	// MaxInputBytes is the maximum allowed input size in bytes.
-	// If <= 0, DefaultMaxInputBytes is used.
 	MaxInputBytes int64
 	// MaxConcurrentProcessing is the most images processed at once.
-	// If <= 0, config.DefaultMaxConcurrentProcessing() is used.
 	MaxConcurrentProcessing int
 }
 
 // New creates a new image processor with the given parameters.
-// A zero-value Params{} uses sensible defaults.
 func New(params Params) *ImageProcessor {
 	initVips()
 
-	maxInputBytes := params.MaxInputBytes
-	if maxInputBytes <= 0 {
-		maxInputBytes = DefaultMaxInputBytes
-	}
-
-	maxConcurrentProcessing := params.MaxConcurrentProcessing
-	if maxConcurrentProcessing <= 0 {
-		maxConcurrentProcessing = config.DefaultMaxConcurrentProcessing()
-	}
-
 	return &ImageProcessor{
-		maxInputBytes:         maxInputBytes,
-		processingSemaphore:   make(chan struct{}, maxConcurrentProcessing),
+		maxInputBytes:         params.MaxInputBytes,
+		processingSemaphore:   make(chan struct{}, params.MaxConcurrentProcessing),
 		processingWaitTimeout: ProcessingWaitTimeout,
 	}
 }
@@ -322,30 +304,6 @@ const (
 	mimeJXL  = "image/jxl"
 )
 
-// SupportedInputFormats returns MIME types this processor can read.
-func (p *ImageProcessor) SupportedInputFormats() []string {
-	return []string{
-		mimeJPEG,
-		mimePNG,
-		mimeGIF,
-		mimeWebP,
-		mimeAVIF,
-		mimeJXL,
-	}
-}
-
-// SupportedOutputFormats returns formats this processor can write.
-func (p *ImageProcessor) SupportedOutputFormats() []imagerequest.Format {
-	return []imagerequest.Format{
-		imagerequest.FormatJPEG,
-		imagerequest.FormatPNG,
-		imagerequest.FormatGIF,
-		imagerequest.FormatWebP,
-		imagerequest.FormatAVIF,
-		imagerequest.FormatJXL,
-	}
-}
-
 // FormatToMIME converts a Format to its MIME type string.
 func FormatToMIME(format imagerequest.Format) string {
 	switch format {

internal/imageprocessor/imageprocessor_internal_test.go

@@ -119,7 +119,7 @@ func detectMIME(data []byte) string {
 func TestImageProcessor_ResizeJPEG(t *testing.T) {
 	t.Parallel()
 
-	proc := New(Params{})
+	proc := New(Params{MaxInputBytes: 50 << 20, MaxConcurrentProcessing: 1})
 	ctx := context.Background()
 
 	input := createTestJPEG(t, 800, 600)
@@ -165,7 +165,7 @@ func TestImageProcessor_ResizeJPEG(t *testing.T) {
 func TestImageProcessor_ConvertToPNG(t *testing.T) {
 	t.Parallel()
 
-	proc := New(Params{})
+	proc := New(Params{MaxInputBytes: 50 << 20, MaxConcurrentProcessing: 1})
 	ctx := context.Background()
 
 	input := createTestJPEG(t, 200, 150)
@@ -201,7 +201,7 @@ func processAndCheckSize(
 ) {
 	t.Helper()
 
-	proc := New(Params{})
+	proc := New(Params{MaxInputBytes: 50 << 20, MaxConcurrentProcessing: 1})
 	ctx := context.Background()
 
 	input := createTestJPEG(t, inputW, inputH)
@@ -239,7 +239,7 @@ func TestImageProcessor_OriginalSize(t *testing.T) {
 func TestImageProcessor_FitContain(t *testing.T) {
 	t.Parallel()
 
-	proc := New(Params{})
+	proc := New(Params{MaxInputBytes: 50 << 20, MaxConcurrentProcessing: 1})
 	ctx := context.Background()
 
 	// 800x400 image (2:1 aspect) into 400x400 box with contain
@@ -285,7 +285,7 @@ func TestImageProcessor_ProportionalScale_HeightOnly(t *testing.T) {
 func TestImageProcessor_ProcessPNG(t *testing.T) {
 	t.Parallel()
 
-	proc := New(Params{})
+	proc := New(Params{MaxInputBytes: 50 << 20, MaxConcurrentProcessing: 1})
 	ctx := context.Background()
 
 	input := createTestPNG(t, 400, 300)
@@ -312,22 +312,6 @@ func TestImageProcessor_ProcessPNG(t *testing.T) {
 	}
 }
 
-func TestImageProcessor_SupportedFormats(t *testing.T) {
-	t.Parallel()
-
-	proc := New(Params{})
-
-	inputFormats := proc.SupportedInputFormats()
-	if len(inputFormats) == 0 {
-		t.Error("SupportedInputFormats() returned empty slice")
-	}
-
-	outputFormats := proc.SupportedOutputFormats()
-	if len(outputFormats) == 0 {
-		t.Error("SupportedOutputFormats() returned empty slice")
-	}
-}
-
 func TestImageProcessor_RejectsOversizedInput(t *testing.T) {
 	t.Parallel()
 
@@ -346,7 +330,7 @@ func TestImageProcessor_RejectsOversizedInput(t *testing.T) {
 		t.Run(tt.name, func(t *testing.T) {
 			t.Parallel()
 
-			proc := New(Params{})
+			proc := New(Params{MaxInputBytes: 50 << 20, MaxConcurrentProcessing: 1})
 			ctx := context.Background()
 			input := createTestJPEG(t, tt.width, tt.height)
 
@@ -372,7 +356,7 @@ func TestImageProcessor_RejectsOversizedInput(t *testing.T) {
 func TestImageProcessor_AcceptsMaxDimensionInput(t *testing.T) {
 	t.Parallel()
 
-	proc := New(Params{})
+	proc := New(Params{MaxInputBytes: 50 << 20, MaxConcurrentProcessing: 1})
 	ctx := context.Background()
 
 	// Create an image at exactly MaxInputDimension - should be accepted
@@ -403,7 +387,7 @@ func encodeAndCheck(
 ) {
 	t.Helper()
 
-	proc := New(Params{})
+	proc := New(Params{MaxInputBytes: 50 << 20, MaxConcurrentProcessing: 1})
 	ctx := context.Background()
 
 	input := createTestJPEG(t, 200, 150)
@@ -452,7 +436,7 @@ func TestImageProcessor_EncodeWebP(t *testing.T) {
 func TestImageProcessor_DecodeAVIF(t *testing.T) {
 	t.Parallel()
 
-	proc := New(Params{})
+	proc := New(Params{MaxInputBytes: 50 << 20, MaxConcurrentProcessing: 1})
 	ctx := context.Background()
 
 	// Load test AVIF file
@@ -494,7 +478,7 @@ func TestImageProcessor_RejectsOversizedInputData(t *testing.T) {
 	// Create a processor with a very small byte limit
 	const limit = 1024
 
-	proc := New(Params{MaxInputBytes: limit})
+	proc := New(Params{MaxInputBytes: limit, MaxConcurrentProcessing: 1})
 	ctx := context.Background()
 
 	// Create a valid JPEG that exceeds the byte limit
@@ -527,7 +511,7 @@ func TestImageProcessor_AcceptsInputWithinLimit(t *testing.T) {
 	input := createTestJPEG(t, 10, 10)
 	limit := int64(len(input)) * 10 // 10× headroom
 
-	proc := New(Params{MaxInputBytes: limit})
+	proc := New(Params{MaxInputBytes: limit, MaxConcurrentProcessing: 1})
 	ctx := context.Background()
 
 	transform := &imagerequest.Transform{
@@ -545,22 +529,6 @@ func TestImageProcessor_AcceptsInputWithinLimit(t *testing.T) {
 	defer func() { _ = result.Content.Close() }()
 }
 
-func TestImageProcessor_DefaultMaxInputBytes(t *testing.T) {
-	t.Parallel()
-
-	// Passing 0 should use the default
-	proc := New(Params{})
-	if proc.maxInputBytes != DefaultMaxInputBytes {
-		t.Errorf("maxInputBytes = %d, want %d", proc.maxInputBytes, DefaultMaxInputBytes)
-	}
-
-	// Passing negative should also use the default
-	proc = New(Params{MaxInputBytes: -1})
-	if proc.maxInputBytes != DefaultMaxInputBytes {
-		t.Errorf("maxInputBytes = %d, want %d", proc.maxInputBytes, DefaultMaxInputBytes)
-	}
-}
-
 func TestImageProcessor_EncodeAVIF(t *testing.T) {
 	t.Parallel()
 
@@ -574,7 +542,10 @@ func processAndDecode(
 ) *vips.ImageRef {
 	t.Helper()
 
-	result, err := New(Params{}).Process(
+	result, err := New(Params{
+		MaxInputBytes:           50 << 20,
+		MaxConcurrentProcessing: 1,
+	}).Process(
 		context.Background(), bytes.NewReader(input), transform,
 	)
 	if err != nil {

internal/imageprocessor/jpegxl_output_internal_test.go

@@ -21,7 +21,10 @@ func TestImageProcessor_EncodeJPEGXL(t *testing.T) {
 		Format: imagerequest.FormatJXL,
 	}
 
-	result, err := New(Params{}).Process(
+	result, err := New(Params{
+		MaxInputBytes:           50 << 20,
+		MaxConcurrentProcessing: 1,
+	}).Process(
 		t.Context(), bytes.NewReader(createTestJPEG(t, 200, 150)), transform,
 	)
 	if err != nil {
@@ -64,7 +67,10 @@ func TestImageProcessor_JPEGXLQuality(t *testing.T) {
 	outputBytes := make(map[int]int64)
 
 	for _, quality := range []int{30, 90} {
-		result, err := New(Params{}).Process(t.Context(),
+		result, err := New(Params{
+			MaxInputBytes:           50 << 20,
+			MaxConcurrentProcessing: 1,
+		}).Process(t.Context(),
 			bytes.NewReader(input),
 			&imagerequest.Transform{Format: imagerequest.FormatJXL, Quality: quality})
 		if err != nil {

internal/imageprocessor/max_concurrent_processing_internal_test.go

@@ -5,7 +5,6 @@ import (
 	"context"
 	"errors"
 	"io"
-	"runtime"
 	"strings"
 	"sync"
 	"testing"
@@ -117,47 +116,15 @@ func waitForEntries(t *testing.T, entered <-chan struct{}, count int) {
 	}
 }
 
-func TestNewDefaultsMaxConcurrentProcessingToHalfTheCPUs(t *testing.T) {
+func TestNewMakesMaxConcurrentProcessingSlots(t *testing.T) {
 	t.Parallel()
 
-	for _, limit := range []int{0, -1} {
-		proc := New(Params{MaxConcurrentProcessing: limit})
-		if got := cap(proc.processingSemaphore); got != max(runtime.GOMAXPROCS(0)/2, 1) {
-			t.Errorf("MaxConcurrentProcessing %d: %d slots, want %d, half the CPUs",
-				limit, got, max(runtime.GOMAXPROCS(0)/2, 1))
-		}
-	}
-
-	proc := New(Params{MaxConcurrentProcessing: 3})
+	proc := New(Params{MaxInputBytes: 50 << 20, MaxConcurrentProcessing: 3})
 	if got := cap(proc.processingSemaphore); got != 3 {
 		t.Errorf("MaxConcurrentProcessing 3: %d slots, want 3", got)
 	}
 }
 
-// TestNewDefaultIsHalfTheCPUsAndAtLeastOne checks that New, given no
-// MaxConcurrentProcessing, makes one slot for every two CPUs Go uses,
-// rounded down, and one slot when Go uses one CPU.
-//
-//nolint:paralleltest // it sets GOMAXPROCS, which other tests read
-func TestNewDefaultIsHalfTheCPUsAndAtLeastOne(t *testing.T) {
-	cpusBefore := runtime.GOMAXPROCS(0)
-
-	t.Cleanup(func() { runtime.GOMAXPROCS(cpusBefore) })
-
-	for _, tc := range []struct{ cpus, want int }{
-		{1, 1}, {2, 1}, {3, 1}, {8, 4},
-	} {
-		runtime.GOMAXPROCS(tc.cpus)
-
-		got := cap(New(Params{}).processingSemaphore)
-		t.Logf("GOMAXPROCS %d: %d slots", tc.cpus, got)
-
-		if got != tc.want {
-			t.Errorf("GOMAXPROCS %d: %d slots, want %d", tc.cpus, got, tc.want)
-		}
-	}
-}
-
 // TestProcessNeverExceedsMaxConcurrentProcessing starts more Process calls
 // than MaxConcurrentProcessing allows and holds the first ones inside
 // Process until the test lets them go. No more than the limit may be
@@ -171,7 +138,7 @@ func TestProcessNeverExceedsMaxConcurrentProcessing(t *testing.T) {
 		calls = 6
 	)
 
-	proc := New(Params{MaxConcurrentProcessing: limit})
+	proc := New(Params{MaxInputBytes: 50 << 20, MaxConcurrentProcessing: limit})
 	input := createTestJPEG(t, 50, 50)
 
 	counter := &readingCounter{}
@@ -218,7 +185,7 @@ func TestProcessNeverExceedsMaxConcurrentProcessing(t *testing.T) {
 func TestProcessWaitsThenFailsWhenNoSlotFrees(t *testing.T) {
 	t.Parallel()
 
-	proc := New(Params{MaxConcurrentProcessing: 1})
+	proc := New(Params{MaxInputBytes: 50 << 20, MaxConcurrentProcessing: 1})
 	proc.processingWaitTimeout = 100 * time.Millisecond
 
 	input := createTestJPEG(t, 10, 10)
@@ -334,7 +301,7 @@ func TestProcessReleasesSlotOnError(t *testing.T) {
 func TestWaitForProcessing(t *testing.T) {
 	t.Parallel()
 
-	proc := New(Params{MaxConcurrentProcessing: 2})
+	proc := New(Params{MaxInputBytes: 50 << 20, MaxConcurrentProcessing: 2})
 
 	gate := make(chan struct{})
 	entered := make(chan struct{}, 1)

internal/imageprocessor/orig_format_internal_test.go

@@ -27,7 +27,10 @@ func TestImageProcessor_OrigOfUnnamedFormatRefused(t *testing.T) {
 		t.Fatalf("failed to save test TIFF: %v", err)
 	}
 
-	result, err := New(Params{}).Process(
+	result, err := New(Params{
+		MaxInputBytes:           50 << 20,
+		MaxConcurrentProcessing: 1,
+	}).Process(
 		t.Context(), bytes.NewReader(tiff),
 		&imagerequest.Transform{Format: imagerequest.FormatOriginal},
 	)

internal/imageprocessor/source_formats_internal_test.go

@@ -14,7 +14,7 @@ import (
 func TestImageProcessor_LoadsEachSourceFormat(t *testing.T) {
 	t.Parallel()
 
-	proc := New(Params{})
+	proc := New(Params{MaxInputBytes: 50 << 20, MaxConcurrentProcessing: 1})
 
 	for _, format := range []imagerequest.Format{
 		imagerequest.FormatJPEG, imagerequest.FormatPNG, imagerequest.FormatGIF,

internal/imageproxy/cache_hit_internal_test.go

@@ -0,0 +1,100 @@
+package imageproxy
+
+import (
+	"bytes"
+	"os"
+	"testing"
+	"time"
+
+	"sneak.berlin/go/pixa/internal/imagerequest"
+	"sneak.berlin/go/pixa/internal/imgcache"
+)
+
+// TestProxy_Get_MakesAgainVariantOfNoOutputFormat replaces a cached
+// variant's file with text, and checks that the next request is a miss
+// answered with the variant made again, and the one after a hit.
+func TestProxy_Get_MakesAgainVariantOfNoOutputFormat(t *testing.T) {
+	t.Parallel()
+
+	svc, fixtures := setupTestProxy(t)
+	req := photoVariant(fixtures, 85, imagerequest.FitCover)
+
+	want, _ := getImage(t, svc, req)
+
+	path := variantFilePath(svc, imagerequest.CacheKey(req))
+
+	err := os.WriteFile(path, []byte("this is text, not an image"),
+		imgcache.StorageFilePerm)
+	if err != nil {
+		t.Fatalf("replacing the variant file: %v", err)
+	}
+
+	got, status := getImage(t, svc, req)
+	t.Logf("after the replacement: %d bytes, %s", len(got), status)
+
+	if status != CacheMiss || !bytes.Equal(got, want) {
+		t.Errorf("after the replacement: Get() = %d bytes, %s; want the %d bytes "+
+			"made again, %s", len(got), status, len(want), CacheMiss)
+	}
+
+	got, status = getImage(t, svc, req)
+
+	if status != CacheHit || !bytes.Equal(got, want) {
+		t.Errorf("next request: Get() = %d bytes, %s; want the %d bytes, %s",
+			len(got), status, len(want), CacheHit)
+	}
+}
+
+// TestProxy_Get_HitWaitsForNoDatabaseWrite holds the database's one
+// connection while a cached variant is requested. The hit must be answered
+// while the connection is still held, and its access time must reach the
+// database once it is free.
+func TestProxy_Get_HitWaitsForNoDatabaseWrite(t *testing.T) {
+	t.Parallel()
+
+	svc, fixtures := setupTestProxy(t)
+	req := photoVariant(fixtures, 85, imagerequest.FitCover)
+	key := imagerequest.CacheKey(req)
+
+	getImage(t, svc, req)
+
+	// Each connection to an in-memory SQLite database opens a new, empty one,
+	// so the goroutine that writes the access times must share the
+	// connection that holds the schema.
+	svc.db.SetMaxOpenConns(1)
+	svc.cache.StartPendingAccessTimeWrites()
+
+	defer func() { _ = svc.cache.StopPendingAccessTimeWrites(t.Context()) }()
+
+	anHourAgo := time.Now().Add(-time.Hour)
+	setVariantLastAccessed(t, svc.db, key, anHourAgo)
+
+	releaseDatabase := holdDatabase(t, svc.db)
+	defer releaseDatabase()
+
+	results := startGet(t.Context(), svc, req)
+
+	select {
+	case got := <-results:
+		if got.err != nil {
+			t.Fatalf("Get() of the cached variant error = %v", got.err)
+		}
+	case <-time.After(pendingAccessTimesWait):
+		t.Fatal("the cache hit waited for the database")
+	}
+
+	releaseDatabase()
+
+	deadline := time.Now().Add(pendingAccessTimesWait)
+
+	for countRows(t, svc.db, `
+		SELECT COUNT(*) FROM variant_content
+		WHERE cache_key = ? AND last_accessed_at > ?
+	`, string(key), anHourAgo.UTC().Format(sqliteTimestampFormat)) == 0 {
+		if time.Now().After(deadline) {
+			t.Fatal("the hit's access time was not written")
+		}
+
+		time.Sleep(10 * time.Millisecond)
+	}
+}

internal/imageproxy/concurrent_misses_internal_test.go

@@ -1,4 +1,4 @@
-package imgcache
+package imageproxy
 
 import (
 	"bytes"
@@ -36,7 +36,7 @@ type heldFetcher struct {
 }
 
 func (f *heldFetcher) Fetch(
-	ctx context.Context, url string,
+	ctx context.Context, source imagerequest.Source,
 ) (*httpfetcher.FetchResult, error) {
 	f.fetches.Add(1)
 
@@ -48,7 +48,7 @@ func (f *heldFetcher) Fetch(
 		return nil, ctx.Err()
 	}
 
-	return f.upstream.Fetch(ctx, url)
+	return f.upstream.Fetch(ctx, source)
 }
 
 // releaseFetches lets every held fetch, and every later one, go on.
@@ -56,15 +56,15 @@ func (f *heldFetcher) releaseFetches() {
 	f.releaseOnce.Do(func() { close(f.release) })
 }
 
-// setupHeldFetchService returns a test service whose fetches go through a
+// setupHeldFetchProxy returns a test proxy whose fetches go through a
 // heldFetcher. Its database is limited to one connection: each connection to
 // an in-memory SQLite database opens a new, empty one, so requests running at
 // once must share the connection that holds the schema.
-func setupHeldFetchService(t *testing.T) (*Service, *TestFixtures, *heldFetcher) {
+func setupHeldFetchProxy(t *testing.T) (*testProxy, *TestFixtures, *heldFetcher) {
 	t.Helper()
 
-	svc, fixtures := SetupTestService(t)
-	svc.cache.db.SetMaxOpenConns(1)
+	svc, fixtures := setupTestProxy(t)
+	svc.db.SetMaxOpenConns(1)
 
 	fetcher := &heldFetcher{
 		upstream: svc.fetcher,
@@ -79,8 +79,8 @@ func setupHeldFetchService(t *testing.T) (*Service, *TestFixtures, *heldFetcher)
 }
 
 // photoVariant asks for the test photo, 100x100, at 50x25 as a JPEG of the
-// given quality and fit mode. Each call returns a new request, as Get writes
-// to the request it is given.
+// given quality and fit mode. Each call returns a new request, which a test
+// may change.
 func photoVariant(
 	fixtures *TestFixtures, quality int, fit imagerequest.Fit,
 ) *imagerequest.Request {
@@ -107,7 +107,7 @@ type getResult struct {
 // startGet calls Get in a goroutine of its own and delivers what it returned
 // on the channel.
 func startGet(
-	ctx context.Context, svc *Service, req *imagerequest.Request,
+	ctx context.Context, svc *testProxy, req *imagerequest.Request,
 ) <-chan getResult {
 	results := make(chan getResult, 1)
 
@@ -139,14 +139,14 @@ func jpegSize(data []byte) (int, int) {
 	return config.Width, config.Height
 }
 
-// TestService_Get_ConcurrentMissesShareOneFetch starts several requests for
+// TestProxy_Get_ConcurrentMissesShareOneFetch starts several requests for
 // one uncached variant while the first one's fetch is held. Between them they
 // must fetch the source once and transcode it once, every one must be answered
 // with the same 50x25 JPEG, and each must count one miss.
-func TestService_Get_ConcurrentMissesShareOneFetch(t *testing.T) {
+func TestProxy_Get_ConcurrentMissesShareOneFetch(t *testing.T) {
 	t.Parallel()
 
-	svc, fixtures, fetcher := setupHeldFetchService(t)
+	svc, fixtures, fetcher := setupHeldFetchProxy(t)
 
 	const requests = 8
 
@@ -183,7 +183,7 @@ func TestService_Get_ConcurrentMissesShareOneFetch(t *testing.T) {
 		t.Errorf("%d requests made %d upstream fetches, want 1", requests, fetches)
 	}
 
-	// NewTestFS builds the same files the test service's fetcher serves.
+	// NewTestFS builds the same files the test proxy's fetcher serves.
 	testFS, _ := NewTestFS(t)
 
 	photo, err := fs.ReadFile(testFS, fixtures.GoodHostJPEG)
@@ -191,20 +191,20 @@ func TestService_Get_ConcurrentMissesShareOneFetch(t *testing.T) {
 		t.Fatal(err)
 	}
 
-	want := cacheStatsCounters{0, requests, 1, int64(len(photo)), 1}
+	want := counterValues{0, requests, 1, int64(len(photo)), 1}
 
-	if got := readCacheStatsCounters(t, svc.cache); got != want {
+	if got := readCounters(t, svc); got != want {
 		t.Errorf("counters = %+v, want %+v", got, want)
 	}
 }
 
-// TestService_Get_ConcurrentVariantsStayApart requests three variants of the
+// TestProxy_Get_ConcurrentVariantsStayApart requests three variants of the
 // test photo at once that differ only in quality or fit. Each must be made by
 // a fetch and a transcode of its own, and each answer must be its own variant.
-func TestService_Get_ConcurrentVariantsStayApart(t *testing.T) {
+func TestProxy_Get_ConcurrentVariantsStayApart(t *testing.T) {
 	t.Parallel()
 
-	svc, fixtures, fetcher := setupHeldFetchService(t)
+	svc, fixtures, fetcher := setupHeldFetchProxy(t)
 
 	cover := startGet(t.Context(), svc, photoVariant(fixtures, 85, imagerequest.FitCover))
 	lowQuality := startGet(t.Context(), svc,
@@ -253,14 +253,14 @@ func TestService_Get_ConcurrentVariantsStayApart(t *testing.T) {
 	}
 }
 
-// TestService_Get_WaiterStopsWhenItsContextEnds has a second request for a
+// TestProxy_Get_WaiterStopsWhenItsContextEnds has a second request for a
 // variant join the first one's held fetch, then ends the second request's
 // context. The second request must return at once with the context's error,
 // while the fetch is still held, and the first must still be answered.
-func TestService_Get_WaiterStopsWhenItsContextEnds(t *testing.T) {
+func TestProxy_Get_WaiterStopsWhenItsContextEnds(t *testing.T) {
 	t.Parallel()
 
-	svc, fixtures, fetcher := setupHeldFetchService(t)
+	svc, fixtures, fetcher := setupHeldFetchProxy(t)
 
 	first := startGet(t.Context(), svc, photoVariant(fixtures, 85, imagerequest.FitCover))
 
@@ -298,15 +298,15 @@ func TestService_Get_WaiterStopsWhenItsContextEnds(t *testing.T) {
 	}
 }
 
-// TestService_Get_FirstRequestLeavingKeepsTheWork ends the context of the
+// TestProxy_Get_FirstRequestLeavingKeepsTheWork ends the context of the
 // request whose fetch is held, after a second request has joined it. The fetch
 // and transcode must go on and answer the second request. The first request
 // waits for its own work, as every request did before misses were shared, and
 // is answered too.
-func TestService_Get_FirstRequestLeavingKeepsTheWork(t *testing.T) {
+func TestProxy_Get_FirstRequestLeavingKeepsTheWork(t *testing.T) {
 	t.Parallel()
 
-	svc, fixtures, fetcher := setupHeldFetchService(t)
+	svc, fixtures, fetcher := setupHeldFetchProxy(t)
 
 	firstCtx, cancelFirst := context.WithCancel(t.Context())
 	first := startGet(firstCtx, svc, photoVariant(fixtures, 85, imagerequest.FitCover))
@@ -337,11 +337,11 @@ func TestService_Get_FirstRequestLeavingKeepsTheWork(t *testing.T) {
 	}
 }
 
-// TestService_Get_ConcurrentMissesShareAFailure has several requests for an
+// TestProxy_Get_ConcurrentMissesShareAFailure has several requests for an
 // image that cannot be served arrive while its fetch is held: the one fetch
 // answers all of them with its error. The request after them is answered from
 // the negative cache, where the failure is kept.
-func TestService_Get_ConcurrentMissesShareAFailure(t *testing.T) {
+func TestProxy_Get_ConcurrentMissesShareAFailure(t *testing.T) {
 	t.Parallel()
 
 	tests := []struct {
@@ -362,7 +362,7 @@ func TestService_Get_ConcurrentMissesShareAFailure(t *testing.T) {
 		t.Run(tc.name, func(t *testing.T) {
 			t.Parallel()
 
-			svc, fixtures, fetcher := setupHeldFetchService(t)
+			svc, fixtures, fetcher := setupHeldFetchProxy(t)
 
 			request := func() *imagerequest.Request {
 				req := photoVariant(fixtures, 85, imagerequest.FitCover)
@@ -398,12 +398,12 @@ func TestService_Get_ConcurrentMissesShareAFailure(t *testing.T) {
 	}
 }
 
-// TestService_Get_EndedRequestFetchesNothing checks that a request whose
+// TestProxy_Get_EndedRequestFetchesNothing checks that a request whose
 // context has already ended when it misses the cache starts no fetch.
-func TestService_Get_EndedRequestFetchesNothing(t *testing.T) {
+func TestProxy_Get_EndedRequestFetchesNothing(t *testing.T) {
 	t.Parallel()
 
-	svc, fixtures, fetcher := setupHeldFetchService(t)
+	svc, fixtures, fetcher := setupHeldFetchProxy(t)
 
 	ctx, cancel := context.WithCancel(t.Context())
 	cancel()
@@ -418,14 +418,14 @@ func TestService_Get_EndedRequestFetchesNothing(t *testing.T) {
 	}
 }
 
-// TestService_Get_ReturnsByItsDeadline gives a request a deadline and holds
+// TestProxy_Get_ReturnsByItsDeadline gives a request a deadline and holds
 // its fetch until the fetch's context ends, as the fetcher does while it waits
 // for a free connection to the host. The request must return by its deadline
 // with the deadline's error.
-func TestService_Get_ReturnsByItsDeadline(t *testing.T) {
+func TestProxy_Get_ReturnsByItsDeadline(t *testing.T) {
 	t.Parallel()
 
-	svc, fixtures, _ := setupHeldFetchService(t)
+	svc, fixtures, _ := setupHeldFetchProxy(t)
 
 	const timeout = 200 * time.Millisecond
 
@@ -444,18 +444,18 @@ func TestService_Get_ReturnsByItsDeadline(t *testing.T) {
 	}
 }
 
-// TestService_Get_WaiterGetsProcessingTimedOut has a request with no deadline
+// TestProxy_Get_WaiterGetsProcessingTimedOut has a request with no deadline
 // join the held fetch of a first request with a short one. With the only
 // processing slot held, the first request's transcode waits for it until the
 // first request's deadline ends, and stops then. The waiting request, whose
 // own context has not ended, must get ErrProcessingTimedOut, which the image
 // routes answer 503 as when pixa is busy.
-func TestService_Get_WaiterGetsProcessingTimedOut(t *testing.T) {
+func TestProxy_Get_WaiterGetsProcessingTimedOut(t *testing.T) {
 	t.Parallel()
 
-	svc, fixtures, fetcher := setupHeldFetchService(t)
+	svc, fixtures, fetcher := setupHeldFetchProxy(t)
 	svc.processor = imageprocessor.New(
-		imageprocessor.Params{MaxConcurrentProcessing: 1},
+		imageprocessor.Params{MaxInputBytes: 50 << 20, MaxConcurrentProcessing: 1},
 	)
 
 	holdProcessingSlot(t, svc.processor)
@@ -489,18 +489,18 @@ func TestService_Get_WaiterGetsProcessingTimedOut(t *testing.T) {
 type panickingFetcher struct{}
 
 func (panickingFetcher) Fetch(
-	context.Context, string,
+	context.Context, imagerequest.Source,
 ) (*httpfetcher.FetchResult, error) {
 	panic("upstream fetcher panicked")
 }
 
-// TestService_Get_PanicBecomesAnError checks that a panic while a variant is
+// TestProxy_Get_PanicBecomesAnError checks that a panic while a variant is
 // being made reaches its request as an error naming the panic, instead of
 // being raised again.
-func TestService_Get_PanicBecomesAnError(t *testing.T) {
+func TestProxy_Get_PanicBecomesAnError(t *testing.T) {
 	t.Parallel()
 
-	svc, fixtures := SetupTestService(t)
+	svc, fixtures := setupTestProxy(t)
 	svc.fetcher = panickingFetcher{}
 
 	var err error
@@ -520,13 +520,13 @@ func TestService_Get_PanicBecomesAnError(t *testing.T) {
 	}
 }
 
-// TestService_Get_PanicIsReportedToSentry checks that a panic while a variant
+// TestProxy_Get_PanicIsReportedToSentry checks that a panic while a variant
 // is being made is reported through the Sentry hub on the request's context,
 // where the Sentry middleware puts one when sentry_dsn is set.
-func TestService_Get_PanicIsReportedToSentry(t *testing.T) {
+func TestProxy_Get_PanicIsReportedToSentry(t *testing.T) {
 	t.Parallel()
 
-	svc, fixtures := SetupTestService(t)
+	svc, fixtures := setupTestProxy(t)
 	svc.fetcher = panickingFetcher{}
 
 	transport := &sentry.MockTransport{}

internal/imageproxy/divzero_internal_test.go

@@ -1,4 +1,4 @@
-package imgcache
+package imageproxy
 
 import (
 	"fmt"

internal/imageproxy/imageproxy.go

@@ -1,4 +1,7 @@
-package imgcache
+// Package imageproxy fetches, transforms, caches and serves images. Its
+// Proxy serves an image from the cache when it can, and otherwise makes it
+// from the cached or fetched source image.
+package imageproxy
 
 import (
 	"bytes"
@@ -7,101 +10,61 @@ import (
 	"fmt"
 	"io"
 	"log/slog"
-	"net/url"
 	"runtime/debug"
-	"strings"
 	"time"
 
 	"github.com/dustin/go-humanize"
 	"github.com/getsentry/sentry-go"
 	"github.com/go-chi/chi/v5/middleware"
 	"golang.org/x/sync/singleflight"
-	"sneak.berlin/go/pixa/internal/allowlist"
 	"sneak.berlin/go/pixa/internal/httpfetcher"
 	"sneak.berlin/go/pixa/internal/imageprocessor"
 	"sneak.berlin/go/pixa/internal/imagerequest"
+	"sneak.berlin/go/pixa/internal/imgcache"
 	"sneak.berlin/go/pixa/internal/magic"
-	"sneak.berlin/go/pixa/internal/signature"
 )
 
-// Service implements the ImageCache interface, orchestrating cache,
-// fetcher, and processor.
-type Service struct {
-	cache           *Cache
+// Proxy orchestrates the cache, fetcher, and processor.
+type Proxy struct {
+	cache           *imgcache.Cache
 	fetcher         httpfetcher.Fetcher
 	processor       *imageprocessor.ImageProcessor
-	signer          *signature.Signer
-	allowlist       *allowlist.HostAllowList
 	log             *slog.Logger
-	allowHTTP       bool
 	maxResponseSize int64
 	// variantsInProgress lets the requests that miss the same variant at the
 	// same time share one fetch and one transcode.
 	variantsInProgress singleflight.Group
 }
 
-// ServiceConfig holds configuration for the image service.
-type ServiceConfig struct {
+// Config holds the proxy's configuration.
+type Config struct {
 	// Cache is the cache instance
-	Cache *Cache
-	// FetcherConfig configures the upstream fetcher built when Fetcher is
-	// not set. Its AllowHTTP and MaxResponseSize are used either way.
-	FetcherConfig *httpfetcher.Config
-	// Fetcher is an optional custom fetcher (for testing)
+	Cache *imgcache.Cache
+	// Fetcher fetches source images from upstream hosts
 	Fetcher httpfetcher.Fetcher
-	// SigningKey is the HMAC signing key (empty disables signing)
-	SigningKey string
-	// Allowlist is the list of hosts that don't require signatures
-	Allowlist []string
-	// MaxConcurrentProcessing is the most images processed at once; zero
-	// uses the image processor's default, half the CPUs
-	MaxConcurrentProcessing int
+	// Processor converts and resizes the images
+	Processor *imageprocessor.ImageProcessor
+	// MaxResponseSize is the largest source accepted, in bytes; a cached
+	// source larger than this is fetched again
+	MaxResponseSize int64
 	// Logger for logging
 	Logger *slog.Logger
 }
 
-// Static errors for service construction.
-var (
-	errCacheRequired      = errors.New("cache is required")
-	errSigningKeyRequired = errors.New("signing key is required")
-)
+// errCacheRequired is returned when the proxy is built without a cache.
+var errCacheRequired = errors.New("cache is required")
 
-// NewService creates a new image service.
-func NewService(cfg *ServiceConfig) (*Service, error) {
+// NewProxy creates a new image proxy.
+func NewProxy(cfg *Config) (*Proxy, error) {
 	if cfg.Cache == nil {
 		return nil, errCacheRequired
 	}
 
-	if cfg.SigningKey == "" {
-		return nil, errSigningKeyRequired
-	}
-
-	// Resolve fetcher config for defaults
-	fetcherCfg := cfg.FetcherConfig
-	if fetcherCfg == nil {
-		fetcherCfg = httpfetcher.DefaultConfig()
-	}
-
-	// Use custom fetcher if provided, otherwise create HTTP fetcher
-	var fetcher httpfetcher.Fetcher
-	if cfg.Fetcher != nil {
-		fetcher = cfg.Fetcher
-	} else {
-		fetcher = httpfetcher.New(fetcherCfg)
-	}
-
-	signer := signature.New(cfg.SigningKey)
-
 	log := cfg.Logger
 	if log == nil {
 		log = slog.Default()
 	}
 
-	allowHTTP := false
-	if cfg.FetcherConfig != nil {
-		allowHTTP = cfg.FetcherConfig.AllowHTTP
-	}
-
 	// JPEG XL is the default output format, so pixad does not start
 	// without it.
 	err := imageprocessor.CheckJPEGXLSupport()
@@ -109,21 +72,12 @@ func NewService(cfg *ServiceConfig) (*Service, error) {
 		return nil, err
 	}
 
-	maxResponseSize := fetcherCfg.MaxResponseSize
-	processor := imageprocessor.New(imageprocessor.Params{
-		MaxInputBytes:           maxResponseSize,
-		MaxConcurrentProcessing: cfg.MaxConcurrentProcessing,
-	})
-
-	return &Service{
+	return &Proxy{
 		cache:           cfg.Cache,
-		fetcher:         fetcher,
-		processor:       processor,
-		signer:          signer,
-		allowlist:       allowlist.New(cfg.Allowlist),
+		fetcher:         cfg.Fetcher,
+		processor:       cfg.Processor,
 		log:             log,
-		allowHTTP:       allowHTTP,
-		maxResponseSize: maxResponseSize,
+		maxResponseSize: cfg.MaxResponseSize,
 	}, nil
 }
 
@@ -136,48 +90,70 @@ var ErrNegativeCached = errors.New("request is in negative cache (recently faile
 var ErrProcessingTimedOut = errors.New(
 	"image processing stopped at the deadline of the request doing it")
 
+// ImageResponse represents a processed image ready to serve
+type ImageResponse struct {
+	// Content is the image data reader
+	Content io.ReadCloser
+	// ContentLength is the size in bytes (-1 if unknown)
+	ContentLength int64
+	// ContentType is the MIME type of the response
+	ContentType string
+	// ETag is the entity tag for caching
+	ETag string
+	// CacheStatus indicates HIT or MISS
+	CacheStatus CacheStatus
+	// FetchedBytes is the number of bytes fetched from upstream (0 if cache hit)
+	FetchedBytes int64
+}
+
+// CacheStatus indicates whether the response was served from cache.
+type CacheStatus string
+
+// Cache status values for response headers.
+const (
+	CacheHit  CacheStatus = "HIT"
+	CacheMiss CacheStatus = "MISS"
+)
+
 // Get retrieves a processed image, fetching and processing if necessary.
 // While its source URL is in the negative cache, a cached variant is still
 // served and a variant not in the cache is still made from a cached source;
 // only a request that needs the source fetched is refused.
-func (s *Service) Get(
+//
+// A cache hit opens the variant file and makes no database call: the cache
+// hands its access time to the goroutine that writes access times, without
+// waiting for it, and its hit is counted in memory. A variant file that
+// cannot be served, such as one whose first bytes are not those of an output
+// format, is a miss, and the variant is made again.
+func (p *Proxy) Get(
 	ctx context.Context, req *imagerequest.Request,
 ) (*ImageResponse, error) {
-	// Propagate AllowHTTP setting to the request
-	req.AllowHTTP = s.allowHTTP
+	cacheKey := imagerequest.CacheKey(req)
 
-	// Check variant cache first (disk only, no DB)
-	result, err := s.cache.Lookup(ctx, req)
-	if err != nil {
-		s.log.Warn("cache lookup failed", "error", err)
+	reader, size, contentType, err := p.cache.Variant(cacheKey)
+	if err == nil {
+		p.cache.IncrementStats(true, 0)
+
+		return &ImageResponse{
+			Content:       reader,
+			ContentLength: size,
+			ContentType:   contentType,
+			CacheStatus:   CacheHit,
+			ETag:          formatETag(cacheKey),
+		}, nil
 	}
 
-	// Cache hit - serve directly from disk
-	if result != nil && result.Hit {
-		reader, size, contentType, err := s.cache.GetVariant(result.CacheKey)
-		if err != nil {
-			s.log.Error("failed to get cached variant", "key", result.CacheKey, "error", err)
-			// Fall through to re-process
-		} else {
-			// Counted also when the request context has ended meanwhile
-			s.cache.IncrementStats(ctx, true, 0)
-
-			return &ImageResponse{
-				Content:       reader,
-				ContentLength: size,
-				ContentType:   contentType,
-				CacheStatus:   CacheHit,
-				ETag:          formatETag(result.CacheKey),
-			}, nil
-		}
+	if !errors.Is(err, imgcache.ErrNotFound) {
+		p.log.Warn("cached variant not served, making it again",
+			"cache_key", cacheKey, "error", err)
 	}
 
 	// Cache miss - get the variant, processed once for all the requests that
 	// miss it at the same time, then count this request's miss, also when it
 	// failed or the request context has ended meanwhile
-	response, err := s.processOrWait(ctx, req)
+	response, err := p.processOrWait(ctx, req)
 
-	s.cache.IncrementStats(ctx, false, 0)
+	p.cache.IncrementStats(false, 0)
 
 	if err != nil {
 		return nil, err
@@ -188,73 +164,10 @@ func (s *Service) Get(
 	return response, nil
 }
 
-// Warm pre-fetches and caches an image without returning it.
-func (s *Service) Warm(ctx context.Context, req *imagerequest.Request) error {
-	_, err := s.Get(ctx, req)
-
-	return err
-}
-
-// Stats returns cache statistics.
-func (s *Service) Stats(ctx context.Context) (*CacheStats, error) {
-	return s.cache.Stats(ctx)
-}
-
 // WaitForProcessing waits until no image is being processed, or until ctx
 // ends, and returns how many images were still being processed then.
-func (s *Service) WaitForProcessing(ctx context.Context) int {
-	return s.processor.WaitForProcessing(ctx)
-}
-
-// ValidateRequest validates the request signature if required.
-func (s *Service) ValidateRequest(req *imagerequest.Request) error {
-	// Check if host is allowed (no signature required)
-	sourceURL := req.SourceURL()
-
-	parsedURL, err := url.Parse(sourceURL)
-	if err != nil {
-		return fmt.Errorf("invalid source URL: %w", err)
-	}
-
-	// An allowlist entry names no port, so a host followed by ":", with or
-	// without a port, needs a signature. The port is read from the parsed
-	// URL, as the fetcher reads it: in an IPv6 address written without
-	// brackets, such as 2001:db8::1:8443, it is what follows the last ":".
-	// This is checked here and not in allowlist.IsAllowed, which also matches
-	// referer_blocklist, where a Referer must be blocked whatever its port.
-	hostHasPort := parsedURL.Port() != "" ||
-		strings.HasSuffix(parsedURL.Host, ":")
-
-	if !hostHasPort && s.allowlist.IsAllowed(parsedURL) {
-		return nil
-	}
-
-	// Signature required for non-allowed hosts
-	return s.signer.Verify(req)
-}
-
-// GenerateSignedURL generates a signed URL for the given request. The URL
-// carries q and fit next to sig and exp, so the image route verifies it for
-// the quality and fit it was signed with. An unset quality or fit is first
-// set to 85 or cover, the values the route uses when a URL has no q or fit.
-func (s *Service) GenerateSignedURL(
-	baseURL string,
-	req *imagerequest.Request,
-	ttl time.Duration,
-) (string, error) {
-	if req.Transform.Quality == 0 {
-		req.Transform.Quality = 85
-	}
-
-	if req.Transform.FitMode == "" {
-		req.Transform.FitMode = imagerequest.FitCover
-	}
-
-	// Sets the request's Expires and Signature
-	path, sig, exp := s.signer.GenerateSignedURL(req, ttl)
-
-	return fmt.Sprintf("%s%s?sig=%s&exp=%d&q=%d&fit=%s",
-		baseURL, path, sig, exp, req.Transform.Quality, req.Transform.FitMode), nil
+func (p *Proxy) WaitForProcessing(ctx context.Context) int {
+	return p.processor.WaitForProcessing(ctx)
 }
 
 // errPanicked is returned when processing a variant panicked.
@@ -266,7 +179,7 @@ var errPanicked = errors.New("panic while processing image")
 // and take no upstream connection or processing slot. The processing ignores
 // the first request's cancellation, so the others are still served if that
 // client goes away, but keeps its deadline.
-func (s *Service) processOrWait(
+func (p *Proxy) processOrWait(
 	ctx context.Context, req *imagerequest.Request,
 ) (*ImageResponse, error) {
 	// A request that has already ended starts no processing
@@ -280,7 +193,7 @@ func (s *Service) processOrWait(
 	// only when no other request is processing the variant
 	processing := make(chan struct{})
 
-	results := s.variantsInProgress.DoChan(string(cacheKey),
+	results := p.variantsInProgress.DoChan(string(cacheKey),
 		func() (_ any, err error) {
 			close(processing)
 
@@ -291,7 +204,7 @@ func (s *Service) processOrWait(
 			defer func() {
 				recovered := recover()
 				if recovered != nil {
-					s.log.Error("panic while processing image",
+					p.log.Error("panic while processing image",
 						"host", req.Source.Host, "path", req.Source.Path,
 						"panic", recovered, "stack", string(debug.Stack()))
 
@@ -306,7 +219,7 @@ func (s *Service) processOrWait(
 			processingCtx, cancel := withoutCancelKeepingDeadline(ctx)
 			defer cancel()
 
-			return s.processFromSourceOrFetch(processingCtx, req, cacheKey)
+			return p.processFromSourceOrFetch(processingCtx, req, cacheKey)
 		})
 
 	var result singleflight.Result
@@ -355,25 +268,31 @@ func withoutCancelKeepingDeadline(
 	return context.WithDeadline(context.WithoutCancel(ctx), deadline)
 }
 
-// loadCachedSource opens source content from cache, without reading it, and
-// returns it with its size; nil if the cached data is unavailable, empty or
-// exceeds maxResponseSize.
-func (s *Service) loadCachedSource(
-	sourceKey imagerequest.SourceKey,
+// loadCachedSource opens src's cached source, without reading it, and
+// returns it with its size; nil if it is not in the cache, cannot be read,
+// is empty or exceeds maxResponseSize.
+func (p *Proxy) loadCachedSource(
+	ctx context.Context, src imagerequest.Source,
 ) (io.ReadCloser, int64) {
-	reader, size, err := s.cache.GetSourceContent(sourceKey)
+	reader, size, err := p.cache.Source(ctx, src)
+	if errors.Is(err, imgcache.ErrNotFound) {
+		return nil, 0
+	}
+
 	if err != nil {
-		s.log.Warn("failed to load cached source, fetching", "error", err)
+		p.log.Warn("failed to load cached source, fetching", "error", err)
 
 		return nil, 0
 	}
 
-	if size > s.maxResponseSize {
+	p.log.Debug("using cached source", "source_key", src.Key())
+
+	if size > p.maxResponseSize {
 		_ = reader.Close()
 
-		s.log.Warn("cached source exceeds max response size, discarding",
-			"source_key", sourceKey,
-			"max_bytes", s.maxResponseSize,
+		p.log.Warn("cached source exceeds max response size, discarding",
+			"source_key", src.Key(),
+			"max_bytes", p.maxResponseSize,
 		)
 
 		return nil, 0
@@ -390,30 +309,16 @@ func (s *Service) loadCachedSource(
 
 // processFromSourceOrFetch processes an image, using cached source content
 // if available.
-func (s *Service) processFromSourceOrFetch(
+func (p *Proxy) processFromSourceOrFetch(
 	ctx context.Context,
 	req *imagerequest.Request,
 	cacheKey imagerequest.VariantKey,
 ) (*processedVariant, error) {
-	// Check if we have cached source content
-	sourceKey, _, err := s.cache.LookupSource(ctx, req)
-	if err != nil {
-		s.log.Warn("source lookup failed", "error", err)
-	}
-
-	var (
-		source     io.ReadCloser
-		sourceSize int64
-	)
-
-	if sourceKey != "" {
-		s.log.Debug("using cached source", "source_key", sourceKey)
-		source, sourceSize = s.loadCachedSource(sourceKey)
-	}
+	source, sourceSize := p.loadCachedSource(ctx, req.Source)
 
 	// Fetch from upstream if we don't have source data or it's empty
 	if source == nil {
-		return s.fetchAndProcess(ctx, req, cacheKey)
+		return p.fetchAndProcess(ctx, req, cacheKey)
 	}
 
 	defer func() { _ = source.Close() }()
@@ -421,26 +326,26 @@ func (s *Service) processFromSourceOrFetch(
 	// Process using cached source; nothing was fetched from upstream. The
 	// image processor reads the source only once it has a processing slot,
 	// so a request waiting for one holds none of it in memory.
-	return s.processAndStore(ctx, req, cacheKey, source, sourceSize)
+	return p.processAndStore(ctx, req, cacheKey, source, sourceSize)
 }
 
 // fetchAndProcess fetches from upstream, processes, and caches the result.
 // It counts the fetch with the bytes read from upstream, including when
 // reading the response or a later step fails. It fetches nothing while the
 // source URL is in the negative cache.
-func (s *Service) fetchAndProcess(
+func (p *Proxy) fetchAndProcess(
 	ctx context.Context,
 	req *imagerequest.Request,
 	cacheKey imagerequest.VariantKey,
 ) (*processedVariant, error) {
 	// Check negative cache - skip fetching for recently-failed URLs
-	negHit, err := s.cache.checkNegativeCache(ctx, req)
+	negHit, err := p.cache.Failure(ctx, req.Source)
 	if err != nil {
-		s.log.Warn("negative cache check failed", "error", err)
+		p.log.Warn("negative cache check failed", "error", err)
 	}
 
 	if negHit {
-		s.log.Debug("negative cache hit",
+		p.log.Debug("negative cache hit",
 			"host", req.Source.Host,
 			"path", req.Source.Path,
 		)
@@ -449,15 +354,14 @@ func (s *Service) fetchAndProcess(
 	}
 
 	// Fetch from upstream
-	sourceURL := req.SourceURL()
+	p.log.Debug("fetching from upstream", "host", req.Source.Host,
+		"path", req.Source.Path, "query", req.Source.Query)
 
-	s.log.Debug("fetching from upstream", "url", sourceURL)
-
-	fetchResult, err := s.fetcher.Fetch(ctx, sourceURL)
+	fetchResult, err := p.fetcher.Fetch(ctx, req.Source)
 	if err != nil {
 		// Store negative cache for certain errors
 		if isNegativeCacheable(err) {
-			s.storeNegative(ctx, req, extractStatusCode(err), err)
+			p.storeNegative(ctx, req)
 		}
 
 		return nil, fmt.Errorf("upstream fetch failed: %w", err)
@@ -474,7 +378,7 @@ func (s *Service) fetchAndProcess(
 	fetchBytes := int64(len(sourceData))
 
 	// Counted also when the request context has ended meanwhile
-	s.cache.IncrementUpstreamFetch(ctx, fetchBytes)
+	p.cache.IncrementUpstreamFetch(fetchBytes)
 
 	if err != nil {
 		return nil, fmt.Errorf("failed to read upstream response: %w", err)
@@ -490,7 +394,7 @@ func (s *Service) fetchAndProcess(
 	}
 
 	// Log upstream fetch details
-	s.log.Info("upstream fetched",
+	p.log.Info("upstream fetched",
 		"request_id", middleware.GetReqID(ctx),
 		"host", req.Source.Host,
 		"path", req.Source.Path,
@@ -508,36 +412,34 @@ func (s *Service) fetchAndProcess(
 	// error status does.
 	err = magic.ValidateMagicBytes(sourceData, fetchResult.ContentType)
 	if err != nil {
-		s.log.Warn("source refused", "request_id", middleware.GetReqID(ctx),
+		p.log.Warn("source refused", "request_id", middleware.GetReqID(ctx),
 			"host", req.Source.Host, "path", req.Source.Path,
 			"content_type", fetchResult.ContentType, "error", err)
-		s.storeNegative(ctx, req, httpStatusUnsupportedMediaType, err)
+		p.storeNegative(ctx, req)
 
 		return nil, fmt.Errorf("content validation failed: %w", err)
 	}
 
 	// Store source content
-	_, err = s.cache.StoreSource(ctx, req, bytes.NewReader(sourceData), fetchResult)
+	_, err = p.cache.PutSource(ctx, req.Source, bytes.NewReader(sourceData))
 	if err != nil {
-		s.log.Warn("failed to store source content", "error", err)
+		p.log.Warn("failed to store source content", "error", err)
 		// Continue even if caching fails
 	}
 
-	return s.processAndStore(
+	return p.processAndStore(
 		ctx, req, cacheKey, bytes.NewReader(sourceData), fetchBytes,
 	)
 }
 
 // storeNegative stores a negative cache entry for req's source, whose fetch
-// failed with err, so the source is not fetched again until the entry
-// expires. A failure to store the entry is only logged.
-func (s *Service) storeNegative(
-	ctx context.Context, req *imagerequest.Request, statusCode int, err error,
-) {
-	storeErr := s.cache.StoreNegative(ctx, req, statusCode, err.Error())
-	if storeErr != nil {
-		s.log.Warn("failed to store negative cache entry",
-			"host", req.Source.Host, "path", req.Source.Path, "error", storeErr)
+// failed, so the source is not fetched again until the entry expires. A
+// failure to store the entry is only logged.
+func (p *Proxy) storeNegative(ctx context.Context, req *imagerequest.Request) {
+	err := p.cache.PutFailure(ctx, req.Source)
+	if err != nil {
+		p.log.Warn("failed to store negative cache entry",
+			"host", req.Source.Host, "path", req.Source.Path, "error", err)
 	}
 }
 
@@ -551,7 +453,7 @@ type processedVariant struct {
 
 // processAndStore processes the image read from source and stores the
 // result.
-func (s *Service) processAndStore(
+func (p *Proxy) processAndStore(
 	ctx context.Context,
 	req *imagerequest.Request,
 	cacheKey imagerequest.VariantKey,
@@ -563,7 +465,7 @@ func (s *Service) processAndStore(
 
 	// ctx has the deadline of the request doing the processing, which can
 	// end while a request waiting for its image has time left
-	processResult, err := s.processor.Process(ctx, source, &req.Transform)
+	processResult, err := p.processor.Process(ctx, source, &req.Transform)
 	if errors.Is(err, context.DeadlineExceeded) {
 		return nil, fmt.Errorf("%w: %w", ErrProcessingTimedOut, err)
 	}
@@ -575,7 +477,7 @@ func (s *Service) processAndStore(
 	processDuration := time.Since(processStart)
 
 	// Counted also when the request context has ended meanwhile
-	s.cache.IncrementTransformCount(ctx)
+	p.cache.IncrementTransformCount()
 
 	// Read processed content
 	processedData, err := io.ReadAll(processResult.Content)
@@ -594,7 +496,7 @@ func (s *Service) processAndStore(
 		sizePercent = float64(outputSize) / float64(fetchBytes) * 100.0
 	}
 
-	s.log.Info("image converted",
+	p.log.Info("image converted",
 		"request_id", middleware.GetReqID(ctx),
 		"host", req.Source.Host,
 		"path", req.Source.Path,
@@ -613,11 +515,9 @@ func (s *Service) processAndStore(
 	)
 
 	// Store variant to cache
-	err = s.cache.StoreVariant(
-		ctx, cacheKey, bytes.NewReader(processedData), processResult.ContentType,
-	)
+	err = p.cache.PutVariant(ctx, cacheKey, bytes.NewReader(processedData))
 	if err != nil {
-		s.log.Warn("failed to store variant", "error", err)
+		p.log.Warn("failed to store variant", "error", err)
 		// Continue even if caching fails
 	}
 
@@ -628,28 +528,11 @@ func (s *Service) processAndStore(
 	}, nil
 }
 
-// HTTP status codes for error responses.
-const (
-	httpStatusBadGateway           = 502
-	httpStatusInternalError        = 500
-	httpStatusUnsupportedMediaType = 415
-)
-
 // isNegativeCacheable returns true if the error should be cached.
 func isNegativeCacheable(err error) bool {
 	return errors.Is(err, httpfetcher.ErrUpstreamError)
 }
 
-// extractStatusCode extracts HTTP status code from error message.
-func extractStatusCode(err error) int {
-	// Default to 502 Bad Gateway for upstream errors
-	if errors.Is(err, httpfetcher.ErrUpstreamError) {
-		return httpStatusBadGateway
-	}
-
-	return httpStatusInternalError
-}
-
 // etagHashLength is the number of hash characters to use for ETags.
 const etagHashLength = 16
 

internal/imageproxy/imageproxy_internal_test.go

@@ -1,8 +1,7 @@
-package imgcache
+package imageproxy
 
 import (
 	"context"
-	"encoding/json"
 	"io"
 	"os"
 	"testing"
@@ -10,20 +9,18 @@ import (
 
 	"sneak.berlin/go/pixa/internal/imagerequest"
 	"sneak.berlin/go/pixa/internal/magic"
-	"sneak.berlin/go/pixa/internal/signature"
 )
 
 // Test data literals used repeatedly in this file (goconst).
 const (
 	testPathPhoto  = "/images/photo.jpg"
 	testPathUpload = "/uploads/image.jpg"
-	testSigningKey = "test-signing-key-12345"
 )
 
-func TestService_Get_AllowlistedHost(t *testing.T) {
+func TestProxy_Get_GoodHost(t *testing.T) {
 	t.Parallel()
 
-	svc, fixtures := SetupTestService(t)
+	svc, fixtures := setupTestProxy(t)
 	ctx := context.Background()
 
 	req := &imagerequest.Request{
@@ -61,36 +58,12 @@ func TestService_Get_AllowlistedHost(t *testing.T) {
 	}
 }
 
-func TestService_Get_NonAllowlistedHost_NoSignature(t *testing.T) {
+// TestProxy_Get_OtherHost checks that the proxy serves a second host as it
+// serves GoodHost.
+func TestProxy_Get_OtherHost(t *testing.T) {
 	t.Parallel()
 
-	svc, fixtures := SetupTestService(t, WithSigningKey("test-key"))
-
-	req := &imagerequest.Request{
-		Source: imagerequest.Source{
-			Host: fixtures.OtherHost,
-			Path: testPathUpload,
-		},
-		Transform: imagerequest.Transform{
-			Size:    imagerequest.Size{Width: 50, Height: 50},
-			Format:  imagerequest.FormatJPEG,
-			Quality: 85,
-			FitMode: imagerequest.FitCover,
-		},
-	}
-
-	// Should fail validation - not allowlisted and no signature
-	err := svc.ValidateRequest(req)
-	if err == nil {
-		t.Error("ValidateRequest() expected error for non-allowlisted host without signature")
-	}
-}
-
-func TestService_Get_NonAllowlistedHost_ValidSignature(t *testing.T) {
-	t.Parallel()
-
-	signingKey := testSigningKey
-	svc, fixtures := SetupTestService(t, WithSigningKey(signingKey))
+	svc, fixtures := setupTestProxy(t)
 	ctx := context.Background()
 
 	req := &imagerequest.Request{
@@ -106,17 +79,6 @@ func TestService_Get_NonAllowlistedHost_ValidSignature(t *testing.T) {
 		},
 	}
 
-	// Generate a valid signature
-	signer := signature.New(signingKey)
-	req.Expires = time.Now().Add(time.Hour)
-	req.Signature = signer.Sign(req)
-
-	// Should pass validation
-	err := svc.ValidateRequest(req)
-	if err != nil {
-		t.Errorf("ValidateRequest() error = %v", err)
-	}
-
 	// Should fetch successfully
 	resp, err := svc.Get(ctx, req)
 	if err != nil {
@@ -135,248 +97,10 @@ func TestService_Get_NonAllowlistedHost_ValidSignature(t *testing.T) {
 	}
 }
 
-func TestService_Get_NonAllowlistedHost_ExpiredSignature(t *testing.T) {
+func TestProxy_Get_InvalidFile(t *testing.T) {
 	t.Parallel()
 
-	signingKey := testSigningKey
-	svc, fixtures := SetupTestService(t, WithSigningKey(signingKey))
-
-	req := &imagerequest.Request{
-		Source: imagerequest.Source{
-			Host: fixtures.OtherHost,
-			Path: testPathUpload,
-		},
-		Transform: imagerequest.Transform{
-			Size:    imagerequest.Size{Width: 50, Height: 50},
-			Format:  imagerequest.FormatJPEG,
-			Quality: 85,
-			FitMode: imagerequest.FitCover,
-		},
-	}
-
-	// Generate an expired signature
-	signer := signature.New(signingKey)
-	req.Expires = time.Now().Add(-time.Hour) // Already expired
-	req.Signature = signer.Sign(req)
-
-	// Should fail validation
-	err := svc.ValidateRequest(req)
-	if err == nil {
-		t.Error("ValidateRequest() expected error for expired signature")
-	}
-}
-
-func TestService_Get_NonAllowlistedHost_InvalidSignature(t *testing.T) {
-	t.Parallel()
-
-	signingKey := testSigningKey
-	svc, fixtures := SetupTestService(t, WithSigningKey(signingKey))
-
-	req := &imagerequest.Request{
-		Source: imagerequest.Source{
-			Host: fixtures.OtherHost,
-			Path: testPathUpload,
-		},
-		Transform: imagerequest.Transform{
-			Size:    imagerequest.Size{Width: 50, Height: 50},
-			Format:  imagerequest.FormatJPEG,
-			Quality: 85,
-			FitMode: imagerequest.FitCover,
-		},
-	}
-
-	// Set an invalid signature
-	req.Signature = "invalid-signature"
-	req.Expires = time.Now().Add(time.Hour)
-
-	// Should fail validation
-	err := svc.ValidateRequest(req)
-	if err == nil {
-		t.Error("ValidateRequest() expected error for invalid signature")
-	}
-}
-
-// TestService_ValidateRequest_SignatureExactHostMatch verifies that
-// ValidateRequest enforces exact host matching for signatures. A
-// signature for one host must not verify for a different host, even
-// if they share a domain suffix.
-func TestService_ValidateRequest_SignatureExactHostMatch(t *testing.T) {
-	t.Parallel()
-
-	signingKey := "test-signing-key-must-be-32-chars"
-	svc, _ := SetupTestService(t,
-		WithSigningKey(signingKey),
-		WithNoAllowlist(),
-	)
-
-	signer := signature.New(signingKey)
-
-	// Sign a request for "cdn.example.com"
-	signedReq := &imagerequest.Request{
-		Source: imagerequest.Source{
-			Host: testHostCDN,
-			Path: testPathCat,
-		},
-		Transform: imagerequest.Transform{
-			Size:    imagerequest.Size{Width: 50, Height: 50},
-			Format:  imagerequest.FormatJPEG,
-			Quality: 85,
-			FitMode: imagerequest.FitCover,
-		},
-		Expires: time.Now().Add(time.Hour),
-	}
-	signedReq.Signature = signer.Sign(signedReq)
-
-	// The original request should pass validation
-	t.Run("exact host passes", func(t *testing.T) {
-		t.Parallel()
-
-		err := svc.ValidateRequest(signedReq)
-		if err != nil {
-			t.Errorf("ValidateRequest() exact host failed: %v", err)
-		}
-	})
-
-	// Try to reuse the signature with different hosts
-	tests := []struct {
-		name string
-		host string
-	}{
-		{"parent domain", testHostExample},
-		{"sibling subdomain", "images.example.com"},
-		{"deeper subdomain", "a.cdn.example.com"},
-		{"evil suffix domain", "cdn.example.com.evil.com"},
-		{"prefixed host", "evilcdn.example.com"},
-	}
-
-	for _, tt := range tests {
-		t.Run(tt.name+" rejected", func(t *testing.T) {
-			t.Parallel()
-
-			req := &imagerequest.Request{
-				Source: imagerequest.Source{
-					Host:  tt.host,
-					Path:  signedReq.Source.Path,
-					Query: signedReq.Source.Query,
-				},
-				Transform: signedReq.Transform,
-				Expires:   signedReq.Expires,
-				Signature: signedReq.Signature,
-			}
-
-			err := svc.ValidateRequest(req)
-			if err == nil {
-				t.Errorf(
-					"ValidateRequest() should reject signature for host %q (signed for %q)",
-					tt.host, signedReq.Source.Host)
-			}
-		})
-	}
-}
-
-// TestService_ValidateRequest_HostWithPortNeedsSignature verifies that a host
-// followed by a port, or by a colon with no port, matches no allowlist entry,
-// so that a request for it without a signature is refused, while the same
-// host with no port is allowlisted.
-func TestService_ValidateRequest_HostWithPortNeedsSignature(t *testing.T) {
-	t.Parallel()
-
-	svc, _ := SetupTestService(t,
-		WithSigningKey(testSigningKey),
-		WithAllowlist(testHostCDN, "203.0.113.5", "2001:db8::1"),
-	)
-
-	tests := []struct {
-		host        string
-		allowlisted bool
-	}{
-		{testHostCDN, true},
-		{"cdn.example.com:8443", false},
-		{"cdn.example.com:443", false},
-		{"cdn.example.com:", false},
-		{"203.0.113.5", true},
-		{"203.0.113.5:8080", false},
-		{"[2001:db8::1]", true},
-		{"[2001:db8::1]:8443", false},
-	}
-
-	for _, tt := range tests {
-		t.Run(tt.host, func(t *testing.T) {
-			t.Parallel()
-
-			req := &imagerequest.Request{
-				Source: imagerequest.Source{
-					Host: tt.host,
-					Path: testPathCat,
-				},
-				Transform: imagerequest.Transform{
-					Size:    imagerequest.Size{Width: 50, Height: 50},
-					Format:  imagerequest.FormatJPEG,
-					Quality: 85,
-					FitMode: imagerequest.FitCover,
-				},
-			}
-
-			err := svc.ValidateRequest(req)
-			t.Logf("ValidateRequest() for host %q with no signature: %v",
-				tt.host, err)
-
-			if allowlisted := err == nil; allowlisted != tt.allowlisted {
-				t.Errorf("host %q served without a signature = %v, want %v",
-					tt.host, allowlisted, tt.allowlisted)
-			}
-		})
-	}
-}
-
-// TestService_ValidateRequest_UnbracketedIPv6PortNeedsSignature verifies that
-// an allowlisted IPv6 address written without brackets and followed by a
-// port, or by a colon with no port, needs a signature. Go's URL parser, which
-// the fetcher uses, reads 2001:db8::1:8443 as the host 2001:db8::1 with the
-// port 8443.
-func TestService_ValidateRequest_UnbracketedIPv6PortNeedsSignature(
-	t *testing.T,
-) {
-	t.Parallel()
-
-	svc, _ := SetupTestService(t,
-		WithSigningKey(testSigningKey),
-		WithAllowlist("2001:db8::1"),
-	)
-
-	for _, host := range []string{"2001:db8::1:8443", "2001:db8::1:"} {
-		t.Run(host, func(t *testing.T) {
-			t.Parallel()
-
-			req := &imagerequest.Request{
-				Source: imagerequest.Source{
-					Host: host,
-					Path: testPathCat,
-				},
-				Transform: imagerequest.Transform{
-					Size:    imagerequest.Size{Width: 50, Height: 50},
-					Format:  imagerequest.FormatJPEG,
-					Quality: 85,
-					FitMode: imagerequest.FitCover,
-				},
-			}
-
-			err := svc.ValidateRequest(req)
-			t.Logf("ValidateRequest() for host %q with no signature: %v",
-				host, err)
-
-			if err == nil {
-				t.Errorf("host %q served without a signature, want refused",
-					host)
-			}
-		})
-	}
-}
-
-func TestService_Get_InvalidFile(t *testing.T) {
-	t.Parallel()
-
-	svc, fixtures := SetupTestService(t)
+	svc, fixtures := setupTestProxy(t)
 	ctx := context.Background()
 
 	req := photoVariant(fixtures, 85, imagerequest.FitCover)
@@ -390,10 +114,10 @@ func TestService_Get_InvalidFile(t *testing.T) {
 	}
 }
 
-func TestService_Get_NotFound(t *testing.T) {
+func TestProxy_Get_NotFound(t *testing.T) {
 	t.Parallel()
 
-	svc, fixtures := SetupTestService(t)
+	svc, fixtures := setupTestProxy(t)
 	ctx := context.Background()
 
 	req := photoVariant(fixtures, 85, imagerequest.FitCover)
@@ -406,10 +130,10 @@ func TestService_Get_NotFound(t *testing.T) {
 	}
 }
 
-func TestService_Get_FormatConversion(t *testing.T) {
+func TestProxy_Get_FormatConversion(t *testing.T) {
 	t.Parallel()
 
-	svc, fixtures := SetupTestService(t)
+	svc, fixtures := setupTestProxy(t)
 	ctx := context.Background()
 
 	tests := []struct {
@@ -489,10 +213,10 @@ func TestService_Get_FormatConversion(t *testing.T) {
 	}
 }
 
-func TestService_Get_Caching(t *testing.T) {
+func TestProxy_Get_Caching(t *testing.T) {
 	t.Parallel()
 
-	svc, fixtures := SetupTestService(t)
+	svc, fixtures := setupTestProxy(t)
 	ctx := context.Background()
 
 	req := &imagerequest.Request{
@@ -548,23 +272,21 @@ func TestService_Get_Caching(t *testing.T) {
 	}
 }
 
-// TestService_Get_ServesSourceAndVariantCachedTwoDaysAgo caches a source and
+// TestProxy_Get_ServesSourceAndVariantCachedTwoDaysAgo caches a source and
 // a variant of it and sets back two days every time pixa stored for them.
-// For the source: fetched_at and last_accessed_at in source_content, and the
-// access and modification times of its file. For the variant: created_at and
-// last_accessed_at in variant_content, the created_at in its .meta file, and
-// the access and modification times of its file and of its .meta file. It
+// For the source: last_accessed_at in source_content, and the access and
+// modification times of its file. For the variant: last_accessed_at in
+// variant_content, and the access and modification times of its file. It
 // then checks that the variant is still served from the cache and that a new
 // variant is made from the cached source without fetching it again.
 //
 // Left at the present: the files' status change and birth times, which a
 // program cannot set, and the times of the directories holding the files,
-// which other entries share. The request_cache and output_content tables are
-// never written.
-func TestService_Get_ServesSourceAndVariantCachedTwoDaysAgo(t *testing.T) {
+// which other entries share.
+func TestProxy_Get_ServesSourceAndVariantCachedTwoDaysAgo(t *testing.T) {
 	t.Parallel()
 
-	svc, fixtures := SetupTestService(t)
+	svc, fixtures := setupTestProxy(t)
 	variant := photoVariant(fixtures, 85, imagerequest.FitCover)
 
 	resp, err := svc.Get(t.Context(), variant)
@@ -575,8 +297,8 @@ func TestService_Get_ServesSourceAndVariantCachedTwoDaysAgo(t *testing.T) {
 	_ = resp.Content.Close()
 
 	twoDaysAgo := time.Now().AddDate(0, 0, -2)
-	backdateSource(t, svc.cache, variant, twoDaysAgo)
-	backdateVariant(t, svc.cache, variant, twoDaysAgo)
+	backdateSource(t, svc, variant, twoDaysAgo)
+	backdateVariant(t, svc, variant, twoDaysAgo)
 
 	resp, err = svc.Get(t.Context(), variant)
 	if err != nil {
@@ -598,28 +320,23 @@ func TestService_Get_ServesSourceAndVariantCachedTwoDaysAgo(t *testing.T) {
 
 	_ = resp.Content.Close()
 
-	fetches := readCacheStatsCounters(t, svc.cache).upstreamFetchCount
+	fetches := readCounters(t, svc).upstreamFetchCount
 	if fetches != 1 {
 		t.Errorf("upstream fetches = %d, want 1", fetches)
 	}
 }
 
 // backdateSource sets every time pixa stored for req's cached source to when,
-// as listed on TestService_Get_ServesSourceAndVariantCachedTwoDaysAgo.
+// as listed on TestProxy_Get_ServesSourceAndVariantCachedTwoDaysAgo.
 func backdateSource(
-	t *testing.T, cache *Cache, req *imagerequest.Request, when time.Time,
+	t *testing.T, svc *testProxy, req *imagerequest.Request, when time.Time,
 ) {
 	t.Helper()
 
 	key := req.Source.Key()
-	dbTime := when.UTC().Format(sqliteTimestampFormat)
+	setSourceLastAccessed(t, svc.db, key, when)
 
-	updateOneRow(t, cache,
-		`UPDATE source_content SET fetched_at = ? WHERE source_key = ?`,
-		dbTime, string(key))
-	setSourceLastAccessed(t, cache, key, when)
-
-	path := cache.srcContent.keyToPath(key)
+	path := sourceFilePath(svc, key)
 
 	err := os.Chtimes(path, when, when)
 	if err != nil {
@@ -629,71 +346,27 @@ func backdateSource(
 
 // backdateVariant sets every time pixa stored for the cached variant req asks
 // for to when, as listed on
-// TestService_Get_ServesSourceAndVariantCachedTwoDaysAgo.
+// TestProxy_Get_ServesSourceAndVariantCachedTwoDaysAgo.
 func backdateVariant(
-	t *testing.T, cache *Cache, req *imagerequest.Request, when time.Time,
+	t *testing.T, svc *testProxy, req *imagerequest.Request, when time.Time,
 ) {
 	t.Helper()
 
 	key := imagerequest.CacheKey(req)
+	path := variantFilePath(svc, key)
 
-	content, size, contentType, err := cache.variants.LoadWithMeta(key)
-	if err != nil || contentType == "" {
-		t.Fatalf("reading the variant's .meta file: content type %q, error %v",
-			contentType, err)
-	}
+	setVariantLastAccessed(t, svc.db, key, when)
 
-	_ = content.Close()
-
-	meta, err := json.Marshal(VariantMeta{
-		ContentType: contentType,
-		Size:        size,
-		CreatedAt:   when.Unix(),
-	})
+	err := os.Chtimes(path, when, when)
 	if err != nil {
-		t.Fatalf("encoding the variant's .meta file: %v", err)
-	}
-
-	path := cache.variants.keyToPath(key)
-
-	err = os.WriteFile(path+variantMetaSuffix, meta, StorageFilePerm)
-	if err != nil {
-		t.Fatalf("writing the variant's .meta file: %v", err)
-	}
-
-	updateOneRow(t, cache,
-		`UPDATE variant_content SET created_at = ? WHERE cache_key = ?`,
-		when.UTC().Format(sqliteTimestampFormat), string(key))
-	setVariantLastAccessed(t, cache, key, when)
-
-	for _, file := range []string{path, path + variantMetaSuffix} {
-		err = os.Chtimes(file, when, when)
-		if err != nil {
-			t.Fatalf("setting back the times of %s: %v", file, err)
-		}
+		t.Fatalf("setting back the times of %s: %v", path, err)
 	}
 }
 
-// updateOneRow runs the UPDATE statement query and fails the test unless it
-// changed exactly one row.
-func updateOneRow(t *testing.T, cache *Cache, query string, args ...any) {
-	t.Helper()
-
-	result, err := cache.db.ExecContext(t.Context(), query, args...)
-	if err != nil {
-		t.Fatalf("%s: %v", query, err)
-	}
-
-	rows, err := result.RowsAffected()
-	if err != nil || rows != 1 {
-		t.Fatalf("%s: rows = %d, error = %v, want 1 row", query, rows, err)
-	}
-}
-
-func TestService_Get_DifferentSizes(t *testing.T) {
+func TestProxy_Get_DifferentSizes(t *testing.T) {
 	t.Parallel()
 
-	svc, fixtures := SetupTestService(t)
+	svc, fixtures := setupTestProxy(t)
 	ctx := context.Background()
 
 	// Request same image at different sizes
@@ -744,37 +417,10 @@ func TestService_Get_DifferentSizes(t *testing.T) {
 	}
 }
 
-func TestService_ValidateRequest_NoSigningKey(t *testing.T) {
+func TestProxy_Get_ContextCancellation(t *testing.T) {
 	t.Parallel()
 
-	// Service with no signing key - all non-allowlisted requests should fail
-	svc, fixtures := SetupTestService(t, WithNoAllowlist())
-
-	req := &imagerequest.Request{
-		Source: imagerequest.Source{
-			Host: fixtures.OtherHost,
-			Path: testPathUpload,
-		},
-		Transform: imagerequest.Transform{
-			Size:    imagerequest.Size{Width: 50, Height: 50},
-			Format:  imagerequest.FormatJPEG,
-			Quality: 85,
-			FitMode: imagerequest.FitCover,
-		},
-	}
-
-	err := svc.ValidateRequest(req)
-	if err == nil {
-		t.Error(
-			"ValidateRequest() expected error when no signing key and host not allowlisted",
-		)
-	}
-}
-
-func TestService_Get_ContextCancellation(t *testing.T) {
-	t.Parallel()
-
-	svc, fixtures := SetupTestService(t)
+	svc, fixtures := setupTestProxy(t)
 
 	ctx, cancel := context.WithCancel(context.Background())
 	cancel() // Cancel immediately
@@ -798,10 +444,10 @@ func TestService_Get_ContextCancellation(t *testing.T) {
 	}
 }
 
-func TestService_Get_ReturnsETag(t *testing.T) {
+func TestProxy_Get_ReturnsETag(t *testing.T) {
 	t.Parallel()
 
-	svc, fixtures := SetupTestService(t)
+	svc, fixtures := setupTestProxy(t)
 	ctx := context.Background()
 
 	req := &imagerequest.Request{
@@ -835,10 +481,10 @@ func TestService_Get_ReturnsETag(t *testing.T) {
 	}
 }
 
-func TestService_Get_ETagConsistency(t *testing.T) {
+func TestProxy_Get_ETagConsistency(t *testing.T) {
 	t.Parallel()
 
-	svc, fixtures := SetupTestService(t)
+	svc, fixtures := setupTestProxy(t)
 	ctx := context.Background()
 
 	req := &imagerequest.Request{
@@ -880,10 +526,10 @@ func TestService_Get_ETagConsistency(t *testing.T) {
 	}
 }
 
-func TestService_Get_DifferentETagsForDifferentContent(t *testing.T) {
+func TestProxy_Get_DifferentETagsForDifferentContent(t *testing.T) {
 	t.Parallel()
 
-	svc, fixtures := SetupTestService(t)
+	svc, fixtures := setupTestProxy(t)
 	ctx := context.Background()
 
 	// Request same image at different sizes - should get different ETags

internal/imageproxy/max_concurrent_processing_internal_test.go

@@ -1,4 +1,4 @@
-package imgcache
+package imageproxy
 
 import (
 	"image/color"
@@ -55,18 +55,18 @@ func holdProcessingSlot(
 	return release
 }
 
-// TestService_Get_WaitsForSlotBeforeReadingCachedSource checks that a
+// TestProxy_Get_WaitsForSlotBeforeReadingCachedSource checks that a
 // request whose source is cached holds none of it while it waits for a
 // processing slot: it reads the cached file only once it has a slot. With
 // the only slot held, a request for a new width of the cached 100x100 photo
 // waits; the cached file is then rewritten as a 100x50 image before the slot
 // is freed, so the request must answer with that image scaled to 40x20.
-func TestService_Get_WaitsForSlotBeforeReadingCachedSource(t *testing.T) {
+func TestProxy_Get_WaitsForSlotBeforeReadingCachedSource(t *testing.T) {
 	t.Parallel()
 
-	svc, fixtures := SetupTestService(t)
+	svc, fixtures := setupTestProxy(t)
 	svc.processor = imageprocessor.New(
-		imageprocessor.Params{MaxConcurrentProcessing: 1},
+		imageprocessor.Params{MaxInputBytes: 50 << 20, MaxConcurrentProcessing: 1},
 	)
 
 	// A first request caches the photo as a source.
@@ -77,13 +77,15 @@ func TestService_Get_WaitsForSlotBeforeReadingCachedSource(t *testing.T) {
 
 	_ = resp.Content.Close()
 
-	sourceKey, _, err := svc.cache.LookupSource(t.Context(),
-		widthOnlyRequest(fixtures, 50))
-	if err != nil || sourceKey == "" {
-		t.Fatalf("LookupSource() = %q, %v; want the cached source",
-			sourceKey, err)
+	cachedSource := widthOnlyRequest(fixtures, 50).Source
+
+	reader, _, err := svc.cache.Source(t.Context(), cachedSource)
+	if err != nil {
+		t.Fatalf("Source() error = %v; want the cached source", err)
 	}
 
+	_ = reader.Close()
+
 	release := holdProcessingSlot(t, svc.processor)
 
 	var (
@@ -103,7 +105,7 @@ func TestService_Get_WaitsForSlotBeforeReadingCachedSource(t *testing.T) {
 	// before waiting, it would have read it by now.
 	time.Sleep(100 * time.Millisecond)
 
-	err = os.WriteFile(svc.cache.srcContent.keyToPath(sourceKey),
+	err = os.WriteFile(sourceFilePath(svc, cachedSource.Key()),
 		generateTestJPEG(t, 100, 50, color.RGBA{0, 0, 255, 255}), 0o600)
 	if err != nil {
 		t.Fatalf("failed to rewrite the cached source: %v", err)

internal/imageproxy/negative_cache_internal_test.go

@@ -0,0 +1,96 @@
+package imageproxy
+
+import (
+	"errors"
+	"os"
+	"testing"
+
+	"sneak.berlin/go/pixa/internal/imagerequest"
+)
+
+// TestProxy_Get_ServesCachedVariantOfNegativeCachedURL caches a variant of
+// the test photo, then keeps a failed fetch of the photo's URL in the negative
+// cache, as a failed fetch for another variant does once the source has been
+// evicted. The cached variant is still served, and a variant not in the cache
+// is made from the cached source. Once the source is evicted, a variant not in
+// the cache needs a fetch and is refused.
+func TestProxy_Get_ServesCachedVariantOfNegativeCachedURL(t *testing.T) {
+	t.Parallel()
+
+	svc, fixtures := setupTestProxy(t)
+
+	resp, err := svc.Get(t.Context(), photoVariant(fixtures, 85, imagerequest.FitCover))
+	if err != nil {
+		t.Fatalf("Get() first request error = %v", err)
+	}
+
+	_ = resp.Content.Close()
+
+	err = svc.cache.PutFailure(t.Context(),
+		photoVariant(fixtures, 85, imagerequest.FitCover).Source)
+	if err != nil {
+		t.Fatalf("PutFailure() error = %v", err)
+	}
+
+	resp, err = svc.Get(t.Context(), photoVariant(fixtures, 85, imagerequest.FitCover))
+	if err != nil {
+		t.Fatalf("Get() of the cached variant error = %v", err)
+	}
+
+	_ = resp.Content.Close()
+
+	if resp.CacheStatus != CacheHit {
+		t.Errorf("cached variant CacheStatus = %q, want %q",
+			resp.CacheStatus, CacheHit)
+	}
+
+	resp, err = svc.Get(t.Context(), photoVariant(fixtures, 40, imagerequest.FitCover))
+	t.Logf("Get() of a variant made from the cached source: error = %v", err)
+
+	if err != nil {
+		t.Fatalf("Get() of a variant made from the cached source error = %v", err)
+	}
+
+	_ = resp.Content.Close()
+
+	source := photoVariant(fixtures, 85, imagerequest.FitCover).Source
+
+	reader, _, err := svc.cache.Source(t.Context(), source)
+	if err != nil {
+		t.Fatalf("Source() error = %v, want the cached source", err)
+	}
+
+	_ = reader.Close()
+
+	// Evicts the source as the cache's eviction does: its row, then its file
+	_, err = svc.db.ExecContext(t.Context(),
+		`DELETE FROM source_content WHERE source_key = ?`, string(source.Key()))
+	if err != nil {
+		t.Fatalf("deleting the source's row: %v", err)
+	}
+
+	err = os.Remove(sourceFilePath(svc, source.Key()))
+	if err != nil {
+		t.Fatalf("removing the source's file: %v", err)
+	}
+
+	_, err = svc.Get(t.Context(), photoVariant(fixtures, 60, imagerequest.FitCover))
+	t.Logf("Get() of a variant whose source was evicted: error = %v", err)
+
+	if !errors.Is(err, ErrNegativeCached) {
+		t.Errorf("Get() of a variant whose source was evicted error = %v, want %v",
+			err, ErrNegativeCached)
+	}
+}
+
+func TestProxy_Get_ReturnsErrorForNegativeCachedURL(t *testing.T) {
+	t.Parallel()
+
+	// This test verifies that Proxy.Get() checks the negative cache
+	// We can't easily test the full pipeline without vips, but we can
+	// verify the error type
+	err := ErrNegativeCached
+	if !errors.Is(err, ErrNegativeCached) {
+		t.Error("ErrNegativeCached should be identifiable with errors.Is")
+	}
+}

internal/imageproxy/partial_cache_files_internal_test.go

@@ -0,0 +1,273 @@
+package imageproxy
+
+import (
+	"bytes"
+	"errors"
+	"io"
+	"io/fs"
+	"log/slog"
+	"os"
+	"path/filepath"
+	"strings"
+	"testing"
+	"time"
+
+	"github.com/prometheus/client_golang/prometheus"
+	"sneak.berlin/go/pixa/internal/imagerequest"
+	"sneak.berlin/go/pixa/internal/imgcache"
+)
+
+// getImage calls Get for req and returns the image it answered with and
+// its cache status.
+func getImage(
+	t *testing.T, svc *testProxy, req *imagerequest.Request,
+) ([]byte, CacheStatus) {
+	t.Helper()
+
+	resp, err := svc.Get(t.Context(), req)
+	if err != nil {
+		t.Fatalf("Get() of quality %d error = %v", req.Transform.Quality, err)
+	}
+
+	defer func() { _ = resp.Content.Close() }()
+
+	body, err := io.ReadAll(resp.Content)
+	if err != nil {
+		t.Fatalf("reading the image of quality %d: %v", req.Transform.Quality, err)
+	}
+
+	return body, resp.CacheStatus
+}
+
+// fileSize returns the size of the file at path.
+func fileSize(t *testing.T, path string) int64 {
+	t.Helper()
+
+	info, err := os.Stat(path)
+	if err != nil {
+		t.Fatalf("stat %s: %v", path, err)
+	}
+
+	return info.Size()
+}
+
+// truncateFile cuts the file at path down to size bytes.
+func truncateFile(t *testing.T, path string, size int64) {
+	t.Helper()
+
+	t.Logf("cutting %s from %d to %d bytes",
+		filepath.Base(path), fileSize(t, path), size)
+
+	err := os.Truncate(path, size)
+	if err != nil {
+		t.Fatalf("truncating %s: %v", path, err)
+	}
+}
+
+// cachedSourcePath returns the path of the cached source file req uses.
+func cachedSourcePath(
+	t *testing.T, svc *testProxy, req *imagerequest.Request,
+) string {
+	t.Helper()
+
+	reader, _, err := svc.cache.Source(t.Context(), req.Source)
+	if err != nil {
+		t.Fatalf("Source() error = %v; want the cached source", err)
+	}
+
+	_ = reader.Close()
+
+	return sourceFilePath(svc, req.Source.Key())
+}
+
+// reconcileAsAtStartup runs the reconciliation pass of a second cache over
+// the database and the state directory of svc's cache, as pixad does when it
+// starts. It waits for the pass to remove the file at path, stops the second
+// cache, and returns what that cache logged.
+//
+// The database is limited to one connection, the one that holds the
+// in-memory schema. A statement cancelled while it runs can close that
+// connection, and the database with it. The test therefore takes the
+// connection before it stops the second cache, so that the stop only cancels
+// a wait for the connection.
+func reconcileAsAtStartup(t *testing.T, svc *testProxy, path string) string {
+	t.Helper()
+
+	var logBuf bytes.Buffer
+
+	cache, err := imgcache.NewCache(svc.db, imgcache.Config{
+		StateDir: svc.stateDir,
+		MaxBytes: 1 << 30,
+		Logger:   slog.New(slog.NewJSONHandler(&logBuf, nil)),
+	}, prometheus.NewRegistry())
+	if err != nil {
+		t.Fatalf("failed to create the second cache: %v", err)
+	}
+
+	svc.db.SetMaxOpenConns(1)
+	cache.StartEviction(imgcache.DefaultEvictionInterval)
+
+	deadline := time.Now().Add(5 * time.Second)
+
+	_, statErr := os.Stat(path)
+	for statErr == nil && time.Now().Before(deadline) {
+		time.Sleep(10 * time.Millisecond)
+
+		_, statErr = os.Stat(path)
+	}
+
+	releaseDatabase := holdDatabase(t, svc.db)
+	err = cache.StopEviction(t.Context())
+
+	releaseDatabase()
+
+	if err != nil {
+		t.Fatalf("StopEviction() error = %v", err)
+	}
+
+	t.Logf("reconciliation log: %s", logBuf.String())
+
+	if !errors.Is(statErr, fs.ErrNotExist) {
+		t.Fatalf("the reconciliation pass did not remove %s: %v", path, statErr)
+	}
+
+	return logBuf.String()
+}
+
+// assertWarningNames checks that logs hold a warning naming the file at
+// path.
+func assertWarningNames(t *testing.T, logs, path string) {
+	t.Helper()
+
+	for line := range strings.Lines(logs) {
+		if strings.Contains(line, `"level":"WARN"`) &&
+			strings.Contains(line, `"path":"`+path+`"`) {
+			return
+		}
+	}
+
+	t.Errorf("reconciliation logged no warning naming %s", path)
+}
+
+// TestProxy_Get_FetchesAgainForEmptyCachedSource empties the cached source
+// file, as a crash or a restored snapshot could, and runs the reconciliation
+// pass, which must remove it with a warning naming it. It then asks for two
+// new sizes of the source: the first must fetch it again and cache it whole,
+// so that the second is made from the cache without a third fetch.
+func TestProxy_Get_FetchesAgainForEmptyCachedSource(t *testing.T) {
+	t.Parallel()
+
+	svc, fixtures := setupTestProxy(t)
+
+	getImage(t, svc, photoVariant(fixtures, 85, imagerequest.FitCover))
+
+	path := cachedSourcePath(t, svc, photoVariant(fixtures, 85, imagerequest.FitCover))
+	wholeSize := fileSize(t, path)
+
+	truncateFile(t, path, 0)
+	assertWarningNames(t, reconcileAsAtStartup(t, svc, path), path)
+
+	getImage(t, svc, photoVariant(fixtures, 40, imagerequest.FitCover))
+	getImage(t, svc, photoVariant(fixtures, 30, imagerequest.FitCover))
+
+	fetches := readCounters(t, svc).upstreamFetchCount
+	t.Logf("upstream fetches = %d", fetches)
+
+	if fetches != 2 {
+		t.Errorf("upstream fetches = %d, want 2: the empty cached source "+
+			"was kept and fetched again on every new size", fetches)
+	}
+
+	size := fileSize(t, path)
+	if size != wholeSize {
+		t.Errorf("cached source is %d bytes, want %d", size, wholeSize)
+	}
+}
+
+// TestProxy_Get_FetchesAgainForPartialCachedSource cuts the cached source
+// file to half its size, as a crash or a restored snapshot could, and runs
+// the reconciliation pass, which must remove it with a warning naming it.
+// It then asks for two new sizes of the source: the first must fetch it
+// again rather than fail to decode the partial file, and the second is
+// made from the cache.
+func TestProxy_Get_FetchesAgainForPartialCachedSource(t *testing.T) {
+	t.Parallel()
+
+	svc, fixtures := setupTestProxy(t)
+
+	getImage(t, svc, photoVariant(fixtures, 85, imagerequest.FitCover))
+
+	path := cachedSourcePath(t, svc, photoVariant(fixtures, 85, imagerequest.FitCover))
+	wholeSize := fileSize(t, path)
+
+	truncateFile(t, path, wholeSize/2)
+	assertWarningNames(t, reconcileAsAtStartup(t, svc, path), path)
+
+	getImage(t, svc, photoVariant(fixtures, 40, imagerequest.FitCover))
+	getImage(t, svc, photoVariant(fixtures, 30, imagerequest.FitCover))
+
+	fetches := readCounters(t, svc).upstreamFetchCount
+	t.Logf("upstream fetches = %d", fetches)
+
+	if fetches != 2 {
+		t.Errorf("upstream fetches = %d, want 2", fetches)
+	}
+
+	size := fileSize(t, path)
+	if size != wholeSize {
+		t.Errorf("cached source is %d bytes, want %d", size, wholeSize)
+	}
+}
+
+// assertConvertedAgain caches a variant of the photo, cuts its file to size
+// bytes and runs the reconciliation pass, which must remove it with a
+// warning naming it. It then checks that Get converts the image again
+// instead of serving the file, and that the next Get serves the whole
+// variant from the cache.
+func assertConvertedAgain(t *testing.T, size int64) {
+	t.Helper()
+
+	svc, fixtures := setupTestProxy(t)
+	req := photoVariant(fixtures, 85, imagerequest.FitCover)
+
+	want, _ := getImage(t, svc, req)
+
+	path := variantFilePath(svc, imagerequest.CacheKey(req))
+
+	truncateFile(t, path, size)
+	assertWarningNames(t, reconcileAsAtStartup(t, svc, path), path)
+
+	got, status := getImage(t, svc, req)
+	t.Logf("after the cut: %d bytes, %s", len(got), status)
+
+	if status != CacheMiss || !bytes.Equal(got, want) {
+		t.Errorf("after the cut: Get() = %d bytes, %s; want the %d bytes "+
+			"converted again, %s", len(got), status, len(want), CacheMiss)
+	}
+
+	got, status = getImage(t, svc, req)
+	t.Logf("next request: %d bytes, %s", len(got), status)
+
+	if status != CacheHit || !bytes.Equal(got, want) {
+		t.Errorf("next request: Get() = %d bytes, %s; want the %d bytes, %s",
+			len(got), status, len(want), CacheHit)
+	}
+}
+
+// TestProxy_Get_ConvertsAgainForEmptyCachedVariant empties a cached
+// variant file, as a crash or a restored snapshot could, and checks that it
+// is converted again rather than served empty.
+func TestProxy_Get_ConvertsAgainForEmptyCachedVariant(t *testing.T) {
+	t.Parallel()
+
+	assertConvertedAgain(t, 0)
+}
+
+// TestProxy_Get_ConvertsAgainForPartialCachedVariant cuts a cached
+// variant file to ten bytes, as a crash or a restored snapshot could, and
+// checks that it is converted again rather than served cut short.
+func TestProxy_Get_ConvertsAgainForPartialCachedVariant(t *testing.T) {
+	t.Parallel()
+
+	assertConvertedAgain(t, 10)
+}

internal/imageproxy/pendingaccesstimes_internal_test.go

@@ -0,0 +1,78 @@
+package imageproxy
+
+import (
+	"context"
+	"database/sql"
+	"errors"
+	"testing"
+	"time"
+
+	"sneak.berlin/go/pixa/internal/imagerequest"
+)
+
+// pendingAccessTimesWait is how long a test waits for the database's
+// connection, for a cache hit to be answered, for an access time to reach
+// the database, or for a request to return past its deadline.
+const pendingAccessTimesWait = 5 * time.Second
+
+// holdDatabase takes the one connection of db, so that every other query
+// waits for it, and returns the func that frees it.
+func holdDatabase(t *testing.T, db *sql.DB) func() {
+	t.Helper()
+
+	ctx, cancel := context.WithTimeout(t.Context(), pendingAccessTimesWait)
+	defer cancel()
+
+	conn, err := db.Conn(ctx)
+	if err != nil {
+		t.Fatalf("failed to take the database connection: %v", err)
+	}
+
+	return func() { _ = conn.Close() }
+}
+
+// TestProxy_Get_ReturnsByItsDeadlineWhileTheDatabaseIsBusy holds the
+// database's one connection while a request whose fetch is held reaches its
+// deadline. The request must return with the deadline's error while the
+// connection is still held, as it waits for no database write past its
+// deadline.
+func TestProxy_Get_ReturnsByItsDeadlineWhileTheDatabaseIsBusy(t *testing.T) {
+	t.Parallel()
+
+	svc, fixtures, fetcher := setupHeldFetchProxy(t)
+
+	svc.cache.StartPendingAccessTimeWrites()
+	defer func() { _ = svc.cache.StopPendingAccessTimeWrites(t.Context()) }()
+
+	// Room for the request to reach its fetch on a busy host
+	const timeout = 2 * time.Second
+
+	ctx, cancel := context.WithTimeout(t.Context(), timeout)
+	defer cancel()
+
+	deadline, _ := ctx.Deadline()
+
+	results := startGet(ctx, svc, photoVariant(fixtures, 85, imagerequest.FitCover))
+
+	// The request has made its database reads by the time it fetches.
+	select {
+	case <-fetcher.started:
+	case <-time.After(timeout):
+		t.Fatal("request did not reach its fetch by its deadline")
+	}
+
+	releaseDatabase := holdDatabase(t, svc.db)
+	defer releaseDatabase()
+
+	select {
+	case got := <-results:
+		t.Logf("Get() returned %v after its deadline, error = %v",
+			time.Since(deadline), got.err)
+
+		if !errors.Is(got.err, context.DeadlineExceeded) {
+			t.Errorf("Get() error = %v, want %v", got.err, context.DeadlineExceeded)
+		}
+	case <-time.After(time.Until(deadline) + pendingAccessTimesWait):
+		t.Fatal("request did not return while the database was busy")
+	}
+}

internal/imageproxy/stats_internal_test.go

@@ -0,0 +1,243 @@
+package imageproxy
+
+import (
+	"bytes"
+	"context"
+	"image/color"
+	"io"
+	"io/fs"
+	"testing"
+
+	"sneak.berlin/go/pixa/internal/httpfetcher"
+	"sneak.berlin/go/pixa/internal/imagerequest"
+)
+
+// TestProxy_Get_CountsStats walks Get through a miss that fetches the
+// source, a hit, a miss that reuses the cached source, and two misses whose
+// source cannot be used, checking every counter of the cache after each.
+func TestProxy_Get_CountsStats(t *testing.T) {
+	t.Parallel()
+
+	svc, fixtures := setupTestProxy(t)
+
+	// NewTestFS builds the same files the test proxy's fetcher serves.
+	testFS, _ := NewTestFS(t)
+
+	photo, err := fs.ReadFile(testFS, fixtures.GoodHostJPEG)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	fake, err := fs.ReadFile(testFS, fixtures.InvalidFile)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	photoBytes, fakeBytes := int64(len(photo)), int64(len(fake))
+
+	// want is hits, misses, upstream fetches, upstream bytes, transforms.
+	steps := []struct {
+		name    string
+		path    string
+		size    int
+		wantErr bool
+		want    counterValues
+	}{
+		{"miss that fetches the source", testPathPhoto, 50, false,
+			counterValues{0, 1, 1, photoBytes, 1}},
+		{"hit", testPathPhoto, 50, false,
+			counterValues{1, 1, 1, photoBytes, 1}},
+		{"miss that reuses the cached source", testPathPhoto, 25, false,
+			counterValues{1, 2, 1, photoBytes, 2}},
+		{"miss whose source fails the magic byte check", "/images/fake.jpg", 50, true,
+			counterValues{1, 3, 2, photoBytes + fakeBytes, 2}},
+		{"miss whose source is not found", "/images/nonexistent.jpg", 50, true,
+			counterValues{1, 4, 2, photoBytes + fakeBytes, 2}},
+	}
+
+	for _, step := range steps {
+		resp, err := svc.Get(t.Context(), &imagerequest.Request{
+			Source: imagerequest.Source{
+				Host: fixtures.GoodHost,
+				Path: step.path,
+			},
+			Transform: imagerequest.Transform{
+				Size:    imagerequest.Size{Width: step.size, Height: step.size},
+				Format:  imagerequest.FormatJPEG,
+				Quality: 85,
+				FitMode: imagerequest.FitCover,
+			},
+		})
+		if (err != nil) != step.wantErr {
+			t.Fatalf("%s: Get() error = %v, want error %t", step.name, err, step.wantErr)
+		}
+
+		if err == nil {
+			_ = resp.Content.Close()
+		}
+
+		got := readCounters(t, svc)
+		if got != step.want {
+			t.Fatalf("after the %s: counters = %+v, want %+v", step.name, got, step.want)
+		}
+	}
+}
+
+// fakeUpstream answers every fetch with itself as a JPEG body. The body
+// serves data, then calls cancel, when set, and returns err; io.EOF ends
+// the body normally.
+type fakeUpstream struct {
+	data   *bytes.Reader
+	cancel context.CancelFunc
+	err    error
+}
+
+func (u *fakeUpstream) Fetch(
+	context.Context, imagerequest.Source,
+) (*httpfetcher.FetchResult, error) {
+	return &httpfetcher.FetchResult{
+		Content:       io.NopCloser(u),
+		ContentLength: -1,
+		ContentType:   testContentTypeJPEG,
+	}, nil
+}
+
+func (u *fakeUpstream) Read(p []byte) (int, error) {
+	if u.data.Len() > 0 {
+		return u.data.Read(p)
+	}
+
+	if u.cancel != nil {
+		u.cancel()
+	}
+
+	return 0, u.err
+}
+
+// TestProxy_Get_CountsInterruptedMisses checks every counter of the cache
+// after a miss whose request context ends during or after the upstream
+// fetch, and after a miss whose upstream body is over the size limit.
+func TestProxy_Get_CountsInterruptedMisses(t *testing.T) {
+	t.Parallel()
+
+	photo := generateTestJPEG(t, 100, 100, color.RGBA{255, 0, 0, 255})
+	half := len(photo) / 2
+
+	// want is hits, misses, upstream fetches, upstream bytes, transforms.
+	tests := []struct {
+		name    string
+		served  int   // bytes of the photo the upstream body serves
+		cancel  bool  // whether the body then ends the request context
+		readErr error // what the body then returns
+		wantErr bool
+		want    counterValues
+	}{
+		{"request context ends during the fetch", half, true, context.Canceled, true,
+			counterValues{0, 1, 1, int64(half), 0}},
+		{"request context ends after the fetch", len(photo), true, io.EOF, false,
+			counterValues{0, 1, 1, int64(len(photo)), 1}},
+		{"upstream body over the size limit", half, false,
+			httpfetcher.ErrResponseTooLarge, true,
+			counterValues{0, 1, 1, int64(half), 0}},
+	}
+
+	for _, tc := range tests {
+		t.Run(tc.name, func(t *testing.T) {
+			t.Parallel()
+
+			svc, fixtures := setupTestProxy(t)
+
+			ctx, cancel := context.WithCancel(t.Context())
+			defer cancel()
+
+			upstream := &fakeUpstream{
+				data: bytes.NewReader(photo[:tc.served]),
+				err:  tc.readErr,
+			}
+			if tc.cancel {
+				upstream.cancel = cancel
+			}
+
+			svc.fetcher = upstream
+
+			resp, err := svc.Get(ctx, &imagerequest.Request{
+				Source: imagerequest.Source{
+					Host: fixtures.GoodHost,
+					Path: testPathPhoto,
+				},
+				Transform: imagerequest.Transform{
+					Size:    imagerequest.Size{Width: 50, Height: 50},
+					Format:  imagerequest.FormatJPEG,
+					Quality: 85,
+					FitMode: imagerequest.FitCover,
+				},
+			})
+			t.Logf("Get() error = %v", err)
+
+			if (err != nil) != tc.wantErr {
+				t.Fatalf("Get() error = %v, want error %t", err, tc.wantErr)
+			}
+
+			if err == nil {
+				_ = resp.Content.Close()
+			}
+
+			got := readCounters(t, svc)
+			if got != tc.want {
+				t.Errorf("counters = %+v, want %+v", got, tc.want)
+			}
+		})
+	}
+}
+
+// TestProxy_Get_CountsHitAfterRequestEnds checks every counter of the
+// cache after a hit served with a request context that has already ended:
+// only the hit count moves.
+func TestProxy_Get_CountsHitAfterRequestEnds(t *testing.T) {
+	t.Parallel()
+
+	svc, fixtures := setupTestProxy(t)
+
+	req := &imagerequest.Request{
+		Source: imagerequest.Source{
+			Host: fixtures.GoodHost,
+			Path: testPathPhoto,
+		},
+		Transform: imagerequest.Transform{
+			Size:    imagerequest.Size{Width: 50, Height: 50},
+			Format:  imagerequest.FormatJPEG,
+			Quality: 85,
+			FitMode: imagerequest.FitCover,
+		},
+	}
+
+	// A first request caches the variant.
+	resp, err := svc.Get(t.Context(), req)
+	if err != nil {
+		t.Fatalf("first Get() error = %v", err)
+	}
+
+	_ = resp.Content.Close()
+
+	want := readCounters(t, svc)
+	want.hitCount++
+
+	ctx, cancel := context.WithCancel(t.Context())
+	cancel()
+
+	resp, err = svc.Get(ctx, req)
+	if err != nil {
+		t.Fatalf("Get() with an ended request context: error = %v", err)
+	}
+
+	_ = resp.Content.Close()
+
+	if resp.CacheStatus != CacheHit {
+		t.Fatalf("CacheStatus = %v, want %v", resp.CacheStatus, CacheHit)
+	}
+
+	got := readCounters(t, svc)
+	if got != want {
+		t.Errorf("counters = %+v, want %+v", got, want)
+	}
+}

internal/imageproxy/testutil_internal_test.go

@@ -0,0 +1,354 @@
+package imageproxy
+
+import (
+	"bytes"
+	"context"
+	"database/sql"
+	"image"
+	"image/color"
+	"image/gif"
+	"image/jpeg"
+	"image/png"
+	"io/fs"
+	"path/filepath"
+	"testing"
+	"testing/fstest"
+	"time"
+
+	"github.com/prometheus/client_golang/prometheus"
+	"sneak.berlin/go/pixa/internal/database"
+	"sneak.berlin/go/pixa/internal/httpfetcher"
+	"sneak.berlin/go/pixa/internal/imageprocessor"
+	"sneak.berlin/go/pixa/internal/imagerequest"
+	"sneak.berlin/go/pixa/internal/imgcache"
+)
+
+// Shared test data literals, extracted as constants for goconst.
+const (
+	testContentTypeJPEG = "image/jpeg"
+)
+
+// sqliteTimestampFormat matches the format SQLite's CURRENT_TIMESTAMP
+// produces, so injected timestamps compare correctly against ones the
+// cache writes.
+const sqliteTimestampFormat = "2006-01-02 15:04:05"
+
+// TestFixtures contains paths to test files in the mock filesystem.
+type TestFixtures struct {
+	// Valid image files
+	GoodHostJPEG  string // valid JPEG on GoodHost
+	GoodHostPNG   string // valid PNG on GoodHost
+	GoodHostGIF   string // valid GIF on GoodHost
+	OtherHostJPEG string // valid JPEG on OtherHost
+	OtherHostPNG  string // valid PNG on OtherHost
+
+	// Invalid/edge case files
+	InvalidFile string // file with wrong magic bytes
+	EmptyFile   string // zero-byte file
+	TextFile    string // text file masquerading as image
+
+	// Hostnames
+	GoodHost  string // a host the mock fetcher serves
+	OtherHost string // a second host the mock fetcher serves
+}
+
+// DefaultFixtures returns the standard test fixture paths.
+func DefaultFixtures() *TestFixtures {
+	return &TestFixtures{
+		GoodHostJPEG:  "goodhost.example.com/images/photo.jpg",
+		GoodHostPNG:   "goodhost.example.com/images/logo.png",
+		GoodHostGIF:   "goodhost.example.com/images/animation.gif",
+		OtherHostJPEG: "otherhost.example.com/uploads/image.jpg",
+		OtherHostPNG:  "otherhost.example.com/uploads/icon.png",
+		InvalidFile:   "goodhost.example.com/images/fake.jpg",
+		EmptyFile:     "goodhost.example.com/images/empty.jpg",
+		TextFile:      "goodhost.example.com/images/text.png",
+		GoodHost:      "goodhost.example.com",
+		OtherHost:     "otherhost.example.com",
+	}
+}
+
+// NewTestFS creates a mock filesystem with test images.
+func NewTestFS(t *testing.T) (fs.FS, *TestFixtures) {
+	t.Helper()
+
+	fixtures := DefaultFixtures()
+
+	// Generate test images
+	jpegData := generateTestJPEG(t, 100, 100, color.RGBA{255, 0, 0, 255})
+	pngData := generateTestPNG(t, 100, 100, color.RGBA{0, 255, 0, 255})
+	gifData := generateTestGIF(t, 100, 100, color.RGBA{0, 0, 255, 255})
+
+	// Create the mock filesystem
+	mockFS := fstest.MapFS{
+		// Good host files
+		fixtures.GoodHostJPEG: &fstest.MapFile{Data: jpegData},
+		fixtures.GoodHostPNG:  &fstest.MapFile{Data: pngData},
+		fixtures.GoodHostGIF:  &fstest.MapFile{Data: gifData},
+
+		// Other host files
+		fixtures.OtherHostJPEG: &fstest.MapFile{Data: jpegData},
+		fixtures.OtherHostPNG:  &fstest.MapFile{Data: pngData},
+
+		// Invalid files
+		fixtures.InvalidFile: &fstest.MapFile{Data: []byte("not a real image file content")},
+		fixtures.EmptyFile:   &fstest.MapFile{Data: []byte{}},
+		fixtures.TextFile:    &fstest.MapFile{Data: []byte("This is just text, not a PNG")},
+	}
+
+	return mockFS, fixtures
+}
+
+// generateTestJPEG creates a minimal valid JPEG image.
+func generateTestJPEG(t *testing.T, width, height int, c color.Color) []byte {
+	t.Helper()
+
+	img := image.NewRGBA(image.Rect(0, 0, width, height))
+	for y := range height {
+		for x := range width {
+			img.Set(x, y, c)
+		}
+	}
+
+	var buf bytes.Buffer
+
+	err := jpeg.Encode(&buf, img, &jpeg.Options{Quality: 85})
+	if err != nil {
+		t.Fatalf("failed to encode test JPEG: %v", err)
+	}
+
+	return buf.Bytes()
+}
+
+// generateTestPNG creates a minimal valid PNG image.
+func generateTestPNG(t *testing.T, width, height int, c color.Color) []byte {
+	t.Helper()
+
+	img := image.NewRGBA(image.Rect(0, 0, width, height))
+	for y := range height {
+		for x := range width {
+			img.Set(x, y, c)
+		}
+	}
+
+	var buf bytes.Buffer
+
+	err := png.Encode(&buf, img)
+	if err != nil {
+		t.Fatalf("failed to encode test PNG: %v", err)
+	}
+
+	return buf.Bytes()
+}
+
+// generateTestGIF creates a minimal valid GIF image.
+func generateTestGIF(t *testing.T, width, height int, c color.Color) []byte {
+	t.Helper()
+
+	img := image.NewPaletted(
+		image.Rect(0, 0, width, height),
+		[]color.Color{c, color.White},
+	)
+	for y := range height {
+		for x := range width {
+			img.SetColorIndex(x, y, 0)
+		}
+	}
+
+	var buf bytes.Buffer
+
+	err := gif.Encode(&buf, img, nil)
+	if err != nil {
+		t.Fatalf("failed to encode test GIF: %v", err)
+	}
+
+	return buf.Bytes()
+}
+
+// testProxy is a Proxy with the database, the state directory and the
+// metrics registry of its cache, through which a test reads and changes the
+// cache behind the proxy.
+type testProxy struct {
+	*Proxy
+
+	db       *sql.DB
+	stateDir string
+	registry *prometheus.Registry
+}
+
+// setupTestProxy creates a Proxy with mock fetcher for testing.
+func setupTestProxy(t *testing.T) (*testProxy, *TestFixtures) {
+	t.Helper()
+
+	mockFS, fixtures := NewTestFS(t)
+
+	// Create temp directory for cache
+	tmpDir := t.TempDir()
+
+	// Create in-memory database
+	db := setupTestDB(t)
+
+	registry := prometheus.NewRegistry()
+
+	cache, err := imgcache.NewCache(db, imgcache.Config{
+		StateDir:    tmpDir,
+		NegativeTTL: 5 * time.Minute,
+		MaxBytes:    1 << 30,
+	}, registry)
+	if err != nil {
+		t.Fatalf("failed to create cache: %v", err)
+	}
+
+	proxy, err := NewProxy(&Config{
+		Cache:   cache,
+		Fetcher: httpfetcher.NewMock(mockFS),
+		Processor: imageprocessor.New(imageprocessor.Params{
+			MaxInputBytes:           50 << 20,
+			MaxConcurrentProcessing: 2,
+		}),
+		MaxResponseSize: 50 << 20,
+	})
+	if err != nil {
+		t.Fatalf("failed to create proxy: %v", err)
+	}
+
+	return &testProxy{
+		Proxy:    proxy,
+		db:       db,
+		stateDir: tmpDir,
+		registry: registry,
+	}, fixtures
+}
+
+// counterValues holds the values of a cache's counters.
+type counterValues struct {
+	hitCount           int64
+	missCount          int64
+	upstreamFetchCount int64
+	upstreamFetchBytes int64
+	transformCount     int64
+}
+
+// readCounters reads the counters of svc's cache from its metrics registry.
+func readCounters(t *testing.T, svc *testProxy) counterValues {
+	t.Helper()
+
+	families, err := svc.registry.Gather()
+	if err != nil {
+		t.Fatalf("failed to gather the cache's counters: %v", err)
+	}
+
+	values := make(map[string]int64, len(families))
+
+	for _, family := range families {
+		values[family.GetName()] = int64(family.GetMetric()[0].GetCounter().GetValue())
+	}
+
+	return counterValues{
+		hitCount:           values["pixa_cache_hits_total"],
+		missCount:          values["pixa_cache_misses_total"],
+		upstreamFetchCount: values["pixa_upstream_fetches_total"],
+		upstreamFetchBytes: values["pixa_upstream_fetch_bytes_total"],
+		transformCount:     values["pixa_transforms_total"],
+	}
+}
+
+// sourceFilePath returns the path of the file of the source key in svc's
+// cache, as README.md gives it: <state_dir>/cache/sources/<ab>/<cd>/<key>.
+func sourceFilePath(svc *testProxy, key imagerequest.SourceKey) string {
+	name := string(key)
+
+	return filepath.Join(svc.stateDir, "cache", "sources", name[0:2], name[2:4], name)
+}
+
+// variantFilePath returns the path of the file of the variant key in svc's
+// cache, as README.md gives it: <state_dir>/cache/variants/<ab>/<cd>/<key>.
+func variantFilePath(svc *testProxy, key imagerequest.VariantKey) string {
+	name := string(key)
+
+	return filepath.Join(svc.stateDir, "cache", "variants", name[0:2], name[2:4], name)
+}
+
+// setVariantLastAccessed backdates the last access time of a tracked
+// variant.
+func setVariantLastAccessed(
+	t *testing.T, db *sql.DB, key imagerequest.VariantKey, when time.Time,
+) {
+	t.Helper()
+
+	res, err := db.ExecContext(t.Context(),
+		`UPDATE variant_content SET last_accessed_at = ? WHERE cache_key = ?`,
+		when.UTC().Format(sqliteTimestampFormat), string(key),
+	)
+	if err != nil {
+		t.Fatalf("failed to set variant last_accessed_at: %v", err)
+	}
+
+	affected, err := res.RowsAffected()
+	if err != nil {
+		t.Fatalf("failed to read affected rows: %v", err)
+	}
+
+	if affected != 1 {
+		t.Fatalf("variant %s has no accounting row (affected=%d); "+
+			"stores must track variants in the database", key, affected)
+	}
+}
+
+// setSourceLastAccessed backdates the last access time of a tracked
+// source content blob.
+func setSourceLastAccessed(
+	t *testing.T, db *sql.DB, key imagerequest.SourceKey, when time.Time,
+) {
+	t.Helper()
+
+	res, err := db.ExecContext(t.Context(),
+		`UPDATE source_content SET last_accessed_at = ? WHERE source_key = ?`,
+		when.UTC().Format(sqliteTimestampFormat), string(key),
+	)
+	if err != nil {
+		t.Fatalf("failed to set source last_accessed_at: %v", err)
+	}
+
+	affected, err := res.RowsAffected()
+	if err != nil {
+		t.Fatalf("failed to read affected rows: %v", err)
+	}
+
+	if affected != 1 {
+		t.Fatalf("source %s has no accounting row (affected=%d)", key, affected)
+	}
+}
+
+// countRows returns the number of rows the given query yields.
+func countRows(t *testing.T, db *sql.DB, query string, args ...any) int {
+	t.Helper()
+
+	var n int
+
+	err := db.QueryRowContext(t.Context(), query, args...).Scan(&n)
+	if err != nil {
+		t.Fatalf("count query %q failed: %v", query, err)
+	}
+
+	return n
+}
+
+// setupTestDB creates an in-memory SQLite database for testing
+// using the production schema.
+func setupTestDB(t *testing.T) *sql.DB {
+	t.Helper()
+
+	db, err := sql.Open("sqlite", ":memory:")
+	if err != nil {
+		t.Fatalf("failed to open test db: %v", err)
+	}
+
+	// Use the real production schema via migrations
+	err = database.ApplyMigrations(context.Background(), db, nil)
+	if err != nil {
+		t.Fatalf("failed to apply migrations: %v", err)
+	}
+
+	return db
+}

internal/imagerequest/imagerequest.go

@@ -1,13 +1,12 @@
 // Package imagerequest holds the image request type, its parts, its checks,
-// the parsing of the /v1/image/ path, the variant key and the source key. It
-// imports no other package of this repo, so every package that handles an
-// image request can import it.
+// the parsing of the /v1/image/ path, the variant key, the source key and the
+// source's upstream URL. It imports no other package of this repo, so every
+// package that handles an image request can import it.
 package imagerequest
 
 import (
 	"errors"
 	"net/url"
-	"time"
 )
 
 // ErrInvalidFitMode is returned when an invalid fit mode is provided.
@@ -85,19 +84,13 @@ func ValidateImageRequest(req *Request) error {
 }
 
 // Request represents a request for a processed image: the source to fetch
-// and the transform to apply to it.
+// and the transform to apply to it. It holds exactly what the variant key
+// covers.
 type Request struct {
 	// Source is what to fetch
 	Source Source
 	// Transform is what to do to it
 	Transform Transform
-	// Signature is the HMAC signature for non-allowlisted hosts
-	Signature string
-	// Expires is when the URL expires: the exp of a signed URL, or the expiry
-	// of an encrypted URL; the zero time if it has none
-	Expires time.Time
-	// AllowHTTP indicates whether HTTP (non-TLS) is allowed for this request
-	AllowHTTP bool
 }
 
 // Source is the image a request fetches from the origin.
@@ -125,23 +118,20 @@ type Transform struct {
 	FitMode Fit
 }
 
-// SourceURL returns the full upstream URL to fetch: the source path
-// percent-encoded again, as net/url writes a path, so each source path is one
-// upstream path, and the source query as it is, with the characters
-// escapeQuery encodes percent-encoded. Uses http:// scheme when AllowHTTP is
-// true, otherwise https://.
-func (r *Request) SourceURL() string {
+// URL returns the source's upstream URL, with the scheme http when allowHTTP
+// is true and https otherwise. net/url writes its path percent-encoded again,
+// so each source path is one upstream path. Its query is the source query as
+// it is, with the characters escapeQuery encodes percent-encoded.
+func (s Source) URL(allowHTTP bool) *url.URL {
 	scheme := "https"
-	if r.AllowHTTP {
+	if allowHTTP {
 		scheme = "http"
 	}
 
-	sourceURL := url.URL{
+	return &url.URL{
 		Scheme:   scheme,
-		Host:     r.Source.Host,
-		Path:     r.Source.Path,
-		RawQuery: escapeQuery(r.Source.Query),
+		Host:     s.Host,
+		Path:     s.Path,
+		RawQuery: escapeQuery(s.Query),
 	}
-
-	return sourceURL.String()
 }

internal/imagerequest/sourceurl_internal_test.go

@@ -2,57 +2,49 @@ package imagerequest
 
 import "testing"
 
-func TestRequest_SourceURL_DefaultHTTPS(t *testing.T) {
+func TestSource_URL_DefaultHTTPS(t *testing.T) {
 	t.Parallel()
 
-	req := &Request{
-		Source: Source{
-			Host:  testHostCDN,
-			Path:  testPathCat,
-			Query: "v=2",
-		},
+	source := Source{
+		Host:  testHostCDN,
+		Path:  testPathCat,
+		Query: "v=2",
 	}
 
-	got := req.SourceURL()
+	got := source.URL(false).String()
 
 	want := "https://cdn.example.com/photos/cat.jpg?v=2"
 	if got != want {
-		t.Errorf("SourceURL() = %q, want %q", got, want)
+		t.Errorf("URL() = %q, want %q", got, want)
 	}
 }
 
-func TestRequest_SourceURL_AllowHTTP(t *testing.T) {
+func TestSource_URL_AllowHTTP(t *testing.T) {
 	t.Parallel()
 
-	req := &Request{
-		Source: Source{
-			Host: "localhost:8080",
-			Path: testPathCat,
-		},
-		AllowHTTP: true,
+	source := Source{
+		Host: "localhost:8080",
+		Path: testPathCat,
 	}
 
-	got := req.SourceURL()
+	got := source.URL(true).String()
 
 	want := "http://localhost:8080/photos/cat.jpg"
 	if got != want {
-		t.Errorf("SourceURL() = %q, want %q", got, want)
+		t.Errorf("URL() = %q, want %q", got, want)
 	}
 }
 
-func TestRequest_SourceURL_AllowHTTPFalse(t *testing.T) {
+func TestSource_URL_AllowHTTPFalse(t *testing.T) {
 	t.Parallel()
 
-	req := &Request{
-		Source: Source{
-			Host: testHostCDN,
-			Path: "/img.jpg",
-		},
-		AllowHTTP: false,
+	source := Source{
+		Host: testHostCDN,
+		Path: "/img.jpg",
 	}
 
-	got := req.SourceURL()
+	got := source.URL(false).String()
 	if got != "https://cdn.example.com/img.jpg" {
-		t.Errorf("SourceURL() = %q, want https scheme", got)
+		t.Errorf("URL() = %q, want https scheme", got)
 	}
 }

internal/imagerequest/urlparser.go

@@ -66,23 +66,6 @@ var queryEscapeRegex = regexp.MustCompile(`%[0-9A-Fa-f]{2}|[^!-~]|["#'<>%]`)
 var hostRegex = regexp.MustCompile(
 	`^(?:[A-Za-z0-9_.-]+|\[([0-9A-Fa-f:.]+)\])(?::[0-9]*)?$`)
 
-// ParseImagePath parses an image URL's path after /v1/image/, decoded once:
-// <host>/<path>/<size>.<format>, or <host>/<path>/<size> for JPEG XL.
-// Examples:
-//   - cdn.example.com/photos/cat.jpg/800x600.webp
-//   - cdn.example.com/photos/cat.jpg/0x0.jpeg
-//   - cdn.example.com/photos/cat.jpg/orig.png
-//   - cdn.example.com/photos/cat.jpg/800x600
-func ParseImagePath(path string) (*Request, error) {
-	// Strip leading slash if present (chi may include it)
-	path = strings.TrimPrefix(path, "/")
-	if path == "" {
-		return nil, ErrMissingHost
-	}
-
-	return parseImageComponents(path)
-}
-
 // ParseImageURL parses an image URL's path, decoded once as r.URL.Path holds
 // it: /v1/image/<host>/<path>/<size>.<format>, or /v1/image/<host>/<path>/<size>
 // for JPEG XL. A source query follows the source path after a "?", which the
@@ -143,15 +126,15 @@ func parseImageComponents(remainder string) (*Request, error) {
 		return nil, ErrMissingHost
 	}
 
-	// Only the forms hostRegex describes are accepted. A "?", "#" or "@" in
-	// the host would make Go's URL parser, which the allowlist check and the
-	// fetcher use, read a different host from the one allowlisted and cached
+	// Later steps put the host unparsed into the allowlist check, the cache key
+	// and the upstream URL, so only the forms hostRegex describes are accepted
 	hostMatch := hostRegex.FindStringSubmatch(host)
 	if hostMatch == nil {
 		return nil, fmt.Errorf("%w: %q", ErrInvalidHost, host)
 	}
 
-	// Go's URL parser refuses brackets around anything but an IPv6 address
+	// The allowlist check and the fetcher drop the brackets, so brackets
+	// around anything but an IPv6 address would give a host a second spelling
 	if inBrackets := hostMatch[1]; inBrackets != "" {
 		addr, err := netip.ParseAddr(inBrackets)
 		if err != nil || !addr.Is6() {
@@ -164,9 +147,8 @@ func parseImageComponents(remainder string) (*Request, error) {
 	// host
 	host = strings.ToLower(host)
 
-	// Go's URL parser, which the fetcher uses, reads a "#" as the start of a
-	// fragment and drops the rest, so each text after it would be another
-	// spelling, cached under its own keys, of one upstream request
+	// A "#" starts a URL's fragment, so neither the source path nor the source
+	// query may hold one
 	if strings.Contains(path, "#") {
 		return nil, fmt.Errorf(
 			`%w: the source path or query holds a "#"`, ErrInvalidPath)

internal/imagerequest/urlparser_internal_test.go

@@ -192,76 +192,16 @@ func TestParseImageURL_Errors(t *testing.T) {
 	}
 }
 
-func TestParseImagePath(t *testing.T) {
-	t.Parallel()
-
-	// ParseImagePath is for chi wildcard capture (no /v1/image/ prefix)
-	tests := []struct {
-		name    string
-		input   string
-		want    *Request
-		wantErr bool
-	}{
-		{
-			name:  "chi wildcard capture",
-			input: "cdn.example.com/photos/cat.jpg/800x600.webp",
-			want: &Request{
-				Source: Source{
-					Host: testHostCDN,
-					Path: testPathCat,
-				},
-				Transform: Transform{
-					Size:   Size{Width: 800, Height: 600},
-					Format: FormatWebP,
-				},
-			},
-		},
-		{
-			name:  "with leading slash from chi",
-			input: "/cdn.example.com/photos/cat.jpg/800x600.webp",
-			want: &Request{
-				Source: Source{
-					Host: testHostCDN,
-					Path: testPathCat,
-				},
-				Transform: Transform{
-					Size:   Size{Width: 800, Height: 600},
-					Format: FormatWebP,
-				},
-			},
-		},
-	}
-
-	for _, tt := range tests {
-		t.Run(tt.name, func(t *testing.T) {
-			t.Parallel()
-
-			got, err := ParseImagePath(tt.input)
-			if (err != nil) != tt.wantErr {
-				t.Errorf("ParseImagePath() error = %v, wantErr %v", err, tt.wantErr)
-
-				return
-			}
-
-			if err != nil {
-				return
-			}
-
-			assertRequest(t, got, tt.want)
-		})
-	}
-}
-
-// TestParseImagePath_LowerCasesHost verifies that the host is lower-cased, so
+// TestParseImageURL_LowerCasesHost verifies that the host is lower-cased, so
 // that every spelling of its letter case is the same host to the allowlist,
 // the upstream connection limits and the cache, and that the path keeps its
 // case.
-func TestParseImagePath_LowerCasesHost(t *testing.T) {
+func TestParseImageURL_LowerCasesHost(t *testing.T) {
 	t.Parallel()
 
-	got, err := ParseImagePath("CDN.Example.COM/Photos/Cat.jpg/800x600.webp")
+	got, err := ParseImageURL("/v1/image/CDN.Example.COM/Photos/Cat.jpg/800x600.webp")
 	if err != nil {
-		t.Fatalf("ParseImagePath() error = %v", err)
+		t.Fatalf("ParseImageURL() error = %v", err)
 	}
 
 	t.Logf("parsed host %q, path %q", got.Source.Host, got.Source.Path)
@@ -275,21 +215,21 @@ func TestParseImagePath_LowerCasesHost(t *testing.T) {
 	}
 }
 
-// TestParseImagePath_RefusesUserinfo verifies that a host holding "@" is
+// TestParseImageURL_RefusesUserinfo verifies that a host holding "@" is
 // refused: a URL reads what comes before the "@" as a user name and password
 // and the rest as the host.
-func TestParseImagePath_RefusesUserinfo(t *testing.T) {
+func TestParseImageURL_RefusesUserinfo(t *testing.T) {
 	t.Parallel()
 
 	for _, input := range []string{
-		"user@cdn.example.com/photos/cat.jpg/800x600.webp",
-		"user:password@cdn.example.com/photos/cat.jpg/800x600.webp",
+		"/v1/image/user@cdn.example.com/photos/cat.jpg/800x600.webp",
+		"/v1/image/user:password@cdn.example.com/photos/cat.jpg/800x600.webp",
 	} {
-		got, err := ParseImagePath(input)
-		t.Logf("ParseImagePath(%q) = %+v, error = %v", input, got, err)
+		got, err := ParseImageURL(input)
+		t.Logf("ParseImageURL(%q) = %+v, error = %v", input, got, err)
 
 		if !errors.Is(err, ErrInvalidHost) {
-			t.Errorf("ParseImagePath(%q) error = %v, want %v",
+			t.Errorf("ParseImageURL(%q) error = %v, want %v",
 				input, err, ErrInvalidHost)
 		}
 	}
@@ -315,6 +255,10 @@ func TestParseImageURL_PathTraversal(t *testing.T) {
 			name:  "parent in middle of path",
 			input: "/v1/image/cdn.example.com/photos/../../../etc/passwd/800x600.jpeg",
 		},
+		{
+			name:  "parent directory in path",
+			input: "/v1/image/cdn.example.com/photos/../secret/800x600.jpeg",
+		},
 		{
 			name:  "encoded parent directory",
 			input: "/v1/image/cdn.example.com/photos/%2e%2e/secret/800x600.jpeg",
@@ -357,40 +301,6 @@ func TestParseImageURL_PathTraversal(t *testing.T) {
 	}
 }
 
-func TestParseImagePath_PathTraversal(t *testing.T) {
-	t.Parallel()
-
-	// Test path traversal via ParseImagePath (chi wildcard)
-	tests := []struct {
-		name  string
-		input string
-	}{
-		{
-			name:  "parent directory in path",
-			input: "cdn.example.com/photos/../secret/800x600.jpeg",
-		},
-		{
-			name:  "encoded traversal",
-			input: "cdn.example.com/photos/%2e%2e/secret/800x600.jpeg",
-		},
-	}
-
-	for _, tt := range tests {
-		t.Run(tt.name, func(t *testing.T) {
-			t.Parallel()
-
-			_, err := ParseImagePath(tt.input)
-			if err == nil {
-				t.Error("ParseImagePath() should reject path traversal attempts")
-			}
-
-			if !errors.Is(err, ErrPathTraversal) {
-				t.Errorf("ParseImagePath() error = %v, want ErrPathTraversal", err)
-			}
-		})
-	}
-}
-
 // TestParseImageURL_SourceURLHoldingPercent25 verifies that a source path and
 // a source query holding %25, the percent-escape of a literal "%", are
 // accepted as they are, although decoding them once more leaves a "%" that

internal/imgcache/cache.go

@@ -1,3 +1,10 @@
+// Package imgcache is the image cache. It keeps:
+//   - the source images fetched from upstream
+//   - the variants made from them
+//   - the negative cache: the source URLs whose fetch failed recently
+//
+// Each image is a file with a row in the database. Eviction removes the
+// least recently used images to keep the cache under its size limit.
 package imgcache
 
 import (
@@ -8,24 +15,17 @@ import (
 	"io"
 	"log/slog"
 	"path/filepath"
-	"sync"
-	"sync/atomic"
 	"time"
 
-	lru "github.com/hashicorp/golang-lru/v2"
-	"sneak.berlin/go/pixa/internal/httpfetcher"
+	"github.com/prometheus/client_golang/prometheus"
 	"sneak.berlin/go/pixa/internal/imagerequest"
 )
 
-// Cache errors.
-var (
-	ErrCacheMiss     = errors.New("cache miss")
-	ErrNegativeCache = errors.New("negative cache hit")
-)
-
-// metaCacheSize is how many variants' content types metaCache holds. A
-// variant not among them is served as before, reading its .meta file.
-const metaCacheSize = 10000
+// maxPendingAccessTimes is how many access times of variants served from
+// the cache, and how many of cached sources a miss used, may each wait to be
+// written, at about 80 bytes each. An access time handed over while that
+// many of its kind wait is dropped.
+const maxPendingAccessTimes = 10000
 
 // cleanExpiredBatchSize is the most expired negative cache rows one
 // statement of CleanExpired deletes. Each batch is a statement of its
@@ -33,29 +33,22 @@ const metaCacheSize = 10000
 // have expired.
 const cleanExpiredBatchSize = 1000
 
-// CacheConfig holds cache configuration.
-type CacheConfig struct {
+// DefaultNegativeTTL is how long pixad keeps a failed fetch in the negative
+// cache.
+const DefaultNegativeTTL = 5 * time.Minute
+
+// Config holds cache configuration.
+type Config struct {
 	StateDir    string
 	NegativeTTL time.Duration
 
 	// MaxBytes is the disk cache size limit in bytes that eviction
-	// enforces. Zero means no limit is enforced (no eviction).
+	// enforces. Zero turns the disk cache off entirely: no cache
+	// directories are created, lookups always miss, stores are no-ops,
+	// and the eviction goroutine only removes the expired negative cache
+	// rows.
 	MaxBytes int64
 
-	// UseDefaultMaxBytes makes NewCache replace MaxBytes with the
-	// default limit: 75% of the sum of the space free on the filesystem
-	// holding the cache and the bytes the cache already holds, at least
-	// DefaultCacheMaxBytesFloor. The config layer sets this when the
-	// operator omits cache_max_bytes.
-	UseDefaultMaxBytes bool
-
-	// DisableDiskCache turns the disk cache off entirely: no cache
-	// directories are created, lookups always miss, stores are
-	// no-ops, and the eviction goroutine only removes the expired
-	// negative cache rows. The config layer sets this when the
-	// operator configures cache_max_bytes: 0.
-	DisableDiskCache bool
-
 	// Logger receives accounting and eviction log output. A nil
 	// Logger means slog.Default().
 	Logger *slog.Logger
@@ -66,13 +59,9 @@ type Cache struct {
 	db         *sql.DB
 	srcContent *ContentStorage // source images by source key
 	variants   *VariantStorage // processed variants by cache key
-	config     CacheConfig
+	config     Config
 	log        *slog.Logger
 
-	// disabled means the disk cache is turned off entirely: lookups
-	// always miss, stores are no-ops, and no eviction runs.
-	disabled bool
-
 	// Eviction machinery. The channels are created in NewCache so
 	// stores can signal write pressure without racing StartEviction.
 	// evictionCancel, set by StartEviction, cancels the eviction
@@ -81,34 +70,30 @@ type Cache struct {
 	evictionDone     chan struct{}
 	evictionCancel   context.CancelFunc
 
-	// The pending counts: hits, misses, upstream fetches and transforms
-	// whose write to the database missed its request's deadline, kept
-	// here until they are written by the goroutine that
-	// StartPendingCountWrites starts. As for eviction, the channels are
-	// created in NewCache: pendingCountsAdded wakes that goroutine, and
-	// pendingCountsDone is closed when it returns. pendingCountsCancel,
-	// set by StartPendingCountWrites, cancels its context, which tells it
-	// to finish.
-	pendingHits               atomic.Int64
-	pendingMisses             atomic.Int64
-	pendingUpstreamFetches    atomic.Int64
-	pendingUpstreamFetchBytes atomic.Int64
-	pendingTransforms         atomic.Int64
-	pendingCountsAdded        chan struct{}
-	pendingCountsDone         chan struct{}
-	pendingCountsCancel       context.CancelFunc
+	// pendingVariantAccessTimes holds the variants served from the cache,
+	// and pendingSourceAccessTimes the cached sources a miss used, whose
+	// access time is still to be written by the goroutine that
+	// StartPendingAccessTimeWrites starts, so that no request waits for the
+	// database to record it. As for eviction, the channels are created in
+	// NewCache, so that requests can hand over access times before that
+	// goroutine starts; pendingAccessTimesDone is closed when it returns.
+	// pendingAccessTimesCancel, set by StartPendingAccessTimeWrites,
+	// cancels its context, which tells it to finish.
+	pendingVariantAccessTimes chan imagerequest.VariantKey
+	pendingSourceAccessTimes  chan imagerequest.SourceKey
+	pendingAccessTimesDone    chan struct{}
+	pendingAccessTimesCancel  context.CancelFunc
 
-	// Held through each write of the pending counts, and by
-	// readHitAndMissCounts to read them
-	pendingCountsWriteMutex sync.Mutex
-
-	// metaCache holds the content types of the variants most recently
-	// stored or served, so a hit does not read the variant's .meta file.
-	// It never stands in for the variant file, which is always opened.
-	metaCache *lru.Cache[imagerequest.VariantKey, string]
+	// The counts of what the cache does, made and registered by
+	// registerCounters (see counters.go)
+	cacheHits          prometheus.Counter
+	cacheMisses        prometheus.Counter
+	upstreamFetches    prometheus.Counter
+	upstreamFetchBytes prometheus.Counter
+	transforms         prometheus.Counter
 
 	// reconciliationPageSize is the most rows one read of a content table
-	// returns in the reconciliation pass and in Stats. newCache sets it to
+	// returns in the reconciliation pass. NewCache sets it to
 	// defaultReconciliationPageSize; tests set it smaller.
 	reconciliationPageSize int
 
@@ -119,42 +104,37 @@ type Cache struct {
 	reconciliationReadTestHook func(rows int)
 }
 
-// NewCache creates a new cache instance.
-func NewCache(db *sql.DB, config CacheConfig) (*Cache, error) {
-	return newCache(db, config, defaultFreeSpaceProbe)
-}
-
-// newCache is NewCache with the free-space probe passed in, so tests
-// can fake the free space the default limit is worked out from.
-func newCache(
-	db *sql.DB, config CacheConfig, probe FreeSpaceProbeFunc,
+// NewCache creates a new cache instance and registers its counters on
+// metricsRegistry. pixad passes Prometheus' default registry, the one
+// /metrics serves. It makes no database call.
+func NewCache(
+	db *sql.DB, config Config, metricsRegistry prometheus.Registerer,
 ) (*Cache, error) {
 	log := config.Logger
 	if log == nil {
 		log = slog.Default()
 	}
 
-	metaCache, err := lru.New[imagerequest.VariantKey, string](metaCacheSize)
-	if err != nil {
-		return nil, fmt.Errorf("failed to create variant content type cache: %w", err)
-	}
-
 	c := &Cache{
 		db:               db,
 		config:           config,
 		log:              log,
-		disabled:         config.DisableDiskCache,
 		evictionPressure: make(chan struct{}, 1),
 		evictionDone:     make(chan struct{}),
-		metaCache:        metaCache,
 
-		pendingCountsAdded: make(chan struct{}, 1),
-		pendingCountsDone:  make(chan struct{}),
+		pendingVariantAccessTimes: make(chan imagerequest.VariantKey, maxPendingAccessTimes),
+		pendingSourceAccessTimes:  make(chan imagerequest.SourceKey, maxPendingAccessTimes),
+		pendingAccessTimesDone:    make(chan struct{}),
 
 		reconciliationPageSize: defaultReconciliationPageSize,
 	}
 
-	if c.disabled {
+	err := c.registerCounters(metricsRegistry)
+	if err != nil {
+		return nil, err
+	}
+
+	if config.MaxBytes == 0 {
 		return c, nil
 	}
 
@@ -166,7 +146,7 @@ func newCache(
 	}
 
 	variants, err := NewVariantStorage(
-		filepath.Join(config.StateDir, "cache", "variants"), log,
+		filepath.Join(config.StateDir, "cache", "variants"),
 	)
 	if err != nil {
 		return nil, fmt.Errorf("failed to create variant storage: %w", err)
@@ -175,98 +155,114 @@ func newCache(
 	c.srcContent = srcContent
 	c.variants = variants
 
-	if config.UseDefaultMaxBytes {
-		limit, err := c.computeDefaultMaxBytes(context.Background(), probe)
-		if err != nil {
-			return nil, err
-		}
-
-		c.config.MaxBytes = limit
-
-		log.Info("computed default cache size limit from free space and cache contents",
-			"cache_max_bytes", limit,
-			"cache_dir", filepath.Join(config.StateDir, "cache"),
-		)
-	}
-
 	return c, nil
 }
 
-// LookupResult contains the result of a cache lookup.
-type LookupResult struct {
-	Hit         bool
-	CacheKey    imagerequest.VariantKey
-	ContentType string
-	SizeBytes   int64
-	CacheStatus CacheStatus
-}
-
-// Lookup checks if a processed variant exists on disk. Hits touch the
-// variant's LRU timestamp; a disabled cache always misses.
-func (c *Cache) Lookup(
-	ctx context.Context, req *imagerequest.Request,
-) (*LookupResult, error) {
-	cacheKey := imagerequest.CacheKey(req)
-
-	// Check variant storage directly - no DB needed for cache hits
-	if !c.disabled && c.variants.Exists(cacheKey) {
-		c.touchVariant(ctx, cacheKey)
-
-		return &LookupResult{
-			Hit:         true,
-			CacheKey:    cacheKey,
-			CacheStatus: CacheHit,
-		}, nil
-	}
-
-	return &LookupResult{
-		Hit:         false,
-		CacheKey:    cacheKey,
-		CacheStatus: CacheMiss,
-	}, nil
-}
-
-// GetVariant returns a reader, size, and content type for a cached
-// variant. The content type comes from metaCache, or else from the
-// variant's .meta file and is then kept in metaCache. A variant with
-// no .meta file is served as application/octet-stream, which is not
-// kept.
-func (c *Cache) GetVariant(
+// Variant opens a cached variant and returns a reader, its size and its
+// content type, read from the variant file's first bytes. It opens that one
+// file and makes no database call. A variant it opens has its access time
+// handed to the background writer without waiting (see touchVariant). A
+// variant not in the cache, or any variant of a disabled cache, is
+// ErrNotFound; a variant file whose first bytes are not those of an output
+// format is another error.
+func (c *Cache) Variant(
 	cacheKey imagerequest.VariantKey,
 ) (io.ReadCloser, int64, string, error) {
-	if c.disabled {
+	if c.config.MaxBytes == 0 {
 		return nil, 0, "", ErrNotFound
 	}
 
-	contentType, known := c.metaCache.Get(cacheKey)
-	if !known {
-		return c.loadVariantWithMeta(cacheKey)
-	}
-
-	reader, size, err := c.variants.LoadWithSize(cacheKey)
+	reader, size, contentType, err := c.variants.LoadWithContentType(cacheKey)
 	if err != nil {
-		// The file is gone, e.g. deleted outside pixa
-		c.metaCache.Remove(cacheKey)
-
 		return nil, 0, "", err
 	}
 
+	c.touchVariant(cacheKey)
+
 	return reader, size, contentType, nil
 }
 
-// StoreSource stores fetched source content: it writes the file named by
+// PutVariant stores a processed variant by its cache key and records
+// it in the size accounting. On a disabled cache it is a no-op. The
+// accounting insert is best-effort (the startup reconciliation pass
+// adopts any variant file that misses its accounting row).
+func (c *Cache) PutVariant(
+	ctx context.Context, cacheKey imagerequest.VariantKey, content io.Reader,
+) error {
+	if c.config.MaxBytes == 0 {
+		return nil
+	}
+
+	size, err := c.variants.Store(cacheKey, content)
+	if err != nil {
+		return err
+	}
+
+	_, err = c.db.ExecContext(ctx, `
+		INSERT INTO variant_content (cache_key, size_bytes)
+		VALUES (?, ?)
+		ON CONFLICT(cache_key) DO UPDATE SET
+			size_bytes = excluded.size_bytes,
+			last_accessed_at = CURRENT_TIMESTAMP
+	`, string(cacheKey), size)
+	if err != nil {
+		c.log.Warn("failed to record variant in size accounting",
+			"cache_key", cacheKey, "error", err)
+	}
+
+	c.notifyWritePressure()
+
+	return nil
+}
+
+// Source opens the cached source of src and returns a reader and its size
+// in bytes. A source whose row or file is missing is ErrNotFound, as is any
+// source of a disabled cache. A source it opens has its access time handed
+// to the background writer without waiting (see touchSourceContent).
+func (c *Cache) Source(
+	ctx context.Context, src imagerequest.Source,
+) (io.ReadCloser, int64, error) {
+	if c.config.MaxBytes == 0 {
+		return nil, 0, ErrNotFound
+	}
+
+	key := src.Key()
+
+	var rowExists bool
+
+	err := c.db.QueryRowContext(ctx, `
+		SELECT EXISTS(SELECT 1 FROM source_content WHERE source_key = ?)
+	`, string(key)).Scan(&rowExists)
+	if err != nil {
+		return nil, 0, fmt.Errorf("failed to lookup source: %w", err)
+	}
+
+	if !rowExists {
+		return nil, 0, ErrNotFound
+	}
+
+	reader, size, err := c.srcContent.LoadWithSize(key)
+	if err != nil {
+		return nil, 0, err
+	}
+
+	c.touchSourceContent(key)
+
+	return reader, size, nil
+}
+
+// PutSource stores fetched source content: it writes the file named by
 // the source's key, then the source's row. When the row cannot be written
 // the file is removed and the store fails, as a file with no row is never
 // served or evicted; if that removal fails too, the next reconciliation
 // pass removes the file. On a disabled cache it is a no-op returning an
 // empty key.
-func (c *Cache) StoreSource(
+func (c *Cache) PutSource(
 	ctx context.Context,
-	req *imagerequest.Request,
+	src imagerequest.Source,
 	content io.Reader,
-	result *httpfetcher.FetchResult,
 ) (imagerequest.SourceKey, error) {
-	if c.disabled {
+	if c.config.MaxBytes == 0 {
 		return "", nil
 	}
 
@@ -275,7 +271,7 @@ func (c *Cache) StoreSource(
 		return "", fmt.Errorf("failed to read source content: %w", err)
 	}
 
-	key := req.Source.Key()
+	key := src.Key()
 
 	size, err := c.srcContent.Store(key, data)
 	if err != nil {
@@ -283,17 +279,12 @@ func (c *Cache) StoreSource(
 	}
 
 	_, err = c.db.ExecContext(ctx, `
-		INSERT INTO source_content
-			(source_key, source_host, source_path, source_query,
-			 content_type, size_bytes)
-		VALUES (?, ?, ?, ?, ?, ?)
+		INSERT INTO source_content (source_key, size_bytes)
+		VALUES (?, ?)
 		ON CONFLICT(source_key) DO UPDATE SET
-			content_type = excluded.content_type,
 			size_bytes = excluded.size_bytes,
-			fetched_at = CURRENT_TIMESTAMP,
 			last_accessed_at = CURRENT_TIMESTAMP
-	`, string(key), req.Source.Host, req.Source.Path, req.Source.Query,
-		result.ContentType, size)
+	`, string(key), size)
 	if err != nil {
 		deleteErr := c.srcContent.Delete(key)
 		if deleteErr != nil {
@@ -309,98 +300,43 @@ func (c *Cache) StoreSource(
 	return key, nil
 }
 
-// StoreVariant stores a processed variant by its cache key and records
-// it in the size accounting. On a disabled cache it is a no-op. The
-// accounting insert is best-effort (the startup reconciliation pass
-// adopts any variant file that misses its accounting row).
-func (c *Cache) StoreVariant(
-	ctx context.Context, cacheKey imagerequest.VariantKey,
-	content io.Reader, contentType string,
-) error {
-	if c.disabled {
-		return nil
-	}
-
-	size, err := c.variants.Store(cacheKey, content, contentType)
-	if err != nil {
-		return err
-	}
-
-	c.metaCache.Add(cacheKey, contentType)
-
-	_, err = c.db.ExecContext(ctx, `
-		INSERT INTO variant_content (cache_key, size_bytes, content_type)
-		VALUES (?, ?, ?)
-		ON CONFLICT(cache_key) DO UPDATE SET
-			size_bytes = excluded.size_bytes,
-			content_type = excluded.content_type,
-			last_accessed_at = CURRENT_TIMESTAMP
-	`, string(cacheKey), size, contentType)
-	if err != nil {
-		c.log.Warn("failed to record variant in size accounting",
-			"cache_key", cacheKey, "error", err)
-	}
-
-	c.notifyWritePressure()
-
-	return nil
-}
-
-// LookupSource checks if we have cached source content for a request.
-// Returns the source key and content type if found, or empty values
-// if not; a row whose file is missing is not found. Hits touch the
-// source's LRU timestamp; a disabled cache always reports no cached
-// source.
-func (c *Cache) LookupSource(
-	ctx context.Context, req *imagerequest.Request,
-) (imagerequest.SourceKey, string, error) {
-	if c.disabled {
-		return "", "", nil
-	}
-
-	key := req.Source.Key()
-
-	var contentType string
+// Failure reports whether src is in the negative cache. It only reads: a
+// row past its expiry is not a hit, and the eviction goroutine deletes it
+// with the other expired rows (see CleanExpired).
+func (c *Cache) Failure(
+	ctx context.Context, src imagerequest.Source,
+) (bool, error) {
+	var expiresAt time.Time
 
 	err := c.db.QueryRowContext(ctx, `
-		SELECT content_type FROM source_content WHERE source_key = ?
-	`, string(key)).Scan(&contentType)
+		SELECT expires_at FROM negative_cache
+		WHERE source_host = ? AND source_path = ? AND source_query = ?
+	`, src.Host, src.Path, src.Query).Scan(&expiresAt)
 
 	if errors.Is(err, sql.ErrNoRows) {
-		return "", "", nil
+		return false, nil
 	}
 
 	if err != nil {
-		return "", "", fmt.Errorf("failed to lookup source: %w", err)
+		return false, fmt.Errorf("failed to check negative cache: %w", err)
 	}
 
-	// Verify the content file exists
-	if !c.srcContent.Exists(key) {
-		return "", "", nil
-	}
+	expired := time.Now().After(expiresAt)
 
-	c.touchSourceContent(ctx, key)
-
-	return key, contentType, nil
+	return !expired, nil
 }
 
-// StoreNegative stores a negative cache entry for a failed fetch.
-func (c *Cache) StoreNegative(
-	ctx context.Context, req *imagerequest.Request, statusCode int, errMsg string,
-) error {
+// PutFailure stores a negative cache entry for src, whose fetch failed.
+func (c *Cache) PutFailure(ctx context.Context, src imagerequest.Source) error {
 	expiresAt := time.Now().UTC().Add(c.config.NegativeTTL)
 
 	_, err := c.db.ExecContext(ctx, `
 		INSERT INTO negative_cache
-			(source_host, source_path, source_query, status_code,
-			 error_message, expires_at)
-		VALUES (?, ?, ?, ?, ?, ?)
+			(source_host, source_path, source_query, expires_at)
+		VALUES (?, ?, ?, ?)
 		ON CONFLICT(source_host, source_path, source_query) DO UPDATE SET
-			status_code = excluded.status_code,
-			error_message = excluded.error_message,
-			fetched_at = CURRENT_TIMESTAMP,
 			expires_at = excluded.expires_at
-	`, req.Source.Host, req.Source.Path, req.Source.Query, statusCode, errMsg, expiresAt)
+	`, src.Host, src.Path, src.Query, expiresAt)
 	if err != nil {
 		return fmt.Errorf("failed to insert negative cache: %w", err)
 	}
@@ -408,18 +344,6 @@ func (c *Cache) StoreNegative(
 	return nil
 }
 
-// GetSourceContent returns a reader for cached source content by its key,
-// and the content's size in bytes.
-func (c *Cache) GetSourceContent(
-	key imagerequest.SourceKey,
-) (io.ReadCloser, int64, error) {
-	if c.disabled {
-		return nil, 0, ErrNotFound
-	}
-
-	return c.srcContent.LoadWithSize(key)
-}
-
 // CleanExpired removes the negative cache rows past their expiry, one
 // batch per statement, until none is left. Once ctx ends it starts no
 // further batch and returns ctx's error. The eviction goroutine calls it
@@ -443,236 +367,58 @@ func (c *Cache) CleanExpired(ctx context.Context) error {
 	}
 }
 
-// Stats returns cache statistics.
-func (c *Cache) Stats(ctx context.Context) (*CacheStats, error) {
-	var stats CacheStats
-
-	err := c.readHitAndMissCounts(ctx, &stats)
-	if err != nil && !errors.Is(err, sql.ErrNoRows) {
-		return nil, fmt.Errorf("failed to get cache stats: %w", err)
-	}
-
-	// Count and size the cached source images and processed variants from
-	// their tables. A disabled cache holds none, whatever rows an earlier
-	// run left.
-	if !c.disabled {
-		err = c.db.QueryRowContext(ctx, `
-			SELECT (SELECT COUNT(*) FROM source_content)
-			     + (SELECT COUNT(*) FROM variant_content)
-		`).Scan(&stats.TotalItems)
-		if err != nil {
-			c.log.Warn("failed to count cache items for stats", "error", err)
-		}
-
-		stats.TotalSizeBytes, err = c.sumContentSizeBytes(ctx)
-		if err != nil {
-			c.log.Warn("failed to sum cache size for stats", "error", err)
-		}
-	}
-
-	// Compute hit rate as a ratio
-	if stats.HitCount+stats.MissCount > 0 {
-		stats.HitRate = float64(stats.HitCount) / float64(stats.HitCount+stats.MissCount)
-	}
-
-	return &stats, nil
-}
-
 // IncrementStats counts a cache hit or miss, and an upstream fetch that read
 // fetchBytes bytes, as IncrementUpstreamFetch does. Like the other Increment
-// methods, it writes to the database with ctx's deadline but not its
-// cancellation, so an ended request is still counted but does not wait for
-// the database past its deadline; a count not written by then becomes a
-// pending count (see StartPendingCountWrites).
-func (c *Cache) IncrementStats(ctx context.Context, hit bool, fetchBytes int64) {
-	countCtx, cancel := withoutCancelKeepingDeadline(ctx)
-	defer cancel()
-
-	var err error
-
-	pendingCount := &c.pendingMisses
-
+// methods, it only adds to the cache's Prometheus counters in memory (see
+// counters.go), so it never waits for the database or fails.
+func (c *Cache) IncrementStats(hit bool, fetchBytes int64) {
 	if hit {
-		pendingCount = &c.pendingHits
-
-		_, err = c.db.ExecContext(countCtx, `
-			UPDATE cache_stats
-			SET hit_count = hit_count + 1,
-			    last_updated_at = CURRENT_TIMESTAMP
-			WHERE id = 1
-		`)
+		c.cacheHits.Inc()
 	} else {
-		_, err = c.db.ExecContext(countCtx, `
-			UPDATE cache_stats
-			SET miss_count = miss_count + 1,
-			    last_updated_at = CURRENT_TIMESTAMP
-			WHERE id = 1
-		`)
+		c.cacheMisses.Inc()
 	}
 
-	switch {
-	case errors.Is(err, context.DeadlineExceeded):
-		c.addPendingCount(pendingCount, 1)
-	case err != nil:
-		c.log.Warn("failed to count cache hit or miss", "hit", hit, "error", err)
-	}
-
-	c.IncrementUpstreamFetch(ctx, fetchBytes)
+	c.IncrementUpstreamFetch(fetchBytes)
 }
 
 // IncrementUpstreamFetch counts one upstream fetch that read fetchBytes bytes.
 // A fetch that read no bytes is not counted.
-func (c *Cache) IncrementUpstreamFetch(ctx context.Context, fetchBytes int64) {
+func (c *Cache) IncrementUpstreamFetch(fetchBytes int64) {
 	if fetchBytes <= 0 {
 		return
 	}
 
-	countCtx, cancel := withoutCancelKeepingDeadline(ctx)
-	defer cancel()
-
-	_, err := c.db.ExecContext(countCtx, `
-		UPDATE cache_stats
-		SET upstream_fetch_count = upstream_fetch_count + 1,
-		    upstream_fetch_bytes = upstream_fetch_bytes + ?,
-		    last_updated_at = CURRENT_TIMESTAMP
-		WHERE id = 1
-	`, fetchBytes)
-
-	switch {
-	case errors.Is(err, context.DeadlineExceeded):
-		c.addPendingCount(&c.pendingUpstreamFetches, 1)
-		c.addPendingCount(&c.pendingUpstreamFetchBytes, fetchBytes)
-	case err != nil:
-		c.log.Warn("failed to count upstream fetch",
-			"fetch_bytes", fetchBytes, "error", err)
-	}
+	c.upstreamFetches.Inc()
+	c.upstreamFetchBytes.Add(float64(fetchBytes))
 }
 
 // IncrementTransformCount counts one image transcoded by the image processor.
-func (c *Cache) IncrementTransformCount(ctx context.Context) {
-	countCtx, cancel := withoutCancelKeepingDeadline(ctx)
-	defer cancel()
+func (c *Cache) IncrementTransformCount() {
+	c.transforms.Inc()
+}
 
-	_, err := c.db.ExecContext(countCtx, `
-		UPDATE cache_stats
-		SET transform_count = transform_count + 1,
-		    last_updated_at = CURRENT_TIMESTAMP
-		WHERE id = 1
-	`)
-
-	switch {
-	case errors.Is(err, context.DeadlineExceeded):
-		c.addPendingCount(&c.pendingTransforms, 1)
-	case err != nil:
-		c.log.Warn("failed to count transform", "error", err)
+// touchVariant hands the access time of a variant served from the cache to
+// the goroutine StartPendingAccessTimeWrites starts, which writes it in its
+// next batch. It never waits: while maxPendingAccessTimes access times of
+// variants wait to be written, this one is dropped, and the variant only
+// looks colder to eviction.
+func (c *Cache) touchVariant(cacheKey imagerequest.VariantKey) {
+	select {
+	case c.pendingVariantAccessTimes <- cacheKey:
+	default:
 	}
 }
 
-// readHitAndMissCounts sets the hit and miss counts of stats to those of the
-// cache_stats row plus the pending hits and misses (see
-// StartPendingCountWrites). It holds pendingCountsWriteMutex, so that no
-// write of the pending counts falls between the two reads.
-func (c *Cache) readHitAndMissCounts(ctx context.Context, stats *CacheStats) error {
-	c.pendingCountsWriteMutex.Lock()
-	defer c.pendingCountsWriteMutex.Unlock()
-
-	err := c.db.QueryRowContext(ctx, `
-		SELECT hit_count, miss_count
-		FROM cache_stats WHERE id = 1
-	`).Scan(&stats.HitCount, &stats.MissCount)
-
-	stats.HitCount += c.pendingHits.Load()
-	stats.MissCount += c.pendingMisses.Load()
-
-	return err
-}
-
-// loadVariantWithMeta is GetVariant for a variant metaCache does not
-// hold: it reads the content type from the variant's .meta file and
-// keeps it in metaCache, unless a StoreVariant has put one there
-// meanwhile, as the store's is newer. A read that finds no .meta file,
-// as one can when the store's .meta write failed or an eviction removed
-// the file after the read opened the variant, serves
-// application/octet-stream and keeps nothing, so metaCache only ever
-// holds a type read from a .meta file or passed to StoreVariant.
-func (c *Cache) loadVariantWithMeta(
-	cacheKey imagerequest.VariantKey,
-) (io.ReadCloser, int64, string, error) {
-	reader, size, contentType, err := c.variants.LoadWithMeta(cacheKey)
-	if err != nil {
-		return nil, 0, "", err
+// touchSourceContent hands the access time of a cached source a miss used to
+// the goroutine StartPendingAccessTimeWrites starts, which writes it in its
+// next batch. It never waits: while maxPendingAccessTimes access times of
+// sources wait to be written, this one is dropped, and the source only looks
+// colder to eviction.
+func (c *Cache) touchSourceContent(key imagerequest.SourceKey) {
+	select {
+	case c.pendingSourceAccessTimes <- key:
+	default:
 	}
-
-	if contentType == "" {
-		return reader, size, fallbackContentType, nil
-	}
-
-	c.metaCache.ContainsOrAdd(cacheKey, contentType)
-
-	return reader, size, contentType, nil
-}
-
-// touchVariant updates the LRU timestamp of a variant, best-effort:
-// a failed touch only makes the entry look colder to eviction.
-func (c *Cache) touchVariant(ctx context.Context, cacheKey imagerequest.VariantKey) {
-	_, err := c.db.ExecContext(ctx, `
-		UPDATE variant_content SET last_accessed_at = CURRENT_TIMESTAMP
-		WHERE cache_key = ?
-	`, string(cacheKey))
-	if err != nil {
-		c.log.Debug("failed to touch variant LRU timestamp",
-			"cache_key", cacheKey, "error", err)
-	}
-}
-
-// touchSourceContent updates the LRU timestamp of a cached source,
-// best-effort: a failed touch only makes the source look colder.
-func (c *Cache) touchSourceContent(ctx context.Context, key imagerequest.SourceKey) {
-	_, err := c.db.ExecContext(ctx, `
-		UPDATE source_content SET last_accessed_at = CURRENT_TIMESTAMP
-		WHERE source_key = ?
-	`, string(key))
-	if err != nil {
-		c.log.Debug("failed to touch source content LRU timestamp",
-			"source_key", key, "error", err)
-	}
-}
-
-// checkNegativeCache checks if a request is in the negative cache.
-func (c *Cache) checkNegativeCache(
-	ctx context.Context, req *imagerequest.Request,
-) (bool, error) {
-	var expiresAt time.Time
-
-	err := c.db.QueryRowContext(ctx, `
-		SELECT expires_at FROM negative_cache
-		WHERE source_host = ? AND source_path = ? AND source_query = ?
-	`, req.Source.Host, req.Source.Path, req.Source.Query).Scan(&expiresAt)
-
-	if errors.Is(err, sql.ErrNoRows) {
-		return false, nil
-	}
-
-	if err != nil {
-		return false, fmt.Errorf("failed to check negative cache: %w", err)
-	}
-
-	// Check if expired
-	if time.Now().After(expiresAt) {
-		// Clean up expired entry
-		_, err = c.db.ExecContext(ctx, `
-			DELETE FROM negative_cache
-			WHERE source_host = ? AND source_path = ? AND source_query = ?
-		`, req.Source.Host, req.Source.Path, req.Source.Query)
-		if err != nil {
-			c.log.Warn("failed to delete expired negative cache entry",
-				"host", req.Source.Host, "path", req.Source.Path, "error", err)
-		}
-
-		return false, nil
-	}
-
-	return true, nil
 }
 
 // cleanExpiredBatch deletes up to cleanExpiredBatchSize negative cache

internal/imgcache/cache_hit_internal_test.go

@@ -0,0 +1,214 @@
+package imgcache
+
+import (
+	"bytes"
+	"errors"
+	"fmt"
+	"image/color"
+	"io"
+	"testing"
+	"time"
+
+	"sneak.berlin/go/pixa/internal/imageprocessor"
+	"sneak.berlin/go/pixa/internal/imagerequest"
+	"sneak.berlin/go/pixa/internal/magic"
+)
+
+// TestVariant_ContentTypeFromFirstBytes stores an image in each output
+// format as a variant and checks that Variant serves it whole with the
+// content type of its format, read from the file.
+func TestVariant_ContentTypeFromFirstBytes(t *testing.T) {
+	t.Parallel()
+
+	cache, _ := newEvictionTestCache(t, 1<<30)
+	processor := imageprocessor.New(imageprocessor.Params{
+		MaxInputBytes:           50 << 20,
+		MaxConcurrentProcessing: 1,
+	})
+	source := generateTestJPEG(t, 40, 30, color.RGBA{200, 40, 40, 255})
+
+	for _, format := range []imagerequest.Format{
+		imagerequest.FormatJPEG, imagerequest.FormatPNG, imagerequest.FormatGIF,
+		imagerequest.FormatWebP, imagerequest.FormatAVIF, imagerequest.FormatJXL,
+	} {
+		result, err := processor.Process(t.Context(), bytes.NewReader(source),
+			&imagerequest.Transform{
+				Size:    imagerequest.Size{Width: 20, Height: 15},
+				Format:  format,
+				Quality: 85,
+				FitMode: imagerequest.FitCover,
+			})
+		if err != nil {
+			t.Fatalf("making a %s image: %v", format, err)
+		}
+
+		image, err := io.ReadAll(result.Content)
+		if err != nil {
+			t.Fatalf("reading the %s image: %v", format, err)
+		}
+
+		key := imagerequest.VariantKey("aabbccdd10" + string(format))
+
+		err = cache.PutVariant(t.Context(), key, bytes.NewReader(image))
+		if err != nil {
+			t.Fatalf("PutVariant(%s) error = %v", format, err)
+		}
+
+		reader, size, contentType, err := cache.Variant(key)
+		if err != nil {
+			t.Fatalf("Variant(%s) error = %v", format, err)
+		}
+
+		served, err := io.ReadAll(reader)
+		_ = reader.Close()
+
+		if err != nil {
+			t.Fatalf("reading the %s variant: %v", format, err)
+		}
+
+		t.Logf("%s: %d bytes served as %s", format, size, contentType)
+
+		if want := imageprocessor.FormatToMIME(format); contentType != want {
+			t.Errorf("%s variant content type = %q, want %q", format, contentType, want)
+		}
+
+		if size != int64(len(image)) || !bytes.Equal(served, image) {
+			t.Errorf("%s variant served %d bytes, size %d, want the %d bytes stored",
+				format, len(served), size, len(image))
+		}
+	}
+}
+
+// TestVariant_RefusesFileOfNoOutputFormat checks that Variant refuses
+// a variant file whose first bytes are not those of an output format, with
+// an error other than ErrNotFound, and that a variant with no file, or any
+// variant of a disabled cache, is ErrNotFound.
+func TestVariant_RefusesFileOfNoOutputFormat(t *testing.T) {
+	t.Parallel()
+
+	cache, _ := newEvictionTestCache(t, 1<<30)
+	jpegImage := generateTestJPEG(t, 10, 10, color.RGBA{0, 0, 255, 255})
+
+	tests := []struct {
+		name    string
+		content []byte
+		wantErr error
+	}{
+		{"text", []byte("this is text, not an image"), magic.ErrUnknownFormat},
+		{"SVG", []byte(`<svg xmlns="http://www.w3.org/2000/svg"></svg>`),
+			errNotAnOutputFormat},
+		{"JPEG cut to 10 bytes", jpegImage[:10], io.EOF},
+		{"empty", nil, io.EOF},
+	}
+
+	for i, tt := range tests {
+		key := imagerequest.VariantKey(fmt.Sprintf("aabbccdd20%02d", i))
+
+		_, err := cache.variants.Store(key, bytes.NewReader(tt.content))
+		if err != nil {
+			t.Fatalf("%s: storing the variant file: %v", tt.name, err)
+		}
+
+		reader, _, contentType, err := cache.Variant(key)
+		t.Logf("%s: Variant() error = %v", tt.name, err)
+
+		if reader != nil || contentType != "" || !errors.Is(err, tt.wantErr) {
+			t.Errorf("%s: Variant() = reader %v, content type %q, error %v; "+
+				"want no reader and %v", tt.name, reader, contentType, err, tt.wantErr)
+		}
+	}
+
+	reader, _, _, err := cache.Variant(testVariantKeyFour)
+	if reader != nil || !errors.Is(err, ErrNotFound) {
+		t.Errorf("Variant() of a variant with no file = reader %v, error %v; "+
+			"want no reader and %v", reader, err, ErrNotFound)
+	}
+
+	disabled, _ := newEvictionTestCache(t, 0)
+
+	reader, _, _, err = disabled.Variant(testVariantKeyOne)
+	if reader != nil || !errors.Is(err, ErrNotFound) {
+		t.Errorf("Variant() on a disabled cache = reader %v, error %v; "+
+			"want no reader and %v", reader, err, ErrNotFound)
+	}
+}
+
+// TestTouchVariantAndTouchSourceContent_DropAccessTimesPastTheLimit hands one
+// more access time of a variant, and one more of a source, than may wait to
+// be written. touchVariant and touchSourceContent must return each time
+// without waiting, and the last of each must be dropped.
+func TestTouchVariantAndTouchSourceContent_DropAccessTimesPastTheLimit(t *testing.T) {
+	t.Parallel()
+
+	cache, _ := newEvictionTestCache(t, 1<<30)
+
+	done := make(chan struct{})
+
+	go func() {
+		defer close(done)
+
+		for range maxPendingAccessTimes + 1 {
+			cache.touchVariant(testVariantKeyOne)
+			cache.touchSourceContent("aabbccdd3001")
+		}
+	}()
+
+	select {
+	case <-done:
+	case <-time.After(pendingAccessTimesWait):
+		t.Fatal("touchVariant or touchSourceContent waited once the access " +
+			"times reached the limit")
+	}
+
+	if waiting := len(cache.pendingVariantAccessTimes); waiting != maxPendingAccessTimes {
+		t.Errorf("%d access times of variants wait to be written, want %d",
+			waiting, maxPendingAccessTimes)
+	}
+
+	if waiting := len(cache.pendingSourceAccessTimes); waiting != maxPendingAccessTimes {
+		t.Errorf("%d access times of sources wait to be written, want %d",
+			waiting, maxPendingAccessTimes)
+	}
+}
+
+// TestStopPendingAccessTimeWrites_WritesPendingAccessTimes hands the access
+// time of one of two variants, three times, then stops the goroutine that
+// writes them, which never started. The stop must write that access time and
+// leave the other variant's.
+func TestStopPendingAccessTimeWrites_WritesPendingAccessTimes(t *testing.T) {
+	t.Parallel()
+
+	cache, _ := newEvictionTestCache(t, 1<<30)
+	anHourAgo := time.Now().Add(-time.Hour)
+
+	for _, key := range []imagerequest.VariantKey{testVariantKeyOne, testVariantKeyTwo} {
+		storeEvictionTestVariant(t, cache, key, []byte("variant bytes"))
+		setVariantLastAccessed(t, cache, key, anHourAgo)
+	}
+
+	for range 3 {
+		cache.touchVariant(testVariantKeyOne)
+	}
+
+	err := cache.StopPendingAccessTimeWrites(t.Context())
+	if err != nil {
+		t.Fatalf("StopPendingAccessTimeWrites() error = %v", err)
+	}
+
+	for key, wantTouched := range map[imagerequest.VariantKey]int{
+		testVariantKeyOne: 1, testVariantKeyTwo: 0,
+	} {
+		touched := countRows(t, cache, `
+			SELECT COUNT(*) FROM variant_content
+			WHERE cache_key = ? AND last_accessed_at > ?
+		`, string(key), anHourAgo.UTC().Format(sqliteTimestampFormat))
+		if touched != wantTouched {
+			t.Errorf("variant %s: %d rows with a new access time, want %d",
+				key, touched, wantTouched)
+		}
+	}
+
+	if waiting := len(cache.pendingVariantAccessTimes); waiting != 0 {
+		t.Errorf("%d access times still wait to be written, want 0", waiting)
+	}
+}

internal/imgcache/cache_internal_test.go

@@ -1,16 +1,14 @@
 package imgcache
 
 import (
-	"bytes"
 	"context"
 	"database/sql"
 	"os"
 	"testing"
 	"time"
 
+	"github.com/prometheus/client_golang/prometheus"
 	_ "modernc.org/sqlite"
-	"sneak.berlin/go/pixa/internal/httpfetcher"
-	"sneak.berlin/go/pixa/internal/imagerequest"
 )
 
 func setupTestDB(t *testing.T) *sql.DB {
@@ -25,54 +23,17 @@ func setupTestDB(t *testing.T) *sql.DB {
 	schema := `
 		CREATE TABLE source_content (
 			source_key TEXT PRIMARY KEY,
-			source_host TEXT NOT NULL,
-			source_path TEXT NOT NULL,
-			source_query TEXT NOT NULL DEFAULT '',
-			content_type TEXT NOT NULL,
 			size_bytes INTEGER NOT NULL,
-			fetched_at DATETIME DEFAULT CURRENT_TIMESTAMP,
 			last_accessed_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP
 		);
-		CREATE TABLE output_content (
-			content_hash TEXT PRIMARY KEY,
-			content_type TEXT NOT NULL,
-			size_bytes INTEGER NOT NULL,
-			fetched_at DATETIME DEFAULT CURRENT_TIMESTAMP
-		);
-		CREATE TABLE request_cache (
-			id INTEGER PRIMARY KEY AUTOINCREMENT,
-			cache_key TEXT NOT NULL UNIQUE,
-			source_metadata_id INTEGER NOT NULL,
-			output_hash TEXT NOT NULL,
-			width INTEGER NOT NULL,
-			height INTEGER NOT NULL,
-			format TEXT NOT NULL,
-			quality INTEGER NOT NULL DEFAULT 85,
-			fit_mode TEXT NOT NULL DEFAULT 'cover',
-			fetched_at DATETIME DEFAULT CURRENT_TIMESTAMP,
-			access_count INTEGER NOT NULL DEFAULT 1
-		);
 		CREATE TABLE negative_cache (
 			id INTEGER PRIMARY KEY AUTOINCREMENT,
 			source_host TEXT NOT NULL,
 			source_path TEXT NOT NULL,
 			source_query TEXT NOT NULL DEFAULT '',
-			status_code INTEGER NOT NULL,
-			error_message TEXT,
-			fetched_at DATETIME DEFAULT CURRENT_TIMESTAMP,
 			expires_at DATETIME NOT NULL,
 			UNIQUE(source_host, source_path, source_query)
 		);
-		CREATE TABLE cache_stats (
-			id INTEGER PRIMARY KEY CHECK (id = 1),
-			hit_count INTEGER NOT NULL DEFAULT 0,
-			miss_count INTEGER NOT NULL DEFAULT 0,
-			upstream_fetch_count INTEGER NOT NULL DEFAULT 0,
-			upstream_fetch_bytes INTEGER NOT NULL DEFAULT 0,
-			transform_count INTEGER NOT NULL DEFAULT 0,
-			last_updated_at DATETIME DEFAULT CURRENT_TIMESTAMP
-		);
-		INSERT INTO cache_stats (id) VALUES (1);
 	`
 
 	_, err = db.ExecContext(t.Context(), schema)
@@ -83,407 +44,25 @@ func setupTestDB(t *testing.T) *sql.DB {
 	return db
 }
 
-func setupTestCache(t *testing.T) *Cache {
-	t.Helper()
-
-	tmpDir := t.TempDir()
-	db := setupTestDB(t)
-
-	cache, err := NewCache(db, CacheConfig{
-		StateDir:    tmpDir,
-		NegativeTTL: 5 * time.Minute,
-	})
-	if err != nil {
-		t.Fatalf("failed to create cache: %v", err)
-	}
-
-	return cache
-}
-
-func TestCache_LookupMiss(t *testing.T) {
-	t.Parallel()
-
-	cache := setupTestCache(t)
-	ctx := context.Background()
-
-	req := &imagerequest.Request{
-		Source: imagerequest.Source{
-			Host: testHostCDN,
-			Path: testPathCat,
-		},
-		Transform: imagerequest.Transform{
-			Size:    imagerequest.Size{Width: 800, Height: 600},
-			Format:  imagerequest.FormatWebP,
-			Quality: 85,
-			FitMode: imagerequest.FitCover,
-		},
-	}
-
-	result, err := cache.Lookup(ctx, req)
-	if err != nil {
-		t.Fatalf("Lookup() error = %v", err)
-	}
-
-	if result.Hit {
-		t.Error("Lookup() hit = true, want false for cache miss")
-	}
-
-	if result.CacheStatus != CacheMiss {
-		t.Errorf("Lookup() status = %v, want %v", result.CacheStatus, CacheMiss)
-	}
-}
-
-func TestCache_StoreAndLookup(t *testing.T) {
-	t.Parallel()
-
-	cache := setupTestCache(t)
-	ctx := context.Background()
-
-	req := &imagerequest.Request{
-		Source: imagerequest.Source{
-			Host: testHostCDN,
-			Path: testPathCat,
-		},
-		Transform: imagerequest.Transform{
-			Size:    imagerequest.Size{Width: 800, Height: 600},
-			Format:  imagerequest.FormatWebP,
-			Quality: 85,
-			FitMode: imagerequest.FitCover,
-		},
-	}
-
-	// Store source content
-	sourceContent := []byte("fake jpeg data")
-	fetchResult := &httpfetcher.FetchResult{
-		ContentType: testContentTypeJPEG,
-		Headers:     map[string][]string{testHeaderContentType: {testContentTypeJPEG}},
-	}
-
-	sourceKey, err := cache.StoreSource(
-		ctx, req, bytes.NewReader(sourceContent), fetchResult)
-	if err != nil {
-		t.Fatalf("StoreSource() error = %v", err)
-	}
-
-	if sourceKey == "" {
-		t.Error("StoreSource() returned empty key")
-	}
-
-	// Store variant
-	cacheKey := imagerequest.CacheKey(req)
-	outputContent := []byte("fake webp data")
-
-	err = cache.StoreVariant(
-		t.Context(), cacheKey, bytes.NewReader(outputContent), "image/webp")
-	if err != nil {
-		t.Fatalf("StoreVariant() error = %v", err)
-	}
-
-	// Now lookup should hit
-	result, err := cache.Lookup(ctx, req)
-	if err != nil {
-		t.Fatalf("Lookup() error = %v", err)
-	}
-
-	if !result.Hit {
-		t.Error("Lookup() hit = false, want true after store")
-	}
-
-	if result.CacheStatus != CacheHit {
-		t.Errorf("Lookup() status = %v, want %v", result.CacheStatus, CacheHit)
-	}
-
-	if result.CacheKey == "" {
-		t.Error("Lookup() returned empty cache key")
-	}
-}
-
-func TestCache_NegativeCache(t *testing.T) {
-	t.Parallel()
-
-	cache := setupTestCache(t)
-	ctx := context.Background()
-
-	req := &imagerequest.Request{
-		Source: imagerequest.Source{
-			Host: testHostCDN,
-			Path: "/photos/notfound.jpg",
-		},
-		Transform: imagerequest.Transform{
-			Size:   imagerequest.Size{Width: 800, Height: 600},
-			Format: imagerequest.FormatWebP,
-		},
-	}
-
-	// Store negative cache entry
-	err := cache.StoreNegative(ctx, req, 404, "not found")
-	if err != nil {
-		t.Fatalf("StoreNegative() error = %v", err)
-	}
-
-	// Lookup should return miss (negative cache is checked at service level)
-	result, err := cache.Lookup(ctx, req)
-	if err != nil {
-		t.Fatalf("Lookup() error = %v", err)
-	}
-
-	if result.Hit {
-		t.Error("Lookup() hit = true, want false for negative cache entry")
-	}
-}
-
-func TestCache_NegativeCacheExpiry(t *testing.T) {
-	t.Parallel()
-
-	tmpDir := t.TempDir()
-	db := setupTestDB(t)
-
-	// Very short negative TTL for testing
-	cache, err := NewCache(db, CacheConfig{
-		StateDir:    tmpDir,
-		NegativeTTL: 1 * time.Millisecond,
-	})
-	if err != nil {
-		t.Fatalf("failed to create cache: %v", err)
-	}
-
-	ctx := context.Background()
-
-	req := &imagerequest.Request{
-		Source: imagerequest.Source{
-			Host: testHostCDN,
-			Path: "/photos/expired.jpg",
-		},
-		Transform: imagerequest.Transform{
-			Size:   imagerequest.Size{Width: 800, Height: 600},
-			Format: imagerequest.FormatWebP,
-		},
-	}
-
-	// Store negative cache entry
-	err = cache.StoreNegative(ctx, req, 404, "not found")
-	if err != nil {
-		t.Fatalf("StoreNegative() error = %v", err)
-	}
-
-	// Wait for expiry
-	time.Sleep(10 * time.Millisecond)
-
-	// Lookup should return miss after expiry
-	result, err := cache.Lookup(ctx, req)
-	if err != nil {
-		t.Fatalf("Lookup() error = %v, want nil after expiry", err)
-	}
-
-	if result.Hit {
-		t.Error("Lookup() hit = true, want false after negative cache expiry")
-	}
-}
-
-func TestCache_VariantLookup(t *testing.T) {
-	t.Parallel()
-
-	cache := setupTestCache(t)
-	ctx := context.Background()
-
-	req := &imagerequest.Request{
-		Source: imagerequest.Source{
-			Host: testHostCDN,
-			Path: "/photos/variant.jpg",
-		},
-		Transform: imagerequest.Transform{
-			Size:    imagerequest.Size{Width: 800, Height: 600},
-			Format:  imagerequest.FormatWebP,
-			Quality: 85,
-			FitMode: imagerequest.FitCover,
-		},
-	}
-
-	// Store variant
-	cacheKey := imagerequest.CacheKey(req)
-	outputContent := []byte("output data")
-
-	err := cache.StoreVariant(
-		t.Context(), cacheKey, bytes.NewReader(outputContent), "image/webp")
-	if err != nil {
-		t.Fatalf("StoreVariant() error = %v", err)
-	}
-
-	// First lookup
-	result1, err := cache.Lookup(ctx, req)
-	if err != nil {
-		t.Fatalf("Lookup() first error = %v", err)
-	}
-
-	if !result1.Hit {
-		t.Error("Lookup() first hit = false")
-	}
-
-	// Second lookup should also hit (from disk)
-	result2, err := cache.Lookup(ctx, req)
-	if err != nil {
-		t.Fatalf("Lookup() second error = %v", err)
-	}
-
-	if !result2.Hit {
-		t.Error("Lookup() second hit = false")
-	}
-
-	if result1.CacheKey != result2.CacheKey {
-		t.Error("Lookup() returned different cache keys")
-	}
-}
-
-func TestCache_GetVariant_ReturnsContentType(t *testing.T) {
-	t.Parallel()
-
-	cache := setupTestCache(t)
-	ctx := context.Background()
-
-	req := &imagerequest.Request{
-		Source: imagerequest.Source{
-			Host: testHostCDN,
-			Path: "/photos/variantct.jpg",
-		},
-		Transform: imagerequest.Transform{
-			Size:    imagerequest.Size{Width: 800, Height: 600},
-			Format:  imagerequest.FormatWebP,
-			Quality: 85,
-			FitMode: imagerequest.FitCover,
-		},
-	}
-
-	// Store variant
-	cacheKey := imagerequest.CacheKey(req)
-	outputContent := []byte("output webp data")
-
-	err := cache.StoreVariant(
-		t.Context(), cacheKey, bytes.NewReader(outputContent), "image/webp")
-	if err != nil {
-		t.Fatalf("StoreVariant() error = %v", err)
-	}
-
-	// Lookup
-	result, err := cache.Lookup(ctx, req)
-	if err != nil {
-		t.Fatalf("Lookup() error = %v", err)
-	}
-
-	if !result.Hit {
-		t.Fatal("Lookup() hit = false, want true")
-	}
-
-	// GetVariant should return content type
-	reader, size, contentType, err := cache.GetVariant(result.CacheKey)
-	if err != nil {
-		t.Fatalf("GetVariant() error = %v", err)
-	}
-
-	defer func() { _ = reader.Close() }()
-
-	if contentType != "image/webp" {
-		t.Errorf("GetVariant() ContentType = %q, want %q", contentType, "image/webp")
-	}
-
-	if size != int64(len(outputContent)) {
-		t.Errorf("GetVariant() size = %d, want %d", size, len(outputContent))
-	}
-}
-
-func TestCache_GetVariant(t *testing.T) {
-	t.Parallel()
-
-	cache := setupTestCache(t)
-	ctx := context.Background()
-
-	req := &imagerequest.Request{
-		Source: imagerequest.Source{
-			Host: testHostCDN,
-			Path: "/photos/output.jpg",
-		},
-		Transform: imagerequest.Transform{
-			Size:    imagerequest.Size{Width: 800, Height: 600},
-			Format:  imagerequest.FormatWebP,
-			Quality: 85,
-			FitMode: imagerequest.FitCover,
-		},
-	}
-
-	// Store variant
-	cacheKey := imagerequest.CacheKey(req)
-	outputContent := []byte("the actual output content")
-
-	err := cache.StoreVariant(
-		t.Context(), cacheKey, bytes.NewReader(outputContent), "image/webp")
-	if err != nil {
-		t.Fatalf("StoreVariant() error = %v", err)
-	}
-
-	// Lookup to get cache key
-	result, err := cache.Lookup(ctx, req)
-	if err != nil {
-		t.Fatalf("Lookup() error = %v", err)
-	}
-
-	// Get variant content
-	reader, _, _, err := cache.GetVariant(result.CacheKey)
-	if err != nil {
-		t.Fatalf("GetVariant() error = %v", err)
-	}
-
-	defer func() { _ = reader.Close() }()
-
-	buf := make([]byte, 100)
-	n, _ := reader.Read(buf)
-
-	if !bytes.Equal(buf[:n], outputContent) {
-		t.Errorf("GetVariant() content = %q, want %q", buf[:n], outputContent)
-	}
-}
-
-func TestCache_Stats(t *testing.T) {
-	t.Parallel()
-
-	cache := setupTestCache(t)
-	ctx := context.Background()
-
-	// Increment some stats
-	cache.IncrementStats(ctx, true, 0)
-	cache.IncrementStats(ctx, true, 0)
-	cache.IncrementStats(ctx, false, 1024)
-
-	stats, err := cache.Stats(ctx)
-	if err != nil {
-		t.Fatalf("Stats() error = %v", err)
-	}
-
-	if stats.HitCount != 2 {
-		t.Errorf("Stats() HitCount = %d, want 2", stats.HitCount)
-	}
-
-	if stats.MissCount != 1 {
-		t.Errorf("Stats() MissCount = %d, want 1", stats.MissCount)
-	}
-}
-
 func TestCache_CleanExpired(t *testing.T) {
 	t.Parallel()
 
 	tmpDir := t.TempDir()
 	db := setupTestDB(t)
 
-	cache, _ := NewCache(db, CacheConfig{
+	cache, _ := NewCache(db, Config{
 		StateDir:    tmpDir,
 		NegativeTTL: 1 * time.Millisecond,
-	})
+		MaxBytes:    1 << 30,
+	}, prometheus.NewRegistry())
 
 	ctx := context.Background()
 
 	// Insert expired negative cache entry directly
 	_, err := db.ExecContext(ctx, `
 		INSERT INTO negative_cache
-			(source_host, source_path, source_query, status_code, expires_at)
-		VALUES ('example.com', '/old.jpg', '', 404, datetime('now', '-1 hour'))
+			(source_host, source_path, source_query, expires_at)
+		VALUES ('example.com', '/old.jpg', '', datetime('now', '-1 hour'))
 	`)
 	if err != nil {
 		t.Fatalf("failed to insert test data: %v", err)
@@ -524,9 +103,10 @@ func TestCache_StorageDirectoriesCreated(t *testing.T) {
 	tmpDir := t.TempDir()
 	db := setupTestDB(t)
 
-	_, err := NewCache(db, CacheConfig{
+	_, err := NewCache(db, Config{
 		StateDir: tmpDir,
-	})
+		MaxBytes: 1 << 30,
+	}, prometheus.NewRegistry())
 	if err != nil {
 		t.Fatalf("NewCache() error = %v", err)
 	}

internal/imgcache/cachesize.go

@@ -1,85 +0,0 @@
-package imgcache
-
-import (
-	"context"
-	"errors"
-	"fmt"
-	"math"
-	"path/filepath"
-	"syscall"
-)
-
-// DefaultCacheMaxBytesFloor is the minimum computed default for the
-// cache_max_bytes setting: 500 MiB. The floor applies only to the
-// computed default (when the key is omitted from the configuration),
-// never to explicitly configured values.
-const DefaultCacheMaxBytesFloor int64 = 524288000
-
-// freeSpaceFractionNumerator and freeSpaceFractionDenominator express
-// the 75% share used for the computed default limit as integer
-// arithmetic (dividing before multiplying avoids overflow).
-const (
-	freeSpaceFractionNumerator   uint64 = 3
-	freeSpaceFractionDenominator uint64 = 4
-)
-
-var errNegativeBlockSize = errors.New("statfs reported negative block size")
-
-// FreeSpaceProbeFunc reports the number of free bytes available on the
-// filesystem containing path. It is a function type so tests can
-// inject a fake probe instead of depending on the host disk.
-type FreeSpaceProbeFunc func(path string) (uint64, error)
-
-// defaultFreeSpaceProbe reports free filesystem bytes via statfs on
-// the given path, as available to unprivileged processes.
-func defaultFreeSpaceProbe(path string) (uint64, error) {
-	var stat syscall.Statfs_t
-
-	err := syscall.Statfs(path, &stat)
-	if err != nil {
-		return 0, err
-	}
-
-	if stat.Bsize < 0 {
-		return 0, fmt.Errorf("%w %d for %q", errNegativeBlockSize, stat.Bsize, path)
-	}
-
-	blockSize := uint64(stat.Bsize)
-
-	return stat.Bavail * blockSize, nil
-}
-
-// computeDefaultMaxBytes returns the default cache size limit: 75% of
-// the sum of the free bytes probe reports for <state_dir>/cache/ and
-// the bytes the cache already holds, with a floor of
-// DefaultCacheMaxBytesFloor. Counting what the cache holds keeps the
-// limit from shrinking as the cache fills.
-func (c *Cache) computeDefaultMaxBytes(
-	ctx context.Context, probe FreeSpaceProbeFunc,
-) (int64, error) {
-	cacheDir := filepath.Join(c.config.StateDir, "cache")
-
-	freeBytes, err := probe(cacheDir)
-	if err != nil {
-		return 0, fmt.Errorf(
-			"default cache_max_bytes: cannot determine free space for %q: %w",
-			cacheDir, err)
-	}
-
-	usedBytes, err := c.UsageBytes(ctx)
-	if err != nil {
-		return 0, err
-	}
-
-	// Both terms are at most math.MaxInt64, so the sum cannot overflow.
-	//nolint:gosec // G115: UsageBytes returns the total cache usage, never negative
-	spaceBytes := min(freeBytes, math.MaxInt64) + uint64(usedBytes)
-
-	computed := spaceBytes / freeSpaceFractionDenominator * freeSpaceFractionNumerator
-	computed = min(computed, math.MaxInt64)
-
-	limit := int64(computed)
-	limit = max(limit, DefaultCacheMaxBytesFloor)
-
-	return limit, nil
-}

internal/imgcache/cachesize_internal_test.go

@@ -1,188 +0,0 @@
-package imgcache
-
-import (
-	"errors"
-	"os"
-	"path/filepath"
-	"strings"
-	"testing"
-)
-
-// Static errors returned by the stub free-space probes below.
-var (
-	errTestStatfsFailed     = errors.New("statfs failed")
-	errTestProbeNotExpected = errors.New("probe must not be called")
-)
-
-// TestComputeDefaultMaxBytesCountsWhatTheCacheHolds verifies that the
-// default limit is 75% of the free space plus what the cache already
-// holds, so a cache filled to its limit keeps that limit across a
-// restart instead of shrinking to 75% of the space left free.
-func TestComputeDefaultMaxBytesCountsWhatTheCacheHolds(t *testing.T) {
-	t.Parallel()
-
-	cache, _ := newEvictionTestCache(t, 1<<30)
-
-	// Empty cache, 4 GiB free -> 3 GiB default.
-	got, err := cache.computeDefaultMaxBytes(t.Context(),
-		func(string) (uint64, error) { return 4294967296, nil })
-	if err != nil {
-		t.Fatalf("computeDefaultMaxBytes returned error: %v", err)
-	}
-
-	t.Logf("default for an empty cache with 4 GiB free: %d", got)
-
-	if got != 3221225472 {
-		t.Errorf("default for an empty cache = %d, want 3221225472 (75%% of 4 GiB)",
-			got)
-	}
-
-	// The cache now holds those 3 GiB, which leaves 1 GiB free.
-	_, err = cache.db.ExecContext(t.Context(),
-		`INSERT INTO variant_content (cache_key, size_bytes, content_type)
-		 VALUES (?, ?, ?)`,
-		string(testVariantKeyOne), 3221225472, testContentTypeWebP,
-	)
-	if err != nil {
-		t.Fatalf("failed to insert variant accounting row: %v", err)
-	}
-
-	got, err = cache.computeDefaultMaxBytes(t.Context(),
-		func(string) (uint64, error) { return 1073741824, nil })
-	if err != nil {
-		t.Fatalf("computeDefaultMaxBytes returned error: %v", err)
-	}
-
-	t.Logf("default for a cache holding 3 GiB with 1 GiB free: %d", got)
-
-	if got != 3221225472 {
-		t.Errorf("default for a cache holding 3 GiB with 1 GiB free = %d, "+
-			"want 3221225472 (75%% of 1 GiB + 3 GiB)", got)
-	}
-}
-
-// TestComputeDefaultMaxBytesAppliesFloor verifies that when 75% of the
-// free space plus what the cache holds is below 500 MiB, the default
-// is floored at DefaultCacheMaxBytesFloor.
-func TestComputeDefaultMaxBytesAppliesFloor(t *testing.T) {
-	t.Parallel()
-
-	cases := []struct {
-		name      string
-		freeBytes uint64
-	}{
-		{name: "100 MiB free", freeBytes: 104857600},
-		{name: "zero free", freeBytes: 0},
-		{name: "just below floor threshold", freeBytes: 699050665},
-	}
-
-	for _, tc := range cases {
-		t.Run(tc.name, func(t *testing.T) {
-			t.Parallel()
-
-			cache, _ := newEvictionTestCache(t, 1<<30)
-
-			got, err := cache.computeDefaultMaxBytes(t.Context(),
-				func(string) (uint64, error) { return tc.freeBytes, nil })
-			if err != nil {
-				t.Fatalf("computeDefaultMaxBytes returned error: %v", err)
-			}
-
-			if got != DefaultCacheMaxBytesFloor {
-				t.Errorf("computeDefaultMaxBytes = %d, want floor %d",
-					got, DefaultCacheMaxBytesFloor)
-			}
-		})
-	}
-}
-
-// TestComputeDefaultMaxBytesPropagatesProbeError verifies that a
-// failing free-space probe produces an error naming cache_max_bytes,
-// instead of a silently wrong default.
-func TestComputeDefaultMaxBytesPropagatesProbeError(t *testing.T) {
-	t.Parallel()
-
-	cache, _ := newEvictionTestCache(t, 1<<30)
-
-	_, err := cache.computeDefaultMaxBytes(t.Context(),
-		func(string) (uint64, error) { return 0, errTestStatfsFailed })
-	if err == nil {
-		t.Fatal("probe failure must produce an error, got nil")
-	}
-
-	t.Logf("got expected error: %v", err)
-
-	if !strings.Contains(err.Error(), "cache_max_bytes") {
-		t.Errorf("error %q does not name cache_max_bytes", err.Error())
-	}
-}
-
-// TestNewCacheComputesDefaultMaxBytesWhenAsked verifies that with
-// UseDefaultMaxBytes set, the cache's limit becomes the computed
-// default, and that the probe is pointed at <state_dir>/cache/, which
-// must be created first so statfs measures the right filesystem.
-func TestNewCacheComputesDefaultMaxBytesWhenAsked(t *testing.T) {
-	t.Parallel()
-
-	stateDir := t.TempDir()
-	wantCacheDir := filepath.Join(stateDir, "cache")
-
-	var probedPath string
-
-	// 4 GiB free -> 3 GiB default.
-	probe := func(path string) (uint64, error) {
-		probedPath = path
-
-		info, err := os.Stat(path)
-		if err != nil || !info.IsDir() {
-			t.Errorf("cache directory %q was not created before probing: info=%v err=%v",
-				path, info, err)
-		}
-
-		return 4294967296, nil
-	}
-
-	cache, err := newCache(evictionTestDB(t), CacheConfig{
-		StateDir:           stateDir,
-		UseDefaultMaxBytes: true,
-	}, probe)
-	if err != nil {
-		t.Fatalf("newCache returned error: %v", err)
-	}
-
-	if cache.config.MaxBytes != 3221225472 {
-		t.Errorf("MaxBytes = %d, want computed default 3221225472",
-			cache.config.MaxBytes)
-	}
-
-	if probedPath != wantCacheDir {
-		t.Errorf("free space probed at %q, want cache directory %q",
-			probedPath, wantCacheDir)
-	}
-}
-
-// TestNewCacheKeepsExplicitMaxBytes verifies that without
-// UseDefaultMaxBytes the cache keeps MaxBytes exactly as given and
-// never consults the free-space probe.
-func TestNewCacheKeepsExplicitMaxBytes(t *testing.T) {
-	t.Parallel()
-
-	probe := func(string) (uint64, error) {
-		t.Error("free-space probe must not be consulted for explicit values")
-
-		return 0, errTestProbeNotExpected
-	}
-
-	cache, err := newCache(evictionTestDB(t), CacheConfig{
-		StateDir: t.TempDir(),
-		MaxBytes: 1024,
-	}, probe)
-	if err != nil {
-		t.Fatalf("newCache returned error: %v", err)
-	}
-
-	if cache.config.MaxBytes != 1024 {
-		t.Errorf("MaxBytes = %d, want explicit 1024 (no floor, no recompute)",
-			cache.config.MaxBytes)
-	}
-}

internal/imgcache/counters.go

@@ -0,0 +1,45 @@
+package imgcache
+
+import (
+	"fmt"
+
+	"github.com/prometheus/client_golang/prometheus"
+)
+
+// registerCounters makes the cache's counters and registers them on
+// metricsRegistry. The counters are kept in memory, so adding to one waits
+// for nothing and makes no database call. It fails when metricsRegistry
+// already holds counters of the same names.
+func (c *Cache) registerCounters(metricsRegistry prometheus.Registerer) error {
+	c.cacheHits = prometheus.NewCounter(prometheus.CounterOpts{
+		Name: "pixa_cache_hits_total",
+		Help: "Image requests served from the cache.",
+	})
+	c.cacheMisses = prometheus.NewCounter(prometheus.CounterOpts{
+		Name: "pixa_cache_misses_total",
+		Help: "Image requests not served from the cache.",
+	})
+	c.upstreamFetches = prometheus.NewCounter(prometheus.CounterOpts{
+		Name: "pixa_upstream_fetches_total",
+		Help: "Fetches of a source image from upstream that read any bytes.",
+	})
+	c.upstreamFetchBytes = prometheus.NewCounter(prometheus.CounterOpts{
+		Name: "pixa_upstream_fetch_bytes_total",
+		Help: "Bytes read from upstream by fetches of source images.",
+	})
+	c.transforms = prometheus.NewCounter(prometheus.CounterOpts{
+		Name: "pixa_transforms_total",
+		Help: "Images the image processor transformed.",
+	})
+
+	for _, counter := range []prometheus.Counter{
+		c.cacheHits, c.cacheMisses, c.upstreamFetches, c.upstreamFetchBytes, c.transforms,
+	} {
+		err := metricsRegistry.Register(counter)
+		if err != nil {
+			return fmt.Errorf("failed to register the cache's counters: %w", err)
+		}
+	}
+
+	return nil
+}

internal/imgcache/counters_internal_test.go

@@ -0,0 +1,67 @@
+package imgcache
+
+import (
+	"errors"
+	"testing"
+
+	"github.com/prometheus/client_golang/prometheus"
+	"github.com/prometheus/client_golang/prometheus/testutil"
+)
+
+// TestNewCache_RegistersItsCountersOnItsRegistry checks that NewCache
+// registers the cache's counters on the registry it is given, under the
+// names README.md gives them.
+func TestNewCache_RegistersItsCountersOnItsRegistry(t *testing.T) {
+	t.Parallel()
+
+	registry := prometheus.NewRegistry()
+
+	_, err := NewCache(evictionTestDB(t),
+		Config{StateDir: t.TempDir(), MaxBytes: 1 << 30}, registry)
+	if err != nil {
+		t.Fatalf("NewCache() error = %v", err)
+	}
+
+	names := []string{
+		"pixa_cache_hits_total",
+		"pixa_cache_misses_total",
+		"pixa_upstream_fetches_total",
+		"pixa_upstream_fetch_bytes_total",
+		"pixa_transforms_total",
+	}
+
+	count, err := testutil.GatherAndCount(registry, names...)
+	if err != nil {
+		t.Fatalf("testutil.GatherAndCount() error = %v", err)
+	}
+
+	if count != len(names) {
+		t.Errorf("%d of the counters %v are on the registry, want %d",
+			count, names, len(names))
+	}
+}
+
+// TestNewCache_FailsWhenItsCountersCannotBeRegistered makes a second cache
+// on the registry of a first. Its counters have the same names, so the
+// registry refuses them, and NewCache must return that error.
+func TestNewCache_FailsWhenItsCountersCannotBeRegistered(t *testing.T) {
+	t.Parallel()
+
+	registry := prometheus.NewRegistry()
+
+	_, err := NewCache(evictionTestDB(t),
+		Config{StateDir: t.TempDir(), MaxBytes: 1 << 30}, registry)
+	if err != nil {
+		t.Fatalf("NewCache() of the first cache error = %v", err)
+	}
+
+	_, err = NewCache(evictionTestDB(t),
+		Config{StateDir: t.TempDir(), MaxBytes: 1 << 30}, registry)
+	t.Logf("NewCache() of the second cache error = %v", err)
+
+	var alreadyRegistered prometheus.AlreadyRegisteredError
+	if !errors.As(err, &alreadyRegistered) {
+		t.Errorf("NewCache() of the second cache error = %v, want %T",
+			err, alreadyRegistered)
+	}
+}

internal/imgcache/eviction.go

@@ -3,7 +3,6 @@ package imgcache
 import (
 	"context"
 	"database/sql"
-	"encoding/json"
 	"errors"
 	"fmt"
 	"io/fs"
@@ -52,27 +51,26 @@ const sqliteTimestampLayout = "2006-01-02 15:04:05"
 // writes (".tmp-*" patterns in the storage layer).
 const tempFilePrefix = ".tmp-"
 
-// variantMetaSuffix is the sidecar suffix VariantStorage writes next
-// to each variant file.
-const variantMetaSuffix = ".meta"
-
-// fallbackContentType is the content type given to a variant file that
-// has no readable .meta sidecar, when it is served or reconciled.
-const fallbackContentType = "application/octet-stream"
-
 // UsageBytes returns the total number of bytes of cache content
-// tracked in the database (source content blobs plus processed
-// variants). It reads the total the database keeps up to date as rows
-// are added and removed, so it neither scans the cache directories nor
-// sums the tables.
+// tracked in the database, as the package function UsageBytes does, or 0
+// when the disk cache is off.
 func (c *Cache) UsageBytes(ctx context.Context) (int64, error) {
-	if c.disabled {
+	if c.config.MaxBytes == 0 {
 		return 0, nil
 	}
 
+	return UsageBytes(ctx, c.db)
+}
+
+// UsageBytes returns the total number of bytes of cache content tracked
+// in db (source content blobs plus processed variants). It reads the
+// total the database keeps up to date as rows are added and removed, so
+// it neither scans the cache directories nor sums the tables. pixad reads
+// it before it builds the cache, to work out the default cache_max_bytes.
+func UsageBytes(ctx context.Context, db *sql.DB) (int64, error) {
 	var total int64
 
-	err := c.db.QueryRowContext(ctx,
+	err := db.QueryRowContext(ctx,
 		`SELECT total_size_bytes FROM cache_usage WHERE id = 1`,
 	).Scan(&total)
 	if err != nil {
@@ -95,9 +93,9 @@ type evictionCandidate struct {
 
 // EvictToLimit evicts least-recently-used cache entries until total
 // tracked usage is at or below the configured MaxBytes limit. It is a
-// no-op when the cache is disabled or no limit is configured.
+// no-op when the cache is disabled.
 func (c *Cache) EvictToLimit(ctx context.Context) error {
-	if c.disabled || c.config.MaxBytes <= 0 {
+	if c.config.MaxBytes == 0 {
 		return nil
 	}
 
@@ -298,10 +296,7 @@ func (c *Cache) sourceCandidates(ctx context.Context) ([]evictionCandidate, erro
 }
 
 // evictVariant removes one variant: accounting row first, then the
-// content and .meta files, so the database never references a deleted
-// file. The metaCache entry goes before the files; a GetVariant that
-// read them just before may put it back, and the next GetVariant then
-// fails to open the file and removes it again.
+// file, so the database never references a deleted file.
 func (c *Cache) evictVariant(
 	ctx context.Context, cacheKey imagerequest.VariantKey,
 ) error {
@@ -311,8 +306,6 @@ func (c *Cache) evictVariant(
 		return fmt.Errorf("failed to delete variant accounting row: %w", err)
 	}
 
-	c.metaCache.Remove(cacheKey)
-
 	err = c.variants.Delete(cacheKey)
 	if err != nil {
 		return err
@@ -324,7 +317,7 @@ func (c *Cache) evictVariant(
 // evictSourceBlob removes one source: its row first, then its file, as
 // evictVariant does for a variant. It holds no lock: a store of the same
 // source in between can leave the source's row with no file, which
-// LookupSource treats as a miss.
+// Source treats as a miss.
 func (c *Cache) evictSourceBlob(ctx context.Context, key imagerequest.SourceKey) error {
 	_, err := c.db.ExecContext(ctx,
 		`DELETE FROM source_content WHERE source_key = ?`, string(key))
@@ -340,7 +333,7 @@ func (c *Cache) evictSourceBlob(ctx context.Context, key imagerequest.SourceKey)
 // storing request. The notification channel has capacity one and drops
 // when a wakeup is already pending.
 func (c *Cache) notifyWritePressure() {
-	if c.disabled || c.config.MaxBytes <= 0 {
+	if c.config.MaxBytes == 0 {
 		return
 	}
 
@@ -486,7 +479,7 @@ func (c *Cache) runCleanExpired(ctx context.Context) {
 // then on the first periodic eviction tick that comes at least ten times
 // as long after the last pass ended as that pass took, off the request
 // hot path: it adopts variant files that predate the accounting table
-// (or whose accounting insert failed, e.g. StoreVariant's best-effort
+// (or whose accounting insert failed, e.g. PutVariant's best-effort
 // insert under transient DB contention), drops variant accounting rows
 // whose files are missing, removes source blob files the database does
 // not know, removes source and variant files that are empty or not the
@@ -499,7 +492,7 @@ func (c *Cache) runCleanExpired(ctx context.Context) {
 // Once ctx is cancelled, it stops at the next file or row and returns
 // ctx's error.
 func (c *Cache) reconcileAccounting(ctx context.Context) error {
-	if c.disabled {
+	if c.config.MaxBytes == 0 {
 		return nil
 	}
 
@@ -521,12 +514,16 @@ func (c *Cache) reconcileAccounting(ctx context.Context) error {
 	return c.reconcileUsageTotal(ctx)
 }
 
-// reconcileVariantFiles walks the variant storage directory, checking
-// each variant file against its accounting row (reconcileVariantFile)
-// and sweeping stale temp files.
-func (c *Cache) reconcileVariantFiles(ctx context.Context) error {
+// reconcileFiles walks baseDir, one of the cache's storage directories,
+// sweeping stale temp files and passing each other file, with its name, to
+// reconcileFile. Once ctx is cancelled it stops at the next file and
+// returns ctx's error.
+func (c *Cache) reconcileFiles(
+	ctx context.Context, baseDir string,
+	reconcileFile func(path string, entry fs.DirEntry, name string) error,
+) error {
 	return filepath.WalkDir(
-		c.variants.baseDir,
+		baseDir,
 		func(path string, entry fs.DirEntry, err error) error {
 			if ctx.Err() != nil {
 				return ctx.Err()
@@ -544,15 +541,21 @@ func (c *Cache) reconcileVariantFiles(ctx context.Context) error {
 				return nil
 			}
 
-			if strings.HasSuffix(name, variantMetaSuffix) {
-				return nil
-			}
-
-			return c.reconcileVariantFile(ctx, path, entry, imagerequest.VariantKey(name))
+			return reconcileFile(path, entry, name)
 		},
 	)
 }
 
+// reconcileVariantFiles walks the variant storage directory, checking
+// each variant file against its accounting row (reconcileVariantFile)
+// and sweeping stale temp files.
+func (c *Cache) reconcileVariantFiles(ctx context.Context) error {
+	return c.reconcileFiles(ctx, c.variants.baseDir,
+		func(path string, entry fs.DirEntry, name string) error {
+			return c.reconcileVariantFile(ctx, path, entry, imagerequest.VariantKey(name))
+		})
+}
+
 // reconcileVariantFile checks one variant file against its accounting
 // row. A file without a row is adopted into the accounting, unless it is
 // empty. A file that is empty, or not the size its row records, as a
@@ -574,7 +577,7 @@ func (c *Cache) reconcileVariantFile(
 
 	switch {
 	case errors.Is(err, sql.ErrNoRows) && info.Size() > 0:
-		return c.adoptVariantFile(ctx, path, info, cacheKey)
+		return c.adoptVariantFile(ctx, info, cacheKey)
 	case err != nil && !errors.Is(err, sql.ErrNoRows):
 		return fmt.Errorf("failed to check variant accounting row: %w", err)
 	case info.Size() > 0 && info.Size() == recordedSize:
@@ -591,16 +594,14 @@ func (c *Cache) reconcileVariantFile(
 // adoptVariantFile inserts an accounting row for a variant file that
 // has none, using the file's size and modification time.
 func (c *Cache) adoptVariantFile(
-	ctx context.Context, path string, info fs.FileInfo, cacheKey imagerequest.VariantKey,
+	ctx context.Context, info fs.FileInfo, cacheKey imagerequest.VariantKey,
 ) error {
 	modTime := info.ModTime().UTC().Format(sqliteTimestampLayout)
-	contentType := c.variantContentTypeFromSidecar(path)
 
 	_, err := c.db.ExecContext(ctx, `
-		INSERT INTO variant_content
-			(cache_key, size_bytes, content_type, created_at, last_accessed_at)
-		VALUES (?, ?, ?, ?, ?)
-	`, string(cacheKey), info.Size(), contentType, modTime, modTime)
+		INSERT INTO variant_content (cache_key, size_bytes, last_accessed_at)
+		VALUES (?, ?, ?)
+	`, string(cacheKey), info.Size(), modTime)
 	if err != nil {
 		return fmt.Errorf("failed to adopt variant file into accounting: %w", err)
 	}
@@ -611,23 +612,6 @@ func (c *Cache) adoptVariantFile(
 	return nil
 }
 
-// variantContentTypeFromSidecar reads the content type from a variant
-// .meta sidecar, falling back to application/octet-stream.
-func (c *Cache) variantContentTypeFromSidecar(variantPath string) string {
-	//nolint:gosec // path from cache walk
-	metaData, err := os.ReadFile(variantPath + variantMetaSuffix)
-	if err != nil {
-		return fallbackContentType
-	}
-
-	var meta VariantMeta
-	if json.Unmarshal(metaData, &meta) != nil || meta.ContentType == "" {
-		return fallbackContentType
-	}
-
-	return meta.ContentType
-}
-
 // reconcileVariantRows drops accounting rows whose variant files are
 // missing, so the database never references deleted content.
 func (c *Cache) reconcileVariantRows(ctx context.Context) error {
@@ -719,28 +703,10 @@ func queryStringColumn[T ~string](
 // each blob file against its source_content row (reconcileSourceFile)
 // and sweeping stale temp files.
 func (c *Cache) reconcileSourceFiles(ctx context.Context) error {
-	return filepath.WalkDir(
-		c.srcContent.baseDir,
-		func(path string, entry fs.DirEntry, err error) error {
-			if ctx.Err() != nil {
-				return ctx.Err()
-			}
-
-			if err != nil || entry.IsDir() {
-				return err
-			}
-
-			name := entry.Name()
-
-			if strings.HasPrefix(name, tempFilePrefix) {
-				c.sweepStaleTempFile(path, entry)
-
-				return nil
-			}
-
+	return c.reconcileFiles(ctx, c.srcContent.baseDir,
+		func(path string, entry fs.DirEntry, name string) error {
 			return c.reconcileSourceFile(ctx, path, entry, imagerequest.SourceKey(name))
-		},
-	)
+		})
 }
 
 // reconcileSourceFile checks one source blob file against its

internal/imgcache/eviction_internal_test.go

@@ -14,9 +14,9 @@ import (
 	"testing"
 	"time"
 
+	"github.com/prometheus/client_golang/prometheus"
 	"modernc.org/sqlite"
 	"sneak.berlin/go/pixa/internal/database"
-	"sneak.berlin/go/pixa/internal/httpfetcher"
 	"sneak.berlin/go/pixa/internal/imagerequest"
 )
 
@@ -66,16 +66,12 @@ func newEvictionTestCache(t *testing.T, maxBytes int64) (*Cache, string) {
 	tmpDir := t.TempDir()
 	db := evictionTestDB(t)
 
-	// maxBytes zero mirrors the production mapping of
-	// cache_max_bytes: 0 (handlers sets DisableDiskCache); at the
-	// CacheConfig layer itself a zero MaxBytes means "no limit" for
-	// backwards compatibility with existing fixtures.
-	cache, err := NewCache(db, CacheConfig{
-		StateDir:         tmpDir,
-		NegativeTTL:      5 * time.Minute,
-		MaxBytes:         maxBytes,
-		DisableDiskCache: maxBytes == 0,
-	})
+	// maxBytes zero turns the disk cache off, as cache_max_bytes: 0 does.
+	cache, err := NewCache(db, Config{
+		StateDir:    tmpDir,
+		NegativeTTL: 5 * time.Minute,
+		MaxBytes:    maxBytes,
+	}, prometheus.NewRegistry())
 	if err != nil {
 		t.Fatalf("failed to create cache: %v", err)
 	}
@@ -102,20 +98,11 @@ func storeEvictionTestSource(
 		},
 	}
 
-	result := &httpfetcher.FetchResult{
-		StatusCode:    200,
-		ContentType:   testContentTypeJPEG,
-		ContentLength: int64(len(content)),
-		Headers: map[string][]string{
-			testHeaderContentType: {testContentTypeJPEG},
-		},
-	}
-
-	key, err := cache.StoreSource(
-		context.Background(), req, bytes.NewReader(content), result,
+	key, err := cache.PutSource(
+		context.Background(), req.Source, bytes.NewReader(content),
 	)
 	if err != nil {
-		t.Fatalf("StoreSource(%s%s) failed: %v", host, path, err)
+		t.Fatalf("PutSource(%s%s) failed: %v", host, path, err)
 	}
 
 	return key
@@ -128,9 +115,9 @@ func storeEvictionTestVariant(
 ) {
 	t.Helper()
 
-	err := cache.StoreVariant(t.Context(), key, bytes.NewReader(content), "image/webp")
+	err := cache.PutVariant(t.Context(), key, bytes.NewReader(content))
 	if err != nil {
-		t.Fatalf("StoreVariant(%s) failed: %v", key, err)
+		t.Fatalf("PutVariant(%s) failed: %v", key, err)
 	}
 }
 
@@ -436,27 +423,20 @@ func assertDisabledCacheWritesAreNoOps(
 
 	ctx := t.Context()
 
-	err := cache.StoreVariant(
-		ctx, imagerequest.CacheKey(req), bytes.NewReader([]byte("data")), "image/webp",
+	err := cache.PutVariant(
+		ctx, imagerequest.CacheKey(req), bytes.NewReader([]byte("data")),
 	)
 	if err != nil {
-		t.Fatalf("StoreVariant on disabled cache must be a no-op, got error: %v", err)
+		t.Fatalf("PutVariant on disabled cache must be a no-op, got error: %v", err)
 	}
 
-	result := &httpfetcher.FetchResult{
-		StatusCode:    200,
-		ContentType:   testContentTypeJPEG,
-		ContentLength: 4,
-		Headers:       map[string][]string{},
-	}
-
-	key, err := cache.StoreSource(ctx, req, bytes.NewReader([]byte("data")), result)
+	key, err := cache.PutSource(ctx, req.Source, bytes.NewReader([]byte("data")))
 	if err != nil {
-		t.Fatalf("StoreSource on disabled cache must be a no-op, got error: %v", err)
+		t.Fatalf("PutSource on disabled cache must be a no-op, got error: %v", err)
 	}
 
 	if key != "" {
-		t.Errorf("StoreSource on disabled cache returned key %q, want empty", key)
+		t.Errorf("PutSource on disabled cache returned key %q, want empty", key)
 	}
 }
 
@@ -469,23 +449,13 @@ func assertDisabledCacheReadsAlwaysMiss(
 
 	ctx := t.Context()
 
-	lookup, err := cache.Lookup(ctx, req)
-	if err != nil {
-		t.Fatalf("Lookup on disabled cache failed: %v", err)
+	reader, _, err := cache.Source(ctx, req.Source)
+	if !errors.Is(err, ErrNotFound) {
+		t.Fatalf("Source on disabled cache error = %v, want %v", err, ErrNotFound)
 	}
 
-	if lookup.Hit {
-		t.Error("Lookup on disabled cache must always miss")
-	}
-
-	key, srcType, err := cache.LookupSource(ctx, req)
-	if err != nil {
-		t.Fatalf("LookupSource on disabled cache failed: %v", err)
-	}
-
-	if key != "" || srcType != "" {
-		t.Errorf("LookupSource on disabled cache = (%q, %q), want empty",
-			key, srcType)
+	if reader != nil {
+		t.Errorf("Source on disabled cache = reader %v, want none", reader)
 	}
 }
 
@@ -574,7 +544,7 @@ func TestEvictionRunsUnderWritePressure(t *testing.T) {
 }
 
 // TestEvictionRunsOnPeriodicSchedule writes three variant files straight
-// to disk, bypassing StoreVariant, so they have no accounting rows and no
+// to disk, bypassing PutVariant, so they have no accounting rows and no
 // write-pressure notification fires. Only a periodic reconciliation pass
 // can then adopt them, and only the eviction pass that follows it can
 // evict them.
@@ -616,7 +586,7 @@ func TestEvictionRunsOnPeriodicSchedule(t *testing.T) {
 	for i, key := range keys {
 		content := bytes.Repeat([]byte{fills[i]}, 1000)
 
-		_, err = cache.variants.Store(key, bytes.NewReader(content), "image/webp")
+		_, err = cache.variants.Store(key, bytes.NewReader(content))
 		if err != nil {
 			t.Fatalf("failed to store variant file: %v", err)
 		}
@@ -668,7 +638,7 @@ func TestStartEvictionReconcilesAccountingWithDisk(t *testing.T) {
 	untracked := bytes.Repeat([]byte{0x31}, 1000)
 
 	_, err := cache.variants.Store(
-		testVariantKeyOne, bytes.NewReader(untracked), "image/webp",
+		testVariantKeyOne, bytes.NewReader(untracked),
 	)
 	if err != nil {
 		t.Fatalf("failed to store untracked variant file: %v", err)
@@ -676,9 +646,8 @@ func TestStartEvictionReconcilesAccountingWithDisk(t *testing.T) {
 
 	// An accounting row whose file is missing must be dropped.
 	_, err = cache.db.ExecContext(t.Context(),
-		`INSERT INTO variant_content (cache_key, size_bytes, content_type)
-		 VALUES (?, ?, ?)`,
-		"deadbeef0001", 700, "image/webp",
+		`INSERT INTO variant_content (cache_key, size_bytes) VALUES (?, ?)`,
+		"deadbeef0001", 700,
 	)
 	if err != nil {
 		t.Fatalf("failed to insert stale variant accounting row: %v", err)
@@ -728,7 +697,7 @@ func TestStartEvictionReconcilesAccountingWithDisk(t *testing.T) {
 // TestPeriodicReconciliationAdoptsFileThatAppearsAfterStartup proves
 // reconciliation is not a one-shot startup-only pass: it must also run
 // on the periodic ticker, so a variant file that lands on disk with no
-// accounting row well after startup (e.g. because StoreVariant's
+// accounting row well after startup (e.g. because PutVariant's
 // best-effort accounting insert failed under transient contention, or
 // any other cause of an untracked file appearing during steady-state
 // operation) is still adopted into accounting eventually, rather than
@@ -764,13 +733,13 @@ func TestPeriodicReconciliationAdoptsFileThatAppearsAfterStartup(t *testing.T) {
 
 	// Simulate a variant whose accounting insert failed after the
 	// process was already running and serving requests: the content
-	// file is written directly, bypassing StoreVariant's (and thus its
+	// file is written directly, bypassing PutVariant's (and thus its
 	// accounting insert) entirely, exactly as would happen if that
 	// insert had failed and only the file write had succeeded.
 	untracked := bytes.Repeat([]byte{0x41}, 900)
 
 	_, err = cache.variants.Store(
-		"aabbccdd0099", bytes.NewReader(untracked), "image/webp",
+		"aabbccdd0099", bytes.NewReader(untracked),
 	)
 	if err != nil {
 		t.Fatalf("failed to store untracked variant file: %v", err)

internal/imgcache/imgcache.go

@@ -1,66 +0,0 @@
-// Package imgcache provides interfaces and types for the image caching proxy.
-package imgcache
-
-import (
-	"context"
-	"io"
-	"time"
-
-	"sneak.berlin/go/pixa/internal/imagerequest"
-)
-
-// ImageResponse represents a processed image ready to serve
-type ImageResponse struct {
-	// Content is the image data reader
-	Content io.ReadCloser
-	// ContentLength is the size in bytes (-1 if unknown)
-	ContentLength int64
-	// ContentType is the MIME type of the response
-	ContentType string
-	// ETag is the entity tag for caching
-	ETag string
-	// LastModified is when the content was last modified
-	LastModified time.Time
-	// CacheStatus indicates HIT, MISS, or STALE
-	CacheStatus CacheStatus
-	// FetchedBytes is the number of bytes fetched from upstream (0 if cache hit)
-	FetchedBytes int64
-}
-
-// CacheStatus indicates whether the response was served from cache.
-type CacheStatus string
-
-// Cache status values for response headers.
-const (
-	CacheHit   CacheStatus = "HIT"
-	CacheMiss  CacheStatus = "MISS"
-	CacheStale CacheStatus = "STALE"
-)
-
-// ImageCache is the main interface for the image caching proxy
-type ImageCache interface {
-	// Get retrieves a processed image, fetching and processing if necessary
-	Get(ctx context.Context, req *imagerequest.Request) (*ImageResponse, error)
-
-	// Warm pre-fetches and caches an image without returning it
-	Warm(ctx context.Context, req *imagerequest.Request) error
-
-	// Stats returns cache statistics
-	Stats(ctx context.Context) (*CacheStats, error)
-}
-
-// CacheStats contains cache statistics
-type CacheStats struct {
-	// TotalItems is the number of cached source images plus processed
-	// variants
-	TotalItems int64
-	// TotalSizeBytes is the total size of cached source images and
-	// processed variants
-	TotalSizeBytes int64
-	// HitCount is the number of cache hits
-	HitCount int64
-	// MissCount is the number of cache misses
-	MissCount int64
-	// HitRate is HitCount / (HitCount + MissCount)
-	HitRate float64
-}

internal/imgcache/metacache_internal_test.go

@@ -1,361 +0,0 @@
-package imgcache
-
-import (
-	"bytes"
-	"encoding/json"
-	"errors"
-	"fmt"
-	"io"
-	"os"
-	"sync"
-	"testing"
-	"time"
-
-	"sneak.berlin/go/pixa/internal/imagerequest"
-)
-
-// webpRequest returns a request for a 100x100 WebP variant of path.
-func webpRequest(path string) *imagerequest.Request {
-	return &imagerequest.Request{
-		Source: imagerequest.Source{
-			Host: testHostCDN,
-			Path: path,
-		},
-		Transform: imagerequest.Transform{
-			Size:    imagerequest.Size{Width: 100, Height: 100},
-			Format:  imagerequest.FormatWebP,
-			Quality: 85,
-			FitMode: imagerequest.FitCover,
-		},
-	}
-}
-
-// assertVariantServed checks that GetVariant serves key with the given
-// content and the image/webp content type storeEvictionTestVariant stores.
-func assertVariantServed(
-	t *testing.T, cache *Cache, key imagerequest.VariantKey, content []byte,
-) {
-	t.Helper()
-
-	reader, size, contentType, err := cache.GetVariant(key)
-	if err != nil {
-		t.Fatalf("GetVariant(%s) error = %v", key, err)
-	}
-
-	defer func() { _ = reader.Close() }()
-
-	got, err := io.ReadAll(reader)
-	if err != nil {
-		t.Fatalf("reading variant %s: %v", key, err)
-	}
-
-	if !bytes.Equal(got, content) {
-		t.Errorf("GetVariant(%s) content = %q, want %q", key, got, content)
-	}
-
-	if size != int64(len(content)) {
-		t.Errorf("GetVariant(%s) size = %d, want %d", key, size, len(content))
-	}
-
-	if contentType != testContentTypeWebP {
-		t.Errorf("GetVariant(%s) content type = %q, want %q",
-			key, contentType, testContentTypeWebP)
-	}
-}
-
-// assertVariantNotFound checks that GetVariant refuses key with
-// ErrNotFound.
-func assertVariantNotFound(
-	t *testing.T, cache *Cache, key imagerequest.VariantKey,
-) {
-	t.Helper()
-
-	reader, _, _, err := cache.GetVariant(key)
-	if err == nil {
-		_ = reader.Close()
-	}
-
-	if !errors.Is(err, ErrNotFound) {
-		t.Errorf("GetVariant(%s) error = %v, want ErrNotFound", key, err)
-	}
-}
-
-// assertLookupMisses checks that Lookup reports request as a miss.
-func assertLookupMisses(
-	t *testing.T, cache *Cache, request *imagerequest.Request,
-) {
-	t.Helper()
-
-	lookup, err := cache.Lookup(t.Context(), request)
-	if err != nil {
-		t.Fatalf("Lookup(%s) error = %v", request.Source.Path, err)
-	}
-
-	if lookup.Hit {
-		t.Errorf("Lookup(%s) is a hit, want a miss", request.Source.Path)
-	}
-}
-
-// renameFile renames the file at from to to.
-func renameFile(t *testing.T, from, to string) {
-	t.Helper()
-
-	err := os.Rename(from, to)
-	if err != nil {
-		t.Fatalf("renaming %s: %v", from, err)
-	}
-}
-
-// TestSecondHitDoesNotReadMetaFile checks that once a variant has been
-// stored or read, a hit takes its content type from memory: with the
-// .meta file deleted, GetVariant must still return the stored content
-// type rather than the application/octet-stream it uses without one.
-func TestSecondHitDoesNotReadMetaFile(t *testing.T) {
-	t.Parallel()
-
-	cache, _ := newEvictionTestCache(t, 1<<20)
-	content := []byte("webp variant bytes")
-
-	storeEvictionTestVariant(t, cache, testVariantKeyOne, content)
-
-	// A second Cache on the same state directory starts with nothing in
-	// memory, as pixad does after a restart, so its first read uses the
-	// .meta file.
-	restarted, err := NewCache(cache.db, cache.config)
-	if err != nil {
-		t.Fatalf("NewCache() error = %v", err)
-	}
-
-	assertVariantServed(t, restarted, testVariantKeyOne, content)
-
-	err = os.Remove(cache.variants.keyToPath(testVariantKeyOne) + ".meta")
-	if err != nil {
-		t.Fatalf("removing .meta file: %v", err)
-	}
-
-	assertVariantServed(t, cache, testVariantKeyOne, content)
-	assertVariantServed(t, restarted, testVariantKeyOne, content)
-}
-
-// TestReadDuringStoreKeepsStoredContentType checks that a GetVariant
-// which began before StoreVariant finished cannot replace the content
-// type the store kept in memory. Such a read can find the variant file
-// but not yet its .meta file, and so gets application/octet-stream. The
-// test deletes the .meta file after the store, then runs the part of
-// GetVariant that comes after its check of memory.
-func TestReadDuringStoreKeepsStoredContentType(t *testing.T) {
-	t.Parallel()
-
-	cache, _ := newEvictionTestCache(t, 1<<20)
-	content := []byte("webp variant bytes")
-
-	storeEvictionTestVariant(t, cache, testVariantKeyOne, content)
-
-	err := os.Remove(cache.variants.keyToPath(testVariantKeyOne) + ".meta")
-	if err != nil {
-		t.Fatalf("removing .meta file: %v", err)
-	}
-
-	reader, _, contentType, err := cache.loadVariantWithMeta(testVariantKeyOne)
-	if err != nil {
-		t.Fatalf("loadVariantWithMeta(%s) error = %v", testVariantKeyOne, err)
-	}
-
-	_ = reader.Close()
-
-	t.Logf("the read without a .meta file got content type %q", contentType)
-
-	assertVariantServed(t, cache, testVariantKeyOne, content)
-}
-
-// TestReadOfOlderMetaFileKeepsStoredContentType checks that a read
-// which got its content type from a .meta file that StoreVariant had
-// not yet rewritten cannot replace the type the store kept in memory.
-// The test writes such a .meta file, with a different content type,
-// after the store, then runs the part of GetVariant that comes after
-// its check of memory.
-func TestReadOfOlderMetaFileKeepsStoredContentType(t *testing.T) {
-	t.Parallel()
-
-	cache, _ := newEvictionTestCache(t, 1<<20)
-	content := []byte("webp variant bytes")
-
-	storeEvictionTestVariant(t, cache, testVariantKeyOne, content)
-
-	olderMeta, err := json.Marshal(VariantMeta{
-		ContentType: testContentTypeJPEG,
-		Size:        int64(len(content)),
-	})
-	if err != nil {
-		t.Fatalf("encoding .meta file: %v", err)
-	}
-
-	metaPath := cache.variants.keyToPath(testVariantKeyOne) + ".meta"
-
-	err = os.WriteFile(metaPath, olderMeta, StorageFilePerm)
-	if err != nil {
-		t.Fatalf("writing .meta file: %v", err)
-	}
-
-	reader, _, contentType, err := cache.loadVariantWithMeta(testVariantKeyOne)
-	if err != nil {
-		t.Fatalf("loadVariantWithMeta(%s) error = %v", testVariantKeyOne, err)
-	}
-
-	_ = reader.Close()
-
-	if contentType != testContentTypeJPEG {
-		t.Fatalf("loadVariantWithMeta(%s) content type = %q, want %q from the .meta file",
-			testVariantKeyOne, contentType, testContentTypeJPEG)
-	}
-
-	kept, _ := cache.metaCache.Get(testVariantKeyOne)
-	if kept != testContentTypeWebP {
-		t.Errorf("content type in memory = %q, want the stored %q",
-			kept, testContentTypeWebP)
-	}
-}
-
-// TestFailedReadDuringStoreKeepsStoredContentType checks that a read
-// which found no .meta file cannot leave application/octet-stream in
-// memory, even when another read has removed the content type
-// StoreVariant kept there. In this order: the store; a read that finds
-// the variant in memory but cannot open its file, and so removes it
-// from memory; a read that opened the variant file before the store
-// wrote its .meta file. Later hits must get the stored content type.
-func TestFailedReadDuringStoreKeepsStoredContentType(t *testing.T) {
-	t.Parallel()
-
-	cache, _ := newEvictionTestCache(t, 1<<20)
-	content := []byte("webp variant bytes")
-	variantPath := cache.variants.keyToPath(testVariantKeyOne)
-	metaPath := variantPath + ".meta"
-
-	storeEvictionTestVariant(t, cache, testVariantKeyOne, content)
-
-	renameFile(t, variantPath, variantPath+".hidden")
-	assertVariantNotFound(t, cache, testVariantKeyOne)
-	renameFile(t, variantPath+".hidden", variantPath)
-
-	renameFile(t, metaPath, metaPath+".hidden")
-
-	reader, _, contentType, err := cache.GetVariant(testVariantKeyOne)
-	if err != nil {
-		t.Fatalf("GetVariant(%s) error = %v", testVariantKeyOne, err)
-	}
-
-	_ = reader.Close()
-
-	t.Logf("the read without a .meta file got content type %q", contentType)
-
-	renameFile(t, metaPath+".hidden", metaPath)
-
-	assertVariantServed(t, cache, testVariantKeyOne, content)
-}
-
-// TestEvictedVariantIsNotServed checks that a variant the evictor
-// removed is a miss and cannot be read, although it had been stored
-// and served before.
-func TestEvictedVariantIsNotServed(t *testing.T) {
-	t.Parallel()
-
-	cache, _ := newEvictionTestCache(t, 1500)
-
-	oldRequest := webpRequest("/old.jpg")
-	newRequest := webpRequest("/new.jpg")
-	oldKey := imagerequest.CacheKey(oldRequest)
-	newKey := imagerequest.CacheKey(newRequest)
-	oldContent := bytes.Repeat([]byte{0x01}, 1000)
-	newContent := bytes.Repeat([]byte{0x02}, 1000)
-
-	storeEvictionTestVariant(t, cache, oldKey, oldContent)
-	storeEvictionTestVariant(t, cache, newKey, newContent)
-	assertVariantServed(t, cache, oldKey, oldContent)
-	setVariantLastAccessed(t, cache, oldKey, time.Now().Add(-time.Hour))
-
-	err := cache.EvictToLimit(t.Context())
-	if err != nil {
-		t.Fatalf("EvictToLimit() error = %v", err)
-	}
-
-	assertLookupMisses(t, cache, oldRequest)
-	assertVariantNotFound(t, cache, oldKey)
-	assertVariantServed(t, cache, newKey, newContent)
-}
-
-// TestVariantDeletedFromDiskIsNotServed checks that a variant whose
-// file was deleted by something other than the evictor cannot be read,
-// and is a miss afterwards.
-func TestVariantDeletedFromDiskIsNotServed(t *testing.T) {
-	t.Parallel()
-
-	cache, _ := newEvictionTestCache(t, 1<<20)
-
-	request := webpRequest("/deleted.jpg")
-	key := imagerequest.CacheKey(request)
-	content := []byte("webp variant bytes")
-
-	storeEvictionTestVariant(t, cache, key, content)
-	assertVariantServed(t, cache, key, content)
-
-	err := os.Remove(cache.variants.keyToPath(key))
-	if err != nil {
-		t.Fatalf("removing variant file: %v", err)
-	}
-
-	assertVariantNotFound(t, cache, key)
-	assertLookupMisses(t, cache, request)
-}
-
-// TestConcurrentVariantStoreReadAndEvict stores, reads and evicts
-// variants from several goroutines at once, for the race detector.
-func TestConcurrentVariantStoreReadAndEvict(t *testing.T) {
-	t.Parallel()
-
-	cache, _ := newEvictionTestCache(t, 1<<20)
-	ctx := t.Context()
-
-	var wg sync.WaitGroup
-
-	for goroutine := range 8 {
-		wg.Go(func() {
-			key := imagerequest.VariantKey(fmt.Sprintf("aabbccdd01%02d", goroutine))
-			content := []byte(key)
-
-			for range 20 {
-				err := cache.StoreVariant(
-					ctx, key, bytes.NewReader(content), testContentTypeWebP)
-				if err != nil {
-					t.Errorf("StoreVariant(%s) error = %v", key, err)
-
-					return
-				}
-
-				reader, _, contentType, err := cache.GetVariant(key)
-				if err != nil {
-					t.Errorf("GetVariant(%s) error = %v", key, err)
-
-					return
-				}
-
-				_ = reader.Close()
-
-				if contentType != testContentTypeWebP {
-					t.Errorf("GetVariant(%s) content type = %q, want %q",
-						key, contentType, testContentTypeWebP)
-				}
-
-				err = cache.evictVariant(ctx, key)
-				if err != nil {
-					t.Errorf("evictVariant(%s) error = %v", key, err)
-
-					return
-				}
-
-				assertVariantNotFound(t, cache, key)
-			}
-		})
-	}
-
-	wg.Wait()
-}

internal/imgcache/module.go

@@ -1,8 +0,0 @@
-package imgcache
-
-import "time"
-
-// CacheConfig defaults.
-const (
-	DefaultNegativeTTL = 5 * time.Minute
-)

internal/imgcache/negative_cache_internal_test.go

@@ -7,6 +7,7 @@ import (
 	"testing"
 	"time"
 
+	"github.com/prometheus/client_golang/prometheus"
 	"sneak.berlin/go/pixa/internal/imagerequest"
 )
 
@@ -16,10 +17,11 @@ func TestNegativeCache_StoreAndCheck(t *testing.T) {
 	db := setupTestDB(t)
 	dir := t.TempDir()
 
-	cache, err := NewCache(db, CacheConfig{
+	cache, err := NewCache(db, Config{
 		StateDir:    dir,
 		NegativeTTL: 5 * time.Minute,
-	})
+		MaxBytes:    1 << 30,
+	}, prometheus.NewRegistry())
 	if err != nil {
 		t.Fatal(err)
 	}
@@ -33,7 +35,7 @@ func TestNegativeCache_StoreAndCheck(t *testing.T) {
 	}
 
 	// Initially should not be in negative cache
-	hit, err := cache.checkNegativeCache(ctx, req)
+	hit, err := cache.Failure(ctx, req.Source)
 	if err != nil {
 		t.Fatal(err)
 	}
@@ -43,13 +45,13 @@ func TestNegativeCache_StoreAndCheck(t *testing.T) {
 	}
 
 	// Store a negative entry
-	err = cache.StoreNegative(ctx, req, 404, "not found")
+	err = cache.PutFailure(ctx, req.Source)
 	if err != nil {
 		t.Fatal(err)
 	}
 
 	// Now should be in negative cache
-	hit, err = cache.checkNegativeCache(ctx, req)
+	hit, err = cache.Failure(ctx, req.Source)
 	if err != nil {
 		t.Fatal(err)
 	}
@@ -65,10 +67,11 @@ func TestNegativeCache_Expired(t *testing.T) {
 	db := setupTestDB(t)
 	dir := t.TempDir()
 
-	cache, err := NewCache(db, CacheConfig{
+	cache, err := NewCache(db, Config{
 		StateDir:    dir,
 		NegativeTTL: 1 * time.Millisecond, // very short TTL
-	})
+		MaxBytes:    1 << 30,
+	}, prometheus.NewRegistry())
 	if err != nil {
 		t.Fatal(err)
 	}
@@ -82,7 +85,7 @@ func TestNegativeCache_Expired(t *testing.T) {
 	}
 
 	// Store a negative entry with very short TTL
-	err = cache.StoreNegative(ctx, req, 500, "server error")
+	err = cache.PutFailure(ctx, req.Source)
 	if err != nil {
 		t.Fatal(err)
 	}
@@ -91,7 +94,7 @@ func TestNegativeCache_Expired(t *testing.T) {
 	time.Sleep(10 * time.Millisecond)
 
 	// Should no longer be in negative cache
-	hit, err := cache.checkNegativeCache(ctx, req)
+	hit, err := cache.Failure(ctx, req.Source)
 	if err != nil {
 		t.Fatal(err)
 	}
@@ -101,72 +104,6 @@ func TestNegativeCache_Expired(t *testing.T) {
 	}
 }
 
-// TestService_Get_ServesCachedVariantOfNegativeCachedURL caches a variant of
-// the test photo, then keeps a failed fetch of the photo's URL in the negative
-// cache, as a failed fetch for another variant does once the source has been
-// evicted. The cached variant is still served, and a variant not in the cache
-// is made from the cached source. Once the source is evicted, a variant not in
-// the cache needs a fetch and is refused.
-func TestService_Get_ServesCachedVariantOfNegativeCachedURL(t *testing.T) {
-	t.Parallel()
-
-	svc, fixtures := SetupTestService(t)
-
-	resp, err := svc.Get(t.Context(), photoVariant(fixtures, 85, imagerequest.FitCover))
-	if err != nil {
-		t.Fatalf("Get() first request error = %v", err)
-	}
-
-	_ = resp.Content.Close()
-
-	err = svc.cache.StoreNegative(t.Context(),
-		photoVariant(fixtures, 85, imagerequest.FitCover),
-		httpStatusBadGateway, "upstream answered 503")
-	if err != nil {
-		t.Fatalf("StoreNegative() error = %v", err)
-	}
-
-	resp, err = svc.Get(t.Context(), photoVariant(fixtures, 85, imagerequest.FitCover))
-	if err != nil {
-		t.Fatalf("Get() of the cached variant error = %v", err)
-	}
-
-	_ = resp.Content.Close()
-
-	if resp.CacheStatus != CacheHit {
-		t.Errorf("cached variant CacheStatus = %q, want %q",
-			resp.CacheStatus, CacheHit)
-	}
-
-	resp, err = svc.Get(t.Context(), photoVariant(fixtures, 40, imagerequest.FitCover))
-	t.Logf("Get() of a variant made from the cached source: error = %v", err)
-
-	if err != nil {
-		t.Fatalf("Get() of a variant made from the cached source error = %v", err)
-	}
-
-	_ = resp.Content.Close()
-
-	sourceKey, _, err := svc.cache.LookupSource(t.Context(),
-		photoVariant(fixtures, 85, imagerequest.FitCover))
-	if err != nil || sourceKey == "" {
-		t.Fatalf("LookupSource() = %q, %v, want the cached source", sourceKey, err)
-	}
-
-	err = svc.cache.evictSourceBlob(t.Context(), sourceKey)
-	if err != nil {
-		t.Fatalf("evictSourceBlob() error = %v", err)
-	}
-
-	_, err = svc.Get(t.Context(), photoVariant(fixtures, 60, imagerequest.FitCover))
-	t.Logf("Get() of a variant whose source was evicted: error = %v", err)
-
-	if !errors.Is(err, ErrNegativeCached) {
-		t.Errorf("Get() of a variant whose source was evicted error = %v, want %v",
-			err, ErrNegativeCached)
-	}
-}
-
 // TestEvictionLoopRemovesExpiredNegativeCacheRows stores a failed fetch
 // that expires a second later and starts the eviction loop with a short
 // interval: a later tick must remove its row although its URL is never
@@ -177,22 +114,22 @@ func TestEvictionLoopRemovesExpiredNegativeCacheRows(t *testing.T) {
 	t.Parallel()
 
 	tests := []struct {
-		name             string
-		disableDiskCache bool
+		name     string
+		maxBytes int64
 	}{
-		{name: "disk cache enabled", disableDiskCache: false},
-		{name: "disk cache disabled", disableDiskCache: true},
+		{name: "disk cache enabled", maxBytes: 1 << 30},
+		{name: "disk cache disabled", maxBytes: 0},
 	}
 
 	for _, tt := range tests {
 		t.Run(tt.name, func(t *testing.T) {
 			t.Parallel()
 
-			cache, err := NewCache(evictionTestDB(t), CacheConfig{
-				StateDir:         t.TempDir(),
-				NegativeTTL:      time.Second,
-				DisableDiskCache: tt.disableDiskCache,
-			})
+			cache, err := NewCache(evictionTestDB(t), Config{
+				StateDir:    t.TempDir(),
+				NegativeTTL: time.Second,
+				MaxBytes:    tt.maxBytes,
+			}, prometheus.NewRegistry())
 			if err != nil {
 				t.Fatalf("failed to create cache: %v", err)
 			}
@@ -201,17 +138,17 @@ func TestEvictionLoopRemovesExpiredNegativeCacheRows(t *testing.T) {
 				Source: imagerequest.Source{Host: testHostExample, Path: "/gone.jpg"},
 			}
 
-			err = cache.StoreNegative(t.Context(), expired, 404, "not found")
+			err = cache.PutFailure(t.Context(), expired.Source)
 			if err != nil {
-				t.Fatalf("StoreNegative failed: %v", err)
+				t.Fatalf("PutFailure failed: %v", err)
 			}
 
 			const notExpiredPath = "/later.jpg"
 
 			_, err = cache.db.ExecContext(t.Context(), `
 				INSERT INTO negative_cache
-					(source_host, source_path, status_code, expires_at)
-				VALUES (?, ?, 404, datetime('now', '+1 hour'))
+					(source_host, source_path, expires_at)
+				VALUES (?, ?, datetime('now', '+1 hour'))
 			`, testHostExample, notExpiredPath)
 			if err != nil {
 				t.Fatalf("failed to insert a row that has not expired: %v", err)
@@ -266,7 +203,9 @@ func negativeCacheRowCount(tb testing.TB, cache *Cache, sourcePath string) int {
 func TestCleanExpiredDeletesInBatches(t *testing.T) {
 	t.Parallel()
 
-	cache, err := NewCache(evictionTestDB(t), CacheConfig{StateDir: t.TempDir()})
+	cache, err := NewCache(evictionTestDB(t),
+		Config{StateDir: t.TempDir(), MaxBytes: 1 << 30},
+		prometheus.NewRegistry())
 	if err != nil {
 		t.Fatalf("failed to create cache: %v", err)
 	}
@@ -279,8 +218,8 @@ func TestCleanExpiredDeletesInBatches(t *testing.T) {
 
 	_, err = cache.db.ExecContext(ctx, `
 		INSERT INTO negative_cache
-			(source_host, source_path, status_code, expires_at)
-		VALUES (?, ?, 404, datetime('now', '+1 hour'))
+			(source_host, source_path, expires_at)
+		VALUES (?, ?, datetime('now', '+1 hour'))
 	`, testHostExample, notExpiredPath)
 	if err != nil {
 		t.Fatalf("failed to insert a row that has not expired: %v", err)
@@ -320,7 +259,9 @@ func TestCleanExpiredDeletesInBatches(t *testing.T) {
 func TestCleanExpiredRemovesMoreThanOneBatch(t *testing.T) {
 	t.Parallel()
 
-	cache, err := NewCache(evictionTestDB(t), CacheConfig{StateDir: t.TempDir()})
+	cache, err := NewCache(evictionTestDB(t),
+		Config{StateDir: t.TempDir(), MaxBytes: 1 << 30},
+		prometheus.NewRegistry())
 	if err != nil {
 		t.Fatalf("failed to create cache: %v", err)
 	}
@@ -350,7 +291,9 @@ func TestCleanExpiredRemovesMoreThanOneBatch(t *testing.T) {
 func TestCleanExpiredStopsOnceContextEnds(t *testing.T) {
 	t.Parallel()
 
-	cache, err := NewCache(evictionTestDB(t), CacheConfig{StateDir: t.TempDir()})
+	cache, err := NewCache(evictionTestDB(t),
+		Config{StateDir: t.TempDir(), MaxBytes: 1 << 30},
+		prometheus.NewRegistry())
 	if err != nil {
 		t.Fatalf("failed to create cache: %v", err)
 	}
@@ -378,8 +321,8 @@ func insertExpiredNegativeCacheRows(t *testing.T, cache *Cache, count int) {
 	for i := range count {
 		_, err := cache.db.ExecContext(t.Context(), `
 			INSERT INTO negative_cache
-				(source_host, source_path, status_code, expires_at)
-			VALUES (?, ?, 404, datetime('now', '-1 hour'))
+				(source_host, source_path, expires_at)
+			VALUES (?, ?, datetime('now', '-1 hour'))
 		`, testHostExample, "/expired/"+strconv.Itoa(i)+".jpg")
 		if err != nil {
 			t.Fatalf("failed to insert an expired row: %v", err)
@@ -402,15 +345,3 @@ func negativeCacheTableRowCount(tb testing.TB, cache *Cache) int {
 
 	return count
 }
-
-func TestService_Get_ReturnsErrorForNegativeCachedURL(t *testing.T) {
-	t.Parallel()
-
-	// This test verifies that Service.Get() checks the negative cache
-	// We can't easily test the full pipeline without vips, but we can
-	// verify the error type
-	err := ErrNegativeCached
-	if !errors.Is(err, ErrNegativeCached) {
-		t.Error("ErrNegativeCached should be identifiable with errors.Is")
-	}
-}

internal/imgcache/partial_cache_files_internal_test.go

@@ -12,67 +12,6 @@ import (
 	"sneak.berlin/go/pixa/internal/imagerequest"
 )
 
-// getImage calls Get for req and returns the image it answered with and
-// its cache status.
-func getImage(
-	t *testing.T, svc *Service, req *imagerequest.Request,
-) ([]byte, CacheStatus) {
-	t.Helper()
-
-	resp, err := svc.Get(t.Context(), req)
-	if err != nil {
-		t.Fatalf("Get() of quality %d error = %v", req.Transform.Quality, err)
-	}
-
-	defer func() { _ = resp.Content.Close() }()
-
-	body, err := io.ReadAll(resp.Content)
-	if err != nil {
-		t.Fatalf("reading the image of quality %d: %v", req.Transform.Quality, err)
-	}
-
-	return body, resp.CacheStatus
-}
-
-// fileSize returns the size of the file at path.
-func fileSize(t *testing.T, path string) int64 {
-	t.Helper()
-
-	info, err := os.Stat(path)
-	if err != nil {
-		t.Fatalf("stat %s: %v", path, err)
-	}
-
-	return info.Size()
-}
-
-// truncateFile cuts the file at path down to size bytes.
-func truncateFile(t *testing.T, path string, size int64) {
-	t.Helper()
-
-	t.Logf("cutting %s from %d to %d bytes",
-		filepath.Base(path), fileSize(t, path), size)
-
-	err := os.Truncate(path, size)
-	if err != nil {
-		t.Fatalf("truncating %s: %v", path, err)
-	}
-}
-
-// cachedSourcePath returns the path of the cached source file req uses.
-func cachedSourcePath(
-	t *testing.T, svc *Service, req *imagerequest.Request,
-) string {
-	t.Helper()
-
-	sourceKey, _, err := svc.cache.LookupSource(t.Context(), req)
-	if err != nil || sourceKey == "" {
-		t.Fatalf("LookupSource() = %q, %v; want the cached source", sourceKey, err)
-	}
-
-	return svc.cache.srcContent.keyToPath(sourceKey)
-}
-
 // reconcileAsAtStartup runs the cache's reconciliation pass, as pixa does
 // when it starts, and returns what the pass logged.
 func reconcileAsAtStartup(t *testing.T, cache *Cache) string {
@@ -107,129 +46,6 @@ func assertWarningNames(t *testing.T, logs, path string) {
 	t.Errorf("reconciliation logged no warning naming %s", path)
 }
 
-// TestService_Get_FetchesAgainForEmptyCachedSource empties the cached source
-// file, as a crash or a restored snapshot could, and runs the reconciliation
-// pass, which must remove it with a warning naming it. It then asks for two
-// new sizes of the source: the first must fetch it again and cache it whole,
-// so that the second is made from the cache without a third fetch.
-func TestService_Get_FetchesAgainForEmptyCachedSource(t *testing.T) {
-	t.Parallel()
-
-	svc, fixtures := SetupTestService(t)
-
-	getImage(t, svc, photoVariant(fixtures, 85, imagerequest.FitCover))
-
-	path := cachedSourcePath(t, svc, photoVariant(fixtures, 85, imagerequest.FitCover))
-	wholeSize := fileSize(t, path)
-
-	truncateFile(t, path, 0)
-	assertWarningNames(t, reconcileAsAtStartup(t, svc.cache), path)
-
-	getImage(t, svc, photoVariant(fixtures, 40, imagerequest.FitCover))
-	getImage(t, svc, photoVariant(fixtures, 30, imagerequest.FitCover))
-
-	fetches := readCacheStatsCounters(t, svc.cache).upstreamFetchCount
-	t.Logf("upstream fetches = %d", fetches)
-
-	if fetches != 2 {
-		t.Errorf("upstream fetches = %d, want 2: the empty cached source "+
-			"was kept and fetched again on every new size", fetches)
-	}
-
-	size := fileSize(t, path)
-	if size != wholeSize {
-		t.Errorf("cached source is %d bytes, want %d", size, wholeSize)
-	}
-}
-
-// TestService_Get_FetchesAgainForPartialCachedSource cuts the cached source
-// file to half its size, as a crash or a restored snapshot could, and runs
-// the reconciliation pass, which must remove it with a warning naming it.
-// It then asks for two new sizes of the source: the first must fetch it
-// again rather than fail to decode the partial file, and the second is
-// made from the cache.
-func TestService_Get_FetchesAgainForPartialCachedSource(t *testing.T) {
-	t.Parallel()
-
-	svc, fixtures := SetupTestService(t)
-
-	getImage(t, svc, photoVariant(fixtures, 85, imagerequest.FitCover))
-
-	path := cachedSourcePath(t, svc, photoVariant(fixtures, 85, imagerequest.FitCover))
-	wholeSize := fileSize(t, path)
-
-	truncateFile(t, path, wholeSize/2)
-	assertWarningNames(t, reconcileAsAtStartup(t, svc.cache), path)
-
-	getImage(t, svc, photoVariant(fixtures, 40, imagerequest.FitCover))
-	getImage(t, svc, photoVariant(fixtures, 30, imagerequest.FitCover))
-
-	fetches := readCacheStatsCounters(t, svc.cache).upstreamFetchCount
-	t.Logf("upstream fetches = %d", fetches)
-
-	if fetches != 2 {
-		t.Errorf("upstream fetches = %d, want 2", fetches)
-	}
-
-	size := fileSize(t, path)
-	if size != wholeSize {
-		t.Errorf("cached source is %d bytes, want %d", size, wholeSize)
-	}
-}
-
-// assertConvertedAgain caches a variant of the photo, cuts its file to size
-// bytes and runs the reconciliation pass, which must remove it with a
-// warning naming it. It then checks that Get converts the image again
-// instead of serving the file, and that the next Get serves the whole
-// variant from the cache.
-func assertConvertedAgain(t *testing.T, size int64) {
-	t.Helper()
-
-	svc, fixtures := SetupTestService(t)
-	req := photoVariant(fixtures, 85, imagerequest.FitCover)
-
-	want, _ := getImage(t, svc, req)
-
-	path := svc.cache.variants.keyToPath(imagerequest.CacheKey(req))
-
-	truncateFile(t, path, size)
-	assertWarningNames(t, reconcileAsAtStartup(t, svc.cache), path)
-
-	got, status := getImage(t, svc, req)
-	t.Logf("after the cut: %d bytes, %s", len(got), status)
-
-	if status != CacheMiss || !bytes.Equal(got, want) {
-		t.Errorf("after the cut: Get() = %d bytes, %s; want the %d bytes "+
-			"converted again, %s", len(got), status, len(want), CacheMiss)
-	}
-
-	got, status = getImage(t, svc, req)
-	t.Logf("next request: %d bytes, %s", len(got), status)
-
-	if status != CacheHit || !bytes.Equal(got, want) {
-		t.Errorf("next request: Get() = %d bytes, %s; want the %d bytes, %s",
-			len(got), status, len(want), CacheHit)
-	}
-}
-
-// TestService_Get_ConvertsAgainForEmptyCachedVariant empties a cached
-// variant file, as a crash or a restored snapshot could, and checks that it
-// is converted again rather than served empty.
-func TestService_Get_ConvertsAgainForEmptyCachedVariant(t *testing.T) {
-	t.Parallel()
-
-	assertConvertedAgain(t, 0)
-}
-
-// TestService_Get_ConvertsAgainForPartialCachedVariant cuts a cached
-// variant file to ten bytes, as a crash or a restored snapshot could, and
-// checks that it is converted again rather than served cut short.
-func TestService_Get_ConvertsAgainForPartialCachedVariant(t *testing.T) {
-	t.Parallel()
-
-	assertConvertedAgain(t, 10)
-}
-
 // TestReconciliationRemovesEmptyVariantFileWithoutRow checks that the
 // reconciliation pass removes an empty variant file that has no
 // accounting row, with a warning naming it, instead of adding a row for
@@ -239,8 +55,7 @@ func TestReconciliationRemovesEmptyVariantFileWithoutRow(t *testing.T) {
 
 	cache, _ := newEvictionTestCache(t, 1<<30)
 
-	_, err := cache.variants.Store(testVariantKeyOne, bytes.NewReader(nil),
-		testContentTypeWebP)
+	_, err := cache.variants.Store(testVariantKeyOne, bytes.NewReader(nil))
 	if err != nil {
 		t.Fatalf("failed to store an empty variant file: %v", err)
 	}
@@ -261,59 +76,6 @@ func TestReconciliationRemovesEmptyVariantFileWithoutRow(t *testing.T) {
 	}
 }
 
-// TestVariantStorage_StoreReplacesMetaFile stores a variant over one
-// already stored and checks that Store puts a new .meta file in place of
-// the old one, as it does the image, rather than writing into the old
-// file, which a crash could leave empty. The old .meta file is given a
-// second name before the store, so that it can be told apart from the new
-// one after it.
-func TestVariantStorage_StoreReplacesMetaFile(t *testing.T) {
-	t.Parallel()
-
-	storage, key := storeTestVariant(t)
-	metaPath := storage.keyToPath(key) + variantMetaSuffix
-	oldMetaPath := metaPath + ".old"
-
-	err := os.Link(metaPath, oldMetaPath)
-	if err != nil {
-		t.Fatalf("giving the .meta file a second name: %v", err)
-	}
-
-	_, err = storage.Store(key, bytes.NewReader([]byte("new variant data")),
-		testContentTypeJPEG)
-	if err != nil {
-		t.Fatalf("Store() error = %v", err)
-	}
-
-	oldMeta, err := os.Stat(oldMetaPath)
-	if err != nil {
-		t.Fatalf("stat of the old .meta file: %v", err)
-	}
-
-	newMeta, err := os.Stat(metaPath)
-	if err != nil {
-		t.Fatalf("stat of the new .meta file: %v", err)
-	}
-
-	if os.SameFile(oldMeta, newMeta) {
-		t.Error("Store() wrote into the old .meta file instead of putting " +
-			"a new one in its place")
-	}
-
-	reader, _, contentType, err := storage.LoadWithMeta(key)
-	if err != nil {
-		t.Fatalf("LoadWithMeta() error = %v", err)
-	}
-
-	_ = reader.Close()
-
-	t.Logf("content type after the store: %s", contentType)
-
-	if contentType != testContentTypeJPEG {
-		t.Errorf("content type = %q, want %q", contentType, testContentTypeJPEG)
-	}
-}
-
 // TestContentStorage_StoreReplacesFileOfAnotherSize checks that a
 // store over a file of another size at the source's path, such as a
 // partial file a crash left, replaces it with the content.
@@ -344,9 +106,9 @@ func TestContentStorage_StoreReplacesFileOfAnotherSize(t *testing.T) {
 		t.Fatalf("Store() error = %v", err)
 	}
 
-	stored, err := storage.Load(key)
+	stored, _, err := storage.LoadWithSize(key)
 	if err != nil {
-		t.Fatalf("Load() error = %v", err)
+		t.Fatalf("LoadWithSize() error = %v", err)
 	}
 
 	defer func() { _ = stored.Close() }()

internal/imgcache/pendingaccesstimes.go

@@ -0,0 +1,160 @@
+package imgcache
+
+import (
+	"context"
+	"fmt"
+	"time"
+
+	"sneak.berlin/go/pixa/internal/imagerequest"
+)
+
+// pendingWriteInterval is how often the goroutine
+// StartPendingAccessTimeWrites starts writes the pending access times.
+// SQLite's CURRENT_TIMESTAMP, which an access time is written as, counts
+// whole seconds.
+const pendingWriteInterval = time.Second
+
+// StartPendingAccessTimeWrites starts the goroutine that writes, every
+// pendingWriteInterval, the pending access times in one transaction.
+// Requests only hand these to it, so none waits for the database to record
+// them, and only this goroutine's write ever waits for it. It is a no-op
+// when already started. The goroutine outlives the caller, so it runs with
+// its own context, which StopPendingAccessTimeWrites cancels to tell it to
+// finish.
+func (c *Cache) StartPendingAccessTimeWrites() {
+	if c.pendingAccessTimesCancel != nil {
+		return
+	}
+
+	ctx, cancel := context.WithCancel(context.Background())
+	c.pendingAccessTimesCancel = cancel
+
+	go func() {
+		c.pendingAccessTimeWriteLoop(ctx)
+	}()
+}
+
+// StopPendingAccessTimeWrites tells the goroutine
+// StartPendingAccessTimeWrites started, if it did, to finish, and waits for
+// its write under way to end and for it to return. It then writes the
+// pending access times left, once, so that they are written before the
+// database closes. It waits for all of this at most until ctx ends. When
+// access times are not written, it logs that and returns an error.
+func (c *Cache) StopPendingAccessTimeWrites(ctx context.Context) error {
+	var err error
+
+	if c.pendingAccessTimesCancel != nil {
+		c.pendingAccessTimesCancel()
+
+		select {
+		case <-c.pendingAccessTimesDone:
+		case <-ctx.Done():
+			// The write given up on here may not reach the database:
+			// database/sql closes without waiting for a query, and a write
+			// still waiting for the connection fails when the database
+			// closes. Nothing is written here.
+			err = fmt.Errorf("access times still being written: %w", ctx.Err())
+		}
+	}
+
+	if err == nil {
+		err = c.writePendingAccessTimes(ctx)
+	}
+
+	if err != nil {
+		c.log.Error("access times not written at shutdown", "error", err)
+	}
+
+	return err
+}
+
+// pendingAccessTimeWriteLoop is the body of the goroutine
+// StartPendingAccessTimeWrites starts. Once ctx is cancelled it starts no
+// write and returns, but does not cut off a write under way then. Access
+// times whose write fails are dropped.
+func (c *Cache) pendingAccessTimeWriteLoop(ctx context.Context) {
+	defer close(c.pendingAccessTimesDone)
+
+	ticker := time.NewTicker(pendingWriteInterval)
+	defer ticker.Stop()
+
+	for {
+		select {
+		case <-ctx.Done():
+			return
+		case <-ticker.C:
+		}
+
+		// select picks at random when ctx is cancelled and a tick waits
+		// too.
+		if ctx.Err() != nil {
+			return
+		}
+
+		err := c.writePendingAccessTimes(context.WithoutCancel(ctx))
+		if err != nil {
+			c.log.Warn("failed to write access times", "error", err)
+		}
+	}
+}
+
+// writePendingAccessTimes takes out the variants waiting in
+// pendingVariantAccessTimes and the sources waiting in
+// pendingSourceAccessTimes, and sets their last_accessed_at to now in one
+// transaction, once for each however often it was used. Nothing else takes
+// from those channels while it runs: only the goroutine
+// StartPendingAccessTimeWrites starts calls it, or
+// StopPendingAccessTimeWrites once that goroutine has returned.
+func (c *Cache) writePendingAccessTimes(ctx context.Context) error {
+	variants := make(map[imagerequest.VariantKey]struct{})
+
+	for range len(c.pendingVariantAccessTimes) {
+		variants[<-c.pendingVariantAccessTimes] = struct{}{}
+	}
+
+	sources := make(map[imagerequest.SourceKey]struct{})
+
+	for range len(c.pendingSourceAccessTimes) {
+		sources[<-c.pendingSourceAccessTimes] = struct{}{}
+	}
+
+	accessTimes := len(variants) + len(sources)
+	if accessTimes == 0 {
+		return nil
+	}
+
+	tx, err := c.db.BeginTx(ctx, nil)
+	if err != nil {
+		return fmt.Errorf("failed to write %d access times: %w", accessTimes, err)
+	}
+
+	// Does nothing once the transaction is committed
+	defer func() { _ = tx.Rollback() }()
+
+	for cacheKey := range variants {
+		_, err = tx.ExecContext(ctx, `
+			UPDATE variant_content SET last_accessed_at = CURRENT_TIMESTAMP
+			WHERE cache_key = ?
+		`, string(cacheKey))
+		if err != nil {
+			return fmt.Errorf("failed to write %d access times: %w", accessTimes, err)
+		}
+	}
+
+	for sourceKey := range sources {
+		_, err = tx.ExecContext(ctx, `
+			UPDATE source_content SET last_accessed_at = CURRENT_TIMESTAMP
+			WHERE source_key = ?
+		`, string(sourceKey))
+		if err != nil {
+			return fmt.Errorf("failed to write %d access times: %w", accessTimes, err)
+		}
+	}
+
+	err = tx.Commit()
+	if err != nil {
+		return fmt.Errorf("failed to write %d access times: %w", accessTimes, err)
+	}
+
+	return nil
+}

internal/imgcache/pendingaccesstimes_internal_test.go

@@ -0,0 +1,279 @@
+package imgcache
+
+import (
+	"bytes"
+	"context"
+	"database/sql"
+	"errors"
+	"io"
+	"log/slog"
+	"strings"
+	"testing"
+	"time"
+
+	"sneak.berlin/go/pixa/internal/imagerequest"
+)
+
+// pendingAccessTimesWait is how long a test waits for the database's
+// connection, for an access time to reach the database, for a write to start
+// waiting for the connection, or for StopPendingAccessTimeWrites to return.
+const pendingAccessTimesWait = 5 * time.Second
+
+// otherWritesWait is how long a test gives other writes, once one write waits
+// for the database's connection, to start waiting for it too.
+const otherWritesWait = 100 * time.Millisecond
+
+// holdDatabase takes the one connection of cache's database, so that every
+// other query waits for it, and returns the func that frees it.
+func holdDatabase(t *testing.T, cache *Cache) func() {
+	t.Helper()
+
+	ctx, cancel := context.WithTimeout(t.Context(), pendingAccessTimesWait)
+	defer cancel()
+
+	conn, err := cache.db.Conn(ctx)
+	if err != nil {
+		t.Fatalf("failed to take the database connection: %v", err)
+	}
+
+	return func() { _ = conn.Close() }
+}
+
+// waitForConnectionWaits waits until db.Stats().WaitCount, the number of
+// times a caller has waited for a connection, reaches waits.
+func waitForConnectionWaits(t *testing.T, db *sql.DB, waits int64) {
+	t.Helper()
+
+	deadline := time.Now().Add(pendingAccessTimesWait)
+
+	for db.Stats().WaitCount < waits {
+		if time.Now().After(deadline) {
+			t.Fatalf("callers waited for the database connection %d times, want %d",
+				db.Stats().WaitCount, waits)
+		}
+
+		time.Sleep(10 * time.Millisecond)
+	}
+}
+
+// TestStopPendingAccessTimeWrites_LetsTheWriteUnderWayFinish holds the
+// database's one connection while the goroutine that writes the pending
+// access times waits for it, hands over another access time, then stops that
+// goroutine. The stop must leave the write under way to finish rather than
+// cut it off, and once the connection is free it must write the access time
+// that write left.
+func TestStopPendingAccessTimeWrites_LetsTheWriteUnderWayFinish(t *testing.T) {
+	t.Parallel()
+
+	cache, _ := newEvictionTestCache(t, 1<<30)
+	anHourAgo := time.Now().Add(-time.Hour)
+
+	for _, key := range []imagerequest.VariantKey{testVariantKeyOne, testVariantKeyTwo} {
+		storeEvictionTestVariant(t, cache, key, []byte("variant bytes"))
+		setVariantLastAccessed(t, cache, key, anHourAgo)
+	}
+
+	cache.StartPendingAccessTimeWrites()
+	defer func() { _ = cache.StopPendingAccessTimeWrites(t.Context()) }()
+
+	releaseDatabase := holdDatabase(t, cache)
+	defer releaseDatabase()
+
+	waitsBefore := cache.db.Stats().WaitCount
+
+	cache.touchVariant(testVariantKeyOne)
+
+	// The goroutine's write waits for the connection.
+	waitForConnectionWaits(t, cache.db, waitsBefore+1)
+
+	// Handed over after that write took the pending access times
+	cache.touchVariant(testVariantKeyTwo)
+
+	// The goroutine is told to finish here as well as by the stop, so that it
+	// is told before its write under way ends, however late the stop runs.
+	// The access time above then reaches the database only through the
+	// stop's own write.
+	cache.pendingAccessTimesCancel()
+
+	stopped := make(chan error, 1)
+
+	go func() {
+		stopped <- cache.StopPendingAccessTimeWrites(t.Context())
+	}()
+
+	// Had the stop cut off the write under way, its own write would wait for
+	// the connection too.
+	time.Sleep(otherWritesWait)
+
+	waits := cache.db.Stats().WaitCount - waitsBefore
+	t.Logf("%d writes waited for the database", waits)
+
+	if waits > 1 {
+		t.Errorf("%d writes waited for the database, want 1: "+
+			"the stop cut off the write under way", waits)
+	}
+
+	releaseDatabase()
+
+	select {
+	case err := <-stopped:
+		if err != nil {
+			t.Fatalf("StopPendingAccessTimeWrites() error = %v", err)
+		}
+	case <-time.After(pendingAccessTimesWait):
+		t.Fatal("StopPendingAccessTimeWrites() did not return once the database was free")
+	}
+
+	for _, key := range []imagerequest.VariantKey{testVariantKeyOne, testVariantKeyTwo} {
+		if countRows(t, cache, `
+			SELECT COUNT(*) FROM variant_content
+			WHERE cache_key = ? AND last_accessed_at > ?
+		`, string(key), anHourAgo.UTC().Format(sqliteTimestampFormat)) != 1 {
+			t.Errorf("the access time of variant %s was not written", key)
+		}
+	}
+}
+
+// TestStopPendingAccessTimeWrites_ReturnsWhenItsContextEnds holds the
+// database's one connection while the goroutine that writes the pending
+// access times waits for it, then stops that goroutine with a context that
+// ends first. The stop must return the context's error, and the write under
+// way must still reach the database once the connection is free.
+func TestStopPendingAccessTimeWrites_ReturnsWhenItsContextEnds(t *testing.T) {
+	t.Parallel()
+
+	cache, _ := newEvictionTestCache(t, 1<<30)
+	storeEvictionTestVariant(t, cache, testVariantKeyOne, []byte("variant bytes"))
+
+	anHourAgo := time.Now().Add(-time.Hour)
+	setVariantLastAccessed(t, cache, testVariantKeyOne, anHourAgo)
+
+	cache.StartPendingAccessTimeWrites()
+	defer func() { _ = cache.StopPendingAccessTimeWrites(t.Context()) }()
+
+	releaseDatabase := holdDatabase(t, cache)
+	defer releaseDatabase()
+
+	waitsBefore := cache.db.Stats().WaitCount
+
+	cache.touchVariant(testVariantKeyOne)
+
+	// The goroutine's write waits for the connection.
+	waitForConnectionWaits(t, cache.db, waitsBefore+1)
+
+	stopCtx, cancelStop := context.WithCancel(t.Context())
+	cancelStop()
+
+	stopped := make(chan error, 1)
+
+	go func() {
+		stopped <- cache.StopPendingAccessTimeWrites(stopCtx)
+	}()
+
+	select {
+	case err := <-stopped:
+		if !errors.Is(err, context.Canceled) {
+			t.Errorf("StopPendingAccessTimeWrites() error = %v, want %v",
+				err, context.Canceled)
+		}
+	case <-time.After(pendingAccessTimesWait):
+		t.Fatal("StopPendingAccessTimeWrites() did not return once its context ended")
+	}
+
+	releaseDatabase()
+
+	deadline := time.Now().Add(pendingAccessTimesWait)
+
+	for countRows(t, cache, `
+		SELECT COUNT(*) FROM variant_content
+		WHERE cache_key = ? AND last_accessed_at > ?
+	`, string(testVariantKeyOne), anHourAgo.UTC().Format(sqliteTimestampFormat)) == 0 {
+		if time.Now().After(deadline) {
+			t.Fatal("the access time being written was not written")
+		}
+
+		time.Sleep(10 * time.Millisecond)
+	}
+}
+
+// TestSource_HandsAccessTimeToTheBackgroundWriter finds a cached source
+// whose access time is an hour old. Source must not write the access
+// time itself but hand it over to be written, and StopPendingAccessTimeWrites
+// must then write it.
+func TestSource_HandsAccessTimeToTheBackgroundWriter(t *testing.T) {
+	t.Parallel()
+
+	cache, _ := newEvictionTestCache(t, 1<<30)
+	key := storeEvictionTestSource(t, cache, "example.com", "/photo.jpg",
+		[]byte("source bytes"))
+	anHourAgo := time.Now().Add(-time.Hour)
+	setSourceLastAccessed(t, cache, key, anHourAgo)
+
+	touchedQuery := `
+		SELECT COUNT(*) FROM source_content
+		WHERE source_key = ? AND last_accessed_at > ?
+	`
+	anHourAgoInDatabase := anHourAgo.UTC().Format(sqliteTimestampFormat)
+
+	reader, _, err := cache.Source(t.Context(),
+		imagerequest.Source{Host: "example.com", Path: "/photo.jpg"})
+	if err != nil {
+		t.Fatalf("Source() error = %v; want the source of %q", err, key)
+	}
+
+	found, err := io.ReadAll(reader)
+	_ = reader.Close()
+
+	if err != nil || string(found) != "source bytes" {
+		t.Fatalf("Source() reads %q, %v; want the source of %q", found, err, key)
+	}
+
+	if countRows(t, cache, touchedQuery, string(key), anHourAgoInDatabase) != 0 {
+		t.Error("Source wrote the source's access time itself")
+	}
+
+	if waiting := len(cache.pendingSourceAccessTimes); waiting != 1 {
+		t.Errorf("%d access times of sources wait to be written, want 1", waiting)
+	}
+
+	err = cache.StopPendingAccessTimeWrites(t.Context())
+	if err != nil {
+		t.Fatalf("StopPendingAccessTimeWrites() error = %v", err)
+	}
+
+	if countRows(t, cache, touchedQuery, string(key), anHourAgoInDatabase) != 1 {
+		t.Error("the source's access time was not written")
+	}
+}
+
+// TestStopPendingAccessTimeWrites_LogsWhatWasNotWritten makes the write of
+// the access times fail at shutdown, by dropping the table they are written
+// to. The stop must return an error and log that access times were not
+// written.
+func TestStopPendingAccessTimeWrites_LogsWhatWasNotWritten(t *testing.T) {
+	t.Parallel()
+
+	cache, _ := newEvictionTestCache(t, 1<<30)
+
+	var logs bytes.Buffer
+
+	cache.log = slog.New(slog.NewJSONHandler(&logs, nil))
+
+	cache.touchVariant(testVariantKeyOne)
+
+	_, err := cache.db.ExecContext(t.Context(), `DROP TABLE variant_content`)
+	if err != nil {
+		t.Fatalf("dropping variant_content: %v", err)
+	}
+
+	err = cache.StopPendingAccessTimeWrites(t.Context())
+	t.Logf("StopPendingAccessTimeWrites() error = %v; logged:\n%s", err, logs.String())
+
+	if err == nil {
+		t.Error("StopPendingAccessTimeWrites() error = nil, want the access times' error")
+	}
+
+	if !strings.Contains(logs.String(), "access times not written at shutdown") {
+		t.Error("the access times not written were not logged")
+	}
+}

internal/imgcache/pendingcounts.go

@@ -1,149 +0,0 @@
-package imgcache
-
-import (
-	"context"
-	"fmt"
-	"sync/atomic"
-)
-
-// StartPendingCountWrites starts the goroutine that writes the pending
-// counts to the database whenever there are some, one UPDATE at a time:
-// however many requests pass their deadline before their counts are
-// written, at most this one write waits for the database. It is a no-op
-// when already started. The goroutine outlives the caller, so it runs with
-// its own context, which StopPendingCountWrites cancels to tell it to
-// finish.
-func (c *Cache) StartPendingCountWrites() {
-	if c.pendingCountsCancel != nil {
-		return
-	}
-
-	ctx, cancel := context.WithCancel(context.Background())
-	c.pendingCountsCancel = cancel
-
-	go func() {
-		c.pendingCountWriteLoop(ctx)
-	}()
-}
-
-// StopPendingCountWrites tells the goroutine StartPendingCountWrites
-// started, if it did, to finish, and waits for its write under way to end
-// and for it to return. It then writes the pending counts left, so that
-// they are written before the database closes. It waits for all of this at
-// most until ctx ends, then logs the counts not written and returns an
-// error.
-func (c *Cache) StopPendingCountWrites(ctx context.Context) error {
-	var err error
-
-	if c.pendingCountsCancel != nil {
-		c.pendingCountsCancel()
-
-		select {
-		case <-c.pendingCountsDone:
-		case <-ctx.Done():
-			// The write given up on here may not reach the database:
-			// database/sql closes without waiting for a query, and a
-			// write still waiting for the connection fails when the
-			// database closes.
-			err = fmt.Errorf("pending counts still being written: %w", ctx.Err())
-		}
-	}
-
-	if err == nil {
-		err = c.writePendingCounts(ctx)
-	}
-
-	if err != nil {
-		c.log.Error("counts not written at shutdown",
-			"hits", c.pendingHits.Load(),
-			"misses", c.pendingMisses.Load(),
-			"upstream_fetches", c.pendingUpstreamFetches.Load(),
-			"upstream_fetch_bytes", c.pendingUpstreamFetchBytes.Load(),
-			"transforms", c.pendingTransforms.Load(),
-			"error", err,
-		)
-
-		return err
-	}
-
-	return nil
-}
-
-// pendingCountWriteLoop is the body of the goroutine
-// StartPendingCountWrites starts. Once ctx is cancelled it starts no write
-// and returns, but does not cut off a write under way then. A write that
-// fails leaves the counts pending, for the next write.
-func (c *Cache) pendingCountWriteLoop(ctx context.Context) {
-	defer close(c.pendingCountsDone)
-
-	for {
-		select {
-		case <-ctx.Done():
-			return
-		case <-c.pendingCountsAdded:
-		}
-
-		// select picks at random when ctx is cancelled and a wakeup
-		// waits too.
-		if ctx.Err() != nil {
-			return
-		}
-
-		err := c.writePendingCounts(context.WithoutCancel(ctx))
-		if err != nil {
-			c.log.Warn("failed to write pending counts", "error", err)
-		}
-	}
-}
-
-// addPendingCount adds n to pendingCount, one of the pending counts, and
-// wakes the goroutine that writes them. pendingCountsAdded has capacity one,
-// so a wakeup already waiting is enough.
-func (c *Cache) addPendingCount(pendingCount *atomic.Int64, n int64) {
-	pendingCount.Add(n)
-
-	select {
-	case c.pendingCountsAdded <- struct{}{}:
-	default:
-	}
-}
-
-// writePendingCounts adds the pending counts to the cache_stats row in one
-// UPDATE, then takes what it wrote out of them, leaving any added
-// meanwhile. When the UPDATE fails, they all stay pending.
-func (c *Cache) writePendingCounts(ctx context.Context) error {
-	c.pendingCountsWriteMutex.Lock()
-	defer c.pendingCountsWriteMutex.Unlock()
-
-	hits := c.pendingHits.Load()
-	misses := c.pendingMisses.Load()
-	upstreamFetches := c.pendingUpstreamFetches.Load()
-	upstreamFetchBytes := c.pendingUpstreamFetchBytes.Load()
-	transforms := c.pendingTransforms.Load()
-
-	if hits+misses+upstreamFetches+upstreamFetchBytes+transforms == 0 {
-		return nil
-	}
-
-	_, err := c.db.ExecContext(ctx, `
-		UPDATE cache_stats
-		SET hit_count = hit_count + ?,
-		    miss_count = miss_count + ?,
-		    upstream_fetch_count = upstream_fetch_count + ?,
-		    upstream_fetch_bytes = upstream_fetch_bytes + ?,
-		    transform_count = transform_count + ?,
-		    last_updated_at = CURRENT_TIMESTAMP
-		WHERE id = 1
-	`, hits, misses, upstreamFetches, upstreamFetchBytes, transforms)
-	if err != nil {
-		return fmt.Errorf("failed to write pending counts: %w", err)
-	}
-
-	c.pendingHits.Add(-hits)
-	c.pendingMisses.Add(-misses)
-	c.pendingUpstreamFetches.Add(-upstreamFetches)
-	c.pendingUpstreamFetchBytes.Add(-upstreamFetchBytes)
-	c.pendingTransforms.Add(-transforms)
-
-	return nil
-}

internal/imgcache/pendingcounts_internal_test.go

@@ -1,329 +0,0 @@
-package imgcache
-
-import (
-	"context"
-	"database/sql"
-	"errors"
-	"testing"
-	"time"
-
-	"sneak.berlin/go/pixa/internal/imagerequest"
-)
-
-// pendingCountsWait is how long a test waits for the database's connection,
-// for a count to reach the database, for a write to start waiting for the
-// connection, for a request to return past its deadline, or for
-// StopPendingCountWrites to return.
-const pendingCountsWait = 5 * time.Second
-
-// otherWritesWait is how long a test gives other writes, once one write waits
-// for the database's connection, to start waiting for it too.
-const otherWritesWait = 100 * time.Millisecond
-
-// holdDatabase takes the one connection of cache's database, so that every
-// other query waits for it, and returns the func that frees it.
-func holdDatabase(t *testing.T, cache *Cache) func() {
-	t.Helper()
-
-	ctx, cancel := context.WithTimeout(t.Context(), pendingCountsWait)
-	defer cancel()
-
-	conn, err := cache.db.Conn(ctx)
-	if err != nil {
-		t.Fatalf("failed to take the database connection: %v", err)
-	}
-
-	return func() { _ = conn.Close() }
-}
-
-// waitForConnectionWaits waits until db.Stats().WaitCount, the number of
-// times a caller has waited for a connection, reaches waits.
-func waitForConnectionWaits(t *testing.T, db *sql.DB, waits int64) {
-	t.Helper()
-
-	deadline := time.Now().Add(pendingCountsWait)
-
-	for db.Stats().WaitCount < waits {
-		if time.Now().After(deadline) {
-			t.Fatalf("callers waited for the database connection %d times, want %d",
-				db.Stats().WaitCount, waits)
-		}
-
-		time.Sleep(10 * time.Millisecond)
-	}
-}
-
-// waitForCounters waits until the cache_stats row holds want.
-func waitForCounters(t *testing.T, cache *Cache, want cacheStatsCounters) {
-	t.Helper()
-
-	deadline := time.Now().Add(pendingCountsWait)
-
-	for {
-		got := readCacheStatsCounters(t, cache)
-		if got == want {
-			return
-		}
-
-		if time.Now().After(deadline) {
-			t.Fatalf("counters = %+v, want %+v", got, want)
-		}
-
-		time.Sleep(10 * time.Millisecond)
-	}
-}
-
-// TestService_Get_ReturnsByItsDeadlineWhileTheDatabaseIsBusy holds the
-// database's one connection while a request whose fetch is held reaches its
-// deadline. The request must return with the deadline's error while the
-// connection is still held, as it waits for no database write past its
-// deadline, and its miss must reach the database once the connection is free.
-func TestService_Get_ReturnsByItsDeadlineWhileTheDatabaseIsBusy(t *testing.T) {
-	t.Parallel()
-
-	svc, fixtures, fetcher := setupHeldFetchService(t)
-
-	svc.cache.StartPendingCountWrites()
-	defer func() { _ = svc.cache.StopPendingCountWrites(t.Context()) }()
-
-	// Room for the request to reach its fetch on a busy host
-	const timeout = 2 * time.Second
-
-	ctx, cancel := context.WithTimeout(t.Context(), timeout)
-	defer cancel()
-
-	deadline, _ := ctx.Deadline()
-
-	results := startGet(ctx, svc, photoVariant(fixtures, 85, imagerequest.FitCover))
-
-	// The request has made its database reads by the time it fetches.
-	select {
-	case <-fetcher.started:
-	case <-time.After(timeout):
-		t.Fatal("request did not reach its fetch by its deadline")
-	}
-
-	releaseDatabase := holdDatabase(t, svc.cache)
-	defer releaseDatabase()
-
-	select {
-	case got := <-results:
-		t.Logf("Get() returned %v after its deadline, error = %v",
-			time.Since(deadline), got.err)
-
-		if !errors.Is(got.err, context.DeadlineExceeded) {
-			t.Errorf("Get() error = %v, want %v", got.err, context.DeadlineExceeded)
-		}
-	case <-time.After(time.Until(deadline) + pendingCountsWait):
-		t.Fatal("request did not return while the database was busy")
-	}
-
-	releaseDatabase()
-
-	waitForCounters(t, svc.cache, cacheStatsCounters{missCount: 1})
-}
-
-// TestIncrementStats_ManyCountsPastTheirDeadlineLeaveOneWriteWaiting holds the
-// database's one connection while many misses are counted past their
-// deadline. At most one write may then wait for the connection, and every
-// miss must reach the database once it is free.
-func TestIncrementStats_ManyCountsPastTheirDeadlineLeaveOneWriteWaiting(t *testing.T) {
-	t.Parallel()
-
-	cache, _ := newEvictionTestCache(t, 1<<30)
-
-	cache.StartPendingCountWrites()
-	defer func() { _ = cache.StopPendingCountWrites(t.Context()) }()
-
-	releaseDatabase := holdDatabase(t, cache)
-	defer releaseDatabase()
-
-	waitsBefore := cache.db.Stats().WaitCount
-
-	// Writes given a context past its deadline fail at once.
-	ended, cancel := context.WithDeadline(t.Context(), time.Now())
-	defer cancel()
-
-	const misses = 20
-
-	for range misses {
-		cache.IncrementStats(ended, false, 0)
-	}
-
-	// Once one write waits for the connection, any others start waiting too.
-	waitForConnectionWaits(t, cache.db, waitsBefore+1)
-	time.Sleep(otherWritesWait)
-
-	waits := cache.db.Stats().WaitCount - waitsBefore
-	t.Logf("%d writes waited for the database", waits)
-
-	if waits > 1 {
-		t.Errorf("%d writes waited for the database, want at most 1", waits)
-	}
-
-	releaseDatabase()
-
-	waitForCounters(t, cache, cacheStatsCounters{missCount: misses})
-}
-
-// TestStats_IncludesPendingCounts counts hits and misses whose writes miss
-// their deadline, and checks that Stats adds them to the database's counts
-// before they are written.
-func TestStats_IncludesPendingCounts(t *testing.T) {
-	t.Parallel()
-
-	cache, _ := newEvictionTestCache(t, 1<<30)
-
-	_, err := cache.db.ExecContext(t.Context(),
-		`UPDATE cache_stats SET hit_count = 75, miss_count = 25 WHERE id = 1`)
-	if err != nil {
-		t.Fatal(err)
-	}
-
-	// Writes given a context past its deadline fail at once.
-	ended, cancel := context.WithDeadline(t.Context(), time.Now())
-	defer cancel()
-
-	cache.IncrementStats(ended, true, 0)
-	cache.IncrementStats(ended, true, 0)
-	cache.IncrementStats(ended, false, 0)
-
-	stats, err := cache.Stats(t.Context())
-	if err != nil {
-		t.Fatalf("Stats() error = %v", err)
-	}
-
-	if stats.HitCount != 77 || stats.MissCount != 26 {
-		t.Errorf("HitCount = %d, MissCount = %d, want 77 and 26",
-			stats.HitCount, stats.MissCount)
-	}
-
-	want := cacheStatsCounters{hitCount: 75, missCount: 25}
-	if got := readCacheStatsCounters(t, cache); got != want {
-		t.Errorf("counters in the database = %+v, want %+v", got, want)
-	}
-}
-
-// TestStopPendingCountWrites_WritesPendingCounts holds the database's one
-// connection while the goroutine that writes the pending counts waits for it,
-// counts more, then stops that goroutine. The stop must leave the write under
-// way to finish rather than cut it off, and once the connection is free it
-// must write the counts that write left.
-func TestStopPendingCountWrites_WritesPendingCounts(t *testing.T) {
-	t.Parallel()
-
-	cache, _ := newEvictionTestCache(t, 1<<30)
-
-	cache.StartPendingCountWrites()
-	defer func() { _ = cache.StopPendingCountWrites(t.Context()) }()
-
-	releaseDatabase := holdDatabase(t, cache)
-	defer releaseDatabase()
-
-	waitsBefore := cache.db.Stats().WaitCount
-
-	// Writes given a context past its deadline fail at once.
-	ended, cancel := context.WithDeadline(t.Context(), time.Now())
-	defer cancel()
-
-	cache.IncrementStats(ended, true, 0)
-
-	// The goroutine's write waits for the connection.
-	waitForConnectionWaits(t, cache.db, waitsBefore+1)
-
-	// Counted after that write read the pending counts
-	cache.IncrementStats(ended, false, 0)
-	cache.IncrementUpstreamFetch(ended, 1024)
-	cache.IncrementTransformCount(ended)
-
-	// The goroutine is told to finish here as well as by the stop, so that it
-	// is told before its write under way ends, however late the stop runs.
-	// The counts above then reach the database only through the stop's own
-	// write.
-	cache.pendingCountsCancel()
-
-	stopped := make(chan error, 1)
-
-	go func() {
-		stopped <- cache.StopPendingCountWrites(t.Context())
-	}()
-
-	// Had the stop cut off the write under way, its own write would wait for
-	// the connection too.
-	time.Sleep(otherWritesWait)
-
-	waits := cache.db.Stats().WaitCount - waitsBefore
-	t.Logf("%d writes waited for the database", waits)
-
-	if waits > 1 {
-		t.Errorf("%d writes waited for the database, want 1: "+
-			"the stop cut off the write under way", waits)
-	}
-
-	releaseDatabase()
-
-	select {
-	case err := <-stopped:
-		if err != nil {
-			t.Fatalf("StopPendingCountWrites() error = %v", err)
-		}
-	case <-time.After(pendingCountsWait):
-		t.Fatal("StopPendingCountWrites() did not return once the database was free")
-	}
-
-	want := cacheStatsCounters{1, 1, 1, 1024, 1}
-	if got := readCacheStatsCounters(t, cache); got != want {
-		t.Errorf("counters = %+v, want %+v", got, want)
-	}
-}
-
-// TestStopPendingCountWrites_ReturnsWhenItsContextEnds holds the database's
-// one connection while the goroutine that writes the pending counts waits for
-// it, then stops that goroutine with a context that ends first. The stop must
-// return the context's error, and the write under way must still reach the
-// database once the connection is free.
-func TestStopPendingCountWrites_ReturnsWhenItsContextEnds(t *testing.T) {
-	t.Parallel()
-
-	cache, _ := newEvictionTestCache(t, 1<<30)
-
-	cache.StartPendingCountWrites()
-	defer func() { _ = cache.StopPendingCountWrites(t.Context()) }()
-
-	releaseDatabase := holdDatabase(t, cache)
-	defer releaseDatabase()
-
-	waitsBefore := cache.db.Stats().WaitCount
-
-	// Writes given a context past its deadline fail at once.
-	ended, cancel := context.WithDeadline(t.Context(), time.Now())
-	defer cancel()
-
-	cache.IncrementStats(ended, true, 0)
-
-	// The goroutine's write waits for the connection.
-	waitForConnectionWaits(t, cache.db, waitsBefore+1)
-
-	stopCtx, cancelStop := context.WithCancel(t.Context())
-	cancelStop()
-
-	stopped := make(chan error, 1)
-
-	go func() {
-		stopped <- cache.StopPendingCountWrites(stopCtx)
-	}()
-
-	select {
-	case err := <-stopped:
-		if !errors.Is(err, context.Canceled) {
-			t.Errorf("StopPendingCountWrites() error = %v, want %v",
-				err, context.Canceled)
-		}
-	case <-time.After(pendingCountsWait):
-		t.Fatal("StopPendingCountWrites() did not return once its context ended")
-	}
-
-	releaseDatabase()
-
-	waitForCounters(t, cache, cacheStatsCounters{hitCount: 1})
-}

internal/imgcache/source_content_internal_test.go

@@ -7,15 +7,14 @@ import (
 	"path/filepath"
 	"testing"
 
-	"sneak.berlin/go/pixa/internal/httpfetcher"
 	"sneak.berlin/go/pixa/internal/imagerequest"
 )
 
-// TestStoreSource_StoresIdenticalBytesOfTwoURLsTwice stores the same bytes
+// TestPutSource_StoresIdenticalBytesOfTwoURLsTwice stores the same bytes
 // for two source URLs and checks that each URL has a source_content row of
 // its own and a file of its own, named by its source key under
 // cache/sources/<ab>/<cd>/.
-func TestStoreSource_StoresIdenticalBytesOfTwoURLsTwice(t *testing.T) {
+func TestPutSource_StoresIdenticalBytesOfTwoURLsTwice(t *testing.T) {
 	t.Parallel()
 
 	cache, stateDir := newEvictionTestCache(t, 1<<30)
@@ -32,10 +31,8 @@ func TestStoreSource_StoresIdenticalBytesOfTwoURLsTwice(t *testing.T) {
 	}
 
 	for sourcePath, key := range keys {
-		rows := countRows(t, cache, `
-			SELECT COUNT(*) FROM source_content
-			WHERE source_key = ? AND source_host = ? AND source_path = ?
-		`, string(key), testHostCDN, sourcePath)
+		rows := countRows(t, cache,
+			`SELECT COUNT(*) FROM source_content WHERE source_key = ?`, string(key))
 		if rows != 1 {
 			t.Errorf("%s has %d source_content rows, want 1", sourcePath, rows)
 		}
@@ -87,18 +84,12 @@ func TestEvictSourceBlob_LeavesSourceWithIdenticalBytesServed(t *testing.T) {
 		t.Errorf("the evicted source still has %d source_content rows", n)
 	}
 
-	kept, contentType, err := cache.LookupSource(t.Context(), &imagerequest.Request{
-		Source: imagerequest.Source{Host: testHostCDN, Path: "/kept.jpg"},
-	})
-	t.Logf("LookupSource() of the kept source = %q, %q, error %v", kept, contentType, err)
+	reader, _, err := cache.Source(t.Context(),
+		imagerequest.Source{Host: testHostCDN, Path: "/kept.jpg"})
+	t.Logf("Source() of the kept source: error %v", err)
 
-	if err != nil || kept == "" {
-		t.Fatalf("LookupSource() of the kept source = %q, %v; want it cached", kept, err)
-	}
-
-	reader, _, err := cache.GetSourceContent(kept)
 	if err != nil {
-		t.Fatalf("GetSourceContent() of the kept source error = %v", err)
+		t.Fatalf("Source() of the kept source error = %v; want it cached", err)
 	}
 
 	defer func() { _ = reader.Close() }()
@@ -114,10 +105,10 @@ func TestEvictSourceBlob_LeavesSourceWithIdenticalBytesServed(t *testing.T) {
 	}
 }
 
-// TestStoreSource_FailedRowWriteLeavesNoFile drops source_content, so that
-// StoreSource cannot write the source's row, and checks that the store fails
+// TestPutSource_FailedRowWriteLeavesNoFile drops source_content, so that
+// PutSource cannot write the source's row, and checks that the store fails
 // and leaves no file under the source's key.
-func TestStoreSource_FailedRowWriteLeavesNoFile(t *testing.T) {
+func TestPutSource_FailedRowWriteLeavesNoFile(t *testing.T) {
 	t.Parallel()
 
 	cache, _ := newEvictionTestCache(t, 1<<30)
@@ -130,14 +121,12 @@ func TestStoreSource_FailedRowWriteLeavesNoFile(t *testing.T) {
 	req := &imagerequest.Request{
 		Source: imagerequest.Source{Host: testHostCDN, Path: testPathCat},
 	}
-	result := &httpfetcher.FetchResult{ContentType: testContentTypeJPEG}
-
-	key, err := cache.StoreSource(t.Context(), req,
-		bytes.NewReader([]byte("source bytes")), result)
-	t.Logf("StoreSource() = %q, error = %v", key, err)
+	key, err := cache.PutSource(t.Context(), req.Source,
+		bytes.NewReader([]byte("source bytes")))
+	t.Logf("PutSource() = %q, error = %v", key, err)
 
 	if err == nil {
-		t.Error("StoreSource() error = nil, want the failed row write")
+		t.Error("PutSource() error = nil, want the failed row write")
 	}
 
 	if cache.srcContent.Exists(req.Source.Key()) {