pixa/TODO.md

# Workflow

* branch per issue from `next`
* do the work in Next Step
* move Next Step to the top of Completed Steps
* move the top item of Future Steps into Next Step
* commit (`TODO.md` changes in the same commit as the work)
* open a PR based on `next`
* an independent reviewer who did not write the change gates it
* the manager squash-merges the PR into `next` once review passes
* `next` stays green and mergeable to `main` at any time; only the owner
  merges `next` into `main`, via the single milestone PR
* push

# Status

pre-1.0. No git tags exist. The `1.0.0` milestone is in progress; work
lands on `next`, and `main` receives only the milestone PR that the
owner merges. `next` is at the canonical `golangci-lint` v2.12.2 config
and is green. Recent work extracted the internal/magic,
internal/allowlist, internal/httpfetcher, and internal/signature
packages. The gosec findings from the 2026-07-06 survey are resolved.
The disk cache is now size-bounded with LRU eviction
(`cache_max_bytes`), closing the unbounded disk growth DoS vector.

# Next Step

P2: security: referer blacklist

# Completed Steps

- 2026-09-29 variant content types kept in memory (closes #70):
  `Cache.metaCache` holds the content types of up to 10,000 variants in an LRU
  (`github.com/hashicorp/golang-lru/v2`), filled by `StoreVariant` and by
  `GetVariant` after it reads a `.meta` file, where a type `StoreVariant` added
  meanwhile is kept over the one read; for a variant it holds, `Lookup`
  skips the check of the disk and `GetVariant` skips the `.meta` read, still
  opening the variant file and taking the size from it; eviction removes the
  entry before deleting the files, and `GetVariant` removes it when the file
  will not open; the cap is a constant, not a setting; the unused `variantMeta`
  type is gone; `README.md` describes it.
- 2026-09-29 maintenance mode refuses image requests (closes #71): while
  `maintenance_mode` is on, `/v1/image/` and `/v1/e/` answer 503 with a
  `Retry-After` header and the JSON error body, from one middleware in
  `internal/server/routes.go`; the health check stays 200 and reports
  `maintenance_mode`, as the image's Docker `HEALTHCHECK` requests it and upaas
  marks a deploy failed when its container is unhealthy; the login and URL
  generator pages and `/metrics` keep working; documented in `README.md` and
  `config.example.yml`.
- 2026-09-29 bound concurrent image processing and upstream fetches (closes
  #64): `max_concurrent_processing` (default the number of CPUs pixa can use)
  limits the images decoded and encoded at once, and `upstream_connections`
  (default 64) the connections to all upstream hosts together, on top of
  `upstream_connections_per_host`; a fetch holds its connection until its image
  has been processed, and a request whose source is cached reads it only once it
  has a processing slot; a request that finds either limit reached waits up to
  10 seconds for a free one, then gets 503 `server busy, try again later`;
  libvips runs one worker thread per image with its operation cache off;
  documented in `README.md` and `config.example.yml`.
- 2026-09-29 Dockerfiles install through `script/bootstrap` (closes #95): the
  `Dockerfile` lint and build stages and `Dockerfile.lint` copy `script/`,
  `go.mod` and `go.sum`, then run `script/bootstrap` in place of their own
  `apk add` lines, so the build dependencies are listed in one place;
  `script/bootstrap` now also installs a C compiler when `gcc` is missing; the
  build uses `-trimpath` and `-s -w` and keeps `CGO_ENABLED=1` for govips;
  `ARG VERSION` sits just above the build, so a new version reruns neither
  `script/bootstrap` nor the tests.
- 2026-09-29 migrations at the path `REPO_POLICIES.md` sets (closes #96): the
  migration files moved, contents unchanged, from `internal/database/schema/`
  to `internal/db/migrations/` as `000_migration.sql` and `001_schema.sql`; the
  `internal/db/migrations` package embeds them and `internal/database` reads
  them through its `FS()`; the `internal/database` package itself stays; the
  version still comes from the filename prefix, so a database that has recorded
  versions 0 and 1 runs neither again.
- 2026-09-29 `trusted_proxies` advice and signature padding in `README.md`
  (closes #150): the login-limit paragraph, the `trusted_proxies` entry and
  `config.example.yml` say to set `trusted_proxies` to the address pixa sees for
  requests that come through the proxy, which the request log shows as
  `remoteIP` while it is not trusted; for a proxy on the Docker host that
  connects over `127.0.0.1` that is the Docker network's gateway, not the
  proxy's own address; the signature section says `sig` is base64url with the
  `=` padding kept, and gives the example's `sig` for a stated signing key.
- 2026-09-29 fixed uid and gid for `pixad` (closes #151): the image creates the
  `pixad` group with gid 65532 and the `pixad` user with uid 65532, instead of
  the first free uid 1000, so a bind-mounted `/var/lib/pixa` given to `pixad`
  is not owned on the host by a person's login account; the first-run step of
  "Running under upaas" in `README.md` names the uid and gid.
- 2026-09-29 `max-age` never outlives an expiring URL (closes #63): both image
  routes build `Cache-Control` from the request's `Expires`, which an encrypted
  URL's expiry now fills too; `max-age` is one year, or the whole seconds left
  until the `exp` of a `/v1/image/` URL or the expiry of an encrypted URL when
  that is sooner, never negative; an allowlisted host's URL that has an `exp`
  follows it too; `immutable` stays, as freshness now ends at the expiry;
  documented in `README.md`.
- 2026-09-28 add the four settings `README.md` documented but pixa did not
  have, which aborted startup as unknown keys (closes #61):
  `access_control_allow_origin` (default `*`, the CORS origin),
  `upstream_fetch_timeout` (default `30s`), `upstream_max_response_size`
  (default 50 MiB) and `downstream_timeout` (default `60s`, both the
  server's write timeout and the per-request timeout); each has a
  `PIXA_` variable; durations are positive Go duration strings, the size a
  whole number of bytes up to 1 GiB, the origin `*` or one `http` or
  `https` origin as `README.md` describes it; an invalid value
  aborts startup naming the key and the value; documented in
  `config.example.yml` and `README.md`.
- 2026-09-28 cache stats report real numbers (closes #56): `Cache.Stats`
  counts the cached source images and processed variants (`source_content`
  plus `variant_content`) and takes their size from `Cache.UsageBytes`,
  instead of reading `request_cache` and `output_content`, which nothing
  writes; those two tables are left in the schema; a disabled disk cache
  reports no items and no size. A hit is counted even when the request
  context has ended. A miss is counted after it is served or fails, also
  when the request context has ended by then, with the bytes it read from
  upstream, so `upstream_fetch_count` and `upstream_fetch_bytes` move,
  including for an upstream body that fails partway or a fetched source
  that then fails the magic byte check; `transform_count` counts each image
  the image processor transcodes.
- 2026-09-28 strip metadata from processed images (closes #82): every output is
  exported with govips' `StripMetadata`, so it carries no EXIF, XMP, IPTC or ICC
  profile; the image is first turned upright with `AutoRotate` (before sizes are
  worked out) and, when it has an ICC profile, converted to sRGB; the `orig`
  format is re-encoded and stripped like any other, as pixa never serves the
  source bytes; there is no setting to keep metadata; documented in `README.md`.
- 2026-09-28 rate limit the login form (closes #66): `POST /` is limited to 5
  attempts per minute per client address, and an attempt over the limit is
  refused with 429 and a `Retry-After` header; the address is the one
  `internal/clientip` resolves through `trusted_proxies`, an IPv6 client is
  counted by its /64, and an IPv4-mapped address as the IPv4 address it
  carries; the limit is a `RateLimit` middleware in `internal/middleware` on
  `github.com/go-chi/httprate`, which the image routes can reuse; the library
  keeps counts for the current and the previous minute only; documented in
  `README.md`.
- 2026-09-28 refuse an unparseable `exp` on `/v1/image/` and log swallowed
  cache errors (closes #72): an `exp` in the URL that is not a whole
  number, an empty `exp=` included, is a 400 naming `exp` and the value,
  instead of being ignored and answered with 401 as if the URL had no
  `exp`; only an `exp` missing from the URL is unchanged; `README.md` says
  so where it documents `exp`. A failed variant `.meta` write, source
  metadata JSON write, `Stats` count query, stats counter update, negative
  cache write or expired negative cache delete is now logged at `warn`
  with the path or key and the error, and stays non-fatal.
- 2026-09-28 refuse an empty `fit` on `/v1/image/` (closes #139): a
  `fit` in the URL with an empty value (`fit=`) is a 400 naming `fit`,
  instead of being served as `cover` and verified against a signature
  made for `cover`; only a `fit` missing from the URL is still `cover`;
  any other value still goes through the existing fit-mode check;
  `README.md` says so where it documents `fit`.
- 2026-09-28 refuse an invalid `q` on `/v1/image/` (closes #134): a `q`
  that is not a whole number from 1 to 100, an empty `q` included, is a
  400 naming `q` and the value, instead of being served at the default
  85; the route reads `q` with the generator's quality check
  (`parseFormInt` with `minQuality` and `maxQuality`); only a `q` missing
  from the URL is still 85; a query string that cannot be decoded, such
  as `q=80%`, is a 400 showing it; any query parameter given more than
  once (`q`, `fit`, `sig`, `exp` alike) is a 400 naming it, so none is
  read from its first value only; `README.md` states the range and both
  query-string rules.
- 2026-09-28 unknown `PIXA_` environment variables abort startup (closes
  #133): a variable whose name starts with `PIXA_` but is neither a
  setting's variable nor `PIXA_CONFIG_PATH` aborts startup naming it, as
  an unknown config key does, and `PIXA_PORT` is named with a pointer to
  `PORT`; the check runs after the config file loads, so the variables
  the file's `env:` section sets are checked too; documented in
  `README.md`.
- 2026-09-28 start on a fresh upaas volume (closes #129): the image
  starts as root only to give `/var/lib/pixa` to `pixad` when `pixad`
  does not own it (`deploy/docker-entrypoint.sh`), then runs the server
  as `pixad` through `su-exec`, so a root-owned host directory
  bind-mounted there no longer stops the container at startup;
  `README.md` gains a "Running under upaas" section.
- 2026-09-28 run all linting in Docker via `Dockerfile.lint` +
  `script/lint` (closes #104): `make lint` calls `script/lint`, the only
  way the linter is run; inside a container (both Dockerfiles set
  `container=docker`) it runs `golangci-lint`, anywhere else it builds the
  hash-pinned `Dockerfile.lint`, whose last step runs `script/lint` again;
  the `Dockerfile` lint stage runs `make lint`; no host or nix-shell
  `golangci-lint` path remains (`script/bootstrap` installs no linter);
  a per-run `CACHEBUST` build-arg keeps the lint step from being served
  from cache, and a tmpfs mount on that step keeps Go's and
  golangci-lint's caches out of its layer, so a run leaves no large build
  cache behind; `golangci-lint config verify` stays out, as it fetches its
  schema over an unpinned live HTTPS call
- 2026-09-28 every setting as an environment variable (closes #128, also
  covers #99): each config key can be set by `PIXA_` plus the key in upper
  case (`.` written as `_`), and the port by `PORT`; a variable present in
  the environment, even empty, wins over the config file, which wins over
  the default; the typed getters read the variable first, so every existing
  check applies to it and a bad value aborts startup naming the variable;
  lists are comma-separated, and an empty variable (or `""` in the file) is
  an empty list; the Docker image no longer bakes in `config.docker.yml` or
  passes `--config`, and its `HEALTHCHECK` probes `PORT` (default `8080`);
  the config file is looked for under `/etc/pixa` and `~/.config/pixa`
  instead of the daemon name `pixad`; documented in `README.md` and
  `config.example.yml`.
- 2026-09-28 quality and fit in the URL signature (closes #60): the signed
  data is now `host:path:query:width:height:format:expiration:quality:fit`,
  using `85` and `cover` when the URL has no `q` or `fit`, so one signed
  URL can no longer be replayed across other quality and fit values to
  create unauthorized cache entries and transcodes; the known-answer
  vectors in `internal/signature/golden_test.go` and the README signature
  specification describe the new format.
- 2026-09-28 Docker image healthcheck (closes #111): a `HEALTHCHECK` in
  the runtime stage probing `/.well-known/healthcheck.json` with busybox
  `wget`; `script/docker-smoke` (`make docker-smoke`) builds the image,
  starts it with a throwaway `PIXA_SIGNING_KEY`, and passes only once
  Docker reports it healthy within 30 seconds, removing the container on
  exit; the Gitea workflow runs it after `script/cibuild`.
- 2026-09-21 trusted-proxy client IP resolution (closes #94): a
  `trusted_proxies` config key taking a list of CIDRs, parsed by the same
  `net/netip` list parser as `blocked_networks` (an invalid entry aborts
  startup naming the key and value; an omitted key defaults to the RFC 1918
  private ranges, an explicitly empty list trusts no one, and an explicit
  list replaces the default); a new
  `internal/clientip` package resolves the client address by honoring
  `X-Forwarded-For` only when the direct peer is a trusted proxy, walking
  the chain right-to-left to the rightmost non-proxy entry, so a client
  connecting directly cannot spoof its address; the resolved address is
  stored in the request context by a new middleware and used by the
  request-logging middleware and the login-attempt logs in place of the
  raw peer address; documented in `README.md` and `config.example.yml`.
- 2026-09-21 blocked networks configuration extending SSRF protection: a
  `blocked_networks` config key taking a list of CIDRs (parsed with
  `net/netip`, an invalid entry aborts startup naming the key and value),
  added to the built-in blocklist rather than replacing it; the built-in
  ranges extended to CGNAT `100.64.0.0/10`, IETF protocol assignments
  `192.0.0.0/24`, benchmark `198.18.0.0/15`, and NAT64 `64:ff9b::/96`
  (IPv4-mapped forms covered); enforcement stays in the dial-time
  re-resolution so the DNS-rebinding window remains closed; documented in
  `README.md` and `config.example.yml`.
- 2026-09-21 validate dimensions and fit mode on the encrypted-URL
  route and the token generator (closes #62): `imgcache.ValidateDimension`
  alone holds the `MaxDimension` bound and is used by the path parser, by
  the new `ValidateImageRequest` (which also applies `ValidateFitMode`)
  and by the generator; both the `/v1/image/` and `/v1/e/` routes call
  `ValidateImageRequest`, so an over-limit size or an unknown fit mode is a
  400 rather than an out-of-memory or a 500 from the processor; the URL
  generator answers 400 naming the field for a `width` or `height` that is
  not a number or fails the shared check, a `quality` that is not a number
  from 1 to 100, a `ttl` that is not a number from 0 to the largest number
  of seconds the expiry calculation can hold, or an unknown `fit`; an empty
  `quality` is 85 and
  an empty `ttl` never expires; the form's width and height inputs stop at
  8192
- 2026-09-21 http.Server hardening (closes #92): added
  `HTTPReadHeaderTimeout` (10s, bounds the slowloris header dribble) and
  `HTTPIdleTimeout` (120s, bounds keep-alive reuse) alongside the
  existing timeouts and wired them onto the server; added a `LimitBody`
  middleware capping the two form POST bodies (`POST /`, `POST /generate`)
  at `MaxFormBytes` (1 MiB) and returning 413, applied ahead of the CSRF
  middleware so an oversized body is refused as 413 rather than being read
  as a missing CSRF token (403); left `WriteTimeout` at 60s unchanged
- 2026-08-07 update golangci-lint to v2.12.2 with the canonical
  `.golangci.yml` (v2 schema, `default: all` minus six disabled
  linters, `lll` 88, tests included): bumped the pinned
  `golangci/golangci-lint:v2.12.2-alpine` image in `Dockerfile` and the
  release-archive sha256 pins in `script/bootstrap`; fixed the findings
  the stricter config surfaced (notably `paralleltest`, `wsl_v5`,
  `goconst`, `lll`, `noinlineerr`, `err113`, `errcheck`, `testpackage` —
  white-box test files renamed to `*_internal_test.go`), including #55's
  code absorbed after it merged, iterating the pinned linter to
  `0 issues.`; no single finding total is substantiable, since
  golangci-lint's `uniq-by-line` reveals new findings on a line as
  others there are fixed — the documented re-measurements were 81 after
  the #53 merge and 149 after the #55 merge; three behavior changes, so
  not a pure no-op: `Cache.StoreVariant` now takes a `context.Context`
  (`noctx`), so a cancelled request skips its best-effort accounting
  row; `MetadataStorage.Store`'s cleanup defer was dead on `main` and
  leaked `.tmp-*.json` on failure, now fixed with explicit removals; and
  the `signing_key` validation error text gained `value too short: `;
  the eviction loop's uncancellable context is deferred to #102 under a
  `//nolint:contextcheck`; three `//nolint:tagliatelle` directives keep
  the snake_case JSON wire/disk formats unchanged; `make check` green
- 2026-08-07 implement cache size management and eviction (closes
  #51): new `cache_max_bytes` config key validated by the startup
  framework (explicit values used exactly with no floor, `0` disables
  the disk cache entirely, omitted defaults to max(75% of free space
  on the filesystem containing `<state_dir>/cache/`, 500 MiB), logged
  at startup); processed variants are now tracked in the database (a
  new `variant_content` table and an LRU timestamp on `source_content`)
  so total usage is two SUMs, never a directory scan on the hot path; a
  background goroutine evicts globally least-recently-used entries
  (variants and source blobs merged) to the limit, woken by a periodic
  ticker and by write-pressure notifications from stores; a source
  blob and ALL of its `source_metadata` references are deleted in one
  transaction before the file is unlinked, so multi-referenced blobs
  are never removed while referenced and rows never point at deleted
  files; a startup and periodic reconciliation pass adopts untracked
  variant files, drops rows for missing files, removes unreachable
  source blobs, and sweeps stale temp files
- 2026-08-07 validate configuration on startup, fail fast on bad
  config (closes #52): a config value that is set but unparseable or
  invalid aborts startup naming the key and value (defaults apply only
  to omitted keys), unknown config keys abort startup, a malformed
  config file aborts instead of being skipped, and `state_dir` is
  verified creatable and writable before the listener binds
- 2026-08-07 manual test pass of the auth and encrypted URL flows
  against a locally built and running `pixad` (built from `main` at
  `6573b9d`, port 18099, local throwaway config); all six checks
  passed, plus all nine tests in `scripts/manual-test.sh` (closes #49):
    - [x] visit `/` and see the login form: HTTP 200, `Pixa - Login`
      page with `name="key"` password form
    - [x] wrong key shows an error: POST `/` with `key=wrong-key`
      returned HTTP 200 login page containing "Invalid signing key"
    - [x] correct signing key shows the generator form: POST `/`
      returned HTTP 303 to `/` with
      `Set-Cookie: pixa_session=...; HttpOnly; Secure; SameSite=Strict`;
      GET `/` with that cookie rendered `Pixa - URL Generator` with the
      `/generate` form and logout link
    - [x] a generated encrypted URL serves the image: POST `/generate`
      (ttl=3600) produced a `/v1/e/<token>/img.jpeg` URL that returned
      HTTP 200, `Content-Type: image/jpeg`, an 800x600 baseline JPEG of
      61706 bytes
    - [x] an expired URL (short TTL) returns 410: a ttl=1 URL fetched
      after 3 s returned HTTP 410 Gone with
      `{"error":"URL has expired","status":410,...}`
    - [x] logout redirects back to login: GET `/logout` returned HTTP
      303 to `/` with `Set-Cookie: pixa_session=; Max-Age=0`;
      subsequent GET `/` rendered the login form again
- 2026-08-07 fix the two remaining gosec findings (G124 in
  internal/session): session cookies now always carry
  Secure/HttpOnly/SameSite=Strict on both the set and clear paths;
  `make check` green (closes #47)
- 2026-07-07 Adopted scripts-to-rule-them-all: `script/` entrypoints,
  Makefile shims, README Entrypoints section
- 2026-04-07 extract magic byte detection into internal/magic (#42)
- 2026-03-25 extract allowlist package from internal/imgcache (#41)
- 2026-03-25 move schema_migrations table creation into 000.sql (#36)
- 2026-03-20 enforce and document exact-match-only signature
  verification (#40)
- 2026-03-20 bound imageprocessor.Process input read to prevent
  unbounded memory use (#37); consolidate appname into an
  internal/globals constant (#34)
- 2026-03-18 parse version prefix from migration filenames (#33)
- 2026-03-15 QA audit fixes for 1.0/MVP readiness (#25)
- 2026-03-02 split Dockerfile with pre-built golangci-lint stage for
  faster CI (#23)
- 2026-02-25 repo policy compliance: CI workflow, hash-pinned images,
  golangci-lint and gosec fixes of that date (#14); arm64 Docker build
  fix (#16)
- 2026-01-08 WebP and AVIF encoding support via govips (both former P0
  image processing items, now done)

# Future Steps

- P2: security
    - per-IP rate limiting on the image routes
    - per-origin rate limiting
- P2: HTTP response handling
    - Last-Modified headers
    - Vary header for content negotiation
    - X-Request-ID propagation
- P2: auto format selection (format=auto based on Accept header)
- P2: configuration
    - YAML config file support
- P2: operational
    - optional Sentry error reporting
    - comprehensive request logging
    - Prometheus performance metrics
    - integration tests for the image proxy flow
    - load tests to verify the 1k to 5k req/s target
- P2: documentation
    - configuration options
    - API endpoints
    - deployment guide
    - example nginx or caddy reverse proxy config