pixa/TODO.md

Workflow

• branch per issue from next
• do the work in Next Step
• move Next Step to the top of Completed Steps
• move the top item of Future Steps into Next Step
• commit (TODO.md changes in the same commit as the work)
• open a PR based on next
• an independent reviewer who did not write the change gates it
• the manager squash-merges the PR into next once review passes
• next stays green and mergeable to main at any time; only the owner merges next into main, via the single milestone PR
• push

Status

pre-1.0. No git tags exist. The 1.0.0 milestone is in progress; work lands on next, and main receives only the milestone PR that the owner merges. next is at the canonical golangci-lint v2.12.2 config and is green. Recent work extracted the internal/magic, internal/allowlist, internal/httpfetcher, and internal/signature packages. The gosec findings from the 2026-07-06 survey are resolved. The disk cache is now size-bounded with LRU eviction (cache_max_bytes), closing the unbounded disk growth DoS vector.

Next Step

P2: security: referer blacklist

Completed Steps

• 2026-09-29 variant content types kept in memory (closes #70): Cache.metaCache holds the content types of up to 10,000 variants in an LRU (github.com/hashicorp/golang-lru/v2), filled by StoreVariant and by GetVariant after it reads a .meta file, where a type StoreVariant added meanwhile is kept over the one read; for a variant it holds, Lookup skips the check of the disk and GetVariant skips the .meta read, still opening the variant file and taking the size from it; eviction removes the entry before deleting the files, and GetVariant removes it when the file will not open; the cap is a constant, not a setting; the unused variantMeta type is gone; README.md describes it.
• 2026-09-29 maintenance mode refuses image requests (closes #71): while maintenance_mode is on, /v1/image/ and /v1/e/ answer 503 with a Retry-After header and the JSON error body, from one middleware in internal/server/routes.go; the health check stays 200 and reports maintenance_mode, as the image's Docker HEALTHCHECK requests it and upaas marks a deploy failed when its container is unhealthy; the login and URL generator pages and /metrics keep working; documented in README.md and config.example.yml.
• 2026-09-29 bound concurrent image processing and upstream fetches (closes #64): max_concurrent_processing (default the number of CPUs pixa can use) limits the images decoded and encoded at once, and upstream_connections (default 64) the connections to all upstream hosts together, on top of upstream_connections_per_host; a fetch holds its connection until its image has been processed, and a request whose source is cached reads it only once it has a processing slot; a request that finds either limit reached waits up to 10 seconds for a free one, then gets 503 server busy, try again later; libvips runs one worker thread per image with its operation cache off; documented in README.md and config.example.yml.
• 2026-09-29 Dockerfiles install through script/bootstrap (closes #95): the Dockerfile lint and build stages and Dockerfile.lint copy script/, go.mod and go.sum, then run script/bootstrap in place of their own apk add lines, so the build dependencies are listed in one place; script/bootstrap now also installs a C compiler when gcc is missing; the build uses -trimpath and -s -w and keeps CGO_ENABLED=1 for govips; ARG VERSION sits just above the build, so a new version reruns neither script/bootstrap nor the tests.
• 2026-09-29 migrations at the path REPO_POLICIES.md sets (closes #96): the migration files moved, contents unchanged, from internal/database/schema/ to internal/db/migrations/ as 000_migration.sql and 001_schema.sql; the internal/db/migrations package embeds them and internal/database reads them through its FS(); the internal/database package itself stays; the version still comes from the filename prefix, so a database that has recorded versions 0 and 1 runs neither again.
• 2026-09-29 trusted_proxies advice and signature padding in README.md (closes #150): the login-limit paragraph, the trusted_proxies entry and config.example.yml say to set trusted_proxies to the address pixa sees for requests that come through the proxy, which the request log shows as remoteIP while it is not trusted; for a proxy on the Docker host that connects over 127.0.0.1 that is the Docker network's gateway, not the proxy's own address; the signature section says sig is base64url with the = padding kept, and gives the example's sig for a stated signing key.
• 2026-09-29 fixed uid and gid for pixad (closes #151): the image creates the pixad group with gid 65532 and the pixad user with uid 65532, instead of the first free uid 1000, so a bind-mounted /var/lib/pixa given to pixad is not owned on the host by a person's login account; the first-run step of "Running under upaas" in README.md names the uid and gid.
• 2026-09-29 max-age never outlives an expiring URL (closes #63): both image routes build Cache-Control from the request's Expires, which an encrypted URL's expiry now fills too; max-age is one year, or the whole seconds left until the exp of a /v1/image/ URL or the expiry of an encrypted URL when that is sooner, never negative; an allowlisted host's URL that has an exp follows it too; immutable stays, as freshness now ends at the expiry; documented in README.md.
• 2026-09-28 add the four settings README.md documented but pixa did not have, which aborted startup as unknown keys (closes #61): access_control_allow_origin (default *, the CORS origin), upstream_fetch_timeout (default 30s), upstream_max_response_size (default 50 MiB) and downstream_timeout (default 60s, both the server's write timeout and the per-request timeout); each has a PIXA_ variable; durations are positive Go duration strings, the size a whole number of bytes up to 1 GiB, the origin * or one http or https origin as README.md describes it; an invalid value aborts startup naming the key and the value; documented in config.example.yml and README.md.
• 2026-09-28 cache stats report real numbers (closes #56): Cache.Stats counts the cached source images and processed variants (source_content plus variant_content) and takes their size from Cache.UsageBytes, instead of reading request_cache and output_content, which nothing writes; those two tables are left in the schema; a disabled disk cache reports no items and no size. A hit is counted even when the request context has ended. A miss is counted after it is served or fails, also when the request context has ended by then, with the bytes it read from upstream, so upstream_fetch_count and upstream_fetch_bytes move, including for an upstream body that fails partway or a fetched source that then fails the magic byte check; transform_count counts each image the image processor transcodes.
• 2026-09-28 strip metadata from processed images (closes #82): every output is exported with govips' StripMetadata, so it carries no EXIF, XMP, IPTC or ICC profile; the image is first turned upright with AutoRotate (before sizes are worked out) and, when it has an ICC profile, converted to sRGB; the orig format is re-encoded and stripped like any other, as pixa never serves the source bytes; there is no setting to keep metadata; documented in README.md.
• 2026-09-28 rate limit the login form (closes #66): POST / is limited to 5 attempts per minute per client address, and an attempt over the limit is refused with 429 and a Retry-After header; the address is the one internal/clientip resolves through trusted_proxies, an IPv6 client is counted by its /64, and an IPv4-mapped address as the IPv4 address it carries; the limit is a RateLimit middleware in internal/middleware on github.com/go-chi/httprate, which the image routes can reuse; the library keeps counts for the current and the previous minute only; documented in README.md.
• 2026-09-28 refuse an unparseable exp on /v1/image/ and log swallowed cache errors (closes #72): an exp in the URL that is not a whole number, an empty exp= included, is a 400 naming exp and the value, instead of being ignored and answered with 401 as if the URL had no exp; only an exp missing from the URL is unchanged; README.md says so where it documents exp. A failed variant .meta write, source metadata JSON write, Stats count query, stats counter update, negative cache write or expired negative cache delete is now logged at warn with the path or key and the error, and stays non-fatal.
• 2026-09-28 refuse an empty fit on /v1/image/ (closes #139): a fit in the URL with an empty value (fit=) is a 400 naming fit, instead of being served as cover and verified against a signature made for cover; only a fit missing from the URL is still cover; any other value still goes through the existing fit-mode check; README.md says so where it documents fit.
• 2026-09-28 refuse an invalid q on /v1/image/ (closes #134): a q that is not a whole number from 1 to 100, an empty q included, is a 400 naming q and the value, instead of being served at the default 85; the route reads q with the generator's quality check (parseFormInt with minQuality and maxQuality); only a q missing from the URL is still 85; a query string that cannot be decoded, such as q=80%, is a 400 showing it; any query parameter given more than once (q, fit, sig, exp alike) is a 400 naming it, so none is read from its first value only; README.md states the range and both query-string rules.
• 2026-09-28 unknown PIXA_ environment variables abort startup (closes #133): a variable whose name starts with PIXA_ but is neither a setting's variable nor PIXA_CONFIG_PATH aborts startup naming it, as an unknown config key does, and PIXA_PORT is named with a pointer to PORT; the check runs after the config file loads, so the variables the file's env: section sets are checked too; documented in README.md.
• 2026-09-28 start on a fresh upaas volume (closes #129): the image starts as root only to give /var/lib/pixa to pixad when pixad does not own it (deploy/docker-entrypoint.sh), then runs the server as pixad through su-exec, so a root-owned host directory bind-mounted there no longer stops the container at startup; README.md gains a "Running under upaas" section.
• 2026-09-28 run all linting in Docker via Dockerfile.lint + script/lint (closes #104): make lint calls script/lint, the only way the linter is run; inside a container (both Dockerfiles set container=docker) it runs golangci-lint, anywhere else it builds the hash-pinned Dockerfile.lint, whose last step runs script/lint again; the Dockerfile lint stage runs make lint; no host or nix-shell golangci-lint path remains (script/bootstrap installs no linter); a per-run CACHEBUST build-arg keeps the lint step from being served from cache, and a tmpfs mount on that step keeps Go's and golangci-lint's caches out of its layer, so a run leaves no large build cache behind; golangci-lint config verify stays out, as it fetches its schema over an unpinned live HTTPS call
• 2026-09-28 every setting as an environment variable (closes #128, also covers #99): each config key can be set by PIXA_ plus the key in upper case (. written as _), and the port by PORT; a variable present in the environment, even empty, wins over the config file, which wins over the default; the typed getters read the variable first, so every existing check applies to it and a bad value aborts startup naming the variable; lists are comma-separated, and an empty variable (or "" in the file) is an empty list; the Docker image no longer bakes in config.docker.yml or passes --config, and its HEALTHCHECK probes PORT (default 8080); the config file is looked for under /etc/pixa and ~/.config/pixa instead of the daemon name pixad; documented in README.md and config.example.yml.
• 2026-09-28 quality and fit in the URL signature (closes #60): the signed data is now host:path:query:width:height:format:expiration:quality:fit, using 85 and cover when the URL has no q or fit, so one signed URL can no longer be replayed across other quality and fit values to create unauthorized cache entries and transcodes; the known-answer vectors in internal/signature/golden_test.go and the README signature specification describe the new format.
• 2026-09-28 Docker image healthcheck (closes #111): a HEALTHCHECK in the runtime stage probing /.well-known/healthcheck.json with busybox wget; script/docker-smoke (make docker-smoke) builds the image, starts it with a throwaway PIXA_SIGNING_KEY, and passes only once Docker reports it healthy within 30 seconds, removing the container on exit; the Gitea workflow runs it after script/cibuild.
• 2026-09-21 trusted-proxy client IP resolution (closes #94): a trusted_proxies config key taking a list of CIDRs, parsed by the same net/netip list parser as blocked_networks (an invalid entry aborts startup naming the key and value; an omitted key defaults to the RFC 1918 private ranges, an explicitly empty list trusts no one, and an explicit list replaces the default); a new internal/clientip package resolves the client address by honoring X-Forwarded-For only when the direct peer is a trusted proxy, walking the chain right-to-left to the rightmost non-proxy entry, so a client connecting directly cannot spoof its address; the resolved address is stored in the request context by a new middleware and used by the request-logging middleware and the login-attempt logs in place of the raw peer address; documented in README.md and config.example.yml.
• 2026-09-21 blocked networks configuration extending SSRF protection: a blocked_networks config key taking a list of CIDRs (parsed with net/netip, an invalid entry aborts startup naming the key and value), added to the built-in blocklist rather than replacing it; the built-in ranges extended to CGNAT 100.64.0.0/10, IETF protocol assignments 192.0.0.0/24, benchmark 198.18.0.0/15, and NAT64 64:ff9b::/96 (IPv4-mapped forms covered); enforcement stays in the dial-time re-resolution so the DNS-rebinding window remains closed; documented in README.md and config.example.yml.
• 2026-09-21 validate dimensions and fit mode on the encrypted-URL route and the token generator (closes #62): imgcache.ValidateDimension alone holds the MaxDimension bound and is used by the path parser, by the new ValidateImageRequest (which also applies ValidateFitMode) and by the generator; both the /v1/image/ and /v1/e/ routes call ValidateImageRequest, so an over-limit size or an unknown fit mode is a 400 rather than an out-of-memory or a 500 from the processor; the URL generator answers 400 naming the field for a width or height that is not a number or fails the shared check, a quality that is not a number from 1 to 100, a ttl that is not a number from 0 to the largest number of seconds the expiry calculation can hold, or an unknown fit; an empty quality is 85 and an empty ttl never expires; the form's width and height inputs stop at 8192
• 2026-09-21 http.Server hardening (closes #92): added HTTPReadHeaderTimeout (10s, bounds the slowloris header dribble) and HTTPIdleTimeout (120s, bounds keep-alive reuse) alongside the existing timeouts and wired them onto the server; added a LimitBody middleware capping the two form POST bodies (POST /, POST /generate) at MaxFormBytes (1 MiB) and returning 413, applied ahead of the CSRF middleware so an oversized body is refused as 413 rather than being read as a missing CSRF token (403); left WriteTimeout at 60s unchanged
• 2026-08-07 update golangci-lint to v2.12.2 with the canonical .golangci.yml (v2 schema, default: all minus six disabled linters, lll 88, tests included): bumped the pinned golangci/golangci-lint:v2.12.2-alpine image in Dockerfile and the release-archive sha256 pins in script/bootstrap; fixed the findings the stricter config surfaced (notably paralleltest, wsl_v5, goconst, lll, noinlineerr, err113, errcheck, testpackage — white-box test files renamed to *_internal_test.go), including #55's code absorbed after it merged, iterating the pinned linter to 0 issues.; no single finding total is substantiable, since golangci-lint's uniq-by-line reveals new findings on a line as others there are fixed — the documented re-measurements were 81 after the #53 merge and 149 after the #55 merge; three behavior changes, so not a pure no-op: Cache.StoreVariant now takes a context.Context (noctx), so a cancelled request skips its best-effort accounting row; MetadataStorage.Store's cleanup defer was dead on main and leaked .tmp-*.json on failure, now fixed with explicit removals; and the signing_key validation error text gained value too short: ; the eviction loop's uncancellable context is deferred to #102 under a //nolint:contextcheck; three //nolint:tagliatelle directives keep the snake_case JSON wire/disk formats unchanged; make check green
• 2026-08-07 implement cache size management and eviction (closes #51): new cache_max_bytes config key validated by the startup framework (explicit values used exactly with no floor, 0 disables the disk cache entirely, omitted defaults to max(75% of free space on the filesystem containing <state_dir>/cache/, 500 MiB), logged at startup); processed variants are now tracked in the database (a new variant_content table and an LRU timestamp on source_content) so total usage is two SUMs, never a directory scan on the hot path; a background goroutine evicts globally least-recently-used entries (variants and source blobs merged) to the limit, woken by a periodic ticker and by write-pressure notifications from stores; a source blob and ALL of its source_metadata references are deleted in one transaction before the file is unlinked, so multi-referenced blobs are never removed while referenced and rows never point at deleted files; a startup and periodic reconciliation pass adopts untracked variant files, drops rows for missing files, removes unreachable source blobs, and sweeps stale temp files
• 2026-08-07 validate configuration on startup, fail fast on bad config (closes #52): a config value that is set but unparseable or invalid aborts startup naming the key and value (defaults apply only to omitted keys), unknown config keys abort startup, a malformed config file aborts instead of being skipped, and state_dir is verified creatable and writable before the listener binds
• 2026-08-07 manual test pass of the auth and encrypted URL flows against a locally built and running pixad (built from main at 6573b9d, port 18099, local throwaway config); all six checks passed, plus all nine tests in scripts/manual-test.sh (closes #49):
  • visit / and see the login form: HTTP 200, Pixa - Login page with name="key" password form
  • wrong key shows an error: POST / with key=wrong-key returned HTTP 200 login page containing "Invalid signing key"
  • correct signing key shows the generator form: POST / returned HTTP 303 to / with Set-Cookie: pixa_session=...; HttpOnly; Secure; SameSite=Strict; GET / with that cookie rendered Pixa - URL Generator with the /generate form and logout link
  • a generated encrypted URL serves the image: POST /generate (ttl=3600) produced a /v1/e/<token>/img.jpeg URL that returned HTTP 200, Content-Type: image/jpeg, an 800x600 baseline JPEG of 61706 bytes
  • an expired URL (short TTL) returns 410: a ttl=1 URL fetched after 3 s returned HTTP 410 Gone with {"error":"URL has expired","status":410,...}
  • logout redirects back to login: GET /logout returned HTTP 303 to / with Set-Cookie: pixa_session=; Max-Age=0; subsequent GET / rendered the login form again
• 2026-08-07 fix the two remaining gosec findings (G124 in internal/session): session cookies now always carry Secure/HttpOnly/SameSite=Strict on both the set and clear paths; make check green (closes #47)
• 2026-07-07 Adopted scripts-to-rule-them-all: script/ entrypoints, Makefile shims, README Entrypoints section
• 2026-04-07 extract magic byte detection into internal/magic (#42)
• 2026-03-25 extract allowlist package from internal/imgcache (#41)
• 2026-03-25 move schema_migrations table creation into 000.sql (#36)
• 2026-03-20 enforce and document exact-match-only signature verification (#40)
• 2026-03-20 bound imageprocessor.Process input read to prevent unbounded memory use (#37); consolidate appname into an internal/globals constant (#34)
• 2026-03-18 parse version prefix from migration filenames (#33)
• 2026-03-15 QA audit fixes for 1.0/MVP readiness (#25)
• 2026-03-02 split Dockerfile with pre-built golangci-lint stage for faster CI (#23)
• 2026-02-25 repo policy compliance: CI workflow, hash-pinned images, golangci-lint and gosec fixes of that date (#14); arm64 Docker build fix (#16)
• 2026-01-08 WebP and AVIF encoding support via govips (both former P0 image processing items, now done)

Future Steps

• P2: security
  • per-IP rate limiting on the image routes
  • per-origin rate limiting
• P2: HTTP response handling
  • Last-Modified headers
  • Vary header for content negotiation
  • X-Request-ID propagation
• P2: auto format selection (format=auto based on Accept header)
• P2: configuration
  • YAML config file support
• P2: operational
  • optional Sentry error reporting
  • comprehensive request logging
  • Prometheus performance metrics
  • integration tests for the image proxy flow
  • load tests to verify the 1k to 5k req/s target
• P2: documentation
  • configuration options
  • API endpoints
  • deployment guide
  • example nginx or caddy reverse proxy config