An omitted max_concurrent_processing, and New given none, must be half
the CPUs Go uses, rounded down, and at least 1. The tests set GOMAXPROCS
to 1, 2, 3 and 8 and fail until the default changes
(#242).
Model: opus-5-5
An empty, fractional, negative or non-numeric value must abort startup
naming the variable and the key, never fall back to the default. The
config loader already refuses these; this pins it before the default
changes (#242).
Model: opus-5-5
Process looked at its context only while waiting for a processing
slot, so a transcode kept its slot and worked on after its request's
downstream_timeout. acquireSlot now gives the slot back when the
context has ended by the time it has one, and encode returns the
context's error without beginning. govips gives pixa no way to stop
libvips once an encode has begun, so that encode keeps its slot. A
request waiting on a shared transcode stopped at the first request's
deadline is answered 503, not 500. README.md "Routes" says both.
Model: opus-5-5
"Running under upaas" in README.md listed PIXA_CACHE_MAX_BYTES as
optional. Under upaas the volume is a directory on the host's
filesystem, and the default limit is 75% of the sum of that
filesystem's free space and what the cache holds, at least 500 MiB, so
one pixa can take three quarters or more of the space every app has
left, and all of it when that is 500 MiB or less. The section now tells
the operator to set the variable and says what the default does there.
The default is unchanged.
Model: opus-5-5
The metrics middleware labelled each request with its path, so every
new path added series to the registry that were never removed, and
/metrics listed the token of every encrypted URL requested. The handler
label is now the pattern of the chi route the request will match, looked
up before routing, or "unmatched" when none does. /v1 and /v1/ ask the
image subrouter, and an empty path is looked up as /, as chi routes it.
TestMetricsRecordsServedRequest expects "unmatched" in place of "/test",
as the owner approved.
Model: opus-5-5
A negative cache row was deleted only when its URL was requested
again, so a client asking for a new missing path each time grew the
database without limit. The eviction goroutine now removes expired rows
at startup and on every tick, so a row stays about NegativeTTL plus one
eviction interval. It deletes at most 1000 rows per statement, so a
request on the one database connection waits behind one batch at most.
The goroutine now also runs when the disk cache is disabled, since the
negative cache is used either way.
Model: opus-5-5
Get checked the negative cache before the variant cache, so after one
failed fetch of a source every variant of it answered 502 for five
minutes, also one on disk or one that could be made from the cached
source. The check now sits at the start of fetchAndProcess, which runs
only when no cached source can be used. A refusal from the negative
cache now counts as a miss.
Model: opus-5-5
With `cdn.example.com` allowlisted, other letter cases and a host with a port also passed the allowlist; each spelling was fetched and cached on its own with its own per-host connection limit, and a port let an unsigned URL reach any port of the host.
The image URL parser now lower-cases the host and refuses one holding `@` with 400. A host followed by `:`, with or without a port (read as Go's URL parser reads it, unbracketed IPv6 included), never counts as allowlisted, so it needs a signature. The host is signed lower-cased, as the URL writes it, so a signature made over an upper-case spelling is refused with 401.
Model: opus-5-5
The signed data and the cache key joined host, path and query with ":" and no escaping, so path /a:b with no query and path /a with query b: gave the same string: a URL signed for https://h/a:b verified for https://h/a?b:, and the two shared a cached image. Both now write each % in the path and the query as %25 and each : as %3A before joining them. The host never holds / and the path always starts with /, so the joined string reads back one way only. A path and query holding neither character are signed and keyed as before, so the existing known-answer vectors and the README example are unchanged; new vectors cover the escaped cases. Signature Specification in README.md states the escaping.
Model: opus-5-5
Once upstream_max_response_size bytes were read, the fetcher refused the next read without looking for the end of the body. Go's HTTP client returns the end of a body sent with a Content-Length together with its last bytes, but the end of a chunked body can come in a later read, so a chunked body of exactly the limit was refused after it was downloaded whole and the image routes answered 500. Once the limit is used up, the fetcher now reads one more byte: if the body has ended it is accepted, and if there is more it is refused with ErrResponseTooLarge.
Model: opus-5-5
The example config allowlisted github.com and user-images.githubusercontent.com, where anyone can put an image, so a deployment set up as Getting Started says served every image there to anyone without a signature. The example now allowlists nothing and shows one commented entry under example.com. README.md says in Getting Started and in "Running under upaas" that every image on an allowlisted host is served without a signature, so an allowlisted host must be one the operator controls. A new test loads the example config and fails if it allowlists any host.
Model: opus-5-5
fit=outside scales the source to cover the requested size, and the side
scaled past it was held to no maximum: 1x8192?fit=outside of a 3000x600
source was resized to 40960x8192 and encoded. resize now refuses such an
output with ErrOutputTooLarge before resizing, so both image routes answer
it 400 with the error a size with one side 0 already gets. The other fit
modes make the requested size or one within it. README.md and TODO.md
say so.
Model: opus-5-5
A side given as 0 is computed from the source's proportions, and
nothing held it to 8192: 0x8192 of a 3000x600 source was resized to
40960x8192 and encoded. Process now refuses such a size with
ErrOutputTooLarge once it knows the source's dimensions, before the
resize and the encode, and both image routes answer it 400 with the
JSON error body. It is a processing error, not a failed fetch, so it
is never negative-cached. imgcache.MaxDimension is now
imageprocessor.MaxOutputDimension, so the request limit and the output
limit are one value. README.md says it under Routes and Encrypted URLs.
Model: opus-5-5
CacheConfig.CacheTTL was set to DefaultCacheTTL (24 hours) and never
read, so cached images never expired. Remove the field, the constant
and the line that set it. Cached source and transformed images stay
until eviction removes them to keep the cache within cache_max_bytes.
A new test caches a source and a transformed image, sets every time
pixa stores for them two days back (database rows, metadata files and
file times), and checks that both are still served from the cache
without a new upstream fetch.
Existing tests lose only the CacheTTL line of their cache config.
Model: opus-5-5
The upstream fetch no longer accepts image/svg+xml, and both image
routes answer 415, not 500, for any upstream Content-Type outside the
source formats. Before decoding, the image processor names the input's
format from govips' DetermineImageType and refuses, with 415, input
whose first bytes libvips reads as anything but the six source formats,
so an SVG passing the first-bytes check for AVIF never reaches librsvg.
orig of such an input is refused instead of being saved as JPEG.
README.md drops SVG from the source formats.
SVG detection stays in internal/magic, unused on the request path, for
#168. An AVIF image sequence
(brand avis) answers 415.
Model: opus-5-5
TestService_Get_ReturnsByItsDeadline failed on a busy host because its
request, past its deadline, still waited to write its miss count, a write
with no deadline; in pixad that write waits for the one database
connection every request shares. Each count write now keeps the request's
deadline but not its cancellation. A count not written by then is kept in
memory, where Stats includes it, and one goroutine of the cache writes
those counts one UPDATE at a time, and once more at shutdown before the
database closes. The test phase also runs go test with -parallel 4: on a
busy host, as many tests at once as there are CPUs wait so long to be
scheduled that a timed request can fail.
Model: opus-5-5
The new setting image_requests_per_minute (PIXA_IMAGE_REQUESTS_PER_MINUTE),
default 600, caps the GET and HEAD requests one client sends to /v1/image/
and /v1/e/ per minute; the next is answered 429 with Retry-After and the
image routes' JSON error body. It uses the login limit's RateLimit
middleware, so clients are told apart the same way, behind the CORS
middleware, so a preflight is not counted. 0 turns it off, as
script/loadtest does; a negative or non-integer value stops startup.
TestEnvironmentSetsEveryKey gains the new variable, under the standing
approval in #64. To stay within the
linters' limits, the sentry_dsn check moves unchanged into
validateSentryDSN, and the cache_max_bytes explicit flag is set in one
line.
Model: opus-5-5
Each output format now has its own govips export with its settings
named, in place of govips' generic Export, which sent libvips a zero for
some settings it was not given: PNG had no compression and WebP effort
0. PNG now gets libvips' default compression, 6, and WebP its default
effort, 4. GIF and JPEG output is unchanged.
AVIF was at libvips' default effort, 4, which takes minutes for an
8192x8192 image with one libvips thread, far past the default
downstream_timeout. Effort 1, the lowest govips can set, takes about 51
seconds for an image of random pixels, the worst case, and WebP at 4
about 43. A 16-bit source gets 8 bits per sample, not libvips' 12, which
take over four times as long.
Model: opus-5-5
A /v1/image/ URL whose last segment is a size with no format, such as
800x600 or orig, is served as JPEG XL and signed as jxl, so it shares
the signature of the same URL ending in .jxl. An encrypted URL whose
token holds no format is served as JPEG XL, as encurl.DefaultFormat is
now jxl. The generator page selects JPEG XL by default, and a form
with an empty format, or none, makes a URL whose name ends in .jxl.
The image processor no longer takes an empty format as orig: both
routes give every request a format, so it refuses a request with none
instead of keeping a second default. auto still ends with JPEG.
Model: opus-5-5
A JPEG XL source is accepted, and orig of one is JPEG XL. The format
jxl works in plain and encrypted URLs and on the generator page,
served as image/jxl; auto chooses it first when Accept names
image/jxl.
govips sends libvips a JPEG XL distance, which overrides the quality,
so q becomes a distance, keeping 100 lossy. Metadata is removed from
the image before the JPEG XL save, as govips cannot have libvips strip
it, and the image is given 72 dpi so that the EXIF block libvips 8.16
adds holds nothing from the source. The sRGB conversion moved ahead of
the format switch, and a CMYK image with no ICC profile is converted
to sRGB, as libvips cannot save CMYK as JPEG XL.
Model: opus-5-5
The runtime stage of the Dockerfile moves from alpine:3.21 to
alpine:3.22, pinned by digest. The test phase and the build stage use
the golang image built on Alpine 3.22, so pixad was compiled against
libvips 8.16 and ran against 8.15. All three stages now install their
packages from the same Alpine release, where the runtime packages keep
their names. The golang pins now name that release as
golang:1.25.4-alpine3.22, with the same digest, and say that the runtime
stage uses it too. README.md now says the image has libvips 8.16.
Model: opus-5-5
UsageBytes now reads the new cache_usage row, which triggers on
source_content and variant_content keep up to date in the statement
that adds, removes or resizes a row. The reconciliation pass reads both
tables 1000 rows per query, sums them, and corrects the total when it
differs, unless a row changed while it summed. Source rows now get
last_accessed_at when added, so choosing source images to evict reads
that column's index instead of sorting the whole table. Stats still
sums the tables, now in pages: an existing test drops both tables and
expects that sum to fail.
Model: opus-5-5
script/bootstrap --cgo installs vips-jxl when the package manager it
uses is apk, as Alpine's vips package lacks JPEG XL support; the nix
and brew libvips include it, as does apt's from Debian 12 and Ubuntu
24.04 on. The runtime stage of the Dockerfile installs vips-jxl too.
imgcache.NewService calls the new imageprocessor.CheckJPEGXLSupport
before it builds the image processor and fails with its error, which
names the fix, so pixad does not start without the support. JPEG XL
becomes the default output later in the issue. New tests check the
support and save an image as JPEG XL with govips and load it back.
Model: opus-5-5
internal/database now opens the database with one connection. pixa's
own reads and writes run on it one at a time instead of competing for
SQLite's lock, where a write that kept losing could wait past the
five-second busy timeout and fail with "database is locked". The busy
timeout stays, for another program writing to the same file.
With one connection, a query run while rows or a transaction are still
open would wait forever. No code in internal/database or
internal/imgcache does that; the comment on DB() tells callers.
README.md says pixa uses one connection and that requests wait while
eviction runs one of its queries.
Model: opus-5-5
auto is a format in the /v1/image/ path, an encrypted URL's token and
the generator page. Once the signature or token is checked, pixa
chooses AVIF when Accept names image/avif, else WebP when it names
image/webp, else JPEG when the most specific of image/jpeg, image/* and
*/* allows it or Accept is absent. q=0 refuses a format; a header
allowing none of the three answers 406, one that does not parse 400.
The signature and token cover auto itself; the cache key and ETag use
the chosen format. Answers from then on carry Vary: Accept.
Model: opus-5-5
script/fmt and script/fmt-check run prettier 3.8.1 on the markdown after
gofmt, with the same yarn helper and arguments as the copies in
sneak/prompts. prettier is pinned in package.json and yarn.lock;
.prettierrc sets four-space tabs and proseWrap always, and
.prettierignore keeps prettier off REPO_POLICIES.md and vendor/. Plain
script/bootstrap installs Node and Yarn the way the one in sneak/prompts
does; with --cgo it does not, as the Dockerfile stages that pass it
format nothing. The HTML templates stay out: prettier cannot parse a Go
template action inside a tag. This commit also holds the reflow that
make fmt then produced (lines rewrapped, bullets as dashes, no word
changed), which the PR kept as a separate commit for review.
Model: opus-5-5
script/check, cibuild, docker, lint, test, setup and install-precommit
are now the sneak/prompts main copies, unchanged: lint and test each
build their Dockerfile phase with --no-cache. The lint phase runs
golangci-lint from the image REPO_POLICIES.md names, with libvips-dev
from apt-get; the test phase runs the tests with a 90-second timeout;
the build stage depends on both. script/bootstrap installs the C
compiler and image libraries only with --cgo, which the test phase and
build stage pass, and refreshes the apt lists before its first apt
install. Dockerfile.lint and CHECK_EPOCH are gone, and make
docker-versioned and docker-test call the scripts. Without VERSION the
build stage still uses git describe, per issue 166.
Model: opus-5-5
script/loadtest [duration [clients]] (make loadtest, defaults 10s and 4)
measures pixad in three scenarios: hit (one cached image), miss (a new
source image every request) and herd (each new source image asked for
by all clients at once). For each it prints vegeta's report (requests
per second, p50/p95/p99 latency, status codes), pixad's peak resident
memory and how many requests reached the origin. It is a benchmark, not
run by script/check. The origin it uses, internal/loadtestorigin behind
cmd/loadtest-origin, answers every path with one generated JPEG. Bad
arguments are refused before the build. README.md says how to run and
read it and keeps 1-5k r/s as a target not yet measured at scale.
Model: opus-5-5
The standard checkout action clones shallow and fetches no tags, so the
version the build takes from `git describe --tags --always` would be a
bare commit even on a tagged commit. The checkout step now sets
`fetch-depth: 0`, as REPO_POLICIES.md asks of a repo that takes its
version from the tags.
Model: opus-5-5
Getting Started has you create config.yml at the repository root with a
real signing key, but neither .gitignore nor .dockerignore left it out,
so it could be committed and, through COPY . ., reach a build-stage
layer. .gitignore now ignores it next to config.yaml, and .dockerignore
leaves it out in every directory and in any letter case, as it already
does config.yaml and config.dev.yml.
Model: opus-5-5
config.yaml and config.dev.yml are kept out of git because they can hold
the signing key, but .dockerignore did not leave them out, so a local
copy in the working tree reached the build context and, through
COPY . ., a build-stage layer. .dockerignore now leaves them out in
every directory and in any letter case. configs/config.example.yml is
still sent.
Model: opus-5-5
cmd/pixad/main.go built the command line and its --config flag, set
PIXA_CONFIG_PATH from that flag, ignored SIGPIPE and started the fx
app. REPO_POLICIES.md now requires cmd/ to hold only one call into
internal/ or pkg/, so that code moves unchanged to Run in the new
internal/app package, and main calls app.Run(Version). Version stays
in main, so the -X main.Version build flags in the Dockerfile and the
Makefile do not change.
Model: opus-5-5
REPO_POLICIES.md says in-repo agent scratch belongs in both .gitignore and
.dockerignore. .dockerignore already has .claude; .gitignore now has the
.claude/ entry and its comment exactly as the canonical .gitignore in
sneak/prompts has them, unanchored so it matches at every depth.
Model: opus-5-5
.dockerignore patterns without a leading **/ match only at the root of
the build context, so a nested .env or private key still reached it and,
through COPY . ., a build-stage layer. The file is now the standard one
from sneak/prompts: every pattern that should match anywhere has **/,
and private keys and environment files are matched in any letter case.
pixa keeps its own differences: .git is still sent without .git/config
in place of the standard .git line, which the version stamp needs, and
.gitignore, /bin and /data stay out.
Model: opus-5-5
REPO_POLICIES.md is fetched unchanged from prompts/REPO_POLICIES.md on
sneak/prompts main. The new rules pixa's tree breaks are filed as
#202 through
#206 and
#208 and not fixed here. Its rule
that no build stage runs git describe is not followed, per
#166.
Model: opus-5-5
TestImageProxyFlow in internal/server starts the database, handlers and
middleware from the constructors pixad uses, with a fresh state
directory, and replaces only the upstream origin with an httptest
server. For a resize with a format change and for orig it checks a 200
MISS with the right type and size, then a HIT after one upstream request,
and the files and rows the cache keeps. Two optional test seams make
that possible: httpfetcher.Config.DialContext and handlers.Params.Fetcher.
pixad sets neither and the config file and environment cannot, and tests
show production still uses the checked dialer and builds its own fetcher.
Model: opus-5-5
A new handler test makes a URL on the generator page with a ttl of one
second, checks that /v1/e/ serves it at once, waits two seconds and
checks that it then answers 410. The expiry is kept in whole seconds,
so two seconds is the longest a one-second ttl can take to pass. Test
only.
Model: opus-5-5
A new setting, referer_blocklist (PIXA_REFERER_BLOCKLIST), lists hosts
whose pages may not show pixa's images. Entries are written and matched
as allowlist_hosts are, with the same matcher. Both image routes check
the Referer before the signature, the cache and the upstream fetch, and
answer 403 with the JSON error, so a blocked request costs nothing and
is refused whether or not the image is cached. No Referer, or one that
does not parse, is served; README.md says this makes the list easy to
get around. An entry of either host list that is not a host name
(letters, digits, hyphens, underscores, dots, at most one leading dot)
or an IP address now aborts startup naming the setting and the entry.
Model: opus-5-5
config.example.yml moves unchanged to configs/config.example.yml, the
directory REPO_POLICIES.md names for configuration examples. README.md,
the comments in internal/config/config.go and the startup error for the
placeholder signing key name the new path. scripts/manual-test.sh and
its directory are deleted. The handler tests in internal/handlers cover
every check it made except two: fetching a real image from the internet,
and a URL made on the generator page with a ttl answering 410 once the
ttl has passed (#199).
CONVENTIONS.md, a reformatted copy of the Go HTTP server conventions, is
deleted, as REPO_POLICIES.md links the canonical document.
Model: opus-5-5
Requests and the eviction pass write on separate connections, and with
no busy timeout a write that met another one failed at once with
"database is locked" and was lost. internal/database now adds
_pragma=busy_timeout(5000) to every db_url, the default or one the
operator sets, so such a write waits up to five seconds. The default
db_url's _journal_mode=WAL is not a parameter the driver reads, so it
is now _pragma=journal_mode(WAL). README.md and config.example.yml say
what pixa adds to db_url.
Model: opus-5-5
TestPeriodicReconciliationAdoptsFileThatAppearsAfterStartup slept for
three eviction intervals before writing its file, so on a slow start the
startup pass could still be running and adopt the file itself, and the
test passed without a periodic pass. It now holds the test database's
only connection until the startup pass waits for it, writes the file and
lets the connection go, as TestEvictionRunsOnPeriodicSchedule does, so
only a periodic reconciliation pass can adopt the file. Test only.
Model: opus-5-5
The default cache_max_bytes was 75% of the space free at startup. The
cache's own files are not free space, so a fuller cache got a smaller
limit after a restart and eviction then deleted most of it. The default
is now 75% of the sum of the free space and what the cache already holds
by its own size accounting, at least 500 MiB. The cache works it out
when it opens, after the database is open, so the computation and its
tests moved from internal/config to internal/imgcache. The config only
records whether cache_max_bytes was set; newCacheConfig in the handlers
turns the disk cache off only for an explicit 0, and is tested for an
omitted, a zero and a positive value.
Model: opus-5-5
New handler tests in internal/handlers, with no network: GET / shows the
login form without a session; a wrong key shows it again with an error and
sets no session cookie; the right key answers 303 with a session cookie
marked Secure, HttpOnly and SameSite=Strict, with which GET / shows the
generator page; GET /logout empties the cookie with Max-Age=0; POST
/generate without a session answers 303 to /; /v1/e/ serves a valid
token's image, answers 410 for an expired token and 400 for one changed,
cut short or made with another signing key; a URL made on the generator
page is served by /v1/e/. No code changes.
Model: opus-5-5
script-src and style-src now allow only 'self'. The generator page's
two inline onclick handlers, which selected the generated URL and
copied it, move into internal/static/generator.js and are attached
with addEventListener. The bundled Tailwind script, which built styles
in the browser and injected them at runtime, is replaced by a small
hand-written internal/static/style.css holding only the rules the
login and generator pages use; the templates carry a few plain class
names in place of Tailwind's. No build step. The pages keep their
layout, not every pixel of it.
Model: opus-5-5
.golangci.yml is the canonical copy from the main branch of
sneak/prompts, fetched unchanged. It switches off the deprecated
gomodguard, whose deprecation warning was printed on every lint run,
and turns on its successor gomodguard_v2 with the shared module block
list. It also turns on depguard with the rule that keeps
net/http/httptest out of files that are not tests. pixa has no deny
entries of its own to carry forward, and the tree needs no code changes
under the new linters. The owner approved this config in sneak/prompts.
Model: opus-5-5
Of the places pixa looks for its config file on its own, it passed over
any place where os.Stat failed, so a file in a directory pixa may not
enter was skipped without a word and pixa started on a later file or on
the environment and defaults. Now only a path that does not exist, or
that runs through a file (such as under a HOME of /dev/null), is passed
over; any other error aborts startup naming the file, as a file that
does not parse already did. README.md says so where it gives the search
order. A config.yml that links to itself tests this as root too.
Model: opus-5-5
Every PR adds an entry at the top of Completed Steps in TODO.md, so each
merge to next left the other open PRs conflicting there. A root
.gitattributes, copied from sneak/prompts, marks TODO.md merge=union: two
branches that each add an entry at the same place merge without a conflict
and keep both. Git then never reports a conflict in TODO.md, so the
Workflow now says to read the merged entries after every merge or rebase.
Model: opus-5-5
The test wrote each variant file and then inserted its accounting row by
hand while the evictor was running. A reconciliation pass between the two
steps adopted the file first, and the hand insert failed on the unique key.
The test now writes the files only, while it holds the test database's
only connection, so the evictor's startup pass waits after walking the
still empty variant directory. A periodic reconciliation pass then adopts
the files and the eviction pass after it evicts them; no write-pressure
notification fires. The test waits until two of the three files are gone,
then checks that usage is within the limit and that no row points at a
missing file.
Model: opus-5-5
README.md gains a "Deployment" section: what the reverse proxy in front
of pixa must do (terminate TLS, pass Host, Origin and Referer on
unchanged, set X-Forwarded-For with trusted_proxies to match, wait at
least downstream_timeout, optionally refuse /metrics) and what pixa does
itself; that the state directory needs a persistent volume, what
cache_max_bytes counts and why to set it; the health check for a load
balancer; what SIGTERM does and the exit codes; and what running outside
Docker needs. configs/Caddyfile is the example, as Caddy needs no
settings beyond the host name and pixa's address.
Model: opus-5-5
New tests only. MetricsAuth on its own answers 401 with a challenge
without credentials or with a wrong username or password, and lets the
configured ones through. A CORS preflight request gets the same
Access-Control-Allow-Origin as a GET. A POST / carrying the signing key
leaves the key out of the request log line, and the login handler's own
log lines leave out the submitted key. The metrics middleware on its own
records a request it served; the router records nothing while no metrics
username is set.
Not tested through the router: the basic auth in front of /metrics and
recording with a metrics username set (#180).
The pinned basicauth-go compares the password in constant time.
Model: opus-5-5
sneak
clawbot