Compare commits

..
2 Commits
Author SHA1 Message Date
clawbot 410dd92292 Rewrite README.md sentence by sentence (closes #327)
check / check (push) Canceled after 0s
Rewrites README.md from scratch against the code on next, for
#327: what pixa is and how to
use it, deployment, design, the settings reference, then development
and testing. The Configuration list keeps one item per setting, as
TestReadmeNamesEverySetting reads it; the test is unchanged. The
statements where the old text disagreed with the code are listed in
the PR body.

Model: fable-5-1 (writing), opus-5-5 (rework checks and commit)
2026-11-11 09:32:04 +00:00
clawbot 9f1e9f5253 Rewrite AGENTS.md in short plain sentences (#327)
check / check (push) Canceled after 0s
Every rule keeps its meaning and emphasis. Sentences are split so each
holds one idea, with the main rule before its exception and every
command, file name and identifier in backticks.

Three statements were false against the repo or the standing rulings
on record and are corrected:

- the format check is `make fmt-check`, not `make check-fmt`
- a feature branch reaches `next` through a PR that passes review, is
  merged only when the feature is completely finished and the code is
  up to standards, and is deleted after the merge; only the user
  merges `next` into `main`
- the naming example uses the code's term: `al` is bad,
  `SourceHostAllowlist` is good

The `Last Updated` line takes today's date.

Model: fable-5-1
2026-10-27 05:06:36 +01:00
2 changed files with 89 additions and 84 deletions

No files matched your search

+52 -51
View File
@@ -1,75 +1,76 @@
# Repository Rules # Repository Rules
Last Updated 2026-01-08 Last Updated 2026-10-27
These rules MUST be followed at all times, it is very important. These rules MUST be followed at all times. This is very important.
- Never use `git add -A` - add specific changes to a deliberate commit. A commit - Never use `git add -A`. Stage specific changes for a deliberate commit. A
should contain one change. After each change, make a commit with a good commit holds one change. After each change, make a commit with a good one-line
one-line summary. summary.
- NEVER modify the linter config without asking first. - NEVER modify the linter config without asking first.
- NEVER modify tests to exclude special cases or otherwise get them to pass - NEVER modify a test to exclude special cases or otherwise make it pass without
without asking first. In almost all cases, the code should be changed, NOT the asking first. In almost all cases, change the code, NOT the test. You need
tests. If you think the test needs to be changed, make your case for that and explicit user approval to modify an existing test. If you think a test needs
ask for permission to proceed, then stop. You need explicit user approval to to change, make your case and ask for permission to proceed. Then stop. A bug
modify existing tests. A bug fix may change an existing test without asking fix may change an existing test without asking only when that test pins
when that test pins exactly the behavior the filed bug calls wrong, and the exactly the behavior the filed bug calls wrong. The change must then move the
change moves it to the fixed behavior and nothing else; any other change to an test to the fixed behavior and nothing else. Any other change to an existing
existing test still needs explicit user approval. (You do not need user test needs explicit user approval. Writing NEW tests needs no approval.
approval for writing NEW tests.)
- When linting, assume the linter config is CORRECT, and that each item output - When linting, assume the linter config is CORRECT. Treat each item the linter
by the linter is something that legitimately needs fixing in the code. reports as something that legitimately needs fixing in the code.
- When running tests, use `make test`. - When running tests, use `make test`.
- Before commits, run `make check`. This runs `make lint` and `make test` and - Before committing, run `make check`. It runs `make test`, `make lint` and
`make check-fmt`. Any issues discovered MUST be resolved before committing `make fmt-check`. Any issue it finds MUST be resolved before committing,
unless explicitly told otherwise. unless you are explicitly told otherwise.
- When fixing a bug, write a failing test for the bug FIRST. Add appropriate - When fixing a bug, write a failing test for the bug FIRST. Add appropriate
logging to the test to ensure it is written correctly. Commit that. Then go logging to the test, to make sure the test is written correctly. Commit that.
about fixing the bug until the test passes (without modifying the test Then work on the fix until the test passes, without changing the test further.
further). Then commit that. Then commit the fix.
- When adding a new feature, do the same - implement a test first (TDD). It - When adding a new feature, do the same: write a test first (test-driven
doesn't have to be super complex. Commit the test, then commit the feature. development, TDD). The test does not need to be very complex. Commit the test,
then commit the feature.
- When adding a new feature, use a feature branch. When the feature is - When adding a new feature, use a feature branch. The feature branch reaches
completely finished and the code is up to standards (passes `make check`) then `next` through a PR that passes review. It is merged only when the feature is
and only then can the feature branch be merged into `main` and the branch completely finished and the code is up to standards. The code is up to
deleted. standards when it passes `make check`. The branch is deleted after the merge.
Only the user merges `next` into `main`.
- Write godoc documentation comments for all exported types and functions as you - Write `godoc` comments for all exported types and functions as you go along.
go along.
- ALWAYS be consistent in naming. If you name something one thing in one place, - ALWAYS be consistent in naming. If you name something one way in one place,
name it the EXACT SAME THING in another place. use the EXACT SAME name in every other place.
- Be descriptive and specific in naming. `wl` is bad; `SourceHostWhitelist` is - Be descriptive and specific in naming. `al` is bad; `SourceHostAllowlist` is
good. `ConnsPerHost` is bad; `MaxConnectionsPerHost` is good. good. `ConnsPerHost` is bad; `MaxConnectionsPerHost` is good.
- This is not prototype or teaching code - this is designed for production. Any - This is not prototype or teaching code. It is designed for production. Any
security issues (such as denial of service) or other web vulnerabilities are security issue (such as denial of service) or other web vulnerability is a P1
P1 bugs and must be added to TODO.md at the top. bug. It must be added at the top of `TODO.md`.
- As this is production code, no stubbing of implementations unless specifically - Do not stub an implementation unless you are specifically instructed to. This
instructed. We need working implementations. is production code, and we need working implementations.
- NEVER silently fall back to a different setting when a user's parameter - NEVER silently fall back to a different setting when a user's parameter
explicitly specifies a value. If a user requests format=webp and WebP encoding explicitly specifies a value. For example, if a user requests `format=webp`
is not supported, return an error - do NOT silently output PNG instead. If a and WebP encoding is not supported, return an error. Do NOT silently output
user specifies fit=invalid and that fit mode doesn't exist, return an error - PNG instead. If a user specifies `fit=invalid` and that fit mode does not
do NOT silently default to "cover". Silent fallbacks violate the principle of exist, return an error. Do NOT silently default to `cover`. A silent fallback
least surprise and mask bugs. The only acceptable defaults are for OMITTED violates the principle of least surprise and masks bugs. Defaults are
parameters, never for INVALID explicit values. acceptable only for OMITTED parameters, never for INVALID explicit values.
- Avoid vendoring deps unless specifically instructed to. NEVER commit the - Avoid vendoring dependencies unless specifically instructed to. NEVER commit
vendor directory, NEVER commit compiled binaries. If these directories or the `vendor/` directory. NEVER commit compiled binaries. If such a directory
files exist, add them to .gitignore (and commit the .gitignore) if they are or file exists and `.gitignore` does not list it, add it to `.gitignore`.
not already in there. Keep the entire git repository (with history) small - Commit that `.gitignore` change. Keep the entire git repository, history
under 20MiB, unless you specifically must commit larger files (e.g. test included, under 20 MiB. Go over 20 MiB only when you specifically must commit
fixture example media files). Only OUR source code and immediately supporting a large file, such as a media file used as a test fixture. Only OUR source
files (such as test examples) goes into the repo/history. code and its immediately supporting files, such as test examples, go into the
repository and its history.
+37 -33
View File
@@ -266,8 +266,8 @@ to any method the list does not give for it.
- `POST /generate` — make an encrypted URL from the generator page's form. Needs - `POST /generate` — make an encrypted URL from the generator page's form. Needs
a login session and the generator page's form. Without a login session it a login session and the generator page's form. Without a login session it
answers 303 to `/`. Answers 200 with the page showing the URL. Answers 400 answers 303 to `/`. Answers 200 with the page showing the URL. Answers 400
with the page naming a field that is not valid. Answers 500 when the URL with the page naming the field it refused (see Encrypted URLs). Answers 500
cannot be made. when the URL cannot be made.
- `POST /logout` — log out with the generator page's Logout button. That ends - `POST /logout` — log out with the generator page's Logout button. That ends
every login session, in every browser (see Encrypted URLs), and removes the every login session, in every browser (see Encrypted URLs), and removes the
cookie. Needs a login session and the generator page's form. Without a login cookie. Needs a login session and the generator page's form. Without a login
@@ -347,7 +347,8 @@ to any method the list does not give for it.
A browser's CORS preflight request to any path under `/v1/` answers 200, in A browser's CORS preflight request to any path under `/v1/` answers 200, in
maintenance mode too. CORS is the browser rule that lets a page on one site read maintenance mode too. CORS is the browser rule that lets a page on one site read
responses from another. The preflight is the `OPTIONS` request the browser sends responses from another. The preflight is the `OPTIONS` request the browser sends
first. first. It carries an `Origin` header and an `Access-Control-Request-Method`
header.
Every response carries an `X-Request-Id` header with the request's ID. A client Every response carries an `X-Request-Id` header with the request's ID. A client
can quote it when reporting a problem. A reverse proxy may send an can quote it when reporting a problem. A reverse proxy may send an
@@ -474,9 +475,10 @@ it would come out above 8192, the request is refused with 400. The fit modes:
An AVIF output of more than 16,777,216 pixels is refused with 400 before An AVIF output of more than 16,777,216 pixels is refused with 400 before
anything is decoded. That is the number of pixels in 4096x4096 or 8192x2048. A anything is decoded. That is the number of pixels in 4096x4096 or 8192x2048. A
larger AVIF encode could take more than 40 seconds (see Limits). The limit larger AVIF encode could take more than 40 seconds (see Limits). The limit also
includes `orig` of an AVIF source, and `auto` when it chooses AVIF. So a client applies to the format `orig` when the source is AVIF, and to `auto` when it
whose `Accept` names `image/avif` but not `image/jxl` is refused such a size. chooses AVIF. So a client whose `Accept` names `image/avif` but not `image/jxl`
is refused such a size.
**Source formats.** The source may be JPEG, PNG, GIF, WebP, AVIF or JPEG XL. The **Source formats.** The source may be JPEG, PNG, GIF, WebP, AVIF or JPEG XL. The
upstream's `Content-Type` must name its format, and the image's first bytes must upstream's `Content-Type` must name its format, and the image's first bytes must
@@ -620,7 +622,8 @@ The form's fields:
- The quality, a whole number from 1 to 100. The page offers five values: `25`, - The quality, a whole number from 1 to 100. The page offers five values: `25`,
`50`, `70`, `85` and `100`. `85` is selected unless another is chosen. `50`, `70`, `85` and `100`. `85` is selected unless another is chosen.
- The fit, one of the five fit modes (see Image URLs). `cover` is selected - The fit, one of the five fit modes (see Image URLs). `cover` is selected
unless another is chosen. unless another is chosen. A form sent with an empty fit, or none, makes a
`cover` URL.
- How long the URL lasts, in seconds, or `0` for never. The page offers seven - How long the URL lasts, in seconds, or `0` for never. The page offers seven
choices: never, 1 minute, 1 hour, 1 day, 1 week, 30 days and 1 year. Never is choices: never, 1 minute, 1 hour, 1 day, 1 week, 30 days and 1 year. Never is
selected unless another is chosen. selected unless another is chosen.
@@ -743,7 +746,7 @@ processing slot. If its `downstream_timeout` ends after it has a slot but before
it begins to encode, it gives the slot back. For a shared conversion, the first it begins to encode, it gives the slot back. For a shared conversion, the first
request's deadline counts. When a shared conversion stops this way, each request request's deadline counts. When a shared conversion stops this way, each request
waiting for it is answered 503 `server busy, try again later`, even if its own waiting for it is answered 503 `server busy, try again later`, even if its own
`downstream_timeout` has not ended. govips, the library pixa drives libvips `downstream_timeout` has not ended. `govips`, the library pixa drives libvips
through, gives no way to stop an encode once it has begun. So an encode keeps through, gives no way to stop an encode once it has begun. So an encode keeps
its slot until it finishes, which can be after `downstream_timeout`. its slot until it finishes, which can be after `downstream_timeout`.
@@ -751,8 +754,8 @@ its slot until it finishes, which can be after `downstream_timeout`.
They are 8192 on each side of the source and of the output, and 16,777,216 They are 8192 on each side of the source and of the output, and 16,777,216
pixels for an AVIF output. Decoding an 8192x8192 source takes up to about 11 pixels for an AVIF output. Decoding an 8192x8192 source takes up to about 11
seconds. The WebP and JPEG XL encoders run below libvips' default effort. AVIF seconds. The WebP and JPEG XL encoders run below libvips' default effort. AVIF
runs at the lowest effort govips can set. With one libvips thread, as pixa runs runs at the lowest effort `govips` can set. With one libvips thread, as pixa
it, encoding 8192x8192 random pixels takes up to about: runs it, encoding 8192x8192 random pixels takes up to about:
- 1 second as JPEG; - 1 second as JPEG;
- 15 seconds as PNG; - 15 seconds as PNG;
@@ -762,10 +765,10 @@ it, encoding 8192x8192 random pixels takes up to about:
An AVIF output of 4096x4096, 8192x2048 or 2048x8192 random pixels takes up to An AVIF output of 4096x4096, 8192x2048 or 2048x8192 random pixels takes up to
about 34 seconds with an alpha channel, and 22 without. At 8192x8192 and quality about 34 seconds with an alpha channel, and 22 without. At 8192x8192 and quality
85 it would take about 96 seconds with an alpha channel, and 52 without. These 85 it would take about 96 seconds with an alpha channel, and 52 without. The 34
times hold at any quality and at 8 or 16 bits per sample. No encode within the seconds hold at any quality and at 8 or 16 bits per sample. No encode within the
limits takes more than about 40 seconds of the default `downstream_timeout` limits takes more than about 40 seconds. The default `downstream_timeout` is 60
of 60. seconds.
**Processing and connections.** `max_concurrent_processing` images are decoded **Processing and connections.** `max_concurrent_processing` images are decoded
and encoded at once. A request that finds every slot in use waits up to 10 and encoded at once. A request that finds every slot in use waits up to 10
@@ -1042,15 +1045,15 @@ Each item below is one setting, named by its config key:
time allowed for answering one client request. Keep it longer than time allowed for answering one client request. Keep it longer than
`upstream_fetch_timeout` plus 20 seconds (see Limits). `upstream_fetch_timeout` plus 20 seconds (see Limits).
- `access_control_allow_origin`: variable `PIXA_ACCESS_CONTROL_ALLOW_ORIGIN`, - `access_control_allow_origin`: variable `PIXA_ACCESS_CONTROL_ALLOW_ORIGIN`,
default `*`. The origin a browser lets read the responses of `/v1/image/` and default `*`. The origin whose pages may read the responses of `/v1/image/` and
`/v1/e/`, sent as the CORS `Access-Control-Allow-Origin` header. No other `/v1/e/` in a browser. pixa sends it as the CORS `Access-Control-Allow-Origin`
route sends it. `*` is any site. Otherwise it is one `http` or `https` origin header. No other route sends it. `*` is any site. Otherwise it is one `http`
such as `https://example.com`. Its host is a lowercase host name (letters, or `https` origin such as `https://example.com`. Its host is a lowercase host
digits, hyphens and dots, with a letter in its last part) or an IP address name (letters, digits, hyphens and dots, with a letter in its last part) or an
(IPv6 in brackets, in its shortest form). It may have a port from 1 to 65535 IP address (IPv6 in brackets, in its shortest form). It may have a port from 1
that has no leading zero and is not the scheme's default. Any other value to 65535 that has no leading zero and is not the scheme's default. Any other
aborts startup, including an origin with another scheme, such as a browser value aborts startup, including an origin with another scheme, such as a
extension's. browser extension's.
- `metrics.username`: variable `PIXA_METRICS_USERNAME`, default empty. The - `metrics.username`: variable `PIXA_METRICS_USERNAME`, default empty. The
username for `/metrics`. pixa serves `/metrics`, and measures every HTTP username for `/metrics`. pixa serves `/metrics`, and measures every HTTP
request, only when the username and the password are both set. Setting one request, only when the username and the password are both set. Setting one
@@ -1080,8 +1083,9 @@ Each item below is one setting, named by its config key:
image cache, the image proxy and the other parts that serve images and logins. image cache, the image proxy and the other parts that serve images and logins.
Each setting's default is declared once, in `internal/config`. Each setting's default is declared once, in `internal/config`.
- **Packages**: these packages under `internal/` work with the concepts above: - **Packages**: these packages under `internal/` work with the concepts above:
- `handlers`: the routes, which read the image request from the URL, check - `handlers`: the functions the routes call, which read the image request
the signed or encrypted URL, and make the login session from the URL, check the signed or encrypted URL, and make the login
session
- `imagerequest`: the image request, with the source key and the variant key - `imagerequest`: the image request, with the source key and the variant key
- `imageproxy`: the image proxy - `imageproxy`: the image proxy
- `imgcache`: the image cache, which keeps its sources in one `FileStore` - `imgcache`: the image cache, which keeps its sources in one `FileStore`
@@ -1095,7 +1099,7 @@ Each item below is one setting, named by its config key:
tokens tokens
- `session`: the login session - `session`: the login session
- **HTTP router**: `chi` (`github.com/go-chi/chi/v5`) - **HTTP router**: `chi` (`github.com/go-chi/chi/v5`)
- **Image processing**: govips, a Go wrapper for libvips - **Image processing**: `govips`, a Go wrapper for libvips
- **Database**: SQLite via `modernc.org/sqlite` - **Database**: SQLite via `modernc.org/sqlite`
- **Static assets**: embedded in the binary via `//go:embed` - **Static assets**: embedded in the binary via `//go:embed`
- **Metrics**: Prometheus, at `/metrics`. Prometheus is a monitoring system that - **Metrics**: Prometheus, at `/metrics`. Prometheus is a monitoring system that
@@ -1159,8 +1163,8 @@ workflow, and most `Makefile` targets are thin shims that call them. We provide:
It is a benchmark, not part of `script/check` (see Load Test). It is a benchmark, not part of `script/check` (see Load Test).
- `script/cibuild` — the CI entrypoint: run `script/bootstrap` (without - `script/cibuild` — the CI entrypoint: run `script/bootstrap` (without
`--cgo`), then `script/check`, then build the image as `script/docker` does. `--cgo`), then `script/check`, then build the image as `script/docker` does.
- `script/precommit` — the pre-commit checks: the `go mod tidy` guard, then - `script/precommit` — run `go mod tidy` and fail when it changes `go.mod` or
`script/check`. `go.sum`, then run `script/check`.
- `script/install-precommit` — install the git pre-commit hook that runs - `script/install-precommit` — install the git pre-commit hook that runs
`script/precommit`. `script/precommit`.
@@ -1210,10 +1214,10 @@ script/loadtest 30s 32 # 30 seconds per scenario, 32 clients
``` ```
It builds the image with `script/docker`. It also builds the load tool, It builds the image with `script/docker`. It also builds the load tool,
[vegeta](https://github.com/tsenart/vegeta), from a pinned commit. Each scenario [`vegeta`](https://github.com/tsenart/vegeta), from a pinned commit. Each
starts a new `pixad` container and a new origin container, scenario starts a new `pixad` container and a new origin container,
`cmd/loadtest-origin`. The origin is an upstream host that answers every path `cmd/loadtest-origin`. The origin is an upstream host that answers every path
with the same generated 1600x1200 JPEG. vegeta then sends requests from the with the same generated 1600x1200 JPEG. `vegeta` then sends requests from the
given number of clients. Each client sends its next request as soon as its last given number of clients. Each client sends its next request as soon as its last
one is answered. Every request asks for an image resized to 400x300 WebP: one is answered. Every request asks for an image resized to 400x300 WebP:
@@ -1228,7 +1232,7 @@ containers share a Docker network in `203.0.113.0/24`, a range set aside for
documentation. A second run on the same Docker host while one is going fails, as documentation. A second run on the same Docker host while one is going fails, as
it cannot create that network. it cannot create that network.
For each scenario the script prints vegeta's report and two lines of its own: For each scenario the script prints `vegeta`'s report and two lines of its own:
- `Requests [total, rate, throughput]`: the requests sent, how many were sent - `Requests [total, rate, throughput]`: the requests sent, how many were sent
per second, and how many were answered successfully per second; per second, and how many were answered successfully per second;
@@ -1238,7 +1242,7 @@ For each scenario the script prints vegeta's report and two lines of its own:
- `Status Codes` and `Error Set`: anything other than `200` means the other - `Status Codes` and `Error Set`: anything other than `200` means the other
numbers are not for the scenario described, such as `503` when `pixad` was numbers are not for the scenario described, such as `503` when `pixad` was
busy; busy;
- `Bytes In`: `0`, as vegeta is told not to keep the images it receives; - `Bytes In`: `0`, as `vegeta` is told not to keep the images it receives;
- `pixad peak memory (VmHWM)`: the peak resident memory of the `pixad` process - `pixad peak memory (VmHWM)`: the peak resident memory of the `pixad` process
since its container started, in kB; for `hit` it includes the request that put since its container started, in kB; for `hit` it includes the request that put
the image in the cache; the image in the cache;