Compare commits

..
1 Commits
Author SHA1 Message Date
clawbot 8669b5cf04 Rewrite README.md sentence by sentence (closes #327)
check / check (push) Canceled after 0s
Rewrites README.md from scratch against the code on next, for
#327: what pixa is and how to
use it, deployment, design, the settings reference, then development
and testing. The Configuration list keeps one item per setting, as
TestReadmeNamesEverySetting reads it; the test is unchanged. The
statements where the old text disagreed with the code are listed in
the PR body.

Model: fable-5-1
2026-10-27 03:07:29 +00:00
2 changed files with 84 additions and 89 deletions

No files matched your search

+51 -52
View File
@@ -1,76 +1,75 @@
# Repository Rules # Repository Rules
Last Updated 2026-10-27 Last Updated 2026-01-08
These rules MUST be followed at all times. This is very important. These rules MUST be followed at all times, it is very important.
- Never use `git add -A`. Stage specific changes for a deliberate commit. A - Never use `git add -A` - add specific changes to a deliberate commit. A commit
commit holds one change. After each change, make a commit with a good one-line should contain one change. After each change, make a commit with a good
summary. one-line summary.
- NEVER modify the linter config without asking first. - NEVER modify the linter config without asking first.
- NEVER modify a test to exclude special cases or otherwise make it pass without - NEVER modify tests to exclude special cases or otherwise get them to pass
asking first. In almost all cases, change the code, NOT the test. You need without asking first. In almost all cases, the code should be changed, NOT the
explicit user approval to modify an existing test. If you think a test needs tests. If you think the test needs to be changed, make your case for that and
to change, make your case and ask for permission to proceed. Then stop. A bug ask for permission to proceed, then stop. You need explicit user approval to
fix may change an existing test without asking only when that test pins modify existing tests. A bug fix may change an existing test without asking
exactly the behavior the filed bug calls wrong. The change must then move the when that test pins exactly the behavior the filed bug calls wrong, and the
test to the fixed behavior and nothing else. Any other change to an existing change moves it to the fixed behavior and nothing else; any other change to an
test needs explicit user approval. Writing NEW tests needs no approval. existing test still needs explicit user approval. (You do not need user
approval for writing NEW tests.)
- When linting, assume the linter config is CORRECT. Treat each item the linter - When linting, assume the linter config is CORRECT, and that each item output
reports as something that legitimately needs fixing in the code. by the linter is something that legitimately needs fixing in the code.
- When running tests, use `make test`. - When running tests, use `make test`.
- Before committing, run `make check`. It runs `make test`, `make lint` and - Before commits, run `make check`. This runs `make lint` and `make test` and
`make fmt-check`. Any issue it finds MUST be resolved before committing, `make check-fmt`. Any issues discovered MUST be resolved before committing
unless you are explicitly told otherwise. unless explicitly told otherwise.
- When fixing a bug, write a failing test for the bug FIRST. Add appropriate - When fixing a bug, write a failing test for the bug FIRST. Add appropriate
logging to the test, to make sure the test is written correctly. Commit that. logging to the test to ensure it is written correctly. Commit that. Then go
Then work on the fix until the test passes, without changing the test further. about fixing the bug until the test passes (without modifying the test
Then commit the fix. further). Then commit that.
- When adding a new feature, do the same: write a test first (test-driven - When adding a new feature, do the same - implement a test first (TDD). It
development, TDD). The test does not need to be very complex. Commit the test, doesn't have to be super complex. Commit the test, then commit the feature.
then commit the feature.
- When adding a new feature, use a feature branch. The feature branch reaches - When adding a new feature, use a feature branch. When the feature is
`next` through a PR that passes review. It is merged only when the feature is completely finished and the code is up to standards (passes `make check`) then
completely finished and the code is up to standards. The code is up to and only then can the feature branch be merged into `main` and the branch
standards when it passes `make check`. The branch is deleted after the merge. deleted.
Only the user merges `next` into `main`.
- Write `godoc` comments for all exported types and functions as you go along. - Write godoc documentation comments for all exported types and functions as you
go along.
- ALWAYS be consistent in naming. If you name something one way in one place, - ALWAYS be consistent in naming. If you name something one thing in one place,
use the EXACT SAME name in every other place. name it the EXACT SAME THING in another place.
- Be descriptive and specific in naming. `al` is bad; `SourceHostAllowlist` is - Be descriptive and specific in naming. `wl` is bad; `SourceHostWhitelist` is
good. `ConnsPerHost` is bad; `MaxConnectionsPerHost` is good. good. `ConnsPerHost` is bad; `MaxConnectionsPerHost` is good.
- This is not prototype or teaching code. It is designed for production. Any - This is not prototype or teaching code - this is designed for production. Any
security issue (such as denial of service) or other web vulnerability is a P1 security issues (such as denial of service) or other web vulnerabilities are
bug. It must be added at the top of `TODO.md`. P1 bugs and must be added to TODO.md at the top.
- Do not stub an implementation unless you are specifically instructed to. This - As this is production code, no stubbing of implementations unless specifically
is production code, and we need working implementations. instructed. We need working implementations.
- NEVER silently fall back to a different setting when a user's parameter - NEVER silently fall back to a different setting when a user's parameter
explicitly specifies a value. For example, if a user requests `format=webp` explicitly specifies a value. If a user requests format=webp and WebP encoding
and WebP encoding is not supported, return an error. Do NOT silently output is not supported, return an error - do NOT silently output PNG instead. If a
PNG instead. If a user specifies `fit=invalid` and that fit mode does not user specifies fit=invalid and that fit mode doesn't exist, return an error -
exist, return an error. Do NOT silently default to `cover`. A silent fallback do NOT silently default to "cover". Silent fallbacks violate the principle of
violates the principle of least surprise and masks bugs. Defaults are least surprise and mask bugs. The only acceptable defaults are for OMITTED
acceptable only for OMITTED parameters, never for INVALID explicit values. parameters, never for INVALID explicit values.
- Avoid vendoring dependencies unless specifically instructed to. NEVER commit - Avoid vendoring deps unless specifically instructed to. NEVER commit the
the `vendor/` directory. NEVER commit compiled binaries. If such a directory vendor directory, NEVER commit compiled binaries. If these directories or
or file exists and `.gitignore` does not list it, add it to `.gitignore`. files exist, add them to .gitignore (and commit the .gitignore) if they are
Commit that `.gitignore` change. Keep the entire git repository, history not already in there. Keep the entire git repository (with history) small -
included, under 20 MiB. Go over 20 MiB only when you specifically must commit under 20MiB, unless you specifically must commit larger files (e.g. test
a large file, such as a media file used as a test fixture. Only OUR source fixture example media files). Only OUR source code and immediately supporting
code and its immediately supporting files, such as test examples, go into the files (such as test examples) goes into the repo/history.
repository and its history.
+33 -37
View File
@@ -266,8 +266,8 @@ to any method the list does not give for it.
- `POST /generate` — make an encrypted URL from the generator page's form. Needs - `POST /generate` — make an encrypted URL from the generator page's form. Needs
a login session and the generator page's form. Without a login session it a login session and the generator page's form. Without a login session it
answers 303 to `/`. Answers 200 with the page showing the URL. Answers 400 answers 303 to `/`. Answers 200 with the page showing the URL. Answers 400
with the page naming the field it refused (see Encrypted URLs). Answers 500 with the page naming a field that is not valid. Answers 500 when the URL
when the URL cannot be made. cannot be made.
- `POST /logout` — log out with the generator page's Logout button. That ends - `POST /logout` — log out with the generator page's Logout button. That ends
every login session, in every browser (see Encrypted URLs), and removes the every login session, in every browser (see Encrypted URLs), and removes the
cookie. Needs a login session and the generator page's form. Without a login cookie. Needs a login session and the generator page's form. Without a login
@@ -347,8 +347,7 @@ to any method the list does not give for it.
A browser's CORS preflight request to any path under `/v1/` answers 200, in A browser's CORS preflight request to any path under `/v1/` answers 200, in
maintenance mode too. CORS is the browser rule that lets a page on one site read maintenance mode too. CORS is the browser rule that lets a page on one site read
responses from another. The preflight is the `OPTIONS` request the browser sends responses from another. The preflight is the `OPTIONS` request the browser sends
first. It carries an `Origin` header and an `Access-Control-Request-Method` first.
header.
Every response carries an `X-Request-Id` header with the request's ID. A client Every response carries an `X-Request-Id` header with the request's ID. A client
can quote it when reporting a problem. A reverse proxy may send an can quote it when reporting a problem. A reverse proxy may send an
@@ -475,10 +474,9 @@ it would come out above 8192, the request is refused with 400. The fit modes:
An AVIF output of more than 16,777,216 pixels is refused with 400 before An AVIF output of more than 16,777,216 pixels is refused with 400 before
anything is decoded. That is the number of pixels in 4096x4096 or 8192x2048. A anything is decoded. That is the number of pixels in 4096x4096 or 8192x2048. A
larger AVIF encode could take more than 40 seconds (see Limits). The limit also larger AVIF encode could take more than 40 seconds (see Limits). The limit
applies to the format `orig` when the source is AVIF, and to `auto` when it includes `orig` of an AVIF source, and `auto` when it chooses AVIF. So a client
chooses AVIF. So a client whose `Accept` names `image/avif` but not `image/jxl` whose `Accept` names `image/avif` but not `image/jxl` is refused such a size.
is refused such a size.
**Source formats.** The source may be JPEG, PNG, GIF, WebP, AVIF or JPEG XL. The **Source formats.** The source may be JPEG, PNG, GIF, WebP, AVIF or JPEG XL. The
upstream's `Content-Type` must name its format, and the image's first bytes must upstream's `Content-Type` must name its format, and the image's first bytes must
@@ -622,8 +620,7 @@ The form's fields:
- The quality, a whole number from 1 to 100. The page offers five values: `25`, - The quality, a whole number from 1 to 100. The page offers five values: `25`,
`50`, `70`, `85` and `100`. `85` is selected unless another is chosen. `50`, `70`, `85` and `100`. `85` is selected unless another is chosen.
- The fit, one of the five fit modes (see Image URLs). `cover` is selected - The fit, one of the five fit modes (see Image URLs). `cover` is selected
unless another is chosen. A form sent with an empty fit, or none, makes a unless another is chosen.
`cover` URL.
- How long the URL lasts, in seconds, or `0` for never. The page offers seven - How long the URL lasts, in seconds, or `0` for never. The page offers seven
choices: never, 1 minute, 1 hour, 1 day, 1 week, 30 days and 1 year. Never is choices: never, 1 minute, 1 hour, 1 day, 1 week, 30 days and 1 year. Never is
selected unless another is chosen. selected unless another is chosen.
@@ -746,7 +743,7 @@ processing slot. If its `downstream_timeout` ends after it has a slot but before
it begins to encode, it gives the slot back. For a shared conversion, the first it begins to encode, it gives the slot back. For a shared conversion, the first
request's deadline counts. When a shared conversion stops this way, each request request's deadline counts. When a shared conversion stops this way, each request
waiting for it is answered 503 `server busy, try again later`, even if its own waiting for it is answered 503 `server busy, try again later`, even if its own
`downstream_timeout` has not ended. `govips`, the library pixa drives libvips `downstream_timeout` has not ended. govips, the library pixa drives libvips
through, gives no way to stop an encode once it has begun. So an encode keeps through, gives no way to stop an encode once it has begun. So an encode keeps
its slot until it finishes, which can be after `downstream_timeout`. its slot until it finishes, which can be after `downstream_timeout`.
@@ -754,8 +751,8 @@ its slot until it finishes, which can be after `downstream_timeout`.
They are 8192 on each side of the source and of the output, and 16,777,216 They are 8192 on each side of the source and of the output, and 16,777,216
pixels for an AVIF output. Decoding an 8192x8192 source takes up to about 11 pixels for an AVIF output. Decoding an 8192x8192 source takes up to about 11
seconds. The WebP and JPEG XL encoders run below libvips' default effort. AVIF seconds. The WebP and JPEG XL encoders run below libvips' default effort. AVIF
runs at the lowest effort `govips` can set. With one libvips thread, as pixa runs at the lowest effort govips can set. With one libvips thread, as pixa runs
runs it, encoding 8192x8192 random pixels takes up to about: it, encoding 8192x8192 random pixels takes up to about:
- 1 second as JPEG; - 1 second as JPEG;
- 15 seconds as PNG; - 15 seconds as PNG;
@@ -765,10 +762,10 @@ runs it, encoding 8192x8192 random pixels takes up to about:
An AVIF output of 4096x4096, 8192x2048 or 2048x8192 random pixels takes up to An AVIF output of 4096x4096, 8192x2048 or 2048x8192 random pixels takes up to
about 34 seconds with an alpha channel, and 22 without. At 8192x8192 and quality about 34 seconds with an alpha channel, and 22 without. At 8192x8192 and quality
85 it would take about 96 seconds with an alpha channel, and 52 without. The 34 85 it would take about 96 seconds with an alpha channel, and 52 without. These
seconds hold at any quality and at 8 or 16 bits per sample. No encode within the times hold at any quality and at 8 or 16 bits per sample. No encode within the
limits takes more than about 40 seconds. The default `downstream_timeout` is 60 limits takes more than about 40 seconds of the default `downstream_timeout`
seconds. of 60.
**Processing and connections.** `max_concurrent_processing` images are decoded **Processing and connections.** `max_concurrent_processing` images are decoded
and encoded at once. A request that finds every slot in use waits up to 10 and encoded at once. A request that finds every slot in use waits up to 10
@@ -1045,15 +1042,15 @@ Each item below is one setting, named by its config key:
time allowed for answering one client request. Keep it longer than time allowed for answering one client request. Keep it longer than
`upstream_fetch_timeout` plus 20 seconds (see Limits). `upstream_fetch_timeout` plus 20 seconds (see Limits).
- `access_control_allow_origin`: variable `PIXA_ACCESS_CONTROL_ALLOW_ORIGIN`, - `access_control_allow_origin`: variable `PIXA_ACCESS_CONTROL_ALLOW_ORIGIN`,
default `*`. The origin whose pages may read the responses of `/v1/image/` and default `*`. The origin a browser lets read the responses of `/v1/image/` and
`/v1/e/` in a browser. pixa sends it as the CORS `Access-Control-Allow-Origin` `/v1/e/`, sent as the CORS `Access-Control-Allow-Origin` header. No other
header. No other route sends it. `*` is any site. Otherwise it is one `http` route sends it. `*` is any site. Otherwise it is one `http` or `https` origin
or `https` origin such as `https://example.com`. Its host is a lowercase host such as `https://example.com`. Its host is a lowercase host name (letters,
name (letters, digits, hyphens and dots, with a letter in its last part) or an digits, hyphens and dots, with a letter in its last part) or an IP address
IP address (IPv6 in brackets, in its shortest form). It may have a port from 1 (IPv6 in brackets, in its shortest form). It may have a port from 1 to 65535
to 65535 that has no leading zero and is not the scheme's default. Any other that has no leading zero and is not the scheme's default. Any other value
value aborts startup, including an origin with another scheme, such as a aborts startup, including an origin with another scheme, such as a browser
browser extension's. extension's.
- `metrics.username`: variable `PIXA_METRICS_USERNAME`, default empty. The - `metrics.username`: variable `PIXA_METRICS_USERNAME`, default empty. The
username for `/metrics`. pixa serves `/metrics`, and measures every HTTP username for `/metrics`. pixa serves `/metrics`, and measures every HTTP
request, only when the username and the password are both set. Setting one request, only when the username and the password are both set. Setting one
@@ -1083,9 +1080,8 @@ Each item below is one setting, named by its config key:
image cache, the image proxy and the other parts that serve images and logins. image cache, the image proxy and the other parts that serve images and logins.
Each setting's default is declared once, in `internal/config`. Each setting's default is declared once, in `internal/config`.
- **Packages**: these packages under `internal/` work with the concepts above: - **Packages**: these packages under `internal/` work with the concepts above:
- `handlers`: the functions the routes call, which read the image request - `handlers`: the routes, which read the image request from the URL, check
from the URL, check the signed or encrypted URL, and make the login the signed or encrypted URL, and make the login session
session
- `imagerequest`: the image request, with the source key and the variant key - `imagerequest`: the image request, with the source key and the variant key
- `imageproxy`: the image proxy - `imageproxy`: the image proxy
- `imgcache`: the image cache, which keeps its sources in one `FileStore` - `imgcache`: the image cache, which keeps its sources in one `FileStore`
@@ -1099,7 +1095,7 @@ Each item below is one setting, named by its config key:
tokens tokens
- `session`: the login session - `session`: the login session
- **HTTP router**: `chi` (`github.com/go-chi/chi/v5`) - **HTTP router**: `chi` (`github.com/go-chi/chi/v5`)
- **Image processing**: `govips`, a Go wrapper for libvips - **Image processing**: govips, a Go wrapper for libvips
- **Database**: SQLite via `modernc.org/sqlite` - **Database**: SQLite via `modernc.org/sqlite`
- **Static assets**: embedded in the binary via `//go:embed` - **Static assets**: embedded in the binary via `//go:embed`
- **Metrics**: Prometheus, at `/metrics`. Prometheus is a monitoring system that - **Metrics**: Prometheus, at `/metrics`. Prometheus is a monitoring system that
@@ -1163,8 +1159,8 @@ workflow, and most `Makefile` targets are thin shims that call them. We provide:
It is a benchmark, not part of `script/check` (see Load Test). It is a benchmark, not part of `script/check` (see Load Test).
- `script/cibuild` — the CI entrypoint: run `script/bootstrap` (without - `script/cibuild` — the CI entrypoint: run `script/bootstrap` (without
`--cgo`), then `script/check`, then build the image as `script/docker` does. `--cgo`), then `script/check`, then build the image as `script/docker` does.
- `script/precommit` — run `go mod tidy` and fail when it changes `go.mod` or - `script/precommit` — the pre-commit checks: the `go mod tidy` guard, then
`go.sum`, then run `script/check`. `script/check`.
- `script/install-precommit` — install the git pre-commit hook that runs - `script/install-precommit` — install the git pre-commit hook that runs
`script/precommit`. `script/precommit`.
@@ -1214,10 +1210,10 @@ script/loadtest 30s 32 # 30 seconds per scenario, 32 clients
``` ```
It builds the image with `script/docker`. It also builds the load tool, It builds the image with `script/docker`. It also builds the load tool,
[`vegeta`](https://github.com/tsenart/vegeta), from a pinned commit. Each [vegeta](https://github.com/tsenart/vegeta), from a pinned commit. Each scenario
scenario starts a new `pixad` container and a new origin container, starts a new `pixad` container and a new origin container,
`cmd/loadtest-origin`. The origin is an upstream host that answers every path `cmd/loadtest-origin`. The origin is an upstream host that answers every path
with the same generated 1600x1200 JPEG. `vegeta` then sends requests from the with the same generated 1600x1200 JPEG. vegeta then sends requests from the
given number of clients. Each client sends its next request as soon as its last given number of clients. Each client sends its next request as soon as its last
one is answered. Every request asks for an image resized to 400x300 WebP: one is answered. Every request asks for an image resized to 400x300 WebP:
@@ -1232,7 +1228,7 @@ containers share a Docker network in `203.0.113.0/24`, a range set aside for
documentation. A second run on the same Docker host while one is going fails, as documentation. A second run on the same Docker host while one is going fails, as
it cannot create that network. it cannot create that network.
For each scenario the script prints `vegeta`'s report and two lines of its own: For each scenario the script prints vegeta's report and two lines of its own:
- `Requests [total, rate, throughput]`: the requests sent, how many were sent - `Requests [total, rate, throughput]`: the requests sent, how many were sent
per second, and how many were answered successfully per second; per second, and how many were answered successfully per second;
@@ -1242,7 +1238,7 @@ For each scenario the script prints `vegeta`'s report and two lines of its own:
- `Status Codes` and `Error Set`: anything other than `200` means the other - `Status Codes` and `Error Set`: anything other than `200` means the other
numbers are not for the scenario described, such as `503` when `pixad` was numbers are not for the scenario described, such as `503` when `pixad` was
busy; busy;
- `Bytes In`: `0`, as `vegeta` is told not to keep the images it receives; - `Bytes In`: `0`, as vegeta is told not to keep the images it receives;
- `pixad peak memory (VmHWM)`: the peak resident memory of the `pixad` process - `pixad peak memory (VmHWM)`: the peak resident memory of the `pixad` process
since its container started, in kB; for `hit` it includes the request that put since its container started, in kB; for `hit` it includes the request that put
the image in the cache; the image in the cache;