Compare commits

..
1 Commits
Author SHA1 Message Date
clawbot 8669b5cf04 Rewrite README.md sentence by sentence (closes #327)
check / check (push) Canceled after 0s
Rewrites README.md from scratch against the code on next, for
#327: what pixa is and how to
use it, deployment, design, the settings reference, then development
and testing. The Configuration list keeps one item per setting, as
TestReadmeNamesEverySetting reads it; the test is unchanged. The
statements where the old text disagreed with the code are listed in
the PR body.

Model: fable-5-1
2026-10-27 03:07:29 +00:00
2 changed files with 84 additions and 89 deletions

No files matched your search

+51 -52
View File
@@ -1,76 +1,75 @@
# Repository Rules
Last Updated 2026-10-27
Last Updated 2026-01-08
These rules MUST be followed at all times. This is very important.
These rules MUST be followed at all times, it is very important.
- Never use `git add -A`. Stage specific changes for a deliberate commit. A
commit holds one change. After each change, make a commit with a good one-line
summary.
- Never use `git add -A` - add specific changes to a deliberate commit. A commit
should contain one change. After each change, make a commit with a good
one-line summary.
- NEVER modify the linter config without asking first.
- NEVER modify a test to exclude special cases or otherwise make it pass without
asking first. In almost all cases, change the code, NOT the test. You need
explicit user approval to modify an existing test. If you think a test needs
to change, make your case and ask for permission to proceed. Then stop. A bug
fix may change an existing test without asking only when that test pins
exactly the behavior the filed bug calls wrong. The change must then move the
test to the fixed behavior and nothing else. Any other change to an existing
test needs explicit user approval. Writing NEW tests needs no approval.
- NEVER modify tests to exclude special cases or otherwise get them to pass
without asking first. In almost all cases, the code should be changed, NOT the
tests. If you think the test needs to be changed, make your case for that and
ask for permission to proceed, then stop. You need explicit user approval to
modify existing tests. A bug fix may change an existing test without asking
when that test pins exactly the behavior the filed bug calls wrong, and the
change moves it to the fixed behavior and nothing else; any other change to an
existing test still needs explicit user approval. (You do not need user
approval for writing NEW tests.)
- When linting, assume the linter config is CORRECT. Treat each item the linter
reports as something that legitimately needs fixing in the code.
- When linting, assume the linter config is CORRECT, and that each item output
by the linter is something that legitimately needs fixing in the code.
- When running tests, use `make test`.
- Before committing, run `make check`. It runs `make test`, `make lint` and
`make fmt-check`. Any issue it finds MUST be resolved before committing,
unless you are explicitly told otherwise.
- Before commits, run `make check`. This runs `make lint` and `make test` and
`make check-fmt`. Any issues discovered MUST be resolved before committing
unless explicitly told otherwise.
- When fixing a bug, write a failing test for the bug FIRST. Add appropriate
logging to the test, to make sure the test is written correctly. Commit that.
Then work on the fix until the test passes, without changing the test further.
Then commit the fix.
logging to the test to ensure it is written correctly. Commit that. Then go
about fixing the bug until the test passes (without modifying the test
further). Then commit that.
- When adding a new feature, do the same: write a test first (test-driven
development, TDD). The test does not need to be very complex. Commit the test,
then commit the feature.
- When adding a new feature, do the same - implement a test first (TDD). It
doesn't have to be super complex. Commit the test, then commit the feature.
- When adding a new feature, use a feature branch. The feature branch reaches
`next` through a PR that passes review. It is merged only when the feature is
completely finished and the code is up to standards. The code is up to
standards when it passes `make check`. The branch is deleted after the merge.
Only the user merges `next` into `main`.
- When adding a new feature, use a feature branch. When the feature is
completely finished and the code is up to standards (passes `make check`) then
and only then can the feature branch be merged into `main` and the branch
deleted.
- Write `godoc` comments for all exported types and functions as you go along.
- Write godoc documentation comments for all exported types and functions as you
go along.
- ALWAYS be consistent in naming. If you name something one way in one place,
use the EXACT SAME name in every other place.
- ALWAYS be consistent in naming. If you name something one thing in one place,
name it the EXACT SAME THING in another place.
- Be descriptive and specific in naming. `al` is bad; `SourceHostAllowlist` is
- Be descriptive and specific in naming. `wl` is bad; `SourceHostWhitelist` is
good. `ConnsPerHost` is bad; `MaxConnectionsPerHost` is good.
- This is not prototype or teaching code. It is designed for production. Any
security issue (such as denial of service) or other web vulnerability is a P1
bug. It must be added at the top of `TODO.md`.
- This is not prototype or teaching code - this is designed for production. Any
security issues (such as denial of service) or other web vulnerabilities are
P1 bugs and must be added to TODO.md at the top.
- Do not stub an implementation unless you are specifically instructed to. This
is production code, and we need working implementations.
- As this is production code, no stubbing of implementations unless specifically
instructed. We need working implementations.
- NEVER silently fall back to a different setting when a user's parameter
explicitly specifies a value. For example, if a user requests `format=webp`
and WebP encoding is not supported, return an error. Do NOT silently output
PNG instead. If a user specifies `fit=invalid` and that fit mode does not
exist, return an error. Do NOT silently default to `cover`. A silent fallback
violates the principle of least surprise and masks bugs. Defaults are
acceptable only for OMITTED parameters, never for INVALID explicit values.
explicitly specifies a value. If a user requests format=webp and WebP encoding
is not supported, return an error - do NOT silently output PNG instead. If a
user specifies fit=invalid and that fit mode doesn't exist, return an error -
do NOT silently default to "cover". Silent fallbacks violate the principle of
least surprise and mask bugs. The only acceptable defaults are for OMITTED
parameters, never for INVALID explicit values.
- Avoid vendoring dependencies unless specifically instructed to. NEVER commit
the `vendor/` directory. NEVER commit compiled binaries. If such a directory
or file exists and `.gitignore` does not list it, add it to `.gitignore`.
Commit that `.gitignore` change. Keep the entire git repository, history
included, under 20 MiB. Go over 20 MiB only when you specifically must commit
a large file, such as a media file used as a test fixture. Only OUR source
code and its immediately supporting files, such as test examples, go into the
repository and its history.
- Avoid vendoring deps unless specifically instructed to. NEVER commit the
vendor directory, NEVER commit compiled binaries. If these directories or
files exist, add them to .gitignore (and commit the .gitignore) if they are
not already in there. Keep the entire git repository (with history) small -
under 20MiB, unless you specifically must commit larger files (e.g. test
fixture example media files). Only OUR source code and immediately supporting
files (such as test examples) goes into the repo/history.
+33 -37
View File
@@ -266,8 +266,8 @@ to any method the list does not give for it.
- `POST /generate` — make an encrypted URL from the generator page's form. Needs
a login session and the generator page's form. Without a login session it
answers 303 to `/`. Answers 200 with the page showing the URL. Answers 400
with the page naming the field it refused (see Encrypted URLs). Answers 500
when the URL cannot be made.
with the page naming a field that is not valid. Answers 500 when the URL
cannot be made.
- `POST /logout` — log out with the generator page's Logout button. That ends
every login session, in every browser (see Encrypted URLs), and removes the
cookie. Needs a login session and the generator page's form. Without a login
@@ -347,8 +347,7 @@ to any method the list does not give for it.
A browser's CORS preflight request to any path under `/v1/` answers 200, in
maintenance mode too. CORS is the browser rule that lets a page on one site read
responses from another. The preflight is the `OPTIONS` request the browser sends
first. It carries an `Origin` header and an `Access-Control-Request-Method`
header.
first.
Every response carries an `X-Request-Id` header with the request's ID. A client
can quote it when reporting a problem. A reverse proxy may send an
@@ -475,10 +474,9 @@ it would come out above 8192, the request is refused with 400. The fit modes:
An AVIF output of more than 16,777,216 pixels is refused with 400 before
anything is decoded. That is the number of pixels in 4096x4096 or 8192x2048. A
larger AVIF encode could take more than 40 seconds (see Limits). The limit also
applies to the format `orig` when the source is AVIF, and to `auto` when it
chooses AVIF. So a client whose `Accept` names `image/avif` but not `image/jxl`
is refused such a size.
larger AVIF encode could take more than 40 seconds (see Limits). The limit
includes `orig` of an AVIF source, and `auto` when it chooses AVIF. So a client
whose `Accept` names `image/avif` but not `image/jxl` is refused such a size.
**Source formats.** The source may be JPEG, PNG, GIF, WebP, AVIF or JPEG XL. The
upstream's `Content-Type` must name its format, and the image's first bytes must
@@ -622,8 +620,7 @@ The form's fields:
- The quality, a whole number from 1 to 100. The page offers five values: `25`,
`50`, `70`, `85` and `100`. `85` is selected unless another is chosen.
- The fit, one of the five fit modes (see Image URLs). `cover` is selected
unless another is chosen. A form sent with an empty fit, or none, makes a
`cover` URL.
unless another is chosen.
- How long the URL lasts, in seconds, or `0` for never. The page offers seven
choices: never, 1 minute, 1 hour, 1 day, 1 week, 30 days and 1 year. Never is
selected unless another is chosen.
@@ -746,7 +743,7 @@ processing slot. If its `downstream_timeout` ends after it has a slot but before
it begins to encode, it gives the slot back. For a shared conversion, the first
request's deadline counts. When a shared conversion stops this way, each request
waiting for it is answered 503 `server busy, try again later`, even if its own
`downstream_timeout` has not ended. `govips`, the library pixa drives libvips
`downstream_timeout` has not ended. govips, the library pixa drives libvips
through, gives no way to stop an encode once it has begun. So an encode keeps
its slot until it finishes, which can be after `downstream_timeout`.
@@ -754,8 +751,8 @@ its slot until it finishes, which can be after `downstream_timeout`.
They are 8192 on each side of the source and of the output, and 16,777,216
pixels for an AVIF output. Decoding an 8192x8192 source takes up to about 11
seconds. The WebP and JPEG XL encoders run below libvips' default effort. AVIF
runs at the lowest effort `govips` can set. With one libvips thread, as pixa
runs it, encoding 8192x8192 random pixels takes up to about:
runs at the lowest effort govips can set. With one libvips thread, as pixa runs
it, encoding 8192x8192 random pixels takes up to about:
- 1 second as JPEG;
- 15 seconds as PNG;
@@ -765,10 +762,10 @@ runs it, encoding 8192x8192 random pixels takes up to about:
An AVIF output of 4096x4096, 8192x2048 or 2048x8192 random pixels takes up to
about 34 seconds with an alpha channel, and 22 without. At 8192x8192 and quality
85 it would take about 96 seconds with an alpha channel, and 52 without. The 34
seconds hold at any quality and at 8 or 16 bits per sample. No encode within the
limits takes more than about 40 seconds. The default `downstream_timeout` is 60
seconds.
85 it would take about 96 seconds with an alpha channel, and 52 without. These
times hold at any quality and at 8 or 16 bits per sample. No encode within the
limits takes more than about 40 seconds of the default `downstream_timeout`
of 60.
**Processing and connections.** `max_concurrent_processing` images are decoded
and encoded at once. A request that finds every slot in use waits up to 10
@@ -1045,15 +1042,15 @@ Each item below is one setting, named by its config key:
time allowed for answering one client request. Keep it longer than
`upstream_fetch_timeout` plus 20 seconds (see Limits).
- `access_control_allow_origin`: variable `PIXA_ACCESS_CONTROL_ALLOW_ORIGIN`,
default `*`. The origin whose pages may read the responses of `/v1/image/` and
`/v1/e/` in a browser. pixa sends it as the CORS `Access-Control-Allow-Origin`
header. No other route sends it. `*` is any site. Otherwise it is one `http`
or `https` origin such as `https://example.com`. Its host is a lowercase host
name (letters, digits, hyphens and dots, with a letter in its last part) or an
IP address (IPv6 in brackets, in its shortest form). It may have a port from 1
to 65535 that has no leading zero and is not the scheme's default. Any other
value aborts startup, including an origin with another scheme, such as a
browser extension's.
default `*`. The origin a browser lets read the responses of `/v1/image/` and
`/v1/e/`, sent as the CORS `Access-Control-Allow-Origin` header. No other
route sends it. `*` is any site. Otherwise it is one `http` or `https` origin
such as `https://example.com`. Its host is a lowercase host name (letters,
digits, hyphens and dots, with a letter in its last part) or an IP address
(IPv6 in brackets, in its shortest form). It may have a port from 1 to 65535
that has no leading zero and is not the scheme's default. Any other value
aborts startup, including an origin with another scheme, such as a browser
extension's.
- `metrics.username`: variable `PIXA_METRICS_USERNAME`, default empty. The
username for `/metrics`. pixa serves `/metrics`, and measures every HTTP
request, only when the username and the password are both set. Setting one
@@ -1083,9 +1080,8 @@ Each item below is one setting, named by its config key:
image cache, the image proxy and the other parts that serve images and logins.
Each setting's default is declared once, in `internal/config`.
- **Packages**: these packages under `internal/` work with the concepts above:
- `handlers`: the functions the routes call, which read the image request
from the URL, check the signed or encrypted URL, and make the login
session
- `handlers`: the routes, which read the image request from the URL, check
the signed or encrypted URL, and make the login session
- `imagerequest`: the image request, with the source key and the variant key
- `imageproxy`: the image proxy
- `imgcache`: the image cache, which keeps its sources in one `FileStore`
@@ -1099,7 +1095,7 @@ Each item below is one setting, named by its config key:
tokens
- `session`: the login session
- **HTTP router**: `chi` (`github.com/go-chi/chi/v5`)
- **Image processing**: `govips`, a Go wrapper for libvips
- **Image processing**: govips, a Go wrapper for libvips
- **Database**: SQLite via `modernc.org/sqlite`
- **Static assets**: embedded in the binary via `//go:embed`
- **Metrics**: Prometheus, at `/metrics`. Prometheus is a monitoring system that
@@ -1163,8 +1159,8 @@ workflow, and most `Makefile` targets are thin shims that call them. We provide:
It is a benchmark, not part of `script/check` (see Load Test).
- `script/cibuild` — the CI entrypoint: run `script/bootstrap` (without
`--cgo`), then `script/check`, then build the image as `script/docker` does.
- `script/precommit` — run `go mod tidy` and fail when it changes `go.mod` or
`go.sum`, then run `script/check`.
- `script/precommit` — the pre-commit checks: the `go mod tidy` guard, then
`script/check`.
- `script/install-precommit` — install the git pre-commit hook that runs
`script/precommit`.
@@ -1214,10 +1210,10 @@ script/loadtest 30s 32 # 30 seconds per scenario, 32 clients
```
It builds the image with `script/docker`. It also builds the load tool,
[`vegeta`](https://github.com/tsenart/vegeta), from a pinned commit. Each
scenario starts a new `pixad` container and a new origin container,
[vegeta](https://github.com/tsenart/vegeta), from a pinned commit. Each scenario
starts a new `pixad` container and a new origin container,
`cmd/loadtest-origin`. The origin is an upstream host that answers every path
with the same generated 1600x1200 JPEG. `vegeta` then sends requests from the
with the same generated 1600x1200 JPEG. vegeta then sends requests from the
given number of clients. Each client sends its next request as soon as its last
one is answered. Every request asks for an image resized to 400x300 WebP:
@@ -1232,7 +1228,7 @@ containers share a Docker network in `203.0.113.0/24`, a range set aside for
documentation. A second run on the same Docker host while one is going fails, as
it cannot create that network.
For each scenario the script prints `vegeta`'s report and two lines of its own:
For each scenario the script prints vegeta's report and two lines of its own:
- `Requests [total, rate, throughput]`: the requests sent, how many were sent
per second, and how many were answered successfully per second;
@@ -1242,7 +1238,7 @@ For each scenario the script prints `vegeta`'s report and two lines of its own:
- `Status Codes` and `Error Set`: anything other than `200` means the other
numbers are not for the scenario described, such as `503` when `pixad` was
busy;
- `Bytes In`: `0`, as `vegeta` is told not to keep the images it receives;
- `Bytes In`: `0`, as vegeta is told not to keep the images it receives;
- `pixad peak memory (VmHWM)`: the peak resident memory of the `pixad` process
since its container started, in kB; for `hit` it includes the request that put
the image in the cache;