Compare commits
2
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
410dd92292 | ||
|
|
9f1e9f5253 |
No files matched your search
@@ -1,75 +1,76 @@
|
||||
# Repository Rules
|
||||
|
||||
Last Updated 2026-01-08
|
||||
Last Updated 2026-10-27
|
||||
|
||||
These rules MUST be followed at all times, it is very important.
|
||||
These rules MUST be followed at all times. This is very important.
|
||||
|
||||
- Never use `git add -A` - add specific changes to a deliberate commit. A commit
|
||||
should contain one change. After each change, make a commit with a good
|
||||
one-line summary.
|
||||
- Never use `git add -A`. Stage specific changes for a deliberate commit. A
|
||||
commit holds one change. After each change, make a commit with a good one-line
|
||||
summary.
|
||||
|
||||
- NEVER modify the linter config without asking first.
|
||||
|
||||
- NEVER modify tests to exclude special cases or otherwise get them to pass
|
||||
without asking first. In almost all cases, the code should be changed, NOT the
|
||||
tests. If you think the test needs to be changed, make your case for that and
|
||||
ask for permission to proceed, then stop. You need explicit user approval to
|
||||
modify existing tests. A bug fix may change an existing test without asking
|
||||
when that test pins exactly the behavior the filed bug calls wrong, and the
|
||||
change moves it to the fixed behavior and nothing else; any other change to an
|
||||
existing test still needs explicit user approval. (You do not need user
|
||||
approval for writing NEW tests.)
|
||||
- NEVER modify a test to exclude special cases or otherwise make it pass without
|
||||
asking first. In almost all cases, change the code, NOT the test. You need
|
||||
explicit user approval to modify an existing test. If you think a test needs
|
||||
to change, make your case and ask for permission to proceed. Then stop. A bug
|
||||
fix may change an existing test without asking only when that test pins
|
||||
exactly the behavior the filed bug calls wrong. The change must then move the
|
||||
test to the fixed behavior and nothing else. Any other change to an existing
|
||||
test needs explicit user approval. Writing NEW tests needs no approval.
|
||||
|
||||
- When linting, assume the linter config is CORRECT, and that each item output
|
||||
by the linter is something that legitimately needs fixing in the code.
|
||||
- When linting, assume the linter config is CORRECT. Treat each item the linter
|
||||
reports as something that legitimately needs fixing in the code.
|
||||
|
||||
- When running tests, use `make test`.
|
||||
|
||||
- Before commits, run `make check`. This runs `make lint` and `make test` and
|
||||
`make check-fmt`. Any issues discovered MUST be resolved before committing
|
||||
unless explicitly told otherwise.
|
||||
- Before committing, run `make check`. It runs `make test`, `make lint` and
|
||||
`make fmt-check`. Any issue it finds MUST be resolved before committing,
|
||||
unless you are explicitly told otherwise.
|
||||
|
||||
- When fixing a bug, write a failing test for the bug FIRST. Add appropriate
|
||||
logging to the test to ensure it is written correctly. Commit that. Then go
|
||||
about fixing the bug until the test passes (without modifying the test
|
||||
further). Then commit that.
|
||||
logging to the test, to make sure the test is written correctly. Commit that.
|
||||
Then work on the fix until the test passes, without changing the test further.
|
||||
Then commit the fix.
|
||||
|
||||
- When adding a new feature, do the same - implement a test first (TDD). It
|
||||
doesn't have to be super complex. Commit the test, then commit the feature.
|
||||
- When adding a new feature, do the same: write a test first (test-driven
|
||||
development, TDD). The test does not need to be very complex. Commit the test,
|
||||
then commit the feature.
|
||||
|
||||
- When adding a new feature, use a feature branch. When the feature is
|
||||
completely finished and the code is up to standards (passes `make check`) then
|
||||
and only then can the feature branch be merged into `main` and the branch
|
||||
deleted.
|
||||
- When adding a new feature, use a feature branch. The feature branch reaches
|
||||
`next` through a PR that passes review. It is merged only when the feature is
|
||||
completely finished and the code is up to standards. The code is up to
|
||||
standards when it passes `make check`. The branch is deleted after the merge.
|
||||
Only the user merges `next` into `main`.
|
||||
|
||||
- Write godoc documentation comments for all exported types and functions as you
|
||||
go along.
|
||||
- Write `godoc` comments for all exported types and functions as you go along.
|
||||
|
||||
- ALWAYS be consistent in naming. If you name something one thing in one place,
|
||||
name it the EXACT SAME THING in another place.
|
||||
- ALWAYS be consistent in naming. If you name something one way in one place,
|
||||
use the EXACT SAME name in every other place.
|
||||
|
||||
- Be descriptive and specific in naming. `wl` is bad; `SourceHostWhitelist` is
|
||||
- Be descriptive and specific in naming. `al` is bad; `SourceHostAllowlist` is
|
||||
good. `ConnsPerHost` is bad; `MaxConnectionsPerHost` is good.
|
||||
|
||||
- This is not prototype or teaching code - this is designed for production. Any
|
||||
security issues (such as denial of service) or other web vulnerabilities are
|
||||
P1 bugs and must be added to TODO.md at the top.
|
||||
- This is not prototype or teaching code. It is designed for production. Any
|
||||
security issue (such as denial of service) or other web vulnerability is a P1
|
||||
bug. It must be added at the top of `TODO.md`.
|
||||
|
||||
- As this is production code, no stubbing of implementations unless specifically
|
||||
instructed. We need working implementations.
|
||||
- Do not stub an implementation unless you are specifically instructed to. This
|
||||
is production code, and we need working implementations.
|
||||
|
||||
- NEVER silently fall back to a different setting when a user's parameter
|
||||
explicitly specifies a value. If a user requests format=webp and WebP encoding
|
||||
is not supported, return an error - do NOT silently output PNG instead. If a
|
||||
user specifies fit=invalid and that fit mode doesn't exist, return an error -
|
||||
do NOT silently default to "cover". Silent fallbacks violate the principle of
|
||||
least surprise and mask bugs. The only acceptable defaults are for OMITTED
|
||||
parameters, never for INVALID explicit values.
|
||||
explicitly specifies a value. For example, if a user requests `format=webp`
|
||||
and WebP encoding is not supported, return an error. Do NOT silently output
|
||||
PNG instead. If a user specifies `fit=invalid` and that fit mode does not
|
||||
exist, return an error. Do NOT silently default to `cover`. A silent fallback
|
||||
violates the principle of least surprise and masks bugs. Defaults are
|
||||
acceptable only for OMITTED parameters, never for INVALID explicit values.
|
||||
|
||||
- Avoid vendoring deps unless specifically instructed to. NEVER commit the
|
||||
vendor directory, NEVER commit compiled binaries. If these directories or
|
||||
files exist, add them to .gitignore (and commit the .gitignore) if they are
|
||||
not already in there. Keep the entire git repository (with history) small -
|
||||
under 20MiB, unless you specifically must commit larger files (e.g. test
|
||||
fixture example media files). Only OUR source code and immediately supporting
|
||||
files (such as test examples) goes into the repo/history.
|
||||
- Avoid vendoring dependencies unless specifically instructed to. NEVER commit
|
||||
the `vendor/` directory. NEVER commit compiled binaries. If such a directory
|
||||
or file exists and `.gitignore` does not list it, add it to `.gitignore`.
|
||||
Commit that `.gitignore` change. Keep the entire git repository, history
|
||||
included, under 20 MiB. Go over 20 MiB only when you specifically must commit
|
||||
a large file, such as a media file used as a test fixture. Only OUR source
|
||||
code and its immediately supporting files, such as test examples, go into the
|
||||
repository and its history.
|
||||
@@ -266,8 +266,8 @@ to any method the list does not give for it.
|
||||
- `POST /generate` — make an encrypted URL from the generator page's form. Needs
|
||||
a login session and the generator page's form. Without a login session it
|
||||
answers 303 to `/`. Answers 200 with the page showing the URL. Answers 400
|
||||
with the page naming a field that is not valid. Answers 500 when the URL
|
||||
cannot be made.
|
||||
with the page naming the field it refused (see Encrypted URLs). Answers 500
|
||||
when the URL cannot be made.
|
||||
- `POST /logout` — log out with the generator page's Logout button. That ends
|
||||
every login session, in every browser (see Encrypted URLs), and removes the
|
||||
cookie. Needs a login session and the generator page's form. Without a login
|
||||
@@ -347,7 +347,8 @@ to any method the list does not give for it.
|
||||
A browser's CORS preflight request to any path under `/v1/` answers 200, in
|
||||
maintenance mode too. CORS is the browser rule that lets a page on one site read
|
||||
responses from another. The preflight is the `OPTIONS` request the browser sends
|
||||
first.
|
||||
first. It carries an `Origin` header and an `Access-Control-Request-Method`
|
||||
header.
|
||||
|
||||
Every response carries an `X-Request-Id` header with the request's ID. A client
|
||||
can quote it when reporting a problem. A reverse proxy may send an
|
||||
@@ -474,9 +475,10 @@ it would come out above 8192, the request is refused with 400. The fit modes:
|
||||
|
||||
An AVIF output of more than 16,777,216 pixels is refused with 400 before
|
||||
anything is decoded. That is the number of pixels in 4096x4096 or 8192x2048. A
|
||||
larger AVIF encode could take more than 40 seconds (see Limits). The limit
|
||||
includes `orig` of an AVIF source, and `auto` when it chooses AVIF. So a client
|
||||
whose `Accept` names `image/avif` but not `image/jxl` is refused such a size.
|
||||
larger AVIF encode could take more than 40 seconds (see Limits). The limit also
|
||||
applies to the format `orig` when the source is AVIF, and to `auto` when it
|
||||
chooses AVIF. So a client whose `Accept` names `image/avif` but not `image/jxl`
|
||||
is refused such a size.
|
||||
|
||||
**Source formats.** The source may be JPEG, PNG, GIF, WebP, AVIF or JPEG XL. The
|
||||
upstream's `Content-Type` must name its format, and the image's first bytes must
|
||||
@@ -620,7 +622,8 @@ The form's fields:
|
||||
- The quality, a whole number from 1 to 100. The page offers five values: `25`,
|
||||
`50`, `70`, `85` and `100`. `85` is selected unless another is chosen.
|
||||
- The fit, one of the five fit modes (see Image URLs). `cover` is selected
|
||||
unless another is chosen.
|
||||
unless another is chosen. A form sent with an empty fit, or none, makes a
|
||||
`cover` URL.
|
||||
- How long the URL lasts, in seconds, or `0` for never. The page offers seven
|
||||
choices: never, 1 minute, 1 hour, 1 day, 1 week, 30 days and 1 year. Never is
|
||||
selected unless another is chosen.
|
||||
@@ -743,7 +746,7 @@ processing slot. If its `downstream_timeout` ends after it has a slot but before
|
||||
it begins to encode, it gives the slot back. For a shared conversion, the first
|
||||
request's deadline counts. When a shared conversion stops this way, each request
|
||||
waiting for it is answered 503 `server busy, try again later`, even if its own
|
||||
`downstream_timeout` has not ended. govips, the library pixa drives libvips
|
||||
`downstream_timeout` has not ended. `govips`, the library pixa drives libvips
|
||||
through, gives no way to stop an encode once it has begun. So an encode keeps
|
||||
its slot until it finishes, which can be after `downstream_timeout`.
|
||||
|
||||
@@ -751,8 +754,8 @@ its slot until it finishes, which can be after `downstream_timeout`.
|
||||
They are 8192 on each side of the source and of the output, and 16,777,216
|
||||
pixels for an AVIF output. Decoding an 8192x8192 source takes up to about 11
|
||||
seconds. The WebP and JPEG XL encoders run below libvips' default effort. AVIF
|
||||
runs at the lowest effort govips can set. With one libvips thread, as pixa runs
|
||||
it, encoding 8192x8192 random pixels takes up to about:
|
||||
runs at the lowest effort `govips` can set. With one libvips thread, as pixa
|
||||
runs it, encoding 8192x8192 random pixels takes up to about:
|
||||
|
||||
- 1 second as JPEG;
|
||||
- 15 seconds as PNG;
|
||||
@@ -762,10 +765,10 @@ it, encoding 8192x8192 random pixels takes up to about:
|
||||
|
||||
An AVIF output of 4096x4096, 8192x2048 or 2048x8192 random pixels takes up to
|
||||
about 34 seconds with an alpha channel, and 22 without. At 8192x8192 and quality
|
||||
85 it would take about 96 seconds with an alpha channel, and 52 without. These
|
||||
times hold at any quality and at 8 or 16 bits per sample. No encode within the
|
||||
limits takes more than about 40 seconds of the default `downstream_timeout`
|
||||
of 60.
|
||||
85 it would take about 96 seconds with an alpha channel, and 52 without. The 34
|
||||
seconds hold at any quality and at 8 or 16 bits per sample. No encode within the
|
||||
limits takes more than about 40 seconds. The default `downstream_timeout` is 60
|
||||
seconds.
|
||||
|
||||
**Processing and connections.** `max_concurrent_processing` images are decoded
|
||||
and encoded at once. A request that finds every slot in use waits up to 10
|
||||
@@ -1042,15 +1045,15 @@ Each item below is one setting, named by its config key:
|
||||
time allowed for answering one client request. Keep it longer than
|
||||
`upstream_fetch_timeout` plus 20 seconds (see Limits).
|
||||
- `access_control_allow_origin`: variable `PIXA_ACCESS_CONTROL_ALLOW_ORIGIN`,
|
||||
default `*`. The origin a browser lets read the responses of `/v1/image/` and
|
||||
`/v1/e/`, sent as the CORS `Access-Control-Allow-Origin` header. No other
|
||||
route sends it. `*` is any site. Otherwise it is one `http` or `https` origin
|
||||
such as `https://example.com`. Its host is a lowercase host name (letters,
|
||||
digits, hyphens and dots, with a letter in its last part) or an IP address
|
||||
(IPv6 in brackets, in its shortest form). It may have a port from 1 to 65535
|
||||
that has no leading zero and is not the scheme's default. Any other value
|
||||
aborts startup, including an origin with another scheme, such as a browser
|
||||
extension's.
|
||||
default `*`. The origin whose pages may read the responses of `/v1/image/` and
|
||||
`/v1/e/` in a browser. pixa sends it as the CORS `Access-Control-Allow-Origin`
|
||||
header. No other route sends it. `*` is any site. Otherwise it is one `http`
|
||||
or `https` origin such as `https://example.com`. Its host is a lowercase host
|
||||
name (letters, digits, hyphens and dots, with a letter in its last part) or an
|
||||
IP address (IPv6 in brackets, in its shortest form). It may have a port from 1
|
||||
to 65535 that has no leading zero and is not the scheme's default. Any other
|
||||
value aborts startup, including an origin with another scheme, such as a
|
||||
browser extension's.
|
||||
- `metrics.username`: variable `PIXA_METRICS_USERNAME`, default empty. The
|
||||
username for `/metrics`. pixa serves `/metrics`, and measures every HTTP
|
||||
request, only when the username and the password are both set. Setting one
|
||||
@@ -1080,8 +1083,9 @@ Each item below is one setting, named by its config key:
|
||||
image cache, the image proxy and the other parts that serve images and logins.
|
||||
Each setting's default is declared once, in `internal/config`.
|
||||
- **Packages**: these packages under `internal/` work with the concepts above:
|
||||
- `handlers`: the routes, which read the image request from the URL, check
|
||||
the signed or encrypted URL, and make the login session
|
||||
- `handlers`: the functions the routes call, which read the image request
|
||||
from the URL, check the signed or encrypted URL, and make the login
|
||||
session
|
||||
- `imagerequest`: the image request, with the source key and the variant key
|
||||
- `imageproxy`: the image proxy
|
||||
- `imgcache`: the image cache, which keeps its sources in one `FileStore`
|
||||
@@ -1095,7 +1099,7 @@ Each item below is one setting, named by its config key:
|
||||
tokens
|
||||
- `session`: the login session
|
||||
- **HTTP router**: `chi` (`github.com/go-chi/chi/v5`)
|
||||
- **Image processing**: govips, a Go wrapper for libvips
|
||||
- **Image processing**: `govips`, a Go wrapper for libvips
|
||||
- **Database**: SQLite via `modernc.org/sqlite`
|
||||
- **Static assets**: embedded in the binary via `//go:embed`
|
||||
- **Metrics**: Prometheus, at `/metrics`. Prometheus is a monitoring system that
|
||||
@@ -1159,8 +1163,8 @@ workflow, and most `Makefile` targets are thin shims that call them. We provide:
|
||||
It is a benchmark, not part of `script/check` (see Load Test).
|
||||
- `script/cibuild` — the CI entrypoint: run `script/bootstrap` (without
|
||||
`--cgo`), then `script/check`, then build the image as `script/docker` does.
|
||||
- `script/precommit` — the pre-commit checks: the `go mod tidy` guard, then
|
||||
`script/check`.
|
||||
- `script/precommit` — run `go mod tidy` and fail when it changes `go.mod` or
|
||||
`go.sum`, then run `script/check`.
|
||||
- `script/install-precommit` — install the git pre-commit hook that runs
|
||||
`script/precommit`.
|
||||
|
||||
@@ -1210,10 +1214,10 @@ script/loadtest 30s 32 # 30 seconds per scenario, 32 clients
|
||||
```
|
||||
|
||||
It builds the image with `script/docker`. It also builds the load tool,
|
||||
[vegeta](https://github.com/tsenart/vegeta), from a pinned commit. Each scenario
|
||||
starts a new `pixad` container and a new origin container,
|
||||
[`vegeta`](https://github.com/tsenart/vegeta), from a pinned commit. Each
|
||||
scenario starts a new `pixad` container and a new origin container,
|
||||
`cmd/loadtest-origin`. The origin is an upstream host that answers every path
|
||||
with the same generated 1600x1200 JPEG. vegeta then sends requests from the
|
||||
with the same generated 1600x1200 JPEG. `vegeta` then sends requests from the
|
||||
given number of clients. Each client sends its next request as soon as its last
|
||||
one is answered. Every request asks for an image resized to 400x300 WebP:
|
||||
|
||||
@@ -1228,7 +1232,7 @@ containers share a Docker network in `203.0.113.0/24`, a range set aside for
|
||||
documentation. A second run on the same Docker host while one is going fails, as
|
||||
it cannot create that network.
|
||||
|
||||
For each scenario the script prints vegeta's report and two lines of its own:
|
||||
For each scenario the script prints `vegeta`'s report and two lines of its own:
|
||||
|
||||
- `Requests [total, rate, throughput]`: the requests sent, how many were sent
|
||||
per second, and how many were answered successfully per second;
|
||||
@@ -1238,7 +1242,7 @@ For each scenario the script prints vegeta's report and two lines of its own:
|
||||
- `Status Codes` and `Error Set`: anything other than `200` means the other
|
||||
numbers are not for the scenario described, such as `503` when `pixad` was
|
||||
busy;
|
||||
- `Bytes In`: `0`, as vegeta is told not to keep the images it receives;
|
||||
- `Bytes In`: `0`, as `vegeta` is told not to keep the images it receives;
|
||||
- `pixad peak memory (VmHWM)`: the peak resident memory of the `pixad` process
|
||||
since its container started, in kB; for `hit` it includes the request that put
|
||||
the image in the cache;
|
||||
|
||||
Reference in new issue
Block a user