Add script/loadtest to measure throughput, latency and memory (closes #81)

script/loadtest [duration [clients]], or make loadtest, is a benchmark
that script/check does not run. It builds the image and vegeta from a
pinned commit, then for each scenario starts a new pixad container and
a new cmd/loadtest-origin container, an upstream host that answers every
path with one generated JPEG: a cached image (hit), a new source image
per request (miss), and each new source image asked for by all clients
at once (herd). It prints vegeta's report, pixad's peak resident memory
and the requests the origin got. The containers share a network in
203.0.113.0/24, as pixad refuses private and local upstream addresses.
README.md says how to run and read it; TODO.md records a small baseline.

Model: opus-5-5
This commit is contained in:
2026-10-04 23:08:28 +00:00
parent 721502c7b8
commit 18b4ee38aa
5 changed files with 328 additions and 6 deletions
+17 -1
View File
@@ -40,6 +40,21 @@ P2: security: per-IP rate limiting on the image routes
comment are copied from the canonical `.gitignore` in `sneak/prompts`,
unanchored so it matches at every depth. `.dockerignore` already has
`.claude`.
- 2026-10-04 load test (closes #81): `script/loadtest [duration [clients]]`
(`make loadtest`, defaults `10s` and `4`), a benchmark that `script/check`
does not run, measures three scenarios, each against a new pixad container and
a new upstream host, `cmd/loadtest-origin`: `hit` (one cached image), `miss`
(a new source image every request) and `herd` (each new source image asked for
by all clients at once). For each it prints vegeta's report (requests per
second, latency percentiles, status codes), pixad's peak resident memory and
the requests that reached the origin. `README.md` says how to run it and read
it, and keeps 1-5k r/s as a target not yet measured. First measurement, with
the defaults on a shared 48-CPU machine with other work running: a baseline
for later changes, not a test of the target. `hit` 1413 r/s, p50 0.7 ms, p95
8.7 ms, p99 44 ms, peak 53 MiB (4 clients that each wait for their answer, so
not pixad's limit); `miss` 70 r/s, p50 52 ms, p95 91 ms, p99 122 ms, peak 100
MiB, one fetch per request; `herd` 74 r/s, p50 52 ms, p95 69 ms, p99 111 ms,
peak 60 MiB, 188 fetches for 749 requests.
- 2026-10-04 `.dockerignore` keeps secrets out at every depth (closes #205): the
file is now the standard one from `sneak/prompts`, whose patterns match in
every directory and, for environment files and private keys, in any letter
@@ -636,4 +651,5 @@ P2: security: per-IP rate limiting on the image routes
- optional Sentry error reporting
- comprehensive request logging
- Prometheus performance metrics
- load tests to verify the 1k to 5k req/s target
- measure the 1k to 5k req/s target with `script/loadtest` on a machine not
shared with other work