Compare commits
1
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
a90c19aed1 |
@@ -30,12 +30,11 @@ make docker
|
||||
docker run -p 8080:8080 -e PIXA_SIGNING_KEY="$(openssl rand -base64 32)" pixa:latest
|
||||
```
|
||||
|
||||
`make build` compiles pixa on your machine into `bin/pixad`. `pixad` is the
|
||||
server program. Compiling needs the libraries `script/bootstrap --cgo` installs
|
||||
(see Deployment).
|
||||
`make build` compiles pixa on your machine. That needs the libraries
|
||||
`script/bootstrap --cgo` installs (see Deployment).
|
||||
|
||||
The signing key is pixa's one required secret. It must be 32 characters or
|
||||
longer. The placeholder in the example file is refused at startup.
|
||||
The signing key is pixa's one secret. It must be 32 characters or longer. The
|
||||
placeholder in the example file is refused at startup.
|
||||
|
||||
Once pixa runs, it serves images through URLs of this form:
|
||||
|
||||
@@ -70,6 +69,8 @@ variable wins over the same setting in the file.
|
||||
pixa listens on plain HTTP. It runs behind a reverse proxy: a server in front of
|
||||
it that receives the public requests and passes them on. The reverse proxy
|
||||
terminates TLS, the encryption behind `https`.
|
||||
[`configs/Caddyfile`](configs/Caddyfile) is an example for Caddy. Caddy gets its
|
||||
own TLS certificate and does everything in this list without further settings.
|
||||
|
||||
The reverse proxy must:
|
||||
|
||||
@@ -79,16 +80,12 @@ The reverse proxy must:
|
||||
form from its pages unless `Origin` or `Referer` names the host in `Host`. It
|
||||
builds encrypted URLs from `Host`. It checks `Referer` against
|
||||
`referer_blocklist`.
|
||||
- set `X-Forwarded-For` to the client's address. `trusted_proxies` must hold the
|
||||
proxy's address as pixa sees it. The login limit and
|
||||
- set `X-Forwarded-For` to the client's address, with `trusted_proxies` naming
|
||||
the address its requests reach pixa from. The login limit and
|
||||
`image_requests_per_minute` then count each client by its own address.
|
||||
- wait for pixa's answer for at least `downstream_timeout` (default `60s`). That
|
||||
is the longest pixa takes to fetch, convert and send an image.
|
||||
|
||||
[`configs/Caddyfile`](configs/Caddyfile) is an example for Caddy. Caddy does
|
||||
everything in this list without further settings, and gets its own TLS
|
||||
certificate.
|
||||
|
||||
The reverse proxy may also refuse `/metrics`, as the example does. Only a
|
||||
scraper that reaches pixa directly can then read it. pixa itself asks for
|
||||
`metrics.username` and `metrics.password` there.
|
||||
@@ -117,7 +114,7 @@ database and the disk cache:
|
||||
the limit for a while. Leave room on the volume beyond it.
|
||||
- Set `cache_max_bytes` for a lasting deployment. Its default is worked out at
|
||||
each start from the free space on the volume (see Configuration). Anything
|
||||
else that fills or frees space on the volume moves that default.
|
||||
else that fills or frees space on the volume moves it.
|
||||
|
||||
A load balancer's health check can request `/.well-known/healthcheck.json`. It
|
||||
answers 200 whenever pixa is running, in maintenance mode too (see
|
||||
@@ -125,28 +122,28 @@ answers 200 whenever pixa is running, in maintenance mode too (see
|
||||
|
||||
On `SIGTERM` or `SIGINT`, pixa stops accepting connections. The requests in
|
||||
progress and the images being processed get 5 seconds to finish. A request not
|
||||
finished by then is cut off. pixa then saves the last-used times it has not yet
|
||||
written (see Storage), and exits. It exits with status 1 when:
|
||||
finished by then is cut off. pixa then writes the last-used times of cached
|
||||
images that still wait to be written (see Storage), and exits. It exits with
|
||||
status 1 when:
|
||||
|
||||
- images were still being processed after those 5 seconds;
|
||||
- some of those times could not be saved;
|
||||
- some of those times could not be written;
|
||||
- another part of pixa failed to stop.
|
||||
|
||||
Otherwise it exits with 0. `docker stop` waits 10 seconds before it kills the
|
||||
container.
|
||||
|
||||
Outside Docker, pixa needs these at run time:
|
||||
|
||||
- libvips, the image library pixa converts images with, including its JPEG XL
|
||||
support. pixa does not start without that support. On Alpine it is the
|
||||
`vips-jxl` package. The nix and brew packages of libvips include it. So do the
|
||||
apt packages from Debian 12 and Ubuntu 24.04 on.
|
||||
- libheif, which libvips needs for AVIF.
|
||||
- the system's CA certificates, which HTTPS fetches need.
|
||||
|
||||
Compiling pixa needs the development files of libvips and libheif, `pkg-config`
|
||||
and a C compiler. `script/bootstrap --cgo` installs them, and libvips and
|
||||
libheif with them. The `Dockerfile` installs its own packages.
|
||||
Outside Docker, pixa needs libvips and libheif at run time. libvips is the image
|
||||
library pixa converts images with. pixa does not start unless libvips has its
|
||||
JPEG XL support. On Alpine that is the `vips-jxl` package. The nix and brew
|
||||
packages of libvips include it. So do the apt packages from Debian 12 and Ubuntu
|
||||
24.04 on. Compiling pixa also needs the development files of libvips and
|
||||
libheif, `pkg-config` and a C compiler. `script/bootstrap --cgo` installs all of
|
||||
these. The `Dockerfile` installs its own packages. Plain `script/bootstrap`
|
||||
installs git, make and Go; `script/setup` and `script/cibuild` run it. It also
|
||||
installs Node, Yarn and prettier, which format the markdown. It installs none of
|
||||
the C libraries: the checks compile pixa in Docker. Docker itself must already
|
||||
be installed.
|
||||
|
||||
## Running under upaas
|
||||
|
||||
@@ -164,20 +161,19 @@ run under. Its app for pixa needs:
|
||||
comma-separated; default none. Every image on a listed host is served to
|
||||
anyone, so list only hosts you control, where nobody else can put images.
|
||||
- `PIXA_CACHE_MAX_BYTES`: the disk cache limit in bytes; `0` turns the disk
|
||||
cache off. Set it. The volume is a directory on the host's filesystem. The
|
||||
host and every other app use that filesystem too. The default is 75% of
|
||||
the sum of that filesystem's free space and what the cache already holds.
|
||||
So by default the cache may grow to three quarters of the space the host
|
||||
and the other apps have left. The default is also at least 500 MiB. So
|
||||
when that space is 500 MiB or less, the cache may take all of it.
|
||||
cache off. Set it. The volume is a directory on the host's filesystem,
|
||||
shared with the host and every other app. The default is 75% of the sum of
|
||||
that filesystem's free space and what the cache holds, and at least 500
|
||||
MiB. So the cache can grow to three quarters or more of the space they all
|
||||
have left, and to all of it when that is 500 MiB or less.
|
||||
- the rest are in the list under Configuration.
|
||||
- **Memory:** pixa can use up to `max_concurrent_processing` times 4.2 GiB for
|
||||
the images it processes at once. On top of that, the fetched source images
|
||||
waiting for processing can take `upstream_connections` times
|
||||
`upstream_max_response_size`. 4.2 GiB is a measured value. It is the largest
|
||||
rise in resident memory measured while pixa saved one 8192x8192 source, the
|
||||
largest it accepts, to each output format at full size. The rise was largest
|
||||
for a 16-bit JPEG XL of random pixels saved as WebP. At the defaults the total
|
||||
`upstream_max_response_size`. 4.2 GiB is a measured value. It is the most
|
||||
resident memory grew while pixa processed one 8192x8192 source, the largest it
|
||||
accepts, saving it at full size to each output format in turn. Memory grew
|
||||
most for a 16-bit JPEG XL of random pixels saved as WebP. At the defaults that
|
||||
is half the CPUs pixa sees times 4.2 GiB, plus 64 times 50 MiB. On a host
|
||||
where pixa sees 48 CPUs, that is 24 times 4.2 GiB plus 3.1 GiB, about 104 GiB.
|
||||
Set `PIXA_MAX_CONCURRENT_PROCESSING` and `PIXA_UPSTREAM_CONNECTIONS` so that
|
||||
@@ -201,18 +197,17 @@ run under. Its app for pixa needs:
|
||||
- **Health check:** the image's `HEALTHCHECK` requests
|
||||
`/.well-known/healthcheck.json`. upaas reads the container's health 60 seconds
|
||||
after a deploy and marks the deploy failed unless it is `healthy`. The probe
|
||||
takes the port from `PORT` (default `8080`). It does not see a port changed
|
||||
only in a mounted config file, so change the port with `PORT`.
|
||||
takes the port from `PORT` (default `8080`). A port changed only in a mounted
|
||||
config file is not seen by it, so change the port with `PORT`.
|
||||
|
||||
## Rationale
|
||||
|
||||
A web application that shows images needs each one at several sizes and in
|
||||
several formats. pixa makes them on request from one original, and keeps what it
|
||||
made. The application stores one image per picture and hands out URLs. pixa is
|
||||
one binary. At run time it needs only the image libraries and the CA
|
||||
certificates named under Deployment. Signed URLs with an expiry keep strangers
|
||||
from using it to resize anything they like. Allowlisted hosts give open access
|
||||
where that is wanted.
|
||||
one binary. At run time it needs only the image libraries named under
|
||||
Deployment. Signed URLs with an expiry keep strangers from using it to resize
|
||||
anything they like. Allowlisted hosts give open access where that is wanted.
|
||||
|
||||
## Design
|
||||
|
||||
@@ -254,15 +249,18 @@ These terms are the names the code uses:
|
||||
|
||||
### Routes
|
||||
|
||||
pixa answers these routes. Any other path answers 404. A listed path answers 405
|
||||
to any method the list does not give for it.
|
||||
pixa answers these routes. Any other path answers 404. A path in this list asked
|
||||
with a method the list does not give answers 405. A browser's CORS preflight
|
||||
request to any path under `/v1/` answers 200, in maintenance mode too. CORS is
|
||||
the browser rule that lets a page on one site read responses from another. The
|
||||
preflight is the `OPTIONS` request the browser sends first.
|
||||
|
||||
- `GET /` — the login page, or the generator page with a login session (see
|
||||
Encrypted URLs). Needs nothing. Answers 200.
|
||||
- `POST /` — log in with the signing key typed into the login page. Needs the
|
||||
login page's form (below). For the right key it answers 303 to `/` and sets
|
||||
the login session cookie. For a wrong key it answers 200 with the login page
|
||||
and an error. Answers 429 over the login limit (see Limits).
|
||||
login page's form (below). Answers 303 to `/`, with a login session cookie,
|
||||
for the right key. Answers 200 with the login page and an error for a wrong
|
||||
key. Answers 429 over the login limit (see Limits).
|
||||
- `POST /generate` — make an encrypted URL from the generator page's form. Needs
|
||||
a login session and the generator page's form. Without a login session it
|
||||
answers 303 to `/`. Answers 200 with the page showing the URL. Answers 400
|
||||
@@ -344,36 +342,28 @@ to any method the list does not give for it.
|
||||
Answers 401 without them. Answers 404 when they are not set, as the route then
|
||||
does not exist.
|
||||
|
||||
A browser's CORS preflight request to any path under `/v1/` answers 200, in
|
||||
maintenance mode too. CORS is the browser rule that lets a page on one site read
|
||||
responses from another. The preflight is the `OPTIONS` request the browser sends
|
||||
first.
|
||||
|
||||
Every response carries an `X-Request-Id` header with the request's ID. A client
|
||||
can quote it when reporting a problem. A reverse proxy may send an
|
||||
`X-Request-Id` header with the request. pixa keeps that ID when it is 1 to 64
|
||||
letters, digits, `-`, `_` or `.`. Otherwise pixa makes a random ID, which tells
|
||||
nothing about the machine or the other requests. pixa's log line for the request
|
||||
carries the same ID as `request_id`. So do the lines it logs when it fetches,
|
||||
converts and serves an image. The fetch sends it to the upstream host as
|
||||
`X-Request-Id`.
|
||||
can quote it when reporting a problem. The ID is the request's own
|
||||
`X-Request-Id` header, which a reverse proxy may send, when that is 1 to 64
|
||||
letters, digits, `-`, `_` or `.`. Otherwise it is a random one pixa makes, which
|
||||
tells nothing about the machine or the other requests. pixa's log line for the
|
||||
request carries the same ID as `request_id`. So do the lines it logs when it
|
||||
fetches, converts and serves an image. The fetch sends it to the upstream host
|
||||
as `X-Request-Id`.
|
||||
|
||||
The `POST` routes accept only a form that pixa's own page served. The page puts
|
||||
a token in the form and sets a cookie to match. A request without both is
|
||||
refused with 403. So another site cannot submit the form from a visitor's
|
||||
browser. A form is also refused with 403 when the page's host is not the `Host`
|
||||
header pixa receives. So a reverse proxy in front of pixa must pass that header
|
||||
on unchanged. A form body over 1 MiB is refused with 413.
|
||||
|
||||
The login and generator pages are meant to be opened over HTTPS. While `debug`
|
||||
is off, a form sent from a page opened over plain HTTP is refused with 403.
|
||||
While `debug` is on, a form sent from a page opened over HTTPS is refused
|
||||
instead. Plain HTTP is for development on the browser's own machine. The login
|
||||
session cookie is always marked `Secure`. Over plain HTTP a browser keeps such a
|
||||
cookie only for its own machine (`localhost`), if at all.
|
||||
|
||||
The image routes answer the errors listed for them with a JSON body holding
|
||||
`error`, `status` and `timestamp`.
|
||||
browser. The login and generator pages are meant to be opened over HTTPS. While
|
||||
`debug` is off, a form sent from a page opened over plain HTTP is refused
|
||||
with 403. While `debug` is on, a form sent from a page opened over HTTPS is
|
||||
refused instead. Plain HTTP is for development on the browser's own machine. The
|
||||
login session cookie is always marked `Secure`. Over plain HTTP a browser keeps
|
||||
such a cookie only for its own machine (`localhost`), if at all. A form is also
|
||||
refused with 403 when the page's host is not the `Host` header pixa receives. So
|
||||
a reverse proxy in front of pixa must pass that header on unchanged. A form body
|
||||
over 1 MiB is refused with 413. The image routes answer the errors listed for
|
||||
them with a JSON body holding `error`, `status` and `timestamp`.
|
||||
|
||||
### Image URLs
|
||||
|
||||
@@ -472,11 +462,11 @@ it would come out above 8192, the request is refused with 400. The fit modes:
|
||||
at `100x0` is 100x1 by its proportions, but comes out at 8192x1. A side that
|
||||
would come out above 8192 is refused with 400.
|
||||
|
||||
An AVIF output of more than 16,777,216 pixels is refused with 400 before
|
||||
anything is decoded. That is the number of pixels in 4096x4096 or 8192x2048. A
|
||||
larger AVIF encode could take more than 40 seconds (see Limits). The limit
|
||||
includes `orig` of an AVIF source, and `auto` when it chooses AVIF. So a client
|
||||
whose `Accept` names `image/avif` but not `image/jxl` is refused such a size.
|
||||
An AVIF image of more than 16,777,216 pixels is refused with 400 before anything
|
||||
is decoded. That is the number of pixels in 4096x4096 or 8192x2048. A larger
|
||||
AVIF encode could take more than 40 seconds (see Limits). The limit includes
|
||||
`orig` of an AVIF source, and `auto` when it chooses AVIF. So a client whose
|
||||
`Accept` names `image/avif` but not `image/jxl` is refused such a size.
|
||||
|
||||
**Source formats.** The source may be JPEG, PNG, GIF, WebP, AVIF or JPEG XL. The
|
||||
upstream's `Content-Type` must name its format, and the image's first bytes must
|
||||
@@ -495,9 +485,8 @@ from its `Accept` header, in this order:
|
||||
1. JPEG XL, when the header names `image/jxl`;
|
||||
2. AVIF, when it names `image/avif`;
|
||||
3. WebP, when it names `image/webp`;
|
||||
4. JPEG, when the most specific of `image/jpeg`, `image/*` and `*/*` that the
|
||||
header names has a `q` above 0, or when there is no `Accept` header or it is
|
||||
empty.
|
||||
4. JPEG, when the first of `image/jpeg`, `image/*` and `*/*` that it names
|
||||
allows it, or when there is no `Accept` header or it is empty.
|
||||
|
||||
An entry with `q=0` refuses its format. Other `q` values do not change the
|
||||
order. JPEG XL, AVIF and WebP must be named, as clients that cannot show them
|
||||
@@ -600,8 +589,9 @@ change what it asks for.
|
||||
1. Open `/` in a browser over HTTPS and log in with the signing key. The login
|
||||
session lasts 30 days, or until someone logs out with the generator page's
|
||||
Logout button.
|
||||
2. On the generator page, fill in the form. Its fields are listed below. Then
|
||||
submit it (`POST /generate`).
|
||||
2. On the generator page, give the source image's URL, the width and height, the
|
||||
format, quality and fit, and how long the URL lasts. Then submit the form
|
||||
(`POST /generate`).
|
||||
3. The page shows the URL, `https://<host>/v1/e/<token>/img.<format>`, and when
|
||||
it expires. `<host>` is the host the page was opened on. While `debug` is on,
|
||||
the URL starts with `http` instead. The name after the token is ignored. It
|
||||
@@ -610,20 +600,12 @@ change what it asks for.
|
||||
|
||||
The form's fields:
|
||||
|
||||
- The source image's URL. Its scheme is not kept (below).
|
||||
- The width and height. Both empty or `0` keep the source's size. When only one
|
||||
of them is empty or `0`, that side is worked out from the source's proportions
|
||||
- The format is JPEG XL unless another is chosen. A form sent with an empty
|
||||
format, or none, also makes a JPEG XL URL.
|
||||
- Width and height both empty or `0` keep the source's size. When only one of
|
||||
them is empty or `0`, that side is worked out from the source's proportions
|
||||
when the image is made (see Image URLs).
|
||||
- The format: `orig`, `auto`, `jpeg`, `png`, `webp`, `avif`, `jxl` or `gif`. It
|
||||
is JPEG XL unless another is chosen. A form sent with an empty format, or
|
||||
none, also makes a JPEG XL URL.
|
||||
- The quality, a whole number from 1 to 100. The page offers five values, and
|
||||
`85` unless another is chosen.
|
||||
- The fit, one of the five fit modes (see Image URLs). It is `cover` unless
|
||||
another is chosen.
|
||||
- How long the URL lasts, in seconds, or `0` for never. The page offers never, 1
|
||||
minute, 1 hour, 1 day, 1 week, 30 days and 1 year, and never unless another is
|
||||
chosen.
|
||||
- How long the URL lasts is chosen from 1 minute to 1 year, or never.
|
||||
|
||||
The page refuses a form with 400 when:
|
||||
|
||||
@@ -631,25 +613,25 @@ The page refuses a form with 400 when:
|
||||
- the source URL's path holds a `..` segment, as `/v1/image/` refuses one;
|
||||
- a width, height, quality or expiry is not a number, or is out of range;
|
||||
- the fit is not one of the five fit modes;
|
||||
- pixa would fetch the source URL in another form (below).
|
||||
- the source URL would be fetched written another way (below).
|
||||
|
||||
The page refuses a source URL that pixa would fetch in another form, and shows
|
||||
the URL pixa would fetch. A URL with a user name and password is one example:
|
||||
pixa does not send them. A URL ending in `?` with no query is another. A path
|
||||
holding `%2F` is another: pixa would fetch it as `/`. So is a path holding `(`:
|
||||
pixa would fetch it as `%28` (see Image URLs).
|
||||
pixa fetches a source URL only as it was written. The page refuses one that pixa
|
||||
would fetch another way, and shows the URL pixa would fetch. A URL with a user
|
||||
name and password, which pixa does not send, is one example. A path holding
|
||||
`%2F`, which pixa would fetch as `/`, or `(`, which it would fetch as `%28`, is
|
||||
another (see Image URLs).
|
||||
|
||||
The image is made only when the URL is requested. A side that would come out
|
||||
above 8192, or an AVIF output of more than 16,777,216 pixels, is refused with
|
||||
400 then (see Image URLs).
|
||||
|
||||
Every login is made with the one signing key. So logging out ends every login
|
||||
session, in every browser, and a copied session cookie too. pixa keeps a random
|
||||
value in its database. It makes the value when the database has none, and
|
||||
Every login is made with the one signing key, so logging out ends every login
|
||||
session, in every browser, a copy of the session cookie included. pixa keeps a
|
||||
random value in its database. It makes the value when the database has none, and
|
||||
replaces it at each logout. A login session carries the value current at its
|
||||
login. Once the value has changed, the session is refused, after a restart too.
|
||||
The check compares no time, so a login after a logout works whatever the host's
|
||||
clock says. The 30 days are still measured by the host's clock.
|
||||
login. It is refused once the value has changed, also after a restart. The check
|
||||
compares no time, so a login after a logout works whatever the host's clock
|
||||
says. The 30 days are still measured by the host's clock.
|
||||
|
||||
The token holds the source's host, path and query, and the size, format,
|
||||
quality, fit and expiry. It is encrypted with a key derived from `signing_key`.
|
||||
@@ -716,8 +698,7 @@ The rest of the rules:
|
||||
- JPEG XL output is the one exception, with libvips 8.16 and later. libvips then
|
||||
writes an EXIF block of its own, which pixa cannot stop. The block holds the
|
||||
image's size and fixed values: an orientation of 1 and a resolution of 72 dpi.
|
||||
Nothing in it comes from the source. The Docker image has libvips 8.16, so its
|
||||
JPEG XL output carries this block.
|
||||
Nothing in it comes from the source.
|
||||
|
||||
### Limits
|
||||
|
||||
@@ -731,40 +712,34 @@ client goes away. The shared fetch sends the first request's ID upstream. The
|
||||
lines logged for the fetch and the conversion carry that ID.
|
||||
|
||||
**Timeouts.** `downstream_timeout` (default `60s`) bounds the whole answer to
|
||||
one request. The upstream fetch counts toward it. So do the waits for a
|
||||
connection slot and for a processing slot. The wait for a shared
|
||||
`upstream_connections` slot ends after 10 seconds, and so does the wait for a
|
||||
processing slot. The wait for an `upstream_connections_per_host` slot ends only
|
||||
with `downstream_timeout`. Keep `downstream_timeout` longer than
|
||||
`upstream_fetch_timeout` plus 20 seconds.
|
||||
one request. The upstream fetch counts toward it. So do the waits for an
|
||||
upstream connection and for a processing slot, up to 10 seconds each. Keep it
|
||||
longer than `upstream_fetch_timeout` plus 20 seconds.
|
||||
|
||||
A conversion whose request's `downstream_timeout` has ended stops waiting for a
|
||||
processing slot. Once it has one, it gives the slot back without beginning to
|
||||
encode. For a shared conversion, the first request's deadline counts. When a
|
||||
shared conversion stops this way, each request waiting for it is answered 503
|
||||
`server busy, try again later`, even if its own `downstream_timeout` has not
|
||||
encode. For a shared conversion, the first request's deadline counts. A request
|
||||
waiting for a shared conversion that stops this way is answered 503
|
||||
`server busy, try again later`, though its own `downstream_timeout` may not have
|
||||
ended. govips, the library pixa drives libvips through, gives no way to stop an
|
||||
encode once it has begun. So an encode keeps its slot until it finishes, which
|
||||
can be after `downstream_timeout`.
|
||||
|
||||
**Encode times.** The limits on image size bound how long a conversion can take.
|
||||
**Encode times.** The limits on image size bound how long an encode can take.
|
||||
They are 8192 on each side of the source and of the output, and 16,777,216
|
||||
pixels for an AVIF output. Decoding an 8192x8192 source takes up to about 11
|
||||
seconds. The WebP and JPEG XL encoders run below libvips' default effort. AVIF
|
||||
runs at the lowest effort govips can set. With one libvips thread, as pixa runs
|
||||
it, encoding 8192x8192 random pixels takes up to about:
|
||||
pixels for an AVIF output. The WebP and JPEG XL encoders run below libvips'
|
||||
default effort. AVIF runs at the lowest effort govips can set. With one libvips
|
||||
thread, as pixa runs it, 8192x8192 random pixels with an alpha channel take up
|
||||
to about:
|
||||
|
||||
- 1 second as JPEG;
|
||||
- 15 seconds as PNG;
|
||||
- 16 seconds as GIF;
|
||||
- 23 seconds as WebP with an alpha channel, and 16 without;
|
||||
- 27 seconds as JPEG XL with an alpha channel, and 13 without.
|
||||
- 23 seconds as WebP, and 16 without an alpha channel;
|
||||
- 27 seconds as JPEG XL, and 13 without an alpha channel.
|
||||
|
||||
An AVIF output of 4096x4096, 8192x2048 or 2048x8192 random pixels takes up to
|
||||
about 34 seconds with an alpha channel, and 22 without. At 8192x8192 and quality
|
||||
85 it would take about 96 seconds. These times hold at any quality and at 8 or
|
||||
16 bits per sample. No encode within the limits takes more than about 40 seconds
|
||||
of the default `downstream_timeout` of 60.
|
||||
An AVIF output of 4096x4096, 8192x2048 or 2048x8192 random pixels with an alpha
|
||||
channel takes up to about 34 seconds. At 8192x8192 and quality 85 it would take
|
||||
about 96. These times hold at any quality and at 8 or 16 bits per sample. No
|
||||
encode within the limits takes more than about 40 seconds of the default
|
||||
`downstream_timeout` of 60.
|
||||
|
||||
**Processing and connections.** `max_concurrent_processing` images are decoded
|
||||
and encoded at once. A request that finds every slot in use waits up to 10
|
||||
@@ -774,26 +749,24 @@ seconds for one. If none frees up, it is answered 503
|
||||
together, and `upstream_connections_per_host` those to one host. A fetch holds
|
||||
its connection until its image has been processed. A fetch that finds every
|
||||
shared connection in use waits up to 10 seconds the same way, then answers 503.
|
||||
A fetch that finds every connection to its host in use waits for one until
|
||||
`downstream_timeout` ends.
|
||||
|
||||
**Client limits.** The login form (`POST /`) is limited to 5 attempts per minute
|
||||
per client address. An attempt over the limit is refused with 429 and a
|
||||
`Retry-After` header. One client may send `image_requests_per_minute` requests
|
||||
per minute (default 600) to `/v1/image/` and `/v1/e/` together, `GET` and
|
||||
`HEAD`, cache hits included. A request over the limit is refused with 429 and a
|
||||
`Retry-After` header. A browser's CORS preflight is not counted. `0` turns the
|
||||
limit off. Both limits count an IPv6 client by its `/64`.
|
||||
per client address. An IPv6 client is counted by its `/64`. An attempt over the
|
||||
limit is refused with 429 and a `Retry-After` header. One client may send
|
||||
`image_requests_per_minute` requests per minute (default 600) to `/v1/image/`
|
||||
and `/v1/e/` together, `GET` and `HEAD`, cache hits included. A request over the
|
||||
limit is refused with 429 and a `Retry-After` header. A browser's CORS preflight
|
||||
is not counted. `0` turns the limit off.
|
||||
|
||||
**The client address.** Behind a reverse proxy, pixa takes the client address
|
||||
from `X-Forwarded-For` only when the request arrives from an address in
|
||||
`trusted_proxies`. Otherwise all users behind the proxy count as one client. The
|
||||
address the request arrives from is not always the proxy's own. A proxy on the
|
||||
Docker host that connects to pixa over `127.0.0.1` is seen as the gateway of the
|
||||
container's Docker network, such as `172.17.0.1` on the default bridge. A proxy
|
||||
that connects through another of the host's addresses is seen with that address.
|
||||
To be sure, read the address as `remoteIP` in pixa's request log while it is not
|
||||
in `trusted_proxies` (see `trusted_proxies` under Configuration). The default
|
||||
**The client address.** Behind a reverse proxy, the client address comes from
|
||||
`X-Forwarded-For` only when the address pixa sees the request come from is in
|
||||
`trusted_proxies`. Otherwise all users behind the proxy count as one client.
|
||||
That address is not always the proxy's own. A proxy on the Docker host that
|
||||
connects to pixa over `127.0.0.1` is seen as the gateway of the container's
|
||||
Docker network, such as `172.17.0.1` on the default bridge. A proxy that
|
||||
connects through another of the host's addresses is seen with that address. To
|
||||
be sure, read it as `remoteIP` in pixa's request log while it is not in
|
||||
`trusted_proxies` (see `trusted_proxies` under Configuration). The default
|
||||
`trusted_proxies` is the RFC 1918 ranges, the private address ranges. With that
|
||||
default, a client with a private address can choose the address it is counted
|
||||
by, through its own `X-Forwarded-For`, because its own address is trusted too.
|
||||
@@ -816,8 +789,7 @@ with 413.
|
||||
|
||||
`<ab>` and `<cd>` are the first and second pairs of characters of the file's
|
||||
name. The database is one SQLite file. SQLite is the database library pixa
|
||||
embeds; nothing else needs to run. In WAL mode, the default, SQLite keeps
|
||||
`state.sqlite3-wal` and `state.sqlite3-shm` beside it while pixa runs.
|
||||
embeds; nothing else needs to run.
|
||||
|
||||
The image cache keeps these tables in the database:
|
||||
|
||||
@@ -843,7 +815,7 @@ Each source and variant file is written under a temporary name and synced to
|
||||
disk before it is renamed into place. So a crash never leaves part of a file
|
||||
under its own name.
|
||||
|
||||
When pixa starts, and then at the spacing below, it checks the cached files
|
||||
When pixa starts, and every five minutes after, it checks the cached files
|
||||
against the database. It removes each source or variant whose file is empty, or
|
||||
not the size its row records. It logs a warning naming the file. A later request
|
||||
for that image fetches the source or makes the variant again. The check also
|
||||
@@ -881,26 +853,20 @@ also removes the expired rows of the negative cache every five minutes, at most
|
||||
|
||||
Every setting can be given as an environment variable, in a YAML config file
|
||||
(`--config`), or both. A variable present in the environment wins over the file,
|
||||
even when it is empty. The file wins over the built-in default. A variable's
|
||||
value is read as the same text in quotes in the file would be. A setting that
|
||||
holds a list takes comma-separated entries, with the spaces around each trimmed.
|
||||
An empty variable is an empty list.
|
||||
|
||||
The file's `env:` section is the one exception to that order. pixa sets each
|
||||
variable named there while it loads the file. So that variable wins over the
|
||||
environment pixa started with, and over the file's key for the same setting.
|
||||
|
||||
pixa refuses to start when a value does not parse or is invalid. The error names
|
||||
the setting. It also refuses to start when a key in the file is set to null,
|
||||
`key:` with nothing after it included. Omit the key to use the default. It
|
||||
refuses to start when the file has a key that is not in the list below, and the
|
||||
error names the key. It refuses to start when a variable starts with `PIXA_` but
|
||||
is not in the list below, such as a misspelled one or `PIXA_PORT`. The error
|
||||
names the variable. The variables set by the file's `env:` section are checked
|
||||
the same way.
|
||||
|
||||
pixa also accepts `PIXA_CONFIG_PATH`, which names the config file, like
|
||||
`--config`.
|
||||
even when it is empty. The file wins over the built-in default. The one
|
||||
exception is a variable set in the file's `env:` section. pixa sets that
|
||||
variable while loading the file, so it wins over the environment pixa started
|
||||
with. It also wins over the file's key for the same setting. A variable's value
|
||||
is parsed as the same text in the file would be. A setting that holds a list
|
||||
takes comma-separated entries, with the spaces around each trimmed. An empty
|
||||
variable is an empty list. A value that does not parse or is invalid aborts
|
||||
startup, naming the setting and its variable. A key in the file set to null,
|
||||
`key:` with nothing after it included, aborts startup too: omit the key to use
|
||||
the default. pixa refuses to start when a variable starts with `PIXA_` but is
|
||||
not in the list below, such as a misspelled one or `PIXA_PORT`. The error names
|
||||
the variable, as it names an unknown key in the file. pixa also accepts
|
||||
`PIXA_CONFIG_PATH`, which names the config file, like `--config`. The variables
|
||||
set by the file's `env:` section are checked the same way.
|
||||
|
||||
pixa reads at most one config file. It reads the file given with `--config` (or
|
||||
`-c`), or else the one `PIXA_CONFIG_PATH` names. pixa refuses to start when that
|
||||
@@ -970,27 +936,14 @@ Each item below is one setting, named by its config key:
|
||||
`Referer`, for example with `Referrer-Policy: no-referrer`, is not stopped.
|
||||
The login and generator pages are not covered.
|
||||
- `blocked_networks`: variable `PIXA_BLOCKED_NETWORKS`, default none. CIDR
|
||||
ranges pixa never fetches from, on top of the built-in ones. An invalid CIDR
|
||||
aborts startup. An IPv6 address in the IPv4-mapped form (`::ffff:a.b.c.d`) is
|
||||
matched only as the IPv4 address it holds. So an IPv6 range such as `::/0`
|
||||
refuses neither it nor a plain IPv4 address. An entry written in that form is
|
||||
read as the IPv4 range it covers: `::ffff:10.0.0.0/104` refuses what
|
||||
`10.0.0.0/8` refuses. Such an entry shorter than `/96` covers more than IPv4
|
||||
addresses and aborts startup. The built-in ranges always apply. They are:
|
||||
- `0.0.0.0/8`, this network, `0.0.0.0` included;
|
||||
- `10.0.0.0/8`, `172.16.0.0/12` and `192.168.0.0/16`, the private ranges;
|
||||
- `100.64.0.0/10`, carrier-grade NAT;
|
||||
- `127.0.0.0/8`, loopback;
|
||||
- `169.254.0.0/16`, link-local;
|
||||
- `192.0.0.0/24`, IETF protocol assignments;
|
||||
- `198.18.0.0/15`, benchmarking;
|
||||
- `224.0.0.0/4`, multicast;
|
||||
- `::`, the unspecified address;
|
||||
- `::1`, loopback;
|
||||
- `64:ff9b::/96` and `64:ff9b:1::/48`, NAT64;
|
||||
- `fc00::/7`, the unique local (private) range;
|
||||
- `fe80::/10`, link-local;
|
||||
- `ff00::/8`, multicast.
|
||||
ranges pixa never fetches from, on top of the built-in ones. The built-in
|
||||
ranges are loopback, private, link-local, carrier-grade NAT, benchmarking,
|
||||
NAT64 and the like, and always apply. An invalid CIDR aborts startup. An IPv6
|
||||
address in the IPv4-mapped form (`::ffff:a.b.c.d`) is matched only as the IPv4
|
||||
address it holds. So an IPv6 range such as `::/0` refuses neither it nor a
|
||||
plain IPv4 address. An entry written in that form is read as the IPv4 range it
|
||||
covers: `::ffff:10.0.0.0/104` refuses what `10.0.0.0/8` refuses. Such an entry
|
||||
shorter than `/96` covers more than IPv4 addresses and aborts startup.
|
||||
- `trusted_proxies`: variable `PIXA_TRUSTED_PROXIES`, default the RFC 1918
|
||||
private ranges: `10.0.0.0/8`, `172.16.0.0/12` and `192.168.0.0/16`. CIDR
|
||||
ranges of the reverse proxies in front of pixa. `X-Forwarded-For` is believed
|
||||
@@ -1011,18 +964,16 @@ Each item below is one setting, named by its config key:
|
||||
is for testing only.
|
||||
- `upstream_connections_per_host`: variable
|
||||
`PIXA_UPSTREAM_CONNECTIONS_PER_HOST`, default `20`. The most connections to
|
||||
one upstream host at once. It must be at least 1. A fetch that finds every
|
||||
connection to its host in use waits for one until `downstream_timeout` ends
|
||||
(see Limits).
|
||||
one upstream host at once. It must be at least 1.
|
||||
- `upstream_connections`: variable `PIXA_UPSTREAM_CONNECTIONS`, default `64`.
|
||||
The most connections to all upstream hosts together at once, on top of the
|
||||
limit per host. It must be at least 1. A fetch holds its connection until its
|
||||
image has been processed (see Limits).
|
||||
- `max_concurrent_processing`: variable `PIXA_MAX_CONCURRENT_PROCESSING`,
|
||||
default half the CPUs Go uses (`GOMAXPROCS`), rounded down, and at least 1.
|
||||
The most images decoded and encoded at once. `GOMAXPROCS` follows a
|
||||
container's CPU limit. A value that is not a whole number of at least 1 aborts
|
||||
startup. Each image can take a lot of memory (see Running under upaas).
|
||||
default half the CPUs pixa can use, rounded down, and at least 1. The most
|
||||
images decoded and encoded at once. The number of CPUs follows a container's
|
||||
CPU limit. A value that is not a whole number of at least 1 aborts startup.
|
||||
Each image can take a lot of memory (see Running under upaas).
|
||||
- `image_requests_per_minute`: variable `PIXA_IMAGE_REQUESTS_PER_MINUTE`,
|
||||
default `600`. How many requests one client may send per minute to
|
||||
`/v1/image/` and `/v1/e/` together (see Limits). `0` turns the limit off. A
|
||||
@@ -1045,12 +996,11 @@ Each item below is one setting, named by its config key:
|
||||
digits, hyphens and dots, with a letter in its last part) or an IP address
|
||||
(IPv6 in brackets, in its shortest form). It may have a port from 1 to 65535
|
||||
that has no leading zero and is not the scheme's default. Any other value
|
||||
aborts startup, including an origin with another scheme, such as a browser
|
||||
extension's.
|
||||
aborts startup, another scheme such as a browser extension's included.
|
||||
- `metrics.username`: variable `PIXA_METRICS_USERNAME`, default empty. The
|
||||
username for `/metrics`. pixa serves `/metrics`, and measures every HTTP
|
||||
request, only when the username and the password are both set. Setting one
|
||||
without the other aborts startup.
|
||||
username for `/metrics`. pixa serves `/metrics`, and measures requests for it,
|
||||
only when the username and the password are both set. Setting one without the
|
||||
other aborts startup.
|
||||
- `metrics.password`: variable `PIXA_METRICS_PASSWORD`, default empty. The
|
||||
password for `/metrics`. Set it together with the username.
|
||||
- `sentry_dsn`: variable `PIXA_SENTRY_DSN`, default empty. Sentry is an error
|
||||
@@ -1059,12 +1009,12 @@ Each item below is one setting, named by its config key:
|
||||
- `debug`: variable `PIXA_DEBUG`, default `false`. Turns on debug logging and
|
||||
plain-HTTP local development (see Routes).
|
||||
- `maintenance_mode`: variable `PIXA_MAINTENANCE_MODE`, default `false`. While
|
||||
`true`, `/v1/image/` and `/v1/e/` answer every `GET` and `HEAD` request with
|
||||
503, a `Retry-After` header of 300 seconds and a JSON error body. The health
|
||||
check still answers 200 and reports `"maintenance_mode": true`. It stays 200
|
||||
because the image's Docker `HEALTHCHECK` requests it, and upaas marks a deploy
|
||||
failed when its container is unhealthy. The login and generator pages and
|
||||
`/metrics` keep working.
|
||||
`true`, `/v1/image/` and `/v1/e/` answer every request with 503, a
|
||||
`Retry-After` header of 300 seconds and a JSON error body. The health check
|
||||
still answers 200 and reports `"maintenance_mode": true`. It stays 200 because
|
||||
the image's Docker `HEALTHCHECK` requests it, and upaas marks a deploy failed
|
||||
when its container is unhealthy. The login and generator pages and `/metrics`
|
||||
keep working.
|
||||
|
||||
`configs/config.example.yml` names every setting with its default.
|
||||
|
||||
@@ -1072,7 +1022,7 @@ Each item below is one setting, named by its config key:
|
||||
|
||||
- **Dependency injection**: Uber fx, a library that builds the program's parts
|
||||
in dependency order and starts and stops them. `internal/app` puts every part
|
||||
of `pixad` in one fx graph. `internal/providers` builds the image cache, the
|
||||
of pixad in one fx graph. `internal/providers` builds the image cache, the
|
||||
image proxy and the other parts that serve images and logins. Each setting's
|
||||
default is declared once, in `internal/config`.
|
||||
- **Packages**: these packages under `internal/` work with the concepts above:
|
||||
@@ -1092,7 +1042,7 @@ Each item below is one setting, named by its config key:
|
||||
- `session`: the login session
|
||||
- **HTTP router**: go-chi
|
||||
- **Image processing**: govips, a Go wrapper for libvips
|
||||
- **Database**: SQLite via `modernc.org/sqlite`
|
||||
- **Database**: SQLite via modernc.org/sqlite
|
||||
- **Static assets**: embedded in the binary via `//go:embed`
|
||||
- **Metrics**: Prometheus, at `/metrics`. Prometheus is a monitoring system that
|
||||
reads metrics from that route. pixa serves `/metrics` and measures HTTP
|
||||
@@ -1124,22 +1074,21 @@ Each item below is one setting, named by its config key:
|
||||
This repository adheres to the
|
||||
[Scripts to Rule Them All](https://github.com/github/scripts-to-rule-them-all)
|
||||
standard. The scripts in `script/` are the entrypoints for the development
|
||||
workflow, and most Makefile targets are thin shims that call them. We provide:
|
||||
workflow, and the Makefile targets are thin shims that call them. We provide:
|
||||
|
||||
- `script/bootstrap` — install git, make, Go, Node, Yarn and prettier, and
|
||||
download the Go modules. It can be run again at any time. With `--cgo`, it
|
||||
installs a C compiler, `pkg-config`, libvips (with its JPEG XL support) and
|
||||
libheif instead of Node, Yarn and prettier. Compiling pixa outside Docker, as
|
||||
`make build` and `script/bench` do, needs those. Docker itself must already be
|
||||
installed.
|
||||
installs the C compiler and the libvips (with its JPEG XL support) and libheif
|
||||
libraries instead of Node, Yarn and prettier. Compiling and testing pixa need
|
||||
those.
|
||||
- `script/setup` — make a fresh clone ready for development: bootstrap, then
|
||||
install-precommit.
|
||||
- `script/projectname` — output the project name (`pixa`).
|
||||
- `script/test` — run the test suite: build the `test` phase of the
|
||||
`Dockerfile`, writing no image.
|
||||
- `script/lint` — run `golangci-lint`: build the `lint` phase of the
|
||||
`Dockerfile`, writing no image. The linter never runs on the host.
|
||||
- `script/fmt` — format the Go code with `gofmt` and the markdown with prettier
|
||||
- `script/lint` — run golangci-lint: build the `lint` phase of the `Dockerfile`,
|
||||
writing no image. The linter never runs on the host.
|
||||
- `script/fmt` — format the Go code with gofmt and the markdown with prettier
|
||||
(writes).
|
||||
- `script/fmt-check` — check the same formatting (read-only), on the host.
|
||||
- `script/check` — run test, lint, and fmt-check.
|
||||
@@ -1148,8 +1097,8 @@ workflow, and most Makefile targets are thin shims that call them. We provide:
|
||||
and `test` phases, so this runs them too.
|
||||
- `script/docker-smoke` — build the image, start it, and wait for it to be
|
||||
healthy.
|
||||
- `script/loadtest` — measure `pixad`'s throughput, latency and peak memory. It
|
||||
is a benchmark, not part of `script/check` (see Load Test).
|
||||
- `script/loadtest` — measure pixad's throughput, latency and peak memory. It is
|
||||
a benchmark, not part of `script/check` (see Load Test).
|
||||
- `script/bench` — run the Go benchmark that serves cache hits in parallel
|
||||
through the image handler, with Go on the host after `script/bootstrap --cgo`.
|
||||
It is a benchmark, not part of `script/check` (see Load Test).
|
||||
@@ -1161,42 +1110,28 @@ workflow, and most Makefile targets are thin shims that call them. We provide:
|
||||
`script/precommit`.
|
||||
|
||||
Every `docker build` in these scripts passes a new value of the `CACHEBUST`
|
||||
build argument. Every stage of the `Dockerfile` declares it with an
|
||||
`ARG CACHEBUST` line. So every `RUN` step after that line, the lint and test
|
||||
steps included, runs on every build instead of coming from the build cache. The
|
||||
steps above that line install system packages and download the Go modules.
|
||||
Docker keeps their layers until the base image, the step itself, `go.mod` or
|
||||
`go.sum` changes.
|
||||
|
||||
Go's build cache is kept from build to build: the steps that compile mount it,
|
||||
so a package whose inputs are unchanged is not compiled again. The test step
|
||||
runs `go test`, and reruns it with `-v` after a failure. Both runs pass
|
||||
build argument, which every stage of the `Dockerfile` declares. So every `RUN`
|
||||
step after that line, the lint and test steps included, runs on every build
|
||||
instead of coming from the build cache. Above the line are the steps that
|
||||
install system packages and download the Go modules. Docker keeps their layers
|
||||
until the base image, the step itself, `go.mod` or `go.sum` changes. Go's build
|
||||
cache is kept from build to build: the steps that compile mount it, so a package
|
||||
whose inputs are unchanged is not compiled again. Both `go test` runs pass
|
||||
`-count=1`, so no stored test result is replayed. The lint step also mounts
|
||||
`golangci-lint`'s cache. That cache reuses a package's findings only while its
|
||||
files, its imports, `go.mod`, `.golangci.yml` and the `golangci-lint` version
|
||||
are unchanged.
|
||||
|
||||
The `test` phase runs on the Debian Go image, with Debian's libvips and libheif.
|
||||
The image pixa ships is built and runs on Alpine.
|
||||
|
||||
These scripts are the standard copies from `sneak/prompts`, kept identical to
|
||||
them:
|
||||
|
||||
- `script/check`
|
||||
- `script/cibuild`
|
||||
- `script/docker`
|
||||
- `script/lint`
|
||||
- `script/test`
|
||||
- `script/setup`
|
||||
- `script/install-precommit`
|
||||
|
||||
`script/fmt` and `script/fmt-check` are the standard copies with pixa's `gofmt`
|
||||
step kept before prettier. prettier formats the markdown only, not the HTML
|
||||
templates, as it cannot parse a Go template action inside a tag.
|
||||
golangci-lint's cache. That cache reuses a package's findings only while its
|
||||
files, its imports, `go.mod`, `.golangci.yml` and the golangci-lint version are
|
||||
unchanged. The `test` phase runs on the Debian Go image, with Debian's libvips
|
||||
and libheif. The image pixa ships is built and runs on Alpine. These scripts are
|
||||
the standard copies from `sneak/prompts`, kept identical to them:
|
||||
`script/check`, `script/cibuild`, `script/docker`, `script/lint`, `script/test`,
|
||||
`script/setup` and `script/install-precommit`. `script/fmt` and
|
||||
`script/fmt-check` are the standard copies with pixa's `gofmt` step kept before
|
||||
prettier. prettier formats the markdown only, not the HTML templates, as it
|
||||
cannot parse a Go template action inside a tag.
|
||||
|
||||
## Load Test
|
||||
|
||||
`script/loadtest` (or `make loadtest`) measures how fast `pixad` answers and how
|
||||
`script/loadtest` (or `make loadtest`) measures how fast pixad answers and how
|
||||
much memory it uses. It is a benchmark, not a check: `script/check` does not run
|
||||
it. It needs Docker and Go.
|
||||
|
||||
@@ -1205,24 +1140,23 @@ script/loadtest # 10 seconds per scenario, 4 clients
|
||||
script/loadtest 30s 32 # 30 seconds per scenario, 32 clients
|
||||
```
|
||||
|
||||
It builds the image with `script/docker`. It also builds the load tool,
|
||||
It builds the image with `script/docker` and the load tool,
|
||||
[vegeta](https://github.com/tsenart/vegeta), from a pinned commit. Each scenario
|
||||
starts a new `pixad` container and a new origin container,
|
||||
`cmd/loadtest-origin`. The origin is an upstream host that answers every path
|
||||
with the same generated 1600x1200 JPEG. vegeta then sends requests from the
|
||||
given number of clients. Each client sends its next request as soon as its last
|
||||
one is answered. Every request asks for an image resized to 400x300 WebP:
|
||||
starts a new pixad container and a new origin container, `cmd/loadtest-origin`.
|
||||
The origin is an upstream host that answers every path with the same generated
|
||||
1600x1200 JPEG. vegeta then sends requests from the given number of clients.
|
||||
Each client sends its next request as soon as its last one is answered. Every
|
||||
request asks for an image resized to 400x300 WebP:
|
||||
|
||||
- `hit`: the same image every time, put in the cache first;
|
||||
- `miss`: a new source image every time, so `pixad` fetches and converts each
|
||||
one;
|
||||
- `herd`: every client asks for each new source image at the same moment. The
|
||||
clients then share one fetch and one conversion (see Limits).
|
||||
- `miss`: a new source image every time, so pixad fetches and converts each one;
|
||||
- `herd`: each new source image once per client in a row. All clients then ask
|
||||
for it at the same time and share one fetch and one conversion (see Limits).
|
||||
|
||||
`pixad` refuses upstream hosts with private or local addresses, so the
|
||||
containers share a Docker network in `203.0.113.0/24`, a range set aside for
|
||||
documentation. A second run on the same Docker host while one is going fails, as
|
||||
it cannot create that network.
|
||||
pixad refuses upstream hosts with private or local addresses, so the containers
|
||||
share a Docker network in `203.0.113.0/24`, a range set aside for documentation.
|
||||
A second run on the same Docker host while one is going fails, as it cannot
|
||||
create that network.
|
||||
|
||||
For each scenario the script prints vegeta's report and two lines of its own:
|
||||
|
||||
@@ -1232,20 +1166,19 @@ For each scenario the script prints vegeta's report and two lines of its own:
|
||||
to the end of its answer. `50`, `95` and `99` are the 50th, 95th and 99th
|
||||
percentiles;
|
||||
- `Status Codes` and `Error Set`: anything other than `200` means the other
|
||||
numbers are not for the scenario described, such as `503` when `pixad` was
|
||||
busy;
|
||||
numbers are not for the scenario described, such as `503` when pixad was busy;
|
||||
- `Bytes In`: `0`, as vegeta is told not to keep the images it receives;
|
||||
- `pixad peak memory (VmHWM)`: the peak resident memory of the `pixad` process
|
||||
since its container started, in kB; for `hit` it includes the request that put
|
||||
the image in the cache;
|
||||
- `requests to the origin`: the fetches `pixad` made. That is one for `hit`, one
|
||||
- `pixad peak memory (VmHWM)`: the peak resident memory of pixad's process since
|
||||
its container started, in kB; for `hit` it includes the request that put the
|
||||
image in the cache;
|
||||
- `requests to the origin`: the fetches pixad made. That is one for `hit`, one
|
||||
per request for `miss`, and one per image for `herd`: the requests sent
|
||||
divided by the number of clients.
|
||||
|
||||
The numbers depend on the machine and on whatever else runs on it.
|
||||
|
||||
`script/bench` (or `make bench`) measures cache hits alone, without the network
|
||||
and with no `pixad` process. A Go benchmark in `internal/handlers` serves one
|
||||
and with no pixad process. A Go benchmark in `internal/handlers` serves one
|
||||
cached image through the `/v1/image/` handler from as many goroutines at once as
|
||||
the machine has CPUs. It reports the hits served per second as `hits/s`. Each
|
||||
run starts with an empty cache and an empty database, a SQLite file of its own,
|
||||
|
||||
Reference in New Issue
Block a user