check / check (push) Waiting to run
README.md gains a "Deployment" section: what the reverse proxy in front of pixa must do (terminate TLS, pass Host, Origin and Referer on unchanged, set X-Forwarded-For with trusted_proxies to match, wait at least downstream_timeout, optionally refuse /metrics) and what pixa does itself; that the state directory needs a persistent volume, what cache_max_bytes counts and why to set it; the health check for a load balancer; what SIGTERM does and the exit codes; and what running outside Docker needs. configs/Caddyfile is the example, as Caddy needs no settings beyond the host name and pixa's address. Model: opus-5-5
567 lines
32 KiB
Markdown
567 lines
32 KiB
Markdown
# pixa
|
|
|
|
pixa is a GPL-3.0-licensed Go web server by
|
|
[@sneak](https://sneak.berlin) that proxies images from upstream
|
|
sources, optionally resizing or transforming them, and serves the
|
|
results. Both source and transformed images are cached to disk so that
|
|
subsequent requests are served without origin fetches or additional
|
|
processing.
|
|
|
|
## Getting Started
|
|
|
|
```bash
|
|
# clone and build
|
|
git clone https://git.eeqj.de/sneak/pixa.git
|
|
cd pixa
|
|
make build
|
|
|
|
# run with a config file: copy the example and set a real signing key
|
|
# (the example placeholder is refused at startup), e.g. with
|
|
# openssl rand -base64 32
|
|
cp config.example.yml config.yml
|
|
$EDITOR config.yml # replace the signing_key placeholder
|
|
./bin/pixad --config config.yml
|
|
|
|
# or build and run via Docker
|
|
make docker
|
|
docker run -p 8080:8080 -e PIXA_SIGNING_KEY="$(openssl rand -base64 32)" pixa:latest
|
|
```
|
|
|
|
A container takes its settings from environment variables (see
|
|
Configuration below for the list). Only `PIXA_SIGNING_KEY` is required; if
|
|
it is unset the container exits at startup naming the variable. Everything
|
|
else has a built-in default. A config file mounted at `/etc/pixa/config.yml`
|
|
is optional: it is read when present, and an environment variable wins over
|
|
the same setting in it.
|
|
|
|
## Deployment
|
|
|
|
pixa listens on plain HTTP and runs behind a reverse proxy that terminates TLS.
|
|
[`configs/Caddyfile`](configs/Caddyfile) is an example for Caddy, chosen because
|
|
it is the smallest correct one: Caddy gets the TLS certificate itself and does
|
|
everything in this list without further settings. The reverse proxy must:
|
|
|
|
- terminate TLS, as the login and generator pages work only over HTTPS (see
|
|
Routes);
|
|
- pass the `Host`, `Origin` and `Referer` headers on unchanged, as pixa refuses
|
|
a form from those pages unless `Origin` or `Referer` names the host in `Host`,
|
|
and builds encrypted URLs from `Host`;
|
|
- set `X-Forwarded-For` to the client's address, with `trusted_proxies` set to
|
|
the address pixa sees the proxy's requests come from, so the login limit
|
|
counts each client by its own address (see `trusted_proxies` under
|
|
Configuration);
|
|
- wait for pixa's answer for at least `downstream_timeout` (default `60s`), the
|
|
longest pixa takes to fetch, convert and send an image.
|
|
|
|
It may also refuse `/metrics`, as the example does, so that only a scraper that
|
|
reaches pixa directly can read it; pixa itself asks for the metrics username and
|
|
password there.
|
|
|
|
pixa does the rest itself: it checks signatures and encrypted URLs, applies the
|
|
allowlist, refuses upstream hosts with private or local addresses, limits login
|
|
attempts, upstream response size and image dimensions, and sends the security
|
|
headers, `Strict-Transport-Security` included, with every response.
|
|
|
|
The state directory (`state_dir`, `/var/lib/pixa` in the container) holds the
|
|
database and the disk cache:
|
|
|
|
- It needs a persistent volume: without one, every restart starts with an empty
|
|
cache. In the container, the startup script gives the directory to the user
|
|
pixa runs as (uid 65532) and sets its mode to `750`; outside it, that user
|
|
must be able to write the directory.
|
|
- `cache_max_bytes` limits the source and transformed images together. The
|
|
database, the metadata files, the `.meta` file beside each transformed image
|
|
and files still being written come on top, and eviction runs in the
|
|
background, so the cache can pass the limit for a while: leave room on the
|
|
volume beyond it.
|
|
- Set `cache_max_bytes` for a lasting deployment. Its default is 75% of the
|
|
space free when pixa starts, which the cache's own files reduce, so a fuller
|
|
cache gives a smaller limit after a restart.
|
|
|
|
A load balancer's health check can request `/.well-known/healthcheck.json`,
|
|
which answers 200 whenever pixa is running, in maintenance mode too (see
|
|
`maintenance_mode`).
|
|
|
|
On SIGTERM or SIGINT pixa stops accepting connections, gives the requests in
|
|
progress and the images being processed 5 seconds to finish, and exits: with 0,
|
|
or with 1 when images were still being processed after those 5 seconds or
|
|
another part of pixa failed to stop. A request not finished by then is cut off.
|
|
`docker stop` waits 10 seconds before it kills the container.
|
|
|
|
Outside Docker, pixa needs libvips (the image has 8.15) and libheif to run, as
|
|
it uses libvips through CGO; building it also needs their development files,
|
|
`pkg-config` and a C compiler. `script/bootstrap` installs all of these with
|
|
nix, apt, brew or apk.
|
|
|
|
## Running under upaas
|
|
|
|
What the [upaas](https://git.eeqj.de/sneak/upaas) app for pixa needs:
|
|
|
|
- **Port:** pixa listens on container port `8080`.
|
|
- **Volume:** container path `/var/lib/pixa`, where pixa keeps its
|
|
database and cache. Creating the host directory when it is missing is
|
|
upaas's job, tracked in https://git.eeqj.de/sneak/upaas/issues/235.
|
|
- **Environment variables:**
|
|
- `PIXA_SIGNING_KEY` (required): secret for signed and encrypted URLs
|
|
and login, 32+ characters, for example from
|
|
`openssl rand -base64 32`
|
|
- `PIXA_ALLOWLIST_HOSTS`: upstream hosts served without a signature,
|
|
comma-separated
|
|
- `PIXA_CACHE_MAX_BYTES`: disk cache limit in bytes; `0` disables it;
|
|
default 75% of free space
|
|
- the rest are in the table under Configuration below
|
|
- **Health check:** the image's `HEALTHCHECK` requests
|
|
`/.well-known/healthcheck.json`. upaas reads the container's health 60
|
|
seconds after a deploy and marks the deploy failed unless it is
|
|
`healthy`. The probe uses the port from `PORT` (default `8080`), so a
|
|
port changed only in a mounted config file is not seen by it: change
|
|
the port with `PORT`.
|
|
|
|
## Rationale
|
|
|
|
Image-heavy web applications need a fast, caching reverse proxy that
|
|
can resize and transcode images on the fly. pixa fills that role as a
|
|
single, self-contained binary with no external runtime dependencies
|
|
beyond libvips. It supports HMAC-SHA256 signed URLs with expiration to
|
|
prevent abuse, and allowlisted source hosts for open access.
|
|
|
|
## Design
|
|
|
|
### Storage
|
|
|
|
- **Source content**:
|
|
`<state_dir>/cache/sources/<ab>/<cd>/<sha256 of source content>`
|
|
- **Source metadata**:
|
|
`<state_dir>/cache/metadata/<hostname>/<sha256 of path and query>.json`
|
|
(host, path and query, content hash, upstream status and headers, fetch time)
|
|
- **Database**: `<state_dir>/state.sqlite3` (SQLite)
|
|
- **Transformed images**:
|
|
`<state_dir>/cache/variants/<ab>/<cd>/<sha256 of host, path, query, size, format, quality and fit>`,
|
|
each with a `.meta` file beside it holding its content type
|
|
|
|
`<ab>` and `<cd>` are the first and second pairs of characters of the file's
|
|
name.
|
|
|
|
Multiple source paths may reference the same content blob; the
|
|
database tracks references rather than using filesystem refcounting.
|
|
Toward a target of 1-5k r/s, pixa keeps in memory the content types of
|
|
the 10,000 transformed images most recently cached or served, so a
|
|
cache hit on one of them reads only the image file from disk and not
|
|
the metadata file stored beside it.
|
|
|
|
### Routes
|
|
|
|
pixa answers these routes; any other path answers 404. A path in this list asked
|
|
with a method the list does not give answers 405, except `/static/<file>`, which
|
|
answers any method as it answers `GET`. A browser's CORS preflight request
|
|
(`OPTIONS` with `Origin` and `Access-Control-Request-Method` headers) to any
|
|
path under `/v1/` answers 200, in maintenance mode too.
|
|
|
|
- `GET /` — the login page, or the URL generator page with a login session
|
|
(see Encrypted URLs). Needs: nothing. Answers: 200.
|
|
- `POST /` — log in with the signing key typed into the login page. Needs: the
|
|
login page's form (below). Answers: 303 to `/` with a login session cookie
|
|
that lasts 30 days for the right key; 200 with the login page and an error for
|
|
a wrong key; 429 over the login limit (below).
|
|
- `POST /generate` — make an encrypted URL from the generator page's form.
|
|
Needs: a login session and the generator page's form (below); without a login
|
|
session it answers 303 to `/`. Answers: 200 with the page showing the URL; 400
|
|
with the page naming a field that is not valid; 500 when the URL cannot be
|
|
made.
|
|
- `GET /logout` — end the login session. Needs: nothing. Answers: 303 to `/`.
|
|
- `GET` or `HEAD` `/v1/image/<host>/<path>/<size>.<format>` — an image, fetched,
|
|
resized and converted (below). Needs: a signature, unless the host is
|
|
allowlisted (see Source Hosts). Answers: 200; 304 when `If-None-Match` matches
|
|
the image's `ETag`; 400 for a URL or parameter that is not valid; 401 for a
|
|
missing or wrong signature, a missing `exp` or an `exp` in the past; 403 when
|
|
the upstream host, or a host it redirects to, is `localhost`, ends in
|
|
`.localhost` or `.local`, or has an address in a blocked network (see
|
|
`blocked_networks`); 502 when the upstream answered with an error status, and
|
|
for 5 minutes after that for the same source URL; 503 when pixa is busy or in
|
|
maintenance mode; 500 for any other failure.
|
|
- `GET` or `HEAD` `/v1/e/<token>/<name>` — an image through an encrypted URL
|
|
(see Encrypted URLs). Needs: nothing but the URL. Answers: 200; 304 when
|
|
`If-None-Match` matches the image's `ETag`; 400 for a token that does not
|
|
decrypt, or that asks for a size or fit that is not valid; 410 once it has
|
|
expired; 504 when the upstream has not sent its response headers within
|
|
`upstream_fetch_timeout`, but 500 when that time runs out while the image
|
|
itself is still arriving; 403, 502, 503 and 500 as for `/v1/image/`.
|
|
- `GET /robots.txt` — asks every crawler to stay away (`Disallow: /`). Needs:
|
|
nothing. Answers: 200.
|
|
- `GET /.well-known/healthcheck.json` — JSON with `status` (`ok`), `now`,
|
|
`uptime_seconds`, `uptime_human`, `version`, `appname` and
|
|
`maintenance_mode`. Needs: nothing. Answers: 200, always.
|
|
- `GET /static/<file>` — the script the login and generator pages load. Needs:
|
|
nothing. Answers: 200, or 404 for a file that does not exist.
|
|
- `GET /metrics` — Prometheus metrics (see Architecture). Needs: HTTP basic
|
|
authentication with `metrics.username` and `metrics.password`. Answers: 200;
|
|
401 without them; 404 when they are not set, as the route then does not exist.
|
|
|
|
Every response carries an `X-Request-ID` header holding the request's ID, which
|
|
a client can quote when reporting a problem: the request's own `X-Request-ID`,
|
|
as a reverse proxy in front of pixa may send, when it is at most 64 letters,
|
|
digits, `-`, `_` or `.`; otherwise a random one pixa makes for the request,
|
|
which tells nothing about the machine or the other requests. pixa's log line for
|
|
the request carries the same ID as `request_id`, and so do the lines it logs
|
|
when it fetches, converts and serves an image; the fetch sends it to the
|
|
upstream host as `X-Request-ID`.
|
|
|
|
Both `POST` routes accept only a form that pixa's own page served: the page puts
|
|
a token in the form and sets a cookie to match, and a request without both is
|
|
refused with 403, so another site cannot submit the form from a visitor's
|
|
browser. The login and generator pages are meant to be opened over HTTPS: while
|
|
`debug` is off, a form sent from a page opened over plain HTTP is refused with
|
|
403, and while it is on, so is one sent from a page opened over HTTPS. Plain
|
|
HTTP is for development on the browser's own machine: the login session cookie
|
|
is always marked `Secure`, and over plain HTTP a browser keeps such a cookie
|
|
only for its own machine (`localhost`), if at all. A form is also refused with
|
|
403 when the page's host is not the `Host` header pixa receives, so a reverse
|
|
proxy in front of pixa must pass that header on unchanged. A form body over
|
|
1 MiB is refused with 413. The image routes answer the errors listed for them
|
|
with JSON holding `error`, `status` and `timestamp`.
|
|
|
|
An image URL has this form:
|
|
|
|
```
|
|
/v1/image/<host>/<path>/<size>.<format>?sig=<signature>&exp=<expiration>&q=<quality>&fit=<fit>
|
|
```
|
|
|
|
Images are only fetched from origins using TLS with valid certificates, unless
|
|
`allow_http` is set: then pixa fetches every image over plain HTTP, which is for
|
|
testing only.
|
|
|
|
A request whose query string cannot be decoded, or gives any parameter more
|
|
than once, is refused with 400.
|
|
|
|
- `<format>`: one of `orig` (or `original`), `jpeg` (or `jpg`), `png`, `webp`,
|
|
`avif`, `gif`
|
|
- `<size>`: `orig` or `<width>x<height>` (e.g. `800x600`)
|
|
- `sig` and `exp`: the signature and its expiry, needed unless the host is
|
|
allowlisted (see Signature Specification)
|
|
- `q` and `fit`: the output quality and how the image is fitted to `<size>`,
|
|
both optional (values under Signature Specification). Both are part of what
|
|
is cached, so each value of either is a separate cached image.
|
|
|
|
An image is served with `Cache-Control: public, max-age=<seconds>, immutable`.
|
|
When the URL has an expiry (an `exp`, or the TTL of an encrypted URL),
|
|
`max-age` is the whole seconds left until then, at most one year, so no browser
|
|
or proxy cache keeps the image after pixa would refuse the URL. A URL with no
|
|
expiry gets one year. `immutable` only stops a client revalidating while its
|
|
copy is fresh.
|
|
|
|
When several requests for the same image, size, format, quality and fit miss
|
|
the cache at once, they share one upstream fetch (or one read of the cached
|
|
source) and one transcode: the first request does the work, and the others wait
|
|
for its image or its error, holding no upstream connection or processing slot
|
|
of their own. A waiting request stops waiting when its own client goes away.
|
|
The work goes on for the others even if the first request's client goes away,
|
|
until that request's `downstream_timeout` ends. The shared fetch sends the first
|
|
request's ID upstream, and the lines logged for the fetch and the transcode
|
|
carry that ID.
|
|
|
|
The login form (`POST /`) is limited to 5 attempts per minute per client
|
|
address, counting an IPv6 client by its /64; an attempt over the limit is
|
|
refused with 429 and a `Retry-After` header. Behind a reverse proxy the client
|
|
address comes from `X-Forwarded-For` only when the address pixa sees for
|
|
requests that come through the proxy is in `trusted_proxies`; otherwise all
|
|
users behind the proxy are counted as one client. That address is not always
|
|
the proxy's own: a proxy on the Docker host that connects to pixa over
|
|
`127.0.0.1` is seen as the gateway of the container's Docker network, such as
|
|
`172.17.0.1` on the default bridge, and one that connects through another of the
|
|
host's addresses is seen with that address. To be sure, read it as `remoteIP` in
|
|
pixa's request log while it is not in `trusted_proxies` (see `trusted_proxies`
|
|
under Configuration). With the default `trusted_proxies` (the RFC 1918 ranges),
|
|
a client with a private address can choose the address it is counted by through
|
|
its own `X-Forwarded-For`, whether it connects directly or through the proxy,
|
|
because its own address is trusted too. Setting `trusted_proxies` to only the
|
|
address pixa sees for requests that come through the proxy closes this.
|
|
|
|
### Encrypted URLs
|
|
|
|
An encrypted URL is an image URL made on pixa's own web page by someone who
|
|
knows the signing key. It works for any upstream host, allowlisted or not,
|
|
without a signature, and whoever gets it can neither read the source URL from it
|
|
nor change what it asks for.
|
|
|
|
1. Open `/` in a browser over HTTPS (or over plain HTTP while `debug` is on, see
|
|
Routes) and log in with the signing key (`signing_key`). The login session
|
|
lasts 30 days, or until `/logout`.
|
|
2. On the generator page, give the source image's URL, the width and height, the
|
|
format, quality and fit, and how long the URL lasts, then submit the form
|
|
(`POST /generate`). Width and height both empty or `0` keep the original
|
|
size; if only one of them is empty or `0`, that side is scaled to keep the
|
|
image's proportions.
|
|
3. The page shows the URL, `https://<host>/v1/e/<token>/img.<format>`, and when
|
|
it expires. `<host>` is the host the page was opened on, and the URL starts
|
|
with `http` instead while `debug` is on. The name after the token is ignored
|
|
and only gives the URL a file extension, `jpg` for `orig`.
|
|
|
|
The token holds the source's host, path and query and the size, format,
|
|
quality, fit and expiry, encrypted with a key derived from `signing_key`. The
|
|
source URL's scheme is not kept: the image is fetched like any other (see
|
|
Routes), and the blocked networks still apply.
|
|
|
|
How long the URL lasts is chosen on the page, from 1 minute to 1 year, or
|
|
never. The expiry is fixed in the token when the URL is made and cannot be
|
|
changed or revoked afterwards. Until then the image is served with a `max-age`
|
|
that ends at the expiry (see Routes); after it the URL answers 410
|
|
`URL has expired`. A URL made to last forever stops working only when
|
|
`signing_key` changes: changing it makes every encrypted URL already handed out
|
|
answer 400, and ends every login session.
|
|
|
|
### Image Metadata
|
|
|
|
pixa decodes and re-encodes every image it serves, and removes all metadata from
|
|
the output: EXIF (GPS position, camera make, model and serial number, capture
|
|
time, embedded thumbnail), XMP, IPTC and the ICC colour profile. This cannot be
|
|
turned off.
|
|
|
|
- The `orig` format means the source's own format, not the source's bytes: an
|
|
`orig` image is re-encoded and stripped like any other.
|
|
- An image with an EXIF orientation is turned upright first, so it displays the
|
|
same without the tag; a requested size applies to the upright image.
|
|
- An image with an ICC profile is converted to sRGB first, since clients show an
|
|
image with no profile as sRGB. Colours outside sRGB, such as the most
|
|
saturated ones in a Display P3 photo, are clipped.
|
|
|
|
### Source Hosts
|
|
|
|
Source hosts may be allowlisted in the configuration. Non-allowlisted
|
|
hosts require an HMAC-SHA256 signature.
|
|
|
|
#### Signature Specification
|
|
|
|
Signatures use HMAC-SHA256 and include an expiration timestamp to
|
|
prevent replay attacks. Signatures are **exact match only**: every
|
|
component (host, path, query, dimensions, format, expiration, quality,
|
|
fit) must match exactly what was signed. No suffix matching, wildcard
|
|
matching, or partial matching is supported.
|
|
|
|
**Signed data format** (colon-separated):
|
|
|
|
```
|
|
HMAC-SHA256(secret, "host:path:query:width:height:format:expiration:quality:fit")
|
|
```
|
|
|
|
Where:
|
|
|
|
- `host` — source origin hostname (e.g. `cdn.example.com`)
|
|
- `path` — source path (e.g. `/photos/cat.jpg`)
|
|
- `query` — source query string, empty string if none
|
|
- `width` — requested width in pixels, `0` for original
|
|
- `height` — requested height in pixels, `0` for original
|
|
- `format` — output format, one of those listed under Routes, with `original`
|
|
signed as `orig` and `jpg` as `jpeg`
|
|
- `expiration` — the URL's `exp` query parameter, the Unix timestamp when
|
|
the signature expires; a request whose `exp` is not a whole number, an
|
|
empty `exp=` included, is refused with 400
|
|
- `quality` — the URL's `q` query parameter, a whole number from 1 to 100,
|
|
or `85` when the URL has no `q`; a request whose `q` is anything else is
|
|
refused with 400
|
|
- `fit` — the URL's `fit` query parameter (cover, contain, fill, inside,
|
|
outside), or `cover` when the URL has no `fit`; a request whose `fit` is
|
|
anything else, an empty `fit=` included, is refused with 400
|
|
|
|
The URL's `sig` is the HMAC-SHA256 result in base64url (the URL-safe alphabet
|
|
of RFC 4648) with the trailing `=` padding kept, 44 characters in all. pixa
|
|
compares it exactly, so a signature encoded without padding, as Node's
|
|
`base64url` and Go's `base64.RawURLEncoding` do, is refused with 401.
|
|
|
|
**Example:** with the signing key `example-signing-key-for-documentation`,
|
|
resize `https://cdn.example.com/photos/cat.jpg` to 800x600 WebP with
|
|
expiration 1704067200, default quality and fit:
|
|
|
|
1. Build input:
|
|
`cdn.example.com:/photos/cat.jpg::800:600:webp:1704067200:85:cover`
|
|
2. Compute HMAC-SHA256 of it with the signing key
|
|
3. Base64URL-encode the result, keeping the `=` padding:
|
|
`-ay7KHpfqmtIGbibDGbUuBDkymi-Ymdn0NkC6j5EJag=`
|
|
4. URL:
|
|
`/v1/image/cdn.example.com/photos/cat.jpg/800x600.webp?sig=-ay7KHpfqmtIGbibDGbUuBDkymi-Ymdn0NkC6j5EJag=&exp=1704067200`
|
|
|
|
For the same image at quality 40 with fit `contain`, the input ends in
|
|
`:40:contain`, the signature is `5IwXUx6vf7yefhaUvFzgXZvG2o0Df4RJxPTK3pKq5VU=`,
|
|
and the URL is
|
|
`/v1/image/cdn.example.com/photos/cat.jpg/800x600.webp?sig=5IwXUx6vf7yefhaUvFzgXZvG2o0Df4RJxPTK3pKq5VU=&exp=1704067200&q=40&fit=contain`.
|
|
|
|
**Allowlist patterns:**
|
|
|
|
- **Exact match**: `cdn.example.com` — matches only that host
|
|
- **Suffix match**: `.example.com` — matches `cdn.example.com`,
|
|
`images.example.com`, and `example.com`
|
|
|
|
### Configuration
|
|
|
|
Every setting can be given as an environment variable, in a YAML config
|
|
file (`--config`), or both. A variable present in the environment wins over
|
|
the file, even when it is empty, and the file wins over the built-in
|
|
default. The one exception is a variable named in the file's `env:` section:
|
|
it is set while the file loads, so it overrides both the environment the
|
|
process was started with and the file's own key. A variable's value is
|
|
parsed as the same text in the file would be. The three lists take
|
|
comma-separated entries, with the spaces around each trimmed; an empty
|
|
variable is an empty list. A value that does not parse or is invalid aborts
|
|
startup, naming the variable. A variable whose name starts with `PIXA_` but
|
|
is not in the table below, such as a misspelled one or `PIXA_PORT`, aborts
|
|
startup naming it, as an unknown config key does. The one other accepted
|
|
name is `PIXA_CONFIG_PATH`, the config file's path (like `--config`). The
|
|
variables set by the file's `env:` section are checked the same way.
|
|
|
|
pixa reads at most one config file: the one given with `--config` (or `-c`),
|
|
otherwise the one `PIXA_CONFIG_PATH` names, otherwise the first of these that
|
|
pixa finds: `/etc/pixa/config.yml`, `/etc/pixa/config.yaml`,
|
|
`~/.config/pixa/config.yml`, `~/.config/pixa/config.yaml`, then `config.yml`
|
|
and `config.yaml` in the working directory. A named file that does not exist,
|
|
cannot be read or does not parse aborts startup. Of the files pixa looks for on
|
|
its own, one it finds but cannot read or parse aborts startup; one it cannot
|
|
find, for any reason, is passed over without a message, even when the file is
|
|
there in a directory pixa may not enter. With no file, pixa uses the environment
|
|
and the defaults.
|
|
|
|
| Variable | Config key | Meaning |
|
|
| ------------------------------------ | ------------------------------- | ---------------------------------------------------------------------------- |
|
|
| `PIXA_SIGNING_KEY` | `signing_key` | Required: secret for signed and encrypted URLs and login, 32+ characters |
|
|
| `PORT` | `port` | Port to listen on; default `8080` |
|
|
| `PIXA_STATE_DIR` | `state_dir` | Directory for the database and the disk cache; default `/var/lib/pixa` |
|
|
| `PIXA_DB_URL` | `db_url` | SQLite database URL; default `state.sqlite3` in the state directory |
|
|
| `PIXA_CACHE_MAX_BYTES` | `cache_max_bytes` | Disk cache limit in bytes; `0` disables it; default 75% of free space |
|
|
| `PIXA_ALLOWLIST_HOSTS` | `allowlist_hosts` | Upstream hosts served without a signature |
|
|
| `PIXA_BLOCKED_NETWORKS` | `blocked_networks` | CIDR ranges never fetched from, on top of the built-in ones |
|
|
| `PIXA_TRUSTED_PROXIES` | `trusted_proxies` | CIDR ranges of proxies whose `X-Forwarded-For` is believed; default RFC 1918 |
|
|
| `PIXA_ALLOW_HTTP` | `allow_http` | Allow plain-HTTP upstreams, for testing only; default `false` |
|
|
| `PIXA_UPSTREAM_CONNECTIONS_PER_HOST` | `upstream_connections_per_host` | Concurrent connections per upstream host; default `20` |
|
|
| `PIXA_UPSTREAM_CONNECTIONS` | `upstream_connections` | Concurrent connections to all upstream hosts together; default `64` |
|
|
| `PIXA_MAX_CONCURRENT_PROCESSING` | `max_concurrent_processing` | Images processed at once; default the number of CPUs |
|
|
| `PIXA_UPSTREAM_FETCH_TIMEOUT` | `upstream_fetch_timeout` | Time allowed for one fetch from an upstream host; default `30s` |
|
|
| `PIXA_UPSTREAM_MAX_RESPONSE_SIZE` | `upstream_max_response_size` | Largest upstream response accepted, in bytes; default 50 MiB |
|
|
| `PIXA_DOWNSTREAM_TIMEOUT` | `downstream_timeout` | Time allowed for answering one client request; default `60s` |
|
|
| `PIXA_ACCESS_CONTROL_ALLOW_ORIGIN` | `access_control_allow_origin` | CORS origin allowed to read image responses: `*` or one origin; default `*` |
|
|
| `PIXA_METRICS_USERNAME` | `metrics.username` | Username for `/metrics`, which is served only when both are set |
|
|
| `PIXA_METRICS_PASSWORD` | `metrics.password` | Password for `/metrics`; set together with the username |
|
|
| `PIXA_SENTRY_DSN` | `sentry_dsn` | Sentry DSN for error reporting; empty disables it |
|
|
| `PIXA_DEBUG` | `debug` | Debug logging and plain-HTTP local development; default `false` |
|
|
| `PIXA_MAINTENANCE_MODE` | `maintenance_mode` | Answer image requests with 503; the health check stays 200; default `false` |
|
|
|
|
Key settings in more detail:
|
|
|
|
- `access_control_allow_origin` — the origin a browser lets read the responses
|
|
of the image routes, `/v1/image/` and `/v1/e/`, sent as the CORS
|
|
`Access-Control-Allow-Origin` header; no other route sends it. `*`, the
|
|
default, is any site; otherwise one `http` or `https` origin such as
|
|
`https://example.com`, whose host is a lowercase host name (letters,
|
|
digits, hyphens and dots, with a letter in its last part) or an IP address
|
|
(IPv6 in brackets, in its shortest form), with an optional port 1-65535
|
|
that has no leading zero and is not the scheme's default. Any other value,
|
|
including another scheme such as a browser extension's, aborts startup
|
|
- `allowlist_hosts` — list of allowed upstream hosts
|
|
- `blocked_networks` — list of CIDR ranges to refuse for SSRF protection,
|
|
added to the always-enforced built-in ranges (loopback, private,
|
|
link-local, CGNAT, benchmark, NAT64, and the like); an invalid CIDR
|
|
aborts startup
|
|
- `trusted_proxies` — list of CIDR ranges of the reverse proxies in front
|
|
of pixa. `X-Forwarded-For` is believed only when the direct peer falls
|
|
inside one of these ranges; the logged and login-recorded client
|
|
address is then the rightmost forwarded entry that is not itself a
|
|
trusted proxy. Otherwise the direct peer address is used and the header
|
|
is ignored, so a client connecting directly from an address outside
|
|
these ranges cannot spoof its address.
|
|
An omitted key defaults to the RFC 1918 private ranges (`10.0.0.0/8`,
|
|
`172.16.0.0/12`, `192.168.0.0/16`), since pixa is deployed behind a
|
|
proxy on a private network; an explicitly empty list (`[]`) trusts no
|
|
one, and an explicit list replaces the default. An invalid CIDR aborts
|
|
startup. Set this to the address pixa sees for requests that come through
|
|
your proxy, such as `172.17.0.1/32`, when the defaults do not cover it, or
|
|
to trust nothing else (see the login limit under Routes). For a proxy on
|
|
the Docker host that connects to pixa over `127.0.0.1`, that address is the
|
|
gateway of the container's Docker network (`172.17.0.1` on the default
|
|
bridge), not the proxy's own address; a proxy that connects through another of
|
|
the host's addresses is seen with that address. To be sure which address it
|
|
is, set this to `[]` (or `PIXA_TRUSTED_PROXIES` to empty), send a request
|
|
through the proxy, and read `remoteIP` in pixa's request log line for it
|
|
- `upstream_fetch_timeout` — time allowed for one fetch from an upstream
|
|
host, as a duration such as `30s` (the default) or `2m`
|
|
- `upstream_max_response_size` — largest upstream response accepted, in
|
|
bytes; default `52428800` (50 MiB). It also limits the image data pixa
|
|
decodes
|
|
- `downstream_timeout` — time allowed for answering one client request, as a
|
|
duration; default `60s`. The upstream fetch counts toward it, and so do the
|
|
waits for an upstream connection and for a processing slot (up to 10 seconds
|
|
each), so keep it longer than `upstream_fetch_timeout` plus 20 seconds
|
|
- `signing_key` — HMAC secret for URL signatures
|
|
- `cache_max_bytes` — disk cache size limit in bytes; `0` disables the
|
|
disk cache entirely; omitted defaults to 75% of the free space on
|
|
the filesystem containing `<state_dir>/cache/` (minimum 500 MiB)
|
|
- `upstream_connections` — the most connections to upstream hosts at once, all
|
|
hosts together, on top of `upstream_connections_per_host`; default `64`. A
|
|
fetch holds its connection until its image has been processed. A fetch that
|
|
finds all of them in use waits up to 10 seconds for one to free up; if none
|
|
does, and `downstream_timeout` has not ended first, the request is answered
|
|
503 with the error `server busy, try again later`
|
|
- `max_concurrent_processing` — the most images decoded and encoded at once;
|
|
default the number of CPUs pixa can use (`GOMAXPROCS`), which follows a
|
|
container's CPU limit. A request that finds all of them in use waits up to 10
|
|
seconds for one to free up; if none does, and `downstream_timeout` has not
|
|
ended first, it is answered 503 the same way
|
|
- `maintenance_mode` — while `true`, the image routes (`/v1/image/` and
|
|
`/v1/e/`) answer every request for an image with 503, a `Retry-After` header
|
|
and a JSON error body. The health check (`/.well-known/healthcheck.json`)
|
|
still answers 200 and reports `"maintenance_mode": true`. It stays 200
|
|
because the image's Docker `HEALTHCHECK` requests it: a 503 there would make
|
|
the container unhealthy, and upaas marks a deploy failed when its container
|
|
is unhealthy. The login and URL generator pages and `/metrics` keep working
|
|
|
|
See `config.example.yml` for all options with defaults.
|
|
|
|
### Architecture
|
|
|
|
- **Dependency injection**: Uber fx
|
|
- **HTTP router**: go-chi
|
|
- **Image processing**: govips (CGO wrapper for libvips)
|
|
- **Database**: SQLite via modernc.org/sqlite
|
|
- **Static assets**: embedded via `//go:embed`
|
|
- **Metrics**: Prometheus, at `/metrics`: generic HTTP request metrics
|
|
(duration, response size, requests in flight) and the Go runtime and process
|
|
metrics; requests are measured and `/metrics` is served only when
|
|
`metrics.username` and `metrics.password` are set
|
|
- **Logging**: stdlib slog
|
|
|
|
## Entrypoints
|
|
|
|
This repository adheres to the
|
|
[Scripts to Rule Them All](https://github.com/github/scripts-to-rule-them-all)
|
|
standard: normalized scripts in `script/` are the entrypoints for the
|
|
development workflow, and the Makefile targets are thin shims that call
|
|
them. We provide:
|
|
|
|
- `script/bootstrap` — install all dependencies (idempotent)
|
|
- `script/setup` — make a fresh clone ready for development
|
|
(bootstrap, then install-precommit)
|
|
- `script/projectname` — output the project name ("pixa")
|
|
- `script/test` — run the test suite
|
|
- `script/lint` — run golangci-lint, always in a container (builds
|
|
`Dockerfile.lint` when run outside one)
|
|
- `script/fmt` — format all code (writes)
|
|
- `script/fmt-check` — check formatting (read-only)
|
|
- `script/check` — run test, lint, and fmt-check
|
|
- `script/docker` — build the Docker image tagged via `script/projectname`
|
|
- `script/docker-smoke` — build the image, start it, wait for it to be healthy
|
|
- `script/cibuild` — CI entrypoint: `docker build .` with a new
|
|
`CHECK_EPOCH` on every run, so the Dockerfile's checks run instead of
|
|
coming from the build cache, and a green run implies a green repo
|
|
- `script/precommit` — pre-commit checks (`go mod tidy` guard, then
|
|
`script/check`)
|
|
- `script/install-precommit` — install the git pre-commit hook that
|
|
runs `script/precommit`
|
|
|
|
## TODO
|
|
|
|
See [TODO.md](TODO.md) for the full prioritized task list.
|
|
|
|
## License
|
|
|
|
GPL-3.0. See [LICENSE](LICENSE).
|
|
|
|
## Author
|
|
|
|
[@sneak](https://sneak.berlin)
|