sneak/pixa
1
0
Fork 0
Code Issues 31 Pull Requests 4 Actions 3 Packages Projects Releases Wiki Activity
Files
807de97a9d450e808034a1cf461050832c0d555b
pixa/README.md
T
clawbot aaf1bbbd4e Add the four settings the README documented but pixa lacked (closes #61) 
access_control_allow_origin, upstream_fetch_timeout,
upstream_max_response_size and downstream_timeout were in the README
but unknown to pixa, so a config that followed it aborted startup. Each
is now a setting with its PIXA_ variable, defaulting to the value that
was fixed in the code: *, 30s, 50 MiB and 60s. Durations are positive Go
duration strings; the size is a positive whole number of bytes; the
origin is * or one scheme and host. An invalid value aborts startup
naming the key and value. downstream_timeout replaces HTTPWriteTimeout
for the server's write timeout and the per-request timeout.

Model: opus-5-5
2026-09-29 06:11:44 +00:00

342 lines
18 KiB
Markdown
Raw Blame History

# pixa
pixa is a GPL-3.0-licensed Go web server by
[@sneak](https://sneak.berlin) that proxies images from upstream
sources, optionally resizing or transforming them, and serves the
results. Both source and transformed images are cached to disk so that
subsequent requests are served without origin fetches or additional
processing.
## Getting Started
```bash
# clone and build
git clone https://git.eeqj.de/sneak/pixa.git
cd pixa
make build
# run with a config file: copy the example and set a real signing key
# (the example placeholder is refused at startup), e.g. with
# openssl rand -base64 32
cp config.example.yml config.yml
$EDITOR config.yml # replace the signing_key placeholder
./bin/pixad --config config.yml
# or build and run via Docker
make docker
docker run -p 8080:8080 -e PIXA_SIGNING_KEY="$(openssl rand -base64 32)" pixa:latest
```
A container takes its settings from environment variables (see
Configuration below for the list). Only `PIXA_SIGNING_KEY` is required; if
it is unset the container exits at startup naming the variable. Everything
else has a built-in default. A config file mounted at `/etc/pixa/config.yml`
is optional: it is read when present, and an environment variable wins over
the same setting in it.
## Running under upaas
What the [upaas](https://git.eeqj.de/sneak/upaas) app for pixa needs:
- **Port:** pixa listens on container port `8080`.
- **Volume:** container path `/var/lib/pixa`, where pixa keeps its
database and cache. upaas bind-mounts the host path it is given and
does not create it, so the host directory must exist before the first
deploy.
- **Environment variables:**
- `PIXA_SIGNING_KEY` (required): secret for signed and encrypted URLs
and login, 32+ characters, for example from
`openssl rand -base64 32`
- `PIXA_ALLOWLIST_HOSTS`: upstream hosts served without a signature,
comma-separated
- `PIXA_CACHE_MAX_BYTES`: disk cache limit in bytes; `0` disables it;
default 75% of free space
- the rest are in the table under Configuration below
- **Health check:** the image's `HEALTHCHECK` requests
`/.well-known/healthcheck.json`. upaas reads the container's health 60
seconds after a deploy and marks the deploy failed unless it is
`healthy`. The probe uses the port from `PORT` (default `8080`), so a
port changed only in a mounted config file is not seen by it: change
the port with `PORT`.
- **First run:** create the host directory, owned by root or by uid
`65532` and gid `65532`. The server runs as the container's `pixad`
user, which has that uid and gid, and the container gives the
directory to `pixad` when it starts.
## Rationale
Image-heavy web applications need a fast, caching reverse proxy that
can resize and transcode images on the fly. pixa fills that role as a
single, self-contained binary with no external runtime dependencies
beyond libvips. It supports HMAC-SHA256 signed URLs with expiration to
prevent abuse, and allowlisted source hosts for open access.
## Design
### Storage
- **Source content**:
`<statedir>/cache/src-content/<ab>/<cd>/<sha256 of source content>`
- **Source metadata**:
`<statedir>/cache/src-metadata/<hostname>/<sha256 of path>.json`
(fetch time, original headers, request, content hash)
- **Database**: `<statedir>/state.sqlite3` (SQLite)
- **Output documents**:
`<statedir>/cache/dst-content/<ab>/<cd>/<sha256 of output content>`
Multiple source paths may reference the same content blob; the
database tracks references rather than using filesystem refcounting.
In-process caching of request-to-output mappings targets 1-5k r/s.
### Routes
```
/v1/image/<host>/<path>/<size>.<format>?sig=<signature>&exp=<expiration>
```
Images are only fetched from origins using TLS with valid certificates.
A request whose query string cannot be decoded, or gives any parameter more
than once, is refused with 400.
- `<format>`: one of `orig`, `png`, `jpeg`, `webp`
- `<size>`: `orig` or `<width>x<height>` (e.g. `800x600`)
An image is served with `Cache-Control: public, max-age=<seconds>, immutable`.
When the URL has an expiry (an `exp`, or the TTL of an encrypted URL),
`max-age` is the whole seconds left until then, at most one year, so no browser
or proxy cache keeps the image after pixa would refuse the URL. A URL with no
expiry gets one year. `immutable` only stops a client revalidating while its
copy is fresh.
The login form (`POST /`) is limited to 5 attempts per minute per client
address, counting an IPv6 client by its /64; an attempt over the limit is
refused with 429 and a `Retry-After` header. Behind a reverse proxy the client
address comes from `X-Forwarded-For` only when the address pixa sees for
requests that come through the proxy is in `trusted_proxies`; otherwise all
users behind the proxy are counted as one client. That address is not always
the proxy's own: a proxy on the Docker host that connects to pixa over
`127.0.0.1` is seen as the gateway of the container's Docker network, such as
`172.17.0.1` on the default bridge, and one that connects through another of the
host's addresses is seen with that address. To be sure, read it as `remoteIP` in
pixa's request log while it is not in `trusted_proxies` (see `trusted_proxies`
under Configuration). With the default `trusted_proxies` (the RFC 1918 ranges),
a client with a private address can choose the address it is counted by through
its own `X-Forwarded-For`, whether it connects directly or through the proxy,
because its own address is trusted too. Setting `trusted_proxies` to only the
address pixa sees for requests that come through the proxy closes this.
### Image Metadata
pixa decodes and re-encodes every image it serves, and removes all metadata from
the output: EXIF (GPS position, camera make, model and serial number, capture
time, embedded thumbnail), XMP, IPTC and the ICC colour profile. This cannot be
turned off.
- The `orig` format means the source's own format, not the source's bytes: an
`orig` image is re-encoded and stripped like any other.
- An image with an EXIF orientation is turned upright first, so it displays the
same without the tag; a requested size applies to the upright image.
- An image with an ICC profile is converted to sRGB first, since clients show an
image with no profile as sRGB. Colours outside sRGB, such as the most
saturated ones in a Display P3 photo, are clipped.
### Source Hosts
Source hosts may be allowlisted in the configuration. Non-allowlisted
hosts require an HMAC-SHA256 signature.
#### Signature Specification
Signatures use HMAC-SHA256 and include an expiration timestamp to
prevent replay attacks. Signatures are **exact match only**: every
component (host, path, query, dimensions, format, expiration, quality,
fit) must match exactly what was signed. No suffix matching, wildcard
matching, or partial matching is supported.
**Signed data format** (colon-separated):
```
HMAC-SHA256(secret, "host:path:query:width:height:format:expiration:quality:fit")
```
Where:
- `host` — source origin hostname (e.g. `cdn.example.com`)
- `path` — source path (e.g. `/photos/cat.jpg`)
- `query` — source query string, empty string if none
- `width` — requested width in pixels, `0` for original
- `height` — requested height in pixels, `0` for original
- `format` — output format (jpeg, png, webp, avif, gif, orig)
- `expiration` — the URL's `exp` query parameter, the Unix timestamp when
the signature expires; a request whose `exp` is not a whole number, an
empty `exp=` included, is refused with 400
- `quality` — the URL's `q` query parameter, a whole number from 1 to 100,
or `85` when the URL has no `q`; a request whose `q` is anything else is
refused with 400
- `fit` — the URL's `fit` query parameter (cover, contain, fill, inside,
outside), or `cover` when the URL has no `fit`; a request whose `fit` is
anything else, an empty `fit=` included, is refused with 400
The URL's `sig` is the HMAC-SHA256 result in base64url (the URL-safe alphabet
of RFC 4648) with the trailing `=` padding kept, 44 characters in all. pixa
compares it exactly, so a signature encoded without padding, as Node's
`base64url` and Go's `base64.RawURLEncoding` do, is refused with 401.
**Example:** with the signing key `example-signing-key-for-documentation`,
resize `https://cdn.example.com/photos/cat.jpg` to 800x600 WebP with
expiration 1704067200, default quality and fit:
1. Build input:
`cdn.example.com:/photos/cat.jpg::800:600:webp:1704067200:85:cover`
2. Compute HMAC-SHA256 of it with the signing key
3. Base64URL-encode the result, keeping the `=` padding:
`-ay7KHpfqmtIGbibDGbUuBDkymi-Ymdn0NkC6j5EJag=`
4. URL:
`/v1/image/cdn.example.com/photos/cat.jpg/800x600.webp?sig=-ay7KHpfqmtIGbibDGbUuBDkymi-Ymdn0NkC6j5EJag=&exp=1704067200`
For the same image at quality 40 with fit `contain`, the input ends in
`:40:contain`, the signature is `5IwXUx6vf7yefhaUvFzgXZvG2o0Df4RJxPTK3pKq5VU=`,
and the URL is
`/v1/image/cdn.example.com/photos/cat.jpg/800x600.webp?sig=5IwXUx6vf7yefhaUvFzgXZvG2o0Df4RJxPTK3pKq5VU=&exp=1704067200&q=40&fit=contain`.
**Allowlist patterns:**
- **Exact match**: `cdn.example.com` — matches only that host
- **Suffix match**: `.example.com` — matches `cdn.example.com`,
`images.example.com`, and `example.com`
### Configuration
Every setting can be given as an environment variable, in a YAML config
file (`--config`), or both. A variable present in the environment wins over
the file, even when it is empty, and the file wins over the built-in
default. The one exception is a variable named in the file's `env:` section:
it is set while the file loads, so it overrides both the environment the
process was started with and the file's own key. A variable's value is
parsed as the same text in the file would be. The three lists take
comma-separated entries, with the spaces around each trimmed; an empty
variable is an empty list. A value that does not parse or is invalid aborts
startup, naming the variable. A variable whose name starts with `PIXA_` but
is not in the table below, such as a misspelled one or `PIXA_PORT`, aborts
startup naming it, as an unknown config key does. The one other accepted
name is `PIXA_CONFIG_PATH`, the config file's path (like `--config`). The
variables set by the file's `env:` section are checked the same way.
| Variable | Config key | Meaning |
| ------------------------------------ | ------------------------------- | ---------------------------------------------------------------------------- |
| `PIXA_SIGNING_KEY` | `signing_key` | Required: secret for signed and encrypted URLs and login, 32+ characters |
| `PORT` | `port` | Port to listen on; default `8080` |
| `PIXA_STATE_DIR` | `state_dir` | Directory for the database and the disk cache; default `/var/lib/pixa` |
| `PIXA_DB_URL` | `db_url` | SQLite database URL; default `state.sqlite3` in the state directory |
| `PIXA_CACHE_MAX_BYTES` | `cache_max_bytes` | Disk cache limit in bytes; `0` disables it; default 75% of free space |
| `PIXA_ALLOWLIST_HOSTS` | `allowlist_hosts` | Upstream hosts served without a signature |
| `PIXA_BLOCKED_NETWORKS` | `blocked_networks` | CIDR ranges never fetched from, on top of the built-in ones |
| `PIXA_TRUSTED_PROXIES` | `trusted_proxies` | CIDR ranges of proxies whose `X-Forwarded-For` is believed; default RFC 1918 |
| `PIXA_ALLOW_HTTP` | `allow_http` | Allow plain-HTTP upstreams, for testing only; default `false` |
| `PIXA_UPSTREAM_CONNECTIONS_PER_HOST` | `upstream_connections_per_host` | Concurrent connections per upstream host; default `20` |
| `PIXA_UPSTREAM_FETCH_TIMEOUT` | `upstream_fetch_timeout` | Time allowed for one fetch from an upstream host; default `30s` |
| `PIXA_UPSTREAM_MAX_RESPONSE_SIZE` | `upstream_max_response_size` | Largest upstream response accepted, in bytes; default 50 MiB |
| `PIXA_DOWNSTREAM_TIMEOUT` | `downstream_timeout` | Time allowed for answering one client request; default `60s` |
| `PIXA_ACCESS_CONTROL_ALLOW_ORIGIN` | `access_control_allow_origin` | CORS origin allowed to read responses: `*` or one origin; default `*` |
| `PIXA_METRICS_USERNAME` | `metrics.username` | Username for `/metrics`, which is served only when both are set |
| `PIXA_METRICS_PASSWORD` | `metrics.password` | Password for `/metrics`; set together with the username |
| `PIXA_SENTRY_DSN` | `sentry_dsn` | Sentry DSN for error reporting; empty disables it |
| `PIXA_DEBUG` | `debug` | Debug logging and plain-HTTP local development; default `false` |
| `PIXA_MAINTENANCE_MODE` | `maintenance_mode` | Maintenance flag reported by the health check; default `false` |
Key settings in more detail:
- `access_control_allow_origin` — the origin a browser lets read pixa's
responses, sent as the CORS `Access-Control-Allow-Origin` header: `*`, the
default, is any site; otherwise one origin, scheme and host only, such as
`https://example.com`. Anything else aborts startup
- `allowlist_hosts` — list of allowed upstream hosts
- `blocked_networks` — list of CIDR ranges to refuse for SSRF protection,
added to the always-enforced built-in ranges (loopback, private,
link-local, CGNAT, benchmark, NAT64, and the like); an invalid CIDR
aborts startup
- `trusted_proxies` — list of CIDR ranges of the reverse proxies in front
of pixa. `X-Forwarded-For` is believed only when the direct peer falls
inside one of these ranges; the logged and login-recorded client
address is then the rightmost forwarded entry that is not itself a
trusted proxy. Otherwise the direct peer address is used and the header
is ignored, so a client connecting directly from an address outside
these ranges cannot spoof its address.
An omitted key defaults to the RFC 1918 private ranges (`10.0.0.0/8`,
`172.16.0.0/12`, `192.168.0.0/16`), since pixa is deployed behind a
proxy on a private network; an explicitly empty list (`[]`) trusts no
one, and an explicit list replaces the default. An invalid CIDR aborts
startup. Set this to the address pixa sees for requests that come through
your proxy, such as `172.17.0.1/32`, when the defaults do not cover it, or
to trust nothing else (see the login limit under Routes). For a proxy on
the Docker host that connects to pixa over `127.0.0.1`, that address is the
gateway of the container's Docker network (`172.17.0.1` on the default
bridge), not the proxy's own address; a proxy that connects through another of
the host's addresses is seen with that address. To be sure which address it
is, set this to `[]` (or `PIXA_TRUSTED_PROXIES` to empty), send a request
through the proxy, and read `remoteIP` in pixa's request log line for it
- `upstream_fetch_timeout` — time allowed for one fetch from an upstream
host, as a duration such as `30s` (the default) or `2m`
- `upstream_max_response_size` — largest upstream response accepted, in
bytes; default `52428800` (50 MiB). It also limits the image data pixa
decodes
- `downstream_timeout` — time allowed for answering one client request, as a
duration; default `60s`. The upstream fetch counts toward it, so keep it
longer than `upstream_fetch_timeout`
- `signing_key` — HMAC secret for URL signatures
- `cache_max_bytes` — disk cache size limit in bytes; `0` disables the
disk cache entirely; omitted defaults to 75% of the free space on
the filesystem containing `<state_dir>/cache/` (minimum 500 MiB)
See `config.example.yml` for all options with defaults.
### Architecture
- **Dependency injection**: Uber fx
- **HTTP router**: go-chi
- **Image processing**: govips (CGO wrapper for libvips)
- **Database**: SQLite via modernc.org/sqlite
- **Static assets**: embedded via `//go:embed`
- **Metrics**: Prometheus
- **Logging**: stdlib slog
## Entrypoints
This repository adheres to the
[Scripts to Rule Them All](https://github.com/github/scripts-to-rule-them-all)
standard: normalized scripts in `script/` are the entrypoints for the
development workflow, and the Makefile targets are thin shims that call
them. We provide:
- `script/bootstrap` — install all dependencies (idempotent)
- `script/setup` — make a fresh clone ready for development
(bootstrap, then install-precommit)
- `script/projectname` — output the project name ("pixa")
- `script/test` — run the test suite
- `script/lint` — run golangci-lint, always in a container (builds
`Dockerfile.lint` when run outside one)
- `script/fmt` — format all code (writes)
- `script/fmt-check` — check formatting (read-only)
- `script/check` — run test, lint, and fmt-check
- `script/docker` — build the Docker image tagged via `script/projectname`
- `script/docker-smoke` — build the image, start it, wait for it to be healthy
- `script/cibuild` — CI entrypoint: `docker build .` (the Dockerfile
runs the checks, so a green build implies a green repo)
- `script/precommit` — pre-commit checks (`go mod tidy` guard, then
`script/check`)
- `script/install-precommit` — install the git pre-commit hook that
runs `script/precommit`
## TODO
See [TODO.md](TODO.md) for the full prioritized task list.
## License
GPL-3.0. See [LICENSE](LICENSE).
## Author
[@sneak](https://sneak.berlin)
Reference in New Issue View Git Blame Copy Permalink