sneak
9b49a9d5b3
check / check (push) Successful in 3m22s
A variable named in the config file's env: section is set while the file loads, so it overrides both the environment the process was started with and the file's own key. README.md and the config.example.yml header said only that a variable wins over the file; they now state this exception where the precedence is given. Model: opus-5-5
234 lines
11 KiB
Markdown
234 lines
11 KiB
Markdown
# pixa
|
|
|
|
pixa is a GPL-3.0-licensed Go web server by
|
|
[@sneak](https://sneak.berlin) that proxies images from upstream
|
|
sources, optionally resizing or transforming them, and serves the
|
|
results. Both source and transformed images are cached to disk so that
|
|
subsequent requests are served without origin fetches or additional
|
|
processing.
|
|
|
|
## Getting Started
|
|
|
|
```bash
|
|
# clone and build
|
|
git clone https://git.eeqj.de/sneak/pixa.git
|
|
cd pixa
|
|
make build
|
|
|
|
# run with a config file: copy the example and set a real signing key
|
|
# (the example placeholder is refused at startup), e.g. with
|
|
# openssl rand -base64 32
|
|
cp config.example.yml config.yml
|
|
$EDITOR config.yml # replace the signing_key placeholder
|
|
./bin/pixad --config config.yml
|
|
|
|
# or build and run via Docker
|
|
make docker
|
|
docker run -p 8080:8080 -e PIXA_SIGNING_KEY="$(openssl rand -base64 32)" pixa:latest
|
|
```
|
|
|
|
A container takes its settings from environment variables (see
|
|
Configuration below for the list). Only `PIXA_SIGNING_KEY` is required; if
|
|
it is unset the container exits at startup naming the variable. Everything
|
|
else has a built-in default. A config file mounted at `/etc/pixa/config.yml`
|
|
is optional: it is read when present, and an environment variable wins over
|
|
the same setting in it.
|
|
|
|
## Rationale
|
|
|
|
Image-heavy web applications need a fast, caching reverse proxy that
|
|
can resize and transcode images on the fly. pixa fills that role as a
|
|
single, self-contained binary with no external runtime dependencies
|
|
beyond libvips. It supports HMAC-SHA256 signed URLs with expiration to
|
|
prevent abuse, and allowlisted source hosts for open access.
|
|
|
|
## Design
|
|
|
|
### Storage
|
|
|
|
- **Source content**:
|
|
`<statedir>/cache/src-content/<ab>/<cd>/<sha256 of source content>`
|
|
- **Source metadata**:
|
|
`<statedir>/cache/src-metadata/<hostname>/<sha256 of path>.json`
|
|
(fetch time, original headers, request, content hash)
|
|
- **Database**: `<statedir>/state.sqlite3` (SQLite)
|
|
- **Output documents**:
|
|
`<statedir>/cache/dst-content/<ab>/<cd>/<sha256 of output content>`
|
|
|
|
Multiple source paths may reference the same content blob; the
|
|
database tracks references rather than using filesystem refcounting.
|
|
In-process caching of request-to-output mappings targets 1-5k r/s.
|
|
|
|
### Routes
|
|
|
|
```
|
|
/v1/image/<host>/<path>/<size>.<format>?sig=<signature>&exp=<expiration>
|
|
```
|
|
|
|
Images are only fetched from origins using TLS with valid certificates.
|
|
|
|
- `<format>`: one of `orig`, `png`, `jpeg`, `webp`
|
|
- `<size>`: `orig` or `<width>x<height>` (e.g. `800x600`)
|
|
|
|
### Source Hosts
|
|
|
|
Source hosts may be allowlisted in the configuration. Non-allowlisted
|
|
hosts require an HMAC-SHA256 signature.
|
|
|
|
#### Signature Specification
|
|
|
|
Signatures use HMAC-SHA256 and include an expiration timestamp to
|
|
prevent replay attacks. Signatures are **exact match only**: every
|
|
component (host, path, query, dimensions, format, expiration, quality,
|
|
fit) must match exactly what was signed. No suffix matching, wildcard
|
|
matching, or partial matching is supported.
|
|
|
|
**Signed data format** (colon-separated):
|
|
|
|
```
|
|
HMAC-SHA256(secret, "host:path:query:width:height:format:expiration:quality:fit")
|
|
```
|
|
|
|
Where:
|
|
|
|
- `host` — source origin hostname (e.g. `cdn.example.com`)
|
|
- `path` — source path (e.g. `/photos/cat.jpg`)
|
|
- `query` — source query string, empty string if none
|
|
- `width` — requested width in pixels, `0` for original
|
|
- `height` — requested height in pixels, `0` for original
|
|
- `format` — output format (jpeg, png, webp, avif, gif, orig)
|
|
- `expiration` — Unix timestamp when signature expires
|
|
- `quality` — the URL's `q` query parameter (1-100), or `85` when the URL
|
|
has no `q`
|
|
- `fit` — the URL's `fit` query parameter (cover, contain, fill, inside,
|
|
outside), or `cover` when the URL has no `fit`
|
|
|
|
**Example:** resize `https://cdn.example.com/photos/cat.jpg` to 800x600
|
|
WebP with expiration 1704067200, default quality and fit:
|
|
|
|
1. Build input:
|
|
`cdn.example.com:/photos/cat.jpg::800:600:webp:1704067200:85:cover`
|
|
2. Compute HMAC-SHA256 with your secret key
|
|
3. Base64URL-encode the result
|
|
4. URL:
|
|
`/v1/image/cdn.example.com/photos/cat.jpg/800x600.webp?sig=<base64url>&exp=1704067200`
|
|
|
|
For the same image at quality 40 with fit `contain`, the input ends in
|
|
`:40:contain` and the URL is
|
|
`/v1/image/cdn.example.com/photos/cat.jpg/800x600.webp?sig=<base64url>&exp=1704067200&q=40&fit=contain`.
|
|
|
|
**Allowlist patterns:**
|
|
|
|
- **Exact match**: `cdn.example.com` — matches only that host
|
|
- **Suffix match**: `.example.com` — matches `cdn.example.com`,
|
|
`images.example.com`, and `example.com`
|
|
|
|
### Configuration
|
|
|
|
Every setting can be given as an environment variable, in a YAML config
|
|
file (`--config`), or both. A variable present in the environment wins over
|
|
the file, even when it is empty, and the file wins over the built-in
|
|
default. The one exception is a variable named in the file's `env:` section:
|
|
it is set while the file loads, so it overrides both the environment the
|
|
process was started with and the file's own key. A variable's value is
|
|
parsed as the same text in the file would be. The three lists take
|
|
comma-separated entries, with the spaces around each trimmed; an empty
|
|
variable is an empty list. A value that does not parse or is invalid aborts
|
|
startup, naming the variable.
|
|
|
|
| Variable | Config key | Meaning |
|
|
| ------------------------------------ | ------------------------------- | ---------------------------------------------------------------------------- |
|
|
| `PIXA_SIGNING_KEY` | `signing_key` | Required: secret for signed and encrypted URLs and login, 32+ characters |
|
|
| `PORT` | `port` | Port to listen on; default `8080` |
|
|
| `PIXA_STATE_DIR` | `state_dir` | Directory for the database and the disk cache; default `/var/lib/pixa` |
|
|
| `PIXA_DB_URL` | `db_url` | SQLite database URL; default `state.sqlite3` in the state directory |
|
|
| `PIXA_CACHE_MAX_BYTES` | `cache_max_bytes` | Disk cache limit in bytes; `0` disables it; default 75% of free space |
|
|
| `PIXA_ALLOWLIST_HOSTS` | `allowlist_hosts` | Upstream hosts served without a signature |
|
|
| `PIXA_BLOCKED_NETWORKS` | `blocked_networks` | CIDR ranges never fetched from, on top of the built-in ones |
|
|
| `PIXA_TRUSTED_PROXIES` | `trusted_proxies` | CIDR ranges of proxies whose `X-Forwarded-For` is believed; default RFC 1918 |
|
|
| `PIXA_ALLOW_HTTP` | `allow_http` | Allow plain-HTTP upstreams, for testing only; default `false` |
|
|
| `PIXA_UPSTREAM_CONNECTIONS_PER_HOST` | `upstream_connections_per_host` | Concurrent connections per upstream host; default `20` |
|
|
| `PIXA_METRICS_USERNAME` | `metrics.username` | Username for `/metrics`, which is served only when both are set |
|
|
| `PIXA_METRICS_PASSWORD` | `metrics.password` | Password for `/metrics`; set together with the username |
|
|
| `PIXA_SENTRY_DSN` | `sentry_dsn` | Sentry DSN for error reporting; empty disables it |
|
|
| `PIXA_DEBUG` | `debug` | Debug logging and plain-HTTP local development; default `false` |
|
|
| `PIXA_MAINTENANCE_MODE` | `maintenance_mode` | Maintenance flag reported by the health check; default `false` |
|
|
|
|
Key settings in more detail:
|
|
|
|
- `access_control_allow_origin` — CORS origin
|
|
- `allowlist_hosts` — list of allowed upstream hosts
|
|
- `blocked_networks` — list of CIDR ranges to refuse for SSRF protection,
|
|
added to the always-enforced built-in ranges (loopback, private,
|
|
link-local, CGNAT, benchmark, NAT64, and the like); an invalid CIDR
|
|
aborts startup
|
|
- `trusted_proxies` — list of CIDR ranges of the reverse proxies in front
|
|
of pixa. `X-Forwarded-For` is believed only when the direct peer falls
|
|
inside one of these ranges; the logged and login-recorded client
|
|
address is then the rightmost forwarded entry that is not itself a
|
|
trusted proxy. Otherwise the direct peer address is used and the header
|
|
is ignored, so a client connecting directly cannot spoof its address.
|
|
An omitted key defaults to the RFC 1918 private ranges (`10.0.0.0/8`,
|
|
`172.16.0.0/12`, `192.168.0.0/16`), since pixa is deployed behind a
|
|
proxy on a private network; an explicitly empty list (`[]`) trusts no
|
|
one, and an explicit list replaces the default. An invalid CIDR aborts
|
|
startup. Set this to your proxy's address range if it is not already
|
|
covered by the defaults
|
|
- `upstream_fetch_timeout` — timeout for origin requests
|
|
- `upstream_max_response_size` — max origin response size
|
|
- `downstream_timeout` — client response timeout
|
|
- `signing_key` — HMAC secret for URL signatures
|
|
- `cache_max_bytes` — disk cache size limit in bytes; `0` disables the
|
|
disk cache entirely; omitted defaults to 75% of the free space on
|
|
the filesystem containing `<state_dir>/cache/` (minimum 500 MiB)
|
|
|
|
See `config.example.yml` for all options with defaults.
|
|
|
|
### Architecture
|
|
|
|
- **Dependency injection**: Uber fx
|
|
- **HTTP router**: go-chi
|
|
- **Image processing**: govips (CGO wrapper for libvips)
|
|
- **Database**: SQLite via modernc.org/sqlite
|
|
- **Static assets**: embedded via `//go:embed`
|
|
- **Metrics**: Prometheus
|
|
- **Logging**: stdlib slog
|
|
|
|
## Entrypoints
|
|
|
|
This repository adheres to the
|
|
[Scripts to Rule Them All](https://github.com/github/scripts-to-rule-them-all)
|
|
standard: normalized scripts in `script/` are the entrypoints for the
|
|
development workflow, and the Makefile targets are thin shims that call
|
|
them. We provide:
|
|
|
|
- `script/bootstrap` — install all dependencies (idempotent)
|
|
- `script/setup` — make a fresh clone ready for development
|
|
(bootstrap, then install-precommit)
|
|
- `script/projectname` — output the project name ("pixa")
|
|
- `script/test` — run the test suite
|
|
- `script/lint` — run golangci-lint
|
|
- `script/fmt` — format all code (writes)
|
|
- `script/fmt-check` — check formatting (read-only)
|
|
- `script/check` — run test, lint, and fmt-check
|
|
- `script/docker` — build the Docker image tagged via `script/projectname`
|
|
- `script/docker-smoke` — build the image, start it, wait for it to be healthy
|
|
- `script/cibuild` — CI entrypoint: `docker build .` (the Dockerfile
|
|
runs the checks, so a green build implies a green repo)
|
|
- `script/precommit` — pre-commit checks (`go mod tidy` guard, then
|
|
`script/check`)
|
|
- `script/install-precommit` — install the git pre-commit hook that
|
|
runs `script/precommit`
|
|
|
|
## TODO
|
|
|
|
See [TODO.md](TODO.md) for the full prioritized task list.
|
|
|
|
## License
|
|
|
|
GPL-3.0. See [LICENSE](LICENSE).
|
|
|
|
## Author
|
|
|
|
[@sneak](https://sneak.berlin)
|