diff --git a/README.md b/README.md index b6610e4..588e2db 100644 --- a/README.md +++ b/README.md @@ -92,8 +92,69 @@ the metadata file stored beside it. ### Routes +pixa answers these routes; any other path answers 404. A path in this list asked +with a method the list does not give answers 405, except `/static/`, which +answers any method as it answers `GET`. A browser's CORS preflight request +(`OPTIONS` with `Origin` and `Access-Control-Request-Method` headers) to any +path under `/v1/` answers 200, in maintenance mode too. + +- `GET /` — the login page, or the URL generator page with a login session + (see Encrypted URLs). Needs: nothing. Answers: 200. +- `POST /` — log in with the signing key typed into the login page. Needs: the + login page's form (below). Answers: 303 to `/` with a login session cookie + that lasts 30 days for the right key; 200 with the login page and an error for + a wrong key; 429 over the login limit (below). +- `POST /generate` — make an encrypted URL from the generator page's form. + Needs: a login session and the generator page's form (below); without a login + session it answers 303 to `/`. Answers: 200 with the page showing the URL; 400 + with the page naming a field that is not valid; 500 when the URL cannot be + made. +- `GET /logout` — end the login session. Needs: nothing. Answers: 303 to `/`. +- `GET` or `HEAD` `/v1/image///.` — an image, fetched, + resized and converted (below). Needs: a signature, unless the host is + allowlisted (see Source Hosts). Answers: 200; 304 when `If-None-Match` matches + the image's `ETag`; 400 for a URL or parameter that is not valid; 401 for a + missing or wrong signature, a missing `exp` or an `exp` in the past; 403 when + the upstream host, or a host it redirects to, is `localhost`, ends in + `.localhost` or `.local`, or has an address in a blocked network (see + `blocked_networks`); 502 when the upstream answered with an error status, and + for 5 minutes after that for the same source URL; 503 when pixa is busy or in + maintenance mode; 500 for any other failure. +- `GET /v1/e//` — an image through an encrypted URL (see Encrypted + URLs). Needs: nothing but the URL. Answers: 200; 400 for a token that does not + decrypt, or that asks for a size or fit that is not valid; 410 once it has + expired; 504 when the upstream has not sent its response headers within + `upstream_fetch_timeout`, but 500 when that time runs out while the image + itself is still arriving; 403, 502, 503 and 500 as for `/v1/image/`. +- `GET /robots.txt` — asks every crawler to stay away (`Disallow: /`). Needs: + nothing. Answers: 200. +- `GET /.well-known/healthcheck.json` — JSON with `status` (`ok`), `now`, + `uptime_seconds`, `uptime_human`, `version`, `appname` and + `maintenance_mode`. Needs: nothing. Answers: 200, always. +- `GET /static/` — the script the login and generator pages load. Needs: + nothing. Answers: 200, or 404 for a file that does not exist. +- `GET /metrics` — Prometheus metrics (see Architecture). Needs: HTTP basic + authentication with `metrics.username` and `metrics.password`. Answers: 200; + 401 without them; 404 when they are not set, as the route then does not exist. + +Both `POST` routes accept only a form that pixa's own page served: the page puts +a token in the form and sets a cookie to match, and a request without both is +refused with 403, so another site cannot submit the form from a visitor's +browser. The login and generator pages are meant to be opened over HTTPS: while +`debug` is off, a form sent from a page opened over plain HTTP is refused with +403, and while it is on, so is one sent from a page opened over HTTPS. Plain +HTTP is for development on the browser's own machine: the login session cookie +is always marked `Secure`, and over plain HTTP a browser keeps such a cookie +only for its own machine (`localhost`), if at all. A form is also refused with +403 when the page's host is not the `Host` header pixa receives, so a reverse +proxy in front of pixa must pass that header on unchanged. A form body over +1 MiB is refused with 413. The image routes answer the errors listed for them +with JSON holding `error`, `status` and `timestamp`. + +An image URL has this form: + ``` -/v1/image///.?sig=&exp= +/v1/image///.?sig=&exp=&q=&fit= ``` Images are only fetched from origins using TLS with valid certificates, unless @@ -106,6 +167,11 @@ than once, is refused with 400. - ``: one of `orig` (or `original`), `jpeg` (or `jpg`), `png`, `webp`, `avif`, `gif` - ``: `orig` or `x` (e.g. `800x600`) +- `sig` and `exp`: the signature and its expiry, needed unless the host is + allowlisted (see Signature Specification) +- `q` and `fit`: the output quality and how the image is fitted to ``, + both optional (values under Signature Specification). Both are part of what + is cached, so each value of either is a separate cached image. An image is served with `Cache-Control: public, max-age=, immutable`. When the URL has an expiry (an `exp`, or the TTL of an encrypted URL), @@ -139,6 +205,39 @@ its own `X-Forwarded-For`, whether it connects directly or through the proxy, because its own address is trusted too. Setting `trusted_proxies` to only the address pixa sees for requests that come through the proxy closes this. +### Encrypted URLs + +An encrypted URL is an image URL made on pixa's own web page by someone who +knows the signing key. It works for any upstream host, allowlisted or not, +without a signature, and whoever gets it can neither read the source URL from it +nor change what it asks for. + +1. Open `/` in a browser over HTTPS (or over plain HTTP while `debug` is on, see + Routes) and log in with the signing key (`signing_key`). The login session + lasts 30 days, or until `/logout`. +2. On the generator page, give the source image's URL, the width and height, the + format, quality and fit, and how long the URL lasts, then submit the form + (`POST /generate`). Width and height both empty or `0` keep the original + size; if only one of them is empty or `0`, that side is scaled to keep the + image's proportions. +3. The page shows the URL, `https:///v1/e//img.`, and when + it expires. `` is the host the page was opened on, and the URL starts + with `http` instead while `debug` is on. The name after the token is ignored + and only gives the URL a file extension, `jpg` for `orig`. + +The token holds the source's host, path and query and the size, format, +quality, fit and expiry, encrypted with a key derived from `signing_key`. The +source URL's scheme is not kept: the image is fetched like any other (see +Routes), and the blocked networks still apply. + +How long the URL lasts is chosen on the page, from 1 minute to 1 year, or +never. The expiry is fixed in the token when the URL is made and cannot be +changed or revoked afterwards. Until then the image is served with a `max-age` +that ends at the expiry (see Routes); after it the URL answers 410 +`URL has expired`. A URL made to last forever stops working only when +`signing_key` changes: changing it makes every encrypted URL already handed out +answer 400, and ends every login session. + ### Image Metadata pixa decodes and re-encodes every image it serves, and removes all metadata from @@ -237,6 +336,17 @@ startup naming it, as an unknown config key does. The one other accepted name is `PIXA_CONFIG_PATH`, the config file's path (like `--config`). The variables set by the file's `env:` section are checked the same way. +pixa reads at most one config file: the one given with `--config` (or `-c`), +otherwise the one `PIXA_CONFIG_PATH` names, otherwise the first of these that +pixa finds: `/etc/pixa/config.yml`, `/etc/pixa/config.yaml`, +`~/.config/pixa/config.yml`, `~/.config/pixa/config.yaml`, then `config.yml` +and `config.yaml` in the working directory. A named file that does not exist, +cannot be read or does not parse aborts startup. Of the files pixa looks for on +its own, one it finds but cannot read or parse aborts startup; one it cannot +find, for any reason, is passed over without a message, even when the file is +there in a directory pixa may not enter. With no file, pixa uses the environment +and the defaults. + | Variable | Config key | Meaning | | ------------------------------------ | ------------------------------- | ---------------------------------------------------------------------------- | | `PIXA_SIGNING_KEY` | `signing_key` | Required: secret for signed and encrypted URLs and login, 32+ characters | @@ -322,12 +432,12 @@ Key settings in more detail: seconds for one to free up; if none does, and `downstream_timeout` has not ended first, it is answered 503 the same way - `maintenance_mode` — while `true`, the image routes (`/v1/image/` and - `/v1/e/`) answer every request with 503, a `Retry-After` header and a JSON - error body. The health check (`/.well-known/healthcheck.json`) still answers - 200 and reports `"maintenance_mode": true`. It stays 200 because the image's - Docker `HEALTHCHECK` requests it: a 503 there would make the container - unhealthy, and upaas marks a deploy failed when its container is unhealthy. - The login and URL generator pages and `/metrics` keep working + `/v1/e/`) answer every request for an image with 503, a `Retry-After` header + and a JSON error body. The health check (`/.well-known/healthcheck.json`) + still answers 200 and reports `"maintenance_mode": true`. It stays 200 + because the image's Docker `HEALTHCHECK` requests it: a 503 there would make + the container unhealthy, and upaas marks a deploy failed when its container + is unhealthy. The login and URL generator pages and `/metrics` keep working See `config.example.yml` for all options with defaults. diff --git a/TODO.md b/TODO.md index a15e955..0c4ac74 100644 --- a/TODO.md +++ b/TODO.md @@ -29,6 +29,14 @@ P2: security: referer blacklist # Completed Steps +- 2026-10-04 routes, encrypted URLs and config file documented (closes #75): + "Routes" in `README.md` lists every route with its method, purpose, what it + needs and the status codes it answers with, and says `q` and `fit` are part + of what is cached; "Encrypted URLs" covers logging in, making one on the + generator page, how long it lasts and the 410 once it has expired; + "Configuration" gives the order in which pixa looks for its config file; + `config.example.yml` lists `db_url` and `env` and gives every key's default; + `scripts/manual-test.sh` is left to #97. - 2026-10-04 shutdown stops cache eviction in progress (closes #102): `StartEviction` runs the eviction goroutine with its own context, which `StopEviction` cancels, so a pass in progress stops at its next database @@ -441,7 +449,5 @@ P2: security: referer blacklist - integration tests for the image proxy flow - load tests to verify the 1k to 5k req/s target - P2: documentation - - configuration options - - API endpoints - deployment guide - example nginx or caddy reverse proxy config diff --git a/config.example.yml b/config.example.yml index ac32c98..2133dd9 100644 --- a/config.example.yml +++ b/config.example.yml @@ -12,27 +12,38 @@ # Durations are Go duration strings such as 30s or 2m and must be # positive; a bare number has no unit and aborts startup. Sizes are a # whole number of bytes. +# +# A key left out takes the default its comment gives. -# Server settings +# Port to listen on (default: 8080) port: 8080 + +# Debug logging and plain-HTTP local development (default: false) debug: false # While true, the image routes (/v1/image/ and /v1/e/) answer every request -# with 503 and a Retry-After header. The health check keeps answering 200 and -# reports maintenance_mode as true. It stays 200 because the image's Docker -# HEALTHCHECK requests it: a 503 there would make the container unhealthy, and -# upaas marks a deploy failed when its container is unhealthy. +# for an image with 503 and a Retry-After header. The health check keeps +# answering 200 and reports maintenance_mode as true. It stays 200 because +# the image's Docker HEALTHCHECK requests it: a 503 there would make the +# container unhealthy, and upaas marks a deploy failed when its container is +# unhealthy. (default: false) maintenance_mode: false # Data directory for SQLite database and cache files +# (default: /var/lib/pixa) state_dir: ./data +# SQLite database URL (default: +# file:/state.sqlite3?_journal_mode=WAL). An empty value aborts +# startup; leave the key out to use the default. +# db_url: "file:./data/state.sqlite3?_journal_mode=WAL" + # Image proxy settings # HMAC signing key for URL signatures (required, at least 32 characters) # Generate with: openssl rand -base64 32 signing_key: "CHANGE_ME_generate_with_openssl_rand_base64_32" -# Hosts that don't require signatures +# Hosts that don't require signatures (default: none) # Use "." prefix for wildcard subdomain matching (e.g., ".example.com" matches "cdn.example.com") allowlist_hosts: - s3.sneak.cloud @@ -45,7 +56,7 @@ allowlist_hosts: # SSRF protection. These are added to the always-enforced built-in ranges # (loopback, RFC 1918 private, link-local, CGNAT, benchmark, NAT64, and # similar), never replacing them. Each entry must be a valid CIDR in IPv4 -# or IPv6 form; an invalid entry aborts startup. +# or IPv6 form; an invalid entry aborts startup. (default: none) # blocked_networks: # - 100.64.0.0/10 # - 2001:db8::/32 @@ -72,6 +83,7 @@ allowlist_hosts: # - 2001:db8::/32 # Allow HTTP upstream (only for testing, always use HTTPS in production) +# (default: false) allow_http: false # Maximum concurrent connections per upstream host (default: 20) @@ -121,10 +133,16 @@ access_control_allow_origin: "*" # with a minimum of 500 MiB. # cache_max_bytes: 10737418240 -# Sentry error reporting (optional) +# Sentry DSN for error reporting (default: empty, which turns it off) sentry_dsn: "" -# Metrics endpoint authentication (optional) +# Username and password for /metrics, set together (default: unset). Metrics +# are measured and /metrics is served only when both are set. # metrics: # username: "admin" # password: "secret" + +# Environment variables set while this file loads, as described at the top +# (default: none) +# env: +# PIXA_DEBUG: "true"