clawbot
6573b9d1ef
All checks were successful
check / check (push) Successful in 5s
Extracts HMAC-SHA256 request signing out of `internal/imgcache/` into its own `internal/signature/` package, per the plan in [issue #39](#39). This is one of the remaining "easily separable" extractions (`imageprocessor`, `allowlist`, `magic`, and `httpfetcher` already landed). Only the signer is moved here so the diff stays reviewable. ## What moved From `internal/imgcache/signature.go` and its tests into `internal/signature/`: - `Signer` type, its `New` constructor, `Sign`, `Verify`, `GenerateSignedURL` - `ParseParams` (query-string signature/expiration parsing) - Signature error sentinels ## One-way import edge To keep the import edge one-way (`imgcache` depends on `signature`, never the reverse), the package defines a standalone `Request` type carrying just the fields the signature covers, instead of importing `imgcache.ImageRequest`. `imgcache` projects its `ImageRequest` onto `signature.Request` via a small unexported `signatureRequest` helper. This mirrors how the `magic` extraction defined its own `ImageFormat` type. ## Renames (no stuttering) - `NewSigner` -> `signature.New` - `ParseSignatureParams` -> `signature.ParseParams` - `ErrSignatureRequired`/`Invalid`/`Expired` -> `signature.ErrRequired`/`Invalid`/`Expired` The `ErrRequired` message is updated from "non-whitelisted host" to "non-allowlisted host" for inclusive terminology, consistent with the `allowlist` rename. ## Rework (post-review) Three commits added after review feedback: - `d69019b` — golden known-answer test pinning the exact HMAC signatures and signed URL paths for three fixed vectors (resized, resized+query, orig size), cross-validated against an independent HMAC implementation. Any change to the signed byte format now fails loudly. - `43b9f1c` — whitelist→allowlist rename completed across `internal/imgcache` and `internal/handlers` (`ServiceConfig.Allowlist`, `Allowlist` interface, `IsAllowlisted`, test helpers and test names). - `3dc1999` — one-pass config surface rename, no back-compat alias: YAML key `whitelist_hosts` → `allowlist_hosts`, `Config.WhitelistHosts` → `Config.AllowlistHosts`, `config.example.yml`, `scripts/manual-test.sh`, and `README.md` (which also documented a nonexistent `source_host_whitelist` key — now fixed to the real one). ## Behavior Pure refactor apart from the config key rename above. The bytes fed to the HMAC are unchanged (`host:path:query:width:height:format:expiration`), so previously issued signatures remain valid — now enforced by the golden test. All existing tests move with the package. `script/cibuild` passes at head `3dc1999` (fmt-check, lint, test, build). refs #39 Co-authored-by: sneak <sneak@sneak.berlin> Co-authored-by: Jeffrey Paul <sneak@noreply.example.org> Reviewed-on: #46 Co-authored-by: clawbot <clawbot@noreply.example.org> Co-committed-by: clawbot <clawbot@noreply.example.org>
167 lines
5.3 KiB
Markdown
167 lines
5.3 KiB
Markdown
# pixa
|
|
|
|
pixa is a GPL-3.0-licensed Go web server by
|
|
[@sneak](https://sneak.berlin) that proxies images from upstream
|
|
sources, optionally resizing or transforming them, and serves the
|
|
results. Both source and transformed images are cached to disk so that
|
|
subsequent requests are served without origin fetches or additional
|
|
processing.
|
|
|
|
## Getting Started
|
|
|
|
```bash
|
|
# clone and build
|
|
git clone https://git.eeqj.de/sneak/pixa.git
|
|
cd pixa
|
|
make build
|
|
|
|
# run with a config file
|
|
./bin/pixad --config config.example.yml
|
|
|
|
# or build and run via Docker
|
|
make docker
|
|
docker run -p 8080:8080 pixad:latest
|
|
```
|
|
|
|
## Rationale
|
|
|
|
Image-heavy web applications need a fast, caching reverse proxy that
|
|
can resize and transcode images on the fly. pixa fills that role as a
|
|
single, self-contained binary with no external runtime dependencies
|
|
beyond libvips. It supports HMAC-SHA256 signed URLs with expiration to
|
|
prevent abuse, and allowlisted source hosts for open access.
|
|
|
|
## Design
|
|
|
|
### Storage
|
|
|
|
- **Source content**:
|
|
`<statedir>/cache/src-content/<ab>/<cd>/<sha256 of source content>`
|
|
- **Source metadata**:
|
|
`<statedir>/cache/src-metadata/<hostname>/<sha256 of path>.json`
|
|
(fetch time, original headers, request, content hash)
|
|
- **Database**: `<statedir>/state.sqlite3` (SQLite)
|
|
- **Output documents**:
|
|
`<statedir>/cache/dst-content/<ab>/<cd>/<sha256 of output content>`
|
|
|
|
Multiple source paths may reference the same content blob; the
|
|
database tracks references rather than using filesystem refcounting.
|
|
In-process caching of request-to-output mappings targets 1-5k r/s.
|
|
|
|
### Routes
|
|
|
|
```
|
|
/v1/image/<host>/<path>/<size>.<format>?sig=<signature>&exp=<expiration>
|
|
```
|
|
|
|
Images are only fetched from origins using TLS with valid certificates.
|
|
|
|
- `<format>`: one of `orig`, `png`, `jpeg`, `webp`
|
|
- `<size>`: `orig` or `<width>x<height>` (e.g. `800x600`)
|
|
|
|
### Source Hosts
|
|
|
|
Source hosts may be allowlisted in the configuration. Non-allowlisted
|
|
hosts require an HMAC-SHA256 signature.
|
|
|
|
#### Signature Specification
|
|
|
|
Signatures use HMAC-SHA256 and include an expiration timestamp to
|
|
prevent replay attacks. Signatures are **exact match only**: every
|
|
component (host, path, query, dimensions, format, expiration) must
|
|
match exactly what was signed. No suffix matching, wildcard matching,
|
|
or partial matching is supported.
|
|
|
|
**Signed data format** (colon-separated):
|
|
|
|
```
|
|
HMAC-SHA256(secret, "host:path:query:width:height:format:expiration")
|
|
```
|
|
|
|
Where:
|
|
|
|
- `host` — source origin hostname (e.g. `cdn.example.com`)
|
|
- `path` — source path (e.g. `/photos/cat.jpg`)
|
|
- `query` — source query string, empty string if none
|
|
- `width` — requested width in pixels, `0` for original
|
|
- `height` — requested height in pixels, `0` for original
|
|
- `format` — output format (jpeg, png, webp, avif, gif, orig)
|
|
- `expiration` — Unix timestamp when signature expires
|
|
|
|
**Example:** resize
|
|
`https://cdn.example.com/photos/cat.jpg` to 800x600 WebP with
|
|
expiration 1704067200:
|
|
|
|
1. Build input:
|
|
`cdn.example.com:/photos/cat.jpg::800:600:webp:1704067200`
|
|
2. Compute HMAC-SHA256 with your secret key
|
|
3. Base64URL-encode the result
|
|
4. URL:
|
|
`/v1/image/cdn.example.com/photos/cat.jpg/800x600.webp?sig=<base64url>&exp=1704067200`
|
|
|
|
**Allowlist patterns:**
|
|
|
|
- **Exact match**: `cdn.example.com` — matches only that host
|
|
- **Suffix match**: `.example.com` — matches `cdn.example.com`,
|
|
`images.example.com`, and `example.com`
|
|
|
|
### Configuration
|
|
|
|
Configured via YAML file (`--config`). Key settings:
|
|
|
|
- `access_control_allow_origin` — CORS origin
|
|
- `allowlist_hosts` — list of allowed upstream hosts
|
|
- `upstream_fetch_timeout` — timeout for origin requests
|
|
- `upstream_max_response_size` — max origin response size
|
|
- `downstream_timeout` — client response timeout
|
|
- `signing_key` — HMAC secret for URL signatures
|
|
|
|
See `config.example.yml` for all options with defaults.
|
|
|
|
### Architecture
|
|
|
|
- **Dependency injection**: Uber fx
|
|
- **HTTP router**: go-chi
|
|
- **Image processing**: govips (CGO wrapper for libvips)
|
|
- **Database**: SQLite via modernc.org/sqlite
|
|
- **Static assets**: embedded via `//go:embed`
|
|
- **Metrics**: Prometheus
|
|
- **Logging**: stdlib slog
|
|
|
|
## Entrypoints
|
|
|
|
This repository adheres to the
|
|
[Scripts to Rule Them All](https://github.com/github/scripts-to-rule-them-all)
|
|
standard: normalized scripts in `script/` are the entrypoints for the
|
|
development workflow, and the Makefile targets are thin shims that call
|
|
them. We provide:
|
|
|
|
- `script/bootstrap` — install all dependencies (idempotent)
|
|
- `script/setup` — make a fresh clone ready for development
|
|
(bootstrap, then install-precommit)
|
|
- `script/projectname` — output the project name ("pixa")
|
|
- `script/test` — run the test suite
|
|
- `script/lint` — run golangci-lint
|
|
- `script/fmt` — format all code (writes)
|
|
- `script/fmt-check` — check formatting (read-only)
|
|
- `script/check` — run test, lint, and fmt-check
|
|
- `script/docker` — build the Docker image tagged via `script/projectname`
|
|
- `script/cibuild` — CI entrypoint: `docker build .` (the Dockerfile
|
|
runs the checks, so a green build implies a green repo)
|
|
- `script/precommit` — pre-commit checks (`go mod tidy` guard, then
|
|
`script/check`)
|
|
- `script/install-precommit` — install the git pre-commit hook that
|
|
runs `script/precommit`
|
|
|
|
## TODO
|
|
|
|
See [TODO.md](TODO.md) for the full prioritized task list.
|
|
|
|
## License
|
|
|
|
GPL-3.0. See [LICENSE](LICENSE).
|
|
|
|
## Author
|
|
|
|
[@sneak](https://sneak.berlin)
|