3 Commits
Author SHA1 Message Date
sneak b1dc65761b Say that a config file's env: section overrides the environment (closes #128)
check / check (push) Successful in 2m56s
A variable named in the config file's env: section is set while the
file loads, so it overrides both the environment the process was started
with and the file's own key. README.md and the config.example.yml header
said only that a variable wins over the file; they now state this
exception where the precedence is given.

Model: opus-5-5
2026-09-28 10:32:05 +00:00
sneak 2005143e3d Take every setting from PIXA_ variables and PORT (closes #128)
check / check (push) Successful in 2m57s
Each config key can now be set by PIXA_ plus the key in upper case
("." written as "_"), and the port by PORT. One list pairs keys with
variables; the typed getters read a present variable, even an empty
one, before the config file, so every existing check covers it, and
errors a variable can reach name both the key and the variable. An
empty string for blocked_networks or trusted_proxies is now an empty
list, as for allowlist_hosts. The image no longer bakes in
config.docker.yml or passes --config, and its HEALTHCHECK probes
${PORT:-8080}; the config file is looked for under /etc/pixa rather
than /etc/pixad, so a file mounted at /etc/pixa/config.yml is still
read.

Model: opus-5-5
2026-09-28 10:08:37 +00:00
sneak f16b6bd6a6 test: every setting from its environment variable (closes #128)
Tests, written before the change, for the PIXA_ variables and PORT:
every key set from the environment with no config file, PORT over the
file's port, a list variable replacing the file's list, an empty
variable as a set value, invalid values aborting startup naming the
variable, and the signing key and metrics password never printed. All
fail until the change lands, except the check that a config file alone
behaves as before. TestMain unsets PORT and every PIXA_ variable so the
shell running the tests cannot change their result.

Model: opus-5-5
2026-09-28 10:07:50 +00:00
42 changed files with 187 additions and 2152 deletions
+3 -10
View File
@@ -1,5 +1,4 @@
# Lint stage
# Same image as Dockerfile.lint: change both pins together.
# golangci/golangci-lint:v2.12.2-alpine, 2026-08-07
FROM golangci/golangci-lint:v2.12.2-alpine@sha256:91b27804074a0bacea298707f016911e60cf0cdbc6c7bf5ccacb5f0606d18d60 AS lint
@@ -14,9 +13,6 @@ RUN go mod download
# Copy source code
COPY . .
# Tells script/lint it is inside a container, so it runs the linter.
ENV container=docker
# Run formatting check and linter
RUN make fmt-check
RUN make lint
@@ -61,20 +57,17 @@ RUN apk add --no-cache \
vips \
libheif \
ca-certificates \
tzdata \
su-exec
tzdata
# Copy binary from builder
COPY --from=builder /pixad /usr/local/bin/pixad
COPY deploy/docker-entrypoint.sh /usr/local/bin/docker-entrypoint.sh
# Create non-root user, config directory, and data directory
RUN adduser -D -H -s /sbin/nologin pixad && \
mkdir -p /var/lib/pixa /etc/pixa && \
chown pixad:pixad /var/lib/pixa
# No USER: the entrypoint must start as root to give a bind-mounted
# /var/lib/pixa to pixad; it then runs the server as pixad.
USER pixad
WORKDIR /var/lib/pixa
EXPOSE 8080
@@ -87,4 +80,4 @@ HEALTHCHECK --interval=30s --timeout=5s --start-period=10s --retries=3 \
# Settings come from PORT and the PIXA_ environment variables; only
# PIXA_SIGNING_KEY is required. A config file mounted at
# /etc/pixa/config.yml is optional and is read when present.
ENTRYPOINT ["/usr/local/bin/docker-entrypoint.sh"]
ENTRYPOINT ["/usr/local/bin/pixad"]
-34
View File
@@ -1,34 +0,0 @@
# Dockerfile.lint: the container script/lint builds to run golangci-lint,
# which is never installed on the host. Pinned to the same image as the
# Dockerfile lint stage: change both pins together, or the two run
# different linter versions.
#
# golangci/golangci-lint:v2.12.2-alpine, 2026-08-07
FROM golangci/golangci-lint:v2.12.2-alpine@sha256:91b27804074a0bacea298707f016911e60cf0cdbc6c7bf5ccacb5f0606d18d60
# pixa is CGO/libvips: the type-aware linters compile every package, so
# this image needs the same C libraries the build does.
RUN apk add --no-cache build-base vips-dev libheif-dev pkgconfig
WORKDIR /src
# Modules first for layer caching; go.mod/go.sum settle this layer's
# result, so it may safely be reused between runs.
COPY go.mod go.sum ./
RUN go mod download
COPY . .
# Tells script/lint it is inside a container, so it runs the linter.
ENV container=docker
# script/lint passes a different CACHEBUST on every run, and BuildKit
# keys every RUN after this ARG on its value, so the lint step always
# runs instead of returning a cached success that linted nothing.
#
# Go's and golangci-lint's caches (/root/.cache, hundreds of MB) go on a
# tmpfs that is discarded after the step. Written into the layer, they
# would pile up as build cache on every run, since no later run, with
# its new CACHEBUST, can reuse that layer.
ARG CACHEBUST
RUN --mount=type=tmpfs,target=/root/.cache script/lint
+1 -1
View File
@@ -10,7 +10,7 @@ ifdef HAS_PKGCONFIG
NIX_RUN_PREFIX =
NIX_RUN_SUFFIX =
else
NIX_RUN_PREFIX = nix-shell -p pkg-config vips libheif git --run '
NIX_RUN_PREFIX = nix-shell -p pkg-config vips libheif golangci-lint git --run '
NIX_RUN_SUFFIX = '
endif
+12 -85
View File
@@ -34,33 +34,6 @@ else has a built-in default. A config file mounted at `/etc/pixa/config.yml`
is optional: it is read when present, and an environment variable wins over
the same setting in it.
## Running under upaas
What the [upaas](https://git.eeqj.de/sneak/upaas) app for pixa needs:
- **Port:** pixa listens on container port `8080`.
- **Volume:** container path `/var/lib/pixa`, where pixa keeps its
database and cache. upaas bind-mounts the host path it is given and
does not create it, so the host directory must exist before the first
deploy.
- **Environment variables:**
- `PIXA_SIGNING_KEY` (required): secret for signed and encrypted URLs
and login, 32+ characters, for example from
`openssl rand -base64 32`
- `PIXA_ALLOWLIST_HOSTS`: upstream hosts served without a signature,
comma-separated
- `PIXA_CACHE_MAX_BYTES`: disk cache limit in bytes; `0` disables it;
default 75% of free space
- the rest are in the table under Configuration below
- **Health check:** the image's `HEALTHCHECK` requests
`/.well-known/healthcheck.json`. upaas reads the container's health 60
seconds after a deploy and marks the deploy failed unless it is
`healthy`. The probe uses the port from `PORT` (default `8080`), so a
port changed only in a mounted config file is not seen by it: change
the port with `PORT`.
- **First run:** create the host directory. It may be owned by root: the
container gives it to its `pixad` user when it starts.
## Rationale
Image-heavy web applications need a fast, caching reverse proxy that
@@ -94,38 +67,9 @@ In-process caching of request-to-output mappings targets 1-5k r/s.
Images are only fetched from origins using TLS with valid certificates.
A request whose query string cannot be decoded, or gives any parameter more
than once, is refused with 400.
- `<format>`: one of `orig`, `png`, `jpeg`, `webp`
- `<size>`: `orig` or `<width>x<height>` (e.g. `800x600`)
The login form (`POST /`) is limited to 5 attempts per minute per client
address, counting an IPv6 client by its /64; an attempt over the limit is
refused with 429 and a `Retry-After` header. Behind a reverse proxy the client
address comes from `X-Forwarded-For` only when the proxy's address is in
`trusted_proxies`; otherwise all users behind the proxy are counted as one
client. With the default `trusted_proxies` (the RFC 1918 ranges), a client
with a private address can choose the address it is counted by through its own
`X-Forwarded-For`, whether it connects directly or through the proxy, because
its own address is trusted too. Setting `trusted_proxies` to the proxy's own
address closes this.
### Image Metadata
pixa decodes and re-encodes every image it serves, and removes all metadata from
the output: EXIF (GPS position, camera make, model and serial number, capture
time, embedded thumbnail), XMP, IPTC and the ICC colour profile. This cannot be
turned off.
- The `orig` format means the source's own format, not the source's bytes: an
`orig` image is re-encoded and stripped like any other.
- An image with an EXIF orientation is turned upright first, so it displays the
same without the tag; a requested size applies to the upright image.
- An image with an ICC profile is converted to sRGB first, since clients show an
image with no profile as sRGB. Colours outside sRGB, such as the most
saturated ones in a Display P3 photo, are clipped.
### Source Hosts
Source hosts may be allowlisted in the configuration. Non-allowlisted
@@ -135,14 +79,14 @@ hosts require an HMAC-SHA256 signature.
Signatures use HMAC-SHA256 and include an expiration timestamp to
prevent replay attacks. Signatures are **exact match only**: every
component (host, path, query, dimensions, format, expiration, quality,
fit) must match exactly what was signed. No suffix matching, wildcard
matching, or partial matching is supported.
component (host, path, query, dimensions, format, expiration) must
match exactly what was signed. No suffix matching, wildcard matching,
or partial matching is supported.
**Signed data format** (colon-separated):
```
HMAC-SHA256(secret, "host:path:query:width:height:format:expiration:quality:fit")
HMAC-SHA256(secret, "host:path:query:width:height:format:expiration")
```
Where:
@@ -153,30 +97,19 @@ Where:
- `width` — requested width in pixels, `0` for original
- `height` — requested height in pixels, `0` for original
- `format` — output format (jpeg, png, webp, avif, gif, orig)
- `expiration` — the URL's `exp` query parameter, the Unix timestamp when
the signature expires; a request whose `exp` is not a whole number, an
empty `exp=` included, is refused with 400
- `quality` — the URL's `q` query parameter, a whole number from 1 to 100,
or `85` when the URL has no `q`; a request whose `q` is anything else is
refused with 400
- `fit` — the URL's `fit` query parameter (cover, contain, fill, inside,
outside), or `cover` when the URL has no `fit`; a request whose `fit` is
anything else, an empty `fit=` included, is refused with 400
- `expiration` — Unix timestamp when signature expires
**Example:** resize `https://cdn.example.com/photos/cat.jpg` to 800x600
WebP with expiration 1704067200, default quality and fit:
**Example:** resize
`https://cdn.example.com/photos/cat.jpg` to 800x600 WebP with
expiration 1704067200:
1. Build input:
`cdn.example.com:/photos/cat.jpg::800:600:webp:1704067200:85:cover`
`cdn.example.com:/photos/cat.jpg::800:600:webp:1704067200`
2. Compute HMAC-SHA256 with your secret key
3. Base64URL-encode the result
4. URL:
`/v1/image/cdn.example.com/photos/cat.jpg/800x600.webp?sig=<base64url>&exp=1704067200`
For the same image at quality 40 with fit `contain`, the input ends in
`:40:contain` and the URL is
`/v1/image/cdn.example.com/photos/cat.jpg/800x600.webp?sig=<base64url>&exp=1704067200&q=40&fit=contain`.
**Allowlist patterns:**
- **Exact match**: `cdn.example.com` — matches only that host
@@ -194,11 +127,7 @@ process was started with and the file's own key. A variable's value is
parsed as the same text in the file would be. The three lists take
comma-separated entries, with the spaces around each trimmed; an empty
variable is an empty list. A value that does not parse or is invalid aborts
startup, naming the variable. A variable whose name starts with `PIXA_` but
is not in the table below, such as a misspelled one or `PIXA_PORT`, aborts
startup naming it, as an unknown config key does. The one other accepted
name is `PIXA_CONFIG_PATH`, the config file's path (like `--config`). The
variables set by the file's `env:` section are checked the same way.
startup, naming the variable.
| Variable | Config key | Meaning |
| ------------------------------------ | ------------------------------- | ---------------------------------------------------------------------------- |
@@ -231,8 +160,7 @@ Key settings in more detail:
inside one of these ranges; the logged and login-recorded client
address is then the rightmost forwarded entry that is not itself a
trusted proxy. Otherwise the direct peer address is used and the header
is ignored, so a client connecting directly from an address outside
these ranges cannot spoof its address.
is ignored, so a client connecting directly cannot spoof its address.
An omitted key defaults to the RFC 1918 private ranges (`10.0.0.0/8`,
`172.16.0.0/12`, `192.168.0.0/16`), since pixa is deployed behind a
proxy on a private network; an explicitly empty list (`[]`) trusts no
@@ -272,8 +200,7 @@ them. We provide:
(bootstrap, then install-precommit)
- `script/projectname` — output the project name ("pixa")
- `script/test` — run the test suite
- `script/lint` — run golangci-lint, always in a container (builds
`Dockerfile.lint` when run outside one)
- `script/lint` — run golangci-lint
- `script/fmt` — format all code (writes)
- `script/fmt-check` — check formatting (read-only)
- `script/check` — run test, lint, and fmt-check
+2 -87
View File
@@ -30,71 +30,6 @@ exhaustion
# Completed Steps
- 2026-09-28 strip metadata from processed images (closes #82): every output is
exported with govips' `StripMetadata`, so it carries no EXIF, XMP, IPTC or ICC
profile; the image is first turned upright with `AutoRotate` (before sizes are
worked out) and, when it has an ICC profile, converted to sRGB; the `orig`
format is re-encoded and stripped like any other, as pixa never serves the
source bytes; there is no setting to keep metadata; documented in `README.md`.
- 2026-09-28 rate limit the login form (closes #66): `POST /` is limited to 5
attempts per minute per client address, and an attempt over the limit is
refused with 429 and a `Retry-After` header; the address is the one
`internal/clientip` resolves through `trusted_proxies`, an IPv6 client is
counted by its /64, and an IPv4-mapped address as the IPv4 address it
carries; the limit is a `RateLimit` middleware in `internal/middleware` on
`github.com/go-chi/httprate`, which the image routes can reuse; the library
keeps counts for the current and the previous minute only; documented in
`README.md`.
- 2026-09-28 refuse an unparseable `exp` on `/v1/image/` and log swallowed
cache errors (closes #72): an `exp` in the URL that is not a whole
number, an empty `exp=` included, is a 400 naming `exp` and the value,
instead of being ignored and answered with 401 as if the URL had no
`exp`; only an `exp` missing from the URL is unchanged; `README.md` says
so where it documents `exp`. A failed variant `.meta` write, source
metadata JSON write, `Stats` count query, stats counter update, negative
cache write or expired negative cache delete is now logged at `warn`
with the path or key and the error, and stays non-fatal.
- 2026-09-28 refuse an empty `fit` on `/v1/image/` (closes #139): a
`fit` in the URL with an empty value (`fit=`) is a 400 naming `fit`,
instead of being served as `cover` and verified against a signature
made for `cover`; only a `fit` missing from the URL is still `cover`;
any other value still goes through the existing fit-mode check;
`README.md` says so where it documents `fit`.
- 2026-09-28 refuse an invalid `q` on `/v1/image/` (closes #134): a `q`
that is not a whole number from 1 to 100, an empty `q` included, is a
400 naming `q` and the value, instead of being served at the default
85; the route reads `q` with the generator's quality check
(`parseFormInt` with `minQuality` and `maxQuality`); only a `q` missing
from the URL is still 85; a query string that cannot be decoded, such
as `q=80%`, is a 400 showing it; any query parameter given more than
once (`q`, `fit`, `sig`, `exp` alike) is a 400 naming it, so none is
read from its first value only; `README.md` states the range and both
query-string rules.
- 2026-09-28 unknown `PIXA_` environment variables abort startup (closes
#133): a variable whose name starts with `PIXA_` but is neither a
setting's variable nor `PIXA_CONFIG_PATH` aborts startup naming it, as
an unknown config key does, and `PIXA_PORT` is named with a pointer to
`PORT`; the check runs after the config file loads, so the variables
the file's `env:` section sets are checked too; documented in
`README.md`.
- 2026-09-28 start on a fresh upaas volume (closes #129): the image
starts as root only to give `/var/lib/pixa` to `pixad` when `pixad`
does not own it (`deploy/docker-entrypoint.sh`), then runs the server
as `pixad` through `su-exec`, so a root-owned host directory
bind-mounted there no longer stops the container at startup;
`README.md` gains a "Running under upaas" section.
- 2026-09-28 run all linting in Docker via `Dockerfile.lint` +
`script/lint` (closes #104): `make lint` calls `script/lint`, the only
way the linter is run; inside a container (both Dockerfiles set
`container=docker`) it runs `golangci-lint`, anywhere else it builds the
hash-pinned `Dockerfile.lint`, whose last step runs `script/lint` again;
the `Dockerfile` lint stage runs `make lint`; no host or nix-shell
`golangci-lint` path remains (`script/bootstrap` installs no linter);
a per-run `CACHEBUST` build-arg keeps the lint step from being served
from cache, and a tmpfs mount on that step keeps Go's and
golangci-lint's caches out of its layer, so a run leaves no large build
cache behind; `golangci-lint config verify` stays out, as it fetches its
schema over an unpinned live HTTPS call
- 2026-09-28 every setting as an environment variable (closes #128, also
covers #99): each config key can be set by `PIXA_` plus the key in upper
case (`.` written as `_`), and the port by `PORT`; a variable present in
@@ -107,13 +42,6 @@ exhaustion
the config file is looked for under `/etc/pixa` and `~/.config/pixa`
instead of the daemon name `pixad`; documented in `README.md` and
`config.example.yml`.
- 2026-09-28 quality and fit in the URL signature (closes #60): the signed
data is now `host:path:query:width:height:format:expiration:quality:fit`,
using `85` and `cover` when the URL has no `q` or `fit`, so one signed
URL can no longer be replayed across other quality and fit values to
create unauthorized cache entries and transcodes; the known-answer
vectors in `internal/signature/golden_test.go` and the README signature
specification describe the new format.
- 2026-09-28 Docker image healthcheck (closes #111): a `HEALTHCHECK` in
the runtime stage probing `/.well-known/healthcheck.json` with busybox
`wget`; `script/docker-smoke` (`make docker-smoke`) builds the image,
@@ -142,20 +70,6 @@ exhaustion
(IPv4-mapped forms covered); enforcement stays in the dial-time
re-resolution so the DNS-rebinding window remains closed; documented in
`README.md` and `config.example.yml`.
- 2026-09-21 validate dimensions and fit mode on the encrypted-URL
route and the token generator (closes #62): `imgcache.ValidateDimension`
alone holds the `MaxDimension` bound and is used by the path parser, by
the new `ValidateImageRequest` (which also applies `ValidateFitMode`)
and by the generator; both the `/v1/image/` and `/v1/e/` routes call
`ValidateImageRequest`, so an over-limit size or an unknown fit mode is a
400 rather than an out-of-memory or a 500 from the processor; the URL
generator answers 400 naming the field for a `width` or `height` that is
not a number or fails the shared check, a `quality` that is not a number
from 1 to 100, a `ttl` that is not a number from 0 to the largest number
of seconds the expiry calculation can hold, or an unknown `fit`; an empty
`quality` is 85 and
an empty `ttl` never expires; the form's width and height inputs stop at
8192
- 2026-09-21 http.Server hardening (closes #92): added
`HTTPReadHeaderTimeout` (10s, bounds the slowloris header dribble) and
`HTTPIdleTimeout` (120s, bounds keep-alive reuse) alongside the
@@ -257,9 +171,10 @@ exhaustion
# Future Steps
- P1: strip EXIF and other metadata from processed images (privacy)
- P2: security
- referer blacklist
- per-IP rate limiting on the image routes
- per-IP rate limiting
- per-origin rate limiting
- P2: HTTP response handling
- Last-Modified headers
-15
View File
@@ -1,15 +0,0 @@
#!/bin/sh
# deploy/docker-entrypoint.sh: the Docker image's ENTRYPOINT. It runs as
# root only to give /var/lib/pixa to pixad: a host directory
# bind-mounted there keeps its host owner, often root, and pixad could
# not write to it. The server itself always runs as pixad.
set -eu
main() {
if [ "$(stat -c %U /var/lib/pixa)" != pixad ]; then
chown pixad:pixad /var/lib/pixa
fi
exec su-exec pixad /usr/local/bin/pixad "$@"
}
main "$@"
-3
View File
@@ -11,7 +11,6 @@ require (
github.com/getsentry/sentry-go v0.40.0
github.com/go-chi/chi/v5 v5.2.3
github.com/go-chi/cors v1.2.2
github.com/go-chi/httprate v0.16.0
github.com/gorilla/csrf v1.7.3
github.com/gorilla/securecookie v1.1.2
github.com/prometheus/client_golang v1.23.2
@@ -92,7 +91,6 @@ require (
github.com/inconshreveable/mousetrap v1.1.0 // indirect
github.com/josharian/intern v1.0.0 // indirect
github.com/json-iterator/go v1.1.12 // indirect
github.com/klauspost/cpuid/v2 v2.2.10 // indirect
github.com/kylelemons/godebug v1.1.0 // indirect
github.com/mailru/easyjson v0.7.7 // indirect
github.com/mattn/go-colorable v0.1.13 // indirect
@@ -115,7 +113,6 @@ require (
github.com/tidwall/match v1.1.1 // indirect
github.com/tidwall/pretty v1.2.0 // indirect
github.com/x448/float16 v0.8.4 // indirect
github.com/zeebo/xxh3 v1.0.2 // indirect
go.etcd.io/etcd/api/v3 v3.6.2 // indirect
go.etcd.io/etcd/client/pkg/v3 v3.6.2 // indirect
go.etcd.io/etcd/client/v3 v3.6.2 // indirect
-8
View File
@@ -114,8 +114,6 @@ github.com/go-chi/chi/v5 v5.2.3 h1:WQIt9uxdsAbgIYgid+BpYc+liqQZGMHRaUwp0JUcvdE=
github.com/go-chi/chi/v5 v5.2.3/go.mod h1:L2yAIGWB3H+phAw1NxKwWM+7eUH/lU8pOMm5hHcoops=
github.com/go-chi/cors v1.2.2 h1:Jmey33TE+b+rB7fT8MUy1u0I4L+NARQlK6LhzKPSyQE=
github.com/go-chi/cors v1.2.2/go.mod h1:sSbTewc+6wYHBBCW7ytsFSn836hqM7JxpglAy2Vzc58=
github.com/go-chi/httprate v0.16.0 h1:8V5DH9j6pSK6UQoBsTpvMyFxycqaKEIToyPKzHJjUa8=
github.com/go-chi/httprate v0.16.0/go.mod h1:A8lo+qRhk+s9LiuP5saS7XCGDXRXMcrueq0NfIuCa/I=
github.com/go-errors/errors v1.4.2 h1:J6MZopCL4uSllY1OfXM374weqZFFItUbrImctkmUxIA=
github.com/go-errors/errors v1.4.2/go.mod h1:sIVyrIiJhuEF+Pj9Ebtd6P/rEYROXFi3BopGUQ5a5Og=
github.com/go-jose/go-jose/v4 v4.0.5 h1:M6T8+mKZl/+fNNuFHvGIzDz7BTLQPIounk/b9dw3AaE=
@@ -253,8 +251,6 @@ github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI
github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
github.com/klauspost/compress v1.18.0 h1:c/Cqfb0r+Yi+JtIEq73FWXVkRonBlf0CRNYc8Zttxdo=
github.com/klauspost/compress v1.18.0/go.mod h1:2Pp+KzxcywXVXMr50+X0Q/Lsb43OQHYWRCY2AiWywWQ=
github.com/klauspost/cpuid/v2 v2.2.10 h1:tBs3QSyvjDyFTq3uoc/9xFpCuOsJQFNPiAhYdw2skhE=
github.com/klauspost/cpuid/v2 v2.2.10/go.mod h1:hqwkgyIinND0mEev00jJYCxPNVRVXFQeu1XKlok6oO0=
github.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
github.com/kr/logfmt v0.0.0-20140226030751-b84e30acd515/go.mod h1:+0opPa2QZZtGFBFZlji/RkVcI2GknAs/DXo4wKdlNEc=
github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
@@ -400,10 +396,6 @@ github.com/x448/float16 v0.8.4/go.mod h1:14CWIYCyZA/cWjXOioeEpHeN/83MdbZDRQHoFcY
github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=
github.com/zeebo/assert v1.3.0 h1:g7C04CbJuIDKNPFHmsk4hwZDO5O+kntRxzaUoNXj+IQ=
github.com/zeebo/assert v1.3.0/go.mod h1:Pq9JiuJQpG8JLJdtkwrJESF0Foym2/D9XMU5ciN/wJ0=
github.com/zeebo/xxh3 v1.0.2 h1:xZmwmqxHZA8AI603jOQ0tMqmBr9lPeFwGg6d+xy9DC0=
github.com/zeebo/xxh3 v1.0.2/go.mod h1:5NWz9Sef7zIDm2JHfFlcQvNekmcEl9ekUZQQKCYaDcA=
go.etcd.io/etcd/api/v3 v3.6.2 h1:25aCkIMjUmiiOtnBIp6PhNj4KdcURuBak0hU2P1fgRc=
go.etcd.io/etcd/api/v3 v3.6.2/go.mod h1:eFhhvfR8Px1P6SEuLT600v+vrhdDTdcfMzmnxVXXSbk=
go.etcd.io/etcd/client/pkg/v3 v3.6.2 h1:zw+HRghi/G8fKpgKdOcEKpnBTE4OO39T6MegA0RopVU=
-45
View File
@@ -58,7 +58,6 @@ var (
errValueRequired = errors.New("a value is required")
errValueEmpty = errors.New("value must not be empty")
errUnknownConfigKeys = errors.New("unknown config keys")
errUnknownEnvVars = errors.New("unknown environment variables")
errNotAString = errors.New("not a string")
errNotAnInteger = errors.New("not an integer")
errNotABoolean = errors.New("not a boolean")
@@ -155,13 +154,6 @@ func New(_ fx.Lifecycle, params Params) (*Config, error) {
return nil, err
}
// Loading the config file sets the variables in its env section,
// so this also checks their names.
err = validateKnownEnvVars()
if err != nil {
return nil, err
}
if sc == nil {
log.Info("no config file found, using environment variables and defaults")
}
@@ -385,43 +377,6 @@ func envVarNames() map[string]string {
}
}
// validateKnownEnvVars rejects environment variables whose names start
// with PIXA_ but that are neither a setting's variable nor
// PIXA_CONFIG_PATH, so a misspelled variable fails at startup instead of
// being silently ignored, as validateKnownKeys does for config file keys.
// New calls it after loading the config file, so the variables the
// file's env section sets are checked too.
func validateKnownEnvVars() error {
known := map[string]bool{"PIXA_CONFIG_PATH": true}
for _, name := range envVarNames() {
known[name] = true
}
var unknown []string
for _, entry := range os.Environ() {
name, _, _ := strings.Cut(entry, "=")
switch {
case !strings.HasPrefix(name, "PIXA_") || known[name]:
continue
case name == "PIXA_PORT":
unknown = append(unknown, name+" (use PORT for the port)")
default:
unknown = append(unknown, name)
}
}
if len(unknown) > 0 {
sort.Strings(unknown)
return fmt.Errorf("%w: %s", errUnknownEnvVars, strings.Join(unknown, ", "))
}
return nil
}
// lookupValue returns the value set for key and whether one is set. The
// key's environment variable wins when it is present, even when empty;
// its value is a string, read exactly as the same text quoted in the
-112
View File
@@ -3,14 +3,10 @@ package config
import (
"net/netip"
"os"
"path/filepath"
"reflect"
"slices"
"strings"
"testing"
"sneak.berlin/go/pixa/internal/globals"
"sneak.berlin/go/pixa/internal/logger"
)
// TestMain unsets PORT and every PIXA_ environment variable before the
@@ -99,114 +95,6 @@ func TestEnvironmentSetsEveryKey(t *testing.T) {
}
}
// TestUnknownPixaVariableAbortsStartup checks that a PIXA_ variable that
// is not a setting's variable, such as a misspelled one, aborts startup
// naming it, as an unknown config key does, instead of being ignored.
func TestUnknownPixaVariableAbortsStartup(t *testing.T) {
t.Setenv("PIXA_TRUSTED_PROXY", "192.0.2.0/24")
t.Setenv("PIXA_SIGNINGKEY", validTestSigningKey)
err := validateKnownEnvVars()
wantStartupError(t, err, "PIXA_TRUSTED_PROXY", "PIXA_SIGNINGKEY")
}
// TestPixaPortAbortsStartupPointingToPort checks that PIXA_PORT aborts
// startup with a message saying to use PORT, which sets the port.
func TestPixaPortAbortsStartupPointingToPort(t *testing.T) {
t.Setenv("PIXA_PORT", "9090")
err := validateKnownEnvVars()
wantStartupError(t, err, "PIXA_PORT", "use PORT")
}
// TestSettingVariablesAndConfigPathAreAccepted checks that every
// setting's variable and PIXA_CONFIG_PATH pass the check for unknown
// PIXA_ variables. TestEnvironmentSetsEveryKey pins the names in the list.
func TestSettingVariablesAndConfigPathAreAccepted(t *testing.T) {
// A config file's env section loaded by another test can leave a
// PIXA_ variable set for the whole process, so every one is unset
// here first; t.Setenv restores each when the test ends.
for _, entry := range os.Environ() {
name, _, _ := strings.Cut(entry, "=")
if !strings.HasPrefix(name, "PIXA_") {
continue
}
t.Setenv(name, "")
err := os.Unsetenv(name)
if err != nil {
t.Fatalf("failed to unset %s: %v", name, err)
}
}
t.Setenv("PIXA_CONFIG_PATH", "/etc/pixa/config.yml")
for _, name := range envVarNames() {
t.Setenv(name, "")
}
err := validateKnownEnvVars()
if err != nil {
t.Fatalf("PIXA_CONFIG_PATH and every setting's variable "+
"must be accepted: %v", err)
}
}
// configFromNew writes yamlContent to a temporary config file, points
// PIXA_CONFIG_PATH at it, and runs New, as the server does at startup.
// The state directory is a temporary one and the disk cache is off, so
// New succeeds unless something in the test is wrong.
func configFromNew(t *testing.T, yamlContent string) (*Config, error) {
t.Helper()
tmpDir := t.TempDir()
configPath := filepath.Join(tmpDir, "config.yml")
err := os.WriteFile(configPath, []byte(yamlContent), 0o600)
if err != nil {
t.Fatalf("failed to write test config: %v", err)
}
t.Setenv("PIXA_CONFIG_PATH", configPath)
t.Setenv("PIXA_STATE_DIR", filepath.Join(tmpDir, "state"))
t.Setenv("PIXA_CACHE_MAX_BYTES", "0")
testLogger, err := logger.New(nil, logger.Params{Globals: &globals.Globals{}})
if err != nil {
t.Fatalf("failed to create logger: %v", err)
}
return New(nil, Params{Logger: testLogger})
}
// TestUnknownPixaVariableAbortsNew checks that New, which the server
// calls at startup, aborts on a misspelled PIXA_ variable.
func TestUnknownPixaVariableAbortsNew(t *testing.T) {
t.Setenv("PIXA_TRUSTED_PROXY", "192.0.2.0/24")
_, err := configFromNew(t, signingKeyLine)
wantStartupError(t, err, "PIXA_TRUSTED_PROXY")
}
// TestUnknownPixaVariableInEnvSectionAbortsNew checks that New aborts
// on a misspelled PIXA_ name in the config file's env section, which
// loading the file sets as an environment variable.
func TestUnknownPixaVariableInEnvSectionAbortsNew(t *testing.T) {
// The variable must be absent until the file loads. t.Setenv makes
// sure the one the file sets is removed when the test ends.
t.Setenv("PIXA_TRUSTED_PROXY", "")
err := os.Unsetenv("PIXA_TRUSTED_PROXY")
if err != nil {
t.Fatalf("failed to unset PIXA_TRUSTED_PROXY: %v", err)
}
_, err = configFromNew(t, signingKeyLine+
"env:\n PIXA_TRUSTED_PROXY: 192.0.2.0/24\n")
wantStartupError(t, err, "PIXA_TRUSTED_PROXY")
}
// TestPortFromEnvironmentOverridesConfigFile checks that PORT wins over
// the port in the config file.
func TestPortFromEnvironmentOverridesConfigFile(t *testing.T) {
+17 -133
View File
@@ -1,12 +1,8 @@
package handlers
import (
"bytes"
"crypto/subtle"
"errors"
"fmt"
"html/template"
"math"
"net/http"
"net/url"
"strconv"
@@ -18,20 +14,6 @@ import (
"sneak.berlin/go/pixa/internal/templates"
)
// errInvalidFormField reports a generator form field, or the q or exp
// parameter of /v1/image/, whose value is non-numeric or out of range. The
// offending field name is wrapped in so the response can name it.
var errInvalidFormField = errors.New("invalid")
// Bounds for the generator's quality and ttl fields; the quality bounds also
// apply to the q parameter of /v1/image/. maxTTL is in seconds: the expiry
// calculation time.Duration(ttl) * time.Second overflows above it.
const (
minQuality = 1
maxQuality = 100
maxTTL = int(math.MaxInt64 / time.Second)
)
// HandleRoot serves the login page or generator page based on authentication state.
func (s *Handlers) HandleRoot() http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
@@ -119,26 +101,18 @@ func (s *Handlers) HandleGenerateURL() http.HandlerFunc {
// Validate source URL
parsed, err := url.Parse(sourceURL)
if err != nil || parsed.Host == "" {
s.renderGeneratorWithForm(w, r, "Invalid source URL", r.Form,
http.StatusBadRequest)
s.renderGeneratorWithForm(w, r, "Invalid source URL", r.Form)
return
}
payload, expiresAt, ttl, err := buildGeneratePayload(parsed, r.Form)
if err != nil {
s.renderGeneratorWithForm(w, r, err.Error(), r.Form,
http.StatusBadRequest)
return
}
payload, expiresAt, ttl := buildGeneratePayload(parsed, r.Form)
// Generate encrypted token
token, err := s.encGen.Generate(payload)
if err != nil {
s.log.Error("failed to generate encrypted URL", "error", err)
s.renderGeneratorWithForm(w, r, "Failed to generate URL", r.Form,
http.StatusInternalServerError)
s.renderGeneratorWithForm(w, r, "Failed to generate URL", r.Form)
return
}
@@ -166,40 +140,17 @@ func (s *Handlers) HandleGenerateURL() http.HandlerFunc {
}
// buildGeneratePayload parses the numeric form fields and assembles the
// encrypted URL payload. ttl=0 means never expires (ExpiresAt stays 0). A
// non-numeric or out-of-range width, height, quality or ttl, or an
// unrecognized fit mode, is a client error naming the offending field. The
// format field is passed through unchecked.
// encrypted URL payload. ttl=0 means never expires (ExpiresAt stays 0).
func buildGeneratePayload(
parsed *url.URL, form url.Values,
) (*encurl.Payload, time.Time, int, error) {
width, err := parseFormDimension(form, "width")
if err != nil {
return nil, time.Time{}, 0, err
}
) (*encurl.Payload, time.Time, int) {
width, _ := strconv.Atoi(form.Get("width"))
height, _ := strconv.Atoi(form.Get("height"))
quality, _ := strconv.Atoi(form.Get("quality"))
ttl, _ := strconv.Atoi(form.Get("ttl"))
height, err := parseFormDimension(form, "height")
if err != nil {
return nil, time.Time{}, 0, err
}
quality, err := parseFormInt(form, "quality",
encurl.DefaultQuality, minQuality, maxQuality)
if err != nil {
return nil, time.Time{}, 0, err
}
ttl, err := parseFormInt(form, "ttl", 0, 0, maxTTL)
if err != nil {
return nil, time.Time{}, 0, err
}
fitMode := imgcache.FitMode(form.Get("fit"))
err = imgcache.ValidateFitMode(fitMode)
if err != nil {
return nil, time.Time{}, 0,
fmt.Errorf("%w: %s", imgcache.ErrInvalidFitMode, form.Get("fit"))
if quality <= 0 {
quality = 85
}
var (
@@ -220,57 +171,11 @@ func buildGeneratePayload(
Height: height,
Format: imgcache.ImageFormat(form.Get("format")),
Quality: quality,
FitMode: fitMode,
FitMode: imgcache.FitMode(form.Get("fit")),
ExpiresAt: expiresAtUnix,
}
return payload, expiresAt, ttl, nil
}
// parseFormDimension reads an optional width or height form field. An empty
// value means "original size" (0). A non-numeric value, or one
// imgcache.ValidateDimension rejects, is an error naming the field.
func parseFormDimension(form url.Values, field string) (int, error) {
raw := form.Get(field)
if raw == "" {
return 0, nil
}
value, err := strconv.Atoi(raw)
if err != nil {
return 0, fmt.Errorf("%w %s: not a number", errInvalidFormField, field)
}
err = imgcache.ValidateDimension(field, value)
if err != nil {
return 0, err
}
return value, nil
}
// parseFormInt reads an optional integer form field or URL query parameter,
// returning def when the field is empty and an error naming the field when the
// value is non-numeric or outside minValue to maxValue.
func parseFormInt(
form url.Values, field string, def, minValue, maxValue int,
) (int, error) {
raw := form.Get(field)
if raw == "" {
return def, nil
}
value, err := strconv.Atoi(raw)
if err != nil {
return 0, fmt.Errorf("%w %s: not a number", errInvalidFormField, field)
}
if value < minValue || value > maxValue {
return 0, fmt.Errorf("%w %s: must be from %d to %d",
errInvalidFormField, field, minValue, maxValue)
}
return value, nil
return payload, expiresAt, ttl
}
// generatorData holds template data for the generator page.
@@ -310,15 +215,6 @@ func (s *Handlers) renderLogin(
func (s *Handlers) renderGenerator(
w http.ResponseWriter, r *http.Request, data *generatorData,
) {
s.renderGeneratorStatus(w, r, data, http.StatusOK)
}
// renderGeneratorStatus renders the generator page with an explicit HTTP
// status; a rejected form uses 400. The page is rendered into a buffer before
// the status is written, so a template failure can still answer 500.
func (s *Handlers) renderGeneratorStatus(
w http.ResponseWriter, r *http.Request, data *generatorData, status int,
) {
w.Header().Set("Content-Type", "text/html; charset=utf-8")
@@ -328,29 +224,17 @@ func (s *Handlers) renderGeneratorStatus(
data.CSRFField = csrfField(r)
var page bytes.Buffer
err := templates.Render(&page, "generator.html", data)
err := templates.Render(w, "generator.html", data)
if err != nil {
s.log.Error("failed to render generator template", "error", err)
http.Error(w, "Internal server error", http.StatusInternalServerError)
return
}
w.WriteHeader(status)
_, err = page.WriteTo(w)
if err != nil {
s.log.Error("failed to write generator page", "error", err)
}
}
func (s *Handlers) renderGeneratorWithForm(
w http.ResponseWriter, r *http.Request, errorMsg string,
form url.Values, status int,
w http.ResponseWriter, r *http.Request, errorMsg string, form url.Values,
) {
s.renderGeneratorStatus(w, r, &generatorData{
s.renderGenerator(w, r, &generatorData{
Error: errorMsg,
FormURL: form.Get("url"),
FormWidth: form.Get("width"),
@@ -359,7 +243,7 @@ func (s *Handlers) renderGeneratorWithForm(
FormQuality: form.Get("quality"),
FormFit: form.Get("fit"),
FormTTL: form.Get("ttl"),
}, status)
})
}
func (s *Handlers) buildGeneratedURL(r *http.Request, token, format string) string {
@@ -1,163 +0,0 @@
package handlers
import (
"maps"
"net/http"
"net/http/httptest"
"net/url"
"strconv"
"strings"
"testing"
"time"
"sneak.berlin/go/pixa/internal/encurl"
"sneak.berlin/go/pixa/internal/imgcache"
)
// Generator form field names, and a value that is not a number.
const (
widthField = "width"
heightField = "height"
qualityField = "quality"
ttlField = "ttl"
fitField = "fit"
notANumber = "abc"
)
// generatePost submits the /generate form with a valid session and CSRF token
// plus the caller's extra fields, returning the recorder.
func generatePost(
t *testing.T, extra url.Values,
) *httptest.ResponseRecorder {
t.Helper()
h, srv := newCSRFTestRouter(t)
sessionCookie := newSessionCookie(t, h)
cookies, token := csrfCredentials(t, srv, []*http.Cookie{sessionCookie})
cookies = append(cookies, sessionCookie)
form := url.Values{
sourceURLField: {testSourceURL},
csrfTokenField: {token},
}
maps.Copy(form, extra)
return postForm(srv, "/generate", cookies, form)
}
// TestGeneratePostRejectsNonNumericWidth verifies that a non-numeric width is
// rejected with 400 naming the field rather than being coerced to 0 and
// minting a 0-width token.
func TestGeneratePostRejectsNonNumericWidth(t *testing.T) {
t.Parallel()
rec := generatePost(t, url.Values{"width": {"abc"}})
if rec.Code != http.StatusBadRequest {
t.Fatalf("status = %d, want %d", rec.Code, http.StatusBadRequest)
}
if strings.Contains(rec.Body.String(), "/v1/e/") {
t.Error("a token was generated for non-numeric width")
}
}
// TestGeneratePostRejectsOverLimitWidth verifies that a width beyond
// MaxDimension is rejected at generation time so an unusable token cannot be
// minted.
func TestGeneratePostRejectsOverLimitWidth(t *testing.T) {
t.Parallel()
rec := generatePost(t, url.Values{"width": {"100000"}})
if rec.Code != http.StatusBadRequest {
t.Fatalf("status = %d, want %d", rec.Code, http.StatusBadRequest)
}
if strings.Contains(rec.Body.String(), "/v1/e/") {
t.Error("a token was generated for an over-limit width")
}
}
// TestGeneratePostRejectsBadField verifies that each generator field whose
// value is not a number, out of range, or unrecognized is rejected with 400,
// mints no token, and is named in the error shown on the page.
func TestGeneratePostRejectsBadField(t *testing.T) {
t.Parallel()
tests := []struct {
field, value, wantError string
}{
{widthField, notANumber, "invalid width: not a number"},
{widthField, "-1", "width is negative"},
{widthField, "8193", "width is above 8192"},
{heightField, notANumber, "invalid height: not a number"},
{heightField, "8193", "height is above 8192"},
{qualityField, notANumber, "invalid quality: not a number"},
{qualityField, "0", "invalid quality: must be from 1 to 100"},
{qualityField, "101", "invalid quality: must be from 1 to 100"},
{ttlField, notANumber, "invalid ttl: not a number"},
{ttlField, "-1", "invalid ttl: must be from 0 to"},
{ttlField, "10000000000", "invalid ttl: must be from 0 to"},
{fitField, "bogus", "invalid fit mode: bogus"},
}
for _, tt := range tests {
t.Run(tt.field+"="+tt.value, func(t *testing.T) {
t.Parallel()
rec := generatePost(t, url.Values{tt.field: {tt.value}})
body := rec.Body.String()
if rec.Code != http.StatusBadRequest {
t.Fatalf("status = %d, want %d", rec.Code, http.StatusBadRequest)
}
if strings.Contains(body, "/v1/e/") {
t.Error("a token was generated")
}
if !strings.Contains(body, tt.wantError) {
t.Errorf("page does not show %q", tt.wantError)
}
})
}
}
// TestBuildGeneratePayloadDefaultAndLimits verifies that an empty quality
// takes the default, and that the largest accepted width, height, quality and
// ttl are accepted with an expiry still in the future.
func TestBuildGeneratePayloadDefaultAndLimits(t *testing.T) {
t.Parallel()
parsed, err := url.Parse(testSourceURL)
if err != nil {
t.Fatalf("url.Parse() error = %v", err)
}
payload, _, _, err := buildGeneratePayload(parsed, url.Values{})
if err != nil {
t.Fatalf("empty form: error = %v", err)
}
if payload.Quality != encurl.DefaultQuality {
t.Errorf("empty quality gave %d, want %d",
payload.Quality, encurl.DefaultQuality)
}
_, expiresAt, _, err := buildGeneratePayload(parsed, url.Values{
widthField: {strconv.Itoa(imgcache.MaxDimension)},
heightField: {strconv.Itoa(imgcache.MaxDimension)},
qualityField: {strconv.Itoa(maxQuality)},
ttlField: {strconv.Itoa(maxTTL)},
})
if err != nil {
t.Fatalf("largest accepted values: error = %v", err)
}
if !expiresAt.After(time.Now()) {
t.Errorf("ttl %d gave expiry %v, want a time in the future",
maxTTL, expiresAt)
}
}
-118
View File
@@ -4,7 +4,6 @@ import (
"bytes"
"context"
"database/sql"
"encoding/json"
"image"
"image/color"
"image/jpeg"
@@ -277,120 +276,3 @@ func TestHandleImage_ETagHeader(t *testing.T) {
t.Errorf("ETag should be quoted, got %q", etag)
}
}
// TestHandleImage_InvalidFitMode_Returns400 verifies that the plain image
// route rejects an unrecognized fit mode with 400.
func TestHandleImage_InvalidFitMode_Returns400(t *testing.T) {
t.Parallel()
fix := setupTestHandler(t)
status := getImage(t, fix,
"/v1/image/"+fix.goodHost+"/images/photo.jpg/50x50.jpeg?fit=bogus")
if status != http.StatusBadRequest {
t.Fatalf("status = %d, want %d", status, http.StatusBadRequest)
}
}
// TestHandleImage_InvalidQuery_Returns400 verifies that the plain image route
// answers a q that is not a whole number from 1 to 100, an empty one
// included, with 400 naming q and the value, a parameter given more than once
// with 400 naming it, and a query string that cannot be decoded with 400
// showing it, instead of serving the image at the default quality 85 or at
// the first value given.
func TestHandleImage_InvalidQuery_Returns400(t *testing.T) {
t.Parallel()
tests := []struct {
query, wantError string
}{
{"q=banana", `invalid q: not a number, got "banana"`},
{"q=0", `invalid q: must be from 1 to 100, got "0"`},
{"q=101", `invalid q: must be from 1 to 100, got "101"`},
{"q=", `invalid q: not a number, got ""`},
{"q=80&q=500", `invalid q: given more than once`},
{"q=80&q=", `invalid q: given more than once`},
{"fit=cover&fit=contain", `invalid fit: given more than once`},
{"q=80%", `invalid query string "q=80%": invalid URL escape "%"`},
{
"q=50;fit=contain",
`invalid query string "q=50;fit=contain": ` +
`invalid semicolon separator in query`,
},
}
for _, tt := range tests {
t.Run(tt.query, func(t *testing.T) {
t.Parallel()
fix := setupTestHandler(t)
r := chi.NewRouter()
r.Get("/v1/image/*", fix.handler.HandleImage())
req := httptest.NewRequestWithContext(t.Context(), http.MethodGet,
"/v1/image/"+fix.goodHost+"/images/photo.jpg/50x50.jpeg?"+tt.query, nil)
rec := httptest.NewRecorder()
r.ServeHTTP(rec, req)
if rec.Code != http.StatusBadRequest {
t.Fatalf("status = %d, want %d", rec.Code, http.StatusBadRequest)
}
t.Logf("GET %s: %d %s", req.URL, rec.Code, rec.Body)
var body struct {
Error string `json:"error"`
}
err := json.NewDecoder(rec.Body).Decode(&body)
if err != nil {
t.Fatalf("decoding response body: %v", err)
}
if body.Error != tt.wantError {
t.Errorf("error = %q, want %q", body.Error, tt.wantError)
}
})
}
}
// TestHandleImage_EmptyFit_Returns400 verifies that the plain image route
// answers a fit that is in the URL but empty with 400 naming fit, instead of
// serving the image as cover. Only a fit missing from the URL means cover.
func TestHandleImage_EmptyFit_Returns400(t *testing.T) {
t.Parallel()
fix := setupTestHandler(t)
r := chi.NewRouter()
r.Get("/v1/image/*", fix.handler.HandleImage())
req := httptest.NewRequestWithContext(t.Context(), http.MethodGet,
"/v1/image/"+fix.goodHost+"/images/photo.jpg/50x50.jpeg?fit=", nil)
rec := httptest.NewRecorder()
r.ServeHTTP(rec, req)
if rec.Code != http.StatusBadRequest {
t.Fatalf("status = %d, want %d", rec.Code, http.StatusBadRequest)
}
t.Logf("GET %s: %d %s", req.URL, rec.Code, rec.Body)
var body struct {
Error string `json:"error"`
}
err := json.NewDecoder(rec.Body).Decode(&body)
if err != nil {
t.Fatalf("decoding response body: %v", err)
}
wantError := `invalid fit: not a fit mode, got ""`
if body.Error != wantError {
t.Errorf("error = %q, want %q", body.Error, wantError)
}
}
+25 -87
View File
@@ -2,15 +2,12 @@ package handlers
import (
"errors"
"fmt"
"io"
"net/http"
"net/url"
"strconv"
"time"
"github.com/go-chi/chi/v5"
"sneak.berlin/go/pixa/internal/encurl"
"sneak.berlin/go/pixa/internal/httpfetcher"
"sneak.berlin/go/pixa/internal/imgcache"
)
@@ -92,66 +89,39 @@ func (s *Handlers) parseImageRequest(
// Convert to ImageRequest
req := parsed.ToImageRequest()
// Parse signature params from query string. r.URL.Query() would silently
// drop a pair it cannot decode, such as q=80%, so that q would be served
// at 85; a query string that cannot be decoded is refused instead. A
// parameter given more than once is refused too, as only its first value
// would be read.
query, err := url.ParseQuery(r.URL.RawQuery)
if err != nil {
s.respondError(w, fmt.Sprintf("invalid query string %q: %v",
r.URL.RawQuery, err), http.StatusBadRequest)
// Parse signature params from query string
query := r.URL.Query()
req.Signature = query.Get("sig")
return nil, false
if expStr := query.Get("exp"); expStr != "" {
exp, parseErr := strconv.ParseInt(expStr, 10, 64)
if parseErr == nil {
req.Expires = time.Unix(exp, 0)
}
}
for name, values := range query {
if len(values) > 1 {
s.respondError(w, fmt.Sprintf("invalid %s: given more than once",
name), http.StatusBadRequest)
// Parse optional quality and fit params
if qStr := query.Get("q"); qStr != "" {
q, parseErr := strconv.Atoi(qStr)
if parseErr == nil && q > 0 && q <= 100 {
req.Quality = q
}
}
if fit := query.Get("fit"); fit != "" {
req.FitMode = imgcache.FitMode(fit)
fitErr := imgcache.ValidateFitMode(req.FitMode)
if fitErr != nil {
s.respondError(w, "invalid fit mode: "+fit, http.StatusBadRequest)
return nil, false
}
}
req.Signature = query.Get("sig")
req.Expires, err = parseExpires(query)
if err != nil {
s.respondError(w, err.Error(), http.StatusBadRequest)
return nil, false
}
// Parse optional quality and fit params. Only a q missing from the URL is
// 85. A q in the URL that is not a whole number from 1 to 100, an empty
// one included, is refused, checked as the generator checks its quality
// field; that check alone would take an empty q as missing.
qStr := query.Get("q")
if query.Has("q") && qStr == "" {
s.respondError(w, `invalid q: not a number, got ""`,
http.StatusBadRequest)
return nil, false
}
req.Quality, err = parseFormInt(query, "q",
encurl.DefaultQuality, minQuality, maxQuality)
if err != nil {
s.respondError(w, fmt.Sprintf("%v, got %q", err, qStr),
http.StatusBadRequest)
return nil, false
}
// Only a fit missing from the URL is cover. A fit in the URL that is not a
// fit mode is refused by the fit-mode check below; that check would take an
// empty fit as missing, so an empty one is refused here.
req.FitMode = imgcache.FitMode(query.Get("fit"))
if query.Has("fit") && req.FitMode == "" {
s.respondError(w, `invalid fit: not a fit mode, got ""`, http.StatusBadRequest)
return nil, false
// Default quality if not set
if req.Quality == 0 {
req.Quality = 85
}
// Default fit mode if not set
@@ -159,41 +129,9 @@ func (s *Handlers) parseImageRequest(
req.FitMode = imgcache.FitCover
}
// Enforce dimension and fit-mode bounds, shared with the encrypted-URL
// route. Dimensions are already bounded by the path parser above; this
// also rejects an unrecognized fit mode with 400 instead of letting it
// reach the processor as a 500.
err = imgcache.ValidateImageRequest(req)
if err != nil {
s.respondError(w, "invalid image request: "+err.Error(),
http.StatusBadRequest)
return nil, false
}
return req, true
}
// parseExpires reads the exp query parameter, a Unix time in seconds. An exp
// missing from the URL gives the zero time, which the signature check takes
// as no expiration. An exp in the URL that is not a whole number, an empty
// one included, is an error naming exp and the value.
func parseExpires(query url.Values) (time.Time, error) {
if !query.Has("exp") {
return time.Time{}, nil
}
expStr := query.Get("exp")
exp, err := strconv.ParseInt(expStr, 10, 64)
if err != nil {
return time.Time{}, fmt.Errorf("%w exp: not a number, got %q",
errInvalidFormField, expStr)
}
return time.Unix(exp, 0), nil
}
// respondImageError maps image retrieval errors to HTTP responses.
func (s *Handlers) respondImageError(
w http.ResponseWriter, req *imgcache.ImageRequest, err error,
@@ -1,171 +0,0 @@
package handlers
import (
"encoding/json"
"fmt"
"net/http"
"net/http/httptest"
"testing"
"time"
"github.com/go-chi/chi/v5"
"sneak.berlin/go/pixa/internal/imgcache"
"sneak.berlin/go/pixa/internal/signature"
)
// signedHost is not on the allowlist setupTestHandler builds, so a request
// for it needs a valid signature. No image is served for it: a request that
// passes the signature check gets 502 from the failed fetch, and one that
// fails the check gets 401.
const signedHost = "signed.example.com"
// getImage sends a GET for target to the image route of fix and returns the
// response status.
func getImage(t *testing.T, fix *testFixtures, target string) int {
t.Helper()
r := chi.NewRouter()
r.Get("/v1/image/*", fix.handler.HandleImage())
req := httptest.NewRequestWithContext(t.Context(), http.MethodGet, target, nil)
rec := httptest.NewRecorder()
r.ServeHTTP(rec, req)
t.Logf("GET %s: %d", target, rec.Code)
return rec.Code
}
// TestHandleImage_SignatureCoversQualityAndFit signs a URL for quality 85
// and fit cover, the values the route uses when a URL has no q or fit, and
// sends that signature with each q and fit below.
func TestHandleImage_SignatureCoversQualityAndFit(t *testing.T) {
t.Parallel()
expires := time.Now().Add(time.Hour)
signer := signature.New("test-signing-key-must-be-32-chars")
sig := signer.Sign(&signature.Request{
SourceHost: signedHost,
SourcePath: "/images/photo.jpg",
Width: 50,
Height: 50,
Format: string(imgcache.FormatJPEG),
Quality: 85,
FitMode: string(imgcache.FitCover),
Expires: expires,
})
signedURL := fmt.Sprintf("/v1/image/%s/images/photo.jpg/50x50.jpeg?sig=%s&exp=%d",
signedHost, sig, expires.Unix())
tests := []struct {
name string
query string
wantStatus int
}{
{"no q or fit", "", http.StatusBadGateway},
{"q=85 and fit=cover", "&q=85&fit=cover", http.StatusBadGateway},
{"replayed with q=40", "&q=40", http.StatusUnauthorized},
{"replayed with fit=contain", "&fit=contain", http.StatusUnauthorized},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
status := getImage(t, setupTestHandler(t), signedURL+tt.query)
if status != tt.wantStatus {
t.Errorf("status = %d, want %d", status, tt.wantStatus)
}
})
}
}
// TestHandleImage_GeneratedSignedURLVerifies sends URLs built by the
// service's signed-URL generator to the route.
func TestHandleImage_GeneratedSignedURLVerifies(t *testing.T) {
t.Parallel()
tests := []struct {
name string
quality int
fitMode imgcache.FitMode
}{
{"quality 40 and fit contain", 40, imgcache.FitContain},
{"quality and fit unset", 0, ""},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
fix := setupTestHandler(t)
signedURL, err := fix.service.GenerateSignedURL("", &imgcache.ImageRequest{
SourceHost: signedHost,
SourcePath: "/images/photo.jpg",
Size: imgcache.Size{Width: 50, Height: 50},
Format: imgcache.FormatJPEG,
Quality: tt.quality,
FitMode: tt.fitMode,
}, time.Hour)
if err != nil {
t.Fatalf("GenerateSignedURL() error = %v", err)
}
status := getImage(t, fix, signedURL)
if status != http.StatusBadGateway {
t.Errorf("status = %d, want %d", status, http.StatusBadGateway)
}
})
}
}
// TestHandleImage_InvalidExp_Returns400 sends a signed-host URL whose exp is
// not a whole number, and one whose exp is empty. Each is refused with 400
// naming exp and the value, not with the 401 a URL without exp still gets.
func TestHandleImage_InvalidExp_Returns400(t *testing.T) {
t.Parallel()
tests := []struct {
query string
wantStatus int
wantError string
}{
{"sig=x&exp=banana", http.StatusBadRequest,
`invalid exp: not a number, got "banana"`},
{"sig=x&exp=", http.StatusBadRequest, `invalid exp: not a number, got ""`},
{"sig=x", http.StatusUnauthorized, "unauthorized"},
}
for _, tt := range tests {
t.Run(tt.query, func(t *testing.T) {
t.Parallel()
fix := setupTestHandler(t)
r := chi.NewRouter()
r.Get("/v1/image/*", fix.handler.HandleImage())
req := httptest.NewRequestWithContext(t.Context(), http.MethodGet,
"/v1/image/"+signedHost+"/images/photo.jpg/50x50.jpeg?"+tt.query, nil)
rec := httptest.NewRecorder()
r.ServeHTTP(rec, req)
t.Logf("GET %s: %d %s", req.URL, rec.Code, rec.Body)
var body struct {
Error string `json:"error"`
}
err := json.NewDecoder(rec.Body).Decode(&body)
if err != nil {
t.Fatalf("decoding response body: %v", err)
}
if rec.Code != tt.wantStatus || body.Error != tt.wantError {
t.Errorf("got %d %q, want %d %q",
rec.Code, body.Error, tt.wantStatus, tt.wantError)
}
})
}
}
-13
View File
@@ -50,19 +50,6 @@ func (s *Handlers) HandleImageEnc() http.HandlerFunc {
// Convert payload to ImageRequest
req := payload.ToImageRequest()
// Apply the same dimension and fit-mode bounds as the plain image
// route: a sealed payload is trusted for its origin, not for staying
// within limits, so an over-limit size or unknown fit mode is a 400
// here rather than an out-of-memory or a 500 from the processor.
err = imgcache.ValidateImageRequest(req)
if err != nil {
s.log.Debug("encrypted URL failed validation", "error", err)
s.respondError(w, "invalid encrypted URL: "+err.Error(),
http.StatusBadRequest)
return
}
// Log the request
s.log.Debug("encrypted image request",
"host", req.SourceHost,
@@ -1,98 +0,0 @@
package handlers
import (
"context"
"log/slog"
"net/http"
"net/http/httptest"
"testing"
"github.com/go-chi/chi/v5"
"sneak.berlin/go/pixa/internal/encurl"
"sneak.berlin/go/pixa/internal/imgcache"
)
// newEncTestServer builds a router serving the encrypted-URL route with a
// generator seeded by the shared test signing key. The image service is left
// nil: these tests exercise validation that rejects a token before any image
// is fetched, so the handler must never reach the service.
func newEncTestServer(t *testing.T) (*encurl.Generator, http.Handler) {
t.Helper()
encGen, err := encurl.NewGenerator(testSigningKey)
if err != nil {
t.Fatalf("encurl.NewGenerator() error = %v", err)
}
h := &Handlers{
log: slog.New(slog.DiscardHandler),
encGen: encGen,
}
r := chi.NewRouter()
r.Get("/v1/e/{token}/*", h.HandleImageEnc())
return encGen, r
}
// getEncToken issues a GET for the given token and returns the recorder.
func getEncToken(srv http.Handler, token string) *httptest.ResponseRecorder {
req := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/v1/e/"+token+"/img.jpg", nil)
rec := httptest.NewRecorder()
srv.ServeHTTP(rec, req)
return rec
}
// TestHandleImageEnc_OverLimitDimension_Returns400 verifies that a decrypted
// token requesting a dimension beyond MaxDimension is rejected with 400
// instead of reaching the image processor and libvips.
func TestHandleImageEnc_OverLimitDimension_Returns400(t *testing.T) {
t.Parallel()
encGen, srv := newEncTestServer(t)
token, err := encGen.Generate(&encurl.Payload{
SourceHost: "cdn.example.com",
SourcePath: "/photo.jpg",
Width: 100000,
Height: 100000,
})
if err != nil {
t.Fatalf("Generate() error = %v", err)
}
rec := getEncToken(srv, token)
if rec.Code != http.StatusBadRequest {
t.Fatalf("status = %d, want %d", rec.Code, http.StatusBadRequest)
}
}
// TestHandleImageEnc_InvalidFitMode_Returns400 verifies that a decrypted token
// carrying an unrecognized fit mode is rejected with 400 rather than surfacing
// as a 500 from the image processor's default branch.
func TestHandleImageEnc_InvalidFitMode_Returns400(t *testing.T) {
t.Parallel()
encGen, srv := newEncTestServer(t)
token, err := encGen.Generate(&encurl.Payload{
SourceHost: "cdn.example.com",
SourcePath: "/photo.jpg",
Width: 800,
Height: 600,
FitMode: imgcache.FitMode("bogus"),
})
if err != nil {
t.Fatalf("Generate() error = %v", err)
}
rec := getEncToken(srv, token)
if rec.Code != http.StatusBadRequest {
t.Fatalf("status = %d, want %d", rec.Code, http.StatusBadRequest)
}
}
-22
View File
@@ -161,13 +161,6 @@ func (p *ImageProcessor) Process(
}
defer img.Close()
// Turn the image upright now: encode strips the EXIF orientation tag,
// and sizes below must be worked out on the upright image.
err = img.AutoRotate()
if err != nil {
return nil, fmt.Errorf("failed to auto-rotate: %w", err)
}
// Get original dimensions
origWidth := img.Width()
origHeight := img.Height()
@@ -411,21 +404,6 @@ func (p *ImageProcessor) encode(
return nil, fmt.Errorf("%w: %s", ErrUnsupportedOutputFormat, format)
}
// Stripping drops the ICC profile as well, and clients show an image
// with no profile as sRGB, so convert to sRGB first. "srgb" names
// libvips' built-in profile; govips' own sRGB path variable is set on
// first use but read without a lock, so concurrent requests race on it.
if img.HasICCProfile() {
err := img.TransformICCProfileWithFallback("srgb", "srgb")
if err != nil {
return nil, fmt.Errorf("failed to convert to sRGB: %w", err)
}
}
// Drop EXIF, XMP, IPTC and the ICC profile. govips ignores this for
// GIF, which carries none of them.
params.StripMetadata = true
output, _, err := img.Export(&params)
if err != nil {
return nil, err
@@ -9,9 +9,7 @@ import (
"image/jpeg"
"image/png"
"io"
"math"
"os"
"slices"
"testing"
"github.com/davidbyttow/govips/v2/vips"
@@ -563,152 +561,3 @@ func TestImageProcessor_EncodeAVIF(t *testing.T) {
encodeAndCheck(t, FormatAVIF, 85, mimeAVIF)
}
// processAndDecode runs input through Process and decodes the output with
// vips, so a test can inspect the image a client would receive.
func processAndDecode(t *testing.T, input []byte, req *Request) *vips.ImageRef {
t.Helper()
result, err := New(Params{}).Process(
context.Background(), bytes.NewReader(input), req,
)
if err != nil {
t.Fatalf("Process() error = %v", err)
}
defer func() { _ = result.Content.Close() }()
data, err := io.ReadAll(result.Content)
if err != nil {
t.Fatalf("failed to read result: %v", err)
}
output, err := vips.NewImageFromBuffer(data)
if err != nil {
t.Fatalf("failed to decode output: %v", err)
}
t.Cleanup(output.Close)
return output
}
func TestImageProcessor_StripsEXIF(t *testing.T) {
t.Parallel()
// gps-exif.jpg carries GPS coordinates, a camera make, model and serial
// number, and a capture time.
input, err := os.ReadFile("testdata/gps-exif.jpg")
if err != nil {
t.Fatalf("failed to read test JPEG: %v", err)
}
fixture, err := vips.NewImageFromBuffer(input)
if err != nil {
t.Fatalf("failed to decode test JPEG: %v", err)
}
t.Cleanup(fixture.Close)
if !slices.Contains(fixture.GetFields(), "exif-ifd3-GPSLatitude") {
t.Fatal("testdata/gps-exif.jpg has no GPS latitude")
}
formats := []Format{
FormatJPEG, FormatPNG, FormatWebP, FormatAVIF, FormatGIF, FormatOriginal,
}
for _, format := range formats {
t.Run(string(format), func(t *testing.T) {
t.Parallel()
output := processAndDecode(t, input, &Request{Format: format})
if output.HasExif() {
t.Errorf("output has EXIF: %v", output.GetExif())
}
})
}
}
func TestImageProcessor_AppliesEXIFOrientation(t *testing.T) {
t.Parallel()
// orientation-6.jpg is stored 16x8, red on the left and blue on the
// right, with EXIF orientation 6 (turn 90 degrees clockwise to view).
// Upright it is 8x16, red on top and blue below.
input, err := os.ReadFile("testdata/orientation-6.jpg")
if err != nil {
t.Fatalf("failed to read test JPEG: %v", err)
}
tests := []struct {
name string
size Size
wantW int
wantH int
}{
{name: "original size", size: Size{}, wantW: 8, wantH: 16},
{name: "width only", size: Size{Width: 4}, wantW: 4, wantH: 8},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
output := processAndDecode(t, input, &Request{
Size: tt.size,
Format: FormatPNG,
})
if output.Width() != tt.wantW || output.Height() != tt.wantH {
t.Fatalf("output is %dx%d, want %dx%d",
output.Width(), output.Height(), tt.wantW, tt.wantH)
}
top, err := output.GetPoint(tt.wantW/2, 0)
if err != nil {
t.Fatalf("GetPoint() error = %v", err)
}
bottom, err := output.GetPoint(tt.wantW/2, tt.wantH-1)
if err != nil {
t.Fatalf("GetPoint() error = %v", err)
}
if top[0] <= top[2] || bottom[2] <= bottom[0] {
t.Errorf("top pixel = %v, bottom pixel = %v, want red above blue",
top, bottom)
}
})
}
}
func TestImageProcessor_ConvertsWideGamutToSRGB(t *testing.T) {
t.Parallel()
// display-p3.jpg is a flat 8x8 image with the Display P3 profile
// embedded, filled with Display P3 (234, 51, 35), which is sRGB red.
input, err := os.ReadFile("testdata/display-p3.jpg")
if err != nil {
t.Fatalf("failed to read test JPEG: %v", err)
}
output := processAndDecode(t, input, &Request{Format: FormatPNG})
if output.HasICCProfile() {
t.Error("output has an ICC profile")
}
pixel, err := output.GetPoint(4, 4)
if err != nil {
t.Fatalf("GetPoint() error = %v", err)
}
want := []float64{255, 0, 0}
for i := range want {
if math.Abs(pixel[i]-want[i]) > 5 {
t.Fatalf("pixel = %v, want within 5 of %v", pixel, want)
}
}
}
Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.3 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.0 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 811 B

+24 -59
View File
@@ -125,7 +125,7 @@ func NewCache(db *sql.DB, config CacheConfig) (*Cache, error) {
}
variants, err := NewVariantStorage(
filepath.Join(config.StateDir, "cache", "variants"), log,
filepath.Join(config.StateDir, "cache", "variants"),
)
if err != nil {
return nil, fmt.Errorf("failed to create variant storage: %w", err)
@@ -263,7 +263,23 @@ func (c *Cache) StoreSource(
return "", fmt.Errorf("failed to insert source metadata: %w", err)
}
c.writeMetadataSidecar(req, pathHash, contentHash, result)
// Store metadata JSON file
meta := &SourceMetadata{
Host: req.SourceHost,
Path: req.SourcePath,
Query: req.SourceQuery,
ContentHash: string(contentHash),
StatusCode: result.StatusCode,
ContentType: result.ContentType,
ContentLength: result.ContentLength,
ResponseHeaders: result.Headers,
FetchedAt: time.Now().UTC().Unix(),
FetchDurationMs: result.FetchDurationMs,
RemoteAddr: result.RemoteAddr,
}
// A failure here is non-fatal; the metadata is in the database.
_ = c.srcMetadata.Store(req.SourceHost, pathHash, meta)
c.notifyWritePressure()
@@ -420,19 +436,12 @@ func (c *Cache) Stats(ctx context.Context) (*CacheStats, error) {
}
// Get actual item count and total size from content tables
err = c.db.QueryRowContext(ctx,
_ = c.db.QueryRowContext(ctx,
`SELECT COUNT(*) FROM request_cache`,
).Scan(&stats.TotalItems)
if err != nil {
c.log.Warn("failed to count cache items for stats", "error", err)
}
err = c.db.QueryRowContext(ctx,
_ = c.db.QueryRowContext(ctx,
`SELECT COALESCE(SUM(size_bytes), 0) FROM output_content`,
).Scan(&stats.TotalSizeBytes)
if err != nil {
c.log.Warn("failed to sum cache size for stats", "error", err)
}
// Compute hit rate as a ratio
if stats.HitCount+stats.MissCount > 0 {
@@ -444,17 +453,15 @@ func (c *Cache) Stats(ctx context.Context) (*CacheStats, error) {
// IncrementStats increments cache statistics.
func (c *Cache) IncrementStats(ctx context.Context, hit bool, fetchBytes int64) {
var err error
if hit {
_, err = c.db.ExecContext(ctx, `
_, _ = c.db.ExecContext(ctx, `
UPDATE cache_stats
SET hit_count = hit_count + 1,
last_updated_at = CURRENT_TIMESTAMP
WHERE id = 1
`)
} else {
_, err = c.db.ExecContext(ctx, `
_, _ = c.db.ExecContext(ctx, `
UPDATE cache_stats
SET miss_count = miss_count + 1,
last_updated_at = CURRENT_TIMESTAMP
@@ -462,52 +469,14 @@ func (c *Cache) IncrementStats(ctx context.Context, hit bool, fetchBytes int64)
`)
}
if err != nil {
c.log.Warn("failed to count cache hit or miss", "hit", hit, "error", err)
}
if fetchBytes > 0 {
_, err = c.db.ExecContext(ctx, `
_, _ = c.db.ExecContext(ctx, `
UPDATE cache_stats
SET upstream_fetch_count = upstream_fetch_count + 1,
upstream_fetch_bytes = upstream_fetch_bytes + ?,
last_updated_at = CURRENT_TIMESTAMP
WHERE id = 1
`, fetchBytes)
if err != nil {
c.log.Warn("failed to count upstream fetch",
"fetch_bytes", fetchBytes, "error", err)
}
}
}
// writeMetadataSidecar writes the JSON metadata sidecar of a stored source.
// A failure is logged and is otherwise non-fatal; the metadata is in the
// database.
func (c *Cache) writeMetadataSidecar(
req *ImageRequest,
pathHash PathHash,
contentHash ContentHash,
result *httpfetcher.FetchResult,
) {
meta := &SourceMetadata{
Host: req.SourceHost,
Path: req.SourcePath,
Query: req.SourceQuery,
ContentHash: string(contentHash),
StatusCode: result.StatusCode,
ContentType: result.ContentType,
ContentLength: result.ContentLength,
ResponseHeaders: result.Headers,
FetchedAt: time.Now().UTC().Unix(),
FetchDurationMs: result.FetchDurationMs,
RemoteAddr: result.RemoteAddr,
}
err := c.srcMetadata.Store(req.SourceHost, pathHash, meta)
if err != nil {
c.log.Warn("failed to write metadata sidecar",
"host", req.SourceHost, "path_hash", pathHash, "error", err)
}
}
@@ -559,14 +528,10 @@ func (c *Cache) checkNegativeCache(
// Check if expired
if time.Now().After(expiresAt) {
// Clean up expired entry
_, err = c.db.ExecContext(ctx, `
_, _ = c.db.ExecContext(ctx, `
DELETE FROM negative_cache
WHERE source_host = ? AND source_path = ? AND source_query = ?
`, req.SourceHost, req.SourcePath, req.SourceQuery)
if err != nil {
c.log.Warn("failed to delete expired negative cache entry",
"host", req.SourceHost, "path", req.SourcePath, "error", err)
}
return false, nil
}
-18
View File
@@ -59,24 +59,6 @@ func ValidateFitMode(fit FitMode) error {
}
}
// ValidateImageRequest checks a request's width and height with
// ValidateDimension and its fit mode with ValidateFitMode. Both the plain
// /v1/image/ route and the encrypted /v1/e/ route validate through this
// function so a request from either source enforces identical bounds.
func ValidateImageRequest(req *ImageRequest) error {
err := ValidateDimension("width", req.Size.Width)
if err != nil {
return err
}
err = ValidateDimension("height", req.Size.Height)
if err != nil {
return err
}
return ValidateFitMode(req.FitMode)
}
// ImageRequest represents a request for a processed image
type ImageRequest struct {
// SourceHost is the origin host (e.g., "cdn.example.com")
+3 -22
View File
@@ -205,23 +205,12 @@ func (s *Service) ValidateRequest(req *ImageRequest) error {
return s.signer.Verify(signatureRequest(req))
}
// GenerateSignedURL generates a signed URL for the given request. The URL
// carries q and fit next to sig and exp, so the image route verifies it for
// the quality and fit it was signed with. An unset quality or fit is first
// set to 85 or cover, the values the route uses when a URL has no q or fit.
// GenerateSignedURL generates a signed URL for the given request.
func (s *Service) GenerateSignedURL(
baseURL string,
req *ImageRequest,
ttl time.Duration,
) (string, error) {
if req.Quality == 0 {
req.Quality = 85
}
if req.FitMode == "" {
req.FitMode = FitCover
}
sigReq := signatureRequest(req)
path, sig, exp := s.signer.GenerateSignedURL(sigReq, ttl)
@@ -229,8 +218,7 @@ func (s *Service) GenerateSignedURL(
req.Expires = sigReq.Expires
req.Signature = sigReq.Signature
return fmt.Sprintf("%s%s?sig=%s&exp=%d&q=%d&fit=%s",
baseURL, path, sig, exp, req.Quality, req.FitMode), nil
return fmt.Sprintf("%s%s?sig=%s&exp=%d", baseURL, path, sig, exp), nil
}
// loadCachedSource attempts to load source content from cache, returning nil
@@ -322,12 +310,7 @@ func (s *Service) fetchAndProcess(
// Store negative cache for certain errors
if isNegativeCacheable(err) {
statusCode := extractStatusCode(err)
storeErr := s.cache.StoreNegative(ctx, req, statusCode, err.Error())
if storeErr != nil {
s.log.Warn("failed to store negative cache entry",
"host", req.SourceHost, "path", req.SourcePath, "error", storeErr)
}
_ = s.cache.StoreNegative(ctx, req, statusCode, err.Error())
}
return nil, fmt.Errorf("upstream fetch failed: %w", err)
@@ -469,8 +452,6 @@ func signatureRequest(req *ImageRequest) *signature.Request {
Width: req.Size.Width,
Height: req.Size.Height,
Format: string(req.Format),
Quality: req.Quality,
FitMode: string(req.FitMode),
Signature: req.Signature,
Expires: req.Expires,
}
-82
View File
@@ -1,12 +1,9 @@
package imgcache
import (
"bytes"
"context"
"database/sql"
"log/slog"
"math"
"strings"
"testing"
"time"
@@ -104,82 +101,3 @@ func TestStats_ZeroCounts(t *testing.T) {
t.Errorf("HitRate = %f, want 0.0 for zero counts", stats.HitRate)
}
}
// TestStats_LogsFailedCountQueries verifies that a failed item count query
// and a failed size query are each logged at warn and Stats still succeeds.
func TestStats_LogsFailedCountQueries(t *testing.T) {
t.Parallel()
db := setupStatsTestDB(t)
var logBuf bytes.Buffer
cache, err := NewCache(db, CacheConfig{
StateDir: t.TempDir(),
CacheTTL: time.Hour,
NegativeTTL: 5 * time.Minute,
Logger: slog.New(slog.NewJSONHandler(&logBuf, nil)),
})
if err != nil {
t.Fatal(err)
}
_, err = db.ExecContext(t.Context(),
`DROP TABLE request_cache; DROP TABLE output_content`)
if err != nil {
t.Fatal(err)
}
_, err = cache.Stats(t.Context())
if err != nil {
t.Fatalf("Stats() error = %v, want nil", err)
}
for _, msg := range []string{
"failed to count cache items for stats",
"failed to sum cache size for stats",
} {
want := `"level":"WARN","msg":"` + msg + `"`
if !strings.Contains(logBuf.String(), want) {
t.Errorf("log missing %s; got %q", want, logBuf.String())
}
}
}
// TestIncrementStats_LogsFailedUpdates verifies that a failed hit or miss
// count update and a failed upstream fetch count update are each logged at
// warn.
func TestIncrementStats_LogsFailedUpdates(t *testing.T) {
t.Parallel()
db := setupStatsTestDB(t)
var logBuf bytes.Buffer
cache, err := NewCache(db, CacheConfig{
StateDir: t.TempDir(),
CacheTTL: time.Hour,
NegativeTTL: 5 * time.Minute,
Logger: slog.New(slog.NewJSONHandler(&logBuf, nil)),
})
if err != nil {
t.Fatal(err)
}
_, err = db.ExecContext(t.Context(), `DROP TABLE cache_stats`)
if err != nil {
t.Fatal(err)
}
cache.IncrementStats(t.Context(), false, 1024)
for _, msg := range []string{
"failed to count cache hit or miss",
"failed to count upstream fetch",
} {
want := `"level":"WARN","msg":"` + msg + `"`
if !strings.Contains(logBuf.String(), want) {
t.Errorf("log missing %s; got %q", want, logBuf.String())
}
}
}
+3 -10
View File
@@ -7,7 +7,6 @@ import (
"errors"
"fmt"
"io"
"log/slog"
"os"
"path/filepath"
"time"
@@ -393,7 +392,6 @@ func CacheKey(req *ImageRequest) VariantKey {
// Unlike ContentStorage, the key is provided by the caller (not computed from content).
type VariantStorage struct {
baseDir string
log *slog.Logger
}
// VariantMeta contains metadata about a cached variant.
@@ -406,14 +404,13 @@ type VariantMeta struct {
}
// NewVariantStorage creates a new variant storage at the given base directory.
// A failed .meta write is logged to log.
func NewVariantStorage(baseDir string, log *slog.Logger) (*VariantStorage, error) {
func NewVariantStorage(baseDir string) (*VariantStorage, error) {
err := os.MkdirAll(baseDir, StorageDirPerm)
if err != nil {
return nil, fmt.Errorf("failed to create variant storage directory: %w", err)
}
return &VariantStorage{baseDir: baseDir, log: log}, nil
return &VariantStorage{baseDir: baseDir}, nil
}
// Store writes content and metadata to storage at the given key.
@@ -481,11 +478,7 @@ func (s *VariantStorage) Store(
}
// Metadata write failure is non-fatal; content is already stored.
err = os.WriteFile(metaPath, metaData, StorageFilePerm)
if err != nil {
s.log.Warn("failed to write variant metadata sidecar",
"path", metaPath, "error", err)
}
_ = os.WriteFile(metaPath, metaData, StorageFilePerm)
return size, nil
}
@@ -4,10 +4,8 @@ import (
"bytes"
"errors"
"io"
"log/slog"
"os"
"path/filepath"
"strings"
"testing"
)
@@ -406,35 +404,3 @@ func TestCacheKey(t *testing.T) {
t.Error("CacheKey() produced same key for different quality")
}
}
// TestVariantStorage_StoreLogsFailedMetaWrite verifies that a .meta write
// that fails is logged at warn and the store still succeeds.
func TestVariantStorage_StoreLogsFailedMetaWrite(t *testing.T) {
t.Parallel()
var logBuf bytes.Buffer
storage, err := NewVariantStorage(
t.TempDir(), slog.New(slog.NewJSONHandler(&logBuf, nil)))
if err != nil {
t.Fatalf("NewVariantStorage() error = %v", err)
}
key := CacheKey(&ImageRequest{SourceHost: testHostCDN, SourcePath: testPathCat})
// A directory where the .meta file goes makes the .meta write fail.
err = os.MkdirAll(storage.keyToPath(key)+".meta", StorageDirPerm)
if err != nil {
t.Fatalf("failed to create directory: %v", err)
}
_, err = storage.Store(key, bytes.NewReader([]byte("variant data")), "image/webp")
if err != nil {
t.Fatalf("Store() error = %v, want nil", err)
}
want := `"level":"WARN","msg":"failed to write variant metadata sidecar"`
if !strings.Contains(logBuf.String(), want) {
t.Errorf("log missing %s; got %q", want, logBuf.String())
}
}
+4 -25
View File
@@ -23,21 +23,6 @@ var (
// MaxDimension is the maximum allowed width or height.
const MaxDimension = 8192
// ValidateDimension checks one requested width or height; name ("width" or
// "height") appears in the error. 0 means "original size" and is valid.
func ValidateDimension(name string, value int) error {
if value < 0 {
return fmt.Errorf("%w: %s is negative", ErrInvalidSize, name)
}
if value > MaxDimension {
return fmt.Errorf("%w: %s is above %d",
ErrDimensionTooLarge, name, MaxDimension)
}
return nil
}
// sizeFormatRegex matches patterns like "800x600.webp", "0x0.jpeg", "orig.png"
var sizeFormatRegex = regexp.MustCompile(`^(\d+)x(\d+)\.(\w+)$|^(orig)\.(\w+)$`)
@@ -240,20 +225,14 @@ func parseSizeFormat(s string) (Size, ImageFormat, error) {
return Size{}, "", ErrInvalidSize
}
if width > MaxDimension || height > MaxDimension {
return Size{}, "", ErrDimensionTooLarge
}
size = Size{Width: width, Height: height}
formatStr = matches[3]
}
err := ValidateDimension("width", size.Width)
if err != nil {
return Size{}, "", err
}
err = ValidateDimension("height", size.Height)
if err != nil {
return Size{}, "", err
}
format, err := parseFormat(formatStr)
if err != nil {
return Size{}, "", err
@@ -1,64 +0,0 @@
package imgcache
import (
"errors"
"testing"
)
func TestValidateImageRequest(t *testing.T) {
t.Parallel()
tests := []struct {
name string
req ImageRequest
wantErr error
}{
{
name: "within bounds",
req: ImageRequest{Size: Size{Width: 800, Height: 600}, FitMode: FitCover},
},
{
name: "original size and empty fit",
req: ImageRequest{Size: Size{Width: 0, Height: 0}},
},
{
name: "width over limit",
req: ImageRequest{Size: Size{Width: MaxDimension + 1, Height: 600}},
wantErr: ErrDimensionTooLarge,
},
{
name: "height over limit",
req: ImageRequest{Size: Size{Width: 800, Height: MaxDimension + 1}},
wantErr: ErrDimensionTooLarge,
},
{
name: "negative width",
req: ImageRequest{Size: Size{Width: -1, Height: 600}},
wantErr: ErrInvalidSize,
},
{
name: "invalid fit mode",
req: ImageRequest{Size: Size{Width: 800, Height: 600}, FitMode: "bogus"},
wantErr: ErrInvalidFitMode,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
err := ValidateImageRequest(&tt.req)
if tt.wantErr == nil {
if err != nil {
t.Fatalf("ValidateImageRequest() error = %v, want nil", err)
}
return
}
if !errors.Is(err, tt.wantErr) {
t.Fatalf("ValidateImageRequest() error = %v, want %v", err, tt.wantErr)
}
})
}
}
-27
View File
@@ -4,13 +4,11 @@ package middleware
import (
"log/slog"
"net/http"
"net/netip"
"time"
basicauth "github.com/99designs/basicauth-go"
"github.com/go-chi/chi/v5/middleware"
"github.com/go-chi/cors"
"github.com/go-chi/httprate"
metrics "github.com/slok/go-http-metrics/metrics/prometheus"
ghmm "github.com/slok/go-http-metrics/middleware"
"github.com/slok/go-http-metrics/middleware/std"
@@ -90,31 +88,6 @@ func (s *Middleware) ClientIP() func(http.Handler) http.Handler {
}
}
// RateLimit returns a middleware that limits each client to requestLimit
// requests per window and refuses a request over the limit with 429 Too Many
// Requests and a Retry-After header. Clients are told apart by the address
// the ClientIP middleware stored in the request context, so ClientIP must
// run first. An IPv6 client is counted by its /64, which one client usually
// holds whole; an IPv4-mapped address (::ffff:a.b.c.d) is counted as the
// IPv4 address it carries, since every such address falls in the same /64.
// Counts are kept only for the current and the previous window, so memory
// stays bounded.
func (s *Middleware) RateLimit(
requestLimit int, window time.Duration,
) func(http.Handler) http.Handler {
return httprate.LimitBy(requestLimit, window,
func(r *http.Request) (string, error) {
ip := clientip.FromContext(r.Context())
addr, err := netip.ParseAddr(ip)
if err == nil {
ip = addr.Unmap().String()
}
return httprate.CanonicalizeIP(ip), nil
})
}
type loggingResponseWriter struct {
http.ResponseWriter
@@ -1,280 +0,0 @@
package server
import (
"io"
"net/http"
"net/http/httptest"
"net/netip"
"net/url"
"path/filepath"
"regexp"
"strconv"
"strings"
"testing"
"go.uber.org/fx/fxtest"
"sneak.berlin/go/pixa/internal/config"
"sneak.berlin/go/pixa/internal/database"
"sneak.berlin/go/pixa/internal/globals"
"sneak.berlin/go/pixa/internal/handlers"
"sneak.berlin/go/pixa/internal/logger"
"sneak.berlin/go/pixa/internal/middleware"
)
// testSigningKey is a throwaway signing key; submitting it logs in.
const testSigningKey = "test-signing-key-0123456789abcdef"
// wrongKey is submitted for a failed login.
const wrongKey = "not-the-signing-key"
// Addresses for the login rate limit tests. The test server trusts
// 10.0.0.0/8 as its proxies, so the X-Forwarded-For sent by proxyPeer is
// believed and the one sent by firstClient or secondClient is ignored.
const (
firstClient = "198.51.100.1:40000"
secondClient = "198.51.100.2:40000"
proxyPeer = "10.0.0.1:40000"
firstForwarded = "203.0.113.1"
secondForwarded = "203.0.113.2"
)
// csrfFieldPattern extracts the CSRF token rendered into the login form.
var csrfFieldPattern = regexp.MustCompile(
`name="gorilla\.csrf\.Token" value="([^"]+)"`)
// newTestServer builds the server's real routes from the constructors
// cmd/pixad uses, with a throwaway state directory. Debug marks requests
// as plain HTTP, so the CSRF check runs without an https Referer.
func newTestServer(t *testing.T) *Server {
t.Helper()
stateDir := t.TempDir()
cfg := &config.Config{
Debug: true,
SigningKey: testSigningKey,
StateDir: stateDir,
DBURL: "file:" + filepath.Join(stateDir, "state.sqlite3"),
TrustedProxies: []netip.Prefix{netip.MustParsePrefix("10.0.0.0/8")},
}
lc := fxtest.NewLifecycle(t)
log, err := logger.New(lc, logger.Params{Globals: &globals.Globals{}})
if err != nil {
t.Fatalf("logger.New() error = %v", err)
}
db, err := database.New(lc, database.Params{Logger: log, Config: cfg})
if err != nil {
t.Fatalf("database.New() error = %v", err)
}
h, err := handlers.New(lc, handlers.Params{
Logger: log, Database: db, Config: cfg,
})
if err != nil {
t.Fatalf("handlers.New() error = %v", err)
}
mw, err := middleware.New(lc, middleware.Params{Logger: log, Config: cfg})
if err != nil {
t.Fatalf("middleware.New() error = %v", err)
}
lc.RequireStart()
t.Cleanup(lc.RequireStop)
s := &Server{config: cfg, mw: mw, h: h}
s.SetupRoutes()
return s
}
// clientRequest builds a request for / arriving from remoteAddr, carrying
// forwardedFor as its X-Forwarded-For header when that is not empty.
func clientRequest(
t *testing.T, method string, body io.Reader, remoteAddr, forwardedFor string,
) *http.Request {
t.Helper()
req := httptest.NewRequestWithContext(t.Context(), method, "/", body)
req.RemoteAddr = remoteAddr
if forwardedFor != "" {
req.Header.Set("X-Forwarded-For", forwardedFor)
}
return req
}
// postLogin loads the login form with GET / and submits key in it with
// POST /, as a browser does, both from the same client. GET / is not rate
// limited, so the form must load even for a client over the limit.
func postLogin(
t *testing.T, s *Server, remoteAddr, forwardedFor, key string,
) *httptest.ResponseRecorder {
t.Helper()
page := httptest.NewRecorder()
s.ServeHTTP(page,
clientRequest(t, http.MethodGet, nil, remoteAddr, forwardedFor))
if page.Code != http.StatusOK {
t.Fatalf("GET / status = %d, want %d", page.Code, http.StatusOK)
}
match := csrfFieldPattern.FindStringSubmatch(page.Body.String())
if match == nil {
t.Fatalf("no CSRF token field found in the login form")
}
form := url.Values{"key": {key}, "gorilla.csrf.Token": {match[1]}}
req := clientRequest(t, http.MethodPost,
strings.NewReader(form.Encode()), remoteAddr, forwardedFor)
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
for _, c := range page.Result().Cookies() {
req.AddCookie(c)
}
rec := httptest.NewRecorder()
s.ServeHTTP(rec, req)
return rec
}
// tripLoginRateLimit makes LoginAttemptsPerMinute failed logins from one
// client, each answered with the login form again, then one more, which
// must be refused with 429. It returns the response to that last attempt.
func tripLoginRateLimit(
t *testing.T, s *Server, remoteAddr, forwardedFor string,
) *httptest.ResponseRecorder {
t.Helper()
for attempt := range LoginAttemptsPerMinute {
rec := postLogin(t, s, remoteAddr, forwardedFor, wrongKey)
if rec.Code != http.StatusOK {
t.Fatalf("failed login %d status = %d, want %d",
attempt+1, rec.Code, http.StatusOK)
}
}
rec := postLogin(t, s, remoteAddr, forwardedFor, wrongKey)
if rec.Code != http.StatusTooManyRequests {
t.Fatalf("login over the limit status = %d, want %d",
rec.Code, http.StatusTooManyRequests)
}
return rec
}
// TestLoginRateLimitRefusesAttemptOverLimit verifies the login attempt
// after LoginAttemptsPerMinute failed ones from one client is refused with
// 429 and a Retry-After header, and that the client cannot get around the
// limit by sending X-Forwarded-For: from a peer that is not a trusted
// proxy, the header is ignored.
func TestLoginRateLimitRefusesAttemptOverLimit(t *testing.T) {
t.Parallel()
s := newTestServer(t)
rec := tripLoginRateLimit(t, s, firstClient, "")
retryAfter := rec.Header().Get("Retry-After")
seconds, err := strconv.Atoi(retryAfter)
if err != nil || seconds <= 0 {
t.Errorf("Retry-After = %q, want a positive number of seconds",
retryAfter)
}
rec = postLogin(t, s, firstClient, secondForwarded, wrongKey)
if rec.Code != http.StatusTooManyRequests {
t.Errorf("login with X-Forwarded-For from an untrusted peer "+
"status = %d, want %d", rec.Code, http.StatusTooManyRequests)
}
}
// TestLoginRateLimitLeavesOtherClientsAlone verifies one client going over
// the limit does not limit another: a failed login from a different
// address is answered with the login form, and the signing key still logs
// it in.
func TestLoginRateLimitLeavesOtherClientsAlone(t *testing.T) {
t.Parallel()
s := newTestServer(t)
tripLoginRateLimit(t, s, firstClient, "")
rec := postLogin(t, s, secondClient, "", wrongKey)
if rec.Code != http.StatusOK {
t.Errorf("failed login from another client status = %d, want %d",
rec.Code, http.StatusOK)
}
rec = postLogin(t, s, secondClient, "", testSigningKey)
if rec.Code != http.StatusSeeOther {
t.Errorf("login with the signing key from another client "+
"status = %d, want %d", rec.Code, http.StatusSeeOther)
}
}
// TestLoginRateLimitCountsClientsBehindProxySeparately verifies the limit
// counts the client address resolved from X-Forwarded-For, not the address
// of the trusted proxy the requests arrive from, so two clients behind the
// same proxy are counted separately.
func TestLoginRateLimitCountsClientsBehindProxySeparately(t *testing.T) {
t.Parallel()
s := newTestServer(t)
tripLoginRateLimit(t, s, proxyPeer, firstForwarded)
rec := postLogin(t, s, proxyPeer, secondForwarded, wrongKey)
if rec.Code != http.StatusOK {
t.Errorf("failed login from a second client behind the proxy "+
"status = %d, want %d", rec.Code, http.StatusOK)
}
}
// TestLoginRateLimitCountsIPv6ClientsByPrefix verifies an IPv6 client is
// counted by its /64: another address in the same /64 is refused too,
// while an address in a different /64 is not.
func TestLoginRateLimitCountsIPv6ClientsByPrefix(t *testing.T) {
t.Parallel()
s := newTestServer(t)
tripLoginRateLimit(t, s, proxyPeer, "2001:db8::1")
rec := postLogin(t, s, proxyPeer, "2001:db8::2", wrongKey)
if rec.Code != http.StatusTooManyRequests {
t.Errorf("login from the same /64 status = %d, want %d",
rec.Code, http.StatusTooManyRequests)
}
rec = postLogin(t, s, proxyPeer, "2001:db8:0:1::1", wrongKey)
if rec.Code != http.StatusOK {
t.Errorf("login from another /64 status = %d, want %d",
rec.Code, http.StatusOK)
}
}
// TestLoginRateLimitCountsIPv4MappedClientsSeparately verifies an IPv4
// client that the proxy forwards in IPv4-mapped IPv6 form (::ffff:a.b.c.d)
// is counted by its IPv4 address, not by the /64 that every such address
// shares, so two of them behind the proxy are counted separately.
func TestLoginRateLimitCountsIPv4MappedClientsSeparately(t *testing.T) {
t.Parallel()
s := newTestServer(t)
tripLoginRateLimit(t, s, proxyPeer, "::ffff:"+firstForwarded)
rec := postLogin(t, s, proxyPeer, "::ffff:"+secondForwarded, wrongKey)
if rec.Code != http.StatusOK {
t.Errorf("failed login from a second IPv4-mapped client "+
"status = %d, want %d", rec.Code, http.StatusOK)
}
}
+1 -9
View File
@@ -2,7 +2,6 @@ package server
import (
"net/http"
"time"
sentryhttp "github.com/getsentry/sentry-go/http"
"github.com/go-chi/chi/v5"
@@ -13,10 +12,6 @@ import (
"sneak.berlin/go/pixa/internal/static"
)
// LoginAttemptsPerMinute is how many login attempts (POST /) one client may
// make per minute; the next is refused with 429 Too Many Requests.
const LoginAttemptsPerMinute = 5
// SetupRoutes configures all HTTP routes.
func (s *Server) SetupRoutes() {
s.router = chi.NewRouter()
@@ -55,14 +50,11 @@ func (s *Server) SetupRoutes() {
// token cookie is independent of the session cookie, so it also
// covers the login POST, where no session exists yet. LimitBody caps
// the POST body ahead of CSRF, which reads its token from that body.
// The login POST is rate limited per client after both, so every
// attempt that reaches the signing key comparison is counted.
s.router.Group(func(r chi.Router) {
r.Use(s.h.LimitBody(handlers.MaxFormBytes))
r.Use(s.h.CSRF())
r.Get("/", s.h.HandleRoot())
r.With(s.mw.RateLimit(LoginAttemptsPerMinute, time.Minute)).
Post("/", s.h.HandleRoot())
r.Post("/", s.h.HandleRoot())
r.Post("/generate", s.h.HandleGenerateURL())
})
+11 -37
View File
@@ -41,12 +41,9 @@ func goldenVectors() []goldenVector {
Width: 800,
Height: 600,
Format: testFormatWebP,
Quality: 85,
FitMode: testFitCover,
},
// Signed data:
// "cdn.example.com:/photos/cat.jpg::800:600:webp:1704067200:85:cover"
wantSignature: "kdqeGoW2SX7qnaYtoB970wEnLydn0UnIgQYQLfAnjXQ=",
// Signed data: "cdn.example.com:/photos/cat.jpg::800:600:webp:1704067200"
wantSignature: "x5PfPp8QSDo0cJT96od-AEgrQyOVLfqifH5sst61_-w=",
wantSignedPath: testSignedPath,
},
{
@@ -58,12 +55,10 @@ func goldenVectors() []goldenVector {
Width: 800,
Height: 600,
Format: testFormatWebP,
Quality: 85,
FitMode: testFitCover,
},
// Signed data:
// "cdn.example.com:/photos/cat.jpg:token=abc&v=2:800:600:webp:1704067200:85:cover"
wantSignature: "pKgVBOTd_Q_EikI7MNQLC9Q8Hurdxzyv3EIYvVhqc2I=",
// "cdn.example.com:/photos/cat.jpg:token=abc&v=2:800:600:webp:1704067200"
wantSignature: "394_Vf9TdQFkpQ3XKFDQSyxgqKq8N7mApf2S4QaHqyo=",
wantSignedPath: "/v1/image/cdn.example.com/photos/cat.jpg" +
"%3Ftoken=abc&v=2/800x600.webp",
},
@@ -76,34 +71,11 @@ func goldenVectors() []goldenVector {
Width: 0,
Height: 0,
Format: testFormatPNG,
Quality: 85,
FitMode: testFitCover,
},
// Signed data:
// "cdn.example.com:/photos/cat.jpg::0:0:png:1704067200:85:cover"
wantSignature: "6_rZ0yyVbGZRs8kG7n7HLgLi5Jt8vjiWQljIEL1jbIs=",
// Signed data: "cdn.example.com:/photos/cat.jpg::0:0:png:1704067200"
wantSignature: "7Be7oteeQwvnSPU4bchyQ4ZGYGsAGBKpeEtuQ02ox60=",
wantSignedPath: "/v1/image/cdn.example.com/photos/cat.jpg/orig.png",
},
{
name: "non-default quality and fit",
req: signature.Request{
SourceHost: testHost,
SourcePath: testPath,
SourceQuery: "",
Width: 800,
Height: 600,
Format: testFormatWebP,
Quality: 40,
FitMode: testFitContain,
},
// Signed data:
// "cdn.example.com:/photos/cat.jpg::800:600:webp:1704067200:40:contain"
wantSignature: "pGaXpPUbI3A7nMx-4T9bfq9bYWBNL0kY4bxlcv3g1F8=",
// The path is the same as for the default quality and fit:
// q=40&fit=contain go in the query string next to sig and
// exp (imgcache.Service.GenerateSignedURL adds all four).
wantSignedPath: testSignedPath,
},
}
}
@@ -112,9 +84,11 @@ func goldenVectors() []goldenVector {
// hardcoded signing key.
//
// If any of these assertions fail, the signed byte format
// ("host:path:query:width:height:format:expiration:quality:fit"), the
// base64url encoding, or the signed URL layout has changed. Update these
// constants only when that change is intended.
// ("host:path:query:width:height:format:expiration"), the base64url
// encoding, or the signed URL layout has changed. Such a change breaks
// every signature already issued to clients, so it must be made
// deliberately: update these constants only as part of an intentional,
// documented signature format migration.
func TestSigner_GoldenVectors(t *testing.T) {
t.Parallel()
-68
View File
@@ -1,68 +0,0 @@
package signature_test
import (
"errors"
"testing"
"time"
"sneak.berlin/go/pixa/internal/signature"
)
// signedQualityFitRequest returns a request signed for quality 85 and fit
// mode "cover", the effective defaults the handler applies before
// verification.
func signedQualityFitRequest(signer *signature.Signer) *signature.Request {
req := &signature.Request{
SourceHost: testHost,
SourcePath: testPath,
Width: 800,
Height: 600,
Format: testFormatWebP,
Quality: 85,
FitMode: testFitCover,
Expires: time.Now().Add(1 * time.Hour),
}
req.Signature = signer.Sign(req)
return req
}
// TestSigner_Verify_QualityAndFitAreSigned proves that quality and fit are
// covered by the signature: a URL signed for one quality or fit mode must
// not verify when replayed with a different quality or fit mode. This is the
// amplification vector from the issue — one signed URL replayed across many
// quality and fit values yields many unauthorized cache entries and
// transcodes — so it must be rejected.
func TestSigner_Verify_QualityAndFitAreSigned(t *testing.T) {
t.Parallel()
signer := signature.New("test-secret-key")
cases := []struct {
name string
tamper func(r *signature.Request)
}{
{
name: "replayed with different quality",
tamper: func(r *signature.Request) { r.Quality = 40 },
},
{
name: "replayed with different fit mode",
tamper: func(r *signature.Request) { r.FitMode = testFitContain },
},
}
for _, tt := range cases {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
req := signedQualityFitRequest(signer)
tt.tamper(req)
err := signer.Verify(req)
if !errors.Is(err, signature.ErrInvalid) {
t.Errorf("Verify() = %v, want %v", err, signature.ErrInvalid)
}
})
}
}
+5 -20
View File
@@ -37,15 +37,6 @@ type Request struct {
Height int
// Format is the requested output format (e.g. "webp").
Format string
// Quality is the requested output quality (1-100) for lossy formats.
// It is the effective value the request resolves to: callers pass the
// default quality when the request omits the parameter, so an omitted
// quality signs identically to that same value stated explicitly.
Quality int
// FitMode is how the image is fit into the requested dimensions
// (e.g. "cover"). Like Quality it is the effective value: callers pass
// the default fit mode when the request omits the parameter.
FitMode string
// Signature is the HMAC signature to verify.
Signature string
// Expires is the signature expiration timestamp.
@@ -65,8 +56,7 @@ func New(secretKey string) *Signer {
}
// Sign generates an HMAC-SHA256 signature for the given request.
// The signature covers: host + path + query + width + height + format +
// expiration + quality + fit.
// The signature covers: host + path + query + width + height + format + expiration.
func (s *Signer) Sign(req *Request) string {
data := s.buildSignatureData(req)
mac := hmac.New(sha256.New, s.secretKey)
@@ -78,8 +68,7 @@ func (s *Signer) Sign(req *Request) string {
// Verify checks if the signature on the request is valid and not expired.
// Signatures are exact-match only: every component of the signed data
// (host, path, query, dimensions, format, expiration, quality, fit) must
// match exactly.
// (host, path, query, dimensions, format, expiration) must match exactly.
// No suffix matching, wildcard matching, or partial matching is supported.
// A signature for "cdn.example.com" will NOT verify for "example.com" or
// "other.cdn.example.com", and vice versa.
@@ -153,13 +142,11 @@ func (s *Signer) GenerateSignedURL(
}
// buildSignatureData creates the string to be signed.
// Format: "host:path:query:width:height:format:expiration:quality:fit"
// Format: "host:path:query:width:height:format:expiration"
// All components are used verbatim (exact match). No normalization,
// suffix matching, or wildcard expansion is performed. Quality and fit
// are the effective transform values, so replaying a signed URL with a
// different quality or fit mode fails verification.
// suffix matching, or wildcard expansion is performed.
func (s *Signer) buildSignatureData(req *Request) string {
return fmt.Sprintf("%s:%s:%s:%d:%d:%s:%d:%d:%s",
return fmt.Sprintf("%s:%s:%s:%d:%d:%s:%d",
req.SourceHost,
req.SourcePath,
req.SourceQuery,
@@ -167,8 +154,6 @@ func (s *Signer) buildSignatureData(req *Request) string {
req.Height,
req.Format,
req.Expires.Unix(),
req.Quality,
req.FitMode,
)
}
-2
View File
@@ -15,8 +15,6 @@ const (
testPath = "/photos/cat.jpg"
testFormatWebP = "webp"
testFormatPNG = "png"
testFitCover = "cover"
testFitContain = "contain"
testSignedPath = "/v1/image/cdn.example.com/photos/cat.jpg/800x600.webp"
testSig = "abc123"
)
+2 -2
View File
@@ -73,7 +73,7 @@
id="width"
name="width"
min="0"
max="8192"
max="10000"
value="{{if .FormWidth}}{{.FormWidth}}{{else}}0{{end}}"
placeholder="0 = original"
class="w-full px-3 py-2 border border-gray-300 rounded-md shadow-sm focus:outline-none focus:ring-2 focus:ring-blue-500 focus:border-blue-500"
@@ -88,7 +88,7 @@
id="height"
name="height"
min="0"
max="8192"
max="10000"
value="{{if .FormHeight}}{{.FormHeight}}{{else}}0{{end}}"
placeholder="0 = original"
class="w-full px-3 py-2 border border-gray-300 rounded-md shadow-sm focus:outline-none focus:ring-2 focus:ring-blue-500 focus:border-blue-500"
+58 -5
View File
@@ -3,14 +3,20 @@
# this repo. Idempotent: every install is guarded by a check so already
# installed tools are skipped. Base tooling comes from nix, apt, brew,
# or apk (detected in that order); assumes NOTHING is present (not git,
# make, or go). The linter is never installed on the host: golangci-lint
# runs only inside a container, Dockerfile.lint or the Dockerfile lint
# stage (see script/lint). CGO image libraries (pkg-config, vips,
# libheif) are installed for the govips bindings.
# make, or go). golangci-lint is packaged in nix, brew, and apk; on apt
# it is installed from a hash-verified GitHub release archive (never
# curl | sh). CGO image libraries (pkg-config, vips, libheif) are
# installed for the govips bindings.
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
# Pinned versions, 2026-08-07. Never "latest"; exact versions only.
GOLANGCI_LINT_VERSION="2.12.2"
# sha256 of golangci-lint-2.12.2-linux-<arch>.tar.gz release archives
GOLANGCI_LINT_SHA256_AMD64="8df580d2670fed8fa984aac0507099af8df275e665215f5c7a2ae3943893a553"
GOLANGCI_LINT_SHA256_ARM64="44cd40a8c76c86755375adfeea52cfd3533cb43d7bd647771e0ae065e166df3a"
PKGMGR=""
SUDO=""
@@ -51,6 +57,52 @@ missing() {
! command -v "$1" >/dev/null 2>&1
}
# verify_sha256 <file> <expected-hash>
verify_sha256() {
if command -v sha256sum >/dev/null 2>&1; then
actual="$(sha256sum "$1" | cut -d' ' -f1)"
else
actual="$(shasum -a 256 "$1" | cut -d' ' -f1)"
fi
if [ "$actual" != "$2" ]; then
echo "bootstrap: sha256 mismatch for $1" >&2
echo " expected: $2" >&2
echo " actual: $actual" >&2
exit 1
fi
}
# apt has no golangci-lint package: install a pinned release archive
# from GitHub, verified by hardcoded sha256 (never curl | sh).
install_golangci_lint_release() {
case "$(uname -m)" in
x86_64) goarch="amd64"; sha="$GOLANGCI_LINT_SHA256_AMD64" ;;
aarch64|arm64) goarch="arm64"; sha="$GOLANGCI_LINT_SHA256_ARM64" ;;
*)
echo "bootstrap: unsupported architecture $(uname -m)" >&2
exit 1
;;
esac
if missing curl; then pkg_install curl curl curl curl; fi
name="golangci-lint-${GOLANGCI_LINT_VERSION}-linux-${goarch}"
tmp="$(mktemp -d)"
curl -fsSL -o "$tmp/$name.tar.gz" \
"https://github.com/golangci/golangci-lint/releases/download/v${GOLANGCI_LINT_VERSION}/${name}.tar.gz"
verify_sha256 "$tmp/$name.tar.gz" "$sha"
tar -xzf "$tmp/$name.tar.gz" -C "$tmp"
$SUDO install -m 0755 "$tmp/$name/golangci-lint" /usr/local/bin/golangci-lint
rm -rf "$tmp"
}
ensure_golangci_lint() {
if ! missing golangci-lint; then return 0; fi
detect_pkgmgr
case "$PKGMGR" in
apt) install_golangci_lint_release ;;
*) pkg_install golangci-lint golangci-lint golangci-lint golangci-lint ;;
esac
}
# CGO dependencies for govips (image processing)
ensure_cgo_deps() {
if missing pkg-config; then
@@ -71,8 +123,9 @@ main() {
if missing git; then pkg_install git git git git; fi
if missing make; then pkg_install gnumake make make make; fi
# Go toolchain
# Go toolchain and linter
if missing go; then pkg_install go golang go go; fi
ensure_golangci_lint
# CGO image libraries
ensure_cgo_deps
+2 -4
View File
@@ -16,10 +16,8 @@ main() {
# removed on exit.
key="$(head -c 32 /dev/urandom | base64)"
# Docker 25 and later probe every 5 seconds during the image's
# 10-second start period. --health-interval=1s makes it probe every
# second after that instead of every 30, so a slow start still shows
# as healthy within the 30-second wait.
# --health-interval=1s overrides the image's 30s interval so the
# first probe does not use up the whole wait.
cid="$(docker create --health-interval=1s \
-e PIXA_SIGNING_KEY="$key" "$("$SCRIPT_DIR/projectname")")"
# Remove the container on any exit; turning signals into exit makes
+13 -27
View File
@@ -1,37 +1,23 @@
#!/bin/sh
# script/lint: run golangci-lint over the whole tree. This is the only
# way the linter is run, everywhere; it is never installed on the host.
#
# Inside a container it runs the linter. Anywhere else it builds
# Dockerfile.lint, whose last step runs this script again inside that
# container.
#
# Dockerfile.lint and the Dockerfile lint stage set container=docker
# (the systemd convention for marking a container) to say where we are.
# /.dockerenv cannot: it is missing inside build steps, and present on
# hosts that are themselves containers.
# script/lint: run the linter. CGO dependencies (pkg-config, vips,
# libheif) come from nix-shell when not already available (e.g. inside
# a Docker build or an existing nix-shell).
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
main() {
cd "$ROOT"
if [ "${container:-}" = docker ]; then
# `golangci-lint config verify` is not run: it fetches its JSON
# schema over an unpinned live HTTPS call, which REPO_POLICIES.md
# forbids.
echo "Running linter..."
golangci-lint run --config .golangci.yml ./...
run_with_cgo_deps() {
if command -v pkg-config >/dev/null 2>&1; then
sh -c "$1"
else
# A new CACHEBUST on every run means the lint step is never
# served from cache (see Dockerfile.lint). The cacheonly output
# leaves no image behind.
docker build \
--progress=plain \
--build-arg CACHEBUST="$(date +%s)-$$" \
--output=type=cacheonly \
-f Dockerfile.lint .
nix-shell -p pkg-config vips libheif golangci-lint git --run "$1"
fi
}
main() {
cd "$ROOT"
echo "Running linter..."
run_with_cgo_deps "golangci-lint run"
}
main "$@"
+1 -1
View File
@@ -10,7 +10,7 @@ run_with_cgo_deps() {
if command -v pkg-config >/dev/null 2>&1; then
sh -c "$1"
else
nix-shell -p pkg-config vips libheif git --run "$1"
nix-shell -p pkg-config vips libheif golangci-lint git --run "$1"
fi
}