12 Commits
Author SHA1 Message Date
sneak 05678eaae5 next -> main (1.0.0 milestone) (#118)
check / check (push) Successful in 12s
Reviewed-on: #118
2026-09-29 03:01:06 +02:00
clawbot be060a8305 Strip metadata from processed images (closes #82)
check / check (push) Successful in 13s
Every output is exported with govips' StripMetadata, so it carries no
EXIF (GPS, serial numbers, embedded thumbnails), XMP, IPTC or ICC
profile, the orig format included: it is always re-encoded, and pixa
never serves the source bytes. The image is turned upright with
AutoRotate right after decoding, so dropping the orientation tag does
not leave it rotated, and a requested size applies to the upright
image. An image with an ICC profile is converted to sRGB before export.
No setting turns this off. README.md documents it.

Model: opus-5-5
2026-09-29 02:18:25 +02:00
clawbot e410146fb6 Rate limit login attempts per client address (closes #66)
check / check (push) Successful in 12s
POST / had no limit, so the signing key could be guessed at no cost. It
is now limited to 5 attempts per minute per client by a new RateLimit
middleware on github.com/go-chi/httprate; an attempt over the limit gets
429 with Retry-After. It counts by the address the ClientIP middleware
resolved through trusted_proxies (an IPv4-mapped address as its IPv4
address, IPv6 by its /64) and runs after the body-size and CSRF checks,
so every attempt that reaches the key comparison is counted. README says
that with the default trusted_proxies a client with a private address
can choose its counted address, and how to close that.

Model: opus-5-5
2026-09-29 01:03:37 +02:00
clawbot 6010f5beb0 Refuse an unparseable exp with 400; log swallowed cache errors (closes #72)
check / check (push) Successful in 12s
An exp that was not a whole number, or empty, was ignored, so a URL for
a host that needs a signature got 401 as if it had no exp. It is now a
400 naming exp and the value, on every host; only an exp missing from
the URL is unchanged.

A failed variant .meta write, source metadata JSON write, Stats count
query, stats counter update, negative cache write or expired negative
cache delete was discarded without a trace. Each is now logged at warn
with the path or key and the error, and stays non-fatal, with tests for
those that can be made to fail. VariantStorage takes the cache's logger.

Model: opus-5-5
2026-09-28 19:59:39 +02:00
clawbot f149813c7e Refuse an empty fit on /v1/image/ with 400 (closes #139)
check / check (push) Successful in 13s
A fit in the URL with an empty value (fit=) was treated as missing, so
it was served as cover and verified against a signature made for cover.
It is now a 400 naming fit, the same rule the route applies to an empty
q. It is checked before the existing fit-mode check, which takes an
empty fit as missing; any other value still goes through that check
unchanged. Only a fit missing from the URL is cover.

Model: opus-5-5
2026-09-28 18:07:11 +02:00
clawbot 45869572ff Refuse a q outside 1-100 on /v1/image/ with 400 (closes #134)
check / check (push) Successful in 13s
A q that was not a number or was outside 1-100 was dropped and 85 used,
so q=500 was served and verified against a signature made for 85. It is
now a 400 naming q and the value, read with the generator's quality
check; only a q missing from the URL is 85.

The route also refuses with 400 a query string that cannot be decoded
(r.URL.Query() drops such a pair, so q=80% arrived as no q) and any
parameter given more than once, which was read from its first value only
(q=80&q=500 was served at 80).

Model: opus-5-5
2026-09-28 17:46:49 +02:00
clawbot 0f3700f7f5 Abort startup on an unknown PIXA_ environment variable (closes #133)
check / check (push) Successful in 11s
A variable whose name starts with PIXA_ but is neither a setting's
variable, from the list pairing each config key with its variable, nor
PIXA_CONFIG_PATH now aborts startup naming it, as an unknown config key
does. PIXA_PORT is named with a pointer to PORT. The check runs after
the config file loads, so the variables the file's env section sets are
checked too. README.md says so under Configuration.

Model: opus-5-5
2026-09-28 16:12:57 +02:00
clawbot 582ff66ba6 Describe what --health-interval does in docker-smoke on current Docker (closes #132)
check / check (push) Successful in 14s
The old comment said the flag stops the image's 30-second interval from
delaying the first probe past the wait. From Docker 25 on, the first
probe runs 5 seconds after start either way; the flag makes probes
after the 10-second start period come every second instead of every
30. Only the comment changes; TODO.md is left alone because the issue
limits the change to this script.

Model: opus-5-5
2026-09-28 15:48:53 +02:00
clawbot f8d40b89a7 Validate dimensions and fit mode on encrypted URLs (closes #62)
check / check (push) Successful in 3m6s
The encrypted /v1/e/ route used the decrypted payload unchecked, so a
token could request an over-limit size or an unknown fit mode; the
generator turned unparseable numbers into 0.

imgcache.ValidateDimension alone holds the MaxDimension bound and is
used by the path parser, by the new ValidateImageRequest (which adds
ValidateFitMode) and by the generator. Both image routes call
ValidateImageRequest, so each answers 400. The generator answers 400
naming the field for a width or height that is not a number or fails
that check, a quality that is not a number from 1 to 100, a ttl that is
not a number from 0 to the largest the expiry calculation can hold, or
an unknown fit. Empty quality is 85; empty ttl never expires. The
form's size inputs stop at 8192.

Model: opus-4-8 (implementation); opus-5-5 (rework)
2026-09-28 15:24:32 +02:00
clawbot 50123b2a6d Start on a fresh upaas volume and document running under upaas (closes #129)
check / check (push) Successful in 11s
upaas bind-mounts an existing host directory and sets no container
user, so a directory made with mkdir as root left pixad unable to
write /var/lib/pixa, and the container exited at startup.

The image now starts as root: deploy/docker-entrypoint.sh gives
/var/lib/pixa to pixad when pixad does not own it, then runs the
server as pixad through su-exec (alpine's package), so the server
never runs as root. README.md gains a "Running under upaas" section:
port, volume, environment variables, health check, first-run step.

Model: opus-5-5
2026-09-28 15:12:48 +02:00
clawbot 2f7365cc9b Run all linting in Docker through script/lint (closes #104)
check / check (push) Successful in 12s
make lint calls script/lint, the only way golangci-lint is run. Inside a
container it runs the linter; anywhere else it builds Dockerfile.lint,
whose last step runs script/lint again. Both Dockerfiles set
container=docker to mark the container, since /.dockerenv is missing in
build steps and present on hosts that are themselves containers. The
Dockerfile lint stage runs make lint.

A new CACHEBUST build-arg on every run keeps the lint step from being
served from cache; a tmpfs mount keeps Go's and golangci-lint's caches
out of that step's layer, so runs do not pile up build cache.
script/bootstrap and the nix-shell package lists no longer carry
golangci-lint. golangci-lint config verify is not run: it fetches its
schema over an unpinned live HTTPS call.

Model: opus-4-8 (implementation); opus-5-5 (rework)
2026-09-28 14:27:35 +02:00
clawbot 0f5bd51b09 Every setting can be given as an environment variable (closes #128)
check / check (push) Successful in 13s
Each config key can now be set by PIXA_ plus the key in upper case
("." written as "_"), and the port by PORT. A present variable, even
an empty one, is read before the config file through the existing
typed getters, so every existing check covers it; errors name the key
and the variable, never the signing key or metrics password. A
variable named in the file's env: section overrides both. An empty
string for blocked_networks or trusted_proxies is now an empty list.
The image no longer bakes in config.docker.yml or passes --config; its
HEALTHCHECK probes ${PORT:-8080}. The config file is looked for under
/etc/pixa rather than /etc/pixad. Also covers #99.

Model: opus-5-5
2026-09-28 13:46:56 +02:00
40 changed files with 2316 additions and 288 deletions
+16 -8
View File
@@ -1,4 +1,5 @@
# Lint stage # Lint stage
# Same image as Dockerfile.lint: change both pins together.
# golangci/golangci-lint:v2.12.2-alpine, 2026-08-07 # golangci/golangci-lint:v2.12.2-alpine, 2026-08-07
FROM golangci/golangci-lint:v2.12.2-alpine@sha256:91b27804074a0bacea298707f016911e60cf0cdbc6c7bf5ccacb5f0606d18d60 AS lint FROM golangci/golangci-lint:v2.12.2-alpine@sha256:91b27804074a0bacea298707f016911e60cf0cdbc6c7bf5ccacb5f0606d18d60 AS lint
@@ -13,6 +14,9 @@ RUN go mod download
# Copy source code # Copy source code
COPY . . COPY . .
# Tells script/lint it is inside a container, so it runs the linter.
ENV container=docker
# Run formatting check and linter # Run formatting check and linter
RUN make fmt-check RUN make fmt-check
RUN make lint RUN make lint
@@ -57,26 +61,30 @@ RUN apk add --no-cache \
vips \ vips \
libheif \ libheif \
ca-certificates \ ca-certificates \
tzdata tzdata \
su-exec
# Copy binary from builder # Copy binary from builder
COPY --from=builder /pixad /usr/local/bin/pixad COPY --from=builder /pixad /usr/local/bin/pixad
COPY deploy/docker-entrypoint.sh /usr/local/bin/docker-entrypoint.sh
# Create non-root user, config directory, and data directory # Create non-root user, config directory, and data directory
RUN adduser -D -H -s /sbin/nologin pixad && \ RUN adduser -D -H -s /sbin/nologin pixad && \
mkdir -p /var/lib/pixa /etc/pixa && \ mkdir -p /var/lib/pixa /etc/pixa && \
chown pixad:pixad /var/lib/pixa chown pixad:pixad /var/lib/pixa
# Copy the image config; signing_key comes from PIXA_SIGNING_KEY. # No USER: the entrypoint must start as root to give a bind-mounted
# Mount a file over /etc/pixa/config.yml to override anything else. # /var/lib/pixa to pixad; it then runs the server as pixad.
COPY config.docker.yml /etc/pixa/config.yml
USER pixad
WORKDIR /var/lib/pixa WORKDIR /var/lib/pixa
EXPOSE 8080 EXPOSE 8080
# Shell form so the probe follows PORT; a port set only in a mounted
# config file is not seen here.
HEALTHCHECK --interval=30s --timeout=5s --start-period=10s --retries=3 \ HEALTHCHECK --interval=30s --timeout=5s --start-period=10s --retries=3 \
CMD wget --spider -q http://localhost:8080/.well-known/healthcheck.json || exit 1 CMD wget --spider -q "http://localhost:${PORT:-8080}/.well-known/healthcheck.json" || exit 1
ENTRYPOINT ["/usr/local/bin/pixad", "--config", "/etc/pixa/config.yml"] # Settings come from PORT and the PIXA_ environment variables; only
# PIXA_SIGNING_KEY is required. A config file mounted at
# /etc/pixa/config.yml is optional and is read when present.
ENTRYPOINT ["/usr/local/bin/docker-entrypoint.sh"]
+34
View File
@@ -0,0 +1,34 @@
# Dockerfile.lint: the container script/lint builds to run golangci-lint,
# which is never installed on the host. Pinned to the same image as the
# Dockerfile lint stage: change both pins together, or the two run
# different linter versions.
#
# golangci/golangci-lint:v2.12.2-alpine, 2026-08-07
FROM golangci/golangci-lint:v2.12.2-alpine@sha256:91b27804074a0bacea298707f016911e60cf0cdbc6c7bf5ccacb5f0606d18d60
# pixa is CGO/libvips: the type-aware linters compile every package, so
# this image needs the same C libraries the build does.
RUN apk add --no-cache build-base vips-dev libheif-dev pkgconfig
WORKDIR /src
# Modules first for layer caching; go.mod/go.sum settle this layer's
# result, so it may safely be reused between runs.
COPY go.mod go.sum ./
RUN go mod download
COPY . .
# Tells script/lint it is inside a container, so it runs the linter.
ENV container=docker
# script/lint passes a different CACHEBUST on every run, and BuildKit
# keys every RUN after this ARG on its value, so the lint step always
# runs instead of returning a cached success that linted nothing.
#
# Go's and golangci-lint's caches (/root/.cache, hundreds of MB) go on a
# tmpfs that is discarded after the step. Written into the layer, they
# would pile up as build cache on every run, since no later run, with
# its new CACHEBUST, can reuse that layer.
ARG CACHEBUST
RUN --mount=type=tmpfs,target=/root/.cache script/lint
+1 -1
View File
@@ -10,7 +10,7 @@ ifdef HAS_PKGCONFIG
NIX_RUN_PREFIX = NIX_RUN_PREFIX =
NIX_RUN_SUFFIX = NIX_RUN_SUFFIX =
else else
NIX_RUN_PREFIX = nix-shell -p pkg-config vips libheif golangci-lint git --run ' NIX_RUN_PREFIX = nix-shell -p pkg-config vips libheif git --run '
NIX_RUN_SUFFIX = ' NIX_RUN_SUFFIX = '
endif endif
+108 -13
View File
@@ -27,12 +27,39 @@ make docker
docker run -p 8080:8080 -e PIXA_SIGNING_KEY="$(openssl rand -base64 32)" pixa:latest docker run -p 8080:8080 -e PIXA_SIGNING_KEY="$(openssl rand -base64 32)" pixa:latest
``` ```
A container is configured two ways. The signing key comes from the A container takes its settings from environment variables (see
`PIXA_SIGNING_KEY` environment variable, which the baked-in config Configuration below for the list). Only `PIXA_SIGNING_KEY` is required; if
reads; if it is unset the container exits at startup naming the it is unset the container exits at startup naming the variable. Everything
variable. Everything else uses built-in defaults, so to change any else has a built-in default. A config file mounted at `/etc/pixa/config.yml`
other setting mount your own file over `/etc/pixa/config.yml` (see is optional: it is read when present, and an environment variable wins over
`config.example.yml` for the full set of keys). the same setting in it.
## Running under upaas
What the [upaas](https://git.eeqj.de/sneak/upaas) app for pixa needs:
- **Port:** pixa listens on container port `8080`.
- **Volume:** container path `/var/lib/pixa`, where pixa keeps its
database and cache. upaas bind-mounts the host path it is given and
does not create it, so the host directory must exist before the first
deploy.
- **Environment variables:**
- `PIXA_SIGNING_KEY` (required): secret for signed and encrypted URLs
and login, 32+ characters, for example from
`openssl rand -base64 32`
- `PIXA_ALLOWLIST_HOSTS`: upstream hosts served without a signature,
comma-separated
- `PIXA_CACHE_MAX_BYTES`: disk cache limit in bytes; `0` disables it;
default 75% of free space
- the rest are in the table under Configuration below
- **Health check:** the image's `HEALTHCHECK` requests
`/.well-known/healthcheck.json`. upaas reads the container's health 60
seconds after a deploy and marks the deploy failed unless it is
`healthy`. The probe uses the port from `PORT` (default `8080`), so a
port changed only in a mounted config file is not seen by it: change
the port with `PORT`.
- **First run:** create the host directory. It may be owned by root: the
container gives it to its `pixad` user when it starts.
## Rationale ## Rationale
@@ -67,9 +94,38 @@ In-process caching of request-to-output mappings targets 1-5k r/s.
Images are only fetched from origins using TLS with valid certificates. Images are only fetched from origins using TLS with valid certificates.
A request whose query string cannot be decoded, or gives any parameter more
than once, is refused with 400.
- `<format>`: one of `orig`, `png`, `jpeg`, `webp` - `<format>`: one of `orig`, `png`, `jpeg`, `webp`
- `<size>`: `orig` or `<width>x<height>` (e.g. `800x600`) - `<size>`: `orig` or `<width>x<height>` (e.g. `800x600`)
The login form (`POST /`) is limited to 5 attempts per minute per client
address, counting an IPv6 client by its /64; an attempt over the limit is
refused with 429 and a `Retry-After` header. Behind a reverse proxy the client
address comes from `X-Forwarded-For` only when the proxy's address is in
`trusted_proxies`; otherwise all users behind the proxy are counted as one
client. With the default `trusted_proxies` (the RFC 1918 ranges), a client
with a private address can choose the address it is counted by through its own
`X-Forwarded-For`, whether it connects directly or through the proxy, because
its own address is trusted too. Setting `trusted_proxies` to the proxy's own
address closes this.
### Image Metadata
pixa decodes and re-encodes every image it serves, and removes all metadata from
the output: EXIF (GPS position, camera make, model and serial number, capture
time, embedded thumbnail), XMP, IPTC and the ICC colour profile. This cannot be
turned off.
- The `orig` format means the source's own format, not the source's bytes: an
`orig` image is re-encoded and stripped like any other.
- An image with an EXIF orientation is turned upright first, so it displays the
same without the tag; a requested size applies to the upright image.
- An image with an ICC profile is converted to sRGB first, since clients show an
image with no profile as sRGB. Colours outside sRGB, such as the most
saturated ones in a Display P3 photo, are clipped.
### Source Hosts ### Source Hosts
Source hosts may be allowlisted in the configuration. Non-allowlisted Source hosts may be allowlisted in the configuration. Non-allowlisted
@@ -97,11 +153,15 @@ Where:
- `width` — requested width in pixels, `0` for original - `width` — requested width in pixels, `0` for original
- `height` — requested height in pixels, `0` for original - `height` — requested height in pixels, `0` for original
- `format` — output format (jpeg, png, webp, avif, gif, orig) - `format` — output format (jpeg, png, webp, avif, gif, orig)
- `expiration` — Unix timestamp when signature expires - `expiration` — the URL's `exp` query parameter, the Unix timestamp when
- `quality` — the URL's `q` query parameter (1-100), or `85` when the URL the signature expires; a request whose `exp` is not a whole number, an
has no `q` empty `exp=` included, is refused with 400
- `quality` — the URL's `q` query parameter, a whole number from 1 to 100,
or `85` when the URL has no `q`; a request whose `q` is anything else is
refused with 400
- `fit` — the URL's `fit` query parameter (cover, contain, fill, inside, - `fit` — the URL's `fit` query parameter (cover, contain, fill, inside,
outside), or `cover` when the URL has no `fit` outside), or `cover` when the URL has no `fit`; a request whose `fit` is
anything else, an empty `fit=` included, is refused with 400
**Example:** resize `https://cdn.example.com/photos/cat.jpg` to 800x600 **Example:** resize `https://cdn.example.com/photos/cat.jpg` to 800x600
WebP with expiration 1704067200, default quality and fit: WebP with expiration 1704067200, default quality and fit:
@@ -125,7 +185,40 @@ For the same image at quality 40 with fit `contain`, the input ends in
### Configuration ### Configuration
Configured via YAML file (`--config`). Key settings: Every setting can be given as an environment variable, in a YAML config
file (`--config`), or both. A variable present in the environment wins over
the file, even when it is empty, and the file wins over the built-in
default. The one exception is a variable named in the file's `env:` section:
it is set while the file loads, so it overrides both the environment the
process was started with and the file's own key. A variable's value is
parsed as the same text in the file would be. The three lists take
comma-separated entries, with the spaces around each trimmed; an empty
variable is an empty list. A value that does not parse or is invalid aborts
startup, naming the variable. A variable whose name starts with `PIXA_` but
is not in the table below, such as a misspelled one or `PIXA_PORT`, aborts
startup naming it, as an unknown config key does. The one other accepted
name is `PIXA_CONFIG_PATH`, the config file's path (like `--config`). The
variables set by the file's `env:` section are checked the same way.
| Variable | Config key | Meaning |
| ------------------------------------ | ------------------------------- | ---------------------------------------------------------------------------- |
| `PIXA_SIGNING_KEY` | `signing_key` | Required: secret for signed and encrypted URLs and login, 32+ characters |
| `PORT` | `port` | Port to listen on; default `8080` |
| `PIXA_STATE_DIR` | `state_dir` | Directory for the database and the disk cache; default `/var/lib/pixa` |
| `PIXA_DB_URL` | `db_url` | SQLite database URL; default `state.sqlite3` in the state directory |
| `PIXA_CACHE_MAX_BYTES` | `cache_max_bytes` | Disk cache limit in bytes; `0` disables it; default 75% of free space |
| `PIXA_ALLOWLIST_HOSTS` | `allowlist_hosts` | Upstream hosts served without a signature |
| `PIXA_BLOCKED_NETWORKS` | `blocked_networks` | CIDR ranges never fetched from, on top of the built-in ones |
| `PIXA_TRUSTED_PROXIES` | `trusted_proxies` | CIDR ranges of proxies whose `X-Forwarded-For` is believed; default RFC 1918 |
| `PIXA_ALLOW_HTTP` | `allow_http` | Allow plain-HTTP upstreams, for testing only; default `false` |
| `PIXA_UPSTREAM_CONNECTIONS_PER_HOST` | `upstream_connections_per_host` | Concurrent connections per upstream host; default `20` |
| `PIXA_METRICS_USERNAME` | `metrics.username` | Username for `/metrics`, which is served only when both are set |
| `PIXA_METRICS_PASSWORD` | `metrics.password` | Password for `/metrics`; set together with the username |
| `PIXA_SENTRY_DSN` | `sentry_dsn` | Sentry DSN for error reporting; empty disables it |
| `PIXA_DEBUG` | `debug` | Debug logging and plain-HTTP local development; default `false` |
| `PIXA_MAINTENANCE_MODE` | `maintenance_mode` | Maintenance flag reported by the health check; default `false` |
Key settings in more detail:
- `access_control_allow_origin` — CORS origin - `access_control_allow_origin` — CORS origin
- `allowlist_hosts` — list of allowed upstream hosts - `allowlist_hosts` — list of allowed upstream hosts
@@ -138,7 +231,8 @@ Configured via YAML file (`--config`). Key settings:
inside one of these ranges; the logged and login-recorded client inside one of these ranges; the logged and login-recorded client
address is then the rightmost forwarded entry that is not itself a address is then the rightmost forwarded entry that is not itself a
trusted proxy. Otherwise the direct peer address is used and the header trusted proxy. Otherwise the direct peer address is used and the header
is ignored, so a client connecting directly cannot spoof its address. is ignored, so a client connecting directly from an address outside
these ranges cannot spoof its address.
An omitted key defaults to the RFC 1918 private ranges (`10.0.0.0/8`, An omitted key defaults to the RFC 1918 private ranges (`10.0.0.0/8`,
`172.16.0.0/12`, `192.168.0.0/16`), since pixa is deployed behind a `172.16.0.0/12`, `192.168.0.0/16`), since pixa is deployed behind a
proxy on a private network; an explicitly empty list (`[]`) trusts no proxy on a private network; an explicitly empty list (`[]`) trusts no
@@ -178,7 +272,8 @@ them. We provide:
(bootstrap, then install-precommit) (bootstrap, then install-precommit)
- `script/projectname` — output the project name ("pixa") - `script/projectname` — output the project name ("pixa")
- `script/test` — run the test suite - `script/test` — run the test suite
- `script/lint` — run golangci-lint - `script/lint` — run golangci-lint, always in a container (builds
`Dockerfile.lint` when run outside one)
- `script/fmt` — format all code (writes) - `script/fmt` — format all code (writes)
- `script/fmt-check` — check formatting (read-only) - `script/fmt-check` — check formatting (read-only)
- `script/check` — run test, lint, and fmt-check - `script/check` — run test, lint, and fmt-check
+92 -3
View File
@@ -30,6 +30,83 @@ exhaustion
# Completed Steps # Completed Steps
- 2026-09-28 strip metadata from processed images (closes #82): every output is
exported with govips' `StripMetadata`, so it carries no EXIF, XMP, IPTC or ICC
profile; the image is first turned upright with `AutoRotate` (before sizes are
worked out) and, when it has an ICC profile, converted to sRGB; the `orig`
format is re-encoded and stripped like any other, as pixa never serves the
source bytes; there is no setting to keep metadata; documented in `README.md`.
- 2026-09-28 rate limit the login form (closes #66): `POST /` is limited to 5
attempts per minute per client address, and an attempt over the limit is
refused with 429 and a `Retry-After` header; the address is the one
`internal/clientip` resolves through `trusted_proxies`, an IPv6 client is
counted by its /64, and an IPv4-mapped address as the IPv4 address it
carries; the limit is a `RateLimit` middleware in `internal/middleware` on
`github.com/go-chi/httprate`, which the image routes can reuse; the library
keeps counts for the current and the previous minute only; documented in
`README.md`.
- 2026-09-28 refuse an unparseable `exp` on `/v1/image/` and log swallowed
cache errors (closes #72): an `exp` in the URL that is not a whole
number, an empty `exp=` included, is a 400 naming `exp` and the value,
instead of being ignored and answered with 401 as if the URL had no
`exp`; only an `exp` missing from the URL is unchanged; `README.md` says
so where it documents `exp`. A failed variant `.meta` write, source
metadata JSON write, `Stats` count query, stats counter update, negative
cache write or expired negative cache delete is now logged at `warn`
with the path or key and the error, and stays non-fatal.
- 2026-09-28 refuse an empty `fit` on `/v1/image/` (closes #139): a
`fit` in the URL with an empty value (`fit=`) is a 400 naming `fit`,
instead of being served as `cover` and verified against a signature
made for `cover`; only a `fit` missing from the URL is still `cover`;
any other value still goes through the existing fit-mode check;
`README.md` says so where it documents `fit`.
- 2026-09-28 refuse an invalid `q` on `/v1/image/` (closes #134): a `q`
that is not a whole number from 1 to 100, an empty `q` included, is a
400 naming `q` and the value, instead of being served at the default
85; the route reads `q` with the generator's quality check
(`parseFormInt` with `minQuality` and `maxQuality`); only a `q` missing
from the URL is still 85; a query string that cannot be decoded, such
as `q=80%`, is a 400 showing it; any query parameter given more than
once (`q`, `fit`, `sig`, `exp` alike) is a 400 naming it, so none is
read from its first value only; `README.md` states the range and both
query-string rules.
- 2026-09-28 unknown `PIXA_` environment variables abort startup (closes
#133): a variable whose name starts with `PIXA_` but is neither a
setting's variable nor `PIXA_CONFIG_PATH` aborts startup naming it, as
an unknown config key does, and `PIXA_PORT` is named with a pointer to
`PORT`; the check runs after the config file loads, so the variables
the file's `env:` section sets are checked too; documented in
`README.md`.
- 2026-09-28 start on a fresh upaas volume (closes #129): the image
starts as root only to give `/var/lib/pixa` to `pixad` when `pixad`
does not own it (`deploy/docker-entrypoint.sh`), then runs the server
as `pixad` through `su-exec`, so a root-owned host directory
bind-mounted there no longer stops the container at startup;
`README.md` gains a "Running under upaas" section.
- 2026-09-28 run all linting in Docker via `Dockerfile.lint` +
`script/lint` (closes #104): `make lint` calls `script/lint`, the only
way the linter is run; inside a container (both Dockerfiles set
`container=docker`) it runs `golangci-lint`, anywhere else it builds the
hash-pinned `Dockerfile.lint`, whose last step runs `script/lint` again;
the `Dockerfile` lint stage runs `make lint`; no host or nix-shell
`golangci-lint` path remains (`script/bootstrap` installs no linter);
a per-run `CACHEBUST` build-arg keeps the lint step from being served
from cache, and a tmpfs mount on that step keeps Go's and
golangci-lint's caches out of its layer, so a run leaves no large build
cache behind; `golangci-lint config verify` stays out, as it fetches its
schema over an unpinned live HTTPS call
- 2026-09-28 every setting as an environment variable (closes #128, also
covers #99): each config key can be set by `PIXA_` plus the key in upper
case (`.` written as `_`), and the port by `PORT`; a variable present in
the environment, even empty, wins over the config file, which wins over
the default; the typed getters read the variable first, so every existing
check applies to it and a bad value aborts startup naming the variable;
lists are comma-separated, and an empty variable (or `""` in the file) is
an empty list; the Docker image no longer bakes in `config.docker.yml` or
passes `--config`, and its `HEALTHCHECK` probes `PORT` (default `8080`);
the config file is looked for under `/etc/pixa` and `~/.config/pixa`
instead of the daemon name `pixad`; documented in `README.md` and
`config.example.yml`.
- 2026-09-28 quality and fit in the URL signature (closes #60): the signed - 2026-09-28 quality and fit in the URL signature (closes #60): the signed
data is now `host:path:query:width:height:format:expiration:quality:fit`, data is now `host:path:query:width:height:format:expiration:quality:fit`,
using `85` and `cover` when the URL has no `q` or `fit`, so one signed using `85` and `cover` when the URL has no `q` or `fit`, so one signed
@@ -65,6 +142,20 @@ exhaustion
(IPv4-mapped forms covered); enforcement stays in the dial-time (IPv4-mapped forms covered); enforcement stays in the dial-time
re-resolution so the DNS-rebinding window remains closed; documented in re-resolution so the DNS-rebinding window remains closed; documented in
`README.md` and `config.example.yml`. `README.md` and `config.example.yml`.
- 2026-09-21 validate dimensions and fit mode on the encrypted-URL
route and the token generator (closes #62): `imgcache.ValidateDimension`
alone holds the `MaxDimension` bound and is used by the path parser, by
the new `ValidateImageRequest` (which also applies `ValidateFitMode`)
and by the generator; both the `/v1/image/` and `/v1/e/` routes call
`ValidateImageRequest`, so an over-limit size or an unknown fit mode is a
400 rather than an out-of-memory or a 500 from the processor; the URL
generator answers 400 naming the field for a `width` or `height` that is
not a number or fails the shared check, a `quality` that is not a number
from 1 to 100, a `ttl` that is not a number from 0 to the largest number
of seconds the expiry calculation can hold, or an unknown `fit`; an empty
`quality` is 85 and
an empty `ttl` never expires; the form's width and height inputs stop at
8192
- 2026-09-21 http.Server hardening (closes #92): added - 2026-09-21 http.Server hardening (closes #92): added
`HTTPReadHeaderTimeout` (10s, bounds the slowloris header dribble) and `HTTPReadHeaderTimeout` (10s, bounds the slowloris header dribble) and
`HTTPIdleTimeout` (120s, bounds keep-alive reuse) alongside the `HTTPIdleTimeout` (120s, bounds keep-alive reuse) alongside the
@@ -166,10 +257,9 @@ exhaustion
# Future Steps # Future Steps
- P1: strip EXIF and other metadata from processed images (privacy)
- P2: security - P2: security
- referer blacklist - referer blacklist
- per-IP rate limiting - per-IP rate limiting on the image routes
- per-origin rate limiting - per-origin rate limiting
- P2: HTTP response handling - P2: HTTP response handling
- Last-Modified headers - Last-Modified headers
@@ -178,7 +268,6 @@ exhaustion
- P2: auto format selection (format=auto based on Accept header) - P2: auto format selection (format=auto based on Accept header)
- P2: configuration - P2: configuration
- add all configuration options from README - add all configuration options from README
- environment variable overrides
- YAML config file support - YAML config file support
- P2: operational - P2: operational
- optional Sentry error reporting - optional Sentry error reporting
-11
View File
@@ -1,11 +0,0 @@
# Pixa configuration baked into the Docker image.
#
# The signing key is read from the PIXA_SIGNING_KEY environment
# variable; startup aborts naming it when it is unset. Every other key
# is omitted so its default applies. Operators who need more (an
# allowlist, metrics, and so on) mount their own file over
# /etc/pixa/config.yml.
signing_key: "${ENV:PIXA_SIGNING_KEY}"
state_dir: /var/lib/pixa
port: 8080
+9
View File
@@ -1,4 +1,13 @@
# Pixa Example Configuration # Pixa Example Configuration
#
# Every key can also be set by an environment variable, which wins over
# this file: PIXA_ plus the key in upper case, with "." written as "_"
# (state_dir is PIXA_STATE_DIR, metrics.username is
# PIXA_METRICS_USERNAME). The one exception is port, which is set by
# PORT. In a variable, a list is comma-separated. A variable named in
# this file's env: section is set while the file loads, so it overrides
# both the environment the process was started with and this file's own
# key.
# Server settings # Server settings
port: 8080 port: 8080
+15
View File
@@ -0,0 +1,15 @@
#!/bin/sh
# deploy/docker-entrypoint.sh: the Docker image's ENTRYPOINT. It runs as
# root only to give /var/lib/pixa to pixad: a host directory
# bind-mounted there keeps its host owner, often root, and pixad could
# not write to it. The server itself always runs as pixad.
set -eu
main() {
if [ "$(stat -c %U /var/lib/pixa)" != pixad ]; then
chown pixad:pixad /var/lib/pixa
fi
exec su-exec pixad /usr/local/bin/pixad "$@"
}
main "$@"
+3
View File
@@ -11,6 +11,7 @@ require (
github.com/getsentry/sentry-go v0.40.0 github.com/getsentry/sentry-go v0.40.0
github.com/go-chi/chi/v5 v5.2.3 github.com/go-chi/chi/v5 v5.2.3
github.com/go-chi/cors v1.2.2 github.com/go-chi/cors v1.2.2
github.com/go-chi/httprate v0.16.0
github.com/gorilla/csrf v1.7.3 github.com/gorilla/csrf v1.7.3
github.com/gorilla/securecookie v1.1.2 github.com/gorilla/securecookie v1.1.2
github.com/prometheus/client_golang v1.23.2 github.com/prometheus/client_golang v1.23.2
@@ -91,6 +92,7 @@ require (
github.com/inconshreveable/mousetrap v1.1.0 // indirect github.com/inconshreveable/mousetrap v1.1.0 // indirect
github.com/josharian/intern v1.0.0 // indirect github.com/josharian/intern v1.0.0 // indirect
github.com/json-iterator/go v1.1.12 // indirect github.com/json-iterator/go v1.1.12 // indirect
github.com/klauspost/cpuid/v2 v2.2.10 // indirect
github.com/kylelemons/godebug v1.1.0 // indirect github.com/kylelemons/godebug v1.1.0 // indirect
github.com/mailru/easyjson v0.7.7 // indirect github.com/mailru/easyjson v0.7.7 // indirect
github.com/mattn/go-colorable v0.1.13 // indirect github.com/mattn/go-colorable v0.1.13 // indirect
@@ -113,6 +115,7 @@ require (
github.com/tidwall/match v1.1.1 // indirect github.com/tidwall/match v1.1.1 // indirect
github.com/tidwall/pretty v1.2.0 // indirect github.com/tidwall/pretty v1.2.0 // indirect
github.com/x448/float16 v0.8.4 // indirect github.com/x448/float16 v0.8.4 // indirect
github.com/zeebo/xxh3 v1.0.2 // indirect
go.etcd.io/etcd/api/v3 v3.6.2 // indirect go.etcd.io/etcd/api/v3 v3.6.2 // indirect
go.etcd.io/etcd/client/pkg/v3 v3.6.2 // indirect go.etcd.io/etcd/client/pkg/v3 v3.6.2 // indirect
go.etcd.io/etcd/client/v3 v3.6.2 // indirect go.etcd.io/etcd/client/v3 v3.6.2 // indirect
+8
View File
@@ -114,6 +114,8 @@ github.com/go-chi/chi/v5 v5.2.3 h1:WQIt9uxdsAbgIYgid+BpYc+liqQZGMHRaUwp0JUcvdE=
github.com/go-chi/chi/v5 v5.2.3/go.mod h1:L2yAIGWB3H+phAw1NxKwWM+7eUH/lU8pOMm5hHcoops= github.com/go-chi/chi/v5 v5.2.3/go.mod h1:L2yAIGWB3H+phAw1NxKwWM+7eUH/lU8pOMm5hHcoops=
github.com/go-chi/cors v1.2.2 h1:Jmey33TE+b+rB7fT8MUy1u0I4L+NARQlK6LhzKPSyQE= github.com/go-chi/cors v1.2.2 h1:Jmey33TE+b+rB7fT8MUy1u0I4L+NARQlK6LhzKPSyQE=
github.com/go-chi/cors v1.2.2/go.mod h1:sSbTewc+6wYHBBCW7ytsFSn836hqM7JxpglAy2Vzc58= github.com/go-chi/cors v1.2.2/go.mod h1:sSbTewc+6wYHBBCW7ytsFSn836hqM7JxpglAy2Vzc58=
github.com/go-chi/httprate v0.16.0 h1:8V5DH9j6pSK6UQoBsTpvMyFxycqaKEIToyPKzHJjUa8=
github.com/go-chi/httprate v0.16.0/go.mod h1:A8lo+qRhk+s9LiuP5saS7XCGDXRXMcrueq0NfIuCa/I=
github.com/go-errors/errors v1.4.2 h1:J6MZopCL4uSllY1OfXM374weqZFFItUbrImctkmUxIA= github.com/go-errors/errors v1.4.2 h1:J6MZopCL4uSllY1OfXM374weqZFFItUbrImctkmUxIA=
github.com/go-errors/errors v1.4.2/go.mod h1:sIVyrIiJhuEF+Pj9Ebtd6P/rEYROXFi3BopGUQ5a5Og= github.com/go-errors/errors v1.4.2/go.mod h1:sIVyrIiJhuEF+Pj9Ebtd6P/rEYROXFi3BopGUQ5a5Og=
github.com/go-jose/go-jose/v4 v4.0.5 h1:M6T8+mKZl/+fNNuFHvGIzDz7BTLQPIounk/b9dw3AaE= github.com/go-jose/go-jose/v4 v4.0.5 h1:M6T8+mKZl/+fNNuFHvGIzDz7BTLQPIounk/b9dw3AaE=
@@ -251,6 +253,8 @@ github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI
github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck= github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
github.com/klauspost/compress v1.18.0 h1:c/Cqfb0r+Yi+JtIEq73FWXVkRonBlf0CRNYc8Zttxdo= github.com/klauspost/compress v1.18.0 h1:c/Cqfb0r+Yi+JtIEq73FWXVkRonBlf0CRNYc8Zttxdo=
github.com/klauspost/compress v1.18.0/go.mod h1:2Pp+KzxcywXVXMr50+X0Q/Lsb43OQHYWRCY2AiWywWQ= github.com/klauspost/compress v1.18.0/go.mod h1:2Pp+KzxcywXVXMr50+X0Q/Lsb43OQHYWRCY2AiWywWQ=
github.com/klauspost/cpuid/v2 v2.2.10 h1:tBs3QSyvjDyFTq3uoc/9xFpCuOsJQFNPiAhYdw2skhE=
github.com/klauspost/cpuid/v2 v2.2.10/go.mod h1:hqwkgyIinND0mEev00jJYCxPNVRVXFQeu1XKlok6oO0=
github.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ= github.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
github.com/kr/logfmt v0.0.0-20140226030751-b84e30acd515/go.mod h1:+0opPa2QZZtGFBFZlji/RkVcI2GknAs/DXo4wKdlNEc= github.com/kr/logfmt v0.0.0-20140226030751-b84e30acd515/go.mod h1:+0opPa2QZZtGFBFZlji/RkVcI2GknAs/DXo4wKdlNEc=
github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo= github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
@@ -396,6 +400,10 @@ github.com/x448/float16 v0.8.4/go.mod h1:14CWIYCyZA/cWjXOioeEpHeN/83MdbZDRQHoFcY
github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY= github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=
github.com/zeebo/assert v1.3.0 h1:g7C04CbJuIDKNPFHmsk4hwZDO5O+kntRxzaUoNXj+IQ=
github.com/zeebo/assert v1.3.0/go.mod h1:Pq9JiuJQpG8JLJdtkwrJESF0Foym2/D9XMU5ciN/wJ0=
github.com/zeebo/xxh3 v1.0.2 h1:xZmwmqxHZA8AI603jOQ0tMqmBr9lPeFwGg6d+xy9DC0=
github.com/zeebo/xxh3 v1.0.2/go.mod h1:5NWz9Sef7zIDm2JHfFlcQvNekmcEl9ekUZQQKCYaDcA=
go.etcd.io/etcd/api/v3 v3.6.2 h1:25aCkIMjUmiiOtnBIp6PhNj4KdcURuBak0hU2P1fgRc= go.etcd.io/etcd/api/v3 v3.6.2 h1:25aCkIMjUmiiOtnBIp6PhNj4KdcURuBak0hU2P1fgRc=
go.etcd.io/etcd/api/v3 v3.6.2/go.mod h1:eFhhvfR8Px1P6SEuLT600v+vrhdDTdcfMzmnxVXXSbk= go.etcd.io/etcd/api/v3 v3.6.2/go.mod h1:eFhhvfR8Px1P6SEuLT600v+vrhdDTdcfMzmnxVXXSbk=
go.etcd.io/etcd/client/pkg/v3 v3.6.2 h1:zw+HRghi/G8fKpgKdOcEKpnBTE4OO39T6MegA0RopVU= go.etcd.io/etcd/client/pkg/v3 v3.6.2 h1:zw+HRghi/G8fKpgKdOcEKpnBTE4OO39T6MegA0RopVU=
+174 -101
View File
@@ -16,7 +16,6 @@ import (
"git.eeqj.de/sneak/smartconfig" "git.eeqj.de/sneak/smartconfig"
"go.uber.org/fx" "go.uber.org/fx"
"sneak.berlin/go/pixa/internal/globals"
"sneak.berlin/go/pixa/internal/logger" "sneak.berlin/go/pixa/internal/logger"
) )
@@ -59,6 +58,7 @@ var (
errValueRequired = errors.New("a value is required") errValueRequired = errors.New("a value is required")
errValueEmpty = errors.New("value must not be empty") errValueEmpty = errors.New("value must not be empty")
errUnknownConfigKeys = errors.New("unknown config keys") errUnknownConfigKeys = errors.New("unknown config keys")
errUnknownEnvVars = errors.New("unknown environment variables")
errNotAString = errors.New("not a string") errNotAString = errors.New("not a string")
errNotAnInteger = errors.New("not an integer") errNotAnInteger = errors.New("not an integer")
errNotABoolean = errors.New("not a boolean") errNotABoolean = errors.New("not a boolean")
@@ -92,8 +92,7 @@ var (
type Params struct { type Params struct {
fx.In fx.In
Globals *globals.Globals Logger *logger.Logger
Logger *logger.Logger
} }
// Config holds application configuration values. // Config holds application configuration values.
@@ -137,24 +136,34 @@ type Config struct {
CacheMaxBytes int64 CacheMaxBytes int64
// cacheMaxBytesExplicit records whether cache_max_bytes was // cacheMaxBytesExplicit records whether cache_max_bytes was
// explicitly set in the configuration file. Explicit values are // explicitly set, in the environment or the configuration file.
// used exactly as given; only an omitted key gets the computed // Explicit values are used exactly as given; only an omitted key
// default (and its floor) in resolveCacheMaxBytes. // gets the computed default (and its floor) in resolveCacheMaxBytes.
cacheMaxBytesExplicit bool cacheMaxBytesExplicit bool
} }
// New creates a new Config instance by loading configuration from file. // New creates a new Config instance from the environment and the
// config file.
func New(_ fx.Lifecycle, params Params) (*Config, error) { func New(_ fx.Lifecycle, params Params) (*Config, error) {
log := params.Logger.Get() log := params.Logger.Get()
name := params.Globals.Appname
sc, err := loadConfigFile(log, name) // Look for the config file under the project name (/etc/pixa/,
// ~/.config/pixa/), matching the /var/lib/pixa state directory,
// not under the daemon name pixad.
sc, err := loadConfigFile(log, "pixa")
if err != nil {
return nil, err
}
// Loading the config file sets the variables in its env section,
// so this also checks their names.
err = validateKnownEnvVars()
if err != nil { if err != nil {
return nil, err return nil, err
} }
if sc == nil { if sc == nil {
log.Info("no config file found, using defaults") log.Info("no config file found, using environment variables and defaults")
} }
c, err := newFromSmartConfig(sc) c, err := newFromSmartConfig(sc)
@@ -179,22 +188,23 @@ func New(_ fx.Lifecycle, params Params) (*Config, error) {
return c, nil return c, nil
} }
// newFromSmartConfig constructs a Config from a loaded smartconfig // newFromSmartConfig constructs a Config from the environment and a
// instance and validates it. A nil sc means no config file was found, // loaded smartconfig instance, and validates it. A nil sc means no
// in which case every option takes its default value. A key that is // config file was found, in which case every option the environment
// present but unparseable or invalid is an error: defaults apply only // does not set takes its default value. A key that is present but
// to omitted keys, never to invalid explicit values. // unparseable or invalid is an error: defaults apply only to omitted
// keys, never to invalid explicit values.
func newFromSmartConfig(sc *smartconfig.Config) (*Config, error) { func newFromSmartConfig(sc *smartconfig.Config) (*Config, error) {
if sc != nil { if sc != nil {
err := validateKnownKeys(sc) err := validateKnownKeys(sc)
if err != nil { if err != nil {
return nil, err return nil, err
} }
}
err = validateAllowlistHostsValue(sc) err := validateAllowlistHostsValue(sc)
if err != nil { if err != nil {
return nil, err return nil, err
}
} }
blockedNetworks, err := parseCIDRList(sc, keyBlockedNetworks) blockedNetworks, err := parseCIDRList(sc, keyBlockedNetworks)
@@ -238,10 +248,8 @@ func newFromSmartConfig(sc *smartconfig.Config) (*Config, error) {
// The computed default for cache_max_bytes needs a validated // The computed default for cache_max_bytes needs a validated
// state_dir, so it is resolved later (resolveCacheMaxBytes); here // state_dir, so it is resolved later (resolveCacheMaxBytes); here
// we only record whether the operator set the key explicitly. // we only record whether the operator set the key explicitly.
if sc != nil { if _, present := lookupValue(sc, keyCacheMaxBytes); present {
if _, present := sc.Get(keyCacheMaxBytes); present { c.cacheMaxBytesExplicit = true
c.cacheMaxBytesExplicit = true
}
} }
// Build DBURL from StateDir if not explicitly set. The derived URL // Build DBURL from StateDir if not explicitly set. The derived URL
@@ -249,12 +257,9 @@ func newFromSmartConfig(sc *smartconfig.Config) (*Config, error) {
// explicitly empty value. // explicitly empty value.
c.DBURL = loader.stringVal(keyDBURL, "") c.DBURL = loader.stringVal(keyDBURL, "")
if c.DBURL == "" && loader.err == nil { if c.DBURL == "" && loader.err == nil {
if sc != nil { if _, present := lookupValue(sc, keyDBURL); present {
if _, present := sc.Get(keyDBURL); present { return nil, fmt.Errorf("%s: %w; omit it to derive it from state_dir",
return nil, fmt.Errorf( settingName(keyDBURL), errValueEmpty)
"config key %q: %w; omit the key to derive it from state_dir",
keyDBURL, errValueEmpty)
}
} }
c.DBURL = fmt.Sprintf("file:%s/state.sqlite3?_journal_mode=WAL", c.StateDir) c.DBURL = fmt.Sprintf("file:%s/state.sqlite3?_journal_mode=WAL", c.StateDir)
@@ -356,6 +361,91 @@ func isKnownConfigKey(key string) bool {
return false return false
} }
// envVarNames returns, for each configuration key, the environment
// variable that also sets it: PIXA_ plus the key in upper case, with "."
// written as "_", except the port, which REPO_POLICIES.md requires to be
// PORT. metrics is set through its two subkeys; env has no variable.
func envVarNames() map[string]string {
return map[string]string{ //nolint:gosec // G101: variable names, not secrets
keyDebug: "PIXA_DEBUG",
keyMaintenanceMode: "PIXA_MAINTENANCE_MODE",
keyPort: "PORT",
keyStateDir: "PIXA_STATE_DIR",
keySentryDSN: "PIXA_SENTRY_DSN",
keyDBURL: "PIXA_DB_URL",
keyMetricsUsername: "PIXA_METRICS_USERNAME",
keyMetricsPassword: "PIXA_METRICS_PASSWORD",
keySigningKey: "PIXA_SIGNING_KEY",
keyAllowlistHosts: "PIXA_ALLOWLIST_HOSTS",
keyAllowHTTP: "PIXA_ALLOW_HTTP",
keyUpstreamConnectionsPerHost: "PIXA_UPSTREAM_CONNECTIONS_PER_HOST",
keyCacheMaxBytes: "PIXA_CACHE_MAX_BYTES",
keyBlockedNetworks: "PIXA_BLOCKED_NETWORKS",
keyTrustedProxies: "PIXA_TRUSTED_PROXIES",
}
}
// validateKnownEnvVars rejects environment variables whose names start
// with PIXA_ but that are neither a setting's variable nor
// PIXA_CONFIG_PATH, so a misspelled variable fails at startup instead of
// being silently ignored, as validateKnownKeys does for config file keys.
// New calls it after loading the config file, so the variables the
// file's env section sets are checked too.
func validateKnownEnvVars() error {
known := map[string]bool{"PIXA_CONFIG_PATH": true}
for _, name := range envVarNames() {
known[name] = true
}
var unknown []string
for _, entry := range os.Environ() {
name, _, _ := strings.Cut(entry, "=")
switch {
case !strings.HasPrefix(name, "PIXA_") || known[name]:
continue
case name == "PIXA_PORT":
unknown = append(unknown, name+" (use PORT for the port)")
default:
unknown = append(unknown, name)
}
}
if len(unknown) > 0 {
sort.Strings(unknown)
return fmt.Errorf("%w: %s", errUnknownEnvVars, strings.Join(unknown, ", "))
}
return nil
}
// lookupValue returns the value set for key and whether one is set. The
// key's environment variable wins when it is present, even when empty;
// its value is a string, read exactly as the same text quoted in the
// config file would be. Otherwise the config file's value is used.
func lookupValue(sc *smartconfig.Config, key string) (any, bool) {
value, present := os.LookupEnv(envVarNames()[key])
if present {
return value, true
}
if sc == nil {
return nil, false
}
return sc.Get(key)
}
// settingName names key in an error message together with its
// environment variable, since either one may have set the value.
func settingName(key string) string {
return fmt.Sprintf("config key %q (environment variable %s)",
key, envVarNames()[key])
}
// ensureStateDirWritable verifies at startup that StateDir can be // ensureStateDirWritable verifies at startup that StateDir can be
// created and written to, so a misconfigured path aborts startup // created and written to, so a misconfigured path aborts startup
// instead of failing later at first use. // instead of failing later at first use.
@@ -364,28 +454,28 @@ func (c *Config) ensureStateDirWritable() error {
err := os.MkdirAll(c.StateDir, stateDirPerms) err := os.MkdirAll(c.StateDir, stateDirPerms)
if err != nil { if err != nil {
return fmt.Errorf("config key %q: cannot create directory %q: %w", return fmt.Errorf("%s: cannot create directory %q: %w",
keyStateDir, c.StateDir, err) settingName(keyStateDir), c.StateDir, err)
} }
probe, err := os.CreateTemp(c.StateDir, ".startup-write-probe-*") probe, err := os.CreateTemp(c.StateDir, ".startup-write-probe-*")
if err != nil { if err != nil {
return fmt.Errorf("config key %q: directory %q is not writable: %w", return fmt.Errorf("%s: directory %q is not writable: %w",
keyStateDir, c.StateDir, err) settingName(keyStateDir), c.StateDir, err)
} }
probePath := probe.Name() probePath := probe.Name()
err = probe.Close() err = probe.Close()
if err != nil { if err != nil {
return fmt.Errorf("config key %q: cannot close probe file %q: %w", return fmt.Errorf("%s: cannot close probe file %q: %w",
keyStateDir, probePath, err) settingName(keyStateDir), probePath, err)
} }
err = os.Remove(probePath) err = os.Remove(probePath)
if err != nil { if err != nil {
return fmt.Errorf("config key %q: cannot remove probe file %q: %w", return fmt.Errorf("%s: cannot remove probe file %q: %w",
keyStateDir, probePath, err) settingName(keyStateDir), probePath, err)
} }
return nil return nil
@@ -396,18 +486,19 @@ func (c *Config) ensureStateDirWritable() error {
// key value itself is never echoed in error messages. // key value itself is never echoed in error messages.
func (c *Config) validateSigningKey() error { func (c *Config) validateSigningKey() error {
if c.SigningKey == "" { if c.SigningKey == "" {
return fmt.Errorf("config key %q: %w", keySigningKey, errValueRequired) return fmt.Errorf("%s: %w", settingName(keySigningKey), errValueRequired)
} }
// Minimum key length for security (32 bytes = 256 bits) // Minimum key length for security (32 bytes = 256 bits)
const minKeyLength = 32 const minKeyLength = 32
if len(c.SigningKey) < minKeyLength { if len(c.SigningKey) < minKeyLength {
return fmt.Errorf("config key %q: %w: must be at least %d characters, got %d", return fmt.Errorf("%s: %w: must be at least %d characters, got %d",
keySigningKey, errValueTooShort, minKeyLength, len(c.SigningKey)) settingName(keySigningKey), errValueTooShort, minKeyLength,
len(c.SigningKey))
} }
if c.SigningKey == placeholderSigningKey { if c.SigningKey == placeholderSigningKey {
return fmt.Errorf("config key %q: %w", keySigningKey, errPlaceholderKey) return fmt.Errorf("%s: %w", settingName(keySigningKey), errPlaceholderKey)
} }
return nil return nil
@@ -423,25 +514,25 @@ func (c *Config) validate() error {
const maxPort = 65535 const maxPort = 65535
if c.Port < 1 || c.Port > maxPort { if c.Port < 1 || c.Port > maxPort {
return fmt.Errorf("config key %q: value %d is %w 1-%d", return fmt.Errorf("%s: value %d is %w 1-%d",
keyPort, c.Port, errPortOutOfRange, maxPort) settingName(keyPort), c.Port, errPortOutOfRange, maxPort)
} }
if c.UpstreamConnectionsPerHost < 1 { if c.UpstreamConnectionsPerHost < 1 {
return fmt.Errorf("config key %q: value %d %w", return fmt.Errorf("%s: value %d %w",
keyUpstreamConnectionsPerHost, c.UpstreamConnectionsPerHost, settingName(keyUpstreamConnectionsPerHost),
errTooFewConnections) c.UpstreamConnectionsPerHost, errTooFewConnections)
} }
if c.StateDir == "" { if c.StateDir == "" {
return fmt.Errorf("config key %q: %w", keyStateDir, errValueEmpty) return fmt.Errorf("%s: %w", settingName(keyStateDir), errValueEmpty)
} }
// Zero is valid (it disables the disk cache); only negative // Zero is valid (it disables the disk cache); only negative
// values are rejected. No floor applies to explicit values. // values are rejected. No floor applies to explicit values.
if c.CacheMaxBytes < 0 { if c.CacheMaxBytes < 0 {
return fmt.Errorf("config key %q: value %d %w", return fmt.Errorf("%s: value %d %w",
keyCacheMaxBytes, c.CacheMaxBytes, errMustNotBeNegative) settingName(keyCacheMaxBytes), c.CacheMaxBytes, errMustNotBeNegative)
} }
for _, host := range c.AllowlistHosts { for _, host := range c.AllowlistHosts {
@@ -454,14 +545,15 @@ func (c *Config) validate() error {
if c.SentryDSN != "" { if c.SentryDSN != "" {
parsed, err := url.Parse(c.SentryDSN) parsed, err := url.Parse(c.SentryDSN)
if err != nil || parsed.Scheme == "" || parsed.Host == "" { if err != nil || parsed.Scheme == "" || parsed.Host == "" {
return fmt.Errorf("config key %q: value %q is %w", return fmt.Errorf("%s: value %q is %w",
keySentryDSN, c.SentryDSN, errNotAValidURL) settingName(keySentryDSN), c.SentryDSN, errNotAValidURL)
} }
} }
if (c.MetricsUsername == "") != (c.MetricsPassword == "") { if (c.MetricsUsername == "") != (c.MetricsPassword == "") {
return fmt.Errorf("config keys %q and %q %w", return fmt.Errorf("%s and %s %w",
keyMetricsUsername, keyMetricsPassword, errMustBeSetTogether) settingName(keyMetricsUsername), settingName(keyMetricsPassword),
errMustBeSetTogether)
} }
return nil return nil
@@ -476,13 +568,13 @@ func (c *Config) validate() error {
// disable URL signing. // disable URL signing.
func validateAllowlistHost(host string) error { func validateAllowlistHost(host string) error {
if strings.Contains(host, "://") || strings.ContainsAny(host, "/ \t") { if strings.Contains(host, "://") || strings.ContainsAny(host, "/ \t") {
return fmt.Errorf("config key %q: entry %q %w", return fmt.Errorf("%s: entry %q %w",
keyAllowlistHosts, host, errNotBareHostname) settingName(keyAllowlistHosts), host, errNotBareHostname)
} }
if strings.Trim(host, ".") == "" { if strings.Trim(host, ".") == "" {
return fmt.Errorf("config key %q: entry %q %w", return fmt.Errorf("%s: entry %q %w",
keyAllowlistHosts, host, errNoHostnameLabels) settingName(keyAllowlistHosts), host, errNoHostnameLabels)
} }
return nil return nil
@@ -598,11 +690,7 @@ func (l *strictLoader) boolVal(key string, defaultVal bool) bool {
// is omitted. A present value that is not a string, or is explicitly // is omitted. A present value that is not a string, or is explicitly
// null, is an error. // null, is an error.
func getString(sc *smartconfig.Config, key, defaultVal string) (string, error) { func getString(sc *smartconfig.Config, key, defaultVal string) (string, error) {
if sc == nil { raw, ok := lookupValue(sc, key)
return defaultVal, nil
}
raw, ok := sc.Get(key)
if !ok { if !ok {
return defaultVal, nil return defaultVal, nil
} }
@@ -624,11 +712,7 @@ func getString(sc *smartconfig.Config, key, defaultVal string) (string, error) {
// omitted. A present value that is not a whole number, or is explicitly // omitted. A present value that is not a whole number, or is explicitly
// null, is an error; fractional values are never truncated. // null, is an error; fractional values are never truncated.
func getInt(sc *smartconfig.Config, key string, defaultVal int) (int, error) { func getInt(sc *smartconfig.Config, key string, defaultVal int) (int, error) {
if sc == nil { raw, ok := lookupValue(sc, key)
return defaultVal, nil
}
raw, ok := sc.Get(key)
if !ok { if !ok {
return defaultVal, nil return defaultVal, nil
} }
@@ -652,8 +736,8 @@ func getInt(sc *smartconfig.Config, key string, defaultVal int) (int, error) {
case string: case string:
parsed, err := strconv.Atoi(strings.TrimSpace(val)) parsed, err := strconv.Atoi(strings.TrimSpace(val))
if err != nil { if err != nil {
return 0, fmt.Errorf("config key %q: value %q is %w", return 0, fmt.Errorf("%s: value %q is %w",
key, val, errNotAnInteger) settingName(key), val, errNotAnInteger)
} }
return parsed, nil return parsed, nil
@@ -668,11 +752,7 @@ func getInt(sc *smartconfig.Config, key string, defaultVal int) (int, error) {
// is explicitly null, is an error; fractional values are never // is explicitly null, is an error; fractional values are never
// truncated and out-of-range values are never clamped. // truncated and out-of-range values are never clamped.
func getInt64(sc *smartconfig.Config, key string, defaultVal int64) (int64, error) { func getInt64(sc *smartconfig.Config, key string, defaultVal int64) (int64, error) {
if sc == nil { raw, ok := lookupValue(sc, key)
return defaultVal, nil
}
raw, ok := sc.Get(key)
if !ok { if !ok {
return defaultVal, nil return defaultVal, nil
} }
@@ -703,8 +783,8 @@ func getInt64(sc *smartconfig.Config, key string, defaultVal int64) (int64, erro
case string: case string:
parsed, err := strconv.ParseInt(strings.TrimSpace(val), 10, 64) parsed, err := strconv.ParseInt(strings.TrimSpace(val), 10, 64)
if err != nil { if err != nil {
return 0, fmt.Errorf("config key %q: value %q is %w", return 0, fmt.Errorf("%s: value %q is %w",
key, val, errNotAnInteger) settingName(key), val, errNotAnInteger)
} }
return parsed, nil return parsed, nil
@@ -719,11 +799,7 @@ func getInt64(sc *smartconfig.Config, key string, defaultVal int64) (int64, erro
// string), or is explicitly null, is an error; numbers are not accepted // string), or is explicitly null, is an error; numbers are not accepted
// as booleans. // as booleans.
func getBool(sc *smartconfig.Config, key string, defaultVal bool) (bool, error) { func getBool(sc *smartconfig.Config, key string, defaultVal bool) (bool, error) {
if sc == nil { raw, ok := lookupValue(sc, key)
return defaultVal, nil
}
raw, ok := sc.Get(key)
if !ok { if !ok {
return defaultVal, nil return defaultVal, nil
} }
@@ -738,8 +814,8 @@ func getBool(sc *smartconfig.Config, key string, defaultVal bool) (bool, error)
case string: case string:
parsed, err := strconv.ParseBool(strings.TrimSpace(val)) parsed, err := strconv.ParseBool(strings.TrimSpace(val))
if err != nil { if err != nil {
return false, fmt.Errorf("config key %q: value %q is %w", return false, fmt.Errorf("%s: value %q is %w",
key, val, errNotABoolean) settingName(key), val, errNotABoolean)
} }
return parsed, nil return parsed, nil
@@ -755,7 +831,7 @@ func getBool(sc *smartconfig.Config, key string, defaultVal bool) (bool, error)
// (or a comma-separated string), a non-string entry, or an empty entry // (or a comma-separated string), a non-string entry, or an empty entry
// is an error, never silently skipped. // is an error, never silently skipped.
func validateAllowlistHostsValue(sc *smartconfig.Config) error { func validateAllowlistHostsValue(sc *smartconfig.Config) error {
raw, ok := sc.Get(keyAllowlistHosts) raw, ok := lookupValue(sc, keyAllowlistHosts)
if !ok { if !ok {
return nil return nil
} }
@@ -785,8 +861,8 @@ func validateAllowlistHostsValue(sc *smartconfig.Config) error {
for part := range strings.SplitSeq(val, ",") { for part := range strings.SplitSeq(val, ",") {
if strings.TrimSpace(part) == "" { if strings.TrimSpace(part) == "" {
return fmt.Errorf("config key %q: value %q %w", return fmt.Errorf("%s: value %q %w",
keyAllowlistHosts, val, errEmptyEntry) settingName(keyAllowlistHosts), val, errEmptyEntry)
} }
} }
default: default:
@@ -802,11 +878,7 @@ func validateAllowlistHostsValue(sc *smartconfig.Config) error {
// comma-separated string (backwards compatibility). Malformed entries // comma-separated string (backwards compatibility). Malformed entries
// are rejected beforehand by validateAllowlistHostsValue. // are rejected beforehand by validateAllowlistHostsValue.
func getStringSlice(sc *smartconfig.Config) []string { func getStringSlice(sc *smartconfig.Config) []string {
if sc == nil { val, ok := lookupValue(sc, keyAllowlistHosts)
return nil
}
val, ok := sc.Get(keyAllowlistHosts)
if !ok || val == nil { if !ok || val == nil {
return nil return nil
} }
@@ -866,11 +938,7 @@ func defaultTrustedProxies() []netip.Prefix {
// aborts startup naming the key and the offending value; the default // aborts startup naming the key and the offending value; the default
// (an empty list) applies only to an omitted key. // (an empty list) applies only to an omitted key.
func parseCIDRList(sc *smartconfig.Config, key string) ([]netip.Prefix, error) { func parseCIDRList(sc *smartconfig.Config, key string) ([]netip.Prefix, error) {
if sc == nil { raw, ok := lookupValue(sc, key)
return nil, nil
}
raw, ok := sc.Get(key)
if !ok { if !ok {
return nil, nil return nil, nil
} }
@@ -889,8 +957,8 @@ func parseCIDRList(sc *smartconfig.Config, key string) ([]netip.Prefix, error) {
for _, entry := range entries { for _, entry := range entries {
prefix, err := netip.ParsePrefix(entry) prefix, err := netip.ParsePrefix(entry)
if err != nil { if err != nil {
return nil, fmt.Errorf("config key %q: value %q is %w", return nil, fmt.Errorf("%s: value %q is %w",
key, entry, errNotAValidCIDR) settingName(key), entry, errNotAValidCIDR)
} }
prefixes = append(prefixes, prefix) prefixes = append(prefixes, prefix)
@@ -901,7 +969,8 @@ func parseCIDRList(sc *smartconfig.Config, key string) ([]netip.Prefix, error) {
// cidrListEntries extracts the raw entries of the named CIDR-list key as // cidrListEntries extracts the raw entries of the named CIDR-list key as
// trimmed, non-empty strings, from either a YAML list of strings or a // trimmed, non-empty strings, from either a YAML list of strings or a
// comma-separated string. Any other shape is a configuration error. // comma-separated string; an empty string is an empty list, as for
// allowlist_hosts. Any other shape is a configuration error.
func cidrListEntries(raw any, key string) ([]string, error) { func cidrListEntries(raw any, key string) ([]string, error) {
switch val := raw.(type) { switch val := raw.(type) {
case []any: case []any:
@@ -926,11 +995,15 @@ func cidrListEntries(raw any, key string) ([]string, error) {
case string: case string:
entries := make([]string, 0) entries := make([]string, 0)
if strings.TrimSpace(val) == "" {
return entries, nil
}
for part := range strings.SplitSeq(val, ",") { for part := range strings.SplitSeq(val, ",") {
trimmed := strings.TrimSpace(part) trimmed := strings.TrimSpace(part)
if trimmed == "" { if trimmed == "" {
return nil, fmt.Errorf("config key %q: value %q %w", return nil, fmt.Errorf("%s: value %q %w",
key, val, errEmptyEntry) settingName(key), val, errEmptyEntry)
} }
entries = append(entries, trimmed) entries = append(entries, trimmed)
+367
View File
@@ -0,0 +1,367 @@
package config
import (
"net/netip"
"os"
"path/filepath"
"reflect"
"slices"
"strings"
"testing"
"sneak.berlin/go/pixa/internal/globals"
"sneak.berlin/go/pixa/internal/logger"
)
// TestMain unsets PORT and every PIXA_ environment variable before the
// tests run, so each test sees only the variables it sets itself, not
// whatever the shell running the tests exports.
func TestMain(m *testing.M) {
for _, entry := range os.Environ() {
name, _, _ := strings.Cut(entry, "=")
if name != "PORT" && !strings.HasPrefix(name, "PIXA_") {
continue
}
err := os.Unsetenv(name)
if err != nil {
panic(err)
}
}
m.Run()
}
// wantStartupError fails the test unless err is a startup error that
// mentions every one of wants.
func wantStartupError(t *testing.T, err error, wants ...string) {
t.Helper()
if err == nil {
t.Fatalf("want a startup error mentioning %q, got none", wants)
}
t.Logf("got expected error: %v", err)
for _, want := range wants {
if !strings.Contains(err.Error(), want) {
t.Errorf("error %q does not mention %q", err.Error(), want)
}
}
}
// TestEnvironmentSetsEveryKey sets every key from its environment
// variable, with no config file at all: PORT for the port, and PIXA_
// plus the key in upper case, "." written as "_", for every other key.
func TestEnvironmentSetsEveryKey(t *testing.T) {
t.Setenv("PIXA_DEBUG", "true")
t.Setenv("PIXA_MAINTENANCE_MODE", "1")
t.Setenv("PORT", "9090")
t.Setenv("PIXA_STATE_DIR", "/srv/pixa-env")
t.Setenv("PIXA_SENTRY_DSN", "https://abc123@sentry.example.com/42")
t.Setenv("PIXA_DB_URL", "file:/srv/pixa-env/other.sqlite3")
t.Setenv("PIXA_METRICS_USERNAME", "metricsuser")
t.Setenv("PIXA_METRICS_PASSWORD", "metricspass")
t.Setenv("PIXA_SIGNING_KEY", validTestSigningKey)
t.Setenv("PIXA_ALLOWLIST_HOSTS", "s3.sneak.cloud,.example.com")
t.Setenv("PIXA_ALLOW_HTTP", "true")
t.Setenv("PIXA_UPSTREAM_CONNECTIONS_PER_HOST", "5")
t.Setenv("PIXA_CACHE_MAX_BYTES", "1024")
t.Setenv("PIXA_BLOCKED_NETWORKS", "203.0.113.0/24")
t.Setenv("PIXA_TRUSTED_PROXIES", "192.0.2.0/24")
c, err := newFromSmartConfig(nil)
if err != nil {
t.Fatalf("configuration from the environment alone should load: %v", err)
}
want := Config{
Debug: true,
MaintenanceMode: true,
Port: 9090,
StateDir: "/srv/pixa-env",
SentryDSN: "https://abc123@sentry.example.com/42",
DBURL: "file:/srv/pixa-env/other.sqlite3",
MetricsUsername: "metricsuser",
MetricsPassword: "metricspass",
SigningKey: validTestSigningKey,
AllowlistHosts: []string{testHostS3, ".example.com"},
AllowHTTP: true,
UpstreamConnectionsPerHost: 5,
CacheMaxBytes: 1024,
cacheMaxBytesExplicit: true,
BlockedNetworks: []netip.Prefix{netip.MustParsePrefix("203.0.113.0/24")},
TrustedProxies: []netip.Prefix{netip.MustParsePrefix("192.0.2.0/24")},
}
if !reflect.DeepEqual(*c, want) {
t.Errorf("config from the environment =\n%+v\nwant\n%+v", *c, want)
}
}
// TestUnknownPixaVariableAbortsStartup checks that a PIXA_ variable that
// is not a setting's variable, such as a misspelled one, aborts startup
// naming it, as an unknown config key does, instead of being ignored.
func TestUnknownPixaVariableAbortsStartup(t *testing.T) {
t.Setenv("PIXA_TRUSTED_PROXY", "192.0.2.0/24")
t.Setenv("PIXA_SIGNINGKEY", validTestSigningKey)
err := validateKnownEnvVars()
wantStartupError(t, err, "PIXA_TRUSTED_PROXY", "PIXA_SIGNINGKEY")
}
// TestPixaPortAbortsStartupPointingToPort checks that PIXA_PORT aborts
// startup with a message saying to use PORT, which sets the port.
func TestPixaPortAbortsStartupPointingToPort(t *testing.T) {
t.Setenv("PIXA_PORT", "9090")
err := validateKnownEnvVars()
wantStartupError(t, err, "PIXA_PORT", "use PORT")
}
// TestSettingVariablesAndConfigPathAreAccepted checks that every
// setting's variable and PIXA_CONFIG_PATH pass the check for unknown
// PIXA_ variables. TestEnvironmentSetsEveryKey pins the names in the list.
func TestSettingVariablesAndConfigPathAreAccepted(t *testing.T) {
// A config file's env section loaded by another test can leave a
// PIXA_ variable set for the whole process, so every one is unset
// here first; t.Setenv restores each when the test ends.
for _, entry := range os.Environ() {
name, _, _ := strings.Cut(entry, "=")
if !strings.HasPrefix(name, "PIXA_") {
continue
}
t.Setenv(name, "")
err := os.Unsetenv(name)
if err != nil {
t.Fatalf("failed to unset %s: %v", name, err)
}
}
t.Setenv("PIXA_CONFIG_PATH", "/etc/pixa/config.yml")
for _, name := range envVarNames() {
t.Setenv(name, "")
}
err := validateKnownEnvVars()
if err != nil {
t.Fatalf("PIXA_CONFIG_PATH and every setting's variable "+
"must be accepted: %v", err)
}
}
// configFromNew writes yamlContent to a temporary config file, points
// PIXA_CONFIG_PATH at it, and runs New, as the server does at startup.
// The state directory is a temporary one and the disk cache is off, so
// New succeeds unless something in the test is wrong.
func configFromNew(t *testing.T, yamlContent string) (*Config, error) {
t.Helper()
tmpDir := t.TempDir()
configPath := filepath.Join(tmpDir, "config.yml")
err := os.WriteFile(configPath, []byte(yamlContent), 0o600)
if err != nil {
t.Fatalf("failed to write test config: %v", err)
}
t.Setenv("PIXA_CONFIG_PATH", configPath)
t.Setenv("PIXA_STATE_DIR", filepath.Join(tmpDir, "state"))
t.Setenv("PIXA_CACHE_MAX_BYTES", "0")
testLogger, err := logger.New(nil, logger.Params{Globals: &globals.Globals{}})
if err != nil {
t.Fatalf("failed to create logger: %v", err)
}
return New(nil, Params{Logger: testLogger})
}
// TestUnknownPixaVariableAbortsNew checks that New, which the server
// calls at startup, aborts on a misspelled PIXA_ variable.
func TestUnknownPixaVariableAbortsNew(t *testing.T) {
t.Setenv("PIXA_TRUSTED_PROXY", "192.0.2.0/24")
_, err := configFromNew(t, signingKeyLine)
wantStartupError(t, err, "PIXA_TRUSTED_PROXY")
}
// TestUnknownPixaVariableInEnvSectionAbortsNew checks that New aborts
// on a misspelled PIXA_ name in the config file's env section, which
// loading the file sets as an environment variable.
func TestUnknownPixaVariableInEnvSectionAbortsNew(t *testing.T) {
// The variable must be absent until the file loads. t.Setenv makes
// sure the one the file sets is removed when the test ends.
t.Setenv("PIXA_TRUSTED_PROXY", "")
err := os.Unsetenv("PIXA_TRUSTED_PROXY")
if err != nil {
t.Fatalf("failed to unset PIXA_TRUSTED_PROXY: %v", err)
}
_, err = configFromNew(t, signingKeyLine+
"env:\n PIXA_TRUSTED_PROXY: 192.0.2.0/24\n")
wantStartupError(t, err, "PIXA_TRUSTED_PROXY")
}
// TestPortFromEnvironmentOverridesConfigFile checks that PORT wins over
// the port in the config file.
func TestPortFromEnvironmentOverridesConfigFile(t *testing.T) {
t.Setenv("PORT", "9090")
c, err := configFromYAML(t, signingKeyLine+"port: 8080\n")
if err != nil {
t.Fatalf("PORT=9090 with port 8080 in the file should load: %v", err)
}
if c.Port != 9090 {
t.Errorf("Port = %d, want 9090 from PORT, not 8080 from the file", c.Port)
}
}
// TestInvalidPortFromEnvironmentAbortsStartup checks that a PORT that is
// not a number, or is outside the port range, aborts startup naming PORT
// and the value, even though the file's port is valid.
func TestInvalidPortFromEnvironmentAbortsStartup(t *testing.T) {
t.Setenv("PORT", "banana")
_, err := configFromYAML(t, signingKeyLine+"port: 8080\n")
wantStartupError(t, err, "PORT", "banana")
t.Setenv("PORT", "70000")
_, err = configFromYAML(t, signingKeyLine+"port: 8080\n")
wantStartupError(t, err, "PORT", "70000")
}
// TestListFromEnvironmentReplacesConfigFileList checks that a list
// variable replaces the file's list, split on commas with the spaces
// around each entry trimmed.
func TestListFromEnvironmentReplacesConfigFileList(t *testing.T) {
t.Setenv("PIXA_ALLOWLIST_HOSTS", " cdn.example.com , .example.org ")
c, err := configFromYAML(t, signingKeyLine+
"allowlist_hosts:\n - s3.sneak.cloud\n - sneak.berlin\n")
if err != nil {
t.Fatalf("PIXA_ALLOWLIST_HOSTS should load: %v", err)
}
want := []string{"cdn.example.com", ".example.org"}
if !slices.Equal(c.AllowlistHosts, want) {
t.Errorf("AllowlistHosts = %v, want %v from PIXA_ALLOWLIST_HOSTS",
c.AllowlistHosts, want)
}
}
// TestInvalidBlockedNetworksFromEnvironmentAbortsStartup checks that an
// invalid CIDR, or an empty entry, in PIXA_BLOCKED_NETWORKS aborts
// startup naming the variable, as the same list in the file does.
func TestInvalidBlockedNetworksFromEnvironmentAbortsStartup(t *testing.T) {
t.Setenv("PIXA_BLOCKED_NETWORKS", "203.0.113.0/24,not-a-cidr")
_, err := configFromYAML(t, signingKeyLine)
wantStartupError(t, err, "PIXA_BLOCKED_NETWORKS", "not-a-cidr")
t.Setenv("PIXA_BLOCKED_NETWORKS", "203.0.113.0/24,,198.51.100.0/24")
_, err = configFromYAML(t, signingKeyLine)
wantStartupError(t, err, "PIXA_BLOCKED_NETWORKS")
}
// TestInvalidDebugFromEnvironmentAbortsStartup checks that a PIXA_DEBUG
// that strconv.ParseBool rejects aborts startup instead of defaulting.
func TestInvalidDebugFromEnvironmentAbortsStartup(t *testing.T) {
t.Setenv("PIXA_DEBUG", "maybe")
_, err := configFromYAML(t, signingKeyLine)
wantStartupError(t, err, "PIXA_DEBUG", "maybe")
}
// TestConfigFileAloneBehavesAsBefore checks that with no variables set
// (TestMain unsets them) the config file's values are used and omitted
// keys take their defaults.
func TestConfigFileAloneBehavesAsBefore(t *testing.T) {
t.Parallel()
c, err := configFromYAML(t, signingKeyLine+"port: 9191\n")
if err != nil {
t.Fatalf("config file should load: %v", err)
}
if c.Port != 9191 {
t.Errorf("Port = %d, want 9191 from the file", c.Port)
}
if c.StateDir != DefaultStateDir {
t.Errorf("StateDir = %q, want default %q", c.StateDir, DefaultStateDir)
}
if !slices.Equal(c.TrustedProxies, defaultTrustedProxies()) {
t.Errorf("TrustedProxies = %v, want default %v",
c.TrustedProxies, defaultTrustedProxies())
}
}
// TestEmptyTrustedProxiesFromEnvironmentTrustsNoOne checks that an empty
// PIXA_TRUSTED_PROXIES is an empty list, like [] in the file: it trusts
// no proxy instead of taking the default ranges.
func TestEmptyTrustedProxiesFromEnvironmentTrustsNoOne(t *testing.T) {
t.Setenv("PIXA_TRUSTED_PROXIES", "")
c, err := configFromYAML(t, signingKeyLine)
if err != nil {
t.Fatalf("empty PIXA_TRUSTED_PROXIES should load: %v", err)
}
if len(c.TrustedProxies) != 0 {
t.Errorf("TrustedProxies = %v, want none", c.TrustedProxies)
}
}
// TestEmptyVariableDoesNotFallBackToConfigFile checks that a variable
// that is present but empty is a set value: an empty PIXA_STATE_DIR
// aborts startup like state_dir: "" in the file, instead of falling
// through to the file's state_dir.
func TestEmptyVariableDoesNotFallBackToConfigFile(t *testing.T) {
t.Setenv("PIXA_STATE_DIR", "")
_, err := configFromYAML(t, signingKeyLine+"state_dir: /srv/pixa-file\n")
wantStartupError(t, err, "PIXA_STATE_DIR")
}
// TestMissingSigningKeyNamesItsVariable checks that with no config file
// and no PIXA_SIGNING_KEY, startup aborts naming the variable, which is
// how a container started without it reports the problem.
func TestMissingSigningKeyNamesItsVariable(t *testing.T) {
t.Parallel()
_, err := newFromSmartConfig(nil)
wantStartupError(t, err, "PIXA_SIGNING_KEY")
}
// TestSecretsFromEnvironmentAreNotPrinted checks that errors about the
// signing key and the metrics password name their variables but never
// print their values.
func TestSecretsFromEnvironmentAreNotPrinted(t *testing.T) {
t.Setenv("PIXA_SIGNING_KEY", "short-signing-secret")
_, err := newFromSmartConfig(nil)
wantStartupError(t, err, "PIXA_SIGNING_KEY")
if strings.Contains(err.Error(), "short-signing-secret") {
t.Errorf("error %q prints the signing key", err.Error())
}
t.Setenv("PIXA_SIGNING_KEY", validTestSigningKey)
t.Setenv("PIXA_METRICS_PASSWORD", "metrics-password-secret")
_, err = newFromSmartConfig(nil)
wantStartupError(t, err, "PIXA_METRICS_PASSWORD")
if strings.Contains(err.Error(), "metrics-password-secret") {
t.Errorf("error %q prints the metrics password", err.Error())
}
}
+133 -17
View File
@@ -1,8 +1,12 @@
package handlers package handlers
import ( import (
"bytes"
"crypto/subtle" "crypto/subtle"
"errors"
"fmt"
"html/template" "html/template"
"math"
"net/http" "net/http"
"net/url" "net/url"
"strconv" "strconv"
@@ -14,6 +18,20 @@ import (
"sneak.berlin/go/pixa/internal/templates" "sneak.berlin/go/pixa/internal/templates"
) )
// errInvalidFormField reports a generator form field, or the q or exp
// parameter of /v1/image/, whose value is non-numeric or out of range. The
// offending field name is wrapped in so the response can name it.
var errInvalidFormField = errors.New("invalid")
// Bounds for the generator's quality and ttl fields; the quality bounds also
// apply to the q parameter of /v1/image/. maxTTL is in seconds: the expiry
// calculation time.Duration(ttl) * time.Second overflows above it.
const (
minQuality = 1
maxQuality = 100
maxTTL = int(math.MaxInt64 / time.Second)
)
// HandleRoot serves the login page or generator page based on authentication state. // HandleRoot serves the login page or generator page based on authentication state.
func (s *Handlers) HandleRoot() http.HandlerFunc { func (s *Handlers) HandleRoot() http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) { return func(w http.ResponseWriter, r *http.Request) {
@@ -101,18 +119,26 @@ func (s *Handlers) HandleGenerateURL() http.HandlerFunc {
// Validate source URL // Validate source URL
parsed, err := url.Parse(sourceURL) parsed, err := url.Parse(sourceURL)
if err != nil || parsed.Host == "" { if err != nil || parsed.Host == "" {
s.renderGeneratorWithForm(w, r, "Invalid source URL", r.Form) s.renderGeneratorWithForm(w, r, "Invalid source URL", r.Form,
http.StatusBadRequest)
return return
} }
payload, expiresAt, ttl := buildGeneratePayload(parsed, r.Form) payload, expiresAt, ttl, err := buildGeneratePayload(parsed, r.Form)
if err != nil {
s.renderGeneratorWithForm(w, r, err.Error(), r.Form,
http.StatusBadRequest)
return
}
// Generate encrypted token // Generate encrypted token
token, err := s.encGen.Generate(payload) token, err := s.encGen.Generate(payload)
if err != nil { if err != nil {
s.log.Error("failed to generate encrypted URL", "error", err) s.log.Error("failed to generate encrypted URL", "error", err)
s.renderGeneratorWithForm(w, r, "Failed to generate URL", r.Form) s.renderGeneratorWithForm(w, r, "Failed to generate URL", r.Form,
http.StatusInternalServerError)
return return
} }
@@ -140,17 +166,40 @@ func (s *Handlers) HandleGenerateURL() http.HandlerFunc {
} }
// buildGeneratePayload parses the numeric form fields and assembles the // buildGeneratePayload parses the numeric form fields and assembles the
// encrypted URL payload. ttl=0 means never expires (ExpiresAt stays 0). // encrypted URL payload. ttl=0 means never expires (ExpiresAt stays 0). A
// non-numeric or out-of-range width, height, quality or ttl, or an
// unrecognized fit mode, is a client error naming the offending field. The
// format field is passed through unchecked.
func buildGeneratePayload( func buildGeneratePayload(
parsed *url.URL, form url.Values, parsed *url.URL, form url.Values,
) (*encurl.Payload, time.Time, int) { ) (*encurl.Payload, time.Time, int, error) {
width, _ := strconv.Atoi(form.Get("width")) width, err := parseFormDimension(form, "width")
height, _ := strconv.Atoi(form.Get("height")) if err != nil {
quality, _ := strconv.Atoi(form.Get("quality")) return nil, time.Time{}, 0, err
ttl, _ := strconv.Atoi(form.Get("ttl")) }
if quality <= 0 { height, err := parseFormDimension(form, "height")
quality = 85 if err != nil {
return nil, time.Time{}, 0, err
}
quality, err := parseFormInt(form, "quality",
encurl.DefaultQuality, minQuality, maxQuality)
if err != nil {
return nil, time.Time{}, 0, err
}
ttl, err := parseFormInt(form, "ttl", 0, 0, maxTTL)
if err != nil {
return nil, time.Time{}, 0, err
}
fitMode := imgcache.FitMode(form.Get("fit"))
err = imgcache.ValidateFitMode(fitMode)
if err != nil {
return nil, time.Time{}, 0,
fmt.Errorf("%w: %s", imgcache.ErrInvalidFitMode, form.Get("fit"))
} }
var ( var (
@@ -171,11 +220,57 @@ func buildGeneratePayload(
Height: height, Height: height,
Format: imgcache.ImageFormat(form.Get("format")), Format: imgcache.ImageFormat(form.Get("format")),
Quality: quality, Quality: quality,
FitMode: imgcache.FitMode(form.Get("fit")), FitMode: fitMode,
ExpiresAt: expiresAtUnix, ExpiresAt: expiresAtUnix,
} }
return payload, expiresAt, ttl return payload, expiresAt, ttl, nil
}
// parseFormDimension reads an optional width or height form field. An empty
// value means "original size" (0). A non-numeric value, or one
// imgcache.ValidateDimension rejects, is an error naming the field.
func parseFormDimension(form url.Values, field string) (int, error) {
raw := form.Get(field)
if raw == "" {
return 0, nil
}
value, err := strconv.Atoi(raw)
if err != nil {
return 0, fmt.Errorf("%w %s: not a number", errInvalidFormField, field)
}
err = imgcache.ValidateDimension(field, value)
if err != nil {
return 0, err
}
return value, nil
}
// parseFormInt reads an optional integer form field or URL query parameter,
// returning def when the field is empty and an error naming the field when the
// value is non-numeric or outside minValue to maxValue.
func parseFormInt(
form url.Values, field string, def, minValue, maxValue int,
) (int, error) {
raw := form.Get(field)
if raw == "" {
return def, nil
}
value, err := strconv.Atoi(raw)
if err != nil {
return 0, fmt.Errorf("%w %s: not a number", errInvalidFormField, field)
}
if value < minValue || value > maxValue {
return 0, fmt.Errorf("%w %s: must be from %d to %d",
errInvalidFormField, field, minValue, maxValue)
}
return value, nil
} }
// generatorData holds template data for the generator page. // generatorData holds template data for the generator page.
@@ -215,6 +310,15 @@ func (s *Handlers) renderLogin(
func (s *Handlers) renderGenerator( func (s *Handlers) renderGenerator(
w http.ResponseWriter, r *http.Request, data *generatorData, w http.ResponseWriter, r *http.Request, data *generatorData,
) {
s.renderGeneratorStatus(w, r, data, http.StatusOK)
}
// renderGeneratorStatus renders the generator page with an explicit HTTP
// status; a rejected form uses 400. The page is rendered into a buffer before
// the status is written, so a template failure can still answer 500.
func (s *Handlers) renderGeneratorStatus(
w http.ResponseWriter, r *http.Request, data *generatorData, status int,
) { ) {
w.Header().Set("Content-Type", "text/html; charset=utf-8") w.Header().Set("Content-Type", "text/html; charset=utf-8")
@@ -224,17 +328,29 @@ func (s *Handlers) renderGenerator(
data.CSRFField = csrfField(r) data.CSRFField = csrfField(r)
err := templates.Render(w, "generator.html", data) var page bytes.Buffer
err := templates.Render(&page, "generator.html", data)
if err != nil { if err != nil {
s.log.Error("failed to render generator template", "error", err) s.log.Error("failed to render generator template", "error", err)
http.Error(w, "Internal server error", http.StatusInternalServerError) http.Error(w, "Internal server error", http.StatusInternalServerError)
return
}
w.WriteHeader(status)
_, err = page.WriteTo(w)
if err != nil {
s.log.Error("failed to write generator page", "error", err)
} }
} }
func (s *Handlers) renderGeneratorWithForm( func (s *Handlers) renderGeneratorWithForm(
w http.ResponseWriter, r *http.Request, errorMsg string, form url.Values, w http.ResponseWriter, r *http.Request, errorMsg string,
form url.Values, status int,
) { ) {
s.renderGenerator(w, r, &generatorData{ s.renderGeneratorStatus(w, r, &generatorData{
Error: errorMsg, Error: errorMsg,
FormURL: form.Get("url"), FormURL: form.Get("url"),
FormWidth: form.Get("width"), FormWidth: form.Get("width"),
@@ -243,7 +359,7 @@ func (s *Handlers) renderGeneratorWithForm(
FormQuality: form.Get("quality"), FormQuality: form.Get("quality"),
FormFit: form.Get("fit"), FormFit: form.Get("fit"),
FormTTL: form.Get("ttl"), FormTTL: form.Get("ttl"),
}) }, status)
} }
func (s *Handlers) buildGeneratedURL(r *http.Request, token, format string) string { func (s *Handlers) buildGeneratedURL(r *http.Request, token, format string) string {
@@ -0,0 +1,163 @@
package handlers
import (
"maps"
"net/http"
"net/http/httptest"
"net/url"
"strconv"
"strings"
"testing"
"time"
"sneak.berlin/go/pixa/internal/encurl"
"sneak.berlin/go/pixa/internal/imgcache"
)
// Generator form field names, and a value that is not a number.
const (
widthField = "width"
heightField = "height"
qualityField = "quality"
ttlField = "ttl"
fitField = "fit"
notANumber = "abc"
)
// generatePost submits the /generate form with a valid session and CSRF token
// plus the caller's extra fields, returning the recorder.
func generatePost(
t *testing.T, extra url.Values,
) *httptest.ResponseRecorder {
t.Helper()
h, srv := newCSRFTestRouter(t)
sessionCookie := newSessionCookie(t, h)
cookies, token := csrfCredentials(t, srv, []*http.Cookie{sessionCookie})
cookies = append(cookies, sessionCookie)
form := url.Values{
sourceURLField: {testSourceURL},
csrfTokenField: {token},
}
maps.Copy(form, extra)
return postForm(srv, "/generate", cookies, form)
}
// TestGeneratePostRejectsNonNumericWidth verifies that a non-numeric width is
// rejected with 400 naming the field rather than being coerced to 0 and
// minting a 0-width token.
func TestGeneratePostRejectsNonNumericWidth(t *testing.T) {
t.Parallel()
rec := generatePost(t, url.Values{"width": {"abc"}})
if rec.Code != http.StatusBadRequest {
t.Fatalf("status = %d, want %d", rec.Code, http.StatusBadRequest)
}
if strings.Contains(rec.Body.String(), "/v1/e/") {
t.Error("a token was generated for non-numeric width")
}
}
// TestGeneratePostRejectsOverLimitWidth verifies that a width beyond
// MaxDimension is rejected at generation time so an unusable token cannot be
// minted.
func TestGeneratePostRejectsOverLimitWidth(t *testing.T) {
t.Parallel()
rec := generatePost(t, url.Values{"width": {"100000"}})
if rec.Code != http.StatusBadRequest {
t.Fatalf("status = %d, want %d", rec.Code, http.StatusBadRequest)
}
if strings.Contains(rec.Body.String(), "/v1/e/") {
t.Error("a token was generated for an over-limit width")
}
}
// TestGeneratePostRejectsBadField verifies that each generator field whose
// value is not a number, out of range, or unrecognized is rejected with 400,
// mints no token, and is named in the error shown on the page.
func TestGeneratePostRejectsBadField(t *testing.T) {
t.Parallel()
tests := []struct {
field, value, wantError string
}{
{widthField, notANumber, "invalid width: not a number"},
{widthField, "-1", "width is negative"},
{widthField, "8193", "width is above 8192"},
{heightField, notANumber, "invalid height: not a number"},
{heightField, "8193", "height is above 8192"},
{qualityField, notANumber, "invalid quality: not a number"},
{qualityField, "0", "invalid quality: must be from 1 to 100"},
{qualityField, "101", "invalid quality: must be from 1 to 100"},
{ttlField, notANumber, "invalid ttl: not a number"},
{ttlField, "-1", "invalid ttl: must be from 0 to"},
{ttlField, "10000000000", "invalid ttl: must be from 0 to"},
{fitField, "bogus", "invalid fit mode: bogus"},
}
for _, tt := range tests {
t.Run(tt.field+"="+tt.value, func(t *testing.T) {
t.Parallel()
rec := generatePost(t, url.Values{tt.field: {tt.value}})
body := rec.Body.String()
if rec.Code != http.StatusBadRequest {
t.Fatalf("status = %d, want %d", rec.Code, http.StatusBadRequest)
}
if strings.Contains(body, "/v1/e/") {
t.Error("a token was generated")
}
if !strings.Contains(body, tt.wantError) {
t.Errorf("page does not show %q", tt.wantError)
}
})
}
}
// TestBuildGeneratePayloadDefaultAndLimits verifies that an empty quality
// takes the default, and that the largest accepted width, height, quality and
// ttl are accepted with an expiry still in the future.
func TestBuildGeneratePayloadDefaultAndLimits(t *testing.T) {
t.Parallel()
parsed, err := url.Parse(testSourceURL)
if err != nil {
t.Fatalf("url.Parse() error = %v", err)
}
payload, _, _, err := buildGeneratePayload(parsed, url.Values{})
if err != nil {
t.Fatalf("empty form: error = %v", err)
}
if payload.Quality != encurl.DefaultQuality {
t.Errorf("empty quality gave %d, want %d",
payload.Quality, encurl.DefaultQuality)
}
_, expiresAt, _, err := buildGeneratePayload(parsed, url.Values{
widthField: {strconv.Itoa(imgcache.MaxDimension)},
heightField: {strconv.Itoa(imgcache.MaxDimension)},
qualityField: {strconv.Itoa(maxQuality)},
ttlField: {strconv.Itoa(maxTTL)},
})
if err != nil {
t.Fatalf("largest accepted values: error = %v", err)
}
if !expiresAt.After(time.Now()) {
t.Errorf("ttl %d gave expiry %v, want a time in the future",
maxTTL, expiresAt)
}
}
+118
View File
@@ -4,6 +4,7 @@ import (
"bytes" "bytes"
"context" "context"
"database/sql" "database/sql"
"encoding/json"
"image" "image"
"image/color" "image/color"
"image/jpeg" "image/jpeg"
@@ -276,3 +277,120 @@ func TestHandleImage_ETagHeader(t *testing.T) {
t.Errorf("ETag should be quoted, got %q", etag) t.Errorf("ETag should be quoted, got %q", etag)
} }
} }
// TestHandleImage_InvalidFitMode_Returns400 verifies that the plain image
// route rejects an unrecognized fit mode with 400.
func TestHandleImage_InvalidFitMode_Returns400(t *testing.T) {
t.Parallel()
fix := setupTestHandler(t)
status := getImage(t, fix,
"/v1/image/"+fix.goodHost+"/images/photo.jpg/50x50.jpeg?fit=bogus")
if status != http.StatusBadRequest {
t.Fatalf("status = %d, want %d", status, http.StatusBadRequest)
}
}
// TestHandleImage_InvalidQuery_Returns400 verifies that the plain image route
// answers a q that is not a whole number from 1 to 100, an empty one
// included, with 400 naming q and the value, a parameter given more than once
// with 400 naming it, and a query string that cannot be decoded with 400
// showing it, instead of serving the image at the default quality 85 or at
// the first value given.
func TestHandleImage_InvalidQuery_Returns400(t *testing.T) {
t.Parallel()
tests := []struct {
query, wantError string
}{
{"q=banana", `invalid q: not a number, got "banana"`},
{"q=0", `invalid q: must be from 1 to 100, got "0"`},
{"q=101", `invalid q: must be from 1 to 100, got "101"`},
{"q=", `invalid q: not a number, got ""`},
{"q=80&q=500", `invalid q: given more than once`},
{"q=80&q=", `invalid q: given more than once`},
{"fit=cover&fit=contain", `invalid fit: given more than once`},
{"q=80%", `invalid query string "q=80%": invalid URL escape "%"`},
{
"q=50;fit=contain",
`invalid query string "q=50;fit=contain": ` +
`invalid semicolon separator in query`,
},
}
for _, tt := range tests {
t.Run(tt.query, func(t *testing.T) {
t.Parallel()
fix := setupTestHandler(t)
r := chi.NewRouter()
r.Get("/v1/image/*", fix.handler.HandleImage())
req := httptest.NewRequestWithContext(t.Context(), http.MethodGet,
"/v1/image/"+fix.goodHost+"/images/photo.jpg/50x50.jpeg?"+tt.query, nil)
rec := httptest.NewRecorder()
r.ServeHTTP(rec, req)
if rec.Code != http.StatusBadRequest {
t.Fatalf("status = %d, want %d", rec.Code, http.StatusBadRequest)
}
t.Logf("GET %s: %d %s", req.URL, rec.Code, rec.Body)
var body struct {
Error string `json:"error"`
}
err := json.NewDecoder(rec.Body).Decode(&body)
if err != nil {
t.Fatalf("decoding response body: %v", err)
}
if body.Error != tt.wantError {
t.Errorf("error = %q, want %q", body.Error, tt.wantError)
}
})
}
}
// TestHandleImage_EmptyFit_Returns400 verifies that the plain image route
// answers a fit that is in the URL but empty with 400 naming fit, instead of
// serving the image as cover. Only a fit missing from the URL means cover.
func TestHandleImage_EmptyFit_Returns400(t *testing.T) {
t.Parallel()
fix := setupTestHandler(t)
r := chi.NewRouter()
r.Get("/v1/image/*", fix.handler.HandleImage())
req := httptest.NewRequestWithContext(t.Context(), http.MethodGet,
"/v1/image/"+fix.goodHost+"/images/photo.jpg/50x50.jpeg?fit=", nil)
rec := httptest.NewRecorder()
r.ServeHTTP(rec, req)
if rec.Code != http.StatusBadRequest {
t.Fatalf("status = %d, want %d", rec.Code, http.StatusBadRequest)
}
t.Logf("GET %s: %d %s", req.URL, rec.Code, rec.Body)
var body struct {
Error string `json:"error"`
}
err := json.NewDecoder(rec.Body).Decode(&body)
if err != nil {
t.Fatalf("decoding response body: %v", err)
}
wantError := `invalid fit: not a fit mode, got ""`
if body.Error != wantError {
t.Errorf("error = %q, want %q", body.Error, wantError)
}
}
+87 -25
View File
@@ -2,12 +2,15 @@ package handlers
import ( import (
"errors" "errors"
"fmt"
"io" "io"
"net/http" "net/http"
"net/url"
"strconv" "strconv"
"time" "time"
"github.com/go-chi/chi/v5" "github.com/go-chi/chi/v5"
"sneak.berlin/go/pixa/internal/encurl"
"sneak.berlin/go/pixa/internal/httpfetcher" "sneak.berlin/go/pixa/internal/httpfetcher"
"sneak.berlin/go/pixa/internal/imgcache" "sneak.berlin/go/pixa/internal/imgcache"
) )
@@ -89,39 +92,66 @@ func (s *Handlers) parseImageRequest(
// Convert to ImageRequest // Convert to ImageRequest
req := parsed.ToImageRequest() req := parsed.ToImageRequest()
// Parse signature params from query string // Parse signature params from query string. r.URL.Query() would silently
query := r.URL.Query() // drop a pair it cannot decode, such as q=80%, so that q would be served
req.Signature = query.Get("sig") // at 85; a query string that cannot be decoded is refused instead. A
// parameter given more than once is refused too, as only its first value
// would be read.
query, err := url.ParseQuery(r.URL.RawQuery)
if err != nil {
s.respondError(w, fmt.Sprintf("invalid query string %q: %v",
r.URL.RawQuery, err), http.StatusBadRequest)
if expStr := query.Get("exp"); expStr != "" { return nil, false
exp, parseErr := strconv.ParseInt(expStr, 10, 64)
if parseErr == nil {
req.Expires = time.Unix(exp, 0)
}
} }
// Parse optional quality and fit params for name, values := range query {
if qStr := query.Get("q"); qStr != "" { if len(values) > 1 {
q, parseErr := strconv.Atoi(qStr) s.respondError(w, fmt.Sprintf("invalid %s: given more than once",
if parseErr == nil && q > 0 && q <= 100 { name), http.StatusBadRequest)
req.Quality = q
}
}
if fit := query.Get("fit"); fit != "" {
req.FitMode = imgcache.FitMode(fit)
fitErr := imgcache.ValidateFitMode(req.FitMode)
if fitErr != nil {
s.respondError(w, "invalid fit mode: "+fit, http.StatusBadRequest)
return nil, false return nil, false
} }
} }
// Default quality if not set req.Signature = query.Get("sig")
if req.Quality == 0 {
req.Quality = 85 req.Expires, err = parseExpires(query)
if err != nil {
s.respondError(w, err.Error(), http.StatusBadRequest)
return nil, false
}
// Parse optional quality and fit params. Only a q missing from the URL is
// 85. A q in the URL that is not a whole number from 1 to 100, an empty
// one included, is refused, checked as the generator checks its quality
// field; that check alone would take an empty q as missing.
qStr := query.Get("q")
if query.Has("q") && qStr == "" {
s.respondError(w, `invalid q: not a number, got ""`,
http.StatusBadRequest)
return nil, false
}
req.Quality, err = parseFormInt(query, "q",
encurl.DefaultQuality, minQuality, maxQuality)
if err != nil {
s.respondError(w, fmt.Sprintf("%v, got %q", err, qStr),
http.StatusBadRequest)
return nil, false
}
// Only a fit missing from the URL is cover. A fit in the URL that is not a
// fit mode is refused by the fit-mode check below; that check would take an
// empty fit as missing, so an empty one is refused here.
req.FitMode = imgcache.FitMode(query.Get("fit"))
if query.Has("fit") && req.FitMode == "" {
s.respondError(w, `invalid fit: not a fit mode, got ""`, http.StatusBadRequest)
return nil, false
} }
// Default fit mode if not set // Default fit mode if not set
@@ -129,9 +159,41 @@ func (s *Handlers) parseImageRequest(
req.FitMode = imgcache.FitCover req.FitMode = imgcache.FitCover
} }
// Enforce dimension and fit-mode bounds, shared with the encrypted-URL
// route. Dimensions are already bounded by the path parser above; this
// also rejects an unrecognized fit mode with 400 instead of letting it
// reach the processor as a 500.
err = imgcache.ValidateImageRequest(req)
if err != nil {
s.respondError(w, "invalid image request: "+err.Error(),
http.StatusBadRequest)
return nil, false
}
return req, true return req, true
} }
// parseExpires reads the exp query parameter, a Unix time in seconds. An exp
// missing from the URL gives the zero time, which the signature check takes
// as no expiration. An exp in the URL that is not a whole number, an empty
// one included, is an error naming exp and the value.
func parseExpires(query url.Values) (time.Time, error) {
if !query.Has("exp") {
return time.Time{}, nil
}
expStr := query.Get("exp")
exp, err := strconv.ParseInt(expStr, 10, 64)
if err != nil {
return time.Time{}, fmt.Errorf("%w exp: not a number, got %q",
errInvalidFormField, expStr)
}
return time.Unix(exp, 0), nil
}
// respondImageError maps image retrieval errors to HTTP responses. // respondImageError maps image retrieval errors to HTTP responses.
func (s *Handlers) respondImageError( func (s *Handlers) respondImageError(
w http.ResponseWriter, req *imgcache.ImageRequest, err error, w http.ResponseWriter, req *imgcache.ImageRequest, err error,
@@ -1,6 +1,7 @@
package handlers package handlers
import ( import (
"encoding/json"
"fmt" "fmt"
"net/http" "net/http"
"net/http/httptest" "net/http/httptest"
@@ -118,3 +119,53 @@ func TestHandleImage_GeneratedSignedURLVerifies(t *testing.T) {
}) })
} }
} }
// TestHandleImage_InvalidExp_Returns400 sends a signed-host URL whose exp is
// not a whole number, and one whose exp is empty. Each is refused with 400
// naming exp and the value, not with the 401 a URL without exp still gets.
func TestHandleImage_InvalidExp_Returns400(t *testing.T) {
t.Parallel()
tests := []struct {
query string
wantStatus int
wantError string
}{
{"sig=x&exp=banana", http.StatusBadRequest,
`invalid exp: not a number, got "banana"`},
{"sig=x&exp=", http.StatusBadRequest, `invalid exp: not a number, got ""`},
{"sig=x", http.StatusUnauthorized, "unauthorized"},
}
for _, tt := range tests {
t.Run(tt.query, func(t *testing.T) {
t.Parallel()
fix := setupTestHandler(t)
r := chi.NewRouter()
r.Get("/v1/image/*", fix.handler.HandleImage())
req := httptest.NewRequestWithContext(t.Context(), http.MethodGet,
"/v1/image/"+signedHost+"/images/photo.jpg/50x50.jpeg?"+tt.query, nil)
rec := httptest.NewRecorder()
r.ServeHTTP(rec, req)
t.Logf("GET %s: %d %s", req.URL, rec.Code, rec.Body)
var body struct {
Error string `json:"error"`
}
err := json.NewDecoder(rec.Body).Decode(&body)
if err != nil {
t.Fatalf("decoding response body: %v", err)
}
if rec.Code != tt.wantStatus || body.Error != tt.wantError {
t.Errorf("got %d %q, want %d %q",
rec.Code, body.Error, tt.wantStatus, tt.wantError)
}
})
}
}
+13
View File
@@ -50,6 +50,19 @@ func (s *Handlers) HandleImageEnc() http.HandlerFunc {
// Convert payload to ImageRequest // Convert payload to ImageRequest
req := payload.ToImageRequest() req := payload.ToImageRequest()
// Apply the same dimension and fit-mode bounds as the plain image
// route: a sealed payload is trusted for its origin, not for staying
// within limits, so an over-limit size or unknown fit mode is a 400
// here rather than an out-of-memory or a 500 from the processor.
err = imgcache.ValidateImageRequest(req)
if err != nil {
s.log.Debug("encrypted URL failed validation", "error", err)
s.respondError(w, "invalid encrypted URL: "+err.Error(),
http.StatusBadRequest)
return
}
// Log the request // Log the request
s.log.Debug("encrypted image request", s.log.Debug("encrypted image request",
"host", req.SourceHost, "host", req.SourceHost,
@@ -0,0 +1,98 @@
package handlers
import (
"context"
"log/slog"
"net/http"
"net/http/httptest"
"testing"
"github.com/go-chi/chi/v5"
"sneak.berlin/go/pixa/internal/encurl"
"sneak.berlin/go/pixa/internal/imgcache"
)
// newEncTestServer builds a router serving the encrypted-URL route with a
// generator seeded by the shared test signing key. The image service is left
// nil: these tests exercise validation that rejects a token before any image
// is fetched, so the handler must never reach the service.
func newEncTestServer(t *testing.T) (*encurl.Generator, http.Handler) {
t.Helper()
encGen, err := encurl.NewGenerator(testSigningKey)
if err != nil {
t.Fatalf("encurl.NewGenerator() error = %v", err)
}
h := &Handlers{
log: slog.New(slog.DiscardHandler),
encGen: encGen,
}
r := chi.NewRouter()
r.Get("/v1/e/{token}/*", h.HandleImageEnc())
return encGen, r
}
// getEncToken issues a GET for the given token and returns the recorder.
func getEncToken(srv http.Handler, token string) *httptest.ResponseRecorder {
req := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/v1/e/"+token+"/img.jpg", nil)
rec := httptest.NewRecorder()
srv.ServeHTTP(rec, req)
return rec
}
// TestHandleImageEnc_OverLimitDimension_Returns400 verifies that a decrypted
// token requesting a dimension beyond MaxDimension is rejected with 400
// instead of reaching the image processor and libvips.
func TestHandleImageEnc_OverLimitDimension_Returns400(t *testing.T) {
t.Parallel()
encGen, srv := newEncTestServer(t)
token, err := encGen.Generate(&encurl.Payload{
SourceHost: "cdn.example.com",
SourcePath: "/photo.jpg",
Width: 100000,
Height: 100000,
})
if err != nil {
t.Fatalf("Generate() error = %v", err)
}
rec := getEncToken(srv, token)
if rec.Code != http.StatusBadRequest {
t.Fatalf("status = %d, want %d", rec.Code, http.StatusBadRequest)
}
}
// TestHandleImageEnc_InvalidFitMode_Returns400 verifies that a decrypted token
// carrying an unrecognized fit mode is rejected with 400 rather than surfacing
// as a 500 from the image processor's default branch.
func TestHandleImageEnc_InvalidFitMode_Returns400(t *testing.T) {
t.Parallel()
encGen, srv := newEncTestServer(t)
token, err := encGen.Generate(&encurl.Payload{
SourceHost: "cdn.example.com",
SourcePath: "/photo.jpg",
Width: 800,
Height: 600,
FitMode: imgcache.FitMode("bogus"),
})
if err != nil {
t.Fatalf("Generate() error = %v", err)
}
rec := getEncToken(srv, token)
if rec.Code != http.StatusBadRequest {
t.Fatalf("status = %d, want %d", rec.Code, http.StatusBadRequest)
}
}
+22
View File
@@ -161,6 +161,13 @@ func (p *ImageProcessor) Process(
} }
defer img.Close() defer img.Close()
// Turn the image upright now: encode strips the EXIF orientation tag,
// and sizes below must be worked out on the upright image.
err = img.AutoRotate()
if err != nil {
return nil, fmt.Errorf("failed to auto-rotate: %w", err)
}
// Get original dimensions // Get original dimensions
origWidth := img.Width() origWidth := img.Width()
origHeight := img.Height() origHeight := img.Height()
@@ -404,6 +411,21 @@ func (p *ImageProcessor) encode(
return nil, fmt.Errorf("%w: %s", ErrUnsupportedOutputFormat, format) return nil, fmt.Errorf("%w: %s", ErrUnsupportedOutputFormat, format)
} }
// Stripping drops the ICC profile as well, and clients show an image
// with no profile as sRGB, so convert to sRGB first. "srgb" names
// libvips' built-in profile; govips' own sRGB path variable is set on
// first use but read without a lock, so concurrent requests race on it.
if img.HasICCProfile() {
err := img.TransformICCProfileWithFallback("srgb", "srgb")
if err != nil {
return nil, fmt.Errorf("failed to convert to sRGB: %w", err)
}
}
// Drop EXIF, XMP, IPTC and the ICC profile. govips ignores this for
// GIF, which carries none of them.
params.StripMetadata = true
output, _, err := img.Export(&params) output, _, err := img.Export(&params)
if err != nil { if err != nil {
return nil, err return nil, err
@@ -9,7 +9,9 @@ import (
"image/jpeg" "image/jpeg"
"image/png" "image/png"
"io" "io"
"math"
"os" "os"
"slices"
"testing" "testing"
"github.com/davidbyttow/govips/v2/vips" "github.com/davidbyttow/govips/v2/vips"
@@ -561,3 +563,152 @@ func TestImageProcessor_EncodeAVIF(t *testing.T) {
encodeAndCheck(t, FormatAVIF, 85, mimeAVIF) encodeAndCheck(t, FormatAVIF, 85, mimeAVIF)
} }
// processAndDecode runs input through Process and decodes the output with
// vips, so a test can inspect the image a client would receive.
func processAndDecode(t *testing.T, input []byte, req *Request) *vips.ImageRef {
t.Helper()
result, err := New(Params{}).Process(
context.Background(), bytes.NewReader(input), req,
)
if err != nil {
t.Fatalf("Process() error = %v", err)
}
defer func() { _ = result.Content.Close() }()
data, err := io.ReadAll(result.Content)
if err != nil {
t.Fatalf("failed to read result: %v", err)
}
output, err := vips.NewImageFromBuffer(data)
if err != nil {
t.Fatalf("failed to decode output: %v", err)
}
t.Cleanup(output.Close)
return output
}
func TestImageProcessor_StripsEXIF(t *testing.T) {
t.Parallel()
// gps-exif.jpg carries GPS coordinates, a camera make, model and serial
// number, and a capture time.
input, err := os.ReadFile("testdata/gps-exif.jpg")
if err != nil {
t.Fatalf("failed to read test JPEG: %v", err)
}
fixture, err := vips.NewImageFromBuffer(input)
if err != nil {
t.Fatalf("failed to decode test JPEG: %v", err)
}
t.Cleanup(fixture.Close)
if !slices.Contains(fixture.GetFields(), "exif-ifd3-GPSLatitude") {
t.Fatal("testdata/gps-exif.jpg has no GPS latitude")
}
formats := []Format{
FormatJPEG, FormatPNG, FormatWebP, FormatAVIF, FormatGIF, FormatOriginal,
}
for _, format := range formats {
t.Run(string(format), func(t *testing.T) {
t.Parallel()
output := processAndDecode(t, input, &Request{Format: format})
if output.HasExif() {
t.Errorf("output has EXIF: %v", output.GetExif())
}
})
}
}
func TestImageProcessor_AppliesEXIFOrientation(t *testing.T) {
t.Parallel()
// orientation-6.jpg is stored 16x8, red on the left and blue on the
// right, with EXIF orientation 6 (turn 90 degrees clockwise to view).
// Upright it is 8x16, red on top and blue below.
input, err := os.ReadFile("testdata/orientation-6.jpg")
if err != nil {
t.Fatalf("failed to read test JPEG: %v", err)
}
tests := []struct {
name string
size Size
wantW int
wantH int
}{
{name: "original size", size: Size{}, wantW: 8, wantH: 16},
{name: "width only", size: Size{Width: 4}, wantW: 4, wantH: 8},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
output := processAndDecode(t, input, &Request{
Size: tt.size,
Format: FormatPNG,
})
if output.Width() != tt.wantW || output.Height() != tt.wantH {
t.Fatalf("output is %dx%d, want %dx%d",
output.Width(), output.Height(), tt.wantW, tt.wantH)
}
top, err := output.GetPoint(tt.wantW/2, 0)
if err != nil {
t.Fatalf("GetPoint() error = %v", err)
}
bottom, err := output.GetPoint(tt.wantW/2, tt.wantH-1)
if err != nil {
t.Fatalf("GetPoint() error = %v", err)
}
if top[0] <= top[2] || bottom[2] <= bottom[0] {
t.Errorf("top pixel = %v, bottom pixel = %v, want red above blue",
top, bottom)
}
})
}
}
func TestImageProcessor_ConvertsWideGamutToSRGB(t *testing.T) {
t.Parallel()
// display-p3.jpg is a flat 8x8 image with the Display P3 profile
// embedded, filled with Display P3 (234, 51, 35), which is sRGB red.
input, err := os.ReadFile("testdata/display-p3.jpg")
if err != nil {
t.Fatalf("failed to read test JPEG: %v", err)
}
output := processAndDecode(t, input, &Request{Format: FormatPNG})
if output.HasICCProfile() {
t.Error("output has an ICC profile")
}
pixel, err := output.GetPoint(4, 4)
if err != nil {
t.Fatalf("GetPoint() error = %v", err)
}
want := []float64{255, 0, 0}
for i := range want {
if math.Abs(pixel[i]-want[i]) > 5 {
t.Fatalf("pixel = %v, want within 5 of %v", pixel, want)
}
}
}
Binary file not shown.

After

Width:  |  Height:  |  Size: 1.3 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.0 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 811 B

+59 -24
View File
@@ -125,7 +125,7 @@ func NewCache(db *sql.DB, config CacheConfig) (*Cache, error) {
} }
variants, err := NewVariantStorage( variants, err := NewVariantStorage(
filepath.Join(config.StateDir, "cache", "variants"), filepath.Join(config.StateDir, "cache", "variants"), log,
) )
if err != nil { if err != nil {
return nil, fmt.Errorf("failed to create variant storage: %w", err) return nil, fmt.Errorf("failed to create variant storage: %w", err)
@@ -263,23 +263,7 @@ func (c *Cache) StoreSource(
return "", fmt.Errorf("failed to insert source metadata: %w", err) return "", fmt.Errorf("failed to insert source metadata: %w", err)
} }
// Store metadata JSON file c.writeMetadataSidecar(req, pathHash, contentHash, result)
meta := &SourceMetadata{
Host: req.SourceHost,
Path: req.SourcePath,
Query: req.SourceQuery,
ContentHash: string(contentHash),
StatusCode: result.StatusCode,
ContentType: result.ContentType,
ContentLength: result.ContentLength,
ResponseHeaders: result.Headers,
FetchedAt: time.Now().UTC().Unix(),
FetchDurationMs: result.FetchDurationMs,
RemoteAddr: result.RemoteAddr,
}
// A failure here is non-fatal; the metadata is in the database.
_ = c.srcMetadata.Store(req.SourceHost, pathHash, meta)
c.notifyWritePressure() c.notifyWritePressure()
@@ -436,12 +420,19 @@ func (c *Cache) Stats(ctx context.Context) (*CacheStats, error) {
} }
// Get actual item count and total size from content tables // Get actual item count and total size from content tables
_ = c.db.QueryRowContext(ctx, err = c.db.QueryRowContext(ctx,
`SELECT COUNT(*) FROM request_cache`, `SELECT COUNT(*) FROM request_cache`,
).Scan(&stats.TotalItems) ).Scan(&stats.TotalItems)
_ = c.db.QueryRowContext(ctx, if err != nil {
c.log.Warn("failed to count cache items for stats", "error", err)
}
err = c.db.QueryRowContext(ctx,
`SELECT COALESCE(SUM(size_bytes), 0) FROM output_content`, `SELECT COALESCE(SUM(size_bytes), 0) FROM output_content`,
).Scan(&stats.TotalSizeBytes) ).Scan(&stats.TotalSizeBytes)
if err != nil {
c.log.Warn("failed to sum cache size for stats", "error", err)
}
// Compute hit rate as a ratio // Compute hit rate as a ratio
if stats.HitCount+stats.MissCount > 0 { if stats.HitCount+stats.MissCount > 0 {
@@ -453,15 +444,17 @@ func (c *Cache) Stats(ctx context.Context) (*CacheStats, error) {
// IncrementStats increments cache statistics. // IncrementStats increments cache statistics.
func (c *Cache) IncrementStats(ctx context.Context, hit bool, fetchBytes int64) { func (c *Cache) IncrementStats(ctx context.Context, hit bool, fetchBytes int64) {
var err error
if hit { if hit {
_, _ = c.db.ExecContext(ctx, ` _, err = c.db.ExecContext(ctx, `
UPDATE cache_stats UPDATE cache_stats
SET hit_count = hit_count + 1, SET hit_count = hit_count + 1,
last_updated_at = CURRENT_TIMESTAMP last_updated_at = CURRENT_TIMESTAMP
WHERE id = 1 WHERE id = 1
`) `)
} else { } else {
_, _ = c.db.ExecContext(ctx, ` _, err = c.db.ExecContext(ctx, `
UPDATE cache_stats UPDATE cache_stats
SET miss_count = miss_count + 1, SET miss_count = miss_count + 1,
last_updated_at = CURRENT_TIMESTAMP last_updated_at = CURRENT_TIMESTAMP
@@ -469,14 +462,52 @@ func (c *Cache) IncrementStats(ctx context.Context, hit bool, fetchBytes int64)
`) `)
} }
if err != nil {
c.log.Warn("failed to count cache hit or miss", "hit", hit, "error", err)
}
if fetchBytes > 0 { if fetchBytes > 0 {
_, _ = c.db.ExecContext(ctx, ` _, err = c.db.ExecContext(ctx, `
UPDATE cache_stats UPDATE cache_stats
SET upstream_fetch_count = upstream_fetch_count + 1, SET upstream_fetch_count = upstream_fetch_count + 1,
upstream_fetch_bytes = upstream_fetch_bytes + ?, upstream_fetch_bytes = upstream_fetch_bytes + ?,
last_updated_at = CURRENT_TIMESTAMP last_updated_at = CURRENT_TIMESTAMP
WHERE id = 1 WHERE id = 1
`, fetchBytes) `, fetchBytes)
if err != nil {
c.log.Warn("failed to count upstream fetch",
"fetch_bytes", fetchBytes, "error", err)
}
}
}
// writeMetadataSidecar writes the JSON metadata sidecar of a stored source.
// A failure is logged and is otherwise non-fatal; the metadata is in the
// database.
func (c *Cache) writeMetadataSidecar(
req *ImageRequest,
pathHash PathHash,
contentHash ContentHash,
result *httpfetcher.FetchResult,
) {
meta := &SourceMetadata{
Host: req.SourceHost,
Path: req.SourcePath,
Query: req.SourceQuery,
ContentHash: string(contentHash),
StatusCode: result.StatusCode,
ContentType: result.ContentType,
ContentLength: result.ContentLength,
ResponseHeaders: result.Headers,
FetchedAt: time.Now().UTC().Unix(),
FetchDurationMs: result.FetchDurationMs,
RemoteAddr: result.RemoteAddr,
}
err := c.srcMetadata.Store(req.SourceHost, pathHash, meta)
if err != nil {
c.log.Warn("failed to write metadata sidecar",
"host", req.SourceHost, "path_hash", pathHash, "error", err)
} }
} }
@@ -528,10 +559,14 @@ func (c *Cache) checkNegativeCache(
// Check if expired // Check if expired
if time.Now().After(expiresAt) { if time.Now().After(expiresAt) {
// Clean up expired entry // Clean up expired entry
_, _ = c.db.ExecContext(ctx, ` _, err = c.db.ExecContext(ctx, `
DELETE FROM negative_cache DELETE FROM negative_cache
WHERE source_host = ? AND source_path = ? AND source_query = ? WHERE source_host = ? AND source_path = ? AND source_query = ?
`, req.SourceHost, req.SourcePath, req.SourceQuery) `, req.SourceHost, req.SourcePath, req.SourceQuery)
if err != nil {
c.log.Warn("failed to delete expired negative cache entry",
"host", req.SourceHost, "path", req.SourcePath, "error", err)
}
return false, nil return false, nil
} }
+18
View File
@@ -59,6 +59,24 @@ func ValidateFitMode(fit FitMode) error {
} }
} }
// ValidateImageRequest checks a request's width and height with
// ValidateDimension and its fit mode with ValidateFitMode. Both the plain
// /v1/image/ route and the encrypted /v1/e/ route validate through this
// function so a request from either source enforces identical bounds.
func ValidateImageRequest(req *ImageRequest) error {
err := ValidateDimension("width", req.Size.Width)
if err != nil {
return err
}
err = ValidateDimension("height", req.Size.Height)
if err != nil {
return err
}
return ValidateFitMode(req.FitMode)
}
// ImageRequest represents a request for a processed image // ImageRequest represents a request for a processed image
type ImageRequest struct { type ImageRequest struct {
// SourceHost is the origin host (e.g., "cdn.example.com") // SourceHost is the origin host (e.g., "cdn.example.com")
+6 -1
View File
@@ -322,7 +322,12 @@ func (s *Service) fetchAndProcess(
// Store negative cache for certain errors // Store negative cache for certain errors
if isNegativeCacheable(err) { if isNegativeCacheable(err) {
statusCode := extractStatusCode(err) statusCode := extractStatusCode(err)
_ = s.cache.StoreNegative(ctx, req, statusCode, err.Error())
storeErr := s.cache.StoreNegative(ctx, req, statusCode, err.Error())
if storeErr != nil {
s.log.Warn("failed to store negative cache entry",
"host", req.SourceHost, "path", req.SourcePath, "error", storeErr)
}
} }
return nil, fmt.Errorf("upstream fetch failed: %w", err) return nil, fmt.Errorf("upstream fetch failed: %w", err)
+82
View File
@@ -1,9 +1,12 @@
package imgcache package imgcache
import ( import (
"bytes"
"context" "context"
"database/sql" "database/sql"
"log/slog"
"math" "math"
"strings"
"testing" "testing"
"time" "time"
@@ -101,3 +104,82 @@ func TestStats_ZeroCounts(t *testing.T) {
t.Errorf("HitRate = %f, want 0.0 for zero counts", stats.HitRate) t.Errorf("HitRate = %f, want 0.0 for zero counts", stats.HitRate)
} }
} }
// TestStats_LogsFailedCountQueries verifies that a failed item count query
// and a failed size query are each logged at warn and Stats still succeeds.
func TestStats_LogsFailedCountQueries(t *testing.T) {
t.Parallel()
db := setupStatsTestDB(t)
var logBuf bytes.Buffer
cache, err := NewCache(db, CacheConfig{
StateDir: t.TempDir(),
CacheTTL: time.Hour,
NegativeTTL: 5 * time.Minute,
Logger: slog.New(slog.NewJSONHandler(&logBuf, nil)),
})
if err != nil {
t.Fatal(err)
}
_, err = db.ExecContext(t.Context(),
`DROP TABLE request_cache; DROP TABLE output_content`)
if err != nil {
t.Fatal(err)
}
_, err = cache.Stats(t.Context())
if err != nil {
t.Fatalf("Stats() error = %v, want nil", err)
}
for _, msg := range []string{
"failed to count cache items for stats",
"failed to sum cache size for stats",
} {
want := `"level":"WARN","msg":"` + msg + `"`
if !strings.Contains(logBuf.String(), want) {
t.Errorf("log missing %s; got %q", want, logBuf.String())
}
}
}
// TestIncrementStats_LogsFailedUpdates verifies that a failed hit or miss
// count update and a failed upstream fetch count update are each logged at
// warn.
func TestIncrementStats_LogsFailedUpdates(t *testing.T) {
t.Parallel()
db := setupStatsTestDB(t)
var logBuf bytes.Buffer
cache, err := NewCache(db, CacheConfig{
StateDir: t.TempDir(),
CacheTTL: time.Hour,
NegativeTTL: 5 * time.Minute,
Logger: slog.New(slog.NewJSONHandler(&logBuf, nil)),
})
if err != nil {
t.Fatal(err)
}
_, err = db.ExecContext(t.Context(), `DROP TABLE cache_stats`)
if err != nil {
t.Fatal(err)
}
cache.IncrementStats(t.Context(), false, 1024)
for _, msg := range []string{
"failed to count cache hit or miss",
"failed to count upstream fetch",
} {
want := `"level":"WARN","msg":"` + msg + `"`
if !strings.Contains(logBuf.String(), want) {
t.Errorf("log missing %s; got %q", want, logBuf.String())
}
}
}
+10 -3
View File
@@ -7,6 +7,7 @@ import (
"errors" "errors"
"fmt" "fmt"
"io" "io"
"log/slog"
"os" "os"
"path/filepath" "path/filepath"
"time" "time"
@@ -392,6 +393,7 @@ func CacheKey(req *ImageRequest) VariantKey {
// Unlike ContentStorage, the key is provided by the caller (not computed from content). // Unlike ContentStorage, the key is provided by the caller (not computed from content).
type VariantStorage struct { type VariantStorage struct {
baseDir string baseDir string
log *slog.Logger
} }
// VariantMeta contains metadata about a cached variant. // VariantMeta contains metadata about a cached variant.
@@ -404,13 +406,14 @@ type VariantMeta struct {
} }
// NewVariantStorage creates a new variant storage at the given base directory. // NewVariantStorage creates a new variant storage at the given base directory.
func NewVariantStorage(baseDir string) (*VariantStorage, error) { // A failed .meta write is logged to log.
func NewVariantStorage(baseDir string, log *slog.Logger) (*VariantStorage, error) {
err := os.MkdirAll(baseDir, StorageDirPerm) err := os.MkdirAll(baseDir, StorageDirPerm)
if err != nil { if err != nil {
return nil, fmt.Errorf("failed to create variant storage directory: %w", err) return nil, fmt.Errorf("failed to create variant storage directory: %w", err)
} }
return &VariantStorage{baseDir: baseDir}, nil return &VariantStorage{baseDir: baseDir, log: log}, nil
} }
// Store writes content and metadata to storage at the given key. // Store writes content and metadata to storage at the given key.
@@ -478,7 +481,11 @@ func (s *VariantStorage) Store(
} }
// Metadata write failure is non-fatal; content is already stored. // Metadata write failure is non-fatal; content is already stored.
_ = os.WriteFile(metaPath, metaData, StorageFilePerm) err = os.WriteFile(metaPath, metaData, StorageFilePerm)
if err != nil {
s.log.Warn("failed to write variant metadata sidecar",
"path", metaPath, "error", err)
}
return size, nil return size, nil
} }
@@ -4,8 +4,10 @@ import (
"bytes" "bytes"
"errors" "errors"
"io" "io"
"log/slog"
"os" "os"
"path/filepath" "path/filepath"
"strings"
"testing" "testing"
) )
@@ -404,3 +406,35 @@ func TestCacheKey(t *testing.T) {
t.Error("CacheKey() produced same key for different quality") t.Error("CacheKey() produced same key for different quality")
} }
} }
// TestVariantStorage_StoreLogsFailedMetaWrite verifies that a .meta write
// that fails is logged at warn and the store still succeeds.
func TestVariantStorage_StoreLogsFailedMetaWrite(t *testing.T) {
t.Parallel()
var logBuf bytes.Buffer
storage, err := NewVariantStorage(
t.TempDir(), slog.New(slog.NewJSONHandler(&logBuf, nil)))
if err != nil {
t.Fatalf("NewVariantStorage() error = %v", err)
}
key := CacheKey(&ImageRequest{SourceHost: testHostCDN, SourcePath: testPathCat})
// A directory where the .meta file goes makes the .meta write fail.
err = os.MkdirAll(storage.keyToPath(key)+".meta", StorageDirPerm)
if err != nil {
t.Fatalf("failed to create directory: %v", err)
}
_, err = storage.Store(key, bytes.NewReader([]byte("variant data")), "image/webp")
if err != nil {
t.Fatalf("Store() error = %v, want nil", err)
}
want := `"level":"WARN","msg":"failed to write variant metadata sidecar"`
if !strings.Contains(logBuf.String(), want) {
t.Errorf("log missing %s; got %q", want, logBuf.String())
}
}
+25 -4
View File
@@ -23,6 +23,21 @@ var (
// MaxDimension is the maximum allowed width or height. // MaxDimension is the maximum allowed width or height.
const MaxDimension = 8192 const MaxDimension = 8192
// ValidateDimension checks one requested width or height; name ("width" or
// "height") appears in the error. 0 means "original size" and is valid.
func ValidateDimension(name string, value int) error {
if value < 0 {
return fmt.Errorf("%w: %s is negative", ErrInvalidSize, name)
}
if value > MaxDimension {
return fmt.Errorf("%w: %s is above %d",
ErrDimensionTooLarge, name, MaxDimension)
}
return nil
}
// sizeFormatRegex matches patterns like "800x600.webp", "0x0.jpeg", "orig.png" // sizeFormatRegex matches patterns like "800x600.webp", "0x0.jpeg", "orig.png"
var sizeFormatRegex = regexp.MustCompile(`^(\d+)x(\d+)\.(\w+)$|^(orig)\.(\w+)$`) var sizeFormatRegex = regexp.MustCompile(`^(\d+)x(\d+)\.(\w+)$|^(orig)\.(\w+)$`)
@@ -225,14 +240,20 @@ func parseSizeFormat(s string) (Size, ImageFormat, error) {
return Size{}, "", ErrInvalidSize return Size{}, "", ErrInvalidSize
} }
if width > MaxDimension || height > MaxDimension {
return Size{}, "", ErrDimensionTooLarge
}
size = Size{Width: width, Height: height} size = Size{Width: width, Height: height}
formatStr = matches[3] formatStr = matches[3]
} }
err := ValidateDimension("width", size.Width)
if err != nil {
return Size{}, "", err
}
err = ValidateDimension("height", size.Height)
if err != nil {
return Size{}, "", err
}
format, err := parseFormat(formatStr) format, err := parseFormat(formatStr)
if err != nil { if err != nil {
return Size{}, "", err return Size{}, "", err
@@ -0,0 +1,64 @@
package imgcache
import (
"errors"
"testing"
)
func TestValidateImageRequest(t *testing.T) {
t.Parallel()
tests := []struct {
name string
req ImageRequest
wantErr error
}{
{
name: "within bounds",
req: ImageRequest{Size: Size{Width: 800, Height: 600}, FitMode: FitCover},
},
{
name: "original size and empty fit",
req: ImageRequest{Size: Size{Width: 0, Height: 0}},
},
{
name: "width over limit",
req: ImageRequest{Size: Size{Width: MaxDimension + 1, Height: 600}},
wantErr: ErrDimensionTooLarge,
},
{
name: "height over limit",
req: ImageRequest{Size: Size{Width: 800, Height: MaxDimension + 1}},
wantErr: ErrDimensionTooLarge,
},
{
name: "negative width",
req: ImageRequest{Size: Size{Width: -1, Height: 600}},
wantErr: ErrInvalidSize,
},
{
name: "invalid fit mode",
req: ImageRequest{Size: Size{Width: 800, Height: 600}, FitMode: "bogus"},
wantErr: ErrInvalidFitMode,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
err := ValidateImageRequest(&tt.req)
if tt.wantErr == nil {
if err != nil {
t.Fatalf("ValidateImageRequest() error = %v, want nil", err)
}
return
}
if !errors.Is(err, tt.wantErr) {
t.Fatalf("ValidateImageRequest() error = %v, want %v", err, tt.wantErr)
}
})
}
}
+27
View File
@@ -4,11 +4,13 @@ package middleware
import ( import (
"log/slog" "log/slog"
"net/http" "net/http"
"net/netip"
"time" "time"
basicauth "github.com/99designs/basicauth-go" basicauth "github.com/99designs/basicauth-go"
"github.com/go-chi/chi/v5/middleware" "github.com/go-chi/chi/v5/middleware"
"github.com/go-chi/cors" "github.com/go-chi/cors"
"github.com/go-chi/httprate"
metrics "github.com/slok/go-http-metrics/metrics/prometheus" metrics "github.com/slok/go-http-metrics/metrics/prometheus"
ghmm "github.com/slok/go-http-metrics/middleware" ghmm "github.com/slok/go-http-metrics/middleware"
"github.com/slok/go-http-metrics/middleware/std" "github.com/slok/go-http-metrics/middleware/std"
@@ -88,6 +90,31 @@ func (s *Middleware) ClientIP() func(http.Handler) http.Handler {
} }
} }
// RateLimit returns a middleware that limits each client to requestLimit
// requests per window and refuses a request over the limit with 429 Too Many
// Requests and a Retry-After header. Clients are told apart by the address
// the ClientIP middleware stored in the request context, so ClientIP must
// run first. An IPv6 client is counted by its /64, which one client usually
// holds whole; an IPv4-mapped address (::ffff:a.b.c.d) is counted as the
// IPv4 address it carries, since every such address falls in the same /64.
// Counts are kept only for the current and the previous window, so memory
// stays bounded.
func (s *Middleware) RateLimit(
requestLimit int, window time.Duration,
) func(http.Handler) http.Handler {
return httprate.LimitBy(requestLimit, window,
func(r *http.Request) (string, error) {
ip := clientip.FromContext(r.Context())
addr, err := netip.ParseAddr(ip)
if err == nil {
ip = addr.Unmap().String()
}
return httprate.CanonicalizeIP(ip), nil
})
}
type loggingResponseWriter struct { type loggingResponseWriter struct {
http.ResponseWriter http.ResponseWriter
@@ -0,0 +1,280 @@
package server
import (
"io"
"net/http"
"net/http/httptest"
"net/netip"
"net/url"
"path/filepath"
"regexp"
"strconv"
"strings"
"testing"
"go.uber.org/fx/fxtest"
"sneak.berlin/go/pixa/internal/config"
"sneak.berlin/go/pixa/internal/database"
"sneak.berlin/go/pixa/internal/globals"
"sneak.berlin/go/pixa/internal/handlers"
"sneak.berlin/go/pixa/internal/logger"
"sneak.berlin/go/pixa/internal/middleware"
)
// testSigningKey is a throwaway signing key; submitting it logs in.
const testSigningKey = "test-signing-key-0123456789abcdef"
// wrongKey is submitted for a failed login.
const wrongKey = "not-the-signing-key"
// Addresses for the login rate limit tests. The test server trusts
// 10.0.0.0/8 as its proxies, so the X-Forwarded-For sent by proxyPeer is
// believed and the one sent by firstClient or secondClient is ignored.
const (
firstClient = "198.51.100.1:40000"
secondClient = "198.51.100.2:40000"
proxyPeer = "10.0.0.1:40000"
firstForwarded = "203.0.113.1"
secondForwarded = "203.0.113.2"
)
// csrfFieldPattern extracts the CSRF token rendered into the login form.
var csrfFieldPattern = regexp.MustCompile(
`name="gorilla\.csrf\.Token" value="([^"]+)"`)
// newTestServer builds the server's real routes from the constructors
// cmd/pixad uses, with a throwaway state directory. Debug marks requests
// as plain HTTP, so the CSRF check runs without an https Referer.
func newTestServer(t *testing.T) *Server {
t.Helper()
stateDir := t.TempDir()
cfg := &config.Config{
Debug: true,
SigningKey: testSigningKey,
StateDir: stateDir,
DBURL: "file:" + filepath.Join(stateDir, "state.sqlite3"),
TrustedProxies: []netip.Prefix{netip.MustParsePrefix("10.0.0.0/8")},
}
lc := fxtest.NewLifecycle(t)
log, err := logger.New(lc, logger.Params{Globals: &globals.Globals{}})
if err != nil {
t.Fatalf("logger.New() error = %v", err)
}
db, err := database.New(lc, database.Params{Logger: log, Config: cfg})
if err != nil {
t.Fatalf("database.New() error = %v", err)
}
h, err := handlers.New(lc, handlers.Params{
Logger: log, Database: db, Config: cfg,
})
if err != nil {
t.Fatalf("handlers.New() error = %v", err)
}
mw, err := middleware.New(lc, middleware.Params{Logger: log, Config: cfg})
if err != nil {
t.Fatalf("middleware.New() error = %v", err)
}
lc.RequireStart()
t.Cleanup(lc.RequireStop)
s := &Server{config: cfg, mw: mw, h: h}
s.SetupRoutes()
return s
}
// clientRequest builds a request for / arriving from remoteAddr, carrying
// forwardedFor as its X-Forwarded-For header when that is not empty.
func clientRequest(
t *testing.T, method string, body io.Reader, remoteAddr, forwardedFor string,
) *http.Request {
t.Helper()
req := httptest.NewRequestWithContext(t.Context(), method, "/", body)
req.RemoteAddr = remoteAddr
if forwardedFor != "" {
req.Header.Set("X-Forwarded-For", forwardedFor)
}
return req
}
// postLogin loads the login form with GET / and submits key in it with
// POST /, as a browser does, both from the same client. GET / is not rate
// limited, so the form must load even for a client over the limit.
func postLogin(
t *testing.T, s *Server, remoteAddr, forwardedFor, key string,
) *httptest.ResponseRecorder {
t.Helper()
page := httptest.NewRecorder()
s.ServeHTTP(page,
clientRequest(t, http.MethodGet, nil, remoteAddr, forwardedFor))
if page.Code != http.StatusOK {
t.Fatalf("GET / status = %d, want %d", page.Code, http.StatusOK)
}
match := csrfFieldPattern.FindStringSubmatch(page.Body.String())
if match == nil {
t.Fatalf("no CSRF token field found in the login form")
}
form := url.Values{"key": {key}, "gorilla.csrf.Token": {match[1]}}
req := clientRequest(t, http.MethodPost,
strings.NewReader(form.Encode()), remoteAddr, forwardedFor)
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
for _, c := range page.Result().Cookies() {
req.AddCookie(c)
}
rec := httptest.NewRecorder()
s.ServeHTTP(rec, req)
return rec
}
// tripLoginRateLimit makes LoginAttemptsPerMinute failed logins from one
// client, each answered with the login form again, then one more, which
// must be refused with 429. It returns the response to that last attempt.
func tripLoginRateLimit(
t *testing.T, s *Server, remoteAddr, forwardedFor string,
) *httptest.ResponseRecorder {
t.Helper()
for attempt := range LoginAttemptsPerMinute {
rec := postLogin(t, s, remoteAddr, forwardedFor, wrongKey)
if rec.Code != http.StatusOK {
t.Fatalf("failed login %d status = %d, want %d",
attempt+1, rec.Code, http.StatusOK)
}
}
rec := postLogin(t, s, remoteAddr, forwardedFor, wrongKey)
if rec.Code != http.StatusTooManyRequests {
t.Fatalf("login over the limit status = %d, want %d",
rec.Code, http.StatusTooManyRequests)
}
return rec
}
// TestLoginRateLimitRefusesAttemptOverLimit verifies the login attempt
// after LoginAttemptsPerMinute failed ones from one client is refused with
// 429 and a Retry-After header, and that the client cannot get around the
// limit by sending X-Forwarded-For: from a peer that is not a trusted
// proxy, the header is ignored.
func TestLoginRateLimitRefusesAttemptOverLimit(t *testing.T) {
t.Parallel()
s := newTestServer(t)
rec := tripLoginRateLimit(t, s, firstClient, "")
retryAfter := rec.Header().Get("Retry-After")
seconds, err := strconv.Atoi(retryAfter)
if err != nil || seconds <= 0 {
t.Errorf("Retry-After = %q, want a positive number of seconds",
retryAfter)
}
rec = postLogin(t, s, firstClient, secondForwarded, wrongKey)
if rec.Code != http.StatusTooManyRequests {
t.Errorf("login with X-Forwarded-For from an untrusted peer "+
"status = %d, want %d", rec.Code, http.StatusTooManyRequests)
}
}
// TestLoginRateLimitLeavesOtherClientsAlone verifies one client going over
// the limit does not limit another: a failed login from a different
// address is answered with the login form, and the signing key still logs
// it in.
func TestLoginRateLimitLeavesOtherClientsAlone(t *testing.T) {
t.Parallel()
s := newTestServer(t)
tripLoginRateLimit(t, s, firstClient, "")
rec := postLogin(t, s, secondClient, "", wrongKey)
if rec.Code != http.StatusOK {
t.Errorf("failed login from another client status = %d, want %d",
rec.Code, http.StatusOK)
}
rec = postLogin(t, s, secondClient, "", testSigningKey)
if rec.Code != http.StatusSeeOther {
t.Errorf("login with the signing key from another client "+
"status = %d, want %d", rec.Code, http.StatusSeeOther)
}
}
// TestLoginRateLimitCountsClientsBehindProxySeparately verifies the limit
// counts the client address resolved from X-Forwarded-For, not the address
// of the trusted proxy the requests arrive from, so two clients behind the
// same proxy are counted separately.
func TestLoginRateLimitCountsClientsBehindProxySeparately(t *testing.T) {
t.Parallel()
s := newTestServer(t)
tripLoginRateLimit(t, s, proxyPeer, firstForwarded)
rec := postLogin(t, s, proxyPeer, secondForwarded, wrongKey)
if rec.Code != http.StatusOK {
t.Errorf("failed login from a second client behind the proxy "+
"status = %d, want %d", rec.Code, http.StatusOK)
}
}
// TestLoginRateLimitCountsIPv6ClientsByPrefix verifies an IPv6 client is
// counted by its /64: another address in the same /64 is refused too,
// while an address in a different /64 is not.
func TestLoginRateLimitCountsIPv6ClientsByPrefix(t *testing.T) {
t.Parallel()
s := newTestServer(t)
tripLoginRateLimit(t, s, proxyPeer, "2001:db8::1")
rec := postLogin(t, s, proxyPeer, "2001:db8::2", wrongKey)
if rec.Code != http.StatusTooManyRequests {
t.Errorf("login from the same /64 status = %d, want %d",
rec.Code, http.StatusTooManyRequests)
}
rec = postLogin(t, s, proxyPeer, "2001:db8:0:1::1", wrongKey)
if rec.Code != http.StatusOK {
t.Errorf("login from another /64 status = %d, want %d",
rec.Code, http.StatusOK)
}
}
// TestLoginRateLimitCountsIPv4MappedClientsSeparately verifies an IPv4
// client that the proxy forwards in IPv4-mapped IPv6 form (::ffff:a.b.c.d)
// is counted by its IPv4 address, not by the /64 that every such address
// shares, so two of them behind the proxy are counted separately.
func TestLoginRateLimitCountsIPv4MappedClientsSeparately(t *testing.T) {
t.Parallel()
s := newTestServer(t)
tripLoginRateLimit(t, s, proxyPeer, "::ffff:"+firstForwarded)
rec := postLogin(t, s, proxyPeer, "::ffff:"+secondForwarded, wrongKey)
if rec.Code != http.StatusOK {
t.Errorf("failed login from a second IPv4-mapped client "+
"status = %d, want %d", rec.Code, http.StatusOK)
}
}
+9 -1
View File
@@ -2,6 +2,7 @@ package server
import ( import (
"net/http" "net/http"
"time"
sentryhttp "github.com/getsentry/sentry-go/http" sentryhttp "github.com/getsentry/sentry-go/http"
"github.com/go-chi/chi/v5" "github.com/go-chi/chi/v5"
@@ -12,6 +13,10 @@ import (
"sneak.berlin/go/pixa/internal/static" "sneak.berlin/go/pixa/internal/static"
) )
// LoginAttemptsPerMinute is how many login attempts (POST /) one client may
// make per minute; the next is refused with 429 Too Many Requests.
const LoginAttemptsPerMinute = 5
// SetupRoutes configures all HTTP routes. // SetupRoutes configures all HTTP routes.
func (s *Server) SetupRoutes() { func (s *Server) SetupRoutes() {
s.router = chi.NewRouter() s.router = chi.NewRouter()
@@ -50,11 +55,14 @@ func (s *Server) SetupRoutes() {
// token cookie is independent of the session cookie, so it also // token cookie is independent of the session cookie, so it also
// covers the login POST, where no session exists yet. LimitBody caps // covers the login POST, where no session exists yet. LimitBody caps
// the POST body ahead of CSRF, which reads its token from that body. // the POST body ahead of CSRF, which reads its token from that body.
// The login POST is rate limited per client after both, so every
// attempt that reaches the signing key comparison is counted.
s.router.Group(func(r chi.Router) { s.router.Group(func(r chi.Router) {
r.Use(s.h.LimitBody(handlers.MaxFormBytes)) r.Use(s.h.LimitBody(handlers.MaxFormBytes))
r.Use(s.h.CSRF()) r.Use(s.h.CSRF())
r.Get("/", s.h.HandleRoot()) r.Get("/", s.h.HandleRoot())
r.Post("/", s.h.HandleRoot()) r.With(s.mw.RateLimit(LoginAttemptsPerMinute, time.Minute)).
Post("/", s.h.HandleRoot())
r.Post("/generate", s.h.HandleGenerateURL()) r.Post("/generate", s.h.HandleGenerateURL())
}) })
+2 -2
View File
@@ -73,7 +73,7 @@
id="width" id="width"
name="width" name="width"
min="0" min="0"
max="10000" max="8192"
value="{{if .FormWidth}}{{.FormWidth}}{{else}}0{{end}}" value="{{if .FormWidth}}{{.FormWidth}}{{else}}0{{end}}"
placeholder="0 = original" placeholder="0 = original"
class="w-full px-3 py-2 border border-gray-300 rounded-md shadow-sm focus:outline-none focus:ring-2 focus:ring-blue-500 focus:border-blue-500" class="w-full px-3 py-2 border border-gray-300 rounded-md shadow-sm focus:outline-none focus:ring-2 focus:ring-blue-500 focus:border-blue-500"
@@ -88,7 +88,7 @@
id="height" id="height"
name="height" name="height"
min="0" min="0"
max="10000" max="8192"
value="{{if .FormHeight}}{{.FormHeight}}{{else}}0{{end}}" value="{{if .FormHeight}}{{.FormHeight}}{{else}}0{{end}}"
placeholder="0 = original" placeholder="0 = original"
class="w-full px-3 py-2 border border-gray-300 rounded-md shadow-sm focus:outline-none focus:ring-2 focus:ring-blue-500 focus:border-blue-500" class="w-full px-3 py-2 border border-gray-300 rounded-md shadow-sm focus:outline-none focus:ring-2 focus:ring-blue-500 focus:border-blue-500"
+5 -58
View File
@@ -3,20 +3,14 @@
# this repo. Idempotent: every install is guarded by a check so already # this repo. Idempotent: every install is guarded by a check so already
# installed tools are skipped. Base tooling comes from nix, apt, brew, # installed tools are skipped. Base tooling comes from nix, apt, brew,
# or apk (detected in that order); assumes NOTHING is present (not git, # or apk (detected in that order); assumes NOTHING is present (not git,
# make, or go). golangci-lint is packaged in nix, brew, and apk; on apt # make, or go). The linter is never installed on the host: golangci-lint
# it is installed from a hash-verified GitHub release archive (never # runs only inside a container, Dockerfile.lint or the Dockerfile lint
# curl | sh). CGO image libraries (pkg-config, vips, libheif) are # stage (see script/lint). CGO image libraries (pkg-config, vips,
# installed for the govips bindings. # libheif) are installed for the govips bindings.
set -eu set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)" ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
# Pinned versions, 2026-08-07. Never "latest"; exact versions only.
GOLANGCI_LINT_VERSION="2.12.2"
# sha256 of golangci-lint-2.12.2-linux-<arch>.tar.gz release archives
GOLANGCI_LINT_SHA256_AMD64="8df580d2670fed8fa984aac0507099af8df275e665215f5c7a2ae3943893a553"
GOLANGCI_LINT_SHA256_ARM64="44cd40a8c76c86755375adfeea52cfd3533cb43d7bd647771e0ae065e166df3a"
PKGMGR="" PKGMGR=""
SUDO="" SUDO=""
@@ -57,52 +51,6 @@ missing() {
! command -v "$1" >/dev/null 2>&1 ! command -v "$1" >/dev/null 2>&1
} }
# verify_sha256 <file> <expected-hash>
verify_sha256() {
if command -v sha256sum >/dev/null 2>&1; then
actual="$(sha256sum "$1" | cut -d' ' -f1)"
else
actual="$(shasum -a 256 "$1" | cut -d' ' -f1)"
fi
if [ "$actual" != "$2" ]; then
echo "bootstrap: sha256 mismatch for $1" >&2
echo " expected: $2" >&2
echo " actual: $actual" >&2
exit 1
fi
}
# apt has no golangci-lint package: install a pinned release archive
# from GitHub, verified by hardcoded sha256 (never curl | sh).
install_golangci_lint_release() {
case "$(uname -m)" in
x86_64) goarch="amd64"; sha="$GOLANGCI_LINT_SHA256_AMD64" ;;
aarch64|arm64) goarch="arm64"; sha="$GOLANGCI_LINT_SHA256_ARM64" ;;
*)
echo "bootstrap: unsupported architecture $(uname -m)" >&2
exit 1
;;
esac
if missing curl; then pkg_install curl curl curl curl; fi
name="golangci-lint-${GOLANGCI_LINT_VERSION}-linux-${goarch}"
tmp="$(mktemp -d)"
curl -fsSL -o "$tmp/$name.tar.gz" \
"https://github.com/golangci/golangci-lint/releases/download/v${GOLANGCI_LINT_VERSION}/${name}.tar.gz"
verify_sha256 "$tmp/$name.tar.gz" "$sha"
tar -xzf "$tmp/$name.tar.gz" -C "$tmp"
$SUDO install -m 0755 "$tmp/$name/golangci-lint" /usr/local/bin/golangci-lint
rm -rf "$tmp"
}
ensure_golangci_lint() {
if ! missing golangci-lint; then return 0; fi
detect_pkgmgr
case "$PKGMGR" in
apt) install_golangci_lint_release ;;
*) pkg_install golangci-lint golangci-lint golangci-lint golangci-lint ;;
esac
}
# CGO dependencies for govips (image processing) # CGO dependencies for govips (image processing)
ensure_cgo_deps() { ensure_cgo_deps() {
if missing pkg-config; then if missing pkg-config; then
@@ -123,9 +71,8 @@ main() {
if missing git; then pkg_install git git git git; fi if missing git; then pkg_install git git git git; fi
if missing make; then pkg_install gnumake make make make; fi if missing make; then pkg_install gnumake make make make; fi
# Go toolchain and linter # Go toolchain
if missing go; then pkg_install go golang go go; fi if missing go; then pkg_install go golang go go; fi
ensure_golangci_lint
# CGO image libraries # CGO image libraries
ensure_cgo_deps ensure_cgo_deps
+4 -2
View File
@@ -16,8 +16,10 @@ main() {
# removed on exit. # removed on exit.
key="$(head -c 32 /dev/urandom | base64)" key="$(head -c 32 /dev/urandom | base64)"
# --health-interval=1s overrides the image's 30s interval so the # Docker 25 and later probe every 5 seconds during the image's
# first probe does not use up the whole wait. # 10-second start period. --health-interval=1s makes it probe every
# second after that instead of every 30, so a slow start still shows
# as healthy within the 30-second wait.
cid="$(docker create --health-interval=1s \ cid="$(docker create --health-interval=1s \
-e PIXA_SIGNING_KEY="$key" "$("$SCRIPT_DIR/projectname")")" -e PIXA_SIGNING_KEY="$key" "$("$SCRIPT_DIR/projectname")")"
# Remove the container on any exit; turning signals into exit makes # Remove the container on any exit; turning signals into exit makes
+27 -13
View File
@@ -1,23 +1,37 @@
#!/bin/sh #!/bin/sh
# script/lint: run the linter. CGO dependencies (pkg-config, vips, # script/lint: run golangci-lint over the whole tree. This is the only
# libheif) come from nix-shell when not already available (e.g. inside # way the linter is run, everywhere; it is never installed on the host.
# a Docker build or an existing nix-shell). #
# Inside a container it runs the linter. Anywhere else it builds
# Dockerfile.lint, whose last step runs this script again inside that
# container.
#
# Dockerfile.lint and the Dockerfile lint stage set container=docker
# (the systemd convention for marking a container) to say where we are.
# /.dockerenv cannot: it is missing inside build steps, and present on
# hosts that are themselves containers.
set -eu set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)" ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
run_with_cgo_deps() { main() {
if command -v pkg-config >/dev/null 2>&1; then cd "$ROOT"
sh -c "$1" if [ "${container:-}" = docker ]; then
# `golangci-lint config verify` is not run: it fetches its JSON
# schema over an unpinned live HTTPS call, which REPO_POLICIES.md
# forbids.
echo "Running linter..."
golangci-lint run --config .golangci.yml ./...
else else
nix-shell -p pkg-config vips libheif golangci-lint git --run "$1" # A new CACHEBUST on every run means the lint step is never
# served from cache (see Dockerfile.lint). The cacheonly output
# leaves no image behind.
docker build \
--progress=plain \
--build-arg CACHEBUST="$(date +%s)-$$" \
--output=type=cacheonly \
-f Dockerfile.lint .
fi fi
} }
main() {
cd "$ROOT"
echo "Running linter..."
run_with_cgo_deps "golangci-lint run"
}
main "$@" main "$@"
+1 -1
View File
@@ -10,7 +10,7 @@ run_with_cgo_deps() {
if command -v pkg-config >/dev/null 2>&1; then if command -v pkg-config >/dev/null 2>&1; then
sh -c "$1" sh -c "$1"
else else
nix-shell -p pkg-config vips libheif golangci-lint git --run "$1" nix-shell -p pkg-config vips libheif git --run "$1"
fi fi
} }