4 Commits
Author SHA1 Message Date
sneak 9b49a9d5b3 Say that a config file's env: section overrides the environment (closes #128)
check / check (push) Successful in 3m22s
A variable named in the config file's env: section is set while the
file loads, so it overrides both the environment the process was started
with and the file's own key. README.md and the config.example.yml header
said only that a variable wins over the file; they now state this
exception where the precedence is given.

Model: opus-5-5
2026-09-28 11:10:23 +00:00
sneak a7947da6f1 Take every setting from PIXA_ variables and PORT (closes #128)
Each config key can now be set by PIXA_ plus the key in upper case
("." written as "_"), and the port by PORT. One list pairs keys with
variables; the typed getters read a present variable, even an empty
one, before the config file, so every existing check covers it, and
errors a variable can reach name both the key and the variable. An
empty string for blocked_networks or trusted_proxies is now an empty
list, as for allowlist_hosts. The image no longer bakes in
config.docker.yml or passes --config, and its HEALTHCHECK probes
${PORT:-8080}; the config file is looked for under /etc/pixa rather
than /etc/pixad, so a file mounted at /etc/pixa/config.yml is still
read.

Model: opus-5-5
2026-09-28 11:10:23 +00:00
sneak 7dc0ad7902 test: every setting from its environment variable (closes #128)
Tests, written before the change, for the PIXA_ variables and PORT:
every key set from the environment with no config file, PORT over the
file's port, a list variable replacing the file's list, an empty
variable as a set value, invalid values aborting startup naming the
variable, and the signing key and metrics password never printed. All
fail until the change lands, except the check that a config file alone
behaves as before. TestMain unsets PORT and every PIXA_ variable so the
shell running the tests cannot change their result.

Model: opus-5-5
2026-09-28 11:10:07 +00:00
clawbot db784bf561 Include quality and fit in the URL signature (closes #60)
check / check (push) Successful in 12s
The signed data is now
host:path:query:width:height:format:expiration:quality:fit. The route
turns a missing q into 85 and a missing fit into cover before checking
the signature, so those are the values signed for a URL without them;
imgcache fills both from the parsed request.

imgcache.Service.GenerateSignedURL now writes q and fit into the URL
next to sig and exp, first setting an unset quality or fit to 85 or
cover, so a generated URL verifies for the values it signed.

The known-answer vectors in golden_test.go, including one for quality
40 and fit contain, and the README signature section describe the new
format.

Model: opus-4-8 (implementation); opus-5-5 (rework)
2026-09-28 13:02:21 +02:00
13 changed files with 733 additions and 152 deletions
+7 -6
View File
@@ -67,16 +67,17 @@ RUN adduser -D -H -s /sbin/nologin pixad && \
mkdir -p /var/lib/pixa /etc/pixa && \ mkdir -p /var/lib/pixa /etc/pixa && \
chown pixad:pixad /var/lib/pixa chown pixad:pixad /var/lib/pixa
# Copy the image config; signing_key comes from PIXA_SIGNING_KEY.
# Mount a file over /etc/pixa/config.yml to override anything else.
COPY config.docker.yml /etc/pixa/config.yml
USER pixad USER pixad
WORKDIR /var/lib/pixa WORKDIR /var/lib/pixa
EXPOSE 8080 EXPOSE 8080
# Shell form so the probe follows PORT; a port set only in a mounted
# config file is not seen here.
HEALTHCHECK --interval=30s --timeout=5s --start-period=10s --retries=3 \ HEALTHCHECK --interval=30s --timeout=5s --start-period=10s --retries=3 \
CMD wget --spider -q http://localhost:8080/.well-known/healthcheck.json || exit 1 CMD wget --spider -q "http://localhost:${PORT:-8080}/.well-known/healthcheck.json" || exit 1
ENTRYPOINT ["/usr/local/bin/pixad", "--config", "/etc/pixa/config.yml"] # Settings come from PORT and the PIXA_ environment variables; only
# PIXA_SIGNING_KEY is required. A config file mounted at
# /etc/pixa/config.yml is optional and is read when present.
ENTRYPOINT ["/usr/local/bin/pixad"]
+51 -15
View File
@@ -27,12 +27,12 @@ make docker
docker run -p 8080:8080 -e PIXA_SIGNING_KEY="$(openssl rand -base64 32)" pixa:latest docker run -p 8080:8080 -e PIXA_SIGNING_KEY="$(openssl rand -base64 32)" pixa:latest
``` ```
A container is configured two ways. The signing key comes from the A container takes its settings from environment variables (see
`PIXA_SIGNING_KEY` environment variable, which the baked-in config Configuration below for the list). Only `PIXA_SIGNING_KEY` is required; if
reads; if it is unset the container exits at startup naming the it is unset the container exits at startup naming the variable. Everything
variable. Everything else uses built-in defaults, so to change any else has a built-in default. A config file mounted at `/etc/pixa/config.yml`
other setting mount your own file over `/etc/pixa/config.yml` (see is optional: it is read when present, and an environment variable wins over
`config.example.yml` for the full set of keys). the same setting in it.
## Rationale ## Rationale
@@ -79,14 +79,14 @@ hosts require an HMAC-SHA256 signature.
Signatures use HMAC-SHA256 and include an expiration timestamp to Signatures use HMAC-SHA256 and include an expiration timestamp to
prevent replay attacks. Signatures are **exact match only**: every prevent replay attacks. Signatures are **exact match only**: every
component (host, path, query, dimensions, format, expiration) must component (host, path, query, dimensions, format, expiration, quality,
match exactly what was signed. No suffix matching, wildcard matching, fit) must match exactly what was signed. No suffix matching, wildcard
or partial matching is supported. matching, or partial matching is supported.
**Signed data format** (colon-separated): **Signed data format** (colon-separated):
``` ```
HMAC-SHA256(secret, "host:path:query:width:height:format:expiration") HMAC-SHA256(secret, "host:path:query:width:height:format:expiration:quality:fit")
``` ```
Where: Where:
@@ -98,18 +98,25 @@ Where:
- `height` — requested height in pixels, `0` for original - `height` — requested height in pixels, `0` for original
- `format` — output format (jpeg, png, webp, avif, gif, orig) - `format` — output format (jpeg, png, webp, avif, gif, orig)
- `expiration` — Unix timestamp when signature expires - `expiration` — Unix timestamp when signature expires
- `quality` — the URL's `q` query parameter (1-100), or `85` when the URL
has no `q`
- `fit` — the URL's `fit` query parameter (cover, contain, fill, inside,
outside), or `cover` when the URL has no `fit`
**Example:** resize **Example:** resize `https://cdn.example.com/photos/cat.jpg` to 800x600
`https://cdn.example.com/photos/cat.jpg` to 800x600 WebP with WebP with expiration 1704067200, default quality and fit:
expiration 1704067200:
1. Build input: 1. Build input:
`cdn.example.com:/photos/cat.jpg::800:600:webp:1704067200` `cdn.example.com:/photos/cat.jpg::800:600:webp:1704067200:85:cover`
2. Compute HMAC-SHA256 with your secret key 2. Compute HMAC-SHA256 with your secret key
3. Base64URL-encode the result 3. Base64URL-encode the result
4. URL: 4. URL:
`/v1/image/cdn.example.com/photos/cat.jpg/800x600.webp?sig=<base64url>&exp=1704067200` `/v1/image/cdn.example.com/photos/cat.jpg/800x600.webp?sig=<base64url>&exp=1704067200`
For the same image at quality 40 with fit `contain`, the input ends in
`:40:contain` and the URL is
`/v1/image/cdn.example.com/photos/cat.jpg/800x600.webp?sig=<base64url>&exp=1704067200&q=40&fit=contain`.
**Allowlist patterns:** **Allowlist patterns:**
- **Exact match**: `cdn.example.com` — matches only that host - **Exact match**: `cdn.example.com` — matches only that host
@@ -118,7 +125,36 @@ expiration 1704067200:
### Configuration ### Configuration
Configured via YAML file (`--config`). Key settings: Every setting can be given as an environment variable, in a YAML config
file (`--config`), or both. A variable present in the environment wins over
the file, even when it is empty, and the file wins over the built-in
default. The one exception is a variable named in the file's `env:` section:
it is set while the file loads, so it overrides both the environment the
process was started with and the file's own key. A variable's value is
parsed as the same text in the file would be. The three lists take
comma-separated entries, with the spaces around each trimmed; an empty
variable is an empty list. A value that does not parse or is invalid aborts
startup, naming the variable.
| Variable | Config key | Meaning |
| ------------------------------------ | ------------------------------- | ---------------------------------------------------------------------------- |
| `PIXA_SIGNING_KEY` | `signing_key` | Required: secret for signed and encrypted URLs and login, 32+ characters |
| `PORT` | `port` | Port to listen on; default `8080` |
| `PIXA_STATE_DIR` | `state_dir` | Directory for the database and the disk cache; default `/var/lib/pixa` |
| `PIXA_DB_URL` | `db_url` | SQLite database URL; default `state.sqlite3` in the state directory |
| `PIXA_CACHE_MAX_BYTES` | `cache_max_bytes` | Disk cache limit in bytes; `0` disables it; default 75% of free space |
| `PIXA_ALLOWLIST_HOSTS` | `allowlist_hosts` | Upstream hosts served without a signature |
| `PIXA_BLOCKED_NETWORKS` | `blocked_networks` | CIDR ranges never fetched from, on top of the built-in ones |
| `PIXA_TRUSTED_PROXIES` | `trusted_proxies` | CIDR ranges of proxies whose `X-Forwarded-For` is believed; default RFC 1918 |
| `PIXA_ALLOW_HTTP` | `allow_http` | Allow plain-HTTP upstreams, for testing only; default `false` |
| `PIXA_UPSTREAM_CONNECTIONS_PER_HOST` | `upstream_connections_per_host` | Concurrent connections per upstream host; default `20` |
| `PIXA_METRICS_USERNAME` | `metrics.username` | Username for `/metrics`, which is served only when both are set |
| `PIXA_METRICS_PASSWORD` | `metrics.password` | Password for `/metrics`; set together with the username |
| `PIXA_SENTRY_DSN` | `sentry_dsn` | Sentry DSN for error reporting; empty disables it |
| `PIXA_DEBUG` | `debug` | Debug logging and plain-HTTP local development; default `false` |
| `PIXA_MAINTENANCE_MODE` | `maintenance_mode` | Maintenance flag reported by the health check; default `false` |
Key settings in more detail:
- `access_control_allow_origin` — CORS origin - `access_control_allow_origin` — CORS origin
- `allowlist_hosts` — list of allowed upstream hosts - `allowlist_hosts` — list of allowed upstream hosts
+19 -1
View File
@@ -30,6 +30,25 @@ exhaustion
# Completed Steps # Completed Steps
- 2026-09-28 every setting as an environment variable (closes #128, also
covers #99): each config key can be set by `PIXA_` plus the key in upper
case (`.` written as `_`), and the port by `PORT`; a variable present in
the environment, even empty, wins over the config file, which wins over
the default; the typed getters read the variable first, so every existing
check applies to it and a bad value aborts startup naming the variable;
lists are comma-separated, and an empty variable (or `""` in the file) is
an empty list; the Docker image no longer bakes in `config.docker.yml` or
passes `--config`, and its `HEALTHCHECK` probes `PORT` (default `8080`);
the config file is looked for under `/etc/pixa` and `~/.config/pixa`
instead of the daemon name `pixad`; documented in `README.md` and
`config.example.yml`.
- 2026-09-28 quality and fit in the URL signature (closes #60): the signed
data is now `host:path:query:width:height:format:expiration:quality:fit`,
using `85` and `cover` when the URL has no `q` or `fit`, so one signed
URL can no longer be replayed across other quality and fit values to
create unauthorized cache entries and transcodes; the known-answer
vectors in `internal/signature/golden_test.go` and the README signature
specification describe the new format.
- 2026-09-28 Docker image healthcheck (closes #111): a `HEALTHCHECK` in - 2026-09-28 Docker image healthcheck (closes #111): a `HEALTHCHECK` in
the runtime stage probing `/.well-known/healthcheck.json` with busybox the runtime stage probing `/.well-known/healthcheck.json` with busybox
`wget`; `script/docker-smoke` (`make docker-smoke`) builds the image, `wget`; `script/docker-smoke` (`make docker-smoke`) builds the image,
@@ -171,7 +190,6 @@ exhaustion
- P2: auto format selection (format=auto based on Accept header) - P2: auto format selection (format=auto based on Accept header)
- P2: configuration - P2: configuration
- add all configuration options from README - add all configuration options from README
- environment variable overrides
- YAML config file support - YAML config file support
- P2: operational - P2: operational
- optional Sentry error reporting - optional Sentry error reporting
-11
View File
@@ -1,11 +0,0 @@
# Pixa configuration baked into the Docker image.
#
# The signing key is read from the PIXA_SIGNING_KEY environment
# variable; startup aborts naming it when it is unset. Every other key
# is omitted so its default applies. Operators who need more (an
# allowlist, metrics, and so on) mount their own file over
# /etc/pixa/config.yml.
signing_key: "${ENV:PIXA_SIGNING_KEY}"
state_dir: /var/lib/pixa
port: 8080
+9
View File
@@ -1,4 +1,13 @@
# Pixa Example Configuration # Pixa Example Configuration
#
# Every key can also be set by an environment variable, which wins over
# this file: PIXA_ plus the key in upper case, with "." written as "_"
# (state_dir is PIXA_STATE_DIR, metrics.username is
# PIXA_METRICS_USERNAME). The one exception is port, which is set by
# PORT. In a variable, a list is comma-separated. A variable named in
# this file's env: section is set while the file loads, so it overrides
# both the environment the process was started with and this file's own
# key.
# Server settings # Server settings
port: 8080 port: 8080
+125 -97
View File
@@ -16,7 +16,6 @@ import (
"git.eeqj.de/sneak/smartconfig" "git.eeqj.de/sneak/smartconfig"
"go.uber.org/fx" "go.uber.org/fx"
"sneak.berlin/go/pixa/internal/globals"
"sneak.berlin/go/pixa/internal/logger" "sneak.berlin/go/pixa/internal/logger"
) )
@@ -92,7 +91,6 @@ var (
type Params struct { type Params struct {
fx.In fx.In
Globals *globals.Globals
Logger *logger.Logger Logger *logger.Logger
} }
@@ -137,24 +135,27 @@ type Config struct {
CacheMaxBytes int64 CacheMaxBytes int64
// cacheMaxBytesExplicit records whether cache_max_bytes was // cacheMaxBytesExplicit records whether cache_max_bytes was
// explicitly set in the configuration file. Explicit values are // explicitly set, in the environment or the configuration file.
// used exactly as given; only an omitted key gets the computed // Explicit values are used exactly as given; only an omitted key
// default (and its floor) in resolveCacheMaxBytes. // gets the computed default (and its floor) in resolveCacheMaxBytes.
cacheMaxBytesExplicit bool cacheMaxBytesExplicit bool
} }
// New creates a new Config instance by loading configuration from file. // New creates a new Config instance from the environment and the
// config file.
func New(_ fx.Lifecycle, params Params) (*Config, error) { func New(_ fx.Lifecycle, params Params) (*Config, error) {
log := params.Logger.Get() log := params.Logger.Get()
name := params.Globals.Appname
sc, err := loadConfigFile(log, name) // Look for the config file under the project name (/etc/pixa/,
// ~/.config/pixa/), matching the /var/lib/pixa state directory,
// not under the daemon name pixad.
sc, err := loadConfigFile(log, "pixa")
if err != nil { if err != nil {
return nil, err return nil, err
} }
if sc == nil { if sc == nil {
log.Info("no config file found, using defaults") log.Info("no config file found, using environment variables and defaults")
} }
c, err := newFromSmartConfig(sc) c, err := newFromSmartConfig(sc)
@@ -179,23 +180,24 @@ func New(_ fx.Lifecycle, params Params) (*Config, error) {
return c, nil return c, nil
} }
// newFromSmartConfig constructs a Config from a loaded smartconfig // newFromSmartConfig constructs a Config from the environment and a
// instance and validates it. A nil sc means no config file was found, // loaded smartconfig instance, and validates it. A nil sc means no
// in which case every option takes its default value. A key that is // config file was found, in which case every option the environment
// present but unparseable or invalid is an error: defaults apply only // does not set takes its default value. A key that is present but
// to omitted keys, never to invalid explicit values. // unparseable or invalid is an error: defaults apply only to omitted
// keys, never to invalid explicit values.
func newFromSmartConfig(sc *smartconfig.Config) (*Config, error) { func newFromSmartConfig(sc *smartconfig.Config) (*Config, error) {
if sc != nil { if sc != nil {
err := validateKnownKeys(sc) err := validateKnownKeys(sc)
if err != nil { if err != nil {
return nil, err return nil, err
} }
}
err = validateAllowlistHostsValue(sc) err := validateAllowlistHostsValue(sc)
if err != nil { if err != nil {
return nil, err return nil, err
} }
}
blockedNetworks, err := parseCIDRList(sc, keyBlockedNetworks) blockedNetworks, err := parseCIDRList(sc, keyBlockedNetworks)
if err != nil { if err != nil {
@@ -238,23 +240,18 @@ func newFromSmartConfig(sc *smartconfig.Config) (*Config, error) {
// The computed default for cache_max_bytes needs a validated // The computed default for cache_max_bytes needs a validated
// state_dir, so it is resolved later (resolveCacheMaxBytes); here // state_dir, so it is resolved later (resolveCacheMaxBytes); here
// we only record whether the operator set the key explicitly. // we only record whether the operator set the key explicitly.
if sc != nil { if _, present := lookupValue(sc, keyCacheMaxBytes); present {
if _, present := sc.Get(keyCacheMaxBytes); present {
c.cacheMaxBytesExplicit = true c.cacheMaxBytesExplicit = true
} }
}
// Build DBURL from StateDir if not explicitly set. The derived URL // Build DBURL from StateDir if not explicitly set. The derived URL
// is a default: it applies only when db_url is omitted, never to an // is a default: it applies only when db_url is omitted, never to an
// explicitly empty value. // explicitly empty value.
c.DBURL = loader.stringVal(keyDBURL, "") c.DBURL = loader.stringVal(keyDBURL, "")
if c.DBURL == "" && loader.err == nil { if c.DBURL == "" && loader.err == nil {
if sc != nil { if _, present := lookupValue(sc, keyDBURL); present {
if _, present := sc.Get(keyDBURL); present { return nil, fmt.Errorf("%s: %w; omit it to derive it from state_dir",
return nil, fmt.Errorf( settingName(keyDBURL), errValueEmpty)
"config key %q: %w; omit the key to derive it from state_dir",
keyDBURL, errValueEmpty)
}
} }
c.DBURL = fmt.Sprintf("file:%s/state.sqlite3?_journal_mode=WAL", c.StateDir) c.DBURL = fmt.Sprintf("file:%s/state.sqlite3?_journal_mode=WAL", c.StateDir)
@@ -356,6 +353,54 @@ func isKnownConfigKey(key string) bool {
return false return false
} }
// envVarNames returns, for each configuration key, the environment
// variable that also sets it: PIXA_ plus the key in upper case, with "."
// written as "_", except the port, which REPO_POLICIES.md requires to be
// PORT. metrics is set through its two subkeys; env has no variable.
func envVarNames() map[string]string {
return map[string]string{ //nolint:gosec // G101: variable names, not secrets
keyDebug: "PIXA_DEBUG",
keyMaintenanceMode: "PIXA_MAINTENANCE_MODE",
keyPort: "PORT",
keyStateDir: "PIXA_STATE_DIR",
keySentryDSN: "PIXA_SENTRY_DSN",
keyDBURL: "PIXA_DB_URL",
keyMetricsUsername: "PIXA_METRICS_USERNAME",
keyMetricsPassword: "PIXA_METRICS_PASSWORD",
keySigningKey: "PIXA_SIGNING_KEY",
keyAllowlistHosts: "PIXA_ALLOWLIST_HOSTS",
keyAllowHTTP: "PIXA_ALLOW_HTTP",
keyUpstreamConnectionsPerHost: "PIXA_UPSTREAM_CONNECTIONS_PER_HOST",
keyCacheMaxBytes: "PIXA_CACHE_MAX_BYTES",
keyBlockedNetworks: "PIXA_BLOCKED_NETWORKS",
keyTrustedProxies: "PIXA_TRUSTED_PROXIES",
}
}
// lookupValue returns the value set for key and whether one is set. The
// key's environment variable wins when it is present, even when empty;
// its value is a string, read exactly as the same text quoted in the
// config file would be. Otherwise the config file's value is used.
func lookupValue(sc *smartconfig.Config, key string) (any, bool) {
value, present := os.LookupEnv(envVarNames()[key])
if present {
return value, true
}
if sc == nil {
return nil, false
}
return sc.Get(key)
}
// settingName names key in an error message together with its
// environment variable, since either one may have set the value.
func settingName(key string) string {
return fmt.Sprintf("config key %q (environment variable %s)",
key, envVarNames()[key])
}
// ensureStateDirWritable verifies at startup that StateDir can be // ensureStateDirWritable verifies at startup that StateDir can be
// created and written to, so a misconfigured path aborts startup // created and written to, so a misconfigured path aborts startup
// instead of failing later at first use. // instead of failing later at first use.
@@ -364,28 +409,28 @@ func (c *Config) ensureStateDirWritable() error {
err := os.MkdirAll(c.StateDir, stateDirPerms) err := os.MkdirAll(c.StateDir, stateDirPerms)
if err != nil { if err != nil {
return fmt.Errorf("config key %q: cannot create directory %q: %w", return fmt.Errorf("%s: cannot create directory %q: %w",
keyStateDir, c.StateDir, err) settingName(keyStateDir), c.StateDir, err)
} }
probe, err := os.CreateTemp(c.StateDir, ".startup-write-probe-*") probe, err := os.CreateTemp(c.StateDir, ".startup-write-probe-*")
if err != nil { if err != nil {
return fmt.Errorf("config key %q: directory %q is not writable: %w", return fmt.Errorf("%s: directory %q is not writable: %w",
keyStateDir, c.StateDir, err) settingName(keyStateDir), c.StateDir, err)
} }
probePath := probe.Name() probePath := probe.Name()
err = probe.Close() err = probe.Close()
if err != nil { if err != nil {
return fmt.Errorf("config key %q: cannot close probe file %q: %w", return fmt.Errorf("%s: cannot close probe file %q: %w",
keyStateDir, probePath, err) settingName(keyStateDir), probePath, err)
} }
err = os.Remove(probePath) err = os.Remove(probePath)
if err != nil { if err != nil {
return fmt.Errorf("config key %q: cannot remove probe file %q: %w", return fmt.Errorf("%s: cannot remove probe file %q: %w",
keyStateDir, probePath, err) settingName(keyStateDir), probePath, err)
} }
return nil return nil
@@ -396,18 +441,19 @@ func (c *Config) ensureStateDirWritable() error {
// key value itself is never echoed in error messages. // key value itself is never echoed in error messages.
func (c *Config) validateSigningKey() error { func (c *Config) validateSigningKey() error {
if c.SigningKey == "" { if c.SigningKey == "" {
return fmt.Errorf("config key %q: %w", keySigningKey, errValueRequired) return fmt.Errorf("%s: %w", settingName(keySigningKey), errValueRequired)
} }
// Minimum key length for security (32 bytes = 256 bits) // Minimum key length for security (32 bytes = 256 bits)
const minKeyLength = 32 const minKeyLength = 32
if len(c.SigningKey) < minKeyLength { if len(c.SigningKey) < minKeyLength {
return fmt.Errorf("config key %q: %w: must be at least %d characters, got %d", return fmt.Errorf("%s: %w: must be at least %d characters, got %d",
keySigningKey, errValueTooShort, minKeyLength, len(c.SigningKey)) settingName(keySigningKey), errValueTooShort, minKeyLength,
len(c.SigningKey))
} }
if c.SigningKey == placeholderSigningKey { if c.SigningKey == placeholderSigningKey {
return fmt.Errorf("config key %q: %w", keySigningKey, errPlaceholderKey) return fmt.Errorf("%s: %w", settingName(keySigningKey), errPlaceholderKey)
} }
return nil return nil
@@ -423,25 +469,25 @@ func (c *Config) validate() error {
const maxPort = 65535 const maxPort = 65535
if c.Port < 1 || c.Port > maxPort { if c.Port < 1 || c.Port > maxPort {
return fmt.Errorf("config key %q: value %d is %w 1-%d", return fmt.Errorf("%s: value %d is %w 1-%d",
keyPort, c.Port, errPortOutOfRange, maxPort) settingName(keyPort), c.Port, errPortOutOfRange, maxPort)
} }
if c.UpstreamConnectionsPerHost < 1 { if c.UpstreamConnectionsPerHost < 1 {
return fmt.Errorf("config key %q: value %d %w", return fmt.Errorf("%s: value %d %w",
keyUpstreamConnectionsPerHost, c.UpstreamConnectionsPerHost, settingName(keyUpstreamConnectionsPerHost),
errTooFewConnections) c.UpstreamConnectionsPerHost, errTooFewConnections)
} }
if c.StateDir == "" { if c.StateDir == "" {
return fmt.Errorf("config key %q: %w", keyStateDir, errValueEmpty) return fmt.Errorf("%s: %w", settingName(keyStateDir), errValueEmpty)
} }
// Zero is valid (it disables the disk cache); only negative // Zero is valid (it disables the disk cache); only negative
// values are rejected. No floor applies to explicit values. // values are rejected. No floor applies to explicit values.
if c.CacheMaxBytes < 0 { if c.CacheMaxBytes < 0 {
return fmt.Errorf("config key %q: value %d %w", return fmt.Errorf("%s: value %d %w",
keyCacheMaxBytes, c.CacheMaxBytes, errMustNotBeNegative) settingName(keyCacheMaxBytes), c.CacheMaxBytes, errMustNotBeNegative)
} }
for _, host := range c.AllowlistHosts { for _, host := range c.AllowlistHosts {
@@ -454,14 +500,15 @@ func (c *Config) validate() error {
if c.SentryDSN != "" { if c.SentryDSN != "" {
parsed, err := url.Parse(c.SentryDSN) parsed, err := url.Parse(c.SentryDSN)
if err != nil || parsed.Scheme == "" || parsed.Host == "" { if err != nil || parsed.Scheme == "" || parsed.Host == "" {
return fmt.Errorf("config key %q: value %q is %w", return fmt.Errorf("%s: value %q is %w",
keySentryDSN, c.SentryDSN, errNotAValidURL) settingName(keySentryDSN), c.SentryDSN, errNotAValidURL)
} }
} }
if (c.MetricsUsername == "") != (c.MetricsPassword == "") { if (c.MetricsUsername == "") != (c.MetricsPassword == "") {
return fmt.Errorf("config keys %q and %q %w", return fmt.Errorf("%s and %s %w",
keyMetricsUsername, keyMetricsPassword, errMustBeSetTogether) settingName(keyMetricsUsername), settingName(keyMetricsPassword),
errMustBeSetTogether)
} }
return nil return nil
@@ -476,13 +523,13 @@ func (c *Config) validate() error {
// disable URL signing. // disable URL signing.
func validateAllowlistHost(host string) error { func validateAllowlistHost(host string) error {
if strings.Contains(host, "://") || strings.ContainsAny(host, "/ \t") { if strings.Contains(host, "://") || strings.ContainsAny(host, "/ \t") {
return fmt.Errorf("config key %q: entry %q %w", return fmt.Errorf("%s: entry %q %w",
keyAllowlistHosts, host, errNotBareHostname) settingName(keyAllowlistHosts), host, errNotBareHostname)
} }
if strings.Trim(host, ".") == "" { if strings.Trim(host, ".") == "" {
return fmt.Errorf("config key %q: entry %q %w", return fmt.Errorf("%s: entry %q %w",
keyAllowlistHosts, host, errNoHostnameLabels) settingName(keyAllowlistHosts), host, errNoHostnameLabels)
} }
return nil return nil
@@ -598,11 +645,7 @@ func (l *strictLoader) boolVal(key string, defaultVal bool) bool {
// is omitted. A present value that is not a string, or is explicitly // is omitted. A present value that is not a string, or is explicitly
// null, is an error. // null, is an error.
func getString(sc *smartconfig.Config, key, defaultVal string) (string, error) { func getString(sc *smartconfig.Config, key, defaultVal string) (string, error) {
if sc == nil { raw, ok := lookupValue(sc, key)
return defaultVal, nil
}
raw, ok := sc.Get(key)
if !ok { if !ok {
return defaultVal, nil return defaultVal, nil
} }
@@ -624,11 +667,7 @@ func getString(sc *smartconfig.Config, key, defaultVal string) (string, error) {
// omitted. A present value that is not a whole number, or is explicitly // omitted. A present value that is not a whole number, or is explicitly
// null, is an error; fractional values are never truncated. // null, is an error; fractional values are never truncated.
func getInt(sc *smartconfig.Config, key string, defaultVal int) (int, error) { func getInt(sc *smartconfig.Config, key string, defaultVal int) (int, error) {
if sc == nil { raw, ok := lookupValue(sc, key)
return defaultVal, nil
}
raw, ok := sc.Get(key)
if !ok { if !ok {
return defaultVal, nil return defaultVal, nil
} }
@@ -652,8 +691,8 @@ func getInt(sc *smartconfig.Config, key string, defaultVal int) (int, error) {
case string: case string:
parsed, err := strconv.Atoi(strings.TrimSpace(val)) parsed, err := strconv.Atoi(strings.TrimSpace(val))
if err != nil { if err != nil {
return 0, fmt.Errorf("config key %q: value %q is %w", return 0, fmt.Errorf("%s: value %q is %w",
key, val, errNotAnInteger) settingName(key), val, errNotAnInteger)
} }
return parsed, nil return parsed, nil
@@ -668,11 +707,7 @@ func getInt(sc *smartconfig.Config, key string, defaultVal int) (int, error) {
// is explicitly null, is an error; fractional values are never // is explicitly null, is an error; fractional values are never
// truncated and out-of-range values are never clamped. // truncated and out-of-range values are never clamped.
func getInt64(sc *smartconfig.Config, key string, defaultVal int64) (int64, error) { func getInt64(sc *smartconfig.Config, key string, defaultVal int64) (int64, error) {
if sc == nil { raw, ok := lookupValue(sc, key)
return defaultVal, nil
}
raw, ok := sc.Get(key)
if !ok { if !ok {
return defaultVal, nil return defaultVal, nil
} }
@@ -703,8 +738,8 @@ func getInt64(sc *smartconfig.Config, key string, defaultVal int64) (int64, erro
case string: case string:
parsed, err := strconv.ParseInt(strings.TrimSpace(val), 10, 64) parsed, err := strconv.ParseInt(strings.TrimSpace(val), 10, 64)
if err != nil { if err != nil {
return 0, fmt.Errorf("config key %q: value %q is %w", return 0, fmt.Errorf("%s: value %q is %w",
key, val, errNotAnInteger) settingName(key), val, errNotAnInteger)
} }
return parsed, nil return parsed, nil
@@ -719,11 +754,7 @@ func getInt64(sc *smartconfig.Config, key string, defaultVal int64) (int64, erro
// string), or is explicitly null, is an error; numbers are not accepted // string), or is explicitly null, is an error; numbers are not accepted
// as booleans. // as booleans.
func getBool(sc *smartconfig.Config, key string, defaultVal bool) (bool, error) { func getBool(sc *smartconfig.Config, key string, defaultVal bool) (bool, error) {
if sc == nil { raw, ok := lookupValue(sc, key)
return defaultVal, nil
}
raw, ok := sc.Get(key)
if !ok { if !ok {
return defaultVal, nil return defaultVal, nil
} }
@@ -738,8 +769,8 @@ func getBool(sc *smartconfig.Config, key string, defaultVal bool) (bool, error)
case string: case string:
parsed, err := strconv.ParseBool(strings.TrimSpace(val)) parsed, err := strconv.ParseBool(strings.TrimSpace(val))
if err != nil { if err != nil {
return false, fmt.Errorf("config key %q: value %q is %w", return false, fmt.Errorf("%s: value %q is %w",
key, val, errNotABoolean) settingName(key), val, errNotABoolean)
} }
return parsed, nil return parsed, nil
@@ -755,7 +786,7 @@ func getBool(sc *smartconfig.Config, key string, defaultVal bool) (bool, error)
// (or a comma-separated string), a non-string entry, or an empty entry // (or a comma-separated string), a non-string entry, or an empty entry
// is an error, never silently skipped. // is an error, never silently skipped.
func validateAllowlistHostsValue(sc *smartconfig.Config) error { func validateAllowlistHostsValue(sc *smartconfig.Config) error {
raw, ok := sc.Get(keyAllowlistHosts) raw, ok := lookupValue(sc, keyAllowlistHosts)
if !ok { if !ok {
return nil return nil
} }
@@ -785,8 +816,8 @@ func validateAllowlistHostsValue(sc *smartconfig.Config) error {
for part := range strings.SplitSeq(val, ",") { for part := range strings.SplitSeq(val, ",") {
if strings.TrimSpace(part) == "" { if strings.TrimSpace(part) == "" {
return fmt.Errorf("config key %q: value %q %w", return fmt.Errorf("%s: value %q %w",
keyAllowlistHosts, val, errEmptyEntry) settingName(keyAllowlistHosts), val, errEmptyEntry)
} }
} }
default: default:
@@ -802,11 +833,7 @@ func validateAllowlistHostsValue(sc *smartconfig.Config) error {
// comma-separated string (backwards compatibility). Malformed entries // comma-separated string (backwards compatibility). Malformed entries
// are rejected beforehand by validateAllowlistHostsValue. // are rejected beforehand by validateAllowlistHostsValue.
func getStringSlice(sc *smartconfig.Config) []string { func getStringSlice(sc *smartconfig.Config) []string {
if sc == nil { val, ok := lookupValue(sc, keyAllowlistHosts)
return nil
}
val, ok := sc.Get(keyAllowlistHosts)
if !ok || val == nil { if !ok || val == nil {
return nil return nil
} }
@@ -866,11 +893,7 @@ func defaultTrustedProxies() []netip.Prefix {
// aborts startup naming the key and the offending value; the default // aborts startup naming the key and the offending value; the default
// (an empty list) applies only to an omitted key. // (an empty list) applies only to an omitted key.
func parseCIDRList(sc *smartconfig.Config, key string) ([]netip.Prefix, error) { func parseCIDRList(sc *smartconfig.Config, key string) ([]netip.Prefix, error) {
if sc == nil { raw, ok := lookupValue(sc, key)
return nil, nil
}
raw, ok := sc.Get(key)
if !ok { if !ok {
return nil, nil return nil, nil
} }
@@ -889,8 +912,8 @@ func parseCIDRList(sc *smartconfig.Config, key string) ([]netip.Prefix, error) {
for _, entry := range entries { for _, entry := range entries {
prefix, err := netip.ParsePrefix(entry) prefix, err := netip.ParsePrefix(entry)
if err != nil { if err != nil {
return nil, fmt.Errorf("config key %q: value %q is %w", return nil, fmt.Errorf("%s: value %q is %w",
key, entry, errNotAValidCIDR) settingName(key), entry, errNotAValidCIDR)
} }
prefixes = append(prefixes, prefix) prefixes = append(prefixes, prefix)
@@ -901,7 +924,8 @@ func parseCIDRList(sc *smartconfig.Config, key string) ([]netip.Prefix, error) {
// cidrListEntries extracts the raw entries of the named CIDR-list key as // cidrListEntries extracts the raw entries of the named CIDR-list key as
// trimmed, non-empty strings, from either a YAML list of strings or a // trimmed, non-empty strings, from either a YAML list of strings or a
// comma-separated string. Any other shape is a configuration error. // comma-separated string; an empty string is an empty list, as for
// allowlist_hosts. Any other shape is a configuration error.
func cidrListEntries(raw any, key string) ([]string, error) { func cidrListEntries(raw any, key string) ([]string, error) {
switch val := raw.(type) { switch val := raw.(type) {
case []any: case []any:
@@ -926,11 +950,15 @@ func cidrListEntries(raw any, key string) ([]string, error) {
case string: case string:
entries := make([]string, 0) entries := make([]string, 0)
if strings.TrimSpace(val) == "" {
return entries, nil
}
for part := range strings.SplitSeq(val, ",") { for part := range strings.SplitSeq(val, ",") {
trimmed := strings.TrimSpace(part) trimmed := strings.TrimSpace(part)
if trimmed == "" { if trimmed == "" {
return nil, fmt.Errorf("config key %q: value %q %w", return nil, fmt.Errorf("%s: value %q %w",
key, val, errEmptyEntry) settingName(key), val, errEmptyEntry)
} }
entries = append(entries, trimmed) entries = append(entries, trimmed)
+255
View File
@@ -0,0 +1,255 @@
package config
import (
"net/netip"
"os"
"reflect"
"slices"
"strings"
"testing"
)
// TestMain unsets PORT and every PIXA_ environment variable before the
// tests run, so each test sees only the variables it sets itself, not
// whatever the shell running the tests exports.
func TestMain(m *testing.M) {
for _, entry := range os.Environ() {
name, _, _ := strings.Cut(entry, "=")
if name != "PORT" && !strings.HasPrefix(name, "PIXA_") {
continue
}
err := os.Unsetenv(name)
if err != nil {
panic(err)
}
}
m.Run()
}
// wantStartupError fails the test unless err is a startup error that
// mentions every one of wants.
func wantStartupError(t *testing.T, err error, wants ...string) {
t.Helper()
if err == nil {
t.Fatalf("want a startup error mentioning %q, got none", wants)
}
t.Logf("got expected error: %v", err)
for _, want := range wants {
if !strings.Contains(err.Error(), want) {
t.Errorf("error %q does not mention %q", err.Error(), want)
}
}
}
// TestEnvironmentSetsEveryKey sets every key from its environment
// variable, with no config file at all: PORT for the port, and PIXA_
// plus the key in upper case, "." written as "_", for every other key.
func TestEnvironmentSetsEveryKey(t *testing.T) {
t.Setenv("PIXA_DEBUG", "true")
t.Setenv("PIXA_MAINTENANCE_MODE", "1")
t.Setenv("PORT", "9090")
t.Setenv("PIXA_STATE_DIR", "/srv/pixa-env")
t.Setenv("PIXA_SENTRY_DSN", "https://abc123@sentry.example.com/42")
t.Setenv("PIXA_DB_URL", "file:/srv/pixa-env/other.sqlite3")
t.Setenv("PIXA_METRICS_USERNAME", "metricsuser")
t.Setenv("PIXA_METRICS_PASSWORD", "metricspass")
t.Setenv("PIXA_SIGNING_KEY", validTestSigningKey)
t.Setenv("PIXA_ALLOWLIST_HOSTS", "s3.sneak.cloud,.example.com")
t.Setenv("PIXA_ALLOW_HTTP", "true")
t.Setenv("PIXA_UPSTREAM_CONNECTIONS_PER_HOST", "5")
t.Setenv("PIXA_CACHE_MAX_BYTES", "1024")
t.Setenv("PIXA_BLOCKED_NETWORKS", "203.0.113.0/24")
t.Setenv("PIXA_TRUSTED_PROXIES", "192.0.2.0/24")
c, err := newFromSmartConfig(nil)
if err != nil {
t.Fatalf("configuration from the environment alone should load: %v", err)
}
want := Config{
Debug: true,
MaintenanceMode: true,
Port: 9090,
StateDir: "/srv/pixa-env",
SentryDSN: "https://abc123@sentry.example.com/42",
DBURL: "file:/srv/pixa-env/other.sqlite3",
MetricsUsername: "metricsuser",
MetricsPassword: "metricspass",
SigningKey: validTestSigningKey,
AllowlistHosts: []string{testHostS3, ".example.com"},
AllowHTTP: true,
UpstreamConnectionsPerHost: 5,
CacheMaxBytes: 1024,
cacheMaxBytesExplicit: true,
BlockedNetworks: []netip.Prefix{netip.MustParsePrefix("203.0.113.0/24")},
TrustedProxies: []netip.Prefix{netip.MustParsePrefix("192.0.2.0/24")},
}
if !reflect.DeepEqual(*c, want) {
t.Errorf("config from the environment =\n%+v\nwant\n%+v", *c, want)
}
}
// TestPortFromEnvironmentOverridesConfigFile checks that PORT wins over
// the port in the config file.
func TestPortFromEnvironmentOverridesConfigFile(t *testing.T) {
t.Setenv("PORT", "9090")
c, err := configFromYAML(t, signingKeyLine+"port: 8080\n")
if err != nil {
t.Fatalf("PORT=9090 with port 8080 in the file should load: %v", err)
}
if c.Port != 9090 {
t.Errorf("Port = %d, want 9090 from PORT, not 8080 from the file", c.Port)
}
}
// TestInvalidPortFromEnvironmentAbortsStartup checks that a PORT that is
// not a number, or is outside the port range, aborts startup naming PORT
// and the value, even though the file's port is valid.
func TestInvalidPortFromEnvironmentAbortsStartup(t *testing.T) {
t.Setenv("PORT", "banana")
_, err := configFromYAML(t, signingKeyLine+"port: 8080\n")
wantStartupError(t, err, "PORT", "banana")
t.Setenv("PORT", "70000")
_, err = configFromYAML(t, signingKeyLine+"port: 8080\n")
wantStartupError(t, err, "PORT", "70000")
}
// TestListFromEnvironmentReplacesConfigFileList checks that a list
// variable replaces the file's list, split on commas with the spaces
// around each entry trimmed.
func TestListFromEnvironmentReplacesConfigFileList(t *testing.T) {
t.Setenv("PIXA_ALLOWLIST_HOSTS", " cdn.example.com , .example.org ")
c, err := configFromYAML(t, signingKeyLine+
"allowlist_hosts:\n - s3.sneak.cloud\n - sneak.berlin\n")
if err != nil {
t.Fatalf("PIXA_ALLOWLIST_HOSTS should load: %v", err)
}
want := []string{"cdn.example.com", ".example.org"}
if !slices.Equal(c.AllowlistHosts, want) {
t.Errorf("AllowlistHosts = %v, want %v from PIXA_ALLOWLIST_HOSTS",
c.AllowlistHosts, want)
}
}
// TestInvalidBlockedNetworksFromEnvironmentAbortsStartup checks that an
// invalid CIDR, or an empty entry, in PIXA_BLOCKED_NETWORKS aborts
// startup naming the variable, as the same list in the file does.
func TestInvalidBlockedNetworksFromEnvironmentAbortsStartup(t *testing.T) {
t.Setenv("PIXA_BLOCKED_NETWORKS", "203.0.113.0/24,not-a-cidr")
_, err := configFromYAML(t, signingKeyLine)
wantStartupError(t, err, "PIXA_BLOCKED_NETWORKS", "not-a-cidr")
t.Setenv("PIXA_BLOCKED_NETWORKS", "203.0.113.0/24,,198.51.100.0/24")
_, err = configFromYAML(t, signingKeyLine)
wantStartupError(t, err, "PIXA_BLOCKED_NETWORKS")
}
// TestInvalidDebugFromEnvironmentAbortsStartup checks that a PIXA_DEBUG
// that strconv.ParseBool rejects aborts startup instead of defaulting.
func TestInvalidDebugFromEnvironmentAbortsStartup(t *testing.T) {
t.Setenv("PIXA_DEBUG", "maybe")
_, err := configFromYAML(t, signingKeyLine)
wantStartupError(t, err, "PIXA_DEBUG", "maybe")
}
// TestConfigFileAloneBehavesAsBefore checks that with no variables set
// (TestMain unsets them) the config file's values are used and omitted
// keys take their defaults.
func TestConfigFileAloneBehavesAsBefore(t *testing.T) {
t.Parallel()
c, err := configFromYAML(t, signingKeyLine+"port: 9191\n")
if err != nil {
t.Fatalf("config file should load: %v", err)
}
if c.Port != 9191 {
t.Errorf("Port = %d, want 9191 from the file", c.Port)
}
if c.StateDir != DefaultStateDir {
t.Errorf("StateDir = %q, want default %q", c.StateDir, DefaultStateDir)
}
if !slices.Equal(c.TrustedProxies, defaultTrustedProxies()) {
t.Errorf("TrustedProxies = %v, want default %v",
c.TrustedProxies, defaultTrustedProxies())
}
}
// TestEmptyTrustedProxiesFromEnvironmentTrustsNoOne checks that an empty
// PIXA_TRUSTED_PROXIES is an empty list, like [] in the file: it trusts
// no proxy instead of taking the default ranges.
func TestEmptyTrustedProxiesFromEnvironmentTrustsNoOne(t *testing.T) {
t.Setenv("PIXA_TRUSTED_PROXIES", "")
c, err := configFromYAML(t, signingKeyLine)
if err != nil {
t.Fatalf("empty PIXA_TRUSTED_PROXIES should load: %v", err)
}
if len(c.TrustedProxies) != 0 {
t.Errorf("TrustedProxies = %v, want none", c.TrustedProxies)
}
}
// TestEmptyVariableDoesNotFallBackToConfigFile checks that a variable
// that is present but empty is a set value: an empty PIXA_STATE_DIR
// aborts startup like state_dir: "" in the file, instead of falling
// through to the file's state_dir.
func TestEmptyVariableDoesNotFallBackToConfigFile(t *testing.T) {
t.Setenv("PIXA_STATE_DIR", "")
_, err := configFromYAML(t, signingKeyLine+"state_dir: /srv/pixa-file\n")
wantStartupError(t, err, "PIXA_STATE_DIR")
}
// TestMissingSigningKeyNamesItsVariable checks that with no config file
// and no PIXA_SIGNING_KEY, startup aborts naming the variable, which is
// how a container started without it reports the problem.
func TestMissingSigningKeyNamesItsVariable(t *testing.T) {
t.Parallel()
_, err := newFromSmartConfig(nil)
wantStartupError(t, err, "PIXA_SIGNING_KEY")
}
// TestSecretsFromEnvironmentAreNotPrinted checks that errors about the
// signing key and the metrics password name their variables but never
// print their values.
func TestSecretsFromEnvironmentAreNotPrinted(t *testing.T) {
t.Setenv("PIXA_SIGNING_KEY", "short-signing-secret")
_, err := newFromSmartConfig(nil)
wantStartupError(t, err, "PIXA_SIGNING_KEY")
if strings.Contains(err.Error(), "short-signing-secret") {
t.Errorf("error %q prints the signing key", err.Error())
}
t.Setenv("PIXA_SIGNING_KEY", validTestSigningKey)
t.Setenv("PIXA_METRICS_PASSWORD", "metrics-password-secret")
_, err = newFromSmartConfig(nil)
wantStartupError(t, err, "PIXA_METRICS_PASSWORD")
if strings.Contains(err.Error(), "metrics-password-secret") {
t.Errorf("error %q prints the metrics password", err.Error())
}
}
@@ -0,0 +1,120 @@
package handlers
import (
"fmt"
"net/http"
"net/http/httptest"
"testing"
"time"
"github.com/go-chi/chi/v5"
"sneak.berlin/go/pixa/internal/imgcache"
"sneak.berlin/go/pixa/internal/signature"
)
// signedHost is not on the allowlist setupTestHandler builds, so a request
// for it needs a valid signature. No image is served for it: a request that
// passes the signature check gets 502 from the failed fetch, and one that
// fails the check gets 401.
const signedHost = "signed.example.com"
// getImage sends a GET for target to the image route of fix and returns the
// response status.
func getImage(t *testing.T, fix *testFixtures, target string) int {
t.Helper()
r := chi.NewRouter()
r.Get("/v1/image/*", fix.handler.HandleImage())
req := httptest.NewRequestWithContext(t.Context(), http.MethodGet, target, nil)
rec := httptest.NewRecorder()
r.ServeHTTP(rec, req)
t.Logf("GET %s: %d", target, rec.Code)
return rec.Code
}
// TestHandleImage_SignatureCoversQualityAndFit signs a URL for quality 85
// and fit cover, the values the route uses when a URL has no q or fit, and
// sends that signature with each q and fit below.
func TestHandleImage_SignatureCoversQualityAndFit(t *testing.T) {
t.Parallel()
expires := time.Now().Add(time.Hour)
signer := signature.New("test-signing-key-must-be-32-chars")
sig := signer.Sign(&signature.Request{
SourceHost: signedHost,
SourcePath: "/images/photo.jpg",
Width: 50,
Height: 50,
Format: string(imgcache.FormatJPEG),
Quality: 85,
FitMode: string(imgcache.FitCover),
Expires: expires,
})
signedURL := fmt.Sprintf("/v1/image/%s/images/photo.jpg/50x50.jpeg?sig=%s&exp=%d",
signedHost, sig, expires.Unix())
tests := []struct {
name string
query string
wantStatus int
}{
{"no q or fit", "", http.StatusBadGateway},
{"q=85 and fit=cover", "&q=85&fit=cover", http.StatusBadGateway},
{"replayed with q=40", "&q=40", http.StatusUnauthorized},
{"replayed with fit=contain", "&fit=contain", http.StatusUnauthorized},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
status := getImage(t, setupTestHandler(t), signedURL+tt.query)
if status != tt.wantStatus {
t.Errorf("status = %d, want %d", status, tt.wantStatus)
}
})
}
}
// TestHandleImage_GeneratedSignedURLVerifies sends URLs built by the
// service's signed-URL generator to the route.
func TestHandleImage_GeneratedSignedURLVerifies(t *testing.T) {
t.Parallel()
tests := []struct {
name string
quality int
fitMode imgcache.FitMode
}{
{"quality 40 and fit contain", 40, imgcache.FitContain},
{"quality and fit unset", 0, ""},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
fix := setupTestHandler(t)
signedURL, err := fix.service.GenerateSignedURL("", &imgcache.ImageRequest{
SourceHost: signedHost,
SourcePath: "/images/photo.jpg",
Size: imgcache.Size{Width: 50, Height: 50},
Format: imgcache.FormatJPEG,
Quality: tt.quality,
FitMode: tt.fitMode,
}, time.Hour)
if err != nil {
t.Fatalf("GenerateSignedURL() error = %v", err)
}
status := getImage(t, fix, signedURL)
if status != http.StatusBadGateway {
t.Errorf("status = %d, want %d", status, http.StatusBadGateway)
}
})
}
}
+16 -2
View File
@@ -205,12 +205,23 @@ func (s *Service) ValidateRequest(req *ImageRequest) error {
return s.signer.Verify(signatureRequest(req)) return s.signer.Verify(signatureRequest(req))
} }
// GenerateSignedURL generates a signed URL for the given request. // GenerateSignedURL generates a signed URL for the given request. The URL
// carries q and fit next to sig and exp, so the image route verifies it for
// the quality and fit it was signed with. An unset quality or fit is first
// set to 85 or cover, the values the route uses when a URL has no q or fit.
func (s *Service) GenerateSignedURL( func (s *Service) GenerateSignedURL(
baseURL string, baseURL string,
req *ImageRequest, req *ImageRequest,
ttl time.Duration, ttl time.Duration,
) (string, error) { ) (string, error) {
if req.Quality == 0 {
req.Quality = 85
}
if req.FitMode == "" {
req.FitMode = FitCover
}
sigReq := signatureRequest(req) sigReq := signatureRequest(req)
path, sig, exp := s.signer.GenerateSignedURL(sigReq, ttl) path, sig, exp := s.signer.GenerateSignedURL(sigReq, ttl)
@@ -218,7 +229,8 @@ func (s *Service) GenerateSignedURL(
req.Expires = sigReq.Expires req.Expires = sigReq.Expires
req.Signature = sigReq.Signature req.Signature = sigReq.Signature
return fmt.Sprintf("%s%s?sig=%s&exp=%d", baseURL, path, sig, exp), nil return fmt.Sprintf("%s%s?sig=%s&exp=%d&q=%d&fit=%s",
baseURL, path, sig, exp, req.Quality, req.FitMode), nil
} }
// loadCachedSource attempts to load source content from cache, returning nil // loadCachedSource attempts to load source content from cache, returning nil
@@ -452,6 +464,8 @@ func signatureRequest(req *ImageRequest) *signature.Request {
Width: req.Size.Width, Width: req.Size.Width,
Height: req.Size.Height, Height: req.Size.Height,
Format: string(req.Format), Format: string(req.Format),
Quality: req.Quality,
FitMode: string(req.FitMode),
Signature: req.Signature, Signature: req.Signature,
Expires: req.Expires, Expires: req.Expires,
} }
+37 -11
View File
@@ -41,9 +41,12 @@ func goldenVectors() []goldenVector {
Width: 800, Width: 800,
Height: 600, Height: 600,
Format: testFormatWebP, Format: testFormatWebP,
Quality: 85,
FitMode: testFitCover,
}, },
// Signed data: "cdn.example.com:/photos/cat.jpg::800:600:webp:1704067200" // Signed data:
wantSignature: "x5PfPp8QSDo0cJT96od-AEgrQyOVLfqifH5sst61_-w=", // "cdn.example.com:/photos/cat.jpg::800:600:webp:1704067200:85:cover"
wantSignature: "kdqeGoW2SX7qnaYtoB970wEnLydn0UnIgQYQLfAnjXQ=",
wantSignedPath: testSignedPath, wantSignedPath: testSignedPath,
}, },
{ {
@@ -55,10 +58,12 @@ func goldenVectors() []goldenVector {
Width: 800, Width: 800,
Height: 600, Height: 600,
Format: testFormatWebP, Format: testFormatWebP,
Quality: 85,
FitMode: testFitCover,
}, },
// Signed data: // Signed data:
// "cdn.example.com:/photos/cat.jpg:token=abc&v=2:800:600:webp:1704067200" // "cdn.example.com:/photos/cat.jpg:token=abc&v=2:800:600:webp:1704067200:85:cover"
wantSignature: "394_Vf9TdQFkpQ3XKFDQSyxgqKq8N7mApf2S4QaHqyo=", wantSignature: "pKgVBOTd_Q_EikI7MNQLC9Q8Hurdxzyv3EIYvVhqc2I=",
wantSignedPath: "/v1/image/cdn.example.com/photos/cat.jpg" + wantSignedPath: "/v1/image/cdn.example.com/photos/cat.jpg" +
"%3Ftoken=abc&v=2/800x600.webp", "%3Ftoken=abc&v=2/800x600.webp",
}, },
@@ -71,11 +76,34 @@ func goldenVectors() []goldenVector {
Width: 0, Width: 0,
Height: 0, Height: 0,
Format: testFormatPNG, Format: testFormatPNG,
Quality: 85,
FitMode: testFitCover,
}, },
// Signed data: "cdn.example.com:/photos/cat.jpg::0:0:png:1704067200" // Signed data:
wantSignature: "7Be7oteeQwvnSPU4bchyQ4ZGYGsAGBKpeEtuQ02ox60=", // "cdn.example.com:/photos/cat.jpg::0:0:png:1704067200:85:cover"
wantSignature: "6_rZ0yyVbGZRs8kG7n7HLgLi5Jt8vjiWQljIEL1jbIs=",
wantSignedPath: "/v1/image/cdn.example.com/photos/cat.jpg/orig.png", wantSignedPath: "/v1/image/cdn.example.com/photos/cat.jpg/orig.png",
}, },
{
name: "non-default quality and fit",
req: signature.Request{
SourceHost: testHost,
SourcePath: testPath,
SourceQuery: "",
Width: 800,
Height: 600,
Format: testFormatWebP,
Quality: 40,
FitMode: testFitContain,
},
// Signed data:
// "cdn.example.com:/photos/cat.jpg::800:600:webp:1704067200:40:contain"
wantSignature: "pGaXpPUbI3A7nMx-4T9bfq9bYWBNL0kY4bxlcv3g1F8=",
// The path is the same as for the default quality and fit:
// q=40&fit=contain go in the query string next to sig and
// exp (imgcache.Service.GenerateSignedURL adds all four).
wantSignedPath: testSignedPath,
},
} }
} }
@@ -84,11 +112,9 @@ func goldenVectors() []goldenVector {
// hardcoded signing key. // hardcoded signing key.
// //
// If any of these assertions fail, the signed byte format // If any of these assertions fail, the signed byte format
// ("host:path:query:width:height:format:expiration"), the base64url // ("host:path:query:width:height:format:expiration:quality:fit"), the
// encoding, or the signed URL layout has changed. Such a change breaks // base64url encoding, or the signed URL layout has changed. Update these
// every signature already issued to clients, so it must be made // constants only when that change is intended.
// deliberately: update these constants only as part of an intentional,
// documented signature format migration.
func TestSigner_GoldenVectors(t *testing.T) { func TestSigner_GoldenVectors(t *testing.T) {
t.Parallel() t.Parallel()
+68
View File
@@ -0,0 +1,68 @@
package signature_test
import (
"errors"
"testing"
"time"
"sneak.berlin/go/pixa/internal/signature"
)
// signedQualityFitRequest returns a request signed for quality 85 and fit
// mode "cover", the effective defaults the handler applies before
// verification.
func signedQualityFitRequest(signer *signature.Signer) *signature.Request {
req := &signature.Request{
SourceHost: testHost,
SourcePath: testPath,
Width: 800,
Height: 600,
Format: testFormatWebP,
Quality: 85,
FitMode: testFitCover,
Expires: time.Now().Add(1 * time.Hour),
}
req.Signature = signer.Sign(req)
return req
}
// TestSigner_Verify_QualityAndFitAreSigned proves that quality and fit are
// covered by the signature: a URL signed for one quality or fit mode must
// not verify when replayed with a different quality or fit mode. This is the
// amplification vector from the issue — one signed URL replayed across many
// quality and fit values yields many unauthorized cache entries and
// transcodes — so it must be rejected.
func TestSigner_Verify_QualityAndFitAreSigned(t *testing.T) {
t.Parallel()
signer := signature.New("test-secret-key")
cases := []struct {
name string
tamper func(r *signature.Request)
}{
{
name: "replayed with different quality",
tamper: func(r *signature.Request) { r.Quality = 40 },
},
{
name: "replayed with different fit mode",
tamper: func(r *signature.Request) { r.FitMode = testFitContain },
},
}
for _, tt := range cases {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
req := signedQualityFitRequest(signer)
tt.tamper(req)
err := signer.Verify(req)
if !errors.Is(err, signature.ErrInvalid) {
t.Errorf("Verify() = %v, want %v", err, signature.ErrInvalid)
}
})
}
}
+20 -5
View File
@@ -37,6 +37,15 @@ type Request struct {
Height int Height int
// Format is the requested output format (e.g. "webp"). // Format is the requested output format (e.g. "webp").
Format string Format string
// Quality is the requested output quality (1-100) for lossy formats.
// It is the effective value the request resolves to: callers pass the
// default quality when the request omits the parameter, so an omitted
// quality signs identically to that same value stated explicitly.
Quality int
// FitMode is how the image is fit into the requested dimensions
// (e.g. "cover"). Like Quality it is the effective value: callers pass
// the default fit mode when the request omits the parameter.
FitMode string
// Signature is the HMAC signature to verify. // Signature is the HMAC signature to verify.
Signature string Signature string
// Expires is the signature expiration timestamp. // Expires is the signature expiration timestamp.
@@ -56,7 +65,8 @@ func New(secretKey string) *Signer {
} }
// Sign generates an HMAC-SHA256 signature for the given request. // Sign generates an HMAC-SHA256 signature for the given request.
// The signature covers: host + path + query + width + height + format + expiration. // The signature covers: host + path + query + width + height + format +
// expiration + quality + fit.
func (s *Signer) Sign(req *Request) string { func (s *Signer) Sign(req *Request) string {
data := s.buildSignatureData(req) data := s.buildSignatureData(req)
mac := hmac.New(sha256.New, s.secretKey) mac := hmac.New(sha256.New, s.secretKey)
@@ -68,7 +78,8 @@ func (s *Signer) Sign(req *Request) string {
// Verify checks if the signature on the request is valid and not expired. // Verify checks if the signature on the request is valid and not expired.
// Signatures are exact-match only: every component of the signed data // Signatures are exact-match only: every component of the signed data
// (host, path, query, dimensions, format, expiration) must match exactly. // (host, path, query, dimensions, format, expiration, quality, fit) must
// match exactly.
// No suffix matching, wildcard matching, or partial matching is supported. // No suffix matching, wildcard matching, or partial matching is supported.
// A signature for "cdn.example.com" will NOT verify for "example.com" or // A signature for "cdn.example.com" will NOT verify for "example.com" or
// "other.cdn.example.com", and vice versa. // "other.cdn.example.com", and vice versa.
@@ -142,11 +153,13 @@ func (s *Signer) GenerateSignedURL(
} }
// buildSignatureData creates the string to be signed. // buildSignatureData creates the string to be signed.
// Format: "host:path:query:width:height:format:expiration" // Format: "host:path:query:width:height:format:expiration:quality:fit"
// All components are used verbatim (exact match). No normalization, // All components are used verbatim (exact match). No normalization,
// suffix matching, or wildcard expansion is performed. // suffix matching, or wildcard expansion is performed. Quality and fit
// are the effective transform values, so replaying a signed URL with a
// different quality or fit mode fails verification.
func (s *Signer) buildSignatureData(req *Request) string { func (s *Signer) buildSignatureData(req *Request) string {
return fmt.Sprintf("%s:%s:%s:%d:%d:%s:%d", return fmt.Sprintf("%s:%s:%s:%d:%d:%s:%d:%d:%s",
req.SourceHost, req.SourceHost,
req.SourcePath, req.SourcePath,
req.SourceQuery, req.SourceQuery,
@@ -154,6 +167,8 @@ func (s *Signer) buildSignatureData(req *Request) string {
req.Height, req.Height,
req.Format, req.Format,
req.Expires.Unix(), req.Expires.Unix(),
req.Quality,
req.FitMode,
) )
} }
+2
View File
@@ -15,6 +15,8 @@ const (
testPath = "/photos/cat.jpg" testPath = "/photos/cat.jpg"
testFormatWebP = "webp" testFormatWebP = "webp"
testFormatPNG = "png" testFormatPNG = "png"
testFitCover = "cover"
testFitContain = "contain"
testSignedPath = "/v1/image/cdn.example.com/photos/cat.jpg/800x600.webp" testSignedPath = "/v1/image/cdn.example.com/photos/cat.jpg/800x600.webp"
testSig = "abc123" testSig = "abc123"
) )