diff --git a/README.md b/README.md index dda724b..1f23d26 100644 --- a/README.md +++ b/README.md @@ -177,11 +177,12 @@ path under `/v1/` answers 200, in maintenance mode too. the image's `ETag`; 400 for a URL or parameter that is not valid; 401 for a missing or wrong signature, a missing `exp` or an `exp` in the past; 403 when the request's `Referer` names a host in `referer_blocklist`, checked before - anything else; 403 when the upstream host, or a host it redirects to, is - `localhost`, ends in `.localhost` or `.local`, or has an address in a blocked - network (see `blocked_networks`); 502 when the upstream answered with an error - status, and for 5 minutes after that for the same source URL; 503 when pixa is - busy or in maintenance mode; 500 for any other failure. + the signature, the cache and the upstream fetch; 403 when the upstream host, + or a host it redirects to, is `localhost`, ends in `.localhost` or `.local`, + or has an address in a blocked network (see `blocked_networks`); 502 when the + upstream answered with an error status, and for 5 minutes after that for the + same source URL; 503 when pixa is busy or in maintenance mode; 500 for any + other failure. - `GET` or `HEAD` `/v1/e//` — an image through an encrypted URL (see Encrypted URLs). Needs: nothing but the URL. Answers: 200; 304 when `If-None-Match` matches the image's `ETag`; 400 for a token that does not @@ -394,6 +395,11 @@ and the URL is - **Suffix match**: `.example.com` — matches `cdn.example.com`, `images.example.com`, and `example.com` +An IP address is matched exactly; write an IPv6 address without brackets. An +entry that is neither a host name (letters, digits, hyphens and dots, with at +most one leading dot) nor an IP address, such as one with a port or a `*.` +wildcard, aborts startup. + ### Configuration Every setting can be given as an environment variable, in a YAML config @@ -461,11 +467,12 @@ Key settings in more detail: - `allowlist_hosts` — list of allowed upstream hosts - `referer_blocklist` — list of hosts whose pages may not show pixa's images, to stop other sites hotlinking them. Entries are written and matched as for - `allowlist_hosts` (see Allowlist patterns); one that is not a bare host aborts - startup. A request to `/v1/image/` or `/v1/e/` whose `Referer` header names a - listed host is refused with 403 before anything else is done for it, so it - fetches nothing from the upstream host, and it is refused even when the image - is cached. A request with no `Referer`, or one that does not parse as a URL + `allowlist_hosts` (see Allowlist patterns), and an entry that is neither a + host name nor an IP address aborts startup. A request to `/v1/image/` or + `/v1/e/` whose `Referer` header names a listed host is refused with 403 before + its signature or token is checked and before the cache or the upstream host is + used, so it fetches nothing, and it is refused even when the image is cached. + A request with no `Referer`, or one that does not parse as a URL with a host, is served, as many clients send none. So this is easily got around: a site whose pages send no `Referer` (for example with `Referrer-Policy: no-referrer`) is not stopped. It does not apply to the login diff --git a/TODO.md b/TODO.md index 8840ea1..5eb1d53 100644 --- a/TODO.md +++ b/TODO.md @@ -31,6 +31,18 @@ P2: security: per-IP rate limiting on the image routes # Completed Steps +- 2026-10-04 referer blocklist (closes #90): `referer_blocklist` + (`PIXA_REFERER_BLOCKLIST`) lists hosts, written and matched as for + `allowlist_hosts` with the same matcher; an entry of either list that is + neither a host name (letters, digits, hyphens and dots, with at most one + leading dot) nor an IP address, such as one with a port or a `*.` wildcard, + aborts startup naming the setting and the entry. Both image routes refuse a + request whose `Referer` names a listed host with 403 and a JSON error before + the signature, the cache and the upstream fetch, so it fetches nothing and is + refused whether or not the image is cached. A request with no `Referer`, or + one that does not parse as a URL with a host, is served, so the list is easily + got around; `README.md` and `configs/config.example.yml` say so. It does not + apply to the login and generator pages. - 2026-10-04 fewer files in the repository root (closes #97): `config.example.yml` moved unchanged to `configs/config.example.yml`, and `README.md`, the comments in `internal/config/config.go` and the startup error @@ -47,16 +59,6 @@ P2: security: per-IP rate limiting on the image routes for it, and the default `db_url` turns on WAL mode with `_pragma=journal_mode(WAL)`. The old default's `_journal_mode=WAL` is not a parameter the driver reads, so the database was never in WAL mode. -- 2026-10-04 referer blocklist (closes #90): `referer_blocklist` - (`PIXA_REFERER_BLOCKLIST`) lists hosts, written and matched as for - `allowlist_hosts` with the same matcher; an entry that is not a bare host - aborts startup naming the setting and the entry. Both image routes refuse a - request whose `Referer` names a listed host with 403 and a JSON error before - anything else is done for it, so it fetches nothing and is refused whether or - not the image is cached. A request with no `Referer`, or one that does not - parse as a URL with a host, is served, so the list is easily got around; - `README.md` and `config.example.yml` say so. It does not apply to the login - and generator pages. - 2026-10-04 `TestPeriodicReconciliationAdoptsFileThatAppearsAfterStartup` only passes through a periodic pass (closes #189): it slept for three eviction intervals before writing its file, and a startup pass still running diff --git a/configs/config.example.yml b/configs/config.example.yml index 3bcef54..cede448 100644 --- a/configs/config.example.yml +++ b/configs/config.example.yml @@ -46,6 +46,8 @@ signing_key: "CHANGE_ME_generate_with_openssl_rand_base64_32" # Hosts that don't require signatures (default: none) # Use "." prefix for wildcard subdomain matching (e.g., ".example.com" matches "cdn.example.com") +# An entry that is neither a host name nor an IP address (IPv6 without +# brackets), such as one with a port or a "*." wildcard, aborts startup. allowlist_hosts: - s3.sneak.cloud - static.sneak.cloud @@ -58,7 +60,7 @@ allowlist_hosts: # names one of them is answered 403 before anything is fetched, even when # the image is cached. A request with no Referer, or one that does not # parse, is served, so a site whose pages send no Referer is not stopped. -# An entry that is not a host aborts startup. (default: none) +# The login and generator pages are not covered. (default: none) # referer_blocklist: # - leech.example # - .hotlinker.example diff --git a/internal/config/config.go b/internal/config/config.go index be6a7c2..0a5991e 100644 --- a/internal/config/config.go +++ b/internal/config/config.go @@ -11,6 +11,7 @@ import ( "net/url" "os" "path/filepath" + "regexp" "runtime" "sort" "strconv" @@ -98,12 +99,11 @@ var ( "value is null; omit the key entirely to use the default") errValuesNull = errors.New( "value is null; omit a key entirely to use its default") - errNotBareHostname = errors.New( - "must be a bare hostname without scheme, path, or whitespace") - errNoHostnameLabels = errors.New("contains no hostname labels") - errNotADuration = errors.New("not a duration such as 30s or 2m") - errMustBePositive = errors.New("must be positive") - errNotAnOrigin = errors.New( + errNotAHost = errors.New("must be a host name such as " + + "cdn.example.com or .example.com, or an IP address") + errNotADuration = errors.New("not a duration such as 30s or 2m") + errMustBePositive = errors.New("must be positive") + errNotAnOrigin = errors.New( `not "*" or an origin such as https://example.com`) ) @@ -742,25 +742,23 @@ func (c *Config) validateConcurrencyLimits() error { return nil } +// hostNamePattern matches a host name: letters, digits, hyphens and dots, +// optionally after one leading dot. +var hostNamePattern = regexp.MustCompile(`^\.?[A-Za-z0-9-][A-Za-z0-9.-]*$`) + // validateHostPattern checks that an entry of the named key, allowlist_hosts -// or referer_blocklist, is a bare hostname, optionally with a leading dot for -// suffix matching. URLs, paths, and whitespace indicate a misconfigured entry. -// An entry with no hostname labels (such as ".") is rejected: the allowlist -// matcher treats a leading dot as a suffix pattern, so a bare "." would match -// any host written in FQDN trailing-dot form, and in allowlist_hosts -// effectively disable URL signing. +// or referer_blocklist, is an IP address or a host name, the host name +// optionally with one leading dot for suffix matching. Anything else, such as +// a URL, a port or a "*." wildcard, can never match a host, so it is refused. +// So is "." alone: the allowlist matcher would match it against any host +// written with a trailing dot, which in allowlist_hosts disables URL signing. func validateHostPattern(key, host string) error { - if strings.Contains(host, "://") || strings.ContainsAny(host, "/ \t") { - return fmt.Errorf("%s: entry %q %w", - settingName(key), host, errNotBareHostname) + _, err := netip.ParseAddr(host) + if err == nil || hostNamePattern.MatchString(host) { + return nil } - if strings.Trim(host, ".") == "" { - return fmt.Errorf("%s: entry %q %w", - settingName(key), host, errNoHostnameLabels) - } - - return nil + return fmt.Errorf("%s: entry %q %w", settingName(key), host, errNotAHost) } // loadConfigFile loads configuration from the PIXA_CONFIG_PATH env var