Refuse host entries that are not a host name or an IP address
check / check (push) Failing after 3s

An entry of allowlist_hosts or referer_blocklist that is neither a host
name (letters, digits, hyphens and dots, with at most one leading dot)
nor an IP address now aborts startup naming the setting and the entry,
so a `*.` wildcard or a port no longer loads and silently matches
nothing. README.md, configs/config.example.yml and the TODO.md entry now
say the Referer check comes before the signature, the cache and the
upstream fetch, since maintenance mode answers first; the example config
says the list does not cover the login and generator pages.

Model: opus-5-5
This commit is contained in:
2026-10-04 19:09:02 +00:00
parent e1737f497c
commit a2effe1e27
4 changed files with 51 additions and 42 deletions
+17 -10
View File
@@ -177,11 +177,12 @@ path under `/v1/` answers 200, in maintenance mode too.
the image's `ETag`; 400 for a URL or parameter that is not valid; 401 for a the image's `ETag`; 400 for a URL or parameter that is not valid; 401 for a
missing or wrong signature, a missing `exp` or an `exp` in the past; 403 when missing or wrong signature, a missing `exp` or an `exp` in the past; 403 when
the request's `Referer` names a host in `referer_blocklist`, checked before the request's `Referer` names a host in `referer_blocklist`, checked before
anything else; 403 when the upstream host, or a host it redirects to, is the signature, the cache and the upstream fetch; 403 when the upstream host,
`localhost`, ends in `.localhost` or `.local`, or has an address in a blocked or a host it redirects to, is `localhost`, ends in `.localhost` or `.local`,
network (see `blocked_networks`); 502 when the upstream answered with an error or has an address in a blocked network (see `blocked_networks`); 502 when the
status, and for 5 minutes after that for the same source URL; 503 when pixa is upstream answered with an error status, and for 5 minutes after that for the
busy or in maintenance mode; 500 for any other failure. same source URL; 503 when pixa is busy or in maintenance mode; 500 for any
other failure.
- `GET` or `HEAD` `/v1/e/<token>/<name>` — an image through an encrypted URL - `GET` or `HEAD` `/v1/e/<token>/<name>` — an image through an encrypted URL
(see Encrypted URLs). Needs: nothing but the URL. Answers: 200; 304 when (see Encrypted URLs). Needs: nothing but the URL. Answers: 200; 304 when
`If-None-Match` matches the image's `ETag`; 400 for a token that does not `If-None-Match` matches the image's `ETag`; 400 for a token that does not
@@ -394,6 +395,11 @@ and the URL is
- **Suffix match**: `.example.com` — matches `cdn.example.com`, - **Suffix match**: `.example.com` — matches `cdn.example.com`,
`images.example.com`, and `example.com` `images.example.com`, and `example.com`
An IP address is matched exactly; write an IPv6 address without brackets. An
entry that is neither a host name (letters, digits, hyphens and dots, with at
most one leading dot) nor an IP address, such as one with a port or a `*.`
wildcard, aborts startup.
### Configuration ### Configuration
Every setting can be given as an environment variable, in a YAML config Every setting can be given as an environment variable, in a YAML config
@@ -461,11 +467,12 @@ Key settings in more detail:
- `allowlist_hosts` — list of allowed upstream hosts - `allowlist_hosts` — list of allowed upstream hosts
- `referer_blocklist` — list of hosts whose pages may not show pixa's images, to - `referer_blocklist` — list of hosts whose pages may not show pixa's images, to
stop other sites hotlinking them. Entries are written and matched as for stop other sites hotlinking them. Entries are written and matched as for
`allowlist_hosts` (see Allowlist patterns); one that is not a bare host aborts `allowlist_hosts` (see Allowlist patterns), and an entry that is neither a
startup. A request to `/v1/image/` or `/v1/e/` whose `Referer` header names a host name nor an IP address aborts startup. A request to `/v1/image/` or
listed host is refused with 403 before anything else is done for it, so it `/v1/e/` whose `Referer` header names a listed host is refused with 403 before
fetches nothing from the upstream host, and it is refused even when the image its signature or token is checked and before the cache or the upstream host is
is cached. A request with no `Referer`, or one that does not parse as a URL used, so it fetches nothing, and it is refused even when the image is cached.
A request with no `Referer`, or one that does not parse as a URL
with a host, is served, as many clients send none. So this is easily got with a host, is served, as many clients send none. So this is easily got
around: a site whose pages send no `Referer` (for example with around: a site whose pages send no `Referer` (for example with
`Referrer-Policy: no-referrer`) is not stopped. It does not apply to the login `Referrer-Policy: no-referrer`) is not stopped. It does not apply to the login
+12 -10
View File
@@ -31,6 +31,18 @@ P2: security: per-IP rate limiting on the image routes
# Completed Steps # Completed Steps
- 2026-10-04 referer blocklist (closes #90): `referer_blocklist`
(`PIXA_REFERER_BLOCKLIST`) lists hosts, written and matched as for
`allowlist_hosts` with the same matcher; an entry of either list that is
neither a host name (letters, digits, hyphens and dots, with at most one
leading dot) nor an IP address, such as one with a port or a `*.` wildcard,
aborts startup naming the setting and the entry. Both image routes refuse a
request whose `Referer` names a listed host with 403 and a JSON error before
the signature, the cache and the upstream fetch, so it fetches nothing and is
refused whether or not the image is cached. A request with no `Referer`, or
one that does not parse as a URL with a host, is served, so the list is easily
got around; `README.md` and `configs/config.example.yml` say so. It does not
apply to the login and generator pages.
- 2026-10-04 fewer files in the repository root (closes #97): - 2026-10-04 fewer files in the repository root (closes #97):
`config.example.yml` moved unchanged to `configs/config.example.yml`, and `config.example.yml` moved unchanged to `configs/config.example.yml`, and
`README.md`, the comments in `internal/config/config.go` and the startup error `README.md`, the comments in `internal/config/config.go` and the startup error
@@ -47,16 +59,6 @@ P2: security: per-IP rate limiting on the image routes
for it, and the default `db_url` turns on WAL mode with for it, and the default `db_url` turns on WAL mode with
`_pragma=journal_mode(WAL)`. The old default's `_journal_mode=WAL` is not a `_pragma=journal_mode(WAL)`. The old default's `_journal_mode=WAL` is not a
parameter the driver reads, so the database was never in WAL mode. parameter the driver reads, so the database was never in WAL mode.
- 2026-10-04 referer blocklist (closes #90): `referer_blocklist`
(`PIXA_REFERER_BLOCKLIST`) lists hosts, written and matched as for
`allowlist_hosts` with the same matcher; an entry that is not a bare host
aborts startup naming the setting and the entry. Both image routes refuse a
request whose `Referer` names a listed host with 403 and a JSON error before
anything else is done for it, so it fetches nothing and is refused whether or
not the image is cached. A request with no `Referer`, or one that does not
parse as a URL with a host, is served, so the list is easily got around;
`README.md` and `config.example.yml` say so. It does not apply to the login
and generator pages.
- 2026-10-04 `TestPeriodicReconciliationAdoptsFileThatAppearsAfterStartup` - 2026-10-04 `TestPeriodicReconciliationAdoptsFileThatAppearsAfterStartup`
only passes through a periodic pass (closes #189): it slept for three only passes through a periodic pass (closes #189): it slept for three
eviction intervals before writing its file, and a startup pass still running eviction intervals before writing its file, and a startup pass still running
+3 -1
View File
@@ -46,6 +46,8 @@ signing_key: "CHANGE_ME_generate_with_openssl_rand_base64_32"
# Hosts that don't require signatures (default: none) # Hosts that don't require signatures (default: none)
# Use "." prefix for wildcard subdomain matching (e.g., ".example.com" matches "cdn.example.com") # Use "." prefix for wildcard subdomain matching (e.g., ".example.com" matches "cdn.example.com")
# An entry that is neither a host name nor an IP address (IPv6 without
# brackets), such as one with a port or a "*." wildcard, aborts startup.
allowlist_hosts: allowlist_hosts:
- s3.sneak.cloud - s3.sneak.cloud
- static.sneak.cloud - static.sneak.cloud
@@ -58,7 +60,7 @@ allowlist_hosts:
# names one of them is answered 403 before anything is fetched, even when # names one of them is answered 403 before anything is fetched, even when
# the image is cached. A request with no Referer, or one that does not # the image is cached. A request with no Referer, or one that does not
# parse, is served, so a site whose pages send no Referer is not stopped. # parse, is served, so a site whose pages send no Referer is not stopped.
# An entry that is not a host aborts startup. (default: none) # The login and generator pages are not covered. (default: none)
# referer_blocklist: # referer_blocklist:
# - leech.example # - leech.example
# - .hotlinker.example # - .hotlinker.example
+17 -19
View File
@@ -11,6 +11,7 @@ import (
"net/url" "net/url"
"os" "os"
"path/filepath" "path/filepath"
"regexp"
"runtime" "runtime"
"sort" "sort"
"strconv" "strconv"
@@ -98,9 +99,8 @@ var (
"value is null; omit the key entirely to use the default") "value is null; omit the key entirely to use the default")
errValuesNull = errors.New( errValuesNull = errors.New(
"value is null; omit a key entirely to use its default") "value is null; omit a key entirely to use its default")
errNotBareHostname = errors.New( errNotAHost = errors.New("must be a host name such as " +
"must be a bare hostname without scheme, path, or whitespace") "cdn.example.com or .example.com, or an IP address")
errNoHostnameLabels = errors.New("contains no hostname labels")
errNotADuration = errors.New("not a duration such as 30s or 2m") errNotADuration = errors.New("not a duration such as 30s or 2m")
errMustBePositive = errors.New("must be positive") errMustBePositive = errors.New("must be positive")
errNotAnOrigin = errors.New( errNotAnOrigin = errors.New(
@@ -742,27 +742,25 @@ func (c *Config) validateConcurrencyLimits() error {
return nil return nil
} }
// hostNamePattern matches a host name: letters, digits, hyphens and dots,
// optionally after one leading dot.
var hostNamePattern = regexp.MustCompile(`^\.?[A-Za-z0-9-][A-Za-z0-9.-]*$`)
// validateHostPattern checks that an entry of the named key, allowlist_hosts // validateHostPattern checks that an entry of the named key, allowlist_hosts
// or referer_blocklist, is a bare hostname, optionally with a leading dot for // or referer_blocklist, is an IP address or a host name, the host name
// suffix matching. URLs, paths, and whitespace indicate a misconfigured entry. // optionally with one leading dot for suffix matching. Anything else, such as
// An entry with no hostname labels (such as ".") is rejected: the allowlist // a URL, a port or a "*." wildcard, can never match a host, so it is refused.
// matcher treats a leading dot as a suffix pattern, so a bare "." would match // So is "." alone: the allowlist matcher would match it against any host
// any host written in FQDN trailing-dot form, and in allowlist_hosts // written with a trailing dot, which in allowlist_hosts disables URL signing.
// effectively disable URL signing.
func validateHostPattern(key, host string) error { func validateHostPattern(key, host string) error {
if strings.Contains(host, "://") || strings.ContainsAny(host, "/ \t") { _, err := netip.ParseAddr(host)
return fmt.Errorf("%s: entry %q %w", if err == nil || hostNamePattern.MatchString(host) {
settingName(key), host, errNotBareHostname)
}
if strings.Trim(host, ".") == "" {
return fmt.Errorf("%s: entry %q %w",
settingName(key), host, errNoHostnameLabels)
}
return nil return nil
} }
return fmt.Errorf("%s: entry %q %w", settingName(key), host, errNotAHost)
}
// loadConfigFile loads configuration from the PIXA_CONFIG_PATH env var // loadConfigFile loads configuration from the PIXA_CONFIG_PATH env var
// or standard locations. // or standard locations.
func loadConfigFile(log *slog.Logger, appName string) (*smartconfig.Config, error) { func loadConfigFile(log *slog.Logger, appName string) (*smartconfig.Config, error) {