An entry of allowlist_hosts or referer_blocklist that is neither a host name (letters, digits, hyphens and dots, with at most one leading dot) nor an IP address now aborts startup naming the setting and the entry, so a `*.` wildcard or a port no longer loads and silently matches nothing. README.md, config.example.yml and the TODO.md entry now say the Referer check comes before the signature, the cache and the upstream fetch, since maintenance mode answers first; config.example.yml says the list does not cover the login and generator pages. Model: opus-5-5
This commit is contained in:
@@ -177,11 +177,12 @@ path under `/v1/` answers 200, in maintenance mode too.
|
|||||||
the image's `ETag`; 400 for a URL or parameter that is not valid; 401 for a
|
the image's `ETag`; 400 for a URL or parameter that is not valid; 401 for a
|
||||||
missing or wrong signature, a missing `exp` or an `exp` in the past; 403 when
|
missing or wrong signature, a missing `exp` or an `exp` in the past; 403 when
|
||||||
the request's `Referer` names a host in `referer_blocklist`, checked before
|
the request's `Referer` names a host in `referer_blocklist`, checked before
|
||||||
anything else; 403 when the upstream host, or a host it redirects to, is
|
the signature, the cache and the upstream fetch; 403 when the upstream host,
|
||||||
`localhost`, ends in `.localhost` or `.local`, or has an address in a blocked
|
or a host it redirects to, is `localhost`, ends in `.localhost` or `.local`,
|
||||||
network (see `blocked_networks`); 502 when the upstream answered with an error
|
or has an address in a blocked network (see `blocked_networks`); 502 when the
|
||||||
status, and for 5 minutes after that for the same source URL; 503 when pixa is
|
upstream answered with an error status, and for 5 minutes after that for the
|
||||||
busy or in maintenance mode; 500 for any other failure.
|
same source URL; 503 when pixa is busy or in maintenance mode; 500 for any
|
||||||
|
other failure.
|
||||||
- `GET` or `HEAD` `/v1/e/<token>/<name>` — an image through an encrypted URL
|
- `GET` or `HEAD` `/v1/e/<token>/<name>` — an image through an encrypted URL
|
||||||
(see Encrypted URLs). Needs: nothing but the URL. Answers: 200; 304 when
|
(see Encrypted URLs). Needs: nothing but the URL. Answers: 200; 304 when
|
||||||
`If-None-Match` matches the image's `ETag`; 400 for a token that does not
|
`If-None-Match` matches the image's `ETag`; 400 for a token that does not
|
||||||
@@ -394,6 +395,11 @@ and the URL is
|
|||||||
- **Suffix match**: `.example.com` — matches `cdn.example.com`,
|
- **Suffix match**: `.example.com` — matches `cdn.example.com`,
|
||||||
`images.example.com`, and `example.com`
|
`images.example.com`, and `example.com`
|
||||||
|
|
||||||
|
An IP address is matched exactly; write an IPv6 address without brackets. An
|
||||||
|
entry that is neither a host name (letters, digits, hyphens and dots, with at
|
||||||
|
most one leading dot) nor an IP address, such as one with a port or a `*.`
|
||||||
|
wildcard, aborts startup.
|
||||||
|
|
||||||
### Configuration
|
### Configuration
|
||||||
|
|
||||||
Every setting can be given as an environment variable, in a YAML config
|
Every setting can be given as an environment variable, in a YAML config
|
||||||
@@ -461,11 +467,12 @@ Key settings in more detail:
|
|||||||
- `allowlist_hosts` — list of allowed upstream hosts
|
- `allowlist_hosts` — list of allowed upstream hosts
|
||||||
- `referer_blocklist` — list of hosts whose pages may not show pixa's images, to
|
- `referer_blocklist` — list of hosts whose pages may not show pixa's images, to
|
||||||
stop other sites hotlinking them. Entries are written and matched as for
|
stop other sites hotlinking them. Entries are written and matched as for
|
||||||
`allowlist_hosts` (see Allowlist patterns); one that is not a bare host aborts
|
`allowlist_hosts` (see Allowlist patterns), and an entry that is neither a
|
||||||
startup. A request to `/v1/image/` or `/v1/e/` whose `Referer` header names a
|
host name nor an IP address aborts startup. A request to `/v1/image/` or
|
||||||
listed host is refused with 403 before anything else is done for it, so it
|
`/v1/e/` whose `Referer` header names a listed host is refused with 403 before
|
||||||
fetches nothing from the upstream host, and it is refused even when the image
|
its signature or token is checked and before the cache or the upstream host is
|
||||||
is cached. A request with no `Referer`, or one that does not parse as a URL
|
used, so it fetches nothing, and it is refused even when the image is cached.
|
||||||
|
A request with no `Referer`, or one that does not parse as a URL
|
||||||
with a host, is served, as many clients send none. So this is easily got
|
with a host, is served, as many clients send none. So this is easily got
|
||||||
around: a site whose pages send no `Referer` (for example with
|
around: a site whose pages send no `Referer` (for example with
|
||||||
`Referrer-Policy: no-referrer`) is not stopped. It does not apply to the login
|
`Referrer-Policy: no-referrer`) is not stopped. It does not apply to the login
|
||||||
|
|||||||
@@ -31,22 +31,24 @@ P2: security: per-IP rate limiting on the image routes
|
|||||||
|
|
||||||
# Completed Steps
|
# Completed Steps
|
||||||
|
|
||||||
|
- 2026-10-04 referer blocklist (closes #90): `referer_blocklist`
|
||||||
|
(`PIXA_REFERER_BLOCKLIST`) lists hosts, written and matched as for
|
||||||
|
`allowlist_hosts` with the same matcher; an entry of either list that is
|
||||||
|
neither a host name (letters, digits, hyphens and dots, with at most one
|
||||||
|
leading dot) nor an IP address, such as one with a port or a `*.` wildcard,
|
||||||
|
aborts startup naming the setting and the entry. Both image routes refuse a
|
||||||
|
request whose `Referer` names a listed host with 403 and a JSON error before
|
||||||
|
the signature, the cache and the upstream fetch, so it fetches nothing and is
|
||||||
|
refused whether or not the image is cached. A request with no `Referer`, or
|
||||||
|
one that does not parse as a URL with a host, is served, so the list is easily
|
||||||
|
got around; `README.md` and `config.example.yml` say so. It does not apply to
|
||||||
|
the login and generator pages.
|
||||||
- 2026-10-04 SQLite writes no longer fail with "database is locked" (closes
|
- 2026-10-04 SQLite writes no longer fail with "database is locked" (closes
|
||||||
#198): pixa adds `_pragma=busy_timeout(5000)` to every `db_url`, so a write
|
#198): pixa adds `_pragma=busy_timeout(5000)` to every `db_url`, so a write
|
||||||
that finds another in progress on another connection waits up to five seconds
|
that finds another in progress on another connection waits up to five seconds
|
||||||
for it, and the default `db_url` turns on WAL mode with
|
for it, and the default `db_url` turns on WAL mode with
|
||||||
`_pragma=journal_mode(WAL)`. The old default's `_journal_mode=WAL` is not a
|
`_pragma=journal_mode(WAL)`. The old default's `_journal_mode=WAL` is not a
|
||||||
parameter the driver reads, so the database was never in WAL mode.
|
parameter the driver reads, so the database was never in WAL mode.
|
||||||
- 2026-10-04 referer blocklist (closes #90): `referer_blocklist`
|
|
||||||
(`PIXA_REFERER_BLOCKLIST`) lists hosts, written and matched as for
|
|
||||||
`allowlist_hosts` with the same matcher; an entry that is not a bare host
|
|
||||||
aborts startup naming the setting and the entry. Both image routes refuse a
|
|
||||||
request whose `Referer` names a listed host with 403 and a JSON error before
|
|
||||||
anything else is done for it, so it fetches nothing and is refused whether or
|
|
||||||
not the image is cached. A request with no `Referer`, or one that does not
|
|
||||||
parse as a URL with a host, is served, so the list is easily got around;
|
|
||||||
`README.md` and `config.example.yml` say so. It does not apply to the login
|
|
||||||
and generator pages.
|
|
||||||
- 2026-10-04 `TestPeriodicReconciliationAdoptsFileThatAppearsAfterStartup`
|
- 2026-10-04 `TestPeriodicReconciliationAdoptsFileThatAppearsAfterStartup`
|
||||||
only passes through a periodic pass (closes #189): it slept for three
|
only passes through a periodic pass (closes #189): it slept for three
|
||||||
eviction intervals before writing its file, and a startup pass still running
|
eviction intervals before writing its file, and a startup pass still running
|
||||||
|
|||||||
+3
-1
@@ -46,6 +46,8 @@ signing_key: "CHANGE_ME_generate_with_openssl_rand_base64_32"
|
|||||||
|
|
||||||
# Hosts that don't require signatures (default: none)
|
# Hosts that don't require signatures (default: none)
|
||||||
# Use "." prefix for wildcard subdomain matching (e.g., ".example.com" matches "cdn.example.com")
|
# Use "." prefix for wildcard subdomain matching (e.g., ".example.com" matches "cdn.example.com")
|
||||||
|
# An entry that is neither a host name nor an IP address (IPv6 without
|
||||||
|
# brackets), such as one with a port or a "*." wildcard, aborts startup.
|
||||||
allowlist_hosts:
|
allowlist_hosts:
|
||||||
- s3.sneak.cloud
|
- s3.sneak.cloud
|
||||||
- static.sneak.cloud
|
- static.sneak.cloud
|
||||||
@@ -58,7 +60,7 @@ allowlist_hosts:
|
|||||||
# names one of them is answered 403 before anything is fetched, even when
|
# names one of them is answered 403 before anything is fetched, even when
|
||||||
# the image is cached. A request with no Referer, or one that does not
|
# the image is cached. A request with no Referer, or one that does not
|
||||||
# parse, is served, so a site whose pages send no Referer is not stopped.
|
# parse, is served, so a site whose pages send no Referer is not stopped.
|
||||||
# An entry that is not a host aborts startup. (default: none)
|
# The login and generator pages are not covered. (default: none)
|
||||||
# referer_blocklist:
|
# referer_blocklist:
|
||||||
# - leech.example
|
# - leech.example
|
||||||
# - .hotlinker.example
|
# - .hotlinker.example
|
||||||
|
|||||||
+19
-21
@@ -11,6 +11,7 @@ import (
|
|||||||
"net/url"
|
"net/url"
|
||||||
"os"
|
"os"
|
||||||
"path/filepath"
|
"path/filepath"
|
||||||
|
"regexp"
|
||||||
"runtime"
|
"runtime"
|
||||||
"sort"
|
"sort"
|
||||||
"strconv"
|
"strconv"
|
||||||
@@ -98,12 +99,11 @@ var (
|
|||||||
"value is null; omit the key entirely to use the default")
|
"value is null; omit the key entirely to use the default")
|
||||||
errValuesNull = errors.New(
|
errValuesNull = errors.New(
|
||||||
"value is null; omit a key entirely to use its default")
|
"value is null; omit a key entirely to use its default")
|
||||||
errNotBareHostname = errors.New(
|
errNotAHost = errors.New("must be a host name such as " +
|
||||||
"must be a bare hostname without scheme, path, or whitespace")
|
"cdn.example.com or .example.com, or an IP address")
|
||||||
errNoHostnameLabels = errors.New("contains no hostname labels")
|
errNotADuration = errors.New("not a duration such as 30s or 2m")
|
||||||
errNotADuration = errors.New("not a duration such as 30s or 2m")
|
errMustBePositive = errors.New("must be positive")
|
||||||
errMustBePositive = errors.New("must be positive")
|
errNotAnOrigin = errors.New(
|
||||||
errNotAnOrigin = errors.New(
|
|
||||||
`not "*" or an origin such as https://example.com`)
|
`not "*" or an origin such as https://example.com`)
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -742,25 +742,23 @@ func (c *Config) validateConcurrencyLimits() error {
|
|||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// hostNamePattern matches a host name: letters, digits, hyphens and dots,
|
||||||
|
// optionally after one leading dot.
|
||||||
|
var hostNamePattern = regexp.MustCompile(`^\.?[A-Za-z0-9-][A-Za-z0-9.-]*$`)
|
||||||
|
|
||||||
// validateHostPattern checks that an entry of the named key, allowlist_hosts
|
// validateHostPattern checks that an entry of the named key, allowlist_hosts
|
||||||
// or referer_blocklist, is a bare hostname, optionally with a leading dot for
|
// or referer_blocklist, is an IP address or a host name, the host name
|
||||||
// suffix matching. URLs, paths, and whitespace indicate a misconfigured entry.
|
// optionally with one leading dot for suffix matching. Anything else, such as
|
||||||
// An entry with no hostname labels (such as ".") is rejected: the allowlist
|
// a URL, a port or a "*." wildcard, can never match a host, so it is refused.
|
||||||
// matcher treats a leading dot as a suffix pattern, so a bare "." would match
|
// So is "." alone: the allowlist matcher would match it against any host
|
||||||
// any host written in FQDN trailing-dot form, and in allowlist_hosts
|
// written with a trailing dot, which in allowlist_hosts disables URL signing.
|
||||||
// effectively disable URL signing.
|
|
||||||
func validateHostPattern(key, host string) error {
|
func validateHostPattern(key, host string) error {
|
||||||
if strings.Contains(host, "://") || strings.ContainsAny(host, "/ \t") {
|
_, err := netip.ParseAddr(host)
|
||||||
return fmt.Errorf("%s: entry %q %w",
|
if err == nil || hostNamePattern.MatchString(host) {
|
||||||
settingName(key), host, errNotBareHostname)
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
if strings.Trim(host, ".") == "" {
|
return fmt.Errorf("%s: entry %q %w", settingName(key), host, errNotAHost)
|
||||||
return fmt.Errorf("%s: entry %q %w",
|
|
||||||
settingName(key), host, errNoHostnameLabels)
|
|
||||||
}
|
|
||||||
|
|
||||||
return nil
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// loadConfigFile loads configuration from the PIXA_CONFIG_PATH env var
|
// loadConfigFile loads configuration from the PIXA_CONFIG_PATH env var
|
||||||
|
|||||||
Reference in New Issue
Block a user