An entry of allowlist_hosts or referer_blocklist that is neither a host name (letters, digits, hyphens and dots, with at most one leading dot) nor an IP address now aborts startup naming the setting and the entry, so a `*.` wildcard or a port no longer loads and silently matches nothing. README.md, configs/config.example.yml and the TODO.md entry now say the Referer check comes before the signature, the cache and the upstream fetch, since maintenance mode answers first; the example config says the list does not cover the login and generator pages. Model: opus-5-5
This commit is contained in:
@@ -177,11 +177,12 @@ path under `/v1/` answers 200, in maintenance mode too.
|
|||||||
the image's `ETag`; 400 for a URL or parameter that is not valid; 401 for a
|
the image's `ETag`; 400 for a URL or parameter that is not valid; 401 for a
|
||||||
missing or wrong signature, a missing `exp` or an `exp` in the past; 403 when
|
missing or wrong signature, a missing `exp` or an `exp` in the past; 403 when
|
||||||
the request's `Referer` names a host in `referer_blocklist`, checked before
|
the request's `Referer` names a host in `referer_blocklist`, checked before
|
||||||
anything else; 403 when the upstream host, or a host it redirects to, is
|
the signature, the cache and the upstream fetch; 403 when the upstream host,
|
||||||
`localhost`, ends in `.localhost` or `.local`, or has an address in a blocked
|
or a host it redirects to, is `localhost`, ends in `.localhost` or `.local`,
|
||||||
network (see `blocked_networks`); 502 when the upstream answered with an error
|
or has an address in a blocked network (see `blocked_networks`); 502 when the
|
||||||
status, and for 5 minutes after that for the same source URL; 503 when pixa is
|
upstream answered with an error status, and for 5 minutes after that for the
|
||||||
busy or in maintenance mode; 500 for any other failure.
|
same source URL; 503 when pixa is busy or in maintenance mode; 500 for any
|
||||||
|
other failure.
|
||||||
- `GET` or `HEAD` `/v1/e/<token>/<name>` — an image through an encrypted URL
|
- `GET` or `HEAD` `/v1/e/<token>/<name>` — an image through an encrypted URL
|
||||||
(see Encrypted URLs). Needs: nothing but the URL. Answers: 200; 304 when
|
(see Encrypted URLs). Needs: nothing but the URL. Answers: 200; 304 when
|
||||||
`If-None-Match` matches the image's `ETag`; 400 for a token that does not
|
`If-None-Match` matches the image's `ETag`; 400 for a token that does not
|
||||||
@@ -394,6 +395,11 @@ and the URL is
|
|||||||
- **Suffix match**: `.example.com` — matches `cdn.example.com`,
|
- **Suffix match**: `.example.com` — matches `cdn.example.com`,
|
||||||
`images.example.com`, and `example.com`
|
`images.example.com`, and `example.com`
|
||||||
|
|
||||||
|
An IP address is matched exactly; write an IPv6 address without brackets. An
|
||||||
|
entry that is neither a host name (letters, digits, hyphens and dots, with at
|
||||||
|
most one leading dot) nor an IP address, such as one with a port or a `*.`
|
||||||
|
wildcard, aborts startup.
|
||||||
|
|
||||||
### Configuration
|
### Configuration
|
||||||
|
|
||||||
Every setting can be given as an environment variable, in a YAML config
|
Every setting can be given as an environment variable, in a YAML config
|
||||||
@@ -461,11 +467,12 @@ Key settings in more detail:
|
|||||||
- `allowlist_hosts` — list of allowed upstream hosts
|
- `allowlist_hosts` — list of allowed upstream hosts
|
||||||
- `referer_blocklist` — list of hosts whose pages may not show pixa's images, to
|
- `referer_blocklist` — list of hosts whose pages may not show pixa's images, to
|
||||||
stop other sites hotlinking them. Entries are written and matched as for
|
stop other sites hotlinking them. Entries are written and matched as for
|
||||||
`allowlist_hosts` (see Allowlist patterns); one that is not a bare host aborts
|
`allowlist_hosts` (see Allowlist patterns), and an entry that is neither a
|
||||||
startup. A request to `/v1/image/` or `/v1/e/` whose `Referer` header names a
|
host name nor an IP address aborts startup. A request to `/v1/image/` or
|
||||||
listed host is refused with 403 before anything else is done for it, so it
|
`/v1/e/` whose `Referer` header names a listed host is refused with 403 before
|
||||||
fetches nothing from the upstream host, and it is refused even when the image
|
its signature or token is checked and before the cache or the upstream host is
|
||||||
is cached. A request with no `Referer`, or one that does not parse as a URL
|
used, so it fetches nothing, and it is refused even when the image is cached.
|
||||||
|
A request with no `Referer`, or one that does not parse as a URL
|
||||||
with a host, is served, as many clients send none. So this is easily got
|
with a host, is served, as many clients send none. So this is easily got
|
||||||
around: a site whose pages send no `Referer` (for example with
|
around: a site whose pages send no `Referer` (for example with
|
||||||
`Referrer-Policy: no-referrer`) is not stopped. It does not apply to the login
|
`Referrer-Policy: no-referrer`) is not stopped. It does not apply to the login
|
||||||
|
|||||||
@@ -31,6 +31,18 @@ P2: security: per-IP rate limiting on the image routes
|
|||||||
|
|
||||||
# Completed Steps
|
# Completed Steps
|
||||||
|
|
||||||
|
- 2026-10-04 referer blocklist (closes #90): `referer_blocklist`
|
||||||
|
(`PIXA_REFERER_BLOCKLIST`) lists hosts, written and matched as for
|
||||||
|
`allowlist_hosts` with the same matcher; an entry of either list that is
|
||||||
|
neither a host name (letters, digits, hyphens and dots, with at most one
|
||||||
|
leading dot) nor an IP address, such as one with a port or a `*.` wildcard,
|
||||||
|
aborts startup naming the setting and the entry. Both image routes refuse a
|
||||||
|
request whose `Referer` names a listed host with 403 and a JSON error before
|
||||||
|
the signature, the cache and the upstream fetch, so it fetches nothing and is
|
||||||
|
refused whether or not the image is cached. A request with no `Referer`, or
|
||||||
|
one that does not parse as a URL with a host, is served, so the list is easily
|
||||||
|
got around; `README.md` and `configs/config.example.yml` say so. It does not
|
||||||
|
apply to the login and generator pages.
|
||||||
- 2026-10-04 fewer files in the repository root (closes #97):
|
- 2026-10-04 fewer files in the repository root (closes #97):
|
||||||
`config.example.yml` moved unchanged to `configs/config.example.yml`, and
|
`config.example.yml` moved unchanged to `configs/config.example.yml`, and
|
||||||
`README.md`, the comments in `internal/config/config.go` and the startup error
|
`README.md`, the comments in `internal/config/config.go` and the startup error
|
||||||
@@ -47,16 +59,6 @@ P2: security: per-IP rate limiting on the image routes
|
|||||||
for it, and the default `db_url` turns on WAL mode with
|
for it, and the default `db_url` turns on WAL mode with
|
||||||
`_pragma=journal_mode(WAL)`. The old default's `_journal_mode=WAL` is not a
|
`_pragma=journal_mode(WAL)`. The old default's `_journal_mode=WAL` is not a
|
||||||
parameter the driver reads, so the database was never in WAL mode.
|
parameter the driver reads, so the database was never in WAL mode.
|
||||||
- 2026-10-04 referer blocklist (closes #90): `referer_blocklist`
|
|
||||||
(`PIXA_REFERER_BLOCKLIST`) lists hosts, written and matched as for
|
|
||||||
`allowlist_hosts` with the same matcher; an entry that is not a bare host
|
|
||||||
aborts startup naming the setting and the entry. Both image routes refuse a
|
|
||||||
request whose `Referer` names a listed host with 403 and a JSON error before
|
|
||||||
anything else is done for it, so it fetches nothing and is refused whether or
|
|
||||||
not the image is cached. A request with no `Referer`, or one that does not
|
|
||||||
parse as a URL with a host, is served, so the list is easily got around;
|
|
||||||
`README.md` and `config.example.yml` say so. It does not apply to the login
|
|
||||||
and generator pages.
|
|
||||||
- 2026-10-04 `TestPeriodicReconciliationAdoptsFileThatAppearsAfterStartup`
|
- 2026-10-04 `TestPeriodicReconciliationAdoptsFileThatAppearsAfterStartup`
|
||||||
only passes through a periodic pass (closes #189): it slept for three
|
only passes through a periodic pass (closes #189): it slept for three
|
||||||
eviction intervals before writing its file, and a startup pass still running
|
eviction intervals before writing its file, and a startup pass still running
|
||||||
|
|||||||
@@ -46,6 +46,8 @@ signing_key: "CHANGE_ME_generate_with_openssl_rand_base64_32"
|
|||||||
|
|
||||||
# Hosts that don't require signatures (default: none)
|
# Hosts that don't require signatures (default: none)
|
||||||
# Use "." prefix for wildcard subdomain matching (e.g., ".example.com" matches "cdn.example.com")
|
# Use "." prefix for wildcard subdomain matching (e.g., ".example.com" matches "cdn.example.com")
|
||||||
|
# An entry that is neither a host name nor an IP address (IPv6 without
|
||||||
|
# brackets), such as one with a port or a "*." wildcard, aborts startup.
|
||||||
allowlist_hosts:
|
allowlist_hosts:
|
||||||
- s3.sneak.cloud
|
- s3.sneak.cloud
|
||||||
- static.sneak.cloud
|
- static.sneak.cloud
|
||||||
@@ -58,7 +60,7 @@ allowlist_hosts:
|
|||||||
# names one of them is answered 403 before anything is fetched, even when
|
# names one of them is answered 403 before anything is fetched, even when
|
||||||
# the image is cached. A request with no Referer, or one that does not
|
# the image is cached. A request with no Referer, or one that does not
|
||||||
# parse, is served, so a site whose pages send no Referer is not stopped.
|
# parse, is served, so a site whose pages send no Referer is not stopped.
|
||||||
# An entry that is not a host aborts startup. (default: none)
|
# The login and generator pages are not covered. (default: none)
|
||||||
# referer_blocklist:
|
# referer_blocklist:
|
||||||
# - leech.example
|
# - leech.example
|
||||||
# - .hotlinker.example
|
# - .hotlinker.example
|
||||||
|
|||||||
+19
-21
@@ -11,6 +11,7 @@ import (
|
|||||||
"net/url"
|
"net/url"
|
||||||
"os"
|
"os"
|
||||||
"path/filepath"
|
"path/filepath"
|
||||||
|
"regexp"
|
||||||
"runtime"
|
"runtime"
|
||||||
"sort"
|
"sort"
|
||||||
"strconv"
|
"strconv"
|
||||||
@@ -98,12 +99,11 @@ var (
|
|||||||
"value is null; omit the key entirely to use the default")
|
"value is null; omit the key entirely to use the default")
|
||||||
errValuesNull = errors.New(
|
errValuesNull = errors.New(
|
||||||
"value is null; omit a key entirely to use its default")
|
"value is null; omit a key entirely to use its default")
|
||||||
errNotBareHostname = errors.New(
|
errNotAHost = errors.New("must be a host name such as " +
|
||||||
"must be a bare hostname without scheme, path, or whitespace")
|
"cdn.example.com or .example.com, or an IP address")
|
||||||
errNoHostnameLabels = errors.New("contains no hostname labels")
|
errNotADuration = errors.New("not a duration such as 30s or 2m")
|
||||||
errNotADuration = errors.New("not a duration such as 30s or 2m")
|
errMustBePositive = errors.New("must be positive")
|
||||||
errMustBePositive = errors.New("must be positive")
|
errNotAnOrigin = errors.New(
|
||||||
errNotAnOrigin = errors.New(
|
|
||||||
`not "*" or an origin such as https://example.com`)
|
`not "*" or an origin such as https://example.com`)
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -742,25 +742,23 @@ func (c *Config) validateConcurrencyLimits() error {
|
|||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// hostNamePattern matches a host name: letters, digits, hyphens and dots,
|
||||||
|
// optionally after one leading dot.
|
||||||
|
var hostNamePattern = regexp.MustCompile(`^\.?[A-Za-z0-9-][A-Za-z0-9.-]*$`)
|
||||||
|
|
||||||
// validateHostPattern checks that an entry of the named key, allowlist_hosts
|
// validateHostPattern checks that an entry of the named key, allowlist_hosts
|
||||||
// or referer_blocklist, is a bare hostname, optionally with a leading dot for
|
// or referer_blocklist, is an IP address or a host name, the host name
|
||||||
// suffix matching. URLs, paths, and whitespace indicate a misconfigured entry.
|
// optionally with one leading dot for suffix matching. Anything else, such as
|
||||||
// An entry with no hostname labels (such as ".") is rejected: the allowlist
|
// a URL, a port or a "*." wildcard, can never match a host, so it is refused.
|
||||||
// matcher treats a leading dot as a suffix pattern, so a bare "." would match
|
// So is "." alone: the allowlist matcher would match it against any host
|
||||||
// any host written in FQDN trailing-dot form, and in allowlist_hosts
|
// written with a trailing dot, which in allowlist_hosts disables URL signing.
|
||||||
// effectively disable URL signing.
|
|
||||||
func validateHostPattern(key, host string) error {
|
func validateHostPattern(key, host string) error {
|
||||||
if strings.Contains(host, "://") || strings.ContainsAny(host, "/ \t") {
|
_, err := netip.ParseAddr(host)
|
||||||
return fmt.Errorf("%s: entry %q %w",
|
if err == nil || hostNamePattern.MatchString(host) {
|
||||||
settingName(key), host, errNotBareHostname)
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
if strings.Trim(host, ".") == "" {
|
return fmt.Errorf("%s: entry %q %w", settingName(key), host, errNotAHost)
|
||||||
return fmt.Errorf("%s: entry %q %w",
|
|
||||||
settingName(key), host, errNoHostnameLabels)
|
|
||||||
}
|
|
||||||
|
|
||||||
return nil
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// loadConfigFile loads configuration from the PIXA_CONFIG_PATH env var
|
// loadConfigFile loads configuration from the PIXA_CONFIG_PATH env var
|
||||||
|
|||||||
Reference in New Issue
Block a user