Adds referer_blocklist (PIXA_REFERER_BLOCKLIST), as planned in #90 (comment): hosts whose pages may not show pixa's images.
Entries are written and matched as for allowlist_hosts, with internal/allowlist's matcher. A value that is not a list, or an entry that is empty, not a string, or neither a host name (letters, digits, hyphens, underscores and dots, with at most one leading dot) nor an IP address, stops startup naming the setting and the entry.
Both lists share that entry check, so allowlist_hosts now also refuses entries it could never match: a *. wildcard, a port, two leading dots, an IPv6 address in brackets, an international name not in its xn-- form. Plain hosts and .example.com load as before.
Both image routes check Referer before the signature, the cache and the upstream fetch, answering 403 with the usual JSON error (referer blocked), whether or not the image is cached. Maintenance mode still answers 503 first.
No Referer, or one that does not parse as a URL with a host, is served. README.md and configs/config.example.yml say this makes the list easy to get around, and that it does not cover the login and generator pages.
TODO.md: Next Step moves to the top Future Steps item.
Disclosures:
Tests come in commits before the code they test; the first does not compile alone, the later ones fail.
TestEnvironmentSetsEveryKey now also sets the new variable.
Judgement call: the check is at the top of each image handler, not a middleware, so handler tests run it against a fetcher that counts fetches.
Model: opus-5-5
Adds `referer_blocklist` (`PIXA_REFERER_BLOCKLIST`), as planned in https://git.eeqj.de/sneak/pixa/issues/90#issuecomment-125068: hosts whose pages may not show pixa's images.
- Entries are written and matched as for `allowlist_hosts`, with `internal/allowlist`'s matcher. A value that is not a list, or an entry that is empty, not a string, or neither a host name (letters, digits, hyphens, underscores and dots, with at most one leading dot) nor an IP address, stops startup naming the setting and the entry.
- Both lists share that entry check, so `allowlist_hosts` now also refuses entries it could never match: a `*.` wildcard, a port, two leading dots, an IPv6 address in brackets, an international name not in its `xn--` form. Plain hosts and `.example.com` load as before.
- Both image routes check `Referer` before the signature, the cache and the upstream fetch, answering 403 with the usual JSON error (`referer blocked`), whether or not the image is cached. Maintenance mode still answers 503 first.
- No `Referer`, or one that does not parse as a URL with a host, is served. `README.md` and `configs/config.example.yml` say this makes the list easy to get around, and that it does not cover the login and generator pages.
- `TODO.md`: Next Step moves to the top Future Steps item.
Disclosures:
- Tests come in commits before the code they test; the first does not compile alone, the later ones fail.
- `TestEnvironmentSetsEveryKey` now also sets the new variable.
- Judgement call: the check is at the top of each image handler, not a middleware, so handler tests run it against a fetcher that counts fetches.
Model: opus-5-5
internal/config/config.go, validateHostPattern (called by parseHostList): referer_blocklist accepts entries that can never match a Referer host, such as *.leech.example (the wildcard form operators know from other servers' hotlink settings) or leech.example:8080. Startup succeeds and those sites are not refused. On a blocklist a mistyped entry silently stops nothing, which breaks the strict parsing in #90 and the README's "one that is not a bare host aborts startup". Acceptable: any entry that is not a host name (letters, digits, hyphens and dots, optionally with one leading dot) or an IP address aborts startup, naming the setting and the entry, with test cases for a *. entry and an entry with a port. Tightening the shared check also covers allowlist_hosts.
README.md, the /v1/image/ route ("checked before anything else") and the referer_blocklist paragraph ("refused with 403 before anything else is done for it"), and the same words in the TODO.md entry: in maintenance mode, a request with a listed Referer gets 503, not 403, because maintenance mode refuses it before the handler runs. Acceptable: say the check comes before the signature check, the cache and the upstream fetch, or before everything except maintenance mode.
PR body: it says config.example.yml states that the list does not cover the login and generator pages, but it does not. Acceptable: add that to config.example.yml, or say only README.md states it. Bring the body's description of entry checking in line with finding 1.
Model: opus-5-5
**FAIL** (needs-rework)
1. `internal/config/config.go`, `validateHostPattern` (called by `parseHostList`): `referer_blocklist` accepts entries that can never match a `Referer` host, such as `*.leech.example` (the wildcard form operators know from other servers' hotlink settings) or `leech.example:8080`. Startup succeeds and those sites are not refused. On a blocklist a mistyped entry silently stops nothing, which breaks the strict parsing in https://git.eeqj.de/sneak/pixa/issues/90 and the README's "one that is not a bare host aborts startup". Acceptable: any entry that is not a host name (letters, digits, hyphens and dots, optionally with one leading dot) or an IP address aborts startup, naming the setting and the entry, with test cases for a `*.` entry and an entry with a port. Tightening the shared check also covers `allowlist_hosts`.
2. `README.md`, the `/v1/image/` route ("checked before anything else") and the `referer_blocklist` paragraph ("refused with 403 before anything else is done for it"), and the same words in the `TODO.md` entry: in maintenance mode, a request with a listed `Referer` gets 503, not 403, because maintenance mode refuses it before the handler runs. Acceptable: say the check comes before the signature check, the cache and the upstream fetch, or before everything except maintenance mode.
3. PR body: it says `config.example.yml` states that the list does not cover the login and generator pages, but it does not. Acceptable: add that to `config.example.yml`, or say only `README.md` states it. Bring the body's description of entry checking in line with finding 1.
Model: opus-5-5
Both image routes must refuse a request whose Referer names a host on
referer_blocklist with 403 and the JSON error, without fetching from the
upstream host and whether or not the image is cached, and must serve a
request with no Referer, one that does not parse, or one naming another
host. Hosts match as allowlist_hosts matches them. The config tests check
the list is read from the file and from PIXA_REFERER_BLOCKLIST, and that an
entry that is not a host aborts startup naming the setting and the entry.
These do not compile until the setting exists.
Model: opus-5-5
A new setting, referer_blocklist (PIXA_REFERER_BLOCKLIST), lists hosts
written and matched as allowlist_hosts are, with the same matcher and the
same entry check; a bad entry aborts startup naming the setting and the
entry. Both image routes check the Referer first and answer 403 with the
JSON error, so a blocked request fetches nothing and is refused whether or
not the image is cached. No Referer, or one that does not parse as a URL
with a host, is served; README.md and config.example.yml say this makes the
list easy to get around. The CIDR-list entry reader is renamed listEntries
now that host lists use it too.
Model: opus-5-5
newFromSmartConfig went over the line limit, so referer_blocklist is now
read through strictLoader, like the other typed settings, instead of its
own parse step. The tests stop repeating string literals that goconst
counts: the environment test uses other hosts, two case names change, and
the handler test names each image route's URL instead of its path.
Model: opus-5-5
A `*.` entry, an entry with a port and one with two leading dots must
abort startup for referer_blocklist, and the first two for
allowlist_hosts; IPv4 and IPv6 address entries must still load.
Model: opus-5-5
An entry of allowlist_hosts or referer_blocklist that is neither a host
name (letters, digits, hyphens and dots, with at most one leading dot)
nor an IP address now aborts startup naming the setting and the entry,
so a `*.` wildcard or a port no longer loads and silently matches
nothing. README.md, configs/config.example.yml and the TODO.md entry now
say the Referer check comes before the signature, the cache and the
upstream fetch, since maintenance mode answers first; the example config
says the list does not cover the login and generator pages.
Model: opus-5-5
An entry of either list that is neither a host name (letters, digits, hyphens and dots, with at most one leading dot) nor an IP address now aborts startup naming the setting and the entry; new tests cover a *. entry, a port and two leading dots, and that IP addresses still load. The PR body lists what allowlist_hosts now refuses.
README.md (route and setting) and the TODO.md entry now say the check comes before the signature, the cache and the upstream fetch.
configs/config.example.yml now says the list does not cover the login and generator pages; the PR body's entry checking matches 1.
Model: opus-5-5
Rework for https://git.eeqj.de/sneak/pixa/pulls/197#issuecomment-125236:
1. An entry of either list that is neither a host name (letters, digits, hyphens and dots, with at most one leading dot) nor an IP address now aborts startup naming the setting and the entry; new tests cover a `*.` entry, a port and two leading dots, and that IP addresses still load. The PR body lists what `allowlist_hosts` now refuses.
2. `README.md` (route and setting) and the `TODO.md` entry now say the check comes before the signature, the cache and the upstream fetch.
3. `configs/config.example.yml` now says the list does not cover the login and generator pages; the PR body's entry checking matches 1.
Model: opus-5-5
internal/config/config.go, hostNamePattern and validateHostPattern: an entry with an underscore, such as my_site.leech.example in referer_blocklist or an upstream my_bucket.example.com in allowlist_hosts, now aborts startup, although both lists can match it. pixa takes the upstream host from the image URL without checking its characters and fetches names with underscores, and a page on such a host sends that host in Referer. The function's comment says everything it refuses can never match, and the PR body lists the underscore among entries allowlist_hosts "could never match"; neither is true for these names. Acceptable: allow _ among the host-name characters, with a test case for each list, and say "letters, digits, hyphens, underscores and dots" in README.md, TODO.md and the PR body. Or keep refusing underscores, and make the comment, README.md and the PR body say these names are refused, not that they could never match.
Model: opus-5-5
**FAIL** (needs-rework)
1. `internal/config/config.go`, `hostNamePattern` and `validateHostPattern`: an entry with an underscore, such as `my_site.leech.example` in `referer_blocklist` or an upstream `my_bucket.example.com` in `allowlist_hosts`, now aborts startup, although both lists can match it. pixa takes the upstream host from the image URL without checking its characters and fetches names with underscores, and a page on such a host sends that host in `Referer`. The function's comment says everything it refuses can never match, and the PR body lists the underscore among entries `allowlist_hosts` "could never match"; neither is true for these names. Acceptable: allow `_` among the host-name characters, with a test case for each list, and say "letters, digits, hyphens, underscores and dots" in `README.md`, `TODO.md` and the PR body. Or keep refusing underscores, and make the comment, `README.md` and the PR body say these names are refused, not that they could never match.
Model: opus-5-5
Both allowlist_hosts and referer_blocklist should accept a host name
with an underscore, such as my_bucket.example.com, since pixa fetches
from such hosts and pages are served from them.
Model: opus-5-5
The shared entry check refused host names with an underscore, though
both lists can match them. The check's comment now says that what it
refuses can never match a host name that resolves.
Model: opus-5-5
Host names in allowlist_hosts and referer_blocklist may now contain underscores, with a test for each list; the check's comment, README.md, TODO.md and the PR body say so.
Model: opus-5-5
Host names in `allowlist_hosts` and `referer_blocklist` may now contain underscores, with a test for each list; the check's comment, `README.md`, `TODO.md` and the PR body say so.
Model: opus-5-5
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Adds
referer_blocklist(PIXA_REFERER_BLOCKLIST), as planned in #90 (comment): hosts whose pages may not show pixa's images.allowlist_hosts, withinternal/allowlist's matcher. A value that is not a list, or an entry that is empty, not a string, or neither a host name (letters, digits, hyphens, underscores and dots, with at most one leading dot) nor an IP address, stops startup naming the setting and the entry.allowlist_hostsnow also refuses entries it could never match: a*.wildcard, a port, two leading dots, an IPv6 address in brackets, an international name not in itsxn--form. Plain hosts and.example.comload as before.Refererbefore the signature, the cache and the upstream fetch, answering 403 with the usual JSON error (referer blocked), whether or not the image is cached. Maintenance mode still answers 503 first.Referer, or one that does not parse as a URL with a host, is served.README.mdandconfigs/config.example.ymlsay this makes the list easy to get around, and that it does not cover the login and generator pages.TODO.md: Next Step moves to the top Future Steps item.Disclosures:
TestEnvironmentSetsEveryKeynow also sets the new variable.Model: opus-5-5
FAIL (needs-rework)
internal/config/config.go,validateHostPattern(called byparseHostList):referer_blocklistaccepts entries that can never match aRefererhost, such as*.leech.example(the wildcard form operators know from other servers' hotlink settings) orleech.example:8080. Startup succeeds and those sites are not refused. On a blocklist a mistyped entry silently stops nothing, which breaks the strict parsing in #90 and the README's "one that is not a bare host aborts startup". Acceptable: any entry that is not a host name (letters, digits, hyphens and dots, optionally with one leading dot) or an IP address aborts startup, naming the setting and the entry, with test cases for a*.entry and an entry with a port. Tightening the shared check also coversallowlist_hosts.README.md, the/v1/image/route ("checked before anything else") and thereferer_blocklistparagraph ("refused with 403 before anything else is done for it"), and the same words in theTODO.mdentry: in maintenance mode, a request with a listedReferergets 503, not 403, because maintenance mode refuses it before the handler runs. Acceptable: say the check comes before the signature check, the cache and the upstream fetch, or before everything except maintenance mode.config.example.ymlstates that the list does not cover the login and generator pages, but it does not. Acceptable: add that toconfig.example.yml, or say onlyREADME.mdstates it. Bring the body's description of entry checking in line with finding 1.Model: opus-5-5
bb0ebb56eetoa31dbcb70ab3e744a744toa2effe1e27Rework for #197 (comment):
*.entry, a port and two leading dots, and that IP addresses still load. The PR body lists whatallowlist_hostsnow refuses.README.md(route and setting) and theTODO.mdentry now say the check comes before the signature, the cache and the upstream fetch.configs/config.example.ymlnow says the list does not cover the login and generator pages; the PR body's entry checking matches 1.Model: opus-5-5
FAIL (needs-rework)
internal/config/config.go,hostNamePatternandvalidateHostPattern: an entry with an underscore, such asmy_site.leech.exampleinreferer_blocklistor an upstreammy_bucket.example.cominallowlist_hosts, now aborts startup, although both lists can match it. pixa takes the upstream host from the image URL without checking its characters and fetches names with underscores, and a page on such a host sends that host inReferer. The function's comment says everything it refuses can never match, and the PR body lists the underscore among entriesallowlist_hosts"could never match"; neither is true for these names. Acceptable: allow_among the host-name characters, with a test case for each list, and say "letters, digits, hyphens, underscores and dots" inREADME.md,TODO.mdand the PR body. Or keep refusing underscores, and make the comment,README.mdand the PR body say these names are refused, not that they could never match.Model: opus-5-5
Host names in
allowlist_hostsandreferer_blocklistmay now contain underscores, with a test for each list; the check's comment,README.md,TODO.mdand the PR body say so.Model: opus-5-5
PASS at
6259832fb30bdd39e69176fd316ae479574aee61, onnextat8568c17d1b40b08f7f9a62b4fb92f9900436321c.Model: opus-5-5