Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
784f3d2fb7 | ||
|
|
2bea6516a8 | ||
|
|
50c50bfcae | ||
|
|
b84a33fb57 | ||
|
|
4dad4fff43 | ||
|
|
94d60358ee |
No files matched your search
@@ -43,8 +43,7 @@ It protects against:
|
||||
- give the clients of chosen networks and countries a fraction of every
|
||||
limit, so that they are banned sooner;
|
||||
- refuse or limit clients on a blocklist, a DNS blocklist, or with a high
|
||||
AbuseIPDB score;
|
||||
- ban clients named by your own CrowdSec engine.
|
||||
AbuseIPDB score.
|
||||
- Slow and oversized requests. Every request and every answer has a time limit
|
||||
and a size limit.
|
||||
|
||||
@@ -261,7 +260,7 @@ Every setting the code reads, with its default:
|
||||
| `SWW_WAF_LOOKUP_SOURCE` | `geojs` | Where each client's AS number and country come from: `geojs` (a web service), `file` (a local database) or `off`. |
|
||||
| `SWW_WAF_LOOKUP_DB_PATH` | empty | The IPinfo Lite database file, in its `.mmdb` form, read while the source is `file`. |
|
||||
| `SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS` | `800` | How long a request waits for its client's first GeoJS answer when a setting needs it, and how long a request to GeoJS may take, in milliseconds. |
|
||||
| `SWW_WAF_ADD_LOOKUP_HEADERS` | `false` | `true` passes the app the client's AS number and country in `X-Client-ASN` and `X-Client-Country`. |
|
||||
| `SWW_WAF_ADD_LOOKUP_HEADERS` | `true`, or `false` while the lookup source is `off` | `true` passes the app the client's AS number and country in `X-Client-ASN` and `X-Client-Country`. |
|
||||
| `SWW_WAF_EXCLUSIVELY_ALLOWED_COUNTRIES` | empty | When set, the only countries whose clients get through; a client the lookup cannot place is refused too. |
|
||||
| `SWW_WAF_TAG_LISTS` | empty | Address lists that each add a tag to their clients' requests, as `name=URL` pairs, such as `torexit=https://example.com/tor.txt`. |
|
||||
| `SWW_WAF_TAG_DENY` | empty | Tags whose requests are refused, such as `torexit,country=CN`. |
|
||||
@@ -290,8 +289,6 @@ Every setting the code reads, with its default:
|
||||
| `SWW_WAF_ABUSEIPDB_KEY` | unset | The key of your AbuseIPDB account; set, a client that has committed an offence is checked in the background. |
|
||||
| `SWW_WAF_ABUSEIPDB_MIN_SCORE` | `75` | The least AbuseIPDB abuse confidence score, from 0 to 100, that is a hit. |
|
||||
| `SWW_WAF_ABUSEIPDB_DAILY_BUDGET` | `900` | The most AbuseIPDB checks made in a day. |
|
||||
| `SWW_WAF_CROWDSEC_LAPI_URL` | unset | The local API of a CrowdSec engine you run, such as `http://172.17.0.1:8080`; its decision list is fetched every minute, and a listed client is banned. |
|
||||
| `SWW_WAF_CROWDSEC_LAPI_KEY` | unset | The key the engine gave for `smallwebwaf`, as `cscli bouncers add smallwebwaf` prints it; set with the URL, never alone. |
|
||||
| `SWW_WAF_DNSBL_ABUSEIPDB_ACTION` | `limit:25` | What a client listed by a DNS blocklist, or scored a hit by AbuseIPDB, gets: `deny`, `limit:<percent>` or `log`. |
|
||||
| `SWW_WAF_IP_REPUTATION_CACHE_TTL` | `24h` | How long a DNS blocklist verdict or an AbuseIPDB score is used. |
|
||||
| `SWW_WAF_DNSBL_ABUSEIPDB_TIMEOUT_MS` | `2000` | How long a DNS blocklist query or an AbuseIPDB check may take, in milliseconds. |
|
||||
@@ -570,8 +567,7 @@ Each request gets tags from what `waf` knows of its client, written in the
|
||||
- `asn=<number>` and `country=<code>`, from the lookup;
|
||||
- one tag per list of `SWW_WAF_TAG_LISTS` that holds the client, such as
|
||||
`torexit` or `vpn=mullvad`;
|
||||
- `crowdsec`, `dnsbl=<zone>` and `abuseipdb`, when that reputation source lists
|
||||
the client;
|
||||
- `dnsbl=<zone>` and `abuseipdb`, when that reputation source lists the client;
|
||||
- `exploitsignature`, when a rule file or the Core Rule Set matched the request;
|
||||
- `asn=<number>` and `prefix=<netblock>`, from a routewatch instance, while
|
||||
`SWW_WAF_ROUTEWATCH_URL` is set.
|
||||
@@ -609,9 +605,9 @@ lowers a client's limits by tag. See "Request tags" in [`SPEC.md`](SPEC.md).
|
||||
- `nextcloud`, whose fixes let through its sync client, its WebDAV methods,
|
||||
its searches, its file names, its account names, its comments and its
|
||||
documents;
|
||||
- `ghost`, whose fixes let through its filters, its searches, its names, its
|
||||
email addresses, its code injection, its settings and a post saved from
|
||||
its editor, HTML included.
|
||||
- `ghost`, whose fixes let through the text its admin and members type, its
|
||||
analytics, its code injection, ActivityPub and a post saved from its
|
||||
editor, HTML included.
|
||||
- A random password, as a password manager makes one, gets through these forms:
|
||||
- signing in, signing up, and changing or resetting a password, on
|
||||
WordPress, Drupal, MediaWiki, Mastodon, Nextcloud, gitea and Ghost;
|
||||
|
||||
@@ -161,22 +161,21 @@ field gives that name. `refusal_limit` and `byte_limits` give theirs in
|
||||
(`deny_nets`); a client in both lists is let through.
|
||||
5. Bans. A client under a ban is refused (`active_ban`; see "Bans").
|
||||
6. The lookup, unless `SWW_WAF_LOOKUP_SOURCE` is `off` (see "Lookup of AS number
|
||||
and country"). The request goes on at once. It waits for the first GeoJS
|
||||
answer, up to `SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS`, only when one of
|
||||
these needs it:
|
||||
and country"). The request waits for the first GeoJS answer, up to
|
||||
`SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS`, when one of these needs it:
|
||||
- `SWW_WAF_ADD_LOOKUP_HEADERS`, which is `true` by default;
|
||||
- the country list;
|
||||
- a biased threshold;
|
||||
- `SWW_WAF_TAG_DENY` or `SWW_WAF_TAG_LIMIT_PERCENT` naming an `asn=` or
|
||||
`country=` tag;
|
||||
- `SWW_WAF_ADD_LOOKUP_HEADERS`.
|
||||
`country=` tag.
|
||||
|
||||
A request that ends before the answer comes is logged without it.
|
||||
Otherwise the request goes on at once. A request that ends before the answer
|
||||
comes is logged without it.
|
||||
|
||||
7. The country list (`countries`; see "Country list").
|
||||
8. The reputation sources (see "Reputation"), in this order:
|
||||
- A client on a fetched blocklist gets `SWW_WAF_BLOCKLIST_ACTION`
|
||||
(`blocklists`).
|
||||
- A client on the CrowdSec decision list is banned (`crowdsec`).
|
||||
- A client listed by a DNS blocklist gets `SWW_WAF_DNSBL_ABUSEIPDB_ACTION`
|
||||
(`dnsbl`).
|
||||
- A client scored at least `SWW_WAF_ABUSEIPDB_MIN_SCORE` by AbuseIPDB gets
|
||||
@@ -198,10 +197,10 @@ field gives that name. `refusal_limit` and `byte_limits` give theirs in
|
||||
16. A path under `/_sww/` is answered `404`, so that the app never sees that
|
||||
prefix.
|
||||
17. `rproxy` passes the request to `SWW_RPROXY_UPSTREAM_URL`, streaming (see
|
||||
"Passing a request to the app"). With `SWW_WAF_ADD_LOOKUP_HEADERS`,
|
||||
`X-Client-ASN` and `X-Client-Country` are added for the app's own logs; a
|
||||
header whose value is unknown is left out. Whatever that setting says, any
|
||||
such header the client sent is removed.
|
||||
"Passing a request to the app"). While `SWW_WAF_ADD_LOOKUP_HEADERS` is
|
||||
`true`, as by default, `X-Client-ASN` and `X-Client-Country` are added for
|
||||
the app's own logs; a header whose value is unknown is left out. Whatever
|
||||
that setting says, any such header the client sent is removed.
|
||||
18. After the visitor's answer has ended, `rproxy` sends the report (see "The
|
||||
report"). `waf` then counts:
|
||||
- For a request it allowed, the body bytes of the response, of the request
|
||||
@@ -301,14 +300,6 @@ without a ban, because the Core Rule Set has false positives. A `block` rule
|
||||
does the same. Those refusals count toward the refusal limit, so a client that
|
||||
keeps setting them off is banned for a broken limit.
|
||||
|
||||
A client on the CrowdSec decision list is banned until the decision ends, with
|
||||
the cause `crowdsec`. Only a client that sends a request is banned, so a long
|
||||
list does not fill `bans.json`. Such a ban is never made permanent. It does not
|
||||
make the netblock's next ban for a broken limit longer. A ban lifted while the
|
||||
decision is in force is made again at the client's next request. To lift it for
|
||||
good, delete the decision in CrowdSec, then lift the ban once the list has been
|
||||
fetched again.
|
||||
|
||||
These refuse each request they cover, record nothing in `bans.json`, and count
|
||||
no offence:
|
||||
|
||||
@@ -338,10 +329,10 @@ toward `SWW_WAF_MAX_BANS`, so such bans can never fill the table.
|
||||
|
||||
A request refused under a ban that runs out is logged with the action
|
||||
`temporarily_banned`. Under a permanent ban the action is `permanently_banned`.
|
||||
A request refused for a clear sign of attack or a CrowdSec decision takes the
|
||||
action for the ban it made. A request that makes a ban permanent is logged as
|
||||
`permanently_banned`. A request over a rate limit is logged as
|
||||
`client_is_rate_limited`, whichever ban it makes.
|
||||
A request refused for a clear sign of attack takes the action for the ban it
|
||||
made. A request that makes a ban permanent is logged as `permanently_banned`. A
|
||||
request over a rate limit is logged as `client_is_rate_limited`, whichever ban
|
||||
it makes.
|
||||
|
||||
## Settings
|
||||
|
||||
@@ -575,9 +566,9 @@ Each tag and where it comes from:
|
||||
- One tag per list of `SWW_WAF_TAG_LISTS` that holds the client (`tag_lists`),
|
||||
the tag being the list's name, such as `torexit` or `vpn=mullvad`. These lists
|
||||
are fetched and kept like a blocklist. `waf` ships none of its own.
|
||||
- `crowdsec`, `dnsbl=<zone>` and `abuseipdb`, when that reputation source lists
|
||||
the client, whatever the source's action. A zone with a key is named with the
|
||||
key masked, as the `reputation` field gives it.
|
||||
- `dnsbl=<zone>` and `abuseipdb`, when that reputation source lists the client,
|
||||
whatever the source's action. A zone with a key is named with the key masked,
|
||||
as the `reputation` field gives it.
|
||||
- `exploitsignature`, when a rule file or the Core Rule Set matched the request.
|
||||
For the Core Rule Set a match is a score at or over
|
||||
`SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD`.
|
||||
@@ -883,11 +874,13 @@ one entry of `apps` in `internal/coreruleset/coreruleset.go`. gitea's fixes are:
|
||||
which refused `org.apache.commons.logging`.
|
||||
|
||||
A branch, tag or package name can hold a slash, as `feat/docker-support`
|
||||
does. So `q` skips the rule for shell paths (932160) too on these pages,
|
||||
where it refused `/bin/sh`:
|
||||
does. So `q` skips the rule for shell paths (932160) too on these
|
||||
searches, where it refused `/bin/sh`:
|
||||
- a repository's branches, tags and packages pages;
|
||||
- an owner's packages page;
|
||||
- the admin page of packages.
|
||||
- the admin page of packages;
|
||||
- the API's search of packages;
|
||||
- the searches of the Composer and Conan registries.
|
||||
|
||||
- `q` on gitea's other searches, for the same four rules. Words such as
|
||||
`python3`, `powershell`, `ls -la` and `.htaccess` were refused there. The
|
||||
@@ -902,12 +895,46 @@ one entry of `apps` in `internal/coreruleset/coreruleset.go`. gitea's fixes are:
|
||||
- the runner lists of the site, a user, an organisation and a
|
||||
repository;
|
||||
- the admin lists of users, organisations, repositories, unadopted
|
||||
repositories and emails.
|
||||
repositories and emails;
|
||||
- the API's searches of topics, of an organisation's teams and of
|
||||
emails;
|
||||
- the searches of the Cargo, Chef, Composer and Conan registries, and
|
||||
the NuGet registry's search service.
|
||||
|
||||
Most send `q` in the query string. The admin forms that adopt a
|
||||
repository or activate an email send it in the body. gitea serves its
|
||||
admin pages under `/-/admin/` and Forgejo under `/admin/`.
|
||||
|
||||
- Two fields of the API's searches other than `q`, for the same four rules:
|
||||
- `query` of the search of unadopted repositories, which holds an owner
|
||||
and a name, such as `alice/python3-app`;
|
||||
- `name` of the search of a repository's milestones.
|
||||
|
||||
- Fields of the package registries' searches:
|
||||
- `text` of the npm registry's search, for the same four rules and the
|
||||
rule for shell paths (932160). An npm name can hold a slash, as
|
||||
`@alice/tools-cli` does.
|
||||
- `searchTerm` of the NuGet v2 API's searches, for the same four rules.
|
||||
These are the searches:
|
||||
- `Search()` and `Packages()`;
|
||||
- `Search()/$count` and `Packages()/$count`, which count their
|
||||
results.
|
||||
- `$filter` of those searches, for the lists of system files and Windows
|
||||
commands (930120, 932120). It holds an expression such as
|
||||
`tolower(Id) eq 'python3'`.
|
||||
- The names `$filter`, `$orderby` and `$top` on those searches, for the
|
||||
rule for MongoDB operators (942290). It refused every such search.
|
||||
- `q` of a Conan search, for the rule for SQL injection (942100), which
|
||||
refused the pattern `zlib/*`. `q` skips it only when the request has
|
||||
one `q`, in the query string or the body. That `q` has the shape of a
|
||||
Conan reference. A reference is a name, then optionally `/` and a
|
||||
version, then optionally `@` and a user, then optionally `/` and a
|
||||
channel. Each part is made of letters, digits, `_`, `.`, `-` and `*`.
|
||||
A part holds no `+`, since 942100 reads `+` as a space. `q` also holds
|
||||
no `*/`, which ends an SQL comment, and no `--`, which starts one. And
|
||||
942100 finds no SQL injection in `q` once each comment that `/*`
|
||||
starts, an unclosed one included, is replaced by a space.
|
||||
|
||||
- The path of these requests, which names a file:
|
||||
- a file's page;
|
||||
- its raw file;
|
||||
@@ -1756,35 +1783,87 @@ Ghost's fixes: on Ghost's paths, some fields skip the rules that refused
|
||||
ordinary Ghost requests there. `internal/coreruleset/ghost.conf` lists each rule
|
||||
and the request it refused. Ghost's admin and integrations call the Admin API
|
||||
under `/ghost/api/admin/`. Themes and Ghost's scripts call the Content API under
|
||||
`/ghost/api/content/`, and members call `/members/api/`.
|
||||
`/ghost/api/content/`, and members call `/members/api/`. The admin calls
|
||||
ActivityPub, a service of its own, under `/.ghost/activitypub/`.
|
||||
|
||||
Three query fields skip some rules:
|
||||
Eight rules refused words that name commands and files, such as `python3`,
|
||||
`ls -la`, `.htaccess` and the title `powershell & python3`. They are 930120,
|
||||
932120, 932125, 932230, 932235, 932250, 932260 and 932380.
|
||||
|
||||
- `filter`, a query in Ghost's own filter language, skips 932120 on the Admin
|
||||
API and the Content API, so `tag:powershell` passes;
|
||||
These query fields skip some rules:
|
||||
|
||||
- `filter`, a query in Ghost's own filter language, skips those eight rules
|
||||
except 930120, on the Admin API and the Content API;
|
||||
- `search`, a search for a member, skips 930120, 932120, 932250 and 932260 on
|
||||
the Admin API;
|
||||
- `url`, the address the editor looks up for an embed or a bookmark, skips
|
||||
930120 and 932120.
|
||||
930120 and 932120;
|
||||
- `source` and the `utm_` fields, which filter the analytics, skip those eight
|
||||
rules under `/ghost/api/admin/stats/`;
|
||||
- `pathname`, the page the analytics show, skips 930120, 932120 and 932260
|
||||
there;
|
||||
- `email_segment`, the members a post is sent to as a newsletter, skips 932120
|
||||
on the Admin API's posts;
|
||||
- `query`, a search for an account in ActivityPub, skips those eight rules;
|
||||
- `email`, when an invitation is checked, skips 930120, 932120 and 932260.
|
||||
|
||||
The fields of JSON bodies where a person types plain text skip 930120, 932120,
|
||||
932125, 932230, 932250, 932260 and 932380. They are these fields of posts,
|
||||
pages, tags, labels, members, newsletters and snippets, on the Admin API:
|
||||
The fields of JSON bodies where a person types plain text skip those eight
|
||||
rules. They are these fields of posts, pages, tags, labels, members, newsletters
|
||||
and snippets, on the Admin API:
|
||||
|
||||
- `title`, `name` and `slug`;
|
||||
- a post's `tags` and a member's `labels`, by name or slug;
|
||||
- `custom_excerpt`, `description`, `note` and `sender_name`;
|
||||
- the `meta_`, `og_` and `twitter_` titles and descriptions.
|
||||
|
||||
A member's `name` at `/members/api/` skips them too. A member's `email` skips
|
||||
930120, 932120 and 932260, at `/members/api/` and when the admin adds or edits a
|
||||
member. A staff user's address skips them too: `username` when signing in, and
|
||||
`email` when creating the first staff user, asking for a password reset or
|
||||
saving a profile.
|
||||
These fields on the Admin API skip them too:
|
||||
|
||||
The `canonical_url` of a post or a page skips 930120 and 932120, as `url` does.
|
||||
A post's skips them under `/ghost/api/admin/posts/`, and a page's under
|
||||
`/ghost/api/admin/pages/`.
|
||||
- a staff user's `name`, `slug`, `bio`, `location` and `website`;
|
||||
- `blogTitle` and `name` when creating the first staff user;
|
||||
- `name` when accepting an invitation;
|
||||
- `name`, `description` and `benefits` of tiers;
|
||||
- `name`, `code`, `display_title` and `display_description` of offers;
|
||||
- `title`, `description` and `excerpt` of recommendations;
|
||||
- `name` and `description` of integrations, and `name` of webhooks;
|
||||
- `feature_image_alt` and `feature_image_caption` of posts and pages.
|
||||
|
||||
These fields skip them on their own paths:
|
||||
|
||||
- a member's `name` at `/members/api/`;
|
||||
- `html` of a member's comment at `/members/api/comments/`;
|
||||
- `labels`, when the admin imports members from a CSV file;
|
||||
- `content` of a note or a reply in ActivityPub, and `image.altText`, the
|
||||
description of its image;
|
||||
- `name`, `username` and `bio` of the account in ActivityPub.
|
||||
|
||||
A member's `email` skips 930120, 932120 and 932260, at `/members/api/` and when
|
||||
the admin adds or edits a member. A staff user's address skips them too. It is
|
||||
sent as `username` when signing in, and as `email` in these requests:
|
||||
|
||||
- creating the first staff user;
|
||||
- asking for a password reset;
|
||||
- saving a profile;
|
||||
- inviting a staff user;
|
||||
- accepting an invitation.
|
||||
|
||||
In ActivityPub, an account's handle and its address skip 930120 and 932120. The
|
||||
domain of the site's handle skips 930120, 932120 and 932260. Each field skips
|
||||
its rules only on its own path:
|
||||
|
||||
- `sourceHandle`, an account that moves here, when it is added at
|
||||
`/.ghost/activitypub/v1/aliases`;
|
||||
- `actorUri`, that account's address, when the admin removes it there;
|
||||
- `targetHandle`, the account this one moves to, at
|
||||
`/.ghost/activitypub/v1/migration`;
|
||||
- `domain`, saved at `/.ghost/activitypub/v1/domain` and checked at
|
||||
`/.ghost/activitypub/v1/domain/validate`.
|
||||
|
||||
The `canonical_url` of a post, a page or a tag skips 930120 and 932120, as `url`
|
||||
does. Each skips them only on its own path:
|
||||
|
||||
- a post's under `/ghost/api/admin/posts/`;
|
||||
- a page's under `/ghost/api/admin/pages/`;
|
||||
- a tag's under `/ghost/api/admin/tags/`.
|
||||
|
||||
The articles carry HTML on purpose. These fields on the Admin API hold them:
|
||||
|
||||
@@ -1804,12 +1883,14 @@ The articles skip these rules:
|
||||
Code injection is the HTML that Ghost adds to the head and the foot of a page.
|
||||
These fields on the Admin API hold it:
|
||||
|
||||
- `codeinjection_head` and `codeinjection_foot` of posts and pages;
|
||||
- `codeinjection_head` and `codeinjection_foot` of posts, pages and tags;
|
||||
- `value` of settings, since the site's settings hold its code injection.
|
||||
|
||||
Code injection skips these rules:
|
||||
|
||||
- 930120, 932120, 932125, 932230, 932250, 932260 and 932380, as plain text does;
|
||||
- 930120, 932120, 932125, 932230, 932235, 932250, 932260 and 932380, which
|
||||
refused words that name commands and files, such as the site's title
|
||||
`powershell & python3`;
|
||||
- 921130, 941100, 941160 and 941270, which refused `<meta>` and `<link>`
|
||||
elements.
|
||||
|
||||
@@ -1818,15 +1899,22 @@ card can embed a widget and code injection can load analytics. These are the
|
||||
fields, on the Admin API:
|
||||
|
||||
- `lexical`, `mobiledoc` and `html` of posts and pages;
|
||||
- `codeinjection_head` and `codeinjection_foot` of posts and pages;
|
||||
- `codeinjection_head` and `codeinjection_foot` of posts, pages and tags;
|
||||
- `value` of settings.
|
||||
|
||||
`<script>alert(1)</script>` is still refused in snippets and newsletters.
|
||||
`javascript:alert(1)` is still refused in the articles and code injection
|
||||
(941170 and 941210). Path traversal and SQL injection are still refused in every
|
||||
field above. Script injection is too, outside the articles and code injection.
|
||||
`; wget http://example.com/x.sh` is still refused in the query fields, the plain
|
||||
text, the addresses and the canonical URLs.
|
||||
|
||||
`; wget http://example.com/x.sh` is still refused in these fields:
|
||||
|
||||
- the addresses, the handles and the domain;
|
||||
- the `canonical_url` of posts, pages and tags;
|
||||
- `search`, `url`, `pathname`, `email_segment` and `email`.
|
||||
|
||||
The other fields above skip 932235, which refuses that command but also refuses
|
||||
`powershell & python3`.
|
||||
|
||||
Coraza names a staff user's `profile_image` field `json.users.0.profile_image`.
|
||||
That name holds `.profile`, a file on the list of system files, so the name
|
||||
@@ -1863,12 +1951,11 @@ to the surrounding netblock. The four ban lengths are under "Bans".
|
||||
### Reputation
|
||||
|
||||
No source is on by default. A source acts on a list kept elsewhere; the defaults
|
||||
judge a client only by what it does here. AbuseIPDB and CrowdSec also need an
|
||||
account or an engine of the operator's own. The list best suited to be on by
|
||||
default, Spamhaus DROP, has terms, below, that forbid frequent fetching, and
|
||||
Spamhaus may block an address that breaks them. Each `waf` fetches its own copy,
|
||||
so several of them on one host, sharing one address, would often fetch too close
|
||||
together.
|
||||
judge a client only by what it does here. AbuseIPDB also needs an account of the
|
||||
operator's own. The list best suited to be on by default, Spamhaus DROP, has
|
||||
terms, below, that forbid frequent fetching, and Spamhaus may block an address
|
||||
that breaks them. Each `waf` fetches its own copy, so several of them on one
|
||||
host, sharing one address, would often fetch too close together.
|
||||
|
||||
- Blocklists (`SWW_WAF_BLOCKLIST_URLS`) are text files of addresses and
|
||||
netblocks, one to a line, as the Spamhaus DROP list is written. Anything after
|
||||
@@ -1901,18 +1988,6 @@ together.
|
||||
percentage of every request and byte limit, and suits a list of addresses
|
||||
shared with ordinary visitors, such as Tor exits. `log` only notes the hit in
|
||||
the log line and raises an alert.
|
||||
- A CrowdSec engine on the host (`SWW_WAF_CROWDSEC_LAPI_URL`) has its decision
|
||||
list fetched every minute, with `SWW_WAF_CROWDSEC_LAPI_KEY` in the `X-Api-Key`
|
||||
header. The list is kept like a fetched blocklist, and a client on it is
|
||||
banned (see "Bans"). Only decisions of the type `ban` on an address or a
|
||||
netblock are used; any other, such as a captcha or a country, is left out. A
|
||||
decision lasts for the `duration` it had left when the engine answered. This
|
||||
is how a fleet-wide blocklist can arrive without `waf` depending on CrowdSec.
|
||||
A fetch fails when:
|
||||
- the engine answers other than `200`, a redirect included;
|
||||
- the fetch does not finish within a minute;
|
||||
- the answer is longer than 16 MiB;
|
||||
- the answer is not a list of decisions.
|
||||
- Each DNS blocklist (`SWW_WAF_DNSBL_ZONES`) is asked about a client's own
|
||||
address in the background, the first time the client is seen. No request waits
|
||||
for the answer. The name asked about is the one RFC 5782 gives: the numbers of
|
||||
@@ -2298,9 +2373,9 @@ runs no check, writes no request log line and counts nothing in the metrics.
|
||||
answer but the last below carries `X-Sww-Disposition`.
|
||||
|
||||
- `X-Sww-Disposition: allow`, status `200`, no body. `rproxy` passes the request
|
||||
to the app. With `SWW_WAF_ADD_LOOKUP_HEADERS` the answer also carries
|
||||
`X-Client-ASN` and `X-Client-Country`, which `rproxy` adds to the request it
|
||||
passes on. A header whose value is unknown is left out.
|
||||
to the app. While `SWW_WAF_ADD_LOOKUP_HEADERS` is `true`, the answer also
|
||||
carries `X-Client-ASN` and `X-Client-Country`, which `rproxy` adds to the
|
||||
request it passes on. A header whose value is unknown is left out.
|
||||
- `X-Sww-Disposition: deny`, with the status, headers and body the visitor is to
|
||||
get. `rproxy` gives them to the visitor, less this header. The status is one
|
||||
of:
|
||||
@@ -2340,14 +2415,13 @@ client without an answer by then counts as unknown. No question waits on these,
|
||||
which run in the background:
|
||||
|
||||
- the DNS blocklists and AbuseIPDB;
|
||||
- fetching the blocklists, the tag lists, the CrowdSec decision list and the
|
||||
file `SWW_WAF_ASN_LIMIT_PERCENT_URL` names.
|
||||
- fetching the blocklists, the tag lists and the file
|
||||
`SWW_WAF_ASN_LIMIT_PERCENT_URL` names.
|
||||
|
||||
In `enforce` mode, a question does not wait for GeoJS when it is a request:
|
||||
|
||||
- from a client on a fetched blocklist, while `SWW_WAF_BLOCKLIST_ACTION` is
|
||||
`deny`;
|
||||
- from a client on the CrowdSec decision list;
|
||||
- that would take its client over a rate limit at 100% of it, since a percentage
|
||||
can only lower a limit.
|
||||
|
||||
|
||||
+3
-5
@@ -13,9 +13,8 @@ back or dropped.
|
||||
threshold. The request was refused unless the alert's `detail` gives `mode`:
|
||||
`detect` or `observe`.
|
||||
- `anomaly`: a count over an anomaly threshold.
|
||||
- `reputation_hit`: a request from a client on a blocklist, the CrowdSec
|
||||
decision list or a DNS blocklist, or with an AbuseIPDB score that is a hit.
|
||||
One alert per source.
|
||||
- `reputation_hit`: a request from a client on a blocklist or a DNS blocklist,
|
||||
or with an AbuseIPDB score that is a hit. One alert per source.
|
||||
- `source_failure`: one of:
|
||||
- GeoJS failing;
|
||||
- a list that cannot be fetched;
|
||||
@@ -69,8 +68,7 @@ on one line:
|
||||
"earlier_bans": {
|
||||
"limit": 0,
|
||||
"attack": 0,
|
||||
"admin": 0,
|
||||
"crowdsec": 0
|
||||
"admin": 0
|
||||
}
|
||||
}
|
||||
},
|
||||
|
||||
+1
-4
@@ -97,7 +97,7 @@ Every one of these lines has:
|
||||
- `client_denied`: refused for `SWW_WAF_DENY_NETS`, for a reputation source
|
||||
with the action `deny`, or for a tag in `SWW_WAF_TAG_DENY`;
|
||||
- `temporarily_banned`: refused under a ban that runs out, or for a `ban`
|
||||
rule, a trap path or a CrowdSec decision that made such a ban;
|
||||
rule or a trap path that made such a ban;
|
||||
- `permanently_banned`: refused under a ban that never runs out, including a
|
||||
ban this request made or made permanent;
|
||||
- `country_denied`, `client_is_rate_limited`, `blocked_by_block_rule` and
|
||||
@@ -118,8 +118,6 @@ Every one of these lines has:
|
||||
- `countries`: `country`, and `setting`, the country list that refused it;
|
||||
- `tag_deny`: `tag`, the tag of `SWW_WAF_TAG_DENY` the request carries;
|
||||
- `blocklists`: `lists`, the blocklist URLs that list the client;
|
||||
- `crowdsec`: `scenario`, the decision's scenario, and `expires`, when the
|
||||
decision ends;
|
||||
- `dnsbl`: `zones`, the DNS blocklist zones that list the client, their keys
|
||||
masked;
|
||||
- `abuseipdb`: `score`, and `min_score`, `SWW_WAF_ABUSEIPDB_MIN_SCORE`;
|
||||
@@ -171,7 +169,6 @@ Every one of these lines has:
|
||||
and the rules that matched, for a request it inspected.
|
||||
- `reputation`: the sources the client is on, in this order:
|
||||
- blocklist URLs;
|
||||
- the CrowdSec decision list's URL;
|
||||
- DNS blocklist zones;
|
||||
- `abuseipdb`.
|
||||
- `tags`: the request's tags, sorted, left out when it has none. Each tag and
|
||||
|
||||
+7
-16
@@ -150,7 +150,7 @@ cut off; the client is banned.
|
||||
- `SWW_WAF_LOOKUP_SOURCE` (default `geojs`): `geojs`, `file` or `off`. `off`
|
||||
stops the start while a setting that needs a lookup is set:
|
||||
- `SWW_WAF_EXCLUSIVELY_ALLOWED_COUNTRIES`;
|
||||
- `SWW_WAF_ADD_LOOKUP_HEADERS`;
|
||||
- `SWW_WAF_ADD_LOOKUP_HEADERS` set to `true`;
|
||||
- `SWW_WAF_TAG_DENY` or `SWW_WAF_TAG_LIMIT_PERCENT` naming a `country=` tag,
|
||||
or an `asn=` tag while `SWW_WAF_ROUTEWATCH_URL` is unset;
|
||||
- the two byte percentage lists and `SWW_WAF_ASN_LIMIT_PERCENT_URL`;
|
||||
@@ -161,9 +161,11 @@ cut off; the client is banned.
|
||||
- `SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS` (default `800`): how long a
|
||||
request waits for its client's first answer from GeoJS, and how long a request
|
||||
to GeoJS may take. A time in milliseconds; never `off`.
|
||||
- `SWW_WAF_ADD_LOOKUP_HEADERS` (default `false`): `true` has `rproxy` pass the
|
||||
- `SWW_WAF_ADD_LOOKUP_HEADERS` (default `true`): `true` has `rproxy` pass the
|
||||
app the client's AS number in `X-Client-ASN` and its country in
|
||||
`X-Client-Country`.
|
||||
`X-Client-Country`. The default is `false` while `SWW_WAF_LOOKUP_SOURCE` is
|
||||
`off`. With GeoJS, a new client's first request waits for its answer, up to
|
||||
`SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS`.
|
||||
|
||||
## Country list
|
||||
|
||||
@@ -180,8 +182,8 @@ in [`SPEC.md`](../SPEC.md).
|
||||
`name=URL` pairs, such as `torexit=https://example.com/tor.txt`. A name is the
|
||||
tag, and may hold `=`, such as `vpn=mullvad`. Each list is an `http` or
|
||||
`https` URL fetched like a blocklist. A name or URL listed twice is refused.
|
||||
So is a URL that is also in `SWW_WAF_BLOCKLIST_URLS`,
|
||||
`SWW_WAF_ASN_LIMIT_PERCENT_URL`, or the CrowdSec decision list.
|
||||
So is a URL that is also in `SWW_WAF_BLOCKLIST_URLS` or is
|
||||
`SWW_WAF_ASN_LIMIT_PERCENT_URL`.
|
||||
- `SWW_WAF_TAG_DENY` (default empty): tags whose requests are refused, such as
|
||||
`torexit,country=CN`. A tag listed twice, an `asn=` tag without a number, or a
|
||||
`country=` tag without a country, is refused.
|
||||
@@ -214,9 +216,6 @@ tag that no request can get stops the start. A request can get only these:
|
||||
`SWW_WAF_CORE_RULE_SET_MODE` nor `SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD` is
|
||||
`off`.
|
||||
|
||||
`crowdsec` stops the start in both settings, because a CrowdSec decision refuses
|
||||
the request first.
|
||||
|
||||
## Biased thresholds
|
||||
|
||||
- `SWW_WAF_ASN_BYTES_LIMIT_PERCENT` and `SWW_WAF_COUNTRY_BYTES_LIMIT_PERCENT`
|
||||
@@ -333,14 +332,6 @@ the request first.
|
||||
score, from 0 to 100, that is a hit.
|
||||
- `SWW_WAF_ABUSEIPDB_DAILY_BUDGET` (default `900`): the most checks made in a
|
||||
day, a whole number above zero. AbuseIPDB's free accounts may make 1,000.
|
||||
- `SWW_WAF_CROWDSEC_LAPI_URL` (default unset): the local API of a CrowdSec
|
||||
engine you run, such as `http://172.17.0.1:8080`. An `http` or `https` URL
|
||||
without a user or a fragment; its decision list is at `v1/decisions` under it.
|
||||
Set without the key, it is refused. So is a decision list URL that is also in
|
||||
`SWW_WAF_BLOCKLIST_URLS`, or that is `SWW_WAF_ASN_LIMIT_PERCENT_URL`.
|
||||
- `SWW_WAF_CROWDSEC_LAPI_KEY` (default unset): the key the engine gave for
|
||||
`smallwebwaf`, as `cscli bouncers add smallwebwaf` prints it. Set without the
|
||||
URL, it is refused.
|
||||
- `SWW_WAF_DNSBL_ABUSEIPDB_ACTION` (default `limit:25`): `deny`,
|
||||
`limit:<percent>` or `log`, for DNS blocklist and AbuseIPDB hits.
|
||||
- `SWW_WAF_IP_REPUTATION_CACHE_TTL` (default `24h`): how long a zone's verdict
|
||||
|
||||
+1
-1
@@ -25,7 +25,7 @@ Active and past bans: every ban an admin made, and up to `SWW_WAF_MAX_BANS` of
|
||||
those `waf` made. Per entry:
|
||||
|
||||
- `netblock`, `start` and `expires` (`null` for a permanent ban);
|
||||
- `cause`: `attack`, `limit`, `admin` or `crowdsec`;
|
||||
- `cause`: `attack`, `limit` or `admin`;
|
||||
- `reason`: a short reason;
|
||||
- `lifted`: when an admin lifted it, if one did;
|
||||
- `notes`: the notes below.
|
||||
|
||||
@@ -48,9 +48,8 @@ const (
|
||||
// SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD, refused in block mode, let through in
|
||||
// detect mode.
|
||||
EventCoreRuleSetMatch = "core_rule_set_match"
|
||||
// EventReputationHit is a request whose client a blocklist, the
|
||||
// CrowdSec decision list or a DNSBL zone lists, or whose AbuseIPDB score
|
||||
// is a hit.
|
||||
// EventReputationHit is a request whose client a blocklist or a DNSBL
|
||||
// zone lists, or whose AbuseIPDB score is a hit.
|
||||
EventReputationHit = "reputation_hit"
|
||||
// EventSourceFailure is GeoJS failing or refusing sww, a fetch
|
||||
// of a list failing, a query to a DNSBL zone or a check with AbuseIPDB
|
||||
|
||||
+35
-108
@@ -1,10 +1,8 @@
|
||||
// Package bans is the ban ledger: the bans sww makes on the
|
||||
// netblocks of clients that break a rate limit, a byte limit or the refusal
|
||||
// limit, show a clear sign of attack or are listed by the CrowdSec
|
||||
// decision list, and
|
||||
// those an admin makes, with their notes, as the "Bans" section of SPEC.md
|
||||
// describes. The bans are kept in memory, and written to bans.json and
|
||||
// read from it by the state package.
|
||||
// limit or show a clear sign of attack, and those an admin makes, with their
|
||||
// notes, as the "Bans" section of SPEC.md describes. The bans are kept in
|
||||
// memory, and written to bans.json and read from it by the state package.
|
||||
package bans
|
||||
|
||||
import (
|
||||
@@ -28,9 +26,6 @@ const (
|
||||
// CauseAdmin is a ban an admin made, or one sww made that an
|
||||
// admin keeps. It is never dropped.
|
||||
CauseAdmin = "admin"
|
||||
// CauseCrowdSec is a ban sww made for a client the CrowdSec
|
||||
// decision list lists. It ends when CrowdSec's decision does.
|
||||
CauseCrowdSec = "crowdsec"
|
||||
)
|
||||
|
||||
// repeatFactor is how many times as long as the netblock's last ban a ban
|
||||
@@ -45,8 +40,8 @@ type Params struct {
|
||||
// LimitBanDuration is how long a first ban for a broken limit lasts.
|
||||
LimitBanDuration time.Duration
|
||||
// LimitBanRepeatWindow is how soon after the end of the netblock's
|
||||
// ban that ended last, other than one for a clear sign of attack or for
|
||||
// CrowdSec's decision, a broken limit counts as a repeat, which bans for
|
||||
// ban that ended last, other than one for a clear sign of attack, a
|
||||
// broken limit counts as a repeat, which bans for
|
||||
// repeatFactor times as long as that ban.
|
||||
LimitBanRepeatWindow time.Duration
|
||||
// A ban for a broken limit that would be longer than
|
||||
@@ -68,11 +63,10 @@ type Ban struct {
|
||||
Start time.Time
|
||||
// Expires is when the ban ends, zero for a permanent ban.
|
||||
Expires time.Time
|
||||
// Cause is CauseLimit, CauseAttack, CauseAdmin or CauseCrowdSec.
|
||||
// Cause is CauseLimit, CauseAttack or CauseAdmin.
|
||||
Cause string
|
||||
// Reason is a short text: for a ban sww made, the limit broken,
|
||||
// the rule that matched or the scenario of CrowdSec's decision; for an
|
||||
// admin's, what the admin wrote.
|
||||
// Reason is a short text: for a ban sww made, the limit broken or
|
||||
// the rule that matched; for an admin's, what the admin wrote.
|
||||
Reason string
|
||||
// Lifted is when an admin lifted the ban, zero while no admin has. A
|
||||
// lifted ban refuses nothing, and does not make the netblock's next
|
||||
@@ -133,8 +127,7 @@ type Notes struct {
|
||||
// log's reputation names them. It is left out when none did.
|
||||
Reputation []ReputationHit `json:"reputation,omitempty"`
|
||||
// Request is the request that broke the limit, or whose bytes broke
|
||||
// it, that was the clear sign of attack, or that came from a client the
|
||||
// CrowdSec decision list lists.
|
||||
// it, or that was the clear sign of attack.
|
||||
Request Request `json:"request"`
|
||||
// Requests is how many requests the netblock has sent since it was
|
||||
// first seen, and Refused how many of them the ban has refused so
|
||||
@@ -148,9 +141,8 @@ type Notes struct {
|
||||
|
||||
// ReputationHit is a reputation source that listed a client, as a
|
||||
// reputation_hit alert's detail gives it: Source is the URL of the
|
||||
// blocklist or of the CrowdSec decision list, the DNSBL zone with its key
|
||||
// masked, or "abuseipdb", and Score, for AbuseIPDB alone, its score of the
|
||||
// client.
|
||||
// blocklist, the DNSBL zone with its key masked, or "abuseipdb", and Score,
|
||||
// for AbuseIPDB alone, its score of the client.
|
||||
type ReputationHit struct {
|
||||
Source string `json:"source"`
|
||||
Score *int64 `json:"score,omitempty"`
|
||||
@@ -158,10 +150,9 @@ type ReputationHit struct {
|
||||
|
||||
// EarlierBans counts a netblock's bans before a ban, by cause.
|
||||
type EarlierBans struct {
|
||||
Limit int `json:"limit"`
|
||||
Attack int `json:"attack"`
|
||||
Admin int `json:"admin"`
|
||||
CrowdSec int `json:"crowdsec"`
|
||||
Limit int `json:"limit"`
|
||||
Attack int `json:"attack"`
|
||||
Admin int `json:"admin"`
|
||||
}
|
||||
|
||||
// Request is a request in a ban's notes. Each text is cut to 256 bytes.
|
||||
@@ -303,8 +294,7 @@ func activeBan(bans []Ban, now time.Time) *Ban {
|
||||
// BanForLimit bans netblock at now for a broken limit, with notes, and
|
||||
// returns the ban, and true. A first ban lasts LimitBanDuration. A ban
|
||||
// made within LimitBanRepeatWindow after the netblock's ban that ended
|
||||
// last, other than one for a clear sign of attack or for CrowdSec's
|
||||
// decision, or a lifted one, lasts
|
||||
// last, other than one for a clear sign of attack or a lifted one, lasts
|
||||
// repeatFactor times as long as that one. A ban that would be longer
|
||||
// than LimitBanPermanentAbove is permanent instead. If a ban on netblock is still
|
||||
// active, as when two of its requests break a limit at once, that ban is
|
||||
@@ -318,7 +308,6 @@ func (l *Ledger) BanForLimit(
|
||||
return l.ban(
|
||||
netblock,
|
||||
now,
|
||||
time.Time{},
|
||||
CauseLimit,
|
||||
limitReason(notes),
|
||||
notes,
|
||||
@@ -334,7 +323,6 @@ func (l *Ledger) WouldBanForLimit(
|
||||
return l.ban(
|
||||
netblock,
|
||||
now,
|
||||
time.Time{},
|
||||
CauseLimit,
|
||||
limitReason(notes),
|
||||
notes,
|
||||
@@ -353,7 +341,6 @@ func (l *Ledger) BanForAttack(
|
||||
return l.ban(
|
||||
netblock,
|
||||
now,
|
||||
time.Time{},
|
||||
CauseAttack,
|
||||
attackReason(notes),
|
||||
notes,
|
||||
@@ -369,7 +356,6 @@ func (l *Ledger) WouldBanForAttack(
|
||||
return l.ban(
|
||||
netblock,
|
||||
now,
|
||||
time.Time{},
|
||||
CauseAttack,
|
||||
attackReason(notes),
|
||||
notes,
|
||||
@@ -377,51 +363,9 @@ func (l *Ledger) WouldBanForAttack(
|
||||
)
|
||||
}
|
||||
|
||||
// BanForCrowdSec bans netblock at now until expires, when CrowdSec's
|
||||
// decision on the client ends, with notes, and returns the ban, and
|
||||
// whether it made it, as BanForLimit does. Its reason is "CrowdSec's
|
||||
// decision for <scenario>", the scenario that made the decision.
|
||||
func (l *Ledger) BanForCrowdSec(
|
||||
netblock netip.Prefix,
|
||||
now, expires time.Time,
|
||||
scenario string,
|
||||
notes Notes,
|
||||
) (Ban, bool) {
|
||||
return l.ban(
|
||||
netblock,
|
||||
now,
|
||||
expires,
|
||||
CauseCrowdSec,
|
||||
crowdSecReason(scenario),
|
||||
notes,
|
||||
true,
|
||||
)
|
||||
}
|
||||
|
||||
// WouldBanForCrowdSec returns what BanForCrowdSec would, without making
|
||||
// the ban: what observe mode would have done.
|
||||
func (l *Ledger) WouldBanForCrowdSec(
|
||||
netblock netip.Prefix,
|
||||
now, expires time.Time,
|
||||
scenario string,
|
||||
notes Notes,
|
||||
) (Ban, bool) {
|
||||
return l.ban(
|
||||
netblock,
|
||||
now,
|
||||
expires,
|
||||
CauseCrowdSec,
|
||||
crowdSecReason(scenario),
|
||||
notes,
|
||||
false,
|
||||
)
|
||||
}
|
||||
|
||||
// WouldBePermanent reports whether a ban on netblock for cause, CauseLimit,
|
||||
// CauseAttack or CauseCrowdSec, made at now would be permanent, as
|
||||
// BanForLimit, BanForAttack or BanForCrowdSec would make it. It works out
|
||||
// nothing else of the ban. A ban for CrowdSec's decision is never
|
||||
// permanent: it ends with the decision.
|
||||
// WouldBePermanent reports whether a ban on netblock for cause, CauseLimit
|
||||
// or CauseAttack, made at now would be permanent, as BanForLimit or
|
||||
// BanForAttack would make it. It works out nothing else of the ban.
|
||||
func (l *Ledger) WouldBePermanent(
|
||||
netblock netip.Prefix, now time.Time, cause string,
|
||||
) bool {
|
||||
@@ -433,14 +377,11 @@ func (l *Ledger) WouldBePermanent(
|
||||
held = *bans
|
||||
}
|
||||
|
||||
switch cause {
|
||||
case CauseAttack:
|
||||
if cause == CauseAttack {
|
||||
return l.attackExpiry(held, now).IsZero()
|
||||
case CauseLimit:
|
||||
return l.limitExpiry(held, now).IsZero()
|
||||
default: // CauseCrowdSec
|
||||
return false
|
||||
}
|
||||
|
||||
return l.limitExpiry(held, now).IsZero()
|
||||
}
|
||||
|
||||
// limitReason is the reason of a ban for a broken limit, with notes.
|
||||
@@ -459,12 +400,6 @@ func attackReason(notes Notes) string {
|
||||
return "matched the rule " + notes.RuleID
|
||||
}
|
||||
|
||||
// crowdSecReason is the reason of a ban for CrowdSec's decision, which
|
||||
// scenario made.
|
||||
func crowdSecReason(scenario string) string {
|
||||
return "CrowdSec's decision for " + scenario
|
||||
}
|
||||
|
||||
// BanForAdmin bans netblock at now for an admin, with reason, until
|
||||
// expires, or for good when expires is zero, and returns the ban, whose
|
||||
// cause is CauseAdmin. Unlike BanForLimit and BanForAttack, it makes the
|
||||
@@ -704,15 +639,13 @@ func (l *Ledger) holds(netblock netip.Prefix, start time.Time) bool {
|
||||
})
|
||||
}
|
||||
|
||||
// ban bans netblock at now for cause, with reason and notes, as
|
||||
// BanForLimit, BanForAttack and BanForCrowdSec describe, and returns the
|
||||
// ban, and whether it made it. expires is when a ban for CauseCrowdSec
|
||||
// ends, and zero for the others, whose end the ledger works out. Unless
|
||||
// keep is true, the ban is not made, only returned: it is the ban that
|
||||
// would have been made.
|
||||
// ban bans netblock at now for cause, CauseLimit or CauseAttack, with
|
||||
// reason and notes, as BanForLimit and BanForAttack describe, and returns
|
||||
// the ban, and whether it made it. Unless keep is true, the ban is not
|
||||
// made, only returned: it is the ban that would have been made.
|
||||
func (l *Ledger) ban(
|
||||
netblock netip.Prefix,
|
||||
now, expires time.Time,
|
||||
now time.Time,
|
||||
cause, reason string,
|
||||
notes Notes,
|
||||
keep bool,
|
||||
@@ -743,13 +676,10 @@ func (l *Ledger) ban(
|
||||
Notes: notes,
|
||||
}
|
||||
|
||||
switch cause {
|
||||
case CauseAttack:
|
||||
if cause == CauseAttack {
|
||||
ban.Expires = l.attackExpiry(held, now)
|
||||
case CauseLimit:
|
||||
} else {
|
||||
ban.Expires = l.limitExpiry(held, now)
|
||||
default: // CauseCrowdSec
|
||||
ban.Expires = expires
|
||||
}
|
||||
|
||||
if !keep {
|
||||
@@ -784,16 +714,13 @@ func earlierBans(held []Ban) EarlierBans {
|
||||
fromHere.Attack++
|
||||
case CauseAdmin:
|
||||
fromHere.Admin++
|
||||
case CauseCrowdSec:
|
||||
fromHere.CrowdSec++
|
||||
}
|
||||
|
||||
notes := ban.Notes.EarlierBans
|
||||
count := EarlierBans{
|
||||
Limit: notes.Limit + fromHere.Limit,
|
||||
Attack: notes.Attack + fromHere.Attack,
|
||||
Admin: notes.Admin + fromHere.Admin,
|
||||
CrowdSec: notes.CrowdSec + fromHere.CrowdSec,
|
||||
Limit: notes.Limit + fromHere.Limit,
|
||||
Attack: notes.Attack + fromHere.Attack,
|
||||
Admin: notes.Admin + fromHere.Admin,
|
||||
}
|
||||
|
||||
// Of counts that name as many bans, the earlier ban's is taken: it
|
||||
@@ -808,7 +735,7 @@ func earlierBans(held []Ban) EarlierBans {
|
||||
|
||||
// total returns how many bans e counts, of all causes.
|
||||
func (e EarlierBans) total() int {
|
||||
return e.Limit + e.Attack + e.Admin + e.CrowdSec
|
||||
return e.Limit + e.Attack + e.Admin
|
||||
}
|
||||
|
||||
// markChanged has Changed receive a value, unless one is waiting already.
|
||||
@@ -913,9 +840,9 @@ func (l *Ledger) add(ban Ban) {
|
||||
// limitExpiry returns when a ban for a broken limit made at now ends, or
|
||||
// zero when it is permanent. held are the netblock's bans, none of them
|
||||
// active, of which the one that ended last, other than a ban for a clear
|
||||
// sign of attack or for CrowdSec's decision, or a lifted one, can make the
|
||||
// new ban longer. A ban an admin adds to bans.json can start after
|
||||
// another and end before it, so that one is looked for among them all.
|
||||
// sign of attack or a lifted one, can make the new ban longer. A ban an
|
||||
// admin adds to bans.json can start after another and end before it, so
|
||||
// that one is looked for among them all.
|
||||
func (l *Ledger) limitExpiry(held []Ban, now time.Time) time.Time {
|
||||
length := l.params.LimitBanDuration
|
||||
|
||||
|
||||
@@ -1,136 +0,0 @@
|
||||
package bans_test
|
||||
|
||||
import (
|
||||
"net/netip"
|
||||
"reflect"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"sneak.berlin/go/smallwebwaf/internal/bans"
|
||||
)
|
||||
|
||||
// scenario is the scenario of the tests' CrowdSec decisions.
|
||||
const scenario = "crowdsecurity/ssh-bf"
|
||||
|
||||
func TestCrowdSecBanLastsUntilTheDecisionEnds(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
ledger := bans.New(defaultParams())
|
||||
netblock := netip.MustParsePrefix("203.0.113.9/32")
|
||||
expires := midnight().Add(4 * time.Hour)
|
||||
|
||||
// The ban that would be made is not made.
|
||||
would, wouldBan := ledger.WouldBanForCrowdSec(
|
||||
netblock,
|
||||
midnight(),
|
||||
expires,
|
||||
scenario,
|
||||
bans.Notes{},
|
||||
)
|
||||
if !wouldBan || len(ledger.Bans(netblock)) != 0 {
|
||||
t.Errorf(
|
||||
"would ban %t, and the ledger holds %+v, want true and nothing",
|
||||
wouldBan,
|
||||
ledger.Bans(netblock),
|
||||
)
|
||||
}
|
||||
|
||||
const reason = "CrowdSec's decision for " + scenario
|
||||
|
||||
ban, made := ledger.BanForCrowdSec(
|
||||
netblock,
|
||||
midnight(),
|
||||
expires,
|
||||
scenario,
|
||||
bans.Notes{},
|
||||
)
|
||||
if !made || !reflect.DeepEqual(ban, would) ||
|
||||
ban.Cause != bans.CauseCrowdSec ||
|
||||
!ban.Expires.Equal(expires) ||
|
||||
ban.Reason != reason ||
|
||||
ledger.Made(bans.CauseCrowdSec) != 1 {
|
||||
t.Errorf(
|
||||
"made %t the ban %+v, want the one that would be made, %+v, for "+
|
||||
"crowdsec until %s",
|
||||
made,
|
||||
ban,
|
||||
would,
|
||||
expires,
|
||||
)
|
||||
}
|
||||
|
||||
// A second decision on the netblock while the ban lasts makes no other.
|
||||
again, made := ledger.BanForCrowdSec(netblock, midnight().Add(time.Hour),
|
||||
expires.Add(time.Hour), scenario, bans.Notes{})
|
||||
if made || !again.Expires.Equal(expires) ||
|
||||
ledger.Made(bans.CauseCrowdSec) != 1 {
|
||||
t.Errorf(
|
||||
"made %t the ban %+v while the first lasts, want none",
|
||||
made,
|
||||
again,
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCrowdSecBanIsNeverMadePermanent(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
ledger := bans.New(defaultParams())
|
||||
netblock := netip.MustParsePrefix("203.0.113.9/32")
|
||||
expires := midnight().Add(4 * time.Hour)
|
||||
ledger.BanForCrowdSec(
|
||||
netblock,
|
||||
midnight(),
|
||||
expires,
|
||||
scenario,
|
||||
bans.Notes{},
|
||||
)
|
||||
|
||||
// A request as the ban ends is refused, and leaves it as it is.
|
||||
last := expires.Add(-time.Nanosecond)
|
||||
|
||||
held, banned, madePermanent := ledger.Check(netblock.Addr(), last)
|
||||
if !banned || madePermanent || !held.Expires.Equal(expires) ||
|
||||
ledger.WouldBePermanent(netblock, last, bans.CauseCrowdSec) {
|
||||
t.Errorf(
|
||||
"as the ban ends, banned %t with %+v, made permanent %t, want "+
|
||||
"refused under the ban as it was",
|
||||
banned,
|
||||
held,
|
||||
madePermanent,
|
||||
)
|
||||
}
|
||||
|
||||
if _, banned, _ := ledger.Check(netblock.Addr(), expires); banned {
|
||||
t.Error("the ban refuses a request once the decision has ended")
|
||||
}
|
||||
}
|
||||
|
||||
func TestCrowdSecBanIsCountedAndDoesNotLengthenTheNextBanForALimit(
|
||||
t *testing.T,
|
||||
) {
|
||||
t.Parallel()
|
||||
|
||||
ledger := bans.New(defaultParams())
|
||||
netblock := netip.MustParsePrefix("203.0.113.9/32")
|
||||
|
||||
// Three times the three days would be permanent; a limit broken as the
|
||||
// ban for CrowdSec's decision ends bans for an hour, as a first broken
|
||||
// limit does.
|
||||
crowdSec, _ := ledger.BanForCrowdSec(
|
||||
netblock,
|
||||
midnight(),
|
||||
midnight().Add(3*day),
|
||||
scenario,
|
||||
bans.Notes{},
|
||||
)
|
||||
limit, _ := ledger.BanForLimit(netblock, crowdSec.Expires, bans.Notes{})
|
||||
|
||||
if limit.Expires.Sub(limit.Start) != time.Hour ||
|
||||
limit.Notes.EarlierBans != (bans.EarlierBans{CrowdSec: 1}) {
|
||||
t.Errorf(
|
||||
"the ban for a limit is %+v, want one of an hour after one for crowdsec",
|
||||
limit,
|
||||
)
|
||||
}
|
||||
}
|
||||
+8
-81
@@ -84,7 +84,8 @@ type WAF struct {
|
||||
// client's first answer from GeoJS while a setting needs it
|
||||
// (SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS), which cannot be off.
|
||||
// AddLookupHeaders is true when the app is passed the client's AS number
|
||||
// and country in headers (SWW_WAF_ADD_LOOKUP_HEADERS).
|
||||
// and country in headers (SWW_WAF_ADD_LOOKUP_HEADERS), by default unless
|
||||
// LookupSource is off.
|
||||
LookupSource string
|
||||
LookupDBPath string
|
||||
ASNAndCountryLookupTimeout time.Duration
|
||||
@@ -157,13 +158,6 @@ type WAF struct {
|
||||
DNSBLAbuseIPDBLimitPercent int64
|
||||
IPReputationCacheTTL time.Duration
|
||||
DNSBLAbuseIPDBTimeout time.Duration
|
||||
// CrowdSecDecisionsURL is where the decision list of the CrowdSec
|
||||
// engine whose local API SWW_WAF_CROWDSEC_LAPI_URL names is fetched from:
|
||||
// that URL with v1/decisions added to its path, "" while it is unset and
|
||||
// none is. CrowdSecKey is the key the engine is asked with
|
||||
// (SWW_WAF_CROWDSEC_LAPI_KEY).
|
||||
CrowdSecDecisionsURL string
|
||||
CrowdSecKey string
|
||||
// BanResponse is the status a refused client is answered with, 403
|
||||
// or 429, or 0 to close the connection without an answer
|
||||
// (SWW_WAF_BAN_RESPONSE). It answers a banned client, a request that
|
||||
@@ -473,21 +467,12 @@ var (
|
||||
"is not an IP address with an optional port, " +
|
||||
"such as 192.0.2.53 or [2001:db8::53]:5353",
|
||||
)
|
||||
errNotLAPIURL = errors.New(
|
||||
"is not an http or https URL without a user or a fragment, " +
|
||||
"such as http://172.17.0.1:8080")
|
||||
errNotRouteWatchURL = errors.New(
|
||||
"is not an http or https URL without a user or a fragment, " +
|
||||
"such as https://routewatch.example")
|
||||
errNeedsLAPIKey = errors.New("the engine answers no request without it")
|
||||
errLAPIKeyUnused = errors.New(
|
||||
"it is sent only to the engine at that URL",
|
||||
)
|
||||
errAnotherList = errors.New(
|
||||
errFetchedOtherwise = errors.New(
|
||||
"is in SWW_WAF_BLOCKLIST_URLS or is SWW_WAF_ASN_LIMIT_PERCENT_URL too",
|
||||
)
|
||||
errFetchedOtherwise = errors.New("is in SWW_WAF_BLOCKLIST_URLS, is " +
|
||||
"SWW_WAF_ASN_LIMIT_PERCENT_URL or is the CrowdSec decision list too")
|
||||
errNotTagList = errors.New(
|
||||
"is not a name, = and an http or https URL, such as " +
|
||||
"torexit=https://example.com/tor.txt")
|
||||
@@ -502,8 +487,6 @@ var (
|
||||
errExploitTag = errors.New(
|
||||
"cannot lower limits: the rate limits count a request before its " +
|
||||
"rules are matched")
|
||||
errCrowdSecTag = errors.New(
|
||||
"cannot be used: a CrowdSec decision refuses the request first")
|
||||
errNoRequestGetsTag = errors.New(
|
||||
"is a tag no request can get: it is not a name in SWW_WAF_TAG_LISTS, an " +
|
||||
"asn= or country= tag, a dnsbl= tag of a zone in SWW_WAF_DNSBL_ZONES, " +
|
||||
@@ -567,7 +550,6 @@ func WAFFromEnvironment(v *viper.Viper) (*WAF, error) {
|
||||
"SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS",
|
||||
"800",
|
||||
),
|
||||
AddLookupHeaders: env.boolean("SWW_WAF_ADD_LOOKUP_HEADERS", "false"),
|
||||
ExclusivelyAllowedCountries: env.countries(
|
||||
"SWW_WAF_EXCLUSIVELY_ALLOWED_COUNTRIES", ""),
|
||||
TagLists: env.tagLists("SWW_WAF_TAG_LISTS"),
|
||||
@@ -605,10 +587,6 @@ func WAFFromEnvironment(v *viper.Viper) (*WAF, error) {
|
||||
"SWW_WAF_DNSBL_ABUSEIPDB_TIMEOUT_MS",
|
||||
"2000",
|
||||
),
|
||||
CrowdSecDecisionsURL: env.crowdSecDecisionsURL(
|
||||
"SWW_WAF_CROWDSEC_LAPI_URL",
|
||||
),
|
||||
CrowdSecKey: env.secret("SWW_WAF_CROWDSEC_LAPI_KEY"),
|
||||
BanResponse: env.banResponse("SWW_WAF_BAN_RESPONSE", "403"),
|
||||
LimitBanDuration: env.durationNotOff(
|
||||
"SWW_WAF_LIMIT_BAN_DURATION",
|
||||
@@ -746,6 +724,9 @@ func WAFFromEnvironment(v *viper.Viper) (*WAF, error) {
|
||||
|
||||
cfg.LogRemoteAppName = env.appName("SWW_WAF_LOG_REMOTE_APP_NAME",
|
||||
cfg.InstanceName, cfg.LogRemoteURL != nil)
|
||||
// The app gets the headers by default, unless no client is looked up.
|
||||
cfg.AddLookupHeaders = env.boolean("SWW_WAF_ADD_LOOKUP_HEADERS",
|
||||
strconv.FormatBool(cfg.LookupSource != off))
|
||||
cfg.BlocklistAction, cfg.BlocklistLimitPercent = env.action(
|
||||
"SWW_WAF_BLOCKLIST_ACTION", "deny")
|
||||
cfg.DNSBLAbuseIPDBAction, cfg.DNSBLAbuseIPDBLimitPercent = env.action(
|
||||
@@ -755,7 +736,6 @@ func WAFFromEnvironment(v *viper.Viper) (*WAF, error) {
|
||||
env.checkLookupDBPath(cfg)
|
||||
env.checkCountriesAndLookups(cfg)
|
||||
env.checkASNLimitPercentURL(cfg)
|
||||
env.checkCrowdSec(cfg)
|
||||
env.checkTagLists(cfg)
|
||||
env.checkTags(cfg)
|
||||
|
||||
@@ -1061,26 +1041,6 @@ func (e *environment) resolver(name string) netip.AddrPort {
|
||||
return resolver
|
||||
}
|
||||
|
||||
// crowdSecDecisionsURL reads the setting that is the URL of the CrowdSec
|
||||
// engine's local API, such as http://172.17.0.1:8080, and returns the URL
|
||||
// its decision list is fetched from, that URL with v1/decisions added to
|
||||
// its path, "" while it is unset or empty.
|
||||
func (e *environment) crowdSecDecisionsURL(name string) string {
|
||||
value := e.value(name, "")
|
||||
if value == "" {
|
||||
return ""
|
||||
}
|
||||
|
||||
lapi, err := url.Parse(value)
|
||||
if err != nil || !isHTTPURL(lapi) {
|
||||
e.check(name, fmt.Errorf("%q %w", value, errNotLAPIURL))
|
||||
|
||||
return ""
|
||||
}
|
||||
|
||||
return lapi.JoinPath("v1", "decisions").String()
|
||||
}
|
||||
|
||||
// routeWatchURL reads the setting that is the URL of a routewatch instance,
|
||||
// "" while it is unset or empty.
|
||||
func (e *environment) routeWatchURL(name string) string {
|
||||
@@ -1190,39 +1150,12 @@ func (e *environment) checkASNLimitPercentURL(cfg *WAF) {
|
||||
}
|
||||
}
|
||||
|
||||
// checkCrowdSec refuses SWW_WAF_CROWDSEC_LAPI_URL without
|
||||
// SWW_WAF_CROWDSEC_LAPI_KEY, the key without the URL, and a decision list
|
||||
// that is fetched as another list too.
|
||||
func (e *environment) checkCrowdSec(cfg *WAF) {
|
||||
decisionsURL := cfg.CrowdSecDecisionsURL
|
||||
|
||||
switch {
|
||||
case decisionsURL != "" && cfg.CrowdSecKey == "":
|
||||
e.check("SWW_WAF_CROWDSEC_LAPI_URL", fmt.Errorf(
|
||||
"is set while SWW_WAF_CROWDSEC_LAPI_KEY is unset; %w",
|
||||
errNeedsLAPIKey,
|
||||
))
|
||||
case decisionsURL == "" && cfg.CrowdSecKey != "":
|
||||
e.check("SWW_WAF_CROWDSEC_LAPI_KEY", fmt.Errorf(
|
||||
"is set while SWW_WAF_CROWDSEC_LAPI_URL is unset; %w",
|
||||
errLAPIKeyUnused,
|
||||
))
|
||||
case decisionsURL != "" && (slices.Contains(cfg.BlocklistURLs, decisionsURL) ||
|
||||
decisionsURL == cfg.ASNLimitPercentURL):
|
||||
e.check(
|
||||
"SWW_WAF_CROWDSEC_LAPI_URL",
|
||||
fmt.Errorf("gives the decision list %q, which %w",
|
||||
decisionsURL, errAnotherList),
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
// checkTagLists refuses a list of SWW_WAF_TAG_LISTS whose URL is fetched as
|
||||
// another list too: the file at a URL is read as one kind of list.
|
||||
func (e *environment) checkTagLists(cfg *WAF) {
|
||||
for _, list := range cfg.TagLists {
|
||||
if slices.Contains(cfg.BlocklistURLs, list.URL) ||
|
||||
list.URL == cfg.ASNLimitPercentURL || list.URL == cfg.CrowdSecDecisionsURL {
|
||||
list.URL == cfg.ASNLimitPercentURL {
|
||||
e.check(
|
||||
"SWW_WAF_TAG_LISTS",
|
||||
fmt.Errorf("%q %w", list.URL, errFetchedOtherwise),
|
||||
@@ -1232,8 +1165,7 @@ func (e *environment) checkTagLists(cfg *WAF) {
|
||||
}
|
||||
|
||||
// checkTags refuses a tag of SWW_WAF_TAG_DENY or SWW_WAF_TAG_LIMIT_PERCENT that would
|
||||
// never apply: crowdsec, whose request is refused before tags are acted on, and
|
||||
// any tag these settings cannot give a request.
|
||||
// never apply: any tag these settings cannot give a request.
|
||||
func (e *environment) checkTags(cfg *WAF) {
|
||||
for _, tag := range cfg.TagDeny {
|
||||
e.check("SWW_WAF_TAG_DENY", checkTag(cfg, tag))
|
||||
@@ -1246,10 +1178,6 @@ func (e *environment) checkTags(cfg *WAF) {
|
||||
|
||||
// checkTag returns an error when tag would never apply, as checkTags says.
|
||||
func checkTag(cfg *WAF, tag string) error {
|
||||
if tag == CrowdSecTag {
|
||||
return fmt.Errorf("%q %w", tag, errCrowdSecTag)
|
||||
}
|
||||
|
||||
if !requestCanGetTag(cfg, tag) {
|
||||
return fmt.Errorf("%q %w", MaskZoneKey(tag), errNoRequestGetsTag)
|
||||
}
|
||||
@@ -2010,7 +1938,6 @@ const (
|
||||
DNSBLTagPrefix = "dnsbl="
|
||||
PrefixTagPrefix = "prefix="
|
||||
AbuseIPDBTag = "abuseipdb"
|
||||
CrowdSecTag = "crowdsec"
|
||||
// ExploitSignatureTag is the tag of a request a rule file or the Core
|
||||
// Rule Set matched.
|
||||
ExploitSignatureTag = "exploitsignature"
|
||||
|
||||
+12
-152
@@ -65,8 +65,6 @@ const (
|
||||
ipReputationCacheTTL = "SWW_WAF_IP_REPUTATION_CACHE_TTL"
|
||||
dnsblAbuseIPDBTimeout = "SWW_WAF_DNSBL_ABUSEIPDB_TIMEOUT_MS"
|
||||
routeWatchURL = "SWW_WAF_ROUTEWATCH_URL"
|
||||
crowdSecURL = "SWW_WAF_CROWDSEC_LAPI_URL"
|
||||
crowdSecKey = "SWW_WAF_CROWDSEC_LAPI_KEY"
|
||||
banResponse = "SWW_WAF_BAN_RESPONSE"
|
||||
limitBanDuration = "SWW_WAF_LIMIT_BAN_DURATION"
|
||||
limitBanRepeatWindow = "SWW_WAF_LIMIT_BAN_REPEAT_WINDOW"
|
||||
@@ -289,6 +287,7 @@ func TestDefaults(t *testing.T) {
|
||||
})
|
||||
wantLookupSettings(t, cfg, config.WAF{
|
||||
LookupSource: defaultLookupSource, ASNAndCountryLookupTimeout: 800 * time.Millisecond,
|
||||
AddLookupHeaders: true,
|
||||
})
|
||||
wantByteLimitSettings(t, cfg, config.WAF{
|
||||
BytesLimitPerMinute: 0, BytesLimitPerHour: 50 << 30,
|
||||
@@ -1397,13 +1396,13 @@ func TestLookupSettingsAsSet(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
cfg := wafFromEnvironment(t, environment{
|
||||
asnAndCountryLookupTimeout: "500", addLookupHeaders: enabled,
|
||||
asnAndCountryLookupTimeout: "500", addLookupHeaders: disabled,
|
||||
})
|
||||
wantLookupSettings(t, cfg, config.WAF{
|
||||
LookupSource: defaultLookupSource, ASNAndCountryLookupTimeout: 500 * time.Millisecond,
|
||||
AddLookupHeaders: true,
|
||||
})
|
||||
|
||||
// With no client looked up, the headers are off unless set.
|
||||
cfg = wafFromEnvironment(t, environment{lookupSource: off})
|
||||
wantLookupSettings(t, cfg, config.WAF{
|
||||
LookupSource: off, ASNAndCountryLookupTimeout: 800 * time.Millisecond,
|
||||
@@ -1424,6 +1423,10 @@ func TestLookupDBPathGoesWithTheFileSourceAlone(t *testing.T) {
|
||||
cfg.LookupSource, cfg.LookupDBPath, path)
|
||||
}
|
||||
|
||||
if !cfg.AddLookupHeaders {
|
||||
t.Error("headers off by default with lookups from file, want on")
|
||||
}
|
||||
|
||||
for _, tc := range []struct {
|
||||
env environment
|
||||
want string
|
||||
@@ -1813,9 +1816,6 @@ func TestTagThatNoRequestCanGetIsToldApartFromOneItCan(t *testing.T) {
|
||||
"abuseipdb with a key",
|
||||
environment{abuseIPDBKey: "k", tagLimitPercent: "abuseipdb:50"}, "",
|
||||
},
|
||||
{"crowdsec, though a list has its name", environment{
|
||||
tagLists: "crowdsec=https://example.com/c.txt", tagDeny: "crowdsec",
|
||||
}, `"crowdsec" cannot be used`},
|
||||
{exploitTag, environment{tagDeny: exploitTag}, ""},
|
||||
{
|
||||
exploitTag + " with rule files off",
|
||||
@@ -1887,26 +1887,20 @@ func TestErrorOfATagSettingLeavesOutAZoneKey(t *testing.T) {
|
||||
func TestTagListURLFetchedAsAnotherListStopsTheStart(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
const lapi = "http://172.17.0.1:8080"
|
||||
|
||||
for _, tc := range []struct {
|
||||
name, listURL string
|
||||
env environment
|
||||
}{
|
||||
{"a blocklist", dropURL, environment{blocklistURLs: dropURL}},
|
||||
{"the file of AS:percent lines", asnURL, environment{asnLimitPercentURL: asnURL}},
|
||||
{
|
||||
"the CrowdSec decision list", lapi + "/v1/decisions",
|
||||
environment{"SWW_WAF_CROWDSEC_LAPI_URL": lapi, "SWW_WAF_CROWDSEC_LAPI_KEY": "key"},
|
||||
},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
tc.env[tagLists] = "dup=" + tc.listURL
|
||||
|
||||
want := tagLists + `: "` + tc.listURL + `" is in SWW_WAF_BLOCKLIST_URLS, is ` +
|
||||
"SWW_WAF_ASN_LIMIT_PERCENT_URL or is the CrowdSec decision list too"
|
||||
want := tagLists + `: "` + tc.listURL + `" is in SWW_WAF_BLOCKLIST_URLS ` +
|
||||
"or is SWW_WAF_ASN_LIMIT_PERCENT_URL too"
|
||||
|
||||
_, err := config.WAFFromEnvironment(tc.env.viper())
|
||||
if err == nil || err.Error() != want {
|
||||
@@ -2456,138 +2450,6 @@ func TestAbuseIPDBKeyIsLoggedMasked(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestCrowdSecSettingsGiveTheDecisionListAndTheKey(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
cfg := wafFromEnvironment(t, environment{})
|
||||
if cfg.CrowdSecDecisionsURL != "" || cfg.CrowdSecKey != "" {
|
||||
t.Errorf(
|
||||
"by default, the decision list %q and the key %q, want neither",
|
||||
cfg.CrowdSecDecisionsURL,
|
||||
cfg.CrowdSecKey,
|
||||
)
|
||||
}
|
||||
|
||||
for lapi, want := range map[string]string{
|
||||
"http://172.17.0.1:8080": "http://172.17.0.1:8080/v1/decisions",
|
||||
"http://172.17.0.1:8080/": "http://172.17.0.1:8080/v1/decisions",
|
||||
"https://crowdsec.example/lapi/": "https://crowdsec.example/lapi/v1/decisions",
|
||||
"https://crowdsec.example:8443/x": "https://crowdsec.example:8443/x/v1/decisions",
|
||||
} {
|
||||
cfg := wafFromEnvironment(
|
||||
t,
|
||||
environment{crowdSecURL: lapi, crowdSecKey: token},
|
||||
)
|
||||
if cfg.CrowdSecDecisionsURL != want || cfg.CrowdSecKey != token {
|
||||
t.Errorf(
|
||||
"%s=%s gave the decision list %q and the key %q, want %s and %s",
|
||||
crowdSecURL,
|
||||
lapi,
|
||||
cfg.CrowdSecDecisionsURL,
|
||||
cfg.CrowdSecKey,
|
||||
want,
|
||||
token,
|
||||
)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestInvalidCrowdSecSettingStopsTheStartSayingWhatIsWrong(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
const (
|
||||
lapi = "http://172.17.0.1:8080"
|
||||
notLAPIURL = " is not an http or https URL without a user or a fragment, " +
|
||||
"such as http://172.17.0.1:8080"
|
||||
anotherList = `gives the decision list "` + lapi + `/v1/decisions", which is in ` +
|
||||
`SWW_WAF_BLOCKLIST_URLS or is SWW_WAF_ASN_LIMIT_PERCENT_URL too`
|
||||
)
|
||||
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
env environment
|
||||
want string
|
||||
}{
|
||||
{
|
||||
"a URL that is not http",
|
||||
environment{crowdSecURL: "ftp://172.17.0.1", crowdSecKey: token},
|
||||
crowdSecURL + `: "ftp://172.17.0.1"` + notLAPIURL,
|
||||
},
|
||||
{
|
||||
"a URL with a user",
|
||||
environment{crowdSecURL: "http://bouncer@172.17.0.1:8080", crowdSecKey: token},
|
||||
crowdSecURL + `: "http://bouncer@172.17.0.1:8080"` + notLAPIURL,
|
||||
},
|
||||
{
|
||||
"the URL without the key",
|
||||
environment{crowdSecURL: lapi},
|
||||
crowdSecURL + ": is set while " + crowdSecKey + " is unset; the engine " +
|
||||
"answers no request without it",
|
||||
},
|
||||
{
|
||||
"the key without the URL",
|
||||
environment{crowdSecKey: token},
|
||||
crowdSecKey + ": is set while " + crowdSecURL + " is unset; it is sent only " +
|
||||
"to the engine at that URL",
|
||||
},
|
||||
{
|
||||
"the decision list as a blocklist too",
|
||||
environment{
|
||||
crowdSecURL: lapi, crowdSecKey: token,
|
||||
blocklistURLs: "https://lists.example/drop.txt," + lapi + "/v1/decisions",
|
||||
},
|
||||
crowdSecURL + ": " + anotherList,
|
||||
},
|
||||
{
|
||||
"the decision list as the file of AS:percent lines too",
|
||||
environment{
|
||||
crowdSecURL: lapi + "/", crowdSecKey: token,
|
||||
asnLimitPercentURL: lapi + "/v1/decisions",
|
||||
},
|
||||
crowdSecURL + ": " + anotherList,
|
||||
},
|
||||
// The key itself is never shown.
|
||||
{
|
||||
"a key with a control character",
|
||||
environment{crowdSecURL: lapi, crowdSecKey: token + "\r"},
|
||||
crowdSecKey + ": holds a control character, such as the carriage return " +
|
||||
"of a Windows line end",
|
||||
},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
_, err := config.WAFFromEnvironment(tc.env.viper())
|
||||
if err == nil || err.Error() != tc.want {
|
||||
t.Errorf("error %v, want %s", err, tc.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestCrowdSecKeyIsLoggedMasked(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
cfg := wafFromEnvironment(t, environment{
|
||||
crowdSecURL: "http://172.17.0.1:8080", crowdSecKey: token,
|
||||
})
|
||||
|
||||
var out bytes.Buffer
|
||||
|
||||
slog.New(slog.NewJSONHandler(&out, nil)).
|
||||
Info("starting", "settings", cfg)
|
||||
|
||||
logged := out.String()
|
||||
if strings.Contains(logged, token) ||
|
||||
!strings.Contains(logged, `"`+crowdSecKey+`":"********"`) ||
|
||||
!strings.Contains(
|
||||
logged,
|
||||
`"`+crowdSecURL+`":"http://172.17.0.1:8080"`,
|
||||
) {
|
||||
t.Errorf("the key is not logged masked beside the URL: %s", logged)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSizesAndOff(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
@@ -2714,10 +2576,10 @@ func TestInvalidLookupTagOrLogValueStopsTheStart(t *testing.T) {
|
||||
{tagDeny, "a,,b"}, {tagDeny, "country=nk"}, {tagDeny, "asn=abc"},
|
||||
{tagDeny, "asn="},
|
||||
{tagDeny, "torexit,TOREXIT"}, {tagDeny, "torexti"},
|
||||
{tagDeny, zenTag}, {tagDeny, "abuseipdb"}, {tagDeny, "crowdsec"},
|
||||
{tagDeny, zenTag}, {tagDeny, "abuseipdb"},
|
||||
{tagLimitPercent, "torexti:50"},
|
||||
{tagLimitPercent, "dnsbl=zen.example:50"},
|
||||
{tagLimitPercent, "abuseipdb:50"}, {tagLimitPercent, "crowdsec:50"},
|
||||
{tagLimitPercent, "abuseipdb:50"},
|
||||
{tagLimitPercent, "vpn :50"},
|
||||
{tagLimitPercent, "vpn"}, {tagLimitPercent, ":50"},
|
||||
{tagLimitPercent, "vpn:"},
|
||||
@@ -3070,7 +2932,7 @@ func TestLogsEachSettingWithItsValue(t *testing.T) {
|
||||
lookupSource: defaultLookupSource,
|
||||
lookupDBPath: "",
|
||||
asnAndCountryLookupTimeout: "800",
|
||||
addLookupHeaders: disabled,
|
||||
addLookupHeaders: enabled,
|
||||
allowedCountries: "",
|
||||
tagLists: "",
|
||||
tagDeny: "",
|
||||
@@ -3091,8 +2953,6 @@ func TestLogsEachSettingWithItsValue(t *testing.T) {
|
||||
ipReputationCacheTTL: defaultIPReputationCacheTTL,
|
||||
dnsblAbuseIPDBTimeout: "2000",
|
||||
routeWatchURL: "",
|
||||
crowdSecURL: "",
|
||||
crowdSecKey: "",
|
||||
banResponse: "403",
|
||||
limitBanDuration: "1h",
|
||||
limitBanRepeatWindow: "24h",
|
||||
|
||||
@@ -919,6 +919,10 @@ const retypeKey = "retype"
|
||||
// system files (930120) holds.
|
||||
const htaccess = ".htaccess"
|
||||
|
||||
// binSh is the shell path /bin/sh as a query or form value, which the rule for
|
||||
// shell paths (932160) refuses.
|
||||
const binSh = "%2Fbin%2Fsh"
|
||||
|
||||
// hardValues are words that name commands and files, as a query or form
|
||||
// value. An app's tests check that its fields let them through.
|
||||
//
|
||||
|
||||
@@ -145,7 +145,7 @@ func TestForgejoExclusionsAreNarrow(t *testing.T) {
|
||||
"/alice/-/packages/npm/x?q=",
|
||||
"/alice/tools/branches?other=",
|
||||
} {
|
||||
for _, value := range slices.Concat([]string{"%2Fbin%2Fsh"}, hardValues) {
|
||||
for _, value := range slices.Concat([]string{binSh}, hardValues) {
|
||||
if got := inspect(t, crs, get(target+value)); got.Score < threshold {
|
||||
t.Errorf("%s%s let through: %+v", target, value, got)
|
||||
}
|
||||
|
||||
@@ -7,7 +7,8 @@
|
||||
#
|
||||
# Ghost's admin and its integrations call the Admin API under /ghost/api/admin/,
|
||||
# themes and Ghost's own scripts call the Content API under /ghost/api/content/,
|
||||
# and members call /members/api/. Coraza names a field of a JSON body by its
|
||||
# and members call /members/api/. The admin calls ActivityPub, a service of its
|
||||
# own, under /.ghost/activitypub/. Coraza names a field of a JSON body by its
|
||||
# path, so the title of the first post saved is json.posts.0.title.
|
||||
|
||||
# Every rule below is skipped unless Inspect has set tx.app_ghost. It sets it
|
||||
@@ -17,11 +18,22 @@
|
||||
SecRule &TX:app_ghost "@eq 0" \
|
||||
"id:900365,phase:1,pass,nolog,skipAfter:END_GHOST"
|
||||
|
||||
# filter, a query in Ghost's own filter language, skips 932120 on the Admin API
|
||||
# and the Content API: filter=tag:powershell, filter=name:~'powershell'.
|
||||
# filter, a query in Ghost's own filter language, skips seven rules on the
|
||||
# Admin API and the Content API:
|
||||
# 932120: filter=tag:powershell, filter=name:~'powershell'
|
||||
# 932125, 932230, 932250: filter=title:~'powershell & ls -la'
|
||||
# 932235, 932260: filter=title:~'powershell & python3', which the admin's
|
||||
# post picker sends when a person types that title
|
||||
# 932380: filter=title:~'python3 & powershell'
|
||||
SecRule REQUEST_FILENAME "@rx ^/ghost/api/(?:admin|content)/" \
|
||||
"id:900350,phase:1,pass,nolog,\
|
||||
ctl:ruleRemoveTargetById=932120;ARGS:filter"
|
||||
ctl:ruleRemoveTargetById=932120;ARGS:filter,\
|
||||
ctl:ruleRemoveTargetById=932125;ARGS:filter,\
|
||||
ctl:ruleRemoveTargetById=932230;ARGS:filter,\
|
||||
ctl:ruleRemoveTargetById=932235;ARGS:filter,\
|
||||
ctl:ruleRemoveTargetById=932250;ARGS:filter,\
|
||||
ctl:ruleRemoveTargetById=932260;ARGS:filter,\
|
||||
ctl:ruleRemoveTargetById=932380;ARGS:filter"
|
||||
|
||||
# search, the search for a member by name or address, skips four rules on the
|
||||
# Admin API:
|
||||
@@ -45,10 +57,61 @@ SecRule REQUEST_FILENAME "@streq /ghost/api/admin/oembed/" \
|
||||
ctl:ruleRemoveTargetById=930120;ARGS:url,\
|
||||
ctl:ruleRemoveTargetById=932120;ARGS:url"
|
||||
|
||||
# The fields a person types plain text into skip seven rules on the Admin API:
|
||||
# The query fields that filter the analytics skip the eight rules that 900353
|
||||
# lists, on /ghost/api/admin/stats/:
|
||||
# 932120: source=powershell.example.com
|
||||
# 932235, 932260: utm_campaign=powershell & python3
|
||||
# 932250: utm_term=ls -la
|
||||
# These fields filter the figures by the site a visit came from, or a
|
||||
# campaign: source, utm_source, utm_medium, utm_campaign, utm_content and
|
||||
# utm_term.
|
||||
SecRule REQUEST_FILENAME "@beginsWith /ghost/api/admin/stats/" \
|
||||
"id:900367,phase:1,pass,nolog,\
|
||||
ctl:ruleRemoveTargetById=930120;ARGS:/^(?:source|utm_(?:source|medium|campaign|content|term))$/,\
|
||||
ctl:ruleRemoveTargetById=932120;ARGS:/^(?:source|utm_(?:source|medium|campaign|content|term))$/,\
|
||||
ctl:ruleRemoveTargetById=932125;ARGS:/^(?:source|utm_(?:source|medium|campaign|content|term))$/,\
|
||||
ctl:ruleRemoveTargetById=932230;ARGS:/^(?:source|utm_(?:source|medium|campaign|content|term))$/,\
|
||||
ctl:ruleRemoveTargetById=932235;ARGS:/^(?:source|utm_(?:source|medium|campaign|content|term))$/,\
|
||||
ctl:ruleRemoveTargetById=932250;ARGS:/^(?:source|utm_(?:source|medium|campaign|content|term))$/,\
|
||||
ctl:ruleRemoveTargetById=932260;ARGS:/^(?:source|utm_(?:source|medium|campaign|content|term))$/,\
|
||||
ctl:ruleRemoveTargetById=932380;ARGS:/^(?:source|utm_(?:source|medium|campaign|content|term))$/"
|
||||
|
||||
# pathname, the page the analytics show, skips three rules on
|
||||
# /ghost/api/admin/stats/, which refused the path of a post and of tag pages:
|
||||
# 930120: pathname=/tag/proc/
|
||||
# 932120: pathname=/python3-and-powershell/
|
||||
# 932260: pathname=/tag/python3/
|
||||
SecRule REQUEST_FILENAME "@beginsWith /ghost/api/admin/stats/" \
|
||||
"id:900385,phase:1,pass,nolog,\
|
||||
ctl:ruleRemoveTargetById=930120;ARGS:pathname,\
|
||||
ctl:ruleRemoveTargetById=932120;ARGS:pathname,\
|
||||
ctl:ruleRemoveTargetById=932260;ARGS:pathname"
|
||||
|
||||
# email_segment, the members a post is sent to as a newsletter, skips 932120
|
||||
# on the Admin API's posts: email_segment=label:powershell-users.
|
||||
SecRule REQUEST_FILENAME "@beginsWith /ghost/api/admin/posts/" \
|
||||
"id:900368,phase:1,pass,nolog,\
|
||||
ctl:ruleRemoveTargetById=932120;ARGS:email_segment"
|
||||
|
||||
# query, a search for an account in ActivityPub, skips the eight rules that
|
||||
# 900353 lists: query=python3, query=@powershell@example.com.
|
||||
SecRule REQUEST_FILENAME "@streq /.ghost/activitypub/v1/actions/search" \
|
||||
"id:900369,phase:1,pass,nolog,\
|
||||
ctl:ruleRemoveTargetById=930120;ARGS:query,\
|
||||
ctl:ruleRemoveTargetById=932120;ARGS:query,\
|
||||
ctl:ruleRemoveTargetById=932125;ARGS:query,\
|
||||
ctl:ruleRemoveTargetById=932230;ARGS:query,\
|
||||
ctl:ruleRemoveTargetById=932235;ARGS:query,\
|
||||
ctl:ruleRemoveTargetById=932250;ARGS:query,\
|
||||
ctl:ruleRemoveTargetById=932260;ARGS:query,\
|
||||
ctl:ruleRemoveTargetById=932380;ARGS:query"
|
||||
|
||||
# The fields a person types plain text into skip eight rules on the Admin API,
|
||||
# which refused words that name commands and files:
|
||||
# 930120: the title "Using python3 and ls -la to tidy .htaccess files"
|
||||
# 932120: the page's slug about-python3-powershell-htaccess
|
||||
# 932125, 932230, 932250: the tag "Linux & ls -la"
|
||||
# 932235: the title "powershell & python3"
|
||||
# 932260: the tag python3
|
||||
# 932380: the newsletter "Shell tips weekly: python3 & powershell"
|
||||
# These fields hold plain text:
|
||||
@@ -64,11 +127,12 @@ SecRule REQUEST_FILENAME "@beginsWith /ghost/api/admin/" \
|
||||
ctl:ruleRemoveTargetById=932120;ARGS:/^json\.(?:posts|pages|tags|labels|members|newsletters|snippets)\.\d+\.(?:title|name|slug|custom_excerpt|description|note|sender_name|(?:meta|og|twitter)_(?:title|description)|(?:tags|labels)\.\d+(?:\.(?:name|slug))?)$/,\
|
||||
ctl:ruleRemoveTargetById=932125;ARGS:/^json\.(?:posts|pages|tags|labels|members|newsletters|snippets)\.\d+\.(?:title|name|slug|custom_excerpt|description|note|sender_name|(?:meta|og|twitter)_(?:title|description)|(?:tags|labels)\.\d+(?:\.(?:name|slug))?)$/,\
|
||||
ctl:ruleRemoveTargetById=932230;ARGS:/^json\.(?:posts|pages|tags|labels|members|newsletters|snippets)\.\d+\.(?:title|name|slug|custom_excerpt|description|note|sender_name|(?:meta|og|twitter)_(?:title|description)|(?:tags|labels)\.\d+(?:\.(?:name|slug))?)$/,\
|
||||
ctl:ruleRemoveTargetById=932235;ARGS:/^json\.(?:posts|pages|tags|labels|members|newsletters|snippets)\.\d+\.(?:title|name|slug|custom_excerpt|description|note|sender_name|(?:meta|og|twitter)_(?:title|description)|(?:tags|labels)\.\d+(?:\.(?:name|slug))?)$/,\
|
||||
ctl:ruleRemoveTargetById=932250;ARGS:/^json\.(?:posts|pages|tags|labels|members|newsletters|snippets)\.\d+\.(?:title|name|slug|custom_excerpt|description|note|sender_name|(?:meta|og|twitter)_(?:title|description)|(?:tags|labels)\.\d+(?:\.(?:name|slug))?)$/,\
|
||||
ctl:ruleRemoveTargetById=932260;ARGS:/^json\.(?:posts|pages|tags|labels|members|newsletters|snippets)\.\d+\.(?:title|name|slug|custom_excerpt|description|note|sender_name|(?:meta|og|twitter)_(?:title|description)|(?:tags|labels)\.\d+(?:\.(?:name|slug))?)$/,\
|
||||
ctl:ruleRemoveTargetById=932380;ARGS:/^json\.(?:posts|pages|tags|labels|members|newsletters|snippets)\.\d+\.(?:title|name|slug|custom_excerpt|description|note|sender_name|(?:meta|og|twitter)_(?:title|description)|(?:tags|labels)\.\d+(?:\.(?:name|slug))?)$/"
|
||||
|
||||
# name, the name a member saves at /members/api/, skips the same seven rules.
|
||||
# name, the name a member saves at /members/api/, skips the same eight rules.
|
||||
SecRule REQUEST_FILENAME "@beginsWith /members/api/" \
|
||||
"id:900359,phase:1,pass,nolog,chain"
|
||||
SecRule &ARGS_GET:/^json\.name$/ "@eq 0" \
|
||||
@@ -76,10 +140,99 @@ SecRule REQUEST_FILENAME "@beginsWith /members/api/" \
|
||||
ctl:ruleRemoveTargetById=932120;ARGS:/^json\.name$/,\
|
||||
ctl:ruleRemoveTargetById=932125;ARGS:/^json\.name$/,\
|
||||
ctl:ruleRemoveTargetById=932230;ARGS:/^json\.name$/,\
|
||||
ctl:ruleRemoveTargetById=932235;ARGS:/^json\.name$/,\
|
||||
ctl:ruleRemoveTargetById=932250;ARGS:/^json\.name$/,\
|
||||
ctl:ruleRemoveTargetById=932260;ARGS:/^json\.name$/,\
|
||||
ctl:ruleRemoveTargetById=932380;ARGS:/^json\.name$/"
|
||||
|
||||
# More fields of plain text skip the same eight rules on the Admin API, which
|
||||
# refused "python3 & powershell club", "Weekly ls -la and .htaccess tips" and
|
||||
# "powershell & python3 deal" in them:
|
||||
# users.<n>.name, slug, bio, location and website, a staff user's profile
|
||||
# setup.<n>.blogTitle and name, creating the first staff user
|
||||
# invitation.<n>.name, accepting a staff invitation
|
||||
# tiers.<n>.name, description and benefits
|
||||
# offers.<n>.name, code, display_title and display_description
|
||||
# recommendations.<n>.title, description and excerpt
|
||||
# integrations.<n>.name and description, and webhooks.<n>.name
|
||||
# posts.<n>.feature_image_alt and feature_image_caption, and the same fields
|
||||
# of pages
|
||||
# Coraza matches a field's name in lower case, so blogTitle is blogtitle.
|
||||
SecRule REQUEST_FILENAME "@beginsWith /ghost/api/admin/" \
|
||||
"id:900370,phase:1,pass,nolog,chain"
|
||||
SecRule &ARGS_GET:/^json\.(?:users\.\d+\.(?:name|slug|bio|location|website)|setup\.\d+\.(?:blogtitle|name)|invitation\.\d+\.name|tiers\.\d+\.(?:name|description|benefits\.\d+)|offers\.\d+\.(?:name|code|display_title|display_description)|recommendations\.\d+\.(?:title|description|excerpt)|integrations\.\d+\.(?:name|description)|webhooks\.\d+\.name|(?:posts|pages)\.\d+\.feature_image_(?:alt|caption))$/ "@eq 0" \
|
||||
"ctl:ruleRemoveTargetById=930120;ARGS:/^json\.(?:users\.\d+\.(?:name|slug|bio|location|website)|setup\.\d+\.(?:blogtitle|name)|invitation\.\d+\.name|tiers\.\d+\.(?:name|description|benefits\.\d+)|offers\.\d+\.(?:name|code|display_title|display_description)|recommendations\.\d+\.(?:title|description|excerpt)|integrations\.\d+\.(?:name|description)|webhooks\.\d+\.name|(?:posts|pages)\.\d+\.feature_image_(?:alt|caption))$/,\
|
||||
ctl:ruleRemoveTargetById=932120;ARGS:/^json\.(?:users\.\d+\.(?:name|slug|bio|location|website)|setup\.\d+\.(?:blogtitle|name)|invitation\.\d+\.name|tiers\.\d+\.(?:name|description|benefits\.\d+)|offers\.\d+\.(?:name|code|display_title|display_description)|recommendations\.\d+\.(?:title|description|excerpt)|integrations\.\d+\.(?:name|description)|webhooks\.\d+\.name|(?:posts|pages)\.\d+\.feature_image_(?:alt|caption))$/,\
|
||||
ctl:ruleRemoveTargetById=932125;ARGS:/^json\.(?:users\.\d+\.(?:name|slug|bio|location|website)|setup\.\d+\.(?:blogtitle|name)|invitation\.\d+\.name|tiers\.\d+\.(?:name|description|benefits\.\d+)|offers\.\d+\.(?:name|code|display_title|display_description)|recommendations\.\d+\.(?:title|description|excerpt)|integrations\.\d+\.(?:name|description)|webhooks\.\d+\.name|(?:posts|pages)\.\d+\.feature_image_(?:alt|caption))$/,\
|
||||
ctl:ruleRemoveTargetById=932230;ARGS:/^json\.(?:users\.\d+\.(?:name|slug|bio|location|website)|setup\.\d+\.(?:blogtitle|name)|invitation\.\d+\.name|tiers\.\d+\.(?:name|description|benefits\.\d+)|offers\.\d+\.(?:name|code|display_title|display_description)|recommendations\.\d+\.(?:title|description|excerpt)|integrations\.\d+\.(?:name|description)|webhooks\.\d+\.name|(?:posts|pages)\.\d+\.feature_image_(?:alt|caption))$/,\
|
||||
ctl:ruleRemoveTargetById=932235;ARGS:/^json\.(?:users\.\d+\.(?:name|slug|bio|location|website)|setup\.\d+\.(?:blogtitle|name)|invitation\.\d+\.name|tiers\.\d+\.(?:name|description|benefits\.\d+)|offers\.\d+\.(?:name|code|display_title|display_description)|recommendations\.\d+\.(?:title|description|excerpt)|integrations\.\d+\.(?:name|description)|webhooks\.\d+\.name|(?:posts|pages)\.\d+\.feature_image_(?:alt|caption))$/,\
|
||||
ctl:ruleRemoveTargetById=932250;ARGS:/^json\.(?:users\.\d+\.(?:name|slug|bio|location|website)|setup\.\d+\.(?:blogtitle|name)|invitation\.\d+\.name|tiers\.\d+\.(?:name|description|benefits\.\d+)|offers\.\d+\.(?:name|code|display_title|display_description)|recommendations\.\d+\.(?:title|description|excerpt)|integrations\.\d+\.(?:name|description)|webhooks\.\d+\.name|(?:posts|pages)\.\d+\.feature_image_(?:alt|caption))$/,\
|
||||
ctl:ruleRemoveTargetById=932260;ARGS:/^json\.(?:users\.\d+\.(?:name|slug|bio|location|website)|setup\.\d+\.(?:blogtitle|name)|invitation\.\d+\.name|tiers\.\d+\.(?:name|description|benefits\.\d+)|offers\.\d+\.(?:name|code|display_title|display_description)|recommendations\.\d+\.(?:title|description|excerpt)|integrations\.\d+\.(?:name|description)|webhooks\.\d+\.name|(?:posts|pages)\.\d+\.feature_image_(?:alt|caption))$/,\
|
||||
ctl:ruleRemoveTargetById=932380;ARGS:/^json\.(?:users\.\d+\.(?:name|slug|bio|location|website)|setup\.\d+\.(?:blogtitle|name)|invitation\.\d+\.name|tiers\.\d+\.(?:name|description|benefits\.\d+)|offers\.\d+\.(?:name|code|display_title|display_description)|recommendations\.\d+\.(?:title|description|excerpt)|integrations\.\d+\.(?:name|description)|webhooks\.\d+\.name|(?:posts|pages)\.\d+\.feature_image_(?:alt|caption))$/"
|
||||
|
||||
# A member's comment, html at /members/api/comments/, skips the same eight
|
||||
# rules, which refused "I ran ls -la and fixed my .htaccess with python3". The
|
||||
# rules for scripts and HTML still refuse a <script> element, an event handler
|
||||
# and a javascript: link in it.
|
||||
SecRule REQUEST_FILENAME "@beginsWith /members/api/comments/" \
|
||||
"id:900371,phase:1,pass,nolog,chain"
|
||||
SecRule &ARGS_GET:/^json\.comments\.\d+\.html$/ "@eq 0" \
|
||||
"ctl:ruleRemoveTargetById=930120;ARGS:/^json\.comments\.\d+\.html$/,\
|
||||
ctl:ruleRemoveTargetById=932120;ARGS:/^json\.comments\.\d+\.html$/,\
|
||||
ctl:ruleRemoveTargetById=932125;ARGS:/^json\.comments\.\d+\.html$/,\
|
||||
ctl:ruleRemoveTargetById=932230;ARGS:/^json\.comments\.\d+\.html$/,\
|
||||
ctl:ruleRemoveTargetById=932235;ARGS:/^json\.comments\.\d+\.html$/,\
|
||||
ctl:ruleRemoveTargetById=932250;ARGS:/^json\.comments\.\d+\.html$/,\
|
||||
ctl:ruleRemoveTargetById=932260;ARGS:/^json\.comments\.\d+\.html$/,\
|
||||
ctl:ruleRemoveTargetById=932380;ARGS:/^json\.comments\.\d+\.html$/"
|
||||
|
||||
# labels, which the admin gives the members imported from a CSV file at
|
||||
# /ghost/api/admin/members/upload/, skips the same eight rules, which refused
|
||||
# "python3 import" and "powershell & .htaccess".
|
||||
SecRule REQUEST_FILENAME "@streq /ghost/api/admin/members/upload/" \
|
||||
"id:900372,phase:1,pass,nolog,chain"
|
||||
SecRule &ARGS_GET:labels "@eq 0" \
|
||||
"ctl:ruleRemoveTargetById=930120;ARGS:labels,\
|
||||
ctl:ruleRemoveTargetById=932120;ARGS:labels,\
|
||||
ctl:ruleRemoveTargetById=932125;ARGS:labels,\
|
||||
ctl:ruleRemoveTargetById=932230;ARGS:labels,\
|
||||
ctl:ruleRemoveTargetById=932235;ARGS:labels,\
|
||||
ctl:ruleRemoveTargetById=932250;ARGS:labels,\
|
||||
ctl:ruleRemoveTargetById=932260;ARGS:labels,\
|
||||
ctl:ruleRemoveTargetById=932380;ARGS:labels"
|
||||
|
||||
# In ActivityPub, a note's or a reply's text and the description of its image,
|
||||
# and an account's name, username and bio, skip the same eight rules. They
|
||||
# refused "Trying python3 & powershell", the image's description "A terminal
|
||||
# running ls -la on .htaccess with python3" and the bio "ls -la and .htaccess
|
||||
# tips". Each field skips them on its own path:
|
||||
# content and image.altText, posting a note at
|
||||
# /.ghost/activitypub/v1/actions/note or a reply under
|
||||
# /.ghost/activitypub/v1/actions/reply/
|
||||
# name, username and bio, saving the account at /.ghost/activitypub/v1/account
|
||||
SecRule REQUEST_FILENAME "@rx ^/\.ghost/activitypub/v1/actions/(?:note$|reply/)" \
|
||||
"id:900373,phase:1,pass,nolog,chain"
|
||||
SecRule &ARGS_GET:/^json\.(?:content|image\.alttext)$/ "@eq 0" \
|
||||
"ctl:ruleRemoveTargetById=930120;ARGS:/^json\.(?:content|image\.alttext)$/,\
|
||||
ctl:ruleRemoveTargetById=932120;ARGS:/^json\.(?:content|image\.alttext)$/,\
|
||||
ctl:ruleRemoveTargetById=932125;ARGS:/^json\.(?:content|image\.alttext)$/,\
|
||||
ctl:ruleRemoveTargetById=932230;ARGS:/^json\.(?:content|image\.alttext)$/,\
|
||||
ctl:ruleRemoveTargetById=932235;ARGS:/^json\.(?:content|image\.alttext)$/,\
|
||||
ctl:ruleRemoveTargetById=932250;ARGS:/^json\.(?:content|image\.alttext)$/,\
|
||||
ctl:ruleRemoveTargetById=932260;ARGS:/^json\.(?:content|image\.alttext)$/,\
|
||||
ctl:ruleRemoveTargetById=932380;ARGS:/^json\.(?:content|image\.alttext)$/"
|
||||
SecRule REQUEST_FILENAME "@streq /.ghost/activitypub/v1/account" \
|
||||
"id:900374,phase:1,pass,nolog,chain"
|
||||
SecRule &ARGS_GET:/^json\.(?:name|username|bio)$/ "@eq 0" \
|
||||
"ctl:ruleRemoveTargetById=930120;ARGS:/^json\.(?:name|username|bio)$/,\
|
||||
ctl:ruleRemoveTargetById=932120;ARGS:/^json\.(?:name|username|bio)$/,\
|
||||
ctl:ruleRemoveTargetById=932125;ARGS:/^json\.(?:name|username|bio)$/,\
|
||||
ctl:ruleRemoveTargetById=932230;ARGS:/^json\.(?:name|username|bio)$/,\
|
||||
ctl:ruleRemoveTargetById=932235;ARGS:/^json\.(?:name|username|bio)$/,\
|
||||
ctl:ruleRemoveTargetById=932250;ARGS:/^json\.(?:name|username|bio)$/,\
|
||||
ctl:ruleRemoveTargetById=932260;ARGS:/^json\.(?:name|username|bio)$/,\
|
||||
ctl:ruleRemoveTargetById=932380;ARGS:/^json\.(?:name|username|bio)$/"
|
||||
|
||||
# A member's email address skips 930120, 932120 and 932260, which refused
|
||||
# erin.htaccess@example.com, dave.powershell@example.com and
|
||||
# python3.dev@example.com. These fields hold it:
|
||||
@@ -132,6 +285,63 @@ SecRule REQUEST_FILENAME "@rx ^/ghost/api/admin/users/[0-9a-f]{24}/$" \
|
||||
ctl:ruleRemoveTargetById=932120;ARGS:/^json\.users\.\d+\.email$/,\
|
||||
ctl:ruleRemoveTargetById=932260;ARGS:/^json\.users\.\d+\.email$/"
|
||||
|
||||
# The address of a staff invitation skips the same three rules. Each of these
|
||||
# fields skips them only on its own path:
|
||||
# invites.<n>.email, inviting a staff user at /ghost/api/admin/invites/
|
||||
# email in the query, checking an invitation at
|
||||
# /ghost/api/admin/authentication/invitation/
|
||||
# invitation.<n>.email, accepting it at the same path
|
||||
SecRule REQUEST_FILENAME "@streq /ghost/api/admin/invites/" \
|
||||
"id:900375,phase:1,pass,nolog,chain"
|
||||
SecRule &ARGS_GET:/^json\.invites\.\d+\.email$/ "@eq 0" \
|
||||
"ctl:ruleRemoveTargetById=930120;ARGS:/^json\.invites\.\d+\.email$/,\
|
||||
ctl:ruleRemoveTargetById=932120;ARGS:/^json\.invites\.\d+\.email$/,\
|
||||
ctl:ruleRemoveTargetById=932260;ARGS:/^json\.invites\.\d+\.email$/"
|
||||
SecRule REQUEST_FILENAME "@streq /ghost/api/admin/authentication/invitation/" \
|
||||
"id:900376,phase:1,pass,nolog,\
|
||||
ctl:ruleRemoveTargetById=930120;ARGS:email,\
|
||||
ctl:ruleRemoveTargetById=932120;ARGS:email,\
|
||||
ctl:ruleRemoveTargetById=932260;ARGS:email"
|
||||
SecRule REQUEST_FILENAME "@streq /ghost/api/admin/authentication/invitation/" \
|
||||
"id:900377,phase:1,pass,nolog,chain"
|
||||
SecRule &ARGS_GET:/^json\.invitation\.\d+\.email$/ "@eq 0" \
|
||||
"ctl:ruleRemoveTargetById=930120;ARGS:/^json\.invitation\.\d+\.email$/,\
|
||||
ctl:ruleRemoveTargetById=932120;ARGS:/^json\.invitation\.\d+\.email$/,\
|
||||
ctl:ruleRemoveTargetById=932260;ARGS:/^json\.invitation\.\d+\.email$/"
|
||||
|
||||
# In ActivityPub, an account's handle and its address skip 930120 and 932120.
|
||||
# They refused the handles @erin.htaccess@example.com and
|
||||
# @powershell@example.com, and the addresses
|
||||
# https://example.com/users/erin.htaccess and
|
||||
# https://example.com/users/powershell. A handle starts with an @, so 932260
|
||||
# does not refuse @python3.dev@example.com. The domain of the site's handle
|
||||
# skips the same three rules as an email address, which refused
|
||||
# erin.htaccess.example.com, powershell.example.com and python3.example.com.
|
||||
# Each of these fields skips them only on its own path:
|
||||
# sourceHandle, an account that moves here, added at
|
||||
# /.ghost/activitypub/v1/aliases
|
||||
# actorUri, that account's address, when the admin removes it there
|
||||
# targetHandle, the account this one moves to, at
|
||||
# /.ghost/activitypub/v1/migration
|
||||
# domain, saved at /.ghost/activitypub/v1/domain and checked at
|
||||
# /.ghost/activitypub/v1/domain/validate
|
||||
SecRule REQUEST_FILENAME "@streq /.ghost/activitypub/v1/aliases" \
|
||||
"id:900379,phase:1,pass,nolog,chain"
|
||||
SecRule &ARGS_GET:/^json\.(?:sourcehandle|actoruri)$/ "@eq 0" \
|
||||
"ctl:ruleRemoveTargetById=930120;ARGS:/^json\.(?:sourcehandle|actoruri)$/,\
|
||||
ctl:ruleRemoveTargetById=932120;ARGS:/^json\.(?:sourcehandle|actoruri)$/"
|
||||
SecRule REQUEST_FILENAME "@streq /.ghost/activitypub/v1/migration" \
|
||||
"id:900380,phase:1,pass,nolog,chain"
|
||||
SecRule &ARGS_GET:/^json\.targethandle$/ "@eq 0" \
|
||||
"ctl:ruleRemoveTargetById=930120;ARGS:/^json\.targethandle$/,\
|
||||
ctl:ruleRemoveTargetById=932120;ARGS:/^json\.targethandle$/"
|
||||
SecRule REQUEST_FILENAME "@rx ^/\.ghost/activitypub/v1/domain(?:/validate)?$" \
|
||||
"id:900381,phase:1,pass,nolog,chain"
|
||||
SecRule &ARGS_GET:/^json\.domain$/ "@eq 0" \
|
||||
"ctl:ruleRemoveTargetById=930120;ARGS:/^json\.domain$/,\
|
||||
ctl:ruleRemoveTargetById=932120;ARGS:/^json\.domain$/,\
|
||||
ctl:ruleRemoveTargetById=932260;ARGS:/^json\.domain$/"
|
||||
|
||||
# The articles skip these rules on the Admin API:
|
||||
# 930120: ls -la /var/www/html/.htaccess in a code block
|
||||
# 932120: powershell -ExecutionPolicy Bypass -File build.ps1
|
||||
@@ -178,9 +388,10 @@ SecRule REQUEST_FILENAME "@beginsWith /ghost/api/admin/" \
|
||||
|
||||
# Code injection, the HTML Ghost adds to the head and the foot of a page, and
|
||||
# the site's settings, which hold the site-wide code injection besides the
|
||||
# title and the description, skip the rules of plain text above and these:
|
||||
# title and the description, skip the eight rules that 900353 lists and these:
|
||||
# 921130, 941100, 941160: <meta name="google-site-verification" ...>
|
||||
# 941270: <link rel="preconnect" href="https://fonts.example.com">
|
||||
# 932235, one of the eight, refused the site's title "powershell & python3".
|
||||
# These fields hold it, and skip the rules anywhere on the Admin API:
|
||||
# posts.<n>.codeinjection_head and _foot, and the same fields of pages
|
||||
# settings.<n>.value, the value of a setting
|
||||
@@ -194,6 +405,7 @@ SecRule REQUEST_FILENAME "@beginsWith /ghost/api/admin/" \
|
||||
ctl:ruleRemoveTargetById=932120;ARGS:/^json\.(?:(?:posts|pages)\.\d+\.codeinjection_(?:head|foot)|settings\.\d+\.value)$/,\
|
||||
ctl:ruleRemoveTargetById=932125;ARGS:/^json\.(?:(?:posts|pages)\.\d+\.codeinjection_(?:head|foot)|settings\.\d+\.value)$/,\
|
||||
ctl:ruleRemoveTargetById=932230;ARGS:/^json\.(?:(?:posts|pages)\.\d+\.codeinjection_(?:head|foot)|settings\.\d+\.value)$/,\
|
||||
ctl:ruleRemoveTargetById=932235;ARGS:/^json\.(?:(?:posts|pages)\.\d+\.codeinjection_(?:head|foot)|settings\.\d+\.value)$/,\
|
||||
ctl:ruleRemoveTargetById=932250;ARGS:/^json\.(?:(?:posts|pages)\.\d+\.codeinjection_(?:head|foot)|settings\.\d+\.value)$/,\
|
||||
ctl:ruleRemoveTargetById=932260;ARGS:/^json\.(?:(?:posts|pages)\.\d+\.codeinjection_(?:head|foot)|settings\.\d+\.value)$/,\
|
||||
ctl:ruleRemoveTargetById=932380;ARGS:/^json\.(?:(?:posts|pages)\.\d+\.codeinjection_(?:head|foot)|settings\.\d+\.value)$/,\
|
||||
@@ -240,6 +452,36 @@ SecRule REQUEST_FILENAME "@rx ^/ghost/api/admin/users/[0-9a-f]{24}/$" \
|
||||
SecRule &ARGS_GET:/^json\.users\.\d+\.profile_image$/ "@eq 0" \
|
||||
"ctl:ruleRemoveTargetById=930120;ARGS_NAMES:/^json\.users\.\d+\.profile_image$/"
|
||||
|
||||
# A tag's code injection, tags.<n>.codeinjection_head and _foot under
|
||||
# /ghost/api/admin/tags/, skips the rules that 900356 and 900366 switch off in
|
||||
# the code injection of posts. A javascript: link is still refused in it.
|
||||
SecRule REQUEST_FILENAME "@beginsWith /ghost/api/admin/tags/" \
|
||||
"id:900378,phase:1,pass,nolog,chain"
|
||||
SecRule &ARGS_GET:/^json\.tags\.\d+\.codeinjection_(?:head|foot)$/ "@eq 0" \
|
||||
"ctl:ruleRemoveTargetById=921130;ARGS:/^json\.tags\.\d+\.codeinjection_(?:head|foot)$/,\
|
||||
ctl:ruleRemoveTargetById=930120;ARGS:/^json\.tags\.\d+\.codeinjection_(?:head|foot)$/,\
|
||||
ctl:ruleRemoveTargetById=932120;ARGS:/^json\.tags\.\d+\.codeinjection_(?:head|foot)$/,\
|
||||
ctl:ruleRemoveTargetById=932125;ARGS:/^json\.tags\.\d+\.codeinjection_(?:head|foot)$/,\
|
||||
ctl:ruleRemoveTargetById=932230;ARGS:/^json\.tags\.\d+\.codeinjection_(?:head|foot)$/,\
|
||||
ctl:ruleRemoveTargetById=932235;ARGS:/^json\.tags\.\d+\.codeinjection_(?:head|foot)$/,\
|
||||
ctl:ruleRemoveTargetById=932250;ARGS:/^json\.tags\.\d+\.codeinjection_(?:head|foot)$/,\
|
||||
ctl:ruleRemoveTargetById=932260;ARGS:/^json\.tags\.\d+\.codeinjection_(?:head|foot)$/,\
|
||||
ctl:ruleRemoveTargetById=932380;ARGS:/^json\.tags\.\d+\.codeinjection_(?:head|foot)$/,\
|
||||
ctl:ruleRemoveTargetById=941100;ARGS:/^json\.tags\.\d+\.codeinjection_(?:head|foot)$/,\
|
||||
ctl:ruleRemoveTargetById=941110;ARGS:/^json\.tags\.\d+\.codeinjection_(?:head|foot)$/,\
|
||||
ctl:ruleRemoveTargetById=941160;ARGS:/^json\.tags\.\d+\.codeinjection_(?:head|foot)$/,\
|
||||
ctl:ruleRemoveTargetById=941270;ARGS:/^json\.tags\.\d+\.codeinjection_(?:head|foot)$/"
|
||||
|
||||
# A tag's canonical URL, tags.<n>.canonical_url under /ghost/api/admin/tags/,
|
||||
# skips the two rules that url skips:
|
||||
# 930120: https://example.com/tag/.htaccess/
|
||||
# 932120: https://example.com/tag/powershell/
|
||||
SecRule REQUEST_FILENAME "@beginsWith /ghost/api/admin/tags/" \
|
||||
"id:900382,phase:1,pass,nolog,chain"
|
||||
SecRule &ARGS_GET:/^json\.tags\.\d+\.canonical_url$/ "@eq 0" \
|
||||
"ctl:ruleRemoveTargetById=930120;ARGS:/^json\.tags\.\d+\.canonical_url$/,\
|
||||
ctl:ruleRemoveTargetById=932120;ARGS:/^json\.tags\.\d+\.canonical_url$/"
|
||||
|
||||
# The passwords, which skip each rule that random passwords set off:
|
||||
# password, signing in at /ghost/api/admin/session/
|
||||
# setup.<n>.password, creating the first staff user
|
||||
|
||||
@@ -28,6 +28,19 @@ const (
|
||||
ghostUser = "/ghost/api/admin/users/6af4570fd281b00001a59891/"
|
||||
ghostMember = "/members/api/member/"
|
||||
ghostMagicLink = "/members/api/send-magic-link/"
|
||||
ghostInvites = "/ghost/api/admin/invites/"
|
||||
ghostInvitation = "/ghost/api/admin/authentication/invitation/"
|
||||
ghostTiers = "/ghost/api/admin/tiers/6af45710d281b00001a598a3/"
|
||||
ghostOffers = "/ghost/api/admin/offers/"
|
||||
ghostRecommended = "/ghost/api/admin/recommendations/"
|
||||
ghostTags = "/ghost/api/admin/tags/"
|
||||
ghostImport = "/ghost/api/admin/members/upload/"
|
||||
ghostComments = "/members/api/comments/"
|
||||
ghostNote = "/.ghost/activitypub/v1/actions/note"
|
||||
ghostAccount = "/.ghost/activitypub/v1/account"
|
||||
ghostAliases = "/.ghost/activitypub/v1/aliases"
|
||||
ghostMigration = "/.ghost/activitypub/v1/migration"
|
||||
ghostDomain = "/.ghost/activitypub/v1/domain"
|
||||
)
|
||||
|
||||
func TestGhostFixesApplyOnlyToAHostnameThatServesGhost(t *testing.T) {
|
||||
@@ -64,7 +77,7 @@ func (f ghostField) with(t *testing.T, value string) string {
|
||||
)
|
||||
}
|
||||
|
||||
// ghostTextFields hold plain text and skip the rules of 900353 and 900359.
|
||||
// ghostTextFields hold plain text and skip the eight rules that 900353 lists.
|
||||
//
|
||||
//nolint:gochecknoglobals // a constant cannot be a list
|
||||
var ghostTextFields = []ghostField{
|
||||
@@ -116,8 +129,111 @@ var ghostTextFields = []ghostField{
|
||||
`{"snippets":[{"name":"VALUE"}]}`,
|
||||
},
|
||||
{http.MethodPut, ghostMember, ghostMemberName},
|
||||
|
||||
// A staff user's profile, the first staff user, an invitation accepted.
|
||||
{http.MethodPut, ghostUser, `{"users":[{"name":"VALUE"}]}`},
|
||||
{http.MethodPut, ghostUser, `{"users":[{"slug":"VALUE"}]}`},
|
||||
{http.MethodPut, ghostUser, `{"users":[{"bio":"VALUE"}]}`},
|
||||
{http.MethodPut, ghostUser, `{"users":[{"location":"VALUE"}]}`},
|
||||
{http.MethodPut, ghostUser, `{"users":[{"website":"VALUE"}]}`},
|
||||
{http.MethodPost, ghostSetup, `{"setup":[{"blogTitle":"VALUE"}]}`},
|
||||
{http.MethodPost, ghostSetup, `{"setup":[{"name":"VALUE"}]}`},
|
||||
{http.MethodPost, ghostInvitation, `{"invitation":[{"name":"VALUE"}]}`},
|
||||
|
||||
// Tiers, offers, recommendations, integrations and webhooks.
|
||||
{http.MethodPut, ghostTiers, `{"tiers":[{"name":"VALUE"}]}`},
|
||||
{http.MethodPut, ghostTiers, `{"tiers":[{"description":"VALUE"}]}`},
|
||||
{
|
||||
http.MethodPut,
|
||||
ghostTiers,
|
||||
`{"tiers":[{"benefits":["Tips","VALUE"]}]}`,
|
||||
},
|
||||
{http.MethodPost, ghostOffers, `{"offers":[{"name":"VALUE"}]}`},
|
||||
{http.MethodPost, ghostOffers, `{"offers":[{"code":"VALUE"}]}`},
|
||||
{http.MethodPost, ghostOffers, `{"offers":[{"display_title":"VALUE"}]}`},
|
||||
{
|
||||
http.MethodPost,
|
||||
ghostOffers,
|
||||
`{"offers":[{"display_description":"VALUE"}]}`,
|
||||
},
|
||||
{
|
||||
http.MethodPost,
|
||||
ghostRecommended,
|
||||
`{"recommendations":[{"title":"VALUE"}]}`,
|
||||
},
|
||||
{
|
||||
http.MethodPost,
|
||||
ghostRecommended,
|
||||
`{"recommendations":[{"description":"VALUE"}]}`,
|
||||
},
|
||||
{
|
||||
http.MethodPost,
|
||||
ghostRecommended,
|
||||
`{"recommendations":[{"excerpt":"VALUE"}]}`,
|
||||
},
|
||||
{
|
||||
http.MethodPost,
|
||||
"/ghost/api/admin/integrations/",
|
||||
`{"integrations":[{"name":"VALUE","description":"VALUE"}]}`,
|
||||
},
|
||||
{
|
||||
http.MethodPost,
|
||||
"/ghost/api/admin/webhooks/",
|
||||
`{"webhooks":[{"event":"post.published","name":"VALUE"}]}`,
|
||||
},
|
||||
|
||||
// A feature image's alt text and caption.
|
||||
{http.MethodPut, ghostPost, `{"posts":[{"feature_image_alt":"VALUE"}]}`},
|
||||
{
|
||||
http.MethodPut,
|
||||
ghostPost,
|
||||
`{"posts":[{"feature_image_caption":` +
|
||||
`"Photo: <a href=\"https://example.com/\">VALUE</a>"}]}`,
|
||||
},
|
||||
{http.MethodPut, ghostPage, `{"pages":[{"feature_image_alt":"VALUE"}]}`},
|
||||
|
||||
// A member's comment, written and edited.
|
||||
ghostNewComment,
|
||||
{
|
||||
http.MethodPut,
|
||||
ghostComments + "6af5426ae56ecb0001938894/",
|
||||
`{"comments":[{"id":"6af5426ae56ecb0001938894","html":"<p>VALUE</p>"}]}`,
|
||||
},
|
||||
|
||||
// ActivityPub: a note, a reply, the description of their image, and the
|
||||
// account.
|
||||
{http.MethodPost, ghostNote, `{"content":"VALUE"}`},
|
||||
{http.MethodPost, ghostReply, `{"content":"VALUE"}`},
|
||||
{http.MethodPost, ghostNote, ghostNoteImage},
|
||||
{http.MethodPost, ghostReply, ghostNoteImage},
|
||||
{http.MethodPut, ghostAccount, `{"name":"VALUE","username":"alice"}`},
|
||||
{http.MethodPut, ghostAccount, `{"username":"VALUE"}`},
|
||||
{http.MethodPut, ghostAccount, `{"bio":"VALUE"}`},
|
||||
}
|
||||
|
||||
// ghostNewComment is a member's comment on a post, with VALUE in a paragraph.
|
||||
//
|
||||
//nolint:gochecknoglobals // a constant cannot be a struct
|
||||
var ghostNewComment = ghostField{
|
||||
http.MethodPost,
|
||||
ghostComments,
|
||||
`{"comments":[{"post_id":"6af458a5d281b00001a59969","html":"<p>VALUE</p>"}]}`,
|
||||
}
|
||||
|
||||
// ghostComment is the HTML of an ordinary comment, with a link.
|
||||
const ghostComment = `Try <strong>python3</strong>, then ` +
|
||||
`<a href="https://example.com/ls-la" rel="ugc">ls -la</a>.`
|
||||
|
||||
// ghostNoteImage is the body of a note or a reply in ActivityPub with an
|
||||
// image, with VALUE as the image's description.
|
||||
const ghostNoteImage = `{"content":"My desk","image":{"url":` +
|
||||
`"https://example.com/content/images/desk.png","altText":"VALUE"}}`
|
||||
|
||||
// ghostReply is the path that posts a reply to a note in ActivityPub.
|
||||
const ghostReply = "/.ghost/activitypub/v1/actions/reply/" +
|
||||
"https%3A%2F%2Fexample.com%2F.ghost%2Factivitypub%2Fnote%2F" +
|
||||
"0d6e3b4c-2a1f-4b5e-9c8d-7e6f5a4b3c2d"
|
||||
|
||||
// ghostMemberName is the body that saves a member's name at /members/api/.
|
||||
const ghostMemberName = `{"name":"VALUE"}`
|
||||
|
||||
@@ -135,6 +251,8 @@ var ghostAddressFields = []ghostField{
|
||||
{http.MethodPost, ghostMembers, `{"members":[{"email":"VALUE"}]}`},
|
||||
{http.MethodPost, ghostSession, `{"username":"VALUE"}`},
|
||||
{http.MethodPost, ghostSetup, `{"setup":[{"email":"VALUE"}]}`},
|
||||
{http.MethodPost, ghostInvites, `{"invites":[{"email":"VALUE"}]}`},
|
||||
{http.MethodPost, ghostInvitation, `{"invitation":[{"email":"VALUE"}]}`},
|
||||
{
|
||||
http.MethodPost,
|
||||
ghostPasswordReset,
|
||||
@@ -143,14 +261,41 @@ var ghostAddressFields = []ghostField{
|
||||
{http.MethodPut, ghostUser, `{"users":[{"email":"VALUE"}]}`},
|
||||
}
|
||||
|
||||
// ghostCanonicalURLFields hold the canonical URL of a post or a page, and skip
|
||||
// 930120 and 932120, as url does.
|
||||
// ghostHandleFields hold an account's handle in ActivityPub, which the admin
|
||||
// sends as an @ followed by an address, with no spaces. They skip 930120 and
|
||||
// 932120.
|
||||
//
|
||||
//nolint:gochecknoglobals // a constant cannot be a list
|
||||
var ghostCanonicalURLFields = []ghostField{
|
||||
var ghostHandleFields = []ghostField{
|
||||
{http.MethodPost, ghostAliases, `{"sourceHandle":"@VALUE"}`},
|
||||
{http.MethodPost, ghostMigration, `{"targetHandle":"@VALUE"}`},
|
||||
}
|
||||
|
||||
// ghostDomainFields hold the domain of the site's handle in ActivityPub and
|
||||
// skip 930120, 932120 and 932260.
|
||||
//
|
||||
//nolint:gochecknoglobals // a constant cannot be a list
|
||||
var ghostDomainFields = []ghostField{
|
||||
{http.MethodPut, ghostDomain, `{"domain":"VALUE"}`},
|
||||
{http.MethodPost, ghostDomain + "/validate", `{"domain":"VALUE"}`},
|
||||
}
|
||||
|
||||
// ghostURLFields hold the canonical URL of a post, a page or a tag, or the
|
||||
// address of an account removed from those that move here in ActivityPub.
|
||||
// They skip 930120 and 932120, as url does.
|
||||
//
|
||||
//nolint:gochecknoglobals // a constant cannot be a list
|
||||
var ghostURLFields = []ghostField{
|
||||
{http.MethodPut, ghostPost, `{"posts":[{"canonical_url":"VALUE"}]}`},
|
||||
{http.MethodPost, ghostPosts, `{"posts":[{"canonical_url":"VALUE"}]}`},
|
||||
{http.MethodPut, ghostPage, `{"pages":[{"canonical_url":"VALUE"}]}`},
|
||||
{http.MethodPost, ghostTags, `{"tags":[{"canonical_url":"VALUE"}]}`},
|
||||
{
|
||||
http.MethodPut,
|
||||
ghostTags + "6af45d4ed281b00001a599a8/",
|
||||
`{"tags":[{"canonical_url":"VALUE"}]}`,
|
||||
},
|
||||
{http.MethodDelete, ghostAliases, `{"actorUri":"VALUE"}`},
|
||||
}
|
||||
|
||||
// ghostPostFields hold a post or a page, skip the rules of 900355, and take a
|
||||
@@ -193,8 +338,8 @@ var ghostArticleFields = slices.Concat(
|
||||
ghostSnippetAndNewsletterFooterFields,
|
||||
)
|
||||
|
||||
// ghostCodeFields hold code injection or a setting, skip the rules of 900356,
|
||||
// and take a <script> element by 900366.
|
||||
// ghostCodeFields hold code injection or a setting, skip the rules of 900356
|
||||
// and 900378, and take a <script> element by 900366 and 900378.
|
||||
//
|
||||
//nolint:gochecknoglobals // a constant cannot be a list
|
||||
var ghostCodeFields = []ghostField{
|
||||
@@ -213,6 +358,16 @@ var ghostCodeFields = []ghostField{
|
||||
`{"settings":[{"key":"title","value":"Blog"},` +
|
||||
`{"key":"codeinjection_head","value":"VALUE"}]}`,
|
||||
},
|
||||
{
|
||||
http.MethodPost,
|
||||
ghostTags,
|
||||
`{"tags":[{"codeinjection_head":"VALUE"}]}`,
|
||||
},
|
||||
{
|
||||
http.MethodPut,
|
||||
ghostTags + "6af45d4ed281b00001a599a8/",
|
||||
`{"tags":[{"codeinjection_foot":"VALUE"}]}`,
|
||||
},
|
||||
}
|
||||
|
||||
// The query fields that skip some rules, each ending in "=".
|
||||
@@ -221,17 +376,42 @@ const (
|
||||
ghostContentFilter = "/ghost/api/content/posts/?key=c695aa1f5cdfa62f7b677c3c83&filter="
|
||||
ghostSearch = ghostMembers + "?limit=100&search="
|
||||
ghostEmbed = "/ghost/api/admin/oembed/?type=bookmark&url="
|
||||
ghostStats = "/ghost/api/admin/stats/top-content/?date_from=2026-10-13"
|
||||
ghostPathname = ghostStats + "&pathname="
|
||||
ghostSource = ghostStats + "&source="
|
||||
ghostCampaign = ghostStats + "&utm_campaign="
|
||||
ghostSegment = ghostPost + "?newsletter=default-newsletter&email_segment="
|
||||
ghostAccountSearch = "/.ghost/activitypub/v1/actions/search?query="
|
||||
ghostInvitedEmail = ghostInvitation + "?email="
|
||||
)
|
||||
|
||||
// ghostAnalyticsFilters are source and the utm_ fields, which filter the
|
||||
// analytics, each ending in "=".
|
||||
//
|
||||
//nolint:gochecknoglobals // a constant cannot be a list
|
||||
var ghostAnalyticsFilters = []string{
|
||||
ghostSource,
|
||||
ghostStats + "&utm_source=",
|
||||
ghostStats + "&utm_medium=",
|
||||
ghostCampaign,
|
||||
ghostStats + "&utm_content=",
|
||||
ghostStats + "&utm_term=",
|
||||
}
|
||||
|
||||
// ghostQueryFields are the query fields that skip some rules.
|
||||
//
|
||||
//nolint:gochecknoglobals // a constant cannot be a list
|
||||
var ghostQueryFields = []string{
|
||||
ghostAdminFilter, ghostContentFilter, ghostSearch, ghostEmbed,
|
||||
ghostPathname, ghostSource, ghostCampaign, ghostSegment,
|
||||
ghostAccountSearch, ghostInvitedEmail,
|
||||
}
|
||||
|
||||
// ghostURLs are addresses 930120 and 932120 refused in url and in a canonical
|
||||
// URL.
|
||||
// ghostCommandNames joins two command names, as an ordinary title can.
|
||||
const ghostCommandNames = "powershell & python3"
|
||||
|
||||
// ghostURLs are addresses 930120 and 932120 refused in url and in
|
||||
// ghostURLFields.
|
||||
//
|
||||
//nolint:gochecknoglobals // a constant cannot be a list
|
||||
var ghostURLs = []string{
|
||||
@@ -251,6 +431,15 @@ var ghostAddresses = []string{
|
||||
"dave.powershell@example.com",
|
||||
}
|
||||
|
||||
// ghostDomains are domains 930120, 932120 and 932260 refused.
|
||||
//
|
||||
//nolint:gochecknoglobals // a constant cannot be a list
|
||||
var ghostDomains = []string{
|
||||
"erin.htaccess.example.com",
|
||||
"python3.example.com",
|
||||
"powershell.example.com",
|
||||
}
|
||||
|
||||
// ghostArticle is an ordinary article with a code block, an HTML card and an
|
||||
// embed, which each of the rules of 900355 refused.
|
||||
const ghostArticle = `<h2>Tidy .htaccess with python3</h2>` +
|
||||
@@ -341,17 +530,23 @@ func TestGhostOrdinaryValuesPassInTheExcludedFields(t *testing.T) {
|
||||
for _, address := range ghostURLs {
|
||||
wantGhostQuery(t, ghostEmbed+url.QueryEscape(address), false)
|
||||
|
||||
for _, f := range ghostCanonicalURLFields {
|
||||
for _, f := range ghostURLFields {
|
||||
wantGhostField(t, f, address, false)
|
||||
}
|
||||
}
|
||||
|
||||
for _, address := range ghostAddresses {
|
||||
for _, f := range ghostAddressFields {
|
||||
for _, f := range slices.Concat(ghostAddressFields, ghostHandleFields) {
|
||||
wantGhostField(t, f, address, false)
|
||||
}
|
||||
}
|
||||
|
||||
for _, domain := range ghostDomains {
|
||||
for _, f := range ghostDomainFields {
|
||||
wantGhostField(t, f, domain, false)
|
||||
}
|
||||
}
|
||||
|
||||
for _, f := range ghostArticleFields {
|
||||
wantGhostField(t, f, ghostArticle, false)
|
||||
}
|
||||
@@ -360,6 +555,14 @@ func TestGhostOrdinaryValuesPassInTheExcludedFields(t *testing.T) {
|
||||
wantGhostField(t, f, ghostCodeInjection, false)
|
||||
}
|
||||
|
||||
wantGhostField(t, ghostNewComment, ghostComment, false)
|
||||
wantGhostField(
|
||||
t,
|
||||
ghostField{http.MethodPost, ghostNote, ghostNoteImage},
|
||||
"A terminal running ls -la on .htaccess with python3",
|
||||
false,
|
||||
)
|
||||
|
||||
wantGhostQuery(t, "/ghost/api/admin/posts/?filter="+
|
||||
url.QueryEscape(
|
||||
"tag:powershell+status:[draft,scheduled,published,sent]",
|
||||
@@ -393,7 +596,8 @@ func TestGhostScriptsPassInPostsAndCodeInjection(t *testing.T) {
|
||||
wantGhostField(t, f, ghostWidget, false)
|
||||
}
|
||||
|
||||
// Site code injection, saved in settings, and a post's or a page's own.
|
||||
// Site code injection, saved in settings, and a post's, a page's or a
|
||||
// tag's own.
|
||||
for _, f := range ghostCodeFields {
|
||||
wantGhostField(t, f, ghostAnalytics, false)
|
||||
}
|
||||
@@ -412,7 +616,8 @@ func TestGhostAttacksAreRefusedInTheExcludedFields(t *testing.T) {
|
||||
|
||||
for _, attack := range ghostAttacks {
|
||||
for _, f := range slices.Concat(ghostTextFields, ghostAddressFields,
|
||||
ghostCanonicalURLFields, ghostArticleFields, ghostCodeFields) {
|
||||
ghostHandleFields, ghostDomainFields, ghostURLFields,
|
||||
ghostArticleFields, ghostCodeFields) {
|
||||
wantGhostField(t, f, attack, true)
|
||||
}
|
||||
|
||||
@@ -424,7 +629,8 @@ func TestGhostAttacksAreRefusedInTheExcludedFields(t *testing.T) {
|
||||
// A script injection stays refused in every field that does not take a
|
||||
// <script> element.
|
||||
for _, f := range slices.Concat(ghostTextFields, ghostAddressFields,
|
||||
ghostCanonicalURLFields, ghostSnippetAndNewsletterFooterFields) {
|
||||
ghostHandleFields, ghostDomainFields, ghostURLFields,
|
||||
ghostSnippetAndNewsletterFooterFields) {
|
||||
wantGhostField(t, f, scriptTag, true)
|
||||
}
|
||||
|
||||
@@ -432,15 +638,19 @@ func TestGhostAttacksAreRefusedInTheExcludedFields(t *testing.T) {
|
||||
wantGhostQuery(t, target+url.QueryEscape(scriptTag), true)
|
||||
}
|
||||
|
||||
// A command after a separator stays refused wherever its rules do not
|
||||
// misfire: in the query fields, the plain text, the addresses and the
|
||||
// canonical URLs.
|
||||
for _, f := range slices.Concat(ghostTextFields, ghostAddressFields,
|
||||
ghostCanonicalURLFields) {
|
||||
// A command after a separator stays refused wherever 932235 does not
|
||||
// misfire: in the addresses, handles, the domain, ghostURLFields, search,
|
||||
// url, pathname, email_segment and email. The other fields skip 932235,
|
||||
// which refuses "powershell & python3" as it refuses this.
|
||||
for _, f := range slices.Concat(ghostAddressFields, ghostHandleFields,
|
||||
ghostDomainFields, ghostURLFields) {
|
||||
wantGhostField(t, f, commandAfterASeparator, true)
|
||||
}
|
||||
|
||||
for _, target := range ghostQueryFields {
|
||||
for _, target := range []string{
|
||||
ghostSearch, ghostEmbed, ghostPathname, ghostSegment,
|
||||
ghostInvitedEmail,
|
||||
} {
|
||||
wantGhostQuery(
|
||||
t,
|
||||
target+url.QueryEscape(commandAfterASeparator),
|
||||
@@ -448,6 +658,15 @@ func TestGhostAttacksAreRefusedInTheExcludedFields(t *testing.T) {
|
||||
)
|
||||
}
|
||||
|
||||
// A comment keeps the rules for scripts and HTML.
|
||||
wantGhostField(t, ghostNewComment, `<img src=x onerror=alert(1)>`, true)
|
||||
wantGhostField(
|
||||
t,
|
||||
ghostNewComment,
|
||||
`<a href="javascript:alert(1)">x</a>`,
|
||||
true,
|
||||
)
|
||||
|
||||
// The articles and code injection carry HTML on purpose, but a
|
||||
// javascript: link is still refused there, and a script element in
|
||||
// snippets and a newsletter's footer.
|
||||
@@ -459,12 +678,138 @@ func TestGhostAttacksAreRefusedInTheExcludedFields(t *testing.T) {
|
||||
wantGhostField(t, f, `<p>Hi</p><script src="/x.js"></script>`, true)
|
||||
}
|
||||
|
||||
// An address keeps 932250, which refuses a listing of files.
|
||||
for _, f := range ghostAddressFields {
|
||||
// An address and the domain keep 932250, which refuses a listing of
|
||||
// files. A handle keeps it too, but its @ comes before any command.
|
||||
for _, f := range slices.Concat(ghostAddressFields, ghostDomainFields) {
|
||||
wantGhostField(t, f, "ls -la /root", true)
|
||||
}
|
||||
}
|
||||
|
||||
func TestGhostTwoCommandNamesPass(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
// The site's title, saved through settings.
|
||||
siteTitle := ghostField{
|
||||
http.MethodPut,
|
||||
ghostSettings,
|
||||
`{"settings":[{"key":"title","value":"VALUE"}]}`,
|
||||
}
|
||||
wantGhostField(t, siteTitle, ghostCommandNames, false)
|
||||
|
||||
// Values that join two command names, as a person types them.
|
||||
for _, typed := range []string{
|
||||
ghostCommandNames, "python3 & powershell", "powershell & ls -la",
|
||||
"ls -la & python3",
|
||||
} {
|
||||
for _, f := range slices.Concat(ghostTextFields, ghostCodeFields) {
|
||||
wantGhostField(t, f, typed, false)
|
||||
}
|
||||
|
||||
wantGhostImportLabel(t, ghostImport, typed, false)
|
||||
|
||||
for _, target := range slices.Concat(
|
||||
ghostAnalyticsFilters,
|
||||
[]string{ghostAccountSearch},
|
||||
) {
|
||||
wantGhostQuery(t, target+url.QueryEscape(typed), false)
|
||||
}
|
||||
|
||||
// filter as the admin's post picker writes it, after a person types
|
||||
// such a title.
|
||||
filter := "status:published+title:~'" + typed + "'"
|
||||
wantGhostQuery(t, ghostAdminFilter+url.QueryEscape(filter), false)
|
||||
}
|
||||
}
|
||||
|
||||
// TestGhostOrdinaryQueriesPass checks the query fields of the analytics
|
||||
// pages, email_segment, the search of ActivityPub and the check of an
|
||||
// invitation.
|
||||
func TestGhostOrdinaryQueriesPass(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
for _, encoded := range hardValues {
|
||||
value, _ := url.QueryUnescape(encoded)
|
||||
text := url.QueryEscape("Notes on " + value + " & tidy files")
|
||||
|
||||
for _, target := range ghostAnalyticsFilters {
|
||||
wantGhostQuery(t, target+encoded, false)
|
||||
}
|
||||
|
||||
wantGhostQuery(t, ghostCampaign+text, false)
|
||||
wantGhostQuery(t, ghostAccountSearch+text, false)
|
||||
}
|
||||
|
||||
// pathname, the page the analytics show: a post and two tag pages.
|
||||
for _, path := range []string{
|
||||
"/tag/proc/", "/python3-and-powershell/", "/tag/python3/",
|
||||
} {
|
||||
wantGhostQuery(t, ghostPathname+url.QueryEscape(path), false)
|
||||
}
|
||||
|
||||
wantGhostQuery(
|
||||
t,
|
||||
ghostCampaign+url.QueryEscape(ghostCommandNames),
|
||||
false,
|
||||
)
|
||||
wantGhostQuery(
|
||||
t,
|
||||
ghostAccountSearch+"%40powershell%40example.com",
|
||||
false,
|
||||
)
|
||||
|
||||
for _, segment := range []string{
|
||||
"label:powershell-users", "label:python3", "label:ls-la",
|
||||
"label:htaccess", "status:free,tier:powershell-club",
|
||||
} {
|
||||
wantGhostQuery(t, ghostSegment+url.QueryEscape(segment), false)
|
||||
}
|
||||
|
||||
for _, address := range ghostAddresses {
|
||||
wantGhostQuery(t, ghostInvitedEmail+url.QueryEscape(address), false)
|
||||
}
|
||||
}
|
||||
|
||||
// wantGhostImportLabel checks that the Core Rule Set, reading bodies,
|
||||
// refuses label, sent as labels in a CSV import to target, or lets it
|
||||
// through.
|
||||
func wantGhostImportLabel(
|
||||
t *testing.T, target, label string, refused bool,
|
||||
) {
|
||||
t.Helper()
|
||||
|
||||
r := request{http.MethodPost, target, []string{multipart}}
|
||||
body := field("labels", label) + field("mapping[email]", "email") + end
|
||||
|
||||
got, _ := inspectBody(t, readingBodies(t), r, body)
|
||||
if refused && got.Score < ghostThreshold {
|
||||
t.Errorf("labels=%q at %s let through: %+v", label, target, got)
|
||||
}
|
||||
|
||||
if !refused && got.Score >= ghostThreshold {
|
||||
t.Errorf("labels=%q at %s refused: %+v", label, target, got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestGhostImportLabelsPass(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
for _, encoded := range hardValues {
|
||||
value, _ := url.QueryUnescape(encoded)
|
||||
wantGhostImportLabel(t, ghostImport, value+" import", false)
|
||||
}
|
||||
|
||||
wantGhostImportLabel(t, ghostImport, ghostCommandNames, false)
|
||||
|
||||
for _, attack := range ghostAttacks {
|
||||
wantGhostImportLabel(t, ghostImport, attack, true)
|
||||
}
|
||||
|
||||
// labels keeps its rules on other paths, and when the query has a field
|
||||
// of the same name.
|
||||
wantGhostImportLabel(t, ghostMembers, "python3 import", true)
|
||||
wantGhostImportLabel(t, ghostImport+"?labels=x", "python3 import", true)
|
||||
}
|
||||
|
||||
func TestGhostExclusionsAreNarrow(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
@@ -502,7 +847,6 @@ func TestGhostExclusionsAreNarrow(t *testing.T) {
|
||||
`{"settings":[{"value":"<meta name=x content=y>"}]}`,
|
||||
},
|
||||
{"/ghost/api/admin/members/", `{"members":[{"geolocation":"python3"}]}`},
|
||||
{"/ghost/api/admin/invites/", `{"invites":[{"email":"python3.dev@example.com"}]}`},
|
||||
{ghostPosts, email},
|
||||
{ghostMembers, email},
|
||||
{ghostMember, `{"members":[{"email":"python3.dev@example.com"}]}`},
|
||||
@@ -560,6 +904,7 @@ func TestGhostScriptExclusionIsNarrow(t *testing.T) {
|
||||
script = `<script defer src=\"/script.js\"></script>`
|
||||
scriptInPost = `{"posts":[{"lexical":"` + script + `"}]}`
|
||||
scriptInSettings = `{"settings":[{"value":"` + script + `"}]}`
|
||||
scriptInTag = `{"tags":[{"codeinjection_head":"` + script + `"}]}`
|
||||
)
|
||||
|
||||
for _, refused := range []struct{ target, body string }{
|
||||
@@ -569,6 +914,8 @@ func TestGhostScriptExclusionIsNarrow(t *testing.T) {
|
||||
{"/ghost/api/admin/users/me/", `{"users":[{"bio":"` + script + `"}]}`},
|
||||
{"/api/settings/", scriptInSettings},
|
||||
{ghostSettings + "?json.settings.0.value=x", scriptInSettings},
|
||||
{ghostPosts, scriptInTag},
|
||||
{ghostTags + "?json.tags.0.codeinjection_head=x", scriptInTag},
|
||||
} {
|
||||
r := request{http.MethodPut, refused.target, []string{jsonBody}}
|
||||
|
||||
@@ -590,7 +937,8 @@ func TestGhostQueryExclusionsAreNarrow(t *testing.T) {
|
||||
|
||||
// The query fields keep their rules off Ghost's paths, and in other
|
||||
// query fields there. search skips its rules only on the Admin API, and
|
||||
// url only on the editor's lookup of embeds.
|
||||
// url only on the editor's lookup of embeds. pathname, email_segment,
|
||||
// query and email skip theirs only on their own paths.
|
||||
for _, target := range []string{
|
||||
"/api/posts/?filter=powershell",
|
||||
"/ghost/posts/?filter=powershell",
|
||||
@@ -599,11 +947,79 @@ func TestGhostQueryExclusionsAreNarrow(t *testing.T) {
|
||||
"/members/api/member/?search=python3",
|
||||
"/ghost/api/admin/posts/?url=powershell",
|
||||
"/oembed/?url=powershell",
|
||||
"/ghost/api/admin/posts/?pathname=%2Fpowershell%2F",
|
||||
"/ghost/api/admin/stats/top-content/?order=powershell",
|
||||
"/ghost/api/admin/pages/?email_segment=label%3Apowershell-users",
|
||||
"/ghost/api/admin/posts/?query=python3",
|
||||
"/.ghost/activitypub/v1/actions/note?query=python3",
|
||||
"/ghost/api/admin/invites/?email=python3.dev%40example.com",
|
||||
} {
|
||||
wantGhostQuery(t, target, true)
|
||||
}
|
||||
}
|
||||
|
||||
// TestGhostStaffCommentAndActivityPubExclusionsAreNarrow checks the body
|
||||
// fields that 900370 to 900382 name, as TestGhostExclusionsAreNarrow checks
|
||||
// the others: each keeps its rules on another path, and when the query has
|
||||
// a field of the same name.
|
||||
func TestGhostStaffCommentAndActivityPubExclusionsAreNarrow(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
const (
|
||||
bio = `{"users":[{"bio":"python3"}]}`
|
||||
comment = `{"comments":[{"html":"<p>Notes on .htaccess</p>"}]}`
|
||||
content = `{"content":"python3"}`
|
||||
altText = `{"image":{"altText":"python3"}}`
|
||||
accountBio = `{"bio":"python3"}`
|
||||
invited = `{"invites":[{"email":"python3.dev@example.com"}]}`
|
||||
accepted = `{"invitation":[{"email":"python3.dev@example.com"}]}`
|
||||
sourceHandle = `{"sourceHandle":"@powershell@example.com"}`
|
||||
actorURI = `{"actorUri":"https://example.com/users/powershell"}`
|
||||
targetHandle = `{"targetHandle":"@powershell@example.com"}`
|
||||
domain = `{"domain":"python3.example.com"}`
|
||||
tagCode = `{"tags":[{"codeinjection_head":"<meta name=x content=y>"}]}`
|
||||
tagURL = `{"tags":[{"canonical_url":"https://example.com/tag/powershell/"}]}`
|
||||
)
|
||||
|
||||
for _, refused := range []struct{ target, body string }{
|
||||
{"/api/users/1/", bio},
|
||||
{ghostMember, bio},
|
||||
{ghostUser + "?json.users.0.bio=x", bio},
|
||||
{ghostPosts, comment},
|
||||
{ghostComments + "?json.comments.0.html=x", comment},
|
||||
{ghostAccount, content},
|
||||
{ghostNote + "?json.content=x", content},
|
||||
{ghostAccount, altText},
|
||||
{ghostNote + "?json.image.altText=x", altText},
|
||||
{ghostNote, accountBio},
|
||||
{ghostAccount + "?json.bio=x", accountBio},
|
||||
{ghostPosts, invited},
|
||||
{ghostInvites + "?json.invites.0.email=x", invited},
|
||||
{ghostInvites, accepted},
|
||||
{ghostInvitation + "?json.invitation.0.email=x", accepted},
|
||||
{ghostMigration, sourceHandle},
|
||||
{ghostAliases + "?json.sourceHandle=x", sourceHandle},
|
||||
{ghostMigration, actorURI},
|
||||
{ghostAliases + "?json.actorUri=x", actorURI},
|
||||
{ghostAliases, targetHandle},
|
||||
{ghostMigration + "?json.targetHandle=x", targetHandle},
|
||||
{ghostAccount, domain},
|
||||
{ghostDomain + "?json.domain=x", domain},
|
||||
{ghostPosts, tagCode},
|
||||
{ghostTags + "?json.tags.0.codeinjection_head=x", tagCode},
|
||||
{ghostPosts, tagURL},
|
||||
{ghostTags + "?json.tags.0.canonical_url=x", tagURL},
|
||||
} {
|
||||
r := request{http.MethodPut, refused.target, []string{jsonBody}}
|
||||
|
||||
got, _ := inspectBody(t, readingBodies(t), r, refused.body)
|
||||
if got.Score < ghostThreshold {
|
||||
t.Errorf("PUT %s %s let through: %+v", refused.target,
|
||||
refused.body, got)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestGhostPasswordsPass(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
|
||||
@@ -172,11 +172,13 @@ SecRule REQUEST_FILENAME "@rx ^/(?:explore/repos|user/search_candidates|\
|
||||
|
||||
# A branch, tag or package name can hold a slash, as feat/docker-support and
|
||||
# npm's @scope/name do. So q on a repository's branches, tags and packages
|
||||
# pages, on an owner's packages page and on the admin page of packages also
|
||||
# skips this rule:
|
||||
# pages, on an owner's packages page, on the admin page of packages, on the
|
||||
# API's search of packages and on the Composer and Conan registries' searches
|
||||
# also skips this rule:
|
||||
# 932160: /bin/sh
|
||||
SecRule REQUEST_FILENAME "@rx ^/(?:[^/]+/[^/]+/(?:branches|tags|packages)|\
|
||||
admin/packages)$" \
|
||||
admin/packages|api/v1/packages/[^/]+|\
|
||||
api/packages/[^/]+/(?:composer/search\.json|conan/v[12]/conans/search))$" \
|
||||
"id:900605,phase:1,pass,nolog,ctl:ruleRemoveTargetById=932160;ARGS:q"
|
||||
|
||||
# gitea's other searches send a name or a few words in q too, which skips only
|
||||
@@ -188,7 +190,11 @@ SecRule REQUEST_FILENAME "@rx ^/(?:[^/]+/[^/]+/(?:branches|tags|packages)|\
|
||||
# The searches:
|
||||
# - the dashboard's filter of repositories;
|
||||
# - the author filter of the issue and pull request lists;
|
||||
# - the lists of users and organisations, and the API's search for users;
|
||||
# - the lists of users and organisations;
|
||||
# - the API's searches of users, of topics, of an organisation's teams and of
|
||||
# emails;
|
||||
# - the searches of the Cargo, Chef, Composer and Conan registries, and the
|
||||
# NuGet registry's search service;
|
||||
# - the list of repositories a user watches;
|
||||
# - an organisation's milestones of one team, and its search for teams;
|
||||
# - the milestones of a repository or an organisation, and the projects of a
|
||||
@@ -205,7 +211,11 @@ SecRule REQUEST_FILENAME "@rx ^/(?:[^/]+/[^/]+/(?:branches|tags|packages)|\
|
||||
# versions holds the package's name, which can hold a slash, as npm's
|
||||
# @scope/name does.
|
||||
SecRule REQUEST_FILENAME "@rx ^/(?:repo/search|notifications/watching|\
|
||||
explore/(?:users|organizations)|api/v1/users/search|\
|
||||
explore/(?:users|organizations)|\
|
||||
api/v1/(?:users/search|topics/search|orgs/[^/]+/teams/search|\
|
||||
admin/emails/search)|\
|
||||
api/packages/[^/]+/(?:cargo/api/v1/crates|chef/api/v1/(?:search|cookbooks)|\
|
||||
composer/search\.json|conan/v[12]/conans/search|nuget/query)|\
|
||||
org/[^/]+/(?:milestones/[^/]+|teams/-/search)|\
|
||||
[^/]+/-/packages/[^/]+/.+/versions|\
|
||||
[^/]+/[^/]+/(?:(?:issues|pulls)/posters|issues/suggestions|milestones|\
|
||||
@@ -219,6 +229,99 @@ SecRule REQUEST_FILENAME "@rx ^/(?:repo/search|notifications/watching|\
|
||||
ctl:ruleRemoveTargetById=932250;ARGS:q,\
|
||||
ctl:ruleRemoveTargetById=932260;ARGS:q"
|
||||
|
||||
# Two of the API's searches send their words in a field other than q, which
|
||||
# skips these rules there:
|
||||
# 930120: .htaccess
|
||||
# 932120: powershell
|
||||
# 932250: ls -la
|
||||
# 932260: python3, alice/python3-app
|
||||
# The search of unadopted repositories sends an owner and a name, such as
|
||||
# alice/python3-app, in query.
|
||||
SecRule REQUEST_FILENAME "@rx ^/api/v1/admin/unadopted$" \
|
||||
"id:900607,phase:1,pass,nolog,\
|
||||
ctl:ruleRemoveTargetById=930120;ARGS:query,\
|
||||
ctl:ruleRemoveTargetById=932120;ARGS:query,\
|
||||
ctl:ruleRemoveTargetById=932250;ARGS:query,\
|
||||
ctl:ruleRemoveTargetById=932260;ARGS:query"
|
||||
|
||||
# The search of a repository's milestones sends a milestone's name in name.
|
||||
SecRule REQUEST_FILENAME "@rx ^/api/v1/repos/[^/]+/[^/]+/milestones$" \
|
||||
"id:900608,phase:1,pass,nolog,\
|
||||
ctl:ruleRemoveTargetById=930120;ARGS:name,\
|
||||
ctl:ruleRemoveTargetById=932120;ARGS:name,\
|
||||
ctl:ruleRemoveTargetById=932250;ARGS:name,\
|
||||
ctl:ruleRemoveTargetById=932260;ARGS:name"
|
||||
|
||||
# The npm registry's search sends its words in text, which skips these rules
|
||||
# there:
|
||||
# 930120: .htaccess
|
||||
# 932120: powershell
|
||||
# 932160: /bin/sh, as an npm name such as @alice/tools-cli holds a slash
|
||||
# 932250: ls -la
|
||||
# 932260: python3
|
||||
SecRule REQUEST_FILENAME "@rx ^/api/packages/[^/]+/npm/-/v1/search$" \
|
||||
"id:900609,phase:1,pass,nolog,\
|
||||
ctl:ruleRemoveTargetById=930120;ARGS:text,\
|
||||
ctl:ruleRemoveTargetById=932120;ARGS:text,\
|
||||
ctl:ruleRemoveTargetById=932160;ARGS:text,\
|
||||
ctl:ruleRemoveTargetById=932250;ARGS:text,\
|
||||
ctl:ruleRemoveTargetById=932260;ARGS:text"
|
||||
|
||||
# A Conan search takes a pattern such as zlib/*, which the rule for SQL
|
||||
# injection (942100) refused in q. q skips that rule only when the request has
|
||||
# one q, in the query string or the body, with the shape of a Conan reference.
|
||||
# A reference is a name, then optionally / and a version, then optionally @
|
||||
# and a user, then optionally / and a channel. Each part is made of letters,
|
||||
# digits, _, ., - and *. A part holds no +, since 942100 reads + as a space.
|
||||
# q must also hold no */, which ends an SQL comment, and no --, which starts
|
||||
# one. Last, 942100 must find no SQL injection in q once each comment that /*
|
||||
# starts, an unclosed one included, is replaced by a space. So zlib/* passes,
|
||||
# and SQL injection such as these stays refused:
|
||||
# 1' OR '1'='1
|
||||
# 1/**/or/**/1
|
||||
# 1/*@*/or-1
|
||||
# 1+or+1--
|
||||
# 1-or-1--
|
||||
# x/1.or.1
|
||||
# 1.having.1
|
||||
# This rule is in phase 2, because a phase 1 rule runs before the body is read
|
||||
# and cannot see a q there. It checks tx.app_gitea, since the skip above does
|
||||
# not reach phase 2.
|
||||
SecRule REQUEST_FILENAME "@rx ^/api/packages/[^/]+/conan/v[12]/conans/search$" \
|
||||
"id:900611,phase:2,pass,nolog,chain"
|
||||
SecRule &TX:app_gitea "@eq 1" "chain"
|
||||
SecRule &ARGS:q "@eq 1" "chain"
|
||||
SecRule ARGS:q "!@contains */" "chain"
|
||||
SecRule ARGS:q "!@contains --" "chain"
|
||||
SecRule ARGS:q "!@detectSQLi" "t:none,t:replaceComments,chain"
|
||||
SecRule ARGS:q "@rx ^[\w.*-]+(?:/[\w.*-]+)?(?:@[\w.*-]+(?:/[\w.*-]+)?)?$" \
|
||||
"ctl:ruleRemoveTargetById=942100;ARGS:q"
|
||||
|
||||
# The NuGet v2 API's searches, Search() and Packages(), send their words in
|
||||
# searchTerm, or in $filter as an expression such as
|
||||
# tolower(Id) eq 'python3'. Search()/$count and Packages()/$count count a
|
||||
# search's results and read the same fields. searchTerm skips these rules:
|
||||
# 930120: .htaccess
|
||||
# 932120: powershell
|
||||
# 932250: ls -la
|
||||
# 932260: python3
|
||||
# $filter skips these:
|
||||
# 930120: .htaccess
|
||||
# 932120: powershell
|
||||
# The names $filter, $orderby and $top skip the rule for MongoDB operators
|
||||
# (942290), which refused every such search.
|
||||
SecRule REQUEST_FILENAME "@rx ^/api/packages/[^/]+/nuget/(?:Search|Packages)\(\)(?:/\$count)?$" \
|
||||
"id:900612,phase:1,pass,nolog,\
|
||||
ctl:ruleRemoveTargetById=930120;ARGS:searchTerm,\
|
||||
ctl:ruleRemoveTargetById=932120;ARGS:searchTerm,\
|
||||
ctl:ruleRemoveTargetById=932250;ARGS:searchTerm,\
|
||||
ctl:ruleRemoveTargetById=932260;ARGS:searchTerm,\
|
||||
ctl:ruleRemoveTargetById=930120;ARGS:$filter,\
|
||||
ctl:ruleRemoveTargetById=932120;ARGS:$filter,\
|
||||
ctl:ruleRemoveTargetById=942290;ARGS_NAMES:$filter,\
|
||||
ctl:ruleRemoveTargetById=942290;ARGS_NAMES:$orderby,\
|
||||
ctl:ruleRemoveTargetById=942290;ARGS_NAMES:$top"
|
||||
|
||||
# The repositories on a user's or an organisation's page are searched with q
|
||||
# too. Their path has one segment, so q skips these rules on every path of one
|
||||
# segment, other apps' pages such as /index.php included:
|
||||
|
||||
@@ -3,6 +3,7 @@ package coreruleset_test
|
||||
import (
|
||||
"net/http"
|
||||
"slices"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"sneak.berlin/go/smallwebwaf/internal/coreruleset"
|
||||
@@ -182,7 +183,7 @@ func TestGiteaSearchExclusionsAreNarrow(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
attacks := []string{"%2Fbin%2Fsh", commandInjection, pathTraversal}
|
||||
attacks := []string{binSh, commandInjection, pathTraversal}
|
||||
|
||||
// /bin/sh, a shell command after a ";", a path traversal and the hard
|
||||
// values stay refused in q on paths near the searches, and in another
|
||||
@@ -248,3 +249,253 @@ func TestGiteaSearchExclusionsAreNarrow(t *testing.T) {
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// giteaAPISearches are the API's searches that gitea and Forgejo share whose
|
||||
// field skips the lists of system files and command names. Each ends in the
|
||||
// field it sends its words in.
|
||||
//
|
||||
//nolint:gochecknoglobals // a constant cannot be a list
|
||||
var giteaAPISearches = []string{
|
||||
"/api/v1/users/search?q=",
|
||||
"/api/v1/topics/search?q=",
|
||||
"/api/v1/orgs/acme/teams/search?q=",
|
||||
"/api/v1/admin/emails/search?q=",
|
||||
"/api/v1/admin/unadopted?query=",
|
||||
"/api/v1/repos/alice/tools/milestones?name=",
|
||||
}
|
||||
|
||||
// giteaAPIPackageSearch is the API's search of an owner's packages, whose q
|
||||
// also lets /bin/sh through, since a package name can hold a slash.
|
||||
const giteaAPIPackageSearch = "/api/v1/packages/alice?q="
|
||||
|
||||
func TestGiteaAttacksAreRefusedInItsAPISearches(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
checkSearchesRefuseAttacks(
|
||||
t,
|
||||
giteaAPISearches,
|
||||
[]string{giteaAPIPackageSearch},
|
||||
)
|
||||
}
|
||||
|
||||
// checkSearchesRefuseAttacks checks that SQL injection, a path traversal,
|
||||
// script injection and a shell command stay refused in the field of each
|
||||
// search, in the query string and in a form data body. Each search ends in
|
||||
// its field. /bin/sh stays refused too in the field of each of
|
||||
// refusingBinSh, but not of lettingBinSh.
|
||||
func checkSearchesRefuseAttacks(
|
||||
t *testing.T, refusingBinSh, lettingBinSh []string,
|
||||
) {
|
||||
t.Helper()
|
||||
|
||||
const threshold = 5 // SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD's default
|
||||
|
||||
defaults, bodies := atDefaults(t), readingBodies(t)
|
||||
|
||||
for _, search := range slices.Concat(refusingBinSh, lettingBinSh) {
|
||||
path, field, _ := strings.Cut(search, "?")
|
||||
|
||||
for _, attack := range []string{
|
||||
sqlInjection, pathTraversal, scriptInjection, commandInjection,
|
||||
} {
|
||||
if got := inspect(t, defaults, get(search+attack)); got.Score < threshold {
|
||||
t.Errorf("GET %s%s let through: %+v", search, attack, got)
|
||||
}
|
||||
|
||||
// The same field in a form data body.
|
||||
r := request{http.MethodPost, path, []string{formData}}
|
||||
if got, _ := inspectBody(t, bodies, r, field+attack); got.Score < threshold {
|
||||
t.Errorf(
|
||||
"POST %s %s%s let through: %+v",
|
||||
path,
|
||||
field,
|
||||
attack,
|
||||
got,
|
||||
)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
for _, search := range refusingBinSh {
|
||||
if got := inspect(t, defaults, get(search+binSh)); got.Score < threshold {
|
||||
t.Errorf("%s%s let through: %+v", search, binSh, got)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestGiteaAPISearchExclusionsAreNarrow(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
// Each search's field on paths near the searches, and another field on
|
||||
// them.
|
||||
checkValuesStayRefused(t, []string{
|
||||
"/api/v1/topics/search/x?q=",
|
||||
"/api/v1/orgs/acme/teams?q=",
|
||||
"/api/v1/orgs/acme/teams/search/x?q=",
|
||||
"/api/v1/admin/emails?q=",
|
||||
"/api/v1/admin/unadopted/alice/legacy-site?query=",
|
||||
"/api/v1/admin/users?query=",
|
||||
"/api/v1/repos/alice/tools/milestones/1?name=",
|
||||
"/api/v1/repos/alice/tools/labels?name=",
|
||||
"/api/v1/packages/alice/generic/tools-cli?q=",
|
||||
"/api/v1/topics/search?other=",
|
||||
"/api/v1/orgs/acme/teams/search?other=",
|
||||
"/api/v1/admin/emails/search?other=",
|
||||
"/api/v1/admin/unadopted?q=",
|
||||
"/api/v1/repos/alice/tools/milestones?q=",
|
||||
"/api/v1/packages/alice?other=",
|
||||
})
|
||||
}
|
||||
|
||||
// checkValuesStayRefused checks that /bin/sh, a shell command after a ";", a
|
||||
// path traversal and the hard values stay refused at the end of each target.
|
||||
func checkValuesStayRefused(t *testing.T, targets []string) {
|
||||
t.Helper()
|
||||
|
||||
const threshold = 5 // SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD's default
|
||||
|
||||
crs := atDefaults(t)
|
||||
|
||||
for _, target := range targets {
|
||||
for _, value := range slices.Concat(
|
||||
[]string{binSh, commandInjection, pathTraversal},
|
||||
hardValues,
|
||||
) {
|
||||
if got := inspect(t, crs, get(target+value)); got.Score < threshold {
|
||||
t.Errorf("%s%s let through: %+v", target, value, got)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// giteaRegistrySearches are the package registries' searches whose field
|
||||
// skips the lists of system files and command names. Each ends in the field
|
||||
// it sends its words in.
|
||||
//
|
||||
//nolint:gochecknoglobals // a constant cannot be a list
|
||||
var giteaRegistrySearches = []string{
|
||||
"/api/packages/alice/cargo/api/v1/crates?q=",
|
||||
"/api/packages/alice/chef/api/v1/search?q=",
|
||||
"/api/packages/alice/chef/api/v1/cookbooks?q=",
|
||||
"/api/packages/alice/nuget/query?q=",
|
||||
"/api/packages/alice/nuget/Search()?searchTerm=",
|
||||
"/api/packages/alice/nuget/Packages()?$filter=",
|
||||
"/api/packages/alice/nuget/Search()/$count?searchTerm=",
|
||||
"/api/packages/alice/nuget/Packages()/$count?$filter=",
|
||||
}
|
||||
|
||||
// giteaRegistrySearchesLettingBinSh are the package registries' searches
|
||||
// whose field also lets /bin/sh through, since a Composer, Conan or npm name
|
||||
// can hold a slash.
|
||||
//
|
||||
//nolint:gochecknoglobals // a constant cannot be a list
|
||||
var giteaRegistrySearchesLettingBinSh = []string{
|
||||
"/api/packages/alice/composer/search.json?q=",
|
||||
"/api/packages/alice/conan/v1/conans/search?q=",
|
||||
"/api/packages/alice/conan/v2/conans/search?q=",
|
||||
"/api/packages/alice/npm/-/v1/search?text=",
|
||||
}
|
||||
|
||||
func TestGiteaAttacksAreRefusedInItsRegistrySearches(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
checkSearchesRefuseAttacks(
|
||||
t,
|
||||
giteaRegistrySearches,
|
||||
giteaRegistrySearchesLettingBinSh,
|
||||
)
|
||||
}
|
||||
|
||||
func TestGiteaRegistrySearchExclusionsAreNarrow(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
// Each search's field on paths near the searches, and another field on
|
||||
// them.
|
||||
checkValuesStayRefused(t, []string{
|
||||
"/api/packages/alice/cargo/api/v1/crates/tools-cli?q=",
|
||||
"/api/packages/alice/chef/api/v1/cookbooks/tools?q=",
|
||||
"/api/packages/alice/composer/packages.json?q=",
|
||||
"/api/packages/alice/conan/v1/conans/tools/1.0/alice/stable/search?q=",
|
||||
"/api/packages/alice/nuget/query/x?q=",
|
||||
"/api/packages/alice/nuget/FindPackagesById()?id=",
|
||||
"/api/packages/alice/nuget/FindPackagesById()/$count?id=",
|
||||
"/api/packages/alice/nuget/Search()/$count/x?searchTerm=",
|
||||
"/api/packages/alice/npm/tools-cli?text=",
|
||||
"/api/packages/alice/cargo/api/v1/crates?other=",
|
||||
"/api/packages/alice/chef/api/v1/search?other=",
|
||||
"/api/packages/alice/composer/search.json?type=",
|
||||
"/api/packages/alice/conan/v2/conans/search?other=",
|
||||
"/api/packages/alice/nuget/query?other=",
|
||||
"/api/packages/alice/nuget/Search()?targetFramework=",
|
||||
"/api/packages/alice/nuget/Packages()?$orderby=",
|
||||
"/api/packages/alice/nuget/Search()/$count?targetFramework=",
|
||||
"/api/packages/alice/nuget/Packages()/$count?$orderby=",
|
||||
"/api/packages/alice/npm/-/v1/search?q=",
|
||||
})
|
||||
|
||||
const threshold = 5 // SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD's default
|
||||
|
||||
crs := atDefaults(t)
|
||||
|
||||
// The rule for MongoDB operators (942290) still refuses the name of
|
||||
// another operator on NuGet's searches, and $filter on NuGet's other
|
||||
// paths. The rule for SQL injection (942100) still refuses tools/* in a
|
||||
// field other than a Conan search's, SQL injection in a second q after a
|
||||
// Conan pattern, SQL comments or dots that stand in for spaces, and SQL
|
||||
// injection holding + or --.
|
||||
for _, target := range []string{
|
||||
"/api/packages/alice/nuget/Search()?$where=1",
|
||||
"/api/packages/alice/nuget/Packages()?$ne=1",
|
||||
"/api/packages/alice/nuget/Search()/$count?$where=1",
|
||||
"/api/packages/alice/nuget/FindPackagesById()?$filter=IsLatestVersion",
|
||||
"/api/packages/alice/nuget/FindPackagesById()/$count?$filter=IsLatestVersion",
|
||||
"/api/packages/alice/composer/search.json?q=tools%2F%2A",
|
||||
"/api/packages/alice/conan/v2/conans/search?other=tools%2F%2A",
|
||||
"/api/packages/alice/conan/v2/conans/search?q=tools%2F%2A&q=" +
|
||||
sqlInjection,
|
||||
"/api/packages/alice/conan/v2/conans/search?q=1%2F%2A%2A%2For%2F%2A%2A%2F1",
|
||||
"/api/packages/alice/conan/v2/conans/search?q=1%2F%2A%40%2A%2For-1",
|
||||
"/api/packages/alice/conan/v2/conans/search?q=1%2Bor%2B1--",
|
||||
"/api/packages/alice/conan/v2/conans/search?q=1-or-1--",
|
||||
"/api/packages/alice/conan/v2/conans/search?q=x%2F1.or.1",
|
||||
"/api/packages/alice/conan/v2/conans/search?q=x%2F1.and.1",
|
||||
"/api/packages/alice/conan/v2/conans/search?q=1.having.1",
|
||||
} {
|
||||
if got := inspect(t, crs, get(target)); got.Score < threshold {
|
||||
t.Errorf("%s let through: %+v", target, got)
|
||||
}
|
||||
}
|
||||
|
||||
// SQL injection in a q in the body is refused, even after a Conan
|
||||
// pattern in the query string.
|
||||
search := request{
|
||||
http.MethodPost,
|
||||
"/api/packages/alice/conan/v2/conans/search?q=zlib",
|
||||
[]string{formData},
|
||||
}
|
||||
|
||||
body := "q=" + sqlInjection
|
||||
if got, _ := inspectBody(t, readingBodies(t), search, body); got.Score < threshold {
|
||||
t.Errorf("POST %s %s let through: %+v", search.target, body, got)
|
||||
}
|
||||
|
||||
// So is SQL injection with dots for spaces, in a q sent only in the body.
|
||||
search.target = "/api/packages/alice/conan/v2/conans/search"
|
||||
|
||||
body = "q=x%2F1.or.1"
|
||||
if got, _ := inspectBody(t, readingBodies(t), search, body); got.Score < threshold {
|
||||
t.Errorf("POST %s %s let through: %+v", search.target, body, got)
|
||||
}
|
||||
|
||||
// The rule that lets a Conan pattern through is in phase 2, where it
|
||||
// checks tx.app_gitea itself: a hostname that does not serve gitea
|
||||
// refuses the pattern.
|
||||
pattern := get(
|
||||
"/api/packages/alice/conan/v2/conans/search?q=tools%2F%2A",
|
||||
)
|
||||
apps := servingApps(t, map[string][]string{
|
||||
giteaHost: {gitea}, otherHost: {wordpress},
|
||||
})
|
||||
wantResult(t, apps, withHost(pattern, giteaHost), coreruleset.Result{})
|
||||
wantResult(t, apps, withHost(pattern, otherHost), matched(942100))
|
||||
}
|
||||
+160
@@ -974,3 +974,163 @@
|
||||
{"method": "GET", "target": "/org/acme/issues/developers?type=your_repositories&sort=recentupdate&project=ZgotmplZ&state=open&q=SELECT%20%2A%20FROM%20users%20WHERE", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/org/acme/issues/developers", "Accept-Encoding: gzip, deflate", "Cookie: session=5d2e8a17c4b9f063; lang=en-US"], "body": ""}
|
||||
{"method": "GET", "target": "/org/acme/pulls/developers?type=your_repositories&sort=recentupdate&project=ZgotmplZ&state=open&q=SELECT%20%2A%20FROM%20users%20WHERE", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/org/acme/pulls/developers", "Accept-Encoding: gzip, deflate", "Cookie: session=5d2e8a17c4b9f063; lang=en-US"], "body": ""}
|
||||
{"method": "GET", "target": "/alice/tools/milestone/1?q=SELECT%20%2A%20FROM%20users%20WHERE", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/milestone/1", "Accept-Encoding: gzip, deflate", "Cookie: session=5d2e8a17c4b9f063; lang=en-US"], "body": ""}
|
||||
# The API's searches that gitea and Forgejo share, sent with curl to Forgejo
|
||||
# 17.0.1. Each searches for words that name commands and files, such as python3
|
||||
# and .htaccess, and for plain words. A package name can hold a slash, as npm's
|
||||
# @alice/tools-cli does, so the search of packages is also sent /bin/sh. The
|
||||
# search of unadopted repositories takes an owner and a name.
|
||||
{"method": "GET", "target": "/api/v1/repos/search?q=python3&limit=10", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*", "Authorization: token 772deb2aefef4c9484e8c673197b82bc880fc7ca"], "body": ""}
|
||||
{"method": "GET", "target": "/api/v1/repos/search?q=powershell&limit=10", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*", "Authorization: token 772deb2aefef4c9484e8c673197b82bc880fc7ca"], "body": ""}
|
||||
{"method": "GET", "target": "/api/v1/repos/search?q=ls%20-la&limit=10", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*", "Authorization: token 772deb2aefef4c9484e8c673197b82bc880fc7ca"], "body": ""}
|
||||
{"method": "GET", "target": "/api/v1/repos/search?q=.htaccess&limit=10", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*", "Authorization: token 772deb2aefef4c9484e8c673197b82bc880fc7ca"], "body": ""}
|
||||
{"method": "GET", "target": "/api/v1/repos/search?q=docker-compose.yml&limit=10", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*", "Authorization: token 772deb2aefef4c9484e8c673197b82bc880fc7ca"], "body": ""}
|
||||
{"method": "GET", "target": "/api/v1/repos/search?q=tools&limit=10", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*", "Authorization: token 772deb2aefef4c9484e8c673197b82bc880fc7ca"], "body": ""}
|
||||
{"method": "GET", "target": "/api/v1/users/search?q=python3", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*", "Authorization: token 772deb2aefef4c9484e8c673197b82bc880fc7ca"], "body": ""}
|
||||
{"method": "GET", "target": "/api/v1/users/search?q=powershell", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*", "Authorization: token 772deb2aefef4c9484e8c673197b82bc880fc7ca"], "body": ""}
|
||||
{"method": "GET", "target": "/api/v1/users/search?q=ls%20-la", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*", "Authorization: token 772deb2aefef4c9484e8c673197b82bc880fc7ca"], "body": ""}
|
||||
{"method": "GET", "target": "/api/v1/users/search?q=.htaccess", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*", "Authorization: token 772deb2aefef4c9484e8c673197b82bc880fc7ca"], "body": ""}
|
||||
{"method": "GET", "target": "/api/v1/users/search?q=bob", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*", "Authorization: token 772deb2aefef4c9484e8c673197b82bc880fc7ca"], "body": ""}
|
||||
{"method": "GET", "target": "/api/v1/repos/issues/search?q=python3&type=issues&state=open", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*", "Authorization: token 772deb2aefef4c9484e8c673197b82bc880fc7ca"], "body": ""}
|
||||
{"method": "GET", "target": "/api/v1/repos/issues/search?q=powershell&type=issues&state=open", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*", "Authorization: token 772deb2aefef4c9484e8c673197b82bc880fc7ca"], "body": ""}
|
||||
{"method": "GET", "target": "/api/v1/repos/issues/search?q=ls%20-la&type=issues&state=open", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*", "Authorization: token 772deb2aefef4c9484e8c673197b82bc880fc7ca"], "body": ""}
|
||||
{"method": "GET", "target": "/api/v1/repos/issues/search?q=.htaccess&type=issues&state=open", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*", "Authorization: token 772deb2aefef4c9484e8c673197b82bc880fc7ca"], "body": ""}
|
||||
{"method": "GET", "target": "/api/v1/repos/issues/search?q=docker-compose.yml&type=issues&state=open", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*", "Authorization: token 772deb2aefef4c9484e8c673197b82bc880fc7ca"], "body": ""}
|
||||
{"method": "GET", "target": "/api/v1/repos/issues/search?q=build&type=issues&state=open", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*", "Authorization: token 772deb2aefef4c9484e8c673197b82bc880fc7ca"], "body": ""}
|
||||
{"method": "GET", "target": "/api/v1/repos/alice/tools/issues?state=open&q=python3", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*", "Authorization: token 772deb2aefef4c9484e8c673197b82bc880fc7ca"], "body": ""}
|
||||
{"method": "GET", "target": "/api/v1/repos/alice/tools/issues?state=open&q=powershell", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*", "Authorization: token 772deb2aefef4c9484e8c673197b82bc880fc7ca"], "body": ""}
|
||||
{"method": "GET", "target": "/api/v1/repos/alice/tools/issues?state=open&q=ls%20-la", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*", "Authorization: token 772deb2aefef4c9484e8c673197b82bc880fc7ca"], "body": ""}
|
||||
{"method": "GET", "target": "/api/v1/repos/alice/tools/issues?state=open&q=.htaccess", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*", "Authorization: token 772deb2aefef4c9484e8c673197b82bc880fc7ca"], "body": ""}
|
||||
{"method": "GET", "target": "/api/v1/repos/alice/tools/issues?state=open&q=docker-compose.yml", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*", "Authorization: token 772deb2aefef4c9484e8c673197b82bc880fc7ca"], "body": ""}
|
||||
{"method": "GET", "target": "/api/v1/packages/alice?q=python3", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*", "Authorization: token 772deb2aefef4c9484e8c673197b82bc880fc7ca"], "body": ""}
|
||||
{"method": "GET", "target": "/api/v1/packages/alice?q=powershell", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*", "Authorization: token 772deb2aefef4c9484e8c673197b82bc880fc7ca"], "body": ""}
|
||||
{"method": "GET", "target": "/api/v1/packages/alice?q=ls%20-la", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*", "Authorization: token 772deb2aefef4c9484e8c673197b82bc880fc7ca"], "body": ""}
|
||||
{"method": "GET", "target": "/api/v1/packages/alice?q=.htaccess", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*", "Authorization: token 772deb2aefef4c9484e8c673197b82bc880fc7ca"], "body": ""}
|
||||
{"method": "GET", "target": "/api/v1/packages/alice?q=%2Fbin%2Fsh", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*", "Authorization: token 772deb2aefef4c9484e8c673197b82bc880fc7ca"], "body": ""}
|
||||
{"method": "GET", "target": "/api/v1/packages/alice?q=install-script", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*", "Authorization: token 772deb2aefef4c9484e8c673197b82bc880fc7ca"], "body": ""}
|
||||
{"method": "GET", "target": "/api/v1/packages/alice?q=%40alice%2Ftools-cli", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*", "Authorization: token 772deb2aefef4c9484e8c673197b82bc880fc7ca"], "body": ""}
|
||||
{"method": "GET", "target": "/api/v1/topics/search?q=python3", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*", "Authorization: token 772deb2aefef4c9484e8c673197b82bc880fc7ca"], "body": ""}
|
||||
{"method": "GET", "target": "/api/v1/topics/search?q=powershell", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*", "Authorization: token 772deb2aefef4c9484e8c673197b82bc880fc7ca"], "body": ""}
|
||||
{"method": "GET", "target": "/api/v1/topics/search?q=ls%20-la", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*", "Authorization: token 772deb2aefef4c9484e8c673197b82bc880fc7ca"], "body": ""}
|
||||
{"method": "GET", "target": "/api/v1/topics/search?q=.htaccess", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*", "Authorization: token 772deb2aefef4c9484e8c673197b82bc880fc7ca"], "body": ""}
|
||||
{"method": "GET", "target": "/api/v1/topics/search?q=docker", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*", "Authorization: token 772deb2aefef4c9484e8c673197b82bc880fc7ca"], "body": ""}
|
||||
{"method": "GET", "target": "/api/v1/orgs/acme/teams/search?q=python3", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*", "Authorization: token 772deb2aefef4c9484e8c673197b82bc880fc7ca"], "body": ""}
|
||||
{"method": "GET", "target": "/api/v1/orgs/acme/teams/search?q=powershell", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*", "Authorization: token 772deb2aefef4c9484e8c673197b82bc880fc7ca"], "body": ""}
|
||||
{"method": "GET", "target": "/api/v1/orgs/acme/teams/search?q=ls%20-la", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*", "Authorization: token 772deb2aefef4c9484e8c673197b82bc880fc7ca"], "body": ""}
|
||||
{"method": "GET", "target": "/api/v1/orgs/acme/teams/search?q=.htaccess", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*", "Authorization: token 772deb2aefef4c9484e8c673197b82bc880fc7ca"], "body": ""}
|
||||
{"method": "GET", "target": "/api/v1/orgs/acme/teams/search?q=developers", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*", "Authorization: token 772deb2aefef4c9484e8c673197b82bc880fc7ca"], "body": ""}
|
||||
{"method": "GET", "target": "/api/v1/admin/emails/search?q=python3", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*", "Authorization: token 772deb2aefef4c9484e8c673197b82bc880fc7ca"], "body": ""}
|
||||
{"method": "GET", "target": "/api/v1/admin/emails/search?q=powershell", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*", "Authorization: token 772deb2aefef4c9484e8c673197b82bc880fc7ca"], "body": ""}
|
||||
{"method": "GET", "target": "/api/v1/admin/emails/search?q=ls%20-la", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*", "Authorization: token 772deb2aefef4c9484e8c673197b82bc880fc7ca"], "body": ""}
|
||||
{"method": "GET", "target": "/api/v1/admin/emails/search?q=.htaccess", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*", "Authorization: token 772deb2aefef4c9484e8c673197b82bc880fc7ca"], "body": ""}
|
||||
{"method": "GET", "target": "/api/v1/admin/emails/search?q=alice", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*", "Authorization: token 772deb2aefef4c9484e8c673197b82bc880fc7ca"], "body": ""}
|
||||
{"method": "GET", "target": "/api/v1/admin/unadopted?query=python3", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*", "Authorization: token 772deb2aefef4c9484e8c673197b82bc880fc7ca"], "body": ""}
|
||||
{"method": "GET", "target": "/api/v1/admin/unadopted?query=powershell", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*", "Authorization: token 772deb2aefef4c9484e8c673197b82bc880fc7ca"], "body": ""}
|
||||
{"method": "GET", "target": "/api/v1/admin/unadopted?query=ls%20-la", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*", "Authorization: token 772deb2aefef4c9484e8c673197b82bc880fc7ca"], "body": ""}
|
||||
{"method": "GET", "target": "/api/v1/admin/unadopted?query=.htaccess", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*", "Authorization: token 772deb2aefef4c9484e8c673197b82bc880fc7ca"], "body": ""}
|
||||
{"method": "GET", "target": "/api/v1/admin/unadopted?query=alice%2Fpython3-app", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*", "Authorization: token 772deb2aefef4c9484e8c673197b82bc880fc7ca"], "body": ""}
|
||||
{"method": "GET", "target": "/api/v1/admin/unadopted?query=alice%2Flegacy-site", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*", "Authorization: token 772deb2aefef4c9484e8c673197b82bc880fc7ca"], "body": ""}
|
||||
{"method": "GET", "target": "/api/v1/repos/alice/tools/milestones?state=open&name=python3", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*", "Authorization: token 772deb2aefef4c9484e8c673197b82bc880fc7ca"], "body": ""}
|
||||
{"method": "GET", "target": "/api/v1/repos/alice/tools/milestones?state=open&name=powershell", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*", "Authorization: token 772deb2aefef4c9484e8c673197b82bc880fc7ca"], "body": ""}
|
||||
{"method": "GET", "target": "/api/v1/repos/alice/tools/milestones?state=open&name=ls%20-la", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*", "Authorization: token 772deb2aefef4c9484e8c673197b82bc880fc7ca"], "body": ""}
|
||||
{"method": "GET", "target": "/api/v1/repos/alice/tools/milestones?state=open&name=.htaccess", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*", "Authorization: token 772deb2aefef4c9484e8c673197b82bc880fc7ca"], "body": ""}
|
||||
{"method": "GET", "target": "/api/v1/repos/alice/tools/milestones?state=open&name=v1.0", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*", "Authorization: token 772deb2aefef4c9484e8c673197b82bc880fc7ca"], "body": ""}
|
||||
# The package registries' searches, sent to Forgejo 17.0.1 by each registry's
|
||||
# own client: npm, composer, conan 2 and 1, cargo, dotnet and knife. curl sent
|
||||
# the requests no client sends: the Chef cookbook list with a search, and
|
||||
# NuGet's Packages() with the search in $filter, as NuGet's client builds it for
|
||||
# a feed without Search(). curl also sent each NuGet Search() and Packages()
|
||||
# search to Search()/$count and Packages()/$count, which count its results. Each
|
||||
# searches for words that name commands and files, such as python3 and
|
||||
# .htaccess, and for plain words. A Composer, Conan or npm name can hold a
|
||||
# slash, so those searches are also sent /bin/sh.
|
||||
{"method": "GET", "target": "/api/packages/alice/npm/-/v1/search?text=python3&size=20&from=0&quality=0.65&popularity=0.98&maintenance=0.5", "headers": ["user-agent: npm/10.9.9 node/v22.23.5 linux x64 workspaces/false", "npm-auth-type: web", "npm-command: search", "Accept: */*", "Accept-Encoding: gzip,deflate", "Host: example.com", "connection: keep-alive"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/npm/-/v1/search?text=powershell&size=20&from=0&quality=0.65&popularity=0.98&maintenance=0.5", "headers": ["user-agent: npm/10.9.9 node/v22.23.5 linux x64 workspaces/false", "npm-auth-type: web", "npm-command: search", "Accept: */*", "Accept-Encoding: gzip,deflate", "Host: example.com", "connection: keep-alive"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/npm/-/v1/search?text=ls+-la&size=20&from=0&quality=0.65&popularity=0.98&maintenance=0.5", "headers": ["user-agent: npm/10.9.9 node/v22.23.5 linux x64 workspaces/false", "npm-auth-type: web", "npm-command: search", "Accept: */*", "Accept-Encoding: gzip,deflate", "Host: example.com", "connection: keep-alive"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/npm/-/v1/search?text=.htaccess&size=20&from=0&quality=0.65&popularity=0.98&maintenance=0.5", "headers": ["user-agent: npm/10.9.9 node/v22.23.5 linux x64 workspaces/false", "npm-auth-type: web", "npm-command: search", "Accept: */*", "Accept-Encoding: gzip,deflate", "Host: example.com", "connection: keep-alive"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/npm/-/v1/search?text=%2Fbin%2Fsh&size=20&from=0&quality=0.65&popularity=0.98&maintenance=0.5", "headers": ["user-agent: npm/10.9.9 node/v22.23.5 linux x64 workspaces/false", "npm-auth-type: web", "npm-command: search", "Accept: */*", "Accept-Encoding: gzip,deflate", "Host: example.com", "connection: keep-alive"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/npm/-/v1/search?text=%40alice%2Ftools-cli&size=20&from=0&quality=0.65&popularity=0.98&maintenance=0.5", "headers": ["user-agent: npm/10.9.9 node/v22.23.5 linux x64 workspaces/false", "npm-auth-type: web", "npm-command: search", "Accept: */*", "Accept-Encoding: gzip,deflate", "Host: example.com", "connection: keep-alive"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/npm/-/v1/search?text=docker-compose.yml&size=20&from=0&quality=0.65&popularity=0.98&maintenance=0.5", "headers": ["user-agent: npm/10.9.9 node/v22.23.5 linux x64 workspaces/false", "npm-auth-type: web", "npm-command: search", "Accept: */*", "Accept-Encoding: gzip,deflate", "Host: example.com", "connection: keep-alive"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/npm/-/v1/search?text=tools&size=20&from=0&quality=0.65&popularity=0.98&maintenance=0.5", "headers": ["user-agent: npm/10.9.9 node/v22.23.5 linux x64 workspaces/false", "npm-auth-type: web", "npm-command: search", "Accept: */*", "Accept-Encoding: gzip,deflate", "Host: example.com", "connection: keep-alive"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/composer/search.json?q=python3&type=", "headers": ["Host: example.com", "Accept: */*", "Accept-Encoding: deflate, gzip, br, zstd", "User-Agent: Composer/2.11.2 (Linux; 6.8.0-111-generic; PHP 8.5.12; cURL 8.23.0; cmd:search)", "Connection: keep-alive"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/composer/search.json?q=powershell&type=", "headers": ["Host: example.com", "Accept: */*", "Accept-Encoding: deflate, gzip, br, zstd", "User-Agent: Composer/2.11.2 (Linux; 6.8.0-111-generic; PHP 8.5.12; cURL 8.23.0; cmd:search)", "Connection: keep-alive"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/composer/search.json?q=ls+-la&type=", "headers": ["Host: example.com", "Accept: */*", "Accept-Encoding: deflate, gzip, br, zstd", "User-Agent: Composer/2.11.2 (Linux; 6.8.0-111-generic; PHP 8.5.12; cURL 8.23.0; cmd:search)", "Connection: keep-alive"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/composer/search.json?q=.htaccess&type=", "headers": ["Host: example.com", "Accept: */*", "Accept-Encoding: deflate, gzip, br, zstd", "User-Agent: Composer/2.11.2 (Linux; 6.8.0-111-generic; PHP 8.5.12; cURL 8.23.0; cmd:search)", "Connection: keep-alive"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/composer/search.json?q=%2Fbin%2Fsh&type=", "headers": ["Host: example.com", "Accept: */*", "Accept-Encoding: deflate, gzip, br, zstd", "User-Agent: Composer/2.11.2 (Linux; 6.8.0-111-generic; PHP 8.5.12; cURL 8.23.0; cmd:search)", "Connection: keep-alive"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/composer/search.json?q=alice%2Ftools-cli&type=", "headers": ["Host: example.com", "Accept: */*", "Accept-Encoding: deflate, gzip, br, zstd", "User-Agent: Composer/2.11.2 (Linux; 6.8.0-111-generic; PHP 8.5.12; cURL 8.23.0; cmd:search)", "Connection: keep-alive"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/composer/search.json?q=docker-compose.yml&type=", "headers": ["Host: example.com", "Accept: */*", "Accept-Encoding: deflate, gzip, br, zstd", "User-Agent: Composer/2.11.2 (Linux; 6.8.0-111-generic; PHP 8.5.12; cURL 8.23.0; cmd:search)", "Connection: keep-alive"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/composer/search.json?q=tools&type=", "headers": ["Host: example.com", "Accept: */*", "Accept-Encoding: deflate, gzip, br, zstd", "User-Agent: Composer/2.11.2 (Linux; 6.8.0-111-generic; PHP 8.5.12; cURL 8.23.0; cmd:search)", "Connection: keep-alive"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/conan/v2/conans/search?q=python3", "headers": ["Host: example.com", "User-Agent: Conan/2.34.0 (Linux 6.8.0-111-generic; Python 3.13.18; x86_64)", "Accept-Encoding: gzip, deflate", "Accept: */*", "Connection: keep-alive", "X-Client-Anonymous-Id: afce410baeaee94a86997417d92f7788a91b00c5"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/conan/v2/conans/search?q=powershell", "headers": ["Host: example.com", "User-Agent: Conan/2.34.0 (Linux 6.8.0-111-generic; Python 3.13.18; x86_64)", "Accept-Encoding: gzip, deflate", "Accept: */*", "Connection: keep-alive", "X-Client-Anonymous-Id: afce410baeaee94a86997417d92f7788a91b00c5"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/conan/v2/conans/search?q=ls+-la", "headers": ["Host: example.com", "User-Agent: Conan/2.34.0 (Linux 6.8.0-111-generic; Python 3.13.18; x86_64)", "Accept-Encoding: gzip, deflate", "Accept: */*", "Connection: keep-alive", "X-Client-Anonymous-Id: afce410baeaee94a86997417d92f7788a91b00c5"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/conan/v2/conans/search?q=.htaccess", "headers": ["Host: example.com", "User-Agent: Conan/2.34.0 (Linux 6.8.0-111-generic; Python 3.13.18; x86_64)", "Accept-Encoding: gzip, deflate", "Accept: */*", "Connection: keep-alive", "X-Client-Anonymous-Id: afce410baeaee94a86997417d92f7788a91b00c5"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/conan/v2/conans/search?q=%2Fbin%2Fsh", "headers": ["Host: example.com", "User-Agent: Conan/2.34.0 (Linux 6.8.0-111-generic; Python 3.13.18; x86_64)", "Accept-Encoding: gzip, deflate", "Accept: */*", "Connection: keep-alive", "X-Client-Anonymous-Id: afce410baeaee94a86997417d92f7788a91b00c5"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/conan/v2/conans/search?q=tools%2F%2A", "headers": ["Host: example.com", "User-Agent: Conan/2.34.0 (Linux 6.8.0-111-generic; Python 3.13.18; x86_64)", "Accept-Encoding: gzip, deflate", "Accept: */*", "Connection: keep-alive", "X-Client-Anonymous-Id: afce410baeaee94a86997417d92f7788a91b00c5"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/conan/v2/conans/search?q=tools%2F1.0%40alice%2Fstable", "headers": ["Host: example.com", "User-Agent: Conan/2.34.0 (Linux 6.8.0-111-generic; Python 3.13.18; x86_64)", "Accept-Encoding: gzip, deflate", "Accept: */*", "Connection: keep-alive", "X-Client-Anonymous-Id: afce410baeaee94a86997417d92f7788a91b00c5"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/conan/v2/conans/search?q=docker-compose.yml", "headers": ["Host: example.com", "User-Agent: Conan/2.34.0 (Linux 6.8.0-111-generic; Python 3.13.18; x86_64)", "Accept-Encoding: gzip, deflate", "Accept: */*", "Connection: keep-alive", "X-Client-Anonymous-Id: afce410baeaee94a86997417d92f7788a91b00c5"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/conan/v2/conans/search?q=tools", "headers": ["Host: example.com", "User-Agent: Conan/2.34.0 (Linux 6.8.0-111-generic; Python 3.13.18; x86_64)", "Accept-Encoding: gzip, deflate", "Accept: */*", "Connection: keep-alive", "X-Client-Anonymous-Id: afce410baeaee94a86997417d92f7788a91b00c5"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/conan/v1/conans/search?q=python3", "headers": ["Host: example.com", "User-Agent: Conan/1.66.0 (Linux 6.8.0-111-generic; Python 3.13.18; x86_64)", "Accept-Encoding: gzip, deflate", "Accept: */*", "Connection: keep-alive", "X-Client-Anonymous-Id: afce410baeaee94a86997417d92f7788a91b00c5", "X-Client-Id: "], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/conan/v1/conans/search?q=powershell", "headers": ["Host: example.com", "User-Agent: Conan/1.66.0 (Linux 6.8.0-111-generic; Python 3.13.18; x86_64)", "Accept-Encoding: gzip, deflate", "Accept: */*", "Connection: keep-alive", "X-Client-Anonymous-Id: afce410baeaee94a86997417d92f7788a91b00c5", "X-Client-Id: "], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/conan/v1/conans/search?q=ls+-la", "headers": ["Host: example.com", "User-Agent: Conan/1.66.0 (Linux 6.8.0-111-generic; Python 3.13.18; x86_64)", "Accept-Encoding: gzip, deflate", "Accept: */*", "Connection: keep-alive", "X-Client-Anonymous-Id: afce410baeaee94a86997417d92f7788a91b00c5", "X-Client-Id: "], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/conan/v1/conans/search?q=.htaccess", "headers": ["Host: example.com", "User-Agent: Conan/1.66.0 (Linux 6.8.0-111-generic; Python 3.13.18; x86_64)", "Accept-Encoding: gzip, deflate", "Accept: */*", "Connection: keep-alive", "X-Client-Anonymous-Id: afce410baeaee94a86997417d92f7788a91b00c5", "X-Client-Id: "], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/conan/v1/conans/search?q=%2Fbin%2Fsh", "headers": ["Host: example.com", "User-Agent: Conan/1.66.0 (Linux 6.8.0-111-generic; Python 3.13.18; x86_64)", "Accept-Encoding: gzip, deflate", "Accept: */*", "Connection: keep-alive", "X-Client-Anonymous-Id: afce410baeaee94a86997417d92f7788a91b00c5", "X-Client-Id: "], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/conan/v1/conans/search?q=tools%2F%2A", "headers": ["Host: example.com", "User-Agent: Conan/1.66.0 (Linux 6.8.0-111-generic; Python 3.13.18; x86_64)", "Accept-Encoding: gzip, deflate", "Accept: */*", "Connection: keep-alive", "X-Client-Anonymous-Id: afce410baeaee94a86997417d92f7788a91b00c5", "X-Client-Id: "], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/conan/v1/conans/search?q=docker-compose.yml", "headers": ["Host: example.com", "User-Agent: Conan/1.66.0 (Linux 6.8.0-111-generic; Python 3.13.18; x86_64)", "Accept-Encoding: gzip, deflate", "Accept: */*", "Connection: keep-alive", "X-Client-Anonymous-Id: afce410baeaee94a86997417d92f7788a91b00c5", "X-Client-Id: "], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/conan/v1/conans/search?q=tools", "headers": ["Host: example.com", "User-Agent: Conan/1.66.0 (Linux 6.8.0-111-generic; Python 3.13.18; x86_64)", "Accept-Encoding: gzip, deflate", "Accept: */*", "Connection: keep-alive", "X-Client-Anonymous-Id: afce410baeaee94a86997417d92f7788a91b00c5", "X-Client-Id: "], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/cargo/api/v1/crates?q=python3&per_page=10", "headers": ["Host: example.com", "User-Agent: cargo/1.99.1 (5f94df478 2026-08-27)", "Accept-Encoding: deflate, gzip", "Upgrade: h2c", "HTTP2-Settings: AAMAAABkAAQAAQAAAAIAAAAA", "accept: application/json", "Connection: Upgrade, HTTP2-Settings"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/cargo/api/v1/crates?q=powershell&per_page=10", "headers": ["Host: example.com", "User-Agent: cargo/1.99.1 (5f94df478 2026-08-27)", "Accept-Encoding: deflate, gzip", "Upgrade: h2c", "HTTP2-Settings: AAMAAABkAAQAAQAAAAIAAAAA", "accept: application/json", "Connection: Upgrade, HTTP2-Settings"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/cargo/api/v1/crates?q=ls%20%2Dla&per_page=10", "headers": ["Host: example.com", "User-Agent: cargo/1.99.1 (5f94df478 2026-08-27)", "Accept-Encoding: deflate, gzip", "Upgrade: h2c", "HTTP2-Settings: AAMAAABkAAQAAQAAAAIAAAAA", "accept: application/json", "Connection: Upgrade, HTTP2-Settings"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/cargo/api/v1/crates?q=%2Ehtaccess&per_page=10", "headers": ["Host: example.com", "User-Agent: cargo/1.99.1 (5f94df478 2026-08-27)", "Accept-Encoding: deflate, gzip", "Upgrade: h2c", "HTTP2-Settings: AAMAAABkAAQAAQAAAAIAAAAA", "accept: application/json", "Connection: Upgrade, HTTP2-Settings"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/cargo/api/v1/crates?q=docker%2Dcompose%2Eyml&per_page=10", "headers": ["Host: example.com", "User-Agent: cargo/1.99.1 (5f94df478 2026-08-27)", "Accept-Encoding: deflate, gzip", "Upgrade: h2c", "HTTP2-Settings: AAMAAABkAAQAAQAAAAIAAAAA", "accept: application/json", "Connection: Upgrade, HTTP2-Settings"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/cargo/api/v1/crates?q=tools&per_page=10", "headers": ["Host: example.com", "User-Agent: cargo/1.99.1 (5f94df478 2026-08-27)", "Accept-Encoding: deflate, gzip", "Upgrade: h2c", "HTTP2-Settings: AAMAAABkAAQAAQAAAAIAAAAA", "accept: application/json", "Connection: Upgrade, HTTP2-Settings"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/nuget/query?q=python3&skip=0&take=20&prerelease=false&semVerLevel=2.0.0", "headers": ["Host: example.com", "X-NuGet-Session-Id: bc433449-14e0-4cb0-907b-fcca1ead302b", "User-Agent: NuGet Client V3/6.14.3 (LINUX)", "X-NuGet-Client-Version: 6.14.3", "Accept-Encoding: gzip, deflate"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/nuget/query?q=powershell&skip=0&take=20&prerelease=false&semVerLevel=2.0.0", "headers": ["Host: example.com", "X-NuGet-Session-Id: e691cf8b-1859-45b1-aa67-e0987eb7474a", "User-Agent: NuGet Client V3/6.14.3 (LINUX)", "X-NuGet-Client-Version: 6.14.3", "Accept-Encoding: gzip, deflate"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/nuget/query?q=ls%20-la&skip=0&take=20&prerelease=false&semVerLevel=2.0.0", "headers": ["Host: example.com", "X-NuGet-Session-Id: 725a8ab1-a2c3-4924-84dd-a741bd6b8c07", "User-Agent: NuGet Client V3/6.14.3 (LINUX)", "X-NuGet-Client-Version: 6.14.3", "Accept-Encoding: gzip, deflate"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/nuget/query?q=.htaccess&skip=0&take=20&prerelease=false&semVerLevel=2.0.0", "headers": ["Host: example.com", "X-NuGet-Session-Id: 5b1d333f-6583-4f78-9c23-d776ebbde5cf", "User-Agent: NuGet Client V3/6.14.3 (LINUX)", "X-NuGet-Client-Version: 6.14.3", "Accept-Encoding: gzip, deflate"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/nuget/query?q=docker-compose.yml&skip=0&take=20&prerelease=false&semVerLevel=2.0.0", "headers": ["Host: example.com", "X-NuGet-Session-Id: 23a81a25-0828-4311-8d2d-66f0d5161b6b", "User-Agent: NuGet Client V3/6.14.3 (LINUX)", "X-NuGet-Client-Version: 6.14.3", "Accept-Encoding: gzip, deflate"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/nuget/query?q=tools&skip=0&take=20&prerelease=false&semVerLevel=2.0.0", "headers": ["Host: example.com", "X-NuGet-Session-Id: ebf39a79-969d-4ac5-b5a0-5119c2d43ccc", "User-Agent: NuGet Client V3/6.14.3 (LINUX)", "X-NuGet-Client-Version: 6.14.3", "Accept-Encoding: gzip, deflate"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/nuget/Search()?$filter=IsLatestVersion&searchTerm='python3'&targetFramework=''&includePrerelease=false&$skip=0&$top=20&semVerLevel=2.0.0", "headers": ["Host: example.com", "Accept: application/atom+xml, application/xml", "X-NuGet-Session-Id: 825cbe04-63e7-414a-8394-2ecf74c68211", "User-Agent: NuGet Client V3/6.14.3 (LINUX)", "X-NuGet-Client-Version: 6.14.3", "Accept-Encoding: gzip, deflate"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/nuget/Search()?$filter=IsLatestVersion&searchTerm='powershell'&targetFramework=''&includePrerelease=false&$skip=0&$top=20&semVerLevel=2.0.0", "headers": ["Host: example.com", "Accept: application/atom+xml, application/xml", "X-NuGet-Session-Id: f61bfff6-1c26-4024-bc17-6b78dcef2edc", "User-Agent: NuGet Client V3/6.14.3 (LINUX)", "X-NuGet-Client-Version: 6.14.3", "Accept-Encoding: gzip, deflate"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/nuget/Search()?$filter=IsLatestVersion&searchTerm='ls%20-la'&targetFramework=''&includePrerelease=false&$skip=0&$top=20&semVerLevel=2.0.0", "headers": ["Host: example.com", "Accept: application/atom+xml, application/xml", "X-NuGet-Session-Id: e894f984-8abf-4a23-960a-d1a09c94eaa3", "User-Agent: NuGet Client V3/6.14.3 (LINUX)", "X-NuGet-Client-Version: 6.14.3", "Accept-Encoding: gzip, deflate"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/nuget/Search()?$filter=IsLatestVersion&searchTerm='.htaccess'&targetFramework=''&includePrerelease=false&$skip=0&$top=20&semVerLevel=2.0.0", "headers": ["Host: example.com", "Accept: application/atom+xml, application/xml", "X-NuGet-Session-Id: 33e3aa1b-14cb-4854-821a-c3eb9e5be7ee", "User-Agent: NuGet Client V3/6.14.3 (LINUX)", "X-NuGet-Client-Version: 6.14.3", "Accept-Encoding: gzip, deflate"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/nuget/Search()?$filter=IsLatestVersion&searchTerm='docker-compose.yml'&targetFramework=''&includePrerelease=false&$skip=0&$top=20&semVerLevel=2.0.0", "headers": ["Host: example.com", "Accept: application/atom+xml, application/xml", "X-NuGet-Session-Id: c478a564-7cf9-4282-9f00-bf5b7966607d", "User-Agent: NuGet Client V3/6.14.3 (LINUX)", "X-NuGet-Client-Version: 6.14.3", "Accept-Encoding: gzip, deflate"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/nuget/Search()?$filter=IsLatestVersion&searchTerm='tools'&targetFramework=''&includePrerelease=false&$skip=0&$top=20&semVerLevel=2.0.0", "headers": ["Host: example.com", "Accept: application/atom+xml, application/xml", "X-NuGet-Session-Id: 8dfa5583-4005-4134-9343-ddb4c68c125b", "User-Agent: NuGet Client V3/6.14.3 (LINUX)", "X-NuGet-Client-Version: 6.14.3", "Accept-Encoding: gzip, deflate"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/chef/api/v1/search?q=python3&items=9999999&start=0", "headers": ["Accept: application/json", "Accept-Encoding: gzip;q=1.0,deflate;q=0.6,identity;q=0.3", "X-Remote-Request-Id: e7210054-9f40-4a58-8228-beeab1e3bb05", "X-Chef-Version: 19.4.46", "Host: example.com", "User-Agent: Cinc Client Knife/19.4.46 (ruby-3.4.12-p148; ohai-19.1.40; x86_64-linux; +https://cinc.sh)", "Connection: close"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/chef/api/v1/search?q=powershell&items=9999999&start=0", "headers": ["Accept: application/json", "Accept-Encoding: gzip;q=1.0,deflate;q=0.6,identity;q=0.3", "X-Remote-Request-Id: 0f3b26a8-7ca2-4b52-a9bb-c23a25c4172c", "X-Chef-Version: 19.4.46", "Host: example.com", "User-Agent: Cinc Client Knife/19.4.46 (ruby-3.4.12-p148; ohai-19.1.40; x86_64-linux; +https://cinc.sh)", "Connection: close"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/chef/api/v1/search?q=ls%20-la&items=9999999&start=0", "headers": ["Accept: application/json", "Accept-Encoding: gzip;q=1.0,deflate;q=0.6,identity;q=0.3", "X-Remote-Request-Id: d0fa60d5-6948-4149-845f-e38d340cc09a", "X-Chef-Version: 19.4.46", "Host: example.com", "User-Agent: Cinc Client Knife/19.4.46 (ruby-3.4.12-p148; ohai-19.1.40; x86_64-linux; +https://cinc.sh)", "Connection: close"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/chef/api/v1/search?q=.htaccess&items=9999999&start=0", "headers": ["Accept: application/json", "Accept-Encoding: gzip;q=1.0,deflate;q=0.6,identity;q=0.3", "X-Remote-Request-Id: 8c22d5d2-1691-4488-89ca-a405dd7a15cc", "X-Chef-Version: 19.4.46", "Host: example.com", "User-Agent: Cinc Client Knife/19.4.46 (ruby-3.4.12-p148; ohai-19.1.40; x86_64-linux; +https://cinc.sh)", "Connection: close"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/chef/api/v1/search?q=docker-compose.yml&items=9999999&start=0", "headers": ["Accept: application/json", "Accept-Encoding: gzip;q=1.0,deflate;q=0.6,identity;q=0.3", "X-Remote-Request-Id: 0a1bd4b1-0009-4d1d-931d-d1c19b26dd8b", "X-Chef-Version: 19.4.46", "Host: example.com", "User-Agent: Cinc Client Knife/19.4.46 (ruby-3.4.12-p148; ohai-19.1.40; x86_64-linux; +https://cinc.sh)", "Connection: close"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/chef/api/v1/search?q=tools&items=9999999&start=0", "headers": ["Accept: application/json", "Accept-Encoding: gzip;q=1.0,deflate;q=0.6,identity;q=0.3", "X-Remote-Request-Id: f858e4b5-10bc-4633-a517-f165a65ac4cb", "X-Chef-Version: 19.4.46", "Host: example.com", "User-Agent: Cinc Client Knife/19.4.46 (ruby-3.4.12-p148; ohai-19.1.40; x86_64-linux; +https://cinc.sh)", "Connection: close"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/chef/api/v1/cookbooks?q=python3&items=9999999&start=0", "headers": ["Host: example.com", "User-Agent: curl/8.17.0", "Accept: application/json"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/chef/api/v1/cookbooks?q=powershell&items=9999999&start=0", "headers": ["Host: example.com", "User-Agent: curl/8.17.0", "Accept: application/json"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/chef/api/v1/cookbooks?q=ls%20-la&items=9999999&start=0", "headers": ["Host: example.com", "User-Agent: curl/8.17.0", "Accept: application/json"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/chef/api/v1/cookbooks?q=.htaccess&items=9999999&start=0", "headers": ["Host: example.com", "User-Agent: curl/8.17.0", "Accept: application/json"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/chef/api/v1/cookbooks?q=docker-compose.yml&items=9999999&start=0", "headers": ["Host: example.com", "User-Agent: curl/8.17.0", "Accept: application/json"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/chef/api/v1/cookbooks?q=tools&items=9999999&start=0", "headers": ["Host: example.com", "User-Agent: curl/8.17.0", "Accept: application/json"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/nuget/Packages()?$filter=((((Id%20ne%20null)%20and%20substringof('python3',tolower(Id)))%20or%20((Description%20ne%20null)%20and%20substringof('python3',tolower(Description))))%20or%20((Tags%20ne%20null)%20and%20substringof('%20python3%20',tolower(Tags))))%20and%20IsLatestVersion&$orderby=Id&$skip=0&$top=30&semVerLevel=2.0.0", "headers": ["Host: example.com", "User-Agent: curl/8.17.0", "Accept: application/atom+xml, application/xml"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/nuget/Packages()?$filter=((((Id%20ne%20null)%20and%20substringof('powershell',tolower(Id)))%20or%20((Description%20ne%20null)%20and%20substringof('powershell',tolower(Description))))%20or%20((Tags%20ne%20null)%20and%20substringof('%20powershell%20',tolower(Tags))))%20and%20IsLatestVersion&$orderby=Id&$skip=0&$top=30&semVerLevel=2.0.0", "headers": ["Host: example.com", "User-Agent: curl/8.17.0", "Accept: application/atom+xml, application/xml"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/nuget/Packages()?$filter=((((Id%20ne%20null)%20and%20substringof('.htaccess',tolower(Id)))%20or%20((Description%20ne%20null)%20and%20substringof('.htaccess',tolower(Description))))%20or%20((Tags%20ne%20null)%20and%20substringof('%20.htaccess%20',tolower(Tags))))%20and%20IsLatestVersion&$orderby=Id&$skip=0&$top=30&semVerLevel=2.0.0", "headers": ["Host: example.com", "User-Agent: curl/8.17.0", "Accept: application/atom+xml, application/xml"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/nuget/Packages()?$filter=((((Id%20ne%20null)%20and%20substringof('docker-compose.yml',tolower(Id)))%20or%20((Description%20ne%20null)%20and%20substringof('docker-compose.yml',tolower(Description))))%20or%20((Tags%20ne%20null)%20and%20substringof('%20docker-compose.yml%20',tolower(Tags))))%20and%20IsLatestVersion&$orderby=Id&$skip=0&$top=30&semVerLevel=2.0.0", "headers": ["Host: example.com", "User-Agent: curl/8.17.0", "Accept: application/atom+xml, application/xml"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/nuget/Packages()?$filter=((((Id%20ne%20null)%20and%20substringof('tools',tolower(Id)))%20or%20((Description%20ne%20null)%20and%20substringof('tools',tolower(Description))))%20or%20((Tags%20ne%20null)%20and%20substringof('%20tools%20',tolower(Tags))))%20and%20IsLatestVersion&$orderby=Id&$skip=0&$top=30&semVerLevel=2.0.0", "headers": ["Host: example.com", "User-Agent: curl/8.17.0", "Accept: application/atom+xml, application/xml"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/nuget/Packages()?$filter=(((((((Id%20ne%20null)%20and%20substringof('ls',tolower(Id)))%20or%20((Description%20ne%20null)%20and%20substringof('ls',tolower(Description))))%20or%20((Tags%20ne%20null)%20and%20substringof('%20ls%20',tolower(Tags))))%20or%20((Id%20ne%20null)%20and%20substringof('-la',tolower(Id))))%20or%20((Description%20ne%20null)%20and%20substringof('-la',tolower(Description))))%20or%20((Tags%20ne%20null)%20and%20substringof('%20-la%20',tolower(Tags))))%20and%20IsLatestVersion&$orderby=Id&$skip=0&$top=30&semVerLevel=2.0.0", "headers": ["Host: example.com", "User-Agent: curl/8.17.0", "Accept: application/atom+xml, application/xml"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/nuget/Packages()?$filter=tolower(Id)%20eq%20'python3'&$top=30", "headers": ["Host: example.com", "User-Agent: curl/8.17.0", "Accept: application/atom+xml, application/xml"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/nuget/Packages()?$filter=tolower(Id)%20eq%20'powershell'&$top=30", "headers": ["Host: example.com", "User-Agent: curl/8.17.0", "Accept: application/atom+xml, application/xml"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/nuget/Packages()?$filter=tolower(Id)%20eq%20'.htaccess'&$top=30", "headers": ["Host: example.com", "User-Agent: curl/8.17.0", "Accept: application/atom+xml, application/xml"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/nuget/Packages()?$filter=tolower(Id)%20eq%20'docker-compose.yml'&$top=30", "headers": ["Host: example.com", "User-Agent: curl/8.17.0", "Accept: application/atom+xml, application/xml"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/nuget/Packages()?$filter=tolower(Id)%20eq%20'tools'&$top=30", "headers": ["Host: example.com", "User-Agent: curl/8.17.0", "Accept: application/atom+xml, application/xml"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/nuget/Search()/$count?$filter=IsLatestVersion&searchTerm='python3'&targetFramework=''&includePrerelease=false&$skip=0&$top=20&semVerLevel=2.0.0", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/nuget/Search()/$count?$filter=IsLatestVersion&searchTerm='powershell'&targetFramework=''&includePrerelease=false&$skip=0&$top=20&semVerLevel=2.0.0", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/nuget/Search()/$count?$filter=IsLatestVersion&searchTerm='ls%20-la'&targetFramework=''&includePrerelease=false&$skip=0&$top=20&semVerLevel=2.0.0", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/nuget/Search()/$count?$filter=IsLatestVersion&searchTerm='.htaccess'&targetFramework=''&includePrerelease=false&$skip=0&$top=20&semVerLevel=2.0.0", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/nuget/Search()/$count?$filter=IsLatestVersion&searchTerm='docker-compose.yml'&targetFramework=''&includePrerelease=false&$skip=0&$top=20&semVerLevel=2.0.0", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/nuget/Search()/$count?$filter=IsLatestVersion&searchTerm='tools'&targetFramework=''&includePrerelease=false&$skip=0&$top=20&semVerLevel=2.0.0", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/nuget/Packages()/$count?$filter=((((Id%20ne%20null)%20and%20substringof('python3',tolower(Id)))%20or%20((Description%20ne%20null)%20and%20substringof('python3',tolower(Description))))%20or%20((Tags%20ne%20null)%20and%20substringof('%20python3%20',tolower(Tags))))%20and%20IsLatestVersion&$orderby=Id&$skip=0&$top=30&semVerLevel=2.0.0", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/nuget/Packages()/$count?$filter=((((Id%20ne%20null)%20and%20substringof('powershell',tolower(Id)))%20or%20((Description%20ne%20null)%20and%20substringof('powershell',tolower(Description))))%20or%20((Tags%20ne%20null)%20and%20substringof('%20powershell%20',tolower(Tags))))%20and%20IsLatestVersion&$orderby=Id&$skip=0&$top=30&semVerLevel=2.0.0", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/nuget/Packages()/$count?$filter=(((((((Id%20ne%20null)%20and%20substringof('ls',tolower(Id)))%20or%20((Description%20ne%20null)%20and%20substringof('ls',tolower(Description))))%20or%20((Tags%20ne%20null)%20and%20substringof('%20ls%20',tolower(Tags))))%20or%20((Id%20ne%20null)%20and%20substringof('-la',tolower(Id))))%20or%20((Description%20ne%20null)%20and%20substringof('-la',tolower(Description))))%20or%20((Tags%20ne%20null)%20and%20substringof('%20-la%20',tolower(Tags))))%20and%20IsLatestVersion&$orderby=Id&$skip=0&$top=30&semVerLevel=2.0.0", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/nuget/Packages()/$count?$filter=((((Id%20ne%20null)%20and%20substringof('.htaccess',tolower(Id)))%20or%20((Description%20ne%20null)%20and%20substringof('.htaccess',tolower(Description))))%20or%20((Tags%20ne%20null)%20and%20substringof('%20.htaccess%20',tolower(Tags))))%20and%20IsLatestVersion&$orderby=Id&$skip=0&$top=30&semVerLevel=2.0.0", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/nuget/Packages()/$count?$filter=((((Id%20ne%20null)%20and%20substringof('docker-compose.yml',tolower(Id)))%20or%20((Description%20ne%20null)%20and%20substringof('docker-compose.yml',tolower(Description))))%20or%20((Tags%20ne%20null)%20and%20substringof('%20docker-compose.yml%20',tolower(Tags))))%20and%20IsLatestVersion&$orderby=Id&$skip=0&$top=30&semVerLevel=2.0.0", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/nuget/Packages()/$count?$filter=((((Id%20ne%20null)%20and%20substringof('tools',tolower(Id)))%20or%20((Description%20ne%20null)%20and%20substringof('tools',tolower(Description))))%20or%20((Tags%20ne%20null)%20and%20substringof('%20tools%20',tolower(Tags))))%20and%20IsLatestVersion&$orderby=Id&$skip=0&$top=30&semVerLevel=2.0.0", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/nuget/Packages()/$count?$filter=tolower(Id)%20eq%20'python3'&$top=30", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/nuget/Packages()/$count?$filter=tolower(Id)%20eq%20'powershell'&$top=30", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/nuget/Packages()/$count?$filter=tolower(Id)%20eq%20'.htaccess'&$top=30", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/nuget/Packages()/$count?$filter=tolower(Id)%20eq%20'docker-compose.yml'&$top=30", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*"], "body": ""}
|
||||
{"method": "GET", "target": "/api/packages/alice/nuget/Packages()/$count?$filter=tolower(Id)%20eq%20'tools'&$top=30", "headers": ["Host: example.com", "User-Agent: curl/8.18.0", "Accept: */*"], "body": ""}
|
||||
@@ -762,3 +762,33 @@
|
||||
{"method": "POST", "target": "/ghost/api/admin/posts/", "headers": ["Host: example.com", "User-Agent: curl/8.11.1", "Accept: */*", "Authorization: Ghost eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjZhZjQ1Y2VjZDI4MWIwMDAwMWE1OTk5YiJ9.eyJpYXQiOjE3OTQ0MDA2OTQsImV4cCI6MTc5NDQwMDk5NCwiYXVkIjoiL2FkbWluLyJ9.ayqaBc1Hg4fbMZj_XbvUGlhloCnW_D5KDMN8A6uM68M", "Accept-Version: v6.76", "Content-Type: application/json", "Content-Length: 465"], "body": "{\"posts\": [{\"title\": \"Older editor format: python3 notes\", \"mobiledoc\": \"{\\\"version\\\": \\\"0.3.1\\\", \\\"atoms\\\": [], \\\"markups\\\": [[\\\"code\\\"]], \\\"cards\\\": [[\\\"code\\\", {\\\"code\\\": \\\"ls -la /var/www\\\\npython3 tidy.py .htaccess\\\", \\\"language\\\": \\\"bash\\\"}]], \\\"sections\\\": [[1, \\\"h2\\\", [[0, [], 0, \\\"Mobiledoc post about powershell\\\"]]], [1, \\\"p\\\", [[0, [], 0, \\\"Run \\\"], [0, [0], 1, \\\"ls -la\\\"], [0, [], 0, \\\" and keep .htaccess tidy.\\\"]]], [10, 0]]}\", \"status\": \"draft\"}]}"}
|
||||
{"method": "POST", "target": "/ghost/api/admin/snippets/?formats=mobiledoc%2Clexical", "headers": ["Host: example.com", "Content-Length: 519", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "content-type: application/json", "app-pragma: no-cache", "Accept: */*", "Origin: http://example.com", "Referer: http://example.com/ghost/", "Accept-Encoding: gzip, deflate", "Cookie: ghost-admin-api-session=s%3A8kdYjooVavGjq0rNroahDCvgTG3XukhZ.ioil5%2Fg42yqQcdl%2B%2FlhvXc4HoOV6hGDetkdr5aPXvuo"], "body": "{\"snippets\":[{\"name\":\"python3 & .htaccess tip\",\"mobiledoc\":\"{}\",\"lexical\": \"{\\\"root\\\": {\\\"children\\\": [{\\\"children\\\": [{\\\"detail\\\": 0, \\\"format\\\": 0, \\\"mode\\\": \\\"normal\\\", \\\"style\\\": \\\"\\\", \\\"text\\\": \\\"Run ls -la, then python3 tidy.py .htaccess; on Windows use powershell.\\\", \\\"type\\\": \\\"extended-text\\\", \\\"version\\\": 1}], \\\"direction\\\": \\\"ltr\\\", \\\"format\\\": \\\"\\\", \\\"indent\\\": 0, \\\"type\\\": \\\"paragraph\\\", \\\"version\\\": 1}], \\\"direction\\\": \\\"ltr\\\", \\\"format\\\": \\\"\\\", \\\"indent\\\": 0, \\\"type\\\": \\\"root\\\", \\\"version\\\": 1}}\"}]}"}
|
||||
{"method": "POST", "target": "/ghost/api/admin/themes/upload/", "headers": ["Host: example.com", "Content-Length: 2138", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryzhPTWHSPscackPlN", "app-pragma: no-cache", "Accept: */*", "Origin: http://example.com", "Referer: http://example.com/ghost/", "Accept-Encoding: gzip, deflate", "Cookie: ghost-admin-api-session=s%3A8kdYjooVavGjq0rNroahDCvgTG3XukhZ.ioil5%2Fg42yqQcdl%2B%2FlhvXc4HoOV6hGDetkdr5aPXvuo"], "body": "------WebKitFormBoundaryzhPTWHSPscackPlN\r\nContent-Disposition: form-data; name=\"file\"; filename=\"notes-theme.zip\"\r\nContent-Type: application/zip\r\n\r\nUEsDBAoAAAAAAM9la10AAAAAAAAAAAAAAAAMABwAbm90ZXMtdGhlbWUvVVQJAAOmY/RqpmP0anV4CwABBOgDAAAE6AMAAFBLAwQUAAAACADPZWtdXfnn43sAAACmAAAAFQAcAG5vdGVzLXRoZW1lL2luZGV4Lmhic1VUCQADpmP0aqZj9Gp1eAsAAQToAwAABOgDAAAtjlEKAyEMRP/3FHZ7AKHfqVcpwcYqWF00C4WQu1dtP+fNZCYiFzBPCnhmVt3gjak4kWuojdBHc9TOXRWwcfKZjM/Y+30XmcZjKdXdQbw5QBMbhWmeLU8qwokzjXOLDuzIiNDHUzt4sV/ngPY/pzqK8ZUKcqplZtY/2xdQSwMECgAAAAAAz2VrXQAAAAAAAAAAAAAAABUAHABub3Rlcy10aGVtZS9wYXJ0aWFscy9VVAkAA6Zj9GqmY/RqdXgLAAEE6AMAAAToAwAAUEsDBBQAAAAIAM9la11tuH02lQAAAPgAAAAUABwAbm90ZXMtdGhlbWUvcGFnZS5oYnNVVAkAA6Zj9GqmY/RqdXgLAAEE6AMAAAToAwAAXU+7DsMgDNz7FZTMVdSd5lcQoiaxBEkFzmTx77XTdGi3e+l0x3x15gkp7Jl6vzAPr60JcqESxgwm5tDawzKr7g/Wu53ccp+YCSmDhMeDDZhMgkB7BY8lzOpgmU2rUQsE+r3m/4g1IZP6P7oPusdK64hJehpEwm39zpmXW9xWgpU0ckIdcuYmN54HDrsUsWW3tH3uXd5QSwMEFAAAAAgAz2VrXY4bR/y4AAAABwEAABgAHABub3Rlcy10aGVtZS9wYWNrYWdlLmpzb25VVAkAA6Zj9GqmY/RqdXgLAAEE6AMAAAToAwAALY/BbsIwDIbvPIWVM6uKEJdJm+BJKis1baTEjuIAG6jvjpPtZvv7/Vl+OcZE7hMcSyX9qCtZuwc3k/oScg3CjV5AE8YIncNVCvQ8CEP+ravwEZBnyPKgoivF2Bx3q//3D8M4jG6/A0e8BCa14cstq2ht+Pvr1AObbeHNfKVzShhi4xiDpzP9YMqRBi/Jkubywtew9Gg2k06ZypRxaQ8dRnOFZM2k4fl3sOk9lnlCVaptVMuNtm33BlBLAwQUAAAACADPZWtdbbh9NpUAAAD4AAAAFAAcAG5vdGVzLXRoZW1lL3Bvc3QuaGJzVVQJAAOmY/RqpmP0anV4CwABBOgDAAAE6AMAAF1Puw7DIAzc+xWUzFXUneZXEKImsQRJBc5k8e+103Rot3vpdMd8deYJKeyZer8wD6+tCXKhEsYMJubQ2sMyq+4P1rud3HKfmAkpg4THgw2YTIJAewWPJczqYJlNq1ELBPq95v+INSGT+j+6D7rHSuuISXoaRMJt/c6Zl1vcVoKVNHJCHXLmJjeeBw67FLFlt7R97l3eUEsDBBQAAAAIAM9la12WkoJXiwAAAMEAAAAXABwAbm90ZXMtdGhlbWUvZGVmYXVsdC5oYnNVVAkAA6Zj9GqmY/RqdXgLAAEE6AMAAAToAwAANY6xDsIwDET3fkXIDqwMboQEzDCwMEWmdZtKKZGIGVDkf6cO6mTdO92dYXO+nu6P28UEnqNrQI+J+BpbW8oxT0y7mDqMJGIdBMLewUyMpgv4zsSt/fCwPSweTxzJlaKur0IE9isdQ8rsNa+09jTwTP3XdBFz1jVVvirdKn8gImt4SIk1rNgtHfXhH1BLAQIeAwoAAAAAAM9la10AAAAAAAAAAAAAAAAMABgAAAAAAAAAEAD9QQAAAABub3Rlcy10aGVtZS9VVAUAA6Zj9Gp1eAsAAQToAwAABOgDAABQSwECHgMUAAAACADPZWtdXfnn43sAAACmAAAAFQAYAAAAAAABAAAAtIFGAAAAbm90ZXMtdGhlbWUvaW5kZXguaGJzVVQFAAOmY/RqdXgLAAEE6AMAAAToAwAAUEsBAh4DCgAAAAAAz2VrXQAAAAAAAAAAAAAAABUAGAAAAAAAAAAQAP1BEAEAAG5vdGVzLXRoZW1lL3BhcnRpYWxzL1VUBQADpmP0anV4CwABBOgDAAAE6AMAAFBLAQIeAxQAAAAIAM9la11tuH02lQAAAPgAAAAUABgAAAAAAAEAAAC0gV8BAABub3Rlcy10aGVtZS9wYWdlLmhic1VUBQADpmP0anV4CwABBOgDAAAE6AMAAFBLAQIeAxQAAAAIAM9la12OG0f8uAAAAAcBAAAYABgAAAAAAAEAAAC0gUICAABub3Rlcy10aGVtZS9wYWNrYWdlLmpzb25VVAUAA6Zj9Gp1eAsAAQToAwAABOgDAABQSwECHgMUAAAACADPZWtdbbh9NpUAAAD4AAAAFAAYAAAAAAABAAAAtIFMAwAAbm90ZXMtdGhlbWUvcG9zdC5oYnNVVAUAA6Zj9Gp1eAsAAQToAwAABOgDAABQSwECHgMUAAAACADPZWtdlpKCV4sAAADBAAAAFwAYAAAAAAABAAAAtIEvBAAAbm90ZXMtdGhlbWUvZGVmYXVsdC5oYnNVVAUAA6Zj9Gp1eAsAAQToAwAABOgDAABQSwUGAAAAAAcABwB3AgAACwUAAAAA\r\n------WebKitFormBoundaryzhPTWHSPscackPlN--\r\n"}
|
||||
# The requests below were sent to Ghost by a script with the admin's or a
|
||||
# member's headers. They hold words that name commands and files in more fields.
|
||||
{"method": "POST", "target": "/ghost/api/admin/authentication/setup/", "headers": ["Host: example.com", "Content-Length: 158", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "content-type: application/json", "app-pragma: no-cache", "Accept: */*", "Origin: http://example.com", "Referer: http://example.com/ghost/", "Accept-Encoding: gzip, deflate"], "body": "{\"setup\":[{\"blogTitle\":\"Notes on python3, powershell & .htaccess\",\"name\":\"Alice ls -la Example\",\"email\":\"alice@example.com\",\"password\":\"Hv4-mQz8-Lr2x-Tp9w\"}]}"}
|
||||
{"method": "POST", "target": "/ghost/api/admin/session/", "headers": ["Host: example.com", "Content-Length: 64", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "content-type: application/json", "app-pragma: no-cache", "Accept: */*", "Origin: http://example.com", "Referer: http://example.com/ghost/", "Accept-Encoding: gzip, deflate"], "body": "{\"username\":\"alice@example.com\",\"password\":\"Hv4-mQz8-Lr2x-Tp9w\"}"}
|
||||
{"method": "PUT", "target": "/ghost/api/admin/users/6af54436a509340001efcee2/?include=roles", "headers": ["Host: example.com", "Content-Length: 1256", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "content-type: application/json", "app-pragma: no-cache", "Accept: */*", "Origin: http://example.com", "Referer: http://example.com/ghost/", "Accept-Encoding: gzip, deflate", "Cookie: ghost-admin-api-session=s%3A0oed4vmdPUacOz5202Gz4DuzSj2EdRr8.yDcWOi%2FjeAs2xhgQ%2Brrm6eEUWSpZp3YslrhATMDAtCs"], "body": "{\"users\":[{\"id\":\"6af54436a509340001efcee2\",\"name\":\"Alice python3 & powershell\",\"slug\":\"alice-python3-powershell\",\"email\":\"alice@example.com\",\"profile_image\":null,\"cover_image\":null,\"bio\":\"I write about ls -la, python3, powershell and .htaccess files.\",\"website\":\"https://example.com/python3/powershell\",\"location\":\"Berlin & the python3 meetup\",\"facebook\":null,\"twitter\":null,\"threads\":null,\"bluesky\":null,\"mastodon\":null,\"tiktok\":null,\"youtube\":null,\"instagram\":null,\"linkedin\":null,\"accessibility\":\"{\\\"nightShift\\\":\\\"system\\\"}\",\"status\":\"active\",\"meta_title\":null,\"meta_description\":null,\"tour\":null,\"last_seen\":\"2026-11-12T04:44:42.000Z\",\"comment_notifications\":true,\"free_member_signup_notification\":true,\"paid_subscription_started_notification\":true,\"paid_subscription_canceled_notification\":false,\"mention_notifications\":true,\"recommendation_notifications\":true,\"milestone_notifications\":true,\"donation_notifications\":true,\"gift_subscription_notifications\":true,\"created_at\":\"2026-11-12T04:44:38.000Z\",\"updated_at\":\"2026-11-12T04:44:42.000Z\",\"roles\":[{\"id\":\"6af54436a509340001efcee7\",\"name\":\"Owner\",\"description\":\"Blog Owner\",\"created_at\":\"2026-11-12T04:44:38.000Z\",\"updated_at\":\"2026-11-12T04:44:38.000Z\"}],\"url\":\"http://example.com/author/alice/\"}]}"}
|
||||
{"method": "POST", "target": "/ghost/api/admin/invites/", "headers": ["Host: example.com", "Content-Length: 130", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "content-type: application/json", "app-pragma: no-cache", "Accept: */*", "Origin: http://example.com", "Referer: http://example.com/ghost/", "Accept-Encoding: gzip, deflate", "Cookie: ghost-admin-api-session=s%3A0oed4vmdPUacOz5202Gz4DuzSj2EdRr8.yDcWOi%2FjeAs2xhgQ%2Brrm6eEUWSpZp3YslrhATMDAtCs"], "body": "{\"invites\":[{\"email\":\"erin.htaccess@example.com\",\"role_id\":\"6af54436a509340001efcee6\",\"expires\":null,\"status\":null,\"token\":null}]}"}
|
||||
{"method": "POST", "target": "/ghost/api/admin/invites/", "headers": ["Host: example.com", "Content-Length: 128", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "content-type: application/json", "app-pragma: no-cache", "Accept: */*", "Origin: http://example.com", "Referer: http://example.com/ghost/", "Accept-Encoding: gzip, deflate", "Cookie: ghost-admin-api-session=s%3A0oed4vmdPUacOz5202Gz4DuzSj2EdRr8.yDcWOi%2FjeAs2xhgQ%2Brrm6eEUWSpZp3YslrhATMDAtCs"], "body": "{\"invites\":[{\"email\":\"python3.dev@example.com\",\"role_id\":\"6af54436a509340001efcee6\",\"expires\":null,\"status\":null,\"token\":null}]}"}
|
||||
{"method": "POST", "target": "/ghost/api/admin/invites/", "headers": ["Host: example.com", "Content-Length: 132", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "content-type: application/json", "app-pragma: no-cache", "Accept: */*", "Origin: http://example.com", "Referer: http://example.com/ghost/", "Accept-Encoding: gzip, deflate", "Cookie: ghost-admin-api-session=s%3A0oed4vmdPUacOz5202Gz4DuzSj2EdRr8.yDcWOi%2FjeAs2xhgQ%2Brrm6eEUWSpZp3YslrhATMDAtCs"], "body": "{\"invites\":[{\"email\":\"dave.powershell@example.com\",\"role_id\":\"6af54436a509340001efcee6\",\"expires\":null,\"status\":null,\"token\":null}]}"}
|
||||
{"method": "GET", "target": "/ghost/api/admin/authentication/invitation/?email=erin.htaccess%40example.com", "headers": ["Host: example.com", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "app-pragma: no-cache", "Accept: application/json, text/plain, */*", "Origin: http://example.com", "Referer: http://example.com/ghost/", "Accept-Encoding: gzip, deflate"], "body": ""}
|
||||
{"method": "POST", "target": "/ghost/api/admin/authentication/invitation/", "headers": ["Host: example.com", "Content-Length: 243", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "content-type: application/json", "app-pragma: no-cache", "Accept: */*", "Origin: http://example.com", "Referer: http://example.com/ghost/", "Accept-Encoding: gzip, deflate"], "body": "{\"invitation\":[{\"token\":\"MTc5NTA2MzQ4MjU5MnxlcmluLmh0YWNjZXNzQGV4YW1wbGUuY29tfDUxbkp4aWJ4SzA4NzN0eUtIQWl5YzhyWFhKTjA0aWJ1ZDExendTVHQrSU09\",\"name\":\"Erin python3 & .htaccess\",\"password\":\"Rk7-Wq2z-Np5c-Lx8v\",\"email\":\"erin.htaccess@example.com\"}]}"}
|
||||
{"method": "PUT", "target": "/ghost/api/admin/tiers/6af54437a509340001efcef1/", "headers": ["Host: example.com", "Content-Length: 474", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "content-type: application/json", "app-pragma: no-cache", "Accept: */*", "Origin: http://example.com", "Referer: http://example.com/ghost/", "Accept-Encoding: gzip, deflate", "Cookie: ghost-admin-api-session=s%3A0oed4vmdPUacOz5202Gz4DuzSj2EdRr8.yDcWOi%2FjeAs2xhgQ%2Brrm6eEUWSpZp3YslrhATMDAtCs"], "body": "{\"tiers\":[{\"id\":\"6af54437a509340001efcef1\",\"name\":\"python3 & powershell club\",\"description\":\"Weekly ls -la and .htaccess tips\",\"slug\":\"default-product\",\"active\":true,\"type\":\"paid\",\"welcome_page_url\":null,\"created_at\":\"2026-11-12T04:44:39.000Z\",\"updated_at\":\"2026-11-12T04:44:39.000Z\",\"visibility\":\"public\",\"benefits\":[\"python3 scripts\",\"powershell recipes\",\"ls -la cheat sheet\",\".htaccess examples\"],\"currency\":\"USD\",\"monthly_price\":500,\"yearly_price\":5000,\"trial_days\":0}]}"}
|
||||
{"method": "POST", "target": "/ghost/api/admin/offers/", "headers": ["Host: example.com", "Content-Length: 391", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "content-type: application/json", "app-pragma: no-cache", "Accept: */*", "Origin: http://example.com", "Referer: http://example.com/ghost/", "Accept-Encoding: gzip, deflate", "Cookie: ghost-admin-api-session=s%3A0oed4vmdPUacOz5202Gz4DuzSj2EdRr8.yDcWOi%2FjeAs2xhgQ%2Brrm6eEUWSpZp3YslrhATMDAtCs"], "body": "{\"offers\":[{\"name\":\"python3 launch\",\"code\":\"python3-launch\",\"display_title\":\"powershell & python3 deal\",\"display_description\":\"Half off for ls -la and .htaccess fans\",\"type\":\"percent\",\"cadence\":\"month\",\"amount\":50,\"duration\":\"once\",\"duration_in_months\":null,\"currency_restriction\":false,\"currency\":null,\"status\":\"active\",\"redemption_type\":\"signup\",\"tier\":{\"id\":\"6af54437a509340001efcef1\"}}]}"}
|
||||
{"method": "POST", "target": "/ghost/api/admin/recommendations/", "headers": ["Host: example.com", "Content-Length: 288", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "content-type: application/json", "app-pragma: no-cache", "Accept: */*", "Origin: http://example.com", "Referer: http://example.com/ghost/", "Accept-Encoding: gzip, deflate", "Cookie: ghost-admin-api-session=s%3A0oed4vmdPUacOz5202Gz4DuzSj2EdRr8.yDcWOi%2FjeAs2xhgQ%2Brrm6eEUWSpZp3YslrhATMDAtCs"], "body": "{\"recommendations\":[{\"title\":\"python3 & powershell weekly\",\"description\":\"Tips on ls -la and .htaccess files\",\"excerpt\":\"A newsletter about python3, powershell, ls -la and .htaccess.\",\"featured_image\":null,\"favicon\":null,\"url\":\"https://example.com/python3/\",\"one_click_subscribe\":false}]}"}
|
||||
{"method": "POST", "target": "/ghost/api/admin/integrations/?include=api_keys%2Cwebhooks", "headers": ["Host: example.com", "Content-Length: 55", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "content-type: application/json", "app-pragma: no-cache", "Accept: */*", "Origin: http://example.com", "Referer: http://example.com/ghost/", "Accept-Encoding: gzip, deflate", "Cookie: ghost-admin-api-session=s%3A0oed4vmdPUacOz5202Gz4DuzSj2EdRr8.yDcWOi%2FjeAs2xhgQ%2Brrm6eEUWSpZp3YslrhATMDAtCs"], "body": "{\"integrations\":[{\"name\":\"python3 & powershell sync\"}]}"}
|
||||
{"method": "PUT", "target": "/ghost/api/admin/integrations/6af5443aa509340001efcfc6/?include=api_keys%2Cwebhooks", "headers": ["Host: example.com", "Content-Length: 128", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "content-type: application/json", "app-pragma: no-cache", "Accept: */*", "Origin: http://example.com", "Referer: http://example.com/ghost/", "Accept-Encoding: gzip, deflate", "Cookie: ghost-admin-api-session=s%3A0oed4vmdPUacOz5202Gz4DuzSj2EdRr8.yDcWOi%2FjeAs2xhgQ%2Brrm6eEUWSpZp3YslrhATMDAtCs"], "body": "{\"integrations\":[{\"name\":\"python3 & powershell sync\",\"description\":\"Runs ls -la and tidies .htaccess files\",\"icon_image\":null}]}"}
|
||||
{"method": "POST", "target": "/ghost/api/admin/webhooks/", "headers": ["Host: example.com", "Content-Length: 202", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "content-type: application/json", "app-pragma: no-cache", "Accept: */*", "Origin: http://example.com", "Referer: http://example.com/ghost/", "Accept-Encoding: gzip, deflate", "Cookie: ghost-admin-api-session=s%3A0oed4vmdPUacOz5202Gz4DuzSj2EdRr8.yDcWOi%2FjeAs2xhgQ%2Brrm6eEUWSpZp3YslrhATMDAtCs"], "body": "{\"webhooks\":[{\"event\":\"post.published\",\"target_url\":\"https://example.com/hooks/python3\",\"name\":\"python3 & powershell hook\",\"secret\":null,\"api_version\":\"v6\",\"integration_id\":\"6af5443aa509340001efcfc6\"}]}"}
|
||||
{"method": "POST", "target": "/ghost/api/admin/tags/?include=count.posts", "headers": ["Host: example.com", "Content-Length: 702", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "content-type: application/json", "app-pragma: no-cache", "Accept: */*", "Origin: http://example.com", "Referer: http://example.com/ghost/", "Accept-Encoding: gzip, deflate", "Cookie: ghost-admin-api-session=s%3AWHEHmHzgM_62oH0uUMxh6TgpQCaauWmB.h%2FuJNd7nfwPSaFGmMarZNVguG8U1ZIIJsaLFE4%2BqXMA"], "body": "{\"tags\":[{\"name\":\"Shell\",\"slug\":\"shell\",\"description\":null,\"feature_image\":null,\"meta_title\":\"python3 & powershell tips\",\"meta_description\":\"Run ls -la and tidy .htaccess files with python3.\",\"canonical_url\":\"https://example.com/tag/powershell/\",\"twitter_image\":null,\"twitter_title\":\"powershell & python3\",\"twitter_description\":\"ls -la, .htaccess and more\",\"og_image\":null,\"og_title\":\"python3 & powershell\",\"og_description\":\"ls -la and .htaccess tips\",\"codeinjection_head\":\"<meta name=\\\"google-site-verification\\\" content=\\\"x\\\"><link rel=\\\"preconnect\\\" href=\\\"https://fonts.example.com\\\">\",\"codeinjection_foot\":\"<style>.tag-python3 { color: #333; }</style>\",\"accent_color\":null,\"visibility\":\"public\"}]}"}
|
||||
{"method": "PUT", "target": "/ghost/api/admin/posts/6af5443ba509340001efcfd1/?formats=mobiledoc%2Clexical", "headers": ["Host: example.com", "Content-Length: 315", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "content-type: application/json", "app-pragma: no-cache", "Accept: */*", "Origin: http://example.com", "Referer: http://example.com/ghost/", "Accept-Encoding: gzip, deflate", "Cookie: ghost-admin-api-session=s%3A0oed4vmdPUacOz5202Gz4DuzSj2EdRr8.yDcWOi%2FjeAs2xhgQ%2Brrm6eEUWSpZp3YslrhATMDAtCs"], "body": "{\"posts\":[{\"updated_at\":\"2026-11-12T04:44:43.000Z\",\"title\":\"Shell notes\",\"feature_image\":\"https://images.example.com/terminal.png\",\"feature_image_alt\":\"A terminal running ls -la on .htaccess with python3\",\"feature_image_caption\":\"python3 & powershell, photo by <a href=\\\"https://example.com/ls-la\\\">Erin</a>\"}]}"}
|
||||
{"method": "POST", "target": "/ghost/api/admin/members/upload/", "headers": ["Host: example.com", "Content-Length: 990", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "content-type: multipart/form-data; boundary=----WebKitFormBoundaryq3Lm8vXc2TzR7wKd", "app-pragma: no-cache", "Accept: */*", "Origin: http://example.com", "Referer: http://example.com/ghost/", "Accept-Encoding: gzip, deflate", "Cookie: ghost-admin-api-session=s%3A0oed4vmdPUacOz5202Gz4DuzSj2EdRr8.yDcWOi%2FjeAs2xhgQ%2Brrm6eEUWSpZp3YslrhATMDAtCs"], "body": "------WebKitFormBoundaryq3Lm8vXc2TzR7wKd\r\nContent-Disposition: form-data; name=\"membersfile\"; filename=\"members.csv\"\r\nContent-Type: text/csv\r\n\r\nemail,name,note,labels\r\npython3.dev@example.com,Grace python3,Asked about ls -la,powershell\r\nerin.htaccess@example.com,Erin,Tidies .htaccess files,python3\r\n\r\n------WebKitFormBoundaryq3Lm8vXc2TzR7wKd\r\nContent-Disposition: form-data; name=\"labels\"\r\n\r\npython3 import\r\n------WebKitFormBoundaryq3Lm8vXc2TzR7wKd\r\nContent-Disposition: form-data; name=\"labels\"\r\n\r\npowershell & .htaccess\r\n------WebKitFormBoundaryq3Lm8vXc2TzR7wKd\r\nContent-Disposition: form-data; name=\"mapping[email]\"\r\n\r\nemail\r\n------WebKitFormBoundaryq3Lm8vXc2TzR7wKd\r\nContent-Disposition: form-data; name=\"mapping[name]\"\r\n\r\nname\r\n------WebKitFormBoundaryq3Lm8vXc2TzR7wKd\r\nContent-Disposition: form-data; name=\"mapping[note]\"\r\n\r\nnote\r\n------WebKitFormBoundaryq3Lm8vXc2TzR7wKd\r\nContent-Disposition: form-data; name=\"mapping[labels]\"\r\n\r\nlabels\r\n------WebKitFormBoundaryq3Lm8vXc2TzR7wKd--\r\n"}
|
||||
{"method": "GET", "target": "/ghost/api/admin/stats/top-content/?date_from=2026-10-13&date_to=2026-11-12&timezone=Etc%2FUTC&member_status=all&pathname=%2Fpython3-and-powershell%2F", "headers": ["Host: example.com", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "app-pragma: no-cache", "Accept: application/json, text/plain, */*", "Origin: http://example.com", "Referer: http://example.com/ghost/", "Accept-Encoding: gzip, deflate", "Cookie: ghost-admin-api-session=s%3A0oed4vmdPUacOz5202Gz4DuzSj2EdRr8.yDcWOi%2FjeAs2xhgQ%2Brrm6eEUWSpZp3YslrhATMDAtCs"], "body": ""}
|
||||
{"method": "GET", "target": "/ghost/api/admin/stats/top-content/?date_from=2026-10-13&date_to=2026-11-12&timezone=Etc%2FUTC&member_status=all&source=powershell.example.com", "headers": ["Host: example.com", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "app-pragma: no-cache", "Accept: application/json, text/plain, */*", "Origin: http://example.com", "Referer: http://example.com/ghost/", "Accept-Encoding: gzip, deflate", "Cookie: ghost-admin-api-session=s%3A0oed4vmdPUacOz5202Gz4DuzSj2EdRr8.yDcWOi%2FjeAs2xhgQ%2Brrm6eEUWSpZp3YslrhATMDAtCs"], "body": ""}
|
||||
{"method": "GET", "target": "/ghost/api/admin/stats/top-content/?date_from=2026-10-13&date_to=2026-11-12&timezone=Etc%2FUTC&member_status=all&utm_campaign=powershell+%26+python3", "headers": ["Host: example.com", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "app-pragma: no-cache", "Accept: application/json, text/plain, */*", "Origin: http://example.com", "Referer: http://example.com/ghost/", "Accept-Encoding: gzip, deflate", "Cookie: ghost-admin-api-session=s%3A0oed4vmdPUacOz5202Gz4DuzSj2EdRr8.yDcWOi%2FjeAs2xhgQ%2Brrm6eEUWSpZp3YslrhATMDAtCs"], "body": ""}
|
||||
{"method": "PUT", "target": "/ghost/api/admin/posts/6af5443ba509340001efcfd1/?formats=mobiledoc%2Clexical&newsletter=default-newsletter&email_segment=label%3Apowershell-users", "headers": ["Host: example.com", "Content-Length: 74", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "content-type: application/json", "app-pragma: no-cache", "Accept: */*", "Origin: http://example.com", "Referer: http://example.com/ghost/", "Accept-Encoding: gzip, deflate", "Cookie: ghost-admin-api-session=s%3A0oed4vmdPUacOz5202Gz4DuzSj2EdRr8.yDcWOi%2FjeAs2xhgQ%2Brrm6eEUWSpZp3YslrhATMDAtCs"], "body": "{\"posts\":[{\"updated_at\":\"2026-11-12T04:44:43.000Z\",\"status\":\"published\"}]}"}
|
||||
{"method": "GET", "target": "/ghost/api/admin/posts/?filter=status%3Apublished%2Btitle%3A%7E%27powershell+%26+python3%27&limit=25&fields=id%2Ctitle&order=published_at+DESC", "headers": ["Host: example.com", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "app-pragma: no-cache", "Accept: application/json, text/plain, */*", "Origin: http://example.com", "Referer: http://example.com/ghost/", "Accept-Encoding: gzip, deflate", "Cookie: ghost-admin-api-session=s%3A0oed4vmdPUacOz5202Gz4DuzSj2EdRr8.yDcWOi%2FjeAs2xhgQ%2Brrm6eEUWSpZp3YslrhATMDAtCs"], "body": ""}
|
||||
{"method": "GET", "target": "/ghost/api/admin/posts/?filter=status%3Apublished%2Btitle%3A%7E%27ls+-la+%26+.htaccess%27&limit=25&fields=id%2Ctitle&order=published_at+DESC", "headers": ["Host: example.com", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "app-pragma: no-cache", "Accept: application/json, text/plain, */*", "Origin: http://example.com", "Referer: http://example.com/ghost/", "Accept-Encoding: gzip, deflate", "Cookie: ghost-admin-api-session=s%3A0oed4vmdPUacOz5202Gz4DuzSj2EdRr8.yDcWOi%2FjeAs2xhgQ%2Brrm6eEUWSpZp3YslrhATMDAtCs"], "body": ""}
|
||||
{"method": "POST", "target": "/members/api/comments/", "headers": ["Host: example.com", "Content-Length: 161", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Content-Type: application/json", "Accept: */*", "Origin: http://example.com", "Referer: http://example.com/", "Accept-Encoding: gzip, deflate", "Cookie: ghost-members-ssr=5174dad7-2125-4e37-bb05-2aac35f959e9; ghost-members-ssr.sig=kPwob0aJ4RTyxR4cixOd33LpDNg"], "body": "{\"comments\":[{\"post_id\":\"6af5443ba509340001efcfd1\",\"status\":\"published\",\"html\":\"<p>Thanks! I ran <code>ls -la</code> and fixed my .htaccess with python3.</p>\"}]}"}
|
||||
{"method": "PUT", "target": "/members/api/comments/6af5443ba509340001efd00b/", "headers": ["Host: example.com", "Content-Length: 140", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Content-Type: application/json", "Accept: */*", "Origin: http://example.com", "Referer: http://example.com/", "Accept-Encoding: gzip, deflate", "Cookie: ghost-members-ssr=5174dad7-2125-4e37-bb05-2aac35f959e9; ghost-members-ssr.sig=kPwob0aJ4RTyxR4cixOd33LpDNg"], "body": "{\"comments\":[{\"id\":\"6af5443ba509340001efd00b\",\"html\":\"<p>Thanks! I ran ls -la, then powershell, and fixed my .htaccess with python3.</p>\"}]}"}
|
||||
{"method": "POST", "target": "/members/api/comments/", "headers": ["Host: example.com", "Content-Length: 220", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Content-Type: application/json", "Accept: */*", "Origin: http://example.com", "Referer: http://example.com/", "Accept-Encoding: gzip, deflate", "Cookie: ghost-members-ssr=5174dad7-2125-4e37-bb05-2aac35f959e9; ghost-members-ssr.sig=kPwob0aJ4RTyxR4cixOd33LpDNg"], "body": "{\"comments\":[{\"post_id\":\"6af5443ba509340001efcfd1\",\"parent_id\":\"6af5443ba509340001efd00b\",\"in_reply_to_id\":\"6af5443ba509340001efd00b\",\"status\":\"published\",\"html\":\"<p>Same here: python3 & powershell both work.</p>\"}]}"}
|
||||
{"method": "GET", "target": "/members/api/comments/post/6af5443ba509340001efcfd1/?limit=20&filter=created_at%3A%3C%3D2026-11-12T04%3A44%3A43.000Z&page=2&order=created_at+desc", "headers": ["Host: example.com", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: */*", "Origin: http://example.com", "Referer: http://example.com/", "Accept-Encoding: gzip, deflate", "Cookie: ghost-members-ssr=5174dad7-2125-4e37-bb05-2aac35f959e9; ghost-members-ssr.sig=kPwob0aJ4RTyxR4cixOd33LpDNg"], "body": ""}
|
||||
{"method": "GET", "target": "/members/api/comments/6af5443ba509340001efd00b/replies/?limit=5&filter=id%3A%3E%276af5443ba509340001efd010%27", "headers": ["Host: example.com", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: */*", "Origin: http://example.com", "Referer: http://example.com/", "Accept-Encoding: gzip, deflate", "Cookie: ghost-members-ssr=5174dad7-2125-4e37-bb05-2aac35f959e9; ghost-members-ssr.sig=kPwob0aJ4RTyxR4cixOd33LpDNg"], "body": ""}
|
||||
+161
@@ -381,6 +381,167 @@
|
||||
/org/acme/pulls/developers?type=your_repositories&sort=recentupdate&state=open&q=SELECT%20%2A%20FROM%20users%20WHERE&search_mode=exact
|
||||
/alice/tools/milestone/1?q=SELECT%20%2A%20FROM%20users%20WHERE
|
||||
|
||||
# The API's searches that gitea and Forgejo share, sent with curl to gitea
|
||||
# 1.27.3. Each searches for words that name commands and files, such as python3
|
||||
# and .htaccess, and for plain words. A package name can hold a slash, as npm's
|
||||
# @alice/tools-cli does, so the search of packages is also sent /bin/sh. The
|
||||
# search of unadopted repositories takes an owner and a name.
|
||||
/api/v1/repos/search?q=python3&limit=10
|
||||
/api/v1/repos/search?q=powershell&limit=10
|
||||
/api/v1/repos/search?q=ls%20-la&limit=10
|
||||
/api/v1/repos/search?q=.htaccess&limit=10
|
||||
/api/v1/repos/search?q=docker-compose.yml&limit=10
|
||||
/api/v1/repos/search?q=tools&limit=10
|
||||
/api/v1/users/search?q=python3
|
||||
/api/v1/users/search?q=powershell
|
||||
/api/v1/users/search?q=ls%20-la
|
||||
/api/v1/users/search?q=.htaccess
|
||||
/api/v1/users/search?q=bob
|
||||
/api/v1/repos/issues/search?q=python3&type=issues&state=open
|
||||
/api/v1/repos/issues/search?q=powershell&type=issues&state=open
|
||||
/api/v1/repos/issues/search?q=ls%20-la&type=issues&state=open
|
||||
/api/v1/repos/issues/search?q=.htaccess&type=issues&state=open
|
||||
/api/v1/repos/issues/search?q=docker-compose.yml&type=issues&state=open
|
||||
/api/v1/repos/issues/search?q=build&type=issues&state=open
|
||||
/api/v1/repos/alice/tools/issues?state=open&q=python3
|
||||
/api/v1/repos/alice/tools/issues?state=open&q=powershell
|
||||
/api/v1/repos/alice/tools/issues?state=open&q=ls%20-la
|
||||
/api/v1/repos/alice/tools/issues?state=open&q=.htaccess
|
||||
/api/v1/repos/alice/tools/issues?state=open&q=docker-compose.yml
|
||||
/api/v1/packages/alice?q=python3
|
||||
/api/v1/packages/alice?q=powershell
|
||||
/api/v1/packages/alice?q=ls%20-la
|
||||
/api/v1/packages/alice?q=.htaccess
|
||||
/api/v1/packages/alice?q=%2Fbin%2Fsh
|
||||
/api/v1/packages/alice?q=install-script
|
||||
/api/v1/packages/alice?q=%40alice%2Ftools-cli
|
||||
/api/v1/topics/search?q=python3
|
||||
/api/v1/topics/search?q=powershell
|
||||
/api/v1/topics/search?q=ls%20-la
|
||||
/api/v1/topics/search?q=.htaccess
|
||||
/api/v1/topics/search?q=docker
|
||||
/api/v1/orgs/acme/teams/search?q=python3
|
||||
/api/v1/orgs/acme/teams/search?q=powershell
|
||||
/api/v1/orgs/acme/teams/search?q=ls%20-la
|
||||
/api/v1/orgs/acme/teams/search?q=.htaccess
|
||||
/api/v1/orgs/acme/teams/search?q=developers
|
||||
/api/v1/admin/emails/search?q=python3
|
||||
/api/v1/admin/emails/search?q=powershell
|
||||
/api/v1/admin/emails/search?q=ls%20-la
|
||||
/api/v1/admin/emails/search?q=.htaccess
|
||||
/api/v1/admin/emails/search?q=alice
|
||||
/api/v1/admin/unadopted?query=python3
|
||||
/api/v1/admin/unadopted?query=powershell
|
||||
/api/v1/admin/unadopted?query=ls%20-la
|
||||
/api/v1/admin/unadopted?query=.htaccess
|
||||
/api/v1/admin/unadopted?query=alice%2Fpython3-app
|
||||
/api/v1/admin/unadopted?query=alice%2Flegacy-site
|
||||
/api/v1/repos/alice/tools/milestones?state=open&name=python3
|
||||
/api/v1/repos/alice/tools/milestones?state=open&name=powershell
|
||||
/api/v1/repos/alice/tools/milestones?state=open&name=ls%20-la
|
||||
/api/v1/repos/alice/tools/milestones?state=open&name=.htaccess
|
||||
/api/v1/repos/alice/tools/milestones?state=open&name=v1.0
|
||||
|
||||
# The package registries' searches, sent to gitea 1.27.3 by each registry's own
|
||||
# client: cargo, knife, composer, conan 1 and 2, dotnet and npm. curl sent the
|
||||
# requests no client sends: the Chef cookbook list with a search, and NuGet's
|
||||
# Packages() with the search in $filter, as NuGet's client builds it for a feed
|
||||
# without Search(). curl also sent each NuGet Search() and Packages() search to
|
||||
# Search()/$count and Packages()/$count, which count its results. Each searches
|
||||
# for words that name commands and files, such as python3 and .htaccess, and
|
||||
# for plain words. A Composer, Conan or npm name can hold a slash, so those
|
||||
# searches are also sent /bin/sh.
|
||||
/api/packages/alice/cargo/api/v1/crates?q=python3&per_page=10
|
||||
/api/packages/alice/cargo/api/v1/crates?q=powershell&per_page=10
|
||||
/api/packages/alice/cargo/api/v1/crates?q=ls%20%2Dla&per_page=10
|
||||
/api/packages/alice/cargo/api/v1/crates?q=%2Ehtaccess&per_page=10
|
||||
/api/packages/alice/cargo/api/v1/crates?q=docker%2Dcompose%2Eyml&per_page=10
|
||||
/api/packages/alice/cargo/api/v1/crates?q=tools&per_page=10
|
||||
/api/packages/alice/chef/api/v1/search?q=python3&items=9999999&start=0
|
||||
/api/packages/alice/chef/api/v1/search?q=powershell&items=9999999&start=0
|
||||
/api/packages/alice/chef/api/v1/search?q=ls%20-la&items=9999999&start=0
|
||||
/api/packages/alice/chef/api/v1/search?q=.htaccess&items=9999999&start=0
|
||||
/api/packages/alice/chef/api/v1/search?q=docker-compose.yml&items=9999999&start=0
|
||||
/api/packages/alice/chef/api/v1/search?q=tools&items=9999999&start=0
|
||||
/api/packages/alice/chef/api/v1/cookbooks?q=python3&items=9999999&start=0
|
||||
/api/packages/alice/chef/api/v1/cookbooks?q=powershell&items=9999999&start=0
|
||||
/api/packages/alice/chef/api/v1/cookbooks?q=ls%20-la&items=9999999&start=0
|
||||
/api/packages/alice/chef/api/v1/cookbooks?q=.htaccess&items=9999999&start=0
|
||||
/api/packages/alice/chef/api/v1/cookbooks?q=docker-compose.yml&items=9999999&start=0
|
||||
/api/packages/alice/chef/api/v1/cookbooks?q=tools&items=9999999&start=0
|
||||
/api/packages/alice/composer/search.json?q=python3&type=
|
||||
/api/packages/alice/composer/search.json?q=powershell&type=
|
||||
/api/packages/alice/composer/search.json?q=ls+-la&type=
|
||||
/api/packages/alice/composer/search.json?q=.htaccess&type=
|
||||
/api/packages/alice/composer/search.json?q=%2Fbin%2Fsh&type=
|
||||
/api/packages/alice/composer/search.json?q=alice%2Ftools-cli&type=
|
||||
/api/packages/alice/composer/search.json?q=docker-compose.yml&type=
|
||||
/api/packages/alice/composer/search.json?q=tools&type=
|
||||
/api/packages/alice/conan/v1/conans/search?q=python3
|
||||
/api/packages/alice/conan/v1/conans/search?q=powershell
|
||||
/api/packages/alice/conan/v1/conans/search?q=ls+-la
|
||||
/api/packages/alice/conan/v1/conans/search?q=.htaccess
|
||||
/api/packages/alice/conan/v1/conans/search?q=%2Fbin%2Fsh
|
||||
/api/packages/alice/conan/v1/conans/search?q=tools%2F%2A
|
||||
/api/packages/alice/conan/v1/conans/search?q=docker-compose.yml
|
||||
/api/packages/alice/conan/v1/conans/search?q=tools
|
||||
/api/packages/alice/conan/v2/conans/search?q=python3
|
||||
/api/packages/alice/conan/v2/conans/search?q=powershell
|
||||
/api/packages/alice/conan/v2/conans/search?q=ls+-la
|
||||
/api/packages/alice/conan/v2/conans/search?q=.htaccess
|
||||
/api/packages/alice/conan/v2/conans/search?q=%2Fbin%2Fsh
|
||||
/api/packages/alice/conan/v2/conans/search?q=tools%2F1.0%40alice%2Fstable
|
||||
/api/packages/alice/conan/v2/conans/search?q=docker-compose.yml
|
||||
/api/packages/alice/conan/v2/conans/search?q=tools
|
||||
/api/packages/alice/nuget/query?q=python3&skip=0&take=20&prerelease=false&semVerLevel=2.0.0
|
||||
/api/packages/alice/nuget/query?q=powershell&skip=0&take=20&prerelease=false&semVerLevel=2.0.0
|
||||
/api/packages/alice/nuget/query?q=ls%20-la&skip=0&take=20&prerelease=false&semVerLevel=2.0.0
|
||||
/api/packages/alice/nuget/query?q=.htaccess&skip=0&take=20&prerelease=false&semVerLevel=2.0.0
|
||||
/api/packages/alice/nuget/query?q=docker-compose.yml&skip=0&take=20&prerelease=false&semVerLevel=2.0.0
|
||||
/api/packages/alice/nuget/query?q=tools&skip=0&take=20&prerelease=false&semVerLevel=2.0.0
|
||||
/api/packages/alice/nuget/Search()?$filter=IsLatestVersion&searchTerm='python3'&targetFramework=''&includePrerelease=false&$skip=0&$top=20&semVerLevel=2.0.0
|
||||
/api/packages/alice/nuget/Search()?$filter=IsLatestVersion&searchTerm='powershell'&targetFramework=''&includePrerelease=false&$skip=0&$top=20&semVerLevel=2.0.0
|
||||
/api/packages/alice/nuget/Search()?$filter=IsLatestVersion&searchTerm='ls%20-la'&targetFramework=''&includePrerelease=false&$skip=0&$top=20&semVerLevel=2.0.0
|
||||
/api/packages/alice/nuget/Search()?$filter=IsLatestVersion&searchTerm='.htaccess'&targetFramework=''&includePrerelease=false&$skip=0&$top=20&semVerLevel=2.0.0
|
||||
/api/packages/alice/nuget/Search()?$filter=IsLatestVersion&searchTerm='docker-compose.yml'&targetFramework=''&includePrerelease=false&$skip=0&$top=20&semVerLevel=2.0.0
|
||||
/api/packages/alice/nuget/Search()?$filter=IsLatestVersion&searchTerm='tools'&targetFramework=''&includePrerelease=false&$skip=0&$top=20&semVerLevel=2.0.0
|
||||
/api/packages/alice/nuget/Packages()?$filter=((((Id%20ne%20null)%20and%20substringof('python3',tolower(Id)))%20or%20((Description%20ne%20null)%20and%20substringof('python3',tolower(Description))))%20or%20((Tags%20ne%20null)%20and%20substringof('%20python3%20',tolower(Tags))))%20and%20IsLatestVersion&$orderby=Id&$skip=0&$top=30&semVerLevel=2.0.0
|
||||
/api/packages/alice/nuget/Packages()?$filter=((((Id%20ne%20null)%20and%20substringof('powershell',tolower(Id)))%20or%20((Description%20ne%20null)%20and%20substringof('powershell',tolower(Description))))%20or%20((Tags%20ne%20null)%20and%20substringof('%20powershell%20',tolower(Tags))))%20and%20IsLatestVersion&$orderby=Id&$skip=0&$top=30&semVerLevel=2.0.0
|
||||
/api/packages/alice/nuget/Packages()?$filter=(((((((Id%20ne%20null)%20and%20substringof('ls',tolower(Id)))%20or%20((Description%20ne%20null)%20and%20substringof('ls',tolower(Description))))%20or%20((Tags%20ne%20null)%20and%20substringof('%20ls%20',tolower(Tags))))%20or%20((Id%20ne%20null)%20and%20substringof('-la',tolower(Id))))%20or%20((Description%20ne%20null)%20and%20substringof('-la',tolower(Description))))%20or%20((Tags%20ne%20null)%20and%20substringof('%20-la%20',tolower(Tags))))%20and%20IsLatestVersion&$orderby=Id&$skip=0&$top=30&semVerLevel=2.0.0
|
||||
/api/packages/alice/nuget/Packages()?$filter=((((Id%20ne%20null)%20and%20substringof('.htaccess',tolower(Id)))%20or%20((Description%20ne%20null)%20and%20substringof('.htaccess',tolower(Description))))%20or%20((Tags%20ne%20null)%20and%20substringof('%20.htaccess%20',tolower(Tags))))%20and%20IsLatestVersion&$orderby=Id&$skip=0&$top=30&semVerLevel=2.0.0
|
||||
/api/packages/alice/nuget/Packages()?$filter=((((Id%20ne%20null)%20and%20substringof('docker-compose.yml',tolower(Id)))%20or%20((Description%20ne%20null)%20and%20substringof('docker-compose.yml',tolower(Description))))%20or%20((Tags%20ne%20null)%20and%20substringof('%20docker-compose.yml%20',tolower(Tags))))%20and%20IsLatestVersion&$orderby=Id&$skip=0&$top=30&semVerLevel=2.0.0
|
||||
/api/packages/alice/nuget/Packages()?$filter=((((Id%20ne%20null)%20and%20substringof('tools',tolower(Id)))%20or%20((Description%20ne%20null)%20and%20substringof('tools',tolower(Description))))%20or%20((Tags%20ne%20null)%20and%20substringof('%20tools%20',tolower(Tags))))%20and%20IsLatestVersion&$orderby=Id&$skip=0&$top=30&semVerLevel=2.0.0
|
||||
/api/packages/alice/nuget/Packages()?$filter=tolower(Id)%20eq%20'python3'&$top=30
|
||||
/api/packages/alice/nuget/Packages()?$filter=tolower(Id)%20eq%20'powershell'&$top=30
|
||||
/api/packages/alice/nuget/Packages()?$filter=tolower(Id)%20eq%20'.htaccess'&$top=30
|
||||
/api/packages/alice/nuget/Packages()?$filter=tolower(Id)%20eq%20'docker-compose.yml'&$top=30
|
||||
/api/packages/alice/nuget/Packages()?$filter=tolower(Id)%20eq%20'tools'&$top=30
|
||||
/api/packages/alice/nuget/Search()/$count?$filter=IsLatestVersion&searchTerm='python3'&targetFramework=''&includePrerelease=false&$skip=0&$top=20&semVerLevel=2.0.0
|
||||
/api/packages/alice/nuget/Search()/$count?$filter=IsLatestVersion&searchTerm='powershell'&targetFramework=''&includePrerelease=false&$skip=0&$top=20&semVerLevel=2.0.0
|
||||
/api/packages/alice/nuget/Search()/$count?$filter=IsLatestVersion&searchTerm='ls%20-la'&targetFramework=''&includePrerelease=false&$skip=0&$top=20&semVerLevel=2.0.0
|
||||
/api/packages/alice/nuget/Search()/$count?$filter=IsLatestVersion&searchTerm='.htaccess'&targetFramework=''&includePrerelease=false&$skip=0&$top=20&semVerLevel=2.0.0
|
||||
/api/packages/alice/nuget/Search()/$count?$filter=IsLatestVersion&searchTerm='docker-compose.yml'&targetFramework=''&includePrerelease=false&$skip=0&$top=20&semVerLevel=2.0.0
|
||||
/api/packages/alice/nuget/Search()/$count?$filter=IsLatestVersion&searchTerm='tools'&targetFramework=''&includePrerelease=false&$skip=0&$top=20&semVerLevel=2.0.0
|
||||
/api/packages/alice/nuget/Packages()/$count?$filter=((((Id%20ne%20null)%20and%20substringof('python3',tolower(Id)))%20or%20((Description%20ne%20null)%20and%20substringof('python3',tolower(Description))))%20or%20((Tags%20ne%20null)%20and%20substringof('%20python3%20',tolower(Tags))))%20and%20IsLatestVersion&$orderby=Id&$skip=0&$top=30&semVerLevel=2.0.0
|
||||
/api/packages/alice/nuget/Packages()/$count?$filter=((((Id%20ne%20null)%20and%20substringof('powershell',tolower(Id)))%20or%20((Description%20ne%20null)%20and%20substringof('powershell',tolower(Description))))%20or%20((Tags%20ne%20null)%20and%20substringof('%20powershell%20',tolower(Tags))))%20and%20IsLatestVersion&$orderby=Id&$skip=0&$top=30&semVerLevel=2.0.0
|
||||
/api/packages/alice/nuget/Packages()/$count?$filter=(((((((Id%20ne%20null)%20and%20substringof('ls',tolower(Id)))%20or%20((Description%20ne%20null)%20and%20substringof('ls',tolower(Description))))%20or%20((Tags%20ne%20null)%20and%20substringof('%20ls%20',tolower(Tags))))%20or%20((Id%20ne%20null)%20and%20substringof('-la',tolower(Id))))%20or%20((Description%20ne%20null)%20and%20substringof('-la',tolower(Description))))%20or%20((Tags%20ne%20null)%20and%20substringof('%20-la%20',tolower(Tags))))%20and%20IsLatestVersion&$orderby=Id&$skip=0&$top=30&semVerLevel=2.0.0
|
||||
/api/packages/alice/nuget/Packages()/$count?$filter=((((Id%20ne%20null)%20and%20substringof('.htaccess',tolower(Id)))%20or%20((Description%20ne%20null)%20and%20substringof('.htaccess',tolower(Description))))%20or%20((Tags%20ne%20null)%20and%20substringof('%20.htaccess%20',tolower(Tags))))%20and%20IsLatestVersion&$orderby=Id&$skip=0&$top=30&semVerLevel=2.0.0
|
||||
/api/packages/alice/nuget/Packages()/$count?$filter=((((Id%20ne%20null)%20and%20substringof('docker-compose.yml',tolower(Id)))%20or%20((Description%20ne%20null)%20and%20substringof('docker-compose.yml',tolower(Description))))%20or%20((Tags%20ne%20null)%20and%20substringof('%20docker-compose.yml%20',tolower(Tags))))%20and%20IsLatestVersion&$orderby=Id&$skip=0&$top=30&semVerLevel=2.0.0
|
||||
/api/packages/alice/nuget/Packages()/$count?$filter=((((Id%20ne%20null)%20and%20substringof('tools',tolower(Id)))%20or%20((Description%20ne%20null)%20and%20substringof('tools',tolower(Description))))%20or%20((Tags%20ne%20null)%20and%20substringof('%20tools%20',tolower(Tags))))%20and%20IsLatestVersion&$orderby=Id&$skip=0&$top=30&semVerLevel=2.0.0
|
||||
/api/packages/alice/nuget/Packages()/$count?$filter=tolower(Id)%20eq%20'python3'&$top=30
|
||||
/api/packages/alice/nuget/Packages()/$count?$filter=tolower(Id)%20eq%20'powershell'&$top=30
|
||||
/api/packages/alice/nuget/Packages()/$count?$filter=tolower(Id)%20eq%20'.htaccess'&$top=30
|
||||
/api/packages/alice/nuget/Packages()/$count?$filter=tolower(Id)%20eq%20'docker-compose.yml'&$top=30
|
||||
/api/packages/alice/nuget/Packages()/$count?$filter=tolower(Id)%20eq%20'tools'&$top=30
|
||||
/api/packages/alice/npm/-/v1/search?text=python3&size=20&from=0&quality=0.65&popularity=0.98&maintenance=0.5
|
||||
/api/packages/alice/npm/-/v1/search?text=powershell&size=20&from=0&quality=0.65&popularity=0.98&maintenance=0.5
|
||||
/api/packages/alice/npm/-/v1/search?text=ls+-la&size=20&from=0&quality=0.65&popularity=0.98&maintenance=0.5
|
||||
/api/packages/alice/npm/-/v1/search?text=.htaccess&size=20&from=0&quality=0.65&popularity=0.98&maintenance=0.5
|
||||
/api/packages/alice/npm/-/v1/search?text=%2Fbin%2Fsh&size=20&from=0&quality=0.65&popularity=0.98&maintenance=0.5
|
||||
/api/packages/alice/npm/-/v1/search?text=%40alice%2Ftools-cli&size=20&from=0&quality=0.65&popularity=0.98&maintenance=0.5
|
||||
/api/packages/alice/npm/-/v1/search?text=docker-compose.yml&size=20&from=0&quality=0.65&popularity=0.98&maintenance=0.5
|
||||
/api/packages/alice/npm/-/v1/search?text=tools&size=20&from=0&quality=0.65&popularity=0.98&maintenance=0.5
|
||||
|
||||
# Pages, raw files and edit pages of files, and the API's call for their
|
||||
# contents.
|
||||
/owner/repo/src/branch/main/package.json
|
||||
|
||||
+11
-12
@@ -185,7 +185,7 @@ func (m *Metrics) AddBansAndClients(
|
||||
ledger *bans.Ledger, limiter *ratelimit.Limiter, now func() time.Time,
|
||||
) {
|
||||
for _, cause := range []string{
|
||||
bans.CauseLimit, bans.CauseAttack, bans.CauseAdmin, bans.CauseCrowdSec,
|
||||
bans.CauseLimit, bans.CauseAttack, bans.CauseAdmin,
|
||||
} {
|
||||
m.registry.MustRegister(
|
||||
prometheus.NewCounterFunc(prometheus.CounterOpts{
|
||||
@@ -302,20 +302,20 @@ const sourceLabel = "source"
|
||||
|
||||
// AddReputation adds the metrics of the lists fetched from URLs and of the
|
||||
// DNSBL zones, by source, each list's URL or each zone, its key masked as
|
||||
// config.MaskZoneKey masks it: the requests whose client a blocklist, the
|
||||
// CrowdSec decision list, a zone's verdict or AbuseIPDB's score lists,
|
||||
// which ReputationHit counts, and, read from lists and dnsbl as the
|
||||
// metrics are asked for, for a list, the fetches that failed and when the
|
||||
// copy in use was fetched, and for a zone, the queries made and those that
|
||||
// failed. It is called once, before ReputationHit.
|
||||
// config.MaskZoneKey masks it: the requests whose client a blocklist, a
|
||||
// zone's verdict or AbuseIPDB's score lists, which ReputationHit counts,
|
||||
// and, read from lists and dnsbl as the metrics are asked for, for a list,
|
||||
// the fetches that failed and when the copy in use was fetched, and for a
|
||||
// zone, the queries made and those that failed. It is called once, before
|
||||
// ReputationHit.
|
||||
func (m *Metrics) AddReputation(
|
||||
lists *reputation.Lists,
|
||||
dnsbl *reputation.DNSBL,
|
||||
) {
|
||||
m.reputationHits = counterVec(
|
||||
"sww_reputation_hits_total",
|
||||
"Requests whose client a blocklist, the CrowdSec decision list, a DNSBL "+
|
||||
"zone or AbuseIPDB lists, by the list's URL, the zone, or abuseipdb.",
|
||||
"Requests whose client a blocklist, a DNSBL zone or AbuseIPDB lists, "+
|
||||
"by the list's URL, the zone, or abuseipdb.",
|
||||
[]string{sourceLabel},
|
||||
)
|
||||
m.registry.MustRegister(m.reputationHits)
|
||||
@@ -378,9 +378,8 @@ func (m *Metrics) AddAbuseIPDB(abuseIPDB *reputation.AbuseIPDB) {
|
||||
)
|
||||
}
|
||||
|
||||
// ReputationHit counts a request whose client source lists: a blocklist
|
||||
// or the CrowdSec decision list, by its URL, a DNSBL zone, its key masked,
|
||||
// or AbuseIPDB, abuseipdb.
|
||||
// ReputationHit counts a request whose client source lists: a blocklist,
|
||||
// by its URL, a DNSBL zone, its key masked, or AbuseIPDB, abuseipdb.
|
||||
func (m *Metrics) ReputationHit(source string) {
|
||||
m.reputationHits.WithLabelValues(source).Inc()
|
||||
}
|
||||
|
||||
@@ -1,622 +0,0 @@
|
||||
package reputation_test
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"log/slog"
|
||||
"net/http"
|
||||
"net/netip"
|
||||
"reflect"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
"testing/synctest"
|
||||
"time"
|
||||
|
||||
"sneak.berlin/go/smallwebwaf/internal/alerts"
|
||||
"sneak.berlin/go/smallwebwaf/internal/reputation"
|
||||
)
|
||||
|
||||
// The tests run in a synctest bubble, as those of the blocklists do, and
|
||||
// fetch the decision list from engine, a stand-in for a CrowdSec engine
|
||||
// that answers without the network.
|
||||
|
||||
const (
|
||||
// decisionsURL is the decision list of the tests' engine, and engineKey
|
||||
// the key it answers.
|
||||
decisionsURL = "http://crowdsec.example:8080/v1/decisions"
|
||||
engineKey = "crowdsec-key-0123456789abcdef"
|
||||
// sshBF and probing are scenarios of the engine's decisions.
|
||||
sshBF = "crowdsecurity/ssh-bf"
|
||||
probing = "crowdsecurity/http-probing"
|
||||
// ban is the type of a decision to ban, and rangeScope the scope of a
|
||||
// decision on a netblock, as CrowdSec names them.
|
||||
ban = "ban"
|
||||
rangeScope = "Range"
|
||||
)
|
||||
|
||||
func TestCrowdSecDecisionBansItsNetblockUntilItEndsEvenWithTheEngineDown(
|
||||
t *testing.T,
|
||||
) {
|
||||
t.Parallel()
|
||||
|
||||
synctest.Test(t, func(t *testing.T) {
|
||||
began := time.Now()
|
||||
manual := "manual 'ban' from 'localhost'"
|
||||
e := &engine{key: engineKey, decisions: []decision{
|
||||
{"Ip", suspect, ban, manual, began.Add(6 * time.Hour)},
|
||||
// A shorter decision on the same address, which is not the one
|
||||
// used.
|
||||
{"Ip", suspect, ban, sshBF, began.Add(4 * time.Hour)},
|
||||
{
|
||||
rangeScope,
|
||||
"198.51.100.0/24",
|
||||
ban,
|
||||
probing,
|
||||
began.Add(time.Hour),
|
||||
},
|
||||
{"Ip", "2001:db8::1", ban, sshBF, began.Add(2 * time.Hour)},
|
||||
// Left out: a decision to show a captcha, and one on a country.
|
||||
{"Ip", "192.0.2.50", "captcha", probing, began.Add(time.Hour)},
|
||||
{"Country", "KP", ban, manual, began.Add(time.Hour)},
|
||||
}}
|
||||
lists := start(t, e, crowdSecParams())
|
||||
|
||||
for addr, want := range map[string]reputation.Decision{
|
||||
suspect: {Expires: began.Add(6 * time.Hour), Scenario: manual},
|
||||
"198.51.100.0": {Expires: began.Add(time.Hour), Scenario: probing},
|
||||
"198.51.100.255": {Expires: began.Add(time.Hour), Scenario: probing},
|
||||
"2001:db8::1": {Expires: began.Add(2 * time.Hour), Scenario: sshBF},
|
||||
"203.0.113.10": {},
|
||||
"198.51.101.0": {},
|
||||
"2001:db8::2": {},
|
||||
"192.0.2.50": {},
|
||||
} {
|
||||
wantDecision(t, lists, addr, want)
|
||||
}
|
||||
|
||||
// With the engine down, the copy kept still holds the decision on
|
||||
// 198.51.100.0/24, which no longer bans once it has ended.
|
||||
e.set(func(e *engine) { e.failing = true })
|
||||
time.Sleep(time.Hour - time.Nanosecond)
|
||||
synctest.Wait()
|
||||
wantDecision(
|
||||
t,
|
||||
lists,
|
||||
"198.51.100.7",
|
||||
reputation.Decision{
|
||||
Expires: began.Add(time.Hour),
|
||||
Scenario: probing,
|
||||
},
|
||||
)
|
||||
|
||||
time.Sleep(time.Nanosecond)
|
||||
synctest.Wait()
|
||||
wantDecision(t, lists, "198.51.100.7", reputation.Decision{})
|
||||
wantDecision(
|
||||
t,
|
||||
lists,
|
||||
suspect,
|
||||
reputation.Decision{
|
||||
Expires: began.Add(6 * time.Hour),
|
||||
Scenario: manual,
|
||||
},
|
||||
)
|
||||
})
|
||||
}
|
||||
|
||||
func TestCrowdSecDecisionListFetchedAgainEveryMinute(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
synctest.Test(t, func(t *testing.T) {
|
||||
began := time.Now()
|
||||
e := &engine{key: engineKey, decisions: []decision{
|
||||
{"Ip", suspect, ban, sshBF, began.Add(4 * time.Hour)},
|
||||
}}
|
||||
lists := start(t, e, crowdSecParams())
|
||||
wantEngineFetches(t, e, 1)
|
||||
|
||||
added := reputation.Decision{
|
||||
Expires: began.Add(2 * time.Hour),
|
||||
Scenario: probing,
|
||||
}
|
||||
|
||||
e.set(func(e *engine) {
|
||||
e.decisions = append(e.decisions,
|
||||
decision{"Ip", "203.0.113.10", ban, probing, added.Expires})
|
||||
})
|
||||
|
||||
time.Sleep(time.Minute - time.Nanosecond)
|
||||
wantEngineFetches(t, e, 1)
|
||||
wantDecision(t, lists, "203.0.113.10", reputation.Decision{})
|
||||
|
||||
time.Sleep(time.Nanosecond)
|
||||
wantEngineFetches(t, e, 2)
|
||||
wantDecision(t, lists, "203.0.113.10", added)
|
||||
})
|
||||
}
|
||||
|
||||
func TestCrowdSecDecisionOnAClientIsTheOneThatEndsLast(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
synctest.Test(t, func(t *testing.T) {
|
||||
began := time.Now()
|
||||
e := &engine{key: engineKey, decisions: []decision{
|
||||
// Two decisions on one address, the shorter listed first.
|
||||
{"Ip", suspect, ban, sshBF, began.Add(2 * time.Hour)},
|
||||
{"Ip", suspect, ban, probing, began.Add(4 * time.Hour)},
|
||||
// 198.51.100.130 is held by a decision on its address that ends
|
||||
// after the one on its netblock, and 192.0.2.20 by one that ends
|
||||
// before.
|
||||
{
|
||||
rangeScope,
|
||||
"198.51.100.128/25",
|
||||
ban,
|
||||
sshBF,
|
||||
began.Add(time.Hour),
|
||||
},
|
||||
{"Ip", "198.51.100.130", ban, probing, began.Add(3 * time.Hour)},
|
||||
{
|
||||
rangeScope,
|
||||
"192.0.2.0/24",
|
||||
ban,
|
||||
probing,
|
||||
began.Add(5 * time.Hour),
|
||||
},
|
||||
{"Ip", "192.0.2.20", ban, sshBF, began.Add(2 * time.Hour)},
|
||||
}}
|
||||
lists := start(t, e, crowdSecParams())
|
||||
|
||||
wantDecision(
|
||||
t,
|
||||
lists,
|
||||
suspect,
|
||||
reputation.Decision{
|
||||
Expires: began.Add(4 * time.Hour),
|
||||
Scenario: probing,
|
||||
},
|
||||
)
|
||||
wantDecision(
|
||||
t,
|
||||
lists,
|
||||
"198.51.100.130",
|
||||
reputation.Decision{
|
||||
Expires: began.Add(3 * time.Hour),
|
||||
Scenario: probing,
|
||||
},
|
||||
)
|
||||
wantDecision(
|
||||
t,
|
||||
lists,
|
||||
"192.0.2.20",
|
||||
reputation.Decision{
|
||||
Expires: began.Add(5 * time.Hour),
|
||||
Scenario: probing,
|
||||
},
|
||||
)
|
||||
})
|
||||
}
|
||||
|
||||
func TestCrowdSecAnswerOfNoDecisionIsAGoodCopyThatListsNoClient(
|
||||
t *testing.T,
|
||||
) {
|
||||
t.Parallel()
|
||||
|
||||
synctest.Test(t, func(t *testing.T) {
|
||||
began := time.Now()
|
||||
e := &engine{key: engineKey, decisions: []decision{
|
||||
{"Ip", suspect, ban, sshBF, began.Add(4 * time.Hour)},
|
||||
}}
|
||||
lists := start(t, e, crowdSecParams())
|
||||
|
||||
// With its decision deleted, the engine answers null.
|
||||
e.set(func(e *engine) { e.decisions = nil })
|
||||
time.Sleep(time.Minute)
|
||||
wantEngineFetches(t, e, 2)
|
||||
|
||||
want := []reputation.List{{
|
||||
URL: decisionsURL, Tried: time.Now(), Fetched: time.Now(), Lines: []string{"null"},
|
||||
}}
|
||||
if got := lists.Snapshot(); !reflect.DeepEqual(got, want) {
|
||||
t.Errorf("lists %+v, want %+v", got, want)
|
||||
}
|
||||
|
||||
if lists.Failures(decisionsURL) != 0 {
|
||||
t.Errorf("%d failures, want 0", lists.Failures(decisionsURL))
|
||||
}
|
||||
|
||||
wantDecision(t, lists, suspect, reputation.Decision{})
|
||||
})
|
||||
}
|
||||
|
||||
func TestCrowdSecFailureKeepsTheLastGoodCopyAlertsOncePerCooldownAndHidesTheKey(
|
||||
t *testing.T,
|
||||
) {
|
||||
t.Parallel()
|
||||
|
||||
for _, tc := range crowdSecFailures() {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
synctest.Test(t, func(t *testing.T) {
|
||||
var log bytes.Buffer
|
||||
|
||||
began := time.Now()
|
||||
e := &engine{key: engineKey, decisions: []decision{
|
||||
{"Ip", suspect, ban, sshBF, began.Add(4 * time.Hour)},
|
||||
}}
|
||||
queue := newQueue()
|
||||
p := crowdSecParams()
|
||||
p.ProcessLog = slog.New(slog.NewJSONHandler(&log, nil))
|
||||
p.Alerts = queue
|
||||
lists := start(t, e, p)
|
||||
kept := lists.Snapshot()
|
||||
|
||||
e.set(tc.fail)
|
||||
|
||||
// Each failure is tried again a minute after it.
|
||||
for range 2 {
|
||||
time.Sleep(time.Minute)
|
||||
synctest.Wait()
|
||||
}
|
||||
|
||||
wantEngineFetches(t, e, 3)
|
||||
wantDecision(
|
||||
t,
|
||||
lists,
|
||||
suspect,
|
||||
reputation.Decision{
|
||||
Expires: began.Add(4 * time.Hour),
|
||||
Scenario: sshBF,
|
||||
},
|
||||
)
|
||||
|
||||
want := kept[0]
|
||||
want.Tried = time.Now()
|
||||
|
||||
got := lists.Snapshot()
|
||||
if !reflect.DeepEqual(got, []reputation.List{want}) {
|
||||
t.Errorf(
|
||||
"lists %+v, want the first copy, last tried now, %+v",
|
||||
got,
|
||||
want,
|
||||
)
|
||||
}
|
||||
|
||||
if n := lists.Failures(decisionsURL); n != 2 {
|
||||
t.Errorf("%d failures, want 2", n)
|
||||
}
|
||||
|
||||
// One alert for the first failure; the cooldown holds back the
|
||||
// second.
|
||||
failed := time.Now().Add(-time.Minute)
|
||||
alert := fetchFailure(failed, decisionsURL, tc.error)
|
||||
wantAlert(t, queue, alert)
|
||||
|
||||
if !strings.Contains(
|
||||
log.String(),
|
||||
`"msg":"fetching a list failed",`+
|
||||
`"url":"`+decisionsURL+`","error":"`+tc.error,
|
||||
) {
|
||||
t.Errorf("logged\n%s\nwant the failures", log.String())
|
||||
}
|
||||
|
||||
wantKeyNotShown(t, e, log.String(), lists, queue)
|
||||
})
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// crowdSecFailure is a way for the engine to fail: fail has it answer the
|
||||
// fetches after the first so that they fail with error.
|
||||
type crowdSecFailure struct {
|
||||
name string
|
||||
fail func(e *engine)
|
||||
error string
|
||||
}
|
||||
|
||||
// crowdSecFailures returns the ways the engine can fail.
|
||||
func crowdSecFailures() []crowdSecFailure {
|
||||
const notDecision = " does not give an address or a netblock and a duration, " +
|
||||
"such as 4h0m0s"
|
||||
|
||||
return []crowdSecFailure{
|
||||
{
|
||||
"an answer other than 200",
|
||||
func(e *engine) { e.failing = true },
|
||||
"the server answered 503 Service Unavailable",
|
||||
},
|
||||
{
|
||||
"a key the engine refuses",
|
||||
func(e *engine) { e.key = "another-key-0123456789abcdef" },
|
||||
"the server answered 403 Forbidden",
|
||||
},
|
||||
{
|
||||
"a redirect",
|
||||
func(e *engine) { e.redirect = "http://elsewhere.example/v1/decisions" },
|
||||
"the server answered 302 Found",
|
||||
},
|
||||
{
|
||||
"an answer that does not read",
|
||||
func(e *engine) { e.answer = "<html>" },
|
||||
"read the answer: invalid character '<' looking for beginning of value",
|
||||
},
|
||||
{
|
||||
"a decision to ban whose value does not read",
|
||||
func(e *engine) {
|
||||
e.answer = `[{"duration": "4h", "scenario": "` + sshBF + `", ` +
|
||||
`"scope": "Ip", "type": "ban", "value": "203.0.113.300"}]`
|
||||
},
|
||||
"decision 1" + notDecision,
|
||||
},
|
||||
{
|
||||
"a decision to ban whose duration does not read",
|
||||
func(e *engine) {
|
||||
e.answer = `[{"duration": "4h", "scope": "Country", "type": "ban", ` +
|
||||
`"value": "KP"}, {"duration": "four hours", "scope": "Range", ` +
|
||||
`"type": "ban", "value": "198.51.100.0/24"}]`
|
||||
},
|
||||
"decision 2" + notDecision,
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
// wantKeyNotShown checks that no fetch carried the engine's key to a URL
|
||||
// other than its decision list, such as the one a redirect names, and that
|
||||
// the key is in none of what the fetches leave behind: log, the process
|
||||
// log, the alerts waiting in queue, and the copies of lists, which
|
||||
// lists.json keeps.
|
||||
func wantKeyNotShown(
|
||||
t *testing.T,
|
||||
e *engine,
|
||||
log string,
|
||||
lists *reputation.Lists,
|
||||
queue *alerts.Queue,
|
||||
) {
|
||||
t.Helper()
|
||||
|
||||
e.mu.Lock()
|
||||
keySentTo := e.keySentTo
|
||||
e.mu.Unlock()
|
||||
|
||||
if len(keySentTo) != 0 {
|
||||
t.Errorf("the key was sent to %v", keySentTo)
|
||||
}
|
||||
|
||||
shown, err := json.Marshal([]any{lists.Snapshot(), waiting(queue)})
|
||||
if err != nil {
|
||||
t.Fatalf("encode: %v", err)
|
||||
}
|
||||
|
||||
if strings.Contains(log+string(shown), engineKey) {
|
||||
t.Errorf("the key is shown in\n%s\n%s", log, shown)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCrowdSecDecisionListKeptAcrossARestartEndsWhenItsDecisionsDo(
|
||||
t *testing.T,
|
||||
) {
|
||||
t.Parallel()
|
||||
|
||||
synctest.Test(t, func(t *testing.T) {
|
||||
began := time.Now()
|
||||
e := &engine{key: engineKey, decisions: []decision{
|
||||
{
|
||||
rangeScope,
|
||||
"198.51.100.0/24",
|
||||
ban,
|
||||
probing,
|
||||
began.Add(time.Hour),
|
||||
},
|
||||
}}
|
||||
lists := start(t, e, crowdSecParams())
|
||||
kept := lists.Snapshot()
|
||||
|
||||
// Restarted half an hour later with what lists.json keeps, and
|
||||
// the engine down, the decision still bans, until the end it had at
|
||||
// the fetch, half an hour on.
|
||||
time.Sleep(30 * time.Minute)
|
||||
|
||||
down := &engine{key: engineKey, failing: true}
|
||||
again := reputation.New(crowdSecParams())
|
||||
again.SetTransport(down)
|
||||
|
||||
err := again.Load(kept)
|
||||
if err != nil {
|
||||
t.Fatalf("load: %v", err)
|
||||
}
|
||||
|
||||
run(t, again)
|
||||
|
||||
want := reputation.Decision{
|
||||
Expires: began.Add(time.Hour),
|
||||
Scenario: probing,
|
||||
}
|
||||
wantDecision(t, again, "198.51.100.7", want)
|
||||
|
||||
time.Sleep(30*time.Minute - time.Nanosecond)
|
||||
synctest.Wait()
|
||||
wantDecision(t, again, "198.51.100.7", want)
|
||||
|
||||
time.Sleep(time.Nanosecond)
|
||||
synctest.Wait()
|
||||
wantDecision(t, again, "198.51.100.7", reputation.Decision{})
|
||||
})
|
||||
}
|
||||
|
||||
func TestLoadTakesACrowdSecListNeverFetchedAndRefusesACopyThatDoesNotRead(
|
||||
t *testing.T,
|
||||
) {
|
||||
t.Parallel()
|
||||
|
||||
now := time.Date(2026, 10, 6, 0, 0, 0, 0, time.UTC)
|
||||
lists := reputation.New(crowdSecParams())
|
||||
|
||||
// Tried, and never fetched: there is no copy to read.
|
||||
err := lists.Load([]reputation.List{{URL: decisionsURL, Tried: now}})
|
||||
if err != nil {
|
||||
t.Errorf("load the list never fetched: %v", err)
|
||||
}
|
||||
|
||||
err = lists.Load([]reputation.List{{
|
||||
URL: decisionsURL, Tried: now, Fetched: now, Lines: []string{
|
||||
`[{"duration": "4h", "scope": "Range", "type": "ban", ` +
|
||||
`"value": "198.51.100.0/33"}]`,
|
||||
},
|
||||
}})
|
||||
|
||||
const want = "the copy of " + decisionsURL + ": decision 1 does not give an " +
|
||||
"address or a netblock and a duration, such as 4h0m0s"
|
||||
if err == nil || err.Error() != want {
|
||||
t.Errorf("error %v, want %s", err, want)
|
||||
}
|
||||
}
|
||||
|
||||
// engine is a stand-in for the local API of a CrowdSec engine. It answers
|
||||
// a fetch of the decision list that carries its key in X-Api-Key with its
|
||||
// decisions still in force, each with the time it has left as it answers,
|
||||
// by the bubble's clock, as an engine does, or with answer while that is
|
||||
// not "". It answers 403 to a fetch without its key, as an engine does,
|
||||
// with a redirect to redirect while that is not "", and 503 while failing.
|
||||
// It counts the fetches, and notes in keySentTo the URL of each fetch of
|
||||
// another URL that carries a key, as one following a redirect would.
|
||||
type engine struct {
|
||||
mu sync.Mutex
|
||||
key string
|
||||
decisions []decision
|
||||
answer string
|
||||
redirect string
|
||||
failing bool
|
||||
fetches int
|
||||
keySentTo []string
|
||||
}
|
||||
|
||||
// decision is a decision of the engine, which ends at expires.
|
||||
type decision struct {
|
||||
scope, value, kind, scenario string
|
||||
expires time.Time
|
||||
}
|
||||
|
||||
// RoundTrip has the engine answer req, in place of the network.
|
||||
func (e *engine) RoundTrip(req *http.Request) (*http.Response, error) {
|
||||
e.mu.Lock()
|
||||
defer e.mu.Unlock()
|
||||
|
||||
e.fetches++
|
||||
|
||||
if req.URL.String() != decisionsURL &&
|
||||
req.Header.Get("X-Api-Key") != "" {
|
||||
e.keySentTo = append(e.keySentTo, req.URL.String())
|
||||
}
|
||||
|
||||
status, header, body := http.StatusOK, http.Header{}, e.answer
|
||||
|
||||
switch {
|
||||
case req.URL.String() != decisionsURL || req.Header.Get("X-Api-Key") != e.key:
|
||||
status, body = http.StatusForbidden, `{"message":"access forbidden"}`
|
||||
case e.redirect != "":
|
||||
status, header = http.StatusFound, http.Header{
|
||||
"Location": {e.redirect},
|
||||
}
|
||||
case e.failing:
|
||||
status, body = http.StatusServiceUnavailable, ""
|
||||
case body == "":
|
||||
body = e.inForce(time.Now())
|
||||
}
|
||||
|
||||
return &http.Response{
|
||||
StatusCode: status,
|
||||
Status: fmt.Sprintf("%d %s", status, http.StatusText(status)),
|
||||
Header: header,
|
||||
Body: io.NopCloser(strings.NewReader(body)),
|
||||
Request: req,
|
||||
}, nil
|
||||
}
|
||||
|
||||
// inForce returns the decisions in force at now, as the engine answers
|
||||
// them: a JSON list, null for none.
|
||||
func (e *engine) inForce(now time.Time) string {
|
||||
var answer []map[string]string
|
||||
|
||||
for _, d := range e.decisions {
|
||||
if now.Before(d.expires) {
|
||||
answer = append(answer, map[string]string{
|
||||
"duration": d.expires.Sub(now).
|
||||
String(),
|
||||
"origin": "crowdsec",
|
||||
"scenario": d.scenario, "scope": d.scope, "type": d.kind, "value": d.value,
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
body, err := json.Marshal(answer)
|
||||
if err != nil {
|
||||
panic(err) // a list of maps of strings always encodes
|
||||
}
|
||||
|
||||
return string(body)
|
||||
}
|
||||
|
||||
// set changes the engine with change.
|
||||
func (e *engine) set(change func(e *engine)) {
|
||||
e.mu.Lock()
|
||||
defer e.mu.Unlock()
|
||||
|
||||
change(e)
|
||||
}
|
||||
|
||||
// crowdSecParams returns the Params of the decision list of the tests'
|
||||
// engine, fetched with its key, by the bubble's clock, with alerts to a
|
||||
// queue that sends none.
|
||||
func crowdSecParams() reputation.Params {
|
||||
p := params()
|
||||
p.CrowdSecDecisionsURL = decisionsURL
|
||||
p.CrowdSecKey = engineKey
|
||||
|
||||
return p
|
||||
}
|
||||
|
||||
// wantEngineFetches waits until Run has made the fetches due, and checks
|
||||
// how many the engine has had.
|
||||
func wantEngineFetches(t *testing.T, e *engine, want int) {
|
||||
t.Helper()
|
||||
|
||||
synctest.Wait()
|
||||
|
||||
e.mu.Lock()
|
||||
got := e.fetches
|
||||
e.mu.Unlock()
|
||||
|
||||
if got != want {
|
||||
t.Errorf("%d fetches, want %d", got, want)
|
||||
}
|
||||
}
|
||||
|
||||
// wantDecision checks the decision lists says is in force on addr now,
|
||||
// the zero Decision for none.
|
||||
func wantDecision(
|
||||
t *testing.T,
|
||||
lists *reputation.Lists,
|
||||
addr string,
|
||||
want reputation.Decision,
|
||||
) {
|
||||
t.Helper()
|
||||
|
||||
got, listed := lists.CrowdSecDecision(
|
||||
netip.MustParseAddr(addr),
|
||||
time.Now(),
|
||||
)
|
||||
if listed != !want.Expires.IsZero() ||
|
||||
listed &&
|
||||
(!got.Expires.Equal(want.Expires) || got.Scenario != want.Scenario) {
|
||||
t.Errorf(
|
||||
"%s has the decision %+v in force %t, want %+v",
|
||||
addr,
|
||||
got,
|
||||
listed,
|
||||
want,
|
||||
)
|
||||
}
|
||||
}
|
||||
@@ -11,7 +11,6 @@ import (
|
||||
// network.
|
||||
func (l *Lists) SetTransport(transport http.RoundTripper) {
|
||||
l.httpClient.Transport = transport
|
||||
l.crowdSecClient.Transport = transport
|
||||
}
|
||||
|
||||
// SetTransport has a's checks go through transport instead of the
|
||||
|
||||
@@ -1,8 +1,7 @@
|
||||
// Package reputation fetches the lists the settings name by URL: the
|
||||
// blocklists of SWW_WAF_BLOCKLIST_URLS, the file of AS:percent lines
|
||||
// SWW_WAF_ASN_LIMIT_PERCENT_URL names, the address lists of SWW_WAF_TAG_LISTS, and
|
||||
// the decision list of the CrowdSec engine SWW_WAF_CROWDSEC_LAPI_URL names. It
|
||||
// keeps the last good copy of
|
||||
// SWW_WAF_ASN_LIMIT_PERCENT_URL names, and the address lists of
|
||||
// SWW_WAF_TAG_LISTS. It keeps the last good copy of
|
||||
// each, whole, comment lines included, which is used while a fetch fails,
|
||||
// and when each was last tried. It also asks the DNSBL zones of
|
||||
// SWW_WAF_DNSBL_ZONES about clients, and keeps their verdicts, and checks
|
||||
@@ -13,7 +12,6 @@ package reputation
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
@@ -35,10 +33,6 @@ const (
|
||||
maxListBytes = 16 << 20
|
||||
// fetchTimeout bounds one fetch of a list.
|
||||
fetchTimeout = time.Minute
|
||||
// crowdSecRefresh is how long after the CrowdSec decision list was last
|
||||
// fetched or tried it is fetched again: the engine is the operator's
|
||||
// own, and makes and ends decisions all the time.
|
||||
crowdSecRefresh = time.Minute
|
||||
// mappedBits is the length of ::ffff:0.0.0.0/96, the netblock of every
|
||||
// IPv4-mapped address.
|
||||
mappedBits = 96
|
||||
@@ -52,9 +46,6 @@ var (
|
||||
)
|
||||
errNotASNPercent = errors.New(
|
||||
"is not an AS number, : and a percentage, such as AS64496:50")
|
||||
errNotDecision = errors.New(
|
||||
"does not give an address or a netblock and a duration, such as 4h0m0s",
|
||||
)
|
||||
)
|
||||
|
||||
// List is a list as lists.json holds it: the URL it is fetched from,
|
||||
@@ -77,14 +68,8 @@ type Params struct {
|
||||
ASNLimitPercentURL string
|
||||
// TagLists are the address lists that add a tag (SWW_WAF_TAG_LISTS).
|
||||
TagLists []config.TagList
|
||||
// CrowdSecDecisionsURL is the CrowdSec decision list, "" while
|
||||
// SWW_WAF_CROWDSEC_LAPI_URL is unset, fetched with CrowdSecKey
|
||||
// (SWW_WAF_CROWDSEC_LAPI_KEY).
|
||||
CrowdSecDecisionsURL string
|
||||
CrowdSecKey string
|
||||
// Refresh is how long after a list was last fetched or tried it is
|
||||
// fetched again (SWW_WAF_BLOCKLIST_REFRESH), but for the CrowdSec decision
|
||||
// list, which is fetched again crowdSecRefresh after.
|
||||
// fetched again (SWW_WAF_BLOCKLIST_REFRESH).
|
||||
Refresh time.Duration
|
||||
// Now tells the time, normally time.Now in UTC.
|
||||
Now func() time.Time
|
||||
@@ -99,10 +84,6 @@ type Params struct {
|
||||
type Lists struct {
|
||||
params Params
|
||||
httpClient *http.Client
|
||||
// crowdSecClient fetches the CrowdSec decision list. It follows no
|
||||
// redirect, so that the key goes to the engine alone: a redirect is a
|
||||
// failure.
|
||||
crowdSecClient *http.Client
|
||||
|
||||
mu sync.Mutex
|
||||
// lists are by URL, one for each URL Params names.
|
||||
@@ -119,22 +100,12 @@ type list struct {
|
||||
}
|
||||
|
||||
// entries are what the lines of a copy say: for a blocklist or a tag list, the
|
||||
// netblocks it names, with the lengths among them, for the file of AS:percent
|
||||
// lines, the percentage it gives each AS number, and for the CrowdSec decision
|
||||
// list, the decision on each netblock that ends last, with the lengths among
|
||||
// them.
|
||||
// netblocks it names, with the lengths among them, and for the file of
|
||||
// AS:percent lines, the percentage it gives each AS number.
|
||||
type entries struct {
|
||||
netblocks map[netip.Prefix]bool
|
||||
lengths []int
|
||||
percents map[string]int64
|
||||
decisions map[netip.Prefix]Decision
|
||||
}
|
||||
|
||||
// Decision is a decision of the CrowdSec engine to ban a netblock: when
|
||||
// it ends, and the scenario that made it, such as crowdsecurity/ssh-bf.
|
||||
type Decision struct {
|
||||
Expires time.Time
|
||||
Scenario string
|
||||
}
|
||||
|
||||
// New returns the lists, without a copy of any yet.
|
||||
@@ -142,12 +113,7 @@ func New(params Params) *Lists {
|
||||
l := &Lists{
|
||||
params: params,
|
||||
httpClient: &http.Client{},
|
||||
crowdSecClient: &http.Client{
|
||||
CheckRedirect: func(*http.Request, []*http.Request) error {
|
||||
return http.ErrUseLastResponse
|
||||
},
|
||||
},
|
||||
lists: map[string]*list{},
|
||||
lists: map[string]*list{},
|
||||
}
|
||||
|
||||
for _, listURL := range l.URLs() {
|
||||
@@ -159,7 +125,7 @@ func New(params Params) *Lists {
|
||||
|
||||
// URLs returns the URL of every list: the blocklists' in the order
|
||||
// SWW_WAF_BLOCKLIST_URLS names them, then SWW_WAF_ASN_LIMIT_PERCENT_URL, then the
|
||||
// tag lists', then the CrowdSec decision list's.
|
||||
// tag lists'.
|
||||
func (l *Lists) URLs() []string {
|
||||
urls := slices.Clone(l.params.BlocklistURLs)
|
||||
if l.params.ASNLimitPercentURL != "" {
|
||||
@@ -170,10 +136,6 @@ func (l *Lists) URLs() []string {
|
||||
urls = append(urls, list.URL)
|
||||
}
|
||||
|
||||
if l.params.CrowdSecDecisionsURL != "" {
|
||||
urls = append(urls, l.params.CrowdSecDecisionsURL)
|
||||
}
|
||||
|
||||
return urls
|
||||
}
|
||||
|
||||
@@ -225,40 +187,6 @@ func (l *Lists) ASNLimitPercent(asn string) (int64, bool) {
|
||||
return percent, listed
|
||||
}
|
||||
|
||||
// CrowdSecDecision returns the decision of the copy of the CrowdSec
|
||||
// decision list on a netblock that holds addr and that ends last, and
|
||||
// whether it is still in force at now. A decision that has ended no
|
||||
// longer bans, even before the next fetch drops it.
|
||||
func (l *Lists) CrowdSecDecision(
|
||||
addr netip.Addr,
|
||||
now time.Time,
|
||||
) (Decision, bool) {
|
||||
if l.params.CrowdSecDecisionsURL == "" {
|
||||
return Decision{}, false
|
||||
}
|
||||
|
||||
l.mu.Lock()
|
||||
defer l.mu.Unlock()
|
||||
|
||||
kept := l.lists[l.params.CrowdSecDecisionsURL].entries
|
||||
|
||||
var last Decision
|
||||
|
||||
for _, length := range kept.lengths {
|
||||
netblock, err := addr.Prefix(length)
|
||||
if err != nil {
|
||||
continue // an IPv6 netblock's length, past an IPv4 address's 32 bits
|
||||
}
|
||||
|
||||
decision := kept.decisions[netblock]
|
||||
if decision.Expires.After(last.Expires) {
|
||||
last = decision
|
||||
}
|
||||
}
|
||||
|
||||
return last, now.Before(last.Expires)
|
||||
}
|
||||
|
||||
// Fetched returns when the copy in use of the list at listURL was
|
||||
// fetched, or zero while there is none.
|
||||
func (l *Lists) Fetched(listURL string) time.Time {
|
||||
@@ -331,7 +259,7 @@ func (l *Lists) Load(lists []List) error {
|
||||
continue // dropped, or a list tried but never fetched, without a copy
|
||||
}
|
||||
|
||||
read, err := l.parse(kept.URL, kept.Lines, kept.Fetched)
|
||||
read, err := l.parse(kept.URL, kept.Lines)
|
||||
if err != nil {
|
||||
return fmt.Errorf("the copy of %s: %w", kept.URL, err)
|
||||
}
|
||||
@@ -377,8 +305,7 @@ func (l *Lists) fetchDue(ctx context.Context) time.Time {
|
||||
}
|
||||
|
||||
// due returns when the list at listURL is to be fetched: Refresh after it
|
||||
// was last fetched or tried, the later of the two, or crowdSecRefresh
|
||||
// after for the CrowdSec decision list.
|
||||
// was last fetched or tried, the later of the two.
|
||||
func (l *Lists) due(listURL string) time.Time {
|
||||
l.mu.Lock()
|
||||
defer l.mu.Unlock()
|
||||
@@ -390,10 +317,6 @@ func (l *Lists) due(listURL string) time.Time {
|
||||
last = held.kept.Tried
|
||||
}
|
||||
|
||||
if listURL == l.params.CrowdSecDecisionsURL {
|
||||
return last.Add(crowdSecRefresh)
|
||||
}
|
||||
|
||||
return last.Add(l.params.Refresh)
|
||||
}
|
||||
|
||||
@@ -408,7 +331,7 @@ func (l *Lists) fetch(ctx context.Context, listURL string) {
|
||||
|
||||
var found entries
|
||||
if err == nil {
|
||||
found, err = l.parse(listURL, lines, now)
|
||||
found, err = l.parse(listURL, lines)
|
||||
}
|
||||
|
||||
cutOff := err != nil && ctx.Err() != nil
|
||||
@@ -468,11 +391,8 @@ func raiseFailure(queue *alerts.Queue, reason, source string, err error) {
|
||||
})
|
||||
}
|
||||
|
||||
// get fetches the list at listURL, and returns its lines. The CrowdSec
|
||||
// decision list is fetched with CrowdSecKey in the header X-Api-Key, where
|
||||
// the engine looks for it, by crowdSecClient, which follows no redirect.
|
||||
// An answer other than 200, or a list longer than maxListBytes, is a
|
||||
// failure.
|
||||
// get fetches the list at listURL, and returns its lines. An answer other
|
||||
// than 200, or a list longer than maxListBytes, is a failure.
|
||||
func (l *Lists) get(ctx context.Context, listURL string) ([]string, error) {
|
||||
ctx, cancel := context.WithTimeout(ctx, fetchTimeout)
|
||||
defer cancel()
|
||||
@@ -487,14 +407,7 @@ func (l *Lists) get(ctx context.Context, listURL string) ([]string, error) {
|
||||
return nil, fmt.Errorf("make the request: %w", err)
|
||||
}
|
||||
|
||||
client := l.httpClient
|
||||
|
||||
if listURL == l.params.CrowdSecDecisionsURL {
|
||||
req.Header.Set("X-Api-Key", l.params.CrowdSecKey)
|
||||
client = l.crowdSecClient
|
||||
}
|
||||
|
||||
res, err := client.Do(req)
|
||||
res, err := l.httpClient.Do(req)
|
||||
if err != nil {
|
||||
// Do's error names the URL, which the log line and the alert name
|
||||
// already: only what went wrong is kept.
|
||||
@@ -526,22 +439,16 @@ func (l *Lists) get(ctx context.Context, listURL string) ([]string, error) {
|
||||
return lines, nil
|
||||
}
|
||||
|
||||
// parse reads the lines of the list at listURL, fetched at fetched: those
|
||||
// of a blocklist or a tag list, of the file of AS:percent lines, or of the
|
||||
// CrowdSec decision list. In all but the decision list, anything after a ; or
|
||||
// a # on a line is left out, and so is a line left blank. Any other line that
|
||||
// parse reads the lines of the list at listURL: those of a blocklist or a
|
||||
// tag list, or of the file of AS:percent lines. Anything after a ; or a #
|
||||
// on a line is left out, and so is a line left blank. Any other line that
|
||||
// does not read is an error naming it by its number.
|
||||
func (l *Lists) parse(
|
||||
listURL string, lines []string, fetched time.Time,
|
||||
) (entries, error) {
|
||||
switch listURL {
|
||||
case l.params.ASNLimitPercentURL:
|
||||
func (l *Lists) parse(listURL string, lines []string) (entries, error) {
|
||||
if listURL == l.params.ASNLimitPercentURL {
|
||||
return parsePercents(lines)
|
||||
case l.params.CrowdSecDecisionsURL:
|
||||
return parseDecisions(lines, fetched)
|
||||
default:
|
||||
return parseNetblocks(lines)
|
||||
}
|
||||
|
||||
return parseNetblocks(lines)
|
||||
}
|
||||
|
||||
// parseNetblocks reads the lines of a blocklist or a tag list, each an address
|
||||
@@ -628,64 +535,6 @@ func parsePercents(lines []string) (entries, error) {
|
||||
return found, nil
|
||||
}
|
||||
|
||||
// parseDecisions reads the lines of the CrowdSec decision list fetched at
|
||||
// fetched: the engine's answer, a JSON list of its decisions in force,
|
||||
// null while it has none. A decision of the type ban whose scope is Ip or
|
||||
// Range, as CrowdSec names them, bans its value, an address or a netblock
|
||||
// as parseNetblock reads it, until its duration, the time it had left as
|
||||
// the engine answered, has passed since fetched. Any other decision, such
|
||||
// as one to show a captcha or one on a country, is left out. A decision
|
||||
// to ban whose value or duration does not read is an error naming it by
|
||||
// its number.
|
||||
func parseDecisions(lines []string, fetched time.Time) (entries, error) {
|
||||
var answer []struct {
|
||||
Duration string `json:"duration"`
|
||||
Scenario string `json:"scenario"`
|
||||
Scope string `json:"scope"`
|
||||
Type string `json:"type"`
|
||||
Value string `json:"value"`
|
||||
}
|
||||
|
||||
err := json.Unmarshal([]byte(strings.Join(lines, "\n")), &answer)
|
||||
if err != nil {
|
||||
return entries{}, fmt.Errorf("read the answer: %w", err)
|
||||
}
|
||||
|
||||
found := entries{decisions: map[netip.Prefix]Decision{}}
|
||||
|
||||
for i, decision := range answer {
|
||||
if decision.Type != "ban" ||
|
||||
(decision.Scope != "Ip" && decision.Scope != "Range") {
|
||||
continue
|
||||
}
|
||||
|
||||
netblock, ok := parseNetblock(decision.Value)
|
||||
|
||||
duration, err := time.ParseDuration(decision.Duration)
|
||||
if !ok || err != nil {
|
||||
return entries{}, fmt.Errorf(
|
||||
"decision %d %w",
|
||||
i+1,
|
||||
errNotDecision,
|
||||
)
|
||||
}
|
||||
|
||||
expires := fetched.Add(duration)
|
||||
if expires.After(found.decisions[netblock].Expires) {
|
||||
found.decisions[netblock] = Decision{
|
||||
Expires: expires,
|
||||
Scenario: decision.Scenario,
|
||||
}
|
||||
}
|
||||
|
||||
if !slices.Contains(found.lengths, netblock.Bits()) {
|
||||
found.lengths = append(found.lengths, netblock.Bits())
|
||||
}
|
||||
}
|
||||
|
||||
return found, nil
|
||||
}
|
||||
|
||||
// withoutComment returns line without anything after a ; or a #, and
|
||||
// without the spaces around what is left.
|
||||
func withoutComment(line string) string {
|
||||
|
||||
@@ -29,9 +29,8 @@ const (
|
||||
ActionClientIsRateLimited = "client_is_rate_limited"
|
||||
|
||||
// ActionTemporarilyBanned is a request refused because a ban that runs
|
||||
// out covers its client, or because it matched a ban rule, asked for a
|
||||
// trap path or the CrowdSec decision list lists its client, and so made
|
||||
// such a ban.
|
||||
// out covers its client, or because it matched a ban rule or asked for a
|
||||
// trap path, and so made such a ban.
|
||||
ActionTemporarilyBanned = "temporarily_banned"
|
||||
|
||||
// ActionPermanentlyBanned is a request refused because a ban that never
|
||||
@@ -184,8 +183,7 @@ type Line struct {
|
||||
// LimitHitMinuteRefusals for the refusal limit.
|
||||
LimitHit string `json:"limit_hit,omitempty"`
|
||||
// Reputation are the URLs of the blocklists that list the client, then
|
||||
// that of the CrowdSec decision list when it does, then the DNSBL zones
|
||||
// whose verdict lists it, their keys masked, then abuseipdb when its
|
||||
// the DNSBL zones whose verdict lists it, their keys masked, then abuseipdb when its
|
||||
// score is a hit.
|
||||
Reputation []string `json:"reputation,omitempty"`
|
||||
// Tags are the request's tags, sorted.
|
||||
|
||||
@@ -104,7 +104,7 @@ var (
|
||||
errVersion = errors.New("unknown version")
|
||||
// errMissing is for an entry without a field it needs.
|
||||
errMissing = errors.New("has no")
|
||||
errCause = errors.New("is not limit, attack, admin or crowdsec")
|
||||
errCause = errors.New("is not limit, attack or admin")
|
||||
errExpires = errors.New("is not after its start")
|
||||
errDestination = errors.New("is not webhook, slack or ntfy")
|
||||
errScope = errors.New(
|
||||
@@ -967,7 +967,7 @@ func (e BanEntry) ban() bans.Ban {
|
||||
// each expires is read again as written. An expires not after the start
|
||||
// is refused too, since the netblock's next ban for a broken limit, three
|
||||
// times as long as this one, would then refuse nothing; and so is a cause
|
||||
// other than limit, attack, admin or crowdsec, most likely misspelt.
|
||||
// other than limit, attack or admin, most likely misspelt.
|
||||
func (f *bansFile) check(data []byte) error {
|
||||
var written struct {
|
||||
Bans []struct {
|
||||
@@ -991,8 +991,7 @@ func (f *bansFile) check(data []byte) error {
|
||||
case entry.Expires != nil && !entry.Expires.After(entry.Start):
|
||||
return fmt.Errorf("entry %d's expires %w", i+1, errExpires)
|
||||
case entry.Cause != "" && entry.Cause != bans.CauseLimit &&
|
||||
entry.Cause != bans.CauseAttack && entry.Cause != bans.CauseAdmin &&
|
||||
entry.Cause != bans.CauseCrowdSec:
|
||||
entry.Cause != bans.CauseAttack && entry.Cause != bans.CauseAdmin:
|
||||
return fmt.Errorf(
|
||||
"entry %d's cause %q %w",
|
||||
i+1,
|
||||
|
||||
@@ -114,8 +114,7 @@ const permanentBansJSON = `{
|
||||
"earlier_bans": {
|
||||
"limit": 3,
|
||||
"attack": 1,
|
||||
"admin": 1,
|
||||
"crowdsec": 2
|
||||
"admin": 1
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1188,10 +1187,10 @@ func TestBanWithAnotherCauseStopsTheStart(t *testing.T) {
|
||||
`{"netblock": "203.0.113.10/32", "start": "2026-10-06T00:00:00Z", `+
|
||||
`"expires": null, "cause": "admin"}, `+
|
||||
`{"netblock": "203.0.113.11/32", "start": "2026-10-06T00:00:00Z", `+
|
||||
`"expires": "2026-10-06T04:00:00Z", "cause": "crowdsec"}, `+
|
||||
`"expires": "2026-10-06T04:00:00Z", "cause": "limit"}, `+
|
||||
`{"netblock": "203.0.113.12/32", "start": "2026-10-06T00:00:00Z", `+
|
||||
`"expires": null, "cause": "atack"}]}`,
|
||||
`: entry 4's cause "atack" is not limit, attack, admin or crowdsec`)
|
||||
`: entry 4's cause "atack" is not limit, attack or admin`)
|
||||
}
|
||||
|
||||
func TestBanThatDoesNotEndAfterItsStartStopsTheStart(t *testing.T) {
|
||||
@@ -2466,9 +2465,8 @@ func office() netip.Prefix {
|
||||
return netip.MustParsePrefix("203.0.113.0/24")
|
||||
}
|
||||
|
||||
// fill puts a permanent ban an admin made, a ban for a broken limit, one
|
||||
// for a clear sign of attack and one for CrowdSec's decision, clients
|
||||
// with counts and histories,
|
||||
// fill puts a permanent ban an admin made, a ban for a broken limit and one
|
||||
// for a clear sign of attack, clients with counts and histories,
|
||||
// GeoJS answers, the blocklists' last tries and the copy of one, two
|
||||
// verdicts of a DNSBL zone, the AbuseIPDB checks spent today with two
|
||||
// scores, and two answers of routewatch, as filledListsJSON,
|
||||
@@ -2485,13 +2483,6 @@ func fill(params state.Params) {
|
||||
})
|
||||
params.Ledger.BanForAttack(netip.MustParsePrefix("192.0.2.1/32"), now,
|
||||
bans.Notes{RuleID: "env-file", Target: "path"})
|
||||
params.Ledger.BanForCrowdSec(
|
||||
netip.MustParsePrefix("198.51.100.9/32"),
|
||||
now,
|
||||
now.Add(4*time.Hour),
|
||||
"crowdsecurity/ssh-bf",
|
||||
bans.Notes{},
|
||||
)
|
||||
|
||||
for _, c := range []string{"2001:db8::/64", "203.0.113.9/32", "192.0.2.1/32"} {
|
||||
params.Limiter.Count(netip.MustParsePrefix(c), now, whole)
|
||||
@@ -2634,10 +2625,9 @@ func permanentBan() bans.Ban {
|
||||
Requests: 1500,
|
||||
Refused: 3,
|
||||
EarlierBans: bans.EarlierBans{
|
||||
Limit: 3,
|
||||
Attack: 1,
|
||||
Admin: 1,
|
||||
CrowdSec: 2,
|
||||
Limit: 3,
|
||||
Attack: 1,
|
||||
Admin: 1,
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
@@ -847,92 +847,6 @@ func TestBlocklistTriesAndCopiesKeptInListsJSONAcrossRestarts(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestCrowdSecDecisionListKeptInListsJSONAcrossARestart(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
const bouncerKey = "crowdsec-key-0123456789abcdef"
|
||||
|
||||
// A stand-in for the engine's local API, which bans 203.0.113.0/24 for
|
||||
// four hours, and answers only a request with its key, while it is up.
|
||||
down := new(atomic.Bool)
|
||||
engine := httptest.NewServer(http.HandlerFunc(
|
||||
func(w http.ResponseWriter, r *http.Request) {
|
||||
switch {
|
||||
case r.URL.Path != "/v1/decisions" || r.Header.Get("X-Api-Key") != bouncerKey:
|
||||
w.WriteHeader(http.StatusForbidden)
|
||||
case down.Load():
|
||||
w.WriteHeader(http.StatusServiceUnavailable)
|
||||
default:
|
||||
_, _ = io.WriteString(
|
||||
w,
|
||||
`[{"duration": "4h0m0s", "origin": "crowdsec", `+
|
||||
`"scenario": "crowdsecurity/http-probing", "scope": "Range", `+
|
||||
`"type": "ban", "value": "203.0.113.0/24"}]`,
|
||||
)
|
||||
}
|
||||
}))
|
||||
t.Cleanup(engine.Close)
|
||||
|
||||
dir, cache := t.TempDir(), t.TempDir()
|
||||
env := map[string]string{
|
||||
listenAddr: localhost + ":0",
|
||||
stateDir: dir,
|
||||
cacheDir: cache,
|
||||
ruleFilesDir: t.TempDir(),
|
||||
"SWW_WAF_CROWDSEC_LAPI_URL": engine.URL,
|
||||
"SWW_WAF_CROWDSEC_LAPI_KEY": bouncerKey,
|
||||
}
|
||||
|
||||
// Once the list is fetched, the client's request bans it.
|
||||
first := runUntilStopped(t, env, func(url string) {
|
||||
for statusFrom(t, url, placed) != http.StatusForbidden {
|
||||
time.Sleep(pollInterval)
|
||||
}
|
||||
})
|
||||
|
||||
ban := onlyBan(t, dir)
|
||||
if ban["netblock"] != placed+"/32" || ban["cause"] != "crowdsec" ||
|
||||
ban["reason"] != "CrowdSec's decision for crowdsecurity/http-probing" {
|
||||
t.Errorf(
|
||||
"bans.json holds %v, want the ban for crowdsec on %s",
|
||||
ban,
|
||||
placed,
|
||||
)
|
||||
}
|
||||
|
||||
// Restarted with the engine down, the copy kept in lists.json bans
|
||||
// another client in the netblock from its first request.
|
||||
down.Store(true)
|
||||
|
||||
second := runUntilStopped(t, env, func(url string) {
|
||||
wantStatus(t, url, "203.0.113.10", http.StatusForbidden)
|
||||
})
|
||||
|
||||
// The key is in neither run's output, nor in a state or cache file.
|
||||
files := []string{
|
||||
filepath.Join(dir, "bans.json"), filepath.Join(dir, "clients.json"),
|
||||
filepath.Join(cache, "lists.json"),
|
||||
}
|
||||
shown := make([]string, 0, len(files)+2)
|
||||
shown = append(shown, first.text(), second.text())
|
||||
|
||||
for _, path := range files {
|
||||
data, err := os.ReadFile(path) //nolint:gosec // the test's
|
||||
if err != nil {
|
||||
t.Fatalf("read %s: %v", path, err)
|
||||
}
|
||||
|
||||
shown = append(shown, string(data))
|
||||
}
|
||||
|
||||
if all := strings.Join(shown, "\n"); strings.Contains(all, bouncerKey) {
|
||||
t.Errorf(
|
||||
"the key is shown in the output or the state and cache files:\n%s",
|
||||
all,
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
// wantDeniedByList checks that the request log line is of a request the
|
||||
// blocklist at listURL refused.
|
||||
func wantDeniedByList(t *testing.T, line map[string]any, listURL string) {
|
||||
|
||||
@@ -349,6 +349,7 @@ func TestRequestCountsForTheASNumberGeoJSGivesBeforeItEnds(t *testing.T) {
|
||||
map[string]string{
|
||||
trustedProxies: trustLocalhost,
|
||||
asnAndCountryLookupTimeout: "3600000",
|
||||
addLookupHeaders: falseValue,
|
||||
anomalyASN + requestsPerMinute: aLot,
|
||||
})
|
||||
server.Store(started)
|
||||
|
||||
@@ -48,9 +48,6 @@ const (
|
||||
silentAbuseIPDBURL = "silent://abuseipdb/api/v2/check"
|
||||
// wholeLimit is the percentage of a rate limit that lowers it not at all.
|
||||
wholeLimit = 100
|
||||
// clientsDecision is the CrowdSec decision list holding client alone.
|
||||
clientsDecision = `[{"duration": "4h0m0s", "scenario": "crowdsecurity/ssh-bf", ` +
|
||||
`"scope": "Ip", "type": "ban", "value": "` + client + `"}]`
|
||||
)
|
||||
|
||||
// cookies returns a Cookie header's value with count cookies.
|
||||
@@ -296,10 +293,6 @@ func TestTheHeaderLimitsComeBeforeEveryCheckThatRefuses(t *testing.T) {
|
||||
"/", leaveAsItIs,
|
||||
},
|
||||
{"a blocklist", map[string]string{blocklistURLs: dropURL}, "/", listClient},
|
||||
{
|
||||
"the CrowdSec decision list",
|
||||
map[string]string{crowdSecURL: lapi, crowdSecKey: bouncerKey}, "/", decideOnClient,
|
||||
},
|
||||
{
|
||||
"a DNS blocklist", map[string]string{
|
||||
dnsblZones: dnsblZone, dnsblResolver: noResolver, dnsblAbuseIPDBAction: actionDeny,
|
||||
@@ -592,12 +585,6 @@ func listClient(t *testing.T, server *waf.Server, now time.Time) {
|
||||
}
|
||||
}
|
||||
|
||||
// decideOnClient puts client on the CrowdSec decision list.
|
||||
func decideOnClient(t *testing.T, server *waf.Server, now time.Time) {
|
||||
t.Helper()
|
||||
loadDecisions(t, server, now, clientsDecision)
|
||||
}
|
||||
|
||||
// leaveAsItIs prepares nothing.
|
||||
func leaveAsItIs(*testing.T, *waf.Server, time.Time) {}
|
||||
|
||||
@@ -641,12 +628,6 @@ func TestARefusedRequestDoesNotWaitForASlowGeoJS(t *testing.T) {
|
||||
name: "a blocklist", env: map[string]string{blocklistURLs: dropURL},
|
||||
prepare: listClient, status: forbidden, action: requestlog.ActionClientDenied,
|
||||
},
|
||||
{
|
||||
name: "the CrowdSec decision list",
|
||||
env: map[string]string{crowdSecURL: lapi, crowdSecKey: bouncerKey},
|
||||
prepare: decideOnClient, status: forbidden,
|
||||
action: requestlog.ActionTemporarilyBanned,
|
||||
},
|
||||
{
|
||||
name: "the header limits", cookie: cookies(cookieLimit + 1),
|
||||
prepare: leaveAsItIs, status: tooLarge,
|
||||
@@ -774,7 +755,6 @@ func TestWAFAnswersWithinTheTimeLimitWhileEveryOutsideSourceHangs(
|
||||
dnsblZones: dnsblZone, dnsblResolver: resolver,
|
||||
abuseIPDBKey: accountKey,
|
||||
blocklistURLs: held + "/drop.txt", asnLimitPercentURL: held + "/asn.txt",
|
||||
crowdSecURL: held, crowdSecKey: bouncerKey,
|
||||
})
|
||||
|
||||
// The lists are fetched outside the synctest bubble below: a wait on the
|
||||
|
||||
@@ -11,7 +11,6 @@ import (
|
||||
"sneak.berlin/go/smallwebwaf/internal/alerts"
|
||||
"sneak.berlin/go/smallwebwaf/internal/bans"
|
||||
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
|
||||
"sneak.berlin/go/smallwebwaf/internal/reputation"
|
||||
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||
)
|
||||
|
||||
@@ -206,57 +205,6 @@ func (rq *request) banForAttack(now time.Time, notes bans.Notes) string {
|
||||
return banAction(ban.Permanent())
|
||||
}
|
||||
|
||||
// banForCrowdSec bans the client's netblock at now until decision,
|
||||
// CrowdSec's decision on the client, ends. In observe mode it makes no
|
||||
// ban, and raises the alert for the ban it would have made, if that alert
|
||||
// would be sent. It returns the action for that ban.
|
||||
func (rq *request) banForCrowdSec(
|
||||
now time.Time,
|
||||
decision reputation.Decision,
|
||||
) string {
|
||||
netblock := rq.h.netblock(rq.client)
|
||||
if rq.h.config.Observe &&
|
||||
!rq.wouldAlertBan(netblock, now, bans.CauseCrowdSec) {
|
||||
return banAction(
|
||||
rq.h.ledger.WouldBePermanent(netblock, now, bans.CauseCrowdSec),
|
||||
)
|
||||
}
|
||||
|
||||
notes := bans.Notes{
|
||||
ASN: rq.line.ASN,
|
||||
ASName: rq.line.ASName,
|
||||
Country: rq.line.Country,
|
||||
Reputation: rq.reputation,
|
||||
Request: rq.noted(now, rq.h.config.BanResponse),
|
||||
Requests: rq.netblockRequests(netblock),
|
||||
}
|
||||
|
||||
if rq.h.config.Observe {
|
||||
ban, wouldBan := rq.h.ledger.WouldBanForCrowdSec(
|
||||
netblock,
|
||||
now,
|
||||
decision.Expires,
|
||||
decision.Scenario,
|
||||
notes,
|
||||
)
|
||||
if wouldBan {
|
||||
rq.alertBan(ban)
|
||||
}
|
||||
|
||||
return banAction(ban.Permanent())
|
||||
}
|
||||
|
||||
ban, made := rq.h.ledger.BanForCrowdSec(netblock, now, decision.Expires,
|
||||
decision.Scenario, notes)
|
||||
rq.line.BanExpires = banExpires(ban)
|
||||
|
||||
if made {
|
||||
rq.alertBan(ban)
|
||||
}
|
||||
|
||||
return banAction(ban.Permanent())
|
||||
}
|
||||
|
||||
// wouldAlertBan reports whether the alert for a ban on netblock for cause
|
||||
// made at now would be sent. In observe mode the ban the request would
|
||||
// have made is worked out only then, at most once per
|
||||
|
||||
@@ -387,14 +387,16 @@ func TestRequestWaitsForItsLookupWhileABiasedThresholdIsSet(t *testing.T) {
|
||||
|
||||
// In a synctest bubble, as
|
||||
// TestRequestWaitsAsLongAsTheASNAndCountryLookupTimeoutSays says,
|
||||
// with a GeoJS that never answers.
|
||||
// with a GeoJS that never answers. The headers, which need the
|
||||
// answer too, are off.
|
||||
synctest.Test(t, func(t *testing.T) {
|
||||
server, out, _ := newWAF(t, silentGeoJSURL, time.Now,
|
||||
map[string]string{
|
||||
asnAndCountryLookupTimeout: strconv.FormatInt(
|
||||
timeout.Milliseconds(), 10),
|
||||
tagLists: "vpn=" + vpnList,
|
||||
tc.setting: tc.value,
|
||||
addLookupHeaders: falseValue,
|
||||
tagLists: "vpn=" + vpnList,
|
||||
tc.setting: tc.value,
|
||||
})
|
||||
began := time.Now()
|
||||
|
||||
|
||||
@@ -108,8 +108,11 @@ func TestCountryRefusalNamesItsHeuristicCountryAndSetting(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
geojsURL, _ := startGeoJS(t)
|
||||
// The headers are off, so that only the country list makes the request
|
||||
// wait for its answer.
|
||||
s, _, _ := startWithClock(t, geojsURL, map[string]string{
|
||||
asnAndCountryLookupTimeout: "3600000", allowedCountries: "de",
|
||||
asnAndCountryLookupTimeout: "3600000", addLookupHeaders: falseValue,
|
||||
allowedCountries: "de",
|
||||
})
|
||||
|
||||
line := s.get(
|
||||
@@ -239,7 +242,10 @@ func TestPrivateAddressIsNeverLookedUp(t *testing.T) {
|
||||
name string
|
||||
env map[string]string
|
||||
}{
|
||||
{"no setting needs the lookup", nil},
|
||||
{
|
||||
"no setting needs the lookup",
|
||||
map[string]string{addLookupHeaders: falseValue},
|
||||
},
|
||||
{"a tag setting needs it", map[string]string{tagDeny: denyKP}},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
|
||||
@@ -1,54 +0,0 @@
|
||||
package waf
|
||||
|
||||
// The heuristic crowdsec. It reads SWW_WAF_CROWDSEC_LAPI_URL, and the decisions
|
||||
// through the reputation lists.
|
||||
|
||||
import (
|
||||
"context"
|
||||
"time"
|
||||
|
||||
"sneak.berlin/go/smallwebwaf/internal/bans"
|
||||
"sneak.berlin/go/smallwebwaf/internal/config"
|
||||
"sneak.berlin/go/smallwebwaf/internal/reputation"
|
||||
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||
)
|
||||
|
||||
// crowdSec refuses a request when a decision of the CrowdSec decision list on
|
||||
// the client is in force at now. It then adds the tag crowdsec, notes the
|
||||
// list, as noteHit does, and bans the client until that decision ends. Its
|
||||
// figures are the decision's scenario and when it ends.
|
||||
type crowdSec struct{}
|
||||
|
||||
func (crowdSec) name() string { return "crowdsec" }
|
||||
|
||||
func (crowdSec) check(
|
||||
_ context.Context,
|
||||
rq *request,
|
||||
now time.Time,
|
||||
) *verdict {
|
||||
decision, listed := rq.crowdSecDecisionInForce(now)
|
||||
if !listed {
|
||||
return nil
|
||||
}
|
||||
|
||||
rq.addTag(config.CrowdSecTag)
|
||||
rq.noteHit(bans.ReputationHit{Source: rq.h.config.CrowdSecDecisionsURL},
|
||||
"listed by the CrowdSec decision list")
|
||||
|
||||
return &verdict{
|
||||
action: rq.banForCrowdSec(now, decision),
|
||||
figures: map[string]any{
|
||||
"scenario": decision.Scenario,
|
||||
"expires": requestlog.FormatTime(decision.Expires),
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
// crowdSecDecisionInForce returns the decision of the CrowdSec decision list on
|
||||
// the client that is in force at now, and reports whether there is one, which
|
||||
// the heuristic crowdsec refuses the request for.
|
||||
func (rq *request) crowdSecDecisionInForce(
|
||||
now time.Time,
|
||||
) (reputation.Decision, bool) {
|
||||
return rq.h.lists.CrowdSecDecision(rq.client, now)
|
||||
}
|
||||
@@ -1,249 +0,0 @@
|
||||
package waf_test
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
"net/netip"
|
||||
"reflect"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"sneak.berlin/go/smallwebwaf/internal/alerts"
|
||||
"sneak.berlin/go/smallwebwaf/internal/bans"
|
||||
"sneak.berlin/go/smallwebwaf/internal/reputation"
|
||||
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||
"sneak.berlin/go/smallwebwaf/internal/waf"
|
||||
)
|
||||
|
||||
// The CrowdSec settings, and the tests' engine, which is never asked: each
|
||||
// test puts in the copy of its decision list, at decisionsURL, that it
|
||||
// needs, as lists.json would at start.
|
||||
const (
|
||||
crowdSecURL = "SWW_WAF_CROWDSEC_LAPI_URL"
|
||||
crowdSecKey = "SWW_WAF_CROWDSEC_LAPI_KEY"
|
||||
lapi = "http://crowdsec.example:8080"
|
||||
decisionsURL = lapi + "/v1/decisions"
|
||||
bouncerKey = "crowdsec-key-0123456789abcdef"
|
||||
)
|
||||
|
||||
func TestClientTheCrowdSecDecisionListListsIsBannedUntilTheDecisionEnds(
|
||||
t *testing.T,
|
||||
) {
|
||||
t.Parallel()
|
||||
|
||||
s, clk, server, queue := startWithAlerts(t, map[string]string{
|
||||
crowdSecURL: lapi, crowdSecKey: bouncerKey, metricsToken: token,
|
||||
})
|
||||
// client had four hours left on its decision as the engine answered.
|
||||
fetched := clk.Now()
|
||||
loadDecisions(t, server, fetched, `[{"duration": "4h0m0s", `+
|
||||
`"scenario": "crowdsecurity/ssh-bf", "scope": "Ip", "type": "ban", `+
|
||||
`"value": "`+client+`"}]`)
|
||||
expires := requestlog.FormatTime(fetched.Add(4 * time.Hour))
|
||||
|
||||
// Its first request is refused, and bans it until the decision ends.
|
||||
banned := requestlog.ActionTemporarilyBanned
|
||||
line := s.get(client, http.StatusForbidden, banned)
|
||||
wantReputation(t, line, decisionsURL)
|
||||
|
||||
if line.BanExpires != expires {
|
||||
t.Errorf("log line has ban_expires %q, want %s",
|
||||
line.BanExpires, expires)
|
||||
}
|
||||
|
||||
listed := []bans.ReputationHit{{Source: decisionsURL}}
|
||||
|
||||
held := server.Ledger.Bans(netip.MustParsePrefix(client + "/32"))
|
||||
if len(held) != 1 || held[0].Cause != bans.CauseCrowdSec ||
|
||||
!held[0].Start.Equal(fetched) ||
|
||||
!held[0].Expires.Equal(fetched.Add(4*time.Hour)) ||
|
||||
held[0].Reason != "CrowdSec's decision for crowdsecurity/ssh-bf" ||
|
||||
!reflect.DeepEqual(held[0].Notes.Reputation, listed) ||
|
||||
held[0].Notes.Request.Path != "/" || held[0].Notes.Requests != 1 {
|
||||
t.Fatalf(
|
||||
"bans %+v, want one for crowdsec of four hours, with the list and "+
|
||||
"the request in its notes",
|
||||
held,
|
||||
)
|
||||
}
|
||||
|
||||
// The listing raises a reputation_hit alert, and the ban its own.
|
||||
waiting := queue.Snapshot().Waiting[alerts.DestinationWebhook]
|
||||
if len(waiting) != 2 || waiting[0].Event != alerts.EventReputationHit ||
|
||||
waiting[0].Reason != "listed by the CrowdSec decision list" ||
|
||||
!reflect.DeepEqual(
|
||||
waiting[1],
|
||||
banAlert(alerts.EventBan, fetched, client, held[0],
|
||||
expires),
|
||||
) {
|
||||
t.Errorf(
|
||||
"alerts waiting %+v, want a reputation_hit alert, then the ban's",
|
||||
waiting,
|
||||
)
|
||||
}
|
||||
|
||||
// Each request while the ban lasts is refused under it, as under any
|
||||
// ban, and once it has ended the client is let through.
|
||||
clk.advance(4*time.Hour - time.Second)
|
||||
|
||||
line = s.get(client, http.StatusForbidden, banned)
|
||||
wantReputation(t, line)
|
||||
|
||||
if line.BanExpires != expires {
|
||||
t.Errorf("log line has ban_expires %q, want %s",
|
||||
line.BanExpires, expires)
|
||||
}
|
||||
|
||||
clk.advance(time.Second)
|
||||
s.get(client, http.StatusOK, requestlog.ActionForward)
|
||||
|
||||
// The ban and the hit are counted, and the list has the metrics of any
|
||||
// list fetched from a URL.
|
||||
metrics := s.scrape()
|
||||
labels := `{instance="` + alertInstance + `",source="` + decisionsURL + `"}`
|
||||
|
||||
wantMetric(t, metrics, `sww_bans_made_total{cause="crowdsec",`+
|
||||
`instance="`+alertInstance+`"}`, 1)
|
||||
wantMetric(t, metrics, "sww_reputation_hits_total"+labels, 1)
|
||||
wantMetric(t, metrics, "sww_reputation_failures_total"+labels, 0)
|
||||
wantMetric(
|
||||
t,
|
||||
metrics,
|
||||
"sww_reputation_last_fetch_timestamp_seconds"+labels,
|
||||
float64(fetched.Unix()),
|
||||
)
|
||||
}
|
||||
|
||||
func TestCrowdSecBanNamesItsHeuristicScenarioAndEnd(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
s, clk, server, _ := startWithAlerts(t, map[string]string{
|
||||
crowdSecURL: lapi, crowdSecKey: bouncerKey,
|
||||
})
|
||||
loadDecisions(t, server, clk.Now(), `[{"duration": "4h0m0s", `+
|
||||
`"scenario": "crowdsecurity/ssh-bf", "scope": "Ip", "type": "ban", `+
|
||||
`"value": "`+client+`"}]`)
|
||||
|
||||
line := s.get(
|
||||
client,
|
||||
http.StatusForbidden,
|
||||
requestlog.ActionTemporarilyBanned,
|
||||
)
|
||||
wantHeuristic(
|
||||
t,
|
||||
line,
|
||||
"crowdsec",
|
||||
`{"scenario": "crowdsecurity/ssh-bf", `+
|
||||
`"expires": "`+requestlog.FormatTime(clk.Now().Add(4*time.Hour))+`"}`,
|
||||
)
|
||||
wantTags(t, line, "crowdsec")
|
||||
}
|
||||
|
||||
func TestEndedCrowdSecDecisionNoLongerBansThoughTheCopyStillHoldsIt(
|
||||
t *testing.T,
|
||||
) {
|
||||
t.Parallel()
|
||||
|
||||
s, clk, server := startWithClock(t, "", map[string]string{
|
||||
crowdSecURL: lapi, crowdSecKey: bouncerKey,
|
||||
})
|
||||
// 198.51.100.0/24 and 2001:db8::9 had a minute left as the engine
|
||||
// answered.
|
||||
fetched := clk.Now()
|
||||
loadDecisions(t, server, fetched, `[{"duration": "1m0s", `+
|
||||
`"scenario": "crowdsecurity/http-probing", "scope": "Range", "type": "ban", `+
|
||||
`"value": "198.51.100.0/24"}, {"duration": "1m0s", `+
|
||||
`"scenario": "crowdsecurity/http-probing", "scope": "Ip", "type": "ban", `+
|
||||
`"value": "2001:db8::9"}]`)
|
||||
|
||||
// Just before its end, the decision bans a client in the netblock, and
|
||||
// one on an IPv6 address bans the address's group, the /64.
|
||||
clk.advance(time.Minute - time.Nanosecond)
|
||||
s.get(
|
||||
"198.51.100.7",
|
||||
http.StatusForbidden,
|
||||
requestlog.ActionTemporarilyBanned,
|
||||
)
|
||||
s.get(
|
||||
"2001:db8::9",
|
||||
http.StatusForbidden,
|
||||
requestlog.ActionTemporarilyBanned,
|
||||
)
|
||||
s.get(
|
||||
"2001:db8::5",
|
||||
http.StatusForbidden,
|
||||
requestlog.ActionTemporarilyBanned,
|
||||
)
|
||||
|
||||
// Once it has ended, it bans no other client, and the bans it made end
|
||||
// with it.
|
||||
clk.advance(time.Nanosecond)
|
||||
|
||||
for _, from := range []string{"198.51.100.8", "198.51.100.7", "2001:db8::5"} {
|
||||
wantReputation(
|
||||
t,
|
||||
s.get(from, http.StatusOK, requestlog.ActionForward),
|
||||
)
|
||||
}
|
||||
|
||||
if made := server.Ledger.Made(bans.CauseCrowdSec); made != 2 {
|
||||
t.Errorf(
|
||||
"%d bans made for crowdsec, want 2, on 198.51.100.7/32 and "+
|
||||
"2001:db8::/64",
|
||||
made,
|
||||
)
|
||||
}
|
||||
|
||||
if held := server.Ledger.Bans(netip.MustParsePrefix("2001:db8::/64")); len(
|
||||
held,
|
||||
) != 1 {
|
||||
t.Errorf("bans of 2001:db8::/64 %+v, want one", held)
|
||||
}
|
||||
}
|
||||
|
||||
func TestObserveModeForwardsAClientTheCrowdSecDecisionListListsAndAlertsTheBan(
|
||||
t *testing.T,
|
||||
) {
|
||||
t.Parallel()
|
||||
|
||||
s, clk, server, queue := startWithAlerts(t, map[string]string{
|
||||
crowdSecURL: lapi, crowdSecKey: bouncerKey, mode: observe,
|
||||
})
|
||||
loadDecisions(t, server, clk.Now(), `[{"duration": "4h0m0s", `+
|
||||
`"scenario": "crowdsecurity/ssh-bf", "scope": "Ip", "type": "ban", `+
|
||||
`"value": "`+client+`"}]`)
|
||||
|
||||
line := s.get(client, http.StatusOK, requestlog.ActionForward)
|
||||
wantWouldAction(t, line, requestlog.ActionTemporarilyBanned)
|
||||
wantReputation(t, line, decisionsURL)
|
||||
|
||||
if held := server.Ledger.Snapshot(); len(held) != 0 {
|
||||
t.Errorf("bans %+v, want none", held)
|
||||
}
|
||||
|
||||
waiting := queue.Snapshot().Waiting[alerts.DestinationWebhook]
|
||||
if len(waiting) != 2 || waiting[1].Event != alerts.EventBan ||
|
||||
waiting[1].Detail["cause"] != bans.CauseCrowdSec ||
|
||||
waiting[1].Detail["mode"] != observe {
|
||||
t.Errorf(
|
||||
"alerts waiting %+v, want a reputation_hit alert, then the ban alert "+
|
||||
"marked observe",
|
||||
waiting,
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
// loadDecisions puts into server's lists the copy of the decision list at
|
||||
// decisionsURL, answer, the engine's answer, fetched at fetched, as
|
||||
// lists.json would at start.
|
||||
func loadDecisions(
|
||||
t *testing.T, server *waf.Server, fetched time.Time, answer string,
|
||||
) {
|
||||
t.Helper()
|
||||
|
||||
err := server.Lists.Load([]reputation.List{{
|
||||
URL: decisionsURL, Tried: fetched, Fetched: fetched, Lines: []string{answer},
|
||||
}})
|
||||
if err != nil {
|
||||
t.Fatalf("load the decision list: %v", err)
|
||||
}
|
||||
}
|
||||
@@ -60,7 +60,6 @@ func afterLookup() []heuristic {
|
||||
return []heuristic{
|
||||
countries{},
|
||||
blocklists{},
|
||||
crowdSec{},
|
||||
dnsbl{},
|
||||
abuseIPDB{},
|
||||
tagDeny{},
|
||||
|
||||
@@ -12,7 +12,7 @@ import (
|
||||
)
|
||||
|
||||
// The headers in which an answer of allow gives rproxy the client's AS number
|
||||
// and country, for the app, while SWW_WAF_ADD_LOOKUP_HEADERS is set. Go writes
|
||||
// and country, for the app, while SWW_WAF_ADD_LOOKUP_HEADERS is true. Go writes
|
||||
// every header name in this form, as it sends it and as it receives it, so
|
||||
// X-Client-ASN arrives as X-Client-Asn; header names are not case-sensitive.
|
||||
const (
|
||||
|
||||
@@ -36,6 +36,7 @@ func TestEveryClientIsLookedUpWithoutWaitingWhileNoSettingNeedsIt(
|
||||
geojsURL, asked, release := startHeldGeoJS(t)
|
||||
s, _, server := startWithClock(t, geojsURL, map[string]string{
|
||||
asnAndCountryLookupTimeout: "3600000",
|
||||
addLookupHeaders: falseValue,
|
||||
rateLimitPerMinute: "1",
|
||||
})
|
||||
|
||||
@@ -418,7 +419,7 @@ func TestClientsOwnLookupHeadersAreRemovedWhileTheSettingIsOff(
|
||||
geojsURL,
|
||||
time.Now,
|
||||
map[string]string{
|
||||
trustedProxies: trustLocalhost,
|
||||
trustedProxies: trustLocalhost, addLookupHeaders: falseValue,
|
||||
},
|
||||
)
|
||||
s := &sender{t: t, addr: addr, out: out}
|
||||
@@ -457,6 +458,8 @@ func TestRequestWaitsAsLongAsTheASNAndCountryLookupTimeoutSays(
|
||||
map[string]string{
|
||||
asnAndCountryLookupTimeout: strconv.FormatInt(
|
||||
timeout.Milliseconds(), 10),
|
||||
// Off, so that only the country list makes the request wait.
|
||||
addLookupHeaders: falseValue,
|
||||
allowedCountries: "DE",
|
||||
})
|
||||
|
||||
|
||||
@@ -75,8 +75,8 @@ type Params struct {
|
||||
Rules *rules.Files
|
||||
// Alerts receive the alert for each ban waf makes or makes
|
||||
// permanent, for each count over an anomaly threshold, for each request
|
||||
// whose client a blocklist, the CrowdSec decision list, a DNSBL zone or
|
||||
// AbuseIPDB lists, for each request the Core Rule Set scores at or over
|
||||
// whose client a blocklist, a DNSBL zone or AbuseIPDB lists, for each
|
||||
// request the Core Rule Set scores at or over
|
||||
// SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD, and for GeoJS failing, a fetch of a list
|
||||
// failing, a query to a DNSBL zone or a check with AbuseIPDB failing,
|
||||
// or the day's AbuseIPDB checks used up.
|
||||
@@ -194,11 +194,13 @@ func newReputation(
|
||||
) (*reputation.Lists, *reputation.DNSBL, *reputation.AbuseIPDB) {
|
||||
cfg := params.Config
|
||||
lists := reputation.New(reputation.Params{
|
||||
BlocklistURLs: cfg.BlocklistURLs, Refresh: cfg.BlocklistRefresh,
|
||||
ASNLimitPercentURL: cfg.ASNLimitPercentURL,
|
||||
TagLists: cfg.TagLists,
|
||||
CrowdSecDecisionsURL: cfg.CrowdSecDecisionsURL, CrowdSecKey: cfg.CrowdSecKey,
|
||||
Now: params.Now, ProcessLog: params.ProcessLog, Alerts: params.Alerts,
|
||||
BlocklistURLs: cfg.BlocklistURLs,
|
||||
Refresh: cfg.BlocklistRefresh,
|
||||
ASNLimitPercentURL: cfg.ASNLimitPercentURL,
|
||||
TagLists: cfg.TagLists,
|
||||
Now: params.Now,
|
||||
ProcessLog: params.ProcessLog,
|
||||
Alerts: params.Alerts,
|
||||
})
|
||||
dnsbl := reputation.NewDNSBL(reputation.DNSBLParams{
|
||||
Zones: cfg.DNSBLZones, Resolver: cfg.DNSBLResolver,
|
||||
|
||||
@@ -93,6 +93,10 @@ const off = "off"
|
||||
// on is the value that switches a setting that takes true or false on.
|
||||
const on = "true"
|
||||
|
||||
// falseValue is the value that switches a setting that takes true or false
|
||||
// off.
|
||||
const falseValue = "false"
|
||||
|
||||
// output collects what sww writes on stdout.
|
||||
type output struct {
|
||||
mu sync.Mutex
|
||||
|
||||
@@ -1,7 +1,6 @@
|
||||
package waf
|
||||
|
||||
// What the reputation heuristics share: blocklists, crowdsec, dnsbl and
|
||||
// abuseipdb.
|
||||
// What the reputation heuristics share: blocklists, dnsbl and abuseipdb.
|
||||
|
||||
import (
|
||||
"sneak.berlin/go/smallwebwaf/internal/alerts"
|
||||
|
||||
+4
-7
@@ -146,18 +146,15 @@ func (h *handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
|
||||
|
||||
// refusedWhateverTheAnswer reports whether the request is one that does not wait
|
||||
// for GeoJS, since the checks will refuse it at now whatever GeoJS answers about
|
||||
// its client: the heuristic crowdsec, blocklists or rate_limits will refuse it,
|
||||
// as a function in that heuristic's file says. In observe mode, which refuses
|
||||
// its client: the heuristic blocklists or rate_limits will refuse it, as a
|
||||
// function in that heuristic's file says. In observe mode, which refuses
|
||||
// nothing, none is. rproxy lets a request through when waf answers too late.
|
||||
func (rq *request) refusedWhateverTheAnswer(now time.Time) bool {
|
||||
if rq.h.config.Observe {
|
||||
return false
|
||||
}
|
||||
|
||||
_, decided := rq.crowdSecDecisionInForce(now)
|
||||
|
||||
return decided ||
|
||||
rq.refusedByBlocklists(rq.h.lists.ListedBy(rq.client)) ||
|
||||
return rq.refusedByBlocklists(rq.h.lists.ListedBy(rq.client)) ||
|
||||
rq.overAWholeRateLimit(now)
|
||||
}
|
||||
|
||||
@@ -323,7 +320,7 @@ func asTheVisitorSentIt(r *http.Request, body []byte) *http.Request {
|
||||
}
|
||||
|
||||
// allow answers a question with DispositionAllow and, while
|
||||
// SWW_WAF_ADD_LOOKUP_HEADERS is set, the client's AS number and country.
|
||||
// SWW_WAF_ADD_LOOKUP_HEADERS is true, the client's AS number and country.
|
||||
func (rq *request) allow() {
|
||||
rq.allowed = true
|
||||
|
||||
|
||||
@@ -869,7 +869,7 @@ func TestAllowCarriesTheLookupHeadersWhileTheSettingIsSet(t *testing.T) {
|
||||
wantASN, wantCountry string
|
||||
}{
|
||||
{on, "AS64496", "DE"},
|
||||
{"false", "", ""},
|
||||
{falseValue, "", ""},
|
||||
} {
|
||||
t.Run(tc.setting, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
Reference in new issue
Block a user