Compare commits
2
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
f24d50eed4 | ||
|
|
719691d1e8 |
No files matched your search
@@ -280,6 +280,7 @@ Every setting the code reads, with its default:
|
|||||||
| `SWW_WAF_EXEMPT_PATHS` | empty | Path prefixes the Core Rule Set does not inspect. |
|
| `SWW_WAF_EXEMPT_PATHS` | empty | Path prefixes the Core Rule Set does not inspect. |
|
||||||
| `SWW_WAF_BODY_LIMIT` | `off` | How much of a request body the Core Rule Set reads, a size up to `64K`; `off` reads none. |
|
| `SWW_WAF_BODY_LIMIT` | `off` | How much of a request body the Core Rule Set reads, a size up to `64K`; `off` reads none. |
|
||||||
| `SWW_CORE_RULE_SET_EXCLUSIONS_FILE` | unset | A file of Core Rule Set exclusions, as Coraza directives, that switch rules off for some fields or paths only. |
|
| `SWW_CORE_RULE_SET_EXCLUSIONS_FILE` | unset | A file of Core Rule Set exclusions, as Coraza directives, that switch rules off for some fields or paths only. |
|
||||||
|
| `SWW_APPS` | unset | The apps each hostname serves, such as `git.example.com:gitea;cloud.example.com:gitea`; only those apps' fixes apply there. Unset, every app's fixes apply to every hostname. |
|
||||||
| `SWW_BLOCKLIST_URLS` | empty | Blocklists to fetch, text files of addresses and netblocks one to a line, such as `https://www.spamhaus.org/drop/drop.txt`. |
|
| `SWW_BLOCKLIST_URLS` | empty | Blocklists to fetch, text files of addresses and netblocks one to a line, such as `https://www.spamhaus.org/drop/drop.txt`. |
|
||||||
| `SWW_BLOCKLIST_REFRESH` | `24h` | How long after a list was last tried it is fetched again, `1h` or more. |
|
| `SWW_BLOCKLIST_REFRESH` | `24h` | How long after a list was last tried it is fetched again, `1h` or more. |
|
||||||
| `SWW_BLOCKLIST_ACTION` | `deny` | What a client on a blocklist gets: `deny`, `limit:<percent>` of every limit, or `log`. |
|
| `SWW_BLOCKLIST_ACTION` | `deny` | What a client on a blocklist gets: `deny`, `limit:<percent>` of every limit, or `log`. |
|
||||||
@@ -581,11 +582,15 @@ client's limits by tag. See "Request tags" in [`SPEC.md`](SPEC.md).
|
|||||||
headers of every request, and no body unless `SWW_WAF_BODY_LIMIT` is set. In
|
headers of every request, and no body unless `SWW_WAF_BODY_LIMIT` is set. In
|
||||||
front of a code forge, bodies carry what people write and publish, which the
|
front of a code forge, bodies carry what people write and publish, which the
|
||||||
rules take for attacks.
|
rules take for attacks.
|
||||||
- `smallwebwaf` changes the Core Rule Set in eight ways so that it lets ordinary
|
- `smallwebwaf` changes the Core Rule Set in six ways so that it lets ordinary
|
||||||
requests to a code forge through, such as git over HTTP and sign-in through
|
requests to a code forge through, such as git over HTTP and sign-in through
|
||||||
Git Credential Manager. On gitea's own paths, its searches, file names and
|
Git Credential Manager. No setting undoes the changes; "Attack detection" in
|
||||||
branch names get through too. No setting undoes the changes; "Attack
|
[`SPEC.md`](SPEC.md) lists them.
|
||||||
detection" in [`SPEC.md`](SPEC.md) lists them.
|
- Each supported app also has its own fixes, which let its ordinary requests
|
||||||
|
through. gitea's let its searches, file names and branch names through.
|
||||||
|
`SWW_APPS` names the apps each hostname serves. A hostname gets only its own
|
||||||
|
apps' fixes, and a hostname it does not list gets none. Unset, every app's
|
||||||
|
fixes apply to every hostname.
|
||||||
- `SWW_CORE_RULE_SET_EXCLUSIONS_FILE` switches single rules off for an app's own
|
- `SWW_CORE_RULE_SET_EXCLUSIONS_FILE` switches single rules off for an app's own
|
||||||
fields or paths.
|
fields or paths.
|
||||||
- Rule files are plain text, one rule to a line, edited by hand and read again
|
- Rule files are plain text, one rule to a line, edited by hand and read again
|
||||||
|
|||||||
@@ -81,8 +81,8 @@ to them.
|
|||||||
of bans is full.
|
of bans is full.
|
||||||
- `internal/rules`: reads the rule files at start and again when they
|
- `internal/rules`: reads the rule files at start and again when they
|
||||||
change, and tells which rules a request matches.
|
change, and tells which rules a request matches.
|
||||||
- `internal/coreruleset`: the Core Rule Set, with the eight changes listed
|
- `internal/coreruleset`: the Core Rule Set, with the six changes and the
|
||||||
under "Attack detection", as Coraza directives.
|
apps' fixes listed under "Attack detection", as Coraza directives.
|
||||||
- `internal/lookup`: looks clients up through GeoJS or in the lookup
|
- `internal/lookup`: looks clients up through GeoJS or in the lookup
|
||||||
database.
|
database.
|
||||||
- `internal/reputation`: fetches the lists, asks the DNS blocklists and
|
- `internal/reputation`: fetches the lists, asks the DNS blocklists and
|
||||||
@@ -674,7 +674,7 @@ files, the limits and the bans still apply to the client that sends it.
|
|||||||
- `SWW_CORE_RULE_SET_EXCLUSIONS_FILE` names a file of exclusions, written as
|
- `SWW_CORE_RULE_SET_EXCLUSIONS_FILE` names a file of exclusions, written as
|
||||||
Coraza directives. An exclusion switches one rule off for some fields or paths
|
Coraza directives. An exclusion switches one rule off for some fields or paths
|
||||||
only, where `SWW_WAF_DISABLED_RULES` switches it off everywhere. The file
|
only, where `SWW_WAF_DISABLED_RULES` switches it off everywhere. The file
|
||||||
holds an app's own exclusions; the eighth change below holds gitea's.
|
holds an app's own exclusions; gitea's fixes below hold gitea's.
|
||||||
- `discriminator` loads the file after `smallwebwaf`'s own setup rules and
|
- `discriminator` loads the file after `smallwebwaf`'s own setup rules and
|
||||||
before the Core Rule Set's rules. The Core Rule Set loads its own
|
before the Core Rule Set's rules. The Core Rule Set loads its own
|
||||||
`REQUEST-900-EXCLUSION-RULES-BEFORE-CRS.conf` at the same point.
|
`REQUEST-900-EXCLUSION-RULES-BEFORE-CRS.conf` at the same point.
|
||||||
@@ -720,7 +720,7 @@ files, the limits and the bans still apply to the client that sends it.
|
|||||||
scanner from an ordinary client. In front of a code forge, registry and git
|
scanner from an ordinary client. In front of a code forge, registry and git
|
||||||
clients get `404` and `401` by design, hundreds of times a minute.
|
clients get `404` and `401` by design, hundreds of times a minute.
|
||||||
|
|
||||||
`smallwebwaf` changes the Core Rule Set 4.25.0 in eight ways, since in front of
|
`smallwebwaf` changes the Core Rule Set 4.25.0 in six ways, since in front of
|
||||||
gitea it would otherwise refuse ordinary requests. The changes hold in front of
|
gitea it would otherwise refuse ordinary requests. The changes hold in front of
|
||||||
every app, and no setting undoes them:
|
every app, and no setting undoes them:
|
||||||
|
|
||||||
@@ -739,8 +739,52 @@ every app, and no setting undoes them:
|
|||||||
and tea, which gitea registers for OAuth sign-in, ask to be sent back to
|
and tea, which gitea registers for OAuth sign-in, ask to be sent back to
|
||||||
`http://127.0.0.1` on the user's own machine. The server never fetches that
|
`http://127.0.0.1` on the user's own machine. The server never fetches that
|
||||||
address.
|
address.
|
||||||
4. Ten query parameters are not checked against the Core Rule Set's lists of
|
4. `Referer` is not checked for a Unix command given without arguments (932340),
|
||||||
system files (930120), shell paths (932160) and command names (932260):
|
for Java starting a process (944110) or for a Java class name (944130).
|
||||||
|
`Referer` names the page a request comes from, on everything that page loads
|
||||||
|
and on every link followed from it. The Core Rule Set takes some page
|
||||||
|
addresses for attacks:
|
||||||
|
- a search for `env`;
|
||||||
|
- OpenJDK's `Runtime.java`, whose path holds both `runtime` and `java.`;
|
||||||
|
- a file named `java.lang.String.md`.
|
||||||
|
|
||||||
|
Every request from such a page would be refused. `Referer` keeps the other
|
||||||
|
rules, except after some gitea searches (gitea's fixes). An app that passes
|
||||||
|
`Referer` to a shell must check it itself.
|
||||||
|
|
||||||
|
5. Responses are not inspected. A raw file from a repository, such as a shell
|
||||||
|
script, looks to the response rules like source code leaking from the server.
|
||||||
|
6. The `from` parameter of a container registry's upload path,
|
||||||
|
`/v2/<image>/blobs/uploads/`, is not checked for command names (932260). When
|
||||||
|
a push reuses a layer, `from` names the image the layer comes from. An image
|
||||||
|
such as `owner/python3-app` would have that push refused.
|
||||||
|
|
||||||
|
Each supported app also has its own fixes, which let its ordinary requests
|
||||||
|
through. An app's fixes hold for the hostnames `SWW_APPS` lists the app for:
|
||||||
|
|
||||||
|
- `SWW_APPS` holds entries separated by semicolons. Each is a hostname, a colon,
|
||||||
|
then one or more apps separated by commas, for example
|
||||||
|
`git.example.com:gitea;code.example.com:gitea`.
|
||||||
|
- The request's `Host` is matched by exact name, without its port and without
|
||||||
|
regard to case. There are no wildcards.
|
||||||
|
- A hostname not listed gets no app's fixes, so every rule checks it in full.
|
||||||
|
Set empty, `SWW_APPS` lists no hostname.
|
||||||
|
- Unset, every app's fixes hold for every hostname. An app's fixes then weaken
|
||||||
|
the check of any other app that serves the same paths or fields.
|
||||||
|
- Any of these mistakes stops the start:
|
||||||
|
- an unknown app;
|
||||||
|
- a hostname that is not a DNS name, such as `*.example.com`;
|
||||||
|
- a hostname listed twice;
|
||||||
|
- an app listed twice for one hostname;
|
||||||
|
- an entry without a colon.
|
||||||
|
|
||||||
|
`gitea` is the only app supported. Each app's fixes are in one file, such as
|
||||||
|
`internal/coreruleset/gitea.conf`, and one entry of `apps` in
|
||||||
|
`internal/coreruleset/coreruleset.go`. gitea's fixes are:
|
||||||
|
|
||||||
|
1. Ten query parameters are not checked against the Core Rule Set's lists of
|
||||||
|
system files (930120), shell paths (932160) and command names (932260), on
|
||||||
|
any path:
|
||||||
- `path`, `files`, `skip-to` and `sub_path`;
|
- `path`, `files`, `skip-to` and `sub_path`;
|
||||||
- `ref`, `sha`, `branch` and `workflow`;
|
- `ref`, `sha`, `branch` and `workflow`;
|
||||||
- `artifactName` and `redirect_to`.
|
- `artifactName` and `redirect_to`.
|
||||||
@@ -753,41 +797,18 @@ every app, and no setting undoes them:
|
|||||||
catch reaches the app, such as the command `|cat /etc/passwd`. Path
|
catch reaches the app, such as the command `|cat /etc/passwd`. Path
|
||||||
traversal, SQL and script injection, and PHP, Java and Node.js code are
|
traversal, SQL and script injection, and PHP, Java and Node.js code are
|
||||||
still refused there. Other parameters keep all three rules, except on the
|
still refused there. Other parameters keep all three rules, except on the
|
||||||
gitea paths the eighth change names. An app that uses one of these
|
gitea paths the third fix names.
|
||||||
parameters as a file on the server, or passes it to a shell, must check the
|
|
||||||
value itself. A rule file can refuse it instead.
|
|
||||||
|
|
||||||
5. Two cookies are left out, and `Referer` keeps fewer rules:
|
2. The Core Rule Set reads the request without the `gitea_flash` and
|
||||||
- The Core Rule Set reads the request without the `gitea_flash` and
|
`redirect_to` cookies. gitea puts a message naming what was just done into
|
||||||
`redirect_to` cookies. gitea puts a message naming what was just done into
|
`gitea_flash`. It puts into `redirect_to` the page to return to after signing
|
||||||
`gitea_flash`. It puts into `redirect_to` the page to return to after
|
in. The Core Rule Set takes file names in them, such as `package.json`, for
|
||||||
signing in. The Core Rule Set takes file names in them, such as
|
attacks. A browser sends such a cookie with every request until gitea
|
||||||
`package.json`, for attacks. A browser sends such a cookie with every
|
replaces it. gitea cannot replace it while those requests are refused, so one
|
||||||
request until gitea replaces it. gitea cannot replace it while those
|
refusal would keep the browser out of the whole site. gitea shows the message
|
||||||
requests are refused, so one refusal would keep the browser out of the
|
with any script removed and returns only to its own pages. Every other cookie
|
||||||
whole site. gitea shows the message with any script removed and returns
|
is read in full.
|
||||||
only to its own pages. Every other cookie is read in full. An app that
|
3. On gitea's paths, some fields skip the rules that refused ordinary gitea
|
||||||
reads a cookie of either name must check it itself.
|
|
||||||
- `Referer` is not checked for a Unix command given without arguments
|
|
||||||
(932340), for Java starting a process (944110) or for a Java class name
|
|
||||||
(944130). `Referer` names the page a request comes from, on everything
|
|
||||||
that page loads and on every link followed from it. The Core Rule Set
|
|
||||||
takes some page addresses for attacks:
|
|
||||||
- a search for `env`;
|
|
||||||
- OpenJDK's `Runtime.java`, whose path holds both `runtime` and `java.`;
|
|
||||||
- a file named `java.lang.String.md`.
|
|
||||||
|
|
||||||
Every request from such a page would be refused. `Referer` keeps the
|
|
||||||
other rules, except after some gitea searches (the eighth change). An
|
|
||||||
app that passes `Referer` to a shell must check it itself.
|
|
||||||
|
|
||||||
6. Responses are not inspected. A raw file from a repository, such as a shell
|
|
||||||
script, looks to the response rules like source code leaking from the server.
|
|
||||||
7. The `from` parameter of a container registry's upload path,
|
|
||||||
`/v2/<image>/blobs/uploads/`, is not checked for command names (932260). When
|
|
||||||
a push reuses a layer, `from` names the image the layer comes from. An image
|
|
||||||
such as `owner/python3-app` would have that push refused.
|
|
||||||
8. On gitea's paths, some fields skip the rules that refused ordinary gitea
|
|
||||||
requests there. They are the fields where gitea sends a search, a file name
|
requests there. They are the fields where gitea sends a search, a file name
|
||||||
or a branch name. `internal/coreruleset/gitea.conf` lists each rule and the
|
or a branch name. `internal/coreruleset/gitea.conf` lists each rule and the
|
||||||
request it refused. The fields are:
|
request it refused. The fields are:
|
||||||
@@ -805,8 +826,7 @@ every app, and no setting undoes them:
|
|||||||
- packages.
|
- packages.
|
||||||
|
|
||||||
A user's or an organisation's page is any path of one segment, so on
|
A user's or an organisation's page is any path of one segment, so on
|
||||||
every such path, on any app, `q` skips these three rules, for example on
|
every such path `q` skips these three rules, for example on `/search`.
|
||||||
`/search`.
|
|
||||||
|
|
||||||
Some of these searches also skip the rule for Java class names (944130),
|
Some of these searches also skip the rule for Java class names (944130),
|
||||||
which refused `org.apache.commons.logging`.
|
which refused `org.apache.commons.logging`.
|
||||||
@@ -844,11 +864,14 @@ every app, and no setting undoes them:
|
|||||||
`actions/upload-artifact@v3` (920450).
|
`actions/upload-artifact@v3` (920450).
|
||||||
|
|
||||||
Elsewhere these fields keep every rule. The paths are those of gitea at the
|
Elsewhere these fields keep every rule. The paths are those of gitea at the
|
||||||
site root. An app other than gitea that serves one of them must check these
|
site root.
|
||||||
fields itself.
|
|
||||||
|
|
||||||
The parameter and cookie names in these changes are matched without regard to
|
On a hostname that gets gitea's fixes, an app other than gitea must check these
|
||||||
case, as Coraza matches them.
|
fields and cookies itself. Listing each hostname's own apps in `SWW_APPS` keeps
|
||||||
|
the full check for the others.
|
||||||
|
|
||||||
|
The parameter names in these changes and fixes are matched without regard to
|
||||||
|
case, and the cookie names exactly.
|
||||||
|
|
||||||
### Ban lengths, response and scope
|
### Ban lengths, response and scope
|
||||||
|
|
||||||
|
|||||||
@@ -255,6 +255,24 @@ the request first.
|
|||||||
a range. So does a directive that removes or changes rules as the file loads,
|
a range. So does a directive that removes or changes rules as the file loads,
|
||||||
such as `SecRuleRemoveById`, since none of the Core Rule Set's rules is loaded
|
such as `SecRuleRemoveById`, since none of the Core Rule Set's rules is loaded
|
||||||
yet. It has no `_FILE` form.
|
yet. It has no `_FILE` form.
|
||||||
|
- `SWW_APPS` (default unset): the apps each hostname serves. A request gets the
|
||||||
|
fixes of its hostname's apps only (see "Attack detection" in
|
||||||
|
[`SPEC.md`](../SPEC.md)).
|
||||||
|
- Entries are separated by semicolons. Each is a hostname, a colon, then one
|
||||||
|
or more apps separated by commas, for example
|
||||||
|
`git.example.com:gitea;code.example.com:gitea`.
|
||||||
|
- `gitea` is the only app supported.
|
||||||
|
- The request's `Host` is matched by exact name, without its port and
|
||||||
|
without regard to case. There are no wildcards.
|
||||||
|
- A hostname not listed gets no app's fixes, so every rule checks it in
|
||||||
|
full. Set empty, it lists no hostname.
|
||||||
|
- Unset, every app's fixes apply to every hostname.
|
||||||
|
- Any of these mistakes stops the start:
|
||||||
|
- an unknown app;
|
||||||
|
- a hostname that is not a DNS name, such as `*.example.com`;
|
||||||
|
- a hostname listed twice;
|
||||||
|
- an app listed twice for one hostname;
|
||||||
|
- an entry without a colon.
|
||||||
- `SWW_ERROR_BURST_THRESHOLD` (default `30`): the most refusals a client may get
|
- `SWW_ERROR_BURST_THRESHOLD` (default `30`): the most refusals a client may get
|
||||||
in a minute from a rule file, a trap path or the Core Rule Set, a count or
|
in a minute from a rule file, a trap path or the Core Rule Set, a count or
|
||||||
`off`. The next one bans the client (see "Attack detection" in `SPEC.md`).
|
`off`. The next one bans the client (see "Attack detection" in `SPEC.md`).
|
||||||
|
|||||||
@@ -220,7 +220,7 @@ type State struct {
|
|||||||
// requests to it failed, and how many alerts were dropped from its full
|
// requests to it failed, and how many alerts were dropped from its full
|
||||||
// queue or given up as it refused them.
|
// queue or given up as it refused them.
|
||||||
type Counts struct {
|
type Counts struct {
|
||||||
Sent, Failed, Dropped int64
|
Sent, SendFailures, Dropped int64
|
||||||
}
|
}
|
||||||
|
|
||||||
// Queue takes the alerts raised, holds back those it must, and sends the
|
// Queue takes the alerts raised, holds back those it must, and sends the
|
||||||
@@ -264,7 +264,7 @@ type destination struct {
|
|||||||
// waiting are the alerts waiting to be sent, oldest first.
|
// waiting are the alerts waiting to be sent, oldest first.
|
||||||
waiting []*Alert
|
waiting []*Alert
|
||||||
|
|
||||||
sent, failed, dropped atomic.Int64
|
sent, sendFailures, dropped atomic.Int64
|
||||||
}
|
}
|
||||||
|
|
||||||
// cooldownKey is what makes an alert a repeat of another: the same event
|
// cooldownKey is what makes an alert a repeat of another: the same event
|
||||||
@@ -426,7 +426,9 @@ func (q *Queue) Counts(name string) Counts {
|
|||||||
for _, d := range q.destinations {
|
for _, d := range q.destinations {
|
||||||
if d.name == name {
|
if d.name == name {
|
||||||
return Counts{
|
return Counts{
|
||||||
Sent: d.sent.Load(), Failed: d.failed.Load(), Dropped: d.dropped.Load(),
|
Sent: d.sent.Load(),
|
||||||
|
SendFailures: d.sendFailures.Load(),
|
||||||
|
Dropped: d.dropped.Load(),
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -675,7 +677,7 @@ func (d *destination) run(ctx context.Context) {
|
|||||||
retryAt = time.Time{}
|
retryAt = time.Time{}
|
||||||
case errors.Is(err, errRefused):
|
case errors.Is(err, errRefused):
|
||||||
d.remove(alert)
|
d.remove(alert)
|
||||||
d.failed.Add(1)
|
d.sendFailures.Add(1)
|
||||||
d.dropped.Add(1)
|
d.dropped.Add(1)
|
||||||
|
|
||||||
retryDelay = 0
|
retryDelay = 0
|
||||||
@@ -684,7 +686,7 @@ func (d *destination) run(ctx context.Context) {
|
|||||||
d.processLog.Warn("gave up an alert "+d.setting+" refused",
|
d.processLog.Warn("gave up an alert "+d.setting+" refused",
|
||||||
"event", alert.Event, "error", err.Error())
|
"event", alert.Event, "error", err.Error())
|
||||||
case ctx.Err() == nil: // not cut off as sww stops
|
case ctx.Err() == nil: // not cut off as sww stops
|
||||||
d.failed.Add(1)
|
d.sendFailures.Add(1)
|
||||||
|
|
||||||
retryDelay = min(max(retryDelayFactor*retryDelay, firstRetryDelay),
|
retryDelay = min(max(retryDelayFactor*retryDelay, firstRetryDelay),
|
||||||
maxRetryDelay)
|
maxRetryDelay)
|
||||||
|
|||||||
@@ -897,7 +897,7 @@ func TestDestinationThatDoesNotAnswerHoldsUpNeitherOther(t *testing.T) {
|
|||||||
synctest.Wait()
|
synctest.Wait()
|
||||||
|
|
||||||
wantDestinationCounts(t, q, alerts.DestinationSlack,
|
wantDestinationCounts(t, q, alerts.DestinationSlack,
|
||||||
alerts.Counts{Sent: 3, Failed: 1})
|
alerts.Counts{Sent: 3, SendFailures: 1})
|
||||||
|
|
||||||
logged := log.String()
|
logged := log.String()
|
||||||
if !strings.Contains(logged,
|
if !strings.Contains(logged,
|
||||||
@@ -934,7 +934,7 @@ func TestDropsAreCountedForEachDestination(t *testing.T) {
|
|||||||
for destination, want := range map[string]alerts.Counts{
|
for destination, want := range map[string]alerts.Counts{
|
||||||
alerts.DestinationWebhook: {Sent: all},
|
alerts.DestinationWebhook: {Sent: all},
|
||||||
alerts.DestinationSlack: {Dropped: 1},
|
alerts.DestinationSlack: {Dropped: 1},
|
||||||
alerts.DestinationNtfy: {Failed: all, Dropped: all},
|
alerts.DestinationNtfy: {SendFailures: all, Dropped: all},
|
||||||
} {
|
} {
|
||||||
wantDestinationCounts(t, q, destination, want)
|
wantDestinationCounts(t, q, destination, want)
|
||||||
}
|
}
|
||||||
@@ -1223,12 +1223,12 @@ func wantEvents(t *testing.T, webhook *standIn, want ...string) {
|
|||||||
// requests to it that it counts as failed, the alerts it counts as held
|
// requests to it that it counts as failed, the alerts it counts as held
|
||||||
// back, and those it counts as dropped for the webhook.
|
// back, and those it counts as dropped for the webhook.
|
||||||
func wantCounts(
|
func wantCounts(
|
||||||
t *testing.T, q *alerts.Queue, sent, failed, suppressed, dropped int64,
|
t *testing.T, q *alerts.Queue, sent, sendFailures, suppressed, dropped int64,
|
||||||
) {
|
) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
wantDestinationCounts(t, q, alerts.DestinationWebhook,
|
wantDestinationCounts(t, q, alerts.DestinationWebhook,
|
||||||
alerts.Counts{Sent: sent, Failed: failed, Dropped: dropped})
|
alerts.Counts{Sent: sent, SendFailures: sendFailures, Dropped: dropped})
|
||||||
|
|
||||||
if q.Suppressed() != suppressed {
|
if q.Suppressed() != suppressed {
|
||||||
t.Errorf("%d alerts held back, want %d", q.Suppressed(), suppressed)
|
t.Errorf("%d alerts held back, want %d", q.Suppressed(), suppressed)
|
||||||
|
|||||||
@@ -391,7 +391,7 @@ func alertFor(r Request, k key, o overThreshold) alerts.Alert {
|
|||||||
detail["asn"] = k.asn
|
detail["asn"] = k.asn
|
||||||
case ScopeTotal:
|
case ScopeTotal:
|
||||||
counted = "the whole service"
|
counted = "the whole service"
|
||||||
default: // watch
|
default: // watched_net
|
||||||
counted = "the named netblock " + k.name + ", " + k.netblock.String()
|
counted = "the named netblock " + k.name + ", " + k.netblock.String()
|
||||||
detail["name"] = k.name
|
detail["name"] = k.name
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -29,6 +29,7 @@ import (
|
|||||||
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/alerts"
|
"sneak.berlin/go/smallwebwaf/internal/alerts"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/anomaly"
|
"sneak.berlin/go/smallwebwaf/internal/anomaly"
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/coreruleset"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/remotelog"
|
"sneak.berlin/go/smallwebwaf/internal/remotelog"
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -235,6 +236,11 @@ type Config struct {
|
|||||||
WAFExemptPaths []string
|
WAFExemptPaths []string
|
||||||
WAFBodyLimit int64
|
WAFBodyLimit int64
|
||||||
CoreRuleSetExclusionsFile string
|
CoreRuleSetExclusionsFile string
|
||||||
|
// Apps are the names of the apps each hostname serves, by hostname in
|
||||||
|
// lowercase (SWW_APPS): a request gets only its hostname's apps' fixes
|
||||||
|
// to the Core Rule Set. Apps is nil while SWW_APPS is unset, and every
|
||||||
|
// request gets every app's fixes.
|
||||||
|
Apps map[string][]string
|
||||||
// TrapPaths are the paths a request for which is a clear sign of
|
// TrapPaths are the paths a request for which is a clear sign of
|
||||||
// attack (SWW_TRAP_PATHS), each starting with / and without a ?.
|
// attack (SWW_TRAP_PATHS), each starting with / and without a ?.
|
||||||
TrapPaths []string
|
TrapPaths []string
|
||||||
@@ -404,6 +410,11 @@ var (
|
|||||||
errNotTrapPath = errors.New(
|
errNotTrapPath = errors.New(
|
||||||
"is not a path starting with / and without a ?, such as /wp-login.php",
|
"is not a path starting with / and without a ?, such as /wp-login.php",
|
||||||
)
|
)
|
||||||
|
errNotAppsEntry = errors.New(
|
||||||
|
"is not a hostname, : and a list of apps, such as git.example.com:gitea",
|
||||||
|
)
|
||||||
|
errNotHostname = errors.New("is not a DNS name such as git.example.com")
|
||||||
|
errNotApp = errors.New("is not an app sww has fixes for; the apps are")
|
||||||
errNotWAFMode = errors.New("is not off, detect or block")
|
errNotWAFMode = errors.New("is not off, detect or block")
|
||||||
errNotParanoiaLevel = errors.New("is not a paranoia level, from 1 to 4")
|
errNotParanoiaLevel = errors.New("is not a paranoia level, from 1 to 4")
|
||||||
errNotRuleID = errors.New(
|
errNotRuleID = errors.New(
|
||||||
@@ -577,6 +588,7 @@ func FromEnvironment(
|
|||||||
WAFBodyLimit: env.wafBodyLimit("SWW_WAF_BODY_LIMIT", off),
|
WAFBodyLimit: env.wafBodyLimit("SWW_WAF_BODY_LIMIT", off),
|
||||||
CoreRuleSetExclusionsFile: env.filePath(
|
CoreRuleSetExclusionsFile: env.filePath(
|
||||||
"SWW_CORE_RULE_SET_EXCLUSIONS_FILE"),
|
"SWW_CORE_RULE_SET_EXCLUSIONS_FILE"),
|
||||||
|
Apps: env.apps("SWW_APPS"),
|
||||||
TrapPaths: env.trapPaths("SWW_TRAP_PATHS"),
|
TrapPaths: env.trapPaths("SWW_TRAP_PATHS"),
|
||||||
ErrorBurstThreshold: env.count("SWW_ERROR_BURST_THRESHOLD", "30"),
|
ErrorBurstThreshold: env.count("SWW_ERROR_BURST_THRESHOLD", "30"),
|
||||||
LogRemoteURL: env.logRemoteURL("SWW_LOG_REMOTE_URL"),
|
LogRemoteURL: env.logRemoteURL("SWW_LOG_REMOTE_URL"),
|
||||||
@@ -832,6 +844,25 @@ func (e *environment) pathPrefixes(name, defaultValue string) []string {
|
|||||||
return prefixes
|
return prefixes
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// apps reads the setting that is the apps each hostname serves. It is nil
|
||||||
|
// while the setting is unset, and logged as null, unlike a setting set
|
||||||
|
// empty.
|
||||||
|
func (e *environment) apps(name string) map[string][]string {
|
||||||
|
value, set := e.lookup(name)
|
||||||
|
if !set {
|
||||||
|
e.settings = append(e.settings, slog.Any(name, nil))
|
||||||
|
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
e.settings = append(e.settings, slog.String(name, value))
|
||||||
|
|
||||||
|
apps, err := parseApps(value)
|
||||||
|
e.check(name, err)
|
||||||
|
|
||||||
|
return apps
|
||||||
|
}
|
||||||
|
|
||||||
// trapPaths reads the setting that is the list of trap paths. It is empty
|
// trapPaths reads the setting that is the list of trap paths. It is empty
|
||||||
// by default.
|
// by default.
|
||||||
func (e *environment) trapPaths(name string) []string {
|
func (e *environment) trapPaths(name string) []string {
|
||||||
@@ -1849,6 +1880,65 @@ func parseTrapPaths(value string) ([]string, error) {
|
|||||||
return paths, nil
|
return paths, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// parseApps reads the apps each hostname serves: entries separated by
|
||||||
|
// semicolons, each a hostname, a colon, and a comma-separated list of the
|
||||||
|
// names coreruleset.AppNames returns, such as
|
||||||
|
// git.example.com:gitea;cloud.example.com:nextcloud,wordpress. Each
|
||||||
|
// hostname is a DNS name, without a wildcard. It returns them by hostname
|
||||||
|
// in lowercase, since a hostname is matched without regard to case. An
|
||||||
|
// empty value lists no hostname.
|
||||||
|
func parseApps(value string) (map[string][]string, error) {
|
||||||
|
apps := map[string][]string{}
|
||||||
|
|
||||||
|
if strings.TrimSpace(value) == "" {
|
||||||
|
return apps, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
for entry := range strings.SplitSeq(value, ";") {
|
||||||
|
hostname, list, found := strings.Cut(entry, ":")
|
||||||
|
if !found {
|
||||||
|
return nil, fmt.Errorf("%q %w", entry, errNotAppsEntry)
|
||||||
|
}
|
||||||
|
|
||||||
|
hostname = strings.ToLower(strings.TrimSpace(hostname))
|
||||||
|
if hostname == "" {
|
||||||
|
return nil, fmt.Errorf("%q %w", entry, errNotAppsEntry)
|
||||||
|
}
|
||||||
|
|
||||||
|
if !isZone(hostname) {
|
||||||
|
return nil, fmt.Errorf("%q %w", hostname, errNotHostname)
|
||||||
|
}
|
||||||
|
|
||||||
|
if _, listed := apps[hostname]; listed {
|
||||||
|
return nil, fmt.Errorf("%q %w", hostname, errListedTwice)
|
||||||
|
}
|
||||||
|
|
||||||
|
names, err := parseList(list)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
if len(names) == 0 {
|
||||||
|
return nil, fmt.Errorf("%q %w", entry, errNotAppsEntry)
|
||||||
|
}
|
||||||
|
|
||||||
|
for i, name := range names {
|
||||||
|
if !slices.Contains(coreruleset.AppNames(), name) {
|
||||||
|
return nil, fmt.Errorf("%q %w %s", name, errNotApp,
|
||||||
|
strings.Join(coreruleset.AppNames(), ", "))
|
||||||
|
}
|
||||||
|
|
||||||
|
if slices.Contains(names[:i], name) {
|
||||||
|
return nil, fmt.Errorf("%q %w", name, errListedTwice)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
apps[hostname] = names
|
||||||
|
}
|
||||||
|
|
||||||
|
return apps, nil
|
||||||
|
}
|
||||||
|
|
||||||
// parseRuleIDs reads a comma-separated list of the ids of Core Rule Set
|
// parseRuleIDs reads a comma-separated list of the ids of Core Rule Set
|
||||||
// rules, each a whole number above zero and outside firstSetupRuleID to
|
// rules, each a whole number above zero and outside firstSetupRuleID to
|
||||||
// lastSetupRuleID.
|
// lastSetupRuleID.
|
||||||
@@ -2455,7 +2545,8 @@ func MaskZoneKey(zone string) string {
|
|||||||
return masked + "." + rest
|
return masked + "." + rest
|
||||||
}
|
}
|
||||||
|
|
||||||
// isZone reports whether each label of zone is as parseZones takes it.
|
// isZone reports whether each label of zone is as parseZones takes it. It
|
||||||
|
// also checks the hostnames of SWW_APPS, which are DNS names too.
|
||||||
func isZone(zone string) bool {
|
func isZone(zone string) bool {
|
||||||
for label := range strings.SplitSeq(zone, ".") {
|
for label := range strings.SplitSeq(zone, ".") {
|
||||||
badChar := strings.ContainsFunc(label, func(char rune) bool {
|
badChar := strings.ContainsFunc(label, func(char rune) bool {
|
||||||
|
|||||||
@@ -89,6 +89,7 @@ const (
|
|||||||
wafExemptPaths = "SWW_WAF_EXEMPT_PATHS"
|
wafExemptPaths = "SWW_WAF_EXEMPT_PATHS"
|
||||||
wafBodyLimit = "SWW_WAF_BODY_LIMIT"
|
wafBodyLimit = "SWW_WAF_BODY_LIMIT"
|
||||||
exclusionsFile = "SWW_CORE_RULE_SET_EXCLUSIONS_FILE"
|
exclusionsFile = "SWW_CORE_RULE_SET_EXCLUSIONS_FILE"
|
||||||
|
apps = "SWW_APPS"
|
||||||
trapPaths = "SWW_TRAP_PATHS"
|
trapPaths = "SWW_TRAP_PATHS"
|
||||||
errorBurstThreshold = "SWW_ERROR_BURST_THRESHOLD"
|
errorBurstThreshold = "SWW_ERROR_BURST_THRESHOLD"
|
||||||
logRemoteURL = "SWW_LOG_REMOTE_URL"
|
logRemoteURL = "SWW_LOG_REMOTE_URL"
|
||||||
@@ -582,6 +583,92 @@ func TestCoreRuleSetSettings(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestApps(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
for _, tc := range []struct {
|
||||||
|
env environment
|
||||||
|
want map[string][]string
|
||||||
|
}{
|
||||||
|
{environment{}, nil},
|
||||||
|
{environment{apps: ""}, map[string][]string{}},
|
||||||
|
{
|
||||||
|
environment{apps: "Git.Example.com:gitea; cloud.example.com : gitea "},
|
||||||
|
map[string][]string{
|
||||||
|
"git.example.com": {"gitea"}, "cloud.example.com": {"gitea"},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
} {
|
||||||
|
cfg := fromEnvironment(t, tc.env)
|
||||||
|
if !reflect.DeepEqual(cfg.Apps, tc.want) {
|
||||||
|
t.Errorf("%v gave %#v, want %#v", tc.env, cfg.Apps, tc.want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Unset and set empty, SWW_APPS has opposite effects, so it logs differently.
|
||||||
|
func TestUnsetAndEmptyAppsAreLoggedDifferently(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
for _, tc := range []struct {
|
||||||
|
env environment
|
||||||
|
want string
|
||||||
|
}{
|
||||||
|
{environment{}, `"` + apps + `":null`},
|
||||||
|
{environment{apps: ""}, `"` + apps + `":""`},
|
||||||
|
} {
|
||||||
|
var out bytes.Buffer
|
||||||
|
|
||||||
|
slog.New(slog.NewJSONHandler(&out, nil)).
|
||||||
|
Info("starting", "settings", fromEnvironment(t, tc.env))
|
||||||
|
|
||||||
|
if !strings.Contains(out.String(), tc.want) {
|
||||||
|
t.Errorf("%v logged %s, want %s in it", tc.env, out.String(), tc.want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestInvalidAppsStopsTheStart(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
const (
|
||||||
|
notEntry = " is not a hostname, : and a list of apps, " +
|
||||||
|
"such as git.example.com:gitea"
|
||||||
|
notHostname = " is not a DNS name such as git.example.com"
|
||||||
|
)
|
||||||
|
|
||||||
|
for _, tc := range []struct{ value, want string }{
|
||||||
|
{"git.example.com:gitea;blog.example.com", `"blog.example.com"` + notEntry},
|
||||||
|
{"git.example.com:gitea;", `""` + notEntry},
|
||||||
|
{":gitea", `":gitea"` + notEntry},
|
||||||
|
{"git.example.com:", `"git.example.com:"` + notEntry},
|
||||||
|
{"*.example.com:gitea", `"*.example.com"` + notHostname},
|
||||||
|
{"git.example.com/:gitea", `"git.example.com/"` + notHostname},
|
||||||
|
{"git example.com:gitea", `"git example.com"` + notHostname},
|
||||||
|
{
|
||||||
|
"git.example.com:gitea,nextcloud",
|
||||||
|
`"nextcloud" is not an app sww has fixes for; the apps are gitea`,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"git.example.com:gitea;GIT.example.com:gitea",
|
||||||
|
`"git.example.com" is listed twice`,
|
||||||
|
},
|
||||||
|
{"git.example.com:gitea,gitea", `"gitea" is listed twice`},
|
||||||
|
{"git.example.com:gitea,", `"gitea," has an empty item in its list`},
|
||||||
|
} {
|
||||||
|
t.Run(tc.value, func(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
_, err := config.FromEnvironment(environment{apps: tc.value}.lookupEnv)
|
||||||
|
|
||||||
|
want := apps + ": " + tc.want
|
||||||
|
if err == nil || err.Error() != want {
|
||||||
|
t.Errorf("error %v, want %s", err, want)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func TestWAFBodyLimitOf64K(t *testing.T) {
|
func TestWAFBodyLimitOf64K(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
@@ -2589,6 +2676,7 @@ func TestLogsEachSettingWithItsValue(t *testing.T) {
|
|||||||
wafExemptPaths: "",
|
wafExemptPaths: "",
|
||||||
wafBodyLimit: off,
|
wafBodyLimit: off,
|
||||||
exclusionsFile: "",
|
exclusionsFile: "",
|
||||||
|
apps: "", // logged as null, which decodes as ""
|
||||||
trapPaths: "",
|
trapPaths: "",
|
||||||
errorBurstThreshold: "30",
|
errorBurstThreshold: "30",
|
||||||
logRemoteURL: "",
|
logRemoteURL: "",
|
||||||
|
|||||||
@@ -1,9 +1,9 @@
|
|||||||
// Package coreruleset runs the OWASP Core Rule Set 4.25.0, through Coraza, on the
|
// Package coreruleset runs the OWASP Core Rule Set 4.25.0, through Coraza, on the
|
||||||
// method, the URL with its query and the headers of a request, and on its
|
// method, the URL with its query and the headers of a request, and on its
|
||||||
// body while SWW_WAF_BODY_LIMIT is set, with the eight changes sww
|
// body while SWW_WAF_BODY_LIMIT is set, with the six changes sww
|
||||||
// makes to it and the exclusions in SWW_CORE_RULE_SET_EXCLUSIONS_FILE, as
|
// makes to it, the fixes for the apps SWW_APPS names, and the exclusions in
|
||||||
// "Attack detection" under "Settings" in SPEC.md describes them. It reads no
|
// SWW_CORE_RULE_SET_EXCLUSIONS_FILE, as "Attack detection" under "Settings"
|
||||||
// response.
|
// in SPEC.md describes them. It reads no response.
|
||||||
//
|
//
|
||||||
// sww writes only to its state directory, so Coraza is built with
|
// sww writes only to its state directory, so Coraza is built with
|
||||||
// its no_fs_access tag, as the Dockerfile and script/build build it: of a
|
// its no_fs_access tag, as the Dockerfile and script/build build it: of a
|
||||||
@@ -16,6 +16,8 @@ import (
|
|||||||
"errors"
|
"errors"
|
||||||
"fmt"
|
"fmt"
|
||||||
"io"
|
"io"
|
||||||
|
"maps"
|
||||||
|
"net"
|
||||||
"net/http"
|
"net/http"
|
||||||
"net/netip"
|
"net/netip"
|
||||||
"os"
|
"os"
|
||||||
@@ -32,8 +34,8 @@ import (
|
|||||||
|
|
||||||
// directives are the Core Rule Set as sww runs it, with the
|
// directives are the Core Rule Set as sww runs it, with the
|
||||||
// paranoia level for %d, bodyDirectives for the first %s while
|
// paranoia level for %d, bodyDirectives for the first %s while
|
||||||
// SWW_WAF_BODY_LIMIT is set, the exclusions file for the second, and
|
// SWW_WAF_BODY_LIMIT is set, the exclusions file for the second, and the
|
||||||
// giteaExclusions for the third.
|
// directives of every app in apps for the third.
|
||||||
// Each rule sww adds has an id from 900000 to 900999, the ids the
|
// Each rule sww adds has an id from 900000 to 900999, the ids the
|
||||||
// Core Rule Set keeps for the rules that set it up, which
|
// Core Rule Set keeps for the rules that set it up, which
|
||||||
// SWW_WAF_DISABLED_RULES and the exclusions file refuse, so that no
|
// SWW_WAF_DISABLED_RULES and the exclusions file refuse, so that no
|
||||||
@@ -81,44 +83,28 @@ SecRule ARGUMENTS_LIMIT_REACHED "@eq 1" "id:900300,phase:2,pass,\
|
|||||||
# before the rules of phase 1 too.
|
# before the rules of phase 1 too.
|
||||||
%s
|
%s
|
||||||
|
|
||||||
# The eighth change comes before the rules for the same reason.
|
# The apps' fixes come before the rules for the same reason.
|
||||||
%s
|
%s
|
||||||
|
|
||||||
# The sixth: only the rules for requests are loaded, and no response is
|
# The fifth: only the rules for requests are loaded, and no response is
|
||||||
# inspected.
|
# inspected.
|
||||||
Include @owasp_crs/REQUEST-*.conf
|
Include @owasp_crs/REQUEST-*.conf
|
||||||
|
|
||||||
# The third: redirect_uri is not checked for a URL naming an IP address or
|
# The third: redirect_uri is not checked for a URL naming an IP address or
|
||||||
# localhost. Coraza matches a parameter name here, and in the fourth,
|
# localhost. Coraza matches a parameter name here without regard to case.
|
||||||
# without regard to case. ARGS holds the fields of a form data or multipart
|
# ARGS holds the fields of a form data or multipart body Coraza reads as
|
||||||
# body Coraza reads as well as the query parameters, so a field of one of
|
# well as the query parameters, so a field of this name is left out too.
|
||||||
# these names is left out too.
|
|
||||||
SecRuleUpdateTargetById 931100 "!ARGS:redirect_uri"
|
SecRuleUpdateTargetById 931100 "!ARGS:redirect_uri"
|
||||||
SecRuleUpdateTargetById 934110 "!ARGS:redirect_uri"
|
SecRuleUpdateTargetById 934110 "!ARGS:redirect_uri"
|
||||||
|
|
||||||
# The fourth: the query parameters in which gitea sends names within a
|
# The fourth: Referer is not checked for a Unix command without arguments,
|
||||||
# repository or its own records, or a page of its own site, are not
|
# for Java starting a process, or for a Java class name. 944130 refused
|
||||||
# checked against the lists of system files, shell paths and command
|
# every page opened from the page of java.lang.String.md.
|
||||||
# names. Coraza takes one rule id per directive.
|
|
||||||
SecRuleUpdateTargetById 930120 "!ARGS:path|!ARGS:files|!ARGS:skip-to|\
|
|
||||||
!ARGS:sub_path|!ARGS:ref|!ARGS:sha|!ARGS:branch|!ARGS:workflow|\
|
|
||||||
!ARGS:artifactName|!ARGS:redirect_to"
|
|
||||||
SecRuleUpdateTargetById 932160 "!ARGS:path|!ARGS:files|!ARGS:skip-to|\
|
|
||||||
!ARGS:sub_path|!ARGS:ref|!ARGS:sha|!ARGS:branch|!ARGS:workflow|\
|
|
||||||
!ARGS:artifactName|!ARGS:redirect_to"
|
|
||||||
SecRuleUpdateTargetById 932260 "!ARGS:path|!ARGS:files|!ARGS:skip-to|\
|
|
||||||
!ARGS:sub_path|!ARGS:ref|!ARGS:sha|!ARGS:branch|!ARGS:workflow|\
|
|
||||||
!ARGS:artifactName|!ARGS:redirect_to"
|
|
||||||
|
|
||||||
# The fifth, for Referer: it is not checked for a Unix command without
|
|
||||||
# arguments, for Java starting a process, or for a Java class name. 944130
|
|
||||||
# refused every page opened from the page of java.lang.String.md. The
|
|
||||||
# cookies are left out in Inspect.
|
|
||||||
SecRuleUpdateTargetById 932340 "!REQUEST_HEADERS:Referer"
|
SecRuleUpdateTargetById 932340 "!REQUEST_HEADERS:Referer"
|
||||||
SecRuleUpdateTargetById 944110 "!REQUEST_HEADERS:Referer"
|
SecRuleUpdateTargetById 944110 "!REQUEST_HEADERS:Referer"
|
||||||
SecRuleUpdateTargetById 944130 "!REQUEST_HEADERS:Referer"
|
SecRuleUpdateTargetById 944130 "!REQUEST_HEADERS:Referer"
|
||||||
|
|
||||||
# The seventh: a container image push that reuses a layer names the image
|
# The sixth: a container image push that reuses a layer names the image
|
||||||
# it takes the layer from in the query parameter from, which is not
|
# it takes the layer from in the query parameter from, which is not
|
||||||
# checked for command names. 932260 refused the push of a layer from
|
# checked for command names. 932260 refused the push of a layer from
|
||||||
# owner/python3-app.
|
# owner/python3-app.
|
||||||
@@ -168,18 +154,36 @@ SecRule MULTIPART_STRICT_ERROR "!@eq 0" "id:900450,phase:2,pass,\
|
|||||||
severity:'CRITICAL',setvar:'tx.inbound_anomaly_score_pl1=+5'"
|
severity:'CRITICAL',setvar:'tx.inbound_anomaly_score_pl1=+5'"
|
||||||
`
|
`
|
||||||
|
|
||||||
// giteaExclusions are the eighth change: the fields of gitea's own
|
// app is an app whose ordinary requests the Core Rule Set would refuse,
|
||||||
// requests that skip the rules that refused them there, as Coraza
|
// with the fixes that let them pass.
|
||||||
// directives.
|
type app struct {
|
||||||
//
|
// directives are Coraza exclusions, loaded before the Core Rule Set's
|
||||||
//go:embed gitea.conf
|
// rules. Their first rule skips the rest unless Inspect sets the
|
||||||
var giteaExclusions string
|
// transaction variable app_ and the app's name, and they end with the
|
||||||
|
// marker it skips to.
|
||||||
|
directives string
|
||||||
|
// cookies are the cookies the Core Rule Set reads a request without.
|
||||||
|
cookies []string
|
||||||
|
}
|
||||||
|
|
||||||
// cookiesNotRead are the cookies the Core Rule Set reads a request
|
//go:embed gitea.conf
|
||||||
// without, the rest of the fifth change.
|
var giteaDirectives string
|
||||||
|
|
||||||
|
// apps are the apps whose fixes sww has, by the name SWW_APPS gives each.
|
||||||
//
|
//
|
||||||
//nolint:gochecknoglobals // a constant cannot be a list
|
//nolint:gochecknoglobals // a constant cannot be a map
|
||||||
var cookiesNotRead = []string{"gitea_flash", "redirect_to"}
|
var apps = map[string]app{
|
||||||
|
"gitea": {
|
||||||
|
directives: giteaDirectives,
|
||||||
|
cookies: []string{"gitea_flash", "redirect_to"},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
// AppNames returns the names of the apps whose fixes sww has, in
|
||||||
|
// alphabetical order.
|
||||||
|
func AppNames() []string {
|
||||||
|
return slices.Sorted(maps.Keys(apps))
|
||||||
|
}
|
||||||
|
|
||||||
// Params are what New needs.
|
// Params are what New needs.
|
||||||
type Params struct {
|
type Params struct {
|
||||||
@@ -195,6 +199,10 @@ type Params struct {
|
|||||||
// Core Rule Set's rules (SWW_CORE_RULE_SET_EXCLUSIONS_FILE), "" while
|
// Core Rule Set's rules (SWW_CORE_RULE_SET_EXCLUSIONS_FILE), "" while
|
||||||
// there is none.
|
// there is none.
|
||||||
ExclusionsFile string
|
ExclusionsFile string
|
||||||
|
// Apps are the names of the apps each hostname serves, by hostname in
|
||||||
|
// lowercase (SWW_APPS). A request gets the fixes of its hostname's
|
||||||
|
// apps only, or of every app while Apps is nil.
|
||||||
|
Apps map[string][]string
|
||||||
}
|
}
|
||||||
|
|
||||||
// The ids of sww's own rules, which an exclusions file may not
|
// The ids of sww's own rules, which an exclusions file may not
|
||||||
@@ -284,15 +292,17 @@ func checkExclusions(exclusions string) error {
|
|||||||
// CoreRuleSet is the Core Rule Set, ready to inspect requests. It is safe
|
// CoreRuleSet is the Core Rule Set, ready to inspect requests. It is safe
|
||||||
// for concurrent use.
|
// for concurrent use.
|
||||||
type CoreRuleSet struct {
|
type CoreRuleSet struct {
|
||||||
waf coraza.WAF
|
engine coraza.WAF
|
||||||
bodyLimit int64
|
bodyLimit int64
|
||||||
|
apps map[string][]string
|
||||||
}
|
}
|
||||||
|
|
||||||
// New returns the Core Rule Set with the eight changes, at params'
|
// New returns the Core Rule Set with the six changes and the apps' fixes,
|
||||||
// paranoia level, without the rules it switches off, reading request
|
// at params' paranoia level, without the rules it switches off, reading
|
||||||
// bodies up to params' limit, and with params' exclusions file loaded
|
// request bodies up to params' limit, and with params' exclusions file
|
||||||
// before the Core Rule Set's rules. An exclusions file is an error if it
|
// loaded before the Core Rule Set's rules. An exclusions file is an error
|
||||||
// cannot be read or does not parse, or if checkExclusions refuses it.
|
// if it cannot be read or does not parse, or if checkExclusions refuses
|
||||||
|
// it.
|
||||||
func New(params Params) (*CoreRuleSet, error) {
|
func New(params Params) (*CoreRuleSet, error) {
|
||||||
body := ""
|
body := ""
|
||||||
if params.BodyLimit > 0 {
|
if params.BodyLimit > 0 {
|
||||||
@@ -315,8 +325,13 @@ func New(params Params) (*CoreRuleSet, error) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
var fixes strings.Builder
|
||||||
|
for _, name := range AppNames() {
|
||||||
|
fixes.WriteString(apps[name].directives + "\n")
|
||||||
|
}
|
||||||
|
|
||||||
text := fmt.Sprintf(directives, params.ParanoiaLevel, body, exclusions,
|
text := fmt.Sprintf(directives, params.ParanoiaLevel, body, exclusions,
|
||||||
giteaExclusions)
|
fixes.String())
|
||||||
|
|
||||||
if len(params.DisabledRules) > 0 {
|
if len(params.DisabledRules) > 0 {
|
||||||
ids := make([]string, len(params.DisabledRules))
|
ids := make([]string, len(params.DisabledRules))
|
||||||
@@ -327,14 +342,16 @@ func New(params Params) (*CoreRuleSet, error) {
|
|||||||
text += "SecRuleRemoveById " + strings.Join(ids, " ") + "\n"
|
text += "SecRuleRemoveById " + strings.Join(ids, " ") + "\n"
|
||||||
}
|
}
|
||||||
|
|
||||||
waf, err := coraza.NewWAF(coraza.NewWAFConfig().
|
engine, err := coraza.NewWAF(coraza.NewWAFConfig().
|
||||||
WithRootFS(coreruleset.FS).
|
WithRootFS(coreruleset.FS).
|
||||||
WithDirectives(text))
|
WithDirectives(text))
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, fmt.Errorf("load the Core Rule Set: %w", err)
|
return nil, fmt.Errorf("load the Core Rule Set: %w", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
return &CoreRuleSet{waf: waf, bodyLimit: params.BodyLimit}, nil
|
return &CoreRuleSet{
|
||||||
|
engine: engine, bodyLimit: params.BodyLimit, apps: params.Apps,
|
||||||
|
}, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
// Result is what the Core Rule Set found in a request.
|
// Result is what the Core Rule Set found in a request.
|
||||||
@@ -346,25 +363,37 @@ type Result struct {
|
|||||||
Score int
|
Score int
|
||||||
}
|
}
|
||||||
|
|
||||||
// Inspect runs the Core Rule Set on r, a request from client: on its
|
// Inspect runs the Core Rule Set on r, a request from client, with the
|
||||||
// method, its URL with the query, and its headers, the Cookie header
|
// fixes of the apps appsFor names for r's hostname: on its method, its URL
|
||||||
// without the cookies in cookiesNotRead, and on body, r's body as the
|
// with the query, and its headers, the Cookie header without those apps'
|
||||||
// caller has it, as readBody reads it. It returns what it found, what it
|
// cookies, and on body, r's body as the caller has it, as readBody reads
|
||||||
// read of body, and the error that ended the reading early, if one did.
|
// it. It returns what it found, what it read of body, and the error that
|
||||||
|
// ended the reading early, if one did.
|
||||||
func (c *CoreRuleSet) Inspect(
|
func (c *CoreRuleSet) Inspect(
|
||||||
r *http.Request, client netip.Addr, body io.Reader,
|
r *http.Request, client netip.Addr, body io.Reader,
|
||||||
) (Result, []byte, error) {
|
) (Result, []byte, error) {
|
||||||
tx := c.waf.NewTransaction()
|
tx := c.engine.NewTransaction()
|
||||||
// Closing would remove the files Coraza wrote, and it writes none.
|
// Closing would remove the files Coraza wrote, and it writes none.
|
||||||
defer func() { _ = tx.Close() }()
|
defer func() { _ = tx.Close() }()
|
||||||
|
|
||||||
|
// The transaction's variables are set only through Coraza's interface
|
||||||
|
// for plugins.
|
||||||
|
state := tx.(plugintypes.TransactionState) //nolint:forcetypeassert // every one is
|
||||||
|
|
||||||
|
var cookiesNotRead []string
|
||||||
|
|
||||||
|
for _, name := range c.appsFor(r.Host) {
|
||||||
|
state.Variables().TX().Set("app_"+name, []string{"1"})
|
||||||
|
cookiesNotRead = append(cookiesNotRead, apps[name].cookies...)
|
||||||
|
}
|
||||||
|
|
||||||
tx.ProcessConnection(client.String(), 0, "", 0)
|
tx.ProcessConnection(client.String(), 0, "", 0)
|
||||||
tx.ProcessURI(r.URL.String(), r.Method, r.Proto)
|
tx.ProcessURI(r.URL.String(), r.Method, r.Proto)
|
||||||
|
|
||||||
for name, values := range r.Header {
|
for name, values := range r.Header {
|
||||||
for _, value := range values {
|
for _, value := range values {
|
||||||
if name == "Cookie" {
|
if name == "Cookie" {
|
||||||
value = withoutCookiesNotRead(value)
|
value = withoutCookies(value, cookiesNotRead)
|
||||||
if value == "" {
|
if value == "" {
|
||||||
continue // it held those cookies alone
|
continue // it held those cookies alone
|
||||||
}
|
}
|
||||||
@@ -457,9 +486,26 @@ func score(tx types.Transaction) int {
|
|||||||
return n
|
return n
|
||||||
}
|
}
|
||||||
|
|
||||||
// withoutCookiesNotRead returns value, a Cookie header's, without the
|
// appsFor returns the names of the apps whose fixes a request for host,
|
||||||
// cookies in cookiesNotRead.
|
// its Host header, gets: every app while SWW_APPS is unset, and otherwise
|
||||||
func withoutCookiesNotRead(value string) string {
|
// those SWW_APPS lists for host's name, without the port and in
|
||||||
|
// lowercase.
|
||||||
|
func (c *CoreRuleSet) appsFor(host string) []string {
|
||||||
|
if c.apps == nil {
|
||||||
|
return AppNames()
|
||||||
|
}
|
||||||
|
|
||||||
|
hostname, _, err := net.SplitHostPort(host)
|
||||||
|
if err != nil {
|
||||||
|
hostname = host // it has no port
|
||||||
|
}
|
||||||
|
|
||||||
|
return c.apps[strings.ToLower(hostname)]
|
||||||
|
}
|
||||||
|
|
||||||
|
// withoutCookies returns value, a Cookie header's, without the cookies
|
||||||
|
// named in cookiesNotRead.
|
||||||
|
func withoutCookies(value string, cookiesNotRead []string) string {
|
||||||
var kept []string
|
var kept []string
|
||||||
|
|
||||||
for cookie := range strings.SplitSeq(value, ";") {
|
for cookie := range strings.SplitSeq(value, ";") {
|
||||||
|
|||||||
@@ -124,8 +124,8 @@ func atDefaults(t *testing.T) *coreruleset.CoreRuleSet {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// wantChange checks that crs lets through passes, a gitea request one of
|
// wantChange checks that crs lets through passes, a gitea request one of
|
||||||
// the eight changes is for, and still finds result in refused, a request
|
// the six changes or gitea's fixes are for, and still finds result in
|
||||||
// like it that the change is not for.
|
// refused, a request like it that they are not for.
|
||||||
func wantChange(
|
func wantChange(
|
||||||
t *testing.T, crs *coreruleset.CoreRuleSet, passes, refused request,
|
t *testing.T, crs *coreruleset.CoreRuleSet, passes, refused request,
|
||||||
result coreruleset.Result,
|
result coreruleset.Result,
|
||||||
@@ -386,6 +386,96 @@ func TestImagePushThatReusesALayerIsNotCheckedForCommandNames(t *testing.T) {
|
|||||||
matched(930120, 932160))
|
matched(930120, 932160))
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// giteaFixed are gitea requests that only gitea's fixes let through, one
|
||||||
|
// for each kind of fix, each with what the Core Rule Set finds in it
|
||||||
|
// without them.
|
||||||
|
//
|
||||||
|
//nolint:gochecknoglobals // a constant cannot be a list
|
||||||
|
var giteaFixed = []struct {
|
||||||
|
request request
|
||||||
|
result coreruleset.Result
|
||||||
|
}{
|
||||||
|
// A search of code, on the path of gitea's search.
|
||||||
|
{get("/explore/code?q=SELECT%20*%20FROM%20users%20WHERE"), matched(942100)},
|
||||||
|
// A file name, in a parameter gitea sends names in, on any path.
|
||||||
|
{get("/?path=.gitignore"), matched(930120)},
|
||||||
|
// A file name, in a cookie of gitea's.
|
||||||
|
{
|
||||||
|
get("/owner/repo", "Cookie: gitea_flash=success%3DFile%2Bpackage.json%2Bdeleted"),
|
||||||
|
matched(930120),
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
// The hostnames of the tests of SWW_APPS: the one that serves gitea, and
|
||||||
|
// one that serves another app.
|
||||||
|
const (
|
||||||
|
giteaHost = "git.example"
|
||||||
|
otherHost = "blog.example"
|
||||||
|
)
|
||||||
|
|
||||||
|
// withHost returns r sent to host.
|
||||||
|
func withHost(r request, host string) request {
|
||||||
|
return request{r.method, r.target, append([]string{"Host: " + host}, r.headers...)}
|
||||||
|
}
|
||||||
|
|
||||||
|
// servingApps returns the Core Rule Set at the defaults, with SWW_APPS
|
||||||
|
// giving apps.
|
||||||
|
func servingApps(t *testing.T, apps map[string][]string) *coreruleset.CoreRuleSet {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
crs, err := coreruleset.New(coreruleset.Params{
|
||||||
|
ParanoiaLevel: 1, DisabledRules: defaultDisabledRules, Apps: apps,
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("load the Core Rule Set: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
return crs
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestAppsUnsetGivesEveryHostnameEveryAppsFixes(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
crs := servingApps(t, nil)
|
||||||
|
|
||||||
|
for _, fixed := range giteaFixed {
|
||||||
|
for _, host := range []string{giteaHost, otherHost} {
|
||||||
|
wantResult(t, crs, withHost(fixed.request, host), coreruleset.Result{})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestListedHostnameGetsItsAppsFixes(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
crs := servingApps(t, map[string][]string{giteaHost: {"gitea"}})
|
||||||
|
|
||||||
|
// The hostname is matched without the port and without regard to case.
|
||||||
|
for _, host := range []string{giteaHost, "GIT.Example:8443"} {
|
||||||
|
for _, fixed := range giteaFixed {
|
||||||
|
wantResult(t, crs, withHost(fixed.request, host), coreruleset.Result{})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestUnlistedHostnameGetsTheFullCheck(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
for _, apps := range []map[string][]string{
|
||||||
|
{giteaHost: {"gitea"}},
|
||||||
|
// SWW_APPS set empty lists no hostname.
|
||||||
|
{},
|
||||||
|
} {
|
||||||
|
crs := servingApps(t, apps)
|
||||||
|
|
||||||
|
for _, host := range []string{otherHost, otherHost + ":8443", "git.example.org"} {
|
||||||
|
for _, fixed := range giteaFixed {
|
||||||
|
wantResult(t, crs, withHost(fixed.request, host), fixed.result)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// withExclusions returns the Core Rule Set at the defaults, with the
|
// withExclusions returns the Core Rule Set at the defaults, with the
|
||||||
// exclusions file at path.
|
// exclusions file at path.
|
||||||
func withExclusions(path string) (*coreruleset.CoreRuleSet, error) {
|
func withExclusions(path string) (*coreruleset.CoreRuleSet, error) {
|
||||||
|
|||||||
@@ -1,8 +1,51 @@
|
|||||||
# The eighth change: on gitea's paths, some fields skip the rules that refused
|
# gitea's fixes: some fields skip the rules that refused ordinary gitea requests.
|
||||||
# ordinary gitea requests there. They are the fields where gitea sends a search,
|
# They are the fields where gitea sends a search, a file name or a branch name.
|
||||||
# a file name or a branch name. Each exclusion runs with the request, before the
|
# Each exclusion runs with the request, before the rules it switches off, those
|
||||||
# rules it switches off, those of phase 1 included, so this file is loaded before
|
# of phase 1 included, so this file is loaded before the Core Rule Set's rules.
|
||||||
# the Core Rule Set's rules. Each comment names a request the rule refused.
|
# Each comment names a request the rule refused.
|
||||||
|
|
||||||
|
# A request whose hostname SWW_APPS does not list gitea for skips every rule
|
||||||
|
# below. Inspect sets tx.app_gitea on the others.
|
||||||
|
SecRule &TX:app_gitea "@eq 0" \
|
||||||
|
"id:900590,phase:1,pass,nolog,skipAfter:END_GITEA"
|
||||||
|
|
||||||
|
# The query parameters in which gitea sends names within a repository or its
|
||||||
|
# own records, or a page of its own site, are not checked against the lists of
|
||||||
|
# system files, shell paths and command names, on any path:
|
||||||
|
# 930120: .gitignore
|
||||||
|
# 932160: bin/docker-entrypoint
|
||||||
|
# 932260: docker-build
|
||||||
|
SecAction "id:900591,phase:1,pass,nolog,\
|
||||||
|
ctl:ruleRemoveTargetById=930120;ARGS:path,\
|
||||||
|
ctl:ruleRemoveTargetById=930120;ARGS:files,\
|
||||||
|
ctl:ruleRemoveTargetById=930120;ARGS:skip-to,\
|
||||||
|
ctl:ruleRemoveTargetById=930120;ARGS:sub_path,\
|
||||||
|
ctl:ruleRemoveTargetById=930120;ARGS:ref,\
|
||||||
|
ctl:ruleRemoveTargetById=930120;ARGS:sha,\
|
||||||
|
ctl:ruleRemoveTargetById=930120;ARGS:branch,\
|
||||||
|
ctl:ruleRemoveTargetById=930120;ARGS:workflow,\
|
||||||
|
ctl:ruleRemoveTargetById=930120;ARGS:artifactName,\
|
||||||
|
ctl:ruleRemoveTargetById=930120;ARGS:redirect_to,\
|
||||||
|
ctl:ruleRemoveTargetById=932160;ARGS:path,\
|
||||||
|
ctl:ruleRemoveTargetById=932160;ARGS:files,\
|
||||||
|
ctl:ruleRemoveTargetById=932160;ARGS:skip-to,\
|
||||||
|
ctl:ruleRemoveTargetById=932160;ARGS:sub_path,\
|
||||||
|
ctl:ruleRemoveTargetById=932160;ARGS:ref,\
|
||||||
|
ctl:ruleRemoveTargetById=932160;ARGS:sha,\
|
||||||
|
ctl:ruleRemoveTargetById=932160;ARGS:branch,\
|
||||||
|
ctl:ruleRemoveTargetById=932160;ARGS:workflow,\
|
||||||
|
ctl:ruleRemoveTargetById=932160;ARGS:artifactName,\
|
||||||
|
ctl:ruleRemoveTargetById=932160;ARGS:redirect_to,\
|
||||||
|
ctl:ruleRemoveTargetById=932260;ARGS:path,\
|
||||||
|
ctl:ruleRemoveTargetById=932260;ARGS:files,\
|
||||||
|
ctl:ruleRemoveTargetById=932260;ARGS:skip-to,\
|
||||||
|
ctl:ruleRemoveTargetById=932260;ARGS:sub_path,\
|
||||||
|
ctl:ruleRemoveTargetById=932260;ARGS:ref,\
|
||||||
|
ctl:ruleRemoveTargetById=932260;ARGS:sha,\
|
||||||
|
ctl:ruleRemoveTargetById=932260;ARGS:branch,\
|
||||||
|
ctl:ruleRemoveTargetById=932260;ARGS:workflow,\
|
||||||
|
ctl:ruleRemoveTargetById=932260;ARGS:artifactName,\
|
||||||
|
ctl:ruleRemoveTargetById=932260;ARGS:redirect_to"
|
||||||
|
|
||||||
# gitea's searches of code, of issues and pull requests, and of commits send the
|
# gitea's searches of code, of issues and pull requests, and of commits send the
|
||||||
# text searched for in q, which skips these rules there:
|
# text searched for in q, which skips these rules there:
|
||||||
@@ -155,8 +198,8 @@ SecRule REQUEST_FILENAME \
|
|||||||
# 942140: information_schema.sql
|
# 942140: information_schema.sql
|
||||||
# 944110: java/lang/Runtime.java
|
# 944110: java/lang/Runtime.java
|
||||||
# 944130: java.lang.String.md
|
# 944130: java.lang.String.md
|
||||||
# Each request skips the rules that refused it. The fourth change already has
|
# Each request skips the rules that refused it. Rule 900591 already has path and
|
||||||
# path and redirect_to skip 930120, 932160 and 932260 on every path.
|
# redirect_to skip 930120, 932160 and 932260 on every path.
|
||||||
|
|
||||||
# A file's commit graph, in file.
|
# A file's commit graph, in file.
|
||||||
SecRule REQUEST_FILENAME "@rx ^/[^/]+/[^/]+/graph$" \
|
SecRule REQUEST_FILENAME "@rx ^/[^/]+/[^/]+/graph$" \
|
||||||
@@ -281,3 +324,5 @@ SecRule REQUEST_FILENAME \
|
|||||||
"@rx ^/api/actions_pipeline/_apis/pipelines/workflows/[0-9]+/artifacts/[^/]+/upload$" \
|
"@rx ^/api/actions_pipeline/_apis/pipelines/workflows/[0-9]+/artifacts/[^/]+/upload$" \
|
||||||
"id:900640,phase:1,pass,nolog,\
|
"id:900640,phase:1,pass,nolog,\
|
||||||
ctl:ruleRemoveTargetById=920450;REQUEST_HEADERS_NAMES:Content-Range"
|
ctl:ruleRemoveTargetById=920450;REQUEST_HEADERS_NAMES:Content-Range"
|
||||||
|
|
||||||
|
SecMarker END_GITEA
|
||||||
@@ -40,7 +40,7 @@ func TestAdminEndpointsAreOffWhileTheTokenIsUnset(t *testing.T) {
|
|||||||
// An empty token does not match the unset one either.
|
// An empty token does not match the unset one either.
|
||||||
for _, authorization := range []string{adminBearer, bearer, "Bearer ", ""} {
|
for _, authorization := range []string{adminBearer, bearer, "Bearer ", ""} {
|
||||||
for _, e := range adminEndpoints() {
|
for _, e := range adminEndpoints() {
|
||||||
s.adminRequest(authorization, e.method, e.path, e.body, http.StatusNotFound)
|
s.ownEndpointRequest(authorization, e.method, e.path, e.body, http.StatusNotFound)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -67,7 +67,7 @@ func TestAdminEndpointsNeedTheAdminToken(t *testing.T) {
|
|||||||
for _, authorization := range []string{
|
for _, authorization := range []string{
|
||||||
"", bearer, "Bearer " + strings.ToUpper(adminSecret), "Basic " + adminSecret,
|
"", bearer, "Bearer " + strings.ToUpper(adminSecret), "Basic " + adminSecret,
|
||||||
} {
|
} {
|
||||||
got := s.adminRequest(authorization, e.method, e.path, e.body,
|
got := s.ownEndpointRequest(authorization, e.method, e.path, e.body,
|
||||||
http.StatusUnauthorized)
|
http.StatusUnauthorized)
|
||||||
if got.header.Get("WWW-Authenticate") != "Bearer" {
|
if got.header.Get("WWW-Authenticate") != "Bearer" {
|
||||||
t.Errorf("%s %s with %q was answered without WWW-Authenticate: Bearer",
|
t.Errorf("%s %s with %q was answered without WWW-Authenticate: Bearer",
|
||||||
@@ -329,7 +329,7 @@ func TestAdminEndpointsNeedTheTokenInObserveMode(t *testing.T) {
|
|||||||
})
|
})
|
||||||
|
|
||||||
for _, e := range adminEndpoints() {
|
for _, e := range adminEndpoints() {
|
||||||
s.adminRequest("", e.method, e.path, e.body, http.StatusUnauthorized)
|
s.ownEndpointRequest("", e.method, e.path, e.body, http.StatusUnauthorized)
|
||||||
}
|
}
|
||||||
|
|
||||||
if held := server.Ledger.Snapshot(); len(held) != 0 {
|
if held := server.Ledger.Snapshot(); len(held) != 0 {
|
||||||
@@ -360,12 +360,12 @@ func adminEndpoints() []adminEndpoint {
|
|||||||
func (s *sender) admin(method, path, body string, status int) answer {
|
func (s *sender) admin(method, path, body string, status int) answer {
|
||||||
s.t.Helper()
|
s.t.Helper()
|
||||||
|
|
||||||
return s.adminRequest(adminBearer, method, path, body, status)
|
return s.ownEndpointRequest(adminBearer, method, path, body, status)
|
||||||
}
|
}
|
||||||
|
|
||||||
// adminRequest is admin with authorization as the request's Authorization header
|
// ownEndpointRequest is admin with authorization as the request's Authorization
|
||||||
// unless it is "".
|
// header unless it is "".
|
||||||
func (s *sender) adminRequest(
|
func (s *sender) ownEndpointRequest(
|
||||||
authorization, method, path, body string, status int,
|
authorization, method, path, body string, status int,
|
||||||
) answer {
|
) answer {
|
||||||
s.t.Helper()
|
s.t.Helper()
|
||||||
|
|||||||
@@ -36,8 +36,8 @@ const (
|
|||||||
// tooLarge is how discriminator refuses a request over the header limits.
|
// tooLarge is how discriminator refuses a request over the header limits.
|
||||||
tooLarge = http.StatusRequestHeaderFieldsTooLarge
|
tooLarge = http.StatusRequestHeaderFieldsTooLarge
|
||||||
// silentAbuseIPDBURL is where AbuseIPDB never answers: at the stand-in
|
// silentAbuseIPDBURL is where AbuseIPDB never answers: at the stand-in
|
||||||
// that unansweredGeoJSURL names too.
|
// that silentGeoJSURL names too.
|
||||||
silentAbuseIPDBURL = "unanswered://abuseipdb/api/v2/check"
|
silentAbuseIPDBURL = "silent://abuseipdb/api/v2/check"
|
||||||
// wholeLimit is the percentage of a rate limit that lowers it not at all.
|
// wholeLimit is the percentage of a rate limit that lowers it not at all.
|
||||||
wholeLimit = 100
|
wholeLimit = 100
|
||||||
// clientsDecision is the CrowdSec decision list holding client alone.
|
// clientsDecision is the CrowdSec decision list holding client alone.
|
||||||
@@ -340,7 +340,7 @@ func newDiscriminatorInBubble(
|
|||||||
settings := map[string]string{tagDeny: "country=KP"}
|
settings := map[string]string{tagDeny: "country=KP"}
|
||||||
maps.Copy(settings, env)
|
maps.Copy(settings, env)
|
||||||
|
|
||||||
server, out, _ := newDiscriminator(t, unansweredGeoJSURL, time.Now, settings)
|
server, out, _ := newDiscriminator(t, silentGeoJSURL, time.Now, settings)
|
||||||
|
|
||||||
return server, out
|
return server, out
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -134,9 +134,9 @@ func (rq *request) banForLimit(now time.Time, hit ratelimit.Hit, status int) {
|
|||||||
|
|
||||||
switch hit.Kind {
|
switch hit.Kind {
|
||||||
case ratelimit.KindRequests:
|
case ratelimit.KindRequests:
|
||||||
notes.LimitPercent, notes.LimitPercentSetting = rq.limitPercent.logged()
|
notes.LimitPercent, notes.LimitPercentSetting = rq.rateLimitPercent.logged()
|
||||||
case ratelimit.KindBytes:
|
case ratelimit.KindBytes:
|
||||||
notes.LimitPercent, notes.LimitPercentSetting = rq.bytesPercent.logged()
|
notes.LimitPercent, notes.LimitPercentSetting = rq.bytesLimitPercent.logged()
|
||||||
}
|
}
|
||||||
|
|
||||||
if rq.h.config.Observe {
|
if rq.h.config.Observe {
|
||||||
|
|||||||
@@ -415,7 +415,7 @@ func startWithClock(
|
|||||||
|
|
||||||
// sender sends requests to rproxy one after another, each on a connection of its
|
// sender sends requests to rproxy one after another, each on a connection of its
|
||||||
// own, and checks each one's answer and log line. discriminatorAddr is where
|
// own, and checks each one's answer and log line. discriminatorAddr is where
|
||||||
// discriminator's own endpoints are, which adminRequest asks. Every request
|
// discriminator's own endpoints are, which ownEndpointRequest asks. Every request
|
||||||
// discriminator gets must come through the sender, since the log lines are
|
// discriminator gets must come through the sender, since the log lines are
|
||||||
// matched to them in order.
|
// matched to them in order.
|
||||||
type sender struct {
|
type sender struct {
|
||||||
|
|||||||
@@ -143,9 +143,9 @@ func TestBytesPercentSettingsTakeThePlaceOfTheOthersForByteLimits(t *testing.T)
|
|||||||
for _, tc := range []struct {
|
for _, tc := range []struct {
|
||||||
name string
|
name string
|
||||||
env map[string]string
|
env map[string]string
|
||||||
// limitPercent and bytesPercent are the log line's, as percentText
|
// rateLimitPercent and bytesLimitPercent are the log line's, as
|
||||||
// gives them, and limitHit is its limit_hit.
|
// percentText gives them, and limitHit is its limit_hit.
|
||||||
limitPercent, bytesPercent, limitHit string
|
rateLimitPercent, bytesLimitPercent, limitHit string
|
||||||
}{
|
}{
|
||||||
{
|
{
|
||||||
"lowering the byte limits alone",
|
"lowering the byte limits alone",
|
||||||
@@ -183,10 +183,10 @@ func TestBytesPercentSettingsTakeThePlaceOfTheOthersForByteLimits(t *testing.T)
|
|||||||
|
|
||||||
wantPercent(t, "rate_limit_percent",
|
wantPercent(t, "rate_limit_percent",
|
||||||
line.RateLimitPercent, line.RateLimitPercentSetting,
|
line.RateLimitPercent, line.RateLimitPercentSetting,
|
||||||
tc.limitPercent)
|
tc.rateLimitPercent)
|
||||||
wantPercent(t, "bytes_limit_percent",
|
wantPercent(t, "bytes_limit_percent",
|
||||||
line.BytesLimitPercent, line.BytesLimitPercentSetting,
|
line.BytesLimitPercent, line.BytesLimitPercentSetting,
|
||||||
tc.bytesPercent)
|
tc.bytesLimitPercent)
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -291,7 +291,7 @@ func TestClientWithoutAnAnswerInTimeHasTheUnknownLimitPercent(t *testing.T) {
|
|||||||
// In a synctest bubble, as TestRequestWaitsAsLongAsTheLookupTimeoutSays
|
// In a synctest bubble, as TestRequestWaitsAsLongAsTheLookupTimeoutSays
|
||||||
// says, with a GeoJS that never answers.
|
// says, with a GeoJS that never answers.
|
||||||
synctest.Test(t, func(t *testing.T) {
|
synctest.Test(t, func(t *testing.T) {
|
||||||
server, out, _ := newDiscriminator(t, unansweredGeoJSURL, time.Now,
|
server, out, _ := newDiscriminator(t, silentGeoJSURL, time.Now,
|
||||||
map[string]string{unknownLimitPercent: "0"})
|
map[string]string{unknownLimitPercent: "0"})
|
||||||
|
|
||||||
// Once the time the request waits for its answer is up, the client
|
// Once the time the request waits for its answer is up, the client
|
||||||
@@ -335,7 +335,7 @@ func TestRequestWaitsForItsLookupWhileABiasedThresholdIsSet(t *testing.T) {
|
|||||||
// In a synctest bubble, as TestRequestWaitsAsLongAsTheLookupTimeoutSays
|
// In a synctest bubble, as TestRequestWaitsAsLongAsTheLookupTimeoutSays
|
||||||
// says, with a GeoJS that never answers.
|
// says, with a GeoJS that never answers.
|
||||||
synctest.Test(t, func(t *testing.T) {
|
synctest.Test(t, func(t *testing.T) {
|
||||||
server, out, _ := newDiscriminator(t, unansweredGeoJSURL, time.Now,
|
server, out, _ := newDiscriminator(t, silentGeoJSURL, time.Now,
|
||||||
map[string]string{
|
map[string]string{
|
||||||
lookupTimeout: timeout.String(), tagLists: "vpn=" + vpnList, tc.setting: tc.value,
|
lookupTimeout: timeout.String(), tagLists: "vpn=" + vpnList, tc.setting: tc.value,
|
||||||
})
|
})
|
||||||
|
|||||||
@@ -26,7 +26,7 @@ func (byteLimits) check(_ context.Context, rq *request, now time.Time) *verdict
|
|||||||
}
|
}
|
||||||
|
|
||||||
counts, hit, over := rq.h.limiter.CountBytes(rq.h.clientGroup(rq.client), now,
|
counts, hit, over := rq.h.limiter.CountBytes(rq.h.clientGroup(rq.client), now,
|
||||||
rq.countedBytes(), rq.bytesPercent.percent)
|
rq.countedBytes(), rq.bytesLimitPercent.percent)
|
||||||
rq.line.Counts.MinuteBytes = counts.MinuteBytes
|
rq.line.Counts.MinuteBytes = counts.MinuteBytes
|
||||||
rq.line.Counts.HourBytes = counts.HourBytes
|
rq.line.Counts.HourBytes = counts.HourBytes
|
||||||
rq.line.Counts.DayBytes = counts.DayBytes
|
rq.line.Counts.DayBytes = counts.DayBytes
|
||||||
|
|||||||
@@ -10,6 +10,7 @@ import (
|
|||||||
"testing"
|
"testing"
|
||||||
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/alerts"
|
"sneak.berlin/go/smallwebwaf/internal/alerts"
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/discriminator"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
|
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||||
)
|
)
|
||||||
@@ -21,6 +22,7 @@ const (
|
|||||||
wafDisabledRules = "SWW_WAF_DISABLED_RULES"
|
wafDisabledRules = "SWW_WAF_DISABLED_RULES"
|
||||||
wafExemptPaths = "SWW_WAF_EXEMPT_PATHS"
|
wafExemptPaths = "SWW_WAF_EXEMPT_PATHS"
|
||||||
wafBodyLimit = "SWW_WAF_BODY_LIMIT"
|
wafBodyLimit = "SWW_WAF_BODY_LIMIT"
|
||||||
|
apps = "SWW_APPS"
|
||||||
block = "block"
|
block = "block"
|
||||||
detect = "detect"
|
detect = "detect"
|
||||||
)
|
)
|
||||||
@@ -290,6 +292,27 @@ func TestDisabledRulesSwitchOffWhatGiteaWouldBeRefused(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestGiteasFixesApplyOnlyToAHostnameThatServesGitea(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
// Only gitea's fixes let a file name through in path, a parameter
|
||||||
|
// gitea sends file names in.
|
||||||
|
const fileName = "/?path=.gitignore"
|
||||||
|
|
||||||
|
s := startDiscriminator(t, "", map[string]string{
|
||||||
|
wafMode: block, apps: siteA + ":gitea",
|
||||||
|
})
|
||||||
|
|
||||||
|
wantDisposition(t, do(t, s.question(siteA, client, fileName)),
|
||||||
|
http.StatusOK, discriminator.DispositionAllow)
|
||||||
|
wantDisposition(t, do(t, s.question(siteB, client, fileName)),
|
||||||
|
http.StatusForbidden, discriminator.DispositionDeny)
|
||||||
|
|
||||||
|
lines := s.out.requestLines(t, 2)
|
||||||
|
wantLine(t, lines[0], http.StatusOK, requestlog.ActionForward)
|
||||||
|
wantLine(t, lines[1], http.StatusForbidden, requestlog.ActionBlockedByCoreRuleSet)
|
||||||
|
}
|
||||||
|
|
||||||
func TestAttackInAFormBodyIsRefusedOnlyWhileBodiesAreRead(t *testing.T) {
|
func TestAttackInAFormBodyIsRefusedOnlyWhileBodiesAreRead(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
|
|||||||
@@ -183,7 +183,7 @@ func (h *handler) answerQuestion(w http.ResponseWriter, r *http.Request) {
|
|||||||
rq.line.DiscriminatorLine = requestlog.DiscriminatorLineDecision
|
rq.line.DiscriminatorLine = requestlog.DiscriminatorLineDecision
|
||||||
refused := rq.check(r.Context())
|
refused := rq.check(r.Context())
|
||||||
rq.checked = time.Now()
|
rq.checked = time.Now()
|
||||||
underOwnPrefix := strings.HasPrefix(r.URL.Path, adminPrefix)
|
underOwnPrefix := strings.HasPrefix(r.URL.Path, ownEndpointPrefix)
|
||||||
|
|
||||||
if refused == nil && !underOwnPrefix {
|
if refused == nil && !underOwnPrefix {
|
||||||
// In observe mode, what enforce mode would have refused counts here.
|
// In observe mode, what enforce mode would have refused counts here.
|
||||||
|
|||||||
@@ -21,7 +21,8 @@ func TestHealthEndpointCountsTowardNoLimit(t *testing.T) {
|
|||||||
)
|
)
|
||||||
|
|
||||||
for range healthChecks {
|
for range healthChecks {
|
||||||
got := s.adminRequest("", http.MethodGet, discriminator.HealthPath, "", http.StatusOK)
|
got := s.ownEndpointRequest("", http.MethodGet, discriminator.HealthPath, "",
|
||||||
|
http.StatusOK)
|
||||||
|
|
||||||
if string(got.body) != "ok\n" || got.header.Get("Content-Type") != contentType {
|
if string(got.body) != "ok\n" || got.header.Get("Content-Type") != contentType {
|
||||||
t.Errorf("health endpoint answered %q with Content-Type %q, want ok "+
|
t.Errorf("health endpoint answered %q with Content-Type %q, want ok "+
|
||||||
|
|||||||
@@ -100,8 +100,8 @@ func TestOwnEndpointsAreNotInTheHistory(t *testing.T) {
|
|||||||
|
|
||||||
s, _, server := startWithClock(t, "", map[string]string{metricsToken: token})
|
s, _, server := startWithClock(t, "", map[string]string{metricsToken: token})
|
||||||
|
|
||||||
s.adminRequest("", http.MethodGet, discriminator.HealthPath, "", http.StatusOK)
|
s.ownEndpointRequest("", http.MethodGet, discriminator.HealthPath, "", http.StatusOK)
|
||||||
s.adminRequest("", http.MethodGet, discriminator.MetricsPath, "",
|
s.ownEndpointRequest("", http.MethodGet, discriminator.MetricsPath, "",
|
||||||
http.StatusUnauthorized)
|
http.StatusUnauthorized)
|
||||||
s.scrape()
|
s.scrape()
|
||||||
|
|
||||||
|
|||||||
@@ -385,7 +385,7 @@ func TestRequestWaitsAsLongAsTheLookupTimeoutSays(t *testing.T) {
|
|||||||
// is never reached.
|
// is never reached.
|
||||||
const timeout = 3 * time.Second
|
const timeout = 3 * time.Second
|
||||||
|
|
||||||
server, out, _ := newDiscriminator(t, unansweredGeoJSURL, time.Now,
|
server, out, _ := newDiscriminator(t, silentGeoJSURL, time.Now,
|
||||||
map[string]string{
|
map[string]string{
|
||||||
lookupTimeout: timeout.String(),
|
lookupTimeout: timeout.String(),
|
||||||
allowedCountries: "DE",
|
allowedCountries: "DE",
|
||||||
@@ -405,25 +405,25 @@ func TestRequestWaitsAsLongAsTheLookupTimeoutSays(t *testing.T) {
|
|||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
// unansweredGeoJSURL is where a GeoJS that never answers is asked: a
|
// silentGeoJSURL is where a GeoJS that never answers is asked: a
|
||||||
// request to it waits, without the network, until it is abandoned.
|
// request to it waits, without the network, until it is abandoned.
|
||||||
// TestMain registers it with Go's default transport, through which GeoJS
|
// TestMain registers it with Go's default transport, through which GeoJS
|
||||||
// is asked.
|
// is asked.
|
||||||
const unansweredGeoJSURL = "unanswered://geojs/v1/ip/geo.json"
|
const silentGeoJSURL = "silent://geojs/v1/ip/geo.json"
|
||||||
|
|
||||||
func TestMain(m *testing.M) {
|
func TestMain(m *testing.M) {
|
||||||
transport, _ := http.DefaultTransport.(*http.Transport)
|
transport, _ := http.DefaultTransport.(*http.Transport)
|
||||||
transport.RegisterProtocol("unanswered", unansweredGeoJS{})
|
transport.RegisterProtocol("silent", silentGeoJS{})
|
||||||
transport.RegisterProtocol("abuseipdb", abuseIPDBStandIn{})
|
transport.RegisterProtocol("abuseipdb", abuseIPDBStandIn{})
|
||||||
|
|
||||||
m.Run()
|
m.Run()
|
||||||
}
|
}
|
||||||
|
|
||||||
// unansweredGeoJS is the GeoJS at unansweredGeoJSURL.
|
// silentGeoJS is the GeoJS at silentGeoJSURL.
|
||||||
type unansweredGeoJS struct{}
|
type silentGeoJS struct{}
|
||||||
|
|
||||||
// RoundTrip waits until req is abandoned.
|
// RoundTrip waits until req is abandoned.
|
||||||
func (unansweredGeoJS) RoundTrip(req *http.Request) (*http.Response, error) {
|
func (silentGeoJS) RoundTrip(req *http.Request) (*http.Response, error) {
|
||||||
<-req.Context().Done()
|
<-req.Context().Done()
|
||||||
|
|
||||||
return nil, req.Context().Err()
|
return nil, req.Context().Err()
|
||||||
|
|||||||
@@ -32,7 +32,7 @@ func TestMetricsAreOffWhileTheTokenIsUnset(t *testing.T) {
|
|||||||
|
|
||||||
// An empty token does not match the unset one either.
|
// An empty token does not match the unset one either.
|
||||||
for _, authorization := range []string{bearer, "Bearer ", ""} {
|
for _, authorization := range []string{bearer, "Bearer ", ""} {
|
||||||
s.adminRequest(authorization, http.MethodGet, discriminator.MetricsPath, "",
|
s.ownEndpointRequest(authorization, http.MethodGet, discriminator.MetricsPath, "",
|
||||||
http.StatusNotFound)
|
http.StatusNotFound)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -60,7 +60,7 @@ func TestMetricsNeedTheTokenAndOtherPathsAreNotFound(t *testing.T) {
|
|||||||
{http.MethodGet, "/_sww/bans", bearer, http.StatusNotFound},
|
{http.MethodGet, "/_sww/bans", bearer, http.StatusNotFound},
|
||||||
{http.MethodPost, discriminator.HealthPath, "", http.StatusNotFound},
|
{http.MethodPost, discriminator.HealthPath, "", http.StatusNotFound},
|
||||||
} {
|
} {
|
||||||
got := s.adminRequest(tc.authorization, tc.method, tc.path, "", tc.status)
|
got := s.ownEndpointRequest(tc.authorization, tc.method, tc.path, "", tc.status)
|
||||||
|
|
||||||
if tc.status == http.StatusUnauthorized &&
|
if tc.status == http.StatusUnauthorized &&
|
||||||
got.header.Get("WWW-Authenticate") != "Bearer" {
|
got.header.Get("WWW-Authenticate") != "Bearer" {
|
||||||
@@ -381,8 +381,8 @@ func scrape(t *testing.T, addr string) string {
|
|||||||
func (s *sender) scrape() string {
|
func (s *sender) scrape() string {
|
||||||
s.t.Helper()
|
s.t.Helper()
|
||||||
|
|
||||||
return string(s.adminRequest(bearer, http.MethodGet, discriminator.MetricsPath, "",
|
return string(s.ownEndpointRequest(bearer, http.MethodGet, discriminator.MetricsPath,
|
||||||
http.StatusOK).body)
|
"", http.StatusOK).body)
|
||||||
}
|
}
|
||||||
|
|
||||||
// metric returns the value of series in metrics, which are in the
|
// metric returns the value of series in metrics, which are in the
|
||||||
|
|||||||
@@ -25,9 +25,9 @@ import (
|
|||||||
"sneak.berlin/go/smallwebwaf/internal/rules"
|
"sneak.berlin/go/smallwebwaf/internal/rules"
|
||||||
)
|
)
|
||||||
|
|
||||||
// adminPrefix starts the path of each of discriminator's own endpoints. The app never
|
// ownEndpointPrefix starts the path of each of discriminator's own endpoints. The app
|
||||||
// sees a path under it.
|
// never sees a path under it.
|
||||||
const adminPrefix = "/_sww/"
|
const ownEndpointPrefix = "/_sww/"
|
||||||
|
|
||||||
// HealthPath is the health endpoint of discriminator and of rproxy, which the
|
// HealthPath is the health endpoint of discriminator and of rproxy, which the
|
||||||
// container's health check asks.
|
// container's health check asks.
|
||||||
@@ -221,7 +221,8 @@ func newReputation(
|
|||||||
// NewCoreRuleSet returns the Core Rule Set at SWW_WAF_PARANOIA_LEVEL,
|
// NewCoreRuleSet returns the Core Rule Set at SWW_WAF_PARANOIA_LEVEL,
|
||||||
// without the rules SWW_WAF_DISABLED_RULES switches off, reading bodies
|
// without the rules SWW_WAF_DISABLED_RULES switches off, reading bodies
|
||||||
// up to SWW_WAF_BODY_LIMIT, with the exclusions in
|
// up to SWW_WAF_BODY_LIMIT, with the exclusions in
|
||||||
// SWW_CORE_RULE_SET_EXCLUSIONS_FILE, or nil while SWW_WAF_MODE is off. It
|
// SWW_CORE_RULE_SET_EXCLUSIONS_FILE and the fixes of the apps in SWW_APPS,
|
||||||
|
// or nil while SWW_WAF_MODE is off. It
|
||||||
// reads at most QuestionBodyMaxBytes less one byte: a question's body of
|
// reads at most QuestionBodyMaxBytes less one byte: a question's body of
|
||||||
// QuestionBodyMaxBytes may be cut short, and the Core Rule Set does not read a
|
// QuestionBodyMaxBytes may be cut short, and the Core Rule Set does not read a
|
||||||
// JSON or XML body over its limit. An exclusions file that coreruleset.New refuses is
|
// JSON or XML body over its limit. An exclusions file that coreruleset.New refuses is
|
||||||
@@ -236,6 +237,7 @@ func NewCoreRuleSet(cfg *config.Config) (*coreruleset.CoreRuleSet, error) {
|
|||||||
ParanoiaLevel: cfg.WAFParanoiaLevel, DisabledRules: cfg.WAFDisabledRules,
|
ParanoiaLevel: cfg.WAFParanoiaLevel, DisabledRules: cfg.WAFDisabledRules,
|
||||||
BodyLimit: min(cfg.WAFBodyLimit, QuestionBodyMaxBytes-1),
|
BodyLimit: min(cfg.WAFBodyLimit, QuestionBodyMaxBytes-1),
|
||||||
ExclusionsFile: cfg.CoreRuleSetExclusionsFile,
|
ExclusionsFile: cfg.CoreRuleSetExclusionsFile,
|
||||||
|
Apps: cfg.Apps,
|
||||||
})
|
})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, fmt.Errorf("SWW_CORE_RULE_SET_EXCLUSIONS_FILE: %w", err)
|
return nil, fmt.Errorf("SWW_CORE_RULE_SET_EXCLUSIONS_FILE: %w", err)
|
||||||
|
|||||||
@@ -28,12 +28,14 @@ func (rateLimits) check(_ context.Context, rq *request, now time.Time) *verdict
|
|||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
rq.limitPercent, rq.bytesPercent = rq.limitPercentages()
|
rq.rateLimitPercent, rq.bytesLimitPercent = rq.limitPercentages()
|
||||||
rq.line.RateLimitPercent, rq.line.RateLimitPercentSetting = rq.limitPercent.logged()
|
rq.line.RateLimitPercent, rq.line.RateLimitPercentSetting =
|
||||||
rq.line.BytesLimitPercent, rq.line.BytesLimitPercentSetting = rq.bytesPercent.logged()
|
rq.rateLimitPercent.logged()
|
||||||
|
rq.line.BytesLimitPercent, rq.line.BytesLimitPercentSetting =
|
||||||
|
rq.bytesLimitPercent.logged()
|
||||||
|
|
||||||
counts, hit, over := rq.h.limiter.Count(rq.h.clientGroup(rq.client), now,
|
counts, hit, over := rq.h.limiter.Count(rq.h.clientGroup(rq.client), now,
|
||||||
rq.limitPercent.percent)
|
rq.rateLimitPercent.percent)
|
||||||
rq.line.Counts = counts
|
rq.line.Counts = counts
|
||||||
|
|
||||||
if !over {
|
if !over {
|
||||||
|
|||||||
@@ -256,9 +256,9 @@ func TestASNLimitPercentFileCountsAsTheSettingDoesTheLowerWinning(t *testing.T)
|
|||||||
name string
|
name string
|
||||||
env map[string]string
|
env map[string]string
|
||||||
file string
|
file string
|
||||||
// limitPercent and bytesPercent are the upload's, as percentText
|
// rateLimitPercent and bytesLimitPercent are the upload's, as
|
||||||
// gives them.
|
// percentText gives them.
|
||||||
limitPercent, bytesPercent string
|
rateLimitPercent, bytesLimitPercent string
|
||||||
}{
|
}{
|
||||||
{"the file's alone", nil, asnDEQuarter, fromURL, fromURL},
|
{"the file's alone", nil, asnDEQuarter, fromURL, fromURL},
|
||||||
{
|
{
|
||||||
@@ -293,10 +293,10 @@ func TestASNLimitPercentFileCountsAsTheSettingDoesTheLowerWinning(t *testing.T)
|
|||||||
line := s.uploadFrom(fromDE)
|
line := s.uploadFrom(fromDE)
|
||||||
wantPercent(t, "rate_limit_percent",
|
wantPercent(t, "rate_limit_percent",
|
||||||
line.RateLimitPercent, line.RateLimitPercentSetting,
|
line.RateLimitPercent, line.RateLimitPercentSetting,
|
||||||
tc.limitPercent)
|
tc.rateLimitPercent)
|
||||||
wantPercent(t, "bytes_limit_percent",
|
wantPercent(t, "bytes_limit_percent",
|
||||||
line.BytesLimitPercent, line.BytesLimitPercentSetting,
|
line.BytesLimitPercent, line.BytesLimitPercentSetting,
|
||||||
tc.bytesPercent)
|
tc.bytesLimitPercent)
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -48,11 +48,11 @@ type decision struct {
|
|||||||
// so that not waiting never lowers its client's limits.
|
// so that not waiting never lowers its client's limits.
|
||||||
skippedLookupWait bool
|
skippedLookupWait bool
|
||||||
// counted is true for a request the rate limits counted, whose bytes
|
// counted is true for a request the rate limits counted, whose bytes
|
||||||
// the byte limits count once it has ended. limitPercent and
|
// the byte limits count once it has ended. rateLimitPercent and
|
||||||
// bytesPercent are then its client's limit percentages for the rate
|
// bytesLimitPercent are then its client's limit percentages for the rate
|
||||||
// limits and for the byte limits.
|
// limits and for the byte limits.
|
||||||
counted bool
|
counted bool
|
||||||
limitPercent, bytesPercent percentage
|
rateLimitPercent, bytesLimitPercent percentage
|
||||||
// attack is true for a request that matched a ban rule or asked for a
|
// attack is true for a request that matched a ban rule or asked for a
|
||||||
// trap path, blockedByBlockRule for one a block rule refused, and
|
// trap path, blockedByBlockRule for one a block rule refused, and
|
||||||
// blockedByCoreRuleSet for one the Core Rule Set refused, each an offence
|
// blockedByCoreRuleSet for one the Core Rule Set refused, each an offence
|
||||||
|
|||||||
@@ -362,7 +362,7 @@ func TestWithoutWaitTheRequestGoesOnAtOnceAndTheAnswerIsGivenWhenItComes(
|
|||||||
mu.Unlock()
|
mu.Unlock()
|
||||||
|
|
||||||
wantCountry(t, g, client, germany)
|
wantCountry(t, g, client, germany)
|
||||||
wantUnanswered(t, m, 0)
|
wantRequestsWithoutGeoJSAnswer(t, m, 0)
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -596,7 +596,7 @@ func TestClientsWithoutAnAnswerAreCounted(t *testing.T) {
|
|||||||
// GeoJS fails, so the first client goes without an answer, and GeoJS
|
// GeoJS fails, so the first client goes without an answer, and GeoJS
|
||||||
// is left alone for a second, which does not pass in this test.
|
// is left alone for a second, which does not pass in this test.
|
||||||
wantCountry(t, g, clients(), "")
|
wantCountry(t, g, clients(), "")
|
||||||
wantUnanswered(t, m, 1)
|
wantRequestsWithoutGeoJSAnswer(t, m, 1)
|
||||||
|
|
||||||
// Meanwhile each new client goes without one at once, while there is
|
// Meanwhile each new client goes without one at once, while there is
|
||||||
// room for it among the 10,000 that may wait.
|
// room for it among the 10,000 that may wait.
|
||||||
@@ -604,11 +604,11 @@ func TestClientsWithoutAnAnswerAreCounted(t *testing.T) {
|
|||||||
wantCountry(t, g, clients(), "")
|
wantCountry(t, g, clients(), "")
|
||||||
}
|
}
|
||||||
|
|
||||||
wantUnanswered(t, m, 10000)
|
wantRequestsWithoutGeoJSAnswer(t, m, 10000)
|
||||||
|
|
||||||
// One more, for which there is no room, goes without one too.
|
// One more, for which there is no room, goes without one too.
|
||||||
wantCountry(t, g, clients(), "")
|
wantCountry(t, g, clients(), "")
|
||||||
wantUnanswered(t, m, 10001)
|
wantRequestsWithoutGeoJSAnswer(t, m, 10001)
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -841,9 +841,9 @@ func wantAsked(t *testing.T, geojs *standIn, i int, want ...string) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// wantUnanswered checks how many requests m counts as having gone without
|
// wantRequestsWithoutGeoJSAnswer checks how many requests m counts as having
|
||||||
// an answer from GeoJS.
|
// gone without an answer from GeoJS.
|
||||||
func wantUnanswered(t *testing.T, m *metrics.Metrics, want float64) {
|
func wantRequestsWithoutGeoJSAnswer(t *testing.T, m *metrics.Metrics, want float64) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
got := testutil.ToFloat64(m.RequestsWithoutGeoJSAnswer)
|
got := testutil.ToFloat64(m.RequestsWithoutGeoJSAnswer)
|
||||||
|
|||||||
@@ -359,7 +359,7 @@ func (m *Metrics) AddAlerts(queue *alerts.Queue) {
|
|||||||
Help: "Requests to the destination that failed.",
|
Help: "Requests to the destination that failed.",
|
||||||
ConstLabels: destination,
|
ConstLabels: destination,
|
||||||
}, func() float64 {
|
}, func() float64 {
|
||||||
return float64(queue.Counts(name).Failed)
|
return float64(queue.Counts(name).SendFailures)
|
||||||
}),
|
}),
|
||||||
prometheus.NewCounterFunc(prometheus.CounterOpts{
|
prometheus.NewCounterFunc(prometheus.CounterOpts{
|
||||||
Name: "sww_alerts_suppressed_total",
|
Name: "sww_alerts_suppressed_total",
|
||||||
|
|||||||
@@ -45,9 +45,9 @@ const (
|
|||||||
idleTimeout = 120 * time.Second
|
idleTimeout = 120 * time.Second
|
||||||
)
|
)
|
||||||
|
|
||||||
// adminPrefix starts the path of every request for sww itself, which
|
// ownEndpointPrefix starts the path of every request for sww itself, which
|
||||||
// never reaches the app.
|
// never reaches the app.
|
||||||
const adminPrefix = "/_sww/"
|
const ownEndpointPrefix = "/_sww/"
|
||||||
|
|
||||||
// allowedMethods lists the methods rproxy passes on, as an Allow header gives
|
// allowedMethods lists the methods rproxy passes on, as an Allow header gives
|
||||||
// them. It answers any other method 405 itself.
|
// them. It answers any other method 405 itself.
|
||||||
@@ -211,8 +211,8 @@ func (h *handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// passToApp passes the request on to the app, unless a limit or its path keeps
|
// passToApp passes the request on to the app, unless a limit or its path keeps
|
||||||
// it from the app. discriminator answers a path under adminPrefix itself, so such a
|
// it from the app. discriminator answers a path under ownEndpointPrefix itself, so
|
||||||
// request gets here only when SWRP_DEFAULT_DISPOSITION allowed it. ctx is the
|
// such a request gets here only when SWRP_DEFAULT_DISPOSITION allowed it. ctx is the
|
||||||
// request's own context.
|
// request's own context.
|
||||||
func (rq *request) passToApp(ctx context.Context) {
|
func (rq *request) passToApp(ctx context.Context) {
|
||||||
maxBytes := rq.h.config.MaxRequestSize
|
maxBytes := rq.h.config.MaxRequestSize
|
||||||
@@ -220,7 +220,7 @@ func (rq *request) passToApp(ctx context.Context) {
|
|||||||
switch {
|
switch {
|
||||||
case rq.refused.Load() != 0 || (maxBytes > 0 && rq.in.ContentLength > maxBytes):
|
case rq.refused.Load() != 0 || (maxBytes > 0 && rq.in.ContentLength > maxBytes):
|
||||||
rq.answer(http.StatusRequestEntityTooLarge)
|
rq.answer(http.StatusRequestEntityTooLarge)
|
||||||
case strings.HasPrefix(rq.in.URL.Path, adminPrefix):
|
case strings.HasPrefix(rq.in.URL.Path, ownEndpointPrefix):
|
||||||
rq.answer(http.StatusNotFound)
|
rq.answer(http.StatusNotFound)
|
||||||
default:
|
default:
|
||||||
rq.forward(ctx)
|
rq.forward(ctx)
|
||||||
|
|||||||
@@ -64,7 +64,7 @@ var (
|
|||||||
errCause = errors.New("is not limit, attack, admin or crowdsec")
|
errCause = errors.New("is not limit, attack, admin or crowdsec")
|
||||||
errExpires = errors.New("is not after its start")
|
errExpires = errors.New("is not after its start")
|
||||||
errDestination = errors.New("is not webhook, slack or ntfy")
|
errDestination = errors.New("is not webhook, slack or ntfy")
|
||||||
errScope = errors.New("is not client, net, asn, total or watch")
|
errScope = errors.New("is not client, net, asn, total or watched_net")
|
||||||
errWaitingList = errors.New(`waiting is a list, but now lists the alerts by ` +
|
errWaitingList = errors.New(`waiting is a list, but now lists the alerts by ` +
|
||||||
`destination: put the list under "webhook", as "waiting": {"webhook": [...]}, ` +
|
`destination: put the list under "webhook", as "waiting": {"webhook": [...]}, ` +
|
||||||
`or remove the file`)
|
`or remove the file`)
|
||||||
@@ -858,8 +858,8 @@ func (f *alertsFile) check([]byte) error {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// checkAnomalyCounters refuses an anomaly counter whose scope is not
|
// checkAnomalyCounters refuses an anomaly counter whose scope is not
|
||||||
// client, net, asn, total or watch, most likely misspelt, and one without
|
// client, net, asn, total or watched_net, most likely misspelt, and one
|
||||||
// a field it needs, as missingFromCounter tells.
|
// without a field it needs, as missingFromCounter tells.
|
||||||
func checkAnomalyCounters(counters []anomaly.Counter) error {
|
func checkAnomalyCounters(counters []anomaly.Counter) error {
|
||||||
for i, counter := range counters {
|
for i, counter := range counters {
|
||||||
if !slices.Contains(anomaly.Scopes(), counter.Scope) {
|
if !slices.Contains(anomaly.Scopes(), counter.Scope) {
|
||||||
@@ -879,9 +879,9 @@ func checkAnomalyCounters(counters []anomaly.Counter) error {
|
|||||||
// missingFromCounter returns the first field counter, an anomaly counter,
|
// missingFromCounter returns the first field counter, an anomaly counter,
|
||||||
// needs and has not, or "" when it has them all: what tells it from the
|
// needs and has not, or "" when it has them all: what tells it from the
|
||||||
// others in its scope, without which it would never be counted again, the
|
// others in its scope, without which it would never be counted again, the
|
||||||
// netblock of a client, net or watch counter, the AS number of an asn one
|
// netblock of a client, net or watched_net counter, the AS number of an
|
||||||
// and the name of a watch one; and the start of a window in which it has
|
// asn one and the name of a watched_net one; and the start of a window in
|
||||||
// requests or bytes, without which they would be dropped.
|
// which it has requests or bytes, without which they would be dropped.
|
||||||
func missingFromCounter(counter anomaly.Counter) string {
|
func missingFromCounter(counter anomaly.Counter) string {
|
||||||
scope := counter.Scope
|
scope := counter.Scope
|
||||||
|
|
||||||
|
|||||||
@@ -534,7 +534,7 @@ func TestFileThatDoesNotParseStopsTheStart(t *testing.T) {
|
|||||||
`{"version": 1, "anomaly_counters": [{"scope": "total"}, ` +
|
`{"version": 1, "anomaly_counters": [{"scope": "total"}, ` +
|
||||||
`{"scope": "nett", "netblock": "203.0.113.0/24"}]}`,
|
`{"scope": "nett", "netblock": "203.0.113.0/24"}]}`,
|
||||||
`: anomaly_counters entry 2's scope "nett" is not client, net, asn, total ` +
|
`: anomaly_counters entry 2's scope "nett" is not client, net, asn, total ` +
|
||||||
`or watch`,
|
`or watched_net`,
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"a copy of a list with a line that does not read", reputationJSON,
|
"a copy of a list with a line that does not read", reputationJSON,
|
||||||
|
|||||||
Reference in new issue
Block a user