|
|
@@ -100,12 +100,10 @@ to them.
|
|
|
|
## What happens to a request
|
|
|
|
## What happens to a request
|
|
|
|
|
|
|
|
|
|
|
|
The steps run in this order. The first step that produces a final answer ends
|
|
|
|
The steps run in this order. The first step that produces a final answer ends
|
|
|
|
the checks; step 15 runs for every request but the health check. The size and
|
|
|
|
the checks; step 15 runs for every request. The size and time limits apply to
|
|
|
|
time limits apply to the whole exchange. Each step names the section that holds
|
|
|
|
the whole exchange. Each step names the section that holds its rules.
|
|
|
|
its rules.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1. `GET /_smallwebwaf/healthz` is answered at once, before any check, so a
|
|
|
|
1. `GET /_smallwebwaf/healthz` is answered at once, before any check.
|
|
|
|
health checker never needs an exemption and is never refused.
|
|
|
|
|
|
|
|
2. The client is identified.
|
|
|
|
2. The client is identified.
|
|
|
|
- If the TCP peer is inside `SWWAF_TRUSTED_PROXIES`, `X-Forwarded-For` is
|
|
|
|
- If the TCP peer is inside `SWWAF_TRUSTED_PROXIES`, `X-Forwarded-For` is
|
|
|
|
walked from the right, and the first address outside those netblocks is
|
|
|
|
walked from the right, and the first address outside those netblocks is
|
|
|
@@ -118,9 +116,9 @@ its rules.
|
|
|
|
around its address, a /64 by default, since one abuser usually controls a
|
|
|
|
around its address, a /64 by default, since one abuser usually controls a
|
|
|
|
whole /64. A client is therefore one IPv4 address or one IPv6 group.
|
|
|
|
whole /64. A client is therefore one IPv4 address or one IPv6 group.
|
|
|
|
3. The static lists. A client in `SWWAF_ALLOW_NETS` skips steps 4 to 12 and the
|
|
|
|
3. The static lists. A client in `SWWAF_ALLOW_NETS` skips steps 4 to 12 and the
|
|
|
|
byte limits and the error burst at step 15. It still needs a token at
|
|
|
|
byte limits and the error burst at step 15. It is still counted for the
|
|
|
|
step 13. A client in `SWWAF_DENY_NETS` is refused; a client in both lists is
|
|
|
|
anomaly thresholds, and still needs a token at step 13. A client in
|
|
|
|
let through.
|
|
|
|
`SWWAF_DENY_NETS` is refused; a client in both lists is let through.
|
|
|
|
4. Bans. A client under a ban is refused (see "Bans").
|
|
|
|
4. Bans. A client under a ban is refused (see "Bans").
|
|
|
|
5. The lookup, unless `SWWAF_LOOKUP_SOURCE` is `off` (see "Lookup of AS number
|
|
|
|
5. The lookup, unless `SWWAF_LOOKUP_SOURCE` is `off` (see "Lookup of AS number
|
|
|
|
and country"). The request goes on at once. It waits for the first GeoJS
|
|
|
|
and country"). The request goes on at once. It waits for the first GeoJS
|
|
|
@@ -136,6 +134,8 @@ its rules.
|
|
|
|
gets `SWWAF_BLOCKLIST_ACTION`. A client on the CrowdSec decision list is
|
|
|
|
gets `SWWAF_BLOCKLIST_ACTION`. A client on the CrowdSec decision list is
|
|
|
|
banned. A client listed by a DNS blocklist, or scored at least
|
|
|
|
banned. A client listed by a DNS blocklist, or scored at least
|
|
|
|
`SWWAF_ABUSEIPDB_MIN_SCORE` by AbuseIPDB, gets `SWWAF_REPUTATION_ACTION`.
|
|
|
|
`SWWAF_ABUSEIPDB_MIN_SCORE` by AbuseIPDB, gets `SWWAF_REPUTATION_ACTION`.
|
|
|
|
|
|
|
|
With no cached verdict, a query is queued in the background and the request
|
|
|
|
|
|
|
|
goes on, so a first request is never delayed by a reputation query.
|
|
|
|
8. The client's limit percentage (see "Biased thresholds").
|
|
|
|
8. The client's limit percentage (see "Biased thresholds").
|
|
|
|
9. Rate limiting (see "Request rate limits, per client"). A request over a limit
|
|
|
|
9. Rate limiting (see "Request rate limits, per client"). A request over a limit
|
|
|
|
times the percentage is refused and bans the client.
|
|
|
|
times the percentage is refused and bans the client.
|
|
|
@@ -164,8 +164,8 @@ its rules.
|
|
|
|
thresholds are evaluated.
|
|
|
|
thresholds are evaluated.
|
|
|
|
|
|
|
|
|
|
|
|
In `SWWAF_MODE=observe`, every refusal above is logged as what would have
|
|
|
|
In `SWWAF_MODE=observe`, every refusal above is logged as what would have
|
|
|
|
happened, nothing is refused, and `smallwebwaf` makes no ban of its own. The
|
|
|
|
happened, nothing is refused, and no ban is made. The size and time limits and
|
|
|
|
size and time limits and the token check still apply.
|
|
|
|
the token check still apply.
|
|
|
|
|
|
|
|
|
|
|
|
## Counting method
|
|
|
|
## Counting method
|
|
|
|
|
|
|
|
|
|
|
@@ -183,8 +183,8 @@ size and time limits and the token check still apply.
|
|
|
|
GeoJS answers, the reputation verdicts and scores, the DNS queries under way
|
|
|
|
GeoJS answers, the reputation verdicts and scores, the DNS queries under way
|
|
|
|
and the anomaly counters have fixed bounds, listed under "Fixed limits" in
|
|
|
|
and the anomaly counters have fixed bounds, listed under "Fixed limits" in
|
|
|
|
`docs/settings.md`. Past its bound, the GeoJS answer or anomaly counter used
|
|
|
|
`docs/settings.md`. Past its bound, the GeoJS answer or anomaly counter used
|
|
|
|
longest ago goes first, and the verdict or score fetched longest ago. Past the
|
|
|
|
longest ago goes first, and the verdict or score fetched longest ago. Past
|
|
|
|
bound on DNS queries under way, a zone is asked at the client's next request.
|
|
|
|
1,000 DNS queries under way, a zone is asked at the client's next request.
|
|
|
|
|
|
|
|
|
|
|
|
## Bans
|
|
|
|
## Bans
|
|
|
|
|
|
|
|
|
|
|
@@ -193,7 +193,8 @@ Until a ban ends, every request from its netblock is refused with
|
|
|
|
`SWWAF_BAN_SCOPE_V4_PREFIX`, or its IPv6 group. At the defaults a ban covers one
|
|
|
|
`SWWAF_BAN_SCOPE_V4_PREFIX`, or its IPv6 group. At the defaults a ban covers one
|
|
|
|
address or one /64, so a permanent ban does not reach neighbours who did
|
|
|
|
address or one /64, so a permanent ban does not reach neighbours who did
|
|
|
|
nothing. Requests refused under a ban are not counted against limits. They are
|
|
|
|
nothing. Requests refused under a ban are not counted against limits. They are
|
|
|
|
counted in the client's history and in the ban's notes.
|
|
|
|
counted in the client's history and in the ban's notes. A banned client is not
|
|
|
|
|
|
|
|
looked up.
|
|
|
|
|
|
|
|
|
|
|
|
An offence is a request `smallwebwaf` holds against the client. It is one of:
|
|
|
|
An offence is a request `smallwebwaf` holds against the client. It is one of:
|
|
|
|
|
|
|
|
|
|
|
@@ -215,12 +216,9 @@ to a ban, each by its own rule:
|
|
|
|
permanent this way.
|
|
|
|
permanent this way.
|
|
|
|
- Once that ban has run out, the netblock is served like any other. Its next
|
|
|
|
- Once that ban has run out, the netblock is served like any other. Its next
|
|
|
|
clear sign of attack bans it permanently at once.
|
|
|
|
clear sign of attack bans it permanently at once.
|
|
|
|
- A broken limit, one of:
|
|
|
|
- A broken limit: a request over a rate limit, a response that takes the
|
|
|
|
- a request over a rate limit;
|
|
|
|
client's byte total past a byte limit, or more than
|
|
|
|
- a response that takes the client's byte total past a byte limit;
|
|
|
|
`SWWAF_ERROR_BURST_THRESHOLD` refusals within a minute.
|
|
|
|
- more than `SWWAF_ERROR_BURST_THRESHOLD` refusals within a minute.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
The bans for a broken limit:
|
|
|
|
|
|
|
|
- The first such ban lasts `SWWAF_LIMIT_BAN_DURATION` (default `1h`), with
|
|
|
|
- The first such ban lasts `SWWAF_LIMIT_BAN_DURATION` (default `1h`), with
|
|
|
|
the cause `limit`. So does one that comes more than
|
|
|
|
the cause `limit`. So does one that comes more than
|
|
|
|
`SWWAF_LIMIT_BAN_REPEAT_WINDOW` (default `24h`) after the last such ban
|
|
|
|
`SWWAF_LIMIT_BAN_REPEAT_WINDOW` (default `24h`) after the last such ban
|
|
|
@@ -236,7 +234,7 @@ to a ban, each by its own rule:
|
|
|
|
A Core Rule Set match refuses only the request it matched, with `403` and
|
|
|
|
A Core Rule Set match refuses only the request it matched, with `403` and
|
|
|
|
without a ban, because the Core Rule Set has false positives. A `block` rule
|
|
|
|
without a ban, because the Core Rule Set has false positives. A `block` rule
|
|
|
|
does the same. Those refusals count toward the error burst, so a client that
|
|
|
|
does the same. Those refusals count toward the error burst, so a client that
|
|
|
|
keeps setting them off is banned for a broken limit.
|
|
|
|
keeps setting them off is banned under the second rule.
|
|
|
|
|
|
|
|
|
|
|
|
A client on the CrowdSec decision list is banned until the decision ends, with
|
|
|
|
A client on the CrowdSec decision list is banned until the decision ends, with
|
|
|
|
the cause `crowdsec`. Only a client that sends a request is banned, so a long
|
|
|
|
the cause `crowdsec`. Only a client that sends a request is banned, so a long
|
|
|
@@ -331,8 +329,8 @@ Conventions:
|
|
|
|
- `SWWAF_ADMIN_TOKEN` switches on the ban endpoints and
|
|
|
|
- `SWWAF_ADMIN_TOKEN` switches on the ban endpoints and
|
|
|
|
`/_smallwebwaf/clients/<ip>`. The metrics are off until `SWWAF_METRICS_TOKEN`
|
|
|
|
`/_smallwebwaf/clients/<ip>`. The metrics are off until `SWWAF_METRICS_TOKEN`
|
|
|
|
is set, since without a token they would be open to anyone on the internet.
|
|
|
|
is set, since without a token they would be open to anyone on the internet.
|
|
|
|
The two are separate, so the metrics collector, which holds the metrics token,
|
|
|
|
The two are separate, so a scraper that holds the metrics token cannot manage
|
|
|
|
cannot manage bans.
|
|
|
|
bans.
|
|
|
|
- A token is a long random value, since the endpoints can be reached from the
|
|
|
|
- A token is a long random value, since the endpoints can be reached from the
|
|
|
|
internet. One shorter than 32 characters stops the start. The app starts with
|
|
|
|
internet. One shorter than 32 characters stops the start. The app starts with
|
|
|
|
the same environment variables as `smallwebwaf`, so it can read a token set as
|
|
|
|
the same environment variables as `smallwebwaf`, so it can read a token set as
|
|
|
@@ -368,23 +366,16 @@ Conventions:
|
|
|
|
- `SWWAF_ALLOW_NETS` skips the checks step 3 names, for monitoring or the
|
|
|
|
- `SWWAF_ALLOW_NETS` skips the checks step 3 names, for monitoring or the
|
|
|
|
owner's own networks. `SWWAF_RATE_LIMIT_EXEMPT_NETS` bypasses the request and
|
|
|
|
owner's own networks. `SWWAF_RATE_LIMIT_EXEMPT_NETS` bypasses the request and
|
|
|
|
byte limits only; the error burst, attack detection and bans still apply.
|
|
|
|
byte limits only; the error burst, attack detection and bans still apply.
|
|
|
|
- Many people can share one address, as in an office, on a mobile carrier or
|
|
|
|
- Many clients share one address (mobile carriers, offices, Tor), and so share
|
|
|
|
through Tor. They are one client, and share its limits and its bans. The
|
|
|
|
its limits and its bans. The defaults sit well above one person's use, and the
|
|
|
|
default limits are sized for about 40 busy people on one address. A known
|
|
|
|
first two lists take known shared addresses out.
|
|
|
|
shared address busier than that belongs in `SWWAF_ALLOW_NETS` or
|
|
|
|
|
|
|
|
`SWWAF_RATE_LIMIT_EXEMPT_NETS`.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
### Request rate limits, per client
|
|
|
|
### Request rate limits, per client
|
|
|
|
|
|
|
|
|
|
|
|
Breaking one bans the client, so the defaults sit about twice above what 40
|
|
|
|
Breaking one bans the client, so the defaults sit several times above what one
|
|
|
|
people busy with heavy web apps make together on one address. Each such person
|
|
|
|
busy person produces. A browser loading a heavy page makes a few hundred
|
|
|
|
makes about 75 requests in a busy minute, and 1,250 an hour for ten hours a day.
|
|
|
|
requests, and a git clone a handful. A machine that talks to the app all day can
|
|
|
|
|
|
|
|
still pass the day limit. Its address belongs in `SWWAF_RATE_LIMIT_EXEMPT_NETS`.
|
|
|
|
A client slower than those 40 people cannot be told from them by counting. Only
|
|
|
|
|
|
|
|
the other checks can catch an abuser that slow. They include the trap paths, the
|
|
|
|
|
|
|
|
rule files, the Core Rule Set and the blocklists. An abuser that trips none of
|
|
|
|
|
|
|
|
them is not caught. A machine that talks to the app all day can still pass the
|
|
|
|
|
|
|
|
day limit. Its address belongs in `SWWAF_RATE_LIMIT_EXEMPT_NETS`.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
`SWWAF_RATE_LIMIT_EXEMPT_PATHS` takes path prefixes out of the count, such as
|
|
|
|
`SWWAF_RATE_LIMIT_EXEMPT_PATHS` takes path prefixes out of the count, such as
|
|
|
|
static assets or health checks; their bytes are not counted either. A prefix is
|
|
|
|
static assets or health checks; their bytes are not counted either. A prefix is
|
|
|
@@ -397,12 +388,9 @@ matched the same way.
|
|
|
|
|
|
|
|
|
|
|
|
### Byte limits, per client
|
|
|
|
### Byte limits, per client
|
|
|
|
|
|
|
|
|
|
|
|
Each of the same 40 people moves about 256 MiB an hour, for ten hours a day. The
|
|
|
|
A response's bytes are counted when it ends (see step 15). So every default sits
|
|
|
|
hour and day limits sit about twice above what they move together. A response's
|
|
|
|
above the largest request and answer together, and no single download breaks a
|
|
|
|
bytes are counted when it ends (see step 15). So the minute limit holds two of
|
|
|
|
limit.
|
|
|
|
the largest responses, with their requests, ending in the same minute. No single
|
|
|
|
|
|
|
|
download then breaks a limit. An operator who raises `SWWAF_RESPONSE_MAX_BYTES`
|
|
|
|
|
|
|
|
raises the byte limits with it.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
### Size and time limits
|
|
|
|
### Size and time limits
|
|
|
|
|
|
|
|
|
|
|
@@ -454,8 +442,12 @@ service that keeps its visitors' addresses on its own host.
|
|
|
|
|
|
|
|
|
|
|
|
- Only one source is used. The start stops, naming both settings, when `file` is
|
|
|
|
- Only one source is used. The start stops, naming both settings, when `file` is
|
|
|
|
set without `SWWAF_LOOKUP_DB_PATH`, or `SWWAF_LOOKUP_DB_PATH` with any other
|
|
|
|
set without `SWWAF_LOOKUP_DB_PATH`, or `SWWAF_LOOKUP_DB_PATH` with any other
|
|
|
|
source. The start also stops when the source is `off` and a setting that needs
|
|
|
|
source. The start also stops when the source is `off` and one of these needs
|
|
|
|
a lookup is set (`docs/settings.md` lists them).
|
|
|
|
lookups:
|
|
|
|
|
|
|
|
- a country list;
|
|
|
|
|
|
|
|
- a biased threshold;
|
|
|
|
|
|
|
|
- `SWWAF_ADD_LOOKUP_HEADERS`;
|
|
|
|
|
|
|
|
- an anomaly threshold per AS number.
|
|
|
|
- GeoJS is told the address of every new visitor, whether or not a setting uses
|
|
|
|
- GeoJS is told the address of every new visitor, whether or not a setting uses
|
|
|
|
the answer, since the request log, the metrics and the ban notes carry it.
|
|
|
|
the answer, since the request log, the metrics and the ban notes carry it.
|
|
|
|
These addresses are never sent:
|
|
|
|
These addresses are never sent:
|
|
|
@@ -466,8 +458,8 @@ service that keeps its visitors' addresses on its own host.
|
|
|
|
Addresses are asked about together, up to 200 in one request
|
|
|
|
Addresses are asked about together, up to 200 in one request
|
|
|
|
(`https://get.geojs.io/v1/ip/geo.json?ip=a,b,c`), one request at a time, which
|
|
|
|
(`https://get.geojs.io/v1/ip/geo.json?ip=a,b,c`), one request at a time, which
|
|
|
|
keeps the number of requests to GeoJS low. Each answer's `country_code`, `asn`
|
|
|
|
keeps the number of requests to GeoJS low. Each answer's `country_code`, `asn`
|
|
|
|
and `organization_name` are read. An answer without a `country_code` leaves
|
|
|
|
and `organization_name` are read. An answer without a `country_code`, or with
|
|
|
|
the country unknown, and the `asn` `64512` leaves the AS number unknown.
|
|
|
|
the `asn` `64512`, counts as unknown.
|
|
|
|
- Each answer is kept for seven days, in memory and in `lookups.json`. Answers
|
|
|
|
- Each answer is kept for seven days, in memory and in `lookups.json`. Answers
|
|
|
|
are kept apart from the table of clients, so that an answer outlasts a client
|
|
|
|
are kept apart from the table of clients, so that an answer outlasts a client
|
|
|
|
dropped from that table. The answer reaches the client's history and ban notes
|
|
|
|
dropped from that table. The answer reaches the client's history and ban notes
|
|
|
@@ -484,8 +476,8 @@ service that keeps its visitors' addresses on its own host.
|
|
|
|
API requests", judged by GeoJS alone, and let it block a caller. While GeoJS
|
|
|
|
API requests", judged by GeoJS alone, and let it block a caller. While GeoJS
|
|
|
|
is slow, down or refusing `smallwebwaf`, new clients count as unknown. GeoJS
|
|
|
|
is slow, down or refusing `smallwebwaf`, new clients count as unknown. GeoJS
|
|
|
|
is then left alone for a second, and twice as long after each further failure,
|
|
|
|
is then left alone for a second, and twice as long after each further failure,
|
|
|
|
up to five minutes. Each failure raises a `source_failure` alert (see
|
|
|
|
up to five minutes, and one `source_failure` alert is sent per cooldown. A
|
|
|
|
"Alerting"). A service that cannot accept this uses the file.
|
|
|
|
service that cannot accept this uses the file.
|
|
|
|
- The file is the free IPinfo Lite database (`ipinfo_lite.mmdb`), whose `asn`,
|
|
|
|
- The file is the free IPinfo Lite database (`ipinfo_lite.mmdb`), whose `asn`,
|
|
|
|
`as_name` and `country_code` fields are read. The operator downloads it with a
|
|
|
|
`as_name` and `country_code` fields are read. The operator downloads it with a
|
|
|
|
free IPinfo account and mounts it read-only; `smallwebwaf` never fetches it
|
|
|
|
free IPinfo account and mounts it read-only; `smallwebwaf` never fetches it
|
|
|
@@ -494,9 +486,9 @@ service that keeps its visitors' addresses on its own host.
|
|
|
|
on the site that uses it, such as "IP address data is powered by IPinfo".
|
|
|
|
on the site that uses it, such as "IP address data is powered by IPinfo".
|
|
|
|
- The file is read whole at start; one that is missing or unreadable stops the
|
|
|
|
- The file is read whole at start; one that is missing or unreadable stops the
|
|
|
|
start. It is read again once it has gone 2 seconds without a change after
|
|
|
|
start. It is read again once it has gone 2 seconds without a change after
|
|
|
|
being replaced. A replacement that cannot be read is logged, sent as one
|
|
|
|
being replaced. One that cannot be read is logged, sent as one `file_error`
|
|
|
|
`file_error` alert and ignored. The database already loaded stays in use. A
|
|
|
|
alert and ignored. The database already loaded stays in use. A client whose
|
|
|
|
client whose address the file does not hold counts as unknown.
|
|
|
|
address the file does not hold counts as unknown.
|
|
|
|
|
|
|
|
|
|
|
|
### Country lists
|
|
|
|
### Country lists
|
|
|
|
|
|
|
|
|
|
|
@@ -516,8 +508,9 @@ where it comes from.
|
|
|
|
exclusive list already refuses every other country. A code on both lists stops
|
|
|
|
exclusive list already refuses every other country. A code on both lists stops
|
|
|
|
the start with a message naming it.
|
|
|
|
the start with a message naming it.
|
|
|
|
- A refused request is answered with `SWWAF_BAN_RESPONSE`, as soon as the
|
|
|
|
- A refused request is answered with `SWWAF_BAN_RESPONSE`, as soon as the
|
|
|
|
client's address has been looked up, before its body is read. It is logged
|
|
|
|
client's address has been looked up, before its body is read. It skips the
|
|
|
|
with the action `country_denied`, and counted in the metrics.
|
|
|
|
reputation checks, the limits, the rule files and the Core Rule Set. It is
|
|
|
|
|
|
|
|
logged with the action `country_denied`, and counted in the metrics.
|
|
|
|
|
|
|
|
|
|
|
|
### Biased thresholds
|
|
|
|
### Biased thresholds
|
|
|
|
|
|
|
|
|
|
|
@@ -526,11 +519,10 @@ countries. The lists are empty by default, for the same reason as the country
|
|
|
|
lists. The request log and the metrics show which AS numbers and countries a
|
|
|
|
lists. The request log and the metrics show which AS numbers and countries a
|
|
|
|
service's abuse comes from.
|
|
|
|
service's abuse comes from.
|
|
|
|
|
|
|
|
|
|
|
|
- Each list gives AS numbers or countries a percentage, such as `AS14061:50`.
|
|
|
|
- Clients in a listed AS number or country get that percentage of every request
|
|
|
|
Their clients get that percentage of every request and byte limit
|
|
|
|
and byte limit (`SWWAF_ASN_LIMIT_PERCENT`, `SWWAF_COUNTRY_LIMIT_PERCENT`). A
|
|
|
|
(`SWWAF_ASN_LIMIT_PERCENT`, `SWWAF_COUNTRY_LIMIT_PERCENT`). A lowered limit is
|
|
|
|
lowered limit is rounded down. `0` is a zero allowance: the client's first
|
|
|
|
rounded down. `0` is a zero allowance: the client's first request breaks a
|
|
|
|
request breaks a limit and bans it.
|
|
|
|
limit and bans it.
|
|
|
|
|
|
|
|
- `SWWAF_ASN_BYTES_PERCENT` and `SWWAF_COUNTRY_BYTES_PERCENT` set the byte
|
|
|
|
- `SWWAF_ASN_BYTES_PERCENT` and `SWWAF_COUNTRY_BYTES_PERCENT` set the byte
|
|
|
|
limits' percentage alone, when it should differ from the request limits'. For
|
|
|
|
limits' percentage alone, when it should differ from the request limits'. For
|
|
|
|
an AS number they list, they replace its percentage from
|
|
|
|
an AS number they list, they replace its percentage from
|
|
|
@@ -541,12 +533,12 @@ service's abuse comes from.
|
|
|
|
fetched and kept like a blocklist (see "Reputation"). Where it and
|
|
|
|
fetched and kept like a blocklist (see "Reputation"). Where it and
|
|
|
|
`SWWAF_ASN_LIMIT_PERCENT` both give a percentage, the lower applies.
|
|
|
|
`SWWAF_ASN_LIMIT_PERCENT` both give a percentage, the lower applies.
|
|
|
|
- A client's limit percentage is the lowest of those that apply to it: its AS
|
|
|
|
- A client's limit percentage is the lowest of those that apply to it: its AS
|
|
|
|
number's, its country's, or a reputation hit's `limit:<percent>`. A client
|
|
|
|
number's, its country's, or a reputation hit's `limit:<percent>`. A client the
|
|
|
|
without a country gets `SWWAF_UNKNOWN_LIMIT_PERCENT` in place of a country's.
|
|
|
|
lookup cannot place gets `SWWAF_UNKNOWN_LIMIT_PERCENT` in place of an AS
|
|
|
|
The percentage is 100 if none applies. Clients given a lower percentage are
|
|
|
|
number's or a country's. The percentage is 100 if none applies. Clients given
|
|
|
|
banned after fewer requests, first for a while and then, for repeated abuse,
|
|
|
|
a lower percentage are banned after fewer requests, first for a while and
|
|
|
|
permanently. The log line of each counted request, and the notes of each ban,
|
|
|
|
then, for repeated abuse, permanently. The log line of each counted request,
|
|
|
|
say which setting lowered the limit.
|
|
|
|
and the notes of each ban, say which setting lowered the limit.
|
|
|
|
- No budget is shared by a whole country or AS number. One abuser could use it
|
|
|
|
- No budget is shared by a whole country or AS number. One abuser could use it
|
|
|
|
up and lock out everyone else there: a denial of service nobody chose, and the
|
|
|
|
up and lock out everyone else there: a denial of service nobody chose, and the
|
|
|
|
one this tool exists to prevent. Refusing a whole country is left to the
|
|
|
|
one this tool exists to prevent. Refusing a whole country is left to the
|
|
|
@@ -557,14 +549,16 @@ service's abuse comes from.
|
|
|
|
By default the Core Rule Set reads the method, the URL with its query string,
|
|
|
|
By default the Core Rule Set reads the method, the URL with its query string,
|
|
|
|
and the headers of every request. That is where automated attacks show:
|
|
|
|
and the headers of every request. That is where automated attacks show:
|
|
|
|
injection in query strings, path traversal, attacks carried in headers,
|
|
|
|
injection in query strings, path traversal, attacks carried in headers,
|
|
|
|
scanners' user agents. It reads no request body. In front of a code forge such
|
|
|
|
scanners' user agents. It reads no request body. On a code forge, the example
|
|
|
|
as gitea, bodies carry what people write and publish, such as issue text and
|
|
|
|
deployment, bodies carry what people write and publish, such as issue text and
|
|
|
|
files saved in the web editor. The Core Rule Set takes the shell commands and
|
|
|
|
files saved in the web editor. The Core Rule Set takes the shell commands and
|
|
|
|
code in them for attacks. The default gives up refusing an attack carried in a
|
|
|
|
code in them for attacks. The default gives up refusing an attack carried in a
|
|
|
|
body. The rule files, the limits and the bans still apply to the client that
|
|
|
|
body. The rule files, the limits and the bans still apply to the client that
|
|
|
|
sends it.
|
|
|
|
sends it.
|
|
|
|
|
|
|
|
|
|
|
|
- The Core Rule Set skips a path under `SWWAF_WAF_EXEMPT_PATHS`.
|
|
|
|
- The Core Rule Set inspects only a request that every earlier check let
|
|
|
|
|
|
|
|
through, and skips a path under `SWWAF_WAF_EXEMPT_PATHS`. An engine error on a
|
|
|
|
|
|
|
|
request forwards the request, and is logged and counted.
|
|
|
|
- `SWWAF_WAF_PARANOIA_LEVEL` and `SWWAF_WAF_ANOMALY_THRESHOLD` are the Core Rule
|
|
|
|
- `SWWAF_WAF_PARANOIA_LEVEL` and `SWWAF_WAF_ANOMALY_THRESHOLD` are the Core Rule
|
|
|
|
Set's own two tuning values, at its own defaults. Each level up runs more
|
|
|
|
Set's own two tuning values, at its own defaults. Each level up runs more
|
|
|
|
rules, which find more attacks and take more ordinary requests for attacks.
|
|
|
|
rules, which find more attacks and take more ordinary requests for attacks.
|
|
|
@@ -607,18 +601,14 @@ sends it.
|
|
|
|
percent-decoded: `/wp-login.php` matches `/wp-login.php?x=1` and
|
|
|
|
percent-decoded: `/wp-login.php` matches `/wp-login.php?x=1` and
|
|
|
|
`/%77p-login.php`, and does not match `/wp-login.php/` or
|
|
|
|
`/%77p-login.php`, and does not match `/wp-login.php/` or
|
|
|
|
`/blog/wp-login.php`.
|
|
|
|
`/blog/wp-login.php`.
|
|
|
|
- `SWWAF_ERROR_BURST_THRESHOLD` counts a client's refusals per minute of these
|
|
|
|
- `SWWAF_ERROR_BURST_THRESHOLD` counts a client's refusals per minute for a rule
|
|
|
|
kinds, and of no other:
|
|
|
|
file match, a trap path, a Core Rule Set match, or a missing or wrong token;
|
|
|
|
- a rule file match;
|
|
|
|
no other refusal counts. More than this breaks a limit (see "Bans"). A client
|
|
|
|
- a trap path;
|
|
|
|
trying one attack or token after another is refused many times a minute; a
|
|
|
|
- a Core Rule Set match;
|
|
|
|
person rarely more than a few times. Answers the app gives are not counted,
|
|
|
|
- a missing or wrong token.
|
|
|
|
since they do not tell a scanner from an ordinary client. In front of gitea,
|
|
|
|
|
|
|
|
registry and git clients get `404` and `401` by design, hundreds of times a
|
|
|
|
More than this breaks a limit (see "Bans"). A client trying one attack or
|
|
|
|
minute.
|
|
|
|
token after another is refused many times a minute; a person rarely more
|
|
|
|
|
|
|
|
than a few times. Answers the app gives are not counted, since they do not
|
|
|
|
|
|
|
|
tell a scanner from an ordinary client. In front of gitea, registry and git
|
|
|
|
|
|
|
|
clients get `404` and `401` by design, hundreds of times a minute.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
`smallwebwaf` changes the Core Rule Set 4.25.0 in six ways, since in front of
|
|
|
|
`smallwebwaf` changes the Core Rule Set 4.25.0 in six ways, since in front of
|
|
|
|
gitea it would otherwise refuse ordinary requests. The changes hold in front of
|
|
|
|
gitea it would otherwise refuse ordinary requests. The changes hold in front of
|
|
|
@@ -645,12 +635,12 @@ every app, and no setting undoes them:
|
|
|
|
- `ref`, `sha`, `branch` and `workflow`;
|
|
|
|
- `ref`, `sha`, `branch` and `workflow`;
|
|
|
|
- `artifactName` and `redirect_to`.
|
|
|
|
- `artifactName` and `redirect_to`.
|
|
|
|
|
|
|
|
|
|
|
|
gitea puts into them file paths, branch and workflow names, and the page to
|
|
|
|
gitea puts file paths, branch and workflow names, and the page to return to
|
|
|
|
return to after signing in. It reads these values as names in a repository
|
|
|
|
after signing in into them. It reads these values as names in a repository
|
|
|
|
or in its own records, or as a page of its own site, never as a command. Any
|
|
|
|
or in its own records, or as a page of its own site, never as a command. Any
|
|
|
|
name on those lists can be an ordinary file or branch, such as `.gitignore`
|
|
|
|
name on those lists can be an ordinary file or branch, such as `.gitignore`
|
|
|
|
or `docker-build`. In these parameters, what only these three rules would
|
|
|
|
or `docker-build`. In these parameters, anything only these three rules
|
|
|
|
catch reaches the app, such as the command `|cat /etc/passwd`. Path
|
|
|
|
catch reaches the app, commands such as `|cat /etc/passwd` included. Path
|
|
|
|
traversal, SQL and script injection, and PHP, Java and Node.js code are
|
|
|
|
traversal, SQL and script injection, and PHP, Java and Node.js code are
|
|
|
|
still refused there. Every other parameter keeps all three rules. An app
|
|
|
|
still refused there. Every other parameter keeps all three rules. An app
|
|
|
|
that uses one of these parameters as a file on the server, or passes it to a
|
|
|
|
that uses one of these parameters as a file on the server, or passes it to a
|
|
|
@@ -659,8 +649,8 @@ every app, and no setting undoes them:
|
|
|
|
5. Two cookies are left out, and `Referer` keeps fewer rules:
|
|
|
|
5. Two cookies are left out, and `Referer` keeps fewer rules:
|
|
|
|
- The Core Rule Set reads the request without the `gitea_flash` and
|
|
|
|
- The Core Rule Set reads the request without the `gitea_flash` and
|
|
|
|
`redirect_to` cookies. gitea puts a message naming what was just done into
|
|
|
|
`redirect_to` cookies. gitea puts a message naming what was just done into
|
|
|
|
`gitea_flash`. It puts into `redirect_to` the page to return to after
|
|
|
|
`gitea_flash`. It puts the page to return to after signing in into
|
|
|
|
signing in. The Core Rule Set takes file names in them, such as
|
|
|
|
`redirect_to`. The Core Rule Set takes file names in them, such as
|
|
|
|
`package.json`, for attacks. A browser sends such a cookie with every
|
|
|
|
`package.json`, for attacks. A browser sends such a cookie with every
|
|
|
|
request until gitea replaces it. gitea cannot replace it while those
|
|
|
|
request until gitea replaces it. gitea cannot replace it while those
|
|
|
|
requests are refused, so one refusal would keep the browser out of the
|
|
|
|
requests are refused, so one refusal would keep the browser out of the
|
|
|
@@ -783,9 +773,9 @@ too close together.
|
|
|
|
by default. Such verdicts are less certain than a blocklist, so a listed
|
|
|
|
by default. Such verdicts are less certain than a blocklist, so a listed
|
|
|
|
client gets a quarter of every request and byte limit rather than a refusal.
|
|
|
|
client gets a quarter of every request and byte limit rather than a refusal.
|
|
|
|
- Verdicts and scores are kept for `SWWAF_REPUTATION_CACHE_TTL`, in memory and
|
|
|
|
- Verdicts and scores are kept for `SWWAF_REPUTATION_CACHE_TTL`, in memory and
|
|
|
|
in `reputation.json`. A source that is down or over quota gives no verdict.
|
|
|
|
in `reputation.json`, with the fixed bounds under "Fixed limits" in
|
|
|
|
The service continues. Each failure raises a `source_failure` alert (see
|
|
|
|
`docs/settings.md`. A source that is down or over quota gives no verdict. The
|
|
|
|
"Alerting").
|
|
|
|
service continues, with one `source_failure` alert per cooldown.
|
|
|
|
|
|
|
|
|
|
|
|
### Alerting
|
|
|
|
### Alerting
|
|
|
|
|
|
|
|
|
|
|
@@ -797,8 +787,9 @@ too close together.
|
|
|
|
the ban's alert, with `mode` set to `observe` in its `detail`.
|
|
|
|
the ban's alert, with `mode` set to `observe` in its `detail`.
|
|
|
|
- A repeat is an alert with the same event, and the same netblock, source, file
|
|
|
|
- A repeat is an alert with the same event, and the same netblock, source, file
|
|
|
|
or anomaly scope, as one sent less than `SWWAF_ALERT_COOLDOWN` before. An
|
|
|
|
or anomaly scope, as one sent less than `SWWAF_ALERT_COOLDOWN` before. An
|
|
|
|
anomaly scope is one of the things counted under "Anomaly thresholds". A
|
|
|
|
anomaly scope is the thing counted: one client, netblock, AS number or named
|
|
|
|
repeat is held back and counted, and the next such alert sent gives the count.
|
|
|
|
netblock, or the whole service. A repeat is held back and counted, and the
|
|
|
|
|
|
|
|
next such alert sent gives the count.
|
|
|
|
- Past `SWWAF_ALERT_MAX_PER_HOUR` alerts in an hour, the rest are held back and
|
|
|
|
- Past `SWWAF_ALERT_MAX_PER_HOUR` alerts in an hour, the rest are held back and
|
|
|
|
counted by event, so a wide attack cannot flood the channel. Once the hour
|
|
|
|
counted by event, so a wide attack cannot flood the channel. Once the hour
|
|
|
|
ends, one alert with the event `summary` gives those counts. It also counts
|
|
|
|
ends, one alert with the event `summary` gives those counts. It also counts
|
|
|
@@ -829,7 +820,7 @@ addresses at low rates each, which the per-client limits do not see. A low
|
|
|
|
- Every request but the health check is counted, whatever was done with it,
|
|
|
|
- Every request but the health check is counted, whatever was done with it,
|
|
|
|
clients in `SWWAF_ALLOW_NETS` and `SWWAF_RATE_LIMIT_EXEMPT_NETS` included,
|
|
|
|
clients in `SWWAF_ALLOW_NETS` and `SWWAF_RATE_LIMIT_EXEMPT_NETS` included,
|
|
|
|
since an exempt client misbehaving is worth knowing about. The counters are
|
|
|
|
since an exempt client misbehaving is worth knowing about. The counters are
|
|
|
|
kept in memory and in `alerts.json`.
|
|
|
|
kept in memory and in `alerts.json`, with a fixed bound.
|
|
|
|
|
|
|
|
|
|
|
|
## Rule files
|
|
|
|
## Rule files
|
|
|
|
|
|
|
|
|
|
|
@@ -855,10 +846,10 @@ and the default file.
|
|
|
|
of every request; a rule naming one stops the start like any bad line. Request
|
|
|
|
of every request; a rule naming one stops the start like any bad line. Request
|
|
|
|
bodies are not available to rule files; reading them is the Core Rule Set's
|
|
|
|
bodies are not available to rule files; reading them is the Core Rule Set's
|
|
|
|
job.
|
|
|
|
job.
|
|
|
|
- `log` only notes the match in the request log. `block` refuses the request;
|
|
|
|
- `log` only notes the match in the request log. `block` refuses the request
|
|
|
|
`ban` makes the request a clear sign of attack (see "Bans"), and is for
|
|
|
|
without a ban; `ban` makes the request a clear sign of attack (see "Bans"),
|
|
|
|
requests no real visitor sends. A visitor banned by mistake is let back in by
|
|
|
|
and is for requests no real visitor sends. A visitor banned by mistake is let
|
|
|
|
lifting the ban.
|
|
|
|
back in by lifting the ban.
|
|
|
|
- At start, these stop the process with a message naming the file and line:
|
|
|
|
- At start, these stop the process with a message naming the file and line:
|
|
|
|
- a line that does not parse;
|
|
|
|
- a line that does not parse;
|
|
|
|
- a regex that does not compile;
|
|
|
|
- a regex that does not compile;
|
|
|
@@ -874,11 +865,11 @@ and the default file.
|
|
|
|
wrong for every app, with its path rules anchored at the site root. In front
|
|
|
|
wrong for every app, with its path rules anchored at the site root. In front
|
|
|
|
of gitea, `/.env` is a probe, while
|
|
|
|
of gitea, `/.env` is a probe, while
|
|
|
|
`/<owner>/<repo>/src/branch/main/.env.example` is a file in a repository that
|
|
|
|
`/<owner>/<repo>/src/branch/main/.env.example` is a file in a repository that
|
|
|
|
any visitor may open. Anchor the paths of your own `ban` rules at the site
|
|
|
|
any visitor may open. `ban` rules of your own follow the same rule. Anything
|
|
|
|
root too. Anything app-specific, such as a rule banning WordPress probes in
|
|
|
|
app-specific, such as a rule banning WordPress probes in front of gitea,
|
|
|
|
front of gitea, belongs in a file the app's Dockerfile adds or the deployer
|
|
|
|
belongs in a file the app's Dockerfile adds or the deployer mounts. A
|
|
|
|
mounts. A directory mounted over `SWWAF_RULES_DIR` replaces the default file;
|
|
|
|
directory mounted over `SWWAF_RULES_DIR` replaces the default file; single
|
|
|
|
single files mounted into it add to it.
|
|
|
|
files mounted into it add to it.
|
|
|
|
- Rules meant to be edited while `smallwebwaf` runs belong in a mounted
|
|
|
|
- Rules meant to be edited while `smallwebwaf` runs belong in a mounted
|
|
|
|
directory, not a mounted file. Many editors save a file by replacing it, and a
|
|
|
|
directory, not a mounted file. Many editors save a file by replacing it, and a
|
|
|
|
container keeps seeing the old one when the file alone is mounted. The lookup
|
|
|
|
container keeps seeing the old one when the file alone is mounted. The lookup
|
|
|
@@ -887,11 +878,11 @@ and the default file.
|
|
|
|
|
|
|
|
|
|
|
|
## Persistent state
|
|
|
|
## Persistent state
|
|
|
|
|
|
|
|
|
|
|
|
All state lives in memory, and the files in `SWWAF_STATE_DIR` hold a copy of it,
|
|
|
|
All state lives in memory, and the files in `SWWAF_STATE_DIR` hold a copy of all
|
|
|
|
so an orderly stop loses nothing. The files are meant for people as well: an
|
|
|
|
of it, so an orderly stop loses nothing (the remote log is the exception; see
|
|
|
|
admin can read or edit them at any time, and the running `smallwebwaf` takes the
|
|
|
|
"Request log"). The files are meant for people as well: an admin can read or
|
|
|
|
edit in. `docs/state-files.md` lists the files, what each holds, and their
|
|
|
|
edit them at any time, and the running `smallwebwaf` takes the edit in.
|
|
|
|
sizes.
|
|
|
|
`docs/state-files.md` lists the files, what each holds, and their sizes.
|
|
|
|
|
|
|
|
|
|
|
|
- Each file holds one kind of state, as JSON with:
|
|
|
|
- Each file holds one kind of state, as JSON with:
|
|
|
|
- a top-level `version` number;
|
|
|
|
- a top-level `version` number;
|
|
|
@@ -949,8 +940,9 @@ sizes.
|
|
|
|
|
|
|
|
|
|
|
|
## Request log
|
|
|
|
## Request log
|
|
|
|
|
|
|
|
|
|
|
|
Every request `smallwebwaf` sees gets one JSON object on one line on stdout,
|
|
|
|
Every request gets one JSON object on one line on stdout, refused ones included,
|
|
|
|
refused ones included. `docs/request-log.md` lists the fields. Bodies are never
|
|
|
|
apart from those Go's HTTP server ends before `smallwebwaf` sees them (see "Size
|
|
|
|
|
|
|
|
and time limits"). `docs/request-log.md` lists the fields. Bodies are never
|
|
|
|
logged, and no header but those listed there. Query strings are logged as
|
|
|
|
logged, and no header but those listed there. Query strings are logged as
|
|
|
|
received; an app that carries secrets in query strings needs that fixed in the
|
|
|
|
received; an app that carries secrets in query strings needs that fixed in the
|
|
|
|
app. The process's own messages (start, the settings, stop, errors) share the
|
|
|
|
app. The process's own messages (start, the settings, stop, errors) share the
|
|
|
@@ -979,7 +971,8 @@ while `docker logs` keeps working.
|
|
|
|
## Metrics endpoint
|
|
|
|
## Metrics endpoint
|
|
|
|
|
|
|
|
|
|
|
|
`GET /_smallwebwaf/metrics` answers in the Prometheus text format, for a request
|
|
|
|
`GET /_smallwebwaf/metrics` answers in the Prometheus text format, for a request
|
|
|
|
carrying `SWWAF_METRICS_TOKEN`. `docs/metrics.md` lists the metrics. No metric
|
|
|
|
carrying `SWWAF_METRICS_TOKEN`. `docs/metrics.md` lists the metrics. Every
|
|
|
|
|
|
|
|
metric carries the label `instance`, set to `SWWAF_INSTANCE_NAME`. No metric
|
|
|
|
carries a client IP address as a label. Per-address questions are answered by
|
|
|
|
carries a client IP address as a label. Per-address questions are answered by
|
|
|
|
the request log and `GET /_smallwebwaf/clients/<ip>`.
|
|
|
|
the request log and `GET /_smallwebwaf/clients/<ip>`.
|
|
|
|
|
|
|
|
|
|
|
@@ -987,14 +980,15 @@ the request log and `GET /_smallwebwaf/clients/<ip>`.
|
|
|
|
|
|
|
|
|
|
|
|
A request whose path starts with `/_smallwebwaf/` is for `smallwebwaf` itself:
|
|
|
|
A request whose path starts with `/_smallwebwaf/` is for `smallwebwaf` itself:
|
|
|
|
it answers it and never passes it to the app. The prefix carries the tool's
|
|
|
|
it answers it and never passes it to the app. The prefix carries the tool's
|
|
|
|
name, so it takes no path an app uses. Admins and the metrics collector reach
|
|
|
|
name, so it takes no path an app uses. Admins and scrapers reach these endpoints
|
|
|
|
these endpoints through traefik, like any other request.
|
|
|
|
through traefik, like any other request. `docs/admin-endpoints.md` lists the
|
|
|
|
`docs/admin-endpoints.md` lists the endpoints, what each takes and what each
|
|
|
|
endpoints, what each takes and what each answers.
|
|
|
|
answers.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
- `GET /_smallwebwaf/healthz` answers `200` with `ok` to anyone, without a
|
|
|
|
- `GET /_smallwebwaf/healthz` answers `200` with `ok` to anyone, without a
|
|
|
|
token, while `smallwebwaf` is running. It does not ask the app; the
|
|
|
|
token, while `smallwebwaf` is running. It does not ask the app; the
|
|
|
|
container's health check does.
|
|
|
|
container's health check does. It is answered before any check, so a health
|
|
|
|
|
|
|
|
checker never needs an exemption and is never refused, for example by
|
|
|
|
|
|
|
|
`SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES`.
|
|
|
|
- Every other endpoint needs its token, sent as `Authorization: Bearer <token>`
|
|
|
|
- Every other endpoint needs its token, sent as `Authorization: Bearer <token>`
|
|
|
|
(see "Tokens" under "Settings"). While a token is unset, the endpoints that
|
|
|
|
(see "Tokens" under "Settings"). While a token is unset, the endpoints that
|
|
|
|
need it answer `404`, as does any other path under the prefix.
|
|
|
|
need it answer `404`, as does any other path under the prefix.
|
|
|
@@ -1002,9 +996,10 @@ answers.
|
|
|
|
other request goes through. They are answered at the point where another
|
|
|
|
other request goes through. They are answered at the point where another
|
|
|
|
request would be forwarded to the app: a banned or refused client stays
|
|
|
|
request would be forwarded to the app: a banned or refused client stays
|
|
|
|
refused, and each request counts toward the client's limits. A missing or
|
|
|
|
refused, and each request counts toward the client's limits. A missing or
|
|
|
|
wrong token is answered `401` (see "Attack detection" for what guessing
|
|
|
|
wrong token is answered `401`, in `SWWAF_MODE=observe` too, and counts toward
|
|
|
|
costs).
|
|
|
|
the error burst. A client guessing tokens is therefore banned once it passes
|
|
|
|
- A ban made through the endpoints starts at once.
|
|
|
|
`SWWAF_ERROR_BURST_THRESHOLD` guesses in a minute.
|
|
|
|
|
|
|
|
- A ban made through the endpoints starts at once, with the cause `admin`.
|
|
|
|
|
|
|
|
|
|
|
|
## Deployment
|
|
|
|
## Deployment
|
|
|
|
|
|
|
|
|
|
|
@@ -1032,40 +1027,40 @@ what the image holds and how each part is pinned.
|
|
|
|
digest.
|
|
|
|
digest.
|
|
|
|
- nixpkgs comes from a release file on `releases.nixos.org`, named with its
|
|
|
|
- nixpkgs comes from a release file on `releases.nixos.org`, named with its
|
|
|
|
hash, not from a GitHub archive, whose bytes can change. Root uses Nix
|
|
|
|
hash, not from a GitHub archive, whose bytes can change. Root uses Nix
|
|
|
|
directly; no Nix daemon runs. `nix-bin` creates no build users, so
|
|
|
|
directly; no Nix daemon runs, and since `nix-bin` creates no build users,
|
|
|
|
`/etc/nix/nix.conf` sets `build-users-group =`, which lets root build without
|
|
|
|
`build-users-group =` in `/etc/nix/nix.conf` lets root build without them.
|
|
|
|
them. Root's Nix profile is last on the `PATH`, so that no package hides the
|
|
|
|
Root's Nix profile is last on the `PATH`, so that no package hides the image's
|
|
|
|
image's own commands; busybox's `sv`, for one, looks for services elsewhere.
|
|
|
|
own commands; busybox's `sv`, for one, looks for services elsewhere. An app
|
|
|
|
An app built on the image gets the same packages each time. Security fixes
|
|
|
|
built on the image gets the same packages each time. Security fixes from
|
|
|
|
from Ubuntu and nixpkgs come with a newer `smallwebwaf` image, which an app
|
|
|
|
Ubuntu and nixpkgs come with a newer `smallwebwaf` image, which an app takes
|
|
|
|
takes by changing the digest in its `FROM` line. The image moves to each new
|
|
|
|
by changing the digest in its `FROM` line. The image moves to each new nixpkgs
|
|
|
|
nixpkgs release, every six months.
|
|
|
|
release, every six months.
|
|
|
|
- runit supervises the two processes, with `runsvinit` as the entrypoint, as the
|
|
|
|
- runit supervises the two processes, with `runsvinit` as the entrypoint, as the
|
|
|
|
owner's code style guide asks for service containers. No package carries
|
|
|
|
owner's code style guide asks for service containers. No package carries
|
|
|
|
`runsvinit`, so the image builds it from its archived source, unchanged since
|
|
|
|
`runsvinit`, so the image builds it from its archived source, unchanged since
|
|
|
|
2015, at a fixed commit hash, with a `go.mod` the build writes. `runsvinit`
|
|
|
|
2015, at a fixed commit hash, with a `go.mod` the build writes. `runsvinit`
|
|
|
|
starts runit's `runsvdir`, which runs the `run` script of each directory under
|
|
|
|
starts runit's `runsvdir`, which runs the `run` script of each directory under
|
|
|
|
`/etc/service`. Ubuntu's `sv` looks for services there too.
|
|
|
|
`/etc/service`. Ubuntu's `sv` looks for services there too.
|
|
|
|
- The image's `run` script for `smallwebwaf` makes `SWWAF_STATE_DIR` and every
|
|
|
|
- The image's `run` script for `smallwebwaf` waits one second and makes
|
|
|
|
file in it belong to the `smallwebwaf` user (uid and gid 65532). Then it
|
|
|
|
`SWWAF_STATE_DIR` and every file in it belong to the `smallwebwaf` user (uid
|
|
|
|
starts `smallwebwaf` as that user with runit's `chpst`.
|
|
|
|
and gid 65532). Then it starts `smallwebwaf` as that user with runit's
|
|
|
|
|
|
|
|
`chpst`.
|
|
|
|
|
|
|
|
|
|
|
|
Every `run` script, the app's included, is a bash script. It starts with
|
|
|
|
Every `run` script, the app's included, is a bash script. It starts with
|
|
|
|
`#!/usr/bin/env bash` and `set -euo pipefail`, and puts its code in a `main`
|
|
|
|
`#!/usr/bin/env bash` and `set -euo pipefail`, and puts its code in a `main`
|
|
|
|
function, called on its last line, as the owner's code style guide asks. Ubuntu
|
|
|
|
function, called on its last line, as the owner's code style guide asks. Ubuntu
|
|
|
|
ships bash. Each waits one second (`sleep 1`) before starting its program, so a
|
|
|
|
ships bash.
|
|
|
|
process that stops at start is started again every second, not in a tight loop.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Beyond its `FROM` line, the app's Dockerfile adds:
|
|
|
|
Beyond its `FROM` line, the app's Dockerfile adds:
|
|
|
|
|
|
|
|
|
|
|
|
- its binary;
|
|
|
|
- its binary;
|
|
|
|
- any packages it needs, from nixpkgs, with `nix-env -iA nixpkgs.<name>`;
|
|
|
|
- any packages it needs, from nixpkgs, with `nix-env -iA nixpkgs.<name>`;
|
|
|
|
- its runit service: a directory under `/etc/service` named after the app, never
|
|
|
|
- its runit service: a directory under `/etc/service` named after the app, never
|
|
|
|
`smallwebwaf`. Its `run` script starts the app with `chpst`, listening where
|
|
|
|
`smallwebwaf`. Its `run` script waits one second (`sleep 1`) and then starts
|
|
|
|
`SWWAF_UPSTREAM_URL` points (see "Ports"). The app runs as a user of its own,
|
|
|
|
the app with `chpst`, listening where `SWWAF_UPSTREAM_URL` points (see
|
|
|
|
which the Dockerfile creates with `useradd`. That user is neither root nor
|
|
|
|
"Ports"). The app runs as a user of its own, which the Dockerfile creates with
|
|
|
|
`smallwebwaf`, so that the app cannot touch the state files or the process of
|
|
|
|
`useradd`. That user is neither root nor `smallwebwaf`, so that the app cannot
|
|
|
|
`smallwebwaf`.
|
|
|
|
touch the state files or the process of `smallwebwaf`.
|
|
|
|
|
|
|
|
|
|
|
|
It sets no `ENTRYPOINT` or `USER` of its own. The container must start
|
|
|
|
It sets no `ENTRYPOINT` or `USER` of its own. The container must start
|
|
|
|
`runsvinit`, as root, so that it can start each service as its own user.
|
|
|
|
`runsvinit`, as root, so that it can start each service as its own user.
|
|
|
@@ -1077,21 +1072,20 @@ It sets no `ENTRYPOINT` or `USER` of its own. The container must start
|
|
|
|
script starts as root and hands over to its program's user with `chpst`. Both
|
|
|
|
script starts as root and hands over to its program's user with `chpst`. Both
|
|
|
|
services start with the container's environment variables and write to the
|
|
|
|
services start with the container's environment variables and write to the
|
|
|
|
container's output, which `docker logs` shows.
|
|
|
|
container's output, which `docker logs` shows.
|
|
|
|
- When `smallwebwaf` or the app exits, runit runs its `run` script again. The
|
|
|
|
- When `smallwebwaf` or the app exits, runit runs its `run` script again, which
|
|
|
|
other process keeps running, and the container does not stop. While
|
|
|
|
waits a second before starting it. The other process keeps running, and the
|
|
|
|
`smallwebwaf` is down, traefik answers `502`. While the app is down,
|
|
|
|
container does not stop. While `smallwebwaf` is down, traefik answers `502`.
|
|
|
|
`smallwebwaf` answers `502`, which is not a client offence.
|
|
|
|
While the app is down, `smallwebwaf` answers `502`, which is not a client
|
|
|
|
- Only faults found at start stop `smallwebwaf`:
|
|
|
|
offence.
|
|
|
|
|
|
|
|
- Only four things stop `smallwebwaf`, all at start:
|
|
|
|
- invalid configuration;
|
|
|
|
- invalid configuration;
|
|
|
|
- a lookup database that is missing or unreadable;
|
|
|
|
- a lookup database that is missing or unreadable;
|
|
|
|
- a rule file or state file that does not parse;
|
|
|
|
- a rule file or state file that does not parse;
|
|
|
|
- an unwritable `SWWAF_STATE_DIR`;
|
|
|
|
- an unwritable `SWWAF_STATE_DIR`.
|
|
|
|
- a `SWWAF_RULES_DIR` that does not exist or cannot be read;
|
|
|
|
- A `smallwebwaf` that stops at start is therefore started again every second,
|
|
|
|
- a `SWWAF_LISTEN_ADDR` it cannot listen on.
|
|
|
|
logging its message each time, until the cause is fixed; the health check
|
|
|
|
- A `smallwebwaf` that stops at start logs its message at each try until the
|
|
|
|
fails meanwhile. Once running, a failing GeoJS, reputation source or alert
|
|
|
|
cause is fixed; the health check fails meanwhile. Once running, a failing
|
|
|
|
destination, or a broken edit, never takes the protected service down.
|
|
|
|
GeoJS, reputation source or alert destination, or a broken edit, never takes
|
|
|
|
|
|
|
|
the protected service down.
|
|
|
|
|
|
|
|
- `docker stop` sends `SIGTERM` to `runsvinit`, which has runit stop both
|
|
|
|
- `docker stop` sends `SIGTERM` to `runsvinit`, which has runit stop both
|
|
|
|
services at once, each with `SIGTERM`, and exits once they have stopped.
|
|
|
|
services at once, each with `SIGTERM`, and exits once they have stopped.
|
|
|
|
`smallwebwaf` stops taking requests, gives those in progress five seconds to
|
|
|
|
`smallwebwaf` stops taking requests, gives those in progress five seconds to
|
|
|
@@ -1160,10 +1154,9 @@ all: `10.0.0.0/8,172.16.0.0/12,192.168.0.0/16,127.0.0.1/32,::1/128`.
|
|
|
|
|
|
|
|
|
|
|
|
### upaas
|
|
|
|
### upaas
|
|
|
|
|
|
|
|
|
|
|
|
An app deployed by upaas, which rebuilds and restarts an app's Docker container
|
|
|
|
An app deployed by upaas takes this shape with no change to upaas. upaas builds
|
|
|
|
when its repository changes, takes this shape with no change to upaas. upaas
|
|
|
|
the app's Dockerfile and runs the one container as it runs any app. The traefik
|
|
|
|
builds the app's Dockerfile and runs the one container as it runs any app. The
|
|
|
|
labels route to port 8080, and any `SWWAF_` settings go with the app's
|
|
|
|
traefik labels route to port 8080, and any `SWWAF_` settings go with the app's
|
|
|
|
|
|
|
|
environment variables. The volume for `/var/lib/smallwebwaf` and the directory
|
|
|
|
environment variables. The volume for `/var/lib/smallwebwaf` and the directory
|
|
|
|
of any token file go beside the app's own volumes. gitea is built on the image
|
|
|
|
of any token file go beside the app's own volumes. gitea is built on the image
|
|
|
|
like any other app; `docs/gitea.md` records what the defaults do in front of it,
|
|
|
|
like any other app; `docs/gitea.md` records what the defaults do in front of it,
|
|
|
|