SWW_ROUTEWATCH_URL names the instance. The new heuristic routewatch_tags
asks it in the background about each client and tags the request. Answers
are kept in memory for an hour. A request without tags gives the reason in
routewatch_tags_missing, and it is never delayed or refused.
Model: opus-5-5
Each request gets tags, logged as `tags`: `asn=`, `country=`, one per `SWW_TAG_LISTS` list holding the client, `crowdsec`, `dnsbl=<zone>`, `abuseipdb` and `exploitsignature`. `SWW_TAG_DENY` refuses by tag and `SWW_TAG_LIMIT_PERCENT` lowers limits by tag. They replace `SWW_ASN_LIMIT_PERCENT`, `SWW_COUNTRY_LIMIT_PERCENT` and `SWW_DENIED_COUNTRIES`, with no aliases. A tag in those settings that nothing configured can give stops the start.
Judgement call: `SWW_ASN_LIMIT_PERCENT_URL` stays, its entries counting like `asn=` tags.
Judgement call: `SWW_TAG_LIMIT_PERCENT` also lowers byte limits, as the settings it replaces did.
Trips a reader: `sww_country_list_refusals_total` now counts only `SWW_EXCLUSIVELY_ALLOWED_COUNTRIES` refusals.
Model: opus-5-5
`script/publish` runs `script/docker` against the Docker host and pushes the image to the registry under four tags: the UTC date, the short commit hash, the branch name and `latest`. It refuses a working tree with uncommitted changes, since the hash tag names a commit.
Also carries sneak's `golines -m 77` line wrapping in three files, with no code change.
Unverified: a real build on the Docker host and a real push from this commit.
Deviation: sneak's FIXME comments are out, because the linter refuses them; they are #227, #228 and #229.
Model: opus-5-5
Co-authored-by: sneak <sneak@sneak.berlin>
Each discriminator check is now a type in its own file behind one small interface, listed in `heuristics.go`. Adding a heuristic is one file and one line. Behaviour does not change.
Refusal and ban log lines gain `heuristic` and `heuristic_figures`. A ban after the answer, by `refusal_limit` or `byte_limits`, logs `after_answer_heuristic` and its figures, so one line can name both.
Deviation: four lists, not the planned three, so `refusal_limit` still bans at the decision.
Rule suppressed: `//nolint:lll` on one struct tag.
Model: opus-5-5
xz-utils, which nix-bin depends on, links its man pages with update-alternatives when it installs, and the image's dpkg config drops man pages, so each link printed a warning. A config file re-includes only the man1 pages for this install; the same step deletes them, their dangling links in /etc/alternatives and the config file. No man page or package is added or removed.
Model: deepseek-v4-pro
The image's `ca-certificates`, `nix-bin` and `runit` now come from Ubuntu's archive instead of a dated snapshot, whose service had failed image builds. apt fetches them over plain HTTP and checks each against the archive's signed package lists, so no CA certificates are borrowed from the Go image. The README, `SPEC.md` and `docs/image.md` say so; the amd64-only limit, which came from the removed hash check, is gone.
Rule cut against: `REPO_POLICIES.md` pins every external reference by hash; sneak's ruling at sneak/prompts#72 (comment) allows Ubuntu's own package index.
Model: deepseek-v4-pro (implementation), opus-5-5 (rework)
New defaults, as sneak approved: 120 requests a minute, 1,500 an hour and 6,000 a day; bytes off a minute, 50G an hour and 100G a day. The README, `SPEC.md`, `docs/settings.md` and `docs/gitea.md` give the new figures and reasoning, and point Actions runners and shared addresses at `SWWAF_RATE_LIMIT_EXEMPT_NETS`. The README examples now hold against the new defaults.
An Actions runner that is not exempted makes more requests an hour than the new limit and will be banned.
Model: opus-5-5
The image's `ca-certificates`, `nix-bin` and `runit` now come from Ubuntu's archive instead of a dated snapshot, whose service had failed image builds. apt fetches them over plain HTTP and checks each against the archive's signed package lists, so no CA certificates are borrowed from the Go image. The README, `SPEC.md` and `docs/image.md` say so; the amd64-only limit, which came from the removed hash check, is gone.
Rule cut against: `REPO_POLICIES.md` pins every external reference by hash; sneak's ruling at sneak/prompts#72 (comment) allows Ubuntu's own package index.
Model: deepseek-v4-pro (implementation), opus-5-5 (rework)
The package passes nothing on since the one-process mode went: it answers rproxy's questions, takes its reports and answers servewaf's own endpoints. It is now named after the command it serves, with `git mv`, and its constructor is `servewaf.New`. `SPEC.md` and the other documents follow. No change of behaviour.
Model: opus-5-5
A compose setup that runs gitea 1.27.3 behind traefik, rproxy and servewaf on one shared network with IPv4 and IPv6, and `script/gitea-setup` (`make gitea-setup`), which brings it up and checks that a page load, a push and a clone pass through rproxy and are checked by servewaf. Containers on the shared network skip servewaf's checks; the gateways do not. gitea gets a secret key made per install and has sign-up off. `docs/gitea.md` explains each setting.
Judgement call: the Core Rule Set runs in `detect` mode here until #198 lands.
Model: deepseek-v4-pro (implementation), opus-5-5 (rework)
Merged by sneak's order of 6 November ("take the current state of smallwebwaf and get it on to main. i am going to deploy it soon in its alpha state").
Model: opus-5-5
`smallwebwaf` with no command now prints the commands and exits with status 2.
The image runs rproxy, or servewaf with SMALLWEBWAF_COMMAND=servewaf. The
passing-on code, its settings and its log fields and metric go. internal/proxy
keeps the checks, answers, reports and own endpoints; its tests run through the
real rproxy. The documents describe the pair.
Judgement call: clients.json keeps token_refused, since state files refuse unknown fields.
Deviation: script/gitea-requests sets SWWAF_LOG_REQUEST_HEADERS, to label requests.
Partially verified: make example-app and make gitea-requests never built the image, as Ubuntu's snapshot service answered 5xx.
Model: opus-5-5
`smallwebwaf rproxy` reads the first 64 KiB of each body, asks servewaf over
HTTP/1.1, and passes what servewaf allows to the app within its limits. Any
other answer goes to the visitor as servewaf gave it. SWRP_DEFAULT_DISPOSITION
decides only when servewaf cannot be connected to, does not answer at all, or
does not answer within SWRP_TIMEOUT_MS. A method SPEC.md does not list gets 405.
Duplication: internal/rproxy copies passing-on code from internal/proxy until
#172 removes the originals.
Rule suppressed: gosec G704 and G705; gosec at
internal/smallwebwaf/rproxy_test.go:431; tagliatelle at
internal/requestlog/requestlog.go:203; paralleltest on one test.
Judgement call: an answer over 1 MiB, or with a status below 200, gets 502.
Judgement call: every target not starting with one / gets 400.
Model: opus-5-5
A question waits on GeoJS at most 800ms, the new default of
SWWAF_LOOKUP_TIMEOUT. In enforce mode it does not wait at all when it is
over the header limits, on a fetched blocklist with deny, on the CrowdSec
decision list, or over a whole rate limit. Not waiting never lowers a
client's limit. Just before the Core Rule Set, servewaf refuses more than
50 cookies or 100 header fields with 431.
Shared default changed: SWWAF_LOOKUP_TIMEOUT also applies to smallwebwaf
with no command.
Judgement call: SWWAF_REPUTATION_TIMEOUT stays 2s, since no request waits
on the DNS blocklists or AbuseIPDB.
Model: opus-5-5
`smallwebwaf servewaf` runs today's checks on the visitor's request each
question carries, answers rproxy, and passes nothing to an app. rproxy reports
every request servewaf answered. A report counts an allowed request's bytes and
writes servewaf's second log line. The table holds a copy of what a report
needs, never the request, so a report cannot reach a request being handled.
Judgement call: a question's body is read before any check; over 64 KiB gets 400.
Judgement call: a report's bytes and durations have ceilings, so they cannot overflow.
Judgement call: a `close` disposition for SWWAF_BAN_RESPONSE=close.
Model: opus-5-5
The default request limits now suit programmatic use of an API, such
as an automation fleet on another host, and still catch abusive bots:
- per minute: 500, room for a fleet's busiest minute and a few first
visits with an empty cache, and passed within seconds by a client
firing as fast as it can;
- per hour: 20000, passed by a client that keeps up 6 a second;
- per day: 300000, passed by a scraper that keeps up 4 a second.
The byte limits and ban lengths are unchanged. SPEC.md, README.md and
docs/settings.md give the reasons; docs/gitea.md redoes the Actions
runner sums.
Judgement call: the docs/alerts.md ban example shows the minute limit.
Model: opus-5-5
TestHealthCheck closed the app's server, and later smallwebwaf, and
expected connections to the freed ports to fail. Another test could take
such a port in between, so the connection succeeded and the test failed.
The failing app check now uses the multicast address 224.0.0.1, to which
TCP cannot connect. The failing smallwebwaf check uses port 0, on which
nothing listens. Neither result depends on other processes. The test now
stops smallwebwaf after its last check, so that it exits before the
test's temporary directories are removed. The code under test is
unchanged.
Model: opus-5-5
The request limits and the hour and day byte limits now sit about twice
above what 40 people busy with heavy web apps make together on one
address:
- requests: 6000 a minute, 100,000 an hour, 1,000,000 a day;
- bytes: 20 GiB an hour, 200 GiB a day.
The largest response drops from 5 GiB to 1 GiB. The minute byte limit
of 4 GiB is sized to hold two of the largest responses with their
requests. SPEC.md gives the new reasoning; README.md and the reference
documents give the new figures.
Model: opus-5-5
SPEC.md is the only place that says how smallwebwaf behaves and why; docs/ holds lists linking to it; the README is short and links to both. Passed independent review in round 8.
Model: fable-5-1 (writing); opus-5-5 (review, merge)
The AbuseIPDB section of README.md listed only a broken rate limit or
byte limit, a ban rule match, and a refusal by a block rule or the Core
Rule Set. A client is checked once its history counts any offence, so it
now also names the error burst, a request for a trap path and a refusal
for a missing or wrong token, as "What it does so far" does. The
definition of an offence under "Bans" in SPEC.md now names a refusal for
a missing or wrong token too, as the code counts one.
Model: opus-5-5
A netblock's next ban for a broken limit within
SWWAF_LIMIT_BAN_REPEAT_WINDOW lasts three times its last limit or admin
ban. A hand-written entry with an expires at or before its start made
that length zero or less, so the next limit ban refused nothing. The
bans.json checks now refuse such an entry: it stops the start, and an
edit holding one is set aside as bans.json.bad while running. README's
"State files" lists it with the other refused entries.
Model: opus-5-5
The run script gave ${SWWAF_STATE_DIR:-/var/lib/smallwebwaf} to the
smallwebwaf user and never read SWWAF_STATE_DIR_FILE, so a new volume
at the directory that file names stayed root's. The run script now
takes the setting from the variable or the file, and changes the owner
only when it is the absolute path of a directory that exists;
otherwise it starts smallwebwaf, which stops with its message naming
the setting. A chown that fails, as on a read-only volume, no longer
ends the run script. script/example-app, and its entry in README.md,
gain two steps: a container on such a volume becomes healthy, and a
missing state directory reaches smallwebwaf's message.
Model: opus-5-5
Go's server adds a Content-Type it guesses from the body when the
response's headers have no Content-Type key, which happened to some
of the app's answers and not others, as the first part of the body or
the headers went out first. When the app sends none, smallwebwaf now
sets the key with no value on the client's response, which sends none.
Model: opus-5-5
A client too slow to send its request line and headers within
SWWAF_CLIENT_REQUEST_TIMEOUT gets either no answer or an error answer
from Go's server, mostly 400 Bad Request; either way its connection is
closed, nothing reaches the app, and no request log line is written.
The README's "Request log" and SPEC's size and time limits said such a
client always gets no answer. Both documents' summaries of
smallwebwaf's own answers now cover only limits passed after the
request line and headers have been read, and give 408 only for a slow
body; SPEC names Go's server as the one answering 431.
Model: opus-5-5
ReverseProxy sends the path URL.EscapedPath gives, which decodes and
escapes again a path holding a character such as |, turning %2F into /.
The path as sent now goes in URL.Opaque, which Go sends as it is; the
rate limit and WAF exempt paths and the log line's path use the same
path. A path starting with // is left as before: Go would send it in
Opaque as a scheme and host.
Model: opus-5-5
ReverseProxy refuses a 101 whose Upgrade is not the one the client asked
for, but modifyResponse had already counted the 101 as sent, so
answerError sent nothing and the client got an empty 200. The 101 now
counts as sent only once ReverseProxy has taken over the client's
connection, and answerError closes the app's switched connection, which
ReverseProxy leaves open when it refuses the switch.
Model: opus-5-5
runit stops smallwebwaf and the app at the same time, so the five seconds
smallwebwaf gives requests in progress help only while the app still
answers them. Both documents now say so, say what the client gets when the
app exits at once on SIGTERM, and say what an app does to let its requests
in progress finish.
Model: opus-5-5
When less than 256 KiB of a client's body is left after the app has
answered, the rest is read by closing the body, which the transport,
ReverseProxy and Go's server each do, not through requestBody.Read.
waiting was then false, so a request timeout that ran out took the 504
branch, which set no read deadline, and the request held its connection
for as long as the client kept it open. Once the app has answered, that
branch now ends reading the client's body at once, whichever of
SWWAF_CLIENT_REQUEST_TIMEOUT and SWWAF_UPSTREAM_REQUEST_TIMEOUT ran out.
Model: opus-5-5
script/gitea-requests runs gitea 1.27.3 behind the image at its
defaults, sends through it the requests that reading gitea's source
found the Core Rule Set refuses, with a short sample of ordinary use,
and prints each request with the rule ids that matched it. make
gitea-requests runs it; make check does not.
Every case still holds, and each fix would let through what SPEC.md
says stays refused, so the notes specific to gitea now name them with
how each refusal plays out: file names in query parameters that read as
Java, script, PowerShell or SQL, such paths, and an image push that
reuses a layer of an image named like a command.
Model: opus-5-5
earlierBans took the counts in the notes of the netblock's first ban
held, so a ban dropped after an admin's ban, which came first, went
uncounted. Each ban held now gives a count, its notes' counts plus one
for it and each ban held after it, and the count that names the most
bans is taken whole. Counting from the newest ban alone would lose the
bans before an admin's ban added to bans.json without notes; taking
each cause from a different count would count twice a ban an admin
keeps, which later bans' notes count under its old cause.
Model: opus-5-5
When the app answered before the client had sent its whole body, and its
answer ended with its last bytes, every timeout was stopped,
SWWAF_CLIENT_REQUEST_TIMEOUT included. The transport went on waiting to
read the body, and Go's server cannot send the answer until that read
returns, so a client that stopped sending held the request until it
closed the connection. The timeouts now run on until the client has sent
its body, as they do before the app answers: once
SWWAF_CLIENT_REQUEST_TIMEOUT runs out the client is sent the app's
answer, the connection is closed, and the log line says timed_out.
Model: opus-5-5
Both apt-get calls in the Dockerfile now pass -o Acquire::Retries=5.
apt already retries a download that fails with 503 three times,
waiting 1, 2 and 4 seconds, which a short outage of the snapshot
service outlasted; five retries wait about half a minute in all.
Model: opus-5-5
checkClient read the time before the lookup, which can wait for GeoJS
up to SWWAF_LOOKUP_TIMEOUT, and the rate limits counted the request at
that time. Two requests of a new client waiting across the end of a
minute could then be counted out of order, and the one dated more than a
second before the minute under way began made the rate limits start
that window afresh, as for a clock set back: the other request's count
was lost. The checks after the lookup now read the time again.
Model: opus-5-5
A path rule matched only the path as the client sent it, so /%2eenv got
past the default env-file rule while an app that decodes the path
serves /.env. A path rule now matches the path as sent or decoded once,
as a uri rule already did, and so does a path in SWWAF_TRAP_PATHS:
/%77p-login.php matches /wp-login.php. The query is cut off before the
path is decoded, and the path is decoded only once, so /%252eenv is not
/.env. A ban's notes keep the path as sent. SPEC.md and README.md say
so.
Model: opus-5-5
SWWAF_WAF_BODY_LIMIT (default off, at most 1G) has the Core Rule Set read
form data and multipart up to the limit, the rest streaming on, and JSON
and XML no larger than it, with text/json and the application and text
types ending in +json or +xml. The part read is held for the app. A size
or time limit met while reading ends the request. Content-Encoding is
refused again on these kinds. A body Coraza cannot parse, or a multipart
body failing its strict checks, adds 5, as does a multipart body the limit
cuts in a part's headers before a colon or a line feed. Coraza is built
with no_fs_access, so writes no file. Rule 900300 moves to phase 2.
Judgement call: Content-Encoding is refused on a JSON or XML body too
large to read, as SPEC.md allows.
Model: opus-5-5
Coraza v3.8.1 runs the Core Rule Set 4.25.0 (coraza-coreruleset v4.25.0)
after the rule files, with the six changes and the default
SWWAF_WAF_DISABLED_RULES that SPEC.md gives; no body, no response. The
parameter names in the third and fourth changes are matched in any case,
as Coraza does. SWWAF_WAF_DISABLED_RULES refuses 900000 to 900999,
smallwebwaf's own rules. A request with more than 1000 query parameters
adds 5 (rule 900300). In block mode a match is refused with 403, an
offence counted toward the error burst; in detect mode it is let
through. Both log waf_rule_ids, waf_score and duration_waf, raise
waf_block, and count smallwebwaf_waf_matches_total.
Judgement call: waf_block is raised in block mode too.
Deviation: no engine-error path; with no body read, Coraza cannot fail.
Model: opus-5-5
Twelve tests in internal/proxy needed the GeoJS stand-in to be asked or
to answer within the default SWWAF_LOOKUP_TIMEOUT of one second on the
real clock. A hold-up of the test process past it abandoned the request
to the stand-in, or left the client unknown. Each now sets
SWWAF_LOOKUP_TIMEOUT to an hour, and the comment on startGeoJS asks the
same of later tests.
Judgement call: set in each test, not as a default in newProxy, where it
would change two tests that rely on the default second.
Model: opus-5-5
SWWAF_TRAP_PATHS: a request whose path, as a path rule sees it, is one
of them is a clear sign of attack, banned as a ban rule's match is; the
ban's notes give its trap_path. Checked after the rate limits, before the
rule files.
SWWAF_ERROR_BURST_THRESHOLD (default 30, or off): more refusals in a
minute after a block or ban rule or a trap path, or for a missing or
wrong token, ban the client as a broken limit does. Counted in
clients.json's minute_refusals; limit_hit error_burst, notes kind
refusals.
A token refusal is now the offence token_refused, and
smallwebwaf_offences_total counts every kind the history does.
Judgement call: the threshold is not lowered by a client's limit percentage.
Model: opus-5-5
SWWAF_CROWDSEC_LAPI_URL and SWWAF_CROWDSEC_LAPI_KEY name an engine whose
decision list, <url>/v1/decisions, is fetched every minute with the key in
X-Api-Key, following no redirect, and kept as a blocklist is: used while a
fetch fails, and across restarts through reputation.json. Ban decisions on an
Ip or a Range end at the fetch time plus their duration. A listed client's
request is refused and bans its netblock with the cause crowdsec until the
decision ends; bans.json, ban notes and metrics take the cause.
Judgement call: fetched every minute, not a setting.
Judgement call: a crowdsec ban never lengthens a limit ban.
Judgement call: a lifted crowdsec ban is remade while its decision lasts.
Model: opus-5-5
A ban's notes, in bans.json and in its alert, gain `reputation`: each
blocklist, DNSBL zone or AbuseIPDB that listed the client when the ban
was made, as its `source`, named and ordered as in the request log's
`reputation`, with AbuseIPDB's `score`. It is left out when none did.
README.md shows it in a bans.json example.
Notes now hold a list, so bans can no longer be compared with ==: the
tests compare them with reflect.DeepEqual.
Judgement call: the score is a pointer, so a score of 0, a hit while
SWWAF_ABUSEIPDB_MIN_SCORE is 0, is still written.
Model: opus-5-5
The IPv6 group that is one client, the size of the table of clients and
the level of the process's own lines become settings. clientGroup reads
the group length from them, so limits, bans, history, lookups, AbuseIPDB
scores and per-client anomaly counters all follow it; ratelimit.New
takes the table size; the process logger takes the level once the
settings are read, and request lines, written apart from it, are never
held back.
Judgement call: SWWAF_IPV6_GROUP_PREFIX accepts 32 to 128, the issue's example range.
Model: opus-5-5
sneak
clawbot