waf: the asks to GeoJS, the DNSBL zones, AbuseIPDB and routewatch end before the last state write #439

Open
opened 2026-11-19 17:13:13 +01:00 by clawbot · 1 comment
Collaborator

From the architecture review, "Construction, start and stop", under "Background work is started and waited for by name": #283 (comment). Approved for building: #283 (comment). The code was read at 809d266 on next. Line numbers are that commit's.

The fx wiring (#421) built the first half of this proposal: each background loop is a lifecycle hook that is waited for at the stop. This issue is the other half.

What is there

Four parts start a goroutine from inside a request that nothing waits for:

  • internal/geojs/geojs.go:336: go g.askAboutWaiting(context.WithoutCancel(ctx));
  • internal/reputation/dnsbl.go:166: go d.ask(context.WithoutCancel(ctx), q);
  • internal/reputation/abuseipdb.go:177: go a.check(context.WithoutCancel(ctx), client, addr);
  • internal/routewatch/routewatch.go:153: go c.ask(context.WithoutCancel(ctx), client, addr).

Why it matters

  • At an orderly stop, these goroutines can still be asking when waf writes its state and cache files. An answer that comes then changes a table after it was written, or after the process has begun to exit.
  • GeoJS's answer also calls back into the table of clients and the ban ledger, and a failure raises an alert into a queue nobody sends from any more.
  • The SQLite state store (#217) says nothing uses the store or its tables after it is closed. A goroutine nobody waits for can.

What changes

  • Each of the four types gets one method that keeps it from starting another ask, ends the asks under way, and returns once their goroutines have returned.
  • Each provider in internal/sww/waf.go (newGeoJS, newDNSBL, newAbuseIPDB, newRouteWatch) adds that method as a stop hook, as the loops' providers do with background. fx stops serving first, so no request starts an ask after it. The last write of the state and cache files stays run's last step, after every hook.
  • An ask ended by the stop is not a failure of its source: it raises no alert, counts no failure in the metrics, and sets no wait before the source is asked again.

Deviation from the review, and why

  • The review asked that every ask a part had started be waited for, so that an orderly stop loses nothing.
  • That wait cannot be kept inside the stop. A request to GeoJS may take 10 seconds, a query to routewatch 5, and GeoJS goes on to its next batch of clients after each. waf gives requests five seconds at the stop, and runit and docker stop kill the process a little after that. A stop that waits out an ask would be killed before the last write, and lose the state.
  • So the asks are ended, not waited out. What is lost is what is lost today: an answer that had not come yet, which is asked for again after the next start. For AbuseIPDB that is one check of the day's budget.
  • Reversal cost: low. Waiting up to a limit in place of ending at once is a few lines in each of the four methods.

What stays

  • While waf serves, nothing changes: a request still never waits for these asks, and an ask still goes on after the request that started it has ended.
  • The two goroutines a request starts for itself are left alone: the wait for GeoJS in internal/waf/lookup.go:81 and the checks beside the lookup in internal/waf/besidelookup.go:61. Neither changes a table.
  • The remote log sender stays outside fx, as the fx wiring left it on purpose.

Done when

  • after its method returns, each of the four has no goroutine of its own, and starts none;
  • a test for each type shows that: an ask under way when the method is called is ended, raises no alert and counts no failure, and a later call starts none;
  • make check passes;
  • the PR body lists the four methods' names. Each says what the method does to a reader of that one file, the standard of the symbol names audit (#213).

If this is more than one PR can carry, say so on this issue instead of starting.

Model: fable-5-2

From the architecture review, "Construction, start and stop", under "Background work is started and waited for by name": https://git.eeqj.de/sneak/smallwebwaf/issues/283#issuecomment-173831. Approved for building: https://git.eeqj.de/sneak/smallwebwaf/issues/283#issuecomment-183550. The code was read at `809d266` on `next`. Line numbers are that commit's. The fx wiring (https://git.eeqj.de/sneak/smallwebwaf/pulls/421) built the first half of this proposal: each background loop is a lifecycle hook that is waited for at the stop. This issue is the other half. ### What is there Four parts start a goroutine from inside a request that nothing waits for: - `internal/geojs/geojs.go:336`: `go g.askAboutWaiting(context.WithoutCancel(ctx))`; - `internal/reputation/dnsbl.go:166`: `go d.ask(context.WithoutCancel(ctx), q)`; - `internal/reputation/abuseipdb.go:177`: `go a.check(context.WithoutCancel(ctx), client, addr)`; - `internal/routewatch/routewatch.go:153`: `go c.ask(context.WithoutCancel(ctx), client, addr)`. ### Why it matters - At an orderly stop, these goroutines can still be asking when `waf` writes its state and cache files. An answer that comes then changes a table after it was written, or after the process has begun to exit. - GeoJS's answer also calls back into the table of clients and the ban ledger, and a failure raises an alert into a queue nobody sends from any more. - The SQLite state store (https://git.eeqj.de/sneak/smallwebwaf/issues/217) says nothing uses the store or its tables after it is closed. A goroutine nobody waits for can. ### What changes - Each of the four types gets one method that keeps it from starting another ask, ends the asks under way, and returns once their goroutines have returned. - Each provider in `internal/sww/waf.go` (`newGeoJS`, `newDNSBL`, `newAbuseIPDB`, `newRouteWatch`) adds that method as a stop hook, as the loops' providers do with `background`. fx stops serving first, so no request starts an ask after it. The last write of the state and cache files stays `run`'s last step, after every hook. - An ask ended by the stop is not a failure of its source: it raises no alert, counts no failure in the metrics, and sets no wait before the source is asked again. ### Deviation from the review, and why - The review asked that every ask a part had started be waited for, so that an orderly stop loses nothing. - That wait cannot be kept inside the stop. A request to GeoJS may take 10 seconds, a query to routewatch 5, and GeoJS goes on to its next batch of clients after each. `waf` gives requests five seconds at the stop, and runit and `docker stop` kill the process a little after that. A stop that waits out an ask would be killed before the last write, and lose the state. - So the asks are ended, not waited out. What is lost is what is lost today: an answer that had not come yet, which is asked for again after the next start. For AbuseIPDB that is one check of the day's budget. - Reversal cost: low. Waiting up to a limit in place of ending at once is a few lines in each of the four methods. ### What stays - While `waf` serves, nothing changes: a request still never waits for these asks, and an ask still goes on after the request that started it has ended. - The two goroutines a request starts for itself are left alone: the wait for GeoJS in `internal/waf/lookup.go:81` and the checks beside the lookup in `internal/waf/besidelookup.go:61`. Neither changes a table. - The remote log sender stays outside fx, as the fx wiring left it on purpose. ### Done when - after its method returns, each of the four has no goroutine of its own, and starts none; - a test for each type shows that: an ask under way when the method is called is ended, raises no alert and counts no failure, and a later call starts none; - `make check` passes; - the PR body lists the four methods' names. Each says what the method does to a reader of that one file, the standard of the symbol names audit (https://git.eeqj.de/sneak/smallwebwaf/issues/213). If this is more than one PR can carry, say so on this issue instead of starting. Model: fable-5-2
clawbot self-assigned this 2026-11-19 17:13:13 +01:00
Author
Collaborator

Built in #463 on branch issue-439-stop-asks-at-stop, into next.

  • The four types get GeoJS.StopAsking, DNSBL.StopAsking, AbuseIPDB.StopChecking and routewatch's Client.StopAsking. Each is added as a stop hook by its provider in internal/sww/waf.go.
  • The asks are ended at the stop, not waited out, as this issue's deviation section says.

Model: opus-5-5

Built in https://git.eeqj.de/sneak/smallwebwaf/pulls/463 on branch `issue-439-stop-asks-at-stop`, into `next`. - The four types get `GeoJS.StopAsking`, `DNSBL.StopAsking`, `AbuseIPDB.StopChecking` and routewatch's `Client.StopAsking`. Each is added as a stop hook by its provider in `internal/sww/waf.go`. - The asks are ended at the stop, not waited out, as this issue's deviation section says. Model: opus-5-5
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sneak/smallwebwaf#439