Caches of outside APIs live in /var/cache/sww #314

Closed
opened 2026-11-11 00:57:23 +01:00 by clawbot · 2 comments
Collaborator

sneak, on #217:

put all of the caches of responses/results from external http apis (abuseipdb, geojs, routewatch, etc) either into their own files, or a single sqlite file together, and make sure it lives in /var/cache/ so it can be properly handled (excluded from backups etc).

Reading: #217 (comment). The state files today all live in SWW_STATE_DIR (/var/lib/sww), caches and state together.

Definition of done, one PR into next, on the JSON files as they are (the SQLite spec on that issue is not approved yet):

  • Every cache of answers from an outside HTTP API (AbuseIPDB, GeoJS, routewatch, DNS blocklists if their answers are kept, and any other) is written to its own file in /var/cache/sww. State that must survive (bans, counts, anything not re-fetchable) stays in SWW_STATE_DIR.
  • The directory is a setting only if SWW_STATE_DIR is one: then SWW_CACHE_DIR, default /var/cache/sww, validated the same way. The image's entrypoint creates it and gives it to the app user, as it does the state directory.
  • A missing or unreadable cache file is not an error: the cache starts empty and fills again.
  • docs/state-files.md, SPEC.md and README.md say which files are caches, where they live, and that the cache directory can be left out of backups. The compose examples need no volume for it.
  • Before 1.0 there is no installed base: no migration of old files.

If the rename of the discriminator to waf (#227) has landed by then, use the new names. Follow AGENTS.md: checks only through make targets. make fmt, then flock -w 1800 /srv/code/tmp/.gate-smallwebwaf.lock make check; a flock timeout also exits 1, so read the output. Push over SSH and post with tea. Label the PR needs-review.

Model: opus-5-5

sneak, on https://git.eeqj.de/sneak/smallwebwaf/issues/217: > put all of the caches of responses/results from external http apis (abuseipdb, geojs, routewatch, etc) either into their own files, or a single sqlite file together, and make sure it lives in /var/cache/<app> so it can be properly handled (excluded from backups etc). Reading: https://git.eeqj.de/sneak/smallwebwaf/issues/217#issuecomment-176005. The state files today all live in `SWW_STATE_DIR` (`/var/lib/sww`), caches and state together. **Definition of done**, one PR into `next`, on the JSON files as they are (the SQLite spec on that issue is not approved yet): - Every cache of answers from an outside HTTP API (AbuseIPDB, GeoJS, routewatch, DNS blocklists if their answers are kept, and any other) is written to its own file in `/var/cache/sww`. State that must survive (bans, counts, anything not re-fetchable) stays in `SWW_STATE_DIR`. - The directory is a setting only if `SWW_STATE_DIR` is one: then `SWW_CACHE_DIR`, default `/var/cache/sww`, validated the same way. The image's entrypoint creates it and gives it to the app user, as it does the state directory. - A missing or unreadable cache file is not an error: the cache starts empty and fills again. - `docs/state-files.md`, `SPEC.md` and `README.md` say which files are caches, where they live, and that the cache directory can be left out of backups. The compose examples need no volume for it. - Before 1.0 there is no installed base: no migration of old files. If the rename of the discriminator to `waf` (https://git.eeqj.de/sneak/smallwebwaf/issues/227) has landed by then, use the new names. Follow `AGENTS.md`: checks only through `make` targets. `make fmt`, then `flock -w 1800 /srv/code/tmp/.gate-smallwebwaf.lock make check`; a `flock` timeout also exits 1, so read the output. Push over SSH and post with `tea`. Label the PR `needs-review`. Model: opus-5-5
clawbot self-assigned this 2026-11-11 00:57:23 +01:00
Author
Collaborator

Plan. The rename to waf (#319) has landed, so this starts from current next with the new names. Packages: internal/config for the cache directory setting, and the packages that keep each cache (internal/reputation, internal/lookup, internal/routewatch, and any other that fetches from an outside HTTP API). The Dockerfile's entrypoint creates the directory as it does the state directory. If #317 (UTC and days) lands first, rebase onto it. Follow AGENTS.md: checks only through make targets. make fmt, then flock -w 1800 /srv/code/tmp/.gate-smallwebwaf.lock make check; a flock timeout also exits 1, so read the output. Edit by hand. Title ending (closes #314).

Model: opus-5-5

**Plan.** The rename to `waf` (https://git.eeqj.de/sneak/smallwebwaf/pulls/319) has landed, so this starts from current `next` with the new names. Packages: `internal/config` for the cache directory setting, and the packages that keep each cache (`internal/reputation`, `internal/lookup`, `internal/routewatch`, and any other that fetches from an outside HTTP API). The `Dockerfile`'s entrypoint creates the directory as it does the state directory. If https://git.eeqj.de/sneak/smallwebwaf/pulls/317 (UTC and days) lands first, rebase onto it. Follow `AGENTS.md`: checks only through `make` targets. `make fmt`, then `flock -w 1800 /srv/code/tmp/.gate-smallwebwaf.lock make check`; a `flock` timeout also exits 1, so read the output. Edit by hand. Title ending ` (closes #314)`. Model: opus-5-5
Author
Collaborator

Built in #336.

  • Four cache files now live in SWW_CACHE_DIR (default /var/cache/sww): lookups.json, lists.json, dnsbl.json and abuseipdb-scores.json. reputation.json is gone.
  • The AbuseIPDB checks spent today are a count, so they stay in SWW_STATE_DIR as abuseipdb-checks.json.
  • routewatch answers are kept in memory only, so they have no file.

Model: opus-5-5

Built in https://git.eeqj.de/sneak/smallwebwaf/pulls/336. - Four cache files now live in `SWW_CACHE_DIR` (default `/var/cache/sww`): `lookups.json`, `lists.json`, `dnsbl.json` and `abuseipdb-scores.json`. `reputation.json` is gone. - The AbuseIPDB checks spent today are a count, so they stay in `SWW_STATE_DIR` as `abuseipdb-checks.json`. - routewatch answers are kept in memory only, so they have no file. Model: opus-5-5
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sneak/smallwebwaf#314