Commit Graph
2 Commits
Author SHA1 Message Date
clawbot 8a62f12802 Targets whose home page redirects are checked at /robots.txt (closes #194)
check / check (push) Canceled after 0s
A check follows a redirect and times the request to the new URL too. So the AWS Console, the Google Cloud Console, Akamai, B2 and the eight S3 regions are now checked at /robots.txt on their own host, which answers without a redirect. Before, the S3 and B2 figures also timed a page on another host. The S3 regions are checked with HEAD, because Chrome closes the connection after a GET of their /robots.txt and the figure would then include connection setup every time. README.md says what a figure includes.

Reports name a target by its URL, so these targets' samples are stored under the new URLs.

Model: opus-5-5
2026-10-26 04:11:48 +01:00
clawbot 406e073aee Move to the layout REPO_POLICIES.md gives (closes #30)
check / check (push) Canceled after 0s
The Go module moves from backend/ to the root (go.mod, go.sum,
.golangci.yml, cmd/, internal/), the page's source to web/, and nginx's
two files to configs/. backend/ is gone: its LICENSE and .dockerignore
are deleted, and its .gitignore entries, README and scripts are folded
into the root ones. script/build builds the server binary after the
frontend; script/run and script/clean move up from backend/script/;
make test, lint, fmt and fmt-check do what the backend's did. Vite's
root is web/ and it still builds into dist/. go.mod ignores
node_modules/, whose npm packages can ship .go files. script/bootstrap
no longer installs gcc, which only the host Go tests needed.

Model: opus-5-5
2026-10-14 12:01:32 +02:00