clawbot 8be53283d3 Targets whose home page redirects are checked at /robots.txt (closes #194) ...

The browser follows a redirect, so the figures for the AWS Console, Akamai,
B2 and the S3 regions also timed the page it led to. Each of these targets
is now checked at /robots.txt on its own host, which answers without a
redirect. So is the Google Cloud Console, whose home page redirects curl's
requests. The S3 regions are checked with a HEAD request, as Chrome closes
their connection after most GET requests. README.md says what a figure
includes.

Model: opus-5-5

2026-10-26 02:52:06 +00:00

..

src

Targets whose home page redirects are checked at /robots.txt (closes #194)

2026-10-26 02:52:06 +00:00

index.html

Move to the layout REPO_POLICIES.md gives (closes #30)

2026-10-14 12:01:32 +02:00