Targets whose home page redirects are checked at /robots.txt (closes #194)

A check follows a redirect and times the request to the new URL too. So the AWS Console, the Google Cloud Console, Akamai, B2 and the eight S3 regions are now checked at /robots.txt on their own host, which answers without a redirect. Before, the S3 and B2 figures also timed a page on another host. The S3 regions are checked with HEAD, because Chrome closes the connection after a GET of their /robots.txt and the figure would then include connection setup every time. README.md says what a figure includes.

Reports name a target by its URL, so these targets' samples are stored under the new URLs.

Model: opus-5-5

TODO.md

@@ -23,6 +23,14 @@ Run `make frontend-viewport-test` in CI as its own step; it is not part of
 
 # Completed Steps
 
+- 2026-10-26: a target whose home page answers with a redirect is checked at
+  `/robots.txt` on the same host, which answers without one
+  ([#194](https://git.eeqj.de/sneak/netwatch/issues/194)). These are the AWS
+  Console, Akamai, B2 and the 8 S3 regions. The browser followed each redirect,
+  so their figures also timed the page it led to. The Google Cloud Console is
+  checked at `/robots.txt` too, as its home page redirects `curl`'s requests.
+  The S3 regions are checked with a HEAD request, as Chrome closes their
+  connection after most GET requests. `README.md` says what a figure includes
 - 2026-10-26: the six Hetzner targets answer wherever the page is served from
   ([#191](https://git.eeqj.de/sneak/netwatch/issues/191)). They close the
   connection without an answer when the request has a `Referer` header. The