Targets whose home page redirects are checked at /robots.txt (closes #194)

A check follows a redirect and times the request to the new URL too. So the AWS Console, the Google Cloud Console, Akamai, B2 and the eight S3 regions are now checked at /robots.txt on their own host, which answers without a redirect. Before, the S3 and B2 figures also timed a page on another host. The S3 regions are checked with HEAD, because Chrome closes the connection after a GET of their /robots.txt and the figure would then include connection setup every time. README.md says what a figure includes.

Reports name a target by its URL, so these targets' samples are stored under the new URLs.

Model: opus-5-5

configs/security-headers.conf

@@ -8,10 +8,9 @@ add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" alway
 # Scripts and styles load only from the page's own origin. Inline ones
 # are blocked, style attributes in markup included, so style elements
 # through classes or element.style. data: images are for the favicon
-# in web/index.html. connect-src is * because the browser checks each
-# probe in web/src/main.js against it, and also every redirect the probe
-# follows, and several of those hosts redirect to others; a list of
-# hosts here would block those probes. It also covers the reports the
+# in web/index.html. connect-src is * because the page checks every
+# target in web/src/main.js. A list of hosts here would have to change
+# with every change to those targets. It also covers the reports the
 # page sends to its own origin.
 add_header Content-Security-Policy "default-src 'self'; connect-src *; img-src 'self' data:; object-src 'none'; base-uri 'none'; form-action 'none'; frame-ancestors 'none'" always;