sneak/netwatch
1
0 Fork 0
Code Issues 5 Pull Requests 1 Actions 8 Packages Projects Releases Wiki Activity

Acceptance test: use netwatch end to end as a user before a release candidate #204

New Issue
Closed
opened 2026-11-11 20:55:35 +01:00 by clawbot · 1 comment
clawbot commented 2026-11-11 20:55:35 +01:00
Collaborator
Copy link
Copy source

Hands-on test before a release candidate, by the top-level manager's standing practice (sneak asked on 10 November whether vaultik had been tested by hand; it had not, and sfdupes then showed missing basics on first real use).

What to do

Build next fresh with the repo's own entrypoints and use netwatch as a new user would, following README.md literally, every command and claim in it. Use small synthetic data in local containers; nothing RAM-heavy, and no real third-party accounts or credentials (stop and say so where one is needed).

Definition of done

  • Every defect filed as its own issue: steps, what happened, what should happen.
  • Every missing basic a user would expect filed as its own issue titled proposal: ..., assigned to sneak.
  • One comment here: a plain verdict (ready for a release candidate, or not), the issues filed, each marked defect or proposal, and anything in the README that turned out wrong. No code changes.

Model: opus-5-5

Hands-on test before a release candidate, by the top-level manager's standing practice (sneak asked on 10 November whether vaultik had been tested by hand; it had not, and sfdupes then showed missing basics on first real use). ## What to do Build `next` fresh with the repo's own entrypoints and use netwatch as a new user would, following `README.md` literally, every command and claim in it. Use small synthetic data in local containers; nothing RAM-heavy, and no real third-party accounts or credentials (stop and say so where one is needed). ## Definition of done - Every defect filed as its own issue: steps, what happened, what should happen. - Every missing basic a user would expect filed as its own issue titled `proposal: ...`, assigned to sneak. - One comment here: a plain verdict (ready for a release candidate, or not), the issues filed, each marked defect or proposal, and anything in the README that turned out wrong. No code changes. Model: opus-5-5
clawbot commented 2026-11-11 21:17:13 +01:00
Author
Collaborator
Copy link
Copy source

Verdict: not ready for a release candidate. The six defects below are open. One of them, the Bahrain target, fails for every user. #190 also remains open.

Tested on next at 93bbd80, with the page in headless Chrome.

Issues filed

  • defect: #205: the S3 Bahrain target never answers, so Reachable never reaches 26/26.
  • defect: #206: the health box and summary wait for the round's slowest check, and say OFFLINE for a while after the network is back.
  • defect: #207: the dot beside each host name is always gray.
  • defect: #208: the footer says every check is a GET and that CORS affects figures.
  • defect: #209: the backend logs 43 lines of framework internals at each start, and a bad setting's reason is buried in them.
  • defect: #210: four README gaps, listed below.
  • proposal: #211: a way to read the reports the backend stores.
  • proposal: #212: remember the interval and the pinned rows across page loads.
  • proposal: #213: show Local CPE only when it answers.

What the README got wrong

  • Getting Started never says which address to open. After make run, http://localhost:8080/ answers 404, since netwatch-server serves no page.
  • "gray is no answer": in the graph, a failed check is a faint red band.
  • The report's timestamp has no stated form. It is an ISO 8601 string, and a number is refused with 400.
  • The YAML settings file is named, but not what goes in it.
  • Targets lists Bahrain, which has not answered for months (#205).

Disclosures

  • Not checked: the local targets against a real modem or router, since a container has none. Also not checked: browsers other than Chrome, make add-dependency (it changes dependencies), CORS_ALLOWED_ORIGINS, SENTRY_DSN, the 1 GiB size cap, the 30 s report abandonment, and a deployment under upaas.
  • Unverified: the cause of the Bahrain outage rests on press reports. The 48 s lag at the 60 s interval is computed from the timeout rule, not measured.
  • Deviation: the Go and yarn caches were pointed into the session's temporary directory, to keep files out of $HOME. This does not change behaviour.
  • Judgement call: the README gaps are one issue, fixable in one PR, not four.

Model: opus-5-5

**Verdict: not ready for a release candidate.** The six defects below are open. One of them, the Bahrain target, fails for every user. https://git.eeqj.de/sneak/netwatch/issues/190 also remains open. Tested on `next` at `93bbd80`, with the page in headless Chrome. ## Issues filed - defect: https://git.eeqj.de/sneak/netwatch/issues/205: the S3 Bahrain target never answers, so `Reachable` never reaches `26/26`. - defect: https://git.eeqj.de/sneak/netwatch/issues/206: the health box and summary wait for the round's slowest check, and say `OFFLINE` for a while after the network is back. - defect: https://git.eeqj.de/sneak/netwatch/issues/207: the dot beside each host name is always gray. - defect: https://git.eeqj.de/sneak/netwatch/issues/208: the footer says every check is a GET and that CORS affects figures. - defect: https://git.eeqj.de/sneak/netwatch/issues/209: the backend logs 43 lines of framework internals at each start, and a bad setting's reason is buried in them. - defect: https://git.eeqj.de/sneak/netwatch/issues/210: four README gaps, listed below. - proposal: https://git.eeqj.de/sneak/netwatch/issues/211: a way to read the reports the backend stores. - proposal: https://git.eeqj.de/sneak/netwatch/issues/212: remember the interval and the pinned rows across page loads. - proposal: https://git.eeqj.de/sneak/netwatch/issues/213: show `Local CPE` only when it answers. ## What the README got wrong - Getting Started never says which address to open. After `make run`, `http://localhost:8080/` answers 404, since `netwatch-server` serves no page. - "gray is no answer": in the graph, a failed check is a faint red band. - The report's `timestamp` has no stated form. It is an ISO 8601 string, and a number is refused with 400. - The YAML settings file is named, but not what goes in it. - Targets lists Bahrain, which has not answered for months (https://git.eeqj.de/sneak/netwatch/issues/205). ## Disclosures - Not checked: the local targets against a real modem or router, since a container has none. Also not checked: browsers other than Chrome, `make add-dependency` (it changes dependencies), `CORS_ALLOWED_ORIGINS`, `SENTRY_DSN`, the 1 GiB size cap, the 30 s report abandonment, and a deployment under upaas. - Unverified: the cause of the Bahrain outage rests on press reports. The 48 s lag at the 60 s interval is computed from the timeout rule, not measured. - Deviation: the Go and yarn caches were pointed into the session's temporary directory, to keep files out of `$HOME`. This does not change behaviour. - Judgement call: the README gaps are one issue, fixable in one PR, not four. Model: opus-5-5
Sign in to join this conversation.
No Branch/Tag Specified
Labels
Clear labels
critical
Extremely important: data loss, a security hole, production down, or basic use impossible
 merge-ready
needs-checks
needs-rebase
needs-review
needs-rework
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
clawbot sneak (Jeffrey Paul)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sneak/netwatch#204