Make the test gate unfakeable and stop test-integration lying (closes #93)
All checks were successful
check / check (push) Successful in 3m42s
All checks were successful
check / check (push) Successful in 3m42s
Closes #69. script/test ran `go test` without -count=1, so Go's test cache satisfied the gate without running anything: a repeat `make test` printed all 14 ok lines in 0.42 seconds, every one marked (cached). Those lines count as ok lines, so the evidence signal this repo relies on was forgeable. It sits below the Docker layer cache - CHECK_EPOCH forces `RUN make test` to re-execute, but a GOCACHE baked into an earlier image layer survives into the re-executed step, so the step can run and still do no work. -count=1 is applied unconditionally rather than only in the container, because the pre-commit hook runs the same script and a gate honest only in CI is dishonest where it is leaned on most. It costs about 11 seconds on every repeat run, which is what it costs for a repeat run to mean anything. test-coverage had the same omission and is fixed too; a coverage profile assembled from cached results describes a run that did not happen. Both invocations in script/test now share one run_tests function so the quiet run and the verbose rerun cannot drift apart in flags. make test-integration passed -tags=integration while no file in the repo carries any build tag, so it was an exact duplicate of make test. Removed rather than given a tag scheme: the whole suite is 12s on the host, so gating saves seconds in exchange for a mechanism whose failure mode is "some tests silently stopped running" - a poor trade in a repo that has found several ways for a gate to report an unearned green. -timeout goes 30s to 120s. This DIVERGES from REPO_POLICIES.md:192, which mandates 30s; the divergence is deliberate, recorded in script/test's comment, and proposed upstream as #101. Measured worst case is 10.2s and each fresh measurement has come in above the last, leaving 30s at 2.9x - too thin for a loaded runner. A -timeout is a hang backstop, not a performance budget. Note for the record: cold-cache compilation is NOT charged against -timeout. The flag reaches the test binary as -test.timeout and its clock starts inside testing.M.Run, after compilation. Verified twice independently - a run with an empty GOCACHE spent ~46s compiling and then reported per-package durations within noise of warm. A shell `timeout 30 go test ./...` does include compilation, but that is a different mechanism.
This commit was merged in pull request #98.
This commit is contained in:
33
TODO.md
33
TODO.md
@@ -18,6 +18,39 @@ Define remaining scope for a first tagged release and cut v0.1.0.
|
||||
|
||||
# Completed Steps
|
||||
|
||||
- 2026-08-09: Closed the fifth false-green mechanism (issues #93, #69).
|
||||
`script/test` omitted `-count=1`, so Go's test result cache could
|
||||
satisfy the gate outright: a second back-to-back `make test` printed
|
||||
the full set of 14 `ok` lines, every one marked `(cached)`, having
|
||||
executed no test at all. Since `ok <pkg> (cached)` is an `ok` line,
|
||||
the "14 `ok` lines means the suite ran" signal this repo leans on was
|
||||
forgeable, one level below the Docker layer cache that #85 addressed.
|
||||
Fixed with `-count=1` unconditionally rather than only in the
|
||||
container, because the pre-commit hook runs the same script and a
|
||||
gate honest only in CI is dishonest where people rely on it most;
|
||||
`test-coverage` got the same flag, and `script/check` inherits it by
|
||||
calling `script/test`. In the same area, `make test-integration` was
|
||||
deleted rather than made real: no file in the repo carried a build
|
||||
tag, so `-tags=integration` selected nothing and the target was an
|
||||
exact duplicate of `make test`. Tagging a subset was rejected because
|
||||
the entire suite runs in well under a minute, and a scheme whose
|
||||
failure mode is "some tests silently stopped running" is a poor trade
|
||||
for those seconds in a repo with this particular history. The
|
||||
`-timeout` was raised from 30s after measuring rather than after
|
||||
assuming: the standing claim that cold-cache compilation is charged
|
||||
against `-timeout` is **false**, disproved by a containerised run
|
||||
that spent 46s compiling and still reported per-package durations
|
||||
within noise of a warm host run. `-timeout` reaches the test binary
|
||||
as `-test.timeout` and its clock starts inside `testing.M.Run`, after
|
||||
the build. The real exposure was margin, not compilation. The 120s
|
||||
landed on is a **deliberate, documented divergence** from
|
||||
`REPO_POLICIES.md:192`, which mandates 30s, and from that file's
|
||||
canonical recipe at `:212-214`; the divergence is recorded in
|
||||
`script/test`'s comment because `REPO_POLICIES.md` is org-canonical
|
||||
and not editable here, and issue #101 proposes amending the policy
|
||||
text upstream. Numbers and the full verification are recorded once,
|
||||
on the pull request, and are deliberately not restated here.
|
||||
|
||||
- 2026-08-09: Triaged all fifteen stale remote branches (issue #71) and
|
||||
deleted fourteen of them; the full per-branch disposition with
|
||||
evidence is recorded on that issue. Method mattered more than the
|
||||
|
||||
Reference in New Issue
Block a user