Terminal QA harness: drive the real binary under tmux at 80x24 and assert on screen contents #42
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
The release gate #8 was filed as "needs a human" on the premise that nothing here can exercise a real terminal. That premise is wrong, and I verified it rather than assuming:
tmuxis present on this host, and a detached session created with-x 80 -y 24gives a real 80x24 terminal with a real PTY, into whichtmux send-keysfeeds keystrokes and out of whichtmux capture-pane -preturns the exact rendered cell grid. A probe placing text with a raw cursor-position escape came back at the expected row and column.That is enough to test everything the existing suite structurally cannot: real
tcellrendering, a real process boundary, and the actual death and win paths, none of which the headless suite reaches because it pins HP/food/exp viafortify().Definition of done
A harness, driven through a
maketarget, that starts the built binary in a detached 80x24tmuxsession, sends keys, and asserts against captured screen contents. It must:tmuxis absent rather than silently skipping — a skipped terminal test that reports green is worse than no test.ROGUEOPTS, fixed geometry, and a bounded wait for each expected screen rather than a fixed sleep.Cases to cover, taken from the gate's own list:
S, quit, relaunch against the save file, confirm state is intact and the save file is removed on restore.--More--handling.ROGUEOPTShandling:name=,terse,jump,fruit=.Scores are written to a real path in
$HOME; point the harness at a private throwawayHOMEso it never touches a real~/.rogue.scores.Dependency
Blocked on #19 — the harness needs a built binary at a known path, and the make-targets-only rule means it must come from a
make buildtarget, which does not exist yet. Do 19 first.Unblocked: #19 landed, so
make buildexists and puts the binary at./build/rogue(gitignored, underbuild/).Implementation requirements settled before dispatch.
Determinism over sleeps. Every wait is a poll on
tmux capture-panefor an expected screen, with a bounded timeout and a clear failure message naming what was expected and dumping the pane that was actually there. A fixedsleepwill pass on an idle host and flake under load — this repo has already de-flaked one wall-clock-assumption test (#36), so do not introduce another.A skipped terminal test that reports green is worse than no test. If
tmuxis missing the harness fails loudly. It must nott.Skip.Isolate
$HOME. Scores go to~/.rogue.scoresand saves to a real path; point the harness at a private throwawayHOMEper run so it can never touch a real one, and assert on the score file inside it.Do not weaken the game to make it testable. No new production hooks, no test-only branches in game code, no widening of exported surface. Wizard mode already exists and is the sanctioned lever. If a case genuinely cannot be driven without a production change, stop and report rather than adding one.
Wizard-assisted is not a playthrough. Any case that uses wizard mode to reach a state says so in its name and its output. #8 turns on an honest game completing; a wizard descent must never be able to be mistaken for that evidence.
Start with the cases that are cheap and decisive — death path, save/restore across a real process boundary,
ROGUEOPTS, screen integrity. Amulet retrieval is the ambitious one; land the harness with the first four working rather than holding everything for it.Every defect found gets its own issue. Those are the deliverable, not a nuisance — do not batch them into one, and do not fix them drive-by in this unit.
Plan.
Placement. New package
qa/, its ownmake qatarget that depends onbuild. Not inmake test: the harness needstmuxand a built binary, and putting it intestwould dragbuildintomake check, which must stay non-file-modifying.GO_PKGSbecomesgo list ./... | grep -v /qa, sotestandcoverexclude it automatically instead of by a hand-maintained list, with a guard that fails the target if the list comes back empty. No build tag: a separate package keeps the harness insidegolangci-lint run ./..., where a tagged file would silently escape the lint gate.Synchronisation. One primitive: poll
capture-pane -p -N(which yields a stable 24x80 grid) until a predicate holds, bounded. On timeout, fail naming the expectation and dumping the pane. No fixed sleeps anywhere. Sessions are uniquely named per test and killed fromt.Cleanup; the pane command is additionally wrapped intimeoutso nothing can outlive a hard panic on this shared host.$HOMEis a per-testt.TempDir(), passed viatmux new-session -e. Missingtmuxist.Fatal, never a skip.Cases.
-dfor the score-file assertion, decoding$HOME/.rogue.scoresand checking the entry. Split from case 1 deliberately: a death with an empty purse never places on the scoreboard, because C'sscore()breaks onamount > scp->sc_score. That is faithful, not a defect, so the score assertion needs a path with a non-zero purse.S,y, process exits, relaunch against the file, compare the restored grid to the pre-save grid and assert the save file is gone.ROGUEOPTS: theoscreen forname=,jump,fruit=, plus a behavioural check forterse(terse and non-terse forms of the same message).--More--(eat the starting ration) asserted to absorb non-space input and clear on space.total_winnervia a created amulet and the stairs, and asserts the win banner and the appraisal list. It also asserts no scoreboard entry is written, since wizard mode setsnoscore— which is exactly why it cannot stand in for #8.Defects found while probing get their own issues; none are fixed here.