Make main and next green: a thumbnail test times out on the first test pass #153

Closed
opened 2026-10-02 01:50:35 +02:00 by clawbot · 1 comment
Collaborator

Top priority: this comes before all other work in this repo. Owner's words (sneak/project-management#20): "main and next should be green on all managed repos at all times. any divergence is a top priority."

Red: main (9e6e0385) and next (10afa7a7). script/cibuild fails in the test stage of the Dockerfile (RUN timeout 90 yarn run vitest run ...): on the first pass, test/thumbnails/thumbnails.test.ts "fixMissingThumbnails > re-encodes smaller until the thumbnail fits the recorded size" took 5.4 s on main and 6.1 s on next and failed with "Test timed out in 5000ms". On the second (verbose) pass it passed in about 3 s, and the step still exits 1 as written. Every other test passed. The test re-encodes a noisy 400x300 JPEG several times, so its run time sits near vitest's default 5 s limit and depends on how busy the machine is: it is flaky, and the flake is the defect.

Reproduce: clone the branch fresh and run script/cibuild while the host is busy (here three builds ran at once).

Definition of done:

  • The cause is fixed for real: the test does the same check with much less work (for example a smaller image that still cannot fit at the default quality), or, if it cannot, it gets an explicit timeout well above its measured time, with the reason in a one-line comment. No test is skipped, deleted or retried into passing.
  • next: through a reviewed PR to next, a fresh clone's script/cibuild passes several times in a row with nothing failing on the first pass.
  • main: the same fix, and nothing else from next, lands on main through its own reviewed PR targeting main, then main passes the same way.

Model: opus-5-5

Top priority: this comes before all other work in this repo. Owner's words (https://git.eeqj.de/sneak/project-management/issues/20): "main and next should be green on all managed repos at all times. any divergence is a top priority." Red: `main` (`9e6e0385`) and `next` (`10afa7a7`). `script/cibuild` fails in the `test` stage of the `Dockerfile` (`RUN timeout 90 yarn run vitest run ...`): on the first pass, `test/thumbnails/thumbnails.test.ts` "fixMissingThumbnails > re-encodes smaller until the thumbnail fits the recorded size" took 5.4 s on `main` and 6.1 s on `next` and failed with "Test timed out in 5000ms". On the second (verbose) pass it passed in about 3 s, and the step still exits 1 as written. Every other test passed. The test re-encodes a noisy 400x300 JPEG several times, so its run time sits near vitest's default 5 s limit and depends on how busy the machine is: it is flaky, and the flake is the defect. Reproduce: clone the branch fresh and run `script/cibuild` while the host is busy (here three builds ran at once). Definition of done: - The cause is fixed for real: the test does the same check with much less work (for example a smaller image that still cannot fit at the default quality), or, if it cannot, it gets an explicit timeout well above its measured time, with the reason in a one-line comment. No test is skipped, deleted or retried into passing. - `next`: through a reviewed PR to `next`, a fresh clone's `script/cibuild` passes several times in a row with nothing failing on the first pass. - `main`: the same fix, and nothing else from `next`, lands on `main` through its own reviewed PR targeting `main`, then `main` passes the same way. Model: opus-5-5
clawbot self-assigned this 2026-10-02 01:50:35 +02:00
Author
Collaborator

Plan. This goes ahead of everything else in quak.

  • Cause. The test builds a noisy 400x300 JPEG. fixMissingThumbnails then decodes it and re-encodes it at falling quality until it fits. Each encode of that much random data takes most of a second, and on a busy host the total crosses vitest's 5 s limit.
  • Fix for next. Make the image much smaller, for example 64x48 random pixels, which the default quality-50 encoding still cannot fit under the recorded size. Keep every assertion. The test must still prove that quak re-encodes smaller until the thumbnail fits. Add no timeout and no retry, unless the smaller image cannot keep that proof; in that case add one explicit timeout, with a one-line reason.
  • Proof. In a fresh clone, run script/cibuild several times in a row, at least once while other builds are running. Nothing may fail on the first test pass. The PR targets next and gets an independent review.
  • main. Once the next PR passes, the same change alone goes to main through its own PR. It is a branch from main with only that test edit, independently reviewed, and squash-merged after script/cibuild passes on it the same way.

Model: opus-5-5

Plan. This goes ahead of everything else in quak. - **Cause.** The test builds a noisy 400x300 JPEG. `fixMissingThumbnails` then decodes it and re-encodes it at falling quality until it fits. Each encode of that much random data takes most of a second, and on a busy host the total crosses vitest's 5 s limit. - **Fix for `next`.** Make the image much smaller, for example 64x48 random pixels, which the default quality-50 encoding still cannot fit under the recorded size. Keep every assertion. The test must still prove that quak re-encodes smaller until the thumbnail fits. Add no timeout and no retry, unless the smaller image cannot keep that proof; in that case add one explicit timeout, with a one-line reason. - **Proof.** In a fresh clone, run `script/cibuild` several times in a row, at least once while other builds are running. Nothing may fail on the first test pass. The PR targets `next` and gets an independent review. - **`main`.** Once the `next` PR passes, the same change alone goes to `main` through its own PR. It is a branch from `main` with only that test edit, independently reviewed, and squash-merged after `script/cibuild` passes on it the same way. Model: opus-5-5
clawbot reopened this issue 2026-10-02 03:49:38 +02:00
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sneak/quak#153