ircserver tests fail under heavy host load #265

Closed
opened 2026-11-10 21:15:47 +01:00 by clawbot · 3 comments
Collaborator

Tests in internal/ircserver fail when the host is heavily loaded, and pass on a rerun with no code change. Seen in make check on a branch that does not touch the package, with a load average near 200 on 48 cores.

Tests that failed, a different set on each run:

  • TestQuietClientNotAnsweringPingIsClosed: closed 2.011338126s after the client's last line, want 1.3s to 2s
  • TestRelayNickDiffersOnlyInCase: read tcp ...: i/o timeout
  • TestPassPostRegistrationLength, TestPassBeforeRegistrationLength, TestPassPostRegistrationNotStored, TestHandlePassPostRegistration
  • TestIntegrationModeHashcash, TestIntegrationModeInviteOnlyNeedsOperator, TestIntegrationTwoClients

In the failing runs the database OnStart hook took 2 to 3.6 seconds, which uses up the tests' fixed read deadlines and timing windows.

Definition of done

  • The listed tests pass under heavy host load: their deadlines allow for slow startup, or they wait for an event instead of a fixed time.
  • make check stays green.

Model: opus-5-5

Tests in `internal/ircserver` fail when the host is heavily loaded, and pass on a rerun with no code change. Seen in `make check` on a branch that does not touch the package, with a load average near 200 on 48 cores. Tests that failed, a different set on each run: - `TestQuietClientNotAnsweringPingIsClosed`: `closed 2.011338126s after the client's last line, want 1.3s to 2s` - `TestRelayNickDiffersOnlyInCase`: `read tcp ...: i/o timeout` - `TestPassPostRegistrationLength`, `TestPassBeforeRegistrationLength`, `TestPassPostRegistrationNotStored`, `TestHandlePassPostRegistration` - `TestIntegrationModeHashcash`, `TestIntegrationModeInviteOnlyNeedsOperator`, `TestIntegrationTwoClients` In the failing runs the database `OnStart` hook took 2 to 3.6 seconds, which uses up the tests' fixed read deadlines and timing windows. ## Definition of done - The listed tests pass under heavy host load: their deadlines allow for slow startup, or they wait for an event instead of a fixed time. - `make check` stays green. Model: opus-5-5
clawbot self-assigned this 2026-11-10 21:24:54 +01:00
Author
Collaborator

Also in scope: TestForeignKeysOnAfterCancelledQuery/no_parameters in internal/db fails the same way under heavy host load ("the pool kept its first connection") and passes on a rerun. The definition of done covers it beside the internal/ircserver tests.

Model: opus-5-5

Also in scope: `TestForeignKeysOnAfterCancelledQuery/no_parameters` in `internal/db` fails the same way under heavy host load ("the pool kept its first connection") and passes on a rerun. The definition of done covers it beside the `internal/ircserver` tests. Model: opus-5-5
clawbot added the critical label 2026-11-11 00:12:26 +01:00
Author
Collaborator

Top priority: make check on next at 72d8f2c failed on TestQuietClientNotAnsweringPingIsClosed (closed 2.008 s after the last line, window 1.3 to 2 s) under host load near 100. This unit starts now instead of after #267; whichever lands second rebases.

Every test this issue names must pass under such load: widen the timing windows by what the test is not measuring (setup and scheduling delay), or wait for the event instead of a fixed time. The one-sided bound each test checks (closed only after the quiet period, never before it) stays.

Model: opus-5-5

Top priority: `make check` on `next` at `72d8f2c` failed on `TestQuietClientNotAnsweringPingIsClosed` (closed 2.008 s after the last line, window 1.3 to 2 s) under host load near 100. This unit starts now instead of after https://git.eeqj.de/sneak/neoirc/pulls/267; whichever lands second rebases. Every test this issue names must pass under such load: widen the timing windows by what the test is not measuring (setup and scheduling delay), or wait for the event instead of a fixed time. The one-sided bound each test checks (closed only after the quiet period, never before it) stays. Model: opus-5-5
Author
Collaborator

Built in #272: the internal/ircserver read and wait limit goes from 5 to 30 seconds. The ping-close test gets a 3 second interval, and its upper bound is measured from the end of registration; the lower bound is unchanged. The internal/db cancelled-query timeout goes from 100 ms to 2 seconds, because a context that ends before the statement starts leaves the connection in the pool.

Model: opus-5-5

Built in https://git.eeqj.de/sneak/neoirc/pulls/272: the `internal/ircserver` read and wait limit goes from 5 to 30 seconds. The ping-close test gets a 3 second interval, and its upper bound is measured from the end of registration; the lower bound is unchanged. The `internal/db` cancelled-query timeout goes from 100 ms to 2 seconds, because a context that ends before the statement starts leaves the connection in the pool. Model: opus-5-5
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sneak/neoirc#265