Index the event-tier columns the sweeps, event log and retention scan #319

Open
clawbot wants to merge 1 commits from issue-314-event-tier-indexes into next
Collaborator

Adds secondary indexes to the per-webhook event databases so the background work stops scanning whole tables: deliveries.status (recovery and sweep, once a minute), deliveries.event_id and delivery_results.delivery_id (each page of the event log), and events.created_at (retention by age). They are declared in GORM model tags, so AutoMigrate creates them on both a fresh and an existing per-webhook database, since every open runs the migration.

events.created_at is the one that needs care: created_at lives on the shared BaseModel, and only the events table should carry the index. Event therefore redeclares CreatedAt with an index tag, which shadows the embedded field for that model alone; the other tables keep their created_at unindexed. A newcomer sees two CreatedAt on Event, and the comment there says why.

Test: opening an existing index-less database creates the indexes. It stands in for an older file by dropping the four indexes from a freshly opened database, closing it, and reopening, then asserts all four exist.

Gate: make check is red only on internal/handlers, which timed out at 91.8s against its 90s budget under heavy shared-host load (later reruns were OOM-killed by the host before finishing). It is not this change: internal/database, the only package touched, passes with the new test, and no assertion failed anywhere. Re-run the gate when the host is quieter.

Disclosure: judgement call, reported the gate red rather than retry further, to avoid adding load to an out-of-memory host.

Model: opus-4-8

Adds secondary indexes to the per-webhook event databases so the background work stops scanning whole tables: `deliveries.status` (recovery and sweep, once a minute), `deliveries.event_id` and `delivery_results.delivery_id` (each page of the event log), and `events.created_at` (retention by age). They are declared in GORM model tags, so `AutoMigrate` creates them on both a fresh and an existing per-webhook database, since every open runs the migration. `events.created_at` is the one that needs care: `created_at` lives on the shared `BaseModel`, and only the events table should carry the index. Event therefore redeclares `CreatedAt` with an `index` tag, which shadows the embedded field for that model alone; the other tables keep their `created_at` unindexed. A newcomer sees two `CreatedAt` on Event, and the comment there says why. Test: opening an existing index-less database creates the indexes. It stands in for an older file by dropping the four indexes from a freshly opened database, closing it, and reopening, then asserts all four exist. Gate: `make check` is red only on `internal/handlers`, which timed out at 91.8s against its 90s budget under heavy shared-host load (later reruns were OOM-killed by the host before finishing). It is not this change: `internal/database`, the only package touched, passes with the new test, and no assertion failed anywhere. Re-run the gate when the host is quieter. Disclosure: judgement call, reported the gate red rather than retry further, to avoid adding load to an out-of-memory host. Model: opus-4-8
clawbot added the needs-review label 2026-09-21 09:53:44 +02:00
clawbot self-assigned this 2026-09-21 09:53:44 +02:00
clawbot added 1 commit 2026-09-21 09:53:45 +02:00
The per-webhook tables declared no secondary indexes, so the recovery
and sweep queries (by delivery status, every minute), the event log
(deliveries by event, results by delivery) and retention (events by
age) each scanned a whole table. Add indexes through GORM model tags so
AutoMigrate creates them on a fresh and on an existing per-webhook
database. events.created_at is indexed by overriding the embedded
BaseModel field on Event alone, leaving the other tables' created_at
unindexed. A test drops the indexes from an opened database, reopens it,
and asserts the open recreated them. The README Data Model section lists
the indexes.

Model: opus-4-8
Some required checks failed
check / check (push) Failing after 2s
You are not authorized to merge this pull request.
This pull request can be merged automatically.
This branch is out-of-date with the base branch
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u origin issue-314-event-tier-indexes:issue-314-event-tier-indexes
git checkout issue-314-event-tier-indexes
Sign in to join this conversation.
No Reviewers
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sneak/webhooker#319