sneak/webhooker
1
0
Fork 0
Code Issues 5 Pull Requests 1 Actions Packages Projects Releases Wiki Activity

The event log's resubmit count reads every live event instead of using the resubmitted_from_id index #325

New Issue
Closed
opened 2026-09-28 13:36:41 +02:00 by clawbot · 2 comments
clawbot commented 2026-09-28 13:36:41 +02:00
Collaborator
Copy Link
Copy Source

resubmitCounts in internal/handlers/source_management.go runs on every event log page: it counts how many events were resubmitted from each of the page's events, with resubmitted_from_id IN (...) over up to 25 event ids. GORM adds deleted_at IS NULL, and SQLite, which keeps no statistics on the per-webhook databases, then reads through the deleted_at index instead of idx_events_resubmitted_from_id whenever the page holds two or more events. Every page load reads every live event in that webhook's database.

This is the cause #319 fixes for the other event-tier indexes, by adding deleted_at to each index.

Definition of done

  • The statement resubmitCounts sends for a page of events uses an index on resubmitted_from_id, shown by a test that checks SQLite's plan for that statement as GORM builds it, as #319 does for its statements.
  • An existing per-webhook database gains the index on open.
  • The README's Event-tier indexes table lists it.

Model: opus-5-5

`resubmitCounts` in `internal/handlers/source_management.go` runs on every event log page: it counts how many events were resubmitted from each of the page's events, with `resubmitted_from_id IN (...)` over up to 25 event ids. GORM adds `deleted_at IS NULL`, and SQLite, which keeps no statistics on the per-webhook databases, then reads through the `deleted_at` index instead of `idx_events_resubmitted_from_id` whenever the page holds two or more events. Every page load reads every live event in that webhook's database. This is the cause https://git.eeqj.de/sneak/webhooker/pulls/319 fixes for the other event-tier indexes, by adding `deleted_at` to each index. ## Definition of done - The statement `resubmitCounts` sends for a page of events uses an index on `resubmitted_from_id`, shown by a test that checks SQLite's plan for that statement as GORM builds it, as https://git.eeqj.de/sneak/webhooker/pulls/319 does for its statements. - An existing per-webhook database gains the index on open. - The README's Event-tier indexes table lists it. Model: opus-5-5
clawbot commented 2026-10-02 20:32:40 +02:00
Author
Collaborator
Copy Link
Copy Source

Plan. Check against next first. The statement resubmitCounts sends for a page of events uses an index on resubmitted_from_id: add deleted_at to that index, as #319 did for the other event-tier indexes, so SQLite picks it over the deleted_at index. A test checks SQLite's plan for the statement as GORM builds it, as the tests from 319 do. The README's event-tier indexes table lists it. Pre-1.0: the index goes into the schema in place; nothing is added for databases from older builds, so the issue's "existing database gains the index on open" item is dropped.

Model: opus-5-5

Plan. Check against `next` first. The statement `resubmitCounts` sends for a page of events uses an index on `resubmitted_from_id`: add `deleted_at` to that index, as https://git.eeqj.de/sneak/webhooker/pulls/319 did for the other event-tier indexes, so SQLite picks it over the `deleted_at` index. A test checks SQLite's plan for the statement as GORM builds it, as the tests from 319 do. The README's event-tier indexes table lists it. Pre-1.0: the index goes into the schema in place; nothing is added for databases from older builds, so the issue's "existing database gains the index on open" item is dropped. Model: opus-5-5
clawbot commented 2026-10-02 20:53:08 +02:00
Author
Collaborator
Copy Link
Copy Source

#468 makes deleted_at the second column of idx_events_resubmitted_from_id, so the event log's resubmit count is answered from that index for a whole page of events. A new test checks SQLite's plan for the statement as GORM builds it, and the README's event-tier indexes table lists the index.

  • Deviation, per the plan: nothing is added for databases from older builds. Such a database keeps the old one-column index under the same name until it is recreated.
  • Judgement call: the plan test passes 25 ids, a full page. With the old index, SQLite only switches to the deleted_at index from five ids, so the three ids the other plan tests use would not show the problem.

Model: opus-5-5

https://git.eeqj.de/sneak/webhooker/pulls/468 makes `deleted_at` the second column of `idx_events_resubmitted_from_id`, so the event log's resubmit count is answered from that index for a whole page of events. A new test checks SQLite's plan for the statement as GORM builds it, and the README's event-tier indexes table lists the index. - Deviation, per the plan: nothing is added for databases from older builds. Such a database keeps the old one-column index under the same name until it is recreated. - Judgement call: the plan test passes 25 ids, a full page. With the old index, SQLite only switches to the `deleted_at` index from five ids, so the three ids the other plan tests use would not show the problem. Model: opus-5-5
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
critical
Extremely important: data loss, a security hole, production down, or basic use impossible
 merge-ready
needs-checks
needs-rebase
needs-review
needs-rework
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
clawbot sneak (Jeffrey Paul)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sneak/webhooker#325