Index the event log's resubmit count with deleted_at (closes #325)
check / check (push) Waiting to run
check / check (push) Waiting to run
The event log's resubmit count, run on every page load over up to 25 event ids, read every live event in the webhook's database: GORM adds deleted_at IS NULL, and SQLite, keeping no statistics there, chose the deleted_at index over the resubmitted_from_id one. deleted_at is now the second column of idx_events_resubmitted_from_id, so the count is answered from that index for a whole page of events. A test checks SQLite's plan for the statement as GORM builds it, with a full page of ids. The README's event-tier indexes table lists the index. Pre-1.0: the index changes in the schema in place, with nothing for older databases. Model: opus-5-5
This commit was merged in pull request #468.
This commit is contained in:
@@ -1879,16 +1879,19 @@ tags, so `AutoMigrate` creates them on a fresh database:
|
||||
| `deliveries` | `event_id`, `deleted_at` | The event log, which loads each event's deliveries, and retention, which counts and deletes the deliveries of expired events |
|
||||
| `delivery_results` | `delivery_id`, `deleted_at` | The event log, which loads the attempts of a page's deliveries, and retention, which deletes the attempts of expired events |
|
||||
| `events` | `deleted_at`, `created_at` | The webhook page's statistics, which count recent events |
|
||||
| `events` | `resubmitted_from_id`, `deleted_at` | The event log, which counts the events resubmitted from each event on a page |
|
||||
| `events` | `created_at` | Retention, which selects expired events by age |
|
||||
|
||||
GORM's soft delete adds `deleted_at IS NULL` to these queries; retention
|
||||
leaves it out. SQLite keeps no statistics on these tables, and without them it
|
||||
rates the `deleted_at` index, which every live row matches, above an index on
|
||||
a column matched against several values or compared with a range. So every
|
||||
index but the last also covers `deleted_at`. It comes second, so that
|
||||
retention can use the index without it, except in `events`, where the
|
||||
statistics compare `created_at` with a range (`>=`) and SQLite narrows by a
|
||||
range only on the last column it uses.
|
||||
index but the last also covers `deleted_at`. It comes second in the `event_id`
|
||||
and `delivery_id` indexes, so that retention can use them without it. The event
|
||||
log's count, the one query on the `resubmitted_from_id` index, always carries
|
||||
`deleted_at IS NULL` and uses both columns. In the statistics' `events` index
|
||||
`deleted_at` comes first, because they compare `created_at` with a range (`>=`)
|
||||
and SQLite narrows by a range only on the last column it uses.
|
||||
|
||||
#### Common Fields
|
||||
|
||||
|
||||
@@ -199,6 +199,40 @@ func TestStatisticsQueriesUseTheirIndexes(t *testing.T) {
|
||||
"(deleted_at=? AND created_at>?)")
|
||||
}
|
||||
|
||||
// TestResubmitCountUsesItsIndex does the same for the event log's count
|
||||
// of the events resubmitted from each of a page's events (resubmitCounts
|
||||
// in the handlers). It passes a full page of 25 ids: with an index on
|
||||
// resubmitted_from_id alone, SQLite uses it for three ids and turns to
|
||||
// the deleted_at index from five.
|
||||
func TestResubmitCountUsesItsIndex(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
mgr, lc := setupTestWebhookDBManager(t)
|
||||
ctx := context.Background()
|
||||
require.NoError(t, lc.Start(ctx))
|
||||
|
||||
defer func() { require.NoError(t, lc.Stop(ctx)) }()
|
||||
|
||||
db, err := mgr.GetDB(uuid.New().String())
|
||||
require.NoError(t, err)
|
||||
|
||||
dry := db.Session(&gorm.Session{DryRun: true})
|
||||
|
||||
page := make([]string, 25)
|
||||
for i := range page {
|
||||
page[i] = uuid.New().String()
|
||||
}
|
||||
|
||||
var counts []struct{ Total int }
|
||||
|
||||
assertPlanUses(t, db, dry.Model(&database.Event{}).
|
||||
Select("resubmitted_from_id, count(*) AS total").
|
||||
Where("resubmitted_from_id IN ?", page).
|
||||
Group("resubmitted_from_id").Find(&counts),
|
||||
"idx_events_resubmitted_from_id "+
|
||||
"(resubmitted_from_id=? AND deleted_at=?)")
|
||||
}
|
||||
|
||||
// assertPlanUses asserts that SQLite's plan for a statement GORM built
|
||||
// in a dry run, run with the same SQL and arguments GORM would send,
|
||||
// names each of the given indexes.
|
||||
|
||||
@@ -19,8 +19,10 @@ type Event struct {
|
||||
// narrows by a < only on the last column it uses. Its final delete
|
||||
// has no deleted_at condition and uses the index on created_at
|
||||
// alone. The other tables keep the unindexed BaseModel created_at.
|
||||
CreatedAt time.Time `gorm:"index;index:idx_events_deleted_at_created_at,priority:2" json:"createdAt"`
|
||||
DeletedAt gorm.DeletedAt `gorm:"index:idx_events_deleted_at_created_at,priority:1" json:"deletedAt,omitzero"`
|
||||
// DeletedAt is also the second column of the resubmitted_from_id
|
||||
// index, for the reason DeliveryResult gives.
|
||||
CreatedAt time.Time `gorm:"index;index:idx_events_deleted_at_created_at,priority:2" json:"createdAt"`
|
||||
DeletedAt gorm.DeletedAt `gorm:"index:idx_events_deleted_at_created_at,priority:1;index:idx_events_resubmitted_from_id,priority:2" json:"deletedAt,omitzero"`
|
||||
|
||||
WebhookID string `gorm:"type:uuid;not null" json:"webhookId"`
|
||||
EntrypointID string `gorm:"type:uuid;not null" json:"entrypointId"`
|
||||
@@ -42,7 +44,7 @@ type Event struct {
|
||||
// existed. It is not a foreign key: the source event can be
|
||||
// reaped by retention while its copies remain, and the id is
|
||||
// kept as the record of where the copy came from either way.
|
||||
ResubmittedFromID *string `gorm:"type:uuid;index" json:"resubmittedFromId,omitempty"`
|
||||
ResubmittedFromID *string `gorm:"type:uuid;index:idx_events_resubmitted_from_id,priority:1" json:"resubmittedFromId,omitempty"`
|
||||
|
||||
// Relations. No model marshals the record it belongs to, so
|
||||
// Webhook and Entrypoint are left out of the JSON.
|
||||
|
||||
Reference in New Issue
Block a user