2 Commits
Author SHA1 Message Date
sneak ded9b0398e Index the event log's resubmit count with deleted_at (closes #325)
check / check (push) Successful in 3m14s
The resubmit count matches resubmitted_from_id against a page of event
ids, and GORM adds deleted_at IS NULL. With no statistics, SQLite
preferred the deleted_at index once the page held five or more ids, so
every event log page read every live event. deleted_at now joins
resubmitted_from_id in idx_events_resubmitted_from_id, as for the other
event-tier indexes, and the plan test checks the statement with a full
page of ids.

Model: opus-5-5
2026-10-02 19:10:20 +00:00
clawbot 35d2f28c67 Name each embedded static file so a missing Alpine.js fails the build (closes #166)
check / check (push) Successful in 3m20s
static/static.go embedded the css and js directories, so a build without the extracted Alpine.js file compiled and produced a binary whose admin pages silently had no Alpine. It now names the four files the pages load, so such a build fails naming js/alpine.min.js; make build extracts the file first, so the failure shows when that step is skipped. Both lint stages extract Alpine.js before linting, since the static package no longer compiles without it, and the README's lint stage says so. static/css/input.css, the Tailwind source no page loads, is no longer embedded or served.

Model: opus-5-5
2026-10-02 20:56:35 +02:00
6 changed files with 62 additions and 12 deletions
+1
View File
@@ -25,6 +25,7 @@ COPY . .
# would need a docker daemon inside the build. Keep these steps in step with # would need a docker daemon inside the build. Keep these steps in step with
# Dockerfile.lint, including --network=none (see its header for why). # Dockerfile.lint, including --network=none (see its header for why).
RUN make fmt-check RUN make fmt-check
RUN script/assets
RUN --network=none golangci-lint config verify --config .golangci.yml RUN --network=none golangci-lint config verify --config .golangci.yml
RUN --network=none golangci-lint run --config .golangci.yml --build-tags browser ./... RUN --network=none golangci-lint run --config .golangci.yml --build-tags browser ./...
+4
View File
@@ -31,6 +31,10 @@ FROM deps AS lint
COPY . . COPY . .
# static/static.go embeds the Alpine.js file this extracts from 3p/; without
# it the static package does not compile and cannot be linted.
RUN script/assets
# `run` silently ignores config keys it does not recognize, so a typo would # `run` silently ignores config keys it does not recognize, so a typo would
# disable a setting without a word. `config verify` is what catches that. # disable a setting without a word. `config verify` is what catches that.
RUN --network=none golangci-lint config verify --config .golangci.yml RUN --network=none golangci-lint config verify --config .golangci.yml
+13 -7
View File
@@ -1348,7 +1348,9 @@ apply. The directory is `3p/` rather than `vendor/` because Go treats a root
where `go:embed` picks it up. `script/test`, `make build` and `make dev` run where `go:embed` picks it up. `script/test`, `make build` and `make dev` run
it first, and the Dockerfile builds through `make test` and `make build`, so it first, and the Dockerfile builds through `make test` and `make build`, so
nothing downloads Alpine.js. The extracted file is not committed, and nothing downloads Alpine.js. The extracted file is not committed, and
`.dockerignore` keeps any host copy out of the build context. `.dockerignore` keeps any host copy out of the build context. `static/static.go`
names every file it embeds, so a build that skips the extraction, such as a
bare `go build`, fails with an error naming `js/alpine.min.js`.
To move to a new version: download To move to a new version: download
`https://registry.npmjs.org/@alpinejs/csp/-/csp-<version>.tgz`, check it against `https://registry.npmjs.org/@alpinejs/csp/-/csp-<version>.tgz`, check it against
@@ -1876,16 +1878,19 @@ tags, so `AutoMigrate` creates them on a fresh database:
| `deliveries` | `event_id`, `deleted_at` | The event log, which loads each event's deliveries, and retention, which counts and deletes the deliveries of expired events | | `deliveries` | `event_id`, `deleted_at` | The event log, which loads each event's deliveries, and retention, which counts and deletes the deliveries of expired events |
| `delivery_results` | `delivery_id`, `deleted_at` | The event log, which loads the attempts of a page's deliveries, and retention, which deletes the attempts of expired events | | `delivery_results` | `delivery_id`, `deleted_at` | The event log, which loads the attempts of a page's deliveries, and retention, which deletes the attempts of expired events |
| `events` | `deleted_at`, `created_at` | The webhook page's statistics, which count recent events | | `events` | `deleted_at`, `created_at` | The webhook page's statistics, which count recent events |
| `events` | `resubmitted_from_id`, `deleted_at` | The event log, which counts the events resubmitted from each event on a page |
| `events` | `created_at` | Retention, which selects expired events by age | | `events` | `created_at` | Retention, which selects expired events by age |
GORM's soft delete adds `deleted_at IS NULL` to these queries; retention GORM's soft delete adds `deleted_at IS NULL` to these queries; retention
leaves it out. SQLite keeps no statistics on these tables, and without them it leaves it out. SQLite keeps no statistics on these tables, and without them it
rates the `deleted_at` index, which every live row matches, above an index on rates the `deleted_at` index, which every live row matches, above an index on
a column matched against several values or compared with a range. So every a column matched against several values or compared with a range. So every
index but the last also covers `deleted_at`. It comes second, so that index but the last also covers `deleted_at`. It comes second in the `event_id`
retention can use the index without it, except in `events`, where the and `delivery_id` indexes, so that retention can use them without it. The event
statistics compare `created_at` with a range (`>=`) and SQLite narrows by a log's count, the one query on the `resubmitted_from_id` index, always carries
range only on the last column it uses. `deleted_at IS NULL` and uses both columns. In the statistics' `events` index
`deleted_at` comes first, because they compare `created_at` with a range (`>=`)
and SQLite narrows by a range only on the last column it uses.
#### Common Fields #### Common Fields
@@ -3379,8 +3384,9 @@ version is fixed independently of the compiler's:
1. **Lint stage** (`golangci/golangci-lint:v2.12.2`, Debian-based) — 1. **Lint stage** (`golangci/golangci-lint:v2.12.2`, Debian-based) —
installs `make`, downloads dependencies, copies the source, and runs installs `make`, downloads dependencies, copies the source, and runs
`make fmt-check`, then `golangci-lint config verify` and `make fmt-check`, then `script/assets` to extract Alpine.js from
`golangci-lint run`, both with `--network=none`. `3p/`, then `golangci-lint config verify` and `golangci-lint run`,
both with `--network=none`.
2. **Builder stage** (`golang:1.26.1-bookworm`) — depends on the lint 2. **Builder stage** (`golang:1.26.1-bookworm`) — depends on the lint
stage passing (it copies a file from it), runs `make test` and stage passing (it copies a file from it), runs `make test` and
`make build` (both extract Alpine.js from `3p/` first), and finally `make build` (both extract Alpine.js from `3p/` first), and finally
@@ -199,6 +199,40 @@ func TestStatisticsQueriesUseTheirIndexes(t *testing.T) {
"(deleted_at=? AND created_at>?)") "(deleted_at=? AND created_at>?)")
} }
// TestResubmitCountUsesItsIndex does the same for the event log's count
// of the events resubmitted from each of a page's events (resubmitCounts
// in the handlers). It passes a full page of 25 ids: with an index on
// resubmitted_from_id alone, SQLite uses it for three ids and turns to
// the deleted_at index from five.
func TestResubmitCountUsesItsIndex(t *testing.T) {
t.Parallel()
mgr, lc := setupTestWebhookDBManager(t)
ctx := context.Background()
require.NoError(t, lc.Start(ctx))
defer func() { require.NoError(t, lc.Stop(ctx)) }()
db, err := mgr.GetDB(uuid.New().String())
require.NoError(t, err)
dry := db.Session(&gorm.Session{DryRun: true})
page := make([]string, 25)
for i := range page {
page[i] = uuid.New().String()
}
var counts []struct{ Total int }
assertPlanUses(t, db, dry.Model(&database.Event{}).
Select("resubmitted_from_id, count(*) AS total").
Where("resubmitted_from_id IN ?", page).
Group("resubmitted_from_id").Find(&counts),
"idx_events_resubmitted_from_id "+
"(resubmitted_from_id=? AND deleted_at=?)")
}
// assertPlanUses asserts that SQLite's plan for a statement GORM built // assertPlanUses asserts that SQLite's plan for a statement GORM built
// in a dry run, run with the same SQL and arguments GORM would send, // in a dry run, run with the same SQL and arguments GORM would send,
// names each of the given indexes. // names each of the given indexes.
+5 -3
View File
@@ -19,8 +19,10 @@ type Event struct {
// narrows by a < only on the last column it uses. Its final delete // narrows by a < only on the last column it uses. Its final delete
// has no deleted_at condition and uses the index on created_at // has no deleted_at condition and uses the index on created_at
// alone. The other tables keep the unindexed BaseModel created_at. // alone. The other tables keep the unindexed BaseModel created_at.
CreatedAt time.Time `gorm:"index;index:idx_events_deleted_at_created_at,priority:2" json:"createdAt"` // DeletedAt is also the second column of the resubmitted_from_id
DeletedAt gorm.DeletedAt `gorm:"index:idx_events_deleted_at_created_at,priority:1" json:"deletedAt,omitzero"` // index, for the reason DeliveryResult gives.
CreatedAt time.Time `gorm:"index;index:idx_events_deleted_at_created_at,priority:2" json:"createdAt"`
DeletedAt gorm.DeletedAt `gorm:"index:idx_events_deleted_at_created_at,priority:1;index:idx_events_resubmitted_from_id,priority:2" json:"deletedAt,omitzero"`
WebhookID string `gorm:"type:uuid;not null" json:"webhookId"` WebhookID string `gorm:"type:uuid;not null" json:"webhookId"`
EntrypointID string `gorm:"type:uuid;not null" json:"entrypointId"` EntrypointID string `gorm:"type:uuid;not null" json:"entrypointId"`
@@ -42,7 +44,7 @@ type Event struct {
// existed. It is not a foreign key: the source event can be // existed. It is not a foreign key: the source event can be
// reaped by retention while its copies remain, and the id is // reaped by retention while its copies remain, and the id is
// kept as the record of where the copy came from either way. // kept as the record of where the copy came from either way.
ResubmittedFromID *string `gorm:"type:uuid;index" json:"resubmittedFromId,omitempty"` ResubmittedFromID *string `gorm:"type:uuid;index:idx_events_resubmitted_from_id,priority:1" json:"resubmittedFromId,omitempty"`
// Relations. No model marshals the record it belongs to, so // Relations. No model marshals the record it belongs to, so
// Webhook and Entrypoint are left out of the JSON. // Webhook and Entrypoint are left out of the JSON.
+5 -2
View File
@@ -5,7 +5,10 @@ import (
"embed" "embed"
) )
// Static holds the embedded CSS and JavaScript files for the web UI. // Static holds the CSS and JavaScript files the web UI's pages load. They
// are named one by one so that a missing js/alpine.min.js, which make
// assets extracts and git does not track, fails the build instead of
// leaving the pages without Alpine.js.
// //
//go:embed css js //go:embed css/tailwind.css css/style.css js/app.js js/alpine.min.js
var Static embed.FS var Static embed.FS