Adds a ## Backup, Restore, and Upgrades section to README.md, placed
right after ### Running with Docker so it sits next to the volume-mount
instructions that create the problem. Docs-only.
What it covers
All five definition-of-done bullets, each verified against the source rather
than asserted from memory.
The backup set.webhooker.db (internal/database/database.go:139), one events-{webhook_uuid}.db per webhook
(internal/database/webhook_db_manager.go:227), one archive-{webhook_uuid}.db per webhook that has a database target
(internal/delivery/target_database.go:218), all under DATA_DIR (default /var/lib/webhooker, internal/config/config.go:492). Two things worth
flagging that the issue did not state:
the archive file is keyed on the webhook UUID, not the target UUID, so
several database targets on one webhook share one archive file
.env is read from the process working directory, not DATA_DIR, so it is
outside the backup set and needs backing up separately
Hot copy is not safe. No journal_mode pragma is issued at any of the
three sql.Open sites, so every database runs SQLite's default rollback
journal and there is no WAL. WebhookDBManager also caches event-database
handles for the whole process lifetime, closing them only on webhook deletion
or shutdown, so an apparently idle instance is no guarantee. Documents
stop-copy-start and sqlite3 .backup, noting that the sqlite3 CLI is not in the runtime image (alpine:3.21 plus ca-certificates only) and
that .backup is per-file, so only stop-copy-start gives one point in time
across the whole set. The existing archive move-away affordance is referenced
and explicitly scoped as not a substitute.
Restore. Restore the whole set together: every database is opened mode=rwc, so a missing events-{uuid}.db is created empty on first
access instead of erroring, and the webhook returns with its config intact
and its history silently gone. Ownership: chown -R 1000:1000, and the
directory as well as the files, because SQLite writes the rollback journal
beside the database.
Upgrade.AutoMigrate runs unconditionally on every start against all
three database kinds (internal/database/models.go:8, internal/database/webhook_db_manager.go:266, internal/delivery/target_database_archive.go:301). There is no schema
version table, no migration ledger and no down migrations anywhere in the
repo. Back up first; downgrade is stated as unsupported, with the reason
that AutoMigrate is additive so an older binary will generally open
migrated files and appear to work while writing against a schema it does not
know — silent divergence rather than a startup error.
Secrets. Event and archive databases hold full payload bodies and headers
(archivedEvent carries Headers and Body). Until #206 is fixed the event databases
also carry target credentials via the association upsert. Target config in webhooker.db is unencrypted, tracked at #212.
Gate evidence
make check — green, run again after the rebase onto next:
The lint stage really executed rather than replaying from cache: script/lint
passes --no-cache-filter=lint, the stage shows DONE 72.3s rather than CACHED, and golangci-lint printed its own 0 issues. summary line. Tests
pass (ok sneak.berlin/go/webhooker/... across the tree). No containers or
images were left behind — script/lint builds with --output=type=cacheonly, and docker ps -a is empty. No prune was run.
make fmt — run, exit 0, no changes produced. Note for the record: script/fmt in this repo is gofmt -s -w . plus goimports only, and there
is no prettier config in the tree, so make fmt does not touch Markdown. The
new prose is hand-wrapped to match the file: no line in the added block
exceeds 72 columns, which is the README's existing width.
Scope
git diff --name-only origin/next...HEAD returns README.md and nothing
else. Zero changes to code, config, scripts, CI or build files, and TODO.md
is untouched.
One pre-existing item noticed and deliberately not acted on: the lint run
warns that gomodguard is deprecated since golangci-lint v2.12.0 and
replaced by gomodguard_v2. It is unrelated to this change and out of scope
here.
Closes https://git.eeqj.de/sneak/webhooker/issues/210.
Adds a `## Backup, Restore, and Upgrades` section to `README.md`, placed
right after `### Running with Docker` so it sits next to the volume-mount
instructions that create the problem. Docs-only.
## What it covers
All five definition-of-done bullets, each verified against the source rather
than asserted from memory.
**The backup set.** `webhooker.db` (`internal/database/database.go:139`), one
`events-{webhook_uuid}.db` per webhook
(`internal/database/webhook_db_manager.go:227`), one
`archive-{webhook_uuid}.db` per webhook that has a `database` target
(`internal/delivery/target_database.go:218`), all under `DATA_DIR` (default
`/var/lib/webhooker`, `internal/config/config.go:492`). Two things worth
flagging that the issue did not state:
- the archive file is keyed on the **webhook** UUID, not the target UUID, so
several `database` targets on one webhook share one archive file
- `.env` is read from the process working directory, not `DATA_DIR`, so it is
outside the backup set and needs backing up separately
**Hot copy is not safe.** No `journal_mode` pragma is issued at any of the
three `sql.Open` sites, so every database runs SQLite's default rollback
journal and there is no WAL. `WebhookDBManager` also caches event-database
handles for the whole process lifetime, closing them only on webhook deletion
or shutdown, so an apparently idle instance is no guarantee. Documents
stop-copy-start and `sqlite3 .backup`, noting that the `sqlite3` CLI is
**not** in the runtime image (`alpine:3.21` plus `ca-certificates` only) and
that `.backup` is per-file, so only stop-copy-start gives one point in time
across the whole set. The existing archive move-away affordance is referenced
and explicitly scoped as not a substitute.
**Restore.** Restore the whole set together: every database is opened
`mode=rwc`, so a missing `events-{uuid}.db` is created **empty** on first
access instead of erroring, and the webhook returns with its config intact
and its history silently gone. Ownership: `chown -R 1000:1000`, and the
directory as well as the files, because SQLite writes the rollback journal
beside the database.
**Upgrade.** `AutoMigrate` runs unconditionally on every start against all
three database kinds (`internal/database/models.go:8`,
`internal/database/webhook_db_manager.go:266`,
`internal/delivery/target_database_archive.go:301`). There is no schema
version table, no migration ledger and no down migrations anywhere in the
repo. Back up first; downgrade is stated as **unsupported**, with the reason
that `AutoMigrate` is additive so an older binary will generally open
migrated files and appear to work while writing against a schema it does not
know — silent divergence rather than a startup error.
**Secrets.** Event and archive databases hold full payload bodies and headers
(`archivedEvent` carries `Headers` and `Body`). Until
https://git.eeqj.de/sneak/webhooker/issues/206 is fixed the event databases
also carry target credentials via the association upsert. Target config in
`webhooker.db` is unencrypted, tracked at
https://git.eeqj.de/sneak/webhooker/issues/212.
## Gate evidence
`make check` — green, run again after the rebase onto `next`:
```
#12 [lint 3/3] RUN --network=none golangci-lint run --config .golangci.yml ./...
#12 72.15 0 issues.
#12 DONE 72.3s
=== PIPE STATUS: 0 ===
```
The lint stage really executed rather than replaying from cache: `script/lint`
passes `--no-cache-filter=lint`, the stage shows `DONE 72.3s` rather than
`CACHED`, and golangci-lint printed its own `0 issues.` summary line. Tests
pass (`ok sneak.berlin/go/webhooker/...` across the tree). No containers or
images were left behind — `script/lint` builds with
`--output=type=cacheonly`, and `docker ps -a` is empty. No prune was run.
`make fmt` — run, exit 0, no changes produced. Note for the record:
`script/fmt` in this repo is `gofmt -s -w .` plus `goimports` only, and there
is no prettier config in the tree, so `make fmt` does not touch Markdown. The
new prose is hand-wrapped to match the file: no line in the added block
exceeds 72 columns, which is the README's existing width.
## Scope
`git diff --name-only origin/next...HEAD` returns `README.md` and nothing
else. Zero changes to code, config, scripts, CI or build files, and `TODO.md`
is untouched.
One pre-existing item noticed and deliberately not acted on: the lint run
warns that `gomodguard` is deprecated since golangci-lint v2.12.0 and
replaced by `gomodguard_v2`. It is unrelated to this change and out of scope
here.
The README had nothing on any of the three, leaving an operator with
no answer to which files to back up, whether a hot copy is safe, how
to restore, or what a newer image does to their data.
Adds a "Backup, Restore, and Upgrades" section covering:
- the backup set: webhooker.db, one events-{uuid}.db per webhook, one
archive-{uuid}.db per webhook with a database target, all under
DATA_DIR, plus the .env that is not under it
- why a hot copy is unsafe (no journal_mode pragma is issued on any
DSN, so every database runs the default rollback journal, and the
main and event handles stay open for the process lifetime), with
stop-copy-start and sqlite3 .backup as the safe procedures
- restore: the whole set together, since a missing events DB is
created empty rather than erroring, and chown to UID 1000 because
SQLite needs to write the journal beside the database
- upgrade: AutoMigrate runs unconditionally on every start against
all three database kinds, there is no schema version table and no
down migrations, so back up first and downgrade is unsupported
- that event and archive databases hold full payload bodies and
headers, that event databases also carry target credentials until
#206 is fixed, and that target config in webhooker.db is
unencrypted, tracked in #212
Docs only; no code, config, script or build changes.
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Closes #210.
Adds a
## Backup, Restore, and Upgradessection toREADME.md, placedright after
### Running with Dockerso it sits next to the volume-mountinstructions that create the problem. Docs-only.
What it covers
All five definition-of-done bullets, each verified against the source rather
than asserted from memory.
The backup set.
webhooker.db(internal/database/database.go:139), oneevents-{webhook_uuid}.dbper webhook(
internal/database/webhook_db_manager.go:227), onearchive-{webhook_uuid}.dbper webhook that has adatabasetarget(
internal/delivery/target_database.go:218), all underDATA_DIR(default/var/lib/webhooker,internal/config/config.go:492). Two things worthflagging that the issue did not state:
several
databasetargets on one webhook share one archive file.envis read from the process working directory, notDATA_DIR, so it isoutside the backup set and needs backing up separately
Hot copy is not safe. No
journal_modepragma is issued at any of thethree
sql.Opensites, so every database runs SQLite's default rollbackjournal and there is no WAL.
WebhookDBManageralso caches event-databasehandles for the whole process lifetime, closing them only on webhook deletion
or shutdown, so an apparently idle instance is no guarantee. Documents
stop-copy-start and
sqlite3 .backup, noting that thesqlite3CLI isnot in the runtime image (
alpine:3.21plusca-certificatesonly) andthat
.backupis per-file, so only stop-copy-start gives one point in timeacross the whole set. The existing archive move-away affordance is referenced
and explicitly scoped as not a substitute.
Restore. Restore the whole set together: every database is opened
mode=rwc, so a missingevents-{uuid}.dbis created empty on firstaccess instead of erroring, and the webhook returns with its config intact
and its history silently gone. Ownership:
chown -R 1000:1000, and thedirectory as well as the files, because SQLite writes the rollback journal
beside the database.
Upgrade.
AutoMigrateruns unconditionally on every start against allthree database kinds (
internal/database/models.go:8,internal/database/webhook_db_manager.go:266,internal/delivery/target_database_archive.go:301). There is no schemaversion table, no migration ledger and no down migrations anywhere in the
repo. Back up first; downgrade is stated as unsupported, with the reason
that
AutoMigrateis additive so an older binary will generally openmigrated files and appear to work while writing against a schema it does not
know — silent divergence rather than a startup error.
Secrets. Event and archive databases hold full payload bodies and headers
(
archivedEventcarriesHeadersandBody). Until#206 is fixed the event databases
also carry target credentials via the association upsert. Target config in
webhooker.dbis unencrypted, tracked at#212.
Gate evidence
make check— green, run again after the rebase ontonext:The lint stage really executed rather than replaying from cache:
script/lintpasses
--no-cache-filter=lint, the stage showsDONE 72.3srather thanCACHED, and golangci-lint printed its own0 issues.summary line. Testspass (
ok sneak.berlin/go/webhooker/...across the tree). No containers orimages were left behind —
script/lintbuilds with--output=type=cacheonly, anddocker ps -ais empty. No prune was run.make fmt— run, exit 0, no changes produced. Note for the record:script/fmtin this repo isgofmt -s -w .plusgoimportsonly, and thereis no prettier config in the tree, so
make fmtdoes not touch Markdown. Thenew prose is hand-wrapped to match the file: no line in the added block
exceeds 72 columns, which is the README's existing width.
Scope
git diff --name-only origin/next...HEADreturnsREADME.mdand nothingelse. Zero changes to code, config, scripts, CI or build files, and
TODO.mdis untouched.
One pre-existing item noticed and deliberately not acted on: the lint run
warns that
gomodguardis deprecated since golangci-lint v2.12.0 andreplaced by
gomodguard_v2. It is unrelated to this change and out of scopehere.
The README had nothing on any of the three, leaving an operator with no answer to which files to back up, whether a hot copy is safe, how to restore, or what a newer image does to their data. Adds a "Backup, Restore, and Upgrades" section covering: - the backup set: webhooker.db, one events-{uuid}.db per webhook, one archive-{uuid}.db per webhook with a database target, all under DATA_DIR, plus the .env that is not under it - why a hot copy is unsafe (no journal_mode pragma is issued on any DSN, so every database runs the default rollback journal, and the main and event handles stay open for the process lifetime), with stop-copy-start and sqlite3 .backup as the safe procedures - restore: the whole set together, since a missing events DB is created empty rather than erroring, and chown to UID 1000 because SQLite needs to write the journal beside the database - upgrade: AutoMigrate runs unconditionally on every start against all three database kinds, there is no schema version table and no down migrations, so back up first and downgrade is unsupported - that event and archive databases hold full payload bodies and headers, that event databases also carry target credentials until #206 is fixed, and that target config in webhooker.db is unencrypted, tracked in #212 Docs only; no code, config, script or build changes.