sneak/webhooker
1
0
Fork 0
Code Issues 49 Pull Requests 1 Actions Packages Projects Releases Wiki Activity

Document backup, restore and upgrade procedures (closes #210) #214

Merged
clawbot merged 1 commits from issue-210-backup-restore-upgrade-docs into next 2026-08-20 06:05:15 +02:00
Conversation 0 Commits 1 Files Changed 1 +162
clawbot commented 2026-08-20 06:04:26 +02:00
Collaborator
Copy Link

Closes #210.

Adds a ## Backup, Restore, and Upgrades section to README.md, placed
right after ### Running with Docker so it sits next to the volume-mount
instructions that create the problem. Docs-only.

What it covers

All five definition-of-done bullets, each verified against the source rather
than asserted from memory.

The backup set. webhooker.db (internal/database/database.go:139), one
events-{webhook_uuid}.db per webhook
(internal/database/webhook_db_manager.go:227), one
archive-{webhook_uuid}.db per webhook that has a database target
(internal/delivery/target_database.go:218), all under DATA_DIR (default
/var/lib/webhooker, internal/config/config.go:492). Two things worth
flagging that the issue did not state:

  • the archive file is keyed on the webhook UUID, not the target UUID, so
    several database targets on one webhook share one archive file
  • .env is read from the process working directory, not DATA_DIR, so it is
    outside the backup set and needs backing up separately

Hot copy is not safe. No journal_mode pragma is issued at any of the
three sql.Open sites, so every database runs SQLite's default rollback
journal and there is no WAL. WebhookDBManager also caches event-database
handles for the whole process lifetime, closing them only on webhook deletion
or shutdown, so an apparently idle instance is no guarantee. Documents
stop-copy-start and sqlite3 .backup, noting that the sqlite3 CLI is
not in the runtime image (alpine:3.21 plus ca-certificates only) and
that .backup is per-file, so only stop-copy-start gives one point in time
across the whole set. The existing archive move-away affordance is referenced
and explicitly scoped as not a substitute.

Restore. Restore the whole set together: every database is opened
mode=rwc, so a missing events-{uuid}.db is created empty on first
access instead of erroring, and the webhook returns with its config intact
and its history silently gone. Ownership: chown -R 1000:1000, and the
directory as well as the files, because SQLite writes the rollback journal
beside the database.

Upgrade. AutoMigrate runs unconditionally on every start against all
three database kinds (internal/database/models.go:8,
internal/database/webhook_db_manager.go:266,
internal/delivery/target_database_archive.go:301). There is no schema
version table, no migration ledger and no down migrations anywhere in the
repo. Back up first; downgrade is stated as unsupported, with the reason
that AutoMigrate is additive so an older binary will generally open
migrated files and appear to work while writing against a schema it does not
know — silent divergence rather than a startup error.

Secrets. Event and archive databases hold full payload bodies and headers
(archivedEvent carries Headers and Body). Until
#206 is fixed the event databases
also carry target credentials via the association upsert. Target config in
webhooker.db is unencrypted, tracked at
#212.

Gate evidence

make check — green, run again after the rebase onto next:

#12 [lint 3/3] RUN --network=none golangci-lint run --config .golangci.yml ./...
#12 72.15 0 issues.
#12 DONE 72.3s
=== PIPE STATUS: 0 ===

The lint stage really executed rather than replaying from cache: script/lint
passes --no-cache-filter=lint, the stage shows DONE 72.3s rather than
CACHED, and golangci-lint printed its own 0 issues. summary line. Tests
pass (ok sneak.berlin/go/webhooker/... across the tree). No containers or
images were left behind — script/lint builds with
--output=type=cacheonly, and docker ps -a is empty. No prune was run.

make fmt — run, exit 0, no changes produced. Note for the record:
script/fmt in this repo is gofmt -s -w . plus goimports only, and there
is no prettier config in the tree, so make fmt does not touch Markdown. The
new prose is hand-wrapped to match the file: no line in the added block
exceeds 72 columns, which is the README's existing width.

Scope

git diff --name-only origin/next...HEAD returns README.md and nothing
else. Zero changes to code, config, scripts, CI or build files, and TODO.md
is untouched.

One pre-existing item noticed and deliberately not acted on: the lint run
warns that gomodguard is deprecated since golangci-lint v2.12.0 and
replaced by gomodguard_v2. It is unrelated to this change and out of scope
here.

Closes https://git.eeqj.de/sneak/webhooker/issues/210. Adds a `## Backup, Restore, and Upgrades` section to `README.md`, placed right after `### Running with Docker` so it sits next to the volume-mount instructions that create the problem. Docs-only. ## What it covers All five definition-of-done bullets, each verified against the source rather than asserted from memory. **The backup set.** `webhooker.db` (`internal/database/database.go:139`), one `events-{webhook_uuid}.db` per webhook (`internal/database/webhook_db_manager.go:227`), one `archive-{webhook_uuid}.db` per webhook that has a `database` target (`internal/delivery/target_database.go:218`), all under `DATA_DIR` (default `/var/lib/webhooker`, `internal/config/config.go:492`). Two things worth flagging that the issue did not state: - the archive file is keyed on the **webhook** UUID, not the target UUID, so several `database` targets on one webhook share one archive file - `.env` is read from the process working directory, not `DATA_DIR`, so it is outside the backup set and needs backing up separately **Hot copy is not safe.** No `journal_mode` pragma is issued at any of the three `sql.Open` sites, so every database runs SQLite's default rollback journal and there is no WAL. `WebhookDBManager` also caches event-database handles for the whole process lifetime, closing them only on webhook deletion or shutdown, so an apparently idle instance is no guarantee. Documents stop-copy-start and `sqlite3 .backup`, noting that the `sqlite3` CLI is **not** in the runtime image (`alpine:3.21` plus `ca-certificates` only) and that `.backup` is per-file, so only stop-copy-start gives one point in time across the whole set. The existing archive move-away affordance is referenced and explicitly scoped as not a substitute. **Restore.** Restore the whole set together: every database is opened `mode=rwc`, so a missing `events-{uuid}.db` is created **empty** on first access instead of erroring, and the webhook returns with its config intact and its history silently gone. Ownership: `chown -R 1000:1000`, and the directory as well as the files, because SQLite writes the rollback journal beside the database. **Upgrade.** `AutoMigrate` runs unconditionally on every start against all three database kinds (`internal/database/models.go:8`, `internal/database/webhook_db_manager.go:266`, `internal/delivery/target_database_archive.go:301`). There is no schema version table, no migration ledger and no down migrations anywhere in the repo. Back up first; downgrade is stated as **unsupported**, with the reason that `AutoMigrate` is additive so an older binary will generally open migrated files and appear to work while writing against a schema it does not know — silent divergence rather than a startup error. **Secrets.** Event and archive databases hold full payload bodies and headers (`archivedEvent` carries `Headers` and `Body`). Until https://git.eeqj.de/sneak/webhooker/issues/206 is fixed the event databases also carry target credentials via the association upsert. Target config in `webhooker.db` is unencrypted, tracked at https://git.eeqj.de/sneak/webhooker/issues/212. ## Gate evidence `make check` — green, run again after the rebase onto `next`: ``` #12 [lint 3/3] RUN --network=none golangci-lint run --config .golangci.yml ./... #12 72.15 0 issues. #12 DONE 72.3s === PIPE STATUS: 0 === ``` The lint stage really executed rather than replaying from cache: `script/lint` passes `--no-cache-filter=lint`, the stage shows `DONE 72.3s` rather than `CACHED`, and golangci-lint printed its own `0 issues.` summary line. Tests pass (`ok sneak.berlin/go/webhooker/...` across the tree). No containers or images were left behind — `script/lint` builds with `--output=type=cacheonly`, and `docker ps -a` is empty. No prune was run. `make fmt` — run, exit 0, no changes produced. Note for the record: `script/fmt` in this repo is `gofmt -s -w .` plus `goimports` only, and there is no prettier config in the tree, so `make fmt` does not touch Markdown. The new prose is hand-wrapped to match the file: no line in the added block exceeds 72 columns, which is the README's existing width. ## Scope `git diff --name-only origin/next...HEAD` returns `README.md` and nothing else. Zero changes to code, config, scripts, CI or build files, and `TODO.md` is untouched. One pre-existing item noticed and deliberately not acted on: the lint run warns that `gomodguard` is deprecated since golangci-lint v2.12.0 and replaced by `gomodguard_v2`. It is unrelated to this change and out of scope here.
clawbot added 1 commit 2026-08-20 06:04:26 +02:00
Document backup, restore and upgrade procedures (closes #210)
All checks were successful
check / check (push) Successful in 10s
Details
0f2ac5a3c8 
The README had nothing on any of the three, leaving an operator with
no answer to which files to back up, whether a hot copy is safe, how
to restore, or what a newer image does to their data.

Adds a "Backup, Restore, and Upgrades" section covering:

- the backup set: webhooker.db, one events-{uuid}.db per webhook, one
  archive-{uuid}.db per webhook with a database target, all under
  DATA_DIR, plus the .env that is not under it
- why a hot copy is unsafe (no journal_mode pragma is issued on any
  DSN, so every database runs the default rollback journal, and the
  main and event handles stay open for the process lifetime), with
  stop-copy-start and sqlite3 .backup as the safe procedures
- restore: the whole set together, since a missing events DB is
  created empty rather than erroring, and chown to UID 1000 because
  SQLite needs to write the journal beside the database
- upgrade: AutoMigrate runs unconditionally on every start against
  all three database kinds, there is no schema version table and no
  down migrations, so back up first and downgrade is unsupported
- that event and archive databases hold full payload bodies and
  headers, that event databases also carry target credentials until
  #206 is fixed, and that target config in webhooker.db is
  unencrypted, tracked in #212

Docs only; no code, config, script or build changes.
clawbot added the needs-review label 2026-08-20 06:04:32 +02:00
clawbot self-assigned this 2026-08-20 06:04:35 +02:00
clawbot merged commit 10c8dd2331 into next 2026-08-20 06:05:15 +02:00
clawbot deleted branch issue-210-backup-restore-upgrade-docs 2026-08-20 06:05:16 +02:00
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
Clear labels
merge-ready
needs-checks
needs-rebase
needs-review
needs-rework
No Label needs-review
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
clawbot sneak
No Assignees clawbot
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sneak/webhooker#214
Delete Branch "issue-210-backup-restore-upgrade-docs"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?