clawbot
2bf51d65e3
check / check (push) Failing after 19m7s
- snapshot verify and snapshot restore also list the destination for a snapshot ID that is not in the local index: README, REPOSTRUCTURE, ARCHITECTURE, DATAMODEL and the two commands' help. - prune on a separate machine deletes only unused data and does not remove old snapshots. - The --quiet usage string names everything the flag leaves out. - A warning written as a log record stays on stderr. - config set on a config that does not load refuses three things. - No test is held back by a build tag. - A plain docker build . adds no -dirty. - The backup's order of work records directories. - TODO.md says what parallel downloads would speed up and names restorePlan and restoreSweeper. Model: fable-5-1
81 lines
3.5 KiB
Markdown
81 lines
3.5 KiB
Markdown
# Workflow
|
|
|
|
- branch (from `main`)
|
|
- do the work in Next Step
|
|
- delete Next Step from `TODO.md`
|
|
- move the top item of Future Steps into Next Step
|
|
- commit (`TODO.md` changes in the same commit as the work)
|
|
- merge to `main` if the branch is not protected, otherwise open a PR
|
|
- push
|
|
|
|
# Status
|
|
|
|
pre-1.0
|
|
|
|
# Next Step
|
|
|
|
Fix the open bugs on the issue tracker that block the release. Merging `next`
|
|
into `main` is the owner's.
|
|
|
|
# Future Steps
|
|
|
|
Work planned after 1.0, loosely ordered by priority.
|
|
|
|
## Correctness and operability
|
|
|
|
- **Outside security audit.** Before 1.0 the encryption and blob-generation code
|
|
was reviewed. Every bug the review found was fixed. The risks it accepted are
|
|
listed in [Accepted risks](REPOSTRUCTURE.md#accepted-risks). No outside audit
|
|
has been done. age, zstd and content-defined chunking are mostly off-the-shelf
|
|
pieces. The code that joins them needs an outside read:
|
|
- key handling
|
|
- recipient parsing
|
|
- manifest trust boundary
|
|
- restore-time identity validation
|
|
- **Verify restored content end-to-end in CI.** The current integration test
|
|
does this for a small synthetic snapshot but not at scale. A nightly job
|
|
against a multi-GB representative snapshot would catch silent regressions in
|
|
the chunker, packer, or restore planner.
|
|
|
|
## Performance
|
|
|
|
- **Parallel blob downloads during restore.** A restore downloads one blob at a
|
|
time now. With a fast S3 endpoint and a multi-core machine, restore speed is
|
|
bound by the fetch, decryption and decompression of each blob. Fetching,
|
|
decrypting and decompressing several blobs at once would make such a restore
|
|
faster. The change must work with `restorePlan`, which orders the files, and
|
|
with `restoreSweeper`, which deletes a cached blob once no remaining file
|
|
needs it.
|
|
- **Bandwidth limiting (`--bwlimit`).** For both upload and download. It is
|
|
useful for backing up over a shared link. It is tricky to make work correctly
|
|
with parallel downloads.
|
|
- **Restart of interrupted restore.** Today a restore can be started again, and
|
|
it then overwrites the partial output. It does not resume from where it
|
|
stopped, and it does not skip files that are already present. A `--resume`
|
|
mode that checks targets before fetching blobs would matter for very large
|
|
restores.
|
|
- **Daemon mode.** A long-running mode that watches for file changes, so that
|
|
frequent backups, such as hourly ones, skip the full scan. It adds little for
|
|
the usual backups from cron every 12 to 36 hours. See
|
|
[issue #204](https://git.eeqj.de/sneak/vaultik/issues/204).
|
|
|
|
## Usability
|
|
|
|
- **Man pages and richer `--help` examples.** Cobra generates basic help. Man
|
|
pages would be a separate target.
|
|
- **`vaultik snapshot diff <a> <b>`.** Show which files changed between two
|
|
snapshots, without restoring either.
|
|
- **Status reporting hook for `--cron`.** When a backup fails silently in cron,
|
|
the user has no idea. A configurable hook on completion, for both success and
|
|
failure, would close the loop. It could be a webhook, an email or
|
|
`notify-send`.
|
|
|
|
## Infrastructure
|
|
|
|
- **Schema migrations after 1.0.** Migrations are supported, but none are added
|
|
before 1.0 because there is no installed base yet. After 1.0, each schema
|
|
change is a new migration, so an existing local index is migrated when
|
|
`vaultik` is updated. See [`DATAMODEL.md`](DATAMODEL.md#schema-migrations).
|
|
- **Storage backend coverage tests.** S3, `file://` and `rclone://` all share
|
|
the `Storer` interface, but the rclone path is the least exercised in CI.
|