From 79fccfecce82e042eae2361e84e68fd211034f86 Mon Sep 17 00:00:00 2001 From: sneak Date: Fri, 6 Nov 2026 19:34:28 +0000 Subject: [PATCH] Move the roadmap into TODO.md and fix a test comment The rewrite removed the README's roadmap, which TODO.md linked to. The roadmap items are now under Future Steps in TODO.md, in place of the link. A comment in internal/storage/s3_test.go cited the README for the destination layout. That layout is in docs/REPOSTRUCTURE.md. Model: fable-5-1 --- TODO.md | 63 +++++++++++++++++++++++++++++++++++-- internal/storage/s3_test.go | 2 +- 2 files changed, 62 insertions(+), 3 deletions(-) diff --git a/TODO.md b/TODO.md index c32824f..849ebf8 100644 --- a/TODO.md +++ b/TODO.md @@ -1316,5 +1316,64 @@ owner's; a `v*` tag runs the release build in # Future Steps -Work planned after 1.0 is listed in the README -[roadmap](README.md#roadmap). +Work planned after 1.0, loosely ordered by priority. + +## Correctness and operability + +- **Outside security audit.** Before 1.0 the encryption and + blob-generation code was reviewed. Every bug the review found was + fixed. The risks it accepted are listed in + [Accepted risks](docs/REPOSTRUCTURE.md#accepted-risks). No outside + audit has been done. age, zstd and content-defined chunking are mostly + off-the-shelf pieces. The code that joins them needs an outside read: + - key handling + - recipient parsing + - manifest trust boundary + - restore-time identity validation +- **Verify restored content end-to-end in CI.** The current integration + test does this for a small synthetic snapshot but not at scale. A + nightly job against a multi-GB representative snapshot would catch + silent regressions in the chunker, packer, or restore planner. + +## Performance + +- **Parallel blob downloads during restore.** A restore downloads one + blob at a time now. With a fast S3 endpoint and a multi-core machine, + restore speed is bound by the fetch, decryption and decompression of + each blob. Running several of those in parallel against the disk + cache would close most of the remaining gap. This needs to interact + correctly with the locality planner and the sweeper. +- **Bandwidth limiting (`--bwlimit`).** For both upload and download. + It is useful for backing up over a shared link. It is tricky to make + work correctly with parallel downloads. +- **Restart of interrupted restore.** Today a restore can be started + again, and it then overwrites the partial output. It does not resume + from where it stopped, and it does not skip files that are already + present. A `--resume` mode that checks targets before fetching blobs + would matter for very large restores. +- **Daemon mode.** A long-running mode that watches for file changes, + so that frequent backups, such as hourly ones, skip the full scan. It + adds little for the usual backups from cron every 12 to 36 hours. See + [issue #204](https://git.eeqj.de/sneak/vaultik/issues/204). + +## Usability + +- **Man pages and richer `--help` examples.** Cobra generates basic + help. Man pages would be a separate target. +- **`vaultik snapshot diff `.** Show which files changed between + two snapshots, without restoring either. +- **Status reporting hook for `--cron`.** When a backup fails silently + in cron, the user has no idea. A configurable hook on completion, for + both success and failure, would close the loop. It could be a + webhook, an email or `notify-send`. + +## Infrastructure + +- **Schema migrations after 1.0.** Migrations are supported, but none + are added before 1.0 because there is no installed base yet. After + 1.0, each schema change is a new migration, so an existing local + index is migrated when `vaultik` is updated. See + [`docs/DATAMODEL.md`](docs/DATAMODEL.md#schema-migrations). +- **Storage backend coverage tests.** S3, `file://` and `rclone://` all + share the `Storer` interface, but the rclone path is the least + exercised in CI. diff --git a/internal/storage/s3_test.go b/internal/storage/s3_test.go index 8902bc7..7758b38 100644 --- a/internal/storage/s3_test.go +++ b/internal/storage/s3_test.go @@ -161,7 +161,7 @@ func TestS3ListingCutShortIsAnError(t *testing.T) { } // TestS3URLPrefixKeyLayout pins the bucket keys an s3:// URL reads and -// writes: the README's remote storage layout, with the prefix joined to +// writes: the layout in docs/REPOSTRUCTURE.md, with the prefix joined to // each key by one "/". s3://b/p and s3://b/p/ must be the same // destination, or a host that writes the URL the other way finds no // snapshots. The listed object is put straight into the bucket, as