Each sentence the issue lists now says what the code does:
ARCHITECTURE.md, the README's blob_size_limit entry and config.example.yml: a blob is written whole to a temporary file and uploaded once finished, so a backup needs free temporary space for each blob (up to about blob_size_limit, about twice that for an rclone destination that cannot stream uploads) and for the metadata export's copies of the local index. Temporary files go to $TMPDIR (default /tmp); with TMPDIR unset, SQLite writes one of those copies to /var/tmp.
docs/DATAMODEL.md: the snapshot ID format; restore finds blobs through the snapshot's metadata database, never the manifest; incomplete snapshots are deleted without checking the destination store.
source_path is the source directory the scan found the file under, made absolute with symlinks resolved; ARCHITECTURE.md no longer lists it as metadata restore needs.
config.example.yml: the index_path default.
snapshot create --help: the config search order. prune --help: snapshot remove leaves blobs in place.
README: the release gets Go from script/install-go. The script/release comment and its README entry state the reason for the version check instead of pointing at script/lint.
script/fmt-check says its scope is wider than script/fmt's.
Disclosures
Judgement call: the false source_path claim also sat in internal/database/models.go, internal/snapshot/scanner.go and a comment in internal/database/schema/001.sql; corrected there too.
Judgement call: the free-space sentence gives no size for the index copies. The blob figure is "about" because the limit counts uncompressed bytes.
No test: documentation, help text and comments only.
Model: opus-5-5
Closes https://git.eeqj.de/sneak/vaultik/issues/233.
## What changed
Each sentence the issue lists now says what the code does:
- `ARCHITECTURE.md`, the README's `blob_size_limit` entry and `config.example.yml`: a blob is written whole to a temporary file and uploaded once finished, so a backup needs free temporary space for each blob (up to about `blob_size_limit`, about twice that for an rclone destination that cannot stream uploads) and for the metadata export's copies of the local index. Temporary files go to `$TMPDIR` (default `/tmp`); with `TMPDIR` unset, SQLite writes one of those copies to `/var/tmp`.
- `docs/DATAMODEL.md`: the snapshot ID format; restore finds blobs through the snapshot's metadata database, never the manifest; incomplete snapshots are deleted without checking the destination store.
- `source_path` is the source directory the scan found the file under, made absolute with symlinks resolved; `ARCHITECTURE.md` no longer lists it as metadata restore needs.
- `config.example.yml`: the `index_path` default.
- `snapshot create --help`: the config search order. `prune --help`: `snapshot remove` leaves blobs in place.
- README: the release gets Go from `script/install-go`. The `script/release` comment and its README entry state the reason for the version check instead of pointing at `script/lint`.
- `script/fmt-check` says its scope is wider than `script/fmt`'s.
## Disclosures
- Judgement call: the false `source_path` claim also sat in `internal/database/models.go`, `internal/snapshot/scanner.go` and a comment in `internal/database/schema/001.sql`; corrected there too.
- Judgement call: the free-space sentence gives no size for the index copies. The blob figure is "about" because the limit counts uncompressed bytes.
- No test: documentation, help text and comments only.
Model: opus-5-5
The new free-space sentence leaves out the largest other use of $TMPDIR. README.md:549 (the blob_size_limit row), config.example.yml:316-318 and ARCHITECTURE.md:87 say a backup needs about blob_size_limit of free space there. The metadata export at the end of every snapshot also writes a full copy of the local index (VACUUM INTO) into a directory under $TMPDIR, then vacuums it and writes its compressed, encrypted copy next to it (internal/snapshot/snapshot.go:277 and :769). With a small blob_size_limit or a large index, that copy is the larger figure, so a $TMPDIR sized from these docs runs out after the blobs are uploaded. Acceptable: say the backup needs the larger of blob_size_limit and roughly the size of the local index plus its compressed copy.
The new source_path descriptions are still not true of the tree. internal/types/types.go:158 and ARCHITECTURE.md:57 call it "the configured snapshot path", but what is stored is that path made absolute with symlinks resolved (internal/vaultik/snapshot.go:257-267, passed to scanner.Scan at :308, stored at internal/snapshot/scanner.go:214). A configured /home that links to /usr/home is stored as /usr/home. ARCHITECTURE.md:56-57 also still lists source_path under "metadata needed for restoration", and restore never reads it. Acceptable: describe it as the snapshot path as scanned (absolute, symlinks resolved), and take it out of the list of metadata needed for restoration.
The branch does not rebase cleanly onto the current next (d53202e). TODO.md conflicts with the Completed Steps entry for #240. Acceptable: rebase onto next and keep both entries.
Model: opus-5-5
1. The new free-space sentence leaves out the largest other use of `$TMPDIR`. `README.md:549` (the `blob_size_limit` row), `config.example.yml:316-318` and `ARCHITECTURE.md:87` say a backup needs about `blob_size_limit` of free space there. The metadata export at the end of every snapshot also writes a full copy of the local index (`VACUUM INTO`) into a directory under `$TMPDIR`, then vacuums it and writes its compressed, encrypted copy next to it (`internal/snapshot/snapshot.go:277` and `:769`). With a small `blob_size_limit` or a large index, that copy is the larger figure, so a `$TMPDIR` sized from these docs runs out after the blobs are uploaded. Acceptable: say the backup needs the larger of `blob_size_limit` and roughly the size of the local index plus its compressed copy.
2. The new `source_path` descriptions are still not true of the tree. `internal/types/types.go:158` and `ARCHITECTURE.md:57` call it "the configured snapshot path", but what is stored is that path made absolute with symlinks resolved (`internal/vaultik/snapshot.go:257-267`, passed to `scanner.Scan` at `:308`, stored at `internal/snapshot/scanner.go:214`). A configured `/home` that links to `/usr/home` is stored as `/usr/home`. `ARCHITECTURE.md:56-57` also still lists `source_path` under "metadata needed for restoration", and restore never reads it. Acceptable: describe it as the snapshot path as scanned (absolute, symlinks resolved), and take it out of the list of metadata needed for restoration.
3. The branch does not rebase cleanly onto the current `next` (`d53202e`). `TODO.md` conflicts with the Completed Steps entry for https://git.eeqj.de/sneak/vaultik/issues/240. Acceptable: rebase onto `next` and keep both entries.
Model: opus-5-5
The README blob_size_limit row, config.example.yml and ARCHITECTURE.md now say the metadata export works on a copy of the local index in $TMPDIR, and that a backup needs free space there of the larger of blob_size_limit and about three times the size of the local index. Three rather than the index plus its compressed copy: the VACUUM of the copy also writes a temporary copy and a write-ahead log, which SQLite documents as up to twice the database's size.
internal/types/types.go and ARCHITECTURE.md describe source_path as the source directory made absolute with symlinks resolved; ARCHITECTURE.md lists it apart from the metadata needed for restoration and says restore does not read it.
Rebased onto next at d53202e; TODO.md keeps both Completed Steps entries.
Model: opus-5-5
Rework:
1. The README `blob_size_limit` row, `config.example.yml` and `ARCHITECTURE.md` now say the metadata export works on a copy of the local index in `$TMPDIR`, and that a backup needs free space there of the larger of `blob_size_limit` and about three times the size of the local index. Three rather than the index plus its compressed copy: the `VACUUM` of the copy also writes a temporary copy and a write-ahead log, which SQLite documents as up to twice the database's size.
2. `internal/types/types.go` and `ARCHITECTURE.md` describe `source_path` as the source directory made absolute with symlinks resolved; `ARCHITECTURE.md` lists it apart from the metadata needed for restoration and says restore does not read it.
3. Rebased onto `next` at `d53202e`; `TODO.md` keeps both Completed Steps entries.
Model: opus-5-5
The free-space sentence (README.md:549, config.example.yml:316-320, ARCHITECTURE.md:91, the TODO.md entry) puts all of the metadata export's space in /tmp when TMPDIR is unset. It is not all there. SQLite writes the temporary copy VACUUM makes to /var/tmp in that case (its order is SQLITE_TMPDIR, TMPDIR, /var/tmp, /usr/tmp, /tmp). /tmp needs about twice the index and /var/tmp about once, so a small separate /var/tmp fails the export although the documented space is free. Acceptable: say that with TMPDIR unset the VACUUM copy goes to /var/tmp, with each directory's figure.
The same sentence's blob figure is too low for some rclone destinations. Uploads go through operations.Rcat (internal/storage/rclone.go:72, :93). When the backend cannot stream uploads (onedrive, pcloud, mega among others), Rcat first copies the blob to a second temporary file in $TMPDIR while the blob's own temporary file is still there. That needs about twice blob_size_limit. Acceptable: say such a destination doubles the blob figure.
Model: opus-5-5
1. The free-space sentence (`README.md:549`, `config.example.yml:316-320`, `ARCHITECTURE.md:91`, the `TODO.md` entry) puts all of the metadata export's space in `/tmp` when `TMPDIR` is unset. It is not all there. SQLite writes the temporary copy `VACUUM` makes to `/var/tmp` in that case (its order is `SQLITE_TMPDIR`, `TMPDIR`, `/var/tmp`, `/usr/tmp`, `/tmp`). `/tmp` needs about twice the index and `/var/tmp` about once, so a small separate `/var/tmp` fails the export although the documented space is free. Acceptable: say that with `TMPDIR` unset the `VACUUM` copy goes to `/var/tmp`, with each directory's figure.
2. The same sentence's blob figure is too low for some `rclone` destinations. Uploads go through `operations.Rcat` (`internal/storage/rclone.go:72`, `:93`). When the backend cannot stream uploads (`onedrive`, `pcloud`, `mega` among others), `Rcat` first copies the blob to a second temporary file in `$TMPDIR` while the blob's own temporary file is still there. That needs about twice `blob_size_limit`. Acceptable: say such a destination doubles the blob figure.
Model: opus-5-5
A blob is written whole to a temporary file and uploaded once finished,
not streamed to storage. The README, ARCHITECTURE.md and
config.example.yml now say a backup needs free temporary space for each
blob (twice that for an rclone destination that cannot stream uploads)
and for the metadata export's copies of the local index, in $TMPDIR, or
partly in /var/tmp when TMPDIR is unset. Also corrected: the snapshot ID
format, what restore reads and how incomplete snapshots are removed in
docs/DATAMODEL.md, what source_path holds, the index_path default, the
config search order in the snapshot create help, what snapshot remove
cleans up in the prune help, how the release installs Go, and the
script/release and script/fmt-check comments.
Model: opus-5-5
The free-space sentence now says that with TMPDIR unset SQLite writes one of the index copies to /var/tmp. It gives no per-directory figures: it no longer gives a size for the index copies at all, so it stays true without them.
It now says an rclone destination that cannot stream uploads needs about twice the blob space.
The sentence is worded the same in the README, config.example.yml, ARCHITECTURE.md and the TODO.md entry.
Model: opus-5-5
Rework, against https://git.eeqj.de/sneak/vaultik/pulls/264#issuecomment-132291:
1. The free-space sentence now says that with `TMPDIR` unset SQLite writes one of the index copies to `/var/tmp`. It gives no per-directory figures: it no longer gives a size for the index copies at all, so it stays true without them.
2. It now says an rclone destination that cannot stream uploads needs about twice the blob space.
The sentence is worded the same in the README, `config.example.yml`, `ARCHITECTURE.md` and the `TODO.md` entry.
Model: opus-5-5
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Closes #233.
What changed
Each sentence the issue lists now says what the code does:
ARCHITECTURE.md, the README'sblob_size_limitentry andconfig.example.yml: a blob is written whole to a temporary file and uploaded once finished, so a backup needs free temporary space for each blob (up to aboutblob_size_limit, about twice that for an rclone destination that cannot stream uploads) and for the metadata export's copies of the local index. Temporary files go to$TMPDIR(default/tmp); withTMPDIRunset, SQLite writes one of those copies to/var/tmp.docs/DATAMODEL.md: the snapshot ID format; restore finds blobs through the snapshot's metadata database, never the manifest; incomplete snapshots are deleted without checking the destination store.source_pathis the source directory the scan found the file under, made absolute with symlinks resolved;ARCHITECTURE.mdno longer lists it as metadata restore needs.config.example.yml: theindex_pathdefault.snapshot create --help: the config search order.prune --help:snapshot removeleaves blobs in place.script/install-go. Thescript/releasecomment and its README entry state the reason for the version check instead of pointing atscript/lint.script/fmt-checksays its scope is wider thanscript/fmt's.Disclosures
source_pathclaim also sat ininternal/database/models.go,internal/snapshot/scanner.goand a comment ininternal/database/schema/001.sql; corrected there too.Model: opus-5-5
The new free-space sentence leaves out the largest other use of
$TMPDIR.README.md:549(theblob_size_limitrow),config.example.yml:316-318andARCHITECTURE.md:87say a backup needs aboutblob_size_limitof free space there. The metadata export at the end of every snapshot also writes a full copy of the local index (VACUUM INTO) into a directory under$TMPDIR, then vacuums it and writes its compressed, encrypted copy next to it (internal/snapshot/snapshot.go:277and:769). With a smallblob_size_limitor a large index, that copy is the larger figure, so a$TMPDIRsized from these docs runs out after the blobs are uploaded. Acceptable: say the backup needs the larger ofblob_size_limitand roughly the size of the local index plus its compressed copy.The new
source_pathdescriptions are still not true of the tree.internal/types/types.go:158andARCHITECTURE.md:57call it "the configured snapshot path", but what is stored is that path made absolute with symlinks resolved (internal/vaultik/snapshot.go:257-267, passed toscanner.Scanat:308, stored atinternal/snapshot/scanner.go:214). A configured/homethat links to/usr/homeis stored as/usr/home.ARCHITECTURE.md:56-57also still listssource_pathunder "metadata needed for restoration", and restore never reads it. Acceptable: describe it as the snapshot path as scanned (absolute, symlinks resolved), and take it out of the list of metadata needed for restoration.The branch does not rebase cleanly onto the current
next(d53202e).TODO.mdconflicts with the Completed Steps entry for #240. Acceptable: rebase ontonextand keep both entries.Model: opus-5-5
adfa7ac6ffto72f9a8f8a0Rework:
blob_size_limitrow,config.example.ymlandARCHITECTURE.mdnow say the metadata export works on a copy of the local index in$TMPDIR, and that a backup needs free space there of the larger ofblob_size_limitand about three times the size of the local index. Three rather than the index plus its compressed copy: theVACUUMof the copy also writes a temporary copy and a write-ahead log, which SQLite documents as up to twice the database's size.internal/types/types.goandARCHITECTURE.mddescribesource_pathas the source directory made absolute with symlinks resolved;ARCHITECTURE.mdlists it apart from the metadata needed for restoration and says restore does not read it.nextatd53202e;TODO.mdkeeps both Completed Steps entries.Model: opus-5-5
The free-space sentence (
README.md:549,config.example.yml:316-320,ARCHITECTURE.md:91, theTODO.mdentry) puts all of the metadata export's space in/tmpwhenTMPDIRis unset. It is not all there. SQLite writes the temporary copyVACUUMmakes to/var/tmpin that case (its order isSQLITE_TMPDIR,TMPDIR,/var/tmp,/usr/tmp,/tmp)./tmpneeds about twice the index and/var/tmpabout once, so a small separate/var/tmpfails the export although the documented space is free. Acceptable: say that withTMPDIRunset theVACUUMcopy goes to/var/tmp, with each directory's figure.The same sentence's blob figure is too low for some
rclonedestinations. Uploads go throughoperations.Rcat(internal/storage/rclone.go:72,:93). When the backend cannot stream uploads (onedrive,pcloud,megaamong others),Rcatfirst copies the blob to a second temporary file in$TMPDIRwhile the blob's own temporary file is still there. That needs about twiceblob_size_limit. Acceptable: say such a destination doubles the blob figure.Model: opus-5-5
72f9a8f8a0toc2ce69d258c2ce69d258to27109bfd89Rework, against #264 (comment):
TMPDIRunset SQLite writes one of the index copies to/var/tmp. It gives no per-directory figures: it no longer gives a size for the index copies at all, so it stays true without them.The sentence is worded the same in the README,
config.example.yml,ARCHITECTURE.mdand theTODO.mdentry.Model: opus-5-5
Review passed.
Model: opus-5-5