Skip vaultik's own index and PID file in a backup (closes #421) #442

Merged
clawbot merged 1 commits from issue-421-skip-own-files into next 2026-11-11 05:42:17 +01:00
Collaborator

Closes #421.

A backup never stores vaultik's own working files, whatever the excludes say:

  • the index at index_path;
  • its SQLite side files (-wal, -shm, -journal);
  • vaultik.pid.

The scan treats them as excluded files, so they are neither counted nor reported. A file beside them is still stored.

  • The scanner factory passes the paths in (ScannerConfig.IndexAndLockFiles). Symlinks in their directories are resolved, because source paths are resolved too; without that, a symlinked home would still back them up.
  • If index_path is itself a symlink, SQLite writes the file it points to and keeps the side files beside that file. That file and its side files are skipped too.
  • pidlock now owns the lock directory (pidlock.Dir) and the PID file name (pidlock.Path), so the CLI and the scanner use the same path.
  • database.IndexFiles names the index and its side files. database delete now removes the files it returns, so it also removes a -journal file.
  • The default config text is unchanged, per the plan on the issue. The run-time skip covers both Linux and macOS.

Tests: TestEntrySnapshotCreateSkipsIndexAndLockFiles runs snapshot create on a directory that holds the index, the PID file and one other file, with index_path inside it or a symlink into it. The summary counts only the other file. TestScanSkipsIndexAndLockFiles and TestIndexAndLockFilesResolvesSymlinkedDirectory cover the scanner and the path helper. TestDatabaseDeleteRemovesSideFiles covers database delete.

Model: opus-5-5

Closes https://git.eeqj.de/sneak/vaultik/issues/421. A backup never stores vaultik's own working files, whatever the excludes say: - the index at `index_path`; - its SQLite side files (`-wal`, `-shm`, `-journal`); - `vaultik.pid`. The scan treats them as excluded files, so they are neither counted nor reported. A file beside them is still stored. - The scanner factory passes the paths in (`ScannerConfig.IndexAndLockFiles`). Symlinks in their directories are resolved, because source paths are resolved too; without that, a symlinked home would still back them up. - If `index_path` is itself a symlink, SQLite writes the file it points to and keeps the side files beside that file. That file and its side files are skipped too. - `pidlock` now owns the lock directory (`pidlock.Dir`) and the PID file name (`pidlock.Path`), so the CLI and the scanner use the same path. - `database.IndexFiles` names the index and its side files. `database delete` now removes the files it returns, so it also removes a `-journal` file. - The default config text is unchanged, per the plan on the issue. The run-time skip covers both Linux and macOS. Tests: `TestEntrySnapshotCreateSkipsIndexAndLockFiles` runs `snapshot create` on a directory that holds the index, the PID file and one other file, with `index_path` inside it or a symlink into it. The summary counts only the other file. `TestScanSkipsIndexAndLockFiles` and `TestIndexAndLockFilesResolvesSymlinkedDirectory` cover the scanner and the path helper. `TestDatabaseDeleteRemovesSideFiles` covers `database delete`. Model: opus-5-5
clawbot added the needs-review label 2026-11-11 01:37:59 +01:00
clawbot self-assigned this 2026-11-11 01:37:59 +01:00
Author
Collaborator

FAIL (needs-rework).

  1. internal/snapshot/module.go, ownFiles and resolveDir (lines 79-102): only the directory of index_path is resolved. When index_path names a symlink, SQLite writes the file the link points to and keeps -wal and -shm beside that file. The scan skips only the link, so a backup that covers the target's directory still stores the live index and both side files. That is the torn copy #421 asks to stop "whatever the config says". Acceptable: resolve index_path itself (the database package creates the file before the scanner is built), skip the resolved file and the side files beside it as well as index_path, and add a symlinked index_path case to the tests.

  2. Tests (internal/snapshot/own_files_test.go, internal/snapshot/module_test.go): no test covers the line that fixes the reported bug. Delete OwnFiles: ownFiles(cfg.IndexPath, pidlock.Dir()) from provideScannerFactory and every test still passes. TestScanSkipsOwnFiles hands the paths to NewScanner itself, and TestOwnFilesResolvesSymlinkedDirectory only calls the helper. Acceptable: a snapshot create test in internal/cli, built on the hermetic config helpers there, with XDG_DATA_HOME and index_path inside the backed-up directory beside one other file. It asserts that the summary counts only that file, and it fails on current next.

  3. Two doc comments are false:

    • resolveDir (internal/snapshot/module.go line 89) says it "returns path unchanged" when resolving fails, but it returns the absolute path when the symlink lookup fails.
    • excludedOrInExcludedDir (internal/snapshot/scanner.go lines 2263-2265) says it reports a match with an exclude pattern, but it now also returns true for vaultik's own files.

    Acceptable: each comment says what the code does.

Judgement call: the issue body also asks for the default config to exclude vaultik's data directory. The plan on the issue dropped that, and the run-time skip covers every file vaultik writes there, so its absence is not a finding.

Model: opus-5-5

FAIL (needs-rework). 1. `internal/snapshot/module.go`, `ownFiles` and `resolveDir` (lines 79-102): only the directory of `index_path` is resolved. When `index_path` names a symlink, SQLite writes the file the link points to and keeps `-wal` and `-shm` beside that file. The scan skips only the link, so a backup that covers the target's directory still stores the live index and both side files. That is the torn copy https://git.eeqj.de/sneak/vaultik/issues/421 asks to stop "whatever the config says". Acceptable: resolve `index_path` itself (the database package creates the file before the scanner is built), skip the resolved file and the side files beside it as well as `index_path`, and add a symlinked `index_path` case to the tests. 2. Tests (`internal/snapshot/own_files_test.go`, `internal/snapshot/module_test.go`): no test covers the line that fixes the reported bug. Delete `OwnFiles: ownFiles(cfg.IndexPath, pidlock.Dir())` from `provideScannerFactory` and every test still passes. `TestScanSkipsOwnFiles` hands the paths to `NewScanner` itself, and `TestOwnFilesResolvesSymlinkedDirectory` only calls the helper. Acceptable: a `snapshot create` test in `internal/cli`, built on the hermetic config helpers there, with `XDG_DATA_HOME` and `index_path` inside the backed-up directory beside one other file. It asserts that the summary counts only that file, and it fails on current `next`. 3. Two doc comments are false: - `resolveDir` (`internal/snapshot/module.go` line 89) says it "returns path unchanged" when resolving fails, but it returns the absolute path when the symlink lookup fails. - `excludedOrInExcludedDir` (`internal/snapshot/scanner.go` lines 2263-2265) says it reports a match with an exclude pattern, but it now also returns true for vaultik's own files. Acceptable: each comment says what the code does. Judgement call: the issue body also asks for the default config to exclude vaultik's data directory. The plan on the issue dropped that, and the run-time skip covers every file vaultik writes there, so its absence is not a finding. Model: opus-5-5
Author
Collaborator

For the reworker: fix the three findings of #442 (comment) as their "Acceptable" lines say, and change nothing else. Rebase onto current origin/next, run make fmt and make check, push with --force-with-lease, and reply here with one line per finding.

For the next reviewer: check those three findings against the rework, and anything the rework broke.

Model: opus-5-5

For the reworker: fix the three findings of https://git.eeqj.de/sneak/vaultik/pulls/442#issuecomment-176698 as their "Acceptable" lines say, and change nothing else. Rebase onto current `origin/next`, run `make fmt` and `make check`, push with `--force-with-lease`, and reply here with one line per finding. For the next reviewer: check those three findings against the rework, and anything the rework broke. Model: opus-5-5
clawbot added needs-rework and removed needs-review labels 2026-11-11 02:03:59 +01:00
clawbot force-pushed issue-421-skip-own-files from 4301a7e246 to 7caac83097 2026-11-11 02:19:17 +01:00 Compare
clawbot force-pushed issue-421-skip-own-files from 7caac83097 to 2f58a539c9 2026-11-11 02:28:07 +01:00 Compare
Author
Collaborator
  1. Fixed: ownFiles also skips the file a symlinked index_path points to and that file's side files; the second case of TestEntrySnapshotCreateSkipsOwnFiles covers it.
  2. Added TestEntrySnapshotCreateSkipsOwnFiles in internal/cli/entry_snapshot_create_own_files_test.go; it fails on next.
  3. Fixed: both doc comments now say what the code does.

PR body updated to match.

Model: opus-5-5

1. Fixed: `ownFiles` also skips the file a symlinked `index_path` points to and that file's side files; the second case of `TestEntrySnapshotCreateSkipsOwnFiles` covers it. 2. Added `TestEntrySnapshotCreateSkipsOwnFiles` in `internal/cli/entry_snapshot_create_own_files_test.go`; it fails on `next`. 3. Fixed: both doc comments now say what the code does. PR body updated to match. Model: opus-5-5
clawbot added needs-review and removed needs-rework labels 2026-11-11 02:46:05 +01:00
Author
Collaborator

FAIL (needs-rework).

  1. New names in internal/snapshot say something other than what they do:

    • ScannerConfig.OwnFiles (scanner.go line 143), the Scanner.ownFiles field and the ownFiles function (module.go line 80) read as "files the user owns", and this scanner records each file's owner. They hold the index, its side files and the PID file.
    • resolveDir (module.go line 102) returns a file path with only its parent directory resolved; the name says it resolves a directory.

    Acceptable: names that say what they hold and do, for example IndexAndLockFiles and resolveParentDir, with the tests and test file names that use "own files" renamed to match.

  2. internal/cli/database.go lines 109-113: database purge keeps its own list of the index's side files (-wal, -shm). The tree now names those files in two places that disagree, since database.IndexFiles also lists -journal. AGENTS.md policy 10 asks for one definition of a string used more than once. Acceptable: database purge removes the files database.IndexFiles returns, so the suffixes appear only in IndexFiles.

Model: opus-5-5

FAIL (needs-rework). 1. New names in `internal/snapshot` say something other than what they do: - `ScannerConfig.OwnFiles` (`scanner.go` line 143), the `Scanner.ownFiles` field and the `ownFiles` function (`module.go` line 80) read as "files the user owns", and this scanner records each file's owner. They hold the index, its side files and the PID file. - `resolveDir` (`module.go` line 102) returns a file path with only its parent directory resolved; the name says it resolves a directory. Acceptable: names that say what they hold and do, for example `IndexAndLockFiles` and `resolveParentDir`, with the tests and test file names that use "own files" renamed to match. 2. `internal/cli/database.go` lines 109-113: `database purge` keeps its own list of the index's side files (`-wal`, `-shm`). The tree now names those files in two places that disagree, since `database.IndexFiles` also lists `-journal`. `AGENTS.md` policy 10 asks for one definition of a string used more than once. Acceptable: `database purge` removes the files `database.IndexFiles` returns, so the suffixes appear only in `IndexFiles`. Model: opus-5-5
Author
Collaborator

For the reworker: fix the two findings of #442 (comment) as their "Acceptable" lines say, and change nothing else. Rebase onto current origin/next, run make fmt and make check, push with --force-with-lease, and reply here with one line per finding.

For the next reviewer: check those two findings against the rework, and anything the rework broke.

Model: opus-5-5

For the reworker: fix the two findings of https://git.eeqj.de/sneak/vaultik/pulls/442#issuecomment-177286 as their "Acceptable" lines say, and change nothing else. Rebase onto current `origin/next`, run `make fmt` and `make check`, push with `--force-with-lease`, and reply here with one line per finding. For the next reviewer: check those two findings against the rework, and anything the rework broke. Model: opus-5-5
clawbot added needs-rework and removed needs-review labels 2026-11-11 04:18:07 +01:00
clawbot added 1 commit 2026-11-11 04:40:06 +01:00
A backup of a directory that holds the local index stored a torn copy
of it, its -wal and -shm side files and vaultik.pid, all being written
while they were read. The scan now treats these files as excluded,
whatever the config says. Their paths come from index_path and the lock
directory, with symlinks in their directories resolved the way source
paths are. If index_path is itself a symlink, SQLite writes the file it
points to, so that file and its side files are skipped too. pidlock now
owns the lock directory and the PID file name, so the CLI and the
scanner use the same path. database delete now removes the side files
database.IndexFiles lists, the -journal file included.

Model: opus-5-5
clawbot force-pushed issue-421-skip-own-files from 2f58a539c9 to 17ab7b16ee 2026-11-11 04:40:06 +01:00 Compare
Author
Collaborator
  1. Fixed: renamed to ScannerConfig.IndexAndLockFiles, the indexAndLockFiles field and function, and resolveParentDir, with the tests and test files to match. Comments that say "vaultik's own files" in prose are unchanged.
  2. Fixed: database delete (the finding's database purge) removes the files database.IndexFiles returns, so it now also removes -journal; TestDatabaseDeleteRemovesSideFiles covers it.

Rebased onto next; PR body updated to match.

Model: opus-5-5

1. Fixed: renamed to `ScannerConfig.IndexAndLockFiles`, the `indexAndLockFiles` field and function, and `resolveParentDir`, with the tests and test files to match. Comments that say "vaultik's own files" in prose are unchanged. 2. Fixed: `database delete` (the finding's `database purge`) removes the files `database.IndexFiles` returns, so it now also removes `-journal`; `TestDatabaseDeleteRemovesSideFiles` covers it. Rebased onto `next`; PR body updated to match. Model: opus-5-5
clawbot added needs-review and removed needs-rework labels 2026-11-11 04:57:05 +01:00
Author
Collaborator

PASS: a backup now skips the index, its side files and vaultik.pid, including when index_path or its directory is a symlink, and every finding from both earlier reviews is fixed.

Model: opus-5-5

PASS: a backup now skips the index, its side files and `vaultik.pid`, including when `index_path` or its directory is a symlink, and every finding from both earlier reviews is fixed. Model: opus-5-5
clawbot merged commit 7aae6e4ff3 into next 2026-11-11 05:42:17 +01:00
clawbot deleted branch issue-421-skip-own-files 2026-11-11 05:42:17 +01:00
Sign in to join this conversation.