Go on with a backup when the destination marker cannot be checked (closes #389) #392

Merged
clawbot merged 1 commits from issue-389-marker-check-put-only into next 2026-10-15 02:02:35 +02:00
Collaborator

Fixes #389.

A source host whose credentials allow only PUT, the setup the README and docs/REPOSTRUCTURE.md promise is enough, failed every backup after its first: the marker check looks up vaultik-destination.txt with an S3 HEAD, which such credentials are refused, and any error other than not found was fatal.

checkDestinationMarker now logs any other error at info level and returns nil, so the backup goes on and writes the marker as before. Not found still returns the not-present error; it is what a file:// volume's empty mount point answers, the case the check exists for.

Worth knowing:

  • The change is in the shared check, so prune, snapshot purge, snapshot remove, snapshot list and remote nuke also go on past such an error. Each then lists or deletes on the destination, which fails by itself on a store that cannot be reached.
  • Judgement call (from the plan): info level, not a warning, so --cron stays silent on a PUT-only host's successful runs.
  • The fault-injecting test store (internal/storage/faultstore) gains an OnStat hook, in the same form as OnDelete, for the new test.
  • A PUT-only host therefore does not notice a destination emptied since its last backup. The marker sections of docs/REPOSTRUCTURE.md and the README now say so, and that vaultik database delete must then be run on such a host; ARCHITECTURE.md and the snapshot create bullet say it briefly.

Model: opus-5-5

Fixes https://git.eeqj.de/sneak/vaultik/issues/389. A source host whose credentials allow only PUT, the setup the README and `docs/REPOSTRUCTURE.md` promise is enough, failed every backup after its first: the marker check looks up `vaultik-destination.txt` with an S3 HEAD, which such credentials are refused, and any error other than not found was fatal. `checkDestinationMarker` now logs any other error at info level and returns nil, so the backup goes on and writes the marker as before. Not found still returns the not-present error; it is what a `file://` volume's empty mount point answers, the case the check exists for. Worth knowing: - The change is in the shared check, so `prune`, `snapshot purge`, `snapshot remove`, `snapshot list` and `remote nuke` also go on past such an error. Each then lists or deletes on the destination, which fails by itself on a store that cannot be reached. - Judgement call (from the plan): info level, not a warning, so `--cron` stays silent on a PUT-only host's successful runs. - The fault-injecting test store (`internal/storage/faultstore`) gains an `OnStat` hook, in the same form as `OnDelete`, for the new test. - A PUT-only host therefore does not notice a destination emptied since its last backup. The marker sections of `docs/REPOSTRUCTURE.md` and the README now say so, and that `vaultik database delete` must then be run on such a host; `ARCHITECTURE.md` and the `snapshot create` bullet say it briefly. Model: opus-5-5
clawbot added the needs-review label 2026-10-14 23:14:45 +02:00
clawbot self-assigned this 2026-10-14 23:14:45 +02:00
Author
Collaborator

FAIL (needs-rework)

  1. docs/REPOSTRUCTURE.md line 30, README.md lines 333-336 and 579-587, ARCHITECTURE.md lines 385-392: these say, with no exception, that while the local index holds snapshots a destination without vaultik-destination.txt is refused (snapshot create will not write to it; prune, snapshot purge and remote nuke fail). The code now does that only when the store answers the look-up with not found. When the store refuses the look-up, as S3 does on every run for credentials that allow only PUT, the check is skipped with an info-level log line, which is not shown without --verbose, and the command goes on. So on such a host a destination emptied since its last backup (for example by remote nuke run from the trusted host) is not caught, although the documents say it is. The PR description's reason for leaving the documents unchanged ("a missing marker behaves as before") does not hold for that host. Acceptable: the marker section of docs/REPOSTRUCTURE.md and the README's marker bullet say that when the store refuses the look-up, the check is skipped and the command goes on. They also say that a host with PUT-only credentials therefore does not notice an emptied destination, and that vaultik database delete must be run on it after the destination is emptied. ARCHITECTURE.md and the snapshot create bullet say the same or point there.

Unverified: not run against a real S3 store with a PUT-only policy.

Model: opus-5-5

**FAIL (needs-rework)** 1. `docs/REPOSTRUCTURE.md` line 30, `README.md` lines 333-336 and 579-587, `ARCHITECTURE.md` lines 385-392: these say, with no exception, that while the local index holds snapshots a destination without `vaultik-destination.txt` is refused (`snapshot create` will not write to it; `prune`, `snapshot purge` and `remote nuke` fail). The code now does that only when the store answers the look-up with not found. When the store refuses the look-up, as S3 does on every run for credentials that allow only PUT, the check is skipped with an info-level log line, which is not shown without `--verbose`, and the command goes on. So on such a host a destination emptied since its last backup (for example by `remote nuke` run from the trusted host) is not caught, although the documents say it is. The PR description's reason for leaving the documents unchanged ("a missing marker behaves as before") does not hold for that host. Acceptable: the marker section of `docs/REPOSTRUCTURE.md` and the README's marker bullet say that when the store refuses the look-up, the check is skipped and the command goes on. They also say that a host with PUT-only credentials therefore does not notice an emptied destination, and that `vaultik database delete` must be run on it after the destination is emptied. `ARCHITECTURE.md` and the `snapshot create` bullet say the same or point there. Unverified: not run against a real S3 store with a PUT-only policy. Model: opus-5-5
clawbot added needs-rework and removed needs-review labels 2026-10-14 23:49:39 +02:00
clawbot force-pushed issue-389-marker-check-put-only from 1722100267 to 64f6ca7f4d 2026-10-15 00:05:12 +02:00 Compare
Author
Collaborator

Addressed: the marker section of docs/REPOSTRUCTURE.md and the README's marker bullet now say a refused look-up skips the check, so a PUT-only host does not notice an emptied destination and vaultik database delete must be run on it afterwards; ARCHITECTURE.md and the snapshot create bullet say so briefly, and three code comments stating the old rule are corrected.

Model: opus-5-5

Addressed: the marker section of `docs/REPOSTRUCTURE.md` and the README's marker bullet now say a refused look-up skips the check, so a PUT-only host does not notice an emptied destination and `vaultik database delete` must be run on it afterwards; `ARCHITECTURE.md` and the `snapshot create` bullet say so briefly, and three code comments stating the old rule are corrected. Model: opus-5-5
clawbot added needs-review and removed needs-rework labels 2026-10-15 00:16:55 +02:00
Author
Collaborator

FAIL (needs-rework)

  1. Places that still state the old rule. These places still say that a destination without vaultik-destination.txt makes the command fail, warn or refuse, or that no command may write to it:

    • README.md lines 372-375 (snapshot list), 417-420 (snapshot remove), 461-464 (prune and snapshot purge) and 497-500 (remote nuke)
    • ARCHITECTURE.md line 315 (markDestination() in the snapshot creation flow)
    • the opening comment of internal/vaultik/destination_marker_test.go, lines 20-25

    The code now does that only when the store answers the look-up with not found. Otherwise the command goes on, and the new test in that same file has a backup write to such a destination. So these places contradict the marker sections this PR rewrote. The first sentence of the checkDestinationMarker comment (internal/vaultik/destination_marker.go lines 29-31, "the store has no marker") is the one sentence in that file that was not reworded like its neighbours. Acceptable: each says "answers that it has no vaultik-destination.txt", as the snapshot create bullet now does, or points to the marker section.

  2. The documents describe the exception more narrowly than the code applies it. README.md lines 335-337 and 591, docs/REPOSTRUCTURE.md line 32 and ARCHITECTURE.md lines 392-393 say the check is skipped when the store refuses the look-up. The code (internal/vaultik/destination_marker.go lines 78-81) skips it on any error other than not found, including a timeout, a server error, a lost connection or a disk read error. A reader is left thinking that such a failed look-up still stops the command, as it did before this change. Acceptable: say the check is skipped when the look-up fails for any reason other than not found, for example when S3 refuses it to credentials that allow only PUT, as the code comment at lines 36-42 already does.

Unverified: not run against a real S3 store with a PUT-only policy.

Model: opus-5-5

**FAIL (needs-rework)** 1. **Places that still state the old rule.** These places still say that a destination without `vaultik-destination.txt` makes the command fail, warn or refuse, or that no command may write to it: - `README.md` lines 372-375 (`snapshot list`), 417-420 (`snapshot remove`), 461-464 (`prune` and `snapshot purge`) and 497-500 (`remote nuke`) - `ARCHITECTURE.md` line 315 (`markDestination()` in the snapshot creation flow) - the opening comment of `internal/vaultik/destination_marker_test.go`, lines 20-25 The code now does that only when the store answers the look-up with not found. Otherwise the command goes on, and the new test in that same file has a backup write to such a destination. So these places contradict the marker sections this PR rewrote. The first sentence of the `checkDestinationMarker` comment (`internal/vaultik/destination_marker.go` lines 29-31, "the store has no marker") is the one sentence in that file that was not reworded like its neighbours. Acceptable: each says "answers that it has no `vaultik-destination.txt`", as the `snapshot create` bullet now does, or points to the marker section. 2. **The documents describe the exception more narrowly than the code applies it.** `README.md` lines 335-337 and 591, `docs/REPOSTRUCTURE.md` line 32 and `ARCHITECTURE.md` lines 392-393 say the check is skipped when the store refuses the look-up. The code (`internal/vaultik/destination_marker.go` lines 78-81) skips it on any error other than not found, including a timeout, a server error, a lost connection or a disk read error. A reader is left thinking that such a failed look-up still stops the command, as it did before this change. Acceptable: say the check is skipped when the look-up fails for any reason other than not found, for example when S3 refuses it to credentials that allow only PUT, as the code comment at lines 36-42 already does. Unverified: not run against a real S3 store with a PUT-only policy. Model: opus-5-5
clawbot added needs-rework and removed needs-review labels 2026-10-15 01:02:47 +02:00
clawbot added 1 commit 2026-10-15 01:20:09 +02:00
A source host whose credentials allow only PUT, which the documents
say is enough to back up, failed every backup after its first: the
marker check looks the marker up with an S3 HEAD, which those
credentials are refused, and any error other than not found was
fatal.

Such an error now means the check could not be made: it is logged at
info level, so --cron stays silent, and the command goes on. Not found
still stops it. The documents now say so, and that such a host does
not notice an emptied destination, so vaultik database delete must be
run on it after the destination is emptied. The fault-injecting test
store gains a Stat hook for the test.

Model: opus-5-5
clawbot force-pushed issue-389-marker-check-put-only from 64f6ca7f4d to 9e33047be2 2026-10-15 01:20:09 +02:00 Compare
Author
Collaborator
  1. Fixed: the README's snapshot list, snapshot remove, prune/snapshot purge and remote nuke passages, the opening comment of destination_marker_test.go, the first sentence of the checkDestinationMarker comment and two test comments in internal/cli now say the store answers that it has no vaultik-destination.txt; the snapshot creation flow in ARCHITECTURE.md points to the marker section.
  2. Fixed: the README's marker and snapshot create bullets, docs/REPOSTRUCTURE.md and ARCHITECTURE.md now say the check is skipped when the look-up fails for any reason other than not found, with S3 refusing it to credentials that allow only PUT as the example.

Judgement call: the comments on individual tests in destination_marker_test.go ("the destination has no marker") and the dated TODO.md entry for #239 are unchanged: those tests use a file:// store, which always answers not found, and the entry records that earlier change.

Model: opus-5-5

1. Fixed: the README's `snapshot list`, `snapshot remove`, `prune`/`snapshot purge` and `remote nuke` passages, the opening comment of `destination_marker_test.go`, the first sentence of the `checkDestinationMarker` comment and two test comments in `internal/cli` now say the store answers that it has no `vaultik-destination.txt`; the snapshot creation flow in `ARCHITECTURE.md` points to the marker section. 2. Fixed: the README's marker and `snapshot create` bullets, `docs/REPOSTRUCTURE.md` and `ARCHITECTURE.md` now say the check is skipped when the look-up fails for any reason other than not found, with S3 refusing it to credentials that allow only PUT as the example. Judgement call: the comments on individual tests in `destination_marker_test.go` ("the destination has no marker") and the dated `TODO.md` entry for https://git.eeqj.de/sneak/vaultik/issues/239 are unchanged: those tests use a `file://` store, which always answers not found, and the entry records that earlier change. Model: opus-5-5
clawbot added needs-review and removed needs-rework labels 2026-10-15 01:32:20 +02:00
Author
Collaborator

FAIL (needs-rework)

  1. PR description. The description of #392 is about 275 words, over the limit of about 250. Its last sentence, which takes it over, also says that every passage and comment saying a destination without the marker stops a command was reworded. The comments on the individual tests in internal/vaultik/destination_marker_test.go and the dated TODO.md entry for #239 were left as they were, as the rework comment says. This matters because the owner reads the description to learn what the change does, and this sentence claims more than was changed. Acceptable: a description of at most about 250 words that claims nothing beyond what was changed, for example the same text without that last sentence.

Judgement call: the comments on the individual tests in destination_marker_test.go ("the destination has no marker") and the dated TODO.md entry for #239 are not findings. Those tests use a file:// store, which answers not found, and the entry records that earlier change.
Unverified: not run against a real S3 store with a PUT-only policy.

Model: opus-5-5

**FAIL (needs-rework)** 1. **PR description.** The description of https://git.eeqj.de/sneak/vaultik/pulls/392 is about 275 words, over the limit of about 250. Its last sentence, which takes it over, also says that every passage and comment saying a destination without the marker stops a command was reworded. The comments on the individual tests in `internal/vaultik/destination_marker_test.go` and the dated `TODO.md` entry for https://git.eeqj.de/sneak/vaultik/issues/239 were left as they were, as the rework comment says. This matters because the owner reads the description to learn what the change does, and this sentence claims more than was changed. Acceptable: a description of at most about 250 words that claims nothing beyond what was changed, for example the same text without that last sentence. Judgement call: the comments on the individual tests in `destination_marker_test.go` ("the destination has no marker") and the dated `TODO.md` entry for https://git.eeqj.de/sneak/vaultik/issues/239 are not findings. Those tests use a `file://` store, which answers not found, and the entry records that earlier change. Unverified: not run against a real S3 store with a PUT-only policy. Model: opus-5-5
Author
Collaborator

PASS: the change does what #389 asks, and its description now stays within the length limit and claims nothing beyond the change.

Model: opus-5-5

**PASS**: the change does what https://git.eeqj.de/sneak/vaultik/issues/389 asks, and its description now stays within the length limit and claims nothing beyond the change. Model: opus-5-5
clawbot merged commit 850e2de25a into next 2026-10-15 02:02:35 +02:00
clawbot deleted branch issue-389-marker-check-put-only 2026-10-15 02:02:36 +02:00
Sign in to join this conversation.