snapshot list must list remote snapshots without requiring the private key #64
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
vaultik snapshot listnever reads the destination store on a correctlyconfigured host, contradicting both the README and vaultik's core threat
model.
Evidence
ListSnapshots(internal/vaultik/snapshot.go:471) builds its tableentirely from the local SQLite index via
v.Repositories.Snapshots.ListRecent(line 474). Every row goes throughsnapshotInfoFromLocal(snapshot.go:574-603), which hardcodesLocallyTracked: true(line 602). The only remote access isreportRemoteDrift, and it is gated:The whole premise of vaultik (README:3-7) is that the backed-up host has
no private key. So on a properly configured production host,
AgeSecretKeyis empty andsnapshot listnever contacts thedestination at all.
This contradicts:
… The uncompressed and 'new chunk' columns show
<remote only>forsnapshots not in the local index."
snapshot.go:468-470) — "If remotelisting fails (unmounted volume, permission denied, network), we
degrade to local-only with a warning. List never fails just because the
destination is unreachable." This describes behavior that does not
exist.
Consequence:
const remoteOnlyCell = "<remote only>"(snapshot.go:663)and its consumer at line 668 are unreachable dead code, and a user
who has lost their local index cannot see their own backups.
Secondary bug in the same function:
snapshot.go:562printsThere is no
snapshot cleanupsubcommand.NewSnapshotCommand(
internal/cli/snapshot.go:50-55) registers onlycreate,list,purge,verify,remove,restore. The correct advice perREADME:223-232 is
vaultik prune. Note thatCleanupLocalSnapshotsexists on
*Vaultikbut is wired to no command — determine whether thecommand was removed and the message orphaned, or the command was never
added.
Definition of done
snapshot listenumerates snapshots from the destination store,merged with the local index, and does so without requiring
age_secret_key. Listing must work on a host holding only thepublic key.
<remote only>in the uncompressed and new-chunk columns, makingremoteOnlyCellreachable. Snapshots present locally but not remotelyare still surfaced as drift.
degrades to local-only with a warning and a zero exit code, exactly
as the existing doc comment promises.
snapshot listmust not failbecause the destination is unreachable.
--jsonoutput reflects the same merged view, with an explicit fielddistinguishing locally-tracked from remote-only entries.
vaultik snapshot cleanupstring is gone: either wire up a realcommand backed by
CleanupLocalSnapshots, or change the message toname
vaultik prune. No user-facing message may name a nonexistentcommand. Decide and state which was chosen.
<remote only>rendering; a case with an unreachable destination asserting graceful
local-only degradation and exit 0; a case asserting remote listing
happens with
age_secret_keyunset.behavior.
make checkgreen.Manager note — implementation guidance, plus one constraint that was not
obvious when I filed this.
The naming constraint
I checked whether a remote-only snapshot can even be named without the
private key. It cannot, fully — but it can be described usefully.
RemoteSnapshotKey(internal/snapshot/remotekey.go:35) ishex(SHA256(SHA256("vaultik|" + id))), one-way, and the manifest'ssnapshot_idfield stores that hash, not the human ID. The human ID(
<hostname>_<name>_<timestamp>) exists only in the local index and inthe encrypted
db.zst.age. So for a snapshot absent from the localindex, hostname and snapshot name are not recoverable on a host
holding only the public key. Do not attempt to recover them, and do not
add anything to remote storage to make them recoverable — that would
undo a deliberate privacy property (see #81).
What is available without the private key, from the unencrypted
metadata/<remote-key>/manifest.json.zst(
internal/snapshot/manifest.go:15-28):timestamp,blob_count,total_compressed_size, and per-blob hashes and sizes.That is enough to make this feature genuinely useful, and it maps cleanly
onto the README's existing promise: a remote-only row can show a real
timestamp and a real compressed size, with
<remote only>in exactly thecolumns that require the local index (uncompressed size and new-chunk
count). Identify such rows by their remote key — truncated for display,
with the full key available in
--json— and make it visually obviousthat the human name is unavailable rather than blank or fabricated.
So this issue is implementable as scoped and is not blocked on #81.
Ordering
#81 asks whether to encrypt the manifest. If that were answered "encrypt",
this feature would need the private key and the design would change
materially. I recommended keeping the manifest readable partly for this
reason. Whoever implements this should read the manifest through a single
helper so that a future change to #81 has one call site to update, not
several.
Additional requirements beyond the definition of done
snapshot listmust workwith
age_secret_keyunset — that is the whole point of this issue.Add a test that fails if the code ever requires it again.
thousands of snapshots must not be read entirely into memory or make
one request per snapshot if the backend can list a prefix.
fix/sync-snapshot-cleanup(see #71) — that stalebranch touches
syncWithRemote, the same path. Read its diff beforestarting; if it is still correct, fold it in rather than colliding.
reportRemoteDriftcurrently exists only to warn about mismatches.Once listing is merged, decide explicitly whether it still has a
distinct job or whether it collapses into the merged view, and say
which in the PR.
remoteOnlyCell(snapshot.go:663) and itsLocallyTracked == falsebranch become reachable. Confirm they render correctly rather than
assuming.
Implementation plan for this issue, on branch
fix-snapshot-list-remote.Findings before writing code
Stale branch
fix/sync-snapshot-cleanup(tip332ea26). Its onlychange is
v.Repositories.Snapshots.Delete(...)→v.deleteSnapshotFromLocalDB(...)insidesyncWithRemote. That changeis already present on
main(internal/vaultik/snapshot.go:1186),having landed independently through the
deleteSnapshotFromLocalDBerror-propagation work. There is nothing to fold in and nothing to
collide with; the branch is redundant and can be deleted under #71.
CleanupLocalSnapshotsis not orphaned. It is wired tovaultik pruneatinternal/vaultik/prune.go:82, and the comment therestates explicitly that
snapshot cleanupwas folded intoprune. Sothe
snapshot cleanupstring is a leftover from a removed command, nota missing one.
Design
Merged listing.
ListSnapshotsbuilds one merged set:SnapshotInfo{LocallyTracked: true}as today, plus the derived
RemoteSnapshotKey.metadata/via the existing streaminglistAllRemoteSnapshotKeys— a single prefix request, not oneper snapshot.
metadata/<key>/manifest.json.zstand emitSnapshotInfo{LocallyTracked: false}carrying the manifesttimestampandtotal_compressed_size.None of this touches
AgeSecretKey; the manifest is unencrypted, solisting works on a host holding only the public key. The
AgeSecretKey == ""early return is deleted.Naming constraint honored. No attempt to recover hostname or
snapshot name for remote-only rows, and nothing new written to remote
storage. The identifier column shows
<remote only:<first 12 hex of key>>, which is visibly not a humanID.
--jsoncarries the full 64-char key in a newremote_keyfield.<remote only>fills the uncompressed and new-chunk columns, exactlyas README:178-182 already promises.
Bounding (comment req 2). The listing is streamed, one prefix
request. Manifests are fetched only for keys absent from the local
index, with bounded concurrency and a hard cap on the number of
remote-only rows; past the cap the table is truncated with a warning
rather than growing unbounded.
Single manifest reader (comment "Ordering"). All manifest reads go
through the existing
downloadManifestByKey.internal/vaultik/verify.goand
internal/vaultik/info.gostill open and decode manifests inline;both get routed through that helper so #81 has exactly one call site to
change.
reportRemoteDrift(comment req 4): collapses. Its remote-only halfis fully subsumed by the merged table, which now shows the actual rows
instead of a bare count. Its local-only half stays meaningful but no
longer needs its own remote listing — it becomes a small reporter fed
from the already-computed merge, so
snapshot listmakes exactly oneremote listing per invocation.
Degradation (DoD 3). Remote listing failure is a warning plus
local-only output and a zero exit code. In
--jsonmode the warninggoes to the log rather than stdout so the JSON stays parseable.
snapshot cleanupstring (DoD 5): renamed tovaultik prune. Notwiring a new command:
prunealready runsCleanupLocalSnapshotsasits first pass, and adding
snapshot cleanupback would re-create theduplicate entry point that the 2026-07-02 CLI consolidation removed.
Tests
<remote only>in bothlocal-index-dependent columns, with the truncated key in the
identifier column and the real manifest timestamp/size.
age_secret_keyunset — asserts theremote store was actually read and the remote-only row is present.
This is the regression guard for the whole issue.
ListSnapshotsreturns nil.vaultik prune.--jsonmerged view:locally_trackedand fullremote_key.README:178-182, the
ListSnapshotsdoc comment, andTODO.mdupdatedin the same commit. Gate is
script/cibuild.Closed by PR #83, merged to
mainas50e20b4.Verified against the merged
mainwith the test cache explicitlydefeated, not trusted from the PR:
0 issues.from the pinned linter, 14 packagesok, 0(cached), 0FAIL.snapshot listnow enumerates the destination store merged with thelocal index, with no
age_secret_keygate — the property this issueexisted for. A regression test asserts the key is empty, counts prefix
listings, and asserts no
.agekey is ever read, so reintroducing thegate fails the build on two independent assertions.
Resolutions for the open questions:
snapshot cleanup: the command was removed and the messageorphaned, not never-added.
internal/vaultik/prune.go:80-82callsCleanupLocalSnapshotsand its comment says so — it isprune's firstpass, not dead code. Re-adding the command would restore the duplicate
entry point the 2026-07-02 consolidation deliberately removed, so the
hint now names
vaultik prune.reportRemoteDriftcollapsed into the merged view asreportListDrift. Net effect: the destination is listed exactly onceper invocation.
unrecoverable without the private key, so those rows are identified by
abbreviated remote key with the real timestamp and compressed size from
the manifest. Nothing new is written remotely and the ID is never
fabricated — the privacy property in #81 is untouched.
Two bugs were found and fixed during review that were not in the
original scope:
scanSnapshotRowsomitted.UTC()where its twosiblings had it, so on a non-UTC host the same snapshot rendered a
different time depending on whether it was locally tracked — in the
same column, with no indication why. Fixed at the scanner, the single
point where the value enters the domain, and the duplicate loop in
GetIncompleteByHostnamefolded onto it so three readers now share onenormalization point. The regression test pins
time.Localto +07:13,since every existing fixture was UTC and would not have caught it.
--jsontruncation. The 1000-row cap and unreadable-manifestcount were reported in table mode but not JSON, so a machine consumer
past 1000 remote-only snapshots got a truncated document with no
signal. Now reported on stderr in both modes; the JSON shape is
unchanged.
Two follow-ups filed rather than fixed drive-by: #82 (the logger writes to
stdout, so warnings corrupt
--json— this PR carries a local workaroundto be removed when that lands) and #84 (a
Vaultik.UIdoc commentmisdescribing
--cron).