sneak/vaultik
1
0
Fork 0
Code Issues 19 Pull Requests 1 Actions Packages Projects Releases Wiki Activity

The unencrypted manifest publishes backup timestamps and a per-blob size profile #81

New Issue
Closed
opened 2026-08-09 04:57:02 +02:00 by clawbot · 2 comments
clawbot commented 2026-08-09 04:57:02 +02:00
Collaborator
Copy Link

Split out of #67, where I initially got the privacy posture wrong. This is
the underlying design question and it needs an owner decision.

What is published in the clear

Every snapshot writes metadata/<remote-key>/manifest.json.zst — zstd
compressed, not .age, so unencrypted at rest. Anyone who can list
or read the destination can read it. internal/snapshot/manifest.go:15-28:

field protected?
snapshot_id hashed (hex(SHA256(SHA256("vaultik|" + id))))
timestamp plaintext RFC3339
blob_count plaintext
total_compressed_size plaintext
blobs[].hash plaintext
blobs[].compressed_size plaintext

Hashing the snapshot ID successfully hides the hostname and
snapshot name. It does not hide the timestamp, because the timestamp
is written separately in the clear alongside it.

Why this is worth a decision

Taken together these give an observer, per snapshot: when it ran, how
many blobs it produced, the total size, and the individual size of every
blob. That is a timing-and-size profile. It supports inferences the
hashing was clearly meant to prevent — backup cadence, when a machine was
active or went quiet, roughly how much changed between runs, and
correlation of the same content across destinations via blob hashes.

There is also a false comment in the code, snapshot.go:842-844:

// Create manifest. SnapshotID in the unencrypted manifest is the
// double-SHA256 remote key, not the human ID, so the public bytes
// don't reveal hostname/snapshot-name/timestamp metadata.

The next statement writes Timestamp: time.Now().UTC().Format(time.RFC3339)
into those exact public bytes. remotekey.go's doc comment makes the same
overreaching claim. Whatever is decided here, both comments must stop
asserting something untrue — that is what misled me, and it will mislead
the next reader.

Options

  1. Encrypt the manifest (manifest.json.zst.age), leaving only the
    hashed directory name public. Strongest privacy. Cost: prune,
    verify, and remote listing currently read the manifest without a
    private key — check every consumer before choosing this, since it may
    force those operations to require the key, which cuts against the
    product's core premise. This interacts directly with #64.
  2. Trim the manifest to only what genuinely must be public, e.g. drop
    timestamp and per-blob sizes, keep blob hashes if prune needs
    them. Partial improvement, no key required.
  3. Accept and document it. Keep the format, and state plainly in the
    threat model exactly what an observer learns. Cheapest, and honest,
    but it permanently concedes the timing channel.

My recommendation: option 3 for 1.0, then revisit. Option 1 is the
right long-term answer but it risks requiring the private key for
listing/prune/verify, which would undermine the tool's central claim, and
that trade deserves its own design pass rather than being rushed into a
release. Option 2 is a half-measure that still leaks cadence via object
mtimes on most backends. Documenting the real posture is achievable now
and unblocks #67.

Note the timing channel is not fully closable by encrypting the manifest
anyway — object creation times and per-object sizes remain visible at the
storage layer on S3 and file:// alike. Any claim we make should account
for that.

Definition of done

  1. A decision recorded here, with reasoning.
  2. The false comments at snapshot.go:842-844 and in
    remotekey.go corrected regardless of which option is chosen.
  3. If option 1 or 2: implemented, with every existing manifest consumer
    (prune, verify, remote listing, snapshot rm) confirmed still
    working without a private key, or the loss of that property explicitly
    accepted and documented.
  4. docs/REPOSTRUCTURE.md threat model matches the outcome — this feeds
    #67, which is blocked on this decision.
  5. script/cibuild exits 0.

Assigning to sneak: this is a threat-model trade-off, not an
implementation detail.

Split out of #67, where I initially got the privacy posture wrong. This is the underlying design question and it needs an owner decision. ## What is published in the clear Every snapshot writes `metadata/<remote-key>/manifest.json.zst` — zstd compressed, **not** `.age`, so unencrypted at rest. Anyone who can list or read the destination can read it. `internal/snapshot/manifest.go:15-28`: | field | protected? | | --- | --- | | `snapshot_id` | hashed (`hex(SHA256(SHA256("vaultik\|" + id)))`) | | `timestamp` | **plaintext RFC3339** | | `blob_count` | plaintext | | `total_compressed_size` | plaintext | | `blobs[].hash` | plaintext | | `blobs[].compressed_size` | plaintext | Hashing the snapshot ID successfully hides the **hostname** and **snapshot name**. It does not hide the timestamp, because the timestamp is written separately in the clear alongside it. ## Why this is worth a decision Taken together these give an observer, per snapshot: when it ran, how many blobs it produced, the total size, and the individual size of every blob. That is a timing-and-size profile. It supports inferences the hashing was clearly meant to prevent — backup cadence, when a machine was active or went quiet, roughly how much changed between runs, and correlation of the same content across destinations via blob hashes. There is also a **false comment in the code**, `snapshot.go:842-844`: ```go // Create manifest. SnapshotID in the unencrypted manifest is the // double-SHA256 remote key, not the human ID, so the public bytes // don't reveal hostname/snapshot-name/timestamp metadata. ``` The next statement writes `Timestamp: time.Now().UTC().Format(time.RFC3339)` into those exact public bytes. `remotekey.go`'s doc comment makes the same overreaching claim. Whatever is decided here, both comments must stop asserting something untrue — that is what misled me, and it will mislead the next reader. ## Options 1. **Encrypt the manifest** (`manifest.json.zst.age`), leaving only the hashed directory name public. Strongest privacy. Cost: `prune`, `verify`, and remote listing currently read the manifest without a private key — check every consumer before choosing this, since it may force those operations to require the key, which cuts against the product's core premise. **This interacts directly with #64.** 2. **Trim the manifest** to only what genuinely must be public, e.g. drop `timestamp` and per-blob sizes, keep blob hashes if `prune` needs them. Partial improvement, no key required. 3. **Accept and document it.** Keep the format, and state plainly in the threat model exactly what an observer learns. Cheapest, and honest, but it permanently concedes the timing channel. **My recommendation: option 3 for 1.0, then revisit.** Option 1 is the right long-term answer but it risks requiring the private key for listing/prune/verify, which would undermine the tool's central claim, and that trade deserves its own design pass rather than being rushed into a release. Option 2 is a half-measure that still leaks cadence via object mtimes on most backends. Documenting the real posture is achievable now and unblocks #67. Note the timing channel is not fully closable by encrypting the manifest anyway — object creation times and per-object sizes remain visible at the storage layer on S3 and `file://` alike. Any claim we make should account for that. ## Definition of done 1. A decision recorded here, with reasoning. 2. The false comments at `snapshot.go:842-844` and in `remotekey.go` corrected regardless of which option is chosen. 3. If option 1 or 2: implemented, with every existing manifest consumer (`prune`, `verify`, remote listing, `snapshot rm`) confirmed still working without a private key, or the loss of that property explicitly accepted and documented. 4. `docs/REPOSTRUCTURE.md` threat model matches the outcome — this feeds #67, which is blocked on this decision. 5. `script/cibuild` exits 0. Assigning to `sneak`: this is a threat-model trade-off, not an implementation detail.
clawbot added this to the 1.0.0 milestone 2026-08-09 04:57:02 +02:00
sneak was assigned by clawbot 2026-08-09 04:57:06 +02:00
sneak commented 2026-08-09 16:03:48 +02:00
Owner
Copy Link

3 is the obvious previously made decision. don’t second guess my design decisions already implemented.

3 is the obvious previously made decision. don’t second guess my design decisions already implemented.
clawbot commented 2026-08-09 16:10:13 +02:00
Author
Collaborator
Copy Link

Closing per sneak's ruling: option 3. The unencrypted manifest's
backup timestamps and per-blob size profile are an accepted, previously
made design decision — not a defect, and not open for revision here.

His words: "3 is the obvious previously made decision. don't second
guess my design decisions already implemented."

I was wrong to file this as an issue. Filing implies a defect and a
remediation obligation, and applying that framing to a deliberate,
already-implemented choice misrepresents it and spends the owner's
attention re-explaining a decision he had already made. The observation
was worth raising; the tracker was the wrong place for it. Chat would
have been correct.

Taking the standing rule from this: do not file the owner's
already-implemented design decisions as defects. Genuine defects still
get filed, and so do choices that are merely undocumented — the
distinction is whether the code reflects an intentional decision or an
oversight.

What carries forward, and where

One item from this issue is a genuine defect and survives its closure —
the code comment at internal/snapshot/snapshot.go:842-844:

// Create manifest. SnapshotID in the unencrypted manifest is the
// double-SHA256 remote key, not the human ID, so the public bytes
// don't reveal hostname/snapshot-name/timestamp metadata.

The next statement writes Timestamp: time.Now().UTC().Format(time.RFC3339)
into exactly those public bytes. That comment is false regardless of which
option was chosen, and remotekey.go's doc comment overreaches the same
way. A comment asserting the opposite of the adjacent code is an
oversight, not a design decision — it is what misled me into filing this
in the first place, and it will mislead the next reader.

Moved to #67, which is now unblocked and rescoped: the threat model
should describe the manifest's exposure as intended behaviour, stated
plainly so users can evaluate it, rather than as something to be fixed.
docs/REPOSTRUCTURE.md's existing claim that backup times are observable
was correct all along — it just attributed them to the snapshot ID rather
than to the manifest.

Closing per `sneak`'s ruling: **option 3**. The unencrypted manifest's backup timestamps and per-blob size profile are an accepted, previously made design decision — not a defect, and not open for revision here. His words: *"3 is the obvious previously made decision. don't second guess my design decisions already implemented."* I was wrong to file this as an issue. Filing implies a defect and a remediation obligation, and applying that framing to a deliberate, already-implemented choice misrepresents it and spends the owner's attention re-explaining a decision he had already made. The observation was worth raising; the tracker was the wrong place for it. Chat would have been correct. Taking the standing rule from this: **do not file the owner's already-implemented design decisions as defects.** Genuine defects still get filed, and so do choices that are merely undocumented — the distinction is whether the code reflects an intentional decision or an oversight. ## What carries forward, and where One item from this issue is a genuine defect and survives its closure — the code comment at `internal/snapshot/snapshot.go:842-844`: ```go // Create manifest. SnapshotID in the unencrypted manifest is the // double-SHA256 remote key, not the human ID, so the public bytes // don't reveal hostname/snapshot-name/timestamp metadata. ``` The next statement writes `Timestamp: time.Now().UTC().Format(time.RFC3339)` into exactly those public bytes. That comment is false regardless of which option was chosen, and `remotekey.go`'s doc comment overreaches the same way. A comment asserting the opposite of the adjacent code is an oversight, not a design decision — it is what misled me into filing this in the first place, and it will mislead the next reader. Moved to **#67**, which is now unblocked and rescoped: the threat model should describe the manifest's exposure as **intended behaviour**, stated plainly so users can evaluate it, rather than as something to be fixed. `docs/REPOSTRUCTURE.md`'s existing claim that backup times are observable was correct all along — it just attributed them to the snapshot ID rather than to the manifest.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
merge-ready
All checks pass, reviewed, rebased, ready for merge
 needs-checks
make check does not pass cleanly
 needs-rebase
PR has merge conflicts or is behind main
 needs-review
Code review not yet done
 needs-rework
Code review found issues to fix
No Label
Milestone
No items
No Milestone 1.0.0
Projects
Clear projects
No project
Assignees
Clear assignees
clawbot sneak
No Assignees sneak
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sneak/vaultik#81
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?