Correct the security claims in docs and comments, and record the accepted risks (closes #171)
check / check (pull_request) Successful in 1m58s
check / check (push) Successful in 2m49s

Docs and comments only; no behaviour change. Corrects ten overclaims the security review found: snapshot names are hashed but the hash uses no secret, so a guessed hostname and name can be confirmed; a blob is named by hex(SHA256(SHA256(uncompressed contents))), stated once in docs/REPOSTRUCTURE.md and referenced elsewhere; double hashing does not hide known content (blob packing does); age uses ChaCha20-Poly1305, not XChaCha20; encryption is required, not optional; a snapshot is marked complete before its metadata is uploaded; the export comment now matches its only caller; deep verify detects corruption, not authorship; adding a recipient does not reach existing data; restore examples target a user-owned directory.

Adds an Accepted Risks subsection under Security Considerations with the seven documented risks, cross-referenced from the README.

Model: opus-4-8
This commit was merged in pull request #199.
This commit is contained in:
2026-09-22 19:28:31 +02:00
parent c3bec7d3aa
commit 1548c0f933
14 changed files with 108 additions and 67 deletions
+9 -7
View File
@@ -1,14 +1,16 @@
// Package blob handles the creation of blobs - the final storage units for Vaultik.
// A blob is a large file (up to 10GB) containing many compressed and encrypted chunks
// from multiple source files. Blobs are content-addressed, meaning their filename
// is derived from the SHA256 hash of their compressed and encrypted content.
// from multiple source files. Blobs are content-addressed: a blob's filename is
// hex(SHA256(SHA256(uncompressed blob contents))), computed from the concatenated
// chunk data before compression and encryption, not from the stored bytes. See
// blobgen.DoubleSHA256 and docs/REPOSTRUCTURE.md.
//
// The blob creation process:
// 1. Chunks are accumulated from multiple files
// 2. The collection is compressed using zstd
// 3. The compressed data is encrypted using age
// 4. The encrypted blob is hashed to create its content-addressed name
// 5. The blob is uploaded to S3 using the hash as the filename
// 1. Chunks are accumulated from multiple files
// 2. Each chunk's uncompressed bytes are fed to a running SHA-256 and, in the same
// pass, compressed with zstd and encrypted with age into the temp file
// 3. On finalize, the name is the double SHA-256 of that uncompressed content
// 4. The blob is uploaded to S3 using the name as the filename
//
// This design optimizes storage efficiency by batching many small chunks into
// larger blobs, reducing the number of S3 operations and associated costs.
+13 -7
View File
@@ -16,11 +16,17 @@ import (
)
// DoubleSHA256 returns the double SHA-256 of content whose single SHA-256
// digest is sum: it hashes that digest once more. Stored objects are named by
// this second hash so that a name never reveals whether known content is
// present — an attacker who knows a plaintext, and thus its SHA-256, still
// cannot derive the stored name without hashing the digest again. Both a blob
// and the metadata database export are named this way.
// digest is sum: it hashes that digest once more. Stored objects — a blob, and
// the metadata database export — are named by this second hash.
//
// The second hash does not hide whether known content is stored: an attacker
// who can reproduce an object's entire plaintext computes the same name simply
// by hashing twice, exactly as this code does. What limits that is blob
// packing, not the double hash — a blob's name covers all of its concatenated
// chunk plaintext, so a name can be confirmed only by someone who can
// reproduce the whole blob (a snapshot made entirely of known content, or a
// known file large enough to fill blobs on its own). An ordinary file that
// shares a blob with other, unknown data cannot be confirmed this way.
func DoubleSHA256(sum []byte) []byte {
h := sha256.Sum256(sum)
@@ -147,8 +153,8 @@ func (w *Writer) Close() error {
// ContentID returns the double SHA-256 of the uncompressed input data: the
// name under which this content is stored. It is the second hash of the
// running SHA-256, via DoubleSHA256; see that function for why content is
// named this way rather than by its plain SHA-256.
// running SHA-256, via DoubleSHA256; see that function for what naming content
// this way does and does not hide.
func (w *Writer) ContentID() []byte {
return DoubleSHA256(w.hasher.Sum(nil))
}
+5 -2
View File
@@ -46,8 +46,11 @@ const defaultConfigTemplate = `# vaultik configuration
# ─── REQUIRED ────────────────────────────────────────────────────────────────
# Age recipient public keys for encryption.
# Backups are encrypted to ALL listed recipients. Any one of the corresponding
# private keys can decrypt. Generate a keypair with:
# Backups are encrypted to ALL listed recipients; any one of the corresponding
# private keys can decrypt. Adding a recipient later does not re-encrypt data
# already stored: deduplicated chunks and existing blobs stay encrypted to the
# earlier recipients, so a newly added key cannot restore them on its own (see
# docs/REPOSTRUCTURE.md, Accepted Risks). Generate a keypair with:
# age-keygen -o vaultik_backup_private_key.txt
# grep 'public key' vaultik_backup_private_key.txt
age_recipients:
+2 -1
View File
@@ -192,7 +192,8 @@ func newSnapshotVerifyCommand() *cobra.Command {
Long: "Checks that every blob the snapshot's manifest lists is present\n" +
"in storage with the size the manifest records, and that the\n" +
"snapshot's encrypted database is present. It does not read blob\n" +
"contents; use --deep to download and cryptographically verify them.\n\n" +
"contents; use --deep to download, decrypt, and re-hash every blob\n" +
"to detect corruption -- integrity, not who wrote it.\n\n" +
"The snapshot may be named by its ID or, on a host with no local\n" +
"index, by the remote key that 'snapshot list' prints for a\n" +
"remote-only snapshot (an unambiguous leading part is enough).",
+4 -2
View File
@@ -3,8 +3,10 @@
//
// Blobs in Vaultik are the final storage units uploaded to S3. Each blob is a
// large (up to 10GB) file containing many compressed and encrypted chunks from
// multiple source files. Blobs are content-addressed, meaning their filename
// is derived from their SHA256 hash after compression and encryption.
// multiple source files. Blobs are content-addressed: the filename in S3 is
// hex(SHA256(SHA256(uncompressed blob contents))), computed from the chunk data
// before compression and encryption (not from the stored bytes). See
// blobgen.DoubleSHA256 and docs/REPOSTRUCTURE.md.
//
// Schema is managed via numbered SQL migrations embedded in the schema/
// directory. Migration 000.sql bootstraps the schema_migrations tracking
+6 -6
View File
@@ -51,15 +51,15 @@ type Chunk struct {
// Blob represents a blob record in the database.
// A blob is Vaultik's final storage unit - a large file (up to 10GB) containing
// many compressed and encrypted chunks from multiple source files.
// Blobs are content-addressed, meaning their filename in S3 is derived from
// the SHA256 hash of their compressed and encrypted content.
// The blob creation process is: chunks are accumulated -> compressed with zstd
// -> encrypted with age -> hashed -> uploaded to S3 with the hash as filename.
// Blobs are content-addressed: the filename in S3 is
// hex(SHA256(SHA256(uncompressed blob contents))), computed from the chunk data
// before compression and encryption (not from the stored bytes). See
// blobgen.DoubleSHA256 and docs/REPOSTRUCTURE.md.
type Blob struct {
ID types.BlobID // UUID assigned when blob creation starts
// Hash is the SHA256 of the final compressed+encrypted content
// (empty until finalized).
// Hash is hex(SHA256(SHA256(uncompressed blob contents)))
// (empty until finalized); see the type comment above.
Hash types.BlobHash
CreatedTS time.Time // When blob creation started
FinishedTS *time.Time // When blob was finalized (nil if still packing)
+1 -1
View File
@@ -119,7 +119,7 @@ type ScannerConfig struct {
Storage storage.Storer
MaxBlobSize int64
CompressionLevel int
AgeRecipients []string // Optional, empty means no encryption
AgeRecipients []string // required; output is always encrypted
EnableProgress bool // Enable the live progress reporter (ETAs, throughput)
UI *ui.Writer // Where user-facing scanner messages go; nil = discard
Exclude []string // Glob patterns for files/directories to exclude
+15 -14
View File
@@ -24,7 +24,7 @@ package snapshot
// 7. Close the temporary database
// 8. VACUUM the database to remove deleted data and compact (security critical)
// 9. Compress the binary database with zstd
// 10. Encrypt the compressed database with age (if encryption is enabled)
// 10. Encrypt the compressed database with age (always; recipients are required)
// 11. Upload to S3 as: metadata/{snapshot-id}/db.zst.age
// 12. Reopen the main database
//
@@ -238,14 +238,12 @@ func (sm *SnapshotManager) CompleteSnapshot(
// 3. Cleans the copy to contain only current snapshot data
// 4. Dumps the cleaned database to SQL
// 5. Compresses the SQL dump with zstd
// 6. Encrypts the compressed data (if encryption is enabled)
// 6. Encrypts the compressed data with age (always; recipients are required)
// 7. Uploads to S3 at: snapshots/{snapshot-id}.sql.zst[.age]
//
// The caller is responsible for:
// - Ensuring the main database is closed before calling this method
// - Reopening the main database after this method returns
//
// This ensures database consistency during the copy operation.
// The only caller (finalizeSnapshotMetadata) does not close the main database
// before calling this method: the index is copied at dbPath while it is still
// open, and every step here operates on that copy, never on the live index.
func (sm *SnapshotManager) ExportSnapshotMetadata(
ctx context.Context, dbPath string, snapshotID string,
) error {
@@ -415,9 +413,11 @@ func (sm *SnapshotManager) prepareExportDB(
// uploadSnapshotArtifacts uploads the database backup and blob manifest
// to remote storage at metadata/<remote-key>/, where remote-key is the
// double-SHA256 derivation of the snapshot ID (see RemoteSnapshotKey).
// We never write the human-readable snapshot ID into any unencrypted
// part of remote storage so a listing of the destination bucket leaks
// no host, configuration, or scheduling information.
// The human-readable snapshot ID is never written into an unencrypted part
// of remote storage, so a plain listing shows only the hashed key, not the
// hostname or snapshot name. The hash uses no secret, so a guessed hostname
// and snapshot name can still be confirmed against a listing, and the backup
// time is public: the manifest carries a plaintext timestamp.
func (sm *SnapshotManager) uploadSnapshotArtifacts(
ctx context.Context, snapshotID string, dbData, manifestData []byte,
) error {
@@ -814,10 +814,11 @@ func (sm *SnapshotManager) generateBlobManifest(
}
// Create manifest. SnapshotID in the unencrypted manifest is the
// double-SHA256 remote key (see RemoteSnapshotKey), not the human ID,
// so neither this field nor the directory name reveals the hostname or
// snapshot name. Timestamp below is written in the clear, so the backup
// time is observable to anyone who can read the manifest.
// double-SHA256 remote key (see RemoteSnapshotKey), not the human ID, so
// neither this field nor the directory name spells out the hostname or
// snapshot name — but the key uses no secret, so a guessed hostname and
// snapshot name can be confirmed. Timestamp below is written in the clear,
// so the backup time is observable to anyone who can read the manifest.
manifest := &Manifest{
SnapshotID: RemoteSnapshotKey(snapshotID),
Timestamp: time.Now().UTC().Format(time.RFC3339),
+4 -2
View File
@@ -146,8 +146,10 @@ type SnapshotID string
// Used for content-addressing and deduplication of file chunks.
type ChunkHash string
// BlobHash is the SHA256 hash of a blob's compressed and encrypted content.
// This is used as the filename in S3 storage for content-addressed retrieval.
// BlobHash is hex(SHA256(SHA256(uncompressed blob contents))), computed before
// compression and encryption (see blobgen.DoubleSHA256 and
// docs/REPOSTRUCTURE.md). It is used as the filename in S3 storage for
// content-addressed retrieval.
type BlobHash string
// FilePath represents an absolute path to a file or directory.