Compare commits
1
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
6a57750a7b |
@@ -99,8 +99,10 @@ vaultik snapshot restore <snapshot-id> ~/vaultik-restore
|
||||
On a destination shared between hosts or users, the prune in that cron line
|
||||
relies on `prune_grace_period` (default `24h`) to leave the blobs of a backup
|
||||
another one is still running, so it must be longer than the longest backup
|
||||
run; otherwise give each host its own prefix or directory. See
|
||||
[locking](#locking).
|
||||
run. It does not protect a blob that backup reuses from an earlier run that
|
||||
stopped before its manifest: the prune can delete such a blob, and the new
|
||||
snapshot then cannot be restored. To not depend on the grace period, give each
|
||||
host its own prefix or directory. See [locking](#locking).
|
||||
|
||||
## restoring on another machine
|
||||
|
||||
@@ -206,9 +208,14 @@ time. A backup uploads its blobs before the manifest that lists them, so a
|
||||
prune from elsewhere at that moment finds blobs no snapshot lists. It
|
||||
deletes such a blob only once it is older than `prune_grace_period`
|
||||
(default `24h`), which therefore must be longer than the longest backup run
|
||||
of any host or user sharing the destination. `remote nuke` deletes
|
||||
everything regardless. To not depend on the grace period, give each host
|
||||
or user its own prefix (or directory) on the destination.
|
||||
of any host or user sharing the destination. The grace period does not
|
||||
protect a blob a backup reuses from an earlier run that stopped before its
|
||||
manifest: a backup skips the upload of a blob already on the destination, so
|
||||
that blob keeps the time of its first upload, and a prune from elsewhere
|
||||
during the backup can delete it; the new snapshot then cannot be restored.
|
||||
`remote nuke` deletes everything regardless. To not depend on the grace
|
||||
period, give each host or user its own prefix (or directory) on the
|
||||
destination.
|
||||
|
||||
Read-only commands — `info`, `snapshot list`, `snapshot verify`, and
|
||||
`remote info` — do not take the lock and are never blocked, so they run
|
||||
@@ -443,8 +450,11 @@ manifest on the destination store to compute the still-referenced blob
|
||||
set and delete any blob not in that set that is older than
|
||||
`prune_grace_period` (default `24h`). A newer one is kept and counted: it
|
||||
may belong to a backup still running, on this host or another sharing the
|
||||
destination store, that has not yet uploaded its manifest. Step (3) reads
|
||||
all remote manifests — network cost scales with the number of snapshots.
|
||||
destination store, that has not yet uploaded its manifest. The grace period
|
||||
does not protect a blob such a backup reuses from an earlier run that stopped
|
||||
before its manifest: that blob keeps the time of its first upload, so it can
|
||||
be deleted. Step (3) reads all remote manifests — network cost scales with
|
||||
the number of snapshots.
|
||||
`snapshot create --prune` runs the same cleanup automatically; this is the
|
||||
manual entry point for the same work. If the destination store cannot
|
||||
be listed, or the local index holds snapshots and the destination has
|
||||
@@ -493,8 +503,8 @@ deletes what is left.
|
||||
|
||||
**`database delete`**: Delete the local SQLite state database file
|
||||
entirely. Remote storage is unaffected; the next backup will do a full
|
||||
scan and upload all the data again, and the local index will re-bind to
|
||||
the currently configured storage destination.
|
||||
scan and re-deduplicate against existing remote blobs, and the local
|
||||
index will re-bind to the currently configured storage destination.
|
||||
Use this after changing `storage_url`, or `s3.endpoint`, `s3.bucket` or
|
||||
`s3.prefix`, to a different destination.
|
||||
* `--force`: Skip confirmation prompt
|
||||
@@ -536,7 +546,9 @@ and can be deleted.
|
||||
A destination shared between hosts or users, each with its own local index,
|
||||
relies on `prune_grace_period` to keep one's prune from deleting the blobs of
|
||||
another's backup still running, so it must be longer than the longest backup
|
||||
run; otherwise give each host its own prefix (or directory). See
|
||||
run. It does not protect a blob that backup reuses from an earlier run that
|
||||
stopped before its manifest, which the prune can delete. To not depend on the
|
||||
grace period, give each host its own prefix (or directory). See
|
||||
[locking](#locking).
|
||||
|
||||
Legacy S3 configuration via `s3.*` fields (endpoint, bucket, prefix, etc.) is
|
||||
@@ -671,7 +683,7 @@ complete annotated example also lives in
|
||||
| `chunk_size` | `10MB` | Average chunk size for content-defined chunking |
|
||||
| `blob_size_limit` | `10GB` | Maximum blob size before splitting. Must be at least four times `chunk_size` (the largest chunk the chunker can emit), otherwise a single-chunk blob could exceed the limit. A backup needs free temporary space, because each blob is written whole to a temporary file before it is uploaded (up to about `blob_size_limit`; an rclone destination that cannot stream uploads needs about twice that) and the metadata export writes copies of the local index. Temporary files go to `$TMPDIR` (default `/tmp`); with `TMPDIR` unset, SQLite writes one of those copies to `/var/tmp` |
|
||||
| `compression_level` | `3` | zstd compression level (1-19) |
|
||||
| `prune_grace_period` | `24h` | How old a blob that no snapshot lists must be before `prune` deletes it. A backup uploads its blobs before the manifest that lists them, so on a destination shared between hosts or users this must be longer than the longest backup run (see [locking](#locking)). A duration such as `36h` or `90m`; `0s` deletes such blobs at once |
|
||||
| `prune_grace_period` | `24h` | How old a blob that no snapshot lists must be before `prune` deletes it. A backup uploads its blobs before the manifest that lists them, so on a destination shared between hosts or users this must be longer than the longest backup run (see [locking](#locking)). It does not protect a blob a backup reuses from an earlier run that stopped before its manifest: that blob keeps the time of its first upload, so a prune can delete it. A duration such as `36h` or `90m`; `0s` deletes such blobs at once |
|
||||
| `hostname` | system hostname | Hostname used in snapshot IDs |
|
||||
| `index_path` | platform data dir | Local SQLite index path |
|
||||
|
||||
|
||||
@@ -333,6 +333,9 @@ compression_level: 5
|
||||
# A backup uploads its blobs before the manifest that lists them, so on a
|
||||
# destination shared between hosts or users this must be longer than the
|
||||
# longest backup run
|
||||
# It does not protect a blob a backup reuses from an earlier run that stopped
|
||||
# before its manifest: that blob keeps the time of its first upload, so a
|
||||
# prune can delete it
|
||||
# Supports: 36h, 90m, etc.; 0s deletes them at once
|
||||
# Default: 24h
|
||||
#prune_grace_period: 24h
|
||||
|
||||
+2
-2
@@ -90,7 +90,7 @@ Stores information about packed, compressed, and encrypted blob files.
|
||||
- `created_ts` (INTEGER NOT NULL) - Creation timestamp
|
||||
- `finished_ts` (INTEGER) - Finalization timestamp (NULL if in progress)
|
||||
- `uncompressed_size` (INTEGER NOT NULL DEFAULT 0) - Total size of chunks before compression
|
||||
- `compressed_size` (INTEGER NOT NULL DEFAULT 0) - Size after compression and encryption. Once uploaded, the size of the object in storage
|
||||
- `compressed_size` (INTEGER NOT NULL DEFAULT 0) - Size after compression and encryption. Once uploaded, the size of the object in storage: when the blob was already stored, under another compression level or other recipients, this can differ from the size of the local copy
|
||||
- `uploaded_ts` (INTEGER) - Upload completion timestamp (NULL if not uploaded)
|
||||
|
||||
**Purpose:** Tracks blob lifecycle from creation through upload. The UUID primary key allows immediate association of chunks with blobs.
|
||||
@@ -185,7 +185,7 @@ Tracks blob upload metrics.
|
||||
- `UPDATE blobs SET blob_hash = ?, finished_ts = ?` - Finalize blob after packing
|
||||
|
||||
4. **Upload**
|
||||
- `UPDATE blobs SET uploaded_ts = ?, compressed_size = ?` - Mark blob as uploaded, with the size of the object in storage (a blob already stored is uploaded again, so that a prune does not take it for an unreferenced blob older than `prune_grace_period`)
|
||||
- `UPDATE blobs SET uploaded_ts = ?, compressed_size = ?` - Mark blob as uploaded, with the size of the object in storage (the upload is skipped when the blob is already stored)
|
||||
- `INSERT INTO uploads` - Record upload metrics with snapshot_id
|
||||
- `INSERT INTO snapshot_blobs` - Associate blob with snapshot
|
||||
|
||||
|
||||
@@ -150,7 +150,7 @@ These are known, deliberate properties of the format and the tooling, recorded s
|
||||
3. **Chunking uses no secret.** The FastCDC parameters are fixed and public. The default 10 MB average yields chunks between 2.5 MB and 40 MB, and any file of 2.5 MB or less is a single chunk. At the default 10 GB `blob_size_limit` a blob holds hundreds of chunks, so individual chunk lengths are not visible in the blob's size; lowering the limit toward the chunk size begins to expose them.
|
||||
4. **Decrypted data on local disk.** Several commands stage plaintext under `$TMPDIR`: `snapshot restore` writes decrypted blobs under `vaultik-blobcache-*/` (no size cap) and the decrypted metadata database at `vaultik-restore-*/snapshot.db`; `verify --deep` writes that database at `vaultik-verify-*/snapshot.db`; `snapshot create` keeps a plaintext copy of the index at `vaultik-snapshot-*/snapshot.db`. These files are created `0600` and removed on success, but a `kill -9` or a power loss leaves them behind — delete any leftover `vaultik-*` directory under `$TMPDIR` by hand. `$TMPDIR` should be trusted to the same degree as the restore target.
|
||||
5. **Store permissions differ per command.** The backed-up host needs only PUT to run `snapshot create`: it writes blobs and metadata and neither reads nor deletes them. Other commands need more — `snapshot verify`, `snapshot restore`, and `prune` list and read; `prune`, `snapshot purge`, `snapshot remove`, and `remote nuke` also delete. The recommended cron line uses `--prune`, which runs `prune` on the backed-up host, so granting that host `--prune` gives it credentials that can delete its own backups. To keep the source host to PUT only, prune from a separate trusted host instead.
|
||||
6. **Changing recipients does not re-encrypt existing data.** Deduplicated chunks already on the destination stay encrypted to the recipients in force when they were written. A new snapshot that reuses them cannot be restored with a newly added recipient's key alone, because those reused objects were never encrypted to it. To make everything readable by a new key, run `vaultik database delete` and take a full backup to a fresh destination or prefix.
|
||||
6. **Changing recipients does not re-encrypt existing data.** Deduplicated chunks and same-named blobs already on the destination stay encrypted to the recipients in force when they were written. A new snapshot that reuses them cannot be restored with a newly added recipient's key alone, because those reused objects were never encrypted to it. To make everything readable by a new key, run `vaultik database delete` and take a full backup to a fresh destination or prefix.
|
||||
7. **X25519 recipients only.** vaultik rejects age ssh and plugin recipients. Long-lived ciphertext held by a third party (the destination operator) has no fallback if X25519 is ever broken: there is no second recipient type and no post-quantum option.
|
||||
|
||||
## Consistency Guarantees
|
||||
@@ -166,7 +166,7 @@ The prune operation is safe because:
|
||||
1. It keeps every blob listed in any snapshot's manifest and deletes only blobs that no manifest references
|
||||
2. Manifests are unencrypted and can be read without keys
|
||||
3. If any manifest cannot be downloaded or decoded, prune deletes nothing and exits with an error, rather than treating that snapshot's blobs as unreferenced
|
||||
4. It deletes an unreferenced blob only once the blob is older than `prune_grace_period` (default `24h`). A backup uploads its blobs before the manifest that lists them, so a prune running meanwhile, on this host or another sharing the destination, leaves them as long as no backup run takes longer than the grace period. `remote nuke` has no such check
|
||||
4. It deletes an unreferenced blob only once the blob is older than `prune_grace_period` (default `24h`). A backup uploads its blobs before the manifest that lists them, so a prune running meanwhile, on this host or another sharing the destination, leaves them as long as no backup run takes longer than the grace period. The grace period does not protect a blob a backup reuses from an earlier run that stopped before its manifest: the backup skips the upload of a blob already on the destination, so that blob keeps the time of its first upload, and a prune before the backup's manifest is written can delete it; the new snapshot then cannot be restored. `remote nuke` has no such check
|
||||
|
||||
## Restoration Requirements
|
||||
|
||||
|
||||
@@ -242,7 +242,10 @@ storage_url: ""
|
||||
# How old a blob that no snapshot lists must be before prune deletes it.
|
||||
# A backup uploads its blobs before the manifest that lists them, so on a
|
||||
# destination shared between hosts or users this must be longer than the
|
||||
# longest backup run. Accepts: 36h, 90m, etc. 0s deletes them at once.
|
||||
# longest backup run. It does not protect a blob a backup reuses from an
|
||||
# earlier run that stopped before its manifest: that blob keeps the time of
|
||||
# its first upload, so a prune can delete it.
|
||||
# Accepts: 36h, 90m, etc. 0s deletes them at once.
|
||||
# Default: 24h
|
||||
# prune_grace_period: 24h
|
||||
|
||||
|
||||
@@ -48,9 +48,9 @@ This will erase all local tracking of:
|
||||
- The storage-binding record
|
||||
|
||||
The remote storage is NOT affected. After deletion, the next backup
|
||||
will perform a full scan and upload all the data again, and the local
|
||||
index will re-bind to the currently configured storage destination on
|
||||
that run.
|
||||
will perform a full scan and re-deduplicate against existing remote
|
||||
blobs, and the local index will re-bind to the currently configured
|
||||
storage destination on that run.
|
||||
|
||||
Use --force to skip the confirmation prompt.`,
|
||||
Args: cobra.NoArgs,
|
||||
|
||||
@@ -23,7 +23,9 @@ set of still-referenced blob hashes, and deletes any blob not in that
|
||||
set that is older than prune_grace_period (default 24h). A newer one
|
||||
is kept: it may belong to a backup still running, on this host or
|
||||
another sharing the destination store, that has not yet uploaded its
|
||||
manifest.
|
||||
manifest. The grace period does not protect a blob such a backup
|
||||
reuses from an earlier run that stopped before its manifest: that
|
||||
blob keeps the time of its first upload, so it can be deleted.
|
||||
|
||||
Snapshot create --prune runs the same cleanup automatically; this
|
||||
command is the manual entry point for the same work (e.g. after a
|
||||
|
||||
@@ -6,6 +6,7 @@ import (
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"sneak.berlin/go/vaultik/internal/chunker"
|
||||
"sneak.berlin/go/vaultik/internal/log"
|
||||
@@ -450,6 +451,35 @@ func TestValidatePruneGracePeriod(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// TestLoadPruneGracePeriod checks that a config file without
|
||||
// prune_grace_period loads with the documented 24h. The value is written out
|
||||
// here rather than taken from defaultPruneGracePeriod, so that a change of the
|
||||
// default fails this test.
|
||||
func TestLoadPruneGracePeriod(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
const withoutGracePeriod = "snapshots:\n" +
|
||||
" test:\n" +
|
||||
" paths: [/tmp/vaultik-test-source]\n" +
|
||||
"storage_url: file:///tmp/vaultik-test-storage\n"
|
||||
|
||||
path := filepath.Join(t.TempDir(), "config.yml")
|
||||
|
||||
err := os.WriteFile(path, []byte(withoutGracePeriod), 0o600)
|
||||
if err != nil {
|
||||
t.Fatalf("write config: %v", err)
|
||||
}
|
||||
|
||||
cfg, err := Load(path)
|
||||
if err != nil {
|
||||
t.Fatalf("Load() unexpected error: %v", err)
|
||||
}
|
||||
|
||||
if cfg.PruneGracePeriod != 24*time.Hour {
|
||||
t.Errorf("prune_grace_period = %s, want 24h", cfg.PruneGracePeriod)
|
||||
}
|
||||
}
|
||||
|
||||
// TestLoadRejectsRelativeFileURL checks that a file:// storage_url with a
|
||||
// relative path fails at load. Accepted, it would resolve against the
|
||||
// working directory, so a backup run from another directory would write
|
||||
|
||||
@@ -155,7 +155,9 @@ func (r *BlobRepository) UpdateFinished(
|
||||
}
|
||||
|
||||
// UpdateUploaded marks a blob as uploaded and records the size of the
|
||||
// stored object as its compressed size.
|
||||
// stored object as its compressed size. That size can differ from the
|
||||
// local copy's when an identical blob was already stored with another
|
||||
// compression level or other recipients.
|
||||
func (r *BlobRepository) UpdateUploaded(
|
||||
ctx context.Context, tx *sql.Tx, id string, storedSize int64,
|
||||
) error {
|
||||
|
||||
@@ -1641,7 +1641,8 @@ func (s *Scanner) handleBlobReady(
|
||||
blobPath := fmt.Sprintf("blobs/%s/%s/%s",
|
||||
finishedBlob.Hash[:2], finishedBlob.Hash[2:4], finishedBlob.Hash)
|
||||
|
||||
err := s.uploadBlob(ctx, blobPath, blobWithReader, startTime, result)
|
||||
storedSize, blobExists, err := s.uploadBlobIfNeeded(
|
||||
ctx, blobPath, blobWithReader, startTime, result)
|
||||
if err != nil {
|
||||
cleanupErr := s.cleanupBlobTempFile(blobWithReader)
|
||||
|
||||
@@ -1649,7 +1650,8 @@ func (s *Scanner) handleBlobReady(
|
||||
fmt.Errorf("uploading blob %s: %w", finishedBlob.Hash, err), cleanupErr)
|
||||
}
|
||||
|
||||
err = s.recordBlobMetadata(ctx, finishedBlob, startTime)
|
||||
err = s.recordBlobMetadata(
|
||||
ctx, finishedBlob, storedSize, blobExists, startTime)
|
||||
if err != nil {
|
||||
cleanupErr := s.cleanupBlobTempFile(blobWithReader)
|
||||
|
||||
@@ -1673,33 +1675,46 @@ func (s *Scanner) handleBlobReady(
|
||||
return nil
|
||||
}
|
||||
|
||||
// uploadBlob uploads the blob to storage. A blob already there, left by
|
||||
// an earlier run stopped before its manifest, is uploaded again rather
|
||||
// than skipped: keeping its old time would let a prune elsewhere delete
|
||||
// it as an unreferenced blob older than prune_grace_period before this
|
||||
// backup's manifest lists it.
|
||||
func (s *Scanner) uploadBlob(
|
||||
// uploadBlobIfNeeded uploads the blob to storage if it doesn't already
|
||||
// exist. It returns the size of the object in storage, which for an
|
||||
// existing object can differ from the local copy's, and whether it
|
||||
// already existed.
|
||||
func (s *Scanner) uploadBlobIfNeeded(
|
||||
ctx context.Context,
|
||||
blobPath string,
|
||||
blobWithReader *blob.WithReader,
|
||||
startTime time.Time,
|
||||
result *ScanResult,
|
||||
) error {
|
||||
) (int64, bool, error) {
|
||||
finishedBlob := blobWithReader.FinishedBlob
|
||||
|
||||
// Check if blob already exists (deduplication after restart)
|
||||
destination := s.storage.Info().Location
|
||||
|
||||
stored, err := s.storage.Stat(ctx, blobPath)
|
||||
if err == nil {
|
||||
log.Info("Blob already exists in storage, skipping upload",
|
||||
"hash", finishedBlob.Hash,
|
||||
"size", humanize.Bytes(safeUint64(finishedBlob.Compressed)))
|
||||
s.ui.Infof("Blob %s (%s) already exists at %s. Skipping upload.",
|
||||
s.ui.Hex(finishedBlob.Hash), s.ui.Size(finishedBlob.Compressed),
|
||||
s.ui.Path(destination))
|
||||
|
||||
return stored.Size, true, nil
|
||||
}
|
||||
|
||||
s.ui.Beginf("Uploading blob %s (%s) to %s.",
|
||||
s.ui.Hex(finishedBlob.Hash), s.ui.Size(finishedBlob.Compressed),
|
||||
s.ui.Path(destination))
|
||||
|
||||
progressCallback := s.makeUploadProgressCallback(ctx, finishedBlob, startTime)
|
||||
|
||||
err := s.storage.PutWithProgress(ctx, blobPath, blobWithReader.Reader,
|
||||
err = s.storage.PutWithProgress(ctx, blobPath, blobWithReader.Reader,
|
||||
finishedBlob.Compressed, progressCallback)
|
||||
if err != nil {
|
||||
log.Error("Failed to upload blob", "hash", finishedBlob.Hash, "error", err)
|
||||
|
||||
return fmt.Errorf("uploading blob to storage: %w", err)
|
||||
return 0, false, fmt.Errorf("uploading blob to storage: %w", err)
|
||||
}
|
||||
|
||||
uploadDuration := time.Since(startTime)
|
||||
@@ -1730,7 +1745,7 @@ func (s *Scanner) uploadBlob(
|
||||
stats.BytesUploaded.Add(finishedBlob.Compressed)
|
||||
}
|
||||
|
||||
return nil
|
||||
return finishedBlob.Compressed, false, nil
|
||||
}
|
||||
|
||||
// makeUploadProgressCallback creates a progress callback for blob uploads.
|
||||
@@ -1800,6 +1815,8 @@ func (s *Scanner) makeUploadProgressCallback(
|
||||
func (s *Scanner) recordBlobMetadata(
|
||||
ctx context.Context,
|
||||
finishedBlob *blob.FinishedBlob,
|
||||
storedSize int64,
|
||||
blobExists bool,
|
||||
startTime time.Time,
|
||||
) error {
|
||||
finishedBlobID, err := types.ParseBlobID(finishedBlob.ID)
|
||||
@@ -1811,7 +1828,7 @@ func (s *Scanner) recordBlobMetadata(
|
||||
|
||||
return s.repos.WithTx(ctx, func(txCtx context.Context, tx *sql.Tx) error {
|
||||
err := s.repos.Blobs.UpdateUploaded(
|
||||
txCtx, tx, finishedBlob.ID, finishedBlob.Compressed)
|
||||
txCtx, tx, finishedBlob.ID, storedSize)
|
||||
if err != nil {
|
||||
return fmt.Errorf("updating blob upload timestamp: %w", err)
|
||||
}
|
||||
@@ -1822,17 +1839,19 @@ func (s *Scanner) recordBlobMetadata(
|
||||
return fmt.Errorf("adding blob to snapshot: %w", err)
|
||||
}
|
||||
|
||||
upload := &database.Upload{
|
||||
BlobHash: finishedBlob.Hash,
|
||||
SnapshotID: s.snapshotID,
|
||||
UploadedAt: startTime,
|
||||
Size: finishedBlob.Compressed,
|
||||
DurationMs: uploadDuration.Milliseconds(),
|
||||
}
|
||||
if !blobExists {
|
||||
upload := &database.Upload{
|
||||
BlobHash: finishedBlob.Hash,
|
||||
SnapshotID: s.snapshotID,
|
||||
UploadedAt: startTime,
|
||||
Size: finishedBlob.Compressed,
|
||||
DurationMs: uploadDuration.Milliseconds(),
|
||||
}
|
||||
|
||||
err = s.repos.Uploads.Create(txCtx, tx, upload)
|
||||
if err != nil {
|
||||
return fmt.Errorf("recording upload metrics: %w", err)
|
||||
err := s.repos.Uploads.Create(txCtx, tx, upload)
|
||||
if err != nil {
|
||||
return fmt.Errorf("recording upload metrics: %w", err)
|
||||
}
|
||||
}
|
||||
|
||||
return nil
|
||||
|
||||
@@ -84,100 +84,6 @@ func TestPruneDuringBackupKeepsItsBlobs(t *testing.T) {
|
||||
assertRestoredTree(t, fs, restoreDir, testFiles)
|
||||
}
|
||||
|
||||
// TestPruneDuringBackupKeepsBlobsOfKilledRun covers a backup that builds
|
||||
// the blobs an earlier run, stopped before its manifest upload, left on
|
||||
// the destination store, older than the grace period by the time of the
|
||||
// backup. A prune from another host runs before the backup's manifest
|
||||
// upload, and the backup must then restore.
|
||||
func TestPruneDuringBackupKeepsBlobsOfKilledRun(t *testing.T) {
|
||||
log.Initialize(log.Config{})
|
||||
t.Parallel()
|
||||
|
||||
fs := afero.NewOsFs()
|
||||
tempDir := t.TempDir()
|
||||
dataDir := filepath.Join(tempDir, "src")
|
||||
storeDir := t.TempDir()
|
||||
restoreDir := filepath.Join(tempDir, "restored")
|
||||
dbPath := filepath.Join(tempDir, "index.sqlite")
|
||||
|
||||
ctx := context.Background()
|
||||
testFiles := writeFaultSourceTree(t, fs, dataDir)
|
||||
|
||||
cfg := faultTestConfig()
|
||||
cfg.IndexPath = dbPath
|
||||
cfg.ChunkSize = config.Size(faultChunkSize)
|
||||
cfg.Snapshots = map[string]config.SnapshotConfig{
|
||||
"data": {Paths: []string{dataDir}},
|
||||
}
|
||||
|
||||
inner, err := storage.NewFileStorer(storeDir)
|
||||
require.NoError(t, err)
|
||||
|
||||
otherHostCfg := faultTestConfig()
|
||||
otherHostCfg.PruneGracePeriod = time.Hour
|
||||
otherHost := newReaderVaultik(ctx, otherHostCfg, inner, nil, fs)
|
||||
|
||||
var (
|
||||
killedRun = true
|
||||
pruned bool
|
||||
pruneErr error
|
||||
)
|
||||
|
||||
store := faultstore.New(inner)
|
||||
store.OnPut = func(key string) faultstore.PutAction {
|
||||
if !strings.HasSuffix(key, "/manifest.json.zst") {
|
||||
return faultstore.PutNormal
|
||||
}
|
||||
|
||||
if killedRun {
|
||||
return faultstore.PutFail
|
||||
}
|
||||
|
||||
pruned = true
|
||||
pruneErr = otherHost.PruneBlobs(&vaultik.PruneOptions{Force: true})
|
||||
|
||||
return faultstore.PutNormal
|
||||
}
|
||||
|
||||
db, err := database.New(ctx, dbPath)
|
||||
require.NoError(t, err)
|
||||
|
||||
repos := database.NewRepositories(db)
|
||||
v := newBackupVaultik(ctx, cfg, store, repos, db, fs)
|
||||
opts := &vaultik.SnapshotCreateOptions{Cron: true}
|
||||
|
||||
require.Error(t, v.CreateSnapshot(opts),
|
||||
"the first run must stop at its manifest upload")
|
||||
|
||||
beforeGracePeriod := time.Now().Add(-2 * otherHostCfg.PruneGracePeriod)
|
||||
|
||||
for object := range inner.ListStream(ctx, "blobs/") {
|
||||
require.NoError(t, object.Err)
|
||||
require.NoError(t, os.Chtimes(filepath.Join(storeDir, object.Key),
|
||||
beforeGracePeriod, beforeGracePeriod))
|
||||
}
|
||||
|
||||
killedRun = false
|
||||
|
||||
require.NoError(t, v.CreateSnapshot(opts))
|
||||
require.True(t, pruned, "the prune must run before the manifest upload")
|
||||
require.NoError(t, pruneErr)
|
||||
|
||||
local, err := repos.Snapshots.ListRecent(ctx, listRecentTestLimit)
|
||||
require.NoError(t, err)
|
||||
require.Len(t, local, 1)
|
||||
require.NoError(t, db.Close())
|
||||
|
||||
reader := newReaderVaultik(ctx, cfg, inner, nil, fs)
|
||||
require.NoError(t, reader.Restore(&vaultik.RestoreOptions{
|
||||
SnapshotID: local[0].ID.String(),
|
||||
TargetDir: restoreDir,
|
||||
Verify: true,
|
||||
}), "a prune during the backup must not delete the blobs it rebuilt")
|
||||
|
||||
assertRestoredTree(t, fs, restoreDir, testFiles)
|
||||
}
|
||||
|
||||
// TestPruneKeepsUnreferencedBlobWithNoTime checks that a blob no
|
||||
// manifest lists, whose listing gives no modification time (as an S3
|
||||
// listing entry without LastModified does), is kept and counted with
|
||||
|
||||
@@ -14,13 +14,13 @@ import (
|
||||
"sneak.berlin/go/vaultik/internal/vaultik"
|
||||
)
|
||||
|
||||
// TestVerifyPassesWhenBackupRebuildsStoredBlob backs up a file, then backs
|
||||
// it up again from an empty local index, as after `database delete`, with
|
||||
// an extra age recipient. The second backup builds the same blob, already
|
||||
// stored, and uploads it again; its copy is larger, because age writes a
|
||||
// header entry for each recipient. The second snapshot must record the
|
||||
// size of the object now stored, so shallow and deep verify pass.
|
||||
func TestVerifyPassesWhenBackupRebuildsStoredBlob(t *testing.T) {
|
||||
// TestVerifyPassesWhenBackupReusesStoredBlob backs up a file, then backs it
|
||||
// up again from an empty local index, as after `database delete`, with an
|
||||
// extra age recipient. The second backup builds the same blob, finds it
|
||||
// already stored and skips the upload, but its own copy is larger, because
|
||||
// age writes a header entry for each recipient. The second snapshot must
|
||||
// record the size of the stored object, so shallow and deep verify pass.
|
||||
func TestVerifyPassesWhenBackupReusesStoredBlob(t *testing.T) {
|
||||
log.Initialize(log.Config{})
|
||||
t.Parallel()
|
||||
|
||||
@@ -58,8 +58,8 @@ func TestVerifyPassesWhenBackupRebuildsStoredBlob(t *testing.T) {
|
||||
snapshotID, result := runDedupSnapshot(ctx, t, env.sm, env.makeScanner(),
|
||||
env.cfg.Hostname, "second", dataDir, secondDBPath)
|
||||
require.Positive(t, result.BlobsCreated)
|
||||
require.Equal(t, result.BlobsCreated, result.BlobsUploaded,
|
||||
"the second backup should upload every blob again")
|
||||
require.Zero(t, result.BlobsUploaded,
|
||||
"the second backup should find every blob already stored")
|
||||
|
||||
require.NoError(t, env.db.Close())
|
||||
|
||||
|
||||
Reference in New Issue
Block a user