Compare commits

...
2 Commits
Author SHA1 Message Date
sneak 27109bfd89 Correct doc and help sentences that are false about the code (closes #233)
check / check (push) Waiting to run
A blob is written whole to a temporary file and uploaded once finished,
not streamed to storage. The README, ARCHITECTURE.md and
config.example.yml now say a backup needs free temporary space for each
blob (twice that for an rclone destination that cannot stream uploads)
and for the metadata export's copies of the local index, in $TMPDIR, or
partly in /var/tmp when TMPDIR is unset. Also corrected: the snapshot ID
format, what restore reads and how incomplete snapshots are removed in
docs/DATAMODEL.md, what source_path holds, the index_path default, the
config search order in the snapshot create help, what snapshot remove
cleans up in the prune help, how the release installs Go, and the
script/release and script/fmt-check comments.

Model: opus-5-5
2026-10-07 15:17:08 +00:00
clawbot e161343eac Speed up the vaultik and database package tests (closes #235)
check / check (push) Waiting to run
In the Dockerfile test phase, internal/vaultik spent 23.6s of 31.9s in
36 tests run one at a time. 24 of them were serial only because they
call log.Initialize; they now call it before t.Parallel(), so the
logger is replaced before any parallel test runs. The 12 still serial
change the umask, TMPDIR, os.Stderr or time.Local.

TestLargeDatasets took 8.4s of the 9.8s internal/database run by
committing each of its 1,500 inserts on its own; it now makes them in
one transaction. TestDedupOnlySnapshotRestores gives its second backup
its own snapshot name instead of sleeping 1.1s for a new snapshot ID.

Model: opus-5-5
2026-10-07 17:12:07 +02:00
24 changed files with 161 additions and 130 deletions
+7 -3
View File
@@ -54,10 +54,12 @@ The database tracks five primary entities and their relationships:
#### File (`database.File`)
Represents a file, directory, or symlink in the backup system. Stores metadata needed for restoration:
- Path, source_path (for restore path stripping), mtime
- Path, mtime
- Size, mode, ownership (uid, gid)
- Symlink target (if applicable)
It also stores `source_path`, the source directory the scan found it under, made absolute and with symlinks resolved. Restore does not read it.
#### Chunk (`database.Chunk`)
A content-addressed unit of data. Files are split into variable-size chunks using the FastCDC algorithm:
- `ChunkHash`: SHA256 hash of chunk content (primary key)
@@ -82,9 +84,11 @@ The final storage unit uploaded to S3. Contains many compressed and encrypted ch
Blob creation process:
1. Chunks are accumulated (up to MaxBlobSize, typically 10GB)
2. As each chunk is added, its uncompressed bytes are fed to a running SHA-256
3. Concurrently, the same bytes are compressed with zstd, then encrypted with age (recipients configured in config), and streamed to storage
3. Concurrently, the same bytes are compressed with zstd, then encrypted with age (recipients configured in config), and written to a temporary file
4. On finalize, the blob's name is the double SHA-256 of the uncompressed contents — `hex(SHA256(SHA256(...)))` — not a hash of the compressed, encrypted bytes
5. Uploaded to `blobs/{hash[0:2]}/{hash[2:4]}/{hash}`
5. The finished file is uploaded to `blobs/{hash[0:2]}/{hash[2:4]}/{hash}` and then deleted
A backup needs free temporary space, because each blob is written whole to a temporary file before it is uploaded (up to about `blob_size_limit`; an rclone destination that cannot stream uploads needs about twice that) and the metadata export writes copies of the local index. Temporary files go to `$TMPDIR` (default `/tmp`); with `TMPDIR` unset, SQLite writes one of those copies to `/var/tmp`.
#### BlobChunk (`database.BlobChunk`)
Maps chunks to their position within blobs:
+9 -8
View File
@@ -546,7 +546,7 @@ complete annotated example also lives in
| `s3.*` | | Legacy S3 configuration (endpoint, bucket, credentials) |
| `exclude` | | Global exclude patterns (applied to all snapshots) |
| `chunk_size` | `10MB` | Average chunk size for content-defined chunking |
| `blob_size_limit` | `10GB` | Maximum blob size before splitting. Must be at least four times `chunk_size` (the largest chunk the chunker can emit), otherwise a single-chunk blob could exceed the limit |
| `blob_size_limit` | `10GB` | Maximum blob size before splitting. Must be at least four times `chunk_size` (the largest chunk the chunker can emit), otherwise a single-chunk blob could exceed the limit. A backup needs free temporary space, because each blob is written whole to a temporary file before it is uploaded (up to about `blob_size_limit`; an rclone destination that cannot stream uploads needs about twice that) and the metadata export writes copies of the local index. Temporary files go to `$TMPDIR` (default `/tmp`); with `TMPDIR` unset, SQLite writes one of those copies to `/var/tmp` |
| `compression_level` | `3` | zstd compression level (1-19) |
| `hostname` | system hostname | Hostname used in snapshot IDs |
| `index_path` | platform data dir | Local SQLite index path |
@@ -805,7 +805,8 @@ them. We provide:
in the `Dockerfile` together.
* `script/release` — cross-compile and publish the release artifacts
with the pinned `goreleaser`. Refuses a `goreleaser` on `PATH` whose
version is not the pinned one, on the same reasoning as `script/lint`.
version is not the pinned one, because a different version would build
a different release from the same tag.
* `script/release-snapshot` — the same build with no publishing and no
tagging, into `./dist`
* `script/test` — run the test suite by building the `test` phase of
@@ -915,14 +916,14 @@ It is passed to `goreleaser` as `GITEA_TOKEN`. The runner's automatic
token is deliberately not used: it is not guaranteed to carry release
write access.
The Go toolchain that compiles the released binaries comes from an
`actions/setup-go` step pinned by commit sha, reading its version from
`go.mod` (currently `1.26.1`, the same version the `Dockerfile` builder
stage pins by digest). `goreleaser` shells out to `go` for every
The Go toolchain that compiles the released binaries is installed by
`script/install-go`, which downloads the version named by `go.mod`
(currently `1.26.1`, the same version the `Dockerfile` builder stage
pins by digest) and refuses the archive unless its sha256 matches the
value committed in the script. `goreleaser` shells out to `go` for every
cross-compile, so without that step the release would either fail
outright or ship binaries built by whatever unpinned toolchain the
runner happened to carry — the one unpinned thing in an otherwise
hash-pinned release path.
runner happened to carry.
To rehearse the whole build without publishing or tagging anything:
+26
View File
@@ -22,6 +22,32 @@ the tag exists and is exercised; what is left is merging `next` to
# Completed Steps
- 2026-10-07: Corrected documentation, help text and comments that were
false about the code
([issue #233](https://git.eeqj.de/sneak/vaultik/issues/233)). A blob
is not streamed to storage. The README, `ARCHITECTURE.md` and
`config.example.yml` now say a backup needs free temporary space,
because each blob is written whole to a temporary file before it is
uploaded (up to about `blob_size_limit`; an rclone destination that
cannot stream uploads needs about twice that) and the metadata export
writes copies of the local index. Temporary files go to `$TMPDIR`
(default `/tmp`); with `TMPDIR` unset, SQLite writes one of those
copies to `/var/tmp`. Also corrected: the snapshot ID format, what restore
reads and how incomplete snapshots are removed in `docs/DATAMODEL.md`,
what `source_path` holds, the `index_path` and config file defaults,
what `snapshot remove` cleans up, how the release gets its Go
toolchain, and the `script/release` and `script/fmt-check` comments.
- 2026-10-07: Cut the time the `internal/vaultik` and `internal/database`
tests take ([issue #235](https://git.eeqj.de/sneak/vaultik/issues/235)).
Most of the `internal/vaultik` time went to 24 tests that ran one at a
time only because they call `log.Initialize`; they now call it before
`t.Parallel()`, as the package's other tests do. `TestLargeDatasets`
committed each of its 1,500 inserts on its own and now makes them in
one transaction. `TestDedupOnlySnapshotRestores` gives its second
backup its own snapshot name instead of sleeping past the one-second
timestamp in the snapshot ID.
- 2026-10-07: Made two messages say only what is true
([issue #240](https://git.eeqj.de/sneak/vaultik/issues/240)). A config
file that others can read was warned about as containing S3
+10 -2
View File
@@ -295,8 +295,10 @@ storage_url: "rclone://myremote/path/to/backups"
# Path to local SQLite index database
# This database tracks file state for incremental backups
# Default: /var/lib/vaultik/index.sqlite
#index_path: /var/lib/vaultik/index.sqlite
# Default: the platform data directory, e.g.
# macOS: ~/Library/Application Support/vaultik/index.sqlite
# Linux: ~/.local/share/vaultik/index.sqlite
#index_path: /path/to/index.sqlite
# Average chunk size for content-defined chunking
# Smaller chunks = better deduplication but more metadata
@@ -311,6 +313,12 @@ storage_url: "rclone://myremote/path/to/backups"
# Chunking uses no secret (the FastCDC parameters are fixed and public). At a
# large limit a blob holds hundreds of chunks, so individual chunk lengths are
# not visible in its size; lowering the limit toward chunk_size exposes them.
# A backup needs free temporary space, because each blob is written whole to
# a temporary file before it is uploaded (up to about blob_size_limit; an
# rclone destination that cannot stream uploads needs about twice that) and
# the metadata export writes copies of the local index. Temporary files go to
# $TMPDIR (default /tmp); with TMPDIR unset, SQLite writes one of those copies
# to /var/tmp.
# Supports: 1GB, 10G, 500MB, 1GiB, etc.
# Default: 10GB
#blob_size_limit: 10GB
+5 -6
View File
@@ -111,7 +111,7 @@ Maps chunks to the blobs that contain them.
Tracks backup snapshots.
**Columns:**
- `id` (TEXT PRIMARY KEY) - Snapshot ID (format: hostname-YYYYMMDD-HHMMSSZ)
- `id` (TEXT PRIMARY KEY) - Snapshot ID (format: `hostname_name_timestamp`, e.g. `server1_home_2025-06-01T12:00:00Z`: the hostname up to its first `.`, the snapshot name, and an RFC 3339 UTC timestamp)
- `hostname` (TEXT) - Hostname where backup was created
- `vaultik_version` (TEXT) - Version of Vaultik used
- `vaultik_git_revision` (TEXT) - Git revision of Vaultik used
@@ -218,8 +218,8 @@ The `{remote-key}` directory name is a one-way hash of the human snapshot ID, so
### 4. Restore Process
The restore process doesn't use the local database. Instead:
1. Downloads snapshot metadata from S3
2. Downloads required blobs based on manifest
1. Downloads and decrypts the snapshot's metadata database (`db.zst.age`) from S3
2. Downloads the blobs holding the chunks of the files being restored, found through that database's `blob_chunks` table; the manifest is not read
3. Reconstructs files from decrypted and decompressed chunks
### 5. Pruning
@@ -232,9 +232,8 @@ The restore process doesn't use the local database. Instead:
Before each backup:
1. Query incomplete snapshots (where `completed_at IS NULL`)
2. Check if metadata exists in S3
3. If no metadata, delete snapshot and all associations
4. Clean up orphaned files, chunks, and blobs
2. Delete each one and all its associations, without checking S3 for its metadata
3. Clean up orphaned files, chunks, and blobs
## Repository Pattern
+5 -3
View File
@@ -22,9 +22,11 @@ scans every snapshot manifest in the destination store, builds the
set of still-referenced blob hashes, and deletes any blob not in that
set.
Snapshot create --prune and snapshot remove run the same cleanup
automatically; this command is the manual entry point for the same
work (e.g. after a crashed backup or to reclaim storage).`,
Snapshot create --prune runs the same cleanup automatically; this
command is the manual entry point for the same work (e.g. after a
crashed backup or to reclaim storage). Snapshot remove leaves blobs in
place; run this command afterwards to delete the ones no longer
referenced.`,
Args: cobra.NoArgs,
RunE: func(cmd *cobra.Command, _ []string) error {
// Use unified config resolution
+3 -2
View File
@@ -66,8 +66,9 @@ func newSnapshotCreateCommand() *cobra.Command {
If snapshot names are provided, only those snapshots are created.
If no names are provided, all configured snapshots are created.
Config is located at /etc/vaultik/config.yml by default, but can be overridden by
specifying a path using --config or by setting VAULTIK_CONFIG to a path.`,
The config is read from the path given by --config or VAULTIK_CONFIG;
otherwise from the platform config directory (~/.config/vaultik/config.yml
on Linux), then /etc/vaultik/config.yml.`,
Args: cobra.ArbitraryArgs,
RunE: func(cmd *cobra.Command, args []string) error {
// Pass snapshot names from args
+1 -2
View File
@@ -14,8 +14,7 @@ type File struct {
ID types.FileID // UUID primary key
Path types.FilePath // Absolute path of the file
// SourcePath is the source directory this file came from (used for
// restore path stripping).
// SourcePath is the source directory this file came from.
SourcePath types.SourcePath
MTime time.Time
Size int64
@@ -3,6 +3,7 @@ package database
import (
"context"
"database/sql"
"fmt"
"strings"
"testing"
@@ -367,7 +368,7 @@ func verifyBlobNullUploadTS(
}
// createLargeDatasetFiles creates fileCount files and adds every other
// one to the snapshot.
// one to the snapshot, in one transaction as a backup writes them.
func createLargeDatasetFiles(
t *testing.T,
repos *Repositories,
@@ -376,9 +377,10 @@ func createLargeDatasetFiles(
) {
t.Helper()
ctx := context.Background()
start := time.Now()
err := repos.WithTx(context.Background(),
func(ctx context.Context, tx *sql.Tx) error {
for i := range fileCount {
file := &File{
Path: types.FilePath(fmt.Sprintf("/large/file%05d.txt", i)),
@@ -389,19 +391,25 @@ func createLargeDatasetFiles(
GID: uint32(1000 + (i % 10)),
}
err := repos.Files.Create(ctx, nil, file)
err := repos.Files.Create(ctx, tx, file)
if err != nil {
t.Fatalf("failed to create file %d: %v", i, err)
return fmt.Errorf("creating file %d: %w", i, err)
}
// Add half to snapshot
if i%2 == 0 {
err = repos.Snapshots.AddFileByID(ctx, nil, snapshotID, file.ID)
err = repos.Snapshots.AddFileByID(ctx, tx, snapshotID, file.ID)
if err != nil {
return err
}
}
}
return nil
})
if err != nil {
t.Fatal(err)
}
}
}
t.Logf("Created %d files in %v", fileCount, time.Since(start))
}
+1 -1
View File
@@ -5,7 +5,7 @@
CREATE TABLE IF NOT EXISTS files (
id TEXT PRIMARY KEY, -- UUID
path TEXT NOT NULL UNIQUE,
source_path TEXT NOT NULL DEFAULT '', -- The source directory this file came from (for restore path stripping)
source_path TEXT NOT NULL DEFAULT '', -- The source directory this file came from
mtime INTEGER NOT NULL, -- whole seconds since the Unix epoch
mtime_nsec INTEGER NOT NULL, -- nanoseconds within that second, 0 to 999999999
size INTEGER NOT NULL,
+2 -4
View File
@@ -58,8 +58,8 @@ type Scanner struct {
compressionLevel int
ageRecipient string
snapshotID string // Current snapshot being processed
// currentSourcePath is the source directory being scanned (used for
// restore path stripping).
// currentSourcePath is the source directory being scanned, stored with
// each file record.
currentSourcePath string
exclude []string // Glob patterns for files/directories to exclude
compiledExclude []compiledPattern // Compiled glob patterns
@@ -211,7 +211,6 @@ func (s *Scanner) Scan(
ctx context.Context, path string, snapshotID string,
) (*ScanResult, error) {
s.snapshotID = snapshotID
// Store source path for file records (used during restore)
s.currentSourcePath = path
result := &ScanResult{
StartTime: time.Now().UTC(),
@@ -1200,7 +1199,6 @@ func (s *Scanner) checkFileInMemory(
file := &database.File{
ID: fileID,
Path: types.FilePath(path),
// Store source directory for restore path stripping
SourcePath: types.SourcePath(s.currentSourcePath),
MTime: info.ModTime(),
Size: info.Size(),
+2 -2
View File
@@ -155,8 +155,8 @@ type BlobHash string
// FilePath represents an absolute path to a file or directory.
type FilePath string
// SourcePath represents the root directory from which files are backed up.
// Used during restore to strip the source prefix from paths.
// SourcePath is the source directory a scan found a file under, made
// absolute and with symlinks resolved.
type SourcePath string
// Hostname identifies a host machine.
@@ -130,10 +130,9 @@ func assertThirdSnapshotRestores(
// up, and that snapshot is removed. The first snapshot keeps the file row,
// which now lists the appended content's chunks, while removal drops the
// blob that held them.
//
//nolint:paralleltest // installs the global logger via log.Initialize
func TestBackupAfterRemovingNewestSnapshotRestoresChangedFile(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
fs := afero.NewOsFs()
tempDir := t.TempDir()
@@ -178,10 +177,9 @@ func TestBackupAfterRemovingNewestSnapshotRestoresChangedFile(t *testing.T) {
// The next run's prune drops that incomplete snapshot and its blob, while
// the first snapshot keeps the file row, which now lists the appended
// content's chunks.
//
//nolint:paralleltest // installs the global logger via log.Initialize
func TestBackupAfterInterruptedRunRestoresChangedFile(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
fs := afero.NewOsFs()
tempDir := t.TempDir()
+15 -22
View File
@@ -38,11 +38,9 @@ import (
// (https://git.eeqj.de/sneak/vaultik/issues/130) and is not re-tested
// here; these tests target the layers above the backend.
//
// The tests run serially, not with t.Parallel: each calls
// log.Initialize, which replaces the package-global logger, and a
// backup or restore running concurrently reads that same logger. Under
// -race the two collide. Running one at a time is the same choice
// prune_count_test.go already makes for the same reason.
// log.Initialize replaces the package-global logger that a running
// backup or restore reads, so each test calls it before t.Parallel,
// while no parallel test is running yet.
const (
faultChunkSize = int64(64 * 1024)
@@ -165,17 +163,19 @@ func newReaderVaultik(
// Scenario 3: a stored blob's bytes are flipped before restore reads
// them. Restore must fail loudly, and no file must be left on the
// restore target holding corrupt content.
//
//nolint:paralleltest // installs the global logger via log.Initialize
func TestRestoreRejectsCorruptBlob(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
assertRestoreRejectsDamagedBlob(t, faultstore.GetCorrupt, "corrupt")
}
// Scenario 4: a stored blob is truncated before restore reads it. Same
// contract as the corrupt case.
//
//nolint:paralleltest // installs the global logger via log.Initialize
func TestRestoreRejectsTruncatedBlob(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
assertRestoreRejectsDamagedBlob(t, faultstore.GetTruncate, "truncated")
}
@@ -188,7 +188,6 @@ func assertRestoreRejectsDamagedBlob(
t *testing.T, fault faultstore.GetFault, name string,
) {
t.Helper()
log.Initialize(log.Config{})
fs := afero.NewOsFs()
tempDir := t.TempDir()
@@ -232,10 +231,9 @@ func assertRestoreRejectsDamagedBlob(
// Scenario 6: the backend accepts blob uploads and reports success but
// stores nothing. verify --deep must catch it.
//
//nolint:paralleltest // installs the global logger via log.Initialize
func TestDeepVerifyCatchesLyingBackend(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
fs := afero.NewOsFs()
tempDir := t.TempDir()
@@ -285,10 +283,9 @@ func TestDeepVerifyCatchesLyingBackend(t *testing.T) {
// Scenario 1a: a blob upload fails partway through. The interrupted run
// must not record the blob as uploaded, must not reference it from the
// snapshot, and must leave no blob object at the destination.
//
//nolint:paralleltest // installs the global logger via log.Initialize
func TestInterruptedBlobUploadRecordsNoUploadedBlob(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
fs := afero.NewOsFs()
tempDir := t.TempDir()
@@ -361,10 +358,9 @@ func TestInterruptedBlobUploadRecordsNoUploadedBlob(t *testing.T) {
// chunks in a blob that was actually uploaded, so the retry re-chunks and
// re-uploads the affected data instead of silently referencing data that
// never reached storage.
//
//nolint:paralleltest // installs the global logger via log.Initialize
func TestBackupRetryAfterInterruptedUploadIsRestorable(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
fs := afero.NewOsFs()
tempDir := t.TempDir()
@@ -426,10 +422,9 @@ func TestBackupRetryAfterInterruptedUploadIsRestorable(t *testing.T) {
// covered by TestBackupCompletesOnlyAfterMetadataExport
// (https://git.eeqj.de/sneak/vaultik/issues/177); this test exercises the
// lower-level export path in isolation.
//
//nolint:paralleltest // installs the global logger via log.Initialize
func TestBackupSurvivesMetadataExportInterruption(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
fs := afero.NewOsFs()
tempDir := t.TempDir()
@@ -509,10 +504,9 @@ func TestBackupSurvivesMetadataExportInterruption(t *testing.T) {
// destination. Rerunning the backup must then prune the incomplete
// snapshot, produce a snapshot whose destination metadata and local index
// agree, and restore. See https://git.eeqj.de/sneak/vaultik/issues/177.
//
//nolint:paralleltest // installs the global logger via log.Initialize
func TestBackupCompletesOnlyAfterMetadataExport(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
fs := afero.NewOsFs()
tempDir := t.TempDir()
@@ -685,10 +679,9 @@ func faultScannerFactory(
// Scenario 5: the restore target runs out of space mid-file. Restore
// must fail with an out-of-space error, and must not leave a truncated
// file at the target path presenting as a complete restore.
//
//nolint:paralleltest // installs the global logger via log.Initialize
func TestRestoreReportsDiskFull(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
osFS := afero.NewOsFs()
tempDir := t.TempDir()
+10 -11
View File
@@ -928,17 +928,17 @@ func setupDedupBackupEnv(
}
}
// runDedupSnapshot creates a "dedup" snapshot, scans dataDir into it,
// completes it, and exports its metadata, returning the snapshot ID and
// scan result.
// runDedupSnapshot creates a snapshot with the given name, scans dataDir
// into it, completes it, and exports its metadata, returning the snapshot
// ID and scan result.
func runDedupSnapshot(
ctx context.Context, t *testing.T,
sm *snapshot.SnapshotManager, scanner *snapshot.Scanner,
hostname, dataDir, dbPath string,
hostname, name, dataDir, dbPath string,
) (string, *snapshot.ScanResult) {
t.Helper()
id, err := sm.CreateSnapshotWithName(ctx, hostname, "dedup", "v", "g")
id, err := sm.CreateSnapshotWithName(ctx, hostname, name, "v", "g")
require.NoError(t, err)
result, err := scanner.Scan(ctx, dataDir, id)
@@ -980,16 +980,15 @@ func TestDedupOnlySnapshotRestores(t *testing.T) {
// First snapshot — uploads all blobs.
_, r1 := runDedupSnapshot(ctx, t, sm, makeScanner(),
cfg.Hostname, dataDir, dbPath)
cfg.Hostname, "first", dataDir, dbPath)
require.Positive(t, r1.BlobsCreated,
"first snapshot should upload at least one blob")
// Second snapshot — same data, every chunk dedups. Sleep past the
// second-precision timestamp so the snapshot IDs differ.
time.Sleep(1100 * time.Millisecond)
// Second snapshot — same data, every chunk dedups. Its own name gives
// it a different snapshot ID without waiting for the one-second
// timestamp in the ID to tick over.
id2, r2 := runDedupSnapshot(ctx, t, sm, makeScanner(),
cfg.Hostname, dataDir, dbPath)
cfg.Hostname, "second", dataDir, dbPath)
require.Equal(t, 0, r2.BlobsCreated,
"second snapshot should upload zero new blobs (fully dedup'd)")
+5 -10
View File
@@ -78,10 +78,9 @@ func backUpThenUnplug(
// TestFirstBackupCreatesDestinationDirectory checks that a first backup
// to a destination directory that does not exist yet creates it, and
// that the destination can be listed afterwards.
//
//nolint:paralleltest // installs the global logger via log.Initialize
func TestFirstBackupCreatesDestinationDirectory(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
ctx := context.Background()
storeDir := filepath.Join(t.TempDir(), "volume", "backup")
@@ -96,10 +95,9 @@ func TestFirstBackupCreatesDestinationDirectory(t *testing.T) {
// TestListSnapshotsWarnsWhenDestinationMissing checks that snapshot list
// warns and shows the local index alone, without reporting the local
// snapshot as missing from the destination.
//
//nolint:paralleltest // installs the global logger via log.Initialize
func TestListSnapshotsWarnsWhenDestinationMissing(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
ctx := context.Background()
v, repos, out := backUpThenUnplug(ctx, t)
@@ -116,10 +114,9 @@ func TestListSnapshotsWarnsWhenDestinationMissing(t *testing.T) {
// TestRemoveSnapshotWarnsWhenDestinationMissing checks that snapshot
// remove warns that the metadata could not be removed from the
// destination, instead of reporting that it was.
//
//nolint:paralleltest // installs the global logger via log.Initialize
func TestRemoveSnapshotWarnsWhenDestinationMissing(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
ctx := context.Background()
v, repos, out := backUpThenUnplug(ctx, t)
@@ -138,10 +135,9 @@ func TestRemoveSnapshotWarnsWhenDestinationMissing(t *testing.T) {
// TestPruneKeepsLocalRecordsWhenDestinationMissing checks that prune
// fails on a destination it cannot list and deletes no local snapshot
// record.
//
//nolint:paralleltest // installs the global logger via log.Initialize
func TestPruneKeepsLocalRecordsWhenDestinationMissing(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
ctx := context.Background()
v, repos, _ := backUpThenUnplug(ctx, t)
@@ -158,10 +154,9 @@ func TestPruneKeepsLocalRecordsWhenDestinationMissing(t *testing.T) {
// TestPurgeSaysListingFailedOnceWhenDestinationMissing checks that
// snapshot purge fails on a destination it cannot list, with an error
// that says "listing remote snapshots" once.
//
//nolint:paralleltest // installs the global logger via log.Initialize
func TestPurgeSaysListingFailedOnceWhenDestinationMissing(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
ctx := context.Background()
v, _, _ := backUpThenUnplug(ctx, t)
+1 -2
View File
@@ -14,10 +14,9 @@ import (
// the discarded-error bug: getTableCount for a table its query cannot
// resolve must not silently become 0. A count that could not be read is
// reported as unknown, which a reader can tell apart from an empty table.
//
//nolint:paralleltest // installs the global logger via log.Initialize
func TestTableCountForReportSurfacesReadFailure(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
ctx := context.Background()
+1 -2
View File
@@ -164,10 +164,9 @@ func scratchEntries(t *testing.T, dir string) []string {
// restore while a blob download is in progress. The download fails only
// because of the cancel, so Restore must return context.Canceled without
// reporting the file that needs the blob as failed.
//
//nolint:paralleltest // installs the global logger via log.Initialize
func TestRestoreSkipErrorsCancelDuringBlobDownload(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
fs := afero.NewOsFs()
tempDir := t.TempDir()
+6 -5
View File
@@ -45,9 +45,10 @@ type missingBlobBackup struct {
// after one blob of a two-blob snapshot was deleted. Every file stored in
// that blob must be reported as failed and left absent, every other file
// must be restored intact, and Restore must still return an error.
//
//nolint:paralleltest // installs the global logger via log.Initialize
func TestRestoreSkipErrorsSkipsFilesOfMissingBlob(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
ctx := context.Background()
backup := backupThenDeleteOneBlob(ctx, t)
@@ -85,9 +86,10 @@ func TestRestoreSkipErrorsSkipsFilesOfMissingBlob(t *testing.T) {
// TestRestoreMissingBlobAbortsWithoutSkipErrors checks that a deleted blob
// still ends the restore with an error when SkipErrors is not set.
//
//nolint:paralleltest // installs the global logger via log.Initialize
func TestRestoreMissingBlobAbortsWithoutSkipErrors(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
ctx := context.Background()
backup := backupThenDeleteOneBlob(ctx, t)
@@ -107,7 +109,6 @@ func backupThenDeleteOneBlob(
ctx context.Context, t *testing.T,
) *missingBlobBackup {
t.Helper()
log.Initialize(log.Config{})
fs := afero.NewOsFs()
tempDir := t.TempDir()
+1 -2
View File
@@ -18,10 +18,9 @@ import (
// A file rewritten with its size unchanged and a new mtime in the same
// second as the mtime the index holds must still be backed up. See
// https://git.eeqj.de/sneak/vaultik/issues/226.
//
//nolint:paralleltest // installs the global logger via log.Initialize
func TestBackupOfSameSecondRewriteRestoresNewContent(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
fs := afero.NewOsFs()
tempDir := t.TempDir()
+9 -8
View File
@@ -54,8 +54,6 @@ type summaryEnv struct {
func newSummaryEnv(t *testing.T) *summaryEnv {
t.Helper()
log.Initialize(log.Config{})
fs := afero.NewOsFs()
tempDir := t.TempDir()
srcDir := filepath.Join(tempDir, "src")
@@ -193,9 +191,10 @@ func (e *summaryEnv) dataLine(total, backedUp int64) string {
// A first backup stores copy.bin's chunks while backing up a.bin, so
// copy.bin's chunks are deduplicated within the run. Each file and byte
// is still counted once.
//
//nolint:paralleltest // installs the global logger via log.Initialize
func TestSnapshotSummaryFirstRun(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
env := newSummaryEnv(t)
summary := env.backUp(t, "first", false)
@@ -219,9 +218,10 @@ func TestSnapshotSummaryFirstRun(t *testing.T) {
// An incremental backup where a.bin's mtime changed but its content did
// not: a.bin is backed up again and every one of its chunks is already
// stored.
//
//nolint:paralleltest // installs the global logger via log.Initialize
func TestSnapshotSummaryIncrementalRunWithDeduplicatedChunks(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
env := newSummaryEnv(t)
env.backUp(t, "first", false)
@@ -251,9 +251,10 @@ func TestSnapshotSummaryIncrementalRunWithDeduplicatedChunks(t *testing.T) {
// Under --cron the progress reporter is off; the upload figures must
// still reach the summary and the snapshots row. The snapshot has two
// paths, each backed up by its own scan.
//
//nolint:paralleltest // installs the global logger via log.Initialize
func TestSnapshotSummaryCronRunRecordsUploads(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
env := newSummaryEnv(t)
summary := env.backUp(t, "split", true)
+1 -2
View File
@@ -18,10 +18,9 @@ import (
// A backup without --cron runs the progress reporter while one scanner
// scans each path of the snapshot in turn. See
// https://git.eeqj.de/sneak/vaultik/issues/253.
//
//nolint:paralleltest // installs the global logger via log.Initialize
func TestBackupWithoutCronOfTwoPathSnapshotRestoresBothPaths(t *testing.T) {
log.Initialize(log.Config{})
t.Parallel()
const snapshotName = "data"
+5 -2
View File
@@ -1,6 +1,9 @@
#!/bin/sh
# script/fmt-check: check formatting (read-only). Same scope as
# script/fmt, but fails instead of writing.
# script/fmt-check: check formatting (read-only). Fails instead of
# writing. It checks every Go file outside .tool, which is more than
# script/fmt formats: `go fmt ./...` skips `testdata` directories and
# files and directories whose names start with `.` or `_`. Fix a file
# only this reports with `gofmt -w`.
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
+3 -4
View File
@@ -21,10 +21,9 @@ goreleaser_version() {
head -n 1
}
# Resolve the goreleaser to run, on the same rule script/lint uses for
# golangci-lint: a binary on PATH is accepted only when it is exactly
# the pinned version, because a differently versioned tool would
# produce a differently built release from the same tag. Anything else
# Resolve the goreleaser to run. A binary on PATH is accepted only when
# it is exactly the pinned version, because a differently versioned tool
# would produce a differently built release from the same tag. Anything else
# comes from .tool/bin, and a missing one is a loud failure naming the
# script that installs it rather than a silent fallback.
resolve_goreleaser() {