Compare commits
1
Commits
next
..
0e57ea874a
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
0e57ea874a |
@@ -104,12 +104,7 @@ Version: 2025-06-08
|
|||||||
|
|
||||||
13. Pre-1.0: NEVER write database migrations. There are no live databases
|
13. Pre-1.0: NEVER write database migrations. There are no live databases
|
||||||
anywhere — every user's local index can be rebuilt from a fresh full
|
anywhere — every user's local index can be rebuilt from a fresh full
|
||||||
backup. To change the schema, edit `internal/database/schema/001.sql`
|
backup. When the schema changes, just change `schema.sql` (and any code
|
||||||
(and any code that touches the affected tables) directly; do not add new
|
that touches the affected tables). The local index is disposable until
|
||||||
numbered schema files. Those numbered files and the `schema_migrations`
|
1.0 ships and is tagged.
|
||||||
table they populate only bootstrap a fresh database — they are not an
|
|
||||||
upgrade path. The local index is disposable until 1.0 ships and is
|
|
||||||
tagged; once 1.0 is tagged that clause expires and the question of
|
|
||||||
upgrading existing indexes returns. See [`docs/DATAMODEL.md`](docs/DATAMODEL.md)
|
|
||||||
for the full explanation.
|
|
||||||
|
|
||||||
|
|||||||
+3
-24
@@ -20,6 +20,8 @@
|
|||||||
# golang:1.26.1-alpine, 2026-03-17
|
# golang:1.26.1-alpine, 2026-03-17
|
||||||
FROM golang:1.26.1-alpine@sha256:2389ebfa5b7f43eeafbd6be0c3700cc46690ef842ad962f6c5bd6be49ed82039 AS builder
|
FROM golang:1.26.1-alpine@sha256:2389ebfa5b7f43eeafbd6be0c3700cc46690ef842ad962f6c5bd6be49ed82039 AS builder
|
||||||
|
|
||||||
|
ARG VERSION=dev
|
||||||
|
|
||||||
# Build tooling: make, plus a C toolchain because `go test -race` needs cgo.
|
# Build tooling: make, plus a C toolchain because `go test -race` needs cgo.
|
||||||
# The sqlite driver is pure Go (modernc.org/sqlite), so no sqlite library or
|
# The sqlite driver is pure Go (modernc.org/sqlite), so no sqlite library or
|
||||||
# CLI is required.
|
# CLI is required.
|
||||||
@@ -64,31 +66,8 @@ RUN [ -n "$CHECK_EPOCH" ] || exit 1
|
|||||||
RUN echo "check epoch: ${CHECK_EPOCH}" && make fmt-check
|
RUN echo "check epoch: ${CHECK_EPOCH}" && make fmt-check
|
||||||
RUN echo "check epoch: ${CHECK_EPOCH}" && make test
|
RUN echo "check epoch: ${CHECK_EPOCH}" && make test
|
||||||
|
|
||||||
# Version, commit and build date are computed on the host by
|
|
||||||
# script/docker and script/cibuild (where .git exists) and passed in as
|
|
||||||
# build args. The build context excludes .git (see .dockerignore), so
|
|
||||||
# the build cannot derive them itself: it used to try, with `git
|
|
||||||
# rev-parse` inside this stage, and always got "unknown". VERSION comes
|
|
||||||
# from script/version, the source of truth shared with the Makefile, so
|
|
||||||
# it carries the same tag / dev-<sha> / -dirty rules and a Docker image
|
|
||||||
# reports the same string a local build of the same tree would.
|
|
||||||
#
|
|
||||||
# The defaults are the fallback for a bare `docker build .` that passes
|
|
||||||
# none of them: an unset arg would otherwise stamp an empty string and
|
|
||||||
# produce an image that cannot report its own version, commit or date.
|
|
||||||
# They match what an out-of-git build reports elsewhere.
|
|
||||||
#
|
|
||||||
# These ARGs sit here, after the checks, rather than at the top of the
|
|
||||||
# stage: every commit changes their values, and a value change
|
|
||||||
# invalidates all layers below the ARG. Declared up top they would bust
|
|
||||||
# `go mod download`; here they only rekey this build layer, which the
|
|
||||||
# COPY of the sources above already rebuilds on any change anyway.
|
|
||||||
ARG VERSION=dev
|
|
||||||
ARG COMMIT=unknown
|
|
||||||
ARG COMMIT_DATE=unknown
|
|
||||||
|
|
||||||
# Build (pure Go, no CGO required since we use modernc.org/sqlite)
|
# Build (pure Go, no CGO required since we use modernc.org/sqlite)
|
||||||
RUN CGO_ENABLED=0 go build -ldflags "-X 'sneak.berlin/go/vaultik/internal/globals.Version=${VERSION}' -X 'sneak.berlin/go/vaultik/internal/globals.Commit=${COMMIT}' -X 'sneak.berlin/go/vaultik/internal/globals.CommitDate=${COMMIT_DATE}'" -o /vaultik ./cmd/vaultik
|
RUN CGO_ENABLED=0 go build -ldflags "-X 'sneak.berlin/go/vaultik/internal/globals.Version=${VERSION}' -X 'sneak.berlin/go/vaultik/internal/globals.Commit=$(git rev-parse HEAD 2>/dev/null || echo unknown)' -X 'sneak.berlin/go/vaultik/internal/globals.CommitDate=$(git show -s --format=%cs HEAD 2>/dev/null || echo unknown)'" -o /vaultik ./cmd/vaultik
|
||||||
|
|
||||||
# Runtime stage
|
# Runtime stage
|
||||||
# alpine:3.21, 2026-02-25
|
# alpine:3.21, 2026-02-25
|
||||||
|
|||||||
@@ -251,8 +251,7 @@ local index alone, and still exits zero.
|
|||||||
per-snapshot-name (`--keep-latest` keeps the latest of each name, not the
|
per-snapshot-name (`--keep-latest` keeps the latest of each name, not the
|
||||||
latest globally).
|
latest globally).
|
||||||
* `--keep-latest`: Keep only the most recent snapshot of each name
|
* `--keep-latest`: Keep only the most recent snapshot of each name
|
||||||
* `--older-than <duration>`: Remove snapshots older than duration (e.g. `30d`,
|
* `--older-than <duration>`: Remove snapshots older than duration (e.g. `30d`, `6m`, `1y`)
|
||||||
`4w`, `6mo`, `1y`; `m` is minutes, `mo` is months)
|
|
||||||
* `--snapshot <name>`: Restrict to specific snapshot names (repeat for multiple)
|
* `--snapshot <name>`: Restrict to specific snapshot names (repeat for multiple)
|
||||||
* `--force`: Skip confirmation prompt
|
* `--force`: Skip confirmation prompt
|
||||||
|
|
||||||
@@ -457,13 +456,9 @@ Key fields:
|
|||||||
sequentially. Restore speed is bound by single-stream throughput.
|
sequentially. Restore speed is bound by single-stream throughput.
|
||||||
* **Device nodes, named pipes, and sockets are silently skipped.** Only
|
* **Device nodes, named pipes, and sockets are silently skipped.** Only
|
||||||
regular files, directories, and symlinks are backed up.
|
regular files, directories, and symlinks are backed up.
|
||||||
* **No upgrade path between versions.** There is no supported way to carry
|
* **No database migrations.** If the local SQLite schema changes between
|
||||||
an existing local index across a schema change; if the local SQLite
|
versions, delete the local database (`vaultik database delete`) and run
|
||||||
schema changes between versions, delete the local database (`vaultik
|
a full backup. Remote storage is unaffected.
|
||||||
database delete`) and run a full backup. Remote storage is unaffected.
|
|
||||||
(The binary does embed numbered schema files and a `schema_migrations`
|
|
||||||
table to bootstrap a fresh database — see [`docs/DATAMODEL.md`](docs/DATAMODEL.md)
|
|
||||||
— but that is not an upgrade path.)
|
|
||||||
* **Files that change during backup may be inconsistent.** There is no
|
* **Files that change during backup may be inconsistent.** There is no
|
||||||
filesystem snapshot or freeze. If a file is modified between the scan
|
filesystem snapshot or freeze. If a file is modified between the scan
|
||||||
and chunk phases, the backed-up copy may reflect a partial write.
|
and chunk phases, the backed-up copy may reflect a partial write.
|
||||||
@@ -533,12 +528,10 @@ priority.
|
|||||||
another host" workflow works but isn't documented as a
|
another host" workflow works but isn't documented as a
|
||||||
first-class operation in this README. Worth a dedicated section
|
first-class operation in this README. Worth a dedicated section
|
||||||
once it's settled.
|
once it's settled.
|
||||||
* **Cross-version schema upgrades.** There is no upgrade path between
|
* **Schema migrations.** Currently nonexistent — pre-1.0 schema
|
||||||
released versions — pre-1.0 schema changes are handled by `vaultik
|
changes are handled by `vaultik database delete` plus a full
|
||||||
database delete` plus a full re-scan (see
|
re-scan. Post-1.0 we'll need a migration story to keep existing
|
||||||
[`docs/DATAMODEL.md`](docs/DATAMODEL.md)). Post-1.0 we'll need a
|
index databases usable across upgrades.
|
||||||
migration story to keep existing index databases usable across
|
|
||||||
upgrades.
|
|
||||||
* **Storage backend coverage tests.** S3, file://, and rclone://
|
* **Storage backend coverage tests.** S3, file://, and rclone://
|
||||||
all share the Storer interface but the rclone path is the least
|
all share the Storer interface but the rclone path is the least
|
||||||
exercised in CI.
|
exercised in CI.
|
||||||
|
|||||||
@@ -34,15 +34,6 @@ release" is exactly the contradiction
|
|||||||
is distinguishable from one that could not be queried. The counts have
|
is distinguishable from one that could not be queried. The counts have
|
||||||
no `--json` representation — under `--json` the summary is suppressed
|
no `--json` representation — under `--json` the summary is suppressed
|
||||||
entirely — so nothing there can show a false `0`.
|
entirely — so nothing there can show a false `0`.
|
||||||
|
|
||||||
- 2026-09-21: Made the s3 storage backend report a missing object as
|
|
||||||
`storage.ErrNotFound`, like the `file` and `rclone` backends and as the
|
|
||||||
`Storer` interface documents. `S3Storer.Get` and `Stat` returned the raw
|
|
||||||
AWS SDK error, so `errors.Is(err, storage.ErrNotFound)` was false on s3
|
|
||||||
and callers branched differently per backend. Added a small `s3.IsNotFound`
|
|
||||||
helper (reused by `HeadObject`) and a test that a missing key maps to
|
|
||||||
`ErrNotFound`
|
|
||||||
([issue #129](https://git.eeqj.de/sneak/vaultik/issues/129)).
|
|
||||||
- 2026-09-21: Fixed `verify --deep` reporting healthy snapshots as
|
- 2026-09-21: Fixed `verify --deep` reporting healthy snapshots as
|
||||||
corrupt. Its final blob-integrity check hashed the encrypted
|
corrupt. Its final blob-integrity check hashed the encrypted
|
||||||
downloaded bytes with a single SHA256 and compared that to the blob
|
downloaded bytes with a single SHA256 and compared that to the blob
|
||||||
@@ -78,24 +69,6 @@ release" is exactly the contradiction
|
|||||||
keeps that exact compiler from auto-switching. Bumping Go now touches
|
keeps that exact compiler from auto-switching. Bumping Go now touches
|
||||||
`go.mod`, the checksum, and the `Dockerfile` `golang` digest together.
|
`go.mod`, the checksum, and the `Dockerfile` `golang` digest together.
|
||||||
|
|
||||||
- 2026-09-21: Collapsed the two duration parsers into one and fixed the
|
|
||||||
`--older-than` months example
|
|
||||||
([issue #123](https://git.eeqj.de/sneak/vaultik/issues/123)). Two
|
|
||||||
functions named `parseDuration` existed with different grammars;
|
|
||||||
`snapshot purge --older-than` and `--keep-newer-than` both already went
|
|
||||||
through the one in `internal/vaultik`, while the richer copy in
|
|
||||||
`internal/cli/duration.go` was reachable only from its own test. Kept
|
|
||||||
the live-path parser and deleted the unused one, so no flag's accepted
|
|
||||||
grammar changes. The trap the issue was filed over: `README.md`
|
|
||||||
documented `6m` as the months example for `--older-than`, but `m` is
|
|
||||||
minutes, so the documented command deleted every snapshot older than
|
|
||||||
six minutes on a destructive flag. Corrected the doc to `6mo` and put
|
|
||||||
both flags' help text on one example list that states `m` is minutes
|
|
||||||
and `mo` is months. The surviving parser now rejects negatives, which
|
|
||||||
it previously accepted (`-5h`) or silently made positive (`-5d`).
|
|
||||||
Table-driven tests cover every unit, `6m` as six minutes, `6mo` as 180
|
|
||||||
days, and rejection of a bare number, an unknown unit, and a negative.
|
|
||||||
|
|
||||||
- 2026-08-10: Moved every lint run into its own container, as a build
|
- 2026-08-10: Moved every lint run into its own container, as a build
|
||||||
step ([issue #113](https://git.eeqj.de/sneak/vaultik/issues/113)).
|
step ([issue #113](https://git.eeqj.de/sneak/vaultik/issues/113)).
|
||||||
New root `Dockerfile.lint`, built by `script/lint`, runs
|
New root `Dockerfile.lint`, built by `script/lint`, runs
|
||||||
|
|||||||
@@ -1,102 +0,0 @@
|
|||||||
package main_test
|
|
||||||
|
|
||||||
import (
|
|
||||||
"strings"
|
|
||||||
"testing"
|
|
||||||
|
|
||||||
"github.com/stretchr/testify/assert"
|
|
||||||
"github.com/stretchr/testify/require"
|
|
||||||
)
|
|
||||||
|
|
||||||
// This file guards the version stamping of the product image (issue
|
|
||||||
// #75). The failure it protects against is silent: the image still
|
|
||||||
// builds and runs, but `vaultik version` inside it reports "commit:
|
|
||||||
// unknown", so an operator cannot tell which source produced a given
|
|
||||||
// backup. .dockerignore excludes .git, so the build cannot derive the
|
|
||||||
// commit itself; the values must be computed on the host and passed in.
|
|
||||||
//
|
|
||||||
// These are parses of the committed files, for the same reason the lint
|
|
||||||
// guards next door are: shelling out to docker would nest a build
|
|
||||||
// inside `make test`. That `vaultik version` in the built image really
|
|
||||||
// prints the host's version is verified by hand and recorded on the
|
|
||||||
// pull request.
|
|
||||||
|
|
||||||
// dockerScript is script/docker, relative to the repository root.
|
|
||||||
const dockerScript = "script/docker"
|
|
||||||
|
|
||||||
// versionArgs are the ldflag targets the build stamps and, matching
|
|
||||||
// them, the build args the host must supply. The names line up so the
|
|
||||||
// same list checks both files.
|
|
||||||
func versionArgs() []string {
|
|
||||||
return []string{"VERSION", "COMMIT", "COMMIT_DATE"}
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestProductDockerfileTakesVersionAsBuildArgs fails unless the build
|
|
||||||
// declares each version arg and stamps it into the binary by ldflag
|
|
||||||
// reference, rather than computing it in the container.
|
|
||||||
func TestProductDockerfileTakesVersionAsBuildArgs(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
found := instructions(t, productDockerfile)
|
|
||||||
|
|
||||||
for _, arg := range versionArgs() {
|
|
||||||
require.GreaterOrEqual(t, indexOf(found, "ARG "+arg), 0,
|
|
||||||
"%s must declare `ARG %s` so the host can pass it in",
|
|
||||||
productDockerfile, arg)
|
|
||||||
|
|
||||||
assertLdflagReferences(t, found, arg)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestProductDockerfileDoesNotDeriveVersionItself is the anti-regression
|
|
||||||
// for the original defect: the container ran `git rev-parse`, but .git
|
|
||||||
// is not in the build context, so it always resolved to "unknown". No
|
|
||||||
// git command may reach into a build that cannot see the history.
|
|
||||||
func TestProductDockerfileDoesNotDeriveVersionItself(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
text := instructionText(readRepoFile(t, productDockerfile))
|
|
||||||
|
|
||||||
assert.NotContains(t, text, "git ",
|
|
||||||
"%s must not run git: .git is excluded from the build context, so"+
|
|
||||||
" any value it derives is wrong. Pass version, commit and date"+
|
|
||||||
" in as build args instead.", productDockerfile)
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestDockerScriptComputesVersionOnTheHost fails unless script/docker
|
|
||||||
// derives each value where .git exists and passes it as a build arg,
|
|
||||||
// with VERSION coming from script/version so a Docker build reports the
|
|
||||||
// same string a local build of the same tree would.
|
|
||||||
func TestDockerScriptComputesVersionOnTheHost(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
script := readRepoFile(t, dockerScript)
|
|
||||||
|
|
||||||
for _, arg := range versionArgs() {
|
|
||||||
assert.Contains(t, script, "--build-arg "+arg+"=",
|
|
||||||
"%s must pass --build-arg %s to the build", dockerScript, arg)
|
|
||||||
}
|
|
||||||
|
|
||||||
assert.Contains(t, script, "/version",
|
|
||||||
"%s must take VERSION from script/version, the source of truth"+
|
|
||||||
" shared with the Makefile", dockerScript)
|
|
||||||
}
|
|
||||||
|
|
||||||
// assertLdflagReferences fails unless some build instruction stamps the
|
|
||||||
// named variable from the ARG (a ${arg} reference), not from a value
|
|
||||||
// computed inside the container.
|
|
||||||
func assertLdflagReferences(t *testing.T, found []string, arg string) {
|
|
||||||
t.Helper()
|
|
||||||
|
|
||||||
for _, instruction := range found {
|
|
||||||
if strings.HasPrefix(instruction, "RUN ") &&
|
|
||||||
strings.Contains(instruction, "go build") &&
|
|
||||||
strings.Contains(instruction, "${"+arg+"}") {
|
|
||||||
return
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
assert.Fail(t, "version arg is declared but never stamped",
|
|
||||||
"the go build in %s must reference ${%s} in its ldflags, or the"+
|
|
||||||
" arg is passed and discarded", productDockerfile, arg)
|
|
||||||
}
|
|
||||||
@@ -304,14 +304,10 @@ func instructionText(contents string) string {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// indexOf returns the position of the first instruction equal to, or
|
// indexOf returns the position of the first instruction equal to, or
|
||||||
// beginning with, want; -1 if there is none. An `ARG NAME=default`
|
// beginning with, want; -1 if there is none.
|
||||||
// counts as beginning with `ARG NAME`, so a declared arg is found
|
|
||||||
// whether or not it carries a default.
|
|
||||||
func indexOf(found []string, want string) int {
|
func indexOf(found []string, want string) int {
|
||||||
for i, instruction := range found {
|
for i, instruction := range found {
|
||||||
if instruction == want ||
|
if instruction == want || strings.HasPrefix(instruction, want+" ") {
|
||||||
strings.HasPrefix(instruction, want+" ") ||
|
|
||||||
strings.HasPrefix(instruction, want+"=") {
|
|
||||||
return i
|
return i
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
+1
-11
@@ -10,16 +10,6 @@ import (
|
|||||||
)
|
)
|
||||||
|
|
||||||
func main() {
|
func main() {
|
||||||
os.Exit(run())
|
|
||||||
}
|
|
||||||
|
|
||||||
// run sets up optional profiling, runs the CLI, and returns the process
|
|
||||||
// exit code. os.Exit lives in main so it fires only after run's deferred
|
|
||||||
// profile writers have flushed. cli.Entry returns a status code rather
|
|
||||||
// than calling os.Exit itself: an os.Exit from inside it would skip
|
|
||||||
// these defers and truncate the profile of a failing command -- exactly
|
|
||||||
// the command one most often wants to profile.
|
|
||||||
func run() int {
|
|
||||||
// CPU profiling: set VAULTIK_CPUPROFILE=/path/to/cpu.prof
|
// CPU profiling: set VAULTIK_CPUPROFILE=/path/to/cpu.prof
|
||||||
if cpuProfile := os.Getenv("VAULTIK_CPUPROFILE"); cpuProfile != "" {
|
if cpuProfile := os.Getenv("VAULTIK_CPUPROFILE"); cpuProfile != "" {
|
||||||
f, err := os.Create(cpuProfile) //nolint:gosec // G304: operator-set path
|
f, err := os.Create(cpuProfile) //nolint:gosec // G304: operator-set path
|
||||||
@@ -56,5 +46,5 @@ func run() int {
|
|||||||
}()
|
}()
|
||||||
}
|
}
|
||||||
|
|
||||||
return cli.Entry()
|
cli.Entry()
|
||||||
}
|
}
|
||||||
|
|||||||
+5
-24
@@ -5,30 +5,11 @@
|
|||||||
Vaultik uses a local SQLite database to track file metadata, chunk mappings, and blob associations during the backup process. This database serves as an index for incremental backups and enables efficient deduplication.
|
Vaultik uses a local SQLite database to track file metadata, chunk mappings, and blob associations during the backup process. This database serves as an index for incremental backups and enables efficient deduplication.
|
||||||
|
|
||||||
**Important Notes:**
|
**Important Notes:**
|
||||||
|
- **No Migration Support (pre-1.0)**: Vaultik does not support database schema
|
||||||
This section is the authoritative explanation of the schema/migration story;
|
migrations. The local index is treated as disposable — if the schema changes,
|
||||||
other documents (the README and `AGENTS.md`) link here.
|
delete the local SQLite database (`vaultik database delete`) and run a full
|
||||||
|
backup. The remote storage is unaffected; the new index will re-deduplicate
|
||||||
- **No upgrade path between versions (pre-1.0)**: Vaultik has no supported way to
|
against existing remote blobs.
|
||||||
carry an existing local index across a schema change. The index is disposable
|
|
||||||
— if the on-disk schema changes between versions, delete the local SQLite
|
|
||||||
database (`vaultik database delete`) and run a full backup. Remote storage is
|
|
||||||
unaffected; the new index re-deduplicates against existing remote blobs. This
|
|
||||||
is the standing project policy, and it is separate from the schema bootstrap
|
|
||||||
described next.
|
|
||||||
- **Schema bootstrap**: a fresh database is populated from numbered SQL files
|
|
||||||
embedded in the binary under `internal/database/schema/`. `000.sql` creates the
|
|
||||||
`schema_migrations` table; `001.sql` creates the application tables. On opening
|
|
||||||
a database the code applies each numbered file that has not yet run and records
|
|
||||||
its version in `schema_migrations`. This bootstraps a new database; it does not
|
|
||||||
upgrade an existing one between released versions.
|
|
||||||
- **Changing the schema (pre-1.0)**: edit `internal/database/schema/001.sql` (and
|
|
||||||
the code that touches the affected tables) directly. Do not add new numbered
|
|
||||||
files — there is no installed base to migrate.
|
|
||||||
- **Disposability expires at 1.0**: the index is treated as disposable only until
|
|
||||||
1.0 ships and is tagged. Once 1.0 is tagged that clause expires and the
|
|
||||||
question of upgrading existing indexes returns. It is deliberately left open
|
|
||||||
here.
|
|
||||||
- **Version Compatibility**: In rare cases, you may need to use the same version
|
- **Version Compatibility**: In rare cases, you may need to use the same version
|
||||||
of Vaultik to restore a backup as was used to create it. This ensures
|
of Vaultik to restore a backup as was used to create it. This ensures
|
||||||
compatibility with the metadata format stored in S3.
|
compatibility with the metadata format stored in S3.
|
||||||
|
|||||||
+39
-90
@@ -11,7 +11,6 @@ import (
|
|||||||
"os/signal"
|
"os/signal"
|
||||||
"path/filepath"
|
"path/filepath"
|
||||||
"strings"
|
"strings"
|
||||||
"sync"
|
|
||||||
"syscall"
|
"syscall"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
@@ -197,90 +196,13 @@ func RunApp(ctx context.Context, app *fx.App) error {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// errReported marks a failure the operation has already shown the user
|
|
||||||
// (and deliberately withheld under --json). Entry turns it into a
|
|
||||||
// non-zero exit status without printing anything further, so the error
|
|
||||||
// line is not doubled. It flows up from RunOperation through cobra to
|
|
||||||
// Entry.
|
|
||||||
var errReported = errors.New("operation failed")
|
|
||||||
|
|
||||||
// RunOperation runs op against the Vaultik instance inside the fx app
|
|
||||||
// and turns a failure into a returned error rather than an os.Exit from
|
|
||||||
// within the goroutine. An os.Exit there skipped main's deferred
|
|
||||||
// profile writers -- so profiling a failing command yielded a truncated
|
|
||||||
// profile (issue #75) -- and RunWithApp's PID-lock release, and denied
|
|
||||||
// the app any graceful shutdown; returning the error to the top runs
|
|
||||||
// all three.
|
|
||||||
//
|
|
||||||
// op runs in a goroutine so OnStart returns promptly and an interrupt
|
|
||||||
// can still cancel through OnStop; when it finishes, success or failure,
|
|
||||||
// it triggers shutdown, which is what lets RunWithApp return. report is
|
|
||||||
// called with a non-canceled failure so the caller can log it (and
|
|
||||||
// suppress it under --json) before it becomes errReported. A context
|
|
||||||
// cancellation is the interrupt path, not a failure: it is neither
|
|
||||||
// reported nor counted as one.
|
|
||||||
func RunOperation(
|
|
||||||
ctx context.Context, opts AppOptions,
|
|
||||||
op func(v *vaultik.Vaultik) error, report func(err error),
|
|
||||||
) error {
|
|
||||||
var (
|
|
||||||
mu sync.Mutex
|
|
||||||
failed bool
|
|
||||||
)
|
|
||||||
|
|
||||||
opts.Invokes = append(opts.Invokes,
|
|
||||||
fx.Invoke(func(v *vaultik.Vaultik, lc fx.Lifecycle) {
|
|
||||||
lc.Append(fx.Hook{
|
|
||||||
OnStart: func(_ context.Context) error {
|
|
||||||
go func() {
|
|
||||||
err := op(v)
|
|
||||||
if err != nil && !errors.Is(err, context.Canceled) {
|
|
||||||
report(err)
|
|
||||||
|
|
||||||
mu.Lock()
|
|
||||||
failed = true
|
|
||||||
mu.Unlock()
|
|
||||||
}
|
|
||||||
|
|
||||||
stopErr := v.Shutdowner.Shutdown()
|
|
||||||
if stopErr != nil {
|
|
||||||
log.Error("Failed to shutdown", "error", stopErr)
|
|
||||||
}
|
|
||||||
}()
|
|
||||||
|
|
||||||
return nil
|
|
||||||
},
|
|
||||||
OnStop: func(_ context.Context) error {
|
|
||||||
v.Cancel()
|
|
||||||
|
|
||||||
return nil
|
|
||||||
},
|
|
||||||
})
|
|
||||||
}))
|
|
||||||
|
|
||||||
err := RunWithApp(ctx, opts)
|
|
||||||
if err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
|
|
||||||
// The goroutine sets failed before triggering the shutdown that lets
|
|
||||||
// RunWithApp return, so the write is in place by the time we read it.
|
|
||||||
mu.Lock()
|
|
||||||
defer mu.Unlock()
|
|
||||||
|
|
||||||
if failed {
|
|
||||||
return errReported
|
|
||||||
}
|
|
||||||
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// runVaultikApp runs the standard single-operation command lifecycle
|
// runVaultikApp runs the standard single-operation command lifecycle
|
||||||
// shared by the list/purge/verify/remove/remote-info subcommands:
|
// shared by the list/purge/verify/remove/remote-info subcommands:
|
||||||
// resolve the config, then run op against the Vaultik instance through
|
// resolve the config, start the fx app, run op against the Vaultik
|
||||||
// RunOperation, reporting a failure prefixed with failMsg (suppressed
|
// instance in a goroutine, report a failure prefixed with failMsg
|
||||||
// while suppressErrors is true, e.g. under --json). extraQuiet is OR-ed
|
// (suppressed while suppressErrors is true, e.g. under --json), then
|
||||||
// into LogOptions.Quiet (e.g. --json output modes).
|
// trigger shutdown. The operation is cancelled when the app stops.
|
||||||
|
// extraQuiet is OR-ed into LogOptions.Quiet (e.g. --json output modes).
|
||||||
func runVaultikApp(
|
func runVaultikApp(
|
||||||
cmd *cobra.Command, extraQuiet, suppressErrors bool,
|
cmd *cobra.Command, extraQuiet, suppressErrors bool,
|
||||||
failMsg string, op func(v *vaultik.Vaultik) error,
|
failMsg string, op func(v *vaultik.Vaultik) error,
|
||||||
@@ -292,20 +214,47 @@ func runVaultikApp(
|
|||||||
|
|
||||||
rootFlags := GetRootFlags()
|
rootFlags := GetRootFlags()
|
||||||
|
|
||||||
return RunOperation(cmd.Context(), AppOptions{
|
return RunWithApp(cmd.Context(), AppOptions{
|
||||||
ConfigPath: configPath,
|
ConfigPath: configPath,
|
||||||
LogOptions: log.Options{
|
LogOptions: log.Options{
|
||||||
Verbose: rootFlags.Verbose,
|
Verbose: rootFlags.Verbose,
|
||||||
Debug: rootFlags.Debug,
|
Debug: rootFlags.Debug,
|
||||||
Quiet: rootFlags.Quiet || extraQuiet,
|
Quiet: rootFlags.Quiet || extraQuiet,
|
||||||
},
|
},
|
||||||
}, op, func(err error) {
|
Modules: []fx.Option{},
|
||||||
if suppressErrors {
|
Invokes: []fx.Option{
|
||||||
return
|
fx.Invoke(func(v *vaultik.Vaultik, lc fx.Lifecycle) {
|
||||||
}
|
lc.Append(fx.Hook{
|
||||||
|
OnStart: func(_ context.Context) error {
|
||||||
|
go func() {
|
||||||
|
err := op(v)
|
||||||
|
if err != nil {
|
||||||
|
if !errors.Is(err, context.Canceled) {
|
||||||
|
if !suppressErrors {
|
||||||
|
log.Error(failMsg, "error", err)
|
||||||
|
ReportErrorf("%s: %v", failMsg, err)
|
||||||
|
}
|
||||||
|
|
||||||
log.Error(failMsg, "error", err)
|
os.Exit(1)
|
||||||
ReportErrorf("%s: %v", failMsg, err)
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
err = v.Shutdowner.Shutdown()
|
||||||
|
if err != nil {
|
||||||
|
log.Error("Failed to shutdown", "error", err)
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
|
||||||
|
return nil
|
||||||
|
},
|
||||||
|
OnStop: func(_ context.Context) error {
|
||||||
|
v.Cancel()
|
||||||
|
|
||||||
|
return nil
|
||||||
|
},
|
||||||
|
})
|
||||||
|
}),
|
||||||
|
},
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,126 @@
|
|||||||
|
package cli
|
||||||
|
|
||||||
|
import (
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"regexp"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Approximate lengths of the extended calendar units accepted by
|
||||||
|
// parseDuration.
|
||||||
|
const (
|
||||||
|
durationDay = 24 * time.Hour
|
||||||
|
durationWeek = 7 * durationDay
|
||||||
|
durationMonth = 30 * durationDay
|
||||||
|
durationYear = 365 * durationDay
|
||||||
|
)
|
||||||
|
|
||||||
|
var (
|
||||||
|
errNegativeDuration = errors.New("negative durations are not supported")
|
||||||
|
errInvalidDuration = errors.New("invalid duration format")
|
||||||
|
errUnknownTimeUnit = errors.New("unknown time unit")
|
||||||
|
)
|
||||||
|
|
||||||
|
// parseDuration parses duration strings. Supports standard Go duration format
|
||||||
|
// (e.g., "3h30m", "1h45m30s") as well as extended units:
|
||||||
|
// - d: days (e.g., "30d", "7d")
|
||||||
|
// - w: weeks (e.g., "2w", "4w")
|
||||||
|
// - mo: months (30 days) (e.g., "6mo", "1mo")
|
||||||
|
// - y: years (365 days) (e.g., "1y", "2y")
|
||||||
|
//
|
||||||
|
// Can combine units: "1y6mo", "2w3d", "1d12h30m"
|
||||||
|
func parseDuration(s string) (time.Duration, error) {
|
||||||
|
// First try standard Go duration parsing
|
||||||
|
d, err := time.ParseDuration(s)
|
||||||
|
if err == nil {
|
||||||
|
return d, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Extended duration parsing
|
||||||
|
// Check for negative values
|
||||||
|
if strings.HasPrefix(strings.TrimSpace(s), "-") {
|
||||||
|
return 0, errNegativeDuration
|
||||||
|
}
|
||||||
|
|
||||||
|
// Pattern matches: number + unit, repeated
|
||||||
|
re := regexp.MustCompile(`(\d+(?:\.\d+)?)\s*([a-zA-Z]+)`)
|
||||||
|
matches := re.FindAllStringSubmatch(s, -1)
|
||||||
|
|
||||||
|
if len(matches) == 0 {
|
||||||
|
return 0, fmt.Errorf("%w: %q", errInvalidDuration, s)
|
||||||
|
}
|
||||||
|
|
||||||
|
var total time.Duration
|
||||||
|
|
||||||
|
for _, match := range matches {
|
||||||
|
valueStr := match[1]
|
||||||
|
unit := strings.ToLower(match[2])
|
||||||
|
|
||||||
|
value, err := strconv.ParseFloat(valueStr, 64)
|
||||||
|
if err != nil {
|
||||||
|
return 0, fmt.Errorf("invalid number %q: %w", valueStr, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
d, err := durationForUnit(value, unit)
|
||||||
|
if err != nil {
|
||||||
|
return 0, err
|
||||||
|
}
|
||||||
|
|
||||||
|
total += d
|
||||||
|
}
|
||||||
|
|
||||||
|
return total, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// durationForUnit converts a value with a (case-normalized) unit suffix
|
||||||
|
// into a time.Duration, accepting Go's standard units plus the extended
|
||||||
|
// calendar units.
|
||||||
|
func durationForUnit(value float64, unit string) (time.Duration, error) {
|
||||||
|
switch unit {
|
||||||
|
// Standard time units
|
||||||
|
case "ns", "nanosecond", "nanoseconds":
|
||||||
|
return time.Duration(value), nil
|
||||||
|
case "us", "µs", "microsecond", "microseconds":
|
||||||
|
return time.Duration(value * float64(time.Microsecond)), nil
|
||||||
|
case "ms", "millisecond", "milliseconds":
|
||||||
|
return time.Duration(value * float64(time.Millisecond)), nil
|
||||||
|
case "s", "sec", "second", "seconds":
|
||||||
|
return time.Duration(value * float64(time.Second)), nil
|
||||||
|
case "m", "min", "minute", "minutes":
|
||||||
|
return time.Duration(value * float64(time.Minute)), nil
|
||||||
|
case "h", "hr", "hour", "hours":
|
||||||
|
return time.Duration(value * float64(time.Hour)), nil
|
||||||
|
// Extended units
|
||||||
|
case "d", "day", "days":
|
||||||
|
return time.Duration(value * float64(durationDay)), nil
|
||||||
|
case "w", "week", "weeks":
|
||||||
|
return time.Duration(value * float64(durationWeek)), nil
|
||||||
|
case "mo", "month", "months":
|
||||||
|
// Using 30 days as approximation
|
||||||
|
return time.Duration(value * float64(durationMonth)), nil
|
||||||
|
case "y", "year", "years":
|
||||||
|
// Using 365 days as approximation
|
||||||
|
return time.Duration(value * float64(durationYear)), nil
|
||||||
|
default:
|
||||||
|
// Try parsing as standard Go duration unit
|
||||||
|
testStr := "1" + unit
|
||||||
|
|
||||||
|
_, err := time.ParseDuration(testStr)
|
||||||
|
if err != nil {
|
||||||
|
return 0, fmt.Errorf("%w: %q", errUnknownTimeUnit, unit)
|
||||||
|
}
|
||||||
|
|
||||||
|
// It's a valid Go duration unit, parse the full value
|
||||||
|
fullStr := fmt.Sprintf("%g%s", value, unit)
|
||||||
|
|
||||||
|
d, err := time.ParseDuration(fullStr)
|
||||||
|
if err != nil {
|
||||||
|
return 0, fmt.Errorf("invalid duration %q: %w", fullStr, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
return d, nil
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,299 @@
|
|||||||
|
package cli //nolint:testpackage // needs access to unexported parseDuration
|
||||||
|
|
||||||
|
import (
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/stretchr/testify/assert"
|
||||||
|
"github.com/stretchr/testify/require"
|
||||||
|
)
|
||||||
|
|
||||||
|
type parseDurationCase struct {
|
||||||
|
name string
|
||||||
|
input string
|
||||||
|
expected time.Duration
|
||||||
|
wantErr bool
|
||||||
|
}
|
||||||
|
|
||||||
|
// runParseDurationCases executes a table of parseDuration cases as
|
||||||
|
// parallel subtests.
|
||||||
|
func runParseDurationCases(t *testing.T, tests []parseDurationCase) {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
for _, tt := range tests {
|
||||||
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
got, err := parseDuration(tt.input)
|
||||||
|
|
||||||
|
if tt.wantErr {
|
||||||
|
require.Error(t, err, "expected error for input %q", tt.input)
|
||||||
|
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
require.NoError(t, err, "unexpected error for input %q", tt.input)
|
||||||
|
assert.Equal(t, tt.expected, got, "duration mismatch for input %q", tt.input)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestParseDurationStandard(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
runParseDurationCases(t, []parseDurationCase{
|
||||||
|
{
|
||||||
|
name: "standard seconds",
|
||||||
|
input: "30s",
|
||||||
|
expected: 30 * time.Second,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "standard minutes",
|
||||||
|
input: "45m",
|
||||||
|
expected: 45 * time.Minute,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "standard hours",
|
||||||
|
input: "2h",
|
||||||
|
expected: 2 * time.Hour,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "standard combined",
|
||||||
|
input: "3h30m",
|
||||||
|
expected: 3*time.Hour + 30*time.Minute,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "standard complex",
|
||||||
|
input: "1h45m30s",
|
||||||
|
expected: 1*time.Hour + 45*time.Minute + 30*time.Second,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "standard with milliseconds",
|
||||||
|
input: "1s500ms",
|
||||||
|
expected: 1*time.Second + 500*time.Millisecond,
|
||||||
|
},
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestParseDurationExtendedUnits(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
runParseDurationCases(t, []parseDurationCase{
|
||||||
|
// Extended units - days
|
||||||
|
{
|
||||||
|
name: "single day",
|
||||||
|
input: "1d",
|
||||||
|
expected: 24 * time.Hour,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "multiple days",
|
||||||
|
input: "7d",
|
||||||
|
expected: 7 * 24 * time.Hour,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "fractional days",
|
||||||
|
input: "1.5d",
|
||||||
|
expected: 36 * time.Hour,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "days spelled out",
|
||||||
|
input: "3days",
|
||||||
|
expected: 3 * 24 * time.Hour,
|
||||||
|
},
|
||||||
|
// Extended units - weeks
|
||||||
|
{
|
||||||
|
name: "single week",
|
||||||
|
input: "1w",
|
||||||
|
expected: 7 * 24 * time.Hour,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "multiple weeks",
|
||||||
|
input: "4w",
|
||||||
|
expected: 4 * 7 * 24 * time.Hour,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "weeks spelled out",
|
||||||
|
input: "2weeks",
|
||||||
|
expected: 2 * 7 * 24 * time.Hour,
|
||||||
|
},
|
||||||
|
// Extended units - months
|
||||||
|
{
|
||||||
|
name: "single month",
|
||||||
|
input: "1mo",
|
||||||
|
expected: 30 * 24 * time.Hour,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "multiple months",
|
||||||
|
input: "6mo",
|
||||||
|
expected: 6 * 30 * 24 * time.Hour,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "months spelled out",
|
||||||
|
input: "3months",
|
||||||
|
expected: 3 * 30 * 24 * time.Hour,
|
||||||
|
},
|
||||||
|
// Extended units - years
|
||||||
|
{
|
||||||
|
name: "single year",
|
||||||
|
input: "1y",
|
||||||
|
expected: 365 * 24 * time.Hour,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "multiple years",
|
||||||
|
input: "2y",
|
||||||
|
expected: 2 * 365 * 24 * time.Hour,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "years spelled out",
|
||||||
|
input: "1year",
|
||||||
|
expected: 365 * 24 * time.Hour,
|
||||||
|
},
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestParseDurationCombinedAndErrors(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
runParseDurationCases(t, []parseDurationCase{
|
||||||
|
// Combined extended units
|
||||||
|
{
|
||||||
|
name: "weeks and days",
|
||||||
|
input: "2w3d",
|
||||||
|
expected: 2*7*24*time.Hour + 3*24*time.Hour,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "years and months",
|
||||||
|
input: "1y6mo",
|
||||||
|
expected: 365*24*time.Hour + 6*30*24*time.Hour,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "days and hours",
|
||||||
|
input: "1d12h",
|
||||||
|
expected: 24*time.Hour + 12*time.Hour,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "complex combination",
|
||||||
|
input: "1y2mo3w4d5h6m7s",
|
||||||
|
expected: 365*24*time.Hour + 2*30*24*time.Hour +
|
||||||
|
3*7*24*time.Hour + 4*24*time.Hour +
|
||||||
|
5*time.Hour + 6*time.Minute + 7*time.Second,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "with spaces",
|
||||||
|
input: "1d 12h 30m",
|
||||||
|
expected: 24*time.Hour + 12*time.Hour + 30*time.Minute,
|
||||||
|
},
|
||||||
|
// Edge cases
|
||||||
|
{
|
||||||
|
name: "zero duration",
|
||||||
|
input: "0s",
|
||||||
|
expected: 0,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "large duration",
|
||||||
|
input: "10y",
|
||||||
|
expected: 10 * 365 * 24 * time.Hour,
|
||||||
|
},
|
||||||
|
// Error cases
|
||||||
|
{
|
||||||
|
name: "empty string",
|
||||||
|
input: "",
|
||||||
|
wantErr: true,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "invalid format",
|
||||||
|
input: "abc",
|
||||||
|
wantErr: true,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "unknown unit",
|
||||||
|
input: "5x",
|
||||||
|
wantErr: true,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "invalid number",
|
||||||
|
input: "xyzd",
|
||||||
|
wantErr: true,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "negative not supported",
|
||||||
|
input: "-5d",
|
||||||
|
wantErr: true,
|
||||||
|
},
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestParseDurationSpecialCases(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
// Test that standard Go durations work exactly as expected
|
||||||
|
standardDurations := []string{
|
||||||
|
"300ms",
|
||||||
|
"1.5h",
|
||||||
|
"2h45m",
|
||||||
|
"72h",
|
||||||
|
"1us",
|
||||||
|
"1µs",
|
||||||
|
"1ns",
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, d := range standardDurations {
|
||||||
|
expected, err := time.ParseDuration(d)
|
||||||
|
require.NoError(t, err)
|
||||||
|
|
||||||
|
got, err := parseDuration(d)
|
||||||
|
require.NoError(t, err)
|
||||||
|
assert.Equal(t, expected, got, "standard duration %q should parse identically", d)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestParseDurationRealWorldExamples(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
// Test real-world snapshot purge scenarios
|
||||||
|
tests := []struct {
|
||||||
|
description string
|
||||||
|
input string
|
||||||
|
olderThan time.Duration
|
||||||
|
}{
|
||||||
|
{
|
||||||
|
description: "keep snapshots from last 30 days",
|
||||||
|
input: "30d",
|
||||||
|
olderThan: 30 * 24 * time.Hour,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
description: "keep snapshots from last 6 months",
|
||||||
|
input: "6mo",
|
||||||
|
olderThan: 6 * 30 * 24 * time.Hour,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
description: "keep snapshots from last year",
|
||||||
|
input: "1y",
|
||||||
|
olderThan: 365 * 24 * time.Hour,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
description: "keep snapshots from last week and a half",
|
||||||
|
input: "1w3d",
|
||||||
|
olderThan: 10 * 24 * time.Hour,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
description: "keep snapshots from last 90 days",
|
||||||
|
input: "90d",
|
||||||
|
olderThan: 90 * 24 * time.Hour,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, tt := range tests {
|
||||||
|
t.Run(tt.description, func(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
got, err := parseDuration(tt.input)
|
||||||
|
require.NoError(t, err)
|
||||||
|
assert.Equal(t, tt.olderThan, got)
|
||||||
|
|
||||||
|
// Verify the duration makes sense for snapshot purging
|
||||||
|
assert.Greater(t, got, time.Hour,
|
||||||
|
"snapshot purge duration should be at least an hour")
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
+3
-18
@@ -1,7 +1,6 @@
|
|||||||
package cli
|
package cli
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"errors"
|
|
||||||
"io"
|
"io"
|
||||||
"os"
|
"os"
|
||||||
"strings"
|
"strings"
|
||||||
@@ -20,11 +19,7 @@ const shortCommitLen = 12
|
|||||||
// flag is present in os.Args — see bannerSuppressedInArgs), executes the
|
// flag is present in os.Args — see bannerSuppressedInArgs), executes the
|
||||||
// root cobra command, and routes any returned error through the
|
// root cobra command, and routes any returned error through the
|
||||||
// ui.Writer so the user sees a properly formatted "🛑 ERROR:" line.
|
// ui.Writer so the user sees a properly formatted "🛑 ERROR:" line.
|
||||||
//
|
func Entry() {
|
||||||
// It returns the process exit code (0 on success, 1 on error) rather
|
|
||||||
// than calling os.Exit, so that main's deferred profile writers run
|
|
||||||
// before the process ends. See run in cmd/vaultik/main.go.
|
|
||||||
func Entry() int {
|
|
||||||
emitStartupBanner(os.Args[1:], os.Stdout)
|
emitStartupBanner(os.Args[1:], os.Stdout)
|
||||||
|
|
||||||
rootCmd := NewRootCommand()
|
rootCmd := NewRootCommand()
|
||||||
@@ -32,19 +27,9 @@ func Entry() int {
|
|||||||
|
|
||||||
err := rootCmd.Execute()
|
err := rootCmd.Execute()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
// An operation that ran inside the fx app has already reported
|
ReportErrorf("%s", err.Error())
|
||||||
// its own failure (and suppressed it under --json); errReported
|
os.Exit(1)
|
||||||
// says so. Printing it again here would double the error line.
|
|
||||||
// Every other error — bad arguments, a config that would not
|
|
||||||
// load — reaches Entry unreported, so it is shown here.
|
|
||||||
if !errors.Is(err, errReported) {
|
|
||||||
ReportErrorf("%s", err.Error())
|
|
||||||
}
|
|
||||||
|
|
||||||
return 1
|
|
||||||
}
|
}
|
||||||
|
|
||||||
return 0
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// emitStartupBanner writes the startup banner to w unless args (the
|
// emitStartupBanner writes the startup banner to w unless args (the
|
||||||
|
|||||||
@@ -230,7 +230,7 @@ func TestEntryJSONStdoutIsExactlyOneDocument(t *testing.T) {
|
|||||||
programName, flagConfig, configPath, cmdSnapshot, cmdList, flagJSON,
|
programName, flagConfig, configPath, cmdSnapshot, cmdList, flagJSON,
|
||||||
}
|
}
|
||||||
|
|
||||||
stdout := captureProcessStdout(t, func() { _ = Entry() })
|
stdout := captureProcessStdout(t, Entry)
|
||||||
|
|
||||||
requireExactlyOneJSONDocument(t, stdout)
|
requireExactlyOneJSONDocument(t, stdout)
|
||||||
|
|
||||||
|
|||||||
@@ -81,7 +81,7 @@ func TestEntryPruneJSONStdoutIsExactlyOneDocument(t *testing.T) {
|
|||||||
programName, flagConfig, configPath, cmdPrune, flagJSON,
|
programName, flagConfig, configPath, cmdPrune, flagJSON,
|
||||||
}
|
}
|
||||||
|
|
||||||
stdout := captureProcessStdout(t, func() { _ = Entry() })
|
stdout := captureProcessStdout(t, Entry)
|
||||||
|
|
||||||
requireExactlyOneJSONDocument(t, stdout)
|
requireExactlyOneJSONDocument(t, stdout)
|
||||||
|
|
||||||
|
|||||||
@@ -1,58 +0,0 @@
|
|||||||
package cli //nolint:testpackage // shares programName and the capture helpers
|
|
||||||
|
|
||||||
import (
|
|
||||||
"os"
|
|
||||||
"testing"
|
|
||||||
|
|
||||||
"github.com/stretchr/testify/assert"
|
|
||||||
)
|
|
||||||
|
|
||||||
// TestEntryReturnsStatusCode pins the contract main() relies on for
|
|
||||||
// issue #75: Entry reports success or failure through its return value
|
|
||||||
// and never calls os.Exit. An os.Exit from inside Entry would skip
|
|
||||||
// main's deferred profile writers and truncate the profile of a failing
|
|
||||||
// command. main turns this code into os.Exit only after those defers
|
|
||||||
// run, so a failing command must come back with a non-zero code rather
|
|
||||||
// than ending the process here.
|
|
||||||
//
|
|
||||||
// Stdout is captured only to keep the banner and command output off the
|
|
||||||
// test log; the assertion is on the returned code.
|
|
||||||
//
|
|
||||||
//nolint:paralleltest // replaces os.Args and rootFlags
|
|
||||||
func TestEntryReturnsStatusCode(t *testing.T) {
|
|
||||||
for _, testCase := range []struct {
|
|
||||||
name string
|
|
||||||
args []string
|
|
||||||
want int
|
|
||||||
}{
|
|
||||||
{
|
|
||||||
// version is self-contained: it needs no config and no
|
|
||||||
// destination store, so it exercises the success path.
|
|
||||||
name: "successful command returns zero",
|
|
||||||
args: []string{programName, "version"},
|
|
||||||
want: 0,
|
|
||||||
},
|
|
||||||
{
|
|
||||||
name: "unknown command returns one",
|
|
||||||
args: []string{programName, "no-such-command"},
|
|
||||||
want: 1,
|
|
||||||
},
|
|
||||||
} {
|
|
||||||
t.Run(testCase.name, func(t *testing.T) {
|
|
||||||
previousArgs := os.Args
|
|
||||||
|
|
||||||
t.Cleanup(func() {
|
|
||||||
os.Args = previousArgs
|
|
||||||
rootFlags = RootFlags{}
|
|
||||||
})
|
|
||||||
|
|
||||||
os.Args = testCase.args
|
|
||||||
|
|
||||||
var code int
|
|
||||||
|
|
||||||
_ = captureProcessStdout(t, func() { code = Entry() })
|
|
||||||
|
|
||||||
assert.Equal(t, testCase.want, code)
|
|
||||||
})
|
|
||||||
}
|
|
||||||
}
|
|
||||||
+37
-6
@@ -1,7 +1,12 @@
|
|||||||
package cli
|
package cli
|
||||||
|
|
||||||
import (
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"os"
|
||||||
|
|
||||||
"github.com/spf13/cobra"
|
"github.com/spf13/cobra"
|
||||||
|
"go.uber.org/fx"
|
||||||
"sneak.berlin/go/vaultik/internal/log"
|
"sneak.berlin/go/vaultik/internal/log"
|
||||||
"sneak.berlin/go/vaultik/internal/vaultik"
|
"sneak.berlin/go/vaultik/internal/vaultik"
|
||||||
)
|
)
|
||||||
@@ -28,18 +33,44 @@ func NewInfoCommand() *cobra.Command {
|
|||||||
// Use the app framework
|
// Use the app framework
|
||||||
rootFlags := GetRootFlags()
|
rootFlags := GetRootFlags()
|
||||||
|
|
||||||
return RunOperation(cmd.Context(), AppOptions{
|
return RunWithApp(cmd.Context(), AppOptions{
|
||||||
ConfigPath: configPath,
|
ConfigPath: configPath,
|
||||||
LogOptions: log.Options{
|
LogOptions: log.Options{
|
||||||
Verbose: rootFlags.Verbose,
|
Verbose: rootFlags.Verbose,
|
||||||
Debug: rootFlags.Debug,
|
Debug: rootFlags.Debug,
|
||||||
Quiet: rootFlags.Quiet,
|
Quiet: rootFlags.Quiet,
|
||||||
},
|
},
|
||||||
}, func(v *vaultik.Vaultik) error {
|
Modules: []fx.Option{},
|
||||||
return v.ShowInfo()
|
Invokes: []fx.Option{
|
||||||
}, func(err error) {
|
fx.Invoke(func(v *vaultik.Vaultik, lc fx.Lifecycle) {
|
||||||
log.Error("Failed to show info", "error", err)
|
lc.Append(fx.Hook{
|
||||||
ReportErrorf("Failed to show info: %v", err)
|
OnStart: func(_ context.Context) error {
|
||||||
|
go func() {
|
||||||
|
err := v.ShowInfo()
|
||||||
|
if err != nil {
|
||||||
|
if !errors.Is(err, context.Canceled) {
|
||||||
|
log.Error("Failed to show info", "error", err)
|
||||||
|
ReportErrorf("Failed to show info: %v", err)
|
||||||
|
os.Exit(1)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
err = v.Shutdowner.Shutdown()
|
||||||
|
if err != nil {
|
||||||
|
log.Error("Failed to shutdown", "error", err)
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
|
||||||
|
return nil
|
||||||
|
},
|
||||||
|
OnStop: func(_ context.Context) error {
|
||||||
|
v.Cancel()
|
||||||
|
|
||||||
|
return nil
|
||||||
|
},
|
||||||
|
})
|
||||||
|
}),
|
||||||
|
},
|
||||||
})
|
})
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
|
|||||||
+43
-9
@@ -1,7 +1,12 @@
|
|||||||
package cli
|
package cli
|
||||||
|
|
||||||
import (
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"os"
|
||||||
|
|
||||||
"github.com/spf13/cobra"
|
"github.com/spf13/cobra"
|
||||||
|
"go.uber.org/fx"
|
||||||
"sneak.berlin/go/vaultik/internal/log"
|
"sneak.berlin/go/vaultik/internal/log"
|
||||||
"sneak.berlin/go/vaultik/internal/vaultik"
|
"sneak.berlin/go/vaultik/internal/vaultik"
|
||||||
)
|
)
|
||||||
@@ -36,22 +41,51 @@ work (e.g. after a crashed backup or to reclaim storage).`,
|
|||||||
// Use the app framework like other commands
|
// Use the app framework like other commands
|
||||||
rootFlags := GetRootFlags()
|
rootFlags := GetRootFlags()
|
||||||
|
|
||||||
return RunOperation(cmd.Context(), AppOptions{
|
return RunWithApp(cmd.Context(), AppOptions{
|
||||||
ConfigPath: configPath,
|
ConfigPath: configPath,
|
||||||
LogOptions: log.Options{
|
LogOptions: log.Options{
|
||||||
Verbose: rootFlags.Verbose,
|
Verbose: rootFlags.Verbose,
|
||||||
Debug: rootFlags.Debug,
|
Debug: rootFlags.Debug,
|
||||||
Quiet: rootFlags.Quiet || opts.JSON,
|
Quiet: rootFlags.Quiet || opts.JSON,
|
||||||
},
|
},
|
||||||
}, func(v *vaultik.Vaultik) error {
|
Modules: []fx.Option{},
|
||||||
return v.Prune(opts)
|
Invokes: []fx.Option{
|
||||||
}, func(err error) {
|
fx.Invoke(func(v *vaultik.Vaultik, lc fx.Lifecycle) {
|
||||||
if opts.JSON {
|
lc.Append(fx.Hook{
|
||||||
return
|
OnStart: func(_ context.Context) error {
|
||||||
}
|
// Start the prune operation in a goroutine
|
||||||
|
go func() {
|
||||||
|
// Run the prune operation
|
||||||
|
err := v.Prune(opts)
|
||||||
|
if err != nil {
|
||||||
|
if !errors.Is(err, context.Canceled) {
|
||||||
|
if !opts.JSON {
|
||||||
|
log.Error("Prune operation failed", "error", err)
|
||||||
|
ReportErrorf("Prune failed: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
log.Error("Prune operation failed", "error", err)
|
os.Exit(1)
|
||||||
ReportErrorf("Prune failed: %v", err)
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Shutdown the app when prune completes
|
||||||
|
err = v.Shutdowner.Shutdown()
|
||||||
|
if err != nil {
|
||||||
|
log.Error("Failed to shutdown", "error", err)
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
|
||||||
|
return nil
|
||||||
|
},
|
||||||
|
OnStop: func(_ context.Context) error {
|
||||||
|
log.Debug("Stopping prune operation")
|
||||||
|
v.Cancel()
|
||||||
|
|
||||||
|
return nil
|
||||||
|
},
|
||||||
|
})
|
||||||
|
}),
|
||||||
|
},
|
||||||
})
|
})
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
|
|||||||
+37
-9
@@ -1,9 +1,12 @@
|
|||||||
package cli
|
package cli
|
||||||
|
|
||||||
import (
|
import (
|
||||||
|
"context"
|
||||||
"errors"
|
"errors"
|
||||||
|
"os"
|
||||||
|
|
||||||
"github.com/spf13/cobra"
|
"github.com/spf13/cobra"
|
||||||
|
"go.uber.org/fx"
|
||||||
"sneak.berlin/go/vaultik/internal/log"
|
"sneak.berlin/go/vaultik/internal/log"
|
||||||
"sneak.berlin/go/vaultik/internal/vaultik"
|
"sneak.berlin/go/vaultik/internal/vaultik"
|
||||||
)
|
)
|
||||||
@@ -80,22 +83,47 @@ func newRemoteInfoCommand() *cobra.Command {
|
|||||||
|
|
||||||
rootFlags := GetRootFlags()
|
rootFlags := GetRootFlags()
|
||||||
|
|
||||||
return RunOperation(cmd.Context(), AppOptions{
|
return RunWithApp(cmd.Context(), AppOptions{
|
||||||
ConfigPath: configPath,
|
ConfigPath: configPath,
|
||||||
LogOptions: log.Options{
|
LogOptions: log.Options{
|
||||||
Verbose: rootFlags.Verbose,
|
Verbose: rootFlags.Verbose,
|
||||||
Debug: rootFlags.Debug,
|
Debug: rootFlags.Debug,
|
||||||
Quiet: rootFlags.Quiet || jsonOutput,
|
Quiet: rootFlags.Quiet || jsonOutput,
|
||||||
},
|
},
|
||||||
}, func(v *vaultik.Vaultik) error {
|
Modules: []fx.Option{},
|
||||||
return v.RemoteInfo(jsonOutput)
|
Invokes: []fx.Option{
|
||||||
}, func(err error) {
|
fx.Invoke(func(v *vaultik.Vaultik, lc fx.Lifecycle) {
|
||||||
if jsonOutput {
|
lc.Append(fx.Hook{
|
||||||
return
|
OnStart: func(_ context.Context) error {
|
||||||
}
|
go func() {
|
||||||
|
err := v.RemoteInfo(jsonOutput)
|
||||||
|
if err != nil {
|
||||||
|
if !errors.Is(err, context.Canceled) {
|
||||||
|
if !jsonOutput {
|
||||||
|
log.Error("Failed to get remote info", "error", err)
|
||||||
|
ReportErrorf("Failed to get remote info: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
log.Error("Failed to get remote info", "error", err)
|
os.Exit(1)
|
||||||
ReportErrorf("Failed to get remote info: %v", err)
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
err = v.Shutdowner.Shutdown()
|
||||||
|
if err != nil {
|
||||||
|
log.Error("Failed to shutdown", "error", err)
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
|
||||||
|
return nil
|
||||||
|
},
|
||||||
|
OnStop: func(_ context.Context) error {
|
||||||
|
v.Cancel()
|
||||||
|
|
||||||
|
return nil
|
||||||
|
},
|
||||||
|
})
|
||||||
|
}),
|
||||||
|
},
|
||||||
})
|
})
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
|
|||||||
+76
-20
@@ -1,10 +1,13 @@
|
|||||||
package cli
|
package cli
|
||||||
|
|
||||||
import (
|
import (
|
||||||
|
"context"
|
||||||
"errors"
|
"errors"
|
||||||
"fmt"
|
"fmt"
|
||||||
|
"os"
|
||||||
|
|
||||||
"github.com/spf13/cobra"
|
"github.com/spf13/cobra"
|
||||||
|
"go.uber.org/fx"
|
||||||
"sneak.berlin/go/vaultik/internal/log"
|
"sneak.berlin/go/vaultik/internal/log"
|
||||||
"sneak.berlin/go/vaultik/internal/vaultik"
|
"sneak.berlin/go/vaultik/internal/vaultik"
|
||||||
)
|
)
|
||||||
@@ -83,8 +86,7 @@ specifying a path using --config or by setting VAULTIK_CONFIG to a path.`,
|
|||||||
// Use the backup functionality from cli package
|
// Use the backup functionality from cli package
|
||||||
rootFlags := GetRootFlags()
|
rootFlags := GetRootFlags()
|
||||||
|
|
||||||
// --cron suppression is wired through v.UI by setupGlobals.
|
return RunWithApp(cmd.Context(), AppOptions{
|
||||||
return RunOperation(cmd.Context(), AppOptions{
|
|
||||||
ConfigPath: configPath,
|
ConfigPath: configPath,
|
||||||
LogOptions: log.Options{
|
LogOptions: log.Options{
|
||||||
Verbose: rootFlags.Verbose,
|
Verbose: rootFlags.Verbose,
|
||||||
@@ -92,11 +94,42 @@ specifying a path using --config or by setting VAULTIK_CONFIG to a path.`,
|
|||||||
Cron: opts.Cron,
|
Cron: opts.Cron,
|
||||||
Quiet: rootFlags.Quiet,
|
Quiet: rootFlags.Quiet,
|
||||||
},
|
},
|
||||||
}, func(v *vaultik.Vaultik) error {
|
Modules: []fx.Option{},
|
||||||
return v.CreateSnapshot(opts)
|
Invokes: []fx.Option{
|
||||||
}, func(err error) {
|
fx.Invoke(func(v *vaultik.Vaultik, lc fx.Lifecycle) {
|
||||||
log.Error("Snapshot creation failed", "error", err)
|
lc.Append(fx.Hook{
|
||||||
ReportErrorf("Snapshot creation failed: %v", err)
|
OnStart: func(_ context.Context) error {
|
||||||
|
// Start the snapshot creation in a goroutine
|
||||||
|
go func() {
|
||||||
|
// --cron suppression is wired through v.UI by setupGlobals.
|
||||||
|
err := v.CreateSnapshot(opts)
|
||||||
|
if err != nil {
|
||||||
|
if !errors.Is(err, context.Canceled) {
|
||||||
|
log.Error("Snapshot creation failed", "error", err)
|
||||||
|
ReportErrorf("Snapshot creation failed: %v", err)
|
||||||
|
os.Exit(1)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Shutdown the app when snapshot completes
|
||||||
|
err = v.Shutdowner.Shutdown()
|
||||||
|
if err != nil {
|
||||||
|
log.Error("Failed to shutdown", "error", err)
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
|
||||||
|
return nil
|
||||||
|
},
|
||||||
|
OnStop: func(_ context.Context) error {
|
||||||
|
log.Debug("Stopping snapshot creation")
|
||||||
|
// Cancel the Vaultik context
|
||||||
|
v.Cancel()
|
||||||
|
|
||||||
|
return nil
|
||||||
|
},
|
||||||
|
})
|
||||||
|
}),
|
||||||
|
},
|
||||||
})
|
})
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
@@ -108,8 +141,7 @@ specifying a path using --config or by setting VAULTIK_CONFIG to a path.`,
|
|||||||
"orphaned blobs")
|
"orphaned blobs")
|
||||||
cmd.Flags().StringVar(&opts.KeepNewerThan, "keep-newer-than", "",
|
cmd.Flags().StringVar(&opts.KeepNewerThan, "keep-newer-than", "",
|
||||||
"With --prune: keep snapshots newer than this duration "+
|
"With --prune: keep snapshots newer than this duration "+
|
||||||
"(e.g. 30d, 4w, 6mo, 1y; m is minutes, mo is months) "+
|
"(e.g. 4w, 30d, 6mo) instead of only the latest")
|
||||||
"instead of only the latest")
|
|
||||||
|
|
||||||
return cmd
|
return cmd
|
||||||
}
|
}
|
||||||
@@ -172,8 +204,7 @@ restrict the operation to specific snapshot names.`,
|
|||||||
cmd.Flags().BoolVar(&opts.KeepLatest, "keep-latest", false,
|
cmd.Flags().BoolVar(&opts.KeepLatest, "keep-latest", false,
|
||||||
"Keep only the latest snapshot of each name")
|
"Keep only the latest snapshot of each name")
|
||||||
cmd.Flags().StringVar(&opts.OlderThan, "older-than", "",
|
cmd.Flags().StringVar(&opts.OlderThan, "older-than", "",
|
||||||
"Remove snapshots older than duration "+
|
"Remove snapshots older than duration (e.g., 30d, 6m, 1y)")
|
||||||
"(e.g. 30d, 4w, 6mo, 1y; m is minutes, mo is months)")
|
|
||||||
cmd.Flags().BoolVar(&opts.Force, "force", false, "Skip confirmation prompt")
|
cmd.Flags().BoolVar(&opts.Force, "force", false, "Skip confirmation prompt")
|
||||||
cmd.Flags().StringArrayVar(&opts.Names, "snapshot", nil,
|
cmd.Flags().StringArrayVar(&opts.Names, "snapshot", nil,
|
||||||
"Restrict to snapshots with these names (repeat for multiple)")
|
"Restrict to snapshots with these names (repeat for multiple)")
|
||||||
@@ -201,22 +232,47 @@ func newSnapshotVerifyCommand() *cobra.Command {
|
|||||||
|
|
||||||
rootFlags := GetRootFlags()
|
rootFlags := GetRootFlags()
|
||||||
|
|
||||||
return RunOperation(cmd.Context(), AppOptions{
|
return RunWithApp(cmd.Context(), AppOptions{
|
||||||
ConfigPath: configPath,
|
ConfigPath: configPath,
|
||||||
LogOptions: log.Options{
|
LogOptions: log.Options{
|
||||||
Verbose: rootFlags.Verbose,
|
Verbose: rootFlags.Verbose,
|
||||||
Debug: rootFlags.Debug,
|
Debug: rootFlags.Debug,
|
||||||
Quiet: rootFlags.Quiet || opts.JSON,
|
Quiet: rootFlags.Quiet || opts.JSON,
|
||||||
},
|
},
|
||||||
}, func(v *vaultik.Vaultik) error {
|
Modules: []fx.Option{},
|
||||||
return v.VerifySnapshotWithOptions(snapshotID, opts)
|
Invokes: []fx.Option{
|
||||||
}, func(err error) {
|
fx.Invoke(func(v *vaultik.Vaultik, lc fx.Lifecycle) {
|
||||||
if opts.JSON {
|
lc.Append(fx.Hook{
|
||||||
return
|
OnStart: func(_ context.Context) error {
|
||||||
}
|
go func() {
|
||||||
|
err := v.VerifySnapshotWithOptions(snapshotID, opts)
|
||||||
|
if err != nil {
|
||||||
|
if !errors.Is(err, context.Canceled) {
|
||||||
|
if !opts.JSON {
|
||||||
|
log.Error("Verification failed", "error", err)
|
||||||
|
ReportErrorf("Verification failed: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
log.Error("Verification failed", "error", err)
|
os.Exit(1)
|
||||||
ReportErrorf("Verification failed: %v", err)
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
err = v.Shutdowner.Shutdown()
|
||||||
|
if err != nil {
|
||||||
|
log.Error("Failed to shutdown", "error", err)
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
|
||||||
|
return nil
|
||||||
|
},
|
||||||
|
OnStop: func(_ context.Context) error {
|
||||||
|
v.Cancel()
|
||||||
|
|
||||||
|
return nil
|
||||||
|
},
|
||||||
|
})
|
||||||
|
}),
|
||||||
|
},
|
||||||
})
|
})
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1,8 +1,16 @@
|
|||||||
package cli
|
package cli
|
||||||
|
|
||||||
import (
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"os"
|
||||||
|
|
||||||
"github.com/spf13/cobra"
|
"github.com/spf13/cobra"
|
||||||
|
"go.uber.org/fx"
|
||||||
|
"sneak.berlin/go/vaultik/internal/config"
|
||||||
|
"sneak.berlin/go/vaultik/internal/globals"
|
||||||
"sneak.berlin/go/vaultik/internal/log"
|
"sneak.berlin/go/vaultik/internal/log"
|
||||||
|
"sneak.berlin/go/vaultik/internal/storage"
|
||||||
"sneak.berlin/go/vaultik/internal/vaultik"
|
"sneak.berlin/go/vaultik/internal/vaultik"
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -17,6 +25,15 @@ type RestoreOptions struct {
|
|||||||
Verify bool // Verify restored files after restore
|
Verify bool // Verify restored files after restore
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// RestoreApp contains all dependencies needed for restore
|
||||||
|
type RestoreApp struct {
|
||||||
|
Globals *globals.Globals
|
||||||
|
Config *config.Config
|
||||||
|
Storage storage.Storer
|
||||||
|
Vaultik *vaultik.Vaultik
|
||||||
|
Shutdowner fx.Shutdowner
|
||||||
|
}
|
||||||
|
|
||||||
// newSnapshotRestoreCommand creates the 'snapshot restore' subcommand
|
// newSnapshotRestoreCommand creates the 'snapshot restore' subcommand
|
||||||
func newSnapshotRestoreCommand() *cobra.Command {
|
func newSnapshotRestoreCommand() *cobra.Command {
|
||||||
opts := &RestoreOptions{}
|
opts := &RestoreOptions{}
|
||||||
@@ -60,8 +77,7 @@ Examples:
|
|||||||
return cmd
|
return cmd
|
||||||
}
|
}
|
||||||
|
|
||||||
// runRestore parses arguments and runs the restore operation through the
|
// runRestore parses arguments and runs the restore operation through the app framework
|
||||||
// app framework.
|
|
||||||
func runRestore(cmd *cobra.Command, args []string, opts *RestoreOptions) error {
|
func runRestore(cmd *cobra.Command, args []string, opts *RestoreOptions) error {
|
||||||
snapshotID := args[0]
|
snapshotID := args[0]
|
||||||
|
|
||||||
@@ -70,30 +86,87 @@ func runRestore(cmd *cobra.Command, args []string, opts *RestoreOptions) error {
|
|||||||
opts.Paths = args[restoreMinArgs:]
|
opts.Paths = args[restoreMinArgs:]
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Use unified config resolution
|
||||||
configPath, err := ResolveConfigPath()
|
configPath, err := ResolveConfigPath()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Use the app framework like other commands
|
||||||
rootFlags := GetRootFlags()
|
rootFlags := GetRootFlags()
|
||||||
|
|
||||||
return RunOperation(cmd.Context(), AppOptions{
|
return RunWithApp(cmd.Context(), AppOptions{
|
||||||
ConfigPath: configPath,
|
ConfigPath: configPath,
|
||||||
LogOptions: log.Options{
|
LogOptions: log.Options{
|
||||||
Verbose: rootFlags.Verbose,
|
Verbose: rootFlags.Verbose,
|
||||||
Debug: rootFlags.Debug,
|
Debug: rootFlags.Debug,
|
||||||
Quiet: rootFlags.Quiet,
|
Quiet: rootFlags.Quiet,
|
||||||
},
|
},
|
||||||
}, func(v *vaultik.Vaultik) error {
|
Modules: buildRestoreModules(),
|
||||||
return v.Restore(&vaultik.RestoreOptions{
|
Invokes: buildRestoreInvokes(snapshotID, opts),
|
||||||
SnapshotID: snapshotID,
|
|
||||||
TargetDir: opts.TargetDir,
|
|
||||||
Paths: opts.Paths,
|
|
||||||
Verify: opts.Verify,
|
|
||||||
SkipErrors: rootFlags.SkipErrors,
|
|
||||||
})
|
|
||||||
}, func(err error) {
|
|
||||||
log.Error("Restore operation failed", "error", err)
|
|
||||||
ReportErrorf("Restore failed: %v", err)
|
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// buildRestoreModules returns the fx.Options for dependency injection in restore
|
||||||
|
func buildRestoreModules() []fx.Option {
|
||||||
|
return []fx.Option{
|
||||||
|
fx.Provide(fx.Annotate(
|
||||||
|
func(g *globals.Globals, cfg *config.Config,
|
||||||
|
storer storage.Storer, v *vaultik.Vaultik, shutdowner fx.Shutdowner) *RestoreApp {
|
||||||
|
return &RestoreApp{
|
||||||
|
Globals: g,
|
||||||
|
Config: cfg,
|
||||||
|
Storage: storer,
|
||||||
|
Vaultik: v,
|
||||||
|
Shutdowner: shutdowner,
|
||||||
|
}
|
||||||
|
},
|
||||||
|
)),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// buildRestoreInvokes returns the fx.Options that wire up the restore lifecycle
|
||||||
|
func buildRestoreInvokes(snapshotID string, opts *RestoreOptions) []fx.Option {
|
||||||
|
return []fx.Option{
|
||||||
|
fx.Invoke(func(app *RestoreApp, lc fx.Lifecycle) {
|
||||||
|
lc.Append(fx.Hook{
|
||||||
|
OnStart: func(_ context.Context) error {
|
||||||
|
// Start the restore operation in a goroutine
|
||||||
|
go func() {
|
||||||
|
// Run the restore operation
|
||||||
|
restoreOpts := &vaultik.RestoreOptions{
|
||||||
|
SnapshotID: snapshotID,
|
||||||
|
TargetDir: opts.TargetDir,
|
||||||
|
Paths: opts.Paths,
|
||||||
|
Verify: opts.Verify,
|
||||||
|
SkipErrors: GetRootFlags().SkipErrors,
|
||||||
|
}
|
||||||
|
|
||||||
|
err := app.Vaultik.Restore(restoreOpts)
|
||||||
|
if err != nil {
|
||||||
|
if !errors.Is(err, context.Canceled) {
|
||||||
|
log.Error("Restore operation failed", "error", err)
|
||||||
|
ReportErrorf("Restore failed: %v", err)
|
||||||
|
os.Exit(1)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Shutdown the app when restore completes
|
||||||
|
err = app.Shutdowner.Shutdown()
|
||||||
|
if err != nil {
|
||||||
|
log.Error("Failed to shutdown", "error", err)
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
|
||||||
|
return nil
|
||||||
|
},
|
||||||
|
OnStop: func(_ context.Context) error {
|
||||||
|
log.Debug("Stopping restore operation")
|
||||||
|
app.Vaultik.Cancel()
|
||||||
|
|
||||||
|
return nil
|
||||||
|
},
|
||||||
|
})
|
||||||
|
}),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
+5
-13
@@ -219,7 +219,11 @@ func (c *Client) HeadObject(ctx context.Context, key string) (bool, error) {
|
|||||||
Key: aws.String(fullKey),
|
Key: aws.String(fullKey),
|
||||||
})
|
})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
if IsNotFound(err) {
|
var (
|
||||||
|
notFound *s3types.NotFound
|
||||||
|
noSuchKey *s3types.NoSuchKey
|
||||||
|
)
|
||||||
|
if errors.As(err, ¬Found) || errors.As(err, &noSuchKey) {
|
||||||
return false, nil
|
return false, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -229,18 +233,6 @@ func (c *Client) HeadObject(ctx context.Context, key string) (bool, error) {
|
|||||||
return true, nil
|
return true, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
// IsNotFound reports whether err indicates that an object does not exist.
|
|
||||||
// Head and Get requests surface a missing object as different SDK types,
|
|
||||||
// so both are checked here.
|
|
||||||
func IsNotFound(err error) bool {
|
|
||||||
var (
|
|
||||||
notFound *s3types.NotFound
|
|
||||||
noSuchKey *s3types.NoSuchKey
|
|
||||||
)
|
|
||||||
|
|
||||||
return errors.As(err, ¬Found) || errors.As(err, &noSuchKey)
|
|
||||||
}
|
|
||||||
|
|
||||||
// ObjectInfo contains information about an S3 object.
|
// ObjectInfo contains information about an S3 object.
|
||||||
// It is used by ListObjectsStream to return object metadata
|
// It is used by ListObjectsStream to return object metadata
|
||||||
// along with any errors encountered during listing.
|
// along with any errors encountered during listing.
|
||||||
|
|||||||
+54
-79
@@ -46,18 +46,31 @@ func (f *FileStorer) SetFilesystem(fs afero.Fs) {
|
|||||||
// storage base path.
|
// storage base path.
|
||||||
const storageDirPerm = 0o755
|
const storageDirPerm = 0o755
|
||||||
|
|
||||||
// tempSuffix marks a partially written object. writeAtomic streams into a
|
|
||||||
// temp file carrying this suffix and only renames it onto the real key once
|
|
||||||
// the whole object is on disk, so an interrupted write can never leave a
|
|
||||||
// truncated object at the key a later run would Stat and trust as a complete
|
|
||||||
// blob. List and ListStream skip these files, so a leftover from an
|
|
||||||
// interrupted write is never listed or trusted as a blob; it is otherwise
|
|
||||||
// harmless and is overwritten when the same key is written again.
|
|
||||||
const tempSuffix = ".partial"
|
|
||||||
|
|
||||||
// Put stores data at the specified key.
|
// Put stores data at the specified key.
|
||||||
func (f *FileStorer) Put(_ context.Context, key string, data io.Reader) error {
|
func (f *FileStorer) Put(_ context.Context, key string, data io.Reader) error {
|
||||||
return f.writeAtomic(key, data, nil)
|
path := f.fullPath(key)
|
||||||
|
|
||||||
|
// Create parent directories
|
||||||
|
dir := filepath.Dir(path)
|
||||||
|
|
||||||
|
err := f.fs.MkdirAll(dir, storageDirPerm)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("creating directories: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
file, err := f.fs.Create(path)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("creating file: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
defer func() { _ = file.Close() }()
|
||||||
|
|
||||||
|
_, err = io.Copy(file, data)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("writing file: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
// PutWithProgress stores data with progress reporting.
|
// PutWithProgress stores data with progress reporting.
|
||||||
@@ -65,7 +78,35 @@ func (f *FileStorer) PutWithProgress(
|
|||||||
_ context.Context, key string, data io.Reader,
|
_ context.Context, key string, data io.Reader,
|
||||||
_ int64, progress ProgressCallback,
|
_ int64, progress ProgressCallback,
|
||||||
) error {
|
) error {
|
||||||
return f.writeAtomic(key, data, progress)
|
path := f.fullPath(key)
|
||||||
|
|
||||||
|
// Create parent directories
|
||||||
|
dir := filepath.Dir(path)
|
||||||
|
|
||||||
|
err := f.fs.MkdirAll(dir, storageDirPerm)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("creating directories: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
file, err := f.fs.Create(path)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("creating file: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
defer func() { _ = file.Close() }()
|
||||||
|
|
||||||
|
// Wrap with progress tracking
|
||||||
|
pw := &progressWriter{
|
||||||
|
writer: file,
|
||||||
|
callback: progress,
|
||||||
|
}
|
||||||
|
|
||||||
|
_, err = io.Copy(pw, data)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("writing file: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
// Get retrieves data from the specified key.
|
// Get retrieves data from the specified key.
|
||||||
@@ -147,7 +188,7 @@ func (f *FileStorer) List(ctx context.Context, prefix string) ([]string, error)
|
|||||||
default:
|
default:
|
||||||
}
|
}
|
||||||
|
|
||||||
if !info.IsDir() && !strings.HasSuffix(info.Name(), tempSuffix) {
|
if !info.IsDir() {
|
||||||
// Convert back to key (relative path from basePath)
|
// Convert back to key (relative path from basePath)
|
||||||
relPath, err := filepath.Rel(f.basePath, path)
|
relPath, err := filepath.Rel(f.basePath, path)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -204,7 +245,7 @@ func (f *FileStorer) ListStream(ctx context.Context, prefix string) <-chan Objec
|
|||||||
return nil //nolint:nilerr // continue walking despite errors
|
return nil //nolint:nilerr // continue walking despite errors
|
||||||
}
|
}
|
||||||
|
|
||||||
if !info.IsDir() && !strings.HasSuffix(info.Name(), tempSuffix) {
|
if !info.IsDir() {
|
||||||
relPath, err := filepath.Rel(f.basePath, path)
|
relPath, err := filepath.Rel(f.basePath, path)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
ch <- ObjectInfo{Err: fmt.Errorf("computing relative path: %w", err)}
|
ch <- ObjectInfo{Err: fmt.Errorf("computing relative path: %w", err)}
|
||||||
@@ -234,72 +275,6 @@ func (f *FileStorer) Info() Info {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// writeAtomic streams data into a temp file in the destination directory,
|
|
||||||
// fsyncs it, and renames it onto the final key. The key therefore appears
|
|
||||||
// only once the whole object has been durably written; a failure part-way
|
|
||||||
// leaves a temp file (removed here on the failing path) rather than a
|
|
||||||
// truncated object at the key.
|
|
||||||
func (f *FileStorer) writeAtomic(
|
|
||||||
key string, data io.Reader, progress ProgressCallback,
|
|
||||||
) error {
|
|
||||||
path := f.fullPath(key)
|
|
||||||
dir := filepath.Dir(path)
|
|
||||||
|
|
||||||
err := f.fs.MkdirAll(dir, storageDirPerm)
|
|
||||||
if err != nil {
|
|
||||||
return fmt.Errorf("creating directories: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
tmp, err := afero.TempFile(f.fs, dir, filepath.Base(path)+"-*"+tempSuffix)
|
|
||||||
if err != nil {
|
|
||||||
return fmt.Errorf("creating temp file: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
tmpPath := tmp.Name()
|
|
||||||
|
|
||||||
// Remove the temp file unless the rename below claims it. On the success
|
|
||||||
// path renamed is true, so the deferred Close and Remove are harmless
|
|
||||||
// no-ops on a name that no longer exists.
|
|
||||||
renamed := false
|
|
||||||
|
|
||||||
defer func() {
|
|
||||||
_ = tmp.Close()
|
|
||||||
|
|
||||||
if !renamed {
|
|
||||||
_ = f.fs.Remove(tmpPath)
|
|
||||||
}
|
|
||||||
}()
|
|
||||||
|
|
||||||
var w io.Writer = tmp
|
|
||||||
if progress != nil {
|
|
||||||
w = &progressWriter{writer: tmp, callback: progress}
|
|
||||||
}
|
|
||||||
|
|
||||||
_, err = io.Copy(w, data)
|
|
||||||
if err != nil {
|
|
||||||
return fmt.Errorf("writing file: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
err = tmp.Sync()
|
|
||||||
if err != nil {
|
|
||||||
return fmt.Errorf("syncing temp file: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
err = tmp.Close()
|
|
||||||
if err != nil {
|
|
||||||
return fmt.Errorf("closing temp file: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
err = f.fs.Rename(tmpPath, path)
|
|
||||||
if err != nil {
|
|
||||||
return fmt.Errorf("renaming temp file: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
renamed = true
|
|
||||||
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// fullPath returns the full filesystem path for a key.
|
// fullPath returns the full filesystem path for a key.
|
||||||
func (f *FileStorer) fullPath(key string) string {
|
func (f *FileStorer) fullPath(key string) string {
|
||||||
return filepath.Join(f.basePath, key)
|
return filepath.Join(f.basePath, key)
|
||||||
|
|||||||
@@ -1,119 +0,0 @@
|
|||||||
package storage_test
|
|
||||||
|
|
||||||
import (
|
|
||||||
"context"
|
|
||||||
"errors"
|
|
||||||
"os"
|
|
||||||
"path/filepath"
|
|
||||||
"strings"
|
|
||||||
"testing"
|
|
||||||
|
|
||||||
"sneak.berlin/go/vaultik/internal/storage"
|
|
||||||
)
|
|
||||||
|
|
||||||
// errStreamInterrupted stands in for an upload cut off mid-stream.
|
|
||||||
var errStreamInterrupted = errors.New("connection reset mid-upload")
|
|
||||||
|
|
||||||
// failingReader yields its data once, then fails.
|
|
||||||
type failingReader struct {
|
|
||||||
data []byte
|
|
||||||
done bool
|
|
||||||
}
|
|
||||||
|
|
||||||
func (r *failingReader) Read(p []byte) (int, error) {
|
|
||||||
if r.done {
|
|
||||||
return 0, errStreamInterrupted
|
|
||||||
}
|
|
||||||
|
|
||||||
n := copy(p, r.data)
|
|
||||||
r.done = true
|
|
||||||
|
|
||||||
return n, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestFileStorer_InterruptedWriteLeavesNoTrustedObject checks that a write
|
|
||||||
// cut off mid-stream leaves nothing at the destination key, so a later run
|
|
||||||
// cannot Stat a truncated object and trust it as a complete blob.
|
|
||||||
func TestFileStorer_InterruptedWriteLeavesNoTrustedObject(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
f, err := storage.NewFileStorer(t.TempDir())
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("NewFileStorer: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
ctx := context.Background()
|
|
||||||
key := "blobs/aa/bb/aabbccddeeff"
|
|
||||||
|
|
||||||
err = f.PutWithProgress(ctx, key, &failingReader{data: []byte("partial")}, 4096, nil)
|
|
||||||
if err == nil {
|
|
||||||
t.Fatal("expected the interrupted write to fail, got nil")
|
|
||||||
}
|
|
||||||
|
|
||||||
_, err = f.Stat(ctx, key)
|
|
||||||
if !errors.Is(err, storage.ErrNotFound) {
|
|
||||||
t.Fatalf("expected key absent after interrupted write, got Stat err %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
keys, err := f.List(ctx, "blobs/")
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("List: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
if len(keys) != 0 {
|
|
||||||
t.Fatalf("expected no keys listed after interrupted write, got %v", keys)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestFileStorer_ListSkipsPartialFiles checks that a leftover temp file (the
|
|
||||||
// storage layer names them with a ".partial" suffix) is never surfaced as a
|
|
||||||
// key by List or ListStream.
|
|
||||||
func TestFileStorer_ListSkipsPartialFiles(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
base := t.TempDir()
|
|
||||||
|
|
||||||
f, err := storage.NewFileStorer(base)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("NewFileStorer: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
ctx := context.Background()
|
|
||||||
realKey := "blobs/aa/bb/aabbccddeeff"
|
|
||||||
|
|
||||||
err = f.Put(ctx, realKey, strings.NewReader("blob-bytes"))
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("Put: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
// A stray temp file, as an interrupted write would leave behind.
|
|
||||||
leftover := filepath.Join(base, "blobs/aa/bb/aabbccddeeff-123456.partial")
|
|
||||||
|
|
||||||
err = os.WriteFile(leftover, []byte("half"), 0o600)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("writing leftover temp file: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
keys, err := f.List(ctx, "blobs/")
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("List: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
if len(keys) != 1 || keys[0] != realKey {
|
|
||||||
t.Fatalf("List should return only the real key, got %v", keys)
|
|
||||||
}
|
|
||||||
|
|
||||||
var streamed []string
|
|
||||||
|
|
||||||
for obj := range f.ListStream(ctx, "blobs/") {
|
|
||||||
if obj.Err != nil {
|
|
||||||
t.Fatalf("ListStream: %v", obj.Err)
|
|
||||||
}
|
|
||||||
|
|
||||||
streamed = append(streamed, obj.Key)
|
|
||||||
}
|
|
||||||
|
|
||||||
if len(streamed) != 1 || streamed[0] != realKey {
|
|
||||||
t.Fatalf("ListStream should return only the real key, got %v", streamed)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
+1
-16
@@ -38,29 +38,14 @@ func (s *S3Storer) PutWithProgress(
|
|||||||
}
|
}
|
||||||
|
|
||||||
// Get retrieves data from the specified key.
|
// Get retrieves data from the specified key.
|
||||||
// Returns ErrNotFound if the object does not exist.
|
|
||||||
func (s *S3Storer) Get(ctx context.Context, key string) (io.ReadCloser, error) {
|
func (s *S3Storer) Get(ctx context.Context, key string) (io.ReadCloser, error) {
|
||||||
rc, err := s.client.GetObject(ctx, key)
|
return s.client.GetObject(ctx, key)
|
||||||
if err != nil {
|
|
||||||
if s3.IsNotFound(err) {
|
|
||||||
return nil, fmt.Errorf("get %q: %w", key, ErrNotFound)
|
|
||||||
}
|
|
||||||
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
|
|
||||||
return rc, nil
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// Stat returns metadata about an object without retrieving its contents.
|
// Stat returns metadata about an object without retrieving its contents.
|
||||||
// Returns ErrNotFound if the object does not exist.
|
|
||||||
func (s *S3Storer) Stat(ctx context.Context, key string) (*ObjectInfo, error) {
|
func (s *S3Storer) Stat(ctx context.Context, key string) (*ObjectInfo, error) {
|
||||||
info, err := s.client.StatObject(ctx, key)
|
info, err := s.client.StatObject(ctx, key)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
if s3.IsNotFound(err) {
|
|
||||||
return nil, fmt.Errorf("stat %q: %w", key, ErrNotFound)
|
|
||||||
}
|
|
||||||
|
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -1,59 +0,0 @@
|
|||||||
package storage_test
|
|
||||||
|
|
||||||
import (
|
|
||||||
"context"
|
|
||||||
"errors"
|
|
||||||
"net/http/httptest"
|
|
||||||
"testing"
|
|
||||||
|
|
||||||
"github.com/johannesboyne/gofakes3"
|
|
||||||
"github.com/johannesboyne/gofakes3/backend/s3mem"
|
|
||||||
|
|
||||||
"sneak.berlin/go/vaultik/internal/s3"
|
|
||||||
"sneak.berlin/go/vaultik/internal/storage"
|
|
||||||
)
|
|
||||||
|
|
||||||
// TestS3StorerMissingKeyMapsToErrNotFound verifies that the s3 backend reports
|
|
||||||
// a missing object as storage.ErrNotFound, matching the file and rclone
|
|
||||||
// backends and the Storer contract. Without the mapping, Get and Stat leak the
|
|
||||||
// raw SDK error and errors.Is(err, storage.ErrNotFound) is false.
|
|
||||||
//
|
|
||||||
//nolint:paralleltest // shares an in-process S3 server via t.Cleanup
|
|
||||||
func TestS3StorerMissingKeyMapsToErrNotFound(t *testing.T) {
|
|
||||||
const bucket = "test-bucket"
|
|
||||||
|
|
||||||
backend := s3mem.New()
|
|
||||||
|
|
||||||
err := backend.CreateBucket(bucket)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("create bucket: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
srv := httptest.NewServer(gofakes3.New(backend).Server())
|
|
||||||
t.Cleanup(srv.Close)
|
|
||||||
|
|
||||||
ctx := context.Background()
|
|
||||||
|
|
||||||
client, err := s3.NewClient(ctx, s3.Config{
|
|
||||||
Endpoint: srv.URL,
|
|
||||||
Bucket: bucket,
|
|
||||||
AccessKeyID: "test",
|
|
||||||
SecretAccessKey: "test",
|
|
||||||
Region: "us-east-1",
|
|
||||||
})
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("new client: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
storer := storage.NewS3Storer(client)
|
|
||||||
|
|
||||||
_, err = storer.Get(ctx, "does-not-exist")
|
|
||||||
if !errors.Is(err, storage.ErrNotFound) {
|
|
||||||
t.Errorf("Get on missing key: got %v, want ErrNotFound", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
_, err = storer.Stat(ctx, "does-not-exist")
|
|
||||||
if !errors.Is(err, storage.ErrNotFound) {
|
|
||||||
t.Errorf("Stat on missing key: got %v, want ErrNotFound", err)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -33,9 +33,8 @@ func ubytes(n int64) string {
|
|||||||
var (
|
var (
|
||||||
errMalformedSnapshotID = errors.New(
|
errMalformedSnapshotID = errors.New(
|
||||||
"invalid snapshot ID format: expected hostname_snapshotname_timestamp")
|
"invalid snapshot ID format: expected hostname_snapshotname_timestamp")
|
||||||
errInvalidDuration = errors.New("invalid duration")
|
errInvalidDuration = errors.New("invalid duration")
|
||||||
errUnknownTimeUnit = errors.New("unknown time unit")
|
errUnknownTimeUnit = errors.New("unknown time unit")
|
||||||
errNegativeDuration = errors.New("negative durations are not supported")
|
|
||||||
)
|
)
|
||||||
|
|
||||||
// Time-unit lengths used by parseDuration.
|
// Time-unit lengths used by parseDuration.
|
||||||
@@ -139,13 +138,8 @@ func parseSnapshotName(snapshotID string) string {
|
|||||||
|
|
||||||
// parseDuration parses a duration string with support for human-friendly units:
|
// parseDuration parses a duration string with support for human-friendly units:
|
||||||
// d/day/days, w/week/weeks, mo/month/months, y/year/years, plus standard Go
|
// d/day/days, w/week/weeks, mo/month/months, y/year/years, plus standard Go
|
||||||
// duration units. Following Go, m is minutes and mo is months. A bare number,
|
// duration units (h, m, s).
|
||||||
// an unknown unit, and a negative value are all rejected.
|
|
||||||
func parseDuration(s string) (time.Duration, error) {
|
func parseDuration(s string) (time.Duration, error) {
|
||||||
if strings.HasPrefix(strings.TrimSpace(s), "-") {
|
|
||||||
return 0, errNegativeDuration
|
|
||||||
}
|
|
||||||
|
|
||||||
d, err := time.ParseDuration(s)
|
d, err := time.ParseDuration(s)
|
||||||
if err == nil {
|
if err == nil {
|
||||||
return d, nil
|
return d, nil
|
||||||
|
|||||||
@@ -51,32 +51,13 @@ func TestParseDuration(t *testing.T) {
|
|||||||
want time.Duration
|
want time.Duration
|
||||||
err bool
|
err bool
|
||||||
}{
|
}{
|
||||||
// Go units, including the m-is-minutes / mo-is-months distinction
|
|
||||||
// that this parser exists to keep straight.
|
|
||||||
{"10ns", 10 * time.Nanosecond, false},
|
|
||||||
{"10us", 10 * time.Microsecond, false},
|
|
||||||
{"500ms", 500 * time.Millisecond, false},
|
|
||||||
{"30s", 30 * time.Second, false},
|
|
||||||
{"6m", 6 * time.Minute, false},
|
|
||||||
{"1h", time.Hour, false},
|
|
||||||
// Extended calendar units.
|
|
||||||
{"30d", 30 * 24 * time.Hour, false},
|
{"30d", 30 * 24 * time.Hour, false},
|
||||||
{"3days", 3 * 24 * time.Hour, false},
|
|
||||||
{"4w", 4 * 7 * 24 * time.Hour, false},
|
{"4w", 4 * 7 * 24 * time.Hour, false},
|
||||||
{"2weeks", 2 * 7 * 24 * time.Hour, false},
|
{"6mo", 6 * 30 * 24 * time.Hour, false},
|
||||||
{"6mo", 180 * 24 * time.Hour, false},
|
|
||||||
{"1month", 30 * 24 * time.Hour, false},
|
|
||||||
{"1y", 365 * 24 * time.Hour, false},
|
{"1y", 365 * 24 * time.Hour, false},
|
||||||
{"2years", 2 * 365 * 24 * time.Hour, false},
|
|
||||||
// Combined units.
|
|
||||||
{"2w3d", 2*7*24*time.Hour + 3*24*time.Hour, false},
|
{"2w3d", 2*7*24*time.Hour + 3*24*time.Hour, false},
|
||||||
{"1y6mo", 365*24*time.Hour + 180*24*time.Hour, false},
|
{"1h", time.Hour, false},
|
||||||
// Rejected inputs.
|
{"30s", 30 * time.Second, false},
|
||||||
{"6", 0, true}, // bare number, no unit
|
|
||||||
{"5x", 0, true}, // unknown unit
|
|
||||||
{"-5d", 0, true}, // negative, extended unit
|
|
||||||
{"-5h", 0, true}, // negative, Go unit
|
|
||||||
{"", 0, true}, // empty
|
|
||||||
{"garbage", 0, true},
|
{"garbage", 0, true},
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
+1
-16
@@ -56,23 +56,8 @@ main() {
|
|||||||
docker build --output=type=cacheonly \
|
docker build --output=type=cacheonly \
|
||||||
--build-arg CHECK_EPOCH="$epoch" -f Dockerfile.lint .
|
--build-arg CHECK_EPOCH="$epoch" -f Dockerfile.lint .
|
||||||
|
|
||||||
# Version, commit and build date are computed here on the host, the
|
|
||||||
# same way script/docker does, and passed into the product build so
|
|
||||||
# the CI-built image reports its real source. The build context
|
|
||||||
# excludes .git (see .dockerignore), so the build cannot derive them
|
|
||||||
# itself; without these it would stamp the Dockerfile's dev/unknown
|
|
||||||
# fallbacks. VERSION comes from script/version, the source of truth
|
|
||||||
# shared with the Makefile.
|
|
||||||
version="$("$ROOT/script/version")"
|
|
||||||
commit="$(git rev-parse HEAD 2>/dev/null || echo unknown)"
|
|
||||||
commit_date="$(git show -s --format=%cs HEAD 2>/dev/null || echo unknown)"
|
|
||||||
|
|
||||||
epoch="$(date +%s%N)$$"
|
epoch="$(date +%s%N)$$"
|
||||||
docker build --build-arg CHECK_EPOCH="$epoch" \
|
docker build --build-arg CHECK_EPOCH="$epoch" .
|
||||||
--build-arg VERSION="$version" \
|
|
||||||
--build-arg COMMIT="$commit" \
|
|
||||||
--build-arg COMMIT_DATE="$commit_date" \
|
|
||||||
.
|
|
||||||
}
|
}
|
||||||
|
|
||||||
main "$@"
|
main "$@"
|
||||||
|
|||||||
@@ -24,24 +24,7 @@ main() {
|
|||||||
# whether the tree is clean. The Dockerfile now refuses to build
|
# whether the tree is clean. The Dockerfile now refuses to build
|
||||||
# without a non-empty value, so this is required, not optional.
|
# without a non-empty value, so this is required, not optional.
|
||||||
epoch="$(date +%s%N)$$"
|
epoch="$(date +%s%N)$$"
|
||||||
|
|
||||||
# Version, commit and build date are computed here on the host,
|
|
||||||
# where .git exists, and passed into the build. The build context
|
|
||||||
# excludes .git (see .dockerignore), so the container cannot derive
|
|
||||||
# them itself -- it used to try and always got "unknown", giving
|
|
||||||
# every image a "commit: unknown" it could not be traced from.
|
|
||||||
# VERSION comes from script/version, the source of truth shared with
|
|
||||||
# the Makefile, so a Docker build reports the same string (tag,
|
|
||||||
# dev-<sha>, or a -dirty variant) that a local build of the same
|
|
||||||
# tree would.
|
|
||||||
version="$("$SCRIPT_DIR/version")"
|
|
||||||
commit="$(git rev-parse HEAD 2>/dev/null || echo unknown)"
|
|
||||||
commit_date="$(git show -s --format=%cs HEAD 2>/dev/null || echo unknown)"
|
|
||||||
|
|
||||||
docker build --build-arg CHECK_EPOCH="$epoch" \
|
docker build --build-arg CHECK_EPOCH="$epoch" \
|
||||||
--build-arg VERSION="$version" \
|
|
||||||
--build-arg COMMIT="$commit" \
|
|
||||||
--build-arg COMMIT_DATE="$commit_date" \
|
|
||||||
-t "$("$SCRIPT_DIR/projectname")" .
|
-t "$("$SCRIPT_DIR/projectname")" .
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user