739de1e1014db613f3a936b83c44abd32474a05a
29
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 clawbot
|
696ed9ab4d |
Gate prune's local-cleanup output on --json, and make make build build (closes #108)
check / check (push) Successful in 2m16s
Closes #110. CleanupLocalSnapshots wrote three prose lines to stdout with no --json awareness, covering every branch, so `vaultik prune --json | jq` failed on any input. -q never helped either: printlnStdout and stdoutf write straight to v.Stdout and never consult v.UI, which is what SetQuiet affects. It now takes *PruneOptions, symmetric with its sibling phase PruneBlobs, and gates all three writes. Threading opts.JSON was chosen over moving the lines to log.Info, because internal/log/log.go defaults the level to Warn: log.Info would not have relocated them to stderr, it would have deleted them from a plain `vaultik prune`, and "Removing stale local record" narrates the deletion of local index rows. The stale-record count is deliberately not added to PruneBlobsResult - every field there is blob-scoped and produced by the phase that runs after this reconciliation, so adding it would change a published --json schema as a side effect of a stream fix. Note for anyone reading the --json contract: under --json the stale-record removal now produces no signal in either stream. stdout is correctly gated, stderr is level-pinned to Warn because --json sets Quiet, and the count is not in the document. That is inherited behaviour - PruneBlobs' own log.Info calls are equally invisible under --json - not something this change introduced, and it is tracked separately. make build exited 0 and produced nothing: .PHONY listed build with no build: rule, and a phony target with no prerequisites and no recipe is considered already satisfied, which turns what would be a hard error into a silent success. In a repo where `make build` is the documented way to build, a caller checking the exit code concluded the build worked. Now `build: vaultik`, verified in both directions - a clean build produces the binary, a deliberately broken one exits non-zero and produces none. All 19 .PHONY names were audited; build was the only one lacking a rule. TestPhonyTargetsAllHaveRules keeps that true for names added later, so the class is closed rather than the instance. |
||
|
clawbot
|
f21e7c9e70 |
Suppress the startup banner for --json (closes #106)
check / check (push) Successful in 2m31s
The banner is printed to stdout before cobra parses, and bannerSuppressedInArgs recognised only --quiet, -q and --cron. So every --json document was preceded by two banner lines and a blank one, and `vaultik snapshot list --json | jq` failed. Passing opts.JSON as extraQuiet did not help: that calls UI.SetQuiet in an fx OnStart hook, long after Entry has printed. The raw-argv scan is extended rather than the banner moved after parsing. root.go documents that the banner must survive cobra rejecting its arguments and --help, and no single post-parse location covers those paths. The subcommand-versus-persistent distinction does not decide it: --cron is already in the suppression list and is itself subcommand-only, existing on snapshot create alone, so this adds another instance of an accepted imprecision rather than a new kind. The error directions are asymmetric - a false positive loses a decorative banner, a false negative corrupts a document - so the scan errs toward suppression, which is also why --json=false suppresses, exactly as --quiet=false already does. Four of the five --json commands now pipe into jq cleanly with no other flags: snapshot list, snapshot verify, snapshot remove, remote info. prune does not, because pruneLocalSnapshots writes three prose lines to stdout with no --json awareness. That reproduces identically before this change and -q never suppressed it either, since printlnStdout and stdoutf bypass v.UI entirely. Tracked as #108. Also fixed: TTYHandler's human-readable byte formatting did not survive grouping, because the key check compared against the bare attribute name and a grouped record presents it qualified. AGENTS.md policy 9 keyed the log format on stdout's TTY-ness, which #82 made false by moving the logger to stderr; it now names the log stream. Vaultik.Stderr keeps its field with the comment amended to say outright that nothing writes to it, and the dead listEnv.stderr is removed. |
||
|
clawbot
|
c16ef476a9 |
Log to stderr and stop discarding With attributes (closes #82)
check / check (push) Successful in 4m20s
Closes #97. internal/log attached both handlers to os.Stdout, so any record that was not suppressed landed in the middle of a --json document. WARN and ERROR are never suppressed, so this was not hypothetical: a config file with permissions looser than 0600 was enough to break `vaultik snapshot list --json | jq`. Both handlers now write to os.Stderr, and the TTY-vs-JSON format choice tests os.Stderr rather than os.Stdout - the format has to follow the stream the records land on, or a redirected stderr gets colorized whenever stdout happens to be a terminal. User-visible: --verbose and --debug output moves to stderr too, so `vaultik snapshot list -v > out.txt` no longer captures diagnostics. --quiet and --cron semantics are unchanged. TTYHandler.WithAttrs and WithGroup discarded their arguments and returned the receiver, while their doc comments claimed otherwise, so attributes passed through the exported log.With vanished. The effect was environment-dependent in the worst direction: handler choice is by TTY-ness, so attributes disappeared on a terminal - where a developer is debugging - and appeared correctly in CI. Both now return a new handler with copied state rather than mutating the receiver, since slog permits a handler to be shared and derived from concurrently. A test asserts the TTY and JSON handlers emit the same attribute set, which is the test that would have caught the original defect. The local workaround in snapshot_list.go is removed now that the logger no longer writes to stdout. The collect-then-emit machinery is kept, but for a different reason than it was added: emitting from the fetch workers would order warnings by network timing, whereas key-order emission after group.Wait() is deterministic run to run. Not yet complete: --json stdout still carries the startup banner, which internal/cli/entry.go writes before cobra parses and which bannerSuppressedInArgs does not recognise --json for. That is the remaining stdout contamination path and is tracked in #106. |
||
|
clawbot
|
e3f407b440 |
Make the tagged-release path work on Gitea (closes #65)
check / check (push) Successful in 3m7s
No tag could be cut at all: .goreleaser.yaml had no gitea_urls block, so
goreleaser defaulted to the GitHub API, and the repo has zero tags.
.goreleaser.yaml now points at git.eeqj.de. Version derives from git via
a new script/version - exact tag with any leading v stripped, else
dev-<12-char sha>, with a -dirty suffix when tracked files are modified -
replacing the hardcoded 1.0.0-rc.1 that every local build was stamping
regardless of git state. A tag-triggered .gitea/workflows/release.yml
runs goreleaser with a scoped token (RELEASE_TOKEN); script/bootstrap
installs a sha256-verified goreleaser, and make release / release-snapshot
become script shims like every other target.
Two fabrications were removed rather than merely replaced. goreleaser's
snapshot.version_template was `{{ incpatch .Version }}-next`, which
invents a release number from the last tag - and with no tags, from
goreleaser's own fabricated v0.0.0. And internal/cli/version.go gated its
development-build notice on Version == "dev" exactly, so the moment
untagged builds carried a sha that notice would have gone silent and an
unreleased binary would have read as a release. Replaced with a tested
IsDevVersion predicate, and closed at both layers: the Makefile now
refuses to build when script/version yields nothing, and an empty version
counts as a development build - reachable today via
`docker build --build-arg VERSION=`.
The release workflow installs Go from a sha-pinned actions/setup-go
(v5.6.0) using go-version-file, so the compiler that produces released
binaries is pinned like every other external reference. Without it the
first tag push would either fail at goreleaser's before-hook or compile
the published artifacts with whatever unpinned Go the runner happened to
carry - the one unpinned thing in a release path that already refuses an
unpinned goreleaser.
Known gap: the Go tarball setup-go fetches is version-pinned but not
checksum-verified against a value in this repo, unlike the goreleaser
install and the Dockerfile digest.
|
||
|
clawbot
|
b6e4a218a3 |
Isolate the lint cache and context-gate the native path (closes #99)
check / check (push) Successful in 2m23s
Closes #80. script/lint pointed GOLANGCI_LINT_CACHE at a path shared by every worktree of this repo. Two worktrees have identical Go file contents, so their cache keys collided and one tree's stored findings replayed for another, paths included - observed as 231 findings all citing another session's worktree, with no parallel-runner message to signal it. The failure is symmetric and only one direction is loud: a clean tree failed by a dirty sibling gets investigated, a dirty tree passed by a clean sibling does not. The cache is now keyed per worktree on a digest of $ROOT, and remains persistent. Independently of that, script/lint-audit inspects every run's output and fails the run if any finding cites a path outside the tree being linted. That guard is the load-bearing part: it converts a silent unearned green into a hard error regardless of how the cache is keyed. It is deliberately built so it can never certify a pass, only reject, so it cannot itself become a gate that reports green. The native path was gated on version equality alone, which admitted a locally installed matching binary and bypassed the digest pin. It now requires VAULTIK_LINT_IN_CONTAINER=1, set only by the Dockerfile lint stage, in addition to version equality. /.dockerenv was rejected as the signal because dockerd creates it for `docker run` but not reliably during a BuildKit `docker build`, which is the case the exception exists for. A version mismatch inside the container is now a hard error rather than a fall-through. This mattered more than the issue supposed: on this host a matching golangci-lint exists on PATH, so script/lint was taking the native path and linting against the global cache without ever running the pinned image. That is the likely root of the observed contamination, and it is closed here rather than mitigated. The parallel-runner error is retried rather than reported. It is not a lint result, and surfacing it as a non-zero exit is indistinguishable to a caller from real findings; exhausted retries fail saying the tree was never analysed. Note that a private cache alone does not remove lock contention - measured with two concurrent runs using separate cache directories. script/bootstrap no longer reports success on a machine that cannot run the gate: docker is now required by lint, check and precommit, so a missing binary or unreachable daemon is a hard failure naming what will not work. |
||
|
clawbot
|
c51f693527 |
Make the test gate unfakeable and stop test-integration lying (closes #93)
check / check (push) Successful in 3m42s
Closes #69. script/test ran `go test` without -count=1, so Go's test cache satisfied the gate without running anything: a repeat `make test` printed all 14 ok lines in 0.42 seconds, every one marked (cached). Those lines count as ok lines, so the evidence signal this repo relies on was forgeable. It sits below the Docker layer cache - CHECK_EPOCH forces `RUN make test` to re-execute, but a GOCACHE baked into an earlier image layer survives into the re-executed step, so the step can run and still do no work. -count=1 is applied unconditionally rather than only in the container, because the pre-commit hook runs the same script and a gate honest only in CI is dishonest where it is leaned on most. It costs about 11 seconds on every repeat run, which is what it costs for a repeat run to mean anything. test-coverage had the same omission and is fixed too; a coverage profile assembled from cached results describes a run that did not happen. Both invocations in script/test now share one run_tests function so the quiet run and the verbose rerun cannot drift apart in flags. make test-integration passed -tags=integration while no file in the repo carries any build tag, so it was an exact duplicate of make test. Removed rather than given a tag scheme: the whole suite is 12s on the host, so gating saves seconds in exchange for a mechanism whose failure mode is "some tests silently stopped running" - a poor trade in a repo that has found several ways for a gate to report an unearned green. -timeout goes 30s to 120s. This DIVERGES from REPO_POLICIES.md:192, which mandates 30s; the divergence is deliberate, recorded in script/test's comment, and proposed upstream as #101. Measured worst case is 10.2s and each fresh measurement has come in above the last, leaving 30s at 2.9x - too thin for a loaded runner. A -timeout is a hang backstop, not a performance budget. Note for the record: cold-cache compilation is NOT charged against -timeout. The flag reaches the test binary as -test.timeout and its clock starts inside testing.M.Run, after compilation. Verified twice independently - a run with an empty GOCACHE spent ~46s compiling and then reported per-package durations within noise of warm. A shell `timeout 30 go test ./...` does include compilation, but that is a different mechanism. |
||
|
clawbot
|
3f9c2e5033 |
Record the stale-branch triage and advance TODO.md (closes #71)
check / check (push) Successful in 2m5s
Twelve stale remote branches retired. Nine were ancestors of main with no unique commits; golangci-v2.12.2 pointed at a tree byte-identical to the one main's |
||
|
clawbot
|
50816b7415 |
Make a missing CHECK_EPOCH fail the build (closes #91)
check / check (push) Successful in 3m2s
PR #89 stopped script/cibuild replaying cached check layers, but left a gap: a bare `docker build .` with no --build-arg still faked. An unset ARG is an empty string, an empty string is a stable cache key, and the check layers replay from it. That gap mattered because REPO_POLICIES.md names `docker build .` verbatim as a command that must be green, so the documented command was the one that lied. Both check stages now carry `RUN [ -n "$CHECK_EPOCH" ] || exit 1` immediately under their own ARG. Failed steps are never cached, so this fails on every invocation rather than once - a bare build now stops with a named error instead of reporting a green it did not earn. Each stage needs its own guard because ARG scope is per-stage; a gate-carrying stage without one is a silent hole if ordering ever changes. The check RUNs now reference the value (`echo "check epoch: ${CHECK_EPOCH}" && make <target>`), so the cache miss is contractual rather than resting on BuildKit's current treatment of unreferenced ARGs, and the epoch is visible in the build log. The epoch becomes "$(date +%s%N)$$" so concurrent invocations in the same second cannot collide. busybox silently drops %N and exits 0, so $$ is what makes it correct there. The bare-assignment form is retained deliberately: inlining the substitution into --build-arg would, under set -eu, yield an empty and therefore constant epoch without aborting. script/docker gets the same treatment - it is not the gate, but two entrypoints disagreeing about whether the tree is green is its own hazard, and local builds are almost always warm. Verified by negative control rather than inspection: a bare build fails twice consecutively here and succeeds twice on the parent commit, so the change is demonstrably not a no-op. The builder-stage guard was fired directly with a targeted probe build, since the lint stage otherwise fails first and would leave it unexercised. |
||
|
clawbot
|
c3bb3b5580 |
Make script/cibuild unable to report an unearned green (closes #85)
check / check (push) Successful in 3m13s
script/cibuild was a bare `docker build .`. On an unchanged tree Docker served the check RUN layers from cache, so make fmt-check, make lint and make test never executed - and the build still exited 0. Measured at 221ms with zero ok lines and every check layer CACHED, against 162s for a real run. CI showed the same signature: 6 second "successes" on main. An ARG CHECK_EPOCH now sits immediately above the check RUNs in both stages - each stage declares its own, since ARG scope is per-stage - and script/cibuild passes a fresh value per invocation. Dependency and module layers sit above the ARG and still cache, so this does not make every build cold. The epoch is assigned before the build rather than inlined into the --build-arg. Under `set -eu` a command substitution that fails inside an argument does not abort the script: CHECK_EPOCH would become an empty string, an empty string is a constant, a constant CHECK_EPOCH restores the cached false green, and the guard would silently disarm itself while still exiting 0. As a bare assignment, set -e catches a failing date and no build starts. The README and Dockerfile state the guarantee conditionally. It holds per build context and CHECK_EPOCH value, and depends on script/cibuild passing a fresh one - a bare `docker build .` with no --build-arg still replays the check layers from the second consecutive run onward. That residual gap is tracked in #91 along with the remaining upstream hardening. Verification is recorded once, in the PR's verification comment, rather than restated with differing numbers in three places. |
||
|
clawbot
|
3bcdbcfd83 |
Correct what --cron actually suppresses (closes #84)
check / check (push) Successful in 6s
The Vaultik.UI doc comment claimed the cli layer replaces the writer with
a discarding writer in --cron mode. It does not. UI is built once as
ui.New(os.Stdout) and never reassigned; internal/cli/app.go calls
UI.SetQuiet(true) when --cron or --quiet is set, which drops Begin,
Complete, Info, Notice, Detail, Progress and Banner - but Warningf and
Errorf have no quiet check and are still emitted.
That distinction matters: the end-of-run summary is deliberately routed
through UI.Warningf so cron delivers something, so a reader who believed
the comment would have concluded the opposite of how the code is meant to
work.
The README's --cron description carried the same imprecision ("Silent
unless error") and is corrected alongside it.
Comment and documentation only - the Go diff contains no non-comment
lines, so there is no behavior change.
|
||
|
clawbot
|
50e20b460e |
List remote snapshots without requiring the private key (closes #64)
check / check (push) Successful in 6s
ListSnapshots built its table entirely from the local SQLite index. The only remote access, reportRemoteDrift, was gated on AgeSecretKey != "", so on a correctly configured host - which by design holds no private key - snapshot list never contacted the destination store at all. A user who lost their local index could not see their own backups, and the "<remote only>" cell the README documents was unreachable dead code. The listing is now the union of the local index and the destination store, with no age_secret_key gate. Remote-only snapshots cannot have their hostname or name recovered - RemoteSnapshotKey is one-way and the manifest stores the hash - so they are listed by abbreviated remote key with the real timestamp and compressed size from the manifest, and "<remote only>" in the two columns that require the local index. Nothing new is written to remote storage and the human ID is never fabricated. An unreachable destination degrades to local-only with a warning and a zero exit code. remote_present is null rather than false in that case, so "absent" and "unknown" stay distinguishable and no drift is claimed from a listing that never happened. Also: - Snapshot timestamps are normalised to UTC in scanSnapshotRows, the single point where they enter the domain. Previously one of three scanners omitted .UTC(), so on a non-UTC host the same snapshot rendered a different time depending on whether it was locally tracked. - The 1000-row cap and the unreadable-manifest count are reported in --json mode as well as table mode, so machine consumers cannot be silently truncated. The JSON shape is unchanged. - Warnings raised while listing are routed to stderr rather than the logger, which writes to stdout and would corrupt the JSON document. This is a local workaround for the logger bug tracked in #82 and should be removed when that lands. - downloadManifestByKey is now the only remote manifest reader, so the manifest privacy question in #81 has a single call site to change. - The orphaned "vaultik snapshot cleanup" hint now names vaultik prune; that command was folded into prune by the 2026-07-02 consolidation. |
||
|
clawbot
|
af607e3597 |
Run the linter at the pinned version locally too (closes #78)
check / check (push) Successful in 6s
script/lint ran bare golangci-lint from PATH while CI and the Dockerfile pinned v2.12.2 by digest, so make lint and CI could disagree about findings. That drift ran both directions: it produced two false green claims during the lint remediation, and on an ambient 2.10.1 it also reported four gosec findings on a tree CI linted clean. script/lint now extracts the image reference - tag and digest - from the Dockerfile lint stage FROM line and runs that exact image under docker. The Dockerfile FROM line is the single source of truth for the linter version; the duplicate pins in the Makefile deps target and in script/bootstrap are removed rather than kept in sync. A golangci-lint on PATH is used only when its version exactly equals the pin, which is what makes the in-container lint stage work (the Dockerfile runs make lint inside the pinned image, where there is no docker daemon). Any other version, or none, goes through docker. When docker is unavailable the script fails with an actionable message and never falls back to a different linter version. script/lint-fix delegates to script/lint --fix so autofixes come from the pinned linter too. The container mounts persistent build and module caches and runs as the invoking uid/gid. Verified by reinstating the four historical nolint directives that 2.10.1 requires and 2.12.2 reports as unused: the old script passed on that tree and the new one fails with four nolintlint findings. |
||
|
clawbot
|
e496aa334b |
Finish the lint remediation: script/cibuild exits 0 (closes #61)
check / check (push) Successful in 5s
Clears the final 80 golangci-lint findings under the canonical .golangci.yml (sha256 021cc83f4e6fc7c31b95b34b846723dfcf20b66b7baeea1dc40406e643346bcb), taking the repo from red to green: script/cibuild exits 0. - wsl_v5 (60): blank line above defer/go statements sharing no variable with the line above; blank-line-only diff. - sqlclosecheck (10): the package-local CloseRows helper hid the close from the analyzer. Helper removed; all 18 call sites now defer an inline rows.Close(), preserving the fatal-on-close-error path. No resource leak existed - the rows were always being closed. - prealloc (3): append targets given a starting capacity. - revive (3): package-name findings suppressed with per-site directives pending the naming decision tracked in #76. No gosec suppressions are needed under the pinned linter. .golangci.yml, Dockerfile, Makefile, .gitea/ and script/ are byte-identical to main. Verified with script/cibuild (digest-pinned golangci-lint v2.12.2), not make check - the latter resolves the linter from PATH and is not a trustworthy gate here; see #78. Closes #59. |
||
 clawbotandsneak
|
cc58583130 |
Update golangci-lint to v2.12.2 with canonical config (#62)
check / check (push) Successful in 5s
Updates golangci-lint to v2.12.2 everywhere it is pinned and installs the canonical `.golangci.yml`, then remediates every finding the new linter/config surfaces so `make check` is green. ## Version bump - `Dockerfile` lint stage: `golangci/golangci-lint:v2.11.3-alpine` -> `v2.12.2-alpine` (digest-pinned, date comment updated) - `Makefile` `deps` target: `go install` moved from the old v1 module path at `@latest` to the pinned `github.com/golangci/golangci-lint/v2/cmd/golangci-lint@v2.12.2` - `.golangci.yml` replaced with the canonical config (v2 schema; settings under `linters.settings` so the thresholds actually apply; `default: all` with the standard six disables) - `script/bootstrap` installs golangci-lint via the system package manager and carries no version pin, so it is unchanged - CI (`.gitea/workflows/check.yml`) only runs `script/cibuild`, so it needed no change ## Lint remediation The canonical config surfaced ~3,300 findings across 56k lines. All are fixed, behavior-preserving; incorporates and supersedes the per-package mechanical passes already merged to `main` (refs #61). Highlights: - `err113`: dynamic errors replaced with package sentinels + `%w` wrapping; comparisons via `errors.Is` - `goprintffuncname`: printf-style helpers renamed with an `f` suffix (`ui.Writer` message methods, `cli.ReportErrorf`, `database.Fatalf`) and all call sites updated - `revive` stutter renames: `blob.Handler`, `blob.WithReader`, `blob.ChunkPosition`, `storage.URL`, `storage.Info`; missing doc comments added - `contextcheck`/`noctx`: `context.Context` threaded through `blob.Packer` and the scanner call sites; context-aware `exec`/`sql` variants - `funlen`/`cyclop`/`gocognit`/`dupl`: oversized and duplicated functions split into focused helpers (production and test code) - tests: `t.Parallel()` added where safe (global logger init kept in the serial phase for `-race`), `t.TempDir()`/`t.Helper()` adopted, several suites converted to external test packages - `gosec`: bounded integer conversions, `ReadHeaderTimeout` on the test HTTP server; remaining warnings suppressed per-site with justifications - remaining `nolint` directives are rare, targeted, and each carries a reason (e.g. `nilnil` not-found contract in the repository layer, fx module globals, on-disk snake_case struct tags) - removed the deprecated `log.LogOptions` alias (callers migrated to `log.Options`) `make check` (tests with `-race`, lint, fmt-check) passes. Co-authored-by: sneak <sneak@sneak.berlin> Reviewed-on: #62 Co-authored-by: clawbot <clawbot@noreply.example.org> Co-committed-by: clawbot <clawbot@noreply.example.org> |
||
|
clawbotandsneak
|
928c389a5a |
Add .editorconfig (refs #59) (#60)
check / check (push) Successful in 4s
Adds `.editorconfig`, copied byte-for-byte from `sneak/dnswatcher` (blob `2fe0ce0`). This is the small, safe half of #59; the `.golangci.yml` half is deferred — adopting the org-standard config surfaces ~2,990 lint findings on vaultik and needs a separate lint-cleanup decision (see #59). Hence `refs #59`, not `closes`. `make check` and `docker build .` are green (a static config file does not affect them). Left open for review (not merged). Co-authored-by: sneak <sneak@sneak.berlin> Reviewed-on: #60 Co-authored-by: clawbot <clawbot@noreply.example.org> Co-committed-by: clawbot <clawbot@noreply.example.org> |
||
 clawbotandclawbot
|
18c14d1507 |
Move schema_migrations table creation into 000.sql with INTEGER version column (#58)
check / check (push) Successful in 2m25s
Closes #57 Adopts the [pixa migration pattern](sneak/pixa#36) for schema management. Replaces the monolithic `schema.sql` embed with a numbered migration system. ## Changes ### New: `schema/000.sql` — Bootstrap migration - Creates `schema_migrations` table with `INTEGER PRIMARY KEY` version column - Self-contained: includes both `CREATE TABLE IF NOT EXISTS` and `INSERT OR IGNORE` for version 0 - Go code does zero INSERTs for bootstrap — just reads and executes 000.sql ### Renamed: `schema.sql` → `schema/001.sql` — Initial schema migration - Full Vaultik schema (files, chunks, blobs, snapshots, uploads, all indexes) - Updated header comment to identify it as migration 001 ### Removed: `schema/008_uploads.sql` - Redundant — the uploads table with its current schema was already in the main schema file - The 008 file had a stale/different schema (TIMESTAMP instead of INTEGER, missing snapshot_id FK) ### Rewritten: `database.go` — Migration engine - `//go:embed schema/*.sql` replaces `//go:embed schema.sql` - `bootstrapMigrationsTable()`: checks if `schema_migrations` table exists, applies 000.sql if missing - `applyMigrations()`: iterates through numbered .sql files, checks `schema_migrations` for each version, applies and records pending ones - `collectMigrations()`: reads embedded schema dir, returns sorted filenames - `ParseMigrationVersion()`: extracts numeric version from filenames like `001.sql` or `001_description.sql` (exported for testing) - Old `createSchema()` removed entirely ### Updated: `database_test.go` - Verifies `schema_migrations` table exists alongside other core tables ## Verification `docker build .` passes — formatting, linting, all tests green. Co-authored-by: clawbot <clawbot@noreply.git.eeqj.de> Reviewed-on: #58 Co-authored-by: clawbot <clawbot@noreply.example.org> Co-committed-by: clawbot <clawbot@noreply.example.org> |
||
|
|
65da291ddf |
feat: per-name purge filtering for snapshot purge (#51)
check / check (push) Successful in 4m50s
## Summary `PurgeSnapshots` now applies `--keep-latest` retention per snapshot name instead of globally across all names. ### Problem Previously, `--keep-latest` would keep only the single most recent snapshot across ALL snapshot names. For example, with snapshots: - `system_2024-01-15` - `home_2024-01-14` - `system_2024-01-13` `--keep-latest` would keep only `system_2024-01-15` and delete the latest `home` snapshot too. ### Solution 1. **Per-name retention**: `--keep-latest` now groups snapshots by name and keeps the latest of each group. In the example above, both `system_2024-01-15` and `home_2024-01-14` would be kept. 2. **`--name` flag**: New flag to filter purge operations to a specific snapshot name. `--name home --keep-latest` only purges `home` snapshots, leaving all `system` snapshots untouched. ### Changes - `internal/vaultik/helpers.go`: Add `parseSnapshotName()` to extract the snapshot name from a snapshot ID (`hostname_name_timestamp` format) - `internal/vaultik/snapshot.go`: Add `SnapshotPurgeOptions` struct with `Name` field, add `PurgeSnapshotsWithOptions()` method, modify `--keep-latest` logic to group by name - `internal/cli/purge.go` and `internal/cli/snapshot.go`: Add `--name` flag to both purge CLI surfaces - `README.md`: Update CLI documentation ### Tests - `helpers_test.go`: Unit tests for `parseSnapshotName()` and `parseSnapshotTimestamp()` - `purge_per_name_test.go`: Integration tests covering: - Per-name retention with multiple names - Single-name retention - `--name` filter with `--keep-latest` - `--name` filter with `--older-than` - No-match name filter (all snapshots retained) - Legacy snapshots without name component - Mixed named and legacy snapshots - Three different snapshot names ### Backward Compatibility The existing `PurgeSnapshots(keepLatest, olderThan, force)` signature is preserved as a wrapper around the new `PurgeSnapshotsWithOptions()`. The `--prune` flag in `snapshot create` continues to work unchanged. `docker build .` passes (lint, fmt-check, all tests). closes [#9](#9) Co-authored-by: user <user@Mac.lan guest wan> Co-authored-by: clawbot <clawbot@noreply.git.eeqj.de> Reviewed-on: #51 Co-authored-by: clawbot <clawbot@noreply.example.org> Co-committed-by: clawbot <clawbot@noreply.example.org> |
||
|
clawbotanduser
|
dcf3ec399a |
feat: concurrent manifest downloads in ListSnapshots (#50)
check / check (push) Successful in 2m27s
## Summary Replace serial `getManifestSize()` calls in `ListSnapshots` with bounded concurrent downloads using `errgroup`. For each remote snapshot not in the local DB, manifest downloads now run in parallel (up to 10 concurrent goroutines) instead of one at a time. ## Changes - Use `errgroup` with `SetLimit(10)` for bounded concurrency - Collect remote-only snapshot IDs first, pre-add entries with zero size - Download manifests concurrently, patch sizes from results - Remove now-unused `getManifestSize` helper (logic inlined into goroutines) - Promote `golang.org/x/sync` from indirect to direct dependency ## Testing - `make check` passes (fmt-check, lint, tests) - `docker build .` passes closes #8 Co-authored-by: user <user@Mac.lan guest wan> Reviewed-on: #50 Co-authored-by: clawbot <clawbot@noreply.example.org> Co-committed-by: clawbot <clawbot@noreply.example.org> |
||
|
clawbotandclawbot
|
495dede1bc |
fix: replace O(n²) duplicate detection with map-based O(1) lookups (#45)
check / check (push) Has been cancelled
Replace linear scan deduplication of snapshot IDs in `RemoveAllSnapshots()` and `PruneBlobs()` with `map[string]bool` for O(1) lookups. Previously, each new snapshot ID was checked against the entire collected slice via a linear scan, resulting in O(n²) overall complexity. Now a `seen` map provides constant-time membership checks while preserving insertion order in the slice. **Changes:** - `internal/vaultik/snapshot.go` (`RemoveAllSnapshots`): replaced linear `for` loop dedup with `seen` map - `internal/vaultik/prune.go` (`PruneBlobs`): replaced linear `for` loop dedup with `seen` map closes #12 Co-authored-by: clawbot <clawbot@noreply.git.eeqj.de> Reviewed-on: #45 Co-authored-by: clawbot <clawbot@noreply.example.org> Co-committed-by: clawbot <clawbot@noreply.example.org> |
||
|
clawbotanduser
|
1c72a37bc8 |
Remove all ctime usage and storage (#55)
check / check (push) Successful in 5s
Remove all ctime from the codebase per sneak's decision on [PR #48](#48). ## Rationale - ctime means different things on macOS (birth time) vs Linux (inode change time) — ambiguous cross-platform - Vaultik never uses ctime operationally (scanning triggers on mtime change) - Cannot be restored on either platform - Write-only forensic data with no consumer ## Changes - **Schema** (`internal/database/schema.sql`): Removed `ctime` column from `files` table - **Model** (`internal/database/models.go`): Removed `CTime` field from `File` struct - **Database layer** (`internal/database/files.go`): Removed ctime from all INSERT/SELECT queries, ON CONFLICT updates, and scan targets in both `scanFile` and `scanFileRows` helpers; updated `CreateBatch` accordingly - **Scanner** (`internal/snapshot/scanner.go`): Removed `CTime: info.ModTime()` assignment in `checkFileInMemory()` - **Tests**: Removed all `CTime` field assignments from 8 test files - **Documentation**: Removed ctime references from `ARCHITECTURE.md` and `docs/DATAMODEL.md` `docker build .` passes clean (lint, fmt-check, all tests). closes #54 Co-authored-by: user <user@Mac.lan guest wan> Reviewed-on: #55 Co-authored-by: clawbot <clawbot@noreply.example.org> Co-committed-by: clawbot <clawbot@noreply.example.org> |
||
|
clawbotanduser
|
60b6746db9 |
schema: add ON DELETE CASCADE to snapshot_files.file_id and snapshot_blobs.blob_id FKs (#46)
check / check (push) Successful in 2m47s
Add `ON DELETE CASCADE` to the two foreign keys that were missing it: - `snapshot_files.file_id` → `files(id)` - `snapshot_blobs.blob_id` → `blobs(id)` This ensures that when a file or blob row is deleted, the corresponding snapshot junction rows are automatically cleaned up, consistent with the other CASCADE FKs already in the schema. closes #19 Co-authored-by: user <user@Mac.lan guest wan> Reviewed-on: #46 Co-authored-by: clawbot <clawbot@noreply.example.org> Co-committed-by: clawbot <clawbot@noreply.example.org> |
||
|
clawbotandclawbot
|
f28c8a73b7 |
fix: add ON DELETE CASCADE to uploads FK on snapshot_id (#44)
check / check (push) Successful in 2m24s
The `uploads` table's foreign key on `snapshot_id` did not cascade deletes, unlike `snapshot_files` and `snapshot_blobs`. This caused FK violations when deleting snapshots with associated upload records (if FK enforcement is enabled) unless uploads were manually deleted first. Adds `ON DELETE CASCADE` to the `snapshot_id` FK in `schema.sql` for consistency with the other snapshot-referencing tables. `docker build .` passes (fmt-check, lint, all tests, build). closes #18 Co-authored-by: clawbot <clawbot@noreply.git.eeqj.de> Reviewed-on: #44 Co-authored-by: clawbot <clawbot@noreply.example.org> Co-committed-by: clawbot <clawbot@noreply.example.org> |
||
|
clawbotanduser
|
1c0f5b8eb2 |
Rename blob_fetch_stub.go to blob_fetch.go (#53)
check / check (push) Successful in 4m28s
Renames `internal/vaultik/blob_fetch_stub.go` to `internal/vaultik/blob_fetch.go`. The file contains production code (`hashVerifyReader`, `FetchAndDecryptBlob`), not stubs. The `_stub` suffix was a misnomer from the original implementation in [PR #39](#39). Pure rename — no code changes. All tests, linting, and formatting pass. closes #52 Co-authored-by: user <user@Mac.lan guest wan> Reviewed-on: #53 Co-authored-by: clawbot <clawbot@noreply.example.org> Co-committed-by: clawbot <clawbot@noreply.example.org> |
||
|
clawbotandclawbot
|
689109a2b8 |
fix: remove destructive sync from ListSnapshots (#49)
check / check (push) Has been cancelled
## Summary `ListSnapshots()` silently deleted local snapshot records not found in remote storage. A list/read operation should not have destructive side effects. ## Changes 1. **Removed destructive sync from `ListSnapshots()`** — the inline loop that deleted local snapshots not present in remote storage has been removed entirely. `ListSnapshots()` now only reads and displays data. 2. **Improved `syncWithRemote()` cascade cleanup** — updated `syncWithRemote()` to use `deleteSnapshotFromLocalDB()` instead of directly calling `Repositories.Snapshots.Delete()`. This ensures proper cascade deletion of related records (`snapshot_files`, `snapshot_blobs`, `snapshot_uploads`) before deleting the snapshot record itself, matching the thorough cleanup that the removed `ListSnapshots` code was doing. The explicit sync behavior remains available via `syncWithRemote()`, which is called by `PurgeSnapshots()`. ## Testing - `docker build .` passes (lint, fmt-check, all tests, compilation) closes #15 Co-authored-by: clawbot <clawbot@eeqj.de> Reviewed-on: #49 Co-authored-by: clawbot <clawbot@noreply.example.org> Co-committed-by: clawbot <clawbot@noreply.example.org> |
||
|
clawbotanduser
|
ac2f21a89d |
Refactor: break up oversized methods into smaller descriptive helpers (#41)
check / check (push) Successful in 4m17s
Closes #40 Per sneak's feedback on PR #37: methods were too long. This PR breaks all methods over 100-150 lines into smaller, descriptively named helper methods. ## Refactored methods (8 total) | Original | Lines | Helpers extracted | |---|---|---| | `createNamedSnapshot` | 214 | `resolveSnapshotPaths`, `scanAllDirectories`, `collectUploadStats`, `finalizeSnapshotMetadata`, `printSnapshotSummary`, `getSnapshotBlobSizes`, `formatUploadSpeed` | | `ListSnapshots` | 159 | `listRemoteSnapshotIDs`, `reconcileLocalWithRemote`, `buildSnapshotInfoList`, `printSnapshotTable` | | `PruneBlobs` | 170 | `collectReferencedBlobs`, `listUniqueSnapshotIDs`, `listAllRemoteBlobs`, `findUnreferencedBlobs`, `deleteUnreferencedBlobs` | | `RunDeepVerify` | 182 | `loadVerificationData`, `runVerificationSteps`, `deepVerifyFailure` | | `RemoteInfo` | 187 | `collectSnapshotMetadata`, `collectReferencedBlobsFromManifests`, `populateRemoteInfoResult`, `scanRemoteBlobStorage`, `printRemoteInfoTable` | | `handleBlobReady` | 173 | `uploadBlobIfNeeded`, `makeUploadProgressCallback`, `recordBlobMetadata`, `cleanupBlobTempFile` | | `processFileStreaming` | 146 | `updateChunkStats`, `addChunkToPacker`, `queueFileForBatchInsert` | | `finalizeCurrentBlob` | 167 | `closeBlobWriter`, `buildChunkRefs`, `commitBlobToDatabase`, `deliverFinishedBlob` | ## Verification - `go build ./...` ✅ - `make test` ✅ (all tests pass) - `golangci-lint run` ✅ (0 issues) - No behavioral changes, pure restructuring Co-authored-by: user <user@Mac.lan guest wan> Reviewed-on: #41 Co-authored-by: clawbot <clawbot@noreply.example.org> Co-committed-by: clawbot <clawbot@noreply.example.org> |
||
|
|
7a5943958d |
feat: add progress bar to restore operation (#23)
Add an interactive progress bar (using schollz/progressbar) to the file restore loop, matching the existing pattern in verify. Shows bytes restored with ETA when output is a terminal. Fixes #20 Co-authored-by: clawbot <clawbot@eeqj.de> Co-authored-by: clawbot <clawbot@noreply.git.eeqj.de> Reviewed-on: #23 Co-authored-by: clawbot <clawbot@noreply.example.org> Co-committed-by: clawbot <clawbot@noreply.example.org> |
||
|
clawbot
|
ed5d777d05 |
fix: set disk cache max size to 4x configured blob size instead of hardcoded 10 GiB
The disk blob cache now uses 4 * BlobSizeLimit from config instead of a hardcoded 10 GiB default. This ensures the cache scales with the configured blob size. |
||
|
clawbot
|
cafb3d45b8 |
fix: track and report file restore failures
Restore previously logged errors for individual files but returned success even if files failed. Now tracks failed files in RestoreResult, reports them in the summary output, and returns an error if any files failed to restore. Fixes #21 |
||
|
clawbot
|
46c2ea3079 |
fix: remove dead deep-verify TODO stub, route to RunDeepVerify
The VerifySnapshotWithOptions method had a dead code path for opts.Deep that printed 'not yet implemented' and returned nil. The CLI already routes --deep to RunDeepVerify (which is fully implemented). Remove the dead branch and update the VerifySnapshot convenience method to also route deep=true to RunDeepVerify. Fixes #2 |