script/test omitted -count=1, so Go's test result cache could satisfy
the gate outright. A cached package prints `ok <pkg> (cached)`, and
that is an `ok` line: two back-to-back `make test` runs on the parent
commit produced the same 14 `ok` lines, the second in 0.42s with every
line marked (cached), having executed no test at all. The evidence
signal this repo leans on was therefore forgeable. This sits one level
below the Docker layer cache #85 addressed: CHECK_EPOCH forces the
`RUN make test` step to re-execute, but a GOCACHE baked into an earlier
image layer would survive into the re-executed step, so the step can
re-run and still do no work.
Add -count=1 unconditionally rather than only in the containerised
path. The pre-commit hook runs this same script, and a gate that is
honest only in CI is dishonest exactly where people rely on it most.
test-coverage gets the same flag -- a coverage profile assembled from
cached results describes a run that did not happen -- and script/check
inherits it by calling script/test.
Delete make test-integration rather than making it real (option (b) of
issue #69). No file in the repo carries a build tag, so -tags=integration
selected nothing extra and the target was an exact duplicate of make
test, while internal/vaultik/integration_test.go ran unconditionally on
every make test -- the opposite of what a reader of the Makefile would
conclude. Tagging a subset was rejected because the entire suite runs in
well under a minute, so gating would save seconds in exchange for a
build-tag scheme and a second CI path that must be kept wired up; in a
repo that has now found five distinct ways for a gate to report an
unearned green, a mechanism whose failure mode is "some tests silently
stopped running" is a bad trade. Deleting it also means nothing can
drop out of CI coverage, since nothing is conditional. The Makefile and
README now say plainly that make test runs everything.
Raise -timeout from 30s to 120s, after measuring rather than assuming.
The standing claim that cold-cache compilation is charged against
-timeout is false: -timeout reaches the test binary as -test.timeout
and its clock starts inside testing.M.Run, after compilation and
linking. A containerised run with an empty GOCACHE spent 46s compiling
and still reported per-package durations within noise of a warm host
run. The real exposure was margin, not compilation: against the slowest
package's worst observed time, 10.2s for internal/database on a cold
run, 30s left only 2.9x, thin for a loaded CI runner, and each fresh
measurement of that package has come in above the last. -timeout is a
hang backstop, not a performance budget, so it should sit far above the
slowest legitimate runtime; 120s leaves about 12x while still bounding
a hung package, including the verbose rerun, to a few minutes.
120s is a deliberate divergence from REPO_POLICIES.md:192, which
mandates a 30-second timeout, and from that file's canonical Go recipe
at :212-214. REPO_POLICIES.md is org-canonical and not editable from
this repo, so the divergence and its reasoning are recorded in
script/test's comment and in TODO.md, and issue #101 proposes amending
the policy text upstream. That issue also carries the trade this
surfaces: the verbose rerun makes a hung package pay the timeout twice,
which at 120s puts a hang-case Docker build over the same policy's
five-minute build limit.
Both invocations in script/test now share one run_tests function so the
quiet run and the verbose rerun cannot drift apart in flags.
Measurements and the full verification are recorded once, on the pull
request, and deliberately not restated here or in TODO.md.
script/lint ran bare golangci-lint from PATH while CI and the Dockerfile
pinned v2.12.2 by digest, so make lint and CI could disagree about
findings. That drift ran both directions: it produced two false green
claims during the lint remediation, and on an ambient 2.10.1 it also
reported four gosec findings on a tree CI linted clean.
script/lint now extracts the image reference - tag and digest - from the
Dockerfile lint stage FROM line and runs that exact image under docker.
The Dockerfile FROM line is the single source of truth for the linter
version; the duplicate pins in the Makefile deps target and in
script/bootstrap are removed rather than kept in sync.
A golangci-lint on PATH is used only when its version exactly equals the
pin, which is what makes the in-container lint stage work (the Dockerfile
runs make lint inside the pinned image, where there is no docker daemon).
Any other version, or none, goes through docker. When docker is
unavailable the script fails with an actionable message and never falls
back to a different linter version.
script/lint-fix delegates to script/lint --fix so autofixes come from the
pinned linter too. The container mounts persistent build and module
caches and runs as the invoking uid/gid.
Verified by reinstating the four historical nolint directives that 2.10.1
requires and 2.12.2 reports as unused: the old script passed on that tree
and the new one fails with four nolintlint findings.
Updates golangci-lint to v2.12.2 everywhere it is pinned and installs the canonical `.golangci.yml`, then remediates every finding the new linter/config surfaces so `make check` is green.
## Version bump
- `Dockerfile` lint stage: `golangci/golangci-lint:v2.11.3-alpine` -> `v2.12.2-alpine` (digest-pinned, date comment updated)
- `Makefile` `deps` target: `go install` moved from the old v1 module path at `@latest` to the pinned `github.com/golangci/golangci-lint/v2/cmd/golangci-lint@v2.12.2`
- `.golangci.yml` replaced with the canonical config (v2 schema; settings under `linters.settings` so the thresholds actually apply; `default: all` with the standard six disables)
- `script/bootstrap` installs golangci-lint via the system package manager and carries no version pin, so it is unchanged
- CI (`.gitea/workflows/check.yml`) only runs `script/cibuild`, so it needed no change
## Lint remediation
The canonical config surfaced ~3,300 findings across 56k lines. All are fixed, behavior-preserving; incorporates and supersedes the per-package mechanical passes already merged to `main` (refs #61). Highlights:
- `err113`: dynamic errors replaced with package sentinels + `%w` wrapping; comparisons via `errors.Is`
- `goprintffuncname`: printf-style helpers renamed with an `f` suffix (`ui.Writer` message methods, `cli.ReportErrorf`, `database.Fatalf`) and all call sites updated
- `revive` stutter renames: `blob.Handler`, `blob.WithReader`, `blob.ChunkPosition`, `storage.URL`, `storage.Info`; missing doc comments added
- `contextcheck`/`noctx`: `context.Context` threaded through `blob.Packer` and the scanner call sites; context-aware `exec`/`sql` variants
- `funlen`/`cyclop`/`gocognit`/`dupl`: oversized and duplicated functions split into focused helpers (production and test code)
- tests: `t.Parallel()` added where safe (global logger init kept in the serial phase for `-race`), `t.TempDir()`/`t.Helper()` adopted, several suites converted to external test packages
- `gosec`: bounded integer conversions, `ReadHeaderTimeout` on the test HTTP server; remaining warnings suppressed per-site with justifications
- remaining `nolint` directives are rare, targeted, and each carries a reason (e.g. `nilnil` not-found contract in the repository layer, fx module globals, on-disk snake_case struct tags)
- removed the deprecated `log.LogOptions` alias (callers migrated to `log.Options`)
`make check` (tests with `-race`, lint, fmt-check) passes.
Co-authored-by: sneak <sneak@sneak.berlin>
Reviewed-on: #62
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
- globals.go: add Homepage and License constants.
- version command: show author, homepage, license, build date.
- Startup banner reformatted to:
vaultik X by Author (commit Y, built on Z) starting up at T.
https://sneak.berlin/go/vaultik
- Commit date now formatted as YYYY-MM-DD (called "build date" in
user-facing output, since the binary was at least compiled once on
the date of commit). Makefile/Dockerfile use git --format=%cs.
goreleaser slices its RFC3339 .CommitDate template var to 10 chars.
All user-facing output now goes through a single ui.Writer with a
uniform style:
》 (white) for begin / info / notice
》 (green) for complete / success
Warning: for warnings (orange)
ERROR: for errors (red)
》 (indented) for progress heartbeats
Color is enabled when stdout is a TTY and NO_COLOR is unset.
Standards:
- Complete-sentence messages with fully qualified terms ("backup
destination store", "local index database", "snapshot source
files enumeration").
- Every Complete has a matching Begin.
- Natural verb tense conveys state ("Uploading" -> "Uploaded"). The
words "begin"/"complete" never appear in message bodies; the marker
color carries that information.
- ETA means clock time, not duration. Progress lines say "estimated
remaining time (<dur>), finish at <time>" with both labeled.
Adds globals.CommitDate (populated by Makefile/Dockerfile/goreleaser
via ldflags from `git show -s --format=%cI HEAD`) and a startup banner
printed once per invocation.
Strips fx call-chain noise from startup errors so users see the actual
underlying error (e.g. "creating base path: mkdir /Volumes/BACKUPS:
permission denied" instead of three layers of "could not build
arguments for function ...").
README documents the output style and the ui package conventions.
Module path changed from git.eeqj.de/sneak/vaultik to
sneak.berlin/go/vaultik (vanity redirect). All imports, ldflags,
Dockerfile, goreleaser config, and docs updated. App data/config
directories now use plain "vaultik" instead of the reverse-DNS name.
README:
- New copy-pasteable quickstart at top: go install, config init,
age keypair, config set for key + file:// destination, home backup
- All command names in command details are code-quoted
- config set/get gained sequence index support (age_recipients.0)
so lists are settable from the CLI
- Dockerfile build is CGO_ENABLED=0 to match the pure-Go build
- Adopt origin's SnapshotPurgeOptions naming and PurgeSnapshotsWithOptions
method, but extend with Names []string (repeatable --snapshot flag) and
Quiet bool for use by --prune.
- Adopt origin's parseSnapshotName helper.
- Fold the duplicate post-backup prune block into one runPostBackupPrune
call that filters retention to the snapshot names just backed up.
- Keep the shallow-verify timestamp parsing fix and the dead deep-verify
branch removal; use origin's printVerifyHeader/verifyManifestBlobsExist
helper extraction.
- Drop top-level vaultik purge and verify (duplicates of snapshot purge
and snapshot verify).
- Drop the resurrected daemon block from info.go (config fields no
longer exist).
- Combine Makefile targets: gofmt -l for fmt-check, -race for tests,
release/release-snapshot/docker/hooks/deps/test-coverage all included.
make targets each do one thing now: lint, fmt, fmt-check, test. Use
'make check' for combined lint + fmt-check + test (the standard
pre-commit gate).
Release builds are pure-Go (CGO_ENABLED=0) cross-compiling to
linux/darwin × amd64/arm64.
Adds a `make check` target that verifies formatting (gofmt), linting (golangci-lint), and tests (go test -race) without modifying files.
Also adds `.gitea/workflows/check.yml` CI workflow that runs on pushes and PRs to main.
`make check` passes cleanly on current main.
Co-authored-by: user <user@Mac.lan guest wan>
Co-authored-by: clawbot <clawbot@noreply.git.eeqj.de>
Co-authored-by: clawbot <clawbot@sneak.berlin>
Reviewed-on: #42
Co-authored-by: clawbot <sneak+clawbot@sneak.cloud>
Co-committed-by: clawbot <sneak+clawbot@sneak.cloud>
- Implement deterministic blob hashing using double SHA256 of uncompressed
plaintext data, enabling deduplication even after local DB is cleared
- Add Stat() check before blob upload to skip existing blobs in storage
- Add rclone storage backend for additional remote storage options
- Add 'vaultik database purge' command to erase local state DB
- Add 'vaultik remote check' command to verify remote connectivity
- Show configured snapshots in 'vaultik snapshot list' output
- Skip macOS resource fork files (._*) when listing remote snapshots
- Use multi-threaded zstd compression (CPUs - 2 threads)
- Add writer tests for double hashing behavior
- Add internal/types package with type-safe wrappers for IDs, hashes,
paths, and credentials (FileID, BlobID, ChunkHash, etc.)
- Implement driver.Valuer and sql.Scanner for UUID-based types
- Add `vaultik version` command showing version, commit, go version
- Add `--verify` flag to restore command that checksums all restored
files against expected chunk hashes with progress bar
- Remove fetch.go (dead code, functionality in restore)
- Clean up TODO.md, remove completed items
- Update all database and snapshot code to use new custom types
- Implement exclude patterns with anchored pattern support:
- Patterns starting with / only match from root of source dir
- Unanchored patterns match anywhere in path
- Support for glob patterns (*.log, .*, **/*.pack)
- Directory patterns skip entire subtrees
- Add gobwas/glob dependency for pattern matching
- Add 16 comprehensive tests for exclude functionality
- Add snapshot prune command to clean orphaned data:
- Removes incomplete snapshots from database
- Cleans orphaned files, chunks, and blobs
- Runs automatically at backup start for consistency
- Add snapshot remove command for deleting snapshots
- Add VAULTIK_AGE_SECRET_KEY environment variable support
- Fix duplicate fx module provider in restore command
- Change snapshot ID format to hostname_YYYY-MM-DDTHH:MM:SSZ
- Add pure Go SQLite driver (modernc.org/sqlite) to avoid CGO dependency
- Implement database connection management with WAL mode
- Add write mutex for serializing concurrent writes
- Create schema for all tables matching DESIGN.md specifications
- Implement repository pattern for all database entities:
- Files, FileChunks, Chunks, Blobs, BlobChunks, ChunkFiles, Snapshots
- Add transaction support with proper rollback handling
- Add fatal error handling for database integrity issues
- Add snapshot fields for tracking file sizes and compression ratios
- Make index path configurable via VAULTIK_INDEX_PATH environment variable
- Add comprehensive test coverage for all repositories
- Add format check to Makefile to ensure code formatting
- Set up cobra CLI with all commands (backup, restore, prune, verify, fetch)
- Integrate uber/fx for dependency injection and lifecycle management
- Add globals package with build-time variables (Version, Commit)
- Implement config loading from YAML with validation
- Create core data models (FileInfo, ChunkInfo, BlobInfo, Snapshot)
- Add Makefile with build, test, lint, and clean targets
- Include minimal test suite for compilation verification
- Update documentation with --quick flag for verify command
- Fix markdown numbering in implementation TODO
