sneak
426a8d7645
All checks were successful
check / check (pull_request) Successful in 3m11s
Closes #80. script/lint decided whether it could skip the pinned image by asking what version was on PATH rather than where it was running, and cache isolation is part of that same question. Both issues are that one defect. The cache was one directory per repo, shared by every worktree on the host. Two checkouts of this repo have identical Go file contents, so their cache keys collide and golangci-lint replays the stored analysis, including the paths recorded when it was produced. The loud direction of that failure - a clean tree failed by a dirty sibling - is the harmless one. The silent direction, a dirty tree passed by a clean sibling, is another way for a gate here to report a green it did not earn. The cache is now keyed on a digest of the worktree path, so a collision is not possible, and it stays persistent per worktree: a warm run is still seconds. Each cache records the worktree it belongs to and is collected when that worktree is gone, so throwaway worktrees do not accumulate caches; the tree lives under XDG_CACHE_HOME and is disposable by definition. script/lint-audit is the backstop, and runs on every lint: it rejects output citing any file that is not in the tree being linted, so a result built out of another checkout's analysis is a hard error instead of a silent pass. It runs on clean output too, because that is the case nobody investigates. It never certifies that a run passed - it does not look at whether there were findings - so it cannot itself become a gate that reports a green. golangci-lint's "parallel golangci-lint is running" refusal is now a bounded retry rather than a verdict. It is not a lint result, and exiting non-zero on it is indistinguishable to a caller from real findings; issue #88 measured that a private cache does not remove the contention. Exhausting the retries fails with a message that says the tree was never analysed. The native path now requires VAULTIK_LINT_IN_CONTAINER=1, which only the Dockerfile's lint stage sets, in addition to matching the pin. A developer's locally installed 2.12.2 is a different build reached by a different code path and no longer bypasses the digest pin. /.dockerenv was considered and rejected as the signal: dockerd creates it for `docker run`, but it is not reliably present during a BuildKit `docker build`, which is exactly the case the exception exists for. Inside the container a version mismatch is now a hard error rather than a fall-through, since there is no daemon there to fall through to. Version detection uses `golangci-lint version --short`, the interface meant for it, keeping the banner scrape only as a fallback. script/bootstrap no longer prints "bootstrap complete" on a machine that cannot run the gate. Docker missing, or present with an unreachable daemon, is a hard failure naming exactly what breaks. Installing docker from bootstrap was rejected: it needs root, a daemon, and on macOS a GUI cask, so the attempt would itself fail in the common case and trade one false success for a second failure mode. TODO.md's claim that `make check` became "as trustworthy as script/cibuild" is corrected to what README.md already said: only the lint leg is equivalent, while tests and gofmt still run against the host toolchain. README.md's requirements section gains docker and sqlite3. Verified by reproduction, not inspection: two concurrent lints from two worktrees of differing cleanliness each reported only their own findings with no lock error; a real run made to report paths outside its tree exits 1; a matching linter shimmed onto PATH is never invoked while the pinned image runs; a PATH without docker makes bootstrap fail. script/ cibuild exits 0 with the lint layer executing in the pinned image, which is what proves the in-container path still works.
294 lines
18 KiB
Markdown
294 lines
18 KiB
Markdown
# Workflow
|
|
|
|
* branch (from `main`)
|
|
* do the work in Next Step
|
|
* move Next Step to the top of Completed Steps
|
|
* move the top item of Future Steps into Next Step
|
|
* commit (`TODO.md` changes in the same commit as the work)
|
|
* merge to `main` if the branch is not protected, otherwise open a PR
|
|
* push
|
|
|
|
# Status
|
|
|
|
pre-1.0
|
|
|
|
# Next Step
|
|
|
|
Define remaining scope for a first tagged release and cut v0.1.0.
|
|
|
|
# Completed Steps
|
|
|
|
- 2026-08-09: Isolated the lint cache per worktree and context-gated the
|
|
native lint path (issues #99, #80). One defect seen twice:
|
|
`script/lint` decided whether it could skip the pinned image by asking
|
|
what version was on `PATH` rather than where it was running, and cache
|
|
isolation is part of that same question. The cache was one directory
|
|
per repo, shared by every worktree on the host, so two checkouts with
|
|
identical Go file contents collided and golangci-lint replayed the
|
|
stored analysis — paths and all. The loud direction of that failure
|
|
(a clean tree failed by a dirty sibling) is the harmless one; the
|
|
silent direction, a dirty tree **passed** by a clean sibling, is a
|
|
sixth way for a gate here to report a green it did not earn. The cache
|
|
is now keyed on a digest of the worktree path, and every run is
|
|
audited by the new `script/lint-audit`, which rejects output citing any
|
|
file that is not in the tree being linted — a backstop that runs on
|
|
clean output too, because that is the case nobody investigates. Caches
|
|
record the worktree they belong to and are collected when it
|
|
disappears, so throwaway worktrees do not accumulate them; the whole
|
|
tree lives under `XDG_CACHE_HOME` and is disposable. The
|
|
`parallel golangci-lint is running` refusal is now a bounded retry
|
|
rather than a verdict: it is not a lint result, and exiting non-zero
|
|
on it is indistinguishable to a caller from real findings (#88 showed
|
|
a private cache does not remove that contention). The native path now
|
|
requires `VAULTIK_LINT_IN_CONTAINER=1`, set only by the `Dockerfile`
|
|
lint stage, in addition to matching the pin, so a developer's locally
|
|
installed 2.12.2 no longer bypasses the digest pin; `/.dockerenv` was
|
|
rejected as the signal because `dockerd` creates it for `docker run`
|
|
and it is not reliably present during a BuildKit `docker build`, which
|
|
is the case the exception exists for. Version detection uses
|
|
`golangci-lint version --short` with the old banner scrape kept only
|
|
as a fallback. `script/bootstrap` no longer prints `bootstrap
|
|
complete` on a machine that cannot run the gate: a missing docker, or
|
|
one whose daemon is unreachable, is a hard failure naming exactly what
|
|
breaks. Verification was by reproduction rather than inspection — two
|
|
concurrent lints from two worktrees of differing cleanliness, a real
|
|
run made to report an outside path, a matching linter shimmed onto
|
|
`PATH`, and a `PATH` with docker removed — and is recorded on the pull
|
|
request.
|
|
- 2026-08-09: Closed the fifth false-green mechanism (issues #93, #69).
|
|
`script/test` omitted `-count=1`, so Go's test result cache could
|
|
satisfy the gate outright: a second back-to-back `make test` printed
|
|
the full set of 14 `ok` lines, every one marked `(cached)`, having
|
|
executed no test at all. Since `ok <pkg> (cached)` is an `ok` line,
|
|
the "14 `ok` lines means the suite ran" signal this repo leans on was
|
|
forgeable, one level below the Docker layer cache that #85 addressed.
|
|
Fixed with `-count=1` unconditionally rather than only in the
|
|
container, because the pre-commit hook runs the same script and a
|
|
gate honest only in CI is dishonest where people rely on it most;
|
|
`test-coverage` got the same flag, and `script/check` inherits it by
|
|
calling `script/test`. In the same area, `make test-integration` was
|
|
deleted rather than made real: no file in the repo carried a build
|
|
tag, so `-tags=integration` selected nothing and the target was an
|
|
exact duplicate of `make test`. Tagging a subset was rejected because
|
|
the entire suite runs in well under a minute, and a scheme whose
|
|
failure mode is "some tests silently stopped running" is a poor trade
|
|
for those seconds in a repo with this particular history. The
|
|
`-timeout` was raised from 30s after measuring rather than after
|
|
assuming: the standing claim that cold-cache compilation is charged
|
|
against `-timeout` is **false**, disproved by a containerised run
|
|
that spent 46s compiling and still reported per-package durations
|
|
within noise of a warm host run. `-timeout` reaches the test binary
|
|
as `-test.timeout` and its clock starts inside `testing.M.Run`, after
|
|
the build. The real exposure was margin, not compilation. The 120s
|
|
landed on is a **deliberate, documented divergence** from
|
|
`REPO_POLICIES.md:192`, which mandates 30s, and from that file's
|
|
canonical recipe at `:212-214`; the divergence is recorded in
|
|
`script/test`'s comment because `REPO_POLICIES.md` is org-canonical
|
|
and not editable here, and issue #101 proposes amending the policy
|
|
text upstream. Numbers and the full verification are recorded once,
|
|
on the pull request, and are deliberately not restated here.
|
|
|
|
- 2026-08-09: Triaged all fifteen stale remote branches (issue #71) and
|
|
deleted fourteen of them; the full per-branch disposition with
|
|
evidence is recorded on that issue. Method mattered more than the
|
|
outcome here: a three-dot `git diff main...branch` diffs from the
|
|
merge base, so it replays everything that landed on `main` after the
|
|
branch diverged and makes any old branch look like it holds unlanded
|
|
work. That artifact is what made `golangci-v2.12.2` appear to carry
|
|
126 files of unpushed changes when its tree was byte-identical to
|
|
`main`'s. Every containment claim here therefore rests on two-dot tip
|
|
diffs, tree-hash equality, `git cherry`, and `git branch -r --merged`.
|
|
Nine branches were plain ancestors of `main` with zero `git cherry`
|
|
`+` commits. `golangci-v2.12.2` had landed squashed as `cc58583`,
|
|
whose tree hash equals the branch tip's exactly; note the hash
|
|
recorded in the issue had gone stale because `main` advanced, so the
|
|
check had to be redone rather than repeated.
|
|
`fix/sync-snapshot-cleanup` was redundant, its one line already on
|
|
`main` in `syncWithRemote`. `feature/restore-progress-bar` was
|
|
superseded by `printRestoreProgress` and the disk-backed blob cache,
|
|
and had become actively regressive — it would have deleted
|
|
`internal/blobgen/compress_test.go`, the #28 regression test that
|
|
landed separately. The two branches this issue was filed for both
|
|
turned out to be closed questions that `main` had already moved past
|
|
by a recorded decision, so neither was landed and no regression test
|
|
was owed: `ctime` no longer exists anywhere in the codebase after
|
|
`1c72a37` removed the column, the `File.CTime` field and every use
|
|
(#54/#55), and change detection compares size, mtime, mode, uid and
|
|
gid only, exactly as `ARCHITECTURE.md` documents — so the
|
|
silently-skipped-file data-loss risk that made this a 1.0 item does
|
|
not exist. The SQL allow-list branch would have reverted `bfd7334`,
|
|
which replaced that very allow-list with regex sanitisation on review
|
|
feedback, and would have broken `getTableCount("snapshots")` because
|
|
its allow-list omits that table. `feature/daemon-mode` is untouched
|
|
and deferred to #94 pending an owner decision, so it is the one
|
|
branch besides `main` still on the remote. The stale `TODO.md` entry
|
|
named in the issue needed no fix: `e496aa3` had already removed it.
|
|
No product code changed.
|
|
- 2026-08-09: Adopted the remaining upstream `CHECK_EPOCH` hardening
|
|
(issue #91), closing the gap #85 knowingly left open. Four changes,
|
|
all four decided as adopt upstream in `sneak/prompts` #26. (1) Each
|
|
check stage now asserts `[ -n "$CHECK_EPOCH" ] || exit 1` before
|
|
running anything, so a build that supplies no `--build-arg` fails
|
|
instead of lying. This is the item that mattered: an unset `ARG` is
|
|
an empty string and an empty string is a stable cache key, so the
|
|
second and every later bare `docker build .` on an unchanged tree
|
|
replayed all three check layers and still exited 0 — and `docker
|
|
build .` is the command `REPO_POLICIES.md` names verbatim as a thing
|
|
that must be green, so the documented command was precisely the one
|
|
that lied. Failed steps are never cached, which is what makes the
|
|
guard fire on every invocation rather than once. (2) The epoch is now
|
|
expanded into each check command rather than left as a bare
|
|
declaration, so the cache miss no longer depends on BuildKit's
|
|
unreferenced-`ARG` handling staying as it is, and the value appears
|
|
in the build log. (3) `script/cibuild` uses
|
|
`epoch="$(date +%s%N)$$"`, unique per invocation rather than per
|
|
second; `%N` alone is insufficient because busybox drops it silently
|
|
and exits 0, and `$$` is what makes the guarantee hold regardless.
|
|
The bare-assignment form is kept deliberately — inlined in an
|
|
argument, a failing substitution does not abort under `set -eu` and
|
|
would yield an empty constant epoch, restoring the exact false green
|
|
being fixed. (4) `script/docker` passes the same fresh arg, so the
|
|
two entrypoints cannot disagree about whether the tree is green;
|
|
local builds are almost always warm, which made it the likelier
|
|
fooling in practice. The `ARG` placement from #85 is unchanged, below
|
|
`apk add`, `COPY go.mod go.sum` and `go mod download`, so dependency
|
|
layers still cache and the build is not cold. Verified by negative
|
|
control rather than inspection — a bare `docker build .` run twice
|
|
back to back, plus back-to-back pairs of both scripts and a host-side
|
|
`make check`; the measurements are recorded once, in the PR
|
|
verification comment, rather than restated here. `.golangci.yml`, the
|
|
lint-stage `FROM` line and its digest, `script/lint`,
|
|
`REPO_POLICIES.md` and `.gitea/workflows/check.yml` are all
|
|
untouched.
|
|
- 2026-08-09: Stopped `script/cibuild` from reporting a green it did
|
|
not earn (issue #85). A bare `docker build .` let Docker serve the
|
|
check layers from the layer cache whenever the tree had not changed:
|
|
the checks never executed and the build still exited 0. The fix is an
|
|
`ARG CHECK_EPOCH` declared immediately above the check `RUN`s in both
|
|
the lint stage and the builder stage (`ARG` scope is per-stage, so
|
|
each declares its own), with `script/cibuild` assigning
|
|
`epoch="$(date +%s)"` and passing `--build-arg CHECK_EPOCH="$epoch"`.
|
|
The assignment is separate on purpose: under `set -eu` a command
|
|
substitution that fails inside an argument does not abort the script,
|
|
which would leave an empty constant `CHECK_EPOCH` and restore the
|
|
very false green being fixed. Placement is the rest of the point —
|
|
the `ARG` sits below the `apk add`, `COPY go.mod go.sum`, and `go mod
|
|
download` layers, so only the checks are invalidated and the
|
|
dependency layers still cache. The guarantee is conditional on a
|
|
fresh value rather than absolute: a bare `docker build .` gets an
|
|
empty `CHECK_EPOCH` and can still serve the check layers from cache,
|
|
which `README.md` and the `Dockerfile` now say plainly, with issue
|
|
#91 tracking the upstream hardening (expanded `ARG` form, unset
|
|
guard, per-invocation epoch, `script/docker`) that would close it.
|
|
Verified by re-running the reproduction plus the withheld-`--build-arg`
|
|
counterfactual; the measurements are recorded once, in the PR #89
|
|
verification comment, rather than restated here. `.golangci.yml`, the
|
|
lint-stage `FROM` line and its digest, `script/lint`, and
|
|
`.gitea/workflows/check.yml` are all untouched.
|
|
- 2026-08-09: Corrected the `Vaultik.UI` doc comment (issue #84). It
|
|
claimed the cli layer replaces the writer with a discarding one in
|
|
`--cron` mode; the actual mechanism is `UI.SetQuiet(true)` in
|
|
`setupGlobals`, which drops Begin/Complete/Info/Notice/Detail/
|
|
Progress/Banner but still emits Warning and Error. The `--cron` line
|
|
in `README.md` said "Silent unless error", which understated what
|
|
survives, and now names warnings too. The other `--cron` comments
|
|
(`internal/log/log.go`, `internal/cli/snapshot.go`,
|
|
`internal/vaultik/snapshot.go`) were audited and already accurate.
|
|
Comments and docs only, no behavior change.
|
|
- 2026-08-09: Made `snapshot list` list the destination store without
|
|
the private key (issue #64). The listing is now the union of the
|
|
local index and a single streamed listing of the `metadata/` prefix,
|
|
with no `age_secret_key` gate — the manifest is unencrypted, so a
|
|
host holding only the public key can enumerate its own backups and a
|
|
host that lost its local index can still see them. A remote-only
|
|
snapshot's hostname and name are deliberately not recovered (they are
|
|
not recoverable without the private key, and making them so would
|
|
undo the privacy property tracked in issue #81); such rows are
|
|
labelled by an abbreviation of their remote key and carry the real
|
|
timestamp and compressed size from the manifest, with `<remote only>`
|
|
in the two columns that require the local index. Local-only snapshots
|
|
are reported as drift, and the hint now names `vaultik prune`, which
|
|
exists, instead of `vaultik snapshot cleanup`, which does not.
|
|
`reportRemoteDrift` collapsed into the merged view. Every remote
|
|
manifest read in the codebase now goes through
|
|
`downloadManifestByKey`, so issue #81 has one call site to change.
|
|
Review rework: snapshot timestamps now normalize to UTC in
|
|
`scanSnapshotRows`, the one place they enter the domain, so the merged
|
|
TIMESTAMP column cannot show local time for a locally tracked row and
|
|
UTC for a remote-only row on a non-UTC host; `GetIncompleteByHostname`
|
|
was folded onto that same scanner. `--json` now reports the
|
|
unreadable-manifest count and the 1000-row truncation on stderr
|
|
instead of returning a silently short document (the document's shape
|
|
is unchanged). The two per-snapshot `log.Warn` calls on the listing
|
|
path now route through the same JSON-aware writer as the existing
|
|
workaround, so one corrupt manifest can no longer put a log line on
|
|
stdout ahead of the document and break `| jq` — still a local
|
|
workaround pending issue #82. Verified with `script/cibuild` and with
|
|
an uncached `make check` (`0 issues.`, no cached test packages), plus
|
|
end to end against a `file://` destination with no secret key present.
|
|
- 2026-08-09: Closed the gap between `make lint` and CI (issue #78).
|
|
`script/lint` now runs the digest-pinned `golangci-lint` image taken
|
|
from the `Dockerfile` lint stage, which is the single source of truth
|
|
for the linter version; the duplicate pin in the `Makefile` `deps`
|
|
target and the unpinned `golangci-lint` install in `script/bootstrap`
|
|
are gone. A `golangci-lint` on `PATH` is used only when its version is
|
|
exactly the pinned one (which is how the lint stage runs it inside the
|
|
container); anything else goes through Docker, and a missing or
|
|
unreachable Docker daemon is a hard error rather than a silent
|
|
fallback. Only the **lint** leg of `make check` became equivalent to
|
|
`script/cibuild`; its tests and `gofmt` still run on the host against
|
|
the host toolchain, as `README.md` states. An earlier version of this
|
|
entry claimed `make check` was "as trustworthy as `script/cibuild`"
|
|
outright, which overstated it; corrected under issue #80.
|
|
- 2026-08-09: Finished the lint remediation under the canonical
|
|
`.golangci.yml` (issue #61, which also unblocks issue #59). The
|
|
remaining findings were fixed behavior-preservingly: `wsl_v5`
|
|
whitespace, `sqlclosecheck`, and `prealloc`. The `sqlclosecheck` sites
|
|
now close `sql.Rows` in a deferred closure instead of via the
|
|
`CloseRows` helper, which the linter could not see through. Only the
|
|
`revive` package-name findings remain suppressed, with per-site
|
|
`//nolint` directives; the package-rename question behind them is
|
|
tracked in issue #76. Verified with `script/cibuild`, which exits 0 —
|
|
that is the only trustworthy gate, because `script/lint` runs whatever
|
|
`golangci-lint` happens to be on `PATH` rather than the pinned
|
|
v2.12.2 that CI and the `Dockerfile` use, so `make check` can report
|
|
green on findings CI still fails. That tooling gap is tracked in issue
|
|
#78.
|
|
- 2026-08-09: The earlier next step "reconcile the uncommitted
|
|
`ARCHITECTURE.md` edits on `main`" needed no work: the working tree is
|
|
clean and `ARCHITECTURE.md` is committed on `main`.
|
|
- 2026-08-07: Updated golangci-lint to v2.12.2 everywhere it is pinned
|
|
(`Dockerfile` lint stage, `Makefile` deps target), replaced
|
|
`.golangci.yml` with the canonical config (v2 schema, `default: all`),
|
|
and remediated the bulk of the lint findings it surfaced (issue #61):
|
|
behavior-preserving fixes across every package, 2,990 findings down to
|
|
80. `make test` and `make fmt-check` were green at that point but
|
|
`make lint` was still red; the commit message claiming `make check`
|
|
was green was wrong.
|
|
- 2026-08-07: Added the standard `.golangci.yml` and `.editorconfig`
|
|
(issue #59); lint findings under the new config are tracked in issue
|
|
#61. `script/bootstrap` now installs sqlite3 (needed by tests).
|
|
- 2026-07-07 Adopted scripts-to-rule-them-all: `script/` entrypoints,
|
|
Makefile shims, README Entrypoints section
|
|
- 2026-07-02: Consolidated CLI verbs, retired overlapping commands; bound
|
|
the local index to its backup destination URL.
|
|
- 2026-06-28: snapshot rm now removes metadata only and prints the prune
|
|
command; restore skips chown when running as non-root.
|
|
- 2026-06-26: Snapshot IDs hashed at the storage boundary; snapshot list
|
|
made resilient to bad remote entries.
|
|
- 2026-06-24: Collapsed snapshot prune into vaultik prune; restore streams
|
|
blobs to disk and restores files in blob-locality order; cron output
|
|
fixes.
|
|
- 2026-06-17: Restore overhaul: ReadAt chunk reads from cached blobs,
|
|
reference-counted blob sweeper, integration tests; new internal/ui
|
|
output layer, banner, and progress lines.
|
|
- 2025-12-18: Added ARCHITECTURE.md and godoc coverage for exported API.
|
|
- 2025-07-26: End-to-end integration tests; manifest format refactor;
|
|
renamed backup to snapshot; afero filesystem abstraction.
|
|
- 2025-07-20: Initial design and implementation: cobra + fx CLI skeleton,
|
|
SQLite index database, UUID blob storage with streaming chunking.
|
|
|
|
# Future Steps
|
|
|
|
None queued; the release-scoping item is now the Next Step.
|