sneak
a0f0050ada
All checks were successful
check / check (push) Successful in 6s
With per-operand walk/hash/update cycles, a multi-operand invocation (e.g. scan /srv/*) showed pass totals that looked like the whole run's: an operator watching operand 3 of 14 hash 300k files concluded the other 20M files were being skipped. Print the operand path and its position before each cycle.
106 lines
4.8 KiB
Markdown
106 lines
4.8 KiB
Markdown
# Workflow
|
|
|
|
- branch (from `main`)
|
|
- do the work in Next Step
|
|
- move Next Step to the top of Completed Steps
|
|
- move the top item of Future Steps into Next Step
|
|
- commit (`TODO.md` changes in the same commit as the work)
|
|
- merge to `main` if the branch is not protected, otherwise open a PR
|
|
- push
|
|
|
|
# Status
|
|
|
|
- pre-1.0
|
|
|
|
# Next Step
|
|
|
|
- convert Makefile targets to scripts-to-rule-them-all `script/`
|
|
entrypoints like the other managed repos
|
|
|
|
# Completed Steps
|
|
|
|
- announce each operand on stderr before its passes (2026-07-24,
|
|
branch `scan-operand-progress`): with per-operand walk/hash/update
|
|
cycles, a multi-operand run (e.g. `scan /srv/*`) showed pass totals
|
|
that looked like the whole run's — an operator watching operand 3 of
|
|
14 hash 300k files concluded 20M files were being skipped
|
|
- parallel walk (2026-07-24, branch `parallel-walk`): the walk pass
|
|
was a single goroutine and took hours at ~20M files on a busy pool
|
|
(observed: 22M files in 4h on a ZFS server); it is now a
|
|
per-directory worker-pool traversal that records size/mtime during
|
|
the walk (folding away the separate stat pass, halving metadata
|
|
I/O), and each `PATH` operand commits in its own transaction so an
|
|
interrupted scan keeps completed operands
|
|
|
|
- persistent scan database (2026-07-24, branch `persistent-database`):
|
|
`scan` now maintains a SQLite database (`modernc.org/sqlite`, pure
|
|
Go, cgo stays disabled) keyed by absolute path that survives between
|
|
runs — a rescan hashes only new or changed files (by mtime/size),
|
|
deletes records for files vanished from under the scanned operands,
|
|
and leaves records outside them untouched, so `scan` can be cronned
|
|
daily; `report` and `trees` read the database (no positional
|
|
arguments) instead of a scan stream. Database at
|
|
`/var/lib/sfdupes/db.sqlite`, overridable via `SFDUPES_DATABASE`;
|
|
WAL journaling plus a single-transaction update keep a report run
|
|
during a scan safe
|
|
- add the `origin` remote (`git@git.eeqj.de:sneak/sfdupes.git`), tag
|
|
`v0.0.1`, and push `main` plus tags (2026-07-23)
|
|
- `scan` CLI rework (2026-07-23, branch `scan-required-paths`): required
|
|
`PATH...` operands via cobra flags replacing the `/srv` `-root`
|
|
default; new `-x`/`--one-file-system` flag (GNU convention) to stop
|
|
at filesystem boundaries, which are crossed by default
|
|
- bring the repo into full policy compliance (2026-07-23, branch
|
|
`repo-policy-compliance`; checklist below)
|
|
- `git init` with README-only first commit; code baseline committed on
|
|
`main` (2026-07-22)
|
|
- implement `scan`, `report`, and `trees` subcommands (pre-git history)
|
|
|
|
# Future Steps
|
|
|
|
- possible later features (explicitly out of scope per README):
|
|
full-content verification of candidates, removal-script helpers
|
|
|
|
# Repo Policy Compliance
|
|
|
|
Audited 2026-07-22 against `REPO_POLICIES.md` (2026-07-06), the existing
|
|
repo checklist, and the Go styleguide. Code is already gofmt-clean, so no
|
|
standalone formatting commit is needed.
|
|
|
|
- [x] `.gitignore` missing — the compiled `sfdupes` binary and
|
|
`files.dat` sit untracked in the tree; needs OS/editor/Go
|
|
artifacts plus secrets patterns
|
|
- [x] `.editorconfig` missing
|
|
- [x] `LICENSE` missing and README has no License section (MIT assumed
|
|
from house convention — user to confirm)
|
|
- [x] `REPO_POLICIES.md` missing from repo root
|
|
- [x] `.golangci.yml` missing (install canonical copy); code must then
|
|
pass `make lint` (150 findings fixed; `make lint` is clean)
|
|
- [x] `Makefile` lacks required targets `test`, `lint`, `fmt`,
|
|
`fmt-check`, `docker`, `hooks`; `check` currently depends on
|
|
`build`, which writes the binary (`make check` must not modify
|
|
files)
|
|
- [x] no tests — `go test ./...` has nothing to run; policy requires
|
|
real tests with a 30-second timeout and the conditional `-v`
|
|
rerun pattern (suite covers parsing, grouping, digests,
|
|
suppression, hashing, and the scan pipeline; 64% coverage)
|
|
- [x] `Dockerfile` missing — Go multistage with hash-pinned images:
|
|
fail-fast lint stage, build stage running `make check`
|
|
- [x] `.dockerignore` missing
|
|
- [x] `.gitea/workflows/check.yml` missing (`docker build .` on push,
|
|
checkout action pinned by commit SHA)
|
|
- [x] README lacks required sections: Description first line
|
|
(name/purpose/category/license/author), Getting Started,
|
|
Rationale, TODO, License, Author
|
|
- [x] README non-goal "no git repository setup and no CI" is stale now
|
|
that the repo is under git with CI
|
|
- [x] pre-commit hook not installed (`make hooks` once the target
|
|
exists)
|
|
|
|
Accepted divergences (no action):
|
|
|
|
- flat single-package layout with `.go` files in the repo root — fine
|
|
for a small single-binary tool per the Go styleguide; the tracker
|
|
audit agrees
|
|
- `go test` runs without `-race` — the repo mandates `CGO_ENABLED=0`
|
|
(pure-Go builds) and the race detector requires cgo
|