sfdupes/TODO.md

Workflow

• branch (from main)
• do the work in Next Step
• move Next Step to the top of Completed Steps
• move the top item of Future Steps into Next Step
• commit (TODO.md changes in the same commit as the work)
• merge to main if the branch is not protected, otherwise open a PR
• push

Status

• pre-1.0

Next Step

• persistent scan database (branch persistent-database): scan maintains a SQLite database that survives between runs so it can be cronned daily; report and trees read the database instead of a scan stream. Database path defaults to /var/lib/sfdupes/db.sqlite, overridable via SFDUPES_DATABASE. Plan:
  • update README.md (the authoritative spec) for the database: new Database section, revised scan/report/trees modes, dependency list (modernc.org/sqlite, pure Go, cgo stays disabled), exit codes, smoke test with incremental steps
  • add modernc.org/sqlite (hash-pinned via go.sum)
  • db.go: path resolution (SFDUPES_DATABASE env, default /var/lib/sfdupes/db.sqlite), open/create with WAL + busy-timeout pragmas and PRAGMA user_version schema check, files table (path BLOB primary key, size, mtime, head, tail), load-all-rows, and single-transaction apply of upserts/deletes
  • scan.go: resolve operands to absolute paths; diff stat results against loaded rows (reuse hashes when size matches and mtime is not newer); hash only new/changed files; delete rows under the scanned operands not successfully processed; leave rows outside the operands untouched; new summary line (added/updated/removed/unchanged/skipped); "update" progress pass for the database write
  • report.go/trees.go: read records from the database (no positional args, no stream parsing); drop the NUL-stream parser and malformed-record handling
  • main.go: updated usage strings; report/trees take no args (usage error otherwise)
  • tests: db open/env-override/schema tests; incremental scan tests (add, mtime update, delete, unchanged-hash-reuse, out-of-scope rows untouched, error paths); rework pipeline test to go through the database; keep walk/stat/hash/grouping unit tests
  • .gitignore: local *.sqlite artifacts
  • make check, make docker, and the README smoke test pass

Completed Steps

• add the origin remote (git@git.eeqj.de:sneak/sfdupes.git), tag v0.0.1, and push main plus tags (2026-07-23)
• scan CLI rework (2026-07-23, branch scan-required-paths): required PATH... operands via cobra flags replacing the /srv -root default; new -x/--one-file-system flag (GNU convention) to stop at filesystem boundaries, which are crossed by default
• bring the repo into full policy compliance (2026-07-23, branch repo-policy-compliance; checklist below)
• git init with README-only first commit; code baseline committed on main (2026-07-22)
• implement scan, report, and trees subcommands (pre-git history)

Future Steps

• convert Makefile targets to scripts-to-rule-them-all script/ entrypoints like the other managed repos
• possible later features (explicitly out of scope per README): full-content verification of candidates, removal-script helpers

Repo Policy Compliance

Audited 2026-07-22 against REPO_POLICIES.md (2026-07-06), the existing repo checklist, and the Go styleguide. Code is already gofmt-clean, so no standalone formatting commit is needed.

• .gitignore missing — the compiled sfdupes binary and files.dat sit untracked in the tree; needs OS/editor/Go artifacts plus secrets patterns
• .editorconfig missing
• LICENSE missing and README has no License section (MIT assumed from house convention — user to confirm)
• REPO_POLICIES.md missing from repo root
• .golangci.yml missing (install canonical copy); code must then pass make lint (150 findings fixed; make lint is clean)
• Makefile lacks required targets test, lint, fmt, fmt-check, docker, hooks; check currently depends on build, which writes the binary (make check must not modify files)
• no tests — go test ./... has nothing to run; policy requires real tests with a 30-second timeout and the conditional -v rerun pattern (suite covers parsing, grouping, digests, suppression, hashing, and the scan pipeline; 64% coverage)
• Dockerfile missing — Go multistage with hash-pinned images: fail-fast lint stage, build stage running make check
• .dockerignore missing
• .gitea/workflows/check.yml missing (docker build . on push, checkout action pinned by commit SHA)
• README lacks required sections: Description first line (name/purpose/category/license/author), Getting Started, Rationale, TODO, License, Author
• README non-goal "no git repository setup and no CI" is stale now that the repo is under git with CI
• pre-commit hook not installed (make hooks once the target exists)

Accepted divergences (no action):

• flat single-package layout with .go files in the repo root — fine for a small single-binary tool per the Go styleguide; the tracker audit agrees
• go test runs without -race — the repo mandates CGO_ENABLED=0 (pure-Go builds) and the race detector requires cgo