1 Commits
Author SHA1 Message Date
sneak c64a3c416d Rewrite TODO.md sentence by sentence (closes #135)
check / check (push) Canceled after 0s
TODO.md is the project's own task list, judged by the same rules as
README.md: short sentences, lists as bullets, nothing a newcomer reads
twice. Every completed step is kept with its date and its issue or
branch, and every fact the old entries recorded is kept too, in short
sentences. The 2026-07-22 policy-compliance checklist is folded into
the step that did the work, and the accepted divergence from the
layout policy gets a section of its own. Workflow, Status and Next
Step keep their rules unchanged.

Model: fable-5-1
2026-10-26 08:18:11 +00:00
+77 -60
View File
@@ -39,7 +39,8 @@ are known.
it. It leads with what the program is and how to use it. A reference section
covers the settings, the database, the output formats and the exit codes;
development and testing come last. Every statement was checked against the
code on `next` (2026-10-26, https://git.eeqj.de/sneak/sfdupes/issues/135)
code on `next`. Three comments in `main_test.go` cite "Settings", where their
rules now live (2026-10-26, https://git.eeqj.de/sneak/sfdupes/issues/135)
- A file or directory removed while the scan runs is not an error, in the walk
or in the hash phase. The scan prints no line for it and does not count it as
@@ -62,7 +63,9 @@ are known.
of the whole file under 100 MiB, and of 1 MiB at each gigabyte from 100 MiB
up. Gone are:
- the 10 MiB and 50 MiB thresholds;
- the size census;
- the size census: the sizes of every walked file and of every record
outside the operands, by which a file was hashed only when another file
shared its size;
- the records without hashes;
- the content phase.
@@ -107,20 +110,20 @@ are known.
- `scan` never records or reads anything on a filesystem that is not persistent:
`tmpfs`, `proc`, `sysfs` and the other kernel or memory-backed types. It
matches them by device ID. As the scan starts, it lists the mounted
filesystems of those types, from `/proc/self/mountinfo` on Linux and from the
mount list on macOS. No flag turns that off. Below an operand, the walk does
not enter the default excludes either, so `sfdupes scan /` skips the kernel's
view of the running system and the temporary directories. On Linux those are
`/proc`, `/sys`, `/dev`, `/run`, `/tmp` and `/var/tmp`. On macOS they are
`/dev`, `/private/tmp`, `/private/var/tmp`, `/private/var/vm`,
`/System/Volumes/VM` and `/System/Volumes/Data`. A Windows list exists in the
code but is not used. An operand at or inside a default exclude is still
walked. The scan prints a line for each path it leaves out, without counting
it as skipped, and the records there count as outside the scanned operands, as
for `-x`. `--no-default-excludes` turns off the default excludes only, and
`scan --help` lists them (2026-10-17,
https://git.eeqj.de/sneak/sfdupes/issues/124)
matches them by device ID, however the walk or the content phase reaches a
path. As the scan starts, it lists the mounted filesystems of those types,
from `/proc/self/mountinfo` on Linux and from the mount list on macOS. No flag
turns that off. Below an operand, the walk does not enter the default excludes
either, so `sfdupes scan /` skips the kernel's view of the running system and
the temporary directories. On Linux those are `/proc`, `/sys`, `/dev`, `/run`,
`/tmp` and `/var/tmp`. On macOS they are `/dev`, `/private/tmp`,
`/private/var/tmp`, `/private/var/vm`, `/System/Volumes/VM` and
`/System/Volumes/Data`. A Windows list exists in the code but is not used. An
operand at or inside a default exclude is still walked. The scan prints a line
for each path it leaves out, without counting it as skipped, and the records
there count as outside the scanned operands, as for `-x`.
`--no-default-excludes` turns off the default excludes only, and `scan --help`
lists them (2026-10-17, https://git.eeqj.de/sneak/sfdupes/issues/124)
- The rate a progress display shows covers the items completed over the last 5
seconds, or since the line before when stderr is not a terminal. It falls to 0
@@ -129,14 +132,15 @@ are known.
phase's average. Once no item has completed for 10 seconds, the word `stalled`
and where the phase waits take the rate's place, as `stalled disk/db`. The
places are `disk`, `db`, `out` for stdout and `cpu`, or two of them for a
phase that may wait on either. Each phase gives its word where it is defined.
A line too wide for the terminal loses its bar first, then its rate or those
words, so the phase, progress, elapsed time and ETA stay whole. On an
80-column terminal, at counts like those of README's example, the content
phase's checks and reads lose those words at most times in minutes. Every
other phase keeps them whenever the elapsed time and the ETA are both in
minutes. The content phase's last partial batch is written under a display of
its own (2026-10-16, https://git.eeqj.de/sneak/sfdupes/issues/125)
phase that may wait on either. Each phase's word is set in the code that
defines the phase. A line too wide for the terminal loses its bar first, then
its rate or those words, so the phase, progress, elapsed time and ETA stay
whole. On an 80-column terminal, at counts like those of README's example, the
content phase's `checks` and `reads` lines were too wide for those words. That
held at most elapsed times and ETAs in minutes. Every other phase kept them
whenever the elapsed time and the ETA were both in minutes. The content
phase's last partial batch is written under a display of its own (2026-10-16,
https://git.eeqj.de/sneak/sfdupes/issues/125)
- New subcommand `info` prints the database's details on stdout, one per line as
a name, a tab and the value:
@@ -189,8 +193,8 @@ are known.
narrow, also after the terminal is made narrower during a phase. A line too
wide leaves out its rate, then its bar, keeping the phase, progress, elapsed
time and ETA whole. It is cut short only when those alone do not fit, as below
about 70 columns late in a large scan. In plain mode the line is printed every
5 seconds whether or not an item completed (2026-10-16,
about 70 columns late in a large scan. When stderr is not a terminal, the line
is printed every 5 seconds whether or not an item completed (2026-10-16,
https://git.eeqj.de/sneak/sfdupes/issues/112)
- `scan`, `report` and `trees` first print a line on stderr naming the database
@@ -314,8 +318,8 @@ are known.
`files` table is refused with a clear schema-version error (2026-10-04,
https://git.eeqj.de/sneak/sfdupes/issues/11)
- README documents install, Docker, a daily cron scan and how to read and check
the reports (2026-10-04, https://git.eeqj.de/sneak/sfdupes/issues/54)
- README documents install, Docker, a daily `cron` scan and how to read and
check the reports (2026-10-04, https://git.eeqj.de/sneak/sfdupes/issues/54)
- The `Dockerfile` build stage kept the Go module cache out of `builder`'s home
and copied the sources with `--chown`, so no `chown -R` walked them. Since
@@ -359,12 +363,12 @@ are known.
and warning paths. The root directory's path is `/` (2026-10-03,
https://git.eeqj.de/sneak/sfdupes/issues/7)
- A plain `docker build .` stamps the git tag or short commit instead of `dev`.
`.dockerignore` sends `.git` without `.git/config`. The build stage stamps the
`VERSION` build argument, else `git describe --tags --always`. It fails if the
context carries `.git` and the version is still empty, `dev` or `unknown`. CI
checks out the full history (`fetch-depth: 0`), so it stamps the same value as
`make build` (2026-10-02, branch `next`,
- A plain `docker build .` stamps the `git` tag or short commit instead of
`dev`. `.dockerignore` sends `.git` without `.git/config`. The build stage
stamps the `VERSION` build argument, else `git describe --tags --always`. It
fails if the context carries `.git` and the version is still empty, `dev` or
`unknown`. CI checks out the full history (`fetch-depth: 0`), so it stamps the
same value as `make build` (2026-10-02, branch `next`,
https://git.eeqj.de/sneak/sfdupes/issues/67)
- The 1 KiB end-window sampling was replaced with the hashes chosen by file
@@ -438,11 +442,20 @@ are known.
installing. On a mismatch it printed both paths and both versions and exited
non-zero; it did not reorder `PATH` or delete anyone's binary. The `--version`
call kept its stderr and was bounded by `timeout(1)` where that exists. `git`,
`make` and `go` kept presence-only checks. `make check` and `make lint` were
clean at v2.12.2, so v2.10.1 had not been hiding findings on `main`.
Superseded by https://git.eeqj.de/sneak/sfdupes/issues/46: the linter is no
longer installed on a host (2026-08-09, branch `bootstrap-version-check`,
https://git.eeqj.de/sneak/sfdupes/issues/24)
`make` and `go` kept presence-only checks. It was verified by bootstrapping
this host from v2.10.1 to v2.12.2, and again to a no-op. Stub runs of the
script under `dash` covered:
- a thirteen-input version-parse matrix;
- a shadowed install, which must exit non-zero;
- an install destination not on `PATH`;
- `GOBIN` set;
- a wedged binary, which must hit the timeout.
`make check` and `make lint` were clean at v2.12.2, so v2.10.1 had not been
hiding findings on `main`. Superseded by
https://git.eeqj.de/sneak/sfdupes/issues/46: the linter is no longer
installed on a host (2026-08-09, branch `bootstrap-version-check`,
https://git.eeqj.de/sneak/sfdupes/issues/24)
- The hash worker pool unwinds on the error path. The pool is an owned,
context-aware `hashPool`. Every blocking send in the feeder and the workers
@@ -454,12 +467,14 @@ are known.
walk pool gets the same treatment, plus a `ctx.Err()` guard after the walk. A
cancelled walk yields a partial size census, and every file it never reached
looks vanished to the update phase. That phase's own `BeginTx` also fails on
the cancelled context before deleting anything, but the guard is the barrier
that still holds once an interrupted scan may commit what it has. Tests drive
`run(scan)` against a database whose insert trigger aborts, and assert that
the scan fails instead of hanging and that the goroutine count returns to its
pre-scan baseline. Other tests cancel a scan part-way through the walk and
assert that it stops at the guard with every record intact. Direct tests of
the cancelled context before deleting anything. The guard is still the barrier
once an interrupted scan may commit what it has. Tests drive `run(scan)`
against a database whose insert trigger aborts, and assert that the scan fails
instead of hanging and that the goroutine count returns to its pre-scan
baseline. Other tests cancel a scan part-way through the walk,
deterministically, by counting the scan's own checks of `ctx.Done()`. They
assert that it stops at the guard, holding a partial size census and a
still-populated record index, with every record intact. Direct tests of
`sendEvent`, the walk workers, `dispatchDirs`, `feedHashJobs`, `hashWorker`
and `hashPhase` cover the remaining cancellation branches of both pools
(2026-08-09, branch `hash-pool-cleanup`,
@@ -469,13 +484,13 @@ are known.
are gone, so the deferred `db.Close()`, and with it the SQLite WAL checkpoint,
runs when a subcommand fails. `runScan`, `runReport`, `runTrees`,
`loadRecords` and `resolveRoots` return errors instead. The single exit point
is `run` in `main.go`. It maps a `fatalError` to exit 1 and cobra's own
argument and flag errors to exit 2, so a runtime failure is never reported as
a usage error or printed with the usage text. `main_test.go` drives the CLI
in-process and asserts the exit codes from README "Error handling" and the
stdout/stderr split. That includes a fatal error after the database is open
leaving no `-wal` or `-shm` file behind, for `scan`, `report` or `trees`
(2026-08-09, branch `db-close-on-fatal`,
is `run` in `main.go`. It maps a `fatalError`, anything a subcommand returned,
to exit 1, and cobra's own argument and flag errors to exit 2. So a runtime
failure is never reported as a usage error, nor printed with the usage text.
`main_test.go` drives the CLI in-process and asserts the exit codes from
README "Error handling" and the stdout/stderr split. That includes a fatal
error after the database is open leaving no `-wal` or `-shm` file behind, for
`scan`, `report` or `trees` (2026-08-09, branch `db-close-on-fatal`,
https://git.eeqj.de/sneak/sfdupes/issues/4)
- `golangci-lint` updated to v2.12.2 with the canonical config, merged as
@@ -486,17 +501,17 @@ are known.
schema, so they are applied. No new lint findings surfaced (2026-08-09, branch
`golangci-v2.12.2`, https://git.eeqj.de/sneak/sfdupes/issues/3)
- Makefile targets converted to scripts-to-rule-them-all `script/` entrypoints,
like the other managed repos. All 12 entrypoints exist:
- `Makefile` targets converted to scripts-to-rule-them-all `script/`
entrypoints, like the other managed repos. All 12 entrypoints exist:
- `bootstrap`, `setup` and `projectname`;
- `test`, `lint`, `fmt`, `fmt-check` and `check`;
- `docker`, `cibuild`, `precommit` and `install-precommit`.
Every Makefile target is a thin shim over them (2026-07-26, commit
Every `Makefile` target does nothing but call its script (2026-07-26, commit
`3abeacf`, https://git.eeqj.de/sneak/sfdupes/issues/1)
- The binary is the default Make target: plain `make` builds `sfdupes`, where it
used to run `check` plus `build`. `make build` remains as an alias
- The binary is the default `make` target: plain `make` builds `sfdupes`, where
it used to run `check` plus `build`. `make build` remains as an alias
(2026-07-24, branch `make-default-target`)
- Scan-wide phases, concurrent operands and batched updates. All operands seed
@@ -558,9 +573,11 @@ are known.
- `LICENSE` and the README License section. MIT was assumed from house
convention, for the owner to confirm;
- `REPO_POLICIES.md` in the repo root;
- the canonical `.golangci.yml`, with the 150 findings it raised fixed;
- the canonical `.golangci.yml`, with the 150 findings it raised fixed, so
that `make lint` is clean;
- the `Makefile` targets `test`, `lint`, `fmt`, `fmt-check`, `docker` and
`hooks`, and a `check` target that no longer depends on `build`;
`hooks`. `check` depended on `build`, which wrote the binary, and
`make check` must not modify files, so `check` no longer depends on it;
- a test suite, which the policy required with a 30-second timeout and the
conditional `-v` rerun. It covers parsing, grouping, digests, suppression,
hashing and the scan pipeline, at 64% coverage;
@@ -572,7 +589,7 @@ are known.
name, purpose, category, license and author, then Getting Started,
Rationale, TODO, License and Author;
- the stale README non-goal "no git repository setup and no CI" removed;
- the pre-commit hook installed.
- the pre-commit hook, installed with `make hooks` once that target existed.
The code was already `gofmt`-clean, so no standalone formatting commit was
needed (2026-07-23, branch `repo-policy-compliance`)