1 Commits
Author SHA1 Message Date
sneak c64a3c416d Rewrite TODO.md sentence by sentence (closes #135)
check / check (push) Canceled after 0s
TODO.md is the project's own task list, judged by the same rules as
README.md: short sentences, lists as bullets, nothing a newcomer reads
twice. Every completed step is kept with its date and its issue or
branch, and every fact the old entries recorded is kept too, in short
sentences. The 2026-07-22 policy-compliance checklist is folded into
the step that did the work, and the accepted divergence from the
layout policy gets a section of its own. Workflow, Status and Next
Step keep their rules unchanged.

Model: fable-5-1
2026-10-26 08:18:11 +00:00
+77 -60
View File
@@ -39,7 +39,8 @@ are known.
it. It leads with what the program is and how to use it. A reference section it. It leads with what the program is and how to use it. A reference section
covers the settings, the database, the output formats and the exit codes; covers the settings, the database, the output formats and the exit codes;
development and testing come last. Every statement was checked against the development and testing come last. Every statement was checked against the
code on `next` (2026-10-26, https://git.eeqj.de/sneak/sfdupes/issues/135) code on `next`. Three comments in `main_test.go` cite "Settings", where their
rules now live (2026-10-26, https://git.eeqj.de/sneak/sfdupes/issues/135)
- A file or directory removed while the scan runs is not an error, in the walk - A file or directory removed while the scan runs is not an error, in the walk
or in the hash phase. The scan prints no line for it and does not count it as or in the hash phase. The scan prints no line for it and does not count it as
@@ -62,7 +63,9 @@ are known.
of the whole file under 100 MiB, and of 1 MiB at each gigabyte from 100 MiB of the whole file under 100 MiB, and of 1 MiB at each gigabyte from 100 MiB
up. Gone are: up. Gone are:
- the 10 MiB and 50 MiB thresholds; - the 10 MiB and 50 MiB thresholds;
- the size census; - the size census: the sizes of every walked file and of every record
outside the operands, by which a file was hashed only when another file
shared its size;
- the records without hashes; - the records without hashes;
- the content phase. - the content phase.
@@ -107,20 +110,20 @@ are known.
- `scan` never records or reads anything on a filesystem that is not persistent: - `scan` never records or reads anything on a filesystem that is not persistent:
`tmpfs`, `proc`, `sysfs` and the other kernel or memory-backed types. It `tmpfs`, `proc`, `sysfs` and the other kernel or memory-backed types. It
matches them by device ID. As the scan starts, it lists the mounted matches them by device ID, however the walk or the content phase reaches a
filesystems of those types, from `/proc/self/mountinfo` on Linux and from the path. As the scan starts, it lists the mounted filesystems of those types,
mount list on macOS. No flag turns that off. Below an operand, the walk does from `/proc/self/mountinfo` on Linux and from the mount list on macOS. No flag
not enter the default excludes either, so `sfdupes scan /` skips the kernel's turns that off. Below an operand, the walk does not enter the default excludes
view of the running system and the temporary directories. On Linux those are either, so `sfdupes scan /` skips the kernel's view of the running system and
`/proc`, `/sys`, `/dev`, `/run`, `/tmp` and `/var/tmp`. On macOS they are the temporary directories. On Linux those are `/proc`, `/sys`, `/dev`, `/run`,
`/dev`, `/private/tmp`, `/private/var/tmp`, `/private/var/vm`, `/tmp` and `/var/tmp`. On macOS they are `/dev`, `/private/tmp`,
`/System/Volumes/VM` and `/System/Volumes/Data`. A Windows list exists in the `/private/var/tmp`, `/private/var/vm`, `/System/Volumes/VM` and
code but is not used. An operand at or inside a default exclude is still `/System/Volumes/Data`. A Windows list exists in the code but is not used. An
walked. The scan prints a line for each path it leaves out, without counting operand at or inside a default exclude is still walked. The scan prints a line
it as skipped, and the records there count as outside the scanned operands, as for each path it leaves out, without counting it as skipped, and the records
for `-x`. `--no-default-excludes` turns off the default excludes only, and there count as outside the scanned operands, as for `-x`.
`scan --help` lists them (2026-10-17, `--no-default-excludes` turns off the default excludes only, and `scan --help`
https://git.eeqj.de/sneak/sfdupes/issues/124) lists them (2026-10-17, https://git.eeqj.de/sneak/sfdupes/issues/124)
- The rate a progress display shows covers the items completed over the last 5 - The rate a progress display shows covers the items completed over the last 5
seconds, or since the line before when stderr is not a terminal. It falls to 0 seconds, or since the line before when stderr is not a terminal. It falls to 0
@@ -129,14 +132,15 @@ are known.
phase's average. Once no item has completed for 10 seconds, the word `stalled` phase's average. Once no item has completed for 10 seconds, the word `stalled`
and where the phase waits take the rate's place, as `stalled disk/db`. The and where the phase waits take the rate's place, as `stalled disk/db`. The
places are `disk`, `db`, `out` for stdout and `cpu`, or two of them for a places are `disk`, `db`, `out` for stdout and `cpu`, or two of them for a
phase that may wait on either. Each phase gives its word where it is defined. phase that may wait on either. Each phase's word is set in the code that
A line too wide for the terminal loses its bar first, then its rate or those defines the phase. A line too wide for the terminal loses its bar first, then
words, so the phase, progress, elapsed time and ETA stay whole. On an its rate or those words, so the phase, progress, elapsed time and ETA stay
80-column terminal, at counts like those of README's example, the content whole. On an 80-column terminal, at counts like those of README's example, the
phase's checks and reads lose those words at most times in minutes. Every content phase's `checks` and `reads` lines were too wide for those words. That
other phase keeps them whenever the elapsed time and the ETA are both in held at most elapsed times and ETAs in minutes. Every other phase kept them
minutes. The content phase's last partial batch is written under a display of whenever the elapsed time and the ETA were both in minutes. The content
its own (2026-10-16, https://git.eeqj.de/sneak/sfdupes/issues/125) phase's last partial batch is written under a display of its own (2026-10-16,
https://git.eeqj.de/sneak/sfdupes/issues/125)
- New subcommand `info` prints the database's details on stdout, one per line as - New subcommand `info` prints the database's details on stdout, one per line as
a name, a tab and the value: a name, a tab and the value:
@@ -189,8 +193,8 @@ are known.
narrow, also after the terminal is made narrower during a phase. A line too narrow, also after the terminal is made narrower during a phase. A line too
wide leaves out its rate, then its bar, keeping the phase, progress, elapsed wide leaves out its rate, then its bar, keeping the phase, progress, elapsed
time and ETA whole. It is cut short only when those alone do not fit, as below time and ETA whole. It is cut short only when those alone do not fit, as below
about 70 columns late in a large scan. In plain mode the line is printed every about 70 columns late in a large scan. When stderr is not a terminal, the line
5 seconds whether or not an item completed (2026-10-16, is printed every 5 seconds whether or not an item completed (2026-10-16,
https://git.eeqj.de/sneak/sfdupes/issues/112) https://git.eeqj.de/sneak/sfdupes/issues/112)
- `scan`, `report` and `trees` first print a line on stderr naming the database - `scan`, `report` and `trees` first print a line on stderr naming the database
@@ -314,8 +318,8 @@ are known.
`files` table is refused with a clear schema-version error (2026-10-04, `files` table is refused with a clear schema-version error (2026-10-04,
https://git.eeqj.de/sneak/sfdupes/issues/11) https://git.eeqj.de/sneak/sfdupes/issues/11)
- README documents install, Docker, a daily cron scan and how to read and check - README documents install, Docker, a daily `cron` scan and how to read and
the reports (2026-10-04, https://git.eeqj.de/sneak/sfdupes/issues/54) check the reports (2026-10-04, https://git.eeqj.de/sneak/sfdupes/issues/54)
- The `Dockerfile` build stage kept the Go module cache out of `builder`'s home - The `Dockerfile` build stage kept the Go module cache out of `builder`'s home
and copied the sources with `--chown`, so no `chown -R` walked them. Since and copied the sources with `--chown`, so no `chown -R` walked them. Since
@@ -359,12 +363,12 @@ are known.
and warning paths. The root directory's path is `/` (2026-10-03, and warning paths. The root directory's path is `/` (2026-10-03,
https://git.eeqj.de/sneak/sfdupes/issues/7) https://git.eeqj.de/sneak/sfdupes/issues/7)
- A plain `docker build .` stamps the git tag or short commit instead of `dev`. - A plain `docker build .` stamps the `git` tag or short commit instead of
`.dockerignore` sends `.git` without `.git/config`. The build stage stamps the `dev`. `.dockerignore` sends `.git` without `.git/config`. The build stage
`VERSION` build argument, else `git describe --tags --always`. It fails if the stamps the `VERSION` build argument, else `git describe --tags --always`. It
context carries `.git` and the version is still empty, `dev` or `unknown`. CI fails if the context carries `.git` and the version is still empty, `dev` or
checks out the full history (`fetch-depth: 0`), so it stamps the same value as `unknown`. CI checks out the full history (`fetch-depth: 0`), so it stamps the
`make build` (2026-10-02, branch `next`, same value as `make build` (2026-10-02, branch `next`,
https://git.eeqj.de/sneak/sfdupes/issues/67) https://git.eeqj.de/sneak/sfdupes/issues/67)
- The 1 KiB end-window sampling was replaced with the hashes chosen by file - The 1 KiB end-window sampling was replaced with the hashes chosen by file
@@ -438,11 +442,20 @@ are known.
installing. On a mismatch it printed both paths and both versions and exited installing. On a mismatch it printed both paths and both versions and exited
non-zero; it did not reorder `PATH` or delete anyone's binary. The `--version` non-zero; it did not reorder `PATH` or delete anyone's binary. The `--version`
call kept its stderr and was bounded by `timeout(1)` where that exists. `git`, call kept its stderr and was bounded by `timeout(1)` where that exists. `git`,
`make` and `go` kept presence-only checks. `make check` and `make lint` were `make` and `go` kept presence-only checks. It was verified by bootstrapping
clean at v2.12.2, so v2.10.1 had not been hiding findings on `main`. this host from v2.10.1 to v2.12.2, and again to a no-op. Stub runs of the
Superseded by https://git.eeqj.de/sneak/sfdupes/issues/46: the linter is no script under `dash` covered:
longer installed on a host (2026-08-09, branch `bootstrap-version-check`, - a thirteen-input version-parse matrix;
https://git.eeqj.de/sneak/sfdupes/issues/24) - a shadowed install, which must exit non-zero;
- an install destination not on `PATH`;
- `GOBIN` set;
- a wedged binary, which must hit the timeout.
`make check` and `make lint` were clean at v2.12.2, so v2.10.1 had not been
hiding findings on `main`. Superseded by
https://git.eeqj.de/sneak/sfdupes/issues/46: the linter is no longer
installed on a host (2026-08-09, branch `bootstrap-version-check`,
https://git.eeqj.de/sneak/sfdupes/issues/24)
- The hash worker pool unwinds on the error path. The pool is an owned, - The hash worker pool unwinds on the error path. The pool is an owned,
context-aware `hashPool`. Every blocking send in the feeder and the workers context-aware `hashPool`. Every blocking send in the feeder and the workers
@@ -454,12 +467,14 @@ are known.
walk pool gets the same treatment, plus a `ctx.Err()` guard after the walk. A walk pool gets the same treatment, plus a `ctx.Err()` guard after the walk. A
cancelled walk yields a partial size census, and every file it never reached cancelled walk yields a partial size census, and every file it never reached
looks vanished to the update phase. That phase's own `BeginTx` also fails on looks vanished to the update phase. That phase's own `BeginTx` also fails on
the cancelled context before deleting anything, but the guard is the barrier the cancelled context before deleting anything. The guard is still the barrier
that still holds once an interrupted scan may commit what it has. Tests drive once an interrupted scan may commit what it has. Tests drive `run(scan)`
`run(scan)` against a database whose insert trigger aborts, and assert that against a database whose insert trigger aborts, and assert that the scan fails
the scan fails instead of hanging and that the goroutine count returns to its instead of hanging and that the goroutine count returns to its pre-scan
pre-scan baseline. Other tests cancel a scan part-way through the walk and baseline. Other tests cancel a scan part-way through the walk,
assert that it stops at the guard with every record intact. Direct tests of deterministically, by counting the scan's own checks of `ctx.Done()`. They
assert that it stops at the guard, holding a partial size census and a
still-populated record index, with every record intact. Direct tests of
`sendEvent`, the walk workers, `dispatchDirs`, `feedHashJobs`, `hashWorker` `sendEvent`, the walk workers, `dispatchDirs`, `feedHashJobs`, `hashWorker`
and `hashPhase` cover the remaining cancellation branches of both pools and `hashPhase` cover the remaining cancellation branches of both pools
(2026-08-09, branch `hash-pool-cleanup`, (2026-08-09, branch `hash-pool-cleanup`,
@@ -469,13 +484,13 @@ are known.
are gone, so the deferred `db.Close()`, and with it the SQLite WAL checkpoint, are gone, so the deferred `db.Close()`, and with it the SQLite WAL checkpoint,
runs when a subcommand fails. `runScan`, `runReport`, `runTrees`, runs when a subcommand fails. `runScan`, `runReport`, `runTrees`,
`loadRecords` and `resolveRoots` return errors instead. The single exit point `loadRecords` and `resolveRoots` return errors instead. The single exit point
is `run` in `main.go`. It maps a `fatalError` to exit 1 and cobra's own is `run` in `main.go`. It maps a `fatalError`, anything a subcommand returned,
argument and flag errors to exit 2, so a runtime failure is never reported as to exit 1, and cobra's own argument and flag errors to exit 2. So a runtime
a usage error or printed with the usage text. `main_test.go` drives the CLI failure is never reported as a usage error, nor printed with the usage text.
in-process and asserts the exit codes from README "Error handling" and the `main_test.go` drives the CLI in-process and asserts the exit codes from
stdout/stderr split. That includes a fatal error after the database is open README "Error handling" and the stdout/stderr split. That includes a fatal
leaving no `-wal` or `-shm` file behind, for `scan`, `report` or `trees` error after the database is open leaving no `-wal` or `-shm` file behind, for
(2026-08-09, branch `db-close-on-fatal`, `scan`, `report` or `trees` (2026-08-09, branch `db-close-on-fatal`,
https://git.eeqj.de/sneak/sfdupes/issues/4) https://git.eeqj.de/sneak/sfdupes/issues/4)
- `golangci-lint` updated to v2.12.2 with the canonical config, merged as - `golangci-lint` updated to v2.12.2 with the canonical config, merged as
@@ -486,17 +501,17 @@ are known.
schema, so they are applied. No new lint findings surfaced (2026-08-09, branch schema, so they are applied. No new lint findings surfaced (2026-08-09, branch
`golangci-v2.12.2`, https://git.eeqj.de/sneak/sfdupes/issues/3) `golangci-v2.12.2`, https://git.eeqj.de/sneak/sfdupes/issues/3)
- Makefile targets converted to scripts-to-rule-them-all `script/` entrypoints, - `Makefile` targets converted to scripts-to-rule-them-all `script/`
like the other managed repos. All 12 entrypoints exist: entrypoints, like the other managed repos. All 12 entrypoints exist:
- `bootstrap`, `setup` and `projectname`; - `bootstrap`, `setup` and `projectname`;
- `test`, `lint`, `fmt`, `fmt-check` and `check`; - `test`, `lint`, `fmt`, `fmt-check` and `check`;
- `docker`, `cibuild`, `precommit` and `install-precommit`. - `docker`, `cibuild`, `precommit` and `install-precommit`.
Every Makefile target is a thin shim over them (2026-07-26, commit Every `Makefile` target does nothing but call its script (2026-07-26, commit
`3abeacf`, https://git.eeqj.de/sneak/sfdupes/issues/1) `3abeacf`, https://git.eeqj.de/sneak/sfdupes/issues/1)
- The binary is the default Make target: plain `make` builds `sfdupes`, where it - The binary is the default `make` target: plain `make` builds `sfdupes`, where
used to run `check` plus `build`. `make build` remains as an alias it used to run `check` plus `build`. `make build` remains as an alias
(2026-07-24, branch `make-default-target`) (2026-07-24, branch `make-default-target`)
- Scan-wide phases, concurrent operands and batched updates. All operands seed - Scan-wide phases, concurrent operands and batched updates. All operands seed
@@ -558,9 +573,11 @@ are known.
- `LICENSE` and the README License section. MIT was assumed from house - `LICENSE` and the README License section. MIT was assumed from house
convention, for the owner to confirm; convention, for the owner to confirm;
- `REPO_POLICIES.md` in the repo root; - `REPO_POLICIES.md` in the repo root;
- the canonical `.golangci.yml`, with the 150 findings it raised fixed; - the canonical `.golangci.yml`, with the 150 findings it raised fixed, so
that `make lint` is clean;
- the `Makefile` targets `test`, `lint`, `fmt`, `fmt-check`, `docker` and - the `Makefile` targets `test`, `lint`, `fmt`, `fmt-check`, `docker` and
`hooks`, and a `check` target that no longer depends on `build`; `hooks`. `check` depended on `build`, which wrote the binary, and
`make check` must not modify files, so `check` no longer depends on it;
- a test suite, which the policy required with a 30-second timeout and the - a test suite, which the policy required with a 30-second timeout and the
conditional `-v` rerun. It covers parsing, grouping, digests, suppression, conditional `-v` rerun. It covers parsing, grouping, digests, suppression,
hashing and the scan pipeline, at 64% coverage; hashing and the scan pipeline, at 64% coverage;
@@ -572,7 +589,7 @@ are known.
name, purpose, category, license and author, then Getting Started, name, purpose, category, license and author, then Getting Started,
Rationale, TODO, License and Author; Rationale, TODO, License and Author;
- the stale README non-goal "no git repository setup and no CI" removed; - the stale README non-goal "no git repository setup and no CI" removed;
- the pre-commit hook installed. - the pre-commit hook, installed with `make hooks` once that target existed.
The code was already `gofmt`-clean, so no standalone formatting commit was The code was already `gofmt`-clean, so no standalone formatting commit was
needed (2026-07-23, branch `repo-policy-compliance`) needed (2026-07-23, branch `repo-policy-compliance`)