Author SHA1 Message Date
sneak 8cccf8117f Rewrite TODO.md sentence by sentence (closes #135)
check / check (push) Canceled after 0s
TODO.md is the project's own task list, judged by the same rules as
README.md: short sentences, lists as bullets, nothing a newcomer reads
twice. Every completed step is kept with its date and its issue or
branch, and every fact the old entries recorded is kept too, in short
sentences. The 2026-07-22 policy-compliance checklist is folded into
the step that did the work, and the accepted divergence from the
layout policy gets a section of its own. Workflow, Status and Next
Step keep their rules unchanged.

Model: fable-5-1
2026-10-26 14:44:34 +00:00
+39 -41
View File
@@ -63,10 +63,10 @@ are known.
`trees`. Reading a record whose hash is not 32 bytes is an error (2026-10-26, `trees`. Reading a record whose hash is not 32 bytes is an error (2026-10-26,
https://git.eeqj.de/sneak/sfdupes/issues/134) https://git.eeqj.de/sneak/sfdupes/issues/134)
- The hash phase gives three hashes to every file that the walk records. `head` - Every file the walk records gets three hashes in the hash phase. `head` and
and `tail` are the SHA-256 of its first and last 64 KiB. `content` is the `tail` are the SHA-256 of its first and last 64 KiB. `content` is the SHA-256
SHA-256 of the whole file under 100 MiB, and of 1 MiB at each gigabyte from of the whole file under 100 MiB, and of 1 MiB at each gigabyte from 100 MiB
100 MiB up. Gone are: up. Gone are:
- the 10 MiB and 50 MiB thresholds; - the 10 MiB and 50 MiB thresholds;
- the size census, which counted the sizes of every walked file and of every - the size census, which counted the sizes of every walked file and of every
record outside the operands. A file was hashed only when the census showed record outside the operands. A file was hashed only when the census showed
@@ -156,12 +156,12 @@ are known.
the default excludes only, and `scan --help` lists them (2026-10-17, the default excludes only, and `scan --help` lists them (2026-10-17,
https://git.eeqj.de/sneak/sfdupes/issues/124) https://git.eeqj.de/sneak/sfdupes/issues/124)
- A progress display's rate covers the items completed over the last 5 seconds. - The rate a progress display shows covers the items completed over the last 5
When stderr is not a terminal, it covers the items since the line before. It seconds. When stderr is not a terminal, it covers the items since the line
falls to 0 while nothing completes. A burst after a wait no longer reads as a before. It falls to 0 while nothing completes. A burst after a wait no longer
steady rate of over 1,000 files a second. The time remaining still comes from reads as a steady rate of over 1,000 files a second. The time remaining still
the phase's average. Once no item has completed for 10 seconds, the line shows comes from the phase's average. Once no item has completed for 10 seconds, the
`stalled` and where the phase waits in place of the rate, as word `stalled` and where the phase waits take the rate's place, as
`stalled disk/db`. The places are: `stalled disk/db`. The places are:
- `disk`; - `disk`;
- `db`; - `db`;
@@ -176,12 +176,12 @@ are known.
- the elapsed time; - the elapsed time;
- the ETA, the time remaining. - the ETA, the time remaining.
The content phase had one display for its checks and one for its reads. Both On an 80-column terminal, at counts like those of README's example, the
lost those words on an 80-column terminal, at counts like those of README's content phase's displays for its checks and its reads lost those words. That
example. That happened for most elapsed times and ETAs in minutes. Every happened for most elapsed times and ETAs in minutes. Every other phase kept
other phase kept them whenever the elapsed time and the ETA were both in them whenever the elapsed time and the ETA were both in minutes. The content
minutes. The content phase writes its last partial batch, the records not phase writes its last partial batch, the records not yet written, under a
yet written, under a display of its own (2026-10-16, display of its own (2026-10-16,
https://git.eeqj.de/sneak/sfdupes/issues/125) https://git.eeqj.de/sneak/sfdupes/issues/125)
- New subcommand `info` prints the database's details on stdout, one per line as - New subcommand `info` prints the database's details on stdout, one per line as
@@ -231,8 +231,7 @@ are known.
on a slow reader (2026-10-16, https://git.eeqj.de/sneak/sfdupes/issues/113) on a slow reader (2026-10-16, https://git.eeqj.de/sneak/sfdupes/issues/113)
- Each `scan` phase's progress display names the phase and its place among the - Each `scan` phase's progress display names the phase and its place among the
five, as `[2/5] walk`. It runs from the moment that the phase starts. It five, as `[2/5] walk`. It runs from the moment the phase starts and shows:
shows:
- the progress; - the progress;
- the elapsed time; - the elapsed time;
- the ETA, the time remaining. - the ETA, the time remaining.
@@ -286,8 +285,8 @@ are known.
- With `-x`, an operand under another operand is no longer dropped when a - With `-x`, an operand under another operand is no longer dropped when a
directory between the two, the inner one included, is on a different directory between the two, the inner one included, is on a different
filesystem than the outer one. The inner operand is then walked on its own. filesystem than the outer one. The inner operand is then walked on its own.
For a regular-file operand the check starts at its directory. The records of For a regular-file operand the directories checked start at its directory. The
files gone from below it are deleted (2026-10-14, records of files gone from below it are deleted (2026-10-14,
https://git.eeqj.de/sneak/sfdupes/issues/106) https://git.eeqj.de/sneak/sfdupes/issues/106)
- With `-x`, `scan` keeps the records below a directory on another filesystem - With `-x`, `scan` keeps the records below a directory on another filesystem
@@ -295,8 +294,8 @@ are known.
census (2026-10-14, https://git.eeqj.de/sneak/sfdupes/issues/99) census (2026-10-14, https://git.eeqj.de/sneak/sfdupes/issues/99)
- The hash and content phases open each file without blocking and check that it - The hash and content phases open each file without blocking and check that it
is still a regular file. A file that is now a FIFO or device node is warned is still a regular file. A file replaced by a FIFO or device node after it was
about and skipped instead of hanging the scan (2026-10-14, statted is warned about and skipped instead of hanging the scan (2026-10-14,
https://git.eeqj.de/sneak/sfdupes/issues/100) https://git.eeqj.de/sneak/sfdupes/issues/100)
- The scan lock file is named from the database's path with symlinks resolved, - The scan lock file is named from the database's path with symlinks resolved,
@@ -450,8 +449,8 @@ are known.
phase. The content phase ran after the update phase. It read a file for phase. The content phase ran after the update phase. It read a file for
`content` only when its size, `head` and `tail` matched another record's, from `content` only when its size, `head` and `tail` matched another record's, from
this scan or an earlier one. `content` was the whole file below 50 MiB, and this scan or an earlier one. `content` was the whole file below 50 MiB, and
gigabyte-spaced 1 MiB samples at or above. It never read a file gone or gigabyte-spaced 1 MiB samples at or above. A file gone or changed since its
changed since its record was written. `report` and `trees` left out any record record was written was never read. `report` and `trees` left out any record
without a `content` hash. The `content` column is part of the version 1 without a `content` hash. The `content` column is part of the version 1
schema. Superseded by https://git.eeqj.de/sneak/sfdupes/issues/132 schema. Superseded by https://git.eeqj.de/sneak/sfdupes/issues/132
(2026-09-22, branch `next`, https://git.eeqj.de/sneak/sfdupes/issues/61) (2026-09-22, branch `next`, https://git.eeqj.de/sneak/sfdupes/issues/61)
@@ -484,9 +483,9 @@ are known.
- The Docker build stage's prerequisites were installed by running - The Docker build stage's prerequisites were installed by running
`script/bootstrap` instead of `apk add --no-cache make` inline. `script/bootstrap` instead of `apk add --no-cache make` inline.
`script/verify-linter-pin` failed the build unless the linter copied from the `script/verify-linter-pin` failed the build unless the linter copied from the
lint stage was the version that `script/bootstrap` pinned. Builds then took up lint stage was the version `script/bootstrap` pinned. Builds then took up to
to 5m14s from an empty cache, mostly in a `chown -R` of the module cache, 5m14s from an empty cache, mostly in a `chown -R` of the module cache, filed
filed as https://git.eeqj.de/sneak/sfdupes/issues/43. Since as https://git.eeqj.de/sneak/sfdupes/issues/43. Since
https://git.eeqj.de/sneak/sfdupes/issues/95 the build stage installs `git` and https://git.eeqj.de/sneak/sfdupes/issues/95 the build stage installs `git` and
`make` with `apk add --no-cache` and only compiles. The `lint` and `test` `make` with `apk add --no-cache` and only compiles. The `lint` and `test`
phases are the gates (2026-08-09, branch `dockerfile-bootstrap`, phases are the gates (2026-08-09, branch `dockerfile-bootstrap`,
@@ -546,18 +545,17 @@ are known.
- the whole database layer. - the whole database layer.
The walk pool gets the same treatment, plus a `ctx.Err()` guard after the The walk pool gets the same treatment, plus a `ctx.Err()` guard after the
walk. A cancelled walk yields a partial size census, and every file that it walk. A cancelled walk yields a partial size census, and every file it never
never reached looks vanished to the update phase. That phase's own `BeginTx` reached looks vanished to the update phase. That phase's own `BeginTx` also
also fails on the cancelled context before deleting anything. The guard is fails on the cancelled context before deleting anything. The guard is still
still the barrier once an interrupted scan may commit what it has. Tests the barrier once an interrupted scan may commit what it has. Tests drive
drive `run(scan)` against a database whose insert trigger aborts. They `run(scan)` against a database whose insert trigger aborts. They assert that
assert that the scan fails instead of hanging, and that the goroutine count the scan fails instead of hanging, and that the goroutine count returns to
returns to its pre-scan baseline. Other tests cancel a scan part-way through its pre-scan baseline. Other tests cancel a scan part-way through the walk,
the walk, deterministically, by counting the scan's own checks of deterministically, by counting the scan's own checks of `ctx.Done()`. They
`ctx.Done()`. They assert that it stops at the guard with every record assert that it stops at the guard with every record intact. It then holds a
intact. It then holds a partial size census and a still-populated record partial size census and a still-populated record index. Direct tests cover
index. Direct tests cover the remaining cancellation branches of both pools. the remaining cancellation branches of both pools. They test:
They test:
- `sendEvent`; - `sendEvent`;
- the walk workers; - the walk workers;
- `dispatchDirs`; - `dispatchDirs`;
@@ -674,8 +672,8 @@ are known.
- `LICENSE` and the README License section. MIT was assumed from house - `LICENSE` and the README License section. MIT was assumed from house
convention, for the owner to confirm; convention, for the owner to confirm;
- `REPO_POLICIES.md` in the repo root; - `REPO_POLICIES.md` in the repo root;
- the canonical `.golangci.yml`, with its 150 findings fixed, so that - the canonical `.golangci.yml`, with the 150 findings it raised fixed, so
`make lint` is clean; that `make lint` is clean;
- the `Makefile` targets the policy requires: - the `Makefile` targets the policy requires:
- `test` and `lint`; - `test` and `lint`;
- `fmt` and `fmt-check`; - `fmt` and `fmt-check`;