Author SHA1 Message Date
sneak 207bd918c2 Rewrite TODO.md sentence by sentence (closes #135)
check / check (push) Canceled after 0s
Rewrites `TODO.md` from scratch for
#135, by the rules applied to
`README.md` in #140. It is the
project's own task list, so every completed step keeps its date, its
issue or branch and every fact that the old entry recorded, in short
sentences, with lists as bullets. An entry superseded later says so in
one sentence.

`TODO.md` records history, so its entries were not corrected against the
code. The 2026-10-17 release-rule entry is known to be stale. One product
name in the 2026-10-15 entry is replaced by a description.

This file carries the `TODO.md` entry of
#140 word for word, so the
conflict once that merges is resolved by taking this file whole.

Model: fable-5-1
2026-10-26 17:14:55 +00:00
+41 -39
View File
@@ -63,10 +63,10 @@ are known.
`trees`. Reading a record whose hash is not 32 bytes is an error (2026-10-26, `trees`. Reading a record whose hash is not 32 bytes is an error (2026-10-26,
https://git.eeqj.de/sneak/sfdupes/issues/134) https://git.eeqj.de/sneak/sfdupes/issues/134)
- Every file the walk records gets three hashes in the hash phase. `head` and - The hash phase gives three hashes to every file that the walk records. `head`
`tail` are the SHA-256 of its first and last 64 KiB. `content` is the SHA-256 and `tail` are the SHA-256 of its first and last 64 KiB. `content` is the
of the whole file under 100 MiB, and of 1 MiB at each gigabyte from 100 MiB SHA-256 of the whole file under 100 MiB, and of 1 MiB at each gigabyte from
up. Gone are: 100 MiB up. Gone are:
- the 10 MiB and 50 MiB thresholds; - the 10 MiB and 50 MiB thresholds;
- the size census, which counted the sizes of every walked file and of every - the size census, which counted the sizes of every walked file and of every
record outside the operands. A file was hashed only when the census showed record outside the operands. A file was hashed only when the census showed
@@ -156,12 +156,12 @@ are known.
the default excludes only, and `scan --help` lists them (2026-10-17, the default excludes only, and `scan --help` lists them (2026-10-17,
https://git.eeqj.de/sneak/sfdupes/issues/124) https://git.eeqj.de/sneak/sfdupes/issues/124)
- The rate a progress display shows covers the items completed over the last 5 - A progress display's rate covers the items completed over the last 5 seconds.
seconds. When stderr is not a terminal, it covers the items since the line When stderr is not a terminal, it covers the items since the line before. It
before. It falls to 0 while nothing completes. A burst after a wait no longer falls to 0 while nothing completes. A burst after a wait no longer reads as a
reads as a steady rate of over 1,000 files a second. The time remaining still steady rate of over 1,000 files a second. The time remaining still comes from
comes from the phase's average. Once no item has completed for 10 seconds, the the phase's average. Once no item has completed for 10 seconds, the line shows
word `stalled` and where the phase waits take the rate's place, as `stalled` and where the phase waits in place of the rate, as
`stalled disk/db`. The places are: `stalled disk/db`. The places are:
- `disk`; - `disk`;
- `db`; - `db`;
@@ -176,12 +176,12 @@ are known.
- the elapsed time; - the elapsed time;
- the ETA, the time remaining. - the ETA, the time remaining.
On an 80-column terminal, at counts like those of README's example, the The content phase had one display for its checks and one for its reads. Both
content phase's displays for its checks and its reads lost those words. That lost those words on an 80-column terminal, at counts like those of README's
happened for most elapsed times and ETAs in minutes. Every other phase kept example. That happened for most elapsed times and ETAs in minutes. Every
them whenever the elapsed time and the ETA were both in minutes. The content other phase kept them whenever the elapsed time and the ETA were both in
phase writes its last partial batch, the records not yet written, under a minutes. The content phase writes its last partial batch, the records not
display of its own (2026-10-16, yet written, under a display of its own (2026-10-16,
https://git.eeqj.de/sneak/sfdupes/issues/125) https://git.eeqj.de/sneak/sfdupes/issues/125)
- New subcommand `info` prints the database's details on stdout, one per line as - New subcommand `info` prints the database's details on stdout, one per line as
@@ -231,7 +231,8 @@ are known.
on a slow reader (2026-10-16, https://git.eeqj.de/sneak/sfdupes/issues/113) on a slow reader (2026-10-16, https://git.eeqj.de/sneak/sfdupes/issues/113)
- Each `scan` phase's progress display names the phase and its place among the - Each `scan` phase's progress display names the phase and its place among the
five, as `[2/5] walk`. It runs from the moment the phase starts and shows: five, as `[2/5] walk`. It runs from the moment that the phase starts. It
shows:
- the progress; - the progress;
- the elapsed time; - the elapsed time;
- the ETA, the time remaining. - the ETA, the time remaining.
@@ -285,8 +286,8 @@ are known.
- With `-x`, an operand under another operand is no longer dropped when a - With `-x`, an operand under another operand is no longer dropped when a
directory between the two, the inner one included, is on a different directory between the two, the inner one included, is on a different
filesystem than the outer one. The inner operand is then walked on its own. filesystem than the outer one. The inner operand is then walked on its own.
For a regular-file operand the directories checked start at its directory. The For a regular-file operand the check starts at its directory. The records of
records of files gone from below it are deleted (2026-10-14, files gone from below it are deleted (2026-10-14,
https://git.eeqj.de/sneak/sfdupes/issues/106) https://git.eeqj.de/sneak/sfdupes/issues/106)
- With `-x`, `scan` keeps the records below a directory on another filesystem - With `-x`, `scan` keeps the records below a directory on another filesystem
@@ -294,8 +295,8 @@ are known.
census (2026-10-14, https://git.eeqj.de/sneak/sfdupes/issues/99) census (2026-10-14, https://git.eeqj.de/sneak/sfdupes/issues/99)
- The hash and content phases open each file without blocking and check that it - The hash and content phases open each file without blocking and check that it
is still a regular file. A file replaced by a FIFO or device node after it was is still a regular file. A file that is now a FIFO or device node is warned
statted is warned about and skipped instead of hanging the scan (2026-10-14, about and skipped instead of hanging the scan (2026-10-14,
https://git.eeqj.de/sneak/sfdupes/issues/100) https://git.eeqj.de/sneak/sfdupes/issues/100)
- The scan lock file is named from the database's path with symlinks resolved, - The scan lock file is named from the database's path with symlinks resolved,
@@ -449,8 +450,8 @@ are known.
phase. The content phase ran after the update phase. It read a file for phase. The content phase ran after the update phase. It read a file for
`content` only when its size, `head` and `tail` matched another record's, from `content` only when its size, `head` and `tail` matched another record's, from
this scan or an earlier one. `content` was the whole file below 50 MiB, and this scan or an earlier one. `content` was the whole file below 50 MiB, and
gigabyte-spaced 1 MiB samples at or above. A file gone or changed since its gigabyte-spaced 1 MiB samples at or above. It never read a file gone or
record was written was never read. `report` and `trees` left out any record changed since its record was written. `report` and `trees` left out any record
without a `content` hash. The `content` column is part of the version 1 without a `content` hash. The `content` column is part of the version 1
schema. Superseded by https://git.eeqj.de/sneak/sfdupes/issues/132 schema. Superseded by https://git.eeqj.de/sneak/sfdupes/issues/132
(2026-09-22, branch `next`, https://git.eeqj.de/sneak/sfdupes/issues/61) (2026-09-22, branch `next`, https://git.eeqj.de/sneak/sfdupes/issues/61)
@@ -483,9 +484,9 @@ are known.
- The Docker build stage's prerequisites were installed by running - The Docker build stage's prerequisites were installed by running
`script/bootstrap` instead of `apk add --no-cache make` inline. `script/bootstrap` instead of `apk add --no-cache make` inline.
`script/verify-linter-pin` failed the build unless the linter copied from the `script/verify-linter-pin` failed the build unless the linter copied from the
lint stage was the version `script/bootstrap` pinned. Builds then took up to lint stage was the version that `script/bootstrap` pinned. Builds then took up
5m14s from an empty cache, mostly in a `chown -R` of the module cache, filed to 5m14s from an empty cache, mostly in a `chown -R` of the module cache,
as https://git.eeqj.de/sneak/sfdupes/issues/43. Since filed as https://git.eeqj.de/sneak/sfdupes/issues/43. Since
https://git.eeqj.de/sneak/sfdupes/issues/95 the build stage installs `git` and https://git.eeqj.de/sneak/sfdupes/issues/95 the build stage installs `git` and
`make` with `apk add --no-cache` and only compiles. The `lint` and `test` `make` with `apk add --no-cache` and only compiles. The `lint` and `test`
phases are the gates (2026-08-09, branch `dockerfile-bootstrap`, phases are the gates (2026-08-09, branch `dockerfile-bootstrap`,
@@ -545,17 +546,18 @@ are known.
- the whole database layer. - the whole database layer.
The walk pool gets the same treatment, plus a `ctx.Err()` guard after the The walk pool gets the same treatment, plus a `ctx.Err()` guard after the
walk. A cancelled walk yields a partial size census, and every file it never walk. A cancelled walk yields a partial size census, and every file that it
reached looks vanished to the update phase. That phase's own `BeginTx` also never reached looks vanished to the update phase. That phase's own `BeginTx`
fails on the cancelled context before deleting anything. The guard is still also fails on the cancelled context before deleting anything. The guard is
the barrier once an interrupted scan may commit what it has. Tests drive still the barrier once an interrupted scan may commit what it has. Tests
`run(scan)` against a database whose insert trigger aborts. They assert that drive `run(scan)` against a database whose insert trigger aborts. They
the scan fails instead of hanging, and that the goroutine count returns to assert that the scan fails instead of hanging, and that the goroutine count
its pre-scan baseline. Other tests cancel a scan part-way through the walk, returns to its pre-scan baseline. Other tests cancel a scan part-way through
deterministically, by counting the scan's own checks of `ctx.Done()`. They the walk, deterministically, by counting the scan's own checks of
assert that it stops at the guard with every record intact. It then holds a `ctx.Done()`. They assert that it stops at the guard with every record
partial size census and a still-populated record index. Direct tests cover intact. It then holds a partial size census and a still-populated record
the remaining cancellation branches of both pools. They test: index. Direct tests cover the remaining cancellation branches of both pools.
They test:
- `sendEvent`; - `sendEvent`;
- the walk workers; - the walk workers;
- `dispatchDirs`; - `dispatchDirs`;
@@ -672,8 +674,8 @@ are known.
- `LICENSE` and the README License section. MIT was assumed from house - `LICENSE` and the README License section. MIT was assumed from house
convention, for the owner to confirm; convention, for the owner to confirm;
- `REPO_POLICIES.md` in the repo root; - `REPO_POLICIES.md` in the repo root;
- the canonical `.golangci.yml`, with the 150 findings it raised fixed, so - the canonical `.golangci.yml`, with its 150 findings fixed, so that
that `make lint` is clean; `make lint` is clean;
- the `Makefile` targets the policy requires: - the `Makefile` targets the policy requires:
- `test` and `lint`; - `test` and `lint`;
- `fmt` and `fmt-check`; - `fmt` and `fmt-check`;