next into main #110

Merged
sneak merged 12 commits from next into main 2026-10-22 18:44:21 +02:00
12 Commits
Author SHA1 Message Date
clawbot f4ddb58ae1 Keep modules, lint cache and system packages in check builds (closes #109)
check / check (push) Canceled after 0s
Re-vendors REPO_POLICIES.md and script/cibuild, docker, lint and test
from sneak/prompts at 0653579, on sneak's ruling in
sneak/prompts#130. In the Dockerfile's lint,
test and builder stages, the system-package install and the module
download now come before ARG CACHEBUST, so Docker keeps those layers
until the base image, the install line, go.mod or go.sum changes.
golangci-lint run also mounts golangci-lint's cache. Every step that
lints, tests or compiles still runs on every check build, and go test
keeps -count=1 and -race. README and TODO.md say the same.

Model: opus-5-5
2026-10-18 19:25:33 +02:00
clawbot 6346319701 Install from @latest, retract v0.0.1, print a go-installed binary's version (closes #116)
check / check (push) Canceled after 0s
README "Install" now leads with go install ...@latest: until 1.0 each
release is a v0.0.N tag on main's tip, so @latest gives that tip once
it has passed the build. It also says what @main and v0.0.1 give and
how to check an installed binary. go.mod retracts v0.0.1, the early
build without a database.

sfdupes --version printed "dev" for every go-installed binary, since
only the linker set Version. When the linker sets none, it now prints
the main module's version from runtime/debug.ReadBuildInfo; "dev" is
left only when neither is known. Version is now empty by default.

The retraction takes effect only once a release carrying it is the
newest.

Model: opus-5-5
2026-10-17 05:44:49 +02:00
clawbot 4e4a94db2b scan: create the database's directory where its path leads (closes #119)
check / check (push) Canceled after 0s
scan created a missing database directory from filepath.Dir of the
path, which cleans it as text and drops each ".." together with the
name before it. After a symlinked directory that is another directory
than the one the system reaches when it opens the file, so scan left an
empty directory beside the symlink and then failed to create the
database. lockScanDatabase and openScanDatabase now both call
createDatabaseDirectory, beside databasePath, which cuts the last
element off the path as text and creates what is missing along it. A
comment in databasePath now says why the working directory has its
symlinks resolved.

Unverified: a missing directory before a ".." in the path is created along the way; no test covers it.

Model: opus-5-5
2026-10-17 04:44:53 +02:00
clawbot dcfe0ea080 scan: never record its own database files (closes #115)
check / check (push) Canceled after 0s
When the database lay under a scanned operand, the walk recorded the
database file, its -wal and -shm files and its .lock file like any other
file, so every rescan updated them and report paired them with empty
files. Once the database and the lock file are open, the scan now notes
their device and inode, and the walk passes over a file with one of
them, as an operand or below one, printing no line and counting
nothing. Matching by device and inode holds through a symlinked
directory, where comparing paths would not. A record an earlier scan
left for one of them is deleted as unverified. README's smoke-test note
on keeping the database out of the tree is gone, as it no longer holds.

Judgement call: walkRules, in exclude.go, carries the noted files to the walk.

Model: opus-5-5
2026-10-17 04:10:50 +02:00
clawbot 003e184f71 scan: skip OS default excludes, and record nothing on filesystems that are not persistent (closes #124)
check / check (push) Canceled after 0s
scan never records or reads anything on a filesystem that is not
persistent (tmpfs, proc, sysfs and the like), however it reaches it; no
flag turns this off. It notes each such filesystem's device ID as it
starts: on Linux from /proc/self/mountinfo, on macOS by lstat of each
listed mount point. Below an operand the walk also skips the default
excludes, a path list per system (Linux, macOS, and Windows, not used
yet); --no-default-excludes turns off only those. A default exclude that
is itself an operand gets no skip line. The tests stop at once, saying
why, when TMPDIR is not on a persistent filesystem.

Judgement call: an unreadable mount list fails the scan, and no flag gets round it.
Judgement call: a file on such a filesystem below an operand gets the same line as a directory.
Judgement call: such an operand that another operand's walk also reaches is named twice.
Unverified: whether macOS's mount list gives the device lstat reports, so macOS keeps lstat, and a hidden mount there can still add another filesystem's device.
Unverified: the macOS code is never built or run here; bind mounts need root to test.

Model: opus-5-5
2026-10-17 03:44:52 +02:00
clawbot 68f6bde5ee Show the progress rate over the last 5 seconds, and where a phase waits after 10 seconds with no item done (closes #125)
check / check (push) Canceled after 0s
The rate was the phase's average, so a burst after a long wait read as
over 1,000 files a second. It now covers the last 5 seconds, or the
time since the previous line off a terminal; the ETA still comes from
the average. Once no item has completed for 10 seconds, the rate gives
way to "stalled" and where the phase waits: disk, db, out (stdout) or
cpu, or two of them, as "stalled disk/db". A line too wide for the
terminal loses its bar first, then its rate or those words, so the
phase, progress, elapsed time and ETA stay whole. The content phase's
last partial batch is now written under a display of its own.

Deviation: the stall words hold no time and name where the phase waits, not what it does.
Judgement call: trees' group phase, which works in memory, says "cpu".

Model: opus-5-5
2026-10-17 01:10:57 +02:00
clawbot e9e09effc6 Add sfdupes info: show the database's details (closes #117)
check / check (push) Canceled after 0s
info prints the database's path and source, the sizes of the database
file and its -wal file, when the database file last changed, the schema
version and the record counts, one per line as a name, a tab and the
value. It opens the database as report does and shows an open and a
count phase on stderr. The schema version and the counts come from one
read-only transaction, the counts from one query.

seedDatabase now closes test databases out of WAL mode, as scan does;
otherwise info's own connection creates an empty -wal file. The open
phase's name and text became constants for the linter, and the scan
command moved to newScanCommand to keep newRootCommand short enough.

Judgement call: the lines are written after the last display ends, so they skip the display's stdout writer.

Model: opus-5-5
2026-10-16 22:41:16 +02:00
clawbot ebe4d4dbeb report: show its phases as N of M, with progress, elapsed time and ETA (closes #114)
check / check (push) Canceled after 0s
report now runs in two phases, open and read, under the progress
display scan and trees use, until the summary line. open covers a wait
on a busy database. read counts the records, then reads them in the
same read-only transaction in signature order, the files_signature
index read backwards, so SQLite sorts nothing and the read ends at
exactly its count. The duplicate groups are formed in Go from runs of
equal signatures; only those of the size being read are held, and they
are written, ordered by first path, before the next size. The rows go
through the display's stdout writer. stdout and the summary line are
unchanged.

Judgement call: writing is part of read, not a phase of its own, since each size's rows are written while the read goes on.

Model: opus-5-5
2026-10-16 21:44:55 +02:00
clawbot 8e0facdcd3 trees: show its phases as N of M, with progress, elapsed time and ETA (closes #113)
check / check (push) Canceled after 0s
trees now runs in four phases, open, read, group and write, each under
the progress display scan uses, until the summary line. open covers a
wait on a busy database. read counts the records and reads them in one
read-only transaction, so the two agree during a scan; group counts the
directories, and write the groups. The rows reach stdout through a
writer. When stdout and stderr are the same file, it clears the display
and holds its lock while writing, so no row shares a line with the
display on a shared terminal; otherwise it writes without the lock, so
a slow reader of stdout cannot stop the display. Each rate names what
its phase counts.

Judgement call: opening the database is a phase of its own, so there are four phases, not the three the issue sketches.

Model: opus-5-5
2026-10-16 21:00:14 +02:00
clawbot 8dc73aa3f5 scan: show each phase as N of M, with progress, elapsed time and ETA, also while it waits (closes #112)
check / check (push) Canceled after 0s
Every scan display names its phase and its place among the five, as
"[2/5] walk", with its progress, elapsed time and ETA. load counts the
records first; the walk estimates its total from the records under the
operands; the content phase adds up the sizes of the matching groups
and says it is searching until the first record returns. Each display
redraws every 100 ms on a terminal and every 5 seconds in plain mode.

On a terminal no line reaches the last column: a line too wide drops
its rate, then its bar. After the terminal is made narrower than the
widest line drawn, the display makes a new library object, whose erase
fits.

Judgement call: load and the content checks draw the bar too, once their totals are known.
Judgement call: the bar is 10 columns wide, not the width the line leaves.

Model: opus-5-5
2026-10-16 19:11:10 +02:00
clawbot 264817c413 Name the database each command uses on stderr (closes #111)
check / check (push) Canceled after 0s
scan, report and trees now print one line on stderr before anything
else, naming the database and whether its path is the default or comes
from SFDUPES_DATABASE; scan also says when the file is new. report and
trees print it before opening the database, so a missing-database error
follows it. A relative path gets the working directory put in front of
it, with its symlinks resolved as the kernel reports it, and is not
cleaned: a ".." after a symlink names another file than the cleaned
path does. README "Database" lists when scan writes each file.

Unverified: the line on a terminal has no test of its own.

Model: opus-5-5
2026-10-16 15:28:02 +02:00
clawbot 198cefa152 Re-vendor the canonical files from sneak/prompts at e3f79c4 (closes #95)
check / check (push) Canceled after 0s
Every vendored file, REPO_POLICIES.md and every model script is the
copy at sneak/prompts e3f79c4, with this repository's own entries kept
after the canonical content. Lint and test are Dockerfile phases that
write no image; make test runs the suite under the race detector as
nobody. Every build in script/ passes a new CACHEBUST value, which every
stage declares first, so every RUN step runs again; the steps that
compile keep Go's build cache, and both go test runs pass -count=1.
Dockerfile.lint, script/verify-lint-image-pin and make test-race are
gone, prettier runs on the host, and .claude/settings.json is deleted.

Deviation: the set comes from e3f79c4 on next rather than dd4027b.
Judgement call: the test phase mounts the cache at nobody's
/tmp/.cache/go-build, owned by nobody.

Model: opus-5-5
2026-10-15 18:28:54 +02:00