Author SHA1 Message Date
sneak 0f5fedc043 Remove dead files.dat references from build config (closes #22)
check / check (push) Failing after 1s
files.dat was the pre-database on-disk scan format; nothing has
produced it since the persistent SQLite database landed. Drop the
stale references from Makefile clean, .gitignore and .dockerignore.
make clean still removes the binary, and .gitignore still covers the
actual database artifacts (*.sqlite, *.sqlite-shm, *.sqlite-wal). The
sole remaining files.dat mention is the historical entry in TODO.md.

Model: opus-4-8
2026-09-21 07:43:42 +00:00
5 changed files with 23 additions and 65 deletions
-1
View File
@@ -2,7 +2,6 @@
.claude .claude
.DS_Store .DS_Store
sfdupes sfdupes
files.dat
*.log *.log
*.out *.out
*.test *.test
-1
View File
@@ -27,7 +27,6 @@ node_modules/
*.log *.log
# Local scan data # Local scan data
files.dat
*.sqlite *.sqlite
*.sqlite-shm *.sqlite-shm
*.sqlite-wal *.sqlite-wal
+1 -1
View File
@@ -46,4 +46,4 @@ hooks:
@script/install-precommit @script/install-precommit
clean: clean:
rm -f $(BINARY) files.dat rm -f $(BINARY)
+3 -4
View File
@@ -29,10 +29,9 @@
# Completed Steps # Completed Steps
- correct four inaccurate comments in `cancel_test.go` and rename - remove the dead `files.dat` references from `Makefile`, `.gitignore`
`walkCancelInFlightDirs` to `walkCancelInFlightFiles` (2026-09-21, and `.dockerignore` (2026-09-21, branch `next`, closes
branch `issue-33-cancel-test-comments`, closes https://git.eeqj.de/sneak/sfdupes/issues/22)
https://git.eeqj.de/sneak/sfdupes/issues/33)
- fix the lint-image pin comments and `FROM` form in `Dockerfile` and - fix the lint-image pin comments and `FROM` form in `Dockerfile` and
`Dockerfile.lint` (2026-08-10, branch `next`, closes `Dockerfile.lint` (2026-08-10, branch `next`, closes
+19 -58
View File
@@ -13,20 +13,11 @@ import (
"time" "time"
) )
// This file gathers the tests for scan cancellation and worker-pool // poolUnwind bounds how long a goroutine is given to leave a pool
// unwinding. Everything it exercises lives in scan.go, so by the repo's // after its context is cancelled. Only a failing run ever waits this
// convention of one test file per source file it would belong in // long: a pool that ignored its cancellation parks forever, and this
// scan_test.go. It is kept separate on purpose: cancellation behaviour // is what turns that into a failed assertion instead of a suite that
// cuts across both the walk pool and the hash pool as a single concern, // hangs until the test binary's own timeout.
// and scan_test.go is already 900+ lines. That is the deliberate
// exception the convention otherwise expects to be stated.
// poolUnwind bounds how long a goroutine is given to leave a pool after
// its context is cancelled. The tests that use it turn a pool that
// ignored its cancellation — and so parks forever — into a failed
// assertion within this bound instead of a hang. Not every test in this
// file has that property: a few catch their regression only as the test
// binary's own timeout, and each of those says so.
const poolUnwind = 2 * time.Second const poolUnwind = 2 * time.Second
// walkClock is a context whose cancellation is driven by the scan's // walkClock is a context whose cancellation is driven by the scan's
@@ -37,12 +28,9 @@ const poolUnwind = 2 * time.Second
// //
// The accounting behind the n chosen by each test: every blocking // The accounting behind the n chosen by each test: every blocking
// channel operation in the walk selects on Done, so the walk spends // channel operation in the walk selects on Done, so the walk spends
// one consultation per file event plus a couple per directory. The // one consultation per file event plus a couple per directory, while
// index load that runs ahead of it also consults Done, but a bounded // the index load that runs ahead of it spends a small fixed number
// number of times that does not grow with the record count. The tests // (three) whatever the record count.
// depend on that property, not on the bound's exact value: each picks
// an n comfortably above it and well short of the walk's total, so the
// cancellation lands deep inside the walk whatever the record count.
type walkClock struct { type walkClock struct {
n int64 n int64
seen atomic.Int64 seen atomic.Int64
@@ -98,15 +86,11 @@ func (c *walkClock) Value(_ any) any {
// directory still queued and only the handful already in flight can // directory still queued and only the handful already in flight can
// emit anything more. // emit anything more.
const ( const (
walkCancelDirs = 100 walkCancelDirs = 100
walkCancelFilesPerDir = 20 walkCancelFilesPerDir = 20
walkCancelFiles = walkCancelDirs * walkCancelFilesPerDir walkCancelFiles = walkCancelDirs * walkCancelFilesPerDir
walkCancelWorkers = 4 walkCancelWorkers = 4
// The files carried by the walkCancelWorkers directories already in walkCancelInFlightDirs = walkCancelWorkers * walkCancelFilesPerDir
// flight when the scan is cancelled: each such directory can still
// emit its walkCancelFilesPerDir files. This is a file count, not a
// directory count.
walkCancelInFlightFiles = walkCancelWorkers * walkCancelFilesPerDir
) )
// walkCancelAtDone is the consultation on which the fixture's context // walkCancelAtDone is the consultation on which the fixture's context
@@ -170,11 +154,6 @@ func assertRecordsIntact(t *testing.T, db *sql.DB, before []string) {
// phase and fail there instead, with a wrapped error rather than the // phase and fail there instead, with a wrapped error rather than the
// bare cancellation. // bare cancellation.
// //
// The syncScan call here is not bounded by poolUnwind: a regression
// that left a worker pool parked would hang it, and that regression is
// caught only by the test binary's own timeout, not by a quick
// assertion.
//
//nolint:paralleltest // counts goroutines: must not run beside others //nolint:paralleltest // counts goroutines: must not run beside others
func TestSyncScanCancelledMidWalkKeepsRecords(t *testing.T) { func TestSyncScanCancelledMidWalkKeepsRecords(t *testing.T) {
dir := buildWalkCancelTree(t) dir := buildWalkCancelTree(t)
@@ -232,11 +211,10 @@ func assertWalkGuardAborted(t *testing.T, st scanStats, err error) {
} }
// The workers drop every directory still queued once the scan is // The workers drop every directory still queued once the scan is
// cancelled, so only the files in the directories already in flight // cancelled, so only the directories already in flight can add to
// can add to the census after the fact. A census beyond that bound // the census after the fact. A census beyond that bound would mean
// would mean the cancellation was not observed where it should have // the cancellation was not observed where it should have been.
// been. limit := walkCancelAtDone + walkCancelInFlightDirs
limit := walkCancelAtDone + walkCancelInFlightFiles
if st.unchanged > limit { if st.unchanged > limit {
t.Errorf("census covers %d files, want at most %d: the walk kept "+ t.Errorf("census covers %d files, want at most %d: the walk kept "+
"taking directories off the queue after cancellation", "taking directories off the queue after cancellation",
@@ -434,11 +412,7 @@ func TestDispatchDirsClosesJobsWhenCancelled(t *testing.T) {
// TestFeedHashJobsClosesJobsWhenCancelled checks that the hash feeder // TestFeedHashJobsClosesJobsWhenCancelled checks that the hash feeder
// abandons the runs it has not queued yet and still closes the job // abandons the runs it has not queued yet and still closes the job
// channel, which is what lets the workers' range terminate. The // channel, which is what lets the workers' range terminate.
// receive on jobs below is not bounded: a feeder that returned without
// closing jobs would leave that receive with no sender and no close, so
// this regression is caught by the test binary's timeout rather than by
// a bounded assertion.
func TestFeedHashJobsClosesJobsWhenCancelled(t *testing.T) { func TestFeedHashJobsClosesJobsWhenCancelled(t *testing.T) {
t.Parallel() t.Parallel()
@@ -466,16 +440,6 @@ func TestFeedHashJobsClosesJobsWhenCancelled(t *testing.T) {
// nobody wants the hashes of — while still letting the range run out // nobody wants the hashes of — while still letting the range run out
// so the pool tears down. The queued run names a file that does not // so the pool tears down. The queued run names a file that does not
// exist, so a worker that hashed it anyway would produce a result. // exist, so a worker that hashed it anyway would produce a result.
//
// hashWorker has a second cancellation exit: the send of a completed
// result on the results channel. That branch is reachable from the
// production scan path, not dead code — pool.stop() cancels the context
// before it starts draining results, so a worker parked on that send
// leaves through this case, freed by the drain rather than by an empty
// jobs channel. It is exercised by TestScanHashWriteFailureUnwindsPool,
// which strands every worker on a full results channel until stop()
// unwinds the pool. With both branches covered, every cancellation
// branch of the walk and hash pools has a test.
func TestHashWorkerDropsQueuedRuns(t *testing.T) { func TestHashWorkerDropsQueuedRuns(t *testing.T) {
t.Parallel() t.Parallel()
@@ -508,10 +472,7 @@ func TestHashWorkerDropsQueuedRuns(t *testing.T) {
// TestHashPhaseCancelledReturnsContextError checks the result loop's // TestHashPhaseCancelledReturnsContextError checks the result loop's
// own exit: with the pool cancelled, no result will ever arrive, and // own exit: with the pool cancelled, no result will ever arrive, and
// the loop must leave through the cancellation rather than wait for a // the loop must leave through the cancellation rather than wait for a
// receive that cannot happen. This call is not bounded by poolUnwind: a // receive that cannot happen.
// loop that dropped its cancellation case would block on that receive,
// so the regression surfaces as the test binary's timeout rather than
// as a bounded assertion.
func TestHashPhaseCancelledReturnsContextError(t *testing.T) { func TestHashPhaseCancelledReturnsContextError(t *testing.T) {
t.Parallel() t.Parallel()