A regular file replaced by a FIFO before the hash phase reads it hangs the scan for good #100

New Issue

2026-10-13T00:54:14+02:00

clawbot commented

2026-10-13 00:54:14 +02:00

The walk records a path as a regular file. If it is a FIFO (or a device node whose open blocks) by the time a hash worker opens it, os.Open blocks until a writer appears, which never happens. The worker never returns a result, so the phase waits forever, the lock stays held, and every later cron scan fails with another scan is running. The first SIGINT does not end it: readRuns returns, but the deferred pool.stop() waits for the blocked worker, so the hashed records still in the batch are never committed; only a second signal (uncaught, status 130) ends the process, leaving the database in WAL mode. The README says per-file errors never abort the run and that the first signal stops the scan cleanly.

Where (v1.0.0-rc1):

https://git.eeqj.de/sneak/sfdupes/src/tag/v1.0.0-rc1/scan.go#L1335 and https://git.eeqj.de/sneak/sfdupes/src/tag/v1.0.0-rc1/scan.go#L1367 (blocking open; no check that the open file is still regular)
https://git.eeqj.de/sneak/sfdupes/src/tag/v1.0.0-rc1/scan.go#L1257 (stop waits for every worker)

Reproduce: 302 same-size 9 MiB files, scan --workers 1; when the hash: line appears, rm the two files with the highest inode numbers and mkfifo them in their place. The bar stops at [300/302]; one thread sleeps in wait_for_partner; SIGINT prints nothing more; a second SIGINT ends it with status 130, an empty database and -wal/-shm files left behind.

Definition of done

The hash and content phases open without blocking (O_NONBLOCK or equivalent) and check with fstat that the open file is regular; anything else is a per-path error: warned about, counted as skipped, no record written.
A test that swaps a walked regular file for a FIFO before it is read fails before the fix (hangs until the test timeout) and passes after (one warning, exit 0).

Severity: minor. It needs a regular file replaced by a FIFO or blocking device between the walk and the read; the outcome is a scan that never ends and holds the lock until killed.

Model: fable-5-1

The walk records a path as a regular file. If it is a FIFO (or a device node whose open blocks) by the time a hash worker opens it, `os.Open` blocks until a writer appears, which never happens. The worker never returns a result, so the phase waits forever, the lock stays held, and every later cron scan fails with `another scan is running`. The first SIGINT does not end it: `readRuns` returns, but the deferred `pool.stop()` waits for the blocked worker, so the hashed records still in the batch are never committed; only a second signal (uncaught, status 130) ends the process, leaving the database in WAL mode. The README says per-file errors never abort the run and that the first signal stops the scan cleanly. Where (v1.0.0-rc1): - https://git.eeqj.de/sneak/sfdupes/src/tag/v1.0.0-rc1/scan.go#L1335 and https://git.eeqj.de/sneak/sfdupes/src/tag/v1.0.0-rc1/scan.go#L1367 (blocking open; no check that the open file is still regular) - https://git.eeqj.de/sneak/sfdupes/src/tag/v1.0.0-rc1/scan.go#L1257 (`stop` waits for every worker) Reproduce: 302 same-size 9 MiB files, `scan --workers 1`; when the `hash:` line appears, `rm` the two files with the highest inode numbers and `mkfifo` them in their place. The bar stops at `[300/302]`; one thread sleeps in `wait_for_partner`; SIGINT prints nothing more; a second SIGINT ends it with status 130, an empty database and `-wal`/`-shm` files left behind. **Definition of done** - The hash and content phases open without blocking (`O_NONBLOCK` or equivalent) and check with `fstat` that the open file is regular; anything else is a per-path error: warned about, counted as skipped, no record written. - A test that swaps a walked regular file for a FIFO before it is read fails before the fix (hangs until the test timeout) and passes after (one warning, exit 0). Severity: minor. It needs a regular file replaced by a FIFO or blocking device between the walk and the read; the outcome is a scan that never ends and holds the lock until killed. Model: fable-5-1

clawbot referenced this issue

2026-10-13 01:15:47 +02:00

Finish 1.0 and tag it; after that, bugfixes only #98

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: sneak/sfdupes#100