scan records its own database files when the database is inside a scanned operand #115

New Issue

2026-10-16T13:51:54+02:00

clawbot commented

2026-10-16 13:51:54 +02:00

Found while diagnosing sneak's trial run (#56 (comment)), where he set SFDUPES_DATABASE=/srv/tmp/sfdupes/db.db and ran scan /srv/.

When the database lies under a scanned operand, the walk reaches the database file, its -wal and -shm files and its .lock file, and records them like any other file. Reproduced on next: with the database inside the scanned directory, a second scan reports 3 records updated in a tree where nothing changed. report then lists db.sqlite-wal and db.sqlite.lock as duplicates of the empty files in the tree, and the -wal file no longer exists once the scan has closed the database.

Definition of done

scan never writes a record for its own database file, its -wal, -shm and -journal files, or its .lock file, wherever the operands put them, including when an operand reaches them through a symlinked directory.
Skipping them prints no warning and does not count as skipped. A record for one of them left by an earlier scan is deleted, like the record of any file the scan no longer verifies.
Recommended way: once the database and the lock file are open, note their device and inode (and those of the -wal and -shm files) and have the walk pass over a file with one of them. That holds through symlinks, unlike comparing paths. A -journal file never exists during the walk, because the database is in WAL mode then.
README "Rules for the walk" says so.
Tests: the database inside the scanned operand, reached directly and through a symlinked directory; a rescan of an unchanged tree reports nothing updated, and report lists none of the database's files.
TODO.md Completed Steps entry in the same commit; script/cibuild green on a fresh clone.

Notes for the implementer

Package main (the repo's only package). Finding the database's files belongs in db.go, the skip in the walk in scan.go.
Small and plain; no new flags.

Model: opus-5-5

Found while diagnosing sneak's trial run (https://git.eeqj.de/sneak/sfdupes/issues/56#issuecomment-145002), where he set `SFDUPES_DATABASE=/srv/tmp/sfdupes/db.db` and ran `scan /srv/`. When the database lies under a scanned operand, the walk reaches the database file, its `-wal` and `-shm` files and its `.lock` file, and records them like any other file. Reproduced on `next`: with the database inside the scanned directory, a second scan reports 3 records updated in a tree where nothing changed. `report` then lists `db.sqlite-wal` and `db.sqlite.lock` as duplicates of the empty files in the tree, and the `-wal` file no longer exists once the scan has closed the database. ## Definition of done - `scan` never writes a record for its own database file, its `-wal`, `-shm` and `-journal` files, or its `.lock` file, wherever the operands put them, including when an operand reaches them through a symlinked directory. - Skipping them prints no warning and does not count as skipped. A record for one of them left by an earlier scan is deleted, like the record of any file the scan no longer verifies. - Recommended way: once the database and the lock file are open, note their device and inode (and those of the `-wal` and `-shm` files) and have the walk pass over a file with one of them. That holds through symlinks, unlike comparing paths. A `-journal` file never exists during the walk, because the database is in WAL mode then. - README "Rules for the walk" says so. - Tests: the database inside the scanned operand, reached directly and through a symlinked directory; a rescan of an unchanged tree reports nothing updated, and `report` lists none of the database's files. - `TODO.md` Completed Steps entry in the same commit; `script/cibuild` green on a fresh clone. ## Notes for the implementer - Package `main` (the repo's only package). Finding the database's files belongs in `db.go`, the skip in the walk in `scan.go`. - Small and plain; no new flags. Model: opus-5-5

clawbot added this to the 1.0.0 milestone 2026-10-16 13:51:54 +02:00

clawbot referenced this issue from a commit

2026-10-17 03:54:52 +02:00

scan: never record its own database files (closes #115)

clawbot referenced a pull request that will close this issue

2026-10-17 03:56:43 +02:00

scan: never record its own database files (closes #115) #128

clawbot commented

2026-10-17 03:56:58 +02:00

Built in #128: once the database and its .lock file are open, scan notes the device and inode of the database file, its -wal and -shm files and its .lock file, and the walk passes over any file with one of them, with no line and no count.

Model: opus-5-5

Built in https://git.eeqj.de/sneak/sfdupes/pulls/128: once the database and its `.lock` file are open, `scan` notes the device and inode of the database file, its `-wal` and `-shm` files and its `.lock` file, and the walk passes over any file with one of them, with no line and no count. Model: opus-5-5

clawbot referenced this issue

2026-10-17 04:09:57 +02:00

scan: never record its own database files (closes #115) #128

clawbot closed this issue

2026-10-17 04:10:51 +02:00

clawbot referenced this issue from a commit

2026-10-17 04:10:51 +02:00

scan: never record its own database files (closes #115)

clawbot referenced this issue

2026-10-17 04:11:06 +02:00

next into main #110

Sign in to join this conversation.