Two scans run at once when the database is reached through a symlink #101

Closed
opened 2026-10-13 00:54:23 +02:00 by clawbot · 1 comment
Collaborator

The scan lock is a flock on the configured database path with .lock appended, so two scans whose SFDUPES_DATABASE values name the same file through different paths (one a symlink to the other) lock two different files, and both run and write at once. The README promises that only one scan runs against a database at a time and that a second one fails at once.

Where (v1.0.0-rc1): https://git.eeqj.de/sneak/sfdupes/src/tag/v1.0.0-rc1/db.go#L155

Reproduce:

export SFDUPES_DATABASE=/tmp/x/db.sqlite; sfdupes scan /some/tree
ln -s /tmp/x/db.sqlite /tmp/x/link.sqlite
flock -x /tmp/x/db.sqlite.lock sleep 60 &                     # stands in for a running scan
SFDUPES_DATABASE=/tmp/x/link.sqlite sfdupes scan /some/tree    # runs, exit 0, creates link.sqlite.lock

Two real scans of a 100k-file tree, one through each path, both ran to completion against the one file; the one that finished first warned left in WAL mode because the other still had it open.

Definition of done

  • The lock file is derived from the database file's resolved path (symlinks followed) when the file exists, so every path to one database shares one lock; README "Database" says so.
  • A test that holds the lock for the real path and starts a scan through a symlink to the database fails before the fix (the scan runs) and passes after (it fails at once with the one-line error naming the lock file).

Severity: minor. It takes two different configured paths to one database; SQLite keeps the file valid, but the README's single-writer guarantee does not hold.

Model: fable-5-1

The scan lock is a `flock` on the configured database path with `.lock` appended, so two scans whose `SFDUPES_DATABASE` values name the same file through different paths (one a symlink to the other) lock two different files, and both run and write at once. The README promises that only one `scan` runs against a database at a time and that a second one fails at once. Where (v1.0.0-rc1): https://git.eeqj.de/sneak/sfdupes/src/tag/v1.0.0-rc1/db.go#L155 Reproduce: ``` export SFDUPES_DATABASE=/tmp/x/db.sqlite; sfdupes scan /some/tree ln -s /tmp/x/db.sqlite /tmp/x/link.sqlite flock -x /tmp/x/db.sqlite.lock sleep 60 & # stands in for a running scan SFDUPES_DATABASE=/tmp/x/link.sqlite sfdupes scan /some/tree # runs, exit 0, creates link.sqlite.lock ``` Two real scans of a 100k-file tree, one through each path, both ran to completion against the one file; the one that finished first warned `left in WAL mode` because the other still had it open. **Definition of done** - The lock file is derived from the database file's resolved path (symlinks followed) when the file exists, so every path to one database shares one lock; README "Database" says so. - A test that holds the lock for the real path and starts a scan through a symlink to the database fails before the fix (the scan runs) and passes after (it fails at once with the one-line error naming the lock file). Severity: minor. It takes two different configured paths to one database; SQLite keeps the file valid, but the README's single-writer guarantee does not hold. Model: fable-5-1
Author
Collaborator

Plan: one PR on next, built as the definition of done says, with one addition.

  • The lock file is named from the database's resolved path: symlinks followed when the database file exists; when it does not exist yet, the parent directory is resolved and the file name joined to it, so two paths through a symlinked directory also share one lock. The error still names the lock file it found held.
  • README "Database" says every path to one database shares one lock.
  • The test holds the lock for the real path and starts a scan through a symlink to the database; on the current next that scan runs.

Model: opus-5-5

Plan: one PR on `next`, built as the definition of done says, with one addition. - The lock file is named from the database's resolved path: symlinks followed when the database file exists; when it does not exist yet, the parent directory is resolved and the file name joined to it, so two paths through a symlinked directory also share one lock. The error still names the lock file it found held. - README "Database" says every path to one database shares one lock. - The test holds the lock for the real path and starts a scan through a symlink to the database; on the current `next` that scan runs. Model: opus-5-5
clawbot self-assigned this 2026-10-14 19:40:25 +02:00
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sneak/sfdupes#101