Nothing stops two sfdupes scan processes from running against the
same database at once. The documented deployment is a daily cron scan; on the target scale (tens of millions of files, a busy pool)
the first full scan, or any scan during a resilver, can run longer than
a day, so tomorrow's cron run starts on top of today's.
The busy timeout keeps this from corrupting anything, but both scans
walk and hash the same trees, doubling the load on a pool that is
already slow, and each one's summary line counts work the other did. README.md §Database only covers a report running during a scan;
two scans are not mentioned.
Definition of done
scan takes an exclusive advisory lock for its whole run, on a
lock file next to the database (<database path>.lock), using flock(2) via golang.org/x/sys/unix (already an indirect
dependency) — not on the SQLite file itself, which would block report.
A second scan against the same database fails immediately with
a one-line fatal error naming the lock file, exit 1, having
touched neither the filesystem nor the database.
report and trees are unaffected and still run during a scan.
The lock is released on every exit path, including a fatal error
and an interrupt.
README.md §Database and §Error handling document the behaviour.
A test starts a scan holding the lock and asserts the second one
fails with exit 1 and the message.
make check green.
Open question for the owner is in the first comment; the definition
of done above is the recommended reading.
Model: fable-5-1
Nothing stops two `sfdupes scan` processes from running against the
same database at once. The documented deployment is a daily cron
`scan`; on the target scale (tens of millions of files, a busy pool)
the first full scan, or any scan during a resilver, can run longer than
a day, so tomorrow's cron run starts on top of today's.
The busy timeout keeps this from corrupting anything, but both scans
walk and hash the same trees, doubling the load on a pool that is
already slow, and each one's summary line counts work the other did.
`README.md` §Database only covers a `report` running during a `scan`;
two scans are not mentioned.
## Definition of done
1. `scan` takes an exclusive advisory lock for its whole run, on a
lock file next to the database (`<database path>.lock`), using
`flock(2)` via `golang.org/x/sys/unix` (already an indirect
dependency) — not on the SQLite file itself, which would block
`report`.
2. A second `scan` against the same database fails immediately with
a one-line fatal error naming the lock file, exit 1, having
touched neither the filesystem nor the database.
3. `report` and `trees` are unaffected and still run during a scan.
4. The lock is released on every exit path, including a fatal error
and an interrupt.
5. `README.md` §Database and §Error handling document the behaviour.
6. A test starts a scan holding the lock and asserts the second one
fails with exit 1 and the message.
7. `make check` green.
Open question for the owner is in the first comment; the definition
of done above is the recommended reading.
Model: fable-5-1
clawbot
added this to the 1.0.0 milestone 2026-09-21 09:17:51 +02:00
Question for sneak: what should a second scan do when one is already
running against the same database?
A (recommended, and what the definition of done describes): fail
at once, exit 1, naming the lock file. Cron mails the failure; the
running scan is undisturbed.
B: wait for the first scan to finish, then run. Scans queue up
behind a slow one and the pool never gets a rest.
C: leave it as it is. Nothing breaks; the pool does double work.
Work proceeds on A unless you say otherwise. It is a new fatal error
in the README, which is the specification, so it is yours to approve.
Model: fable-5-1
Question for sneak: what should a second `scan` do when one is already
running against the same database?
- A (recommended, and what the definition of done describes): fail
at once, exit 1, naming the lock file. Cron mails the failure; the
running scan is undisturbed.
- B: wait for the first scan to finish, then run. Scans queue up
behind a slow one and the pool never gets a rest.
- C: leave it as it is. Nothing breaks; the pool does double work.
Work proceeds on A unless you say otherwise. It is a new fatal error
in the README, which is the specification, so it is yours to approve.
Model: fable-5-1
sneak
was assigned by clawbot2026-09-21 09:18:12 +02:00
Recorded ruling (sneak, 2026-09-22): option A. A second scan against a database that already has one running fails at once with exit 1 and names the lock file; the running scan is not disturbed. The unit is queued for when sfdupes resumes (paused under the current priority rule). Reassigned to clawbot: nothing here waits on sneak.
Model: opus-5-5
Recorded ruling (sneak, 2026-09-22): option A. A second `scan` against a database that already has one running fails at once with exit 1 and names the lock file; the running scan is not disturbed. The unit is queued for when sfdupes resumes (paused under the current priority rule). Reassigned to clawbot: nothing here waits on sneak.
Model: opus-5-5
sneak
was unassigned by clawbot2026-09-23 07:59:08 +02:00
clawbot
self-assigned this 2026-09-23 07:59:08 +02:00
Plan (implementer's brief; option A per sneak's ruling above).
runScan takes the lock right after the operands are resolved and before the database is opened or anything is walked: open (creating if needed) <database path>.lock, then unix.Flock with LOCK_EX|LOCK_NB from golang.org/x/sys/unix, promoted to a direct dependency and listed in README.md §Constraints.
Held for the whole run and released by closing the file in a deferred call, so every return path releases it; the operating system releases it if the process dies. Never delete the lock file: deleting it lets a third scan lock a new file while the second still holds the old one.
A second scan fails at once: one line naming the lock file, for example scan: another scan is running (lock held on /var/lib/sfdupes/db.sqlite.lock), exit 1. Any other error opening or locking the file is fatal too, with its own message.
report and trees never touch the lock.
README.md §Database and §Error handling describe the lock file and the new fatal error.
Test: the test holds the lock on the file itself, runs a scan, and asserts exit 1, the message, and that the database file was not created.
scan.go is also being edited for #9 and #5 (interrupts) builds on this; whichever lands second rebases.
Model: opus-5-5
Plan (implementer's brief; option A per sneak's ruling above).
- `runScan` takes the lock right after the operands are resolved and before the database is opened or anything is walked: open (creating if needed) `<database path>.lock`, then `unix.Flock` with `LOCK_EX|LOCK_NB` from `golang.org/x/sys/unix`, promoted to a direct dependency and listed in `README.md` §Constraints.
- Held for the whole run and released by closing the file in a deferred call, so every return path releases it; the operating system releases it if the process dies. Never delete the lock file: deleting it lets a third scan lock a new file while the second still holds the old one.
- A second scan fails at once: one line naming the lock file, for example `scan: another scan is running (lock held on /var/lib/sfdupes/db.sqlite.lock)`, exit 1. Any other error opening or locking the file is fatal too, with its own message.
- `report` and `trees` never touch the lock.
- `README.md` §Database and §Error handling describe the lock file and the new fatal error.
- Test: the test holds the lock on the file itself, runs a scan, and asserts exit 1, the message, and that the database file was not created.
`scan.go` is also being edited for https://git.eeqj.de/sneak/sfdupes/issues/9 and https://git.eeqj.de/sneak/sfdupes/issues/5 (interrupts) builds on this; whichever lands second rebases.
Model: opus-5-5
Built as planned above in #75. One addition the plan did not name: the README.md smoke test now keeps the database outside the scanned tree, because its empty lock file would otherwise join the empty1/empty2 group.
Model: opus-5-5
Built as planned above in https://git.eeqj.de/sneak/sfdupes/pulls/75. One addition the plan did not name: the `README.md` smoke test now keeps the database outside the scanned tree, because its empty lock file would otherwise join the `empty1`/`empty2` group.
Model: opus-5-5
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Nothing stops two
sfdupes scanprocesses from running against thesame database at once. The documented deployment is a daily cron
scan; on the target scale (tens of millions of files, a busy pool)the first full scan, or any scan during a resilver, can run longer than
a day, so tomorrow's cron run starts on top of today's.
The busy timeout keeps this from corrupting anything, but both scans
walk and hash the same trees, doubling the load on a pool that is
already slow, and each one's summary line counts work the other did.
README.md§Database only covers areportrunning during ascan;two scans are not mentioned.
Definition of done
scantakes an exclusive advisory lock for its whole run, on alock file next to the database (
<database path>.lock), usingflock(2)viagolang.org/x/sys/unix(already an indirectdependency) — not on the SQLite file itself, which would block
report.scanagainst the same database fails immediately witha one-line fatal error naming the lock file, exit 1, having
touched neither the filesystem nor the database.
reportandtreesare unaffected and still run during a scan.and an interrupt.
README.md§Database and §Error handling document the behaviour.fails with exit 1 and the message.
make checkgreen.Open question for the owner is in the first comment; the definition
of done above is the recommended reading.
Model: fable-5-1
Question for sneak: what should a second
scando when one is alreadyrunning against the same database?
at once, exit 1, naming the lock file. Cron mails the failure; the
running scan is undisturbed.
behind a slow one and the pool never gets a rest.
Work proceeds on A unless you say otherwise. It is a new fatal error
in the README, which is the specification, so it is yours to approve.
Model: fable-5-1
options B and C are pathological and you know it. obviously option A is correct.
Recorded ruling (sneak, 2026-09-22): option A. A second
scanagainst a database that already has one running fails at once with exit 1 and names the lock file; the running scan is not disturbed. The unit is queued for when sfdupes resumes (paused under the current priority rule). Reassigned to clawbot: nothing here waits on sneak.Model: opus-5-5
Plan (implementer's brief; option A per sneak's ruling above).
runScantakes the lock right after the operands are resolved and before the database is opened or anything is walked: open (creating if needed)<database path>.lock, thenunix.FlockwithLOCK_EX|LOCK_NBfromgolang.org/x/sys/unix, promoted to a direct dependency and listed inREADME.md§Constraints.scan: another scan is running (lock held on /var/lib/sfdupes/db.sqlite.lock), exit 1. Any other error opening or locking the file is fatal too, with its own message.reportandtreesnever touch the lock.README.md§Database and §Error handling describe the lock file and the new fatal error.scan.gois also being edited for #9 and #5 (interrupts) builds on this; whichever lands second rebases.Model: opus-5-5
Built as planned above in #75. One addition the plan did not name: the
README.mdsmoke test now keeps the database outside the scanned tree, because its empty lock file would otherwise join theempty1/empty2group.Model: opus-5-5