Nothing stops two sfdupes scan processes from running against the
same database at once. The documented deployment is a daily cron scan; on the target scale (tens of millions of files, a busy pool)
the first full scan, or any scan during a resilver, can run longer than
a day, so tomorrow's cron run starts on top of today's.
The busy timeout keeps this from corrupting anything, but both scans
walk and hash the same trees, doubling the load on a pool that is
already slow, and each one's summary line counts work the other did. README.md §Database only covers a report running during a scan;
two scans are not mentioned.
Definition of done
scan takes an exclusive advisory lock for its whole run, on a
lock file next to the database (<database path>.lock), using flock(2) via golang.org/x/sys/unix (already an indirect
dependency) — not on the SQLite file itself, which would block report.
A second scan against the same database fails immediately with
a one-line fatal error naming the lock file, exit 1, having
touched neither the filesystem nor the database.
report and trees are unaffected and still run during a scan.
The lock is released on every exit path, including a fatal error
and an interrupt.
README.md §Database and §Error handling document the behaviour.
A test starts a scan holding the lock and asserts the second one
fails with exit 1 and the message.
make check green.
Open question for the owner is in the first comment; the definition
of done above is the recommended reading.
Model: fable-5-1
Nothing stops two `sfdupes scan` processes from running against the
same database at once. The documented deployment is a daily cron
`scan`; on the target scale (tens of millions of files, a busy pool)
the first full scan, or any scan during a resilver, can run longer than
a day, so tomorrow's cron run starts on top of today's.
The busy timeout keeps this from corrupting anything, but both scans
walk and hash the same trees, doubling the load on a pool that is
already slow, and each one's summary line counts work the other did.
`README.md` §Database only covers a `report` running during a `scan`;
two scans are not mentioned.
## Definition of done
1. `scan` takes an exclusive advisory lock for its whole run, on a
lock file next to the database (`<database path>.lock`), using
`flock(2)` via `golang.org/x/sys/unix` (already an indirect
dependency) — not on the SQLite file itself, which would block
`report`.
2. A second `scan` against the same database fails immediately with
a one-line fatal error naming the lock file, exit 1, having
touched neither the filesystem nor the database.
3. `report` and `trees` are unaffected and still run during a scan.
4. The lock is released on every exit path, including a fatal error
and an interrupt.
5. `README.md` §Database and §Error handling document the behaviour.
6. A test starts a scan holding the lock and asserts the second one
fails with exit 1 and the message.
7. `make check` green.
Open question for the owner is in the first comment; the definition
of done above is the recommended reading.
Model: fable-5-1
clawbot
added this to the 1.0.0 milestone 2026-09-21 09:17:51 +02:00
Question for sneak: what should a second scan do when one is already
running against the same database?
A (recommended, and what the definition of done describes): fail
at once, exit 1, naming the lock file. Cron mails the failure; the
running scan is undisturbed.
B: wait for the first scan to finish, then run. Scans queue up
behind a slow one and the pool never gets a rest.
C: leave it as it is. Nothing breaks; the pool does double work.
Work proceeds on A unless you say otherwise. It is a new fatal error
in the README, which is the specification, so it is yours to approve.
Model: fable-5-1
Question for sneak: what should a second `scan` do when one is already
running against the same database?
- A (recommended, and what the definition of done describes): fail
at once, exit 1, naming the lock file. Cron mails the failure; the
running scan is undisturbed.
- B: wait for the first scan to finish, then run. Scans queue up
behind a slow one and the pool never gets a rest.
- C: leave it as it is. Nothing breaks; the pool does double work.
Work proceeds on A unless you say otherwise. It is a new fatal error
in the README, which is the specification, so it is yours to approve.
Model: fable-5-1
sneak
was assigned by clawbot2026-09-21 09:18:12 +02:00
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Nothing stops two
sfdupes scanprocesses from running against thesame database at once. The documented deployment is a daily cron
scan; on the target scale (tens of millions of files, a busy pool)the first full scan, or any scan during a resilver, can run longer than
a day, so tomorrow's cron run starts on top of today's.
The busy timeout keeps this from corrupting anything, but both scans
walk and hash the same trees, doubling the load on a pool that is
already slow, and each one's summary line counts work the other did.
README.md§Database only covers areportrunning during ascan;two scans are not mentioned.
Definition of done
scantakes an exclusive advisory lock for its whole run, on alock file next to the database (
<database path>.lock), usingflock(2)viagolang.org/x/sys/unix(already an indirectdependency) — not on the SQLite file itself, which would block
report.scanagainst the same database fails immediately witha one-line fatal error naming the lock file, exit 1, having
touched neither the filesystem nor the database.
reportandtreesare unaffected and still run during a scan.and an interrupt.
README.md§Database and §Error handling document the behaviour.fails with exit 1 and the message.
make checkgreen.Open question for the owner is in the first comment; the definition
of done above is the recommended reading.
Model: fable-5-1
Question for sneak: what should a second
scando when one is alreadyrunning against the same database?
at once, exit 1, naming the lock file. Cron mails the failure; the
running scan is undisturbed.
behind a slow one and the pool never gets a rest.
Work proceeds on A unless you say otherwise. It is a new fatal error
in the README, which is the specification, so it is yours to approve.
Model: fable-5-1