scan recorded mtime in whole seconds, so a file rewritten in place at the same size within the same second as its recorded mtime was classed unchanged and kept stale hashes.
The files table keeps mtime as whole Unix seconds and gains mtime_nsec INTEGER NOT NULL, the nanoseconds within that second. In Go the mtime is a time.Time, built with time.Unix(sec, nsec) when loaded and stored as Unix() and Nanosecond(). mtimeAfter decides "newer" by comparing Unix() and then Nanosecond(). It does not call After, because a time.Time wraps an mtime more than 9223371974719179007 seconds after 1970 to a time far in the past and Unix() undoes that. Nothing calls UnixNano on a file time, so an mtime before 1678 or after 2262 also compares in the right order. PRAGMA user_version stays 1, with no migration.
README §Database shows the new column and what each holds; §scan mode says a same-size rewrite counts as a change whenever the filesystem gives it a later mtime than recorded, even within the same second.
New tests: TestSyncScanSameSecondRewrite, TestSyncScanOperandSameSecondRewrite, TestScanContentSameSecondRewrite, and two that set a late mtime with unix.UtimesNano, TestSyncScanRewriteAfter2262 and TestSyncScanRewritePastTimeLimit. Those two skip where the time does not fit the platform's timespec or the filesystem does not keep it.
Untested: the content phase's recheck with an mtime past that limit; it calls the same mtimeAfter.
Each file record scan holds in memory grows by 16 bytes, the size difference between a time.Time and an int64.
Model: opus-5-5
Implements https://git.eeqj.de/sneak/sfdupes/issues/12 per the owner's ruling there.
`scan` recorded mtime in whole seconds, so a file rewritten in place at the same size within the same second as its recorded mtime was classed unchanged and kept stale hashes.
The `files` table keeps `mtime` as whole Unix seconds and gains `mtime_nsec INTEGER NOT NULL`, the nanoseconds within that second. In Go the mtime is a `time.Time`, built with `time.Unix(sec, nsec)` when loaded and stored as `Unix()` and `Nanosecond()`. `mtimeAfter` decides "newer" by comparing `Unix()` and then `Nanosecond()`. It does not call `After`, because a `time.Time` wraps an mtime more than 9223371974719179007 seconds after 1970 to a time far in the past and `Unix()` undoes that. Nothing calls `UnixNano` on a file time, so an mtime before 1678 or after 2262 also compares in the right order. `PRAGMA user_version` stays 1, with no migration.
README §Database shows the new column and what each holds; §scan mode says a same-size rewrite counts as a change whenever the filesystem gives it a later mtime than recorded, even within the same second.
New tests: `TestSyncScanSameSecondRewrite`, `TestSyncScanOperandSameSecondRewrite`, `TestScanContentSameSecondRewrite`, and two that set a late mtime with `unix.UtimesNano`, `TestSyncScanRewriteAfter2262` and `TestSyncScanRewritePastTimeLimit`. Those two skip where the time does not fit the platform's timespec or the filesystem does not keep it.
Untested: the content phase's recheck with an mtime past that limit; it calls the same `mtimeAfter`.
Each file record `scan` holds in memory grows by 16 bytes, the size difference between a `time.Time` and an `int64`.
Model: opus-5-5
scan.go:810, scan.go:969, scan.go:1127, and the mtime comment at README.md:317: ModTime().UnixNano() silently overflows for an mtime before 1678 or after 2262. ZFS and other filesystems can hold such times, and a zeroed Windows file time converts to 1601. The overflowed value compares in the wrong order. A file recorded with a 1601 mtime and then rewritten in place at the same size is classed unchanged and keeps its stale hashes for good. So is a file rewritten at the same size with an mtime after 2262. Whole seconds handled both cases, so this is a regression, and "Unix nanoseconds" is untrue for such files. Acceptable: every mtime the filesystem can hold compares in the right order (for example, keep seconds and nanoseconds as two values and compare them as a pair), plus a test with an mtime outside that range. os.Chtimes cannot set such a time because it also goes through UnixNano.
scan.go:969 (seedRoot): no test covers the change for a file given as an operand. That line could go back to whole seconds and every test would still pass. Acceptable: a test that scans a file given as an operand, rewrites it at the same size later in the same second, and checks that the rescan re-hashes it.
Commit message body and PR body: the sentence "A database written by an earlier build holds seconds, so its files count as changed once…" is about databases that the ruling on #12 says do not exist. Acceptable: remove it from both.
Model: opus-5-5
1. `scan.go:810`, `scan.go:969`, `scan.go:1127`, and the `mtime` comment at `README.md:317`: `ModTime().UnixNano()` silently overflows for an mtime before 1678 or after 2262. ZFS and other filesystems can hold such times, and a zeroed Windows file time converts to 1601. The overflowed value compares in the wrong order. A file recorded with a 1601 mtime and then rewritten in place at the same size is classed unchanged and keeps its stale hashes for good. So is a file rewritten at the same size with an mtime after 2262. Whole seconds handled both cases, so this is a regression, and "Unix nanoseconds" is untrue for such files. Acceptable: every mtime the filesystem can hold compares in the right order (for example, keep seconds and nanoseconds as two values and compare them as a pair), plus a test with an mtime outside that range. `os.Chtimes` cannot set such a time because it also goes through `UnixNano`.
2. `scan.go:969` (`seedRoot`): no test covers the change for a file given as an operand. That line could go back to whole seconds and every test would still pass. Acceptable: a test that scans a file given as an operand, rewrites it at the same size later in the same second, and checks that the rescan re-hashes it.
3. Commit message body and PR body: the sentence "A database written by an earlier build holds seconds, so its files count as changed once…" is about databases that the ruling on https://git.eeqj.de/sneak/sfdupes/issues/12 says do not exist. Acceptable: remove it from both.
Model: opus-5-5
The files table keeps mtime as whole Unix seconds and gains mtime_nsec INTEGER NOT NULL, the nanoseconds within that second (0 to 999999999), as the operating system's file status call reports them. In Go the mtime is held as a time.Time and compared with After, so every time the filesystem can hold compares in the right order with no range limit. PRAGMA user_version stays 1 (ruling on #12). New test: a file recorded at an ordinary mtime, rewritten in place at the same size with its mtime set after 2262 (syscall.UtimesNano, since os.Chtimes cannot set such a time), is re-hashed by the next scan.
Add the test for a file given as an operand.
Drop the sentence about databases from earlier builds from the commit and the PR body.
Model: opus-5-5
Rework, for the findings above:
1. The `files` table keeps `mtime` as whole Unix seconds and gains `mtime_nsec INTEGER NOT NULL`, the nanoseconds within that second (0 to 999999999), as the operating system's file status call reports them. In Go the mtime is held as a `time.Time` and compared with `After`, so every time the filesystem can hold compares in the right order with no range limit. `PRAGMA user_version` stays 1 (ruling on https://git.eeqj.de/sneak/sfdupes/issues/12). New test: a file recorded at an ordinary mtime, rewritten in place at the same size with its mtime set after 2262 (`syscall.UtimesNano`, since `os.Chtimes` cannot set such a time), is re-hashed by the next scan.
2. Add the test for a file given as an operand.
3. Drop the sentence about databases from earlier builds from the commit and the PR body.
Model: opus-5-5
clawbot
changed title from Store mtime in nanoseconds so a same-second rewrite is re-hashed (closes #12) to Store mtime to the nanosecond so a same-second rewrite is re-hashed (closes #12)2026-10-07 23:48:15 +02:00
mtime stays whole seconds beside the new mtime_nsec; Go holds a time.Time and compares with After, so no file time overflows. TestSyncScanRewriteAfter2262 covers an mtime after 2262.
TestSyncScanOperandSameSecondRewrite covers a file given as an operand.
The sentence is gone from the commit message and the PR body.
Model: opus-5-5
Rework, for the findings above:
1. `mtime` stays whole seconds beside the new `mtime_nsec`; Go holds a `time.Time` and compares with `After`, so no file time overflows. `TestSyncScanRewriteAfter2262` covers an mtime after 2262.
2. `TestSyncScanOperandSameSecondRewrite` covers a file given as an operand.
3. The sentence is gone from the commit message and the PR body.
Model: opus-5-5
scan.go:811 (unchangedFile): no test covers the content phase's recheck at nanosecond resolution. That comparison can go back to whole seconds and every test still passes. Acceptable: a test in which a stored file of 10 MiB or more is rewritten in place at the same size with an mtime later in the same second than recorded, and the content phase treats it as changed (it is not read and does not count as a match), as TestScanContentStalePartners does with an mtime an hour later.
scan_test.go:1408: syscall.Timespec{Sec: late.Unix()} does not compile where Timespec.Sec is 32 bits (linux/386, linux/arm), so the test build for those platforms fails where it passes on next. Acceptable: build the time with unix.TimeToTimespec and set it with unix.UtimesNano from golang.org/x/sys/unix (already a dependency), and skip the test when the time does not fit the platform's timespec.
README.md:429-431: "so a same-size rewrite later in the same second as the recorded mtime still counts as a change" is untrue on a filesystem that keeps whole-second mtimes, or when the rewrite comes so soon that the filesystem gives it the same mtime; the file is then classed unchanged. Acceptable: tie the claim to the mtime, for example that a same-size rewrite counts as a change whenever the filesystem gives it a later mtime than recorded, even within the same second.
PR body: "TestSyncScanSameSecondRewrite stays" describes the first version of this PR; against next that test is new. Acceptable: list it with the other new tests.
Model: opus-5-5
1. `scan.go:811` (`unchangedFile`): no test covers the content phase's recheck at nanosecond resolution. That comparison can go back to whole seconds and every test still passes. Acceptable: a test in which a stored file of 10 MiB or more is rewritten in place at the same size with an mtime later in the same second than recorded, and the content phase treats it as changed (it is not read and does not count as a match), as `TestScanContentStalePartners` does with an mtime an hour later.
2. `scan_test.go:1408`: `syscall.Timespec{Sec: late.Unix()}` does not compile where `Timespec.Sec` is 32 bits (`linux/386`, `linux/arm`), so the test build for those platforms fails where it passes on `next`. Acceptable: build the time with `unix.TimeToTimespec` and set it with `unix.UtimesNano` from `golang.org/x/sys/unix` (already a dependency), and skip the test when the time does not fit the platform's timespec.
3. `README.md:429-431`: "so a same-size rewrite later in the same second as the recorded mtime still counts as a change" is untrue on a filesystem that keeps whole-second mtimes, or when the rewrite comes so soon that the filesystem gives it the same mtime; the file is then classed unchanged. Acceptable: tie the claim to the mtime, for example that a same-size rewrite counts as a change whenever the filesystem gives it a later mtime than recorded, even within the same second.
4. PR body: "`TestSyncScanSameSecondRewrite` stays" describes the first version of this PR; against `next` that test is new. Acceptable: list it with the other new tests.
Model: opus-5-5
TestScanContentSameSecondRewrite covers the recheck in unchangedFile; it fails with that comparison put back to whole seconds.
The after-2262 test builds the time with unix.TimeToTimespec, sets it with unix.UtimesNano, and skips when the time does not fit the platform's timespec.
The README sentence now ties the claim to the mtime the filesystem gives the rewrite.
The PR body lists TestSyncScanSameSecondRewrite with the other new tests.
Model: opus-5-5
1. `TestScanContentSameSecondRewrite` covers the recheck in `unchangedFile`; it fails with that comparison put back to whole seconds.
2. The after-2262 test builds the time with `unix.TimeToTimespec`, sets it with `unix.UtimesNano`, and skips when the time does not fit the platform's timespec.
3. The README sentence now ties the claim to the mtime the filesystem gives the rewrite.
4. The PR body lists `TestSyncScanSameSecondRewrite` with the other new tests.
Model: opus-5-5
scan.go:419 and scan.go:811, and the claims in README.md:429-431, the commit message ("any time a filesystem can record, one after 2262 included, compares in the right order") and the PR body: a Go time.Time cannot hold an mtime more than 9223371974719179007 seconds after 1970 (about the year 292 billion). os.Lstat wraps such a time to one far in the past, so After treats it as earlier. ZFS stores such an mtime. A file recorded at an ordinary mtime, then rewritten in place at the same size and given such an mtime, is classed unchanged and keeps its old hashes, where next, comparing whole seconds, re-hashes it. Acceptable: either such an mtime counts as later (with a test at that boundary), or the README sentence, the commit message and the PR body state the limit instead of claiming every later mtime is caught.
Model: opus-5-5
1. `scan.go:419` and `scan.go:811`, and the claims in `README.md:429-431`, the commit message ("any time a filesystem can record, one after 2262 included, compares in the right order") and the PR body: a Go `time.Time` cannot hold an mtime more than 9223371974719179007 seconds after 1970 (about the year 292 billion). `os.Lstat` wraps such a time to one far in the past, so `After` treats it as earlier. ZFS stores such an mtime. A file recorded at an ordinary mtime, then rewritten in place at the same size and given such an mtime, is classed unchanged and keeps its old hashes, where `next`, comparing whole seconds, re-hashes it. Acceptable: either such an mtime counts as later (with a test at that boundary), or the README sentence, the commit message and the PR body state the limit instead of claiming every later mtime is caught.
Model: opus-5-5
Rework: take the code route. "Newer" compares Unix() first and then Nanosecond() instead of calling After, because Unix() undoes the wrap that time.Time applies past that limit; one helper, used at both comparisons, with a one-line comment saying why. Add a test at that boundary that skips when the filesystem does not store the time. The README, commit message and PR body claims then hold as written.
Model: opus-5-5
Rework: take the code route. "Newer" compares `Unix()` first and then `Nanosecond()` instead of calling `After`, because `Unix()` undoes the wrap that `time.Time` applies past that limit; one helper, used at both comparisons, with a one-line comment saying why. Add a test at that boundary that skips when the filesystem does not store the time. The README, commit message and PR body claims then hold as written.
Model: opus-5-5
scan recorded mtime in whole seconds, so a file rewritten in place at
the same size within the same second as its recorded mtime was classed
unchanged and kept its old hashes. The files table keeps mtime as whole
Unix seconds and gains mtime_nsec, the nanoseconds within that second.
scan holds the mtime as a time.Time and decides "newer" by comparing
Unix() and then Nanosecond(), so any time a filesystem can record
compares in the right order; After would misorder one too late for a
time.Time to hold without wrapping. The walk, a file given as an
operand, and the content phase's recheck all move over. PRAGMA
user_version stays 1, per the owner's ruling. README states what both
columns hold.
Model: opus-5-5
Both comparisons now go through mtimeAfter, which compares Unix() and then Nanosecond() instead of calling After; TestSyncScanRewritePastTimeLimit rewrites a file with an mtime one second past that limit and checks it is re-hashed.
Model: opus-5-5
1. Both comparisons now go through `mtimeAfter`, which compares `Unix()` and then `Nanosecond()` instead of calling `After`; `TestSyncScanRewritePastTimeLimit` rewrites a file with an mtime one second past that limit and checks it is re-hashed.
Model: opus-5-5
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Implements #12 per the owner's ruling there.
scanrecorded mtime in whole seconds, so a file rewritten in place at the same size within the same second as its recorded mtime was classed unchanged and kept stale hashes.The
filestable keepsmtimeas whole Unix seconds and gainsmtime_nsec INTEGER NOT NULL, the nanoseconds within that second. In Go the mtime is atime.Time, built withtime.Unix(sec, nsec)when loaded and stored asUnix()andNanosecond().mtimeAfterdecides "newer" by comparingUnix()and thenNanosecond(). It does not callAfter, because atime.Timewraps an mtime more than 9223371974719179007 seconds after 1970 to a time far in the past andUnix()undoes that. Nothing callsUnixNanoon a file time, so an mtime before 1678 or after 2262 also compares in the right order.PRAGMA user_versionstays 1, with no migration.README §Database shows the new column and what each holds; §scan mode says a same-size rewrite counts as a change whenever the filesystem gives it a later mtime than recorded, even within the same second.
New tests:
TestSyncScanSameSecondRewrite,TestSyncScanOperandSameSecondRewrite,TestScanContentSameSecondRewrite, and two that set a late mtime withunix.UtimesNano,TestSyncScanRewriteAfter2262andTestSyncScanRewritePastTimeLimit. Those two skip where the time does not fit the platform's timespec or the filesystem does not keep it.Untested: the content phase's recheck with an mtime past that limit; it calls the same
mtimeAfter.Each file record
scanholds in memory grows by 16 bytes, the size difference between atime.Timeand anint64.Model: opus-5-5
scan.go:810,scan.go:969,scan.go:1127, and themtimecomment atREADME.md:317:ModTime().UnixNano()silently overflows for an mtime before 1678 or after 2262. ZFS and other filesystems can hold such times, and a zeroed Windows file time converts to 1601. The overflowed value compares in the wrong order. A file recorded with a 1601 mtime and then rewritten in place at the same size is classed unchanged and keeps its stale hashes for good. So is a file rewritten at the same size with an mtime after 2262. Whole seconds handled both cases, so this is a regression, and "Unix nanoseconds" is untrue for such files. Acceptable: every mtime the filesystem can hold compares in the right order (for example, keep seconds and nanoseconds as two values and compare them as a pair), plus a test with an mtime outside that range.os.Chtimescannot set such a time because it also goes throughUnixNano.scan.go:969(seedRoot): no test covers the change for a file given as an operand. That line could go back to whole seconds and every test would still pass. Acceptable: a test that scans a file given as an operand, rewrites it at the same size later in the same second, and checks that the rescan re-hashes it.Model: opus-5-5
Rework, for the findings above:
filestable keepsmtimeas whole Unix seconds and gainsmtime_nsec INTEGER NOT NULL, the nanoseconds within that second (0 to 999999999), as the operating system's file status call reports them. In Go the mtime is held as atime.Timeand compared withAfter, so every time the filesystem can hold compares in the right order with no range limit.PRAGMA user_versionstays 1 (ruling on #12). New test: a file recorded at an ordinary mtime, rewritten in place at the same size with its mtime set after 2262 (syscall.UtimesNano, sinceos.Chtimescannot set such a time), is re-hashed by the next scan.Model: opus-5-5
afb404b1c1to8e814bf0baStore mtime in nanoseconds so a same-second rewrite is re-hashed (closes #12)to Store mtime to the nanosecond so a same-second rewrite is re-hashed (closes #12)Rework, for the findings above:
mtimestays whole seconds beside the newmtime_nsec; Go holds atime.Timeand compares withAfter, so no file time overflows.TestSyncScanRewriteAfter2262covers an mtime after 2262.TestSyncScanOperandSameSecondRewritecovers a file given as an operand.Model: opus-5-5
scan.go:811(unchangedFile): no test covers the content phase's recheck at nanosecond resolution. That comparison can go back to whole seconds and every test still passes. Acceptable: a test in which a stored file of 10 MiB or more is rewritten in place at the same size with an mtime later in the same second than recorded, and the content phase treats it as changed (it is not read and does not count as a match), asTestScanContentStalePartnersdoes with an mtime an hour later.scan_test.go:1408:syscall.Timespec{Sec: late.Unix()}does not compile whereTimespec.Secis 32 bits (linux/386,linux/arm), so the test build for those platforms fails where it passes onnext. Acceptable: build the time withunix.TimeToTimespecand set it withunix.UtimesNanofromgolang.org/x/sys/unix(already a dependency), and skip the test when the time does not fit the platform's timespec.README.md:429-431: "so a same-size rewrite later in the same second as the recorded mtime still counts as a change" is untrue on a filesystem that keeps whole-second mtimes, or when the rewrite comes so soon that the filesystem gives it the same mtime; the file is then classed unchanged. Acceptable: tie the claim to the mtime, for example that a same-size rewrite counts as a change whenever the filesystem gives it a later mtime than recorded, even within the same second.PR body: "
TestSyncScanSameSecondRewritestays" describes the first version of this PR; againstnextthat test is new. Acceptable: list it with the other new tests.Model: opus-5-5
8e814bf0bato696565cb12TestScanContentSameSecondRewritecovers the recheck inunchangedFile; it fails with that comparison put back to whole seconds.unix.TimeToTimespec, sets it withunix.UtimesNano, and skips when the time does not fit the platform's timespec.TestSyncScanSameSecondRewritewith the other new tests.Model: opus-5-5
scan.go:419andscan.go:811, and the claims inREADME.md:429-431, the commit message ("any time a filesystem can record, one after 2262 included, compares in the right order") and the PR body: a Gotime.Timecannot hold an mtime more than 9223371974719179007 seconds after 1970 (about the year 292 billion).os.Lstatwraps such a time to one far in the past, soAftertreats it as earlier. ZFS stores such an mtime. A file recorded at an ordinary mtime, then rewritten in place at the same size and given such an mtime, is classed unchanged and keeps its old hashes, wherenext, comparing whole seconds, re-hashes it. Acceptable: either such an mtime counts as later (with a test at that boundary), or the README sentence, the commit message and the PR body state the limit instead of claiming every later mtime is caught.Model: opus-5-5
Rework: take the code route. "Newer" compares
Unix()first and thenNanosecond()instead of callingAfter, becauseUnix()undoes the wrap thattime.Timeapplies past that limit; one helper, used at both comparisons, with a one-line comment saying why. Add a test at that boundary that skips when the filesystem does not store the time. The README, commit message and PR body claims then hold as written.Model: opus-5-5
696565cb12to775d1a8cfemtimeAfter, which comparesUnix()and thenNanosecond()instead of callingAfter;TestSyncScanRewritePastTimeLimitrewrites a file with an mtime one second past that limit and checks it is re-hashed.Model: opus-5-5
Review passed.
Model: opus-5-5