next #50

Merged
sneak merged 5 commits from next into main 2026-09-26 15:27:29 +02:00
Collaborator

next holds the work landed since the last merge to main. Safe to merge at any time; nothing here needs a deploy step.

  • 337b319 lint-image pin comments and FROM lines put in the policy form; digests unchanged (closes #25).
  • 7ac4f6b dead files.dat references removed from Makefile, .gitignore and .dockerignore; build config only (closes #22).
  • 29a6501 new duplicate-detection ladder per the owner order: under 10 MiB whole-file hash; 10 MiB and over gated on 64 KiB head/tail, then whole-file hash under 50 MiB or gigabyte-spaced 1 MiB samples (closes #61).
  • 09a39dd database schema kept at version 1 per the owner's instruction: the ladder's new content column is part of the version 1 schema, changed in place (closes #61).
  • c737490 a file of 10 MiB or more is read for its content hash only when its size and 64 KiB head and tail match another file's, in the same scan or a stored one, per the issue's own words (closes #61).

Nothing waits on you for this branch beyond the merge itself.

Model: fable-5-1 (body); opus-5-5 (edits)

`next` holds the work landed since the last merge to `main`. Safe to merge at any time; nothing here needs a deploy step. - `337b319` lint-image pin comments and `FROM` lines put in the policy form; digests unchanged (closes https://git.eeqj.de/sneak/sfdupes/issues/25). - `7ac4f6b` dead `files.dat` references removed from `Makefile`, `.gitignore` and `.dockerignore`; build config only (closes https://git.eeqj.de/sneak/sfdupes/issues/22). - `29a6501` new duplicate-detection ladder per the owner order: under 10 MiB whole-file hash; 10 MiB and over gated on 64 KiB head/tail, then whole-file hash under 50 MiB or gigabyte-spaced 1 MiB samples (closes https://git.eeqj.de/sneak/sfdupes/issues/61). - `09a39dd` database schema kept at version 1 per the owner's instruction: the ladder's new `content` column is part of the version 1 schema, changed in place (closes https://git.eeqj.de/sneak/sfdupes/issues/61). - `c737490` a file of 10 MiB or more is read for its content hash only when its size and 64 KiB head and tail match another file's, in the same scan or a stored one, per the issue's own words (closes https://git.eeqj.de/sneak/sfdupes/issues/61). Nothing waits on you for this branch beyond the merge itself. Model: fable-5-1 (body); opus-5-5 (edits)
clawbot added 1 commit 2026-08-10 16:07:56 +02:00
The `(Debian-based)` parenthetical broke the required
`# image:vX.Y.Z, YYYY-MM-DD` form and asserted a base change that never
happened (v2.12.1 was Debian too); the tag before the digest left three
FROM lines in one file using two conventions. Digest unchanged, in both
Dockerfile and Dockerfile.lint. The golang and alpine pin comments
already matched the required form.

script/verify-lint-image-pin parses these two FROM lines to keep them
identical and still matches the tagless form; its advice line drops the
now-meaningless "tag and digest". With no tag in either reference a
tag-only disagreement cannot arise; a tag reintroduced on one side is
caught as a plain mismatch.
clawbot added the needs-review label 2026-08-10 16:08:01 +02:00
clawbot self-assigned this 2026-08-10 16:08:01 +02:00
clawbot changed title from next to Normalize the lint-image pin comments and FROM form (closes #25) 2026-09-04 12:07:25 +02:00
clawbot added merge-ready and removed needs-review labels 2026-09-04 12:07:25 +02:00
clawbot removed their assignment 2026-09-04 12:07:26 +02:00
sneak was assigned by clawbot 2026-09-04 12:07:27 +02:00
clawbot added needs-review and removed merge-ready labels 2026-09-06 16:26:37 +02:00
clawbot changed title from Normalize the lint-image pin comments and FROM form (closes #25) to next 2026-09-06 16:26:37 +02:00
sneak was unassigned by clawbot 2026-09-06 16:26:37 +02:00
clawbot added 1 commit 2026-09-21 15:01:59 +02:00
files.dat was the scan format before the SQLite database; nothing has
produced it since. Drop the stale references from the Makefile clean
target, .gitignore and .dockerignore. make clean still removes the
binary and .gitignore still covers the database files. The only
remaining mention is the historical entry in TODO.md.

Model: opus-4-8 (implementation); fable-5-1 (merge)
clawbot self-assigned this 2026-09-21 20:23:02 +02:00
Owner

@clawbot why is 'next' not merge-ready? it should always be green. if the candidate merge isn't green, it shouldn't be modified.

@clawbot why is 'next' not merge-ready? it should *always* be green. if the candidate merge isn't green, it shouldn't be modified.
Author
Collaborator

Why it was not merge-ready: the repo was paused mid-review-cycle under the 2026-09-22 priority ruling before the milestone gate ran, and the label was being withheld until the whole 1.0.0 queue had landed on next — completeness was gating the label. Under your rule that is wrong: next is only ever modified by green, reviewed unit merges, so this candidate PR must be green and mergeable at all times, and the merge-ready label tracks greenness, not milestone completeness.

Actions, immediately:

  • a verification session is running make check (in Docker, per repo policy) on the current next; it posts the pass/fail result here with the tested commit id
  • green: this PR goes merge-ready and is assigned to you right away; later units keep landing behind it only while they are themselves green
  • red: next is frozen — nothing further merges into it until green is restored, and the offending change is fixed or reverted first
  • the rule is recorded fleet-wide: a next→main candidate that is not green freezes that repo's next

The three open unit PRs (58 in review, 59/60 in rework) stay in the normal cycle and merge only green.

(model: claude-fable-5)

Why it was not merge-ready: the repo was paused mid-review-cycle under the 2026-09-22 priority ruling before the milestone gate ran, and the label was being withheld until the whole 1.0.0 queue had landed on `next` — completeness was gating the label. Under your rule that is wrong: `next` is only ever modified by green, reviewed unit merges, so this candidate PR must be green and mergeable at all times, and the merge-ready label tracks greenness, not milestone completeness. Actions, immediately: - a verification session is running `make check` (in Docker, per repo policy) on the current `next`; it posts the pass/fail result here with the tested commit id - green: this PR goes merge-ready and is assigned to you right away; later units keep landing behind it only while they are themselves green - red: `next` is frozen — nothing further merges into it until green is restored, and the offending change is fixed or reverted first - the rule is recorded fleet-wide: a `next`→`main` candidate that is not green freezes that repo's `next` The three open unit PRs (58 in review, 59/60 in rework) stay in the normal cycle and merge only green. (model: claude-fable-5)
Author
Collaborator

make check on branch next at commit 7ac4f6b passed.

Both gates ran in Docker per repo policy: golangci-lint reported 0 issues., and the test gate passed (ok sneak.berlin/go/sfdupes, coverage 88.5% of statements).

model: claude-opus-4-8

`make check` on branch `next` at commit `7ac4f6b` **passed**. Both gates ran in Docker per repo policy: golangci-lint reported `0 issues.`, and the test gate passed (`ok sneak.berlin/go/sfdupes`, coverage 88.5% of statements). model: claude-opus-4-8
clawbot added merge-ready and removed needs-review labels 2026-09-22 15:55:42 +02:00
clawbot removed their assignment 2026-09-22 15:55:45 +02:00
sneak was assigned by clawbot 2026-09-22 15:55:45 +02:00
Author
Collaborator

Green confirmed at 7ac4f6b (comment above) — labeled merge-ready and assigned per the always-green rule. Mergeable whenever you like; units landing on next behind it re-verify green each time. (model: claude-fable-5)

Green confirmed at `7ac4f6b` (comment above) — labeled merge-ready and assigned per the always-green rule. Mergeable whenever you like; units landing on `next` behind it re-verify green each time. (model: claude-fable-5)
clawbot added 1 commit 2026-09-22 16:40:44 +02:00
Author
Collaborator

make check on branch next at commit 29a6501 passed.

Docker lint + test gates both green:

  • test: ok sneak.berlin/go/sfdupes — coverage 88.1%
  • lint: 0 issues

model: claude-opus-4-8

`make check` on branch `next` at commit `29a6501` **passed**. Docker lint + test gates both green: - test: `ok sneak.berlin/go/sfdupes` — coverage 88.1% - lint: `0 issues` model: claude-opus-4-8
clawbot added 1 commit 2026-09-23 13:38:08 +02:00
Keep the database schema at version 1 (closes #61)
check / check (push) Successful in 42s
09a39ddf37
sfdupes is pre-1.0, with no installed base and no databases anywhere,
so the schema is changed in place and its version stays 1.
schemaVersion goes back to 1; the six-column files table, content
included, is the version 1 schema. The check that stops on a database
with any other version stays. README.md and TODO.md no longer describe
a version 2 or rejecting and rescanning version 1 databases. The
main.go package comment still described 1024-byte end windows and said
full file contents are never read; it now describes the hashes the
code computes.

Model: opus-5-5
clawbot added 1 commit 2026-09-23 16:06:11 +02:00
A file of 10 MiB or more now gets only its 64 KiB head and tail in the
hash phase, so its content is read only when it can be a duplicate. A
new content phase after the update phase finds every group of records,
anywhere in the database, that share size, head and tail and include
one without a content hash. It checks every member with lstat and, when
at least two pass, reads those without a content hash through the
existing worker pool; a stale file does not count as a match. report
and trees leave out records without a content hash. The README, help
text and TODO entry describe the gate; the schema stays at version 1.

Lint suppressed: gosec on the file open in hashContentOnly, as in
hashSignature, and on one chmod in a test.

Model: opus-5-5
Owner

@clawbot is sfdupes now fit for purpose?

@clawbot is sfdupes now fit for purpose?
sneak merged commit c2804d088b into main 2026-09-26 15:27:29 +02:00
Author
Collaborator

Not yet. The three commands work on the normal path and everything on main was reviewed, but sfdupes has never run at real scale (#56). The review toward 1.0 (#52 (comment)) found these still open, all filed:

  • Reports can be wrong without warning: a tab or newline in a path corrupts the report (#7), a cut-off report still exits 0 (#30), and a symlink given as a scan target deletes that subtree's records (#9).
  • Running it from cron as described does not work yet: report run as a normal user cannot open root's database (#8), it prints nothing under cron (#13), Ctrl-C mid-scan is not handled (#5), report and trees hold every record in memory, which may not fit at your scale (#14), two scans can run on one database at once (#53), and there is no install or cron guide (#54).

My recommendation: take sfdupes off pause for those nine after webhooker and pixa, then do the full-scale trial run (#56). Until then, use it only for runs you watch; don't rely on unattended cron runs.

Model: opus-5-5

Not yet. The three commands work on the normal path and everything on `main` was reviewed, but sfdupes has never run at real scale (#56). The review toward 1.0 (https://git.eeqj.de/sneak/sfdupes/issues/52#issuecomment-95752) found these still open, all filed: - **Reports can be wrong without warning:** a tab or newline in a path corrupts the report (#7), a cut-off report still exits 0 (#30), and a symlink given as a scan target deletes that subtree's records (#9). - **Running it from cron as described does not work yet:** `report` run as a normal user cannot open root's database (#8), it prints nothing under cron (#13), Ctrl-C mid-scan is not handled (#5), `report` and `trees` hold every record in memory, which may not fit at your scale (#14), two scans can run on one database at once (#53), and there is no install or cron guide (#54). My recommendation: take sfdupes off pause for those nine after webhooker and pixa, then do the full-scale trial run (#56). Until then, use it only for runs you watch; don't rely on unattended cron runs. Model: opus-5-5
Sign in to join this conversation.