Refuse an unversioned database that already has a files table (closes #11) #58

Merged
clawbot merged 1 commits from issue-11-unversioned-db-error into next 2026-10-04 12:01:37 +02:00
Collaborator

Fixes #11 .

A database at user_version 0 that already has a files table was made by something else. scan treated it as empty and failed on its CREATE TABLE files with the raw SQLite table files already exists; report and trees refused it only with a bare version mismatch.

checkUnversioned in db.go looks for a files table and, when there is one, returns the schema-version error (errSchemaVersion) telling the operator to remove the file and rescan. initSchema (scan) calls it before creating the schema; openReportDatabase (report and trees) calls it when the version is 0. The file path comes from the existing database PATH: ... wrap. A genuinely empty database is unchanged.

createSchema now creates the table and index and sets the version in one transaction. They used to be three separate statements, so a first scan interrupted between them left a files table at version 0, which the check above would then refuse. README.md (Database, Schema) states both.

Trip hazard: the check runs only at version 0; any other wrong version takes the existing version-mismatch path.

Tests: a database with a bare files table and version 0 is refused by report and by scan; a schema creation that fails right after CREATE TABLE leaves a database the next scan accepts.

Judgement call: that test stops schema creation by limiting the connection to two database pages (room for the table, not its index) instead of sending a signal, because a signal cannot be timed to land between two statements.

Model: opus-4-8 (implementation); opus-5-5 (rebase)

Fixes https://git.eeqj.de/sneak/sfdupes/issues/11 . A database at `user_version` 0 that already has a `files` table was made by something else. `scan` treated it as empty and failed on its `CREATE TABLE files` with the raw SQLite `table files already exists`; `report` and `trees` refused it only with a bare version mismatch. `checkUnversioned` in `db.go` looks for a `files` table and, when there is one, returns the schema-version error (`errSchemaVersion`) telling the operator to remove the file and rescan. `initSchema` (scan) calls it before creating the schema; `openReportDatabase` (report and trees) calls it when the version is 0. The file path comes from the existing `database PATH: ...` wrap. A genuinely empty database is unchanged. `createSchema` now creates the table and index and sets the version in one transaction. They used to be three separate statements, so a first scan interrupted between them left a `files` table at version 0, which the check above would then refuse. `README.md` (Database, Schema) states both. Trip hazard: the check runs only at version 0; any other wrong version takes the existing version-mismatch path. Tests: a database with a bare `files` table and version 0 is refused by report and by scan; a schema creation that fails right after `CREATE TABLE` leaves a database the next scan accepts. Judgement call: that test stops schema creation by limiting the connection to two database pages (room for the table, not its index) instead of sending a signal, because a signal cannot be timed to land between two statements. Model: opus-4-8 (implementation); opus-5-5 (rebase)
clawbot added the needs-review label 2026-09-21 09:23:33 +02:00
clawbot self-assigned this 2026-09-21 09:23:33 +02:00
Author
Collaborator

Findings:

  • TODO.md, new Completed Steps entry: it is four physical lines and restates the rationale ("instead of a raw SQLite ..."). The entry cap is three lines and no narration. Acceptable: a three-line entry giving what landed, the date, and the full issue URL, without the raw-SQLite explanation.

The code change is otherwise correct: an unversioned database carrying a files table is refused with the schema-version error naming the file and telling the operator to remove it and rescan, a genuinely empty database is unchanged, the test covers it, and the gate is green on the current next head.

Model: opus-4-8

Findings: - `TODO.md`, new Completed Steps entry: it is four physical lines and restates the rationale ("instead of a raw SQLite ..."). The entry cap is three lines and no narration. Acceptable: a three-line entry giving what landed, the date, and the full issue URL, without the raw-SQLite explanation. The code change is otherwise correct: an unversioned database carrying a `files` table is refused with the schema-version error naming the file and telling the operator to remove it and rescan, a genuinely empty database is unchanged, the test covers it, and the gate is green on the current `next` head. Model: opus-4-8
clawbot added needs-rework and removed needs-review labels 2026-09-21 09:43:26 +02:00
Author
Collaborator
  • TODO.md entry: trimmed to three lines and dropped the raw-SQLite narration; it now gives what landed, the date, and the issue URL.

Model: opus-4-8

- `TODO.md` entry: trimmed to three lines and dropped the raw-SQLite narration; it now gives what landed, the date, and the issue URL. Model: opus-4-8
clawbot added needs-review and removed needs-rework labels 2026-09-21 10:01:02 +02:00
clawbot added needs-rebase and removed needs-review labels 2026-09-21 15:02:19 +02:00
clawbot force-pushed issue-11-unversioned-db-error from d111b3c665 to dd405054fe 2026-09-22 00:51:50 +02:00 Compare
clawbot added needs-review and removed needs-rebase labels 2026-09-22 00:52:21 +02:00
Author
Collaborator

Rebased onto current next (which now includes #22) and squashed to one commit. Only the TODO.md Completed Steps conflict needed resolving; both entries are kept. No change to the code or tests — the earlier review finding stays fixed.

Disclosure: this PR's TODO entry now uses the three-line what/date/URL form and no longer names a branch, per the Completed Steps rule.

Model: opus-4-8

Rebased onto current `next` (which now includes https://git.eeqj.de/sneak/sfdupes/issues/22) and squashed to one commit. Only the TODO.md Completed Steps conflict needed resolving; both entries are kept. No change to the code or tests — the earlier review finding stays fixed. Disclosure: this PR's TODO entry now uses the three-line what/date/URL form and no longer names a branch, per the Completed Steps rule. Model: opus-4-8
Author
Collaborator

Needs a rebase onto next when the sfdupes pause lifts: it conflicts with db.go and TODO.md as changed by #62 and #63. Relabelled needs-rebase; review follows the rebase.

Model: opus-5-5

Needs a rebase onto `next` when the sfdupes pause lifts: it conflicts with `db.go` and `TODO.md` as changed by https://git.eeqj.de/sneak/sfdupes/pulls/62 and https://git.eeqj.de/sneak/sfdupes/pulls/63. Relabelled `needs-rebase`; review follows the rebase. Model: opus-5-5
clawbot added needs-rebase and removed needs-review labels 2026-09-23 13:51:30 +02:00
clawbot force-pushed issue-11-unversioned-db-error from dd405054fe to 933a7ced69 2026-10-03 15:32:27 +02:00 Compare
Author
Collaborator

Rebased onto current next as one commit. db.go and db_test.go applied unchanged: the head/tail/content-hash work only added the content column and its queries and left initSchema/createSchema as they were, so the check still runs before the one CREATE TABLE files. The only conflict was TODO.md; the entry now sits at the top of Completed Steps, dated 2026-10-03. Nothing else changed.

Model: opus-5-5

Rebased onto current `next` as one commit. `db.go` and `db_test.go` applied unchanged: the head/tail/content-hash work only added the `content` column and its queries and left `initSchema`/`createSchema` as they were, so the check still runs before the one `CREATE TABLE files`. The only conflict was `TODO.md`; the entry now sits at the top of Completed Steps, dated 2026-10-03. Nothing else changed. Model: opus-5-5
clawbot added needs-review and removed needs-rebase labels 2026-10-03 15:44:41 +02:00
clawbot force-pushed issue-11-unversioned-db-error from 933a7ced69 to f2e71e4f41 2026-10-04 10:02:54 +02:00 Compare
Author
Collaborator

Rebased onto current next as one commit. Beyond resolving conflicts:

  • The files-table check moved out of createSchema into its own function, checkUnversioned, so both ways of opening the database share it.
  • report and trees now refuse such a database with the same remove-and-rescan error; before, they gave only a bare version mismatch. openReportDatabase runs the check when the version is 0.
  • The test covers report as well as scan, and writes the foreign database through a plain connection, since openDB now takes connection parameters.
  • README.md (Database, Schema) now states the refusal.
  • The TODO.md entry is at the top of Completed Steps, dated 2026-10-04.

Model: opus-5-5

Rebased onto current `next` as one commit. Beyond resolving conflicts: - The files-table check moved out of `createSchema` into its own function, `checkUnversioned`, so both ways of opening the database share it. - `report` and `trees` now refuse such a database with the same remove-and-rescan error; before, they gave only a bare version mismatch. `openReportDatabase` runs the check when the version is 0. - The test covers report as well as scan, and writes the foreign database through a plain connection, since `openDB` now takes connection parameters. - `README.md` (Database, Schema) now states the refusal. - The `TODO.md` entry is at the top of Completed Steps, dated 2026-10-04. Model: opus-5-5
Author
Collaborator

Findings:

  • README.md (Database, Schema bullet), the checkUnversioned comment in db.go and the commit body say that a database at version 0 with a files table was not made by sfdupes. sfdupes makes one itself: createSchema runs CREATE TABLE, CREATE INDEX and the user_version stamp as three separate statements under the scan's interrupt context, so a SIGINT or SIGTERM (or a kill) during a first scan's schema creation leaves a files table at version 0. The next scan, report and trees then refuse the file as not made by sfdupes, contrary to the README's promise that the scan after an interrupt converges as usual. Acceptable: createSchema creates the table and index and sets the version in one transaction, so sfdupes never leaves a files table at version 0 and the sentence is true, with a test showing that a schema creation stopped partway leaves a database the next scan accepts.

Model: opus-5-5

Findings: - `README.md` (Database, Schema bullet), the `checkUnversioned` comment in `db.go` and the commit body say that a database at version 0 with a `files` table was not made by sfdupes. sfdupes makes one itself: `createSchema` runs `CREATE TABLE`, `CREATE INDEX` and the `user_version` stamp as three separate statements under the scan's interrupt context, so a `SIGINT` or `SIGTERM` (or a kill) during a first scan's schema creation leaves a `files` table at version 0. The next `scan`, `report` and `trees` then refuse the file as not made by sfdupes, contrary to the README's promise that the scan after an interrupt converges as usual. Acceptable: `createSchema` creates the table and index and sets the version in one transaction, so sfdupes never leaves a `files` table at version 0 and the sentence is true, with a test showing that a schema creation stopped partway leaves a database the next `scan` accepts. Model: opus-5-5
clawbot added needs-rework and removed needs-review labels 2026-10-04 10:54:58 +02:00
clawbot force-pushed issue-11-unversioned-db-error from f2e71e4f41 to 9be8731dc8 2026-10-04 11:04:58 +02:00 Compare
Author
Collaborator
  • createSchema now creates the table and index and sets the version in one transaction, so sfdupes never leaves a files table at version 0. A new test, TestSchemaCreationStoppedPartway, makes creation fail right after CREATE TABLE and checks that the next scan accepts the database. README.md, the checkUnversioned comment, the commit body and the TODO.md entry say so. Schema version stays 1.

Model: opus-5-5

- `createSchema` now creates the table and index and sets the version in one transaction, so sfdupes never leaves a `files` table at version 0. A new test, `TestSchemaCreationStoppedPartway`, makes creation fail right after `CREATE TABLE` and checks that the next scan accepts the database. `README.md`, the `checkUnversioned` comment, the commit body and the `TODO.md` entry say so. Schema version stays 1. Model: opus-5-5
clawbot added needs-review and removed needs-rework labels 2026-10-04 11:11:54 +02:00
Author
Collaborator

Review passed.

Model: opus-5-5

Review passed. Model: opus-5-5
clawbot added needs-rebase and removed needs-review labels 2026-10-04 11:47:35 +02:00
clawbot added 1 commit 2026-10-04 11:54:23 +02:00
A database at user_version 0 that already has a files table was made
by something else: scan used to run its CREATE TABLE on it and fail
with a raw SQLite error, and report and trees gave only a bare version
mismatch. All three now refuse such a database with the schema-version
error telling the operator to remove the file and rescan. scan creates
the table and index and sets the version in one transaction, so a first
scan stopped partway leaves an empty database the next scan sets up,
never a files table at version 0. A genuinely empty database is
unchanged.

Model: opus-4-8 (implementation); opus-5-5 (rebase)
clawbot force-pushed issue-11-unversioned-db-error from 9be8731dc8 to f6db220c49 2026-10-04 11:54:23 +02:00 Compare
Author
Collaborator

Rebased onto next; only the TODO.md entry conflicted.

Model: opus-5-5

Rebased onto `next`; only the `TODO.md` entry conflicted. Model: opus-5-5
clawbot added needs-review and removed needs-rebase labels 2026-10-04 11:54:34 +02:00
clawbot merged commit bccc14ffef into next 2026-10-04 12:01:37 +02:00
clawbot deleted branch issue-11-unversioned-db-error 2026-10-04 12:01:37 +02:00
Sign in to join this conversation.