Keep the database schema at version 1 (closes #61) #63

Merged
clawbot merged 1 commits from issue-61-schema-version-1 into next 2026-09-23 13:38:07 +02:00
4 changed files with 18 additions and 22 deletions
+2 -5
View File
@@ -154,11 +154,8 @@ All three subcommands operate on a single SQLite database file:
that dies partway leaves a valid database holding everything
hashed so far; the next scan skips those records and converges
toward the filesystem.
- Schema (`PRAGMA user_version` is the schema version, currently 2; a
database with any other version is a fatal error). Version 2 added
the `content` column and the 64 KiB head/tail signature (replacing
the version 1 1 KiB end windows), so a version 1 database cannot be
reused: it is rejected and the tree must be rescanned from scratch.
- Schema (`PRAGMA user_version` is the schema version, currently 1; a
database with any other version is a fatal error):
```sql
CREATE TABLE files (
+5 -6
View File
@@ -36,12 +36,11 @@
`head`, `tail`, and `content` all hold the whole-file hash. A file at
10 MiB or above is gated on the 64 KiB `head` and `tail`, then
compared on a `content` hash — the whole file below 50 MiB,
gigabyte-spaced 1 MiB samples at or above. Schema bumps to version 2
(new `content` column); a version 1 database is rejected and must be
rescanned, which is required anyway since every stored hash changed.
`report` and `trees` group by the extended signature, so the ladder is
applied across the whole database. README "Duplicate detection"
documents every rung including the probabilistic large-file path.
gigabyte-spaced 1 MiB samples at or above. The `content` column is
part of the version 1 schema. `report` and `trees` group by the
extended signature, so the ladder is applied across the whole
database. README "Duplicate detection" documents every rung including
the probabilistic large-file path.
- remove the dead `files.dat` references from `Makefile`, `.gitignore`
and `.dockerignore` (2026-09-21, branch `next`, closes
+2 -5
View File
@@ -25,11 +25,8 @@ const defaultDatabasePath = "/var/lib/sfdupes/db.sqlite"
const databaseEnv = "SFDUPES_DATABASE"
// schemaVersion is the database schema version this build reads and
// writes, stored in PRAGMA user_version. Version 2 adds the content
// column and the head/tail/content signature (replacing the version 1
// 1 KiB end windows), so a version 1 database is rejected and must be
// rescanned.
const schemaVersion = 2
// writes, stored in PRAGMA user_version.
const schemaVersion = 1
// dbDirPerm is the mode for a database parent directory created by
// scan.
+9 -6
View File
@@ -1,10 +1,13 @@
// Command sfdupes quickly identifies candidate duplicate files across
// very large filesystems without reading full file contents. Files are
// considered duplicates when they have identical size, identical SHA-256
// of their first 1024 bytes, and identical SHA-256 of their last 1024
// bytes. scan maintains a persistent SQLite database of file signatures
// (SFDUPES_DATABASE, default /var/lib/sfdupes/db.sqlite) that the
// reporting subcommands read.
// very large filesystems without reading every byte of every file.
// Files are considered duplicates when their sizes are equal and they
// agree on a short ladder of SHA-256 hashes. A file under 10 MiB is
// hashed in full. A larger file is compared on the hashes of its first
// and last 64 KiB and on a content hash: of the whole file when it is
// under 50 MiB, or of gigabyte-spaced 1 MiB samples when it is 50 MiB
// or larger. scan maintains a persistent SQLite database of file
// signatures (SFDUPES_DATABASE, default /var/lib/sfdupes/db.sqlite)
// that the reporting subcommands read.
//
// Usage:
//