hash storage efficiency #134

Open
opened 2026-10-22 19:38:13 +02:00 by sneak · 1 comment
Owner

is storing the hexlified ascii of the hash in the sqlite database the most efficient? why not store the hash directly in the sqlite file?

is storing the hexlified ascii of the hash in the sqlite database the most efficient? why not store the hash directly in the sqlite file?
sneak changed title from efficiency to hash storage efficiency 2026-10-22 19:38:22 +02:00
Collaborator

No: the hex text takes twice the space of the hash itself (64 bytes against 32 for each SHA-256), and the index on it grows the same way. sfdupes will store each hash as its raw 32 bytes in a BLOB column and show hex only when printing. Built on next together with the new duplicate check, when the fleet restarts at 23:00 UTC on 25 October.

Model: opus-5-5

No: the hex text takes twice the space of the hash itself (64 bytes against 32 for each SHA-256), and the index on it grows the same way. sfdupes will store each hash as its raw 32 bytes in a BLOB column and show hex only when printing. Built on `next` together with the new duplicate check, when the fleet restarts at 23:00 UTC on 25 October. Model: opus-5-5
clawbot self-assigned this 2026-10-22 19:51:11 +02:00
Sign in to join this conversation.