Adds lib.mldata, the content-similarity search surface over the CLIP index
built in #49, closing #50.
forFile({ fileID }) — the whole stored ML payload (face boxes, landmarks,
embedding), read from disk on demand, or undefined when uncached. The only
method here that touches disk, and the only async one.
similar({ fileID, limit? }) — files nearest a file by cosine over their
CLIP embeddings, the file itself excluded; empty when it has no indexed
embedding.
searchByEmbedding({ embedding, limit? }) — files nearest a caller-supplied
query vector by cosine. quak bundles no text encoder (owner-deferred), so the
caller produces the query.
Cosine is one plain loop over the packed Float32Array index (no new
dependency). Each row's magnitude is computed on the fly rather than cached: the
index mutates as ML data is fetched, so a norm cache would be a staleness bug
for no real speed at this size.
What the diff does not show:
Judgement call: similar/searchByEmbedding return { fileID, score }[]
(cosine in [-1, 1]) descending, ties broken by ascending fileID, rather than
bare fileIDs — the score is cheap and useful, and .map(r => r.fileID)
recovers the ids.
The private ML-store field was renamed mlStore to free mldata for the
public API. The surface exists even when ML fetching is disabled, returning
empty results.
make check green (tests, eslint and prettier, lint in Docker).
Model: opus-4-8
Adds `lib.mldata`, the content-similarity search surface over the CLIP index
built in
[#49](https://git.eeqj.de/sneak/quak/issues/49), closing
[#50](https://git.eeqj.de/sneak/quak/issues/50).
- `forFile({ fileID })` — the whole stored ML payload (face boxes, landmarks,
embedding), read from disk on demand, or undefined when uncached. The only
method here that touches disk, and the only async one.
- `similar({ fileID, limit? })` — files nearest a file by cosine over their
CLIP embeddings, the file itself excluded; empty when it has no indexed
embedding.
- `searchByEmbedding({ embedding, limit? })` — files nearest a caller-supplied
query vector by cosine. quak bundles no text encoder (owner-deferred), so the
caller produces the query.
Cosine is one plain loop over the packed `Float32Array` index (no new
dependency). Each row's magnitude is computed on the fly rather than cached: the
index mutates as ML data is fetched, so a norm cache would be a staleness bug
for no real speed at this size.
What the diff does not show:
- Judgement call: `similar`/`searchByEmbedding` return `{ fileID, score }[]`
(cosine in [-1, 1]) descending, ties broken by ascending fileID, rather than
bare fileIDs — the score is cheap and useful, and `.map(r => r.fileID)`
recovers the ids.
- The private ML-store field was renamed `mlStore` to free `mldata` for the
public API. The surface exists even when ML fetching is disabled, returning
empty results.
`make check` green (tests, eslint and prettier, lint in Docker).
Model: opus-4-8
Add `lib.mldata`, the RAM-only content search over the #49 CLIP index:
- `forFile({ fileID })` — the whole stored ML payload from disk, or undefined
when uncached (the only method here that touches disk).
- `similar({ fileID, limit? })` — files nearest a file by cosine over their
CLIP embeddings, excluding the file itself.
- `searchByEmbedding({ embedding, limit? })` — files nearest a caller-supplied
query vector; quak bundles no text encoder (owner-deferred).
Cosine is one plain loop over the packed Float32Array index, computing each
row's magnitude on the fly rather than caching norms the mutating index would
invalidate. No new dependency. Results are `{ fileID, score }[]` descending,
ties broken by ascending fileID. The private store field was renamed `mlStore`
to free `mldata` for the public API.
Model: opus-4-8
PASS — lib.mldata implements forFile, similar, and searchByEmbedding correctly as a RAM-only cosine search that reuses the #49 packed index and id list; rankings, self-exclusion, tie-breaking, limits, and the empty-store/wrong-length/zero-magnitude guards are all correct, tests are meaningful, and make check is green on the head rebased onto current next.
Disclosures (non-blocking):
Every query vector in the fixture has unit magnitude, so the cosine denominator's query-norm term is never exercised: a regression that dropped it would leave rankings correct and go uncaught. The implementation itself divides by the query norm correctly; only the test coverage of that term is thin. A single case with a non-unit query asserting an exact score would close it.
The landing commit body runs ~127 words, marginally over the ~120-word guideline; the content is all substantive.
Model: opus-4-8
**PASS** — `lib.mldata` implements `forFile`, `similar`, and `searchByEmbedding` correctly as a RAM-only cosine search that reuses the #49 packed index and id list; rankings, self-exclusion, tie-breaking, limits, and the empty-store/wrong-length/zero-magnitude guards are all correct, tests are meaningful, and `make check` is green on the head rebased onto current `next`.
Disclosures (non-blocking):
- Every query vector in the fixture has unit magnitude, so the cosine denominator's query-norm term is never exercised: a regression that dropped it would leave rankings correct and go uncaught. The implementation itself divides by the query norm correctly; only the test coverage of that term is thin. A single case with a non-unit query asserting an exact score would close it.
- The landing commit body runs ~127 words, marginally over the ~120-word guideline; the content is all substantive.
Model: opus-4-8
clawbot
merged commit 224bd101ab into next2026-09-22 18:14:14 +02:00
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Adds
lib.mldata, the content-similarity search surface over the CLIP indexbuilt in
#49, closing
#50.
forFile({ fileID })— the whole stored ML payload (face boxes, landmarks,embedding), read from disk on demand, or undefined when uncached. The only
method here that touches disk, and the only async one.
similar({ fileID, limit? })— files nearest a file by cosine over theirCLIP embeddings, the file itself excluded; empty when it has no indexed
embedding.
searchByEmbedding({ embedding, limit? })— files nearest a caller-suppliedquery vector by cosine. quak bundles no text encoder (owner-deferred), so the
caller produces the query.
Cosine is one plain loop over the packed
Float32Arrayindex (no newdependency). Each row's magnitude is computed on the fly rather than cached: the
index mutates as ML data is fetched, so a norm cache would be a staleness bug
for no real speed at this size.
What the diff does not show:
similar/searchByEmbeddingreturn{ fileID, score }[](cosine in [-1, 1]) descending, ties broken by ascending fileID, rather than
bare fileIDs — the score is cheap and useful, and
.map(r => r.fileID)recovers the ids.
mlStoreto freemldatafor thepublic API. The surface exists even when ML fetching is disabled, returning
empty results.
make checkgreen (tests, eslint and prettier, lint in Docker).Model: opus-4-8
PASS —
lib.mldataimplementsforFile,similar, andsearchByEmbeddingcorrectly as a RAM-only cosine search that reuses the #49 packed index and id list; rankings, self-exclusion, tie-breaking, limits, and the empty-store/wrong-length/zero-magnitude guards are all correct, tests are meaningful, andmake checkis green on the head rebased onto currentnext.Disclosures (non-blocking):
Model: opus-4-8