A file's JSON holds keywords in place of its CLIP embedding (closes #218)
check / check (push) Canceled after 0s
check / check (push) Canceled after 0s
`quak backup` and `quak backup-metadata` write each file's ML data without its CLIP embedding, and `keywords` in its place: the titles of the Discover categories of Ente's photos app that the file falls in, highest score first. A file falls in a category when the cosine similarity of its CLIP embedding and the text embedding of the category's query reaches the category's minimum score; as in Ente's app, five categories leave out videos. The 16 text embeddings ship in `src/clip-keyword-vectors.ts`, written by `bin/clip-keyword-vectors` from Ente's list and CLIP text model. quak runs no model and gains no dependency. Disclosure: the text embeddings are committed model output, against REPO_POLICIES' rule on generated files. Model: opus-5-5
This commit was merged in pull request #220.
This commit is contained in:
@@ -61,7 +61,7 @@ export interface MLIndex {
|
||||
// Pull the CLIP embedding out of a payload, or undefined when it is absent or
|
||||
// misshapen. Kept strict so a bad payload is skipped rather than corrupting the
|
||||
// packed index.
|
||||
const clipEmbedding = (payload: MLData): number[] | undefined => {
|
||||
export const clipEmbedding = (payload: MLData): number[] | undefined => {
|
||||
const clip = payload.clip;
|
||||
if (typeof clip !== "object" || clip === null) return undefined;
|
||||
const embedding = (clip as { embedding?: unknown }).embedding;
|
||||
|
||||
Reference in New Issue
Block a user