Test keywords in place of the CLIP embedding in a file's JSON
check / check (push) Canceled after 0s
check / check (push) Canceled after 0s
clipKeywords takes a file's ML data and whether it is a video, and gives the titles of the Discover categories of Ente's photos app that its CLIP embedding falls in, highest score first. The tests cover the rule with made-up vectors and Ente's real categories with the CLIP embedding of a public-domain photo of a sunset. The backup and backup-metadata tests now expect keywords in a file's JSON and no CLIP embedding in its mlData. These tests fail until the implementation lands. Model: opus-5-5
This commit is contained in:
@@ -86,10 +86,17 @@ const DAY = "2024/2024-03/2024-03-01";
|
||||
const TOAST = new TextEncoder().encode("a few seconds of the wedding toast");
|
||||
|
||||
// Ente's ML data for beach.jpg: a face it found, and the photo's CLIP
|
||||
// embedding.
|
||||
// embedding, here that of a photo of a sunset (see clip-keywords.test.ts).
|
||||
const ML_DATA = {
|
||||
face: { faces: [{ faceID: "100_1", score: 0.98 }] },
|
||||
clip: { embedding: [0.5, 0.6, 0.7] },
|
||||
clip: {
|
||||
embedding: JSON.parse(
|
||||
readFileSync(
|
||||
new URL("../clip-keywords-sunset.json", import.meta.url),
|
||||
"utf-8",
|
||||
),
|
||||
).embedding,
|
||||
},
|
||||
};
|
||||
|
||||
// Each file's ID, its owner, its original as uploaded, and the metadata its
|
||||
@@ -486,7 +493,10 @@ describe("the first backup of the account", () => {
|
||||
},
|
||||
updationTime: TAKEN,
|
||||
});
|
||||
expect(beach.mlData).toEqual(ML_DATA);
|
||||
// The JSON holds the ML data without its CLIP embedding, and the
|
||||
// keywords the embedding gives in its place.
|
||||
expect(beach.mlData).toEqual({ face: ML_DATA.face });
|
||||
expect(beach.keywords).toEqual(["Sunset"]);
|
||||
expect(beach.imageMetadata.exif.Make.description).toBe("Canon");
|
||||
expect(beach.imageMetadata.exif.Model.description).toBe("EOS R5");
|
||||
|
||||
@@ -497,6 +507,7 @@ describe("the first backup of the account", () => {
|
||||
metadata: { title: "toast.mp4", fileType: "video" },
|
||||
});
|
||||
expect(toast).not.toHaveProperty("mlData");
|
||||
expect(toast).not.toHaveProperty("keywords");
|
||||
expect(toast).not.toHaveProperty("imageMetadata");
|
||||
expect(toast).not.toHaveProperty("imageMetadataError");
|
||||
|
||||
|
||||
Reference in New Issue
Block a user