ML data that does not decrypt is recorded with its reason (closes #198)
check / check (push) Waiting to run
check / check (push) Waiting to run
`fetchMLDataBatch` dropped an entry that did not decrypt, gunzip or parse, so the file looked like one with no ML data and the run exited 0. It now returns the reason beside the payloads, also for ML data that is not a JSON object. The library gives the reasons from its last ML data fetch that succeeded in `status().mlDataErrors` and the `fetchMLData` "done" event; a fetch that fails leaves them as they were. `quak backup` writes the reason as `mlDataError` for a file with no ML data cached and records the file in `failures.json`. `backup-metadata` writes `mlDataError` for it and exits 1; `runMetadataBackup` returns `filesWithMLDataError`. Judgement call: a file whose ML data is cached keeps it in its JSON when a newer copy does not decode, as when the fetch fails. Model: opus-5-5
This commit is contained in:
@@ -938,13 +938,14 @@ const ML_PAYLOAD = { face: { faces: [] }, clip: { embedding: [0.5, 0.25] } };
|
||||
|
||||
class MLClient extends MockClient {
|
||||
mlError?: string;
|
||||
async fetchMLData(args: {
|
||||
fileIDs: number[];
|
||||
}): Promise<Map<number, MLData>> {
|
||||
async fetchMLData(args: { fileIDs: number[] }): Promise<{
|
||||
payloads: Map<number, MLData>;
|
||||
errors: Map<number, unknown>;
|
||||
}> {
|
||||
if (this.mlError) throw new Error(this.mlError);
|
||||
const result = new Map<number, MLData>();
|
||||
if (args.fileIDs.includes(100)) result.set(100, ML_PAYLOAD);
|
||||
return result;
|
||||
const payloads = new Map<number, MLData>();
|
||||
if (args.fileIDs.includes(100)) payloads.set(100, ML_PAYLOAD);
|
||||
return { payloads, errors: new Map() };
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1517,6 +1518,7 @@ describe("backup album folders", () => {
|
||||
contentConcurrency: 5,
|
||||
fetchMLData: async () => {},
|
||||
mlData: async () => undefined,
|
||||
mlDataError: () => undefined,
|
||||
whoami: () => ({ email: "backup@example.com", userID: USER_ID }),
|
||||
});
|
||||
|
||||
|
||||
Reference in New Issue
Block a user