quak files COLLECTION fetches that collection's files alone #224

Closed
clawbot wants to merge 1 commits from issue-217-files-one-collection into next
Collaborator

Fixes #217. quak files COLLECTION no longer waits for a full refresh, which on an empty cache fetched every collection's files first.

What changed:

  • Library.refreshCollection(collection) brings one collection up to date in the store, given its record from freshCollections(). It resolves to whether the store then holds it.
  • Once the stored cursor is past 0, it only updates a collection the store holds. For any other it stores nothing and resolves to false. freshFiles(lib, id) then awaits lib.fresh() and lists from the store.
  • A refresh from a stored cursor of 0 deletes all the store holds and fetches every collection's files from the beginning. Collections then sit in the server's order, as if refreshCollection() had never run.
  • MetadataStore.listUndecryptable() sorts by collection ID, then file ID. A backup reported these in fetch order, which refreshCollection() could change.

What the diff does not show:

  • refreshCollection runs in the refresh slot, cycle. It waits for a running refresh, and a fresh read or backup() started meanwhile waits for it. close() waits for it too.
  • open() awaits the first refresh whenever the stored cursor is 0.

Judgement calls:

  • The first refresh fetches again the files of each collection that refreshCollection() stored before it.
  • Files of different collections can sit in another order in metadata.json. No read depends on that order.
  • An account with no collections keeps a cursor of 0, so each open() awaits its refresh, one request.

Model: opus-5-5

Fixes https://git.eeqj.de/sneak/quak/issues/217. `quak files COLLECTION` no longer waits for a full refresh, which on an empty cache fetched every collection's files first. What changed: - `Library.refreshCollection(collection)` brings one collection up to date in the store, given its record from `freshCollections()`. It resolves to whether the store then holds it. - Once the stored cursor is past 0, it only updates a collection the store holds. For any other it stores nothing and resolves to `false`. `freshFiles(lib, id)` then awaits `lib.fresh()` and lists from the store. - A refresh from a stored cursor of 0 deletes all the store holds and fetches every collection's files from the beginning. Collections then sit in the server's order, as if `refreshCollection()` had never run. - `MetadataStore.listUndecryptable()` sorts by collection ID, then file ID. A backup reported these in fetch order, which `refreshCollection()` could change. What the diff does not show: - `refreshCollection` runs in the refresh slot, `cycle`. It waits for a running refresh, and a fresh read or `backup()` started meanwhile waits for it. `close()` waits for it too. - `open()` awaits the first refresh whenever the stored cursor is 0. Judgement calls: - The first refresh fetches again the files of each collection that `refreshCollection()` stored before it. - Files of different collections can sit in another order in `metadata.json`. No read depends on that order. - An account with no collections keeps a cursor of 0, so each `open()` awaits its refresh, one request. Model: opus-5-5
clawbot added the needs-review label 2026-10-26 00:26:46 +01:00
clawbot self-assigned this 2026-10-26 00:26:46 +01:00
Author
Collaborator

Review of #224 (fixes #217): FAIL, needs rework.

  1. src/library/index.ts, refreshCollection(): a refresh that runs while it fetches can leave the store wrong for good. If a refresh starts during the files request and commits first, refreshCollection then commits its older result over it. The collection goes back to the older updationTime, older file records come back, and so do files the refresh had just deleted. The refresh has already moved the cursor past that change, so the next refresh does not fetch the collection again. The store stays wrong until that collection changes on the server again, not just until the next refresh as the PR body's judgement call says. The CLI cannot hit this today, but refreshCollection is documented for other library users, and the library runs its own background refreshes. Acceptable: after the fetch, commit only if the stored copy is still older than the collection passed in (or keep a refresh from running in between). Add a library test in which a refresh commits a newer copy while the files request is still waiting.

  2. src/library/index.ts, comment on commit(): "the cursor moves past it" is now false for one of its two callers. When refreshCollection calls it the cursor does not move. What keeps a later refresh from fetching a file that does not decrypt again is the collection's stored updationTime. Acceptable: one sentence that is true for both callers.

Model: opus-5-5

Review of https://git.eeqj.de/sneak/quak/pulls/224 (fixes https://git.eeqj.de/sneak/quak/issues/217): **FAIL**, needs rework. 1. **`src/library/index.ts`, `refreshCollection()`: a refresh that runs while it fetches can leave the store wrong for good.** If a refresh starts during the files request and commits first, `refreshCollection` then commits its older result over it. The collection goes back to the older `updationTime`, older file records come back, and so do files the refresh had just deleted. The refresh has already moved the cursor past that change, so the next refresh does not fetch the collection again. The store stays wrong until that collection changes on the server again, not just until the next refresh as the PR body's judgement call says. The CLI cannot hit this today, but `refreshCollection` is documented for other library users, and the library runs its own background refreshes. Acceptable: after the fetch, commit only if the stored copy is still older than the `collection` passed in (or keep a refresh from running in between). Add a library test in which a refresh commits a newer copy while the files request is still waiting. 2. **`src/library/index.ts`, comment on `commit()`: "the cursor moves past it" is now false for one of its two callers.** When `refreshCollection` calls it the cursor does not move. What keeps a later refresh from fetching a file that does not decrypt again is the collection's stored `updationTime`. Acceptable: one sentence that is true for both callers. Model: opus-5-5
clawbot added needs-rework and removed needs-review labels 2026-10-26 00:52:15 +01:00
clawbot force-pushed issue-217-files-one-collection from 8e06cd5723 to daa47068bc 2026-10-26 01:04:19 +01:00 Compare
Author
Collaborator

Rework for the review at #224 (comment):

  1. After the files request, refreshCollection() commits only when the stored copy is still older than the collection it was given. A new library test holds that request while a refresh stores a newer copy, and checks that the newer copy stays. The PR body's judgement call now says this.
  2. The commit() comment now says a collection or file that does not decrypt is not seen again until the server's copy changes, which is true for both callers.

Model: opus-5-5

Rework for the review at https://git.eeqj.de/sneak/quak/pulls/224#issuecomment-154790: 1. After the files request, `refreshCollection()` commits only when the stored copy is still older than the collection it was given. A new library test holds that request while a refresh stores a newer copy, and checks that the newer copy stays. The PR body's judgement call now says this. 2. The `commit()` comment now says a collection or file that does not decrypt is not seen again until the server's copy changes, which is true for both callers. Model: opus-5-5
clawbot added needs-review and removed needs-rework labels 2026-10-26 01:04:26 +01:00
Author
Collaborator

Review of #224 (fixes #217): FAIL, needs rework.

  1. src/library/index.ts, refreshCollection(): it brings back a collection that a refresh has already removed. Suppose a refresh deletes the collection, or finds it undecryptable, after freshCollections() returned its record. This can happen during the files request, or before refreshCollection() looks at the store, for example in the refresh it waits for first. refreshCollection() then finds no stored copy and stores the collection and its files again. For an undecryptable one it also drops the recorded reason. That refresh moved the cursor past the change, so no later refresh removes the collection. The PR body discloses the first case. Acceptable: refreshCollection() never stores a collection the store has already dropped, whatever the order of events with a refresh. Library tests cover a refresh that deletes the collection during the files request, and one that deletes it before the call. One plain way: when no copy is stored, store only if the collection's updationTime is later than the stored cursor.

  2. src/library/index.ts, close(): it does not wait for a refreshCollection() in flight. close() resolves while the files request still waits. refreshCollection() then stores and saves anyway, and writes the cache directory again after the caller removed it. The comment on close() says a caller can remove it at that point. Acceptable: close() waits for a running refreshCollection() as it waits for a refresh, with a test. Running it in the same slot as a refresh is one plain way.

  3. Comment sentences this change touches run well past 30 words. Split each into short sentences:

    • src/cli-commands.ts, above openReadLibrary: "Open a library for a single point command: the aggressive background precache … is off — … and collections and files COLLECTION need none …"
    • test/cli/read.test.ts, file header: "files without a collection ID, get and get-thumb force a Library.fresh() round-trip before they read; … would come back empty and fail here."
    • test/cli/commands.test.ts, above "an account with many albums": "The account here has 40 albums of one photo each, … records the path and query of each request quak sends."

Judgement call, not raised as a finding: a library opened on a cache that only quak files COLLECTION wrote serves just that collection until its first refresh ends, which follows from the plan's choice to store it.

Model: opus-5-5

Review of https://git.eeqj.de/sneak/quak/pulls/224 (fixes https://git.eeqj.de/sneak/quak/issues/217): **FAIL**, needs rework. 1. **`src/library/index.ts`, `refreshCollection()`: it brings back a collection that a refresh has already removed.** Suppose a refresh deletes the collection, or finds it undecryptable, after `freshCollections()` returned its record. This can happen during the files request, or before `refreshCollection()` looks at the store, for example in the refresh it waits for first. `refreshCollection()` then finds no stored copy and stores the collection and its files again. For an undecryptable one it also drops the recorded reason. That refresh moved the cursor past the change, so no later refresh removes the collection. The PR body discloses the first case. Acceptable: `refreshCollection()` never stores a collection the store has already dropped, whatever the order of events with a refresh. Library tests cover a refresh that deletes the collection during the files request, and one that deletes it before the call. One plain way: when no copy is stored, store only if the collection's `updationTime` is later than the stored cursor. 2. **`src/library/index.ts`, `close()`: it does not wait for a `refreshCollection()` in flight.** `close()` resolves while the files request still waits. `refreshCollection()` then stores and saves anyway, and writes the cache directory again after the caller removed it. The comment on `close()` says a caller can remove it at that point. Acceptable: `close()` waits for a running `refreshCollection()` as it waits for a refresh, with a test. Running it in the same slot as a refresh is one plain way. 3. **Comment sentences this change touches run well past 30 words.** Split each into short sentences: - `src/cli-commands.ts`, above `openReadLibrary`: "Open a library for a single point command: the aggressive background precache … is off — … and `collections` and `files COLLECTION` need none …" - `test/cli/read.test.ts`, file header: "`files` without a collection ID, `get` and `get-thumb` force a `Library.fresh()` round-trip before they read; … would come back empty and fail here." - `test/cli/commands.test.ts`, above "an account with many albums": "The account here has 40 albums of one photo each, … records the path and query of each request quak sends." Judgement call, not raised as a finding: a library opened on a cache that only `quak files COLLECTION` wrote serves just that collection until its first refresh ends, which follows from the plan's choice to store it. Model: opus-5-5
clawbot added needs-rework and removed needs-review labels 2026-10-26 01:23:33 +01:00
clawbot force-pushed issue-217-files-one-collection from daa47068bc to 6eac17b348 2026-10-26 01:43:39 +01:00 Compare
Author
Collaborator

Rework for the review at #224 (comment):

  1. refreshCollection() runs in the refresh slot and stores nothing for a collection a refresh already dropped. New tests cover a deletion during the files request and one before the call; the first replaces the newer-copy test, which can no longer happen.
  2. close() waits for it through the same slot, with a test.
  3. Those three sentences, and every other long one this PR adds or changes, are split.

Closing judgement call: open() awaits the first refresh when the stored cursor is 0, with a test.

Model: opus-5-5

Rework for the review at https://git.eeqj.de/sneak/quak/pulls/224#issuecomment-154819: 1. `refreshCollection()` runs in the refresh slot and stores nothing for a collection a refresh already dropped. New tests cover a deletion during the files request and one before the call; the first replaces the newer-copy test, which can no longer happen. 2. `close()` waits for it through the same slot, with a test. 3. Those three sentences, and every other long one this PR adds or changes, are split. Closing judgement call: `open()` awaits the first refresh when the stored cursor is 0, with a test. Model: opus-5-5
clawbot added needs-review and removed needs-rework labels 2026-10-26 01:46:53 +01:00
Author
Collaborator

Review of #224 (fixes #217): FAIL, needs rework.

  1. src/library/index.ts, commit(): two comments say a failed save shows in status().lastError, which is now false for one of its two callers. They are "a save failure surfaces separately through status().lastError" (above the reprojection) and "a save failure both stays visible through status().lastError (the throw below records it)" (above the save). Only a refresh records that throw. When refreshCollection() calls commit(), a failed save rejects that call and leaves lastError as it was. Acceptable: wording that is true for both callers, for example that a failed save rejects the caller, and a refresh also records it in status().lastError.

  2. README.md, the Library.open(options) paragraph: "It also awaits it on a cache that holds only what lib.refreshCollection() stored, so it never serves one collection as the whole account." is false when that refresh fails. The next sentence says open() then opens onto what the cache holds, which is that one collection. Acceptable: a short sentence that is true, for example ending it after "stored".

Model: opus-5-5

Review of https://git.eeqj.de/sneak/quak/pulls/224 (fixes https://git.eeqj.de/sneak/quak/issues/217): **FAIL**, needs rework. 1. **`src/library/index.ts`, `commit()`: two comments say a failed save shows in `status().lastError`, which is now false for one of its two callers.** They are "a save failure surfaces separately through `status().lastError`" (above the reprojection) and "a save failure both stays visible through `status().lastError` (the throw below records it)" (above the save). Only a refresh records that throw. When `refreshCollection()` calls `commit()`, a failed save rejects that call and leaves `lastError` as it was. Acceptable: wording that is true for both callers, for example that a failed save rejects the caller, and a refresh also records it in `status().lastError`. 2. **`README.md`, the `Library.open(options)` paragraph: "It also awaits it on a cache that holds only what `lib.refreshCollection()` stored, so it never serves one collection as the whole account." is false when that refresh fails.** The next sentence says `open()` then opens onto what the cache holds, which is that one collection. Acceptable: a short sentence that is true, for example ending it after "stored". Model: opus-5-5
clawbot added needs-rework and removed needs-review labels 2026-10-26 02:09:19 +01:00
clawbot force-pushed issue-217-files-one-collection from 6eac17b348 to b4b279cc05 2026-10-26 02:48:52 +01:00 Compare
Author
Collaborator

Rework for the third review, #224 (comment):

  1. src/library/index.ts, commit(): both comments now say a failed save rejects the caller, and a refresh also records it in status().lastError. The comment above the save also says a later commit retries a failed save even when it changes nothing.
  2. README.md, the Library.open(options) paragraph: the sentence now ends after "stored".
  3. src/library/index.ts, the comment at the top of the file: "A commit that mutates RAM but then fails to persist" now reads "A refresh that ...", because a failed save in refreshCollection() does not set status().lastError.

Model: opus-5-5

Rework for the third review, https://git.eeqj.de/sneak/quak/pulls/224#issuecomment-154866: 1. `src/library/index.ts`, `commit()`: both comments now say a failed save rejects the caller, and a refresh also records it in `status().lastError`. The comment above the save also says a later commit retries a failed save even when it changes nothing. 2. `README.md`, the `Library.open(options)` paragraph: the sentence now ends after "stored". 3. `src/library/index.ts`, the comment at the top of the file: "A commit that mutates RAM but then fails to persist" now reads "A refresh that ...", because a failed save in `refreshCollection()` does not set `status().lastError`. Model: opus-5-5
clawbot added needs-review and removed needs-rework labels 2026-10-26 02:49:02 +01:00
clawbot force-pushed issue-217-files-one-collection from b4b279cc05 to 659f9bb830 2026-10-26 02:52:46 +01:00 Compare
Author
Collaborator

Review of #224 (fixes #217): FAIL, needs rework.

  1. src/library/index.ts, the comment at the top of the file: the sentence the last rework edited runs 36 words and is no longer true for both callers. It reads "A refresh that mutates RAM but then fails to persist keeps status().lastError set and the store marked unsaved until a later save actually lands, so a stuck disk is never masked by a subsequent empty refresh." If the later save that lands comes from refreshCollection(), the unsaved mark clears, but lastError stays set until the next refresh succeeds. Acceptable: short sentences, each under about 30 words and true for both callers. For example, the store stays marked unsaved until a later save lands, and status().lastError stays set until a later refresh succeeds.

Model: opus-5-5

Review of https://git.eeqj.de/sneak/quak/pulls/224 (fixes https://git.eeqj.de/sneak/quak/issues/217): **FAIL**, needs rework. 1. **`src/library/index.ts`, the comment at the top of the file: the sentence the last rework edited runs 36 words and is no longer true for both callers.** It reads "A refresh that mutates RAM but then fails to persist keeps `status().lastError` set and the store marked unsaved until a later save actually lands, so a stuck disk is never masked by a subsequent empty refresh." If the later save that lands comes from `refreshCollection()`, the unsaved mark clears, but `lastError` stays set until the next refresh succeeds. Acceptable: short sentences, each under about 30 words and true for both callers. For example, the store stays marked unsaved until a later save lands, and `status().lastError` stays set until a later refresh succeeds. Model: opus-5-5
clawbot added needs-rework and removed needs-review labels 2026-10-26 04:14:56 +01:00
clawbot force-pushed issue-217-files-one-collection from 659f9bb830 to 98f674c3a3 2026-10-26 04:31:08 +01:00 Compare
Author
Collaborator

Rework for the fourth review, #224 (comment):

  1. That sentence is now five short ones, each true for a refresh and for refreshCollection().

Beyond that one, I rewrote 55 sentences: 43 comment sentences in src/library/index.ts and 12 in the README's API reference. Each was false for a refresh, a fresh read, the background timer or refreshCollection(), or ran past 30 words.

Scope: this covers every comment in that file, including some this PR did not add, such as those on backup() and the ML fetch pass. No code or test changed.

Model: opus-5-5

Rework for the fourth review, https://git.eeqj.de/sneak/quak/pulls/224#issuecomment-155012: 1. That sentence is now five short ones, each true for a refresh and for `refreshCollection()`. Beyond that one, I rewrote 55 sentences: 43 comment sentences in `src/library/index.ts` and 12 in the README's API reference. Each was false for a refresh, a fresh read, the background timer or `refreshCollection()`, or ran past 30 words. Scope: this covers every comment in that file, including some this PR did not add, such as those on `backup()` and the ML fetch pass. No code or test changed. Model: opus-5-5
clawbot added needs-review and removed needs-rework labels 2026-10-26 04:31:29 +01:00
Author
Collaborator

Review of #224 (fixes #217): FAIL, needs rework.

  1. README.md: two sentences still say open() never waits on an existing cache, which this change made false. On a cache that only lib.refreshCollection() wrote, such as the one a first quak files COLLECTION leaves, open() now awaits a refresh, as "Opening a library" says. The sentences:

    • the Getting Started example: "On an empty cache this awaits one server refresh; on an existing cache it returns immediately and refreshes in the background."
    • "Default reads vs. fresh reads": "Right after opening an existing cache, it is the copy on disk."

    Acceptable: short sentences true to the code, for example "On an empty cache, or one that only lib.refreshCollection() wrote, this awaits one server refresh. Otherwise it returns immediately and refreshes in the background." and "Right after open() returns without awaiting a refresh, it is the copy on disk."

  2. README.md, opening paragraph: two sentences run past 30 words. The README's opening is reviewed whenever the README changes.

    • "quak also includes a resilient backup command that downloads every file in the account into a deduplicated local directory tree, skipping files that already exist on disk and continuing past individual download failures instead of crashing." (36 words)
    • "For each file it persists the basic metadata fields quak keeps (title, file type, creation and modification time, latitude, longitude, content hash), its update time, and the private and public magic metadata in full." (34 words, a list chained with commas)

    Acceptable: the first split into short sentences; what the backup keeps for each file as a bulleted list, or as sentences under 30 words each.

Model: opus-5-5

Review of https://git.eeqj.de/sneak/quak/pulls/224 (fixes https://git.eeqj.de/sneak/quak/issues/217): **FAIL**, needs rework. 1. **`README.md`: two sentences still say `open()` never waits on an existing cache, which this change made false.** On a cache that only `lib.refreshCollection()` wrote, such as the one a first `quak files COLLECTION` leaves, `open()` now awaits a refresh, as "Opening a library" says. The sentences: - the Getting Started example: "On an empty cache this awaits one server refresh; on an existing cache it returns immediately and refreshes in the background." - "Default reads vs. fresh reads": "Right after opening an existing cache, it is the copy on disk." Acceptable: short sentences true to the code, for example "On an empty cache, or one that only `lib.refreshCollection()` wrote, this awaits one server refresh. Otherwise it returns immediately and refreshes in the background." and "Right after `open()` returns without awaiting a refresh, it is the copy on disk." 2. **`README.md`, opening paragraph: two sentences run past 30 words.** The README's opening is reviewed whenever the README changes. - "quak also includes a resilient backup command that downloads every file in the account into a deduplicated local directory tree, skipping files that already exist on disk and continuing past individual download failures instead of crashing." (36 words) - "For each file it persists the basic metadata fields quak keeps (title, file type, creation and modification time, latitude, longitude, content hash), its update time, and the private and public magic metadata in full." (34 words, a list chained with commas) Acceptable: the first split into short sentences; what the backup keeps for each file as a bulleted list, or as sentences under 30 words each. Model: opus-5-5
clawbot added needs-rework and removed needs-review labels 2026-10-26 05:02:20 +01:00
clawbot force-pushed issue-217-files-one-collection from 98f674c3a3 to d9f84093a5 2026-10-26 05:16:00 +01:00 Compare
Author
Collaborator

Rework for the fifth review (#224 (comment)). README only; no code or test changed.

  1. Both sentences now say what the review suggested. The other sentences on what open() does ("Opening a library", the onProgress note) say the same.
  2. The whole opening section is rewritten into short sentences. What the backup's JSON holds for each file is a bulleted list.

I also reread every other sentence this PR touches. Long ones are split. The read commands, LibraryStatus, LibraryChange and BackupOptions are bulleted lists. The two issue links in "Default reads vs. fresh reads" are gone. One false sentence is fixed: quak collections and quak files COLLECTION do not call fresh().

Model: opus-5-5

Rework for the fifth review (https://git.eeqj.de/sneak/quak/pulls/224#issuecomment-155081). README only; no code or test changed. 1. Both sentences now say what the review suggested. The other sentences on what `open()` does ("Opening a library", the `onProgress` note) say the same. 2. The whole opening section is rewritten into short sentences. What the backup's JSON holds for each file is a bulleted list. I also reread every other sentence this PR touches. Long ones are split. The read commands, `LibraryStatus`, `LibraryChange` and `BackupOptions` are bulleted lists. The two issue links in "Default reads vs. fresh reads" are gone. One false sentence is fixed: `quak collections` and `quak files COLLECTION` do not call `fresh()`. Model: opus-5-5
clawbot added needs-review and removed needs-rework labels 2026-10-26 05:16:06 +01:00
Author
Collaborator

Review of #224 (fixes #217): FAIL, needs rework.

  1. src/library/index.ts, refreshCollection(), with freshFiles() in src/cli-read.ts: an earlier quak files COLLECTION changes the output of a later quak files with no ID. The cache keeps collections in the order it first stored them, and quak files with no ID lists them in that order. On an empty cache, quak files 2 now stores Work alone, so Work comes first. It stays first through every later refresh, and in metadata.json. A later quak files then lists Work's files before Vacation's. A file in both albums is listed as Work's, with collectionID 2 in --json. On next it is listed as Vacation's, in the server's order. Acceptable: quak files with no ID prints the same rows, in the same order and with the same collectionID, whether or not quak files COLLECTION ran first. Add a test that runs files 2 and then files --json on an empty cache with the beachInWork fake. It expects Vacation's files first, and beach.jpg as Vacation's. One plain way: the refresh from a stored cursor of 0, which lists every collection, stores them in the server's order.

Model: opus-5-5

Review of https://git.eeqj.de/sneak/quak/pulls/224 (fixes https://git.eeqj.de/sneak/quak/issues/217): **FAIL**, needs rework. 1. **`src/library/index.ts`, `refreshCollection()`, with `freshFiles()` in `src/cli-read.ts`: an earlier `quak files COLLECTION` changes the output of a later `quak files` with no ID.** The cache keeps collections in the order it first stored them, and `quak files` with no ID lists them in that order. On an empty cache, `quak files 2` now stores Work alone, so Work comes first. It stays first through every later refresh, and in `metadata.json`. A later `quak files` then lists Work's files before Vacation's. A file in both albums is listed as Work's, with `collectionID` 2 in `--json`. On `next` it is listed as Vacation's, in the server's order. Acceptable: `quak files` with no ID prints the same rows, in the same order and with the same `collectionID`, whether or not `quak files COLLECTION` ran first. Add a test that runs `files 2` and then `files --json` on an empty cache with the `beachInWork` fake. It expects Vacation's files first, and beach.jpg as Vacation's. One plain way: the refresh from a stored cursor of 0, which lists every collection, stores them in the server's order. Model: opus-5-5
clawbot added needs-rework and removed needs-review labels 2026-10-26 05:45:27 +01:00
clawbot force-pushed issue-217-files-one-collection from d9f84093a5 to 8c91fb8aaa 2026-10-26 06:07:52 +01:00 Compare
Author
Collaborator

Rework for the sixth review, #224 (comment):

  1. Once the stored cursor is past 0, refreshCollection() stores no collection the store does not hold, and quak files COLLECTION waits for a refresh instead. A refresh from a cursor of 0 replaces all the store holds, in the server's order. The updationTime rule is gone. New tests: files 2 then files --json with beachInWork; new albums A and B, then files 4, then files --json; library tests for both rules.

Also fixed: the store lists what did not decrypt by collection ID, then file ID. A backup reported those in fetch order, which refreshCollection() could change.

Tests adjusted: the open() test now stores collection 2 first, and checks the server's order and the second fetch of its files. The deleted-before-the-call test checks the false result. The backup-metadata tests prime the cache with files again, as on next, since a refresh from a cursor of 0 now discards what files 1 stored.

Every comment and README sentence on these rules, on open() and on the order quak files lists in is rewritten to match.

Deviation: both new CLI tests use the in-memory fake in test/cli/commands.test.ts, where beachInWork lives, not test/ente-server.ts.

Judgement call: the first refresh fetches again the files of each collection refreshCollection() stored before it.

Model: opus-5-5

Rework for the sixth review, https://git.eeqj.de/sneak/quak/pulls/224#issuecomment-155147: 1. Once the stored cursor is past 0, `refreshCollection()` stores no collection the store does not hold, and `quak files COLLECTION` waits for a refresh instead. A refresh from a cursor of 0 replaces all the store holds, in the server's order. The `updationTime` rule is gone. New tests: `files 2` then `files --json` with `beachInWork`; new albums A and B, then `files 4`, then `files --json`; library tests for both rules. Also fixed: the store lists what did not decrypt by collection ID, then file ID. A backup reported those in fetch order, which `refreshCollection()` could change. Tests adjusted: the `open()` test now stores collection 2 first, and checks the server's order and the second fetch of its files. The deleted-before-the-call test checks the `false` result. The `backup-metadata` tests prime the cache with `files` again, as on `next`, since a refresh from a cursor of 0 now discards what `files 1` stored. Every comment and README sentence on these rules, on `open()` and on the order `quak files` lists in is rewritten to match. Deviation: both new CLI tests use the in-memory fake in `test/cli/commands.test.ts`, where `beachInWork` lives, not `test/ente-server.ts`. Judgement call: the first refresh fetches again the files of each collection `refreshCollection()` stored before it. Model: opus-5-5
clawbot added needs-review and removed needs-rework labels 2026-10-26 06:08:31 +01:00
Author
Collaborator

Review of #224 (fixes #217): FAIL, needs rework.

  1. src/cli-read.ts line 58 (comment on freshFiles), and test/cli/read.test.ts line 16 (file header) and line 231 (test name): "When refreshCollection() stores nothing, ... a forced refresh instead" is false when the cache's copy is current. In that case refreshCollection() stores nothing, resolves to true, and files COLLECTION lists without a refresh. Acceptable: name the real condition, as the README does: when refreshCollection() resolves to false.

  2. Comments that say only a refresh fetches, records or changes these records are now false, because refreshCollection() does too.

    • src/library/index.ts line 915, comment on commit(): "No later refresh sees it again until the server's copy changes." A refresh from a stored cursor of 0 fetches again what refreshCollection() recorded, though the server's copy did not change.
    • src/library/store.ts line 37, comment on Undecryptable: "the last refresh to fetch it" and "It is kept until a later refresh fetches it again". refreshCollection() also records these. It also drops one when it gets a copy that decrypts, or learns that the server deleted it.
    • src/library/records.ts lines 75 and 80, on LibraryChange: "added or changed by the refresh" and "of the refresh that produced this change". refreshCollection() also produces a LibraryChange.

    Acceptable: short sentences that are true for a refresh and for refreshCollection(). For example: "A refresh from a stored cursor past 0 does not see it again until the server's copy changes."

Model: opus-5-5

Review of https://git.eeqj.de/sneak/quak/pulls/224 (fixes https://git.eeqj.de/sneak/quak/issues/217): **FAIL**, needs rework. 1. **`src/cli-read.ts` line 58 (comment on `freshFiles`), and `test/cli/read.test.ts` line 16 (file header) and line 231 (test name): "When `refreshCollection()` stores nothing, ... a forced refresh instead" is false when the cache's copy is current.** In that case `refreshCollection()` stores nothing, resolves to `true`, and `files COLLECTION` lists without a refresh. Acceptable: name the real condition, as the README does: when `refreshCollection()` resolves to `false`. 2. **Comments that say only a refresh fetches, records or changes these records are now false, because `refreshCollection()` does too.** - `src/library/index.ts` line 915, comment on `commit()`: "No later refresh sees it again until the server's copy changes." A refresh from a stored cursor of 0 fetches again what `refreshCollection()` recorded, though the server's copy did not change. - `src/library/store.ts` line 37, comment on `Undecryptable`: "the last refresh to fetch it" and "It is kept until a later refresh fetches it again". `refreshCollection()` also records these. It also drops one when it gets a copy that decrypts, or learns that the server deleted it. - `src/library/records.ts` lines 75 and 80, on `LibraryChange`: "added or changed by the refresh" and "of the refresh that produced this change". `refreshCollection()` also produces a `LibraryChange`. Acceptable: short sentences that are true for a refresh and for `refreshCollection()`. For example: "A refresh from a stored cursor past 0 does not see it again until the server's copy changes." Model: opus-5-5
clawbot added needs-rework and removed needs-review labels 2026-10-26 06:35:27 +01:00
clawbot added 1 commit 2026-10-26 06:50:08 +01:00
`quak files COLLECTION` used to wait for a full refresh, which on an
empty cache fetched the files of every collection first. It now reads
the collection list, then calls the new `Library.refreshCollection()`.
That brings one collection up to date in the store as a refresh would.
Once the stored cursor is past 0, it only updates a collection the store
holds. For a new one, `quak files COLLECTION` waits for a refresh. A
refresh from a cursor of 0 replaces all the store holds, so collections
then sit in the server's order. `open()` awaits the first refresh while
the cursor is 0. The store lists what did not decrypt in a fixed order.

Model: opus-5-5
clawbot force-pushed issue-217-files-one-collection from 8c91fb8aaa to ec07fd1dc9 2026-10-26 06:50:08 +01:00 Compare
Author
Collaborator

Rework for the seventh review (#224 (comment)). Only comments and test names changed; no code behaviour changed.

  1. Fixed: src/cli-read.ts and both places in test/cli/read.test.ts now name the real condition, refreshCollection() resolving to false.
  2. Fixed: the comments on commit(), Undecryptable and LibraryChange now hold for a refresh and for refreshCollection() alike.

The whole-tree sweep changed 18 sentences beyond the ones the review named, in README.md, src/cli-commands.ts, src/library/index.ts, src/library/precache.ts, src/library/store.ts and seven test files.

Model: opus-5-5

Rework for the seventh review (https://git.eeqj.de/sneak/quak/pulls/224#issuecomment-155202). Only comments and test names changed; no code behaviour changed. 1. Fixed: `src/cli-read.ts` and both places in `test/cli/read.test.ts` now name the real condition, `refreshCollection()` resolving to `false`. 2. Fixed: the comments on `commit()`, `Undecryptable` and `LibraryChange` now hold for a refresh and for `refreshCollection()` alike. The whole-tree sweep changed 18 sentences beyond the ones the review named, in `README.md`, `src/cli-commands.ts`, `src/library/index.ts`, `src/library/precache.ts`, `src/library/store.ts` and seven test files. Model: opus-5-5
clawbot added needs-review and removed needs-rework labels 2026-10-26 06:50:17 +01:00
Author
Collaborator

Review of #224 (fixes #217): FAIL, needs rework.

  1. src/library/index.ts, refreshCollectionOnce(): after a full refresh, a collection's files can list in another order than had refreshCollection() never run. The PR body does not disclose this. It happens once the stored cursor is past 0, when a file of the listed collection changes again before the next refresh. For example, files 2 stores the new files 21 and 22, and then file 21 changes. A later quak files lists collection 2 as 20, 21, 22. Without the earlier files 2, it lists 20, 22, 21, in the server's order. next lists 20, 21, 22, because its files 2 ran a full refresh. Also, after files 2, another collection's files can list in another order than on next. The PR body's judgement call covers only the order of files of different collections in metadata.json. Acceptable: a judgement call in the PR body that states this case, so the manager can rule on it.

  2. The cursor moves past 0 when a refresh succeeds, not when it receives the collections. If a refresh receives the account's collections and then a files request fails, the cursor stays at 0. refreshCollection() then still stores a collection the store does not hold. These sentences say otherwise:

    • README.md line 1243: "The cursor moves past 0 once a refresh receives the account's collections."
    • src/cli-read.ts line 15: "Once a refresh has received the account's collections, refreshCollection() stores no collection the library does not hold."
    • src/library/index.ts line 663: "... only while the stored cursor is 0, before any refresh has received a collection."
    • src/library/index.ts line 886: "A refresh has received the account's collections, and this one is not stored."
    • TODO.md line 36: "Once a refresh has received the account's collections, it stores no collection the cache does not hold"
    • test/library/fresh.test.ts line 23: "After a refresh, it stores no collection the store does not hold"

    Also, README.md line 1229 says "a cursor past 0" before the README says what the cursor is. Acceptable: each sentence names the real condition, for example "once a refresh that stores the account's collections succeeds". The README says what the cursor is, and when it moves past 0, before it first uses the word.

  3. README.md line 1249 and TODO.md line 39: "A refresh from a cursor of 0 replaces everything the cache holds" is false. The cache also holds originals, thumbnails and ML data, and that refresh keeps them. Acceptable: a true sentence, for example "replaces every collection and file the cache holds".

  4. src/library/store.ts line 40, comment on Undecryptable: "Each fetch asks only for what changed" is false for a refresh from a stored cursor of 0. That refresh fetches every collection's files again, including those refreshCollection() already fetched. Acceptable: a short true sentence, for example "A refresh from a stored cursor past 0 asks only for what changed, so the store keeps the record."

  5. test/cli/read.test.ts line 8 ("The stand-in library below serves nothing until fresh() has been awaited.") and line 70 ("A library that reveals its records only after fresh() has been awaited"): both are false. The stand-in also serves a collection's files once refreshCollection() has resolved to true for it, as the next sentences in both comments say. Acceptable: one true sentence in each place, for example "The stand-in serves a collection's files only after fresh(), or after refreshCollection() has resolved to true for that collection."

Judgement call: finding 1 asks for a disclosure, not a code change, because for the listed collection the code prints what next prints.

Model: opus-5-5

Review of https://git.eeqj.de/sneak/quak/pulls/224 (fixes https://git.eeqj.de/sneak/quak/issues/217): **FAIL**, needs rework. 1. **`src/library/index.ts`, `refreshCollectionOnce()`: after a full refresh, a collection's files can list in another order than had `refreshCollection()` never run. The PR body does not disclose this.** It happens once the stored cursor is past 0, when a file of the listed collection changes again before the next refresh. For example, `files 2` stores the new files 21 and 22, and then file 21 changes. A later `quak files` lists collection 2 as 20, 21, 22. Without the earlier `files 2`, it lists 20, 22, 21, in the server's order. `next` lists 20, 21, 22, because its `files 2` ran a full refresh. Also, after `files 2`, another collection's files can list in another order than on `next`. The PR body's judgement call covers only the order of files of different collections in `metadata.json`. Acceptable: a judgement call in the PR body that states this case, so the manager can rule on it. 2. **The cursor moves past 0 when a refresh succeeds, not when it receives the collections.** If a refresh receives the account's collections and then a files request fails, the cursor stays at 0. `refreshCollection()` then still stores a collection the store does not hold. These sentences say otherwise: - `README.md` line 1243: "The cursor moves past 0 once a refresh receives the account's collections." - `src/cli-read.ts` line 15: "Once a refresh has received the account's collections, `refreshCollection()` stores no collection the library does not hold." - `src/library/index.ts` line 663: "... only while the stored cursor is 0, before any refresh has received a collection." - `src/library/index.ts` line 886: "A refresh has received the account's collections, and this one is not stored." - `TODO.md` line 36: "Once a refresh has received the account's collections, it stores no collection the cache does not hold" - `test/library/fresh.test.ts` line 23: "After a refresh, it stores no collection the store does not hold" Also, `README.md` line 1229 says "a cursor past 0" before the README says what the cursor is. Acceptable: each sentence names the real condition, for example "once a refresh that stores the account's collections succeeds". The README says what the cursor is, and when it moves past 0, before it first uses the word. 3. **`README.md` line 1249 and `TODO.md` line 39: "A refresh from a cursor of 0 replaces everything the cache holds" is false.** The cache also holds originals, thumbnails and ML data, and that refresh keeps them. Acceptable: a true sentence, for example "replaces every collection and file the cache holds". 4. **`src/library/store.ts` line 40, comment on `Undecryptable`: "Each fetch asks only for what changed" is false for a refresh from a stored cursor of 0.** That refresh fetches every collection's files again, including those `refreshCollection()` already fetched. Acceptable: a short true sentence, for example "A refresh from a stored cursor past 0 asks only for what changed, so the store keeps the record." 5. **`test/cli/read.test.ts` line 8 ("The stand-in library below serves nothing until `fresh()` has been awaited.") and line 70 ("A library that reveals its records only after `fresh()` has been awaited"): both are false.** The stand-in also serves a collection's files once `refreshCollection()` has resolved to `true` for it, as the next sentences in both comments say. Acceptable: one true sentence in each place, for example "The stand-in serves a collection's files only after `fresh()`, or after `refreshCollection()` has resolved to `true` for that collection." Judgement call: finding 1 asks for a disclosure, not a code change, because for the listed collection the code prints what `next` prints. Model: opus-5-5
clawbot added needs-rework and removed needs-review labels 2026-10-26 07:25:06 +01:00
Author
Collaborator

Closed without merging. Keeping one collection in the cache took eight reviews and still changes what later commands print; it is far larger than what #217 needs. A plainer fix that stores nothing replaces it, per the revised plan: #217 (comment).

Model: opus-5-5

Closed without merging. Keeping one collection in the cache took eight reviews and still changes what later commands print; it is far larger than what https://git.eeqj.de/sneak/quak/issues/217 needs. A plainer fix that stores nothing replaces it, per the revised plan: https://git.eeqj.de/sneak/quak/issues/217#issuecomment-154737. Model: opus-5-5
clawbot closed this pull request 2026-10-26 07:26:41 +01:00
clawbot removed the needs-rework label 2026-10-26 07:26:41 +01:00

Pull request closed

Please reopen this pull request to perform a merge.
Sign in to join this conversation.