pulls all collection data when listing files for a single collection #217

Closed
opened 2026-10-22 19:01:38 +02:00 by sneak · 3 comments
Owner
user@andros:~/dev/quak$ ./dist/bin/quak.js files -v -v 1580559963372908
Refresh started
GET https://api.ente.io/collections/v2?sinceTime=0: HTTP 200
GET https://api.ente.io/collections/v2/diff?collectionID=1580559963143563&sinceTime=0: HTTP 200
Fetched the files of 1 of 102 collections
GET https://api.ente.io/collections/v2/diff?collectionID=1580559963002374&sinceTime=0: HTTP 200
Fetched the files of 2 of 102 collections
GET https://api.ente.io/collections/v2/diff?collectionID=1580559962946858&sinceTime=0: HTTP 200
Fetched the files of 3 of 102 collections
GET https://api.ente.io/collections/v2/diff?collectionID=1580559962954821&sinceTime=0: HTTP 200
Fetched the files of 4 of 102 collections
GET https://api.ente.io/collections/v2/diff?collectionID=1580559963196018&sinceTime=0: HTTP 200
GET https://api.ente.io/collections/v2/diff?collectionID=1580559963196018&sinceTime=1733898973271689: HTTP 200
GET https://api.ente.io/collections/v2/diff?collectionID=1580559963196018&sinceTime=1733903920270083: HTTP 200
GET https://api.ente.io/collections/v2/diff?collectionID=1580559963196018&sinceTime=1733908260469651: HTTP 200
Fetched the files of 5 of 102 collections
GET https://api.ente.io/collections/v2/diff?collectionID=1580559962946951&sinceTime=0: HTTP 200
Fetched the files of 6 of 102 collections
GET https://api.ente.io/collections/v2/diff?collectionID=1580559964347222&sinceTime=0: HTTP 200
Fetched the files of 7 of 102 collections
^C
``` user@andros:~/dev/quak$ ./dist/bin/quak.js files -v -v 1580559963372908 Refresh started GET https://api.ente.io/collections/v2?sinceTime=0: HTTP 200 GET https://api.ente.io/collections/v2/diff?collectionID=1580559963143563&sinceTime=0: HTTP 200 Fetched the files of 1 of 102 collections GET https://api.ente.io/collections/v2/diff?collectionID=1580559963002374&sinceTime=0: HTTP 200 Fetched the files of 2 of 102 collections GET https://api.ente.io/collections/v2/diff?collectionID=1580559962946858&sinceTime=0: HTTP 200 Fetched the files of 3 of 102 collections GET https://api.ente.io/collections/v2/diff?collectionID=1580559962954821&sinceTime=0: HTTP 200 Fetched the files of 4 of 102 collections GET https://api.ente.io/collections/v2/diff?collectionID=1580559963196018&sinceTime=0: HTTP 200 GET https://api.ente.io/collections/v2/diff?collectionID=1580559963196018&sinceTime=1733898973271689: HTTP 200 GET https://api.ente.io/collections/v2/diff?collectionID=1580559963196018&sinceTime=1733903920270083: HTTP 200 GET https://api.ente.io/collections/v2/diff?collectionID=1580559963196018&sinceTime=1733908260469651: HTTP 200 Fetched the files of 5 of 102 collections GET https://api.ente.io/collections/v2/diff?collectionID=1580559962946951&sinceTime=0: HTTP 200 Fetched the files of 6 of 102 collections GET https://api.ente.io/collections/v2/diff?collectionID=1580559964347222&sinceTime=0: HTTP 200 Fetched the files of 7 of 102 collections ^C ```
sneak added this to the 1.0.0 milestone 2026-10-22 19:02:34 +02:00
Collaborator

Recorded as a bug: quak files COLLECTION must fetch only that collection's files, not every collection's. quak's manager fixes it, with a test that listing one collection requests no other collection, when the fleet restarts at 23:00 UTC on 25 October.

Model: opus-5-5

Recorded as a bug: `quak files COLLECTION` must fetch only that collection's files, not every collection's. quak's manager fixes it, with a test that listing one collection requests no other collection, when the fleet restarts at 23:00 UTC on 25 October. Model: opus-5-5
clawbot self-assigned this 2026-10-22 19:06:24 +02:00
Collaborator

Plan (revised 26 October, replacing the one #224 built and which is closed).

What is wrong. quak files COLLECTION calls freshFiles in src/cli-read.ts, which awaits lib.fresh(): a full refresh that fetches the files of every new or changed collection before it lists the one asked for. On an empty cache that is every collection in the account.

Why the plan changed. Storing the one collection in the cache, as the first plan said, made later commands print differently depending on what ran before, and needed rules for the cursor, refresh order and opening the library. Eight reviews kept finding more. A command that lists one collection does not need the cache at all.

Fix.

  • Add a Library method in src/library/index.ts, beside freshCollections(), that returns one collection's live files from the server, given its record from freshCollections(). It calls client.filesSince from the beginning of time and returns the files that decrypt, in the server's order. Like freshCollections(), it stores nothing and touches no refresh, cursor or saved cache.
  • freshFiles(lib, collectionID) calls lib.freshCollections(), returns undefined when the ID is not among them, and otherwise returns that collection's files from the new method. quak files with no argument keeps the full refresh.
  • Rows, --json, exit codes and the treatment of files that do not decrypt stay as on next. Rows come in the server's order. The progress lines of a full refresh no longer appear for this case.
  • Nothing else changes: no change to open(), refreshes, the store or the content cache.
  • Update the comment at the top of src/cli-read.ts, the FreshReadLibrary stand-in in test/cli/read.test.ts, and any README sentence about what quak files COLLECTION fetches.

Tests. With quak's real Client against the fake server in test/ente-server.ts, holding at least three collections: listing one sends a /collections/v2/diff request for that collection and for no other. Listing one leaves the saved cache as it was, and a later quak files prints what it prints on next.

Done when both tests pass and nothing else quak files COLLECTION prints changes.

Model: opus-5-5

Plan (revised 26 October, replacing the one https://git.eeqj.de/sneak/quak/pulls/224 built and which is closed). **What is wrong.** `quak files COLLECTION` calls `freshFiles` in `src/cli-read.ts`, which awaits `lib.fresh()`: a full refresh that fetches the files of every new or changed collection before it lists the one asked for. On an empty cache that is every collection in the account. **Why the plan changed.** Storing the one collection in the cache, as the first plan said, made later commands print differently depending on what ran before, and needed rules for the cursor, refresh order and opening the library. Eight reviews kept finding more. A command that lists one collection does not need the cache at all. **Fix.** - Add a `Library` method in `src/library/index.ts`, beside `freshCollections()`, that returns one collection's live files from the server, given its record from `freshCollections()`. It calls `client.filesSince` from the beginning of time and returns the files that decrypt, in the server's order. Like `freshCollections()`, it stores nothing and touches no refresh, cursor or saved cache. - `freshFiles(lib, collectionID)` calls `lib.freshCollections()`, returns `undefined` when the ID is not among them, and otherwise returns that collection's files from the new method. `quak files` with no argument keeps the full refresh. - Rows, `--json`, exit codes and the treatment of files that do not decrypt stay as on `next`. Rows come in the server's order. The progress lines of a full refresh no longer appear for this case. - Nothing else changes: no change to `open()`, refreshes, the store or the content cache. - Update the comment at the top of `src/cli-read.ts`, the `FreshReadLibrary` stand-in in `test/cli/read.test.ts`, and any README sentence about what `quak files COLLECTION` fetches. **Tests.** With quak's real `Client` against the fake server in `test/ente-server.ts`, holding at least three collections: listing one sends a `/collections/v2/diff` request for that collection and for no other. Listing one leaves the saved cache as it was, and a later `quak files` prints what it prints on `next`. **Done when** both tests pass and nothing else `quak files COLLECTION` prints changes. Model: opus-5-5
Collaborator

Built in #224. quak files COLLECTION reads the collection list, then brings that one collection up to date in the cache through the new Library.refreshCollection(). It fetches no other collection's files, and on a repeat run, when the cached copy is current, none at all.

Model: opus-5-5

Built in https://git.eeqj.de/sneak/quak/pulls/224. `quak files COLLECTION` reads the collection list, then brings that one collection up to date in the cache through the new `Library.refreshCollection()`. It fetches no other collection's files, and on a repeat run, when the cached copy is current, none at all. Model: opus-5-5
Sign in to join this conversation.
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sneak/quak#217