Resumable, deletion-aware collection and file enumeration (closes #38) #57

Merged
clawbot merged 1 commits from issue-38-resumable-enumeration into next 2026-09-22 12:37:11 +02:00
Collaborator

Implements #38 and closes
#7.

What changed

Two new Client methods for the cache refresh:

  • collectionsSince({ sinceTime }) returns live decrypted collections, a
    deleted list of tombstoned collection ids, and a cursor (the max
    updationTime seen, tombstones included, falling back to the input when the
    response is empty).
  • filesSince({ collectionID, collectionKey, sinceTime }) paginates the diff
    from the given cursor and returns the same shape for files.

listCollections/listFiles are now thin wrappers that enumerate from
sinceTime 0 and drop deletions, so existing callers and their tests are
unchanged.

Tombstone return shape (the design decision the issue left open)

Live records and deleted ids are kept in separate fields rather than returning a
minimal EnteFile/Collection for a tombstone. A tombstone has no key or
metadata to decrypt, so a hollow record would either lie about its fields or
fail strict tsc. A bare id list is honest and type-clean.

Non-advancing server (issue 7)

The old listFiles loop trusted hasMore and advanced its cursor only from
rows it saw, so a page that reported more while its max updationTime did not
exceed the cursor looped forever. filesSince now throws a clear error in that
case; the wrapper inherits the fix. Covered by a test, alongside cursor
threading (page then empty diff) and tombstone surfacing.

Model: opus-4-8

Implements https://git.eeqj.de/sneak/quak/issues/38 and closes https://git.eeqj.de/sneak/quak/issues/7. ## What changed Two new `Client` methods for the cache refresh: - `collectionsSince({ sinceTime })` returns live decrypted collections, a `deleted` list of tombstoned collection ids, and a `cursor` (the max `updationTime` seen, tombstones included, falling back to the input when the response is empty). - `filesSince({ collectionID, collectionKey, sinceTime })` paginates the diff from the given cursor and returns the same shape for files. `listCollections`/`listFiles` are now thin wrappers that enumerate from `sinceTime` 0 and drop deletions, so existing callers and their tests are unchanged. ## Tombstone return shape (the design decision the issue left open) Live records and deleted ids are kept in separate fields rather than returning a minimal `EnteFile`/`Collection` for a tombstone. A tombstone has no key or metadata to decrypt, so a hollow record would either lie about its fields or fail strict tsc. A bare id list is honest and type-clean. ## Non-advancing server (issue 7) The old `listFiles` loop trusted `hasMore` and advanced its cursor only from rows it saw, so a page that reported more while its max `updationTime` did not exceed the cursor looped forever. `filesSince` now throws a clear error in that case; the wrapper inherits the fix. Covered by a test, alongside cursor threading (page then empty diff) and tombstone surfacing. Model: opus-4-8
clawbot added the needs-review label 2026-09-22 12:20:01 +02:00
clawbot self-assigned this 2026-09-22 12:20:01 +02:00
clawbot added 1 commit 2026-09-22 12:20:01 +02:00
Add collectionsSince/filesSince on Client: they take a starting cursor,
decrypt live records, surface tombstoned ids in a separate `deleted`
list, and return the max updationTime seen as the cursor to resume from.
A tombstone has no key or metadata to decrypt, so it is a bare id rather
than a hollowed-out record that strict tsc would reject.

filesSince paginates the diff from the given cursor and, when the server
reports hasMore but the page's max updationTime does not exceed the
cursor it was fetched with, throws instead of looping forever.

listCollections/listFiles become thin wrappers that enumerate from
sinceTime 0 and drop deletions, so existing callers and tests are
unaffected.

Closes #7.

Model: opus-4-8
Author
Collaborator

PASS — collectionsSince/filesSince take a starting cursor, surface tombstoned ids in a separate deleted list, return the resumable max-updationTime cursor (tombstones included), and filesSince throws instead of looping when the server reports hasMore without advancing (#7); listCollections/listFiles are unaffected thin wrappers, the tombstone shape is a type-clean bare id, tests are meaningful (mutation-checked), and make check is green against current next.

Model: opus-4-8

PASS — `collectionsSince`/`filesSince` take a starting cursor, surface tombstoned ids in a separate `deleted` list, return the resumable max-`updationTime` cursor (tombstones included), and `filesSince` throws instead of looping when the server reports `hasMore` without advancing (https://git.eeqj.de/sneak/quak/issues/7); `listCollections`/`listFiles` are unaffected thin wrappers, the tombstone shape is a type-clean bare id, tests are meaningful (mutation-checked), and `make check` is green against current `next`. Model: opus-4-8
clawbot merged commit 42a6c17d49 into next 2026-09-22 12:37:11 +02:00
clawbot deleted branch issue-38-resumable-enumeration 2026-09-22 12:37:11 +02:00
Sign in to join this conversation.