sneak/quak
1
0
Fork 0
Code Issues 8 Pull Requests 2 Actions Packages Projects Releases Wiki Activity

On-disk content and thumbnail cache, with per-photo fetch and thumbnail prefetch #46

New Issue
Closed
opened 2026-09-22 09:11:06 +02:00 by clawbot · 1 comment
clawbot commented 2026-09-22 09:11:06 +02:00
Collaborator
Copy Link
Copy Source

Cache/API implementation (#36), phase 3.
Depends on the request-pools unit, the streaming-decrypt unit, and the read
surface.

Goal

The on-disk content and thumbnail cache keyed by fileID, plus the per-photo
fetch and the thumbnail-prefetch surface.

Scope

  • Layout under cacheDirectory: originals/<fileID>.<ext> and
    thumbnails/<fileID>.<ext>. Flat dirs; no sharding until a real account
    proves it needed. 0700/0600.
  • Present-means-complete: content appears only by rename (streaming atomic
    writer), so a file that exists is whole. No "downloaded" flag stored anywhere;
    the directory listing taken at open() is the record of what is cached. Reap
    orphan temp files on open.
  • Photo.original({ onProgress? }) / Photo.thumbnail({ onProgress? }):
    return { path, bytes }; if present (cache or downloadDirectory) → one
    skipped event, no network; else fetch through the content/thumbnail pool,
    verify, store, return the path. Set PhotoRecord.thumbnailPath/originalPath
    when cached.
  • lib.thumbnails.ensure({ fileIDs; priority: "visible"|"ahead"|"background"; signal?; onProgress? })
    → per-file { fileID, path?, error? }; drives the thumbnail pool with
    priority; AbortSignal drops queued work (in-flight finishes and is kept);
    dedup so a fileID requested twice downloads once.
  • Integrity: compare content hash to FileMetadata.hash when present and store
    it; fall back to fileSize when hash absent. Confirm the exact hash
    construction (algorithm, live-photo combined case) against fixtures before
    relying on it.

Definition of done

  • Fetch-once-then-cached behaviour; skipped on a present file; paths returned.
  • thumbnails.ensure respects priority, dedup, and abort; tested.
  • Orphan temp reaping on open; integrity check wired. make check green.

Grounding

src/library/. Depends on pools + streaming decrypt + read surface. Eviction is
the next unit; precache the one after.

Dispatch notes: TDD; no scripted edits; no interactive questions; plain language.
Squash subject ends (closes #<this issue>). End every message with
Model: opus-4-8.

Model: opus-4-8

Cache/API implementation (https://git.eeqj.de/sneak/quak/issues/36), phase 3. Depends on the request-pools unit, the streaming-decrypt unit, and the read surface. ## Goal The on-disk content and thumbnail cache keyed by `fileID`, plus the per-photo fetch and the thumbnail-prefetch surface. ## Scope - Layout under `cacheDirectory`: `originals/<fileID>.<ext>` and `thumbnails/<fileID>.<ext>`. Flat dirs; no sharding until a real account proves it needed. `0700`/`0600`. - Present-means-complete: content appears only by rename (streaming atomic writer), so a file that exists is whole. No "downloaded" flag stored anywhere; the directory listing taken at `open()` is the record of what is cached. Reap orphan temp files on open. - `Photo.original({ onProgress? })` / `Photo.thumbnail({ onProgress? })`: return `{ path, bytes }`; if present (cache or `downloadDirectory`) → one `skipped` event, no network; else fetch through the content/thumbnail pool, verify, store, return the path. Set `PhotoRecord.thumbnailPath`/`originalPath` when cached. - `lib.thumbnails.ensure({ fileIDs; priority: "visible"|"ahead"|"background"; signal?; onProgress? })` → per-file `{ fileID, path?, error? }`; drives the thumbnail pool with priority; `AbortSignal` drops queued work (in-flight finishes and is kept); dedup so a fileID requested twice downloads once. - Integrity: compare content hash to `FileMetadata.hash` when present and store it; fall back to `fileSize` when `hash` absent. Confirm the exact hash construction (algorithm, live-photo combined case) against fixtures before relying on it. ## Definition of done - Fetch-once-then-cached behaviour; `skipped` on a present file; paths returned. - `thumbnails.ensure` respects priority, dedup, and abort; tested. - Orphan temp reaping on open; integrity check wired. `make check` green. ## Grounding `src/library/`. Depends on pools + streaming decrypt + read surface. Eviction is the next unit; precache the one after. Dispatch notes: TDD; no scripted edits; no interactive questions; plain language. Squash subject ends ` (closes #<this issue>)`. End every message with `Model: opus-4-8`. Model: opus-4-8
clawbot commented 2026-09-22 17:49:14 +02:00
Author
Collaborator
Copy Link
Copy Source

Built in #66 (branch issue-46-content-cache, base next).

The cache stores originals and thumbnails by fileID under cacheDirectory (0700/0600); a file appears only by the atomic rename, so present means complete, and orphan temp files are reaped at open. Photo.original/thumbnail return the path (a skipped event when already cached or already in the backup directory), and lib.thumbnails.ensure drives the thumbnail pool with priority, dedup, and abort. Cached paths are set on the photo records.

One open question for you on the PR: the content-hash / fileSize integrity comparison is deferred, because the only hash fixture in the repo is a placeholder and the hash construction cannot be confirmed against fixtures (the precondition this issue names). Integrity is meanwhile enforced by the streaming decrypt (authenticated, complete only on TAG_FINAL) plus a non-empty check.

make check is green.

Model: opus-4-8

Built in https://git.eeqj.de/sneak/quak/pulls/66 (branch `issue-46-content-cache`, base `next`). The cache stores originals and thumbnails by fileID under `cacheDirectory` (`0700`/`0600`); a file appears only by the atomic rename, so present means complete, and orphan temp files are reaped at open. `Photo.original`/`thumbnail` return the path (a skipped event when already cached or already in the backup directory), and `lib.thumbnails.ensure` drives the thumbnail pool with priority, dedup, and abort. Cached paths are set on the photo records. One open question for you on the PR: the content-hash / fileSize integrity comparison is deferred, because the only hash fixture in the repo is a placeholder and the hash construction cannot be confirmed against fixtures (the precondition this issue names). Integrity is meanwhile enforced by the streaming decrypt (authenticated, complete only on TAG_FINAL) plus a non-empty check. `make check` is green. Model: opus-4-8
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
merge-ready
Review passed; ready to merge
 needs-checks
CI / make check is not green on this PR
 needs-rebase
PR conflicts with main and must be rebased
 needs-review
PR is awaiting adversarial review
 needs-rework
Review failed; PR needs changes
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
clawbot sneak (Jeffrey Paul)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sneak/quak#46