Three bounded request pools for metadata, content, and thumbnails (closes #45) #63

Merged
clawbot merged 1 commits from issue-45-request-pools into next 2026-09-22 15:31:21 +02:00
Collaborator

Adds src/library/pools.ts: a generic BoundedPool and a RequestPools bundle
of three, resolving #45. Nothing wires them
into Library yet — that belongs to the content-cache unit
(#46), so this PR is the pools module plus
its test and touches nothing else.

What a reader should know that the diff does not show:

  • A pool holds a task's slot for that task's whole lifetime. Retries live inside
    the task function, so a retrying request keeps its one slot between attempts —
    which is how "retries consume pool slots" is met without the pool knowing
    anything about the retry policy.
  • Dedup is by caller-supplied key and covers any still-pending task, queued or
    running; the key frees the instant that task settles (success or failure), so
    a later request for the same id runs afresh. A shared submission takes the
    priority of the first submitter — a deliberate simplification, not a promotion
    queue.
  • Priority is only about which waiter takes a freeing slot (on-demand before
    background, FIFO within a priority); it never preempts a running task.
  • The three pools are independent instances, so an idle pool cannot lend slots
    to a busy one — verified by test rather than by construction alone.

Gate: make check green (lint via Dockerfile.lint in Docker, as designed).

Model: opus-4-8

Adds `src/library/pools.ts`: a generic `BoundedPool` and a `RequestPools` bundle of three, resolving https://git.eeqj.de/sneak/quak/issues/45. Nothing wires them into `Library` yet — that belongs to the content-cache unit (https://git.eeqj.de/sneak/quak/issues/46), so this PR is the pools module plus its test and touches nothing else. What a reader should know that the diff does not show: - A pool holds a task's slot for that task's whole lifetime. Retries live inside the task function, so a retrying request keeps its one slot between attempts — which is how "retries consume pool slots" is met without the pool knowing anything about the retry policy. - Dedup is by caller-supplied `key` and covers any still-pending task, queued or running; the key frees the instant that task settles (success or failure), so a later request for the same id runs afresh. A shared submission takes the priority of the first submitter — a deliberate simplification, not a promotion queue. - Priority is only about which waiter takes a freeing slot (on-demand before background, FIFO within a priority); it never preempts a running task. - The three pools are independent instances, so an idle pool cannot lend slots to a busy one — verified by test rather than by construction alone. Gate: `make check` green (lint via `Dockerfile.lint` in Docker, as designed). Model: opus-4-8
clawbot added 1 commit 2026-09-22 15:20:06 +02:00
A self-contained `src/library/pools.ts`: a generic `BoundedPool` concurrency
limiter and a `RequestPools` bundle of three at the design's caps (metadata 10,
content 5, thumbnails 25, each overridable). Within a pool, on-demand work is
served before background/precache work, first-come-first-served within a
priority; a task submitted under a `key` already pending is shared rather than
run twice, and the key frees the moment that task settles. The slot is held for
the task's whole lifetime, so a request that retries internally stays counted
against the cap between attempts.

Pools only; nothing wires them into `Library` yet — that is a later unit.

Model: opus-4-8
clawbot added the needs-review label 2026-09-22 15:20:10 +02:00
clawbot self-assigned this 2026-09-22 15:20:10 +02:00
Author
Collaborator

PASS — three independent bounded pools (metadata 10, content 5, thumbnails 25, each overridable) with every cap enforced, on-demand-before-background priority, in-flight dedup that runs a shared key once, and a slot held across a task's internal retries, delivered as a self-contained src/library/pools.ts that leaves Library, src/library/index.ts and ApiClient untouched; make check is green on the current next and each behaviour fails the suite under mutation.

Model: opus-4-8

**PASS** — three independent bounded pools (metadata 10, content 5, thumbnails 25, each overridable) with every cap enforced, on-demand-before-background priority, in-flight dedup that runs a shared key once, and a slot held across a task's internal retries, delivered as a self-contained `src/library/pools.ts` that leaves `Library`, `src/library/index.ts` and `ApiClient` untouched; `make check` is green on the current `next` and each behaviour fails the suite under mutation. Model: opus-4-8
clawbot merged commit 000d395c87 into next 2026-09-22 15:31:21 +02:00
clawbot deleted branch issue-45-request-pools 2026-09-22 15:31:21 +02:00
Sign in to join this conversation.