5 Commits
Author SHA1 Message Date
clawbot c44d43be94 Test that a late hit and a disabled cache are counted right (closes #56)
check / check (push) Failing after 2m49s
A hit served after the request context has ended must still move the hit
count, and a disabled disk cache must report no items and no size even
when its database holds rows from an earlier run.

Model: opus-5-5
2026-09-29 02:45:50 +00:00
clawbot 8abac98174 Count interrupted misses and the upstream bytes they read (closes #56)
check / check (push) Successful in 3m2s
The miss and transform counters are written with context.WithoutCancel,
so a client disconnect or the request timeout during or after the work
no longer loses them. A failed read of the upstream body now returns
the bytes read before the error, so an over-size or cut-off body still
moves the upstream fetch counters. processFromSourceOrFetch passes the
cached source's length directly instead of through a local named
fetchBytes.

Model: opus-5-5
2026-09-29 01:56:21 +00:00
clawbot 4543bad14a Test that cache stats count interrupted misses (closes #56)
Checks every cache_stats counter after a miss whose request context
ends during or after the upstream fetch, and after one whose upstream
body is over the size limit.

Model: opus-5-5
2026-09-29 01:56:21 +00:00
clawbot d3c9fb89fa Make the cache stats count what is cached, fetched and transcoded (closes #56)
Stats read request_cache and output_content, which nothing writes, so
TotalItems and TotalSizeBytes were always 0. They now count
source_content plus variant_content, the size through UsageBytes; a
failed query is still logged at warn. Get counts a miss after the work,
passing the bytes fetched from upstream (0 for a cached source; still
counted when the fetched source then fails), so upstream_fetch_count and
upstream_fetch_bytes move. transform_count is incremented after each
successful image processor call. request_cache and output_content stay
in the schema; dropping them is a separate decision.

Model: opus-5-5
2026-09-29 01:56:21 +00:00
clawbot d15178534e Test that the cache stats counters and totals move (closes #56)
Failing tests, committed ahead of the fix. Stats totals are checked
after storing a source image and two processed variants. A walk through
Service.Get (a miss that fetches, a hit, a miss that reuses the cached
source, a source failing the magic byte check, a source not found)
checks every cache_stats counter after each step. The warn-log test for
the Stats queries now drops source_content and variant_content, the
tables Stats will read.

Model: opus-5-5
2026-09-29 01:55:56 +00:00
5 changed files with 22 additions and 39 deletions
+2 -6
View File
@@ -68,12 +68,8 @@ RUN apk add --no-cache \
COPY --from=builder /pixad /usr/local/bin/pixad COPY --from=builder /pixad /usr/local/bin/pixad
COPY deploy/docker-entrypoint.sh /usr/local/bin/docker-entrypoint.sh COPY deploy/docker-entrypoint.sh /usr/local/bin/docker-entrypoint.sh
# Create non-root user, config directory, and data directory. pixad # Create non-root user, config directory, and data directory
# gets uid and gid 65532, which host login and system accounts do not RUN adduser -D -H -s /sbin/nologin pixad && \
# use: a bind-mounted /var/lib/pixa is given to pixad, and on the host
# it must not belong to a person's account.
RUN addgroup -g 65532 pixad && \
adduser -D -H -s /sbin/nologin -u 65532 -G pixad pixad && \
mkdir -p /var/lib/pixa /etc/pixa && \ mkdir -p /var/lib/pixa /etc/pixa && \
chown pixad:pixad /var/lib/pixa chown pixad:pixad /var/lib/pixa
+2 -4
View File
@@ -58,10 +58,8 @@ What the [upaas](https://git.eeqj.de/sneak/upaas) app for pixa needs:
`healthy`. The probe uses the port from `PORT` (default `8080`), so a `healthy`. The probe uses the port from `PORT` (default `8080`), so a
port changed only in a mounted config file is not seen by it: change port changed only in a mounted config file is not seen by it: change
the port with `PORT`. the port with `PORT`.
- **First run:** create the host directory, owned by root or by uid - **First run:** create the host directory. It may be owned by root: the
`65532` and gid `65532`. The server runs as the container's `pixad` container gives it to its `pixad` user when it starts.
user, which has that uid and gid, and the container gives the
directory to `pixad` when it starts.
## Rationale ## Rationale
+6 -13
View File
@@ -30,11 +30,6 @@ exhaustion
# Completed Steps # Completed Steps
- 2026-09-29 fixed uid and gid for `pixad` (closes #151): the image creates the
`pixad` group with gid 65532 and the `pixad` user with uid 65532, instead of
the first free uid 1000, so a bind-mounted `/var/lib/pixa` given to `pixad`
is not owned on the host by a person's login account; the first-run step of
"Running under upaas" in `README.md` names the uid and gid.
- 2026-09-29 `max-age` never outlives an expiring URL (closes #63): both image - 2026-09-29 `max-age` never outlives an expiring URL (closes #63): both image
routes build `Cache-Control` from the request's `Expires`, which an encrypted routes build `Cache-Control` from the request's `Expires`, which an encrypted
URL's expiry now fills too; `max-age` is one year, or the whole seconds left URL's expiry now fills too; `max-age` is one year, or the whole seconds left
@@ -46,14 +41,12 @@ exhaustion
counts the cached source images and processed variants (`source_content` counts the cached source images and processed variants (`source_content`
plus `variant_content`) and takes their size from `Cache.UsageBytes`, plus `variant_content`) and takes their size from `Cache.UsageBytes`,
instead of reading `request_cache` and `output_content`, which nothing instead of reading `request_cache` and `output_content`, which nothing
writes; those two tables are left in the schema; a disabled disk cache writes; those two tables are left in the schema. A miss is counted after
reports no items and no size. A hit is counted even when the request it is served or fails, even when the request context has ended by then,
context has ended. A miss is counted after it is served or fails, also with the bytes it read from upstream, so `upstream_fetch_count` and
when the request context has ended by then, with the bytes it read from `upstream_fetch_bytes` move, including for an upstream body that fails
upstream, so `upstream_fetch_count` and `upstream_fetch_bytes` move, partway or a fetched source that then fails the magic byte check;
including for an upstream body that fails partway or a fetched source `transform_count` counts each image the image processor transcodes.
that then fails the magic byte check; `transform_count` counts each image
the image processor transcodes.
- 2026-09-28 strip metadata from processed images (closes #82): every output is - 2026-09-28 strip metadata from processed images (closes #82): every output is
exported with govips' `StripMetadata`, so it carries no EXIF, XMP, IPTC or ICC exported with govips' `StripMetadata`, so it carries no EXIF, XMP, IPTC or ICC
profile; the image is first turned upright with `AutoRotate` (before sizes are profile; the image is first turned upright with `AutoRotate` (before sizes are
+11 -14
View File
@@ -419,21 +419,18 @@ func (c *Cache) Stats(ctx context.Context) (*CacheStats, error) {
return nil, fmt.Errorf("failed to get cache stats: %w", err) return nil, fmt.Errorf("failed to get cache stats: %w", err)
} }
// Count and size the cached source images and processed variants. A // Count and size the cached source images and processed variants
// disabled cache holds none, whatever rows an earlier run left. err = c.db.QueryRowContext(ctx, `
if !c.disabled { SELECT (SELECT COUNT(*) FROM source_content)
err = c.db.QueryRowContext(ctx, ` + (SELECT COUNT(*) FROM variant_content)
SELECT (SELECT COUNT(*) FROM source_content) `).Scan(&stats.TotalItems)
+ (SELECT COUNT(*) FROM variant_content) if err != nil {
`).Scan(&stats.TotalItems) c.log.Warn("failed to count cache items for stats", "error", err)
if err != nil { }
c.log.Warn("failed to count cache items for stats", "error", err)
}
stats.TotalSizeBytes, err = c.UsageBytes(ctx) stats.TotalSizeBytes, err = c.UsageBytes(ctx)
if err != nil { if err != nil {
c.log.Warn("failed to sum cache size for stats", "error", err) c.log.Warn("failed to sum cache size for stats", "error", err)
}
} }
// Compute hit rate as a ratio // Compute hit rate as a ratio
+1 -2
View File
@@ -143,8 +143,7 @@ func (s *Service) Get(ctx context.Context, req *ImageRequest) (*ImageResponse, e
s.log.Error("failed to get cached variant", "key", result.CacheKey, "error", err) s.log.Error("failed to get cached variant", "key", result.CacheKey, "error", err)
// Fall through to re-process // Fall through to re-process
} else { } else {
// Counted also when the request context has ended meanwhile s.cache.IncrementStats(ctx, true, 0)
s.cache.IncrementStats(context.WithoutCancel(ctx), true, 0)
return &ImageResponse{ return &ImageResponse{
Content: reader, Content: reader,