Compare commits
5
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
47a2c2360a | ||
|
|
84a86274cd | ||
|
|
22bfc432b6 | ||
|
|
f743930f71 | ||
|
|
4e30dc7467 |
+18
-9
@@ -13,12 +13,21 @@
|
||||
# `/myapp`, never `**/myapp`, which also matches `cmd/myapp/` and
|
||||
# deletes the package directory from the context.
|
||||
|
||||
# Unlike the standard file, which leaves out all of .git, pixa sends
|
||||
# .git without its config. Without a VERSION build argument the stage
|
||||
# that compiles runs `git describe --tags --always` on .git, which does
|
||||
# not need .git/config; that file can hold a credential, such as a
|
||||
# .git is sent without its config. Without a VERSION build argument the
|
||||
# stage that compiles runs `git describe --tags --always` on .git, which
|
||||
# does not need .git/config; that file can hold a credential, such as a
|
||||
# password in a remote URL or the token the CI checkout step stores there.
|
||||
.git/config
|
||||
# Each submodule keeps a config with the same exposure in its git directory
|
||||
# under .git/modules/, nested again for a submodule's own submodules, or in
|
||||
# its own .git directory when it keeps one.
|
||||
# KNOWN GAP: a submodule whose name has a `config` segment (`config`,
|
||||
# `deploy/config`, `config/lib`) loses its whole git directory, because
|
||||
# `**/.git/modules/**/config` also matches that segment's directory
|
||||
# under .git/modules/. Go's version stamping then fails the build;
|
||||
# nothing leaks. Name such a submodule without that segment:
|
||||
# `git submodule add --name`.
|
||||
**/.git/config
|
||||
**/.git/modules/**/config
|
||||
|
||||
# Agent scratch: one full checkout of the repo per in-flight agent.
|
||||
# Anchored because it occurs once where agents run at the repo root.
|
||||
@@ -42,7 +51,9 @@
|
||||
**/[iI][dD]_[rR][sS][aA]
|
||||
**/[iI][dD]_[dD][sS][aA]
|
||||
**/[iI][dD]_[eE][cC][dD][sS][aA]
|
||||
**/[iI][dD]_[eE][cC][dD][sS][aA]_[sS][kK]
|
||||
**/[iI][dD]_[eE][dD]25519
|
||||
**/[iI][dD]_[eE][dD]25519_[sS][kK]
|
||||
|
||||
# Dependencies: restored inside the image, never copied in.
|
||||
**/node_modules
|
||||
@@ -60,10 +71,8 @@
|
||||
**/.vscode
|
||||
**/*.sublime-*
|
||||
|
||||
# pixa's own entries. Nothing in the build reads .gitignore. On the
|
||||
# host, `make build` writes bin/pixad, and the example config keeps its
|
||||
# state directory in data/.
|
||||
.gitignore
|
||||
# pixa's own entries. On the host, `make build` writes bin/pixad, and the
|
||||
# example config keeps its state directory in data/.
|
||||
/bin
|
||||
/data
|
||||
|
||||
|
||||
@@ -13,3 +13,6 @@ indent_style = tab
|
||||
|
||||
[*.go]
|
||||
indent_style = tab
|
||||
|
||||
# This repository's own sections, such as one for another language it
|
||||
# uses, go below this comment, and a re-vendor keeps them.
|
||||
|
||||
@@ -1,15 +1,21 @@
|
||||
name: check
|
||||
on: [push]
|
||||
# Free the shared runner: a new push cancels only the same branch's older run.
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
jobs:
|
||||
check:
|
||||
runs-on: ubuntu-latest
|
||||
# Free the shared runner from a hung build.
|
||||
timeout-minutes: 20
|
||||
steps:
|
||||
# actions/checkout v4.2.2, 2026-02-22
|
||||
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683
|
||||
# The default clone is shallow and has no tags, so the
|
||||
# version the build takes from `git describe` would be a
|
||||
# bare commit; this fetches the whole history with its tags.
|
||||
# script/cibuild needs no token, so none is left in .git/config.
|
||||
with:
|
||||
persist-credentials: false
|
||||
# All history and tags, so git describe finds the version tag.
|
||||
fetch-depth: 0
|
||||
- run: script/cibuild
|
||||
- run: script/docker-smoke
|
||||
|
||||
+37
-6
@@ -17,15 +17,46 @@ Thumbs.db
|
||||
# entry and must not be given a `**/` prefix on the way into one.
|
||||
.claude/
|
||||
|
||||
# Environment / secrets
|
||||
.env
|
||||
.env.*
|
||||
*.pem
|
||||
*.key
|
||||
# Node
|
||||
node_modules/
|
||||
|
||||
# Secrets. Unanchored like every entry above, so each matches at every
|
||||
# depth. Matching is case-sensitive on Linux, so names use character
|
||||
# ranges rather than a lowercase form that misses `Server.Key`.
|
||||
|
||||
# Environment files. `*.env` covers bare `.env` and the `prod.env`
|
||||
# convention. Only the templates `example.env` and `sample.env` are
|
||||
# re-included below. A repository that commits any other template adds
|
||||
# its own negation at the end of this file, for example `!.env.example`.
|
||||
*.[eE][nN][vV]
|
||||
.[eE][nN][vV].*
|
||||
.[eE][nN][vV][rR][cC]
|
||||
!example.env
|
||||
!sample.env
|
||||
|
||||
# Private keys and the bundles carrying them.
|
||||
*.[pP][eE][mM]
|
||||
*.[kK][eE][yY]
|
||||
*.[pP]12
|
||||
*.[pP][fF][xX]
|
||||
[iI][dD]_[rR][sS][aA]
|
||||
[iI][dD]_[dD][sS][aA]
|
||||
[iI][dD]_[eE][cC][dD][sS][aA]
|
||||
[iI][dD]_[eE][cC][dD][sS][aA]_[sS][kK]
|
||||
[iI][dD]_[eE][dD]25519
|
||||
[iI][dD]_[eE][dD]25519_[sS][kK]
|
||||
|
||||
# This repository's own entries, such as its build outputs, go below
|
||||
# this comment, and a re-vendor keeps them. Anchor a binary built at the
|
||||
# root: `/myapp`, never `myapp`, which also ignores `cmd/myapp/`.
|
||||
|
||||
# Go
|
||||
*.log
|
||||
*.out
|
||||
*.test
|
||||
|
||||
# Dependencies
|
||||
vendor/
|
||||
node_modules/
|
||||
|
||||
# Build output
|
||||
/bin/
|
||||
|
||||
@@ -17,6 +17,7 @@ linters:
|
||||
disable:
|
||||
# Genuinely incompatible with project patterns
|
||||
- exhaustruct # Requires all struct fields
|
||||
- exhaustruct_v5 # Requires all struct fields (successor to exhaustruct)
|
||||
- godot # Requires comments to end with periods
|
||||
- wrapcheck # Too verbose for internal packages
|
||||
- varnamelen # Short names like db, id are idiomatic Go
|
||||
@@ -24,6 +25,8 @@ linters:
|
||||
# silenced by disabling that name, not by enabling the successor.
|
||||
- wsl # Deprecated, replaced by wsl_v5
|
||||
- gomodguard # Deprecated, replaced by gomodguard_v2
|
||||
# Misses findings at random in v2.14.0; back once a pinned release fixes it
|
||||
- canonicalheader
|
||||
settings:
|
||||
lll:
|
||||
line-length: 88
|
||||
|
||||
@@ -1,7 +1,2 @@
|
||||
node_modules/
|
||||
yarn.lock
|
||||
|
||||
# A byte-for-byte copy of the one in sneak/prompts.
|
||||
REPO_POLICIES.md
|
||||
|
||||
vendor/
|
||||
|
||||
+48
-30
@@ -1,7 +1,12 @@
|
||||
# Every stage declares ARG CACHEBUST first: script/ passes a new value
|
||||
# on every build, which reruns every RUN step after it. The steps that
|
||||
# compile mount Go's build cache, which survives from build to build.
|
||||
|
||||
# Lint phase. script/lint builds it alone. The linter is run directly:
|
||||
# `make lint` and script/lint are themselves a docker build.
|
||||
# golangci/golangci-lint:v2.12.2, 2026-10-04
|
||||
FROM golangci/golangci-lint@sha256:5cceeef04e53efe1470638d4b4b4f5ceefd574955ab3941b2d9a68a8c9ad5240 AS lint
|
||||
# golangci/golangci-lint:v2.14.0, 2026-10-15
|
||||
FROM golangci/golangci-lint@sha256:ad862ba6b3798cbe0fd9fd7408d498fd74fbd2623a92406b2fd3898faf0bf98f AS lint
|
||||
ARG CACHEBUST
|
||||
|
||||
# The linter compiles every package, and govips needs the libvips
|
||||
# headers for that. REPO_POLICIES.md has the lint phase install them
|
||||
@@ -14,17 +19,20 @@ WORKDIR /src
|
||||
COPY go.mod go.sum ./
|
||||
RUN go mod download
|
||||
COPY . .
|
||||
RUN golangci-lint run --config .golangci.yml ./...
|
||||
RUN --mount=type=cache,target=/root/.cache/go-build \
|
||||
golangci-lint run --config .golangci.yml ./...
|
||||
|
||||
# Test phase. script/test builds it alone.
|
||||
# golang:1.25.4-alpine3.22, 2026-02-25; the runtime stage uses Alpine 3.22 too
|
||||
FROM golang:1.25.4-alpine3.22@sha256:d3f0cf7723f3429e3f9ed846243970b20a2de7bae6a5b66fc5914e228d831bbb AS test
|
||||
# Test phase. script/test builds it alone. -race needs cgo and so a C
|
||||
# compiler, which the Debian Go image ships and the alpine one does not.
|
||||
# golang:1.25.4-trixie, 2026-10-15
|
||||
FROM golang@sha256:a02d35efc036053fdf0da8c15919276bf777a80cbfda6a35c5e9f087e652adfc AS test
|
||||
ARG CACHEBUST
|
||||
|
||||
WORKDIR /src
|
||||
|
||||
# script/bootstrap --cgo installs the build dependencies (a C compiler,
|
||||
# the libvips and libheif headers, and libvips' JPEG XL support, which
|
||||
# the tests need) and downloads the Go modules.
|
||||
# script/bootstrap --cgo installs the build dependencies (the libvips
|
||||
# and libheif headers, and libvips' JPEG XL support, which the tests
|
||||
# need) and downloads the Go modules.
|
||||
COPY script/ ./script/
|
||||
COPY go.mod go.sum ./
|
||||
RUN script/bootstrap --cgo
|
||||
@@ -32,11 +40,14 @@ RUN script/bootstrap --cgo
|
||||
COPY . .
|
||||
|
||||
# Without -v first; on a failure, again with -v for the details, and
|
||||
# the step fails even if the second run passes. -parallel 4: by default
|
||||
# a package runs as many of its tests at once as the host has CPUs, and
|
||||
# on a busy host they then wait so long to be scheduled that a test
|
||||
# that times a request can see it return a second late.
|
||||
RUN go test -count=1 -timeout 90s -race -parallel 4 -cover ./... || \
|
||||
# the step fails even if the second run passes. -count=1 because the
|
||||
# mounted cache also holds test results, which go test would otherwise
|
||||
# replay. -parallel 4: by default a package runs as many of its tests at
|
||||
# once as the host has CPUs, and on a busy host they then wait so long
|
||||
# to be scheduled that a test that times a request can see it return a
|
||||
# second late.
|
||||
RUN --mount=type=cache,target=/root/.cache/go-build \
|
||||
go test -count=1 -timeout 90s -race -parallel 4 -cover ./... || \
|
||||
{ echo "--- Rerunning with -v for details ---"; \
|
||||
go test -count=1 -timeout 90s -race -parallel 4 -v ./...; exit 1; }
|
||||
|
||||
@@ -45,38 +56,44 @@ RUN go test -count=1 -timeout 90s -race -parallel 4 -cover ./... || \
|
||||
# test passed.
|
||||
# golang:1.25.4-alpine3.22, 2026-02-25; the runtime stage uses Alpine 3.22 too
|
||||
FROM golang:1.25.4-alpine3.22@sha256:d3f0cf7723f3429e3f9ed846243970b20a2de7bae6a5b66fc5914e228d831bbb AS builder
|
||||
ARG CACHEBUST
|
||||
|
||||
COPY --from=lint /src/go.sum /dev/null
|
||||
COPY --from=test /src/go.sum /dev/null
|
||||
|
||||
WORKDIR /src
|
||||
|
||||
# Build dependencies and Go modules, as in the test phase
|
||||
# Build dependencies, git among them, and Go modules, as in the test
|
||||
# phase
|
||||
COPY script/ ./script/
|
||||
COPY go.mod go.sum ./
|
||||
RUN script/bootstrap --cgo
|
||||
# A tar-stream context keeps the sender's file owners, which git refuses.
|
||||
RUN git config --system --add safe.directory /src
|
||||
|
||||
# Copy source code
|
||||
COPY . .
|
||||
|
||||
# VERSION is declared here, not earlier: a new value reruns only the
|
||||
# build, not script/bootstrap. Given none, the version is
|
||||
# `git describe --tags --always` of the .git in the build context (git
|
||||
# comes from script/bootstrap): the tag on a tagged commit, tag-N-gHASH
|
||||
# after one, the short commit when no tag is reachable. A context that
|
||||
# carries .git and still yields no version fails the build; one without
|
||||
# .git, as from a source tarball, stamps an empty version. CGO stays
|
||||
# enabled for govips; -trimpath keeps build paths out of the binary, and
|
||||
# -s -w leave out the symbol table and debug information.
|
||||
# The VERSION build arg when one is given, otherwise
|
||||
# `git describe --tags --always` on the .git in the build context: the
|
||||
# tag on a tagged commit, tag-N-gHASH after one, the short commit when
|
||||
# no tag is reachable. With .git present, a version that is still empty,
|
||||
# dev or unknown fails the build: git is missing or could not read the
|
||||
# checkout. A context without .git, as from a source tarball, stamps an
|
||||
# empty version. CGO stays enabled for govips; -trimpath keeps build
|
||||
# paths out of the binary, and -s -w leave out the symbol table and
|
||||
# debug information.
|
||||
ARG VERSION
|
||||
RUN version="${VERSION:-$(git describe --tags --always)}"; \
|
||||
if [ -e .git ] && { [ -z "$version" ] || [ "$version" = dev ] || \
|
||||
[ "$version" = unknown ]; }; then \
|
||||
echo "the build context carries .git but yields no version" >&2; \
|
||||
exit 1; \
|
||||
RUN --mount=type=cache,target=/root/.cache/go-build \
|
||||
VERSION="${VERSION:-$(git describe --tags --always)}"; \
|
||||
if [ -e .git ]; then \
|
||||
case "$VERSION" in ""|dev|unknown) \
|
||||
echo "version is '$VERSION' although .git is present" >&2; \
|
||||
exit 1 ;; \
|
||||
esac; \
|
||||
fi; \
|
||||
CGO_ENABLED=1 GOTOOLCHAIN=auto go build -trimpath \
|
||||
-ldflags "-s -w -X main.Version=${version}" \
|
||||
-ldflags="-s -w -X main.Version=${VERSION}" \
|
||||
-o /pixad ./cmd/pixad
|
||||
|
||||
# Runtime stage, and the last one: a plain `docker build .` builds this
|
||||
@@ -85,6 +102,7 @@ RUN version="${VERSION:-$(git describe --tags --always)}"; \
|
||||
# the libvips and musl it was built with.
|
||||
# alpine:3.22, 2026-10-08
|
||||
FROM alpine:3.22@sha256:5291449c3df73caf6ed85e649dec1b9e818b39a5d8c871e97afc13e9cd5e8fa8
|
||||
ARG CACHEBUST
|
||||
|
||||
# Install runtime dependencies only. vips-jxl is libvips' JPEG XL
|
||||
# support, without which pixad does not start.
|
||||
|
||||
@@ -131,6 +131,34 @@ What the [upaas](https://git.eeqj.de/sneak/upaas) app for pixa needs:
|
||||
the cache can grow to three quarters or more of the space they all have
|
||||
left, and to all of it when that is 500 MiB or less.
|
||||
- the rest are in the table under Configuration below
|
||||
- **Memory:** pixa can use up to `max_concurrent_processing` times 4.2 GiB for
|
||||
the images it processes at once, plus `upstream_connections` times
|
||||
`upstream_max_response_size` for the source images it has fetched that wait
|
||||
for processing. 4.2 GiB is the most resident memory was measured to grow while
|
||||
pixa, with pixad's libvips settings, processed one 8192x8192 source, the
|
||||
largest it accepts, saving it at full size to each output format in turn, one
|
||||
at a time. The sources were a JPEG, a GIF, a WebP, an 8-bit and a 16-bit PNG,
|
||||
an 8-bit and a 12-bit AVIF, and an 8-bit, a 16-bit and a 32-bit floating-point
|
||||
JPEG XL; memory grew most for the 16-bit JPEG XL, of random pixels, saved as
|
||||
WebP. At the defaults that is half the CPUs pixa sees times 4.2 GiB, plus 64
|
||||
times 50 MiB: on a host where pixa sees 48 CPUs, 24 times 4.2 GiB plus 3.1
|
||||
GiB, about 104 GiB. Set `PIXA_MAX_CONCURRENT_PROCESSING` and
|
||||
`PIXA_UPSTREAM_CONNECTIONS` so that this fits the memory the container may
|
||||
use.
|
||||
- **Temporary files:** while pixa processes a large source, libvips writes its
|
||||
decoded pixels to a file in `/tmp`. The file disappears from `/tmp` once it is
|
||||
written, so a listing of `/tmp` does not show it, but it keeps its space, on
|
||||
disk or, where `/tmp` is a tmpfs, in memory, until the image is done. For a
|
||||
GIF, WebP, AVIF or JPEG XL source the decoded pixels are held in memory as
|
||||
well as written to that file, and the 4.2 GiB above includes that memory. For
|
||||
an 8192x8192 source the file was measured on pixad at 192 MiB for an 8-bit RGB
|
||||
JPEG, 512 MiB for a 16-bit RGBA PNG or JPEG XL, and 1 GiB for a JPEG XL of
|
||||
32-bit floating-point RGBA, the largest measured: up to 1 GiB per image
|
||||
processed at once, 24 GiB at the defaults on a host where pixa sees 48 CPUs.
|
||||
This `/tmp` is in the container's own filesystem, not on the volume at
|
||||
`/var/lib/pixa`, and `cache_max_bytes` does not count it. The 4.2 GiB above
|
||||
does not include these files; where `/tmp` is a tmpfs, they are memory on top
|
||||
of it.
|
||||
- **Health check:** the image's `HEALTHCHECK` requests
|
||||
`/.well-known/healthcheck.json`. upaas reads the container's health 60 seconds
|
||||
after a deploy and marks the deploy failed unless it is `healthy`. The probe
|
||||
@@ -288,12 +316,32 @@ once, is refused with 400.
|
||||
both optional (values under Signature Specification). Both are part of what is
|
||||
cached, so each value of either is a separate cached image.
|
||||
|
||||
A source URL's query follows its path after `%3F`, the encoded `?`:
|
||||
`/v1/image/<host>/<path>%3F<query>/<size>.<format>`. pixa decodes the
|
||||
percent-escapes in the image URL's path once, then reads the host up to the
|
||||
first `/`, the size and format after the last `/`, and between them the source
|
||||
path up to the first `?` and the source query after it. So the `?` before the
|
||||
query, any `?` in the query, and every `%` in the source path or query must be
|
||||
percent-encoded, as `%3F` and `%25`. Any other character may be written as
|
||||
itself or percent-encoded: `/`, `;`, `,`, `=` and `&` work either way. For
|
||||
example,
|
||||
`https://cdn.example.com/img?url=https://origin.example.com/my%20cat.jpg` at
|
||||
800x600 WebP is
|
||||
`/v1/image/cdn.example.com/img%3Furl=https://origin.example.com/my%2520cat.jpg/800x600.webp`,
|
||||
signed as shown under Signature Specification.
|
||||
|
||||
A size with one side `0` has that side scaled to keep the source's proportions,
|
||||
and is refused with 400 when that side would come out above 8192, the most
|
||||
either side may be; so is a size with `fit=outside`, which scales the image to
|
||||
cover the size and so past it on one side, when that side would come out
|
||||
above 8192.
|
||||
|
||||
An AVIF image of more than 16,777,216 pixels, the number 4096x4096 and 8192x2048
|
||||
have, is refused with 400 before anything is decoded, as its encode could take
|
||||
more than 40 seconds (see below). That includes `orig` of an AVIF source, and
|
||||
`auto` when it chooses AVIF, so a client whose `Accept` names `image/avif` but
|
||||
not `image/jxl` is refused such a size.
|
||||
|
||||
The source image may be JPEG, PNG, GIF, WebP, AVIF or JPEG XL. The upstream's
|
||||
`Content-Type` must name its format, and the image's first bytes must match it.
|
||||
An upstream answering with any other `Content-Type`, SVG included, is refused
|
||||
@@ -343,14 +391,18 @@ transcode that stops this way is answered 503 with the error
|
||||
ended. govips gives pixa no way to stop libvips once an encode has begun, so an
|
||||
encode keeps its slot until it finishes, which can be after
|
||||
`downstream_timeout`. The limit of 8192 on each side of the source and of the
|
||||
output bounds how long that takes. With one libvips thread, as pixad runs it,
|
||||
encoding 8192x8192 random pixels takes up to about 1 second as JPEG, 15 as PNG,
|
||||
16 as GIF, 63 as WebP, 110 as AVIF and 96 as JPEG XL, the slowest of 8 or 16
|
||||
bits per sample, with or without an alpha channel. WebP, AVIF and JPEG XL are
|
||||
that slow only with an alpha channel: 8-bit RGB takes about 44, 53 and 28
|
||||
seconds. Decoding such a source adds up to about 11 seconds. A WebP, AVIF or
|
||||
JPEG XL encode can therefore hold its slot past the default `downstream_timeout`
|
||||
of 60 seconds.
|
||||
output, and for AVIF the limit of 16,777,216 pixels, bound how long that takes.
|
||||
With one libvips thread, as pixad runs it, encoding 8192x8192 random pixels
|
||||
takes up to about 1 second as JPEG, 15 as PNG and 16 as GIF, the slowest of 8 or
|
||||
16 bits per sample, with or without an alpha channel. The WebP and JPEG XL
|
||||
encoders run below libvips' default effort, and AVIF at the lowest effort govips
|
||||
can set, so that no encode takes more than 40 seconds of the default
|
||||
`downstream_timeout` of 60: at any quality and 8 or 16 bits per sample,
|
||||
8192x8192 random pixels with an alpha channel take up to about 23 seconds as
|
||||
WebP and 27 as JPEG XL, and 4096x4096, 8192x2048 or 2048x8192 up to about 34 as
|
||||
AVIF, which at quality 85 would take about 96 at 8192x8192. Without an alpha
|
||||
channel they take up to about 16, 13 and 22 seconds. Decoding such a source adds
|
||||
up to about 11 seconds.
|
||||
|
||||
The login form (`POST /`) is limited to 5 attempts per minute per client
|
||||
address, counting an IPv6 client by its /64; an attempt over the limit is
|
||||
@@ -402,7 +454,8 @@ nor change what it asks for.
|
||||
sent with an empty format, or none, also makes a JPEG XL URL. Width and
|
||||
height both empty or `0` keep the original size; if only one of them is empty
|
||||
or `0`, that side is scaled to keep the image's proportions. A URL whose
|
||||
scaled side would come out above 8192 answers 400.
|
||||
scaled side would come out above 8192 answers 400, and so does an AVIF URL of
|
||||
more than 16,777,216 pixels (see Routes).
|
||||
3. The page shows the URL, `https://<host>/v1/e/<token>/img.<format>`, and when
|
||||
it expires. `<host>` is the host the page was opened on, and the URL starts
|
||||
with `http` instead while `debug` is on. The name after the token is ignored
|
||||
@@ -507,6 +560,17 @@ For the same image at quality 40 with fit `contain`, the input ends in
|
||||
and the URL is
|
||||
`/v1/image/cdn.example.com/photos/cat.jpg/800x600.webp?sig=5IwXUx6vf7yefhaUvFzgXZvG2o0Df4RJxPTK3pKq5VU=&exp=1704067200&q=40&fit=contain`.
|
||||
|
||||
For a source URL with a query,
|
||||
`https://cdn.example.com/img?url=https://origin.example.com/my%20cat.jpg`, at
|
||||
800x600 WebP with the same key and expiration and the default quality and fit:
|
||||
|
||||
1. Build input, with the query's `:` written as `%3A` and its `%` as `%25`:
|
||||
`cdn.example.com:/img:url=https%3A//origin.example.com/my%2520cat.jpg:800:600:webp:1704067200:85:cover`
|
||||
2. The signature is `37aFa9GSVCZM1kRY9SzMOk2fOg_M7Wt5gKNR4xM-FYA=`
|
||||
3. URL, with the `?` before the query written as `%3F` and the query's `%` as
|
||||
`%25` (see Routes):
|
||||
`/v1/image/cdn.example.com/img%3Furl=https://origin.example.com/my%2520cat.jpg/800x600.webp?sig=37aFa9GSVCZM1kRY9SzMOk2fOg_M7Wt5gKNR4xM-FYA=&exp=1704067200`
|
||||
|
||||
**Allowlist patterns:**
|
||||
|
||||
- **Exact match**: `cdn.example.com` — matches only that host
|
||||
@@ -566,7 +630,7 @@ tell. With no file, pixa uses the environment and the defaults.
|
||||
| `PIXA_ALLOW_HTTP` | `allow_http` | Allow plain-HTTP upstreams, for testing only; default `false` |
|
||||
| `PIXA_UPSTREAM_CONNECTIONS_PER_HOST` | `upstream_connections_per_host` | Concurrent connections per upstream host; default `20` |
|
||||
| `PIXA_UPSTREAM_CONNECTIONS` | `upstream_connections` | Concurrent connections to all upstream hosts together; default `64` |
|
||||
| `PIXA_MAX_CONCURRENT_PROCESSING` | `max_concurrent_processing` | Images processed at once; default the number of CPUs |
|
||||
| `PIXA_MAX_CONCURRENT_PROCESSING` | `max_concurrent_processing` | Images processed at once; default half the CPUs, at least 1 |
|
||||
| `PIXA_IMAGE_REQUESTS_PER_MINUTE` | `image_requests_per_minute` | Image requests per client per minute; `0` turns it off; default `600` |
|
||||
| `PIXA_UPSTREAM_FETCH_TIMEOUT` | `upstream_fetch_timeout` | Time allowed for one fetch from an upstream host; default `30s` |
|
||||
| `PIXA_UPSTREAM_MAX_RESPONSE_SIZE` | `upstream_max_response_size` | Largest upstream response accepted, in bytes; default 50 MiB |
|
||||
@@ -652,10 +716,12 @@ Key settings in more detail:
|
||||
does, and `downstream_timeout` has not ended first, the request is answered
|
||||
503 with the error `server busy, try again later`
|
||||
- `max_concurrent_processing` — the most images decoded and encoded at once;
|
||||
default the number of CPUs pixa can use (`GOMAXPROCS`), which follows a
|
||||
container's CPU limit. A request that finds all of them in use waits up to 10
|
||||
seconds for one to free up; if none does, and `downstream_timeout` has not
|
||||
ended first, it is answered 503 the same way
|
||||
default half the CPUs pixa can use (`GOMAXPROCS`, which follows a container's
|
||||
CPU limit), rounded down, and at least 1. A value that is not a whole number
|
||||
of at least 1 aborts startup. A request that finds all of them in use waits up
|
||||
to 10 seconds for one to free up; if none does, and `downstream_timeout` has
|
||||
not ended first, it is answered 503 the same way. Each one can take a lot of
|
||||
memory: see "Running under upaas"
|
||||
- `image_requests_per_minute` — how many requests one client may send to the
|
||||
image routes per minute (see the limit on them under Routes); default `600`.
|
||||
`0` turns the limit off, and a negative or non-integer value aborts startup.
|
||||
@@ -706,9 +772,9 @@ provide:
|
||||
install-precommit)
|
||||
- `script/projectname` — output the project name ("pixa")
|
||||
- `script/test` — run the test suite: build the `test` phase of the
|
||||
`Dockerfile`, tagged `pixa-test`
|
||||
`Dockerfile`, writing no image
|
||||
- `script/lint` — run golangci-lint: build the `lint` phase of the `Dockerfile`,
|
||||
tagged `pixa-lint`; the linter never runs on the host
|
||||
writing no image; the linter never runs on the host
|
||||
- `script/fmt` — format the Go code with gofmt and the markdown with prettier
|
||||
(writes)
|
||||
- `script/fmt-check` — check the same formatting (read-only), on the host
|
||||
@@ -726,15 +792,20 @@ provide:
|
||||
- `script/install-precommit` — install the git pre-commit hook that runs
|
||||
`script/precommit`
|
||||
|
||||
Every `docker build` in these scripts passes `--no-cache`, so the lint and test
|
||||
phases run on every build instead of coming from the build cache.
|
||||
Every `docker build` in these scripts passes a new value of the `CACHEBUST`
|
||||
build argument, which every stage of the `Dockerfile` declares first, so every
|
||||
`RUN` step, the lint and test phases included, runs on every build instead of
|
||||
coming from the build cache. Go's build cache is kept from build to build: the
|
||||
steps that compile mount it, so a package whose inputs are unchanged is not
|
||||
compiled again, and both `go test` runs pass `-count=1` so that no stored test
|
||||
result is replayed. The `test` phase runs on the Debian Go image, with Debian's
|
||||
libvips and libheif; the image pixa ships is built and runs on Alpine.
|
||||
`script/check`, `script/cibuild`, `script/docker`, `script/lint`, `script/test`,
|
||||
`script/setup` and `script/install-precommit` are the standard copies from
|
||||
`sneak/prompts`, kept identical to them. `script/fmt` and `script/fmt-check` are
|
||||
the standard copies with pixa's `gofmt` step kept before prettier. prettier
|
||||
formats the markdown only: not the HTML templates, as it cannot parse a Go
|
||||
template action inside a tag, and not `REPO_POLICIES.md` (see
|
||||
`.prettierignore`), a copy of the one in `sneak/prompts`.
|
||||
formats the markdown only, not the HTML templates, as it cannot parse a Go
|
||||
template action inside a tag.
|
||||
|
||||
## Load Test
|
||||
|
||||
|
||||
+258
-105
@@ -1,6 +1,6 @@
|
||||
---
|
||||
title: Repository Policies
|
||||
last_modified: 2026-09-08
|
||||
last_modified: 2026-10-15
|
||||
---
|
||||
|
||||
This document covers repository structure, tooling, and workflow standards. Code
|
||||
@@ -54,34 +54,39 @@ style conventions are in separate documents:
|
||||
`cibuild`. `script/bootstrap` installs all dependencies idempotently and
|
||||
assumes nothing is present: base tools come from nix, apt, brew, or apk
|
||||
(detected in that order; apt runs noninteractive). For node it uses the
|
||||
installed node if present; otherwise it installs a PINNED node version via
|
||||
nvm, first installing nvm itself if missing — from a hash-verified GitHub
|
||||
release archive (never `curl | sh`), with bash installed as an explicit
|
||||
prerequisite since nvm requires bash. yarn is then pinned via
|
||||
`corepack prepare yarn@<version> --activate`. Never install "latest" or "lts";
|
||||
always exact versions. `script/cibuild` runs the CI build: it changes to the
|
||||
repo root, runs `script/bootstrap`, runs `script/check`, and builds the image
|
||||
with the version; the Gitea workflow calls it. **`script/cibuild` runs
|
||||
installed node only when its major version is the pinned one; otherwise (node
|
||||
missing, or another major, such as the node 24 the CI runner image ships) it
|
||||
installs the PINNED node version via nvm and installs yarn and the packages
|
||||
under it. nvm itself is installed first if missing, from a hash-verified
|
||||
GitHub release archive (never `curl | sh`), with bash installed as an explicit
|
||||
prerequisite since nvm requires bash. Only the major version is compared
|
||||
because the `Dockerfile` stages start from a node image whose exact version is
|
||||
not the pin, and nvm cannot install a prebuilt node on alpine. yarn is pinned
|
||||
via `corepack prepare yarn@<version> --activate`. Never install "latest" or
|
||||
"lts"; always exact versions. `script/cibuild` runs the CI build: it changes
|
||||
to the repo root, runs `script/bootstrap`, runs `script/check`, and builds the
|
||||
image with the version; the Gitea workflow calls it. **`script/cibuild` runs
|
||||
`script/bootstrap` first**, because the workflow checks out the repo and runs
|
||||
nothing else, while `script/fmt-check` runs the formatter on the host: on a
|
||||
pristine checkout with nothing installed the run dies there, after the
|
||||
containerised gates have passed. **The bootstrap alone is not enough**:
|
||||
`script/bootstrap` installs node and yarn under nvm and leaves neither on the
|
||||
`PATH` of the shell that called it, so a bare `yarn` still exits 127. The host
|
||||
entrypoints that need yarn — `script/fmt` and `script/fmt-check` — therefore
|
||||
source nvm for the pinned node version before invoking it, exactly as
|
||||
`script/bootstrap`'s own install step does. A runner carrying nothing but
|
||||
docker and git then gets through `script/check`. Four further scripts are our
|
||||
own extensions to the standard: `script/check` runs `script/test`,
|
||||
`script/lint` and `script/fmt-check`; `script/precommit` is what the git
|
||||
pre-commit hook runs, and it calls `script/check`; `script/install-precommit`
|
||||
installs the git pre-commit hook (the `make hooks` target shims to it); and
|
||||
`script/projectname` (literally that filename) simply outputs the project's
|
||||
name. Scripts that need the name call `script/projectname` — e.g.
|
||||
`script/docker` assembles its image tag from it — so those scripts stay
|
||||
byte-identical across all repos. Repo-type-specific pre-commit extras (e.g.
|
||||
`go mod tidy` verification in Go repos) belong in `script/precommit`, not in
|
||||
the hook itself. Model scripts are at
|
||||
containerised gates have passed. **The bootstrap alone is not enough**: when
|
||||
`script/bootstrap` installs node and yarn under nvm it leaves neither on the
|
||||
`PATH` of the shell that called it, so a bare `yarn` either exits 127 or runs
|
||||
under another node. The host entrypoints that need yarn — `script/fmt` and
|
||||
`script/fmt-check` — therefore use the same test as `script/bootstrap`: when
|
||||
the node on `PATH` has the pinned major version they run the `yarn` on `PATH`,
|
||||
and otherwise they load nvm through `$HOME/.nvm/nvm.sh` and run yarn under the
|
||||
pinned node. A runner carrying nothing but docker and git then gets through
|
||||
`script/check`. Four further scripts are our own extensions to the standard:
|
||||
`script/check` runs `script/test`, `script/lint` and `script/fmt-check`;
|
||||
`script/precommit` is what the git pre-commit hook runs, and it calls
|
||||
`script/check`; `script/install-precommit` installs the git pre-commit hook
|
||||
(the `make hooks` target shims to it); and `script/projectname` (literally
|
||||
that filename) simply outputs the project's name. Scripts that need the name
|
||||
call `script/projectname` — e.g. `script/docker` assembles its image tag from
|
||||
it — so those scripts stay byte-identical across all repos. Repo-type-specific
|
||||
pre-commit extras (e.g. `go mod tidy` verification in Go repos) belong in
|
||||
`script/precommit`, not in the hook itself. Model scripts are at
|
||||
`https://git.eeqj.de/sneak/prompts/raw/branch/main/script/<name>`. The README
|
||||
must document the provided scripts in an **Entrypoints** section (see the
|
||||
README requirements below).
|
||||
@@ -104,18 +109,24 @@ style conventions are in separate documents:
|
||||
`lint` phase and a `test` phase, with the final stage depending on both so the
|
||||
image cannot be built unless they pass. For non-server repos the final stage
|
||||
brings up a development environment; for server repos it is the runtime image.
|
||||
Dockerfiles install development prerequisites by running `script/bootstrap`
|
||||
rather than duplicating installs inline; COPY `script/` and the dependency
|
||||
manifests (`package.json` + `yarn.lock`, `go.mod` + `go.sum`, etc.) before
|
||||
running it.
|
||||
The gate phases and the build stage start from their pinned base images and
|
||||
install what those images lack either inline, as the canonical Go `Dockerfile`
|
||||
below does for `git`, or by running `script/bootstrap`, as the `prompts`
|
||||
repo's own `Dockerfile` does for its yarn packages. The development
|
||||
environment stage installs development prerequisites by running
|
||||
`script/bootstrap` rather than duplicating its installs inline. A stage that
|
||||
runs `script/bootstrap` COPYs `script/` and the dependency manifests
|
||||
(`package.json` + `yarn.lock`, `go.mod` + `go.sum`, etc.) before running it.
|
||||
|
||||
- **Linting and testing run in Docker, as phases of the `Dockerfile`.** There is
|
||||
no separate lint file. `script/lint` and `script/test` each build one phase
|
||||
and nothing else:
|
||||
|
||||
```sh
|
||||
docker build --no-cache --target lint -t "$(script/projectname)-lint" .
|
||||
docker build --no-cache --target test -t "$(script/projectname)-test" .
|
||||
docker build --build-arg CACHEBUST="$(date +%s)-$$" \
|
||||
--target lint --output type=cacheonly .
|
||||
docker build --build-arg CACHEBUST="$(date +%s)-$$" \
|
||||
--target test --output type=cacheonly .
|
||||
```
|
||||
|
||||
**A stage that is not the last one in the file is built only when the final
|
||||
@@ -125,10 +136,15 @@ style conventions are in separate documents:
|
||||
plain `docker build .` builds the last stage alone and exits 0 having linted
|
||||
and tested nothing.
|
||||
|
||||
**Every `docker build` in `script/` is tagged**, here and in
|
||||
`script/cibuild` and `script/docker`. An untagged build leaves a dangling
|
||||
image behind on every invocation, on every developer host and every CI
|
||||
runner; a tagged one replaces the previous image.
|
||||
**The gate builds write no image.** With `--output type=cacheonly` the phase
|
||||
runs and a failing step fails the build, but the result is not exported.
|
||||
Nothing uses those images, and writing one out is slow: a Go test phase's
|
||||
image holds the toolchain. A build given neither `--output` nor `-t` writes
|
||||
an untagged image and leaves it dangling, on every developer host and every
|
||||
CI runner. `script/cibuild` and `script/docker` build the image that ships
|
||||
and tag it, so each build replaces the previous image; each assigns the tag
|
||||
on its own line before the build, so `set -e` stops it where
|
||||
`script/projectname` fails.
|
||||
|
||||
Inside a phase the tool is invoked directly — `golangci-lint`, `go test`,
|
||||
`eslint`, `prettier` — never through `make lint` or `script/test`, which are
|
||||
@@ -143,66 +159,107 @@ style conventions are in separate documents:
|
||||
one's findings, and a host-global lock in `$TMPDIR` makes concurrent runs
|
||||
exit non-zero with `parallel golangci-lint is running` — a status a caller
|
||||
cannot tell from real findings. Both have produced wrong verdicts in this
|
||||
org, in both directions. A container has its own cache, its own `TMPDIR` and
|
||||
a digest-pinned binary, so neither is reachable.
|
||||
org, in both directions. A container has its own golangci-lint cache, its
|
||||
own `TMPDIR` and a digest-pinned binary, so neither is reachable.
|
||||
|
||||
- **Any build that runs checks is built with `--no-cache`.** Docker invalidates
|
||||
a `COPY` layer only when the copied content changes, so on an unchanged tree
|
||||
the check `RUN` is served from cache, nothing executes, and the build still
|
||||
exits 0. Every `docker build` in `script/` therefore passes `--no-cache`:
|
||||
`script/lint`, `script/test`, `script/cibuild` and `script/docker` are the
|
||||
four, and there is no fifth — `script/check` runs the two gate phases and
|
||||
`script/fmt-check`, and builds no image of its own. A bare `docker build .` is
|
||||
not evidence that anything ran: a sub-second build reporting success is a
|
||||
cache hit, not a result. Never invalidate by pruning — `docker builder prune`
|
||||
and friends destroy a build cache shared with every other build on the host.
|
||||
- **Any build that runs checks is given a new `CACHEBUST` value.** Docker
|
||||
invalidates a `COPY` layer only when the copied content changes, so on an
|
||||
unchanged tree the check `RUN` is served from cache, nothing executes, and the
|
||||
build still exits 0. Every `docker build` in `script/` therefore passes
|
||||
`--build-arg CACHEBUST="$(date +%s)-$$"`, the time and the script's process
|
||||
id, so no two builds share a value, not even the three in one
|
||||
`script/cibuild`. `script/lint`, `script/test`, `script/cibuild` and
|
||||
`script/docker` are the four, and there is no fifth — `script/check` runs the
|
||||
two gate phases and `script/fmt-check`, and builds no image of its own. Every
|
||||
stage of the `Dockerfile` declares `ARG CACHEBUST` as its first line after
|
||||
`FROM`, and a new value reruns every `RUN` step after that line; `COPY` steps
|
||||
are compared by content and need nothing. A stage without the `ARG` line is
|
||||
served from cache on an unchanged tree, so a repository re-vendors the scripts
|
||||
and adds the `ARG` lines in one commit. The scripts do not pass `--no-cache`:
|
||||
it reruns every step too, but it also empties every cache mount, and Go's
|
||||
build cache, which the canonical Go `Dockerfile` below mounts so that compiled
|
||||
packages survive between check builds, would start empty every time. A bare
|
||||
`docker build .` is not evidence that anything ran: a sub-second build
|
||||
reporting success is a cache hit, not a result. Never invalidate by pruning —
|
||||
`docker builder prune` and friends destroy a build cache shared with every
|
||||
other build on the host. When a check is added or changed, prove it works by
|
||||
planting a defect it must catch and watching the run fail on it, then revert
|
||||
the defect. A green run alone shows neither that the check ran nor that it
|
||||
covers what it should.
|
||||
|
||||
- **The gate phases are separate stages, and the build stage depends on both.**
|
||||
The lint phase is based on the `golangci/golangci-lint` image (pinned by
|
||||
hash), so lint failures surface in seconds rather than after a full compile,
|
||||
and the test phase is based on the Go image. The canonical Go repo
|
||||
and the test phase is based on the Debian Go image. The canonical Go repo
|
||||
`Dockerfile`:
|
||||
|
||||
```dockerfile
|
||||
# Every stage declares ARG CACHEBUST first: script/ passes a new value
|
||||
# on every build, which reruns every RUN step after it. The steps that
|
||||
# compile mount Go's build cache, which survives from build to build.
|
||||
|
||||
# Lint phase
|
||||
# golangci/golangci-lint:v2.x.x, YYYY-MM-DD
|
||||
FROM golangci/golangci-lint@sha256:... AS lint
|
||||
ARG CACHEBUST
|
||||
WORKDIR /src
|
||||
COPY go.mod go.sum ./
|
||||
RUN go mod download
|
||||
COPY . .
|
||||
RUN golangci-lint run --config .golangci.yml ./...
|
||||
RUN --mount=type=cache,target=/root/.cache/go-build \
|
||||
golangci-lint run --config .golangci.yml ./...
|
||||
|
||||
# Test phase
|
||||
# golang:1.x-alpine, YYYY-MM-DD
|
||||
# Test phase. -race needs cgo and so a C compiler, which the Debian Go
|
||||
# image ships and the alpine one does not. -count=1 because the mounted
|
||||
# cache also holds test results, which go test would otherwise replay.
|
||||
# golang:1.x, YYYY-MM-DD
|
||||
FROM golang@sha256:... AS test
|
||||
ARG CACHEBUST
|
||||
WORKDIR /src
|
||||
COPY go.mod go.sum ./
|
||||
RUN go mod download
|
||||
COPY . .
|
||||
RUN go test -timeout 90s -race -cover ./... || \
|
||||
RUN --mount=type=cache,target=/root/.cache/go-build \
|
||||
go test -count=1 -timeout 90s -race -cover ./... || \
|
||||
{ echo "--- Rerunning with -v for details ---"; \
|
||||
go test -timeout 90s -race -v ./...; exit 1; }
|
||||
go test -count=1 -timeout 90s -race -v ./...; exit 1; }
|
||||
|
||||
# Build stage. Nothing is wanted from either phase above; the copies
|
||||
# are what make BuildKit build them first, so this stage cannot run
|
||||
# unless lint and test passed.
|
||||
# golang:1.x-alpine, YYYY-MM-DD
|
||||
FROM golang@sha256:... AS builder
|
||||
ARG CACHEBUST
|
||||
COPY --from=lint /src/go.sum /dev/null
|
||||
COPY --from=test /src/go.sum /dev/null
|
||||
RUN apk add --no-cache git
|
||||
# A tar-stream context keeps the sender's file owners, which git refuses.
|
||||
RUN git config --system --add safe.directory /src
|
||||
WORKDIR /src
|
||||
COPY go.mod go.sum ./
|
||||
RUN go mod download
|
||||
COPY . .
|
||||
|
||||
ARG VERSION=dev
|
||||
RUN CGO_ENABLED=0 go build -trimpath \
|
||||
-ldflags="-s -w -X main.Version=${VERSION}" \
|
||||
-o /app ./cmd/app/
|
||||
# The VERSION build arg when one is given, otherwise
|
||||
# `git describe --tags --always` on the .git in the build context. With
|
||||
# .git present, a version that is still empty, dev or unknown fails the
|
||||
# build: git is missing or could not read the checkout.
|
||||
ARG VERSION
|
||||
RUN --mount=type=cache,target=/root/.cache/go-build \
|
||||
VERSION="${VERSION:-$(git describe --tags --always)}"; \
|
||||
if [ -e .git ]; then \
|
||||
case "$VERSION" in ""|dev|unknown) \
|
||||
echo "version is '$VERSION' although .git is present" >&2; \
|
||||
exit 1 ;; \
|
||||
esac; \
|
||||
fi; \
|
||||
CGO_ENABLED=0 go build -trimpath \
|
||||
-ldflags="-s -w -X main.Version=${VERSION}" \
|
||||
-o /app ./cmd/app/
|
||||
|
||||
# Runtime stage, and the last one
|
||||
FROM alpine@sha256:...
|
||||
ARG CACHEBUST
|
||||
COPY --from=builder /app /usr/local/bin/app
|
||||
ENTRYPOINT ["app"]
|
||||
```
|
||||
@@ -217,23 +274,88 @@ style conventions are in separate documents:
|
||||
- Keep the runtime stage last, and if you add a stage after it, give it the
|
||||
same two copies. A plain `docker build .` builds the last stage's chain
|
||||
and nothing else.
|
||||
- Only Go's build cache is mounted (`/root/.cache/go-build`, where Go keeps
|
||||
it for root), and only on the steps that compile: `golangci-lint run`,
|
||||
both `go test` runs and `go build`. It keeps compiled packages between
|
||||
check builds, and Go reuses one only when all its inputs are unchanged, so
|
||||
it saves the compile without skipping a check. Nothing else is kept:
|
||||
golangci-lint's own cache would replay stored findings in place of a lint
|
||||
run, and Go builds from whatever files a kept module cache holds without
|
||||
checking them again (only `go mod verify` does), so a file changed there
|
||||
would go unnoticed. Go's build cache also holds test results, which is why
|
||||
both `go test` runs carry `-count=1`.
|
||||
- If the project uses `//go:embed` directives that reference build artifacts
|
||||
(e.g. a web frontend compiled in a separate stage), the lint phase must
|
||||
create placeholder files so the embed directives resolve. Example:
|
||||
`RUN mkdir -p web/dist && touch web/dist/index.html web/dist/style.css`.
|
||||
- If the project requires CGO or system libraries for linting (e.g.
|
||||
`vips-dev`), install them in the lint phase with `apk add`.
|
||||
- `ARG VERSION=dev` is declared in the stage that compiles and supplied by
|
||||
`script/docker` and `script/cibuild`; no stage may call `git describe`.
|
||||
- If the project requires CGO or system libraries for linting, install them
|
||||
in the lint phase. The `golangci/golangci-lint` image is Debian-based and
|
||||
has no `apk`, so install with `apt-get` under the Debian package name
|
||||
(`libvips-dev`, where alpine says `vips-dev`), and delete the package
|
||||
lists in the same `RUN`, so the layer does not keep them:
|
||||
|
||||
```dockerfile
|
||||
RUN apt-get update \
|
||||
&& apt-get install -y --no-install-recommends libvips-dev \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
```
|
||||
|
||||
- `.dockerignore` lets `.git` into the build context. It keeps out every git
|
||||
`config` at any depth (`**/.git/config`, `**/.git/modules/**/config`): the
|
||||
repository's own, each submodule's under `.git/modules/`, and that of a
|
||||
submodule keeping its own `.git` directory. `git describe` does not need
|
||||
them, and each can hold a credential: a password in a remote URL, or the
|
||||
token the CI checkout step stores there. A submodule whose name has a
|
||||
`config` segment (`config`, `deploy/config`, `config/lib`) loses its whole
|
||||
git directory to `**/.git/modules/**/config`, and Go's version stamping
|
||||
then fails the build: give it a name without that segment
|
||||
(`git submodule add --name`). The stage that compiles has `git` (the
|
||||
Debian Go image has it; an alpine one needs `apk add --no-cache git`) and
|
||||
takes the version from the `VERSION` build argument when one is given,
|
||||
otherwise from `git describe --tags --always`. That gives the tag on a
|
||||
tagged commit; on a later commit, the tag, the number of commits since it
|
||||
and the short commit (`v1.2.3-4-gabc1234`); and the short commit when no
|
||||
tag is reachable. The stage that compiles also marks its working directory
|
||||
safe for git (`git config --system --add safe.directory /src`): a context
|
||||
sent as a tar stream keeps the sender's file owners, and git refuses a
|
||||
checkout owned by another user, so the version would come out empty.
|
||||
`ARG VERSION` has no default, and the build fails if the context carries
|
||||
`.git` and the version still comes out empty, `dev` or `unknown`. A plain
|
||||
`docker build .` with no build arguments must succeed; a Dockerfile that
|
||||
refuses an empty build argument drops that refusal and keeps the argument.
|
||||
A checkout whose `.git` is a file (a linked worktree, or a repository
|
||||
checked out as a submodule) is the exception: that file points to a git
|
||||
directory outside the build context, so the build cannot read the version
|
||||
and a plain `docker build .` fails; pass the version with
|
||||
`--build-arg VERSION=...`, as `script/docker` and `script/cibuild` already
|
||||
do.
|
||||
|
||||
- Every repo should have a Gitea Actions workflow (`.gitea/workflows/`) that
|
||||
runs `script/cibuild` on push, and checks out the repo as its only other step.
|
||||
That script bootstraps, runs the gate phases, and then builds the image, so a
|
||||
successful run means every check passed; a bare `docker build .` does not
|
||||
runs `script/cibuild` on push, and checks out the repo as its only other step,
|
||||
with `persist-credentials: false`: `script/cibuild` needs no token, and
|
||||
without it the checkout leaves the job's token in `.git/config` for every
|
||||
later step. The checkout step also sets `fetch-depth: 0`, which fetches the
|
||||
tags `git describe` needs: by default it clones shallow with no tags, and a
|
||||
tagged repository's CI build would stamp a bare short commit id. The
|
||||
workflow's `concurrency` block groups runs by workflow and branch
|
||||
(`${{ github.workflow }}-${{ github.ref }}`) with `cancel-in-progress: true`,
|
||||
so a new push cancels the older run on the same branch, queued or running, and
|
||||
no other: runs for replaced commits do not hold up the shared runner.
|
||||
`script/cibuild` bootstraps, runs the gate phases, and then builds the image,
|
||||
so a successful run means every check passed; a bare `docker build .` does not
|
||||
carry the same guarantee, because its gate phases may come from the cache. The
|
||||
image build is uncached and so runs the gate phases a second time. That is the
|
||||
price of the rule above, and it is worth paying: the image that ships is built
|
||||
from a run of its own gates rather than from a cache entry.
|
||||
image build gets a `CACHEBUST` value of its own and so runs the gate phases a
|
||||
second time. That is the price of the rule above, and it is worth paying: the
|
||||
image that ships is built from a run of its own gates rather than from a cache
|
||||
entry. The `check` job sets `timeout-minutes: 20`, so a hung build frees the
|
||||
shared runner after 20 minutes. That allows for the three Docker builds
|
||||
described above (the test phase, the lint phase, then the image), each held to
|
||||
the 5-minute Docker build limit below, plus the bootstrap. A separate workflow
|
||||
limited to `main` by a `branches` list under `on: push` cannot be checked by
|
||||
review: to try a change to it, add the feature branch to that list and push,
|
||||
then remove the branch from the list again before merging. Keep any job in it
|
||||
that publishes behind `if: github.ref_name == 'main'`, so the run from the
|
||||
feature branch publishes nothing.
|
||||
|
||||
- Use platform-standard formatters: `black` for Python, `prettier` for
|
||||
JS/CSS/Markdown/HTML, `go fmt` for Go. Always use default configuration with
|
||||
@@ -286,17 +408,18 @@ style conventions are in separate documents:
|
||||
```
|
||||
|
||||
`-count=1` is required on both invocations: it defeats Go's test _result_
|
||||
cache, so the target cannot report a pass it did not earn, and the rerun
|
||||
reproduces a failure instead of replaying it. It leaves the build cache
|
||||
alone, so it costs the runtime of the suite and no recompilation.
|
||||
cache, so neither run can report a stored pass in place of running the
|
||||
tests. It leaves the build cache alone, so it costs the runtime of the suite
|
||||
and no recompilation.
|
||||
|
||||
Note that this is a second, independent cache, stacked below the Docker
|
||||
layer cache that [issue #26](https://git.eeqj.de/sneak/prompts/issues/26)
|
||||
addresses. `CHECK_EPOCH` guarantees the `RUN make test` _step_ re-executes;
|
||||
it does not guarantee `go test` inside that step does any work, because the
|
||||
`GOCACHE` baked into earlier image layers survives into the re-executed
|
||||
step. They are two separate defects requiring two separate fixes, and a fix
|
||||
for one must not be recorded as covering the other.
|
||||
That cache is Go's own, separate from Docker's layer cache. Go stores a
|
||||
passing result in its cache directory (`GOCACHE`), and when the same tests
|
||||
run again on unchanged code it prints that result, marked `(cached)`,
|
||||
without running them. The directory lasts from one run to the next on a
|
||||
developer's machine, where this target runs, and in the `test` phase of the
|
||||
`Dockerfile`, which mounts it to keep compiled packages between check
|
||||
builds, so the phase's `go test` runs carry `-count=1` too. A new
|
||||
`CACHEBUST` value (above) is what makes that step run on an unchanged tree.
|
||||
|
||||
Python example:
|
||||
|
||||
@@ -324,9 +447,12 @@ style conventions are in separate documents:
|
||||
|
||||
- `.gitignore` should be comprehensive from the start: OS files (`.DS_Store`),
|
||||
editor files (`.swp`, `*~`), in-repo agent scratch directories (`.claude/`),
|
||||
language build artifacts, and `node_modules/`. Fetch the standard `.gitignore`
|
||||
from `https://git.eeqj.de/sneak/prompts/raw/branch/main/.gitignore` when
|
||||
setting up a new repo. These patterns are written to `.gitignore`'s own
|
||||
`node_modules/`, and the repo's own build outputs. Fetch the standard
|
||||
`.gitignore` from
|
||||
`https://git.eeqj.de/sneak/prompts/raw/branch/main/.gitignore` when setting up
|
||||
a new repo. A repo's `.gitignore` is the standard file followed by the repo's
|
||||
own entries, such as its binaries; a re-vendor replaces the standard part and
|
||||
keeps those entries. These patterns are written to `.gitignore`'s own
|
||||
semantics, in which an unanchored pattern already matches at every depth; they
|
||||
are not a `.dockerignore` and must not be transplanted into one unmodified.
|
||||
|
||||
@@ -340,7 +466,7 @@ style conventions are in separate documents:
|
||||
— which is more dangerous than a short file with no secret patterns at all,
|
||||
because it reads as solved and stops anyone looking. Give every
|
||||
depth-independent pattern the `**/` prefix and leave only genuinely
|
||||
root-anchored entries unprefixed: `.git`, and the repo's own host-built
|
||||
root-anchored entries unprefixed: `.claude`, and the repo's own host-built
|
||||
binary, written `/myapp` and never `**/myapp`, which would also match
|
||||
`cmd/myapp/` and delete the package directory from the context. Matching is
|
||||
case-sensitive, and an ALL-CAPS twin per pattern still misses `Server.Key`, so
|
||||
@@ -365,21 +491,24 @@ style conventions are in separate documents:
|
||||
directory, so a repo running agents in subdirectories still ships
|
||||
`services/api/.claude/` and must add its own anchored entry there.
|
||||
|
||||
- **Excluding `.git` means `git describe` cannot run inside any build stage, and
|
||||
it fails quietly there.** In a build stage there is no repository, so
|
||||
`git describe` writes nothing to stdout, `-X main.Version=` comes out empty,
|
||||
the binary reports no version at all, and the build still exits 0. Compute the
|
||||
version on the host and thread it in as a build arg. `script/docker` and
|
||||
`script/cibuild` do this, byte-identically across repos:
|
||||
- **A plain `docker build .` of a clone stamps the version that
|
||||
`git describe --tags --always` gives**, derived from the `.git` in the build
|
||||
context as the canonical `Dockerfile` above shows. Without its failure check,
|
||||
a missing `git` or an unreadable checkout would leave `-X main.Version=` empty
|
||||
and the build would still exit 0. `script/docker` and `script/cibuild` pass
|
||||
the version they compute on the host; it takes precedence. They do this
|
||||
byte-identically across repos:
|
||||
|
||||
```sh
|
||||
# Own line: a failing command substitution inside an argument does not
|
||||
# trip `set -e`, so the inline form degrades to an empty constant.
|
||||
# The version and the tag each get their own line: a failing command
|
||||
# substitution inside an argument does not trip `set -e`, so the inline
|
||||
# form degrades to an empty constant.
|
||||
version="$(git describe --tags --always --dirty 2>/dev/null || true)"
|
||||
[ -n "$version" ] || version="unknown"
|
||||
docker build --no-cache \
|
||||
tag="$(script/projectname)"
|
||||
docker build --build-arg CACHEBUST="$(date +%s)-$$" \
|
||||
--build-arg VERSION="$version" \
|
||||
-t "$(script/projectname)" .
|
||||
-t "$tag" .
|
||||
```
|
||||
|
||||
`--always` makes an untagged repo yield an abbreviated commit hash rather
|
||||
@@ -387,12 +516,12 @@ style conventions are in separate documents:
|
||||
fallback is applied — a live check that fires on a build from an export with
|
||||
no `.git` and on a repository with no commits yet. Do not fold it into the
|
||||
substitution as `|| echo unknown`, which makes the guard unreachable. The
|
||||
Dockerfile's side is `ARG VERSION=dev` in the stage that compiles, declared
|
||||
Dockerfile's side is `ARG VERSION` in the stage that compiles, declared
|
||||
there because `ARG` is stage-scoped; passing `VERSION` to a repo whose
|
||||
Dockerfile declares no such `ARG` is ignored and costs nothing, which is why
|
||||
the scripts stay byte-identical. One consequence for CI: the standard
|
||||
checkout action clones shallow and fetches no tags, so a repo that embeds a
|
||||
tag-derived version must set `fetch-depth: 0` on its checkout step.
|
||||
checkout action clones shallow and fetches no tags, so the canonical
|
||||
`.gitea/workflows/check.yml` sets `fetch-depth: 0` on its checkout step.
|
||||
|
||||
- **Verify `.dockerignore` by enumerating the image, not by reading the
|
||||
patterns.** Plant files at the root _and_ at least two directories deep, build
|
||||
@@ -426,12 +555,18 @@ style conventions are in separate documents:
|
||||
`test-support` depguard rule, where a repo names its own test-support packages
|
||||
by full import path. A repo adds entries there and changes nothing else, and a
|
||||
re-vendor carries its entries forward. The canonical golangci-lint version is
|
||||
v2.12.2 (released 2026-05-06), pinned as the digest of the lint phase's base
|
||||
v2.14.0 (released 2026-09-24), pinned as the digest of the lint phase's base
|
||||
image
|
||||
(`golangci/golangci-lint@sha256:5cceeef04e53efe1470638d4b4b4f5ceefd574955ab3941b2d9a68a8c9ad5240`,
|
||||
which reports `2.12.2 built with go1.26.2 from c0d3ddc9`). That digest is the
|
||||
only pin, since no repo installs golangci-lint on the host: bumping the
|
||||
version means changing it and nothing else.
|
||||
(`golangci/golangci-lint@sha256:ad862ba6b3798cbe0fd9fd7408d498fd74fbd2623a92406b2fd3898faf0bf98f`,
|
||||
which reports `2.14.0 built with go1.27.0 from 114493f9`). A module's `go`
|
||||
directive must not name a newer Go minor version than the one golangci-lint
|
||||
was built with, or golangci-lint refuses to lint it: this release lints
|
||||
`go 1.27.1` but not `go 1.28`. That digest is the only pin, since no repo
|
||||
installs golangci-lint on the host. A repo sets the lint phase digest to the
|
||||
one named here and re-vendors `.golangci.yml` in the same commit, whichever of
|
||||
the two prompted the change: the canonical copy can name linters that an older
|
||||
golangci-lint rejects, and a newer golangci-lint can add linters that
|
||||
`default: all` switches on until the canonical copy disables them.
|
||||
|
||||
- **`script/bootstrap` installs a pinned tool by comparing versions, never by
|
||||
testing presence.** An `if ! command -v <tool>; then install; fi` guard tests
|
||||
@@ -453,8 +588,18 @@ style conventions are in separate documents:
|
||||
function defined and never invoked has the same exit status and the same
|
||||
empty output as one that worked.
|
||||
|
||||
Node is the exception to the whole-token comparison: the canonical
|
||||
`script/bootstrap` compares only its major version, because the node 22
|
||||
alpine image the `Dockerfile` stages start from is not the exact pin, and
|
||||
nvm cannot install a prebuilt node on alpine.
|
||||
|
||||
Keep it POSIX sh: no arrays, no `[[`, no `grep -P`.
|
||||
|
||||
A Go tool a repo needs on the host is installed with `go install` pinned to
|
||||
a commit hash (`go install <package>@<commit hash>`). It is never tracked as
|
||||
a `go.mod` tool dependency or through a `tools.go` file, either of which
|
||||
pulls the tool's own dependencies into the repo's `go.mod` and `go.sum`.
|
||||
|
||||
- When pinning images or packages by hash, add a comment above the reference
|
||||
with the version and date (YYYY-MM-DD).
|
||||
|
||||
@@ -564,13 +709,17 @@ style conventions are in separate documents:
|
||||
Never edit existing migrations after release.
|
||||
|
||||
- All repos should have an `.editorconfig` enforcing the project's indentation
|
||||
settings.
|
||||
settings: the standard file from
|
||||
`https://git.eeqj.de/sneak/prompts/raw/branch/main/.editorconfig`, which sets
|
||||
tabs for `Makefile` and Go files, followed by the repo's own sections, such as
|
||||
one for another language it uses. A re-vendor replaces the standard part and
|
||||
keeps those sections.
|
||||
|
||||
- Avoid putting files in the repo root unless necessary. Root should contain
|
||||
only project-level config files (`README.md`, `Makefile`, `Dockerfile`,
|
||||
`LICENSE`, `.gitignore`, `.editorconfig`, `REPO_POLICIES.md`, and
|
||||
language-specific config). Everything else goes in a subdirectory. Canonical
|
||||
subdirectory names:
|
||||
only project-level config files (`README.md`, `AGENTS.md`, `Makefile`,
|
||||
`Dockerfile`, `LICENSE`, `.gitignore`, `.editorconfig`, `REPO_POLICIES.md`,
|
||||
and language-specific config). Everything else goes in a subdirectory.
|
||||
Canonical subdirectory names:
|
||||
- `bin/` — executable scripts and tools
|
||||
- `cmd/` — Go command entrypoints; thin only: one `main.go` per binary whose
|
||||
body is a single call into `internal/` or `pkg/`, no project logic in
|
||||
@@ -601,3 +750,7 @@ style conventions are in separate documents:
|
||||
- Go: `go.mod`, `go.sum`, `.golangci.yml`
|
||||
- JS: `package.json`, `yarn.lock`, `.prettierrc`, `.prettierignore`
|
||||
- Python: `pyproject.toml`
|
||||
|
||||
- Guidance for coding agents lives in one `AGENTS.md` at the repository root. It
|
||||
is never committed under a file or directory named after one agent tool, such
|
||||
as `CLAUDE.md` or `.claude/`, and never split into separate memory files.
|
||||
|
||||
@@ -18,7 +18,7 @@
|
||||
|
||||
pre-1.0. No git tags exist. The `1.0.0` milestone is in progress; work lands on
|
||||
`next`, and `main` receives only the milestone PR that the owner merges. `next`
|
||||
is at the canonical `golangci-lint` v2.12.2 config and is green. Recent work
|
||||
is at the canonical `golangci-lint` v2.14.0 config and is green. Recent work
|
||||
extracted the internal/magic, internal/allowlist, internal/httpfetcher, and
|
||||
internal/signature packages. The gosec findings from the 2026-07-06 survey are
|
||||
resolved. The disk cache is now size-bounded with LRU eviction
|
||||
@@ -30,6 +30,64 @@ P2: security: per-origin rate limiting
|
||||
|
||||
# Completed Steps
|
||||
|
||||
- 2026-10-15 a WebP, AVIF or JPEG XL encode at the limits takes at most about 34
|
||||
seconds (closes #272): with one libvips thread, 8192x8192 random pixels with
|
||||
an alpha channel took about 61 seconds to encode as WebP, 96 as AVIF and 81 as
|
||||
JPEG XL, so with the default `downstream_timeout` of 60 seconds such an image
|
||||
was never delivered, yet its encode held a processing slot to the end. WebP is
|
||||
now saved at effort 2 and JPEG XL at effort 4, which take up to about 23 and
|
||||
27 seconds. AVIF was already at the lowest effort govips can set, so an AVIF
|
||||
output of more than 16,777,216 pixels is refused with 400 before anything is
|
||||
decoded; 4096x4096, 8192x2048 or 2048x8192 takes up to about 34 seconds. Each
|
||||
time is the slowest at any quality. Resizing with `fit=cover` already decodes
|
||||
the input, so the output's dimensions are worked out before resizing.
|
||||
`README.md` gives the limit and the new times.
|
||||
- 2026-10-15 the README says how a source URL with a query is written into an
|
||||
image URL, and the image route accepts it (closes #255): the query follows the
|
||||
source path after `%3F`. pixa now reads the image URL's path decoded once, so
|
||||
the `?` before the query, any `?` in it and every `%` in the source path or
|
||||
query must be percent-encoded, and any other character may be. The route used
|
||||
to parse chi's wildcard, which holds the path as the client wrote it whenever
|
||||
that is not Go's own encoding of it; a `/`, `;` or `,` in the query written as
|
||||
`%2F`, `%3B` or `%2C`, as `GenerateSignedURL` wrote them, kept the `%3F` from
|
||||
being read as `?`, and the request failed with 401. `GenerateSignedURL` now
|
||||
writes Go's own encoding of the path. The README has an example with a query
|
||||
and its signature. The path traversal check no longer reads the source query,
|
||||
and keeps a `%` that does not start a percent-escape as it is instead of
|
||||
refusing the URL, so a source path or query holding `%25` is accepted, while a
|
||||
double-encoded `..` next to such a `%` is still refused.
|
||||
- 2026-10-15 `max_concurrent_processing` defaults to half the CPUs pixa sees,
|
||||
rounded down, and at least 1 (closes #242): it defaulted to one image per CPU,
|
||||
and a container with no CPU limit sees every CPU of the host, so on a 48-CPU
|
||||
host pixa processed 48 images at once, each of which can take about 4.2 GiB.
|
||||
`config.DefaultMaxConcurrentProcessing` gives the default to the config and to
|
||||
`imageprocessor.New`; `PIXA_MAX_CONCURRENT_PROCESSING` still overrides it.
|
||||
"Running under upaas" in `README.md` gives the worst-case memory, measured,
|
||||
and tells the operator to set `PIXA_MAX_CONCURRENT_PROCESSING` and
|
||||
`PIXA_UPSTREAM_CONNECTIONS` to fit the container's memory. It also gives the
|
||||
file libvips writes to `/tmp` for each large source, up to 1 GiB, measured,
|
||||
which is outside `/var/lib/pixa` and `cache_max_bytes` and is memory where
|
||||
`/tmp` is a tmpfs.
|
||||
- 2026-10-15 the canonical files are re-vendored from `sneak/prompts` commit
|
||||
`e3f79c4` (closes #221): `.dockerignore`, `.editorconfig`,
|
||||
`.gitea/workflows/check.yml`, `.gitignore`, `.golangci.yml`,
|
||||
`.prettierignore`, `.prettierrc`, `REPO_POLICIES.md` and the standard scripts
|
||||
are its copies. pixa's own entries follow the standard part: `/bin`, `/data`
|
||||
and the local config files in `.dockerignore`; Go's `*.log`, `*.out` and
|
||||
`*.test`, `vendor/`, the binaries, `/data/`, `*.sqlite3` and the local config
|
||||
files in `.gitignore`. The workflow keeps its `script/docker-smoke` step while
|
||||
https://git.eeqj.de/sneak/pixa/issues/203 is open. The scripts pass a new
|
||||
`CACHEBUST` build argument in place of `--no-cache`, and every `Dockerfile`
|
||||
stage declares it first, so every `RUN` step still runs on every build while
|
||||
Go's build cache, mounted on the steps that compile, is kept; both `go test`
|
||||
runs keep `-count=1`. The lint and test builds write no image. The lint phase
|
||||
runs golangci-lint v2.14.0, the test phase runs on the Debian Go image, and
|
||||
the build stage marks `/src` safe for git. `script/bootstrap`, `script/fmt`
|
||||
and `script/fmt-check` use the node on the `PATH` only when it has the pinned
|
||||
major version. Of golangci-lint v2.14.0's two findings, an unused `nolint`
|
||||
directive is removed, and gosec's new path check is marked as not applying to
|
||||
the fixed config file locations. The agent guidance moves unchanged from
|
||||
`CLAUDE.md` to a root `AGENTS.md`, where `REPO_POLICIES.md` keeps it.
|
||||
- 2026-10-15 the request metrics record any method that is not a standard one as
|
||||
`other` (closes #274): their `method` label was each request's method as sent,
|
||||
and Go's HTTP server accepts any method name, so a client sending made-up
|
||||
|
||||
@@ -108,10 +108,11 @@ upstream_connections_per_host: 20
|
||||
# downstream_timeout has ended first.
|
||||
upstream_connections: 64
|
||||
|
||||
# Maximum number of images decoded and encoded at once (default: the
|
||||
# number of CPUs pixa can use, which follows a container's CPU limit). A
|
||||
# request that finds none free waits up to 10 seconds for one, and if none
|
||||
# frees up it is answered 503, unless downstream_timeout has ended first.
|
||||
# Maximum number of images decoded and encoded at once (default: half the
|
||||
# number of CPUs pixa can use, at least 1; the number of CPUs follows a
|
||||
# container's CPU limit). A request that finds none free waits up to 10
|
||||
# seconds for one, and if none frees up it is answered 503, unless
|
||||
# downstream_timeout has ended first.
|
||||
# max_concurrent_processing: 4
|
||||
|
||||
# How many requests to the image routes (/v1/image/ and /v1/e/) one client
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
package config
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"runtime"
|
||||
"testing"
|
||||
)
|
||||
@@ -12,8 +13,8 @@ const (
|
||||
)
|
||||
|
||||
// TestOmittedConcurrencyLimitsUseDefaults checks that an omitted
|
||||
// max_concurrent_processing is the number of CPUs Go uses and an omitted
|
||||
// upstream_connections is 64.
|
||||
// max_concurrent_processing is half the CPUs Go uses, and at least 1, and
|
||||
// an omitted upstream_connections is 64.
|
||||
func TestOmittedConcurrencyLimitsUseDefaults(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
@@ -22,9 +23,9 @@ func TestOmittedConcurrencyLimitsUseDefaults(t *testing.T) {
|
||||
t.Fatalf("minimal config should be valid, got error: %v", err)
|
||||
}
|
||||
|
||||
if c.MaxConcurrentProcessing != runtime.GOMAXPROCS(0) {
|
||||
t.Errorf("MaxConcurrentProcessing = %d, want %d, one per CPU",
|
||||
c.MaxConcurrentProcessing, runtime.GOMAXPROCS(0))
|
||||
if c.MaxConcurrentProcessing != max(runtime.GOMAXPROCS(0)/2, 1) {
|
||||
t.Errorf("MaxConcurrentProcessing = %d, want %d, half the CPUs",
|
||||
c.MaxConcurrentProcessing, max(runtime.GOMAXPROCS(0)/2, 1))
|
||||
}
|
||||
|
||||
if c.UpstreamConnections != 64 {
|
||||
@@ -32,6 +33,36 @@ func TestOmittedConcurrencyLimitsUseDefaults(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// TestOmittedMaxConcurrentProcessingIsHalfTheCPUsAndAtLeastOne checks
|
||||
// that an omitted max_concurrent_processing is half the CPUs Go uses,
|
||||
// rounded down, and 1 when Go uses one CPU.
|
||||
//
|
||||
//nolint:paralleltest // it sets GOMAXPROCS, which other tests read
|
||||
func TestOmittedMaxConcurrentProcessingIsHalfTheCPUsAndAtLeastOne(t *testing.T) {
|
||||
cpusBefore := runtime.GOMAXPROCS(0)
|
||||
|
||||
t.Cleanup(func() { runtime.GOMAXPROCS(cpusBefore) })
|
||||
|
||||
for _, tc := range []struct{ cpus, want int }{
|
||||
{1, 1}, {2, 1}, {3, 1}, {8, 4},
|
||||
} {
|
||||
runtime.GOMAXPROCS(tc.cpus)
|
||||
|
||||
c, err := configFromYAML(t, signingKeyLine)
|
||||
if err != nil {
|
||||
t.Fatalf("minimal config should be valid, got error: %v", err)
|
||||
}
|
||||
|
||||
t.Logf("GOMAXPROCS %d: MaxConcurrentProcessing %d",
|
||||
tc.cpus, c.MaxConcurrentProcessing)
|
||||
|
||||
if c.MaxConcurrentProcessing != tc.want {
|
||||
t.Errorf("GOMAXPROCS %d: MaxConcurrentProcessing = %d, want %d",
|
||||
tc.cpus, c.MaxConcurrentProcessing, tc.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestExplicitConcurrencyLimitsAreUsed checks that valid values for the
|
||||
// two limits are used as given.
|
||||
func TestExplicitConcurrencyLimitsAreUsed(t *testing.T) {
|
||||
@@ -160,3 +191,19 @@ func TestConcurrencyLimitsFromEnvironment(t *testing.T) {
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestInvalidMaxConcurrentProcessingFromEnvironmentAbortsStartup checks
|
||||
// that a PIXA_MAX_CONCURRENT_PROCESSING that is not a whole number of at
|
||||
// least 1, an empty one included, aborts startup naming the variable and
|
||||
// the key instead of taking the default.
|
||||
func TestInvalidMaxConcurrentProcessingFromEnvironmentAbortsStartup(t *testing.T) {
|
||||
for _, value := range []string{"", " ", "0.5", "half", "-1", "4 images"} {
|
||||
t.Run(fmt.Sprintf("value %q", value), func(t *testing.T) {
|
||||
t.Setenv(testMaxConcurrentProcessingVar, value)
|
||||
|
||||
_, err := configFromYAML(t, signingKeyLine)
|
||||
wantStartupError(t, err,
|
||||
testMaxConcurrentProcessingVar, keyMaxConcurrentProcessing)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
@@ -37,6 +37,14 @@ const (
|
||||
DefaultImageRequestsPerMinute = 600
|
||||
)
|
||||
|
||||
// DefaultMaxConcurrentProcessing returns the most images processed at once
|
||||
// when max_concurrent_processing is omitted: half the CPUs Go uses
|
||||
// (runtime.GOMAXPROCS(0), which follows a container's CPU limit), rounded
|
||||
// down, and at least 1.
|
||||
func DefaultMaxConcurrentProcessing() int {
|
||||
return max(runtime.GOMAXPROCS(0)/2, 1) //nolint:mnd // half the CPUs
|
||||
}
|
||||
|
||||
// Configuration key names.
|
||||
const (
|
||||
keyDebug = "debug"
|
||||
@@ -297,10 +305,8 @@ func newFromSmartConfig(sc *smartconfig.Config) (*Config, error) {
|
||||
keyUpstreamConnectionsPerHost, DefaultUpstreamConnectionsPerHost),
|
||||
UpstreamConnections: loader.intVal(
|
||||
keyUpstreamConnections, DefaultUpstreamConnections),
|
||||
// Decoding and encoding are CPU-bound, so the default is one image
|
||||
// per CPU Go uses, which follows a container's CPU limit.
|
||||
MaxConcurrentProcessing: loader.intVal(
|
||||
keyMaxConcurrentProcessing, runtime.GOMAXPROCS(0)),
|
||||
keyMaxConcurrentProcessing, DefaultMaxConcurrentProcessing()),
|
||||
UpstreamFetchTimeout: loader.durationVal(
|
||||
keyUpstreamFetchTimeout, DefaultUpstreamFetchTimeout),
|
||||
UpstreamMaxResponseSize: loader.int64Val(
|
||||
@@ -823,6 +829,7 @@ func loadConfigFile(log *slog.Logger, appName string) (*smartconfig.Config, erro
|
||||
// one whose path runs through a file, such as under a HOME of
|
||||
// /dev/null. One that cannot be read or does not parse is a
|
||||
// fatal startup error.
|
||||
//nolint:gosec // G703: a fixed config location; HOME is the operator's
|
||||
_, statErr := os.Stat(cleanPath)
|
||||
if errors.Is(statErr, fs.ErrNotExist) || errors.Is(statErr, syscall.ENOTDIR) {
|
||||
continue
|
||||
|
||||
@@ -0,0 +1,49 @@
|
||||
package handlers
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"sneak.berlin/go/pixa/internal/imageprocessor"
|
||||
"sneak.berlin/go/pixa/internal/imgcache"
|
||||
)
|
||||
|
||||
// TestAVIFOutputTooLargeAnswers400 checks that respondImageError and
|
||||
// handleImageError, which answer the errors of the two image routes, answer
|
||||
// 400 naming the AVIF pixel limit when the image processor refuses an AVIF
|
||||
// output above it, wrapped as the service wraps it. The request's source only
|
||||
// goes into a log line, so it is left empty.
|
||||
func TestAVIFOutputTooLargeAnswers400(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
h := &Handlers{log: slog.New(slog.DiscardHandler)}
|
||||
err := fmt.Errorf("image processing failed: %w: 4097x4096",
|
||||
imageprocessor.ErrAVIFOutputTooLarge)
|
||||
wantBody := fmt.Sprintf("above %d pixels", imageprocessor.MaxAVIFOutputPixels)
|
||||
|
||||
plain := httptest.NewRecorder()
|
||||
h.respondImageError(plain, &imgcache.ImageRequest{}, err)
|
||||
|
||||
encrypted := httptest.NewRecorder()
|
||||
h.handleImageError(encrypted, err)
|
||||
|
||||
for name, rec := range map[string]*httptest.ResponseRecorder{
|
||||
"respondImageError": plain, "handleImageError": encrypted,
|
||||
} {
|
||||
t.Logf("%s: status = %d, body = %q", name, rec.Code, rec.Body.String())
|
||||
|
||||
if rec.Code != http.StatusBadRequest {
|
||||
t.Errorf("%s: status = %d, want %d",
|
||||
name, rec.Code, http.StatusBadRequest)
|
||||
}
|
||||
|
||||
if !strings.Contains(rec.Body.String(), wantBody) {
|
||||
t.Errorf("%s: body = %q, want it to say %q",
|
||||
name, rec.Body.String(), wantBody)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -9,7 +9,6 @@ import (
|
||||
"strconv"
|
||||
"time"
|
||||
|
||||
"github.com/go-chi/chi/v5"
|
||||
"github.com/go-chi/chi/v5/middleware"
|
||||
"sneak.berlin/go/pixa/internal/encurl"
|
||||
"sneak.berlin/go/pixa/internal/httpfetcher"
|
||||
@@ -80,20 +79,21 @@ func (s *Handlers) HandleRobotsTxt() http.HandlerFunc {
|
||||
}
|
||||
}
|
||||
|
||||
// parseImageRequest parses the wildcard path and query parameters into
|
||||
// parseImageRequest parses the image URL's path and query parameters into
|
||||
// an ImageRequest. On invalid input it writes an error response and
|
||||
// returns false.
|
||||
func (s *Handlers) parseImageRequest(
|
||||
w http.ResponseWriter, r *http.Request,
|
||||
) (*imgcache.ImageRequest, bool) {
|
||||
// Get the wildcard path from chi
|
||||
pathParam := chi.URLParam(r, "*")
|
||||
|
||||
// Parse the URL path
|
||||
parsed, err := imgcache.ParseImagePath(pathParam)
|
||||
// The path is parsed decoded once, as r.URL.Path holds it, and not from
|
||||
// chi's wildcard: chi routes on the path as the client wrote it whenever
|
||||
// that is not Go's own encoding of it, so its wildcard would still hold
|
||||
// the %3F before a source query when, say, a "/" in the query is written
|
||||
// as %2F
|
||||
parsed, err := imgcache.ParseImageURL(r.URL.Path)
|
||||
if err != nil {
|
||||
s.log.Warn("failed to parse image URL",
|
||||
"path", pathParam,
|
||||
"path", r.URL.Path,
|
||||
"error", err,
|
||||
)
|
||||
s.respondError(w, "invalid image URL: "+err.Error(), http.StatusBadRequest)
|
||||
@@ -264,6 +264,14 @@ func (s *Handlers) respondImageError(
|
||||
return
|
||||
}
|
||||
|
||||
if errors.Is(err, imageprocessor.ErrAVIFOutputTooLarge) {
|
||||
s.respondError(w, fmt.Sprintf("size too large for AVIF: the output "+
|
||||
"would be above %d pixels", imageprocessor.MaxAVIFOutputPixels),
|
||||
http.StatusBadRequest)
|
||||
|
||||
return
|
||||
}
|
||||
|
||||
s.respondError(w, "internal error", http.StatusInternalServerError)
|
||||
}
|
||||
|
||||
|
||||
@@ -169,3 +169,166 @@ func TestHandleImage_InvalidExp_Returns400(t *testing.T) {
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestHandleImage_GeneratedSignedURLWithSourceQueryVerifies sends URLs built
|
||||
// by the service's signed-URL generator for source URLs with a query to the
|
||||
// route, through a chi router: chi routes on the path as the client wrote it
|
||||
// whenever that is not Go's own encoding of it. A 502 means the signature
|
||||
// passed, so the route read the source path and query that were signed.
|
||||
func TestHandleImage_GeneratedSignedURLWithSourceQueryVerifies(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
tests := []struct {
|
||||
name string
|
||||
query string
|
||||
}{
|
||||
{"slash and colon", "url=https://origin.example.com/cat.jpg"},
|
||||
{"semicolon", "a=1;b=2"},
|
||||
{"comma", "crop=0,0,50,50"},
|
||||
{"equals and ampersand", "a=1&b=2"},
|
||||
{"percent-escape", "name=my%20cat"},
|
||||
{"question mark", "next=/a?b=1"},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
fix := setupTestHandler(t)
|
||||
|
||||
signedURL, err := fix.service.GenerateSignedURL("", &imgcache.ImageRequest{
|
||||
SourceHost: signedHost,
|
||||
SourcePath: photoPath,
|
||||
SourceQuery: tt.query,
|
||||
Size: imgcache.Size{Width: 50, Height: 50},
|
||||
Format: imgcache.FormatJPEG,
|
||||
}, time.Hour)
|
||||
if err != nil {
|
||||
t.Fatalf("GenerateSignedURL() error = %v", err)
|
||||
}
|
||||
|
||||
status := getImage(t, fix, signedURL)
|
||||
if status != http.StatusBadGateway {
|
||||
t.Errorf("status = %d, want %d", status, http.StatusBadGateway)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestHandleImage_SourceQueryEncodedAnyWay signs one source URL with a query
|
||||
// and writes it into the image URL in ways the README allows: the "?" before
|
||||
// the query and each "%" in it percent-encoded, and every other character as
|
||||
// itself or percent-encoded. A 502 means the signature passed.
|
||||
func TestHandleImage_SourceQueryEncodedAnyWay(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
expires := time.Now().Add(time.Hour)
|
||||
signer := signature.New("test-signing-key-must-be-32-chars")
|
||||
sig := signer.Sign(&signature.Request{
|
||||
SourceHost: signedHost,
|
||||
SourcePath: photoPath,
|
||||
SourceQuery: "url=https://origin.example.com/my%20cat.jpg",
|
||||
Width: 50,
|
||||
Height: 50,
|
||||
Format: string(imgcache.FormatJPEG),
|
||||
Quality: 85,
|
||||
FitMode: string(imgcache.FitCover),
|
||||
Expires: expires,
|
||||
})
|
||||
|
||||
tests := []struct {
|
||||
name string
|
||||
writtenQuery string
|
||||
}{
|
||||
{"only ? and % encoded",
|
||||
"%3Furl=https://origin.example.com/my%2520cat.jpg"},
|
||||
{"every reserved character encoded",
|
||||
"%3Furl%3Dhttps%3A%2F%2Forigin.example.com%2Fmy%2520cat.jpg"},
|
||||
{"lower-case hex digits",
|
||||
"%3furl=https://origin.example.com/my%2520cat.jpg"},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
target := fmt.Sprintf("/v1/image/%s%s%s/50x50.jpeg?sig=%s&exp=%d",
|
||||
signedHost, photoPath, tt.writtenQuery, sig, expires.Unix())
|
||||
|
||||
status := getImage(t, setupTestHandler(t), target)
|
||||
if status != http.StatusBadGateway {
|
||||
t.Errorf("status = %d, want %d", status, http.StatusBadGateway)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestHandleImage_SourceURLHoldingPercent25Verifies signs a source path and a
|
||||
// source query that each hold %25, the percent-escape of a literal "%", and
|
||||
// sends each to the route written as the README says, with that "%" written as
|
||||
// %25, and as GenerateSignedURL writes it. A 502 means the signature passed.
|
||||
func TestHandleImage_SourceURLHoldingPercent25Verifies(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
tests := []struct {
|
||||
name string
|
||||
sourcePath string
|
||||
sourceQuery string
|
||||
// writtenPathAndQuery is the source path and query as the README says
|
||||
// to write them into the image URL
|
||||
writtenPathAndQuery string
|
||||
}{
|
||||
{"path", "/sale/50%25off.jpg", "", "/sale/50%2525off.jpg"},
|
||||
{"query", photoPath, "discount=50%25", photoPath + "%3Fdiscount=50%2525"},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name+" written by the README rule", func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
expires := time.Now().Add(time.Hour)
|
||||
signer := signature.New("test-signing-key-must-be-32-chars")
|
||||
sig := signer.Sign(&signature.Request{
|
||||
SourceHost: signedHost,
|
||||
SourcePath: tt.sourcePath,
|
||||
SourceQuery: tt.sourceQuery,
|
||||
Width: 50,
|
||||
Height: 50,
|
||||
Format: string(imgcache.FormatJPEG),
|
||||
Quality: 85,
|
||||
FitMode: string(imgcache.FitCover),
|
||||
Expires: expires,
|
||||
})
|
||||
|
||||
target := fmt.Sprintf("/v1/image/%s%s/50x50.jpeg?sig=%s&exp=%d",
|
||||
signedHost, tt.writtenPathAndQuery, sig, expires.Unix())
|
||||
|
||||
status := getImage(t, setupTestHandler(t), target)
|
||||
if status != http.StatusBadGateway {
|
||||
t.Errorf("status = %d, want %d", status, http.StatusBadGateway)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run(tt.name+" written by GenerateSignedURL", func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
fix := setupTestHandler(t)
|
||||
|
||||
signedURL, err := fix.service.GenerateSignedURL("", &imgcache.ImageRequest{
|
||||
SourceHost: signedHost,
|
||||
SourcePath: tt.sourcePath,
|
||||
SourceQuery: tt.sourceQuery,
|
||||
Size: imgcache.Size{Width: 50, Height: 50},
|
||||
Format: imgcache.FormatJPEG,
|
||||
}, time.Hour)
|
||||
if err != nil {
|
||||
t.Fatalf("GenerateSignedURL() error = %v", err)
|
||||
}
|
||||
|
||||
status := getImage(t, fix, signedURL)
|
||||
if status != http.StatusBadGateway {
|
||||
t.Errorf("status = %d, want %d", status, http.StatusBadGateway)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
@@ -170,6 +170,10 @@ func (s *Handlers) handleImageError(w http.ResponseWriter, err error) {
|
||||
s.respondError(w, fmt.Sprintf("size too large for the source: the side "+
|
||||
"computed from its proportions is above %d",
|
||||
imageprocessor.MaxOutputDimension), http.StatusBadRequest)
|
||||
case errors.Is(err, imageprocessor.ErrAVIFOutputTooLarge):
|
||||
s.respondError(w, fmt.Sprintf("size too large for AVIF: the output "+
|
||||
"would be above %d pixels", imageprocessor.MaxAVIFOutputPixels),
|
||||
http.StatusBadRequest)
|
||||
default:
|
||||
s.log.Error("image request failed", "error", err)
|
||||
s.respondError(w, "internal error", http.StatusInternalServerError)
|
||||
|
||||
@@ -0,0 +1,64 @@
|
||||
package imageprocessor
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"errors"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// TestImageProcessor_RefusesAVIFAboveMaxPixels verifies that an AVIF output
|
||||
// of more than MaxAVIFOutputPixels is refused with ErrAVIFOutputTooLarge
|
||||
// before anything is decoded, whatever size and fit give it, and that an AVIF
|
||||
// output of at most that many pixels, or one in another format, is not. The
|
||||
// input is a 64x32 PNG cut off in its pixel data: libvips reads its size from
|
||||
// its header but fails when it decodes it, so a request that gets that failure
|
||||
// was not refused, and a refusal came before any decoding.
|
||||
func TestImageProcessor_RefusesAVIFAboveMaxPixels(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
whole := createTestPNG(t, 64, 32)
|
||||
cutOff := whole[:len(whole)/2]
|
||||
|
||||
tests := []struct {
|
||||
name string
|
||||
format Format
|
||||
width, height int
|
||||
fit FitMode
|
||||
refused bool
|
||||
}{
|
||||
{"AVIF at the limit", FormatAVIF, 4096, 4096, FitCover, false},
|
||||
{"AVIF above the limit", FormatAVIF, 4096, 4097, FitCover, true},
|
||||
{"WebP above the AVIF limit", FormatWebP, 4096, 4097, FitCover, false},
|
||||
// FitInside does not enlarge, so the output stays 64x32
|
||||
{"AVIF inside a larger size", FormatAVIF, 8192, 8192, FitInside, false},
|
||||
// FitOutside scales 64x32 by 128 to cover 8192x1: 8192x4096
|
||||
{"AVIF outside a smaller size", FormatAVIF, 8192, 1, FitOutside, true},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
_, err := New(Params{}).Process(t.Context(), bytes.NewReader(cutOff),
|
||||
&Request{
|
||||
Size: Size{Width: tt.width, Height: tt.height},
|
||||
Format: tt.format,
|
||||
FitMode: tt.fit,
|
||||
})
|
||||
t.Logf("Process() error = %v", err)
|
||||
|
||||
if tt.refused {
|
||||
if !errors.Is(err, ErrAVIFOutputTooLarge) {
|
||||
t.Errorf("Process() error = %v, want %v", err, ErrAVIFOutputTooLarge)
|
||||
}
|
||||
|
||||
return
|
||||
}
|
||||
|
||||
if err == nil || errors.Is(err, ErrAVIFOutputTooLarge) {
|
||||
t.Errorf("Process() error = %v, want the failure to decode "+
|
||||
"the cut-off input", err)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -77,10 +77,10 @@ func TestImageProcessor_PNGIsCompressed(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// TestImageProcessor_WebPAtDefaultEffort verifies that WebP is saved at
|
||||
// libvips' default effort, 4: the output is the size of the same image saved
|
||||
// at that effort.
|
||||
func TestImageProcessor_WebPAtDefaultEffort(t *testing.T) {
|
||||
// TestImageProcessor_WebPAtEffort2 verifies that WebP is saved at effort 2,
|
||||
// not libvips' default of 4: the output is the size of the same image saved
|
||||
// at effort 2.
|
||||
func TestImageProcessor_WebPAtEffort2(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
input := createTestJPEG(t, 200, 150)
|
||||
@@ -90,14 +90,14 @@ func TestImageProcessor_WebPAtDefaultEffort(t *testing.T) {
|
||||
want, _, err := decode(t, input).ExportWebp(&vips.WebpExportParams{
|
||||
StripMetadata: true,
|
||||
Quality: 85,
|
||||
ReductionEffort: 4,
|
||||
ReductionEffort: 2,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("ExportWebp() error = %v", err)
|
||||
}
|
||||
|
||||
if size != int64(len(want)) {
|
||||
t.Errorf("WebP output is %d bytes, want %d, the size at effort 4",
|
||||
t.Errorf("WebP output is %d bytes, want %d, the size at effort 2",
|
||||
size, len(want))
|
||||
}
|
||||
}
|
||||
@@ -143,3 +143,29 @@ func TestImageProcessor_AVIFAtEffort1(t *testing.T) {
|
||||
"and 8 bits", size, len(want))
|
||||
}
|
||||
}
|
||||
|
||||
// TestImageProcessor_JPEGXLAtEffort4 verifies that JPEG XL is saved at
|
||||
// effort 4, not libvips' default of 7: the output is the size of the same
|
||||
// image saved at effort 4, with the distance exportJXL gives quality 85.
|
||||
func TestImageProcessor_JPEGXLAtEffort4(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
input := createTestJPEG(t, 200, 150)
|
||||
|
||||
size := processedSize(t, input, &Request{Format: FormatJXL, Quality: 85})
|
||||
|
||||
params := vips.NewJxlExportParams()
|
||||
params.Quality = 85
|
||||
params.Distance = jxlDistance(85)
|
||||
params.Effort = 4
|
||||
|
||||
want, _, err := decode(t, input).ExportJxl(params)
|
||||
if err != nil {
|
||||
t.Fatalf("ExportJxl() error = %v", err)
|
||||
}
|
||||
|
||||
if size != int64(len(want)) {
|
||||
t.Errorf("JPEG XL output is %d bytes, want %d, the size at effort 4",
|
||||
size, len(want))
|
||||
}
|
||||
}
|
||||
|
||||
@@ -7,11 +7,11 @@ import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"runtime"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/davidbyttow/govips/v2/vips"
|
||||
"sneak.berlin/go/pixa/internal/config"
|
||||
)
|
||||
|
||||
// vipsOnce ensures vips is initialized exactly once.
|
||||
@@ -20,11 +20,10 @@ import (
|
||||
var vipsOnce sync.Once
|
||||
|
||||
// initVips initializes libvips with quiet logging, one worker thread per
|
||||
// image and no operation cache. Process already works on one image per CPU
|
||||
// by default, so more threads per image would only compete for the CPUs.
|
||||
// Each request decodes different source bytes, so the operation cache
|
||||
// would rarely be hit and would hold memory outside MaxConcurrentProcessing;
|
||||
// repeated requests are served from pixa's disk cache instead.
|
||||
// image and no operation cache. Each request decodes different source
|
||||
// bytes, so the operation cache would rarely be hit and would hold memory
|
||||
// outside MaxConcurrentProcessing; repeated requests are served from
|
||||
// pixa's disk cache instead.
|
||||
func initVips() {
|
||||
vipsOnce.Do(func() {
|
||||
vips.LoggingSettings(nil, vips.LogLevelError)
|
||||
@@ -143,6 +142,19 @@ const MaxOutputDimension = 8192
|
||||
// requested size.
|
||||
var ErrOutputTooLarge = errors.New("output image dimensions exceed maximum")
|
||||
|
||||
// MaxAVIFOutputPixels is the most pixels an AVIF output may have, so that an
|
||||
// encode takes at most 40 seconds of the default downstream_timeout of 60
|
||||
// seconds; AVIF is already at the lowest effort govips can set (avifEffort).
|
||||
// With one thread, as pixad runs libvips, saving 4096x4096, 8192x2048 or
|
||||
// 2048x8192 random pixels with an alpha channel takes up to about 34
|
||||
// seconds, the slowest at any quality and at 8 or 16 bits per sample, and
|
||||
// 5120x4096 at 8 bits already takes about 36.
|
||||
const MaxAVIFOutputPixels = 4096 * 4096
|
||||
|
||||
// ErrAVIFOutputTooLarge is returned, before anything is decoded, when an AVIF
|
||||
// output would have more than MaxAVIFOutputPixels pixels.
|
||||
var ErrAVIFOutputTooLarge = errors.New("AVIF output has too many pixels")
|
||||
|
||||
// ErrUnsupportedOutputFormat is returned when the requested output format is
|
||||
// not supported.
|
||||
var ErrUnsupportedOutputFormat = errors.New("unsupported output format")
|
||||
@@ -179,7 +191,7 @@ type Params struct {
|
||||
// If <= 0, DefaultMaxInputBytes is used.
|
||||
MaxInputBytes int64
|
||||
// MaxConcurrentProcessing is the most images processed at once.
|
||||
// If <= 0, the number of CPUs Go uses (runtime.GOMAXPROCS(0)) is used.
|
||||
// If <= 0, config.DefaultMaxConcurrentProcessing() is used.
|
||||
MaxConcurrentProcessing int
|
||||
}
|
||||
|
||||
@@ -195,7 +207,7 @@ func New(params Params) *ImageProcessor {
|
||||
|
||||
maxConcurrentProcessing := params.MaxConcurrentProcessing
|
||||
if maxConcurrentProcessing <= 0 {
|
||||
maxConcurrentProcessing = runtime.GOMAXPROCS(0)
|
||||
maxConcurrentProcessing = config.DefaultMaxConcurrentProcessing()
|
||||
}
|
||||
|
||||
return &ImageProcessor{
|
||||
@@ -211,7 +223,8 @@ func New(params Params) *ImageProcessor {
|
||||
// ErrTooManyImages. When ctx has ended once Process has a slot, or before it
|
||||
// encodes, it returns ctx's error and gives the slot back. govips gives pixa
|
||||
// no way to stop libvips once an encode has begun, so an encode keeps its
|
||||
// slot until it finishes.
|
||||
// slot until it finishes. An AVIF output of more than MaxAVIFOutputPixels is
|
||||
// refused with ErrAVIFOutputTooLarge before anything is decoded.
|
||||
func (p *ImageProcessor) Process(
|
||||
ctx context.Context,
|
||||
input io.Reader,
|
||||
@@ -224,18 +237,9 @@ func (p *ImageProcessor) Process(
|
||||
|
||||
defer release()
|
||||
|
||||
// Read input with a size limit to prevent unbounded memory consumption.
|
||||
// We read at most maxInputBytes+1 so we can detect if the input exceeds
|
||||
// the limit without consuming additional memory.
|
||||
limited := io.LimitReader(input, p.maxInputBytes+1)
|
||||
|
||||
data, err := io.ReadAll(limited)
|
||||
data, err := p.readInput(input)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to read input: %w", err)
|
||||
}
|
||||
|
||||
if int64(len(data)) > p.maxInputBytes {
|
||||
return nil, ErrInputDataTooLarge
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// govips picks the libvips loader from the input's first bytes with
|
||||
@@ -278,20 +282,28 @@ func (p *ImageProcessor) Process(
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// Resize if needed
|
||||
if targetWidth != origWidth || targetHeight != origHeight {
|
||||
err := p.resize(img, targetWidth, targetHeight, req.FitMode)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to resize: %w", err)
|
||||
}
|
||||
}
|
||||
|
||||
// orig is the source's own format; encode refuses an empty format
|
||||
outputFormat := req.Format
|
||||
if outputFormat == FormatOriginal {
|
||||
outputFormat = sourceFormat
|
||||
}
|
||||
|
||||
// Determine output dimensions; too large ones are refused before
|
||||
// resizing, which with FitCover already decodes the input
|
||||
outputWidth, outputHeight, err := outputDimensions(origWidth, origHeight,
|
||||
targetWidth, targetHeight, req.FitMode, outputFormat)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// Resize if needed
|
||||
if targetWidth != origWidth || targetHeight != origHeight {
|
||||
err := p.resize(img, outputWidth, outputHeight, req.FitMode)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to resize: %w", err)
|
||||
}
|
||||
}
|
||||
|
||||
// Encode to target format
|
||||
output, err := p.encode(ctx, img, outputFormat, req.Quality)
|
||||
if err != nil {
|
||||
@@ -444,6 +456,23 @@ func (p *ImageProcessor) acquireSlot(ctx context.Context) (func(), error) {
|
||||
return release, nil
|
||||
}
|
||||
|
||||
// readInput reads input with a size limit to prevent unbounded memory
|
||||
// consumption, refusing more than maxInputBytes with ErrInputDataTooLarge. It
|
||||
// reads at most maxInputBytes+1 so it can detect if the input exceeds the
|
||||
// limit without consuming additional memory.
|
||||
func (p *ImageProcessor) readInput(input io.Reader) ([]byte, error) {
|
||||
data, err := io.ReadAll(io.LimitReader(input, p.maxInputBytes+1))
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to read input: %w", err)
|
||||
}
|
||||
|
||||
if int64(len(data)) > p.maxInputBytes {
|
||||
return nil, ErrInputDataTooLarge
|
||||
}
|
||||
|
||||
return data, nil
|
||||
}
|
||||
|
||||
// detectFormat returns the format string for a govips image type.
|
||||
func (p *ImageProcessor) detectFormat(imageType vips.ImageType) string {
|
||||
switch imageType {
|
||||
@@ -468,9 +497,61 @@ func (p *ImageProcessor) detectFormat(imageType vips.ImageType) string {
|
||||
}
|
||||
}
|
||||
|
||||
// resize resizes the image according to the fit mode. With FitOutside one
|
||||
// side can come out larger than requested; when it would be above
|
||||
// MaxOutputDimension, resize returns ErrOutputTooLarge without resizing.
|
||||
// outputDimensions returns the dimensions resize gives an image of imgWidth x
|
||||
// imgHeight for the target dimensions and fit mode, without resizing it, and
|
||||
// refuses an output too large for format. With FitOutside one side can come
|
||||
// out larger than the target; when it would be above MaxOutputDimension,
|
||||
// outputDimensions returns ErrOutputTooLarge. An AVIF output of more than
|
||||
// MaxAVIFOutputPixels is ErrAVIFOutputTooLarge.
|
||||
func outputDimensions(
|
||||
imgWidth, imgHeight, width, height int, fit FitMode, format Format,
|
||||
) (int, int, error) {
|
||||
scaleW := float64(width) / float64(imgWidth)
|
||||
scaleH := float64(height) / float64(imgHeight)
|
||||
outputWidth, outputHeight := width, height
|
||||
|
||||
switch fit {
|
||||
case FitCover, FitFill, "":
|
||||
// Cover crops, and fill stretches, to exactly the target dimensions
|
||||
|
||||
case FitContain:
|
||||
// Fit within the target dimensions, maintaining aspect ratio
|
||||
scale := min(scaleW, scaleH)
|
||||
outputWidth = int(float64(imgWidth) * scale)
|
||||
outputHeight = int(float64(imgHeight) * scale)
|
||||
|
||||
case FitInside:
|
||||
// Same as contain, but only shrink
|
||||
scale := min(scaleW, scaleH, 1)
|
||||
outputWidth = int(float64(imgWidth) * scale)
|
||||
outputHeight = int(float64(imgHeight) * scale)
|
||||
|
||||
case FitOutside:
|
||||
// The smallest dimension fits, the other may exceed the target
|
||||
scale := max(scaleW, scaleH)
|
||||
outputWidth = int(float64(imgWidth) * scale)
|
||||
outputHeight = int(float64(imgHeight) * scale)
|
||||
|
||||
default:
|
||||
return 0, 0, fmt.Errorf("%w: %s", ErrInvalidFitMode, fit)
|
||||
}
|
||||
|
||||
// The side FitOutside scales past the target is held to the maximum too
|
||||
if max(outputWidth, outputHeight) > MaxOutputDimension {
|
||||
return 0, 0, fmt.Errorf("%w: %dx%d is above %d", ErrOutputTooLarge,
|
||||
outputWidth, outputHeight, MaxOutputDimension)
|
||||
}
|
||||
|
||||
if format == FormatAVIF && outputWidth*outputHeight > MaxAVIFOutputPixels {
|
||||
return 0, 0, fmt.Errorf("%w: %dx%d is above %d pixels",
|
||||
ErrAVIFOutputTooLarge, outputWidth, outputHeight, MaxAVIFOutputPixels)
|
||||
}
|
||||
|
||||
return outputWidth, outputHeight, nil
|
||||
}
|
||||
|
||||
// resize resizes the image to width x height, the dimensions
|
||||
// outputDimensions gives for the fit mode.
|
||||
func (p *ImageProcessor) resize(
|
||||
img *vips.ImageRef, width, height int, fit FitMode,
|
||||
) error {
|
||||
@@ -479,52 +560,20 @@ func (p *ImageProcessor) resize(
|
||||
// Resize and crop to fill exact dimensions (default)
|
||||
return img.Thumbnail(width, height, vips.InterestingCentre)
|
||||
|
||||
case FitContain:
|
||||
// Resize to fit within dimensions, maintaining aspect ratio
|
||||
imgW, imgH := img.Width(), img.Height()
|
||||
scaleW := float64(width) / float64(imgW)
|
||||
scaleH := float64(height) / float64(imgH)
|
||||
scale := min(scaleW, scaleH)
|
||||
newW := int(float64(imgW) * scale)
|
||||
newH := int(float64(imgH) * scale)
|
||||
|
||||
return img.Thumbnail(newW, newH, vips.InterestingNone)
|
||||
|
||||
case FitFill:
|
||||
// Resize to exact dimensions (may distort)
|
||||
return img.ThumbnailWithSize(width, height, vips.InterestingNone, vips.SizeForce)
|
||||
|
||||
case FitInside:
|
||||
// Same as contain, but only shrink
|
||||
if img.Width() <= width && img.Height() <= height {
|
||||
return nil // Already fits
|
||||
// outputDimensions keeps the dimensions of an image that already fits
|
||||
if img.Width() == width && img.Height() == height {
|
||||
return nil
|
||||
}
|
||||
|
||||
imgW, imgH := img.Width(), img.Height()
|
||||
scaleW := float64(width) / float64(imgW)
|
||||
scaleH := float64(height) / float64(imgH)
|
||||
scale := min(scaleW, scaleH)
|
||||
newW := int(float64(imgW) * scale)
|
||||
newH := int(float64(imgH) * scale)
|
||||
return img.Thumbnail(width, height, vips.InterestingNone)
|
||||
|
||||
return img.Thumbnail(newW, newH, vips.InterestingNone)
|
||||
|
||||
case FitOutside:
|
||||
// Resize so smallest dimension fits, may exceed target on other dimension
|
||||
imgW, imgH := img.Width(), img.Height()
|
||||
scaleW := float64(width) / float64(imgW)
|
||||
scaleH := float64(height) / float64(imgH)
|
||||
scale := max(scaleW, scaleH)
|
||||
newW := int(float64(imgW) * scale)
|
||||
newH := int(float64(imgH) * scale)
|
||||
|
||||
// The dimension that exceeds the target is held to the maximum too
|
||||
if newW > MaxOutputDimension || newH > MaxOutputDimension {
|
||||
return fmt.Errorf("%w: %dx%d is above %d", ErrOutputTooLarge,
|
||||
newW, newH, MaxOutputDimension)
|
||||
}
|
||||
|
||||
return img.Thumbnail(newW, newH, vips.InterestingNone)
|
||||
case FitContain, FitOutside:
|
||||
return img.Thumbnail(width, height, vips.InterestingNone)
|
||||
|
||||
default:
|
||||
return fmt.Errorf("%w: %s", ErrInvalidFitMode, fit)
|
||||
@@ -628,11 +677,16 @@ func exportGIF(img *vips.ImageRef) ([]byte, error) {
|
||||
return output, err
|
||||
}
|
||||
|
||||
// webpEffort is libvips' default WebP effort, from 0 (fastest) to 6.
|
||||
const webpEffort = 4
|
||||
// webpEffort is the WebP effort, from 0 (fastest) to 6. It is below libvips'
|
||||
// default of 4 so that an encode takes at most 40 seconds of the default
|
||||
// downstream_timeout of 60 seconds. With one thread, as pixad runs libvips,
|
||||
// saving an 8192x8192 image of random pixels with an alpha channel takes up
|
||||
// to about 23 seconds at 2, and up to about 16 without one, at any quality;
|
||||
// with an alpha channel, 3 takes about 50 seconds and 4 about 61.
|
||||
const webpEffort = 2
|
||||
|
||||
// exportWebP encodes img as lossy WebP at quality and libvips' default
|
||||
// effort, without metadata.
|
||||
// exportWebP encodes img as lossy WebP at quality and webpEffort, without
|
||||
// metadata.
|
||||
func exportWebP(img *vips.ImageRef, quality int) ([]byte, error) {
|
||||
output, _, err := img.ExportWebp(&vips.WebpExportParams{
|
||||
StripMetadata: true,
|
||||
@@ -644,12 +698,13 @@ func exportWebP(img *vips.ImageRef, quality int) ([]byte, error) {
|
||||
}
|
||||
|
||||
// avifEffort is the AVIF effort, from 0 (fastest) to 9; 1 is the lowest
|
||||
// govips can set. With one thread, as pixad runs libvips, 1 takes about 51
|
||||
// seconds to save an 8192x8192 RGB image of random pixels, and about 110
|
||||
// with an alpha channel, the worst case, against the default
|
||||
// downstream_timeout of 60 seconds. On an image of milder noise, which 1
|
||||
// saves in about 12 seconds, 2 takes nearly a minute and libvips' default,
|
||||
// 4, takes minutes.
|
||||
// govips can set. With one thread, as pixad runs libvips, 1 takes about 52
|
||||
// seconds to save an 8192x8192 RGB image of random pixels at quality 85,
|
||||
// and about 96 with an alpha channel, against the default
|
||||
// downstream_timeout of 60 seconds, so an AVIF output is held to
|
||||
// MaxAVIFOutputPixels. On an image of milder noise, which 1 saves in about
|
||||
// 12 seconds, 2 takes nearly a minute and libvips' default, 4, takes
|
||||
// minutes.
|
||||
const avifEffort = 1
|
||||
|
||||
// avifBitdepth is the AVIF bit depth, 8 bits per sample for every image.
|
||||
@@ -676,10 +731,18 @@ func exportAVIF(img *vips.ImageRef, quality int) ([]byte, error) {
|
||||
// JPEG that names none.
|
||||
const jxlResolution = 72 / 25.4
|
||||
|
||||
// exportJXL encodes img as JPEG XL at quality, with libvips' default effort
|
||||
// and without metadata. govips sends libvips a distance, the JPEG XL
|
||||
// encoder's own measure of quality, along with the quality, and libvips then
|
||||
// uses the distance alone, so the quality is also given as a distance.
|
||||
// jxlEffort is the JPEG XL effort, from 1 (fastest) to 9. It is below
|
||||
// libvips' default of 7 so that an encode takes at most 40 seconds of the
|
||||
// default downstream_timeout of 60 seconds. With one thread, as pixad runs
|
||||
// libvips, saving an 8192x8192 image of random pixels with an alpha channel
|
||||
// takes up to about 27 seconds at 4, and up to about 13 without one, at any
|
||||
// quality; with an alpha channel, 5 takes about 43 seconds and 7 about 81.
|
||||
const jxlEffort = 4
|
||||
|
||||
// exportJXL encodes img as JPEG XL at quality and jxlEffort, without
|
||||
// metadata. govips sends libvips a distance, the JPEG XL encoder's own
|
||||
// measure of quality, along with the quality, and libvips then uses the
|
||||
// distance alone, so the quality is also given as a distance.
|
||||
func exportJXL(img *vips.ImageRef, quality int) ([]byte, error) {
|
||||
// libvips converts a CMYK image to sRGB before it saves WebP, AVIF or
|
||||
// PNG, but cannot save one as JPEG XL. encode has already converted
|
||||
@@ -718,6 +781,7 @@ func exportJXL(img *vips.ImageRef, quality int) ([]byte, error) {
|
||||
params := vips.NewJxlExportParams()
|
||||
params.Quality = quality
|
||||
params.Distance = jxlDistance(quality)
|
||||
params.Effort = jxlEffort
|
||||
|
||||
output, _, err := toSave.ExportJxl(params)
|
||||
if err != nil {
|
||||
|
||||
@@ -115,14 +115,14 @@ func waitForEntries(t *testing.T, entered <-chan struct{}, count int) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestNewDefaultsMaxConcurrentProcessingToCPUs(t *testing.T) {
|
||||
func TestNewDefaultsMaxConcurrentProcessingToHalfTheCPUs(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
for _, limit := range []int{0, -1} {
|
||||
proc := New(Params{MaxConcurrentProcessing: limit})
|
||||
if got := cap(proc.processingSemaphore); got != runtime.GOMAXPROCS(0) {
|
||||
t.Errorf("MaxConcurrentProcessing %d: %d slots, want %d, one per CPU",
|
||||
limit, got, runtime.GOMAXPROCS(0))
|
||||
if got := cap(proc.processingSemaphore); got != max(runtime.GOMAXPROCS(0)/2, 1) {
|
||||
t.Errorf("MaxConcurrentProcessing %d: %d slots, want %d, half the CPUs",
|
||||
limit, got, max(runtime.GOMAXPROCS(0)/2, 1))
|
||||
}
|
||||
}
|
||||
|
||||
@@ -132,6 +132,30 @@ func TestNewDefaultsMaxConcurrentProcessingToCPUs(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// TestNewDefaultIsHalfTheCPUsAndAtLeastOne checks that New, given no
|
||||
// MaxConcurrentProcessing, makes one slot for every two CPUs Go uses,
|
||||
// rounded down, and one slot when Go uses one CPU.
|
||||
//
|
||||
//nolint:paralleltest // it sets GOMAXPROCS, which other tests read
|
||||
func TestNewDefaultIsHalfTheCPUsAndAtLeastOne(t *testing.T) {
|
||||
cpusBefore := runtime.GOMAXPROCS(0)
|
||||
|
||||
t.Cleanup(func() { runtime.GOMAXPROCS(cpusBefore) })
|
||||
|
||||
for _, tc := range []struct{ cpus, want int }{
|
||||
{1, 1}, {2, 1}, {3, 1}, {8, 4},
|
||||
} {
|
||||
runtime.GOMAXPROCS(tc.cpus)
|
||||
|
||||
got := cap(New(Params{}).processingSemaphore)
|
||||
t.Logf("GOMAXPROCS %d: %d slots", tc.cpus, got)
|
||||
|
||||
if got != tc.want {
|
||||
t.Errorf("GOMAXPROCS %d: %d slots, want %d", tc.cpus, got, tc.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestProcessNeverExceedsMaxConcurrentProcessing starts more Process calls
|
||||
// than MaxConcurrentProcessing allows and holds the first ones inside
|
||||
// Process until the test lets them go. No more than the limit may be
|
||||
|
||||
@@ -78,10 +78,6 @@ func (c *Cache) computeDefaultMaxBytes(
|
||||
computed := spaceBytes / freeSpaceFractionDenominator * freeSpaceFractionNumerator
|
||||
computed = min(computed, math.MaxInt64)
|
||||
|
||||
// gosec cannot see that min() above bounds computed, so it reads
|
||||
// this conversion as potentially overflowing. It cannot: computed is
|
||||
// at most math.MaxInt64 on every path here.
|
||||
//nolint:gosec // G115: clamped to MaxInt64 by min above
|
||||
limit := int64(computed)
|
||||
limit = max(limit, DefaultCacheMaxBytesFloor)
|
||||
|
||||
|
||||
@@ -53,7 +53,7 @@ type ServiceConfig struct {
|
||||
// Allowlist is the list of hosts that don't require signatures
|
||||
Allowlist []string
|
||||
// MaxConcurrentProcessing is the most images processed at once; zero
|
||||
// uses the image processor's default, one per CPU
|
||||
// uses the image processor's default, half the CPUs
|
||||
MaxConcurrentProcessing int
|
||||
// Logger for logging
|
||||
Logger *slog.Logger
|
||||
|
||||
@@ -47,6 +47,9 @@ func ValidateDimension(name string, value int) error {
|
||||
var sizeFormatRegex = regexp.MustCompile(
|
||||
`^(\d+)x(\d+)(?:\.(\w+))?$|^(orig)(?:\.(\w+))?$`)
|
||||
|
||||
// percentEscapeRegex matches one percent-escape, such as "%2e"
|
||||
var percentEscapeRegex = regexp.MustCompile(`%[0-9A-Fa-f]{2}`)
|
||||
|
||||
// ParsedURL contains the parsed components of an image proxy URL.
|
||||
type ParsedURL struct {
|
||||
// Host is the source origin host (e.g., "cdn.example.com")
|
||||
@@ -61,9 +64,8 @@ type ParsedURL struct {
|
||||
Format ImageFormat
|
||||
}
|
||||
|
||||
// ParseImagePath parses the path captured by chi's wildcard:
|
||||
// <host>/<path>/<size>.<format>, or <host>/<path>/<size> for JPEG XL
|
||||
// This is the primary entry point when using chi routing.
|
||||
// ParseImagePath parses an image URL's path after /v1/image/, decoded once:
|
||||
// <host>/<path>/<size>.<format>, or <host>/<path>/<size> for JPEG XL.
|
||||
// Examples:
|
||||
// - cdn.example.com/photos/cat.jpg/800x600.webp
|
||||
// - cdn.example.com/photos/cat.jpg/0x0.jpeg
|
||||
@@ -79,9 +81,10 @@ func ParseImagePath(path string) (*ParsedURL, error) {
|
||||
return parseImageComponents(path)
|
||||
}
|
||||
|
||||
// ParseImageURL parses a full URL path like /v1/image/<host>/<path>/<size>.<format>,
|
||||
// or /v1/image/<host>/<path>/<size> for JPEG XL
|
||||
// Use ParseImagePath instead when working with chi's wildcard capture.
|
||||
// ParseImageURL parses an image URL's path, decoded once as r.URL.Path holds
|
||||
// it: /v1/image/<host>/<path>/<size>.<format>, or /v1/image/<host>/<path>/<size>
|
||||
// for JPEG XL. A source query follows the source path after a "?", which the
|
||||
// URL writes as %3F.
|
||||
func ParseImageURL(urlPath string) (*ParsedURL, error) {
|
||||
// Remove the /v1/image/ prefix
|
||||
const prefix = "/v1/image/"
|
||||
@@ -100,12 +103,6 @@ func ParseImageURL(urlPath string) (*ParsedURL, error) {
|
||||
// parseImageComponents parses <host>/<path>/<size>.<format>, or
|
||||
// <host>/<path>/<size> for JPEG XL.
|
||||
func parseImageComponents(remainder string) (*ParsedURL, error) {
|
||||
// Check for path traversal before any other processing
|
||||
err := checkPathTraversal(remainder)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// Find the last path segment, which holds "size" or "size.format"
|
||||
lastSlash := strings.LastIndex(remainder, "/")
|
||||
if lastSlash == -1 {
|
||||
@@ -161,6 +158,13 @@ func parseImageComponents(remainder string) (*ParsedURL, error) {
|
||||
path = path[:qIndex]
|
||||
}
|
||||
|
||||
// The source query names no file on the upstream, so only the host and
|
||||
// the source path are checked for path traversal
|
||||
err = checkPathTraversal(host + path)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// Ensure path starts with /
|
||||
if !strings.HasPrefix(path, "/") {
|
||||
path = "/" + path
|
||||
@@ -186,12 +190,7 @@ func checkPathTraversal(path string) error {
|
||||
// Decode multiple times to catch double-encoding
|
||||
decoded := path
|
||||
for range 3 {
|
||||
newDecoded, err := url.PathUnescape(decoded)
|
||||
if err != nil {
|
||||
// Malformed encoding is suspicious
|
||||
return ErrPathTraversal
|
||||
}
|
||||
|
||||
newDecoded := decodePercentEscapes(decoded)
|
||||
if newDecoded == decoded {
|
||||
break
|
||||
}
|
||||
@@ -232,6 +231,18 @@ func checkPathTraversal(path string) error {
|
||||
return nil
|
||||
}
|
||||
|
||||
// decodePercentEscapes decodes every percent-escape in s, such as %2e, and
|
||||
// keeps a "%" that does not start one as it is, where url.PathUnescape would
|
||||
// refuse the whole string. A source path holding %25, a literal "%", has such
|
||||
// a "%" after one more decoding.
|
||||
func decodePercentEscapes(s string) string {
|
||||
return percentEscapeRegex.ReplaceAllStringFunc(s, func(escape string) string {
|
||||
decoded, _ := url.PathUnescape(escape)
|
||||
|
||||
return decoded
|
||||
})
|
||||
}
|
||||
|
||||
// parseSizeFormat parses strings like "800x600.webp", "orig.png" or "800x600"
|
||||
func parseSizeFormat(s string) (Size, ImageFormat, error) {
|
||||
matches := sizeFormatRegex.FindStringSubmatch(s)
|
||||
|
||||
@@ -407,6 +407,71 @@ func TestParseImagePath_PathTraversal(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// TestParseImageURL_SourceURLHoldingPercent25 verifies that a source path and
|
||||
// a source query holding %25, the percent-escape of a literal "%", are
|
||||
// accepted as they are, although decoding them once more leaves a "%" that
|
||||
// does not start a percent-escape.
|
||||
func TestParseImageURL_SourceURLHoldingPercent25(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
tests := []struct {
|
||||
name string
|
||||
input string
|
||||
want *ParsedURL
|
||||
}{
|
||||
{
|
||||
name: "path",
|
||||
input: "/v1/image/cdn.example.com/sale/50%25off.jpg/800x600.webp",
|
||||
want: &ParsedURL{
|
||||
Host: testHostCDN, Path: "/sale/50%25off.jpg",
|
||||
Size: Size{Width: 800, Height: 600}, Format: FormatWebP,
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "query",
|
||||
input: "/v1/image/cdn.example.com/photos/cat.jpg?discount=50%25/800x600.webp",
|
||||
want: &ParsedURL{
|
||||
Host: testHostCDN, Path: testPathCat, Query: "discount=50%25",
|
||||
Size: Size{Width: 800, Height: 600}, Format: FormatWebP,
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
got, err := ParseImageURL(tt.input)
|
||||
if err != nil {
|
||||
t.Fatalf("ParseImageURL() unexpected error = %v", err)
|
||||
}
|
||||
|
||||
assertParsedURL(t, got, tt.want)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestParseImageURL_DoubleEncodedTraversalNextToPercent verifies that a
|
||||
// double-encoded ".." is still refused when the path also holds a "%" that
|
||||
// does not start a percent-escape, either as written or after one more
|
||||
// decoding.
|
||||
func TestParseImageURL_DoubleEncodedTraversalNextToPercent(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
for _, input := range []string{
|
||||
"/v1/image/cdn.example.com/photos/%252e%252e/secret/50%25off.jpg/800x600.jpeg",
|
||||
"/v1/image/cdn.example.com/photos/%252e%252e/secret/100%/800x600.jpeg",
|
||||
} {
|
||||
_, err := ParseImageURL(input)
|
||||
t.Logf("ParseImageURL(%q) error = %v", input, err)
|
||||
|
||||
if !errors.Is(err, ErrPathTraversal) {
|
||||
t.Errorf("ParseImageURL(%q) error = %v, want ErrPathTraversal",
|
||||
input, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// errorIs checks if err matches target (handles wrapped errors).
|
||||
func errorIs(err, target error) bool {
|
||||
if errors.Is(err, target) {
|
||||
|
||||
@@ -126,31 +126,20 @@ func (s *Signer) GenerateSignedURL(
|
||||
sizeStr = fmt.Sprintf("%dx%d", req.Width, req.Height)
|
||||
}
|
||||
|
||||
// Build the path.
|
||||
// When a source query is present, it is embedded as a path segment
|
||||
// (e.g. /host/path?query/size.fmt) so that the URL parser can extract
|
||||
// it from the last-slash split. The "?" inside a path segment is
|
||||
// percent-encoded by clients but chi delivers it decoded, which is
|
||||
// exactly what the URL parser expects.
|
||||
var path string
|
||||
// Build the path. A source query follows the source path after a "?".
|
||||
// The image route reads the path decoded once, so the whole path is
|
||||
// written in Go's own percent-encoding of a URL path, which writes "?"
|
||||
// as %3F and "%" as %25, and leaves "/", ";", ",", "=" and "&" as they
|
||||
// are.
|
||||
sourcePathAndQuery := req.SourcePath
|
||||
if req.SourceQuery != "" {
|
||||
path = fmt.Sprintf("/v1/image/%s%s%%3F%s/%s.%s",
|
||||
req.SourceHost,
|
||||
req.SourcePath,
|
||||
url.PathEscape(req.SourceQuery),
|
||||
sizeStr,
|
||||
req.Format,
|
||||
)
|
||||
} else {
|
||||
path = fmt.Sprintf("/v1/image/%s%s/%s.%s",
|
||||
req.SourceHost,
|
||||
req.SourcePath,
|
||||
sizeStr,
|
||||
req.Format,
|
||||
)
|
||||
sourcePathAndQuery += "?" + req.SourceQuery
|
||||
}
|
||||
|
||||
return path, sig, exp
|
||||
unescapedPath := fmt.Sprintf("/v1/image/%s%s/%s.%s",
|
||||
req.SourceHost, sourcePathAndQuery, sizeStr, req.Format)
|
||||
|
||||
return (&url.URL{Path: unescapedPath}).EscapedPath(), sig, exp
|
||||
}
|
||||
|
||||
// buildSignatureData creates the string to be signed.
|
||||
|
||||
@@ -475,6 +475,44 @@ func TestGenerateSignedURL_WithoutQueryString(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// TestGenerateSignedURL_ReadmeQueryExample checks the README's example of a
|
||||
// source URL with a query, the image at
|
||||
// https://cdn.example.com/img?url=https://origin.example.com/my%20cat.jpg: its
|
||||
// signature with the README's key, and the path GenerateSignedURL writes for
|
||||
// it, with the "?" written as %3F and the "%" as %25.
|
||||
func TestGenerateSignedURL_ReadmeQueryExample(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
signer := signature.New("example-signing-key-for-documentation")
|
||||
|
||||
req := signature.Request{
|
||||
SourceHost: testHost,
|
||||
SourcePath: "/img",
|
||||
SourceQuery: "url=https://origin.example.com/my%20cat.jpg",
|
||||
Width: 800,
|
||||
Height: 600,
|
||||
Format: testFormatWebP,
|
||||
Quality: 85,
|
||||
FitMode: testFitCover,
|
||||
Expires: time.Unix(1704067200, 0),
|
||||
}
|
||||
|
||||
const wantSig = "37aFa9GSVCZM1kRY9SzMOk2fOg_M7Wt5gKNR4xM-FYA="
|
||||
|
||||
gotSig := signer.Sign(&req)
|
||||
if gotSig != wantSig {
|
||||
t.Errorf("Sign() = %q, want %q", gotSig, wantSig)
|
||||
}
|
||||
|
||||
const wantPath = "/v1/image/cdn.example.com" +
|
||||
"/img%3Furl=https://origin.example.com/my%2520cat.jpg/800x600.webp"
|
||||
|
||||
gotPath, _, _ := signer.GenerateSignedURL(&req, time.Hour)
|
||||
if gotPath != wantPath {
|
||||
t.Errorf("GenerateSignedURL() path = %q, want %q", gotPath, wantPath)
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseParams(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
|
||||
+33
-12
@@ -3,11 +3,13 @@
|
||||
# this repo. Idempotent: every install is guarded by a check so already
|
||||
# installed tools are skipped. Base tooling comes from nix, apt, brew,
|
||||
# or apk (detected in that order); assumes NOTHING is present (not git,
|
||||
# make, or go). Node is used directly if installed; otherwise it is
|
||||
# installed at a pinned version via nvm (installing nvm itself first,
|
||||
# from a hash-verified release archive, never curl | sh). The linter is
|
||||
# never installed on the host: golangci-lint runs only in the lint phase
|
||||
# of the Dockerfile (see script/lint).
|
||||
# make, or go). The installed node is used only when its major version
|
||||
# is the pinned one; otherwise (node missing, or another major) the
|
||||
# pinned version is installed via nvm (installing nvm itself first,
|
||||
# from a hash-verified release archive, never curl | sh), and yarn and
|
||||
# the packages are installed under it. The linter is never installed on
|
||||
# the host: golangci-lint runs only in the lint phase of the Dockerfile
|
||||
# (see script/lint).
|
||||
#
|
||||
# script/bootstrap git, make, Go, and Node, Yarn and the
|
||||
# prettier in yarn.lock for script/fmt and
|
||||
@@ -110,31 +112,50 @@ ensure_nvm() {
|
||||
rm -rf "$tmp"
|
||||
}
|
||||
|
||||
# True when the node on PATH has the pinned major version. Only the
|
||||
# major is compared, as REPO_POLICIES.md asks. Another major is not
|
||||
# used: the pinned yarn 1 prints a deprecation warning under node 24.
|
||||
# script/fmt and script/fmt-check carry this function unchanged, to run
|
||||
# the yarn installed here.
|
||||
node_is_pinned_major() {
|
||||
if ! command -v node >/dev/null 2>&1; then return 1; fi
|
||||
installed="$(node --version)"
|
||||
installed="${installed#v}"
|
||||
[ "${installed%%.*}" = "${NODE_VERSION%%.*}" ]
|
||||
}
|
||||
|
||||
ensure_node() {
|
||||
if ! missing node; then return 0; fi
|
||||
if node_is_pinned_major; then
|
||||
echo "bootstrap: using node $(node --version)"
|
||||
return 0
|
||||
fi
|
||||
ensure_nvm
|
||||
nvm_sh "nvm install $NODE_VERSION"
|
||||
nvm_sh "nvm use $NODE_VERSION >/dev/null && \
|
||||
echo \"bootstrap: using node \$(node --version)\""
|
||||
}
|
||||
|
||||
ensure_yarn() {
|
||||
if ! node_is_pinned_major; then
|
||||
nvm_sh "nvm use $NODE_VERSION >/dev/null && corepack enable && \
|
||||
corepack prepare yarn@$YARN_VERSION --activate"
|
||||
return 0
|
||||
fi
|
||||
if ! missing yarn; then return 0; fi
|
||||
if ! missing corepack; then
|
||||
corepack enable
|
||||
corepack prepare "yarn@$YARN_VERSION" --activate
|
||||
elif [ -s "$HOME/.nvm/nvm.sh" ]; then
|
||||
nvm_sh "nvm use $NODE_VERSION >/dev/null && corepack enable && \
|
||||
corepack prepare yarn@$YARN_VERSION --activate"
|
||||
else
|
||||
npm install -g "yarn@$YARN_VERSION"
|
||||
fi
|
||||
}
|
||||
|
||||
install_js_deps() {
|
||||
if missing yarn && [ -s "$HOME/.nvm/nvm.sh" ]; then
|
||||
if node_is_pinned_major; then
|
||||
yarn install --frozen-lockfile
|
||||
else
|
||||
nvm_sh "nvm use $NODE_VERSION >/dev/null && cd \"$ROOT\" && \
|
||||
yarn install --frozen-lockfile"
|
||||
else
|
||||
yarn install --frozen-lockfile
|
||||
fi
|
||||
}
|
||||
|
||||
|
||||
+13
-10
@@ -2,9 +2,11 @@
|
||||
# script/cibuild: run the CI build. It bootstraps first: a CI runner
|
||||
# checks out and runs this and nothing else, and script/fmt-check runs
|
||||
# the formatter on the host, which a pristine checkout cannot do.
|
||||
# --no-cache for the same reason as script/docker: the gate phases the
|
||||
# final stage depends on are RUN steps, and a cached one is a check that
|
||||
# did not run.
|
||||
# A new CACHEBUST value reruns every RUN step, since a cached gate phase
|
||||
# is a check that did not run; unlike --no-cache, it keeps Go's build cache.
|
||||
# This relies on every stage of the Dockerfile declaring ARG CACHEBUST
|
||||
# first; a stage without that line is served from cache on an unchanged
|
||||
# tree, with no warning.
|
||||
set -eu
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
|
||||
@@ -14,16 +16,17 @@ main() {
|
||||
cd "$ROOT"
|
||||
"$SCRIPT_DIR/bootstrap"
|
||||
"$SCRIPT_DIR/check"
|
||||
# Own line: a failing command substitution inside an argument does
|
||||
# not trip `set -e`, so the inline form degrades silently to an
|
||||
# empty constant. VERSION is computed here because .dockerignore
|
||||
# excludes .git, so `git describe` in a build stage yields an empty
|
||||
# version without failing.
|
||||
# The version and the tag each get their own line: a failing
|
||||
# command substitution inside an argument does not trip `set -e`,
|
||||
# so the inline form degrades silently to an empty constant. The
|
||||
# VERSION build argument takes precedence over the version a build
|
||||
# stage derives from the .git in the context.
|
||||
version="$(git describe --tags --always --dirty 2>/dev/null || true)"
|
||||
[ -n "$version" ] || version="unknown"
|
||||
docker build --no-cache \
|
||||
tag="$("$SCRIPT_DIR/projectname")"
|
||||
docker build --build-arg CACHEBUST="$(date +%s)-$$" \
|
||||
--build-arg VERSION="$version" \
|
||||
-t "$("$SCRIPT_DIR/projectname")" .
|
||||
-t "$tag" .
|
||||
}
|
||||
|
||||
main "$@"
|
||||
|
||||
+13
-9
@@ -1,8 +1,11 @@
|
||||
#!/bin/sh
|
||||
# script/docker: build the Docker image tagged with the project name.
|
||||
# Identical in all repos; the tag comes from script/projectname.
|
||||
# --no-cache because the gate phases the final stage depends on are RUN
|
||||
# steps, and a cached one is a check that did not run.
|
||||
# A new CACHEBUST value reruns every RUN step, since a cached gate phase
|
||||
# is a check that did not run; unlike --no-cache, it keeps Go's build cache.
|
||||
# This relies on every stage of the Dockerfile declaring ARG CACHEBUST
|
||||
# first; a stage without that line is served from cache on an unchanged
|
||||
# tree, with no warning.
|
||||
set -eu
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
|
||||
@@ -10,16 +13,17 @@ ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
|
||||
|
||||
main() {
|
||||
cd "$ROOT"
|
||||
# Own line: a failing command substitution inside an argument does
|
||||
# not trip `set -e`, so the inline form degrades silently to an
|
||||
# empty constant. VERSION is computed here because .dockerignore
|
||||
# excludes .git, so `git describe` in a build stage yields an empty
|
||||
# version without failing.
|
||||
# The version and the tag each get their own line: a failing
|
||||
# command substitution inside an argument does not trip `set -e`,
|
||||
# so the inline form degrades silently to an empty constant. The
|
||||
# VERSION build argument takes precedence over the version a build
|
||||
# stage derives from the .git in the context.
|
||||
version="$(git describe --tags --always --dirty 2>/dev/null || true)"
|
||||
[ -n "$version" ] || version="unknown"
|
||||
docker build --no-cache \
|
||||
tag="$("$SCRIPT_DIR/projectname")"
|
||||
docker build --build-arg CACHEBUST="$(date +%s)-$$" \
|
||||
--build-arg VERSION="$version" \
|
||||
-t "$("$SCRIPT_DIR/projectname")" .
|
||||
-t "$tag" .
|
||||
}
|
||||
|
||||
main "$@"
|
||||
|
||||
+16
-6
@@ -7,15 +7,25 @@ ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
||||
# Must match the pin in script/bootstrap.
|
||||
NODE_VERSION="22.17.0"
|
||||
|
||||
# script/bootstrap installs node and yarn under nvm and leaves neither
|
||||
# on the PATH of the shell that called it, so resolve the pinned
|
||||
# toolchain here the way bootstrap's own install step does. nvm is a
|
||||
# bash script, hence the subshell.
|
||||
# True when the node on PATH has the pinned major version: the same
|
||||
# function as in script/bootstrap, which uses it to decide where it
|
||||
# installs yarn.
|
||||
node_is_pinned_major() {
|
||||
if ! command -v node >/dev/null 2>&1; then return 1; fi
|
||||
installed="$(node --version)"
|
||||
installed="${installed#v}"
|
||||
[ "${installed%%.*}" = "${NODE_VERSION%%.*}" ]
|
||||
}
|
||||
|
||||
# Run the yarn script/bootstrap installed: the yarn on PATH when the
|
||||
# node on PATH has the pinned major version, and otherwise yarn under
|
||||
# the pinned node in nvm, which bootstrap leaves off the PATH of the
|
||||
# shell that called it. nvm is a bash script, hence the subshell.
|
||||
run_yarn() {
|
||||
if command -v yarn >/dev/null 2>&1; then
|
||||
if node_is_pinned_major && command -v yarn >/dev/null 2>&1; then
|
||||
exec yarn "$@"
|
||||
fi
|
||||
if [ ! -s "$HOME/.nvm/nvm.sh" ]; then
|
||||
if node_is_pinned_major || [ ! -s "$HOME/.nvm/nvm.sh" ]; then
|
||||
echo "fmt: no yarn; run script/bootstrap first" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
+16
-6
@@ -8,15 +8,25 @@ ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
||||
# Must match the pin in script/bootstrap.
|
||||
NODE_VERSION="22.17.0"
|
||||
|
||||
# script/bootstrap installs node and yarn under nvm and leaves neither
|
||||
# on the PATH of the shell that called it, so resolve the pinned
|
||||
# toolchain here the way bootstrap's own install step does. nvm is a
|
||||
# bash script, hence the subshell.
|
||||
# True when the node on PATH has the pinned major version: the same
|
||||
# function as in script/bootstrap, which uses it to decide where it
|
||||
# installs yarn.
|
||||
node_is_pinned_major() {
|
||||
if ! command -v node >/dev/null 2>&1; then return 1; fi
|
||||
installed="$(node --version)"
|
||||
installed="${installed#v}"
|
||||
[ "${installed%%.*}" = "${NODE_VERSION%%.*}" ]
|
||||
}
|
||||
|
||||
# Run the yarn script/bootstrap installed: the yarn on PATH when the
|
||||
# node on PATH has the pinned major version, and otherwise yarn under
|
||||
# the pinned node in nvm, which bootstrap leaves off the PATH of the
|
||||
# shell that called it. nvm is a bash script, hence the subshell.
|
||||
run_yarn() {
|
||||
if command -v yarn >/dev/null 2>&1; then
|
||||
if node_is_pinned_major && command -v yarn >/dev/null 2>&1; then
|
||||
exec yarn "$@"
|
||||
fi
|
||||
if [ ! -s "$HOME/.nvm/nvm.sh" ]; then
|
||||
if node_is_pinned_major || [ ! -s "$HOME/.nvm/nvm.sh" ]; then
|
||||
echo "fmt-check: no yarn; run script/bootstrap first" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
+8
-5
@@ -5,9 +5,12 @@
|
||||
# make its answer untrustworthy.
|
||||
#
|
||||
# The phase is not the last stage in the file, so it is built only when
|
||||
# --target names it. --no-cache because a cached lint layer is a lint
|
||||
# that did not run. The tag makes each build replace the previous image
|
||||
# instead of leaving a dangling one behind.
|
||||
# --target names it. A new CACHEBUST value reruns every RUN step, since
|
||||
# a cached lint layer is a lint that did not run; unlike --no-cache, it
|
||||
# keeps Go's build cache. This relies on every stage of the Dockerfile
|
||||
# declaring ARG CACHEBUST first; a stage without that line is served
|
||||
# from cache on an unchanged tree, with no warning.
|
||||
# --output type=cacheonly writes no image, since nothing uses one.
|
||||
set -eu
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
|
||||
@@ -15,9 +18,9 @@ ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
|
||||
|
||||
main() {
|
||||
cd "$ROOT"
|
||||
docker build --no-cache \
|
||||
docker build --build-arg CACHEBUST="$(date +%s)-$$" \
|
||||
--target lint \
|
||||
-t "$("$SCRIPT_DIR/projectname")-lint" .
|
||||
--output type=cacheonly .
|
||||
}
|
||||
|
||||
main "$@"
|
||||
|
||||
+8
-4
@@ -2,8 +2,12 @@
|
||||
# script/test: run the test suite. Testing is a phase of the Dockerfile
|
||||
# and this builds that phase alone, on the same terms as script/lint:
|
||||
# --target because a phase that is not the last stage is built only when
|
||||
# named, --no-cache because a cached test layer is a test that did not
|
||||
# run, and a tag so each build replaces the previous image.
|
||||
# named, and a new CACHEBUST value because a cached test layer is a test
|
||||
# that did not run; unlike --no-cache, it keeps Go's build cache. This
|
||||
# relies on every stage of the Dockerfile declaring ARG CACHEBUST first;
|
||||
# a stage without that line is served from cache on an unchanged tree,
|
||||
# with no warning. --output type=cacheonly writes no image, since
|
||||
# nothing uses one.
|
||||
set -eu
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
|
||||
@@ -11,9 +15,9 @@ ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
|
||||
|
||||
main() {
|
||||
cd "$ROOT"
|
||||
docker build --no-cache \
|
||||
docker build --build-arg CACHEBUST="$(date +%s)-$$" \
|
||||
--target test \
|
||||
-t "$("$SCRIPT_DIR/projectname")-test" .
|
||||
--output type=cacheonly .
|
||||
}
|
||||
|
||||
main "$@"
|
||||
|
||||
Reference in New Issue
Block a user