Compare commits
12
Commits
0ec6cb00c0
...
a9e4d76b5f
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
a9e4d76b5f | ||
|
|
cead3d57bf | ||
|
|
b12cb4565c | ||
|
|
80e336c937 | ||
|
|
64d96aab87 | ||
|
|
e8fdd49c80 | ||
|
|
5b89f928d7 | ||
|
|
ebdecf9aba | ||
|
|
de7ecfdb5e | ||
|
|
ae7564b769 | ||
|
|
4e30dc7467 | ||
|
|
16f9d1dfd1 |
+18
-9
@@ -13,12 +13,21 @@
|
|||||||
# `/myapp`, never `**/myapp`, which also matches `cmd/myapp/` and
|
# `/myapp`, never `**/myapp`, which also matches `cmd/myapp/` and
|
||||||
# deletes the package directory from the context.
|
# deletes the package directory from the context.
|
||||||
|
|
||||||
# Unlike the standard file, which leaves out all of .git, pixa sends
|
# .git is sent without its config. Without a VERSION build argument the
|
||||||
# .git without its config. Without a VERSION build argument the stage
|
# stage that compiles runs `git describe --tags --always` on .git, which
|
||||||
# that compiles runs `git describe --tags --always` on .git, which does
|
# does not need .git/config; that file can hold a credential, such as a
|
||||||
# not need .git/config; that file can hold a credential, such as a
|
|
||||||
# password in a remote URL or the token the CI checkout step stores there.
|
# password in a remote URL or the token the CI checkout step stores there.
|
||||||
.git/config
|
# Each submodule keeps a config with the same exposure in its git directory
|
||||||
|
# under .git/modules/, nested again for a submodule's own submodules, or in
|
||||||
|
# its own .git directory when it keeps one.
|
||||||
|
# KNOWN GAP: a submodule whose name has a `config` segment (`config`,
|
||||||
|
# `deploy/config`, `config/lib`) loses its whole git directory, because
|
||||||
|
# `**/.git/modules/**/config` also matches that segment's directory
|
||||||
|
# under .git/modules/. Go's version stamping then fails the build;
|
||||||
|
# nothing leaks. Name such a submodule without that segment:
|
||||||
|
# `git submodule add --name`.
|
||||||
|
**/.git/config
|
||||||
|
**/.git/modules/**/config
|
||||||
|
|
||||||
# Agent scratch: one full checkout of the repo per in-flight agent.
|
# Agent scratch: one full checkout of the repo per in-flight agent.
|
||||||
# Anchored because it occurs once where agents run at the repo root.
|
# Anchored because it occurs once where agents run at the repo root.
|
||||||
@@ -42,7 +51,9 @@
|
|||||||
**/[iI][dD]_[rR][sS][aA]
|
**/[iI][dD]_[rR][sS][aA]
|
||||||
**/[iI][dD]_[dD][sS][aA]
|
**/[iI][dD]_[dD][sS][aA]
|
||||||
**/[iI][dD]_[eE][cC][dD][sS][aA]
|
**/[iI][dD]_[eE][cC][dD][sS][aA]
|
||||||
|
**/[iI][dD]_[eE][cC][dD][sS][aA]_[sS][kK]
|
||||||
**/[iI][dD]_[eE][dD]25519
|
**/[iI][dD]_[eE][dD]25519
|
||||||
|
**/[iI][dD]_[eE][dD]25519_[sS][kK]
|
||||||
|
|
||||||
# Dependencies: restored inside the image, never copied in.
|
# Dependencies: restored inside the image, never copied in.
|
||||||
**/node_modules
|
**/node_modules
|
||||||
@@ -60,10 +71,8 @@
|
|||||||
**/.vscode
|
**/.vscode
|
||||||
**/*.sublime-*
|
**/*.sublime-*
|
||||||
|
|
||||||
# pixa's own entries. Nothing in the build reads .gitignore. On the
|
# pixa's own entries. On the host, `make build` writes bin/pixad, and the
|
||||||
# host, `make build` writes bin/pixad, and the example config keeps its
|
# example config keeps its state directory in data/.
|
||||||
# state directory in data/.
|
|
||||||
.gitignore
|
|
||||||
/bin
|
/bin
|
||||||
/data
|
/data
|
||||||
|
|
||||||
|
|||||||
@@ -13,3 +13,6 @@ indent_style = tab
|
|||||||
|
|
||||||
[*.go]
|
[*.go]
|
||||||
indent_style = tab
|
indent_style = tab
|
||||||
|
|
||||||
|
# This repository's own sections, such as one for another language it
|
||||||
|
# uses, go below this comment, and a re-vendor keeps them.
|
||||||
|
|||||||
@@ -1,15 +1,21 @@
|
|||||||
name: check
|
name: check
|
||||||
on: [push]
|
on: [push]
|
||||||
|
# Free the shared runner: a new push cancels only the same branch's older run.
|
||||||
|
concurrency:
|
||||||
|
group: ${{ github.workflow }}-${{ github.ref }}
|
||||||
|
cancel-in-progress: true
|
||||||
jobs:
|
jobs:
|
||||||
check:
|
check:
|
||||||
runs-on: ubuntu-latest
|
runs-on: ubuntu-latest
|
||||||
|
# Free the shared runner from a hung build.
|
||||||
|
timeout-minutes: 20
|
||||||
steps:
|
steps:
|
||||||
# actions/checkout v4.2.2, 2026-02-22
|
# actions/checkout v4.2.2, 2026-02-22
|
||||||
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683
|
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683
|
||||||
# The default clone is shallow and has no tags, so the
|
# script/cibuild needs no token, so none is left in .git/config.
|
||||||
# version the build takes from `git describe` would be a
|
|
||||||
# bare commit; this fetches the whole history with its tags.
|
|
||||||
with:
|
with:
|
||||||
|
persist-credentials: false
|
||||||
|
# All history and tags, so git describe finds the version tag.
|
||||||
fetch-depth: 0
|
fetch-depth: 0
|
||||||
- run: script/cibuild
|
- run: script/cibuild
|
||||||
- run: script/docker-smoke
|
- run: script/docker-smoke
|
||||||
|
|||||||
+37
-6
@@ -17,15 +17,46 @@ Thumbs.db
|
|||||||
# entry and must not be given a `**/` prefix on the way into one.
|
# entry and must not be given a `**/` prefix on the way into one.
|
||||||
.claude/
|
.claude/
|
||||||
|
|
||||||
# Environment / secrets
|
# Node
|
||||||
.env
|
node_modules/
|
||||||
.env.*
|
|
||||||
*.pem
|
# Secrets. Unanchored like every entry above, so each matches at every
|
||||||
*.key
|
# depth. Matching is case-sensitive on Linux, so names use character
|
||||||
|
# ranges rather than a lowercase form that misses `Server.Key`.
|
||||||
|
|
||||||
|
# Environment files. `*.env` covers bare `.env` and the `prod.env`
|
||||||
|
# convention. Only the templates `example.env` and `sample.env` are
|
||||||
|
# re-included below. A repository that commits any other template adds
|
||||||
|
# its own negation at the end of this file, for example `!.env.example`.
|
||||||
|
*.[eE][nN][vV]
|
||||||
|
.[eE][nN][vV].*
|
||||||
|
.[eE][nN][vV][rR][cC]
|
||||||
|
!example.env
|
||||||
|
!sample.env
|
||||||
|
|
||||||
|
# Private keys and the bundles carrying them.
|
||||||
|
*.[pP][eE][mM]
|
||||||
|
*.[kK][eE][yY]
|
||||||
|
*.[pP]12
|
||||||
|
*.[pP][fF][xX]
|
||||||
|
[iI][dD]_[rR][sS][aA]
|
||||||
|
[iI][dD]_[dD][sS][aA]
|
||||||
|
[iI][dD]_[eE][cC][dD][sS][aA]
|
||||||
|
[iI][dD]_[eE][cC][dD][sS][aA]_[sS][kK]
|
||||||
|
[iI][dD]_[eE][dD]25519
|
||||||
|
[iI][dD]_[eE][dD]25519_[sS][kK]
|
||||||
|
|
||||||
|
# This repository's own entries, such as its build outputs, go below
|
||||||
|
# this comment, and a re-vendor keeps them. Anchor a binary built at the
|
||||||
|
# root: `/myapp`, never `myapp`, which also ignores `cmd/myapp/`.
|
||||||
|
|
||||||
|
# Go
|
||||||
|
*.log
|
||||||
|
*.out
|
||||||
|
*.test
|
||||||
|
|
||||||
# Dependencies
|
# Dependencies
|
||||||
vendor/
|
vendor/
|
||||||
node_modules/
|
|
||||||
|
|
||||||
# Build output
|
# Build output
|
||||||
/bin/
|
/bin/
|
||||||
|
|||||||
@@ -17,6 +17,7 @@ linters:
|
|||||||
disable:
|
disable:
|
||||||
# Genuinely incompatible with project patterns
|
# Genuinely incompatible with project patterns
|
||||||
- exhaustruct # Requires all struct fields
|
- exhaustruct # Requires all struct fields
|
||||||
|
- exhaustruct_v5 # Requires all struct fields (successor to exhaustruct)
|
||||||
- godot # Requires comments to end with periods
|
- godot # Requires comments to end with periods
|
||||||
- wrapcheck # Too verbose for internal packages
|
- wrapcheck # Too verbose for internal packages
|
||||||
- varnamelen # Short names like db, id are idiomatic Go
|
- varnamelen # Short names like db, id are idiomatic Go
|
||||||
@@ -24,6 +25,8 @@ linters:
|
|||||||
# silenced by disabling that name, not by enabling the successor.
|
# silenced by disabling that name, not by enabling the successor.
|
||||||
- wsl # Deprecated, replaced by wsl_v5
|
- wsl # Deprecated, replaced by wsl_v5
|
||||||
- gomodguard # Deprecated, replaced by gomodguard_v2
|
- gomodguard # Deprecated, replaced by gomodguard_v2
|
||||||
|
# Misses findings at random in v2.14.0; back once a pinned release fixes it
|
||||||
|
- canonicalheader
|
||||||
settings:
|
settings:
|
||||||
lll:
|
lll:
|
||||||
line-length: 88
|
line-length: 88
|
||||||
|
|||||||
@@ -1,7 +1,2 @@
|
|||||||
node_modules/
|
node_modules/
|
||||||
yarn.lock
|
yarn.lock
|
||||||
|
|
||||||
# A byte-for-byte copy of the one in sneak/prompts.
|
|
||||||
REPO_POLICIES.md
|
|
||||||
|
|
||||||
vendor/
|
|
||||||
|
|||||||
+48
-30
@@ -1,7 +1,12 @@
|
|||||||
|
# Every stage declares ARG CACHEBUST first: script/ passes a new value
|
||||||
|
# on every build, which reruns every RUN step after it. The steps that
|
||||||
|
# compile mount Go's build cache, which survives from build to build.
|
||||||
|
|
||||||
# Lint phase. script/lint builds it alone. The linter is run directly:
|
# Lint phase. script/lint builds it alone. The linter is run directly:
|
||||||
# `make lint` and script/lint are themselves a docker build.
|
# `make lint` and script/lint are themselves a docker build.
|
||||||
# golangci/golangci-lint:v2.12.2, 2026-10-04
|
# golangci/golangci-lint:v2.14.0, 2026-10-15
|
||||||
FROM golangci/golangci-lint@sha256:5cceeef04e53efe1470638d4b4b4f5ceefd574955ab3941b2d9a68a8c9ad5240 AS lint
|
FROM golangci/golangci-lint@sha256:ad862ba6b3798cbe0fd9fd7408d498fd74fbd2623a92406b2fd3898faf0bf98f AS lint
|
||||||
|
ARG CACHEBUST
|
||||||
|
|
||||||
# The linter compiles every package, and govips needs the libvips
|
# The linter compiles every package, and govips needs the libvips
|
||||||
# headers for that. REPO_POLICIES.md has the lint phase install them
|
# headers for that. REPO_POLICIES.md has the lint phase install them
|
||||||
@@ -14,17 +19,20 @@ WORKDIR /src
|
|||||||
COPY go.mod go.sum ./
|
COPY go.mod go.sum ./
|
||||||
RUN go mod download
|
RUN go mod download
|
||||||
COPY . .
|
COPY . .
|
||||||
RUN golangci-lint run --config .golangci.yml ./...
|
RUN --mount=type=cache,target=/root/.cache/go-build \
|
||||||
|
golangci-lint run --config .golangci.yml ./...
|
||||||
|
|
||||||
# Test phase. script/test builds it alone.
|
# Test phase. script/test builds it alone. -race needs cgo and so a C
|
||||||
# golang:1.25.4-alpine3.22, 2026-02-25; the runtime stage uses Alpine 3.22 too
|
# compiler, which the Debian Go image ships and the alpine one does not.
|
||||||
FROM golang:1.25.4-alpine3.22@sha256:d3f0cf7723f3429e3f9ed846243970b20a2de7bae6a5b66fc5914e228d831bbb AS test
|
# golang:1.25.4-trixie, 2026-10-15
|
||||||
|
FROM golang@sha256:a02d35efc036053fdf0da8c15919276bf777a80cbfda6a35c5e9f087e652adfc AS test
|
||||||
|
ARG CACHEBUST
|
||||||
|
|
||||||
WORKDIR /src
|
WORKDIR /src
|
||||||
|
|
||||||
# script/bootstrap --cgo installs the build dependencies (a C compiler,
|
# script/bootstrap --cgo installs the build dependencies (the libvips
|
||||||
# the libvips and libheif headers, and libvips' JPEG XL support, which
|
# and libheif headers, and libvips' JPEG XL support, which the tests
|
||||||
# the tests need) and downloads the Go modules.
|
# need) and downloads the Go modules.
|
||||||
COPY script/ ./script/
|
COPY script/ ./script/
|
||||||
COPY go.mod go.sum ./
|
COPY go.mod go.sum ./
|
||||||
RUN script/bootstrap --cgo
|
RUN script/bootstrap --cgo
|
||||||
@@ -32,11 +40,14 @@ RUN script/bootstrap --cgo
|
|||||||
COPY . .
|
COPY . .
|
||||||
|
|
||||||
# Without -v first; on a failure, again with -v for the details, and
|
# Without -v first; on a failure, again with -v for the details, and
|
||||||
# the step fails even if the second run passes. -parallel 4: by default
|
# the step fails even if the second run passes. -count=1 because the
|
||||||
# a package runs as many of its tests at once as the host has CPUs, and
|
# mounted cache also holds test results, which go test would otherwise
|
||||||
# on a busy host they then wait so long to be scheduled that a test
|
# replay. -parallel 4: by default a package runs as many of its tests at
|
||||||
# that times a request can see it return a second late.
|
# once as the host has CPUs, and on a busy host they then wait so long
|
||||||
RUN go test -count=1 -timeout 90s -race -parallel 4 -cover ./... || \
|
# to be scheduled that a test that times a request can see it return a
|
||||||
|
# second late.
|
||||||
|
RUN --mount=type=cache,target=/root/.cache/go-build \
|
||||||
|
go test -count=1 -timeout 90s -race -parallel 4 -cover ./... || \
|
||||||
{ echo "--- Rerunning with -v for details ---"; \
|
{ echo "--- Rerunning with -v for details ---"; \
|
||||||
go test -count=1 -timeout 90s -race -parallel 4 -v ./...; exit 1; }
|
go test -count=1 -timeout 90s -race -parallel 4 -v ./...; exit 1; }
|
||||||
|
|
||||||
@@ -45,38 +56,44 @@ RUN go test -count=1 -timeout 90s -race -parallel 4 -cover ./... || \
|
|||||||
# test passed.
|
# test passed.
|
||||||
# golang:1.25.4-alpine3.22, 2026-02-25; the runtime stage uses Alpine 3.22 too
|
# golang:1.25.4-alpine3.22, 2026-02-25; the runtime stage uses Alpine 3.22 too
|
||||||
FROM golang:1.25.4-alpine3.22@sha256:d3f0cf7723f3429e3f9ed846243970b20a2de7bae6a5b66fc5914e228d831bbb AS builder
|
FROM golang:1.25.4-alpine3.22@sha256:d3f0cf7723f3429e3f9ed846243970b20a2de7bae6a5b66fc5914e228d831bbb AS builder
|
||||||
|
ARG CACHEBUST
|
||||||
|
|
||||||
COPY --from=lint /src/go.sum /dev/null
|
COPY --from=lint /src/go.sum /dev/null
|
||||||
COPY --from=test /src/go.sum /dev/null
|
COPY --from=test /src/go.sum /dev/null
|
||||||
|
|
||||||
WORKDIR /src
|
WORKDIR /src
|
||||||
|
|
||||||
# Build dependencies and Go modules, as in the test phase
|
# Build dependencies, git among them, and Go modules, as in the test
|
||||||
|
# phase
|
||||||
COPY script/ ./script/
|
COPY script/ ./script/
|
||||||
COPY go.mod go.sum ./
|
COPY go.mod go.sum ./
|
||||||
RUN script/bootstrap --cgo
|
RUN script/bootstrap --cgo
|
||||||
|
# A tar-stream context keeps the sender's file owners, which git refuses.
|
||||||
|
RUN git config --system --add safe.directory /src
|
||||||
|
|
||||||
# Copy source code
|
# Copy source code
|
||||||
COPY . .
|
COPY . .
|
||||||
|
|
||||||
# VERSION is declared here, not earlier: a new value reruns only the
|
# The VERSION build arg when one is given, otherwise
|
||||||
# build, not script/bootstrap. Given none, the version is
|
# `git describe --tags --always` on the .git in the build context: the
|
||||||
# `git describe --tags --always` of the .git in the build context (git
|
# tag on a tagged commit, tag-N-gHASH after one, the short commit when
|
||||||
# comes from script/bootstrap): the tag on a tagged commit, tag-N-gHASH
|
# no tag is reachable. With .git present, a version that is still empty,
|
||||||
# after one, the short commit when no tag is reachable. A context that
|
# dev or unknown fails the build: git is missing or could not read the
|
||||||
# carries .git and still yields no version fails the build; one without
|
# checkout. A context without .git, as from a source tarball, stamps an
|
||||||
# .git, as from a source tarball, stamps an empty version. CGO stays
|
# empty version. CGO stays enabled for govips; -trimpath keeps build
|
||||||
# enabled for govips; -trimpath keeps build paths out of the binary, and
|
# paths out of the binary, and -s -w leave out the symbol table and
|
||||||
# -s -w leave out the symbol table and debug information.
|
# debug information.
|
||||||
ARG VERSION
|
ARG VERSION
|
||||||
RUN version="${VERSION:-$(git describe --tags --always)}"; \
|
RUN --mount=type=cache,target=/root/.cache/go-build \
|
||||||
if [ -e .git ] && { [ -z "$version" ] || [ "$version" = dev ] || \
|
VERSION="${VERSION:-$(git describe --tags --always)}"; \
|
||||||
[ "$version" = unknown ]; }; then \
|
if [ -e .git ]; then \
|
||||||
echo "the build context carries .git but yields no version" >&2; \
|
case "$VERSION" in ""|dev|unknown) \
|
||||||
exit 1; \
|
echo "version is '$VERSION' although .git is present" >&2; \
|
||||||
|
exit 1 ;; \
|
||||||
|
esac; \
|
||||||
fi; \
|
fi; \
|
||||||
CGO_ENABLED=1 GOTOOLCHAIN=auto go build -trimpath \
|
CGO_ENABLED=1 GOTOOLCHAIN=auto go build -trimpath \
|
||||||
-ldflags "-s -w -X main.Version=${version}" \
|
-ldflags="-s -w -X main.Version=${VERSION}" \
|
||||||
-o /pixad ./cmd/pixad
|
-o /pixad ./cmd/pixad
|
||||||
|
|
||||||
# Runtime stage, and the last one: a plain `docker build .` builds this
|
# Runtime stage, and the last one: a plain `docker build .` builds this
|
||||||
@@ -85,6 +102,7 @@ RUN version="${VERSION:-$(git describe --tags --always)}"; \
|
|||||||
# the libvips and musl it was built with.
|
# the libvips and musl it was built with.
|
||||||
# alpine:3.22, 2026-10-08
|
# alpine:3.22, 2026-10-08
|
||||||
FROM alpine:3.22@sha256:5291449c3df73caf6ed85e649dec1b9e818b39a5d8c871e97afc13e9cd5e8fa8
|
FROM alpine:3.22@sha256:5291449c3df73caf6ed85e649dec1b9e818b39a5d8c871e97afc13e9cd5e8fa8
|
||||||
|
ARG CACHEBUST
|
||||||
|
|
||||||
# Install runtime dependencies only. vips-jxl is libvips' JPEG XL
|
# Install runtime dependencies only. vips-jxl is libvips' JPEG XL
|
||||||
# support, without which pixad does not start.
|
# support, without which pixad does not start.
|
||||||
|
|||||||
@@ -131,6 +131,34 @@ What the [upaas](https://git.eeqj.de/sneak/upaas) app for pixa needs:
|
|||||||
the cache can grow to three quarters or more of the space they all have
|
the cache can grow to three quarters or more of the space they all have
|
||||||
left, and to all of it when that is 500 MiB or less.
|
left, and to all of it when that is 500 MiB or less.
|
||||||
- the rest are in the table under Configuration below
|
- the rest are in the table under Configuration below
|
||||||
|
- **Memory:** pixa can use up to `max_concurrent_processing` times 4.2 GiB for
|
||||||
|
the images it processes at once, plus `upstream_connections` times
|
||||||
|
`upstream_max_response_size` for the source images it has fetched that wait
|
||||||
|
for processing. 4.2 GiB is the most resident memory was measured to grow while
|
||||||
|
pixa, with pixad's libvips settings, processed one 8192x8192 source, the
|
||||||
|
largest it accepts, saving it at full size to each output format in turn, one
|
||||||
|
at a time. The sources were a JPEG, a GIF, a WebP, an 8-bit and a 16-bit PNG,
|
||||||
|
an 8-bit and a 12-bit AVIF, and an 8-bit, a 16-bit and a 32-bit floating-point
|
||||||
|
JPEG XL; memory grew most for the 16-bit JPEG XL, of random pixels, saved as
|
||||||
|
WebP. At the defaults that is half the CPUs pixa sees times 4.2 GiB, plus 64
|
||||||
|
times 50 MiB: on a host where pixa sees 48 CPUs, 24 times 4.2 GiB plus 3.1
|
||||||
|
GiB, about 104 GiB. Set `PIXA_MAX_CONCURRENT_PROCESSING` and
|
||||||
|
`PIXA_UPSTREAM_CONNECTIONS` so that this fits the memory the container may
|
||||||
|
use.
|
||||||
|
- **Temporary files:** while pixa processes a large source, libvips writes its
|
||||||
|
decoded pixels to a file in `/tmp`. The file disappears from `/tmp` once it is
|
||||||
|
written, so a listing of `/tmp` does not show it, but it keeps its space, on
|
||||||
|
disk or, where `/tmp` is a tmpfs, in memory, until the image is done. For a
|
||||||
|
GIF, WebP, AVIF or JPEG XL source the decoded pixels are held in memory as
|
||||||
|
well as written to that file, and the 4.2 GiB above includes that memory. For
|
||||||
|
an 8192x8192 source the file was measured on pixad at 192 MiB for an 8-bit RGB
|
||||||
|
JPEG, 512 MiB for a 16-bit RGBA PNG or JPEG XL, and 1 GiB for a JPEG XL of
|
||||||
|
32-bit floating-point RGBA, the largest measured: up to 1 GiB per image
|
||||||
|
processed at once, 24 GiB at the defaults on a host where pixa sees 48 CPUs.
|
||||||
|
This `/tmp` is in the container's own filesystem, not on the volume at
|
||||||
|
`/var/lib/pixa`, and `cache_max_bytes` does not count it. The 4.2 GiB above
|
||||||
|
does not include these files; where `/tmp` is a tmpfs, they are memory on top
|
||||||
|
of it.
|
||||||
- **Health check:** the image's `HEALTHCHECK` requests
|
- **Health check:** the image's `HEALTHCHECK` requests
|
||||||
`/.well-known/healthcheck.json`. upaas reads the container's health 60 seconds
|
`/.well-known/healthcheck.json`. upaas reads the container's health 60 seconds
|
||||||
after a deploy and marks the deploy failed unless it is `healthy`. The probe
|
after a deploy and marks the deploy failed unless it is `healthy`. The probe
|
||||||
@@ -566,7 +594,7 @@ tell. With no file, pixa uses the environment and the defaults.
|
|||||||
| `PIXA_ALLOW_HTTP` | `allow_http` | Allow plain-HTTP upstreams, for testing only; default `false` |
|
| `PIXA_ALLOW_HTTP` | `allow_http` | Allow plain-HTTP upstreams, for testing only; default `false` |
|
||||||
| `PIXA_UPSTREAM_CONNECTIONS_PER_HOST` | `upstream_connections_per_host` | Concurrent connections per upstream host; default `20` |
|
| `PIXA_UPSTREAM_CONNECTIONS_PER_HOST` | `upstream_connections_per_host` | Concurrent connections per upstream host; default `20` |
|
||||||
| `PIXA_UPSTREAM_CONNECTIONS` | `upstream_connections` | Concurrent connections to all upstream hosts together; default `64` |
|
| `PIXA_UPSTREAM_CONNECTIONS` | `upstream_connections` | Concurrent connections to all upstream hosts together; default `64` |
|
||||||
| `PIXA_MAX_CONCURRENT_PROCESSING` | `max_concurrent_processing` | Images processed at once; default the number of CPUs |
|
| `PIXA_MAX_CONCURRENT_PROCESSING` | `max_concurrent_processing` | Images processed at once; default half the CPUs, at least 1 |
|
||||||
| `PIXA_IMAGE_REQUESTS_PER_MINUTE` | `image_requests_per_minute` | Image requests per client per minute; `0` turns it off; default `600` |
|
| `PIXA_IMAGE_REQUESTS_PER_MINUTE` | `image_requests_per_minute` | Image requests per client per minute; `0` turns it off; default `600` |
|
||||||
| `PIXA_UPSTREAM_FETCH_TIMEOUT` | `upstream_fetch_timeout` | Time allowed for one fetch from an upstream host; default `30s` |
|
| `PIXA_UPSTREAM_FETCH_TIMEOUT` | `upstream_fetch_timeout` | Time allowed for one fetch from an upstream host; default `30s` |
|
||||||
| `PIXA_UPSTREAM_MAX_RESPONSE_SIZE` | `upstream_max_response_size` | Largest upstream response accepted, in bytes; default 50 MiB |
|
| `PIXA_UPSTREAM_MAX_RESPONSE_SIZE` | `upstream_max_response_size` | Largest upstream response accepted, in bytes; default 50 MiB |
|
||||||
@@ -652,10 +680,12 @@ Key settings in more detail:
|
|||||||
does, and `downstream_timeout` has not ended first, the request is answered
|
does, and `downstream_timeout` has not ended first, the request is answered
|
||||||
503 with the error `server busy, try again later`
|
503 with the error `server busy, try again later`
|
||||||
- `max_concurrent_processing` — the most images decoded and encoded at once;
|
- `max_concurrent_processing` — the most images decoded and encoded at once;
|
||||||
default the number of CPUs pixa can use (`GOMAXPROCS`), which follows a
|
default half the CPUs pixa can use (`GOMAXPROCS`, which follows a container's
|
||||||
container's CPU limit. A request that finds all of them in use waits up to 10
|
CPU limit), rounded down, and at least 1. A value that is not a whole number
|
||||||
seconds for one to free up; if none does, and `downstream_timeout` has not
|
of at least 1 aborts startup. A request that finds all of them in use waits up
|
||||||
ended first, it is answered 503 the same way
|
to 10 seconds for one to free up; if none does, and `downstream_timeout` has
|
||||||
|
not ended first, it is answered 503 the same way. Each one can take a lot of
|
||||||
|
memory: see "Running under upaas"
|
||||||
- `image_requests_per_minute` — how many requests one client may send to the
|
- `image_requests_per_minute` — how many requests one client may send to the
|
||||||
image routes per minute (see the limit on them under Routes); default `600`.
|
image routes per minute (see the limit on them under Routes); default `600`.
|
||||||
`0` turns the limit off, and a negative or non-integer value aborts startup.
|
`0` turns the limit off, and a negative or non-integer value aborts startup.
|
||||||
@@ -685,7 +715,9 @@ See `configs/config.example.yml` for all options with defaults.
|
|||||||
`metrics.username` and `metrics.password` are set. A request's `handler` label
|
`metrics.username` and `metrics.password` are set. A request's `handler` label
|
||||||
is the pattern of the route it matches, such as `/v1/image/*`, never its path.
|
is the pattern of the route it matches, such as `/v1/image/*`, never its path.
|
||||||
Requests that match no route share the label `unmatched`: those the router
|
Requests that match no route share the label `unmatched`: those the router
|
||||||
answers 404 or 405, and CORS preflights
|
answers 404 or 405, and CORS preflights. A request's `method` label is its
|
||||||
|
method when that is `GET`, `HEAD`, `POST`, `PUT`, `PATCH`, `DELETE`,
|
||||||
|
`OPTIONS`, `CONNECT` or `TRACE`, and `other` for any other method
|
||||||
- **Logging**: stdlib slog
|
- **Logging**: stdlib slog
|
||||||
|
|
||||||
## Entrypoints
|
## Entrypoints
|
||||||
@@ -704,9 +736,9 @@ provide:
|
|||||||
install-precommit)
|
install-precommit)
|
||||||
- `script/projectname` — output the project name ("pixa")
|
- `script/projectname` — output the project name ("pixa")
|
||||||
- `script/test` — run the test suite: build the `test` phase of the
|
- `script/test` — run the test suite: build the `test` phase of the
|
||||||
`Dockerfile`, tagged `pixa-test`
|
`Dockerfile`, writing no image
|
||||||
- `script/lint` — run golangci-lint: build the `lint` phase of the `Dockerfile`,
|
- `script/lint` — run golangci-lint: build the `lint` phase of the `Dockerfile`,
|
||||||
tagged `pixa-lint`; the linter never runs on the host
|
writing no image; the linter never runs on the host
|
||||||
- `script/fmt` — format the Go code with gofmt and the markdown with prettier
|
- `script/fmt` — format the Go code with gofmt and the markdown with prettier
|
||||||
(writes)
|
(writes)
|
||||||
- `script/fmt-check` — check the same formatting (read-only), on the host
|
- `script/fmt-check` — check the same formatting (read-only), on the host
|
||||||
@@ -724,15 +756,20 @@ provide:
|
|||||||
- `script/install-precommit` — install the git pre-commit hook that runs
|
- `script/install-precommit` — install the git pre-commit hook that runs
|
||||||
`script/precommit`
|
`script/precommit`
|
||||||
|
|
||||||
Every `docker build` in these scripts passes `--no-cache`, so the lint and test
|
Every `docker build` in these scripts passes a new value of the `CACHEBUST`
|
||||||
phases run on every build instead of coming from the build cache.
|
build argument, which every stage of the `Dockerfile` declares first, so every
|
||||||
|
`RUN` step, the lint and test phases included, runs on every build instead of
|
||||||
|
coming from the build cache. Go's build cache is kept from build to build: the
|
||||||
|
steps that compile mount it, so a package whose inputs are unchanged is not
|
||||||
|
compiled again, and both `go test` runs pass `-count=1` so that no stored test
|
||||||
|
result is replayed. The `test` phase runs on the Debian Go image, with Debian's
|
||||||
|
libvips and libheif; the image pixa ships is built and runs on Alpine.
|
||||||
`script/check`, `script/cibuild`, `script/docker`, `script/lint`, `script/test`,
|
`script/check`, `script/cibuild`, `script/docker`, `script/lint`, `script/test`,
|
||||||
`script/setup` and `script/install-precommit` are the standard copies from
|
`script/setup` and `script/install-precommit` are the standard copies from
|
||||||
`sneak/prompts`, kept identical to them. `script/fmt` and `script/fmt-check` are
|
`sneak/prompts`, kept identical to them. `script/fmt` and `script/fmt-check` are
|
||||||
the standard copies with pixa's `gofmt` step kept before prettier. prettier
|
the standard copies with pixa's `gofmt` step kept before prettier. prettier
|
||||||
formats the markdown only: not the HTML templates, as it cannot parse a Go
|
formats the markdown only, not the HTML templates, as it cannot parse a Go
|
||||||
template action inside a tag, and not `REPO_POLICIES.md` (see
|
template action inside a tag.
|
||||||
`.prettierignore`), a copy of the one in `sneak/prompts`.
|
|
||||||
|
|
||||||
## Load Test
|
## Load Test
|
||||||
|
|
||||||
|
|||||||
+258
-105
@@ -1,6 +1,6 @@
|
|||||||
---
|
---
|
||||||
title: Repository Policies
|
title: Repository Policies
|
||||||
last_modified: 2026-09-08
|
last_modified: 2026-10-15
|
||||||
---
|
---
|
||||||
|
|
||||||
This document covers repository structure, tooling, and workflow standards. Code
|
This document covers repository structure, tooling, and workflow standards. Code
|
||||||
@@ -54,34 +54,39 @@ style conventions are in separate documents:
|
|||||||
`cibuild`. `script/bootstrap` installs all dependencies idempotently and
|
`cibuild`. `script/bootstrap` installs all dependencies idempotently and
|
||||||
assumes nothing is present: base tools come from nix, apt, brew, or apk
|
assumes nothing is present: base tools come from nix, apt, brew, or apk
|
||||||
(detected in that order; apt runs noninteractive). For node it uses the
|
(detected in that order; apt runs noninteractive). For node it uses the
|
||||||
installed node if present; otherwise it installs a PINNED node version via
|
installed node only when its major version is the pinned one; otherwise (node
|
||||||
nvm, first installing nvm itself if missing — from a hash-verified GitHub
|
missing, or another major, such as the node 24 the CI runner image ships) it
|
||||||
release archive (never `curl | sh`), with bash installed as an explicit
|
installs the PINNED node version via nvm and installs yarn and the packages
|
||||||
prerequisite since nvm requires bash. yarn is then pinned via
|
under it. nvm itself is installed first if missing, from a hash-verified
|
||||||
`corepack prepare yarn@<version> --activate`. Never install "latest" or "lts";
|
GitHub release archive (never `curl | sh`), with bash installed as an explicit
|
||||||
always exact versions. `script/cibuild` runs the CI build: it changes to the
|
prerequisite since nvm requires bash. Only the major version is compared
|
||||||
repo root, runs `script/bootstrap`, runs `script/check`, and builds the image
|
because the `Dockerfile` stages start from a node image whose exact version is
|
||||||
with the version; the Gitea workflow calls it. **`script/cibuild` runs
|
not the pin, and nvm cannot install a prebuilt node on alpine. yarn is pinned
|
||||||
|
via `corepack prepare yarn@<version> --activate`. Never install "latest" or
|
||||||
|
"lts"; always exact versions. `script/cibuild` runs the CI build: it changes
|
||||||
|
to the repo root, runs `script/bootstrap`, runs `script/check`, and builds the
|
||||||
|
image with the version; the Gitea workflow calls it. **`script/cibuild` runs
|
||||||
`script/bootstrap` first**, because the workflow checks out the repo and runs
|
`script/bootstrap` first**, because the workflow checks out the repo and runs
|
||||||
nothing else, while `script/fmt-check` runs the formatter on the host: on a
|
nothing else, while `script/fmt-check` runs the formatter on the host: on a
|
||||||
pristine checkout with nothing installed the run dies there, after the
|
pristine checkout with nothing installed the run dies there, after the
|
||||||
containerised gates have passed. **The bootstrap alone is not enough**:
|
containerised gates have passed. **The bootstrap alone is not enough**: when
|
||||||
`script/bootstrap` installs node and yarn under nvm and leaves neither on the
|
`script/bootstrap` installs node and yarn under nvm it leaves neither on the
|
||||||
`PATH` of the shell that called it, so a bare `yarn` still exits 127. The host
|
`PATH` of the shell that called it, so a bare `yarn` either exits 127 or runs
|
||||||
entrypoints that need yarn — `script/fmt` and `script/fmt-check` — therefore
|
under another node. The host entrypoints that need yarn — `script/fmt` and
|
||||||
source nvm for the pinned node version before invoking it, exactly as
|
`script/fmt-check` — therefore use the same test as `script/bootstrap`: when
|
||||||
`script/bootstrap`'s own install step does. A runner carrying nothing but
|
the node on `PATH` has the pinned major version they run the `yarn` on `PATH`,
|
||||||
docker and git then gets through `script/check`. Four further scripts are our
|
and otherwise they load nvm through `$HOME/.nvm/nvm.sh` and run yarn under the
|
||||||
own extensions to the standard: `script/check` runs `script/test`,
|
pinned node. A runner carrying nothing but docker and git then gets through
|
||||||
`script/lint` and `script/fmt-check`; `script/precommit` is what the git
|
`script/check`. Four further scripts are our own extensions to the standard:
|
||||||
pre-commit hook runs, and it calls `script/check`; `script/install-precommit`
|
`script/check` runs `script/test`, `script/lint` and `script/fmt-check`;
|
||||||
installs the git pre-commit hook (the `make hooks` target shims to it); and
|
`script/precommit` is what the git pre-commit hook runs, and it calls
|
||||||
`script/projectname` (literally that filename) simply outputs the project's
|
`script/check`; `script/install-precommit` installs the git pre-commit hook
|
||||||
name. Scripts that need the name call `script/projectname` — e.g.
|
(the `make hooks` target shims to it); and `script/projectname` (literally
|
||||||
`script/docker` assembles its image tag from it — so those scripts stay
|
that filename) simply outputs the project's name. Scripts that need the name
|
||||||
byte-identical across all repos. Repo-type-specific pre-commit extras (e.g.
|
call `script/projectname` — e.g. `script/docker` assembles its image tag from
|
||||||
`go mod tidy` verification in Go repos) belong in `script/precommit`, not in
|
it — so those scripts stay byte-identical across all repos. Repo-type-specific
|
||||||
the hook itself. Model scripts are at
|
pre-commit extras (e.g. `go mod tidy` verification in Go repos) belong in
|
||||||
|
`script/precommit`, not in the hook itself. Model scripts are at
|
||||||
`https://git.eeqj.de/sneak/prompts/raw/branch/main/script/<name>`. The README
|
`https://git.eeqj.de/sneak/prompts/raw/branch/main/script/<name>`. The README
|
||||||
must document the provided scripts in an **Entrypoints** section (see the
|
must document the provided scripts in an **Entrypoints** section (see the
|
||||||
README requirements below).
|
README requirements below).
|
||||||
@@ -104,18 +109,24 @@ style conventions are in separate documents:
|
|||||||
`lint` phase and a `test` phase, with the final stage depending on both so the
|
`lint` phase and a `test` phase, with the final stage depending on both so the
|
||||||
image cannot be built unless they pass. For non-server repos the final stage
|
image cannot be built unless they pass. For non-server repos the final stage
|
||||||
brings up a development environment; for server repos it is the runtime image.
|
brings up a development environment; for server repos it is the runtime image.
|
||||||
Dockerfiles install development prerequisites by running `script/bootstrap`
|
The gate phases and the build stage start from their pinned base images and
|
||||||
rather than duplicating installs inline; COPY `script/` and the dependency
|
install what those images lack either inline, as the canonical Go `Dockerfile`
|
||||||
manifests (`package.json` + `yarn.lock`, `go.mod` + `go.sum`, etc.) before
|
below does for `git`, or by running `script/bootstrap`, as the `prompts`
|
||||||
running it.
|
repo's own `Dockerfile` does for its yarn packages. The development
|
||||||
|
environment stage installs development prerequisites by running
|
||||||
|
`script/bootstrap` rather than duplicating its installs inline. A stage that
|
||||||
|
runs `script/bootstrap` COPYs `script/` and the dependency manifests
|
||||||
|
(`package.json` + `yarn.lock`, `go.mod` + `go.sum`, etc.) before running it.
|
||||||
|
|
||||||
- **Linting and testing run in Docker, as phases of the `Dockerfile`.** There is
|
- **Linting and testing run in Docker, as phases of the `Dockerfile`.** There is
|
||||||
no separate lint file. `script/lint` and `script/test` each build one phase
|
no separate lint file. `script/lint` and `script/test` each build one phase
|
||||||
and nothing else:
|
and nothing else:
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
docker build --no-cache --target lint -t "$(script/projectname)-lint" .
|
docker build --build-arg CACHEBUST="$(date +%s)-$$" \
|
||||||
docker build --no-cache --target test -t "$(script/projectname)-test" .
|
--target lint --output type=cacheonly .
|
||||||
|
docker build --build-arg CACHEBUST="$(date +%s)-$$" \
|
||||||
|
--target test --output type=cacheonly .
|
||||||
```
|
```
|
||||||
|
|
||||||
**A stage that is not the last one in the file is built only when the final
|
**A stage that is not the last one in the file is built only when the final
|
||||||
@@ -125,10 +136,15 @@ style conventions are in separate documents:
|
|||||||
plain `docker build .` builds the last stage alone and exits 0 having linted
|
plain `docker build .` builds the last stage alone and exits 0 having linted
|
||||||
and tested nothing.
|
and tested nothing.
|
||||||
|
|
||||||
**Every `docker build` in `script/` is tagged**, here and in
|
**The gate builds write no image.** With `--output type=cacheonly` the phase
|
||||||
`script/cibuild` and `script/docker`. An untagged build leaves a dangling
|
runs and a failing step fails the build, but the result is not exported.
|
||||||
image behind on every invocation, on every developer host and every CI
|
Nothing uses those images, and writing one out is slow: a Go test phase's
|
||||||
runner; a tagged one replaces the previous image.
|
image holds the toolchain. A build given neither `--output` nor `-t` writes
|
||||||
|
an untagged image and leaves it dangling, on every developer host and every
|
||||||
|
CI runner. `script/cibuild` and `script/docker` build the image that ships
|
||||||
|
and tag it, so each build replaces the previous image; each assigns the tag
|
||||||
|
on its own line before the build, so `set -e` stops it where
|
||||||
|
`script/projectname` fails.
|
||||||
|
|
||||||
Inside a phase the tool is invoked directly — `golangci-lint`, `go test`,
|
Inside a phase the tool is invoked directly — `golangci-lint`, `go test`,
|
||||||
`eslint`, `prettier` — never through `make lint` or `script/test`, which are
|
`eslint`, `prettier` — never through `make lint` or `script/test`, which are
|
||||||
@@ -143,66 +159,107 @@ style conventions are in separate documents:
|
|||||||
one's findings, and a host-global lock in `$TMPDIR` makes concurrent runs
|
one's findings, and a host-global lock in `$TMPDIR` makes concurrent runs
|
||||||
exit non-zero with `parallel golangci-lint is running` — a status a caller
|
exit non-zero with `parallel golangci-lint is running` — a status a caller
|
||||||
cannot tell from real findings. Both have produced wrong verdicts in this
|
cannot tell from real findings. Both have produced wrong verdicts in this
|
||||||
org, in both directions. A container has its own cache, its own `TMPDIR` and
|
org, in both directions. A container has its own golangci-lint cache, its
|
||||||
a digest-pinned binary, so neither is reachable.
|
own `TMPDIR` and a digest-pinned binary, so neither is reachable.
|
||||||
|
|
||||||
- **Any build that runs checks is built with `--no-cache`.** Docker invalidates
|
- **Any build that runs checks is given a new `CACHEBUST` value.** Docker
|
||||||
a `COPY` layer only when the copied content changes, so on an unchanged tree
|
invalidates a `COPY` layer only when the copied content changes, so on an
|
||||||
the check `RUN` is served from cache, nothing executes, and the build still
|
unchanged tree the check `RUN` is served from cache, nothing executes, and the
|
||||||
exits 0. Every `docker build` in `script/` therefore passes `--no-cache`:
|
build still exits 0. Every `docker build` in `script/` therefore passes
|
||||||
`script/lint`, `script/test`, `script/cibuild` and `script/docker` are the
|
`--build-arg CACHEBUST="$(date +%s)-$$"`, the time and the script's process
|
||||||
four, and there is no fifth — `script/check` runs the two gate phases and
|
id, so no two builds share a value, not even the three in one
|
||||||
`script/fmt-check`, and builds no image of its own. A bare `docker build .` is
|
`script/cibuild`. `script/lint`, `script/test`, `script/cibuild` and
|
||||||
not evidence that anything ran: a sub-second build reporting success is a
|
`script/docker` are the four, and there is no fifth — `script/check` runs the
|
||||||
cache hit, not a result. Never invalidate by pruning — `docker builder prune`
|
two gate phases and `script/fmt-check`, and builds no image of its own. Every
|
||||||
and friends destroy a build cache shared with every other build on the host.
|
stage of the `Dockerfile` declares `ARG CACHEBUST` as its first line after
|
||||||
|
`FROM`, and a new value reruns every `RUN` step after that line; `COPY` steps
|
||||||
|
are compared by content and need nothing. A stage without the `ARG` line is
|
||||||
|
served from cache on an unchanged tree, so a repository re-vendors the scripts
|
||||||
|
and adds the `ARG` lines in one commit. The scripts do not pass `--no-cache`:
|
||||||
|
it reruns every step too, but it also empties every cache mount, and Go's
|
||||||
|
build cache, which the canonical Go `Dockerfile` below mounts so that compiled
|
||||||
|
packages survive between check builds, would start empty every time. A bare
|
||||||
|
`docker build .` is not evidence that anything ran: a sub-second build
|
||||||
|
reporting success is a cache hit, not a result. Never invalidate by pruning —
|
||||||
|
`docker builder prune` and friends destroy a build cache shared with every
|
||||||
|
other build on the host. When a check is added or changed, prove it works by
|
||||||
|
planting a defect it must catch and watching the run fail on it, then revert
|
||||||
|
the defect. A green run alone shows neither that the check ran nor that it
|
||||||
|
covers what it should.
|
||||||
|
|
||||||
- **The gate phases are separate stages, and the build stage depends on both.**
|
- **The gate phases are separate stages, and the build stage depends on both.**
|
||||||
The lint phase is based on the `golangci/golangci-lint` image (pinned by
|
The lint phase is based on the `golangci/golangci-lint` image (pinned by
|
||||||
hash), so lint failures surface in seconds rather than after a full compile,
|
hash), so lint failures surface in seconds rather than after a full compile,
|
||||||
and the test phase is based on the Go image. The canonical Go repo
|
and the test phase is based on the Debian Go image. The canonical Go repo
|
||||||
`Dockerfile`:
|
`Dockerfile`:
|
||||||
|
|
||||||
```dockerfile
|
```dockerfile
|
||||||
|
# Every stage declares ARG CACHEBUST first: script/ passes a new value
|
||||||
|
# on every build, which reruns every RUN step after it. The steps that
|
||||||
|
# compile mount Go's build cache, which survives from build to build.
|
||||||
|
|
||||||
# Lint phase
|
# Lint phase
|
||||||
# golangci/golangci-lint:v2.x.x, YYYY-MM-DD
|
# golangci/golangci-lint:v2.x.x, YYYY-MM-DD
|
||||||
FROM golangci/golangci-lint@sha256:... AS lint
|
FROM golangci/golangci-lint@sha256:... AS lint
|
||||||
|
ARG CACHEBUST
|
||||||
WORKDIR /src
|
WORKDIR /src
|
||||||
COPY go.mod go.sum ./
|
COPY go.mod go.sum ./
|
||||||
RUN go mod download
|
RUN go mod download
|
||||||
COPY . .
|
COPY . .
|
||||||
RUN golangci-lint run --config .golangci.yml ./...
|
RUN --mount=type=cache,target=/root/.cache/go-build \
|
||||||
|
golangci-lint run --config .golangci.yml ./...
|
||||||
|
|
||||||
# Test phase
|
# Test phase. -race needs cgo and so a C compiler, which the Debian Go
|
||||||
# golang:1.x-alpine, YYYY-MM-DD
|
# image ships and the alpine one does not. -count=1 because the mounted
|
||||||
|
# cache also holds test results, which go test would otherwise replay.
|
||||||
|
# golang:1.x, YYYY-MM-DD
|
||||||
FROM golang@sha256:... AS test
|
FROM golang@sha256:... AS test
|
||||||
|
ARG CACHEBUST
|
||||||
WORKDIR /src
|
WORKDIR /src
|
||||||
COPY go.mod go.sum ./
|
COPY go.mod go.sum ./
|
||||||
RUN go mod download
|
RUN go mod download
|
||||||
COPY . .
|
COPY . .
|
||||||
RUN go test -timeout 90s -race -cover ./... || \
|
RUN --mount=type=cache,target=/root/.cache/go-build \
|
||||||
|
go test -count=1 -timeout 90s -race -cover ./... || \
|
||||||
{ echo "--- Rerunning with -v for details ---"; \
|
{ echo "--- Rerunning with -v for details ---"; \
|
||||||
go test -timeout 90s -race -v ./...; exit 1; }
|
go test -count=1 -timeout 90s -race -v ./...; exit 1; }
|
||||||
|
|
||||||
# Build stage. Nothing is wanted from either phase above; the copies
|
# Build stage. Nothing is wanted from either phase above; the copies
|
||||||
# are what make BuildKit build them first, so this stage cannot run
|
# are what make BuildKit build them first, so this stage cannot run
|
||||||
# unless lint and test passed.
|
# unless lint and test passed.
|
||||||
# golang:1.x-alpine, YYYY-MM-DD
|
# golang:1.x-alpine, YYYY-MM-DD
|
||||||
FROM golang@sha256:... AS builder
|
FROM golang@sha256:... AS builder
|
||||||
|
ARG CACHEBUST
|
||||||
COPY --from=lint /src/go.sum /dev/null
|
COPY --from=lint /src/go.sum /dev/null
|
||||||
COPY --from=test /src/go.sum /dev/null
|
COPY --from=test /src/go.sum /dev/null
|
||||||
|
RUN apk add --no-cache git
|
||||||
|
# A tar-stream context keeps the sender's file owners, which git refuses.
|
||||||
|
RUN git config --system --add safe.directory /src
|
||||||
WORKDIR /src
|
WORKDIR /src
|
||||||
COPY go.mod go.sum ./
|
COPY go.mod go.sum ./
|
||||||
RUN go mod download
|
RUN go mod download
|
||||||
COPY . .
|
COPY . .
|
||||||
|
|
||||||
ARG VERSION=dev
|
# The VERSION build arg when one is given, otherwise
|
||||||
RUN CGO_ENABLED=0 go build -trimpath \
|
# `git describe --tags --always` on the .git in the build context. With
|
||||||
-ldflags="-s -w -X main.Version=${VERSION}" \
|
# .git present, a version that is still empty, dev or unknown fails the
|
||||||
-o /app ./cmd/app/
|
# build: git is missing or could not read the checkout.
|
||||||
|
ARG VERSION
|
||||||
|
RUN --mount=type=cache,target=/root/.cache/go-build \
|
||||||
|
VERSION="${VERSION:-$(git describe --tags --always)}"; \
|
||||||
|
if [ -e .git ]; then \
|
||||||
|
case "$VERSION" in ""|dev|unknown) \
|
||||||
|
echo "version is '$VERSION' although .git is present" >&2; \
|
||||||
|
exit 1 ;; \
|
||||||
|
esac; \
|
||||||
|
fi; \
|
||||||
|
CGO_ENABLED=0 go build -trimpath \
|
||||||
|
-ldflags="-s -w -X main.Version=${VERSION}" \
|
||||||
|
-o /app ./cmd/app/
|
||||||
|
|
||||||
# Runtime stage, and the last one
|
# Runtime stage, and the last one
|
||||||
FROM alpine@sha256:...
|
FROM alpine@sha256:...
|
||||||
|
ARG CACHEBUST
|
||||||
COPY --from=builder /app /usr/local/bin/app
|
COPY --from=builder /app /usr/local/bin/app
|
||||||
ENTRYPOINT ["app"]
|
ENTRYPOINT ["app"]
|
||||||
```
|
```
|
||||||
@@ -217,23 +274,88 @@ style conventions are in separate documents:
|
|||||||
- Keep the runtime stage last, and if you add a stage after it, give it the
|
- Keep the runtime stage last, and if you add a stage after it, give it the
|
||||||
same two copies. A plain `docker build .` builds the last stage's chain
|
same two copies. A plain `docker build .` builds the last stage's chain
|
||||||
and nothing else.
|
and nothing else.
|
||||||
|
- Only Go's build cache is mounted (`/root/.cache/go-build`, where Go keeps
|
||||||
|
it for root), and only on the steps that compile: `golangci-lint run`,
|
||||||
|
both `go test` runs and `go build`. It keeps compiled packages between
|
||||||
|
check builds, and Go reuses one only when all its inputs are unchanged, so
|
||||||
|
it saves the compile without skipping a check. Nothing else is kept:
|
||||||
|
golangci-lint's own cache would replay stored findings in place of a lint
|
||||||
|
run, and Go builds from whatever files a kept module cache holds without
|
||||||
|
checking them again (only `go mod verify` does), so a file changed there
|
||||||
|
would go unnoticed. Go's build cache also holds test results, which is why
|
||||||
|
both `go test` runs carry `-count=1`.
|
||||||
- If the project uses `//go:embed` directives that reference build artifacts
|
- If the project uses `//go:embed` directives that reference build artifacts
|
||||||
(e.g. a web frontend compiled in a separate stage), the lint phase must
|
(e.g. a web frontend compiled in a separate stage), the lint phase must
|
||||||
create placeholder files so the embed directives resolve. Example:
|
create placeholder files so the embed directives resolve. Example:
|
||||||
`RUN mkdir -p web/dist && touch web/dist/index.html web/dist/style.css`.
|
`RUN mkdir -p web/dist && touch web/dist/index.html web/dist/style.css`.
|
||||||
- If the project requires CGO or system libraries for linting (e.g.
|
- If the project requires CGO or system libraries for linting, install them
|
||||||
`vips-dev`), install them in the lint phase with `apk add`.
|
in the lint phase. The `golangci/golangci-lint` image is Debian-based and
|
||||||
- `ARG VERSION=dev` is declared in the stage that compiles and supplied by
|
has no `apk`, so install with `apt-get` under the Debian package name
|
||||||
`script/docker` and `script/cibuild`; no stage may call `git describe`.
|
(`libvips-dev`, where alpine says `vips-dev`), and delete the package
|
||||||
|
lists in the same `RUN`, so the layer does not keep them:
|
||||||
|
|
||||||
|
```dockerfile
|
||||||
|
RUN apt-get update \
|
||||||
|
&& apt-get install -y --no-install-recommends libvips-dev \
|
||||||
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
```
|
||||||
|
|
||||||
|
- `.dockerignore` lets `.git` into the build context. It keeps out every git
|
||||||
|
`config` at any depth (`**/.git/config`, `**/.git/modules/**/config`): the
|
||||||
|
repository's own, each submodule's under `.git/modules/`, and that of a
|
||||||
|
submodule keeping its own `.git` directory. `git describe` does not need
|
||||||
|
them, and each can hold a credential: a password in a remote URL, or the
|
||||||
|
token the CI checkout step stores there. A submodule whose name has a
|
||||||
|
`config` segment (`config`, `deploy/config`, `config/lib`) loses its whole
|
||||||
|
git directory to `**/.git/modules/**/config`, and Go's version stamping
|
||||||
|
then fails the build: give it a name without that segment
|
||||||
|
(`git submodule add --name`). The stage that compiles has `git` (the
|
||||||
|
Debian Go image has it; an alpine one needs `apk add --no-cache git`) and
|
||||||
|
takes the version from the `VERSION` build argument when one is given,
|
||||||
|
otherwise from `git describe --tags --always`. That gives the tag on a
|
||||||
|
tagged commit; on a later commit, the tag, the number of commits since it
|
||||||
|
and the short commit (`v1.2.3-4-gabc1234`); and the short commit when no
|
||||||
|
tag is reachable. The stage that compiles also marks its working directory
|
||||||
|
safe for git (`git config --system --add safe.directory /src`): a context
|
||||||
|
sent as a tar stream keeps the sender's file owners, and git refuses a
|
||||||
|
checkout owned by another user, so the version would come out empty.
|
||||||
|
`ARG VERSION` has no default, and the build fails if the context carries
|
||||||
|
`.git` and the version still comes out empty, `dev` or `unknown`. A plain
|
||||||
|
`docker build .` with no build arguments must succeed; a Dockerfile that
|
||||||
|
refuses an empty build argument drops that refusal and keeps the argument.
|
||||||
|
A checkout whose `.git` is a file (a linked worktree, or a repository
|
||||||
|
checked out as a submodule) is the exception: that file points to a git
|
||||||
|
directory outside the build context, so the build cannot read the version
|
||||||
|
and a plain `docker build .` fails; pass the version with
|
||||||
|
`--build-arg VERSION=...`, as `script/docker` and `script/cibuild` already
|
||||||
|
do.
|
||||||
|
|
||||||
- Every repo should have a Gitea Actions workflow (`.gitea/workflows/`) that
|
- Every repo should have a Gitea Actions workflow (`.gitea/workflows/`) that
|
||||||
runs `script/cibuild` on push, and checks out the repo as its only other step.
|
runs `script/cibuild` on push, and checks out the repo as its only other step,
|
||||||
That script bootstraps, runs the gate phases, and then builds the image, so a
|
with `persist-credentials: false`: `script/cibuild` needs no token, and
|
||||||
successful run means every check passed; a bare `docker build .` does not
|
without it the checkout leaves the job's token in `.git/config` for every
|
||||||
|
later step. The checkout step also sets `fetch-depth: 0`, which fetches the
|
||||||
|
tags `git describe` needs: by default it clones shallow with no tags, and a
|
||||||
|
tagged repository's CI build would stamp a bare short commit id. The
|
||||||
|
workflow's `concurrency` block groups runs by workflow and branch
|
||||||
|
(`${{ github.workflow }}-${{ github.ref }}`) with `cancel-in-progress: true`,
|
||||||
|
so a new push cancels the older run on the same branch, queued or running, and
|
||||||
|
no other: runs for replaced commits do not hold up the shared runner.
|
||||||
|
`script/cibuild` bootstraps, runs the gate phases, and then builds the image,
|
||||||
|
so a successful run means every check passed; a bare `docker build .` does not
|
||||||
carry the same guarantee, because its gate phases may come from the cache. The
|
carry the same guarantee, because its gate phases may come from the cache. The
|
||||||
image build is uncached and so runs the gate phases a second time. That is the
|
image build gets a `CACHEBUST` value of its own and so runs the gate phases a
|
||||||
price of the rule above, and it is worth paying: the image that ships is built
|
second time. That is the price of the rule above, and it is worth paying: the
|
||||||
from a run of its own gates rather than from a cache entry.
|
image that ships is built from a run of its own gates rather than from a cache
|
||||||
|
entry. The `check` job sets `timeout-minutes: 20`, so a hung build frees the
|
||||||
|
shared runner after 20 minutes. That allows for the three Docker builds
|
||||||
|
described above (the test phase, the lint phase, then the image), each held to
|
||||||
|
the 5-minute Docker build limit below, plus the bootstrap. A separate workflow
|
||||||
|
limited to `main` by a `branches` list under `on: push` cannot be checked by
|
||||||
|
review: to try a change to it, add the feature branch to that list and push,
|
||||||
|
then remove the branch from the list again before merging. Keep any job in it
|
||||||
|
that publishes behind `if: github.ref_name == 'main'`, so the run from the
|
||||||
|
feature branch publishes nothing.
|
||||||
|
|
||||||
- Use platform-standard formatters: `black` for Python, `prettier` for
|
- Use platform-standard formatters: `black` for Python, `prettier` for
|
||||||
JS/CSS/Markdown/HTML, `go fmt` for Go. Always use default configuration with
|
JS/CSS/Markdown/HTML, `go fmt` for Go. Always use default configuration with
|
||||||
@@ -286,17 +408,18 @@ style conventions are in separate documents:
|
|||||||
```
|
```
|
||||||
|
|
||||||
`-count=1` is required on both invocations: it defeats Go's test _result_
|
`-count=1` is required on both invocations: it defeats Go's test _result_
|
||||||
cache, so the target cannot report a pass it did not earn, and the rerun
|
cache, so neither run can report a stored pass in place of running the
|
||||||
reproduces a failure instead of replaying it. It leaves the build cache
|
tests. It leaves the build cache alone, so it costs the runtime of the suite
|
||||||
alone, so it costs the runtime of the suite and no recompilation.
|
and no recompilation.
|
||||||
|
|
||||||
Note that this is a second, independent cache, stacked below the Docker
|
That cache is Go's own, separate from Docker's layer cache. Go stores a
|
||||||
layer cache that [issue #26](https://git.eeqj.de/sneak/prompts/issues/26)
|
passing result in its cache directory (`GOCACHE`), and when the same tests
|
||||||
addresses. `CHECK_EPOCH` guarantees the `RUN make test` _step_ re-executes;
|
run again on unchanged code it prints that result, marked `(cached)`,
|
||||||
it does not guarantee `go test` inside that step does any work, because the
|
without running them. The directory lasts from one run to the next on a
|
||||||
`GOCACHE` baked into earlier image layers survives into the re-executed
|
developer's machine, where this target runs, and in the `test` phase of the
|
||||||
step. They are two separate defects requiring two separate fixes, and a fix
|
`Dockerfile`, which mounts it to keep compiled packages between check
|
||||||
for one must not be recorded as covering the other.
|
builds, so the phase's `go test` runs carry `-count=1` too. A new
|
||||||
|
`CACHEBUST` value (above) is what makes that step run on an unchanged tree.
|
||||||
|
|
||||||
Python example:
|
Python example:
|
||||||
|
|
||||||
@@ -324,9 +447,12 @@ style conventions are in separate documents:
|
|||||||
|
|
||||||
- `.gitignore` should be comprehensive from the start: OS files (`.DS_Store`),
|
- `.gitignore` should be comprehensive from the start: OS files (`.DS_Store`),
|
||||||
editor files (`.swp`, `*~`), in-repo agent scratch directories (`.claude/`),
|
editor files (`.swp`, `*~`), in-repo agent scratch directories (`.claude/`),
|
||||||
language build artifacts, and `node_modules/`. Fetch the standard `.gitignore`
|
`node_modules/`, and the repo's own build outputs. Fetch the standard
|
||||||
from `https://git.eeqj.de/sneak/prompts/raw/branch/main/.gitignore` when
|
`.gitignore` from
|
||||||
setting up a new repo. These patterns are written to `.gitignore`'s own
|
`https://git.eeqj.de/sneak/prompts/raw/branch/main/.gitignore` when setting up
|
||||||
|
a new repo. A repo's `.gitignore` is the standard file followed by the repo's
|
||||||
|
own entries, such as its binaries; a re-vendor replaces the standard part and
|
||||||
|
keeps those entries. These patterns are written to `.gitignore`'s own
|
||||||
semantics, in which an unanchored pattern already matches at every depth; they
|
semantics, in which an unanchored pattern already matches at every depth; they
|
||||||
are not a `.dockerignore` and must not be transplanted into one unmodified.
|
are not a `.dockerignore` and must not be transplanted into one unmodified.
|
||||||
|
|
||||||
@@ -340,7 +466,7 @@ style conventions are in separate documents:
|
|||||||
— which is more dangerous than a short file with no secret patterns at all,
|
— which is more dangerous than a short file with no secret patterns at all,
|
||||||
because it reads as solved and stops anyone looking. Give every
|
because it reads as solved and stops anyone looking. Give every
|
||||||
depth-independent pattern the `**/` prefix and leave only genuinely
|
depth-independent pattern the `**/` prefix and leave only genuinely
|
||||||
root-anchored entries unprefixed: `.git`, and the repo's own host-built
|
root-anchored entries unprefixed: `.claude`, and the repo's own host-built
|
||||||
binary, written `/myapp` and never `**/myapp`, which would also match
|
binary, written `/myapp` and never `**/myapp`, which would also match
|
||||||
`cmd/myapp/` and delete the package directory from the context. Matching is
|
`cmd/myapp/` and delete the package directory from the context. Matching is
|
||||||
case-sensitive, and an ALL-CAPS twin per pattern still misses `Server.Key`, so
|
case-sensitive, and an ALL-CAPS twin per pattern still misses `Server.Key`, so
|
||||||
@@ -365,21 +491,24 @@ style conventions are in separate documents:
|
|||||||
directory, so a repo running agents in subdirectories still ships
|
directory, so a repo running agents in subdirectories still ships
|
||||||
`services/api/.claude/` and must add its own anchored entry there.
|
`services/api/.claude/` and must add its own anchored entry there.
|
||||||
|
|
||||||
- **Excluding `.git` means `git describe` cannot run inside any build stage, and
|
- **A plain `docker build .` of a clone stamps the version that
|
||||||
it fails quietly there.** In a build stage there is no repository, so
|
`git describe --tags --always` gives**, derived from the `.git` in the build
|
||||||
`git describe` writes nothing to stdout, `-X main.Version=` comes out empty,
|
context as the canonical `Dockerfile` above shows. Without its failure check,
|
||||||
the binary reports no version at all, and the build still exits 0. Compute the
|
a missing `git` or an unreadable checkout would leave `-X main.Version=` empty
|
||||||
version on the host and thread it in as a build arg. `script/docker` and
|
and the build would still exit 0. `script/docker` and `script/cibuild` pass
|
||||||
`script/cibuild` do this, byte-identically across repos:
|
the version they compute on the host; it takes precedence. They do this
|
||||||
|
byte-identically across repos:
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
# Own line: a failing command substitution inside an argument does not
|
# The version and the tag each get their own line: a failing command
|
||||||
# trip `set -e`, so the inline form degrades to an empty constant.
|
# substitution inside an argument does not trip `set -e`, so the inline
|
||||||
|
# form degrades to an empty constant.
|
||||||
version="$(git describe --tags --always --dirty 2>/dev/null || true)"
|
version="$(git describe --tags --always --dirty 2>/dev/null || true)"
|
||||||
[ -n "$version" ] || version="unknown"
|
[ -n "$version" ] || version="unknown"
|
||||||
docker build --no-cache \
|
tag="$(script/projectname)"
|
||||||
|
docker build --build-arg CACHEBUST="$(date +%s)-$$" \
|
||||||
--build-arg VERSION="$version" \
|
--build-arg VERSION="$version" \
|
||||||
-t "$(script/projectname)" .
|
-t "$tag" .
|
||||||
```
|
```
|
||||||
|
|
||||||
`--always` makes an untagged repo yield an abbreviated commit hash rather
|
`--always` makes an untagged repo yield an abbreviated commit hash rather
|
||||||
@@ -387,12 +516,12 @@ style conventions are in separate documents:
|
|||||||
fallback is applied — a live check that fires on a build from an export with
|
fallback is applied — a live check that fires on a build from an export with
|
||||||
no `.git` and on a repository with no commits yet. Do not fold it into the
|
no `.git` and on a repository with no commits yet. Do not fold it into the
|
||||||
substitution as `|| echo unknown`, which makes the guard unreachable. The
|
substitution as `|| echo unknown`, which makes the guard unreachable. The
|
||||||
Dockerfile's side is `ARG VERSION=dev` in the stage that compiles, declared
|
Dockerfile's side is `ARG VERSION` in the stage that compiles, declared
|
||||||
there because `ARG` is stage-scoped; passing `VERSION` to a repo whose
|
there because `ARG` is stage-scoped; passing `VERSION` to a repo whose
|
||||||
Dockerfile declares no such `ARG` is ignored and costs nothing, which is why
|
Dockerfile declares no such `ARG` is ignored and costs nothing, which is why
|
||||||
the scripts stay byte-identical. One consequence for CI: the standard
|
the scripts stay byte-identical. One consequence for CI: the standard
|
||||||
checkout action clones shallow and fetches no tags, so a repo that embeds a
|
checkout action clones shallow and fetches no tags, so the canonical
|
||||||
tag-derived version must set `fetch-depth: 0` on its checkout step.
|
`.gitea/workflows/check.yml` sets `fetch-depth: 0` on its checkout step.
|
||||||
|
|
||||||
- **Verify `.dockerignore` by enumerating the image, not by reading the
|
- **Verify `.dockerignore` by enumerating the image, not by reading the
|
||||||
patterns.** Plant files at the root _and_ at least two directories deep, build
|
patterns.** Plant files at the root _and_ at least two directories deep, build
|
||||||
@@ -426,12 +555,18 @@ style conventions are in separate documents:
|
|||||||
`test-support` depguard rule, where a repo names its own test-support packages
|
`test-support` depguard rule, where a repo names its own test-support packages
|
||||||
by full import path. A repo adds entries there and changes nothing else, and a
|
by full import path. A repo adds entries there and changes nothing else, and a
|
||||||
re-vendor carries its entries forward. The canonical golangci-lint version is
|
re-vendor carries its entries forward. The canonical golangci-lint version is
|
||||||
v2.12.2 (released 2026-05-06), pinned as the digest of the lint phase's base
|
v2.14.0 (released 2026-09-24), pinned as the digest of the lint phase's base
|
||||||
image
|
image
|
||||||
(`golangci/golangci-lint@sha256:5cceeef04e53efe1470638d4b4b4f5ceefd574955ab3941b2d9a68a8c9ad5240`,
|
(`golangci/golangci-lint@sha256:ad862ba6b3798cbe0fd9fd7408d498fd74fbd2623a92406b2fd3898faf0bf98f`,
|
||||||
which reports `2.12.2 built with go1.26.2 from c0d3ddc9`). That digest is the
|
which reports `2.14.0 built with go1.27.0 from 114493f9`). A module's `go`
|
||||||
only pin, since no repo installs golangci-lint on the host: bumping the
|
directive must not name a newer Go minor version than the one golangci-lint
|
||||||
version means changing it and nothing else.
|
was built with, or golangci-lint refuses to lint it: this release lints
|
||||||
|
`go 1.27.1` but not `go 1.28`. That digest is the only pin, since no repo
|
||||||
|
installs golangci-lint on the host. A repo sets the lint phase digest to the
|
||||||
|
one named here and re-vendors `.golangci.yml` in the same commit, whichever of
|
||||||
|
the two prompted the change: the canonical copy can name linters that an older
|
||||||
|
golangci-lint rejects, and a newer golangci-lint can add linters that
|
||||||
|
`default: all` switches on until the canonical copy disables them.
|
||||||
|
|
||||||
- **`script/bootstrap` installs a pinned tool by comparing versions, never by
|
- **`script/bootstrap` installs a pinned tool by comparing versions, never by
|
||||||
testing presence.** An `if ! command -v <tool>; then install; fi` guard tests
|
testing presence.** An `if ! command -v <tool>; then install; fi` guard tests
|
||||||
@@ -453,8 +588,18 @@ style conventions are in separate documents:
|
|||||||
function defined and never invoked has the same exit status and the same
|
function defined and never invoked has the same exit status and the same
|
||||||
empty output as one that worked.
|
empty output as one that worked.
|
||||||
|
|
||||||
|
Node is the exception to the whole-token comparison: the canonical
|
||||||
|
`script/bootstrap` compares only its major version, because the node 22
|
||||||
|
alpine image the `Dockerfile` stages start from is not the exact pin, and
|
||||||
|
nvm cannot install a prebuilt node on alpine.
|
||||||
|
|
||||||
Keep it POSIX sh: no arrays, no `[[`, no `grep -P`.
|
Keep it POSIX sh: no arrays, no `[[`, no `grep -P`.
|
||||||
|
|
||||||
|
A Go tool a repo needs on the host is installed with `go install` pinned to
|
||||||
|
a commit hash (`go install <package>@<commit hash>`). It is never tracked as
|
||||||
|
a `go.mod` tool dependency or through a `tools.go` file, either of which
|
||||||
|
pulls the tool's own dependencies into the repo's `go.mod` and `go.sum`.
|
||||||
|
|
||||||
- When pinning images or packages by hash, add a comment above the reference
|
- When pinning images or packages by hash, add a comment above the reference
|
||||||
with the version and date (YYYY-MM-DD).
|
with the version and date (YYYY-MM-DD).
|
||||||
|
|
||||||
@@ -564,13 +709,17 @@ style conventions are in separate documents:
|
|||||||
Never edit existing migrations after release.
|
Never edit existing migrations after release.
|
||||||
|
|
||||||
- All repos should have an `.editorconfig` enforcing the project's indentation
|
- All repos should have an `.editorconfig` enforcing the project's indentation
|
||||||
settings.
|
settings: the standard file from
|
||||||
|
`https://git.eeqj.de/sneak/prompts/raw/branch/main/.editorconfig`, which sets
|
||||||
|
tabs for `Makefile` and Go files, followed by the repo's own sections, such as
|
||||||
|
one for another language it uses. A re-vendor replaces the standard part and
|
||||||
|
keeps those sections.
|
||||||
|
|
||||||
- Avoid putting files in the repo root unless necessary. Root should contain
|
- Avoid putting files in the repo root unless necessary. Root should contain
|
||||||
only project-level config files (`README.md`, `Makefile`, `Dockerfile`,
|
only project-level config files (`README.md`, `AGENTS.md`, `Makefile`,
|
||||||
`LICENSE`, `.gitignore`, `.editorconfig`, `REPO_POLICIES.md`, and
|
`Dockerfile`, `LICENSE`, `.gitignore`, `.editorconfig`, `REPO_POLICIES.md`,
|
||||||
language-specific config). Everything else goes in a subdirectory. Canonical
|
and language-specific config). Everything else goes in a subdirectory.
|
||||||
subdirectory names:
|
Canonical subdirectory names:
|
||||||
- `bin/` — executable scripts and tools
|
- `bin/` — executable scripts and tools
|
||||||
- `cmd/` — Go command entrypoints; thin only: one `main.go` per binary whose
|
- `cmd/` — Go command entrypoints; thin only: one `main.go` per binary whose
|
||||||
body is a single call into `internal/` or `pkg/`, no project logic in
|
body is a single call into `internal/` or `pkg/`, no project logic in
|
||||||
@@ -601,3 +750,7 @@ style conventions are in separate documents:
|
|||||||
- Go: `go.mod`, `go.sum`, `.golangci.yml`
|
- Go: `go.mod`, `go.sum`, `.golangci.yml`
|
||||||
- JS: `package.json`, `yarn.lock`, `.prettierrc`, `.prettierignore`
|
- JS: `package.json`, `yarn.lock`, `.prettierrc`, `.prettierignore`
|
||||||
- Python: `pyproject.toml`
|
- Python: `pyproject.toml`
|
||||||
|
|
||||||
|
- Guidance for coding agents lives in one `AGENTS.md` at the repository root. It
|
||||||
|
is never committed under a file or directory named after one agent tool, such
|
||||||
|
as `CLAUDE.md` or `.claude/`, and never split into separate memory files.
|
||||||
|
|||||||
@@ -18,7 +18,7 @@
|
|||||||
|
|
||||||
pre-1.0. No git tags exist. The `1.0.0` milestone is in progress; work lands on
|
pre-1.0. No git tags exist. The `1.0.0` milestone is in progress; work lands on
|
||||||
`next`, and `main` receives only the milestone PR that the owner merges. `next`
|
`next`, and `main` receives only the milestone PR that the owner merges. `next`
|
||||||
is at the canonical `golangci-lint` v2.12.2 config and is green. Recent work
|
is at the canonical `golangci-lint` v2.14.0 config and is green. Recent work
|
||||||
extracted the internal/magic, internal/allowlist, internal/httpfetcher, and
|
extracted the internal/magic, internal/allowlist, internal/httpfetcher, and
|
||||||
internal/signature packages. The gosec findings from the 2026-07-06 survey are
|
internal/signature packages. The gosec findings from the 2026-07-06 survey are
|
||||||
resolved. The disk cache is now size-bounded with LRU eviction
|
resolved. The disk cache is now size-bounded with LRU eviction
|
||||||
@@ -30,6 +30,47 @@ P2: security: per-origin rate limiting
|
|||||||
|
|
||||||
# Completed Steps
|
# Completed Steps
|
||||||
|
|
||||||
|
- 2026-10-15 `max_concurrent_processing` defaults to half the CPUs pixa sees,
|
||||||
|
rounded down, and at least 1 (closes #242): it defaulted to one image per CPU,
|
||||||
|
and a container with no CPU limit sees every CPU of the host, so on a 48-CPU
|
||||||
|
host pixa processed 48 images at once, each of which can take about 4.2 GiB.
|
||||||
|
`config.DefaultMaxConcurrentProcessing` gives the default to the config and to
|
||||||
|
`imageprocessor.New`; `PIXA_MAX_CONCURRENT_PROCESSING` still overrides it.
|
||||||
|
"Running under upaas" in `README.md` gives the worst-case memory, measured,
|
||||||
|
and tells the operator to set `PIXA_MAX_CONCURRENT_PROCESSING` and
|
||||||
|
`PIXA_UPSTREAM_CONNECTIONS` to fit the container's memory. It also gives the
|
||||||
|
file libvips writes to `/tmp` for each large source, up to 1 GiB, measured,
|
||||||
|
which is outside `/var/lib/pixa` and `cache_max_bytes` and is memory where
|
||||||
|
`/tmp` is a tmpfs.
|
||||||
|
- 2026-10-15 the canonical files are re-vendored from `sneak/prompts` commit
|
||||||
|
`e3f79c4` (closes #221): `.dockerignore`, `.editorconfig`,
|
||||||
|
`.gitea/workflows/check.yml`, `.gitignore`, `.golangci.yml`,
|
||||||
|
`.prettierignore`, `.prettierrc`, `REPO_POLICIES.md` and the standard scripts
|
||||||
|
are its copies. pixa's own entries follow the standard part: `/bin`, `/data`
|
||||||
|
and the local config files in `.dockerignore`; Go's `*.log`, `*.out` and
|
||||||
|
`*.test`, `vendor/`, the binaries, `/data/`, `*.sqlite3` and the local config
|
||||||
|
files in `.gitignore`. The workflow keeps its `script/docker-smoke` step while
|
||||||
|
https://git.eeqj.de/sneak/pixa/issues/203 is open. The scripts pass a new
|
||||||
|
`CACHEBUST` build argument in place of `--no-cache`, and every `Dockerfile`
|
||||||
|
stage declares it first, so every `RUN` step still runs on every build while
|
||||||
|
Go's build cache, mounted on the steps that compile, is kept; both `go test`
|
||||||
|
runs keep `-count=1`. The lint and test builds write no image. The lint phase
|
||||||
|
runs golangci-lint v2.14.0, the test phase runs on the Debian Go image, and
|
||||||
|
the build stage marks `/src` safe for git. `script/bootstrap`, `script/fmt`
|
||||||
|
and `script/fmt-check` use the node on the `PATH` only when it has the pinned
|
||||||
|
major version. Of golangci-lint v2.14.0's two findings, an unused `nolint`
|
||||||
|
directive is removed, and gosec's new path check is marked as not applying to
|
||||||
|
the fixed config file locations. The agent guidance moves unchanged from
|
||||||
|
`CLAUDE.md` to a root `AGENTS.md`, where `REPO_POLICIES.md` keeps it.
|
||||||
|
- 2026-10-15 the request metrics record any method that is not a standard one as
|
||||||
|
`other` (closes #274): their `method` label was each request's method as sent,
|
||||||
|
and Go's HTTP server accepts any method name, so a client sending made-up
|
||||||
|
methods added series to the Prometheus registry that were never removed. The
|
||||||
|
label is now the request's method only for `GET`, `HEAD`, `POST`, `PUT`,
|
||||||
|
`PATCH`, `DELETE`, `OPTIONS`, `CONNECT` and `TRACE`. The metrics library reads
|
||||||
|
the method from the request it is handed, so the metrics middleware hands it a
|
||||||
|
copy of the request carrying the label; the handlers still see the method the
|
||||||
|
client sent.
|
||||||
- 2026-10-15 logging out ends every login session (closes #256): a login session
|
- 2026-10-15 logging out ends every login session (closes #256): a login session
|
||||||
was a cookie with nothing kept on the server, so `/logout` removed it from one
|
was a cookie with nothing kept on the server, so `/logout` removed it from one
|
||||||
browser and a copy of it kept working for its 30 days. Every login is made
|
browser and a copy of it kept working for its 30 days. Every login is made
|
||||||
|
|||||||
@@ -108,10 +108,11 @@ upstream_connections_per_host: 20
|
|||||||
# downstream_timeout has ended first.
|
# downstream_timeout has ended first.
|
||||||
upstream_connections: 64
|
upstream_connections: 64
|
||||||
|
|
||||||
# Maximum number of images decoded and encoded at once (default: the
|
# Maximum number of images decoded and encoded at once (default: half the
|
||||||
# number of CPUs pixa can use, which follows a container's CPU limit). A
|
# number of CPUs pixa can use, at least 1; the number of CPUs follows a
|
||||||
# request that finds none free waits up to 10 seconds for one, and if none
|
# container's CPU limit). A request that finds none free waits up to 10
|
||||||
# frees up it is answered 503, unless downstream_timeout has ended first.
|
# seconds for one, and if none frees up it is answered 503, unless
|
||||||
|
# downstream_timeout has ended first.
|
||||||
# max_concurrent_processing: 4
|
# max_concurrent_processing: 4
|
||||||
|
|
||||||
# How many requests to the image routes (/v1/image/ and /v1/e/) one client
|
# How many requests to the image routes (/v1/image/ and /v1/e/) one client
|
||||||
|
|||||||
@@ -1,6 +1,7 @@
|
|||||||
package config
|
package config
|
||||||
|
|
||||||
import (
|
import (
|
||||||
|
"fmt"
|
||||||
"runtime"
|
"runtime"
|
||||||
"testing"
|
"testing"
|
||||||
)
|
)
|
||||||
@@ -12,8 +13,8 @@ const (
|
|||||||
)
|
)
|
||||||
|
|
||||||
// TestOmittedConcurrencyLimitsUseDefaults checks that an omitted
|
// TestOmittedConcurrencyLimitsUseDefaults checks that an omitted
|
||||||
// max_concurrent_processing is the number of CPUs Go uses and an omitted
|
// max_concurrent_processing is half the CPUs Go uses, and at least 1, and
|
||||||
// upstream_connections is 64.
|
// an omitted upstream_connections is 64.
|
||||||
func TestOmittedConcurrencyLimitsUseDefaults(t *testing.T) {
|
func TestOmittedConcurrencyLimitsUseDefaults(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
@@ -22,9 +23,9 @@ func TestOmittedConcurrencyLimitsUseDefaults(t *testing.T) {
|
|||||||
t.Fatalf("minimal config should be valid, got error: %v", err)
|
t.Fatalf("minimal config should be valid, got error: %v", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
if c.MaxConcurrentProcessing != runtime.GOMAXPROCS(0) {
|
if c.MaxConcurrentProcessing != max(runtime.GOMAXPROCS(0)/2, 1) {
|
||||||
t.Errorf("MaxConcurrentProcessing = %d, want %d, one per CPU",
|
t.Errorf("MaxConcurrentProcessing = %d, want %d, half the CPUs",
|
||||||
c.MaxConcurrentProcessing, runtime.GOMAXPROCS(0))
|
c.MaxConcurrentProcessing, max(runtime.GOMAXPROCS(0)/2, 1))
|
||||||
}
|
}
|
||||||
|
|
||||||
if c.UpstreamConnections != 64 {
|
if c.UpstreamConnections != 64 {
|
||||||
@@ -32,6 +33,36 @@ func TestOmittedConcurrencyLimitsUseDefaults(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// TestOmittedMaxConcurrentProcessingIsHalfTheCPUsAndAtLeastOne checks
|
||||||
|
// that an omitted max_concurrent_processing is half the CPUs Go uses,
|
||||||
|
// rounded down, and 1 when Go uses one CPU.
|
||||||
|
//
|
||||||
|
//nolint:paralleltest // it sets GOMAXPROCS, which other tests read
|
||||||
|
func TestOmittedMaxConcurrentProcessingIsHalfTheCPUsAndAtLeastOne(t *testing.T) {
|
||||||
|
cpusBefore := runtime.GOMAXPROCS(0)
|
||||||
|
|
||||||
|
t.Cleanup(func() { runtime.GOMAXPROCS(cpusBefore) })
|
||||||
|
|
||||||
|
for _, tc := range []struct{ cpus, want int }{
|
||||||
|
{1, 1}, {2, 1}, {3, 1}, {8, 4},
|
||||||
|
} {
|
||||||
|
runtime.GOMAXPROCS(tc.cpus)
|
||||||
|
|
||||||
|
c, err := configFromYAML(t, signingKeyLine)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("minimal config should be valid, got error: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
t.Logf("GOMAXPROCS %d: MaxConcurrentProcessing %d",
|
||||||
|
tc.cpus, c.MaxConcurrentProcessing)
|
||||||
|
|
||||||
|
if c.MaxConcurrentProcessing != tc.want {
|
||||||
|
t.Errorf("GOMAXPROCS %d: MaxConcurrentProcessing = %d, want %d",
|
||||||
|
tc.cpus, c.MaxConcurrentProcessing, tc.want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// TestExplicitConcurrencyLimitsAreUsed checks that valid values for the
|
// TestExplicitConcurrencyLimitsAreUsed checks that valid values for the
|
||||||
// two limits are used as given.
|
// two limits are used as given.
|
||||||
func TestExplicitConcurrencyLimitsAreUsed(t *testing.T) {
|
func TestExplicitConcurrencyLimitsAreUsed(t *testing.T) {
|
||||||
@@ -160,3 +191,19 @@ func TestConcurrencyLimitsFromEnvironment(t *testing.T) {
|
|||||||
})
|
})
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// TestInvalidMaxConcurrentProcessingFromEnvironmentAbortsStartup checks
|
||||||
|
// that a PIXA_MAX_CONCURRENT_PROCESSING that is not a whole number of at
|
||||||
|
// least 1, an empty one included, aborts startup naming the variable and
|
||||||
|
// the key instead of taking the default.
|
||||||
|
func TestInvalidMaxConcurrentProcessingFromEnvironmentAbortsStartup(t *testing.T) {
|
||||||
|
for _, value := range []string{"", " ", "0.5", "half", "-1", "4 images"} {
|
||||||
|
t.Run(fmt.Sprintf("value %q", value), func(t *testing.T) {
|
||||||
|
t.Setenv(testMaxConcurrentProcessingVar, value)
|
||||||
|
|
||||||
|
_, err := configFromYAML(t, signingKeyLine)
|
||||||
|
wantStartupError(t, err,
|
||||||
|
testMaxConcurrentProcessingVar, keyMaxConcurrentProcessing)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
@@ -37,6 +37,14 @@ const (
|
|||||||
DefaultImageRequestsPerMinute = 600
|
DefaultImageRequestsPerMinute = 600
|
||||||
)
|
)
|
||||||
|
|
||||||
|
// DefaultMaxConcurrentProcessing returns the most images processed at once
|
||||||
|
// when max_concurrent_processing is omitted: half the CPUs Go uses
|
||||||
|
// (runtime.GOMAXPROCS(0), which follows a container's CPU limit), rounded
|
||||||
|
// down, and at least 1.
|
||||||
|
func DefaultMaxConcurrentProcessing() int {
|
||||||
|
return max(runtime.GOMAXPROCS(0)/2, 1) //nolint:mnd // half the CPUs
|
||||||
|
}
|
||||||
|
|
||||||
// Configuration key names.
|
// Configuration key names.
|
||||||
const (
|
const (
|
||||||
keyDebug = "debug"
|
keyDebug = "debug"
|
||||||
@@ -297,10 +305,8 @@ func newFromSmartConfig(sc *smartconfig.Config) (*Config, error) {
|
|||||||
keyUpstreamConnectionsPerHost, DefaultUpstreamConnectionsPerHost),
|
keyUpstreamConnectionsPerHost, DefaultUpstreamConnectionsPerHost),
|
||||||
UpstreamConnections: loader.intVal(
|
UpstreamConnections: loader.intVal(
|
||||||
keyUpstreamConnections, DefaultUpstreamConnections),
|
keyUpstreamConnections, DefaultUpstreamConnections),
|
||||||
// Decoding and encoding are CPU-bound, so the default is one image
|
|
||||||
// per CPU Go uses, which follows a container's CPU limit.
|
|
||||||
MaxConcurrentProcessing: loader.intVal(
|
MaxConcurrentProcessing: loader.intVal(
|
||||||
keyMaxConcurrentProcessing, runtime.GOMAXPROCS(0)),
|
keyMaxConcurrentProcessing, DefaultMaxConcurrentProcessing()),
|
||||||
UpstreamFetchTimeout: loader.durationVal(
|
UpstreamFetchTimeout: loader.durationVal(
|
||||||
keyUpstreamFetchTimeout, DefaultUpstreamFetchTimeout),
|
keyUpstreamFetchTimeout, DefaultUpstreamFetchTimeout),
|
||||||
UpstreamMaxResponseSize: loader.int64Val(
|
UpstreamMaxResponseSize: loader.int64Val(
|
||||||
@@ -823,6 +829,7 @@ func loadConfigFile(log *slog.Logger, appName string) (*smartconfig.Config, erro
|
|||||||
// one whose path runs through a file, such as under a HOME of
|
// one whose path runs through a file, such as under a HOME of
|
||||||
// /dev/null. One that cannot be read or does not parse is a
|
// /dev/null. One that cannot be read or does not parse is a
|
||||||
// fatal startup error.
|
// fatal startup error.
|
||||||
|
//nolint:gosec // G703: a fixed config location; HOME is the operator's
|
||||||
_, statErr := os.Stat(cleanPath)
|
_, statErr := os.Stat(cleanPath)
|
||||||
if errors.Is(statErr, fs.ErrNotExist) || errors.Is(statErr, syscall.ENOTDIR) {
|
if errors.Is(statErr, fs.ErrNotExist) || errors.Is(statErr, syscall.ENOTDIR) {
|
||||||
continue
|
continue
|
||||||
|
|||||||
@@ -7,11 +7,11 @@ import (
|
|||||||
"errors"
|
"errors"
|
||||||
"fmt"
|
"fmt"
|
||||||
"io"
|
"io"
|
||||||
"runtime"
|
|
||||||
"sync"
|
"sync"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
"github.com/davidbyttow/govips/v2/vips"
|
"github.com/davidbyttow/govips/v2/vips"
|
||||||
|
"sneak.berlin/go/pixa/internal/config"
|
||||||
)
|
)
|
||||||
|
|
||||||
// vipsOnce ensures vips is initialized exactly once.
|
// vipsOnce ensures vips is initialized exactly once.
|
||||||
@@ -20,11 +20,10 @@ import (
|
|||||||
var vipsOnce sync.Once
|
var vipsOnce sync.Once
|
||||||
|
|
||||||
// initVips initializes libvips with quiet logging, one worker thread per
|
// initVips initializes libvips with quiet logging, one worker thread per
|
||||||
// image and no operation cache. Process already works on one image per CPU
|
// image and no operation cache. Each request decodes different source
|
||||||
// by default, so more threads per image would only compete for the CPUs.
|
// bytes, so the operation cache would rarely be hit and would hold memory
|
||||||
// Each request decodes different source bytes, so the operation cache
|
// outside MaxConcurrentProcessing; repeated requests are served from
|
||||||
// would rarely be hit and would hold memory outside MaxConcurrentProcessing;
|
// pixa's disk cache instead.
|
||||||
// repeated requests are served from pixa's disk cache instead.
|
|
||||||
func initVips() {
|
func initVips() {
|
||||||
vipsOnce.Do(func() {
|
vipsOnce.Do(func() {
|
||||||
vips.LoggingSettings(nil, vips.LogLevelError)
|
vips.LoggingSettings(nil, vips.LogLevelError)
|
||||||
@@ -179,7 +178,7 @@ type Params struct {
|
|||||||
// If <= 0, DefaultMaxInputBytes is used.
|
// If <= 0, DefaultMaxInputBytes is used.
|
||||||
MaxInputBytes int64
|
MaxInputBytes int64
|
||||||
// MaxConcurrentProcessing is the most images processed at once.
|
// MaxConcurrentProcessing is the most images processed at once.
|
||||||
// If <= 0, the number of CPUs Go uses (runtime.GOMAXPROCS(0)) is used.
|
// If <= 0, config.DefaultMaxConcurrentProcessing() is used.
|
||||||
MaxConcurrentProcessing int
|
MaxConcurrentProcessing int
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -195,7 +194,7 @@ func New(params Params) *ImageProcessor {
|
|||||||
|
|
||||||
maxConcurrentProcessing := params.MaxConcurrentProcessing
|
maxConcurrentProcessing := params.MaxConcurrentProcessing
|
||||||
if maxConcurrentProcessing <= 0 {
|
if maxConcurrentProcessing <= 0 {
|
||||||
maxConcurrentProcessing = runtime.GOMAXPROCS(0)
|
maxConcurrentProcessing = config.DefaultMaxConcurrentProcessing()
|
||||||
}
|
}
|
||||||
|
|
||||||
return &ImageProcessor{
|
return &ImageProcessor{
|
||||||
|
|||||||
@@ -115,14 +115,14 @@ func waitForEntries(t *testing.T, entered <-chan struct{}, count int) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestNewDefaultsMaxConcurrentProcessingToCPUs(t *testing.T) {
|
func TestNewDefaultsMaxConcurrentProcessingToHalfTheCPUs(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
for _, limit := range []int{0, -1} {
|
for _, limit := range []int{0, -1} {
|
||||||
proc := New(Params{MaxConcurrentProcessing: limit})
|
proc := New(Params{MaxConcurrentProcessing: limit})
|
||||||
if got := cap(proc.processingSemaphore); got != runtime.GOMAXPROCS(0) {
|
if got := cap(proc.processingSemaphore); got != max(runtime.GOMAXPROCS(0)/2, 1) {
|
||||||
t.Errorf("MaxConcurrentProcessing %d: %d slots, want %d, one per CPU",
|
t.Errorf("MaxConcurrentProcessing %d: %d slots, want %d, half the CPUs",
|
||||||
limit, got, runtime.GOMAXPROCS(0))
|
limit, got, max(runtime.GOMAXPROCS(0)/2, 1))
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -132,6 +132,30 @@ func TestNewDefaultsMaxConcurrentProcessingToCPUs(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// TestNewDefaultIsHalfTheCPUsAndAtLeastOne checks that New, given no
|
||||||
|
// MaxConcurrentProcessing, makes one slot for every two CPUs Go uses,
|
||||||
|
// rounded down, and one slot when Go uses one CPU.
|
||||||
|
//
|
||||||
|
//nolint:paralleltest // it sets GOMAXPROCS, which other tests read
|
||||||
|
func TestNewDefaultIsHalfTheCPUsAndAtLeastOne(t *testing.T) {
|
||||||
|
cpusBefore := runtime.GOMAXPROCS(0)
|
||||||
|
|
||||||
|
t.Cleanup(func() { runtime.GOMAXPROCS(cpusBefore) })
|
||||||
|
|
||||||
|
for _, tc := range []struct{ cpus, want int }{
|
||||||
|
{1, 1}, {2, 1}, {3, 1}, {8, 4},
|
||||||
|
} {
|
||||||
|
runtime.GOMAXPROCS(tc.cpus)
|
||||||
|
|
||||||
|
got := cap(New(Params{}).processingSemaphore)
|
||||||
|
t.Logf("GOMAXPROCS %d: %d slots", tc.cpus, got)
|
||||||
|
|
||||||
|
if got != tc.want {
|
||||||
|
t.Errorf("GOMAXPROCS %d: %d slots, want %d", tc.cpus, got, tc.want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// TestProcessNeverExceedsMaxConcurrentProcessing starts more Process calls
|
// TestProcessNeverExceedsMaxConcurrentProcessing starts more Process calls
|
||||||
// than MaxConcurrentProcessing allows and holds the first ones inside
|
// than MaxConcurrentProcessing allows and holds the first ones inside
|
||||||
// Process until the test lets them go. No more than the limit may be
|
// Process until the test lets them go. No more than the limit may be
|
||||||
|
|||||||
@@ -78,10 +78,6 @@ func (c *Cache) computeDefaultMaxBytes(
|
|||||||
computed := spaceBytes / freeSpaceFractionDenominator * freeSpaceFractionNumerator
|
computed := spaceBytes / freeSpaceFractionDenominator * freeSpaceFractionNumerator
|
||||||
computed = min(computed, math.MaxInt64)
|
computed = min(computed, math.MaxInt64)
|
||||||
|
|
||||||
// gosec cannot see that min() above bounds computed, so it reads
|
|
||||||
// this conversion as potentially overflowing. It cannot: computed is
|
|
||||||
// at most math.MaxInt64 on every path here.
|
|
||||||
//nolint:gosec // G115: clamped to MaxInt64 by min above
|
|
||||||
limit := int64(computed)
|
limit := int64(computed)
|
||||||
limit = max(limit, DefaultCacheMaxBytesFloor)
|
limit = max(limit, DefaultCacheMaxBytesFloor)
|
||||||
|
|
||||||
|
|||||||
@@ -53,7 +53,7 @@ type ServiceConfig struct {
|
|||||||
// Allowlist is the list of hosts that don't require signatures
|
// Allowlist is the list of hosts that don't require signatures
|
||||||
Allowlist []string
|
Allowlist []string
|
||||||
// MaxConcurrentProcessing is the most images processed at once; zero
|
// MaxConcurrentProcessing is the most images processed at once; zero
|
||||||
// uses the image processor's default, one per CPU
|
// uses the image processor's default, half the CPUs
|
||||||
MaxConcurrentProcessing int
|
MaxConcurrentProcessing int
|
||||||
// Logger for logging
|
// Logger for logging
|
||||||
Logger *slog.Logger
|
Logger *slog.Logger
|
||||||
|
|||||||
@@ -220,7 +220,10 @@ func (s *Middleware) CORS() func(http.Handler) http.Handler {
|
|||||||
// pattern of the chi route a request matches, such as /v1/image/*, or
|
// pattern of the chi route a request matches, such as /v1/image/*, or
|
||||||
// "unmatched" for a request that matches none, so the routes fix how many
|
// "unmatched" for a request that matches none, so the routes fix how many
|
||||||
// values the label has. It is never the path, with which any client could
|
// values the label has. It is never the path, with which any client could
|
||||||
// add series to the registry without limit.
|
// add series to the registry without limit. Its method label is the
|
||||||
|
// request's method when that is a standard one, such as GET, and "other"
|
||||||
|
// for any other: Go's HTTP server accepts any method name, so a label per
|
||||||
|
// method would let a client add series the same way.
|
||||||
func (s *Middleware) Metrics() func(http.Handler) http.Handler {
|
func (s *Middleware) Metrics() func(http.Handler) http.Handler {
|
||||||
mdlw := ghmm.New(ghmm.Config{
|
mdlw := ghmm.New(ghmm.Config{
|
||||||
Recorder: metrics.NewRecorder(metrics.Config{
|
Recorder: metrics.NewRecorder(metrics.Config{
|
||||||
@@ -230,7 +233,19 @@ func (s *Middleware) Metrics() func(http.Handler) http.Handler {
|
|||||||
|
|
||||||
return func(next http.Handler) http.Handler {
|
return func(next http.Handler) http.Handler {
|
||||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
std.Handler(routePattern(r), mdlw, next).ServeHTTP(w, r)
|
// The metrics library takes the method label from the request
|
||||||
|
// it is handed, so it is handed a shallow copy of r whose
|
||||||
|
// method is the label. next is served r itself, through the
|
||||||
|
// response writer the library wraps to see the status and size.
|
||||||
|
labelled := r.WithContext(r.Context())
|
||||||
|
labelled.Method = methodLabel(r.Method)
|
||||||
|
|
||||||
|
serveNext := http.HandlerFunc(
|
||||||
|
func(wrapped http.ResponseWriter, _ *http.Request) {
|
||||||
|
next.ServeHTTP(wrapped, r)
|
||||||
|
})
|
||||||
|
|
||||||
|
std.Handler(routePattern(r), mdlw, serveNext).ServeHTTP(w, labelled)
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -275,6 +290,20 @@ func routePattern(r *http.Request) string {
|
|||||||
return pattern
|
return pattern
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// methodLabel returns method when it is one of the standard methods net/http
|
||||||
|
// names: GET, HEAD, POST, PUT, PATCH, DELETE, OPTIONS, CONNECT and TRACE.
|
||||||
|
// For any other method it returns "other".
|
||||||
|
func methodLabel(method string) string {
|
||||||
|
switch method {
|
||||||
|
case http.MethodGet, http.MethodHead, http.MethodPost, http.MethodPut,
|
||||||
|
http.MethodPatch, http.MethodDelete, http.MethodOptions,
|
||||||
|
http.MethodConnect, http.MethodTrace:
|
||||||
|
return method
|
||||||
|
default:
|
||||||
|
return "other"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// MetricsAuth returns a basic auth middleware for the metrics endpoint.
|
// MetricsAuth returns a basic auth middleware for the metrics endpoint.
|
||||||
func (s *Middleware) MetricsAuth() func(http.Handler) http.Handler {
|
func (s *Middleware) MetricsAuth() func(http.Handler) http.Handler {
|
||||||
return basicauth.New(
|
return basicauth.New(
|
||||||
|
|||||||
@@ -7,6 +7,7 @@ import (
|
|||||||
"net/http"
|
"net/http"
|
||||||
"net/http/httptest"
|
"net/http/httptest"
|
||||||
"net/url"
|
"net/url"
|
||||||
|
"slices"
|
||||||
"strings"
|
"strings"
|
||||||
"testing"
|
"testing"
|
||||||
|
|
||||||
@@ -314,6 +315,58 @@ func requestCountsByHandler(
|
|||||||
return counts
|
return counts
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// TestMetricsGivesMadeUpMethodsOneLabel checks that the metrics middleware
|
||||||
|
// gives requests with methods that are not standard ones a single method
|
||||||
|
// label, "other", and still serves each request with the method it was sent
|
||||||
|
// with. With a label per method, any client could add series to the
|
||||||
|
// registry without limit by making up a new method for each request.
|
||||||
|
func TestMetricsGivesMadeUpMethodsOneLabel(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
// The line /metrics shows once both made-up methods have been served.
|
||||||
|
const want = `http_request_duration_seconds_count{` +
|
||||||
|
`code="200",handler="unmatched",method="other",service=""} 2`
|
||||||
|
|
||||||
|
registry := prometheus.NewRegistry()
|
||||||
|
mw := &Middleware{
|
||||||
|
log: slog.Default(),
|
||||||
|
config: &config.Config{},
|
||||||
|
metricsRegistry: registry,
|
||||||
|
}
|
||||||
|
|
||||||
|
var served []string
|
||||||
|
|
||||||
|
handler := mw.Metrics()(http.HandlerFunc(
|
||||||
|
func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
served = append(served, r.Method)
|
||||||
|
|
||||||
|
w.WriteHeader(http.StatusOK)
|
||||||
|
}))
|
||||||
|
|
||||||
|
methods := []string{"FROBNICATE", "WIBBLE"}
|
||||||
|
for _, method := range methods {
|
||||||
|
handler.ServeHTTP(httptest.NewRecorder(), httptest.NewRequestWithContext(
|
||||||
|
t.Context(), method, "/", nil))
|
||||||
|
}
|
||||||
|
|
||||||
|
t.Logf("methods the handler was served: %v", served)
|
||||||
|
|
||||||
|
if !slices.Equal(served, methods) {
|
||||||
|
t.Errorf("methods the handler was served = %v, want %v",
|
||||||
|
served, methods)
|
||||||
|
}
|
||||||
|
|
||||||
|
rec := httptest.NewRecorder()
|
||||||
|
promhttp.HandlerFor(registry, promhttp.HandlerOpts{}).ServeHTTP(rec,
|
||||||
|
httptest.NewRequestWithContext(
|
||||||
|
t.Context(), http.MethodGet, "/metrics", nil))
|
||||||
|
|
||||||
|
if !strings.Contains(rec.Body.String(), want) {
|
||||||
|
t.Errorf("/metrics does not count both made-up methods as one; "+
|
||||||
|
"want the line %q in:\n%s", want, rec.Body.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// TestLoggingLeavesOutSubmittedSigningKey checks that a login, a POST /
|
// TestLoggingLeavesOutSubmittedSigningKey checks that a login, a POST /
|
||||||
// whose form carries the signing key, leaves no trace of the key in the
|
// whose form carries the signing key, leaves no trace of the key in the
|
||||||
// request's log line.
|
// request's log line.
|
||||||
|
|||||||
+33
-12
@@ -3,11 +3,13 @@
|
|||||||
# this repo. Idempotent: every install is guarded by a check so already
|
# this repo. Idempotent: every install is guarded by a check so already
|
||||||
# installed tools are skipped. Base tooling comes from nix, apt, brew,
|
# installed tools are skipped. Base tooling comes from nix, apt, brew,
|
||||||
# or apk (detected in that order); assumes NOTHING is present (not git,
|
# or apk (detected in that order); assumes NOTHING is present (not git,
|
||||||
# make, or go). Node is used directly if installed; otherwise it is
|
# make, or go). The installed node is used only when its major version
|
||||||
# installed at a pinned version via nvm (installing nvm itself first,
|
# is the pinned one; otherwise (node missing, or another major) the
|
||||||
# from a hash-verified release archive, never curl | sh). The linter is
|
# pinned version is installed via nvm (installing nvm itself first,
|
||||||
# never installed on the host: golangci-lint runs only in the lint phase
|
# from a hash-verified release archive, never curl | sh), and yarn and
|
||||||
# of the Dockerfile (see script/lint).
|
# the packages are installed under it. The linter is never installed on
|
||||||
|
# the host: golangci-lint runs only in the lint phase of the Dockerfile
|
||||||
|
# (see script/lint).
|
||||||
#
|
#
|
||||||
# script/bootstrap git, make, Go, and Node, Yarn and the
|
# script/bootstrap git, make, Go, and Node, Yarn and the
|
||||||
# prettier in yarn.lock for script/fmt and
|
# prettier in yarn.lock for script/fmt and
|
||||||
@@ -110,31 +112,50 @@ ensure_nvm() {
|
|||||||
rm -rf "$tmp"
|
rm -rf "$tmp"
|
||||||
}
|
}
|
||||||
|
|
||||||
|
# True when the node on PATH has the pinned major version. Only the
|
||||||
|
# major is compared, as REPO_POLICIES.md asks. Another major is not
|
||||||
|
# used: the pinned yarn 1 prints a deprecation warning under node 24.
|
||||||
|
# script/fmt and script/fmt-check carry this function unchanged, to run
|
||||||
|
# the yarn installed here.
|
||||||
|
node_is_pinned_major() {
|
||||||
|
if ! command -v node >/dev/null 2>&1; then return 1; fi
|
||||||
|
installed="$(node --version)"
|
||||||
|
installed="${installed#v}"
|
||||||
|
[ "${installed%%.*}" = "${NODE_VERSION%%.*}" ]
|
||||||
|
}
|
||||||
|
|
||||||
ensure_node() {
|
ensure_node() {
|
||||||
if ! missing node; then return 0; fi
|
if node_is_pinned_major; then
|
||||||
|
echo "bootstrap: using node $(node --version)"
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
ensure_nvm
|
ensure_nvm
|
||||||
nvm_sh "nvm install $NODE_VERSION"
|
nvm_sh "nvm install $NODE_VERSION"
|
||||||
|
nvm_sh "nvm use $NODE_VERSION >/dev/null && \
|
||||||
|
echo \"bootstrap: using node \$(node --version)\""
|
||||||
}
|
}
|
||||||
|
|
||||||
ensure_yarn() {
|
ensure_yarn() {
|
||||||
|
if ! node_is_pinned_major; then
|
||||||
|
nvm_sh "nvm use $NODE_VERSION >/dev/null && corepack enable && \
|
||||||
|
corepack prepare yarn@$YARN_VERSION --activate"
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
if ! missing yarn; then return 0; fi
|
if ! missing yarn; then return 0; fi
|
||||||
if ! missing corepack; then
|
if ! missing corepack; then
|
||||||
corepack enable
|
corepack enable
|
||||||
corepack prepare "yarn@$YARN_VERSION" --activate
|
corepack prepare "yarn@$YARN_VERSION" --activate
|
||||||
elif [ -s "$HOME/.nvm/nvm.sh" ]; then
|
|
||||||
nvm_sh "nvm use $NODE_VERSION >/dev/null && corepack enable && \
|
|
||||||
corepack prepare yarn@$YARN_VERSION --activate"
|
|
||||||
else
|
else
|
||||||
npm install -g "yarn@$YARN_VERSION"
|
npm install -g "yarn@$YARN_VERSION"
|
||||||
fi
|
fi
|
||||||
}
|
}
|
||||||
|
|
||||||
install_js_deps() {
|
install_js_deps() {
|
||||||
if missing yarn && [ -s "$HOME/.nvm/nvm.sh" ]; then
|
if node_is_pinned_major; then
|
||||||
|
yarn install --frozen-lockfile
|
||||||
|
else
|
||||||
nvm_sh "nvm use $NODE_VERSION >/dev/null && cd \"$ROOT\" && \
|
nvm_sh "nvm use $NODE_VERSION >/dev/null && cd \"$ROOT\" && \
|
||||||
yarn install --frozen-lockfile"
|
yarn install --frozen-lockfile"
|
||||||
else
|
|
||||||
yarn install --frozen-lockfile
|
|
||||||
fi
|
fi
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
+13
-10
@@ -2,9 +2,11 @@
|
|||||||
# script/cibuild: run the CI build. It bootstraps first: a CI runner
|
# script/cibuild: run the CI build. It bootstraps first: a CI runner
|
||||||
# checks out and runs this and nothing else, and script/fmt-check runs
|
# checks out and runs this and nothing else, and script/fmt-check runs
|
||||||
# the formatter on the host, which a pristine checkout cannot do.
|
# the formatter on the host, which a pristine checkout cannot do.
|
||||||
# --no-cache for the same reason as script/docker: the gate phases the
|
# A new CACHEBUST value reruns every RUN step, since a cached gate phase
|
||||||
# final stage depends on are RUN steps, and a cached one is a check that
|
# is a check that did not run; unlike --no-cache, it keeps Go's build cache.
|
||||||
# did not run.
|
# This relies on every stage of the Dockerfile declaring ARG CACHEBUST
|
||||||
|
# first; a stage without that line is served from cache on an unchanged
|
||||||
|
# tree, with no warning.
|
||||||
set -eu
|
set -eu
|
||||||
|
|
||||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
|
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
|
||||||
@@ -14,16 +16,17 @@ main() {
|
|||||||
cd "$ROOT"
|
cd "$ROOT"
|
||||||
"$SCRIPT_DIR/bootstrap"
|
"$SCRIPT_DIR/bootstrap"
|
||||||
"$SCRIPT_DIR/check"
|
"$SCRIPT_DIR/check"
|
||||||
# Own line: a failing command substitution inside an argument does
|
# The version and the tag each get their own line: a failing
|
||||||
# not trip `set -e`, so the inline form degrades silently to an
|
# command substitution inside an argument does not trip `set -e`,
|
||||||
# empty constant. VERSION is computed here because .dockerignore
|
# so the inline form degrades silently to an empty constant. The
|
||||||
# excludes .git, so `git describe` in a build stage yields an empty
|
# VERSION build argument takes precedence over the version a build
|
||||||
# version without failing.
|
# stage derives from the .git in the context.
|
||||||
version="$(git describe --tags --always --dirty 2>/dev/null || true)"
|
version="$(git describe --tags --always --dirty 2>/dev/null || true)"
|
||||||
[ -n "$version" ] || version="unknown"
|
[ -n "$version" ] || version="unknown"
|
||||||
docker build --no-cache \
|
tag="$("$SCRIPT_DIR/projectname")"
|
||||||
|
docker build --build-arg CACHEBUST="$(date +%s)-$$" \
|
||||||
--build-arg VERSION="$version" \
|
--build-arg VERSION="$version" \
|
||||||
-t "$("$SCRIPT_DIR/projectname")" .
|
-t "$tag" .
|
||||||
}
|
}
|
||||||
|
|
||||||
main "$@"
|
main "$@"
|
||||||
|
|||||||
+13
-9
@@ -1,8 +1,11 @@
|
|||||||
#!/bin/sh
|
#!/bin/sh
|
||||||
# script/docker: build the Docker image tagged with the project name.
|
# script/docker: build the Docker image tagged with the project name.
|
||||||
# Identical in all repos; the tag comes from script/projectname.
|
# Identical in all repos; the tag comes from script/projectname.
|
||||||
# --no-cache because the gate phases the final stage depends on are RUN
|
# A new CACHEBUST value reruns every RUN step, since a cached gate phase
|
||||||
# steps, and a cached one is a check that did not run.
|
# is a check that did not run; unlike --no-cache, it keeps Go's build cache.
|
||||||
|
# This relies on every stage of the Dockerfile declaring ARG CACHEBUST
|
||||||
|
# first; a stage without that line is served from cache on an unchanged
|
||||||
|
# tree, with no warning.
|
||||||
set -eu
|
set -eu
|
||||||
|
|
||||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
|
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
|
||||||
@@ -10,16 +13,17 @@ ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
|
|||||||
|
|
||||||
main() {
|
main() {
|
||||||
cd "$ROOT"
|
cd "$ROOT"
|
||||||
# Own line: a failing command substitution inside an argument does
|
# The version and the tag each get their own line: a failing
|
||||||
# not trip `set -e`, so the inline form degrades silently to an
|
# command substitution inside an argument does not trip `set -e`,
|
||||||
# empty constant. VERSION is computed here because .dockerignore
|
# so the inline form degrades silently to an empty constant. The
|
||||||
# excludes .git, so `git describe` in a build stage yields an empty
|
# VERSION build argument takes precedence over the version a build
|
||||||
# version without failing.
|
# stage derives from the .git in the context.
|
||||||
version="$(git describe --tags --always --dirty 2>/dev/null || true)"
|
version="$(git describe --tags --always --dirty 2>/dev/null || true)"
|
||||||
[ -n "$version" ] || version="unknown"
|
[ -n "$version" ] || version="unknown"
|
||||||
docker build --no-cache \
|
tag="$("$SCRIPT_DIR/projectname")"
|
||||||
|
docker build --build-arg CACHEBUST="$(date +%s)-$$" \
|
||||||
--build-arg VERSION="$version" \
|
--build-arg VERSION="$version" \
|
||||||
-t "$("$SCRIPT_DIR/projectname")" .
|
-t "$tag" .
|
||||||
}
|
}
|
||||||
|
|
||||||
main "$@"
|
main "$@"
|
||||||
|
|||||||
+16
-6
@@ -7,15 +7,25 @@ ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
|||||||
# Must match the pin in script/bootstrap.
|
# Must match the pin in script/bootstrap.
|
||||||
NODE_VERSION="22.17.0"
|
NODE_VERSION="22.17.0"
|
||||||
|
|
||||||
# script/bootstrap installs node and yarn under nvm and leaves neither
|
# True when the node on PATH has the pinned major version: the same
|
||||||
# on the PATH of the shell that called it, so resolve the pinned
|
# function as in script/bootstrap, which uses it to decide where it
|
||||||
# toolchain here the way bootstrap's own install step does. nvm is a
|
# installs yarn.
|
||||||
# bash script, hence the subshell.
|
node_is_pinned_major() {
|
||||||
|
if ! command -v node >/dev/null 2>&1; then return 1; fi
|
||||||
|
installed="$(node --version)"
|
||||||
|
installed="${installed#v}"
|
||||||
|
[ "${installed%%.*}" = "${NODE_VERSION%%.*}" ]
|
||||||
|
}
|
||||||
|
|
||||||
|
# Run the yarn script/bootstrap installed: the yarn on PATH when the
|
||||||
|
# node on PATH has the pinned major version, and otherwise yarn under
|
||||||
|
# the pinned node in nvm, which bootstrap leaves off the PATH of the
|
||||||
|
# shell that called it. nvm is a bash script, hence the subshell.
|
||||||
run_yarn() {
|
run_yarn() {
|
||||||
if command -v yarn >/dev/null 2>&1; then
|
if node_is_pinned_major && command -v yarn >/dev/null 2>&1; then
|
||||||
exec yarn "$@"
|
exec yarn "$@"
|
||||||
fi
|
fi
|
||||||
if [ ! -s "$HOME/.nvm/nvm.sh" ]; then
|
if node_is_pinned_major || [ ! -s "$HOME/.nvm/nvm.sh" ]; then
|
||||||
echo "fmt: no yarn; run script/bootstrap first" >&2
|
echo "fmt: no yarn; run script/bootstrap first" >&2
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
|
|||||||
+16
-6
@@ -8,15 +8,25 @@ ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
|||||||
# Must match the pin in script/bootstrap.
|
# Must match the pin in script/bootstrap.
|
||||||
NODE_VERSION="22.17.0"
|
NODE_VERSION="22.17.0"
|
||||||
|
|
||||||
# script/bootstrap installs node and yarn under nvm and leaves neither
|
# True when the node on PATH has the pinned major version: the same
|
||||||
# on the PATH of the shell that called it, so resolve the pinned
|
# function as in script/bootstrap, which uses it to decide where it
|
||||||
# toolchain here the way bootstrap's own install step does. nvm is a
|
# installs yarn.
|
||||||
# bash script, hence the subshell.
|
node_is_pinned_major() {
|
||||||
|
if ! command -v node >/dev/null 2>&1; then return 1; fi
|
||||||
|
installed="$(node --version)"
|
||||||
|
installed="${installed#v}"
|
||||||
|
[ "${installed%%.*}" = "${NODE_VERSION%%.*}" ]
|
||||||
|
}
|
||||||
|
|
||||||
|
# Run the yarn script/bootstrap installed: the yarn on PATH when the
|
||||||
|
# node on PATH has the pinned major version, and otherwise yarn under
|
||||||
|
# the pinned node in nvm, which bootstrap leaves off the PATH of the
|
||||||
|
# shell that called it. nvm is a bash script, hence the subshell.
|
||||||
run_yarn() {
|
run_yarn() {
|
||||||
if command -v yarn >/dev/null 2>&1; then
|
if node_is_pinned_major && command -v yarn >/dev/null 2>&1; then
|
||||||
exec yarn "$@"
|
exec yarn "$@"
|
||||||
fi
|
fi
|
||||||
if [ ! -s "$HOME/.nvm/nvm.sh" ]; then
|
if node_is_pinned_major || [ ! -s "$HOME/.nvm/nvm.sh" ]; then
|
||||||
echo "fmt-check: no yarn; run script/bootstrap first" >&2
|
echo "fmt-check: no yarn; run script/bootstrap first" >&2
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
|
|||||||
+8
-5
@@ -5,9 +5,12 @@
|
|||||||
# make its answer untrustworthy.
|
# make its answer untrustworthy.
|
||||||
#
|
#
|
||||||
# The phase is not the last stage in the file, so it is built only when
|
# The phase is not the last stage in the file, so it is built only when
|
||||||
# --target names it. --no-cache because a cached lint layer is a lint
|
# --target names it. A new CACHEBUST value reruns every RUN step, since
|
||||||
# that did not run. The tag makes each build replace the previous image
|
# a cached lint layer is a lint that did not run; unlike --no-cache, it
|
||||||
# instead of leaving a dangling one behind.
|
# keeps Go's build cache. This relies on every stage of the Dockerfile
|
||||||
|
# declaring ARG CACHEBUST first; a stage without that line is served
|
||||||
|
# from cache on an unchanged tree, with no warning.
|
||||||
|
# --output type=cacheonly writes no image, since nothing uses one.
|
||||||
set -eu
|
set -eu
|
||||||
|
|
||||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
|
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
|
||||||
@@ -15,9 +18,9 @@ ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
|
|||||||
|
|
||||||
main() {
|
main() {
|
||||||
cd "$ROOT"
|
cd "$ROOT"
|
||||||
docker build --no-cache \
|
docker build --build-arg CACHEBUST="$(date +%s)-$$" \
|
||||||
--target lint \
|
--target lint \
|
||||||
-t "$("$SCRIPT_DIR/projectname")-lint" .
|
--output type=cacheonly .
|
||||||
}
|
}
|
||||||
|
|
||||||
main "$@"
|
main "$@"
|
||||||
|
|||||||
+8
-4
@@ -2,8 +2,12 @@
|
|||||||
# script/test: run the test suite. Testing is a phase of the Dockerfile
|
# script/test: run the test suite. Testing is a phase of the Dockerfile
|
||||||
# and this builds that phase alone, on the same terms as script/lint:
|
# and this builds that phase alone, on the same terms as script/lint:
|
||||||
# --target because a phase that is not the last stage is built only when
|
# --target because a phase that is not the last stage is built only when
|
||||||
# named, --no-cache because a cached test layer is a test that did not
|
# named, and a new CACHEBUST value because a cached test layer is a test
|
||||||
# run, and a tag so each build replaces the previous image.
|
# that did not run; unlike --no-cache, it keeps Go's build cache. This
|
||||||
|
# relies on every stage of the Dockerfile declaring ARG CACHEBUST first;
|
||||||
|
# a stage without that line is served from cache on an unchanged tree,
|
||||||
|
# with no warning. --output type=cacheonly writes no image, since
|
||||||
|
# nothing uses one.
|
||||||
set -eu
|
set -eu
|
||||||
|
|
||||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
|
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
|
||||||
@@ -11,9 +15,9 @@ ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
|
|||||||
|
|
||||||
main() {
|
main() {
|
||||||
cd "$ROOT"
|
cd "$ROOT"
|
||||||
docker build --no-cache \
|
docker build --build-arg CACHEBUST="$(date +%s)-$$" \
|
||||||
--target test \
|
--target test \
|
||||||
-t "$("$SCRIPT_DIR/projectname")-test" .
|
--output type=cacheonly .
|
||||||
}
|
}
|
||||||
|
|
||||||
main "$@"
|
main "$@"
|
||||||
|
|||||||
Reference in New Issue
Block a user