Compare commits
2
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
87648b91d0 | ||
|
|
faad959bea |
+31
-22
@@ -1,12 +1,16 @@
|
||||
# Every stage declares ARG CACHEBUST first: script/ passes a new value
|
||||
# on every build, which reruns every RUN step after it. The steps that
|
||||
# compile mount Go's build cache, which survives from build to build.
|
||||
# In every stage ARG CACHEBUST comes right after the steps that install
|
||||
# system packages and download Go's modules, or right after FROM in a
|
||||
# stage with neither, and every step that lints, tests or compiles
|
||||
# comes after it. script/ passes a new value on every build, which
|
||||
# reruns every RUN step after it. Docker keeps the layers above it
|
||||
# until the base image, their own lines or go.mod and go.sum change.
|
||||
# The steps that compile mount Go's build cache, and golangci-lint run
|
||||
# also mounts golangci-lint's; both survive from build to build.
|
||||
|
||||
# Lint phase. script/lint builds it alone. The linter is run directly:
|
||||
# `make lint` and script/lint are themselves a docker build.
|
||||
# golangci/golangci-lint:v2.14.0, 2026-10-15
|
||||
FROM golangci/golangci-lint@sha256:ad862ba6b3798cbe0fd9fd7408d498fd74fbd2623a92406b2fd3898faf0bf98f AS lint
|
||||
ARG CACHEBUST
|
||||
|
||||
# The linter compiles every package, and govips needs the libvips
|
||||
# headers for that. REPO_POLICIES.md has the lint phase install them
|
||||
@@ -18,25 +22,29 @@ RUN apt-get update \
|
||||
WORKDIR /src
|
||||
COPY go.mod go.sum ./
|
||||
RUN go mod download
|
||||
ARG CACHEBUST
|
||||
COPY . .
|
||||
RUN --mount=type=cache,target=/root/.cache/go-build \
|
||||
--mount=type=cache,target=/root/.cache/golangci-lint \
|
||||
golangci-lint run --config .golangci.yml ./...
|
||||
|
||||
# Test phase. script/test builds it alone. -race needs cgo and so a C
|
||||
# compiler, which the Debian Go image ships and the alpine one does not.
|
||||
# golang:1.25.4-trixie, 2026-10-15
|
||||
FROM golang@sha256:a02d35efc036053fdf0da8c15919276bf777a80cbfda6a35c5e9f087e652adfc AS test
|
||||
ARG CACHEBUST
|
||||
|
||||
# govips compiles against the libvips and libheif headers, and the tests
|
||||
# need libvips' JPEG XL support and libheif's AVIF encoder. Debian's
|
||||
# libvips-dev brings all of them, the encoder as a recommended package,
|
||||
# so recommended packages are installed here.
|
||||
RUN apt-get update \
|
||||
&& apt-get install -y libvips-dev \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
WORKDIR /src
|
||||
|
||||
# script/bootstrap --cgo installs the build dependencies (the libvips
|
||||
# and libheif headers, and libvips' JPEG XL support, which the tests
|
||||
# need) and downloads the Go modules.
|
||||
COPY script/ ./script/
|
||||
COPY go.mod go.sum ./
|
||||
RUN script/bootstrap --cgo
|
||||
|
||||
RUN go mod download
|
||||
ARG CACHEBUST
|
||||
COPY . .
|
||||
|
||||
# Without -v first; on a failure, again with -v for the details, and
|
||||
@@ -52,22 +60,23 @@ RUN --mount=type=cache,target=/root/.cache/go-build \
|
||||
go test -count=1 -timeout 90s -race -parallel 4 -v ./...; exit 1; }
|
||||
|
||||
# Build stage. Nothing is wanted from the two phases above: these copies
|
||||
# make BuildKit build them first, so this stage runs only when lint and
|
||||
# test passed.
|
||||
# make BuildKit build them first, so nothing after them runs unless lint
|
||||
# and test passed.
|
||||
# golang:1.25.4-alpine3.22, 2026-02-25; the runtime stage uses Alpine 3.22 too
|
||||
FROM golang:1.25.4-alpine3.22@sha256:d3f0cf7723f3429e3f9ed846243970b20a2de7bae6a5b66fc5914e228d831bbb AS builder
|
||||
|
||||
# git for the version, and for govips a C compiler, pkg-config and the
|
||||
# libvips and libheif headers
|
||||
RUN apk add --no-cache git build-base pkgconfig vips-dev libheif-dev
|
||||
|
||||
WORKDIR /src
|
||||
COPY go.mod go.sum ./
|
||||
RUN go mod download
|
||||
ARG CACHEBUST
|
||||
|
||||
COPY --from=lint /src/go.sum /dev/null
|
||||
COPY --from=test /src/go.sum /dev/null
|
||||
|
||||
WORKDIR /src
|
||||
|
||||
# Build dependencies, git among them, and Go modules, as in the test
|
||||
# phase
|
||||
COPY script/ ./script/
|
||||
COPY go.mod go.sum ./
|
||||
RUN script/bootstrap --cgo
|
||||
# A tar-stream context keeps the sender's file owners, which git refuses.
|
||||
RUN git config --system --add safe.directory /src
|
||||
|
||||
@@ -102,7 +111,6 @@ RUN --mount=type=cache,target=/root/.cache/go-build \
|
||||
# the libvips and musl it was built with.
|
||||
# alpine:3.22, 2026-10-08
|
||||
FROM alpine:3.22@sha256:5291449c3df73caf6ed85e649dec1b9e818b39a5d8c871e97afc13e9cd5e8fa8
|
||||
ARG CACHEBUST
|
||||
|
||||
# Install runtime dependencies only. vips-jxl is libvips' JPEG XL
|
||||
# support, without which pixad does not start.
|
||||
@@ -113,6 +121,7 @@ RUN apk add --no-cache \
|
||||
ca-certificates \
|
||||
tzdata \
|
||||
su-exec
|
||||
ARG CACHEBUST
|
||||
|
||||
# Copy binary from builder
|
||||
COPY --from=builder /pixad /usr/local/bin/pixad
|
||||
|
||||
@@ -694,7 +694,10 @@ Key settings in more detail:
|
||||
and generator pages. Default: empty
|
||||
- `blocked_networks` — list of CIDR ranges to refuse for SSRF protection, added
|
||||
to the always-enforced built-in ranges (loopback, private, link-local, CGNAT,
|
||||
benchmark, NAT64, and the like); an invalid CIDR aborts startup
|
||||
benchmark, NAT64, and the like); an invalid CIDR aborts startup. An IPv6
|
||||
address in the IPv4-mapped form (`::ffff:a.b.c.d`) is matched only as the IPv4
|
||||
address it holds, so an IPv6 range such as `::/0` refuses neither it nor a
|
||||
plain IPv4 address
|
||||
- `trusted_proxies` — list of CIDR ranges of the reverse proxies in front of
|
||||
pixa. `X-Forwarded-For` is believed only when the direct peer falls inside one
|
||||
of these ranges; the logged and login-recorded client address is then the
|
||||
@@ -822,19 +825,24 @@ provide:
|
||||
`script/precommit`
|
||||
|
||||
Every `docker build` in these scripts passes a new value of the `CACHEBUST`
|
||||
build argument, which every stage of the `Dockerfile` declares first, so every
|
||||
`RUN` step, the lint and test phases included, runs on every build instead of
|
||||
coming from the build cache. Go's build cache is kept from build to build: the
|
||||
steps that compile mount it, so a package whose inputs are unchanged is not
|
||||
compiled again, and both `go test` runs pass `-count=1` so that no stored test
|
||||
result is replayed. The `test` phase runs on the Debian Go image, with Debian's
|
||||
libvips and libheif; the image pixa ships is built and runs on Alpine.
|
||||
`script/check`, `script/cibuild`, `script/docker`, `script/lint`, `script/test`,
|
||||
`script/setup` and `script/install-precommit` are the standard copies from
|
||||
`sneak/prompts`, kept identical to them. `script/fmt` and `script/fmt-check` are
|
||||
the standard copies with pixa's `gofmt` step kept before prettier. prettier
|
||||
formats the markdown only, not the HTML templates, as it cannot parse a Go
|
||||
template action inside a tag.
|
||||
build argument, which every stage of the `Dockerfile` declares, so every `RUN`
|
||||
step after that line, the lint and test steps included, runs on every build
|
||||
instead of coming from the build cache. Above the line are the steps that
|
||||
install system packages and download the Go modules; Docker keeps their layers
|
||||
until the base image, the step itself, `go.mod` or `go.sum` changes. Go's build
|
||||
cache is kept from build to build: the steps that compile mount it, so a package
|
||||
whose inputs are unchanged is not compiled again, and both `go test` runs pass
|
||||
`-count=1` so that no stored test result is replayed. The lint step also mounts
|
||||
golangci-lint's cache, which reuses a package's findings only when its files,
|
||||
those of the packages it imports, `go.mod`, `.golangci.yml` and the
|
||||
golangci-lint version are all unchanged. The `test` phase runs on the Debian Go
|
||||
image, with Debian's libvips and libheif; the image pixa ships is built and runs
|
||||
on Alpine. `script/check`, `script/cibuild`, `script/docker`, `script/lint`,
|
||||
`script/test`, `script/setup` and `script/install-precommit` are the standard
|
||||
copies from `sneak/prompts`, kept identical to them. `script/fmt` and
|
||||
`script/fmt-check` are the standard copies with pixa's `gofmt` step kept before
|
||||
prettier. prettier formats the markdown only, not the HTML templates, as it
|
||||
cannot parse a Go template action inside a tag.
|
||||
|
||||
## Load Test
|
||||
|
||||
|
||||
+68
-35
@@ -1,6 +1,6 @@
|
||||
---
|
||||
title: Repository Policies
|
||||
last_modified: 2026-10-15
|
||||
last_modified: 2026-10-18
|
||||
---
|
||||
|
||||
This document covers repository structure, tooling, and workflow standards. Code
|
||||
@@ -159,8 +159,11 @@ style conventions are in separate documents:
|
||||
one's findings, and a host-global lock in `$TMPDIR` makes concurrent runs
|
||||
exit non-zero with `parallel golangci-lint is running` — a status a caller
|
||||
cannot tell from real findings. Both have produced wrong verdicts in this
|
||||
org, in both directions. A container has its own golangci-lint cache, its
|
||||
own `TMPDIR` and a digest-pinned binary, so neither is reachable.
|
||||
org, in both directions. A container has its own `TMPDIR`, so the lock is
|
||||
out of reach. The lint phase does keep golangci-lint's cache between check
|
||||
builds (below), but it lints every checkout at `/src`, and a finding is
|
||||
taken from that cache only for the same files, settings and golangci-lint
|
||||
version.
|
||||
|
||||
- **Any build that runs checks is given a new `CACHEBUST` value.** Docker
|
||||
invalidates a `COPY` layer only when the copied content changes, so on an
|
||||
@@ -170,15 +173,21 @@ style conventions are in separate documents:
|
||||
id, so no two builds share a value, not even the three in one
|
||||
`script/cibuild`. `script/lint`, `script/test`, `script/cibuild` and
|
||||
`script/docker` are the four, and there is no fifth — `script/check` runs the
|
||||
two gate phases and `script/fmt-check`, and builds no image of its own. Every
|
||||
stage of the `Dockerfile` declares `ARG CACHEBUST` as its first line after
|
||||
`FROM`, and a new value reruns every `RUN` step after that line; `COPY` steps
|
||||
are compared by content and need nothing. A stage without the `ARG` line is
|
||||
served from cache on an unchanged tree, so a repository re-vendors the scripts
|
||||
and adds the `ARG` lines in one commit. The scripts do not pass `--no-cache`:
|
||||
it reruns every step too, but it also empties every cache mount, and Go's
|
||||
build cache, which the canonical Go `Dockerfile` below mounts so that compiled
|
||||
packages survive between check builds, would start empty every time. A bare
|
||||
two gate phases and `script/fmt-check`, and builds no image of its own. In
|
||||
every stage of the `Dockerfile`, `ARG CACHEBUST` comes right after the steps
|
||||
that install system packages and download Go's modules, or right after `FROM`
|
||||
in a stage with neither, and every step that lints, tests or compiles comes
|
||||
after it. A step that installs anything else as well, such as a
|
||||
`script/bootstrap` that also installs node packages, comes after it too. A new
|
||||
value reruns every `RUN` step after that line; `COPY` steps are compared by
|
||||
content and need nothing. Docker keeps the layers above the line until the
|
||||
base image, their own lines or the files they copy change. A stage without the
|
||||
`ARG` line is served from cache on an unchanged tree, so a repository
|
||||
re-vendors the scripts and adds the `ARG` lines in one commit. The scripts do
|
||||
not pass `--no-cache`: it reruns every step too, but it also builds the layers
|
||||
above the line again and empties every cache mount, so Go's build cache and
|
||||
golangci-lint's, which the canonical Go `Dockerfile` below mounts so that they
|
||||
survive between check builds, would start empty every time. A bare
|
||||
`docker build .` is not evidence that anything ran: a sub-second build
|
||||
reporting success is a cache hit, not a result. Never invalidate by pruning —
|
||||
`docker builder prune` and friends destroy a build cache shared with every
|
||||
@@ -194,19 +203,25 @@ style conventions are in separate documents:
|
||||
`Dockerfile`:
|
||||
|
||||
```dockerfile
|
||||
# Every stage declares ARG CACHEBUST first: script/ passes a new value
|
||||
# on every build, which reruns every RUN step after it. The steps that
|
||||
# compile mount Go's build cache, which survives from build to build.
|
||||
# In every stage ARG CACHEBUST comes right after the steps that install
|
||||
# system packages and download Go's modules, or right after FROM in a
|
||||
# stage with neither, and every step that lints, tests or compiles
|
||||
# comes after it. script/ passes a new value on every build, which
|
||||
# reruns every RUN step after it. Docker keeps the layers above it
|
||||
# until the base image, their own lines or go.mod and go.sum change.
|
||||
# The steps that compile mount Go's build cache, and golangci-lint run
|
||||
# also mounts golangci-lint's; both survive from build to build.
|
||||
|
||||
# Lint phase
|
||||
# golangci/golangci-lint:v2.x.x, YYYY-MM-DD
|
||||
FROM golangci/golangci-lint@sha256:... AS lint
|
||||
ARG CACHEBUST
|
||||
WORKDIR /src
|
||||
COPY go.mod go.sum ./
|
||||
RUN go mod download
|
||||
ARG CACHEBUST
|
||||
COPY . .
|
||||
RUN --mount=type=cache,target=/root/.cache/go-build \
|
||||
--mount=type=cache,target=/root/.cache/golangci-lint \
|
||||
golangci-lint run --config .golangci.yml ./...
|
||||
|
||||
# Test phase. -race needs cgo and so a C compiler, which the Debian Go
|
||||
@@ -214,10 +229,10 @@ style conventions are in separate documents:
|
||||
# cache also holds test results, which go test would otherwise replay.
|
||||
# golang:1.x, YYYY-MM-DD
|
||||
FROM golang@sha256:... AS test
|
||||
ARG CACHEBUST
|
||||
WORKDIR /src
|
||||
COPY go.mod go.sum ./
|
||||
RUN go mod download
|
||||
ARG CACHEBUST
|
||||
COPY . .
|
||||
RUN --mount=type=cache,target=/root/.cache/go-build \
|
||||
go test -count=1 -timeout 90s -race -cover ./... || \
|
||||
@@ -225,19 +240,19 @@ style conventions are in separate documents:
|
||||
go test -count=1 -timeout 90s -race -v ./...; exit 1; }
|
||||
|
||||
# Build stage. Nothing is wanted from either phase above; the copies
|
||||
# are what make BuildKit build them first, so this stage cannot run
|
||||
# unless lint and test passed.
|
||||
# are what make BuildKit build them first, so nothing after them can
|
||||
# run unless lint and test passed.
|
||||
# golang:1.x-alpine, YYYY-MM-DD
|
||||
FROM golang@sha256:... AS builder
|
||||
ARG CACHEBUST
|
||||
COPY --from=lint /src/go.sum /dev/null
|
||||
COPY --from=test /src/go.sum /dev/null
|
||||
RUN apk add --no-cache git
|
||||
# A tar-stream context keeps the sender's file owners, which git refuses.
|
||||
RUN git config --system --add safe.directory /src
|
||||
WORKDIR /src
|
||||
COPY go.mod go.sum ./
|
||||
RUN go mod download
|
||||
ARG CACHEBUST
|
||||
COPY --from=lint /src/go.sum /dev/null
|
||||
COPY --from=test /src/go.sum /dev/null
|
||||
# A tar-stream context keeps the sender's file owners, which git refuses.
|
||||
RUN git config --system --add safe.directory /src
|
||||
COPY . .
|
||||
|
||||
# The VERSION build arg when one is given, otherwise
|
||||
@@ -274,16 +289,26 @@ style conventions are in separate documents:
|
||||
- Keep the runtime stage last, and if you add a stage after it, give it the
|
||||
same two copies. A plain `docker build .` builds the last stage's chain
|
||||
and nothing else.
|
||||
- Only Go's build cache is mounted (`/root/.cache/go-build`, where Go keeps
|
||||
it for root), and only on the steps that compile: `golangci-lint run`,
|
||||
both `go test` runs and `go build`. It keeps compiled packages between
|
||||
check builds, and Go reuses one only when all its inputs are unchanged, so
|
||||
it saves the compile without skipping a check. Nothing else is kept:
|
||||
golangci-lint's own cache would replay stored findings in place of a lint
|
||||
run, and Go builds from whatever files a kept module cache holds without
|
||||
checking them again (only `go mod verify` does), so a file changed there
|
||||
would go unnoticed. Go's build cache also holds test results, which is why
|
||||
both `go test` runs carry `-count=1`.
|
||||
- Check builds keep these, and none of them can hand back a stale result:
|
||||
- The layers above `ARG CACHEBUST`: the system packages and Go's
|
||||
modules. Docker installs the packages again when the base image or the
|
||||
install line changes, and downloads the modules again when `go.mod` or
|
||||
`go.sum` changes. Go checks every module against `go.sum` as it
|
||||
downloads it, and no later build can change a kept layer. The modules
|
||||
are kept in a layer rather than a cache mount because a cache mount is
|
||||
shared by every build on the host, and Go builds from whatever files
|
||||
it holds without checking them again (only `go mod verify` does), so a
|
||||
file changed there would go unnoticed.
|
||||
- Go's build cache (`/root/.cache/go-build`, where Go keeps it for
|
||||
root), mounted on the steps that compile: `golangci-lint run`, both
|
||||
`go test` runs and `go build`. Go reuses a compiled package only when
|
||||
all its inputs are unchanged. The cache also holds test results, which
|
||||
is why both `go test` runs carry `-count=1`.
|
||||
- golangci-lint's cache (`/root/.cache/golangci-lint`, where the pinned
|
||||
image keeps it), mounted on `golangci-lint run`. golangci-lint reuses
|
||||
a package's findings only when the package's files and those of every
|
||||
package it imports, `go.mod`, the linters and their settings in
|
||||
`.golangci.yml`, and the golangci-lint version are all unchanged.
|
||||
- If the project uses `//go:embed` directives that reference build artifacts
|
||||
(e.g. a web frontend compiled in a separate stage), the lint phase must
|
||||
create placeholder files so the embed directives resolve. Example:
|
||||
@@ -292,12 +317,20 @@ style conventions are in separate documents:
|
||||
in the lint phase. The `golangci/golangci-lint` image is Debian-based and
|
||||
has no `apk`, so install with `apt-get` under the Debian package name
|
||||
(`libvips-dev`, where alpine says `vips-dev`), and delete the package
|
||||
lists in the same `RUN`, so the layer does not keep them:
|
||||
lists in the same `RUN`, so the layer does not keep them. The test phase,
|
||||
on the Debian Go image, installs the libraries its tests need the same
|
||||
way. Either install goes right after `FROM`, above `ARG CACHEBUST`, so
|
||||
Docker keeps it until the base image or the install line changes:
|
||||
|
||||
```dockerfile
|
||||
FROM golangci/golangci-lint@sha256:... AS lint
|
||||
RUN apt-get update \
|
||||
&& apt-get install -y --no-install-recommends libvips-dev \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
WORKDIR /src
|
||||
COPY go.mod go.sum ./
|
||||
RUN go mod download
|
||||
ARG CACHEBUST
|
||||
```
|
||||
|
||||
- `.dockerignore` lets `.git` into the build context. It keeps out every git
|
||||
|
||||
@@ -30,6 +30,22 @@ P2: security: per-origin rate limiting
|
||||
|
||||
# Completed Steps
|
||||
|
||||
- 2026-10-18 re-vendored from `sneak/prompts` commit `0653579` (closes #221):
|
||||
each `Dockerfile` stage installs its system packages and downloads the Go
|
||||
modules above `ARG CACHEBUST`, so check builds keep them, and the lint step
|
||||
keeps golangci-lint's cache.
|
||||
- 2026-10-16 P1 security bug: the check that keeps pixa from fetching internal
|
||||
addresses refuses the local-use NAT64 range (`64:ff9b:1::/48`, RFC 8215) as it
|
||||
refuses `64:ff9b::/96`, and refuses an IPv6 address in the IPv4-compatible
|
||||
form (`::/96`) when `blocked_networks` contains either the address as written
|
||||
or the IPv4 address it holds (closes #299). A source host such as
|
||||
`[64:ff9b:1::a00:1]` was fetched, which on a network that runs NAT64 on that
|
||||
range reaches the IPv4 address it holds, here `10.0.0.1`; and with
|
||||
`203.0.113.0/24` in `blocked_networks`, `[::ffff:203.0.113.5]` was refused but
|
||||
`[::203.0.113.5]` was fetched. Both now answer 403. The mapped form
|
||||
(`::ffff:a.b.c.d`) is still matched only as the IPv4 address it holds: Go
|
||||
keeps a plain IPv4 address in that form, and an IPv6 entry such as `::/0` must
|
||||
not refuse every IPv4 address.
|
||||
- 2026-10-16 P1 security bug: the check that keeps pixa from fetching internal
|
||||
addresses judges an IPv6 address in the IPv4-compatible form (`::/96`), such
|
||||
as `::127.0.0.1` or `::10.0.0.1`, by the IPv4 address in its last four bytes
|
||||
|
||||
@@ -59,15 +59,17 @@ const (
|
||||
// builtinBlockedPrefixes are internal or special-use ranges that Go's
|
||||
// net.IP predicates (IsPrivate, IsLinkLocalUnicast, and the like) do not
|
||||
// already cover. They are always blocked, in addition to any
|
||||
// operator-supplied networks. IPv4-mapped IPv6 addresses are unmapped
|
||||
// before matching, so these IPv4 ranges are caught in both forms.
|
||||
// operator-supplied networks. IPv4-mapped and IPv4-compatible IPv6
|
||||
// addresses are read as the IPv4 address they hold before matching, so
|
||||
// these IPv4 ranges are caught in all three forms.
|
||||
//
|
||||
//nolint:gochecknoglobals // immutable built-in blocklist
|
||||
var builtinBlockedPrefixes = []netip.Prefix{
|
||||
netip.MustParsePrefix("100.64.0.0/10"), // RFC 6598 CGNAT / carrier-grade NAT
|
||||
netip.MustParsePrefix("192.0.0.0/24"), // RFC 6890 IETF protocol assignments
|
||||
netip.MustParsePrefix("198.18.0.0/15"), // RFC 2544 benchmarking range
|
||||
netip.MustParsePrefix("64:ff9b::/96"), // RFC 6052 NAT64 (maps onto IPv4)
|
||||
netip.MustParsePrefix("100.64.0.0/10"), // RFC 6598 CGNAT / carrier-grade NAT
|
||||
netip.MustParsePrefix("192.0.0.0/24"), // RFC 6890 IETF protocol assignments
|
||||
netip.MustParsePrefix("198.18.0.0/15"), // RFC 2544 benchmarking range
|
||||
netip.MustParsePrefix("64:ff9b::/96"), // RFC 6052 NAT64 (maps onto IPv4)
|
||||
netip.MustParsePrefix("64:ff9b:1::/48"), // RFC 8215 local-use NAT64 (maps onto IPv4)
|
||||
}
|
||||
|
||||
// Fetcher errors.
|
||||
@@ -543,12 +545,8 @@ func isPrivateIP(ip net.IP) bool {
|
||||
return true
|
||||
}
|
||||
|
||||
// An IPv4-compatible IPv6 address (in ::/96, such as ::127.0.0.1) holds
|
||||
// an IPv4 address in its last four bytes and is judged as that address
|
||||
if addr, ok := netip.AddrFromSlice(ip); ok &&
|
||||
netip.MustParsePrefix("::/96").Contains(addr) {
|
||||
return isPrivateIP(ip[12:])
|
||||
}
|
||||
// An IPv4-compatible IPv6 address is judged as the IPv4 address it holds
|
||||
ip = ipv4CompatibleToIPv4(ip)
|
||||
|
||||
// Check for loopback
|
||||
if ip.IsLoopback() {
|
||||
@@ -601,7 +599,12 @@ func isPrivateIP(ip net.IP) bool {
|
||||
}
|
||||
|
||||
// isBlockedIP reports whether ip is refused, either by the built-in
|
||||
// internal-range check or by one of the operator-supplied prefixes.
|
||||
// internal-range check or by one of the operator-supplied prefixes. An
|
||||
// IPv4-compatible address (in ::/96) is refused when a prefix contains the
|
||||
// address as written or the IPv4 address it holds. An IPv4-mapped address (in
|
||||
// ::ffff:0:0/96) is matched only as the IPv4 address it holds: Go keeps a
|
||||
// plain IPv4 address in that form too, and an IPv6 prefix such as ::/0 must
|
||||
// not refuse every IPv4 address.
|
||||
func isBlockedIP(ip net.IP, blocked []netip.Prefix) bool {
|
||||
if isPrivateIP(ip) {
|
||||
return true
|
||||
@@ -614,11 +617,28 @@ func isBlockedIP(ip net.IP, blocked []netip.Prefix) bool {
|
||||
|
||||
addr = addr.Unmap()
|
||||
|
||||
// ip read as IPv4 where it is or holds an IPv4 address, otherwise the
|
||||
// zero netip.Addr, which no prefix contains. Matching asIPv4 as well as
|
||||
// addr refuses more only for an IPv4-compatible address.
|
||||
asIPv4, _ := netip.AddrFromSlice(ipv4CompatibleToIPv4(ip).To4())
|
||||
|
||||
return slices.ContainsFunc(blocked, func(prefix netip.Prefix) bool {
|
||||
return prefix.Contains(addr)
|
||||
return prefix.Contains(addr) || prefix.Contains(asIPv4)
|
||||
})
|
||||
}
|
||||
|
||||
// ipv4CompatibleToIPv4 returns the IPv4 address that an IPv4-compatible IPv6
|
||||
// address (in ::/96, such as ::127.0.0.1) holds in its last four bytes, and
|
||||
// any other address unchanged.
|
||||
func ipv4CompatibleToIPv4(ip net.IP) net.IP {
|
||||
addr, ok := netip.AddrFromSlice(ip)
|
||||
if ok && netip.MustParsePrefix("::/96").Contains(addr) {
|
||||
return ip[12:]
|
||||
}
|
||||
|
||||
return ip
|
||||
}
|
||||
|
||||
// ssrfSafeDialer validates IP addresses against the built-in blocked ranges
|
||||
// before connecting. New wraps dialSSRFSafe with the operator-supplied
|
||||
// blocklist; this entry point enforces the built-in ranges alone.
|
||||
|
||||
@@ -14,10 +14,12 @@ import (
|
||||
"net"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"net/netip"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"sync/atomic"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"go.uber.org/fx"
|
||||
"go.uber.org/fx/fxtest"
|
||||
@@ -183,6 +185,57 @@ func startImageProxy(
|
||||
return s, db.DB(), stateDir
|
||||
}
|
||||
|
||||
// startImageProxyWithBlockedNetworks starts the components pixad's fx app
|
||||
// builds, from a config with a fresh state directory and blockedNetworks as
|
||||
// blocked_networks. Unlike startImageProxy it provides no upstream fetcher, so
|
||||
// the handlers build the one pixad uses, which checks every address it
|
||||
// connects to against the built-in blocked ranges and blocked_networks. It
|
||||
// returns the server with its routes.
|
||||
func startImageProxyWithBlockedNetworks(
|
||||
t *testing.T, blockedNetworks []netip.Prefix,
|
||||
) *Server {
|
||||
t.Helper()
|
||||
|
||||
stateDir := t.TempDir()
|
||||
cfg := &config.Config{
|
||||
SigningKey: testSigningKey,
|
||||
StateDir: stateDir,
|
||||
DBURL: "file:" + filepath.Join(stateDir, "state.sqlite3"),
|
||||
BlockedNetworks: blockedNetworks,
|
||||
// A fetch that gets past the checks and finds nothing at the address
|
||||
// fails within this time at the latest.
|
||||
UpstreamFetchTimeout: 2 * time.Second,
|
||||
// With no connection slots, the fetch would fail before connecting.
|
||||
UpstreamConnections: config.DefaultUpstreamConnections,
|
||||
DownstreamTimeout: config.DefaultDownstreamTimeout,
|
||||
}
|
||||
|
||||
var (
|
||||
h *handlers.Handlers
|
||||
mw *middleware.Middleware
|
||||
)
|
||||
|
||||
app := fxtest.New(t,
|
||||
fx.Supply(cfg),
|
||||
fx.Provide(
|
||||
globals.New,
|
||||
logger.New,
|
||||
database.New,
|
||||
healthcheck.New,
|
||||
handlers.New,
|
||||
middleware.New,
|
||||
),
|
||||
fx.Populate(&h, &mw),
|
||||
)
|
||||
app.RequireStart()
|
||||
t.Cleanup(app.RequireStop)
|
||||
|
||||
s := &Server{config: cfg, mw: mw, h: h}
|
||||
s.SetupRoutes()
|
||||
|
||||
return s
|
||||
}
|
||||
|
||||
// getImage sends a GET for target to s and fails unless it answers 200.
|
||||
func getImage(t *testing.T, s *Server, target string) *httptest.ResponseRecorder {
|
||||
t.Helper()
|
||||
|
||||
@@ -3,6 +3,7 @@ package server
|
||||
import (
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"net/netip"
|
||||
"net/url"
|
||||
"regexp"
|
||||
"slices"
|
||||
@@ -586,3 +587,79 @@ func TestImageRoutesRefuseIPv6SourceHostHoldingInternalIPv4Address(t *testing.T)
|
||||
t.Errorf("upstream received %q, want no request", got)
|
||||
}
|
||||
}
|
||||
|
||||
// TestImageRoutesCheckIPv4AddressInIPv6SourceHost requests ipv6SourcePath on
|
||||
// source hosts that hold an IPv4 address, through a /v1/image/ URL with a
|
||||
// signature and through an encrypted URL, with the fetcher pixad uses.
|
||||
// [64:ff9b:1::a00:1], in the local-use NAT64 range (64:ff9b:1::/48, RFC 8215),
|
||||
// answers 403. [::203.0.113.5] holds the public address 203.0.113.5 in the
|
||||
// IPv4-compatible form (::/96) and answers 403 when blocked_networks lists
|
||||
// 203.0.113.0/24 or ::/96. [::ffff:203.0.113.5] holds it in the IPv4-mapped
|
||||
// form (::ffff:0:0/96) and answers 403 when blocked_networks lists
|
||||
// 203.0.113.0/24. Go keeps the plain address 203.0.113.5 in the mapped form
|
||||
// too, and an IPv6 range must not refuse it: with ::/0 listed it is not
|
||||
// refused. Neither is [::203.0.113.5] with blocked_networks empty. Where a
|
||||
// host is not refused, the fetcher tries to connect to it, finds nothing
|
||||
// there, and the request fails with another status.
|
||||
func TestImageRoutesCheckIPv4AddressInIPv6SourceHost(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
const (
|
||||
ipv4CompatibleHost = "[::203.0.113.5]"
|
||||
ipv4MappedHost = "[::ffff:203.0.113.5]"
|
||||
)
|
||||
|
||||
tests := []struct {
|
||||
name string
|
||||
sourceHost string
|
||||
blockedNetworks []netip.Prefix
|
||||
wantForbidden bool
|
||||
}{
|
||||
{"local-use NAT64 form of 10.0.0.1", "[64:ff9b:1::a00:1]", nil, true},
|
||||
{"IPv4-compatible form of an address in blocked_networks",
|
||||
ipv4CompatibleHost,
|
||||
[]netip.Prefix{netip.MustParsePrefix("203.0.113.0/24")}, true},
|
||||
{"IPv4-compatible form with ::/96 in blocked_networks",
|
||||
ipv4CompatibleHost,
|
||||
[]netip.Prefix{netip.MustParsePrefix("::/96")}, true},
|
||||
{"IPv4-mapped form of an address in blocked_networks",
|
||||
ipv4MappedHost,
|
||||
[]netip.Prefix{netip.MustParsePrefix("203.0.113.0/24")}, true},
|
||||
{"plain IPv4 address with ::/0 in blocked_networks",
|
||||
"203.0.113.5",
|
||||
[]netip.Prefix{netip.MustParsePrefix("::/0")}, false},
|
||||
{"IPv4-compatible form of an address with blocked_networks empty",
|
||||
ipv4CompatibleHost, nil, false},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
imageURLs := map[string]string{
|
||||
"image URL with a signature": signedImageURL(t, tt.sourceHost),
|
||||
"encrypted URL": encryptedImageURL(t, tt.sourceHost),
|
||||
}
|
||||
|
||||
for route, imageURL := range imageURLs {
|
||||
t.Run(tt.name+" "+route, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
s := startImageProxyWithBlockedNetworks(t, tt.blockedNetworks)
|
||||
|
||||
rec := httptest.NewRecorder()
|
||||
s.ServeHTTP(rec, httptest.NewRequestWithContext(
|
||||
t.Context(), http.MethodGet, imageURL, nil))
|
||||
t.Logf("GET %s: %d", imageURL, rec.Code)
|
||||
|
||||
forbidden := rec.Code == http.StatusForbidden
|
||||
if tt.wantForbidden && !forbidden {
|
||||
t.Errorf("status = %d, want %d; body %s",
|
||||
rec.Code, http.StatusForbidden, rec.Body.String())
|
||||
}
|
||||
|
||||
if !tt.wantForbidden && forbidden {
|
||||
t.Errorf("status = %d, want any other status; body %s",
|
||||
rec.Code, rec.Body.String())
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
+2
-3
@@ -19,9 +19,8 @@
|
||||
# CGO image libraries (pkg-config, vips
|
||||
# with its JPEG XL support, libheif) for
|
||||
# the govips bindings instead of Node: to
|
||||
# compile pixa, in the Dockerfile's test
|
||||
# phase and build stage, which format
|
||||
# nothing
|
||||
# compile pixa outside Docker, as
|
||||
# `make build` does
|
||||
set -eu
|
||||
|
||||
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
||||
|
||||
+5
-4
@@ -2,11 +2,12 @@
|
||||
# script/cibuild: run the CI build. It bootstraps first: a CI runner
|
||||
# checks out and runs this and nothing else, and script/fmt-check runs
|
||||
# the formatter on the host, which a pristine checkout cannot do.
|
||||
# A new CACHEBUST value reruns every RUN step, since a cached gate phase
|
||||
# is a check that did not run; unlike --no-cache, it keeps Go's build cache.
|
||||
# A new CACHEBUST value reruns every RUN step after the ARG CACHEBUST
|
||||
# line, since a cached gate phase is a check that did not run; unlike
|
||||
# --no-cache, it keeps the layers above that line and the cache mounts.
|
||||
# This relies on every stage of the Dockerfile declaring ARG CACHEBUST
|
||||
# first; a stage without that line is served from cache on an unchanged
|
||||
# tree, with no warning.
|
||||
# before every step that lints, tests or compiles; a stage without that
|
||||
# line is served from cache on an unchanged tree, with no warning.
|
||||
set -eu
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
|
||||
|
||||
+5
-4
@@ -1,11 +1,12 @@
|
||||
#!/bin/sh
|
||||
# script/docker: build the Docker image tagged with the project name.
|
||||
# Identical in all repos; the tag comes from script/projectname.
|
||||
# A new CACHEBUST value reruns every RUN step, since a cached gate phase
|
||||
# is a check that did not run; unlike --no-cache, it keeps Go's build cache.
|
||||
# A new CACHEBUST value reruns every RUN step after the ARG CACHEBUST
|
||||
# line, since a cached gate phase is a check that did not run; unlike
|
||||
# --no-cache, it keeps the layers above that line and the cache mounts.
|
||||
# This relies on every stage of the Dockerfile declaring ARG CACHEBUST
|
||||
# first; a stage without that line is served from cache on an unchanged
|
||||
# tree, with no warning.
|
||||
# before every step that lints, tests or compiles; a stage without that
|
||||
# line is served from cache on an unchanged tree, with no warning.
|
||||
set -eu
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
|
||||
|
||||
+7
-5
@@ -5,11 +5,13 @@
|
||||
# make its answer untrustworthy.
|
||||
#
|
||||
# The phase is not the last stage in the file, so it is built only when
|
||||
# --target names it. A new CACHEBUST value reruns every RUN step, since
|
||||
# a cached lint layer is a lint that did not run; unlike --no-cache, it
|
||||
# keeps Go's build cache. This relies on every stage of the Dockerfile
|
||||
# declaring ARG CACHEBUST first; a stage without that line is served
|
||||
# from cache on an unchanged tree, with no warning.
|
||||
# --target names it. A new CACHEBUST value reruns every RUN step after
|
||||
# the ARG CACHEBUST line, since a cached lint layer is a lint that did
|
||||
# not run; unlike --no-cache, it keeps the layers above that line and
|
||||
# the cache mounts. This relies on every stage of the Dockerfile
|
||||
# declaring ARG CACHEBUST before every step that lints, tests or
|
||||
# compiles; a stage without that line is served from cache on an
|
||||
# unchanged tree, with no warning.
|
||||
# --output type=cacheonly writes no image, since nothing uses one.
|
||||
set -eu
|
||||
|
||||
|
||||
+6
-5
@@ -3,11 +3,12 @@
|
||||
# and this builds that phase alone, on the same terms as script/lint:
|
||||
# --target because a phase that is not the last stage is built only when
|
||||
# named, and a new CACHEBUST value because a cached test layer is a test
|
||||
# that did not run; unlike --no-cache, it keeps Go's build cache. This
|
||||
# relies on every stage of the Dockerfile declaring ARG CACHEBUST first;
|
||||
# a stage without that line is served from cache on an unchanged tree,
|
||||
# with no warning. --output type=cacheonly writes no image, since
|
||||
# nothing uses one.
|
||||
# that did not run; unlike --no-cache, it keeps the layers above the ARG
|
||||
# CACHEBUST line and the cache mounts. This relies on every stage of the
|
||||
# Dockerfile declaring ARG CACHEBUST before every step that lints, tests
|
||||
# or compiles; a stage without that line is served from cache on an
|
||||
# unchanged tree, with no warning. --output type=cacheonly writes no
|
||||
# image, since nothing uses one.
|
||||
set -eu
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
|
||||
|
||||
Reference in New Issue
Block a user