Compare commits
1
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
e5b68b2df7 |
+4
-5
@@ -1,15 +1,14 @@
|
||||
# .git is deliberately NOT excluded: the build derives the version it stamps
|
||||
# into the binary from it (script/version). Nor is any tracked file: git in
|
||||
# the build would see it as deleted and mark the version -dirty. Only
|
||||
# untracked files belong here.
|
||||
#
|
||||
# .ci-fingerprint is deliberately NOT excluded: it is the CI cache barrier
|
||||
# that keeps the check stages from replaying a cached pass. See the lint
|
||||
# stage of the Dockerfile.
|
||||
.git/
|
||||
bin/
|
||||
# Extracted from 3p/ by `make assets` inside the build; a host copy is not
|
||||
# needed. The tarball in 3p/ must stay in the context.
|
||||
static/js/alpine.min.js
|
||||
*.md
|
||||
LICENSE
|
||||
.editorconfig
|
||||
.env
|
||||
.env.*
|
||||
*.db
|
||||
|
||||
@@ -28,11 +28,12 @@ jobs:
|
||||
run: script/ci-mark-superseded
|
||||
|
||||
- name: Fingerprint the build context
|
||||
# Every commit that changes more than docs writes a new fingerprint
|
||||
# into the context, which invalidates the `COPY . .` layer of both
|
||||
# check stages: a commit that was never linted, formatted-checked,
|
||||
# tested and built cannot report success from cache. Docs-only
|
||||
# commits rebuild too, since the context also carries `.git`.
|
||||
# `.dockerignore` keeps docs out of the build context, so a docs-only
|
||||
# commit legitimately replays the whole image from cache and stays
|
||||
# cheap. Every other commit writes a new fingerprint into the context,
|
||||
# which invalidates the `COPY . .` layer of both check stages: a
|
||||
# commit that was never linted, formatted-checked, tested and built
|
||||
# cannot report success from cache.
|
||||
run: |
|
||||
set -eu
|
||||
fp="$(git log -1 --format=%H -- . ':!*.md' ':!LICENSE' ':!.editorconfig')"
|
||||
|
||||
+7
-15
@@ -38,8 +38,8 @@ FROM golang:1.26.1-bookworm@sha256:4465644228bc2857a954b092167e12aa59c006a349228
|
||||
COPY --from=lint /src/go.sum /dev/null
|
||||
|
||||
# jq is a runtime dependency of script/ci-mark-superseded, which the test
|
||||
# suite executes. git is what script/version derives the version with.
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends make curl ca-certificates jq git && rm -rf /var/lib/apt/lists/*
|
||||
# suite executes.
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends make curl ca-certificates jq && rm -rf /var/lib/apt/lists/*
|
||||
|
||||
WORKDIR /build
|
||||
|
||||
@@ -55,22 +55,14 @@ COPY . .
|
||||
# from its tarball in 3p/.
|
||||
RUN make test
|
||||
|
||||
# Version stamped into the binary: the VERSION build arg when one is
|
||||
# given, otherwise what script/version derives from the .git the build
|
||||
# context carries, so any `docker build .` of a clone stamps its commit.
|
||||
# With neither, as from a source tarball, it is "unknown".
|
||||
# Version stamped into the binary. .dockerignore excludes .git/, so
|
||||
# nothing in this stage can derive it: script/docker resolves it on the
|
||||
# host and passes it in. The default is what a bare `docker build .`
|
||||
# with no --build-arg gets, and it names no tag the tree may not be at.
|
||||
#
|
||||
# Declared here, below the test step, so a changed version does not
|
||||
# invalidate its cached layer.
|
||||
ARG VERSION
|
||||
|
||||
# A context that carries .git must not stamp "unknown": that means git is
|
||||
# missing here or refused to read the checkout, and the image could not be
|
||||
# traced back to its commit.
|
||||
RUN if [ -d .git ] && [ "$(make version VERSION="$VERSION")" = unknown ]; then \
|
||||
echo "version is unknown although the build context carries .git" >&2; \
|
||||
exit 1; \
|
||||
fi
|
||||
ARG VERSION=unknown
|
||||
|
||||
RUN make build VERSION="$VERSION"
|
||||
|
||||
|
||||
@@ -4,12 +4,12 @@
|
||||
.DEFAULT_GOAL := check
|
||||
|
||||
# Version stamped into the binary. Derived from git by script/version;
|
||||
# override it (`make build VERSION=v1.2.3`) to stamp a given value, which is
|
||||
# how the Dockerfile passes its build arg in.
|
||||
# override it (`make build VERSION=v1.2.3`) where git metadata is
|
||||
# unavailable, which is how the Dockerfile passes its build arg in.
|
||||
VERSION ?= $(shell script/version)
|
||||
|
||||
# An empty override (`make build VERSION=`, or the Dockerfile's `make build
|
||||
# VERSION="$VERSION"` when no VERSION build arg was given) means unset,
|
||||
# An empty override (`make build VERSION=`, or a `--build-arg VERSION=`
|
||||
# landing on the Dockerfile's `make build VERSION="$VERSION"`) means unset,
|
||||
# exactly as it does in script/version -- stamping "" would leave the binary
|
||||
# reporting no version and the footer back on its "dev" fallback. `override`
|
||||
# is required: a plain assignment loses to the command-line definition it
|
||||
|
||||
@@ -1123,21 +1123,13 @@ build itself.
|
||||
| Uncommitted changes | the above with a `-dirty` suffix |
|
||||
| No git metadata | `unknown` |
|
||||
|
||||
The image derives it the same way, from the `.git` that the build
|
||||
context carries, so any `docker build .` of a clone stamps the commit it
|
||||
was built from; a shallow clone of one branch has no tags and stamps the
|
||||
short SHA. `.dockerignore` must therefore leave out neither `.git` nor
|
||||
any tracked file, which git in the build would see as deleted, marking
|
||||
the version `-dirty`. A `VERSION` build arg (`--build-arg VERSION=...`)
|
||||
takes precedence; `script/docker` (and so `make docker`) passes the one
|
||||
`script/version` resolves on the host. The image build fails if its
|
||||
context carries `.git` and the version still comes out `unknown`, which
|
||||
means git in the build could not read the checkout.
|
||||
|
||||
`unknown` is what a source tarball, or a `docker build` with no `.git`
|
||||
in its context and no `VERSION` build arg, reports. A build that reports
|
||||
`unknown` is a build nobody told what it was; it is not a failure, but
|
||||
it cannot be traced back to a commit.
|
||||
`unknown` is what a source tarball or a `docker build .` with no
|
||||
`--build-arg VERSION=...` reports. `.dockerignore` excludes `.git/`, so
|
||||
the build context carries no git metadata and the image cannot derive
|
||||
the version itself: `script/docker` (and so `make docker`) resolves it
|
||||
on the host and passes it in as the `VERSION` build arg. A build that
|
||||
reports `unknown` is a build nobody told what it was; it is not a
|
||||
failure, but it cannot be traced back to a commit.
|
||||
|
||||
`make version` prints what the current checkout would stamp, and
|
||||
`make build VERSION=v1.2.3` overrides it. An empty override — from
|
||||
@@ -3173,9 +3165,8 @@ version is fixed independently of the compiler's:
|
||||
rebuilds the binary with `CGO_ENABLED=1` and static linking so it
|
||||
runs on musl. Both builds go through `make build`, the relink adding
|
||||
its `-extldflags` via `GO_LDFLAGS`, so neither can drop the `-X` that
|
||||
stamps the version. The version is the `VERSION` build arg if one is
|
||||
given, otherwise derived from the `.git` in the context, and the
|
||||
stage fails if a context with `.git` would stamp `unknown` (see
|
||||
stamps the version. The version arrives as the `VERSION` build arg,
|
||||
since the context has no `.git` (see
|
||||
[Version stamping](#version-stamping)).
|
||||
3. **Runtime stage** (`alpine:3.21`) — copies the static binary and
|
||||
`deploy/docker-entrypoint.sh`, creates the `/var/lib/webhooker`
|
||||
@@ -3207,17 +3198,16 @@ A layer cache lets `docker build .` exit 0 in seconds with the lint and
|
||||
test stages replayed rather than executed, which would make a green
|
||||
check meaningless. The `check` workflow therefore writes
|
||||
`.ci-fingerprint` into the build context before building. Its value is
|
||||
the hash of the last commit that touched anything other than `*.md`,
|
||||
`LICENSE` and `.editorconfig`, so:
|
||||
the hash of the last commit that touched the build context, so:
|
||||
|
||||
- Any commit that changes code (including a squash merge whose tree
|
||||
matches an already-built branch) gets a new fingerprint, invalidates
|
||||
the `COPY . .` layer of both check stages, and really runs
|
||||
`make fmt-check`, `golangci-lint`, `make test`, and `make build`. A
|
||||
run that reports success ran them.
|
||||
- A docs-only commit leaves the fingerprint unchanged, but it still
|
||||
rebuilds in full: the context also carries `.git`, which changes with
|
||||
every commit (see [Version stamping](#version-stamping)).
|
||||
- A docs-only commit leaves the fingerprint unchanged — `.dockerignore`
|
||||
excludes `*.md`, `LICENSE` and `.editorconfig` from the context
|
||||
anyway — so the image replays from cache and costs seconds.
|
||||
|
||||
The module download layer sits above `COPY . .` and stays cached either
|
||||
way.
|
||||
|
||||
@@ -17,6 +17,10 @@ const (
|
||||
// dnsResolutionTimeout is the maximum time to wait for
|
||||
// DNS resolution during SSRF validation.
|
||||
dnsResolutionTimeout = 5 * time.Second
|
||||
|
||||
// azureWireServer is Azure's WireServer, a public address that
|
||||
// serves VM credentials.
|
||||
azureWireServer = "168.63.129.16"
|
||||
)
|
||||
|
||||
// Sentinel errors for SSRF validation.
|
||||
@@ -25,8 +29,14 @@ var (
|
||||
errNoIPs = errors.New(
|
||||
"hostname resolved to no IP addresses",
|
||||
)
|
||||
errBlockedIP = errors.New(
|
||||
"blocked private, reserved or cloud metadata address",
|
||||
// ErrBlockedIP reports a private or reserved address the
|
||||
// default blocklist refuses, one that ALLOWED_EGRESS_CIDRS
|
||||
// can open.
|
||||
ErrBlockedIP = errors.New(
|
||||
"blocked private or reserved address",
|
||||
)
|
||||
errBlockedWireServer = errors.New(
|
||||
"blocked cloud metadata address",
|
||||
)
|
||||
errBlockedMetadata = errors.New(
|
||||
"blocked link-local or cloud instance metadata " +
|
||||
@@ -123,8 +133,7 @@ func init() {
|
||||
"::1/128",
|
||||
"fc00::/7",
|
||||
"fe80::/10",
|
||||
// Azure WireServer, a public address that serves VM credentials.
|
||||
"168.63.129.16/32",
|
||||
azureWireServer + "/32",
|
||||
})
|
||||
|
||||
// Every entry is named. The set must not grow or shrink
|
||||
@@ -338,9 +347,17 @@ func (g *Guard) checkIP(ip net.IP) error {
|
||||
return nil
|
||||
}
|
||||
|
||||
// WireServer is on the default blocklist but is a public
|
||||
// address, so its refusal does not call it private or reserved.
|
||||
if ip.Equal(net.ParseIP(azureWireServer)) {
|
||||
return fmt.Errorf(
|
||||
"target IP %s: %w", ip, errBlockedWireServer,
|
||||
)
|
||||
}
|
||||
|
||||
if isBlockedIP(ip) {
|
||||
return fmt.Errorf(
|
||||
"target IP %s: %w", ip, errBlockedIP,
|
||||
"target IP %s: %w", ip, ErrBlockedIP,
|
||||
)
|
||||
}
|
||||
|
||||
|
||||
@@ -1577,11 +1577,22 @@ func (h *Handlers) validateTargetURL(
|
||||
"url", delivery.MaskURL(targetURL),
|
||||
"error", err,
|
||||
)
|
||||
http.Error(
|
||||
w,
|
||||
"Invalid target URL: "+err.Error(),
|
||||
http.StatusBadRequest,
|
||||
)
|
||||
|
||||
msg := "Invalid target URL: " + err.Error()
|
||||
|
||||
// Only a private or reserved address's refusal says how
|
||||
// to allow it. Metadata refusals never do: link-local and
|
||||
// the other unconditional metadata addresses cannot be
|
||||
// opened, and Azure's WireServer, which listing does
|
||||
// open, hands out VM credentials.
|
||||
if errors.Is(err, delivery.ErrBlockedIP) {
|
||||
msg += ". Private and reserved addresses are refused " +
|
||||
"by default; the server's ALLOWED_EGRESS_CIDRS " +
|
||||
"setting allows named networks (see \"Allowing " +
|
||||
"egress to your own network\" in the README)."
|
||||
}
|
||||
|
||||
http.Error(w, msg, http.StatusBadRequest)
|
||||
|
||||
return err
|
||||
}
|
||||
|
||||
@@ -0,0 +1,116 @@
|
||||
package handlers_test
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
"net/url"
|
||||
"testing"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
"sneak.berlin/go/webhooker/internal/database"
|
||||
)
|
||||
|
||||
// privateRefusalHint is the sentence that tells an operator a private
|
||||
// destination is refused on purpose, and how to allow one.
|
||||
const privateRefusalHint = "Private and reserved addresses are " +
|
||||
"refused by default; the server's ALLOWED_EGRESS_CIDRS setting " +
|
||||
"allows named networks (see \"Allowing egress to your own " +
|
||||
"network\" in the README)."
|
||||
|
||||
// TestTargetRefusal_PrivateDestinationSaysHowToAllowIt covers both
|
||||
// target types that take a URL, on add and on edit.
|
||||
func TestTargetRefusal_PrivateDestinationSaysHowToAllowIt(
|
||||
t *testing.T,
|
||||
) {
|
||||
t.Parallel()
|
||||
|
||||
env := setupSourceTest(t)
|
||||
|
||||
targetTypes := []database.TargetType{
|
||||
database.TargetTypeHTTP,
|
||||
database.TargetTypeSlack,
|
||||
}
|
||||
|
||||
for _, targetType := range targetTypes {
|
||||
t.Run(string(targetType), func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
webhook := seedWebhookWithRetention(t, env.db, 30)
|
||||
targetsPath := "/source/" + webhook.ID + "/targets"
|
||||
|
||||
form := url.Values{}
|
||||
form.Set("name", "private")
|
||||
form.Set("type", string(targetType))
|
||||
form.Set("url", editBlockedURL)
|
||||
|
||||
added := serveTarget(
|
||||
env, http.MethodPost, targetsPath, form,
|
||||
)
|
||||
assert.Equal(t, http.StatusBadRequest, added.Code)
|
||||
assert.Contains(
|
||||
t, added.Body.String(), privateRefusalHint,
|
||||
)
|
||||
|
||||
form.Set("url", editOriginalURL)
|
||||
|
||||
created := serveTarget(
|
||||
env, http.MethodPost, targetsPath, form,
|
||||
)
|
||||
require.Equal(
|
||||
t, http.StatusSeeOther, created.Code,
|
||||
created.Body.String(),
|
||||
)
|
||||
|
||||
targets := targetsForWebhook(t, env.db, webhook.ID)
|
||||
require.Len(t, targets, 1)
|
||||
|
||||
form.Set("url", editBlockedURL)
|
||||
|
||||
edited := submitTargetEdit(
|
||||
env, webhook.ID, targets[0].ID, form,
|
||||
)
|
||||
assert.Equal(t, http.StatusBadRequest, edited.Code)
|
||||
assert.Contains(
|
||||
t, edited.Body.String(), privateRefusalHint,
|
||||
)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestTargetRefusal_MetadataDestinationDoesNotSayHowToAllowIt: no
|
||||
// setting opens a link-local address, and Azure's WireServer hands out
|
||||
// VM credentials, so neither refusal points at the setting.
|
||||
func TestTargetRefusal_MetadataDestinationDoesNotSayHowToAllowIt(
|
||||
t *testing.T,
|
||||
) {
|
||||
t.Parallel()
|
||||
|
||||
env := setupSourceTest(t)
|
||||
|
||||
metadataURLs := map[string]string{
|
||||
"link-local": "http://169.254.169.254/latest/meta-data/",
|
||||
"wireserver": "http://168.63.129.16/?comp=versions",
|
||||
}
|
||||
|
||||
for name, metadataURL := range metadataURLs {
|
||||
t.Run(name, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
webhook := seedWebhookWithRetention(t, env.db, 30)
|
||||
|
||||
form := url.Values{}
|
||||
form.Set("name", "metadata")
|
||||
form.Set("type", string(database.TargetTypeHTTP))
|
||||
form.Set("url", metadataURL)
|
||||
|
||||
w := serveTarget(
|
||||
env, http.MethodPost,
|
||||
"/source/"+webhook.ID+"/targets", form,
|
||||
)
|
||||
assert.Equal(t, http.StatusBadRequest, w.Code)
|
||||
assert.NotContains(
|
||||
t, w.Body.String(), privateRefusalHint,
|
||||
)
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -115,8 +115,8 @@ func TestVersion_EnclosingRepositoryIsNotUsed(t *testing.T) {
|
||||
require.Equal(t, unknown, runScript(t, inner, nil))
|
||||
}
|
||||
|
||||
// An explicit VERSION, such as the Dockerfile's build arg, wins over
|
||||
// anything derivable.
|
||||
// The Docker build has no git metadata, so the version arrives as an
|
||||
// environment override. It wins over anything derivable.
|
||||
func TestVersion_EnvironmentOverrideWins(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
@@ -128,8 +128,8 @@ func TestVersion_EnvironmentOverrideWins(t *testing.T) {
|
||||
}
|
||||
|
||||
// An empty VERSION is treated as unset rather than stamping an empty
|
||||
// string: a caller exporting VERSION= must not produce a binary
|
||||
// reporting "".
|
||||
// string: the Dockerfile's build arg has a non-empty default, but a
|
||||
// caller exporting VERSION= must not produce a binary reporting "".
|
||||
func TestVersion_EmptyOverrideFallsBackToGit(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
@@ -168,8 +168,8 @@ func TestMakefile_BuildComposesVersionAndExtraFlags(t *testing.T) {
|
||||
}
|
||||
|
||||
// A caller can define VERSION as the empty string -- `make build
|
||||
// VERSION=`, or the Dockerfile's `make build VERSION="$VERSION"` when no
|
||||
// VERSION build arg was given. script/version's own guard does not cover
|
||||
// VERSION=`, or a `--build-arg VERSION=` reaching the Dockerfile's `make
|
||||
// build VERSION="$VERSION"`. script/version's own guard does not cover
|
||||
// that: the value never passes through the script. Stamping "" would
|
||||
// leave the binary reporting no version and the footer on "dev", which
|
||||
// is the defect this package exists for.
|
||||
@@ -231,7 +231,7 @@ func TestDockerfile_BuildsThroughTheMakeTarget(t *testing.T) {
|
||||
|
||||
require.NotContains(t, dockerfile, "go build",
|
||||
"a raw go build bypasses the Makefile's -X flag")
|
||||
require.Contains(t, dockerfile, "ARG VERSION")
|
||||
require.Contains(t, dockerfile, "ARG VERSION=")
|
||||
require.Contains(t, dockerfile,
|
||||
`make build VERSION="$VERSION" GO_LDFLAGS='-extldflags "-static"'`)
|
||||
}
|
||||
|
||||
+3
-3
@@ -2,9 +2,9 @@
|
||||
# script/docker: build the Docker image tagged with the project name.
|
||||
# The tag comes from script/projectname.
|
||||
#
|
||||
# The version script/version resolves here goes in as the VERSION build
|
||||
# arg, which takes precedence over what the build would derive from the
|
||||
# .git in its context.
|
||||
# .dockerignore excludes .git/, so the builder stage cannot derive the
|
||||
# version itself. It is resolved here, where the checkout is, and passed
|
||||
# in as a build arg; without it the image would stamp itself "unknown".
|
||||
set -eu
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
|
||||
|
||||
+7
-5
@@ -7,16 +7,18 @@
|
||||
#
|
||||
# Order of precedence:
|
||||
#
|
||||
# 1. $VERSION, if set and non-empty: an explicit value, such as the
|
||||
# Dockerfile's VERSION build arg.
|
||||
# 1. $VERSION, if set and non-empty. This is how the value reaches a
|
||||
# build that cannot derive it: .dockerignore excludes .git/, so the
|
||||
# builder stage has no git metadata and the Dockerfile takes the
|
||||
# value as a build arg instead.
|
||||
# 2. `git describe --tags --always --dirty` against this checkout. At
|
||||
# a clean tagged commit that is exactly the tag; otherwise it
|
||||
# carries the short SHA, the commit distance when a tag is
|
||||
# reachable, and a -dirty suffix for uncommitted changes.
|
||||
# 3. "unknown", for a tree with no git metadata and no $VERSION -- a
|
||||
# source tarball, or a `docker build` with no .git in its context
|
||||
# and no VERSION build arg. That case must not fail the build and
|
||||
# must not name a tag the tree may not be at, so it names nothing.
|
||||
# source tarball, or `docker build .` with no --build-arg. That
|
||||
# case must not fail the build and must not name a tag the tree may
|
||||
# not be at, so it names nothing.
|
||||
#
|
||||
# The git step insists the enclosing repository is this checkout, not
|
||||
# merely some repository above it: an unpacked tarball sitting inside an
|
||||
|
||||
Reference in New Issue
Block a user