Lint in a container as a build step, via Dockerfile.lint (closes #113)
All checks were successful
check / check (pull_request) Successful in 4m0s
All checks were successful
check / check (pull_request) Successful in 4m0s
Every lint run now happens inside its own container, invoked through script/lint, and linting is a build step rather than a container command: a successful build of the new root Dockerfile.lint IS a clean lint. That shape also works where the docker daemon is remote and bind mounts are impossible. Its FROM line -- golangci/golangci-lint:v2.12.2, pinned by digest -- is now the only pin of the linter version in this repo. A container per run has its own lint cache and its own golangci-lint lock, both discarded with it, so neither cross-worktree contamination nor lock contention exists any more. The machinery that defended against them is therefore gone: the per-worktree cache directories, the lock-retry loop, and script/lint-audit, which existed to catch findings replayed from a cache that no longer exists. So is the host lint path in its entirety -- the native escape hatch, its version detection, and VAULTIK_LINT_IN_CONTAINER in both script/lint and the Dockerfile. Nothing lints on the host, at any version. A cached build lints nothing, so the CHECK_EPOCH mechanism the product Dockerfile already used is what makes a green mean something: ARG CHECK_EPOCH with no default, placed below the module layers so dependency caching survives, a `RUN [ -n "$CHECK_EPOCH" ] || exit 1` guard so a build that withholds the arg fails instead of replaying, and the value expanded into the lint command itself. script/lint computes `epoch="$(date +%s%N)$$"` as a bare assignment on its own line, because inline in the argument a failing substitution does not abort under `set -eu` and yields a constant empty epoch -- which is exactly the false green being prevented. The product Dockerfile loses its lint stage rather than gaining a second linter pin. That stage ran `make lint`, which is now `docker build`: docker-in-docker inside a BuildKit step with no daemon. Calling golangci-lint directly there instead would have meant two independently bumpable digests for one tool. `make fmt-check` moves beside `make test` in the builder stage, and script/cibuild now builds Dockerfile.lint and then Dockerfile, each with its own fresh epoch, failing on either. Consequence, stated in comments rather than left to be discovered: script/docker builds the product image only and no longer lints; script/check and script/cibuild are the gates. Two decisions taken deliberately and documented where they apply. `golangci-lint config verify` is omitted: it fetches its JSON schema over an unpinned live HTTPS call, which would make the gate depend on a remote resource outside this repo's hash-pinning discipline and turn an upstream outage or an egress-less runner into a red that is not a lint verdict. script/lint-fix is kept, reimplemented as a bind-mounted docker run against the image parsed out of Dockerfile.lint -- a build step cannot write fixes back to the worktree -- and its header states outright that it is a developer convenience, never a gate, and needs a local daemon. cmd/vaultik/lintdocker_test.go parses both Dockerfiles and both scripts and fails if any part of the mechanism is dropped: the digest pin, the defaultless ARG below `go mod download`, the emptiness guard, the expansion of the epoch into each check command, the bare per-invocation epoch assignment in both scripts, cibuild building both files, and the absence of any host-lint escape hatch. Every one of those losses is silent -- the build still exits 0 and nothing is checked -- which is why they are asserted rather than trusted. script/lint takes no arguments now, and says so instead of dropping them: a build step has no command line to pass linter flags to.
This commit is contained in:
@@ -48,11 +48,12 @@ missing() {
|
||||
! command -v "$1" >/dev/null 2>&1
|
||||
}
|
||||
|
||||
# Docker is a hard requirement, not a nice-to-have: script/lint runs the
|
||||
# digest-pinned golangci-lint image from the Dockerfile's lint stage, and
|
||||
# script/check and script/precommit both run script/lint. A bootstrap
|
||||
# that prints "bootstrap complete" on a machine where `make check` cannot
|
||||
# run is a false success, so this fails instead.
|
||||
# Docker is a hard requirement, not a nice-to-have: script/lint lints by
|
||||
# building Dockerfile.lint, whose digest-pinned golangci-lint image is
|
||||
# the only place the linter runs, and script/check and script/precommit
|
||||
# both run script/lint. A bootstrap that prints "bootstrap complete" on a
|
||||
# machine where `make check` cannot run is a false success, so this fails
|
||||
# instead.
|
||||
#
|
||||
# Installing docker from here was considered and rejected: it needs root,
|
||||
# a running daemon, and on macOS a GUI cask, so an attempt would itself
|
||||
@@ -79,13 +80,15 @@ bootstrap: FAILED - $reason.
|
||||
|
||||
Docker is required to develop this repo. Without it these do not work:
|
||||
|
||||
script/lint runs the digest-pinned golangci-lint image declared
|
||||
by the Dockerfile's lint stage, which is the single
|
||||
source of truth for the linter version
|
||||
script/lint builds Dockerfile.lint, which runs the linter as a
|
||||
build step in a digest-pinned golangci-lint image.
|
||||
That FROM line is the single source of truth for the
|
||||
linter version
|
||||
script/check runs script/lint
|
||||
script/precommit runs script/check, so commits are blocked by the
|
||||
pre-commit hook installed by script/setup
|
||||
script/cibuild builds the Dockerfile, which is what CI runs
|
||||
script/cibuild builds Dockerfile.lint and Dockerfile, which is what
|
||||
CI runs
|
||||
|
||||
Install docker (and start the daemon, checking DOCKER_HOST and your
|
||||
group membership), then re-run script/bootstrap. golangci-lint on PATH
|
||||
@@ -104,12 +107,12 @@ main() {
|
||||
# Go toolchain
|
||||
if missing go; then pkg_install go golang go go; fi
|
||||
|
||||
# golangci-lint is deliberately NOT installed: script/lint runs the
|
||||
# digest-pinned golangci-lint image from the Dockerfile's lint stage,
|
||||
# so whatever a package manager happens to ship would only be a
|
||||
# shadow of the pinned version that could drift from CI. script/lint
|
||||
# will not use a PATH binary on a host at any version, so installing
|
||||
# one here would buy nothing.
|
||||
# golangci-lint is deliberately NOT installed: script/lint lints by
|
||||
# building Dockerfile.lint, whose digest-pinned image is the only
|
||||
# place the linter runs, so whatever a package manager happens to
|
||||
# ship would only be a shadow of the pinned version that could drift
|
||||
# from CI. Nothing on the host is ever used as a linter, at any
|
||||
# version, so installing one here would buy nothing.
|
||||
|
||||
# sqlite3 CLI: the test suite shells out to it (VACUUM).
|
||||
if missing sqlite3; then pkg_install sqlite sqlite3 sqlite sqlite; fi
|
||||
|
||||
@@ -1,22 +1,31 @@
|
||||
#!/bin/sh
|
||||
# script/cibuild: run the CI build. The Dockerfile does not run
|
||||
# script/check; it runs `make fmt-check` and `make lint` in its lint
|
||||
# stage and `make test` in its builder stage. A successful build
|
||||
# implies those three passed, provided they actually ran -- which is
|
||||
# what the CHECK_EPOCH below is for.
|
||||
# Generic: needs no adaptation. The Gitea workflow runs this on push.
|
||||
# script/cibuild: run the CI build. This is the full gate, and it is two
|
||||
# builds, in this order:
|
||||
#
|
||||
# Dockerfile.lint the linter, as a build step (a clean build IS a
|
||||
# clean lint)
|
||||
# Dockerfile `make fmt-check` and `make test` in the builder
|
||||
# stage, then the product image
|
||||
#
|
||||
# Either one failing fails this script. Note what follows from the
|
||||
# split: script/docker builds only the product image and so no longer
|
||||
# lints -- this script and script/check (which runs script/lint) are the
|
||||
# things that decide whether the tree is clean.
|
||||
#
|
||||
# Generic apart from the two Dockerfiles: the Gitea workflow runs this
|
||||
# on push.
|
||||
set -eu
|
||||
|
||||
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
||||
|
||||
main() {
|
||||
cd "$ROOT"
|
||||
# The Dockerfile's check layers are keyed on CHECK_EPOCH, so a
|
||||
# fresh value here is what forces them to re-run: without it an
|
||||
# Both Dockerfiles key their check layers on CHECK_EPOCH, so a fresh
|
||||
# value is what forces those layers to re-run: without it an
|
||||
# unchanged tree replays them from cache, the checks never execute,
|
||||
# and the build still exits 0. The ARG sits immediately above the
|
||||
# check RUNs, so dependency and module layers still cache. The
|
||||
# Dockerfile also refuses to build at all when CHECK_EPOCH is empty,
|
||||
# and the build still exits 0. Each ARG sits immediately above the
|
||||
# check RUNs, so dependency and module layers still cache. Both
|
||||
# Dockerfiles also refuse to build at all when CHECK_EPOCH is empty,
|
||||
# so a missing value fails loudly here rather than passing quietly.
|
||||
#
|
||||
# The value must be unique per invocation, not per second. `date +%s`
|
||||
@@ -35,6 +44,18 @@ main() {
|
||||
# script exists to prevent -- so the guard would disarm itself and
|
||||
# still exit 0. As a bare assignment, `set -e` catches a failing
|
||||
# `date` and no build starts.
|
||||
#
|
||||
# A separate value per build, because they are separate builds: one
|
||||
# `date` shared between them would still be fresh, but reusing it
|
||||
# invites the two to be collapsed into a single value that is
|
||||
# computed somewhere else and passed in.
|
||||
epoch="$(date +%s%N)$$"
|
||||
# cacheonly for the lint build: its verdict is the exit status and
|
||||
# the image is never run, so exporting it is pure cost. See
|
||||
# script/lint.
|
||||
docker build --output=type=cacheonly \
|
||||
--build-arg CHECK_EPOCH="$epoch" -f Dockerfile.lint .
|
||||
|
||||
epoch="$(date +%s%N)$$"
|
||||
docker build --build-arg CHECK_EPOCH="$epoch" .
|
||||
}
|
||||
|
||||
@@ -2,6 +2,13 @@
|
||||
# script/docker: build the Docker image tagged with the project name.
|
||||
# Identical in all repos; the tag comes from script/projectname.
|
||||
# Generic: needs no adaptation.
|
||||
#
|
||||
# This builds the PRODUCT image only, and the product Dockerfile has no
|
||||
# lint stage: linting lives in Dockerfile.lint and is run by
|
||||
# script/lint. So a green here means `make fmt-check` and `make test`
|
||||
# passed and the image built -- it says nothing about lint. The gates
|
||||
# are script/check (which runs script/lint) and script/cibuild (which
|
||||
# builds both files).
|
||||
set -eu
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
|
||||
|
||||
354
script/lint
354
script/lint
@@ -1,110 +1,41 @@
|
||||
#!/bin/sh
|
||||
# script/lint: run the linter.
|
||||
#
|
||||
# The linter always runs at the version pinned by the Dockerfile's lint
|
||||
# stage, so a local run and a CI run of the same tree cannot disagree.
|
||||
# That FROM line (image tag plus digest) is the single source of truth
|
||||
# for the linter version in this repo: bump it there and nothing else
|
||||
# needs editing.
|
||||
# The linter runs inside the image built by Dockerfile.lint, and it runs
|
||||
# there as a BUILD STEP: a successful build of that file IS a clean
|
||||
# lint. Nothing lints on the host, at any version, ever. That FROM line
|
||||
# is the single source of truth for the linter version in this repo, so
|
||||
# a local run and a CI run of the same tree cannot disagree.
|
||||
#
|
||||
# Normally that means running the pinned image with docker. The one
|
||||
# exception is running INSIDE that image: the Dockerfile's lint stage
|
||||
# runs `make lint`, and there is no docker daemon in there. That stage
|
||||
# sets VAULTIK_LINT_IN_CONTAINER=1, and only when that variable is set
|
||||
# is a golangci-lint on PATH used directly - and then only if its
|
||||
# version is exactly the pin. Version equality alone is deliberately NOT
|
||||
# enough: it also matches a developer's locally installed copy of the
|
||||
# same version, which is a different build with a different Go
|
||||
# toolchain, reached by a different code path, and it would bypass the
|
||||
# digest pin this script exists to enforce. /.dockerenv was considered
|
||||
# as the context signal and rejected: dockerd creates it for `docker
|
||||
# run`, but it is not reliably present during a BuildKit `docker build`,
|
||||
# which is exactly the case the exception exists for.
|
||||
# One container per run means one lint cache and one golangci-lint lock
|
||||
# per run, both private to that run and thrown away with it. That is
|
||||
# what makes concurrent runs on a shared host safe, and it is why this
|
||||
# script no longer carries per-worktree cache directories, a lock-retry
|
||||
# loop, or an output audit: there is no shared state left for them to
|
||||
# defend (issue https://git.eeqj.de/sneak/vaultik/issues/113).
|
||||
#
|
||||
# The linter's output is checked before it is believed: every run is
|
||||
# audited by script/lint-audit for findings that cannot belong to this
|
||||
# tree, and a run refused by golangci-lint's cross-process lock is
|
||||
# retried rather than reported as a verdict. See the lock-retry loop in
|
||||
# main and the header of script/lint-audit.
|
||||
# To watch the linter execute, set BUILDKIT_PROGRESS=plain, which docker
|
||||
# honours directly:
|
||||
#
|
||||
# Extra arguments are passed through to `golangci-lint run`, before
|
||||
# `./...` (see script/lint-fix).
|
||||
# BUILDKIT_PROGRESS=plain script/lint
|
||||
#
|
||||
# The lint layer must appear as executing rather than CACHED on every
|
||||
# run; see the CHECK_EPOCH comment in Dockerfile.lint.
|
||||
set -eu
|
||||
|
||||
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
||||
DOCKERFILE="$ROOT/Dockerfile"
|
||||
|
||||
# golangci-lint takes a cross-process lock and refuses to start while
|
||||
# another instance holds it. That refusal is not a lint result, and
|
||||
# exiting non-zero on it is indistinguishable to a caller from real
|
||||
# findings - so it is retried rather than reported. Bounded, because a
|
||||
# lock that is never released must fail rather than hang.
|
||||
LOCK_MESSAGE="parallel golangci-lint is running"
|
||||
LOCK_ATTEMPTS=6
|
||||
LOCK_SLEEP=15
|
||||
|
||||
# The image reference of the Dockerfile's lint stage, tag and digest
|
||||
# included, e.g.
|
||||
# golangci/golangci-lint:v2.12.2-alpine@sha256:91b2...
|
||||
lint_image() {
|
||||
awk '$1 == "FROM" && $3 == "AS" && $4 == "lint" { print $2; exit }' \
|
||||
"$DOCKERFILE"
|
||||
}
|
||||
|
||||
# The bare version that image reference pins, e.g. 2.12.2
|
||||
pinned_version() {
|
||||
lint_image | sed -e 's/@.*//' -e 's/.*://' -e 's/^v//' -e 's/-.*//'
|
||||
}
|
||||
|
||||
# The version of the golangci-lint on PATH, if any, e.g. 2.12.2
|
||||
#
|
||||
# `version --short` prints the bare version and is the interface meant
|
||||
# for this (checked against 2.10.1 and 2.12.2). The banner scrape below
|
||||
# it is a fallback for a release where --short is absent or silent; the
|
||||
# banner's exact wording is not a stable interface, which is why it is
|
||||
# no longer the primary parse.
|
||||
installed_version() {
|
||||
command -v golangci-lint >/dev/null 2>&1 || return 0
|
||||
|
||||
short="$(golangci-lint version --short 2>/dev/null |
|
||||
tr -d '[:space:]' | sed -e 's/^v//')"
|
||||
case "$short" in
|
||||
*[0-9].[0-9]*.[0-9]*)
|
||||
echo "$short"
|
||||
return 0
|
||||
;;
|
||||
esac
|
||||
|
||||
golangci-lint version 2>/dev/null | awk '
|
||||
{
|
||||
for (i = 1; i <= NF; i++) {
|
||||
if ($i ~ /^[0-9]+\.[0-9]+\.[0-9]+$/) {
|
||||
print $i
|
||||
exit
|
||||
}
|
||||
}
|
||||
}'
|
||||
}
|
||||
|
||||
# True inside the Dockerfile's lint stage, which sets this. Nothing else
|
||||
# sets it: setting it by hand on a host is an explicit, visible decision
|
||||
# to lint with an unpinned binary, not something reached by accident.
|
||||
in_lint_container() {
|
||||
[ "${VAULTIK_LINT_IN_CONTAINER:-}" = "1" ]
|
||||
}
|
||||
DOCKERFILE="$ROOT/Dockerfile.lint"
|
||||
|
||||
require_docker() {
|
||||
image="$1"
|
||||
if ! command -v docker >/dev/null 2>&1; then
|
||||
cat >&2 <<EOF
|
||||
lint: docker is required to run the pinned linter.
|
||||
|
||||
pinned image: $image
|
||||
lint image declared by: $DOCKERFILE
|
||||
|
||||
Install docker. Linting with any other golangci-lint is not supported:
|
||||
it is what lets a local run pass while CI fails. An installed
|
||||
golangci-lint on PATH is not used, whatever its version; only the lint
|
||||
stage of the Dockerfile itself runs the linter natively.
|
||||
it is what lets a local run pass while CI fails. A golangci-lint on
|
||||
PATH is never used, whatever its version.
|
||||
EOF
|
||||
exit 1
|
||||
fi
|
||||
@@ -113,7 +44,7 @@ EOF
|
||||
lint: the docker daemon is not reachable, so the pinned linter cannot
|
||||
run.
|
||||
|
||||
pinned image: $image
|
||||
lint image declared by: $DOCKERFILE
|
||||
|
||||
Start the daemon (and check DOCKER_HOST / your group membership). This
|
||||
script will not fall back to a different linter version or to an
|
||||
@@ -123,213 +54,54 @@ EOF
|
||||
fi
|
||||
}
|
||||
|
||||
# Where the per-worktree caches live.
|
||||
cache_home() {
|
||||
echo "${XDG_CACHE_HOME:-${HOME:-/tmp}/.cache}/vaultik-lint"
|
||||
}
|
||||
usage() {
|
||||
cat >&2 <<EOF
|
||||
usage: $(basename "$0")
|
||||
|
||||
# A short, stable digest of this worktree's path.
|
||||
path_digest() {
|
||||
if command -v sha256sum >/dev/null 2>&1; then
|
||||
printf '%s' "$ROOT" | sha256sum | cut -c1-12
|
||||
elif command -v shasum >/dev/null 2>&1; then
|
||||
printf '%s' "$ROOT" | shasum -a 256 | cut -c1-12
|
||||
else
|
||||
printf '%s' "$ROOT" | cksum | tr -cd '0-9' | cut -c1-12
|
||||
fi
|
||||
}
|
||||
|
||||
# Caches for the containerized linter, private to THIS worktree.
|
||||
#
|
||||
# Keeping them out of the repo and persisting them between runs is what
|
||||
# keeps the inner loop fast: a warm run costs about the same as a native
|
||||
# one plus container startup. Keeping them keyed on the worktree path is
|
||||
# what keeps them correct. One shared cache for the whole repo was the
|
||||
# defect in issue #99: two worktrees of this repo have identical file
|
||||
# contents, so their cache keys collide, and golangci-lint replays the
|
||||
# stored results - including the file paths recorded when they were
|
||||
# produced. That silently reports one worktree's findings, or one
|
||||
# worktree's clean bill of health, for another.
|
||||
cache_dir() {
|
||||
slug="$(printf '%s' "$(basename "$ROOT")" | tr -c 'A-Za-z0-9._-' '-')"
|
||||
echo "$(cache_home)/$slug-$(path_digest)"
|
||||
}
|
||||
|
||||
# One cache per worktree means throwaway worktrees would otherwise leave
|
||||
# caches behind forever. Each cache records the worktree it belongs to,
|
||||
# and any cache whose worktree no longer exists is collected here, so
|
||||
# growth is bounded by the number of worktrees that actually exist. The
|
||||
# whole tree also sits under XDG_CACHE_HOME (~/.cache by default), so it
|
||||
# is disposable by definition: `rm -rf "${XDG_CACHE_HOME:-~/.cache}/vaultik-lint"`
|
||||
# costs nothing but the next run's cold cache.
|
||||
prune_dead_caches() {
|
||||
home="$(cache_home)"
|
||||
if [ ! -d "$home" ]; then
|
||||
return 0
|
||||
fi
|
||||
for dir in "$home"/*; do
|
||||
if [ ! -f "$dir/worktree" ]; then
|
||||
continue
|
||||
fi
|
||||
owner="$(cat "$dir/worktree")"
|
||||
if [ -z "$owner" ]; then
|
||||
continue
|
||||
fi
|
||||
if [ ! -d "$owner" ]; then
|
||||
# The Go module cache inside is deliberately read-only, and
|
||||
# rm(1) cannot unlink a file out of a directory it may not
|
||||
# write, so the tree has to be made writable first. And
|
||||
# failing to tidy up is a housekeeping problem, never a
|
||||
# reason to fail a lint: without the fallback below, `set
|
||||
# -e` turns a stale cache that will not delete into a lint
|
||||
# error, which is a gate failing for a reason that has
|
||||
# nothing to do with the code. (Observed, not theorised.)
|
||||
chmod -R u+w "$dir" 2>/dev/null || true
|
||||
if ! rm -rf "$dir" 2>/dev/null; then
|
||||
# Restore the marker on a partial removal: an
|
||||
# unmarked leftover would be skipped by every future
|
||||
# run and never collected.
|
||||
mkdir -p "$dir" 2>/dev/null || true
|
||||
echo "$owner" >"$dir/worktree" 2>/dev/null || true
|
||||
echo "lint: could not remove stale cache $dir" >&2
|
||||
fi
|
||||
fi
|
||||
done
|
||||
}
|
||||
|
||||
prepare_cache() {
|
||||
cache="$1"
|
||||
mkdir -p "$cache/go-build" "$cache/go-mod" "$cache/golangci-lint"
|
||||
echo "$ROOT" >"$cache/worktree"
|
||||
}
|
||||
|
||||
# Run the linter, wherever it is that this script is allowed to run it.
|
||||
run_linter() {
|
||||
if in_lint_container; then
|
||||
golangci-lint run "$@" ./...
|
||||
return $?
|
||||
fi
|
||||
|
||||
docker run --rm \
|
||||
--user "$(id -u):$(id -g)" \
|
||||
--env HOME=/tmp \
|
||||
--env GOFLAGS=-buildvcs=false \
|
||||
--env GOCACHE=/cache/go-build \
|
||||
--env GOMODCACHE=/cache/go-mod \
|
||||
--env GOLANGCI_LINT_CACHE=/cache/golangci-lint \
|
||||
--volume "$ROOT:/src" \
|
||||
--volume "$CACHE:/cache" \
|
||||
--workdir /src \
|
||||
"$IMAGE" \
|
||||
golangci-lint run "$@" ./...
|
||||
}
|
||||
|
||||
# Run the linter, streaming its combined output while also capturing it,
|
||||
# and hand back its exit status. The output has to be inspected before
|
||||
# it is believed, which is why this script no longer just execs the
|
||||
# linter. `tee` would swallow the status, so it is smuggled out through
|
||||
# a file: there is no pipefail in POSIX sh.
|
||||
run_capture() {
|
||||
capture="$1"
|
||||
shift
|
||||
rm -f "$capture.status"
|
||||
{
|
||||
rc=0
|
||||
# `set -e` is in force inside this subshell too, so the status
|
||||
# has to be caught here: an unguarded non-zero exit (which is
|
||||
# what "the linter found something" looks like) would abort the
|
||||
# subshell before the status was ever written.
|
||||
run_linter "$@" 2>&1 || rc=$?
|
||||
echo "$rc" >"$capture.status"
|
||||
} | tee "$capture"
|
||||
|
||||
if [ ! -s "$capture.status" ]; then
|
||||
echo "lint: the linter did not report an exit status" >&2
|
||||
exit 1
|
||||
fi
|
||||
read -r captured_status <"$capture.status"
|
||||
rm -f "$capture.status"
|
||||
return "$captured_status"
|
||||
}
|
||||
|
||||
# Reject output that cannot describe this tree. See script/lint-audit
|
||||
# for what that means and why: in short, a finding citing a file that is
|
||||
# not here means the result being reported was produced somewhere else,
|
||||
# and a PASS built out of another checkout's analysis is silent (issue
|
||||
# #99). The audit therefore runs on clean output as well.
|
||||
audit_output() {
|
||||
capture="$1"
|
||||
if ! "$ROOT/script/lint-audit" "$capture"; then
|
||||
if [ -n "$CACHE" ]; then
|
||||
echo " this tree's lint cache: $CACHE" >&2
|
||||
fi
|
||||
exit 1
|
||||
fi
|
||||
script/lint takes no arguments. The linter runs as a build step, so
|
||||
there is no command line to pass flags to; anything accepted here would
|
||||
have to be silently dropped. To apply autofixes, use script/lint-fix,
|
||||
which runs the same pinned image as a container for exactly this
|
||||
reason.
|
||||
EOF
|
||||
exit 2
|
||||
}
|
||||
|
||||
main() {
|
||||
[ "$#" -eq 0 ] || usage
|
||||
|
||||
cd "$ROOT"
|
||||
require_docker
|
||||
|
||||
IMAGE="$(lint_image)"
|
||||
if [ -z "$IMAGE" ]; then
|
||||
echo "lint: no lint stage found in $DOCKERFILE" >&2
|
||||
exit 1
|
||||
fi
|
||||
# A fresh epoch per invocation is what forces the lint layer to
|
||||
# execute; the layers above the ARG in Dockerfile.lint still cache,
|
||||
# so a run is not cold. The value must be unique per invocation, not
|
||||
# per second: `date +%s` is second-granular, so two concurrent
|
||||
# invocations in the same second would get identical epochs and the
|
||||
# later one could be served from cache -- the false green in
|
||||
# miniature. `%N` alone does not fix it either, because busybox
|
||||
# silently drops %N, exits 0, and hands back second granularity with
|
||||
# no warning. `$$` is what makes this correct regardless, since
|
||||
# concurrent invocations have different pids.
|
||||
#
|
||||
# Assign it on its own line rather than inline in the argument.
|
||||
# Under `set -eu` a command substitution that fails inside an
|
||||
# argument does NOT abort the script: CHECK_EPOCH would become an
|
||||
# empty string, an empty string is a constant, and a constant epoch
|
||||
# is exactly the cached-lint false green this guards against. As a
|
||||
# bare assignment, `set -e` catches a failing `date` and no build
|
||||
# starts.
|
||||
epoch="$(date +%s%N)$$"
|
||||
|
||||
CACHE=""
|
||||
if in_lint_container; then
|
||||
# No docker daemon in here, so there is no fallback: a mismatch
|
||||
# is a hard error rather than a quiet substitution.
|
||||
installed="$(installed_version)"
|
||||
pinned="$(pinned_version)"
|
||||
if [ -z "$installed" ] || [ "$installed" != "$pinned" ]; then
|
||||
cat >&2 <<EOF
|
||||
lint: VAULTIK_LINT_IN_CONTAINER is set, so this is expected to be
|
||||
running inside the Dockerfile's pinned lint image, but the golangci-lint
|
||||
on PATH does not match the pin.
|
||||
|
||||
pinned: $pinned ($IMAGE)
|
||||
installed: ${installed:-<none>}
|
||||
EOF
|
||||
exit 1
|
||||
fi
|
||||
else
|
||||
require_docker "$IMAGE"
|
||||
prune_dead_caches
|
||||
CACHE="$(cache_dir)"
|
||||
prepare_cache "$CACHE"
|
||||
fi
|
||||
|
||||
capture="$(mktemp "${TMPDIR:-/tmp}/vaultik-lint.XXXXXX")"
|
||||
trap 'rm -f "$capture" "$capture.status"' EXIT HUP INT TERM
|
||||
|
||||
attempt=1
|
||||
while :; do
|
||||
status=0
|
||||
run_capture "$capture" "$@" || status=$?
|
||||
|
||||
if grep -Fq "$LOCK_MESSAGE" "$capture"; then
|
||||
if [ "$attempt" -lt "$LOCK_ATTEMPTS" ]; then
|
||||
echo "lint: another golangci-lint holds the lock;" \
|
||||
"retrying in ${LOCK_SLEEP}s" \
|
||||
"(attempt $attempt of $LOCK_ATTEMPTS)" >&2
|
||||
sleep "$LOCK_SLEEP"
|
||||
attempt=$((attempt + 1))
|
||||
continue
|
||||
fi
|
||||
cat >&2 <<EOF
|
||||
|
||||
lint: gave up after $LOCK_ATTEMPTS attempts, each blocked by another
|
||||
golangci-lint holding the cross-process lock. This is NOT a lint
|
||||
verdict: the tree was never analysed. Re-run when the other run has
|
||||
finished.
|
||||
EOF
|
||||
exit 1
|
||||
fi
|
||||
|
||||
audit_output "$capture"
|
||||
exit "$status"
|
||||
done
|
||||
# cacheonly: the lint verdict is the build's exit status, and the
|
||||
# image it would otherwise produce is never run. Exporting it costs
|
||||
# most of the wall time of a warm run and leaves a dangling image
|
||||
# behind on every invocation, on a host that may be running many.
|
||||
docker build \
|
||||
--output=type=cacheonly \
|
||||
--build-arg CHECK_EPOCH="$epoch" \
|
||||
-f "$DOCKERFILE" \
|
||||
"$ROOT"
|
||||
}
|
||||
|
||||
main "$@"
|
||||
|
||||
@@ -1,113 +0,0 @@
|
||||
#!/bin/sh
|
||||
# script/lint-audit: audit a captured golangci-lint run for output that
|
||||
# cannot describe this tree. Called by script/lint on every run; usable
|
||||
# on its own against any saved lint output.
|
||||
#
|
||||
# script/lint-audit <capture-file>
|
||||
#
|
||||
# Exits 0 when every finding cites a file in this tree, 1 when any does
|
||||
# not. It NEVER certifies that a lint run passed - it has no idea
|
||||
# whether the run found issues, and does not look. It only rejects
|
||||
# output that is impossible for this tree, which is a different and much
|
||||
# weaker claim. Do not use it as a gate; use script/lint.
|
||||
#
|
||||
# Why this exists (issue #99): golangci-lint caches analysis results,
|
||||
# and a cache shared between two checkouts of this repo can serve one
|
||||
# checkout's stored findings for another, file paths included. The
|
||||
# failure is symmetric and only one direction is loud - a clean tree
|
||||
# failed by a dirty sibling gets investigated, while a dirty tree passed
|
||||
# by a clean sibling is silent. This turns the silent direction into a
|
||||
# hard error, which is why it runs on clean output too.
|
||||
#
|
||||
# The primary fix is that script/lint now keys its cache on the worktree
|
||||
# path so the collision cannot happen. This is the backstop, because a
|
||||
# backstop that only runs when we already believe things are fine is
|
||||
# worth more than one more assumption.
|
||||
set -eu
|
||||
|
||||
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
||||
|
||||
# Where script/lint bind-mounts the tree inside the pinned image. A
|
||||
# containerized run that prints absolute paths (`--path-mode abs`)
|
||||
# prints them under this, so they are this tree's files under another
|
||||
# name. Note the consequence, and why the cache key rather than this
|
||||
# check is the real fix: two containerized runs of different checkouts
|
||||
# both call themselves /src, so contamination between two container
|
||||
# runs is not distinguishable by path alone.
|
||||
CONTAINER_ROOT="/src"
|
||||
|
||||
usage() {
|
||||
echo "usage: $(basename "$0") <capture-file>" >&2
|
||||
exit 2
|
||||
}
|
||||
|
||||
# Every path cited by a finding that is not a file in this tree.
|
||||
#
|
||||
# The linter runs with the tree root as its working directory, so a
|
||||
# legitimate finding cites either a relative path that resolves inside
|
||||
# the tree or an absolute path under the root. A path that escapes
|
||||
# (absolute and elsewhere, or with a `..` component) or that names a
|
||||
# file which is not here describes something this run did not analyse.
|
||||
foreign_paths() {
|
||||
capture="$1"
|
||||
awk -F: '$1 ~ /\.go$/ && $2 ~ /^[0-9]+$/ { print $1 }' "$capture" |
|
||||
sort -u |
|
||||
while IFS= read -r path; do
|
||||
case "$path" in
|
||||
"$ROOT"/*)
|
||||
path="${path#"$ROOT"/}"
|
||||
;;
|
||||
"$CONTAINER_ROOT"/*)
|
||||
path="${path#"$CONTAINER_ROOT"/}"
|
||||
;;
|
||||
/*)
|
||||
printf '%s\n' "$path"
|
||||
continue
|
||||
;;
|
||||
../* | */../*)
|
||||
printf '%s\n' "$path"
|
||||
continue
|
||||
;;
|
||||
esac
|
||||
if [ ! -e "$ROOT/$path" ]; then
|
||||
printf '%s\n' "$path"
|
||||
fi
|
||||
done
|
||||
}
|
||||
|
||||
main() {
|
||||
[ "$#" -eq 1 ] || usage
|
||||
capture="$1"
|
||||
if [ ! -f "$capture" ]; then
|
||||
echo "lint-audit: no such capture file: $capture" >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
foreign="$(foreign_paths "$capture")"
|
||||
if [ -z "$foreign" ]; then
|
||||
exit 0
|
||||
fi
|
||||
|
||||
cat >&2 <<EOF
|
||||
|
||||
lint: REJECTED - the linter reported findings for files that are not in
|
||||
this tree, so its output does not describe the tree that was linted.
|
||||
This result is void, whichever way it went: a pass here would be a pass
|
||||
earned by analysing someone else's code.
|
||||
|
||||
tree: $ROOT
|
||||
|
||||
Paths reported that are not in this tree:
|
||||
EOF
|
||||
printf '%s\n' "$foreign" | sed -e 's/^/ /' >&2
|
||||
cat >&2 <<EOF
|
||||
|
||||
This is the signature of analysis replayed from a cache belonging to
|
||||
another checkout (issue #99). Clear this tree's lint cache and re-run:
|
||||
|
||||
rm -rf "\${XDG_CACHE_HOME:-\$HOME/.cache}/vaultik-lint"
|
||||
EOF
|
||||
exit 1
|
||||
}
|
||||
|
||||
main "$@"
|
||||
@@ -1,18 +1,54 @@
|
||||
#!/bin/sh
|
||||
# script/lint-fix: run the linter's autofixer. Rewrites files in place
|
||||
# for every finding the enabled linters know how to fix; findings
|
||||
# without an autofix are reported but left alone (exit status is
|
||||
# nonzero while any remain).
|
||||
# without an autofix are reported but left alone.
|
||||
#
|
||||
# Delegates to script/lint so the autofixer is the same pinned linter
|
||||
# version that script/lint and CI use - fixes written by a different
|
||||
# version are not necessarily fixes for the version that gates.
|
||||
# THIS IS A DEVELOPER CONVENIENCE AND NEVER A GATE. Nothing in
|
||||
# script/check, script/precommit or script/cibuild calls it, and no gate
|
||||
# reads its exit status. The gate is script/lint, which builds
|
||||
# Dockerfile.lint; run that afterwards to find out whether the tree is
|
||||
# actually clean.
|
||||
#
|
||||
# Unlike script/lint this cannot be a build step: a build step writes
|
||||
# into an image, and fixes have to land in the worktree. So it runs the
|
||||
# same pinned image as a container with the tree bind-mounted, which
|
||||
# means it needs a LOCAL docker daemon -- a remote daemon has no access
|
||||
# to these files, and this script will appear to do nothing there. The
|
||||
# image reference is parsed out of Dockerfile.lint's FROM line, so the
|
||||
# autofixer is always the same version as the linter that gates; fixes
|
||||
# written by a different version are not necessarily fixes for the
|
||||
# version that decides.
|
||||
set -eu
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
|
||||
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
||||
DOCKERFILE="$ROOT/Dockerfile.lint"
|
||||
|
||||
# The image reference from Dockerfile.lint, tag and digest included.
|
||||
lint_image() {
|
||||
awk '$1 == "FROM" { print $2; exit }' "$DOCKERFILE"
|
||||
}
|
||||
|
||||
main() {
|
||||
exec "$SCRIPT_DIR/lint" --fix "$@"
|
||||
cd "$ROOT"
|
||||
|
||||
image="$(lint_image)"
|
||||
if [ -z "$image" ]; then
|
||||
echo "lint-fix: no FROM line found in $DOCKERFILE" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Run as the invoking user so the rewritten files stay owned by
|
||||
# them. HOME is set because the Go and golangci-lint caches default
|
||||
# under it and that user has no home inside the container; those
|
||||
# caches are per-container and discarded with it.
|
||||
docker run --rm \
|
||||
--user "$(id -u):$(id -g)" \
|
||||
--env HOME=/tmp \
|
||||
--env GOFLAGS=-buildvcs=false \
|
||||
--volume "$ROOT:/src" \
|
||||
--workdir /src \
|
||||
"$image" \
|
||||
golangci-lint run --config .golangci.yml --fix "$@" ./...
|
||||
}
|
||||
|
||||
main "$@"
|
||||
|
||||
Reference in New Issue
Block a user