Compare commits
2
Commits
next
..
90229152cc
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
90229152cc | ||
|
|
781fb415ff |
+29
-83
@@ -1,84 +1,30 @@
|
|||||||
# .dockerignore does NOT use .gitignore semantics. Docker matches with
|
# .git is sent so the build can derive the version it stamps into the binary
|
||||||
# moby/patternmatcher: filepath.Match plus `**`, so `*` does not cross
|
# (script/version). Its config, which can hold a remote URL carrying a
|
||||||
# `/` and an unprefixed pattern is anchored at the context root. Every
|
# credential and which `git describe` does not need, is left out of a
|
||||||
# depth-independent pattern therefore needs `**/`, or `config/.env` and
|
# directory context. A context sent as a tar is not filtered by this file, so
|
||||||
# `certs/server.key` still ship while this file reads as solved. Only
|
# it carries .git/config unless its sender leaves it out.
|
||||||
# genuinely root-anchored entries go unprefixed. Never transplant these
|
.git/config
|
||||||
# into .gitignore, where `**/` is wrong.
|
|
||||||
|
# No tracked file may be listed here: git in the build would see it as
|
||||||
|
# deleted and mark the version -dirty.
|
||||||
#
|
#
|
||||||
# Matching is case-sensitive, so secrets use character ranges rather
|
# .ci-fingerprint is deliberately NOT excluded: it is the CI cache barrier
|
||||||
# than an ALL-CAPS twin, which would still miss `Server.Key`.
|
# that keeps the check stages from replaying a cached pass. See the lint
|
||||||
#
|
# stage of the Dockerfile.
|
||||||
# Extend with this repo's own host-built artifacts, written anchored:
|
bin/
|
||||||
# `/myapp`, never `**/myapp`, which also matches `cmd/myapp/` and
|
# Extracted from 3p/ by `make assets` inside the build; a host copy is not
|
||||||
# deletes the package directory from the context.
|
# needed. The tarball in 3p/ must stay in the context.
|
||||||
|
static/js/alpine.min.js
|
||||||
# .git is sent without its config. Without a VERSION build argument the
|
# The js-deps stage installs ESLint and prettier; a host copy would overwrite
|
||||||
# stage that compiles runs `git describe --tags --always` on .git, which
|
# them at the `COPY . .` of the stages built on it.
|
||||||
# does not need .git/config; that file can hold a credential, such as a
|
node_modules/
|
||||||
# password in a remote URL or the token the CI checkout step stores there.
|
.env
|
||||||
# Each submodule keeps a config with the same exposure in its git directory
|
.env.*
|
||||||
# under .git/modules/, nested again for a submodule's own submodules, or in
|
*.db
|
||||||
# its own .git directory when it keeps one.
|
*.sqlite
|
||||||
# KNOWN GAP: a submodule whose name has a `config` segment (`config`,
|
*.sqlite3
|
||||||
# `deploy/config`, `config/lib`) loses its whole git directory, because
|
.DS_Store
|
||||||
# `**/.git/modules/**/config` also matches that segment's directory
|
.idea/
|
||||||
# under .git/modules/. Go's version stamping then fails the build;
|
.vscode/
|
||||||
# nothing leaks. Name such a submodule without that segment:
|
tmp/
|
||||||
# `git submodule add --name`.
|
temp/
|
||||||
**/.git/config
|
|
||||||
**/.git/modules/**/config
|
|
||||||
|
|
||||||
# Agent scratch: one full checkout of the repo per in-flight agent.
|
|
||||||
# Anchored because it occurs once where agents run at the repo root.
|
|
||||||
# KNOWN GAP: a repo running agents in subdirectories still ships
|
|
||||||
# `services/api/.claude/` and must add its own anchored entry.
|
|
||||||
.claude
|
|
||||||
|
|
||||||
# Environment files. `*.env` covers bare `.env` and the `prod.env`
|
|
||||||
# convention. Re-include a committed template with a negation if the
|
|
||||||
# build needs one: `!docs/example.env`.
|
|
||||||
**/*.[eE][nN][vV]
|
|
||||||
**/.[eE][nN][vV].*
|
|
||||||
**/.[eE][nN][vV][rR][cC]
|
|
||||||
|
|
||||||
# Private keys and the bundles carrying them. Public certificates
|
|
||||||
# (*.crt, *.cer) are deliberately absent: they are legitimate inputs.
|
|
||||||
**/*.[pP][eE][mM]
|
|
||||||
**/*.[kK][eE][yY]
|
|
||||||
**/*.[pP]12
|
|
||||||
**/*.[pP][fF][xX]
|
|
||||||
**/[iI][dD]_[rR][sS][aA]
|
|
||||||
**/[iI][dD]_[dD][sS][aA]
|
|
||||||
**/[iI][dD]_[eE][cC][dD][sS][aA]
|
|
||||||
**/[iI][dD]_[eE][cC][dD][sS][aA]_[sS][kK]
|
|
||||||
**/[iI][dD]_[eE][dD]25519
|
|
||||||
**/[iI][dD]_[eE][dD]25519_[sS][kK]
|
|
||||||
|
|
||||||
# Dependencies: restored inside the image, never copied in.
|
|
||||||
**/node_modules
|
|
||||||
|
|
||||||
# OS metadata.
|
|
||||||
**/.DS_Store
|
|
||||||
**/Thumbs.db
|
|
||||||
|
|
||||||
# Editor state: never a build input, and it churns COPY.
|
|
||||||
**/*.swp
|
|
||||||
**/*.swo
|
|
||||||
**/*~
|
|
||||||
**/*.bak
|
|
||||||
**/.idea
|
|
||||||
**/.vscode
|
|
||||||
**/*.sublime-*
|
|
||||||
|
|
||||||
# This repository's own host-built artifacts: the binary `make build`
|
|
||||||
# writes, and the Alpine.js file `make assets` extracts from 3p/ (the
|
|
||||||
# build extracts its own).
|
|
||||||
/bin
|
|
||||||
/static/js/alpine.min.js
|
|
||||||
|
|
||||||
# SQLite databases, which hold the session key and webhook payloads, at
|
|
||||||
# any depth.
|
|
||||||
**/*.db
|
|
||||||
**/*.sqlite
|
|
||||||
**/*.sqlite3
|
|
||||||
|
|||||||
@@ -10,6 +10,3 @@ insert_final_newline = true
|
|||||||
|
|
||||||
[Makefile]
|
[Makefile]
|
||||||
indent_style = tab
|
indent_style = tab
|
||||||
|
|
||||||
[*.go]
|
|
||||||
indent_style = tab
|
|
||||||
|
|||||||
@@ -1,9 +1,37 @@
|
|||||||
name: check
|
name: check
|
||||||
on: [push]
|
|
||||||
|
on:
|
||||||
|
push:
|
||||||
|
branches:
|
||||||
|
- '**'
|
||||||
|
|
||||||
jobs:
|
jobs:
|
||||||
check:
|
check:
|
||||||
runs-on: ubuntu-latest
|
runs-on: ubuntu-latest
|
||||||
steps:
|
steps:
|
||||||
# actions/checkout v4.2.2, 2026-02-22
|
- name: Checkout
|
||||||
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683
|
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 2024-10-23
|
||||||
- run: script/cibuild
|
with:
|
||||||
|
# The superseded-status step needs history to walk ancestors (it
|
||||||
|
# aborts on a shallow clone).
|
||||||
|
fetch-depth: 0
|
||||||
|
|
||||||
|
- name: Mark superseded run statuses
|
||||||
|
# Gitea cancels the in-flight run when another commit is pushed to the
|
||||||
|
# same branch and records the cancellation as `failure`, so a commit
|
||||||
|
# that was never tested reads as a test result. The script rewrites
|
||||||
|
# those statuses to say what happened. See its header for why the
|
||||||
|
# state stays `failure` and not `skipped`.
|
||||||
|
env:
|
||||||
|
GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
|
||||||
|
run: script/ci-mark-superseded
|
||||||
|
|
||||||
|
- name: Fingerprint the build context
|
||||||
|
# Writes the hash of the commit being checked into the context, which
|
||||||
|
# invalidates the `COPY . .` layer of every check stage: a commit
|
||||||
|
# that was never linted, format-checked, stylesheet-checked, tested
|
||||||
|
# and built cannot report success from cache.
|
||||||
|
run: git rev-parse HEAD > .ci-fingerprint
|
||||||
|
|
||||||
|
- name: Build Docker image (runs the gofmt check, golangci-lint, the stylesheet check, ESLint, the Markdown check, make test, make build)
|
||||||
|
run: script/cibuild
|
||||||
|
|||||||
+21
-50
@@ -1,53 +1,3 @@
|
|||||||
# OS
|
|
||||||
.DS_Store
|
|
||||||
Thumbs.db
|
|
||||||
|
|
||||||
# Editors
|
|
||||||
*.swp
|
|
||||||
*.swo
|
|
||||||
*~
|
|
||||||
*.bak
|
|
||||||
.idea/
|
|
||||||
.vscode/
|
|
||||||
*.sublime-*
|
|
||||||
|
|
||||||
# Agent scratch (worktrees of this repo, created and destroyed by
|
|
||||||
# in-flight tooling). Unanchored: .gitignore patterns already match at
|
|
||||||
# every depth, so no prefix is wanted here. This is not a .dockerignore
|
|
||||||
# entry and must not be given a `**/` prefix on the way into one.
|
|
||||||
.claude/
|
|
||||||
|
|
||||||
# Node
|
|
||||||
node_modules/
|
|
||||||
|
|
||||||
# Secrets. Unanchored like every entry above, so each matches at every
|
|
||||||
# depth. Matching is case-sensitive on Linux, so names use character
|
|
||||||
# ranges rather than a lowercase form that misses `Server.Key`.
|
|
||||||
|
|
||||||
# Environment files. `*.env` covers bare `.env` and the `prod.env`
|
|
||||||
# convention. Only the templates `example.env` and `sample.env` are
|
|
||||||
# re-included below. A repository that commits any other template adds
|
|
||||||
# its own negation after these lines, for example `!.env.example`.
|
|
||||||
*.[eE][nN][vV]
|
|
||||||
.[eE][nN][vV].*
|
|
||||||
.[eE][nN][vV][rR][cC]
|
|
||||||
!example.env
|
|
||||||
!sample.env
|
|
||||||
|
|
||||||
# Private keys and the bundles carrying them.
|
|
||||||
*.[pP][eE][mM]
|
|
||||||
*.[kK][eE][yY]
|
|
||||||
*.[pP]12
|
|
||||||
*.[pP][fF][xX]
|
|
||||||
[iI][dD]_[rR][sS][aA]
|
|
||||||
[iI][dD]_[dD][sS][aA]
|
|
||||||
[iI][dD]_[eE][cC][dD][sS][aA]
|
|
||||||
[iI][dD]_[eE][cC][dD][sS][aA]_[sS][kK]
|
|
||||||
[iI][dD]_[eE][dD]25519
|
|
||||||
[iI][dD]_[eE][dD]25519_[sS][kK]
|
|
||||||
|
|
||||||
# This repository's own entries, after the shared content above.
|
|
||||||
|
|
||||||
# Binaries
|
# Binaries
|
||||||
*.exe
|
*.exe
|
||||||
*.dll
|
*.dll
|
||||||
@@ -65,6 +15,24 @@ bin/
|
|||||||
# Go vendor directory
|
# Go vendor directory
|
||||||
vendor/
|
vendor/
|
||||||
|
|
||||||
|
# ESLint, prettier and their dependencies, installed from yarn.lock
|
||||||
|
node_modules/
|
||||||
|
|
||||||
|
# IDE specific files
|
||||||
|
.idea/
|
||||||
|
*.swp
|
||||||
|
*.swo
|
||||||
|
*~
|
||||||
|
.vscode/
|
||||||
|
|
||||||
|
# OS specific files
|
||||||
|
.DS_Store
|
||||||
|
Thumbs.db
|
||||||
|
|
||||||
|
# Environment and config files
|
||||||
|
.env
|
||||||
|
.env.local
|
||||||
|
|
||||||
# Data directory (SQLite databases)
|
# Data directory (SQLite databases)
|
||||||
data/
|
data/
|
||||||
*.db
|
*.db
|
||||||
@@ -78,6 +46,9 @@ data/
|
|||||||
tmp/
|
tmp/
|
||||||
temp/
|
temp/
|
||||||
|
|
||||||
|
# CI cache barrier, written into the build context by the check workflow
|
||||||
|
.ci-fingerprint
|
||||||
|
|
||||||
# Alpine.js, extracted by `make assets` from its tarball in 3p/, which is
|
# Alpine.js, extracted by `make assets` from its tarball in 3p/, which is
|
||||||
# what is committed.
|
# what is committed.
|
||||||
/static/js/alpine.min.js
|
/static/js/alpine.min.js
|
||||||
|
|||||||
+2
-67
@@ -10,21 +10,14 @@ run:
|
|||||||
|
|
||||||
linters:
|
linters:
|
||||||
default: all
|
default: all
|
||||||
enable:
|
|
||||||
# Successor to the deprecated gomodguard. Named explicitly, rather than
|
|
||||||
# left to `default: all`, because it carries the module policy below.
|
|
||||||
- gomodguard_v2
|
|
||||||
disable:
|
disable:
|
||||||
# Genuinely incompatible with project patterns
|
# Genuinely incompatible with project patterns
|
||||||
- exhaustruct # Requires all struct fields
|
- exhaustruct # Requires all struct fields
|
||||||
- exhaustruct_v5 # Requires all struct fields (successor to exhaustruct)
|
- depguard # Dependency allow/block lists
|
||||||
- godot # Requires comments to end with periods
|
- godot # Requires comments to end with periods
|
||||||
|
- wsl # Deprecated, replaced by wsl_v5
|
||||||
- wrapcheck # Too verbose for internal packages
|
- wrapcheck # Too verbose for internal packages
|
||||||
- varnamelen # Short names like db, id are idiomatic Go
|
- varnamelen # Short names like db, id are idiomatic Go
|
||||||
# Deprecated: the warning is attached to the old name, so it is
|
|
||||||
# silenced by disabling that name, not by enabling the successor.
|
|
||||||
- wsl # Deprecated, replaced by wsl_v5
|
|
||||||
- gomodguard # Deprecated, replaced by gomodguard_v2
|
|
||||||
settings:
|
settings:
|
||||||
lll:
|
lll:
|
||||||
line-length: 88
|
line-length: 88
|
||||||
@@ -35,64 +28,6 @@ linters:
|
|||||||
max-complexity: 15
|
max-complexity: 15
|
||||||
dupl:
|
dupl:
|
||||||
threshold: 100
|
threshold: 100
|
||||||
depguard:
|
|
||||||
# Test-support code must not be compiled into the shipped binary. A
|
|
||||||
# test-support package exists to hand a test privileges the program
|
|
||||||
# itself must never have, so a file that is not a test must not import
|
|
||||||
# one. Test files, and the files inside a package whose directory name
|
|
||||||
# ends in `test`, are where that code belongs, and are exempt.
|
|
||||||
#
|
|
||||||
# The deny list below is the one part of this file a repository is
|
|
||||||
# expected to extend, and the only part it may. depguard matches an
|
|
||||||
# import path against a list of prefixes, so it cannot be told "any path
|
|
||||||
# whose last segment ends in test"; a repository's own test-support
|
|
||||||
# packages have to be named here one at a time, by full import path,
|
|
||||||
# under a module path that differs from repository to repository. Add
|
|
||||||
# them; change nothing else.
|
|
||||||
rules:
|
|
||||||
test-support:
|
|
||||||
list-mode: lax
|
|
||||||
files:
|
|
||||||
- "$all"
|
|
||||||
- "!$test"
|
|
||||||
- "!**/*test/**"
|
|
||||||
deny:
|
|
||||||
- pkg: net/http/httptest
|
|
||||||
desc: >-
|
|
||||||
Test-support code belongs in test files and in packages whose
|
|
||||||
directory name ends in test, not in the shipped binary.
|
|
||||||
# Only decisions already recorded in the Go package defaults are
|
|
||||||
# listed here. Every entry matches the module path exactly.
|
|
||||||
gomodguard_v2:
|
|
||||||
blocked:
|
|
||||||
- module: github.com/rs/zerolog
|
|
||||||
recommendations:
|
|
||||||
- log/slog
|
|
||||||
reason: "Structured logging is stdlib log/slog."
|
|
||||||
# One entry per pre-fork module path, because the later releases
|
|
||||||
# are separate paths. A prefix match would be shorter but would
|
|
||||||
# also reach github.com/go-redis/redismock, the test double for
|
|
||||||
# the successor these entries recommend.
|
|
||||||
- module: github.com/go-redis/redis
|
|
||||||
recommendations:
|
|
||||||
- github.com/redis/go-redis/v9
|
|
||||||
reason: "Pre-fork module; use the maintained go-redis v9."
|
|
||||||
- module: github.com/go-redis/redis/v7
|
|
||||||
recommendations:
|
|
||||||
- github.com/redis/go-redis/v9
|
|
||||||
reason: "Pre-fork module; use the maintained go-redis v9."
|
|
||||||
- module: github.com/go-redis/redis/v8
|
|
||||||
recommendations:
|
|
||||||
- github.com/redis/go-redis/v9
|
|
||||||
reason: "Pre-fork module; use the maintained go-redis v9."
|
|
||||||
- module: github.com/sergi/go-diff
|
|
||||||
recommendations:
|
|
||||||
- github.com/aymanbagabas/go-udiff
|
|
||||||
reason: "No unified diff output; use go-udiff."
|
|
||||||
- module: github.com/hexops/gotextdiff
|
|
||||||
recommendations:
|
|
||||||
- github.com/aymanbagabas/go-udiff
|
|
||||||
reason: "Unmaintained fork; use go-udiff."
|
|
||||||
|
|
||||||
issues:
|
issues:
|
||||||
max-issues-per-linter: 0
|
max-issues-per-linter: 0
|
||||||
|
|||||||
@@ -1,2 +0,0 @@
|
|||||||
node_modules/
|
|
||||||
yarn.lock
|
|
||||||
@@ -1,3 +0,0 @@
|
|||||||
# Install into node_modules/: the Dockerfile's lint and Markdown stages run
|
|
||||||
# ESLint and prettier from node_modules/.bin.
|
|
||||||
nodeLinker: node-modules
|
|
||||||
+63
-121
@@ -1,3 +1,36 @@
|
|||||||
|
# Lint stage
|
||||||
|
# golangci/golangci-lint:v2.12.2 (Debian-based), 2026-08-07
|
||||||
|
# Using Debian-based image because mattn/go-sqlite3 (CGO) does not
|
||||||
|
# compile on Alpine musl (off64_t is a glibc type).
|
||||||
|
FROM golangci/golangci-lint:v2.12.2@sha256:5cceeef04e53efe1470638d4b4b4f5ceefd574955ab3941b2d9a68a8c9ad5240 AS lint
|
||||||
|
|
||||||
|
RUN apt-get update && apt-get install -y --no-install-recommends make && rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
|
WORKDIR /src
|
||||||
|
|
||||||
|
# Copy go mod files first for better layer caching
|
||||||
|
COPY go.mod go.sum ./
|
||||||
|
RUN go mod download
|
||||||
|
|
||||||
|
# Copy source code. In CI the context also carries .ci-fingerprint, which
|
||||||
|
# holds the hash of the commit being checked (see
|
||||||
|
# .gitea/workflows/check.yml). That invalidates this layer, so the checks
|
||||||
|
# below cannot report success by replaying a cached pass. Do not add it to
|
||||||
|
# .dockerignore.
|
||||||
|
COPY . .
|
||||||
|
|
||||||
|
# Run the Go formatting check and the linter. gofmt and golangci-lint are
|
||||||
|
# invoked directly rather than through `make fmt-check` and `make lint`: this
|
||||||
|
# stage is already the pinned linter image, and both scripts build docker
|
||||||
|
# stages, so calling them here would need a docker daemon inside the build.
|
||||||
|
# The Markdown half of `make fmt-check` is the markdown-check stage below.
|
||||||
|
# Keep the golangci-lint steps in step with Dockerfile.lint, including
|
||||||
|
# --network=none (see its header for why).
|
||||||
|
RUN if [ -n "$(gofmt -s -l .)" ]; then echo "gofmt needed on:"; gofmt -s -l .; exit 1; fi
|
||||||
|
RUN script/assets
|
||||||
|
RUN --network=none golangci-lint config verify --config .golangci.yml
|
||||||
|
RUN --network=none golangci-lint run --config .golangci.yml --build-tags browser ./...
|
||||||
|
|
||||||
# Stylesheet stages. static/css/tailwind.css is generated, by this pinned
|
# Stylesheet stages. static/css/tailwind.css is generated, by this pinned
|
||||||
# tailwindcss, from static/css/input.css and the files its @source lines
|
# tailwindcss, from static/css/input.css and the files its @source lines
|
||||||
# name. `make css` (script/css) writes it out from the css-output stage.
|
# name. `make css` (script/css) writes it out from the css-output stage.
|
||||||
@@ -36,17 +69,15 @@ RUN sed 's/}/}\n/g' static/css/tailwind.css > /tmp/committed.css \
|
|||||||
|
|
||||||
# JavaScript lint stages: ESLint, at the version package.json and yarn.lock
|
# JavaScript lint stages: ESLint, at the version package.json and yarn.lock
|
||||||
# pin, checks static/js/ against eslint.config.mjs. js-deps installs it, and
|
# pin, checks static/js/ against eslint.config.mjs. js-deps installs it, and
|
||||||
# prettier for the Markdown stages below. The lint phase below runs js-lint.
|
# prettier for the Markdown stages below, and stays cached until those two
|
||||||
#
|
# files change. script/lint forces only js-lint to re-run, and the build stage
|
||||||
# The image's own corepack runs the yarn that package.json's packageManager
|
# below runs it too. COPY . . brings in the CI cache barrier described in the
|
||||||
# field names, yarn 4.18.1 (released 2026-09-24), and checks it against the
|
# lint stage above.
|
||||||
# hash there. The image also ships yarn 1, which `corepack enable yarn`
|
# node:24.21.0-alpine (LTS, with yarn 1.22.22), 2026-09-18
|
||||||
# replaces.
|
|
||||||
# node:24.21.0-alpine (LTS), 2026-09-18
|
|
||||||
FROM node:24.21.0-alpine@sha256:ebfe2f90462722a7a4de65e91990e97fe0d401c70e0e762c5b53302f905ec1c1 AS js-deps
|
FROM node:24.21.0-alpine@sha256:ebfe2f90462722a7a4de65e91990e97fe0d401c70e0e762c5b53302f905ec1c1 AS js-deps
|
||||||
WORKDIR /src
|
WORKDIR /src
|
||||||
COPY package.json yarn.lock .yarnrc.yml ./
|
COPY package.json yarn.lock ./
|
||||||
RUN corepack enable yarn && yarn install --immutable --mode=skip-build
|
RUN yarn install --frozen-lockfile --ignore-scripts
|
||||||
|
|
||||||
FROM js-deps AS js-lint
|
FROM js-deps AS js-lint
|
||||||
COPY . .
|
COPY . .
|
||||||
@@ -70,116 +101,23 @@ FROM js-deps AS markdown-check
|
|||||||
COPY . .
|
COPY . .
|
||||||
RUN --network=none node_modules/.bin/prettier --check '**/*.md'
|
RUN --network=none node_modules/.bin/prettier --check '**/*.md'
|
||||||
|
|
||||||
# Lint phase: the Go formatting check and golangci-lint over the Go code,
|
|
||||||
# and ESLint over static/js/ through the copy from js-lint at the end.
|
|
||||||
# `make lint` (script/lint) builds this stage alone; the build stage below
|
|
||||||
# depends on it.
|
|
||||||
#
|
|
||||||
# golangci/golangci-lint:v2.14.0 (Debian-based), 2026-09-24
|
|
||||||
# Using Debian-based image because mattn/go-sqlite3 (CGO) does not
|
|
||||||
# compile on Alpine musl (off64_t is a glibc type).
|
|
||||||
FROM golangci/golangci-lint:v2.14.0@sha256:ad862ba6b3798cbe0fd9fd7408d498fd74fbd2623a92406b2fd3898faf0bf98f AS lint
|
|
||||||
|
|
||||||
WORKDIR /src
|
|
||||||
|
|
||||||
# Copy go mod files first for better layer caching
|
|
||||||
COPY go.mod go.sum ./
|
|
||||||
RUN go mod download
|
|
||||||
|
|
||||||
COPY . .
|
|
||||||
|
|
||||||
# gofmt and golangci-lint are invoked directly rather than through `make
|
|
||||||
# fmt-check` and `make lint`, which are themselves docker builds and would
|
|
||||||
# need a docker daemon inside this one. The Markdown half of `make
|
|
||||||
# fmt-check` is the markdown-check stage above.
|
|
||||||
RUN if [ -n "$(gofmt -s -l .)" ]; then echo "gofmt needed on:"; gofmt -s -l .; exit 1; fi
|
|
||||||
|
|
||||||
# static/static.go embeds the Alpine.js file this extracts from 3p/; without
|
|
||||||
# it the static package does not compile and cannot be linted.
|
|
||||||
RUN script/assets
|
|
||||||
|
|
||||||
# The golangci-lint steps run with --network=none. `golangci-lint config
|
|
||||||
# verify` is documented as fetching its JSON schema over HTTPS; this pinned
|
|
||||||
# image resolves the schema without any network, and --network=none enforces
|
|
||||||
# that. It also proves no linter reaches out at analysis time.
|
|
||||||
#
|
|
||||||
# `run` silently ignores config keys it does not recognize, so a typo would
|
|
||||||
# disable a setting without a word. `config verify` is what catches that.
|
|
||||||
RUN --network=none golangci-lint config verify --config .golangci.yml
|
|
||||||
# --build-tags browser also lints the browser test, which is built only with
|
|
||||||
# that tag (make test-browser).
|
|
||||||
RUN --network=none golangci-lint run --config .golangci.yml --build-tags browser ./...
|
|
||||||
|
|
||||||
# Nothing is wanted from js-lint; the copy is what makes this phase run it.
|
|
||||||
COPY --from=js-lint /src/yarn.lock /dev/null
|
|
||||||
|
|
||||||
# Test phase. -race needs cgo and so a C compiler, which the Debian Go image
|
|
||||||
# ships and the alpine one does not. `make test` (script/test) builds this
|
|
||||||
# stage alone; the build stage below depends on it.
|
|
||||||
#
|
|
||||||
# golang:1.26.1-bookworm (Debian-based), 2026-03-17
|
|
||||||
FROM golang:1.26.1-bookworm@sha256:4465644228bc2857a954b092167e12aa59c006a3492282a6c820bf4755fd64a4 AS test
|
|
||||||
|
|
||||||
WORKDIR /src
|
|
||||||
|
|
||||||
COPY go.mod go.sum ./
|
|
||||||
RUN go mod download
|
|
||||||
|
|
||||||
COPY . .
|
|
||||||
|
|
||||||
# static/static.go embeds the Alpine.js file this extracts from 3p/.
|
|
||||||
RUN script/assets
|
|
||||||
|
|
||||||
# -timeout applies to each package on its own, so 90s has only to clear the
|
|
||||||
# slowest one. -p 4 -parallel 8 keep the run under 2 GB of memory: at most
|
|
||||||
# four test binaries build or run at once, each with at most eight parallel
|
|
||||||
# tests. Under -race every test binary and every link costs a few hundred MB,
|
|
||||||
# so the defaults (one per core) add up to several GB on a many-core host.
|
|
||||||
#
|
|
||||||
# The first run has no -v: go test then prints one result line per package,
|
|
||||||
# with its coverage, and for a package that fails, everything its tests
|
|
||||||
# wrote. Verbose output from the whole suite passes the 2 MiB at which the
|
|
||||||
# Docker build cuts off a step's log, so on a failure only the tests that
|
|
||||||
# failed run again, with -v. go test reports a failed test as a line starting
|
|
||||||
# "--- FAIL: TestName" (a failed subtest's line is indented, and reruns with
|
|
||||||
# its parent) and a failed package as "FAIL<tab>package/path<tab>...". A
|
|
||||||
# failure that names no test, such as a build error or a timeout, is already
|
|
||||||
# shown in full, so there is nothing to rerun. The step fails after the rerun
|
|
||||||
# whatever its result: the first run already showed the suite is broken.
|
|
||||||
#
|
|
||||||
# TMPDIR, where the tests keep their SQLite databases, is a tmpfs: SQLite
|
|
||||||
# waits for the disk at every commit, and on a busy host that waiting was
|
|
||||||
# about 40% of the slowest package's run time. GOTMPDIR keeps go's own
|
|
||||||
# build files, the test binaries among them, on disk.
|
|
||||||
#
|
|
||||||
# bash with pipefail, so that the first run's status is go test's, not tee's.
|
|
||||||
SHELL ["/bin/bash", "-o", "pipefail", "-c"]
|
|
||||||
RUN --mount=type=tmpfs,target=/tmp/tests,size=512m \
|
|
||||||
export TMPDIR=/tmp/tests GOTMPDIR=/tmp; \
|
|
||||||
go test -race -cover -p 4 -parallel 8 -timeout 90s ./... 2>&1 | tee /tmp/go-test.log && exit 0; \
|
|
||||||
tests="$(awk '/^--- FAIL: / { print $3 }' /tmp/go-test.log | paste -s -d '|' -)"; \
|
|
||||||
packages="$(awk '/^FAIL\t/ { print $2 }' /tmp/go-test.log)"; \
|
|
||||||
if [ -n "$tests" ]; then \
|
|
||||||
echo "--- Rerunning the failed tests with -v for details ---"; \
|
|
||||||
go test -race -v -p 4 -parallel 8 -timeout 90s -run "^($tests)\$" $packages; \
|
|
||||||
fi; \
|
|
||||||
exit 1
|
|
||||||
|
|
||||||
# Build stage
|
# Build stage
|
||||||
# golang:1.26.1-bookworm (Debian-based), 2026-03-17
|
# golang:1.26.1-bookworm (Debian-based), 2026-03-17
|
||||||
# Using Debian-based image because gorm.io/driver/sqlite pulls in
|
# Using Debian-based image because gorm.io/driver/sqlite pulls in
|
||||||
# mattn/go-sqlite3 (CGO), which does not compile on Alpine musl. The image
|
# mattn/go-sqlite3 (CGO), which does not compile on Alpine musl.
|
||||||
# ships git and make, which the version step below uses.
|
|
||||||
FROM golang:1.26.1-bookworm@sha256:4465644228bc2857a954b092167e12aa59c006a3492282a6c820bf4755fd64a4 AS builder
|
FROM golang:1.26.1-bookworm@sha256:4465644228bc2857a954b092167e12aa59c006a3492282a6c820bf4755fd64a4 AS builder
|
||||||
|
|
||||||
# Nothing is wanted from the lint and test phases or from the stylesheet and
|
# Depend on the lint, stylesheet check, JavaScript lint and Markdown check
|
||||||
# Markdown checks; the copies are what make BuildKit build them first, so
|
# stages passing
|
||||||
# this stage cannot run unless they all passed.
|
|
||||||
COPY --from=lint /src/go.sum /dev/null
|
COPY --from=lint /src/go.sum /dev/null
|
||||||
COPY --from=test /src/go.sum /dev/null
|
|
||||||
COPY --from=css-check /out/tailwind.css /dev/null
|
COPY --from=css-check /out/tailwind.css /dev/null
|
||||||
|
COPY --from=js-lint /src/yarn.lock /dev/null
|
||||||
COPY --from=markdown-check /src/yarn.lock /dev/null
|
COPY --from=markdown-check /src/yarn.lock /dev/null
|
||||||
|
|
||||||
|
# jq is a runtime dependency of script/ci-mark-superseded, which the test
|
||||||
|
# suite executes. git is what script/version derives the version with.
|
||||||
|
RUN apt-get update && apt-get install -y --no-install-recommends make curl ca-certificates jq git && rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
# A build context sent as a tar archive keeps its files' owners, and git
|
# A build context sent as a tar archive keeps its files' owners, and git
|
||||||
# refuses to read a checkout owned by another user. Trust this one
|
# refuses to read a checkout owned by another user. Trust this one
|
||||||
# whoever owns it.
|
# whoever owns it.
|
||||||
@@ -191,27 +129,31 @@ WORKDIR /build
|
|||||||
COPY go.mod go.sum ./
|
COPY go.mod go.sum ./
|
||||||
RUN go mod download
|
RUN go mod download
|
||||||
|
|
||||||
|
# Copy source code, including the .ci-fingerprint cache barrier described in
|
||||||
|
# the lint stage above.
|
||||||
COPY . .
|
COPY . .
|
||||||
|
|
||||||
|
# Run tests and build. Both first run script/assets, which extracts Alpine.js
|
||||||
|
# from its tarball in 3p/.
|
||||||
|
RUN make test
|
||||||
|
|
||||||
# Version stamped into the binary: the VERSION build arg when one is
|
# Version stamped into the binary: the VERSION build arg when one is
|
||||||
# given, otherwise what script/version derives from the .git the build
|
# given, otherwise what script/version derives from the .git the build
|
||||||
# context carries, so any `docker build .` of a clone stamps its commit.
|
# context carries, so any `docker build .` of a clone stamps its commit.
|
||||||
# With neither, as from a source tarball, it is "unknown".
|
# With neither, as from a source tarball, it is "unknown".
|
||||||
|
#
|
||||||
|
# Declared here, below the test step, so a changed version does not
|
||||||
|
# invalidate its cached layer.
|
||||||
ARG VERSION
|
ARG VERSION
|
||||||
|
|
||||||
# A context that carries .git must not stamp an empty version, "dev" or
|
# A context that carries .git must not stamp "unknown": that means git is
|
||||||
# "unknown": that means git is missing here or could not read the
|
# missing here or could not read the checkout, and the image could not be
|
||||||
# checkout, and the image could not be traced back to its commit.
|
# traced back to its commit.
|
||||||
RUN version="$(make version VERSION="$VERSION")"; \
|
RUN if [ -d .git ] && [ "$(make version VERSION="$VERSION")" = unknown ]; then \
|
||||||
if [ -e .git ]; then \
|
echo "version is unknown although the build context carries .git" >&2; \
|
||||||
case "$version" in ""|dev|unknown) \
|
exit 1; \
|
||||||
echo "version is '$version' although .git is present" >&2; \
|
|
||||||
exit 1 ;; \
|
|
||||||
esac; \
|
|
||||||
fi
|
fi
|
||||||
|
|
||||||
# Builds through the Makefile's build target, which runs script/assets
|
|
||||||
# (Alpine.js, extracted from its tarball in 3p/) first.
|
|
||||||
RUN make build VERSION="$VERSION"
|
RUN make build VERSION="$VERSION"
|
||||||
|
|
||||||
# Rebuild with static linking for Alpine runtime.
|
# Rebuild with static linking for Alpine runtime.
|
||||||
|
|||||||
+1
-1
@@ -16,7 +16,7 @@ COPY . .
|
|||||||
|
|
||||||
# The test binary embeds the templates and static files, so the browser
|
# The test binary embeds the templates and static files, so the browser
|
||||||
# stage needs nothing else. -p 4 keeps the compile's memory down, as in
|
# stage needs nothing else. -p 4 keeps the compile's memory down, as in
|
||||||
# the test phase of Dockerfile.
|
# script/test.
|
||||||
RUN make assets && go test -c -p 4 -tags browser -o /browser.test ./internal/server
|
RUN make assets && go test -c -p 4 -tags browser -o /browser.test ./internal/server
|
||||||
|
|
||||||
# chromedp/headless-shell:151.0.7922.109 (Debian trixie), 2026-08-11. The
|
# chromedp/headless-shell:151.0.7922.109 (Debian trixie), 2026-08-11. The
|
||||||
|
|||||||
@@ -0,0 +1,43 @@
|
|||||||
|
# Lint-only image, built by script/lint. golangci-lint is never installed on
|
||||||
|
# the host: the repo is COPYed into the pinned image and linted as a build
|
||||||
|
# step, so a successful build IS a clean lint. This works even when the docker
|
||||||
|
# daemon is remote and bind mounts are impossible.
|
||||||
|
#
|
||||||
|
# script/lint passes --no-cache-filter=lint. Without it an unchanged tree
|
||||||
|
# replays the lint stage from cache and the build succeeds in under a second
|
||||||
|
# having run no linter at all. Do not drop that flag.
|
||||||
|
#
|
||||||
|
# The lint steps run with --network=none. `golangci-lint config verify` is
|
||||||
|
# documented as fetching its JSON schema over HTTPS, which would make linting
|
||||||
|
# depend on an unpinned remote artifact; this pinned image resolves the schema
|
||||||
|
# without any network, and --network=none enforces that rather than trusting
|
||||||
|
# it. It also proves no linter reaches out at analysis time. If a future image
|
||||||
|
# bump makes either step need the network, this build fails loudly instead of
|
||||||
|
# quietly acquiring an unpinned dependency.
|
||||||
|
|
||||||
|
# golangci/golangci-lint:v2.12.2 (Debian-based), 2026-08-07
|
||||||
|
# Using Debian-based image because mattn/go-sqlite3 (CGO) does not
|
||||||
|
# compile on Alpine musl (off64_t is a glibc type).
|
||||||
|
FROM golangci/golangci-lint:v2.12.2@sha256:5cceeef04e53efe1470638d4b4b4f5ceefd574955ab3941b2d9a68a8c9ad5240 AS deps
|
||||||
|
|
||||||
|
WORKDIR /src
|
||||||
|
|
||||||
|
# Copy go mod files first for better layer caching. This stage is cacheable;
|
||||||
|
# only the lint stage below is forced to re-execute.
|
||||||
|
COPY go.mod go.sum ./
|
||||||
|
RUN go mod download
|
||||||
|
|
||||||
|
FROM deps AS lint
|
||||||
|
|
||||||
|
COPY . .
|
||||||
|
|
||||||
|
# static/static.go embeds the Alpine.js file this extracts from 3p/; without
|
||||||
|
# it the static package does not compile and cannot be linted.
|
||||||
|
RUN script/assets
|
||||||
|
|
||||||
|
# `run` silently ignores config keys it does not recognize, so a typo would
|
||||||
|
# disable a setting without a word. `config verify` is what catches that.
|
||||||
|
RUN --network=none golangci-lint config verify --config .golangci.yml
|
||||||
|
# --build-tags browser also lints the browser test, which is built only with
|
||||||
|
# that tag (make test-browser).
|
||||||
|
RUN --network=none golangci-lint run --config .golangci.yml --build-tags browser ./...
|
||||||
@@ -19,10 +19,6 @@ override VERSION := $(or $(strip $(VERSION)),$(shell script/version))
|
|||||||
# Extra linker flags for the build target. The static relink in the
|
# Extra linker flags for the build target. The static relink in the
|
||||||
# Dockerfile adds -extldflags here rather than passing its own -ldflags,
|
# Dockerfile adds -extldflags here rather than passing its own -ldflags,
|
||||||
# so composing flags cannot drop the version stamp.
|
# so composing flags cannot drop the version stamp.
|
||||||
#
|
|
||||||
# The build target itself always passes -trimpath and -s -w, as the Go
|
|
||||||
# Dockerfile in REPO_POLICIES.md does: no build paths, symbol table or
|
|
||||||
# debug information in the binary.
|
|
||||||
GO_LDFLAGS ?=
|
GO_LDFLAGS ?=
|
||||||
|
|
||||||
bootstrap:
|
bootstrap:
|
||||||
@@ -53,7 +49,7 @@ check:
|
|||||||
@script/check
|
@script/check
|
||||||
|
|
||||||
build: assets
|
build: assets
|
||||||
go build -trimpath -ldflags '$(strip -s -w -X main.version=$(VERSION) $(GO_LDFLAGS))' -o bin/webhooker ./cmd/webhooker
|
go build -ldflags '$(strip -X main.version=$(VERSION) $(GO_LDFLAGS))' -o bin/webhooker ./cmd/webhooker
|
||||||
|
|
||||||
run: build
|
run: build
|
||||||
./bin/webhooker
|
./bin/webhooker
|
||||||
|
|||||||
@@ -17,14 +17,14 @@ before deploying one.
|
|||||||
### Prerequisites
|
### Prerequisites
|
||||||
|
|
||||||
- Go 1.26.1+ (the version in `go.mod`)
|
- Go 1.26.1+ (the version in `go.mod`)
|
||||||
- Docker (for `make test`, `make lint`, `make fmt` and `make css`, and so for
|
- Docker (for `make lint`, `make fmt` and `make css`, and so for `make check`,
|
||||||
`make check`, for the browser test in `make test-browser`, for the CI gate,
|
for the browser test in `make test-browser`, for the CI gate, and for
|
||||||
and for containerized deployment)
|
containerized deployment)
|
||||||
|
|
||||||
golangci-lint is not a prerequisite and must not be installed on the host:
|
golangci-lint is not a prerequisite and must not be installed on the host:
|
||||||
`script/bootstrap` does not install it, and `make lint` runs the digest-pinned
|
`script/bootstrap` does not install it, and `make lint` runs the digest-pinned
|
||||||
linter image in the Dockerfile's `lint` phase. The same holds for tailwindcss
|
linter image via `Dockerfile.lint`. The same holds for tailwindcss (see
|
||||||
(see [Stylesheet](#stylesheet)). ESLint, prettier, node and yarn are not
|
[Stylesheet](#stylesheet)). ESLint, prettier, node and yarn are not
|
||||||
prerequisites either, and `make lint` and `make fmt` never use a host copy of
|
prerequisites either, and `make lint` and `make fmt` never use a host copy of
|
||||||
them (see [Linting](#linting)).
|
them (see [Linting](#linting)).
|
||||||
|
|
||||||
@@ -43,9 +43,8 @@ make check
|
|||||||
|
|
||||||
# Run the server from the clone. DATA_DIR defaults to
|
# Run the server from the clone. DATA_DIR defaults to
|
||||||
# /var/lib/webhooker in every environment, so set it (in .env or the
|
# /var/lib/webhooker in every environment, so set it (in .env or the
|
||||||
# shell) to a writable directory outside the clone: the databases hold
|
# shell) to a writable directory.
|
||||||
# the session key.
|
DATA_DIR=./data make dev
|
||||||
DATA_DIR=../webhooker-data make dev
|
|
||||||
|
|
||||||
# Build Docker image
|
# Build Docker image
|
||||||
make docker
|
make docker
|
||||||
@@ -56,11 +55,11 @@ make docker
|
|||||||
```bash
|
```bash
|
||||||
make bootstrap # Install all dependencies (idempotent)
|
make bootstrap # Install all dependencies (idempotent)
|
||||||
make setup # Bootstrap + install git pre-commit hook
|
make setup # Bootstrap + install git pre-commit hook
|
||||||
make assets # Extract Alpine.js from 3p/ (build and dev run it)
|
make assets # Extract Alpine.js from 3p/ (test, check, build, dev run it)
|
||||||
make fmt # Format Go (gofmt + goimports) and Markdown (prettier, in Docker)
|
make fmt # Format Go (gofmt + goimports) and Markdown (prettier, in Docker)
|
||||||
make fmt-check # Fail if gofmt or prettier would change anything (writes nothing)
|
make fmt-check # Fail if gofmt or prettier would change anything (writes nothing)
|
||||||
make lint # Run golangci-lint and ESLint in Docker
|
make lint # Run golangci-lint and ESLint in Docker
|
||||||
make test # Run tests with race detection, in Docker
|
make test # Run tests with race detection
|
||||||
make test-browser # Run the browser test in Docker (Dockerfile.browser)
|
make test-browser # Run the browser test in Docker (Dockerfile.browser)
|
||||||
make check # test + lint + fmt-check + css-check (CI gate)
|
make check # test + lint + fmt-check + css-check (CI gate)
|
||||||
make build # Build binary to bin/webhooker (version-stamped)
|
make build # Build binary to bin/webhooker (version-stamped)
|
||||||
@@ -1096,9 +1095,7 @@ field), in the UI footer, and in the startup log line (`msg=starting`,
|
|||||||
The value is stamped in at build time by the linker; it is not read from a file
|
The value is stamped in at build time by the linker; it is not read from a file
|
||||||
at runtime, so it identifies the build itself.
|
at runtime, so it identifies the build itself.
|
||||||
|
|
||||||
`script/version` produces the value for `make build`, and the image build runs
|
`script/version` produces the value and both build paths use it:
|
||||||
`make build` too; `script/docker` and `script/cibuild` run the same
|
|
||||||
`git describe --tags --always --dirty` on the host. All of them report:
|
|
||||||
|
|
||||||
| Build | What it reports |
|
| Build | What it reports |
|
||||||
| ----------------------- | --------------------------------------------------- |
|
| ----------------------- | --------------------------------------------------- |
|
||||||
@@ -1113,19 +1110,18 @@ carries, so any `docker build .` of a clone, with no build arguments, stamps the
|
|||||||
commit it was built from; a shallow clone of one branch has no tags and stamps
|
commit it was built from; a shallow clone of one branch has no tags and stamps
|
||||||
the short SHA. `.dockerignore` must therefore leave out neither `.git` nor any
|
the short SHA. `.dockerignore` must therefore leave out neither `.git` nor any
|
||||||
tracked file, which git in the build would see as deleted, marking the version
|
tracked file, which git in the build would see as deleted, marking the version
|
||||||
`-dirty`. It does leave out every git `config` (`**/.git/config`,
|
`-dirty`. It does leave `.git/config`, which can hold a remote URL carrying a
|
||||||
`**/.git/modules/**/config`), which can hold a remote URL carrying a credential
|
credential and which `git describe` does not need, out of a directory context. A
|
||||||
and which `git describe` does not need, from a directory context. A context sent
|
context sent as a tar is not filtered by `.dockerignore`, so it carries
|
||||||
as a tar is not filtered by `.dockerignore`, so it carries `.git/config` unless
|
`.git/config` unless its sender leaves it out; for upaas, that is
|
||||||
its sender leaves it out; for upaas, that is
|
|
||||||
https://git.eeqj.de/sneak/upaas/issues/274. git in the build reads the checkout
|
https://git.eeqj.de/sneak/upaas/issues/274. git in the build reads the checkout
|
||||||
whoever owns its files, since a context sent as a tar archive keeps the sender's
|
whoever owns its files, since a context sent as a tar archive keeps the sender's
|
||||||
owners and git otherwise refuses a checkout owned by another user. A `VERSION`
|
owners and git otherwise refuses a checkout owned by another user. A `VERSION`
|
||||||
build arg (`--build-arg VERSION=...`) takes precedence; `script/docker` (and so
|
build arg (`--build-arg VERSION=...`) takes precedence; `script/docker` (and so
|
||||||
`make docker`) and `script/cibuild` pass the one they resolve on the host, or
|
`make docker`) passes the one `script/version` resolves on the host. The image
|
||||||
`unknown` where git gives none. The image build fails if its context carries
|
build fails if its context carries `.git` and the version still comes out
|
||||||
`.git` and the version still comes out empty, `dev` or `unknown`, which means
|
`unknown`, which means git is missing from the build or could not read the
|
||||||
git is missing from the build or could not read the checkout.
|
checkout.
|
||||||
|
|
||||||
`unknown` is what a source tarball, or a `docker build` with no `.git` in its
|
`unknown` is what a source tarball, or a `docker build` with no `.git` in its
|
||||||
context and no `VERSION` build arg, reports. A build that reports `unknown` is a
|
context and no `VERSION` build arg, reports. A build that reports `unknown` is a
|
||||||
@@ -1139,9 +1135,7 @@ to a commit.
|
|||||||
|
|
||||||
Nothing that varies between two builds of the same commit is stamped — no
|
Nothing that varies between two builds of the same commit is stamped — no
|
||||||
timestamp, no hostname, no builder identity — so two builds of one commit still
|
timestamp, no hostname, no builder identity — so two builds of one commit still
|
||||||
produce a byte-identical binary. `make build` passes `-trimpath`, so the
|
produce a byte-identical binary.
|
||||||
directory it builds in is not recorded either, and `-s -w`, which leave out the
|
|
||||||
symbol table and debug information.
|
|
||||||
|
|
||||||
### Backups contain secrets
|
### Backups contain secrets
|
||||||
|
|
||||||
@@ -1219,14 +1213,11 @@ commands with no script behind them, though `build`, `run` and `dev` first run
|
|||||||
`script/assets`, and `build` and `version` both take their value from
|
`script/assets`, and `build` and `version` both take their value from
|
||||||
`script/version`.
|
`script/version`.
|
||||||
|
|
||||||
`make build` and `make dev` each run `script/assets` first, which writes the
|
`script/test`, `make build` and `make dev` each run `script/assets` first, which
|
||||||
uncommitted `static/js/alpine.min.js` (see
|
writes the ignored `static/js/alpine.min.js` (see
|
||||||
[Third-party browser assets](#third-party-browser-assets)), so they work on a
|
[Third-party browser assets](#third-party-browser-assets)), so `make test`,
|
||||||
fresh clone without a separate step. The Docker stages that compile the code run
|
`make check` and the pre-commit hook work on a fresh clone without a separate
|
||||||
it themselves.
|
step.
|
||||||
|
|
||||||
Every `docker build` in `script/` passes `--no-cache`: a check served from the
|
|
||||||
build cache is a check that did not run.
|
|
||||||
|
|
||||||
We provide:
|
We provide:
|
||||||
|
|
||||||
@@ -1236,12 +1227,10 @@ We provide:
|
|||||||
- `script/projectname` — output the project name ("webhooker")
|
- `script/projectname` — output the project name ("webhooker")
|
||||||
- `script/assets` — extract Alpine.js from its tarball in `3p/` (see
|
- `script/assets` — extract Alpine.js from its tarball in `3p/` (see
|
||||||
[Third-party browser assets](#third-party-browser-assets))
|
[Third-party browser assets](#third-party-browser-assets))
|
||||||
- `script/test` — run the test suite: builds the Dockerfile's `test` phase,
|
- `script/test` — run the test suite
|
||||||
tagged `webhooker-test`
|
|
||||||
- `script/test-browser` — run the browser test in Docker (see
|
- `script/test-browser` — run the browser test in Docker (see
|
||||||
[Third-party browser assets](#third-party-browser-assets))
|
[Third-party browser assets](#third-party-browser-assets))
|
||||||
- `script/lint` — run the `gofmt` check, golangci-lint and ESLint: builds the
|
- `script/lint` — run golangci-lint and ESLint in Docker (see Linting below)
|
||||||
Dockerfile's `lint` phase, tagged `webhooker-lint` (see Linting below)
|
|
||||||
- `script/fmt` — format the Go code and, in Docker, the Markdown (writes)
|
- `script/fmt` — format the Go code and, in Docker, the Markdown (writes)
|
||||||
- `script/fmt-check` — check formatting (read-only)
|
- `script/fmt-check` — check formatting (read-only)
|
||||||
- `script/css` — regenerate `static/css/tailwind.css` in Docker (writes; see
|
- `script/css` — regenerate `static/css/tailwind.css` in Docker (writes; see
|
||||||
@@ -1252,11 +1241,11 @@ We provide:
|
|||||||
- `script/version` — output the version to stamp into the binary (see
|
- `script/version` — output the version to stamp into the binary (see
|
||||||
[Version stamping](#version-stamping))
|
[Version stamping](#version-stamping))
|
||||||
- `script/docker` — build the Docker image tagged via `script/projectname`,
|
- `script/docker` — build the Docker image tagged via `script/projectname`,
|
||||||
passing the version `git describe` gives on the host in as the `VERSION` build
|
passing `script/version`'s output in as the `VERSION` build arg
|
||||||
arg
|
- `script/cibuild` — CI entrypoint: `docker build .` (the Dockerfile runs the
|
||||||
- `script/cibuild` — CI entrypoint: `script/bootstrap`, then `script/check`,
|
checks, so a green build implies a green repo)
|
||||||
then the same image build as `script/docker`, whose gate phases run again (see
|
- `script/ci-mark-superseded` — CI helper: mark the commits whose run a newer
|
||||||
[CI gate honesty](#ci-gate-honesty))
|
push cancelled (see [CI gate honesty](#ci-gate-honesty))
|
||||||
- `script/precommit` — pre-commit checks (`go mod tidy` guard, then
|
- `script/precommit` — pre-commit checks (`go mod tidy` guard, then
|
||||||
`script/check`)
|
`script/check`)
|
||||||
- `script/install-precommit` — install the git pre-commit hook that runs
|
- `script/install-precommit` — install the git pre-commit hook that runs
|
||||||
@@ -1291,9 +1280,7 @@ the event log only the newest starts expanded, and an event there expands and
|
|||||||
collapses when its row's caret or its ID is clicked, and from the keyboard, but
|
collapses when its row's caret or its ID is clicked, and from the keyboard, but
|
||||||
not when its ID is selected with the mouse, and a delivery's attempts inside it
|
not when its ID is selected with the mouse, and a delivery's attempts inside it
|
||||||
expand and collapse; and at phone width the menu button opens and closes the
|
expand and collapse; and at phone width the menu button opens and closes the
|
||||||
mobile menu, and neither the webhook page nor the event log, with a delivery's
|
mobile menu. It also fails if the browser reports a console warning or error, an
|
||||||
attempts open, scrolls sideways or cuts anything off at the page's or a card's
|
|
||||||
edge. It also fails if the browser reports a console warning or error, an
|
|
||||||
uncaught exception, or anything the policy refused. `make check` and the image
|
uncaught exception, or anything the policy refused. `make check` and the image
|
||||||
build lint it but do not run it, and `make test` leaves it out (its file is
|
build lint it but do not run it, and `make test` leaves it out (its file is
|
||||||
built only with the `browser` build tag). Run it with `make test-browser` after
|
built only with the `browser` build tag). Run it with `make test-browser` after
|
||||||
@@ -1309,12 +1296,12 @@ apply. The directory is `3p/` rather than `vendor/` because Go treats a root
|
|||||||
|
|
||||||
`script/assets` (`make assets`) extracts the browser build,
|
`script/assets` (`make assets`) extracts the browser build,
|
||||||
`package/dist/cdn.min.js`, from the tarball to `static/js/alpine.min.js`, where
|
`package/dist/cdn.min.js`, from the tarball to `static/js/alpine.min.js`, where
|
||||||
`go:embed` picks it up. `make build` and `make dev` run it first, and so do the
|
`go:embed` picks it up. `script/test`, `make build` and `make dev` run it first,
|
||||||
Dockerfile's lint, test and build stages, so nothing downloads Alpine.js. The
|
and the Dockerfile builds through `make test` and `make build`, so nothing
|
||||||
extracted file is not committed, and `.dockerignore` keeps any host copy out of
|
downloads Alpine.js. The extracted file is not committed, and `.dockerignore`
|
||||||
the build context. `static/static.go` names every file it embeds, so a build
|
keeps any host copy out of the build context. `static/static.go` names every
|
||||||
that skips the extraction, such as a bare `go build`, fails with an error naming
|
file it embeds, so a build that skips the extraction, such as a bare `go build`,
|
||||||
`js/alpine.min.js`.
|
fails with an error naming `js/alpine.min.js`.
|
||||||
|
|
||||||
To move to a new version: download
|
To move to a new version: download
|
||||||
`https://registry.npmjs.org/@alpinejs/csp/-/csp-<version>.tgz`, check it against
|
`https://registry.npmjs.org/@alpinejs/csp/-/csp-<version>.tgz`, check it against
|
||||||
@@ -1650,18 +1637,10 @@ URL, custom headers, timeout settings).
|
|||||||
**`http` target configuration:**
|
**`http` target configuration:**
|
||||||
|
|
||||||
| Key | Type | Description |
|
| Key | Type | Description |
|
||||||
| -------------- | ------------- | ----------------------------------------------------------------------------------------- |
|
| --------- | ------------- | -------------------------------------------------------------------------------------- |
|
||||||
| `url` | string | Destination the event is POSTed to |
|
| `url` | string | Destination the event is POSTed to |
|
||||||
| `headers` | object | Extra request headers, applied last so they win over the event's own forwarded headers |
|
| `headers` | object | Extra request headers, applied last so they win over the event's own forwarded headers |
|
||||||
| `timeout` | integer (sec) | Per-target request timeout; unset (or 0) uses the shared 30-second client timeout |
|
| `timeout` | integer (sec) | Per-target request timeout; unset (or 0) uses the shared 30-second client timeout |
|
||||||
| `forwardQuery` | boolean | Pass the query string each event arrived with on to the target; unset (or false) does not |
|
|
||||||
|
|
||||||
`forwardQuery` is off by default, and the target URL is then sent exactly as
|
|
||||||
configured. On, each delivery appends the event's query string to the target
|
|
||||||
URL, joined with `&` when the URL already has a query string of its own; a
|
|
||||||
replayed delivery and a resubmitted event's deliveries do the same. Both target
|
|
||||||
forms offer it as "Pass the query string on to this target", and the target list
|
|
||||||
shows it when it is on.
|
|
||||||
|
|
||||||
`timeout` is capped at **300 seconds**, and the form rejects anything above it
|
`timeout` is capped at **300 seconds**, and the form rejects anything above it
|
||||||
rather than substituting the cap. A delivery attempt holds one of the bounded
|
rather than substituting the cap. A delivery attempt holds one of the bounded
|
||||||
@@ -1723,7 +1702,6 @@ auditing, for replay, and for resubmission.
|
|||||||
| `webhook_id` | UUID | Foreign key → Webhook |
|
| `webhook_id` | UUID | Foreign key → Webhook |
|
||||||
| `entrypoint_id` | UUID | Foreign key → Entrypoint |
|
| `entrypoint_id` | UUID | Foreign key → Entrypoint |
|
||||||
| `method` | string | HTTP method of the captured request. Always `POST`: the receiver answers every other method with 405 before an Event is created |
|
| `method` | string | HTTP method of the captured request. Always `POST`: the receiver answers every other method with 405 before an Event is created |
|
||||||
| `raw_query` | text | The query string of the captured request, as sent, without the leading `?`; empty when there was none. A resubmitted copy carries its original's |
|
|
||||||
| `headers` | JSON | Complete request headers |
|
| `headers` | JSON | Complete request headers |
|
||||||
| `body` | text | Raw request body |
|
| `body` | text | Raw request body |
|
||||||
| `content_type` | string | Content-Type header value |
|
| `content_type` | string | Content-Type header value |
|
||||||
@@ -1732,15 +1710,9 @@ auditing, for replay, and for resubmission.
|
|||||||
|
|
||||||
**Relations:** Belongs to Webhook. Belongs to Entrypoint. Has many Deliveries.
|
**Relations:** Belongs to Webhook. Belongs to Entrypoint. Has many Deliveries.
|
||||||
|
|
||||||
When a request arrives at an entrypoint, the full request (method, query string,
|
When a request arrives at an entrypoint, the full request (method, headers,
|
||||||
headers, body) is captured as an Event. The event is then queued for delivery to
|
body) is captured as an Event. The event is then queued for delivery to every
|
||||||
every active target configured on the parent webhook.
|
active target configured on the parent webhook.
|
||||||
|
|
||||||
The event log and the event's own page show the query string with the rest of
|
|
||||||
the request. The event log leaves out one larger than 32 KiB, as it does request
|
|
||||||
headers, and links to the event's page, which shows it whole. The `database` and
|
|
||||||
`log` targets carry it with the rest of the event. An `http` target receives it
|
|
||||||
only when its `forwardQuery` setting is on.
|
|
||||||
|
|
||||||
#### Delivery
|
#### Delivery
|
||||||
|
|
||||||
@@ -1786,14 +1758,14 @@ the webhook's currently active targets.
|
|||||||
|
|
||||||
**Resubmit.** Replay recovers one delivery; **resubmit** re-injects one EVENT.
|
**Resubmit.** Replay recovers one delivery; **resubmit** re-injects one EVENT.
|
||||||
The event log offers a per-event **Resubmit** action that stores a NEW event
|
The event log offers a per-event **Resubmit** action that stores a NEW event
|
||||||
copying the stored one's `method`, `raw_query`, `headers`, `body` and
|
copying the stored one's `method`, `headers`, `body` and `content_type`
|
||||||
`content_type` verbatim, then fans it out to the webhook's currently **active**
|
verbatim, then fans it out to the webhook's currently **active** targets —
|
||||||
targets — resolved fresh by the same query the receiver uses, so a target
|
resolved fresh by the same query the receiver uses, so a target created long
|
||||||
created long after the original event arrived receives it. That is the
|
after the original event arrived receives it. That is the difference that
|
||||||
difference that matters: a target added to test a backend under development has
|
matters: a target added to test a backend under development has no prior
|
||||||
no prior delivery, so there is nothing to replay to it, while a resubmit reaches
|
delivery, so there is nothing to replay to it, while a resubmit reaches it like
|
||||||
it like any other active target. Inactive targets are skipped, exactly as the
|
any other active target. Inactive targets are skipped, exactly as the receiver
|
||||||
receiver skips them.
|
skips them.
|
||||||
|
|
||||||
The new event is a first-class event in the log with its own deliveries, not a
|
The new event is a first-class event in the log with its own deliveries, not a
|
||||||
marker on the one it came from, and the original's deliveries are left
|
marker on the one it came from, and the original's deliveries are left
|
||||||
@@ -2464,8 +2436,7 @@ in front of them, so a query on a fixed 200 URL would otherwise buy the same
|
|||||||
amplification as an invented path. Nothing debuggable is lost: the only query
|
amplification as an invented path. Nothing debuggable is lost: the only query
|
||||||
parameters this service reads are the sign-in page's `next`, the page to return
|
parameters this service reads are the sign-in page's `next`, the page to return
|
||||||
to, `notice`, which names the line a page shows after an action, and the event
|
to, `notice`, which names the line a page shows after an action, and the event
|
||||||
log's `show`, which picks the events it lists. A query string sent to an
|
log's `show`, which picks the events it lists.
|
||||||
entrypoint is not lost either: the event stores it, and the event log shows it.
|
|
||||||
|
|
||||||
Client-supplied request content does not leave the host by the other route
|
Client-supplied request content does not leave the host by the other route
|
||||||
either. The Sentry SDK attaches the request to every event it captures,
|
either. The Sentry SDK attaches the request to every event it captures,
|
||||||
@@ -2684,10 +2655,9 @@ What that ceiling does **not** cover, stated here so the figure is not read as
|
|||||||
more than it is:
|
more than it is:
|
||||||
|
|
||||||
- **Lines carrying an authenticated operator's own input**, which are not
|
- **Lines carrying an authenticated operator's own input**, which are not
|
||||||
truncated at all. `webhook created` logs the submitted `name` verbatim
|
truncated at all. `webhook created` logs the submitted `name` verbatim and
|
||||||
(`internal/handlers/webhook_create.go`) and
|
`target URL blocked by SSRF protection` logs the target host (both
|
||||||
`target URL blocked by SSRF protection` logs the target host
|
`internal/handlers/source_management.go`), as do the `target_name` lines in
|
||||||
(`internal/handlers/target_create.go`), as do the `target_name` lines in
|
|
||||||
`internal/delivery/engine.go` and `internal/delivery/target_http.go`. The only
|
`internal/delivery/engine.go` and `internal/delivery/target_http.go`. The only
|
||||||
bound on any of them is the 1 MB form body cap, so a 100 KB `name` writes a
|
bound on any of them is the 1 MB form body cap, so a 100 KB `name` writes a
|
||||||
single line of roughly 600 KB — measured. This is deliberate: every one of
|
single line of roughly 600 KB — measured. This is deliberate: every one of
|
||||||
@@ -2928,7 +2898,7 @@ page that was asked for.
|
|||||||
| `POST` | `/hook/{id}/edit` | Edit webhook submission |
|
| `POST` | `/hook/{id}/edit` | Edit webhook submission |
|
||||||
| `POST` | `/hook/{id}/delete` | Delete webhook |
|
| `POST` | `/hook/{id}/delete` | Delete webhook |
|
||||||
| `GET` | `/hook/{id}/events` | Full Event Log. `?show=failed` lists only the events with a failed delivery, and `?show=pending` only those with a delivery pending or retrying |
|
| `GET` | `/hook/{id}/events` | Full Event Log. `?show=failed` lists only the events with a failed delivery, and `?show=pending` only those with a delivery pending or retrying |
|
||||||
| `GET` | `/hook/{id}/events/{eventID}` | One event's own page: its details, the entrypoint it arrived at (for a resubmitted copy, the one the request it copies arrived at), its query string, its request headers, its whole body and every delivery of it |
|
| `GET` | `/hook/{id}/events/{eventID}` | One event's own page: its details, the entrypoint it arrived at (for a resubmitted copy, the one the request it copies arrived at), its request headers, its whole body and every delivery of it |
|
||||||
| `GET` | `/hook/{id}/events/{eventID}/body` | Download an event's stored body. The pages show a body as text, cut at 32 KiB in the recent events and the event log, and leave a binary one out, so this is the only route that serves the stored bytes; it is offered wherever a body is cut or binary |
|
| `GET` | `/hook/{id}/events/{eventID}/body` | Download an event's stored body. The pages show a body as text, cut at 32 KiB in the recent events and the event log, and leave a binary one out, so this is the only route that serves the stored bytes; it is offered wherever a body is cut or binary |
|
||||||
| `POST` | `/hook/{id}/deliveries/{deliveryID}/replay` | Replay a finished delivery: creates a new delivery for the same event against the target's current configuration (30 per minute per bucket, then `429`) |
|
| `POST` | `/hook/{id}/deliveries/{deliveryID}/replay` | Replay a finished delivery: creates a new delivery for the same event against the target's current configuration (30 per minute per bucket, then `429`) |
|
||||||
| `POST` | `/hook/{id}/events/{eventID}/resubmit` | Resubmit a stored event: creates a new event copying it and fans that out to every currently active target (30 per minute per bucket, then `429`) |
|
| `POST` | `/hook/{id}/events/{eventID}/resubmit` | Resubmit a stored event: creates a new event copying it and fans that out to every currently active target (30 per minute per bucket, then `429`) |
|
||||||
@@ -2980,6 +2950,8 @@ webhooker/
|
|||||||
├── internal/
|
├── internal/
|
||||||
│ ├── banner/
|
│ ├── banner/
|
||||||
│ │ └── banner.go # Ruled block for the one credential shown in the clear
|
│ │ └── banner.go # Ruled block for the one credential shown in the clear
|
||||||
|
│ ├── ciscript/
|
||||||
|
│ │ └── doc.go # Tests for the CI shell scripts in script/; no runtime code
|
||||||
│ ├── resetpw/
|
│ ├── resetpw/
|
||||||
│ │ └── resetpw.go # `webhooker resetpw`: set an account's password, stopped deployments only
|
│ │ └── resetpw.go # `webhooker resetpw`: set an account's password, stopped deployments only
|
||||||
│ ├── config/
|
│ ├── config/
|
||||||
@@ -3038,17 +3010,7 @@ webhooker/
|
|||||||
│ │ ├── index.go # Index page handler
|
│ │ ├── index.go # Index page handler
|
||||||
│ │ ├── profile.go # User profile handler
|
│ │ ├── profile.go # User profile handler
|
||||||
│ │ ├── settings.go # Read-only Settings page handler
|
│ │ ├── settings.go # Read-only Settings page handler
|
||||||
│ │ ├── webhook_list.go # Webhook list page
|
│ │ ├── source_management.go # Webhook CRUD handlers
|
||||||
│ │ ├── webhook_create.go # Webhook create
|
|
||||||
│ │ ├── webhook_detail.go # Webhook detail page
|
|
||||||
│ │ ├── webhook_edit.go # Webhook edit, archive renaming
|
|
||||||
│ │ ├── webhook_delete.go # Webhook delete, event database and archive writer removal
|
|
||||||
│ │ ├── event_log.go # Event log page: loaders, filters, delivery views
|
|
||||||
│ │ ├── entrypoint.go # Entrypoint create, edit, delete and toggle
|
|
||||||
│ │ ├── target_create.go # Target create, per-type config builders
|
|
||||||
│ │ ├── target_delete.go # Target delete
|
|
||||||
│ │ ├── target_toggle.go # Target toggle
|
|
||||||
│ │ ├── shared.go # Helpers shared by several handlers
|
|
||||||
│ │ └── webhook.go # Webhook receiver handler
|
│ │ └── webhook.go # Webhook receiver handler
|
||||||
│ ├── healthcheck/
|
│ ├── healthcheck/
|
||||||
│ │ └── healthcheck.go # Health check service (uptime, version)
|
│ │ └── healthcheck.go # Health check service (uptime, version)
|
||||||
@@ -3084,15 +3046,14 @@ webhooker/
|
|||||||
│ └── js/alpine.min.js # Alpine.js CSP build, extracted from 3p/ by make assets, not committed
|
│ └── js/alpine.min.js # Alpine.js CSP build, extracted from 3p/ by make assets, not committed
|
||||||
├── templates/ # Go HTML templates (base, login, sources, etc.)
|
├── templates/ # Go HTML templates (base, login, sources, etc.)
|
||||||
├── script/ # Scripts to Rule Them All entrypoints
|
├── script/ # Scripts to Rule Them All entrypoints
|
||||||
├── Dockerfile # Stages: stylesheet, JavaScript lint, Markdown, lint, test, build, Alpine runtime
|
├── Dockerfile # Stages: lint, stylesheet, JavaScript lint, Markdown, test+build, Alpine runtime
|
||||||
|
├── Dockerfile.lint # Lint-only image built by script/lint
|
||||||
├── Dockerfile.browser # Browser test image built by script/test-browser
|
├── Dockerfile.browser # Browser test image built by script/test-browser
|
||||||
├── Makefile # 13 of 19 targets shim script/; 6 are inline
|
├── Makefile # 13 of 19 targets shim script/; 6 are inline
|
||||||
├── go.mod / go.sum
|
├── go.mod / go.sum
|
||||||
├── package.json / yarn.lock # ESLint, prettier and yarn, pinned, for the JavaScript lint and Markdown stages
|
├── package.json / yarn.lock # ESLint and prettier, pinned, for the JavaScript lint and Markdown stages
|
||||||
├── .yarnrc.yml # yarn settings: install into node_modules/
|
|
||||||
├── eslint.config.mjs # ESLint configuration for static/js/
|
├── eslint.config.mjs # ESLint configuration for static/js/
|
||||||
├── .prettierrc # prettier settings for the Markdown
|
├── .prettierrc # prettier settings for the Markdown
|
||||||
├── .prettierignore # Files prettier skips
|
|
||||||
└── .golangci.yml # golangci-lint configuration
|
└── .golangci.yml # golangci-lint configuration
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -3348,36 +3309,43 @@ Two operational consequences follow from bounding the sequence:
|
|||||||
|
|
||||||
### Linting
|
### Linting
|
||||||
|
|
||||||
golangci-lint never runs on the host. `script/lint` builds the Dockerfile's
|
golangci-lint never runs on the host. `script/lint` builds `Dockerfile.lint`,
|
||||||
`lint` phase, which copies the repo into the digest-pinned golangci-lint image
|
which copies the repo into the digest-pinned golangci-lint image and lints as a
|
||||||
and lints as a build step, so a successful build is a clean lint. A host binary
|
build step, so a successful build is a clean lint. A host binary would share one
|
||||||
would share one cache and one lock with every other checkout on the machine,
|
cache and one lock with every other checkout on the machine, which has produced
|
||||||
which has produced both invented findings attributed to other worktrees and
|
both invented findings attributed to other worktrees and unearned passes.
|
||||||
unearned passes.
|
|
||||||
|
|
||||||
Two properties are load-bearing:
|
Three properties are load-bearing:
|
||||||
|
|
||||||
- `script/lint` passes `--no-cache`. Without it an unchanged tree replays the
|
- `script/lint` passes `--no-cache-filter=lint`. Without it an unchanged tree
|
||||||
lint layer from cache and the build exits 0 in under a second having linted
|
replays the lint layer from cache and the build exits 0 in under a second
|
||||||
nothing. Never prune the shared build cache instead.
|
having linted nothing. The `deps` stage stays cacheable, so module downloads
|
||||||
- Both golangci-lint steps use `RUN --network=none`.
|
are not repeated. Invalidation is scoped to the one stage; never prune the
|
||||||
`golangci-lint config verify` is documented as fetching its JSON schema over
|
shared build cache.
|
||||||
HTTPS, which would be an unpinned remote dependency; the pinned image resolves
|
- `script/lint` does not trust that flag. Docker silently ignores
|
||||||
the schema without network access, and `--network=none` enforces that instead
|
`--no-cache-filter` for a stage name that does not match, so a stage rename or
|
||||||
of trusting it. Verify is worth keeping because `golangci-lint run` silently
|
a one-character typo would restore the cached false green with no warning and
|
||||||
ignores config keys it does not recognize, so a typo would disable a setting
|
a fast exit 0. The script therefore tees the build output and treats a run as
|
||||||
with no warning.
|
a pass only if golangci-lint's own summary line (`N issues.` / `N issues:`)
|
||||||
|
appears in it: no summary, no lint, whatever the exit code says.
|
||||||
|
- Both lint steps use `RUN --network=none`. `golangci-lint config verify` is
|
||||||
|
documented as fetching its JSON schema over HTTPS, which would be an unpinned
|
||||||
|
remote dependency; the pinned image resolves the schema without network
|
||||||
|
access, and `--network=none` enforces that instead of trusting it. Verify is
|
||||||
|
worth keeping because `golangci-lint run` silently ignores config keys it does
|
||||||
|
not recognize, so a typo would disable a setting with no warning.
|
||||||
|
|
||||||
ESLint never runs on the host either. It lints `static/js/` (not the extracted
|
ESLint never runs on the host either. It lints `static/js/` (not the extracted
|
||||||
Alpine.js) in the Dockerfile's `js-lint` stage. The `lint` phase copies a file
|
Alpine.js) in the Dockerfile's `js-lint` stage, which `script/lint` builds after
|
||||||
from it, so `make lint` and the image build both run ESLint. Its version is
|
`Dockerfile.lint` and the image build runs before the builder stage. Its version
|
||||||
pinned in `package.json` and every package's hash in `yarn.lock`. The `js-deps`
|
is pinned in `package.json` and every package's hash in `yarn.lock`. The
|
||||||
stage before it installs ESLint with `yarn install --immutable`, which fails
|
`js-deps` stage before it installs ESLint and stays cached until either file
|
||||||
rather than change `yarn.lock`. The yarn it runs is the one the `packageManager`
|
changes, so only the lint step re-runs and ESLint is not downloaded again.
|
||||||
field in `package.json` pins by version and hash, which the node image's own
|
`eslint.config.mjs` turns on the rules of the JavaScript styleguide
|
||||||
corepack fetches and checks. `eslint.config.mjs` turns on the rules of the
|
`REPO_POLICIES.md` links to that a linter can check: `no-var` and
|
||||||
JavaScript styleguide `REPO_POLICIES.md` links to that a linter can check:
|
`prefer-const`. ESLint prints nothing on a pass, so `script/lint` has no summary
|
||||||
`no-var` and `prefer-const`.
|
line to look for; it names the stage once for both `--target` and
|
||||||
|
`--no-cache-filter`, and `--target` fails on a name that matches no stage.
|
||||||
|
|
||||||
prettier formats the Markdown, and it never runs on the host either. It is
|
prettier formats the Markdown, and it never runs on the host either. It is
|
||||||
pinned in `package.json` and `yarn.lock` beside ESLint, installed by the same
|
pinned in `package.json` and `yarn.lock` beside ESLint, installed by the same
|
||||||
@@ -3389,81 +3357,102 @@ on any Markdown file prettier would change.
|
|||||||
### Docker
|
### Docker
|
||||||
|
|
||||||
The Dockerfile uses a multi-stage build. Each stage is pinned by digest, and the
|
The Dockerfile uses a multi-stage build. Each stage is pinned by digest, and the
|
||||||
lint phase and the Go stages are separate images so the linter's version is
|
lint and builder stages are separate images so the linter's version is fixed
|
||||||
fixed independently of the compiler's:
|
independently of the compiler's:
|
||||||
|
|
||||||
1. **Stylesheet stages** (`debian:bookworm-slim`, with the Tailwind standalone
|
1. **Lint stage** (`golangci/golangci-lint:v2.12.2`, Debian-based) — installs
|
||||||
|
`make`, downloads dependencies, copies the source, and runs the `gofmt`
|
||||||
|
check, then `script/assets` to extract Alpine.js from `3p/`, then
|
||||||
|
`golangci-lint config verify` and `golangci-lint run`, both with
|
||||||
|
`--network=none`.
|
||||||
|
2. **Stylesheet stages** (`debian:bookworm-slim`, with the Tailwind standalone
|
||||||
CLI pinned by version and sha256, one binary per architecture) — generate
|
CLI pinned by version and sha256, one binary per architecture) — generate
|
||||||
`static/css/tailwind.css` from `static/css/input.css` and the files its
|
`static/css/tailwind.css` from `static/css/input.css` and the files its
|
||||||
`@source` lines name. `css-check` fails when the committed file differs from
|
`@source` lines name. `css-check` fails when the committed file differs from
|
||||||
the generated one, and `make css` writes the generated file out from
|
the generated one, and `make css` writes the generated file out from
|
||||||
`css-output` (see [Stylesheet](#stylesheet)).
|
`css-output` (see [Stylesheet](#stylesheet)).
|
||||||
2. **JavaScript lint stages** (`node:24.21.0-alpine`, with the yarn
|
3. **JavaScript lint stages** (`node:24.21.0-alpine`, with yarn) — `js-deps`
|
||||||
`package.json` pins, run through the image's corepack) — `js-deps` installs
|
installs ESLint and prettier from `yarn.lock` and `js-lint` runs ESLint over
|
||||||
ESLint and prettier from `yarn.lock` and `js-lint` runs ESLint over
|
|
||||||
`static/js/` (see [Linting](#linting)).
|
`static/js/` (see [Linting](#linting)).
|
||||||
3. **Markdown stages** (on `js-deps`) — `markdown-check` runs prettier over the
|
4. **Markdown stages** (on `js-deps`) — `markdown-check` runs prettier over the
|
||||||
Markdown and fails on any file it would change, and `make fmt` writes the
|
Markdown and fails on any file it would change, and `make fmt` writes the
|
||||||
formatted files out from `markdown-output`.
|
formatted files out from `markdown-output`.
|
||||||
4. **Lint phase** (`lint`, `golangci/golangci-lint:v2.14.0`, Debian-based) —
|
5. **Builder stage** (`golang:1.26.1-bookworm`) — depends on the lint,
|
||||||
downloads dependencies, copies the source, and runs the `gofmt` check, then
|
`css-check`, `js-lint` and `markdown-check` stages passing (it copies a file
|
||||||
`script/assets` to extract Alpine.js from `3p/`, then
|
from each), runs `make test` and `make build` (both extract Alpine.js from
|
||||||
`golangci-lint config verify` and `golangci-lint run`, both with
|
`3p/` first), and finally rebuilds the binary with `CGO_ENABLED=1` and static
|
||||||
`--network=none`, and depends on `js-lint` (it copies a file from it).
|
linking so it runs on musl. Both builds go through `make build`, the relink
|
||||||
`make lint` builds this stage alone.
|
adding its `-extldflags` via `GO_LDFLAGS`, so neither can drop the `-X` that
|
||||||
5. **Test phase** (`test`, `golang:1.26.1-bookworm`, whose C compiler `-race`
|
stamps the version. The version is the `VERSION` build arg if one is given,
|
||||||
needs) — extracts Alpine.js, then runs `go test -race -cover` at most four
|
|
||||||
packages and eight tests at a time, with a 90-second timeout per package. On
|
|
||||||
a failure it runs only the failed tests again with `-v`, since verbose output
|
|
||||||
from the whole suite would pass the 2 MiB at which the Docker build cuts off
|
|
||||||
a step's log, and then fails. `make test` builds this stage alone.
|
|
||||||
6. **Builder stage** (`golang:1.26.1-bookworm`) — depends on the lint and test
|
|
||||||
phases and the `css-check` and `markdown-check` stages passing (it copies a
|
|
||||||
file from each), runs `make build` (which extracts Alpine.js from `3p/`
|
|
||||||
first), and then rebuilds the binary with `CGO_ENABLED=1` and static linking
|
|
||||||
so it runs on musl. Both builds go through `make build`, the relink adding
|
|
||||||
its `-extldflags` via `GO_LDFLAGS`, so neither can drop the `-X` that stamps
|
|
||||||
the version. The version is the `VERSION` build arg if one is given,
|
|
||||||
otherwise derived from the `.git` in the context, and the stage fails if a
|
otherwise derived from the `.git` in the context, and the stage fails if a
|
||||||
context with `.git` would stamp an empty version, `dev` or `unknown` (see
|
context with `.git` would stamp `unknown` (see
|
||||||
[Version stamping](#version-stamping)).
|
[Version stamping](#version-stamping)).
|
||||||
7. **Runtime stage** (`alpine:3.21`) — copies the static binary and
|
6. **Runtime stage** (`alpine:3.21`) — copies the static binary and
|
||||||
`deploy/docker-entrypoint.sh`, creates the `/var/lib/webhooker` directory for
|
`deploy/docker-entrypoint.sh`, creates the `/var/lib/webhooker` directory for
|
||||||
all SQLite databases, exposes port 8080, and includes a health check against
|
all SQLite databases, exposes port 8080, and includes a health check against
|
||||||
`/.well-known/healthcheck`. It sets no `USER`: the `ENTRYPOINT` script starts
|
`/.well-known/healthcheck`. It sets no `USER`: the `ENTRYPOINT` script starts
|
||||||
as root, sets the data directory's owner and mode, and runs the app as the
|
as root, sets the data directory's owner and mode, and runs the app as the
|
||||||
non-root `webhooker` user (UID 1000) through `su-exec`.
|
non-root `webhooker` user (UID 1000) through `su-exec`.
|
||||||
|
|
||||||
The lint and test phases invoke `gofmt`, `golangci-lint` and `go test` directly
|
The lint stage invokes `gofmt` and `golangci-lint` directly rather than
|
||||||
rather than `make fmt-check`, `make lint` and `make test`: those targets build
|
`make fmt-check` and `make lint`: it is already the pinned linter image, and
|
||||||
docker stages, which would need a docker daemon inside this build.
|
both targets build docker stages, which would need a docker daemon inside this
|
||||||
|
build.
|
||||||
|
|
||||||
The lint phase and the Go stages use Debian rather than Alpine because
|
The lint and builder stages use Debian rather than Alpine because
|
||||||
`gorm.io/driver/sqlite` pulls in `mattn/go-sqlite3`, which needs CGO and does
|
`gorm.io/driver/sqlite` pulls in `mattn/go-sqlite3`, which needs CGO and does
|
||||||
not compile against musl. Only the final binary is statically linked, which is
|
not compile against musl. Only the final binary is statically linked, which is
|
||||||
what lets it run on the Alpine runtime image.
|
what lets it run on the Alpine runtime image.
|
||||||
|
|
||||||
`script/cibuild` is the CI gate: it runs `script/bootstrap`, then
|
`script/cibuild` — `docker build .` — is the CI gate: the checks run inside the
|
||||||
`script/check`, then builds the image, whose build runs the lint and test phases
|
image, so a build that succeeds is a repo that is formatted, linted, tested and
|
||||||
and the stylesheet and Markdown checks again. A build that succeeds is a repo
|
compiled, with a current stylesheet. `script/lint` also uses Docker
|
||||||
that is formatted, linted, tested and compiled, with a current stylesheet.
|
(`Dockerfile.lint` and the `js-lint` stage, see Linting above), so `make lint`
|
||||||
`make check` runs the same stages the gate does; of its steps, only the `gofmt`
|
and `make check` run the same pinned linter versions the gate does; of the steps
|
||||||
check in `script/fmt-check` runs on the host.
|
`make check` runs, only `script/test` and the `gofmt` check in
|
||||||
|
`script/fmt-check` run on the host.
|
||||||
|
|
||||||
#### CI gate honesty
|
#### CI gate honesty
|
||||||
|
|
||||||
A layer cache lets `docker build .` exit 0 in seconds with the lint and test
|
A layer cache lets `docker build .` exit 0 in seconds with the lint and test
|
||||||
stages replayed rather than executed, which would make a green check
|
stages replayed rather than executed, which would make a green check
|
||||||
meaningless. Every `docker build` in `script/` therefore passes `--no-cache`, so
|
meaningless. The `check` workflow therefore writes `.ci-fingerprint` into the
|
||||||
on every run the `gofmt` check, `golangci-lint`, ESLint, the stylesheet check,
|
build context before building. Its value is the hash of the commit being
|
||||||
the Markdown check, `go test` and `make build` really execute. A run that
|
checked, so every commit, docs-only ones and a squash merge whose tree matches
|
||||||
reports success ran them. A bare `docker build .` carries no such guarantee.
|
an already-built branch included, gets a new fingerprint, invalidates the
|
||||||
|
`COPY . .` layer of every check stage, and really runs the `gofmt` check,
|
||||||
|
`golangci-lint`, the stylesheet check, ESLint, the Markdown check, `make test`,
|
||||||
|
and `make build`. A run that reports success ran them.
|
||||||
|
|
||||||
The `check` workflow is the shared one from `REPO_POLICIES.md`: it checks out
|
The module download layer sits above `COPY . .` and stays cached.
|
||||||
the repository and runs `script/cibuild`, nothing else. Gitea cancels an
|
|
||||||
in-flight run when a newer commit lands on the same branch and records that as
|
A separate workflow step, run before the fingerprint is written, covers a second
|
||||||
`failure` / `Has been cancelled`: nothing was verified about that commit, so
|
way the gate lied: Gitea cancels an in-flight run when a newer commit lands on
|
||||||
test the commit itself before concluding anything about it.
|
the same branch and records that cancellation as a `failure` status, so a commit
|
||||||
|
nothing ever tested reads as a test result. Cancellation is unconditional
|
||||||
|
server-side for push events, so the superseding run calls
|
||||||
|
`script/ci-mark-superseded`, which rewrites that exact status to `failure` /
|
||||||
|
`Superseded by a newer commit; never tested`.
|
||||||
|
|
||||||
|
The state stays `failure` on purpose: Gitea's combined status folds `skipped`
|
||||||
|
into `success`, so marking a never-tested commit `skipped` made the status API
|
||||||
|
report green for it, indistinguishable from a commit that passed. Reading a
|
||||||
|
commit's status on this repo therefore goes:
|
||||||
|
|
||||||
|
- `success` / `Successful in ...` — the checks ran and passed.
|
||||||
|
- `failure` / `Failing after ...` — the checks ran and failed.
|
||||||
|
- `failure` / `Superseded by a newer commit; never tested` — the run was
|
||||||
|
cancelled, by a newer push or by hand, and nothing was verified about this
|
||||||
|
commit. Test the commit itself before concluding anything about it.
|
||||||
|
|
||||||
|
Genuine failures and successes are never touched, and no status is left
|
||||||
|
`pending`, which would block the commit indefinitely. The step derives its
|
||||||
|
context string from the workflow name, the job **id** and the event. That is
|
||||||
|
deliberately not byte-identical to Gitea's own rule, which uses the job's
|
||||||
|
display `name:` where the runner exports the id, so giving the job a `name:` —
|
||||||
|
or renaming the workflow — makes the derived context stop matching. The step
|
||||||
|
fails loudly when no status on the commit carries that context, so no rename can
|
||||||
|
silently disable the rewrite.
|
||||||
|
|
||||||
## TODO
|
## TODO
|
||||||
|
|
||||||
|
|||||||
+86
-349
@@ -1,6 +1,6 @@
|
|||||||
---
|
---
|
||||||
title: Repository Policies
|
title: Repository Policies
|
||||||
last_modified: 2026-10-04
|
last_modified: 2026-08-07
|
||||||
---
|
---
|
||||||
|
|
||||||
This document covers repository structure, tooling, and workflow standards. Code
|
This document covers repository structure, tooling, and workflow standards. Code
|
||||||
@@ -60,28 +60,17 @@ style conventions are in separate documents:
|
|||||||
prerequisite since nvm requires bash. yarn is then pinned via
|
prerequisite since nvm requires bash. yarn is then pinned via
|
||||||
`corepack prepare yarn@<version> --activate`. Never install "latest" or "lts";
|
`corepack prepare yarn@<version> --activate`. Never install "latest" or "lts";
|
||||||
always exact versions. `script/cibuild` runs the CI build: it changes to the
|
always exact versions. `script/cibuild` runs the CI build: it changes to the
|
||||||
repo root, runs `script/bootstrap`, runs `script/check`, and builds the image
|
repo root and runs `docker build .`; the Gitea workflow calls it. Four further
|
||||||
with the version; the Gitea workflow calls it. **`script/cibuild` runs
|
scripts are our own extensions to the standard: `script/check` runs
|
||||||
`script/bootstrap` first**, because the workflow checks out the repo and runs
|
`script/test`, `script/lint`, and `script/fmt-check`; `script/precommit` is
|
||||||
nothing else, while `script/fmt-check` runs the formatter on the host: on a
|
what the git pre-commit hook runs, and it calls `script/check`;
|
||||||
pristine checkout with nothing installed the run dies there, after the
|
`script/install-precommit` installs the git pre-commit hook (the `make hooks`
|
||||||
containerised gates have passed. **The bootstrap alone is not enough**:
|
target shims to it); and `script/projectname` (literally that filename) simply
|
||||||
`script/bootstrap` installs node and yarn under nvm and leaves neither on the
|
outputs the project's name. Scripts that need the name call
|
||||||
`PATH` of the shell that called it, so a bare `yarn` still exits 127. The host
|
`script/projectname` — e.g. `script/docker` assembles its image tag from it —
|
||||||
entrypoints that need yarn — `script/fmt` and `script/fmt-check` — therefore
|
so those scripts stay byte-identical across all repos. Repo-type-specific
|
||||||
source nvm for the pinned node version before invoking it, exactly as
|
pre-commit extras (e.g. `go mod tidy` verification in Go repos) belong in
|
||||||
`script/bootstrap`'s own install step does. A runner carrying nothing but
|
`script/precommit`, not in the hook itself. Model scripts are at
|
||||||
docker and git then gets through `script/check`. Four further scripts are our
|
|
||||||
own extensions to the standard: `script/check` runs `script/test`,
|
|
||||||
`script/lint` and `script/fmt-check`; `script/precommit` is what the git
|
|
||||||
pre-commit hook runs, and it calls `script/check`; `script/install-precommit`
|
|
||||||
installs the git pre-commit hook (the `make hooks` target shims to it); and
|
|
||||||
`script/projectname` (literally that filename) simply outputs the project's
|
|
||||||
name. Scripts that need the name call `script/projectname` — e.g.
|
|
||||||
`script/docker` assembles its image tag from it — so those scripts stay
|
|
||||||
byte-identical across all repos. Repo-type-specific pre-commit extras (e.g.
|
|
||||||
`go mod tidy` verification in Go repos) belong in `script/precommit`, not in
|
|
||||||
the hook itself. Model scripts are at
|
|
||||||
`https://git.eeqj.de/sneak/prompts/raw/branch/main/script/<name>`. The README
|
`https://git.eeqj.de/sneak/prompts/raw/branch/main/script/<name>`. The README
|
||||||
must document the provided scripts in an **Entrypoints** section (see the
|
must document the provided scripts in an **Entrypoints** section (see the
|
||||||
README requirements below).
|
README requirements below).
|
||||||
@@ -100,198 +89,87 @@ style conventions are in separate documents:
|
|||||||
contributor should be able to understand the entire development workflow by
|
contributor should be able to understand the entire development workflow by
|
||||||
reading the Makefile.
|
reading the Makefile.
|
||||||
|
|
||||||
- Every repo should have a `Dockerfile`, and it carries the repo's gates: a
|
- Every repo should have a `Dockerfile`. All Dockerfiles must run `make check`
|
||||||
`lint` phase and a `test` phase, with the final stage depending on both so the
|
as a build step so the build fails if the branch is not green. For non-server
|
||||||
image cannot be built unless they pass. For non-server repos the final stage
|
repos, the Dockerfile should bring up a development environment and run
|
||||||
brings up a development environment; for server repos it is the runtime image.
|
`make check`. For server repos, `make check` should run as an early build
|
||||||
The gate phases and the build stage start from their pinned base images and
|
stage before the final image is assembled. Dockerfiles install development
|
||||||
install what those images lack either inline, as the canonical Go `Dockerfile`
|
prerequisites by running `script/bootstrap` rather than duplicating installs
|
||||||
below does for `git`, or by running `script/bootstrap`, as the `prompts`
|
inline; COPY `script/` and the dependency manifests (`package.json` +
|
||||||
repo's own `Dockerfile` does for its yarn packages. The development
|
`yarn.lock`, `go.mod` + `go.sum`, etc.) before running it so the bootstrap
|
||||||
environment stage installs development prerequisites by running
|
layer stays cached until dependencies change.
|
||||||
`script/bootstrap` rather than duplicating its installs inline. A stage that
|
|
||||||
runs `script/bootstrap` COPYs `script/` and the dependency manifests
|
|
||||||
(`package.json` + `yarn.lock`, `go.mod` + `go.sum`, etc.) before running it.
|
|
||||||
|
|
||||||
- **Linting and testing run in Docker, as phases of the `Dockerfile`.** There is
|
- **Dockerfiles must use a separate lint stage for fail-fast feedback.** Go
|
||||||
no separate lint file. `script/lint` and `script/test` each build one phase
|
repos use a multistage build where linting runs in an independent stage based
|
||||||
and nothing else:
|
on the `golangci/golangci-lint` image (pinned by hash). This stage runs
|
||||||
|
`make fmt-check` and `make lint` before the full build begins. The build stage
|
||||||
|
then declares an explicit dependency on the lint stage via
|
||||||
|
`COPY --from=lint /src/go.sum /dev/null`, which forces BuildKit to complete
|
||||||
|
linting before proceeding to compilation and tests. This ensures lint failures
|
||||||
|
surface in seconds rather than minutes, without blocking on dependency
|
||||||
|
download or compilation in the build stage.
|
||||||
|
|
||||||
```sh
|
The standard pattern for a Go repo Dockerfile is:
|
||||||
docker build --no-cache --target lint -t "$(script/projectname)-lint" .
|
|
||||||
docker build --no-cache --target test -t "$(script/projectname)-test" .
|
|
||||||
```
|
|
||||||
|
|
||||||
**A stage that is not the last one in the file is built only when the final
|
|
||||||
stage's chain depends on it, or when `--target` names it.** That is why the
|
|
||||||
two gates are always invoked by name here, and why the final stage carries a
|
|
||||||
`COPY --from=` of a harmless file from each of them: without that edge a
|
|
||||||
plain `docker build .` builds the last stage alone and exits 0 having linted
|
|
||||||
and tested nothing.
|
|
||||||
|
|
||||||
**Every `docker build` in `script/` is tagged**, here and in
|
|
||||||
`script/cibuild` and `script/docker`. An untagged build leaves a dangling
|
|
||||||
image behind on every invocation, on every developer host and every CI
|
|
||||||
runner; a tagged one replaces the previous image.
|
|
||||||
|
|
||||||
Inside a phase the tool is invoked directly — `golangci-lint`, `go test`,
|
|
||||||
`eslint`, `prettier` — never through `make lint` or `script/test`, which are
|
|
||||||
themselves a `docker build` and would recurse into a daemon that does not
|
|
||||||
exist in a build step. Formatting is the exception and stays on the host:
|
|
||||||
`script/fmt` writes the working tree, and `script/fmt-check` is its
|
|
||||||
read-only twin.
|
|
||||||
|
|
||||||
**No lint verdict may come from a host invocation of the linter.** On a
|
|
||||||
shared host golangci-lint reads a result cache keyed on file content rather
|
|
||||||
than location, so a second checkout of the same content is served the first
|
|
||||||
one's findings, and a host-global lock in `$TMPDIR` makes concurrent runs
|
|
||||||
exit non-zero with `parallel golangci-lint is running` — a status a caller
|
|
||||||
cannot tell from real findings. Both have produced wrong verdicts in this
|
|
||||||
org, in both directions. A container has its own cache, its own `TMPDIR` and
|
|
||||||
a digest-pinned binary, so neither is reachable.
|
|
||||||
|
|
||||||
- **Any build that runs checks is built with `--no-cache`.** Docker invalidates
|
|
||||||
a `COPY` layer only when the copied content changes, so on an unchanged tree
|
|
||||||
the check `RUN` is served from cache, nothing executes, and the build still
|
|
||||||
exits 0. Every `docker build` in `script/` therefore passes `--no-cache`:
|
|
||||||
`script/lint`, `script/test`, `script/cibuild` and `script/docker` are the
|
|
||||||
four, and there is no fifth — `script/check` runs the two gate phases and
|
|
||||||
`script/fmt-check`, and builds no image of its own. A bare `docker build .` is
|
|
||||||
not evidence that anything ran: a sub-second build reporting success is a
|
|
||||||
cache hit, not a result. Never invalidate by pruning — `docker builder prune`
|
|
||||||
and friends destroy a build cache shared with every other build on the host.
|
|
||||||
When a check is added or changed, prove it works by planting a defect it must
|
|
||||||
catch and watching the run fail on it, then revert the defect. A green run
|
|
||||||
alone shows neither that the check ran nor that it covers what it should.
|
|
||||||
|
|
||||||
- **The gate phases are separate stages, and the build stage depends on both.**
|
|
||||||
The lint phase is based on the `golangci/golangci-lint` image (pinned by
|
|
||||||
hash), so lint failures surface in seconds rather than after a full compile,
|
|
||||||
and the test phase is based on the Debian Go image. The canonical Go repo
|
|
||||||
`Dockerfile`:
|
|
||||||
|
|
||||||
```dockerfile
|
```dockerfile
|
||||||
# Lint phase
|
# Lint stage — fast feedback on formatting and lint issues
|
||||||
# golangci/golangci-lint:v2.x.x, YYYY-MM-DD
|
# golangci/golangci-lint:v2.x.x, YYYY-MM-DD
|
||||||
FROM golangci/golangci-lint@sha256:... AS lint
|
FROM golangci/golangci-lint@sha256:... AS lint
|
||||||
WORKDIR /src
|
WORKDIR /src
|
||||||
COPY go.mod go.sum ./
|
COPY go.mod go.sum ./
|
||||||
RUN go mod download
|
RUN go mod download
|
||||||
COPY . .
|
COPY . .
|
||||||
RUN golangci-lint run --config .golangci.yml ./...
|
RUN make fmt-check
|
||||||
|
RUN make lint
|
||||||
|
|
||||||
# Test phase. -race needs cgo and so a C compiler, which the Debian Go
|
# Build stage
|
||||||
# image ships and the alpine one does not.
|
|
||||||
# golang:1.x, YYYY-MM-DD
|
|
||||||
FROM golang@sha256:... AS test
|
|
||||||
WORKDIR /src
|
|
||||||
COPY go.mod go.sum ./
|
|
||||||
RUN go mod download
|
|
||||||
COPY . .
|
|
||||||
RUN go test -timeout 90s -race -cover ./... || \
|
|
||||||
{ echo "--- Rerunning with -v for details ---"; \
|
|
||||||
go test -timeout 90s -race -v ./...; exit 1; }
|
|
||||||
|
|
||||||
# Build stage. Nothing is wanted from either phase above; the copies
|
|
||||||
# are what make BuildKit build them first, so this stage cannot run
|
|
||||||
# unless lint and test passed.
|
|
||||||
# golang:1.x-alpine, YYYY-MM-DD
|
# golang:1.x-alpine, YYYY-MM-DD
|
||||||
FROM golang@sha256:... AS builder
|
FROM golang@sha256:... AS builder
|
||||||
COPY --from=lint /src/go.sum /dev/null
|
|
||||||
COPY --from=test /src/go.sum /dev/null
|
|
||||||
RUN apk add --no-cache git
|
|
||||||
# A tar-stream context keeps the sender's file owners, which git refuses.
|
|
||||||
RUN git config --system --add safe.directory /src
|
|
||||||
WORKDIR /src
|
WORKDIR /src
|
||||||
|
|
||||||
|
# Force BuildKit to run the lint stage before proceeding
|
||||||
|
COPY --from=lint /src/go.sum /dev/null
|
||||||
|
|
||||||
COPY go.mod go.sum ./
|
COPY go.mod go.sum ./
|
||||||
RUN go mod download
|
RUN go mod download
|
||||||
COPY . .
|
COPY . .
|
||||||
|
RUN make test
|
||||||
|
|
||||||
# The VERSION build arg when one is given, otherwise
|
ARG VERSION=dev
|
||||||
# `git describe --tags --always` on the .git in the build context. With
|
RUN CGO_ENABLED=0 go build -trimpath \
|
||||||
# .git present, a version that is still empty, dev or unknown fails the
|
|
||||||
# build: git is missing or could not read the checkout.
|
|
||||||
ARG VERSION
|
|
||||||
RUN VERSION="${VERSION:-$(git describe --tags --always)}"; \
|
|
||||||
if [ -e .git ]; then \
|
|
||||||
case "$VERSION" in ""|dev|unknown) \
|
|
||||||
echo "version is '$VERSION' although .git is present" >&2; \
|
|
||||||
exit 1 ;; \
|
|
||||||
esac; \
|
|
||||||
fi; \
|
|
||||||
CGO_ENABLED=0 go build -trimpath \
|
|
||||||
-ldflags="-s -w -X main.Version=${VERSION}" \
|
-ldflags="-s -w -X main.Version=${VERSION}" \
|
||||||
-o /app ./cmd/app/
|
-o /app ./cmd/app/
|
||||||
|
|
||||||
# Runtime stage, and the last one
|
# Runtime stage
|
||||||
FROM alpine@sha256:...
|
FROM alpine@sha256:...
|
||||||
COPY --from=builder /app /usr/local/bin/app
|
COPY --from=builder /app /usr/local/bin/app
|
||||||
ENTRYPOINT ["app"]
|
ENTRYPOINT ["app"]
|
||||||
```
|
```
|
||||||
|
|
||||||
Key points:
|
Key points:
|
||||||
- The lint phase uses the `golangci/golangci-lint` image directly (it has
|
- The lint stage uses the `golangci/golangci-lint` image directly (it
|
||||||
both Go and the linter), so nothing needs installing.
|
includes both Go and the linter), so there is no need to install the
|
||||||
- `COPY --from=<phase> /src/go.sum /dev/null` is a no-op copy whose only
|
linter separately.
|
||||||
purpose is the ordering edge. BuildKit runs stages in parallel by default,
|
- `COPY --from=lint /src/go.sum /dev/null` is a no-op file copy that creates
|
||||||
and a stage nothing depends on is not built at all, so without these two
|
a stage dependency. BuildKit runs stages in parallel by default; without
|
||||||
lines a red gate would not fail the build.
|
this line, the build stage would not wait for lint to finish and a lint
|
||||||
- Keep the runtime stage last, and if you add a stage after it, give it the
|
failure might not fail the overall build.
|
||||||
same two copies. A plain `docker build .` builds the last stage's chain
|
|
||||||
and nothing else.
|
|
||||||
- If the project uses `//go:embed` directives that reference build artifacts
|
- If the project uses `//go:embed` directives that reference build artifacts
|
||||||
(e.g. a web frontend compiled in a separate stage), the lint phase must
|
(e.g. a web frontend compiled in a separate stage), the lint stage must
|
||||||
create placeholder files so the embed directives resolve. Example:
|
create placeholder files so the embed directives resolve. Example:
|
||||||
`RUN mkdir -p web/dist && touch web/dist/index.html web/dist/style.css`.
|
`RUN mkdir -p web/dist && touch web/dist/index.html web/dist/style.css`.
|
||||||
- If the project requires CGO or system libraries for linting, install them
|
The lint stage should not depend on the actual build output — it exists to
|
||||||
in the lint phase. The `golangci/golangci-lint` image is Debian-based and
|
fail fast.
|
||||||
has no `apk`, so install with `apt-get` under the Debian package name
|
- If the project requires CGO or system libraries for linting (e.g.
|
||||||
(`libvips-dev`, where alpine says `vips-dev`), and delete the package
|
`vips-dev`), install them in the lint stage with `apk add`.
|
||||||
lists in the same `RUN`, so the layer does not keep them:
|
- The build stage runs `make test` after compilation setup. Tests run in the
|
||||||
|
build stage, not the lint stage, because they may require compiled
|
||||||
```dockerfile
|
artifacts or heavier dependencies.
|
||||||
RUN apt-get update \
|
|
||||||
&& apt-get install -y --no-install-recommends libvips-dev \
|
|
||||||
&& rm -rf /var/lib/apt/lists/*
|
|
||||||
```
|
|
||||||
|
|
||||||
- `.dockerignore` lets `.git` into the build context. It keeps out every git
|
|
||||||
`config` at any depth (`**/.git/config`, `**/.git/modules/**/config`): the
|
|
||||||
repository's own, each submodule's under `.git/modules/`, and that of a
|
|
||||||
submodule keeping its own `.git` directory. `git describe` does not need
|
|
||||||
them, and each can hold a credential: a password in a remote URL, or the
|
|
||||||
token the CI checkout step stores there. A submodule whose name has a
|
|
||||||
`config` segment (`config`, `deploy/config`, `config/lib`) loses its whole
|
|
||||||
git directory to `**/.git/modules/**/config`, and Go's version stamping
|
|
||||||
then fails the build: give it a name without that segment
|
|
||||||
(`git submodule add --name`). The stage that compiles has `git` (the
|
|
||||||
Debian Go image has it; an alpine one needs `apk add --no-cache git`) and
|
|
||||||
takes the version from the `VERSION` build argument when one is given,
|
|
||||||
otherwise from `git describe --tags --always`. That gives the tag on a
|
|
||||||
tagged commit; on a later commit, the tag, the number of commits since it
|
|
||||||
and the short commit (`v1.2.3-4-gabc1234`); and the short commit when no
|
|
||||||
tag is reachable. The stage that compiles also marks its working directory
|
|
||||||
safe for git (`git config --system --add safe.directory /src`): a context
|
|
||||||
sent as a tar stream keeps the sender's file owners, and git refuses a
|
|
||||||
checkout owned by another user, so the version would come out empty.
|
|
||||||
`ARG VERSION` has no default, and the build fails if the context carries
|
|
||||||
`.git` and the version still comes out empty, `dev` or `unknown`. A plain
|
|
||||||
`docker build .` with no build arguments must succeed; a Dockerfile that
|
|
||||||
refuses an empty build argument drops that refusal and keeps the argument.
|
|
||||||
|
|
||||||
- Every repo should have a Gitea Actions workflow (`.gitea/workflows/`) that
|
- Every repo should have a Gitea Actions workflow (`.gitea/workflows/`) that
|
||||||
runs `script/cibuild` on push, and checks out the repo as its only other step.
|
runs `script/cibuild` (which runs `docker build .`) on push. Since the
|
||||||
That script bootstraps, runs the gate phases, and then builds the image, so a
|
Dockerfile already runs `make check`, a successful build implies all checks
|
||||||
successful run means every check passed; a bare `docker build .` does not
|
pass.
|
||||||
carry the same guarantee, because its gate phases may come from the cache. The
|
|
||||||
image build is uncached and so runs the gate phases a second time. That is the
|
|
||||||
price of the rule above, and it is worth paying: the image that ships is built
|
|
||||||
from a run of its own gates rather than from a cache entry. A separate
|
|
||||||
workflow limited to `main` by a `branches` list under `on: push` cannot be
|
|
||||||
checked by review: to try a change to it, add the feature branch to that list
|
|
||||||
and push, then remove the branch from the list again before merging. Keep any
|
|
||||||
job in it that publishes behind `if: github.ref_name == 'main'`, so the run
|
|
||||||
from the feature branch publishes nothing.
|
|
||||||
|
|
||||||
- Use platform-standard formatters: `black` for Python, `prettier` for
|
- Use platform-standard formatters: `black` for Python, `prettier` for
|
||||||
JS/CSS/Markdown/HTML, `go fmt` for Go. Always use default configuration with
|
JS/CSS/Markdown/HTML, `go fmt` for Go. Always use default configuration with
|
||||||
@@ -315,17 +193,15 @@ style conventions are in separate documents:
|
|||||||
suite that exceeds it fails. Under 20 seconds is the target. A suite between
|
suite that exceeds it fails. Under 20 seconds is the target. A suite between
|
||||||
20 and 60 seconds is still green, but the overage must be filed as an
|
20 and 60 seconds is still green, but the overage must be filed as an
|
||||||
improvement bug against that repo. Add a 90-second timeout to the test
|
improvement bug against that repo. Add a 90-second timeout to the test
|
||||||
invocation (`go test -timeout 90s`). The backstop deliberately sits above the
|
invocation in the Makefile (`go test -timeout 90s`). The backstop deliberately
|
||||||
hard cap so that it catches a genuinely hung test rather than a merely slow
|
sits above the hard cap so that it catches a genuinely hung test rather than a
|
||||||
one.
|
merely slow one.
|
||||||
|
|
||||||
- **The test command should use the conditional verbose rerun pattern.** Run
|
- **`make test` should use the conditional verbose rerun pattern.** Run tests
|
||||||
tests without `-v` (verbose) first. If tests fail, automatically rerun with
|
without `-v` (verbose) first. If tests fail, automatically rerun with `-v` to
|
||||||
`-v` to show full output. This keeps CI logs and `docker build` output clean
|
show full output. This keeps CI logs and `docker build` output clean on
|
||||||
on success (just package/suite summaries) while providing full diagnostic
|
success (just package/suite summaries) while providing full diagnostic detail
|
||||||
detail on failure (every test case, every assertion). The command lives in the
|
on failure (every test case, every assertion). The general shell pattern:
|
||||||
`test` phase of the `Dockerfile`, since `script/test` builds that phase; the
|
|
||||||
Makefile form below is the same pattern for any repo-local invocation:
|
|
||||||
|
|
||||||
```makefile
|
```makefile
|
||||||
test:
|
test:
|
||||||
@@ -338,26 +214,11 @@ style conventions are in separate documents:
|
|||||||
|
|
||||||
```makefile
|
```makefile
|
||||||
test:
|
test:
|
||||||
@go test -count=1 -timeout 90s -race -cover ./... || \
|
@go test -timeout 90s -race -cover ./... || \
|
||||||
{ echo "--- Rerunning with -v for details ---"; \
|
{ echo "--- Rerunning with -v for details ---"; \
|
||||||
go test -count=1 -timeout 90s -race -v ./...; exit 1; }
|
go test -timeout 90s -race -v ./...; exit 1; }
|
||||||
```
|
```
|
||||||
|
|
||||||
`-count=1` is required on both invocations: it defeats Go's test _result_
|
|
||||||
cache, so neither run can report a stored pass in place of running the
|
|
||||||
tests. It leaves the build cache alone, so it costs the runtime of the suite
|
|
||||||
and no recompilation.
|
|
||||||
|
|
||||||
That cache is Go's own, separate from Docker's layer cache. Go stores a
|
|
||||||
passing result in its cache directory (`GOCACHE`), and when the same tests
|
|
||||||
run again on unchanged code it prints that result, marked `(cached)`,
|
|
||||||
without running them. That matters on a developer's machine, where this
|
|
||||||
target runs and the directory lasts from one run to the next. The `test`
|
|
||||||
phase of the `Dockerfile` needs no `-count=1`: its base image holds no
|
|
||||||
result for this repo's tests and nothing before its `go test` step runs a
|
|
||||||
test, so there is nothing to replay. `--no-cache` (above) is what makes that
|
|
||||||
step run on an unchanged tree.
|
|
||||||
|
|
||||||
Python example:
|
Python example:
|
||||||
|
|
||||||
```makefile
|
```makefile
|
||||||
@@ -383,84 +244,10 @@ style conventions are in separate documents:
|
|||||||
must be in `.gitignore`. No exceptions.
|
must be in `.gitignore`. No exceptions.
|
||||||
|
|
||||||
- `.gitignore` should be comprehensive from the start: OS files (`.DS_Store`),
|
- `.gitignore` should be comprehensive from the start: OS files (`.DS_Store`),
|
||||||
editor files (`.swp`, `*~`), in-repo agent scratch directories (`.claude/`),
|
editor files (`.swp`, `*~`), language build artifacts, and `node_modules/`.
|
||||||
language build artifacts, and `node_modules/`. Fetch the standard `.gitignore`
|
Fetch the standard `.gitignore` from
|
||||||
from `https://git.eeqj.de/sneak/prompts/raw/branch/main/.gitignore` when
|
`https://git.eeqj.de/sneak/prompts/raw/branch/main/.gitignore` when setting up
|
||||||
setting up a new repo. These patterns are written to `.gitignore`'s own
|
a new repo.
|
||||||
semantics, in which an unanchored pattern already matches at every depth; they
|
|
||||||
are not a `.dockerignore` and must not be transplanted into one unmodified.
|
|
||||||
|
|
||||||
- **`.dockerignore` does not use `.gitignore` semantics, and copying patterns
|
|
||||||
across unmodified leaves secrets in the build context.** Docker matches with
|
|
||||||
`moby/patternmatcher`: `filepath.Match` semantics plus a `**` extension, so
|
|
||||||
`*` does not cross `/` and a pattern without a leading `**/` is anchored at
|
|
||||||
the build-context root. A `.dockerignore` listing `.env`, `*.pem` and `*.key`
|
|
||||||
therefore excludes only the copies at the repository root, while `config/.env`
|
|
||||||
and `certs/server.key` still reach the context and can land in an image layer
|
|
||||||
— which is more dangerous than a short file with no secret patterns at all,
|
|
||||||
because it reads as solved and stops anyone looking. Give every
|
|
||||||
depth-independent pattern the `**/` prefix and leave only genuinely
|
|
||||||
root-anchored entries unprefixed: `.claude`, and the repo's own host-built
|
|
||||||
binary, written `/myapp` and never `**/myapp`, which would also match
|
|
||||||
`cmd/myapp/` and delete the package directory from the context. Matching is
|
|
||||||
case-sensitive, and an ALL-CAPS twin per pattern still misses `Server.Key`, so
|
|
||||||
secret names use character ranges — `**/*.[kK][eE][yY]`, `**/*.[pP][eE][mM]`,
|
|
||||||
and likewise for `.envrc` and the extensionless SSH keys. Where such a pattern
|
|
||||||
also catches something the build needs, re-include it with a negation
|
|
||||||
(`!docs/example.env`); deleting the pattern reopens the exposure for every
|
|
||||||
other file it covers. Fetch the standard `.dockerignore` from
|
|
||||||
`https://git.eeqj.de/sneak/prompts/raw/branch/main/.dockerignore` and extend
|
|
||||||
it with the repo's own artifacts.
|
|
||||||
|
|
||||||
- **In-repo agent scratch belongs in both files, written to each file's own
|
|
||||||
semantics.** `.claude/` holds one worktree per in-flight agent — an entire
|
|
||||||
additional checkout of the repo — so under `COPY . .` the build context
|
|
||||||
inflates by a multiple of the repo and another session's unreviewed work can
|
|
||||||
be copied into an image layer. In `.gitignore` the entry is `.claude/`,
|
|
||||||
unanchored. In `.dockerignore` it is `.claude`, anchored and with **no** `**/`
|
|
||||||
prefix, because the prefixed form would also delete any nested directory of
|
|
||||||
that name from the build. Anchoring carries a known gap that the canonical
|
|
||||||
`.dockerignore` states in its own comment, since consuming repos receive the
|
|
||||||
file and not the tracker: the directory is created in the agent's working
|
|
||||||
directory, so a repo running agents in subdirectories still ships
|
|
||||||
`services/api/.claude/` and must add its own anchored entry there.
|
|
||||||
|
|
||||||
- **A plain `docker build .` of a clone stamps the version that
|
|
||||||
`git describe --tags --always` gives**, derived from the `.git` in the build
|
|
||||||
context as the canonical `Dockerfile` above shows. Without its failure check,
|
|
||||||
a missing `git` or an unreadable checkout would leave `-X main.Version=` empty
|
|
||||||
and the build would still exit 0. `script/docker` and `script/cibuild` pass
|
|
||||||
the version they compute on the host; it takes precedence. They do this
|
|
||||||
byte-identically across repos:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
# Own line: a failing command substitution inside an argument does not
|
|
||||||
# trip `set -e`, so the inline form degrades to an empty constant.
|
|
||||||
version="$(git describe --tags --always --dirty 2>/dev/null || true)"
|
|
||||||
[ -n "$version" ] || version="unknown"
|
|
||||||
docker build --no-cache \
|
|
||||||
--build-arg VERSION="$version" \
|
|
||||||
-t "$(script/projectname)" .
|
|
||||||
```
|
|
||||||
|
|
||||||
`--always` makes an untagged repo yield an abbreviated commit hash rather
|
|
||||||
than failing, and the `[ -n "$version" ]` line is the single place the
|
|
||||||
fallback is applied — a live check that fires on a build from an export with
|
|
||||||
no `.git` and on a repository with no commits yet. Do not fold it into the
|
|
||||||
substitution as `|| echo unknown`, which makes the guard unreachable. The
|
|
||||||
Dockerfile's side is `ARG VERSION` in the stage that compiles, declared
|
|
||||||
there because `ARG` is stage-scoped; passing `VERSION` to a repo whose
|
|
||||||
Dockerfile declares no such `ARG` is ignored and costs nothing, which is why
|
|
||||||
the scripts stay byte-identical. One consequence for CI: the standard
|
|
||||||
checkout action clones shallow and fetches no tags, so a repo that embeds a
|
|
||||||
tag-derived version must set `fetch-depth: 0` on its checkout step.
|
|
||||||
|
|
||||||
- **Verify `.dockerignore` by enumerating the image, not by reading the
|
|
||||||
patterns.** Plant files at the root _and_ at least two directories deep, build
|
|
||||||
a probe image that does `COPY . .`, and list what actually landed
|
|
||||||
(`docker run --rm --entrypoint find IMAGE /app`). The `transferring context`
|
|
||||||
size is not a substitute: a nested secret is a few bytes, and BuildKit
|
|
||||||
transfers only the delta from the previous build.
|
|
||||||
|
|
||||||
- **No build artifacts in version control.** Code-derived data (compiled
|
- **No build artifacts in version control.** Code-derived data (compiled
|
||||||
bundles, minified output, generated assets) must never be committed to the
|
bundles, minified output, generated assets) must never be committed to the
|
||||||
@@ -476,56 +263,12 @@ style conventions are in separate documents:
|
|||||||
- Make all changes on a feature branch. You can do whatever you want on a
|
- Make all changes on a feature branch. You can do whatever you want on a
|
||||||
feature branch.
|
feature branch.
|
||||||
|
|
||||||
- `.golangci.yml` is standardized. The vendored copy in a consuming repo must
|
- `.golangci.yml` is standardized and must _NEVER_ be modified by an agent, only
|
||||||
_NEVER_ be modified by an agent: fetch it from
|
manually by the user. Fetch from
|
||||||
`https://git.eeqj.de/sneak/prompts/raw/branch/main/.golangci.yml` and keep it
|
`https://git.eeqj.de/sneak/prompts/raw/branch/main/.golangci.yml`. The
|
||||||
byte-identical, so that no repo can quietly loosen its own linting. Linter
|
canonical golangci-lint version is v2.12.2 (released 2026-05-06), installed
|
||||||
configuration changes are made to the canonical copy in the `prompts` repo and
|
commit-pinned via
|
||||||
reach consuming repos by re-vendoring; an agent may open a PR against
|
`go install github.com/golangci/golangci-lint/v2/cmd/golangci-lint@c0d3ddc9cf3faa61a4e378e879ece580256d76e5`.
|
||||||
canonical, which only the user merges. One list is exempt from byte-identity,
|
|
||||||
because it cannot be written once for every repo: the `deny` list of the
|
|
||||||
`test-support` depguard rule, where a repo names its own test-support packages
|
|
||||||
by full import path. A repo adds entries there and changes nothing else, and a
|
|
||||||
re-vendor carries its entries forward. The canonical golangci-lint version is
|
|
||||||
v2.14.0 (released 2026-09-24), pinned as the digest of the lint phase's base
|
|
||||||
image
|
|
||||||
(`golangci/golangci-lint@sha256:ad862ba6b3798cbe0fd9fd7408d498fd74fbd2623a92406b2fd3898faf0bf98f`,
|
|
||||||
which reports `2.14.0 built with go1.27.0 from 114493f9`). A module's `go`
|
|
||||||
directive must not name a newer Go minor version than the one golangci-lint
|
|
||||||
was built with, or golangci-lint refuses to lint it: this release lints
|
|
||||||
`go 1.27.1` but not `go 1.28`. That digest is the only pin, since no repo
|
|
||||||
installs golangci-lint on the host. A repo sets the lint phase digest to the
|
|
||||||
one named here and re-vendors `.golangci.yml` in the same commit, whichever of
|
|
||||||
the two prompted the change: the canonical copy can name linters that an older
|
|
||||||
golangci-lint rejects, and a newer golangci-lint can add linters that
|
|
||||||
`default: all` switches on until the canonical copy disables them.
|
|
||||||
|
|
||||||
- **`script/bootstrap` installs a pinned tool by comparing versions, never by
|
|
||||||
testing presence.** An `if ! command -v <tool>; then install; fi` guard tests
|
|
||||||
`PATH` only, so on an already-provisioned machine the pin is inert and a
|
|
||||||
version bump is a silent no-op — while the Dockerfile, installing into a clean
|
|
||||||
image, gets the pinned version, so a local `make check` and `make docker` can
|
|
||||||
disagree about what the tool even is. The canonical form:
|
|
||||||
- compares the installed version against the pin over the **whole** version
|
|
||||||
token; a parser that stops at the first `-` reports `2.12.2` for a host
|
|
||||||
running `2.12.2-rc1` and skips the install;
|
|
||||||
- treats absent, non-zero, empty or unrecognised `--version` output as a
|
|
||||||
mismatch, so the failure direction is a redundant install and never a
|
|
||||||
skipped one;
|
|
||||||
- after installing, re-resolves the binary the way callers do — `hash -r`,
|
|
||||||
then through `PATH`, not through the directory the installer wrote to —
|
|
||||||
and fails naming the resolved path, since an install that a shadowing
|
|
||||||
binary hides succeeds while changing nothing any caller sees;
|
|
||||||
- is actually called, and prints the version on both success paths: a
|
|
||||||
function defined and never invoked has the same exit status and the same
|
|
||||||
empty output as one that worked.
|
|
||||||
|
|
||||||
Keep it POSIX sh: no arrays, no `[[`, no `grep -P`.
|
|
||||||
|
|
||||||
A Go tool a repo needs on the host is installed with `go install` pinned to
|
|
||||||
a commit hash (`go install <package>@<commit hash>`). It is never tracked as
|
|
||||||
a `go.mod` tool dependency or through a `tools.go` file, either of which
|
|
||||||
pulls the tool's own dependencies into the repo's `go.mod` and `go.sum`.
|
|
||||||
|
|
||||||
- When pinning images or packages by hash, add a comment above the reference
|
- When pinning images or packages by hash, add a comment above the reference
|
||||||
with the version and date (YYYY-MM-DD).
|
with the version and date (YYYY-MM-DD).
|
||||||
@@ -639,14 +382,12 @@ style conventions are in separate documents:
|
|||||||
settings.
|
settings.
|
||||||
|
|
||||||
- Avoid putting files in the repo root unless necessary. Root should contain
|
- Avoid putting files in the repo root unless necessary. Root should contain
|
||||||
only project-level config files (`README.md`, `AGENTS.md`, `Makefile`,
|
only project-level config files (`README.md`, `Makefile`, `Dockerfile`,
|
||||||
`Dockerfile`, `LICENSE`, `.gitignore`, `.editorconfig`, `REPO_POLICIES.md`,
|
`LICENSE`, `.gitignore`, `.editorconfig`, `REPO_POLICIES.md`, and
|
||||||
and language-specific config). Everything else goes in a subdirectory.
|
language-specific config). Everything else goes in a subdirectory. Canonical
|
||||||
Canonical subdirectory names:
|
subdirectory names:
|
||||||
- `bin/` — executable scripts and tools
|
- `bin/` — executable scripts and tools
|
||||||
- `cmd/` — Go command entrypoints; thin only: one `main.go` per binary whose
|
- `cmd/` — Go command entrypoints
|
||||||
body is a single call into `internal/` or `pkg/`, no project logic in
|
|
||||||
`cmd/`
|
|
||||||
- `configs/` — configuration templates and examples
|
- `configs/` — configuration templates and examples
|
||||||
- `deploy/` — deployment manifests (k8s, compose, terraform)
|
- `deploy/` — deployment manifests (k8s, compose, terraform)
|
||||||
- `docs/` — documentation and markdown (README.md stays in root)
|
- `docs/` — documentation and markdown (README.md stays in root)
|
||||||
@@ -673,7 +414,3 @@ style conventions are in separate documents:
|
|||||||
- Go: `go.mod`, `go.sum`, `.golangci.yml`
|
- Go: `go.mod`, `go.sum`, `.golangci.yml`
|
||||||
- JS: `package.json`, `yarn.lock`, `.prettierrc`, `.prettierignore`
|
- JS: `package.json`, `yarn.lock`, `.prettierrc`, `.prettierignore`
|
||||||
- Python: `pyproject.toml`
|
- Python: `pyproject.toml`
|
||||||
|
|
||||||
- Guidance for coding agents lives in one `AGENTS.md` at the repository root. It
|
|
||||||
is never committed under a file or directory named after one agent tool, such
|
|
||||||
as `CLAUDE.md` or `.claude/`, and never split into separate memory files.
|
|
||||||
|
|||||||
@@ -22,6 +22,7 @@ require (
|
|||||||
github.com/stretchr/testify v1.11.1
|
github.com/stretchr/testify v1.11.1
|
||||||
go.uber.org/fx v1.24.0
|
go.uber.org/fx v1.24.0
|
||||||
golang.org/x/crypto v0.38.0
|
golang.org/x/crypto v0.38.0
|
||||||
|
gopkg.in/yaml.v3 v3.0.1
|
||||||
gorm.io/driver/sqlite v1.5.4
|
gorm.io/driver/sqlite v1.5.4
|
||||||
gorm.io/gorm v1.25.5
|
gorm.io/gorm v1.25.5
|
||||||
modernc.org/sqlite v1.28.0
|
modernc.org/sqlite v1.28.0
|
||||||
@@ -58,7 +59,6 @@ require (
|
|||||||
golang.org/x/text v0.25.0 // indirect
|
golang.org/x/text v0.25.0 // indirect
|
||||||
golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d // indirect
|
golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d // indirect
|
||||||
google.golang.org/protobuf v1.31.0 // indirect
|
google.golang.org/protobuf v1.31.0 // indirect
|
||||||
gopkg.in/yaml.v3 v3.0.1 // indirect
|
|
||||||
lukechampine.com/uint128 v1.2.0 // indirect
|
lukechampine.com/uint128 v1.2.0 // indirect
|
||||||
modernc.org/cc/v3 v3.40.0 // indirect
|
modernc.org/cc/v3 v3.40.0 // indirect
|
||||||
modernc.org/ccgo/v3 v3.16.13 // indirect
|
modernc.org/ccgo/v3 v3.16.13 // indirect
|
||||||
|
|||||||
@@ -0,0 +1,387 @@
|
|||||||
|
package ciscript_test
|
||||||
|
|
||||||
|
import (
|
||||||
|
"maps"
|
||||||
|
"os"
|
||||||
|
"os/exec"
|
||||||
|
"path/filepath"
|
||||||
|
"slices"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/stretchr/testify/require"
|
||||||
|
"gopkg.in/yaml.v3"
|
||||||
|
)
|
||||||
|
|
||||||
|
const (
|
||||||
|
// supersededDesc is the description script/ci-mark-superseded
|
||||||
|
// writes, and the one an earlier revision of it wrote alongside a
|
||||||
|
// `skipped` state.
|
||||||
|
supersededDesc = "Superseded by a newer commit; never tested"
|
||||||
|
|
||||||
|
// liveContext is the commit-status context Gitea uses for this
|
||||||
|
// repository's runs, as seen in its API. The script derives it from
|
||||||
|
// the workflow and job names rather than hardcoding it; the
|
||||||
|
// derivation is checked against this value below.
|
||||||
|
liveContext = "check / check (push)"
|
||||||
|
|
||||||
|
scriptPath = "../../script/ci-mark-superseded"
|
||||||
|
workflow = "../../.gitea/workflows/check.yml"
|
||||||
|
|
||||||
|
// failure is the only state that neither folds into a combined
|
||||||
|
// `success` (as `skipped` does) nor blocks the commit forever (as
|
||||||
|
// `pending` does).
|
||||||
|
failure = "failure"
|
||||||
|
)
|
||||||
|
|
||||||
|
// repo is a throwaway git history: parent is the commit a run would be
|
||||||
|
// cancelled on, head the commit that superseded it.
|
||||||
|
type repo struct {
|
||||||
|
dir string
|
||||||
|
head string
|
||||||
|
parent string
|
||||||
|
}
|
||||||
|
|
||||||
|
// scriptEnv is the run identity the Gitea runner exports and the script
|
||||||
|
// builds its context string from.
|
||||||
|
type scriptEnv struct {
|
||||||
|
workflow string
|
||||||
|
job string
|
||||||
|
event string
|
||||||
|
}
|
||||||
|
|
||||||
|
func defaultEnv() scriptEnv {
|
||||||
|
return scriptEnv{workflow: "check", job: "check", event: "push"}
|
||||||
|
}
|
||||||
|
|
||||||
|
func cancelled() commitStatus {
|
||||||
|
return commitStatus{
|
||||||
|
Context: liveContext,
|
||||||
|
Status: failure,
|
||||||
|
Description: "Has been cancelled",
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func running() commitStatus {
|
||||||
|
return commitStatus{
|
||||||
|
Context: liveContext,
|
||||||
|
Status: "pending",
|
||||||
|
Description: "Has started running",
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestMarkSuperseded(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
cases := map[string]struct {
|
||||||
|
parent commitStatus
|
||||||
|
wantMark bool
|
||||||
|
}{
|
||||||
|
"a cancelled run is marked": {
|
||||||
|
parent: cancelled(),
|
||||||
|
wantMark: true,
|
||||||
|
},
|
||||||
|
"a laundered skipped status is marked": {
|
||||||
|
parent: commitStatus{
|
||||||
|
Context: liveContext,
|
||||||
|
Status: "skipped",
|
||||||
|
Description: supersededDesc,
|
||||||
|
},
|
||||||
|
wantMark: true,
|
||||||
|
},
|
||||||
|
"a genuine failure is left alone": {
|
||||||
|
parent: commitStatus{
|
||||||
|
Context: liveContext,
|
||||||
|
Status: failure,
|
||||||
|
Description: "Failing after 3m1s",
|
||||||
|
},
|
||||||
|
wantMark: false,
|
||||||
|
},
|
||||||
|
"a passing run is left alone": {
|
||||||
|
parent: commitStatus{
|
||||||
|
Context: liveContext,
|
||||||
|
Status: "success",
|
||||||
|
Description: "Successful in 2m52s",
|
||||||
|
},
|
||||||
|
wantMark: false,
|
||||||
|
},
|
||||||
|
"another context is left alone": {
|
||||||
|
parent: commitStatus{
|
||||||
|
Context: "other / other (push)",
|
||||||
|
Status: failure,
|
||||||
|
Description: "Has been cancelled",
|
||||||
|
},
|
||||||
|
wantMark: false,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
for name, tc := range cases {
|
||||||
|
t.Run(name, func(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
requireTools(t)
|
||||||
|
|
||||||
|
history := newRepo(t)
|
||||||
|
fake, api := newFakeGitea(t)
|
||||||
|
fake.setStatus(history.head, running())
|
||||||
|
fake.setStatus(history.parent, tc.parent)
|
||||||
|
|
||||||
|
out, err := runScript(t, history, api, defaultEnv())
|
||||||
|
require.NoError(t, err, out)
|
||||||
|
|
||||||
|
posted := fake.postedFor(history.parent)
|
||||||
|
if !tc.wantMark {
|
||||||
|
require.Empty(t, posted)
|
||||||
|
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
require.Equal(t, []postedStatus{{
|
||||||
|
Context: liveContext,
|
||||||
|
// Not `skipped`: Gitea's combined status folds
|
||||||
|
// that into `success`, which is what made a
|
||||||
|
// never-tested commit read green.
|
||||||
|
State: failure,
|
||||||
|
Description: supersededDesc,
|
||||||
|
}}, posted)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// A second run must not rewrite what the first one wrote, or every
|
||||||
|
// later push would post a duplicate status.
|
||||||
|
func TestMarkSupersededIsIdempotent(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
requireTools(t)
|
||||||
|
|
||||||
|
history := newRepo(t)
|
||||||
|
fake, api := newFakeGitea(t)
|
||||||
|
fake.setStatus(history.head, running())
|
||||||
|
fake.setStatus(history.parent, cancelled())
|
||||||
|
|
||||||
|
for range 2 {
|
||||||
|
out, err := runScript(t, history, api, defaultEnv())
|
||||||
|
require.NoError(t, err, out)
|
||||||
|
}
|
||||||
|
|
||||||
|
require.Len(t, fake.postedFor(history.parent), 1)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Renaming the workflow or the job changes the context string Gitea
|
||||||
|
// uses. The script must say so instead of quietly matching nothing.
|
||||||
|
func TestMarkSupersededRejectsAnUnknownContext(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
requireTools(t)
|
||||||
|
|
||||||
|
history := newRepo(t)
|
||||||
|
fake, api := newFakeGitea(t)
|
||||||
|
fake.setStatus(history.head, running())
|
||||||
|
fake.setStatus(history.parent, cancelled())
|
||||||
|
|
||||||
|
env := defaultEnv()
|
||||||
|
env.job = "renamed"
|
||||||
|
|
||||||
|
out, err := runScript(t, history, api, env)
|
||||||
|
require.Error(t, err)
|
||||||
|
require.Contains(t, out, "renamed")
|
||||||
|
require.Contains(t, out, liveContext)
|
||||||
|
require.Empty(t, fake.postedFor(history.parent))
|
||||||
|
}
|
||||||
|
|
||||||
|
// ANCESTOR_LIMIT is a documented knob. A value that is set but unusable
|
||||||
|
// must abort: handing it to git and discarding the exit status left the
|
||||||
|
// walk empty and the step green, marking nothing.
|
||||||
|
func TestMarkSupersededRejectsAnUnparseableAncestorLimit(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
requireTools(t)
|
||||||
|
|
||||||
|
history := newRepo(t)
|
||||||
|
fake, api := newFakeGitea(t)
|
||||||
|
fake.setStatus(history.head, running())
|
||||||
|
fake.setStatus(history.parent, cancelled())
|
||||||
|
|
||||||
|
out, err := runScript(
|
||||||
|
t, history, api, defaultEnv(), "ANCESTOR_LIMIT=twenty",
|
||||||
|
)
|
||||||
|
require.Error(t, err)
|
||||||
|
require.Contains(t, out, "ANCESTOR_LIMIT")
|
||||||
|
require.Contains(t, out, "twenty")
|
||||||
|
require.Empty(t, fake.postedFor(history.parent))
|
||||||
|
}
|
||||||
|
|
||||||
|
// A status read that fails is not the same as a commit with nothing to
|
||||||
|
// do. Losing curl's exit status through a pipe made the two identical
|
||||||
|
// and left a laundered commit laundered with no signal.
|
||||||
|
func TestMarkSupersededFailsOnAnUnreadableAncestorStatus(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
requireTools(t)
|
||||||
|
|
||||||
|
history := newRepo(t)
|
||||||
|
fake, api := newFakeGitea(t)
|
||||||
|
fake.setStatus(history.head, running())
|
||||||
|
fake.setStatus(history.parent, cancelled())
|
||||||
|
fake.failStatusRead(history.parent)
|
||||||
|
|
||||||
|
out, err := runScript(t, history, api, defaultEnv())
|
||||||
|
require.Error(t, err)
|
||||||
|
require.Contains(t, out, history.parent)
|
||||||
|
require.Contains(t, out, "cannot read commit statuses")
|
||||||
|
require.Empty(t, fake.postedFor(history.parent))
|
||||||
|
}
|
||||||
|
|
||||||
|
// A shallow clone cannot resolve the parent, so it is indistinguishable
|
||||||
|
// from a root commit to rev-parse and the walk would exit 0 having
|
||||||
|
// marked nothing. It must abort instead: dropping `fetch-depth: 0` from
|
||||||
|
// the checkout step is one edit, and a silent no-op there restores the
|
||||||
|
// false-green bug this script exists to prevent.
|
||||||
|
func TestMarkSupersededRejectsAShallowRepository(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
requireTools(t)
|
||||||
|
|
||||||
|
history := shallowClone(t, newRepo(t))
|
||||||
|
fake, api := newFakeGitea(t)
|
||||||
|
fake.setStatus(history.head, running())
|
||||||
|
fake.setStatus(history.parent, cancelled())
|
||||||
|
|
||||||
|
out, err := runScript(t, history, api, defaultEnv())
|
||||||
|
require.Error(t, err)
|
||||||
|
require.Contains(t, out, "shallow repository")
|
||||||
|
require.Empty(t, fake.postedFor(history.parent))
|
||||||
|
require.Empty(t, fake.postedFor(history.head))
|
||||||
|
}
|
||||||
|
|
||||||
|
// shallowClone returns the same history as a depth-1 clone. The `file://`
|
||||||
|
// URL is required: git ignores --depth for a plain local path.
|
||||||
|
func shallowClone(t *testing.T, history repo) repo {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
dir := t.TempDir()
|
||||||
|
|
||||||
|
//nolint:gosec // fixed argv, arguments are test-local paths
|
||||||
|
cmd := exec.CommandContext(t.Context(), "git", "clone", "-q",
|
||||||
|
"--depth=1", "file://"+history.dir, dir)
|
||||||
|
|
||||||
|
out, err := cmd.CombinedOutput()
|
||||||
|
require.NoError(t, err, string(out))
|
||||||
|
|
||||||
|
return repo{dir: dir, head: history.head, parent: history.parent}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The derived context must equal the one Gitea actually uses, which is
|
||||||
|
// built from the same workflow and job names.
|
||||||
|
func TestDerivedContextMatchesGitea(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
requireTools(t)
|
||||||
|
|
||||||
|
name, job := workflowIdentity(t)
|
||||||
|
|
||||||
|
history := newRepo(t)
|
||||||
|
fake, api := newFakeGitea(t)
|
||||||
|
fake.setStatus(history.head, running())
|
||||||
|
fake.setStatus(history.parent, cancelled())
|
||||||
|
|
||||||
|
out, err := runScript(t, history, api, scriptEnv{
|
||||||
|
workflow: name,
|
||||||
|
job: job,
|
||||||
|
event: "push",
|
||||||
|
})
|
||||||
|
require.NoError(t, err, out)
|
||||||
|
|
||||||
|
posted := fake.postedFor(history.parent)
|
||||||
|
require.Len(t, posted, 1)
|
||||||
|
require.Equal(t, liveContext, posted[0].Context)
|
||||||
|
}
|
||||||
|
|
||||||
|
// workflowIdentity reads the workflow name and its single job id out of
|
||||||
|
// the checked-in workflow file.
|
||||||
|
func workflowIdentity(t *testing.T) (string, string) {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
raw, err := os.ReadFile(workflow)
|
||||||
|
require.NoError(t, err)
|
||||||
|
|
||||||
|
var parsed struct {
|
||||||
|
Name string `yaml:"name"`
|
||||||
|
Jobs map[string]any `yaml:"jobs"`
|
||||||
|
}
|
||||||
|
|
||||||
|
require.NoError(t, yaml.Unmarshal(raw, &parsed))
|
||||||
|
|
||||||
|
jobs := slices.Collect(maps.Keys(parsed.Jobs))
|
||||||
|
require.Len(t, jobs, 1)
|
||||||
|
|
||||||
|
return parsed.Name, jobs[0]
|
||||||
|
}
|
||||||
|
|
||||||
|
func runScript(
|
||||||
|
t *testing.T, history repo, api string, env scriptEnv,
|
||||||
|
extra ...string,
|
||||||
|
) (string, error) {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
script, err := filepath.Abs(scriptPath)
|
||||||
|
require.NoError(t, err)
|
||||||
|
|
||||||
|
//nolint:gosec // fixed argv, repo-local script under test
|
||||||
|
cmd := exec.CommandContext(t.Context(), "sh", script)
|
||||||
|
cmd.Dir = history.dir
|
||||||
|
cmd.Env = append(os.Environ(),
|
||||||
|
"GITHUB_API_URL="+api,
|
||||||
|
"GITHUB_REPOSITORY=sneak/webhooker",
|
||||||
|
"GITHUB_SHA="+history.head,
|
||||||
|
"GITHUB_WORKFLOW="+env.workflow,
|
||||||
|
"GITHUB_JOB="+env.job,
|
||||||
|
"GITHUB_EVENT_NAME="+env.event,
|
||||||
|
"GITEA_TOKEN=test-token",
|
||||||
|
)
|
||||||
|
cmd.Env = append(cmd.Env, extra...)
|
||||||
|
|
||||||
|
out, err := cmd.CombinedOutput()
|
||||||
|
|
||||||
|
return string(out), err
|
||||||
|
}
|
||||||
|
|
||||||
|
func newRepo(t *testing.T) repo {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
dir := t.TempDir()
|
||||||
|
|
||||||
|
git := func(args ...string) string {
|
||||||
|
//nolint:gosec // fixed argv, arguments are test constants
|
||||||
|
cmd := exec.CommandContext(t.Context(), "git", args...)
|
||||||
|
cmd.Dir = dir
|
||||||
|
|
||||||
|
out, err := cmd.CombinedOutput()
|
||||||
|
require.NoError(t, err, string(out))
|
||||||
|
|
||||||
|
return strings.TrimSpace(string(out))
|
||||||
|
}
|
||||||
|
|
||||||
|
commit := func(message string) string {
|
||||||
|
git(
|
||||||
|
"-c", "user.email=ci@example.invalid",
|
||||||
|
"-c", "user.name=ci",
|
||||||
|
"-c", "commit.gpgsign=false",
|
||||||
|
"commit", "-q", "--allow-empty", "-m", message,
|
||||||
|
)
|
||||||
|
|
||||||
|
return git("rev-parse", "HEAD")
|
||||||
|
}
|
||||||
|
|
||||||
|
git("init", "-q", "-b", "main")
|
||||||
|
|
||||||
|
parent := commit("parent")
|
||||||
|
head := commit("head")
|
||||||
|
|
||||||
|
return repo{dir: dir, head: head, parent: parent}
|
||||||
|
}
|
||||||
|
|
||||||
|
func requireTools(t *testing.T) {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
for _, tool := range []string{"sh", "git", "curl", "jq"} {
|
||||||
|
_, err := exec.LookPath(tool)
|
||||||
|
if err != nil {
|
||||||
|
t.Skipf("%s is not installed: %v", tool, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,10 @@
|
|||||||
|
// Package ciscript holds the tests for the repository's CI shell
|
||||||
|
// scripts in script/. It carries no runtime code: the scripts run on
|
||||||
|
// the CI runner, not inside the binary, but their behaviour still has
|
||||||
|
// to be verified by the test suite.
|
||||||
|
//
|
||||||
|
// The scripts under test are outside the Go build graph, so `go test`'s
|
||||||
|
// result cache serves a stale PASS when only a script changed: run the
|
||||||
|
// container build, or GOFLAGS=-count=1, to trust a result here after
|
||||||
|
// editing script/.
|
||||||
|
package ciscript
|
||||||
@@ -0,0 +1,162 @@
|
|||||||
|
package ciscript_test
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/json"
|
||||||
|
"net/http"
|
||||||
|
"net/http/httptest"
|
||||||
|
"sync"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
// commitStatus is the part of an entry in Gitea's combined-status
|
||||||
|
// response that script/ci-mark-superseded reads.
|
||||||
|
type commitStatus struct {
|
||||||
|
Context string `json:"context"`
|
||||||
|
Status string `json:"status"`
|
||||||
|
Description string `json:"description"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// postedStatus is the part of a create-status request body the script
|
||||||
|
// writes.
|
||||||
|
type postedStatus struct {
|
||||||
|
Context string `json:"context"`
|
||||||
|
State string `json:"state"`
|
||||||
|
Description string `json:"description"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// fakeGitea serves the two endpoints the script talks to. Like Gitea,
|
||||||
|
// the newest status for a context replaces the previous one, so a
|
||||||
|
// second run of the script sees what the first one wrote.
|
||||||
|
type fakeGitea struct {
|
||||||
|
mu sync.Mutex
|
||||||
|
statuses map[string][]commitStatus
|
||||||
|
posted map[string][]postedStatus
|
||||||
|
// failRead is a commit whose combined-status read answers HTTP
|
||||||
|
// 500, standing in for a status API that is down.
|
||||||
|
failRead string
|
||||||
|
}
|
||||||
|
|
||||||
|
// newFakeGitea returns the fake and the base URL to hand the script as
|
||||||
|
// GITHUB_API_URL.
|
||||||
|
func newFakeGitea(t *testing.T) (*fakeGitea, string) {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
fake := &fakeGitea{
|
||||||
|
mu: sync.Mutex{},
|
||||||
|
statuses: map[string][]commitStatus{},
|
||||||
|
posted: map[string][]postedStatus{},
|
||||||
|
failRead: "",
|
||||||
|
}
|
||||||
|
|
||||||
|
srv := httptest.NewServer(fake.routes())
|
||||||
|
t.Cleanup(srv.Close)
|
||||||
|
|
||||||
|
return fake, srv.URL
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeGitea) routes() http.Handler {
|
||||||
|
mux := http.NewServeMux()
|
||||||
|
|
||||||
|
mux.HandleFunc(
|
||||||
|
"GET /repos/{owner}/{repo}/commits/{sha}/status",
|
||||||
|
f.handleCombined,
|
||||||
|
)
|
||||||
|
mux.HandleFunc(
|
||||||
|
"POST /repos/{owner}/{repo}/statuses/{sha}",
|
||||||
|
f.handleCreate,
|
||||||
|
)
|
||||||
|
|
||||||
|
return mux
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeGitea) handleCombined(
|
||||||
|
w http.ResponseWriter, r *http.Request,
|
||||||
|
) {
|
||||||
|
f.mu.Lock()
|
||||||
|
defer f.mu.Unlock()
|
||||||
|
|
||||||
|
sha := r.PathValue("sha")
|
||||||
|
if f.failRead != "" && f.failRead == sha {
|
||||||
|
http.Error(w, "boom", http.StatusInternalServerError)
|
||||||
|
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
body := struct {
|
||||||
|
Statuses []commitStatus `json:"statuses"`
|
||||||
|
}{Statuses: f.statuses[sha]}
|
||||||
|
|
||||||
|
payload, err := json.Marshal(body)
|
||||||
|
if err != nil {
|
||||||
|
http.Error(w, err.Error(), http.StatusInternalServerError)
|
||||||
|
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
w.Header().Set("Content-Type", "application/json")
|
||||||
|
|
||||||
|
_, _ = w.Write(payload)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeGitea) handleCreate(w http.ResponseWriter, r *http.Request) {
|
||||||
|
var got postedStatus
|
||||||
|
|
||||||
|
err := json.NewDecoder(r.Body).Decode(&got)
|
||||||
|
if err != nil {
|
||||||
|
http.Error(w, err.Error(), http.StatusBadRequest)
|
||||||
|
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
sha := r.PathValue("sha")
|
||||||
|
|
||||||
|
f.mu.Lock()
|
||||||
|
defer f.mu.Unlock()
|
||||||
|
|
||||||
|
f.posted[sha] = append(f.posted[sha], got)
|
||||||
|
f.replaceLocked(sha, commitStatus{
|
||||||
|
Context: got.Context,
|
||||||
|
Status: got.State,
|
||||||
|
Description: got.Description,
|
||||||
|
})
|
||||||
|
|
||||||
|
w.WriteHeader(http.StatusCreated)
|
||||||
|
}
|
||||||
|
|
||||||
|
// failStatusRead makes the combined-status read for one commit answer
|
||||||
|
// HTTP 500.
|
||||||
|
func (f *fakeGitea) failStatusRead(sha string) {
|
||||||
|
f.mu.Lock()
|
||||||
|
defer f.mu.Unlock()
|
||||||
|
|
||||||
|
f.failRead = sha
|
||||||
|
}
|
||||||
|
|
||||||
|
// setStatus gives a commit its latest status for a context.
|
||||||
|
func (f *fakeGitea) setStatus(sha string, status commitStatus) {
|
||||||
|
f.mu.Lock()
|
||||||
|
defer f.mu.Unlock()
|
||||||
|
|
||||||
|
f.replaceLocked(sha, status)
|
||||||
|
}
|
||||||
|
|
||||||
|
// postedFor returns the statuses the script created for a commit.
|
||||||
|
func (f *fakeGitea) postedFor(sha string) []postedStatus {
|
||||||
|
f.mu.Lock()
|
||||||
|
defer f.mu.Unlock()
|
||||||
|
|
||||||
|
return append([]postedStatus(nil), f.posted[sha]...)
|
||||||
|
}
|
||||||
|
|
||||||
|
// replaceLocked requires f.mu.
|
||||||
|
func (f *fakeGitea) replaceLocked(sha string, status commitStatus) {
|
||||||
|
for i, existing := range f.statuses[sha] {
|
||||||
|
if existing.Context == status.Context {
|
||||||
|
f.statuses[sha][i] = status
|
||||||
|
|
||||||
|
return
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
f.statuses[sha] = append(f.statuses[sha], status)
|
||||||
|
}
|
||||||
@@ -30,10 +30,8 @@ type Event struct {
|
|||||||
WebhookID string `gorm:"type:uuid;not null" json:"webhookId"`
|
WebhookID string `gorm:"type:uuid;not null" json:"webhookId"`
|
||||||
EntrypointID string `gorm:"type:uuid;not null;index:idx_events_entrypoint_id,priority:1" json:"entrypointId"`
|
EntrypointID string `gorm:"type:uuid;not null;index:idx_events_entrypoint_id,priority:1" json:"entrypointId"`
|
||||||
|
|
||||||
// Request data. RawQuery is the receiving request's query string
|
// Request data
|
||||||
// as sent, without the leading "?".
|
|
||||||
Method string `gorm:"not null" json:"method"`
|
Method string `gorm:"not null" json:"method"`
|
||||||
RawQuery string `gorm:"type:text" json:"rawQuery"`
|
|
||||||
Headers string `gorm:"type:text" json:"headers"` // JSON
|
Headers string `gorm:"type:text" json:"headers"` // JSON
|
||||||
Body string `gorm:"type:text" json:"body"`
|
Body string `gorm:"type:text" json:"body"`
|
||||||
ContentType string `json:"contentType"`
|
ContentType string `json:"contentType"`
|
||||||
|
|||||||
@@ -110,7 +110,6 @@ type Task struct {
|
|||||||
MaxRetries int
|
MaxRetries int
|
||||||
|
|
||||||
Method string
|
Method string
|
||||||
RawQuery string
|
|
||||||
Headers string
|
Headers string
|
||||||
ContentType string
|
ContentType string
|
||||||
Body *string
|
Body *string
|
||||||
@@ -1753,7 +1752,6 @@ func buildEventFromTask(task *Task) database.Event {
|
|||||||
event := database.Event{
|
event := database.Event{
|
||||||
EntrypointID: task.EntrypointID,
|
EntrypointID: task.EntrypointID,
|
||||||
Method: task.Method,
|
Method: task.Method,
|
||||||
RawQuery: task.RawQuery,
|
|
||||||
Headers: task.Headers,
|
Headers: task.Headers,
|
||||||
ContentType: task.ContentType,
|
ContentType: task.ContentType,
|
||||||
}
|
}
|
||||||
@@ -2104,7 +2102,6 @@ func buildRecoveryTask(
|
|||||||
TargetConfig: target.Config,
|
TargetConfig: target.Config,
|
||||||
MaxRetries: target.MaxRetries,
|
MaxRetries: target.MaxRetries,
|
||||||
Method: event.Method,
|
Method: event.Method,
|
||||||
RawQuery: event.RawQuery,
|
|
||||||
Headers: event.Headers,
|
Headers: event.Headers,
|
||||||
ContentType: event.ContentType,
|
ContentType: event.ContentType,
|
||||||
Body: bodyPtr,
|
Body: bodyPtr,
|
||||||
|
|||||||
@@ -673,11 +673,6 @@ func TestRecoverPendingDeliveries(t *testing.T) {
|
|||||||
t, s.WebhookDB, s.WebhookID, targetID, 3,
|
t, s.WebhookDB, s.WebhookID, targetID, 3,
|
||||||
)
|
)
|
||||||
|
|
||||||
// A recovered delivery still carries its event's query string.
|
|
||||||
require.NoError(t, s.WebhookDB.Model(&database.Event{}).
|
|
||||||
Where("webhook_id = ?", s.WebhookID).
|
|
||||||
Update("raw_query", eventQuery).Error)
|
|
||||||
|
|
||||||
s.Engine.ExportRecoverPendingDeliveries(
|
s.Engine.ExportRecoverPendingDeliveries(
|
||||||
context.Background(), s.WebhookDB,
|
context.Background(), s.WebhookDB,
|
||||||
s.WebhookID,
|
s.WebhookID,
|
||||||
@@ -692,8 +687,6 @@ func TestRecoverPendingDeliveries(t *testing.T) {
|
|||||||
database.TargetTypeLog,
|
database.TargetTypeLog,
|
||||||
task.TargetType,
|
task.TargetType,
|
||||||
)
|
)
|
||||||
|
|
||||||
assert.Equal(t, eventQuery, task.RawQuery)
|
|
||||||
case <-time.After(2 * time.Second):
|
case <-time.After(2 * time.Second):
|
||||||
t.Fatalf("expected task %d", i)
|
t.Fatalf("expected task %d", i)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -11,7 +11,6 @@ import (
|
|||||||
"net/http/httptest"
|
"net/http/httptest"
|
||||||
"os"
|
"os"
|
||||||
"path/filepath"
|
"path/filepath"
|
||||||
"strconv"
|
|
||||||
"strings"
|
"strings"
|
||||||
"sync"
|
"sync"
|
||||||
"sync/atomic"
|
"sync/atomic"
|
||||||
@@ -1991,10 +1990,6 @@ func assertLogLineComplete(
|
|||||||
"log line must contain the full request headers",
|
"log line must contain the full request headers",
|
||||||
)
|
)
|
||||||
|
|
||||||
assert.Contains(t, out, "raw_query="+strconv.Quote(event.RawQuery),
|
|
||||||
"log line must contain the query string",
|
|
||||||
)
|
|
||||||
|
|
||||||
assert.Contains(t, out, event.EntrypointID,
|
assert.Contains(t, out, event.EntrypointID,
|
||||||
"log line must contain the entrypoint id",
|
"log line must contain the entrypoint id",
|
||||||
)
|
)
|
||||||
@@ -2017,7 +2012,6 @@ func TestDeliverLog_LogsFullContent(t *testing.T) {
|
|||||||
event := seedEvent(
|
event := seedEvent(
|
||||||
t, db, `{"log-body-marker":"abc123"}`,
|
t, db, `{"log-body-marker":"abc123"}`,
|
||||||
)
|
)
|
||||||
event.RawQuery = eventQuery
|
|
||||||
|
|
||||||
dlv := seedDelivery(
|
dlv := seedDelivery(
|
||||||
t, db, event.ID, uuid.New().String(),
|
t, db, event.ID, uuid.New().String(),
|
||||||
|
|||||||
@@ -48,9 +48,8 @@ func fSweepSetup(
|
|||||||
//
|
//
|
||||||
// Every caller drives the dispatch paths synchronously and has already
|
// Every caller drives the dispatch paths synchronously and has already
|
||||||
// waited for them to return, so anything they queued is in the channel
|
// waited for them to return, so anything they queued is in the channel
|
||||||
// by now, and nothing is waited for. A timer here would race the queued
|
// by now. The short grace covers nothing but scheduler jitter, and is
|
||||||
// tasks: on a busy host it can be due by the time select looks, and
|
// kept small because one of these tests runs the drain forty times.
|
||||||
// select picks at random among the cases that are ready.
|
|
||||||
func fDrain(e *delivery.Engine) []delivery.Task {
|
func fDrain(e *delivery.Engine) []delivery.Task {
|
||||||
var out []delivery.Task
|
var out []delivery.Task
|
||||||
|
|
||||||
@@ -60,7 +59,7 @@ func fDrain(e *delivery.Engine) []delivery.Task {
|
|||||||
out = append(out, task)
|
out = append(out, task)
|
||||||
case task := <-e.ExportRetryCh():
|
case task := <-e.ExportRetryCh():
|
||||||
out = append(out, task)
|
out = append(out, task)
|
||||||
default:
|
case <-time.After(25 * time.Millisecond):
|
||||||
return out
|
return out
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -39,9 +39,6 @@ type TargetConfigForm struct {
|
|||||||
// Timeout is the HTTP target's per-request timeout in seconds,
|
// Timeout is the HTTP target's per-request timeout in seconds,
|
||||||
// empty when unset.
|
// empty when unset.
|
||||||
Timeout string
|
Timeout string
|
||||||
// ForwardQuery is the HTTP target's setting that passes each
|
|
||||||
// event's query string on to it.
|
|
||||||
ForwardQuery bool
|
|
||||||
// Expiry is the database (archive) target's row expiry.
|
// Expiry is the database (archive) target's row expiry.
|
||||||
Expiry string
|
Expiry string
|
||||||
// Rotation is the database (archive) target's rotation.
|
// Rotation is the database (archive) target's rotation.
|
||||||
@@ -70,7 +67,6 @@ func NewTargetConfigForm(
|
|||||||
URL: cfg.URL,
|
URL: cfg.URL,
|
||||||
Headers: FormatTargetHeaders(cfg.Headers),
|
Headers: FormatTargetHeaders(cfg.Headers),
|
||||||
Timeout: FormatTargetTimeout(cfg.Timeout),
|
Timeout: FormatTargetTimeout(cfg.Timeout),
|
||||||
ForwardQuery: cfg.ForwardQuery,
|
|
||||||
}, nil
|
}, nil
|
||||||
case database.TargetTypeSlack:
|
case database.TargetTypeSlack:
|
||||||
cfg, err := parseSlackConfig(t.Config)
|
cfg, err := parseSlackConfig(t.Config)
|
||||||
|
|||||||
@@ -171,13 +171,6 @@ func httpConfigFields(t *database.Target) []ConfigField {
|
|||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
if cfg.ForwardQuery {
|
|
||||||
fields = append(fields, ConfigField{
|
|
||||||
Label: "Query string",
|
|
||||||
Value: "passed on to this target",
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
fields = append(fields, maxRetriesField(t))
|
fields = append(fields, maxRetriesField(t))
|
||||||
|
|
||||||
return fields
|
return fields
|
||||||
|
|||||||
@@ -223,8 +223,7 @@ func TestNewTargetViews_HTTP(t *testing.T) {
|
|||||||
Type: database.TargetTypeHTTP,
|
Type: database.TargetTypeHTTP,
|
||||||
Config: `{"url":"` + viewExampleHook + `",` +
|
Config: `{"url":"` + viewExampleHook + `",` +
|
||||||
`"timeout":30,` +
|
`"timeout":30,` +
|
||||||
`"headers":{"Authorization":"Bearer sekrit"},` +
|
`"headers":{"Authorization":"Bearer sekrit"}}`,
|
||||||
`"forwardQuery":true}`,
|
|
||||||
MaxRetries: 5,
|
MaxRetries: 5,
|
||||||
})
|
})
|
||||||
|
|
||||||
@@ -236,7 +235,6 @@ func TestNewTargetViews_HTTP(t *testing.T) {
|
|||||||
"Destination URL": viewMaskedOrigin,
|
"Destination URL": viewMaskedOrigin,
|
||||||
"Timeout": "30s",
|
"Timeout": "30s",
|
||||||
"Headers": "1 configured",
|
"Headers": "1 configured",
|
||||||
"Query string": "passed on to this target",
|
|
||||||
viewMaxRetries: "5",
|
viewMaxRetries: "5",
|
||||||
},
|
},
|
||||||
fields,
|
fields,
|
||||||
|
|||||||
@@ -184,7 +184,6 @@ func (t *databaseTarget) archive(d *database.Delivery) error {
|
|||||||
WebhookID: webhookID,
|
WebhookID: webhookID,
|
||||||
EntrypointID: d.Event.EntrypointID,
|
EntrypointID: d.Event.EntrypointID,
|
||||||
Method: d.Event.Method,
|
Method: d.Event.Method,
|
||||||
RawQuery: d.Event.RawQuery,
|
|
||||||
Headers: d.Event.Headers,
|
Headers: d.Event.Headers,
|
||||||
Body: d.Event.Body,
|
Body: d.Event.Body,
|
||||||
ContentType: d.Event.ContentType,
|
ContentType: d.Event.ContentType,
|
||||||
|
|||||||
@@ -101,7 +101,6 @@ type archivedEvent struct {
|
|||||||
WebhookID string
|
WebhookID string
|
||||||
EntrypointID string
|
EntrypointID string
|
||||||
Method string
|
Method string
|
||||||
RawQuery string
|
|
||||||
Headers string
|
Headers string
|
||||||
Body string
|
Body string
|
||||||
ContentType string
|
ContentType string
|
||||||
|
|||||||
@@ -360,7 +360,6 @@ func writeRow(w io.Writer, ev *archivedEvent, period string) error {
|
|||||||
"webhook_id": ev.WebhookID,
|
"webhook_id": ev.WebhookID,
|
||||||
"entrypoint_id": ev.EntrypointID,
|
"entrypoint_id": ev.EntrypointID,
|
||||||
"method": ev.Method,
|
"method": ev.Method,
|
||||||
"raw_query": ev.RawQuery,
|
|
||||||
"headers": ev.Headers,
|
"headers": ev.Headers,
|
||||||
"body": ev.Body,
|
"body": ev.Body,
|
||||||
"content_type": ev.ContentType,
|
"content_type": ev.ContentType,
|
||||||
|
|||||||
@@ -166,7 +166,6 @@ func TestArchiveExport_MatchesStoredRows(t *testing.T) {
|
|||||||
WebhookID: exportWebhookID,
|
WebhookID: exportWebhookID,
|
||||||
EntrypointID: "ep-1",
|
EntrypointID: "ep-1",
|
||||||
Method: "POST",
|
Method: "POST",
|
||||||
RawQuery: eventQuery,
|
|
||||||
Headers: `{"X-Test":["yes"]}`,
|
Headers: `{"X-Test":["yes"]}`,
|
||||||
Body: body,
|
Body: body,
|
||||||
ContentType: testContentType,
|
ContentType: testContentType,
|
||||||
@@ -216,13 +215,12 @@ func assertExportedRow(
|
|||||||
assert.Equal(t, row.WebhookID, ev["webhook_id"])
|
assert.Equal(t, row.WebhookID, ev["webhook_id"])
|
||||||
assert.Equal(t, row.EntrypointID, ev["entrypoint_id"])
|
assert.Equal(t, row.EntrypointID, ev["entrypoint_id"])
|
||||||
assert.Equal(t, row.Method, ev["method"])
|
assert.Equal(t, row.Method, ev["method"])
|
||||||
assert.Equal(t, row.RawQuery, ev["raw_query"])
|
|
||||||
assert.Equal(t, row.Headers, ev["headers"])
|
assert.Equal(t, row.Headers, ev["headers"])
|
||||||
assert.Equal(t, row.ContentType, ev["content_type"])
|
assert.Equal(t, row.ContentType, ev["content_type"])
|
||||||
|
|
||||||
if row.Body != binaryBody {
|
if row.Body != binaryBody {
|
||||||
assert.Equal(t, row.Body, ev["body"])
|
assert.Equal(t, row.Body, ev["body"])
|
||||||
assert.Len(t, ev, 10, "the ten columns and nothing else: %v", ev)
|
assert.Len(t, ev, 9, "the nine columns and nothing else: %v", ev)
|
||||||
|
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
@@ -231,7 +229,7 @@ func assertExportedRow(
|
|||||||
require.NoError(t, err)
|
require.NoError(t, err)
|
||||||
assert.Equal(t, binaryBody, string(body))
|
assert.Equal(t, binaryBody, string(body))
|
||||||
assert.Equal(t, "base64", ev["body_encoding"])
|
assert.Equal(t, "base64", ev["body_encoding"])
|
||||||
assert.Len(t, ev, 11, "the ten columns and body_encoding: %v", ev)
|
assert.Len(t, ev, 10, "the nine columns and body_encoding: %v", ev)
|
||||||
}
|
}
|
||||||
|
|
||||||
// TestArchiveExport_Empty proves an archive with nothing in it exports
|
// TestArchiveExport_Empty proves an archive with nothing in it exports
|
||||||
@@ -460,13 +458,8 @@ func TestArchiveExport_OneFileOpenAtATime(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// heapPeak is an io.Writer that discards what it is given and records
|
// heapPeak is an io.Writer that discards what it is given and records
|
||||||
// the largest heap it saw at a write. It collects garbage twice before
|
// the largest heap it saw at a write. It collects garbage before each
|
||||||
// each reading, so the heap it reads is what is still held. Once is not
|
// reading, so the heap it reads is what is still held.
|
||||||
// enough: the libraries the export calls (regexp, under GORM's table
|
|
||||||
// names, and encoding/json among them) cache buffers in a sync.Pool,
|
|
||||||
// which keeps them through one collection, so after one the reading
|
|
||||||
// counts however many happen to be cached. That varies from run to run
|
|
||||||
// by about as much as the limit in TestArchiveExport_Streams.
|
|
||||||
type heapPeak struct {
|
type heapPeak struct {
|
||||||
max uint64
|
max uint64
|
||||||
}
|
}
|
||||||
@@ -474,7 +467,6 @@ type heapPeak struct {
|
|||||||
func (p *heapPeak) Write(b []byte) (int, error) {
|
func (p *heapPeak) Write(b []byte) (int, error) {
|
||||||
var m runtime.MemStats
|
var m runtime.MemStats
|
||||||
|
|
||||||
runtime.GC()
|
|
||||||
runtime.GC()
|
runtime.GC()
|
||||||
runtime.ReadMemStats(&m)
|
runtime.ReadMemStats(&m)
|
||||||
p.max = max(p.max, m.HeapAlloc)
|
p.max = max(p.max, m.HeapAlloc)
|
||||||
@@ -504,8 +496,6 @@ func exportHeapGrowth(t *testing.T, rows, bodySize int) uint64 {
|
|||||||
|
|
||||||
export := listExport(t, path)
|
export := listExport(t, path)
|
||||||
|
|
||||||
// Twice, for the reason heapPeak gives.
|
|
||||||
runtime.GC()
|
|
||||||
runtime.GC()
|
runtime.GC()
|
||||||
|
|
||||||
var start runtime.MemStats
|
var start runtime.MemStats
|
||||||
|
|||||||
@@ -85,7 +85,6 @@ func TestDeliverDatabase_ArchivesEvent(t *testing.T) {
|
|||||||
|
|
||||||
webhookDB := testWebhookDB(t)
|
webhookDB := testWebhookDB(t)
|
||||||
event := seedEvent(t, webhookDB, `{"archived":true}`)
|
event := seedEvent(t, webhookDB, `{"archived":true}`)
|
||||||
event.RawQuery = eventQuery
|
|
||||||
d := seedDatabaseTargetDelivery(t, webhookDB, event, tgt)
|
d := seedDatabaseTargetDelivery(t, webhookDB, event, tgt)
|
||||||
|
|
||||||
env.eng.ExportDeliverDatabase(webhookDB, d)
|
env.eng.ExportDeliverDatabase(webhookDB, d)
|
||||||
@@ -114,7 +113,6 @@ func TestDeliverDatabase_ArchivesEvent(t *testing.T) {
|
|||||||
assert.Equal(t, event.ID, rows[0].EventID)
|
assert.Equal(t, event.ID, rows[0].EventID)
|
||||||
assert.Equal(t, event.WebhookID, rows[0].WebhookID)
|
assert.Equal(t, event.WebhookID, rows[0].WebhookID)
|
||||||
assert.Equal(t, event.Method, rows[0].Method)
|
assert.Equal(t, event.Method, rows[0].Method)
|
||||||
assert.Equal(t, eventQuery, rows[0].RawQuery)
|
|
||||||
assert.JSONEq(t, `{"archived":true}`, rows[0].Body)
|
assert.JSONEq(t, `{"archived":true}`, rows[0].Body)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -8,7 +8,6 @@ import (
|
|||||||
"fmt"
|
"fmt"
|
||||||
"io"
|
"io"
|
||||||
"net/http"
|
"net/http"
|
||||||
"net/url"
|
|
||||||
"sort"
|
"sort"
|
||||||
"sync"
|
"sync"
|
||||||
"time"
|
"time"
|
||||||
@@ -33,11 +32,6 @@ type HTTPTargetConfig struct {
|
|||||||
URL string `json:"url"`
|
URL string `json:"url"`
|
||||||
Headers map[string]string `json:"headers,omitempty"`
|
Headers map[string]string `json:"headers,omitempty"`
|
||||||
Timeout int `json:"timeout,omitempty"`
|
Timeout int `json:"timeout,omitempty"`
|
||||||
|
|
||||||
// ForwardQuery passes each event's query string on to the target,
|
|
||||||
// appended to URL. Off, the target URL is sent exactly as
|
|
||||||
// configured.
|
|
||||||
ForwardQuery bool `json:"forwardQuery,omitempty"`
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// httpCore holds the retry, backoff, and circuit-breaker
|
// httpCore holds the retry, backoff, and circuit-breaker
|
||||||
@@ -450,10 +444,6 @@ func (t *httpTarget) doHTTPRequest(
|
|||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
if cfg.ForwardQuery {
|
|
||||||
appendQuery(req.URL, event.RawQuery)
|
|
||||||
}
|
|
||||||
|
|
||||||
originScoped := applyRequestHeaders(
|
originScoped := applyRequestHeaders(
|
||||||
req, event, cfg, t.eng.userAgent(),
|
req, event, cfg, t.eng.userAgent(),
|
||||||
)
|
)
|
||||||
@@ -484,19 +474,6 @@ func (t *httpTarget) doHTTPRequest(
|
|||||||
return resp.StatusCode, string(body), dur, nil
|
return resp.StatusCode, string(body), dur, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
// appendQuery adds an event's query string to a delivery's URL, joined
|
|
||||||
// with "&" to any query string the target URL already has.
|
|
||||||
func appendQuery(u *url.URL, rawQuery string) {
|
|
||||||
switch {
|
|
||||||
case rawQuery == "":
|
|
||||||
return
|
|
||||||
case u.RawQuery == "":
|
|
||||||
u.RawQuery = rawQuery
|
|
||||||
default:
|
|
||||||
u.RawQuery += "&" + rawQuery
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// clientForRequest returns the client for one delivery attempt.
|
// clientForRequest returns the client for one delivery attempt.
|
||||||
// originScoped is the header set applyRequestHeaders built for that
|
// originScoped is the header set applyRequestHeaders built for that
|
||||||
// attempt; a request with neither a per-target timeout nor an
|
// attempt; a request with neither a per-target timeout nor an
|
||||||
|
|||||||
@@ -1,172 +0,0 @@
|
|||||||
package delivery_test
|
|
||||||
|
|
||||||
import (
|
|
||||||
"context"
|
|
||||||
"encoding/json"
|
|
||||||
"net/http"
|
|
||||||
"net/http/httptest"
|
|
||||||
"testing"
|
|
||||||
|
|
||||||
"github.com/google/uuid"
|
|
||||||
"github.com/stretchr/testify/assert"
|
|
||||||
"github.com/stretchr/testify/require"
|
|
||||||
"sneak.berlin/go/webhooker/internal/database"
|
|
||||||
"sneak.berlin/go/webhooker/internal/delivery"
|
|
||||||
)
|
|
||||||
|
|
||||||
// eventQuery is the query string the events in these tests arrived
|
|
||||||
// with.
|
|
||||||
const eventQuery = "a=1&b=2"
|
|
||||||
|
|
||||||
// httpTargetConfig is the stored configuration of an HTTP target at
|
|
||||||
// targetURL.
|
|
||||||
func httpTargetConfig(
|
|
||||||
t *testing.T, targetURL string, forwardQuery bool,
|
|
||||||
) string {
|
|
||||||
t.Helper()
|
|
||||||
|
|
||||||
cfg, err := json.Marshal(delivery.HTTPTargetConfig{
|
|
||||||
URL: targetURL, ForwardQuery: forwardQuery,
|
|
||||||
})
|
|
||||||
require.NoError(t, err)
|
|
||||||
|
|
||||||
return string(cfg)
|
|
||||||
}
|
|
||||||
|
|
||||||
// deliverWithQuery sends one event that arrived with eventQuery to an
|
|
||||||
// HTTP target configured with cfg, through the path a received event's
|
|
||||||
// delivery takes, and returns the attempt it recorded.
|
|
||||||
func deliverWithQuery(t *testing.T, cfg string) database.DeliveryResult {
|
|
||||||
t.Helper()
|
|
||||||
|
|
||||||
s := newISetup(t)
|
|
||||||
|
|
||||||
event := iSeedEvent(t, s.WebhookDB, s.WebhookID, "{}")
|
|
||||||
d := iSeedDelivery(
|
|
||||||
t, s.WebhookDB, event.ID, uuid.NewString(),
|
|
||||||
database.DeliveryStatusPending,
|
|
||||||
)
|
|
||||||
task := iTask(
|
|
||||||
d, event, s.WebhookID, d.TargetID, "query", cfg, 0, 1, &event.Body,
|
|
||||||
)
|
|
||||||
task.RawQuery = eventQuery
|
|
||||||
|
|
||||||
s.Engine.ExportProcessNewTask(context.TODO(), &task)
|
|
||||||
|
|
||||||
var result database.DeliveryResult
|
|
||||||
|
|
||||||
require.NoError(t, s.WebhookDB.Where(
|
|
||||||
"delivery_id = ?", d.ID,
|
|
||||||
).First(&result).Error)
|
|
||||||
|
|
||||||
return result
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestDeliverHTTP_ForwardQuery proves the URL a delivery is sent to:
|
|
||||||
// with the target's setting off, the target URL exactly as configured;
|
|
||||||
// with it on, the event's query string appended, joined with "&" to a
|
|
||||||
// query string the target URL already has.
|
|
||||||
func TestDeliverHTTP_ForwardQuery(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
// The target URL's path, without and with a query string of its
|
|
||||||
// own.
|
|
||||||
const (
|
|
||||||
plain = "/in"
|
|
||||||
withQuery = "/in?key=k"
|
|
||||||
)
|
|
||||||
|
|
||||||
tests := map[string]struct {
|
|
||||||
path string
|
|
||||||
forwardQuery bool
|
|
||||||
want string
|
|
||||||
}{
|
|
||||||
"off": {
|
|
||||||
path: plain, want: plain,
|
|
||||||
},
|
|
||||||
"off, the target URL has a query string": {
|
|
||||||
path: withQuery, want: withQuery,
|
|
||||||
},
|
|
||||||
"on": {
|
|
||||||
path: plain, forwardQuery: true, want: plain + "?" + eventQuery,
|
|
||||||
},
|
|
||||||
"on, the target URL has a query string": {
|
|
||||||
path: withQuery, forwardQuery: true,
|
|
||||||
want: withQuery + "&" + eventQuery,
|
|
||||||
},
|
|
||||||
}
|
|
||||||
|
|
||||||
for name, tc := range tests {
|
|
||||||
t.Run(name, func(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
received := make(chan string, 1)
|
|
||||||
|
|
||||||
ts := httptest.NewServer(http.HandlerFunc(
|
|
||||||
func(w http.ResponseWriter, r *http.Request) {
|
|
||||||
received <- r.RequestURI
|
|
||||||
|
|
||||||
w.WriteHeader(http.StatusOK)
|
|
||||||
},
|
|
||||||
))
|
|
||||||
t.Cleanup(ts.Close)
|
|
||||||
|
|
||||||
result := deliverWithQuery(t, httpTargetConfig(
|
|
||||||
t, ts.URL+tc.path, tc.forwardQuery,
|
|
||||||
))
|
|
||||||
|
|
||||||
assert.True(t, result.Success)
|
|
||||||
require.Len(t, received, 1)
|
|
||||||
assert.Equal(t, tc.want, <-received)
|
|
||||||
})
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestDeliverHTTP_ForwardedQueryKeepsTheTargetURLMasked proves the
|
|
||||||
// credential in a target URL's own query string stays masked once the
|
|
||||||
// event's query string is appended to it: in a response or error that
|
|
||||||
// echoes the URL the target was sent, as the event log's Redactor shows
|
|
||||||
// it, and in the error a failed connection stores.
|
|
||||||
func TestDeliverHTTP_ForwardedQueryKeepsTheTargetURLMasked(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
const secret = "s3cr3t"
|
|
||||||
|
|
||||||
received := make(chan string, 1)
|
|
||||||
|
|
||||||
ts := httptest.NewServer(http.HandlerFunc(
|
|
||||||
func(w http.ResponseWriter, r *http.Request) {
|
|
||||||
received <- r.RequestURI
|
|
||||||
|
|
||||||
w.WriteHeader(http.StatusBadRequest)
|
|
||||||
},
|
|
||||||
))
|
|
||||||
t.Cleanup(ts.Close)
|
|
||||||
|
|
||||||
target := &database.Target{
|
|
||||||
Type: database.TargetTypeHTTP,
|
|
||||||
Config: httpTargetConfig(t, ts.URL+"/in?token="+secret, true),
|
|
||||||
}
|
|
||||||
|
|
||||||
deliverWithQuery(t, target.Config)
|
|
||||||
require.Len(t, received, 1)
|
|
||||||
|
|
||||||
sent := <-received
|
|
||||||
require.Equal(t, "/in?token="+secret+"&"+eventQuery, sent)
|
|
||||||
|
|
||||||
redactor := delivery.NewRedactor(target)
|
|
||||||
|
|
||||||
for _, echoed := range []string{sent, ts.URL + sent} {
|
|
||||||
shown := redactor.Redact("rejected " + echoed)
|
|
||||||
assert.NotContains(t, shown, secret, echoed)
|
|
||||||
assert.Contains(t, shown, delivery.RedactionMarker, echoed)
|
|
||||||
}
|
|
||||||
|
|
||||||
// Nothing listens on port 1.
|
|
||||||
failed := deliverWithQuery(t, httpTargetConfig(
|
|
||||||
t, "http://127.0.0.1:1/in?token="+secret, true,
|
|
||||||
))
|
|
||||||
require.NotEmpty(t, failed.Error)
|
|
||||||
assert.NotContains(t, failed.Error, secret)
|
|
||||||
assert.NotContains(t, failed.Error, eventQuery)
|
|
||||||
}
|
|
||||||
@@ -9,9 +9,9 @@ import (
|
|||||||
)
|
)
|
||||||
|
|
||||||
// logTarget is a fire-and-forget target that logs the entire
|
// logTarget is a fire-and-forget target that logs the entire
|
||||||
// inbound webhook — the full request body, query string and
|
// inbound webhook — the full request body and headers, plus
|
||||||
// headers, plus the method, content type, and the webhook and
|
// the method, content type, and the webhook and entrypoint
|
||||||
// entrypoint ids — then records a single successful attempt.
|
// ids — then records a single successful attempt.
|
||||||
//
|
//
|
||||||
// This is the one log call in the service that deliberately writes
|
// This is the one log call in the service that deliberately writes
|
||||||
// unbounded client-chosen bytes, so it is the one exception to the
|
// unbounded client-chosen bytes, so it is the one exception to the
|
||||||
@@ -46,7 +46,6 @@ func (t *logTarget) Deliver(
|
|||||||
"webhook_id", d.Event.WebhookID,
|
"webhook_id", d.Event.WebhookID,
|
||||||
"entrypoint_id", d.Event.EntrypointID,
|
"entrypoint_id", d.Event.EntrypointID,
|
||||||
"method", d.Event.Method,
|
"method", d.Event.Method,
|
||||||
"raw_query", d.Event.RawQuery,
|
|
||||||
"content_type", d.Event.ContentType,
|
"content_type", d.Event.ContentType,
|
||||||
"headers", d.Event.Headers,
|
"headers", d.Event.Headers,
|
||||||
"body", d.Event.Body,
|
"body", d.Event.Body,
|
||||||
|
|||||||
@@ -162,22 +162,18 @@ func targetSecrets(t *database.Target) []string {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// urlSecrets returns the substrings of a destination URL that
|
// urlSecrets returns the substrings of a destination URL that
|
||||||
// must not survive into a rendered page: the whole URL; its
|
// must not survive into a rendered page: the whole URL, the
|
||||||
// path, unless that is empty or "/"; its query string, and the
|
// parts of it MaskURL elides, and any userinfo.
|
||||||
// request URI that carries it, which a remote echoing the
|
|
||||||
// request line shows even when the URL has no path; and its
|
|
||||||
// userinfo and password.
|
|
||||||
//
|
//
|
||||||
// No length floor is applied to the path, the query string or
|
// No length floor is applied to the path, and none to the
|
||||||
// the userinfo. A short path or a four-byte username is
|
// userinfo. A short path or a four-byte username is treated as
|
||||||
// treated as a credential exactly like a long one, because the
|
// a credential exactly like a long one, because the field takes
|
||||||
// field takes an arbitrary URL and no part of it can be
|
// an arbitrary URL and no part of it can be assumed non-secret —
|
||||||
// assumed non-secret — the same rule MaskURL applies.
|
// the same rule MaskURL applies. headerSecrets does carry a
|
||||||
// headerSecrets does carry a floor, and the difference is
|
// floor, and the difference is deliberate: a header is picked
|
||||||
// deliberate: a header is picked out by a name-shaped guess
|
// out by a name-shaped guess and its value may be ordinary
|
||||||
// and its value may be ordinary text, whereas a URL's path,
|
// text, whereas a URL's path and userinfo are credential
|
||||||
// query string and userinfo are credential material by
|
// material by position.
|
||||||
// position.
|
|
||||||
func urlSecrets(raw string) []string {
|
func urlSecrets(raw string) []string {
|
||||||
raw = strings.TrimSpace(raw)
|
raw = strings.TrimSpace(raw)
|
||||||
if raw == "" {
|
if raw == "" {
|
||||||
@@ -192,11 +188,12 @@ func urlSecrets(raw string) []string {
|
|||||||
}
|
}
|
||||||
|
|
||||||
if parsed.Path != "" && parsed.Path != "/" {
|
if parsed.Path != "" && parsed.Path != "/" {
|
||||||
secrets = append(secrets, parsed.EscapedPath())
|
requestURI := parsed.RequestURI()
|
||||||
}
|
secrets = append(secrets, requestURI)
|
||||||
|
|
||||||
if parsed.RawQuery != "" {
|
if escaped := parsed.EscapedPath(); escaped != requestURI {
|
||||||
secrets = append(secrets, parsed.RequestURI(), parsed.RawQuery)
|
secrets = append(secrets, escaped)
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
if parsed.User != nil {
|
if parsed.User != nil {
|
||||||
|
|||||||
@@ -202,48 +202,6 @@ func TestRedactor_RemovesHTTPURLQueryAndUserinfo(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// TestRedactor_RemovesEchoedQueryOfURLWithoutPath covers an
|
|
||||||
// HTTP target URL whose credential is all in its query string.
|
|
||||||
// Written with or without the "/", the request line sends it
|
|
||||||
// as "/?token=…", and a target passing the event's query string
|
|
||||||
// on sends that after an "&". The event's part stays visible:
|
|
||||||
// the event's page shows it anyway.
|
|
||||||
func TestRedactor_RemovesEchoedQueryOfURLWithoutPath(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
const secret = "s3cr3t"
|
|
||||||
|
|
||||||
marker := delivery.RedactionMarker
|
|
||||||
|
|
||||||
// An echoed request line, and what the event log shows of it.
|
|
||||||
echoes := map[string]string{
|
|
||||||
"POST /?token=" + secret + " HTTP/1.1": "POST " + marker +
|
|
||||||
" HTTP/1.1",
|
|
||||||
"POST ?token=" + secret + " HTTP/1.1": "POST ?" + marker +
|
|
||||||
" HTTP/1.1",
|
|
||||||
"POST /?token=" + secret + "&a=1&b=2 HTTP/1.1": "POST " +
|
|
||||||
marker + "&a=1&b=2 HTTP/1.1",
|
|
||||||
"POST ?token=" + secret + "&a=1&b=2 HTTP/1.1": "POST ?" +
|
|
||||||
marker + "&a=1&b=2 HTTP/1.1",
|
|
||||||
}
|
|
||||||
|
|
||||||
for _, dest := range []string{
|
|
||||||
"https://example.com/?token=" + secret,
|
|
||||||
"https://example.com?token=" + secret,
|
|
||||||
} {
|
|
||||||
r := delivery.NewRedactor(&database.Target{
|
|
||||||
Type: database.TargetTypeHTTP,
|
|
||||||
Config: `{"url":"` + dest + `"}`,
|
|
||||||
})
|
|
||||||
|
|
||||||
for echoed, want := range echoes {
|
|
||||||
assert.Equal(
|
|
||||||
t, want, r.Redact(echoed), "%s: %s", dest, echoed,
|
|
||||||
)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestRedactor_LeavesUnrelatedTextAlone pins that the
|
// TestRedactor_LeavesUnrelatedTextAlone pins that the
|
||||||
// redactor matches literally: it does not guess at what a
|
// redactor matches literally: it does not guess at what a
|
||||||
// secret looks like, so ordinary response content survives.
|
// secret looks like, so ordinary response content survives.
|
||||||
|
|||||||
@@ -310,7 +310,6 @@ func createReplayDelivery(
|
|||||||
TargetConfig: target.Config,
|
TargetConfig: target.Config,
|
||||||
MaxRetries: target.MaxRetries,
|
MaxRetries: target.MaxRetries,
|
||||||
Method: event.Method,
|
Method: event.Method,
|
||||||
RawQuery: event.RawQuery,
|
|
||||||
Headers: event.Headers,
|
Headers: event.Headers,
|
||||||
ContentType: event.ContentType,
|
ContentType: event.ContentType,
|
||||||
Body: replayBody(event.Body),
|
Body: replayBody(event.Body),
|
||||||
|
|||||||
@@ -26,9 +26,6 @@ const paramDeliveryID = "deliveryID"
|
|||||||
// dispatches to it: the notifier is recorded, not run.
|
// dispatches to it: the notifier is recorded, not run.
|
||||||
const replayTargetURL = "http://93.184.216.34/hook"
|
const replayTargetURL = "http://93.184.216.34/hook"
|
||||||
|
|
||||||
// replayEventQuery is the query string a seeded event arrived with.
|
|
||||||
const replayEventQuery = "a=1&b=2"
|
|
||||||
|
|
||||||
// seedFailedDelivery records an event, a terminally failed delivery of
|
// seedFailedDelivery records an event, a terminally failed delivery of
|
||||||
// it to the given target, and the attempt that failed.
|
// it to the given target, and the attempt that failed.
|
||||||
func seedFailedDelivery(
|
func seedFailedDelivery(
|
||||||
@@ -45,7 +42,6 @@ func seedFailedDelivery(
|
|||||||
WebhookID: webhookID,
|
WebhookID: webhookID,
|
||||||
EntrypointID: "entrypoint-" + webhookID,
|
EntrypointID: "entrypoint-" + webhookID,
|
||||||
Method: http.MethodPost,
|
Method: http.MethodPost,
|
||||||
RawQuery: replayEventQuery,
|
|
||||||
Headers: `{"X-Test":["yes"]}`,
|
Headers: `{"X-Test":["yes"]}`,
|
||||||
Body: `{"replay":"me"}`,
|
Body: `{"replay":"me"}`,
|
||||||
ContentType: contentTypeJSON,
|
ContentType: contentTypeJSON,
|
||||||
@@ -300,10 +296,6 @@ func assertReplayTask(
|
|||||||
"replay must use the target's current configuration",
|
"replay must use the target's current configuration",
|
||||||
)
|
)
|
||||||
assert.Equal(t, event.Method, task.Method)
|
assert.Equal(t, event.Method, task.Method)
|
||||||
assert.Equal(
|
|
||||||
t, replayEventQuery, task.RawQuery,
|
|
||||||
"replay re-sends the stored query string",
|
|
||||||
)
|
|
||||||
assert.Equal(t, event.Headers, task.Headers)
|
assert.Equal(t, event.Headers, task.Headers)
|
||||||
assert.Equal(t, event.ContentType, task.ContentType)
|
assert.Equal(t, event.ContentType, task.ContentType)
|
||||||
assert.Equal(t, 1, task.AttemptNum)
|
assert.Equal(t, 1, task.AttemptNum)
|
||||||
|
|||||||
@@ -1,169 +0,0 @@
|
|||||||
package handlers
|
|
||||||
|
|
||||||
import (
|
|
||||||
"net/http"
|
|
||||||
|
|
||||||
"github.com/go-chi/chi"
|
|
||||||
"github.com/google/uuid"
|
|
||||||
"sneak.berlin/go/webhooker/internal/database"
|
|
||||||
)
|
|
||||||
|
|
||||||
// HandleEntrypointCreate handles adding a new entrypoint.
|
|
||||||
func (h *Handlers) HandleEntrypointCreate() http.HandlerFunc {
|
|
||||||
return func(w http.ResponseWriter, r *http.Request) {
|
|
||||||
userID, ok := h.getUserID(r)
|
|
||||||
if !ok {
|
|
||||||
http.Redirect(
|
|
||||||
w, r, "/pages/login", http.StatusSeeOther,
|
|
||||||
)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
sourceID := chi.URLParam(r, "sourceID")
|
|
||||||
|
|
||||||
var webhook database.Webhook
|
|
||||||
|
|
||||||
err := h.db.DB().Where(
|
|
||||||
"id = ? AND user_id = ?", sourceID, userID,
|
|
||||||
).First(&webhook).Error
|
|
||||||
if err != nil {
|
|
||||||
h.renderError(w, r, http.StatusNotFound)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
// The body size cap is enforced by the MaxBodySize
|
|
||||||
// middleware, which runs before CSRF parses the form.
|
|
||||||
err = r.ParseForm()
|
|
||||||
if err != nil {
|
|
||||||
h.renderError(w, r, http.StatusBadRequest)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
description := r.PostFormValue("description")
|
|
||||||
|
|
||||||
entrypoint := &database.Entrypoint{
|
|
||||||
WebhookID: webhook.ID,
|
|
||||||
Path: uuid.New().String(),
|
|
||||||
Description: description,
|
|
||||||
Active: true,
|
|
||||||
}
|
|
||||||
|
|
||||||
err = h.db.DB().Create(entrypoint).Error
|
|
||||||
if err != nil {
|
|
||||||
h.serverError(w, r, "failed to create entrypoint", err)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
http.Redirect(
|
|
||||||
w, r, withNotice("/hook/"+webhook.ID, entrypointAdded),
|
|
||||||
http.StatusSeeOther,
|
|
||||||
)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// HandleEntrypointEdit handles changing an entrypoint's description.
|
|
||||||
// It writes only the description column, so the entrypoint keeps its
|
|
||||||
// URL, and an activate or deactivate saved since the page was shown
|
|
||||||
// is not undone.
|
|
||||||
func (h *Handlers) HandleEntrypointEdit() http.HandlerFunc {
|
|
||||||
return func(w http.ResponseWriter, r *http.Request) {
|
|
||||||
userID, ok := h.getUserID(r)
|
|
||||||
if !ok {
|
|
||||||
http.Redirect(
|
|
||||||
w, r, "/pages/login", http.StatusSeeOther,
|
|
||||||
)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
sourceID := chi.URLParam(r, "sourceID")
|
|
||||||
entrypointID := chi.URLParam(r, "entrypointID")
|
|
||||||
|
|
||||||
var webhook database.Webhook
|
|
||||||
|
|
||||||
err := h.db.DB().Where(
|
|
||||||
"id = ? AND user_id = ?", sourceID, userID,
|
|
||||||
).First(&webhook).Error
|
|
||||||
if err != nil {
|
|
||||||
h.renderError(w, r, http.StatusNotFound)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
// The body size cap is enforced by the MaxBodySize
|
|
||||||
// middleware, which runs before CSRF parses the form.
|
|
||||||
err = r.ParseForm()
|
|
||||||
if err != nil {
|
|
||||||
h.renderError(w, r, http.StatusBadRequest)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
result := h.db.DB().Model(&database.Entrypoint{}).Where(
|
|
||||||
"id = ? AND webhook_id = ?", entrypointID, webhook.ID,
|
|
||||||
).Update("description", r.PostFormValue("description"))
|
|
||||||
if result.Error != nil {
|
|
||||||
h.serverError(
|
|
||||||
w, r, "failed to edit entrypoint", result.Error,
|
|
||||||
)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
// The id came from the URL and may name another webhook's
|
|
||||||
// entrypoint, which this webhook does not have.
|
|
||||||
if result.RowsAffected == 0 {
|
|
||||||
h.renderError(w, r, http.StatusNotFound)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
http.Redirect(
|
|
||||||
w, r, withNotice("/hook/"+webhook.ID, entrypointSaved),
|
|
||||||
http.StatusSeeOther,
|
|
||||||
)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// HandleEntrypointDelete handles deleting an entrypoint.
|
|
||||||
func (h *Handlers) HandleEntrypointDelete() http.HandlerFunc {
|
|
||||||
return h.deleteChildResource(
|
|
||||||
"entrypointID", &database.Entrypoint{},
|
|
||||||
"failed to delete entrypoint",
|
|
||||||
nil,
|
|
||||||
entrypointDeleted,
|
|
||||||
)
|
|
||||||
}
|
|
||||||
|
|
||||||
// HandleEntrypointToggle handles toggling an entrypoint's
|
|
||||||
// active state.
|
|
||||||
func (h *Handlers) HandleEntrypointToggle() http.HandlerFunc {
|
|
||||||
return h.toggleChildResource(
|
|
||||||
"entrypointID",
|
|
||||||
func(webhookID, childID string) (bool, error) {
|
|
||||||
var ep database.Entrypoint
|
|
||||||
|
|
||||||
err := h.db.DB().Where(
|
|
||||||
"id = ? AND webhook_id = ?",
|
|
||||||
childID, webhookID,
|
|
||||||
).First(&ep).Error
|
|
||||||
if err != nil {
|
|
||||||
return false, err
|
|
||||||
}
|
|
||||||
|
|
||||||
// Only the active column: saving the whole row would
|
|
||||||
// write back the description read above over an edit
|
|
||||||
// saved since.
|
|
||||||
active := !ep.Active
|
|
||||||
|
|
||||||
return active, h.db.DB().Model(&ep).
|
|
||||||
Update("active", active).Error
|
|
||||||
},
|
|
||||||
"failed to toggle entrypoint",
|
|
||||||
entrypointActivated, entrypointDeactivated,
|
|
||||||
)
|
|
||||||
}
|
|
||||||
@@ -1,621 +0,0 @@
|
|||||||
package handlers
|
|
||||||
|
|
||||||
import (
|
|
||||||
"net/http"
|
|
||||||
"slices"
|
|
||||||
"time"
|
|
||||||
|
|
||||||
"github.com/dustin/go-humanize"
|
|
||||||
"gorm.io/gorm"
|
|
||||||
"sneak.berlin/go/webhooker/internal/database"
|
|
||||||
"sneak.berlin/go/webhooker/internal/delivery"
|
|
||||||
)
|
|
||||||
|
|
||||||
// DeliveryView is the display-safe projection of a delivery
|
|
||||||
// for the event log page. Its target is a TargetView, so the
|
|
||||||
// stored configuration blob — which holds the target's
|
|
||||||
// credential — has no path to the template.
|
|
||||||
type DeliveryView struct {
|
|
||||||
ID string
|
|
||||||
Status database.DeliveryStatus
|
|
||||||
Target delivery.TargetView
|
|
||||||
|
|
||||||
// Replay is set on a delivery the Replay action created.
|
|
||||||
Replay bool
|
|
||||||
|
|
||||||
// Created is how long ago the delivery was created, and
|
|
||||||
// CreatedUTC the full timestamp the page shows on hover.
|
|
||||||
Created string
|
|
||||||
CreatedUTC string
|
|
||||||
|
|
||||||
// Results is this delivery's attempts in attempt order,
|
|
||||||
// bounded by maxRenderedAttempts. Without them a failure
|
|
||||||
// renders as the status word alone and says nothing about
|
|
||||||
// why.
|
|
||||||
Results []DeliveryResultView
|
|
||||||
|
|
||||||
// AttemptCount is how many attempts were recorded, which
|
|
||||||
// is more than len(Results) once the middle was dropped.
|
|
||||||
AttemptCount int
|
|
||||||
|
|
||||||
// AttemptsOmitted is how many attempts were dropped from
|
|
||||||
// the middle of Results. The page must show it, or the
|
|
||||||
// bound would hide history rather than fold it.
|
|
||||||
AttemptsOmitted int
|
|
||||||
|
|
||||||
// Paused is set while the delivery is retrying and its
|
|
||||||
// target's circuit breaker is open, and nil otherwise.
|
|
||||||
Paused *PausedView
|
|
||||||
}
|
|
||||||
|
|
||||||
// eventLogTarget is what the event log needs to know about
|
|
||||||
// one target: the display-safe view its template renders, and
|
|
||||||
// the redactor that keeps that target's own credential out of
|
|
||||||
// the text its remote peer chose. The two are kept together
|
|
||||||
// so a caller cannot pick up one without the other, and apart
|
|
||||||
// from TargetView so the secrets never reach a template.
|
|
||||||
type eventLogTarget struct {
|
|
||||||
View delivery.TargetView
|
|
||||||
Redactor delivery.Redactor
|
|
||||||
}
|
|
||||||
|
|
||||||
// The event log's show query parameter and its two values: the events
|
|
||||||
// with a failed delivery, and those with a delivery still pending or
|
|
||||||
// retrying.
|
|
||||||
const (
|
|
||||||
showParam = "show"
|
|
||||||
showFailed = "failed"
|
|
||||||
showPending = "pending"
|
|
||||||
)
|
|
||||||
|
|
||||||
// HandleSourceLogs shows the request/response logs for a
|
|
||||||
// webhook.
|
|
||||||
func (h *Handlers) HandleSourceLogs() http.HandlerFunc {
|
|
||||||
return func(w http.ResponseWriter, r *http.Request) {
|
|
||||||
webhook, ok := h.ownedWebhook(w, r)
|
|
||||||
if !ok {
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
targets, err := h.loadTargetMap(webhook.ID)
|
|
||||||
if err != nil {
|
|
||||||
// Without the map every delivery renders through a
|
|
||||||
// zero redactor, so failing the page is the only
|
|
||||||
// safe answer.
|
|
||||||
h.serverError(w, r, "failed to load targets", err)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
// Any other value of show lists every event, as no value
|
|
||||||
// does.
|
|
||||||
show := r.URL.Query().Get(showParam)
|
|
||||||
|
|
||||||
statuses := eventLogStatuses(show)
|
|
||||||
if statuses == nil {
|
|
||||||
show = ""
|
|
||||||
}
|
|
||||||
|
|
||||||
evts, total, ok := h.loadEventsWithDeliveries(
|
|
||||||
w, r, webhook, targets, statuses,
|
|
||||||
)
|
|
||||||
if !ok {
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
failed, pending, err := h.countFailedAndPendingEvents(webhook.ID)
|
|
||||||
if err != nil {
|
|
||||||
h.serverError(w, r, "failed to count events", err)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
data := map[string]any{
|
|
||||||
tmplKeyWebhook: &webhook,
|
|
||||||
"Events": evts,
|
|
||||||
"TotalEvents": total,
|
|
||||||
"Show": show,
|
|
||||||
"FailedEvents": failed,
|
|
||||||
"PendingEvents": pending,
|
|
||||||
}
|
|
||||||
|
|
||||||
h.renderTemplate(w, r, "source_logs.html", data)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// loadTargetMap loads targets into a map of display-safe
|
|
||||||
// views keyed by target ID, each paired with its redactor.
|
|
||||||
// The projection happens here so that no caller can hand a
|
|
||||||
// raw target, configuration blob and all, to a template: the
|
|
||||||
// raw rows do not leave this function.
|
|
||||||
//
|
|
||||||
// The load is Unscoped because deleting a target only soft
|
|
||||||
// deletes the row while its deliveries survive in the
|
|
||||||
// per-webhook database. Both halves of the map need those rows:
|
|
||||||
// a scoped load leaves an old delivery with a zero redactor,
|
|
||||||
// which renders its response bodies unredacted, and with a zero
|
|
||||||
// view, which renders its target as a blank name.
|
|
||||||
//
|
|
||||||
// This map is historical display only. It is built for the event
|
|
||||||
// log and an event's own page, and reaches nothing but
|
|
||||||
// DeliveryView.Target: the target list on the source detail page,
|
|
||||||
// the edit form and the replay path each resolve targets
|
|
||||||
// themselves, and a deleted row is refused there as before.
|
|
||||||
func (h *Handlers) loadTargetMap(
|
|
||||||
webhookID string,
|
|
||||||
) (map[string]eventLogTarget, error) {
|
|
||||||
var targets []database.Target
|
|
||||||
|
|
||||||
err := h.db.DB().Unscoped().Where(
|
|
||||||
"webhook_id = ?", webhookID,
|
|
||||||
).Find(&targets).Error
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
|
|
||||||
targetMap := make(
|
|
||||||
map[string]eventLogTarget, len(targets),
|
|
||||||
)
|
|
||||||
|
|
||||||
for i := range targets {
|
|
||||||
targetMap[targets[i].ID] = eventLogTarget{
|
|
||||||
Redactor: delivery.NewRedactor(&targets[i]),
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// The views come from NewTargetViews rather than being
|
|
||||||
// rebuilt here, so the masking rules stay in one place and a
|
|
||||||
// deleted target's configuration is masked by the same code
|
|
||||||
// that masks a live one's.
|
|
||||||
for _, v := range delivery.NewTargetViews(targets) {
|
|
||||||
entry := targetMap[v.ID]
|
|
||||||
entry.View = v
|
|
||||||
targetMap[v.ID] = entry
|
|
||||||
}
|
|
||||||
|
|
||||||
return targetMap, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// loadEventsWithDeliveries loads the recentEventLimit newest events
|
|
||||||
// and their deliveries from the per-webhook database, and the total
|
|
||||||
// number of events stored. Given delivery statuses, both cover only
|
|
||||||
// the events with a delivery in one of them. Events come back as
|
|
||||||
// capped projections rather than database.Event rows: see
|
|
||||||
// eventLogColumns for why the cut happens in SQL.
|
|
||||||
//
|
|
||||||
// The bool reports whether the load succeeded. It is false
|
|
||||||
// once this has answered the request with an error, and the
|
|
||||||
// caller must then render nothing further.
|
|
||||||
func (h *Handlers) loadEventsWithDeliveries(
|
|
||||||
w http.ResponseWriter,
|
|
||||||
r *http.Request,
|
|
||||||
webhook database.Webhook,
|
|
||||||
targetMap map[string]eventLogTarget,
|
|
||||||
statuses []database.DeliveryStatus,
|
|
||||||
) ([]EventLogView, int64, bool) {
|
|
||||||
if !h.dbMgr.DBExists(webhook.ID) {
|
|
||||||
return nil, 0, true
|
|
||||||
}
|
|
||||||
|
|
||||||
webhookDB, err := h.dbMgr.GetDB(webhook.ID)
|
|
||||||
if err != nil {
|
|
||||||
h.serverError(
|
|
||||||
w, r, "failed to get webhook database", err,
|
|
||||||
)
|
|
||||||
|
|
||||||
return nil, 0, false
|
|
||||||
}
|
|
||||||
|
|
||||||
rows, totalEvents, err := loadEventLogRows(
|
|
||||||
webhookDB, webhook.ID, statuses,
|
|
||||||
)
|
|
||||||
if err != nil {
|
|
||||||
h.serverError(w, r, "failed to load events", err)
|
|
||||||
|
|
||||||
return nil, 0, false
|
|
||||||
}
|
|
||||||
|
|
||||||
result, ok := h.eventLogViews(
|
|
||||||
w, r, webhookDB, webhook.ID, rows, targetMap,
|
|
||||||
maxRenderedBodyBytes,
|
|
||||||
)
|
|
||||||
|
|
||||||
return result, totalEvents, ok
|
|
||||||
}
|
|
||||||
|
|
||||||
// eventLogViews projects loaded events for rendering, each with
|
|
||||||
// its deliveries, how many times it has been resubmitted and the
|
|
||||||
// entrypoint it arrived at (for a resubmitted copy, the one the
|
|
||||||
// request it copies arrived at), and with its request headers only
|
|
||||||
// when their text holds at most maxHeaderBytes. Like
|
|
||||||
// loadEventsWithDeliveries, it reports false once it has answered
|
|
||||||
// the request with an error.
|
|
||||||
func (h *Handlers) eventLogViews(
|
|
||||||
w http.ResponseWriter,
|
|
||||||
r *http.Request,
|
|
||||||
webhookDB *gorm.DB,
|
|
||||||
webhookID string,
|
|
||||||
rows []eventLogRow,
|
|
||||||
targetMap map[string]eventLogTarget,
|
|
||||||
maxHeaderBytes int,
|
|
||||||
) ([]EventLogView, bool) {
|
|
||||||
result := make([]EventLogView, len(rows))
|
|
||||||
eventDeliveries := make([][]database.Delivery, len(rows))
|
|
||||||
|
|
||||||
var deliveryIDs []string
|
|
||||||
|
|
||||||
eventIDs := make([]string, len(rows))
|
|
||||||
|
|
||||||
for i := range rows {
|
|
||||||
result[i] = rows[i].view(webhookID, maxHeaderBytes)
|
|
||||||
eventIDs[i] = rows[i].ID
|
|
||||||
|
|
||||||
webhookDB.Where(
|
|
||||||
"event_id = ?", rows[i].ID,
|
|
||||||
).Find(&eventDeliveries[i])
|
|
||||||
|
|
||||||
for j := range eventDeliveries[i] {
|
|
||||||
deliveryIDs = append(
|
|
||||||
deliveryIDs, eventDeliveries[i][j].ID,
|
|
||||||
)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
attempts, err := h.loadDeliveryResults(
|
|
||||||
webhookDB, deliveryIDs,
|
|
||||||
)
|
|
||||||
if err != nil {
|
|
||||||
h.serverError(
|
|
||||||
w, r, "failed to load delivery attempts", err,
|
|
||||||
)
|
|
||||||
|
|
||||||
return nil, false
|
|
||||||
}
|
|
||||||
|
|
||||||
resubmits, err := resubmitCounts(webhookDB, eventIDs)
|
|
||||||
if err != nil {
|
|
||||||
h.serverError(
|
|
||||||
w, r, "failed to count event resubmissions", err,
|
|
||||||
)
|
|
||||||
|
|
||||||
return nil, false
|
|
||||||
}
|
|
||||||
|
|
||||||
entrypoints, err := h.entrypointNames(webhookID)
|
|
||||||
if err != nil {
|
|
||||||
h.serverError(w, r, "failed to load entrypoints", err)
|
|
||||||
|
|
||||||
return nil, false
|
|
||||||
}
|
|
||||||
|
|
||||||
for i := range rows {
|
|
||||||
result[i].Deliveries = h.newDeliveryViews(
|
|
||||||
eventDeliveries[i], targetMap, attempts,
|
|
||||||
)
|
|
||||||
result[i].ResubmitCount = resubmits[rows[i].ID]
|
|
||||||
|
|
||||||
name, ok := entrypoints[rows[i].EntrypointID]
|
|
||||||
if !ok {
|
|
||||||
name = "deleted entrypoint"
|
|
||||||
}
|
|
||||||
|
|
||||||
result[i].Entrypoint = name
|
|
||||||
}
|
|
||||||
|
|
||||||
return result, true
|
|
||||||
}
|
|
||||||
|
|
||||||
// loadEventLogRows reads the event log projection of the
|
|
||||||
// recentEventLimit newest events, newest first, and the total number
|
|
||||||
// of events stored, both narrowed by statuses as eventsWithStatus
|
|
||||||
// narrows them.
|
|
||||||
func loadEventLogRows(
|
|
||||||
webhookDB *gorm.DB,
|
|
||||||
webhookID string,
|
|
||||||
statuses []database.DeliveryStatus,
|
|
||||||
) ([]eventLogRow, int64, error) {
|
|
||||||
totalEvents, err := countEventsWithStatus(
|
|
||||||
webhookDB, webhookID, statuses,
|
|
||||||
)
|
|
||||||
if err != nil {
|
|
||||||
return nil, 0, err
|
|
||||||
}
|
|
||||||
|
|
||||||
var rows []eventLogRow
|
|
||||||
|
|
||||||
err = eventsWithStatus(webhookDB, webhookID, statuses).Select(
|
|
||||||
eventLogColumns,
|
|
||||||
maxRenderedBodyBytes, maxRenderedBodyBytes, maxRenderedBodyBytes,
|
|
||||||
).Order("created_at DESC").Limit(recentEventLimit).Find(&rows).Error
|
|
||||||
|
|
||||||
return rows, totalEvents, err
|
|
||||||
}
|
|
||||||
|
|
||||||
// eventLogStatuses returns the delivery statuses the event log's show
|
|
||||||
// value lists events by, or nil for one that lists every event.
|
|
||||||
func eventLogStatuses(show string) []database.DeliveryStatus {
|
|
||||||
switch show {
|
|
||||||
case showFailed:
|
|
||||||
return []database.DeliveryStatus{database.DeliveryStatusFailed}
|
|
||||||
case showPending:
|
|
||||||
return []database.DeliveryStatus{
|
|
||||||
database.DeliveryStatusPending,
|
|
||||||
database.DeliveryStatusRetrying,
|
|
||||||
}
|
|
||||||
default:
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// eventsWithStatus selects the webhook's events, or, given statuses,
|
|
||||||
// the recentEventLimit newest of those with at least one delivery in
|
|
||||||
// one of them.
|
|
||||||
//
|
|
||||||
// Given statuses, its cost follows the matching deliveries. SQLite
|
|
||||||
// never reorders a CROSS JOIN, so it reads each join's left side
|
|
||||||
// first: the distinct event IDs of the matching deliveries, through
|
|
||||||
// idx_deliveries_status; then each of those events by ID, sorted to
|
|
||||||
// keep the newest; then the rows of only the events kept, so no other
|
|
||||||
// event's body is read. With a plain "id IN (matching deliveries)"
|
|
||||||
// condition instead, SQLite, which keeps no statistics on these
|
|
||||||
// tables, walks every event newest first.
|
|
||||||
func eventsWithStatus(
|
|
||||||
webhookDB *gorm.DB,
|
|
||||||
webhookID string,
|
|
||||||
statuses []database.DeliveryStatus,
|
|
||||||
) *gorm.DB {
|
|
||||||
if statuses == nil {
|
|
||||||
return webhookDB.Model(&database.Event{}).Where(
|
|
||||||
"webhook_id = ?", webhookID,
|
|
||||||
)
|
|
||||||
}
|
|
||||||
|
|
||||||
matching := webhookDB.Model(&database.Delivery{}).
|
|
||||||
Distinct("event_id").Where("status IN ?", statuses)
|
|
||||||
|
|
||||||
newest := webhookDB.Table("(?) AS matching", matching).
|
|
||||||
Joins("CROSS JOIN events ON events.id = matching.event_id").
|
|
||||||
Where(
|
|
||||||
"events.webhook_id = ? AND events.deleted_at IS NULL",
|
|
||||||
webhookID,
|
|
||||||
).
|
|
||||||
Order("events.created_at DESC").Limit(recentEventLimit).
|
|
||||||
Select("events.id AS event_id")
|
|
||||||
|
|
||||||
return webhookDB.Table("(?) AS newest", newest).
|
|
||||||
Joins("CROSS JOIN events ON events.id = newest.event_id")
|
|
||||||
}
|
|
||||||
|
|
||||||
// countEventsWithStatus counts the webhook's events with at least one
|
|
||||||
// delivery in one of the statuses, or every event when statuses is
|
|
||||||
// nil. Given statuses, it counts the distinct events of the matching
|
|
||||||
// deliveries and reads nothing but those deliveries, through
|
|
||||||
// idx_deliveries_status, where counting the events would read every
|
|
||||||
// event row. That is the same number, because retention deletes an
|
|
||||||
// event's deliveries with it.
|
|
||||||
func countEventsWithStatus(
|
|
||||||
webhookDB *gorm.DB,
|
|
||||||
webhookID string,
|
|
||||||
statuses []database.DeliveryStatus,
|
|
||||||
) (int64, error) {
|
|
||||||
var count int64
|
|
||||||
|
|
||||||
if statuses == nil {
|
|
||||||
err := webhookDB.Model(&database.Event{}).Where(
|
|
||||||
"webhook_id = ?", webhookID,
|
|
||||||
).Count(&count).Error
|
|
||||||
|
|
||||||
return count, err
|
|
||||||
}
|
|
||||||
|
|
||||||
err := webhookDB.Model(&database.Delivery{}).Distinct("event_id").
|
|
||||||
Where("status IN ?", statuses).Count(&count).Error
|
|
||||||
|
|
||||||
return count, err
|
|
||||||
}
|
|
||||||
|
|
||||||
// countFailedAndPendingEvents returns how many of the webhook's events
|
|
||||||
// the event log lists when it shows only those with a failed delivery,
|
|
||||||
// and when it shows only those with a delivery pending or retrying.
|
|
||||||
func (h *Handlers) countFailedAndPendingEvents(
|
|
||||||
webhookID string,
|
|
||||||
) (int64, int64, error) {
|
|
||||||
if !h.dbMgr.DBExists(webhookID) {
|
|
||||||
return 0, 0, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
webhookDB, err := h.dbMgr.GetDB(webhookID)
|
|
||||||
if err != nil {
|
|
||||||
return 0, 0, err
|
|
||||||
}
|
|
||||||
|
|
||||||
failed, err := countEventsWithStatus(
|
|
||||||
webhookDB, webhookID, eventLogStatuses(showFailed),
|
|
||||||
)
|
|
||||||
if err != nil {
|
|
||||||
return 0, 0, err
|
|
||||||
}
|
|
||||||
|
|
||||||
pending, err := countEventsWithStatus(
|
|
||||||
webhookDB, webhookID, eventLogStatuses(showPending),
|
|
||||||
)
|
|
||||||
if err != nil {
|
|
||||||
return 0, 0, err
|
|
||||||
}
|
|
||||||
|
|
||||||
return failed, pending, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// resubmitCounts reports, for each of the page's events, how many
|
|
||||||
// events have been resubmitted from it.
|
|
||||||
//
|
|
||||||
// One grouped query covers the page rather than one query per event.
|
|
||||||
// The page shows at most recentEventLimit events, far below SQLite's
|
|
||||||
// bound parameter ceiling, so it needs no chunking as the delivery
|
|
||||||
// result load does.
|
|
||||||
func resubmitCounts(
|
|
||||||
webhookDB *gorm.DB, eventIDs []string,
|
|
||||||
) (map[string]int, error) {
|
|
||||||
counts := make(map[string]int, len(eventIDs))
|
|
||||||
|
|
||||||
if len(eventIDs) == 0 {
|
|
||||||
return counts, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
var rows []struct {
|
|
||||||
ResubmittedFromID string
|
|
||||||
Total int
|
|
||||||
}
|
|
||||||
|
|
||||||
err := webhookDB.Model(&database.Event{}).
|
|
||||||
Select("resubmitted_from_id, count(*) AS total").
|
|
||||||
Where("resubmitted_from_id IN ?", eventIDs).
|
|
||||||
Group("resubmitted_from_id").
|
|
||||||
Find(&rows).Error
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
|
|
||||||
for _, row := range rows {
|
|
||||||
counts[row.ResubmittedFromID] = row.Total
|
|
||||||
}
|
|
||||||
|
|
||||||
return counts, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// deliveryIDChunkSize bounds how many delivery IDs go into one
|
|
||||||
// IN clause. SQLite refuses a statement carrying more than
|
|
||||||
// SQLITE_MAX_VARIABLE_NUMBER (32766) bound parameters, and a
|
|
||||||
// page holds one delivery per target per event, so a webhook
|
|
||||||
// with enough targets would turn the whole query into an error
|
|
||||||
// and the page into zero attempts.
|
|
||||||
const deliveryIDChunkSize = 500
|
|
||||||
|
|
||||||
// loadDeliveryResults loads the recorded attempts for the
|
|
||||||
// page's deliveries, keyed by delivery ID.
|
|
||||||
//
|
|
||||||
// Each response body is cut by SQLite rather than in Go, for
|
|
||||||
// the reason deliveryResultColumns gives. How many attempts a
|
|
||||||
// delivery has is the target's MaxRetries, which the
|
|
||||||
// authenticated operator sets; how many of them reach the page
|
|
||||||
// is bounded again by maxRenderedAttempts.
|
|
||||||
func (h *Handlers) loadDeliveryResults(
|
|
||||||
webhookDB *gorm.DB,
|
|
||||||
deliveryIDs []string,
|
|
||||||
) (map[string][]deliveryResultRow, error) {
|
|
||||||
byDelivery := make(map[string][]deliveryResultRow)
|
|
||||||
|
|
||||||
for chunk := range slices.Chunk(
|
|
||||||
deliveryIDs, deliveryIDChunkSize,
|
|
||||||
) {
|
|
||||||
var rows []deliveryResultRow
|
|
||||||
|
|
||||||
err := webhookDB.Model(
|
|
||||||
&database.DeliveryResult{},
|
|
||||||
).Select(
|
|
||||||
deliveryResultColumns, maxRenderedResponseBytes,
|
|
||||||
).Where(
|
|
||||||
"delivery_id IN ?", chunk,
|
|
||||||
).Order("attempt_num ASC").Find(&rows).Error
|
|
||||||
if err != nil {
|
|
||||||
// Returning what was loaded so far renders the
|
|
||||||
// deliveries in the failed chunk as never having run,
|
|
||||||
// which is indistinguishable from ones that really
|
|
||||||
// never ran. The page fails instead.
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
|
|
||||||
for i := range rows {
|
|
||||||
byDelivery[rows[i].DeliveryID] = append(
|
|
||||||
byDelivery[rows[i].DeliveryID], rows[i],
|
|
||||||
)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
return byDelivery, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// newDeliveryViews projects deliveries for rendering,
|
|
||||||
// resolving each one's target to its display-safe view and
|
|
||||||
// each one's attempts through that target's redactor. A
|
|
||||||
// retrying delivery also reads its target's circuit breaker.
|
|
||||||
func (h *Handlers) newDeliveryViews(
|
|
||||||
deliveries []database.Delivery,
|
|
||||||
targetMap map[string]eventLogTarget,
|
|
||||||
attempts map[string][]deliveryResultRow,
|
|
||||||
) []DeliveryView {
|
|
||||||
views := make([]DeliveryView, len(deliveries))
|
|
||||||
|
|
||||||
for i := range deliveries {
|
|
||||||
target := targetMap[deliveries[i].TargetID]
|
|
||||||
rows := attempts[deliveries[i].ID]
|
|
||||||
created := deliveries[i].CreatedAt
|
|
||||||
|
|
||||||
results, omitted := renderedAttempts(
|
|
||||||
rows, target.Redactor,
|
|
||||||
)
|
|
||||||
|
|
||||||
views[i] = DeliveryView{
|
|
||||||
ID: deliveries[i].ID,
|
|
||||||
Status: deliveries[i].Status,
|
|
||||||
Target: target.View,
|
|
||||||
Replay: deliveries[i].Replay,
|
|
||||||
Created: humanize.Time(created),
|
|
||||||
CreatedUTC: created.UTC().Format(time.DateTime) + " UTC",
|
|
||||||
Results: results,
|
|
||||||
AttemptCount: len(rows),
|
|
||||||
AttemptsOmitted: omitted,
|
|
||||||
}
|
|
||||||
|
|
||||||
if deliveries[i].Status == database.DeliveryStatusRetrying {
|
|
||||||
views[i].Paused = h.deliveryPausedView(
|
|
||||||
deliveries[i].TargetID, rows,
|
|
||||||
)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
return views
|
|
||||||
}
|
|
||||||
|
|
||||||
// maxRenderedAttempts bounds how many of one delivery's
|
|
||||||
// attempts the page renders. Past it the middle is dropped and
|
|
||||||
// counted, keeping the first attempts and the last ones: how
|
|
||||||
// the delivery started failing and how it ended are what a
|
|
||||||
// reader needs, and the count says plainly that the rest was
|
|
||||||
// dropped rather than never recorded.
|
|
||||||
const (
|
|
||||||
renderedAttemptsHead = 10
|
|
||||||
renderedAttemptsTail = 10
|
|
||||||
maxRenderedAttempts = renderedAttemptsHead +
|
|
||||||
renderedAttemptsTail
|
|
||||||
)
|
|
||||||
|
|
||||||
// renderedAttempts projects a delivery's attempts through the
|
|
||||||
// target's redactor, at most maxRenderedAttempts of them, and
|
|
||||||
// reports how many it dropped.
|
|
||||||
func renderedAttempts(
|
|
||||||
rows []deliveryResultRow,
|
|
||||||
redactor delivery.Redactor,
|
|
||||||
) ([]DeliveryResultView, int) {
|
|
||||||
omitted := 0
|
|
||||||
|
|
||||||
if len(rows) > maxRenderedAttempts {
|
|
||||||
omitted = len(rows) - maxRenderedAttempts
|
|
||||||
|
|
||||||
kept := make(
|
|
||||||
[]deliveryResultRow, 0, maxRenderedAttempts,
|
|
||||||
)
|
|
||||||
kept = append(kept, rows[:renderedAttemptsHead]...)
|
|
||||||
kept = append(
|
|
||||||
kept, rows[len(rows)-renderedAttemptsTail:]...,
|
|
||||||
)
|
|
||||||
rows = kept
|
|
||||||
}
|
|
||||||
|
|
||||||
views := make([]DeliveryResultView, len(rows))
|
|
||||||
for i := range rows {
|
|
||||||
views[i] = rows[i].view(redactor)
|
|
||||||
}
|
|
||||||
|
|
||||||
return views, omitted
|
|
||||||
}
|
|
||||||
@@ -17,23 +17,19 @@ import (
|
|||||||
// bytes rather than characters, so the cap bounds the page in
|
// bytes rather than characters, so the cap bounds the page in
|
||||||
// bytes whatever the payload's encoding. Cutting in SQLite
|
// bytes whatever the payload's encoding. Cutting in SQLite
|
||||||
// rather than in Go is the point of the projection — an
|
// rather than in Go is the point of the projection — an
|
||||||
// oversized body, query string or set of request headers never
|
// oversized body or set of request headers never becomes a Go
|
||||||
// becomes a Go string at all.
|
// string at all.
|
||||||
const eventLogColumns = "id, created_at, method, content_type, " +
|
const eventLogColumns = "id, created_at, method, content_type, " +
|
||||||
"resubmitted_from_id, entrypoint_id, " +
|
"resubmitted_from_id, entrypoint_id, " +
|
||||||
"substr(cast(raw_query as blob), 1, ?) AS raw_query, " +
|
|
||||||
"length(cast(raw_query as blob)) AS raw_query_bytes, " +
|
|
||||||
"substr(cast(headers as blob), 1, ?) AS headers, " +
|
"substr(cast(headers as blob), 1, ?) AS headers, " +
|
||||||
"length(cast(headers as blob)) AS headers_bytes, " +
|
"length(cast(headers as blob)) AS headers_bytes, " +
|
||||||
"substr(cast(body as blob), 1, ?) AS body, " +
|
"substr(cast(body as blob), 1, ?) AS body, " +
|
||||||
"length(cast(body as blob)) AS body_bytes"
|
"length(cast(body as blob)) AS body_bytes"
|
||||||
|
|
||||||
// eventColumns is eventLogColumns for the event's own page, which
|
// eventColumns is eventLogColumns for the event's own page, which
|
||||||
// shows the whole body, the whole query string and every request
|
// shows the whole body and every request header.
|
||||||
// header.
|
|
||||||
const eventColumns = "id, created_at, method, content_type, " +
|
const eventColumns = "id, created_at, method, content_type, " +
|
||||||
"resubmitted_from_id, entrypoint_id, raw_query, " +
|
"resubmitted_from_id, entrypoint_id, headers, " +
|
||||||
"length(cast(raw_query as blob)) AS raw_query_bytes, headers, " +
|
|
||||||
"length(cast(headers as blob)) AS headers_bytes, " +
|
"length(cast(headers as blob)) AS headers_bytes, " +
|
||||||
"cast(body as blob) AS body, " +
|
"cast(body as blob) AS body, " +
|
||||||
"length(cast(body as blob)) AS body_bytes"
|
"length(cast(body as blob)) AS body_bytes"
|
||||||
@@ -61,13 +57,6 @@ type EventLogView struct {
|
|||||||
// entrypoint's secret.
|
// entrypoint's secret.
|
||||||
Entrypoint string
|
Entrypoint string
|
||||||
|
|
||||||
// RawQuery is the query string the event arrived with.
|
|
||||||
// RawQueryCut reports one left out, RawQuery then empty, because
|
|
||||||
// it holds more than maxRenderedBodyBytes; only the event log
|
|
||||||
// leaves it out.
|
|
||||||
RawQuery string
|
|
||||||
RawQueryCut bool
|
|
||||||
|
|
||||||
// Headers is the event's request headers as text, one
|
// Headers is the event's request headers as text, one
|
||||||
// "Name: value" line per value, sorted by name. HeadersCut
|
// "Name: value" line per value, sorted by name. HeadersCut
|
||||||
// reports headers left out because they hold more than
|
// reports headers left out because they hold more than
|
||||||
@@ -96,9 +85,9 @@ func (v EventLogView) ResubmittedFrom() bool {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// eventLogRow is one row of the event log projection, or of
|
// eventLogRow is one row of the event log projection, or of
|
||||||
// eventColumns. In the event log its query string, headers and
|
// eventColumns. In the event log its headers and body columns
|
||||||
// body columns arrive already cut to the cap by SQLite, each with
|
// arrive already cut to the cap by SQLite, each with its true
|
||||||
// its true size beside it.
|
// size beside it.
|
||||||
type eventLogRow struct {
|
type eventLogRow struct {
|
||||||
ID string
|
ID string
|
||||||
CreatedAt time.Time
|
CreatedAt time.Time
|
||||||
@@ -106,8 +95,6 @@ type eventLogRow struct {
|
|||||||
ContentType string
|
ContentType string
|
||||||
ResubmittedFromID *string
|
ResubmittedFromID *string
|
||||||
EntrypointID string
|
EntrypointID string
|
||||||
RawQuery string
|
|
||||||
RawQueryBytes int64
|
|
||||||
Headers string
|
Headers string
|
||||||
HeadersBytes int64
|
HeadersBytes int64
|
||||||
Body []byte
|
Body []byte
|
||||||
@@ -127,13 +114,6 @@ func (r *eventLogRow) view(
|
|||||||
|
|
||||||
headers, fit := requestHeaderLines(r.Headers, maxHeaderBytes)
|
headers, fit := requestHeaderLines(r.Headers, maxHeaderBytes)
|
||||||
|
|
||||||
rawQuery := r.RawQuery
|
|
||||||
rawQueryCut := r.RawQueryBytes > int64(len(rawQuery))
|
|
||||||
|
|
||||||
if rawQueryCut {
|
|
||||||
rawQuery = ""
|
|
||||||
}
|
|
||||||
|
|
||||||
return EventLogView{
|
return EventLogView{
|
||||||
ID: r.ID,
|
ID: r.ID,
|
||||||
Method: r.Method,
|
Method: r.Method,
|
||||||
@@ -143,8 +123,6 @@ func (r *eventLogRow) view(
|
|||||||
Body: newBodyView(
|
Body: newBodyView(
|
||||||
"/hook/"+webhookID+"/events/"+r.ID, r.Body, r.BodyBytes,
|
"/hook/"+webhookID+"/events/"+r.ID, r.Body, r.BodyBytes,
|
||||||
),
|
),
|
||||||
RawQuery: rawQuery,
|
|
||||||
RawQueryCut: rawQueryCut,
|
|
||||||
Headers: strings.Join(headers, "\n"),
|
Headers: strings.Join(headers, "\n"),
|
||||||
HeadersCut: !fit || r.HeadersBytes > int64(len(r.Headers)),
|
HeadersCut: !fit || r.HeadersBytes > int64(len(r.Headers)),
|
||||||
ResubmittedFromID: from,
|
ResubmittedFromID: from,
|
||||||
|
|||||||
@@ -1,16 +1,13 @@
|
|||||||
package handlers_test
|
package handlers_test
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"context"
|
|
||||||
"encoding/json"
|
"encoding/json"
|
||||||
"net/http"
|
"net/http"
|
||||||
"net/http/httptest"
|
|
||||||
"slices"
|
"slices"
|
||||||
"strings"
|
"strings"
|
||||||
"testing"
|
"testing"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
"github.com/go-chi/chi"
|
|
||||||
"github.com/google/uuid"
|
"github.com/google/uuid"
|
||||||
"github.com/stretchr/testify/assert"
|
"github.com/stretchr/testify/assert"
|
||||||
"github.com/stretchr/testify/require"
|
"github.com/stretchr/testify/require"
|
||||||
@@ -20,15 +17,14 @@ import (
|
|||||||
|
|
||||||
// arrivedAt is how a page names the entrypoint an event arrived at.
|
// arrivedAt is how a page names the entrypoint an event arrived at.
|
||||||
func arrivedAt(name string) string {
|
func arrivedAt(name string) string {
|
||||||
return `Arrived at <span class="text-gray-900 wrap-anywhere">` + name +
|
return `Arrived at <span class="text-gray-900">` + name + `</span>`
|
||||||
`</span>`
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// copiedRequestArrivedAt is how a page names, for a resubmitted copy,
|
// copiedRequestArrivedAt is how a page names, for a resubmitted copy,
|
||||||
// the entrypoint the request it copies arrived at.
|
// the entrypoint the request it copies arrived at.
|
||||||
func copiedRequestArrivedAt(name string) string {
|
func copiedRequestArrivedAt(name string) string {
|
||||||
return `The request it copies arrived at ` +
|
return `The request it copies arrived at <span class="text-gray-900">` +
|
||||||
`<span class="text-gray-900 wrap-anywhere">` + name + `</span>`
|
name + `</span>`
|
||||||
}
|
}
|
||||||
|
|
||||||
// headerBox is how a page shows an event's request header lines: as
|
// headerBox is how a page shows an event's request header lines: as
|
||||||
@@ -119,7 +115,6 @@ func TestEventRequest_EachEventShowsItsOwnEntrypointAndHeaders(
|
|||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
assert.Contains(t, page, arrivedAt("Billing sender"))
|
assert.Contains(t, page, arrivedAt("Billing sender"))
|
||||||
assert.Contains(t, page, "No query string.")
|
|
||||||
assert.Contains(t, page, headerBox(
|
assert.Contains(t, page, headerBox(
|
||||||
"Accept: */*",
|
"Accept: */*",
|
||||||
"User-Agent: shop/1 build\t7",
|
"User-Agent: shop/1 build\t7",
|
||||||
@@ -335,70 +330,3 @@ func TestEventRequest_ManyShortHeaderLines(t *testing.T) {
|
|||||||
})
|
})
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// TestHandleWebhook_StoresAndShowsTheQueryString posts to an
|
|
||||||
// entrypoint's URL with a query string and proves the event stores it
|
|
||||||
// as sent, and shows it escaped in the event log and on its own page,
|
|
||||||
// in a box like the one the request headers show in.
|
|
||||||
func TestHandleWebhook_StoresAndShowsTheQueryString(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
f := newRecentEventsFixture(t)
|
|
||||||
ep := seedEntrypoint(t, f.db, f.webhook.ID)
|
|
||||||
|
|
||||||
req := httptest.NewRequestWithContext(
|
|
||||||
context.Background(), http.MethodPost,
|
|
||||||
"/h/"+ep.Path+"?a=1&b=2", strings.NewReader("{}"),
|
|
||||||
)
|
|
||||||
|
|
||||||
rctx := chi.NewRouteContext()
|
|
||||||
rctx.URLParams.Add("uuid", ep.Path)
|
|
||||||
|
|
||||||
req = req.WithContext(context.WithValue(
|
|
||||||
req.Context(), chi.RouteCtxKey, rctx,
|
|
||||||
))
|
|
||||||
|
|
||||||
w := httptest.NewRecorder()
|
|
||||||
f.h.HandleWebhook().ServeHTTP(w, req)
|
|
||||||
|
|
||||||
require.Equal(t, http.StatusOK, w.Code)
|
|
||||||
|
|
||||||
var stored database.Event
|
|
||||||
|
|
||||||
require.NoError(t, f.webhookDB.First(&stored).Error)
|
|
||||||
assert.Equal(t, "a=1&b=2", stored.RawQuery)
|
|
||||||
|
|
||||||
page := renderSourceLogsPage(t, f.h, f.sess, f.webhook.ID)
|
|
||||||
assert.Contains(t, page, headerBox("a=1&b=2"))
|
|
||||||
|
|
||||||
w = serveEventPage(t, f.h, f.sess, f.webhook.ID, stored.ID)
|
|
||||||
require.Equal(t, http.StatusOK, w.Code)
|
|
||||||
assert.Contains(t, w.Body.String(), headerBox("a=1&b=2"))
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestEventRequest_QueryStringOverTheLimit proves the event log leaves
|
|
||||||
// out a query string that holds more than it shows of a body, and links
|
|
||||||
// to the event's own page, which shows it whole.
|
|
||||||
func TestEventRequest_QueryStringOverTheLimit(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
f := newRecentEventsFixture(t)
|
|
||||||
ep := f.entrypoint(t, "Billing sender")
|
|
||||||
event := f.eventAt(t, ep, `{}`, time.Now())
|
|
||||||
query := "q=" + strings.Repeat("x", bodyCap)
|
|
||||||
|
|
||||||
require.NoError(t, f.webhookDB.Model(event).Update(
|
|
||||||
"raw_query", query,
|
|
||||||
).Error)
|
|
||||||
|
|
||||||
page := renderSourceLogsPage(t, f.h, f.sess, f.webhook.ID)
|
|
||||||
assert.Contains(t, page, `<a href="/hook/`+f.webhook.ID+`/events/`+
|
|
||||||
event.ID+`" class="btn-small">Show the query string</a>`)
|
|
||||||
assert.NotContains(t, page, "q=x")
|
|
||||||
assert.Less(t, len(page), 4*bodyCap)
|
|
||||||
|
|
||||||
w := serveEventPage(t, f.h, f.sess, f.webhook.ID, event.ID)
|
|
||||||
require.Equal(t, http.StatusOK, w.Code)
|
|
||||||
assert.Contains(t, w.Body.String(), headerBox(query))
|
|
||||||
assert.NotContains(t, w.Body.String(), "Show the query string")
|
|
||||||
}
|
|
||||||
|
|||||||
@@ -30,7 +30,6 @@ type resubmitSource struct {
|
|||||||
ID string
|
ID string
|
||||||
EntrypointID string
|
EntrypointID string
|
||||||
Method string
|
Method string
|
||||||
RawQuery string
|
|
||||||
Headers string
|
Headers string
|
||||||
ContentType string
|
ContentType string
|
||||||
Body []byte
|
Body []byte
|
||||||
@@ -40,7 +39,7 @@ type resubmitSource struct {
|
|||||||
// The cast to blob is what makes the driver hand back the stored bytes
|
// The cast to blob is what makes the driver hand back the stored bytes
|
||||||
// rather than a string conversion, the same reason eventBodyQuery
|
// rather than a string conversion, the same reason eventBodyQuery
|
||||||
// casts.
|
// casts.
|
||||||
const resubmitColumns = "id, entrypoint_id, method, raw_query, headers, " +
|
const resubmitColumns = "id, entrypoint_id, method, headers, " +
|
||||||
"content_type, cast(body as blob) AS body"
|
"content_type, cast(body as blob) AS body"
|
||||||
|
|
||||||
// HandleEventResubmit re-injects a stored event as a new undelivered
|
// HandleEventResubmit re-injects a stored event as a new undelivered
|
||||||
@@ -194,7 +193,6 @@ func (h *Handlers) queueResubmit(
|
|||||||
WebhookID: webhook.ID,
|
WebhookID: webhook.ID,
|
||||||
EntrypointID: src.EntrypointID,
|
EntrypointID: src.EntrypointID,
|
||||||
Method: src.Method,
|
Method: src.Method,
|
||||||
RawQuery: src.RawQuery,
|
|
||||||
HeadersJSON: src.Headers,
|
HeadersJSON: src.Headers,
|
||||||
ContentType: src.ContentType,
|
ContentType: src.ContentType,
|
||||||
Body: src.Body,
|
Body: src.Body,
|
||||||
|
|||||||
@@ -22,13 +22,9 @@ import (
|
|||||||
// dispatches to it: the notifier is recorded, not run.
|
// dispatches to it: the notifier is recorded, not run.
|
||||||
const resubmitTargetURL = "http://93.184.216.34/hook"
|
const resubmitTargetURL = "http://93.184.216.34/hook"
|
||||||
|
|
||||||
// resubmitEventHeaders and resubmitEventQuery are the stored header
|
// resubmitEventHeaders is the stored header JSON a seeded event
|
||||||
// JSON and query string a seeded event carries, so a test can prove the
|
// carries, so a test can prove the copy takes it verbatim.
|
||||||
// copy takes them verbatim.
|
const resubmitEventHeaders = `{"X-Test":["yes"],"X-Trace":["abc"]}`
|
||||||
const (
|
|
||||||
resubmitEventHeaders = `{"X-Test":["yes"],"X-Trace":["abc"]}`
|
|
||||||
resubmitEventQuery = "a=1&b=2"
|
|
||||||
)
|
|
||||||
|
|
||||||
// seedStoredEvent records one event in a webhook's own database with
|
// seedStoredEvent records one event in a webhook's own database with
|
||||||
// no deliveries at all, which is the state a captured event is in when
|
// no deliveries at all, which is the state a captured event is in when
|
||||||
@@ -47,7 +43,6 @@ func seedStoredEvent(
|
|||||||
WebhookID: webhookID,
|
WebhookID: webhookID,
|
||||||
EntrypointID: "entrypoint-" + webhookID,
|
EntrypointID: "entrypoint-" + webhookID,
|
||||||
Method: http.MethodPost,
|
Method: http.MethodPost,
|
||||||
RawQuery: resubmitEventQuery,
|
|
||||||
Headers: resubmitEventHeaders,
|
Headers: resubmitEventHeaders,
|
||||||
Body: body,
|
Body: body,
|
||||||
ContentType: contentTypeJSON,
|
ContentType: contentTypeJSON,
|
||||||
@@ -207,7 +202,6 @@ func assertEventCopy(
|
|||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
assert.Equal(t, original.Method, fresh.Method)
|
assert.Equal(t, original.Method, fresh.Method)
|
||||||
assert.Equal(t, resubmitEventQuery, fresh.RawQuery)
|
|
||||||
assert.Equal(t, original.Headers, fresh.Headers)
|
assert.Equal(t, original.Headers, fresh.Headers)
|
||||||
assert.Equal(t, original.Body, fresh.Body)
|
assert.Equal(t, original.Body, fresh.Body)
|
||||||
assert.Equal(t, int64(len(original.Body)), fresh.BodyBytes)
|
assert.Equal(t, int64(len(original.Body)), fresh.BodyBytes)
|
||||||
@@ -242,7 +236,6 @@ func assertResubmitTask(
|
|||||||
assert.Equal(t, target.ID, task.TargetID)
|
assert.Equal(t, target.ID, task.TargetID)
|
||||||
assert.Equal(t, target.Type, task.TargetType)
|
assert.Equal(t, target.Type, task.TargetType)
|
||||||
assert.Equal(t, fresh.Method, task.Method)
|
assert.Equal(t, fresh.Method, task.Method)
|
||||||
assert.Equal(t, fresh.RawQuery, task.RawQuery)
|
|
||||||
assert.Equal(t, fresh.Headers, task.Headers)
|
assert.Equal(t, fresh.Headers, task.Headers)
|
||||||
assert.Equal(t, fresh.ContentType, task.ContentType)
|
assert.Equal(t, fresh.ContentType, task.ContentType)
|
||||||
assert.Equal(t, 1, task.AttemptNum)
|
assert.Equal(t, 1, task.AttemptNum)
|
||||||
|
|||||||
@@ -1,230 +0,0 @@
|
|||||||
package handlers
|
|
||||||
|
|
||||||
import (
|
|
||||||
"net/http"
|
|
||||||
"strconv"
|
|
||||||
"strings"
|
|
||||||
|
|
||||||
"github.com/go-chi/chi"
|
|
||||||
"sneak.berlin/go/webhooker/internal/database"
|
|
||||||
)
|
|
||||||
|
|
||||||
// parseRetentionDays interprets a retention_days form value. It
|
|
||||||
// returns the number of days, or, for a value it refuses, the message
|
|
||||||
// the create and edit forms show; the message is empty when the value
|
|
||||||
// is accepted.
|
|
||||||
//
|
|
||||||
// An empty value yields fallback, which lets the create path apply the
|
|
||||||
// default and the edit path leave the stored value unchanged. A value
|
|
||||||
// of 0 is returned as 0 and is rewritten to the retain-forever
|
|
||||||
// sentinel by database.Webhook's BeforeSave hook. Anything unparseable
|
|
||||||
// or negative is refused rather than silently given a default.
|
|
||||||
//
|
|
||||||
// The upper bound is not cosmetic. The reaper computes its cutoff as a
|
|
||||||
// time.Duration, an int64 nanosecond count, so a day count above
|
|
||||||
// database.MaxFiniteRetentionDays overflows, puts the cutoff in the
|
|
||||||
// future, and deletes every event the webhook has. A finite value
|
|
||||||
// above that ceiling is therefore refused, and the message names the
|
|
||||||
// ceiling rather than implying the input was not a number.
|
|
||||||
//
|
|
||||||
// A value at or above the retain-forever sentinel is not out of range:
|
|
||||||
// it is what the edit form pre-fills for a retain-forever webhook, so
|
|
||||||
// submitting the form back unchanged has to keep meaning "forever"
|
|
||||||
// rather than being rejected.
|
|
||||||
func parseRetentionDays(raw string, fallback int) (int, string) {
|
|
||||||
raw = strings.TrimSpace(raw)
|
|
||||||
if raw == "" {
|
|
||||||
return fallback, ""
|
|
||||||
}
|
|
||||||
|
|
||||||
v, err := strconv.Atoi(raw)
|
|
||||||
if err != nil || v < 0 {
|
|
||||||
return 0, "Retention must be a whole number of days, or 0 to " +
|
|
||||||
"retain events forever."
|
|
||||||
}
|
|
||||||
|
|
||||||
if v >= database.RetentionForeverDays {
|
|
||||||
return database.RetentionForeverDays, ""
|
|
||||||
}
|
|
||||||
|
|
||||||
if v > database.MaxFiniteRetentionDays {
|
|
||||||
return 0, "Retention must be at most " +
|
|
||||||
strconv.Itoa(database.MaxFiniteRetentionDays) +
|
|
||||||
" days, or 0 to retain events forever."
|
|
||||||
}
|
|
||||||
|
|
||||||
return v, ""
|
|
||||||
}
|
|
||||||
|
|
||||||
// ownedWebhook resolves the request's sourceID parameter to a
|
|
||||||
// webhook the session's user owns.
|
|
||||||
//
|
|
||||||
// Ownership and existence are decided by one query, so a
|
|
||||||
// webhook belonging to another user is indistinguishable from
|
|
||||||
// one that does not exist: both are a 404, and neither confirms
|
|
||||||
// the id. Callers that reach further into a webhook's data —
|
|
||||||
// the event log page and the event body download — share this
|
|
||||||
// one check rather than restating it, so the download cannot
|
|
||||||
// come to authorize differently from the page that links to it.
|
|
||||||
//
|
|
||||||
// It reports false once it has written the response, which is a
|
|
||||||
// redirect to the login page for an unauthenticated request and
|
|
||||||
// a 404 otherwise. The caller returns without writing more.
|
|
||||||
func (h *Handlers) ownedWebhook(
|
|
||||||
w http.ResponseWriter,
|
|
||||||
r *http.Request,
|
|
||||||
) (database.Webhook, bool) {
|
|
||||||
var webhook database.Webhook
|
|
||||||
|
|
||||||
userID, ok := h.getUserID(r)
|
|
||||||
if !ok {
|
|
||||||
http.Redirect(
|
|
||||||
w, r, "/pages/login", http.StatusSeeOther,
|
|
||||||
)
|
|
||||||
|
|
||||||
return database.Webhook{}, false
|
|
||||||
}
|
|
||||||
|
|
||||||
sourceID := chi.URLParam(r, "sourceID")
|
|
||||||
|
|
||||||
err := h.db.DB().Where(
|
|
||||||
"id = ? AND user_id = ?", sourceID, userID,
|
|
||||||
).First(&webhook).Error
|
|
||||||
if err != nil {
|
|
||||||
h.renderError(w, r, http.StatusNotFound)
|
|
||||||
|
|
||||||
return database.Webhook{}, false
|
|
||||||
}
|
|
||||||
|
|
||||||
return webhook, true
|
|
||||||
}
|
|
||||||
|
|
||||||
// deleteChildResource returns a handler that deletes a child
|
|
||||||
// resource (entrypoint or target) belonging to a webhook. The
|
|
||||||
// optional afterDelete hook runs with the child's id once the
|
|
||||||
// delete has removed it, before the redirect, which carries done as
|
|
||||||
// its notice.
|
|
||||||
func (h *Handlers) deleteChildResource(
|
|
||||||
idParam string,
|
|
||||||
model any,
|
|
||||||
errMsg string,
|
|
||||||
afterDelete func(childID string),
|
|
||||||
done noticeCode,
|
|
||||||
) http.HandlerFunc {
|
|
||||||
return func(w http.ResponseWriter, r *http.Request) {
|
|
||||||
userID, ok := h.getUserID(r)
|
|
||||||
if !ok {
|
|
||||||
http.Redirect(
|
|
||||||
w, r, "/pages/login", http.StatusSeeOther,
|
|
||||||
)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
sourceID := chi.URLParam(r, "sourceID")
|
|
||||||
childID := chi.URLParam(r, idParam)
|
|
||||||
|
|
||||||
var webhook database.Webhook
|
|
||||||
|
|
||||||
err := h.db.DB().Where(
|
|
||||||
"id = ? AND user_id = ?", sourceID, userID,
|
|
||||||
).First(&webhook).Error
|
|
||||||
if err != nil {
|
|
||||||
h.renderError(w, r, http.StatusNotFound)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
result := h.db.DB().Where(
|
|
||||||
"id = ? AND webhook_id = ?",
|
|
||||||
childID, webhook.ID,
|
|
||||||
).Delete(model)
|
|
||||||
if result.Error != nil {
|
|
||||||
h.serverError(w, r, errMsg, result.Error)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
// Only for a row this webhook really had: the id came from
|
|
||||||
// the URL and may name another webhook's child.
|
|
||||||
if afterDelete != nil && result.RowsAffected > 0 {
|
|
||||||
afterDelete(childID)
|
|
||||||
}
|
|
||||||
|
|
||||||
http.Redirect(
|
|
||||||
w, r,
|
|
||||||
withNotice("/hook/"+webhook.ID, done),
|
|
||||||
http.StatusSeeOther,
|
|
||||||
)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// toggleChildResource returns a handler that toggles the active
|
|
||||||
// state of a child resource belonging to a webhook. toggleFn returns
|
|
||||||
// the new state, and the redirect carries activated or deactivated as
|
|
||||||
// its notice to match.
|
|
||||||
func (h *Handlers) toggleChildResource(
|
|
||||||
idParam string,
|
|
||||||
toggleFn func(webhookID, childID string) (bool, error),
|
|
||||||
errMsg string,
|
|
||||||
activated, deactivated noticeCode,
|
|
||||||
) http.HandlerFunc {
|
|
||||||
return func(w http.ResponseWriter, r *http.Request) {
|
|
||||||
userID, ok := h.getUserID(r)
|
|
||||||
if !ok {
|
|
||||||
http.Redirect(
|
|
||||||
w, r, "/pages/login", http.StatusSeeOther,
|
|
||||||
)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
sourceID := chi.URLParam(r, "sourceID")
|
|
||||||
childID := chi.URLParam(r, idParam)
|
|
||||||
|
|
||||||
var webhook database.Webhook
|
|
||||||
|
|
||||||
err := h.db.DB().Where(
|
|
||||||
"id = ? AND user_id = ?", sourceID, userID,
|
|
||||||
).First(&webhook).Error
|
|
||||||
if err != nil {
|
|
||||||
h.renderError(w, r, http.StatusNotFound)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
active, err := toggleFn(webhook.ID, childID)
|
|
||||||
if err != nil {
|
|
||||||
h.serverError(w, r, errMsg, err)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
done := deactivated
|
|
||||||
if active {
|
|
||||||
done = activated
|
|
||||||
}
|
|
||||||
|
|
||||||
http.Redirect(
|
|
||||||
w, r,
|
|
||||||
withNotice("/hook/"+webhook.ID, done),
|
|
||||||
http.StatusSeeOther,
|
|
||||||
)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// getUserID extracts the user ID from the session.
|
|
||||||
func (h *Handlers) getUserID(
|
|
||||||
r *http.Request,
|
|
||||||
) (string, bool) {
|
|
||||||
sess, err := h.session.Get(r)
|
|
||||||
if err != nil {
|
|
||||||
return "", false
|
|
||||||
}
|
|
||||||
|
|
||||||
if !h.session.IsAuthenticated(sess) {
|
|
||||||
return "", false
|
|
||||||
}
|
|
||||||
|
|
||||||
return h.session.GetUserID(sess)
|
|
||||||
}
|
|
||||||
File diff suppressed because it is too large
Load Diff
@@ -1,389 +0,0 @@
|
|||||||
package handlers
|
|
||||||
|
|
||||||
import (
|
|
||||||
"context"
|
|
||||||
"encoding/json"
|
|
||||||
"errors"
|
|
||||||
"fmt"
|
|
||||||
"net/http"
|
|
||||||
"strings"
|
|
||||||
|
|
||||||
"github.com/go-chi/chi"
|
|
||||||
"sneak.berlin/go/webhooker/internal/database"
|
|
||||||
"sneak.berlin/go/webhooker/internal/delivery"
|
|
||||||
)
|
|
||||||
|
|
||||||
// HandleTargetCreate handles adding a new target to a webhook.
|
|
||||||
func (h *Handlers) HandleTargetCreate() http.HandlerFunc {
|
|
||||||
return func(w http.ResponseWriter, r *http.Request) {
|
|
||||||
userID, ok := h.getUserID(r)
|
|
||||||
if !ok {
|
|
||||||
http.Redirect(
|
|
||||||
w, r, "/pages/login", http.StatusSeeOther,
|
|
||||||
)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
sourceID := chi.URLParam(r, "sourceID")
|
|
||||||
|
|
||||||
h.renameMu.Lock()
|
|
||||||
defer h.renameMu.Unlock()
|
|
||||||
|
|
||||||
var webhook database.Webhook
|
|
||||||
|
|
||||||
err := h.db.DB().Where(
|
|
||||||
"id = ? AND user_id = ?", sourceID, userID,
|
|
||||||
).First(&webhook).Error
|
|
||||||
if err != nil {
|
|
||||||
h.renderError(w, r, http.StatusNotFound)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
// The body size cap is enforced by the MaxBodySize
|
|
||||||
// middleware, which runs before CSRF parses the form.
|
|
||||||
err = r.ParseForm()
|
|
||||||
if err != nil {
|
|
||||||
h.renderError(w, r, http.StatusBadRequest)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
h.processTargetCreate(w, r, webhook)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// processTargetCreate validates and creates a new target. A refused
|
|
||||||
// submission shows the webhook page again, with the add target form
|
|
||||||
// open on the chosen type, the values entered, and the reason.
|
|
||||||
func (h *Handlers) processTargetCreate(
|
|
||||||
w http.ResponseWriter,
|
|
||||||
r *http.Request,
|
|
||||||
webhook database.Webhook,
|
|
||||||
) {
|
|
||||||
in := targetFormInputFrom(r)
|
|
||||||
|
|
||||||
target, errMsg, err := h.newTarget(r.Context(), webhook.ID, in)
|
|
||||||
if err != nil {
|
|
||||||
h.serverError(w, r, "failed to encode target config", err)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
if errMsg != "" {
|
|
||||||
h.renderSourceDetail(w, r, webhook, in, errMsg)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
err = h.db.DB().Create(target).Error
|
|
||||||
if err != nil {
|
|
||||||
h.serverError(w, r, "failed to create target", err)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
http.Redirect(
|
|
||||||
w, r, withNotice("/hook/"+webhook.ID, targetAdded),
|
|
||||||
http.StatusSeeOther,
|
|
||||||
)
|
|
||||||
}
|
|
||||||
|
|
||||||
// newTarget validates a new target for a webhook and returns the row
|
|
||||||
// to create, or, when it refuses the target, the message the form
|
|
||||||
// shows. An error is the server's fault, not a refusal: the accepted
|
|
||||||
// configuration could not be encoded. Every form that creates a
|
|
||||||
// target goes through here, so they all accept and refuse the same
|
|
||||||
// things.
|
|
||||||
func (h *Handlers) newTarget(
|
|
||||||
ctx context.Context,
|
|
||||||
webhookID string,
|
|
||||||
in targetFormInput,
|
|
||||||
) (*database.Target, string, error) {
|
|
||||||
target := &database.Target{
|
|
||||||
WebhookID: webhookID,
|
|
||||||
Type: in.Type,
|
|
||||||
Active: true,
|
|
||||||
}
|
|
||||||
|
|
||||||
errMsg, err := h.setTargetFromForm(ctx, target, in)
|
|
||||||
if err != nil || errMsg != "" {
|
|
||||||
return nil, errMsg, err
|
|
||||||
}
|
|
||||||
|
|
||||||
return target, "", nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// setTargetFromForm validates a target form against the target's type
|
|
||||||
// and, when it accepts it, sets the target's name, configuration and
|
|
||||||
// retry count from it. It returns the message the form shows for
|
|
||||||
// anything it refuses, an unknown type among them, and then leaves the
|
|
||||||
// target unchanged; an error is the server's fault, as for newTarget.
|
|
||||||
// The add target form and the target edit form both go through here,
|
|
||||||
// so the two cannot come to disagree about what a target may be.
|
|
||||||
func (h *Handlers) setTargetFromForm(
|
|
||||||
ctx context.Context,
|
|
||||||
target *database.Target,
|
|
||||||
in targetFormInput,
|
|
||||||
) (string, error) {
|
|
||||||
if in.Name == "" {
|
|
||||||
return "Name is required", nil
|
|
||||||
}
|
|
||||||
|
|
||||||
configJSON, errMsg, err := h.buildTargetConfig(ctx, target.Type, in)
|
|
||||||
if err != nil || errMsg != "" {
|
|
||||||
return errMsg, err
|
|
||||||
}
|
|
||||||
|
|
||||||
// An empty max_retries keeps the target's count: the
|
|
||||||
// fire-and-forget default of 0 for a new target, and the stored
|
|
||||||
// count for an edited one, since the forms for target types that
|
|
||||||
// do not retry have no such field. A value that is filled in but
|
|
||||||
// invalid is refused rather than becoming that count, so a typo
|
|
||||||
// cannot destroy the count a target is delivering with.
|
|
||||||
maxRetries, err := parseMaxRetries(in.MaxRetries, target.MaxRetries)
|
|
||||||
if err != nil {
|
|
||||||
return "Invalid delivery attempts: " + retriesErrorMessage(err), nil
|
|
||||||
}
|
|
||||||
|
|
||||||
target.Name = in.Name
|
|
||||||
target.Config = configJSON
|
|
||||||
target.MaxRetries = maxRetries
|
|
||||||
|
|
||||||
return "", nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// targetFormInput carries the raw values of a target form. Both the
|
|
||||||
// create and the edit path fill one and hand it to setTargetFromForm,
|
|
||||||
// so neither can come to validate a target differently from the
|
|
||||||
// other. Both forms are filled from one: the edit form with the
|
|
||||||
// stored values, and a refused form with the values submitted.
|
|
||||||
type targetFormInput struct {
|
|
||||||
// Name is the target's name.
|
|
||||||
Name string
|
|
||||||
// Type is the type chosen on the add target form. The edit form
|
|
||||||
// has none: a target's stored type decides.
|
|
||||||
Type database.TargetType
|
|
||||||
// URL is the destination for an HTTP target and the webhook URL
|
|
||||||
// for a Slack target.
|
|
||||||
URL string
|
|
||||||
// Headers is an HTTP target's headers, one "Name: value" per
|
|
||||||
// line.
|
|
||||||
Headers string
|
|
||||||
// Timeout is an HTTP target's per-request timeout in seconds.
|
|
||||||
Timeout string
|
|
||||||
// ForwardQuery is an HTTP target's checkbox that passes each
|
|
||||||
// event's query string on to it.
|
|
||||||
ForwardQuery bool
|
|
||||||
// MaxRetries is an HTTP or Slack target's max_retries.
|
|
||||||
MaxRetries string
|
|
||||||
// Expiry is a database (archive) target's row expiry.
|
|
||||||
Expiry string
|
|
||||||
// Rotation is a database (archive) target's rotation.
|
|
||||||
Rotation string
|
|
||||||
}
|
|
||||||
|
|
||||||
// targetFormInputFrom reads a target form from a request body. The
|
|
||||||
// body size cap is enforced by the MaxBodySize middleware, which runs
|
|
||||||
// before CSRF parses the form.
|
|
||||||
//
|
|
||||||
// Every field is read with PostFormValue, not FormValue. FormValue
|
|
||||||
// falls back to the query string, which would let
|
|
||||||
// `POST /hook/{id}/targets?url=https://hooks.slack.com/...`
|
|
||||||
// configure a target from a value the request line carries — and the
|
|
||||||
// request line, unlike the body, is what logs, proxies, Referer
|
|
||||||
// headers and error trackers record. The headers field is under the
|
|
||||||
// same rule and for the same reason: its values are authorization
|
|
||||||
// tokens.
|
|
||||||
func targetFormInputFrom(r *http.Request) targetFormInput {
|
|
||||||
return targetFormInput{
|
|
||||||
Name: r.PostFormValue("name"),
|
|
||||||
Type: database.TargetType(r.PostFormValue("type")),
|
|
||||||
URL: r.PostFormValue("url"),
|
|
||||||
Headers: r.PostFormValue("headers"),
|
|
||||||
Timeout: r.PostFormValue("timeout"),
|
|
||||||
ForwardQuery: r.PostFormValue("forward_query") != "",
|
|
||||||
MaxRetries: r.PostFormValue("max_retries"),
|
|
||||||
Expiry: r.PostFormValue("expiry"),
|
|
||||||
Rotation: r.PostFormValue("rotation"),
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// buildTargetConfig builds the JSON config string for a target from
|
|
||||||
// the submitted form values, or returns the message the form shows
|
|
||||||
// for a value it refuses. An error is the server's fault, not a
|
|
||||||
// refusal: the accepted configuration could not be encoded. Which
|
|
||||||
// fields of in apply depends on the target type; a type without a URL
|
|
||||||
// ignores any URL submitted.
|
|
||||||
func (h *Handlers) buildTargetConfig(
|
|
||||||
ctx context.Context,
|
|
||||||
targetType database.TargetType,
|
|
||||||
in targetFormInput,
|
|
||||||
) (string, string, error) {
|
|
||||||
switch targetType {
|
|
||||||
case database.TargetTypeHTTP:
|
|
||||||
return h.buildHTTPTargetConfig(ctx, in)
|
|
||||||
case database.TargetTypeSlack:
|
|
||||||
return h.buildSlackTargetConfig(ctx, in.URL)
|
|
||||||
case database.TargetTypeDatabase:
|
|
||||||
return buildDatabaseTargetConfig(in.Expiry, in.Rotation)
|
|
||||||
case database.TargetTypeLog:
|
|
||||||
return "", "", nil
|
|
||||||
default:
|
|
||||||
return "", "Invalid target type", nil
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// buildHTTPTargetConfig builds config JSON for an HTTP target: an
|
|
||||||
// SSRF-validated destination plus the optional headers, timeout and
|
|
||||||
// query string setting the delivery path honours.
|
|
||||||
func (h *Handlers) buildHTTPTargetConfig(
|
|
||||||
ctx context.Context,
|
|
||||||
in targetFormInput,
|
|
||||||
) (string, string, error) {
|
|
||||||
errMsg := h.validateTargetURL(
|
|
||||||
ctx, in.URL, "URL is required for HTTP targets",
|
|
||||||
)
|
|
||||||
if errMsg != "" {
|
|
||||||
return "", errMsg, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
headers, err := delivery.ParseTargetHeaders(in.Headers)
|
|
||||||
if err != nil {
|
|
||||||
return "", fmt.Sprintf("Invalid headers: %v", err), nil
|
|
||||||
}
|
|
||||||
|
|
||||||
timeout, err := delivery.ParseTargetTimeout(in.Timeout)
|
|
||||||
if err != nil {
|
|
||||||
return "", fmt.Sprintf("Invalid timeout: %v", err), nil
|
|
||||||
}
|
|
||||||
|
|
||||||
configJSON, err := marshalTargetConfig(delivery.HTTPTargetConfig{
|
|
||||||
URL: in.URL,
|
|
||||||
Headers: headers,
|
|
||||||
Timeout: timeout,
|
|
||||||
ForwardQuery: in.ForwardQuery,
|
|
||||||
})
|
|
||||||
|
|
||||||
return configJSON, "", err
|
|
||||||
}
|
|
||||||
|
|
||||||
// buildSlackTargetConfig builds config JSON for a Slack target,
|
|
||||||
// whose whole configuration is one SSRF-validated webhook URL.
|
|
||||||
func (h *Handlers) buildSlackTargetConfig(
|
|
||||||
ctx context.Context,
|
|
||||||
targetURL string,
|
|
||||||
) (string, string, error) {
|
|
||||||
errMsg := h.validateTargetURL(
|
|
||||||
ctx, targetURL,
|
|
||||||
"Webhook URL is required for Slack targets",
|
|
||||||
)
|
|
||||||
if errMsg != "" {
|
|
||||||
return "", errMsg, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
configJSON, err := marshalTargetConfig(delivery.SlackTargetConfig{
|
|
||||||
WebhookURL: targetURL,
|
|
||||||
})
|
|
||||||
|
|
||||||
return configJSON, "", err
|
|
||||||
}
|
|
||||||
|
|
||||||
// validateTargetURL refuses an empty or SSRF-blocked destination,
|
|
||||||
// returning the message the form shows, or "" when the destination
|
|
||||||
// is accepted. missingMsg is the message for no URL at all.
|
|
||||||
//
|
|
||||||
// It is the single point at which a user-supplied destination enters
|
|
||||||
// the SSRF guard, on create and on edit alike. An edit path that
|
|
||||||
// reached storage without passing through here would reopen the hole
|
|
||||||
// the guard closes.
|
|
||||||
func (h *Handlers) validateTargetURL(
|
|
||||||
ctx context.Context,
|
|
||||||
targetURL, missingMsg string,
|
|
||||||
) string {
|
|
||||||
if targetURL == "" {
|
|
||||||
return missingMsg
|
|
||||||
}
|
|
||||||
|
|
||||||
err := h.ssrf.ValidateTargetURL(ctx, targetURL)
|
|
||||||
if err != nil {
|
|
||||||
// The submitted URL can be a credential (a Slack
|
|
||||||
// incoming webhook URL is a bearer token), so the log
|
|
||||||
// records only its scheme and host.
|
|
||||||
h.log.Warn(
|
|
||||||
"target URL blocked by SSRF protection",
|
|
||||||
"url", delivery.MaskURL(targetURL),
|
|
||||||
"error", err,
|
|
||||||
)
|
|
||||||
|
|
||||||
msg := "Invalid target URL: " + err.Error()
|
|
||||||
|
|
||||||
// Only a private or reserved address's refusal says how
|
|
||||||
// to allow it. Other refusals never do: link-local, the
|
|
||||||
// unspecified addresses and the unconditional metadata
|
|
||||||
// addresses cannot be opened, and the default
|
|
||||||
// blocklist's public addresses, which listing does open,
|
|
||||||
// hand out credentials.
|
|
||||||
if errors.Is(err, delivery.ErrBlockedPrivateOrReservedIP) {
|
|
||||||
msg += ". Private and reserved addresses are refused " +
|
|
||||||
"by default; the server's ALLOWED_EGRESS_CIDRS " +
|
|
||||||
"setting allows named networks (see \"Allowing " +
|
|
||||||
"egress to your own network\" in the README)."
|
|
||||||
}
|
|
||||||
|
|
||||||
return msg
|
|
||||||
}
|
|
||||||
|
|
||||||
return ""
|
|
||||||
}
|
|
||||||
|
|
||||||
// marshalTargetConfig serialises a target configuration for storage.
|
|
||||||
func marshalTargetConfig(cfg any) (string, error) {
|
|
||||||
configBytes, err := json.Marshal(cfg)
|
|
||||||
if err != nil {
|
|
||||||
return "", err
|
|
||||||
}
|
|
||||||
|
|
||||||
return string(configBytes), nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// buildDatabaseTargetConfig builds config JSON for a database
|
|
||||||
// (archive) target. The optional expiry and rotation are validated
|
|
||||||
// here, at creation time, so a bad value is refused instead of
|
|
||||||
// failing every subsequent delivery. Each is stored only when set,
|
|
||||||
// and with neither the config is empty (the keep-forever, one-file
|
|
||||||
// default).
|
|
||||||
func buildDatabaseTargetConfig(
|
|
||||||
expiry, rotation string,
|
|
||||||
) (string, string, error) {
|
|
||||||
expiry = strings.TrimSpace(expiry)
|
|
||||||
|
|
||||||
err := delivery.ValidateArchiveExpiry(expiry)
|
|
||||||
if err != nil {
|
|
||||||
return "", fmt.Sprintf("Invalid archive expiry: %v", err), nil
|
|
||||||
}
|
|
||||||
|
|
||||||
err = delivery.ValidateArchiveRotation(rotation)
|
|
||||||
if err != nil {
|
|
||||||
return "", fmt.Sprintf("Invalid archive rotation: %v", err), nil
|
|
||||||
}
|
|
||||||
|
|
||||||
cfg := map[string]any{}
|
|
||||||
|
|
||||||
if expiry != "" {
|
|
||||||
cfg["expiry"] = expiry
|
|
||||||
}
|
|
||||||
|
|
||||||
if rotation != "" {
|
|
||||||
cfg["rotation"] = rotation
|
|
||||||
}
|
|
||||||
|
|
||||||
if len(cfg) == 0 {
|
|
||||||
return "", "", nil
|
|
||||||
}
|
|
||||||
|
|
||||||
configJSON, err := marshalTargetConfig(cfg)
|
|
||||||
|
|
||||||
return configJSON, "", err
|
|
||||||
}
|
|
||||||
@@ -1,31 +0,0 @@
|
|||||||
package handlers
|
|
||||||
|
|
||||||
import (
|
|
||||||
"net/http"
|
|
||||||
|
|
||||||
"sneak.berlin/go/webhooker/internal/database"
|
|
||||||
)
|
|
||||||
|
|
||||||
// HandleTargetDelete handles deleting a target. A deleted
|
|
||||||
// database target's archive writer is evicted and its handle
|
|
||||||
// closed; the archive file is left on disk.
|
|
||||||
func (h *Handlers) HandleTargetDelete() http.HandlerFunc {
|
|
||||||
return h.deleteChildResource(
|
|
||||||
"targetID", &database.Target{},
|
|
||||||
"failed to delete target",
|
|
||||||
h.evictTargetArchiveWriter,
|
|
||||||
targetDeleted,
|
|
||||||
)
|
|
||||||
}
|
|
||||||
|
|
||||||
// evictTargetArchiveWriter is evictArchiveWriter for one deleted
|
|
||||||
// target, and leaves its archive file on disk for the same reason.
|
|
||||||
// A target that is not a database target has no writer, and
|
|
||||||
// evicting it does nothing.
|
|
||||||
func (h *Handlers) evictTargetArchiveWriter(targetID string) {
|
|
||||||
if h.archives == nil {
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
h.archives.EvictTarget(targetID)
|
|
||||||
}
|
|
||||||
@@ -81,7 +81,6 @@ func (h *Handlers) HandleTargetEdit() http.HandlerFunc {
|
|||||||
URL: cfg.URL,
|
URL: cfg.URL,
|
||||||
Headers: cfg.Headers,
|
Headers: cfg.Headers,
|
||||||
Timeout: cfg.Timeout,
|
Timeout: cfg.Timeout,
|
||||||
ForwardQuery: cfg.ForwardQuery,
|
|
||||||
MaxRetries: strconv.Itoa(target.MaxRetries),
|
MaxRetries: strconv.Itoa(target.MaxRetries),
|
||||||
Expiry: cfg.Expiry,
|
Expiry: cfg.Expiry,
|
||||||
Rotation: cfg.Rotation,
|
Rotation: cfg.Rotation,
|
||||||
|
|||||||
@@ -442,59 +442,6 @@ func TestHandleTargetEdit_CallsTheDatabaseTypeArchive(t *testing.T) {
|
|||||||
assert.Contains(t, page, `class="label">Archive rotation</label>`)
|
assert.Contains(t, page, `class="label">Archive rotation</label>`)
|
||||||
}
|
}
|
||||||
|
|
||||||
// TestHandleTarget_ForwardQuery covers the HTTP target's setting that
|
|
||||||
// passes each event's query string on to it: the add target form
|
|
||||||
// stores it checked, the edit form starts with it checked and turns it
|
|
||||||
// off when saved unchecked, and both forms come back with it checked
|
|
||||||
// when refused.
|
|
||||||
func TestHandleTarget_ForwardQuery(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
const checkbox = `name="forward_query" value="on" checked`
|
|
||||||
|
|
||||||
env := setupSourceTest(t)
|
|
||||||
webhook := seedWebhookWithRetention(t, env.db, 30)
|
|
||||||
targetsPath := "/hook/" + webhook.ID + "/targets"
|
|
||||||
|
|
||||||
form := url.Values{}
|
|
||||||
form.Set("name", "forwarding")
|
|
||||||
form.Set("type", string(database.TargetTypeHTTP))
|
|
||||||
form.Set("url", editOriginalURL)
|
|
||||||
form.Set("forward_query", "on")
|
|
||||||
|
|
||||||
w := serveTarget(env, http.MethodPost, targetsPath, form)
|
|
||||||
require.Equal(t, http.StatusSeeOther, w.Code, w.Body.String())
|
|
||||||
|
|
||||||
targets := targetsForWebhook(t, env.db, webhook.ID)
|
|
||||||
require.Len(t, targets, 1)
|
|
||||||
assert.True(t, storedHTTPConfig(t, env, targets[0].ID).ForwardQuery)
|
|
||||||
|
|
||||||
w = serveTarget(
|
|
||||||
env, http.MethodGet, targetsPath+"/"+targets[0].ID+"/edit", nil,
|
|
||||||
)
|
|
||||||
require.Equal(t, http.StatusOK, w.Code)
|
|
||||||
assert.Contains(t, w.Body.String(), checkbox)
|
|
||||||
|
|
||||||
edit := editForm(editOriginalURL, "", "")
|
|
||||||
|
|
||||||
w = submitTargetEdit(env, webhook.ID, targets[0].ID, edit)
|
|
||||||
require.Equal(t, http.StatusSeeOther, w.Code, w.Body.String())
|
|
||||||
assert.False(t, storedHTTPConfig(t, env, targets[0].ID).ForwardQuery)
|
|
||||||
|
|
||||||
edit.Set("url", editBlockedURL)
|
|
||||||
edit.Set("forward_query", "on")
|
|
||||||
|
|
||||||
w = submitTargetEdit(env, webhook.ID, targets[0].ID, edit)
|
|
||||||
require.Equal(t, http.StatusBadRequest, w.Code)
|
|
||||||
assert.Contains(t, w.Body.String(), checkbox)
|
|
||||||
|
|
||||||
form.Set("url", editBlockedURL)
|
|
||||||
|
|
||||||
w = serveTarget(env, http.MethodPost, targetsPath, form)
|
|
||||||
require.Equal(t, http.StatusBadRequest, w.Code)
|
|
||||||
assert.Contains(t, w.Body.String(), "data-forward-query")
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestHandleTargetEditSubmit_Rejects covers every submission that
|
// TestHandleTargetEditSubmit_Rejects covers every submission that
|
||||||
// must not reach storage.
|
// must not reach storage.
|
||||||
//
|
//
|
||||||
|
|||||||
@@ -1,35 +0,0 @@
|
|||||||
package handlers
|
|
||||||
|
|
||||||
import (
|
|
||||||
"net/http"
|
|
||||||
|
|
||||||
"sneak.berlin/go/webhooker/internal/database"
|
|
||||||
)
|
|
||||||
|
|
||||||
// HandleTargetToggle handles toggling a target's active state.
|
|
||||||
func (h *Handlers) HandleTargetToggle() http.HandlerFunc {
|
|
||||||
return h.toggleChildResource(
|
|
||||||
"targetID",
|
|
||||||
func(webhookID, childID string) (bool, error) {
|
|
||||||
var tgt database.Target
|
|
||||||
|
|
||||||
err := h.db.DB().Where(
|
|
||||||
"id = ? AND webhook_id = ?",
|
|
||||||
childID, webhookID,
|
|
||||||
).First(&tgt).Error
|
|
||||||
if err != nil {
|
|
||||||
return false, err
|
|
||||||
}
|
|
||||||
|
|
||||||
// Only the active column: saving the whole row would
|
|
||||||
// write back the name and settings read above over an
|
|
||||||
// edit saved since.
|
|
||||||
active := !tgt.Active
|
|
||||||
|
|
||||||
return active, h.db.DB().Model(&tgt).
|
|
||||||
Update("active", active).Error
|
|
||||||
},
|
|
||||||
"failed to toggle target",
|
|
||||||
targetActivated, targetDeactivated,
|
|
||||||
)
|
|
||||||
}
|
|
||||||
@@ -230,7 +230,6 @@ type eventSource struct {
|
|||||||
WebhookID string
|
WebhookID string
|
||||||
EntrypointID string
|
EntrypointID string
|
||||||
Method string
|
Method string
|
||||||
RawQuery string
|
|
||||||
HeadersJSON string
|
HeadersJSON string
|
||||||
ContentType string
|
ContentType string
|
||||||
Body []byte
|
Body []byte
|
||||||
@@ -246,7 +245,6 @@ func (s eventSource) event() *database.Event {
|
|||||||
WebhookID: s.WebhookID,
|
WebhookID: s.WebhookID,
|
||||||
EntrypointID: s.EntrypointID,
|
EntrypointID: s.EntrypointID,
|
||||||
Method: s.Method,
|
Method: s.Method,
|
||||||
RawQuery: s.RawQuery,
|
|
||||||
Headers: s.HeadersJSON,
|
Headers: s.HeadersJSON,
|
||||||
Body: string(s.Body),
|
Body: string(s.Body),
|
||||||
BodyBytes: int64(len(s.Body)),
|
BodyBytes: int64(len(s.Body)),
|
||||||
@@ -266,7 +264,6 @@ func requestEventSource(
|
|||||||
WebhookID: entrypoint.WebhookID,
|
WebhookID: entrypoint.WebhookID,
|
||||||
EntrypointID: entrypoint.ID,
|
EntrypointID: entrypoint.ID,
|
||||||
Method: r.Method,
|
Method: r.Method,
|
||||||
RawQuery: r.URL.RawQuery,
|
|
||||||
HeadersJSON: string(headersJSON),
|
HeadersJSON: string(headersJSON),
|
||||||
ContentType: r.Header.Get("Content-Type"),
|
ContentType: r.Header.Get("Content-Type"),
|
||||||
Body: body,
|
Body: body,
|
||||||
@@ -444,7 +441,6 @@ func buildDeliveryTasks(
|
|||||||
TargetConfig: targets[i].Config,
|
TargetConfig: targets[i].Config,
|
||||||
MaxRetries: targets[i].MaxRetries,
|
MaxRetries: targets[i].MaxRetries,
|
||||||
Method: event.Method,
|
Method: event.Method,
|
||||||
RawQuery: event.RawQuery,
|
|
||||||
Headers: event.Headers,
|
Headers: event.Headers,
|
||||||
ContentType: event.ContentType,
|
ContentType: event.ContentType,
|
||||||
Body: bodyPtr,
|
Body: bodyPtr,
|
||||||
|
|||||||
@@ -1,263 +0,0 @@
|
|||||||
package handlers
|
|
||||||
|
|
||||||
import (
|
|
||||||
"context"
|
|
||||||
"net/http"
|
|
||||||
"strconv"
|
|
||||||
|
|
||||||
"github.com/google/uuid"
|
|
||||||
"sneak.berlin/go/webhooker/internal/database"
|
|
||||||
)
|
|
||||||
|
|
||||||
// HandleSourceCreate shows the form to create a new webhook.
|
|
||||||
func (h *Handlers) HandleSourceCreate() http.HandlerFunc {
|
|
||||||
return func(w http.ResponseWriter, r *http.Request) {
|
|
||||||
h.renderTemplate(
|
|
||||||
w, r, "sources_new.html",
|
|
||||||
newSourceFormData("", sourceFormInput{
|
|
||||||
RetentionDays: strconv.Itoa(
|
|
||||||
database.DefaultRetentionDays,
|
|
||||||
),
|
|
||||||
}),
|
|
||||||
)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// sourceFormInput carries the raw values of the new webhook form. A
|
|
||||||
// refused submission is shown again from it, so every value entered
|
|
||||||
// comes back, retention included.
|
|
||||||
type sourceFormInput struct {
|
|
||||||
Name string
|
|
||||||
Description string
|
|
||||||
RetentionDays string
|
|
||||||
// HTTPURL, when not empty, asks for an HTTP target with this
|
|
||||||
// destination.
|
|
||||||
HTTPURL string
|
|
||||||
// Archive asks for a database (archive) target, whose rows expire
|
|
||||||
// after ArchiveExpiry and whose files rotate by ArchiveRotation.
|
|
||||||
Archive bool
|
|
||||||
ArchiveExpiry string
|
|
||||||
ArchiveRotation string
|
|
||||||
}
|
|
||||||
|
|
||||||
// newSourceFormData builds the template data for the webhook creation
|
|
||||||
// form. It carries the retention default, which the form's help text
|
|
||||||
// names, from database.DefaultRetentionDays rather than a hardcoded
|
|
||||||
// copy of the same policy.
|
|
||||||
func newSourceFormData(
|
|
||||||
errMsg string, in sourceFormInput,
|
|
||||||
) map[string]any {
|
|
||||||
return map[string]any{
|
|
||||||
tmplKeyError: errMsg,
|
|
||||||
"Form": in,
|
|
||||||
"DefaultRetentionDays": database.DefaultRetentionDays,
|
|
||||||
tmplKeyArchiveExpiryChoices: archiveExpiryOptions(
|
|
||||||
in.ArchiveExpiry,
|
|
||||||
),
|
|
||||||
tmplKeyArchiveRotationChoices: archiveRotationOptions(
|
|
||||||
in.ArchiveRotation,
|
|
||||||
),
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// HandleSourceCreateSubmit handles the webhook creation form
|
|
||||||
// submission.
|
|
||||||
func (h *Handlers) HandleSourceCreateSubmit() http.HandlerFunc {
|
|
||||||
return func(w http.ResponseWriter, r *http.Request) {
|
|
||||||
userID, ok := h.getUserID(r)
|
|
||||||
if !ok {
|
|
||||||
http.Redirect(
|
|
||||||
w, r, "/pages/login", http.StatusSeeOther,
|
|
||||||
)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
// The body size cap is enforced by the MaxBodySize
|
|
||||||
// middleware, which runs before CSRF parses the form.
|
|
||||||
err := r.ParseForm()
|
|
||||||
if err != nil {
|
|
||||||
h.renderError(w, r, http.StatusBadRequest)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
in := sourceFormInput{
|
|
||||||
Name: r.PostFormValue("name"),
|
|
||||||
Description: r.PostFormValue("description"),
|
|
||||||
RetentionDays: r.PostFormValue("retention_days"),
|
|
||||||
HTTPURL: r.PostFormValue("http_url"),
|
|
||||||
Archive: r.PostFormValue("archive") != "",
|
|
||||||
ArchiveExpiry: r.PostFormValue("archive_expiry"),
|
|
||||||
ArchiveRotation: r.PostFormValue("archive_rotation"),
|
|
||||||
}
|
|
||||||
|
|
||||||
refuse := func(errMsg string) {
|
|
||||||
h.renderTemplateStatus(
|
|
||||||
w, r, "sources_new.html",
|
|
||||||
newSourceFormData(errMsg, in),
|
|
||||||
http.StatusBadRequest,
|
|
||||||
)
|
|
||||||
}
|
|
||||||
|
|
||||||
if in.Name == "" {
|
|
||||||
refuse("Name is required")
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
retentionDays, errMsg := parseRetentionDays(
|
|
||||||
in.RetentionDays, database.DefaultRetentionDays,
|
|
||||||
)
|
|
||||||
if errMsg != "" {
|
|
||||||
refuse(errMsg)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
targets, errMsg, err := h.newWebhookTargets(r.Context(), in)
|
|
||||||
if err != nil {
|
|
||||||
h.serverError(w, r, "failed to encode target config", err)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
if errMsg != "" {
|
|
||||||
refuse(errMsg)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
h.createWebhookWithEntrypoint(w, r, &database.Webhook{
|
|
||||||
UserID: userID,
|
|
||||||
Name: in.Name,
|
|
||||||
Description: in.Description,
|
|
||||||
RetentionDays: retentionDays,
|
|
||||||
}, targets)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// newWebhookTargets validates the targets the new webhook form asks
|
|
||||||
// for and returns the rows to create with the webhook, or the message
|
|
||||||
// the form shows for the first one it refuses. A filled-in HTTP URL
|
|
||||||
// asks for an HTTP target named "HTTP", and the archive checkbox for a
|
|
||||||
// database target named "Archive". Each goes through newTarget, as on
|
|
||||||
// the webhook page's add target form. The rows have no WebhookID yet:
|
|
||||||
// the webhook has no ID until it is created.
|
|
||||||
func (h *Handlers) newWebhookTargets(
|
|
||||||
ctx context.Context,
|
|
||||||
in sourceFormInput,
|
|
||||||
) ([]*database.Target, string, error) {
|
|
||||||
var requested []targetFormInput
|
|
||||||
|
|
||||||
if in.HTTPURL != "" {
|
|
||||||
requested = append(requested, targetFormInput{
|
|
||||||
Name: "HTTP",
|
|
||||||
Type: database.TargetTypeHTTP,
|
|
||||||
URL: in.HTTPURL,
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
if in.Archive {
|
|
||||||
requested = append(requested, targetFormInput{
|
|
||||||
Name: "Archive",
|
|
||||||
Type: database.TargetTypeDatabase,
|
|
||||||
Expiry: in.ArchiveExpiry,
|
|
||||||
Rotation: in.ArchiveRotation,
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
targets := make([]*database.Target, 0, len(requested))
|
|
||||||
|
|
||||||
for _, form := range requested {
|
|
||||||
target, errMsg, err := h.newTarget(ctx, "", form)
|
|
||||||
if err != nil || errMsg != "" {
|
|
||||||
return nil, errMsg, err
|
|
||||||
}
|
|
||||||
|
|
||||||
targets = append(targets, target)
|
|
||||||
}
|
|
||||||
|
|
||||||
return targets, "", nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// createWebhookWithEntrypoint creates a webhook, its default
|
|
||||||
// entrypoint and the given targets in a transaction.
|
|
||||||
func (h *Handlers) createWebhookWithEntrypoint(
|
|
||||||
w http.ResponseWriter,
|
|
||||||
r *http.Request,
|
|
||||||
webhook *database.Webhook,
|
|
||||||
targets []*database.Target,
|
|
||||||
) {
|
|
||||||
err := h.commitWebhook(webhook, targets)
|
|
||||||
if err != nil {
|
|
||||||
h.serverError(w, r, "failed to create webhook", err)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
err = h.dbMgr.CreateDB(webhook.ID)
|
|
||||||
if err != nil {
|
|
||||||
h.log.Error(
|
|
||||||
"failed to create webhook event database",
|
|
||||||
"webhook_id", webhook.ID, "error", err,
|
|
||||||
)
|
|
||||||
}
|
|
||||||
|
|
||||||
h.log.Info("webhook created",
|
|
||||||
"webhook_id", webhook.ID,
|
|
||||||
"name", webhook.Name, "user_id", webhook.UserID,
|
|
||||||
)
|
|
||||||
|
|
||||||
http.Redirect(
|
|
||||||
w, r, withNotice("/hook/"+webhook.ID, webhookCreated),
|
|
||||||
http.StatusSeeOther,
|
|
||||||
)
|
|
||||||
}
|
|
||||||
|
|
||||||
// commitWebhook creates a webhook, its default entrypoint and the
|
|
||||||
// given targets in a transaction. Returns an error on failure (rolls
|
|
||||||
// back).
|
|
||||||
func (h *Handlers) commitWebhook(
|
|
||||||
webhook *database.Webhook,
|
|
||||||
targets []*database.Target,
|
|
||||||
) error {
|
|
||||||
tx := h.db.DB().Begin()
|
|
||||||
if tx.Error != nil {
|
|
||||||
return tx.Error
|
|
||||||
}
|
|
||||||
|
|
||||||
err := tx.Create(webhook).Error
|
|
||||||
if err != nil {
|
|
||||||
tx.Rollback()
|
|
||||||
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
|
|
||||||
entrypoint := &database.Entrypoint{
|
|
||||||
WebhookID: webhook.ID,
|
|
||||||
Path: uuid.New().String(),
|
|
||||||
Description: "Default entrypoint",
|
|
||||||
Active: true,
|
|
||||||
}
|
|
||||||
|
|
||||||
err = tx.Create(entrypoint).Error
|
|
||||||
if err != nil {
|
|
||||||
tx.Rollback()
|
|
||||||
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
|
|
||||||
for _, target := range targets {
|
|
||||||
target.WebhookID = webhook.ID
|
|
||||||
|
|
||||||
err = tx.Create(target).Error
|
|
||||||
if err != nil {
|
|
||||||
tx.Rollback()
|
|
||||||
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
return tx.Commit().Error
|
|
||||||
}
|
|
||||||
@@ -1,170 +0,0 @@
|
|||||||
package handlers
|
|
||||||
|
|
||||||
import (
|
|
||||||
"errors"
|
|
||||||
"net/http"
|
|
||||||
|
|
||||||
"github.com/go-chi/chi"
|
|
||||||
"sneak.berlin/go/webhooker/internal/database"
|
|
||||||
)
|
|
||||||
|
|
||||||
// HandleSourceDelete handles webhook deletion.
|
|
||||||
func (h *Handlers) HandleSourceDelete() http.HandlerFunc {
|
|
||||||
return func(w http.ResponseWriter, r *http.Request) {
|
|
||||||
userID, ok := h.getUserID(r)
|
|
||||||
if !ok {
|
|
||||||
http.Redirect(
|
|
||||||
w, r, "/pages/login", http.StatusSeeOther,
|
|
||||||
)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
sourceID := chi.URLParam(r, "sourceID")
|
|
||||||
|
|
||||||
var webhook database.Webhook
|
|
||||||
|
|
||||||
err := h.db.DB().Where(
|
|
||||||
"id = ? AND user_id = ?", sourceID, userID,
|
|
||||||
).First(&webhook).Error
|
|
||||||
if err != nil {
|
|
||||||
h.renderError(w, r, http.StatusNotFound)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
h.deleteWebhookResources(w, r, webhook, userID)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// The messages deleteWebhookResources logs when a file of the event
|
|
||||||
// database cannot be removed: the database file itself, or only a
|
|
||||||
// sidecar once the database file is gone.
|
|
||||||
const (
|
|
||||||
eventDBLeftMsg = "webhook deleted, but its event database file is " +
|
|
||||||
"still on disk; remove it by hand"
|
|
||||||
sidecarLeftMsg = "webhook deleted and its events are gone, but a " +
|
|
||||||
"-wal or -shm sidecar of its event database is " +
|
|
||||||
"still on disk; remove it by hand"
|
|
||||||
)
|
|
||||||
|
|
||||||
// deleteWebhookResources soft-deletes config and hard-deletes
|
|
||||||
// the per-webhook event database.
|
|
||||||
func (h *Handlers) deleteWebhookResources(
|
|
||||||
w http.ResponseWriter,
|
|
||||||
r *http.Request,
|
|
||||||
webhook database.Webhook,
|
|
||||||
userID string,
|
|
||||||
) {
|
|
||||||
// The configuration delete commits before the event database
|
|
||||||
// is touched. No transaction spans the main database and the
|
|
||||||
// filesystem, so one side has to go first: committing the
|
|
||||||
// configuration first means a later failure leaves an unused
|
|
||||||
// event database file on disk, while removing the event
|
|
||||||
// database first would mean a failed commit destroys the
|
|
||||||
// history of a webhook that still exists. A leftover file can
|
|
||||||
// be removed by hand; deleted history cannot be recovered.
|
|
||||||
err := h.commitWebhookDeletion(&webhook)
|
|
||||||
if err != nil {
|
|
||||||
h.serverError(w, r, "failed to delete webhook", err)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
h.log.Info(
|
|
||||||
"webhook deleted",
|
|
||||||
"webhook_id", webhook.ID,
|
|
||||||
"user_id", userID,
|
|
||||||
)
|
|
||||||
|
|
||||||
// Release the delivery engine's per-webhook archiving state
|
|
||||||
// so a deleted webhook's archive writer (and any handle open
|
|
||||||
// within its debounce window) does not linger for the
|
|
||||||
// process lifetime. The archive file itself is deliberately
|
|
||||||
// left on disk; see evictArchiveWriter.
|
|
||||||
h.evictArchiveWriter(webhook.ID)
|
|
||||||
|
|
||||||
err = h.dbMgr.DeleteDB(webhook.ID)
|
|
||||||
if err != nil {
|
|
||||||
// The configuration is committed, so the webhook is gone,
|
|
||||||
// but a file of its event database is still on disk with
|
|
||||||
// nothing referencing it. Report the failure rather than
|
|
||||||
// redirecting as though everything succeeded: the file
|
|
||||||
// needs removing by hand, and the logged error names it.
|
|
||||||
// When only a sidecar is left, the events are already
|
|
||||||
// gone, and the message must not suggest they survive.
|
|
||||||
msg := eventDBLeftMsg
|
|
||||||
if errors.Is(err, database.ErrSidecarNotRemoved) {
|
|
||||||
msg = sidecarLeftMsg
|
|
||||||
}
|
|
||||||
|
|
||||||
h.serverError(w, r, msg, err)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
http.Redirect(
|
|
||||||
w, r, withNotice("/hooks", webhookDeleted), http.StatusSeeOther,
|
|
||||||
)
|
|
||||||
}
|
|
||||||
|
|
||||||
// commitWebhookDeletion soft-deletes a webhook's entrypoints,
|
|
||||||
// targets and the webhook row in one transaction. Every
|
|
||||||
// statement is checked and any failure rolls the whole
|
|
||||||
// transaction back, so a caller that gets an error knows the
|
|
||||||
// configuration is untouched and the event database must be
|
|
||||||
// left alone.
|
|
||||||
func (h *Handlers) commitWebhookDeletion(
|
|
||||||
webhook *database.Webhook,
|
|
||||||
) error {
|
|
||||||
tx := h.db.DB().Begin()
|
|
||||||
if tx.Error != nil {
|
|
||||||
return tx.Error
|
|
||||||
}
|
|
||||||
|
|
||||||
err := tx.Where(
|
|
||||||
"webhook_id = ?", webhook.ID,
|
|
||||||
).Delete(&database.Entrypoint{}).Error
|
|
||||||
if err != nil {
|
|
||||||
tx.Rollback()
|
|
||||||
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
|
|
||||||
err = tx.Where(
|
|
||||||
"webhook_id = ?", webhook.ID,
|
|
||||||
).Delete(&database.Target{}).Error
|
|
||||||
if err != nil {
|
|
||||||
tx.Rollback()
|
|
||||||
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
|
|
||||||
err = tx.Delete(webhook).Error
|
|
||||||
if err != nil {
|
|
||||||
tx.Rollback()
|
|
||||||
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
|
|
||||||
return tx.Commit().Error
|
|
||||||
}
|
|
||||||
|
|
||||||
// evictArchiveWriter asks the delivery engine to drop the cached
|
|
||||||
// archive writers of a webhook's database targets, closing their
|
|
||||||
// archive file handles.
|
|
||||||
//
|
|
||||||
// The archive database files are NOT deleted. Unlike the event
|
|
||||||
// database — which is per-webhook working storage and is
|
|
||||||
// hard-deleted with the webhook — an archive is explicitly
|
|
||||||
// long-term storage that an operator may want to keep or move
|
|
||||||
// away for offline retention. Destroying it as a side effect of
|
|
||||||
// deleting a webhook would be a surprising and unrecoverable
|
|
||||||
// data loss, so the file is left for the operator to handle.
|
|
||||||
func (h *Handlers) evictArchiveWriter(webhookID string) {
|
|
||||||
if h.archives == nil {
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
h.archives.EvictWebhook(webhookID)
|
|
||||||
}
|
|
||||||
@@ -1,133 +0,0 @@
|
|||||||
package handlers
|
|
||||||
|
|
||||||
import (
|
|
||||||
"net/http"
|
|
||||||
"time"
|
|
||||||
|
|
||||||
"github.com/go-chi/chi"
|
|
||||||
"sneak.berlin/go/webhooker/internal/database"
|
|
||||||
"sneak.berlin/go/webhooker/internal/reqtls"
|
|
||||||
)
|
|
||||||
|
|
||||||
// HandleSourceDetail shows details for a specific webhook.
|
|
||||||
func (h *Handlers) HandleSourceDetail() http.HandlerFunc {
|
|
||||||
return func(w http.ResponseWriter, r *http.Request) {
|
|
||||||
userID, ok := h.getUserID(r)
|
|
||||||
if !ok {
|
|
||||||
http.Redirect(
|
|
||||||
w, r, "/pages/login", http.StatusSeeOther,
|
|
||||||
)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
sourceID := chi.URLParam(r, "sourceID")
|
|
||||||
|
|
||||||
var webhook database.Webhook
|
|
||||||
|
|
||||||
err := h.db.DB().Where(
|
|
||||||
"id = ? AND user_id = ?", sourceID, userID,
|
|
||||||
).First(&webhook).Error
|
|
||||||
if err != nil {
|
|
||||||
h.renderError(w, r, http.StatusNotFound)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
h.renderSourceDetail(w, r, webhook, targetFormInput{}, "")
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// renderSourceDetail loads and renders a source detail page. With a
|
|
||||||
// targetErr, it is the page shown again for a refused add target
|
|
||||||
// form: it answers 400, and the form opens on targetForm's type with
|
|
||||||
// its values and the message.
|
|
||||||
func (h *Handlers) renderSourceDetail(
|
|
||||||
w http.ResponseWriter,
|
|
||||||
r *http.Request,
|
|
||||||
webhook database.Webhook,
|
|
||||||
targetForm targetFormInput,
|
|
||||||
targetErr string,
|
|
||||||
) {
|
|
||||||
var entrypoints []database.Entrypoint
|
|
||||||
|
|
||||||
h.db.DB().Where(
|
|
||||||
"webhook_id = ?", webhook.ID,
|
|
||||||
).Find(&entrypoints)
|
|
||||||
|
|
||||||
var targets []database.Target
|
|
||||||
|
|
||||||
h.db.DB().Where(
|
|
||||||
"webhook_id = ?", webhook.ID,
|
|
||||||
).Find(&targets)
|
|
||||||
|
|
||||||
entrypointViews := NewEntrypointViews(entrypoints)
|
|
||||||
|
|
||||||
var events []RecentEventView
|
|
||||||
|
|
||||||
if h.dbMgr.DBExists(webhook.ID) {
|
|
||||||
webhookDB, err := h.dbMgr.GetDB(webhook.ID)
|
|
||||||
if err != nil {
|
|
||||||
h.serverError(w, r, "failed to get webhook database", err)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
events, err = loadRecentEvents(
|
|
||||||
webhookDB, webhook.ID, singleHTTPTargetID(targets),
|
|
||||||
)
|
|
||||||
if err != nil {
|
|
||||||
h.serverError(w, r, "failed to load recent events", err)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
err = addEntrypointEvents(
|
|
||||||
webhookDB, &webhook, entrypointViews, time.Now(),
|
|
||||||
)
|
|
||||||
if err != nil {
|
|
||||||
h.serverError(w, r, "failed to count entrypoint events", err)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
scheme := "http"
|
|
||||||
if reqtls.IsTLS(r) {
|
|
||||||
scheme = "https"
|
|
||||||
}
|
|
||||||
|
|
||||||
// The host is the client's Host header, unvalidated. It is
|
|
||||||
// inert only because source_detail.html renders BaseURL as
|
|
||||||
// text, inside a <code> element and in an entrypoint's delete
|
|
||||||
// prompt; putting it in an href or any other URL context
|
|
||||||
// needs it constrained first.
|
|
||||||
baseURL := scheme + "://" + r.Host
|
|
||||||
|
|
||||||
// The template calls Webhook methods, which take pointer
|
|
||||||
// receivers; html/template cannot address a value stored in a map.
|
|
||||||
data := map[string]any{
|
|
||||||
tmplKeyWebhook: &webhook,
|
|
||||||
// Targets are projected to a display-safe view: a
|
|
||||||
// target's stored config blob holds a credential, and it
|
|
||||||
// must never reach a template.
|
|
||||||
"Entrypoints": entrypointViews,
|
|
||||||
"Targets": h.targetRows(&webhook, targets),
|
|
||||||
"Events": events,
|
|
||||||
"BaseURL": baseURL,
|
|
||||||
"Stats": h.loadWebhookStats(webhook.ID, entrypoints, targets),
|
|
||||||
tmplKeyTargetForm: targetForm,
|
|
||||||
"TargetError": targetErr,
|
|
||||||
// The add target form's selects start on its expiry and
|
|
||||||
// rotation through Alpine, so no choice is selected here.
|
|
||||||
tmplKeyArchiveExpiryChoices: archiveExpiryChoices(),
|
|
||||||
tmplKeyArchiveRotationChoices: archiveRotationChoices(),
|
|
||||||
}
|
|
||||||
|
|
||||||
status := http.StatusOK
|
|
||||||
if targetErr != "" {
|
|
||||||
status = http.StatusBadRequest
|
|
||||||
}
|
|
||||||
|
|
||||||
h.renderTemplateStatus(w, r, "source_detail.html", data, status)
|
|
||||||
}
|
|
||||||
@@ -1,245 +0,0 @@
|
|||||||
package handlers
|
|
||||||
|
|
||||||
import (
|
|
||||||
"errors"
|
|
||||||
"net/http"
|
|
||||||
"strconv"
|
|
||||||
|
|
||||||
"github.com/go-chi/chi"
|
|
||||||
"sneak.berlin/go/webhooker/internal/database"
|
|
||||||
"sneak.berlin/go/webhooker/internal/delivery"
|
|
||||||
)
|
|
||||||
|
|
||||||
// HandleSourceEdit shows the form to edit a webhook.
|
|
||||||
func (h *Handlers) HandleSourceEdit() http.HandlerFunc {
|
|
||||||
return func(w http.ResponseWriter, r *http.Request) {
|
|
||||||
userID, ok := h.getUserID(r)
|
|
||||||
if !ok {
|
|
||||||
http.Redirect(
|
|
||||||
w, r, "/pages/login", http.StatusSeeOther,
|
|
||||||
)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
sourceID := chi.URLParam(r, "sourceID")
|
|
||||||
|
|
||||||
var webhook database.Webhook
|
|
||||||
|
|
||||||
err := h.db.DB().Where(
|
|
||||||
"id = ? AND user_id = ?", sourceID, userID,
|
|
||||||
).First(&webhook).Error
|
|
||||||
if err != nil {
|
|
||||||
h.renderError(w, r, http.StatusNotFound)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
h.renderWebhookEdit(
|
|
||||||
w, r, &webhook,
|
|
||||||
webhook.Name, webhook.Description,
|
|
||||||
strconv.Itoa(webhook.RetentionDays),
|
|
||||||
"", http.StatusOK,
|
|
||||||
)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// HandleSourceEditSubmit handles the webhook edit form
|
|
||||||
// submission.
|
|
||||||
func (h *Handlers) HandleSourceEditSubmit() http.HandlerFunc {
|
|
||||||
return func(w http.ResponseWriter, r *http.Request) {
|
|
||||||
userID, ok := h.getUserID(r)
|
|
||||||
if !ok {
|
|
||||||
http.Redirect(
|
|
||||||
w, r, "/pages/login", http.StatusSeeOther,
|
|
||||||
)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
sourceID := chi.URLParam(r, "sourceID")
|
|
||||||
|
|
||||||
h.renameMu.Lock()
|
|
||||||
defer h.renameMu.Unlock()
|
|
||||||
|
|
||||||
var webhook database.Webhook
|
|
||||||
|
|
||||||
err := h.db.DB().Where(
|
|
||||||
"id = ? AND user_id = ?", sourceID, userID,
|
|
||||||
).First(&webhook).Error
|
|
||||||
if err != nil {
|
|
||||||
h.renderError(w, r, http.StatusNotFound)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
// The body size cap is enforced by the MaxBodySize
|
|
||||||
// middleware, which runs before CSRF parses the form.
|
|
||||||
err = r.ParseForm()
|
|
||||||
if err != nil {
|
|
||||||
h.renderError(w, r, http.StatusBadRequest)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
h.applyWebhookEdit(w, r, &webhook)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// applyWebhookEdit validates and saves webhook edits. A refused save
|
|
||||||
// shows the edit form again with the values submitted and the reason.
|
|
||||||
func (h *Handlers) applyWebhookEdit(
|
|
||||||
w http.ResponseWriter,
|
|
||||||
r *http.Request,
|
|
||||||
webhook *database.Webhook,
|
|
||||||
) {
|
|
||||||
// The body size cap is enforced by the MaxBodySize middleware,
|
|
||||||
// which runs before CSRF parses the form.
|
|
||||||
name := r.PostFormValue("name")
|
|
||||||
description := r.PostFormValue("description")
|
|
||||||
retention := r.PostFormValue("retention_days")
|
|
||||||
|
|
||||||
if name == "" {
|
|
||||||
h.renderWebhookEdit(
|
|
||||||
w, r, webhook, name, description, retention,
|
|
||||||
"Name is required", http.StatusBadRequest,
|
|
||||||
)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
// An empty field falls back to the stored value, so submitting the
|
|
||||||
// form without touching retention leaves the policy alone.
|
|
||||||
retentionDays, errMsg := parseRetentionDays(
|
|
||||||
retention, webhook.RetentionDays,
|
|
||||||
)
|
|
||||||
if errMsg != "" {
|
|
||||||
h.renderWebhookEdit(
|
|
||||||
w, r, webhook, name, description, retention,
|
|
||||||
errMsg, http.StatusBadRequest,
|
|
||||||
)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
// edited is the webhook as the submission leaves it; webhook stays
|
|
||||||
// as stored, for the page shown again when the save is refused.
|
|
||||||
edited := *webhook
|
|
||||||
edited.Name = name
|
|
||||||
edited.Description = description
|
|
||||||
edited.RetentionDays = retentionDays
|
|
||||||
|
|
||||||
// A new name renames the archive files before it is saved (see
|
|
||||||
// delivery.Engine.Rename). If either step fails, the same targets'
|
|
||||||
// archives go back to the name that is still stored, without
|
|
||||||
// reading the main database again.
|
|
||||||
targets, err := h.renameWebhookArchives(
|
|
||||||
webhook.ID, webhook.Name, edited.Name,
|
|
||||||
)
|
|
||||||
if err == nil {
|
|
||||||
err = h.db.DB().Save(&edited).Error
|
|
||||||
}
|
|
||||||
|
|
||||||
if err != nil {
|
|
||||||
restoreErr := h.renameArchives(targets, webhook.Name)
|
|
||||||
if restoreErr != nil {
|
|
||||||
h.log.Error(
|
|
||||||
"failed to rename archives back",
|
|
||||||
"webhook_id", webhook.ID,
|
|
||||||
"error", restoreErr,
|
|
||||||
)
|
|
||||||
}
|
|
||||||
|
|
||||||
if errors.Is(err, delivery.ErrArchiveNameTaken) {
|
|
||||||
h.renderWebhookEdit(
|
|
||||||
w, r, webhook, name, description, retention,
|
|
||||||
"Not saved: "+err.Error()+
|
|
||||||
". Move that archive out of the data directory, "+
|
|
||||||
"its .db together with any -wal and -shm beside "+
|
|
||||||
"it, then save again.",
|
|
||||||
http.StatusConflict,
|
|
||||||
)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
h.serverError(w, r, "failed to update webhook", err)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
http.Redirect(
|
|
||||||
w, r, withNotice("/hook/"+webhook.ID, webhookSaved),
|
|
||||||
http.StatusSeeOther,
|
|
||||||
)
|
|
||||||
}
|
|
||||||
|
|
||||||
// renderWebhookEdit renders the webhook edit page for the webhook as
|
|
||||||
// stored, its form showing name, description and retentionDays, with
|
|
||||||
// an optional error message above it.
|
|
||||||
func (h *Handlers) renderWebhookEdit(
|
|
||||||
w http.ResponseWriter,
|
|
||||||
r *http.Request,
|
|
||||||
webhook *database.Webhook,
|
|
||||||
name, description, retentionDays, errMsg string,
|
|
||||||
status int,
|
|
||||||
) {
|
|
||||||
data := map[string]any{
|
|
||||||
tmplKeyWebhook: webhook,
|
|
||||||
tmplKeyError: errMsg,
|
|
||||||
"Name": name,
|
|
||||||
"Description": description,
|
|
||||||
"RetentionDays": retentionDays,
|
|
||||||
}
|
|
||||||
|
|
||||||
h.renderTemplateStatus(w, r, "source_edit.html", data, status)
|
|
||||||
}
|
|
||||||
|
|
||||||
// renameWebhookArchives renames the archive file of every database
|
|
||||||
// target of a webhook from the webhook name oldName to newName,
|
|
||||||
// keeping each target's own name. It does nothing when the name is
|
|
||||||
// unchanged. It returns the targets it read, so that a failed edit can
|
|
||||||
// move those same archives back with renameArchives.
|
|
||||||
func (h *Handlers) renameWebhookArchives(
|
|
||||||
webhookID, oldName, newName string,
|
|
||||||
) ([]database.Target, error) {
|
|
||||||
if h.archives == nil || oldName == newName {
|
|
||||||
return nil, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
var targets []database.Target
|
|
||||||
|
|
||||||
err := h.db.DB().
|
|
||||||
Where(
|
|
||||||
"webhook_id = ? AND type = ?",
|
|
||||||
webhookID, database.TargetTypeDatabase,
|
|
||||||
).
|
|
||||||
Find(&targets).Error
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
|
|
||||||
return targets, h.renameArchives(targets, newName)
|
|
||||||
}
|
|
||||||
|
|
||||||
// renameArchives renames the archive file of each of the given
|
|
||||||
// database targets to the webhook name webhookName, keeping each
|
|
||||||
// target's own name. It tries every target even after one fails, so
|
|
||||||
// that moving the archives back after a failed edit leaves none under
|
|
||||||
// the new name, and returns every failure joined.
|
|
||||||
func (h *Handlers) renameArchives(
|
|
||||||
targets []database.Target, webhookName string,
|
|
||||||
) error {
|
|
||||||
var errs []error
|
|
||||||
|
|
||||||
for i := range targets {
|
|
||||||
err := h.archives.Rename(
|
|
||||||
targets[i].ID, webhookName, targets[i].Name,
|
|
||||||
)
|
|
||||||
if err != nil {
|
|
||||||
errs = append(errs, err)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
return errors.Join(errs...)
|
|
||||||
}
|
|
||||||
@@ -1,178 +0,0 @@
|
|||||||
package handlers
|
|
||||||
|
|
||||||
import (
|
|
||||||
"fmt"
|
|
||||||
"net/http"
|
|
||||||
"time"
|
|
||||||
|
|
||||||
"sneak.berlin/go/webhooker/internal/database"
|
|
||||||
)
|
|
||||||
|
|
||||||
// WebhookListItem holds data for the webhook list view.
|
|
||||||
type WebhookListItem struct {
|
|
||||||
database.Webhook
|
|
||||||
|
|
||||||
EntrypointCount int
|
|
||||||
InactiveEntrypointCount int
|
|
||||||
TargetCount int
|
|
||||||
InactiveTargetCount int
|
|
||||||
|
|
||||||
// EventCount is how many events the webhook holds, LastEventAt
|
|
||||||
// when the newest arrived (nil before the first), and
|
|
||||||
// FailedLast24Hours how many of its deliveries failed in the last
|
|
||||||
// 24 hours. When the webhook's event database could not be read,
|
|
||||||
// EventsUnreadable is set and these three are not known.
|
|
||||||
EventCount int64
|
|
||||||
LastEventAt *time.Time
|
|
||||||
FailedLast24Hours int64
|
|
||||||
EventsUnreadable bool
|
|
||||||
}
|
|
||||||
|
|
||||||
// HandleSourceList shows a list of user's webhooks.
|
|
||||||
func (h *Handlers) HandleSourceList() http.HandlerFunc {
|
|
||||||
return func(w http.ResponseWriter, r *http.Request) {
|
|
||||||
userID, ok := h.getUserID(r)
|
|
||||||
if !ok {
|
|
||||||
http.Redirect(
|
|
||||||
w, r, "/pages/login", http.StatusSeeOther,
|
|
||||||
)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
var webhooks []database.Webhook
|
|
||||||
|
|
||||||
err := h.db.DB().Where(
|
|
||||||
"user_id = ?", userID,
|
|
||||||
).Order("created_at DESC").Find(&webhooks).Error
|
|
||||||
if err != nil {
|
|
||||||
h.serverError(w, r, "failed to list webhooks", err)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
items, err := h.buildWebhookListItems(webhooks)
|
|
||||||
if err != nil {
|
|
||||||
h.serverError(w, r, "failed to list webhooks", err)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
data := map[string]any{
|
|
||||||
"Webhooks": items,
|
|
||||||
}
|
|
||||||
|
|
||||||
h.renderTemplate(w, r, "sources_list.html", data)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// buildWebhookListItems builds the list's entry for each webhook. It
|
|
||||||
// fails when the main database cannot be read. A webhook whose event
|
|
||||||
// database cannot be read is marked on its own entry, and the error is
|
|
||||||
// logged.
|
|
||||||
func (h *Handlers) buildWebhookListItems(
|
|
||||||
webhooks []database.Webhook,
|
|
||||||
) ([]WebhookListItem, error) {
|
|
||||||
items := make([]WebhookListItem, len(webhooks))
|
|
||||||
since := time.Now().Add(-longWindow)
|
|
||||||
|
|
||||||
for i := range webhooks {
|
|
||||||
item := &items[i]
|
|
||||||
item.Webhook = webhooks[i]
|
|
||||||
|
|
||||||
var err error
|
|
||||||
|
|
||||||
item.EntrypointCount, item.InactiveEntrypointCount, err =
|
|
||||||
h.countWithInactive(&database.Entrypoint{}, item.ID)
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
|
|
||||||
item.TargetCount, item.InactiveTargetCount, err =
|
|
||||||
h.countWithInactive(&database.Target{}, item.ID)
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
|
|
||||||
// Opening an event database that does not exist would create
|
|
||||||
// it, and it would hold nothing to count.
|
|
||||||
if !h.dbMgr.DBExists(item.ID) {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
|
|
||||||
err = h.readListEventFigures(item, since)
|
|
||||||
if err != nil {
|
|
||||||
h.log.Error(
|
|
||||||
"failed to read webhook list figures",
|
|
||||||
"webhook_id", item.ID,
|
|
||||||
"error", err,
|
|
||||||
)
|
|
||||||
|
|
||||||
item.EventsUnreadable = true
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
return items, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// countWithInactive returns how many entrypoints or targets, as model
|
|
||||||
// says, a webhook has, and how many of them are inactive.
|
|
||||||
func (h *Handlers) countWithInactive(
|
|
||||||
model any, webhookID string,
|
|
||||||
) (int, int, error) {
|
|
||||||
var active []bool
|
|
||||||
|
|
||||||
err := h.db.DB().Model(model).
|
|
||||||
Where("webhook_id = ?", webhookID).
|
|
||||||
Pluck("active", &active).Error
|
|
||||||
if err != nil {
|
|
||||||
return 0, 0, fmt.Errorf(
|
|
||||||
"reading active flags of webhook %s: %w", webhookID, err,
|
|
||||||
)
|
|
||||||
}
|
|
||||||
|
|
||||||
inactive := 0
|
|
||||||
|
|
||||||
for _, a := range active {
|
|
||||||
if !a {
|
|
||||||
inactive++
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
return len(active), inactive, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// readListEventFigures fills in the figures the list shows from the
|
|
||||||
// webhook's event database, with the statistics pane's own queries:
|
|
||||||
// the event count and last arrival from the event totals row, and the
|
|
||||||
// deliveries that failed since the given time from the deliveries'
|
|
||||||
// status index.
|
|
||||||
func (h *Handlers) readListEventFigures(
|
|
||||||
item *WebhookListItem, since time.Time,
|
|
||||||
) error {
|
|
||||||
webhookDB, err := h.dbMgr.GetDB(item.ID)
|
|
||||||
if err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
|
|
||||||
var totals database.EventTotals
|
|
||||||
|
|
||||||
err = webhookDB.Take(&totals).Error
|
|
||||||
if err != nil {
|
|
||||||
return fmt.Errorf("reading event totals: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
item.EventCount = totals.Events - totals.EventsRemoved
|
|
||||||
item.LastEventAt = totals.LastEventAt
|
|
||||||
|
|
||||||
byTarget, err := finishedByTarget(webhookDB, since)
|
|
||||||
if err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
|
|
||||||
for _, f := range byTarget {
|
|
||||||
item.FailedLast24Hours += f.Failed
|
|
||||||
}
|
|
||||||
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
@@ -236,9 +236,9 @@ func TestLoginGuard_SemaphoreBoundsConcurrentVerifications(
|
|||||||
|
|
||||||
// rendezvousDeadlock is the deadlock guard described below.
|
// rendezvousDeadlock is the deadlock guard described below.
|
||||||
// It is orders of magnitude longer than any scheduling delay,
|
// It is orders of magnitude longer than any scheduling delay,
|
||||||
// so it never decides the result, and well inside the 90s
|
// so it never decides the result, and well inside script/test's
|
||||||
// package timeout of the Dockerfile's test phase, so a wedge
|
// 30s timeout, so a wedge fails on the assertion instead of
|
||||||
// fails on the assertion instead of blowing that timeout.
|
// blowing the package timeout.
|
||||||
rendezvousDeadlock = 5 * time.Second
|
rendezvousDeadlock = 5 * time.Second
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|||||||
@@ -131,9 +131,8 @@ const (
|
|||||||
//
|
//
|
||||||
// - Lines carrying an AUTHENTICATED operator's own input, which
|
// - Lines carrying an AUTHENTICATED operator's own input, which
|
||||||
// are not truncated at all: the webhook name on "webhook
|
// are not truncated at all: the webhook name on "webhook
|
||||||
// created" (internal/handlers/webhook_create.go) and the target
|
// created" and the target host on "target URL blocked by SSRF
|
||||||
// host on "target URL blocked by SSRF protection"
|
// protection" (both internal/handlers/source_management.go),
|
||||||
// (internal/handlers/target_create.go),
|
|
||||||
// and target_name in internal/delivery/engine.go and
|
// and target_name in internal/delivery/engine.go and
|
||||||
// target_http.go. Each is bounded only by the 1 MB form body
|
// target_http.go. Each is bounded only by the 1 MB form body
|
||||||
// cap, so a 100 KB name writes one line of roughly 600 KB.
|
// cap, so a 100 KB name writes one line of roughly 600 KB.
|
||||||
|
|||||||
@@ -98,25 +98,20 @@ func TestAlpineRunsUnderTheSecurityPolicy(t *testing.T) {
|
|||||||
checkRefusedNewWebhook(ctx, t, srv.URL+"/hooks/new")
|
checkRefusedNewWebhook(ctx, t, srv.URL+"/hooks/new")
|
||||||
checkEventLog(ctx, t, page+"/events", event.ID, older.ID, target.Name)
|
checkEventLog(ctx, t, page+"/events", event.ID, older.ID, target.Name)
|
||||||
checkMobileMenu(ctx, t, page)
|
checkMobileMenu(ctx, t, page)
|
||||||
checkPhoneWidth(ctx, t, page, page+"/events", target.Name)
|
|
||||||
|
|
||||||
assert.Empty(t, problems(), "the browser reported problems")
|
assert.Empty(t, problems(), "the browser reported problems")
|
||||||
}
|
}
|
||||||
|
|
||||||
// seedBrowserWebhook seeds the webhook the browser test loads, owned by
|
// seedBrowserWebhook seeds the webhook the browser test loads, owned by
|
||||||
// userID: an entrypoint, two events, and a target whose delivery of the
|
// userID: an entrypoint, two events, and a target whose delivery of the
|
||||||
// newer event failed once with a 502. The webhook's name and the newer
|
// newer event failed once with a 502. It returns the webhook, the older
|
||||||
// event's content type are each too long for one line on a phone. It
|
// and the newer event, and the target.
|
||||||
// returns the webhook, the older and the newer event, and the target.
|
|
||||||
func seedBrowserWebhook(
|
func seedBrowserWebhook(
|
||||||
t *testing.T, env *testEnv, userID string,
|
t *testing.T, env *testEnv, userID string,
|
||||||
) (*database.Webhook, *database.Event, *database.Event, *database.Target) {
|
) (*database.Webhook, *database.Event, *database.Event, *database.Target) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
webhook := env.seedWebhook(t, userID)
|
webhook := env.seedWebhook(t, userID)
|
||||||
require.NoError(t, env.db.DB().Model(webhook).Update(
|
|
||||||
"name", "payment_provider_production_notifications",
|
|
||||||
).Error)
|
|
||||||
require.NoError(t, env.db.DB().Omit(clause.Associations).Create(
|
require.NoError(t, env.db.DB().Omit(clause.Associations).Create(
|
||||||
&database.Entrypoint{
|
&database.Entrypoint{
|
||||||
WebhookID: webhook.ID,
|
WebhookID: webhook.ID,
|
||||||
@@ -131,9 +126,6 @@ func seedBrowserWebhook(
|
|||||||
|
|
||||||
webhookDB, err := env.dbMgr.GetDB(webhook.ID)
|
webhookDB, err := env.dbMgr.GetDB(webhook.ID)
|
||||||
require.NoError(t, err)
|
require.NoError(t, err)
|
||||||
require.NoError(t, webhookDB.Model(event).Update(
|
|
||||||
"content_type", "application/vnd.paymentprovider.event+json",
|
|
||||||
).Error)
|
|
||||||
require.NoError(t, webhookDB.Omit(clause.Associations).Create(
|
require.NoError(t, webhookDB.Omit(clause.Associations).Create(
|
||||||
&database.DeliveryResult{
|
&database.DeliveryResult{
|
||||||
DeliveryID: dlv.ID,
|
DeliveryID: dlv.ID,
|
||||||
@@ -280,23 +272,6 @@ func click(ctx context.Context, t *testing.T, xpath string) {
|
|||||||
))
|
))
|
||||||
}
|
}
|
||||||
|
|
||||||
// clickAndLoad clicks the link or button matching an XPath expression
|
|
||||||
// and waits, as loadPage does, for the page the click opens to load and
|
|
||||||
// for Alpine.js to start on it. Reading earlier, a check can find an
|
|
||||||
// element of the page being left, gone by the time its value is read;
|
|
||||||
// and the wait in shown is too short for a page load on a busy host.
|
|
||||||
func clickAndLoad(ctx context.Context, t *testing.T, xpath string) {
|
|
||||||
t.Helper()
|
|
||||||
|
|
||||||
_, err := chromedp.RunResponse(
|
|
||||||
ctx, chromedp.Click(xpath, chromedp.BySearch),
|
|
||||||
)
|
|
||||||
require.NoError(t, err)
|
|
||||||
require.NoError(t, chromedp.Run(
|
|
||||||
ctx, chromedp.WaitNotPresent("[x-cloak]", chromedp.ByQuery),
|
|
||||||
))
|
|
||||||
}
|
|
||||||
|
|
||||||
// checkAddEntrypoint loads a webhook page and checks that the add
|
// checkAddEntrypoint loads a webhook page and checks that the add
|
||||||
// entrypoint form stays hidden until the Add button beside its heading
|
// entrypoint form stays hidden until the Add button beside its heading
|
||||||
// is clicked. The click looks for a button element there, so it also
|
// is clicked. The click looks for a button element there, so it also
|
||||||
@@ -439,7 +414,7 @@ func checkAddTarget(
|
|||||||
)))
|
)))
|
||||||
}
|
}
|
||||||
|
|
||||||
clickAndLoad(ctx, t, saveButton)
|
click(ctx, t, saveButton)
|
||||||
assert.Truef(t, shown(ctx, `//span[text()="`+name+
|
assert.Truef(t, shown(ctx, `//span[text()="`+name+
|
||||||
`"]/following-sibling::div/span[text()="`+badge+`"]`),
|
`"]/following-sibling::div/span[text()="`+badge+`"]`),
|
||||||
"%s: the added target is not listed as %s", targetType, badge)
|
"%s: the added target is not listed as %s", targetType, badge)
|
||||||
@@ -502,9 +477,10 @@ func checkArchiveChoices(ctx context.Context, t *testing.T, url string) {
|
|||||||
`/following-sibling::span[text()="daily"]`),
|
`/following-sibling::span[text()="daily"]`),
|
||||||
"a database target added with daily is not listed as daily")
|
"a database target added with daily is not listed as daily")
|
||||||
|
|
||||||
clickAndLoad(ctx, t, row+`//a[text()="Edit"]`)
|
click(ctx, t, row+`//a[text()="Edit"]`)
|
||||||
require.NoError(t, chromedp.Run(
|
require.NoError(t, chromedp.Run(
|
||||||
ctx,
|
ctx,
|
||||||
|
chromedp.WaitReady("#expiry", chromedp.ByQuery),
|
||||||
chromedp.Value("#expiry", &editedExpiry, chromedp.ByQuery),
|
chromedp.Value("#expiry", &editedExpiry, chromedp.ByQuery),
|
||||||
chromedp.Value("#rotation", &editedRotation, chromedp.ByQuery),
|
chromedp.Value("#rotation", &editedRotation, chromedp.ByQuery),
|
||||||
))
|
))
|
||||||
@@ -525,7 +501,6 @@ func checkRefusedTarget(ctx context.Context, t *testing.T, url string) {
|
|||||||
const (
|
const (
|
||||||
refusedURL = "http://127.0.0.1/hook"
|
refusedURL = "http://127.0.0.1/hook"
|
||||||
urlField = `form[action$="/targets"] input[name="url"]`
|
urlField = `form[action$="/targets"] input[name="url"]`
|
||||||
forwardQuery = `form[action$="/targets"] input[name="forward_query"]`
|
|
||||||
reason = `//div[@class="alert-error"]`
|
reason = `//div[@class="alert-error"]`
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -536,34 +511,25 @@ func checkRefusedTarget(ctx context.Context, t *testing.T, url string) {
|
|||||||
ctx,
|
ctx,
|
||||||
chromedp.SetValue(targetName, "refused", chromedp.ByQuery),
|
chromedp.SetValue(targetName, "refused", chromedp.ByQuery),
|
||||||
chromedp.SetValue(urlField, refusedURL, chromedp.ByQuery),
|
chromedp.SetValue(urlField, refusedURL, chromedp.ByQuery),
|
||||||
chromedp.Click(forwardQuery, chromedp.ByQuery),
|
|
||||||
))
|
))
|
||||||
|
|
||||||
clickAndLoad(ctx, t, saveButton)
|
click(ctx, t, saveButton)
|
||||||
|
|
||||||
assert.True(t, shown(ctx, reason),
|
assert.True(t, shown(ctx, reason),
|
||||||
"a refused target does not show the reason")
|
"a refused target does not show the reason")
|
||||||
|
|
||||||
var (
|
var name, typed string
|
||||||
name, typed string
|
|
||||||
checked bool
|
|
||||||
)
|
|
||||||
|
|
||||||
require.NoError(t, chromedp.Run(
|
require.NoError(t, chromedp.Run(
|
||||||
ctx,
|
ctx,
|
||||||
chromedp.Value(targetName, &name, chromedp.ByQuery),
|
chromedp.Value(targetName, &name, chromedp.ByQuery),
|
||||||
chromedp.Value(urlField, &typed, chromedp.ByQuery),
|
chromedp.Value(urlField, &typed, chromedp.ByQuery),
|
||||||
chromedp.JavascriptAttribute(
|
|
||||||
forwardQuery, "checked", &checked, chromedp.ByQuery,
|
|
||||||
),
|
|
||||||
))
|
))
|
||||||
|
|
||||||
assert.Equal(t, "refused", name,
|
assert.Equal(t, "refused", name,
|
||||||
"a refused target does not keep the name entered")
|
"a refused target does not keep the name entered")
|
||||||
assert.Equal(t, refusedURL, typed,
|
assert.Equal(t, refusedURL, typed,
|
||||||
"a refused target does not keep the url entered")
|
"a refused target does not keep the url entered")
|
||||||
assert.True(t, checked,
|
|
||||||
"a refused target does not keep the query string setting checked")
|
|
||||||
assert.True(t, shown(ctx, targetName),
|
assert.True(t, shown(ctx, targetName),
|
||||||
"a refused target does not come back with the form open")
|
"a refused target does not come back with the form open")
|
||||||
assert.True(t, hidden(ctx, typeSelect),
|
assert.True(t, hidden(ctx, typeSelect),
|
||||||
@@ -579,15 +545,10 @@ func checkRefusedTarget(ctx context.Context, t *testing.T, url string) {
|
|||||||
ctx,
|
ctx,
|
||||||
chromedp.Value(targetName, &name, chromedp.ByQuery),
|
chromedp.Value(targetName, &name, chromedp.ByQuery),
|
||||||
chromedp.Value(urlField, &typed, chromedp.ByQuery),
|
chromedp.Value(urlField, &typed, chromedp.ByQuery),
|
||||||
chromedp.JavascriptAttribute(
|
|
||||||
forwardQuery, "checked", &checked, chromedp.ByQuery,
|
|
||||||
),
|
|
||||||
))
|
))
|
||||||
|
|
||||||
assert.Empty(t, name, "after Cancel, the next Add keeps the name entered")
|
assert.Empty(t, name, "after Cancel, the next Add keeps the name entered")
|
||||||
assert.Empty(t, typed, "after Cancel, the next Add keeps the url entered")
|
assert.Empty(t, typed, "after Cancel, the next Add keeps the url entered")
|
||||||
assert.False(t, checked,
|
|
||||||
"after Cancel, the next Add keeps the query string setting checked")
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// checkTargetDeliveries loads a webhook page and checks that the row of
|
// checkTargetDeliveries loads a webhook page and checks that the row of
|
||||||
@@ -652,7 +613,7 @@ func checkRefusedEdits(
|
|||||||
))
|
))
|
||||||
}
|
}
|
||||||
|
|
||||||
clickAndLoad(ctx, t, `//button[text()="Save Changes"]`)
|
click(ctx, t, `//button[text()="Save Changes"]`)
|
||||||
|
|
||||||
assert.Truef(t, shown(ctx, reason),
|
assert.Truef(t, shown(ctx, reason),
|
||||||
"%s: a refused save does not show the reason", edit.url)
|
"%s: a refused save does not show the reason", edit.url)
|
||||||
@@ -776,7 +737,7 @@ func checkEntrypointEdit(
|
|||||||
require.NoError(t, chromedp.Run(
|
require.NoError(t, chromedp.Run(
|
||||||
ctx, chromedp.SendKeys(input, "Billing sender", chromedp.ByQuery),
|
ctx, chromedp.SendKeys(input, "Billing sender", chromedp.ByQuery),
|
||||||
))
|
))
|
||||||
clickAndLoad(ctx, t, saveEdit)
|
click(ctx, t, saveEdit)
|
||||||
|
|
||||||
assert.True(t, shown(ctx, `//span[text()="Billing sender"]`),
|
assert.True(t, shown(ctx, `//span[text()="Billing sender"]`),
|
||||||
"saving the edit form does not change the description")
|
"saving the edit form does not change the description")
|
||||||
@@ -814,7 +775,7 @@ func checkRecentEvents(ctx context.Context, t *testing.T, url string) {
|
|||||||
"clicking the newest event does not collapse it")
|
"clicking the newest event does not collapse it")
|
||||||
|
|
||||||
require.NoError(t, chromedp.Run(ctx, loadPage(url)))
|
require.NoError(t, chromedp.Run(ctx, loadPage(url)))
|
||||||
clickAndLoad(ctx, t, newest+`/ancestor::div[@x-data][1]//a[text()="Open"]`)
|
click(ctx, t, newest+`/ancestor::div[@x-data][1]//a[text()="Open"]`)
|
||||||
|
|
||||||
assert.True(t, shown(ctx, `//h2[text()="Body"]`),
|
assert.True(t, shown(ctx, `//h2[text()="Body"]`),
|
||||||
"Open does not lead to the event's own page")
|
"Open does not lead to the event's own page")
|
||||||
@@ -1215,7 +1176,7 @@ func checkNewWebhookTargets(
|
|||||||
`","rotation":"none"}`
|
`","rotation":"none"}`
|
||||||
}
|
}
|
||||||
|
|
||||||
clickAndLoad(ctx, t, createButton)
|
click(ctx, t, createButton)
|
||||||
require.Truef(t, shown(ctx, `//h1[text()="`+name+`"]`),
|
require.Truef(t, shown(ctx, `//h1[text()="`+name+`"]`),
|
||||||
"%s: the new webhook's page does not open", name)
|
"%s: the new webhook's page does not open", name)
|
||||||
|
|
||||||
@@ -1276,7 +1237,7 @@ func checkRefusedNewWebhook(ctx context.Context, t *testing.T, url string) {
|
|||||||
chromedp.SetValue(pruningChoice, "2160h", chromedp.BySearch),
|
chromedp.SetValue(pruningChoice, "2160h", chromedp.BySearch),
|
||||||
chromedp.SetValue("#archive_rotation", "monthly", chromedp.ByQuery),
|
chromedp.SetValue("#archive_rotation", "monthly", chromedp.ByQuery),
|
||||||
))
|
))
|
||||||
clickAndLoad(ctx, t, createButton)
|
click(ctx, t, createButton)
|
||||||
|
|
||||||
assert.True(t, shown(ctx, `//div[@class="alert-error"]`),
|
assert.True(t, shown(ctx, `//div[@class="alert-error"]`),
|
||||||
"a refused webhook does not show the reason")
|
"a refused webhook does not show the reason")
|
||||||
@@ -1331,60 +1292,3 @@ func checkMobileMenu(ctx context.Context, t *testing.T, url string) {
|
|||||||
click(ctx, t, button)
|
click(ctx, t, button)
|
||||||
assert.True(t, hidden(ctx, menu), "the menu button does not close the menu")
|
assert.True(t, hidden(ctx, menu), "the menu button does not close the menu")
|
||||||
}
|
}
|
||||||
|
|
||||||
// scrollsSideways reports whether the page is wider than the window. A
|
|
||||||
// page's clientWidth is the window's width less its scroll bar.
|
|
||||||
const scrollsSideways = `document.documentElement.scrollWidth >
|
|
||||||
document.documentElement.clientWidth`
|
|
||||||
|
|
||||||
// cutOffElements lists each element, without elements inside it, that
|
|
||||||
// is shown but runs past the page's edge or its card's, by more than a
|
|
||||||
// pixel of rounding. A card hides what runs past its edge.
|
|
||||||
const cutOffElements = `[...document.querySelectorAll("body *")]
|
|
||||||
.filter((el) => {
|
|
||||||
const box = el.getBoundingClientRect();
|
|
||||||
const card = el.closest(".card")?.getBoundingClientRect();
|
|
||||||
const left = card ? card.left : 0;
|
|
||||||
const right = card ? card.right : document.documentElement.clientWidth;
|
|
||||||
return el.children.length === 0 && box.width > 0 &&
|
|
||||||
(box.left < left - 1 || box.right > right + 1);
|
|
||||||
})
|
|
||||||
.map((el) => el.outerHTML.slice(0, 120))`
|
|
||||||
|
|
||||||
// checkPhoneWidth loads the webhook page, url, and its event log,
|
|
||||||
// eventLog, in a phone-sized window, the event log with the attempts of
|
|
||||||
// the newest event's delivery to targetName shown. It checks that
|
|
||||||
// neither page scrolls sideways and that nothing shown on either, no
|
|
||||||
// status, time or control, is cut off at the page's or its card's edge.
|
|
||||||
func checkPhoneWidth(
|
|
||||||
ctx context.Context, t *testing.T, url, eventLog, targetName string,
|
|
||||||
) {
|
|
||||||
t.Helper()
|
|
||||||
|
|
||||||
var (
|
|
||||||
sideways bool
|
|
||||||
cutOff []string
|
|
||||||
)
|
|
||||||
|
|
||||||
measure := chromedp.Tasks{
|
|
||||||
chromedp.Evaluate(scrollsSideways, &sideways),
|
|
||||||
chromedp.Evaluate(cutOffElements, &cutOff),
|
|
||||||
}
|
|
||||||
|
|
||||||
require.NoError(t, chromedp.Run(
|
|
||||||
ctx,
|
|
||||||
chromedp.EmulateViewport(phoneWidth, phoneHeight),
|
|
||||||
loadPage(url),
|
|
||||||
measure,
|
|
||||||
))
|
|
||||||
assert.False(t, sideways, "the webhook page scrolls sideways on a phone")
|
|
||||||
assert.Empty(t, cutOff, "the webhook page cuts these off on a phone")
|
|
||||||
|
|
||||||
require.NoError(t, chromedp.Run(ctx, loadPage(eventLog)))
|
|
||||||
click(ctx, t, `//span[text()="`+targetName+`"]`)
|
|
||||||
require.True(t, shown(ctx, `//span[text()="Attempt 1"]`),
|
|
||||||
"clicking the delivery does not show its attempts")
|
|
||||||
require.NoError(t, chromedp.Run(ctx, measure))
|
|
||||||
assert.False(t, sideways, "the event log scrolls sideways on a phone")
|
|
||||||
assert.Empty(t, cutOff, "the event log cuts these off on a phone")
|
|
||||||
}
|
|
||||||
|
|||||||
+1
-2
@@ -3,6 +3,5 @@
|
|||||||
"devDependencies": {
|
"devDependencies": {
|
||||||
"eslint": "10.11.0",
|
"eslint": "10.11.0",
|
||||||
"prettier": "3.9.9"
|
"prettier": "3.9.9"
|
||||||
},
|
}
|
||||||
"packageManager": "yarn@4.18.1+sha512.b2e1e7524f654f2749d32b4ebcb4622473cb5bcbc485df2007e12a154e50162a4d795526768bc5f5b8f81717bfd79deb2472813d86fb5ae2eb551fa9c872b08f"
|
|
||||||
}
|
}
|
||||||
|
|||||||
+2
-2
@@ -2,8 +2,8 @@
|
|||||||
# script/assets: extract Alpine.js from its npm package tarball, committed
|
# script/assets: extract Alpine.js from its npm package tarball, committed
|
||||||
# in 3p/, to static/js/alpine.min.js, where go:embed reads it. The package
|
# in 3p/, to static/js/alpine.min.js, where go:embed reads it. The package
|
||||||
# is @alpinejs/csp, Alpine's build for pages whose Content-Security-Policy
|
# is @alpinejs/csp, Alpine's build for pages whose Content-Security-Policy
|
||||||
# forbids eval. The extracted file is not committed. make build, make dev
|
# forbids eval. The extracted file is not committed. script/test, make
|
||||||
# and the Dockerfile's lint, test and build stages run this first.
|
# build and make dev run this first.
|
||||||
set -eu
|
set -eu
|
||||||
|
|
||||||
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
||||||
|
|||||||
+4
-12
@@ -11,7 +11,6 @@ ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
|||||||
|
|
||||||
PKGMGR=""
|
PKGMGR=""
|
||||||
SUDO=""
|
SUDO=""
|
||||||
APT_UPDATED=""
|
|
||||||
|
|
||||||
detect_pkgmgr() {
|
detect_pkgmgr() {
|
||||||
[ -n "$PKGMGR" ] && return 0
|
[ -n "$PKGMGR" ] && return 0
|
||||||
@@ -40,14 +39,7 @@ pkg_install() {
|
|||||||
detect_pkgmgr
|
detect_pkgmgr
|
||||||
case "$PKGMGR" in
|
case "$PKGMGR" in
|
||||||
nix) nix-env -iA "nixpkgs.$1" ;;
|
nix) nix-env -iA "nixpkgs.$1" ;;
|
||||||
apt)
|
apt) $SUDO env DEBIAN_FRONTEND=noninteractive apt-get install -y "$2" ;;
|
||||||
# Package lists may be empty (fresh images); refresh once per run.
|
|
||||||
if [ -z "$APT_UPDATED" ]; then
|
|
||||||
$SUDO env DEBIAN_FRONTEND=noninteractive apt-get update
|
|
||||||
APT_UPDATED=1
|
|
||||||
fi
|
|
||||||
$SUDO env DEBIAN_FRONTEND=noninteractive apt-get install -y "$2"
|
|
||||||
;;
|
|
||||||
brew) brew install "$3" ;;
|
brew) brew install "$3" ;;
|
||||||
apk) apk add --no-cache "$4" ;;
|
apk) apk add --no-cache "$4" ;;
|
||||||
esac
|
esac
|
||||||
@@ -68,10 +60,10 @@ main() {
|
|||||||
if missing go; then pkg_install go golang go go; fi
|
if missing go; then pkg_install go golang go go; fi
|
||||||
|
|
||||||
# Not installed here: docker is platform-specific and out of scope for a
|
# Not installed here: docker is platform-specific and out of scope for a
|
||||||
# package-manager bootstrap, but script/test, script/lint, script/fmt and
|
# package-manager bootstrap, but script/lint, script/fmt and script/css
|
||||||
# script/css need it.
|
# need it.
|
||||||
if missing docker; then
|
if missing docker; then
|
||||||
echo "bootstrap: docker not found; script/test, script/lint, script/fmt and script/css require it" >&2
|
echo "bootstrap: docker not found; script/lint, script/fmt and script/css require it" >&2
|
||||||
fi
|
fi
|
||||||
|
|
||||||
go mod download
|
go mod download
|
||||||
|
|||||||
+2
-2
@@ -1,7 +1,7 @@
|
|||||||
#!/bin/sh
|
#!/bin/sh
|
||||||
# script/check: run all checks (test, lint, fmt-check, css-check). Our own
|
# script/check: run all checks (test, lint, fmt-check, css-check). Our own
|
||||||
# extension to scripts-to-rule-them-all. test, lint and css-check are
|
# extension to scripts-to-rule-them-all.
|
||||||
# Docker builds; fmt-check runs gofmt on the host. Writes nothing.
|
# Writes only the ignored static/js/alpine.min.js, through script/test.
|
||||||
# Generic, apart from css-check.
|
# Generic, apart from css-check.
|
||||||
set -eu
|
set -eu
|
||||||
|
|
||||||
|
|||||||
Executable
+152
@@ -0,0 +1,152 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
# script/ci-mark-superseded: record an honest status on commits whose CI
|
||||||
|
# run Gitea cancelled because a newer commit landed on the same branch.
|
||||||
|
# Gitea writes `failure` / "Has been cancelled" for such a run, which
|
||||||
|
# reads as a test result on a commit nothing ever tested. Cancellation is
|
||||||
|
# unconditional server-side for push events, so the superseding run
|
||||||
|
# rewrites those statuses to `failure` with a description that says the
|
||||||
|
# commit was never tested. `skipped` cannot be used: Gitea's combined
|
||||||
|
# status folds `skipped` into `success`, so a never-tested commit would
|
||||||
|
# report green. Genuine failures and successes are never touched.
|
||||||
|
#
|
||||||
|
# Called by the Gitea Actions workflow, which supplies GITHUB_API_URL,
|
||||||
|
# GITHUB_REPOSITORY, GITHUB_SHA, GITHUB_WORKFLOW, GITHUB_JOB,
|
||||||
|
# GITHUB_EVENT_NAME and GITEA_TOKEN. ANCESTOR_LIMIT (default 20) caps how
|
||||||
|
# far back the walk looks; a value that is set but not a positive integer
|
||||||
|
# aborts rather than silently disabling the walk.
|
||||||
|
set -eu
|
||||||
|
|
||||||
|
SUPERSEDED_DESC='Superseded by a newer commit; never tested'
|
||||||
|
|
||||||
|
# Gitea builds the commit-status context as
|
||||||
|
# "<workflow name> / <job name> (<event>)", so derive it rather than
|
||||||
|
# hardcoding the result.
|
||||||
|
#
|
||||||
|
# The derivation is deliberately not byte-exact with Gitea's own rule and
|
||||||
|
# must not be "fixed" into a silent fallback. Gitea uses the job's `name:`
|
||||||
|
# (falling back to the job id) and the workflow's `name:` (falling back to
|
||||||
|
# the workflow filename), while the runner exports GITHUB_JOB as the job
|
||||||
|
# *id* and GITHUB_WORKFLOW as the parsed workflow `name:`. So giving the
|
||||||
|
# job a display `name:`, or dropping the workflow's `name:`, makes the
|
||||||
|
# derived context stop matching --- and require_own_context below then
|
||||||
|
# turns every push red with a message. That loud failure is the point
|
||||||
|
# (https://git.eeqj.de/sneak/webhooker/issues/147 item 2); guessing at a
|
||||||
|
# fallback would restore the silent no-op it replaced.
|
||||||
|
context() {
|
||||||
|
printf '%s / %s (%s)' \
|
||||||
|
"$GITHUB_WORKFLOW" "$GITHUB_JOB" "$GITHUB_EVENT_NAME"
|
||||||
|
}
|
||||||
|
|
||||||
|
# ANCESTOR_LIMIT is a documented knob, so a value that is set but
|
||||||
|
# unusable must fail loudly instead of defaulting
|
||||||
|
# (https://git.eeqj.de/sneak/webhooker/issues/80). Passing it straight to
|
||||||
|
# git would print `fatal: not an integer` into a discarded exit status
|
||||||
|
# and mark nothing.
|
||||||
|
ancestor_limit() {
|
||||||
|
# `-` and not `:-`: an explicitly empty value is set-but-unusable
|
||||||
|
# config, so it aborts like any other bad value rather than silently
|
||||||
|
# running at the default.
|
||||||
|
_limit="${ANCESTOR_LIMIT-20}"
|
||||||
|
case "$_limit" in
|
||||||
|
'' | *[!0-9]* | 0*)
|
||||||
|
echo "ANCESTOR_LIMIT must be a positive integer," \
|
||||||
|
"got '${_limit}'" >&2
|
||||||
|
|
||||||
|
return 1
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
printf '%s' "$_limit"
|
||||||
|
}
|
||||||
|
|
||||||
|
# The status Gitea created for this very job proves which context string
|
||||||
|
# it uses. If the derived one is missing, the workflow or the job was
|
||||||
|
# renamed and the match below would silently stop firing, restoring the
|
||||||
|
# false-red bug with no signal. Fail loudly instead.
|
||||||
|
require_own_context() {
|
||||||
|
if ! _body="$(curl -sf --retry 3 --retry-delay 2 --max-time 30 \
|
||||||
|
"${1}/commits/${GITHUB_SHA}/status")"; then
|
||||||
|
echo "cannot read commit statuses for ${GITHUB_SHA}" >&2
|
||||||
|
return 1
|
||||||
|
fi
|
||||||
|
_found="$(printf '%s' "$_body" | jq -r '(.statuses // [])[].context')"
|
||||||
|
if printf '%s\n' "$_found" | grep -qxF "$2"; then
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
echo "no commit status with context '${2}' on ${GITHUB_SHA}:" >&2
|
||||||
|
echo "workflow or job renamed? contexts present:" >&2
|
||||||
|
printf '%s\n' "$_found" >&2
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
|
||||||
|
# Latest status for our context on a commit, as "state|description".
|
||||||
|
# The read is retried and bounded, and a read that still fails aborts the
|
||||||
|
# step: a laundered commit that cannot be read is not the same as one
|
||||||
|
# with nothing to do, and piping curl into jq would discard the
|
||||||
|
# difference.
|
||||||
|
status_of() {
|
||||||
|
if ! _sbody="$(curl -sf --retry 3 --retry-delay 2 --max-time 30 \
|
||||||
|
"${1}/commits/${2}/status")"; then
|
||||||
|
echo "cannot read commit statuses for ${2}" >&2
|
||||||
|
|
||||||
|
return 1
|
||||||
|
fi
|
||||||
|
printf '%s' "$_sbody" | jq -r --arg c "$3" \
|
||||||
|
'[(.statuses // [])[] | select(.context == $c)][0] // empty
|
||||||
|
| "\(.status)|\(.description)"'
|
||||||
|
}
|
||||||
|
|
||||||
|
mark_superseded() {
|
||||||
|
curl -sf -X POST "${1}/statuses/${2}" \
|
||||||
|
-H "Authorization: token ${GITEA_TOKEN}" \
|
||||||
|
-H 'Content-Type: application/json' \
|
||||||
|
-d "$(jq -nc --arg c "$3" --arg d "$SUPERSEDED_DESC" \
|
||||||
|
'{context: $c, state: "failure", description: $d}')" \
|
||||||
|
>/dev/null
|
||||||
|
}
|
||||||
|
|
||||||
|
main() {
|
||||||
|
_api="${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}"
|
||||||
|
_ctx="$(context)"
|
||||||
|
|
||||||
|
_limit="$(ancestor_limit)"
|
||||||
|
|
||||||
|
require_own_context "$_api" "$_ctx"
|
||||||
|
|
||||||
|
# A shallow clone cannot resolve the parent, so it looks exactly like
|
||||||
|
# a root commit to rev-parse below and would exit 0 having walked
|
||||||
|
# nothing (or, at depth > 1, only the ancestors that happen to be
|
||||||
|
# present). The workflow checks out with `fetch-depth: 0`; verify
|
||||||
|
# that here rather than depend on it silently.
|
||||||
|
if [ "$(git rev-parse --is-shallow-repository)" = 'true' ]; then
|
||||||
|
echo "shallow repository: the ancestor walk needs full history" >&2
|
||||||
|
|
||||||
|
return 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
# A root commit legitimately has no ancestors and is not an error.
|
||||||
|
# A SHA this repository does not have lands here too, since its
|
||||||
|
# parent is equally unresolvable, but require_own_context above has
|
||||||
|
# already aborted on the 404 for it. The walk itself carries no
|
||||||
|
# `|| true`, so a rev-list failure aborts.
|
||||||
|
if ! git rev-parse -q --verify "${GITHUB_SHA}^" >/dev/null; then
|
||||||
|
echo "no ancestor of ${GITHUB_SHA} to check"
|
||||||
|
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
_walk="$(git rev-list --max-count="$_limit" "${GITHUB_SHA}^")"
|
||||||
|
|
||||||
|
for _sha in $_walk; do
|
||||||
|
_latest="$(status_of "$_api" "$_sha" "$_ctx")"
|
||||||
|
# A run that was cancelled, or one an earlier revision of this
|
||||||
|
# script laundered into `skipped`. Anything else stands.
|
||||||
|
case "$_latest" in
|
||||||
|
'failure|Has been cancelled' | "skipped|${SUPERSEDED_DESC}") ;;
|
||||||
|
*) continue ;;
|
||||||
|
esac
|
||||||
|
mark_superseded "$_api" "$_sha" "$_ctx"
|
||||||
|
echo "marked superseded: ${_sha}"
|
||||||
|
done
|
||||||
|
}
|
||||||
|
|
||||||
|
main "$@"
|
||||||
+6
-19
@@ -1,28 +1,15 @@
|
|||||||
#!/bin/sh
|
#!/bin/sh
|
||||||
# script/cibuild: run the CI build. It bootstraps first: a CI runner
|
# script/cibuild: run the CI build. The Dockerfile runs the checks
|
||||||
# checks out and runs this and nothing else, and script/fmt-check runs
|
# (make fmt-check, lint, test), so a successful build implies a green
|
||||||
# the formatter on the host, which a pristine checkout cannot do.
|
# repo. Generic: needs no adaptation. The Gitea workflow runs this on
|
||||||
# --no-cache for the same reason as script/docker: the gate phases the
|
# push.
|
||||||
# final stage depends on are RUN steps, and a cached one is a check that
|
|
||||||
# did not run.
|
|
||||||
set -eu
|
set -eu
|
||||||
|
|
||||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
|
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
||||||
ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
|
|
||||||
|
|
||||||
main() {
|
main() {
|
||||||
cd "$ROOT"
|
cd "$ROOT"
|
||||||
"$SCRIPT_DIR/bootstrap"
|
docker build .
|
||||||
"$SCRIPT_DIR/check"
|
|
||||||
# Own line: a failing command substitution inside an argument does
|
|
||||||
# not trip `set -e`, so the inline form degrades silently to an
|
|
||||||
# empty constant. The VERSION build argument takes precedence over
|
|
||||||
# the version a build stage derives from the .git in the context.
|
|
||||||
version="$(git describe --tags --always --dirty 2>/dev/null || true)"
|
|
||||||
[ -n "$version" ] || version="unknown"
|
|
||||||
docker build --no-cache \
|
|
||||||
--build-arg VERSION="$version" \
|
|
||||||
-t "$("$SCRIPT_DIR/projectname")" .
|
|
||||||
}
|
}
|
||||||
|
|
||||||
main "$@"
|
main "$@"
|
||||||
|
|||||||
+2
-4
@@ -2,16 +2,14 @@
|
|||||||
# script/css: regenerate static/css/tailwind.css (writes). tailwindcss is
|
# script/css: regenerate static/css/tailwind.css (writes). tailwindcss is
|
||||||
# never installed locally: it runs in docker, at the version and sha256
|
# never installed locally: it runs in docker, at the version and sha256
|
||||||
# pinned in the Dockerfile's stylesheet stages, which also say what the
|
# pinned in the Dockerfile's stylesheet stages, which also say what the
|
||||||
# stylesheet is generated from. --no-cache, as on every docker build in
|
# stylesheet is generated from.
|
||||||
# script/, so the stylesheet is generated rather than taken from the cache.
|
|
||||||
set -eu
|
set -eu
|
||||||
|
|
||||||
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
||||||
|
|
||||||
main() {
|
main() {
|
||||||
cd "$ROOT"
|
cd "$ROOT"
|
||||||
docker build --no-cache \
|
docker build --target css-output --output type=local,dest=static/css .
|
||||||
--target css-output --output type=local,dest=static/css .
|
|
||||||
}
|
}
|
||||||
|
|
||||||
main "$@"
|
main "$@"
|
||||||
|
|||||||
+2
-3
@@ -1,15 +1,14 @@
|
|||||||
#!/bin/sh
|
#!/bin/sh
|
||||||
# script/css-check: fail when static/css/tailwind.css differs from what
|
# script/css-check: fail when static/css/tailwind.css differs from what
|
||||||
# script/css would generate (read-only). The comparison is the Dockerfile's
|
# script/css would generate (read-only). The comparison is the Dockerfile's
|
||||||
# css-check stage, which the image build runs too. --no-cache because a
|
# css-check stage, which the image build runs too.
|
||||||
# cached check is a check that did not run.
|
|
||||||
set -eu
|
set -eu
|
||||||
|
|
||||||
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
||||||
|
|
||||||
main() {
|
main() {
|
||||||
cd "$ROOT"
|
cd "$ROOT"
|
||||||
docker build --no-cache --target css-check --output type=cacheonly .
|
docker build --target css-check --output type=cacheonly .
|
||||||
}
|
}
|
||||||
|
|
||||||
main "$@"
|
main "$@"
|
||||||
|
|||||||
+7
-11
@@ -1,8 +1,10 @@
|
|||||||
#!/bin/sh
|
#!/bin/sh
|
||||||
# script/docker: build the Docker image tagged with the project name.
|
# script/docker: build the Docker image tagged with the project name.
|
||||||
# Identical in all repos; the tag comes from script/projectname.
|
# The tag comes from script/projectname.
|
||||||
# --no-cache because the gate phases the final stage depends on are RUN
|
#
|
||||||
# steps, and a cached one is a check that did not run.
|
# The version script/version resolves here goes in as the VERSION build
|
||||||
|
# arg, which takes precedence over what the build would derive from the
|
||||||
|
# .git in its context.
|
||||||
set -eu
|
set -eu
|
||||||
|
|
||||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
|
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
|
||||||
@@ -10,14 +12,8 @@ ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
|
|||||||
|
|
||||||
main() {
|
main() {
|
||||||
cd "$ROOT"
|
cd "$ROOT"
|
||||||
# Own line: a failing command substitution inside an argument does
|
docker build \
|
||||||
# not trip `set -e`, so the inline form degrades silently to an
|
--build-arg VERSION="$("$SCRIPT_DIR/version")" \
|
||||||
# empty constant. The VERSION build argument takes precedence over
|
|
||||||
# the version a build stage derives from the .git in the context.
|
|
||||||
version="$(git describe --tags --always --dirty 2>/dev/null || true)"
|
|
||||||
[ -n "$version" ] || version="unknown"
|
|
||||||
docker build --no-cache \
|
|
||||||
--build-arg VERSION="$version" \
|
|
||||||
-t "$("$SCRIPT_DIR/projectname")" .
|
-t "$("$SCRIPT_DIR/projectname")" .
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
+2
-4
@@ -1,8 +1,7 @@
|
|||||||
#!/bin/sh
|
#!/bin/sh
|
||||||
# script/fmt: format all files (writes): the Go code with gofmt and
|
# script/fmt: format all files (writes): the Go code with gofmt and
|
||||||
# goimports, the Markdown with prettier. prettier is never installed
|
# goimports, the Markdown with prettier. prettier is never installed
|
||||||
# locally: it runs in docker, in the Dockerfile's Markdown stages, built
|
# locally: it runs in docker, in the Dockerfile's Markdown stages.
|
||||||
# with --no-cache like every docker build in script/.
|
|
||||||
set -eu
|
set -eu
|
||||||
|
|
||||||
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
||||||
@@ -13,8 +12,7 @@ main() {
|
|||||||
if command -v goimports >/dev/null 2>&1; then
|
if command -v goimports >/dev/null 2>&1; then
|
||||||
goimports -w .
|
goimports -w .
|
||||||
fi
|
fi
|
||||||
docker build --no-cache \
|
docker build --target markdown-output --output type=local,dest=. .
|
||||||
--target markdown-output --output type=local,dest=. .
|
|
||||||
}
|
}
|
||||||
|
|
||||||
main "$@"
|
main "$@"
|
||||||
|
|||||||
+2
-3
@@ -1,7 +1,6 @@
|
|||||||
#!/bin/sh
|
#!/bin/sh
|
||||||
# script/fmt-check: check formatting (read-only). Same scope as
|
# script/fmt-check: check formatting (read-only). Same scope as
|
||||||
# script/fmt, but fails instead of writing. --no-cache because a cached
|
# script/fmt, but fails instead of writing.
|
||||||
# check is a check that did not run.
|
|
||||||
set -eu
|
set -eu
|
||||||
|
|
||||||
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
||||||
@@ -13,7 +12,7 @@ main() {
|
|||||||
gofmt -s -l .
|
gofmt -s -l .
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
docker build --no-cache --target markdown-check --output type=cacheonly .
|
docker build --target markdown-check --output type=cacheonly .
|
||||||
}
|
}
|
||||||
|
|
||||||
main "$@"
|
main "$@"
|
||||||
|
|||||||
+61
-13
@@ -1,23 +1,71 @@
|
|||||||
#!/bin/sh
|
#!/bin/sh
|
||||||
# script/lint: run the linter. Linting is a phase of the Dockerfile and
|
# script/lint: run the linters, golangci-lint over the Go code and then
|
||||||
# this builds that phase alone; the linter is never installed or run on
|
# ESLint over static/js/. Neither is ever installed locally.
|
||||||
# a developer host, where a shared result cache and a host-global lock
|
|
||||||
# make its answer untrustworthy.
|
|
||||||
#
|
#
|
||||||
# The phase is not the last stage in the file, so it is built only when
|
# golangci-lint runs via docker only, one way, everywhere — script/lint builds
|
||||||
# --target names it. --no-cache because a cached lint layer is a lint
|
# Dockerfile.lint, which COPYs the repo into the pinned golangci-lint image
|
||||||
# that did not run. The tag makes each build replace the previous image
|
# and lints as a build step. This works even when the docker daemon is remote
|
||||||
# instead of leaving a dangling one behind.
|
# and bind mounts are impossible, and it removes the host linter's shared
|
||||||
|
# cache, which has attributed other checkouts' findings to this one.
|
||||||
|
#
|
||||||
|
# --no-cache-filter=lint forces the lint stage to re-execute on every run; a
|
||||||
|
# cached lint stage exits 0 in under a second having linted nothing. The deps
|
||||||
|
# stage keeps its cache, so module downloads are not repeated.
|
||||||
|
# --progress=plain keeps the linter's own output visible on success, so a
|
||||||
|
# passing run shows the issue count rather than nothing.
|
||||||
|
# --output=type=cacheonly leaves no image behind to clean up.
|
||||||
|
#
|
||||||
|
# docker silently ignores --no-cache-filter for a stage name that does not
|
||||||
|
# match, so a rename or a typo would restore the cached false green with no
|
||||||
|
# warning and a fast exit 0. The flag is therefore not trusted: the build
|
||||||
|
# output is teed to a log and a run is only a pass if golangci-lint's own
|
||||||
|
# summary line ("N issues." / "N issues:") is in it. No summary, no lint,
|
||||||
|
# whatever the exit code says.
|
||||||
set -eu
|
set -eu
|
||||||
|
|
||||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
|
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
||||||
ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
|
|
||||||
|
|
||||||
main() {
|
main() {
|
||||||
cd "$ROOT"
|
cd "$ROOT"
|
||||||
docker build --no-cache \
|
|
||||||
--target lint \
|
log="$(mktemp -t webhooker-lint.XXXXXXXX)"
|
||||||
-t "$("$SCRIPT_DIR/projectname")-lint" .
|
rcfile="$(mktemp -t webhooker-lint-rc.XXXXXXXX)"
|
||||||
|
trap 'rm -f "$log" "$rcfile"' EXIT INT TERM
|
||||||
|
|
||||||
|
# The pipeline's status is tee's, and POSIX sh has no pipefail, so the
|
||||||
|
# build's status travels via a file. Output still streams live.
|
||||||
|
{
|
||||||
|
docker build \
|
||||||
|
-f Dockerfile.lint \
|
||||||
|
--no-cache-filter=lint \
|
||||||
|
--progress=plain \
|
||||||
|
--output=type=cacheonly \
|
||||||
|
. 2>&1 && echo 0 >"$rcfile" || echo $? >"$rcfile"
|
||||||
|
} | tee "$log" >&2
|
||||||
|
|
||||||
|
rc="$(cat "$rcfile")"
|
||||||
|
[ "$rc" -eq 0 ] || exit "$rc"
|
||||||
|
|
||||||
|
if ! grep -qE '[0-9]+ issues[.:]' "$log"; then
|
||||||
|
echo "script/lint: golangci-lint printed no summary line; the linter" >&2
|
||||||
|
echo " did not run. Check that the stage named in --no-cache-filter" >&2
|
||||||
|
echo " still matches a stage in Dockerfile.lint." >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
# ESLint runs in the Dockerfile's js-lint stage, which the image build
|
||||||
|
# runs too. It prints nothing on a pass, so there is no summary to look
|
||||||
|
# for. Instead the stage is named once, for both flags: --target fails
|
||||||
|
# on a name that matches no stage, so a rename cannot leave
|
||||||
|
# --no-cache-filter silently ignored. The js-deps stage, which installs
|
||||||
|
# ESLint, keeps its cache, so ESLint is not downloaded again.
|
||||||
|
js_stage=js-lint
|
||||||
|
docker build \
|
||||||
|
--target "$js_stage" \
|
||||||
|
--no-cache-filter="$js_stage" \
|
||||||
|
--progress=plain \
|
||||||
|
--output=type=cacheonly \
|
||||||
|
.
|
||||||
}
|
}
|
||||||
|
|
||||||
main "$@"
|
main "$@"
|
||||||
|
|||||||
+75
-10
@@ -1,19 +1,84 @@
|
|||||||
#!/bin/sh
|
#!/bin/sh
|
||||||
# script/test: run the test suite. Testing is a phase of the Dockerfile
|
# script/test: run the test suite.
|
||||||
# and this builds that phase alone, on the same terms as script/lint:
|
#
|
||||||
# --target because a phase that is not the last stage is built only when
|
# -timeout is applied by `go test` per package, not to the run as a whole, so
|
||||||
# named, --no-cache because a cached test layer is a test that did not
|
# it only has to clear the slowest single package. When this budget was set
|
||||||
# run, and a tag so each build replaces the previous image.
|
# that was internal/handlers, measured in a cache-defeated builder stage on the
|
||||||
|
# 48-core shared build host (2026-08-18); load- and host-dependent, not
|
||||||
|
# invariants:
|
||||||
|
#
|
||||||
|
# 16.9s host load 5-20, GOMAXPROCS 48
|
||||||
|
# 45.9s / 47.3s / 49.0s three runs at deliberate host load 31-73
|
||||||
|
# 30.6s / 39.7s host load 5-20, GOMAXPROCS 6 / 4
|
||||||
|
# 67.3s / 97.5s host load 5-20, GOMAXPROCS 2 / 1
|
||||||
|
# 67.3s GOMAXPROCS 4 at deliberate host load 52-68
|
||||||
|
#
|
||||||
|
# The old 30s budget was breached by every loaded run and by every GOMAXPROCS
|
||||||
|
# at or below 6; at GOMAXPROCS 4 it failed outright ("panic: test timed out
|
||||||
|
# after 30s"), reproduced on 33e4fa4 with no other change.
|
||||||
|
#
|
||||||
|
# 90s matches the org-wide backstop in REPO_POLICIES.md and is sized here
|
||||||
|
# against the figures above: the worst case under native parallelism is 49.0s,
|
||||||
|
# and the compound GOMAXPROCS-4-under-load case at 67.3s sits at 75% of it.
|
||||||
|
# The one figure above 90s is GOMAXPROCS 1, a synthetic core floor rather than
|
||||||
|
# a condition CI runs under. If a CPU-limited runner ever puts a real run near
|
||||||
|
# 67s, that is the datum to revisit the org figure with.
|
||||||
|
#
|
||||||
|
# Those figures predate tests hashing the admin password at 1 MB instead of
|
||||||
|
# 64 MB (https://git.eeqj.de/sneak/webhooker/pulls/404). After that change, in
|
||||||
|
# a cache-defeated build at host load 44-109 (2026-10-02), internal/handlers
|
||||||
|
# took 8.5s and the slowest package was internal/database at 15.8s. Once its
|
||||||
|
# retention tests seeded 50 rows per insert instead of 500
|
||||||
|
# (https://git.eeqj.de/sneak/webhooker/issues/198), internal/database took
|
||||||
|
# 7.3s and the slowest package was internal/handlers at 8.1s to 10.0s, at host
|
||||||
|
# load 25-48 (2026-10-02).
|
||||||
|
#
|
||||||
|
# -p 4 -parallel 8 keep the run under 2 GB of memory: at most four test
|
||||||
|
# binaries build or run at once, each with at most eight parallel tests. Under
|
||||||
|
# -race every test binary and every link costs a few hundred MB, so the
|
||||||
|
# defaults (one per core) add up to several GB on a many-core host.
|
||||||
|
#
|
||||||
|
# The first run has no -v: go test then prints one result line per package,
|
||||||
|
# with its coverage, and for a package that fails, everything its tests wrote,
|
||||||
|
# application log lines included. Verbose output from the whole suite passes
|
||||||
|
# the 2 MiB at which the Docker build cuts off each step's log, so on a failure
|
||||||
|
# only the tests that failed run again, with -v. The script exits 1 after that
|
||||||
|
# rerun whatever its result: the first run already showed the suite is broken.
|
||||||
set -eu
|
set -eu
|
||||||
|
|
||||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
|
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
||||||
ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
|
|
||||||
|
|
||||||
main() {
|
main() {
|
||||||
cd "$ROOT"
|
cd "$ROOT"
|
||||||
docker build --no-cache \
|
"$ROOT/script/assets"
|
||||||
--target test \
|
|
||||||
-t "$("$SCRIPT_DIR/projectname")-test" .
|
log="$(mktemp -t webhooker-test.XXXXXXXX)"
|
||||||
|
rcfile="$(mktemp -t webhooker-test-rc.XXXXXXXX)"
|
||||||
|
trap 'rm -f "$log" "$rcfile"' EXIT INT TERM
|
||||||
|
|
||||||
|
# The pipeline's status is tee's, and POSIX sh has no pipefail, so go
|
||||||
|
# test's status travels via a file. Output still streams live.
|
||||||
|
{
|
||||||
|
go test -race -cover -p 4 -parallel 8 -timeout 90s ./... 2>&1 \
|
||||||
|
&& echo 0 >"$rcfile" || echo $? >"$rcfile"
|
||||||
|
} | tee "$log"
|
||||||
|
if [ "$(cat "$rcfile")" -eq 0 ]; then
|
||||||
|
return
|
||||||
|
fi
|
||||||
|
|
||||||
|
# go test reports a failed test as a line starting "--- FAIL: TestName"
|
||||||
|
# (a failed subtest's line is indented, and reruns with its parent), and
|
||||||
|
# a failed package as "FAIL<tab>package/path<tab>...". A failure that
|
||||||
|
# names no test, such as a build error or a timeout, is already shown in
|
||||||
|
# full above, so there is nothing to rerun.
|
||||||
|
tests="$(awk '/^--- FAIL: / { print $3 }' "$log" | paste -s -d '|' -)"
|
||||||
|
packages="$(awk '/^FAIL\t/ { print $2 }' "$log")"
|
||||||
|
if [ -n "$tests" ]; then
|
||||||
|
echo "--- Rerunning the failed tests with -v for details ---"
|
||||||
|
go test -race -v -p 4 -parallel 8 -timeout 90s \
|
||||||
|
-run "^($tests)\$" $packages || true
|
||||||
|
fi
|
||||||
|
exit 1
|
||||||
}
|
}
|
||||||
|
|
||||||
main "$@"
|
main "$@"
|
||||||
|
|||||||
+3
-2
@@ -3,7 +3,8 @@
|
|||||||
# Docker: Dockerfile.browser builds the test and runs it in a digest-pinned
|
# Docker: Dockerfile.browser builds the test and runs it in a digest-pinned
|
||||||
# headless browser image, so the host needs no browser.
|
# headless browser image, so the host needs no browser.
|
||||||
#
|
#
|
||||||
# --no-cache because a cached test layer is a test that did not run.
|
# --no-cache-filter=browser runs the test again even when nothing changed;
|
||||||
|
# it must name the stage in Dockerfile.browser that runs it.
|
||||||
# --output=type=cacheonly leaves no image behind to clean up.
|
# --output=type=cacheonly leaves no image behind to clean up.
|
||||||
set -eu
|
set -eu
|
||||||
|
|
||||||
@@ -13,7 +14,7 @@ main() {
|
|||||||
cd "$ROOT"
|
cd "$ROOT"
|
||||||
docker build \
|
docker build \
|
||||||
-f Dockerfile.browser \
|
-f Dockerfile.browser \
|
||||||
--no-cache \
|
--no-cache-filter=browser \
|
||||||
--progress=plain \
|
--progress=plain \
|
||||||
--output=type=cacheonly \
|
--output=type=cacheonly \
|
||||||
.
|
.
|
||||||
|
|||||||
+3
-5
@@ -1,11 +1,9 @@
|
|||||||
#!/bin/sh
|
#!/bin/sh
|
||||||
# script/version: output the version string the binary is stamped with.
|
# script/version: output the version string the binary is stamped with.
|
||||||
# Our own extension to scripts-to-rule-them-all. The Makefile's build
|
# Our own extension to scripts-to-rule-them-all. The Makefile's build
|
||||||
# and version targets take the value from here, and the Dockerfile's
|
# target and script/docker both take the value from here, so a `make
|
||||||
# build stage calls them. script/docker and script/cibuild run the same
|
# build` binary and a `make docker` image built from the same checkout
|
||||||
# `git describe` on the host and pass the result in as $VERSION, so a
|
# report the same thing.
|
||||||
# `make build` binary and a `make docker` image built from the same
|
|
||||||
# checkout report the same thing.
|
|
||||||
#
|
#
|
||||||
# Order of precedence:
|
# Order of precedence:
|
||||||
#
|
#
|
||||||
|
|||||||
File diff suppressed because one or more lines are too long
@@ -138,7 +138,6 @@ document.addEventListener("alpine:init", function () {
|
|||||||
url: "",
|
url: "",
|
||||||
headers: "",
|
headers: "",
|
||||||
timeout: "",
|
timeout: "",
|
||||||
forwardQuery: false,
|
|
||||||
maxRetries: "",
|
maxRetries: "",
|
||||||
expiry: "",
|
expiry: "",
|
||||||
rotation: "",
|
rotation: "",
|
||||||
@@ -151,8 +150,6 @@ document.addEventListener("alpine:init", function () {
|
|||||||
this.url = refused.destination;
|
this.url = refused.destination;
|
||||||
this.headers = refused.headers;
|
this.headers = refused.headers;
|
||||||
this.timeout = refused.timeout;
|
this.timeout = refused.timeout;
|
||||||
this.forwardQuery =
|
|
||||||
this.$root.hasAttribute("data-forward-query");
|
|
||||||
this.maxRetries = refused.maxRetries;
|
this.maxRetries = refused.maxRetries;
|
||||||
this.expiry = refused.expiry;
|
this.expiry = refused.expiry;
|
||||||
this.rotation = refused.rotation;
|
this.rotation = refused.rotation;
|
||||||
@@ -172,7 +169,6 @@ document.addEventListener("alpine:init", function () {
|
|||||||
this.url = "";
|
this.url = "";
|
||||||
this.headers = "";
|
this.headers = "";
|
||||||
this.timeout = "";
|
this.timeout = "";
|
||||||
this.forwardQuery = false;
|
|
||||||
this.maxRetries = "";
|
this.maxRetries = "";
|
||||||
this.expiry = "";
|
this.expiry = "";
|
||||||
this.rotation = "";
|
this.rotation = "";
|
||||||
|
|||||||
@@ -4,7 +4,7 @@
|
|||||||
an event's own page. Spans only, since a button may hold no div. -->
|
an event's own page. Spans only, since a button may hold no div. -->
|
||||||
<span class="flex flex-1 flex-wrap items-center justify-between gap-3">
|
<span class="flex flex-1 flex-wrap items-center justify-between gap-3">
|
||||||
<span class="flex flex-wrap items-center gap-3">
|
<span class="flex flex-wrap items-center gap-3">
|
||||||
<span class="text-sm text-gray-700 wrap-anywhere">{{.Target.DisplayName}}</span>
|
<span class="text-sm text-gray-700">{{.Target.DisplayName}}</span>
|
||||||
{{if .Replay}}<span class="text-xs text-gray-500">replay</span>{{end}}
|
{{if .Replay}}<span class="text-xs text-gray-500">replay</span>{{end}}
|
||||||
<span class="text-xs {{if eq .Status "delivered"}}text-green-600{{else if eq .Status "failed"}}text-red-600{{else if eq .Status "retrying"}}text-yellow-600{{else}}text-gray-400{{end}}">{{with .Paused}}waiting: target paused after repeated failures, next try no earlier than {{.Until}} ({{.Relative}}){{else}}{{.Status}}{{end}}</span>
|
<span class="text-xs {{if eq .Status "delivered"}}text-green-600{{else if eq .Status "failed"}}text-red-600{{else if eq .Status "retrying"}}text-yellow-600{{else}}text-gray-400{{end}}">{{with .Paused}}waiting: target paused after repeated failures, next try no earlier than {{.Until}} ({{.Relative}}){{else}}{{.Status}}{{end}}</span>
|
||||||
</span>
|
</span>
|
||||||
|
|||||||
@@ -1,22 +1,14 @@
|
|||||||
{{define "event_request"}}
|
{{define "event_request"}}
|
||||||
<!-- The entrypoint an event arrived at, its query string and its
|
<!-- The entrypoint an event arrived at and its request headers, as
|
||||||
request headers, as handlers.EventLogView carries them: the same in
|
handlers.EventLogView carries them: the same in the event log and
|
||||||
the event log and the event's own page. The entrypoint's URL is
|
the event's own page. The entrypoint's URL is never shown. A
|
||||||
never shown. A resubmitted copy, even a copy of a copy, did not
|
resubmitted copy, even a copy of a copy, did not arrive at an
|
||||||
arrive at an entrypoint; the request it copies did. -->
|
entrypoint; the request it copies did. -->
|
||||||
<div class="space-y-2 text-xs">
|
<div class="space-y-2 text-xs">
|
||||||
{{if .ResubmittedFrom}}
|
{{if .ResubmittedFrom}}
|
||||||
<p class="text-gray-500">The request it copies arrived at <span class="text-gray-900 wrap-anywhere">{{.Entrypoint}}</span></p>
|
<p class="text-gray-500">The request it copies arrived at <span class="text-gray-900">{{.Entrypoint}}</span></p>
|
||||||
{{else}}
|
{{else}}
|
||||||
<p class="text-gray-500">Arrived at <span class="text-gray-900 wrap-anywhere">{{.Entrypoint}}</span></p>
|
<p class="text-gray-500">Arrived at <span class="text-gray-900">{{.Entrypoint}}</span></p>
|
||||||
{{end}}
|
|
||||||
{{if .RawQueryCut}}
|
|
||||||
<p class="text-gray-500">The query string is larger than the event log shows. <a href="{{.Body.EventURL}}" class="btn-small">Show the query string</a></p>
|
|
||||||
{{else if .RawQuery}}
|
|
||||||
<p class="text-gray-500">Query string</p>
|
|
||||||
<pre class="rounded-md border border-gray-200 bg-white p-2 text-xs text-gray-700 overflow-x-auto whitespace-pre-wrap break-all">{{.RawQuery}}</pre>
|
|
||||||
{{else}}
|
|
||||||
<p class="text-gray-500">No query string.</p>
|
|
||||||
{{end}}
|
{{end}}
|
||||||
{{if .HeadersCut}}
|
{{if .HeadersCut}}
|
||||||
<p class="text-gray-500">The request headers are larger than the event log shows. <a href="{{.Body.EventURL}}" class="btn-small">Show the request headers</a></p>
|
<p class="text-gray-500">The request headers are larger than the event log shows. <a href="{{.Body.EventURL}}" class="btn-small">Show the request headers</a></p>
|
||||||
|
|||||||
@@ -6,18 +6,15 @@
|
|||||||
<!-- 108rem, half again the 72rem (max-w-6xl) of the webhook list, the
|
<!-- 108rem, half again the 72rem (max-w-6xl) of the webhook list, the
|
||||||
event log, the navbar and the footer, so an entrypoint URL fits on
|
event log, the navbar and the footer, so an entrypoint URL fits on
|
||||||
one line. An inline style, because the committed tailwind.css has
|
one line. An inline style, because the committed tailwind.css has
|
||||||
no class this wide. wrap-anywhere goes only on names and
|
no class this wide. -->
|
||||||
descriptions: a row too wide for a phone must still run past the
|
|
||||||
edge, where the browser test sees it, rather than break its
|
|
||||||
controls mid-word. -->
|
|
||||||
<div class="mx-auto px-6 py-8" style="max-width: 108rem">
|
<div class="mx-auto px-6 py-8" style="max-width: 108rem">
|
||||||
<div class="mb-6">
|
<div class="mb-6">
|
||||||
<a href="/hooks" class="btn-small">← Back to webhooks</a>
|
<a href="/hooks" class="btn-small">← Back to webhooks</a>
|
||||||
<div class="flex flex-wrap justify-between items-center gap-2 mt-2">
|
<div class="flex flex-wrap justify-between items-center gap-2 mt-2">
|
||||||
<div>
|
<div>
|
||||||
<h1 class="text-2xl font-medium text-gray-900 wrap-anywhere">{{.Webhook.Name}}</h1>
|
<h1 class="text-2xl font-medium text-gray-900">{{.Webhook.Name}}</h1>
|
||||||
{{if .Webhook.Description}}
|
{{if .Webhook.Description}}
|
||||||
<p class="text-sm text-gray-500 mt-1 wrap-anywhere">{{.Webhook.Description}}</p>
|
<p class="text-sm text-gray-500 mt-1">{{.Webhook.Description}}</p>
|
||||||
{{end}}
|
{{end}}
|
||||||
</div>
|
</div>
|
||||||
<div class="flex gap-2">
|
<div class="flex gap-2">
|
||||||
@@ -63,7 +60,7 @@
|
|||||||
{{range .Entrypoints}}
|
{{range .Entrypoints}}
|
||||||
<div class="p-4" x-data="collapsible">
|
<div class="p-4" x-data="collapsible">
|
||||||
<div class="flex flex-wrap items-center justify-between gap-2 mb-1">
|
<div class="flex flex-wrap items-center justify-between gap-2 mb-1">
|
||||||
<span x-show="closed" class="text-sm font-medium text-gray-900 wrap-anywhere">{{if .Description}}{{.Description}}{{else}}Entrypoint{{end}}</span>
|
<span x-show="closed" class="text-sm font-medium text-gray-900">{{if .Description}}{{.Description}}{{else}}Entrypoint{{end}}</span>
|
||||||
<!-- Edit shows this form in place of the
|
<!-- Edit shows this form in place of the
|
||||||
description and hides until it closes, and
|
description and hides until it closes, and
|
||||||
Cancel resets what was typed. With
|
Cancel resets what was typed. With
|
||||||
@@ -135,7 +132,6 @@
|
|||||||
data-destination="{{.TargetForm.URL}}"
|
data-destination="{{.TargetForm.URL}}"
|
||||||
data-headers="{{.TargetForm.Headers}}"
|
data-headers="{{.TargetForm.Headers}}"
|
||||||
data-timeout="{{.TargetForm.Timeout}}"
|
data-timeout="{{.TargetForm.Timeout}}"
|
||||||
{{if .TargetForm.ForwardQuery}}data-forward-query{{end}}
|
|
||||||
data-max-retries="{{.TargetForm.MaxRetries}}"
|
data-max-retries="{{.TargetForm.MaxRetries}}"
|
||||||
data-expiry="{{.TargetForm.Expiry}}"
|
data-expiry="{{.TargetForm.Expiry}}"
|
||||||
data-rotation="{{.TargetForm.Rotation}}">
|
data-rotation="{{.TargetForm.Rotation}}">
|
||||||
@@ -184,13 +180,6 @@
|
|||||||
<label class="text-sm text-gray-700">Timeout (seconds, blank = default):</label>
|
<label class="text-sm text-gray-700">Timeout (seconds, blank = default):</label>
|
||||||
<input type="number" name="timeout" :value="timeout" min="0" max="300" class="input text-sm w-24">
|
<input type="number" name="timeout" :value="timeout" min="0" max="300" class="input text-sm w-24">
|
||||||
</div>
|
</div>
|
||||||
<div>
|
|
||||||
<label class="flex items-center gap-2 text-sm text-gray-700">
|
|
||||||
<input type="checkbox" name="forward_query" value="on" :checked="forwardQuery" class="h-4 w-4">
|
|
||||||
Pass the query string on to this target
|
|
||||||
</label>
|
|
||||||
<p class="text-xs text-gray-500 mt-1">Appends the query string each event arrived with to the URL above, after any query string the URL already has.</p>
|
|
||||||
</div>
|
|
||||||
<div>
|
<div>
|
||||||
<div class="flex gap-2 items-center">
|
<div class="flex gap-2 items-center">
|
||||||
<label class="text-sm text-gray-700">Delivery attempts:</label>
|
<label class="text-sm text-gray-700">Delivery attempts:</label>
|
||||||
@@ -260,7 +249,7 @@
|
|||||||
{{range .Targets}}
|
{{range .Targets}}
|
||||||
<div class="p-4">
|
<div class="p-4">
|
||||||
<div class="flex flex-wrap items-center justify-between gap-2 mb-1">
|
<div class="flex flex-wrap items-center justify-between gap-2 mb-1">
|
||||||
<span class="text-sm font-medium text-gray-900 wrap-anywhere">{{.Name}}</span>
|
<span class="text-sm font-medium text-gray-900">{{.Name}}</span>
|
||||||
<div class="flex flex-wrap items-center gap-2">
|
<div class="flex flex-wrap items-center gap-2">
|
||||||
<span class="badge-info">{{if eq .Type "database"}}archive{{else}}{{.Type}}{{end}}</span>
|
<span class="badge-info">{{if eq .Type "database"}}archive{{else}}{{.Type}}{{end}}</span>
|
||||||
{{if .Active}}
|
{{if .Active}}
|
||||||
|
|||||||
@@ -3,15 +3,10 @@
|
|||||||
{{define "title"}}Full Event Log - {{.Webhook.Name}} - Webhooker{{end}}
|
{{define "title"}}Full Event Log - {{.Webhook.Name}} - Webhooker{{end}}
|
||||||
|
|
||||||
{{define "content"}}
|
{{define "content"}}
|
||||||
<!-- wrap-anywhere goes only on names, IDs and content types: a row too
|
|
||||||
wide for a phone must still run past the edge, where the browser
|
|
||||||
test sees it, rather than break its statuses, times or controls
|
|
||||||
mid-word. So a target's name in an event's row, which shares its
|
|
||||||
element with the delivery's status, goes without. -->
|
|
||||||
<div class="max-w-6xl mx-auto px-6 py-8">
|
<div class="max-w-6xl mx-auto px-6 py-8">
|
||||||
<div class="mb-6">
|
<div class="mb-6">
|
||||||
<a href="/hook/{{.Webhook.ID}}" class="btn-small wrap-anywhere">← Back to {{.Webhook.Name}}</a>
|
<a href="/hook/{{.Webhook.ID}}" class="btn-small">← Back to {{.Webhook.Name}}</a>
|
||||||
<div class="flex flex-wrap justify-between items-center gap-2 mt-2">
|
<div class="flex justify-between items-center mt-2">
|
||||||
<h1 class="text-2xl font-medium text-gray-900">Full Event Log</h1>
|
<h1 class="text-2xl font-medium text-gray-900">Full Event Log</h1>
|
||||||
<!-- Under a filter, this counts the events the filter lists. -->
|
<!-- Under a filter, this counts the events the filter lists. -->
|
||||||
<span class="text-sm text-gray-500">{{if gt .TotalEvents (len .Events)}}{{len .Events}} most recent of {{.TotalEvents}} events{{else}}{{.TotalEvents}}{{if not .Show}} total{{end}} event{{if ne .TotalEvents 1}}s{{end}}{{end}}{{if eq .Show "failed"}} with a failed delivery{{else if eq .Show "pending"}} with a delivery pending or retrying{{end}}</span>
|
<span class="text-sm text-gray-500">{{if gt .TotalEvents (len .Events)}}{{len .Events}} most recent of {{.TotalEvents}} events{{else}}{{.TotalEvents}}{{if not .Show}} total{{end}} event{{if ne .TotalEvents 1}}s{{end}}{{end}}{{if eq .Show "failed"}} with a failed delivery{{else if eq .Show "pending"}} with a delivery pending or retrying{{end}}</span>
|
||||||
@@ -33,8 +28,8 @@
|
|||||||
<div role="button" tabindex="0" class="btn-small w-full flex flex-wrap justify-between gap-2" :aria-expanded="open" @mousedown="cancelPendingToggle" @click="toggleUnlessSelecting" @keydown.enter.prevent="toggle" @keydown.space.prevent="toggle">
|
<div role="button" tabindex="0" class="btn-small w-full flex flex-wrap justify-between gap-2" :aria-expanded="open" @mousedown="cancelPendingToggle" @click="toggleUnlessSelecting" @keydown.enter.prevent="toggle" @keydown.space.prevent="toggle">
|
||||||
<span class="flex flex-wrap items-center gap-3">
|
<span class="flex flex-wrap items-center gap-3">
|
||||||
<span class="badge-info">{{.Method}}</span>
|
<span class="badge-info">{{.Method}}</span>
|
||||||
<span class="text-sm font-mono text-gray-700 wrap-anywhere">{{.ID}}</span>
|
<span class="text-sm font-mono text-gray-700">{{.ID}}</span>
|
||||||
<span class="text-sm text-gray-500 wrap-anywhere">{{.ContentType}}</span>
|
<span class="text-sm text-gray-500">{{.ContentType}}</span>
|
||||||
{{if .ResubmittedFrom}}
|
{{if .ResubmittedFrom}}
|
||||||
<span class="text-xs text-gray-500" title="This event is a copy of {{.ResubmittedFromID}}">resubmitted copy</span>
|
<span class="text-xs text-gray-500" title="This event is a copy of {{.ResubmittedFromID}}">resubmitted copy</span>
|
||||||
{{end}}
|
{{end}}
|
||||||
@@ -59,7 +54,7 @@
|
|||||||
<div x-show="open" x-cloak class="mt-3 p-3 bg-gray-50 rounded-md">
|
<div x-show="open" x-cloak class="mt-3 p-3 bg-gray-50 rounded-md">
|
||||||
<div class="mb-3 flex flex-wrap items-center justify-between gap-2">
|
<div class="mb-3 flex flex-wrap items-center justify-between gap-2">
|
||||||
<div class="text-xs text-gray-500">
|
<div class="text-xs text-gray-500">
|
||||||
{{if .ResubmittedFrom}}Resubmitted from event <a href="/hook/{{$.Webhook.ID}}/events/{{.ResubmittedFromID}}" class="btn-small font-mono wrap-anywhere">{{.ResubmittedFromID}}</a>.{{end}}
|
{{if .ResubmittedFrom}}Resubmitted from event <a href="/hook/{{$.Webhook.ID}}/events/{{.ResubmittedFromID}}" class="btn-small font-mono">{{.ResubmittedFromID}}</a>.{{end}}
|
||||||
{{if .ResubmitCount}}Resubmitted as {{.ResubmitCount}} new event{{if ne .ResubmitCount 1}}s{{end}}.{{end}}
|
{{if .ResubmitCount}}Resubmitted as {{.ResubmitCount}} new event{{if ne .ResubmitCount 1}}s{{end}}.{{end}}
|
||||||
</div>
|
</div>
|
||||||
<form method="POST" action="/hook/{{$.Webhook.ID}}/events/{{.ID}}/resubmit" class="inline">
|
<form method="POST" action="/hook/{{$.Webhook.ID}}/events/{{.ID}}/resubmit" class="inline">
|
||||||
|
|||||||
@@ -47,14 +47,6 @@
|
|||||||
<input type="number" id="timeout" name="timeout" value="{{.TargetForm.Timeout}}" min="0" max="{{.MaxTimeout}}" class="input">
|
<input type="number" id="timeout" name="timeout" value="{{.TargetForm.Timeout}}" min="0" max="{{.MaxTimeout}}" class="input">
|
||||||
<p class="text-xs text-gray-500 mt-1">Per-request timeout, at most {{.MaxTimeout}} seconds. Leave blank to use the default.</p>
|
<p class="text-xs text-gray-500 mt-1">Per-request timeout, at most {{.MaxTimeout}} seconds. Leave blank to use the default.</p>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
<div class="form-group">
|
|
||||||
<label class="flex items-center gap-2 text-sm font-medium text-gray-700">
|
|
||||||
<input type="checkbox" id="forward_query" name="forward_query" value="on"{{if .TargetForm.ForwardQuery}} checked{{end}} class="h-4 w-4">
|
|
||||||
Pass the query string on to this target
|
|
||||||
</label>
|
|
||||||
<p class="text-xs text-gray-500 mt-1">Appends the query string each event arrived with to the destination URL, after any query string the URL already has.</p>
|
|
||||||
</div>
|
|
||||||
{{end}}
|
{{end}}
|
||||||
|
|
||||||
{{if eq .Target.Type "slack"}}
|
{{if eq .Target.Type "slack"}}
|
||||||
|
|||||||
Reference in New Issue
Block a user