sneak/webhooker
1
0
Fork 0
Code Issues 34 Pull Requests 1 Actions Packages Projects Releases Wiki Activity

Compare commits

base: sneak:88f3e1d019d45a4f3f1bba00e832f1da37439242
Branches Tags
sneak:main sneak:next sneak:feat/receiver-rate-limit sneak:issue-70-log-target
..
compare: sneak:e875c3ef22931228e425637c3c7e9071854c2506
Branches Tags
sneak:next sneak:main sneak:feat/receiver-rate-limit sneak:issue-70-log-target

1 Commits
88f3e1d019 ... e875c3ef22

Author SHA1 Message Date
sneak
e875c3ef22 Mark superseded commits honestly instead of skipped (closes #152)
All checks were successful
check / check (push) Successful in 3m5s
Details 
Gitea cancels an in-flight run when a newer commit lands on the same
branch and records the cancellation as `failure` / "Has been
cancelled". The workflow rewrote that to `skipped`, but Gitea's
Combine() folds `skipped` into `success`, so the combined-status API
returned green for a commit nothing had ever tested. Rewrite it to
`failure` / "Superseded by a newer commit; never tested" instead:
red-but-honest, and never `pending`, which would block the commit
forever.

Re-running the superseded commit would have been better still, but is
not reachable on this Gitea (1.25.4): its API exposes no rerun
endpoint, workflow dispatch takes a ref rather than a SHA, and every
replay would be a full uncached build with no bound on how many pile
up behind a burst of merges.

The step also stops hardcoding its status context: the logic moves into
script/ci-mark-superseded, which derives the context from the workflow
name, job name and event -- the same three values Gitea builds it from
-- and fails loudly when no status on the commit being built carries
that context, so renaming the workflow or the job cannot silently
disable the rewrite. That is item 2 of
#147; item 1 there is
untouched.

Tests drive the script against a fake Gitea covering the cancelled,
laundered-skipped, genuinely-failed, passing and renamed cases, so jq
joins the builder image to run them.
 2026-08-17 20:53:28 +00:00
20 changed files with 35 additions and 848 deletions
Expand all files Collapse all files

5
.dockerignore
View File

@@ -3,11 +3,6 @@
# stage of the Dockerfile. # stage of the Dockerfile.
.git/ .git/
bin/ bin/
# Third-party browser assets are fetched and hash-verified inside the build by
# script/fetch-assets. Excluding any host copy keeps a developer's working tree
# from supplying the bytes that get shipped. The script and its
# static/vendor.sha256 manifest stay in the context.
static/js/alpine.min.js
*.md *.md
LICENSE LICENSE
.editorconfig .editorconfig

5
.gitignore vendored
View File

@@ -45,8 +45,3 @@ temp/
# CI cache barrier, written into the build context by the check workflow # CI cache barrier, written into the build context by the check workflow
.ci-fingerprint .ci-fingerprint
# Third-party browser assets, fetched and hash-verified by
# script/fetch-assets against static/vendor.sha256. Not committed:
# REPO_POLICIES.md forbids minified bundles in version control.
/static/js/alpine.min.js

10
Dockerfile
View File

@@ -34,7 +34,7 @@ COPY --from=lint /src/go.sum /dev/null
# jq is a runtime dependency of script/ci-mark-superseded, which the test # jq is a runtime dependency of script/ci-mark-superseded, which the test
# suite executes. # suite executes.
RUN apt-get update && apt-get install -y --no-install-recommends make curl ca-certificates jq && rm -rf /var/lib/apt/lists/* RUN apt-get update && apt-get install -y --no-install-recommends make jq && rm -rf /var/lib/apt/lists/*
WORKDIR /build WORKDIR /build
@@ -46,14 +46,6 @@ RUN go mod download
# the lint stage above. # the lint stage above.
COPY . . COPY . .
# Fetch the third-party browser assets the UI serves. They are not committed
# (REPO_POLICIES.md forbids minified bundles in version control) and
# .dockerignore keeps any host copy out of the build context, so this step is
# the only way they enter the image. Each download is checked against a
# hardcoded sha256 and the build fails on mismatch; make test re-checks the
# hashes against the bytes go:embed actually put in the binary.
RUN script/fetch-assets
# Run tests and build # Run tests and build
RUN make test RUN make test
RUN make build RUN make build

5
Makefile
View File

@@ -1,4 +1,4 @@
.PHONY: bootstrap setup assets test lint fmt fmt-check check build run dev deps docker clean hooks css .PHONY: bootstrap setup test lint fmt fmt-check check build run dev deps docker clean hooks css
# Default target # Default target
.DEFAULT_GOAL := check .DEFAULT_GOAL := check
@@ -9,9 +9,6 @@ bootstrap:
setup: setup:
@script/setup @script/setup
assets:
@script/fetch-assets
test: test:
@script/test @script/test

24
README.md
View File

@@ -40,7 +40,6 @@ make docker
```bash ```bash
make bootstrap # Install all dependencies (idempotent) make bootstrap # Install all dependencies (idempotent)
make setup # Bootstrap + install git pre-commit hook make setup # Bootstrap + install git pre-commit hook
make assets # Fetch + verify third-party browser assets
make fmt # Format code (gofmt + goimports) make fmt # Format code (gofmt + goimports)
make lint # Run golangci-lint make lint # Run golangci-lint
make test # Run tests with race detection make test # Run tests with race detection
@@ -248,8 +247,6 @@ them. We provide:
- `script/setup` — make a fresh clone ready for development - `script/setup` — make a fresh clone ready for development
(bootstrap, then install-precommit) (bootstrap, then install-precommit)
- `script/projectname` — output the project name ("webhooker") - `script/projectname` — output the project name ("webhooker")
- `script/fetch-assets` — download the third-party browser assets into
`static/`, verifying each against its pinned sha256
- `script/test` — run the test suite - `script/test` — run the test suite
- `script/lint` — run golangci-lint - `script/lint` — run golangci-lint
- `script/fmt` — format all code (writes) - `script/fmt` — format all code (writes)
@@ -265,27 +262,6 @@ them. We provide:
- `script/install-precommit` — install the git pre-commit hook that - `script/install-precommit` — install the git pre-commit hook that
runs `script/precommit` runs `script/precommit`
## Third-party browser assets
The web UI serves one third-party script, Alpine.js. It is **not** committed:
a minified bundle in the tree is unreviewable, and `REPO_POLICIES.md` bars
both committed build artifacts and unpinned external references.
Instead `script/fetch-assets` downloads it from a pinned URL, checks the
download against a hardcoded sha256, and installs it under `static/`. The
sha256 of every installed asset is recorded in `static/vendor.sha256`, and
`static/vendor_test.go` re-hashes the bytes `go:embed` put in the binary
against that manifest — so the pin is enforced on what actually ships, not
merely written down. Any mismatch fails the build.
`make bootstrap` runs the fetch for local development, and the Dockerfile
runs it in the build stage; `.gitignore` and `.dockerignore` keep the
artifact out of both the repo and the build context.
To move to a new version: update the version, URL, and tarball sha256 in
`script/fetch-assets` and the asset sha256 in `static/vendor.sha256`, then
run `make assets && make check`.
## Rationale ## Rationale
Webhook integrations between services are inherently fragile. The Webhook integrations between services are inherently fragile. The

43
internal/ciscript/ci_mark_superseded_test.go
View File

@@ -187,47 +187,6 @@ func TestMarkSupersededRejectsAnUnknownContext(t *testing.T) {
require.Empty(t, fake.postedFor(history.parent)) require.Empty(t, fake.postedFor(history.parent))
} }
// ANCESTOR_LIMIT is a documented knob. A value that is set but unusable
// must abort: handing it to git and discarding the exit status left the
// walk empty and the step green, marking nothing.
func TestMarkSupersededRejectsAnUnparseableAncestorLimit(t *testing.T) {
t.Parallel()
requireTools(t)
history := newRepo(t)
fake, api := newFakeGitea(t)
fake.setStatus(history.head, running())
fake.setStatus(history.parent, cancelled())
out, err := runScript(
t, history, api, defaultEnv(), "ANCESTOR_LIMIT=twenty",
)
require.Error(t, err)
require.Contains(t, out, "ANCESTOR_LIMIT")
require.Contains(t, out, "twenty")
require.Empty(t, fake.postedFor(history.parent))
}
// A status read that fails is not the same as a commit with nothing to
// do. Losing curl's exit status through a pipe made the two identical
// and left a laundered commit laundered with no signal.
func TestMarkSupersededFailsOnAnUnreadableAncestorStatus(t *testing.T) {
t.Parallel()
requireTools(t)
history := newRepo(t)
fake, api := newFakeGitea(t)
fake.setStatus(history.head, running())
fake.setStatus(history.parent, cancelled())
fake.failStatusRead(history.parent)
out, err := runScript(t, history, api, defaultEnv())
require.Error(t, err)
require.Contains(t, out, history.parent)
require.Contains(t, out, "cannot read commit statuses")
require.Empty(t, fake.postedFor(history.parent))
}
// The derived context must equal the one Gitea actually uses, which is // The derived context must equal the one Gitea actually uses, which is
// built from the same workflow and job names. // built from the same workflow and job names.
func TestDerivedContextMatchesGitea(t *testing.T) { func TestDerivedContextMatchesGitea(t *testing.T) {
@@ -276,7 +235,6 @@ func workflowIdentity(t *testing.T) (string, string) {
func runScript( func runScript(
t *testing.T, history repo, api string, env scriptEnv, t *testing.T, history repo, api string, env scriptEnv,
extra ...string,
) (string, error) { ) (string, error) {
t.Helper() t.Helper()
@@ -295,7 +253,6 @@ func runScript(
"GITHUB_EVENT_NAME="+env.event, "GITHUB_EVENT_NAME="+env.event,
"GITEA_TOKEN=test-token", "GITEA_TOKEN=test-token",
) )
cmd.Env = append(cmd.Env, extra...)
out, err := cmd.CombinedOutput() out, err := cmd.CombinedOutput()

22
internal/ciscript/fakegitea_test.go
View File

@@ -31,9 +31,6 @@ type fakeGitea struct {
mu sync.Mutex mu sync.Mutex
statuses map[string][]commitStatus statuses map[string][]commitStatus
posted map[string][]postedStatus posted map[string][]postedStatus
// failRead is a commit whose combined-status read answers HTTP
// 500, standing in for a status API that is down.
failRead string
} }
// newFakeGitea returns the fake and the base URL to hand the script as // newFakeGitea returns the fake and the base URL to hand the script as
@@ -45,7 +42,6 @@ func newFakeGitea(t *testing.T) (*fakeGitea, string) {
mu: sync.Mutex{}, mu: sync.Mutex{},
statuses: map[string][]commitStatus{}, statuses: map[string][]commitStatus{},
posted: map[string][]postedStatus{}, posted: map[string][]postedStatus{},
failRead: "",
} }
srv := httptest.NewServer(fake.routes()) srv := httptest.NewServer(fake.routes())
@@ -75,16 +71,9 @@ func (f *fakeGitea) handleCombined(
f.mu.Lock() f.mu.Lock()
defer f.mu.Unlock() defer f.mu.Unlock()
sha := r.PathValue("sha")
if f.failRead != "" && f.failRead == sha {
http.Error(w, "boom", http.StatusInternalServerError)
return
}
body := struct { body := struct {
Statuses []commitStatus `json:"statuses"` Statuses []commitStatus `json:"statuses"`
}{Statuses: f.statuses[sha]} }{Statuses: f.statuses[r.PathValue("sha")]}
payload, err := json.Marshal(body) payload, err := json.Marshal(body)
if err != nil { if err != nil {
@@ -123,15 +112,6 @@ func (f *fakeGitea) handleCreate(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusCreated) w.WriteHeader(http.StatusCreated)
} }
// failStatusRead makes the combined-status read for one commit answer
// HTTP 500.
func (f *fakeGitea) failStatusRead(sha string) {
f.mu.Lock()
defer f.mu.Unlock()
f.failRead = sha
}
// setStatus gives a commit its latest status for a context. // setStatus gives a commit its latest status for a context.
func (f *fakeGitea) setStatus(sha string, status commitStatus) { func (f *fakeGitea) setStatus(sha string, status commitStatus) {
f.mu.Lock() f.mu.Lock()

120
internal/handlers/event_log_view.go
View File

@@ -1,120 +0,0 @@
package handlers
import (
"time"
"unicode/utf8"
)
// maxRenderedBodyBytes caps how many bytes of a stored event
// body reach the event log page. Bodies come from the
// unauthenticated receiver under the 1 MB ingest cap and
// renderTemplate buffers a whole page before writing it, so
// an uncapped page of paginationPerPage events is tens of
// megabytes of resident memory per concurrent viewer.
const maxRenderedBodyBytes = 8192
// eventLogColumns is the event log's projection. The casts to
// blob are load-bearing: they make substr and length count
// bytes rather than characters, so the cap bounds the page in
// bytes whatever the payload's encoding. Cutting in SQLite
// rather than in Go is the point of the projection — an
// oversized body never becomes a Go string at all.
const eventLogColumns = "id, created_at, method, content_type, " +
"substr(cast(body as blob), 1, ?) AS body, " +
"length(cast(body as blob)) AS body_bytes"
// EventLogView is the display-safe projection of an event for
// the event log page, alongside DeliveryView and TargetView.
// It carries a capped body plus the true stored size, so the
// page can mark a body as truncated without ever holding the
// whole thing.
type EventLogView struct {
ID string
CreatedAt time.Time
Method string
ContentType string
// Body holds at most maxRenderedBodyBytes bytes of the
// stored body.
Body string
// BodyBytes is the true size of the stored body.
BodyBytes int64
// BodyTruncated reports that the stored body was larger
// than the cap, so the page owes the reader a marker.
BodyTruncated bool
Deliveries []DeliveryView
}
// BodyShownBytes is how many body bytes the page is actually
// rendering, which the truncation marker reports beside the
// true size.
func (v EventLogView) BodyShownBytes() int {
return len(v.Body)
}
// eventLogRow is one row of the event log projection. Its
// body column arrives already cut to the cap by SQLite, with
// the true size beside it.
type eventLogRow struct {
ID string
CreatedAt time.Time
Method string
ContentType string
Body []byte
BodyBytes int64
}
// view projects a loaded row for rendering.
func (r *eventLogRow) view() EventLogView {
body := r.Body
truncated := r.BodyBytes > int64(len(body))
// Only a cut body can have been left mid-sequence by
// this query. A whole body is passed through exactly as
// stored, however malformed.
if truncated {
body = trimPartialRune(body)
}
return EventLogView{
ID: r.ID,
CreatedAt: r.CreatedAt,
Method: r.Method,
ContentType: r.ContentType,
Body: string(body),
BodyBytes: r.BodyBytes,
BodyTruncated: truncated,
}
}
// trimPartialRune drops a trailing UTF-8 sequence that the
// byte-wise cut left incomplete, so a multi-byte rune severed
// at the cap does not surface as a mojibake tail.
//
// Bytes that are merely invalid UTF-8 are left exactly as
// stored: this service receives binary payloads, and rewriting
// them would misreport what was delivered. The distinction is
// utf8.FullRune's — it reports a complete sequence for an
// invalid encoding too, since that decodes to a width-1 error
// rune, so only a valid prefix still waiting for its
// continuation bytes is removed. A tail with no rune start in
// its last utf8.UTFMax bytes cannot be an incomplete sequence
// either, and is likewise left alone.
func trimPartialRune(b []byte) []byte {
for i := len(b) - 1; i >= 0 && len(b)-i <= utf8.UTFMax; i-- {
if !utf8.RuneStart(b[i]) {
continue
}
if utf8.FullRune(b[i:]) {
return b
}
return b[:i]
}
return b
}

258
internal/handlers/event_log_view_test.go
View File

@@ -1,258 +0,0 @@
package handlers_test
import (
"net/http"
"net/http/httptest"
"strconv"
"strings"
"testing"
"unicode/utf8"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"gorm.io/gorm/clause"
"sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/handlers"
"sneak.berlin/go/webhooker/internal/session"
)
// bodyCap is the number of body bytes the event log page is
// allowed to render for one event.
const bodyCap = handlers.MaxRenderedBodyBytesForTest
// snowman is a three-byte rune, so a body of them straddles the
// byte-wise cut: bodyCap is not a multiple of three.
const snowman = "☃"
// seedEventWithBody records one event with the given body in the
// webhook's own database.
func seedEventWithBody(
t *testing.T,
dbMgr *database.WebhookDBManager,
webhookID string,
body string,
) {
t.Helper()
webhookDB, err := dbMgr.GetDB(webhookID)
require.NoError(t, err)
event := &database.Event{
WebhookID: webhookID,
Method: http.MethodPost,
Body: body,
ContentType: "application/octet-stream",
}
require.NoError(t, webhookDB.Omit(
clause.Associations,
).Create(event).Error)
}
// seedAndProject stores one body and returns the projection the
// event log page would be handed for it.
func seedAndProject(
t *testing.T,
body string,
) handlers.EventLogView {
t.Helper()
var (
h *handlers.Handlers
sess *session.Session
db *database.Database
dbMgr *database.WebhookDBManager
)
app := newTestApp(t, &h, &sess, &db, &dbMgr)
app.RequireStart()
t.Cleanup(app.RequireStop)
wh := seedWebhook(t, db)
seedEventWithBody(t, dbMgr, wh.ID, body)
views := h.LoadEventLogViewsForTest(
httptest.NewRecorder(), *wh, 1,
)
require.Len(t, views, 1)
return views[0]
}
// TestHandleSourceLogs_BoundsOversizeBody proves the rendered
// page is bounded by the cap rather than by the stored payload:
// the body here is 64 times the cap, and the ingest path would
// accept twice as much again.
func TestHandleSourceLogs_BoundsOversizeBody(t *testing.T) {
t.Parallel()
var (
h *handlers.Handlers
sess *session.Session
db *database.Database
dbMgr *database.WebhookDBManager
)
app := newTestApp(t, &h, &sess, &db, &dbMgr)
app.RequireStart()
t.Cleanup(app.RequireStop)
const (
sentinel = "TAIL-SENTINEL-1f4a9c"
storedBytes = 512 * 1024
)
wh := seedWebhook(t, db)
seedEventWithBody(
t, dbMgr, wh.ID,
strings.Repeat("A", storedBytes-len(sentinel))+sentinel,
)
page := renderSourceLogsPage(t, h, sess, wh.ID)
// Nothing past the cap reaches the page, and the whole page
// stays far below the stored body it is reporting on.
assert.NotContains(t, page, sentinel)
assert.Less(t, len(page), 4*bodyCap)
// The marker states the true stored size, not the cut one.
assert.Contains(
t, page,
"showing "+strconv.Itoa(bodyCap)+
" of "+strconv.Itoa(storedBytes)+" bytes",
)
}
// TestHandleSourceLogs_SmallBodyRendersWhole guards the other
// side of the cap: a body under it is shown in full and carries
// no truncation marker.
func TestHandleSourceLogs_SmallBodyRendersWhole(t *testing.T) {
t.Parallel()
var (
h *handlers.Handlers
sess *session.Session
db *database.Database
dbMgr *database.WebhookDBManager
)
app := newTestApp(t, &h, &sess, &db, &dbMgr)
app.RequireStart()
t.Cleanup(app.RequireStop)
wh := seedWebhook(t, db)
seedEventWithBody(t, dbMgr, wh.ID, `{"kept":"whole"}`)
page := renderSourceLogsPage(t, h, sess, wh.ID)
assert.Contains(t, page, "&#34;kept&#34;")
assert.NotContains(t, page, "Body truncated for display")
}
// TestEventLogView_CutMidRune proves a multi-byte rune severed
// by the byte-wise cut is dropped rather than surfaced as a
// mojibake tail.
func TestEventLogView_CutMidRune(t *testing.T) {
t.Parallel()
body := strings.Repeat(snowman, 4096)
view := seedAndProject(t, body)
// bodyCap bytes hold bodyCap/3 whole snowmen and two bytes
// of the next one; those two are dropped.
whole := bodyCap / len(snowman)
assert.True(t, view.BodyTruncated)
assert.Equal(t, int64(len(body)), view.BodyBytes)
assert.Equal(t, strings.Repeat(snowman, whole), view.Body)
assert.True(t, utf8.ValidString(view.Body))
assert.LessOrEqual(t, len(view.Body), bodyCap)
}
// TestEventLogView_BinaryBodyLeftAsStored proves a binary
// payload is passed through byte for byte. Its tail is invalid
// UTF-8 however the cut falls, so repairing it would misreport
// what the sender delivered.
func TestEventLogView_BinaryBodyLeftAsStored(t *testing.T) {
t.Parallel()
raw := make([]byte, bodyCap+808)
for i := range raw {
// 0x80..0xBF: continuation bytes, never a rune start.
raw[i] = 0x80 | byte(i%0x40)
}
view := seedAndProject(t, string(raw))
assert.True(t, view.BodyTruncated)
assert.Equal(t, int64(len(raw)), view.BodyBytes)
assert.Equal(t, string(raw[:bodyCap]), view.Body)
assert.False(t, utf8.ValidString(view.Body))
}
// TestTrimPartialRune covers the distinction the cut repair
// turns on: an incomplete but valid sequence is dropped, while
// bytes that are merely invalid UTF-8 are left alone.
func TestTrimPartialRune(t *testing.T) {
t.Parallel()
cases := []struct {
name string
in []byte
want []byte
}{{
name: "complete ascii",
in: []byte("abc"),
want: []byte("abc"),
}, {
name: "complete multibyte",
in: []byte("ab" + snowman),
want: []byte("ab" + snowman),
}, {
name: "two byte rune cut",
in: []byte{'a', 0xC3},
want: []byte{'a'},
}, {
name: "three byte rune cut after one",
in: []byte{'a', 0xE2},
want: []byte{'a'},
}, {
name: "three byte rune cut after two",
in: []byte{'a', 0xE2, 0x98},
want: []byte{'a'},
}, {
name: "four byte rune cut",
in: []byte{'a', 0xF0, 0x9F, 0x92}, // U+1F4A9 cut
want: []byte{'a'},
}, {
name: "invalid start byte kept",
in: []byte{'a', 0xFF},
want: []byte{'a', 0xFF},
}, {
name: "orphan continuation bytes kept",
in: []byte{0x80, 0x81, 0x82, 0x83, 0x84},
want: []byte{0x80, 0x81, 0x82, 0x83, 0x84},
}, {
name: "truncated sequence followed by junk kept",
in: []byte{0xE2, 0x98, 0xFF},
want: []byte{0xE2, 0x98, 0xFF},
}, {
name: "empty",
in: []byte{},
want: []byte{},
}}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
assert.Equal(
t, tc.want,
handlers.TrimPartialRuneForTest(tc.in),
)
})
}
}

27
internal/handlers/export_test.go
View File

@@ -3,35 +3,8 @@ package handlers
import ( import (
"html/template" "html/template"
"net/http" "net/http"
"sneak.berlin/go/webhooker/internal/database"
) )
// MaxRenderedBodyBytesForTest exposes the event log's body cap
// to the handlers_test package.
const MaxRenderedBodyBytesForTest = maxRenderedBodyBytes
// TrimPartialRuneForTest exposes trimPartialRune for use in the
// handlers_test package.
func TrimPartialRuneForTest(b []byte) []byte {
return trimPartialRune(b)
}
// LoadEventLogViewsForTest exposes loadEventsWithDeliveries for
// use in the handlers_test package. Assertions on the projected
// body need the bytes as loaded: html/template rewrites invalid
// UTF-8 on the way out, so the rendered page cannot show whether
// a binary body survived the projection intact.
func (s *Handlers) LoadEventLogViewsForTest(
w http.ResponseWriter,
webhook database.Webhook,
page int,
) []EventLogView {
views, _ := s.loadEventsWithDeliveries(w, webhook, nil, page)
return views
}
// AddTemplateForTest registers a template under a page name so that // AddTemplateForTest registers a template under a page name so that
// the handlers_test package can drive the render path with a // the handlers_test package can drive the render path with a
// template of its own. // template of its own.

6
internal/handlers/handlers.go
View File

@@ -229,10 +229,8 @@ func (s *Handlers) renderTemplate(
// the response only once rendering has fully succeeded. Executing // the response only once rendering has fully succeeded. Executing
// straight into the ResponseWriter commits a partial body and a 200 // straight into the ResponseWriter commits a partial body and a 200
// status before a mid-render error can be reported, leaving no way // status before a mid-render error can be reported, leaving no way
// to serve a 500. Buffering makes a page's rendered size resident // to serve a 500. These pages are small, so holding one in memory is
// memory per concurrent viewer, so every page owes it a bound: the // the right trade.
// event log caps each stored body at maxRenderedBodyBytes for exactly
// this reason.
func (s *Handlers) executeTemplate( func (s *Handlers) executeTemplate(
w http.ResponseWriter, w http.ResponseWriter,
tmpl *template.Template, tmpl *template.Template,

31
internal/handlers/source_management.go
View File

@@ -92,6 +92,13 @@ func parseRetentionDays(raw string, fallback int) (int, error) {
return v, nil return v, nil
} }
// EventWithDeliveries holds an event and its deliveries.
type EventWithDeliveries struct {
database.Event
Deliveries []DeliveryView
}
// DeliveryView is the display-safe projection of a delivery // DeliveryView is the display-safe projection of a delivery
// for the event log page. Its target is a TargetView, so the // for the event log page. Its target is a TargetView, so the
// stored configuration blob — which holds the target's // stored configuration blob — which holds the target's
@@ -808,18 +815,16 @@ func (h *Handlers) parsePage(r *http.Request) int {
} }
// loadEventsWithDeliveries loads paginated events and their // loadEventsWithDeliveries loads paginated events and their
// deliveries from the per-webhook database. Events come back // deliveries from the per-webhook database.
// as capped projections rather than database.Event rows: see
// eventLogColumns for why the cut happens in SQL.
func (h *Handlers) loadEventsWithDeliveries( func (h *Handlers) loadEventsWithDeliveries(
w http.ResponseWriter, w http.ResponseWriter,
webhook database.Webhook, webhook database.Webhook,
targetMap map[string]delivery.TargetView, targetMap map[string]delivery.TargetView,
page int, page int,
) ([]EventLogView, int64) { ) ([]EventWithDeliveries, int64) {
var totalEvents int64 var totalEvents int64
var result []EventLogView var result []EventWithDeliveries
if !h.dbMgr.DBExists(webhook.ID) { if !h.dbMgr.DBExists(webhook.ID) {
return result, totalEvents return result, totalEvents
@@ -840,25 +845,23 @@ func (h *Handlers) loadEventsWithDeliveries(
offset := (page - 1) * paginationPerPage offset := (page - 1) * paginationPerPage
var rows []eventLogRow var events []database.Event
webhookDB.Model(&database.Event{}).Select( webhookDB.Where(
eventLogColumns, maxRenderedBodyBytes,
).Where(
"webhook_id = ?", webhook.ID, "webhook_id = ?", webhook.ID,
).Order("created_at DESC").Offset(offset).Limit( ).Order("created_at DESC").Offset(offset).Limit(
paginationPerPage, paginationPerPage,
).Find(&rows) ).Find(&events)
result = make([]EventLogView, len(rows)) result = make([]EventWithDeliveries, len(events))
for i := range rows { for i := range events {
result[i] = rows[i].view() result[i].Event = events[i]
var deliveries []database.Delivery var deliveries []database.Delivery
webhookDB.Where( webhookDB.Where(
"event_id = ?", rows[i].ID, "event_id = ?", events[i].ID,
).Find(&deliveries) ).Find(&deliveries)
result[i].Deliveries = newDeliveryViews( result[i].Deliveries = newDeliveryViews(

50
internal/server/static_assets_test.go
View File

@@ -1,50 +0,0 @@
package server_test
import (
"net/http"
"regexp"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"sneak.berlin/go/webhooker/templates"
)
// TestBaseTemplateScriptsAreServed walks every /s/ script the base
// template loads on each page and fetches it through the real router.
// Alpine.js is fetched at build time rather than committed, so nothing
// in the repo guarantees it is present: this is the check that the page
// still gets the JavaScript it asks for.
func TestBaseTemplateScriptsAreServed(t *testing.T) {
t.Parallel()
// scriptSrc matches the src of every <script> tag pointing at the
// /s/ static mount.
scriptSrc := regexp.MustCompile(`<script[^>]+src="(/s/[^"]+)"`)
base, err := templates.Templates.ReadFile("base.html")
require.NoError(t, err)
matches := scriptSrc.FindAllStringSubmatch(string(base), -1)
require.NotEmpty(t, matches, "base.html should load scripts from /s/")
env := newTestEnv(t)
for _, m := range matches {
src := m[1]
t.Run(src, func(t *testing.T) {
t.Parallel()
w := env.get(src, nil)
require.Equalf(
t, http.StatusOK, w.Code,
"base.html loads %s but the server does not serve it", src,
)
assert.NotEmptyf(
t, w.Body.Bytes(), "%s is served but empty", src,
)
})
}
}

8
script/bootstrap
View File

@@ -5,8 +5,7 @@
# or apk (detected in that order); assumes NOTHING is present (not git, # or apk (detected in that order); assumes NOTHING is present (not git,
# make, or go). golangci-lint is packaged in nix, brew, and apk; on apt # make, or go). golangci-lint is packaged in nix, brew, and apk; on apt
# it is installed from a hash-verified GitHub release archive (never # it is installed from a hash-verified GitHub release archive (never
# curl | sh). Finishes by running script/fetch-assets, which installs the # curl | sh).
# hash-pinned third-party browser assets the repo does not commit.
set -eu set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)" ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
@@ -116,11 +115,6 @@ main() {
go mod download go mod download
# Third-party browser assets are not committed; fetch and verify them
# so a fresh clone can build and test.
if missing curl; then pkg_install curl curl curl curl; fi
"$ROOT/script/fetch-assets"
echo "bootstrap complete" echo "bootstrap complete"
} }

62
script/ci-mark-superseded
View File

@@ -12,49 +12,19 @@
# Called by the Gitea Actions workflow, which supplies GITHUB_API_URL, # Called by the Gitea Actions workflow, which supplies GITHUB_API_URL,
# GITHUB_REPOSITORY, GITHUB_SHA, GITHUB_WORKFLOW, GITHUB_JOB, # GITHUB_REPOSITORY, GITHUB_SHA, GITHUB_WORKFLOW, GITHUB_JOB,
# GITHUB_EVENT_NAME and GITEA_TOKEN. ANCESTOR_LIMIT (default 20) caps how # GITHUB_EVENT_NAME and GITEA_TOKEN. ANCESTOR_LIMIT (default 20) caps how
# far back the walk looks; a value that is set but not a positive integer # far back the walk looks.
# aborts rather than silently disabling the walk.
set -eu set -eu
SUPERSEDED_DESC='Superseded by a newer commit; never tested' SUPERSEDED_DESC='Superseded by a newer commit; never tested'
# Gitea builds the commit-status context as # Gitea builds the commit-status context as
# "<workflow name> / <job name> (<event>)", so derive it rather than # "<workflow name> / <job name> (<event>)", the same three values the
# hardcoding the result. # runner exports, so derive it rather than hardcoding the result.
#
# The derivation is deliberately not byte-exact with Gitea's own rule and
# must not be "fixed" into a silent fallback. Gitea uses the job's `name:`
# (falling back to the job id) and the workflow's `name:` (falling back to
# the workflow filename), while the runner exports GITHUB_JOB as the job
# *id* and GITHUB_WORKFLOW as the parsed workflow `name:`. So giving the
# job a display `name:`, or dropping the workflow's `name:`, makes the
# derived context stop matching --- and require_own_context below then
# turns every push red with a message. That loud failure is the point
# (https://git.eeqj.de/sneak/webhooker/issues/147 item 2); guessing at a
# fallback would restore the silent no-op it replaced.
context() { context() {
printf '%s / %s (%s)' \ printf '%s / %s (%s)' \
"$GITHUB_WORKFLOW" "$GITHUB_JOB" "$GITHUB_EVENT_NAME" "$GITHUB_WORKFLOW" "$GITHUB_JOB" "$GITHUB_EVENT_NAME"
} }
# ANCESTOR_LIMIT is a documented knob, so a value that is set but
# unusable must fail loudly instead of defaulting
# (https://git.eeqj.de/sneak/webhooker/issues/80). Passing it straight to
# git would print `fatal: not an integer` into a discarded exit status
# and mark nothing.
ancestor_limit() {
_limit="${ANCESTOR_LIMIT:-20}"
case "$_limit" in
'' | *[!0-9]* | 0*)
echo "ANCESTOR_LIMIT must be a positive integer," \
"got '${_limit}'" >&2
return 1
;;
esac
printf '%s' "$_limit"
}
# The status Gitea created for this very job proves which context string # The status Gitea created for this very job proves which context string
# it uses. If the derived one is missing, the workflow or the job was # it uses. If the derived one is missing, the workflow or the job was
# renamed and the match below would silently stop firing, restoring the # renamed and the match below would silently stop firing, restoring the
@@ -76,18 +46,8 @@ require_own_context() {
} }
# Latest status for our context on a commit, as "state|description". # Latest status for our context on a commit, as "state|description".
# The read is retried and bounded, and a read that still fails aborts the
# step: a laundered commit that cannot be read is not the same as one
# with nothing to do, and piping curl into jq would discard the
# difference.
status_of() { status_of() {
if ! _sbody="$(curl -sf --retry 3 --retry-delay 2 --max-time 30 \ curl -sf "${1}/commits/${2}/status" | jq -r --arg c "$3" \
"${1}/commits/${2}/status")"; then
echo "cannot read commit statuses for ${2}" >&2
return 1
fi
printf '%s' "$_sbody" | jq -r --arg c "$3" \
'[(.statuses // [])[] | select(.context == $c)][0] // empty '[(.statuses // [])[] | select(.context == $c)][0] // empty
| "\(.status)|\(.description)"' | "\(.status)|\(.description)"'
} }
@@ -105,20 +65,10 @@ main() {
_api="${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}" _api="${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}"
_ctx="$(context)" _ctx="$(context)"
_limit="$(ancestor_limit)"
require_own_context "$_api" "$_ctx" require_own_context "$_api" "$_ctx"
# A root commit legitimately has no ancestors; every other rev-list _walk="$(git rev-list \
# failure (a shallow clone, an unknown SHA) must abort, so the walk --max-count="${ANCESTOR_LIMIT:-20}" "${GITHUB_SHA}^" || true)"
# itself carries no `|| true`.
if ! git rev-parse -q --verify "${GITHUB_SHA}^" >/dev/null; then
echo "no ancestor of ${GITHUB_SHA} to check"
return 0
fi
_walk="$(git rev-list --max-count="$_limit" "${GITHUB_SHA}^")"
for _sha in $_walk; do for _sha in $_walk; do
_latest="$(status_of "$_api" "$_sha" "$_ctx")" _latest="$(status_of "$_api" "$_sha" "$_ctx")"

104
script/fetch-assets
View File

@@ -1,104 +0,0 @@
#!/bin/sh
# script/fetch-assets: download the third-party browser assets the web UI
# ships and install them under static/. Minified bundles are not committed
# (REPO_POLICIES.md: no build artifacts in version control), so the build
# fetches them here. Every download is verified against a hardcoded sha256
# before it is installed, and any mismatch aborts. Idempotent: an asset
# already present with its pinned hash is left alone.
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
# The sha256 of each installed asset lives in static/vendor.sha256, in
# sha256sum(1) format, with paths relative to static/. That file is the
# single source of truth: this script verifies against it, and
# static/vendor_test.go asserts the bytes embedded into the binary match
# it, so the hash cannot rot into a value nothing checks.
MANIFEST="static/vendor.sha256"
# Alpine.js 3.14.9, 2026-08-17. Fetched from registry.npmjs.org, the
# publisher of record; the jsDelivr and unpkg copies are mirrors of this
# same tarball. dist/cdn.min.js is the browser build Alpine publishes for
# a <script> tag.
ALPINE_VERSION="3.14.9"
ALPINE_URL="https://registry.npmjs.org/alpinejs/-/alpinejs-${ALPINE_VERSION}.tgz"
# sha256 of alpinejs-3.14.9.tgz
ALPINE_TARBALL_SHA256="97dad7c0c81e659cfc8e7700055da9770f8186187cb9a8a76efb57e00d5ce52a"
ALPINE_MEMBER="package/dist/cdn.min.js"
ALPINE_DEST="js/alpine.min.js"
sha256_of() {
if command -v sha256sum >/dev/null 2>&1; then
sha256sum "$1" | cut -d' ' -f1
else
shasum -a 256 "$1" | cut -d' ' -f1
fi
}
# expected_sha256 <path-relative-to-static>
expected_sha256() {
awk -v want="$1" '$2 == want { print $1; found = 1 }
END { if (!found) exit 1 }' "$ROOT/$MANIFEST"
}
# verify <file> <expected-sha256> <what>
verify() {
actual="$(sha256_of "$1")"
if [ "$actual" != "$2" ]; then
echo "fetch-assets: sha256 mismatch for $3" >&2
echo " expected: $2" >&2
echo " actual: $actual" >&2
exit 1
fi
}
# up_to_date <path-relative-to-static> <expected-sha256>
up_to_date() {
[ -f "$ROOT/static/$1" ] || return 1
[ "$(sha256_of "$ROOT/static/$1")" = "$2" ]
}
fetch_alpine() {
want="$(expected_sha256 "$ALPINE_DEST")"
if up_to_date "$ALPINE_DEST" "$want"; then
echo "fetch-assets: static/$ALPINE_DEST already at $want"
return 0
fi
echo "fetch-assets: fetching Alpine.js $ALPINE_VERSION from $ALPINE_URL"
tmp="$(mktemp -d)"
trap 'rm -rf "$tmp"' EXIT INT TERM
curl -fsSL -o "$tmp/alpine.tgz" "$ALPINE_URL"
verify "$tmp/alpine.tgz" "$ALPINE_TARBALL_SHA256" "alpinejs-${ALPINE_VERSION}.tgz"
tar -xzOf "$tmp/alpine.tgz" "$ALPINE_MEMBER" >"$tmp/alpine.min.js"
verify "$tmp/alpine.min.js" "$want" "$ALPINE_MEMBER from alpinejs-${ALPINE_VERSION}.tgz"
mkdir -p "$(dirname "$ROOT/static/$ALPINE_DEST")"
cp "$tmp/alpine.min.js" "$ROOT/static/$ALPINE_DEST"
rm -rf "$tmp"
trap - EXIT INT TERM
echo "fetch-assets: installed static/$ALPINE_DEST ($want)"
}
# Re-check every manifest entry against what is now on disk, so an entry
# no script installs fails loudly instead of passing silently.
verify_manifest() {
while read -r want path; do
case "$want" in '' | '#'*) continue ;; esac
if [ ! -f "$ROOT/static/$path" ]; then
echo "fetch-assets: $MANIFEST lists static/$path, which is missing" >&2
exit 1
fi
verify "$ROOT/static/$path" "$want" "static/$path"
done <"$ROOT/$MANIFEST"
}
main() {
cd "$ROOT"
fetch_alpine
verify_manifest
echo "fetch-assets: all assets in $MANIFEST verified"
}
main "$@"

5
static/js/alpine.min.js vendored Normal file
View File

File diff suppressed because one or more lines are too long

1
static/vendor.sha256
View File

@@ -1 +0,0 @@
3ed1eed252488921df65e363d6715deb04d7f92aaedb9e52199fdf73cb1e0ad3 js/alpine.min.js

92
static/vendor_test.go
View File

@@ -1,92 +0,0 @@
package static_test
import (
"bufio"
"crypto/sha256"
"encoding/hex"
"os"
"strings"
"testing"
"github.com/stretchr/testify/require"
"sneak.berlin/go/webhooker/static"
)
const manifestPath = "vendor.sha256"
// fetchHint is appended to every failure here: the assets the manifest
// covers are fetched by the build, not committed, so a fresh clone that
// has not run script/fetch-assets fails this test and should be told why.
const fetchHint = "run `script/fetch-assets` (or `make assets`) to install " +
"the pinned third-party assets"
// TestVendoredAssetsMatchManifest asserts that every asset listed in
// static/vendor.sha256 is embedded in the binary with exactly the pinned
// bytes. script/fetch-assets verifies the same hashes at download time;
// this test verifies them again on what actually ships, so a build that
// skipped, cached, or subverted the fetch cannot produce a binary serving
// unpinned third-party JavaScript.
func TestVendoredAssetsMatchManifest(t *testing.T) {
t.Parallel()
entries := readManifest(t)
require.NotEmpty(t, entries, "%s lists no assets", manifestPath)
for path, want := range entries {
t.Run(path, func(t *testing.T) {
t.Parallel()
data, err := static.Static.ReadFile(path)
require.NoErrorf(
t, err,
"%s is listed in %s but is not embedded; %s",
path, manifestPath, fetchHint,
)
sum := sha256.Sum256(data)
got := hex.EncodeToString(sum[:])
require.Equalf(
t, want, got,
"embedded %s does not match its pinned sha256 in %s; %s",
path, manifestPath, fetchHint,
)
})
}
}
// readManifest parses static/vendor.sha256, which is in sha256sum(1)
// format with paths relative to static/.
func readManifest(t *testing.T) map[string]string {
t.Helper()
f, err := os.Open(manifestPath)
require.NoError(t, err, "opening %s", manifestPath)
defer func() { require.NoError(t, f.Close()) }()
entries := make(map[string]string)
scanner := bufio.NewScanner(f)
for scanner.Scan() {
line := strings.TrimSpace(scanner.Text())
if line == "" || strings.HasPrefix(line, "#") {
continue
}
fields := strings.Fields(line)
require.Lenf(
t, fields, 2,
"%s: malformed entry %q, want \"<sha256> <path>\"",
manifestPath, line,
)
sum, path := fields[0], fields[1]
require.Lenf(t, sum, 64, "%s: %q is not a sha256", manifestPath, sum)
entries[path] = sum
}
require.NoError(t, scanner.Err(), "reading %s", manifestPath)
return entries
}

3
templates/source_logs.html
View File

@@ -37,9 +37,6 @@
<div x-show="open" x-cloak class="mt-3 p-3 bg-gray-50 rounded-md"> <div x-show="open" x-cloak class="mt-3 p-3 bg-gray-50 rounded-md">
<pre class="text-xs text-gray-700 overflow-x-auto whitespace-pre-wrap break-all">{{.Body}}</pre> <pre class="text-xs text-gray-700 overflow-x-auto whitespace-pre-wrap break-all">{{.Body}}</pre>
{{if .BodyTruncated}}
<p class="mt-2 text-xs text-gray-500">Body truncated for display: showing {{.BodyShownBytes}} of {{.BodyBytes}} bytes. The stored body is unchanged.</p>
{{end}}
</div> </div>
</div> </div>
{{else}} {{else}}