Compare commits
1
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
b93ecf4cb1 |
+4
-10
@@ -1,20 +1,14 @@
|
||||
# .git is sent so the build can derive the version it stamps into the binary
|
||||
# (script/version). Its config, which can hold a remote URL carrying a
|
||||
# credential and which `git describe` does not need, is left out of a
|
||||
# directory context. A context sent as a tar is not filtered by this file, so
|
||||
# it carries .git/config unless its sender leaves it out.
|
||||
.git/config
|
||||
|
||||
# No tracked file may be listed here: git in the build would see it as
|
||||
# deleted and mark the version -dirty.
|
||||
#
|
||||
# .ci-fingerprint is deliberately NOT excluded: it is the CI cache barrier
|
||||
# that keeps the check stages from replaying a cached pass. See the lint
|
||||
# stage of the Dockerfile.
|
||||
.git/
|
||||
bin/
|
||||
# Extracted from 3p/ by `make assets` inside the build; a host copy is not
|
||||
# needed. The tarball in 3p/ must stay in the context.
|
||||
static/js/alpine.min.js
|
||||
*.md
|
||||
LICENSE
|
||||
.editorconfig
|
||||
.env
|
||||
.env.*
|
||||
*.db
|
||||
|
||||
@@ -12,8 +12,9 @@ jobs:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 2024-10-23
|
||||
with:
|
||||
# The superseded-status step needs history to walk ancestors (it
|
||||
# aborts on a shallow clone).
|
||||
# The fingerprint step below needs history to find the last commit
|
||||
# that touched the Docker build context, and the superseded-status
|
||||
# step needs it to walk ancestors (it aborts on a shallow clone).
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Mark superseded run statuses
|
||||
@@ -27,11 +28,16 @@ jobs:
|
||||
run: script/ci-mark-superseded
|
||||
|
||||
- name: Fingerprint the build context
|
||||
# Writes the hash of the commit being checked into the context, which
|
||||
# invalidates the `COPY . .` layer of both check stages: a commit
|
||||
# that was never linted, format-checked, tested and built cannot
|
||||
# report success from cache.
|
||||
run: git rev-parse HEAD > .ci-fingerprint
|
||||
# `.dockerignore` keeps docs out of the build context, so a docs-only
|
||||
# commit legitimately replays the whole image from cache and stays
|
||||
# cheap. Every other commit writes a new fingerprint into the context,
|
||||
# which invalidates the `COPY . .` layer of both check stages: a
|
||||
# commit that was never linted, formatted-checked, tested and built
|
||||
# cannot report success from cache.
|
||||
run: |
|
||||
set -eu
|
||||
fp="$(git log -1 --format=%H -- . ':!*.md' ':!LICENSE' ':!.editorconfig')"
|
||||
printf '%s\n' "${fp:-$GITHUB_SHA}" > .ci-fingerprint
|
||||
|
||||
- name: Build Docker image (runs make check)
|
||||
run: script/cibuild
|
||||
|
||||
+9
-22
@@ -12,8 +12,8 @@ WORKDIR /src
|
||||
COPY go.mod go.sum ./
|
||||
RUN go mod download
|
||||
|
||||
# Copy source code. In CI the context also carries .ci-fingerprint, which
|
||||
# holds the hash of the commit being checked (see
|
||||
# Copy source code. In CI the context also carries .ci-fingerprint, whose
|
||||
# value changes with every commit that touches the build context (see
|
||||
# .gitea/workflows/check.yml). That invalidates this layer, so the checks
|
||||
# below cannot report success by replaying a cached pass. Do not add it to
|
||||
# .dockerignore.
|
||||
@@ -38,13 +38,8 @@ FROM golang:1.26.1-bookworm@sha256:4465644228bc2857a954b092167e12aa59c006a349228
|
||||
COPY --from=lint /src/go.sum /dev/null
|
||||
|
||||
# jq is a runtime dependency of script/ci-mark-superseded, which the test
|
||||
# suite executes. git is what script/version derives the version with.
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends make curl ca-certificates jq git && rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# A build context sent as a tar archive keeps its files' owners, and git
|
||||
# refuses to read a checkout owned by another user. Trust this one
|
||||
# whoever owns it.
|
||||
RUN git config --system --add safe.directory /build
|
||||
# suite executes.
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends make curl ca-certificates jq && rm -rf /var/lib/apt/lists/*
|
||||
|
||||
WORKDIR /build
|
||||
|
||||
@@ -60,22 +55,14 @@ COPY . .
|
||||
# from its tarball in 3p/.
|
||||
RUN make test
|
||||
|
||||
# Version stamped into the binary: the VERSION build arg when one is
|
||||
# given, otherwise what script/version derives from the .git the build
|
||||
# context carries, so any `docker build .` of a clone stamps its commit.
|
||||
# With neither, as from a source tarball, it is "unknown".
|
||||
# Version stamped into the binary. .dockerignore excludes .git/, so
|
||||
# nothing in this stage can derive it: script/docker resolves it on the
|
||||
# host and passes it in. The default is what a bare `docker build .`
|
||||
# with no --build-arg gets, and it names no tag the tree may not be at.
|
||||
#
|
||||
# Declared here, below the test step, so a changed version does not
|
||||
# invalidate its cached layer.
|
||||
ARG VERSION
|
||||
|
||||
# A context that carries .git must not stamp "unknown": that means git is
|
||||
# missing here or could not read the checkout, and the image could not be
|
||||
# traced back to its commit.
|
||||
RUN if [ -d .git ] && [ "$(make version VERSION="$VERSION")" = unknown ]; then \
|
||||
echo "version is unknown although the build context carries .git" >&2; \
|
||||
exit 1; \
|
||||
fi
|
||||
ARG VERSION=unknown
|
||||
|
||||
RUN make build VERSION="$VERSION"
|
||||
|
||||
|
||||
@@ -4,12 +4,12 @@
|
||||
.DEFAULT_GOAL := check
|
||||
|
||||
# Version stamped into the binary. Derived from git by script/version;
|
||||
# override it (`make build VERSION=v1.2.3`) to stamp a given value, which is
|
||||
# how the Dockerfile passes its build arg in.
|
||||
# override it (`make build VERSION=v1.2.3`) where git metadata is
|
||||
# unavailable, which is how the Dockerfile passes its build arg in.
|
||||
VERSION ?= $(shell script/version)
|
||||
|
||||
# An empty override (`make build VERSION=`, or the Dockerfile's `make build
|
||||
# VERSION="$VERSION"` when no VERSION build arg was given) means unset,
|
||||
# An empty override (`make build VERSION=`, or a `--build-arg VERSION=`
|
||||
# landing on the Dockerfile's `make build VERSION="$VERSION"`) means unset,
|
||||
# exactly as it does in script/version -- stamping "" would leave the binary
|
||||
# reporting no version and the footer back on its "dev" fallback. `override`
|
||||
# is required: a plain assignment loses to the command-line definition it
|
||||
|
||||
@@ -1133,29 +1133,13 @@ build itself.
|
||||
| Uncommitted changes | the above with a `-dirty` suffix |
|
||||
| No git metadata | `unknown` |
|
||||
|
||||
The image derives it the same way, from the `.git` that the build
|
||||
context carries, so any `docker build .` of a clone, with no build
|
||||
arguments, stamps the commit it was built from; a shallow clone of one
|
||||
branch has no tags and stamps the short SHA. `.dockerignore` must
|
||||
therefore leave out neither `.git` nor any tracked file, which git in
|
||||
the build would see as deleted, marking the version `-dirty`. It does
|
||||
leave `.git/config`, which can hold a remote URL carrying a credential
|
||||
and which `git describe` does not need, out of a directory context. A
|
||||
context sent as a tar is not filtered by `.dockerignore`, so it carries
|
||||
`.git/config` unless its sender leaves it out; for upaas, that is
|
||||
https://git.eeqj.de/sneak/upaas/issues/274. git in the build
|
||||
reads the checkout whoever owns its files, since a context sent as a tar
|
||||
archive keeps the sender's owners and git otherwise refuses a checkout
|
||||
owned by another user. A `VERSION` build arg (`--build-arg VERSION=...`)
|
||||
takes precedence; `script/docker` (and so `make docker`) passes the one
|
||||
`script/version` resolves on the host. The image build fails if its
|
||||
context carries `.git` and the version still comes out `unknown`, which
|
||||
means git is missing from the build or could not read the checkout.
|
||||
|
||||
`unknown` is what a source tarball, or a `docker build` with no `.git`
|
||||
in its context and no `VERSION` build arg, reports. A build that reports
|
||||
`unknown` is a build nobody told what it was; it is not a failure, but
|
||||
it cannot be traced back to a commit.
|
||||
`unknown` is what a source tarball or a `docker build .` with no
|
||||
`--build-arg VERSION=...` reports. `.dockerignore` excludes `.git/`, so
|
||||
the build context carries no git metadata and the image cannot derive
|
||||
the version itself: `script/docker` (and so `make docker`) resolves it
|
||||
on the host and passes it in as the `VERSION` build arg. A build that
|
||||
reports `unknown` is a build nobody told what it was; it is not a
|
||||
failure, but it cannot be traced back to a commit.
|
||||
|
||||
`make version` prints what the current checkout would stamp, and
|
||||
`make build VERSION=v1.2.3` overrides it. An empty override — from
|
||||
@@ -1768,7 +1752,7 @@ retries) is individually logged for full observability.
|
||||
#### EventTotals and TargetTotals
|
||||
|
||||
Running counts in each event database, read by the statistics pane at the
|
||||
top of the webhook page. `EventTotals` is one row:
|
||||
top of the webhook page and by the webhook list. `EventTotals` is one row:
|
||||
|
||||
| Field | Type | Description |
|
||||
| ---------------- | --------- | ----------- |
|
||||
@@ -1800,6 +1784,14 @@ target. Its failure percentage for a window is the deliveries that became
|
||||
`failed` in it out of all that became `delivered` or `failed` in it, and
|
||||
a dash when none did.
|
||||
|
||||
The webhook list at `/hooks` shows three of the pane's figures for each
|
||||
webhook: its events within retention and its last event, both from
|
||||
`EventTotals`, and its deliveries that failed in the last 24 hours,
|
||||
counted with the pane's query. It opens each webhook's event database once
|
||||
(the handle stays open) and runs those two reads there, so its cost grows
|
||||
with the number of webhooks and, for each, with the deliveries that
|
||||
finished in the last 24 hours, never with the events stored.
|
||||
|
||||
#### Event-tier indexes
|
||||
|
||||
These indexes on the per-webhook event databases are declared in the model
|
||||
@@ -1807,7 +1799,7 @@ tags, so `AutoMigrate` creates them on a fresh database:
|
||||
|
||||
| Table | Columns | Serves |
|
||||
| ------------------ | --------------------------- | ------ |
|
||||
| `deliveries` | `status`, `deleted_at`, `finished_at`, `target_id` | Startup recovery, the retry and pending sweeps every 60 seconds and the queue-depth sampler every 30 seconds, which select deliveries by status, and the webhook page's statistics, which count each target's deliveries by status and when they finished |
|
||||
| `deliveries` | `status`, `deleted_at`, `finished_at`, `target_id` | Startup recovery, the retry and pending sweeps every 60 seconds and the queue-depth sampler every 30 seconds, which select deliveries by status, and the webhook page's statistics and the webhook list, which count each target's deliveries by status and when they finished |
|
||||
| `deliveries` | `event_id`, `deleted_at` | The event log, which loads each event's deliveries, and retention, which counts and deletes the deliveries of expired events |
|
||||
| `delivery_results` | `delivery_id`, `deleted_at` | The event log, which loads the attempts of a page's deliveries, and retention, which deletes the attempts of expired events |
|
||||
| `events` | `deleted_at`, `created_at` | The webhook page's statistics, which count recent events |
|
||||
@@ -3243,9 +3235,8 @@ version is fixed independently of the compiler's:
|
||||
rebuilds the binary with `CGO_ENABLED=1` and static linking so it
|
||||
runs on musl. Both builds go through `make build`, the relink adding
|
||||
its `-extldflags` via `GO_LDFLAGS`, so neither can drop the `-X` that
|
||||
stamps the version. The version is the `VERSION` build arg if one is
|
||||
given, otherwise derived from the `.git` in the context, and the
|
||||
stage fails if a context with `.git` would stamp `unknown` (see
|
||||
stamps the version. The version arrives as the `VERSION` build arg,
|
||||
since the context has no `.git` (see
|
||||
[Version stamping](#version-stamping)).
|
||||
3. **Runtime stage** (`alpine:3.21`) — copies the static binary and
|
||||
`deploy/docker-entrypoint.sh`, creates the `/var/lib/webhooker`
|
||||
@@ -3277,13 +3268,19 @@ A layer cache lets `docker build .` exit 0 in seconds with the lint and
|
||||
test stages replayed rather than executed, which would make a green
|
||||
check meaningless. The `check` workflow therefore writes
|
||||
`.ci-fingerprint` into the build context before building. Its value is
|
||||
the hash of the commit being checked, so every commit, docs-only ones
|
||||
and a squash merge whose tree matches an already-built branch included,
|
||||
gets a new fingerprint, invalidates the `COPY . .` layer of both check
|
||||
stages, and really runs `make fmt-check`, `golangci-lint`, `make test`,
|
||||
and `make build`. A run that reports success ran them.
|
||||
the hash of the last commit that touched the build context, so:
|
||||
|
||||
The module download layer sits above `COPY . .` and stays cached.
|
||||
- Any commit that changes code (including a squash merge whose tree
|
||||
matches an already-built branch) gets a new fingerprint, invalidates
|
||||
the `COPY . .` layer of both check stages, and really runs
|
||||
`make fmt-check`, `golangci-lint`, `make test`, and `make build`. A
|
||||
run that reports success ran them.
|
||||
- A docs-only commit leaves the fingerprint unchanged — `.dockerignore`
|
||||
excludes `*.md`, `LICENSE` and `.editorconfig` from the context
|
||||
anyway — so the image replays from cache and costs seconds.
|
||||
|
||||
The module download layer sits above `COPY . .` and stays cached either
|
||||
way.
|
||||
|
||||
A separate workflow step, run before the fingerprint is written, covers
|
||||
a second way the gate lied: Gitea cancels an in-flight run when a newer
|
||||
|
||||
@@ -0,0 +1,317 @@
|
||||
package handlers_test
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"regexp"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
"gorm.io/gorm"
|
||||
"gorm.io/gorm/clause"
|
||||
"sneak.berlin/go/webhooker/internal/database"
|
||||
"sneak.berlin/go/webhooker/internal/handlers"
|
||||
"sneak.berlin/go/webhooker/internal/logger"
|
||||
"sneak.berlin/go/webhooker/internal/session"
|
||||
)
|
||||
|
||||
// failedHighlight is how the list marks a number of failed deliveries
|
||||
// that is not zero.
|
||||
const failedHighlight = `class="font-medium text-red-600"`
|
||||
|
||||
// listWebhook adds a webhook with the given name, owned by the test
|
||||
// user.
|
||||
func listWebhook(
|
||||
t *testing.T, db *database.Database, name string,
|
||||
) *database.Webhook {
|
||||
t.Helper()
|
||||
|
||||
wh := &database.Webhook{UserID: deleteTestUserID, Name: name}
|
||||
require.NoError(t, db.DB().Omit(clause.Associations).Create(wh).Error)
|
||||
|
||||
return wh
|
||||
}
|
||||
|
||||
// renderWebhookList runs the real webhook list handler as the test user
|
||||
// and returns the rendered page.
|
||||
func renderWebhookList(
|
||||
t *testing.T, h *handlers.Handlers, sess *session.Session,
|
||||
) string {
|
||||
t.Helper()
|
||||
|
||||
cookies := authenticatedCookies(
|
||||
t, sess, deleteTestUserID, deleteTestUsername,
|
||||
)
|
||||
|
||||
w := httptest.NewRecorder()
|
||||
h.HandleSourceList().ServeHTTP(
|
||||
w, getRequest(t, "/hooks", cookies, nil),
|
||||
)
|
||||
require.Equal(t, http.StatusOK, w.Code)
|
||||
|
||||
return w.Body.String()
|
||||
}
|
||||
|
||||
// listCard returns one webhook's entry in a rendered webhook list, its
|
||||
// markup as rendered and its text with the markup taken out and each
|
||||
// run of space made one space.
|
||||
func listCard(t *testing.T, page, webhookID string) (string, string) {
|
||||
t.Helper()
|
||||
|
||||
_, card, found := strings.Cut(page, `href="/hook/`+webhookID+`"`)
|
||||
require.True(t, found, "the list has no entry for %s", webhookID)
|
||||
|
||||
card, _, _ = strings.Cut(card, "</a>")
|
||||
text := regexp.MustCompile(`<[^>]*>`).ReplaceAllString(card, " ")
|
||||
|
||||
return card, strings.Join(strings.Fields(text), " ")
|
||||
}
|
||||
|
||||
// receiveEvents posts the given number of events to an entrypoint
|
||||
// through the real receiver, and returns the webhook's event database
|
||||
// and its events, oldest first.
|
||||
func receiveEvents(
|
||||
t *testing.T,
|
||||
h *handlers.Handlers,
|
||||
dbMgr *database.WebhookDBManager,
|
||||
webhookID, path string,
|
||||
count int,
|
||||
) (*gorm.DB, []database.Event) {
|
||||
t.Helper()
|
||||
|
||||
router := receiverRouter(h)
|
||||
|
||||
for range count {
|
||||
require.Equal(t, http.StatusOK, postReceiver(t, router, path))
|
||||
}
|
||||
|
||||
webhookDB, err := dbMgr.GetDB(webhookID)
|
||||
require.NoError(t, err)
|
||||
|
||||
events := listEvents(t, webhookDB)
|
||||
require.Len(t, events, count)
|
||||
|
||||
return webhookDB, events
|
||||
}
|
||||
|
||||
// seedFailingWebhook adds a webhook with two entrypoints, one inactive,
|
||||
// and four targets, one inactive. Three events each reach the three
|
||||
// active targets. Two deliveries failed in the last 24 hours, one 30
|
||||
// hours ago, and one was delivered. It returns the webhook and its
|
||||
// newest event.
|
||||
func seedFailingWebhook(
|
||||
t *testing.T,
|
||||
h *handlers.Handlers,
|
||||
db *database.Database,
|
||||
dbMgr *database.WebhookDBManager,
|
||||
) (*database.Webhook, database.Event) {
|
||||
t.Helper()
|
||||
|
||||
wh := listWebhook(t, db, "failing")
|
||||
path := statsEntrypoint(t, db, wh.ID, true)
|
||||
|
||||
statsEntrypoint(t, db, wh.ID, false)
|
||||
|
||||
first := seedTarget(t, db, wh.ID, database.TargetTypeLog)
|
||||
second := seedTarget(t, db, wh.ID, database.TargetTypeLog)
|
||||
|
||||
seedTarget(t, db, wh.ID, database.TargetTypeLog)
|
||||
|
||||
inactive := seedTarget(t, db, wh.ID, database.TargetTypeLog)
|
||||
require.NoError(t, db.DB().Model(inactive).
|
||||
Update("active", false).Error)
|
||||
|
||||
webhookDB, events := receiveEvents(t, h, dbMgr, wh.ID, path, 3)
|
||||
now := time.Now()
|
||||
|
||||
statsFinish(t, webhookDB,
|
||||
statsDelivery(t, webhookDB, events[0].ID, first.ID),
|
||||
database.DeliveryStatusFailed, now.Add(-30*time.Hour))
|
||||
statsFinish(t, webhookDB,
|
||||
statsDelivery(t, webhookDB, events[1].ID, first.ID),
|
||||
database.DeliveryStatusFailed, now.Add(-time.Hour))
|
||||
statsFinish(t, webhookDB,
|
||||
statsDelivery(t, webhookDB, events[2].ID, first.ID),
|
||||
database.DeliveryStatusFailed, now.Add(-time.Minute))
|
||||
statsFinish(t, webhookDB,
|
||||
statsDelivery(t, webhookDB, events[2].ID, second.ID),
|
||||
database.DeliveryStatusDelivered, now.Add(-time.Minute))
|
||||
|
||||
return wh, events[2]
|
||||
}
|
||||
|
||||
// seedHealthyWebhook adds a webhook with one entrypoint and one target,
|
||||
// both active, and two events, both delivered. It returns the webhook
|
||||
// and its newest event.
|
||||
func seedHealthyWebhook(
|
||||
t *testing.T,
|
||||
h *handlers.Handlers,
|
||||
db *database.Database,
|
||||
dbMgr *database.WebhookDBManager,
|
||||
) (*database.Webhook, database.Event) {
|
||||
t.Helper()
|
||||
|
||||
wh := listWebhook(t, db, "healthy")
|
||||
path := statsEntrypoint(t, db, wh.ID, true)
|
||||
target := seedTarget(t, db, wh.ID, database.TargetTypeLog)
|
||||
|
||||
webhookDB, events := receiveEvents(t, h, dbMgr, wh.ID, path, 2)
|
||||
|
||||
for _, ev := range events {
|
||||
statsFinish(t, webhookDB,
|
||||
statsDelivery(t, webhookDB, ev.ID, target.ID),
|
||||
database.DeliveryStatusDelivered, time.Now())
|
||||
}
|
||||
|
||||
return wh, events[1]
|
||||
}
|
||||
|
||||
// lastEventText is how the list shows the arrival of an event.
|
||||
func lastEventText(ev database.Event) string {
|
||||
return ev.CreatedAt.UTC().Format("2006-01-02 15:04:05 UTC")
|
||||
}
|
||||
|
||||
// TestSourceList_ShowsActivityOfEachWebhook checks the figures the list
|
||||
// shows for a webhook with recent failures, a healthy one, a new one
|
||||
// that has received no event, and one without an event database.
|
||||
func TestSourceList_ShowsActivityOfEachWebhook(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
var (
|
||||
h *handlers.Handlers
|
||||
sess *session.Session
|
||||
db *database.Database
|
||||
dbMgr *database.WebhookDBManager
|
||||
)
|
||||
|
||||
app := newTestApp(t, &h, &sess, &db, &dbMgr)
|
||||
app.RequireStart()
|
||||
|
||||
t.Cleanup(app.RequireStop)
|
||||
|
||||
failing, failingNewest := seedFailingWebhook(t, h, db, dbMgr)
|
||||
healthy, healthyNewest := seedHealthyWebhook(t, h, db, dbMgr)
|
||||
|
||||
// Creating a webhook creates its event database.
|
||||
fresh := listWebhook(t, db, "fresh")
|
||||
require.NoError(t, dbMgr.CreateDB(fresh.ID))
|
||||
|
||||
quiet := listWebhook(t, db, "quiet")
|
||||
|
||||
page := renderWebhookList(t, h, sess)
|
||||
|
||||
card, text := listCard(t, page, failing.ID)
|
||||
assert.Contains(t, text, "2 entrypoints, 1 inactive "+
|
||||
"4 targets, 1 inactive "+
|
||||
"3 events within retention "+
|
||||
"Last event "+lastEventText(failingNewest)+" "+
|
||||
"2 failed deliveries in the last 24 hours")
|
||||
assert.Contains(t, card,
|
||||
failedHighlight+">2 failed deliveries in the last 24 hours<")
|
||||
|
||||
card, text = listCard(t, page, healthy.ID)
|
||||
assert.Contains(t, text, "1 entrypoint "+
|
||||
"1 target "+
|
||||
"2 events within retention "+
|
||||
"Last event "+lastEventText(healthyNewest)+" "+
|
||||
"0 failed deliveries in the last 24 hours")
|
||||
assert.NotContains(t, text, "inactive")
|
||||
assert.NotContains(t, card, failedHighlight)
|
||||
|
||||
card, text = listCard(t, page, fresh.ID)
|
||||
assert.Contains(t, text, "0 entrypoints "+
|
||||
"0 targets "+
|
||||
"0 events within retention "+
|
||||
"No events yet "+
|
||||
"0 failed deliveries in the last 24 hours")
|
||||
assert.NotContains(t, card, failedHighlight)
|
||||
|
||||
card, text = listCard(t, page, quiet.ID)
|
||||
assert.Contains(t, text, "0 entrypoints "+
|
||||
"0 targets "+
|
||||
"0 events within retention "+
|
||||
"No events yet "+
|
||||
"0 failed deliveries in the last 24 hours")
|
||||
assert.NotContains(t, card, failedHighlight)
|
||||
assert.False(t, dbMgr.DBExists(quiet.ID),
|
||||
"showing the list must not create an event database")
|
||||
}
|
||||
|
||||
// TestSourceList_CountsOnlyEventsWithinRetention checks that once
|
||||
// retention has removed one of a webhook's three events, the list
|
||||
// counts the two still stored.
|
||||
func TestSourceList_CountsOnlyEventsWithinRetention(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
var (
|
||||
h *handlers.Handlers
|
||||
sess *session.Session
|
||||
db *database.Database
|
||||
dbMgr *database.WebhookDBManager
|
||||
log *logger.Logger
|
||||
)
|
||||
|
||||
app := newTestApp(t, &h, &sess, &db, &dbMgr, &log)
|
||||
app.RequireStart()
|
||||
|
||||
t.Cleanup(app.RequireStop)
|
||||
|
||||
wh := &database.Webhook{
|
||||
UserID: deleteTestUserID, Name: "pruned", RetentionDays: 14,
|
||||
}
|
||||
require.NoError(t, db.DB().Omit(clause.Associations).Create(wh).Error)
|
||||
|
||||
path := statsEntrypoint(t, db, wh.ID, true)
|
||||
webhookDB, events := receiveEvents(t, h, dbMgr, wh.ID, path, 3)
|
||||
|
||||
statsAge(t, webhookDB, events[0].ID, time.Now().Add(-15*24*time.Hour))
|
||||
statsPrune(t, db, dbMgr, log, webhookDB)
|
||||
require.Len(t, listEvents(t, webhookDB), 2)
|
||||
|
||||
_, text := listCard(t, renderWebhookList(t, h, sess), wh.ID)
|
||||
assert.Contains(t, text,
|
||||
"1 entrypoint 0 targets 2 events within retention")
|
||||
}
|
||||
|
||||
// TestSourceList_UnreadableEventDatabase checks that a webhook whose
|
||||
// event database cannot be read says so in its entry instead of
|
||||
// showing zeros, and that the rest of the list is still shown.
|
||||
func TestSourceList_UnreadableEventDatabase(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
var (
|
||||
h *handlers.Handlers
|
||||
sess *session.Session
|
||||
db *database.Database
|
||||
dbMgr *database.WebhookDBManager
|
||||
)
|
||||
|
||||
app := newTestApp(t, &h, &sess, &db, &dbMgr)
|
||||
app.RequireStart()
|
||||
|
||||
t.Cleanup(app.RequireStop)
|
||||
|
||||
broken := listWebhook(t, db, "broken")
|
||||
statsEntrypoint(t, db, broken.ID, true)
|
||||
|
||||
brokenDB, err := dbMgr.GetDB(broken.ID)
|
||||
require.NoError(t, err)
|
||||
require.NoError(t,
|
||||
brokenDB.Migrator().DropTable(&database.EventTotals{}))
|
||||
|
||||
quiet := listWebhook(t, db, "quiet")
|
||||
|
||||
page := renderWebhookList(t, h, sess)
|
||||
|
||||
_, text := listCard(t, page, broken.ID)
|
||||
assert.Contains(t, text,
|
||||
"1 entrypoint 0 targets The event figures could not be read.")
|
||||
assert.NotContains(t, text, "events")
|
||||
assert.NotContains(t, text, "failed")
|
||||
|
||||
_, text = listCard(t, page, quiet.ID)
|
||||
assert.Contains(t, text, "No events yet")
|
||||
}
|
||||
@@ -3,10 +3,12 @@ package handlers
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"slices"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/go-chi/chi"
|
||||
"github.com/google/uuid"
|
||||
@@ -20,9 +22,20 @@ import (
|
||||
type WebhookListItem struct {
|
||||
database.Webhook
|
||||
|
||||
EntrypointCount int64
|
||||
TargetCount int64
|
||||
EntrypointCount int
|
||||
InactiveEntrypointCount int
|
||||
TargetCount int
|
||||
InactiveTargetCount int
|
||||
|
||||
// EventCount is how many events the webhook holds, LastEventAt
|
||||
// when the newest arrived (nil before the first), and
|
||||
// FailedLast24Hours how many of its deliveries failed in the last
|
||||
// 24 hours. When the webhook's event database could not be read,
|
||||
// EventsUnreadable is set and these three are not known.
|
||||
EventCount int64
|
||||
LastEventAt *time.Time
|
||||
FailedLast24Hours int64
|
||||
EventsUnreadable bool
|
||||
}
|
||||
|
||||
// errMissingURL signals that a required URL was not provided.
|
||||
@@ -160,7 +173,18 @@ func (h *Handlers) HandleSourceList() http.HandlerFunc {
|
||||
return
|
||||
}
|
||||
|
||||
items := h.buildWebhookListItems(webhooks)
|
||||
items, err := h.buildWebhookListItems(webhooks)
|
||||
if err != nil {
|
||||
h.log.Error(
|
||||
"failed to list webhooks", "error", err,
|
||||
)
|
||||
http.Error(
|
||||
w, "Internal server error",
|
||||
http.StatusInternalServerError,
|
||||
)
|
||||
|
||||
return
|
||||
}
|
||||
|
||||
data := map[string]any{
|
||||
"Webhooks": items,
|
||||
@@ -170,36 +194,115 @@ func (h *Handlers) HandleSourceList() http.HandlerFunc {
|
||||
}
|
||||
}
|
||||
|
||||
// buildWebhookListItems builds list items with counts.
|
||||
// buildWebhookListItems builds the list's entry for each webhook. It
|
||||
// fails when the main database cannot be read. A webhook whose event
|
||||
// database cannot be read is marked on its own entry, and the error is
|
||||
// logged.
|
||||
func (h *Handlers) buildWebhookListItems(
|
||||
webhooks []database.Webhook,
|
||||
) []WebhookListItem {
|
||||
) ([]WebhookListItem, error) {
|
||||
items := make([]WebhookListItem, len(webhooks))
|
||||
since := time.Now().Add(-longWindow)
|
||||
|
||||
for i := range webhooks {
|
||||
items[i].Webhook = webhooks[i]
|
||||
item := &items[i]
|
||||
item.Webhook = webhooks[i]
|
||||
|
||||
h.db.DB().Model(&database.Entrypoint{}).Where(
|
||||
"webhook_id = ?", webhooks[i].ID,
|
||||
).Count(&items[i].EntrypointCount)
|
||||
var err error
|
||||
|
||||
h.db.DB().Model(&database.Target{}).Where(
|
||||
"webhook_id = ?", webhooks[i].ID,
|
||||
).Count(&items[i].TargetCount)
|
||||
item.EntrypointCount, item.InactiveEntrypointCount, err =
|
||||
h.countWithInactive(&database.Entrypoint{}, item.ID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if h.dbMgr.DBExists(webhooks[i].ID) {
|
||||
webhookDB, err := h.dbMgr.GetDB(
|
||||
webhooks[i].ID,
|
||||
item.TargetCount, item.InactiveTargetCount, err =
|
||||
h.countWithInactive(&database.Target{}, item.ID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// Opening an event database that does not exist would create
|
||||
// it, and it would hold nothing to count.
|
||||
if !h.dbMgr.DBExists(item.ID) {
|
||||
continue
|
||||
}
|
||||
|
||||
err = h.readListEventFigures(item, since)
|
||||
if err != nil {
|
||||
h.log.Error(
|
||||
"failed to read webhook list figures",
|
||||
"webhook_id", item.ID,
|
||||
"error", err,
|
||||
)
|
||||
if err == nil {
|
||||
webhookDB.Model(
|
||||
&database.Event{},
|
||||
).Count(&items[i].EventCount)
|
||||
}
|
||||
|
||||
item.EventsUnreadable = true
|
||||
}
|
||||
}
|
||||
|
||||
return items
|
||||
return items, nil
|
||||
}
|
||||
|
||||
// countWithInactive returns how many entrypoints or targets, as model
|
||||
// says, a webhook has, and how many of them are inactive.
|
||||
func (h *Handlers) countWithInactive(
|
||||
model any, webhookID string,
|
||||
) (int, int, error) {
|
||||
var active []bool
|
||||
|
||||
err := h.db.DB().Model(model).
|
||||
Where("webhook_id = ?", webhookID).
|
||||
Pluck("active", &active).Error
|
||||
if err != nil {
|
||||
return 0, 0, fmt.Errorf(
|
||||
"reading active flags of webhook %s: %w", webhookID, err,
|
||||
)
|
||||
}
|
||||
|
||||
inactive := 0
|
||||
|
||||
for _, a := range active {
|
||||
if !a {
|
||||
inactive++
|
||||
}
|
||||
}
|
||||
|
||||
return len(active), inactive, nil
|
||||
}
|
||||
|
||||
// readListEventFigures fills in the figures the list shows from the
|
||||
// webhook's event database, with the statistics pane's own queries:
|
||||
// the event count and last arrival from the event totals row, and the
|
||||
// deliveries that failed since the given time from the deliveries'
|
||||
// status index.
|
||||
func (h *Handlers) readListEventFigures(
|
||||
item *WebhookListItem, since time.Time,
|
||||
) error {
|
||||
webhookDB, err := h.dbMgr.GetDB(item.ID)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
var totals database.EventTotals
|
||||
|
||||
err = webhookDB.Take(&totals).Error
|
||||
if err != nil {
|
||||
return fmt.Errorf("reading event totals: %w", err)
|
||||
}
|
||||
|
||||
item.EventCount = totals.Events - totals.EventsRemoved
|
||||
item.LastEventAt = totals.LastEventAt
|
||||
|
||||
byTarget, err := finishedByTarget(webhookDB, since)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
for _, f := range byTarget {
|
||||
item.FailedLast24Hours += f.Failed
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// HandleSourceCreate shows the form to create a new webhook.
|
||||
|
||||
@@ -115,8 +115,8 @@ func TestVersion_EnclosingRepositoryIsNotUsed(t *testing.T) {
|
||||
require.Equal(t, unknown, runScript(t, inner, nil))
|
||||
}
|
||||
|
||||
// An explicit VERSION, such as the Dockerfile's build arg, wins over
|
||||
// anything derivable.
|
||||
// The Docker build has no git metadata, so the version arrives as an
|
||||
// environment override. It wins over anything derivable.
|
||||
func TestVersion_EnvironmentOverrideWins(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
@@ -128,8 +128,8 @@ func TestVersion_EnvironmentOverrideWins(t *testing.T) {
|
||||
}
|
||||
|
||||
// An empty VERSION is treated as unset rather than stamping an empty
|
||||
// string: a caller exporting VERSION= must not produce a binary
|
||||
// reporting "".
|
||||
// string: the Dockerfile's build arg has a non-empty default, but a
|
||||
// caller exporting VERSION= must not produce a binary reporting "".
|
||||
func TestVersion_EmptyOverrideFallsBackToGit(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
@@ -168,8 +168,8 @@ func TestMakefile_BuildComposesVersionAndExtraFlags(t *testing.T) {
|
||||
}
|
||||
|
||||
// A caller can define VERSION as the empty string -- `make build
|
||||
// VERSION=`, or the Dockerfile's `make build VERSION="$VERSION"` when no
|
||||
// VERSION build arg was given. script/version's own guard does not cover
|
||||
// VERSION=`, or a `--build-arg VERSION=` reaching the Dockerfile's `make
|
||||
// build VERSION="$VERSION"`. script/version's own guard does not cover
|
||||
// that: the value never passes through the script. Stamping "" would
|
||||
// leave the binary reporting no version and the footer on "dev", which
|
||||
// is the defect this package exists for.
|
||||
@@ -231,7 +231,7 @@ func TestDockerfile_BuildsThroughTheMakeTarget(t *testing.T) {
|
||||
|
||||
require.NotContains(t, dockerfile, "go build",
|
||||
"a raw go build bypasses the Makefile's -X flag")
|
||||
require.Contains(t, dockerfile, "ARG VERSION")
|
||||
require.Contains(t, dockerfile, "ARG VERSION=")
|
||||
require.Contains(t, dockerfile,
|
||||
`make build VERSION="$VERSION" GO_LDFLAGS='-extldflags "-static"'`)
|
||||
}
|
||||
|
||||
+3
-3
@@ -2,9 +2,9 @@
|
||||
# script/docker: build the Docker image tagged with the project name.
|
||||
# The tag comes from script/projectname.
|
||||
#
|
||||
# The version script/version resolves here goes in as the VERSION build
|
||||
# arg, which takes precedence over what the build would derive from the
|
||||
# .git in its context.
|
||||
# .dockerignore excludes .git/, so the builder stage cannot derive the
|
||||
# version itself. It is resolved here, where the checkout is, and passed
|
||||
# in as a build arg; without it the image would stamp itself "unknown".
|
||||
set -eu
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
|
||||
|
||||
+7
-5
@@ -7,16 +7,18 @@
|
||||
#
|
||||
# Order of precedence:
|
||||
#
|
||||
# 1. $VERSION, if set and non-empty: an explicit value, such as the
|
||||
# Dockerfile's VERSION build arg.
|
||||
# 1. $VERSION, if set and non-empty. This is how the value reaches a
|
||||
# build that cannot derive it: .dockerignore excludes .git/, so the
|
||||
# builder stage has no git metadata and the Dockerfile takes the
|
||||
# value as a build arg instead.
|
||||
# 2. `git describe --tags --always --dirty` against this checkout. At
|
||||
# a clean tagged commit that is exactly the tag; otherwise it
|
||||
# carries the short SHA, the commit distance when a tag is
|
||||
# reachable, and a -dirty suffix for uncommitted changes.
|
||||
# 3. "unknown", for a tree with no git metadata and no $VERSION -- a
|
||||
# source tarball, or a `docker build` with no .git in its context
|
||||
# and no VERSION build arg. That case must not fail the build and
|
||||
# must not name a tag the tree may not be at, so it names nothing.
|
||||
# source tarball, or `docker build .` with no --build-arg. That
|
||||
# case must not fail the build and must not name a tag the tree may
|
||||
# not be at, so it names nothing.
|
||||
#
|
||||
# The git step insists the enclosing repository is this checkout, not
|
||||
# merely some repository above it: an unpacked tarball sitting inside an
|
||||
|
||||
@@ -27,10 +27,16 @@
|
||||
</div>
|
||||
<span class="badge-info">Retention: {{.RetentionLabel}}</span>
|
||||
</div>
|
||||
<div class="flex gap-6 mt-4 text-sm text-gray-500">
|
||||
<span>{{.EntrypointCount}} entrypoint{{if ne .EntrypointCount 1}}s{{end}}</span>
|
||||
<span>{{.TargetCount}} target{{if ne .TargetCount 1}}s{{end}}</span>
|
||||
<span>{{.EventCount}} event{{if ne .EventCount 1}}s{{end}}</span>
|
||||
<div class="flex flex-wrap gap-6 mt-4 text-sm text-gray-500">
|
||||
<span>{{.EntrypointCount}} entrypoint{{if ne .EntrypointCount 1}}s{{end}}{{if .InactiveEntrypointCount}}, {{.InactiveEntrypointCount}} inactive{{end}}</span>
|
||||
<span>{{.TargetCount}} target{{if ne .TargetCount 1}}s{{end}}{{if .InactiveTargetCount}}, {{.InactiveTargetCount}} inactive{{end}}</span>
|
||||
{{if .EventsUnreadable}}
|
||||
<span class="text-red-600">The event figures could not be read.</span>
|
||||
{{else}}
|
||||
<span>{{.EventCount}} event{{if ne .EventCount 1}}s{{end}} within retention</span>
|
||||
<span>{{with .LastEventAt}}Last event {{.UTC.Format "2006-01-02 15:04:05 UTC"}}{{else}}No events yet{{end}}</span>
|
||||
<span class="{{if .FailedLast24Hours}}font-medium text-red-600{{end}}">{{.FailedLast24Hours}} failed deliver{{if eq .FailedLast24Hours 1}}y{{else}}ies{{end}} in the last 24 hours</span>
|
||||
{{end}}
|
||||
</div>
|
||||
</a>
|
||||
{{end}}
|
||||
|
||||
Reference in New Issue
Block a user