Compare commits
3
Commits
1cdf2013c8
...
11295f911c
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
11295f911c | ||
|
|
0ea169ec46 | ||
|
|
9285591563 |
@@ -41,11 +41,13 @@ The scripts cover both the frontend and the Go backend, so `make check` fails if
|
||||
either one is broken. We provide:
|
||||
|
||||
- `script/bootstrap` — install all dependencies (the pinned node via nvm unless
|
||||
one new enough for the frontend's dependencies is installed, yarn via
|
||||
corepack, `yarn install --frozen-lockfile`, Go `GO_VERSION` unless the `go` on
|
||||
`PATH` is exactly that version, and the Go modules), linking what it installs
|
||||
itself into `~/.local/bin`, which has to be on `PATH`. It installs no Go
|
||||
linter and not Docker: `make test` and `make lint` run in Docker
|
||||
the `node` on `PATH` has the pinned major version and is new enough for the
|
||||
frontend's dependencies, yarn `YARN_VERSION` via corepack, or npm where there
|
||||
is no corepack, unless the `yarn` on `PATH` is exactly that version,
|
||||
`yarn install --frozen-lockfile`, Go `GO_VERSION` unless the `go` on `PATH` is
|
||||
exactly that version, and the Go modules), linking what it installs itself
|
||||
into `~/.local/bin`, which has to be on `PATH`. It installs no Go linter and
|
||||
not Docker: `make test` and `make lint` run in Docker
|
||||
- `script/setup` — make a fresh clone ready for development: bootstrap plus the
|
||||
git pre-commit hook
|
||||
- `script/dev` — run the Vite dev server, which proxies `/api` to a locally
|
||||
@@ -58,6 +60,12 @@ either one is broken. We provide:
|
||||
defaults (see [Settings](#settings))
|
||||
- `script/clean` — remove what `script/build` writes
|
||||
- `script/projectname` — print the project name (used for the Docker image tag)
|
||||
- `script/yarn` — run yarn with the given arguments under the node
|
||||
`script/bootstrap` would use: with `~/.local/bin` first on `PATH`, the first
|
||||
`node` on that `PATH` when it has the pinned major version and is new enough
|
||||
for the frontend's dependencies, and otherwise the pinned node in nvm. Every
|
||||
script that runs yarn on the host runs it through this one, except
|
||||
`script/bootstrap`, which picks its own node and yarn
|
||||
- `script/test` — build the `test` phase of `Dockerfile`, writing no image: the
|
||||
frontend's unit tests in its `frontend-test` stage, and the backend's Go tests
|
||||
with the race detector and coverage. The frontend's production build is left
|
||||
@@ -199,7 +207,9 @@ stops. Its routes:
|
||||
address may send a limited number a minute, and the report files are capped in
|
||||
size, the oldest deleted first
|
||||
- `GET /.well-known/healthcheck` — answers 200 with `"status":"ok"`, the
|
||||
server's version and its uptime
|
||||
server's version and its uptime; while reports cannot be stored, as the last
|
||||
write of a report file failed, it answers 503 with `"status":"error"` and the
|
||||
same fields (see [Report storage](#report-storage))
|
||||
- `GET /metrics` — Prometheus metrics behind basic auth, only when
|
||||
`METRICS_USERNAME` and `METRICS_PASSWORD` are set; each client address may
|
||||
make a limited number of requests to it a minute
|
||||
@@ -274,10 +284,11 @@ The reports of a failed write stay in memory, and the next flush, within a
|
||||
minute, tries again to write them. The server writes one file at a time, so a
|
||||
file holds its reports in the order they arrived, those of a failed write
|
||||
included. From the failed write until a write succeeds, `POST /api/v1/reports`
|
||||
refuses every report with 500 and keeps nothing of it. The page sends the
|
||||
refused samples again in its next report, as long as it still holds them: at the
|
||||
1-second interval one refused report already loses some of those in the history
|
||||
(see [Reporting](#reporting)). Reports already answered 200 are lost only if the
|
||||
refuses every report with 500 and keeps nothing of it, and the health check
|
||||
answers 503 with `"status":"error"`. The page sends the refused samples again in
|
||||
its next report, as long as it still holds them: at the 1-second interval one
|
||||
refused report already loses some of those in the history (see
|
||||
[Reporting](#reporting)). Reports already answered 200 are lost only if the
|
||||
server stops before a write succeeds; the final flush at stop tries them too,
|
||||
and if it fails, the process exits non-zero.
|
||||
|
||||
@@ -551,9 +562,16 @@ What the [upaas](https://git.eeqj.de/sneak/upaas) app for netwatch needs:
|
||||
stops the container. Empty, the backend sends nothing to Sentry
|
||||
- **Health check:** the image's `HEALTHCHECK` requests
|
||||
`/.well-known/healthcheck` through nginx every 30 seconds, so it fails unless
|
||||
both nginx and the backend answer. upaas reads the container's health 60
|
||||
seconds after a deploy and fails the deploy unless it is `healthy`. The
|
||||
container also stops when either process exits.
|
||||
both nginx and the backend answer. It also fails while reports cannot be
|
||||
stored, as when the volume is full or `DATA_DIR` cannot be written: the health
|
||||
check answers 503 from a failed write of a report file until a write succeeds,
|
||||
and the backend tries the write again every minute. Three failed checks in a
|
||||
row make the container `unhealthy`, and one that passes makes it `healthy`
|
||||
again. upaas reads the container's health once, 60 seconds after a deploy, and
|
||||
fails the deploy unless it is `healthy`. A container that turns `unhealthy`
|
||||
after that is neither restarted nor reported by upaas, whose `unless-stopped`
|
||||
restart policy ignores health; only `docker ps` and `docker inspect` show it.
|
||||
The container also stops when either process exits.
|
||||
|
||||
## Browser Compatibility
|
||||
|
||||
|
||||
@@ -21,6 +21,36 @@ Run `make frontend-viewport-test` in CI as its own step; it is not part of
|
||||
|
||||
# Completed Steps
|
||||
|
||||
- 2026-10-19: `script/bootstrap` keeps the `yarn` on `PATH` only when
|
||||
`yarn --version` is exactly `YARN_VERSION`, and otherwise installs that
|
||||
version as it did when no yarn was installed
|
||||
([#154](https://git.eeqj.de/sneak/netwatch/issues/154)); any yarn used to be
|
||||
kept. It uses the `node` on `PATH` only when that node has the major version
|
||||
of `NODE_VERSION`, as `REPO_POLICIES.md` asks, and is at least
|
||||
`NODE_MIN_VERSION`; a node 24 was kept before. After installing node or yarn
|
||||
it looks the tool up through `PATH` again and stops if that one fails the same
|
||||
check, as it does for Go, and it prints the node and yarn versions it uses.
|
||||
Every script that runs yarn on the host, except `script/bootstrap`, which
|
||||
picks its own node and yarn, runs it through the new `script/yarn`. That
|
||||
script puts `~/.local/bin` first on `PATH`, uses the first `node` on that
|
||||
`PATH` when it passes the same test, and otherwise runs yarn under the pinned
|
||||
node in nvm; `make build` ran yarn under whatever node was first on `PATH`
|
||||
- 2026-10-19: `bin/entrypoint.sh` has each of its steps in a function, and a
|
||||
`main` at the end calls them in order, as the shell section of the code style
|
||||
guide asks ([#163](https://git.eeqj.de/sneak/netwatch/issues/163)). It is
|
||||
still POSIX sh with no `set -e`, and does what it did, with the same messages
|
||||
and exit codes. Checked by running the image with each script: a normal start,
|
||||
bad `PORT`, `TRUSTED_PROXIES` and `DATA_DIR` values, `docker stop`, INT, and
|
||||
nginx exiting on its own
|
||||
- 2026-10-19: while report files cannot be written, the health check answers 503
|
||||
with `"status":"error"` and the same fields, so the image's `HEALTHCHECK`
|
||||
reports the container `unhealthy`
|
||||
([#142](https://git.eeqj.de/sneak/netwatch/issues/142)). It answered 200
|
||||
`"status":"ok"` whatever happened, so the platform saw a healthy container
|
||||
while every report was refused with 500. It answers 200 at start, before any
|
||||
write has failed, and again from the first write that succeeds. A unit test
|
||||
removes `DATA_DIR` to make a write fail and checks the 503, then puts it back
|
||||
and checks the 200 after the next write
|
||||
- 2026-10-19: report file writes no longer overlap
|
||||
([#150](https://git.eeqj.de/sneak/netwatch/issues/150)), so the reports of
|
||||
failed writes are written in the order they arrived. Two writes that failed at
|
||||
|
||||
+91
-63
@@ -13,24 +13,27 @@ set -u
|
||||
# address and start anyway, and reports a bad port without naming
|
||||
# PORT, so a value that is not a usable port stops the container here,
|
||||
# before either process starts.
|
||||
export PORT="${PORT:-8080}"
|
||||
case "$PORT" in
|
||||
*[!0-9]*)
|
||||
echo "entrypoint: PORT must be a port number, not '$PORT'" >&2
|
||||
check_port() {
|
||||
export PORT="${PORT:-8080}"
|
||||
case "$PORT" in
|
||||
*[!0-9]*)
|
||||
echo "entrypoint: PORT must be a port number, not '$PORT'" >&2
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
# The length is checked first because, for a number too big for it,
|
||||
# the shell's test prints an error and is false, so the range checks
|
||||
# alone would let it through.
|
||||
if [ "${#PORT}" -gt 5 ] || [ "$PORT" -lt 1 ] || [ "$PORT" -gt 65535 ]; then
|
||||
echo "entrypoint: PORT must be from 1 to 65535, not '$PORT'" >&2
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
# The length is checked first because, for a number too big for it,
|
||||
# the shell's test prints an error and is false, so the range checks
|
||||
# alone would let it through.
|
||||
if [ "${#PORT}" -gt 5 ] || [ "$PORT" -lt 1 ] || [ "$PORT" -gt 65535 ]; then
|
||||
echo "entrypoint: PORT must be from 1 to 65535, not '$PORT'" >&2
|
||||
exit 1
|
||||
fi
|
||||
if [ "$PORT" -eq 8081 ]; then
|
||||
echo "entrypoint: PORT cannot be 8081, netwatch-server listens there" >&2
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
if [ "$PORT" -eq 8081 ]; then
|
||||
echo "entrypoint: PORT cannot be 8081, netwatch-server" \
|
||||
"listens there" >&2
|
||||
exit 1
|
||||
fi
|
||||
}
|
||||
|
||||
# TRUSTED_PROXIES names the reverse proxies in front of the container,
|
||||
# as IP addresses or CIDRs separated by commas. nginx takes the client
|
||||
@@ -47,21 +50,23 @@ fi
|
||||
# TRUSTED_PROXIES. Its error, naming the CIDR, is dropped for the one
|
||||
# below, naming the entry as written. set -f keeps a * in an entry from
|
||||
# becoming a list of file names.
|
||||
TRUSTED_PROXIES="${TRUSTED_PROXIES-10.0.0.0/8,172.16.0.0/12,192.168.0.0/16}"
|
||||
set -f
|
||||
for proxy in $(printf '%s' "$TRUSTED_PROXIES" | tr ',' ' '); do
|
||||
case "$proxy" in
|
||||
*/*) cidr="$proxy" ;;
|
||||
*:*) cidr="$proxy/128" ;;
|
||||
*) cidr="$proxy/32" ;;
|
||||
esac
|
||||
if ! netwatch-server check-cidr "$cidr" 2> /dev/null; then
|
||||
echo "entrypoint: TRUSTED_PROXIES must be IP addresses or CIDRs" \
|
||||
"separated by commas; '$proxy' is neither" >&2
|
||||
exit 1
|
||||
fi
|
||||
echo "set_real_ip_from $cidr;"
|
||||
done > /etc/nginx/trusted-proxies.conf
|
||||
write_trusted_proxies() {
|
||||
TRUSTED_PROXIES="${TRUSTED_PROXIES-10.0.0.0/8,172.16.0.0/12,192.168.0.0/16}"
|
||||
set -f
|
||||
for proxy in $(printf '%s' "$TRUSTED_PROXIES" | tr ',' ' '); do
|
||||
case "$proxy" in
|
||||
*/*) cidr="$proxy" ;;
|
||||
*:*) cidr="$proxy/128" ;;
|
||||
*) cidr="$proxy/32" ;;
|
||||
esac
|
||||
if ! netwatch-server check-cidr "$cidr" 2> /dev/null; then
|
||||
echo "entrypoint: TRUSTED_PROXIES must be IP addresses or CIDRs" \
|
||||
"separated by commas; '$proxy' is neither" >&2
|
||||
exit 1
|
||||
fi
|
||||
echo "set_real_ip_from $cidr;"
|
||||
done > /etc/nginx/trusted-proxies.conf
|
||||
}
|
||||
|
||||
# netwatch-server keeps its report files in DATA_DIR, on the /data
|
||||
# volume, which may be a host directory owned by root or by another
|
||||
@@ -73,12 +78,10 @@ done > /etc/nginx/trusted-proxies.conf
|
||||
# naming DATA_DIR, unless DATA_DIR is /data or a path below it, and it
|
||||
# acts on nothing outside /data, whatever symbolic or hard links it
|
||||
# meets there.
|
||||
export DATA_DIR="${DATA_DIR:-/data/reports}"
|
||||
netwatch-server prepare-data-dir "$DATA_DIR" || exit 1
|
||||
|
||||
# A stop signal is only noted here; the loop below acts on it.
|
||||
stop_requested=""
|
||||
trap 'stop_requested=yes' TERM INT
|
||||
prepare_data_dir() {
|
||||
export DATA_DIR="${DATA_DIR:-/data/reports}"
|
||||
netwatch-server prepare-data-dir "$DATA_DIR" || exit 1
|
||||
}
|
||||
|
||||
# netwatch-server runs as the netwatch user and listens on loopback
|
||||
# only, on a port other than the public one; configs/nginx.conf proxies
|
||||
@@ -87,9 +90,11 @@ trap 'stop_requested=yes' TERM INT
|
||||
# the container has. The netwatch user has no login shell, hence -s
|
||||
# /bin/sh. busybox su replaces itself with the command instead of
|
||||
# staying on as its parent, so $! is the server's own PID.
|
||||
BIND_ADDRESS=127.0.0.1 PORT=8081 TRUSTED_PROXIES=127.0.0.1/32 \
|
||||
su -s /bin/sh netwatch -c 'exec netwatch-server' &
|
||||
backend=$!
|
||||
start_backend() {
|
||||
BIND_ADDRESS=127.0.0.1 PORT=8081 TRUSTED_PROXIES=127.0.0.1/32 \
|
||||
su -s /bin/sh netwatch -c 'exec netwatch-server' &
|
||||
backend=$!
|
||||
}
|
||||
|
||||
# nginx starts through the nginx image's own entrypoint, which applies
|
||||
# the image's start-up configuration and then replaces itself with
|
||||
@@ -98,9 +103,11 @@ backend=$!
|
||||
# that rendering to PORT: a variable nginx itself uses, such as $uri,
|
||||
# would otherwise be replaced by an environment variable of the same
|
||||
# name.
|
||||
NGINX_ENVSUBST_FILTER='^PORT$' \
|
||||
/docker-entrypoint.sh nginx -g 'daemon off;' &
|
||||
nginx=$!
|
||||
start_nginx() {
|
||||
NGINX_ENVSUBST_FILTER='^PORT$' \
|
||||
/docker-entrypoint.sh nginx -g 'daemon off;' &
|
||||
nginx=$!
|
||||
}
|
||||
|
||||
running() {
|
||||
kill -0 "$1" 2>/dev/null
|
||||
@@ -109,26 +116,47 @@ running() {
|
||||
# POSIX sh cannot wait for whichever of two children exits first, so
|
||||
# look once a second. The shell collects a child that has exited while
|
||||
# it runs sleep, and running() is false for that child from then on.
|
||||
while [ -z "$stop_requested" ] && running "$backend" && running "$nginx"; do
|
||||
sleep 1
|
||||
done
|
||||
wait_for_stop_or_exit() {
|
||||
while [ -z "$stop_requested" ] && running "$backend" && running "$nginx"; do
|
||||
sleep 1
|
||||
done
|
||||
}
|
||||
|
||||
# Stop both, then wait until neither is left.
|
||||
kill -TERM "$backend" "$nginx" 2>/dev/null
|
||||
while running "$backend" || running "$nginx"; do
|
||||
sleep 1
|
||||
done
|
||||
|
||||
wait "$backend"
|
||||
backend_status=$?
|
||||
wait "$nginx"
|
||||
nginx_status=$?
|
||||
echo "entrypoint: netwatch-server exited $backend_status," \
|
||||
"nginx exited $nginx_status"
|
||||
stop_both() {
|
||||
kill -TERM "$backend" "$nginx" 2>/dev/null
|
||||
while running "$backend" || running "$nginx"; do
|
||||
sleep 1
|
||||
done
|
||||
}
|
||||
|
||||
# Success is a requested stop that both processes exited cleanly from.
|
||||
if [ -n "$stop_requested" ] && [ "$backend_status" -eq 0 ] &&
|
||||
[ "$nginx_status" -eq 0 ]; then
|
||||
exit 0
|
||||
fi
|
||||
exit 1
|
||||
exit_with_status() {
|
||||
wait "$backend"
|
||||
backend_status=$?
|
||||
wait "$nginx"
|
||||
nginx_status=$?
|
||||
echo "entrypoint: netwatch-server exited $backend_status," \
|
||||
"nginx exited $nginx_status"
|
||||
if [ -n "$stop_requested" ] && [ "$backend_status" -eq 0 ] &&
|
||||
[ "$nginx_status" -eq 0 ]; then
|
||||
exit 0
|
||||
fi
|
||||
exit 1
|
||||
}
|
||||
|
||||
main() {
|
||||
check_port
|
||||
write_trusted_proxies
|
||||
prepare_data_dir
|
||||
# A stop signal is only noted here; wait_for_stop_or_exit acts on it.
|
||||
stop_requested=""
|
||||
trap 'stop_requested=yes' TERM INT
|
||||
start_backend
|
||||
start_nginx
|
||||
wait_for_stop_or_exit
|
||||
stop_both
|
||||
exit_with_status
|
||||
}
|
||||
|
||||
main "$@"
|
||||
|
||||
@@ -3,9 +3,17 @@ package handlers
|
||||
import "net/http"
|
||||
|
||||
// HandleHealthCheck returns a handler for the health check
|
||||
// endpoint.
|
||||
// endpoint. It answers 200 while the status is "ok" and 503 otherwise,
|
||||
// so that the image's HEALTHCHECK fails.
|
||||
func (s *Handlers) HandleHealthCheck() http.HandlerFunc {
|
||||
return func(w http.ResponseWriter, r *http.Request) {
|
||||
s.respondJSON(w, r, s.hc.Healthcheck(), http.StatusOK)
|
||||
resp := s.hc.Healthcheck()
|
||||
|
||||
code := http.StatusOK
|
||||
if resp.Status != "ok" {
|
||||
code = http.StatusServiceUnavailable
|
||||
}
|
||||
|
||||
s.respondJSON(w, r, resp, code)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -9,17 +9,19 @@ import (
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"sneak.berlin/go/netwatch/internal/config"
|
||||
"sneak.berlin/go/netwatch/internal/globals"
|
||||
"sneak.berlin/go/netwatch/internal/handlers"
|
||||
"sneak.berlin/go/netwatch/internal/healthcheck"
|
||||
"sneak.berlin/go/netwatch/internal/logger"
|
||||
"sneak.berlin/go/netwatch/internal/reportbuf"
|
||||
|
||||
"go.uber.org/fx/fxtest"
|
||||
)
|
||||
|
||||
// newStartedHandlers builds Handlers with a real health check for the
|
||||
// server named in g, and starts them, which records the time the
|
||||
// uptime counts from.
|
||||
// server named in g, and a report buffer in a temporary directory, and
|
||||
// starts them, which records the time the uptime counts from.
|
||||
func newStartedHandlers(t *testing.T, g *globals.Globals) *handlers.Handlers {
|
||||
t.Helper()
|
||||
|
||||
@@ -30,14 +32,23 @@ func newStartedHandlers(t *testing.T, g *globals.Globals) *handlers.Handlers {
|
||||
t.Fatalf("logger: %v", err)
|
||||
}
|
||||
|
||||
buf, err := reportbuf.New(lc, reportbuf.Params{
|
||||
Config: &config.Config{DataDir: t.TempDir(), DataDirMaxBytes: 1 << 20},
|
||||
Logger: log,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("report buffer: %v", err)
|
||||
}
|
||||
|
||||
hc, err := healthcheck.New(lc,
|
||||
healthcheck.Params{Globals: g, Logger: log})
|
||||
healthcheck.Params{Buffer: buf, Globals: g, Logger: log})
|
||||
if err != nil {
|
||||
t.Fatalf("health check: %v", err)
|
||||
}
|
||||
|
||||
h, err := handlers.New(lc,
|
||||
handlers.Params{Globals: g, Healthcheck: hc, Logger: log})
|
||||
h, err := handlers.New(lc, handlers.Params{
|
||||
Buffer: buf, Globals: g, Healthcheck: hc, Logger: log,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("handlers: %v", err)
|
||||
}
|
||||
@@ -48,10 +59,12 @@ func newStartedHandlers(t *testing.T, g *globals.Globals) *handlers.Handlers {
|
||||
return h
|
||||
}
|
||||
|
||||
// TestHandleHealthCheck checks the health check's answer: 200, a JSON
|
||||
// content type, and a JSON object with exactly the fields of
|
||||
// healthcheck.HealthcheckResponse, carrying this server's name and
|
||||
// version and an uptime counted from its start.
|
||||
// TestHandleHealthCheck checks the health check's answer at start, when
|
||||
// no write of a report file has failed: 200, a JSON content type, and a
|
||||
// JSON object with exactly the fields of healthcheck.HealthcheckResponse,
|
||||
// carrying "status":"ok", this server's name and version and an uptime
|
||||
// counted from its start. The answer while writes fail is checked in
|
||||
// internal/reportbuf, whose tests can make a write fail.
|
||||
func TestHandleHealthCheck(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
|
||||
@@ -10,6 +10,7 @@ import (
|
||||
"sneak.berlin/go/netwatch/internal/config"
|
||||
"sneak.berlin/go/netwatch/internal/globals"
|
||||
"sneak.berlin/go/netwatch/internal/logger"
|
||||
"sneak.berlin/go/netwatch/internal/reportbuf"
|
||||
|
||||
"go.uber.org/fx"
|
||||
)
|
||||
@@ -18,6 +19,7 @@ import (
|
||||
type Params struct {
|
||||
fx.In
|
||||
|
||||
Buffer *reportbuf.Buffer
|
||||
Config *config.Config
|
||||
Globals *globals.Globals
|
||||
Logger *logger.Logger
|
||||
@@ -68,12 +70,18 @@ func New(
|
||||
return s, nil
|
||||
}
|
||||
|
||||
// Healthcheck returns the current health status of the application.
|
||||
// Healthcheck returns the current health status of the application:
|
||||
// "ok", or "error" while report files cannot be written.
|
||||
func (s *Healthcheck) Healthcheck() *HealthcheckResponse {
|
||||
status := "ok"
|
||||
if s.params.Buffer.LastWriteFailed() {
|
||||
status = "error"
|
||||
}
|
||||
|
||||
return &HealthcheckResponse{
|
||||
Appname: s.params.Globals.Appname,
|
||||
Now: time.Now().UTC().Format(time.RFC3339Nano),
|
||||
Status: "ok",
|
||||
Status: status,
|
||||
UptimeHuman: s.uptime().String(),
|
||||
UptimeSeconds: int64(s.uptime().Seconds()),
|
||||
Version: s.params.Globals.Version,
|
||||
|
||||
@@ -255,6 +255,16 @@ func (b *Buffer) Append(v any) error {
|
||||
return nil
|
||||
}
|
||||
|
||||
// LastWriteFailed reports whether the last write of a report file to
|
||||
// finish failed: false at start until a write fails, and false again
|
||||
// once a write succeeds. While it is true, Append refuses reports.
|
||||
func (b *Buffer) LastWriteFailed() bool {
|
||||
b.mu.Lock()
|
||||
defer b.mu.Unlock()
|
||||
|
||||
return b.writeErr != nil
|
||||
}
|
||||
|
||||
// deleteOldestFiles deletes report files, oldest first, until n more
|
||||
// bytes fit under maxBytes. It deletes none when the reports waiting
|
||||
// to be written leave no room for n even with every file gone, since
|
||||
|
||||
@@ -1074,6 +1074,51 @@ func TestReportsRefusedWhileWritesFail(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// TestHealthCheckFailsWhileWritesFail asks the health check handler
|
||||
// for its answer around a failed write. It answers 200 with
|
||||
// "status":"ok" at start, 503 with "status":"error" from the failed
|
||||
// write until a write succeeds, and 200 with "status":"ok" again after.
|
||||
func TestHealthCheckFailsWhileWritesFail(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
t.Setenv("DATA_DIR", dir)
|
||||
|
||||
buf, h := startBufferWithHandlers(t)
|
||||
|
||||
checkHealthCheck(t, h, "at start", http.StatusOK, "ok")
|
||||
|
||||
// A flush writes a file only when a report is waiting.
|
||||
err := buf.Append(map[string]string{"id": "kept"})
|
||||
if err != nil {
|
||||
t.Fatalf("append: %v", err)
|
||||
}
|
||||
|
||||
// With the data directory gone, the write has nowhere to go.
|
||||
err = os.Remove(dir)
|
||||
if err != nil {
|
||||
t.Fatalf("remove data dir: %v", err)
|
||||
}
|
||||
|
||||
err = buf.Flush()
|
||||
if err == nil {
|
||||
t.Fatal("flush succeeded with the data directory gone")
|
||||
}
|
||||
|
||||
checkHealthCheck(t, h, "after the failed write",
|
||||
http.StatusServiceUnavailable, "error")
|
||||
|
||||
err = os.Mkdir(dir, 0o750)
|
||||
if err != nil {
|
||||
t.Fatalf("make data dir: %v", err)
|
||||
}
|
||||
|
||||
err = buf.Flush()
|
||||
if err != nil {
|
||||
t.Fatalf("flush with the data directory back: %v", err)
|
||||
}
|
||||
|
||||
checkHealthCheck(t, h, "after a write succeeded", http.StatusOK, "ok")
|
||||
}
|
||||
|
||||
// startBufferWithHandlers starts a Buffer and the Handlers that store
|
||||
// reports in it through fx, as main does, with the DATA_DIR the calling
|
||||
// test has set.
|
||||
@@ -1121,6 +1166,39 @@ func postReport(t *testing.T, h *handlers.Handlers, clientID string) int {
|
||||
return rec.Code
|
||||
}
|
||||
|
||||
// checkHealthCheck asks the health check handler for its answer, and
|
||||
// fails the test, saying when it asked, unless the answer has status
|
||||
// code wantCode and a "status" field of wantStatus.
|
||||
func checkHealthCheck(
|
||||
t *testing.T,
|
||||
h *handlers.Handlers,
|
||||
when string,
|
||||
wantCode int,
|
||||
wantStatus string,
|
||||
) {
|
||||
t.Helper()
|
||||
|
||||
rec := httptest.NewRecorder()
|
||||
req := httptest.NewRequestWithContext(t.Context(),
|
||||
http.MethodGet, "/.well-known/healthcheck", http.NoBody)
|
||||
h.HandleHealthCheck().ServeHTTP(rec, req)
|
||||
|
||||
var body struct {
|
||||
Status string `json:"status"`
|
||||
}
|
||||
|
||||
err := json.Unmarshal(rec.Body.Bytes(), &body)
|
||||
if err != nil {
|
||||
t.Fatalf("health check %s: body not JSON: %v (%q)",
|
||||
when, err, rec.Body.String())
|
||||
}
|
||||
|
||||
if rec.Code != wantCode || body.Status != wantStatus {
|
||||
t.Fatalf("health check %s: %d with status %q, want %d with %q",
|
||||
when, rec.Code, body.Status, wantCode, wantStatus)
|
||||
}
|
||||
}
|
||||
|
||||
// TestConcurrentAppendsStopAtCap appends from many goroutines at once
|
||||
// with room for exactly roomFor reports: exactly that many must be
|
||||
// taken, which holds only if Append checks and counts each report
|
||||
|
||||
@@ -23,8 +23,8 @@ main() {
|
||||
"" | -* | *[[:space:]]*) usage ;;
|
||||
esac
|
||||
cd "$ROOT"
|
||||
yarn add --dev "$1"
|
||||
yarn install --frozen-lockfile
|
||||
"$ROOT/script/yarn" add --dev "$1"
|
||||
"$ROOT/script/yarn" install --frozen-lockfile
|
||||
}
|
||||
|
||||
main "$@"
|
||||
|
||||
+55
-14
@@ -3,11 +3,14 @@
|
||||
# this repo. Idempotent: every install is guarded by a check so already
|
||||
# installed tools are skipped. Base tooling comes from nix, apt, brew,
|
||||
# or apk (detected in that order); assumes nothing is present. Node is
|
||||
# used directly if it is at least NODE_MIN_VERSION; otherwise it is
|
||||
# installed at a pinned version via nvm (installing nvm itself first,
|
||||
# from a hash-verified release archive, never curl | sh). Go, with its
|
||||
# gofmt, is used directly only if it is exactly GO_VERSION; otherwise
|
||||
# that release is installed from its hash-verified archive.
|
||||
# used directly only if it has the major version of NODE_VERSION and is
|
||||
# at least NODE_MIN_VERSION; otherwise NODE_VERSION is installed via nvm
|
||||
# (installing nvm itself first, from a hash-verified release archive,
|
||||
# never curl | sh). Yarn is used directly only if it is exactly
|
||||
# YARN_VERSION; otherwise that version is prepared through corepack,
|
||||
# or installed with npm where there is no corepack.
|
||||
# Go, with its gofmt, is used directly only if it is exactly GO_VERSION;
|
||||
# otherwise that release is installed from its hash-verified archive.
|
||||
#
|
||||
# What this script installs outside the system package manager lives
|
||||
# under $HOME and is linked into ~/.local/bin, where make and the git
|
||||
@@ -142,32 +145,63 @@ ensure_nvm() {
|
||||
rm -rf "$tmp"
|
||||
}
|
||||
|
||||
# node_ok: the node on PATH is at least NODE_MIN_VERSION. node itself
|
||||
# compares the two: major, then minor, then patch.
|
||||
# node_ok: the node on PATH has the major version of NODE_VERSION and is
|
||||
# at least NODE_MIN_VERSION. Only the major is compared with the pin,
|
||||
# as REPO_POLICIES.md asks. node itself compares the versions: major,
|
||||
# then minor, then patch. A node that fails does not pass.
|
||||
# script/bootstrap and script/yarn carry this function unchanged.
|
||||
node_ok() {
|
||||
if missing node; then return 1; fi
|
||||
if ! command -v node >/dev/null 2>&1; then return 1; fi
|
||||
node -e '
|
||||
const have = process.versions.node.split(".").map(Number);
|
||||
const want = process.argv[1].split(".").map(Number);
|
||||
const pinned = process.argv[2].split(".").map(Number);
|
||||
if (have[0] !== pinned[0]) process.exit(1);
|
||||
for (let i = 0; i < 3; i++) {
|
||||
if (have[i] !== want[i]) process.exit(have[i] > want[i] ? 0 : 1);
|
||||
}
|
||||
' "$NODE_MIN_VERSION"
|
||||
' "$NODE_MIN_VERSION" "$NODE_VERSION"
|
||||
}
|
||||
|
||||
# ensure_node: unless node_ok, install NODE_VERSION and link its node.
|
||||
# Afterwards node is looked up through PATH again and must pass node_ok.
|
||||
ensure_node() {
|
||||
if node_ok; then return 0; fi
|
||||
if node_ok; then
|
||||
echo "bootstrap: using node $(node --version)"
|
||||
return 0
|
||||
fi
|
||||
ensure_nvm
|
||||
nvm_sh "nvm install $NODE_VERSION"
|
||||
link_bin "$HOME/.nvm/versions/node/v$NODE_VERSION/bin/node" node
|
||||
hash -r
|
||||
if ! node_ok; then
|
||||
echo "bootstrap: after installing node $NODE_VERSION through nvm," >&2
|
||||
echo " the node on PATH, $(command -v node), is not it" >&2
|
||||
exit 1
|
||||
fi
|
||||
echo "bootstrap: installed node $(node --version)"
|
||||
}
|
||||
|
||||
# ensure_yarn: corepack writes its shims (pnpm and yarnpkg as well as
|
||||
# yarn) into $TOOLCHAIN rather than next to itself, and the npm fallback
|
||||
# installs there too; only yarn is linked.
|
||||
# yarn_ok: yarn --version on PATH prints exactly YARN_VERSION, compared
|
||||
# over the whole version. A yarn that fails or prints anything else does
|
||||
# not pass.
|
||||
yarn_ok() {
|
||||
if missing yarn; then return 1; fi
|
||||
version="$(yarn --version 2>/dev/null)" || return 1
|
||||
[ "$version" = "$YARN_VERSION" ]
|
||||
}
|
||||
|
||||
# ensure_yarn: unless yarn_ok, prepare YARN_VERSION and link its yarn.
|
||||
# corepack writes its shims (pnpm and yarnpkg as well as yarn) into
|
||||
# $TOOLCHAIN rather than next to itself, and the npm fallback installs
|
||||
# there too; only yarn is linked. Afterwards yarn is looked up through
|
||||
# PATH again and must pass yarn_ok, so another yarn that hides the link
|
||||
# stops bootstrap.
|
||||
ensure_yarn() {
|
||||
if ! missing yarn; then return 0; fi
|
||||
if yarn_ok; then
|
||||
echo "bootstrap: using yarn $(yarn --version)"
|
||||
return 0
|
||||
fi
|
||||
shims="$TOOLCHAIN/corepack-shims"
|
||||
mkdir -p "$shims"
|
||||
if ! missing corepack; then
|
||||
@@ -182,6 +216,13 @@ ensure_yarn() {
|
||||
shims="$TOOLCHAIN/npm-global/bin"
|
||||
fi
|
||||
link_bin "$shims/yarn" yarn
|
||||
hash -r
|
||||
if ! yarn_ok; then
|
||||
echo "bootstrap: after installing yarn $YARN_VERSION in $shims," >&2
|
||||
echo " the yarn on PATH, $(command -v yarn), does not report it" >&2
|
||||
exit 1
|
||||
fi
|
||||
echo "bootstrap: installed yarn $(yarn --version)"
|
||||
}
|
||||
|
||||
# go_ok: the go on PATH has its gofmt beside it (a Go release ships the
|
||||
|
||||
+1
-1
@@ -8,7 +8,7 @@ ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
||||
|
||||
main() {
|
||||
cd "$ROOT"
|
||||
yarn build
|
||||
"$ROOT/script/yarn" build
|
||||
|
||||
# VERSION comes from the environment. Unset or empty, it is git
|
||||
# describe, or "dev" where there is no git or no repository history.
|
||||
|
||||
+1
-1
@@ -7,7 +7,7 @@ ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
||||
|
||||
main() {
|
||||
cd "$ROOT"
|
||||
yarn dev
|
||||
"$ROOT/script/yarn" dev
|
||||
}
|
||||
|
||||
main "$@"
|
||||
|
||||
+1
-1
@@ -10,7 +10,7 @@ ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
||||
|
||||
main() {
|
||||
cd "$ROOT"
|
||||
yarn prettier --write '**/*.{js,css,html,md}'
|
||||
"$ROOT/script/yarn" prettier --write '**/*.{js,css,html,md}'
|
||||
}
|
||||
|
||||
main "$@"
|
||||
|
||||
@@ -7,7 +7,7 @@ ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
||||
|
||||
main() {
|
||||
cd "$ROOT"
|
||||
yarn prettier --check '**/*.{js,css,html,md}'
|
||||
"$ROOT/script/yarn" prettier --check '**/*.{js,css,html,md}'
|
||||
}
|
||||
|
||||
main "$@"
|
||||
|
||||
@@ -14,12 +14,14 @@ ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
||||
|
||||
main() {
|
||||
cd "$ROOT"
|
||||
NODE_OPTIONS=--test-reporter=dot timeout 90 yarn --silent run test || {
|
||||
NODE_OPTIONS=--test-reporter=dot \
|
||||
timeout 90 "$ROOT/script/yarn" --silent run test || {
|
||||
echo "--- Rerunning with every test listed for details ---"
|
||||
NODE_OPTIONS=--test-reporter=spec timeout 90 yarn --silent run test
|
||||
NODE_OPTIONS=--test-reporter=spec \
|
||||
timeout 90 "$ROOT/script/yarn" --silent run test
|
||||
exit 1
|
||||
}
|
||||
timeout 30 yarn build
|
||||
timeout 30 "$ROOT/script/yarn" build
|
||||
}
|
||||
|
||||
main "$@"
|
||||
|
||||
Executable
+55
@@ -0,0 +1,55 @@
|
||||
#!/bin/sh
|
||||
# script/yarn: run yarn with the given arguments under the node
|
||||
# script/bootstrap would use. Every script that runs yarn on the host
|
||||
# runs it through this one, except script/bootstrap, which picks its
|
||||
# own node and yarn.
|
||||
set -eu
|
||||
|
||||
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
||||
|
||||
# Must match the pins in script/bootstrap.
|
||||
NODE_VERSION="22.17.0"
|
||||
NODE_MIN_VERSION="22.13.0"
|
||||
|
||||
# node_ok: the node on PATH has the major version of NODE_VERSION and is
|
||||
# at least NODE_MIN_VERSION. Only the major is compared with the pin,
|
||||
# as REPO_POLICIES.md asks. node itself compares the versions: major,
|
||||
# then minor, then patch. A node that fails does not pass.
|
||||
# script/bootstrap and script/yarn carry this function unchanged.
|
||||
node_ok() {
|
||||
if ! command -v node >/dev/null 2>&1; then return 1; fi
|
||||
node -e '
|
||||
const have = process.versions.node.split(".").map(Number);
|
||||
const want = process.argv[1].split(".").map(Number);
|
||||
const pinned = process.argv[2].split(".").map(Number);
|
||||
if (have[0] !== pinned[0]) process.exit(1);
|
||||
for (let i = 0; i < 3; i++) {
|
||||
if (have[i] !== want[i]) process.exit(have[i] > want[i] ? 0 : 1);
|
||||
}
|
||||
' "$NODE_MIN_VERSION" "$NODE_VERSION"
|
||||
}
|
||||
|
||||
# Run the yarn script/bootstrap installed: the yarn on PATH when the
|
||||
# node on PATH passes node_ok, and otherwise yarn under the pinned node
|
||||
# in nvm. nvm is a bash script, hence the subshell.
|
||||
run_yarn() {
|
||||
if node_ok && command -v yarn >/dev/null 2>&1; then
|
||||
exec yarn "$@"
|
||||
fi
|
||||
if node_ok || [ ! -s "$HOME/.nvm/nvm.sh" ]; then
|
||||
echo "script/yarn: no yarn; run script/bootstrap first" >&2
|
||||
exit 1
|
||||
fi
|
||||
exec bash -c '. "$HOME/.nvm/nvm.sh" && nvm use "$1" >/dev/null &&
|
||||
shift && exec yarn "$@"' bash "$NODE_VERSION" "$@"
|
||||
}
|
||||
|
||||
main() {
|
||||
cd "$ROOT"
|
||||
# script/bootstrap links the node and yarn it installs into
|
||||
# ~/.local/bin, which the shell that called it may not have on PATH.
|
||||
PATH="$HOME/.local/bin:$PATH"
|
||||
run_yarn "$@"
|
||||
}
|
||||
|
||||
main "$@"
|
||||
Reference in New Issue
Block a user