5 Commits
Author SHA1 Message Date
sneak fe2deec5bb Page is sent with Cache-Control: no-cache (closes #141)
check / check (push) Waiting to run
nginx sent index.html with no Cache-Control, so a browser could keep
it after a deploy and ask for hashed assets the new build no longer
has, and show an empty page. location / now adds Cache-Control:
no-cache, which covers its try_files fallback to /index.html, and
includes security-headers.conf again, since its own add_header drops
the server's. /assets/ and the locations passed to the backend are
unchanged.

script/frontend-viewport-test fails unless / and a path that falls
back to the page carry Cache-Control: no-cache and
Content-Security-Policy, which comes from the same file as the other
security headers.

Model: opus-5-5
2026-10-19 00:41:51 +00:00
clawbot 007190b541 Clock errors no longer stop probing; legend reads >=500ms (closes #149, closes #152)
check / check (push) Waiting to run
The header's clocks start last in init, after the rounds, and an error
formatting them, as in a browser that rejects the "shortGeneric" time
zone name, leaves both clocks blank instead of stopping the page before
its first round. updateClocks moved out of init so a unit test can call
it.

The footer legend's red entry reads >=500ms, as README.md and
CONFIG.latencyColors have it. buildUI is exported so a unit test can draw
the page and check the legend against every entry of
CONFIG.latencyColors.

Model: opus-5-5
2026-10-19 01:34:45 +02:00
clawbot fd32f348b7 prepare-data-dir leaves files with a second name to their owner (closes #143)
check / check (push) Waiting to run
A hard link in /data shares its owner with a file that can be anywhere
on the same filesystem, so chowning it gave that file to the netwatch
user. A file with more than one name now keeps its owner and its path
is logged.

Owners are now set on the opened directory, file or link after
checking it, as chmodDir sets modes, so a hard link swapped in
afterwards is caught. Every other file is opened without following it;
a symbolic link is given to the netwatch user itself, and devices,
named pipes and sockets keep their owner. This is built only for
Linux: prepare-data-dir runs only in the Linux image, and macOS has
neither O_PATH nor AT_EMPTY_PATH; elsewhere it returns an error. A
failed owner change names the file.

Model: opus-5-5
2026-10-19 01:11:41 +02:00
clawbot 2532da55e9 Client address is the right-most untrusted X-Forwarded-For entry (closes #144)
check / check (push) Waiting to run
From a peer in TRUSTED_PROXIES, clientIP took the left-most
X-Forwarded-For entry, which the client writes itself behind a proxy
that appends to the header, so a client could choose the address the
rate limit counts. It now walks the entries from the right past trusted
proxies and takes the first that is not one. An entry with a port is
read as its address; with only trusted entries, or one that is not an
address at all, it uses the peer. Every X-Forwarded-For header line is
read, and X-Real-IP only when there is no X-Forwarded-For entry.

Model: opus-5-5
2026-10-19 00:28:18 +02:00
clawbot e43a710114 Summary reads --/-- after an interval change, not 0/26 in red (closes #140)
check / check (push) Waiting to run
The interval menu resets every host to pending and updates the summary,
which counted the WAN hosts with a latency recorded, none, and showed
0/26 in red until the second round after the change ended. While every
WAN host still waits for its first check, the Reachable slot now reads
--/-- in white, as at page load. updateSummary is exported for a unit
test that resets the hosts as the interval menu does; a second test
checks that the slot still reads 0/26 in red once every check failed.

Model: opus-5-5
2026-10-18 23:00:35 +02:00
15 changed files with 615 additions and 97 deletions
+16 -4
View File
@@ -86,7 +86,9 @@ either one is broken. We provide:
- `script/frontend-fmt-check` — check prettier formatting (read-only)
- `script/frontend-viewport-test` — responsive-layout verification of the built
frontend in a containerised headless Chrome (see
[test/viewport/README.md](test/viewport/README.md)). Not part of
[test/viewport/README.md](test/viewport/README.md)). Before the browser
starts, it checks that the page, and a path that falls back to it, are served
with `Cache-Control: no-cache` and `Content-Security-Policy`. Not part of
`script/check`: it takes minutes.
- `script/docker` — build the image from `Dockerfile` without the build cache,
tagged `netwatch` via `script/projectname`
@@ -212,8 +214,15 @@ replaces the default; set and empty, the server trusts no proxy. Loopback is not
in the default, so a reverse proxy on the same host has to be named, as in
`TRUSTED_PROXIES=127.0.0.1/32`. A request whose direct peer is outside this set
has its forwarded headers ignored, and the direct peer is logged and
rate-limited instead. The container image does not use this default; see
[Deployment](#deployment).
rate-limited instead. From a direct peer inside the set, the client address is
the right-most `X-Forwarded-For` entry that is not in the set: each proxy
appends the address it took the request from, so the entries further left were
written by the client and are not used. An entry written as an address with a
port, such as `198.51.100.7:5000` or `[2001:db8::1]:443`, is read as that
address. If the search from the right finds only addresses in the set, or meets
an entry that is not an address at all, such as `unknown`, the direct peer is
used. `X-Real-IP` is read only when `X-Forwarded-For` is missing or empty. The
container image does not use this default; see [Deployment](#deployment).
A variable set to a value the server cannot use, such as `PORT=abc`,
`DEBUG=maybe` or a `BIND_ADDRESS` that is not an IP address, stops it from
@@ -428,7 +437,10 @@ address nginx passes on and no other. The image:
each entry with `netwatch-server check-cidr`, which parses it as the backend
parses its own `TRUSTED_PROXIES`
- Sends access logs to stdout
- Caches static assets with immutable headers
- Sends the page with `Cache-Control: no-cache`, so a browser that keeps a copy
checks with the server that it is current before using it again; only the
files under `/assets/`, whose names change whenever their contents do, are
cached as immutable
- Sends the security headers `REPO_POLICIES.md` requires on every response, as
`configs/security-headers.conf` sets them, in place of the backend's own
- Stores reports in `DATA_DIR`, `/data/reports` by default, on the `/data`
+63
View File
@@ -21,6 +21,69 @@ Run `make frontend-viewport-test` in CI as its own step; it is not part of
# Completed Steps
- 2026-10-18: the image's nginx sends the page with `Cache-Control: no-cache`
([#141](https://git.eeqj.de/sneak/netwatch/issues/141)), so a browser that
keeps a copy checks with the server that it is current before using it again.
The page went out with no `Cache-Control`, so a browser could keep it after a
deploy and ask for files under `/assets/` that the new build no longer has,
and show an empty page. A path with no file of its own falls back to the page
and gets the same header. `location /` now includes the security headers
again, as `/assets/` does, since its own `add_header` drops the server's.
`/assets/` keeps its immutable caching. `make frontend-viewport-test` fails
unless `/` and a path that falls back to the page carry
`Cache-Control: no-cache` and `Content-Security-Policy`, which comes from the
same file as the other security headers
- 2026-10-18: a browser that cannot format the header's clocks still probes
([#149](https://git.eeqj.de/sneak/netwatch/issues/149)). The clocks name the
local time zone through `Intl.DateTimeFormat`'s `shortGeneric` option, which
Chrome before 95, Firefox before 91 and Safari before 15.4 reject with an
error; it came before the first round started, so such a browser never checked
a target. The clocks now start last, after the rounds, and an error formatting
them leaves both blank. The footer's color legend reads `>=500ms` for red,
where it read `>500ms`, so 500 ms, which is red, is in one of its colors
([#152](https://git.eeqj.de/sneak/netwatch/issues/152)). Unit tests check the
clocks with a working `Intl.DateTimeFormat` and with one that rejects
`shortGeneric`, and draw the page and check the legend against every entry of
`CONFIG.latencyColors`
- 2026-10-18: `netwatch-server prepare-data-dir` no longer changes the owner of
a file outside `/data` through a hard link in the volume
([#143](https://git.eeqj.de/sneak/netwatch/issues/143)). A file with a second
name (hard link) keeps its owner, and the start logs its path. Owners are now
set on the opened directory, file or link, never by name, so a hard link
swapped in after the check is caught too; a symbolic link is opened without
following it and given to the netwatch user itself, so the server can still
follow one in a sticky directory that anyone can write to. Devices, named
pipes and sockets keep their owner. Setting the owner on an open file that is
not a directory is built only for Linux: `prepare-data-dir` runs only in the
Linux image, and macOS, where the backend is also built for development, has
neither `O_PATH` nor `AT_EMPTY_PATH`. On any other system the package still
builds, and `prepare-data-dir` fails at the first file that is not a
directory. The test that runs as root hard-links a file, a symbolic link and a
named pipe from outside the volume into it and finds their owners unchanged
- 2026-10-18: from a trusted proxy, the backend takes the client address from
the right-most `X-Forwarded-For` entry that is not in `TRUSTED_PROXIES`
([#144](https://git.eeqj.de/sneak/netwatch/issues/144)). It took the left-most
entry, which the client writes itself behind a proxy that appends to the
header, so a client could choose the address the rate limit counts and the
request log records. An entry written as an address with a port is read as
that address. With only trusted addresses in the header, or an entry that is
not an address at all (such as `unknown`) met on the way, the direct peer is
used. `X-Real-IP` is read only without `X-Forwarded-For`, and every
`X-Forwarded-For` header line is read, not only the first. The image's nginx,
and the `TRUSTED_PROXIES=127.0.0.1/32` it starts the backend with, are
unchanged; the comment in `configs/nginx.conf` that said the backend takes the
first entry now says it takes the one address nginx passes on. Unit tests
resolve the address from chains ending in an untrusted address, in trusted
ones, of only trusted ones, with ports, with an entry that is not an address,
and split over two header lines
- 2026-10-18: after an interval change, the summary's Reachable slot reads
`--/--` in white, as at page load, until the second round since the change
ends ([#140](https://git.eeqj.de/sneak/netwatch/issues/140)). It used to read
0/26 in red from the change on, as no WAN host had a check recorded, for up to
two intervals with the network fine. The summary update now writes `--/--`
while every WAN host still waits for its first check. A unit test resets the
hosts as the interval menu does and checks the slot; another checks that it
reads 0/26 in red once every WAN host's check has failed
- 2026-10-14: stopping the server waits for a report file write already under
way, the periodic flush's or the one `Append` starts when 10 MiB are waiting,
before the final flush
+7 -4
View File
@@ -66,10 +66,13 @@ done > /etc/nginx/trusted-proxies.conf
# netwatch-server keeps its report files in DATA_DIR, on the /data
# volume, which may be a host directory owned by root or by another
# uid. Here, as root, netwatch-server prepare-data-dir creates DATA_DIR
# and gives /data and everything in it to the netwatch user, so the
# host directory needs no preparing. It stops the start, naming
# DATA_DIR, unless DATA_DIR is /data or a path below it, and it acts on
# nothing outside /data, whatever symbolic links it meets there.
# and gives /data and the directories, regular files and symbolic links
# in it to the netwatch user, so the host directory needs no preparing.
# A file with a second name (hard link) keeps its owner, as that name
# could be outside /data, and its path is logged. It stops the start,
# naming DATA_DIR, unless DATA_DIR is /data or a path below it, and it
# acts on nothing outside /data, whatever symbolic or hard links it
# meets there.
export DATA_DIR="${DATA_DIR:-/data/reports}"
netwatch-server prepare-data-dir "$DATA_DIR" || exit 1
+8 -1
View File
@@ -30,8 +30,13 @@ server {
access_log /dev/stdout combined;
error_log /dev/stderr warn;
# With no-cache, a browser never uses a copy of the page that names
# assets a newer build no longer has without first checking with the
# server that it is current.
location / {
try_files $uri $uri/ /index.html;
add_header Cache-Control no-cache;
include /etc/nginx/security-headers.conf;
}
# Cache static assets aggressively
@@ -46,7 +51,9 @@ server {
# 127.0.0.1:8081. These headers go with every request passed to it.
# X-Forwarded-For carries only the client address, as resolved by
# the real IP settings above, and not the chain the request came
# with: the backend takes the first entry, which a client can write.
# with. bin/entrypoint.sh starts the backend with
# TRUSTED_PROXIES=127.0.0.1/32, the address nginx connects from, so
# the backend takes that one entry as the client address.
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $remote_addr;
+1 -1
View File
@@ -17,6 +17,7 @@ require (
github.com/slok/go-http-metrics v0.13.0
github.com/spf13/viper v1.21.0
go.uber.org/fx v1.24.0
golang.org/x/sys v0.47.0
)
require (
@@ -41,7 +42,6 @@ require (
go.uber.org/multierr v1.10.0 // indirect
go.uber.org/zap v1.26.0 // indirect
go.yaml.in/yaml/v3 v3.0.4 // indirect
golang.org/x/sys v0.47.0 // indirect
golang.org/x/text v0.40.0 // indirect
google.golang.org/protobuf v1.36.11 // indirect
)
+44 -11
View File
@@ -11,6 +11,7 @@ import (
"net/http"
"net/netip"
"runtime/debug"
"slices"
"strings"
"time"
@@ -133,6 +134,17 @@ func ipFromHostPort(hostPort string) string {
// X-Real-IP are honoured only when the direct peer is a
// trusted proxy; otherwise the direct peer is returned so a
// spoofed header cannot forge the logged address.
//
// Each proxy appends to X-Forwarded-For the address it took the
// request from, so the entries on the right are written by the
// trusted proxies and those further left by the client. The
// client's address is therefore the right-most entry that is not
// a trusted proxy. An entry written as an address with a port is
// read as that address, both for the trust check and as the
// result. When every entry is a trusted proxy, or the walk from
// the right meets an entry that is not an address at all (such as
// "unknown"), the peer is used. X-Real-IP is read only when there
// is no X-Forwarded-For entry at all.
func clientIP(
remoteAddr string,
header http.Header,
@@ -144,28 +156,49 @@ func clientIP(
return peer
}
if xff := firstForwardedFor(header.Get("X-Forwarded-For")); xff != "" {
return xff
forwarded := forwardedFor(header)
if len(forwarded) == 0 {
if xr := strings.TrimSpace(header.Get("X-Real-IP")); validIP(xr) {
return xr
}
return peer
}
if xr := strings.TrimSpace(header.Get("X-Real-IP")); validIP(xr) {
return xr
for _, entry := range slices.Backward(forwarded) {
if !validIP(entry) {
return peer
}
if !addrInAny(entry, trusted) {
return entry
}
}
return peer
}
// firstForwardedFor returns the left-most valid address in an
// X-Forwarded-For list (the original client), or "" if none.
func firstForwardedFor(value string) string {
// forwardedFor returns the entries of X-Forwarded-For in order,
// leaving out blank ones. A proxy may add its entry as a header
// line of its own rather than append it to the line the client
// sent, so every line is read, not only the first. Some proxies
// write an entry as an address with a port, such as
// 198.51.100.7:5000 or [2001:db8::1]:443; the port is removed so
// that the entry is read as its address.
func forwardedFor(header http.Header) []string {
value := strings.Join(header.Values("X-Forwarded-For"), ",")
var entries []string
for part := range strings.SplitSeq(value, ",") {
candidate := strings.TrimSpace(part)
if validIP(candidate) {
return candidate
entry := strings.TrimSpace(part)
if entry != "" {
entries = append(entries, ipFromHostPort(entry))
}
}
return ""
return entries
}
func validIP(s string) bool {
+77 -5
View File
@@ -25,10 +25,16 @@ import (
const (
// loopbackPeer is a remote address inside the trusted-proxy allowlist.
loopbackPeer = "127.0.0.1:5000"
// loopbackIP is the address of loopbackPeer, which ClientIP returns
// when it uses the peer.
loopbackIP = "127.0.0.1"
// forwardedIP is the client address presented via X-Forwarded-For.
forwardedIP = "203.0.113.7"
// realIP is the client address presented via X-Real-IP.
realIP = "203.0.113.9"
// appendedIP is the X-Forwarded-For entry a trusted proxy appended
// after the entries the client wrote.
appendedIP = "198.51.100.7"
)
func mustPrefixes(t *testing.T, cidrs ...string) []netip.Prefix {
@@ -85,10 +91,35 @@ func clientIPCases() []clientIPCase {
want: forwardedIP,
},
{
name: "trusted proxy uses left-most of chain",
remoteAddr: "10.1.2.3:5000",
xff: forwardedIP + ", 10.1.2.3",
want: forwardedIP,
name: "trusted proxy uses right-most of chain",
remoteAddr: loopbackPeer,
xff: "203.0.113.9, " + appendedIP,
want: appendedIP,
},
{
name: "trusted proxy entries are skipped from the right",
remoteAddr: loopbackPeer,
xff: "203.0.113.9, 10.0.0.5",
want: "203.0.113.9",
},
{
name: "chain of only trusted proxies uses peer",
remoteAddr: loopbackPeer,
xff: "10.0.0.5, 10.1.2.3",
want: loopbackIP,
},
{
name: "entry that is not an address uses peer",
remoteAddr: loopbackPeer,
xff: "203.0.113.9, unknown",
want: loopbackIP,
},
{
name: "x-real-ip ignored when forwarded-for has entries",
remoteAddr: loopbackPeer,
xff: "10.0.0.5",
xRealIP: realIP,
want: loopbackIP,
},
{
name: "trusted proxy falls back to x-real-ip",
@@ -117,7 +148,7 @@ func clientIPCases() []clientIPCase {
name: "trusted proxy with garbage header uses peer",
remoteAddr: loopbackPeer,
xff: "not-an-ip",
want: "127.0.0.1",
want: loopbackIP,
},
}
}
@@ -148,6 +179,47 @@ func TestClientIP(t *testing.T) {
}
}
// TestClientIPReadsEveryForwardedForLine checks that an entry a proxy
// adds as an X-Forwarded-For line of its own is read, and the line the
// client sent before it is not taken.
func TestClientIPReadsEveryForwardedForLine(t *testing.T) {
t.Parallel()
header := http.Header{}
header.Add("X-Forwarded-For", "203.0.113.9")
header.Add("X-Forwarded-For", appendedIP)
got := middleware.ClientIP(loopbackPeer, header,
mustPrefixes(t, "127.0.0.1/32"))
if got != appendedIP {
t.Errorf("ClientIP() = %q, want %q", got, appendedIP)
}
}
// TestClientIPReadsAddressWithPort checks that an X-Forwarded-For entry
// written as an address with a port is read as that address, both for
// the trust check and as the result.
func TestClientIPReadsAddressWithPort(t *testing.T) {
t.Parallel()
trusted := mustPrefixes(t, "127.0.0.1/32", "10.0.0.0/8")
for xff, want := range map[string]string{
"203.0.113.9, " + appendedIP + ":5000": appendedIP,
"203.0.113.9, [2001:db8::1]:443": "2001:db8::1",
"203.0.113.9, 10.0.0.5:5000": "203.0.113.9",
} {
header := http.Header{}
header.Set("X-Forwarded-For", xff)
got := middleware.ClientIP(loopbackPeer, header, trusted)
if got != want {
t.Errorf("X-Forwarded-For %q: ClientIP() = %q, want %q",
xff, got, want)
}
}
}
func TestSecurityHeaders(t *testing.T) {
t.Parallel()
+62 -32
View File
@@ -18,9 +18,15 @@ var ErrDataDirOutsideVolume = errors.New(
// PrepareDataDir gets dir, the server's DATA_DIR, ready for owner, the
// user the server runs as, so that a host directory mounted at volume,
// /data in the image, needs no preparing: it creates dir, gives volume
// and everything in it to owner, and gives volume and dir the mode the
// server gives a directory it creates. dir must be volume or a path
// below it, with no '.', '..', empty part or '/' at the end.
// and the directories, regular files and symbolic links in it to
// owner, and gives volume and dir the mode the server gives a directory
// it creates. It leaves a file with a second name (hard link) as it
// is, as that name could be outside volume, and logs its path; it
// leaves a device, named pipe or socket as it is too (see chownFile).
// dir must be volume or a path below it, with no '.', '..', empty part
// or '/' at the end. On any system but Linux, the system of the image, it
// fails at the first file in volume that is not a directory (see
// chownFile).
//
// bin/entrypoint.sh runs this as root, which would follow a symbolic
// link anywhere, so every step goes through an os.Root opened on
@@ -32,10 +38,9 @@ var ErrDataDirOutsideVolume = errors.New(
// open, never following a link at the name it creates, and follows a
// link on the path only as the os.Root allows, so a relative link
// inside volume can lead it to create directories elsewhere inside
// volume. Lchown never changes what a link points to, and the modes
// are set on directories already opened (see chmodDir), so the most
// that process can do is make a step fail or wait, or act on
// something else inside volume.
// volume. The owners and modes are set on files already opened (see
// chownAll and chmodDir), so the most that process can do is make a
// step fail or wait, or act on something else inside volume.
func PrepareDataDir(volume, dir string, owner *user.User) error {
// rel is dir as a path from volume; IsLocal is false for one that
// leads out of it.
@@ -66,7 +71,7 @@ func PrepareDataDir(volume, dir string, owner *user.User) error {
return err
}
err = lchownAll(root, ".", uid, gid)
err = chownAll(root, ".", uid, gid)
if err != nil {
return err
}
@@ -79,28 +84,23 @@ func PrepareDataDir(volume, dir string, owner *user.User) error {
return chmodDir(root, rel)
}
// lchownAll gives name, a directory inside root, and everything in it
// to uid and gid. It reads each directory opened through root, not
// through root.FS(), which refuses a name that is not valid UTF-8, and
// calls Lchown on every entry, which gives a symbolic link itself to
// them, not what it points to. It goes into an entry only when the
// read found a directory there, so it follows no link it finds; one
// swapped in for that directory afterwards is followed only as the
// os.Root allows.
func lchownAll(root *os.Root, name string, uid, gid int) error {
err := root.Lchown(name, uid, gid)
if err != nil {
return err
}
dir, err := root.Open(name)
if err != nil {
return err
}
entries, err := dir.ReadDir(-1)
_ = dir.Close()
// chownAll gives name, a directory inside root, and the directories,
// regular files and symbolic links in it to uid and gid, but not the
// files chownFile leaves. It opens each through root and gives the
// open file to them, as chmodDir sets a mode on the open directory.
// Root.Lchown would not hold: it acts on whatever is at the name by
// then, and a hard link swapped in there shares its owner with a file
// that could be outside root. A symbolic link is given to them itself,
// never what it points to, because in a directory that anyone can
// write to and that has the sticky bit set, as /tmp has, the kernel
// follows a link only for the link's owner or when the link and the
// directory have the same owner. chownAll goes into an entry only when
// the read found a directory there, so it follows no link it finds;
// one swapped in afterwards for a directory is followed only as the
// os.Root allows, and one swapped in for any other file is not
// followed at all (see chownFile).
func chownAll(root *os.Root, name string, uid, gid int) error {
entries, err := chownDir(root, name, uid, gid)
if err != nil {
return err
}
@@ -108,9 +108,9 @@ func lchownAll(root *os.Root, name string, uid, gid int) error {
for _, entry := range entries {
entryName := filepath.Join(name, entry.Name())
if entry.IsDir() {
err = lchownAll(root, entryName, uid, gid)
err = chownAll(root, entryName, uid, gid)
} else {
err = root.Lchown(entryName, uid, gid)
err = chownFile(root, entryName, uid, gid)
}
if err != nil {
@@ -121,6 +121,36 @@ func lchownAll(root *os.Root, name string, uid, gid int) error {
return nil
}
// chownDir gives name, a directory inside root, to uid and gid, and
// returns what it holds. Like chmodDir, it opens name through root and
// refuses anything but a directory, which has no second name (hard
// link). It reads the open directory, not root.FS(), which refuses a
// name that is not valid UTF-8.
func chownDir(root *os.Root, name string, uid, gid int) ([]fs.DirEntry, error) {
dir, err := root.Open(name)
if err != nil {
return nil, err
}
defer func() { _ = dir.Close() }()
info, err := dir.Stat()
if err != nil {
return nil, err
}
if !info.IsDir() {
return nil, &fs.PathError{Op: "chown", Path: name, Err: syscall.ENOTDIR}
}
err = dir.Chown(uid, gid)
if err != nil {
return nil, err
}
return dir.ReadDir(-1)
}
// chmodDir gives name, a directory inside root, the mode the server
// gives a directory it creates. Root.Chmod would not hold: on Linux it
// checks that name is not a symbolic link, then sets the mode by name,
+65
View File
@@ -0,0 +1,65 @@
//go:build linux
package reportbuf
import (
"io/fs"
"log/slog"
"os"
"syscall"
"golang.org/x/sys/unix"
)
// chownFile gives name, a file inside root that is not a directory, to
// uid and gid if it is a regular file or symbolic link with one name.
// It leaves, and logs, a file of any kind with a second name (hard
// link): all the names of a file share its owner, and the other could
// be outside root. It leaves any other file with one name, such as a
// device, named pipe or socket: the server needs to own none of them.
// It opens name through root with O_PATH, which reads nothing from the
// file, so opening a device or named pipe neither waits nor acts on
// it; os.Root adds O_NOFOLLOW, and the two together open a symbolic
// link itself, never what it points to. It reads the kind of file and
// its number of names from the open file and gives that file to them,
// so a hard link swapped in at name is caught too.
//
// This file is built only on Linux: prepare-data-dir runs only in the
// Linux image, and macOS, where the backend is also built for
// development, has neither O_PATH nor AT_EMPTY_PATH. On any other
// system chownFile returns an error (see datadir_other.go).
func chownFile(root *os.Root, name string, uid, gid int) error {
file, err := root.OpenFile(name, unix.O_PATH, 0)
if err != nil {
return err
}
defer func() { _ = file.Close() }()
info, err := file.Stat()
if err != nil {
return err
}
stat, ok := info.Sys().(*syscall.Stat_t)
if !ok || stat.Nlink > 1 {
slog.Warn("owner not changed: file has a second name (hard link)",
"path", file.Name())
return nil
}
if !info.Mode().IsRegular() && info.Mode().Type() != fs.ModeSymlink {
return nil
}
// File.Chown would fail: fchown refuses a file opened with O_PATH.
// Fchownat with an empty name and AT_EMPTY_PATH acts on the open
// file instead.
err = unix.Fchownat(int(file.Fd()), "", uid, gid, unix.AT_EMPTY_PATH)
if err != nil {
return &fs.PathError{Op: "chown", Path: file.Name(), Err: err}
}
return nil
}
+21
View File
@@ -0,0 +1,21 @@
//go:build !linux
package reportbuf
import (
"errors"
"io/fs"
"os"
)
// chownFile returns an error on every system but Linux. Opening a file
// that is not a directory without reading it or following it, and
// giving it to a user on the open file, never by name, is built only
// for Linux (see datadir_linux.go): prepare-data-dir runs only in the
// Linux image, and macOS, where the backend is also built for
// development, has neither O_PATH nor AT_EMPTY_PATH. This file lets
// the package build elsewhere, such as on macOS, where a developer
// runs make build and make run.
func chownFile(_ *os.Root, name string, _, _ int) error {
return &fs.PathError{Op: "chown", Path: name, Err: errors.ErrUnsupported}
}
+40 -8
View File
@@ -254,10 +254,13 @@ func TestPrepareDataDirTakesDirectoryNamedInLatin1(t *testing.T) {
requireOwner(t, filepath.Join(latin1, "f"), uint32(uid), uint32(gid))
}
// TestPrepareDataDirGivesVolumeToOwner gives everything in the volume
// to a uid and gid that own nothing, which only root can do. A
// symbolic link in the volume to a directory outside it is given to
// them itself; what it points to is left as it was.
// TestPrepareDataDirGivesVolumeToOwner gives the directories, regular
// files and symbolic links in the volume to a uid and gid that own
// nothing, which only root can do. A symbolic link in the volume to a
// directory outside it is given to them itself; what it points to is
// left as it was. A hard link to a file outside the volume, one to a
// symbolic link outside it and one to a named pipe outside it are left
// as they were, and so is a named pipe in the volume.
func TestPrepareDataDirGivesVolumeToOwner(t *testing.T) {
t.Parallel()
@@ -274,6 +277,16 @@ func TestPrepareDataDirGivesVolumeToOwner(t *testing.T) {
t.Fatal(err)
}
err = os.Symlink("f", filepath.Join(outside, "link"))
if err != nil {
t.Fatal(err)
}
err = syscall.Mkfifo(filepath.Join(outside, "fifo"), 0o600)
if err != nil {
t.Fatal(err)
}
err = os.Mkdir(old, 0o700)
if err != nil {
t.Fatal(err)
@@ -289,6 +302,21 @@ func TestPrepareDataDirGivesVolumeToOwner(t *testing.T) {
t.Fatal(err)
}
err = syscall.Mkfifo(filepath.Join(old, "fifo"), 0o600)
if err != nil {
t.Fatal(err)
}
// os.Link does not follow a symbolic link: it gives the link itself
// a second name.
for _, name := range []string{"f", "link", "fifo"} {
err = os.Link(filepath.Join(outside, name),
filepath.Join(old, "hardlink-to-"+name))
if err != nil {
t.Fatal(err)
}
}
err = reportbuf.PrepareDataDir(volume, filepath.Join(volume, reports),
&user.User{Uid: "4242", Gid: "4343"})
if err != nil {
@@ -296,12 +324,16 @@ func TestPrepareDataDirGivesVolumeToOwner(t *testing.T) {
}
for _, path := range []string{
volume, filepath.Join(volume, reports), old,
filepath.Join(old, "f"), filepath.Join(old, "link"),
volume, filepath.Join(volume, reports), old, filepath.Join(old, "f"),
filepath.Join(old, "link"),
} {
requireOwner(t, path, 4242, 4343)
}
requireOwner(t, outside, 0, 0)
requireOwner(t, filepath.Join(outside, "f"), 0, 0)
for _, path := range []string{
outside, filepath.Join(outside, "f"), filepath.Join(outside, "link"),
filepath.Join(outside, "fifo"), filepath.Join(old, "fifo"),
} {
requireOwner(t, path, 0, 0)
}
}
+4 -1
View File
@@ -1,6 +1,9 @@
// Package reportbuf accumulates telemetry reports in memory
// and periodically flushes them to zstd-compressed JSONL files,
// deleting the oldest files to keep them under a size cap.
// deleting the oldest files to keep them under a size cap. Before the
// server starts, PrepareDataDir gets the directory those files are
// kept in, DATA_DIR, ready for the user the server runs as: it creates
// it and gives it and the volume it is on to that user.
package reportbuf
import (
+27
View File
@@ -7,6 +7,8 @@
# headless Chrome at it over CDP, and asserts on computed layout at every
# viewport width derived from the app's own CSS. See
# test/viewport/README.md for what this covers and what it cannot.
# Before the browser starts, it checks that the page is served with
# Cache-Control: no-cache and Content-Security-Policy.
#
# Deliberately not part of script/check: it needs Docker and takes far
# longer than the 20s budget make test has to stay inside.
@@ -75,6 +77,31 @@ main() {
-v "$ROOT/configs/security-headers.conf:/etc/nginx/security-headers.conf:ro" \
"$SERVER_IMAGE" > /dev/null
# The page, and a path with no file of its own, which falls back to
# it, must go out with Cache-Control: no-cache. That add_header in
# its location drops the server's own unless the location includes
# the security headers again; only Content-Security-Policy is
# checked, as it comes from the same file as the rest. curl tries up
# to ten more times, a second apart, while nginx starts. It runs
# without -S, which would print an error for every failed attempt,
# even when a later one succeeds.
for path in / /no-such-page; do
if ! headers="$(docker exec "$SERVER" curl -s -o /dev/null -D - \
--retry 10 --retry-connrefused --retry-delay 1 \
"http://127.0.0.1:8080$path")"; then
echo "frontend-viewport-test: nginx did not answer $path" >&2
exit 1
fi
headers="$(printf '%s\n' "$headers" | tr -d '\r')"
if ! printf '%s\n' "$headers" | grep -qix 'Cache-Control: no-cache' ||
! printf '%s\n' "$headers" | grep -qi '^Content-Security-Policy:'; then
echo "frontend-viewport-test: $path needs" \
"Cache-Control: no-cache and Content-Security-Policy; got:" >&2
printf '%s\n' "$headers" >&2
exit 1
fi
done
# The image's own entrypoint already exposes CDP on 9222 and passes
# --no-sandbox, so only extra flags belong here; re-specifying the
# debugging port collides with it and leaves the endpoint bound to
+131 -2
View File
@@ -6,6 +6,7 @@ import { beforeEach, test } from "node:test";
import assert from "node:assert/strict";
import {
AppState,
buildUI,
CONFIG,
greyOutUI,
hostRowHTML,
@@ -17,6 +18,8 @@ import {
measureLatency,
Reporter,
tick,
updateClocks,
updateSummary,
} from "../../web/src/main.js";
// There is no page here, so the tests stand in for it. The debug log looks
@@ -27,8 +30,10 @@ import {
// selector until the next test starts. As on a page, writing its text
// replaces its markup; the status dot greyOutUI looks for in it is not
// there. Drawing a sparkline does nothing; it looks for the pixel ratio on
// window and finds none. In each test, console.error and console.info
// print nothing and keep what they are given.
// window and finds none. The commit the page's footer names, which
// vite.config.js fills in when it builds the page, is blank. In each test,
// console.error and console.info print nothing and keep what they are
// given.
const doNothing = () => {};
let elements;
beforeEach((t) => {
@@ -50,6 +55,8 @@ const canvasContext = {
globalThis.window = {};
globalThis.requestAnimationFrame = doNothing;
globalThis.cancelAnimationFrame = doNothing;
globalThis.__COMMIT_HASH__ = "";
globalThis.__COMMIT_FULL__ = "";
globalThis.document = {
getElementById: () => null,
createElement: () => ({}),
@@ -578,6 +585,32 @@ test("a check that timed out or found its target unreachable has a grey figure a
assert.equal(latencyHex(null), "#6b7280");
});
// buildUI draws the page into #app, here a plain object. Each entry in the
// footer's color legend is a colored dot and the latencies it stands for,
// with < and > written as &lt; and &gt;. A latency takes the color of the
// first entry in CONFIG.latencyColors whose limit it is below, so the
// legend has, for each of those entries in order, a dot of its color (a
// bg- class where CONFIG.latencyColors has a text- one) and the latencies
// below its limit, or, for the last, which has no limit, the latencies
// from the limit before it up. A gray dot for offline comes after them.
test("the footer's color legend has each entry in CONFIG.latencyColors, with its color and limit, then offline", (t) => {
const app = {};
t.mock.method(globalThis.document, "getElementById", () => app);
buildUI(new AppState([]));
const legend = [
...app.innerHTML.matchAll(/(bg-\w+-500) [^"]*"><\/span>(\S+)/g),
].map(([, dot, latencies]) => [dot, latencies]);
const colors = CONFIG.latencyColors;
const expected = colors.map((color, i) => [
color.className.replace("text-", "bg-"),
color.below === Infinity
? `&gt;=${colors[i - 1].below}ms`
: `&lt;${color.below}ms`,
]);
assert.deepEqual(legend, [...expected, ["bg-gray-500", "offline"]]);
});
// A target whose checks, in turn, answered after each of latencies ms, or,
// for null, found it unreachable.
function hostAfter(latencies) {
@@ -654,6 +687,102 @@ test("the summary's min, max, median and average latency over the WAN targets' l
});
});
// In the next two tests each element of the summary that updateSummary
// writes into is a plain object, made the first time it is looked up and
// kept in elements under "#" and its id.
// The interval menu resets each target and counts the rounds from 0 again,
// then updates the summary, before any check since the change is recorded.
test("after an interval change, the summary's Reachable slot reads --/-- in white, as at page load", (t) => {
t.mock.method(
globalThis.document,
"getElementById",
(id) => (elements[`#${id}`] ??= {}),
);
const state = new AppState([]);
for (const host of state.wan) {
host.pushSample(Date.now(), { latency: 30, error: null });
}
state.tickCount = 5;
for (const host of state.allHosts) host.reset();
state.tickCount = 0;
updateSummary(state);
assert.equal(elements["#summary-reachable"].textContent, "--/--");
assert.equal(elements["#summary-reachable"].className, "text-white");
});
test("once every WAN target's check has failed, the summary's Reachable slot reads 0 of them, in red", (t) => {
t.mock.method(
globalThis.document,
"getElementById",
(id) => (elements[`#${id}`] ??= {}),
);
const state = new AppState([]);
for (const host of state.wan) {
host.pushSample(Date.now(), { latency: null, error: "timeout" });
}
updateSummary(state);
assert.equal(
elements["#summary-reachable"].textContent,
`0/${state.wan.length}`,
);
assert.equal(elements["#summary-reachable"].className, "text-red-400");
});
// In the next two tests each of the header's clocks is a plain object, made
// the first time updateClocks looks it up and kept in elements under "#" and
// its id.
// The mocked clock reads 1970-01-01T00:00:00.000Z.
test("the header's clocks show the local time with its time zone's name, and the UTC time", (t) => {
t.mock.method(
globalThis.document,
"getElementById",
(id) => (elements[`#${id}`] ??= {}),
);
t.mock.timers.enable({ apis: ["Date"] });
updateClocks();
assert.match(
elements["#clock-local"].textContent,
/^Local: \d{4}-\d\d-\d\d \d\d:\d\d:\d\d \S/,
);
assert.equal(
elements["#clock-utc"].textContent,
"UTC: 1970-01-01 00:00:00 UTC",
);
});
// Here Intl.DateTimeFormat is as in Chrome before 95, Firefox before 91 or
// Safari before 15.4: it throws a RangeError when asked for the time zone's
// "shortGeneric" name.
test("in a browser that cannot name the time zone, updating the clocks throws nothing and leaves them blank", (t) => {
t.mock.method(
globalThis.document,
"getElementById",
(id) => (elements[`#${id}`] ??= {}),
);
const DateTimeFormat = Intl.DateTimeFormat;
t.mock.method(Intl, "DateTimeFormat", function (locales, options) {
if (options?.timeZoneName === "shortGeneric") {
throw new RangeError(
"Value shortGeneric out of range for Intl.DateTimeFormat options property timeZoneName",
);
}
return new DateTimeFormat(locales, options);
});
updateClocks();
assert.equal(elements["#clock-local"].textContent, "");
assert.equal(elements["#clock-utc"].textContent, "");
});
// An app state in which, of the WAN targets, the first timedOut timed out,
// the next unreachable were found unreachable, the next answered answered
// after latency ms, and the rest have not been checked yet.
+49 -28
View File
@@ -829,7 +829,7 @@ function wanRowsHTML(wanHosts) {
return rows.join("");
}
function buildUI(state) {
export function buildUI(state) {
const app = document.getElementById("app");
app.innerHTML = `
<div class="mx-auto px-[5%] py-8">
@@ -911,7 +911,7 @@ function buildUI(state) {
<span class="inline-block w-3 h-3 rounded-full bg-lime-500 mr-1 ml-3 align-middle"></span>&lt;100ms
<span class="inline-block w-3 h-3 rounded-full bg-yellow-500 mr-1 ml-3 align-middle"></span>&lt;200ms
<span class="inline-block w-3 h-3 rounded-full bg-orange-500 mr-1 ml-3 align-middle"></span>&lt;500ms
<span class="inline-block w-3 h-3 rounded-full bg-red-500 mr-1 ml-3 align-middle"></span>&gt;500ms
<span class="inline-block w-3 h-3 rounded-full bg-red-500 mr-1 ml-3 align-middle"></span>&gt;=500ms
<span class="inline-block w-3 h-3 rounded-full bg-gray-500 mr-1 ml-3 align-middle"></span>offline
</p>
<p class="mt-2"><a href="https://git.eeqj.de/sneak/netwatch/commit/${__COMMIT_FULL__}" target="_blank" rel="noopener" class="text-gray-600 hover:text-gray-400">${__COMMIT_HASH__}</a></p>
@@ -995,7 +995,7 @@ function updateHostRow(host, index) {
SparklineRenderer.draw(canvas, host.history);
}
function updateSummary(state) {
export function updateSummary(state) {
const stats = state.wanStats();
const hstats = state.wanHistoryStats();
@@ -1014,13 +1014,20 @@ function updateSummary(state) {
const hmaxUnitEl = document.getElementById("summary-hmax-unit");
if (!reachableEl) return;
reachableEl.textContent = `${stats.reachable}/${stats.total}`;
reachableEl.className =
stats.reachable === stats.total
? "text-green-400"
: stats.reachable === 0
? "text-red-400"
: "text-yellow-400";
// Before any WAN host's check is recorded, as after an interval change,
// no host has answered or failed, so the slot reads as at page load.
if (state.wan.every((h) => h.status === "pending")) {
reachableEl.textContent = "--/--";
reachableEl.className = "text-white";
} else {
reachableEl.textContent = `${stats.reachable}/${stats.total}`;
reachableEl.className =
stats.reachable === stats.total
? "text-green-400"
: stats.reachable === 0
? "text-red-400"
: "text-yellow-400";
}
if (stats.min !== null) {
minEl.textContent = `${stats.min}`;
@@ -1107,6 +1114,33 @@ function updateHealthBox(state) {
else if (health === "healthy") log.notice(`Health: ${labels[health]}`);
}
// Writes the local time, with its time zone's name, and the UTC time into
// the header's clocks. The clocks only decorate the page: in a browser
// whose Intl.DateTimeFormat rejects the "shortGeneric" time zone name
// (Chrome before 95, Firefox before 91, Safari before 15.4) they are left
// blank.
export function updateClocks() {
let local = "";
let utc = "";
try {
const now = new Date();
const zone = new Intl.DateTimeFormat("en-US", {
timeZoneName: "shortGeneric",
})
.formatToParts(now)
.find((p) => p.type === "timeZoneName").value;
const pad = (n) => String(n).padStart(2, "0");
local = `Local: ${now.toLocaleString("sv-SE", { hour12: false })} ${zone}`;
utc =
`UTC: ${now.getUTCFullYear()}-${pad(now.getUTCMonth() + 1)}-${pad(now.getUTCDate())} ` +
`${pad(now.getUTCHours())}:${pad(now.getUTCMinutes())}:${pad(now.getUTCSeconds())} UTC`;
} catch {
// Both clocks are left blank.
}
document.getElementById("clock-local").textContent = local;
document.getElementById("clock-utc").textContent = utc;
}
// --- Debug Log Renderer ------------------------------------------------------
// One line of the debug log panel, its message set as text.
@@ -1502,24 +1536,6 @@ async function init() {
sortAndRebuildWAN(state);
});
function updateClocks() {
const now = new Date();
const pad = (n) => String(n).padStart(2, "0");
const utc =
`${now.getUTCFullYear()}-${pad(now.getUTCMonth() + 1)}-${pad(now.getUTCDate())} ` +
`${pad(now.getUTCHours())}:${pad(now.getUTCMinutes())}:${pad(now.getUTCSeconds())} UTC`;
const local =
now.toLocaleString("sv-SE", { hour12: false }) +
" " +
new Intl.DateTimeFormat("en-US", { timeZoneName: "shortGeneric" })
.formatToParts(now)
.find((p) => p.type === "timeZoneName").value;
document.getElementById("clock-local").textContent = "Local: " + local;
document.getElementById("clock-utc").textContent = "UTC: " + utc;
}
updateClocks();
setInterval(updateClocks, 1000);
// Rounds never overlap: a round first gives up the last round's checks
// if they are still waiting, and the last round then records nothing
// more. At a steady interval they never are, as they time out at 80% of
@@ -1599,6 +1615,11 @@ async function init() {
window.addEventListener("resize", () => handleResize(state));
setTimeout(() => handleResize(state), CONFIG.resizeDelay);
// The clocks start last, after the rounds: they only decorate the page,
// so nothing in them may stop the monitor from probing.
updateClocks();
setInterval(updateClocks, 1000);
}
// Bootstrap only when loaded as the page: a real DOM containing the #app