Compare commits
1
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 sneak
|
2d7a489f58 |
@@ -86,9 +86,7 @@ either one is broken. We provide:
|
||||
- `script/frontend-fmt-check` — check prettier formatting (read-only)
|
||||
- `script/frontend-viewport-test` — responsive-layout verification of the built
|
||||
frontend in a containerised headless Chrome (see
|
||||
[test/viewport/README.md](test/viewport/README.md)). Before the browser
|
||||
starts, it checks that the page, and a path that falls back to it, are served
|
||||
with `Cache-Control: no-cache` and `Content-Security-Policy`. Not part of
|
||||
[test/viewport/README.md](test/viewport/README.md)). Not part of
|
||||
`script/check`: it takes minutes.
|
||||
- `script/docker` — build the image from `Dockerfile` without the build cache,
|
||||
tagged `netwatch` via `script/projectname`
|
||||
@@ -155,26 +153,8 @@ report a delta, so only new samples are sent; the mark advances only on a
|
||||
delivered report, and while paused nothing is sent. Changing the interval clears
|
||||
each host's history on the page; the samples in it not yet reported still go in
|
||||
the next report. Delivery failure is quiet — one debug-log line per outage,
|
||||
never blocking probing. The report-building step is a pure function of host
|
||||
state.
|
||||
|
||||
A sample in the history that no report has delivered is sent only as long as the
|
||||
history holds it, so after a refused or failed report the next report carries
|
||||
the samples the history still has. Samples an interval change cleared from the
|
||||
history are kept apart, the last 100 per host, until they are delivered. The
|
||||
history keeps each host's last 100 rounds: 100 seconds at the 1-second interval,
|
||||
200 seconds at 2 seconds, 5 minutes at the default of 3 seconds and 100 minutes
|
||||
at 60 seconds. At 1 second, one refused or failed report loses the oldest 20
|
||||
seconds of each host's samples, as the next delivered report comes 2 minutes
|
||||
after the last. At 2 seconds and longer one loses nothing; samples are lost from
|
||||
the third refused report in a row at 2 seconds, from the fifth at 3 seconds, and
|
||||
later still at the longer intervals.
|
||||
|
||||
No report is sent while the page is paused, but each paused round still adds an
|
||||
entry to the history, so each report due during a pause counts as a refused one.
|
||||
A pause longer than the history spans loses the samples in the history recorded
|
||||
between the last delivered report and the pause: up to a minute's worth for each
|
||||
host if the report before the pause was delivered.
|
||||
retried at the next interval, never blocking probing. The report-building step
|
||||
is a pure function of host state.
|
||||
|
||||
### Backend
|
||||
|
||||
@@ -219,7 +199,7 @@ image, `make run` builds it and runs it on port 8080.
|
||||
| `PORT` | `8080` | HTTP listen port |
|
||||
| `DATA_DIR` | `./data/reports` | Directory for compressed reports |
|
||||
| `DATA_DIR_MAX_BYTES` | `1073741824` (1 GiB) | Most bytes of report files kept in `DATA_DIR`, oldest deleted first; see [Report limits](#report-limits) |
|
||||
| `DEBUG` | empty | Debug logging; any non-empty value, `false` and `0` included, turns it on |
|
||||
| `DEBUG` | `false` | Enable debug logging |
|
||||
| `TRUSTED_PROXIES` | RFC 1918 ranges | Comma-separated CIDRs whose `X-Forwarded-For` / `X-Real-IP` headers are trusted for client IP resolution |
|
||||
| `REPORTS_PER_MINUTE` | `60` | Reports each client address may send a minute; see [Report limits](#report-limits) |
|
||||
| `CORS_ALLOWED_ORIGINS` | empty | Comma-separated origins whose pages may call the API; see [CORS](#cors) |
|
||||
@@ -242,10 +222,10 @@ an entry that is not an address at all, such as `unknown`, the direct peer is
|
||||
used. `X-Real-IP` is read only when `X-Forwarded-For` is missing or empty. The
|
||||
container image does not use this default; see [Deployment](#deployment).
|
||||
|
||||
A variable set to a value the server cannot use, such as `PORT=abc` or a
|
||||
`BIND_ADDRESS` that is not an IP address, stops it from starting, with an error
|
||||
naming the variable. An empty variable counts as unset, except
|
||||
`TRUSTED_PROXIES`. No value of `DEBUG` stops the start.
|
||||
A variable set to a value the server cannot use, such as `PORT=abc`,
|
||||
`DEBUG=maybe` or a `BIND_ADDRESS` that is not an IP address, stops it from
|
||||
starting, with an error naming the variable. An empty variable counts as unset,
|
||||
except `TRUSTED_PROXIES`.
|
||||
|
||||
#### Report storage
|
||||
|
||||
@@ -260,19 +240,17 @@ Files are created with `O_EXCL` to prevent overwrites.
|
||||
|
||||
The reports of a failed write stay in memory, and the next flush, within a
|
||||
minute, tries again to write them. From the failed write until a write succeeds,
|
||||
`POST /api/v1/reports` refuses every report with 500 and keeps nothing of it.
|
||||
The page sends the refused samples again in its next report, as long as it still
|
||||
holds them: at the 1-second interval one refused report already loses some of
|
||||
those in the history (see [Reporting](#reporting)). Reports already answered 200
|
||||
are lost only if the server stops before a write succeeds; the final flush at
|
||||
stop tries them too, and if it fails, the process exits non-zero.
|
||||
`POST /api/v1/reports` refuses every report with 500 and keeps nothing of it, so
|
||||
the page keeps its samples and sends them again at its next report. Reports
|
||||
already answered 200 are lost only if the server stops before a write succeeds;
|
||||
the final flush at stop tries them too, and if it fails, the process exits
|
||||
non-zero.
|
||||
|
||||
#### Report limits
|
||||
|
||||
`POST /api/v1/reports` takes reports from anyone who can reach it, without
|
||||
credentials, so it is bounded instead. Each refusal below answers with the same
|
||||
`{"status":"error"}` body as any other error, except nginx's refusal of a body
|
||||
over the limit in the image.
|
||||
`{"status":"error"}` body as any other error.
|
||||
|
||||
- **JSON only.** A report must be sent with `Content-Type: application/json`,
|
||||
parameters such as `charset` allowed, as the page sends it; any other is
|
||||
@@ -280,16 +258,6 @@ over the limit in the image.
|
||||
send a `text/plain` POST, or a form, without a CORS preflight; refusing them
|
||||
keeps any site from making its visitors' browsers store reports (see
|
||||
[CORS](#cors)).
|
||||
- **Body limit.** A report body may be at most 1 MiB, the backend's limit for
|
||||
any request. Run on its own, the backend answers 413 before the route to a
|
||||
request that declares a longer body. Without a declared length, the route
|
||||
answers 413 only when the report it reads runs past 1 MiB; nothing after the
|
||||
end of the report is read, so a longer body whose report ends sooner is
|
||||
stored, and one that stops being valid JSON sooner is answered 400. In the
|
||||
image, nginx refuses a body over 1 MiB first, with or without a declared
|
||||
length, as its `client_max_body_size`, which `configs/nginx.conf` does not
|
||||
set, is 1 MiB by default. nginx answers 413 with its own HTML error page, and
|
||||
the backend never sees the request.
|
||||
- **Rate limit.** Each client address, resolved through `TRUSTED_PROXIES`, may
|
||||
send `REPORTS_PER_MINUTE` reports a minute; past that it gets 429 with
|
||||
`Retry-After: 60`. The minute slides: reports from the minute before still
|
||||
@@ -313,12 +281,6 @@ over the limit in the image.
|
||||
how much of the newest reports is kept: the default of 1 GiB is small enough
|
||||
for any host; set it to the space you can give `DATA_DIR`.
|
||||
|
||||
At the defaults, one address may send 60 reports of up to 1 MiB a minute, and a
|
||||
report made up to compress badly takes about as much room in the report files as
|
||||
its request body, so in about 17 minutes one address can fill the 1 GiB cap with
|
||||
its own reports, every report stored before them deleted to make room. Lowering
|
||||
`REPORTS_PER_MINUTE` or raising `DATA_DIR_MAX_BYTES` makes that take longer.
|
||||
|
||||
#### CORS
|
||||
|
||||
The page calls the API from the origin it is served from, so by default the
|
||||
@@ -340,16 +302,12 @@ the metrics record its duration and response size, labelled with its path,
|
||||
method and status; they also count those requests in progress, and include Go's
|
||||
runtime and process metrics. No other request is recorded: not those to
|
||||
`/metrics` itself, and not those answered before they reach either route, such
|
||||
as a CORS preflight, or a request refused with 404 for a path no route has or
|
||||
405 for a method its route does not take. Run on its own, the backend also
|
||||
answers 413 before the route to a request declaring a body length over the 1 MiB
|
||||
limit, which is not recorded. A body over the limit sent without a declared
|
||||
length reaches the route and is recorded with the status the route answers,
|
||||
which is 413 only when the report it reads runs past the limit: nothing after
|
||||
the end of the report is read. In the image neither reaches the backend: nginx
|
||||
answers both with 413 itself (see [Report limits](#report-limits)), and nothing
|
||||
of them is recorded. Clients can make up any number of paths and methods, and
|
||||
each would add labels to the metrics for as long as the server runs.
|
||||
as a CORS preflight, or a request refused with 404 for a path no route has, 405
|
||||
for a method its route does not take, or 413 for declaring a body length over
|
||||
the 1 MiB limit. A report whose body goes over the limit without declaring its
|
||||
length reaches the route, is answered 413 there, and is recorded with that
|
||||
status. Clients can make up any number of paths and methods, and each would add
|
||||
labels to the metrics for as long as the server runs.
|
||||
|
||||
`/metrics` is rate limited, so that its password cannot be guessed quickly: each
|
||||
client address, resolved through `TRUSTED_PROXIES`, may make 60 requests to it a
|
||||
@@ -477,10 +435,7 @@ address nginx passes on and no other. The image:
|
||||
each entry with `netwatch-server check-cidr`, which parses it as the backend
|
||||
parses its own `TRUSTED_PROXIES`
|
||||
- Sends access logs to stdout
|
||||
- Sends the page with `Cache-Control: no-cache`, so a browser that keeps a copy
|
||||
checks with the server that it is current before using it again; only the
|
||||
files under `/assets/`, whose names change whenever their contents do, are
|
||||
cached as immutable
|
||||
- Caches static assets with immutable headers
|
||||
- Sends the security headers `REPO_POLICIES.md` requires on every response, as
|
||||
`configs/security-headers.conf` sets them, in place of the backend's own
|
||||
- Stores reports in `DATA_DIR`, `/data/reports` by default, on the `/data`
|
||||
@@ -508,8 +463,7 @@ What the [upaas](https://git.eeqj.de/sneak/upaas) app for netwatch needs:
|
||||
report files may take; the oldest are deleted to stay under it
|
||||
- `CORS_ALLOWED_ORIGINS`, default empty: other origins whose pages may call
|
||||
the API
|
||||
- `DEBUG`, default empty: debug logging; any non-empty value, `false` and
|
||||
`0` included, turns it on
|
||||
- `DEBUG`, default `false`: debug logging
|
||||
- `DATA_DIR`, default `/data/reports`: the directory the reports are kept
|
||||
in: `/data` or a path below it, with no `.` or `..` part and no extra `/`.
|
||||
The container also stops if the path goes through a symbolic link that
|
||||
|
||||
@@ -21,86 +21,6 @@ Run `make frontend-viewport-test` in CI as its own step; it is not part of
|
||||
|
||||
# Completed Steps
|
||||
|
||||
- 2026-10-19: `DEBUG` unset or empty leaves debug logging off, and any other
|
||||
value, `false` and `0` included, turns it on, as the code style guide says
|
||||
([#159](https://git.eeqj.de/sneak/netwatch/issues/159)). No value of it stops
|
||||
the start any more; before, only `1`, `t`, `T`, `TRUE`, `true`, `True`, `0`,
|
||||
`f`, `F`, `FALSE`, `false` and `False` were accepted. `README.md` no longer
|
||||
gives `false` as its default
|
||||
- 2026-10-19: the three unit tests of a check whose result is discarded (its
|
||||
round given up, the user paused, or the first round) also check that the
|
||||
host's history is still empty, its status still `pending`, and nothing was
|
||||
written to the console
|
||||
([#151](https://git.eeqj.de/sneak/netwatch/issues/151)). They checked only the
|
||||
row, so a change that recorded the result and skipped only the row would have
|
||||
kept them passing
|
||||
- 2026-10-18: when the backend runs on its own, a `DATA_DIR` that cannot be
|
||||
created or read, or where no file can be created, stops the start with an
|
||||
error naming `DATA_DIR` and its value, as the other settings' errors do
|
||||
([#148](https://git.eeqj.de/sneak/netwatch/issues/148)); the first two named
|
||||
only the path, and the third was found only at the first write of a report
|
||||
file. At start the server creates a file in `DATA_DIR` and removes it. Unit
|
||||
tests start the buffer with `DATA_DIR` below a regular file and with
|
||||
`DATA_DIR` set to `/proc`, and check the error. The report file number is an
|
||||
`atomic.Int64`, as the Go style guide asks for int types
|
||||
([#161](https://git.eeqj.de/sneak/netwatch/issues/161)); file names are
|
||||
unchanged
|
||||
- 2026-10-18: the image's nginx sends the page with `Cache-Control: no-cache`
|
||||
([#141](https://git.eeqj.de/sneak/netwatch/issues/141)), so a browser that
|
||||
keeps a copy checks with the server that it is current before using it again.
|
||||
The page went out with no `Cache-Control`, so a browser could keep it after a
|
||||
deploy and ask for files under `/assets/` that the new build no longer has,
|
||||
and show an empty page. A path that `location /` answers with the page through
|
||||
its `try_files` fallback gets the same header. `location /` now includes the
|
||||
security headers again, as `/assets/` does, since its own `add_header` drops
|
||||
the server's. `/assets/` keeps its immutable caching.
|
||||
`make frontend-viewport-test` fails unless `/` and a path that falls back to
|
||||
the page carry `Cache-Control: no-cache` and `Content-Security-Policy`, which
|
||||
comes from the same file as the other security headers
|
||||
- 2026-10-18: `README.md` says what the page loses while reports are refused or
|
||||
the page is paused ([#145](https://git.eeqj.de/sneak/netwatch/issues/145)),
|
||||
how fast one address can replace the stored reports
|
||||
([#146](https://git.eeqj.de/sneak/netwatch/issues/146)), and that in the image
|
||||
nginx answers a body over 1 MiB itself
|
||||
([#147](https://git.eeqj.de/sneak/netwatch/issues/147)). A sample in the
|
||||
history that no report has delivered is sent only as long as the history holds
|
||||
it: at the 1-second interval one refused or failed report loses the oldest 20
|
||||
seconds of each host's samples, and a pause longer than the history loses
|
||||
every sample between the last delivered report and the pause. Samples an
|
||||
interval change cleared are kept apart, the last 100 per host, until they are
|
||||
delivered. At the defaults, one address sending 1 MiB reports made up to
|
||||
compress badly can fill the 1 GiB cap in about 17 minutes. nginx's
|
||||
`client_max_body_size` is also 1 MiB by default, so in the image it answers a
|
||||
larger body with 413 and its own HTML page, and the backend never sees or
|
||||
records it; the backend's own 413 sentences now say they apply to it run on
|
||||
its own. The reporter, the body limit and nginx are unchanged
|
||||
- 2026-10-18: a browser that cannot format the header's clocks still probes
|
||||
([#149](https://git.eeqj.de/sneak/netwatch/issues/149)). The clocks name the
|
||||
local time zone through `Intl.DateTimeFormat`'s `shortGeneric` option, which
|
||||
Chrome before 95, Firefox before 91 and Safari before 15.4 reject with an
|
||||
error; it came before the first round started, so such a browser never checked
|
||||
a target. The clocks now start last, after the rounds, and an error formatting
|
||||
them leaves both blank. The footer's color legend reads `>=500ms` for red,
|
||||
where it read `>500ms`, so 500 ms, which is red, is in one of its colors
|
||||
([#152](https://git.eeqj.de/sneak/netwatch/issues/152)). Unit tests check the
|
||||
clocks with a working `Intl.DateTimeFormat` and with one that rejects
|
||||
`shortGeneric`, and draw the page and check the legend against every entry of
|
||||
`CONFIG.latencyColors`
|
||||
- 2026-10-18: `netwatch-server prepare-data-dir` no longer changes the owner of
|
||||
a file outside `/data` through a hard link in the volume
|
||||
([#143](https://git.eeqj.de/sneak/netwatch/issues/143)). A file with a second
|
||||
name (hard link) keeps its owner, and the start logs its path. Owners are now
|
||||
set on the opened directory, file or link, never by name, so a hard link
|
||||
swapped in after the check is caught too; a symbolic link is opened without
|
||||
following it and given to the netwatch user itself, so the server can still
|
||||
follow one in a sticky directory that anyone can write to. Devices, named
|
||||
pipes and sockets keep their owner. Setting the owner on an open file that is
|
||||
not a directory is built only for Linux: `prepare-data-dir` runs only in the
|
||||
Linux image, and macOS, where the backend is also built for development, has
|
||||
neither `O_PATH` nor `AT_EMPTY_PATH`. On any other system the package still
|
||||
builds, and `prepare-data-dir` fails at the first file that is not a
|
||||
directory. The test that runs as root hard-links a file, a symbolic link and a
|
||||
named pipe from outside the volume into it and finds their owners unchanged
|
||||
- 2026-10-18: from a trusted proxy, the backend takes the client address from
|
||||
the right-most `X-Forwarded-For` entry that is not in `TRUSTED_PROXIES`
|
||||
([#144](https://git.eeqj.de/sneak/netwatch/issues/144)). It took the left-most
|
||||
@@ -112,11 +32,9 @@ Run `make frontend-viewport-test` in CI as its own step; it is not part of
|
||||
used. `X-Real-IP` is read only without `X-Forwarded-For`, and every
|
||||
`X-Forwarded-For` header line is read, not only the first. The image's nginx,
|
||||
and the `TRUSTED_PROXIES=127.0.0.1/32` it starts the backend with, are
|
||||
unchanged; the comment in `configs/nginx.conf` that said the backend takes the
|
||||
first entry now says it takes the one address nginx passes on. Unit tests
|
||||
resolve the address from chains ending in an untrusted address, in trusted
|
||||
ones, of only trusted ones, with ports, with an entry that is not an address,
|
||||
and split over two header lines
|
||||
unchanged. Unit tests resolve the address from chains ending in an untrusted
|
||||
address, in trusted ones, of only trusted ones, with ports, with an entry that
|
||||
is not an address, and split over two header lines
|
||||
- 2026-10-18: after an interval change, the summary's Reachable slot reads
|
||||
`--/--` in white, as at page load, until the second round since the change
|
||||
ends ([#140](https://git.eeqj.de/sneak/netwatch/issues/140)). It used to read
|
||||
|
||||
+4
-7
@@ -66,13 +66,10 @@ done > /etc/nginx/trusted-proxies.conf
|
||||
# netwatch-server keeps its report files in DATA_DIR, on the /data
|
||||
# volume, which may be a host directory owned by root or by another
|
||||
# uid. Here, as root, netwatch-server prepare-data-dir creates DATA_DIR
|
||||
# and gives /data and the directories, regular files and symbolic links
|
||||
# in it to the netwatch user, so the host directory needs no preparing.
|
||||
# A file with a second name (hard link) keeps its owner, as that name
|
||||
# could be outside /data, and its path is logged. It stops the start,
|
||||
# naming DATA_DIR, unless DATA_DIR is /data or a path below it, and it
|
||||
# acts on nothing outside /data, whatever symbolic or hard links it
|
||||
# meets there.
|
||||
# and gives /data and everything in it to the netwatch user, so the
|
||||
# host directory needs no preparing. It stops the start, naming
|
||||
# DATA_DIR, unless DATA_DIR is /data or a path below it, and it acts on
|
||||
# nothing outside /data, whatever symbolic links it meets there.
|
||||
export DATA_DIR="${DATA_DIR:-/data/reports}"
|
||||
netwatch-server prepare-data-dir "$DATA_DIR" || exit 1
|
||||
|
||||
|
||||
+1
-8
@@ -30,13 +30,8 @@ server {
|
||||
access_log /dev/stdout combined;
|
||||
error_log /dev/stderr warn;
|
||||
|
||||
# With no-cache, a browser never uses a copy of the page that names
|
||||
# assets a newer build no longer has without first checking with the
|
||||
# server that it is current.
|
||||
location / {
|
||||
try_files $uri $uri/ /index.html;
|
||||
add_header Cache-Control no-cache;
|
||||
include /etc/nginx/security-headers.conf;
|
||||
}
|
||||
|
||||
# Cache static assets aggressively
|
||||
@@ -51,9 +46,7 @@ server {
|
||||
# 127.0.0.1:8081. These headers go with every request passed to it.
|
||||
# X-Forwarded-For carries only the client address, as resolved by
|
||||
# the real IP settings above, and not the chain the request came
|
||||
# with. bin/entrypoint.sh starts the backend with
|
||||
# TRUSTED_PROXIES=127.0.0.1/32, the address nginx connects from, so
|
||||
# the backend takes that one entry as the client address.
|
||||
# with: the backend takes the first entry, which a client can write.
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $remote_addr;
|
||||
|
||||
@@ -17,7 +17,6 @@ require (
|
||||
github.com/slok/go-http-metrics v0.13.0
|
||||
github.com/spf13/viper v1.21.0
|
||||
go.uber.org/fx v1.24.0
|
||||
golang.org/x/sys v0.47.0
|
||||
)
|
||||
|
||||
require (
|
||||
@@ -42,6 +41,7 @@ require (
|
||||
go.uber.org/multierr v1.10.0 // indirect
|
||||
go.uber.org/zap v1.26.0 // indirect
|
||||
go.yaml.in/yaml/v3 v3.0.4 // indirect
|
||||
golang.org/x/sys v0.47.0 // indirect
|
||||
golang.org/x/text v0.40.0 // indirect
|
||||
google.golang.org/protobuf v1.36.11 // indirect
|
||||
)
|
||||
|
||||
@@ -41,6 +41,7 @@ var (
|
||||
"must be an origin, scheme://host with an optional port",
|
||||
)
|
||||
errNotPort = errors.New("must be a port number, 1 to 65535")
|
||||
errNotBool = errors.New("must be true or false")
|
||||
errNotIP = errors.New("must be an IP address, or empty")
|
||||
|
||||
errMetricsCredentials = errors.New(
|
||||
@@ -100,7 +101,7 @@ func New(
|
||||
viper.SetDefault("CORS_ALLOWED_ORIGINS", "")
|
||||
viper.SetDefault("DATA_DIR", "./data/reports")
|
||||
viper.SetDefault("DATA_DIR_MAX_BYTES", defaultDataDirMaxBytes)
|
||||
viper.SetDefault("DEBUG", "")
|
||||
viper.SetDefault("DEBUG", "false")
|
||||
// An empty BIND_ADDRESS listens on every interface.
|
||||
viper.SetDefault("BIND_ADDRESS", "")
|
||||
viper.SetDefault("PORT", "8080")
|
||||
@@ -119,17 +120,19 @@ func New(
|
||||
}
|
||||
}
|
||||
|
||||
// Read with strconv: viper's GetInt would read a value it cannot
|
||||
// parse as 0 instead of failing.
|
||||
// Read with strconv: viper's GetInt and GetBool would read a value
|
||||
// they cannot parse as 0 or false instead of failing.
|
||||
port, err := strconv.Atoi(viper.GetString("PORT"))
|
||||
if err != nil || port < 1 || port > math.MaxUint16 {
|
||||
return nil, fmt.Errorf("PORT %q: %w",
|
||||
viper.GetString("PORT"), errNotPort)
|
||||
}
|
||||
|
||||
// Any non-empty value turns debug logging on, "false" and "0"
|
||||
// included, as the code style guide says.
|
||||
debug := viper.GetString("DEBUG") != ""
|
||||
debug, err := strconv.ParseBool(viper.GetString("DEBUG"))
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("DEBUG %q: %w",
|
||||
viper.GetString("DEBUG"), errNotBool)
|
||||
}
|
||||
|
||||
// An empty TRUSTED_PROXIES trusts no proxy, but viper reads an
|
||||
// empty environment variable as unset and would give the default,
|
||||
@@ -169,8 +172,8 @@ func New(
|
||||
}
|
||||
|
||||
// check fails with an error naming the first setting here whose value
|
||||
// the server cannot use. New checks PORT as it reads it, and the
|
||||
// middleware checks TRUSTED_PROXIES as it parses it.
|
||||
// the server cannot use. New checks PORT and DEBUG as it reads them,
|
||||
// and the middleware checks TRUSTED_PROXIES as it parses it.
|
||||
func (s *Config) check() error {
|
||||
// viper reads a value that is not a number as 0, so this also
|
||||
// catches a mistyped setting.
|
||||
|
||||
@@ -127,52 +127,12 @@ func TestPortMustBeAPortNumber(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// TestDebugIsOnForAnyNonEmptyValue: unset or empty, debug logging is
|
||||
// off; any other value turns it on, "false" and "0" included, and none
|
||||
// stops the start.
|
||||
func TestDebugIsOnForAnyNonEmptyValue(t *testing.T) {
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
value string
|
||||
unset bool
|
||||
want bool
|
||||
}{
|
||||
{name: "unset", unset: true, want: false},
|
||||
{name: "empty", value: "", want: false},
|
||||
{name: "false", value: "false", want: true},
|
||||
{name: "0", value: "0", want: true},
|
||||
{name: "yes", value: "yes", want: true},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
// t.Setenv first, so the variable is put back as it was
|
||||
// after the test.
|
||||
t.Setenv("DEBUG", tc.value)
|
||||
// TestDebugMustBeTrueOrFalse: viper reads any other value, such as
|
||||
// "yes", as false.
|
||||
func TestDebugMustBeTrueOrFalse(t *testing.T) {
|
||||
t.Setenv("DEBUG", "yes")
|
||||
|
||||
if tc.unset {
|
||||
err := os.Unsetenv("DEBUG")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
var cfg *config.Config
|
||||
|
||||
app := fx.New(
|
||||
fx.NopLogger,
|
||||
fx.Provide(globals.New, logger.New, config.New),
|
||||
fx.Populate(&cfg),
|
||||
)
|
||||
|
||||
err := app.Err()
|
||||
if err != nil {
|
||||
t.Fatalf("config error = %v", err)
|
||||
}
|
||||
|
||||
if cfg.Debug != tc.want {
|
||||
t.Fatalf("Debug = %t, want %t", cfg.Debug, tc.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
requireConfigError(t, "DEBUG")
|
||||
}
|
||||
|
||||
// TestBindAddressMustBeAnIPAddress: a host name would be looked up
|
||||
|
||||
@@ -32,9 +32,6 @@ const (
|
||||
forwardedIP = "203.0.113.7"
|
||||
// realIP is the client address presented via X-Real-IP.
|
||||
realIP = "203.0.113.9"
|
||||
// appendedIP is the X-Forwarded-For entry a trusted proxy appended
|
||||
// after the entries the client wrote.
|
||||
appendedIP = "198.51.100.7"
|
||||
)
|
||||
|
||||
func mustPrefixes(t *testing.T, cidrs ...string) []netip.Prefix {
|
||||
@@ -93,8 +90,8 @@ func clientIPCases() []clientIPCase {
|
||||
{
|
||||
name: "trusted proxy uses right-most of chain",
|
||||
remoteAddr: loopbackPeer,
|
||||
xff: "203.0.113.9, " + appendedIP,
|
||||
want: appendedIP,
|
||||
xff: "203.0.113.9, 198.51.100.7",
|
||||
want: "198.51.100.7",
|
||||
},
|
||||
{
|
||||
name: "trusted proxy entries are skipped from the right",
|
||||
@@ -108,6 +105,24 @@ func clientIPCases() []clientIPCase {
|
||||
xff: "10.0.0.5, 10.1.2.3",
|
||||
want: loopbackIP,
|
||||
},
|
||||
{
|
||||
name: "entry with a port is read as its address",
|
||||
remoteAddr: loopbackPeer,
|
||||
xff: "203.0.113.9, 198.51.100.7:5000",
|
||||
want: "198.51.100.7",
|
||||
},
|
||||
{
|
||||
name: "ipv6 entry with a port is read as its address",
|
||||
remoteAddr: loopbackPeer,
|
||||
xff: "203.0.113.9, [2001:db8::1]:443",
|
||||
want: "2001:db8::1",
|
||||
},
|
||||
{
|
||||
name: "trusted proxy entry with a port is skipped",
|
||||
remoteAddr: loopbackPeer,
|
||||
xff: "203.0.113.9, 10.0.0.5:5000",
|
||||
want: "203.0.113.9",
|
||||
},
|
||||
{
|
||||
name: "entry that is not an address uses peer",
|
||||
remoteAddr: loopbackPeer,
|
||||
@@ -187,36 +202,12 @@ func TestClientIPReadsEveryForwardedForLine(t *testing.T) {
|
||||
|
||||
header := http.Header{}
|
||||
header.Add("X-Forwarded-For", "203.0.113.9")
|
||||
header.Add("X-Forwarded-For", appendedIP)
|
||||
header.Add("X-Forwarded-For", "198.51.100.7")
|
||||
|
||||
got := middleware.ClientIP(loopbackPeer, header,
|
||||
mustPrefixes(t, "127.0.0.1/32"))
|
||||
if got != appendedIP {
|
||||
t.Errorf("ClientIP() = %q, want %q", got, appendedIP)
|
||||
}
|
||||
}
|
||||
|
||||
// TestClientIPReadsAddressWithPort checks that an X-Forwarded-For entry
|
||||
// written as an address with a port is read as that address, both for
|
||||
// the trust check and as the result.
|
||||
func TestClientIPReadsAddressWithPort(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
trusted := mustPrefixes(t, "127.0.0.1/32", "10.0.0.0/8")
|
||||
|
||||
for xff, want := range map[string]string{
|
||||
"203.0.113.9, " + appendedIP + ":5000": appendedIP,
|
||||
"203.0.113.9, [2001:db8::1]:443": "2001:db8::1",
|
||||
"203.0.113.9, 10.0.0.5:5000": "203.0.113.9",
|
||||
} {
|
||||
header := http.Header{}
|
||||
header.Set("X-Forwarded-For", xff)
|
||||
|
||||
got := middleware.ClientIP(loopbackPeer, header, trusted)
|
||||
if got != want {
|
||||
t.Errorf("X-Forwarded-For %q: ClientIP() = %q, want %q",
|
||||
xff, got, want)
|
||||
}
|
||||
if got != "198.51.100.7" {
|
||||
t.Errorf("ClientIP() = %q, want %q", got, "198.51.100.7")
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -18,15 +18,9 @@ var ErrDataDirOutsideVolume = errors.New(
|
||||
// PrepareDataDir gets dir, the server's DATA_DIR, ready for owner, the
|
||||
// user the server runs as, so that a host directory mounted at volume,
|
||||
// /data in the image, needs no preparing: it creates dir, gives volume
|
||||
// and the directories, regular files and symbolic links in it to
|
||||
// owner, and gives volume and dir the mode the server gives a directory
|
||||
// it creates. It leaves a file with a second name (hard link) as it
|
||||
// is, as that name could be outside volume, and logs its path; it
|
||||
// leaves a device, named pipe or socket as it is too (see chownFile).
|
||||
// dir must be volume or a path below it, with no '.', '..', empty part
|
||||
// or '/' at the end. On any system but Linux, the system of the image, it
|
||||
// fails at the first file in volume that is not a directory (see
|
||||
// chownFile).
|
||||
// and everything in it to owner, and gives volume and dir the mode the
|
||||
// server gives a directory it creates. dir must be volume or a path
|
||||
// below it, with no '.', '..', empty part or '/' at the end.
|
||||
//
|
||||
// bin/entrypoint.sh runs this as root, which would follow a symbolic
|
||||
// link anywhere, so every step goes through an os.Root opened on
|
||||
@@ -38,9 +32,10 @@ var ErrDataDirOutsideVolume = errors.New(
|
||||
// open, never following a link at the name it creates, and follows a
|
||||
// link on the path only as the os.Root allows, so a relative link
|
||||
// inside volume can lead it to create directories elsewhere inside
|
||||
// volume. The owners and modes are set on files already opened (see
|
||||
// chownAll and chmodDir), so the most that process can do is make a
|
||||
// step fail or wait, or act on something else inside volume.
|
||||
// volume. Lchown never changes what a link points to, and the modes
|
||||
// are set on directories already opened (see chmodDir), so the most
|
||||
// that process can do is make a step fail or wait, or act on
|
||||
// something else inside volume.
|
||||
func PrepareDataDir(volume, dir string, owner *user.User) error {
|
||||
// rel is dir as a path from volume; IsLocal is false for one that
|
||||
// leads out of it.
|
||||
@@ -71,7 +66,7 @@ func PrepareDataDir(volume, dir string, owner *user.User) error {
|
||||
return err
|
||||
}
|
||||
|
||||
err = chownAll(root, ".", uid, gid)
|
||||
err = lchownAll(root, ".", uid, gid)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
@@ -84,23 +79,28 @@ func PrepareDataDir(volume, dir string, owner *user.User) error {
|
||||
return chmodDir(root, rel)
|
||||
}
|
||||
|
||||
// chownAll gives name, a directory inside root, and the directories,
|
||||
// regular files and symbolic links in it to uid and gid, but not the
|
||||
// files chownFile leaves. It opens each through root and gives the
|
||||
// open file to them, as chmodDir sets a mode on the open directory.
|
||||
// Root.Lchown would not hold: it acts on whatever is at the name by
|
||||
// then, and a hard link swapped in there shares its owner with a file
|
||||
// that could be outside root. A symbolic link is given to them itself,
|
||||
// never what it points to, because in a directory that anyone can
|
||||
// write to and that has the sticky bit set, as /tmp has, the kernel
|
||||
// follows a link only for the link's owner or when the link and the
|
||||
// directory have the same owner. chownAll goes into an entry only when
|
||||
// the read found a directory there, so it follows no link it finds;
|
||||
// one swapped in afterwards for a directory is followed only as the
|
||||
// os.Root allows, and one swapped in for any other file is not
|
||||
// followed at all (see chownFile).
|
||||
func chownAll(root *os.Root, name string, uid, gid int) error {
|
||||
entries, err := chownDir(root, name, uid, gid)
|
||||
// lchownAll gives name, a directory inside root, and everything in it
|
||||
// to uid and gid. It reads each directory opened through root, not
|
||||
// through root.FS(), which refuses a name that is not valid UTF-8, and
|
||||
// calls Lchown on every entry, which gives a symbolic link itself to
|
||||
// them, not what it points to. It goes into an entry only when the
|
||||
// read found a directory there, so it follows no link it finds; one
|
||||
// swapped in for that directory afterwards is followed only as the
|
||||
// os.Root allows.
|
||||
func lchownAll(root *os.Root, name string, uid, gid int) error {
|
||||
err := root.Lchown(name, uid, gid)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
dir, err := root.Open(name)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
entries, err := dir.ReadDir(-1)
|
||||
_ = dir.Close()
|
||||
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
@@ -108,9 +108,9 @@ func chownAll(root *os.Root, name string, uid, gid int) error {
|
||||
for _, entry := range entries {
|
||||
entryName := filepath.Join(name, entry.Name())
|
||||
if entry.IsDir() {
|
||||
err = chownAll(root, entryName, uid, gid)
|
||||
err = lchownAll(root, entryName, uid, gid)
|
||||
} else {
|
||||
err = chownFile(root, entryName, uid, gid)
|
||||
err = root.Lchown(entryName, uid, gid)
|
||||
}
|
||||
|
||||
if err != nil {
|
||||
@@ -121,36 +121,6 @@ func chownAll(root *os.Root, name string, uid, gid int) error {
|
||||
return nil
|
||||
}
|
||||
|
||||
// chownDir gives name, a directory inside root, to uid and gid, and
|
||||
// returns what it holds. Like chmodDir, it opens name through root and
|
||||
// refuses anything but a directory, which has no second name (hard
|
||||
// link). It reads the open directory, not root.FS(), which refuses a
|
||||
// name that is not valid UTF-8.
|
||||
func chownDir(root *os.Root, name string, uid, gid int) ([]fs.DirEntry, error) {
|
||||
dir, err := root.Open(name)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
defer func() { _ = dir.Close() }()
|
||||
|
||||
info, err := dir.Stat()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if !info.IsDir() {
|
||||
return nil, &fs.PathError{Op: "chown", Path: name, Err: syscall.ENOTDIR}
|
||||
}
|
||||
|
||||
err = dir.Chown(uid, gid)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return dir.ReadDir(-1)
|
||||
}
|
||||
|
||||
// chmodDir gives name, a directory inside root, the mode the server
|
||||
// gives a directory it creates. Root.Chmod would not hold: on Linux it
|
||||
// checks that name is not a symbolic link, then sets the mode by name,
|
||||
|
||||
@@ -1,65 +0,0 @@
|
||||
//go:build linux
|
||||
|
||||
package reportbuf
|
||||
|
||||
import (
|
||||
"io/fs"
|
||||
"log/slog"
|
||||
"os"
|
||||
"syscall"
|
||||
|
||||
"golang.org/x/sys/unix"
|
||||
)
|
||||
|
||||
// chownFile gives name, a file inside root that is not a directory, to
|
||||
// uid and gid if it is a regular file or symbolic link with one name.
|
||||
// It leaves, and logs, a file of any kind with a second name (hard
|
||||
// link): all the names of a file share its owner, and the other could
|
||||
// be outside root. It leaves any other file with one name, such as a
|
||||
// device, named pipe or socket: the server needs to own none of them.
|
||||
// It opens name through root with O_PATH, which reads nothing from the
|
||||
// file, so opening a device or named pipe neither waits nor acts on
|
||||
// it; os.Root adds O_NOFOLLOW, and the two together open a symbolic
|
||||
// link itself, never what it points to. It reads the kind of file and
|
||||
// its number of names from the open file and gives that file to them,
|
||||
// so a hard link swapped in at name is caught too.
|
||||
//
|
||||
// This file is built only on Linux: prepare-data-dir runs only in the
|
||||
// Linux image, and macOS, where the backend is also built for
|
||||
// development, has neither O_PATH nor AT_EMPTY_PATH. On any other
|
||||
// system chownFile returns an error (see datadir_other.go).
|
||||
func chownFile(root *os.Root, name string, uid, gid int) error {
|
||||
file, err := root.OpenFile(name, unix.O_PATH, 0)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
defer func() { _ = file.Close() }()
|
||||
|
||||
info, err := file.Stat()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
stat, ok := info.Sys().(*syscall.Stat_t)
|
||||
if !ok || stat.Nlink > 1 {
|
||||
slog.Warn("owner not changed: file has a second name (hard link)",
|
||||
"path", file.Name())
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
if !info.Mode().IsRegular() && info.Mode().Type() != fs.ModeSymlink {
|
||||
return nil
|
||||
}
|
||||
|
||||
// File.Chown would fail: fchown refuses a file opened with O_PATH.
|
||||
// Fchownat with an empty name and AT_EMPTY_PATH acts on the open
|
||||
// file instead.
|
||||
err = unix.Fchownat(int(file.Fd()), "", uid, gid, unix.AT_EMPTY_PATH)
|
||||
if err != nil {
|
||||
return &fs.PathError{Op: "chown", Path: file.Name(), Err: err}
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
@@ -1,21 +0,0 @@
|
||||
//go:build !linux
|
||||
|
||||
package reportbuf
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"io/fs"
|
||||
"os"
|
||||
)
|
||||
|
||||
// chownFile returns an error on every system but Linux. Opening a file
|
||||
// that is not a directory without reading it or following it, and
|
||||
// giving it to a user on the open file, never by name, is built only
|
||||
// for Linux (see datadir_linux.go): prepare-data-dir runs only in the
|
||||
// Linux image, and macOS, where the backend is also built for
|
||||
// development, has neither O_PATH nor AT_EMPTY_PATH. This file lets
|
||||
// the package build elsewhere, such as on macOS, where a developer
|
||||
// runs make build and make run.
|
||||
func chownFile(_ *os.Root, name string, _, _ int) error {
|
||||
return &fs.PathError{Op: "chown", Path: name, Err: errors.ErrUnsupported}
|
||||
}
|
||||
@@ -254,13 +254,10 @@ func TestPrepareDataDirTakesDirectoryNamedInLatin1(t *testing.T) {
|
||||
requireOwner(t, filepath.Join(latin1, "f"), uint32(uid), uint32(gid))
|
||||
}
|
||||
|
||||
// TestPrepareDataDirGivesVolumeToOwner gives the directories, regular
|
||||
// files and symbolic links in the volume to a uid and gid that own
|
||||
// nothing, which only root can do. A symbolic link in the volume to a
|
||||
// directory outside it is given to them itself; what it points to is
|
||||
// left as it was. A hard link to a file outside the volume, one to a
|
||||
// symbolic link outside it and one to a named pipe outside it are left
|
||||
// as they were, and so is a named pipe in the volume.
|
||||
// TestPrepareDataDirGivesVolumeToOwner gives everything in the volume
|
||||
// to a uid and gid that own nothing, which only root can do. A
|
||||
// symbolic link in the volume to a directory outside it is given to
|
||||
// them itself; what it points to is left as it was.
|
||||
func TestPrepareDataDirGivesVolumeToOwner(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
@@ -277,16 +274,6 @@ func TestPrepareDataDirGivesVolumeToOwner(t *testing.T) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
err = os.Symlink("f", filepath.Join(outside, "link"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
err = syscall.Mkfifo(filepath.Join(outside, "fifo"), 0o600)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
err = os.Mkdir(old, 0o700)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
@@ -302,21 +289,6 @@ func TestPrepareDataDirGivesVolumeToOwner(t *testing.T) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
err = syscall.Mkfifo(filepath.Join(old, "fifo"), 0o600)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
// os.Link does not follow a symbolic link: it gives the link itself
|
||||
// a second name.
|
||||
for _, name := range []string{"f", "link", "fifo"} {
|
||||
err = os.Link(filepath.Join(outside, name),
|
||||
filepath.Join(old, "hardlink-to-"+name))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
err = reportbuf.PrepareDataDir(volume, filepath.Join(volume, reports),
|
||||
&user.User{Uid: "4242", Gid: "4343"})
|
||||
if err != nil {
|
||||
@@ -324,16 +296,12 @@ func TestPrepareDataDirGivesVolumeToOwner(t *testing.T) {
|
||||
}
|
||||
|
||||
for _, path := range []string{
|
||||
volume, filepath.Join(volume, reports), old, filepath.Join(old, "f"),
|
||||
filepath.Join(old, "link"),
|
||||
volume, filepath.Join(volume, reports), old,
|
||||
filepath.Join(old, "f"), filepath.Join(old, "link"),
|
||||
} {
|
||||
requireOwner(t, path, 4242, 4343)
|
||||
}
|
||||
|
||||
for _, path := range []string{
|
||||
outside, filepath.Join(outside, "f"), filepath.Join(outside, "link"),
|
||||
filepath.Join(outside, "fifo"), filepath.Join(old, "fifo"),
|
||||
} {
|
||||
requireOwner(t, path, 0, 0)
|
||||
}
|
||||
requireOwner(t, outside, 0, 0)
|
||||
requireOwner(t, filepath.Join(outside, "f"), 0, 0)
|
||||
}
|
||||
|
||||
@@ -1,9 +1,6 @@
|
||||
// Package reportbuf accumulates telemetry reports in memory
|
||||
// and periodically flushes them to zstd-compressed JSONL files,
|
||||
// deleting the oldest files to keep them under a size cap. Before the
|
||||
// server starts, PrepareDataDir gets the directory those files are
|
||||
// kept in, DATA_DIR, ready for the user the server runs as: it creates
|
||||
// it and gives it and the volume it is on to that user.
|
||||
// deleting the oldest files to keep them under a size cap.
|
||||
package reportbuf
|
||||
|
||||
import (
|
||||
@@ -88,7 +85,7 @@ type Buffer struct {
|
||||
now func() time.Time
|
||||
// seq numbers the report files, so that two named in the same
|
||||
// millisecond still get different names.
|
||||
seq atomic.Int64
|
||||
seq atomic.Uint64
|
||||
stopOnce sync.Once
|
||||
// usedBytes is what Append checks against maxBytes: the size
|
||||
// of the report files in dataDir, plus the reports waiting to
|
||||
@@ -127,33 +124,16 @@ func New(
|
||||
|
||||
lc.Append(fx.Hook{
|
||||
OnStart: func(_ context.Context) error {
|
||||
// The errors name DATA_DIR, as config.New's do for the
|
||||
// settings it checks.
|
||||
err := os.MkdirAll(b.dataDir, dirPerms)
|
||||
if err != nil {
|
||||
return fmt.Errorf("DATA_DIR %q: %w", b.dataDir, err)
|
||||
}
|
||||
|
||||
// Creating and removing a file stops the start when no
|
||||
// file can be created in DATA_DIR, instead of the first
|
||||
// write of a report file failing after reports were
|
||||
// answered 200. A full disk is not caught: the empty
|
||||
// file is still created.
|
||||
f, err := os.CreateTemp(b.dataDir, "")
|
||||
if err != nil {
|
||||
return fmt.Errorf("DATA_DIR %q: %w", b.dataDir, err)
|
||||
}
|
||||
|
||||
err = errors.Join(f.Close(), os.Remove(f.Name()))
|
||||
if err != nil {
|
||||
return fmt.Errorf("DATA_DIR %q: %w", b.dataDir, err)
|
||||
return fmt.Errorf("create data dir: %w", err)
|
||||
}
|
||||
|
||||
// Report files left by earlier runs count too, and are
|
||||
// the first to be deleted to make room.
|
||||
files, err := reportFiles(b.dataDir)
|
||||
if err != nil {
|
||||
return fmt.Errorf("DATA_DIR %q: %w", b.dataDir, err)
|
||||
return err
|
||||
}
|
||||
|
||||
b.mu.Lock()
|
||||
|
||||
@@ -108,68 +108,6 @@ func TestFailedFinalFlushFailsStop(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// TestUncreatableDataDirStopsStart sets DATA_DIR below a regular file,
|
||||
// where no directory can be created, even by root, as the tests run in
|
||||
// the backend image. The start must fail with an error naming DATA_DIR
|
||||
// and its value, as the errors for the other settings do.
|
||||
func TestUncreatableDataDirStopsStart(t *testing.T) {
|
||||
file := filepath.Join(t.TempDir(), "file")
|
||||
writeBytes(t, file, 1)
|
||||
|
||||
dir := filepath.Join(file, "reports")
|
||||
t.Setenv("DATA_DIR", dir)
|
||||
|
||||
app := fxtest.New(t,
|
||||
fx.Provide(
|
||||
globals.New,
|
||||
logger.New,
|
||||
config.New,
|
||||
reportbuf.New,
|
||||
),
|
||||
fx.Invoke(func(*reportbuf.Buffer) {}),
|
||||
)
|
||||
|
||||
err := app.Start(t.Context())
|
||||
if err == nil {
|
||||
t.Fatal("start succeeded with DATA_DIR below a regular file")
|
||||
}
|
||||
|
||||
want := fmt.Sprintf("DATA_DIR %q: ", dir)
|
||||
if !strings.Contains(err.Error(), want) {
|
||||
t.Fatalf("start error = %q, want it to contain %q", err, want)
|
||||
}
|
||||
}
|
||||
|
||||
// TestUnwritableDataDirStopsStart sets DATA_DIR to /proc, which exists
|
||||
// and can be read, but where no file can be created, even by root, as
|
||||
// the tests run in the backend image: root ignores a directory the test
|
||||
// makes read-only. The start must fail with an error naming DATA_DIR and
|
||||
// its value, rather than the first write of a report file.
|
||||
func TestUnwritableDataDirStopsStart(t *testing.T) {
|
||||
dir := "/proc"
|
||||
t.Setenv("DATA_DIR", dir)
|
||||
|
||||
app := fxtest.New(t,
|
||||
fx.Provide(
|
||||
globals.New,
|
||||
logger.New,
|
||||
config.New,
|
||||
reportbuf.New,
|
||||
),
|
||||
fx.Invoke(func(*reportbuf.Buffer) {}),
|
||||
)
|
||||
|
||||
err := app.Start(t.Context())
|
||||
if err == nil {
|
||||
t.Fatal("start succeeded with DATA_DIR /proc")
|
||||
}
|
||||
|
||||
want := fmt.Sprintf("DATA_DIR %q: ", dir)
|
||||
if !strings.Contains(err.Error(), want) {
|
||||
t.Fatalf("start error = %q, want it to contain %q", err, want)
|
||||
}
|
||||
}
|
||||
|
||||
// TestStopWaitsForPeriodicWrite holds open the write the periodic flush
|
||||
// starts, then stops the app. The stop must wait until the write is
|
||||
// released, and the report must then be in its file. The test runs in
|
||||
|
||||
@@ -7,8 +7,6 @@
|
||||
# headless Chrome at it over CDP, and asserts on computed layout at every
|
||||
# viewport width derived from the app's own CSS. See
|
||||
# test/viewport/README.md for what this covers and what it cannot.
|
||||
# Before the browser starts, it checks that the page is served with
|
||||
# Cache-Control: no-cache and Content-Security-Policy.
|
||||
#
|
||||
# Deliberately not part of script/check: it needs Docker and takes far
|
||||
# longer than the 20s budget make test has to stay inside.
|
||||
@@ -77,32 +75,6 @@ main() {
|
||||
-v "$ROOT/configs/security-headers.conf:/etc/nginx/security-headers.conf:ro" \
|
||||
"$SERVER_IMAGE" > /dev/null
|
||||
|
||||
# The page, and /no-such-page, which location / answers with the
|
||||
# page through its try_files fallback, must go out with
|
||||
# Cache-Control: no-cache. That add_header in its location drops the
|
||||
# server's own unless the location includes the security headers
|
||||
# again; only Content-Security-Policy is checked, as it comes from
|
||||
# the same file as the rest. curl tries up to ten more times, a
|
||||
# second apart, while nginx starts. It runs without -S, which would
|
||||
# print an error for every failed attempt, even when a later one
|
||||
# succeeds.
|
||||
for path in / /no-such-page; do
|
||||
if ! headers="$(docker exec "$SERVER" curl -s -o /dev/null -D - \
|
||||
--retry 10 --retry-connrefused --retry-delay 1 \
|
||||
"http://127.0.0.1:8080$path")"; then
|
||||
echo "frontend-viewport-test: nginx did not answer $path" >&2
|
||||
exit 1
|
||||
fi
|
||||
headers="$(printf '%s\n' "$headers" | tr -d '\r')"
|
||||
if ! printf '%s\n' "$headers" | grep -qix 'Cache-Control: no-cache' ||
|
||||
! printf '%s\n' "$headers" | grep -qi '^Content-Security-Policy:'; then
|
||||
echo "frontend-viewport-test: $path needs" \
|
||||
"Cache-Control: no-cache and Content-Security-Policy; got:" >&2
|
||||
printf '%s\n' "$headers" >&2
|
||||
exit 1
|
||||
fi
|
||||
done
|
||||
|
||||
# The image's own entrypoint already exposes CDP on 9222 and passes
|
||||
# --no-sandbox, so only extra flags belong here; re-specifying the
|
||||
# debugging port collides with it and leaves the endpoint bound to
|
||||
|
||||
+11
-106
@@ -6,7 +6,6 @@ import { beforeEach, test } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import {
|
||||
AppState,
|
||||
buildUI,
|
||||
CONFIG,
|
||||
greyOutUI,
|
||||
hostRowHTML,
|
||||
@@ -18,7 +17,6 @@ import {
|
||||
measureLatency,
|
||||
Reporter,
|
||||
tick,
|
||||
updateClocks,
|
||||
updateSummary,
|
||||
} from "../../web/src/main.js";
|
||||
|
||||
@@ -30,10 +28,8 @@ import {
|
||||
// selector until the next test starts. As on a page, writing its text
|
||||
// replaces its markup; the status dot greyOutUI looks for in it is not
|
||||
// there. Drawing a sparkline does nothing; it looks for the pixel ratio on
|
||||
// window and finds none. The commit the page's footer names, which
|
||||
// vite.config.js fills in when it builds the page, is blank. In each test,
|
||||
// console.error and console.info print nothing and keep what they are
|
||||
// given.
|
||||
// window and finds none. In each test, console.error and console.info
|
||||
// print nothing and keep what they are given.
|
||||
const doNothing = () => {};
|
||||
let elements;
|
||||
beforeEach((t) => {
|
||||
@@ -55,8 +51,6 @@ const canvasContext = {
|
||||
globalThis.window = {};
|
||||
globalThis.requestAnimationFrame = doNothing;
|
||||
globalThis.cancelAnimationFrame = doNothing;
|
||||
globalThis.__COMMIT_HASH__ = "";
|
||||
globalThis.__COMMIT_FULL__ = "";
|
||||
globalThis.document = {
|
||||
getElementById: () => null,
|
||||
createElement: () => ({}),
|
||||
@@ -83,11 +77,6 @@ function statusText(state, host) {
|
||||
return elements[`.status-text[data-host="${index}"]`]?.innerHTML;
|
||||
}
|
||||
|
||||
// What this test wrote to the browser console with console[method].
|
||||
function consoleLines(method) {
|
||||
return console[method].mock.calls.map((call) => call.arguments[0]);
|
||||
}
|
||||
|
||||
// Mocks the clock for test t, so that a check lasting seconds takes no real
|
||||
// time, and replaces fetch with targets that each answer after
|
||||
// answerAfter(url) milliseconds of that clock, or never when that is
|
||||
@@ -207,10 +196,9 @@ test("at a 30000ms interval, a target answering after 1000ms shows in its row wh
|
||||
});
|
||||
|
||||
// In the next three tests, the answering target's check is still waiting
|
||||
// when something happens after which its result must not be recorded or
|
||||
// show in its row, and the check must write nothing to the console.
|
||||
// when something happens after which its result must not show.
|
||||
|
||||
test("at a 30000ms interval, a check still waiting when its round is given up is not recorded and does not show in its row", async (t) => {
|
||||
test("at a 30000ms interval, a check still waiting when its round is given up does not show in its row", async (t) => {
|
||||
CONFIG.updateInterval = 30000;
|
||||
const state = new AppState([
|
||||
{ name: "Answering", url: "https://answering.test" },
|
||||
@@ -226,13 +214,9 @@ test("at a 30000ms interval, a check still waiting when its round is given up is
|
||||
roundChecks.abort();
|
||||
assert.notEqual(await settled(round), "still waiting");
|
||||
assert.equal(latencyFigure(state, answering), undefined);
|
||||
assert.deepEqual(answering.history, []);
|
||||
assert.equal(answering.status, "pending");
|
||||
assert.deepEqual(consoleLines("error"), []);
|
||||
assert.deepEqual(consoleLines("info"), []);
|
||||
});
|
||||
|
||||
test("at a 30000ms interval, a check still waiting when the user pauses is not recorded and does not show in its row", async (t) => {
|
||||
test("at a 30000ms interval, a check still waiting when the user pauses does not show in its row", async (t) => {
|
||||
CONFIG.updateInterval = 30000;
|
||||
const state = new AppState([
|
||||
{ name: "Answering", url: "https://answering.test" },
|
||||
@@ -247,13 +231,9 @@ test("at a 30000ms interval, a check still waiting when the user pauses is not r
|
||||
t.mock.timers.tick(500);
|
||||
assert.equal(await settled(round), "still waiting");
|
||||
assert.equal(latencyFigure(state, answering), undefined);
|
||||
assert.deepEqual(answering.history, []);
|
||||
assert.equal(answering.status, "pending");
|
||||
assert.deepEqual(consoleLines("error"), []);
|
||||
assert.deepEqual(consoleLines("info"), []);
|
||||
});
|
||||
|
||||
test("at a 30000ms interval, a check in the first round is not recorded and does not show in its row", async (t) => {
|
||||
test("at a 30000ms interval, a check in the first round does not show in its row", async (t) => {
|
||||
CONFIG.updateInterval = 30000;
|
||||
const state = new AppState([
|
||||
{ name: "Answering", url: "https://answering.test" },
|
||||
@@ -265,10 +245,6 @@ test("at a 30000ms interval, a check in the first round is not recorded and does
|
||||
t.mock.timers.tick(1000);
|
||||
assert.equal(await settled(round), "still waiting");
|
||||
assert.equal(latencyFigure(state, answering), undefined);
|
||||
assert.deepEqual(answering.history, []);
|
||||
assert.equal(answering.status, "pending");
|
||||
assert.deepEqual(consoleLines("error"), []);
|
||||
assert.deepEqual(consoleLines("info"), []);
|
||||
});
|
||||
|
||||
test("at a 30000ms interval, after the user pauses and resumes during a round, no row reads paused once its last check ends", async (t) => {
|
||||
@@ -318,6 +294,11 @@ function stateWithOneTarget() {
|
||||
return { state, target: state.local[0] };
|
||||
}
|
||||
|
||||
// What this test wrote to the browser console with console[method].
|
||||
function consoleLines(method) {
|
||||
return console[method].mock.calls.map((call) => call.arguments[0]);
|
||||
}
|
||||
|
||||
test("a check that fails with a network error writes one console line with the target, the time, the error and how long the request took", async (t) => {
|
||||
const { state, target } = stateWithOneTarget();
|
||||
mockTargets(t, () => 23, target.url);
|
||||
@@ -598,32 +579,6 @@ test("a check that timed out or found its target unreachable has a grey figure a
|
||||
assert.equal(latencyHex(null), "#6b7280");
|
||||
});
|
||||
|
||||
// buildUI draws the page into #app, here a plain object. Each entry in the
|
||||
// footer's color legend is a colored dot and the latencies it stands for,
|
||||
// with < and > written as < and >. A latency takes the color of the
|
||||
// first entry in CONFIG.latencyColors whose limit it is below, so the
|
||||
// legend has, for each of those entries in order, a dot of its color (a
|
||||
// bg- class where CONFIG.latencyColors has a text- one) and the latencies
|
||||
// below its limit, or, for the last, which has no limit, the latencies
|
||||
// from the limit before it up. A gray dot for offline comes after them.
|
||||
test("the footer's color legend has each entry in CONFIG.latencyColors, with its color and limit, then offline", (t) => {
|
||||
const app = {};
|
||||
t.mock.method(globalThis.document, "getElementById", () => app);
|
||||
buildUI(new AppState([]));
|
||||
|
||||
const legend = [
|
||||
...app.innerHTML.matchAll(/(bg-\w+-500) [^"]*"><\/span>(\S+)/g),
|
||||
].map(([, dot, latencies]) => [dot, latencies]);
|
||||
const colors = CONFIG.latencyColors;
|
||||
const expected = colors.map((color, i) => [
|
||||
color.className.replace("text-", "bg-"),
|
||||
color.below === Infinity
|
||||
? `>=${colors[i - 1].below}ms`
|
||||
: `<${color.below}ms`,
|
||||
]);
|
||||
assert.deepEqual(legend, [...expected, ["bg-gray-500", "offline"]]);
|
||||
});
|
||||
|
||||
// A target whose checks, in turn, answered after each of latencies ms, or,
|
||||
// for null, found it unreachable.
|
||||
function hostAfter(latencies) {
|
||||
@@ -746,56 +701,6 @@ test("once every WAN target's check has failed, the summary's Reachable slot rea
|
||||
assert.equal(elements["#summary-reachable"].className, "text-red-400");
|
||||
});
|
||||
|
||||
// In the next two tests each of the header's clocks is a plain object, made
|
||||
// the first time updateClocks looks it up and kept in elements under "#" and
|
||||
// its id.
|
||||
|
||||
// The mocked clock reads 1970-01-01T00:00:00.000Z.
|
||||
test("the header's clocks show the local time with its time zone's name, and the UTC time", (t) => {
|
||||
t.mock.method(
|
||||
globalThis.document,
|
||||
"getElementById",
|
||||
(id) => (elements[`#${id}`] ??= {}),
|
||||
);
|
||||
t.mock.timers.enable({ apis: ["Date"] });
|
||||
|
||||
updateClocks();
|
||||
|
||||
assert.match(
|
||||
elements["#clock-local"].textContent,
|
||||
/^Local: \d{4}-\d\d-\d\d \d\d:\d\d:\d\d \S/,
|
||||
);
|
||||
assert.equal(
|
||||
elements["#clock-utc"].textContent,
|
||||
"UTC: 1970-01-01 00:00:00 UTC",
|
||||
);
|
||||
});
|
||||
|
||||
// Here Intl.DateTimeFormat is as in Chrome before 95, Firefox before 91 or
|
||||
// Safari before 15.4: it throws a RangeError when asked for the time zone's
|
||||
// "shortGeneric" name.
|
||||
test("in a browser that cannot name the time zone, updating the clocks throws nothing and leaves them blank", (t) => {
|
||||
t.mock.method(
|
||||
globalThis.document,
|
||||
"getElementById",
|
||||
(id) => (elements[`#${id}`] ??= {}),
|
||||
);
|
||||
const DateTimeFormat = Intl.DateTimeFormat;
|
||||
t.mock.method(Intl, "DateTimeFormat", function (locales, options) {
|
||||
if (options?.timeZoneName === "shortGeneric") {
|
||||
throw new RangeError(
|
||||
"Value shortGeneric out of range for Intl.DateTimeFormat options property timeZoneName",
|
||||
);
|
||||
}
|
||||
return new DateTimeFormat(locales, options);
|
||||
});
|
||||
|
||||
updateClocks();
|
||||
|
||||
assert.equal(elements["#clock-local"].textContent, "");
|
||||
assert.equal(elements["#clock-utc"].textContent, "");
|
||||
});
|
||||
|
||||
// An app state in which, of the WAN targets, the first timedOut timed out,
|
||||
// the next unreachable were found unreachable, the next answered answered
|
||||
// after latency ms, and the rest have not been checked yet.
|
||||
|
||||
+20
-34
@@ -829,7 +829,7 @@ function wanRowsHTML(wanHosts) {
|
||||
return rows.join("");
|
||||
}
|
||||
|
||||
export function buildUI(state) {
|
||||
function buildUI(state) {
|
||||
const app = document.getElementById("app");
|
||||
app.innerHTML = `
|
||||
<div class="mx-auto px-[5%] py-8">
|
||||
@@ -911,7 +911,7 @@ export function buildUI(state) {
|
||||
<span class="inline-block w-3 h-3 rounded-full bg-lime-500 mr-1 ml-3 align-middle"></span><100ms
|
||||
<span class="inline-block w-3 h-3 rounded-full bg-yellow-500 mr-1 ml-3 align-middle"></span><200ms
|
||||
<span class="inline-block w-3 h-3 rounded-full bg-orange-500 mr-1 ml-3 align-middle"></span><500ms
|
||||
<span class="inline-block w-3 h-3 rounded-full bg-red-500 mr-1 ml-3 align-middle"></span>>=500ms
|
||||
<span class="inline-block w-3 h-3 rounded-full bg-red-500 mr-1 ml-3 align-middle"></span>>500ms
|
||||
<span class="inline-block w-3 h-3 rounded-full bg-gray-500 mr-1 ml-3 align-middle"></span>offline
|
||||
</p>
|
||||
<p class="mt-2"><a href="https://git.eeqj.de/sneak/netwatch/commit/${__COMMIT_FULL__}" target="_blank" rel="noopener" class="text-gray-600 hover:text-gray-400">${__COMMIT_HASH__}</a></p>
|
||||
@@ -1114,33 +1114,6 @@ function updateHealthBox(state) {
|
||||
else if (health === "healthy") log.notice(`Health: ${labels[health]}`);
|
||||
}
|
||||
|
||||
// Writes the local time, with its time zone's name, and the UTC time into
|
||||
// the header's clocks. The clocks only decorate the page: in a browser
|
||||
// whose Intl.DateTimeFormat rejects the "shortGeneric" time zone name
|
||||
// (Chrome before 95, Firefox before 91, Safari before 15.4) they are left
|
||||
// blank.
|
||||
export function updateClocks() {
|
||||
let local = "";
|
||||
let utc = "";
|
||||
try {
|
||||
const now = new Date();
|
||||
const zone = new Intl.DateTimeFormat("en-US", {
|
||||
timeZoneName: "shortGeneric",
|
||||
})
|
||||
.formatToParts(now)
|
||||
.find((p) => p.type === "timeZoneName").value;
|
||||
const pad = (n) => String(n).padStart(2, "0");
|
||||
local = `Local: ${now.toLocaleString("sv-SE", { hour12: false })} ${zone}`;
|
||||
utc =
|
||||
`UTC: ${now.getUTCFullYear()}-${pad(now.getUTCMonth() + 1)}-${pad(now.getUTCDate())} ` +
|
||||
`${pad(now.getUTCHours())}:${pad(now.getUTCMinutes())}:${pad(now.getUTCSeconds())} UTC`;
|
||||
} catch {
|
||||
// Both clocks are left blank.
|
||||
}
|
||||
document.getElementById("clock-local").textContent = local;
|
||||
document.getElementById("clock-utc").textContent = utc;
|
||||
}
|
||||
|
||||
// --- Debug Log Renderer ------------------------------------------------------
|
||||
|
||||
// One line of the debug log panel, its message set as text.
|
||||
@@ -1536,6 +1509,24 @@ async function init() {
|
||||
sortAndRebuildWAN(state);
|
||||
});
|
||||
|
||||
function updateClocks() {
|
||||
const now = new Date();
|
||||
const pad = (n) => String(n).padStart(2, "0");
|
||||
const utc =
|
||||
`${now.getUTCFullYear()}-${pad(now.getUTCMonth() + 1)}-${pad(now.getUTCDate())} ` +
|
||||
`${pad(now.getUTCHours())}:${pad(now.getUTCMinutes())}:${pad(now.getUTCSeconds())} UTC`;
|
||||
const local =
|
||||
now.toLocaleString("sv-SE", { hour12: false }) +
|
||||
" " +
|
||||
new Intl.DateTimeFormat("en-US", { timeZoneName: "shortGeneric" })
|
||||
.formatToParts(now)
|
||||
.find((p) => p.type === "timeZoneName").value;
|
||||
document.getElementById("clock-local").textContent = "Local: " + local;
|
||||
document.getElementById("clock-utc").textContent = "UTC: " + utc;
|
||||
}
|
||||
updateClocks();
|
||||
setInterval(updateClocks, 1000);
|
||||
|
||||
// Rounds never overlap: a round first gives up the last round's checks
|
||||
// if they are still waiting, and the last round then records nothing
|
||||
// more. At a steady interval they never are, as they time out at 80% of
|
||||
@@ -1615,11 +1606,6 @@ async function init() {
|
||||
|
||||
window.addEventListener("resize", () => handleResize(state));
|
||||
setTimeout(() => handleResize(state), CONFIG.resizeDelay);
|
||||
|
||||
// The clocks start last, after the rounds: they only decorate the page,
|
||||
// so nothing in them may stop the monitor from probing.
|
||||
updateClocks();
|
||||
setInterval(updateClocks, 1000);
|
||||
}
|
||||
|
||||
// Bootstrap only when loaded as the page: a real DOM containing the #app
|
||||
|
||||
Reference in New Issue
Block a user