Compare commits
2
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
cf30c7460c | ||
|
|
cf50d195ad |
No files matched your search
@@ -44,8 +44,7 @@ its abuse contact. RouteWatch keeps them in its database and fetches them again
|
|||||||
once they are 30 days old. An IP lookup answers from the stored details and
|
once they are 30 days old. An IP lookup answers from the stored details and
|
||||||
never waits for WHOIS.
|
never waits for WHOIS.
|
||||||
|
|
||||||
An IP lookup also says which public blocklists list the address (see
|
An IP lookup also says which public IP lists hold the address (see IP lists).
|
||||||
Blocklists).
|
|
||||||
|
|
||||||
## Getting started
|
## Getting started
|
||||||
|
|
||||||
@@ -65,7 +64,7 @@ docker run --memory=5g --memory-swap=5g -p 8080:8080 \
|
|||||||
```
|
```
|
||||||
|
|
||||||
The memory limit is explained in Memory. The volume holds the database and the
|
The memory limit is explained in Memory. The volume holds the database and the
|
||||||
copies of the blocklists.
|
copies of the IP lists.
|
||||||
|
|
||||||
To run the binary on the host instead, build it with `make build`. The build
|
To run the binary on the host instead, build it with `make build`. The build
|
||||||
needs Go and a C compiler, because the SQLite library is written in C:
|
needs Go and a C compiler, because the SQLite library is written in C:
|
||||||
@@ -284,8 +283,12 @@ address. Every entry has the same fields:
|
|||||||
- `ip`: the address
|
- `ip`: the address
|
||||||
- `ptr`: the address's PTR names, the host names that DNS gives for it, or an
|
- `ptr`: the address's PTR names, the host names that DNS gives for it, or an
|
||||||
empty list when it has none
|
empty list when it has none
|
||||||
- `blocklists`: the public blocklists the address is on (see Blocklists), or an
|
- `flags`: yes-or-no facts about the address:
|
||||||
empty list when it is on none
|
- `is_tor`: true when the address is on the `tor_exits` list
|
||||||
|
- `is_bogon`: true when the address is reserved, so that no network should
|
||||||
|
announce it on the internet
|
||||||
|
- `ip_lists`: the public IP lists the address is on (see IP lists), or an empty
|
||||||
|
list when it is on none
|
||||||
|
|
||||||
Every other field is about the prefix in the routing table that holds the
|
Every other field is about the prefix in the routing table that holds the
|
||||||
address. Each is null when no prefix in the routing table holds the address:
|
address. Each is null when no prefix in the routing table holds the address:
|
||||||
@@ -331,7 +334,7 @@ Each PTR lookup gets 2 seconds. An address whose lookup takes longer is answered
|
|||||||
without PTR names.
|
without PTR names.
|
||||||
|
|
||||||
A successful answer carries `Cache-Control` with a `max-age` of 15 minutes, the
|
A successful answer carries `Cache-Control` with a `max-age` of 15 minutes, the
|
||||||
shortest blocklist interval. The answer for the caller's own address (`/ip/`) is
|
shortest IP list interval. The answer for the caller's own address (`/ip/`) is
|
||||||
`private`, and the others are `public`.
|
`private`, and the others are `public`.
|
||||||
|
|
||||||
### Errors
|
### Errors
|
||||||
@@ -391,10 +394,10 @@ inline `<style>` and `<script>` blocks. Each answer's policy names a random
|
|||||||
value, a nonce, made for that answer. The browser applies or runs only the
|
value, a nonce, made for that answer. The browser applies or runs only the
|
||||||
blocks that carry it.
|
blocks that carry it.
|
||||||
|
|
||||||
## Blocklists
|
## IP lists
|
||||||
|
|
||||||
An IP lookup reports which public blocklists list each address. Each entry in
|
An IP lookup reports which public IP lists hold each address. Each entry in
|
||||||
`blocklists` has:
|
`ip_lists` has:
|
||||||
|
|
||||||
- `name`: the list's name, from the lists below
|
- `name`: the list's name, from the lists below
|
||||||
- `description`: what being on the list means
|
- `description`: what being on the list means
|
||||||
@@ -431,8 +434,8 @@ can be downloaded whole without an account or a key.
|
|||||||
How RouteWatch keeps the lists:
|
How RouteWatch keeps the lists:
|
||||||
|
|
||||||
- It holds each list in memory, so a lookup makes no outside request.
|
- It holds each list in memory, so a lookup makes no outside request.
|
||||||
- It keeps a copy of each list in the `blocklists` directory of the state
|
- It keeps a copy of each list in the `iplists` directory of the state directory
|
||||||
directory (see Configuration). Each copy is a file named after its list.
|
(see Configuration). Each copy is a file named after its list.
|
||||||
- Beside each copy, a file ending in `.validators` keeps the `Last-Modified` and
|
- Beside each copy, a file ending in `.validators` keeps the `Last-Modified` and
|
||||||
`ETag` headers the publisher sent with that copy.
|
`ETag` headers the publisher sent with that copy.
|
||||||
- At startup it loads each list from its copy. It downloads a list at once when
|
- At startup it loads each list from its copy. It downloads a list at once when
|
||||||
@@ -440,7 +443,7 @@ How RouteWatch keeps the lists:
|
|||||||
- A download sends the copy's `Last-Modified` as `If-Modified-Since` and its
|
- A download sends the copy's `Last-Modified` as `If-Modified-Since` and its
|
||||||
`ETag` as `If-None-Match`. The publisher then sends the list only when it has
|
`ETag` as `If-None-Match`. The publisher then sends the list only when it has
|
||||||
changed.
|
changed.
|
||||||
- A download fails when it takes over a minute or its file is over 16 MiB.
|
- A download fails when it takes over a minute or its file is over 200 MiB.
|
||||||
- When a download fails, or its file does not parse, RouteWatch keeps the
|
- When a download fails, or its file does not parse, RouteWatch keeps the
|
||||||
previous copy and logs the failure. It tries again one interval later.
|
previous copy and logs the failure. It tries again one interval later.
|
||||||
|
|
||||||
@@ -492,7 +495,7 @@ by the client. For a request from any other address, they use the address the
|
|||||||
request came from, whatever headers it carries. `X-Real-IP` and `True-Client-IP`
|
request came from, whatever headers it carries. `X-Real-IP` and `True-Client-IP`
|
||||||
are not read.
|
are not read.
|
||||||
|
|
||||||
The state directory holds the database, `db.sqlite`, and the blocklist copies:
|
The state directory holds the database, `db.sqlite`, and the IP list copies:
|
||||||
|
|
||||||
- macOS: `~/Library/Application Support/berlin.sneak.app.routewatch/`
|
- macOS: `~/Library/Application Support/berlin.sneak.app.routewatch/`
|
||||||
- Linux and the BSDs, as root: `/var/lib/berlin.sneak.app.routewatch/`
|
- Linux and the BSDs, as root: `/var/lib/berlin.sneak.app.routewatch/`
|
||||||
@@ -647,7 +650,7 @@ The parts:
|
|||||||
wait starts at 15 seconds. It halves after a success, down to 1 second, and
|
wait starts at 15 seconds. It halves after a success, down to 1 second, and
|
||||||
grows by half after a failure, up to a minute. An IP lookup that finds an AS
|
grows by half after a failure, up to a minute. An IP lookup that finds an AS
|
||||||
with old or missing details asks the fetcher for it at once, without the wait.
|
with old or missing details asks the fetcher for it at once, without the wait.
|
||||||
- The blocklists part downloads the blocklists and answers which lists hold an
|
- The IP lists part downloads the IP lists and answers which lists hold an
|
||||||
address.
|
address.
|
||||||
- The database maintainer checkpoints the database every 5 seconds: it folds the
|
- The database maintainer checkpoints the database every 5 seconds: it folds the
|
||||||
write-ahead log, where SQLite first writes changes, into the database file. It
|
write-ahead log, where SQLite first writes changes, into the database file. It
|
||||||
@@ -756,7 +759,7 @@ On `SIGTERM` or `SIGINT` the daemon stops its parts in this order:
|
|||||||
2. the handlers, each of which writes what it holds
|
2. the handlers, each of which writes what it holds
|
||||||
3. the WHOIS fetcher
|
3. the WHOIS fetcher
|
||||||
4. the database maintainer
|
4. the database maintainer
|
||||||
5. the blocklist downloads
|
5. the IP list downloads
|
||||||
6. the HTTP server
|
6. the HTTP server
|
||||||
|
|
||||||
Last, the daemon checkpoints and closes the database. It gives all this 8
|
Last, the daemon checkpoints and closes the database. It gives all this 8
|
||||||
@@ -790,10 +793,10 @@ routewatch/
|
|||||||
├── docs/
|
├── docs/
|
||||||
│ └── message-examples.json # Examples of RIS Live messages
|
│ └── message-examples.json # Examples of RIS Live messages
|
||||||
├── internal/
|
├── internal/
|
||||||
│ ├── blocklist/ # Public IP blocklists: downloads and lookups
|
|
||||||
│ ├── bogon/ # Reserved prefixes and AS numbers
|
│ ├── bogon/ # Reserved prefixes and AS numbers
|
||||||
│ ├── config/ # Settings from the environment
|
│ ├── config/ # Settings from the environment
|
||||||
│ ├── database/ # SQLite storage
|
│ ├── database/ # SQLite storage
|
||||||
|
│ ├── iplist/ # Public IP lists: downloads and lookups
|
||||||
│ ├── logger/ # Structured logging
|
│ ├── logger/ # Structured logging
|
||||||
│ ├── metrics/ # Feed and route update figures
|
│ ├── metrics/ # Feed and route update figures
|
||||||
│ ├── ristypes/ # The RIS Live message types
|
│ ├── ristypes/ # The RIS Live message types
|
||||||
|
|||||||
@@ -41,9 +41,6 @@ https://git.eeqj.de/sneak/routewatch/issues/225, also on `next`, takes the peers
|
|||||||
whose sessions go down together out of the table in one walk. A 24-hour run of
|
whose sessions go down together out of the table in one walk. A 24-hour run of
|
||||||
`9bd0e91` on the live feed is checking whether any handler still drops updates.
|
`9bd0e91` on the live feed is checking whether any handler still drops updates.
|
||||||
|
|
||||||
One question waits for sneak: which fields the IP lookup answer gains
|
|
||||||
(https://git.eeqj.de/sneak/routewatch/issues/203).
|
|
||||||
|
|
||||||
The order of the project's remaining work is kept on
|
The order of the project's remaining work is kept on
|
||||||
https://git.eeqj.de/sneak/routewatch/issues/86.
|
https://git.eeqj.de/sneak/routewatch/issues/86.
|
||||||
|
|
||||||
@@ -56,6 +53,14 @@ problem the run finds.
|
|||||||
|
|
||||||
# Completed Steps
|
# Completed Steps
|
||||||
|
|
||||||
|
- 2026-11-10: the tests no longer fail when a busy host stalls for several
|
||||||
|
seconds. Most such waits are now 15 seconds. The test of the stream's idle
|
||||||
|
timeout runs on a fake clock (closes #240)
|
||||||
|
- 2026-11-10: each result of the IP lookup has a `flags` object with `is_tor`
|
||||||
|
and `is_bogon`. The public IP lists are named `ip_lists` in the answer,
|
||||||
|
`internal/iplist` in the code and `iplists` in the state directory. A list
|
||||||
|
file may be up to 200 MiB (closes
|
||||||
|
https://git.eeqj.de/sneak/routewatch/issues/233).
|
||||||
- 2026-11-05: the prefix page lists the prefix's origin changes, newest first.
|
- 2026-11-05: the prefix page lists the prefix's origin changes, newest first.
|
||||||
The routing table records each change, and the prefix handler writes them to a
|
The routing table records each change, and the prefix handler writes them to a
|
||||||
new `origin_changes` table. The database maintainer prunes each prefix to its
|
new `origin_changes` table. The database maintainer prunes each prefix to its
|
||||||
|
|||||||
@@ -12,7 +12,7 @@ import (
|
|||||||
|
|
||||||
// parseTimeLimit is how long a test waits for parseCSV to return before it
|
// parseTimeLimit is how long a test waits for parseCSV to return before it
|
||||||
// fails.
|
// fails.
|
||||||
const parseTimeLimit = 2 * time.Second
|
const parseTimeLimit = 15 * time.Second
|
||||||
|
|
||||||
// TestParseCSVReadsTodaysLayout parses an as.csv with the country code column
|
// TestParseCSVReadsTodaysLayout parses an as.csv with the country code column
|
||||||
// that the source added after the description. It expects each AS with its
|
// that the source added after the description. It expects each AS with its
|
||||||
|
|||||||
@@ -72,17 +72,34 @@ const (
|
|||||||
// lockTestPrefix is the prefix the batch writes in the checkpoint tests
|
// lockTestPrefix is the prefix the batch writes in the checkpoint tests
|
||||||
// store.
|
// store.
|
||||||
lockTestPrefix = "192.0.2.0/24"
|
lockTestPrefix = "192.0.2.0/24"
|
||||||
|
// longBusyTimeoutMs is the busy timeout setLongBusyTimeout gives the
|
||||||
|
// writing connection. A checkpoint that waits for an open read takes this
|
||||||
|
// long.
|
||||||
|
longBusyTimeoutMs = 30000
|
||||||
// readerCheckpointLimit is how long a checkpoint, and a batch write
|
// readerCheckpointLimit is how long a checkpoint, and a batch write
|
||||||
// waiting for it, may take while a reader is open: well under the busy
|
// waiting for it, may take while a reader is open: well under
|
||||||
// timeout, which a checkpoint that waits for the reader takes.
|
// longBusyTimeoutMs, with room for a stall on a busy host.
|
||||||
readerCheckpointLimit = time.Second
|
readerCheckpointLimit = 15 * time.Second
|
||||||
// truncateCheckpointLimit is how long a checkpoint of a WAL file over
|
// truncateCheckpointLimit is how long a checkpoint of a WAL file over
|
||||||
// sqliteWALSizeLimitBytes may take to give up while a reader is open: its
|
// sqliteWALSizeLimitBytes may take to give up while a reader is open: its
|
||||||
// wait for the reader plus a second, well under the busy timeout.
|
// wait for the reader plus readerCheckpointLimit.
|
||||||
truncateCheckpointLimit = truncateBusyTimeoutMs*time.Millisecond +
|
truncateCheckpointLimit = truncateBusyTimeoutMs*time.Millisecond +
|
||||||
time.Second
|
readerCheckpointLimit
|
||||||
)
|
)
|
||||||
|
|
||||||
|
// setLongBusyTimeout gives the writing connection of db longBusyTimeoutMs, so
|
||||||
|
// that a checkpoint that waits for an open read takes far longer than a stall
|
||||||
|
// on a busy host.
|
||||||
|
func setLongBusyTimeout(ctx context.Context, t *testing.T, db *Database) {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
_, err := db.writer.ExecContext(ctx,
|
||||||
|
fmt.Sprintf("PRAGMA busy_timeout=%d", longBusyTimeoutMs))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("failed to set the busy timeout: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// TestConnectionPoolPragmas holds several pooled connections open at once and
|
// TestConnectionPoolPragmas holds several pooled connections open at once and
|
||||||
// checks each one carries the per-connection settings from the DSN, plus the
|
// checks each one carries the per-connection settings from the DSN, plus the
|
||||||
// process-wide heap limits, and sees the new file with auto_vacuum
|
// process-wide heap limits, and sees the new file with auto_vacuum
|
||||||
@@ -579,6 +596,8 @@ func TestCheckpointDoesNotWaitForReader(t *testing.T) {
|
|||||||
ctx := context.Background()
|
ctx := context.Background()
|
||||||
ts := time.Now().UTC()
|
ts := time.Now().UTC()
|
||||||
|
|
||||||
|
setLongBusyTimeout(ctx, t, db)
|
||||||
|
|
||||||
err = db.UpdatePrefixesBatch(map[string]time.Time{"198.51.100.0/24": ts})
|
err = db.UpdatePrefixesBatch(map[string]time.Time{"198.51.100.0/24": ts})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("failed to store the prefix the reader reads: %v", err)
|
t.Fatalf("failed to store the prefix the reader reads: %v", err)
|
||||||
@@ -690,6 +709,7 @@ func TestCheckpointEmptiesWALOnceNoReadIsOpen(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
fillWALPastLimit(ctx, t, db)
|
fillWALPastLimit(ctx, t, db)
|
||||||
|
setLongBusyTimeout(ctx, t, db)
|
||||||
|
|
||||||
started := time.Now()
|
started := time.Now()
|
||||||
|
|
||||||
|
|||||||
@@ -1,4 +1,4 @@
|
|||||||
package blocklist
|
package iplist
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
@@ -22,7 +22,7 @@ const (
|
|||||||
|
|
||||||
// maxDownloadSize is the size of the largest list file accepted, in
|
// maxDownloadSize is the size of the largest list file accepted, in
|
||||||
// bytes. The largest list, FireHOL abusers, is about 2.5 MiB.
|
// bytes. The largest list, FireHOL abusers, is about 2.5 MiB.
|
||||||
maxDownloadSize = 16 << 20
|
maxDownloadSize = 200 << 20
|
||||||
)
|
)
|
||||||
|
|
||||||
// The request headers a download sets.
|
// The request headers a download sets.
|
||||||
@@ -78,10 +78,10 @@ var (
|
|||||||
// downloads each list once its copy is older than its Interval, and a list
|
// downloads each list once its copy is older than its Interval, and a list
|
||||||
// with no copy at once. The goroutine runs until Stop: the end of ctx does
|
// with no copy at once. The goroutine runs until Stop: the end of ctx does
|
||||||
// not stop it.
|
// not stop it.
|
||||||
func (b *Blocklists) Start(ctx context.Context) {
|
func (b *IPLists) Start(ctx context.Context) {
|
||||||
err := os.MkdirAll(b.dir, dirPermissions)
|
err := os.MkdirAll(b.dir, dirPermissions)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
b.logger.Error("Failed to create the blocklists directory",
|
b.logger.Error("Failed to create the IP lists directory",
|
||||||
"dir", b.dir, "error", err)
|
"dir", b.dir, "error", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -90,7 +90,7 @@ func (b *Blocklists) Start(ctx context.Context) {
|
|||||||
for i, list := range b.lists {
|
for i, list := range b.lists {
|
||||||
loadErr := b.loadCopy(list)
|
loadErr := b.loadCopy(list)
|
||||||
if loadErr != nil && !errors.Is(loadErr, fs.ErrNotExist) {
|
if loadErr != nil && !errors.Is(loadErr, fs.ErrNotExist) {
|
||||||
b.logger.Error("Failed to load a blocklist copy",
|
b.logger.Error("Failed to load an IP list copy",
|
||||||
"list", list.Name, "error", loadErr)
|
"list", list.Name, "error", loadErr)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -104,20 +104,20 @@ func (b *Blocklists) Start(ctx context.Context) {
|
|||||||
b.run(ctx, nextDownload)
|
b.run(ctx, nextDownload)
|
||||||
}()
|
}()
|
||||||
|
|
||||||
b.logger.Info("Blocklist downloads started", "dir", b.dir)
|
b.logger.Info("IP list downloads started", "dir", b.dir)
|
||||||
}
|
}
|
||||||
|
|
||||||
// Stop stops the downloads, cancelling one in progress, and waits until the
|
// Stop stops the downloads, cancelling one in progress, and waits until the
|
||||||
// goroutine has returned or ctx ends. The copies in memory stay, so Lookup
|
// goroutine has returned or ctx ends. The copies in memory stay, so Lookup
|
||||||
// still answers.
|
// still answers.
|
||||||
func (b *Blocklists) Stop(ctx context.Context) {
|
func (b *IPLists) Stop(ctx context.Context) {
|
||||||
b.stop()
|
b.stop()
|
||||||
|
|
||||||
select {
|
select {
|
||||||
case <-b.done:
|
case <-b.done:
|
||||||
b.logger.Info("Blocklist downloads stopped")
|
b.logger.Info("IP list downloads stopped")
|
||||||
case <-ctx.Done():
|
case <-ctx.Done():
|
||||||
b.logger.Warn("Stop deadline passed before the blocklist downloads " +
|
b.logger.Warn("Stop deadline passed before the IP list downloads " +
|
||||||
"stopped")
|
"stopped")
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -125,7 +125,7 @@ func (b *Blocklists) Stop(ctx context.Context) {
|
|||||||
// run downloads each list when nextDownload says, until ctx ends. After each
|
// run downloads each list when nextDownload says, until ctx ends. After each
|
||||||
// try, whether it worked or not, the list's next download is an Interval
|
// try, whether it worked or not, the list's next download is an Interval
|
||||||
// later.
|
// later.
|
||||||
func (b *Blocklists) run(ctx context.Context, nextDownload []time.Time) {
|
func (b *IPLists) run(ctx context.Context, nextDownload []time.Time) {
|
||||||
defer close(b.done)
|
defer close(b.done)
|
||||||
|
|
||||||
if len(nextDownload) == 0 {
|
if len(nextDownload) == 0 {
|
||||||
@@ -155,7 +155,7 @@ func (b *Blocklists) run(ctx context.Context, nextDownload []time.Time) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
if err != nil {
|
if err != nil {
|
||||||
b.logger.Error("Blocklist download failed; keeping the "+
|
b.logger.Error("IP list download failed; keeping the "+
|
||||||
"previous copy", "list", list.Name, "error", err)
|
"previous copy", "list", list.Name, "error", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -169,7 +169,7 @@ func (b *Blocklists) run(ctx context.Context, nextDownload []time.Time) {
|
|||||||
// on disk and in memory. An answer that the copy is still current marks the
|
// on disk and in memory. An answer that the copy is still current marks the
|
||||||
// copy as updated now. Anything else leaves the copy as it is and returns an
|
// copy as updated now. Anything else leaves the copy as it is and returns an
|
||||||
// error.
|
// error.
|
||||||
func (b *Blocklists) download(ctx context.Context, list List) error {
|
func (b *IPLists) download(ctx context.Context, list List) error {
|
||||||
ctx, cancel := context.WithTimeout(ctx, downloadTimeout)
|
ctx, cancel := context.WithTimeout(ctx, downloadTimeout)
|
||||||
defer cancel()
|
defer cancel()
|
||||||
|
|
||||||
@@ -212,7 +212,7 @@ func (b *Blocklists) download(ctx context.Context, list List) error {
|
|||||||
// replaceCopy writes a new version of list, read from body, beside the list's
|
// replaceCopy writes a new version of list, read from body, beside the list's
|
||||||
// copy. Only when it parses does it replace the copy, on disk and in memory.
|
// copy. Only when it parses does it replace the copy, on disk and in memory.
|
||||||
// sent are the validators the publisher sent with the new version.
|
// sent are the validators the publisher sent with the new version.
|
||||||
func (b *Blocklists) replaceCopy(list List, body io.Reader, sent validators) error {
|
func (b *IPLists) replaceCopy(list List, body io.Reader, sent validators) error {
|
||||||
root, err := os.OpenRoot(b.dir)
|
root, err := os.OpenRoot(b.dir)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return err
|
return err
|
||||||
@@ -244,13 +244,13 @@ func (b *Blocklists) replaceCopy(list List, body io.Reader, sent validators) err
|
|||||||
b.setCopy(list.Name, listCopy{
|
b.setCopy(list.Name, listCopy{
|
||||||
addresses: addresses, updated: time.Now().UTC(), validators: sent,
|
addresses: addresses, updated: time.Now().UTC(), validators: sent,
|
||||||
})
|
})
|
||||||
b.logger.Info("Downloaded a new version of a blocklist", "list", list.Name)
|
b.logger.Info("Downloaded a new version of an IP list", "list", list.Name)
|
||||||
|
|
||||||
// Without this file, the first download after a restart asks for the
|
// Without this file, the first download after a restart asks for the
|
||||||
// whole list.
|
// whole list.
|
||||||
err = writeValidators(root, list.Name, sent)
|
err = writeValidators(root, list.Name, sent)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
b.logger.Error("Failed to keep the validators of a blocklist copy",
|
b.logger.Error("Failed to keep the validators of an IP list copy",
|
||||||
"list", list.Name, "error", err)
|
"list", list.Name, "error", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -288,7 +288,7 @@ func writeAndParse(
|
|||||||
|
|
||||||
// markCurrent records that the publisher says the list's copy is still
|
// markCurrent records that the publisher says the list's copy is still
|
||||||
// current: the copy, on disk and in memory, is marked as updated now.
|
// current: the copy, on disk and in memory, is marked as updated now.
|
||||||
func (b *Blocklists) markCurrent(list List) error {
|
func (b *IPLists) markCurrent(list List) error {
|
||||||
root, err := os.OpenRoot(b.dir)
|
root, err := os.OpenRoot(b.dir)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return err
|
return err
|
||||||
@@ -304,7 +304,7 @@ func (b *Blocklists) markCurrent(list List) error {
|
|||||||
}
|
}
|
||||||
|
|
||||||
b.setUpdated(list.Name, now)
|
b.setUpdated(list.Name, now)
|
||||||
b.logger.Debug("Blocklist copy is still current", "list", list.Name)
|
b.logger.Debug("IP list copy is still current", "list", list.Name)
|
||||||
|
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
@@ -312,7 +312,7 @@ func (b *Blocklists) markCurrent(list List) error {
|
|||||||
// loadCopy makes the list's copy on disk, with the validators kept beside it,
|
// loadCopy makes the list's copy on disk, with the validators kept beside it,
|
||||||
// its copy in memory. The file's modification time is when the copy was
|
// its copy in memory. The file's modification time is when the copy was
|
||||||
// updated.
|
// updated.
|
||||||
func (b *Blocklists) loadCopy(list List) error {
|
func (b *IPLists) loadCopy(list List) error {
|
||||||
root, err := os.OpenRoot(b.dir)
|
root, err := os.OpenRoot(b.dir)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return err
|
return err
|
||||||
@@ -1,7 +1,7 @@
|
|||||||
// Package blocklist downloads public IP blocklists on a schedule, keeps the
|
// Package iplist downloads public IP lists on a schedule, keeps the last good
|
||||||
// last good copy of each on disk and in memory, and answers which lists
|
// copy of each on disk and in memory, and answers which lists contain an
|
||||||
// contain an address.
|
// address.
|
||||||
package blocklist
|
package iplist
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
@@ -31,7 +31,7 @@ const (
|
|||||||
FormatDShield
|
FormatDShield
|
||||||
)
|
)
|
||||||
|
|
||||||
// List is one public blocklist.
|
// List is one public IP list.
|
||||||
type List struct {
|
type List struct {
|
||||||
// Name names the list in lookup answers, and names its copy on disk.
|
// Name names the list in lookup answers, and names its copy on disk.
|
||||||
Name string
|
Name string
|
||||||
@@ -80,9 +80,9 @@ type Params struct {
|
|||||||
Logger *slog.Logger
|
Logger *slog.Logger
|
||||||
}
|
}
|
||||||
|
|
||||||
// Blocklists keeps a copy of each list and answers which lists contain an
|
// IPLists keeps a copy of each list and answers which lists contain an
|
||||||
// address. Start loads the copies from disk and starts downloading the lists.
|
// address. Start loads the copies from disk and starts downloading the lists.
|
||||||
type Blocklists struct {
|
type IPLists struct {
|
||||||
dir string
|
dir string
|
||||||
lists []List
|
lists []List
|
||||||
logger *slog.Logger
|
logger *slog.Logger
|
||||||
@@ -97,12 +97,12 @@ type Blocklists struct {
|
|||||||
done chan struct{}
|
done chan struct{}
|
||||||
}
|
}
|
||||||
|
|
||||||
// New returns a Blocklists with no copies. Start loads them.
|
// New returns an IPLists with no copies. Start loads them.
|
||||||
func New(params Params) *Blocklists {
|
func New(params Params) *IPLists {
|
||||||
return &Blocklists{
|
return &IPLists{
|
||||||
dir: params.Dir,
|
dir: params.Dir,
|
||||||
lists: params.Lists,
|
lists: params.Lists,
|
||||||
logger: params.Logger.With("component", "blocklists"),
|
logger: params.Logger.With("component", "iplists"),
|
||||||
copies: make(map[string]listCopy),
|
copies: make(map[string]listCopy),
|
||||||
done: make(chan struct{}),
|
done: make(chan struct{}),
|
||||||
}
|
}
|
||||||
@@ -111,7 +111,7 @@ func New(params Params) *Blocklists {
|
|||||||
// Lookup returns a Listing for each list whose copy contains addr, in the
|
// Lookup returns a Listing for each list whose copy contains addr, in the
|
||||||
// order of the lists given to New. It makes no outside request. When no list
|
// order of the lists given to New. It makes no outside request. When no list
|
||||||
// contains addr, it returns an empty list, not nil.
|
// contains addr, it returns an empty list, not nil.
|
||||||
func (b *Blocklists) Lookup(addr netip.Addr) []Listing {
|
func (b *IPLists) Lookup(addr netip.Addr) []Listing {
|
||||||
b.mu.RLock()
|
b.mu.RLock()
|
||||||
defer b.mu.RUnlock()
|
defer b.mu.RUnlock()
|
||||||
|
|
||||||
@@ -133,7 +133,7 @@ func (b *Blocklists) Lookup(addr netip.Addr) []Listing {
|
|||||||
|
|
||||||
// ShortestInterval returns the shortest Interval of the lists. That is the
|
// ShortestInterval returns the shortest Interval of the lists. That is the
|
||||||
// soonest a copy can change, and so how long a lookup answer stays current.
|
// soonest a copy can change, and so how long a lookup answer stays current.
|
||||||
func (b *Blocklists) ShortestInterval() time.Duration {
|
func (b *IPLists) ShortestInterval() time.Duration {
|
||||||
var shortest time.Duration
|
var shortest time.Duration
|
||||||
|
|
||||||
for i, list := range b.lists {
|
for i, list := range b.lists {
|
||||||
@@ -147,7 +147,7 @@ func (b *Blocklists) ShortestInterval() time.Duration {
|
|||||||
|
|
||||||
// updated returns when the named list's copy was last downloaded or found
|
// updated returns when the named list's copy was last downloaded or found
|
||||||
// current, or the zero time when the list has no copy.
|
// current, or the zero time when the list has no copy.
|
||||||
func (b *Blocklists) updated(name string) time.Time {
|
func (b *IPLists) updated(name string) time.Time {
|
||||||
b.mu.RLock()
|
b.mu.RLock()
|
||||||
defer b.mu.RUnlock()
|
defer b.mu.RUnlock()
|
||||||
|
|
||||||
@@ -156,7 +156,7 @@ func (b *Blocklists) updated(name string) time.Time {
|
|||||||
|
|
||||||
// copyValidators returns the validators of the named list's copy. They are
|
// copyValidators returns the validators of the named list's copy. They are
|
||||||
// empty when the list has no copy.
|
// empty when the list has no copy.
|
||||||
func (b *Blocklists) copyValidators(name string) validators {
|
func (b *IPLists) copyValidators(name string) validators {
|
||||||
b.mu.RLock()
|
b.mu.RLock()
|
||||||
defer b.mu.RUnlock()
|
defer b.mu.RUnlock()
|
||||||
|
|
||||||
@@ -164,7 +164,7 @@ func (b *Blocklists) copyValidators(name string) validators {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// setCopy makes c the named list's copy.
|
// setCopy makes c the named list's copy.
|
||||||
func (b *Blocklists) setCopy(name string, c listCopy) {
|
func (b *IPLists) setCopy(name string, c listCopy) {
|
||||||
b.mu.Lock()
|
b.mu.Lock()
|
||||||
defer b.mu.Unlock()
|
defer b.mu.Unlock()
|
||||||
|
|
||||||
@@ -172,7 +172,7 @@ func (b *Blocklists) setCopy(name string, c listCopy) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// setUpdated records that the named list's copy was found current at t.
|
// setUpdated records that the named list's copy was found current at t.
|
||||||
func (b *Blocklists) setUpdated(name string, t time.Time) {
|
func (b *IPLists) setUpdated(name string, t time.Time) {
|
||||||
b.mu.Lock()
|
b.mu.Lock()
|
||||||
defer b.mu.Unlock()
|
defer b.mu.Unlock()
|
||||||
|
|
||||||
+112
-29
@@ -1,4 +1,4 @@
|
|||||||
package blocklist
|
package iplist
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
@@ -121,7 +121,7 @@ func testList(url string, format Format) List {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func newTestBlocklists(dir string, list List) *Blocklists {
|
func newTestIPLists(dir string, list List) *IPLists {
|
||||||
return New(Params{
|
return New(Params{
|
||||||
Dir: dir,
|
Dir: dir,
|
||||||
Lists: []List{list},
|
Lists: []List{list},
|
||||||
@@ -130,7 +130,7 @@ func newTestBlocklists(dir string, list List) *Blocklists {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// lookup looks addr up in b.
|
// lookup looks addr up in b.
|
||||||
func lookup(t *testing.T, b *Blocklists, addr string) []Listing {
|
func lookup(t *testing.T, b *IPLists, addr string) []Listing {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
return b.Lookup(netip.MustParseAddr(addr))
|
return b.Lookup(netip.MustParseAddr(addr))
|
||||||
@@ -188,7 +188,7 @@ func TestLookupFindsListedAddresses(t *testing.T) {
|
|||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
_, url := newPublisher(t, tt.body)
|
_, url := newPublisher(t, tt.body)
|
||||||
b := newTestBlocklists(t.TempDir(), testList(url, tt.format))
|
b := newTestIPLists(t.TempDir(), testList(url, tt.format))
|
||||||
|
|
||||||
err := b.download(t.Context(), b.lists[0])
|
err := b.download(t.Context(), b.lists[0])
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -228,10 +228,6 @@ func TestFailedDownloadKeepsThePreviousCopy(t *testing.T) {
|
|||||||
}{
|
}{
|
||||||
{"server error", http.StatusInternalServerError, "unavailable"},
|
{"server error", http.StatusInternalServerError, "unavailable"},
|
||||||
{"not a list", http.StatusOK, "<!DOCTYPE html>\n<html></html>\n"},
|
{"not a list", http.StatusOK, "<!DOCTYPE html>\n<html></html>\n"},
|
||||||
{
|
|
||||||
"over the size limit", http.StatusOK,
|
|
||||||
strings.Repeat(listedAddr+"\n", maxDownloadSize/len(listedAddr)),
|
|
||||||
},
|
|
||||||
}
|
}
|
||||||
|
|
||||||
for _, tt := range tests {
|
for _, tt := range tests {
|
||||||
@@ -240,7 +236,7 @@ func TestFailedDownloadKeepsThePreviousCopy(t *testing.T) {
|
|||||||
|
|
||||||
dir := t.TempDir()
|
dir := t.TempDir()
|
||||||
p, url := newPublisher(t, goodListBody)
|
p, url := newPublisher(t, goodListBody)
|
||||||
b := newTestBlocklists(dir, testList(url, FormatLines))
|
b := newTestIPLists(dir, testList(url, FormatLines))
|
||||||
|
|
||||||
err := b.download(t.Context(), b.lists[0])
|
err := b.download(t.Context(), b.lists[0])
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -262,23 +258,106 @@ func TestFailedDownloadKeepsThePreviousCopy(t *testing.T) {
|
|||||||
t.Errorf("listings %+v after the failure, want %+v", after, before)
|
t.Errorf("listings %+v after the failure, want %+v", after, before)
|
||||||
}
|
}
|
||||||
|
|
||||||
copies := os.DirFS(dir)
|
checkGoodCopyOnDisk(t, dir)
|
||||||
|
|
||||||
onDisk, err := fs.ReadFile(copies, testListName)
|
|
||||||
if err != nil || string(onDisk) != goodListBody {
|
|
||||||
t.Errorf("copy on disk %q (%v), want %q", onDisk, err, goodListBody)
|
|
||||||
}
|
|
||||||
|
|
||||||
_, err = fs.Stat(copies, testListName+downloadSuffix)
|
|
||||||
if !errors.Is(err, fs.ErrNotExist) {
|
|
||||||
t.Errorf("the failed download's file is still there (%v)", err)
|
|
||||||
}
|
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// checkGoodCopyOnDisk checks that dir holds the good version of the test list
|
||||||
|
// and no part of a failed download.
|
||||||
|
func checkGoodCopyOnDisk(t *testing.T, dir string) {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
copies := os.DirFS(dir)
|
||||||
|
|
||||||
|
onDisk, err := fs.ReadFile(copies, testListName)
|
||||||
|
if err != nil || string(onDisk) != goodListBody {
|
||||||
|
t.Errorf("copy on disk %q (%v), want %q", onDisk, err, goodListBody)
|
||||||
|
}
|
||||||
|
|
||||||
|
_, err = fs.Stat(copies, testListName+downloadSuffix)
|
||||||
|
if !errors.Is(err, fs.ErrNotExist) {
|
||||||
|
t.Errorf("the failed download's file is still there (%v)", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// commentLines is an endless io.Reader of comment lines. It lets a test send
|
||||||
|
// a large list file without holding the file in memory.
|
||||||
|
type commentLines struct {
|
||||||
|
offset int
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *commentLines) Read(p []byte) (int, error) {
|
||||||
|
const line = "# a comment line that pads a large list file\n"
|
||||||
|
|
||||||
|
for filled := 0; filled < len(p); {
|
||||||
|
copied := copy(p[filled:], line[c.offset:])
|
||||||
|
filled += copied
|
||||||
|
c.offset = (c.offset + copied) % len(line)
|
||||||
|
}
|
||||||
|
|
||||||
|
return len(p), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// largeListFile returns a reader of a list file of size bytes: comment lines,
|
||||||
|
// then newListedAddr on the last line.
|
||||||
|
func largeListFile(size int64) io.Reader {
|
||||||
|
lastLine := "\n" + newListedAddr + "\n"
|
||||||
|
|
||||||
|
return io.MultiReader(
|
||||||
|
io.LimitReader(&commentLines{}, size-int64(len(lastLine))),
|
||||||
|
strings.NewReader(lastLine),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestLargeFileUnderTheSizeLimitIsAccepted replaces a list with a 199 MiB
|
||||||
|
// file. The address on its last line is listed, so the whole file was read.
|
||||||
|
func TestLargeFileUnderTheSizeLimitIsAccepted(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
const size = 199 << 20
|
||||||
|
|
||||||
|
b := newTestIPLists(t.TempDir(), testList("", FormatLines))
|
||||||
|
|
||||||
|
err := b.replaceCopy(b.lists[0], largeListFile(size), validators{})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("replaceCopy: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if listings := lookup(t, b, newListedAddr); len(listings) != 1 {
|
||||||
|
t.Errorf("listings %+v, want one", listings)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestFileOverTheSizeLimitKeepsThePreviousCopy downloads a good version of a
|
||||||
|
// list, then replaces it with a file one byte over the limit. The attempt
|
||||||
|
// fails, and the good copy stays in memory and on disk.
|
||||||
|
func TestFileOverTheSizeLimitKeepsThePreviousCopy(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
dir := t.TempDir()
|
||||||
|
_, url := newPublisher(t, goodListBody)
|
||||||
|
b := newTestIPLists(dir, testList(url, FormatLines))
|
||||||
|
|
||||||
|
err := b.download(t.Context(), b.lists[0])
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("download: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
err = b.replaceCopy(b.lists[0], largeListFile(maxDownloadSize+1), validators{})
|
||||||
|
if !errors.Is(err, errTooLarge) {
|
||||||
|
t.Fatalf("replaceCopy: %v, want %v", err, errTooLarge)
|
||||||
|
}
|
||||||
|
|
||||||
|
if listings := lookup(t, b, listedAddr); len(listings) != 1 {
|
||||||
|
t.Errorf("listings of %s %+v, want one", listedAddr, listings)
|
||||||
|
}
|
||||||
|
|
||||||
|
checkGoodCopyOnDisk(t, dir)
|
||||||
|
}
|
||||||
|
|
||||||
// TestRestartLoadsCopiesFromDisk downloads a list, then starts a new
|
// TestRestartLoadsCopiesFromDisk downloads a list, then starts a new
|
||||||
// Blocklists on the same directory, as a restart does. The new one answers
|
// IPLists on the same directory, as a restart does. The new one answers
|
||||||
// from the copy on disk at once, and does not download the list again while
|
// from the copy on disk at once, and does not download the list again while
|
||||||
// the copy is younger than the list's interval.
|
// the copy is younger than the list's interval.
|
||||||
func TestRestartLoadsCopiesFromDisk(t *testing.T) {
|
func TestRestartLoadsCopiesFromDisk(t *testing.T) {
|
||||||
@@ -288,7 +367,7 @@ func TestRestartLoadsCopiesFromDisk(t *testing.T) {
|
|||||||
p, url := newPublisher(t, goodListBody)
|
p, url := newPublisher(t, goodListBody)
|
||||||
list := testList(url, FormatLines)
|
list := testList(url, FormatLines)
|
||||||
|
|
||||||
first := newTestBlocklists(dir, list)
|
first := newTestIPLists(dir, list)
|
||||||
|
|
||||||
err := first.download(t.Context(), list)
|
err := first.download(t.Context(), list)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -297,13 +376,17 @@ func TestRestartLoadsCopiesFromDisk(t *testing.T) {
|
|||||||
|
|
||||||
downloaded := lookup(t, first, listedAddr)[0].Updated
|
downloaded := lookup(t, first, listedAddr)[0].Updated
|
||||||
|
|
||||||
second := newTestBlocklists(dir, list)
|
second := newTestIPLists(dir, list)
|
||||||
second.Start(t.Context())
|
second.Start(t.Context())
|
||||||
second.Stop(t.Context())
|
second.Stop(t.Context())
|
||||||
|
|
||||||
|
// The download takes its time after writing the file, and the restart
|
||||||
|
// reads the file's time. A busy host can put seconds between the two.
|
||||||
|
const maxUpdatedDifference = 15 * time.Second
|
||||||
|
|
||||||
listings := lookup(t, second, listedAddr)
|
listings := lookup(t, second, listedAddr)
|
||||||
if len(listings) != 1 ||
|
if len(listings) != 1 ||
|
||||||
listings[0].Updated.Sub(downloaded).Abs() > time.Second {
|
listings[0].Updated.Sub(downloaded).Abs() > maxUpdatedDifference {
|
||||||
t.Errorf("after the restart, listings %+v, want one updated at %s",
|
t.Errorf("after the restart, listings %+v, want one updated at %s",
|
||||||
listings, downloaded)
|
listings, downloaded)
|
||||||
}
|
}
|
||||||
@@ -328,7 +411,7 @@ func TestStartRefreshesAnOldCopy(t *testing.T) {
|
|||||||
lastModified := time.Now().Add(-3 * list.Interval).UTC().Truncate(time.Second)
|
lastModified := time.Now().Add(-3 * list.Interval).UTC().Truncate(time.Second)
|
||||||
p.publish(goodListBody, lastModified, testETag)
|
p.publish(goodListBody, lastModified, testETag)
|
||||||
|
|
||||||
err := newTestBlocklists(dir, list).download(t.Context(), list)
|
err := newTestIPLists(dir, list).download(t.Context(), list)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("download: %v", err)
|
t.Fatalf("download: %v", err)
|
||||||
}
|
}
|
||||||
@@ -340,7 +423,7 @@ func TestStartRefreshesAnOldCopy(t *testing.T) {
|
|||||||
t.Fatalf("Chtimes: %v", err)
|
t.Fatalf("Chtimes: %v", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
b := newTestBlocklists(dir, list)
|
b := newTestIPLists(dir, list)
|
||||||
b.Start(t.Context())
|
b.Start(t.Context())
|
||||||
|
|
||||||
deadline := time.Now().Add(maxWait)
|
deadline := time.Now().Add(maxWait)
|
||||||
@@ -381,7 +464,7 @@ func TestNewVersionOlderThanTheLastDownloadReplacesTheCopy(t *testing.T) {
|
|||||||
|
|
||||||
p, url := newPublisher(t, goodListBody)
|
p, url := newPublisher(t, goodListBody)
|
||||||
list := testList(url, FormatLines)
|
list := testList(url, FormatLines)
|
||||||
b := newTestBlocklists(t.TempDir(), list)
|
b := newTestIPLists(t.TempDir(), list)
|
||||||
|
|
||||||
firstVersion := time.Now().Add(-2 * list.Interval).UTC().Truncate(time.Second)
|
firstVersion := time.Now().Add(-2 * list.Interval).UTC().Truncate(time.Second)
|
||||||
p.publish(goodListBody, firstVersion, "")
|
p.publish(goodListBody, firstVersion, "")
|
||||||
@@ -418,7 +501,7 @@ func TestDownloadWithoutLastModifiedOrETagSendsNoCondition(t *testing.T) {
|
|||||||
|
|
||||||
p, url := newPublisher(t, goodListBody)
|
p, url := newPublisher(t, goodListBody)
|
||||||
list := testList(url, FormatLines)
|
list := testList(url, FormatLines)
|
||||||
b := newTestBlocklists(t.TempDir(), list)
|
b := newTestIPLists(t.TempDir(), list)
|
||||||
|
|
||||||
err := b.download(t.Context(), list)
|
err := b.download(t.Context(), list)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -455,7 +538,7 @@ func TestStopCancelsADownloadInProgress(t *testing.T) {
|
|||||||
}))
|
}))
|
||||||
t.Cleanup(srv.Close)
|
t.Cleanup(srv.Close)
|
||||||
|
|
||||||
b := newTestBlocklists(t.TempDir(), testList(srv.URL, FormatLines))
|
b := newTestIPLists(t.TempDir(), testList(srv.URL, FormatLines))
|
||||||
b.Start(t.Context())
|
b.Start(t.Context())
|
||||||
|
|
||||||
select {
|
select {
|
||||||
@@ -1,7 +1,10 @@
|
|||||||
package blocklist
|
package iplist
|
||||||
|
|
||||||
import "time"
|
import "time"
|
||||||
|
|
||||||
|
// TorExitsList is the name of the list of Tor exit nodes.
|
||||||
|
const TorExitsList = "tor_exits"
|
||||||
|
|
||||||
// How often each list is downloaded. No list is downloaded more often than
|
// How often each list is downloaded. No list is downloaded more often than
|
||||||
// its publisher updates it or allows.
|
// its publisher updates it or allows.
|
||||||
const (
|
const (
|
||||||
@@ -30,7 +33,7 @@ const (
|
|||||||
torExitsInterval = time.Hour
|
torExitsInterval = time.Hour
|
||||||
)
|
)
|
||||||
|
|
||||||
// Lists returns the public blocklists routewatch downloads. Each can be
|
// Lists returns the public IP lists routewatch downloads. Each can be
|
||||||
// downloaded whole without an account or a key. Spamhaus merged its EDROP
|
// downloaded whole without an account or a key. Spamhaus merged its EDROP
|
||||||
// list into DROP, so DROP covers both.
|
// list into DROP, so DROP covers both.
|
||||||
func Lists() []List {
|
func Lists() []List {
|
||||||
@@ -83,7 +86,7 @@ func Lists() []List {
|
|||||||
Format: FormatSpamhausJSON,
|
Format: FormatSpamhausJSON,
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
Name: "tor_exits",
|
Name: TorExitsList,
|
||||||
Description: "A Tor exit node, where traffic from the Tor network leaves",
|
Description: "A Tor exit node, where traffic from the Tor network leaves",
|
||||||
URL: "https://check.torproject.org/torbulkexitlist",
|
URL: "https://check.torproject.org/torbulkexitlist",
|
||||||
Interval: torExitsInterval,
|
Interval: torExitsInterval,
|
||||||
@@ -1,4 +1,4 @@
|
|||||||
package blocklist
|
package iplist
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"bufio"
|
"bufio"
|
||||||
+16
-16
@@ -15,9 +15,9 @@ import (
|
|||||||
"sync"
|
"sync"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
"git.eeqj.de/sneak/routewatch/internal/blocklist"
|
|
||||||
"git.eeqj.de/sneak/routewatch/internal/config"
|
"git.eeqj.de/sneak/routewatch/internal/config"
|
||||||
"git.eeqj.de/sneak/routewatch/internal/database"
|
"git.eeqj.de/sneak/routewatch/internal/database"
|
||||||
|
"git.eeqj.de/sneak/routewatch/internal/iplist"
|
||||||
"git.eeqj.de/sneak/routewatch/internal/logger"
|
"git.eeqj.de/sneak/routewatch/internal/logger"
|
||||||
"git.eeqj.de/sneak/routewatch/internal/metrics"
|
"git.eeqj.de/sneak/routewatch/internal/metrics"
|
||||||
"git.eeqj.de/sneak/routewatch/internal/routetable"
|
"git.eeqj.de/sneak/routewatch/internal/routetable"
|
||||||
@@ -61,12 +61,12 @@ type RouteWatch struct {
|
|||||||
peeringHandler *PeeringHandler
|
peeringHandler *PeeringHandler
|
||||||
asnFetcher *ASNFetcher
|
asnFetcher *ASNFetcher
|
||||||
dbMaintainer *DBMaintainer
|
dbMaintainer *DBMaintainer
|
||||||
blocklists *blocklist.Blocklists
|
ipLists *iplist.IPLists
|
||||||
}
|
}
|
||||||
|
|
||||||
// blocklistsDirName is the directory in the state directory that holds the
|
// ipListsDirName is the directory in the state directory that holds the IP
|
||||||
// blocklist copies.
|
// list copies.
|
||||||
const blocklistsDirName = "blocklists"
|
const ipListsDirName = "iplists"
|
||||||
|
|
||||||
// New creates a new RouteWatch instance
|
// New creates a new RouteWatch instance
|
||||||
func New(deps Dependencies) *RouteWatch {
|
func New(deps Dependencies) *RouteWatch {
|
||||||
@@ -138,17 +138,17 @@ func (rw *RouteWatch) Run(ctx context.Context) error {
|
|||||||
rw.asnFetcher.Start(ctx)
|
rw.asnFetcher.Start(ctx)
|
||||||
rw.server.SetASNFetcher(rw.asnFetcher)
|
rw.server.SetASNFetcher(rw.asnFetcher)
|
||||||
|
|
||||||
// Load the blocklist copies from disk and start downloading the lists.
|
// Load the IP list copies from disk and start downloading the lists.
|
||||||
// The HTTP handlers read the server's blocklists without a lock, so they
|
// The HTTP handlers read the server's IP lists without a lock, so they
|
||||||
// are set before the handlers run.
|
// are set before the handlers run.
|
||||||
blocklists := blocklist.New(blocklist.Params{
|
ipLists := iplist.New(iplist.Params{
|
||||||
Dir: filepath.Join(rw.config.StateDir, blocklistsDirName),
|
Dir: filepath.Join(rw.config.StateDir, ipListsDirName),
|
||||||
Lists: blocklist.Lists(),
|
Lists: iplist.Lists(),
|
||||||
Logger: rw.logger.Logger,
|
Logger: rw.logger.Logger,
|
||||||
})
|
})
|
||||||
blocklists.Start(ctx)
|
ipLists.Start(ctx)
|
||||||
rw.blocklists = blocklists
|
rw.ipLists = ipLists
|
||||||
rw.server.SetBlocklists(blocklists)
|
rw.server.SetIPLists(ipLists)
|
||||||
|
|
||||||
// Start HTTP server
|
// Start HTTP server
|
||||||
err = rw.server.Start(ctx)
|
err = rw.server.Start(ctx)
|
||||||
@@ -168,7 +168,7 @@ func (rw *RouteWatch) Run(ctx context.Context) error {
|
|||||||
|
|
||||||
// Shutdown stops all services in order: the streamer, the handlers, which
|
// Shutdown stops all services in order: the streamer, the handlers, which
|
||||||
// write what they hold, the WHOIS fetcher, the database maintainer, the
|
// write what they hold, the WHOIS fetcher, the database maintainer, the
|
||||||
// blocklist downloads and the HTTP server, and last the database, which it
|
// IP list downloads and the HTTP server, and last the database, which it
|
||||||
// checkpoints and closes. Each stage waits at most until ctx ends, so
|
// checkpoints and closes. Each stage waits at most until ctx ends, so
|
||||||
// Shutdown returns soon after: a handler logs what it could not write by
|
// Shutdown returns soon after: a handler logs what it could not write by
|
||||||
// then, and the HTTP server closes the connections of requests still
|
// then, and the HTTP server closes the connections of requests still
|
||||||
@@ -219,8 +219,8 @@ func (rw *RouteWatch) Shutdown(ctx context.Context) {
|
|||||||
rw.dbMaintainer.Stop(ctx)
|
rw.dbMaintainer.Stop(ctx)
|
||||||
}
|
}
|
||||||
|
|
||||||
if rw.blocklists != nil {
|
if rw.ipLists != nil {
|
||||||
rw.blocklists.Stop(ctx)
|
rw.ipLists.Stop(ctx)
|
||||||
}
|
}
|
||||||
|
|
||||||
err := rw.server.Stop(ctx)
|
err := rw.server.Stop(ctx)
|
||||||
|
|||||||
@@ -418,7 +418,7 @@ func TestRouteWatchLiveFeed(t *testing.T) {
|
|||||||
// Create streamer
|
// Create streamer
|
||||||
s := streamer.New(logger, metricsTracker)
|
s := streamer.New(logger, metricsTracker)
|
||||||
|
|
||||||
// The state directory holds the blocklist copies Run downloads.
|
// The state directory holds the IP list copies Run downloads.
|
||||||
cfg := &config.Config{
|
cfg := &config.Config{
|
||||||
StateDir: t.TempDir(),
|
StateDir: t.TempDir(),
|
||||||
MaxRuntime: 5 * time.Second,
|
MaxRuntime: 5 * time.Second,
|
||||||
|
|||||||
+73
-22
@@ -15,8 +15,9 @@ import (
|
|||||||
"strings"
|
"strings"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
"git.eeqj.de/sneak/routewatch/internal/blocklist"
|
"git.eeqj.de/sneak/routewatch/internal/bogon"
|
||||||
"git.eeqj.de/sneak/routewatch/internal/database"
|
"git.eeqj.de/sneak/routewatch/internal/database"
|
||||||
|
"git.eeqj.de/sneak/routewatch/internal/iplist"
|
||||||
"git.eeqj.de/sneak/routewatch/internal/metrics"
|
"git.eeqj.de/sneak/routewatch/internal/metrics"
|
||||||
"git.eeqj.de/sneak/routewatch/internal/ristypes"
|
"git.eeqj.de/sneak/routewatch/internal/ristypes"
|
||||||
"git.eeqj.de/sneak/routewatch/internal/routetable"
|
"git.eeqj.de/sneak/routewatch/internal/routetable"
|
||||||
@@ -765,9 +766,12 @@ type IPLookupResponse struct {
|
|||||||
Errors []string `json:"errors,omitempty"`
|
Errors []string `json:"errors,omitempty"`
|
||||||
}
|
}
|
||||||
|
|
||||||
// IPLookupResult is the answer for one address: its PTR names, its live route
|
// IPLookupResult is the answer for one address: its PTR names, its flags, its
|
||||||
// and that route's AS, and the blocklists that list it. Every result has every
|
// live route and that route's AS, and the IP lists that contain it. Every
|
||||||
// field. The fields of IPInfo are null when the address has no live route.
|
// result has every field. The fields of IPInfo are null when the address has
|
||||||
|
// no live route.
|
||||||
|
//
|
||||||
|
//nolint:tagliatelle // snake_case keys, as the Go style guide asks
|
||||||
type IPLookupResult struct {
|
type IPLookupResult struct {
|
||||||
IPInfo
|
IPInfo
|
||||||
|
|
||||||
@@ -775,8 +779,22 @@ type IPLookupResult struct {
|
|||||||
// PTR is empty, never nil, when the address has no PTR names.
|
// PTR is empty, never nil, when the address has no PTR names.
|
||||||
PTR []string `json:"ptr"`
|
PTR []string `json:"ptr"`
|
||||||
|
|
||||||
// Blocklists has one entry for each blocklist the address is on.
|
Flags IPLookupFlags `json:"flags"`
|
||||||
Blocklists []blocklist.Listing `json:"blocklists"`
|
|
||||||
|
// IPLists has one entry for each IP list the address is on.
|
||||||
|
IPLists []iplist.Listing `json:"ip_lists"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// IPLookupFlags are the yes-or-no facts about an address. Every result has
|
||||||
|
// them, whether or not the address has a live route.
|
||||||
|
//
|
||||||
|
//nolint:tagliatelle // snake_case keys, as the Go style guide asks
|
||||||
|
type IPLookupFlags struct {
|
||||||
|
// IsTor says that the address is on the Tor exit list.
|
||||||
|
IsTor bool `json:"is_tor"`
|
||||||
|
|
||||||
|
// IsBogon says that the address is reserved, as package bogon tells.
|
||||||
|
IsBogon bool `json:"is_bogon"`
|
||||||
}
|
}
|
||||||
|
|
||||||
// handleIPInfo returns a handler that provides comprehensive IP information.
|
// handleIPInfo returns a handler that provides comprehensive IP information.
|
||||||
@@ -852,12 +870,7 @@ func (s *Server) handleIPInfo() http.HandlerFunc {
|
|||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
|
||||||
response.Results = append(response.Results, IPLookupResult{
|
response.Results = append(response.Results, s.ipLookupResult(ip, ptrs, ipInfo))
|
||||||
IP: ip,
|
|
||||||
PTR: ptrs,
|
|
||||||
IPInfo: ipInfo,
|
|
||||||
Blocklists: s.blocklistListings(ip),
|
|
||||||
})
|
|
||||||
|
|
||||||
if ipInfo.NeedsWHOISRefresh {
|
if ipInfo.NeedsWHOISRefresh {
|
||||||
refreshASNs[*ipInfo.ASN] = true
|
refreshASNs[*ipInfo.ASN] = true
|
||||||
@@ -958,23 +971,61 @@ func ipVersionOf(prefix netip.Prefix) int {
|
|||||||
return ipVersionV6
|
return ipVersionV6
|
||||||
}
|
}
|
||||||
|
|
||||||
// blocklistListings returns the blocklists that list ip. It returns an empty
|
// ipLookupResult returns the result for ip, adding its flags and IP lists to
|
||||||
// list, never nil, so that an answer's blocklists is [] rather than null.
|
// its PTR names and route.
|
||||||
func (s *Server) blocklistListings(ip string) []blocklist.Listing {
|
func (s *Server) ipLookupResult(ip string, ptrs []string, info IPInfo) IPLookupResult {
|
||||||
|
listings := s.ipListListings(ip)
|
||||||
|
|
||||||
|
return IPLookupResult{
|
||||||
|
IP: ip,
|
||||||
|
PTR: ptrs,
|
||||||
|
IPInfo: info,
|
||||||
|
Flags: ipLookupFlags(ip, listings),
|
||||||
|
IPLists: listings,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// ipListListings returns the IP lists that contain ip. It returns an empty
|
||||||
|
// list, never nil, so that an answer's ip_lists is [] rather than null.
|
||||||
|
func (s *Server) ipListListings(ip string) []iplist.Listing {
|
||||||
addr, err := netip.ParseAddr(ip)
|
addr, err := netip.ParseAddr(ip)
|
||||||
if s.blocklists == nil || err != nil {
|
if s.ipLists == nil || err != nil {
|
||||||
return []blocklist.Listing{}
|
return []iplist.Listing{}
|
||||||
}
|
}
|
||||||
|
|
||||||
return s.blocklists.Lookup(addr)
|
return s.ipLists.Lookup(addr)
|
||||||
|
}
|
||||||
|
|
||||||
|
// ipLookupFlags returns the flags of ip, whose IP list entries are listings.
|
||||||
|
func ipLookupFlags(ip string, listings []iplist.Listing) IPLookupFlags {
|
||||||
|
return IPLookupFlags{
|
||||||
|
IsTor: slices.ContainsFunc(listings, func(l iplist.Listing) bool {
|
||||||
|
return l.Name == iplist.TorExitsList
|
||||||
|
}),
|
||||||
|
IsBogon: isBogon(ip),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// isBogon says whether the address ip is reserved. An IPv4 address written in
|
||||||
|
// IPv6 form counts as the IPv4 address it holds.
|
||||||
|
func isBogon(ip string) bool {
|
||||||
|
addr, err := netip.ParseAddr(ip)
|
||||||
|
if err != nil {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
|
||||||
|
addr = addr.Unmap()
|
||||||
|
_, reserved := bogon.Prefix(netip.PrefixFrom(addr, addr.BitLen()))
|
||||||
|
|
||||||
|
return reserved
|
||||||
}
|
}
|
||||||
|
|
||||||
// setIPLookupCacheControl lets caches keep an IP lookup answer for the
|
// setIPLookupCacheControl lets caches keep an IP lookup answer for the
|
||||||
// shortest blocklist interval, the soonest a blocklist copy can change. The
|
// shortest IP list interval, the soonest an IP list copy can change. The
|
||||||
// answer about the caller's own address is private, so that a shared cache
|
// answer about the caller's own address is private, so that a shared cache
|
||||||
// does not give it to another caller. Without blocklists it sets nothing.
|
// does not give it to another caller. Without IP lists it sets nothing.
|
||||||
func (s *Server) setIPLookupCacheControl(w http.ResponseWriter, ownAddress bool) {
|
func (s *Server) setIPLookupCacheControl(w http.ResponseWriter, ownAddress bool) {
|
||||||
if s.blocklists == nil {
|
if s.ipLists == nil {
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -983,7 +1034,7 @@ func (s *Server) setIPLookupCacheControl(w http.ResponseWriter, ownAddress bool)
|
|||||||
visibility = cacheControlPrivate
|
visibility = cacheControlPrivate
|
||||||
}
|
}
|
||||||
|
|
||||||
maxAgeSeconds := int(s.blocklists.ShortestInterval().Seconds())
|
maxAgeSeconds := int(s.ipLists.ShortestInterval().Seconds())
|
||||||
w.Header().Set(cacheControlHeader,
|
w.Header().Set(cacheControlHeader,
|
||||||
visibility+cacheControlMaxAge+strconv.Itoa(maxAgeSeconds))
|
visibility+cacheControlMaxAge+strconv.Itoa(maxAgeSeconds))
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -17,11 +17,12 @@ import (
|
|||||||
"strings"
|
"strings"
|
||||||
"sync/atomic"
|
"sync/atomic"
|
||||||
"testing"
|
"testing"
|
||||||
|
"testing/synctest"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
"git.eeqj.de/sneak/routewatch/internal/blocklist"
|
|
||||||
"git.eeqj.de/sneak/routewatch/internal/config"
|
"git.eeqj.de/sneak/routewatch/internal/config"
|
||||||
"git.eeqj.de/sneak/routewatch/internal/database"
|
"git.eeqj.de/sneak/routewatch/internal/database"
|
||||||
|
"git.eeqj.de/sneak/routewatch/internal/iplist"
|
||||||
"git.eeqj.de/sneak/routewatch/internal/logger"
|
"git.eeqj.de/sneak/routewatch/internal/logger"
|
||||||
"git.eeqj.de/sneak/routewatch/internal/metrics"
|
"git.eeqj.de/sneak/routewatch/internal/metrics"
|
||||||
"git.eeqj.de/sneak/routewatch/internal/ristypes"
|
"git.eeqj.de/sneak/routewatch/internal/ristypes"
|
||||||
@@ -173,6 +174,9 @@ type tableStats struct {
|
|||||||
// carried 203.0.113.0/24, the two with kept routes withdrew it, so its origin
|
// carried 203.0.113.0/24, the two with kept routes withdrew it, so its origin
|
||||||
// keeps no route. The oldest and newest routes are those of the other two
|
// keeps no route. The oldest and newest routes are those of the other two
|
||||||
// prefixes.
|
// prefixes.
|
||||||
|
//
|
||||||
|
// The requests run in a synctest bubble, whose clock is fake, so a stall on
|
||||||
|
// the host does not use up the handlers' 4-second limit.
|
||||||
func TestStatsHandlersAnswerFromMemory(t *testing.T) {
|
func TestStatsHandlersAnswerFromMemory(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
@@ -218,7 +222,11 @@ func TestStatsHandlersAnswerFromMemory(t *testing.T) {
|
|||||||
|
|
||||||
for name, handler := range handlers {
|
for name, handler := range handlers {
|
||||||
rec := httptest.NewRecorder()
|
rec := httptest.NewRecorder()
|
||||||
handler(rec, httptest.NewRequestWithContext(t.Context(), http.MethodGet, "/", nil))
|
|
||||||
|
synctest.Test(t, func(t *testing.T) {
|
||||||
|
handler(rec,
|
||||||
|
httptest.NewRequestWithContext(t.Context(), http.MethodGet, "/", nil))
|
||||||
|
})
|
||||||
|
|
||||||
if rec.Code != http.StatusOK {
|
if rec.Code != http.StatusOK {
|
||||||
t.Errorf("%s: status %d, want %d; body %s",
|
t.Errorf("%s: status %d, want %d; body %s",
|
||||||
@@ -253,15 +261,20 @@ func TestStatsHandlersAnswerFromMemory(t *testing.T) {
|
|||||||
// TestStatsAnswerCarriesChurnPerMinute checks that the stream object of the
|
// TestStatsAnswerCarriesChurnPerMinute checks that the stream object of the
|
||||||
// /api/v1/stats answer holds route_churn_per_min as a whole number, and no
|
// /api/v1/stats answer holds route_churn_per_min as a whole number, and no
|
||||||
// longer route_churn_per_sec, the moving average it replaces
|
// longer route_churn_per_sec, the moving average it replaces
|
||||||
// (https://git.eeqj.de/sneak/routewatch/issues/160).
|
// (https://git.eeqj.de/sneak/routewatch/issues/160). The request runs in a
|
||||||
|
// synctest bubble, whose clock is fake, so a stall on the host does not use up
|
||||||
|
// the handler's 4-second limit.
|
||||||
func TestStatsAnswerCarriesChurnPerMinute(t *testing.T) {
|
func TestStatsAnswerCarriesChurnPerMinute(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
s := newServerWithTestRoutes(t)
|
s := newServerWithTestRoutes(t)
|
||||||
|
|
||||||
rec := httptest.NewRecorder()
|
rec := httptest.NewRecorder()
|
||||||
s.handleStats()(rec,
|
|
||||||
httptest.NewRequestWithContext(t.Context(), http.MethodGet, statsPath, nil))
|
synctest.Test(t, func(t *testing.T) {
|
||||||
|
s.handleStats()(rec,
|
||||||
|
httptest.NewRequestWithContext(t.Context(), http.MethodGet, statsPath, nil))
|
||||||
|
})
|
||||||
|
|
||||||
if rec.Code != http.StatusOK {
|
if rec.Code != http.StatusOK {
|
||||||
t.Fatalf("status %d, want %d; body %s", rec.Code, http.StatusOK, rec.Body)
|
t.Fatalf("status %d, want %d; body %s", rec.Code, http.StatusOK, rec.Body)
|
||||||
@@ -309,18 +322,26 @@ func (heldBackHandler) HeldBackCount() int { return heldBackMess
|
|||||||
|
|
||||||
// TestStatsAnswerGivesHowManyMessagesAHandlerHoldsBack registers a handler
|
// TestStatsAnswerGivesHowManyMessagesAHandlerHoldsBack registers a handler
|
||||||
// that holds back messages. Its figures in /api/v1/stats, which the status
|
// that holds back messages. Its figures in /api/v1/stats, which the status
|
||||||
// page shows, must give how many.
|
// page shows, must give how many. The request runs in a synctest bubble, whose
|
||||||
|
// clock is fake, so a stall on the host does not use up the handler's 4-second
|
||||||
|
// limit.
|
||||||
func TestStatsAnswerGivesHowManyMessagesAHandlerHoldsBack(t *testing.T) {
|
func TestStatsAnswerGivesHowManyMessagesAHandlerHoldsBack(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
s := newServerWithTestRoutes(t)
|
s := newServerWithTestRoutes(t)
|
||||||
s.streamer.RegisterHandler(heldBackHandler{})
|
s.streamer.RegisterHandler(heldBackHandler{})
|
||||||
|
|
||||||
|
var answer []byte
|
||||||
|
|
||||||
|
synctest.Test(t, func(t *testing.T) {
|
||||||
|
answer = get(t, s, statsPath).Body.Bytes()
|
||||||
|
})
|
||||||
|
|
||||||
var body struct {
|
var body struct {
|
||||||
Data StatsResponse `json:"data"`
|
Data StatsResponse `json:"data"`
|
||||||
}
|
}
|
||||||
|
|
||||||
err := json.Unmarshal(get(t, s, statsPath).Body.Bytes(), &body)
|
err := json.Unmarshal(answer, &body)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("decoding the answer: %v", err)
|
t.Fatalf("decoding the answer: %v", err)
|
||||||
}
|
}
|
||||||
@@ -364,7 +385,9 @@ type unitStats struct {
|
|||||||
// names. They used to be strings with the unit in them, such as "1h18m34s" or
|
// names. They used to be strings with the unit in them, such as "1h18m34s" or
|
||||||
// "1.1 GB" (https://git.eeqj.de/sneak/routewatch/issues/199). The stream has
|
// "1.1 GB" (https://git.eeqj.de/sneak/routewatch/issues/199). The stream has
|
||||||
// never connected, so the uptime is 0. The memory use in /api/v1/stats is the
|
// never connected, so the uptime is 0. The memory use in /api/v1/stats is the
|
||||||
// same number of bytes as its gc.heap_alloc_bytes.
|
// same number of bytes as its gc.heap_alloc_bytes. The requests run in a
|
||||||
|
// synctest bubble, whose clock is fake, so a stall on the host does not use up
|
||||||
|
// the handlers' 4-second limit.
|
||||||
func TestStatsAnswersGiveUnitsInFieldNames(t *testing.T) {
|
func TestStatsAnswersGiveUnitsInFieldNames(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
@@ -374,7 +397,11 @@ func TestStatsAnswersGiveUnitsInFieldNames(t *testing.T) {
|
|||||||
s.SetASNFetcher(fixedIntervalASNFetcher{interval: fetchInterval})
|
s.SetASNFetcher(fixedIntervalASNFetcher{interval: fetchInterval})
|
||||||
|
|
||||||
for _, path := range []string{statusJSONPath, statsPath} {
|
for _, path := range []string{statusJSONPath, statsPath} {
|
||||||
answer := get(t, s, path).Body.Bytes()
|
var answer []byte
|
||||||
|
|
||||||
|
synctest.Test(t, func(t *testing.T) {
|
||||||
|
answer = get(t, s, path).Body.Bytes()
|
||||||
|
})
|
||||||
|
|
||||||
var body struct {
|
var body struct {
|
||||||
Data unitStats `json:"data"`
|
Data unitStats `json:"data"`
|
||||||
@@ -415,49 +442,53 @@ func TestStatsAnswersGiveUnitsInFieldNames(t *testing.T) {
|
|||||||
// connection_duration_seconds of /api/v1/stats is null while the stream is not
|
// connection_duration_seconds of /api/v1/stats is null while the stream is not
|
||||||
// connected, and a whole number of seconds once it is. It used to be the
|
// connected, and a whole number of seconds once it is. It used to be the
|
||||||
// string "disconnected", or a Go duration such as "1h18m34s"
|
// string "disconnected", or a Go duration such as "1h18m34s"
|
||||||
// (https://git.eeqj.de/sneak/routewatch/issues/199).
|
// (https://git.eeqj.de/sneak/routewatch/issues/199). The test runs in a
|
||||||
|
// synctest bubble, whose clock is fake, so a stall on the host does not use up
|
||||||
|
// the handler's 4-second limit.
|
||||||
func TestStatsAnswerGivesConnectionDurationInSeconds(t *testing.T) {
|
func TestStatsAnswerGivesConnectionDurationInSeconds(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
s := newServerWithTestRoutes(t)
|
s := newServerWithTestRoutes(t)
|
||||||
|
|
||||||
connectionDuration := func() json.RawMessage {
|
synctest.Test(t, func(t *testing.T) {
|
||||||
answer := get(t, s, statsPath).Body.Bytes()
|
connectionDuration := func() json.RawMessage {
|
||||||
|
answer := get(t, s, statsPath).Body.Bytes()
|
||||||
|
|
||||||
var body struct {
|
var body struct {
|
||||||
Data map[string]json.RawMessage `json:"data"`
|
Data map[string]json.RawMessage `json:"data"`
|
||||||
|
}
|
||||||
|
|
||||||
|
err := json.Unmarshal(answer, &body)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("decoding the answer: %v; answer %s", err, answer)
|
||||||
|
}
|
||||||
|
|
||||||
|
duration, ok := body.Data["connection_duration_seconds"]
|
||||||
|
if !ok {
|
||||||
|
t.Fatalf("no connection_duration_seconds: %s", answer)
|
||||||
|
}
|
||||||
|
|
||||||
|
return duration
|
||||||
}
|
}
|
||||||
|
|
||||||
err := json.Unmarshal(answer, &body)
|
disconnected := connectionDuration()
|
||||||
if err != nil {
|
if string(disconnected) != "null" {
|
||||||
t.Fatalf("decoding the answer: %v; answer %s", err, answer)
|
t.Errorf("disconnected: connection_duration_seconds = %s, want null",
|
||||||
|
disconnected)
|
||||||
}
|
}
|
||||||
|
|
||||||
duration, ok := body.Data["connection_duration_seconds"]
|
s.streamer.GetMetricsTracker().SetConnected(true)
|
||||||
if !ok {
|
|
||||||
t.Fatalf("no connection_duration_seconds: %s", answer)
|
var seconds int64
|
||||||
|
|
||||||
|
connected := connectionDuration()
|
||||||
|
|
||||||
|
err := json.Unmarshal(connected, &seconds)
|
||||||
|
if err != nil || seconds != 0 {
|
||||||
|
t.Errorf("just connected: connection_duration_seconds = %s, want 0",
|
||||||
|
connected)
|
||||||
}
|
}
|
||||||
|
})
|
||||||
return duration
|
|
||||||
}
|
|
||||||
|
|
||||||
disconnected := connectionDuration()
|
|
||||||
if string(disconnected) != "null" {
|
|
||||||
t.Errorf("disconnected: connection_duration_seconds = %s, want null",
|
|
||||||
disconnected)
|
|
||||||
}
|
|
||||||
|
|
||||||
s.streamer.GetMetricsTracker().SetConnected(true)
|
|
||||||
|
|
||||||
var seconds int64
|
|
||||||
|
|
||||||
connected := connectionDuration()
|
|
||||||
|
|
||||||
err := json.Unmarshal(connected, &seconds)
|
|
||||||
if err != nil || seconds != 0 {
|
|
||||||
t.Errorf("just connected: connection_duration_seconds = %s, want 0",
|
|
||||||
connected)
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// TestStreamDurationsCountSeconds checks that the uptime and the connection
|
// TestStreamDurationsCountSeconds checks that the uptime and the connection
|
||||||
@@ -1134,7 +1165,9 @@ func (failingDB) GetPrefixFirstSeen(
|
|||||||
// with the driver's error text. It also sends a prefix that does not parse.
|
// with the driver's error text. It also sends a prefix that does not parse.
|
||||||
// No answer may carry the driver's or the prefix parser's text. Before the fix
|
// No answer may carry the driver's or the prefix parser's text. Before the fix
|
||||||
// the answers sent it to the client
|
// the answers sent it to the client
|
||||||
// (https://git.eeqj.de/sneak/routewatch/issues/108).
|
// (https://git.eeqj.de/sneak/routewatch/issues/108). The requests run in a
|
||||||
|
// synctest bubble, whose clock is fake, so a stall on the host does not use up
|
||||||
|
// the stats handlers' 4-second limit.
|
||||||
func TestJSONRoutesDoNotSendErrorDetails(t *testing.T) {
|
func TestJSONRoutesDoNotSendErrorDetails(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
@@ -1155,8 +1188,11 @@ func TestJSONRoutesDoNotSendErrorDetails(t *testing.T) {
|
|||||||
|
|
||||||
for _, tt := range tests {
|
for _, tt := range tests {
|
||||||
rec := httptest.NewRecorder()
|
rec := httptest.NewRecorder()
|
||||||
s.router.ServeHTTP(rec, httptest.NewRequestWithContext(
|
|
||||||
t.Context(), http.MethodGet, tt.path, nil))
|
synctest.Test(t, func(t *testing.T) {
|
||||||
|
s.router.ServeHTTP(rec, httptest.NewRequestWithContext(
|
||||||
|
t.Context(), http.MethodGet, tt.path, nil))
|
||||||
|
})
|
||||||
|
|
||||||
if rec.Code != tt.wantCode {
|
if rec.Code != tt.wantCode {
|
||||||
t.Errorf("%s: status %d, want %d; body %s",
|
t.Errorf("%s: status %d, want %d; body %s",
|
||||||
@@ -1397,8 +1433,11 @@ func TestIPLookupKeepsRouteWhenPTRLookupDoesNotAnswer(t *testing.T) {
|
|||||||
s := newServerWithTestRoutes(t)
|
s := newServerWithTestRoutes(t)
|
||||||
s.resolver = &fakeResolver{ptrNeverAnswers: true}
|
s.resolver = &fakeResolver{ptrNeverAnswers: true}
|
||||||
|
|
||||||
// The request's time limit passes after the PTR lookup's.
|
// The request's time limit passes well after the PTR lookup's, so that a
|
||||||
ctx, cancel := context.WithTimeout(t.Context(), 2*ptrLookupTimeout)
|
// stall on a busy host does not use up the difference.
|
||||||
|
const requestTimeLimit = ptrLookupTimeout + 15*time.Second
|
||||||
|
|
||||||
|
ctx, cancel := context.WithTimeout(t.Context(), requestTimeLimit)
|
||||||
defer cancel()
|
defer cancel()
|
||||||
|
|
||||||
rec := httptest.NewRecorder()
|
rec := httptest.NewRecorder()
|
||||||
@@ -1428,50 +1467,112 @@ func TestIPLookupKeepsRouteWhenPTRLookupDoesNotAnswer(t *testing.T) {
|
|||||||
// newServerWithTestRoutes stores a route for.
|
// newServerWithTestRoutes stores a route for.
|
||||||
const addrInTestPrefixV6 = "2001:db8::1"
|
const addrInTestPrefixV6 = "2001:db8::1"
|
||||||
|
|
||||||
// testBlocklistInterval is fakeBlocklists' shortest interval.
|
// testIPListInterval is fakeIPLists' shortest interval.
|
||||||
const testBlocklistInterval = 15 * time.Minute
|
const testIPListInterval = 15 * time.Minute
|
||||||
|
|
||||||
// ownAddressPath is the IP lookup of the caller's own address.
|
// ownAddressPath is the IP lookup of the caller's own address.
|
||||||
const ownAddressPath = "/ip/"
|
const ownAddressPath = "/ip/"
|
||||||
|
|
||||||
// listedUnroutedAddr is an address that fakeBlocklists lists, outside the
|
// listedUnroutedAddr is an address that fakeIPLists lists, outside the routes
|
||||||
// routes newServerWithTestRoutes stores.
|
// newServerWithTestRoutes stores.
|
||||||
const listedUnroutedAddr = "203.0.113.1"
|
const listedUnroutedAddr = "203.0.113.1"
|
||||||
|
|
||||||
// fakeBlocklists has one list, which lists addrInTestPrefixV4 and
|
// torExitAddr is the address that fakeIPLists lists on the Tor exit list.
|
||||||
// listedUnroutedAddr and no other address.
|
const torExitAddr = "203.0.113.2"
|
||||||
type fakeBlocklists struct{}
|
|
||||||
|
|
||||||
func testListing() blocklist.Listing {
|
// fakeIPLists has two lists. The test list holds addrInTestPrefixV4 and
|
||||||
return blocklist.Listing{
|
// listedUnroutedAddr, and the Tor exit list holds torExitAddr.
|
||||||
|
type fakeIPLists struct{}
|
||||||
|
|
||||||
|
func testListing() iplist.Listing {
|
||||||
|
return iplist.Listing{
|
||||||
Name: "test_list",
|
Name: "test_list",
|
||||||
Description: "On the test list",
|
Description: "On the test list",
|
||||||
Updated: time.Date(2026, 10, 27, 4, 0, 0, 0, time.UTC),
|
Updated: time.Date(2026, 10, 27, 4, 0, 0, 0, time.UTC),
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func (fakeBlocklists) Lookup(addr netip.Addr) []blocklist.Listing {
|
func (fakeIPLists) Lookup(addr netip.Addr) []iplist.Listing {
|
||||||
if addr == netip.MustParseAddr(addrInTestPrefixV4) ||
|
if addr == netip.MustParseAddr(addrInTestPrefixV4) ||
|
||||||
addr == netip.MustParseAddr(listedUnroutedAddr) {
|
addr == netip.MustParseAddr(listedUnroutedAddr) {
|
||||||
return []blocklist.Listing{testListing()}
|
return []iplist.Listing{testListing()}
|
||||||
}
|
}
|
||||||
|
|
||||||
return []blocklist.Listing{}
|
if addr == netip.MustParseAddr(torExitAddr) {
|
||||||
|
return []iplist.Listing{{Name: iplist.TorExitsList}}
|
||||||
|
}
|
||||||
|
|
||||||
|
return []iplist.Listing{}
|
||||||
}
|
}
|
||||||
|
|
||||||
func (fakeBlocklists) ShortestInterval() time.Duration {
|
func (fakeIPLists) ShortestInterval() time.Duration {
|
||||||
return testBlocklistInterval
|
return testIPListInterval
|
||||||
}
|
}
|
||||||
|
|
||||||
// TestIPLookupReportsBlocklists looks up two routed addresses, one of them on
|
// TestIPLookupFlags looks up addresses with and without a route. Every result
|
||||||
// a blocklist. The listed address's result names the list, says what it means
|
// has both flags, under the JSON keys the API documents.
|
||||||
// and when it was updated. The other's result has an empty list.
|
func TestIPLookupFlags(t *testing.T) {
|
||||||
func TestIPLookupReportsBlocklists(t *testing.T) {
|
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
s := newServerWithTestRoutes(t)
|
s := newServerWithTestRoutes(t)
|
||||||
s.resolver = &fakeResolver{}
|
s.resolver = &fakeResolver{}
|
||||||
s.SetBlocklists(fakeBlocklists{})
|
s.SetIPLists(fakeIPLists{})
|
||||||
|
|
||||||
|
tests := []struct {
|
||||||
|
name string
|
||||||
|
addr string
|
||||||
|
tor bool
|
||||||
|
bogon bool
|
||||||
|
}{
|
||||||
|
{"on the Tor exit list", torExitAddr, true, true},
|
||||||
|
{"on another list", listedUnroutedAddr, false, true},
|
||||||
|
{"routed IPv4", addrInTestPrefixV4, false, true},
|
||||||
|
{"routed IPv6", addrInTestPrefixV6, false, true},
|
||||||
|
{"private IPv4", "10.0.0.1", false, true},
|
||||||
|
{"loopback IPv4", "127.0.0.1", false, true},
|
||||||
|
{"public IPv4", "1.1.1.1", false, false},
|
||||||
|
{"private IPv6", "fd00::1", false, true},
|
||||||
|
{"loopback IPv6", "::1", false, true},
|
||||||
|
{"public IPv6", "2606:4700:4700::1111", false, false},
|
||||||
|
{"public IPv4 in IPv6 form", "::ffff:1.1.1.1", false, false},
|
||||||
|
{"private IPv4 in IPv6 form", "::ffff:10.0.0.1", false, true},
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, tt := range tests {
|
||||||
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
path := "/api/v1/ip/" + tt.addr
|
||||||
|
|
||||||
|
results := resultFields(t, get(t, s, path).Body.Bytes())
|
||||||
|
if len(results) != 1 {
|
||||||
|
t.Fatalf("%s: %d results, want 1", path, len(results))
|
||||||
|
}
|
||||||
|
|
||||||
|
var got map[string]bool
|
||||||
|
|
||||||
|
err := json.Unmarshal(results[0]["flags"], &got)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("%s: decoding flags %s: %v", path, results[0]["flags"], err)
|
||||||
|
}
|
||||||
|
|
||||||
|
want := map[string]bool{"is_tor": tt.tor, "is_bogon": tt.bogon}
|
||||||
|
if !maps.Equal(got, want) {
|
||||||
|
t.Errorf("%s: flags %v, want %v", path, got, want)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestIPLookupReportsIPLists looks up two routed addresses, one of them on an
|
||||||
|
// IP list. The listed address's result names the list, says what it means and
|
||||||
|
// when it was updated. The other's result has an empty list.
|
||||||
|
func TestIPLookupReportsIPLists(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
s := newServerWithTestRoutes(t)
|
||||||
|
s.resolver = &fakeResolver{}
|
||||||
|
s.SetIPLists(fakeIPLists{})
|
||||||
|
|
||||||
rec := httptest.NewRecorder()
|
rec := httptest.NewRecorder()
|
||||||
s.router.ServeHTTP(rec, httptest.NewRequestWithContext(t.Context(),
|
s.router.ServeHTTP(rec, httptest.NewRequestWithContext(t.Context(),
|
||||||
@@ -1498,15 +1599,15 @@ func TestIPLookupReportsBlocklists(t *testing.T) {
|
|||||||
|
|
||||||
want := testListing()
|
want := testListing()
|
||||||
|
|
||||||
listed := results[0].Blocklists
|
listed := results[0].IPLists
|
||||||
if len(listed) != 1 || listed[0].Name != want.Name ||
|
if len(listed) != 1 || listed[0].Name != want.Name ||
|
||||||
listed[0].Description != want.Description ||
|
listed[0].Description != want.Description ||
|
||||||
!listed[0].Updated.Equal(want.Updated) {
|
!listed[0].Updated.Equal(want.Updated) {
|
||||||
t.Errorf("%s: blocklists %+v, want [%+v]", results[0].IP, listed, want)
|
t.Errorf("%s: ip_lists %+v, want [%+v]", results[0].IP, listed, want)
|
||||||
}
|
}
|
||||||
|
|
||||||
if !strings.Contains(rec.Body.String(), `"blocklists":[]`) {
|
if !strings.Contains(rec.Body.String(), `"ip_lists":[]`) {
|
||||||
t.Errorf("answer %s, want an empty blocklists list for %s",
|
t.Errorf("answer %s, want an empty ip_lists list for %s",
|
||||||
rec.Body, addrInTestPrefixV6)
|
rec.Body, addrInTestPrefixV6)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -1536,20 +1637,19 @@ func resultFields(t *testing.T, answer []byte) []map[string]json.RawMessage {
|
|||||||
return body.Data.Results
|
return body.Data.Results
|
||||||
}
|
}
|
||||||
|
|
||||||
// TestIPLookupReportsBlocklistsOfAnAddressWithNoRoute looks up a listed
|
// TestIPLookupReportsIPListsOfAnAddressWithNoRoute looks up a listed address
|
||||||
// address that has no live route, alone and beside a routed address. Each
|
// that has no live route, alone and beside a routed address. Each answer is
|
||||||
// answer is 200 and has a result for the address, with its blocklists and
|
// 200 and has a result for the address, with its IP lists and every route
|
||||||
// every route field set to null. Before the fix the lookup of that address
|
// field set to null. Before the fix the lookup of that address alone answered
|
||||||
// alone answered 404, and beside a routed address it was only named in
|
// 404, and beside a routed address it was only named in errors, so its IP
|
||||||
// errors, so its blocklists were never reported. After that fix the result
|
// lists were never reported. After that fix the result still left out the
|
||||||
// still left out the route fields
|
// route fields (https://git.eeqj.de/sneak/routewatch/issues/200).
|
||||||
// (https://git.eeqj.de/sneak/routewatch/issues/200).
|
func TestIPLookupReportsIPListsOfAnAddressWithNoRoute(t *testing.T) {
|
||||||
func TestIPLookupReportsBlocklistsOfAnAddressWithNoRoute(t *testing.T) {
|
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
s := newServerWithTestRoutes(t)
|
s := newServerWithTestRoutes(t)
|
||||||
s.resolver = &fakeResolver{}
|
s.resolver = &fakeResolver{}
|
||||||
s.SetBlocklists(fakeBlocklists{})
|
s.SetIPLists(fakeIPLists{})
|
||||||
|
|
||||||
// routeFields are the fields of a result that come from the address's
|
// routeFields are the fields of a result that come from the address's
|
||||||
// route and that route's origin AS.
|
// route and that route's origin AS.
|
||||||
@@ -1605,10 +1705,10 @@ func TestIPLookupReportsBlocklistsOfAnAddressWithNoRoute(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
result := body.Data.Results[i]
|
result := body.Data.Results[i]
|
||||||
if len(result.Blocklists) != 1 ||
|
if len(result.IPLists) != 1 ||
|
||||||
result.Blocklists[0].Name != testListing().Name {
|
result.IPLists[0].Name != testListing().Name {
|
||||||
t.Errorf("%s: blocklists %+v, want [%+v]",
|
t.Errorf("%s: ip_lists %+v, want [%+v]",
|
||||||
target, result.Blocklists, testListing())
|
target, result.IPLists, testListing())
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -1629,7 +1729,7 @@ func TestIPLookupResultsHaveTheSameFields(t *testing.T) {
|
|||||||
s.resolver = &fakeResolver{hosts: map[string][]string{
|
s.resolver = &fakeResolver{hosts: map[string][]string{
|
||||||
twoAddressHostName: {addrInTestPrefixV4, listedUnroutedAddr},
|
twoAddressHostName: {addrInTestPrefixV4, listedUnroutedAddr},
|
||||||
}}
|
}}
|
||||||
s.SetBlocklists(fakeBlocklists{})
|
s.SetIPLists(fakeIPLists{})
|
||||||
|
|
||||||
// want is the fields of the first result.
|
// want is the fields of the first result.
|
||||||
var want []string
|
var want []string
|
||||||
@@ -1684,7 +1784,7 @@ func TestIPLookupOfAnAddressWithoutPTRNamesOrStoredNetblock(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// TestIPLookupAnswersMayBeCached checks the Cache-Control header of the IP
|
// TestIPLookupAnswersMayBeCached checks the Cache-Control header of the IP
|
||||||
// lookup answers. Caches may keep them for the blocklists' shortest interval,
|
// lookup answers. Caches may keep them for the IP lists' shortest interval,
|
||||||
// and only the caller's own browser may keep the answer about the caller's
|
// and only the caller's own browser may keep the answer about the caller's
|
||||||
// own address.
|
// own address.
|
||||||
func TestIPLookupAnswersMayBeCached(t *testing.T) {
|
func TestIPLookupAnswersMayBeCached(t *testing.T) {
|
||||||
@@ -1692,7 +1792,7 @@ func TestIPLookupAnswersMayBeCached(t *testing.T) {
|
|||||||
|
|
||||||
s := newServerWithTestRoutes(t)
|
s := newServerWithTestRoutes(t)
|
||||||
s.resolver = &fakeResolver{}
|
s.resolver = &fakeResolver{}
|
||||||
s.SetBlocklists(fakeBlocklists{})
|
s.SetIPLists(fakeIPLists{})
|
||||||
|
|
||||||
tests := []struct {
|
tests := []struct {
|
||||||
path string
|
path string
|
||||||
@@ -2226,9 +2326,13 @@ func settledGoroutineCount() int {
|
|||||||
return prev
|
return prev
|
||||||
}
|
}
|
||||||
|
|
||||||
// waitForGoroutines waits until the goroutine count drops to target or below.
|
// waitForGoroutines waits up to 15 seconds for the goroutine count to drop to
|
||||||
|
// target or below.
|
||||||
func waitForGoroutines(target int) bool {
|
func waitForGoroutines(target int) bool {
|
||||||
for range 100 {
|
const maxWait = 15 * time.Second
|
||||||
|
|
||||||
|
deadline := time.Now().Add(maxWait)
|
||||||
|
for time.Now().Before(deadline) {
|
||||||
if runtime.NumGoroutine() <= target {
|
if runtime.NumGoroutine() <= target {
|
||||||
return true
|
return true
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -76,7 +76,9 @@ func asLink(asn int) string {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// TestHomePageCardsShowTheTableFigures fills a routing table and the asns
|
// TestHomePageCardsShowTheTableFigures fills a routing table and the asns
|
||||||
// table, works out the cards, and checks that each card shows its figure.
|
// table, works out the cards, and checks that each card shows its figure. The
|
||||||
|
// cards are worked out in a synctest bubble, whose clock is fake, so a stall on
|
||||||
|
// the host does not use up the 4-second limit on their database read.
|
||||||
func TestHomePageCardsShowTheTableFigures(t *testing.T) {
|
func TestHomePageCardsShowTheTableFigures(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
@@ -95,7 +97,9 @@ func TestHomePageCardsShowTheTableFigures(t *testing.T) {
|
|||||||
t.Fatalf("GetOrCreateASNBatch: %v", err)
|
t.Fatalf("GetOrCreateASNBatch: %v", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
s.refreshHomeCards(t.Context())
|
synctest.Test(t, func(t *testing.T) {
|
||||||
|
s.refreshHomeCards(t.Context())
|
||||||
|
})
|
||||||
|
|
||||||
page := get(t, s, "/").Body.String()
|
page := get(t, s, "/").Body.String()
|
||||||
|
|
||||||
@@ -279,7 +283,7 @@ func TestKeepHomeCardsFreshWorksThemOutEveryInterval(t *testing.T) {
|
|||||||
|
|
||||||
const (
|
const (
|
||||||
interval = 10 * time.Millisecond
|
interval = 10 * time.Millisecond
|
||||||
maxWait = 5 * time.Second
|
maxWait = 15 * time.Second
|
||||||
)
|
)
|
||||||
|
|
||||||
table := emptyTable()
|
table := emptyTable()
|
||||||
@@ -321,7 +325,7 @@ func TestKeepHomeCardsFreshWorksThemOutEveryInterval(t *testing.T) {
|
|||||||
func TestStartedServerWorksOutTheCards(t *testing.T) {
|
func TestStartedServerWorksOutTheCards(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
const maxWait = 5 * time.Second
|
const maxWait = 15 * time.Second
|
||||||
|
|
||||||
s := newServerWithTable(t, newCardsTable())
|
s := newServerWithTable(t, newCardsTable())
|
||||||
|
|
||||||
|
|||||||
@@ -14,6 +14,7 @@ import (
|
|||||||
"regexp"
|
"regexp"
|
||||||
"strings"
|
"strings"
|
||||||
"testing"
|
"testing"
|
||||||
|
"testing/synctest"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
"git.eeqj.de/sneak/routewatch/internal/config"
|
"git.eeqj.de/sneak/routewatch/internal/config"
|
||||||
@@ -163,6 +164,10 @@ func TestJSONAnswersPutNoUnitInAString(t *testing.T) {
|
|||||||
// router and the middleware write. Before the fix a panic in a handler, and a
|
// router and the middleware write. Before the fix a panic in a handler, and a
|
||||||
// method /status.json or the health check does not take, answered with an
|
// method /status.json or the health check does not take, answered with an
|
||||||
// empty body (https://git.eeqj.de/sneak/routewatch/issues/201).
|
// empty body (https://git.eeqj.de/sneak/routewatch/issues/201).
|
||||||
|
//
|
||||||
|
// The requests run in a synctest bubble, whose clock is fake, so a stall on
|
||||||
|
// the host does not use up the 2-second limit of the health check or the
|
||||||
|
// 4-second limit of the stats handlers.
|
||||||
func TestJSONAnswersCarryMeta(t *testing.T) {
|
func TestJSONAnswersCarryMeta(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
@@ -181,6 +186,10 @@ func TestJSONAnswersCarryMeta(t *testing.T) {
|
|||||||
s.router.Get(panicsPath, JSONValidationMiddleware(http.HandlerFunc(
|
s.router.Get(panicsPath, JSONValidationMiddleware(http.HandlerFunc(
|
||||||
func(http.ResponseWriter, *http.Request) { panic(errDiskIO) })).ServeHTTP)
|
func(http.ResponseWriter, *http.Request) { panic(errDiskIO) })).ServeHTTP)
|
||||||
|
|
||||||
|
failing := New(failingDB{}, newTestTable(),
|
||||||
|
streamer.New(logger.New(), metrics.New()), logger.New(), &config.Config{})
|
||||||
|
failing.resolver = &fakeResolver{}
|
||||||
|
|
||||||
tests := []struct {
|
tests := []struct {
|
||||||
method string
|
method string
|
||||||
path string
|
path string
|
||||||
@@ -216,42 +225,40 @@ func TestJSONAnswersCarryMeta(t *testing.T) {
|
|||||||
{http.MethodGet, panicsPath, http.StatusInternalServerError},
|
{http.MethodGet, panicsPath, http.StatusInternalServerError},
|
||||||
}
|
}
|
||||||
|
|
||||||
for _, tt := range tests {
|
synctest.Test(t, func(t *testing.T) {
|
||||||
checkAnswerCarriesMeta(t, s, httptest.NewRequestWithContext(
|
for _, tt := range tests {
|
||||||
t.Context(), tt.method, tt.path, nil), tt.wantCode)
|
checkAnswerCarriesMeta(t, s, httptest.NewRequestWithContext(
|
||||||
}
|
t.Context(), tt.method, tt.path, nil), tt.wantCode)
|
||||||
|
}
|
||||||
|
|
||||||
// A request whose time limit has already passed answers 504. The handlers
|
// A request whose time limit has already passed answers 504. The
|
||||||
// write the answers of the first five routes, and the JSON check on the
|
// handlers write the answers of the first five routes, and the JSON
|
||||||
// route writes the last.
|
// check on the route writes the last.
|
||||||
pastTimeLimit, cancel := context.WithDeadline(t.Context(), time.Now())
|
pastTimeLimit, cancel := context.WithDeadline(t.Context(), time.Now())
|
||||||
defer cancel()
|
defer cancel()
|
||||||
|
|
||||||
for _, path := range []string{
|
for _, path := range []string{
|
||||||
statusJSONPath, statsPath, "/ip/" + addrInTestPrefixV4,
|
statusJSONPath, statsPath, "/ip/" + addrInTestPrefixV4,
|
||||||
"/api/v1/ip/" + addrInTestPrefixV4, asJSONPath, writesNothingPath,
|
"/api/v1/ip/" + addrInTestPrefixV4, asJSONPath, writesNothingPath,
|
||||||
} {
|
} {
|
||||||
checkAnswerCarriesMeta(t, s, httptest.NewRequestWithContext(
|
checkAnswerCarriesMeta(t, s, httptest.NewRequestWithContext(
|
||||||
pastTimeLimit, http.MethodGet, path, nil), http.StatusGatewayTimeout)
|
pastTimeLimit, http.MethodGet, path, nil), http.StatusGatewayTimeout)
|
||||||
}
|
}
|
||||||
|
|
||||||
// With a store whose reads fail, the health check answers 503, and the
|
// With a store whose reads fail, the health check answers 503, and
|
||||||
// handlers of the other routes 500.
|
// the handlers of the other routes 500.
|
||||||
failing := New(failingDB{}, newTestTable(),
|
|
||||||
streamer.New(logger.New(), metrics.New()), logger.New(), &config.Config{})
|
|
||||||
failing.resolver = &fakeResolver{}
|
|
||||||
|
|
||||||
checkAnswerCarriesMeta(t, failing, httptest.NewRequestWithContext(
|
|
||||||
t.Context(), http.MethodGet, healthCheckPath, nil),
|
|
||||||
http.StatusServiceUnavailable)
|
|
||||||
|
|
||||||
for _, path := range []string{
|
|
||||||
statusJSONPath, statsPath, asJSONPath,
|
|
||||||
"/api/v1/ip/" + addrInTestPrefixV4, "/ip/" + addrInTestPrefixV4,
|
|
||||||
} {
|
|
||||||
checkAnswerCarriesMeta(t, failing, httptest.NewRequestWithContext(
|
checkAnswerCarriesMeta(t, failing, httptest.NewRequestWithContext(
|
||||||
t.Context(), http.MethodGet, path, nil), http.StatusInternalServerError)
|
t.Context(), http.MethodGet, healthCheckPath, nil),
|
||||||
}
|
http.StatusServiceUnavailable)
|
||||||
|
|
||||||
|
for _, path := range []string{
|
||||||
|
statusJSONPath, statsPath, asJSONPath,
|
||||||
|
"/api/v1/ip/" + addrInTestPrefixV4, "/ip/" + addrInTestPrefixV4,
|
||||||
|
} {
|
||||||
|
checkAnswerCarriesMeta(t, failing, httptest.NewRequestWithContext(
|
||||||
|
t.Context(), http.MethodGet, path, nil), http.StatusInternalServerError)
|
||||||
|
}
|
||||||
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
// checkAnswerCarriesMeta serves req through the server's router, and checks
|
// checkAnswerCarriesMeta serves req through the server's router, and checks
|
||||||
|
|||||||
@@ -5,6 +5,7 @@ import (
|
|||||||
"net/http"
|
"net/http"
|
||||||
"net/http/httptest"
|
"net/http/httptest"
|
||||||
"testing"
|
"testing"
|
||||||
|
"testing/synctest"
|
||||||
)
|
)
|
||||||
|
|
||||||
// TestRoutesAnswerHEAD sends HEAD for a page and JSON routes through the
|
// TestRoutesAnswerHEAD sends HEAD for a page and JSON routes through the
|
||||||
@@ -12,15 +13,23 @@ import (
|
|||||||
// lookup of the caller's own address, are in the list because chi's GetHead
|
// lookup of the caller's own address, are in the list because chi's GetHead
|
||||||
// does not reach them. Before the fix every route answered HEAD with 405
|
// does not reach them. Before the fix every route answered HEAD with 405
|
||||||
// (https://git.eeqj.de/sneak/routewatch/issues/111).
|
// (https://git.eeqj.de/sneak/routewatch/issues/111).
|
||||||
|
//
|
||||||
|
// The requests run in a synctest bubble, whose clock is fake, so a stall on
|
||||||
|
// the host does not use up the stats handlers' 4-second limit. The resolver
|
||||||
|
// is fake because no time limit would cut off a real DNS lookup in a bubble.
|
||||||
func TestRoutesAnswerHEAD(t *testing.T) {
|
func TestRoutesAnswerHEAD(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
s := newServerWithTestRoutes(t)
|
s := newServerWithTestRoutes(t)
|
||||||
|
s.resolver = &fakeResolver{}
|
||||||
|
|
||||||
for _, path := range []string{"/", statusJSONPath, statsPath, "/ip", "/ip/"} {
|
for _, path := range []string{"/", statusJSONPath, statsPath, "/ip", "/ip/"} {
|
||||||
rec := httptest.NewRecorder()
|
rec := httptest.NewRecorder()
|
||||||
s.router.ServeHTTP(rec, httptest.NewRequestWithContext(
|
|
||||||
t.Context(), http.MethodHead, path, nil))
|
synctest.Test(t, func(t *testing.T) {
|
||||||
|
s.router.ServeHTTP(rec, httptest.NewRequestWithContext(
|
||||||
|
t.Context(), http.MethodHead, path, nil))
|
||||||
|
})
|
||||||
|
|
||||||
if rec.Code != http.StatusOK {
|
if rec.Code != http.StatusOK {
|
||||||
t.Errorf("HEAD %s: status %d, want %d; body %s",
|
t.Errorf("HEAD %s: status %d, want %d; body %s",
|
||||||
|
|||||||
+10
-10
@@ -22,9 +22,9 @@ import (
|
|||||||
"sync/atomic"
|
"sync/atomic"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
"git.eeqj.de/sneak/routewatch/internal/blocklist"
|
|
||||||
"git.eeqj.de/sneak/routewatch/internal/config"
|
"git.eeqj.de/sneak/routewatch/internal/config"
|
||||||
"git.eeqj.de/sneak/routewatch/internal/database"
|
"git.eeqj.de/sneak/routewatch/internal/database"
|
||||||
|
"git.eeqj.de/sneak/routewatch/internal/iplist"
|
||||||
"git.eeqj.de/sneak/routewatch/internal/logger"
|
"git.eeqj.de/sneak/routewatch/internal/logger"
|
||||||
"git.eeqj.de/sneak/routewatch/internal/routetable"
|
"git.eeqj.de/sneak/routewatch/internal/routetable"
|
||||||
"git.eeqj.de/sneak/routewatch/internal/streamer"
|
"git.eeqj.de/sneak/routewatch/internal/streamer"
|
||||||
@@ -45,10 +45,10 @@ type ASNFetcher interface {
|
|||||||
GetStats() ASNFetcherStats
|
GetStats() ASNFetcherStats
|
||||||
}
|
}
|
||||||
|
|
||||||
// Blocklists tells which public blocklists list an address, from copies it
|
// IPLists tells which public IP lists contain an address, from copies it keeps
|
||||||
// keeps in memory, as *blocklist.Blocklists does.
|
// in memory, as *iplist.IPLists does.
|
||||||
type Blocklists interface {
|
type IPLists interface {
|
||||||
Lookup(addr netip.Addr) []blocklist.Listing
|
Lookup(addr netip.Addr) []iplist.Listing
|
||||||
ShortestInterval() time.Duration
|
ShortestInterval() time.Duration
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -69,7 +69,7 @@ type Server struct {
|
|||||||
port int
|
port int
|
||||||
srv *http.Server
|
srv *http.Server
|
||||||
asnFetcher ASNFetcher
|
asnFetcher ASNFetcher
|
||||||
blocklists Blocklists
|
ipLists IPLists
|
||||||
|
|
||||||
// trustedProxies are the reverse proxies whose X-Forwarded-For header
|
// trustedProxies are the reverse proxies whose X-Forwarded-For header
|
||||||
// RealIPMiddleware believes
|
// RealIPMiddleware believes
|
||||||
@@ -202,8 +202,8 @@ func (s *Server) SetASNFetcher(fetcher ASNFetcher) {
|
|||||||
s.asnFetcher = fetcher
|
s.asnFetcher = fetcher
|
||||||
}
|
}
|
||||||
|
|
||||||
// SetBlocklists sets the blocklists the IP lookups report. Call it before
|
// SetIPLists sets the IP lists the IP lookups report. Call it before Start:
|
||||||
// Start: the handlers read the blocklists without a lock.
|
// the handlers read the IP lists without a lock.
|
||||||
func (s *Server) SetBlocklists(blocklists Blocklists) {
|
func (s *Server) SetIPLists(ipLists IPLists) {
|
||||||
s.blocklists = blocklists
|
s.ipLists = ipLists
|
||||||
}
|
}
|
||||||
@@ -22,10 +22,10 @@ func TestStopClosesTheConnectionOfARequestStillRunning(t *testing.T) {
|
|||||||
stopDeadline = 100 * time.Millisecond
|
stopDeadline = 100 * time.Millisecond
|
||||||
// waitTimeout is how long the test waits for the request to reach
|
// waitTimeout is how long the test waits for the request to reach
|
||||||
// the handler, and then for the client to see its connection closed.
|
// the handler, and then for the client to see its connection closed.
|
||||||
waitTimeout = 2 * time.Second
|
waitTimeout = 15 * time.Second
|
||||||
// readHeaderTimeout bounds the test server's read of the request
|
// readHeaderTimeout bounds the test server's read of the request
|
||||||
// header, as every http.Server must.
|
// header, as every http.Server must.
|
||||||
readHeaderTimeout = time.Second
|
readHeaderTimeout = waitTimeout
|
||||||
)
|
)
|
||||||
|
|
||||||
release := make(chan struct{})
|
release := make(chan struct{})
|
||||||
|
|||||||
@@ -14,6 +14,7 @@ import (
|
|||||||
"strings"
|
"strings"
|
||||||
"sync/atomic"
|
"sync/atomic"
|
||||||
"testing"
|
"testing"
|
||||||
|
"testing/synctest"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
"git.eeqj.de/sneak/routewatch/internal/logger"
|
"git.eeqj.de/sneak/routewatch/internal/logger"
|
||||||
@@ -398,7 +399,7 @@ func TestStopQueuesTheHeldStateMessages(t *testing.T) {
|
|||||||
const (
|
const (
|
||||||
held = 2
|
held = 2
|
||||||
lines = held + 1
|
lines = held + 1
|
||||||
maxWait = 5 * time.Second
|
maxWait = 15 * time.Second
|
||||||
)
|
)
|
||||||
|
|
||||||
srv := httptest.NewServer(http.HandlerFunc(
|
srv := httptest.NewServer(http.HandlerFunc(
|
||||||
@@ -478,15 +479,15 @@ func TestStopQueuesTheHeldStateMessages(t *testing.T) {
|
|||||||
// TestStopReturnsByItsDeadline stops a streamer whose handler has a full queue
|
// TestStopReturnsByItsDeadline stops a streamer whose handler has a full queue
|
||||||
// and a STATE message held for it, and a worker that stays busy until the test
|
// and a STATE message held for it, and a worker that stays busy until the test
|
||||||
// ends, as one does whose handler waits for a database write. Stop must return
|
// ends, as one does whose handler waits for a database write. Stop must return
|
||||||
// within its deadline plus a small margin instead of waiting for room on the
|
// within its deadline plus a margin instead of waiting for room on the queue
|
||||||
// queue and for the worker, and log as not handed over the held message and
|
// and for the worker, and log as not handed over the held message and every
|
||||||
// every message on the queue.
|
// message on the queue.
|
||||||
func TestStopReturnsByItsDeadline(t *testing.T) {
|
func TestStopReturnsByItsDeadline(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
const (
|
const (
|
||||||
deadline = 100 * time.Millisecond
|
deadline = 100 * time.Millisecond
|
||||||
margin = 2 * time.Second
|
margin = 15 * time.Second
|
||||||
)
|
)
|
||||||
|
|
||||||
var logs bytes.Buffer
|
var logs bytes.Buffer
|
||||||
@@ -610,10 +611,34 @@ func TestReconnectWhenServerNeverAnswers(t *testing.T) {
|
|||||||
checkReconnect(t, neverAnswer, idleTimeout)
|
checkReconnect(t, neverAnswer, idleTimeout)
|
||||||
}
|
}
|
||||||
|
|
||||||
// TestStreamNotDroppedWhileLinesArrive has the server send lines well within
|
// pipeTransport answers every request with a response whose body write fills.
|
||||||
// the idle timeout for several times the timeout, then end the response. Each
|
// The body fails when the request's context ends, as a connection's does.
|
||||||
// line must start the idle timeout over, so the stream lasts until the server
|
// write gets that context, so that it can end when the request does.
|
||||||
// ends it instead of being dropped as idle.
|
type pipeTransport struct {
|
||||||
|
write func(context.Context, io.Writer)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (p pipeTransport) RoundTrip(req *http.Request) (*http.Response, error) {
|
||||||
|
body, w := io.Pipe()
|
||||||
|
|
||||||
|
context.AfterFunc(req.Context(), func() {
|
||||||
|
_ = w.CloseWithError(req.Context().Err())
|
||||||
|
})
|
||||||
|
|
||||||
|
go func(ctx context.Context) {
|
||||||
|
p.write(ctx, w)
|
||||||
|
|
||||||
|
_ = w.Close()
|
||||||
|
}(req.Context())
|
||||||
|
|
||||||
|
return &http.Response{StatusCode: http.StatusOK, Body: body}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestStreamNotDroppedWhileLinesArrive has a response send lines well within
|
||||||
|
// the idle timeout for several times the timeout, then end. Each line must
|
||||||
|
// start the idle timeout over, so the stream lasts until the response ends
|
||||||
|
// instead of being dropped as idle. It runs in a synctest bubble, whose clock
|
||||||
|
// is fake, so a stall on the host does not use up the timeout.
|
||||||
func TestStreamNotDroppedWhileLinesArrive(t *testing.T) {
|
func TestStreamNotDroppedWhileLinesArrive(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
@@ -623,26 +648,35 @@ func TestStreamNotDroppedWhileLinesArrive(t *testing.T) {
|
|||||||
sendFor = 5 * idleTimeout
|
sendFor = 5 * idleTimeout
|
||||||
)
|
)
|
||||||
|
|
||||||
srv := httptest.NewServer(http.HandlerFunc(
|
// The streamer is made outside the bubble. Its metrics start a goroutine
|
||||||
func(w http.ResponseWriter, _ *http.Request) {
|
// that never returns, and a bubble waits for each of its goroutines to
|
||||||
|
// return.
|
||||||
|
s := New(logger.New(), metrics.New())
|
||||||
|
s.idleTimeout = idleTimeout
|
||||||
|
s.client = &http.Client{Transport: pipeTransport{
|
||||||
|
write: func(ctx context.Context, w io.Writer) {
|
||||||
deadline := time.Now().Add(sendFor)
|
deadline := time.Now().Add(sendFor)
|
||||||
for time.Now().Before(deadline) {
|
for time.Now().Before(deadline) {
|
||||||
_, _ = io.WriteString(w, updateLine)
|
_, _ = io.WriteString(w, updateLine)
|
||||||
|
|
||||||
flush(w)
|
// The wait ends with the stream. A bubble's clock stops when
|
||||||
time.Sleep(lineInterval)
|
// the test function returns, and a goroutine then still
|
||||||
|
// asleep panics the test binary.
|
||||||
|
select {
|
||||||
|
case <-ctx.Done():
|
||||||
|
return
|
||||||
|
case <-time.After(lineInterval):
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}))
|
},
|
||||||
defer srv.Close()
|
}}
|
||||||
|
|
||||||
s := New(logger.New(), metrics.New())
|
synctest.Test(t, func(t *testing.T) {
|
||||||
s.url = srv.URL
|
err := s.stream(context.Background())
|
||||||
s.idleTimeout = idleTimeout
|
if !errors.Is(err, errStreamEnded) {
|
||||||
|
t.Fatalf("stream returned %v, want %v", err, errStreamEnded)
|
||||||
err := s.stream(context.Background())
|
}
|
||||||
if !errors.Is(err, errStreamEnded) {
|
})
|
||||||
t.Fatalf("stream returned %v, want %v", err, errStreamEnded)
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// TestStreamReadsLongLine has the server send one UPDATE line of about
|
// TestStreamReadsLongLine has the server send one UPDATE line of about
|
||||||
@@ -706,7 +740,7 @@ func TestHandlerStatsReadWhileMessagesAreDropped(t *testing.T) {
|
|||||||
const (
|
const (
|
||||||
lines = 5
|
lines = 5
|
||||||
wantDropped = lines - 2
|
wantDropped = lines - 2
|
||||||
maxWait = 5 * time.Second
|
maxWait = 15 * time.Second
|
||||||
)
|
)
|
||||||
|
|
||||||
srv := httptest.NewServer(http.HandlerFunc(
|
srv := httptest.NewServer(http.HandlerFunc(
|
||||||
@@ -825,7 +859,7 @@ func checkReconnect(
|
|||||||
) {
|
) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
const maxWait = 5 * time.Second
|
const maxWait = 15 * time.Second
|
||||||
|
|
||||||
tracker := metrics.New()
|
tracker := metrics.New()
|
||||||
connectedAtReconnect := make(chan bool, 1)
|
connectedAtReconnect := make(chan bool, 1)
|
||||||
@@ -896,9 +930,13 @@ func settledGoroutineCount() int {
|
|||||||
return prev
|
return prev
|
||||||
}
|
}
|
||||||
|
|
||||||
// waitForGoroutines waits until the goroutine count drops to target or below.
|
// waitForGoroutines waits up to 15 seconds for the goroutine count to drop to
|
||||||
|
// target or below.
|
||||||
func waitForGoroutines(target int) bool {
|
func waitForGoroutines(target int) bool {
|
||||||
for range 100 {
|
const maxWait = 15 * time.Second
|
||||||
|
|
||||||
|
deadline := time.Now().Add(maxWait)
|
||||||
|
for time.Now().Before(deadline) {
|
||||||
if runtime.NumGoroutine() <= target {
|
if runtime.NumGoroutine() <= target {
|
||||||
return true
|
return true
|
||||||
}
|
}
|
||||||
|
|||||||
Reference in new issue
Block a user