diff --git a/.dockerignore b/.dockerignore index 89e43ce..875a9f5 100644 --- a/.dockerignore +++ b/.dockerignore @@ -1,32 +1,107 @@ -# Docker does not read .gitignore, and a pattern here matches from the root of -# the build context only: a pattern meant for every directory needs `**/`. +# .dockerignore does NOT use .gitignore semantics. Docker matches with +# moby/patternmatcher: filepath.Match plus `**`, so `*` does not cross +# `/` and an unprefixed pattern is anchored at the context root. Every +# depth-independent pattern therefore needs `**/`, or `config/.env` and +# `certs/server.key` still ship while this file reads as solved. Only +# genuinely root-anchored entries go unprefixed. Never transplant these +# into .gitignore, where `**/` is wrong. +# +# Matching is case-sensitive, so secrets use character ranges rather +# than an ALL-CAPS twin, which would still miss `Server.Key`. +# +# Extend with this repo's own host-built artifacts, written anchored: +# `/myapp`, never `**/myapp`, which also matches `cmd/myapp/` and +# deletes the package directory from the context. # .git is sent without its config. Without a VERSION build argument the # stage that compiles runs `git describe --tags --always` on .git, which # does not need .git/config; that file can hold a credential, such as a # password in a remote URL or the token the CI checkout step stores there. -.git/config +# Each submodule keeps a config with the same exposure in its git directory +# under .git/modules/, nested again for a submodule's own submodules, or in +# its own .git directory when it keeps one. +# KNOWN GAP: a submodule whose name has a `config` segment (`config`, +# `deploy/config`, `config/lib`) loses its whole git directory, because +# `**/.git/modules/**/config` also matches that segment's directory +# under .git/modules/. Go's version stamping then fails the build; +# nothing leaks. Name such a submodule without that segment: +# `git submodule add --name`. +**/.git/config +**/.git/modules/**/config -# Local build and debug output: `make build`, `make run`, `make asupdate`, test -# binaries, coverage profiles and source archives. +# Agent scratch: one full checkout of the repo per in-flight agent. +# Anchored because it occurs once where agents run at the repo root. +# KNOWN GAP: a repo running agents in subdirectories still ships +# `services/api/.claude/` and must add its own anchored entry. +.claude + +# Environment files. `*.env` covers bare `.env` and the `prod.env` +# convention. Re-include a committed template with a negation if the +# build needs one: `!docs/example.env`. +**/*.[eE][nN][vV] +**/.[eE][nN][vV].* +**/.[eE][nN][vV][rR][cC] + +# Private keys and the bundles carrying them. Public certificates +# (*.crt, *.cer) are deliberately absent: they are legitimate inputs. +**/*.[pP][eE][mM] +**/*.[kK][eE][yY] +**/*.[pP]12 +**/*.[pP][fF][xX] +**/[iI][dD]_[rR][sS][aA] +**/[iI][dD]_[dD][sS][aA] +**/[iI][dD]_[eE][cC][dD][sS][aA] +**/[iI][dD]_[eE][cC][dD][sS][aA]_[sS][kK] +**/[iI][dD]_[eE][dD]25519 +**/[iI][dD]_[eE][dD]25519_[sS][kK] + +# Dependencies: restored inside the image, never copied in. +**/node_modules + +# OS metadata. +**/.DS_Store +**/Thumbs.db + +# Editor state: never a build input, and it churns COPY. +**/*.swp +**/*.swo +**/*~ +**/*.bak +**/.idea +**/.vscode +**/*.sublime-* + +# This repository's host-built artifacts: `make build` writes bin/, `go +# build` in the root the binaries, `make run` log.txt, `make asupdate` its +# .tmp file; logs, test binaries, coverage profiles, source archives, and +# the binaries and libraries a build for another platform writes. /bin +/routewatch +/asinfo-gen +/streamdumper /log.txt /out /pkg/asinfo/asdata.json **/*.tar.zst +**/*.log **/*.test **/*.out **/*.tmp +**/*.exe +**/*.dll +**/*.so +**/*.dylib -# Local databases and secrets. The image carries a source archive of the whole -# build context, so these would otherwise ship inside it. +# Local databases. The image carries a source archive of the whole build +# context, so these would otherwise ship inside it. The daemon's database +# is db.sqlite, with its -wal and -shm files while it runs. **/*.db **/*.db-journal **/*.db-wal -**/.env +**/*.sqlite +**/*.sqlite-wal +**/*.sqlite-shm # A local Go workspace points at directories outside the build context. /go.work /go.work.sum - -**/.DS_Store diff --git a/.editorconfig b/.editorconfig index 2fe0ce0..de84fae 100644 --- a/.editorconfig +++ b/.editorconfig @@ -10,3 +10,9 @@ insert_final_newline = true [Makefile] indent_style = tab + +[*.go] +indent_style = tab + +# This repository's own sections, such as one for another language it +# uses, go below this comment, and a re-vendor keeps them. diff --git a/.gitea/workflows/check.yml b/.gitea/workflows/check.yml index a55bc55..2e96965 100644 --- a/.gitea/workflows/check.yml +++ b/.gitea/workflows/check.yml @@ -1,9 +1,20 @@ name: check on: [push] +# Free the shared runner: a new push cancels only the same branch's older run. +concurrency: + group: ${{ github.workflow }}-${{ github.ref }} + cancel-in-progress: true jobs: check: runs-on: ubuntu-latest + # Free the shared runner from a hung build. + timeout-minutes: 20 steps: - # actions/checkout v4.2.2, 2026-02-28 + # actions/checkout v4.2.2, 2026-02-22 - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 + # script/cibuild needs no token, so none is left in .git/config. + with: + persist-credentials: false + # All history and tags, so git describe finds the version tag. + fetch-depth: 0 - run: script/cibuild diff --git a/.gitignore b/.gitignore index 4951af1..b42a480 100644 --- a/.gitignore +++ b/.gitignore @@ -1,41 +1,83 @@ -# Binaries for programs and plugins +# OS +.DS_Store +Thumbs.db + +# Editors +*.swp +*.swo +*~ +*.bak +.idea/ +.vscode/ +*.sublime-* + +# Agent scratch (worktrees of this repo, created and destroyed by +# in-flight tooling). Unanchored: .gitignore patterns already match at +# every depth, so no prefix is wanted here. This is not a .dockerignore +# entry and must not be given a `**/` prefix on the way into one. +.claude/ + +# Node +node_modules/ + +# Secrets. Unanchored like every entry above, so each matches at every +# depth. Matching is case-sensitive on Linux, so names use character +# ranges rather than a lowercase form that misses `Server.Key`. + +# Environment files. `*.env` covers bare `.env` and the `prod.env` +# convention. Only the templates `example.env` and `sample.env` are +# re-included below. A repository that commits any other template adds +# its own negation at the end of this file, for example `!.env.example`. +*.[eE][nN][vV] +.[eE][nN][vV].* +.[eE][nN][vV][rR][cC] +!example.env +!sample.env + +# Private keys and the bundles carrying them. +*.[pP][eE][mM] +*.[kK][eE][yY] +*.[pP]12 +*.[pP][fF][xX] +[iI][dD]_[rR][sS][aA] +[iI][dD]_[dD][sS][aA] +[iI][dD]_[eE][cC][dD][sS][aA] +[iI][dD]_[eE][cC][dD][sS][aA]_[sS][kK] +[iI][dD]_[eE][dD]25519 +[iI][dD]_[eE][dD]25519_[sS][kK] + +# This repository's own entries, such as its build outputs, go below +# this comment, and a re-vendor keeps them. Anchor a binary built at the +# root: `/myapp`, never `myapp`, which also ignores `cmd/myapp/`. + +# Go: logs, test binaries, coverage profiles, and the binaries `make build` +# and `go build` in the repository root write. +*.log +*.out +*.test *.exe *.exe~ *.dll *.so *.dylib /bin/ - -.DS_Store -log.txt - -# Test binary, built with `go test -c` -*.test - -# Output of the go coverage tool, specifically when used with LiteIDE -*.out - -# Dependency directories (remove the comment below to include it) -# vendor/ - -# Go workspace file +/routewatch +/asinfo-gen +/streamdumper go.work go.work.sum -# env file -.env - -# Database files +# Local databases, temporary files and debug output. The daemon's +# database is db.sqlite, with its -wal and -shm files while it runs. *.db *.db-journal *.db-wal - -# Temporary files +*.sqlite +*.sqlite-wal +*.sqlite-shm *.tmp - -# Raw AS data (we only commit the gzipped version) -pkg/asinfo/asdata.json - -# Debug output files -out log.txt +out + +# Raw AS data (only the gzipped version is committed). +pkg/asinfo/asdata.json diff --git a/.golangci.yml b/.golangci.yml index 9d7f245..32db9fe 100644 --- a/.golangci.yml +++ b/.golangci.yml @@ -1,92 +1,101 @@ version: "2" +# Config schema uses the golangci-lint v2 layout (settings live under +# linters.settings, not top-level linters-settings) so that the +# thresholds below are actually applied by golangci-lint >= v2. + run: - go: "1.24" - tests: false + timeout: 5m + modules-download-mode: readonly linters: + default: all enable: - # Additional linters requested - - testifylint # Checks usage of github.com/stretchr/testify - - usetesting # usetesting is an analyzer that detects using os.Setenv instead of t.Setenv since Go 1.17 - # - tagliatelle # Disabled: we need snake_case for external API compatibility - - nlreturn # nlreturn checks for a new line before return and branch statements - - nilnil # Checks that there is no simultaneous return of nil error and an invalid value - - nestif # Reports deeply nested if statements - - mnd # An analyzer to detect magic numbers - - lll # Reports long lines - - intrange # intrange is a linter to find places where for loops could make use of an integer range - - gochecknoglobals # Check that no global variables exist - - # Default/existing linters that are commonly useful - - govet - - errcheck - - staticcheck - - unused - - ineffassign - - misspell - - revive - - gosec - - unconvert - - unparam - -linters-settings: - lll: - line-length: 120 - - nestif: - min-complexity: 4 - - nlreturn: - block-size: 2 - - revive: - rules: - - name: var-naming - arguments: - - [] - - [] - - "upperCaseConst=true" - - tagliatelle: - case: + # Successor to the deprecated gomodguard. Named explicitly, rather than + # left to `default: all`, because it carries the module policy below. + - gomodguard_v2 + disable: + # Genuinely incompatible with project patterns + - exhaustruct # Requires all struct fields + - exhaustruct_v5 # Requires all struct fields (successor to exhaustruct) + - godot # Requires comments to end with periods + - wrapcheck # Too verbose for internal packages + - varnamelen # Short names like db, id are idiomatic Go + # Deprecated: the warning is attached to the old name, so it is + # silenced by disabling that name, not by enabling the successor. + - wsl # Deprecated, replaced by wsl_v5 + - gomodguard # Deprecated, replaced by gomodguard_v2 + # Misses findings at random in v2.14.0; back once a pinned release fixes it + - canonicalheader + settings: + lll: + line-length: 88 + funlen: + lines: 80 + statements: 50 + cyclop: + max-complexity: 15 + dupl: + threshold: 100 + depguard: + # Test-support code must not be compiled into the shipped binary. A + # test-support package exists to hand a test privileges the program + # itself must never have, so a file that is not a test must not import + # one. Test files, and the files inside a package whose directory name + # ends in `test`, are where that code belongs, and are exempt. + # + # The deny list below is the one part of this file a repository is + # expected to extend, and the only part it may. depguard matches an + # import path against a list of prefixes, so it cannot be told "any path + # whose last segment ends in test"; a repository's own test-support + # packages have to be named here one at a time, by full import path, + # under a module path that differs from repository to repository. Add + # them; change nothing else. rules: - json: snake - yaml: snake - xml: snake - bson: snake - - testifylint: - enable-all: true - - usetesting: {} + test-support: + list-mode: lax + files: + - "$all" + - "!$test" + - "!**/*test/**" + deny: + - pkg: net/http/httptest + desc: >- + Test-support code belongs in test files and in packages whose + directory name ends in test, not in the shipped binary. + # Only decisions already recorded in the Go package defaults are + # listed here. Every entry matches the module path exactly. + gomodguard_v2: + blocked: + - module: github.com/rs/zerolog + recommendations: + - log/slog + reason: "Structured logging is stdlib log/slog." + # One entry per pre-fork module path, because the later releases + # are separate paths. A prefix match would be shorter but would + # also reach github.com/go-redis/redismock, the test double for + # the successor these entries recommend. + - module: github.com/go-redis/redis + recommendations: + - github.com/redis/go-redis/v9 + reason: "Pre-fork module; use the maintained go-redis v9." + - module: github.com/go-redis/redis/v7 + recommendations: + - github.com/redis/go-redis/v9 + reason: "Pre-fork module; use the maintained go-redis v9." + - module: github.com/go-redis/redis/v8 + recommendations: + - github.com/redis/go-redis/v9 + reason: "Pre-fork module; use the maintained go-redis v9." + - module: github.com/sergi/go-diff + recommendations: + - github.com/aymanbagabas/go-udiff + reason: "No unified diff output; use go-udiff." + - module: github.com/hexops/gotextdiff + recommendations: + - github.com/aymanbagabas/go-udiff + reason: "Unmaintained fork; use go-udiff." issues: max-issues-per-linter: 0 max-same-issues: 0 - exclude-rules: - # Exclude unused parameter warnings for cobra command signatures - - text: "parameter '(args|cmd)' seems to be unused" - linters: - - revive - - # Allow ALL_CAPS constant names - - text: "don't use ALL_CAPS in Go names" - linters: - - revive - - # Allow snake_case JSON tags for external API compatibility - - path: "internal/types/ris.go" - linters: - - tagliatelle - - # Allow snake_case JSON tags for database models - - path: "internal/database/models.go" - linters: - - tagliatelle - - # Allow generic package name for types that define data structures - - path: "internal/types/" - text: "avoid meaningless package names" - linters: - - revive diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..b7ebc67 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,95 @@ +# IMPORTANT RULES + +- Claude is an inanimate tool. The spam that Claude attempts to insert into + commit messages (which it erroneously refers to as "attribution") is not + attribution, as I am the sole author of code created using Claude. It is + corporate advertising for Anthropic and is therefore completely unacceptable + in commit messages. + +- Tests should always be run before committing code. No commits should be made + that do not pass tests. + +- Code should always be formatted before committing. Do not commit unformatted + code. + +- Code should always be linted and linter errors fixed before committing. NEVER + commit code that does not pass the linter. DO NOT modify the linter config + unless specifically instructed. + +- The test suite is fast and local. When running tests, NEVER run individual + parts of the test suite, always run the whole thing by running "make test". + +- Do not stop working on a task until you have reached the definition of done + provided to you in the initial instruction. Don't do part or most of the work, + do all of the work until the criteria for done are met. + +- When you complete each task, if the tests are passing and the code is + formatted and there are no linter errors, always commit and push your work. + Use a good commit message and don't mention any author or co-author + attribution. + +- Do not create additional files in the root directory of the project without + asking permission first. Configuration files, documentation, and build files + are acceptable in the root, but source code and other files should be + organized in appropriate subdirectories. + +- Do not use bare strings or numbers in code, especially if they appear anywhere + more than once. Always define a constant (usually at the top of the file) and + give it a descriptive name, then use that constant in the code instead of the + bare string or number. + +- If you are fixing a bug, write a test first that reproduces the bug and fails, + and then fix the bug in the code, using the test to verify that the fix + worked. + +- When implementing new features, be aware of potential side-effects (such as + state files on disk, data in the database, etc.) and ensure that it is + possible to mock or stub these side-effects in tests when designing an API. + +- When dealing with dates and times or timestamps, always use, display, and + store UTC. Set the local timezone to UTC on startup. If the user needs to see + the time in a different timezone, store the user's timezone in a separate + field and convert the UTC time to the user's timezone when displaying it. For + internal use and internal applications and administrative purposes, always + display UTC. + +- When implementing programs, put the main.go in ./cmd//main.go + and put the program's code in ./internal//. This allows for + multiple programs to be implemented in the same repository without cluttering + the root directory. main.go should simply import and call + .CLIEntry(). The full implementation should be in + ./internal//. + +- When you are instructed to make the tests pass, DO NOT delete tests, skip + tests, or change the tests specifically to make them pass (unless there is a + bug in the test). This is cheating, and it is bad. You should only be + modifying the test if it is incorrect or if the test is no longer relevant. In + almost all cases, you should be fixing the code that is being tested, or + updating the tests to match a refactored implementation. + +- Always write a `Makefile` with the default target being `test`, and with a + `fmt` target that formats the code. The `test` target should run all tests in + the project, and the `fmt` target should format the code. `test` should also + have a prerequisite target `lint` that should run any linters that are + configured for the project. + +- After each completed bugfix or feature, the code must be committed. Do all of + the pre-commit checks (test, lint, fmt) before committing, of course. After + each commit, push to the remote. + +- Always write tests, even if they are extremely simple and just check for + correct syntax (ability to compile/import). If you are writing a new feature, + write a test for it. You don't need to target complete coverage, but you + should at least test any new functionality you add. + +- Always use structured logging. Log any relevant state/context with the + messages (but do not log secrets). If stdout is not a terminal, output the + structured logs in jsonl format. Use go's log/slog. + +- You do not need to summarize your changes in the chat after making them. + Making the changes and committing them is sufficient. If anything out of the + ordinary happened, please explain it, but in the normal case where you found + and fixed the bug, or implemented the feature, there is no need for the + end-of-change summary. + +- When testing daemons, use a 15 second timeout always. diff --git a/CLAUDE.md b/CLAUDE.md deleted file mode 100644 index 3124a8a..0000000 --- a/CLAUDE.md +++ /dev/null @@ -1,97 +0,0 @@ -# IMPORTANT RULES - -* Claude is an inanimate tool. The spam that Claude attempts to insert into - commit messages (which it erroneously refers to as "attribution") is not - attribution, as I am the sole author of code created using Claude. It is - corporate advertising for Anthropic and is therefore completely - unacceptable in commit messages. - -* Tests should always be run before committing code. No commits should be - made that do not pass tests. - -* Code should always be formatted before committing. Do not commit - unformatted code. - -* Code should always be linted and linter errors fixed before committing. - NEVER commit code that does not pass the linter. DO NOT modify the linter - config unless specifically instructed. - -* The test suite is fast and local. When running tests, NEVER run - individual parts of the test suite, always run the whole thing by running - "make test". - -* Do not stop working on a task until you have reached the definition of - done provided to you in the initial instruction. Don't do part or most of - the work, do all of the work until the criteria for done are met. - -* When you complete each task, if the tests are passing and the code is - formatted and there are no linter errors, always commit and push your - work. Use a good commit message and don't mention any author or co-author - attribution. - -* Do not create additional files in the root directory of the project - without asking permission first. Configuration files, documentation, and - build files are acceptable in the root, but source code and other files - should be organized in appropriate subdirectories. - -* Do not use bare strings or numbers in code, especially if they appear - anywhere more than once. Always define a constant (usually at the top of - the file) and give it a descriptive name, then use that constant in the - code instead of the bare string or number. - -* If you are fixing a bug, write a test first that reproduces the bug and - fails, and then fix the bug in the code, using the test to verify that the - fix worked. - -* When implementing new features, be aware of potential side-effects (such - as state files on disk, data in the database, etc.) and ensure that it is - possible to mock or stub these side-effects in tests when designing an - API. - -* When dealing with dates and times or timestamps, always use, display, and - store UTC. Set the local timezone to UTC on startup. If the user needs - to see the time in a different timezone, store the user's timezone in a - separate field and convert the UTC time to the user's timezone when - displaying it. For internal use and internal applications and - administrative purposes, always display UTC. - -* When implementing programs, put the main.go in - ./cmd//main.go and put the program's code in - ./internal//. This allows for multiple programs to be - implemented in the same repository without cluttering the root directory. - main.go should simply import and call .CLIEntry(). The - full implementation should be in ./internal//. - -* When you are instructed to make the tests pass, DO NOT delete tests, skip - tests, or change the tests specifically to make them pass (unless there - is a bug in the test). This is cheating, and it is bad. You should only - be modifying the test if it is incorrect or if the test is no longer - relevant. In almost all cases, you should be fixing the code that is - being tested, or updating the tests to match a refactored implementation. - -* Always write a `Makefile` with the default target being `test`, and with a - `fmt` target that formats the code. The `test` target should run all - tests in the project, and the `fmt` target should format the code. `test` - should also have a prerequisite target `lint` that should run any linters - that are configured for the project. - -* After each completed bugfix or feature, the code must be committed. Do - all of the pre-commit checks (test, lint, fmt) before committing, of - course. After each commit, push to the remote. - -* Always write tests, even if they are extremely simple and just check for - correct syntax (ability to compile/import). If you are writing a new - feature, write a test for it. You don't need to target complete coverage, - but you should at least test any new functionality you add. - -* Always use structured logging. Log any relevant state/context with the - messages (but do not log secrets). If stdout is not a terminal, output - the structured logs in jsonl format. Use go's log/slog. - -* You do not need to summarize your changes in the chat after making them. - Making the changes and committing them is sufficient. If anything out of - the ordinary happened, please explain it, but in the normal case where you - found and fixed the bug, or implemented the feature, there is no need for - the end-of-change summary. - -* When testing daemons, use a 15 second timeout always. diff --git a/Dockerfile b/Dockerfile index dfd9d5a..f2f9d83 100644 --- a/Dockerfile +++ b/Dockerfile @@ -1,22 +1,46 @@ -# Lint stage — fast feedback on formatting and lint issues. -# The golangci-lint image bundles Go, gcc and make, so it can run go vet on -# the CGO sqlite package and golangci-lint without extra installs. -# golangci/golangci-lint:v2.7.2 (Go 1.25.5), 2026-09-21 -FROM golangci/golangci-lint@sha256:5d6d5c70a61f1356adfd9dd6316ce286799fefc9d743421356ff1b00842368ba AS lint +# In every stage ARG CACHEBUST comes right after the steps that install +# system packages and download Go's modules, or right after FROM in a +# stage with neither, and every step that lints, tests or compiles +# comes after it. script/ passes a new value on every build, which +# reruns every RUN step after it. Docker keeps the layers above it +# until the base image, their own lines or go.mod and go.sum change. +# The steps that compile mount Go's build cache, and golangci-lint run +# also mounts golangci-lint's; both survive from build to build. +# Lint phase. The image is Debian-based and has gcc, so it can check the +# cgo SQLite package without extra installs. +# golangci/golangci-lint:v2.14.0 (Go 1.27.0), 2026-10-19 +FROM golangci/golangci-lint@sha256:ad862ba6b3798cbe0fd9fd7408d498fd74fbd2623a92406b2fd3898faf0bf98f AS lint WORKDIR /src - COPY go.mod go.sum ./ RUN go mod download - +ARG CACHEBUST COPY . . +RUN --mount=type=cache,target=/root/.cache/go-build \ + --mount=type=cache,target=/root/.cache/golangci-lint \ + golangci-lint run --config .golangci.yml ./... -RUN make fmt-check -RUN make lint +# Test phase. -race needs cgo and so a C compiler, which the Debian Go +# image ships and the alpine one does not. -count=1 because the mounted +# cache also holds test results, which go test would otherwise replay. +# -short skips the live RIS Live feed test, so the suite needs no network. +# golang:1.27.2-bookworm, 2026-10-19 +FROM golang@sha256:5cf287a799e6b94384bad13d16b14904c531f51ba65792237e122ce42b392f61 AS test +WORKDIR /src +COPY go.mod go.sum ./ +RUN go mod download +ARG CACHEBUST +COPY . . +RUN --mount=type=cache,target=/root/.cache/go-build \ + go test -short -count=1 -timeout 90s -race -cover ./... || \ + { echo "--- Rerunning with -v for details ---"; \ + go test -short -count=1 -timeout 90s -race -v ./...; exit 1; } -# Build stage -# golang:1.24-bookworm, 2026-09-21 -FROM golang@sha256:1a6d4452c65dea36aac2e2d606b01b4a029ec90cc1ae53890540ce6173ea77ac AS builder +# Build stage. Nothing is wanted from either phase above; the copies +# are what make BuildKit build them first, so nothing after them can +# run unless lint and test passed. +# golang:1.27.2-bookworm, 2026-10-19 +FROM golang@sha256:5cf287a799e6b94384bad13d16b14904c531f51ba65792237e122ce42b392f61 AS builder # Install build dependencies (zstd for archive, gcc for CGO/sqlite3) RUN apt-get update && apt-get install -y --no-install-recommends \ @@ -26,36 +50,38 @@ RUN apt-get update && apt-get install -y --no-install-recommends \ && rm -rf /var/lib/apt/lists/* WORKDIR /src - -# Force BuildKit to run the lint stage before compiling or testing. +COPY go.mod go.sum ./ +RUN go mod download +ARG CACHEBUST COPY --from=lint /src/go.sum /dev/null - -# Copy everything +COPY --from=test /src/go.sum /dev/null +# A tar-stream context keeps the sender's file owners, which git refuses. +RUN git config --system --add safe.directory /src COPY . . -# Vendor dependencies (must be after copying source) -RUN go mod download && go mod vendor - -# Run the test suite in the build stage: -race needs cgo and the C compiler -# installed above. The suite is offline (the live-feed test is opt-in). -RUN make test +# Vendor dependencies for the source archive (must be after copying source) +RUN go mod vendor # Build the binary with CGO enabled (required for sqlite3). The version the -# page footer shows is the VERSION build argument when one is given, otherwise -# `git describe --tags --always` of the .git in the build context (git comes -# with this image): the tag on a tagged commit, tag-N-gHASH after one, the -# short commit when no tag is reachable. A context that carries .git and still -# yields no version fails the build. The footer links to the full commit. +# page footer shows is the VERSION build arg when one is given, otherwise +# `git describe --tags --always` on the .git in the build context: the tag +# on a tagged commit, tag-N-gHASH after one, the short commit when no tag is +# reachable. With .git present, a version that is still empty, dev or +# unknown fails the build: git is missing or could not read the checkout. +# Without .git the footer says unknown. The footer links to the full commit. ARG VERSION -RUN version="${VERSION:-$(git describe --tags --always || echo unknown)}"; \ - if [ -e .git ] && { [ -z "$version" ] || [ "$version" = dev ] || \ - [ "$version" = unknown ]; }; then \ - echo "no version could be derived although the build context carries .git" >&2; \ - exit 1; \ +RUN --mount=type=cache,target=/root/.cache/go-build \ + VERSION="${VERSION:-$(git describe --tags --always)}"; \ + if [ -e .git ]; then \ + case "$VERSION" in ""|dev|unknown) \ + echo "version is '$VERSION' although .git is present" >&2; \ + exit 1 ;; \ + esac; \ fi; \ + REVISION="$(git rev-parse --verify HEAD || echo unknown)"; \ CGO_ENABLED=1 GOOS=linux go build -o /routewatch -ldflags "\ - -X git.eeqj.de/sneak/routewatch/internal/version.GitRevision=$(git rev-parse --verify HEAD || echo unknown) \ - -X git.eeqj.de/sneak/routewatch/internal/version.GitRevisionShort=$version" \ + -X git.eeqj.de/sneak/routewatch/internal/version.GitRevision=$REVISION \ + -X git.eeqj.de/sneak/routewatch/internal/version.GitRevisionShort=${VERSION:-unknown}" \ ./cmd/routewatch # Create source archive with vendored dependencies @@ -64,7 +90,7 @@ RUN tar --zstd -cf /routewatch-source.tar.zst \ --exclude='*.tar.zst' \ . -# Runtime stage +# Runtime stage, and the last one # debian:bookworm-slim, 2026-09-21 FROM debian@sha256:3783cc01769c7b2b1b83a5c5ad96c815348e28ed7da68e2e3687004faa906251 @@ -76,6 +102,8 @@ RUN apt-get update && apt-get install -y --no-install-recommends \ curl \ && rm -rf /var/lib/apt/lists/* +ARG CACHEBUST + # Create non-root user RUN useradd -r -u 1000 -m routewatch diff --git a/README.md b/README.md index af8ac6a..7c3629f 100644 --- a/README.md +++ b/README.md @@ -5,6 +5,21 @@ table in real time: it streams BGP UPDATE messages from the RIPE RIS Live service, maintains a live routing table in SQLite, and provides HTTP APIs for querying routing information. +## Rationale + +RIPE RIS Live streams the BGP updates that RIPE's route collectors receive, as +they happen, from the moment a client connects. It reports changes, not the +routing table they add up to, so it cannot say which AS an IP address is routed +to now or which routes a prefix has. RouteWatch applies the feed's updates to +live routing tables in its own SQLite database and answers IP-to-AS lookups and +prefix pages from them. + +The feed sends no table when a client connects, so these tables hold only the +routes the feed has announced since RouteWatch first started: an IP-to-AS lookup +finds nothing for an address whose routes have not been announced since then. +The tables also miss the updates RouteWatch drops when it falls behind under +load and those sent while it is stopped, so a lookup or a prefix page can show a +route the feed has withdrawn. ## Features @@ -37,21 +52,81 @@ DEBUG=routewatch ./routewatch ## HTTP Endpoints +Every route answers `HEAD` as it answers `GET`, without the body. + ### Web Interface -- `GET /` - Redirects to /status -- `GET /status` - HTML status dashboard -- `GET /status.json` - JSON statistics -- `GET /as/{asn}` - AS detail page (HTML) -- `GET /prefix/{prefix}` - Prefix detail page (HTML) -- `GET /prefixlength/{length}` - IPv4 prefixes by mask length -- `GET /prefixlength6/{length}` - IPv6 prefixes by mask length -- `GET /ip/{ip}` - Redirects to prefix containing the IP + +- `GET /` - Home page (HTML): headline figures, and lookups by AS number and by + IP address +- `GET /status` - HTML status dashboard, which reads `/api/v1/stats` +- `GET /status.json` - JSON statistics: feed, memory, database and route figures +- `GET /.well-known/healthcheck.json` - Health check (JSON): 200 when the + database answers and the RIS Live feed is connected, otherwise 503; + `data.status` is `ok` or `error` and `data.checks` says which check failed +- `GET /as/{asn}` - AS detail page (HTML): the prefixes the AS originates and + its peers +- `GET /prefix/{prefix}/{length}` - Prefix detail page (HTML), such as + `/prefix/192.0.2.0/24`: every live route for the prefix, one per peer, and its + origin ASes +- `GET /prefixlength/{length}` - Up to 500 IPv4 prefixes with that mask length, + picked at random, each with its newest route (HTML) +- `GET /prefixlength6/{length}` - The same for IPv6 +- `GET /ip/{addr}` - The same JSON answer as `/api/v1/ip/{ip}` +- `GET /ip/` - The same, for the caller's own address: the address the request + came from or, when that is a trusted proxy (`TRUSTED_PROXIES`, see + Configuration), the client address in its `X-Forwarded-For` ### API v1 -- `GET /api/v1/stats` - Detailed statistics with handler metrics -- `GET /api/v1/ip/{ip}` - Look up AS information for an IP address -- `GET /api/v1/as/{asn}` - Get prefixes announced by an AS -- `GET /api/v1/prefix/{prefix}` - Get routes for a specific prefix + +- `GET /api/v1/stats` - The `/status.json` figures plus connection, garbage + collector, stream and handler queue statistics +- `GET /api/v1/ip/{ip}` - Looks up each address: its most specific live route + (`netblock`), that route's origin AS with its WHOIS details, and the address's + PTR names. `{ip}` is an IP address, a host name, or a comma-separated list of + at most 10 of these. Of the addresses a host name resolves to, only public + ones are looked up: a name with none, such as one that resolves to a private + address, is reported as `host name has no public address`, without its + addresses. `results` has one entry per address with a live route and `errors` + one per address or name that failed. The answer is 400 when `{ip}` lists more + than 10, when it comes to more than 10 addresses once its names are resolved, + or when nothing in it is an IP address or a host name with a public address; + and 404 when none of the addresses has a live route. A PTR lookup gets 2 + seconds; an address whose lookup takes longer is answered without PTR names. +- `GET /api/v1/as/{asn}` - The AS and the prefixes it originates, IPv4 and IPv6 + apart, each with when its newest route was updated +- `GET /api/v1/prefix/{prefix}/{length}` - Every live route for the prefix, one + per peer, and the same routes grouped by origin AS in `origins` + +A JSON answer is `{"status": "ok", "data": ...}`, or for an error +`{"status": "error", "error": {"msg": ..., "code": ...}}`, and carries an +`@meta` object with the time zone, the API version and the time the request +took. The health check is the exception: its 503 answer is also +`{"status": "ok", "data": ...}`, with `error` in `data.status`. + +A request that runs past its time limit answers 504: the limit is 30 seconds, +and 4 seconds for the database reads behind `/status.json` and `/api/v1/stats`. +An AS number, prefix or mask length that does not parse answers 400, as does a +mask length out of range for its IP version. An AS the database does not hold, +or a prefix with no live route, answers 404. The HTML pages give these errors as +plain text. Under `/api/v1` and `/ip/`, a path no route matches answers 404 with +the message `Not found`, and `POST`, `PUT`, `PATCH`, `DELETE`, `OPTIONS`, +`TRACE` or `CONNECT` answers 405 with `Method not allowed`. A 500 answer says +only `Internal server error`, and a host name that does not resolve only +`could not resolve host name`; the detail goes to the log. + +Every answer carries `Strict-Transport-Security`, `Content-Security-Policy`, +`X-Frame-Options`, `X-Content-Type-Options`, `Referrer-Policy` and +`Permissions-Policy`. The Content-Security-Policy lets a page load content only +from the server, and does not allow inline content with `'unsafe-inline'`: the +pages keep their CSS and JavaScript in inline `