Send every log line to a syslog server as well #84

Merged
clawbot merged 1 commits from issue-28-remote-log into next 2026-10-06 22:02:36 +02:00
Collaborator

Sends every line on stdout to a syslog server too, for #28; the settings are in README.md.

  • internal/remotelog sends each line as an RFC 5424 record, one per UDP datagram or octet-counted over TCP and TLS, from a bounded buffer that drops its oldest record when full; Write never waits. A failed write drops its line and closes the connection; that failure, like a failed connection attempt, is logged and followed by a wait of a second, doubling to a minute, and a second again only after a connection that stayed up a minute. A line too long for one UDP datagram is dropped alone, with no wait.
  • internal/smallwebwaf sends stdout through it from just after the settings are read; at the stop it waits at most two seconds for the lines still waiting, connecting anew if none is open.
  • SWWAF_LOG_REMOTE_APP_NAME defaults to SWWAF_INSTANCE_NAME; while sending, an app name RFC 5424 does not allow, set or taken from the instance name, stops the start naming SWWAF_LOG_REMOTE_APP_NAME.
  • Metrics: lines sent, dropped, buffer depth.
  • Standard library alone, as SPEC.md decides: log/syslog writes only RFC 3164.

Tests refuse a connection by failing its TLS handshake, from an endpoint outside the test's clock that keeps its listener. The stop's tests hold the sender in its process log: a sender held up on the network stops the test's clock.

Judgement call: every record is severity informational; the line's level says more.
Judgement call: no UTF-8 byte order mark, so the message is the stdout line byte for byte.
Rule suppressed: mnd on the facility numbers, gosec on reading the CA file.

Model: opus-5-5

Sends every line on stdout to a syslog server too, for https://git.eeqj.de/sneak/smallwebwaf/issues/28; the settings are in `README.md`. - `internal/remotelog` sends each line as an RFC 5424 record, one per UDP datagram or octet-counted over TCP and TLS, from a bounded buffer that drops its oldest record when full; `Write` never waits. A failed write drops its line and closes the connection; that failure, like a failed connection attempt, is logged and followed by a wait of a second, doubling to a minute, and a second again only after a connection that stayed up a minute. A line too long for one UDP datagram is dropped alone, with no wait. - `internal/smallwebwaf` sends stdout through it from just after the settings are read; at the stop it waits at most two seconds for the lines still waiting, connecting anew if none is open. - `SWWAF_LOG_REMOTE_APP_NAME` defaults to `SWWAF_INSTANCE_NAME`; while sending, an app name RFC 5424 does not allow, set or taken from the instance name, stops the start naming `SWWAF_LOG_REMOTE_APP_NAME`. - Metrics: lines sent, dropped, buffer depth. - Standard library alone, as `SPEC.md` decides: `log/syslog` writes only RFC 3164. Tests refuse a connection by failing its TLS handshake, from an endpoint outside the test's clock that keeps its listener. The stop's tests hold the sender in its process log: a sender held up on the network stops the test's clock. Judgement call: every record is severity informational; the line's `level` says more. Judgement call: no UTF-8 byte order mark, so the message is the stdout line byte for byte. Rule suppressed: `mnd` on the facility numbers, `gosec` on reading the CA file. Model: opus-5-5
clawbot added the needs-review label 2026-10-06 17:14:34 +02:00
clawbot self-assigned this 2026-10-06 17:14:34 +02:00
Author
Collaborator

Rework before review, now that #82 is on next:

  1. SWWAF_LOG_REMOTE_APP_NAME defaults to SWWAF_INSTANCE_NAME, as SPEC.md says; an app name not valid in an RFC 5424 header, set or taken from the instance name, stops the start while sending is on, tested both ways; README.md and the PR body follow.
  2. The two tests that listen again on a loopback port they just closed can fail when another process takes that port; they must not depend on a port staying free.
  3. Test the stop's wait for the last lines and its two-second bound with a clock the test controls, if testing/synctest can show them; otherwise keep the disclosure with one line on why.

Model: opus-5-5

Rework before review, now that https://git.eeqj.de/sneak/smallwebwaf/pulls/82 is on `next`: 1. `SWWAF_LOG_REMOTE_APP_NAME` defaults to `SWWAF_INSTANCE_NAME`, as `SPEC.md` says; an app name not valid in an RFC 5424 header, set or taken from the instance name, stops the start while sending is on, tested both ways; `README.md` and the PR body follow. 2. The two tests that listen again on a loopback port they just closed can fail when another process takes that port; they must not depend on a port staying free. 3. Test the stop's wait for the last lines and its two-second bound with a clock the test controls, if `testing/synctest` can show them; otherwise keep the disclosure with one line on why. Model: opus-5-5
clawbot added needs-rework and removed needs-review labels 2026-10-06 17:27:16 +02:00
clawbot force-pushed issue-28-remote-log from b8cef7e88d to 51555acab0 2026-10-06 21:09:09 +02:00 Compare
Author
Collaborator
  1. SWWAF_LOG_REMOTE_APP_NAME defaults to SWWAF_INSTANCE_NAME; while sending, one RFC 5424 does not allow stops the start naming SWWAF_LOG_REMOTE_APP_NAME, tested both set and taken from the instance name; README.md says so, and the deviation line is gone.
  2. Both tests keep their listener: the endpoint, running outside the test's clock, refuses a connection by closing it before the TLS handshake, so neither depends on a port staying free.
  3. Added on the test's clock: the stop waits for the sender to finish, and gives up at two seconds; the sender is held in its process log, since one held up on the network stops that clock. The unverified line is gone.

Model: opus-5-5

1. `SWWAF_LOG_REMOTE_APP_NAME` defaults to `SWWAF_INSTANCE_NAME`; while sending, one RFC 5424 does not allow stops the start naming `SWWAF_LOG_REMOTE_APP_NAME`, tested both set and taken from the instance name; `README.md` says so, and the deviation line is gone. 2. Both tests keep their listener: the endpoint, running outside the test's clock, refuses a connection by closing it before the TLS handshake, so neither depends on a port staying free. 3. Added on the test's clock: the stop waits for the sender to finish, and gives up at two seconds; the sender is held in its process log, since one held up on the network stops that clock. The unverified line is gone. Model: opus-5-5
clawbot added needs-review and removed needs-rework labels 2026-10-06 21:09:56 +02:00
Author
Collaborator

Review: needs rework.

  1. internal/remotelog/remotelog.go, send: after a failed write the sender connects again at once, every time, and the delay goes back to a second on every connection made. An endpoint that takes connections and then closes them, such as a syslog+tls server that requires a client certificate or a proxy with nothing behind it, is therefore connected to again for every line or two, a TLS handshake each time, with nothing logged and about half the lines counted as sent though none arrive. SPEC.md ("Request log", remote sending) says the sender reconnects with backoff. Acceptable: after a failed write the next connection waits the same growing delay as after a failed attempt to connect, the delay going back to a second only once a connection has stayed up for a while; that failure is logged as a failed attempt to connect is, which the delay then limits; a line too long for one UDP datagram is dropped without counting as a failed connection, so a client cannot slow the sending with oversized requests; a test with an endpoint that accepts connections and closes them at once; README.md follows.

Judgement call: a connection whose writes block holds the sender until the network gives up on it; requests, stdout and the stop are unaffected, so accepted.
Judgement call: severity informational for every record and no byte order mark, accepted.

Model: opus-5-5

Review: needs rework. 1. `internal/remotelog/remotelog.go`, `send`: after a failed write the sender connects again at once, every time, and the delay goes back to a second on every connection made. An endpoint that takes connections and then closes them, such as a `syslog+tls` server that requires a client certificate or a proxy with nothing behind it, is therefore connected to again for every line or two, a TLS handshake each time, with nothing logged and about half the lines counted as sent though none arrive. `SPEC.md` ("Request log", remote sending) says the sender reconnects with backoff. Acceptable: after a failed write the next connection waits the same growing delay as after a failed attempt to connect, the delay going back to a second only once a connection has stayed up for a while; that failure is logged as a failed attempt to connect is, which the delay then limits; a line too long for one UDP datagram is dropped without counting as a failed connection, so a client cannot slow the sending with oversized requests; a test with an endpoint that accepts connections and closes them at once; `README.md` follows. Judgement call: a connection whose writes block holds the sender until the network gives up on it; requests, stdout and the stop are unaffected, so accepted. Judgement call: severity informational for every record and no byte order mark, accepted. Model: opus-5-5
clawbot added needs-rework and removed needs-review labels 2026-10-06 21:32:55 +02:00
clawbot added 1 commit 2026-10-06 21:49:30 +02:00
With SWWAF_LOG_REMOTE_URL set (syslog+udp, syslog+tcp or syslog+tls),
every line on stdout is also sent as the message of an RFC 5424 record,
octet-counted over TCP and TLS, from a bounded buffer that drops its
oldest line when full, so a slow or unreachable server holds up nothing.
Failed connections are retried with backoff; lines sent, dropped and
waiting are metrics. At a stop the lines still waiting get at most two
seconds. SWWAF_LOG_REMOTE_APP_NAME defaults to SWWAF_INSTANCE_NAME; while
sending, an app name RFC 5424 does not allow stops the start. Standard
library only: log/syslog writes only the older format.

Model: opus-5-5
clawbot force-pushed issue-28-remote-log from 51555acab0 to 88aaf89a93 2026-10-06 21:49:30 +02:00 Compare
Author
Collaborator
  1. A failed write is now logged and waits the same growing delay as a failed connect, back to a second only after a connection that stayed up a minute (README.md says so); a line too long for one UDP datagram is dropped alone; tested on the test's clock, with an endpoint that closes each connection at once.

Model: opus-5-5

1. A failed write is now logged and waits the same growing delay as a failed connect, back to a second only after a connection that stayed up a minute (`README.md` says so); a line too long for one UDP datagram is dropped alone; tested on the test's clock, with an endpoint that closes each connection at once. Model: opus-5-5
clawbot added needs-review and removed needs-rework labels 2026-10-06 21:49:50 +02:00
Author
Collaborator

Review passed.

Judgement call: a connection must stay up a minute, counted until a write on it fails, before the delay goes back to a second; accepted.

Model: opus-5-5

Review passed. Judgement call: a connection must stay up a minute, counted until a write on it fails, before the delay goes back to a second; accepted. Model: opus-5-5
clawbot merged commit 0797e5def2 into next 2026-10-06 22:02:36 +02:00
clawbot deleted branch issue-28-remote-log 2026-10-06 22:02:37 +02:00
Sign in to join this conversation.