Log to stderr and stop discarding With attributes (closes #82)
check / check (push) Successful in 4m20s
check / check (push) Successful in 4m20s
Closes #97. internal/log attached both handlers to os.Stdout, so any record that was not suppressed landed in the middle of a --json document. WARN and ERROR are never suppressed, so this was not hypothetical: a config file with permissions looser than 0600 was enough to break `vaultik snapshot list --json | jq`. Both handlers now write to os.Stderr, and the TTY-vs-JSON format choice tests os.Stderr rather than os.Stdout - the format has to follow the stream the records land on, or a redirected stderr gets colorized whenever stdout happens to be a terminal. User-visible: --verbose and --debug output moves to stderr too, so `vaultik snapshot list -v > out.txt` no longer captures diagnostics. --quiet and --cron semantics are unchanged. TTYHandler.WithAttrs and WithGroup discarded their arguments and returned the receiver, while their doc comments claimed otherwise, so attributes passed through the exported log.With vanished. The effect was environment-dependent in the worst direction: handler choice is by TTY-ness, so attributes disappeared on a terminal - where a developer is debugging - and appeared correctly in CI. Both now return a new handler with copied state rather than mutating the receiver, since slog permits a handler to be shared and derived from concurrently. A test asserts the TTY and JSON handlers emit the same attribute set, which is the test that would have caught the original defect. The local workaround in snapshot_list.go is removed now that the logger no longer writes to stdout. The collect-then-emit machinery is kept, but for a different reason than it was added: emitting from the fetch workers would order warnings by network timing, whereas key-order emission after group.Wait() is deterministic run to run. Not yet complete: --json stdout still carries the startup banner, which internal/cli/entry.go writes before cobra parses and which bannerSuppressedInArgs does not recognise --json for. That is the remaining stdout contamination path and is tracked in #106.
This commit was merged in pull request #107.
This commit is contained in:
@@ -87,7 +87,7 @@ func (v *Vaultik) ListSnapshots(jsonOutput bool) error {
|
||||
snapshots = append(snapshots, info)
|
||||
}
|
||||
|
||||
listing, remoteErr := v.collectRemoteSnapshots(localKeys, jsonOutput)
|
||||
listing, remoteErr := v.collectRemoteSnapshots(localKeys)
|
||||
if remoteErr != nil {
|
||||
v.warnRemoteListingFailed(remoteErr, jsonOutput)
|
||||
} else {
|
||||
@@ -131,23 +131,23 @@ func (v *Vaultik) ListSnapshots(jsonOutput bool) error {
|
||||
// still worth printing, and `snapshot list` exiting non-zero because a
|
||||
// volume is unmounted would be worse than useless.
|
||||
//
|
||||
// In --json mode the warning goes to stderr rather than through the
|
||||
// logger or the UI writer, both of which emit on stdout — the JSON
|
||||
// document has to be the only thing on stdout for `snapshot list --json
|
||||
// | jq` to work. The failure is also representable in the document
|
||||
// itself: every row's remote_present is null when the destination could
|
||||
// not be listed.
|
||||
// The two output modes report it through different channels. Table mode
|
||||
// uses the UI writer, whose prose and color match the table it sits
|
||||
// under. The UI writer emits on stdout, though, so --json mode uses the
|
||||
// logger instead: stdout has to hold nothing but the JSON document for
|
||||
// `snapshot list --json | jq` to work. Both channels are chosen once,
|
||||
// never both, so the user is not told the same thing twice.
|
||||
//
|
||||
// The failure is also representable in the document itself: every row's
|
||||
// remote_present is null when the destination could not be listed.
|
||||
func (v *Vaultik) warnRemoteListingFailed(err error, jsonOutput bool) {
|
||||
if jsonOutput {
|
||||
_, _ = fmt.Fprintf(v.Stderr,
|
||||
"Warning: could not list backup destination store: %v. "+
|
||||
"Showing snapshots from the local index only.\n", err)
|
||||
log.Warn("Could not list backup destination store; "+
|
||||
"showing snapshots from the local index only", "error", err)
|
||||
|
||||
return
|
||||
}
|
||||
|
||||
// Once only: the logger also writes to stdout, so emitting through
|
||||
// both it and the UI would print the same sentence to the user twice.
|
||||
v.UI.Warningf("Could not list backup destination store: %v.", err)
|
||||
v.UI.Infof("Showing snapshots from the local index only.")
|
||||
}
|
||||
@@ -156,69 +156,29 @@ func (v *Vaultik) warnRemoteListingFailed(err error, jsonOutput bool) {
|
||||
// is about to read is incomplete: manifests that could not be read, and
|
||||
// remote-only snapshots dropped by the maxRemoteOnlyRows cap.
|
||||
//
|
||||
// Table mode reports both below the table (see reportListDrift). In
|
||||
// --json mode they cannot go on stdout — the document has to be the
|
||||
// only thing there for `snapshot list --json | jq` to work — and the
|
||||
// document's shape is deliberately left alone so existing consumers
|
||||
// keep parsing. So they go to stderr, the same place the
|
||||
// unreachable-destination warning already goes. A consumer that must
|
||||
// react to truncation can treat any output on this stream as "this
|
||||
// listing is not the whole picture"; silent truncation of a listing
|
||||
// whose whole purpose is disaster recovery is the worse failure.
|
||||
// Table mode reports both below the table (see reportListDrift) through
|
||||
// the UI writer, which emits on stdout. In --json mode stdout has to
|
||||
// hold nothing but the document for `snapshot list --json | jq` to
|
||||
// work, and the document's shape is deliberately left alone so existing
|
||||
// consumers keep parsing — so these go to the logger, which writes to
|
||||
// stderr. A consumer that must react to truncation can treat any output
|
||||
// on that stream as "this listing is not the whole picture"; silent
|
||||
// truncation of a listing whose whole purpose is disaster recovery is
|
||||
// the worse failure.
|
||||
func (v *Vaultik) reportJSONListingLimits(listing *remoteSnapshotListing) {
|
||||
if listing.unreadable > 0 {
|
||||
_, _ = fmt.Fprintf(v.Stderr,
|
||||
"Warning: %d remote snapshot(s) could not be described: "+
|
||||
"manifest missing or unreadable. They are missing from "+
|
||||
"this listing.\n", listing.unreadable)
|
||||
log.Warn("Some remote snapshot(s) could not be described: "+
|
||||
"manifest missing or unreadable; they are missing from "+
|
||||
"this listing", "unreadable", listing.unreadable)
|
||||
}
|
||||
|
||||
if listing.omitted > 0 {
|
||||
_, _ = fmt.Fprintf(v.Stderr,
|
||||
"Warning: listing truncated: %d further remote-only "+
|
||||
"snapshot(s) not shown (limit %d per listing).\n",
|
||||
listing.omitted, maxRemoteOnlyRows)
|
||||
log.Warn("Listing truncated: further remote-only snapshot(s) "+
|
||||
"not shown", "omitted", listing.omitted,
|
||||
"limit", maxRemoteOnlyRows)
|
||||
}
|
||||
}
|
||||
|
||||
// kvPairSize is the number of variadic arguments that make up one
|
||||
// structured logging key/value pair.
|
||||
const kvPairSize = 2
|
||||
|
||||
// warnWhileListing reports a per-snapshot problem found while
|
||||
// describing the destination store, through a writer that is safe for
|
||||
// the current output mode.
|
||||
//
|
||||
// In --json mode it writes to v.Stderr rather than calling log.Warn,
|
||||
// for the same reason warnRemoteListingFailed does: internal/log builds
|
||||
// its logger over os.Stdout and defaults to level Warn, so one warning
|
||||
// there would put a log line on stdout ahead of the JSON document and
|
||||
// break `snapshot list --json | jq`. A single corrupt manifest is
|
||||
// precisely the degradation this listing is built to survive, so it
|
||||
// must not be the thing that corrupts the output.
|
||||
//
|
||||
// This is a local workaround. Remove it, and the branch in
|
||||
// warnRemoteListingFailed, once issue #82 makes the logger's sink
|
||||
// configurable.
|
||||
func (v *Vaultik) warnWhileListing(jsonOutput bool, msg string, args ...any) {
|
||||
if !jsonOutput {
|
||||
log.Warn(msg, args...)
|
||||
|
||||
return
|
||||
}
|
||||
|
||||
var line strings.Builder
|
||||
|
||||
_, _ = fmt.Fprintf(&line, "Warning: %s", msg)
|
||||
|
||||
for i := 0; i+kvPairSize <= len(args); i += kvPairSize {
|
||||
pair := args[i : i+kvPairSize]
|
||||
_, _ = fmt.Fprintf(&line, " %v=%v", pair[0], pair[1])
|
||||
}
|
||||
|
||||
_, _ = fmt.Fprintln(v.Stderr, line.String())
|
||||
}
|
||||
|
||||
// remoteSnapshotListing is the result of one pass over the destination
|
||||
// store's metadata/ prefix.
|
||||
type remoteSnapshotListing struct {
|
||||
@@ -247,11 +207,8 @@ type remoteSnapshotListing struct {
|
||||
// Manifest reads scale only with the number of snapshots the local
|
||||
// index does not already know about, and are capped at
|
||||
// maxRemoteOnlyRows.
|
||||
//
|
||||
// jsonOutput only selects where per-snapshot warnings are written; see
|
||||
// warnWhileListing.
|
||||
func (v *Vaultik) collectRemoteSnapshots(
|
||||
localKeys map[string]bool, jsonOutput bool,
|
||||
localKeys map[string]bool,
|
||||
) (*remoteSnapshotListing, error) {
|
||||
keys, err := v.listAllRemoteSnapshotKeys()
|
||||
if err != nil {
|
||||
@@ -282,17 +239,22 @@ func (v *Vaultik) collectRemoteSnapshots(
|
||||
}
|
||||
|
||||
listing.remoteOnly, listing.unreadable = v.describeRemoteOnlySnapshots(
|
||||
unknown, jsonOutput)
|
||||
unknown)
|
||||
|
||||
return listing, nil
|
||||
}
|
||||
|
||||
// listingWarning is a problem found with one remote snapshot, recorded
|
||||
// rather than emitted on the spot. Manifest reads run concurrently and
|
||||
// the writer chosen by warnWhileListing is not guaranteed to be safe
|
||||
// for concurrent use, so warnings are held until every read has
|
||||
// finished and then emitted in key order from a single goroutine. That
|
||||
// also makes the warning order deterministic run to run.
|
||||
// rather than emitted on the spot. Manifest reads run concurrently, so
|
||||
// emitting from the worker that found the problem would order the
|
||||
// warnings by fetch completion — which varies run to run with network
|
||||
// timing and tells the reader nothing. Holding them and emitting in key
|
||||
// order from a single goroutine after every read has finished makes two
|
||||
// runs over the same damaged store produce the same diagnostics in the
|
||||
// same order.
|
||||
//
|
||||
// Concurrency safety is no longer part of the reason: these are emitted
|
||||
// through log.Warn, and slog handlers are safe for concurrent use.
|
||||
type listingWarning struct {
|
||||
msg string
|
||||
args []any
|
||||
@@ -306,7 +268,7 @@ type listingWarning struct {
|
||||
// failing the listing: one bad snapshot directory must not hide every
|
||||
// other snapshot the user has.
|
||||
func (v *Vaultik) describeRemoteOnlySnapshots(
|
||||
keys []string, jsonOutput bool,
|
||||
keys []string,
|
||||
) ([]SnapshotInfo, int) {
|
||||
found := make([]SnapshotInfo, len(keys))
|
||||
ok := make([]bool, len(keys))
|
||||
@@ -349,7 +311,7 @@ func (v *Vaultik) describeRemoteOnlySnapshots(
|
||||
|
||||
for i := range keys {
|
||||
if warnings[i] != nil {
|
||||
v.warnWhileListing(jsonOutput, warnings[i].msg, warnings[i].args...)
|
||||
log.Warn(warnings[i].msg, warnings[i].args...)
|
||||
}
|
||||
|
||||
if !ok[i] {
|
||||
|
||||
Reference in New Issue
Block a user