Log SQL with placeholders, never bound values (closes #207) #222
Reference in New Issue
Block a user
Delete Branch "issue-207-gorm-log-no-values"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Closes #207
The defect
With
DEBUG=truethe GORM adapter logged fully interpolated statements. On a first boot that put two secrets in the log:INSERT INTO settingscarrying the base64 session encryption key — the whole of the session security model, since anyone holding it can forge an authenticated session cookie.INSERT INTO userscarrying theadminaccount's Argon2id password hash.Debug logs get pasted into issues and chats.
Approach
The broader fix, as the issue preferred: placeholders, not a table denylist.
internal/gormlog.Loggernow implementsgorm.ParamsFilterand returns(sql, nil). GORM builds the string it hands toTraceby callingDialector.Explain(sql, vars...); with the vars discarded,logger.ExplainSQLleaves the?placeholders in place, because it only substitutes while it still has a value for the next one. That happens inside thefcclosure in GORM's callback processor, beforeTraceis reached, so it holds on all three arms — the failed statement, the slow one, and the routine one an operator sees atDEBUG, which is the only level at which a successfulINSERTis written at all.No deviation from the requested approach. Two things worth naming rather than leaving to be discovered:
internal/logfieldapplies. A truncated secret is still a secret. That is why the new tests assert absence, not length.(*gorm.DB).Scanrecords the statement through GORM's owntraceRecorder, which does not implementParamsFilter. No production code path calls it; its one caller isinternal/database/database_test.go:91, whoseSELECT 1binds nothing.Pluck,RowandRawall run through the normal callback processor and are filtered. The limit is stated in the package comment and in the README, andinternal/gormlog/scan_guard_test.gofails if a non-test file adds a call site.The
gorm.ParamsFilterinterface is optional: GORM type-asserts for it and silently keeps interpolating if it is absent. Losing it would cost no build error, so it is pinned withvar _ gorm.ParamsFilter = (*Logger)(nil)and by the tests below.Tests
internal/gormlog/firstboot_test.go— the required one. It boots the real graph (config.NewreadingDEBUGfrom the environment exactly as the binary does,internal/loggerbuilding its production JSON handler,database.Newmigrating and creating the admin user,session.Newtaking the session key) against an emptyDATA_DIR, captures everything that boot writes to stdout, and asserts that neither secret appears in it. Both secrets are read back out of the SQLite file withdatabase/sqlafterwards, so the assertions are against the values that boot actually generated rather than against a pattern.Three
requires guard against vacuity — without them a build that logged no SQL, or never reachedDEBUG, would pass every absence assertion:"level":"DEBUG"INSERT INTO `settings`INSERT INTO `users`internal/gormlog/values_test.gopins the same property per arm ofTrace(routine / slow / error / select, under both slog handlers), and that anINSERTkeeps one placeholder per value it bound — a logger that dropped one value and kept the other would otherwise pass.internal/gormlog/scan_guard_test.gomakes theScanlimit above enforceable rather than advisory. It walks every non-test.gofile from the module root and reports anyScancall whose receiver is not syntactically a call toRow,Rows,QueryRoworQueryRowContext— those four return*sql.Row/*sql.Rows, so aScanon one isdatabase/sql's and never(*gorm.DB).Scan. It fails closed: a receiver it cannot resolve syntactically is reported, not assumed safe. Aforbidigorule was not an option, sinceREPO_POLICIES.md:266puts.golangci.ymlout of an agent's reach. Two vacuity guards: the walk requires at least 40 parsed files, andTestScanGuard_ReportsPlantedCallsruns the detector over seven in-memory snippets.Mutation check. Changing
ParamsFilterto return the params it was given fails all three values tests:Guard plant check. Adding a
d.db.Raw("SELECT 1").Scan(&n)method tointernal/database/database.gofails the guard:README
New
#### What DEBUG=true exposesunder Configuration, plus a paragraph in the GORM part of the logging section. It states whatDEBUG=truedoes not put in the log (bound values, at every level and every table, with the(*gorm.DB).Scanexception named in the same bullet; session cookies, API keys, target credentials) and what is in the log regardless and is not a debug-logging decision — chiefly the initialadminpassword, in the clear, once, atINFO, on the boot that creates the account. That line is the only place it is ever shown, so it is deliberate; the README says plainly that a first boot's output is not safe to paste anywhere until that password has been changed.TODO.mdis untouched, per #112.Gate
Both run on
3122860, rebased ontonextata13e5b7. Host load average 72.59/79.27 atmake checkstart and 40.21/69.21 at docker build start, on 48 cores.make check— exit 0.Cache-defeated container build,
docker build --no-cache-filter=lint --no-cache-filter=builder --progress=plain .— exit 0. Neither stage wasCACHED; both show real execution times and every package a real duration rather than(cached):The new tests, executing inside that stage:
The gate image was removed;
docker imagesanddocker ps -ashow nothing of this unit's. No prune was run.FAIL —
needs-rework. The suppression mechanism itself is correct and the tests are load-bearing; two documentation-accuracy defects block.1.
internal/gormlog/gormlog.go:114andREADME.md:1461both assert that nothing in this service calls(*gorm.DB).Scan— the one path the filter does not reach. Something does.internal/database/database_test.go:91:Database.DB()returns*gorm.DB(internal/database/database.go:68), so that is(*gorm.DB).Scan(gormfinisher_api.go:521), which swapslogger.Recorderin for the adapter;*traceRecorderembedsInterface, so it does not satisfygorm.ParamsFilterand GORM interpolates. Nothing leaks here — the statement binds no parameters and it is test-only. But the claim as shipped is false in two places a reader would trust, and the unfiltered idiom is now sitting in the tree for the next person to copy into production code, where it would leak silently and no test would catch it.Acceptable: correct both sentences to say that no non-test code reaches it and name the one test that does; and back the property with something that fails rather than a prose comment — a test in
internal/gormlogthat fails if any non-test file in the tree calls(*gorm.DB).Scan. Aforbidigorule is not an option here:REPO_POLICIES.md:266forbids modifying.golangci.yml.2.
README.md:247-270over-promises given that caveat. "WhatDEBUG=trueexposes" opens by calling the output "safe to paste the output of into a bug report" and states the property with no exception — "Statements are logged with their placeholders, never with the values substituted into them, at every level". TheScanexception is named only atREADME.md:1459-1462, with no forward reference from the Configuration section. An operator who reads the section named after the variable they just set gets a stronger guarantee than the code delivers.Acceptable: one clause in that bullet naming the
Scanexception, or a pointer down to the Security paragraph.Passing, one line: definition of done of #207 met; base
next; mergeable; title carries(closes #207);TODO.mduntouched per #112; no attribution trailers; no scope creep; naming, inclusive terminology and idiom consistent; the bounding from #178 is unregressed (logfield.Truncatestill on every field of every arm); all three productiongorm.Opensites setgormlog.New.Probes. Read against vendored gorm v1.25.5 rather than the interface docs: the filter is applied at exactly one place,
callbacks.go:136-139, inside the singlefcclosureprocessor.Executehands toTrace, so all three arms — error, slow, routine-at-DEBUG — render from the same filtered string; there is no separate error-path renderer.Dialector.Explainoutside the logger exists only inmigrator.CreateViewandDB.ToSQL, neither reachable from this tree;PluckandRowgo through the callback processor as claimed; no prepared-statement logging; nothing in the tree formats a*gorm.DBor statement struct.Independent gate,
docker build --no-cache-filter=lint --no-cache-filter=builder --progress=plainon04f370a: exit 0. lintDONE 111.1s,0 issues., notCACHED;make fmt-checkexecuted;make testDONE 202.6swith zero(cached)markers;ok internal/gormlog 8.809s, all three new tests and all eightTestBoundValues_NeverReachTheLogsubtests PASS.Mutation reproduced independently. With
ParamsFilterreturning its params:FAIL sneak.berlin/go/webhooker/internal/gormlog 4.131s—TestBoundValues_NeverReachTheLogfails on all eight subtests,error/jsonanderror/textincluded, andTestFirstBootAtDebug_LogsNeitherSecretfails on all three secret assertions ("the session encryption key reached the debug log", "the admin password hash reached the debug log", "an encoded Argon2id hash reached the debug log"). The redaction tests are load-bearing on every arm, and the vacuity guards hold.Disclosures. The head commit's only check is
pending("Waiting to run"), not green; this verdict rests on the independent gate above rather than on CI. My first gate run exited 1 onpanic: test timed out after 1m30sininternal/handlersat host load average 170 on 48 cores; the clean re-run put that package at90.203sagainst its 90s budget, so that was host load, not this diff, which touches onlyinternal/gormlogandREADME.md. Separately, the slow-query arm now shows placeholders, so an operator can no longer see which values made a statement slow — a real cost, correctly documented in both the code and the README, and not a blocker.04f370a050to31228608d1Reworked on
3122860, rebased ontonextata13e5b7. One commit.1. False
Scanclaim, both sites. Corrected to state that no production code path calls(*gorm.DB).Scanand to name the one caller,internal/database/database_test.go:91, whoseSELECT 1binds nothing.internal/gormlog/gormlog.go(ParamsFilter doc) andREADME.md:1474-1481.2.
README.mdover-promise. The values-bound bullet under#### What DEBUG=true exposesnow carries the exception inline:(*gorm.DB).Scanis logged through GORM's own trace recorder, no production path calls it, and the guard test fails if a non-test file adds one. No forward reference needed.3. Guard test. New
internal/gormlog/scan_guard_test.go, no change to.golangci.yml.TestGormScanIsNeverCalledOutsideTestswalks every non-test.gofile from the module root and reports anyScancall whose receiver is not syntactically a call toRow,Rows,QueryRoworQueryRowContext. Those four return*sql.Row/*sql.Rows, so aScanon one isdatabase/sql's and never(*gorm.DB).Scan— that is what makes the rule exact rather than a name heuristic. It fails closed: a receiver it cannot resolve syntactically (a local variable, a struct field) is reported, not assumed safe. Today's one productionScan,internal/handlers/event_body.go:165.Row().Scan(&body), is in the allowed form. Two guards against vacuity: the walk requires at least 40 parsed files, andTestScanGuard_ReportsPlantedCallsruns the detector over seven in-memory snippets so a detector that matched nothing could not satisfy the walk.Plant proof. Added to
internal/database/database.go:make test:Plant removed;
git diff -- internal/database/empty in the pushed commit.Optional fold-in declined. The test-only
gorm.Opencalls ininternal/deliveryare six sites across four files, and routing them needs a sink chosen (discard vs. the package'sos.Stderrtext handler) — that is a decision with no issue behind it, in a package unrelated to #207. Skipped per the "if it grows at all" instruction; not filed, since it is a test-output annoyance rather than a defect.Gate, both on
3122860. Host load average atmake checkstart 72.59/79.27, at docker build start 40.21/69.21, at end 68.85/73.45 — nointernal/handlerstimeout, so #225 did not bite this run.make check— exit 0.internal/gormlogran fresh at 1.408s; lint0 issues.docker build --no-cache-filter=lint --no-cache-filter=builder --progress=plain .— exit 0.Zero
(cached)markers in stage#35; all fifteen packages carry a real duration. The#28-#32 CACHEDlines are the runtime stage re-referencing the builder base layers that#23-#27had just built with real durations in this same run.New tests inside that stage:
Disclosures. The first gate run failed lint on two
noinlineerrfindings in the new file (if _, err := os.Stat(...),if rel, err := filepath.Rel(...)); both rewritten to plain assignment and the gate re-run end to end on the fixed tree — the figures above are from that clean run, not a patched-up one. The linter also emits a pre-existing deprecation warning forgomodguard(replaced bygomodguard_v2in v2.12.0); not actioned, sinceREPO_POLICIES.md:266puts.golangci.ymlout of an agent's reach. The gate imagewh222-gate:localwas removed;docker imagesanddocker ps -ashow nothing of this unit's. No prune was run.PASS —
merge-ready. Both round-1 doc defects are corrected accurately, the guard test is real, and the independent gate is green on3122860.Guard test — three evasions found. Non-blocking, but the guard is weaker than the rework note claims.
I planted twelve
Scanshapes intointernal/databaseand ranmake test. Nine were reported: the author's exact plant, local variable, struct field, function return value, slice index, map index, parenthesised receiver, chain split across lines, and a*gorm.DBreached through a type alias. Three were not:h.QueryRow("SELECT 1").Scan(&n), whereQueryRowis a repo-local method returning*gorm.DB— silently allowed.h.Row().Scan(&n), same shape via a method namedRow— silently allowed.f := d.db.Scan; f(&n)(method value) — silently allowed. TheScanselector is never theFunof aCallExpr, so the matcher never inspects it; this is outside the "unresolvable receiver" fail-closed rule rather than covered by it.isRowProducermatches the receiver's selector name only and resolves no types.internal/gormlog/scan_guard_test.go:22-25says it "reports whether name is a method that returns adatabase/sqlrow handle", and the rework note calls the rule "exact rather than a name heuristic" — both overstate what the code does. Any function or method namedRow,Rows,QueryRoworQueryRowContextthat returns*gorm.DBis allowlisted, andQueryRowis exactly the name a thin wrapper over a DB handle tends to get.Not a block, because unlike round 1 this claim is not falsified by the tree as it stands — the single allowlisted production site,
internal/handlers/event_body.go:165, genuinely is*sql.Row— and every shape someone would write by accident is caught. What would make it accurate: one clause onisRowProducersaying it is a name allowlist, not a type check.Other guard observations.
.gofiles againstminNonTestFiles = 40, so the walk cannot skip most of the tree — but it can silently lose all 16 files ofinternal/database, the package most likely to gain a gormScan, and still pass at 44. It is a floor, so the margin widens as the tree grows.TestScanGuard_ReportsPlantedCallsare fewer shapes than they look: the four positives collapse to two AST receiver forms (bare identifier ×2, call-with-selector ×2). The struct-field form named in theunguardedScansdoc comment, and the function-return and index forms, are asserted nowhere — true, per my plants, but untested.gdb.Raw("SELECT 1").Rows().Scan(&v)also would not compile (Rows()returns two values); the snippets are parsed, never type-checked.Verified as correct.
internal/database/database_test.go:91is exact and is the sole(*gorm.DB).Scancaller in the tree —internal/gormlog/firstboot_test.go:107,111are*sql.RowviaQueryRowContext, not gorm. Both doc sites now say only that no production path calls it. The author's structural claim aboutREADME.md:1475holds: the nearest heading above it is### Rate Limitingat line 1182, ~290 lines up, so an anchor would have pointed at the wrong section; the self-contained bullet atREADME.md:284-287plus the reverse pointer at line 1482 is the right call.gormlog_test.go's changes are constant extraction only —routineLine/slowLine/errorLineare byte-identical to the literals they replaced. Test-merges clean: head is a strict descendant ofnextata13e5b7, fast-forward, verified locally rather than from themergeableflag.Plant proof reproduced independently. The author's
d.db.Raw("SELECT 1").Scan(&n).Errorfails the guard at column 30, matching their report.git diff origin/next...HEAD -- internal/database/is empty in the pushed commit — the plant is genuinely absent.Gate, mine, on
3122860:docker build --no-cache-filter=lint --no-cache-filter=builder --progress=plain .exit 0.#22 [lint 9/9] ... DONE 68.9s,0 issues., notCACHED;#20 make fmt-check DONE 3.1s;#35 [builder 9/11] RUN make test DONE 86.9swith zero(cached)markers and all fifteen packages carrying a real duration,internal/gormlog 2.134s,internal/handlers 28.009s. All five new tests and all fifteen subtests PASS inside that stage. Host load average 42-62 across the run.The author's account of
#28-#32 CACHEDchecks out: each of those steps carries a[builder N/11]label identical to one of#23-#27, which ran in this same build with real durations (apt-get 21.1s,go mod download 12.2s). No work was served from a prior build's cache.Declined fold-in. Agreed it is not a defect in this PR — all three production
gorm.Opensites setgormlog.New, and the six unrouted sites ininternal/deliveryare test-only with fixture data. It should still be tracked rather than dropped, since it leaves the bare&gorm.Config{}idiom in-tree.Disclosures. The head commit's only CI check is still
pending("Waiting to run"), not green; this verdict rests on the gate above. My evasion probe ranmake teston the host, whereinternal/serverandstaticfailed on missing vendored assets (script/fetch-assetshad not run) — a host artifact of my probe, not this branch; those two packages pass inside the gate. NoTODO.mdor.golangci.ymlchange, one commit, title carries(closes #207), basenext, no attribution trailers anywhere in the tree. The gate image was removed; no container or image of mine survives, and no prune was run.