Author SHA1 Message Date
clawbot e2227ac07e Copy the current canonical .golangci.yml (closes #26)
check / check (push) Waiting to run
The shared lint config in the prompts repo moved from the deprecated
gomodguard linter to gomodguard_v2, with a module block list, and now
enables depguard to keep test-support packages out of non-test files.
This replaces the repo's copy with that file unchanged, so lint no
longer prints the gomodguard deprecation warning.

Model: opus-5-5
2026-10-04 16:13:20 +02:00
clawbot 4a16a41bd7 Test both hashWorker cancellation checks on their own (closes #83)
check / check (push) Waiting to run
TestHashWorkerDropsQueuedRuns now passes hashWorker a hash function
that records being called, so a worker that hashes a run after the
scan is cancelled fails the test every time instead of only when it
then chose to send its result.

TestHashWorkerAbandonsBlockedSend cancels the scan from inside the
hash function and leaves the result channel unread, so the worker can
only return through the cancellation case beside its send. The scan
tests could not show this, because stop drains results and frees a
parked worker anyway.

Model: opus-5-5
2026-10-04 16:01:36 +02:00
clawbot 722675f153 Escape the database path in the SQLite connection string (closes #55)
check / check (push) Waiting to run
openDB put the path into the connection string unescaped, so a ? or #
in it ended the file name and a % started an escape: scan could
silently fill a database under a shortened name. The path now goes
through net/url as a file: URI. An absolute path gets an empty host and
a relative path none, because SQLite reads what follows file:// up to
the next slash as a host name. The path is not cleaned, so it stays
exactly what the operator gave.

A test runs scan, report and trees against such a file name given as an
absolute path, as one starting with //, and as a relative path, and
checks that only that file and its lock file exist afterwards.

Model: opus-5-5
2026-10-04 15:13:19 +02:00
6 changed files with 213 additions and 26 deletions
+66 -2
View File
@@ -10,14 +10,20 @@ run:
linters: linters:
default: all default: all
enable:
# Successor to the deprecated gomodguard. Named explicitly, rather than
# left to `default: all`, because it carries the module policy below.
- gomodguard_v2
disable: disable:
# Genuinely incompatible with project patterns # Genuinely incompatible with project patterns
- exhaustruct # Requires all struct fields - exhaustruct # Requires all struct fields
- depguard # Dependency allow/block lists
- godot # Requires comments to end with periods - godot # Requires comments to end with periods
- wsl # Deprecated, replaced by wsl_v5
- wrapcheck # Too verbose for internal packages - wrapcheck # Too verbose for internal packages
- varnamelen # Short names like db, id are idiomatic Go - varnamelen # Short names like db, id are idiomatic Go
# Deprecated: the warning is attached to the old name, so it is
# silenced by disabling that name, not by enabling the successor.
- wsl # Deprecated, replaced by wsl_v5
- gomodguard # Deprecated, replaced by gomodguard_v2
settings: settings:
lll: lll:
line-length: 88 line-length: 88
@@ -28,6 +34,64 @@ linters:
max-complexity: 15 max-complexity: 15
dupl: dupl:
threshold: 100 threshold: 100
depguard:
# Test-support code must not be compiled into the shipped binary. A
# test-support package exists to hand a test privileges the program
# itself must never have, so a file that is not a test must not import
# one. Test files, and the files inside a package whose directory name
# ends in `test`, are where that code belongs, and are exempt.
#
# The deny list below is the one part of this file a repository is
# expected to extend, and the only part it may. depguard matches an
# import path against a list of prefixes, so it cannot be told "any path
# whose last segment ends in test"; a repository's own test-support
# packages have to be named here one at a time, by full import path,
# under a module path that differs from repository to repository. Add
# them; change nothing else.
rules:
test-support:
list-mode: lax
files:
- "$all"
- "!$test"
- "!**/*test/**"
deny:
- pkg: net/http/httptest
desc: >-
Test-support code belongs in test files and in packages whose
directory name ends in test, not in the shipped binary.
# Only decisions already recorded in the Go package defaults are
# listed here. Every entry matches the module path exactly.
gomodguard_v2:
blocked:
- module: github.com/rs/zerolog
recommendations:
- log/slog
reason: "Structured logging is stdlib log/slog."
# One entry per pre-fork module path, because the later releases
# are separate paths. A prefix match would be shorter but would
# also reach github.com/go-redis/redismock, the test double for
# the successor these entries recommend.
- module: github.com/go-redis/redis
recommendations:
- github.com/redis/go-redis/v9
reason: "Pre-fork module; use the maintained go-redis v9."
- module: github.com/go-redis/redis/v7
recommendations:
- github.com/redis/go-redis/v9
reason: "Pre-fork module; use the maintained go-redis v9."
- module: github.com/go-redis/redis/v8
recommendations:
- github.com/redis/go-redis/v9
reason: "Pre-fork module; use the maintained go-redis v9."
- module: github.com/sergi/go-diff
recommendations:
- github.com/aymanbagabas/go-udiff
reason: "No unified diff output; use go-udiff."
- module: github.com/hexops/gotextdiff
recommendations:
- github.com/aymanbagabas/go-udiff
reason: "Unmaintained fork; use go-udiff."
issues: issues:
max-issues-per-linter: 0 max-issues-per-linter: 0
+3 -1
View File
@@ -283,7 +283,9 @@ All three subcommands operate on a single SQLite database file:
- Location: the value of the `SFDUPES_DATABASE` environment variable - Location: the value of the `SFDUPES_DATABASE` environment variable
when set and non-empty, otherwise `/var/lib/sfdupes/db.sqlite`. when set and non-empty, otherwise `/var/lib/sfdupes/db.sqlite`.
There is no command-line flag. There is no command-line flag. The path names the file exactly,
whatever characters it holds (`?`, `#` and `%` included); a
relative path is relative to the working directory.
- `scan` creates the database (and its parent directory) on first - `scan` creates the database (and its parent directory) on first
use. `report` and `trees` require an existing database; a missing use. `report` and `trees` require an existing database; a missing
database file is a fatal error (exit 1) telling the user to run database file is a fatal error (exit 1) telling the user to run
+10
View File
@@ -29,6 +29,16 @@
# Completed Steps # Completed Steps
- `.golangci.yml` replaced with the current canonical copy, which uses
`gomodguard_v2`, so lint no longer prints a deprecation warning
(2026-10-04, https://git.eeqj.de/sneak/sfdupes/issues/26)
- a test fails when either `hashWorker` cancellation check in `scan.go`
is removed (2026-10-04, https://git.eeqj.de/sneak/sfdupes/issues/83)
- a database path holding `?`, `#` or `%` opens exactly the file it names
(2026-10-04, https://git.eeqj.de/sneak/sfdupes/issues/55)
- `scan` rejects `--workers` below 1 as a usage error instead of - `scan` rejects `--workers` below 1 as a usage error instead of
running single-threaded (2026-10-04, running single-threaded (2026-10-04,
https://git.eeqj.de/sneak/sfdupes/issues/10) https://git.eeqj.de/sneak/sfdupes/issues/10)
+52 -22
View File
@@ -729,47 +729,77 @@ func TestFeedHashJobsClosesJobsWhenCancelled(t *testing.T) {
// TestHashWorkerDropsQueuedRuns checks that a cancelled hash worker // TestHashWorkerDropsQueuedRuns checks that a cancelled hash worker
// keeps reading jobs and drops the runs rather than reading files // keeps reading jobs and drops the runs rather than reading files
// nobody wants the hashes of — while still letting the range run out // nobody wants the hashes of — while still letting the range run out
// so the pool tears down. The queued run names a file that does not // so the pool tears down. The hash function records that it was
// exist, so a worker that hashed it anyway would produce a result. // called, so a worker that hashed the queued run anyway is caught
// // every time.
// hashWorker's other cancellation exit, abandoning the send of a
// result, is reachable from the scan: stop cancels the pool before it
// drains results, so a worker waiting on that send can leave through
// it. The tests that stop a scan mid-hash, among them
// TestScanHashWriteFailureUnwindsPool, reach it in some runs only,
// depending on timing, and no test fails without it, since stop's
// drain frees a waiting worker anyway. This test, for its part, catches
// a removed drop check in some runs only: a worker that hashes the run
// anyway then picks at random between sending the result and leaving.
func TestHashWorkerDropsQueuedRuns(t *testing.T) { func TestHashWorkerDropsQueuedRuns(t *testing.T) {
t.Parallel() t.Parallel()
done := make(chan struct{}) done := make(chan struct{})
jobs := make(chan []fileRec, 1) jobs := make(chan []fileRec, 1)
results := make(chan hashResult, 1) results := make(chan hashResult)
run := []fileRec{{path: filepath.Join(t.TempDir(), "missing"), size: 1}} jobs <- []fileRec{{path: filepath.Join(t.TempDir(), "missing"), size: 1}}
jobs <- run
close(jobs) close(jobs)
var hashed atomic.Bool
hash := func(path string, size int64) (string, string, string, error) {
hashed.Store(true)
return hashSignature(path, size)
}
go func() { go func() {
defer close(done) defer close(done)
hashWorker(cancelledContext(t), jobs, results, hashSignature) hashWorker(cancelledContext(t), jobs, results, hash)
}() }()
awaitReturn(t, done, "hashWorker") awaitReturn(t, done, "hashWorker")
select { if hashed.Load() {
case r := <-results: t.Error("cancelled hash worker hashed the queued run, want it dropped")
t.Errorf("cancelled hash worker produced %+v, want the run dropped",
r)
default:
} }
} }
// TestHashWorkerAbandonsBlockedSend checks that a hash worker with a
// result to deliver and nobody to deliver it to leaves once the scan
// is cancelled, instead of holding the pool open. The scan tests do
// not catch this: stop drains results, which frees a parked worker
// anyway.
func TestHashWorkerAbandonsBlockedSend(t *testing.T) {
t.Parallel()
ctx, cancel := context.WithCancel(t.Context())
defer cancel()
done := make(chan struct{})
jobs := make(chan []fileRec, 1)
// Unbuffered and unread, with jobs left open: the worker's only way
// out is the cancellation case beside its send.
results := make(chan hashResult)
jobs <- []fileRec{{path: filepath.Join(t.TempDir(), "missing"), size: 1}}
// The scan is cancelled while the worker hashes, so the worker has
// already passed the check that drops queued runs.
hash := func(path string, size int64) (string, string, string, error) {
cancel()
return hashSignature(path, size)
}
go func() {
defer close(done)
hashWorker(ctx, jobs, results, hash)
}()
awaitReturn(t, done, "hashWorker")
}
// TestHashPhaseCancelledReturnsContextError checks the result loop's // TestHashPhaseCancelledReturnsContextError checks the result loop's
// own exit: with the pool cancelled, no result will ever arrive, and // own exit: with the pool cancelled, no result will ever arrive, and
// the loop must leave through the cancellation rather than wait for a // the loop must leave through the cancellation rather than wait for a
+14 -1
View File
@@ -6,6 +6,7 @@ import (
"errors" "errors"
"fmt" "fmt"
"io/fs" "io/fs"
"net/url"
"os" "os"
"path/filepath" "path/filepath"
"slices" "slices"
@@ -107,7 +108,19 @@ const reportParams = "mode=ro" +
// openDB opens the SQLite database at path with the connection // openDB opens the SQLite database at path with the connection
// parameters params. It does not create or verify the schema. // parameters params. It does not create or verify the schema.
func openDB(path, params string) (*sql.DB, error) { func openDB(path, params string) (*sql.DB, error) {
db, err := sql.Open("sqlite", "file:"+path+"?"+params) // The path is escaped into a file: URI, so ?, # and % in it stay
// part of the file name. SQLite reads what follows file:// up to
// the next / as a host name, so an absolute path goes after an
// empty host (file:///abs) and a relative path goes without one
// (file:rel).
uri := url.URL{
Scheme: "file",
OmitHost: !filepath.IsAbs(path),
Path: path,
RawQuery: params,
}
db, err := sql.Open("sqlite", uri.String())
if err != nil { if err != nil {
return nil, fmt.Errorf("open database %s: %w", path, err) return nil, fmt.Errorf("open database %s: %w", path, err)
} }
+68
View File
@@ -8,6 +8,7 @@ import (
"io/fs" "io/fs"
"os" "os"
"path/filepath" "path/filepath"
"slices"
"strconv" "strconv"
"strings" "strings"
"testing" "testing"
@@ -639,6 +640,73 @@ func TestRunReportsNeedOnlyReadAccess(t *testing.T) {
} }
} }
// assertRunsUseDatabase runs scan, then report and trees, against the
// database that SFDUPES_DATABASE names, the file name in dir. It fails
// unless the reports find the duplicate pair the scan recorded and dir
// then holds only that file and its lock file: nothing was created
// under a shortened name.
func assertRunsUseDatabase(t *testing.T, dir, name string) {
t.Helper()
dupes := scanFixture(t)
want := "first\tdupe\tsize\n" + dupes[0] + "\t" + dupes[1] + "\t300\n"
if got := runStdout(t, cmdReport); got != want {
t.Errorf("report stdout = %q, want %q", got, want)
}
want = "first\tdupe\tfiles\tsize\n" +
filepath.Dir(dupes[0]) + "\t" + filepath.Dir(dupes[1]) + "\t1\t300\n"
if got := runStdout(t, cmdTrees); got != want {
t.Errorf("trees stdout = %q, want %q", got, want)
}
entries, err := os.ReadDir(dir)
if err != nil {
t.Fatal(err)
}
got := make([]string, 0, len(entries))
for _, e := range entries {
got = append(got, e.Name())
}
if wantFiles := []string{name, name + ".lock"}; !slices.Equal(got, wantFiles) {
t.Errorf("%s holds %q, want %q", dir, got, wantFiles)
}
}
func TestRunDatabasePathUsedAsGiven(t *testing.T) {
// README §Database: the path names the database file exactly. In
// SQLite's connection string an unescaped ? or # would end the file
// name and % would start an escape, and a path starting with //
// could be read as a host name. %25 is a valid escape, so unescaped
// this name opens a file named a without any error.
const name = "a?b#c%25d e.sqlite"
t.Run("absolute", func(t *testing.T) {
dir := t.TempDir()
t.Setenv(databaseEnv, filepath.Join(dir, name))
assertRunsUseDatabase(t, dir, name)
})
t.Run("leading double slash", func(t *testing.T) {
dir := t.TempDir()
t.Setenv(databaseEnv, "/"+filepath.Join(dir, name))
assertRunsUseDatabase(t, dir, name)
})
t.Run("relative", func(t *testing.T) {
dir := t.TempDir()
t.Chdir(dir)
t.Setenv(databaseEnv, name)
assertRunsUseDatabase(t, dir, name)
})
}
// holdScanLock takes the lock on the database at path, as a running // holdScanLock takes the lock on the database at path, as a running
// scan does, and holds it until the test ends. It fails the test when // scan does, and holds it until the test ends. It fails the test when
// the lock is already held. // the lock is already held.