Keep modules, lint cache and system packages in check builds (closes #109)

Re-vendors REPO_POLICIES.md and script/cibuild, docker, lint and test
from sneak/prompts at 0653579, on sneak's ruling in
sneak/prompts#130. In the Dockerfile's lint,
test and builder stages, the system-package install and the module
download now come before ARG CACHEBUST, so Docker keeps those layers
until the base image, the install line, go.mod or go.sum changes.
golangci-lint run also mounts golangci-lint's cache. Every step that
lints, tests or compiles still runs on every check build, and go test
keeps -count=1 and -race. README and TODO.md say the same.

Model: opus-5-5

Dockerfile

@@ -1,16 +1,23 @@
-# Every stage declares ARG CACHEBUST first: script/ passes a new value
+# In every stage ARG CACHEBUST comes right after the steps that install
-# on every build, which reruns every RUN step after it. The steps that
+# system packages and download Go's modules, or right after FROM in a
-# compile mount Go's build cache, which survives from build to build.
+# stage with neither, and every step that lints, tests or compiles
+# comes after it. script/ passes a new value on every build, which
+# reruns every RUN step after it. Docker keeps the layers above it
+# until the base image, their own lines or go.mod and go.sum change.
+# The steps that compile mount Go's build cache, and golangci-lint run
+# also mounts golangci-lint's; both survive from build to build. The
+# test phase runs as nobody and so mounts Go's build cache at
+# /tmp/.cache/go-build; its own comment says why.
 
 # Lint phase, built alone by script/lint. The tools are invoked directly
 # rather than through `make lint`, which runs docker itself and so cannot
 # run inside a build step.
 # golangci/golangci-lint:v2.14.0, 2026-10-07
 FROM golangci/golangci-lint@sha256:ad862ba6b3798cbe0fd9fd7408d498fd74fbd2623a92406b2fd3898faf0bf98f AS lint
-ARG CACHEBUST
 WORKDIR /src
 COPY go.mod go.sum ./
 RUN go mod download
+ARG CACHEBUST
 COPY . .
 
 # The gofmt half of `make fmt-check`. gofmt's output is assigned to a
@@ -24,6 +31,7 @@ RUN files="$(gofmt -s -l .)" && \
 # Validates .golangci.yml against the schema the pinned binary embeds.
 RUN golangci-lint config verify --config .golangci.yml
 RUN --mount=type=cache,target=/root/.cache/go-build \
+    --mount=type=cache,target=/root/.cache/golangci-lint \
     golangci-lint run --config .golangci.yml ./...
 
 # Test phase, built alone by script/test. -race needs cgo and so a C
@@ -38,12 +46,12 @@ RUN --mount=type=cache,target=/root/.cache/go-build \
 # nobody (65534) so the tests can write to it.
 # golang:1.25-trixie, 2026-10-04
 FROM golang@sha256:2c4c60ef415fbfa5e90300722293bef36c5e63fae17570ce18f580af933dbd73 AS test
-ARG CACHEBUST
 USER nobody
 ENV HOME=/tmp
 WORKDIR /src
 COPY go.mod go.sum ./
 RUN go mod download
+ARG CACHEBUST
 COPY . .
 RUN --mount=type=cache,target=/tmp/.cache/go-build,uid=65534,gid=65534 \
     go test -count=1 -timeout 90s -race -cover ./... || \
@@ -51,19 +59,19 @@ RUN --mount=type=cache,target=/tmp/.cache/go-build,uid=65534,gid=65534 \
       go test -count=1 -timeout 90s -race -v ./...; exit 1; }
 
 # Build stage. Nothing is wanted from either phase above; the copies are
-# what make BuildKit build them first, so this stage cannot run unless
+# what make BuildKit build them first, so nothing after them can run
-# lint and test passed.
+# unless lint and test passed.
 # golang:1.25-alpine, 2026-07-23
 FROM golang@sha256:56961d79ea8129efddcc0b8643fd8a5416b4e6228cfd477e3fd61deb2672c587 AS builder
-ARG CACHEBUST
-COPY --from=lint /src/go.sum /dev/null
-COPY --from=test /src/go.sum /dev/null
 RUN apk add --no-cache git make
-# A tar-stream context keeps the sender's file owners, which git refuses.
-RUN git config --system --add safe.directory /src
 WORKDIR /src
 COPY go.mod go.sum ./
 RUN go mod download
+ARG CACHEBUST
+COPY --from=lint /src/go.sum /dev/null
+COPY --from=test /src/go.sum /dev/null
+# A tar-stream context keeps the sender's file owners, which git refuses.
+RUN git config --system --add safe.directory /src
 COPY . .
 
 # The version stamped into the binary: the VERSION build argument when

README.md

@@ -49,15 +49,40 @@ results are as fresh as the last completed scan.
 
 ### Install
 
-With Go installed, this builds and installs the current `main` branch:
+With Go installed, this builds and installs the newest release:
+
+```sh
+go install sneak.berlin/go/sfdupes@latest
+```
+
+Until 1.0, releases are `v0.0.N` tags on the tip of the `main` branch: each time
+`main` moves and its tip passes `script/cibuild`, that tip is tagged with the
+next `v0.0.N`. So `@latest` installs `main`'s tip once it has passed the build.
+`@main` installs the tip before it is tagged:
 
 ```sh
 go install sneak.berlin/go/sfdupes@main
 ```
 
-The binary goes to `$(go env GOPATH)/bin`, or to `$GOBIN` when that is set. A
+`v0.0.1` is an early build without a database. It does not work as this README
-binary installed this way reports its version as `dev`; one built from a clone
+describes, and `go.mod` retracts it.
-or into the Docker image carries the git tag or commit it was built from.
+
+The binary goes to `$(go env GOPATH)/bin`, or to `$GOBIN` when that is set. To
+see which version an installed binary is:
+
+```sh
+sfdupes --version
+go version -m "$(command -v sfdupes)"
+```
+
+`sfdupes --version` prints `sfdupes VERSION`. For a binary from `go install`,
+`VERSION` is the version Go recorded in it: the release tag, such as `v0.0.3`,
+or for `@main` a pseudo-version ending in the first 12 characters of the
+commit's hash. A binary built from a clone or into the Docker image carries the
+git tag or commit it was built from instead, and `VERSION` is `dev` only when
+the binary carries no version at all. `go version -m` prints what Go recorded in
+the binary; its `mod` line names the module's version. Binaries from `v0.0.2`
+and earlier print `dev`, so check those with `go version -m`.
 
 From a clone, `make build` writes the binary to `./sfdupes`:
 
@@ -262,10 +287,10 @@ sfdupes [command] --help
 CPUs), `-x` (`--one-file-system`) keeps the walk of each operand on that
 operand's filesystem, and `--no-default-excludes` lets the walk enter the
 default excludes, a list of paths; all three are described under "`scan` mode".
-`sfdupes --version` (or `-v`) prints one line, `sfdupes VERSION`, to stdout and
+`sfdupes --version` (or `-v`) prints one line, `sfdupes VERSION` (see
-exits 0, writing nothing to stderr. `-h` or `--help`, alone or after a
+"Install"), to stdout and exits 0, writing nothing to stderr. `-h` or `--help`,
-subcommand, prints the help text to stderr and exits 0, writing nothing to
+alone or after a subcommand, prints the help text to stderr and exits 0, writing
-stdout.
+nothing to stdout.
 
 ### Database
 
@@ -496,8 +521,9 @@ operands:
   can be scanned on different schedules into the same database. The one
   exception is the content phase below: a stored file of 10 MiB or more without
   a `content` hash is read for one, wherever it lies, once its size, `head`, and
-  `tail` match another record's. If that file is gone or has changed since its
+  `tail` match another record's. If that file is gone, has changed since its
-  record was written, the record is left as it is.
+  record was written, or is not on a persistent filesystem, it is not read and
+  the record is left as it is.
 - A record at or below a path the walk left out, because that path is a default
   exclude, is not on a persistent filesystem, or, with `-x`, is a directory on
   another filesystem than its operand, counts as outside the scanned operands:
@@ -550,19 +576,20 @@ the hash phase:
     be read are kept in memory, never every file's hashes. Every record sharing
     their size, `head`, and `tail`, including one that already has a `content`
     hash, has its file checked with `lstat` first. A file that is gone, is no
-    longer a regular file, or has changed (a different size, or an mtime newer
+    longer a regular file, has changed (a different size, or an mtime newer than
-    than recorded) keeps its record as it is and does not count as a match for
+    recorded), or is not on a persistent filesystem (see "Rules for the walk")
-    the others. Any other `lstat` error is warned about and counted as skipped,
+    keeps its record as it is and does not count as a match for the others. Any
-    with the same result. If such a record has no `content` hash, it stays out
+    other `lstat` error is warned about and counted as skipped, with the same
-    of duplicate groups; if it has one, it is still reported until a scan
+    result. If such a record has no `content` hash, it stays out of duplicate
-    covering its own tree updates or removes it. The files that pass and have no
+    groups; if it has one, it is still reported until a scan covering its own
-    `content` hash are read only if at least two of those records pass, so a
+    tree updates or removes it. The files that pass and have no `content` hash
-    file whose only matches are stale costs no read; a file that already has a
+    are read only if at least two of those records pass, so a file whose only
-    `content` hash is never read again. They are read by a worker pool as in the
+    matches are stale costs no read; a file that already has a `content` hash is
-    hash phase, in inode order and once per inode, and their content hashes are
+    never read again. They are read by a worker pool as in the hash phase, in
-    committed in batches. A failed read is warned about and counted as skipped;
+    inode order and once per inode, and their content hashes are committed in
-    its record keeps an empty `content`, so it is not a duplicate, and a later
+    batches. A failed read is warned about and counted as skipped; its record
-    scan tries again.
+    keeps an empty `content`, so it is not a duplicate, and a later scan tries
+    again.
 
 Rules for the walk:
 
@@ -573,6 +600,15 @@ Rules for the walk:
 - Never descend into a directory named `.zfs` (ZFS snapshot pseudo-dirs; walking
   them would list every file once per snapshot), not even when it is an operand;
   such an operand is dropped the same way.
+- Never record the scan's own database. Once the database and its `.lock` file
+  are open, `scan` notes the device and inode of the database file, its `-wal`
+  and `-shm` files and its `.lock` file, and the walk passes over a file with
+  one of them, however it reaches it: as an operand, below one, or through a
+  symlinked directory. It prints no line for such a file and does not count it
+  as skipped. A record that an earlier scan left for one of them is deleted,
+  like the record of any file the scan no longer verifies. There is no
+  `-journal` file during the walk, because the database is in WAL mode then (see
+  "Database").
 - Filesystem boundaries are crossed by default. With `-x` (long form
   `--one-file-system`, following the GNU `du`/`rsync` convention), never descend
   into a directory on a different filesystem than its `PATH` operand; each
@@ -589,15 +625,18 @@ Rules for the walk:
   it names the database and before it takes the lock, `scan` lists the mounted
   filesystems, from `/proc/self/mountinfo` on Linux and from the system's list
   of mounted filesystems on macOS (on any other system it lists none), and notes
-  the device ID of the mount point of each one of those types, read with
+  the device ID of each one of those types. On Linux that is the device ID the
-  `lstat`. Where one filesystem is mounted over another, the one mounted last
+  list gives, so a filesystem that a later mount hides still gives its own. On
-  counts. If `scan` cannot list them, it fails (exit 1). The walk then never
+  macOS it is read with `lstat` on the mount point, and where one filesystem is
-  enters a directory, and never records a file, on one of those devices, however
+  mounted over another on the same mount point, the one mounted last counts. If
-  it reaches it: as an operand, below one, or through a bind mount, which keeps
+  `scan` cannot list them, it fails (exit 1). The walk then never enters a
-  the device ID of what it binds. No flag turns this off. For each operand,
+  directory, and never records a file, on one of those devices, however it
-  directory or file it leaves out this way, `scan` prints a line on stderr
+  reaches it: as an operand, below one, or through a bind mount, which keeps the
-  naming it and the type of its filesystem, as it prints a warning, so it never
+  device ID of what it binds. The content phase never reads a file on one of
-  lands inside a progress display:
+  them either. No flag turns this off. For each operand, directory or file it
+  leaves out this way, `scan` prints a line on stderr naming it and the type of
+  its filesystem, as it prints a warning, so it never lands inside a progress
+  display:
 
     ```
     scan: skipped /dev/shm (tmpfs, not a persistent filesystem)
@@ -1086,13 +1125,22 @@ entrypoints are:
 - `script/install-precommit` — install the git pre-commit hook, as
   `.git/hooks/pre-commit`, that runs `script/precommit`.
 
-Every `docker build` in `script/` passes a new `CACHEBUST` build argument, which
+Every `docker build` in `script/` passes a new `CACHEBUST` build argument. In
-every stage of the `Dockerfile` declares first, so every `RUN` step runs again
+every stage of the `Dockerfile`, `ARG CACHEBUST` comes right after the steps
-each time. On an unchanged tree Docker would otherwise serve the gate steps from
+that install system packages and download the Go modules, or right after `FROM`
-its cache, and the build would pass having run no test and no linter. Go's build
+in a stage with neither, and every step that lints, tests or compiles comes
-cache is kept between builds on the steps that compile, so only what changed is
+after it, so every `RUN` step after it runs again each time. On an unchanged
-compiled again; the tests run with `-count=1`, so none reports a stored result.
+tree Docker would otherwise serve the gate steps from its cache, and the build
-Every run downloads the Go modules again and needs the network.
+would pass having run no test and no linter. The layers above that line stay in
+Docker's build cache until the base image, their own lines, `go.mod` or `go.sum`
+change, or until that cache is cleared; only then are the packages installed and
+the modules downloaded again. Go's build cache is kept between builds on the
+steps that compile, so only what changed is compiled again; the tests run with
+`-count=1`, so none reports a stored result. `golangci-lint run` also keeps
+golangci-lint's cache, and takes a package's findings from it only when the
+package's files and those of the packages it imports, `go.mod`, the linters and
+their settings in `.golangci.yml`, and the golangci-lint version are all
+unchanged.
 
 ## Build
 
@@ -1156,10 +1204,6 @@ All of the following, run in this directory, must pass:
     ./sfdupes report
     ```
 
-    (The database lives in a temp directory of its own: inside `$d`, the scan
-    would record it, and its empty lock file would join the `empty1`/`empty2`
-    group.)
-
     Expected from the first `report`: `one.bin`/`copy.bin`/`copy2.bin` form one
     group (two dupe rows, `first` is the lexicographically smallest path);
     `t1/f1`/`t2/f1`/`t3/f1` form one group; `t1/sub/f2`/

REPO_POLICIES.md

@@ -1,6 +1,6 @@
 ---
 title: Repository Policies
-last_modified: 2026-10-15
+last_modified: 2026-10-18
 ---
 
 This document covers repository structure, tooling, and workflow standards. Code
@@ -159,8 +159,11 @@ style conventions are in separate documents:
     one's findings, and a host-global lock in `$TMPDIR` makes concurrent runs
     exit non-zero with `parallel golangci-lint is running` — a status a caller
     cannot tell from real findings. Both have produced wrong verdicts in this
-    org, in both directions. A container has its own golangci-lint cache, its
+    org, in both directions. A container has its own `TMPDIR`, so the lock is
-    own `TMPDIR` and a digest-pinned binary, so neither is reachable.
+    out of reach. The lint phase does keep golangci-lint's cache between check
+    builds (below), but it lints every checkout at `/src`, and a finding is
+    taken from that cache only for the same files, settings and golangci-lint
+    version.
 
 - **Any build that runs checks is given a new `CACHEBUST` value.** Docker
   invalidates a `COPY` layer only when the copied content changes, so on an
@@ -170,15 +173,21 @@ style conventions are in separate documents:
   id, so no two builds share a value, not even the three in one
   `script/cibuild`. `script/lint`, `script/test`, `script/cibuild` and
   `script/docker` are the four, and there is no fifth — `script/check` runs the
-  two gate phases and `script/fmt-check`, and builds no image of its own. Every
+  two gate phases and `script/fmt-check`, and builds no image of its own. In
-  stage of the `Dockerfile` declares `ARG CACHEBUST` as its first line after
+  every stage of the `Dockerfile`, `ARG CACHEBUST` comes right after the steps
-  `FROM`, and a new value reruns every `RUN` step after that line; `COPY` steps
+  that install system packages and download Go's modules, or right after `FROM`
-  are compared by content and need nothing. A stage without the `ARG` line is
+  in a stage with neither, and every step that lints, tests or compiles comes
-  served from cache on an unchanged tree, so a repository re-vendors the scripts
+  after it. A step that installs anything else as well, such as a
-  and adds the `ARG` lines in one commit. The scripts do not pass `--no-cache`:
+  `script/bootstrap` that also installs node packages, comes after it too. A new
-  it reruns every step too, but it also empties every cache mount, and Go's
+  value reruns every `RUN` step after that line; `COPY` steps are compared by
-  build cache, which the canonical Go `Dockerfile` below mounts so that compiled
+  content and need nothing. Docker keeps the layers above the line until the
-  packages survive between check builds, would start empty every time. A bare
+  base image, their own lines or the files they copy change. A stage without the
+  `ARG` line is served from cache on an unchanged tree, so a repository
+  re-vendors the scripts and adds the `ARG` lines in one commit. The scripts do
+  not pass `--no-cache`: it reruns every step too, but it also builds the layers
+  above the line again and empties every cache mount, so Go's build cache and
+  golangci-lint's, which the canonical Go `Dockerfile` below mounts so that they
+  survive between check builds, would start empty every time. A bare
   `docker build .` is not evidence that anything ran: a sub-second build
   reporting success is a cache hit, not a result. Never invalidate by pruning —
   `docker builder prune` and friends destroy a build cache shared with every
@@ -194,19 +203,25 @@ style conventions are in separate documents:
   `Dockerfile`:
 
     ```dockerfile
-    # Every stage declares ARG CACHEBUST first: script/ passes a new value
+    # In every stage ARG CACHEBUST comes right after the steps that install
-    # on every build, which reruns every RUN step after it. The steps that
+    # system packages and download Go's modules, or right after FROM in a
-    # compile mount Go's build cache, which survives from build to build.
+    # stage with neither, and every step that lints, tests or compiles
+    # comes after it. script/ passes a new value on every build, which
+    # reruns every RUN step after it. Docker keeps the layers above it
+    # until the base image, their own lines or go.mod and go.sum change.
+    # The steps that compile mount Go's build cache, and golangci-lint run
+    # also mounts golangci-lint's; both survive from build to build.
 
     # Lint phase
     # golangci/golangci-lint:v2.x.x, YYYY-MM-DD
     FROM golangci/golangci-lint@sha256:... AS lint
-    ARG CACHEBUST
     WORKDIR /src
     COPY go.mod go.sum ./
     RUN go mod download
+    ARG CACHEBUST
     COPY . .
     RUN --mount=type=cache,target=/root/.cache/go-build \
+        --mount=type=cache,target=/root/.cache/golangci-lint \
         golangci-lint run --config .golangci.yml ./...
 
     # Test phase. -race needs cgo and so a C compiler, which the Debian Go
@@ -214,10 +229,10 @@ style conventions are in separate documents:
     # cache also holds test results, which go test would otherwise replay.
     # golang:1.x, YYYY-MM-DD
     FROM golang@sha256:... AS test
-    ARG CACHEBUST
     WORKDIR /src
     COPY go.mod go.sum ./
     RUN go mod download
+    ARG CACHEBUST
     COPY . .
     RUN --mount=type=cache,target=/root/.cache/go-build \
         go test -count=1 -timeout 90s -race -cover ./... || \
@@ -225,19 +240,19 @@ style conventions are in separate documents:
           go test -count=1 -timeout 90s -race -v ./...; exit 1; }
 
     # Build stage. Nothing is wanted from either phase above; the copies
-    # are what make BuildKit build them first, so this stage cannot run
+    # are what make BuildKit build them first, so nothing after them can
-    # unless lint and test passed.
+    # run unless lint and test passed.
     # golang:1.x-alpine, YYYY-MM-DD
     FROM golang@sha256:... AS builder
-    ARG CACHEBUST
-    COPY --from=lint /src/go.sum /dev/null
-    COPY --from=test /src/go.sum /dev/null
     RUN apk add --no-cache git
-    # A tar-stream context keeps the sender's file owners, which git refuses.
-    RUN git config --system --add safe.directory /src
     WORKDIR /src
     COPY go.mod go.sum ./
     RUN go mod download
+    ARG CACHEBUST
+    COPY --from=lint /src/go.sum /dev/null
+    COPY --from=test /src/go.sum /dev/null
+    # A tar-stream context keeps the sender's file owners, which git refuses.
+    RUN git config --system --add safe.directory /src
     COPY . .
 
     # The VERSION build arg when one is given, otherwise
@@ -274,16 +289,26 @@ style conventions are in separate documents:
     - Keep the runtime stage last, and if you add a stage after it, give it the
       same two copies. A plain `docker build .` builds the last stage's chain
       and nothing else.
-    - Only Go's build cache is mounted (`/root/.cache/go-build`, where Go keeps
+    - Check builds keep these, and none of them can hand back a stale result:
-      it for root), and only on the steps that compile: `golangci-lint run`,
+        - The layers above `ARG CACHEBUST`: the system packages and Go's
-      both `go test` runs and `go build`. It keeps compiled packages between
+          modules. Docker installs the packages again when the base image or the
-      check builds, and Go reuses one only when all its inputs are unchanged, so
+          install line changes, and downloads the modules again when `go.mod` or
-      it saves the compile without skipping a check. Nothing else is kept:
+          `go.sum` changes. Go checks every module against `go.sum` as it
-      golangci-lint's own cache would replay stored findings in place of a lint
+          downloads it, and no later build can change a kept layer. The modules
-      run, and Go builds from whatever files a kept module cache holds without
+          are kept in a layer rather than a cache mount because a cache mount is
-      checking them again (only `go mod verify` does), so a file changed there
+          shared by every build on the host, and Go builds from whatever files
-      would go unnoticed. Go's build cache also holds test results, which is why
+          it holds without checking them again (only `go mod verify` does), so a
-      both `go test` runs carry `-count=1`.
+          file changed there would go unnoticed.
+        - Go's build cache (`/root/.cache/go-build`, where Go keeps it for
+          root), mounted on the steps that compile: `golangci-lint run`, both
+          `go test` runs and `go build`. Go reuses a compiled package only when
+          all its inputs are unchanged. The cache also holds test results, which
+          is why both `go test` runs carry `-count=1`.
+        - golangci-lint's cache (`/root/.cache/golangci-lint`, where the pinned
+          image keeps it), mounted on `golangci-lint run`. golangci-lint reuses
+          a package's findings only when the package's files and those of every
+          package it imports, `go.mod`, the linters and their settings in
+          `.golangci.yml`, and the golangci-lint version are all unchanged.
     - If the project uses `//go:embed` directives that reference build artifacts
       (e.g. a web frontend compiled in a separate stage), the lint phase must
       create placeholder files so the embed directives resolve. Example:
@@ -292,12 +317,20 @@ style conventions are in separate documents:
       in the lint phase. The `golangci/golangci-lint` image is Debian-based and
       has no `apk`, so install with `apt-get` under the Debian package name
       (`libvips-dev`, where alpine says `vips-dev`), and delete the package
-      lists in the same `RUN`, so the layer does not keep them:
+      lists in the same `RUN`, so the layer does not keep them. The test phase,
+      on the Debian Go image, installs the libraries its tests need the same
+      way. Either install goes right after `FROM`, above `ARG CACHEBUST`, so
+      Docker keeps it until the base image or the install line changes:
 
         ```dockerfile
+        FROM golangci/golangci-lint@sha256:... AS lint
         RUN apt-get update \
             && apt-get install -y --no-install-recommends libvips-dev \
             && rm -rf /var/lib/apt/lists/*
+        WORKDIR /src
+        COPY go.mod go.sum ./
+        RUN go mod download
+        ARG CACHEBUST
         ```
 
     - `.dockerignore` lets `.git` into the build context. It keeps out every git

TODO.md

@@ -29,20 +29,55 @@
 
 # Completed Steps
 
-- `scan` never records anything that is not on a persistent filesystem (`tmpfs`,
+- re-vendor `REPO_POLICIES.md` and the four scripts that pass `CACHEBUST` from
-  `proc`, `sysfs` and the other kernel or memory-backed types), matched by the
+  `sneak/prompts` `next` at `0653579`: in the `lint`, `test` and `builder`
-  device ID of each such mount point, listed as the scan starts from
+  stages of the `Dockerfile` the module download, and in `builder` the
-  `/proc/self/mountinfo` on Linux and the mounted-filesystem list on macOS,
+  system-package install, now come before `ARG CACHEBUST`, so Docker keeps them
-  however the walk reaches it; no flag turns that off. Below an operand, its
+  until the base image, the install line, `go.mod` or `go.sum` changes, and
-  walk does not enter the default excludes either, so `sfdupes scan /` skips the
+  every step that lints, tests or compiles still comes after it;
-  kernel's view of the running system and the temporary directories: on Linux
+  `golangci-lint run` also keeps golangci-lint's cache. README says the same
-  `/proc`, `/sys`, `/dev`, `/run`, `/tmp` and `/var/tmp`, on macOS `/dev`,
+  (2026-10-18, https://git.eeqj.de/sneak/sfdupes/issues/109)
-  `/private/tmp`, `/private/var/tmp`, `/private/var/vm`, `/System/Volumes/VM`
+
-  and `/System/Volumes/Data`, and a Windows list, not used yet. An operand at or
+- releases until 1.0 are `v0.0.N` tags on `main`'s tip, so
-  inside a default exclude is still walked. The scan prints a line for each path
+  `go install sneak.berlin/go/sfdupes@latest` installs `main`'s tip once it has
-  it leaves out, without counting it as skipped, and the records there count as
+  passed the build; README "Install" says so, says what `@main` and `v0.0.1`
-  outside the scanned operands, as for `-x`. `--no-default-excludes` turns off
+  give, and how to check an installed binary. `go.mod` retracts `v0.0.1`, the
-  the default excludes only, and `scan --help` lists them (2026-10-17,
+  early build without a database. When the linker set no version,
+  `sfdupes --version` prints the main module's version that Go recorded in the
+  binary, the tag or the pseudo-version of a `go install` build, and `dev` only
+  when neither is known (2026-10-17,
+  https://git.eeqj.de/sneak/sfdupes/issues/116)
+
+- `scan` creates the database's directory where the database path leads: the
+  path with its last element cut off as text, never cleaned, so a `..` after a
+  symlinked directory is followed through the symlink, as when the file is
+  opened. With `SFDUPES_DATABASE=/srv/link/../new/db.sqlite` and `/srv/link` a
+  symlink to `/data/x`, it creates `/data/new` and the database in it, where it
+  used to create `/srv/new` and then fail (2026-10-17,
+  https://git.eeqj.de/sneak/sfdupes/issues/119)
+
+- `scan` never records its own database: once the database and its `.lock` file
+  are open, it notes the device and inode of the database file, its `-wal` and
+  `-shm` files and its `.lock` file, and the walk passes over a file with one of
+  them wherever the operands reach it, through a symlinked directory included,
+  printing no line and counting nothing. A record an earlier scan left for one
+  of them is deleted (2026-10-17, https://git.eeqj.de/sneak/sfdupes/issues/115)
+
+- `scan` never records or reads anything that is not on a persistent filesystem
+  (`tmpfs`, `proc`, `sysfs` and the other kernel or memory-backed types),
+  matched by the device ID of each such filesystem, listed as the scan starts
+  from `/proc/self/mountinfo` on Linux and the mounted-filesystem list on macOS,
+  however the walk or the content phase reaches it; no flag turns that off.
+  Below an operand, its walk does not enter the default excludes either, so
+  `sfdupes scan /` skips the kernel's view of the running system and the
+  temporary directories: on Linux `/proc`, `/sys`, `/dev`, `/run`, `/tmp` and
+  `/var/tmp`, on macOS `/dev`, `/private/tmp`, `/private/var/tmp`,
+  `/private/var/vm`, `/System/Volumes/VM` and `/System/Volumes/Data`, and a
+  Windows list, not used yet. An operand at or inside a default exclude is still
+  walked. The scan prints a line for each path it leaves out, without counting
+  it as skipped, and the records there count as outside the scanned operands, as
+  for `-x`. `--no-default-excludes` turns off the default excludes only, and
+  `scan --help` lists them (2026-10-17,
   https://git.eeqj.de/sneak/sfdupes/issues/124)
 
 - the rate a progress display shows covers the items completed over the last 5
@@ -128,7 +163,11 @@
   and the build stage installs `git` and `make` itself; a new push cancels the
   workflow's older run on the same branch, and a run stops after 20 minutes;
   `.claude/settings.json` is deleted (2026-10-15,
-  https://git.eeqj.de/sneak/sfdupes/issues/95)
+  https://git.eeqj.de/sneak/sfdupes/issues/95). Since
+  https://git.eeqj.de/sneak/sfdupes/issues/109 the module download in `lint`,
+  `test` and the build stage, and the install of `git` and `make` in the build
+  stage, come before `ARG CACHEBUST`, and Docker keeps them between builds;
+  every step that lints, tests or compiles still runs on every build
 
 - with `-x`, an operand under another operand is walked on its own instead of
   being dropped when a directory from it (from its directory, for a regular
@@ -329,9 +368,9 @@
   a `CHECK_EPOCH` build argument; since
   https://git.eeqj.de/sneak/sfdupes/issues/95 every `docker build` in `script/`
   passes a new `CACHEBUST` value instead, which every stage of the `Dockerfile`
-  declares first. Run as root, the tests fail
+  declares before every step that lints, tests or compiles. Run as root, the
-  `TestScanHardlinkRunFailsTogether`, because root reads through the `chmod(0)`
+  tests fail `TestScanHardlinkRunFailsTogether`, because root reads through the
-  the test relies on, so the `test` phase runs them as `nobody`
+  `chmod(0)` the test relies on, so the `test` phase runs them as `nobody`
 - check the installed golangci-lint version in `script/bootstrap` instead of
   only its presence (2026-08-09, branch `bootstrap-version-check`, closes
   https://git.eeqj.de/sneak/sfdupes/issues/24): the version lives only in

db.go

@@ -109,10 +109,8 @@ func databasePath() (string, string, error) {
 	}
 
 	// The working directory as the kernel reports it, with no symlink
-	// in it. os.Getwd may return PWD instead, which can name it through
+	// in it, as README "Database" says. os.Getwd may return PWD
-	// a symlink; a ".." after that symlink then leads somewhere else
+	// instead, which can name it through a symlink.
-	// once the path is cleaned, as scan cleans it to create the
-	// database's directory.
 	wd, err := unix.Getwd()
 	if err != nil {
 		return "", "", fmt.Errorf("resolve database path %s: %w", path, err)
@@ -125,6 +123,24 @@ func databasePath() (string, string, error) {
 	return strings.TrimSuffix(wd, "/") + "/" + path, source, nil
 }
 
+// createDatabaseDirectory creates the directory the database at path
+// goes in, and any directory above it, if missing. path is absolute, as
+// databasePath returns it. The directory is path with its last element
+// cut off as text, by filepath.Split. filepath.Dir would clean it,
+// dropping each ".." together with the directory name before it; when
+// that name is a symlink, the cleaned path names another directory than
+// the one the system reaches when it opens path.
+func createDatabaseDirectory(path string) error {
+	dir, _ := filepath.Split(path)
+
+	err := os.MkdirAll(dir, dbDirPerm)
+	if err != nil {
+		return fmt.Errorf("create database directory: %w", err)
+	}
+
+	return nil
+}
+
 // scanParams are the connection parameters for scan: read-write, with
 // WAL journaling and a busy timeout, so a report can run while a cron
 // scan is in progress. closeScanDatabase leaves WAL mode again.
@@ -179,9 +195,9 @@ func openDB(path, params string) (*sql.DB, error) {
 // that deleted it would let the next scan lock a new file while
 // another still holds the old one.
 func lockScanDatabase(path string) (*os.File, error) {
-	err := os.MkdirAll(filepath.Dir(path), dbDirPerm)
+	err := createDatabaseDirectory(path)
 	if err != nil {
-		return nil, fmt.Errorf("create database directory: %w", err)
+		return nil, err
 	}
 
 	// Like SQLite, this follows a symlink at path and creates its
@@ -225,9 +241,9 @@ func lockScanDatabase(path string) (*os.File, error) {
 // openScanDatabase opens the database for the scan subcommand, creating
 // the file, its parent directory, and the schema as needed.
 func openScanDatabase(ctx context.Context, path string) (*sql.DB, error) {
-	err := os.MkdirAll(filepath.Dir(path), dbDirPerm)
+	err := createDatabaseDirectory(path)
 	if err != nil {
-		return nil, fmt.Errorf("create database directory: %w", err)
+		return nil, err
 	}
 
 	db, err := openDB(path, scanParams)
@@ -245,6 +261,42 @@ func openScanDatabase(ctx context.Context, path string) (*sql.DB, error) {
 	return db, nil
 }
 
+// databaseFiles returns, by device and inode, the files of the database
+// at path that the scan keeps open: the database file, its -wal and
+// -shm files and its .lock file, each named from the database's path
+// with symlinks resolved, as SQLite and lockScanDatabase name them. The
+// scan calls it once the database and the lock file are open, so that
+// its walk passes over them however it reaches them, through a
+// symlinked directory included. A file that does not exist then is left
+// out, since no walk can reach it. There is no -journal file while the
+// database is in WAL mode.
+func databaseFiles(path string) ([]fileRec, error) {
+	resolved, err := filepath.EvalSymlinks(path)
+	if err != nil {
+		return nil, fmt.Errorf("resolve database path: %w", err)
+	}
+
+	var files []fileRec
+
+	for _, p := range []string{
+		resolved, resolved + "-wal", resolved + "-shm", resolved + ".lock",
+	} {
+		fi, err := os.Lstat(p)
+		if errors.Is(err, fs.ErrNotExist) {
+			continue
+		}
+
+		if err != nil {
+			return nil, fmt.Errorf("database: %w", err)
+		}
+
+		dev, ino := inodeOfInfo(fi)
+		files = append(files, fileRec{path: p, dev: dev, ino: ino})
+	}
+
+	return files, nil
+}
+
 // closeScanDatabase switches the database at path from WAL back to
 // rollback-journal mode and closes it. Out of WAL mode the database
 // file alone holds the whole database, so a reader needs no -wal or

db_test.go

@@ -5,6 +5,7 @@ import (
 	"database/sql"
 	"errors"
 	"fmt"
+	"io/fs"
 	"os"
 	"path/filepath"
 	"slices"
@@ -118,6 +119,116 @@ func TestOpenScanDatabaseCreates(t *testing.T) {
 	}
 }
 
+// TestScanCreatesDatabaseDirectoryAfterSymlink has a database path go
+// through a symlink to a directory, then "..", to a missing directory.
+// With srv/link a symlink to data/x, srv/link/../new/db.sqlite leads to
+// data/new/db.sqlite. lockScanDatabase and openScanDatabase, the two
+// places scan creates the database's directory, must each create the
+// database there, and nothing at srv/new, where the path leads once
+// cleaned as text.
+func TestScanCreatesDatabaseDirectoryAfterSymlink(t *testing.T) {
+	t.Parallel()
+
+	creates := map[string]func(ctx context.Context, path string) error{
+		"lockScanDatabase": func(_ context.Context, path string) error {
+			lock, err := lockScanDatabase(path)
+			if err == nil {
+				_ = lock.Close()
+			}
+
+			return err
+		},
+		"openScanDatabase": func(ctx context.Context, path string) error {
+			db, err := openScanDatabase(ctx, path)
+			if err == nil {
+				_ = db.Close()
+			}
+
+			return err
+		},
+	}
+
+	for name, create := range creates {
+		t.Run(name, func(t *testing.T) {
+			t.Parallel()
+
+			dir, err := filepath.EvalSymlinks(t.TempDir())
+			if err != nil {
+				t.Fatal(err)
+			}
+
+			err = os.MkdirAll(filepath.Join(dir, "data", "x"), 0o750)
+			if err != nil {
+				t.Fatal(err)
+			}
+
+			err = os.Mkdir(filepath.Join(dir, "srv"), 0o750)
+			if err != nil {
+				t.Fatal(err)
+			}
+
+			err = os.Symlink(filepath.Join(dir, "data", "x"),
+				filepath.Join(dir, "srv", "link"))
+			if err != nil {
+				t.Fatal(err)
+			}
+
+			err = create(t.Context(), dir+"/srv/link/../new/db.sqlite")
+			if err != nil {
+				t.Fatalf("%s: %v", name, err)
+			}
+
+			_, err = os.Stat(filepath.Join(dir, "data", "new", "db.sqlite"))
+			if err != nil {
+				t.Errorf("database the path leads to: %v", err)
+			}
+
+			_, err = os.Lstat(filepath.Join(dir, "srv", "new"))
+			if !errors.Is(err, fs.ErrNotExist) {
+				t.Errorf("srv/new: stat error %v, want it missing", err)
+			}
+		})
+	}
+}
+
+// TestDatabaseFiles opens a database through a symlink to its file:
+// databaseFiles finds the database file and its -wal and -shm files
+// beside the symlink's target, where SQLite keeps them, and leaves out
+// the .lock file, which only a scan creates.
+func TestDatabaseFiles(t *testing.T) {
+	t.Parallel()
+
+	path := testDBPath(t)
+	link := filepath.Join(t.TempDir(), "link.sqlite")
+
+	err := os.Symlink(path, link)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	db, err := openScanDatabase(t.Context(), link)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	defer func() { _ = db.Close() }()
+
+	files, err := databaseFiles(link)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	got := make([]string, 0, len(files))
+	for _, f := range files {
+		got = append(got, f.path)
+	}
+
+	want := []string{path, path + "-wal", path + "-shm"}
+	if !slices.Equal(got, want) {
+		t.Errorf("databaseFiles(%s) = %q, want %q", link, got, want)
+	}
+}
+
 func TestOpenDatabaseUnversionedForeign(t *testing.T) {
 	t.Parallel()
 

exclude.go

@@ -3,8 +3,8 @@ package main
 // This file holds two rules for the walk (README "Rules for the walk"):
 // the default excludes, the paths a scan's walk does not enter below an
 // operand unless scan is given --no-default-excludes, and the
-// filesystems that are not persistent, on which the walk never records
+// filesystems that are not persistent, on which the scan never records
-// anything.
+// or reads anything.
 
 import (
 	"bufio"
@@ -15,6 +15,8 @@ import (
 	"path/filepath"
 	"slices"
 	"strings"
+
+	"golang.org/x/sys/unix"
 )
 
 // The systems that have default excludes, as runtime.GOOS names them.
@@ -25,11 +27,11 @@ const (
 )
 
 // The fields of a line of mountInfoPath that readMountInfo reads,
-// counted from 0 (see proc(5)): the mount point is the fifth field, and
+// counted from 0 (see proc(5)): the device ID, as major:minor, is the
-// after the sixth come optional fields, then a field holding "-", then
+// third field, and after the sixth come optional fields, then a field
-// the filesystem type.
+// holding "-", then the filesystem type.
 const (
-	mountPointField     = 4
+	deviceField         = 2
 	optionalFieldsStart = 6
 )
 
@@ -41,14 +43,17 @@ const typesPerHelpLine = 6
 // readMountInfo reads.
 var errMountInfoLine = errors.New("unexpected line")
 
-// walkRules holds the two rules for the walk that a scan sets up as it
+// walkRules holds the rules for the walk that a scan sets up as it
 // starts. excludes are the default excludes, or none with
 // --no-default-excludes. nonPersistent maps the device ID of every
-// mounted filesystem that is not persistent to its type; the walk
+// mounted filesystem that is not persistent to its type; the scan
-// records nothing on one of those, whatever the flags.
+// records and reads nothing on one of those, whatever the flags.
+// databaseFiles are the scan's own database files, which the walk
+// passes over (see databaseFiles in db.go).
 type walkRules struct {
 	excludes      []string
 	nonPersistent map[uint64]string
+	databaseFiles []fileRec
 }
 
 // defaultExcludes returns the default excludes of the system goos
@@ -105,32 +110,6 @@ func nonPersistentFilesystemTypes() []string {
 	}
 }
 
-// nonPersistentFilesystems returns the device ID of every mounted
-// filesystem that is not persistent, with its type, read with lstat on
-// its mount point. mountedFilesystems lists the mounts. A mount point
-// that lstat cannot reach is left out.
-func nonPersistentFilesystems() (map[uint64]string, error) {
-	mounts, err := mountedFilesystems()
-	if err != nil {
-		return nil, err
-	}
-
-	types := nonPersistentFilesystemTypes()
-	devices := make(map[uint64]string)
-
-	for mountPoint, fsType := range mounts {
-		if !slices.Contains(types, fsType) {
-			continue
-		}
-
-		if dev, ok := deviceOfPath(mountPoint); ok {
-			devices[dev] = fsType
-		}
-	}
-
-	return devices, nil
-}
-
 // nonPersistentLine returns the line the scan prints for path when fi,
 // its lstat, puts it on a filesystem in rules.nonPersistent, and ""
 // when it is on any other.
@@ -150,16 +129,13 @@ func (rules walkRules) nonPersistentLine(path string, fi fs.FileInfo) string {
 }
 
 // readMountInfo reads mountinfo, laid out as mountInfoPath is, and
-// returns every mount point with the type of its filesystem. Where one
+// returns the device ID of every mounted filesystem that is not
-// filesystem is mounted over another, a later line names the one on
+// persistent, with its type. The device ID is the one each line gives,
-// top, so the last line for a mount point decides.
+// not that of the file now at its mount point, so a mount that a later
-func readMountInfo(mountinfo io.Reader) (map[string]string, error) {
+// mount hides never adds the device of another filesystem.
-	// The kernel writes a space, tab, newline or backslash in a mount
+func readMountInfo(mountinfo io.Reader) (map[uint64]string, error) {
-	// point as an octal escape.
+	types := nonPersistentFilesystemTypes()
-	unescape := strings.NewReplacer(
+	devices := make(map[uint64]string)
-		`\040`, " ", `\011`, "\t", `\012`, "\n", `\134`, `\`)
-
-	mounts := make(map[string]string)
 	lines := bufio.NewScanner(mountinfo)
 
 	for lines.Scan() {
@@ -170,10 +146,20 @@ func readMountInfo(mountinfo io.Reader) (map[string]string, error) {
 			return nil, fmt.Errorf("%w: %q", errMountInfoLine, lines.Text())
 		}
 
-		mounts[unescape.Replace(fields[mountPointField])] = fields[sep+1]
+		var major, minor uint32
+
+		_, err := fmt.Sscanf(fields[deviceField], "%d:%d", &major, &minor)
+		if err != nil {
+			return nil, fmt.Errorf("%w: %q", errMountInfoLine, lines.Text())
+		}
+
+		fsType := fields[sep+1]
+		if slices.Contains(types, fsType) {
+			devices[unix.Mkdev(major, minor)] = fsType
+		}
 	}
 
-	return mounts, lines.Err()
+	return devices, lines.Err()
 }
 
 // defaultExcludesHelp describes, for the scan command's help, the

exclude_darwin.go

@@ -2,12 +2,19 @@
 
 package main
 
-import "golang.org/x/sys/unix"
+import (
+	"slices"
 
-// mountedFilesystems returns every mount point with the type of its
+	"golang.org/x/sys/unix"
-// filesystem, from the system's list of mounted filesystems. Where one
+)
-// filesystem is mounted over another, the later one in the list counts.
+
-func mountedFilesystems() (map[string]string, error) {
+// nonPersistentFilesystems returns the device ID of every mounted
+// filesystem that is not persistent, with its type, from the system's
+// list of mounted filesystems. The device ID is read with lstat on the
+// mount point. Where one filesystem is mounted over another on the same
+// mount point, the later one in the list counts. A mount point that
+// lstat cannot reach is left out.
+func nonPersistentFilesystems() (map[uint64]string, error) {
 	// The first call counts the filesystems, the second lists them.
 	n, err := unix.Getfsstat(nil, unix.MNT_NOWAIT)
 	if err != nil {
@@ -28,5 +35,18 @@ func mountedFilesystems() (map[string]string, error) {
 			unix.ByteSliceToString(st.Fstypename[:])
 	}
 
-	return mounts, nil
+	types := nonPersistentFilesystemTypes()
+	devices := make(map[uint64]string)
+
+	for mountPoint, fsType := range mounts {
+		if !slices.Contains(types, fsType) {
+			continue
+		}
+
+		if dev, ok := deviceOfPath(mountPoint); ok {
+			devices[dev] = fsType
+		}
+	}
+
+	return devices, nil
 }

exclude_other.go

@@ -12,12 +12,12 @@ import (
 // process sees.
 const mountInfoPath = "/proc/self/mountinfo"
 
-// mountedFilesystems returns every mount point with the type of its
+// nonPersistentFilesystems returns the device ID of every mounted
-// filesystem: on Linux read from mountInfoPath (see readMountInfo), and
+// filesystem that is not persistent, with its type: on Linux read from
-// on any other system none.
+// mountInfoPath (see readMountInfo), and on any other system none.
-func mountedFilesystems() (map[string]string, error) {
+func nonPersistentFilesystems() (map[uint64]string, error) {
 	if runtime.GOOS != goosLinux {
-		return map[string]string{}, nil
+		return map[uint64]string{}, nil
 	}
 
 	f, err := os.Open(mountInfoPath)
@@ -27,10 +27,10 @@ func mountedFilesystems() (map[string]string, error) {
 
 	defer func() { _ = f.Close() }()
 
-	mounts, err := readMountInfo(f)
+	devices, err := readMountInfo(f)
 	if err != nil {
 		return nil, fmt.Errorf("read %s: %w", mountInfoPath, err)
 	}
 
-	return mounts, nil
+	return devices, nil
 }

exclude_test.go

@@ -10,6 +10,8 @@ import (
 	"slices"
 	"strings"
 	"testing"
+
+	"golang.org/x/sys/unix"
 )
 
 func TestDefaultExcludes(t *testing.T) {
@@ -61,16 +63,17 @@ func TestIsDefaultExclude(t *testing.T) {
 func TestReadMountInfo(t *testing.T) {
 	t.Parallel()
 
-	// The second-to-last line has no optional fields. /mnt/over has an
+	// The second-to-last line has no optional fields. The tmpfs on
-	// ext4 filesystem mounted over its tmpfs.
+	// /data/a/b is hidden by the root filesystem's /srv, mounted over
+	// /data/a after it: the tmpfs still gives its own device, and the
+	// root filesystem's device is not listed.
 	mountinfo := strings.Join([]string{
 		"22 1 8:1 / / rw,relatime shared:1 - ext4 /dev/sda1 rw",
 		"23 22 0:21 / /proc rw,nosuid shared:12 - proc none rw",
 		"24 22 0:22 / /sys rw,nosuid shared:7 - sysfs none rw",
-		`25 22 0:23 / /mnt/ram\040disk rw shared:2 - tmpfs none rw`,
+		"25 22 0:23 / /data/a/b rw shared:2 - tmpfs none rw",
-		"26 22 0:24 / /mnt/over rw shared:3 - tmpfs none rw",
+		"26 22 8:1 /srv /data/a rw - ext4 /dev/sda1 rw",
-		"27 26 8:2 / /mnt/over rw - ext4 /dev/sda2 rw",
+		"27 22 0:25 / /srv/auto rw shared:4 - autofs systemd-1 rw",
-		"28 22 0:25 / /srv/auto rw shared:4 - autofs systemd-1 rw",
 	}, "\n")
 
 	got, err := readMountInfo(strings.NewReader(mountinfo))
@@ -78,18 +81,23 @@ func TestReadMountInfo(t *testing.T) {
 		t.Fatal(err)
 	}
 
-	want := map[string]string{
+	want := map[uint64]string{
-		"/": "ext4", "/proc": "proc", "/sys": "sysfs",
+		unix.Mkdev(0, 21): "proc", unix.Mkdev(0, 22): "sysfs",
-		"/mnt/ram disk": "tmpfs", "/mnt/over": "ext4", "/srv/auto": "autofs",
+		unix.Mkdev(0, 23): "tmpfs", unix.Mkdev(0, 25): "autofs",
 	}
 	if !maps.Equal(got, want) {
-		t.Errorf("readMountInfo = %q, want %q", got, want)
+		t.Errorf("readMountInfo = %v, want %v", got, want)
 	}
 
-	_, err = readMountInfo(strings.NewReader("22 1 8:1 / / rw,relatime\n"))
+	for _, line := range []string{
-	if !errors.Is(err, errMountInfoLine) {
+		"22 1 8:1 / / rw,relatime\n",
-		t.Errorf("readMountInfo of a line without a type = %v, want %v",
+		"22 1 sda / / rw,relatime shared:1 - ext4 /dev/sda1 rw\n",
-			err, errMountInfoLine)
+	} {
+		_, err = readMountInfo(strings.NewReader(line))
+		if !errors.Is(err, errMountInfoLine) {
+			t.Errorf("readMountInfo(%q) = %v, want %v", line, err,
+				errMountInfoLine)
+		}
 	}
 }
 
@@ -442,6 +450,51 @@ func TestScanKeepsRecordsOnNonPersistent(t *testing.T) {
 	}
 }
 
+// TestScanContentSkipsNonPersistent checks that the content phase does
+// not read a file on a filesystem that is not persistent, though an
+// earlier scan left a record for it: it is treated as a changed file,
+// so its record is left as it is and its identical copy in another
+// tree, having no other match, is not read either.
+func TestScanContentSkipsNonPersistent(t *testing.T) {
+	t.Parallel()
+
+	shm, _, rules := shmTempDir(t)
+	db := openTestDB(t)
+
+	// A scan that, without the list of filesystems, records a with a
+	// head and tail but no content hash.
+	a := sparseFileWithoutMatch(t, shm, "a", headTailMin)
+
+	syncTree(t, db, shm)
+
+	before := recordByPath(t, dbRecords(t, db), a)
+	if before.head == "" || before.content != "" {
+		t.Fatalf("after scanning %s: %+v, want head and tail only", shm, before)
+	}
+
+	b := sparseFile(t, t.TempDir(), "b", headTailMin)
+
+	st, err := syncScan(t.Context(), db, []string{filepath.Dir(b)}, 1, false,
+		rules)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	if st.skipped != 0 {
+		t.Errorf("skipped = %d, want 0", st.skipped)
+	}
+
+	recs := dbRecords(t, db)
+	if r := recordByPath(t, recs, a); r != before {
+		t.Errorf("record = %+v, want it left as %+v", r, before)
+	}
+
+	if r := recordByPath(t, recs, b); r.content != "" {
+		t.Errorf("%s: content = %q, want none: its only match is on %s",
+			b, r.content, shm)
+	}
+}
+
 // TestWalkLeavesOutNonPersistent starts the walk itself in a directory
 // on a filesystem that is not persistent, which a scan never does: it
 // drops such an operand first. Below a directory on a persistent

go.mod

@@ -24,3 +24,5 @@ require (
 	modernc.org/mathutil v1.7.1 // indirect
 	modernc.org/memory v1.11.0 // indirect
 )
+
+retract v0.0.1 // early build without a database; does not work as the README describes

main.go

@@ -28,6 +28,7 @@ import (
 	"io"
 	"os"
 	"runtime"
+	"runtime/debug"
 
 	"github.com/spf13/cobra"
 )
@@ -58,10 +59,32 @@ var errNoSubcommand = errors.New("no subcommand")
 var errWorkersBelowOne = errors.New("--workers must be at least 1")
 
 // Version is the build version, injected at link time via -ldflags
-// (see the Makefile); "dev" for a plain go build.
+// (see the Makefile); empty when the linker set none, as for a binary
+// built by go install. buildVersion turns it into what --version prints.
 //
 //nolint:gochecknoglobals // written only by the linker
-var Version = "dev"
+var Version string
+
+// buildVersion returns the version that sfdupes --version prints
+// (README "Install"): the version the linker set, when it set one;
+// otherwise the main module's version that Go recorded in the binary,
+// a tag such as v0.0.2 or the pseudo-version of an @main build; and
+// "dev" when neither is known. info is what debug.ReadBuildInfo
+// returns, nil when the binary carries none. Go records "(devel)" for
+// a build that has no version, such as one inside a copy of the source
+// without git.
+func buildVersion(linkerVersion string, info *debug.BuildInfo) string {
+	if linkerVersion != "" {
+		return linkerVersion
+	}
+
+	if info == nil || info.Main.Version == "" ||
+		info.Main.Version == "(devel)" {
+		return "dev"
+	}
+
+	return info.Main.Version
+}
 
 func main() {
 	// Once the reader of a stdout pipe has gone, as in "sfdupes report |
@@ -116,7 +139,10 @@ func newRootCommand(stdout, stderr io.Writer) *cobra.Command {
 	var showVersion bool
 
 	printVersion := runE(func(context.Context, []string) error {
-		_, err := fmt.Fprintf(stdout, "sfdupes %s\n", Version)
+		info, _ := debug.ReadBuildInfo()
+
+		_, err := fmt.Fprintf(stdout, "sfdupes %s\n",
+			buildVersion(Version, info))
 		if err != nil {
 			return fmt.Errorf("write stdout: %w", err)
 		}

main_test.go

@@ -9,6 +9,7 @@ import (
 	"io/fs"
 	"os"
 	"path/filepath"
+	"runtime/debug"
 	"slices"
 	"strconv"
 	"strings"
@@ -714,6 +715,8 @@ func TestRunVersion(t *testing.T) {
 
 	// README §Subcommands: the version is one line on stdout, with
 	// nothing on stderr, and exits 0.
+	info, _ := debug.ReadBuildInfo()
+
 	for _, arg := range []string{"--version", "-v"} {
 		var stdout, stderr bytes.Buffer
 
@@ -722,7 +725,7 @@ func TestRunVersion(t *testing.T) {
 			t.Errorf("run(%s) = %d, want %d", arg, code, exitOK)
 		}
 
-		want := "sfdupes " + Version + "\n"
+		want := "sfdupes " + buildVersion(Version, info) + "\n"
 		if got := stdout.String(); got != want {
 			t.Errorf("run(%s) stdout = %q, want %q", arg, got, want)
 		}
@@ -733,6 +736,40 @@ func TestRunVersion(t *testing.T) {
 	}
 }
 
+func TestBuildVersion(t *testing.T) {
+	t.Parallel()
+
+	// README "Install": the linker's version wins; otherwise the main
+	// module's version that Go recorded; "dev" when neither is known.
+	tagged := &debug.BuildInfo{Main: debug.Module{Version: "v0.0.2"}}
+	pseudo := &debug.BuildInfo{Main: debug.Module{
+		Version: "v1.0.0-rc2.0.20261016120000-4e4a94d0c2a1",
+	}}
+	devel := &debug.BuildInfo{Main: debug.Module{Version: "(devel)"}}
+
+	cases := []struct {
+		name   string
+		linker string
+		info   *debug.BuildInfo
+		want   string
+	}{
+		{"linker set", "v0.0.2-3-g4e4a94d", tagged, "v0.0.2-3-g4e4a94d"},
+		{"linker set, no build info", "4e4a94d", nil, "4e4a94d"},
+		{"build info with a tag", "", tagged, "v0.0.2"},
+		{"build info with a pseudo-version", "", pseudo,
+			"v1.0.0-rc2.0.20261016120000-4e4a94d0c2a1"},
+		{"(devel)", "", devel, "dev"},
+		{"no build info", "", nil, "dev"},
+	}
+
+	for _, c := range cases {
+		if got := buildVersion(c.linker, c.info); got != c.want {
+			t.Errorf("%s: buildVersion(%q, ...) = %q, want %q",
+				c.name, c.linker, got, c.want)
+		}
+	}
+}
+
 func TestRunVersionWriteFailureIsFatal(t *testing.T) {
 	t.Parallel()
 

scan.go

@@ -118,7 +118,8 @@ func scanPhase(name string) string {
 // rejects anything less. Before the lock is taken, the scan lists the
 // filesystems that are not persistent, on which the walk records
 // nothing; unless noDefaultExcludes is set, the walk does not enter the
-// default excludes either.
+// default excludes either. Once the database is open, the scan notes
+// its files, which the walk passes over.
 func runScan(ctx context.Context, roots []string, workers int,
 	oneFS, noDefaultExcludes bool,
 ) error {
@@ -173,6 +174,11 @@ func runScan(ctx context.Context, roots []string, workers int,
 
 	defer closeScanDatabase(ctx, db, dbPath)
 
+	rules.databaseFiles, err = databaseFiles(dbPath)
+	if err != nil {
+		return err
+	}
+
 	st, err := syncScan(ctx, db, roots, workers, oneFS, rules)
 	if errors.Is(err, context.Canceled) {
 		return interrupted(st.walked)
@@ -435,7 +441,7 @@ func (s *scanState) runPhases(ctx context.Context, roots []string,
 		return err
 	}
 
-	return s.contentPhase(ctx, workers)
+	return s.contentPhase(ctx, workers, rules)
 }
 
 // walkableRoots returns the operands the walk can start from: regular
@@ -710,6 +716,15 @@ func sameInode(a, b fileRec) bool {
 	return (a.dev != 0 || a.ino != 0) && a.dev == b.dev && a.ino == b.ino
 }
 
+// isDatabaseFile reports whether f, a file the walk reached, is one of
+// the scan's own database files in rules, which the walk passes over
+// without a line and without counting it.
+func isDatabaseFile(f fileRec, rules walkRules) bool {
+	return slices.ContainsFunc(rules.databaseFiles, func(d fileRec) bool {
+		return sameInode(d, f)
+	})
+}
+
 // hashPhase hashes every queued file with hashSignature — the head and
 // tail of a file of headTailMin or more, the whole file below that —
 // committing completed records to the database in batches as results
@@ -878,9 +893,12 @@ func (s *scanState) updatePhase(ctx context.Context) error {
 // hash phase's worker pool and their records written back in batches. A
 // failed read is warned about and counted as skipped; the record keeps
 // its empty content, so it is never grouped, and a later scan tries
-// again.
+// again. No file on a filesystem in rules that is not persistent is
-func (s *scanState) contentPhase(ctx context.Context, workers int) error {
+// read.
-	toRead, recs, err := s.contentCandidates(ctx)
+func (s *scanState) contentPhase(ctx context.Context, workers int,
+	rules walkRules,
+) error {
+	toRead, recs, err := s.contentCandidates(ctx, rules)
 	if err != nil {
 		return err
 	}
@@ -915,18 +933,19 @@ func (s *scanState) contentPhase(ctx context.Context, workers int) error {
 // contentCandidates returns the files the content phase reads, and
 // their records by path. Every record contentCandidatesSQL returns has
 // its file checked with lstat, whether or not it already has a content
-// hash: a file that is gone, is no longer a regular file, or has
+// hash: a file that is gone, is no longer a regular file, has changed
-// changed by the walk's rule keeps its record as it is and does not
+// by the walk's rule, or is on a filesystem in rules that is not
-// count as a match for the others, and any other lstat error is warned
+// persistent keeps its record as it is and does not count as a match
-// about and counted as skipped, with the same result. If such a record
+// for the others, and any other lstat error is warned about and counted
-// has no content hash, it stays out of duplicate groups; if it has one,
+// as skipped, with the same result. If such a record has no content
-// it is still reported until a scan covering its own tree updates or
+// hash, it stays out of duplicate groups; if it has one, it is still
-// removes it. The files of a group that pass and have no content hash
+// reported until a scan covering its own tree updates or removes it.
-// are read only if at least minGroupSize of the group's files pass, so
+// The files of a group that pass and have no content hash are read only
-// a group whose other members are all stale costs no reads. Only the
+// if at least minGroupSize of the group's files pass, so a group whose
-// records to be read are kept.
+// other members are all stale costs no reads. Only the records to be
+// read are kept.
 func (s *scanState) contentCandidates(
-	ctx context.Context,
+	ctx context.Context, rules walkRules,
 ) ([]fileRec, map[string]scanRec, error) {
 	// The search and the checks take real time on a large database;
 	// without a display the scan looks hung before the reads begin.
@@ -984,7 +1003,7 @@ func (s *scanState) contentCandidates(
 			first = r
 		}
 
-		f, ok, err := unchangedFile(r)
+		f, ok, err := unchangedFile(r, rules)
 		if err != nil {
 			s.st.skipped++
 
@@ -1018,9 +1037,10 @@ func (s *scanState) contentCandidates(
 
 // unchangedFile lstats the file r names and returns it for reading if
 // it is still the regular file r records: the same size, and an mtime
-// no newer than recorded (the walk's change rule). A file that is gone
+// no newer than recorded (the walk's change rule). A file that is gone,
-// or has changed reports false; any other lstat error is returned.
+// has changed, or is on a filesystem in rules that is not persistent
-func unchangedFile(r scanRec) (fileRec, bool, error) {
+// reports false; any other lstat error is returned.
+func unchangedFile(r scanRec, rules walkRules) (fileRec, bool, error) {
 	fi, err := os.Lstat(r.path)
 	if errors.Is(err, fs.ErrNotExist) {
 		return fileRec{}, false, nil
@@ -1037,6 +1057,12 @@ func unchangedFile(r scanRec) (fileRec, bool, error) {
 
 	dev, ino := inodeOfInfo(fi)
 
+	// The scan reads nothing on a filesystem that is not persistent,
+	// whatever record an earlier scan left for the file.
+	if _, found := rules.nonPersistent[dev]; found {
+		return fileRec{}, false, nil
+	}
+
 	return fileRec{
 		path: r.path, size: r.size, mtime: r.mtime, dev: dev, ino: ino,
 	}, true, nil
@@ -1132,7 +1158,7 @@ func startWalk(ctx context.Context, roots []string, oneFS bool,
 	go func() {
 		initial := make([]dirJob, 0, len(roots))
 		for _, root := range roots {
-			initial = append(initial, seedRoot(ctx, root, events)...)
+			initial = append(initial, seedRoot(ctx, root, rules, events)...)
 		}
 
 		dispatchDirs(ctx, initial, jobs, subdirs)
@@ -1186,13 +1212,14 @@ func operandWarning(root string, fi fs.FileInfo) string {
 }
 
 // seedRoot turns one PATH operand into the walk's starting state: a
-// regular-file operand is statted and emitted directly, and a directory
+// regular-file operand is statted and emitted directly, unless it is
+// one of the scan's own database files in rules, and a directory
 // operand becomes an initial job. walkableRoots has already dropped
 // every other operand. One that has changed into something else since
 // is warned about and skipped here; it is still a root, so the records
 // stored beneath it, other than those under another root, are deleted
 // as unverified.
-func seedRoot(ctx context.Context, root string,
+func seedRoot(ctx context.Context, root string, rules walkRules,
 	events chan<- walkEvent,
 ) []dirJob {
 	fi, err := os.Lstat(root)
@@ -1219,14 +1246,17 @@ func seedRoot(ctx context.Context, root string,
 	}
 
 	dev, ino := inodeOfInfo(fi)
-
+	rec := fileRec{
-	sendEvent(ctx, events, walkEvent{rec: fileRec{
 		path:  root,
 		size:  fi.Size(),
 		mtime: fi.ModTime(),
 		dev:   dev,
 		ino:   ino,
-	}})
+	}
+
+	if !isDatabaseFile(rec, rules) {
+		sendEvent(ctx, events, walkEvent{rec: rec})
+	}
 
 	return nil
 }
@@ -1361,7 +1391,8 @@ func walkOneDir(ctx context.Context, job dirJob, oneFS bool,
 // regular file) between the directory read and the lstat is warned
 // about and skipped. A file on a filesystem in rules that is not
 // persistent, as through a bind mount of one file, is not recorded: it
-// gets an event naming it instead.
+// gets an event naming it instead. One of the scan's own database files
+// in rules gets no event at all.
 func emitFile(ctx context.Context, p string, e fs.DirEntry,
 	rules walkRules, events chan<- walkEvent,
 ) {
@@ -1387,14 +1418,17 @@ func emitFile(ctx context.Context, p string, e fs.DirEntry,
 	}
 
 	dev, ino := inodeOfInfo(info)
-
+	rec := fileRec{
-	sendEvent(ctx, events, walkEvent{rec: fileRec{
 		path:  p,
 		size:  info.Size(),
 		mtime: info.ModTime(),
 		dev:   dev,
 		ino:   ino,
-	}})
+	}
+
+	if !isDatabaseFile(rec, rules) {
+		sendEvent(ctx, events, walkEvent{rec: rec})
+	}
 }
 
 // subdirJob applies the descent rules to directory p: never enter

scan_test.go

@@ -1171,7 +1171,7 @@ func syncOneFilesystem(t *testing.T, db *sql.DB, roots []string,
 	var initial []dirJob
 
 	for _, root := range roots {
-		for _, job := range seedRoot(ctx, root, events) {
+		for _, job := range seedRoot(ctx, root, walkRules{}, events) {
 			job.rootDev = devices[root]
 			initial = append(initial, job)
 		}
@@ -2501,3 +2501,99 @@ func TestUnderRoot(t *testing.T) {
 		}
 	}
 }
+
+// TestRunScanPassesOverDatabase scans a tree that holds the scan's own
+// database, reached directly and through a symlinked directory. The
+// walk passes over the database file, its -wal and -shm files and its
+// .lock file with no line and no count, and the record an earlier scan
+// left for the .lock file is deleted. A rescan of the unchanged tree
+// updates nothing, and report lists none of the database's files.
+func TestRunScanPassesOverDatabase(t *testing.T) {
+	for _, viaSymlink := range []bool{false, true} {
+		t.Run(fmt.Sprintf("via symlink %v", viaSymlink), func(t *testing.T) {
+			base := t.TempDir()
+			tree := filepath.Join(base, "real", "tree")
+			operand := tree
+
+			writeFile(t, tree, "a", pattern(1, 100))
+			writeFile(t, tree, "b", pattern(1, 100))
+			writeFile(t, tree, "empty1", nil)
+			writeFile(t, tree, "empty2", nil)
+
+			if viaSymlink {
+				err := os.Symlink(filepath.Join(base, "real"),
+					filepath.Join(base, "link"))
+				if err != nil {
+					t.Fatal(err)
+				}
+
+				operand = filepath.Join(base, "link", "tree")
+			}
+
+			dbPath := filepath.Join(tree, "db.sqlite")
+			t.Setenv(databaseEnv, dbPath)
+
+			db, err := openScanDatabase(t.Context(), dbPath)
+			if err != nil {
+				t.Fatal(err)
+			}
+
+			err = applyChanges(t.Context(), db, []scanRec{
+				{path: filepath.Join(operand, "db.sqlite.lock")},
+			}, nil, nil)
+			if err != nil {
+				t.Fatal(err)
+			}
+
+			closeScanDatabase(t.Context(), db, dbPath)
+
+			assertScanSummary(t,
+				"4 files seen (4 added, 0 updated, 1 removed, 0 unchanged)",
+				operand)
+			assertScanSummary(t,
+				"4 files seen (0 added, 0 updated, 0 removed, 4 unchanged)",
+				operand)
+
+			report := runStdout(t, cmdReport)
+			if strings.Contains(report, "db.sqlite") ||
+				!strings.Contains(report, filepath.Join(operand, "empty2")) {
+				t.Errorf("report = %q, want the empty files and none of "+
+					"the database's files", report)
+			}
+		})
+	}
+}
+
+// TestRunScanPassesOverDatabaseOperands scans the database file and its
+// .lock file given as operands themselves: neither is recorded or
+// counted.
+func TestRunScanPassesOverDatabaseOperands(t *testing.T) {
+	dbPath := testDBPath(t)
+	t.Setenv(databaseEnv, dbPath)
+
+	// The first scan creates both files.
+	scanOK(t, t.TempDir())
+
+	assertScanSummary(t,
+		"0 files seen (0 added, 0 updated, 0 removed, 0 unchanged)",
+		dbPath, dbPath+".lock")
+}
+
+// assertScanSummary scans operands and fails the test unless the
+// summary line gives counts with nothing skipped, and the line naming
+// the database is the only one that names one of its files.
+func assertScanSummary(t *testing.T, counts string, operands ...string) {
+	t.Helper()
+
+	stderr := scanOK(t, operands...)
+
+	want := "scan: " + counts + ", 0 skipped\n"
+	if !strings.Contains(stderr, want) {
+		t.Errorf("scan %q: stderr = %q, want %q", operands, stderr, want)
+	}
+
+	if strings.Count(stderr, "db.sqlite") != 1 {
+		t.Errorf("scan %q: stderr = %q, want only the line naming the "+
+			"database to name one of its files", operands, stderr)
+	}
+}

script/cibuild

@@ -2,11 +2,12 @@
 # script/cibuild: run the CI build. It bootstraps first: a CI runner
 # checks out and runs this and nothing else, and script/fmt-check runs
 # the formatter on the host, which a pristine checkout cannot do.
-# A new CACHEBUST value reruns every RUN step, since a cached gate phase
+# A new CACHEBUST value reruns every RUN step after the ARG CACHEBUST
-# is a check that did not run; unlike --no-cache, it keeps Go's build cache.
+# line, since a cached gate phase is a check that did not run; unlike
+# --no-cache, it keeps the layers above that line and the cache mounts.
 # This relies on every stage of the Dockerfile declaring ARG CACHEBUST
-# first; a stage without that line is served from cache on an unchanged
+# before every step that lints, tests or compiles; a stage without that
-# tree, with no warning.
+# line is served from cache on an unchanged tree, with no warning.
 set -eu
 
 SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"

script/docker

@@ -1,11 +1,12 @@
 #!/bin/sh
 # script/docker: build the Docker image tagged with the project name.
 # Identical in all repos; the tag comes from script/projectname.
-# A new CACHEBUST value reruns every RUN step, since a cached gate phase
+# A new CACHEBUST value reruns every RUN step after the ARG CACHEBUST
-# is a check that did not run; unlike --no-cache, it keeps Go's build cache.
+# line, since a cached gate phase is a check that did not run; unlike
+# --no-cache, it keeps the layers above that line and the cache mounts.
 # This relies on every stage of the Dockerfile declaring ARG CACHEBUST
-# first; a stage without that line is served from cache on an unchanged
+# before every step that lints, tests or compiles; a stage without that
-# tree, with no warning.
+# line is served from cache on an unchanged tree, with no warning.
 set -eu
 
 SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"

script/lint

@@ -5,11 +5,13 @@
 # make its answer untrustworthy.
 #
 # The phase is not the last stage in the file, so it is built only when
-# --target names it. A new CACHEBUST value reruns every RUN step, since
+# --target names it. A new CACHEBUST value reruns every RUN step after
-# a cached lint layer is a lint that did not run; unlike --no-cache, it
+# the ARG CACHEBUST line, since a cached lint layer is a lint that did
-# keeps Go's build cache. This relies on every stage of the Dockerfile
+# not run; unlike --no-cache, it keeps the layers above that line and
-# declaring ARG CACHEBUST first; a stage without that line is served
+# the cache mounts. This relies on every stage of the Dockerfile
-# from cache on an unchanged tree, with no warning.
+# declaring ARG CACHEBUST before every step that lints, tests or
+# compiles; a stage without that line is served from cache on an
+# unchanged tree, with no warning.
 # --output type=cacheonly writes no image, since nothing uses one.
 set -eu
 

script/test

@@ -3,11 +3,12 @@
 # and this builds that phase alone, on the same terms as script/lint:
 # --target because a phase that is not the last stage is built only when
 # named, and a new CACHEBUST value because a cached test layer is a test
-# that did not run; unlike --no-cache, it keeps Go's build cache. This
+# that did not run; unlike --no-cache, it keeps the layers above the ARG
-# relies on every stage of the Dockerfile declaring ARG CACHEBUST first;
+# CACHEBUST line and the cache mounts. This relies on every stage of the
-# a stage without that line is served from cache on an unchanged tree,
+# Dockerfile declaring ARG CACHEBUST before every step that lints, tests
-# with no warning. --output type=cacheonly writes no image, since
+# or compiles; a stage without that line is served from cache on an
-# nothing uses one.
+# unchanged tree, with no warning. --output type=cacheonly writes no
+# image, since nothing uses one.
 set -eu
 
 SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"