13 Commits

Author SHA1 Message Date
6230bb1c3a Add scripts-to-rule-them-all scaffold (refs #1)
Some checks failed
check / check (push) Failing after 28s
Add the standard STRTA scaffold, mirroring the conformant Go repos:

- script/ POSIX-sh entrypoints (bootstrap, setup, projectname, test,
  lint, fmt, fmt-check, check, docker, cibuild, precommit,
  install-precommit).
- Makefile rewritten as thin shims: .PHONY plus the nine standard
  targets each delegating to script/NAME; repo-specific build, clean,
  and try targets retained.
- .golangci.yml matching the org-standard Go lint config.
- Dockerfile whose build runs make check then make build, so the image
  fails on any check failure.
- .gitea/workflows/check.yml running script/cibuild.

make fmt-check and make test are green. make check is not yet green
because of pre-existing lint findings in the application code, which
are out of scope for this scaffold change; hence refs (not closes).
2026-07-25 18:36:57 +07:00
975116f13d Add standard Workflow section to TODO.md 2026-07-06 21:06:44 +02:00
1bf1c892f4 Add TODO.md 2026-07-06 20:35:51 +02:00
629613de1b Track actual bytes read instead of stale file size
fileMultihash now returns the number of bytes actually read during
hashing. This ensures BytesProcessed reflects the true amount of
data processed, not a potentially stale size from the initial walk.
2026-02-02 13:50:42 -08:00
5c2338d590 Use atomic operations for failure tracking in ProcessCheck
Replace the non-atomic 'bad' bool with atomic comparison of FilesFailed
count before and after the walk. This ensures consistent use of atomic
operations for all shared state and eliminates a potential race if
parallelism is added in the future.
2026-02-02 13:49:12 -08:00
9f86bf1dc1 Detect file modifications during checksum calculation (TOCTOU fix)
- Check file mtime before and after hashing; error if they differ
- Store file's mtime as sumtime instead of wall-clock time
- Use fresh stat for BytesProcessed to get accurate count

This fixes a TOCTOU race where a file could be modified between
hashing and writing the xattr, resulting in a stale checksum.
It also makes sum update comparisons semantically correct by
comparing file mtime against stored mtime rather than wall-clock time.
2026-02-02 13:48:24 -08:00
2e44e5bb78 Return errors from countFiles instead of swallowing them
countFiles and countFilesMultiple now return errors instead of silently
ignoring them. This ensures that issues like non-existent paths or
permission errors are reported early rather than showing a misleading
progress bar with 0 total.
2026-02-02 13:47:40 -08:00
b9d65115c2 Use single progress bar when processing multiple paths
Instead of creating a new progress bar for each path, count total files
across all paths upfront and use a single unified progress bar. This
provides clearer UX when processing multiple directories.
2026-02-02 13:46:18 -08:00
144d2de243 Return error when stdin provides no paths
When using "-" to read paths from stdin, if stdin is empty or contains
only blank lines, return an explicit error instead of silently succeeding
with no work done.
2026-02-02 13:43:18 -08:00
d848c5e51b Remove dead code in symlink handling
filepath.Walk uses Lstat, so symlinks are reported with ModeSymlink set,
never ModeDir. The info.IsDir() check was always false, making the
filepath.SkipDir branch unreachable dead code.
2026-02-02 13:15:39 -08:00
86764abadf Add quiet mode, progress bar, summary report, and stdin support
- Add -q/--quiet flag to suppress all output except errors
- Add progress bar with 250ms refresh, file count, and ETA display
- Print summary report to stderr on completion (files processed, skipped, failed, bytes, duration)
- Support reading paths from stdin with "-" argument (e.g., find | attrsum sum add -)
- Update README with new features and updated TODO section
2026-02-01 04:22:13 -08:00
c856ea25be Support multiple file/directory arguments for all commands
- Change sum add, sum update, check, and clear to accept 1+ paths
- Update README usage examples to show multiple path support
- Add TODO section with planned future improvements
2026-02-01 03:21:36 -08:00
9ad48fb9b0 Update README.md 2025-07-12 08:40:00 +00:00
22 changed files with 1019 additions and 263 deletions

View File

@@ -0,0 +1,9 @@
name: check
on: [push]
jobs:
check:
runs-on: ubuntu-latest
steps:
# actions/checkout v4.2.2, 2026-02-28
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683
- run: script/cibuild

32
.golangci.yml Normal file
View File

@@ -0,0 +1,32 @@
version: "2"
run:
timeout: 5m
modules-download-mode: readonly
linters:
default: all
disable:
# Genuinely incompatible with project patterns
- exhaustruct # Requires all struct fields
- depguard # Dependency allow/block lists
- godot # Requires comments to end with periods
- wsl # Deprecated, replaced by wsl_v5
- wrapcheck # Too verbose for internal packages
- varnamelen # Short names like db, id are idiomatic Go
linters-settings:
lll:
line-length: 88
funlen:
lines: 80
statements: 50
cyclop:
max-complexity: 15
dupl:
threshold: 100
issues:
exclude-use-default: false
max-issues-per-linter: 0
max-same-issues: 0

34
Dockerfile Normal file
View File

@@ -0,0 +1,34 @@
# Build stage
# golang 1.25-alpine, 2026-02-28
FROM golang@sha256:f6751d823c26342f9506c03797d2527668d095b0a15f1862cddb4d927a7a4ced AS builder
RUN apk add --no-cache git make gcc musl-dev binutils-gold
# golangci-lint v2.10.1
RUN go install github.com/golangci/golangci-lint/v2/cmd/golangci-lint@5d1e709b7be35cb2025444e19de266b056b7b7ee
# goimports v0.42.0
RUN go install golang.org/x/tools/cmd/goimports@009367f5c17a8d4c45a961a3a509277190a9a6f0
WORKDIR /src
COPY go.mod go.sum ./
RUN go mod download
COPY . .
# Run all checks - build fails if any check fails
RUN make check
# Build the binary
RUN make build
# Runtime stage
# alpine 3.21, 2026-02-28
FROM alpine@sha256:c3f8e73fdb79deaebaa2037150150191b9dcbfba68b4a46d70103204c53f4709
RUN apk add --no-cache ca-certificates tzdata
WORKDIR /app
COPY --from=builder /src/attrsum /app/attrsum
ENTRYPOINT ["/app/attrsum"]

View File

@@ -1,9 +1,38 @@
.PHONY: default bootstrap setup test lint fmt fmt-check check docker hooks build clean try
TESTDIR := $(HOME)/Documents/_SYSADMIN/cyberdyne TESTDIR := $(HOME)/Documents/_SYSADMIN/cyberdyne
default: test # Standard targets are thin shims; the implementations live in script/
# per the scripts-to-rule-them-all pattern.
default: check build
bootstrap:
@script/bootstrap
setup:
@script/setup
test: test:
@go test ./... -v @script/test
lint:
@script/lint
fmt:
@script/fmt
fmt-check:
@script/fmt-check
check:
@script/check
docker:
@script/docker
hooks:
@script/install-precommit
build: clean build: clean
@go build . @go build .
@@ -21,4 +50,3 @@ try: build
touch $(TESTDIR)/* touch $(TESTDIR)/*
./attrsum sum update -v $(TESTDIR) ./attrsum sum update -v $(TESTDIR)
./attrsum check -v $(TESTDIR) ./attrsum check -v $(TESTDIR)

View File

@@ -31,20 +31,26 @@ Semantic Versioning 2.0.0 is used for tags.
## Usage ## Usage
```bash ```bash
# add checksum & timestamp xattrs to every regular file under DIR # add checksum & timestamp xattrs to every regular file under one or more paths
attrsum sum add DIR attrsum sum add DIR1 DIR2 file.txt
# update checksum only when file mtime is newer than stored sumtime # update checksum only when file mtime is newer than stored sumtime
attrsum sum update DIR attrsum sum update DIR1 DIR2
# verify checksums, stop on first error # verify checksums, stop on first error
attrsum check DIR attrsum check DIR1 DIR2
# verify every file, reporting each result, keep going after errors # verify every file, reporting each result, keep going after errors
attrsum -v check --continue DIR attrsum -v check --continue DIR1 DIR2
# remove checksum & timestamp xattrs # remove checksum & timestamp xattrs
attrsum clear DIR attrsum clear DIR1 DIR2
# read paths from stdin (use - as argument)
find /data -name "*.jpg" | attrsum sum add -
# quiet mode (suppress progress bar and summary)
attrsum -q sum add DIR
``` ```
| xattr key | meaning | | xattr key | meaning |
@@ -55,23 +61,36 @@ attrsum clear DIR
Flags: Flags:
* `-v, --verbose` — per-file log output * `-v, --verbose` — per-file log output
* `-q, --quiet` — suppress all output except errors (no progress bar or summary)
* `--exclude PATTERN` — skip paths matching rsync/Doublestar glob * `--exclude PATTERN` — skip paths matching rsync/Doublestar glob
* `--exclude-dotfiles` — skip any path component that starts with `.` * `--exclude-dotfiles` — skip any path component that starts with `.`
All commands display a progress bar with ETA and print a summary report to stderr on completion (unless `--quiet` is specified).
`attrsum` **never follows symlinks** and skips non-regular files (sockets, devices, …). `attrsum` **never follows symlinks** and skips non-regular files (sockets, devices, …).
--- ---
## Why? ## Why?
Apple APFS and Linux ext3/ext4 **store no per-file metadata checksums**, so Apple APFS and Linux ext3/ext4 **store no per-file content checksums**, so
silent data corruption can pass unnoticed. `attrsum` keeps a portable, silent data corruption can pass unnoticed. `attrsum` keeps a portable checksum **inside each files xattrs**, providing integrity
tamper-evident checksum **inside each files xattrs**, providing integrity
verification that travels with the file itself—no external database verification that travels with the file itself—no external database
required. Now you can trust a USB stick didn't eat your data. required. Now you can trust a USB stick didn't eat your data.
--- ---
## TODO
Future improvements under consideration:
- **Dry-run mode (`--dry-run`, `-n`)** — show what would be done without making changes
- **JSON output (`--json`)** — machine-readable output for scripting and integration
- **Parallel processing (`-j N`)** — use multiple goroutines for faster checksumming on large trees
- **Exit code documentation** — formalize and document exit codes for scripting
---
## Contributing ## Contributing
* Author & maintainer: **sneak** <sneak@sneak.berlin> * Author & maintainer: **sneak** <sneak@sneak.berlin>

55
TODO.md Normal file
View File

@@ -0,0 +1,55 @@
# Workflow
* branch (from `main`)
* do the work in Next Step
* move Next Step to the top of Completed Steps
* move the top item of Future Steps into Next Step
* commit (`TODO.md` changes in the same commit as the work)
* merge to `main` if the branch is not protected, otherwise open a PR
* push
# Status
1.0+
Tagged 1.0.0 (2025-05-08). Substantial correctness fixes and features
have landed since the tag.
# Next Step
Policy scaffold commit: add LICENSE, REPO_POLICIES.md, .editorconfig,
.golangci.yml, and a comprehensive .gitignore (currently only the
attrsum binary), and extend the Makefile (only test/build/clean/try
today) with lint, fmt, fmt-check, check, and hooks targets. Fix the try
target to use a temp fixture instead of the hardcoded
$(HOME)/Documents/_SYSADMIN/cyberdyne path.
# Completed Steps
* 2026-02-02: correctness pass: track actual bytes read instead of
stale file size, atomic failure tracking in ProcessCheck, detect
file modification during checksum (TOCTOU), propagate countFiles
errors, single progress bar across paths, error on empty stdin,
dead code removal
* 2026-02-01: added quiet mode, progress bar, summary report, and stdin
path input; multiple file/directory arguments for all commands
* 2025-07-12: README update
* 2025-05-08: initial working tool with passing tests, skips
non-regular files, Makefile, README; tagged 1.0.0
# Future Steps
* Add Dockerfile and .dockerignore that run make check, images pinned
by sha256, plus a Makefile docker target
* Add .gitea/workflows/check.yml
* Restructure README.md into the standard sections: Description,
Getting Started, Rationale, Design, TODO, License, Author (Getting
Started, Why?, TODO, License exist; Description, Design, Author are
missing)
* Tag a patch release to ship the 2026-02-02 correctness fixes
* Dry-run mode (--dry-run, -n): show what would be done without making
changes (from README TODO)
* JSON output (--json) for scripting and integration (from README TODO)
* Parallel processing (-j N) with multiple goroutines for faster
checksumming on large trees (from README TODO)
* Formalize and document exit codes for scripting (from README TODO)

View File

@@ -1,6 +1,7 @@
package main package main
import ( import (
"bufio"
"bytes" "bytes"
"crypto/sha256" "crypto/sha256"
"errors" "errors"
@@ -10,12 +11,14 @@ import (
"os" "os"
"path/filepath" "path/filepath"
"strings" "strings"
"sync/atomic"
"time" "time"
base58 "github.com/mr-tron/base58/base58"
"github.com/bmatcuk/doublestar/v4" "github.com/bmatcuk/doublestar/v4"
base58 "github.com/mr-tron/base58/base58"
"github.com/multiformats/go-multihash" "github.com/multiformats/go-multihash"
"github.com/pkg/xattr" "github.com/pkg/xattr"
"github.com/schollz/progressbar/v3"
"github.com/spf13/cobra" "github.com/spf13/cobra"
) )
@@ -26,10 +29,51 @@ const (
var ( var (
verbose bool verbose bool
quiet bool
excludePatterns []string excludePatterns []string
excludeDotfiles bool excludeDotfiles bool
) )
// Stats tracks operation statistics for summary reporting
type Stats struct {
FilesProcessed int64
FilesSkipped int64
FilesFailed int64
BytesProcessed int64
StartTime time.Time
}
func (s *Stats) Duration() time.Duration {
return time.Since(s.StartTime)
}
func (s *Stats) Print(operation string) {
if quiet {
return
}
fmt.Fprintf(os.Stderr, "\n%s complete: %d files processed, %d skipped, %d failed, %s bytes in %s\n",
operation,
s.FilesProcessed,
s.FilesSkipped,
s.FilesFailed,
formatBytes(s.BytesProcessed),
s.Duration().Round(time.Millisecond),
)
}
func formatBytes(b int64) string {
const unit = 1024
if b < unit {
return fmt.Sprintf("%d B", b)
}
div, exp := int64(unit), 0
for n := b / unit; n >= unit; n /= unit {
div *= unit
exp++
}
return fmt.Sprintf("%.1f %ciB", float64(b)/float64(div), "KMGTPE"[exp])
}
func main() { func main() {
rootCmd := &cobra.Command{ rootCmd := &cobra.Command{
Use: "attrsum", Use: "attrsum",
@@ -39,6 +83,7 @@ func main() {
rootCmd.SilenceErrors = true rootCmd.SilenceErrors = true
rootCmd.PersistentFlags().BoolVarP(&verbose, "verbose", "v", false, "enable verbose output") rootCmd.PersistentFlags().BoolVarP(&verbose, "verbose", "v", false, "enable verbose output")
rootCmd.PersistentFlags().BoolVarP(&quiet, "quiet", "q", false, "suppress all output except errors")
rootCmd.PersistentFlags().StringArrayVar(&excludePatterns, "exclude", nil, "exclude files/directories matching pattern (rsync-style, repeatable)") rootCmd.PersistentFlags().StringArrayVar(&excludePatterns, "exclude", nil, "exclude files/directories matching pattern (rsync-style, repeatable)")
rootCmd.PersistentFlags().BoolVar(&excludeDotfiles, "exclude-dotfiles", false, "exclude any file or directory whose name starts with '.'") rootCmd.PersistentFlags().BoolVar(&excludeDotfiles, "exclude-dotfiles", false, "exclude any file or directory whose name starts with '.'")
@@ -51,6 +96,36 @@ func main() {
} }
} }
// expandPaths expands the given paths, reading from stdin if "-" is present
func expandPaths(args []string) ([]string, error) {
var paths []string
readFromStdin := false
for _, arg := range args {
if arg == "-" {
readFromStdin = true
scanner := bufio.NewScanner(os.Stdin)
for scanner.Scan() {
line := strings.TrimSpace(scanner.Text())
if line != "" {
paths = append(paths, line)
}
}
if err := scanner.Err(); err != nil {
return nil, fmt.Errorf("reading stdin: %w", err)
}
} else {
paths = append(paths, arg)
}
}
if len(paths) == 0 {
if readFromStdin {
return nil, errors.New("no paths provided on stdin")
}
return nil, errors.New("no paths provided")
}
return paths, nil
}
/////////////////////////////////////////////////////////////////////////////// ///////////////////////////////////////////////////////////////////////////////
// Sum commands // Sum commands
/////////////////////////////////////////////////////////////////////////////// ///////////////////////////////////////////////////////////////////////////////
@@ -62,56 +137,143 @@ func newSumCmd() *cobra.Command {
} }
add := &cobra.Command{ add := &cobra.Command{
Use: "add <dir>", Use: "add <path>... (use - to read paths from stdin)",
Short: "Write checksums for files missing them", Short: "Write checksums for files missing them",
Args: cobra.ExactArgs(1), Args: cobra.MinimumNArgs(1),
RunE: func(_ *cobra.Command, a []string) error { return ProcessSumAdd(a[0]) }, RunE: func(_ *cobra.Command, a []string) error {
paths, err := expandPaths(a)
if err != nil {
return err
}
stats := &Stats{StartTime: time.Now()}
var bar *progressbar.ProgressBar
if !quiet {
total, err := countFilesMultiple(paths)
if err != nil {
return err
}
bar = newProgressBar(total, "Adding checksums")
}
for _, p := range paths {
if err := ProcessSumAdd(p, stats, bar); err != nil {
if bar != nil {
bar.Finish()
}
return err
}
}
if bar != nil {
bar.Finish()
}
stats.Print("sum add")
return nil
},
} }
upd := &cobra.Command{ upd := &cobra.Command{
Use: "update <dir>", Use: "update <path>... (use - to read paths from stdin)",
Short: "Recalculate checksum when file newer than stored sumtime", Short: "Recalculate checksum when file newer than stored sumtime",
Args: cobra.ExactArgs(1), Args: cobra.MinimumNArgs(1),
RunE: func(_ *cobra.Command, a []string) error { return ProcessSumUpdate(a[0]) }, RunE: func(_ *cobra.Command, a []string) error {
paths, err := expandPaths(a)
if err != nil {
return err
}
stats := &Stats{StartTime: time.Now()}
var bar *progressbar.ProgressBar
if !quiet {
total, err := countFilesMultiple(paths)
if err != nil {
return err
}
bar = newProgressBar(total, "Updating checksums")
}
for _, p := range paths {
if err := ProcessSumUpdate(p, stats, bar); err != nil {
if bar != nil {
bar.Finish()
}
return err
}
}
if bar != nil {
bar.Finish()
}
stats.Print("sum update")
return nil
},
} }
cmd.AddCommand(add, upd) cmd.AddCommand(add, upd)
return cmd return cmd
} }
func ProcessSumAdd(dir string) error { func ProcessSumAdd(dir string, stats *Stats, bar *progressbar.ProgressBar) error {
return walkAndProcess(dir, func(p string, _ os.FileInfo) error { return writeChecksumAndTime(p) }) return walkAndProcess(dir, stats, bar, func(p string, info os.FileInfo, s *Stats) error {
if hasXattr(p, checksumKey) {
atomic.AddInt64(&s.FilesSkipped, 1)
return nil
} }
if err := writeChecksumAndTime(p, info, s); err != nil {
func ProcessSumUpdate(dir string) error { atomic.AddInt64(&s.FilesFailed, 1)
return walkAndProcess(dir, func(p string, info os.FileInfo) error { return err
t, err := readSumTime(p)
if err != nil || info.ModTime().After(t) {
return writeChecksumAndTime(p)
} }
return nil return nil
}) })
} }
func writeChecksumAndTime(path string) error { func ProcessSumUpdate(dir string, stats *Stats, bar *progressbar.ProgressBar) error {
hash, err := fileMultihash(path) return walkAndProcess(dir, stats, bar, func(p string, info os.FileInfo, s *Stats) error {
t, err := readSumTime(p)
if err != nil || info.ModTime().After(t) {
if err := writeChecksumAndTime(p, info, s); err != nil {
atomic.AddInt64(&s.FilesFailed, 1)
return err
}
} else {
atomic.AddInt64(&s.FilesSkipped, 1)
}
return nil
})
}
func writeChecksumAndTime(path string, info os.FileInfo, stats *Stats) error {
// Record mtime before hashing to detect modifications during hash
mtimeBefore := info.ModTime()
hash, bytesRead, err := fileMultihash(path)
if err != nil { if err != nil {
return err return err
} }
// Check if file was modified during hashing
infoAfter, err := os.Lstat(path)
if err != nil {
return fmt.Errorf("stat after hash: %w", err)
}
if !infoAfter.ModTime().Equal(mtimeBefore) {
return fmt.Errorf("%s: file modified during checksum calculation", path)
}
if err := xattr.Set(path, checksumKey, hash); err != nil { if err := xattr.Set(path, checksumKey, hash); err != nil {
return fmt.Errorf("set checksum attr: %w", err) return fmt.Errorf("set checksum attr: %w", err)
} }
if verbose { if verbose && !quiet {
fmt.Printf("%s %s written\n", path, hash) fmt.Printf("%s %s written\n", path, hash)
} }
ts := time.Now().UTC().Format(time.RFC3339Nano) // Store the file's mtime as sumtime (not wall-clock time)
// This makes update comparisons semantically correct
ts := mtimeBefore.UTC().Format(time.RFC3339Nano)
if err := xattr.Set(path, sumTimeKey, []byte(ts)); err != nil { if err := xattr.Set(path, sumTimeKey, []byte(ts)); err != nil {
return fmt.Errorf("set sumtime attr: %w", err) return fmt.Errorf("set sumtime attr: %w", err)
} }
if verbose { if verbose && !quiet {
fmt.Printf("%s %s written\n", path, ts) fmt.Printf("%s %s written\n", path, ts)
} }
atomic.AddInt64(&stats.FilesProcessed, 1)
atomic.AddInt64(&stats.BytesProcessed, bytesRead)
return nil return nil
} }
@@ -129,29 +291,66 @@ func readSumTime(path string) (time.Time, error) {
func newClearCmd() *cobra.Command { func newClearCmd() *cobra.Command {
return &cobra.Command{ return &cobra.Command{
Use: "clear <dir>", Use: "clear <path>... (use - to read paths from stdin)",
Short: "Remove checksum xattrs from tree", Short: "Remove checksum xattrs from tree",
Args: cobra.ExactArgs(1), Args: cobra.MinimumNArgs(1),
RunE: func(_ *cobra.Command, a []string) error { return ProcessClear(a[0]) }, RunE: func(_ *cobra.Command, a []string) error {
paths, err := expandPaths(a)
if err != nil {
return err
}
stats := &Stats{StartTime: time.Now()}
var bar *progressbar.ProgressBar
if !quiet {
total, err := countFilesMultiple(paths)
if err != nil {
return err
}
bar = newProgressBar(total, "Clearing checksums")
}
for _, p := range paths {
if err := ProcessClear(p, stats, bar); err != nil {
if bar != nil {
bar.Finish()
}
return err
}
}
if bar != nil {
bar.Finish()
}
stats.Print("clear")
return nil
},
} }
} }
func ProcessClear(dir string) error { func ProcessClear(dir string, stats *Stats, bar *progressbar.ProgressBar) error {
return walkAndProcess(dir, func(p string, _ os.FileInfo) error { return walkAndProcess(dir, stats, bar, func(p string, info os.FileInfo, s *Stats) error {
cleared := false
for _, k := range []string{checksumKey, sumTimeKey} { for _, k := range []string{checksumKey, sumTimeKey} {
v, err := xattr.Get(p, k) v, err := xattr.Get(p, k)
if err != nil { if err != nil {
if errors.Is(err, xattr.ENOATTR) { if errors.Is(err, xattr.ENOATTR) {
continue continue
} }
atomic.AddInt64(&s.FilesFailed, 1)
return err return err
} }
if verbose { if verbose && !quiet {
fmt.Printf("%s %s removed\n", p, string(v)) fmt.Printf("%s %s removed\n", p, string(v))
} }
if err := xattr.Remove(p, k); err != nil { if err := xattr.Remove(p, k); err != nil {
atomic.AddInt64(&s.FilesFailed, 1)
return err return err
} }
cleared = true
}
if cleared {
atomic.AddInt64(&s.FilesProcessed, 1)
atomic.AddInt64(&s.BytesProcessed, info.Size())
} else {
atomic.AddInt64(&s.FilesSkipped, 1)
} }
return nil return nil
}) })
@@ -164,25 +363,59 @@ func ProcessClear(dir string) error {
func newCheckCmd() *cobra.Command { func newCheckCmd() *cobra.Command {
var cont bool var cont bool
cmd := &cobra.Command{ cmd := &cobra.Command{
Use: "check <dir>", Use: "check <path>... (use - to read paths from stdin)",
Short: "Verify stored checksums", Short: "Verify stored checksums",
Args: cobra.ExactArgs(1), Args: cobra.MinimumNArgs(1),
RunE: func(_ *cobra.Command, a []string) error { return ProcessCheck(a[0], cont) }, RunE: func(_ *cobra.Command, a []string) error {
paths, err := expandPaths(a)
if err != nil {
return err
}
stats := &Stats{StartTime: time.Now()}
var bar *progressbar.ProgressBar
if !quiet {
total, err := countFilesMultiple(paths)
if err != nil {
return err
}
bar = newProgressBar(total, "Verifying checksums")
}
var finalErr error
for _, p := range paths {
if err := ProcessCheck(p, cont, stats, bar); err != nil {
if cont {
finalErr = err
} else {
if bar != nil {
bar.Finish()
}
stats.Print("check")
return err
}
}
}
if bar != nil {
bar.Finish()
}
stats.Print("check")
return finalErr
},
} }
cmd.Flags().BoolVar(&cont, "continue", false, "continue after errors and report each file") cmd.Flags().BoolVar(&cont, "continue", false, "continue after errors and report each file")
return cmd return cmd
} }
func ProcessCheck(dir string, cont bool) error { func ProcessCheck(dir string, cont bool, stats *Stats, bar *progressbar.ProgressBar) error {
fail := errors.New("verification failed") fail := errors.New("verification failed")
bad := false // Track initial failed count to detect failures during this walk
initialFailed := atomic.LoadInt64(&stats.FilesFailed)
err := walkAndProcess(dir, func(p string, _ os.FileInfo) error { err := walkAndProcess(dir, stats, bar, func(p string, info os.FileInfo, s *Stats) error {
exp, err := xattr.Get(p, checksumKey) exp, err := xattr.Get(p, checksumKey)
if err != nil { if err != nil {
if errors.Is(err, xattr.ENOATTR) { if errors.Is(err, xattr.ENOATTR) {
bad = true atomic.AddInt64(&s.FilesFailed, 1)
if verbose { if verbose && !quiet {
fmt.Printf("%s <none> ERROR\n", p) fmt.Printf("%s <none> ERROR\n", p)
} }
if cont { if cont {
@@ -193,15 +426,19 @@ func ProcessCheck(dir string, cont bool) error {
return err return err
} }
act, err := fileMultihash(p) act, bytesRead, err := fileMultihash(p)
if err != nil { if err != nil {
atomic.AddInt64(&s.FilesFailed, 1)
return err return err
} }
ok := bytes.Equal(exp, act) ok := bytes.Equal(exp, act)
if !ok { if !ok {
bad = true atomic.AddInt64(&s.FilesFailed, 1)
} else {
atomic.AddInt64(&s.FilesProcessed, 1)
atomic.AddInt64(&s.BytesProcessed, bytesRead)
} }
if verbose { if verbose && !quiet {
status := "OK" status := "OK"
if !ok { if !ok {
status = "ERROR" status = "ERROR"
@@ -220,7 +457,8 @@ func ProcessCheck(dir string, cont bool) error {
} }
return err return err
} }
if bad { // Check if any failures occurred during this walk
if atomic.LoadInt64(&stats.FilesFailed) > initialFailed {
return fail return fail
} }
return nil return nil
@@ -230,21 +468,87 @@ func ProcessCheck(dir string, cont bool) error {
// Helpers // Helpers
/////////////////////////////////////////////////////////////////////////////// ///////////////////////////////////////////////////////////////////////////////
func walkAndProcess(root string, fn func(string, os.FileInfo) error) error { // countFiles counts the total number of regular files that will be processed
func countFiles(root string) (int64, error) {
var count int64
root = filepath.Clean(root) root = filepath.Clean(root)
return filepath.Walk(root, func(p string, info os.FileInfo, err error) error { err := filepath.Walk(root, func(p string, info os.FileInfo, err error) error {
if err != nil {
return err
}
// Skip symlinks - note: filepath.Walk uses Lstat, so symlinks are
// reported as ModeSymlink, never as directories. Walk doesn't follow them.
if info.Mode()&os.ModeSymlink != 0 {
return nil
}
rel, _ := filepath.Rel(root, p)
if shouldExclude(rel, info) {
if info.IsDir() {
return filepath.SkipDir
}
return nil
}
if info.IsDir() {
return nil
}
if !info.Mode().IsRegular() {
return nil
}
count++
return nil
})
return count, err
}
// countFilesMultiple counts files across multiple roots
func countFilesMultiple(roots []string) (int64, error) {
var total int64
for _, root := range roots {
count, err := countFiles(root)
if err != nil {
return total, err
}
total += count
}
return total, nil
}
// newProgressBar creates a new progress bar with standard options
func newProgressBar(total int64, description string) *progressbar.ProgressBar {
return progressbar.NewOptions64(total,
progressbar.OptionSetDescription(description),
progressbar.OptionSetWriter(os.Stderr),
progressbar.OptionShowCount(),
progressbar.OptionShowIts(),
progressbar.OptionSetItsString("files"),
progressbar.OptionThrottle(250*time.Millisecond),
progressbar.OptionShowElapsedTimeOnFinish(),
progressbar.OptionSetPredictTime(true),
progressbar.OptionFullWidth(),
progressbar.OptionSetTheme(progressbar.Theme{
Saucer: "=",
SaucerHead: ">",
SaucerPadding: " ",
BarStart: "[",
BarEnd: "]",
}),
)
}
func walkAndProcess(root string, stats *Stats, bar *progressbar.ProgressBar, fn func(string, os.FileInfo, *Stats) error) error {
root = filepath.Clean(root)
err := filepath.Walk(root, func(p string, info os.FileInfo, err error) error {
if err != nil { if err != nil {
return err return err
} }
// skip symlinks entirely // Skip symlinks - filepath.Walk uses Lstat, so symlinks are reported
// as ModeSymlink, never as directories. Walk doesn't follow them.
if info.Mode()&os.ModeSymlink != 0 { if info.Mode()&os.ModeSymlink != 0 {
if verbose { if verbose && !quiet {
log.Printf("skip symlink %s", p) log.Printf("skip symlink %s", p)
} }
if info.IsDir() {
return filepath.SkipDir
}
return nil return nil
} }
@@ -260,13 +564,20 @@ func walkAndProcess(root string, fn func(string, os.FileInfo) error) error {
return nil return nil
} }
if !info.Mode().IsRegular() { if !info.Mode().IsRegular() {
if verbose { if verbose && !quiet {
log.Printf("skip non-regular %s", p) log.Printf("skip non-regular %s", p)
} }
return nil return nil
} }
return fn(p, info)
fnErr := fn(p, info, stats)
if bar != nil {
bar.Add(1)
}
return fnErr
}) })
return err
} }
func shouldExclude(rel string, info os.FileInfo) bool { func shouldExclude(rel string, info os.FileInfo) bool {
@@ -293,20 +604,21 @@ func hasXattr(path, key string) bool {
return err == nil return err == nil
} }
func fileMultihash(path string) ([]byte, error) { func fileMultihash(path string) (hash []byte, bytesRead int64, err error) {
f, err := os.Open(path) f, err := os.Open(path)
if err != nil { if err != nil {
return nil, err return nil, 0, err
} }
defer f.Close() defer f.Close()
h := sha256.New() h := sha256.New()
if _, err := io.Copy(h, f); err != nil { bytesRead, err = io.Copy(h, f)
return nil, err if err != nil {
return nil, bytesRead, err
} }
mh, err := multihash.Encode(h.Sum(nil), multihash.SHA2_256) mh, err := multihash.Encode(h.Sum(nil), multihash.SHA2_256)
if err != nil { if err != nil {
return nil, err return nil, bytesRead, err
} }
return []byte(base58.Encode(mh)), nil return []byte(base58.Encode(mh)), bytesRead, nil
} }

View File

@@ -30,13 +30,17 @@ func writeFile(t *testing.T, root, name, content string) string {
return p return p
} }
func newTestStats() *Stats {
return &Stats{StartTime: time.Now()}
}
func TestSumAddAndUpdate(t *testing.T) { func TestSumAddAndUpdate(t *testing.T) {
dir := t.TempDir() dir := t.TempDir()
skipIfNoXattr(t, dir) skipIfNoXattr(t, dir)
f := writeFile(t, dir, "a.txt", "hello") f := writeFile(t, dir, "a.txt", "hello")
if err := ProcessSumAdd(dir); err != nil { if err := ProcessSumAdd(dir, newTestStats(), nil); err != nil {
t.Fatalf("add: %v", err) t.Fatalf("add: %v", err)
} }
if _, err := xattr.Get(f, checksumKey); err != nil { if _, err := xattr.Get(f, checksumKey); err != nil {
@@ -49,7 +53,7 @@ func TestSumAddAndUpdate(t *testing.T) {
now := time.Now().Add(2 * time.Second) now := time.Now().Add(2 * time.Second)
os.Chtimes(f, now, now) os.Chtimes(f, now, now)
if err := ProcessSumUpdate(dir); err != nil { if err := ProcessSumUpdate(dir, newTestStats(), nil); err != nil {
t.Fatalf("update: %v", err) t.Fatalf("update: %v", err)
} }
tsb2, _ := xattr.Get(f, sumTimeKey) tsb2, _ := xattr.Get(f, sumTimeKey)
@@ -64,17 +68,17 @@ func TestProcessCheckIntegration(t *testing.T) {
skipIfNoXattr(t, dir) skipIfNoXattr(t, dir)
writeFile(t, dir, "b.txt", "world") writeFile(t, dir, "b.txt", "world")
if err := ProcessSumAdd(dir); err != nil { if err := ProcessSumAdd(dir, newTestStats(), nil); err != nil {
t.Fatalf("add: %v", err) t.Fatalf("add: %v", err)
} }
if err := ProcessCheck(dir, false); err != nil { if err := ProcessCheck(dir, false, newTestStats(), nil); err != nil {
t.Fatalf("check ok: %v", err) t.Fatalf("check ok: %v", err)
} }
f := filepath.Join(dir, "b.txt") f := filepath.Join(dir, "b.txt")
os.WriteFile(f, []byte("corrupt"), 0o644) os.WriteFile(f, []byte("corrupt"), 0o644)
if err := ProcessCheck(dir, false); err == nil { if err := ProcessCheck(dir, false, newTestStats(), nil); err == nil {
t.Fatalf("expected mismatch error, got nil") t.Fatalf("expected mismatch error, got nil")
} }
} }
@@ -84,11 +88,11 @@ func TestClearRemovesAttrs(t *testing.T) {
skipIfNoXattr(t, dir) skipIfNoXattr(t, dir)
f := writeFile(t, dir, "c.txt", "data") f := writeFile(t, dir, "c.txt", "data")
if err := ProcessSumAdd(dir); err != nil { if err := ProcessSumAdd(dir, newTestStats(), nil); err != nil {
t.Fatalf("add: %v", err) t.Fatalf("add: %v", err)
} }
if err := ProcessClear(dir); err != nil { if err := ProcessClear(dir, newTestStats(), nil); err != nil {
t.Fatalf("clear: %v", err) t.Fatalf("clear: %v", err)
} }
if _, err := xattr.Get(f, checksumKey); err == nil { if _, err := xattr.Get(f, checksumKey); err == nil {
@@ -113,7 +117,7 @@ func TestExcludeDotfilesAndPatterns(t *testing.T) {
excludePatterns = []string{"*.me"} excludePatterns = []string{"*.me"}
defer func() { excludeDotfiles, excludePatterns = oldDot, oldPat }() defer func() { excludeDotfiles, excludePatterns = oldDot, oldPat }()
if err := ProcessSumAdd(dir); err != nil { if err := ProcessSumAdd(dir, newTestStats(), nil); err != nil {
t.Fatalf("add with excludes: %v", err) t.Fatalf("add with excludes: %v", err)
} }
@@ -139,7 +143,7 @@ func TestSkipBrokenSymlink(t *testing.T) {
} }
// Should not error and should not create xattrs on link // Should not error and should not create xattrs on link
if err := ProcessSumAdd(dir); err != nil { if err := ProcessSumAdd(dir, newTestStats(), nil); err != nil {
t.Fatalf("ProcessSumAdd with symlink: %v", err) t.Fatalf("ProcessSumAdd with symlink: %v", err)
} }
if _, err := xattr.Get(link, checksumKey); err == nil { if _, err := xattr.Get(link, checksumKey); err == nil {
@@ -155,13 +159,13 @@ func TestPermissionErrors(t *testing.T) {
os.Chmod(secret, 0o000) os.Chmod(secret, 0o000)
defer os.Chmod(secret, 0o644) defer os.Chmod(secret, 0o644)
if err := ProcessSumAdd(dir); err == nil { if err := ProcessSumAdd(dir, newTestStats(), nil); err == nil {
t.Fatalf("expected permission error, got nil") t.Fatalf("expected permission error, got nil")
} }
if err := ProcessSumUpdate(dir); err == nil { if err := ProcessSumUpdate(dir, newTestStats(), nil); err == nil {
t.Fatalf("expected permission error on update, got nil") t.Fatalf("expected permission error on update, got nil")
} }
if err := ProcessCheck(dir, false); err == nil { if err := ProcessCheck(dir, false, newTestStats(), nil); err == nil {
t.Fatalf("expected permission error on check, got nil") t.Fatalf("expected permission error on check, got nil")
} }
} }

6
go.mod
View File

@@ -7,6 +7,7 @@ require (
github.com/mr-tron/base58 v1.2.0 github.com/mr-tron/base58 v1.2.0
github.com/multiformats/go-multihash v0.2.3 github.com/multiformats/go-multihash v0.2.3
github.com/pkg/xattr v0.4.10 github.com/pkg/xattr v0.4.10
github.com/schollz/progressbar/v3 v3.19.0
github.com/spf13/cobra v1.9.1 github.com/spf13/cobra v1.9.1
) )
@@ -14,10 +15,13 @@ require (
github.com/inconshreveable/mousetrap v1.1.0 // indirect github.com/inconshreveable/mousetrap v1.1.0 // indirect
github.com/klauspost/cpuid/v2 v2.0.9 // indirect github.com/klauspost/cpuid/v2 v2.0.9 // indirect
github.com/minio/sha256-simd v1.0.0 // indirect github.com/minio/sha256-simd v1.0.0 // indirect
github.com/mitchellh/colorstring v0.0.0-20190213212951-d06e56a500db // indirect
github.com/multiformats/go-varint v0.0.6 // indirect github.com/multiformats/go-varint v0.0.6 // indirect
github.com/rivo/uniseg v0.4.7 // indirect
github.com/spaolacci/murmur3 v1.1.0 // indirect github.com/spaolacci/murmur3 v1.1.0 // indirect
github.com/spf13/pflag v1.0.6 // indirect github.com/spf13/pflag v1.0.6 // indirect
golang.org/x/crypto v0.0.0-20220525230936-793ad666bf5e // indirect golang.org/x/crypto v0.0.0-20220525230936-793ad666bf5e // indirect
golang.org/x/sys v0.1.0 // indirect golang.org/x/sys v0.29.0 // indirect
golang.org/x/term v0.28.0 // indirect
lukechampine.com/blake3 v1.1.6 // indirect lukechampine.com/blake3 v1.1.6 // indirect
) )

23
go.sum
View File

@@ -1,13 +1,21 @@
github.com/bmatcuk/doublestar/v4 v4.8.1 h1:54Bopc5c2cAvhLRAzqOGCYHYyhcDHsFF4wWIR5wKP38= github.com/bmatcuk/doublestar/v4 v4.8.1 h1:54Bopc5c2cAvhLRAzqOGCYHYyhcDHsFF4wWIR5wKP38=
github.com/bmatcuk/doublestar/v4 v4.8.1/go.mod h1:xBQ8jztBU6kakFMg+8WGxn0c6z1fTSPVIjEY1Wr7jzc= github.com/bmatcuk/doublestar/v4 v4.8.1/go.mod h1:xBQ8jztBU6kakFMg+8WGxn0c6z1fTSPVIjEY1Wr7jzc=
github.com/chengxilo/virtualterm v1.0.4 h1:Z6IpERbRVlfB8WkOmtbHiDbBANU7cimRIof7mk9/PwM=
github.com/chengxilo/virtualterm v1.0.4/go.mod h1:DyxxBZz/x1iqJjFxTFcr6/x+jSpqN0iwWCOK1q10rlY=
github.com/cpuguy83/go-md2man/v2 v2.0.6/go.mod h1:oOW0eioCTA6cOiMLiUPZOpcVxMig6NIQQ7OS05n1F4g= github.com/cpuguy83/go-md2man/v2 v2.0.6/go.mod h1:oOW0eioCTA6cOiMLiUPZOpcVxMig6NIQQ7OS05n1F4g=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8= github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8=
github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw= github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw=
github.com/klauspost/cpuid/v2 v2.0.4/go.mod h1:FInQzS24/EEf25PyTYn52gqo7WaD8xa0213Md/qVLRg= github.com/klauspost/cpuid/v2 v2.0.4/go.mod h1:FInQzS24/EEf25PyTYn52gqo7WaD8xa0213Md/qVLRg=
github.com/klauspost/cpuid/v2 v2.0.9 h1:lgaqFMSdTdQYdZ04uHyN2d/eKdOMyi2YLSvlQIBFYa4= github.com/klauspost/cpuid/v2 v2.0.9 h1:lgaqFMSdTdQYdZ04uHyN2d/eKdOMyi2YLSvlQIBFYa4=
github.com/klauspost/cpuid/v2 v2.0.9/go.mod h1:FInQzS24/EEf25PyTYn52gqo7WaD8xa0213Md/qVLRg= github.com/klauspost/cpuid/v2 v2.0.9/go.mod h1:FInQzS24/EEf25PyTYn52gqo7WaD8xa0213Md/qVLRg=
github.com/mattn/go-runewidth v0.0.16 h1:E5ScNMtiwvlvB5paMFdw9p4kSQzbXFikJ5SQO6TULQc=
github.com/mattn/go-runewidth v0.0.16/go.mod h1:Jdepj2loyihRzMpdS35Xk/zdY8IAYHsh153qUoGf23w=
github.com/minio/sha256-simd v1.0.0 h1:v1ta+49hkWZyvaKwrQB8elexRqm6Y0aMLjCNsrYxo6g= github.com/minio/sha256-simd v1.0.0 h1:v1ta+49hkWZyvaKwrQB8elexRqm6Y0aMLjCNsrYxo6g=
github.com/minio/sha256-simd v1.0.0/go.mod h1:OuYzVNI5vcoYIAmbIvHPl3N3jUzVedXbKy5RFepssQM= github.com/minio/sha256-simd v1.0.0/go.mod h1:OuYzVNI5vcoYIAmbIvHPl3N3jUzVedXbKy5RFepssQM=
github.com/mitchellh/colorstring v0.0.0-20190213212951-d06e56a500db h1:62I3jR2EmQ4l5rM/4FEfDWcRD+abF5XlKShorW5LRoQ=
github.com/mitchellh/colorstring v0.0.0-20190213212951-d06e56a500db/go.mod h1:l0dey0ia/Uv7NcFFVbCLtqEBQbrT4OCwCSKTEv6enCw=
github.com/mr-tron/base58 v1.2.0 h1:T/HDJBh4ZCPbU39/+c3rRvE0uKBQlU27+QI8LJ4t64o= github.com/mr-tron/base58 v1.2.0 h1:T/HDJBh4ZCPbU39/+c3rRvE0uKBQlU27+QI8LJ4t64o=
github.com/mr-tron/base58 v1.2.0/go.mod h1:BinMc/sQntlIE1frQmRFPUoPA1Zkr8VRgBdjWI2mNwc= github.com/mr-tron/base58 v1.2.0/go.mod h1:BinMc/sQntlIE1frQmRFPUoPA1Zkr8VRgBdjWI2mNwc=
github.com/multiformats/go-multihash v0.2.3 h1:7Lyc8XfX/IY2jWb/gI7JP+o7JEq9hOa7BFvVU9RSh+U= github.com/multiformats/go-multihash v0.2.3 h1:7Lyc8XfX/IY2jWb/gI7JP+o7JEq9hOa7BFvVU9RSh+U=
@@ -16,19 +24,30 @@ github.com/multiformats/go-varint v0.0.6 h1:gk85QWKxh3TazbLxED/NlDVv8+q+ReFJk7Y2
github.com/multiformats/go-varint v0.0.6/go.mod h1:3Ls8CIEsrijN6+B7PbrXRPxHRPuXSrVKRY101jdMZYE= github.com/multiformats/go-varint v0.0.6/go.mod h1:3Ls8CIEsrijN6+B7PbrXRPxHRPuXSrVKRY101jdMZYE=
github.com/pkg/xattr v0.4.10 h1:Qe0mtiNFHQZ296vRgUjRCoPHPqH7VdTOrZx3g0T+pGA= github.com/pkg/xattr v0.4.10 h1:Qe0mtiNFHQZ296vRgUjRCoPHPqH7VdTOrZx3g0T+pGA=
github.com/pkg/xattr v0.4.10/go.mod h1:di8WF84zAKk8jzR1UBTEWh9AUlIZZ7M/JNt8e9B6ktU= github.com/pkg/xattr v0.4.10/go.mod h1:di8WF84zAKk8jzR1UBTEWh9AUlIZZ7M/JNt8e9B6ktU=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/rivo/uniseg v0.4.7 h1:WUdvkW8uEhrYfLC4ZzdpI2ztxP1I582+49Oc5Mq64VQ=
github.com/rivo/uniseg v0.4.7/go.mod h1:FN3SvrM+Zdj16jyLfmOkMNblXMcoc8DfTHruCPUcx88=
github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM= github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
github.com/schollz/progressbar/v3 v3.19.0 h1:Ea18xuIRQXLAUidVDox3AbwfUhD0/1IvohyTutOIFoc=
github.com/schollz/progressbar/v3 v3.19.0/go.mod h1:IsO3lpbaGuzh8zIMzgY3+J8l4C8GjO0Y9S69eFvNsec=
github.com/spaolacci/murmur3 v1.1.0 h1:7c1g84S4BPRrfL5Xrdp6fOJ206sU9y293DDHaoy0bLI= github.com/spaolacci/murmur3 v1.1.0 h1:7c1g84S4BPRrfL5Xrdp6fOJ206sU9y293DDHaoy0bLI=
github.com/spaolacci/murmur3 v1.1.0/go.mod h1:JwIasOWyU6f++ZhiEuf87xNszmSA2myDM2Kzu9HwQUA= github.com/spaolacci/murmur3 v1.1.0/go.mod h1:JwIasOWyU6f++ZhiEuf87xNszmSA2myDM2Kzu9HwQUA=
github.com/spf13/cobra v1.9.1 h1:CXSaggrXdbHK9CF+8ywj8Amf7PBRmPCOJugH954Nnlo= github.com/spf13/cobra v1.9.1 h1:CXSaggrXdbHK9CF+8ywj8Amf7PBRmPCOJugH954Nnlo=
github.com/spf13/cobra v1.9.1/go.mod h1:nDyEzZ8ogv936Cinf6g1RU9MRY64Ir93oCnqb9wxYW0= github.com/spf13/cobra v1.9.1/go.mod h1:nDyEzZ8ogv936Cinf6g1RU9MRY64Ir93oCnqb9wxYW0=
github.com/spf13/pflag v1.0.6 h1:jFzHGLGAlb3ruxLB8MhbI6A8+AQX/2eW4qeyNZXNp2o= github.com/spf13/pflag v1.0.6 h1:jFzHGLGAlb3ruxLB8MhbI6A8+AQX/2eW4qeyNZXNp2o=
github.com/spf13/pflag v1.0.6/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg= github.com/spf13/pflag v1.0.6/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsTg=
github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
golang.org/x/crypto v0.0.0-20220525230936-793ad666bf5e h1:T8NU3HyQ8ClP4SEE+KbFlg6n0NhuTsN4MyznaarGsZM= golang.org/x/crypto v0.0.0-20220525230936-793ad666bf5e h1:T8NU3HyQ8ClP4SEE+KbFlg6n0NhuTsN4MyznaarGsZM=
golang.org/x/crypto v0.0.0-20220525230936-793ad666bf5e/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4= golang.org/x/crypto v0.0.0-20220525230936-793ad666bf5e/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4=
golang.org/x/sys v0.0.0-20220408201424-a24fb2fb8a0f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20220408201424-a24fb2fb8a0f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.1.0 h1:kunALQeHf1/185U1i0GOB/fy1IPRDDpuoOOqRReG57U= golang.org/x/sys v0.29.0 h1:TPYlXGxvx1MGTn2GiZDhnjPA9wZzZeGKHHmKhHYvgaU=
golang.org/x/sys v0.1.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.29.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/term v0.28.0 h1:/Ts8HFuMR2E6IP/jlo7QVLZHggjKQbhu/7H0LJFr3Gg=
golang.org/x/term v0.28.0/go.mod h1:Sw/lC2IAUZ92udQNf3WodGtn4k/XoLyZoh8v/8uiwek=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
lukechampine.com/blake3 v1.1.6 h1:H3cROdztr7RCfoaTpGZFQsrqvweFLrqS73j7L7cmR5c= lukechampine.com/blake3 v1.1.6 h1:H3cROdztr7RCfoaTpGZFQsrqvweFLrqS73j7L7cmR5c=
lukechampine.com/blake3 v1.1.6/go.mod h1:tkKEOtDkNtklkXtLNEOGNq5tcV90tJiA1vAA12R78LA= lukechampine.com/blake3 v1.1.6/go.mod h1:tkKEOtDkNtklkXtLNEOGNq5tcV90tJiA1vAA12R78LA=

82
script/bootstrap Executable file
View File

@@ -0,0 +1,82 @@
#!/bin/sh
# script/bootstrap: install all dependencies needed to build and develop
# this repo. Idempotent: every install is guarded by a check so already
# installed tools are skipped. Base tooling comes from nix, apt, brew,
# or apk (detected in that order); assumes nothing is present.
# golangci-lint and goimports are installed via `go install` at the same
# pinned commits the Dockerfile uses (never "latest").
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
# Pinned versions, 2026-07-07 (same pins as the Dockerfile)
# golangci-lint v2.10.1
GOLANGCI_LINT_REF="github.com/golangci/golangci-lint/v2/cmd/golangci-lint@5d1e709b7be35cb2025444e19de266b056b7b7ee"
# goimports v0.42.0
GOIMPORTS_REF="golang.org/x/tools/cmd/goimports@009367f5c17a8d4c45a961a3a509277190a9a6f0"
PKGMGR=""
SUDO=""
APT_UPDATED=""
detect_pkgmgr() {
[ -n "$PKGMGR" ] && return 0
if command -v nix-env >/dev/null 2>&1; then
PKGMGR="nix"
elif command -v apt-get >/dev/null 2>&1; then
PKGMGR="apt"
elif command -v brew >/dev/null 2>&1; then
PKGMGR="brew"
elif command -v apk >/dev/null 2>&1; then
PKGMGR="apk"
else
echo "bootstrap: no supported package manager (nix, apt, brew, apk)" >&2
exit 1
fi
if [ "$PKGMGR" = "apt" ]; then
export DEBIAN_FRONTEND=noninteractive
if [ "$(id -u)" != "0" ]; then
SUDO="sudo"
fi
fi
}
# pkg_install <nix-attr> <apt-pkg> <brew-formula> <apk-pkg>
pkg_install() {
detect_pkgmgr
case "$PKGMGR" in
nix) nix-env -iA "nixpkgs.$1" ;;
apt)
if [ -z "$APT_UPDATED" ]; then
$SUDO env DEBIAN_FRONTEND=noninteractive apt-get update
APT_UPDATED=1
fi
$SUDO env DEBIAN_FRONTEND=noninteractive apt-get install -y "$2"
;;
brew) brew install "$3" ;;
apk) apk add --no-cache "$4" ;;
esac
}
missing() {
! command -v "$1" >/dev/null 2>&1
}
main() {
cd "$ROOT"
if missing git; then pkg_install git git git git; fi
if missing make; then pkg_install gnumake make make make; fi
if missing go; then pkg_install go golang go go; fi
# Lint/format tools, pinned via go install (installs into
# "$(go env GOPATH)/bin"; ensure that is on your PATH).
if missing golangci-lint; then go install "$GOLANGCI_LINT_REF"; fi
if missing goimports; then go install "$GOIMPORTS_REF"; fi
go mod download
echo "bootstrap complete"
}
main "$@"

14
script/check Executable file
View File

@@ -0,0 +1,14 @@
#!/bin/sh
# script/check: run all checks (test, lint, fmt-check). Our own
# extension to scripts-to-rule-them-all. Must not modify any files.
set -eu
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
main() {
"$SCRIPT_DIR/test"
"$SCRIPT_DIR/lint"
"$SCRIPT_DIR/fmt-check"
}
main "$@"

13
script/cibuild Executable file
View File

@@ -0,0 +1,13 @@
#!/bin/sh
# script/cibuild: run the CI build. The Dockerfile runs make check, so
# a successful build implies all checks pass.
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
main() {
cd "$ROOT"
docker build .
}
main "$@"

14
script/docker Executable file
View File

@@ -0,0 +1,14 @@
#!/bin/sh
# script/docker: build the Docker image tagged with the project name.
# Identical in all repos; the tag comes from script/projectname.
set -eu
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
main() {
cd "$ROOT"
docker build -t "$("$SCRIPT_DIR/projectname")" .
}
main "$@"

13
script/fmt Executable file
View File

@@ -0,0 +1,13 @@
#!/bin/sh
# script/fmt: format all files (writes).
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
main() {
cd "$ROOT"
gofmt -s -w .
goimports -w .
}
main "$@"

18
script/fmt-check Executable file
View File

@@ -0,0 +1,18 @@
#!/bin/sh
# script/fmt-check: check formatting (read-only). Same scope as
# script/fmt, but fails instead of writing.
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
main() {
cd "$ROOT"
files="$(gofmt -l .)"
if [ -n "$files" ]; then
echo "gofmt: files not formatted:" >&2
echo "$files" >&2
exit 1
fi
}
main "$@"

16
script/install-precommit Executable file
View File

@@ -0,0 +1,16 @@
#!/bin/sh
# script/install-precommit: install the git pre-commit hook that runs
# script/precommit. Our own extension to scripts-to-rule-them-all.
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
main() {
cd "$ROOT"
hook=".git/hooks/pre-commit"
printf '#!/bin/sh\nset -e\nscript/precommit\n' > "$hook"
chmod +x "$hook"
echo "pre-commit hook installed: runs script/precommit"
}
main "$@"

12
script/lint Executable file
View File

@@ -0,0 +1,12 @@
#!/bin/sh
# script/lint: run the linter.
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
main() {
cd "$ROOT"
golangci-lint run --config .golangci.yml ./...
}
main "$@"

21
script/precommit Executable file
View File

@@ -0,0 +1,21 @@
#!/bin/sh
# script/precommit: run by the git pre-commit hook; fails the commit if
# checks fail. Our own extension to scripts-to-rule-them-all. Go extra:
# go mod tidy must be a no-op before the checks run.
set -eu
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
main() {
cd "$ROOT"
go mod tidy
if ! git diff --exit-code -- go.mod go.sum; then
echo "precommit: go mod tidy changed go.mod/go.sum;" \
"stage the changes and retry" >&2
exit 1
fi
"$SCRIPT_DIR/check"
}
main "$@"

12
script/projectname Executable file
View File

@@ -0,0 +1,12 @@
#!/bin/sh
# script/projectname: output the name of this project. Our own
# extension to scripts-to-rule-them-all. Other scripts that need the
# name (e.g. script/docker) call this, so they can stay identical
# across all repos.
set -eu
main() {
echo "attrsum"
}
main "$@"

13
script/setup Executable file
View File

@@ -0,0 +1,13 @@
#!/bin/sh
# script/setup: set up the repo for development after a fresh clone:
# installs dependencies and the git pre-commit hook.
set -eu
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
main() {
"$SCRIPT_DIR/bootstrap"
"$SCRIPT_DIR/install-precommit"
}
main "$@"

12
script/test Executable file
View File

@@ -0,0 +1,12 @@
#!/bin/sh
# script/test: run the test suite.
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
main() {
cd "$ROOT"
go test -v -race -timeout 30s -cover ./...
}
main "$@"