Commit Graph

109 Commits

Author SHA1 Message Date
4a2060087d Add GPG signature verification on manifest load
- Implement gpgVerify function that creates a temporary keyring to verify
  detached signatures against embedded public keys
- Signature verification happens during deserialization after hash
  validation but before decompression
- Extract signatureString() as a method on manifest for generating the
  canonical signature string (MAGIC-UUID-MULTIHASH)
- Add --require-signature flag to check command to mandate signature from
  a specific GPG key ID
- Expose IsSigned() and Signer() methods on Checker for signature status
2025-12-18 05:56:16 -08:00
213364bab5 Add UUID to manifest and verify integrity before decompression
- Add UUID field to both inner and outer manifest messages
- Generate random v4 UUID when creating manifest
- Hash compressed data (not uncompressed) for integrity check
- Verify hash before decompression to prevent malicious payloads
- Validate UUIDs are proper format and match between inner/outer
- Sign string format: MAGIC-UUID-MULTIHASH
2025-12-18 02:20:51 -08:00
778999a285 Add GPG signing support for manifest generation
- Add --sign-key flag and MFER_SIGN_KEY env var to gen and freshen commands
- Sign inner message multihash with GPG detached signature
- Include signer fingerprint and public key in outer wrapper
- Add comprehensive tests with temporary GPG keyring
- Increase test timeout to 10s for GPG key generation
2025-12-18 02:12:54 -08:00
308c583d57 Remove codebase structure section from README
godoc provides this documentation automatically
2025-12-18 01:38:13 -08:00
019fe41c3d Update .gitignore for new bin/ build directory 2025-12-18 01:30:50 -08:00
fc0b38ea19 Add TODO.md with codebase audit findings
Document issues found during code audit including:
- Critical: broken error comparison, unchecked hash writes, URL path traversal
- Important: goroutine leak, timestamp precision, missing context cancellation
- Code quality: duplicate functions, inefficient calculations, missing validation
2025-12-18 01:30:01 -08:00
61c17ca585 Normalize markdown formatting in documentation
- Use consistent dash-style bullet points
- Remove trailing whitespace
- Add missing blank lines between sections
- Add trailing newline to README.md
2025-12-18 01:29:56 -08:00
dae6c64e24 Change build output path from mfer.cmd to bin/mfer
Use conventional bin/ directory for build output instead of
placing executable in project root.
2025-12-18 01:29:47 -08:00
a5b0343b28 Use Go 1.13+ octal literal syntax throughout codebase
Update file permission literals from legacy octal format (0755, 0644)
to explicit Go 1.13+ format (0o755, 0o644) for improved readability.
2025-12-18 01:29:40 -08:00
e25e309581 Move checker package into mfer package
Consolidate checker functionality into the mfer package alongside
scanner, removing the need for a separate internal/checker package.
2025-12-18 01:28:35 -08:00
dc115c5ba2 Add custom types for type safety throughout codebase
- Add FileCount, FileSize, RelFilePath, AbsFilePath, ModTime, Multihash types
- Add UnixSeconds and UnixNanos types for timestamp handling
- Add URL types (ManifestURL, FileURL, BaseURL) with safe path joining
- Consolidate scanner package into mfer package
- Update checker to use custom types in Result and CheckStatus
- Add ModTime.Timestamp() method for protobuf conversion
- Update all tests to use proper custom types
2025-12-18 01:01:18 -08:00
a9f0d2abe4 Update README to reflect current API (FileProgress was already a channel) 2025-12-17 17:19:08 -08:00
1588e1bb9f Remove unused legacy manifest APIs
Removed:
- New(), NewFromPaths(), NewFromFS() - unused constructors
- Scan(), addFile(), addInputPath(), addInputFS() - unused scanning code
- WriteToFile(), Write() - unused output methods (Builder.Build() is used)
- GetFileCount(), GetTotalFileSize() - unused accessors
- pathIsHidden() - duplicated in internal/scanner
- ManifestScanOptions - unused options struct
- HasError(), AddError(), WithContext() - unused error/context handling
- NewFromProto() - deprecated alias
- manifestFile struct - unused internal type

Kept:
- manifest struct (simplified to just pbInner, pbOuter, output)
- NewManifestFromReader(), NewManifestFromFile() - for loading manifests
- Files() - returns files from loaded manifest
- Builder and its methods - for creating manifests
2025-12-17 17:16:35 -08:00
09e8da0855 Update CLAUDE.md and clean up completed TODOs in README 2025-12-17 17:09:33 -08:00
efa4bb929a Update README: mark FIXMEs as resolved 2025-12-17 17:08:37 -08:00
16e3538ea6 Document WriteToFile overwrite behavior, remove misplaced FIXME 2025-12-17 17:08:11 -08:00
1ae384b6f6 Add context cancellation support to Scan 2025-12-17 17:07:02 -08:00
b55ae961c8 Validate filesystem in addInputFS 2025-12-17 17:05:42 -08:00
ed40673e85 Validate input path exists in addInputPath 2025-12-17 17:04:46 -08:00
2549695ab0 Update README TODO with completed items 2025-12-17 17:03:05 -08:00
e480c3f677 Fix redundant stat call in addFile
Use the FileInfo already provided by Walk instead of calling Stat again.
Only stat if fi is nil (defensive, shouldn't happen in normal Walk usage).
Also fixes potential nil pointer dereference if fi was nil.
2025-12-17 17:02:29 -08:00
d3776d7d7c Add unit tests for checker/scanner and validate input paths
- Add comprehensive unit tests for internal/checker (88.5% coverage)
  - Status string representations
  - NewChecker validation
  - Check operation (OK, missing, size/hash mismatch)
  - Progress reporting and context cancellation
  - FindExtraFiles functionality

- Add comprehensive unit tests for internal/scanner (80.1% coverage)
  - Constructors and options
  - File/path enumeration
  - Dotfile exclusion/inclusion
  - ToManifest with progress and cancellation
  - Non-blocking status channel sends

- Validate input paths before scanning in generate command
  - Fail fast with clear error if paths don't exist
  - Prevents confusing errors deep in enumeration
2025-12-17 16:57:01 -08:00
07db5d434f Humanize file sizes in gen output, clean up temp on signal
- Humanize file sizes in verbose file listing (e.g., "76.8 MiB" not "76836984 bytes")
- Add signal handler to clean up temp file on Ctrl-C/SIGTERM during gen
2025-12-17 16:26:32 -08:00
1d37dd9748 Show only filename (not full path) in debug log caller info 2025-12-17 16:21:06 -08:00
1e81801036 Change gen file listing from debug to verbose level 2025-12-17 16:19:19 -08:00
eaeb84f2cc Change check OK messages from debug to verbose level 2025-12-17 16:16:16 -08:00
a20c3e5104 Add manifest corruption detection test
Generates a ~1MB manifest (20000 files with random names), then:
- Verifies truncated manifest causes check to fail
- Runs 500 iterations of random single-byte corruption
- Each iteration verifies check detects the corruption
2025-12-17 16:11:59 -08:00
c218fe56e9 Add atomic writes, humanized sizes, debug logging, and -v/-q per-command
- Atomic writes for mfer gen: writes to temp file, renames on success,
  cleans up temp on error/interrupt. Prevents empty manifests on Ctrl-C.
- Humanized byte sizes using dustin/go-humanize (e.g., "10 MiB" not "10485760")
- Progress lines clear when done (using ANSI escape \r\033[K])
- Debug logging when files are added to manifest (mfer gen -vv)
- Move -v/-q flags from global to per-command for better UX
- Add tests for atomic write behavior with failing filesystem mock
2025-12-17 15:57:20 -08:00
444a4c8f45 Add list command to show manifest contents
- mfer list: shows file paths one per line
- mfer list -l/--long: shows size, mtime, and path
- mfer list --print0: NUL-separated output for xargs -0
2025-12-17 15:36:48 -08:00
a07209fef5 Skip manifest write when nothing changed in freshen 2025-12-17 15:31:33 -08:00
b3fe38092b Add project URL to banner output 2025-12-17 15:29:58 -08:00
9dbdcbde91 Fix version command and add default action with banner
- Version command now uses mfer.Version constant instead of empty build flags
- Running just 'mfer' shows banner + help
- Unknown commands still return error with exit code 1
2025-12-17 15:28:14 -08:00
6edc798de0 Add version line after banner with release date
Added mfer/constants.go with Version and ReleaseDate constants for
deterministic builds. Banner now shows "mfer by @sneak: v0.1.0 released 2025-12-17"
2025-12-17 15:23:55 -08:00
818358a8a1 Fix pathIsHidden treating base directory as hidden, print banner to stdout
- pathIsHidden(".") was returning true, causing freshen to skip entire
  directory tree when dotfiles excluded
- Banner now prints directly to stdout to avoid log prefix artifacts
2025-12-17 15:20:56 -08:00
5523cb1595 Add Verbose log level between Info and Debug
Implemented full log level hierarchy: Fatal, Error, Warn, Info, Verbose, Debug.
- Verbose level (-v) shows detailed operations like file changes (M/A/D)
- Debug level (-vv) shows low-level tracing with caller info
- Quiet mode (-q) sets level to Error, suppressing Info messages
- Banner and summary output now use log levels for filtering
2025-12-17 15:17:27 -08:00
0e86562c09 Exclude dotfiles by default, add --include-dotfiles flag
Changed the default behavior to exclude dotfiles (files/dirs starting with .)
which is the more common use case. Added --include-dotfiles flag for when
hidden files need to be included in the manifest.
2025-12-17 15:10:22 -08:00
6dc496fa9e Add verbose output for freshen showing M/A/D files with -v 2025-12-17 15:04:43 -08:00
e6cb906d35 Change operational log messages from Debug to Info 2025-12-17 15:02:46 -08:00
45201509ff Fix progress line not clearing before final status 2025-12-17 15:01:32 -08:00
21028af9aa Replace gzip with zstd compression
Use zstd with SpeedBestCompression level for better compression
ratios. Remove gzip support entirely. Include generated protobuf
file to allow building without protoc.
2025-12-17 14:49:30 -08:00
150bac82cf Add TODO.md with 1.0 release tasks 2025-12-17 14:40:27 -08:00
92bd13efde Fix all linter errors
- Add explicit error ignoring with _ = for Close/Remove calls
- Rename WriteTo to Write to avoid io.WriterTo interface conflict
- Fix errcheck warnings in fetch, freshen, gen, mfer, checker,
  deserialize, serialize, and output files
2025-12-17 14:37:52 -08:00
531f460f87 Add CLAUDE.md 2025-12-17 14:33:00 -08:00
c5ca3e2ced Change FileProgress callback to channel-based progress
Replace callback-based progress reporting in Builder.AddFile with
channel-based FileHashProgress for consistency with EnumerateStatus
and ScanStatus patterns. Update scanner.go to use the new channel API.
2025-12-17 14:30:10 -08:00
fded1a0393 Remove generateInner, now handled by Builder
- Remove generateInner() from serialize.go
- Update generate() to error if pbInner not set
- Remove legacy tests that depended on old code path
- Update TODO item to reflect removal
2025-12-17 11:27:41 -08:00
5d7c729efb remove golangci-lint config files 2025-12-17 11:27:41 -08:00
48c3c09d85 Rename ManifestBuilder to Builder 2025-12-17 11:27:41 -08:00
f3be3eba84 Add TODO: change FileProgress callback to channel-based 2025-12-17 11:27:41 -08:00
5e65b3a0fd Add TODO section to README with prioritized task list 2025-12-17 11:27:41 -08:00
79fc5cca6c Add godoc strings to all exported types, functions, and fields
Documents:
- cli: NO_COLOR, RunOptions fields, CLIApp, VersionString
- checker: Result fields, Status constants, CheckStatus fields
- scanner: EnumerateStatus, ScanStatus, Options, FileEntry fields
- log: Level alias, DisableStyling, Init, Info/Debug functions,
  verbosity helpers, GetLogger, GetLevel, WithError
- mfer: ManifestScanOptions, New, NewFromPaths, NewFromFS, MAGIC
2025-12-17 11:27:41 -08:00