Commit Graph

97 Commits

Author SHA1 Message Date
1588e1bb9f Remove unused legacy manifest APIs
Removed:
- New(), NewFromPaths(), NewFromFS() - unused constructors
- Scan(), addFile(), addInputPath(), addInputFS() - unused scanning code
- WriteToFile(), Write() - unused output methods (Builder.Build() is used)
- GetFileCount(), GetTotalFileSize() - unused accessors
- pathIsHidden() - duplicated in internal/scanner
- ManifestScanOptions - unused options struct
- HasError(), AddError(), WithContext() - unused error/context handling
- NewFromProto() - deprecated alias
- manifestFile struct - unused internal type

Kept:
- manifest struct (simplified to just pbInner, pbOuter, output)
- NewManifestFromReader(), NewManifestFromFile() - for loading manifests
- Files() - returns files from loaded manifest
- Builder and its methods - for creating manifests
2025-12-17 17:16:35 -08:00
09e8da0855 Update CLAUDE.md and clean up completed TODOs in README 2025-12-17 17:09:33 -08:00
efa4bb929a Update README: mark FIXMEs as resolved 2025-12-17 17:08:37 -08:00
16e3538ea6 Document WriteToFile overwrite behavior, remove misplaced FIXME 2025-12-17 17:08:11 -08:00
1ae384b6f6 Add context cancellation support to Scan 2025-12-17 17:07:02 -08:00
b55ae961c8 Validate filesystem in addInputFS 2025-12-17 17:05:42 -08:00
ed40673e85 Validate input path exists in addInputPath 2025-12-17 17:04:46 -08:00
2549695ab0 Update README TODO with completed items 2025-12-17 17:03:05 -08:00
e480c3f677 Fix redundant stat call in addFile
Use the FileInfo already provided by Walk instead of calling Stat again.
Only stat if fi is nil (defensive, shouldn't happen in normal Walk usage).
Also fixes potential nil pointer dereference if fi was nil.
2025-12-17 17:02:29 -08:00
d3776d7d7c Add unit tests for checker/scanner and validate input paths
- Add comprehensive unit tests for internal/checker (88.5% coverage)
  - Status string representations
  - NewChecker validation
  - Check operation (OK, missing, size/hash mismatch)
  - Progress reporting and context cancellation
  - FindExtraFiles functionality

- Add comprehensive unit tests for internal/scanner (80.1% coverage)
  - Constructors and options
  - File/path enumeration
  - Dotfile exclusion/inclusion
  - ToManifest with progress and cancellation
  - Non-blocking status channel sends

- Validate input paths before scanning in generate command
  - Fail fast with clear error if paths don't exist
  - Prevents confusing errors deep in enumeration
2025-12-17 16:57:01 -08:00
07db5d434f Humanize file sizes in gen output, clean up temp on signal
- Humanize file sizes in verbose file listing (e.g., "76.8 MiB" not "76836984 bytes")
- Add signal handler to clean up temp file on Ctrl-C/SIGTERM during gen
2025-12-17 16:26:32 -08:00
1d37dd9748 Show only filename (not full path) in debug log caller info 2025-12-17 16:21:06 -08:00
1e81801036 Change gen file listing from debug to verbose level 2025-12-17 16:19:19 -08:00
eaeb84f2cc Change check OK messages from debug to verbose level 2025-12-17 16:16:16 -08:00
a20c3e5104 Add manifest corruption detection test
Generates a ~1MB manifest (20000 files with random names), then:
- Verifies truncated manifest causes check to fail
- Runs 500 iterations of random single-byte corruption
- Each iteration verifies check detects the corruption
2025-12-17 16:11:59 -08:00
c218fe56e9 Add atomic writes, humanized sizes, debug logging, and -v/-q per-command
- Atomic writes for mfer gen: writes to temp file, renames on success,
  cleans up temp on error/interrupt. Prevents empty manifests on Ctrl-C.
- Humanized byte sizes using dustin/go-humanize (e.g., "10 MiB" not "10485760")
- Progress lines clear when done (using ANSI escape \r\033[K])
- Debug logging when files are added to manifest (mfer gen -vv)
- Move -v/-q flags from global to per-command for better UX
- Add tests for atomic write behavior with failing filesystem mock
2025-12-17 15:57:20 -08:00
444a4c8f45 Add list command to show manifest contents
- mfer list: shows file paths one per line
- mfer list -l/--long: shows size, mtime, and path
- mfer list --print0: NUL-separated output for xargs -0
2025-12-17 15:36:48 -08:00
a07209fef5 Skip manifest write when nothing changed in freshen 2025-12-17 15:31:33 -08:00
b3fe38092b Add project URL to banner output 2025-12-17 15:29:58 -08:00
9dbdcbde91 Fix version command and add default action with banner
- Version command now uses mfer.Version constant instead of empty build flags
- Running just 'mfer' shows banner + help
- Unknown commands still return error with exit code 1
2025-12-17 15:28:14 -08:00
6edc798de0 Add version line after banner with release date
Added mfer/constants.go with Version and ReleaseDate constants for
deterministic builds. Banner now shows "mfer by @sneak: v0.1.0 released 2025-12-17"
2025-12-17 15:23:55 -08:00
818358a8a1 Fix pathIsHidden treating base directory as hidden, print banner to stdout
- pathIsHidden(".") was returning true, causing freshen to skip entire
  directory tree when dotfiles excluded
- Banner now prints directly to stdout to avoid log prefix artifacts
2025-12-17 15:20:56 -08:00
5523cb1595 Add Verbose log level between Info and Debug
Implemented full log level hierarchy: Fatal, Error, Warn, Info, Verbose, Debug.
- Verbose level (-v) shows detailed operations like file changes (M/A/D)
- Debug level (-vv) shows low-level tracing with caller info
- Quiet mode (-q) sets level to Error, suppressing Info messages
- Banner and summary output now use log levels for filtering
2025-12-17 15:17:27 -08:00
0e86562c09 Exclude dotfiles by default, add --include-dotfiles flag
Changed the default behavior to exclude dotfiles (files/dirs starting with .)
which is the more common use case. Added --include-dotfiles flag for when
hidden files need to be included in the manifest.
2025-12-17 15:10:22 -08:00
6dc496fa9e Add verbose output for freshen showing M/A/D files with -v 2025-12-17 15:04:43 -08:00
e6cb906d35 Change operational log messages from Debug to Info 2025-12-17 15:02:46 -08:00
45201509ff Fix progress line not clearing before final status 2025-12-17 15:01:32 -08:00
21028af9aa Replace gzip with zstd compression
Use zstd with SpeedBestCompression level for better compression
ratios. Remove gzip support entirely. Include generated protobuf
file to allow building without protoc.
2025-12-17 14:49:30 -08:00
150bac82cf Add TODO.md with 1.0 release tasks 2025-12-17 14:40:27 -08:00
92bd13efde Fix all linter errors
- Add explicit error ignoring with _ = for Close/Remove calls
- Rename WriteTo to Write to avoid io.WriterTo interface conflict
- Fix errcheck warnings in fetch, freshen, gen, mfer, checker,
  deserialize, serialize, and output files
2025-12-17 14:37:52 -08:00
531f460f87 Add CLAUDE.md 2025-12-17 14:33:00 -08:00
c5ca3e2ced Change FileProgress callback to channel-based progress
Replace callback-based progress reporting in Builder.AddFile with
channel-based FileHashProgress for consistency with EnumerateStatus
and ScanStatus patterns. Update scanner.go to use the new channel API.
2025-12-17 14:30:10 -08:00
fded1a0393 Remove generateInner, now handled by Builder
- Remove generateInner() from serialize.go
- Update generate() to error if pbInner not set
- Remove legacy tests that depended on old code path
- Update TODO item to reflect removal
2025-12-17 11:27:41 -08:00
5d7c729efb remove golangci-lint config files 2025-12-17 11:27:41 -08:00
48c3c09d85 Rename ManifestBuilder to Builder 2025-12-17 11:27:41 -08:00
f3be3eba84 Add TODO: change FileProgress callback to channel-based 2025-12-17 11:27:41 -08:00
5e65b3a0fd Add TODO section to README with prioritized task list 2025-12-17 11:27:41 -08:00
79fc5cca6c Add godoc strings to all exported types, functions, and fields
Documents:
- cli: NO_COLOR, RunOptions fields, CLIApp, VersionString
- checker: Result fields, Status constants, CheckStatus fields
- scanner: EnumerateStatus, ScanStatus, Options, FileEntry fields
- log: Level alias, DisableStyling, Init, Info/Debug functions,
  verbosity helpers, GetLogger, GetLevel, WithError
- mfer: ManifestScanOptions, New, NewFromPaths, NewFromFS, MAGIC
2025-12-17 11:27:41 -08:00
155ebe9a78 Merge branch 'main' into next 2025-12-17 19:01:52 +00:00
dc2ea47f6a Add testable CLI with dependency injection and new scanner/checker packages
Major changes:
- Refactor CLI to accept injected I/O streams and filesystem (afero.Fs)
  for testing without touching the real filesystem
- Add RunOptions struct and RunWithOptions() for configurable CLI execution
- Add internal/scanner package with two-phase manifest generation:
  - Phase 1 (Enumeration): walk directories, collect metadata
  - Phase 2 (Scan): read contents, compute hashes, write manifest
- Add internal/checker package for manifest verification with progress
  reporting and channel-based result streaming
- Add mfer/builder.go for incremental manifest construction
- Add --no-extra-files flag to check command to detect files not in manifest
- Add timing summaries showing file count, size, elapsed time, and throughput
- Add comprehensive tests using afero.MemMapFs (no real filesystem access)
- Add contrib/usage.sh integration test script
- Fix banner ASCII art alignment (consistent spacing)
- Fix verbosity levels so summaries display at default log level
- Update internal/log to support configurable output writers
2025-12-17 11:00:55 -08:00
2717685619 update readme to conform with my new readme howto standards (#8)
All checks were successful
continuous-integration/drone/push Build is passing
Reviewed-on: #8

manually approving because CI is offline rn for some reason
2024-12-09 02:50:57 +00:00
13f39d598f remove gofumpt from linting
Some checks failed
continuous-integration/drone/push Build is passing
continuous-integration/drone/pr Build is failing
2024-05-22 14:34:05 -07:00
01bffc8388 latest
All checks were successful
continuous-integration/drone/push Build is passing
2024-05-22 14:25:35 -07:00
7df558d8d0 next (#5)
All checks were successful
continuous-integration/drone/push Build is passing
Co-authored-by: sneak <sneak@sneak.berlin>
Reviewed-on: #5
2022-12-09 00:02:33 +00:00
7a8a1b4a4a rollup from next branch (#4)
All checks were successful
continuous-integration/drone/push Build is passing
Co-authored-by: sneak <sneak@sneak.berlin>
Reviewed-on: #4
2022-12-04 07:59:36 +00:00
eb3b685aa3 i think this will build now
All checks were successful
continuous-integration/drone/push Build is passing
2022-02-02 00:26:18 -08:00
9d8616866f latest 2022-02-02 00:12:03 -08:00
b9c51fdeeb proto3 compiler doesn't like required/optional anymore?
All checks were successful
continuous-integration/drone/push Build is passing
2022-02-01 23:48:42 -08:00
7b869592ce proto3 fixes to pass ci
Some checks failed
continuous-integration/drone/push Build is failing
2022-02-01 23:44:58 -08:00
6628325843 ci debugging
Some checks failed
continuous-integration/drone/push Build is failing
2022-02-01 23:35:33 -08:00