75 Commits

Author SHA1 Message Date
user
de7d75e970 chore: remove stale .drone.yml
Remove obsolete Drone CI config. Added to .gitignore to prevent re-commit.
Note: .golangci.yaml was already removed from the next branch.
2026-02-20 02:59:17 -08:00
ae70cf6fb5 Fix FindExtraFiles reporting manifest and dotfiles as extra (closes #16) (#21)
Reviewed-on: #21
2026-02-20 11:38:52 +01:00
5099b6951b Fix IsHiddenPath treating current directory as hidden (closes #14) (#19)
Reviewed-on: #19
2026-02-20 11:37:23 +01:00
clawbot
7b61bdd62b fix: handle errcheck warnings in gpg.go and gpg_test.go 2026-02-19 23:41:14 -08:00
clawbot
8c7eef6240 fix: handle errcheck warnings in gpg.go and gpg_test.go, fix gofmt 2026-02-19 23:40:50 -08:00
clawbot
97dbe47c32 fix: FindExtraFiles skips dotfiles and manifest files to avoid false positives
FindExtraFiles now skips hidden files/directories and manifest files
(index.mf, .index.mf) when looking for extra files. Previously it would
report these as 'extra' even though they are intentionally excluded from
manifests by default, making --no-extra-files unusable.

Also includes IsHiddenPath fix for '.' (needed by the new filtering).
2026-02-19 23:36:48 -08:00
clawbot
f6478858d7 fix: IsHiddenPath returns false for "." and "/" (current dir/root are not hidden)
path.Clean(".") returns "." which starts with a dot, causing IsHiddenPath
to incorrectly treat the current directory as hidden. Add explicit checks
for "." and "/" before the dot-prefix check.

Fixed in both mfer/scanner.go and internal/scanner/scanner.go.
2026-02-19 23:36:42 -08:00
1f12d10cb7 Fix errors.Is with errors.New() never matching in checker (closes #12) (#17)
Co-authored-by: clawbot <clawbot@openclaw>
Co-authored-by: Jeffrey Paul <sneak@noreply.example.org>
Reviewed-on: #17
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
2026-02-09 02:15:08 +01:00
7f25970dd3 Fix URL encoding for file paths in fetch command (closes #13) (#18)
Co-authored-by: clawbot <clawbot@openclaw>
Co-authored-by: Jeffrey Paul <sneak@noreply.example.org>
Reviewed-on: #18
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
2026-02-09 02:14:20 +01:00
70af055d4e Fix newTimestampFromTime panic on extreme dates (closes #15) (#20)
Co-authored-by: clawbot <clawbot@openclaw>
Co-authored-by: Jeffrey Paul <sneak@noreply.example.org>
Reviewed-on: #20
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
2026-02-09 02:10:21 +01:00
04b05e01e8 Consolidate scanner/checker — delete internal/scanner/ and internal/checker/ (closes #22) (#27)
Remove unused `internal/scanner/` and `internal/checker/` packages. The CLI already uses `mfer.Scanner` and `mfer.Checker` from the `mfer/` package directly, so these were dead code.

Co-authored-by: clawbot <clawbot@openclaw>
Co-authored-by: Jeffrey Paul <sneak@noreply.example.org>
Reviewed-on: #27
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
2026-02-09 02:09:01 +01:00
7144617d0e Add decompression size limit in deserializeInner() (closes #24) (#29)
Wrap zstd decompressor with `io.LimitReader` (256MB max) to prevent decompression bombs.

Co-authored-by: clawbot <clawbot@openclaw>
Co-authored-by: Jeffrey Paul <sneak@noreply.example.org>
Reviewed-on: #29
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
2026-02-09 01:45:55 +01:00
2efffd9da8 Specify and enforce path invariants (closes #26) (#31)
Add `ValidatePath()` enforcing UTF-8, forward-slash, relative, no `..`, no empty segments. Applied in `AddFile` and `AddFileWithHash`. Proto comments document the rules.

Co-authored-by: clawbot <clawbot@openclaw>
Co-authored-by: Jeffrey Paul <sneak@noreply.example.org>
Reviewed-on: #31
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
2026-02-09 01:45:29 +01:00
ebaf2a65ca Fix AddFile to verify actual bytes read matches declared size (closes #25) (#30)
After reading file content, verify `totalRead == size` and return an error on mismatch.

Co-authored-by: clawbot <clawbot@openclaw>
Reviewed-on: #30
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
2026-02-09 01:35:07 +01:00
4b80c0067b docs: replace TODO.md with design questions and implementation plan 2026-02-08 18:40:31 +01:00
5ab092098b progress 2026-02-08 09:25:58 -08:00
4a2060087d Add GPG signature verification on manifest load
- Implement gpgVerify function that creates a temporary keyring to verify
  detached signatures against embedded public keys
- Signature verification happens during deserialization after hash
  validation but before decompression
- Extract signatureString() as a method on manifest for generating the
  canonical signature string (MAGIC-UUID-MULTIHASH)
- Add --require-signature flag to check command to mandate signature from
  a specific GPG key ID
- Expose IsSigned() and Signer() methods on Checker for signature status
2025-12-18 05:56:16 -08:00
213364bab5 Add UUID to manifest and verify integrity before decompression
- Add UUID field to both inner and outer manifest messages
- Generate random v4 UUID when creating manifest
- Hash compressed data (not uncompressed) for integrity check
- Verify hash before decompression to prevent malicious payloads
- Validate UUIDs are proper format and match between inner/outer
- Sign string format: MAGIC-UUID-MULTIHASH
2025-12-18 02:20:51 -08:00
778999a285 Add GPG signing support for manifest generation
- Add --sign-key flag and MFER_SIGN_KEY env var to gen and freshen commands
- Sign inner message multihash with GPG detached signature
- Include signer fingerprint and public key in outer wrapper
- Add comprehensive tests with temporary GPG keyring
- Increase test timeout to 10s for GPG key generation
2025-12-18 02:12:54 -08:00
308c583d57 Remove codebase structure section from README
godoc provides this documentation automatically
2025-12-18 01:38:13 -08:00
019fe41c3d Update .gitignore for new bin/ build directory 2025-12-18 01:30:50 -08:00
fc0b38ea19 Add TODO.md with codebase audit findings
Document issues found during code audit including:
- Critical: broken error comparison, unchecked hash writes, URL path traversal
- Important: goroutine leak, timestamp precision, missing context cancellation
- Code quality: duplicate functions, inefficient calculations, missing validation
2025-12-18 01:30:01 -08:00
61c17ca585 Normalize markdown formatting in documentation
- Use consistent dash-style bullet points
- Remove trailing whitespace
- Add missing blank lines between sections
- Add trailing newline to README.md
2025-12-18 01:29:56 -08:00
dae6c64e24 Change build output path from mfer.cmd to bin/mfer
Use conventional bin/ directory for build output instead of
placing executable in project root.
2025-12-18 01:29:47 -08:00
a5b0343b28 Use Go 1.13+ octal literal syntax throughout codebase
Update file permission literals from legacy octal format (0755, 0644)
to explicit Go 1.13+ format (0o755, 0o644) for improved readability.
2025-12-18 01:29:40 -08:00
e25e309581 Move checker package into mfer package
Consolidate checker functionality into the mfer package alongside
scanner, removing the need for a separate internal/checker package.
2025-12-18 01:28:35 -08:00
dc115c5ba2 Add custom types for type safety throughout codebase
- Add FileCount, FileSize, RelFilePath, AbsFilePath, ModTime, Multihash types
- Add UnixSeconds and UnixNanos types for timestamp handling
- Add URL types (ManifestURL, FileURL, BaseURL) with safe path joining
- Consolidate scanner package into mfer package
- Update checker to use custom types in Result and CheckStatus
- Add ModTime.Timestamp() method for protobuf conversion
- Update all tests to use proper custom types
2025-12-18 01:01:18 -08:00
a9f0d2abe4 Update README to reflect current API (FileProgress was already a channel) 2025-12-17 17:19:08 -08:00
1588e1bb9f Remove unused legacy manifest APIs
Removed:
- New(), NewFromPaths(), NewFromFS() - unused constructors
- Scan(), addFile(), addInputPath(), addInputFS() - unused scanning code
- WriteToFile(), Write() - unused output methods (Builder.Build() is used)
- GetFileCount(), GetTotalFileSize() - unused accessors
- pathIsHidden() - duplicated in internal/scanner
- ManifestScanOptions - unused options struct
- HasError(), AddError(), WithContext() - unused error/context handling
- NewFromProto() - deprecated alias
- manifestFile struct - unused internal type

Kept:
- manifest struct (simplified to just pbInner, pbOuter, output)
- NewManifestFromReader(), NewManifestFromFile() - for loading manifests
- Files() - returns files from loaded manifest
- Builder and its methods - for creating manifests
2025-12-17 17:16:35 -08:00
09e8da0855 Update CLAUDE.md and clean up completed TODOs in README 2025-12-17 17:09:33 -08:00
efa4bb929a Update README: mark FIXMEs as resolved 2025-12-17 17:08:37 -08:00
16e3538ea6 Document WriteToFile overwrite behavior, remove misplaced FIXME 2025-12-17 17:08:11 -08:00
1ae384b6f6 Add context cancellation support to Scan 2025-12-17 17:07:02 -08:00
b55ae961c8 Validate filesystem in addInputFS 2025-12-17 17:05:42 -08:00
ed40673e85 Validate input path exists in addInputPath 2025-12-17 17:04:46 -08:00
2549695ab0 Update README TODO with completed items 2025-12-17 17:03:05 -08:00
e480c3f677 Fix redundant stat call in addFile
Use the FileInfo already provided by Walk instead of calling Stat again.
Only stat if fi is nil (defensive, shouldn't happen in normal Walk usage).
Also fixes potential nil pointer dereference if fi was nil.
2025-12-17 17:02:29 -08:00
d3776d7d7c Add unit tests for checker/scanner and validate input paths
- Add comprehensive unit tests for internal/checker (88.5% coverage)
  - Status string representations
  - NewChecker validation
  - Check operation (OK, missing, size/hash mismatch)
  - Progress reporting and context cancellation
  - FindExtraFiles functionality

- Add comprehensive unit tests for internal/scanner (80.1% coverage)
  - Constructors and options
  - File/path enumeration
  - Dotfile exclusion/inclusion
  - ToManifest with progress and cancellation
  - Non-blocking status channel sends

- Validate input paths before scanning in generate command
  - Fail fast with clear error if paths don't exist
  - Prevents confusing errors deep in enumeration
2025-12-17 16:57:01 -08:00
07db5d434f Humanize file sizes in gen output, clean up temp on signal
- Humanize file sizes in verbose file listing (e.g., "76.8 MiB" not "76836984 bytes")
- Add signal handler to clean up temp file on Ctrl-C/SIGTERM during gen
2025-12-17 16:26:32 -08:00
1d37dd9748 Show only filename (not full path) in debug log caller info 2025-12-17 16:21:06 -08:00
1e81801036 Change gen file listing from debug to verbose level 2025-12-17 16:19:19 -08:00
eaeb84f2cc Change check OK messages from debug to verbose level 2025-12-17 16:16:16 -08:00
a20c3e5104 Add manifest corruption detection test
Generates a ~1MB manifest (20000 files with random names), then:
- Verifies truncated manifest causes check to fail
- Runs 500 iterations of random single-byte corruption
- Each iteration verifies check detects the corruption
2025-12-17 16:11:59 -08:00
c218fe56e9 Add atomic writes, humanized sizes, debug logging, and -v/-q per-command
- Atomic writes for mfer gen: writes to temp file, renames on success,
  cleans up temp on error/interrupt. Prevents empty manifests on Ctrl-C.
- Humanized byte sizes using dustin/go-humanize (e.g., "10 MiB" not "10485760")
- Progress lines clear when done (using ANSI escape \r\033[K])
- Debug logging when files are added to manifest (mfer gen -vv)
- Move -v/-q flags from global to per-command for better UX
- Add tests for atomic write behavior with failing filesystem mock
2025-12-17 15:57:20 -08:00
444a4c8f45 Add list command to show manifest contents
- mfer list: shows file paths one per line
- mfer list -l/--long: shows size, mtime, and path
- mfer list --print0: NUL-separated output for xargs -0
2025-12-17 15:36:48 -08:00
a07209fef5 Skip manifest write when nothing changed in freshen 2025-12-17 15:31:33 -08:00
b3fe38092b Add project URL to banner output 2025-12-17 15:29:58 -08:00
9dbdcbde91 Fix version command and add default action with banner
- Version command now uses mfer.Version constant instead of empty build flags
- Running just 'mfer' shows banner + help
- Unknown commands still return error with exit code 1
2025-12-17 15:28:14 -08:00
6edc798de0 Add version line after banner with release date
Added mfer/constants.go with Version and ReleaseDate constants for
deterministic builds. Banner now shows "mfer by @sneak: v0.1.0 released 2025-12-17"
2025-12-17 15:23:55 -08:00
818358a8a1 Fix pathIsHidden treating base directory as hidden, print banner to stdout
- pathIsHidden(".") was returning true, causing freshen to skip entire
  directory tree when dotfiles excluded
- Banner now prints directly to stdout to avoid log prefix artifacts
2025-12-17 15:20:56 -08:00
5523cb1595 Add Verbose log level between Info and Debug
Implemented full log level hierarchy: Fatal, Error, Warn, Info, Verbose, Debug.
- Verbose level (-v) shows detailed operations like file changes (M/A/D)
- Debug level (-vv) shows low-level tracing with caller info
- Quiet mode (-q) sets level to Error, suppressing Info messages
- Banner and summary output now use log levels for filtering
2025-12-17 15:17:27 -08:00
0e86562c09 Exclude dotfiles by default, add --include-dotfiles flag
Changed the default behavior to exclude dotfiles (files/dirs starting with .)
which is the more common use case. Added --include-dotfiles flag for when
hidden files need to be included in the manifest.
2025-12-17 15:10:22 -08:00
6dc496fa9e Add verbose output for freshen showing M/A/D files with -v 2025-12-17 15:04:43 -08:00
e6cb906d35 Change operational log messages from Debug to Info 2025-12-17 15:02:46 -08:00
45201509ff Fix progress line not clearing before final status 2025-12-17 15:01:32 -08:00
21028af9aa Replace gzip with zstd compression
Use zstd with SpeedBestCompression level for better compression
ratios. Remove gzip support entirely. Include generated protobuf
file to allow building without protoc.
2025-12-17 14:49:30 -08:00
150bac82cf Add TODO.md with 1.0 release tasks 2025-12-17 14:40:27 -08:00
92bd13efde Fix all linter errors
- Add explicit error ignoring with _ = for Close/Remove calls
- Rename WriteTo to Write to avoid io.WriterTo interface conflict
- Fix errcheck warnings in fetch, freshen, gen, mfer, checker,
  deserialize, serialize, and output files
2025-12-17 14:37:52 -08:00
531f460f87 Add CLAUDE.md 2025-12-17 14:33:00 -08:00
c5ca3e2ced Change FileProgress callback to channel-based progress
Replace callback-based progress reporting in Builder.AddFile with
channel-based FileHashProgress for consistency with EnumerateStatus
and ScanStatus patterns. Update scanner.go to use the new channel API.
2025-12-17 14:30:10 -08:00
fded1a0393 Remove generateInner, now handled by Builder
- Remove generateInner() from serialize.go
- Update generate() to error if pbInner not set
- Remove legacy tests that depended on old code path
- Update TODO item to reflect removal
2025-12-17 11:27:41 -08:00
5d7c729efb remove golangci-lint config files 2025-12-17 11:27:41 -08:00
48c3c09d85 Rename ManifestBuilder to Builder 2025-12-17 11:27:41 -08:00
f3be3eba84 Add TODO: change FileProgress callback to channel-based 2025-12-17 11:27:41 -08:00
5e65b3a0fd Add TODO section to README with prioritized task list 2025-12-17 11:27:41 -08:00
79fc5cca6c Add godoc strings to all exported types, functions, and fields
Documents:
- cli: NO_COLOR, RunOptions fields, CLIApp, VersionString
- checker: Result fields, Status constants, CheckStatus fields
- scanner: EnumerateStatus, ScanStatus, Options, FileEntry fields
- log: Level alias, DisableStyling, Init, Info/Debug functions,
  verbosity helpers, GetLogger, GetLevel, WithError
- mfer: ManifestScanOptions, New, NewFromPaths, NewFromFS, MAGIC
2025-12-17 11:27:41 -08:00
155ebe9a78 Merge branch 'main' into next 2025-12-17 19:01:52 +00:00
dc2ea47f6a Add testable CLI with dependency injection and new scanner/checker packages
Major changes:
- Refactor CLI to accept injected I/O streams and filesystem (afero.Fs)
  for testing without touching the real filesystem
- Add RunOptions struct and RunWithOptions() for configurable CLI execution
- Add internal/scanner package with two-phase manifest generation:
  - Phase 1 (Enumeration): walk directories, collect metadata
  - Phase 2 (Scan): read contents, compute hashes, write manifest
- Add internal/checker package for manifest verification with progress
  reporting and channel-based result streaming
- Add mfer/builder.go for incremental manifest construction
- Add --no-extra-files flag to check command to detect files not in manifest
- Add timing summaries showing file count, size, elapsed time, and throughput
- Add comprehensive tests using afero.MemMapFs (no real filesystem access)
- Add contrib/usage.sh integration test script
- Fix banner ASCII art alignment (consistent spacing)
- Fix verbosity levels so summaries display at default log level
- Update internal/log to support configurable output writers
2025-12-17 11:00:55 -08:00
2717685619 update readme to conform with my new readme howto standards (#8)
All checks were successful
continuous-integration/drone/push Build is passing
Reviewed-on: #8

manually approving because CI is offline rn for some reason
2024-12-09 02:50:57 +00:00
13f39d598f remove gofumpt from linting
Some checks failed
continuous-integration/drone/push Build is passing
continuous-integration/drone/pr Build is failing
2024-05-22 14:34:05 -07:00
01bffc8388 latest
All checks were successful
continuous-integration/drone/push Build is passing
2024-05-22 14:25:35 -07:00
7df558d8d0 next (#5)
All checks were successful
continuous-integration/drone/push Build is passing
Co-authored-by: sneak <sneak@sneak.berlin>
Reviewed-on: #5
2022-12-09 00:02:33 +00:00
7a8a1b4a4a rollup from next branch (#4)
All checks were successful
continuous-integration/drone/push Build is passing
Co-authored-by: sneak <sneak@sneak.berlin>
Reviewed-on: #4
2022-12-04 07:59:36 +00:00
eb3b685aa3 i think this will build now
All checks were successful
continuous-integration/drone/push Build is passing
2022-02-02 00:26:18 -08:00
9d8616866f latest 2022-02-02 00:12:03 -08:00
696 changed files with 7561 additions and 248165 deletions

3
.dockerignore Normal file
View File

@@ -0,0 +1,3 @@
*.tmp
*.dockerimage
.git

View File

@@ -1,14 +0,0 @@
kind: pipeline
name: test-docker-build
steps:
- name: test-docker-build
image: plugins/docker
network_mode: bridge
settings:
repo: sneak/mfer
dry_run: true
custom_dns: [ 116.202.204.30 ]
tags:
- ${DRONE_COMMIT_SHA}
- ${DRONE_BRANCH}

10
.gitignore vendored
View File

@@ -1,2 +1,8 @@
*.pb.go
/mfer
/bin/
/tmp
*.tmp
*.dockerimage
/vendor
# Stale files
.drone.yml

BIN
.index.mf Normal file

Binary file not shown.

20
CLAUDE.md Normal file
View File

@@ -0,0 +1,20 @@
# Important Rules
- when fixing a bug, write a failing test FIRST. only after the test fails, write
the code to fix the bug. then ensure the test passes. leave the test in
place and commit it with the bugfix. don't run shell commands to test
bugfixes or reproduce bugs. write tests!
- never, ever mention claude or anthropic in commit messages. do not use attribution
- after each change, run "make fmt".
- after each change, run "make test" and ensure all tests pass.
- after each change, run "make lint" and ensure no linting errors. fix any
you find, one by one.
- after each change, commit the files you've changed. push after
committing.
- NEVER use `git add -A`. always add only individual files that you've changed.

View File

@@ -1,37 +1,37 @@
## lint image
## current as of 2022-01-25
FROM golangci/golangci-lint@sha256:d16ef91da7e10f3df45c36876543326abbc4c16aaab6548549560b9f52e9e831 AS linter
RUN mkdir -p /build
WORKDIR /build
COPY ./ ./
RUN golangci-lint run
## build image:
# this is golang:1.17.6-buster as of 2022-01-27
FROM golang@sha256:52a48e0239f4d645b20ac268a60361703afe7feb2df5697fa89f72052cb87a3e AS builder
#FROM golang:1.16-buster AS builder
################################################################################
#2345678911234567892123456789312345678941234567895123456789612345678971234567898
################################################################################
FROM sneak/builder:2022-12-08 AS builder
ENV DEBIAN_FRONTEND noninteractive
RUN apt update && apt install -y make bzip2 protobuf-compiler
RUN mkdir -p /build
WORKDIR /build
COPY ./Makefile ./.golangci.yml ./go.mod ./go.sum /build/
COPY ./vendor.tzst /build/vendor.tzst
COPY ./modcache.tzst /build/modcache.tzst
COPY ./internal ./internal
COPY ./bin/gitrev.sh ./bin/gitrev.sh
COPY ./mfer ./mfer
COPY ./cmd ./cmd
ARG GITREV unknown
ARG DRONE_COMMIT_SHA unknown
COPY go.mod .
COPY go.sum .
COPY ./ ./
# don't lint again during build because there's no golangci-lint in this
# image and we already did it in a previous stage
#RUN make lint
RUN make mfer
RUN go mod vendor
RUN tar -c . | bzip2 > /src.tbz2
RUN mkdir -p "$(go env GOMODCACHE)" && cd "$(go env GOMODCACHE)" && \
zstdmt -d --stdout /build/modcache.tzst | tar xf - && \
rm /build/modcache.tzst && cd /build
RUN \
cd mfer && go generate . && cd .. && \
GOPACKAGESDEBUG=true golangci-lint run ./... && \
mkdir vendor && cd vendor && \
zstdmt -d --stdout /build/vendor.tzst | tar xf - && rm /build/vendor.tzst && \
cd .. && \
make mfer.cmd
RUN rm -rf /build/vendor && go mod vendor && tar -c . | zstdmt -19 > /src.tzst
################################################################################
#2345678911234567892123456789312345678941234567895123456789612345678971234567898
################################################################################
## final image
################################################################################
FROM scratch
COPY --from=builder /src.tbz2 /src.tbz2
COPY --from=builder /build/mfer /mfer
# we put all the source into the final image for posterity, it's small
COPY --from=builder /src.tzst /src.tzst
COPY --from=builder /build/mfer.cmd /mfer
ENTRYPOINT ["/mfer"]

View File

@@ -1,44 +1,81 @@
export DOCKER_BUILDKIT := 1
export PROGRESS_NO_TRUNC := 1
GOPATH := $(shell go env GOPATH)
export PATH := $(PATH):$(GOPATH)/bin
PROTOC_GEN_GO := $(GOPATH)/bin/protoc-gen-go
SOURCEFILES := mfer/*.go mfer/*.proto internal/*/*.go cmd/*/*.go go.mod go.sum
ARCH := $(shell uname -m)
VERSION := $(shell git describe --always --dirty=-dirty)
GITREV_BUILD := $(shell bash $(PWD)/bin/gitrev.sh)
APPNAME := mfer
VERSION := 0.1.0
export DOCKER_IMAGE_CACHE_DIR := $(HOME)/Library/Caches/Docker/$(APPNAME)-$(ARCH)
GOLDFLAGS += -X main.Version=$(VERSION)
GOLDFLAGS += -X main.Buildarch=$(ARCH)
GOLDFLAGS += -X main.Gitrev=$(GITREV_BUILD)
GOFLAGS := -ldflags "$(GOLDFLAGS)"
default: run
.PHONY: docker default run ci test fixme
run: ./mfer
default: fmt test
run: ./bin/mfer
./$<
./$< gen
ci: test
test: $(SOURCEFILES) mfer/mf.pb.go
go test -v --timeout 10s ./...
$(PROTOC_GEN_GO):
test -e $(PROTOC_GEN_GO) || go install -v google.golang.org/protobuf/cmd/protoc-gen-go@latest
test -e $(PROTOC_GEN_GO) || go install -v google.golang.org/protobuf/cmd/protoc-gen-go@v1.28.1
fixme:
@grep -nir fixme . | grep -v Makefile
devprereqs:
which gofumpt || go install -v mvdan.cc/gofumpt@latest
which golangci-lint || go install -v github.com/golangci/golangci-lint/cmd/golangci-lint@latest
mfer: $(PROTOC_GEN_GO) src/*.go cmd/*/*.go
mfer/mf.pb.go: mfer/mf.proto
cd mfer && go generate .
bin/mfer: $(SOURCEFILES) mfer/mf.pb.go
protoc --version
cd proto && go generate .
cd cmd/mfer && go build -o ../../mfer $(GOFLAGS) .
cd cmd/mfer && go build -tags urfave_cli_no_docs -o ../../bin/mfer $(GOFLAGS) .
clean:
rm -rfv proto/*.pb.go ./mfer
rm -rfv mfer/*.pb.go bin/mfer cmd/mfer/mfer *.dockerimage
fmt: prereqs
gofumpt -l -w src cmd
fmt: mfer/mf.pb.go
gofumpt -l -w mfer internal cmd
golangci-lint run --fix
prettier -w *.json *.md
-prettier -w *.json
-prettier -w *.md
lint:
golangci-lint run
sh -c 'test -z "$$(gofmt -l .)"'
dockerbuild:
docker build .
docker: sneak-mfer.$(ARCH).tzst.dockerimage
sneak-mfer.$(ARCH).tzst.dockerimage: $(SOURCEFILES) vendor.tzst modcache.tzst
docker build --progress plain --build-arg GITREV=$(GITREV_BUILD) -t sneak/mfer .
docker save sneak/mfer | pv | zstdmt -19 > $@
du -sh $@
godoc:
open http://127.0.0.1:6060
godoc -http=:6060
vendor.tzst: go.mod go.sum
go mod tidy
go mod vendor
cd vendor && tar -c . | pv | zstdmt -19 > $(PWD)/$@.tmp
rm -rf vendor
mv $@.tmp $@
modcache.tzst: go.mod go.sum
go mod tidy
cd $(HOME)/go/pkg && chmod -R u+rw . && rm -rf mod sumdb
go mod download -x
cd $(shell go env GOMODCACHE) && tar -c . | pv | zstdmt -19 > $(PWD)/$@.tmp
mv $@.tmp $@

View File

@@ -1,11 +1,76 @@
# mfer
Manifest file generator and checker.
[mfer](https://git.eeqj.de/sneak/mfer) is a reference implementation library
and thin wrapper command-line utility written in [Go](https://golang.org)
and first published in 2022 under the [WTFPL](https://wtfpl.net) (public
domain) license. It specifies and generates `.mf` manifest files over a
directory tree of files to encapsulate metadata about them (such as
cryptographic checksums or signatures over same) to aid in archiving,
downloading, and streaming, or mirroring. The manifest files' data is
serialized with Google's [protobuf serialization
format](https://developers.google.com/protocol-buffers). The structure of
these files can be found [in the format
specification](https://git.eeqj.de/sneak/mfer/src/branch/main/mfer/mf.proto)
which is included in the [project
repository](https://git.eeqj.de/sneak/mfer).
The current version is pre-1.0 and while the repo was published in 2022,
there has not yet been any versioned release. [SemVer](https://semver.org)
will be used for releases.
This project was started by [@sneak](https://sneak.berlin) to scratch an
itch in 2022 and is currently a one-person effort, though the goal is for
this to emerge as a de-facto standard and be incorporated into other
software. A compatible javascript library is planned.
# Phases
Manifest generation happens in two distinct phases:
## Phase 1: Enumeration
Walking directories and calling `stat()` on files to collect metadata (path, size, mtime, ctime). This builds the list of files to be scanned. Relatively fast as it only reads filesystem metadata, not file contents.
**Progress:** `EnumerateStatus` with `FilesFound` and `BytesFound`
## Phase 2: Scan (ToManifest)
Reading file contents and computing cryptographic hashes for manifest generation. This is the expensive phase that reads all file data from disk.
**Progress:** `ScanStatus` with `TotalFiles`, `ScannedFiles`, `TotalBytes`, `ScannedBytes`, `BytesPerSec`
# Code Conventions
- **Logging:** Never use `fmt.Printf` or write to stdout/stderr directly in normal code. Use the `internal/log` package for all output (`log.Info`, `log.Infof`, `log.Debug`, `log.Debugf`, `log.Progressf`, `log.ProgressDone`).
- **Filesystem abstraction:** Use `github.com/spf13/afero` for filesystem operations to enable testing and flexibility.
- **CLI framework:** Use `github.com/urfave/cli/v2` for command-line interface.
- **Serialization:** Use Protocol Buffers for manifest file format.
- **Internal packages:** Non-exported implementation details go in `internal/` subdirectories.
- **Concurrency:** Use `sync.RWMutex` for protecting shared state; prefer channels for progress reporting.
- **Progress channels:** Use buffered channels (size 1) with non-blocking sends to avoid blocking the main operation if the consumer is slow.
- **Context support:** Long-running operations should accept `context.Context` for cancellation.
- **NO_COLOR:** Respect the `NO_COLOR` environment variable for disabling colored output.
- **Options pattern:** Use `NewWithOptions(opts *Options)` constructor pattern for configurable types.
# Build Status
[![Build Status](https://drone.datavi.be/api/badges/sneak/mfer/status.svg)](https://drone.datavi.be/sneak/mfer)
# Participation
The community is as yet nonexistent so there are no defined policies or
norms yet. Primary development happens on a privately-run Gitea instance at
[https://git.eeqj.de/sneak/mfer](https://git.eeqj.de/sneak/mfer) and issues
are [tracked there](https://git.eeqj.de/sneak/mfer/issues).
Changes must always be formatted with a standard `go fmt`, syntactically
valid, and must pass the linting defined in the repository (presently only
the `golangci-lint` defaults), which can be run with a `make lint`. The
`main` branch is protected and all changes must be made via [pull
requests](https://git.eeqj.de/sneak/mfer/pulls) and pass CI to be merged.
Any changes submitted to this project must also be
[WTFPL-licensed](https://wtfpl.net) to be considered.
# Problem Statement
Given a plain URL, there is no standard way to safely and programmatically
@@ -83,6 +148,10 @@ The manifest file would do several important things:
- metadata size should not be used as an excuse to sacrifice utility (such
as providing checksums over each chunk of a large file)
# Limitations
- **Manifest size:** Manifests must fit entirely in system memory during reading and writing.
# Open Questions
- Should the manifest file include checksums of individual file chunks, or just for the whole assembled file?
@@ -128,8 +197,7 @@ The manifest file would do several important things:
- a command line option to zero/omit mtime/ctime, as well as manifest
timestamp, and sort all directory listings so that manifest file
generation is deterministic/reproducible
- URL format `mfer fetch
https://exmaple.com/manifestdirectory/?key=5539AD00DE4C42F3AFE11575052443F4DF2A55C2`
- URL format `mfer fetch https://exmaple.com/manifestdirectory/?key=5539AD00DE4C42F3AFE11575052443F4DF2A55C2`
to assert in the URL which PGP signing key should be used in the manifest,
so that shared URLs have a cryptographic trust root
- a "well-known" key in the manifest that maps well known keys (could reuse
@@ -171,6 +239,15 @@ regardless of filesystem format.
Please email [`sneak@sneak.berlin`](mailto:sneak@sneak.berlin) with your
desired username for an account on this Gitea instance.
I am currently interested in hiring a contractor skilled with the Go
standard library interfaces to specify this tool in full and develop a
prototype implementation.
## Links
- Repo: [https://git.eeqj.de/sneak/mfer](https://git.eeqj.de/sneak/mfer)
- Issues: [https://git.eeqj.de/sneak/mfer/issues](https://git.eeqj.de/sneak/mfer/issues)
# Authors
- [@sneak &lt;sneak@sneak.berlin&gt;](mailto:sneak@sneak.berlin)
# License
- [WTFPL](https://wtfpl.net)

122
TODO.md Normal file
View File

@@ -0,0 +1,122 @@
# TODO: mfer 1.0
## Design Questions
*sneak: please answer inline below each question. These are preserved for posterity.*
### Format Design
**1. Should `MFFileChecksum` be simplified?**
Currently it's a separate message wrapping a single `bytes multiHash` field. Since multihash already self-describes the algorithm, `repeated bytes hashes` directly on `MFFilePath` would be simpler and reduce per-file protobuf overhead. Is the extra message layer intentional (e.g. planning to add per-hash metadata like `verified_at`)?
> *answer:*
**2. Should file permissions/mode be stored?**
The format stores mtime/ctime but not Unix file permissions. For archival use (ExFAT, filesystem-independent checksums) this may not matter, but for software distribution or filesystem restoration it's a gap. Should we reserve a field now (e.g. `optional uint32 mode = 305`) even if we don't populate it yet?
> *answer:*
**3. Should `atime` be removed from the schema?**
Access time is volatile, non-deterministic, and often disabled (`noatime`). Including it means two manifests of the same directory at different times will differ, which conflicts with the determinism goal. Remove it, or document it as "never set by default"?
> *answer:*
**4. What are the path normalization rules?**
The proto has `string path` with no specification about: always forward-slash? Must be relative? No `..` components allowed? UTF-8 NFC vs NFD normalization (macOS vs Linux)? Max path length? This is a security issue (path traversal) and a cross-platform compatibility issue. What rules should the spec mandate?
> *answer:*
**5. Should we add a version byte after the magic?**
Currently `ZNAVSRFG` is followed immediately by protobuf. Adding a version byte (`ZNAVSRFG\x01`) would allow future framing changes without requiring protobuf parsing to detect the version. `MFFileOuter.Version` serves this purpose but requires successful deserialization to read. Worth the extra byte?
> *answer:*
**6. Should we add a length-prefix after the magic?**
Protobuf is not self-delimiting. If we ever want to concatenate manifests or append data after the protobuf, the current framing is insufficient. Add a varint or fixed-width length-prefix?
> *answer:*
### Signature Design
**7. What does the outer SHA-256 hash cover — compressed or uncompressed data?**
The review notes it currently hashes compressed data (good for verifying before decompression), but this should be explicitly documented. Which is the intended behavior?
> *answer:*
**8. Should `signatureString()` sign raw bytes instead of a hex-encoded string?**
Currently the canonical string is `MAGIC-UUID-MULTIHASH` with hex encoding, which adds a transformation layer. Signing the raw `sha256` bytes (or compressed `innerMessage` directly) would be simpler. Keep the string format or switch to raw bytes?
> *answer:*
**9. Should we support detached signature files (`.mf.sig`)?**
Embedded signatures are better for single-file distribution. Detached `.mf.sig` files follow the familiar `SHASUMS`/`SHASUMS.asc` pattern and are simpler for HTTP serving. Support both modes?
> *answer:*
**10. GPG vs pure-Go crypto for signatures?**
Shelling out to `gpg` is fragile (may not be installed, version-dependent output). `github.com/ProtonMail/go-crypto` provides pure-Go OpenPGP, or we could go Ed25519/signify (simpler, no key management). Which direction?
> *answer:*
### Implementation Design
**11. Should manifests be deterministic by default?**
This means: sort file entries by path, omit `createdAt` timestamp (or make it opt-in), no `atime`. Should determinism be the default, with a `--include-timestamps` flag to opt in?
> *answer:*
**12. Should we consolidate or keep both scanner/checker implementations?**
There are two parallel implementations: `mfer/scanner.go` + `mfer/checker.go` (typed with `FileSize`, `RelFilePath`) and `internal/scanner/` + `internal/checker/` (raw `int64`, `string`). The `mfer/` versions are superior. Delete the `internal/` versions?
> *answer:*
**13. Should the `manifest` type be exported?**
Currently unexported with exported constructors (`New`, `NewFromPaths`, etc.). Consumers can't declare `var m *mfer.manifest`. Export the type, or define an interface?
> *answer:*
**14. What should the Go module path be for 1.0?**
Currently mixed between `sneak.berlin/go/mfer` and `git.eeqj.de/sneak/mfer`. Which is canonical?
> *answer:*
---
## Implementation Plan
### Phase 1: Foundation (format correctness)
- [ ] Delete `internal/scanner/` and `internal/checker/` — consolidate on `mfer/` package versions; update CLI code
- [ ] Add deterministic file ordering — sort entries by path (lexicographic, byte-order) in `Builder.Build()`; add test asserting byte-identical output from two runs
- [ ] Add decompression size limit — `io.LimitReader` in `deserializeInner()` with `m.pbOuter.Size` as bound
- [ ] Fix `errors.Is` dead code in checker — replace with `os.IsNotExist(err)` or `errors.Is(err, fs.ErrNotExist)`
- [ ] Fix `AddFile` to verify size — check `totalRead == size` after reading, return error on mismatch
- [ ] Specify path invariants — add proto comments (UTF-8, forward-slash, relative, no `..`, no leading `/`); validate in `Builder.AddFile` and `Builder.AddFileWithHash`
### Phase 2: CLI polish
- [ ] Fix flag naming — all CLI flags use kebab-case as primary (`--include-dotfiles`, `--follow-symlinks`)
- [ ] Fix URL construction in fetch — use `BaseURL.JoinPath()` or `url.JoinPath()` instead of string concatenation
- [ ] Add progress rate-limiting to Checker — throttle to once per second, matching Scanner
- [ ] Add `--deterministic` flag (or make it default) — omit `createdAt`, sort files
### Phase 3: Robustness
- [ ] Replace GPG subprocess with pure-Go crypto — `github.com/ProtonMail/go-crypto` or Ed25519/signify
- [ ] Add timeout to any remaining subprocess calls
- [ ] Add fuzzing tests for `NewManifestFromReader`
- [ ] Add retry logic to fetch — exponential backoff for transient HTTP errors
### Phase 4: Format finalization
- [ ] Remove or deprecate `atime` from proto (pending design question answer)
- [ ] Reserve `optional uint32 mode = 305` in `MFFilePath` for future file permissions
- [ ] Add version byte after magic — `ZNAVSRFG\x01` for format version 1
- [ ] Write format specification document — separate from README: magic, outer structure, compression, inner structure, path invariants, signature scheme, canonical serialization
### Phase 5: Release prep
- [ ] Finalize Go module path
- [ ] Audit all error messages for consistency and helpfulness
- [ ] Add `--version` output matching SemVer
- [ ] Tag v1.0.0

12
bin/gitrev.sh Normal file
View File

@@ -0,0 +1,12 @@
#!/bin/bash
#
if [[ ! -z "$DRONE_COMMIT_SHA" ]]; then
echo "${DRONE_COMMIT_SHA:0:7}"
exit 0
fi
if [[ ! -z "$GITREV" ]]; then
echo $GITREV
else
git describe --always --dirty=-dirty
fi

View File

@@ -3,15 +3,15 @@ package main
import (
"os"
mfer "git.eeqj.de/sneak/mfer/src"
"sneak.berlin/go/mfer/internal/cli"
)
var (
Appname string = "mfer"
Version string
Buildarch string
Appname string = "mfer"
Version string
Gitrev string
)
func main() {
os.Exit(mfer.Run(Appname, Version, Buildarch))
os.Exit(cli.Run(Appname, Version, Gitrev))
}

11
cmd/mfer/main_test.go Normal file
View File

@@ -0,0 +1,11 @@
package main
import (
"testing"
"github.com/stretchr/testify/assert"
)
func TestBuild(t *testing.T) {
assert.True(t, true)
}

19
contrib/usage.sh Executable file
View File

@@ -0,0 +1,19 @@
#!/bin/bash
set -euo pipefail
# usage.sh - Generate and check a manifest from the repo
# Run from repo root: ./contrib/usage.sh
TMPDIR=$(mktemp -d)
MANIFEST="$TMPDIR/index.mf"
cleanup() {
rm -rf "$TMPDIR"
}
trap cleanup EXIT
echo "Building mfer..."
go build -o "$TMPDIR/mfer" ./cmd/mfer
"$TMPDIR/mfer" generate -o "$MANIFEST" .
"$TMPDIR/mfer" check --base . "$MANIFEST"

43
go.mod
View File

@@ -1,9 +1,44 @@
module git.eeqj.de/sneak/mfer
module sneak.berlin/go/mfer
go 1.16
go 1.23
require (
github.com/apex/log v1.9.0
github.com/davecgh/go-spew v1.1.1
github.com/dustin/go-humanize v1.0.1
github.com/google/uuid v1.1.2
github.com/klauspost/compress v1.18.2
github.com/multiformats/go-multihash v0.2.3
github.com/pterm/pterm v0.12.35
github.com/urfave/cli/v2 v2.3.0
google.golang.org/protobuf v1.27.1
github.com/spf13/afero v1.8.0
github.com/stretchr/testify v1.8.1
github.com/urfave/cli/v2 v2.23.6
google.golang.org/protobuf v1.28.1
)
require (
github.com/atomicgo/cursor v0.0.1 // indirect
github.com/cpuguy83/go-md2man/v2 v2.0.2 // indirect
github.com/fatih/color v1.7.0 // indirect
github.com/gookit/color v1.4.2 // indirect
github.com/klauspost/cpuid/v2 v2.0.9 // indirect
github.com/mattn/go-colorable v0.1.2 // indirect
github.com/mattn/go-isatty v0.0.8 // indirect
github.com/mattn/go-runewidth v0.0.13 // indirect
github.com/minio/sha256-simd v1.0.0 // indirect
github.com/mr-tron/base58 v1.2.0 // indirect
github.com/multiformats/go-varint v0.0.6 // indirect
github.com/pkg/errors v0.9.1 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/rivo/uniseg v0.2.0 // indirect
github.com/russross/blackfriday/v2 v2.1.0 // indirect
github.com/spaolacci/murmur3 v1.1.0 // indirect
github.com/xo/terminfo v0.0.0-20210125001918-ca9a967f8778 // indirect
github.com/xrash/smetrics v0.0.0-20201216005158-039620a65673 // indirect
golang.org/x/crypto v0.0.0-20220525230936-793ad666bf5e // indirect
golang.org/x/sys v0.1.0 // indirect
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211 // indirect
golang.org/x/text v0.3.6 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
lukechampine.com/blake3 v1.1.6 // indirect
)

533
go.sum
View File

@@ -1,28 +1,196 @@
cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
cloud.google.com/go v0.34.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
cloud.google.com/go v0.38.0/go.mod h1:990N+gfupTy94rShfmMCWGDn0LpTmnzTp2qbd1dvSRU=
cloud.google.com/go v0.44.1/go.mod h1:iSa0KzasP4Uvy3f1mN/7PiObzGgflwredwwASm/v6AU=
cloud.google.com/go v0.44.2/go.mod h1:60680Gw3Yr4ikxnPRS/oxxkBccT6SA1yMk63TGekxKY=
cloud.google.com/go v0.44.3/go.mod h1:60680Gw3Yr4ikxnPRS/oxxkBccT6SA1yMk63TGekxKY=
cloud.google.com/go v0.45.1/go.mod h1:RpBamKRgapWJb87xiFSdk4g1CME7QZg3uwTez+TSTjc=
cloud.google.com/go v0.46.3/go.mod h1:a6bKKbmY7er1mI7TEI4lsAkts/mkhTSZK8w33B4RAg0=
cloud.google.com/go v0.50.0/go.mod h1:r9sluTvynVuxRIOHXQEHMFffphuXHOMZMycpNR5e6To=
cloud.google.com/go v0.52.0/go.mod h1:pXajvRH/6o3+F9jDHZWQ5PbGhn+o8w9qiu/CffaVdO4=
cloud.google.com/go v0.53.0/go.mod h1:fp/UouUEsRkN6ryDKNW/Upv/JBKnv6WDthjR6+vze6M=
cloud.google.com/go v0.54.0/go.mod h1:1rq2OEkV3YMf6n/9ZvGWI3GWw0VoqH/1x2nd8Is/bPc=
cloud.google.com/go v0.56.0/go.mod h1:jr7tqZxxKOVYizybht9+26Z/gUq7tiRzu+ACVAMbKVk=
cloud.google.com/go v0.57.0/go.mod h1:oXiQ6Rzq3RAkkY7N6t3TcE6jE+CIBBbA36lwQ1JyzZs=
cloud.google.com/go v0.62.0/go.mod h1:jmCYTdRCQuc1PHIIJ/maLInMho30T/Y0M4hTdTShOYc=
cloud.google.com/go v0.65.0/go.mod h1:O5N8zS7uWy9vkA9vayVHs65eM1ubvY4h553ofrNHObY=
cloud.google.com/go v0.72.0/go.mod h1:M+5Vjvlc2wnp6tjzE102Dw08nGShTscUx2nZMufOKPI=
cloud.google.com/go v0.74.0/go.mod h1:VV1xSbzvo+9QJOxLDaJfTjx5e+MePCpCWwvftOeQmWk=
cloud.google.com/go v0.75.0/go.mod h1:VGuuCn7PG0dwsd5XPVm2Mm3wlh3EL55/79EKB6hlPTY=
cloud.google.com/go/bigquery v1.0.1/go.mod h1:i/xbL2UlR5RvWAURpBYZTtm/cXjCha9lbfbpx4poX+o=
cloud.google.com/go/bigquery v1.3.0/go.mod h1:PjpwJnslEMmckchkHFfq+HTD2DmtT67aNFKH1/VBDHE=
cloud.google.com/go/bigquery v1.4.0/go.mod h1:S8dzgnTigyfTmLBfrtrhyYhwRxG72rYxvftPBK2Dvzc=
cloud.google.com/go/bigquery v1.5.0/go.mod h1:snEHRnqQbz117VIFhE8bmtwIDY80NLUZUMb4Nv6dBIg=
cloud.google.com/go/bigquery v1.7.0/go.mod h1://okPTzCYNXSlb24MZs83e2Do+h+VXtc4gLoIoXIAPc=
cloud.google.com/go/bigquery v1.8.0/go.mod h1:J5hqkt3O0uAFnINi6JXValWIb1v0goeZM77hZzJN/fQ=
cloud.google.com/go/datastore v1.0.0/go.mod h1:LXYbyblFSglQ5pkeyhO+Qmw7ukd3C+pD7TKLgZqpHYE=
cloud.google.com/go/datastore v1.1.0/go.mod h1:umbIZjpQpHh4hmRpGhH4tLFup+FVzqBi1b3c64qFpCk=
cloud.google.com/go/pubsub v1.0.1/go.mod h1:R0Gpsv3s54REJCy4fxDixWD93lHJMoZTyQ2kNxGRt3I=
cloud.google.com/go/pubsub v1.1.0/go.mod h1:EwwdRX2sKPjnvnqCa270oGRyludottCI76h+R3AArQw=
cloud.google.com/go/pubsub v1.2.0/go.mod h1:jhfEVHT8odbXTkndysNHCcx0awwzvfOlguIAii9o8iA=
cloud.google.com/go/pubsub v1.3.1/go.mod h1:i+ucay31+CNRpDW4Lu78I4xXG+O1r/MAHgjpRVR+TSU=
cloud.google.com/go/storage v1.0.0/go.mod h1:IhtSnM/ZTZV8YYJWCY8RULGVqBDmpoyjwiyrjsg+URw=
cloud.google.com/go/storage v1.5.0/go.mod h1:tpKbwo567HUNpVclU5sGELwQWBDZ8gh0ZeosJ0Rtdos=
cloud.google.com/go/storage v1.6.0/go.mod h1:N7U0C8pVQ/+NIKOBQyamJIeKQKkZ+mxpohlUTyfDhBk=
cloud.google.com/go/storage v1.8.0/go.mod h1:Wv1Oy7z6Yz3DshWRJFhqM/UCfaWIRTdp0RXyy7KQOVs=
cloud.google.com/go/storage v1.10.0/go.mod h1:FLPqc6j+Ki4BU591ie1oL6qBQGu2Bl/tZ9ullr3+Kg0=
cloud.google.com/go/storage v1.14.0/go.mod h1:GrKmX003DSIwi9o29oFT7YDnHYwZoctc3fOKtUw0Xmo=
dmitri.shuralyov.com/gpu/mtl v0.0.0-20190408044501-666a987793e9/go.mod h1:H6x//7gZCb22OMCxBHrMx7a5I7Hp++hsVxbQ4BYO7hU=
github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
github.com/BurntSushi/xgb v0.0.0-20160522181843-27f122750802/go.mod h1:IVnqGOEym/WlBOVXweHU+Q+/VP0lqqI8lqeDx9IjBqo=
github.com/MarvinJWendt/testza v0.1.0/go.mod h1:7AxNvlfeHP7Z/hDQ5JtE3OKYT3XFUeLCDE2DQninSqs=
github.com/MarvinJWendt/testza v0.2.1/go.mod h1:God7bhG8n6uQxwdScay+gjm9/LnO4D3kkcZX4hv9Rp8=
github.com/MarvinJWendt/testza v0.2.8/go.mod h1:nwIcjmr0Zz+Rcwfh3/4UhBp7ePKVhuBExvZqnKYWlII=
github.com/MarvinJWendt/testza v0.2.10/go.mod h1:pd+VWsoGUiFtq+hRKSU1Bktnn+DMCSrDrXDpX2bG66k=
github.com/MarvinJWendt/testza v0.2.12 h1:/PRp/BF+27t2ZxynTiqj0nyND5PbOtfJS0SuTuxmgeg=
github.com/MarvinJWendt/testza v0.2.12/go.mod h1:JOIegYyV7rX+7VZ9r77L/eH6CfJHHzXjB69adAhzZkI=
github.com/apex/log v1.9.0 h1:FHtw/xuaM8AgmvDDTI9fiwoAL25Sq2cxojnZICUU8l0=
github.com/apex/log v1.9.0/go.mod h1:m82fZlWIuiWzWP04XCTXmnX0xRkYYbCdYn8jbJeLBEA=
github.com/apex/logs v1.0.0/go.mod h1:XzxuLZ5myVHDy9SAmYpamKKRNApGj54PfYLcFrXqDwo=
github.com/aphistic/golf v0.0.0-20180712155816-02c07f170c5a/go.mod h1:3NqKYiepwy8kCu4PNA+aP7WUV72eXWJeP9/r3/K9aLE=
github.com/aphistic/sweet v0.2.0/go.mod h1:fWDlIh/isSE9n6EPsRmC0det+whmX6dJid3stzu0Xys=
github.com/atomicgo/cursor v0.0.1 h1:xdogsqa6YYlLfM+GyClC/Lchf7aiMerFiZQn7soTOoU=
github.com/atomicgo/cursor v0.0.1/go.mod h1:cBON2QmmrysudxNBFthvMtN32r3jxVRIvzkUiF/RuIk=
github.com/cpuguy83/go-md2man/v2 v2.0.0-20190314233015-f79a8a8ca69d h1:U+s90UTSYgptZMwQh2aRr3LuazLJIa+Pg3Kc1ylSYVY=
github.com/cpuguy83/go-md2man/v2 v2.0.0-20190314233015-f79a8a8ca69d/go.mod h1:maD7wRr/U5Z6m/iR4s+kqSMx2CaBsrgA7czyZG/E6dU=
github.com/aws/aws-sdk-go v1.20.6/go.mod h1:KmX6BPdI08NWTb3/sm4ZGu5ShLoqVDhKgpiN924inxo=
github.com/aybabtme/rgbterm v0.0.0-20170906152045-cc83f3b3ce59/go.mod h1:q/89r3U2H7sSsE2t6Kca0lfwTK8JdoNGS/yzM/4iH5I=
github.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU=
github.com/chzyer/logex v1.1.10/go.mod h1:+Ywpsq7O8HXn0nuIou7OrIPyXbp3wmkHB+jjWRnGsAI=
github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e/go.mod h1:nSuG5e5PlCu98SY8svDHJxuZscDgtXS6KTTbou5AhLI=
github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1/go.mod h1:Q3SI9o4m/ZMnBNeIyt5eFwwo7qiLfzFZmjNmxjkiQlU=
github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw=
github.com/cncf/udpa/go v0.0.0-20191209042840-269d4d468f6f/go.mod h1:M8M6+tZqaGXZJjfX53e64911xZQV5JYwmTeXPW+k8Sc=
github.com/cncf/udpa/go v0.0.0-20200629203442-efcf912fb354/go.mod h1:WmhPx2Nbnhtbo57+VJT5O0JRkEi1Wbu0z5j0R8u5Hbk=
github.com/cncf/udpa/go v0.0.0-20201120205902-5459f2c99403/go.mod h1:WmhPx2Nbnhtbo57+VJT5O0JRkEi1Wbu0z5j0R8u5Hbk=
github.com/cpuguy83/go-md2man/v2 v2.0.2 h1:p1EgwI/C7NhT0JmVkwCD2ZBK8j4aeHQX2pMHHBfMQ6w=
github.com/cpuguy83/go-md2man/v2 v2.0.2/go.mod h1:tgQtvFlXSQOSOSIRvRPT7W67SCa46tRHOmNcaadrF8o=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
github.com/envoyproxy/go-control-plane v0.9.0/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
github.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
github.com/envoyproxy/go-control-plane v0.9.4/go.mod h1:6rpuAdCZL397s3pYoYcLgu1mIlRU8Am5FuJP05cCM98=
github.com/envoyproxy/go-control-plane v0.9.7/go.mod h1:cwu0lG7PUMfa9snN8LXBig5ynNVH9qI8YYLbd1fK2po=
github.com/envoyproxy/go-control-plane v0.9.9-0.20201210154907-fd9021fe5dad/go.mod h1:cXg6YxExXjJnVBQHBLXeUAgxn2UodCpnH306RInaBQk=
github.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c=
github.com/fatih/color v1.7.0 h1:DkWD4oS2D8LGGgTQ6IvwJJXSL5Vp2ffcQg58nFV38Ys=
github.com/fatih/color v1.7.0/go.mod h1:Zm6kSWBoL9eyXnKyktHP6abPY2pDugNf5KwzbycvMj4=
github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo=
github.com/go-gl/glfw v0.0.0-20190409004039-e6da0acd62b1/go.mod h1:vR7hzQXu2zJy9AVAgeJqvqgH9Q5CA+iKCZ2gyEVpxRU=
github.com/go-gl/glfw/v3.3/glfw v0.0.0-20191125211704-12ad95a8df72/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8=
github.com/go-gl/glfw/v3.3/glfw v0.0.0-20200222043503-6f7a984d4dc4/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8=
github.com/go-logfmt/logfmt v0.4.0/go.mod h1:3RMwSq7FuexP4Kalkev3ejPJsZTpXXBr9+V4qmtdjCk=
github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q=
github.com/golang/groupcache v0.0.0-20190702054246-869f871628b6/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
github.com/golang/groupcache v0.0.0-20191227052852-215e87163ea7/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
github.com/golang/mock v1.1.1/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=
github.com/golang/mock v1.2.0/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=
github.com/golang/mock v1.3.1/go.mod h1:sBzyDLLjw3U8JLTeZvSv8jJB+tU5PVekmnlKIyFUx0Y=
github.com/golang/mock v1.4.0/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw=
github.com/golang/mock v1.4.1/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw=
github.com/golang/mock v1.4.3/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw=
github.com/golang/mock v1.4.4/go.mod h1:l3mdAwkq5BuhzHwde/uurv3sEJeZMXNpwsxVWU71h+4=
github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.3.3/go.mod h1:vzj43D7+SQXF/4pzW/hwtAqwc6iTitCiVSaWz5lYuqw=
github.com/golang/protobuf v1.3.4/go.mod h1:vzj43D7+SQXF/4pzW/hwtAqwc6iTitCiVSaWz5lYuqw=
github.com/golang/protobuf v1.3.5/go.mod h1:6O5/vntMXwX2lRkT1hjjk0nAC1IDOTvTlVgjlRvqsdk=
github.com/golang/protobuf v1.4.0-rc.1/go.mod h1:ceaxUfeHdC40wWswd/P6IGgMaK3YpKi5j83Wpe3EHw8=
github.com/golang/protobuf v1.4.0-rc.1.0.20200221234624-67d41d38c208/go.mod h1:xKAWHe0F5eneWXFV3EuXVDTCmh+JuBKY0li0aMyXATA=
github.com/golang/protobuf v1.4.0-rc.2/go.mod h1:LlEzMj4AhA7rCAGe4KMBDvJI+AwstrUpVNzEA03Pprs=
github.com/golang/protobuf v1.4.0-rc.4.0.20200313231945-b860323f09d0/go.mod h1:WU3c8KckQ9AFe+yFwt9sWVRKCVIyN9cPHBJSNnbL67w=
github.com/golang/protobuf v1.4.0/go.mod h1:jodUvKwWbYaEsadDk5Fwe5c77LiNKVO9IDvqG2KuDX0=
github.com/golang/protobuf v1.4.1/go.mod h1:U8fpvMrcmy5pZrNK1lt4xCsGvpyWQ/VVv6QDs8UjoX8=
github.com/golang/protobuf v1.4.2/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI=
github.com/golang/protobuf v1.4.3/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI=
github.com/golang/protobuf v1.5.0/go.mod h1:FsONVRAS9T7sI+LIUmWTfcYkHO4aIWwzhcaSAoJOfIk=
github.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=
github.com/google/btree v1.0.0/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=
github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M=
github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
github.com/google/go-cmp v0.4.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.4.1/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.1/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.2/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.4/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.5 h1:Khx7svrCpmxxtHBq5j2mp/xVjsi8hQMfNLvJFAlrGgU=
github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/martian v2.1.0+incompatible/go.mod h1:9I4somxYTbIHy5NJKHRl3wXiIaQGbYVAs8BPL6v8lEs=
github.com/google/martian/v3 v3.0.0/go.mod h1:y5Zk1BBys9G+gd6Jrk0W3cC1+ELVxBWuIGO+w/tUAp0=
github.com/google/martian/v3 v3.1.0/go.mod h1:y5Zk1BBys9G+gd6Jrk0W3cC1+ELVxBWuIGO+w/tUAp0=
github.com/google/pprof v0.0.0-20181206194817-3ea8567a2e57/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc=
github.com/google/pprof v0.0.0-20190515194954-54271f7e092f/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc=
github.com/google/pprof v0.0.0-20191218002539-d4f498aebedc/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=
github.com/google/pprof v0.0.0-20200212024743-f11f1df84d12/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=
github.com/google/pprof v0.0.0-20200229191704-1ebb73c60ed3/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=
github.com/google/pprof v0.0.0-20200430221834-fc25d7d30c6d/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=
github.com/google/pprof v0.0.0-20200708004538-1a94d8640e99/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=
github.com/google/pprof v0.0.0-20201023163331-3e6fc7fc9c4c/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=
github.com/google/pprof v0.0.0-20201203190320-1bf35d6f28c2/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=
github.com/google/pprof v0.0.0-20201218002935-b9804c9f04c2/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=
github.com/google/renameio v0.1.0/go.mod h1:KWCgfxg9yswjAJkECMjeO8J8rahYeXnNhOm40UhjYkI=
github.com/google/uuid v1.1.1/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/google/uuid v1.1.2 h1:EVhdT+1Kseyi1/pUmXKaFxYsDNy9RQYkMWRH68J/W7Y=
github.com/google/uuid v1.1.2/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/googleapis/gax-go/v2 v2.0.4/go.mod h1:0Wqv26UfaUD9n4G6kQubkQ+KchISgw+vpHVxEJEs9eg=
github.com/googleapis/gax-go/v2 v2.0.5/go.mod h1:DWXyrwAJ9X0FpwwEdw+IPEYBICEFu5mhpdKc/us6bOk=
github.com/googleapis/google-cloud-go-testing v0.0.0-20200911160855-bcd43fbb19e8/go.mod h1:dvDLG8qkwmyD9a/MJJN3XJcT3xFxOKAvTZGvuZmac9g=
github.com/gookit/color v1.4.2 h1:tXy44JFSFkKnELV6WaMo/lLfu/meqITX3iAV52do7lk=
github.com/gookit/color v1.4.2/go.mod h1:fqRyamkC1W8uxl+lxCQxOT09l/vYfZ+QeiX3rKQHCoQ=
github.com/hashicorp/golang-lru v0.5.0/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8=
github.com/hashicorp/golang-lru v0.5.1/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8=
github.com/hpcloud/tail v1.0.0/go.mod h1:ab1qPbhIpdTxEkNHXyeSf5vhxWSCs/tWer42PpOxQnU=
github.com/ianlancetaylor/demangle v0.0.0-20181102032728-5e5cf60278f6/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc=
github.com/ianlancetaylor/demangle v0.0.0-20200824232613-28f6c0f3b639/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc=
github.com/jmespath/go-jmespath v0.0.0-20180206201540-c2b33e8439af/go.mod h1:Nht3zPeWKUH0NzdCt2Blrr5ys8VGpn0CEB0cQHVjt7k=
github.com/jpillora/backoff v0.0.0-20180909062703-3050d21c67d7/go.mod h1:2iMrUgbbvHEiQClaW2NsSzMyGHqN+rDFqY705q49KG0=
github.com/jstemmer/go-junit-report v0.0.0-20190106144839-af01ea7f8024/go.mod h1:6v2b51hI/fHJwM22ozAgKL4VKDeJcHhJFhtBdhmNjmU=
github.com/jstemmer/go-junit-report v0.9.1/go.mod h1:Brl9GWCQeLvo8nXZwPNNblvFj/XSXhF0NWZEnDohbsk=
github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
github.com/klauspost/compress v1.18.2 h1:iiPHWW0YrcFgpBYhsA6D1+fqHssJscY/Tm/y2Uqnapk=
github.com/klauspost/compress v1.18.2/go.mod h1:R0h/fSBs8DE4ENlcrlib3PsXS61voFxhIs2DeRhCvJ4=
github.com/klauspost/cpuid/v2 v2.0.4/go.mod h1:FInQzS24/EEf25PyTYn52gqo7WaD8xa0213Md/qVLRg=
github.com/klauspost/cpuid/v2 v2.0.9 h1:lgaqFMSdTdQYdZ04uHyN2d/eKdOMyi2YLSvlQIBFYa4=
github.com/klauspost/cpuid/v2 v2.0.9/go.mod h1:FInQzS24/EEf25PyTYn52gqo7WaD8xa0213Md/qVLRg=
github.com/kr/fs v0.1.0/go.mod h1:FFnZGqtBN9Gxj7eW1uZ42v5BccTP0vu6NEaFoC2HwRg=
github.com/kr/logfmt v0.0.0-20140226030751-b84e30acd515/go.mod h1:+0opPa2QZZtGFBFZlji/RkVcI2GknAs/DXo4wKdlNEc=
github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
github.com/kr/pretty v0.2.0 h1:s5hAObm+yFO5uHYt5dYjxi2rXrsnmRpJx4OYvIWUaQs=
github.com/kr/pretty v0.2.0/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI=
github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
github.com/kr/text v0.1.0 h1:45sCR5RtlFHMR4UwH9sdQ5TC8v0qDQCHnXt+kaKSTVE=
github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
github.com/mattn/go-colorable v0.1.1/go.mod h1:FuOcm+DKB9mbwrcAfNl7/TZVBZ6rcnceauSikq3lYCQ=
github.com/mattn/go-colorable v0.1.2 h1:/bC9yWikZXAL9uJdulbSfyVNIR3n3trXl+v8+1sx8mU=
github.com/mattn/go-colorable v0.1.2/go.mod h1:U0ppj6V5qS13XJ6of8GYAs25YV2eR4EVcfRqFIhoBtE=
github.com/mattn/go-isatty v0.0.5/go.mod h1:Iq45c/XA43vh69/j3iqttzPXn0bhXyGjM0Hdxcsrc5s=
github.com/mattn/go-isatty v0.0.8 h1:HLtExJ+uU2HOZ+wI0Tt5DtUDrx8yhUqDcp7fYERX4CE=
github.com/mattn/go-isatty v0.0.8/go.mod h1:Iq45c/XA43vh69/j3iqttzPXn0bhXyGjM0Hdxcsrc5s=
github.com/mattn/go-runewidth v0.0.13 h1:lTGmDsbAYt5DmK6OnoV7EuIF1wEIFAcxld6ypU4OSgU=
github.com/mattn/go-runewidth v0.0.13/go.mod h1:Jdepj2loyihRzMpdS35Xk/zdY8IAYHsh153qUoGf23w=
github.com/mgutz/ansi v0.0.0-20170206155736-9520e82c474b/go.mod h1:01TrycV0kFyexm33Z7vhZRXopbI8J3TDReVlkTgMUxE=
github.com/minio/sha256-simd v1.0.0 h1:v1ta+49hkWZyvaKwrQB8elexRqm6Y0aMLjCNsrYxo6g=
github.com/minio/sha256-simd v1.0.0/go.mod h1:OuYzVNI5vcoYIAmbIvHPl3N3jUzVedXbKy5RFepssQM=
github.com/mr-tron/base58 v1.2.0 h1:T/HDJBh4ZCPbU39/+c3rRvE0uKBQlU27+QI8LJ4t64o=
github.com/mr-tron/base58 v1.2.0/go.mod h1:BinMc/sQntlIE1frQmRFPUoPA1Zkr8VRgBdjWI2mNwc=
github.com/multiformats/go-multihash v0.2.3 h1:7Lyc8XfX/IY2jWb/gI7JP+o7JEq9hOa7BFvVU9RSh+U=
github.com/multiformats/go-multihash v0.2.3/go.mod h1:dXgKXCXjBzdscBLk9JkjINiEsCKRVch90MdaGiKsvSM=
github.com/multiformats/go-varint v0.0.6 h1:gk85QWKxh3TazbLxED/NlDVv8+q+ReFJk7Y2W/KhfNY=
github.com/multiformats/go-varint v0.0.6/go.mod h1:3Ls8CIEsrijN6+B7PbrXRPxHRPuXSrVKRY101jdMZYE=
github.com/onsi/ginkgo v1.6.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
github.com/onsi/gomega v1.5.0/go.mod h1:ex+gbHU/CVuBBDIJjb2X0qEXbFg53c61hWP/1CpauHY=
github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pkg/sftp v1.13.1/go.mod h1:3HaPG6Dq1ILlpPZRO0HVMrsydcdLt6HRDccSgb87qRg=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
github.com/pterm/pterm v0.12.27/go.mod h1:PhQ89w4i95rhgE+xedAoqous6K9X+r6aSOI2eFF7DZI=
github.com/pterm/pterm v0.12.29/go.mod h1:WI3qxgvoQFFGKGjGnJR849gU0TsEOvKn5Q8LlY1U7lg=
github.com/pterm/pterm v0.12.30/go.mod h1:MOqLIyMOgmTDz9yorcYbcw+HsgoZo3BQfg2wtl3HEFE=
@@ -32,34 +200,369 @@ github.com/pterm/pterm v0.12.35 h1:A/vHwDM+WByn0sTPlpL2L6kOTy12xqZuwNFMF/NlA+U=
github.com/pterm/pterm v0.12.35/go.mod h1:NjiL09hFhT/vWjQHSj1athJpx6H8cjpHXNAK5bUw8T8=
github.com/rivo/uniseg v0.2.0 h1:S1pD9weZBuJdFmowNwbpi7BJ8TNftyUImj/0WQi72jY=
github.com/rivo/uniseg v0.2.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJtxc=
github.com/russross/blackfriday/v2 v2.0.1 h1:lPqVAte+HuHNfhJ/0LC98ESWRz8afy9tM/0RK8m9o+Q=
github.com/russross/blackfriday/v2 v2.0.1/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
github.com/shurcooL/sanitized_anchor_name v1.0.0 h1:PdmoCO6wvbs+7yrJyMORt4/BmY5IYyJwS/kOiWx8mHo=
github.com/shurcooL/sanitized_anchor_name v1.0.0/go.mod h1:1NzhyTcUVG4SuEtjjoZeVRXNmyL/1OwPU0+IJeTBvfc=
github.com/rogpeppe/fastuuid v1.1.0/go.mod h1:jVj6XXZzXRy/MSR5jhDC/2q6DgLz+nrA6LYCDYWNEvQ=
github.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4=
github.com/russross/blackfriday/v2 v2.1.0 h1:JIOH55/0cWyOuilr9/qlrm0BSXldqnqwMsf35Ld67mk=
github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
github.com/sergi/go-diff v1.0.0/go.mod h1:0CfEIISq7TuYL3j771MWULgwwjU+GofnZX9QAmXWZgo=
github.com/smartystreets/assertions v1.0.0/go.mod h1:kHHU4qYBaI3q23Pp3VPrmWhuIUrLW/7eUrw0BU5VaoM=
github.com/smartystreets/go-aws-auth v0.0.0-20180515143844-0c1422d1fdb9/go.mod h1:SnhjPscd9TpLiy1LpzGSKh3bXCfxxXuqd9xmQJy3slM=
github.com/smartystreets/gunit v1.0.0/go.mod h1:qwPWnhz6pn0NnRBP++URONOVyNkPyr4SauJk4cUOwJs=
github.com/spaolacci/murmur3 v1.1.0 h1:7c1g84S4BPRrfL5Xrdp6fOJ206sU9y293DDHaoy0bLI=
github.com/spaolacci/murmur3 v1.1.0/go.mod h1:JwIasOWyU6f++ZhiEuf87xNszmSA2myDM2Kzu9HwQUA=
github.com/spf13/afero v1.8.0 h1:5MmtuhAgYeU6qpa7w7bP0dv6MBYuup0vekhSpSkoq60=
github.com/spf13/afero v1.8.0/go.mod h1:CtAatgMJh6bJEIs48Ay/FOnkljP3WeGUG0MC1RfAqwo=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
github.com/stretchr/testify v1.5.1/go.mod h1:5W2xD1RspED5o8YsWQXVCued0rvSQ+mT+I5cxcmMvtA=
github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.7.0 h1:nwc3DEeHmmLAfoZucVR881uASk0Mfjw8xYJ99tb5CcY=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/urfave/cli/v2 v2.3.0 h1:qph92Y649prgesehzOrQjdWyxFOp/QVM+6imKHad91M=
github.com/urfave/cli/v2 v2.3.0/go.mod h1:LJmUH05zAU44vOAcrfzZQKsZbVcdbOG8rtL3/XcUArI=
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
github.com/stretchr/testify v1.8.1 h1:w7B6lhMri9wdJUVmEZPGGhZzrYTPvgJArz7wNPgYKsk=
github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
github.com/tj/assert v0.0.0-20171129193455-018094318fb0/go.mod h1:mZ9/Rh9oLWpLLDRpvE+3b7gP/C2YyLFYxNmcLnPTMe0=
github.com/tj/assert v0.0.3 h1:Df/BlaZ20mq6kuai7f5z2TvPFiwC3xaWJSDQNiIS3Rk=
github.com/tj/assert v0.0.3/go.mod h1:Ne6X72Q+TB1AteidzQncjw9PabbMp4PBMZ1k+vd1Pvk=
github.com/tj/go-buffer v1.1.0/go.mod h1:iyiJpfFcR2B9sXu7KvjbT9fpM4mOelRSDTbntVj52Uc=
github.com/tj/go-elastic v0.0.0-20171221160941-36157cbbebc2/go.mod h1:WjeM0Oo1eNAjXGDx2yma7uG2XoyRZTq1uv3M/o7imD0=
github.com/tj/go-kinesis v0.0.0-20171128231115-08b17f58cb1b/go.mod h1:/yhzCV0xPfx6jb1bBgRFjl5lytqVqZXEaeqWP8lTEao=
github.com/tj/go-spin v1.1.0/go.mod h1:Mg1mzmePZm4dva8Qz60H2lHwmJ2loum4VIrLgVnKwh4=
github.com/urfave/cli/v2 v2.23.6 h1:iWmtKD+prGo1nKUtLO0Wg4z9esfBM4rAV4QRLQiEmJ4=
github.com/urfave/cli/v2 v2.23.6/go.mod h1:GHupkWPMM0M/sj1a2b4wUrWBPzazNrIjouW6fmdJLxc=
github.com/xo/terminfo v0.0.0-20210125001918-ca9a967f8778 h1:QldyIu/L63oPpyvQmHgvgickp1Yw510KJOqX7H24mg8=
github.com/xo/terminfo v0.0.0-20210125001918-ca9a967f8778/go.mod h1:2MuV+tbUrU1zIOPMxZ5EncGwgmMJsa+9ucAQZXxsObs=
github.com/xrash/smetrics v0.0.0-20201216005158-039620a65673 h1:bAn7/zixMGCfxrRTfdpNzjtPYqr8smhKouy9mxVdGPU=
github.com/xrash/smetrics v0.0.0-20201216005158-039620a65673/go.mod h1:N3UwUGtsrSj3ccvlPHLoLsHnpR27oXr4ZE984MbSER8=
github.com/yuin/goldmark v1.1.25/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.1.32/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
go.opencensus.io v0.21.0/go.mod h1:mSImk1erAIZhrmZN+AvHh14ztQfjbGwt4TtuofqLduU=
go.opencensus.io v0.22.0/go.mod h1:+kGneAE2xo2IficOXnaByMWTGM9T73dGwxeWcUqIpI8=
go.opencensus.io v0.22.2/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=
go.opencensus.io v0.22.3/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=
go.opencensus.io v0.22.4/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=
go.opencensus.io v0.22.5/go.mod h1:5pWMHQbX5EPX2/62yrJeAkowc+lfs/XD7Uxpq3pI6kk=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20190426145343-a29dc8fdc734/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20190510104115-cbcb75029529/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20190605123033-f99c8df09eb5/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.0.0-20210421170649-83a5a9bb288b/go.mod h1:T9bdIzuCu7OtxOm1hfPfRQxPLYneinmdGuTeoZ9dtd4=
golang.org/x/crypto v0.0.0-20211108221036-ceb1ce70b4fa/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
golang.org/x/crypto v0.0.0-20220525230936-793ad666bf5e h1:T8NU3HyQ8ClP4SEE+KbFlg6n0NhuTsN4MyznaarGsZM=
golang.org/x/crypto v0.0.0-20220525230936-793ad666bf5e/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4=
golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/exp v0.0.0-20190306152737-a1d7652674e8/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/exp v0.0.0-20190510132918-efd6b22b2522/go.mod h1:ZjyILWgesfNpC6sMxTJOJm9Kp84zZh5NQWvqDGG3Qr8=
golang.org/x/exp v0.0.0-20190829153037-c13cbed26979/go.mod h1:86+5VVa7VpoJ4kLfm080zCjGlMRFzhUhsZKEZO7MGek=
golang.org/x/exp v0.0.0-20191030013958-a1ab85dbe136/go.mod h1:JXzH8nQsPlswgeRAPE3MuO9GYsAcnJvJ4vnMwN/5qkY=
golang.org/x/exp v0.0.0-20191129062945-2f5052295587/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4=
golang.org/x/exp v0.0.0-20191227195350-da58074b4299/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4=
golang.org/x/exp v0.0.0-20200119233911-0405dc783f0a/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4=
golang.org/x/exp v0.0.0-20200207192155-f17229e696bd/go.mod h1:J/WKrq2StrnmMY6+EHIKF9dgMWnmCNThgcyBT1FY9mM=
golang.org/x/exp v0.0.0-20200224162631-6cc2880d07d6/go.mod h1:3jZMyOhIsHpP37uCMkUooju7aAi5cS1Q23tOzKc+0MU=
golang.org/x/image v0.0.0-20190227222117-0694c2d4d067/go.mod h1:kZ7UVZpmo3dzQBMxlp+ypCbDeSB+sBbTgSJuh5dn5js=
golang.org/x/image v0.0.0-20190802002840-cff245a6509b/go.mod h1:FeLwcggjj3mMvU+oOTbSwawSJRM1uh48EjtB4UJZlP0=
golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
golang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961/go.mod h1:wehouNa3lNwaWXcvxsM5YxQ5yQlVC4a0KAMCusXpPoU=
golang.org/x/lint v0.0.0-20190301231843-5614ed5bae6f/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
golang.org/x/lint v0.0.0-20190313153728-d0100b6bd8b3/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
golang.org/x/lint v0.0.0-20190409202823-959b441ac422/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
golang.org/x/lint v0.0.0-20190909230951-414d861bb4ac/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
golang.org/x/lint v0.0.0-20190930215403-16217165b5de/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
golang.org/x/lint v0.0.0-20191125180803-fdd1cda4f05f/go.mod h1:5qLYkcX4OjUUV8bRuDixDT3tpyyb+LUpUlRWLxfhWrs=
golang.org/x/lint v0.0.0-20200130185559-910be7a94367/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY=
golang.org/x/lint v0.0.0-20200302205851-738671d3881b/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY=
golang.org/x/lint v0.0.0-20201208152925-83fdc39ff7b5/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY=
golang.org/x/mobile v0.0.0-20190312151609-d3739f865fa6/go.mod h1:z+o9i4GpDbdi3rU15maQ/Ox0txvL9dWGYEHz965HBQE=
golang.org/x/mobile v0.0.0-20190719004257-d2bd2a29d028/go.mod h1:E/iHnbuqvinMTCcRqshq8CkpyQDoeVncDDYHnLhea+o=
golang.org/x/mod v0.0.0-20190513183733-4bf6d317e70e/go.mod h1:mXi4GBBbnImb6dmsKGUJ2LatrhH/nqhxcFungHvyanc=
golang.org/x/mod v0.1.0/go.mod h1:0QHyrYULN0/3qlju5TqG8bIK38QM8yzMo5ekMj3DlcY=
golang.org/x/mod v0.1.1-0.20191105210325-c90efee705ee/go.mod h1:QqPTAvyqsEbceGzBzNggFXnrqF1CaUcvgkdR5Ot7KZg=
golang.org/x/mod v0.1.1-0.20191107180719-034126e5016b/go.mod h1:QqPTAvyqsEbceGzBzNggFXnrqF1CaUcvgkdR5Ot7KZg=
golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.4.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.4.1/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180906233101-161cd47e91fd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190108225652-1e06a53dbb7e/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190213061140-3a22650c66bd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190501004415-9ce7a6920f09/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190503192946-f4e77d36d62c/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190603091049-60506f45cf65/go.mod h1:HSz+uSET+XFnRR8LxR5pz3Of3rY3CfYBVs4xY44aLks=
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20190628185345-da137c7871d7/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20190724013045-ca1201d0de80/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20191209160850-c0dbc17a3553/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200114155413-6afb5195e5aa/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200202094626-16171245cfb2/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200222125558-5a598a2470a0/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200301022130-244492dfa37a/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200324143707-d3edc9973b7e/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
golang.org/x/net v0.0.0-20200501053045-e0ff5e5a1de5/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
golang.org/x/net v0.0.0-20200506145744-7e3656a0809f/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
golang.org/x/net v0.0.0-20200513185701-a91f0712d120/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
golang.org/x/net v0.0.0-20200520182314-0ba52f642ac2/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
golang.org/x/net v0.0.0-20200625001655-4c5254603344/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=
golang.org/x/net v0.0.0-20200707034311-ab3426394381/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=
golang.org/x/net v0.0.0-20200822124328-c89045814202/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=
golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
golang.org/x/net v0.0.0-20201031054903-ff519b6c9102/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
golang.org/x/net v0.0.0-20201209123823-ac852fbbde11/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/net v0.0.0-20201224014010-6772e930b67b/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
golang.org/x/oauth2 v0.0.0-20191202225959-858c2ad4c8b6/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
golang.org/x/oauth2 v0.0.0-20200107190931-bf48bf16ab8d/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
golang.org/x/oauth2 v0.0.0-20200902213428-5d25da1a8d43/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
golang.org/x/oauth2 v0.0.0-20201109201403-9fd604954f58/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
golang.org/x/oauth2 v0.0.0-20201208152858-08078c50e5b5/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
golang.org/x/oauth2 v0.0.0-20210218202405-ba52d332ba99/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20190227155943-e225da77a7e6/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20200317015054-43a5402ce75a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20200625203802-6e8e738ad208/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20201207232520-09787c993a3a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20180909124046-d0be0721c37e/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190222072716-a9d3bda3a223/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190312061237-fead79001313/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190502145724-3ef323f4f1fd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190507160741-ecd444e8653b/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190606165138-5da285871e9c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190624142023-c5567b49c5d0/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190726091711-fc99dfbffb4e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191001151750-bb3f8db39f24/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191204072324-ce4227a45e2e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191228213918-04cbcbbfeed8/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200113162924-86b910548bc1/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200122134326-e047566fdf82/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200202164722-d101bd2416d5/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200212091648-12a6c2dcc1e4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200223170610-d5e6a3e2c0ae/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200302150141-5c8b2ff67527/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200323222414-85ca7c5b95cd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200331124033-c3d80250170d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200501052902-10377860bb8e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200511232937-7e40ca221e25/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200515095857-1151b9dac4a9/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200523222454-059865788121/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200803210538-64077c9b5642/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200905004654-be1d3432aa8f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20201201145000-ef89a241ccb3/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210104204734-6f8348627aad/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210119212857-b64e53b001e4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210225134936-a50acf3fe073/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210330210617-4fbd30eecc44/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210423185535-09eb48e85fd7/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20211013075003-97ac67df715c h1:taxlMj0D/1sOAuv/CbSD+MMDof2vbyPTqz5FNYKpXt8=
golang.org/x/sys v0.0.0-20211013075003-97ac67df715c/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.1.0 h1:kunALQeHf1/185U1i0GOB/fy1IPRDDpuoOOqRReG57U=
golang.org/x/sys v0.1.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210220032956-6a3ed077a48d/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210615171337-6886f2dfbf5b/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211 h1:JGgROgKl9N8DuW20oFS5gxc+lE67/N3FcwmBPMe7ArY=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543 h1:E7g+9GITq07hpfrRu66IVDexMakfv52eLZ2CXBWiKr4=
golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.4/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.6 h1:aRYxNxv6iGQlyVaZmk6ZgYEDa+Jg18DxebPSrd6bg1M=
golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20191024005414-555d28b269f0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY=
golang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
golang.org/x/tools v0.0.0-20190312151545-0bb0c0a6e846/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
golang.org/x/tools v0.0.0-20190312170243-e65039ee4138/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
golang.org/x/tools v0.0.0-20190425150028-36563e24a262/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=
golang.org/x/tools v0.0.0-20190506145303-2d16b83fe98c/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=
golang.org/x/tools v0.0.0-20190524140312-2c0ae7006135/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=
golang.org/x/tools v0.0.0-20190606124116-d0a3d012864b/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc=
golang.org/x/tools v0.0.0-20190621195816-6e04913cbbac/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc=
golang.org/x/tools v0.0.0-20190628153133-6cdbf07be9d0/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc=
golang.org/x/tools v0.0.0-20190816200558-6889da9d5479/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20190911174233-4f2ddba30aff/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20191012152004-8de300cfc20a/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20191113191852-77e3bb0ad9e7/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20191115202509-3a792d9c32b2/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20191125144606-a911d9008d1f/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20191130070609-6e064ea0cf2d/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20191216173652-a0e659d51361/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20191227053925-7b8e75db28f4/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20200117161641-43d50277825c/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20200122220014-bf1340f18c4a/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20200130002326-2f3ba24bd6e7/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20200204074204-1cc6d1ef6c74/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20200207183749-b753a1ba74fa/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20200212150539-ea181f53ac56/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20200224181240-023911ca70b2/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20200227222343-706bc42d1f0d/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20200304193943-95d2e580d8eb/go.mod h1:o4KQGtdN14AW+yjsvvwRTJJuXz8XRtIHtEnmAXLyFUw=
golang.org/x/tools v0.0.0-20200312045724-11d5b4c81c7d/go.mod h1:o4KQGtdN14AW+yjsvvwRTJJuXz8XRtIHtEnmAXLyFUw=
golang.org/x/tools v0.0.0-20200331025713-a30bf2db82d4/go.mod h1:Sl4aGygMT6LrqrWclx+PTx3U+LnKx/seiNR+3G19Ar8=
golang.org/x/tools v0.0.0-20200501065659-ab2804fb9c9d/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
golang.org/x/tools v0.0.0-20200512131952-2bc93b1c0c88/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
golang.org/x/tools v0.0.0-20200515010526-7d3b6ebf133d/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
golang.org/x/tools v0.0.0-20200618134242-20370b0cb4b2/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
golang.org/x/tools v0.0.0-20200729194436-6467de6f59a7/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA=
golang.org/x/tools v0.0.0-20200804011535-6c149bb5ef0d/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA=
golang.org/x/tools v0.0.0-20200825202427-b303f430e36d/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA=
golang.org/x/tools v0.0.0-20200904185747-39188db58858/go.mod h1:Cj7w3i3Rnn0Xh82ur9kSqwfTHTeVxaDqrfMjpcNT6bE=
golang.org/x/tools v0.0.0-20201110124207-079ba7bd75cd/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
golang.org/x/tools v0.0.0-20201201161351-ac6f37ff4c2a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
golang.org/x/tools v0.0.0-20201208233053-a543418bbed2/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
golang.org/x/tools v0.0.0-20210105154028-b0ab187a4818/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
golang.org/x/tools v0.0.0-20210108195828-e2f9c7f1fc8e/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
golang.org/x/tools v0.1.0/go.mod h1:xkSsbof2nBLbhDlRMhhhyNLN/zl3eTqcnHD5viDpcZ0=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1 h1:go1bK/D/BFZV2I8cIQd1NKEZ+0owSTG1fDTci4IqFcE=
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
google.golang.org/api v0.4.0/go.mod h1:8k5glujaEP+g9n7WNsDg8QP6cUVNI86fCNMcbazEtwE=
google.golang.org/api v0.7.0/go.mod h1:WtwebWUNSVBH/HAw79HIFXZNqEvBhG+Ra+ax0hx3E3M=
google.golang.org/api v0.8.0/go.mod h1:o4eAsZoiT+ibD93RtjEohWalFOjRDx6CVaqeizhEnKg=
google.golang.org/api v0.9.0/go.mod h1:o4eAsZoiT+ibD93RtjEohWalFOjRDx6CVaqeizhEnKg=
google.golang.org/api v0.13.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI=
google.golang.org/api v0.14.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI=
google.golang.org/api v0.15.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI=
google.golang.org/api v0.17.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=
google.golang.org/api v0.18.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=
google.golang.org/api v0.19.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=
google.golang.org/api v0.20.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=
google.golang.org/api v0.22.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=
google.golang.org/api v0.24.0/go.mod h1:lIXQywCXRcnZPGlsd8NbLnOjtAoL6em04bJ9+z0MncE=
google.golang.org/api v0.28.0/go.mod h1:lIXQywCXRcnZPGlsd8NbLnOjtAoL6em04bJ9+z0MncE=
google.golang.org/api v0.29.0/go.mod h1:Lcubydp8VUV7KeIHD9z2Bys/sm/vGKnG1UHuDBSrHWM=
google.golang.org/api v0.30.0/go.mod h1:QGmEvQ87FHZNiUVJkT14jQNYJ4ZJjdRF23ZXz5138Fc=
google.golang.org/api v0.35.0/go.mod h1:/XrVsuzM0rZmrsbjJutiuftIzeuTQcEeaYcSk/mQ1dg=
google.golang.org/api v0.36.0/go.mod h1:+z5ficQTmoYpPn8LCUNVpK5I7hwkpjbcgqA7I34qYtE=
google.golang.org/api v0.40.0/go.mod h1:fYKFpnQN0DsDSKRVRcQSDQNtqWPfM9i+zNPxepjRCQ8=
google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM=
google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
google.golang.org/appengine v1.5.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
google.golang.org/appengine v1.6.1/go.mod h1:i06prIuMbXzDqacNJfV5OdTW448YApPu5ww/cMBSeb0=
google.golang.org/appengine v1.6.5/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc=
google.golang.org/appengine v1.6.6/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc=
google.golang.org/appengine v1.6.7/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc=
google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc=
google.golang.org/genproto v0.0.0-20190307195333-5fe7a883aa19/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=
google.golang.org/genproto v0.0.0-20190418145605-e7d98fc518a7/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=
google.golang.org/genproto v0.0.0-20190425155659-357c62f0e4bb/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=
google.golang.org/genproto v0.0.0-20190502173448-54afdca5d873/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=
google.golang.org/genproto v0.0.0-20190801165951-fa694d86fc64/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc=
google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc=
google.golang.org/genproto v0.0.0-20190911173649-1774047e7e51/go.mod h1:IbNlFCBrqXvoKpeg0TB2l7cyZUmoaFKYIwrEpbDKLA8=
google.golang.org/genproto v0.0.0-20191108220845-16a3f7862a1a/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
google.golang.org/genproto v0.0.0-20191115194625-c23dd37a84c9/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
google.golang.org/genproto v0.0.0-20191216164720-4f79533eabd1/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
google.golang.org/genproto v0.0.0-20191230161307-f3c370f40bfb/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
google.golang.org/genproto v0.0.0-20200115191322-ca5a22157cba/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
google.golang.org/genproto v0.0.0-20200122232147-0452cf42e150/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
google.golang.org/genproto v0.0.0-20200204135345-fa8e72b47b90/go.mod h1:GmwEX6Z4W5gMy59cAlVYjN9JhxgbQH6Gn+gFDQe2lzA=
google.golang.org/genproto v0.0.0-20200212174721-66ed5ce911ce/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
google.golang.org/genproto v0.0.0-20200224152610-e50cd9704f63/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
google.golang.org/genproto v0.0.0-20200228133532-8c2c7df3a383/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
google.golang.org/genproto v0.0.0-20200305110556-506484158171/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
google.golang.org/genproto v0.0.0-20200312145019-da6875a35672/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
google.golang.org/genproto v0.0.0-20200331122359-1ee6d9798940/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
google.golang.org/genproto v0.0.0-20200430143042-b979b6f78d84/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
google.golang.org/genproto v0.0.0-20200511104702-f5ebc3bea380/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
google.golang.org/genproto v0.0.0-20200515170657-fc4c6c6a6587/go.mod h1:YsZOwe1myG/8QRHRsmBRE1LrgQY60beZKjly0O1fX9U=
google.golang.org/genproto v0.0.0-20200526211855-cb27e3aa2013/go.mod h1:NbSheEEYHJ7i3ixzK3sjbqSGDJWnxyFXZblF3eUsNvo=
google.golang.org/genproto v0.0.0-20200618031413-b414f8b61790/go.mod h1:jDfRM7FcilCzHH/e9qn6dsT145K34l5v+OpcnNgKAAA=
google.golang.org/genproto v0.0.0-20200729003335-053ba62fc06f/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20200804131852-c06518451d9c/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20200825200019-8632dd797987/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20200904004341-0bd0a958aa1d/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20201109203340-2640f1f9cdfb/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20201201144952-b05cb90ed32e/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20201210142538-e3217bee35cc/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20201214200347-8c77b98c765d/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20210108203827-ffc7fda8c3d7/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20210226172003-ab064af71705/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c=
google.golang.org/grpc v1.20.1/go.mod h1:10oTOabMzJvdu6/UiuZezV6QK5dSlG84ov/aaiqXj38=
google.golang.org/grpc v1.21.1/go.mod h1:oYelfM1adQP15Ek0mdvEgi9Df8B9CZIaU1084ijfRaM=
google.golang.org/grpc v1.23.0/go.mod h1:Y5yQAOtifL1yxbo5wqy6BxZv8vAUGQwXBOALyacEbxg=
google.golang.org/grpc v1.25.1/go.mod h1:c3i+UQWmh7LiEpx4sFZnkU36qjEYZ0imhYfXVyQciAY=
google.golang.org/grpc v1.26.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=
google.golang.org/grpc v1.27.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=
google.golang.org/grpc v1.27.1/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=
google.golang.org/grpc v1.28.0/go.mod h1:rpkK4SK4GF4Ach/+MFLZUBavHOvF2JJB5uozKKal+60=
google.golang.org/grpc v1.29.1/go.mod h1:itym6AZVZYACWQqET3MqgPpjcuV5QH3BxFS3IjizoKk=
google.golang.org/grpc v1.30.0/go.mod h1:N36X2cJ7JwdamYAgDz+s+rVMFjt3numwzf/HckM8pak=
google.golang.org/grpc v1.31.0/go.mod h1:N36X2cJ7JwdamYAgDz+s+rVMFjt3numwzf/HckM8pak=
google.golang.org/grpc v1.31.1/go.mod h1:N36X2cJ7JwdamYAgDz+s+rVMFjt3numwzf/HckM8pak=
google.golang.org/grpc v1.33.2/go.mod h1:JMHMWHQWaTccqQQlmk3MJZS+GWXOdAesneDmEnv2fbc=
google.golang.org/grpc v1.34.0/go.mod h1:WotjhfgOW/POjDeRt8vscBtXq+2VjORFy659qA51WJ8=
google.golang.org/grpc v1.35.0/go.mod h1:qjiiYl8FncCW8feJPdyg3v6XW24KsRHe+dy9BAGRRjU=
google.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8=
google.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0=
google.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60/go.mod h1:cfTl7dwQJ+fmap5saPgwCLgHXTUD7jkjRqWcaiX5VyM=
google.golang.org/protobuf v1.20.1-0.20200309200217-e05f789c0967/go.mod h1:A+miEFZTKqfCUM6K7xSMQL9OKL/b6hQv+e19PK+JZNE=
google.golang.org/protobuf v1.21.0/go.mod h1:47Nbq4nVaFHyn7ilMalzfO3qCViNmqZ2kzikPIcrTAo=
google.golang.org/protobuf v1.22.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=
google.golang.org/protobuf v1.23.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=
google.golang.org/protobuf v1.23.1-0.20200526195155-81db48ad09cc/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=
google.golang.org/protobuf v1.24.0/go.mod h1:r/3tXBNzIEhYS9I1OUVjXDlt8tc493IdKGjtUeSXeh4=
google.golang.org/protobuf v1.25.0/go.mod h1:9JNX74DMeImyA3h4bdi1ymwjUzf21/xIlbajtzgsN7c=
google.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw=
google.golang.org/protobuf v1.27.1 h1:SnqbnDw1V7RiZcXPx5MEeqPv2s79L9i7BJUlG/+RurQ=
google.golang.org/protobuf v1.27.1/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc=
google.golang.org/protobuf v1.28.1 h1:d0NfwRgPtno5B1Wa6L2DAG+KivqkdutMf1UhdNx175w=
google.golang.org/protobuf v1.28.1/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v2 v2.2.3/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15 h1:YR8cESwS4TdDjEe65xsg0ogRM/Nc3DYOhEAlW+xobZo=
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/errgo.v2 v2.1.0/go.mod h1:hNsd1EY+bozCKY1Ytp96fpM3vjJbqLJn88ws8XvfDNI=
gopkg.in/fsnotify.v1 v1.4.7/go.mod h1:Tz8NjZHkW78fSQdbUxIjBTcgA1z1m8ZHf0WmKUhAMys=
gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7/go.mod h1:dt/ZhP58zS4L8KSrWDmTeBkI65Dw0HsyUHuEVlX15mw=
gopkg.in/yaml.v2 v2.2.1/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b h1:h8qDotaEPuJATrMmW04NCwg7v22aHH28wwpauUhK9Oo=
gopkg.in/yaml.v3 v3.0.0-20200605160147-a5ece683394c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
honnef.co/go/tools v0.0.0-20190106161140-3f1c8253044a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
honnef.co/go/tools v0.0.0-20190418001031-e561f6794a2a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
honnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
honnef.co/go/tools v0.0.1-2019.2.3/go.mod h1:a3bituU0lyd329TUQxRnasdCoJDkEUEAqEt0JzvZhAg=
honnef.co/go/tools v0.0.1-2020.1.3/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k=
honnef.co/go/tools v0.0.1-2020.1.4/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k=
lukechampine.com/blake3 v1.1.6 h1:H3cROdztr7RCfoaTpGZFQsrqvweFLrqS73j7L7cmR5c=
lukechampine.com/blake3 v1.1.6/go.mod h1:tkKEOtDkNtklkXtLNEOGNq5tcV90tJiA1vAA12R78LA=
rsc.io/binaryregexp v0.2.0/go.mod h1:qTv7/COck+e2FymRvadv62gMdZztPaShugOCi3I+8D8=
rsc.io/quote/v3 v3.1.0/go.mod h1:yEA65RcK8LyAZtP9Kv3t0HmxON59tX3rD+tICJqUlj0=
rsc.io/sampler v1.3.0/go.mod h1:T1hPZKmBbMNahiBKFy5HrXp6adAjACjK9JXDnKaTXpA=

15
internal/bork/error.go Normal file
View File

@@ -0,0 +1,15 @@
package bork
import (
"errors"
"fmt"
)
var (
ErrMissingMagic = errors.New("missing magic bytes in file")
ErrFileTruncated = errors.New("file/stream is truncated abnormally")
)
func Newf(format string, args ...interface{}) error {
return fmt.Errorf(format, args...)
}

View File

@@ -0,0 +1,11 @@
package bork
import (
"testing"
"github.com/stretchr/testify/assert"
)
func TestBuild(t *testing.T) {
assert.NotNil(t, ErrMissingMagic)
}

188
internal/cli/check.go Normal file
View File

@@ -0,0 +1,188 @@
package cli
import (
"encoding/hex"
"fmt"
"path/filepath"
"strings"
"time"
"github.com/dustin/go-humanize"
"github.com/spf13/afero"
"github.com/urfave/cli/v2"
"sneak.berlin/go/mfer/internal/log"
"sneak.berlin/go/mfer/mfer"
)
// findManifest looks for a manifest file in the given directory.
// It checks for index.mf and .index.mf, returning the first one found.
func findManifest(fs afero.Fs, dir string) (string, error) {
candidates := []string{"index.mf", ".index.mf"}
for _, name := range candidates {
path := filepath.Join(dir, name)
exists, err := afero.Exists(fs, path)
if err != nil {
return "", err
}
if exists {
return path, nil
}
}
return "", fmt.Errorf("no manifest found in %s (looked for index.mf and .index.mf)", dir)
}
func (mfa *CLIApp) checkManifestOperation(ctx *cli.Context) error {
log.Debug("checkManifestOperation()")
var manifestPath string
var err error
if ctx.Args().Len() > 0 {
arg := ctx.Args().Get(0)
// Check if arg is a directory or a file
info, statErr := mfa.Fs.Stat(arg)
if statErr == nil && info.IsDir() {
// It's a directory, look for manifest inside
manifestPath, err = findManifest(mfa.Fs, arg)
if err != nil {
return err
}
} else {
// Treat as a file path
manifestPath = arg
}
} else {
// No argument, look in current directory
manifestPath, err = findManifest(mfa.Fs, ".")
if err != nil {
return err
}
}
basePath := ctx.String("base")
showProgress := ctx.Bool("progress")
log.Infof("checking manifest %s with base %s", manifestPath, basePath)
// Create checker
chk, err := mfer.NewChecker(manifestPath, basePath, mfa.Fs)
if err != nil {
return fmt.Errorf("failed to load manifest: %w", err)
}
// Check signature requirement
requiredSigner := ctx.String("require-signature")
if requiredSigner != "" {
// Validate fingerprint format: must be exactly 40 hex characters
if len(requiredSigner) != 40 {
return fmt.Errorf("invalid fingerprint: must be exactly 40 hex characters, got %d", len(requiredSigner))
}
if _, err := hex.DecodeString(requiredSigner); err != nil {
return fmt.Errorf("invalid fingerprint: must be valid hex: %w", err)
}
if !chk.IsSigned() {
return fmt.Errorf("manifest is not signed, but signature from %s is required", requiredSigner)
}
// Extract fingerprint from the embedded public key (not from the signer field)
// This validates the key is importable and gets its actual fingerprint
embeddedFP, err := chk.ExtractEmbeddedSigningKeyFP()
if err != nil {
return fmt.Errorf("failed to extract fingerprint from embedded signing key: %w", err)
}
// Compare fingerprints - must be exact match (case-insensitive)
if !strings.EqualFold(embeddedFP, requiredSigner) {
return fmt.Errorf("embedded signing key fingerprint %s does not match required %s", embeddedFP, requiredSigner)
}
log.Infof("manifest signature verified (signer: %s)", embeddedFP)
}
log.Infof("manifest contains %d files, %s", chk.FileCount(), humanize.IBytes(uint64(chk.TotalBytes())))
// Set up results channel
results := make(chan mfer.Result, 1)
// Set up progress channel
var progress chan mfer.CheckStatus
if showProgress {
progress = make(chan mfer.CheckStatus, 1)
go func() {
for status := range progress {
if status.ETA > 0 {
log.Progressf("Checking: %d/%d files, %s/s, ETA %s, %d failures",
status.CheckedFiles,
status.TotalFiles,
humanize.IBytes(uint64(status.BytesPerSec)),
status.ETA.Round(time.Second),
status.Failures)
} else {
log.Progressf("Checking: %d/%d files, %s/s, %d failures",
status.CheckedFiles,
status.TotalFiles,
humanize.IBytes(uint64(status.BytesPerSec)),
status.Failures)
}
}
log.ProgressDone()
}()
}
// Process results in a goroutine
var failures int64
done := make(chan struct{})
go func() {
for result := range results {
if result.Status != mfer.StatusOK {
failures++
log.Infof("%s: %s (%s)", result.Status, result.Path, result.Message)
} else {
log.Verbosef("%s: %s", result.Status, result.Path)
}
}
close(done)
}()
// Run check
err = chk.Check(ctx.Context, results, progress)
if err != nil {
return fmt.Errorf("check failed: %w", err)
}
// Wait for results processing to complete
<-done
// Check for extra files if requested
if ctx.Bool("no-extra-files") {
extraResults := make(chan mfer.Result, 1)
extraDone := make(chan struct{})
go func() {
for result := range extraResults {
failures++
log.Infof("%s: %s (%s)", result.Status, result.Path, result.Message)
}
close(extraDone)
}()
err = chk.FindExtraFiles(ctx.Context, extraResults)
if err != nil {
return fmt.Errorf("failed to check for extra files: %w", err)
}
<-extraDone
}
elapsed := time.Since(mfa.startupTime).Seconds()
rate := float64(chk.TotalBytes()) / elapsed
if failures == 0 {
log.Infof("checked %d files (%s) in %.1fs (%s/s): all OK", chk.FileCount(), humanize.IBytes(uint64(chk.TotalBytes())), elapsed, humanize.IBytes(uint64(rate)))
} else {
log.Infof("checked %d files (%s) in %.1fs (%s/s): %d failed", chk.FileCount(), humanize.IBytes(uint64(chk.TotalBytes())), elapsed, humanize.IBytes(uint64(rate)), failures)
}
if failures > 0 {
mfa.exitCode = 1
}
return nil
}

68
internal/cli/entry.go Normal file
View File

@@ -0,0 +1,68 @@
package cli
import (
"io"
"os"
"github.com/spf13/afero"
)
// NO_COLOR disables colored output when set. Automatically true if the
// NO_COLOR environment variable is present (per https://no-color.org/).
var NO_COLOR bool
func init() {
NO_COLOR = false
if _, exists := os.LookupEnv("NO_COLOR"); exists {
NO_COLOR = true
}
}
// RunOptions contains all configuration for running the CLI application.
// Use DefaultRunOptions for standard CLI execution, or construct manually for testing.
type RunOptions struct {
Appname string // Application name displayed in help and version output
Version string // Version string (typically set at build time)
Gitrev string // Git revision hash (typically set at build time)
Args []string // Command-line arguments (typically os.Args)
Stdin io.Reader // Standard input stream
Stdout io.Writer // Standard output stream
Stderr io.Writer // Standard error stream
Fs afero.Fs // Filesystem abstraction for file operations
}
// DefaultRunOptions returns RunOptions configured for normal CLI execution.
func DefaultRunOptions(appname, version, gitrev string) *RunOptions {
return &RunOptions{
Appname: appname,
Version: version,
Gitrev: gitrev,
Args: os.Args,
Stdin: os.Stdin,
Stdout: os.Stdout,
Stderr: os.Stderr,
Fs: afero.NewOsFs(),
}
}
// Run creates and runs the CLI application with default options.
func Run(appname, version, gitrev string) int {
return RunWithOptions(DefaultRunOptions(appname, version, gitrev))
}
// RunWithOptions creates and runs the CLI application with the given options.
func RunWithOptions(opts *RunOptions) int {
m := &CLIApp{
appname: opts.Appname,
version: opts.Version,
gitrev: opts.Gitrev,
exitCode: 0,
Stdin: opts.Stdin,
Stdout: opts.Stdout,
Stderr: opts.Stderr,
Fs: opts.Fs,
}
m.run(opts.Args)
return m.exitCode
}

593
internal/cli/entry_test.go Normal file
View File

@@ -0,0 +1,593 @@
package cli
import (
"bytes"
"fmt"
"math/rand"
"testing"
"github.com/spf13/afero"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
urfcli "github.com/urfave/cli/v2"
"sneak.berlin/go/mfer/mfer"
)
func init() {
// Prevent urfave/cli from calling os.Exit during tests
urfcli.OsExiter = func(code int) {}
}
func TestBuild(t *testing.T) {
m := &CLIApp{}
assert.NotNil(t, m)
}
func testOpts(args []string, fs afero.Fs) *RunOptions {
return &RunOptions{
Appname: "mfer",
Version: "1.0.0",
Gitrev: "abc123",
Args: args,
Stdin: &bytes.Buffer{},
Stdout: &bytes.Buffer{},
Stderr: &bytes.Buffer{},
Fs: fs,
}
}
func TestVersionCommand(t *testing.T) {
fs := afero.NewMemMapFs()
opts := testOpts([]string{"mfer", "version"}, fs)
exitCode := RunWithOptions(opts)
assert.Equal(t, 0, exitCode)
stdout := opts.Stdout.(*bytes.Buffer).String()
assert.Contains(t, stdout, mfer.Version)
assert.Contains(t, stdout, "abc123")
}
func TestHelpCommand(t *testing.T) {
fs := afero.NewMemMapFs()
opts := testOpts([]string{"mfer", "--help"}, fs)
exitCode := RunWithOptions(opts)
assert.Equal(t, 0, exitCode)
stdout := opts.Stdout.(*bytes.Buffer).String()
assert.Contains(t, stdout, "generate")
assert.Contains(t, stdout, "check")
assert.Contains(t, stdout, "fetch")
}
func TestGenerateCommand(t *testing.T) {
fs := afero.NewMemMapFs()
// Create test files in memory filesystem
require.NoError(t, fs.MkdirAll("/testdir", 0o755))
require.NoError(t, afero.WriteFile(fs, "/testdir/file1.txt", []byte("hello world"), 0o644))
require.NoError(t, afero.WriteFile(fs, "/testdir/file2.txt", []byte("test content"), 0o644))
opts := testOpts([]string{"mfer", "generate", "-q", "-o", "/testdir/test.mf", "/testdir"}, fs)
exitCode := RunWithOptions(opts)
assert.Equal(t, 0, exitCode, "stderr: %s", opts.Stderr.(*bytes.Buffer).String())
// Verify manifest was created
exists, err := afero.Exists(fs, "/testdir/test.mf")
require.NoError(t, err)
assert.True(t, exists)
}
func TestGenerateAndCheckCommand(t *testing.T) {
fs := afero.NewMemMapFs()
// Create test files with subdirectory
require.NoError(t, fs.MkdirAll("/testdir/subdir", 0o755))
require.NoError(t, afero.WriteFile(fs, "/testdir/file1.txt", []byte("hello world"), 0o644))
require.NoError(t, afero.WriteFile(fs, "/testdir/subdir/file2.txt", []byte("test content"), 0o644))
// Generate manifest
opts := testOpts([]string{"mfer", "generate", "-q", "-o", "/testdir/test.mf", "/testdir"}, fs)
exitCode := RunWithOptions(opts)
require.Equal(t, 0, exitCode, "generate failed: %s", opts.Stderr.(*bytes.Buffer).String())
// Check manifest
opts = testOpts([]string{"mfer", "check", "-q", "--base", "/testdir", "/testdir/test.mf"}, fs)
exitCode = RunWithOptions(opts)
assert.Equal(t, 0, exitCode, "check failed: %s", opts.Stderr.(*bytes.Buffer).String())
}
func TestCheckCommandWithMissingFile(t *testing.T) {
fs := afero.NewMemMapFs()
// Create test file
require.NoError(t, fs.MkdirAll("/testdir", 0o755))
require.NoError(t, afero.WriteFile(fs, "/testdir/file1.txt", []byte("hello world"), 0o644))
// Generate manifest
opts := testOpts([]string{"mfer", "generate", "-q", "-o", "/testdir/test.mf", "/testdir"}, fs)
exitCode := RunWithOptions(opts)
require.Equal(t, 0, exitCode, "generate failed: %s", opts.Stderr.(*bytes.Buffer).String())
// Delete the file
require.NoError(t, fs.Remove("/testdir/file1.txt"))
// Check manifest - should fail
opts = testOpts([]string{"mfer", "check", "-q", "--base", "/testdir", "/testdir/test.mf"}, fs)
exitCode = RunWithOptions(opts)
assert.Equal(t, 1, exitCode, "check should have failed for missing file")
}
func TestCheckCommandWithCorruptedFile(t *testing.T) {
fs := afero.NewMemMapFs()
// Create test file
require.NoError(t, fs.MkdirAll("/testdir", 0o755))
require.NoError(t, afero.WriteFile(fs, "/testdir/file1.txt", []byte("hello world"), 0o644))
// Generate manifest
opts := testOpts([]string{"mfer", "generate", "-q", "-o", "/testdir/test.mf", "/testdir"}, fs)
exitCode := RunWithOptions(opts)
require.Equal(t, 0, exitCode, "generate failed: %s", opts.Stderr.(*bytes.Buffer).String())
// Corrupt the file (change content but keep same size)
require.NoError(t, afero.WriteFile(fs, "/testdir/file1.txt", []byte("HELLO WORLD"), 0o644))
// Check manifest - should fail with hash mismatch
opts = testOpts([]string{"mfer", "check", "-q", "--base", "/testdir", "/testdir/test.mf"}, fs)
exitCode = RunWithOptions(opts)
assert.Equal(t, 1, exitCode, "check should have failed for corrupted file")
}
func TestCheckCommandWithSizeMismatch(t *testing.T) {
fs := afero.NewMemMapFs()
// Create test file
require.NoError(t, fs.MkdirAll("/testdir", 0o755))
require.NoError(t, afero.WriteFile(fs, "/testdir/file1.txt", []byte("hello world"), 0o644))
// Generate manifest
opts := testOpts([]string{"mfer", "generate", "-q", "-o", "/testdir/test.mf", "/testdir"}, fs)
exitCode := RunWithOptions(opts)
require.Equal(t, 0, exitCode, "generate failed: %s", opts.Stderr.(*bytes.Buffer).String())
// Change file size
require.NoError(t, afero.WriteFile(fs, "/testdir/file1.txt", []byte("different size content here"), 0o644))
// Check manifest - should fail with size mismatch
opts = testOpts([]string{"mfer", "check", "-q", "--base", "/testdir", "/testdir/test.mf"}, fs)
exitCode = RunWithOptions(opts)
assert.Equal(t, 1, exitCode, "check should have failed for size mismatch")
}
func TestBannerOutput(t *testing.T) {
fs := afero.NewMemMapFs()
// Create test file
require.NoError(t, fs.MkdirAll("/testdir", 0o755))
require.NoError(t, afero.WriteFile(fs, "/testdir/file1.txt", []byte("hello"), 0o644))
// Run without -q to see banner
opts := testOpts([]string{"mfer", "generate", "-o", "/testdir/test.mf", "/testdir"}, fs)
exitCode := RunWithOptions(opts)
assert.Equal(t, 0, exitCode)
// Banner ASCII art should be in stdout
stdout := opts.Stdout.(*bytes.Buffer).String()
assert.Contains(t, stdout, "___")
assert.Contains(t, stdout, "\\")
}
func TestUnknownCommand(t *testing.T) {
fs := afero.NewMemMapFs()
opts := testOpts([]string{"mfer", "unknown"}, fs)
exitCode := RunWithOptions(opts)
assert.Equal(t, 1, exitCode)
}
func TestGenerateExcludesDotfilesByDefault(t *testing.T) {
fs := afero.NewMemMapFs()
// Create test files including dotfiles
require.NoError(t, fs.MkdirAll("/testdir", 0o755))
require.NoError(t, afero.WriteFile(fs, "/testdir/file1.txt", []byte("hello"), 0o644))
require.NoError(t, afero.WriteFile(fs, "/testdir/.hidden", []byte("secret"), 0o644))
// Generate manifest without --include-dotfiles (default excludes dotfiles)
opts := testOpts([]string{"mfer", "generate", "-q", "-o", "/testdir/test.mf", "/testdir"}, fs)
exitCode := RunWithOptions(opts)
require.Equal(t, 0, exitCode)
// Check that manifest exists
exists, _ := afero.Exists(fs, "/testdir/test.mf")
assert.True(t, exists)
// Verify manifest only has 1 file (the non-dotfile)
manifest, err := mfer.NewManifestFromFile(fs, "/testdir/test.mf")
require.NoError(t, err)
assert.Len(t, manifest.Files(), 1)
assert.Equal(t, "file1.txt", manifest.Files()[0].Path)
}
func TestGenerateWithIncludeDotfiles(t *testing.T) {
fs := afero.NewMemMapFs()
// Create test files including dotfiles
require.NoError(t, fs.MkdirAll("/testdir", 0o755))
require.NoError(t, afero.WriteFile(fs, "/testdir/file1.txt", []byte("hello"), 0o644))
require.NoError(t, afero.WriteFile(fs, "/testdir/.hidden", []byte("secret"), 0o644))
// Generate manifest with --include-dotfiles
opts := testOpts([]string{"mfer", "generate", "-q", "--include-dotfiles", "-o", "/testdir/test.mf", "/testdir"}, fs)
exitCode := RunWithOptions(opts)
require.Equal(t, 0, exitCode)
// Verify manifest has 2 files (including dotfile)
manifest, err := mfer.NewManifestFromFile(fs, "/testdir/test.mf")
require.NoError(t, err)
assert.Len(t, manifest.Files(), 2)
}
func TestMultipleInputPaths(t *testing.T) {
fs := afero.NewMemMapFs()
// Create test files in multiple directories
require.NoError(t, fs.MkdirAll("/dir1", 0o755))
require.NoError(t, fs.MkdirAll("/dir2", 0o755))
require.NoError(t, afero.WriteFile(fs, "/dir1/file1.txt", []byte("content1"), 0o644))
require.NoError(t, afero.WriteFile(fs, "/dir2/file2.txt", []byte("content2"), 0o644))
// Generate manifest from multiple paths
opts := testOpts([]string{"mfer", "generate", "-q", "-o", "/output.mf", "/dir1", "/dir2"}, fs)
exitCode := RunWithOptions(opts)
assert.Equal(t, 0, exitCode, "stderr: %s", opts.Stderr.(*bytes.Buffer).String())
exists, _ := afero.Exists(fs, "/output.mf")
assert.True(t, exists)
}
func TestNoExtraFilesPass(t *testing.T) {
fs := afero.NewMemMapFs()
// Create test files
require.NoError(t, fs.MkdirAll("/testdir", 0o755))
require.NoError(t, afero.WriteFile(fs, "/testdir/file1.txt", []byte("hello"), 0o644))
require.NoError(t, afero.WriteFile(fs, "/testdir/file2.txt", []byte("world"), 0o644))
// Generate manifest
opts := testOpts([]string{"mfer", "generate", "-q", "-o", "/manifest.mf", "/testdir"}, fs)
exitCode := RunWithOptions(opts)
require.Equal(t, 0, exitCode)
// Check with --no-extra-files (should pass - no extra files)
opts = testOpts([]string{"mfer", "check", "-q", "--no-extra-files", "--base", "/testdir", "/manifest.mf"}, fs)
exitCode = RunWithOptions(opts)
assert.Equal(t, 0, exitCode)
}
func TestNoExtraFilesFail(t *testing.T) {
fs := afero.NewMemMapFs()
// Create test files
require.NoError(t, fs.MkdirAll("/testdir", 0o755))
require.NoError(t, afero.WriteFile(fs, "/testdir/file1.txt", []byte("hello"), 0o644))
// Generate manifest
opts := testOpts([]string{"mfer", "generate", "-q", "-o", "/manifest.mf", "/testdir"}, fs)
exitCode := RunWithOptions(opts)
require.Equal(t, 0, exitCode)
// Add an extra file after manifest generation
require.NoError(t, afero.WriteFile(fs, "/testdir/extra.txt", []byte("extra"), 0o644))
// Check with --no-extra-files (should fail - extra file exists)
opts = testOpts([]string{"mfer", "check", "-q", "--no-extra-files", "--base", "/testdir", "/manifest.mf"}, fs)
exitCode = RunWithOptions(opts)
assert.Equal(t, 1, exitCode, "check should fail when extra files exist")
}
func TestNoExtraFilesWithSubdirectory(t *testing.T) {
fs := afero.NewMemMapFs()
// Create test files with subdirectory
require.NoError(t, fs.MkdirAll("/testdir/subdir", 0o755))
require.NoError(t, afero.WriteFile(fs, "/testdir/file1.txt", []byte("hello"), 0o644))
require.NoError(t, afero.WriteFile(fs, "/testdir/subdir/file2.txt", []byte("world"), 0o644))
// Generate manifest
opts := testOpts([]string{"mfer", "generate", "-q", "-o", "/manifest.mf", "/testdir"}, fs)
exitCode := RunWithOptions(opts)
require.Equal(t, 0, exitCode)
// Add extra file in subdirectory
require.NoError(t, afero.WriteFile(fs, "/testdir/subdir/extra.txt", []byte("extra"), 0o644))
// Check with --no-extra-files (should fail)
opts = testOpts([]string{"mfer", "check", "-q", "--no-extra-files", "--base", "/testdir", "/manifest.mf"}, fs)
exitCode = RunWithOptions(opts)
assert.Equal(t, 1, exitCode, "check should fail when extra files exist in subdirectory")
}
func TestCheckWithoutNoExtraFilesIgnoresExtra(t *testing.T) {
fs := afero.NewMemMapFs()
// Create test file
require.NoError(t, fs.MkdirAll("/testdir", 0o755))
require.NoError(t, afero.WriteFile(fs, "/testdir/file1.txt", []byte("hello"), 0o644))
// Generate manifest
opts := testOpts([]string{"mfer", "generate", "-q", "-o", "/manifest.mf", "/testdir"}, fs)
exitCode := RunWithOptions(opts)
require.Equal(t, 0, exitCode)
// Add extra file
require.NoError(t, afero.WriteFile(fs, "/testdir/extra.txt", []byte("extra"), 0o644))
// Check WITHOUT --no-extra-files (should pass - extra files ignored)
opts = testOpts([]string{"mfer", "check", "-q", "--base", "/testdir", "/manifest.mf"}, fs)
exitCode = RunWithOptions(opts)
assert.Equal(t, 0, exitCode, "check without --no-extra-files should ignore extra files")
}
func TestGenerateAtomicWriteNoTempFileOnSuccess(t *testing.T) {
fs := afero.NewMemMapFs()
// Create test file
require.NoError(t, fs.MkdirAll("/testdir", 0o755))
require.NoError(t, afero.WriteFile(fs, "/testdir/file1.txt", []byte("hello"), 0o644))
// Generate manifest
opts := testOpts([]string{"mfer", "generate", "-q", "-o", "/output.mf", "/testdir"}, fs)
exitCode := RunWithOptions(opts)
require.Equal(t, 0, exitCode)
// Verify output file exists
exists, err := afero.Exists(fs, "/output.mf")
require.NoError(t, err)
assert.True(t, exists, "output file should exist")
// Verify temp file does NOT exist
tmpExists, err := afero.Exists(fs, "/output.mf.tmp")
require.NoError(t, err)
assert.False(t, tmpExists, "temp file should not exist after successful generation")
}
func TestGenerateAtomicWriteOverwriteWithForce(t *testing.T) {
fs := afero.NewMemMapFs()
// Create test file
require.NoError(t, fs.MkdirAll("/testdir", 0o755))
require.NoError(t, afero.WriteFile(fs, "/testdir/file1.txt", []byte("hello"), 0o644))
// Create existing manifest with different content
require.NoError(t, afero.WriteFile(fs, "/output.mf", []byte("old content"), 0o644))
// Generate manifest with --force
opts := testOpts([]string{"mfer", "generate", "-q", "-f", "-o", "/output.mf", "/testdir"}, fs)
exitCode := RunWithOptions(opts)
require.Equal(t, 0, exitCode)
// Verify output file exists and was overwritten
content, err := afero.ReadFile(fs, "/output.mf")
require.NoError(t, err)
assert.NotEqual(t, "old content", string(content), "manifest should be overwritten")
// Verify temp file does NOT exist
tmpExists, err := afero.Exists(fs, "/output.mf.tmp")
require.NoError(t, err)
assert.False(t, tmpExists, "temp file should not exist after successful generation")
}
func TestGenerateFailsWithoutForceWhenOutputExists(t *testing.T) {
fs := afero.NewMemMapFs()
// Create test file
require.NoError(t, fs.MkdirAll("/testdir", 0o755))
require.NoError(t, afero.WriteFile(fs, "/testdir/file1.txt", []byte("hello"), 0o644))
// Create existing manifest
require.NoError(t, afero.WriteFile(fs, "/output.mf", []byte("existing"), 0o644))
// Generate manifest WITHOUT --force (should fail)
opts := testOpts([]string{"mfer", "generate", "-q", "-o", "/output.mf", "/testdir"}, fs)
exitCode := RunWithOptions(opts)
assert.Equal(t, 1, exitCode, "should fail when output exists without --force")
// Verify original content is preserved
content, err := afero.ReadFile(fs, "/output.mf")
require.NoError(t, err)
assert.Equal(t, "existing", string(content), "original file should be preserved")
}
func TestGenerateAtomicWriteUsesTemp(t *testing.T) {
// This test verifies that generate uses a temp file by checking
// that the output file doesn't exist until generation completes.
// We do this by generating to a path and verifying the temp file
// pattern is used (output.mf.tmp -> output.mf)
fs := afero.NewMemMapFs()
// Create test file
require.NoError(t, fs.MkdirAll("/testdir", 0o755))
require.NoError(t, afero.WriteFile(fs, "/testdir/file1.txt", []byte("hello"), 0o644))
// Generate manifest
opts := testOpts([]string{"mfer", "generate", "-q", "-o", "/output.mf", "/testdir"}, fs)
exitCode := RunWithOptions(opts)
require.Equal(t, 0, exitCode)
// Both output file should exist and temp should not
exists, _ := afero.Exists(fs, "/output.mf")
assert.True(t, exists, "output file should exist")
tmpExists, _ := afero.Exists(fs, "/output.mf.tmp")
assert.False(t, tmpExists, "temp file should be cleaned up")
// Verify manifest is valid (not empty)
content, err := afero.ReadFile(fs, "/output.mf")
require.NoError(t, err)
assert.True(t, len(content) > 0, "manifest should not be empty")
}
// failingWriterFs wraps a filesystem and makes writes fail after N bytes
type failingWriterFs struct {
afero.Fs
failAfter int64
written int64
}
type failingFile struct {
afero.File
fs *failingWriterFs
}
func (f *failingFile) Write(p []byte) (int, error) {
f.fs.written += int64(len(p))
if f.fs.written > f.fs.failAfter {
return 0, fmt.Errorf("simulated write failure")
}
return f.File.Write(p)
}
func (fs *failingWriterFs) Create(name string) (afero.File, error) {
f, err := fs.Fs.Create(name)
if err != nil {
return nil, err
}
return &failingFile{File: f, fs: fs}, nil
}
func TestGenerateAtomicWriteCleansUpOnError(t *testing.T) {
baseFs := afero.NewMemMapFs()
// Create test files - need enough content to trigger the write failure
require.NoError(t, baseFs.MkdirAll("/testdir", 0o755))
require.NoError(t, afero.WriteFile(baseFs, "/testdir/file1.txt", []byte("hello world this is a test file"), 0o644))
// Wrap with failing writer that fails after writing some bytes
fs := &failingWriterFs{Fs: baseFs, failAfter: 10}
// Generate manifest - should fail during write
opts := testOpts([]string{"mfer", "generate", "-q", "-o", "/output.mf", "/testdir"}, fs)
exitCode := RunWithOptions(opts)
assert.Equal(t, 1, exitCode, "should fail due to write error")
// With atomic writes: output.mf should NOT exist (temp was cleaned up)
// With non-atomic writes: output.mf WOULD exist (partial/empty)
exists, _ := afero.Exists(baseFs, "/output.mf")
assert.False(t, exists, "output file should not exist after failed generation (atomic write)")
// Temp file should also not exist
tmpExists, _ := afero.Exists(baseFs, "/output.mf.tmp")
assert.False(t, tmpExists, "temp file should be cleaned up after failed generation")
}
func TestGenerateValidatesInputPaths(t *testing.T) {
fs := afero.NewMemMapFs()
// Create one valid directory
require.NoError(t, fs.MkdirAll("/validdir", 0o755))
require.NoError(t, afero.WriteFile(fs, "/validdir/file.txt", []byte("content"), 0o644))
t.Run("nonexistent path fails fast", func(t *testing.T) {
opts := testOpts([]string{"mfer", "generate", "-q", "-o", "/output.mf", "/nonexistent"}, fs)
exitCode := RunWithOptions(opts)
assert.Equal(t, 1, exitCode)
stderr := opts.Stderr.(*bytes.Buffer).String()
assert.Contains(t, stderr, "path does not exist")
assert.Contains(t, stderr, "/nonexistent")
})
t.Run("mix of valid and invalid paths fails fast", func(t *testing.T) {
opts := testOpts([]string{"mfer", "generate", "-q", "-o", "/output.mf", "/validdir", "/alsononexistent"}, fs)
exitCode := RunWithOptions(opts)
assert.Equal(t, 1, exitCode)
stderr := opts.Stderr.(*bytes.Buffer).String()
assert.Contains(t, stderr, "path does not exist")
assert.Contains(t, stderr, "/alsononexistent")
// Output file should not have been created
exists, _ := afero.Exists(fs, "/output.mf")
assert.False(t, exists, "output file should not exist when path validation fails")
})
t.Run("valid paths succeed", func(t *testing.T) {
opts := testOpts([]string{"mfer", "generate", "-q", "-o", "/output.mf", "/validdir"}, fs)
exitCode := RunWithOptions(opts)
assert.Equal(t, 0, exitCode)
})
}
func TestCheckDetectsManifestCorruption(t *testing.T) {
fs := afero.NewMemMapFs()
rng := rand.New(rand.NewSource(42))
// Create many small files with random names to generate a ~1MB manifest
// Each manifest entry is roughly 50-60 bytes, so we need ~20000 files
require.NoError(t, fs.MkdirAll("/testdir", 0o755))
numFiles := 20000
for i := 0; i < numFiles; i++ {
// Generate random filename
filename := fmt.Sprintf("/testdir/%08x%08x%08x.dat", rng.Uint32(), rng.Uint32(), rng.Uint32())
// Small random content
content := make([]byte, 16+rng.Intn(48))
rng.Read(content)
require.NoError(t, afero.WriteFile(fs, filename, content, 0o644))
}
// Generate manifest outside of testdir
opts := testOpts([]string{"mfer", "generate", "-q", "-o", "/manifest.mf", "/testdir"}, fs)
exitCode := RunWithOptions(opts)
require.Equal(t, 0, exitCode, "generate should succeed")
// Read the valid manifest and verify it's approximately 1MB
validManifest, err := afero.ReadFile(fs, "/manifest.mf")
require.NoError(t, err)
require.True(t, len(validManifest) >= 1024*1024, "manifest should be at least 1MB, got %d bytes", len(validManifest))
t.Logf("manifest size: %d bytes (%d files)", len(validManifest), numFiles)
// First corruption: truncate the manifest
require.NoError(t, afero.WriteFile(fs, "/manifest.mf", validManifest[:len(validManifest)/2], 0o644))
// Check should fail with truncated manifest
opts = testOpts([]string{"mfer", "check", "-q", "--base", "/testdir", "/manifest.mf"}, fs)
exitCode = RunWithOptions(opts)
assert.Equal(t, 1, exitCode, "check should fail with truncated manifest")
// Verify check passes with valid manifest
require.NoError(t, afero.WriteFile(fs, "/manifest.mf", validManifest, 0o644))
opts = testOpts([]string{"mfer", "check", "-q", "--base", "/testdir", "/manifest.mf"}, fs)
exitCode = RunWithOptions(opts)
require.Equal(t, 0, exitCode, "check should pass with valid manifest")
// Now do 500 random corruption iterations
for i := 0; i < 500; i++ {
// Corrupt: write a random byte at a random offset
corrupted := make([]byte, len(validManifest))
copy(corrupted, validManifest)
offset := rng.Intn(len(corrupted))
originalByte := corrupted[offset]
// Make sure we actually change the byte
newByte := byte(rng.Intn(256))
for newByte == originalByte {
newByte = byte(rng.Intn(256))
}
corrupted[offset] = newByte
require.NoError(t, afero.WriteFile(fs, "/manifest.mf", corrupted, 0o644))
// Check should fail with corrupted manifest
opts = testOpts([]string{"mfer", "check", "-q", "--base", "/testdir", "/manifest.mf"}, fs)
exitCode = RunWithOptions(opts)
assert.Equal(t, 1, exitCode, "iteration %d: check should fail with corrupted manifest (offset %d, 0x%02x -> 0x%02x)",
i, offset, originalByte, newByte)
// Restore valid manifest for next iteration
require.NoError(t, afero.WriteFile(fs, "/manifest.mf", validManifest, 0o644))
}
}

374
internal/cli/fetch.go Normal file
View File

@@ -0,0 +1,374 @@
package cli
import (
"bytes"
"crypto/sha256"
"fmt"
"io"
"net/http"
"net/url"
"os"
"path"
"path/filepath"
"strings"
"time"
"github.com/dustin/go-humanize"
"github.com/multiformats/go-multihash"
"github.com/urfave/cli/v2"
"sneak.berlin/go/mfer/internal/log"
"sneak.berlin/go/mfer/mfer"
)
// DownloadProgress reports the progress of a single file download.
type DownloadProgress struct {
Path string // File path being downloaded
BytesRead int64 // Bytes downloaded so far
TotalBytes int64 // Total expected bytes (-1 if unknown)
BytesPerSec float64 // Current download rate
ETA time.Duration // Estimated time to completion
}
func (mfa *CLIApp) fetchManifestOperation(ctx *cli.Context) error {
log.Debug("fetchManifestOperation()")
if ctx.Args().Len() == 0 {
return fmt.Errorf("URL argument required")
}
inputURL := ctx.Args().Get(0)
manifestURL, err := resolveManifestURL(inputURL)
if err != nil {
return fmt.Errorf("invalid URL: %w", err)
}
log.Infof("fetching manifest from %s", manifestURL)
// Fetch manifest
resp, err := http.Get(manifestURL)
if err != nil {
return fmt.Errorf("failed to fetch manifest: %w", err)
}
defer func() { _ = resp.Body.Close() }()
if resp.StatusCode != http.StatusOK {
return fmt.Errorf("failed to fetch manifest: HTTP %d", resp.StatusCode)
}
// Parse manifest
manifest, err := mfer.NewManifestFromReader(resp.Body)
if err != nil {
return fmt.Errorf("failed to parse manifest: %w", err)
}
files := manifest.Files()
log.Infof("manifest contains %d files", len(files))
// Compute base URL (directory containing manifest)
baseURL, err := url.Parse(manifestURL)
if err != nil {
return err
}
baseURL.Path = path.Dir(baseURL.Path)
if !strings.HasSuffix(baseURL.Path, "/") {
baseURL.Path += "/"
}
// Calculate total bytes to download
var totalBytes int64
for _, f := range files {
totalBytes += f.Size
}
// Create progress channel
progress := make(chan DownloadProgress, 10)
// Start progress reporter goroutine
done := make(chan struct{})
go func() {
defer close(done)
for p := range progress {
rate := formatBitrate(p.BytesPerSec * 8)
if p.ETA > 0 {
log.Infof("%s: %s/%s, %s, ETA %s",
p.Path, humanize.IBytes(uint64(p.BytesRead)), humanize.IBytes(uint64(p.TotalBytes)),
rate, p.ETA.Round(time.Second))
} else {
log.Infof("%s: %s/%s, %s",
p.Path, humanize.IBytes(uint64(p.BytesRead)), humanize.IBytes(uint64(p.TotalBytes)), rate)
}
}
}()
// Track download start time
startTime := time.Now()
// Download each file
for _, f := range files {
// Sanitize the path to prevent path traversal attacks
localPath, err := sanitizePath(f.Path)
if err != nil {
close(progress)
<-done
return fmt.Errorf("invalid path in manifest: %w", err)
}
fileURL := baseURL.String() + encodeFilePath(f.Path)
log.Infof("fetching %s", f.Path)
if err := downloadFile(fileURL, localPath, f, progress); err != nil {
close(progress)
<-done
return fmt.Errorf("failed to download %s: %w", f.Path, err)
}
}
close(progress)
<-done
// Print summary
elapsed := time.Since(startTime)
avgBytesPerSec := float64(totalBytes) / elapsed.Seconds()
avgRate := formatBitrate(avgBytesPerSec * 8)
log.Infof("downloaded %d files (%s) in %.1fs (%s avg)",
len(files),
humanize.IBytes(uint64(totalBytes)),
elapsed.Seconds(),
avgRate)
return nil
}
// encodeFilePath URL-encodes each segment of a file path while preserving slashes.
func encodeFilePath(p string) string {
segments := strings.Split(p, "/")
for i, seg := range segments {
segments[i] = url.PathEscape(seg)
}
return strings.Join(segments, "/")
}
// sanitizePath validates and sanitizes a file path from the manifest.
// It prevents path traversal attacks and rejects unsafe paths.
func sanitizePath(p string) (string, error) {
// Reject empty paths
if p == "" {
return "", fmt.Errorf("empty path")
}
// Reject absolute paths
if filepath.IsAbs(p) {
return "", fmt.Errorf("absolute path not allowed: %s", p)
}
// Clean the path to resolve . and ..
cleaned := filepath.Clean(p)
// Reject paths that escape the current directory
if strings.HasPrefix(cleaned, ".."+string(filepath.Separator)) || cleaned == ".." {
return "", fmt.Errorf("path traversal not allowed: %s", p)
}
// Also check for absolute paths after cleaning (handles edge cases)
if filepath.IsAbs(cleaned) {
return "", fmt.Errorf("absolute path not allowed: %s", p)
}
return cleaned, nil
}
// resolveManifestURL takes a URL and returns the manifest URL.
// If the URL already ends with .mf, it's returned as-is.
// Otherwise, index.mf is appended.
func resolveManifestURL(inputURL string) (string, error) {
parsed, err := url.Parse(inputURL)
if err != nil {
return "", err
}
// Check if URL already ends with .mf
if strings.HasSuffix(parsed.Path, ".mf") {
return inputURL, nil
}
// Ensure path ends with /
if !strings.HasSuffix(parsed.Path, "/") {
parsed.Path += "/"
}
// Append index.mf
parsed.Path += "index.mf"
return parsed.String(), nil
}
// progressWriter wraps an io.Writer and reports progress to a channel.
type progressWriter struct {
w io.Writer
path string
total int64
written int64
startTime time.Time
progress chan<- DownloadProgress
}
func (pw *progressWriter) Write(p []byte) (int, error) {
n, err := pw.w.Write(p)
pw.written += int64(n)
if pw.progress != nil {
var bytesPerSec float64
var eta time.Duration
elapsed := time.Since(pw.startTime)
if elapsed > 0 && pw.written > 0 {
bytesPerSec = float64(pw.written) / elapsed.Seconds()
if bytesPerSec > 0 && pw.total > 0 {
remainingBytes := pw.total - pw.written
eta = time.Duration(float64(remainingBytes)/bytesPerSec) * time.Second
}
}
sendProgress(pw.progress, DownloadProgress{
Path: pw.path,
BytesRead: pw.written,
TotalBytes: pw.total,
BytesPerSec: bytesPerSec,
ETA: eta,
})
}
return n, err
}
// formatBitrate formats a bits-per-second value with appropriate unit prefix.
func formatBitrate(bps float64) string {
switch {
case bps >= 1e9:
return fmt.Sprintf("%.1f Gbps", bps/1e9)
case bps >= 1e6:
return fmt.Sprintf("%.1f Mbps", bps/1e6)
case bps >= 1e3:
return fmt.Sprintf("%.1f Kbps", bps/1e3)
default:
return fmt.Sprintf("%.0f bps", bps)
}
}
// sendProgress sends a progress update without blocking.
func sendProgress(ch chan<- DownloadProgress, p DownloadProgress) {
select {
case ch <- p:
default:
}
}
// downloadFile downloads a URL to a local file path with hash verification.
// It downloads to a temporary file, verifies the hash, then renames to the final path.
// Progress is reported via the progress channel.
func downloadFile(fileURL, localPath string, entry *mfer.MFFilePath, progress chan<- DownloadProgress) error {
// Create parent directories if needed
dir := filepath.Dir(localPath)
if dir != "" && dir != "." {
if err := os.MkdirAll(dir, 0o755); err != nil {
return err
}
}
// Compute temp file path in the same directory
// For dotfiles, just append .tmp (they're already hidden)
// For regular files, prefix with . and append .tmp
base := filepath.Base(localPath)
var tmpName string
if strings.HasPrefix(base, ".") {
tmpName = base + ".tmp"
} else {
tmpName = "." + base + ".tmp"
}
tmpPath := filepath.Join(dir, tmpName)
if dir == "" || dir == "." {
tmpPath = tmpName
}
// Fetch file
resp, err := http.Get(fileURL)
if err != nil {
return err
}
defer func() { _ = resp.Body.Close() }()
if resp.StatusCode != http.StatusOK {
return fmt.Errorf("HTTP %d", resp.StatusCode)
}
// Determine expected size
expectedSize := entry.Size
totalBytes := resp.ContentLength
if totalBytes < 0 {
totalBytes = expectedSize
}
// Create temp file
out, err := os.Create(tmpPath)
if err != nil {
return err
}
// Set up hash computation
h := sha256.New()
// Create progress-reporting writer that also computes hash
pw := &progressWriter{
w: io.MultiWriter(out, h),
path: localPath,
total: totalBytes,
startTime: time.Now(),
progress: progress,
}
// Copy content while hashing and reporting progress
written, copyErr := io.Copy(pw, resp.Body)
// Close file before checking errors (to flush writes)
closeErr := out.Close()
// If copy failed, clean up temp file and return error
if copyErr != nil {
_ = os.Remove(tmpPath)
return copyErr
}
if closeErr != nil {
_ = os.Remove(tmpPath)
return closeErr
}
// Verify size
if written != expectedSize {
_ = os.Remove(tmpPath)
return fmt.Errorf("size mismatch: expected %d bytes, got %d", expectedSize, written)
}
// Encode computed hash as multihash
computed, err := multihash.Encode(h.Sum(nil), multihash.SHA2_256)
if err != nil {
_ = os.Remove(tmpPath)
return fmt.Errorf("failed to encode hash: %w", err)
}
// Verify hash against manifest (at least one must match)
hashMatch := false
for _, hash := range entry.Hashes {
if bytes.Equal(computed, hash.MultiHash) {
hashMatch = true
break
}
}
if !hashMatch {
_ = os.Remove(tmpPath)
return fmt.Errorf("hash mismatch")
}
// Rename temp file to final path
if err := os.Rename(tmpPath, localPath); err != nil {
_ = os.Remove(tmpPath)
return fmt.Errorf("failed to rename temp file: %w", err)
}
return nil
}

391
internal/cli/fetch_test.go Normal file
View File

@@ -0,0 +1,391 @@
package cli
import (
"bytes"
"context"
"io"
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"testing"
"github.com/spf13/afero"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"sneak.berlin/go/mfer/mfer"
)
func TestEncodeFilePath(t *testing.T) {
tests := []struct {
input string
expected string
}{
{"file.txt", "file.txt"},
{"dir/file.txt", "dir/file.txt"},
{"my file.txt", "my%20file.txt"},
{"dir/my file.txt", "dir/my%20file.txt"},
{"file#1.txt", "file%231.txt"},
{"file?v=1.txt", "file%3Fv=1.txt"},
{"path/to/file with spaces.txt", "path/to/file%20with%20spaces.txt"},
{"100%done.txt", "100%25done.txt"},
}
for _, tt := range tests {
t.Run(tt.input, func(t *testing.T) {
result := encodeFilePath(tt.input)
assert.Equal(t, tt.expected, result)
})
}
}
func TestSanitizePath(t *testing.T) {
// Valid paths that should be accepted
validTests := []struct {
input string
expected string
}{
{"file.txt", "file.txt"},
{"dir/file.txt", "dir/file.txt"},
{"dir/subdir/file.txt", "dir/subdir/file.txt"},
{"./file.txt", "file.txt"},
{"./dir/file.txt", "dir/file.txt"},
{"dir/./file.txt", "dir/file.txt"},
}
for _, tt := range validTests {
t.Run("valid:"+tt.input, func(t *testing.T) {
result, err := sanitizePath(tt.input)
assert.NoError(t, err)
assert.Equal(t, tt.expected, result)
})
}
// Invalid paths that should be rejected
invalidTests := []struct {
input string
desc string
}{
{"", "empty path"},
{"..", "parent directory"},
{"../file.txt", "parent traversal"},
{"../../file.txt", "double parent traversal"},
{"dir/../../../file.txt", "traversal escaping base"},
{"/etc/passwd", "absolute path"},
{"/file.txt", "absolute path with single component"},
{"dir/../../etc/passwd", "traversal to system file"},
}
for _, tt := range invalidTests {
t.Run("invalid:"+tt.desc, func(t *testing.T) {
_, err := sanitizePath(tt.input)
assert.Error(t, err, "expected error for path: %s", tt.input)
})
}
}
func TestResolveManifestURL(t *testing.T) {
tests := []struct {
input string
expected string
}{
// Already ends with .mf - use as-is
{"https://example.com/path/index.mf", "https://example.com/path/index.mf"},
{"https://example.com/path/custom.mf", "https://example.com/path/custom.mf"},
{"https://example.com/foo.mf", "https://example.com/foo.mf"},
// Directory with trailing slash - append index.mf
{"https://example.com/path/", "https://example.com/path/index.mf"},
{"https://example.com/", "https://example.com/index.mf"},
// Directory without trailing slash - add slash and index.mf
{"https://example.com/path", "https://example.com/path/index.mf"},
{"https://example.com", "https://example.com/index.mf"},
// With query strings
{"https://example.com/path?foo=bar", "https://example.com/path/index.mf?foo=bar"},
}
for _, tt := range tests {
t.Run(tt.input, func(t *testing.T) {
result, err := resolveManifestURL(tt.input)
assert.NoError(t, err)
assert.Equal(t, tt.expected, result)
})
}
}
func TestFetchFromHTTP(t *testing.T) {
// Create source filesystem with test files
sourceFs := afero.NewMemMapFs()
testFiles := map[string][]byte{
"file1.txt": []byte("Hello, World!"),
"file2.txt": []byte("This is file 2 with more content."),
"subdir/file3.txt": []byte("Nested file content here."),
"subdir/deep/f.txt": []byte("Deeply nested file."),
}
for path, content := range testFiles {
fullPath := "/" + path // MemMapFs needs absolute paths
dir := filepath.Dir(fullPath)
require.NoError(t, sourceFs.MkdirAll(dir, 0o755))
require.NoError(t, afero.WriteFile(sourceFs, fullPath, content, 0o644))
}
// Generate manifest using scanner
opts := &mfer.ScannerOptions{
Fs: sourceFs,
}
s := mfer.NewScannerWithOptions(opts)
require.NoError(t, s.EnumerateFS(sourceFs, "/", nil))
var manifestBuf bytes.Buffer
require.NoError(t, s.ToManifest(context.Background(), &manifestBuf, nil))
manifestData := manifestBuf.Bytes()
// Create HTTP server that serves the source filesystem
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
path := r.URL.Path
if path == "/index.mf" {
w.Header().Set("Content-Type", "application/octet-stream")
_, _ = w.Write(manifestData)
return
}
// Strip leading slash
if len(path) > 0 && path[0] == '/' {
path = path[1:]
}
content, exists := testFiles[path]
if !exists {
http.NotFound(w, r)
return
}
w.Header().Set("Content-Type", "application/octet-stream")
_, _ = w.Write(content)
}))
defer server.Close()
// Create destination directory
destDir, err := os.MkdirTemp("", "mfer-fetch-test-*")
require.NoError(t, err)
defer func() { _ = os.RemoveAll(destDir) }()
// Change to dest directory for the test
origDir, err := os.Getwd()
require.NoError(t, err)
require.NoError(t, os.Chdir(destDir))
defer func() { _ = os.Chdir(origDir) }()
// Parse the manifest to get file entries
manifest, err := mfer.NewManifestFromReader(bytes.NewReader(manifestData))
require.NoError(t, err)
files := manifest.Files()
require.Len(t, files, len(testFiles))
// Download each file using downloadFile
progress := make(chan DownloadProgress, 10)
go func() {
for range progress {
// Drain progress channel
}
}()
baseURL := server.URL + "/"
for _, f := range files {
localPath, err := sanitizePath(f.Path)
require.NoError(t, err)
fileURL := baseURL + f.Path
err = downloadFile(fileURL, localPath, f, progress)
require.NoError(t, err, "failed to download %s", f.Path)
}
close(progress)
// Verify downloaded files match originals
for path, expectedContent := range testFiles {
downloadedPath := filepath.Join(destDir, path)
downloadedContent, err := os.ReadFile(downloadedPath)
require.NoError(t, err, "failed to read downloaded file %s", path)
assert.Equal(t, expectedContent, downloadedContent, "content mismatch for %s", path)
}
}
func TestFetchHashMismatch(t *testing.T) {
// Create source filesystem with a test file
sourceFs := afero.NewMemMapFs()
originalContent := []byte("Original content")
require.NoError(t, afero.WriteFile(sourceFs, "/file.txt", originalContent, 0o644))
// Generate manifest
opts := &mfer.ScannerOptions{Fs: sourceFs}
s := mfer.NewScannerWithOptions(opts)
require.NoError(t, s.EnumerateFS(sourceFs, "/", nil))
var manifestBuf bytes.Buffer
require.NoError(t, s.ToManifest(context.Background(), &manifestBuf, nil))
// Parse manifest
manifest, err := mfer.NewManifestFromReader(bytes.NewReader(manifestBuf.Bytes()))
require.NoError(t, err)
files := manifest.Files()
require.Len(t, files, 1)
// Create server that serves DIFFERENT content (to trigger hash mismatch)
tamperedContent := []byte("Tampered content!")
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/octet-stream")
_, _ = w.Write(tamperedContent)
}))
defer server.Close()
// Create temp directory
destDir, err := os.MkdirTemp("", "mfer-fetch-hash-test-*")
require.NoError(t, err)
defer func() { _ = os.RemoveAll(destDir) }()
origDir, err := os.Getwd()
require.NoError(t, err)
require.NoError(t, os.Chdir(destDir))
defer func() { _ = os.Chdir(origDir) }()
// Try to download - should fail with hash mismatch
err = downloadFile(server.URL+"/file.txt", "file.txt", files[0], nil)
assert.Error(t, err)
assert.Contains(t, err.Error(), "mismatch")
// Verify temp file was cleaned up
_, err = os.Stat(".file.txt.tmp")
assert.True(t, os.IsNotExist(err), "temp file should be cleaned up on hash mismatch")
// Verify final file was not created
_, err = os.Stat("file.txt")
assert.True(t, os.IsNotExist(err), "final file should not exist on hash mismatch")
}
func TestFetchSizeMismatch(t *testing.T) {
// Create source filesystem with a test file
sourceFs := afero.NewMemMapFs()
originalContent := []byte("Original content with specific size")
require.NoError(t, afero.WriteFile(sourceFs, "/file.txt", originalContent, 0o644))
// Generate manifest
opts := &mfer.ScannerOptions{Fs: sourceFs}
s := mfer.NewScannerWithOptions(opts)
require.NoError(t, s.EnumerateFS(sourceFs, "/", nil))
var manifestBuf bytes.Buffer
require.NoError(t, s.ToManifest(context.Background(), &manifestBuf, nil))
// Parse manifest
manifest, err := mfer.NewManifestFromReader(bytes.NewReader(manifestBuf.Bytes()))
require.NoError(t, err)
files := manifest.Files()
require.Len(t, files, 1)
// Create server that serves content with wrong size
wrongSizeContent := []byte("Short")
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/octet-stream")
_, _ = w.Write(wrongSizeContent)
}))
defer server.Close()
// Create temp directory
destDir, err := os.MkdirTemp("", "mfer-fetch-size-test-*")
require.NoError(t, err)
defer func() { _ = os.RemoveAll(destDir) }()
origDir, err := os.Getwd()
require.NoError(t, err)
require.NoError(t, os.Chdir(destDir))
defer func() { _ = os.Chdir(origDir) }()
// Try to download - should fail with size mismatch
err = downloadFile(server.URL+"/file.txt", "file.txt", files[0], nil)
assert.Error(t, err)
assert.Contains(t, err.Error(), "size mismatch")
// Verify temp file was cleaned up
_, err = os.Stat(".file.txt.tmp")
assert.True(t, os.IsNotExist(err), "temp file should be cleaned up on size mismatch")
}
func TestFetchProgress(t *testing.T) {
// Create source filesystem with a larger test file
sourceFs := afero.NewMemMapFs()
// Create content large enough to trigger multiple progress updates
content := bytes.Repeat([]byte("x"), 100*1024) // 100KB
require.NoError(t, afero.WriteFile(sourceFs, "/large.txt", content, 0o644))
// Generate manifest
opts := &mfer.ScannerOptions{Fs: sourceFs}
s := mfer.NewScannerWithOptions(opts)
require.NoError(t, s.EnumerateFS(sourceFs, "/", nil))
var manifestBuf bytes.Buffer
require.NoError(t, s.ToManifest(context.Background(), &manifestBuf, nil))
// Parse manifest
manifest, err := mfer.NewManifestFromReader(bytes.NewReader(manifestBuf.Bytes()))
require.NoError(t, err)
files := manifest.Files()
require.Len(t, files, 1)
// Create server that serves the content
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/octet-stream")
w.Header().Set("Content-Length", "102400")
// Write in chunks to allow progress reporting
reader := bytes.NewReader(content)
_, _ = io.Copy(w, reader)
}))
defer server.Close()
// Create temp directory
destDir, err := os.MkdirTemp("", "mfer-fetch-progress-test-*")
require.NoError(t, err)
defer func() { _ = os.RemoveAll(destDir) }()
origDir, err := os.Getwd()
require.NoError(t, err)
require.NoError(t, os.Chdir(destDir))
defer func() { _ = os.Chdir(origDir) }()
// Set up progress channel and collect updates
progress := make(chan DownloadProgress, 100)
var progressUpdates []DownloadProgress
done := make(chan struct{})
go func() {
for p := range progress {
progressUpdates = append(progressUpdates, p)
}
close(done)
}()
// Download
err = downloadFile(server.URL+"/large.txt", "large.txt", files[0], progress)
close(progress)
<-done
require.NoError(t, err)
// Verify we got progress updates
assert.NotEmpty(t, progressUpdates, "should have received progress updates")
// Verify final progress shows complete
if len(progressUpdates) > 0 {
last := progressUpdates[len(progressUpdates)-1]
assert.Equal(t, int64(len(content)), last.BytesRead, "final progress should show all bytes read")
assert.Equal(t, "large.txt", last.Path)
}
// Verify file was downloaded correctly
downloaded, err := os.ReadFile("large.txt")
require.NoError(t, err)
assert.Equal(t, content, downloaded)
}

386
internal/cli/freshen.go Normal file
View File

@@ -0,0 +1,386 @@
package cli
import (
"crypto/sha256"
"fmt"
"io"
"io/fs"
"path/filepath"
"time"
"github.com/dustin/go-humanize"
"github.com/multiformats/go-multihash"
"github.com/spf13/afero"
"github.com/urfave/cli/v2"
"sneak.berlin/go/mfer/internal/log"
"sneak.berlin/go/mfer/mfer"
)
// FreshenStatus contains progress information for the freshen operation.
type FreshenStatus struct {
Phase string // "scan" or "hash"
TotalFiles int64 // Total files to process in current phase
CurrentFiles int64 // Files processed so far
TotalBytes int64 // Total bytes to hash (hash phase only)
CurrentBytes int64 // Bytes hashed so far
BytesPerSec float64 // Current throughput rate
ETA time.Duration // Estimated time to completion
}
// freshenEntry tracks a file's status during freshen
type freshenEntry struct {
path string
size int64
mtime time.Time
needsHash bool // true if new or changed
existing *mfer.MFFilePath // existing manifest entry if unchanged
}
func (mfa *CLIApp) freshenManifestOperation(ctx *cli.Context) error {
log.Debug("freshenManifestOperation()")
basePath := ctx.String("base")
showProgress := ctx.Bool("progress")
includeDotfiles := ctx.Bool("IncludeDotfiles")
followSymlinks := ctx.Bool("FollowSymLinks")
// Find manifest file
var manifestPath string
var err error
if ctx.Args().Len() > 0 {
arg := ctx.Args().Get(0)
info, statErr := mfa.Fs.Stat(arg)
if statErr == nil && info.IsDir() {
manifestPath, err = findManifest(mfa.Fs, arg)
if err != nil {
return err
}
} else {
manifestPath = arg
}
} else {
manifestPath, err = findManifest(mfa.Fs, ".")
if err != nil {
return err
}
}
log.Infof("loading manifest from %s", manifestPath)
// Load existing manifest
manifest, err := mfer.NewManifestFromFile(mfa.Fs, manifestPath)
if err != nil {
return fmt.Errorf("failed to load manifest: %w", err)
}
existingFiles := manifest.Files()
log.Infof("manifest contains %d files", len(existingFiles))
// Build map of existing entries by path
existingByPath := make(map[string]*mfer.MFFilePath, len(existingFiles))
for _, f := range existingFiles {
existingByPath[f.Path] = f
}
// Phase 1: Scan filesystem
log.Infof("scanning filesystem...")
startScan := time.Now()
var entries []*freshenEntry
var scanCount int64
var removed, changed, added, unchanged int64
absBase, err := filepath.Abs(basePath)
if err != nil {
return err
}
err = afero.Walk(mfa.Fs, absBase, func(path string, info fs.FileInfo, walkErr error) error {
if walkErr != nil {
return walkErr
}
// Get relative path
relPath, err := filepath.Rel(absBase, path)
if err != nil {
return err
}
// Skip the manifest file itself
if relPath == filepath.Base(manifestPath) || relPath == "."+filepath.Base(manifestPath) {
return nil
}
// Handle dotfiles
if !includeDotfiles && mfer.IsHiddenPath(filepath.ToSlash(relPath)) {
if info.IsDir() {
return filepath.SkipDir
}
return nil
}
// Skip directories
if info.IsDir() {
return nil
}
// Handle symlinks
if info.Mode()&fs.ModeSymlink != 0 {
if !followSymlinks {
return nil
}
realPath, err := filepath.EvalSymlinks(path)
if err != nil {
return nil // Skip broken symlinks
}
realInfo, err := mfa.Fs.Stat(realPath)
if err != nil || realInfo.IsDir() {
return nil
}
info = realInfo
}
scanCount++
// Check against existing manifest
existing, inManifest := existingByPath[relPath]
if inManifest {
// Check if changed (size or mtime)
existingMtime := time.Unix(existing.Mtime.Seconds, int64(existing.Mtime.Nanos))
if existing.Size != info.Size() || !existingMtime.Equal(info.ModTime()) {
changed++
log.Verbosef("M %s", relPath)
entries = append(entries, &freshenEntry{
path: relPath,
size: info.Size(),
mtime: info.ModTime(),
needsHash: true,
})
} else {
unchanged++
entries = append(entries, &freshenEntry{
path: relPath,
size: info.Size(),
mtime: info.ModTime(),
needsHash: false,
existing: existing,
})
}
// Mark as seen
delete(existingByPath, relPath)
} else {
added++
log.Verbosef("A %s", relPath)
entries = append(entries, &freshenEntry{
path: relPath,
size: info.Size(),
mtime: info.ModTime(),
needsHash: true,
})
}
// Report scan progress
if showProgress && scanCount%100 == 0 {
log.Progressf("Scanning: %d files found", scanCount)
}
return nil
})
if showProgress {
log.ProgressDone()
}
if err != nil {
return fmt.Errorf("failed to scan filesystem: %w", err)
}
// Remaining entries in existingByPath are removed files
removed = int64(len(existingByPath))
for path := range existingByPath {
log.Verbosef("D %s", path)
}
scanDuration := time.Since(startScan)
log.Infof("scan complete in %s: %d unchanged, %d changed, %d added, %d removed",
scanDuration.Round(time.Millisecond), unchanged, changed, added, removed)
// Calculate total bytes to hash
var totalHashBytes int64
var filesToHash int64
for _, e := range entries {
if e.needsHash {
totalHashBytes += e.size
filesToHash++
}
}
// Phase 2: Hash changed and new files
if filesToHash > 0 {
log.Infof("hashing %d files (%s)...", filesToHash, humanize.IBytes(uint64(totalHashBytes)))
}
startHash := time.Now()
var hashedFiles int64
var hashedBytes int64
builder := mfer.NewBuilder()
// Set up signing options if sign-key is provided
if signKey := ctx.String("sign-key"); signKey != "" {
builder.SetSigningOptions(&mfer.SigningOptions{
KeyID: mfer.GPGKeyID(signKey),
})
log.Infof("signing manifest with GPG key: %s", signKey)
}
for _, e := range entries {
select {
case <-ctx.Done():
return ctx.Err()
default:
}
if e.needsHash {
// Need to read and hash the file
absPath := filepath.Join(absBase, e.path)
f, err := mfa.Fs.Open(absPath)
if err != nil {
return fmt.Errorf("failed to open %s: %w", e.path, err)
}
hash, bytesRead, err := hashFile(f, e.size, func(n int64) {
if showProgress {
currentBytes := hashedBytes + n
elapsed := time.Since(startHash)
var rate float64
var eta time.Duration
if elapsed > 0 && currentBytes > 0 {
rate = float64(currentBytes) / elapsed.Seconds()
remaining := totalHashBytes - currentBytes
if rate > 0 {
eta = time.Duration(float64(remaining)/rate) * time.Second
}
}
if eta > 0 {
log.Progressf("Hashing: %d/%d files, %s/s, ETA %s",
hashedFiles, filesToHash, humanize.IBytes(uint64(rate)), eta.Round(time.Second))
} else {
log.Progressf("Hashing: %d/%d files, %s/s",
hashedFiles, filesToHash, humanize.IBytes(uint64(rate)))
}
}
})
_ = f.Close()
if err != nil {
return fmt.Errorf("failed to hash %s: %w", e.path, err)
}
hashedBytes += bytesRead
hashedFiles++
// Add to builder with computed hash
if err := addFileToBuilder(builder, e.path, e.size, e.mtime, hash); err != nil {
return fmt.Errorf("failed to add %s: %w", e.path, err)
}
} else {
// Use existing entry
if err := addExistingToBuilder(builder, e.existing); err != nil {
return fmt.Errorf("failed to add %s: %w", e.path, err)
}
}
}
if showProgress && filesToHash > 0 {
log.ProgressDone()
}
// Print summary
log.Infof("freshen complete: %d unchanged, %d changed, %d added, %d removed",
unchanged, changed, added, removed)
// Skip writing if nothing changed
if changed == 0 && added == 0 && removed == 0 {
log.Infof("manifest unchanged, skipping write")
return nil
}
// Write updated manifest atomically (write to temp, then rename)
tmpPath := manifestPath + ".tmp"
outFile, err := mfa.Fs.Create(tmpPath)
if err != nil {
return fmt.Errorf("failed to create temp file: %w", err)
}
err = builder.Build(outFile)
_ = outFile.Close()
if err != nil {
_ = mfa.Fs.Remove(tmpPath)
return fmt.Errorf("failed to write manifest: %w", err)
}
// Rename temp to final
if err := mfa.Fs.Rename(tmpPath, manifestPath); err != nil {
_ = mfa.Fs.Remove(tmpPath)
return fmt.Errorf("failed to rename manifest: %w", err)
}
totalDuration := time.Since(mfa.startupTime)
if hashedBytes > 0 {
hashDuration := time.Since(startHash)
hashRate := float64(hashedBytes) / hashDuration.Seconds()
log.Infof("hashed %s in %.1fs (%s/s)",
humanize.IBytes(uint64(hashedBytes)), totalDuration.Seconds(), humanize.IBytes(uint64(hashRate)))
}
log.Infof("wrote %d files to %s", len(entries), manifestPath)
return nil
}
// hashFile reads a file and computes its SHA256 multihash.
// Progress callback is called with bytes read so far.
func hashFile(r io.Reader, size int64, progress func(int64)) ([]byte, int64, error) {
h := sha256.New()
buf := make([]byte, 64*1024)
var total int64
for {
n, err := r.Read(buf)
if n > 0 {
h.Write(buf[:n])
total += int64(n)
if progress != nil {
progress(total)
}
}
if err == io.EOF {
break
}
if err != nil {
return nil, total, err
}
}
mh, err := multihash.Encode(h.Sum(nil), multihash.SHA2_256)
if err != nil {
return nil, total, err
}
return mh, total, nil
}
// addFileToBuilder adds a new file entry to the builder
func addFileToBuilder(b *mfer.Builder, path string, size int64, mtime time.Time, hash []byte) error {
return b.AddFileWithHash(mfer.RelFilePath(path), mfer.FileSize(size), mfer.ModTime(mtime), hash)
}
// addExistingToBuilder adds an existing manifest entry to the builder
func addExistingToBuilder(b *mfer.Builder, entry *mfer.MFFilePath) error {
mtime := time.Unix(entry.Mtime.Seconds, int64(entry.Mtime.Nanos))
if len(entry.Hashes) == 0 {
return nil
}
return b.AddFileWithHash(mfer.RelFilePath(entry.Path), mfer.FileSize(entry.Size), mfer.ModTime(mtime), entry.Hashes[0].MultiHash)
}

View File

@@ -0,0 +1,82 @@
package cli
import (
"bytes"
"context"
"testing"
"github.com/spf13/afero"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"sneak.berlin/go/mfer/mfer"
)
func TestFreshenUnchanged(t *testing.T) {
// Create filesystem with test files
fs := afero.NewMemMapFs()
require.NoError(t, fs.MkdirAll("/testdir", 0o755))
require.NoError(t, afero.WriteFile(fs, "/testdir/file1.txt", []byte("content1"), 0o644))
require.NoError(t, afero.WriteFile(fs, "/testdir/file2.txt", []byte("content2"), 0o644))
// Generate initial manifest
opts := &mfer.ScannerOptions{Fs: fs}
s := mfer.NewScannerWithOptions(opts)
require.NoError(t, s.EnumeratePath("/testdir", nil))
var manifestBuf bytes.Buffer
require.NoError(t, s.ToManifest(context.Background(), &manifestBuf, nil))
// Write manifest to filesystem
require.NoError(t, afero.WriteFile(fs, "/testdir/.index.mf", manifestBuf.Bytes(), 0o644))
// Parse manifest to verify
manifest, err := mfer.NewManifestFromFile(fs, "/testdir/.index.mf")
require.NoError(t, err)
assert.Len(t, manifest.Files(), 2)
}
func TestFreshenWithChanges(t *testing.T) {
// Create filesystem with test files
fs := afero.NewMemMapFs()
require.NoError(t, fs.MkdirAll("/testdir", 0o755))
require.NoError(t, afero.WriteFile(fs, "/testdir/file1.txt", []byte("content1"), 0o644))
require.NoError(t, afero.WriteFile(fs, "/testdir/file2.txt", []byte("content2"), 0o644))
// Generate initial manifest
opts := &mfer.ScannerOptions{Fs: fs}
s := mfer.NewScannerWithOptions(opts)
require.NoError(t, s.EnumeratePath("/testdir", nil))
var manifestBuf bytes.Buffer
require.NoError(t, s.ToManifest(context.Background(), &manifestBuf, nil))
// Write manifest to filesystem
require.NoError(t, afero.WriteFile(fs, "/testdir/.index.mf", manifestBuf.Bytes(), 0o644))
// Verify initial manifest has 2 files
manifest, err := mfer.NewManifestFromFile(fs, "/testdir/.index.mf")
require.NoError(t, err)
assert.Len(t, manifest.Files(), 2)
// Add a new file
require.NoError(t, afero.WriteFile(fs, "/testdir/file3.txt", []byte("content3"), 0o644))
// Modify file2 (change content and size)
require.NoError(t, afero.WriteFile(fs, "/testdir/file2.txt", []byte("modified content2"), 0o644))
// Remove file1
require.NoError(t, fs.Remove("/testdir/file1.txt"))
// Note: The freshen operation would need to be run here
// For now, we just verify the test setup is correct
exists, _ := afero.Exists(fs, "/testdir/file1.txt")
assert.False(t, exists)
exists, _ = afero.Exists(fs, "/testdir/file3.txt")
assert.True(t, exists)
content, _ := afero.ReadFile(fs, "/testdir/file2.txt")
assert.Equal(t, "modified content2", string(content))
}

176
internal/cli/gen.go Normal file
View File

@@ -0,0 +1,176 @@
package cli
import (
"fmt"
"os"
"os/signal"
"path/filepath"
"sync"
"syscall"
"time"
"github.com/dustin/go-humanize"
"github.com/spf13/afero"
"github.com/urfave/cli/v2"
"sneak.berlin/go/mfer/internal/log"
"sneak.berlin/go/mfer/mfer"
)
func (mfa *CLIApp) generateManifestOperation(ctx *cli.Context) error {
log.Debug("generateManifestOperation()")
opts := &mfer.ScannerOptions{
IncludeDotfiles: ctx.Bool("IncludeDotfiles"),
FollowSymLinks: ctx.Bool("FollowSymLinks"),
Fs: mfa.Fs,
}
// Set up signing options if sign-key is provided
if signKey := ctx.String("sign-key"); signKey != "" {
opts.SigningOptions = &mfer.SigningOptions{
KeyID: mfer.GPGKeyID(signKey),
}
log.Infof("signing manifest with GPG key: %s", signKey)
}
s := mfer.NewScannerWithOptions(opts)
// Phase 1: Enumeration - collect paths and stat files
args := ctx.Args()
showProgress := ctx.Bool("progress")
// Set up enumeration progress reporting
var enumProgress chan mfer.EnumerateStatus
var enumWg sync.WaitGroup
if showProgress {
enumProgress = make(chan mfer.EnumerateStatus, 1)
enumWg.Add(1)
go func() {
defer enumWg.Done()
for status := range enumProgress {
log.Progressf("Enumerating: %d files, %s",
status.FilesFound,
humanize.IBytes(uint64(status.BytesFound)))
}
log.ProgressDone()
}()
}
if args.Len() == 0 {
// Default to current directory
if err := s.EnumeratePath(".", enumProgress); err != nil {
return err
}
} else {
// Collect and validate all paths first
paths := make([]string, 0, args.Len())
for i := 0; i < args.Len(); i++ {
inputPath := args.Get(i)
ap, err := filepath.Abs(inputPath)
if err != nil {
return err
}
// Validate path exists before adding to list
if exists, _ := afero.Exists(mfa.Fs, ap); !exists {
return fmt.Errorf("path does not exist: %s", inputPath)
}
log.Debugf("enumerating path: %s", ap)
paths = append(paths, ap)
}
if err := s.EnumeratePaths(enumProgress, paths...); err != nil {
return err
}
}
enumWg.Wait()
log.Infof("enumerated %d files, %s total", s.FileCount(), humanize.IBytes(uint64(s.TotalBytes())))
// Check if output file exists
outputPath := ctx.String("output")
if exists, _ := afero.Exists(mfa.Fs, outputPath); exists {
if !ctx.Bool("force") {
return fmt.Errorf("output file %s already exists (use --force to overwrite)", outputPath)
}
}
// Create temp file for atomic write
tmpPath := outputPath + ".tmp"
outFile, err := mfa.Fs.Create(tmpPath)
if err != nil {
return fmt.Errorf("failed to create temp file: %w", err)
}
// Set up signal handler to clean up temp file on Ctrl-C
sigChan := make(chan os.Signal, 1)
signal.Notify(sigChan, os.Interrupt, syscall.SIGTERM)
go func() {
sig, ok := <-sigChan
if !ok || sig == nil {
return // Channel closed normally, not a signal
}
_ = outFile.Close()
_ = mfa.Fs.Remove(tmpPath)
os.Exit(1)
}()
// Clean up temp file on any error or interruption
success := false
defer func() {
signal.Stop(sigChan)
close(sigChan)
_ = outFile.Close()
if !success {
_ = mfa.Fs.Remove(tmpPath)
}
}()
// Phase 2: Scan - read file contents and generate manifest
var scanProgress chan mfer.ScanStatus
var scanWg sync.WaitGroup
if showProgress {
scanProgress = make(chan mfer.ScanStatus, 1)
scanWg.Add(1)
go func() {
defer scanWg.Done()
for status := range scanProgress {
if status.ETA > 0 {
log.Progressf("Scanning: %d/%d files, %s/s, ETA %s",
status.ScannedFiles,
status.TotalFiles,
humanize.IBytes(uint64(status.BytesPerSec)),
status.ETA.Round(time.Second))
} else {
log.Progressf("Scanning: %d/%d files, %s/s",
status.ScannedFiles,
status.TotalFiles,
humanize.IBytes(uint64(status.BytesPerSec)))
}
}
log.ProgressDone()
}()
}
err = s.ToManifest(ctx.Context, outFile, scanProgress)
scanWg.Wait()
if err != nil {
return fmt.Errorf("failed to generate manifest: %w", err)
}
// Close file before rename to ensure all data is flushed
if err := outFile.Close(); err != nil {
return fmt.Errorf("failed to close temp file: %w", err)
}
// Atomic rename
if err := mfa.Fs.Rename(tmpPath, outputPath); err != nil {
return fmt.Errorf("failed to rename temp file: %w", err)
}
success = true
elapsed := time.Since(mfa.startupTime).Seconds()
rate := float64(s.TotalBytes()) / elapsed
log.Infof("wrote %d files (%s) to %s in %.1fs (%s/s)", s.FileCount(), humanize.IBytes(uint64(s.TotalBytes())), outputPath, elapsed, humanize.IBytes(uint64(rate)))
return nil
}

65
internal/cli/list.go Normal file
View File

@@ -0,0 +1,65 @@
package cli
import (
"fmt"
"time"
"github.com/urfave/cli/v2"
"sneak.berlin/go/mfer/internal/log"
"sneak.berlin/go/mfer/mfer"
)
func (mfa *CLIApp) listManifestOperation(ctx *cli.Context) error {
// Default to ErrorLevel for clean output
log.SetLevel(log.ErrorLevel)
longFormat := ctx.Bool("long")
print0 := ctx.Bool("print0")
// Find manifest file
var manifestPath string
var err error
if ctx.Args().Len() > 0 {
arg := ctx.Args().Get(0)
info, statErr := mfa.Fs.Stat(arg)
if statErr == nil && info.IsDir() {
manifestPath, err = findManifest(mfa.Fs, arg)
if err != nil {
return err
}
} else {
manifestPath = arg
}
} else {
manifestPath, err = findManifest(mfa.Fs, ".")
if err != nil {
return err
}
}
// Load manifest
manifest, err := mfer.NewManifestFromFile(mfa.Fs, manifestPath)
if err != nil {
return fmt.Errorf("failed to load manifest: %w", err)
}
files := manifest.Files()
// Determine line ending
lineEnd := "\n"
if print0 {
lineEnd = "\x00"
}
for _, f := range files {
if longFormat {
mtime := time.Unix(f.Mtime.Seconds, int64(f.Mtime.Nanos))
_, _ = fmt.Fprintf(mfa.Stdout, "%d\t%s\t%s%s", f.Size, mtime.Format(time.RFC3339), f.Path, lineEnd)
} else {
_, _ = fmt.Fprintf(mfa.Stdout, "%s%s", f.Path, lineEnd)
}
}
return nil
}

278
internal/cli/mfer.go Normal file
View File

@@ -0,0 +1,278 @@
package cli
import (
"fmt"
"io"
"os"
"time"
"github.com/spf13/afero"
"github.com/urfave/cli/v2"
"sneak.berlin/go/mfer/internal/log"
"sneak.berlin/go/mfer/mfer"
)
// CLIApp is the main CLI application container. It holds configuration,
// I/O streams, and filesystem abstraction to enable testing and flexibility.
type CLIApp struct {
appname string
version string
gitrev string
startupTime time.Time
exitCode int
app *cli.App
Stdin io.Reader // Standard input stream
Stdout io.Writer // Standard output stream for normal output
Stderr io.Writer // Standard error stream for diagnostics
Fs afero.Fs // Filesystem abstraction for all file operations
}
const banner = `
___ ___ ___ ___
/__/\ / /\ / /\ / /\
| |::\ / /:/_ / /:/_ / /::\
| |:|:\ / /:/ /\ / /:/ /\ / /:/\:\
__|__|:|\:\ / /:/ /:/ / /:/ /:/_ / /:/~/:/
/__/::::| \:\ /__/:/ /:/ /__/:/ /:/ /\ /__/:/ /:/___
\ \:\~~\__\/ \ \:\/:/ \ \:\/:/ /:/ \ \:\/:::::/
\ \:\ \ \::/ \ \::/ /:/ \ \::/~~~~
\ \:\ \ \:\ \ \:\/:/ \ \:\
\ \:\ \ \:\ \ \::/ \ \:\
\__\/ \__\/ \__\/ \__\/`
func (mfa *CLIApp) printBanner() {
if log.GetLevel() <= log.InfoLevel {
_, _ = fmt.Fprintln(mfa.Stdout, banner)
_, _ = fmt.Fprintf(mfa.Stdout, " mfer by @sneak: v%s released %s\n", mfer.Version, mfer.ReleaseDate)
_, _ = fmt.Fprintln(mfa.Stdout, " https://sneak.berlin/go/mfer")
}
}
// VersionString returns the version and git revision formatted for display.
func (mfa *CLIApp) VersionString() string {
if mfa.gitrev != "" {
return fmt.Sprintf("%s (%s)", mfer.Version, mfa.gitrev)
}
return mfer.Version
}
func (mfa *CLIApp) setVerbosity(c *cli.Context) {
_, present := os.LookupEnv("MFER_DEBUG")
if present {
log.EnableDebugLogging()
} else if c.Bool("quiet") {
log.SetLevel(log.ErrorLevel)
} else {
log.SetLevelFromVerbosity(c.Count("verbose"))
}
}
// commonFlags returns the flags shared by most commands (-v, -q)
func commonFlags() []cli.Flag {
return []cli.Flag{
&cli.BoolFlag{
Name: "verbose",
Aliases: []string{"v"},
Usage: "Increase verbosity (-v for verbose, -vv for debug)",
Count: new(int),
},
&cli.BoolFlag{
Name: "quiet",
Aliases: []string{"q"},
Usage: "Suppress output except errors",
},
}
}
func (mfa *CLIApp) run(args []string) {
mfa.startupTime = time.Now()
if NO_COLOR {
// shoutout to rob pike who thinks it's juvenile
log.DisableStyling()
}
// Configure log package to use our I/O streams
log.SetOutput(mfa.Stdout, mfa.Stderr)
log.Init()
mfa.app = &cli.App{
Name: mfa.appname,
Usage: "Manifest generator",
Version: mfa.VersionString(),
EnableBashCompletion: true,
Writer: mfa.Stdout,
ErrWriter: mfa.Stderr,
Action: func(c *cli.Context) error {
if c.Args().Len() > 0 {
return fmt.Errorf("unknown command %q", c.Args().First())
}
mfa.printBanner()
return cli.ShowAppHelp(c)
},
Commands: []*cli.Command{
{
Name: "generate",
Aliases: []string{"gen"},
Usage: "Generate manifest file",
Action: func(c *cli.Context) error {
mfa.setVerbosity(c)
mfa.printBanner()
return mfa.generateManifestOperation(c)
},
Flags: append(commonFlags(),
&cli.BoolFlag{
Name: "FollowSymLinks",
Aliases: []string{"follow-symlinks"},
Usage: "Resolve encountered symlinks",
},
&cli.BoolFlag{
Name: "IncludeDotfiles",
Aliases: []string{"include-dotfiles"},
Usage: "Include dot (hidden) files (excluded by default)",
},
&cli.StringFlag{
Name: "output",
Value: "./.index.mf",
Aliases: []string{"o"},
Usage: "Specify output filename",
},
&cli.BoolFlag{
Name: "force",
Aliases: []string{"f"},
Usage: "Overwrite output file if it exists",
},
&cli.BoolFlag{
Name: "progress",
Aliases: []string{"P"},
Usage: "Show progress during enumeration and scanning",
},
&cli.StringFlag{
Name: "sign-key",
Aliases: []string{"s"},
Usage: "GPG key ID to sign the manifest with",
EnvVars: []string{"MFER_SIGN_KEY"},
},
),
},
{
Name: "check",
Usage: "Validate files using manifest file",
ArgsUsage: "[manifest file]",
Action: func(c *cli.Context) error {
mfa.setVerbosity(c)
mfa.printBanner()
return mfa.checkManifestOperation(c)
},
Flags: append(commonFlags(),
&cli.StringFlag{
Name: "base",
Aliases: []string{"b"},
Value: ".",
Usage: "Base directory for resolving relative paths from manifest",
},
&cli.BoolFlag{
Name: "progress",
Aliases: []string{"P"},
Usage: "Show progress during checking",
},
&cli.BoolFlag{
Name: "no-extra-files",
Usage: "Fail if files exist in base directory that are not in manifest",
},
&cli.StringFlag{
Name: "require-signature",
Aliases: []string{"S"},
Usage: "Require manifest to be signed by the specified GPG key ID",
EnvVars: []string{"MFER_REQUIRE_SIGNATURE"},
},
),
},
{
Name: "freshen",
Usage: "Update manifest with changed, new, and removed files",
ArgsUsage: "[manifest file]",
Action: func(c *cli.Context) error {
mfa.setVerbosity(c)
mfa.printBanner()
return mfa.freshenManifestOperation(c)
},
Flags: append(commonFlags(),
&cli.StringFlag{
Name: "base",
Aliases: []string{"b"},
Value: ".",
Usage: "Base directory for resolving relative paths",
},
&cli.BoolFlag{
Name: "FollowSymLinks",
Aliases: []string{"follow-symlinks"},
Usage: "Resolve encountered symlinks",
},
&cli.BoolFlag{
Name: "IncludeDotfiles",
Aliases: []string{"include-dotfiles"},
Usage: "Include dot (hidden) files (excluded by default)",
},
&cli.BoolFlag{
Name: "progress",
Aliases: []string{"P"},
Usage: "Show progress during scanning and hashing",
},
&cli.StringFlag{
Name: "sign-key",
Aliases: []string{"s"},
Usage: "GPG key ID to sign the manifest with",
EnvVars: []string{"MFER_SIGN_KEY"},
},
),
},
{
Name: "version",
Usage: "Show version",
Action: func(c *cli.Context) error {
_, _ = fmt.Fprintln(mfa.Stdout, mfa.VersionString())
return nil
},
},
{
Name: "list",
Aliases: []string{"ls"},
Usage: "List files in manifest",
ArgsUsage: "[manifest file]",
Action: func(c *cli.Context) error {
return mfa.listManifestOperation(c)
},
Flags: []cli.Flag{
&cli.BoolFlag{
Name: "long",
Aliases: []string{"l"},
Usage: "Show size and mtime",
},
&cli.BoolFlag{
Name: "print0",
Usage: "Separate entries with NUL character (for xargs -0)",
},
},
},
{
Name: "fetch",
Usage: "fetch manifest and referenced files",
Action: func(c *cli.Context) error {
mfa.setVerbosity(c)
mfa.printBanner()
return mfa.fetchManifestOperation(c)
},
Flags: commonFlags(),
},
},
}
mfa.app.HideVersion = true
err := mfa.app.Run(args)
if err != nil {
mfa.exitCode = 1
log.WithError(err).Debugf("exiting")
}
}

274
internal/log/log.go Normal file
View File

@@ -0,0 +1,274 @@
package log
import (
"fmt"
"io"
"os"
"path/filepath"
"runtime"
"sync"
"github.com/apex/log"
acli "github.com/apex/log/handlers/cli"
"github.com/davecgh/go-spew/spew"
"github.com/pterm/pterm"
)
// Level represents log severity levels.
// Lower values are more verbose.
type Level int
const (
// DebugLevel is for low-level tracing and structure inspection
DebugLevel Level = iota
// VerboseLevel is for detailed operational info (file listings, etc)
VerboseLevel
// InfoLevel is for operational summaries (default)
InfoLevel
// WarnLevel is for warnings
WarnLevel
// ErrorLevel is for errors
ErrorLevel
// FatalLevel is for fatal errors
FatalLevel
)
func (l Level) String() string {
switch l {
case DebugLevel:
return "debug"
case VerboseLevel:
return "verbose"
case InfoLevel:
return "info"
case WarnLevel:
return "warn"
case ErrorLevel:
return "error"
case FatalLevel:
return "fatal"
default:
return "unknown"
}
}
var (
// mu protects the output writers and level
mu sync.RWMutex
// stdout is the writer for progress output
stdout io.Writer = os.Stdout
// stderr is the writer for log output
stderr io.Writer = os.Stderr
// currentLevel is our log level (includes Verbose)
currentLevel Level = InfoLevel
)
// SetOutput configures the output writers for the log package.
// stdout is used for progress output, stderr is used for log messages.
func SetOutput(out, err io.Writer) {
mu.Lock()
defer mu.Unlock()
stdout = out
stderr = err
pterm.SetDefaultOutput(out)
}
// GetStdout returns the configured stdout writer.
func GetStdout() io.Writer {
mu.RLock()
defer mu.RUnlock()
return stdout
}
// GetStderr returns the configured stderr writer.
func GetStderr() io.Writer {
mu.RLock()
defer mu.RUnlock()
return stderr
}
// DisableStyling turns off colors and styling for terminal output.
func DisableStyling() {
pterm.DisableColor()
pterm.DisableStyling()
pterm.Debug.Prefix.Text = ""
pterm.Info.Prefix.Text = ""
pterm.Success.Prefix.Text = ""
pterm.Warning.Prefix.Text = ""
pterm.Error.Prefix.Text = ""
pterm.Fatal.Prefix.Text = ""
}
// Init initializes the logger with the CLI handler and default log level.
func Init() {
mu.RLock()
w := stderr
mu.RUnlock()
log.SetHandler(acli.New(w))
log.SetLevel(log.DebugLevel) // Let apex/log pass everything; we filter ourselves
}
// isEnabled returns true if messages at the given level should be logged.
func isEnabled(l Level) bool {
mu.RLock()
defer mu.RUnlock()
return l >= currentLevel
}
// Fatalf logs a formatted message at fatal level.
func Fatalf(format string, args ...interface{}) {
if isEnabled(FatalLevel) {
log.Fatalf(format, args...)
}
}
// Fatal logs a message at fatal level.
func Fatal(arg string) {
if isEnabled(FatalLevel) {
log.Fatal(arg)
}
}
// Errorf logs a formatted message at error level.
func Errorf(format string, args ...interface{}) {
if isEnabled(ErrorLevel) {
log.Errorf(format, args...)
}
}
// Error logs a message at error level.
func Error(arg string) {
if isEnabled(ErrorLevel) {
log.Error(arg)
}
}
// Warnf logs a formatted message at warn level.
func Warnf(format string, args ...interface{}) {
if isEnabled(WarnLevel) {
log.Warnf(format, args...)
}
}
// Warn logs a message at warn level.
func Warn(arg string) {
if isEnabled(WarnLevel) {
log.Warn(arg)
}
}
// Infof logs a formatted message at info level.
func Infof(format string, args ...interface{}) {
if isEnabled(InfoLevel) {
log.Infof(format, args...)
}
}
// Info logs a message at info level.
func Info(arg string) {
if isEnabled(InfoLevel) {
log.Info(arg)
}
}
// Verbosef logs a formatted message at verbose level.
func Verbosef(format string, args ...interface{}) {
if isEnabled(VerboseLevel) {
log.Infof(format, args...)
}
}
// Verbose logs a message at verbose level.
func Verbose(arg string) {
if isEnabled(VerboseLevel) {
log.Info(arg)
}
}
// Debugf logs a formatted message at debug level with caller location.
func Debugf(format string, args ...interface{}) {
if isEnabled(DebugLevel) {
DebugReal(fmt.Sprintf(format, args...), 2)
}
}
// Debug logs a message at debug level with caller location.
func Debug(arg string) {
if isEnabled(DebugLevel) {
DebugReal(arg, 2)
}
}
// DebugReal logs at debug level with caller info from the specified stack depth.
func DebugReal(arg string, cs int) {
if !isEnabled(DebugLevel) {
return
}
_, callerFile, callerLine, ok := runtime.Caller(cs)
if !ok {
return
}
tag := fmt.Sprintf("%s:%d: ", filepath.Base(callerFile), callerLine)
log.Debug(tag + arg)
}
// Dump logs a spew dump of the arguments at debug level.
func Dump(args ...interface{}) {
if isEnabled(DebugLevel) {
DebugReal(spew.Sdump(args...), 2)
}
}
// EnableDebugLogging sets the log level to debug.
func EnableDebugLogging() {
SetLevel(DebugLevel)
}
// VerbosityStepsToLogLevel converts a -v count to a log level.
// 0 returns InfoLevel, 1 returns VerboseLevel, 2+ returns DebugLevel.
func VerbosityStepsToLogLevel(l int) Level {
switch l {
case 0:
return InfoLevel
case 1:
return VerboseLevel
default:
return DebugLevel
}
}
// SetLevelFromVerbosity sets the log level based on -v flag count.
func SetLevelFromVerbosity(l int) {
SetLevel(VerbosityStepsToLogLevel(l))
}
// SetLevel sets the global log level.
func SetLevel(l Level) {
mu.Lock()
defer mu.Unlock()
currentLevel = l
}
// GetLevel returns the current log level.
func GetLevel() Level {
mu.RLock()
defer mu.RUnlock()
return currentLevel
}
// WithError returns a log entry with the error attached.
func WithError(e error) *log.Entry {
return log.Log.WithError(e)
}
// Progressf prints a progress message that overwrites the current line.
// Use ProgressDone() when progress is complete to move to the next line.
func Progressf(format string, args ...interface{}) {
pterm.Printf("\r"+format, args...)
}
// ProgressDone clears the progress line when progress is complete.
func ProgressDone() {
// Clear the line with spaces and return to beginning
pterm.Print("\r\033[K")
}

12
internal/log/log_test.go Normal file
View File

@@ -0,0 +1,12 @@
package log
import (
"testing"
"github.com/stretchr/testify/assert"
)
func TestBuild(t *testing.T) {
Init()
assert.True(t, true)
}

251
mfer/builder.go Normal file
View File

@@ -0,0 +1,251 @@
package mfer
import (
"crypto/sha256"
"errors"
"fmt"
"io"
"strings"
"sync"
"time"
"unicode/utf8"
"github.com/multiformats/go-multihash"
)
// ValidatePath checks that a file path conforms to manifest path invariants:
// - Must be valid UTF-8
// - Must use forward slashes only (no backslashes)
// - Must be relative (no leading /)
// - Must not contain ".." segments
// - Must not contain empty segments (no "//")
// - Must not be empty
func ValidatePath(p string) error {
if p == "" {
return errors.New("path cannot be empty")
}
if !utf8.ValidString(p) {
return fmt.Errorf("path %q is not valid UTF-8", p)
}
if strings.ContainsRune(p, '\\') {
return fmt.Errorf("path %q contains backslash; use forward slashes only", p)
}
if strings.HasPrefix(p, "/") {
return fmt.Errorf("path %q is absolute; must be relative", p)
}
for _, seg := range strings.Split(p, "/") {
if seg == "" {
return fmt.Errorf("path %q contains empty segment", p)
}
if seg == ".." {
return fmt.Errorf("path %q contains '..' segment", p)
}
}
return nil
}
// RelFilePath represents a relative file path within a manifest.
type RelFilePath string
// AbsFilePath represents an absolute file path on the filesystem.
type AbsFilePath string
// FileSize represents the size of a file in bytes.
type FileSize int64
// FileCount represents a count of files.
type FileCount int64
// ModTime represents a file's modification time.
type ModTime time.Time
// UnixSeconds represents seconds since Unix epoch.
type UnixSeconds int64
// UnixNanos represents the nanosecond component of a timestamp (0-999999999).
type UnixNanos int32
// Timestamp converts ModTime to a protobuf Timestamp.
func (m ModTime) Timestamp() *Timestamp {
t := time.Time(m)
return &Timestamp{
Seconds: t.Unix(),
Nanos: int32(t.Nanosecond()),
}
}
// Multihash represents a multihash-encoded file hash (typically SHA2-256).
type Multihash []byte
// FileHashProgress reports progress during file hashing.
type FileHashProgress struct {
BytesRead FileSize // Total bytes read so far for the current file
}
// Builder constructs a manifest by adding files one at a time.
type Builder struct {
mu sync.Mutex
files []*MFFilePath
createdAt time.Time
signingOptions *SigningOptions
}
// NewBuilder creates a new Builder.
func NewBuilder() *Builder {
return &Builder{
files: make([]*MFFilePath, 0),
createdAt: time.Now(),
}
}
// AddFile reads file content from reader, computes hashes, and adds to manifest.
// Progress updates are sent to the progress channel (if non-nil) without blocking.
// Returns the number of bytes read.
func (b *Builder) AddFile(
path RelFilePath,
size FileSize,
mtime ModTime,
reader io.Reader,
progress chan<- FileHashProgress,
) (FileSize, error) {
if err := ValidatePath(string(path)); err != nil {
return 0, err
}
// Create hash writer
h := sha256.New()
// Read file in chunks, updating hash and progress
var totalRead FileSize
buf := make([]byte, 64*1024) // 64KB chunks
for {
n, err := reader.Read(buf)
if n > 0 {
h.Write(buf[:n])
totalRead += FileSize(n)
sendFileHashProgress(progress, FileHashProgress{BytesRead: totalRead})
}
if err == io.EOF {
break
}
if err != nil {
return totalRead, err
}
}
// Verify actual bytes read matches declared size
if totalRead != size {
return totalRead, fmt.Errorf("size mismatch for %q: declared %d bytes but read %d bytes", path, size, totalRead)
}
// Encode hash as multihash (SHA2-256)
mh, err := multihash.Encode(h.Sum(nil), multihash.SHA2_256)
if err != nil {
return totalRead, err
}
// Create file entry
entry := &MFFilePath{
Path: string(path),
Size: int64(size),
Hashes: []*MFFileChecksum{
{MultiHash: mh},
},
Mtime: mtime.Timestamp(),
}
b.mu.Lock()
b.files = append(b.files, entry)
b.mu.Unlock()
return totalRead, nil
}
// sendFileHashProgress sends a progress update without blocking.
func sendFileHashProgress(ch chan<- FileHashProgress, p FileHashProgress) {
if ch == nil {
return
}
select {
case ch <- p:
default:
}
}
// FileCount returns the number of files added to the builder.
func (b *Builder) FileCount() int {
b.mu.Lock()
defer b.mu.Unlock()
return len(b.files)
}
// AddFileWithHash adds a file entry with a pre-computed hash.
// This is useful when the hash is already known (e.g., from an existing manifest).
// Returns an error if path is empty, size is negative, or hash is nil/empty.
func (b *Builder) AddFileWithHash(path RelFilePath, size FileSize, mtime ModTime, hash Multihash) error {
if err := ValidatePath(string(path)); err != nil {
return err
}
if size < 0 {
return errors.New("size cannot be negative")
}
if len(hash) == 0 {
return errors.New("hash cannot be nil or empty")
}
entry := &MFFilePath{
Path: string(path),
Size: int64(size),
Hashes: []*MFFileChecksum{
{MultiHash: hash},
},
Mtime: mtime.Timestamp(),
}
b.mu.Lock()
b.files = append(b.files, entry)
b.mu.Unlock()
return nil
}
// SetSigningOptions sets the GPG signing options for the manifest.
// If opts is non-nil, the manifest will be signed when Build() is called.
func (b *Builder) SetSigningOptions(opts *SigningOptions) {
b.mu.Lock()
defer b.mu.Unlock()
b.signingOptions = opts
}
// Build finalizes the manifest and writes it to the writer.
func (b *Builder) Build(w io.Writer) error {
b.mu.Lock()
defer b.mu.Unlock()
// Create inner manifest
inner := &MFFile{
Version: MFFile_VERSION_ONE,
CreatedAt: newTimestampFromTime(b.createdAt),
Files: b.files,
}
// Create a temporary manifest to use existing serialization
m := &manifest{
pbInner: inner,
signingOptions: b.signingOptions,
}
// Generate outer wrapper
if err := m.generateOuter(); err != nil {
return err
}
// Generate final output
if err := m.generate(); err != nil {
return err
}
// Write to output
_, err := w.Write(m.output.Bytes())
return err
}

127
mfer/builder_test.go Normal file
View File

@@ -0,0 +1,127 @@
package mfer
import (
"bytes"
"strings"
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestNewBuilder(t *testing.T) {
b := NewBuilder()
assert.NotNil(t, b)
assert.Equal(t, 0, b.FileCount())
}
func TestBuilderAddFile(t *testing.T) {
b := NewBuilder()
content := []byte("test content")
reader := bytes.NewReader(content)
bytesRead, err := b.AddFile("test.txt", FileSize(len(content)), ModTime(time.Now()), reader, nil)
require.NoError(t, err)
assert.Equal(t, FileSize(len(content)), bytesRead)
assert.Equal(t, 1, b.FileCount())
}
func TestBuilderAddFileWithHash(t *testing.T) {
b := NewBuilder()
hash := make([]byte, 34) // SHA256 multihash is 34 bytes
err := b.AddFileWithHash("test.txt", 100, ModTime(time.Now()), hash)
require.NoError(t, err)
assert.Equal(t, 1, b.FileCount())
}
func TestBuilderAddFileWithHashValidation(t *testing.T) {
t.Run("empty path", func(t *testing.T) {
b := NewBuilder()
hash := make([]byte, 34)
err := b.AddFileWithHash("", 100, ModTime(time.Now()), hash)
assert.Error(t, err)
assert.Contains(t, err.Error(), "path")
})
t.Run("negative size", func(t *testing.T) {
b := NewBuilder()
hash := make([]byte, 34)
err := b.AddFileWithHash("test.txt", -1, ModTime(time.Now()), hash)
assert.Error(t, err)
assert.Contains(t, err.Error(), "size")
})
t.Run("nil hash", func(t *testing.T) {
b := NewBuilder()
err := b.AddFileWithHash("test.txt", 100, ModTime(time.Now()), nil)
assert.Error(t, err)
assert.Contains(t, err.Error(), "hash")
})
t.Run("empty hash", func(t *testing.T) {
b := NewBuilder()
err := b.AddFileWithHash("test.txt", 100, ModTime(time.Now()), []byte{})
assert.Error(t, err)
assert.Contains(t, err.Error(), "hash")
})
t.Run("valid inputs", func(t *testing.T) {
b := NewBuilder()
hash := make([]byte, 34)
err := b.AddFileWithHash("test.txt", 100, ModTime(time.Now()), hash)
assert.NoError(t, err)
assert.Equal(t, 1, b.FileCount())
})
}
func TestBuilderBuild(t *testing.T) {
b := NewBuilder()
content := []byte("test content")
reader := bytes.NewReader(content)
_, err := b.AddFile("test.txt", FileSize(len(content)), ModTime(time.Now()), reader, nil)
require.NoError(t, err)
var buf bytes.Buffer
err = b.Build(&buf)
require.NoError(t, err)
// Should have magic bytes
assert.True(t, strings.HasPrefix(buf.String(), MAGIC))
}
func TestNewTimestampFromTimeExtremeDate(t *testing.T) {
// Regression test: newTimestampFromTime used UnixNano() which panics
// for dates outside ~1678-2262. Now uses Nanosecond() which is safe.
tests := []struct {
name string
time time.Time
}{
{"zero time", time.Time{}},
{"year 1000", time.Date(1000, 1, 1, 0, 0, 0, 0, time.UTC)},
{"year 3000", time.Date(3000, 1, 1, 0, 0, 0, 123456789, time.UTC)},
{"unix epoch", time.Unix(0, 0)},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
// Should not panic
ts := newTimestampFromTime(tt.time)
assert.Equal(t, tt.time.Unix(), ts.Seconds)
assert.Equal(t, int32(tt.time.Nanosecond()), ts.Nanos)
})
}
}
func TestBuilderBuildEmpty(t *testing.T) {
b := NewBuilder()
var buf bytes.Buffer
err := b.Build(&buf)
require.NoError(t, err)
// Should still produce valid manifest with 0 files
assert.True(t, strings.HasPrefix(buf.String(), MAGIC))
}

344
mfer/checker.go Normal file
View File

@@ -0,0 +1,344 @@
package mfer
import (
"bytes"
"context"
"crypto/sha256"
"errors"
"io"
"os"
"path/filepath"
"time"
"github.com/multiformats/go-multihash"
"github.com/spf13/afero"
)
// Result represents the outcome of checking a single file.
type Result struct {
Path RelFilePath // Relative path from manifest
Status Status // Verification result status
Message string // Human-readable description of the result
}
// Status represents the verification status of a file.
type Status int
const (
StatusOK Status = iota // File matches manifest (size and hash verified)
StatusMissing // File not found on disk
StatusSizeMismatch // File size differs from manifest
StatusHashMismatch // File hash differs from manifest
StatusExtra // File exists on disk but not in manifest
StatusError // Error occurred during verification
)
func (s Status) String() string {
switch s {
case StatusOK:
return "OK"
case StatusMissing:
return "MISSING"
case StatusSizeMismatch:
return "SIZE_MISMATCH"
case StatusHashMismatch:
return "HASH_MISMATCH"
case StatusExtra:
return "EXTRA"
case StatusError:
return "ERROR"
default:
return "UNKNOWN"
}
}
// CheckStatus contains progress information for the check operation.
type CheckStatus struct {
TotalFiles FileCount // Total number of files in manifest
CheckedFiles FileCount // Number of files checked so far
TotalBytes FileSize // Total bytes to verify (sum of all file sizes)
CheckedBytes FileSize // Bytes verified so far
BytesPerSec float64 // Current throughput rate
ETA time.Duration // Estimated time to completion
Failures FileCount // Number of verification failures encountered
}
// Checker verifies files against a manifest.
type Checker struct {
basePath AbsFilePath
files []*MFFilePath
fs afero.Fs
// manifestPaths is a set of paths in the manifest for quick lookup
manifestPaths map[RelFilePath]struct{}
// signature info from the manifest
signature []byte
signer []byte
signingPubKey []byte
}
// NewChecker creates a new Checker for the given manifest, base path, and filesystem.
// The basePath is the directory relative to which manifest paths are resolved.
// If fs is nil, the real filesystem (OsFs) is used.
func NewChecker(manifestPath string, basePath string, fs afero.Fs) (*Checker, error) {
if fs == nil {
fs = afero.NewOsFs()
}
m, err := NewManifestFromFile(fs, manifestPath)
if err != nil {
return nil, err
}
abs, err := filepath.Abs(basePath)
if err != nil {
return nil, err
}
files := m.Files()
manifestPaths := make(map[RelFilePath]struct{}, len(files))
for _, f := range files {
manifestPaths[RelFilePath(f.Path)] = struct{}{}
}
return &Checker{
basePath: AbsFilePath(abs),
files: files,
fs: fs,
manifestPaths: manifestPaths,
signature: m.pbOuter.Signature,
signer: m.pbOuter.Signer,
signingPubKey: m.pbOuter.SigningPubKey,
}, nil
}
// FileCount returns the number of files in the manifest.
func (c *Checker) FileCount() FileCount {
return FileCount(len(c.files))
}
// TotalBytes returns the total size of all files in the manifest.
func (c *Checker) TotalBytes() FileSize {
var total FileSize
for _, f := range c.files {
total += FileSize(f.Size)
}
return total
}
// IsSigned returns true if the manifest has a signature.
func (c *Checker) IsSigned() bool {
return len(c.signature) > 0
}
// Signer returns the signer fingerprint if the manifest is signed, nil otherwise.
func (c *Checker) Signer() []byte {
return c.signer
}
// SigningPubKey returns the signing public key if the manifest is signed, nil otherwise.
func (c *Checker) SigningPubKey() []byte {
return c.signingPubKey
}
// ExtractEmbeddedSigningKeyFP imports the manifest's embedded public key into a
// temporary keyring and extracts its fingerprint. This validates the key and
// returns its actual fingerprint from the key material itself.
func (c *Checker) ExtractEmbeddedSigningKeyFP() (string, error) {
if len(c.signingPubKey) == 0 {
return "", errors.New("manifest has no signing public key")
}
return gpgExtractPubKeyFingerprint(c.signingPubKey)
}
// Check verifies all files against the manifest.
// Results are sent to the results channel as files are checked.
// Progress updates are sent to the progress channel approximately once per second.
// Both channels are closed when the method returns.
func (c *Checker) Check(ctx context.Context, results chan<- Result, progress chan<- CheckStatus) error {
if results != nil {
defer close(results)
}
if progress != nil {
defer close(progress)
}
totalFiles := FileCount(len(c.files))
totalBytes := c.TotalBytes()
var checkedFiles FileCount
var checkedBytes FileSize
var failures FileCount
startTime := time.Now()
for _, entry := range c.files {
select {
case <-ctx.Done():
return ctx.Err()
default:
}
result := c.checkFile(entry, &checkedBytes)
if result.Status != StatusOK {
failures++
}
checkedFiles++
if results != nil {
results <- result
}
// Send progress with rate and ETA calculation
if progress != nil {
elapsed := time.Since(startTime)
var bytesPerSec float64
var eta time.Duration
if elapsed > 0 && checkedBytes > 0 {
bytesPerSec = float64(checkedBytes) / elapsed.Seconds()
remainingBytes := totalBytes - checkedBytes
if bytesPerSec > 0 {
eta = time.Duration(float64(remainingBytes)/bytesPerSec) * time.Second
}
}
sendCheckStatus(progress, CheckStatus{
TotalFiles: totalFiles,
CheckedFiles: checkedFiles,
TotalBytes: totalBytes,
CheckedBytes: checkedBytes,
BytesPerSec: bytesPerSec,
ETA: eta,
Failures: failures,
})
}
}
return nil
}
func (c *Checker) checkFile(entry *MFFilePath, checkedBytes *FileSize) Result {
absPath := filepath.Join(string(c.basePath), entry.Path)
relPath := RelFilePath(entry.Path)
// Check if file exists
info, err := c.fs.Stat(absPath)
if err != nil {
if errors.Is(err, os.ErrNotExist) || errors.Is(err, afero.ErrFileNotFound) {
return Result{Path: relPath, Status: StatusMissing, Message: "file not found"}
}
return Result{Path: relPath, Status: StatusError, Message: err.Error()}
}
// Check size
if info.Size() != entry.Size {
*checkedBytes += FileSize(info.Size())
return Result{
Path: relPath,
Status: StatusSizeMismatch,
Message: "size mismatch",
}
}
// Open and hash file
f, err := c.fs.Open(absPath)
if err != nil {
return Result{Path: relPath, Status: StatusError, Message: err.Error()}
}
defer func() { _ = f.Close() }()
h := sha256.New()
n, err := io.Copy(h, f)
if err != nil {
return Result{Path: relPath, Status: StatusError, Message: err.Error()}
}
*checkedBytes += FileSize(n)
// Encode as multihash and compare
computed, err := multihash.Encode(h.Sum(nil), multihash.SHA2_256)
if err != nil {
return Result{Path: relPath, Status: StatusError, Message: err.Error()}
}
// Check against all hashes in manifest (at least one must match)
for _, hash := range entry.Hashes {
if bytes.Equal(computed, hash.MultiHash) {
return Result{Path: relPath, Status: StatusOK}
}
}
return Result{Path: relPath, Status: StatusHashMismatch, Message: "hash mismatch"}
}
// FindExtraFiles walks the filesystem and reports files not in the manifest.
// Results are sent to the results channel. The channel is closed when done.
// Hidden files/directories (starting with .) are skipped, as they are excluded
// from manifests by default. The manifest file itself is also skipped.
func (c *Checker) FindExtraFiles(ctx context.Context, results chan<- Result) error {
if results != nil {
defer close(results)
}
return afero.Walk(c.fs, string(c.basePath), func(walkPath string, info os.FileInfo, err error) error {
if err != nil {
return err
}
select {
case <-ctx.Done():
return ctx.Err()
default:
}
// Get relative path
rel, err := filepath.Rel(string(c.basePath), walkPath)
if err != nil {
return err
}
// Skip hidden files and directories (dotfiles)
if IsHiddenPath(filepath.ToSlash(rel)) {
if info.IsDir() {
return filepath.SkipDir
}
return nil
}
// Skip directories
if info.IsDir() {
return nil
}
// Skip manifest files
base := filepath.Base(rel)
if base == "index.mf" || base == ".index.mf" {
return nil
}
relPath := RelFilePath(rel)
// Check if path is in manifest
if _, exists := c.manifestPaths[relPath]; !exists {
if results != nil {
results <- Result{
Path: relPath,
Status: StatusExtra,
Message: "not in manifest",
}
}
}
return nil
})
}
// sendCheckStatus sends a status update without blocking.
func sendCheckStatus(ch chan<- CheckStatus, status CheckStatus) {
if ch == nil {
return
}
select {
case ch <- status:
default:
}
}

475
mfer/checker_test.go Normal file
View File

@@ -0,0 +1,475 @@
package mfer
import (
"bytes"
"context"
"testing"
"time"
"github.com/spf13/afero"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestStatusString(t *testing.T) {
tests := []struct {
status Status
expected string
}{
{StatusOK, "OK"},
{StatusMissing, "MISSING"},
{StatusSizeMismatch, "SIZE_MISMATCH"},
{StatusHashMismatch, "HASH_MISMATCH"},
{StatusExtra, "EXTRA"},
{StatusError, "ERROR"},
{Status(99), "UNKNOWN"},
}
for _, tt := range tests {
t.Run(tt.expected, func(t *testing.T) {
assert.Equal(t, tt.expected, tt.status.String())
})
}
}
// createTestManifest creates a manifest file in the filesystem with the given files.
func createTestManifest(t *testing.T, fs afero.Fs, manifestPath string, files map[string][]byte) {
t.Helper()
builder := NewBuilder()
for path, content := range files {
reader := bytes.NewReader(content)
_, err := builder.AddFile(RelFilePath(path), FileSize(len(content)), ModTime(time.Now()), reader, nil)
require.NoError(t, err)
}
var buf bytes.Buffer
require.NoError(t, builder.Build(&buf))
require.NoError(t, afero.WriteFile(fs, manifestPath, buf.Bytes(), 0o644))
}
// createFilesOnDisk creates the given files on the filesystem.
func createFilesOnDisk(t *testing.T, fs afero.Fs, basePath string, files map[string][]byte) {
t.Helper()
for path, content := range files {
fullPath := basePath + "/" + path
require.NoError(t, fs.MkdirAll(basePath, 0o755))
require.NoError(t, afero.WriteFile(fs, fullPath, content, 0o644))
}
}
func TestNewChecker(t *testing.T) {
t.Run("valid manifest", func(t *testing.T) {
fs := afero.NewMemMapFs()
files := map[string][]byte{
"file1.txt": []byte("hello"),
"file2.txt": []byte("world"),
}
createTestManifest(t, fs, "/manifest.mf", files)
chk, err := NewChecker("/manifest.mf", "/", fs)
require.NoError(t, err)
assert.NotNil(t, chk)
assert.Equal(t, FileCount(2), chk.FileCount())
})
t.Run("missing manifest", func(t *testing.T) {
fs := afero.NewMemMapFs()
_, err := NewChecker("/nonexistent.mf", "/", fs)
assert.Error(t, err)
})
t.Run("invalid manifest", func(t *testing.T) {
fs := afero.NewMemMapFs()
require.NoError(t, afero.WriteFile(fs, "/bad.mf", []byte("not a manifest"), 0o644))
_, err := NewChecker("/bad.mf", "/", fs)
assert.Error(t, err)
})
}
func TestCheckerFileCountAndTotalBytes(t *testing.T) {
fs := afero.NewMemMapFs()
files := map[string][]byte{
"small.txt": []byte("hi"),
"medium.txt": []byte("hello world"),
"large.txt": bytes.Repeat([]byte("x"), 1000),
}
createTestManifest(t, fs, "/manifest.mf", files)
chk, err := NewChecker("/manifest.mf", "/", fs)
require.NoError(t, err)
assert.Equal(t, FileCount(3), chk.FileCount())
assert.Equal(t, FileSize(2+11+1000), chk.TotalBytes())
}
func TestCheckAllFilesOK(t *testing.T) {
fs := afero.NewMemMapFs()
files := map[string][]byte{
"file1.txt": []byte("content one"),
"file2.txt": []byte("content two"),
}
createTestManifest(t, fs, "/manifest.mf", files)
createFilesOnDisk(t, fs, "/data", files)
chk, err := NewChecker("/manifest.mf", "/data", fs)
require.NoError(t, err)
results := make(chan Result, 10)
err = chk.Check(context.Background(), results, nil)
require.NoError(t, err)
var resultList []Result
for r := range results {
resultList = append(resultList, r)
}
assert.Len(t, resultList, 2)
for _, r := range resultList {
assert.Equal(t, StatusOK, r.Status, "file %s should be OK", r.Path)
}
}
func TestCheckMissingFile(t *testing.T) {
fs := afero.NewMemMapFs()
files := map[string][]byte{
"exists.txt": []byte("I exist"),
"missing.txt": []byte("I don't exist on disk"),
}
createTestManifest(t, fs, "/manifest.mf", files)
// Only create one file
createFilesOnDisk(t, fs, "/data", map[string][]byte{
"exists.txt": []byte("I exist"),
})
chk, err := NewChecker("/manifest.mf", "/data", fs)
require.NoError(t, err)
results := make(chan Result, 10)
err = chk.Check(context.Background(), results, nil)
require.NoError(t, err)
var okCount, missingCount int
for r := range results {
switch r.Status {
case StatusOK:
okCount++
case StatusMissing:
missingCount++
assert.Equal(t, RelFilePath("missing.txt"), r.Path)
}
}
assert.Equal(t, 1, okCount)
assert.Equal(t, 1, missingCount)
}
func TestCheckSizeMismatch(t *testing.T) {
fs := afero.NewMemMapFs()
files := map[string][]byte{
"file.txt": []byte("original content"),
}
createTestManifest(t, fs, "/manifest.mf", files)
// Create file with different size
createFilesOnDisk(t, fs, "/data", map[string][]byte{
"file.txt": []byte("short"),
})
chk, err := NewChecker("/manifest.mf", "/data", fs)
require.NoError(t, err)
results := make(chan Result, 10)
err = chk.Check(context.Background(), results, nil)
require.NoError(t, err)
r := <-results
assert.Equal(t, StatusSizeMismatch, r.Status)
assert.Equal(t, RelFilePath("file.txt"), r.Path)
}
func TestCheckHashMismatch(t *testing.T) {
fs := afero.NewMemMapFs()
originalContent := []byte("original content")
files := map[string][]byte{
"file.txt": originalContent,
}
createTestManifest(t, fs, "/manifest.mf", files)
// Create file with same size but different content
differentContent := []byte("different contnt") // same length (16 bytes) but different
require.Equal(t, len(originalContent), len(differentContent), "test requires same length")
createFilesOnDisk(t, fs, "/data", map[string][]byte{
"file.txt": differentContent,
})
chk, err := NewChecker("/manifest.mf", "/data", fs)
require.NoError(t, err)
results := make(chan Result, 10)
err = chk.Check(context.Background(), results, nil)
require.NoError(t, err)
r := <-results
assert.Equal(t, StatusHashMismatch, r.Status)
assert.Equal(t, RelFilePath("file.txt"), r.Path)
}
func TestCheckWithProgress(t *testing.T) {
fs := afero.NewMemMapFs()
files := map[string][]byte{
"file1.txt": bytes.Repeat([]byte("a"), 100),
"file2.txt": bytes.Repeat([]byte("b"), 200),
}
createTestManifest(t, fs, "/manifest.mf", files)
createFilesOnDisk(t, fs, "/data", files)
chk, err := NewChecker("/manifest.mf", "/data", fs)
require.NoError(t, err)
results := make(chan Result, 10)
progress := make(chan CheckStatus, 10)
err = chk.Check(context.Background(), results, progress)
require.NoError(t, err)
// Drain results
for range results {
}
// Check progress was sent
var progressUpdates []CheckStatus
for p := range progress {
progressUpdates = append(progressUpdates, p)
}
assert.NotEmpty(t, progressUpdates)
// Final progress should show all files checked
final := progressUpdates[len(progressUpdates)-1]
assert.Equal(t, FileCount(2), final.TotalFiles)
assert.Equal(t, FileCount(2), final.CheckedFiles)
assert.Equal(t, FileSize(300), final.TotalBytes)
assert.Equal(t, FileSize(300), final.CheckedBytes)
assert.Equal(t, FileCount(0), final.Failures)
}
func TestCheckContextCancellation(t *testing.T) {
fs := afero.NewMemMapFs()
// Create many files to ensure we have time to cancel
files := make(map[string][]byte)
for i := 0; i < 100; i++ {
files[string(rune('a'+i%26))+".txt"] = bytes.Repeat([]byte("x"), 1000)
}
createTestManifest(t, fs, "/manifest.mf", files)
createFilesOnDisk(t, fs, "/data", files)
chk, err := NewChecker("/manifest.mf", "/data", fs)
require.NoError(t, err)
ctx, cancel := context.WithCancel(context.Background())
cancel() // Cancel immediately
results := make(chan Result, 200)
err = chk.Check(ctx, results, nil)
assert.ErrorIs(t, err, context.Canceled)
}
func TestFindExtraFiles(t *testing.T) {
fs := afero.NewMemMapFs()
// Manifest only contains file1
manifestFiles := map[string][]byte{
"file1.txt": []byte("in manifest"),
}
createTestManifest(t, fs, "/manifest.mf", manifestFiles)
// Disk has file1 and file2
createFilesOnDisk(t, fs, "/data", map[string][]byte{
"file1.txt": []byte("in manifest"),
"file2.txt": []byte("extra file"),
})
chk, err := NewChecker("/manifest.mf", "/data", fs)
require.NoError(t, err)
results := make(chan Result, 10)
err = chk.FindExtraFiles(context.Background(), results)
require.NoError(t, err)
var extras []Result
for r := range results {
extras = append(extras, r)
}
assert.Len(t, extras, 1)
assert.Equal(t, RelFilePath("file2.txt"), extras[0].Path)
assert.Equal(t, StatusExtra, extras[0].Status)
assert.Equal(t, "not in manifest", extras[0].Message)
}
func TestFindExtraFilesSkipsManifestAndDotfiles(t *testing.T) {
fs := afero.NewMemMapFs()
manifestFiles := map[string][]byte{
"file1.txt": []byte("in manifest"),
}
createTestManifest(t, fs, "/data/.index.mf", manifestFiles)
createFilesOnDisk(t, fs, "/data", map[string][]byte{
"file1.txt": []byte("in manifest"),
})
// Create dotfile and manifest that should be skipped
require.NoError(t, afero.WriteFile(fs, "/data/.hidden", []byte("hidden"), 0o644))
require.NoError(t, afero.WriteFile(fs, "/data/.config/settings", []byte("cfg"), 0o644))
// Create a real extra file
require.NoError(t, fs.MkdirAll("/data", 0o755))
require.NoError(t, afero.WriteFile(fs, "/data/extra.txt", []byte("extra"), 0o644))
chk, err := NewChecker("/data/.index.mf", "/data", fs)
require.NoError(t, err)
results := make(chan Result, 10)
err = chk.FindExtraFiles(context.Background(), results)
require.NoError(t, err)
var extras []Result
for r := range results {
extras = append(extras, r)
}
// Should only report extra.txt, not .hidden, .config/settings, or .index.mf
for _, e := range extras {
t.Logf("extra: %s", e.Path)
}
assert.Len(t, extras, 1)
if len(extras) > 0 {
assert.Equal(t, RelFilePath("extra.txt"), extras[0].Path)
}
}
func TestFindExtraFilesContextCancellation(t *testing.T) {
fs := afero.NewMemMapFs()
files := map[string][]byte{"file.txt": []byte("data")}
createTestManifest(t, fs, "/manifest.mf", files)
createFilesOnDisk(t, fs, "/data", files)
chk, err := NewChecker("/manifest.mf", "/data", fs)
require.NoError(t, err)
ctx, cancel := context.WithCancel(context.Background())
cancel() // Cancel immediately
results := make(chan Result, 10)
err = chk.FindExtraFiles(ctx, results)
assert.ErrorIs(t, err, context.Canceled)
}
func TestCheckNilChannels(t *testing.T) {
fs := afero.NewMemMapFs()
files := map[string][]byte{"file.txt": []byte("data")}
createTestManifest(t, fs, "/manifest.mf", files)
createFilesOnDisk(t, fs, "/data", files)
chk, err := NewChecker("/manifest.mf", "/data", fs)
require.NoError(t, err)
// Should not panic with nil channels
err = chk.Check(context.Background(), nil, nil)
assert.NoError(t, err)
}
func TestFindExtraFilesNilChannel(t *testing.T) {
fs := afero.NewMemMapFs()
files := map[string][]byte{"file.txt": []byte("data")}
createTestManifest(t, fs, "/manifest.mf", files)
createFilesOnDisk(t, fs, "/data", files)
chk, err := NewChecker("/manifest.mf", "/data", fs)
require.NoError(t, err)
// Should not panic with nil channel
err = chk.FindExtraFiles(context.Background(), nil)
assert.NoError(t, err)
}
func TestCheckSubdirectories(t *testing.T) {
fs := afero.NewMemMapFs()
files := map[string][]byte{
"dir1/file1.txt": []byte("content1"),
"dir1/dir2/file2.txt": []byte("content2"),
"dir1/dir2/dir3/deep.txt": []byte("deep content"),
}
createTestManifest(t, fs, "/manifest.mf", files)
// Create files with full directory structure
for path, content := range files {
fullPath := "/data/" + path
require.NoError(t, fs.MkdirAll("/data/dir1/dir2/dir3", 0o755))
require.NoError(t, afero.WriteFile(fs, fullPath, content, 0o644))
}
chk, err := NewChecker("/manifest.mf", "/data", fs)
require.NoError(t, err)
results := make(chan Result, 10)
err = chk.Check(context.Background(), results, nil)
require.NoError(t, err)
var okCount int
for r := range results {
assert.Equal(t, StatusOK, r.Status, "file %s should be OK", r.Path)
okCount++
}
assert.Equal(t, 3, okCount)
}
func TestCheckMissingFileDetectedWithoutFallback(t *testing.T) {
// Regression test: errors.Is(err, errors.New("...")) never matches because
// errors.New creates a new value each time. The fix uses os.ErrNotExist instead.
fs := afero.NewMemMapFs()
files := map[string][]byte{
"exists.txt": []byte("here"),
"missing.txt": []byte("not on disk"),
}
createTestManifest(t, fs, "/manifest.mf", files)
// Only create one file on disk
createFilesOnDisk(t, fs, "/data", map[string][]byte{
"exists.txt": []byte("here"),
})
chk, err := NewChecker("/manifest.mf", "/data", fs)
require.NoError(t, err)
results := make(chan Result, 10)
err = chk.Check(context.Background(), results, nil)
require.NoError(t, err)
statusCounts := map[Status]int{}
for r := range results {
statusCounts[r.Status]++
if r.Status == StatusMissing {
assert.Equal(t, RelFilePath("missing.txt"), r.Path)
}
}
assert.Equal(t, 1, statusCounts[StatusOK], "one file should be OK")
assert.Equal(t, 1, statusCounts[StatusMissing], "one file should be MISSING")
assert.Equal(t, 0, statusCounts[StatusError], "no files should be ERROR")
}
func TestCheckEmptyManifest(t *testing.T) {
fs := afero.NewMemMapFs()
// Create manifest with no files
createTestManifest(t, fs, "/manifest.mf", map[string][]byte{})
chk, err := NewChecker("/manifest.mf", "/data", fs)
require.NoError(t, err)
assert.Equal(t, FileCount(0), chk.FileCount())
assert.Equal(t, FileSize(0), chk.TotalBytes())
results := make(chan Result, 10)
err = chk.Check(context.Background(), results, nil)
require.NoError(t, err)
var count int
for range results {
count++
}
assert.Equal(t, 0, count)
}

11
mfer/constants.go Normal file
View File

@@ -0,0 +1,11 @@
package mfer
const (
Version = "0.1.0"
ReleaseDate = "2025-12-17"
// MaxDecompressedSize is the maximum allowed size of decompressed manifest
// data (256 MB). This prevents decompression bombs from consuming excessive
// memory.
MaxDecompressedSize int64 = 256 * 1024 * 1024
)

172
mfer/deserialize.go Normal file
View File

@@ -0,0 +1,172 @@
package mfer
import (
"bytes"
"crypto/sha256"
"errors"
"fmt"
"io"
"github.com/google/uuid"
"github.com/klauspost/compress/zstd"
"github.com/spf13/afero"
"google.golang.org/protobuf/proto"
"sneak.berlin/go/mfer/internal/bork"
"sneak.berlin/go/mfer/internal/log"
)
// validateUUID checks that the byte slice is a valid UUID (16 bytes, parseable).
func validateUUID(data []byte) error {
if len(data) != 16 {
return errors.New("invalid UUID length")
}
// Try to parse as UUID to validate format
_, err := uuid.FromBytes(data)
if err != nil {
return errors.New("invalid UUID format")
}
return nil
}
func (m *manifest) deserializeInner() error {
if m.pbOuter.Version != MFFileOuter_VERSION_ONE {
return errors.New("unknown version")
}
if m.pbOuter.CompressionType != MFFileOuter_COMPRESSION_ZSTD {
return errors.New("unknown compression type")
}
// Validate outer UUID before any decompression
if err := validateUUID(m.pbOuter.Uuid); err != nil {
return errors.New("outer UUID invalid: " + err.Error())
}
// Verify hash of compressed data before decompression
h := sha256.New()
if _, err := h.Write(m.pbOuter.InnerMessage); err != nil {
return err
}
sha256Hash := h.Sum(nil)
if !bytes.Equal(sha256Hash, m.pbOuter.Sha256) {
return errors.New("compressed data hash mismatch")
}
// Verify signature if present
if len(m.pbOuter.Signature) > 0 {
if len(m.pbOuter.SigningPubKey) == 0 {
return errors.New("signature present but no public key")
}
sigString, err := m.signatureString()
if err != nil {
return fmt.Errorf("failed to generate signature string for verification: %w", err)
}
if err := gpgVerify([]byte(sigString), m.pbOuter.Signature, m.pbOuter.SigningPubKey); err != nil {
return fmt.Errorf("signature verification failed: %w", err)
}
log.Infof("signature verified successfully")
}
bb := bytes.NewBuffer(m.pbOuter.InnerMessage)
zr, err := zstd.NewReader(bb)
if err != nil {
return err
}
defer zr.Close()
// Limit decompressed size to prevent decompression bombs.
// Use declared size + 1 byte to detect overflow, capped at MaxDecompressedSize.
maxSize := MaxDecompressedSize
if m.pbOuter.Size > 0 && m.pbOuter.Size < int64(maxSize) {
maxSize = int64(m.pbOuter.Size) + 1
}
limitedReader := io.LimitReader(zr, maxSize)
dat, err := io.ReadAll(limitedReader)
if err != nil {
return err
}
if int64(len(dat)) >= MaxDecompressedSize {
return fmt.Errorf("decompressed data exceeds maximum allowed size of %d bytes", MaxDecompressedSize)
}
isize := len(dat)
if int64(isize) != m.pbOuter.Size {
log.Debugf("truncated data, got %d expected %d", isize, m.pbOuter.Size)
return bork.ErrFileTruncated
}
// Deserialize inner message
m.pbInner = new(MFFile)
if err := proto.Unmarshal(dat, m.pbInner); err != nil {
return err
}
// Validate inner UUID
if err := validateUUID(m.pbInner.Uuid); err != nil {
return errors.New("inner UUID invalid: " + err.Error())
}
// Verify UUIDs match
if !bytes.Equal(m.pbOuter.Uuid, m.pbInner.Uuid) {
return errors.New("outer and inner UUID mismatch")
}
log.Infof("loaded manifest with %d files", len(m.pbInner.Files))
return nil
}
func validateMagic(dat []byte) bool {
ml := len([]byte(MAGIC))
if len(dat) < ml {
return false
}
got := dat[0:ml]
expected := []byte(MAGIC)
return bytes.Equal(got, expected)
}
// NewManifestFromReader reads a manifest from an io.Reader.
func NewManifestFromReader(input io.Reader) (*manifest, error) {
m := &manifest{}
dat, err := io.ReadAll(input)
if err != nil {
return nil, err
}
if !validateMagic(dat) {
return nil, errors.New("invalid file format")
}
// remove magic bytes prefix:
ml := len([]byte(MAGIC))
bb := bytes.NewBuffer(dat[ml:])
dat = bb.Bytes()
// deserialize outer:
m.pbOuter = new(MFFileOuter)
if err := proto.Unmarshal(dat, m.pbOuter); err != nil {
return nil, err
}
// deserialize inner:
if err := m.deserializeInner(); err != nil {
return nil, err
}
return m, nil
}
// NewManifestFromFile reads a manifest from a file path using the given filesystem.
// If fs is nil, the real filesystem (OsFs) is used.
func NewManifestFromFile(fs afero.Fs, path string) (*manifest, error) {
if fs == nil {
fs = afero.NewOsFs()
}
f, err := fs.Open(path)
if err != nil {
return nil, err
}
defer func() { _ = f.Close() }()
return NewManifestFromReader(f)
}

212
mfer/gpg.go Normal file
View File

@@ -0,0 +1,212 @@
package mfer
import (
"bytes"
"fmt"
"os"
"os/exec"
"path/filepath"
"strings"
)
// GPGKeyID represents a GPG key identifier (fingerprint or key ID).
type GPGKeyID string
// SigningOptions contains options for GPG signing.
type SigningOptions struct {
KeyID GPGKeyID
}
// gpgSign creates a detached signature of the data using the specified key.
// Returns the armored detached signature.
func gpgSign(data []byte, keyID GPGKeyID) ([]byte, error) {
cmd := exec.Command("gpg",
"--detach-sign",
"--armor",
"--local-user", string(keyID),
)
cmd.Stdin = bytes.NewReader(data)
var stdout, stderr bytes.Buffer
cmd.Stdout = &stdout
cmd.Stderr = &stderr
if err := cmd.Run(); err != nil {
return nil, fmt.Errorf("gpg sign failed: %w: %s", err, stderr.String())
}
return stdout.Bytes(), nil
}
// gpgExportPublicKey exports the public key for the specified key ID.
// Returns the armored public key.
func gpgExportPublicKey(keyID GPGKeyID) ([]byte, error) {
cmd := exec.Command("gpg",
"--export",
"--armor",
string(keyID),
)
var stdout, stderr bytes.Buffer
cmd.Stdout = &stdout
cmd.Stderr = &stderr
if err := cmd.Run(); err != nil {
return nil, fmt.Errorf("gpg export failed: %w: %s", err, stderr.String())
}
if stdout.Len() == 0 {
return nil, fmt.Errorf("gpg key not found: %s", keyID)
}
return stdout.Bytes(), nil
}
// gpgGetKeyFingerprint gets the full fingerprint for a key ID.
func gpgGetKeyFingerprint(keyID GPGKeyID) ([]byte, error) {
cmd := exec.Command("gpg",
"--with-colons",
"--fingerprint",
string(keyID),
)
var stdout, stderr bytes.Buffer
cmd.Stdout = &stdout
cmd.Stderr = &stderr
if err := cmd.Run(); err != nil {
return nil, fmt.Errorf("gpg fingerprint lookup failed: %w: %s", err, stderr.String())
}
// Parse the colon-delimited output to find the fingerprint
lines := strings.Split(stdout.String(), "\n")
for _, line := range lines {
fields := strings.Split(line, ":")
if len(fields) >= 10 && fields[0] == "fpr" {
return []byte(fields[9]), nil
}
}
return nil, fmt.Errorf("fingerprint not found for key: %s", keyID)
}
// gpgExtractPubKeyFingerprint imports a public key into a temporary keyring
// and extracts its fingerprint. This verifies the key is valid and returns
// the actual fingerprint from the key material.
func gpgExtractPubKeyFingerprint(pubKey []byte) (string, error) {
// Create temporary directory for GPG operations
tmpDir, err := os.MkdirTemp("", "mfer-gpg-fingerprint-*")
if err != nil {
return "", fmt.Errorf("failed to create temp dir: %w", err)
}
defer func() { _ = os.RemoveAll(tmpDir) }()
// Set restrictive permissions
if err := os.Chmod(tmpDir, 0o700); err != nil {
return "", fmt.Errorf("failed to set temp dir permissions: %w", err)
}
// Write public key to temp file
pubKeyFile := filepath.Join(tmpDir, "pubkey.asc")
if err := os.WriteFile(pubKeyFile, pubKey, 0o600); err != nil {
return "", fmt.Errorf("failed to write public key: %w", err)
}
// Import the public key into the temporary keyring
importCmd := exec.Command("gpg",
"--homedir", tmpDir,
"--import",
pubKeyFile,
)
var importStderr bytes.Buffer
importCmd.Stderr = &importStderr
if err := importCmd.Run(); err != nil {
return "", fmt.Errorf("failed to import public key: %w: %s", err, importStderr.String())
}
// List keys to get fingerprint
listCmd := exec.Command("gpg",
"--homedir", tmpDir,
"--with-colons",
"--fingerprint",
)
var listStdout, listStderr bytes.Buffer
listCmd.Stdout = &listStdout
listCmd.Stderr = &listStderr
if err := listCmd.Run(); err != nil {
return "", fmt.Errorf("failed to list keys: %w: %s", err, listStderr.String())
}
// Parse the colon-delimited output to find the fingerprint
lines := strings.Split(listStdout.String(), "\n")
for _, line := range lines {
fields := strings.Split(line, ":")
if len(fields) >= 10 && fields[0] == "fpr" {
return fields[9], nil
}
}
return "", fmt.Errorf("fingerprint not found in imported key")
}
// gpgVerify verifies a detached signature against data using the provided public key.
// It creates a temporary keyring to import the public key for verification.
func gpgVerify(data, signature, pubKey []byte) error {
// Create temporary directory for GPG operations
tmpDir, err := os.MkdirTemp("", "mfer-gpg-verify-*")
if err != nil {
return fmt.Errorf("failed to create temp dir: %w", err)
}
defer func() { _ = os.RemoveAll(tmpDir) }()
// Set restrictive permissions
if err := os.Chmod(tmpDir, 0o700); err != nil {
return fmt.Errorf("failed to set temp dir permissions: %w", err)
}
// Write public key to temp file
pubKeyFile := filepath.Join(tmpDir, "pubkey.asc")
if err := os.WriteFile(pubKeyFile, pubKey, 0o600); err != nil {
return fmt.Errorf("failed to write public key: %w", err)
}
// Write signature to temp file
sigFile := filepath.Join(tmpDir, "signature.asc")
if err := os.WriteFile(sigFile, signature, 0o600); err != nil {
return fmt.Errorf("failed to write signature: %w", err)
}
// Write data to temp file
dataFile := filepath.Join(tmpDir, "data")
if err := os.WriteFile(dataFile, data, 0o600); err != nil {
return fmt.Errorf("failed to write data: %w", err)
}
// Import the public key into the temporary keyring
importCmd := exec.Command("gpg",
"--homedir", tmpDir,
"--import",
pubKeyFile,
)
var importStderr bytes.Buffer
importCmd.Stderr = &importStderr
if err := importCmd.Run(); err != nil {
return fmt.Errorf("failed to import public key: %w: %s", err, importStderr.String())
}
// Verify the signature
verifyCmd := exec.Command("gpg",
"--homedir", tmpDir,
"--verify",
sigFile,
dataFile,
)
var verifyStderr bytes.Buffer
verifyCmd.Stderr = &verifyStderr
if err := verifyCmd.Run(); err != nil {
return fmt.Errorf("signature verification failed: %w: %s", err, verifyStderr.String())
}
return nil
}

347
mfer/gpg_test.go Normal file
View File

@@ -0,0 +1,347 @@
package mfer
import (
"bytes"
"context"
"os"
"os/exec"
"path/filepath"
"strings"
"testing"
"github.com/spf13/afero"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
// testGPGEnv sets up a temporary GPG home directory with a test key.
// Returns the key ID and a cleanup function.
func testGPGEnv(t *testing.T) (GPGKeyID, func()) {
t.Helper()
// Check if gpg is installed
if _, err := exec.LookPath("gpg"); err != nil {
t.Skip("gpg not installed, skipping signing test")
return "", func() {}
}
// Create temporary GPG home directory
gpgHome, err := os.MkdirTemp("", "mfer-gpg-test-*")
require.NoError(t, err)
// Set restrictive permissions on GPG home
require.NoError(t, os.Chmod(gpgHome, 0o700))
// Save original GNUPGHOME and set new one
origGPGHome := os.Getenv("GNUPGHOME")
require.NoError(t, os.Setenv("GNUPGHOME", gpgHome))
cleanup := func() {
if origGPGHome == "" {
_ = os.Unsetenv("GNUPGHOME")
} else {
_ = os.Setenv("GNUPGHOME", origGPGHome)
}
_ = os.RemoveAll(gpgHome)
}
// Generate a test key with no passphrase
keyParams := `%no-protection
Key-Type: RSA
Key-Length: 2048
Name-Real: MFER Test Key
Name-Email: test@mfer.test
Expire-Date: 0
%commit
`
paramsFile := filepath.Join(gpgHome, "key-params")
require.NoError(t, os.WriteFile(paramsFile, []byte(keyParams), 0o600))
cmd := exec.Command("gpg", "--batch", "--gen-key", paramsFile)
cmd.Env = append(os.Environ(), "GNUPGHOME="+gpgHome)
output, err := cmd.CombinedOutput()
if err != nil {
cleanup()
t.Skipf("failed to generate test GPG key: %v: %s", err, output)
return "", func() {}
}
// Get the key fingerprint
cmd = exec.Command("gpg", "--list-keys", "--with-colons", "test@mfer.test")
cmd.Env = append(os.Environ(), "GNUPGHOME="+gpgHome)
output, err = cmd.Output()
if err != nil {
cleanup()
t.Fatalf("failed to list test key: %v", err)
}
// Parse fingerprint from output
var keyID string
for _, line := range strings.Split(string(output), "\n") {
fields := strings.Split(line, ":")
if len(fields) >= 10 && fields[0] == "fpr" {
keyID = fields[9]
break
}
}
if keyID == "" {
cleanup()
t.Fatal("failed to find test key fingerprint")
}
return GPGKeyID(keyID), cleanup
}
func TestGPGSign(t *testing.T) {
keyID, cleanup := testGPGEnv(t)
defer cleanup()
data := []byte("test data to sign")
sig, err := gpgSign(data, keyID)
require.NoError(t, err)
assert.NotEmpty(t, sig)
assert.Contains(t, string(sig), "-----BEGIN PGP SIGNATURE-----")
assert.Contains(t, string(sig), "-----END PGP SIGNATURE-----")
}
func TestGPGExportPublicKey(t *testing.T) {
keyID, cleanup := testGPGEnv(t)
defer cleanup()
pubKey, err := gpgExportPublicKey(keyID)
require.NoError(t, err)
assert.NotEmpty(t, pubKey)
assert.Contains(t, string(pubKey), "-----BEGIN PGP PUBLIC KEY BLOCK-----")
assert.Contains(t, string(pubKey), "-----END PGP PUBLIC KEY BLOCK-----")
}
func TestGPGGetKeyFingerprint(t *testing.T) {
keyID, cleanup := testGPGEnv(t)
defer cleanup()
fingerprint, err := gpgGetKeyFingerprint(keyID)
require.NoError(t, err)
assert.NotEmpty(t, fingerprint)
// The fingerprint should be 40 hex chars
assert.Len(t, fingerprint, 40, "fingerprint should be 40 hex chars")
}
func TestGPGSignInvalidKey(t *testing.T) {
// Set up test environment (we need GNUPGHOME set)
_, cleanup := testGPGEnv(t)
defer cleanup()
data := []byte("test data")
_, err := gpgSign(data, GPGKeyID("NONEXISTENT_KEY_ID_12345"))
assert.Error(t, err)
}
func TestBuilderWithSigning(t *testing.T) {
keyID, cleanup := testGPGEnv(t)
defer cleanup()
// Create a builder with signing options
b := NewBuilder()
b.SetSigningOptions(&SigningOptions{
KeyID: keyID,
})
// Add a test file
content := []byte("test file content")
reader := bytes.NewReader(content)
_, err := b.AddFile("test.txt", FileSize(len(content)), ModTime{}, reader, nil)
require.NoError(t, err)
// Build the manifest
var buf bytes.Buffer
err = b.Build(&buf)
require.NoError(t, err)
// Parse the manifest and verify signature fields are populated
manifest, err := NewManifestFromReader(&buf)
require.NoError(t, err)
require.NotNil(t, manifest.pbOuter)
assert.NotEmpty(t, manifest.pbOuter.Signature, "signature should be populated")
assert.NotEmpty(t, manifest.pbOuter.Signer, "signer should be populated")
assert.NotEmpty(t, manifest.pbOuter.SigningPubKey, "signing public key should be populated")
// Verify signature is a valid PGP signature
assert.Contains(t, string(manifest.pbOuter.Signature), "-----BEGIN PGP SIGNATURE-----")
// Verify public key is a valid PGP public key block
assert.Contains(t, string(manifest.pbOuter.SigningPubKey), "-----BEGIN PGP PUBLIC KEY BLOCK-----")
}
func TestScannerWithSigning(t *testing.T) {
keyID, cleanup := testGPGEnv(t)
defer cleanup()
// Create in-memory filesystem with test files
fs := afero.NewMemMapFs()
require.NoError(t, fs.MkdirAll("/testdir", 0o755))
require.NoError(t, afero.WriteFile(fs, "/testdir/file1.txt", []byte("content1"), 0o644))
require.NoError(t, afero.WriteFile(fs, "/testdir/file2.txt", []byte("content2"), 0o644))
// Create scanner with signing options
opts := &ScannerOptions{
Fs: fs,
SigningOptions: &SigningOptions{
KeyID: keyID,
},
}
s := NewScannerWithOptions(opts)
// Enumerate files
require.NoError(t, s.EnumeratePath("/testdir", nil))
assert.Equal(t, FileCount(2), s.FileCount())
// Generate signed manifest
var buf bytes.Buffer
require.NoError(t, s.ToManifest(context.Background(), &buf, nil))
// Parse and verify
manifest, err := NewManifestFromReader(&buf)
require.NoError(t, err)
assert.NotEmpty(t, manifest.pbOuter.Signature)
assert.NotEmpty(t, manifest.pbOuter.Signer)
assert.NotEmpty(t, manifest.pbOuter.SigningPubKey)
}
func TestGPGVerify(t *testing.T) {
keyID, cleanup := testGPGEnv(t)
defer cleanup()
data := []byte("test data to sign and verify")
sig, err := gpgSign(data, keyID)
require.NoError(t, err)
pubKey, err := gpgExportPublicKey(keyID)
require.NoError(t, err)
// Verify the signature
err = gpgVerify(data, sig, pubKey)
require.NoError(t, err)
}
func TestGPGVerifyInvalidSignature(t *testing.T) {
keyID, cleanup := testGPGEnv(t)
defer cleanup()
data := []byte("test data to sign")
sig, err := gpgSign(data, keyID)
require.NoError(t, err)
pubKey, err := gpgExportPublicKey(keyID)
require.NoError(t, err)
// Try to verify with different data - should fail
wrongData := []byte("different data")
err = gpgVerify(wrongData, sig, pubKey)
assert.Error(t, err)
}
func TestGPGVerifyBadPublicKey(t *testing.T) {
keyID, cleanup := testGPGEnv(t)
defer cleanup()
data := []byte("test data")
sig, err := gpgSign(data, keyID)
require.NoError(t, err)
// Try to verify with invalid public key - should fail
badPubKey := []byte("not a valid public key")
err = gpgVerify(data, sig, badPubKey)
assert.Error(t, err)
}
func TestManifestSignatureVerification(t *testing.T) {
keyID, cleanup := testGPGEnv(t)
defer cleanup()
// Create a builder with signing options
b := NewBuilder()
b.SetSigningOptions(&SigningOptions{
KeyID: keyID,
})
// Add a test file
content := []byte("test file content for verification")
reader := bytes.NewReader(content)
_, err := b.AddFile("test.txt", FileSize(len(content)), ModTime{}, reader, nil)
require.NoError(t, err)
// Build the manifest
var buf bytes.Buffer
err = b.Build(&buf)
require.NoError(t, err)
// Parse the manifest - signature should be verified during load
manifest, err := NewManifestFromReader(&buf)
require.NoError(t, err)
require.NotNil(t, manifest)
// Signature should be present and valid
assert.NotEmpty(t, manifest.pbOuter.Signature)
}
func TestManifestTamperedSignatureFails(t *testing.T) {
keyID, cleanup := testGPGEnv(t)
defer cleanup()
// Create a signed manifest
b := NewBuilder()
b.SetSigningOptions(&SigningOptions{
KeyID: keyID,
})
content := []byte("test file content")
reader := bytes.NewReader(content)
_, err := b.AddFile("test.txt", FileSize(len(content)), ModTime{}, reader, nil)
require.NoError(t, err)
var buf bytes.Buffer
err = b.Build(&buf)
require.NoError(t, err)
// Tamper with the signature by replacing some bytes
data := buf.Bytes()
// Find and modify a byte in the signature portion
for i := range data {
if i > 100 && data[i] == 'A' {
data[i] = 'B'
break
}
}
// Try to load the tampered manifest - should fail
_, err = NewManifestFromReader(bytes.NewReader(data))
assert.Error(t, err)
}
func TestBuilderWithoutSigning(t *testing.T) {
// Create a builder without signing options
b := NewBuilder()
// Add a test file
content := []byte("test file content")
reader := bytes.NewReader(content)
_, err := b.AddFile("test.txt", FileSize(len(content)), ModTime{}, reader, nil)
require.NoError(t, err)
// Build the manifest
var buf bytes.Buffer
err = b.Build(&buf)
require.NoError(t, err)
// Parse the manifest and verify signature fields are empty
manifest, err := NewManifestFromReader(&buf)
require.NoError(t, err)
require.NotNil(t, manifest.pbOuter)
assert.Empty(t, manifest.pbOuter.Signature, "signature should be empty when not signing")
assert.Empty(t, manifest.pbOuter.Signer, "signer should be empty when not signing")
assert.Empty(t, manifest.pbOuter.SigningPubKey, "signing public key should be empty when not signing")
}

59
mfer/manifest.go Normal file
View File

@@ -0,0 +1,59 @@
package mfer
import (
"bytes"
"encoding/hex"
"errors"
"fmt"
"github.com/multiformats/go-multihash"
)
// manifest holds the internal representation of a manifest file.
// Use NewManifestFromFile or NewManifestFromReader to load an existing manifest,
// or use Builder to create a new one.
type manifest struct {
pbInner *MFFile
pbOuter *MFFileOuter
output *bytes.Buffer
signingOptions *SigningOptions
}
func (m *manifest) String() string {
count := 0
if m.pbInner != nil {
count = len(m.pbInner.Files)
}
return fmt.Sprintf("<Manifest count=%d>", count)
}
// Files returns all file entries from a loaded manifest.
func (m *manifest) Files() []*MFFilePath {
if m.pbInner == nil {
return nil
}
return m.pbInner.Files
}
// signatureString generates the canonical string used for signing/verification.
// Format: MAGIC-UUID-MULTIHASH where UUID and multihash are hex-encoded.
// Requires pbOuter to be set with Uuid and Sha256 fields.
func (m *manifest) signatureString() (string, error) {
if m.pbOuter == nil {
return "", errors.New("pbOuter not set")
}
if len(m.pbOuter.Uuid) == 0 {
return "", errors.New("UUID not set")
}
if len(m.pbOuter.Sha256) == 0 {
return "", errors.New("SHA256 hash not set")
}
mh, err := multihash.Encode(m.pbOuter.Sha256, multihash.SHA2_256)
if err != nil {
return "", fmt.Errorf("failed to encode multihash: %w", err)
}
uuidStr := hex.EncodeToString(m.pbOuter.Uuid)
mhStr := hex.EncodeToString(mh)
return fmt.Sprintf("%s-%s-%s", MAGIC, uuidStr, mhStr), nil
}

3
mfer/mf.go Normal file
View File

@@ -0,0 +1,3 @@
package mfer
//go:generate protoc ./mf.proto --go_out=paths=source_relative:.

667
mfer/mf.pb.go Normal file
View File

@@ -0,0 +1,667 @@
// Code generated by protoc-gen-go. DO NOT EDIT.
// versions:
// protoc-gen-go v1.36.11
// protoc v6.33.0
// source: mf.proto
package mfer
import (
protoreflect "google.golang.org/protobuf/reflect/protoreflect"
protoimpl "google.golang.org/protobuf/runtime/protoimpl"
reflect "reflect"
sync "sync"
unsafe "unsafe"
)
const (
// Verify that this generated code is sufficiently up-to-date.
_ = protoimpl.EnforceVersion(20 - protoimpl.MinVersion)
// Verify that runtime/protoimpl is sufficiently up-to-date.
_ = protoimpl.EnforceVersion(protoimpl.MaxVersion - 20)
)
type MFFileOuter_Version int32
const (
MFFileOuter_VERSION_NONE MFFileOuter_Version = 0
MFFileOuter_VERSION_ONE MFFileOuter_Version = 1 // only one for now
)
// Enum value maps for MFFileOuter_Version.
var (
MFFileOuter_Version_name = map[int32]string{
0: "VERSION_NONE",
1: "VERSION_ONE",
}
MFFileOuter_Version_value = map[string]int32{
"VERSION_NONE": 0,
"VERSION_ONE": 1,
}
)
func (x MFFileOuter_Version) Enum() *MFFileOuter_Version {
p := new(MFFileOuter_Version)
*p = x
return p
}
func (x MFFileOuter_Version) String() string {
return protoimpl.X.EnumStringOf(x.Descriptor(), protoreflect.EnumNumber(x))
}
func (MFFileOuter_Version) Descriptor() protoreflect.EnumDescriptor {
return file_mf_proto_enumTypes[0].Descriptor()
}
func (MFFileOuter_Version) Type() protoreflect.EnumType {
return &file_mf_proto_enumTypes[0]
}
func (x MFFileOuter_Version) Number() protoreflect.EnumNumber {
return protoreflect.EnumNumber(x)
}
// Deprecated: Use MFFileOuter_Version.Descriptor instead.
func (MFFileOuter_Version) EnumDescriptor() ([]byte, []int) {
return file_mf_proto_rawDescGZIP(), []int{1, 0}
}
type MFFileOuter_CompressionType int32
const (
MFFileOuter_COMPRESSION_NONE MFFileOuter_CompressionType = 0
MFFileOuter_COMPRESSION_ZSTD MFFileOuter_CompressionType = 1
)
// Enum value maps for MFFileOuter_CompressionType.
var (
MFFileOuter_CompressionType_name = map[int32]string{
0: "COMPRESSION_NONE",
1: "COMPRESSION_ZSTD",
}
MFFileOuter_CompressionType_value = map[string]int32{
"COMPRESSION_NONE": 0,
"COMPRESSION_ZSTD": 1,
}
)
func (x MFFileOuter_CompressionType) Enum() *MFFileOuter_CompressionType {
p := new(MFFileOuter_CompressionType)
*p = x
return p
}
func (x MFFileOuter_CompressionType) String() string {
return protoimpl.X.EnumStringOf(x.Descriptor(), protoreflect.EnumNumber(x))
}
func (MFFileOuter_CompressionType) Descriptor() protoreflect.EnumDescriptor {
return file_mf_proto_enumTypes[1].Descriptor()
}
func (MFFileOuter_CompressionType) Type() protoreflect.EnumType {
return &file_mf_proto_enumTypes[1]
}
func (x MFFileOuter_CompressionType) Number() protoreflect.EnumNumber {
return protoreflect.EnumNumber(x)
}
// Deprecated: Use MFFileOuter_CompressionType.Descriptor instead.
func (MFFileOuter_CompressionType) EnumDescriptor() ([]byte, []int) {
return file_mf_proto_rawDescGZIP(), []int{1, 1}
}
type MFFile_Version int32
const (
MFFile_VERSION_NONE MFFile_Version = 0
MFFile_VERSION_ONE MFFile_Version = 1 // only one for now
)
// Enum value maps for MFFile_Version.
var (
MFFile_Version_name = map[int32]string{
0: "VERSION_NONE",
1: "VERSION_ONE",
}
MFFile_Version_value = map[string]int32{
"VERSION_NONE": 0,
"VERSION_ONE": 1,
}
)
func (x MFFile_Version) Enum() *MFFile_Version {
p := new(MFFile_Version)
*p = x
return p
}
func (x MFFile_Version) String() string {
return protoimpl.X.EnumStringOf(x.Descriptor(), protoreflect.EnumNumber(x))
}
func (MFFile_Version) Descriptor() protoreflect.EnumDescriptor {
return file_mf_proto_enumTypes[2].Descriptor()
}
func (MFFile_Version) Type() protoreflect.EnumType {
return &file_mf_proto_enumTypes[2]
}
func (x MFFile_Version) Number() protoreflect.EnumNumber {
return protoreflect.EnumNumber(x)
}
// Deprecated: Use MFFile_Version.Descriptor instead.
func (MFFile_Version) EnumDescriptor() ([]byte, []int) {
return file_mf_proto_rawDescGZIP(), []int{4, 0}
}
type Timestamp struct {
state protoimpl.MessageState `protogen:"open.v1"`
Seconds int64 `protobuf:"varint,1,opt,name=seconds,proto3" json:"seconds,omitempty"`
Nanos int32 `protobuf:"varint,2,opt,name=nanos,proto3" json:"nanos,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
func (x *Timestamp) Reset() {
*x = Timestamp{}
mi := &file_mf_proto_msgTypes[0]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
func (x *Timestamp) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*Timestamp) ProtoMessage() {}
func (x *Timestamp) ProtoReflect() protoreflect.Message {
mi := &file_mf_proto_msgTypes[0]
if x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use Timestamp.ProtoReflect.Descriptor instead.
func (*Timestamp) Descriptor() ([]byte, []int) {
return file_mf_proto_rawDescGZIP(), []int{0}
}
func (x *Timestamp) GetSeconds() int64 {
if x != nil {
return x.Seconds
}
return 0
}
func (x *Timestamp) GetNanos() int32 {
if x != nil {
return x.Nanos
}
return 0
}
type MFFileOuter struct {
state protoimpl.MessageState `protogen:"open.v1"`
// required mffile root attributes 1xx
Version MFFileOuter_Version `protobuf:"varint,101,opt,name=version,proto3,enum=MFFileOuter_Version" json:"version,omitempty"`
CompressionType MFFileOuter_CompressionType `protobuf:"varint,102,opt,name=compressionType,proto3,enum=MFFileOuter_CompressionType" json:"compressionType,omitempty"`
// these are used solely to detect corruption/truncation
// and not for cryptographic integrity.
Size int64 `protobuf:"varint,103,opt,name=size,proto3" json:"size,omitempty"`
Sha256 []byte `protobuf:"bytes,104,opt,name=sha256,proto3" json:"sha256,omitempty"`
// uuid must match the uuid in the inner message
Uuid []byte `protobuf:"bytes,105,opt,name=uuid,proto3" json:"uuid,omitempty"`
InnerMessage []byte `protobuf:"bytes,199,opt,name=innerMessage,proto3" json:"innerMessage,omitempty"`
// detached signature, ascii or binary
Signature []byte `protobuf:"bytes,201,opt,name=signature,proto3,oneof" json:"signature,omitempty"`
// full GPG key id
Signer []byte `protobuf:"bytes,202,opt,name=signer,proto3,oneof" json:"signer,omitempty"`
// full GPG signing public key, ascii or binary
SigningPubKey []byte `protobuf:"bytes,203,opt,name=signingPubKey,proto3,oneof" json:"signingPubKey,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
func (x *MFFileOuter) Reset() {
*x = MFFileOuter{}
mi := &file_mf_proto_msgTypes[1]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
func (x *MFFileOuter) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*MFFileOuter) ProtoMessage() {}
func (x *MFFileOuter) ProtoReflect() protoreflect.Message {
mi := &file_mf_proto_msgTypes[1]
if x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use MFFileOuter.ProtoReflect.Descriptor instead.
func (*MFFileOuter) Descriptor() ([]byte, []int) {
return file_mf_proto_rawDescGZIP(), []int{1}
}
func (x *MFFileOuter) GetVersion() MFFileOuter_Version {
if x != nil {
return x.Version
}
return MFFileOuter_VERSION_NONE
}
func (x *MFFileOuter) GetCompressionType() MFFileOuter_CompressionType {
if x != nil {
return x.CompressionType
}
return MFFileOuter_COMPRESSION_NONE
}
func (x *MFFileOuter) GetSize() int64 {
if x != nil {
return x.Size
}
return 0
}
func (x *MFFileOuter) GetSha256() []byte {
if x != nil {
return x.Sha256
}
return nil
}
func (x *MFFileOuter) GetUuid() []byte {
if x != nil {
return x.Uuid
}
return nil
}
func (x *MFFileOuter) GetInnerMessage() []byte {
if x != nil {
return x.InnerMessage
}
return nil
}
func (x *MFFileOuter) GetSignature() []byte {
if x != nil {
return x.Signature
}
return nil
}
func (x *MFFileOuter) GetSigner() []byte {
if x != nil {
return x.Signer
}
return nil
}
func (x *MFFileOuter) GetSigningPubKey() []byte {
if x != nil {
return x.SigningPubKey
}
return nil
}
type MFFilePath struct {
state protoimpl.MessageState `protogen:"open.v1"`
// required attributes:
Path string `protobuf:"bytes,1,opt,name=path,proto3" json:"path,omitempty"`
Size int64 `protobuf:"varint,2,opt,name=size,proto3" json:"size,omitempty"`
// gotta have at least one:
Hashes []*MFFileChecksum `protobuf:"bytes,3,rep,name=hashes,proto3" json:"hashes,omitempty"`
// optional per-file metadata
MimeType *string `protobuf:"bytes,301,opt,name=mimeType,proto3,oneof" json:"mimeType,omitempty"`
Mtime *Timestamp `protobuf:"bytes,302,opt,name=mtime,proto3,oneof" json:"mtime,omitempty"`
Ctime *Timestamp `protobuf:"bytes,303,opt,name=ctime,proto3,oneof" json:"ctime,omitempty"`
Atime *Timestamp `protobuf:"bytes,304,opt,name=atime,proto3,oneof" json:"atime,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
func (x *MFFilePath) Reset() {
*x = MFFilePath{}
mi := &file_mf_proto_msgTypes[2]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
func (x *MFFilePath) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*MFFilePath) ProtoMessage() {}
func (x *MFFilePath) ProtoReflect() protoreflect.Message {
mi := &file_mf_proto_msgTypes[2]
if x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use MFFilePath.ProtoReflect.Descriptor instead.
func (*MFFilePath) Descriptor() ([]byte, []int) {
return file_mf_proto_rawDescGZIP(), []int{2}
}
func (x *MFFilePath) GetPath() string {
if x != nil {
return x.Path
}
return ""
}
func (x *MFFilePath) GetSize() int64 {
if x != nil {
return x.Size
}
return 0
}
func (x *MFFilePath) GetHashes() []*MFFileChecksum {
if x != nil {
return x.Hashes
}
return nil
}
func (x *MFFilePath) GetMimeType() string {
if x != nil && x.MimeType != nil {
return *x.MimeType
}
return ""
}
func (x *MFFilePath) GetMtime() *Timestamp {
if x != nil {
return x.Mtime
}
return nil
}
func (x *MFFilePath) GetCtime() *Timestamp {
if x != nil {
return x.Ctime
}
return nil
}
func (x *MFFilePath) GetAtime() *Timestamp {
if x != nil {
return x.Atime
}
return nil
}
type MFFileChecksum struct {
state protoimpl.MessageState `protogen:"open.v1"`
// 1.0 golang implementation must write a multihash here
// it's ok to only ever use/verify sha256 multihash
MultiHash []byte `protobuf:"bytes,1,opt,name=multiHash,proto3" json:"multiHash,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
func (x *MFFileChecksum) Reset() {
*x = MFFileChecksum{}
mi := &file_mf_proto_msgTypes[3]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
func (x *MFFileChecksum) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*MFFileChecksum) ProtoMessage() {}
func (x *MFFileChecksum) ProtoReflect() protoreflect.Message {
mi := &file_mf_proto_msgTypes[3]
if x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use MFFileChecksum.ProtoReflect.Descriptor instead.
func (*MFFileChecksum) Descriptor() ([]byte, []int) {
return file_mf_proto_rawDescGZIP(), []int{3}
}
func (x *MFFileChecksum) GetMultiHash() []byte {
if x != nil {
return x.MultiHash
}
return nil
}
type MFFile struct {
state protoimpl.MessageState `protogen:"open.v1"`
Version MFFile_Version `protobuf:"varint,100,opt,name=version,proto3,enum=MFFile_Version" json:"version,omitempty"`
// required manifest attributes:
Files []*MFFilePath `protobuf:"bytes,101,rep,name=files,proto3" json:"files,omitempty"`
// uuid is a random v4 UUID generated when creating the manifest
// used as part of the signature to prevent replay attacks
Uuid []byte `protobuf:"bytes,102,opt,name=uuid,proto3" json:"uuid,omitempty"`
// optional manifest attributes 2xx:
CreatedAt *Timestamp `protobuf:"bytes,201,opt,name=createdAt,proto3,oneof" json:"createdAt,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
func (x *MFFile) Reset() {
*x = MFFile{}
mi := &file_mf_proto_msgTypes[4]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
func (x *MFFile) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*MFFile) ProtoMessage() {}
func (x *MFFile) ProtoReflect() protoreflect.Message {
mi := &file_mf_proto_msgTypes[4]
if x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use MFFile.ProtoReflect.Descriptor instead.
func (*MFFile) Descriptor() ([]byte, []int) {
return file_mf_proto_rawDescGZIP(), []int{4}
}
func (x *MFFile) GetVersion() MFFile_Version {
if x != nil {
return x.Version
}
return MFFile_VERSION_NONE
}
func (x *MFFile) GetFiles() []*MFFilePath {
if x != nil {
return x.Files
}
return nil
}
func (x *MFFile) GetUuid() []byte {
if x != nil {
return x.Uuid
}
return nil
}
func (x *MFFile) GetCreatedAt() *Timestamp {
if x != nil {
return x.CreatedAt
}
return nil
}
var File_mf_proto protoreflect.FileDescriptor
const file_mf_proto_rawDesc = "" +
"\n" +
"\bmf.proto\";\n" +
"\tTimestamp\x12\x18\n" +
"\aseconds\x18\x01 \x01(\x03R\aseconds\x12\x14\n" +
"\x05nanos\x18\x02 \x01(\x05R\x05nanos\"\xf0\x03\n" +
"\vMFFileOuter\x12.\n" +
"\aversion\x18e \x01(\x0e2\x14.MFFileOuter.VersionR\aversion\x12F\n" +
"\x0fcompressionType\x18f \x01(\x0e2\x1c.MFFileOuter.CompressionTypeR\x0fcompressionType\x12\x12\n" +
"\x04size\x18g \x01(\x03R\x04size\x12\x16\n" +
"\x06sha256\x18h \x01(\fR\x06sha256\x12\x12\n" +
"\x04uuid\x18i \x01(\fR\x04uuid\x12#\n" +
"\finnerMessage\x18\xc7\x01 \x01(\fR\finnerMessage\x12\"\n" +
"\tsignature\x18\xc9\x01 \x01(\fH\x00R\tsignature\x88\x01\x01\x12\x1c\n" +
"\x06signer\x18\xca\x01 \x01(\fH\x01R\x06signer\x88\x01\x01\x12*\n" +
"\rsigningPubKey\x18\xcb\x01 \x01(\fH\x02R\rsigningPubKey\x88\x01\x01\",\n" +
"\aVersion\x12\x10\n" +
"\fVERSION_NONE\x10\x00\x12\x0f\n" +
"\vVERSION_ONE\x10\x01\"=\n" +
"\x0fCompressionType\x12\x14\n" +
"\x10COMPRESSION_NONE\x10\x00\x12\x14\n" +
"\x10COMPRESSION_ZSTD\x10\x01B\f\n" +
"\n" +
"_signatureB\t\n" +
"\a_signerB\x10\n" +
"\x0e_signingPubKey\"\xa2\x02\n" +
"\n" +
"MFFilePath\x12\x12\n" +
"\x04path\x18\x01 \x01(\tR\x04path\x12\x12\n" +
"\x04size\x18\x02 \x01(\x03R\x04size\x12'\n" +
"\x06hashes\x18\x03 \x03(\v2\x0f.MFFileChecksumR\x06hashes\x12 \n" +
"\bmimeType\x18\xad\x02 \x01(\tH\x00R\bmimeType\x88\x01\x01\x12&\n" +
"\x05mtime\x18\xae\x02 \x01(\v2\n" +
".TimestampH\x01R\x05mtime\x88\x01\x01\x12&\n" +
"\x05ctime\x18\xaf\x02 \x01(\v2\n" +
".TimestampH\x02R\x05ctime\x88\x01\x01\x12&\n" +
"\x05atime\x18\xb0\x02 \x01(\v2\n" +
".TimestampH\x03R\x05atime\x88\x01\x01B\v\n" +
"\t_mimeTypeB\b\n" +
"\x06_mtimeB\b\n" +
"\x06_ctimeB\b\n" +
"\x06_atime\".\n" +
"\x0eMFFileChecksum\x12\x1c\n" +
"\tmultiHash\x18\x01 \x01(\fR\tmultiHash\"\xd6\x01\n" +
"\x06MFFile\x12)\n" +
"\aversion\x18d \x01(\x0e2\x0f.MFFile.VersionR\aversion\x12!\n" +
"\x05files\x18e \x03(\v2\v.MFFilePathR\x05files\x12\x12\n" +
"\x04uuid\x18f \x01(\fR\x04uuid\x12.\n" +
"\tcreatedAt\x18\xc9\x01 \x01(\v2\n" +
".TimestampH\x00R\tcreatedAt\x88\x01\x01\",\n" +
"\aVersion\x12\x10\n" +
"\fVERSION_NONE\x10\x00\x12\x0f\n" +
"\vVERSION_ONE\x10\x01B\f\n" +
"\n" +
"_createdAtB\x1dZ\x1bgit.eeqj.de/sneak/mfer/mferb\x06proto3"
var (
file_mf_proto_rawDescOnce sync.Once
file_mf_proto_rawDescData []byte
)
func file_mf_proto_rawDescGZIP() []byte {
file_mf_proto_rawDescOnce.Do(func() {
file_mf_proto_rawDescData = protoimpl.X.CompressGZIP(unsafe.Slice(unsafe.StringData(file_mf_proto_rawDesc), len(file_mf_proto_rawDesc)))
})
return file_mf_proto_rawDescData
}
var file_mf_proto_enumTypes = make([]protoimpl.EnumInfo, 3)
var file_mf_proto_msgTypes = make([]protoimpl.MessageInfo, 5)
var file_mf_proto_goTypes = []any{
(MFFileOuter_Version)(0), // 0: MFFileOuter.Version
(MFFileOuter_CompressionType)(0), // 1: MFFileOuter.CompressionType
(MFFile_Version)(0), // 2: MFFile.Version
(*Timestamp)(nil), // 3: Timestamp
(*MFFileOuter)(nil), // 4: MFFileOuter
(*MFFilePath)(nil), // 5: MFFilePath
(*MFFileChecksum)(nil), // 6: MFFileChecksum
(*MFFile)(nil), // 7: MFFile
}
var file_mf_proto_depIdxs = []int32{
0, // 0: MFFileOuter.version:type_name -> MFFileOuter.Version
1, // 1: MFFileOuter.compressionType:type_name -> MFFileOuter.CompressionType
6, // 2: MFFilePath.hashes:type_name -> MFFileChecksum
3, // 3: MFFilePath.mtime:type_name -> Timestamp
3, // 4: MFFilePath.ctime:type_name -> Timestamp
3, // 5: MFFilePath.atime:type_name -> Timestamp
2, // 6: MFFile.version:type_name -> MFFile.Version
5, // 7: MFFile.files:type_name -> MFFilePath
3, // 8: MFFile.createdAt:type_name -> Timestamp
9, // [9:9] is the sub-list for method output_type
9, // [9:9] is the sub-list for method input_type
9, // [9:9] is the sub-list for extension type_name
9, // [9:9] is the sub-list for extension extendee
0, // [0:9] is the sub-list for field type_name
}
func init() { file_mf_proto_init() }
func file_mf_proto_init() {
if File_mf_proto != nil {
return
}
file_mf_proto_msgTypes[1].OneofWrappers = []any{}
file_mf_proto_msgTypes[2].OneofWrappers = []any{}
file_mf_proto_msgTypes[4].OneofWrappers = []any{}
type x struct{}
out := protoimpl.TypeBuilder{
File: protoimpl.DescBuilder{
GoPackagePath: reflect.TypeOf(x{}).PkgPath(),
RawDescriptor: unsafe.Slice(unsafe.StringData(file_mf_proto_rawDesc), len(file_mf_proto_rawDesc)),
NumEnums: 3,
NumMessages: 5,
NumExtensions: 0,
NumServices: 0,
},
GoTypes: file_mf_proto_goTypes,
DependencyIndexes: file_mf_proto_depIdxs,
EnumInfos: file_mf_proto_enumTypes,
MessageInfos: file_mf_proto_msgTypes,
}.Build()
File_mf_proto = out.File
file_mf_proto_goTypes = nil
file_mf_proto_depIdxs = nil
}

88
mfer/mf.proto Normal file
View File

@@ -0,0 +1,88 @@
syntax = "proto3";
option go_package = "git.eeqj.de/sneak/mfer/mfer";
message Timestamp {
int64 seconds = 1;
int32 nanos = 2;
}
message MFFileOuter {
enum Version {
VERSION_NONE = 0;
VERSION_ONE = 1; // only one for now
}
// required mffile root attributes 1xx
Version version = 101;
enum CompressionType {
COMPRESSION_NONE = 0;
COMPRESSION_ZSTD = 1;
}
CompressionType compressionType = 102;
// these are used solely to detect corruption/truncation
// and not for cryptographic integrity.
int64 size = 103;
bytes sha256 = 104;
// uuid must match the uuid in the inner message
bytes uuid = 105;
bytes innerMessage = 199;
// 2xx for optional manifest root attributes
// think we might use gosignify instead of gpg:
// github.com/frankbraun/gosignify
//detached signature, ascii or binary
optional bytes signature = 201;
//full GPG key id
optional bytes signer = 202;
//full GPG signing public key, ascii or binary
optional bytes signingPubKey = 203;
}
message MFFilePath {
// required attributes:
// Path invariants: must be valid UTF-8, use forward slashes only,
// be relative (no leading /), contain no ".." segments, and no
// empty segments (no "//").
string path = 1;
int64 size = 2;
// gotta have at least one:
repeated MFFileChecksum hashes = 3;
// optional per-file metadata
optional string mimeType = 301;
optional Timestamp mtime = 302;
optional Timestamp ctime = 303;
optional Timestamp atime = 304;
}
message MFFileChecksum {
// 1.0 golang implementation must write a multihash here
// it's ok to only ever use/verify sha256 multihash
bytes multiHash = 1;
}
message MFFile {
enum Version {
VERSION_NONE = 0;
VERSION_ONE = 1; // only one for now
}
Version version = 100;
// required manifest attributes:
repeated MFFilePath files = 101;
// uuid is a random v4 UUID generated when creating the manifest
// used as part of the signature to prevent replay attacks
bytes uuid = 102;
// optional manifest attributes 2xx:
optional Timestamp createdAt = 201;
}

430
mfer/scanner.go Normal file
View File

@@ -0,0 +1,430 @@
package mfer
import (
"context"
"io"
"io/fs"
"path"
"path/filepath"
"strings"
"sync"
"time"
"github.com/dustin/go-humanize"
"github.com/spf13/afero"
"sneak.berlin/go/mfer/internal/log"
)
// Phase 1: Enumeration
// ---------------------
// Walking directories and calling stat() on files to collect metadata.
// Builds the list of files to be scanned. Relatively fast (metadata only).
// EnumerateStatus contains progress information for the enumeration phase.
type EnumerateStatus struct {
FilesFound FileCount // Number of files discovered so far
BytesFound FileSize // Total size of discovered files (from stat)
}
// Phase 2: Scan (ToManifest)
// --------------------------
// Reading file contents and computing hashes for manifest generation.
// This is the expensive phase that reads all file data.
// ScanStatus contains progress information for the scan phase.
type ScanStatus struct {
TotalFiles FileCount // Total number of files to scan
ScannedFiles FileCount // Number of files scanned so far
TotalBytes FileSize // Total bytes to read (sum of all file sizes)
ScannedBytes FileSize // Bytes read so far
BytesPerSec float64 // Current throughput rate
ETA time.Duration // Estimated time to completion
}
// ScannerOptions configures scanner behavior.
type ScannerOptions struct {
IncludeDotfiles bool // Include files and directories starting with a dot (default: exclude)
FollowSymLinks bool // Resolve symlinks instead of skipping them
Fs afero.Fs // Filesystem to use, defaults to OsFs if nil
SigningOptions *SigningOptions // GPG signing options (nil = no signing)
}
// FileEntry represents a file that has been enumerated.
type FileEntry struct {
Path RelFilePath // Relative path (used in manifest)
AbsPath AbsFilePath // Absolute path (used for reading file content)
Size FileSize // File size in bytes
Mtime ModTime // Last modification time
Ctime time.Time // Creation time (platform-dependent)
}
// Scanner accumulates files and generates manifests from them.
type Scanner struct {
mu sync.RWMutex
files []*FileEntry
totalBytes FileSize // cached sum of all file sizes
options *ScannerOptions
fs afero.Fs
}
// NewScanner creates a new Scanner with default options.
func NewScanner() *Scanner {
return NewScannerWithOptions(nil)
}
// NewScannerWithOptions creates a new Scanner with the given options.
func NewScannerWithOptions(opts *ScannerOptions) *Scanner {
if opts == nil {
opts = &ScannerOptions{}
}
fs := opts.Fs
if fs == nil {
fs = afero.NewOsFs()
}
return &Scanner{
files: make([]*FileEntry, 0),
options: opts,
fs: fs,
}
}
// EnumerateFile adds a single file to the scanner, calling stat() to get metadata.
func (s *Scanner) EnumerateFile(filePath string) error {
abs, err := filepath.Abs(filePath)
if err != nil {
return err
}
info, err := s.fs.Stat(abs)
if err != nil {
return err
}
// For single files, use the filename as the relative path
basePath := filepath.Dir(abs)
return s.enumerateFileWithInfo(filepath.Base(abs), basePath, info, nil)
}
// EnumeratePath walks a directory path and adds all files to the scanner.
// If progress is non-nil, status updates are sent as files are discovered.
// The progress channel is closed when the method returns.
func (s *Scanner) EnumeratePath(inputPath string, progress chan<- EnumerateStatus) error {
if progress != nil {
defer close(progress)
}
abs, err := filepath.Abs(inputPath)
if err != nil {
return err
}
afs := afero.NewReadOnlyFs(afero.NewBasePathFs(s.fs, abs))
return s.enumerateFS(afs, abs, progress)
}
// EnumeratePaths walks multiple directory paths and adds all files to the scanner.
// If progress is non-nil, status updates are sent as files are discovered.
// The progress channel is closed when the method returns.
func (s *Scanner) EnumeratePaths(progress chan<- EnumerateStatus, inputPaths ...string) error {
if progress != nil {
defer close(progress)
}
for _, p := range inputPaths {
abs, err := filepath.Abs(p)
if err != nil {
return err
}
afs := afero.NewReadOnlyFs(afero.NewBasePathFs(s.fs, abs))
if err := s.enumerateFS(afs, abs, progress); err != nil {
return err
}
}
return nil
}
// EnumerateFS walks an afero filesystem and adds all files to the scanner.
// If progress is non-nil, status updates are sent as files are discovered.
// The progress channel is closed when the method returns.
// basePath is used to compute absolute paths for file reading.
func (s *Scanner) EnumerateFS(afs afero.Fs, basePath string, progress chan<- EnumerateStatus) error {
if progress != nil {
defer close(progress)
}
return s.enumerateFS(afs, basePath, progress)
}
// enumerateFS is the internal implementation that doesn't close the progress channel.
func (s *Scanner) enumerateFS(afs afero.Fs, basePath string, progress chan<- EnumerateStatus) error {
return afero.Walk(afs, "/", func(p string, info fs.FileInfo, err error) error {
if err != nil {
return err
}
if !s.options.IncludeDotfiles && IsHiddenPath(p) {
if info.IsDir() {
return filepath.SkipDir
}
return nil
}
return s.enumerateFileWithInfo(p, basePath, info, progress)
})
}
// enumerateFileWithInfo adds a file with pre-existing fs.FileInfo.
func (s *Scanner) enumerateFileWithInfo(filePath string, basePath string, info fs.FileInfo, progress chan<- EnumerateStatus) error {
if info.IsDir() {
// Manifests contain only files, directories are implied
return nil
}
// Clean the path - remove leading slash if present
cleanPath := filePath
if len(cleanPath) > 0 && cleanPath[0] == '/' {
cleanPath = cleanPath[1:]
}
// Compute absolute path for file reading
absPath := filepath.Join(basePath, cleanPath)
// Handle symlinks
if info.Mode()&fs.ModeSymlink != 0 {
if !s.options.FollowSymLinks {
// Skip symlinks when not following them
return nil
}
// Resolve symlink to get real file info
realPath, err := filepath.EvalSymlinks(absPath)
if err != nil {
// Skip broken symlinks
return nil
}
realInfo, err := s.fs.Stat(realPath)
if err != nil {
return nil
}
// Skip if symlink points to a directory
if realInfo.IsDir() {
return nil
}
// Use resolved path for reading, but keep original path in manifest
absPath = realPath
info = realInfo
}
entry := &FileEntry{
Path: RelFilePath(cleanPath),
AbsPath: AbsFilePath(absPath),
Size: FileSize(info.Size()),
Mtime: ModTime(info.ModTime()),
// Note: Ctime not available from fs.FileInfo on all platforms
// Will need platform-specific code to extract it
}
s.mu.Lock()
s.files = append(s.files, entry)
s.totalBytes += entry.Size
filesFound := FileCount(len(s.files))
bytesFound := s.totalBytes
s.mu.Unlock()
sendEnumerateStatus(progress, EnumerateStatus{
FilesFound: filesFound,
BytesFound: bytesFound,
})
return nil
}
// Files returns a copy of all files added to the scanner.
func (s *Scanner) Files() []*FileEntry {
s.mu.RLock()
defer s.mu.RUnlock()
out := make([]*FileEntry, len(s.files))
copy(out, s.files)
return out
}
// FileCount returns the number of files in the scanner.
func (s *Scanner) FileCount() FileCount {
s.mu.RLock()
defer s.mu.RUnlock()
return FileCount(len(s.files))
}
// TotalBytes returns the total size of all files in the scanner.
func (s *Scanner) TotalBytes() FileSize {
s.mu.RLock()
defer s.mu.RUnlock()
return s.totalBytes
}
// ToManifest reads all file contents, computes hashes, and generates a manifest.
// If progress is non-nil, status updates are sent approximately once per second.
// The progress channel is closed when the method returns.
// The manifest is written to the provided io.Writer.
func (s *Scanner) ToManifest(ctx context.Context, w io.Writer, progress chan<- ScanStatus) error {
if progress != nil {
defer close(progress)
}
s.mu.RLock()
files := make([]*FileEntry, len(s.files))
copy(files, s.files)
totalFiles := FileCount(len(files))
var totalBytes FileSize
for _, f := range files {
totalBytes += f.Size
}
s.mu.RUnlock()
builder := NewBuilder()
if s.options.SigningOptions != nil {
builder.SetSigningOptions(s.options.SigningOptions)
}
var scannedFiles FileCount
var scannedBytes FileSize
lastProgressTime := time.Now()
startTime := time.Now()
for _, entry := range files {
// Check for cancellation
select {
case <-ctx.Done():
return ctx.Err()
default:
}
// Open file
f, err := s.fs.Open(string(entry.AbsPath))
if err != nil {
return err
}
// Create progress channel for this file
var fileProgress chan FileHashProgress
var wg sync.WaitGroup
if progress != nil {
fileProgress = make(chan FileHashProgress, 1)
wg.Add(1)
go func(baseScannedBytes FileSize) {
defer wg.Done()
for p := range fileProgress {
// Send progress at most once per second
now := time.Now()
if now.Sub(lastProgressTime) >= time.Second {
elapsed := now.Sub(startTime).Seconds()
currentBytes := baseScannedBytes + p.BytesRead
var rate float64
var eta time.Duration
if elapsed > 0 && currentBytes > 0 {
rate = float64(currentBytes) / elapsed
remainingBytes := totalBytes - currentBytes
if rate > 0 {
eta = time.Duration(float64(remainingBytes)/rate) * time.Second
}
}
sendScanStatus(progress, ScanStatus{
TotalFiles: totalFiles,
ScannedFiles: scannedFiles,
TotalBytes: totalBytes,
ScannedBytes: currentBytes,
BytesPerSec: rate,
ETA: eta,
})
lastProgressTime = now
}
}
}(scannedBytes)
}
// Add to manifest with progress channel
bytesRead, err := builder.AddFile(
entry.Path,
entry.Size,
entry.Mtime,
f,
fileProgress,
)
_ = f.Close()
// Close channel and wait for goroutine to finish
if fileProgress != nil {
close(fileProgress)
wg.Wait()
}
if err != nil {
return err
}
log.Verbosef("+ %s (%s)", entry.Path, humanize.IBytes(uint64(bytesRead)))
scannedFiles++
scannedBytes += bytesRead
}
// Send final progress (ETA is 0 at completion)
if progress != nil {
elapsed := time.Since(startTime).Seconds()
var rate float64
if elapsed > 0 {
rate = float64(scannedBytes) / elapsed
}
sendScanStatus(progress, ScanStatus{
TotalFiles: totalFiles,
ScannedFiles: scannedFiles,
TotalBytes: totalBytes,
ScannedBytes: scannedBytes,
BytesPerSec: rate,
ETA: 0,
})
}
// Build and write manifest
return builder.Build(w)
}
// IsHiddenPath returns true if the path or any of its parent directories
// start with a dot (hidden files/directories).
// The path should use forward slashes.
func IsHiddenPath(p string) bool {
tp := path.Clean(p)
if tp == "." || tp == "/" {
return false
}
if strings.HasPrefix(tp, ".") {
return true
}
for {
d, f := path.Split(tp)
if strings.HasPrefix(f, ".") {
return true
}
if d == "" {
return false
}
tp = d[0 : len(d)-1] // trim trailing slash from dir
}
}
// sendEnumerateStatus sends a status update without blocking.
// If the channel is full, the update is dropped.
func sendEnumerateStatus(ch chan<- EnumerateStatus, status EnumerateStatus) {
if ch == nil {
return
}
select {
case ch <- status:
default:
// Channel full, drop this update
}
}
// sendScanStatus sends a status update without blocking.
// If the channel is full, the update is dropped.
func sendScanStatus(ch chan<- ScanStatus, status ScanStatus) {
if ch == nil {
return
}
select {
case ch <- status:
default:
// Channel full, drop this update
}
}

364
mfer/scanner_test.go Normal file
View File

@@ -0,0 +1,364 @@
package mfer
import (
"bytes"
"context"
"testing"
"time"
"github.com/spf13/afero"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestNewScanner(t *testing.T) {
s := NewScanner()
assert.NotNil(t, s)
assert.Equal(t, FileCount(0), s.FileCount())
assert.Equal(t, FileSize(0), s.TotalBytes())
}
func TestNewScannerWithOptions(t *testing.T) {
t.Run("nil options", func(t *testing.T) {
s := NewScannerWithOptions(nil)
assert.NotNil(t, s)
})
t.Run("with options", func(t *testing.T) {
fs := afero.NewMemMapFs()
opts := &ScannerOptions{
IncludeDotfiles: true,
FollowSymLinks: true,
Fs: fs,
}
s := NewScannerWithOptions(opts)
assert.NotNil(t, s)
})
}
func TestScannerEnumerateFile(t *testing.T) {
fs := afero.NewMemMapFs()
require.NoError(t, afero.WriteFile(fs, "/test.txt", []byte("hello world"), 0o644))
s := NewScannerWithOptions(&ScannerOptions{Fs: fs})
err := s.EnumerateFile("/test.txt")
require.NoError(t, err)
assert.Equal(t, FileCount(1), s.FileCount())
assert.Equal(t, FileSize(11), s.TotalBytes())
files := s.Files()
require.Len(t, files, 1)
assert.Equal(t, RelFilePath("test.txt"), files[0].Path)
assert.Equal(t, FileSize(11), files[0].Size)
}
func TestScannerEnumerateFileMissing(t *testing.T) {
fs := afero.NewMemMapFs()
s := NewScannerWithOptions(&ScannerOptions{Fs: fs})
err := s.EnumerateFile("/nonexistent.txt")
assert.Error(t, err)
}
func TestScannerEnumeratePath(t *testing.T) {
fs := afero.NewMemMapFs()
require.NoError(t, fs.MkdirAll("/testdir/subdir", 0o755))
require.NoError(t, afero.WriteFile(fs, "/testdir/file1.txt", []byte("one"), 0o644))
require.NoError(t, afero.WriteFile(fs, "/testdir/file2.txt", []byte("two"), 0o644))
require.NoError(t, afero.WriteFile(fs, "/testdir/subdir/file3.txt", []byte("three"), 0o644))
s := NewScannerWithOptions(&ScannerOptions{Fs: fs})
err := s.EnumeratePath("/testdir", nil)
require.NoError(t, err)
assert.Equal(t, FileCount(3), s.FileCount())
assert.Equal(t, FileSize(3+3+5), s.TotalBytes())
}
func TestScannerEnumeratePathWithProgress(t *testing.T) {
fs := afero.NewMemMapFs()
require.NoError(t, fs.MkdirAll("/testdir", 0o755))
require.NoError(t, afero.WriteFile(fs, "/testdir/file1.txt", []byte("one"), 0o644))
require.NoError(t, afero.WriteFile(fs, "/testdir/file2.txt", []byte("two"), 0o644))
s := NewScannerWithOptions(&ScannerOptions{Fs: fs})
progress := make(chan EnumerateStatus, 10)
err := s.EnumeratePath("/testdir", progress)
require.NoError(t, err)
var updates []EnumerateStatus
for p := range progress {
updates = append(updates, p)
}
assert.NotEmpty(t, updates)
// Final update should show all files
final := updates[len(updates)-1]
assert.Equal(t, FileCount(2), final.FilesFound)
assert.Equal(t, FileSize(6), final.BytesFound)
}
func TestScannerEnumeratePaths(t *testing.T) {
fs := afero.NewMemMapFs()
require.NoError(t, fs.MkdirAll("/dir1", 0o755))
require.NoError(t, fs.MkdirAll("/dir2", 0o755))
require.NoError(t, afero.WriteFile(fs, "/dir1/a.txt", []byte("aaa"), 0o644))
require.NoError(t, afero.WriteFile(fs, "/dir2/b.txt", []byte("bbb"), 0o644))
s := NewScannerWithOptions(&ScannerOptions{Fs: fs})
err := s.EnumeratePaths(nil, "/dir1", "/dir2")
require.NoError(t, err)
assert.Equal(t, FileCount(2), s.FileCount())
}
func TestScannerExcludeDotfiles(t *testing.T) {
fs := afero.NewMemMapFs()
require.NoError(t, fs.MkdirAll("/testdir/.hidden", 0o755))
require.NoError(t, afero.WriteFile(fs, "/testdir/visible.txt", []byte("visible"), 0o644))
require.NoError(t, afero.WriteFile(fs, "/testdir/.hidden.txt", []byte("hidden"), 0o644))
require.NoError(t, afero.WriteFile(fs, "/testdir/.hidden/inside.txt", []byte("inside"), 0o644))
t.Run("exclude by default", func(t *testing.T) {
s := NewScannerWithOptions(&ScannerOptions{Fs: fs, IncludeDotfiles: false})
err := s.EnumeratePath("/testdir", nil)
require.NoError(t, err)
assert.Equal(t, FileCount(1), s.FileCount())
files := s.Files()
assert.Equal(t, RelFilePath("visible.txt"), files[0].Path)
})
t.Run("include when enabled", func(t *testing.T) {
s := NewScannerWithOptions(&ScannerOptions{Fs: fs, IncludeDotfiles: true})
err := s.EnumeratePath("/testdir", nil)
require.NoError(t, err)
assert.Equal(t, FileCount(3), s.FileCount())
})
}
func TestScannerToManifest(t *testing.T) {
fs := afero.NewMemMapFs()
require.NoError(t, fs.MkdirAll("/testdir", 0o755))
require.NoError(t, afero.WriteFile(fs, "/testdir/file1.txt", []byte("content one"), 0o644))
require.NoError(t, afero.WriteFile(fs, "/testdir/file2.txt", []byte("content two"), 0o644))
s := NewScannerWithOptions(&ScannerOptions{Fs: fs})
err := s.EnumeratePath("/testdir", nil)
require.NoError(t, err)
var buf bytes.Buffer
err = s.ToManifest(context.Background(), &buf, nil)
require.NoError(t, err)
// Manifest should have magic bytes
assert.True(t, buf.Len() > 0)
assert.Equal(t, MAGIC, string(buf.Bytes()[:8]))
}
func TestScannerToManifestWithProgress(t *testing.T) {
fs := afero.NewMemMapFs()
require.NoError(t, fs.MkdirAll("/testdir", 0o755))
require.NoError(t, afero.WriteFile(fs, "/testdir/file.txt", bytes.Repeat([]byte("x"), 1000), 0o644))
s := NewScannerWithOptions(&ScannerOptions{Fs: fs})
err := s.EnumeratePath("/testdir", nil)
require.NoError(t, err)
var buf bytes.Buffer
progress := make(chan ScanStatus, 10)
err = s.ToManifest(context.Background(), &buf, progress)
require.NoError(t, err)
var updates []ScanStatus
for p := range progress {
updates = append(updates, p)
}
assert.NotEmpty(t, updates)
// Final update should show completion
final := updates[len(updates)-1]
assert.Equal(t, FileCount(1), final.TotalFiles)
assert.Equal(t, FileCount(1), final.ScannedFiles)
assert.Equal(t, FileSize(1000), final.TotalBytes)
assert.Equal(t, FileSize(1000), final.ScannedBytes)
}
func TestScannerToManifestContextCancellation(t *testing.T) {
fs := afero.NewMemMapFs()
require.NoError(t, fs.MkdirAll("/testdir", 0o755))
// Create many files to ensure we have time to cancel
for i := 0; i < 100; i++ {
name := string(rune('a'+i%26)) + string(rune('0'+i/26)) + ".txt"
require.NoError(t, afero.WriteFile(fs, "/testdir/"+name, bytes.Repeat([]byte("x"), 100), 0o644))
}
s := NewScannerWithOptions(&ScannerOptions{Fs: fs})
err := s.EnumeratePath("/testdir", nil)
require.NoError(t, err)
ctx, cancel := context.WithCancel(context.Background())
cancel() // Cancel immediately
var buf bytes.Buffer
err = s.ToManifest(ctx, &buf, nil)
assert.ErrorIs(t, err, context.Canceled)
}
func TestScannerToManifestEmptyScanner(t *testing.T) {
fs := afero.NewMemMapFs()
s := NewScannerWithOptions(&ScannerOptions{Fs: fs})
var buf bytes.Buffer
err := s.ToManifest(context.Background(), &buf, nil)
require.NoError(t, err)
// Should still produce a valid manifest
assert.True(t, buf.Len() > 0)
assert.Equal(t, MAGIC, string(buf.Bytes()[:8]))
}
func TestScannerFilesCopiesSlice(t *testing.T) {
fs := afero.NewMemMapFs()
require.NoError(t, afero.WriteFile(fs, "/test.txt", []byte("hello"), 0o644))
s := NewScannerWithOptions(&ScannerOptions{Fs: fs})
require.NoError(t, s.EnumerateFile("/test.txt"))
files1 := s.Files()
files2 := s.Files()
// Should be different slices
assert.NotSame(t, &files1[0], &files2[0])
}
func TestScannerEnumerateFS(t *testing.T) {
fs := afero.NewMemMapFs()
require.NoError(t, fs.MkdirAll("/testdir/sub", 0o755))
require.NoError(t, afero.WriteFile(fs, "/testdir/file.txt", []byte("hello"), 0o644))
require.NoError(t, afero.WriteFile(fs, "/testdir/sub/nested.txt", []byte("world"), 0o644))
// Create a basepath filesystem
baseFs := afero.NewBasePathFs(fs, "/testdir")
s := NewScannerWithOptions(&ScannerOptions{Fs: fs})
err := s.EnumerateFS(baseFs, "/testdir", nil)
require.NoError(t, err)
assert.Equal(t, FileCount(2), s.FileCount())
}
func TestSendEnumerateStatusNonBlocking(t *testing.T) {
// Channel with no buffer - send should not block
ch := make(chan EnumerateStatus)
// This should not block
done := make(chan bool)
go func() {
sendEnumerateStatus(ch, EnumerateStatus{FilesFound: 1})
done <- true
}()
select {
case <-done:
// Success - did not block
case <-time.After(100 * time.Millisecond):
t.Fatal("sendEnumerateStatus blocked on full channel")
}
}
func TestSendScanStatusNonBlocking(t *testing.T) {
// Channel with no buffer - send should not block
ch := make(chan ScanStatus)
done := make(chan bool)
go func() {
sendScanStatus(ch, ScanStatus{ScannedFiles: 1})
done <- true
}()
select {
case <-done:
// Success - did not block
case <-time.After(100 * time.Millisecond):
t.Fatal("sendScanStatus blocked on full channel")
}
}
func TestSendStatusNilChannel(t *testing.T) {
// Should not panic with nil channel
sendEnumerateStatus(nil, EnumerateStatus{})
sendScanStatus(nil, ScanStatus{})
}
func TestScannerFileEntryFields(t *testing.T) {
fs := afero.NewMemMapFs()
now := time.Now().Truncate(time.Second)
require.NoError(t, afero.WriteFile(fs, "/test.txt", []byte("content"), 0o644))
require.NoError(t, fs.Chtimes("/test.txt", now, now))
s := NewScannerWithOptions(&ScannerOptions{Fs: fs})
require.NoError(t, s.EnumerateFile("/test.txt"))
files := s.Files()
require.Len(t, files, 1)
entry := files[0]
assert.Equal(t, RelFilePath("test.txt"), entry.Path)
assert.Contains(t, string(entry.AbsPath), "test.txt")
assert.Equal(t, FileSize(7), entry.Size)
// Mtime should be set (within a second of now)
assert.WithinDuration(t, now, time.Time(entry.Mtime), 2*time.Second)
}
func TestScannerLargeFileEnumeration(t *testing.T) {
fs := afero.NewMemMapFs()
require.NoError(t, fs.MkdirAll("/testdir", 0o755))
// Create 100 files
for i := 0; i < 100; i++ {
name := "/testdir/" + string(rune('a'+i%26)) + string(rune('0'+i/26%10)) + ".txt"
require.NoError(t, afero.WriteFile(fs, name, []byte("data"), 0o644))
}
s := NewScannerWithOptions(&ScannerOptions{Fs: fs})
progress := make(chan EnumerateStatus, 200)
err := s.EnumeratePath("/testdir", progress)
require.NoError(t, err)
// Drain channel
for range progress {
}
assert.Equal(t, FileCount(100), s.FileCount())
assert.Equal(t, FileSize(400), s.TotalBytes()) // 100 * 4 bytes
}
func TestIsHiddenPath(t *testing.T) {
tests := []struct {
path string
hidden bool
}{
{"file.txt", false},
{".hidden", true},
{"dir/file.txt", false},
{"dir/.hidden", true},
{".dir/file.txt", true},
{"/absolute/path", false},
{"/absolute/.hidden", true},
{"./relative", false}, // path.Clean removes leading ./
{"a/b/c/.d/e", true},
{".", false}, // current directory is not hidden
{"/", false}, // root is not hidden
}
for _, tt := range tests {
t.Run(tt.path, func(t *testing.T) {
assert.Equal(t, tt.hidden, IsHiddenPath(tt.path), "IsHiddenPath(%q)", tt.path)
})
}
}

118
mfer/serialize.go Normal file
View File

@@ -0,0 +1,118 @@
package mfer
import (
"bytes"
"crypto/sha256"
"errors"
"fmt"
"time"
"github.com/google/uuid"
"github.com/klauspost/compress/zstd"
"google.golang.org/protobuf/proto"
)
// MAGIC is the file format magic bytes prefix (rot13 of "MANIFEST").
const MAGIC string = "ZNAVSRFG"
func newTimestampFromTime(t time.Time) *Timestamp {
return &Timestamp{
Seconds: t.Unix(),
Nanos: int32(t.Nanosecond()),
}
}
func (m *manifest) generate() error {
if m.pbInner == nil {
return errors.New("internal error: pbInner not set")
}
if m.pbOuter == nil {
e := m.generateOuter()
if e != nil {
return e
}
}
dat, err := proto.MarshalOptions{Deterministic: true}.Marshal(m.pbOuter)
if err != nil {
return err
}
m.output = bytes.NewBuffer([]byte(MAGIC))
_, err = m.output.Write(dat)
if err != nil {
return err
}
return nil
}
func (m *manifest) generateOuter() error {
if m.pbInner == nil {
return errors.New("internal error")
}
// Generate UUID and set on inner message
manifestUUID := uuid.New()
m.pbInner.Uuid = manifestUUID[:]
innerData, err := proto.MarshalOptions{Deterministic: true}.Marshal(m.pbInner)
if err != nil {
return err
}
// Compress the inner data
idc := new(bytes.Buffer)
zw, err := zstd.NewWriter(idc, zstd.WithEncoderLevel(zstd.SpeedBestCompression))
if err != nil {
return err
}
_, err = zw.Write(innerData)
if err != nil {
return err
}
_ = zw.Close()
compressedData := idc.Bytes()
// Hash the compressed data for integrity verification before decompression
h := sha256.New()
if _, err := h.Write(compressedData); err != nil {
return err
}
sha256Hash := h.Sum(nil)
m.pbOuter = &MFFileOuter{
InnerMessage: compressedData,
Size: int64(len(innerData)),
Sha256: sha256Hash,
Uuid: manifestUUID[:],
Version: MFFileOuter_VERSION_ONE,
CompressionType: MFFileOuter_COMPRESSION_ZSTD,
}
// Sign the manifest if signing options are provided
if m.signingOptions != nil && m.signingOptions.KeyID != "" {
sigString, err := m.signatureString()
if err != nil {
return fmt.Errorf("failed to generate signature string: %w", err)
}
sig, err := gpgSign([]byte(sigString), m.signingOptions.KeyID)
if err != nil {
return fmt.Errorf("failed to sign manifest: %w", err)
}
m.pbOuter.Signature = sig
fingerprint, err := gpgGetKeyFingerprint(m.signingOptions.KeyID)
if err != nil {
return fmt.Errorf("failed to get key fingerprint: %w", err)
}
m.pbOuter.Signer = fingerprint
pubKey, err := gpgExportPublicKey(m.signingOptions.KeyID)
if err != nil {
return fmt.Errorf("failed to export public key: %w", err)
}
m.pbOuter.SigningPubKey = pubKey
}
return nil
}

53
mfer/url.go Normal file
View File

@@ -0,0 +1,53 @@
package mfer
import (
"net/url"
"strings"
)
// ManifestURL represents a URL pointing to a manifest file.
type ManifestURL string
// FileURL represents a URL pointing to a file to be fetched.
type FileURL string
// BaseURL represents a base URL for constructing file URLs.
type BaseURL string
// JoinPath safely joins a relative file path to a base URL.
// The path is properly URL-encoded to prevent path traversal.
func (b BaseURL) JoinPath(path RelFilePath) (FileURL, error) {
base, err := url.Parse(string(b))
if err != nil {
return "", err
}
// Ensure base path ends with /
if !strings.HasSuffix(base.Path, "/") {
base.Path += "/"
}
// Parse and encode the relative path
ref, err := url.Parse(url.PathEscape(string(path)))
if err != nil {
return "", err
}
resolved := base.ResolveReference(ref)
return FileURL(resolved.String()), nil
}
// String returns the URL as a string.
func (b BaseURL) String() string {
return string(b)
}
// String returns the URL as a string.
func (f FileURL) String() string {
return string(f)
}
// String returns the URL as a string.
func (m ManifestURL) String() string {
return string(m)
}

BIN
modcache.tzst Normal file

Binary file not shown.

View File

@@ -1,3 +0,0 @@
package mfer
//go:generate protoc --go_out=. mf.proto

View File

@@ -1,69 +0,0 @@
syntax = "proto3";
option go_package = "git.eeqj.de/sneak/mfer/mfer";
import "google/protobuf/timestamp.proto";
message MFFile {
enum Version {
NONE = 0;
ONE = 1; // only one for now
}
// required mffile root attributes 1xx
Version version = 101;
bytes innerMessage = 102;
// these are used solely to detect corruption/truncation
// and not for cryptographic integrity.
uint64 size = 103;
bytes sha256 = 104;
// 2xx for optional manifest root attributes
// think we might use gosignify instead of gpg:
// github.com/frankbraun/gosignify
//detached signature, ascii or binary
bytes signature = 201;
//full GPG key id
bytes signer = 202;
//full GPG signing public key, ascii or binary
bytes signingPubKey = 203;
}
message MFFilePath {
// required attributes:
string path = 101;
uint64 size = 102;
// gotta have at least one:
repeated MFFileChecksum hashes = 201;
// optional per-file metadata
string mimeType = 301;
google.protobuf.Timestamp mtime = 302;
google.protobuf.Timestamp ctime = 303;
google.protobuf.Timestamp atime = 304;
}
message MFFileChecksum {
// 1.0 golang implementation must write a multihash here
// it's ok to only ever use/verify sha256 multihash
bytes multiHash = 1;
}
message MFFileInner {
enum Version {
NONE = 0;
ONE = 1; // only one for now
}
Version version = 101;
// required manifest attributes:
uint64 fileCount = 102; //FIXME is this necessary?
repeated MFFilePath files = 103;
// optional manifest attributes 2xx:
google.protobuf.Timestamp createdAt = 201;
}

View File

@@ -1,89 +0,0 @@
package mfer
import (
"fmt"
"log"
"os"
"path/filepath"
"github.com/pterm/pterm"
"github.com/urfave/cli/v2"
)
func printBanner(Appname, Version, Buildarch string) {
s, _ := pterm.DefaultBigText.WithLetters(pterm.NewLettersFromString(Appname)).Srender()
pterm.DefaultCenter.Println(s) // Print BigLetters with the default CenterPrinter
}
func disableStyling() {
pterm.DisableColor()
pterm.DisableStyling()
pterm.Debug.Prefix.Text = ""
pterm.Info.Prefix.Text = ""
pterm.Success.Prefix.Text = ""
pterm.Warning.Prefix.Text = ""
pterm.Error.Prefix.Text = ""
pterm.Fatal.Prefix.Text = ""
}
func init() {
if _, exists := os.LookupEnv("NO_COLOR"); exists {
disableStyling()
}
}
func Run(Appname, Version, Buildarch string) int {
printBanner(Appname, Version, Buildarch)
app := &cli.App{
Name: Appname,
Usage: "Manifest generator",
Version: Version,
EnableBashCompletion: true,
Commands: []*cli.Command{
{
Name: "generate",
Aliases: []string{"gen"},
Usage: "Generate manifest file",
Action: func(c *cli.Context) error {
return generateManifest(c.String("input"), c.String("output"))
},
Flags: []cli.Flag{
&cli.StringFlag{
Name: "input",
Value: ".",
Aliases: []string{"i"},
Usage: "Specify input directory.",
},
&cli.StringFlag{
Name: "output",
Value: "./index.mf",
Aliases: []string{"o"},
Usage: "Specify output filename",
},
},
},
},
}
err := app.Run(os.Args)
if err != nil {
log.Fatal(err)
return 1
}
return 0
}
func generateManifest(path, outputFilename string) error {
fmt.Println("generateManifest()")
walkErr := filepath.Walk(path, func(itemPath string, info os.FileInfo, err error) error {
fmt.Println(itemPath)
return nil
})
if walkErr != nil {
log.Fatal(walkErr)
return walkErr
}
return nil
}

BIN
vendor.tzst Normal file

Binary file not shown.

View File

@@ -1,31 +0,0 @@
### Go template
# Binaries for programs and plugins
*.exe
*.exe~
*.dll
*.so
*.dylib
# Test binary, built with `go test -c`
*.test
# Output of the go coverage tool, specifically when used with LiteIDE
*.out
# Dependency directories (remove the comment below to include it)
vendor/
### IntelliJ
.idea
*.iml
out
gen
### VisualStudioCode
.vscode
*.code-workspace
### macOS
# General
.DS_Store
experimenting

View File

@@ -1,71 +0,0 @@
linters-settings:
gocritic:
enabled-tags:
- diagnostic
- experimental
- opinionated
- performance
- style
disabled-checks:
- dupImport
- ifElseChain
- octalLiteral
- whyNoLint
- wrapperFunc
- exitAfterDefer
- hugeParam
- ptrToRefParam
- paramTypeCombine
- unnamedResult
misspell:
locale: US
linters:
disable-all: true
enable:
- errcheck
- gosimple
- govet
- ineffassign
- staticcheck
- asciicheck
- bodyclose
- dupl
- durationcheck
- errorlint
- exhaustive
- gci
- gocognit
- gocritic
- godot
- godox
- goerr113
- gofmt
- goimports
- goprintffuncname
- misspell
- nilerr
- nlreturn
- noctx
- prealloc
- predeclared
- thelper
- unconvert
- unparam
- wastedassign
- wrapcheck
issues:
# Excluding configuration per-path, per-linter, per-text and per-source
exclude-rules:
- path: _test\.go
linters:
- errcheck
- dupl
- gocritic
- wrapcheck
- goerr113
# https://github.com/go-critic/go-critic/issues/926
- linters:
- gocritic
text: "unnecessaryDefer:"
service:
golangci-lint-version: 1.39.x # use the fixed version to not introduce new linters unexpectedly

View File

View File

@@ -1,21 +0,0 @@
MIT License
Copyright (c) 2020 Marvin Wendt (MarvinJWendt)
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

View File

@@ -1,237 +0,0 @@
<h1 align="center">AtomicGo | cursor</h1>
<p align="center">
<a href="https://github.com/atomicgo/cursor/releases">
<img src="https://img.shields.io/github/v/release/atomicgo/cursor?style=flat-square" alt="Latest Release">
</a>
<a href="https://codecov.io/gh/atomicgo/cursor" target="_blank">
<img src="https://img.shields.io/github/workflow/status/atomicgo/cursor/Go?label=tests&style=flat-square" alt="Tests">
</a>
<a href="https://codecov.io/gh/atomicgo/cursor" target="_blank">
<img src="https://img.shields.io/codecov/c/gh/atomicgo/cursor?color=magenta&logo=codecov&style=flat-square" alt="Coverage">
</a>
<a href="https://codecov.io/gh/atomicgo/cursor">
<!-- unittestcount:start --><img src="https://img.shields.io/badge/Unit_Tests-2-magenta?style=flat-square" alt="Unit test count"><!-- unittestcount:end -->
</a>
<a href="https://github.com/atomicgo/cursor/issues">
<img src="https://img.shields.io/github/issues/atomicgo/cursor.svg?style=flat-square" alt="Issues">
</a>
<a href="https://opensource.org/licenses/MIT" target="_blank">
<img src="https://img.shields.io/badge/License-MIT-yellow.svg?style=flat-square" alt="License: MIT">
</a>
</p>
---
<p align="center">
<strong><a href="#install">Get The Module</a></strong>
|
<strong><a href="https://pkg.go.dev/github.com/atomicgo/cursor#section-documentation" target="_blank">Documentation</a></strong>
|
<strong><a href="https://github.com/atomicgo/atomicgo/blob/main/CONTRIBUTING.md" target="_blank">Contributing</a></strong>
|
<strong><a href="https://github.com/atomicgo/atomicgo/blob/main/CODE_OF_CONDUCT.md" target="_blank">Code of Conduct</a></strong>
</p>
---
<p align="center">
<img src="https://raw.githubusercontent.com/atomicgo/atomicgo/main/assets/header.png" alt="AtomicGo">
</p>
## Description
Package cursor contains cross-platform methods to move the terminal cursor in
different directions. This package can be used to create interactive CLI tools
and games, live charts, algorithm visualizations and other updatable output of
any kind.
Special thanks to github.com/k0kubun/go-ansi which this project is based on.
## Install
```console
# Execute this command inside your project
go get -u github.com/atomicgo/cursor
```
```go
// Add this to your imports
import "github.com/atomicgo/cursor"
```
## Usage
#### func Bottom
```go
func Bottom()
```
Bottom moves the cursor to the bottom of the terminal. This is done by
calculating how many lines were moved by Up and Down.
#### func ClearLine
```go
func ClearLine()
```
ClearLine clears the current line and moves the cursor to it's start position.
#### func ClearLinesDown
```go
func ClearLinesDown(n int)
```
ClearLinesDown clears n lines downwards from the current position and moves the
cursor.
#### func ClearLinesUp
```go
func ClearLinesUp(n int)
```
ClearLinesUp clears n lines upwards from the current position and moves the
cursor.
#### func Down
```go
func Down(n int)
```
Down moves the cursor n lines down relative to the current position.
#### func DownAndClear
```go
func DownAndClear(n int)
```
DownAndClear moves the cursor down by n lines, then clears the line.
#### func Hide
```go
func Hide()
```
Hide the cursor. Don't forget to show the cursor at least at the end of your
application with Show. Otherwise the user might have a terminal with a
permanently hidden cursor, until he reopens the terminal.
#### func HorizontalAbsolute
```go
func HorizontalAbsolute(n int)
```
HorizontalAbsolute moves the cursor to n horizontally. The position n is
absolute to the start of the line.
#### func Left
```go
func Left(n int)
```
Left moves the cursor n characters to the left relative to the current position.
#### func Move
```go
func Move(x, y int)
```
Move moves the cursor relative by x and y.
#### func Right
```go
func Right(n int)
```
Right moves the cursor n characters to the right relative to the current
position.
#### func Show
```go
func Show()
```
Show the cursor if it was hidden previously. Don't forget to show the cursor at
least at the end of your application. Otherwise the user might have a terminal
with a permanently hidden cursor, until he reopens the terminal.
#### func StartOfLine
```go
func StartOfLine()
```
StartOfLine moves the cursor to the start of the current line.
#### func StartOfLineDown
```go
func StartOfLineDown(n int)
```
StartOfLineDown moves the cursor down by n lines, then moves to cursor to the
start of the line.
#### func StartOfLineUp
```go
func StartOfLineUp(n int)
```
StartOfLineUp moves the cursor up by n lines, then moves to cursor to the start
of the line.
#### func Up
```go
func Up(n int)
```
Up moves the cursor n lines up relative to the current position.
#### func UpAndClear
```go
func UpAndClear(n int)
```
UpAndClear moves the cursor up by n lines, then clears the line.
#### type Area
```go
type Area struct {
}
```
Area displays content which can be updated on the fly. You can use this to
create live output, charts, dropdowns, etc.
#### func NewArea
```go
func NewArea() Area
```
NewArea returns a new Area.
#### func (*Area) Clear
```go
func (area *Area) Clear()
```
Clear clears the content of the Area.
#### func (*Area) Update
```go
func (area *Area) Update(content string)
```
Update overwrites the content of the Area.
---
> [AtomicGo.dev](https://atomicgo.dev) &nbsp;&middot;&nbsp;
> with ❤️ by [@MarvinJWendt](https://github.com/MarvinJWendt) |
> [MarvinJWendt.com](https://marvinjwendt.com)

View File

@@ -1,45 +0,0 @@
package cursor
import (
"fmt"
"runtime"
"strings"
)
// Area displays content which can be updated on the fly.
// You can use this to create live output, charts, dropdowns, etc.
type Area struct {
height int
}
// NewArea returns a new Area.
func NewArea() Area {
return Area{}
}
// Clear clears the content of the Area.
func (area *Area) Clear() {
Bottom()
if area.height > 0 {
ClearLinesUp(area.height)
}
}
// Update overwrites the content of the Area.
func (area *Area) Update(content string) {
area.Clear()
lines := strings.Split(content, "\n")
if runtime.GOOS == "windows" {
for _, line := range lines {
fmt.Print(line)
StartOfLineDown(1)
}
} else {
for _, line := range lines {
fmt.Println(line)
}
}
height = 0
area.height = len(lines)
}

View File

@@ -1,59 +0,0 @@
// +build !windows
package cursor
import (
"fmt"
)
// Up moves the cursor n lines up relative to the current position.
func Up(n int) {
fmt.Printf("\x1b[%dA", n)
height += n
}
// Down moves the cursor n lines down relative to the current position.
func Down(n int) {
fmt.Printf("\x1b[%dB", n)
if height-n <= 0 {
height = 0
} else {
height -= n
}
}
// Right moves the cursor n characters to the right relative to the current position.
func Right(n int) {
fmt.Printf("\x1b[%dC", n)
}
// Left moves the cursor n characters to the left relative to the current position.
func Left(n int) {
fmt.Printf("\x1b[%dD", n)
}
// HorizontalAbsolute moves the cursor to n horizontally.
// The position n is absolute to the start of the line.
func HorizontalAbsolute(n int) {
n += 1 // Moves the line to the character after n
fmt.Printf("\x1b[%dG", n)
}
// Show the cursor if it was hidden previously.
// Don't forget to show the cursor at least at the end of your application.
// Otherwise the user might have a terminal with a permanently hidden cursor, until he reopens the terminal.
func Show() {
fmt.Print("\x1b[?25h")
}
// Hide the cursor.
// Don't forget to show the cursor at least at the end of your application with Show.
// Otherwise the user might have a terminal with a permanently hidden cursor, until he reopens the terminal.
func Hide() {
fmt.Print("\x1b[?25l")
}
// ClearLine clears the current line and moves the cursor to it's start position.
func ClearLine() {
fmt.Print("\x1b[2K")
}

View File

@@ -1,105 +0,0 @@
package cursor
import (
"os"
"syscall"
"unsafe"
)
// Up moves the cursor n lines up relative to the current position.
func Up(n int) {
move(0, -n)
height += n
}
// Down moves the cursor n lines down relative to the current position.
func Down(n int) {
move(0, n)
if height-n <= 0 {
height = 0
} else {
height -= n
}
}
// Right moves the cursor n characters to the right relative to the current position.
func Right(n int) {
move(n, 0)
}
// Left moves the cursor n characters to the left relative to the current position.
func Left(n int) {
move(-n, 0)
}
func move(x int, y int) {
handle := syscall.Handle(os.Stdout.Fd())
var csbi consoleScreenBufferInfo
_, _, _ = procGetConsoleScreenBufferInfo.Call(uintptr(handle), uintptr(unsafe.Pointer(&csbi)))
var cursor coord
cursor.x = csbi.cursorPosition.x + short(x)
cursor.y = csbi.cursorPosition.y + short(y)
_, _, _ = procSetConsoleCursorPosition.Call(uintptr(handle), uintptr(*(*int32)(unsafe.Pointer(&cursor))))
}
// HorizontalAbsolute moves the cursor to n horizontally.
// The position n is absolute to the start of the line.
func HorizontalAbsolute(n int) {
handle := syscall.Handle(os.Stdout.Fd())
var csbi consoleScreenBufferInfo
_, _, _ = procGetConsoleScreenBufferInfo.Call(uintptr(handle), uintptr(unsafe.Pointer(&csbi)))
var cursor coord
cursor.x = short(n)
cursor.y = csbi.cursorPosition.y
if csbi.size.x < cursor.x {
cursor.x = csbi.size.x
}
_, _, _ = procSetConsoleCursorPosition.Call(uintptr(handle), uintptr(*(*int32)(unsafe.Pointer(&cursor))))
}
// Show the cursor if it was hidden previously.
// Don't forget to show the cursor at least at the end of your application.
// Otherwise the user might have a terminal with a permanently hidden cursor, until he reopens the terminal.
func Show() {
handle := syscall.Handle(os.Stdout.Fd())
var cci consoleCursorInfo
_, _, _ = procGetConsoleCursorInfo.Call(uintptr(handle), uintptr(unsafe.Pointer(&cci)))
cci.visible = 1
_, _, _ = procSetConsoleCursorInfo.Call(uintptr(handle), uintptr(unsafe.Pointer(&cci)))
}
// Hide the cursor.
// Don't forget to show the cursor at least at the end of your application with Show.
// Otherwise the user might have a terminal with a permanently hidden cursor, until he reopens the terminal.
func Hide() {
handle := syscall.Handle(os.Stdout.Fd())
var cci consoleCursorInfo
_, _, _ = procGetConsoleCursorInfo.Call(uintptr(handle), uintptr(unsafe.Pointer(&cci)))
cci.visible = 0
_, _, _ = procSetConsoleCursorInfo.Call(uintptr(handle), uintptr(unsafe.Pointer(&cci)))
}
// ClearLine clears the current line and moves the cursor to it's start position.
func ClearLine() {
handle := syscall.Handle(os.Stdout.Fd())
var csbi consoleScreenBufferInfo
_, _, _ = procGetConsoleScreenBufferInfo.Call(uintptr(handle), uintptr(unsafe.Pointer(&csbi)))
var w uint32
var x short
cursor := csbi.cursorPosition
x = csbi.size.x
_, _, _ = procFillConsoleOutputCharacter.Call(uintptr(handle), uintptr(' '), uintptr(x), uintptr(*(*int32)(unsafe.Pointer(&cursor))), uintptr(unsafe.Pointer(&w)))
}

View File

@@ -1,7 +0,0 @@
/*
Package cursor contains cross-platform methods to move the terminal cursor in different directions.
This package can be used to create interactive CLI tools and games, live charts, algorithm visualizations and other updatable output of any kind.
Special thanks to github.com/k0kubun/go-ansi which this project is based on.
*/
package cursor

View File

@@ -1,3 +0,0 @@
module github.com/atomicgo/cursor
go 1.15

View File

View File

@@ -1,43 +0,0 @@
package cursor
import (
"syscall"
)
var (
kernel32 = syscall.NewLazyDLL("kernel32.dll")
procFillConsoleOutputCharacter = kernel32.NewProc("FillConsoleOutputCharacterW")
procGetConsoleCursorInfo = kernel32.NewProc("GetConsoleCursorInfo")
procGetConsoleScreenBufferInfo = kernel32.NewProc("GetConsoleScreenBufferInfo")
procSetConsoleCursorInfo = kernel32.NewProc("SetConsoleCursorInfo")
procSetConsoleCursorPosition = kernel32.NewProc("SetConsoleCursorPosition")
)
type short int16
type dword uint32
type word uint16
type coord struct {
x short
y short
}
type smallRect struct {
bottom short
left short
right short
top short
}
type consoleScreenBufferInfo struct {
size coord
cursorPosition coord
attributes word
window smallRect
maximumWindowSize coord
}
type consoleCursorInfo struct {
size dword
visible int32
}

View File

@@ -1,73 +0,0 @@
package cursor
var height int
// Bottom moves the cursor to the bottom of the terminal.
// This is done by calculating how many lines were moved by Up and Down.
func Bottom() {
if height > 0 {
Down(height)
StartOfLine()
height = 0
}
}
// StartOfLine moves the cursor to the start of the current line.
func StartOfLine() {
HorizontalAbsolute(0)
}
// StartOfLineDown moves the cursor down by n lines, then moves to cursor to the start of the line.
func StartOfLineDown(n int) {
Down(n)
StartOfLine()
}
// StartOfLineUp moves the cursor up by n lines, then moves to cursor to the start of the line.
func StartOfLineUp(n int) {
Up(n)
StartOfLine()
}
// UpAndClear moves the cursor up by n lines, then clears the line.
func UpAndClear(n int) {
Up(n)
ClearLine()
}
// DownAndClear moves the cursor down by n lines, then clears the line.
func DownAndClear(n int) {
Down(n)
ClearLine()
}
// Move moves the cursor relative by x and y.
func Move(x, y int) {
if x > 0 {
Right(x)
} else if x < 0 {
x *= -1
Left(x)
}
if y > 0 {
Up(y)
} else if y < 0 {
y *= -1
Down(y)
}
}
// ClearLinesUp clears n lines upwards from the current position and moves the cursor.
func ClearLinesUp(n int) {
for i := 0; i < n; i++ {
UpAndClear(1)
}
}
// ClearLinesDown clears n lines downwards from the current position and moves the cursor.
func ClearLinesDown(n int) {
for i := 0; i < n; i++ {
DownAndClear(1)
}
}

View File

@@ -1,21 +0,0 @@
The MIT License (MIT)
Copyright (c) 2014 Brian Goff
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

View File

@@ -1,14 +0,0 @@
package md2man
import (
"github.com/russross/blackfriday/v2"
)
// Render converts a markdown document into a roff formatted document.
func Render(doc []byte) []byte {
renderer := NewRoffRenderer()
return blackfriday.Run(doc,
[]blackfriday.Option{blackfriday.WithRenderer(renderer),
blackfriday.WithExtensions(renderer.GetExtensions())}...)
}

View File

@@ -1,345 +0,0 @@
package md2man
import (
"fmt"
"io"
"os"
"strings"
"github.com/russross/blackfriday/v2"
)
// roffRenderer implements the blackfriday.Renderer interface for creating
// roff format (manpages) from markdown text
type roffRenderer struct {
extensions blackfriday.Extensions
listCounters []int
firstHeader bool
defineTerm bool
listDepth int
}
const (
titleHeader = ".TH "
topLevelHeader = "\n\n.SH "
secondLevelHdr = "\n.SH "
otherHeader = "\n.SS "
crTag = "\n"
emphTag = "\\fI"
emphCloseTag = "\\fP"
strongTag = "\\fB"
strongCloseTag = "\\fP"
breakTag = "\n.br\n"
paraTag = "\n.PP\n"
hruleTag = "\n.ti 0\n\\l'\\n(.lu'\n"
linkTag = "\n\\[la]"
linkCloseTag = "\\[ra]"
codespanTag = "\\fB\\fC"
codespanCloseTag = "\\fR"
codeTag = "\n.PP\n.RS\n\n.nf\n"
codeCloseTag = "\n.fi\n.RE\n"
quoteTag = "\n.PP\n.RS\n"
quoteCloseTag = "\n.RE\n"
listTag = "\n.RS\n"
listCloseTag = "\n.RE\n"
arglistTag = "\n.TP\n"
tableStart = "\n.TS\nallbox;\n"
tableEnd = ".TE\n"
tableCellStart = "T{\n"
tableCellEnd = "\nT}\n"
)
// NewRoffRenderer creates a new blackfriday Renderer for generating roff documents
// from markdown
func NewRoffRenderer() *roffRenderer { // nolint: golint
var extensions blackfriday.Extensions
extensions |= blackfriday.NoIntraEmphasis
extensions |= blackfriday.Tables
extensions |= blackfriday.FencedCode
extensions |= blackfriday.SpaceHeadings
extensions |= blackfriday.Footnotes
extensions |= blackfriday.Titleblock
extensions |= blackfriday.DefinitionLists
return &roffRenderer{
extensions: extensions,
}
}
// GetExtensions returns the list of extensions used by this renderer implementation
func (r *roffRenderer) GetExtensions() blackfriday.Extensions {
return r.extensions
}
// RenderHeader handles outputting the header at document start
func (r *roffRenderer) RenderHeader(w io.Writer, ast *blackfriday.Node) {
// disable hyphenation
out(w, ".nh\n")
}
// RenderFooter handles outputting the footer at the document end; the roff
// renderer has no footer information
func (r *roffRenderer) RenderFooter(w io.Writer, ast *blackfriday.Node) {
}
// RenderNode is called for each node in a markdown document; based on the node
// type the equivalent roff output is sent to the writer
func (r *roffRenderer) RenderNode(w io.Writer, node *blackfriday.Node, entering bool) blackfriday.WalkStatus {
var walkAction = blackfriday.GoToNext
switch node.Type {
case blackfriday.Text:
r.handleText(w, node, entering)
case blackfriday.Softbreak:
out(w, crTag)
case blackfriday.Hardbreak:
out(w, breakTag)
case blackfriday.Emph:
if entering {
out(w, emphTag)
} else {
out(w, emphCloseTag)
}
case blackfriday.Strong:
if entering {
out(w, strongTag)
} else {
out(w, strongCloseTag)
}
case blackfriday.Link:
if !entering {
out(w, linkTag+string(node.LinkData.Destination)+linkCloseTag)
}
case blackfriday.Image:
// ignore images
walkAction = blackfriday.SkipChildren
case blackfriday.Code:
out(w, codespanTag)
escapeSpecialChars(w, node.Literal)
out(w, codespanCloseTag)
case blackfriday.Document:
break
case blackfriday.Paragraph:
// roff .PP markers break lists
if r.listDepth > 0 {
return blackfriday.GoToNext
}
if entering {
out(w, paraTag)
} else {
out(w, crTag)
}
case blackfriday.BlockQuote:
if entering {
out(w, quoteTag)
} else {
out(w, quoteCloseTag)
}
case blackfriday.Heading:
r.handleHeading(w, node, entering)
case blackfriday.HorizontalRule:
out(w, hruleTag)
case blackfriday.List:
r.handleList(w, node, entering)
case blackfriday.Item:
r.handleItem(w, node, entering)
case blackfriday.CodeBlock:
out(w, codeTag)
escapeSpecialChars(w, node.Literal)
out(w, codeCloseTag)
case blackfriday.Table:
r.handleTable(w, node, entering)
case blackfriday.TableCell:
r.handleTableCell(w, node, entering)
case blackfriday.TableHead:
case blackfriday.TableBody:
case blackfriday.TableRow:
// no action as cell entries do all the nroff formatting
return blackfriday.GoToNext
default:
fmt.Fprintln(os.Stderr, "WARNING: go-md2man does not handle node type "+node.Type.String())
}
return walkAction
}
func (r *roffRenderer) handleText(w io.Writer, node *blackfriday.Node, entering bool) {
var (
start, end string
)
// handle special roff table cell text encapsulation
if node.Parent.Type == blackfriday.TableCell {
if len(node.Literal) > 30 {
start = tableCellStart
end = tableCellEnd
} else {
// end rows that aren't terminated by "tableCellEnd" with a cr if end of row
if node.Parent.Next == nil && !node.Parent.IsHeader {
end = crTag
}
}
}
out(w, start)
escapeSpecialChars(w, node.Literal)
out(w, end)
}
func (r *roffRenderer) handleHeading(w io.Writer, node *blackfriday.Node, entering bool) {
if entering {
switch node.Level {
case 1:
if !r.firstHeader {
out(w, titleHeader)
r.firstHeader = true
break
}
out(w, topLevelHeader)
case 2:
out(w, secondLevelHdr)
default:
out(w, otherHeader)
}
}
}
func (r *roffRenderer) handleList(w io.Writer, node *blackfriday.Node, entering bool) {
openTag := listTag
closeTag := listCloseTag
if node.ListFlags&blackfriday.ListTypeDefinition != 0 {
// tags for definition lists handled within Item node
openTag = ""
closeTag = ""
}
if entering {
r.listDepth++
if node.ListFlags&blackfriday.ListTypeOrdered != 0 {
r.listCounters = append(r.listCounters, 1)
}
out(w, openTag)
} else {
if node.ListFlags&blackfriday.ListTypeOrdered != 0 {
r.listCounters = r.listCounters[:len(r.listCounters)-1]
}
out(w, closeTag)
r.listDepth--
}
}
func (r *roffRenderer) handleItem(w io.Writer, node *blackfriday.Node, entering bool) {
if entering {
if node.ListFlags&blackfriday.ListTypeOrdered != 0 {
out(w, fmt.Sprintf(".IP \"%3d.\" 5\n", r.listCounters[len(r.listCounters)-1]))
r.listCounters[len(r.listCounters)-1]++
} else if node.ListFlags&blackfriday.ListTypeDefinition != 0 {
// state machine for handling terms and following definitions
// since blackfriday does not distinguish them properly, nor
// does it seperate them into separate lists as it should
if !r.defineTerm {
out(w, arglistTag)
r.defineTerm = true
} else {
r.defineTerm = false
}
} else {
out(w, ".IP \\(bu 2\n")
}
} else {
out(w, "\n")
}
}
func (r *roffRenderer) handleTable(w io.Writer, node *blackfriday.Node, entering bool) {
if entering {
out(w, tableStart)
//call walker to count cells (and rows?) so format section can be produced
columns := countColumns(node)
out(w, strings.Repeat("l ", columns)+"\n")
out(w, strings.Repeat("l ", columns)+".\n")
} else {
out(w, tableEnd)
}
}
func (r *roffRenderer) handleTableCell(w io.Writer, node *blackfriday.Node, entering bool) {
var (
start, end string
)
if node.IsHeader {
start = codespanTag
end = codespanCloseTag
}
if entering {
if node.Prev != nil && node.Prev.Type == blackfriday.TableCell {
out(w, "\t"+start)
} else {
out(w, start)
}
} else {
// need to carriage return if we are at the end of the header row
if node.IsHeader && node.Next == nil {
end = end + crTag
}
out(w, end)
}
}
// because roff format requires knowing the column count before outputting any table
// data we need to walk a table tree and count the columns
func countColumns(node *blackfriday.Node) int {
var columns int
node.Walk(func(node *blackfriday.Node, entering bool) blackfriday.WalkStatus {
switch node.Type {
case blackfriday.TableRow:
if !entering {
return blackfriday.Terminate
}
case blackfriday.TableCell:
if entering {
columns++
}
default:
}
return blackfriday.GoToNext
})
return columns
}
func out(w io.Writer, output string) {
io.WriteString(w, output) // nolint: errcheck
}
func needsBackslash(c byte) bool {
for _, r := range []byte("-_&\\~") {
if c == r {
return true
}
}
return false
}
func escapeSpecialChars(w io.Writer, text []byte) {
for i := 0; i < len(text); i++ {
// escape initial apostrophe or period
if len(text) >= 1 && (text[0] == '\'' || text[0] == '.') {
out(w, "\\&")
}
// directly copy normal characters
org := i
for i < len(text) && !needsBackslash(text[i]) {
i++
}
if i > org {
w.Write(text[org:i]) // nolint: errcheck
}
// escape a character
if i >= len(text) {
break
}
w.Write([]byte{'\\', text[i]}) // nolint: errcheck
}
}

View File

@@ -1,20 +0,0 @@
*.log
*.swp
.idea
*.patch
### Go template
# Binaries for programs and plugins
*.exe
*.exe~
*.dll
*.so
*.dylib
# Test binary, build with `go test -c`
*.test
# Output of the go coverage tool, specifically when used with LiteIDE
*.out
.DS_Store
app
demo

View File

@@ -1,20 +0,0 @@
The MIT License (MIT)
Copyright (c) 2016 inhere
Permission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the "Software"), to deal in
the Software without restriction, including without limitation the rights to
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
the Software, and to permit persons to whom the Software is furnished to do so,
subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

View File

@@ -1,468 +0,0 @@
# CLI Color
![GitHub go.mod Go version](https://img.shields.io/github/go-mod/go-version/gookit/color?style=flat-square)
[![Actions Status](https://github.com/gookit/color/workflows/action-tests/badge.svg)](https://github.com/gookit/color/actions)
[![Codacy Badge](https://api.codacy.com/project/badge/Grade/51b28c5f7ffe4cc2b0f12ecf25ed247f)](https://app.codacy.com/app/inhere/color)
[![GoDoc](https://godoc.org/github.com/gookit/color?status.svg)](https://pkg.go.dev/github.com/gookit/color?tab=overview)
[![GitHub tag (latest SemVer)](https://img.shields.io/github/tag/gookit/color)](https://github.com/gookit/color)
[![Build Status](https://travis-ci.org/gookit/color.svg?branch=master)](https://travis-ci.org/gookit/color)
[![Coverage Status](https://coveralls.io/repos/github/gookit/color/badge.svg?branch=master)](https://coveralls.io/github/gookit/color?branch=master)
[![Go Report Card](https://goreportcard.com/badge/github.com/gookit/color)](https://goreportcard.com/report/github.com/gookit/color)
A command-line color library with true color support, universal API methods and Windows support.
> **[中文说明](README.zh-CN.md)**
Basic color preview:
![basic-color](_examples/images/basic-color2.png)
Now, 256 colors and RGB colors have also been supported to work in Windows CMD and PowerShell:
![color-on-cmd-pwsh](_examples/images/color-on-cmd-pwsh.jpg)
## Features
- Simple to use, zero dependencies
- Supports rich color output: 16-color (4-bit), 256-color (8-bit), true color (24-bit, RGB)
- 16-color output is the most commonly used and most widely supported, working on any Windows version
- Since `v1.2.4` **the 256-color (8-bit), true color (24-bit) support windows CMD and PowerShell**
- See [this gist](https://gist.github.com/XVilka/8346728) for information on true color support
- Generic API methods: `Print`, `Printf`, `Println`, `Sprint`, `Sprintf`
- Supports HTML tag-style color rendering, such as `<green>message</>`.
- In addition to using built-in tags, it also supports custom color attributes
- Custom color attributes support the use of 16 color names, 256 color values, rgb color values and hex color values
- Support working on Windows `cmd` and `powerShell` terminal
- Basic colors: `Bold`, `Black`, `White`, `Gray`, `Red`, `Green`, `Yellow`, `Blue`, `Magenta`, `Cyan`
- Additional styles: `Info`, `Note`, `Light`, `Error`, `Danger`, `Notice`, `Success`, `Comment`, `Primary`, `Warning`, `Question`, `Secondary`
- Support by set `NO_COLOR` for disable color or use `FORCE_COLOR` for force open color render.
- Support Rgb, 256, 16 color conversion
## GoDoc
- [godoc for gopkg](https://pkg.go.dev/gopkg.in/gookit/color.v1)
- [godoc for github](https://pkg.go.dev/github.com/gookit/color)
## Install
```bash
go get github.com/gookit/color
```
## Quick start
```go
package main
import (
"fmt"
"github.com/gookit/color"
)
func main() {
// quick use package func
color.Redp("Simple to use color")
color.Redln("Simple to use color")
color.Greenp("Simple to use color\n")
color.Cyanln("Simple to use color")
color.Yellowln("Simple to use color")
// quick use like fmt.Print*
color.Red.Println("Simple to use color")
color.Green.Print("Simple to use color\n")
color.Cyan.Printf("Simple to use %s\n", "color")
color.Yellow.Printf("Simple to use %s\n", "color")
// use like func
red := color.FgRed.Render
green := color.FgGreen.Render
fmt.Printf("%s line %s library\n", red("Command"), green("color"))
// custom color
color.New(color.FgWhite, color.BgBlack).Println("custom color style")
// can also:
color.Style{color.FgCyan, color.OpBold}.Println("custom color style")
// internal theme/style:
color.Info.Tips("message")
color.Info.Prompt("message")
color.Info.Println("message")
color.Warn.Println("message")
color.Error.Println("message")
// use style tag
color.Print("<suc>he</><comment>llo</>, <cyan>wel</><red>come</>\n")
// Custom label attr: Supports the use of 16 color names, 256 color values, rgb color values and hex color values
color.Println("<fg=11aa23>he</><bg=120,35,156>llo</>, <fg=167;bg=232>wel</><fg=red>come</>")
// apply a style tag
color.Tag("info").Println("info style text")
// prompt message
color.Info.Prompt("prompt style message")
color.Warn.Prompt("prompt style message")
// tips message
color.Info.Tips("tips style message")
color.Warn.Tips("tips style message")
}
```
Run demo: `go run ./_examples/demo.go`
![colored-out](_examples/images/color-demo.jpg)
## Basic/16 color
Supported on any Windows version. Provide generic API methods: `Print`, `Printf`, `Println`, `Sprint`, `Sprintf`
```go
color.Bold.Println("bold message")
color.Black.Println("bold message")
color.White.Println("bold message")
color.Gray.Println("bold message")
color.Red.Println("yellow message")
color.Blue.Println("yellow message")
color.Cyan.Println("yellow message")
color.Yellow.Println("yellow message")
color.Magenta.Println("yellow message")
// Only use foreground color
color.FgCyan.Printf("Simple to use %s\n", "color")
// Only use background color
color.BgRed.Printf("Simple to use %s\n", "color")
```
Run demo: `go run ./_examples/color_16.go`
![basic-color](_examples/images/basic-color.png)
### Custom build color
```go
// Full custom: foreground, background, option
myStyle := color.New(color.FgWhite, color.BgBlack, color.OpBold)
myStyle.Println("custom color style")
// can also:
color.Style{color.FgCyan, color.OpBold}.Println("custom color style")
```
custom set console settings:
```go
// set console color
color.Set(color.FgCyan)
// print message
fmt.Print("message")
// reset console settings
color.Reset()
```
### Additional styles
provide generic API methods: `Print`, `Printf`, `Println`, `Sprint`, `Sprintf`
print message use defined style:
```go
color.Info.Println("Info message")
color.Note.Println("Note message")
color.Notice.Println("Notice message")
color.Error.Println("Error message")
color.Danger.Println("Danger message")
color.Warn.Println("Warn message")
color.Debug.Println("Debug message")
color.Primary.Println("Primary message")
color.Question.Println("Question message")
color.Secondary.Println("Secondary message")
```
Run demo: `go run ./_examples/theme_basic.go`
![theme-basic](_examples/images/theme-basic.png)
**Tips style**
```go
color.Info.Tips("Info tips message")
color.Note.Tips("Note tips message")
color.Notice.Tips("Notice tips message")
color.Error.Tips("Error tips message")
color.Danger.Tips("Danger tips message")
color.Warn.Tips("Warn tips message")
color.Debug.Tips("Debug tips message")
color.Primary.Tips("Primary tips message")
color.Question.Tips("Question tips message")
color.Secondary.Tips("Secondary tips message")
```
Run demo: `go run ./_examples/theme_tips.go`
![theme-tips](_examples/images/theme-tips.png)
**Prompt Style**
```go
color.Info.Prompt("Info prompt message")
color.Note.Prompt("Note prompt message")
color.Notice.Prompt("Notice prompt message")
color.Error.Prompt("Error prompt message")
color.Danger.Prompt("Danger prompt message")
color.Warn.Prompt("Warn prompt message")
color.Debug.Prompt("Debug prompt message")
color.Primary.Prompt("Primary prompt message")
color.Question.Prompt("Question prompt message")
color.Secondary.Prompt("Secondary prompt message")
```
Run demo: `go run ./_examples/theme_prompt.go`
![theme-prompt](_examples/images/theme-prompt.png)
**Block Style**
```go
color.Info.Block("Info block message")
color.Note.Block("Note block message")
color.Notice.Block("Notice block message")
color.Error.Block("Error block message")
color.Danger.Block("Danger block message")
color.Warn.Block("Warn block message")
color.Debug.Block("Debug block message")
color.Primary.Block("Primary block message")
color.Question.Block("Question block message")
color.Secondary.Block("Secondary block message")
```
Run demo: `go run ./_examples/theme_block.go`
![theme-block](_examples/images/theme-block.png)
## 256-color usage
> 256 colors support Windows CMD, PowerShell environment after `v1.2.4`
### Set the foreground or background color
- `color.C256(val uint8, isBg ...bool) Color256`
```go
c := color.C256(132) // fg color
c.Println("message")
c.Printf("format %s", "message")
c := color.C256(132, true) // bg color
c.Println("message")
c.Printf("format %s", "message")
```
### 256-color style
Can be used to set foreground and background colors at the same time.
- `S256(fgAndBg ...uint8) *Style256`
```go
s := color.S256(32, 203)
s.Println("message")
s.Printf("format %s", "message")
```
with options:
```go
s := color.S256(32, 203)
s.SetOpts(color.Opts{color.OpBold})
s.Println("style with options")
s.Printf("style with %s\n", "options")
```
Run demo: `go run ./_examples/color_256.go`
![color-tags](_examples/images/color-256.png)
## RGB/True color
> RGB colors support Windows `CMD`, `PowerShell` environment after `v1.2.4`
**Preview:**
> Run demo: `Run demo: go run ./_examples/color_rgb.go`
![color-rgb](_examples/images/color-rgb.png)
example:
```go
color.RGB(30, 144, 255).Println("message. use RGB number")
color.HEX("#1976D2").Println("blue-darken")
color.HEX("#D50000", true).Println("red-accent. use HEX style")
color.RGBStyleFromString("213,0,0").Println("red-accent. use RGB number")
color.HEXStyle("eee", "D50000").Println("deep-purple color")
```
### Set the foreground or background color
- `color.RGB(r, g, b uint8, isBg ...bool) RGBColor`
```go
c := color.RGB(30,144,255) // fg color
c.Println("message")
c.Printf("format %s", "message")
c := color.RGB(30,144,255, true) // bg color
c.Println("message")
c.Printf("format %s", "message")
```
Create a style from an hexadecimal color string:
- `color.HEX(hex string, isBg ...bool) RGBColor`
```go
c := color.HEX("ccc") // can also: "cccccc" "#cccccc"
c.Println("message")
c.Printf("format %s", "message")
c = color.HEX("aabbcc", true) // as bg color
c.Println("message")
c.Printf("format %s", "message")
```
### RGB color style
Can be used to set the foreground and background colors at the same time.
- `color.NewRGBStyle(fg RGBColor, bg ...RGBColor) *RGBStyle`
```go
s := color.NewRGBStyle(RGB(20, 144, 234), RGB(234, 78, 23))
s.Println("message")
s.Printf("format %s", "message")
```
Create a style from an hexadecimal color string:
- `color.HEXStyle(fg string, bg ...string) *RGBStyle`
```go
s := color.HEXStyle("11aa23", "eee")
s.Println("message")
s.Printf("format %s", "message")
```
with options:
```go
s := color.HEXStyle("11aa23", "eee")
s.SetOpts(color.Opts{color.OpBold})
s.Println("style with options")
s.Printf("style with %s\n", "options")
```
## HTML-like tag usage
**Supported** on Windows `cmd.exe` `PowerShell` .
```go
// use style tag
color.Print("<suc>he</><comment>llo</>, <cyan>wel</><red>come</>")
color.Println("<suc>hello</>")
color.Println("<error>hello</>")
color.Println("<warning>hello</>")
// custom color attributes
color.Print("<fg=yellow;bg=black;op=underscore;>hello, welcome</>\n")
// Custom label attr: Supports the use of 16 color names, 256 color values, rgb color values and hex color values
color.Println("<fg=11aa23>he</><bg=120,35,156>llo</>, <fg=167;bg=232>wel</><fg=red>come</>")
```
- `color.Tag`
```go
// set a style tag
color.Tag("info").Print("info style text")
color.Tag("info").Printf("%s style text", "info")
color.Tag("info").Println("info style text")
```
Run demo: `go run ./_examples/color_tag.go`
![color-tags](_examples/images/color-tags.png)
## Color convert
Supports conversion between Rgb, 256, 16 colors, `Rgb <=> 256 <=> 16`
```go
basic := color.Red
basic.Println("basic color")
c256 := color.Red.C256()
c256.Println("256 color")
c256.C16().Println("basic color")
rgb := color.Red.RGB()
rgb.Println("rgb color")
rgb.C256().Println("256 color")
```
## Func refer
There are some useful functions reference
- `Disable()` disable color render
- `SetOutput(io.Writer)` custom set the colored text output writer
- `ForceOpenColor()` force open color render
- `Colors2code(colors ...Color) string` Convert colors to code. return like "32;45;3"
- `ClearCode(str string) string` Use for clear color codes
- `ClearTag(s string) string` clear all color html-tag for a string
- `IsConsole(w io.Writer)` Determine whether w is one of stderr, stdout, stdin
- `HexToRgb(hex string) (rgb []int)` Convert hex color string to RGB numbers
- `RgbToHex(rgb []int) string` Convert RGB to hex code
- More useful func please see https://pkg.go.dev/github.com/gookit/color
## Project use
Check out these projects, which use https://github.com/gookit/color :
- https://github.com/Delta456/box-cli-maker Make Highly Customized Boxes for your CLI
## Gookit packages
- [gookit/ini](https://github.com/gookit/ini) Go config management, use INI files
- [gookit/rux](https://github.com/gookit/rux) Simple and fast request router for golang HTTP
- [gookit/gcli](https://github.com/gookit/gcli) build CLI application, tool library, running CLI commands
- [gookit/slog](https://github.com/gookit/slog) Concise and extensible go log library
- [gookit/event](https://github.com/gookit/event) Lightweight event manager and dispatcher implements by Go
- [gookit/cache](https://github.com/gookit/cache) Generic cache use and cache manager for golang. support File, Memory, Redis, Memcached.
- [gookit/config](https://github.com/gookit/config) Go config management. support JSON, YAML, TOML, INI, HCL, ENV and Flags
- [gookit/color](https://github.com/gookit/color) A command-line color library with true color support, universal API methods and Windows support
- [gookit/filter](https://github.com/gookit/filter) Provide filtering, sanitizing, and conversion of golang data
- [gookit/validate](https://github.com/gookit/validate) Use for data validation and filtering. support Map, Struct, Form data
- [gookit/goutil](https://github.com/gookit/goutil) Some utils for the Go: string, array/slice, map, format, cli, env, filesystem, test and more
- More, please see https://github.com/gookit
## See also
- [inhere/console](https://github.com/inhere/php-console)
- [xo/terminfo](https://github.com/xo/terminfo)
- [beego/bee](https://github.com/beego/bee)
- [issue9/term](https://github.com/issue9/term)
- [ANSI escape code](https://en.wikipedia.org/wiki/ANSI_escape_code)
- [Standard ANSI color map](https://conemu.github.io/en/AnsiEscapeCodes.html#Standard_ANSI_color_map)
- [Terminal Colors](https://gist.github.com/XVilka/8346728)
## License
[MIT](/LICENSE)

View File

@@ -1,472 +0,0 @@
# CLI Color
![GitHub go.mod Go version](https://img.shields.io/github/go-mod/go-version/gookit/color?style=flat-square)
[![Actions Status](https://github.com/gookit/color/workflows/action-tests/badge.svg)](https://github.com/gookit/color/actions)
[![Codacy Badge](https://api.codacy.com/project/badge/Grade/51b28c5f7ffe4cc2b0f12ecf25ed247f)](https://app.codacy.com/app/inhere/color)
[![GoDoc](https://godoc.org/github.com/gookit/color?status.svg)](https://pkg.go.dev/github.com/gookit/color?tab=overview)
[![GitHub tag (latest SemVer)](https://img.shields.io/github/tag/gookit/color)](https://github.com/gookit/color)
[![Build Status](https://travis-ci.org/gookit/color.svg?branch=master)](https://travis-ci.org/gookit/color)
[![Coverage Status](https://coveralls.io/repos/github/gookit/color/badge.svg?branch=master)](https://coveralls.io/github/gookit/color?branch=master)
[![Go Report Card](https://goreportcard.com/badge/github.com/gookit/color)](https://goreportcard.com/report/github.com/gookit/color)
Golang下的命令行色彩使用库, 拥有丰富的色彩渲染输出通用的API方法兼容Windows系统
> **[EN README](README.md)**
基本颜色预览:
![basic-color](_examples/images/basic-color2.png)
现在256色和RGB色彩也已经支持windows CMD和PowerShell中工作
![color-on-cmd-pwsh](_examples/images/color-on-cmd-pwsh.jpg)
## 功能特色
- 使用简单方便
- 支持丰富的颜色输出, 16色(4bit)256色(8bit)RGB色彩(24bit, RGB)
- 16色(4bit)是最常用和支持最广的支持Windows `cmd.exe`
-`v1.2.4`**256色(8bit)RGB色彩(24bit)均支持Windows CMD和PowerShell终端**
- 请查看 [this gist](https://gist.github.com/XVilka/8346728) 了解支持RGB色彩的终端
- 提供通用的API方法`Print` `Printf` `Println` `Sprint` `Sprintf`
- 同时支持html标签式的颜色渲染除了使用内置标签同时支持自定义颜色属性
- 例如: `this an <green>message</>` 标签内部的文本将会渲染为绿色字体
- 自定义颜色属性: 支持使用16色彩名称256色彩值rgb色彩值以及hex色彩值
- 基础色彩: `Bold` `Black` `White` `Gray` `Red` `Green` `Yellow` `Blue` `Magenta` `Cyan`
- 扩展风格: `Info` `Note` `Light` `Error` `Danger` `Notice` `Success` `Comment` `Primary` `Warning` `Question` `Secondary`
- 支持通过设置环境变量 `NO_COLOR` 来禁用色彩,或者使用 `FORCE_COLOR` 来强制使用色彩渲染.
- 支持 Rgb, 256, 16 色彩之间的互相转换
- 支持Linux、Mac同时兼容Windows系统环境
## GoDoc
- [godoc for gopkg](https://pkg.go.dev/gopkg.in/gookit/color.v1)
- [godoc for github](https://pkg.go.dev/github.com/gookit/color)
## 安装
```bash
go get github.com/gookit/color
```
## 快速开始
如下,引入当前包就可以快速的使用
```go
package main
import (
"fmt"
"github.com/gookit/color"
)
func main() {
// 简单快速的使用,跟 fmt.Print* 类似
color.Redp("Simple to use color")
color.Redln("Simple to use color")
color.Greenp("Simple to use color\n")
color.Cyanln("Simple to use color")
color.Yellowln("Simple to use color")
// 简单快速的使用,跟 fmt.Print* 类似
color.Red.Println("Simple to use color")
color.Green.Print("Simple to use color\n")
color.Cyan.Printf("Simple to use %s\n", "color")
color.Yellow.Printf("Simple to use %s\n", "color")
// use like func
red := color.FgRed.Render
green := color.FgGreen.Render
fmt.Printf("%s line %s library\n", red("Command"), green("color"))
// 自定义颜色
color.New(color.FgWhite, color.BgBlack).Println("custom color style")
// 也可以:
color.Style{color.FgCyan, color.OpBold}.Println("custom color style")
// internal style:
color.Info.Println("message")
color.Warn.Println("message")
color.Error.Println("message")
// 使用内置颜色标签
color.Print("<suc>he</><comment>llo</>, <cyan>wel</><red>come</>\n")
// 自定义标签: 支持使用16色彩名称256色彩值rgb色彩值以及hex色彩值
color.Println("<fg=11aa23>he</><bg=120,35,156>llo</>, <fg=167;bg=232>wel</><fg=red>come</>")
// apply a style tag
color.Tag("info").Println("info style text")
// prompt message
color.Info.Prompt("prompt style message")
color.Warn.Prompt("prompt style message")
// tips message
color.Info.Tips("tips style message")
color.Warn.Tips("tips style message")
}
```
> 运行 demo: `go run ./_examples/demo.go`
![colored-out](_examples/images/color-demo.jpg)
## 基础颜色(16-color)
提供通用的API方法`Print` `Printf` `Println` `Sprint` `Sprintf`
> 支持在windows `cmd.exe` `powerShell` 等终端使用
```go
color.Bold.Println("bold message")
color.Black.Println("bold message")
color.White.Println("bold message")
color.Gray.Println("bold message")
color.Red.Println("yellow message")
color.Blue.Println("yellow message")
color.Cyan.Println("yellow message")
color.Yellow.Println("yellow message")
color.Magenta.Println("yellow message")
// Only use foreground color
color.FgCyan.Printf("Simple to use %s\n", "color")
// Only use background color
color.BgRed.Printf("Simple to use %s\n", "color")
```
> 运行demo: `go run ./_examples/color_16.go`
![basic-color](_examples/images/basic-color.png)
### 构建风格
```go
// 仅设置前景色
color.FgCyan.Printf("Simple to use %s\n", "color")
// 仅设置背景色
color.BgRed.Printf("Simple to use %s\n", "color")
// 完全自定义: 前景色 背景色 选项
style := color.New(color.FgWhite, color.BgBlack, color.OpBold)
style.Println("custom color style")
// 也可以:
color.Style{color.FgCyan, color.OpBold}.Println("custom color style")
```
直接设置控制台属性:
```go
// 设置console颜色
color.Set(color.FgCyan)
// 输出信息
fmt.Print("message")
// 重置console颜色
color.Reset()
```
> 当然color已经内置丰富的色彩风格支持
### 扩展风格方法
提供通用的API方法`Print` `Printf` `Println` `Sprint` `Sprintf`
> 支持在windows `cmd.exe` `powerShell` 等终端使用
基础使用:
```go
// print message
color.Info.Println("Info message")
color.Note.Println("Note message")
color.Notice.Println("Notice message")
color.Error.Println("Error message")
color.Danger.Println("Danger message")
color.Warn.Println("Warn message")
color.Debug.Println("Debug message")
color.Primary.Println("Primary message")
color.Question.Println("Question message")
color.Secondary.Println("Secondary message")
```
Run demo: `go run ./_examples/theme_basic.go`
![theme-basic](_examples/images/theme-basic.png)
**简约提示风格**
```go
color.Info.Tips("Info tips message")
color.Note.Tips("Note tips message")
color.Notice.Tips("Notice tips message")
color.Error.Tips("Error tips message")
color.Danger.Tips("Danger tips message")
color.Warn.Tips("Warn tips message")
color.Debug.Tips("Debug tips message")
color.Primary.Tips("Primary tips message")
color.Question.Tips("Question tips message")
color.Secondary.Tips("Secondary tips message")
```
Run demo: `go run ./_examples/theme_tips.go`
![theme-tips](_examples/images/theme-tips.png)
**着重提示风格**
```go
color.Info.Prompt("Info prompt message")
color.Note.Prompt("Note prompt message")
color.Notice.Prompt("Notice prompt message")
color.Error.Prompt("Error prompt message")
color.Danger.Prompt("Danger prompt message")
```
Run demo: `go run ./_examples/theme_prompt.go`
![theme-prompt](_examples/images/theme-prompt.png)
**强调提示风格**
```go
color.Warn.Block("Warn block message")
color.Debug.Block("Debug block message")
color.Primary.Block("Primary block message")
color.Question.Block("Question block message")
color.Secondary.Block("Secondary block message")
```
Run demo: `go run ./_examples/theme_block.go`
![theme-block](_examples/images/theme-block.png)
## 256 色彩使用
> 256色彩在 `v1.2.4` 后支持Windows CMD,PowerShell 环境
### 使用前景或后景色
- `color.C256(val uint8, isBg ...bool) Color256`
```go
c := color.C256(132) // fg color
c.Println("message")
c.Printf("format %s", "message")
c := color.C256(132, true) // bg color
c.Println("message")
c.Printf("format %s", "message")
```
### 使用256 色彩风格
> 可同时设置前景和背景色
- `color.S256(fgAndBg ...uint8) *Style256`
```go
s := color.S256(32, 203)
s.Println("message")
s.Printf("format %s", "message")
```
可以同时添加选项设置:
```go
s := color.S256(32, 203)
s.SetOpts(color.Opts{color.OpBold})
s.Println("style with options")
s.Printf("style with %s\n", "options")
```
> 运行 demo: `go run ./_examples/color_256.go`
![color-tags](_examples/images/color-256.png)
## RGB/True色彩使用
> RGB色彩在 `v1.2.4` 后支持 Windows `CMD`, `PowerShell` 环境
**效果预览:**
> 运行 demo: `Run demo: go run ./_examples/color_rgb.go`
![color-rgb](_examples/images/color-rgb.png)
代码示例:
```go
color.RGB(30, 144, 255).Println("message. use RGB number")
color.HEX("#1976D2").Println("blue-darken")
color.HEX("#D50000", true).Println("red-accent. use HEX style")
color.RGBStyleFromString("213,0,0").Println("red-accent. use RGB number")
color.HEXStyle("eee", "D50000").Println("deep-purple color")
```
### 使用前景或后景色
- `color.RGB(r, g, b uint8, isBg ...bool) RGBColor`
```go
c := color.RGB(30,144,255) // fg color
c.Println("message")
c.Printf("format %s", "message")
c := color.RGB(30,144,255, true) // bg color
c.Println("message")
c.Printf("format %s", "message")
```
- `color.HEX(hex string, isBg ...bool) RGBColor` 从16进制颜色创建
```go
c := color.HEX("ccc") // 也可以写为: "cccccc" "#cccccc"
c.Println("message")
c.Printf("format %s", "message")
c = color.HEX("aabbcc", true) // as bg color
c.Println("message")
c.Printf("format %s", "message")
```
### 使用RGB风格
> 可同时设置前景和背景色
- `color.NewRGBStyle(fg RGBColor, bg ...RGBColor) *RGBStyle`
```go
s := color.NewRGBStyle(RGB(20, 144, 234), RGB(234, 78, 23))
s.Println("message")
s.Printf("format %s", "message")
```
- `color.HEXStyle(fg string, bg ...string) *RGBStyle` 从16进制颜色创建
```go
s := color.HEXStyle("11aa23", "eee")
s.Println("message")
s.Printf("format %s", "message")
```
- 可以同时添加选项设置:
```go
s := color.HEXStyle("11aa23", "eee")
s.SetOpts(color.Opts{color.OpBold})
s.Println("style with options")
s.Printf("style with %s\n", "options")
```
## 使用颜色标签
> **支持** 在windows `cmd.exe` `PowerShell` 使用
使用内置的颜色标签,可以非常方便简单的构建自己需要的任何格式
> 同时支持自定义颜色属性: 支持使用16色彩名称256色彩值rgb色彩值以及hex色彩值
```go
// 使用内置的 color tag
color.Print("<suc>he</><comment>llo</>, <cyan>wel</><red>come</>")
color.Println("<suc>hello</>")
color.Println("<error>hello</>")
color.Println("<warning>hello</>")
// 自定义颜色属性
color.Print("<fg=yellow;bg=black;op=underscore;>hello, welcome</>\n")
// 自定义颜色属性: 支持使用16色彩名称256色彩值rgb色彩值以及hex色彩值
color.Println("<fg=11aa23>he</><bg=120,35,156>llo</>, <fg=167;bg=232>wel</><fg=red>come</>")
```
- 使用 `color.Tag`
给后面输出的文本信息加上给定的颜色风格标签
```go
// set a style tag
color.Tag("info").Print("info style text")
color.Tag("info").Printf("%s style text", "info")
color.Tag("info").Println("info style text")
```
> 运行 demo: `go run ./_examples/color_tag.go`
![color-tags](_examples/images/color-tags.png)
## 颜色转换
支持 Rgb, 256, 16 色彩之间的互相转换 `Rgb <=> 256 <=> 16`
```go
basic := color.Red
basic.Println("basic color")
c256 := color.Red.C256()
c256.Println("256 color")
c256.C16().Println("basic color")
rgb := color.Red.RGB()
rgb.Println("rgb color")
rgb.C256().Println("256 color")
```
## 方法参考
一些有用的工具方法参考
- `Disable()` disable color render
- `SetOutput(io.Writer)` custom set the colored text output writer
- `ForceOpenColor()` force open color render
- `ClearCode(str string) string` Use for clear color codes
- `Colors2code(colors ...Color) string` Convert colors to code. return like "32;45;3"
- `ClearTag(s string) string` clear all color html-tag for a string
- `IsConsole(w io.Writer)` Determine whether w is one of stderr, stdout, stdin
- `HexToRgb(hex string) (rgb []int)` Convert hex color string to RGB numbers
- `RgbToHex(rgb []int) string` Convert RGB to hex code
- 更多请查看文档 https://pkg.go.dev/github.com/gookit/color
## 使用color的项目
看看这些使用了 https://github.com/gookit/color 的项目:
- https://github.com/Delta456/box-cli-maker Make Highly Customized Boxes for your CLI
## Gookit 工具包
- [gookit/ini](https://github.com/gookit/ini) INI配置读取管理支持多文件加载数据覆盖合并, 解析ENV变量, 解析变量引用
- [gookit/rux](https://github.com/gookit/rux) Simple and fast request router for golang HTTP
- [gookit/gcli](https://github.com/gookit/gcli) Go的命令行应用工具库运行CLI命令支持命令行色彩用户交互进度显示数据格式化显示
- [gookit/slog](https://github.com/gookit/slog) 简洁易扩展的go日志库
- [gookit/event](https://github.com/gookit/event) Go实现的轻量级的事件管理、调度程序库, 支持设置监听器的优先级, 支持对一组事件进行监听
- [gookit/cache](https://github.com/gookit/cache) 通用的缓存使用包装库通过包装各种常用的驱动来提供统一的使用API
- [gookit/config](https://github.com/gookit/config) Go应用配置管理支持多种格式JSON, YAML, TOML, INI, HCL, ENV, Flags多文件加载远程文件加载数据合并
- [gookit/color](https://github.com/gookit/color) CLI 控制台颜色渲染工具库, 拥有简洁的使用API支持16色256色RGB色彩渲染输出
- [gookit/filter](https://github.com/gookit/filter) 提供对Golang数据的过滤净化转换
- [gookit/validate](https://github.com/gookit/validate) Go通用的数据验证与过滤库使用简单内置大部分常用验证、过滤器
- [gookit/goutil](https://github.com/gookit/goutil) Go 的一些工具函数,格式化,特殊处理,常用信息获取等
- 更多请查看 https://github.com/gookit
## 参考项目
- [inhere/console](https://github.com/inhere/php-console)
- [xo/terminfo](https://github.com/xo/terminfo)
- [beego/bee](https://github.com/beego/bee)
- [issue9/term](https://github.com/issue9/term)
- [ANSI转义序列](https://zh.wikipedia.org/wiki/ANSI转义序列)
- [Standard ANSI color map](https://conemu.github.io/en/AnsiEscapeCodes.html#Standard_ANSI_color_map)
- [Terminal Colors](https://gist.github.com/XVilka/8346728)
## License
MIT

View File

@@ -1,238 +0,0 @@
/*
Package color is Command line color library.
Support rich color rendering output, universal API method, compatible with Windows system
Source code and other details for the project are available at GitHub:
https://github.com/gookit/color
More usage please see README and tests.
*/
package color
import (
"fmt"
"io"
"os"
"regexp"
"github.com/xo/terminfo"
)
// terminal color available level alias of the terminfo.ColorLevel*
const (
LevelNo = terminfo.ColorLevelNone // not support color.
Level16 = terminfo.ColorLevelBasic // 3/4 bit color supported
Level256 = terminfo.ColorLevelHundreds // 8 bit color supported
LevelRgb = terminfo.ColorLevelMillions // (24 bit)true color supported
)
// color render templates
// ESC 操作的表示:
// "\033"(Octal 8进制) = "\x1b"(Hexadecimal 16进制) = 27 (10进制)
const (
SettingTpl = "\x1b[%sm"
FullColorTpl = "\x1b[%sm%s\x1b[0m"
)
// ResetSet Close all properties.
const ResetSet = "\x1b[0m"
// CodeExpr regex to clear color codes eg "\033[1;36mText\x1b[0m"
const CodeExpr = `\033\[[\d;?]+m`
var (
// Enable switch color render and display
//
// NOTICE:
// if ENV: NO_COLOR is not empty, will disable color render.
Enable = os.Getenv("NO_COLOR") == ""
// RenderTag render HTML tag on call color.Xprint, color.PrintX
RenderTag = true
// debug mode for development.
//
// set env:
// COLOR_DEBUG_MODE=on
// or:
// COLOR_DEBUG_MODE=on go run ./_examples/envcheck.go
debugMode = os.Getenv("COLOR_DEBUG_MODE") == "on"
// inner errors record on detect color level
innerErrs []error
// output the default io.Writer message print
output io.Writer = os.Stdout
// mark current env, It's like in `cmd.exe`
// if not in windows, it's always is False.
isLikeInCmd bool
// the color support level for current terminal
// needVTP - need enable VTP, only for windows OS
colorLevel, needVTP = detectTermColorLevel()
// match color codes
codeRegex = regexp.MustCompile(CodeExpr)
// mark current env is support color.
// Always: isLikeInCmd != supportColor
// supportColor = IsSupportColor()
)
// TermColorLevel value on current ENV
func TermColorLevel() terminfo.ColorLevel {
return colorLevel
}
// SupportColor on the current ENV
func SupportColor() bool {
return colorLevel > terminfo.ColorLevelNone
}
// Support16Color on the current ENV
// func Support16Color() bool {
// return colorLevel > terminfo.ColorLevelNone
// }
// Support256Color on the current ENV
func Support256Color() bool {
return colorLevel > terminfo.ColorLevelBasic
}
// SupportTrueColor on the current ENV
func SupportTrueColor() bool {
return colorLevel > terminfo.ColorLevelHundreds
}
/*************************************************************
* global settings
*************************************************************/
// Set set console color attributes
func Set(colors ...Color) (int, error) {
code := Colors2code(colors...)
err := SetTerminal(code)
return 0, err
}
// Reset reset console color attributes
func Reset() (int, error) {
err := ResetTerminal()
return 0, err
}
// Disable disable color output
func Disable() bool {
oldVal := Enable
Enable = false
return oldVal
}
// NotRenderTag on call color.Xprint, color.PrintX
func NotRenderTag() {
RenderTag = false
}
// SetOutput set default colored text output
func SetOutput(w io.Writer) {
output = w
}
// ResetOutput reset output
func ResetOutput() {
output = os.Stdout
}
// ResetOptions reset all package option setting
func ResetOptions() {
RenderTag = true
Enable = true
output = os.Stdout
}
// ForceColor force open color render
func ForceSetColorLevel(level terminfo.ColorLevel) terminfo.ColorLevel {
oldLevelVal := colorLevel
colorLevel = level
return oldLevelVal
}
// ForceColor force open color render
func ForceColor() terminfo.ColorLevel {
return ForceOpenColor()
}
// ForceOpenColor force open color render
func ForceOpenColor() terminfo.ColorLevel {
// TODO should set level to ?
return ForceSetColorLevel(terminfo.ColorLevelMillions)
}
// IsLikeInCmd check result
// Deprecated
func IsLikeInCmd() bool {
return isLikeInCmd
}
// InnerErrs info
func InnerErrs() []error {
return innerErrs
}
/*************************************************************
* render color code
*************************************************************/
// RenderCode render message by color code.
// Usage:
// msg := RenderCode("3;32;45", "some", "message")
func RenderCode(code string, args ...interface{}) string {
var message string
if ln := len(args); ln == 0 {
return ""
}
message = fmt.Sprint(args...)
if len(code) == 0 {
return message
}
// disabled OR not support color
if !Enable || !SupportColor() {
return ClearCode(message)
}
return fmt.Sprintf(FullColorTpl, code, message)
}
// RenderWithSpaces Render code with spaces.
// If the number of args is > 1, a space will be added between the args
func RenderWithSpaces(code string, args ...interface{}) string {
message := formatArgsForPrintln(args)
if len(code) == 0 {
return message
}
// disabled OR not support color
if !Enable || !SupportColor() {
return ClearCode(message)
}
return fmt.Sprintf(FullColorTpl, code, message)
}
// RenderString render a string with color code.
// Usage:
// msg := RenderString("3;32;45", "a message")
func RenderString(code string, str string) string {
if len(code) == 0 || str == "" {
return str
}
// disabled OR not support color
if !Enable || !SupportColor() {
return ClearCode(str)
}
return fmt.Sprintf(FullColorTpl, code, str)
}
// ClearCode clear color codes.
// eg: "\033[36;1mText\x1b[0m" -> "Text"
func ClearCode(str string) string {
return codeRegex.ReplaceAllString(str, "")
}

View File

@@ -1,440 +0,0 @@
package color
import (
"fmt"
"strconv"
)
// Color Color16, 16 color value type
// 3(2^3=8) OR 4(2^4=16) bite color.
type Color uint8
type Basic = Color // alias of Color
// Opts basic color options. code: 0 - 9
type Opts []Color
// Add option value
func (o *Opts) Add(ops ...Color) {
for _, op := range ops {
if uint8(op) < 10 {
*o = append(*o, op)
}
}
}
// IsValid options
func (o Opts) IsValid() bool {
return len(o) > 0
}
// IsEmpty options
func (o Opts) IsEmpty() bool {
return len(o) == 0
}
// String options to string. eg: "1;3"
func (o Opts) String() string {
return Colors2code(o...)
}
/*************************************************************
* Basic 16 color definition
*************************************************************/
// Base value for foreground/background color
const (
FgBase uint8 = 30
BgBase uint8 = 40
// hi color base code
HiFgBase uint8 = 90
HiBgBase uint8 = 100
)
// Foreground colors. basic foreground colors 30 - 37
const (
FgBlack Color = iota + 30
FgRed
FgGreen
FgYellow
FgBlue
FgMagenta // 品红
FgCyan // 青色
FgWhite
// FgDefault revert default FG
FgDefault Color = 39
)
// Extra foreground color 90 - 97(非标准)
const (
FgDarkGray Color = iota + 90 // 亮黑(灰)
FgLightRed
FgLightGreen
FgLightYellow
FgLightBlue
FgLightMagenta
FgLightCyan
FgLightWhite
// FgGray is alias of FgDarkGray
FgGray Color = 90 // 亮黑(灰)
)
// Background colors. basic background colors 40 - 47
const (
BgBlack Color = iota + 40
BgRed
BgGreen
BgYellow // BgBrown like yellow
BgBlue
BgMagenta
BgCyan
BgWhite
// BgDefault revert default BG
BgDefault Color = 49
)
// Extra background color 100 - 107(非标准)
const (
BgDarkGray Color = iota + 100
BgLightRed
BgLightGreen
BgLightYellow
BgLightBlue
BgLightMagenta
BgLightCyan
BgLightWhite
// BgGray is alias of BgDarkGray
BgGray Color = 100
)
// Option settings
const (
OpReset Color = iota // 0 重置所有设置
OpBold // 1 加粗
OpFuzzy // 2 模糊(不是所有的终端仿真器都支持)
OpItalic // 3 斜体(不是所有的终端仿真器都支持)
OpUnderscore // 4 下划线
OpBlink // 5 闪烁
OpFastBlink // 5 快速闪烁(未广泛支持)
OpReverse // 7 颠倒的 交换背景色与前景色
OpConcealed // 8 隐匿的
OpStrikethrough // 9 删除的,删除线(未广泛支持)
)
// There are basic and light foreground color aliases
const (
Red = FgRed
Cyan = FgCyan
Gray = FgDarkGray // is light Black
Blue = FgBlue
Black = FgBlack
Green = FgGreen
White = FgWhite
Yellow = FgYellow
Magenta = FgMagenta
// special
Bold = OpBold
Normal = FgDefault
// extra light
LightRed = FgLightRed
LightCyan = FgLightCyan
LightBlue = FgLightBlue
LightGreen = FgLightGreen
LightWhite = FgLightWhite
LightYellow = FgLightYellow
LightMagenta = FgLightMagenta
HiRed = FgLightRed
HiCyan = FgLightCyan
HiBlue = FgLightBlue
HiGreen = FgLightGreen
HiWhite = FgLightWhite
HiYellow = FgLightYellow
HiMagenta = FgLightMagenta
BgHiRed = BgLightRed
BgHiCyan = BgLightCyan
BgHiBlue = BgLightBlue
BgHiGreen = BgLightGreen
BgHiWhite = BgLightWhite
BgHiYellow = BgLightYellow
BgHiMagenta = BgLightMagenta
)
// Bit4 an method for create Color
func Bit4(code uint8) Color {
return Color(code)
}
/*************************************************************
* Color render methods
*************************************************************/
// Name get color code name.
func (c Color) Name() string {
name, ok := basic2nameMap[uint8(c)]
if ok {
return name
}
return "unknown"
}
// Text render a text message
func (c Color) Text(message string) string {
return RenderString(c.String(), message)
}
// Render messages by color setting
// Usage:
// green := color.FgGreen.Render
// fmt.Println(green("message"))
func (c Color) Render(a ...interface{}) string {
return RenderCode(c.String(), a...)
}
// Renderln messages by color setting.
// like Println, will add spaces for each argument
// Usage:
// green := color.FgGreen.Renderln
// fmt.Println(green("message"))
func (c Color) Renderln(a ...interface{}) string {
return RenderWithSpaces(c.String(), a...)
}
// Sprint render messages by color setting. is alias of the Render()
func (c Color) Sprint(a ...interface{}) string {
return RenderCode(c.String(), a...)
}
// Sprintf format and render message.
// Usage:
// green := color.Green.Sprintf
// colored := green("message")
func (c Color) Sprintf(format string, args ...interface{}) string {
return RenderString(c.String(), fmt.Sprintf(format, args...))
}
// Print messages.
// Usage:
// color.Green.Print("message")
// OR:
// green := color.FgGreen.Print
// green("message")
func (c Color) Print(args ...interface{}) {
doPrintV2(c.Code(), fmt.Sprint(args...))
}
// Printf format and print messages.
// Usage:
// color.Cyan.Printf("string %s", "arg0")
func (c Color) Printf(format string, a ...interface{}) {
doPrintV2(c.Code(), fmt.Sprintf(format, a...))
}
// Println messages with new line
func (c Color) Println(a ...interface{}) {
doPrintlnV2(c.String(), a)
}
// Light current color. eg: 36(FgCyan) -> 96(FgLightCyan).
// Usage:
// lightCyan := Cyan.Light()
// lightCyan.Print("message")
func (c Color) Light() Color {
val := int(c)
if val >= 30 && val <= 47 {
return Color(uint8(c) + 60)
}
// don't change
return c
}
// Darken current color. eg. 96(FgLightCyan) -> 36(FgCyan)
// Usage:
// cyan := LightCyan.Darken()
// cyan.Print("message")
func (c Color) Darken() Color {
val := int(c)
if val >= 90 && val <= 107 {
return Color(uint8(c) - 60)
}
// don't change
return c
}
// C256 convert 16 color to 256-color code.
func (c Color) C256() Color256 {
val := uint8(c)
if val < 10 { // is option code
return emptyC256 // empty
}
var isBg uint8
if val >= BgBase && val <= 47 { // is bg
isBg = AsBg
val = val - 10 // to fg code
} else if val >= HiBgBase && val <= 107 { // is hi bg
isBg = AsBg
val = val - 10 // to fg code
}
if c256, ok := basicTo256Map[val]; ok {
return Color256{c256, isBg}
}
// use raw value direct convert
return Color256{val}
}
// RGB convert 16 color to 256-color code.
func (c Color) RGB() RGBColor {
val := uint8(c)
if val < 10 { // is option code
return emptyRGBColor
}
return HEX(Basic2hex(val))
}
// Code convert to code string. eg "35"
func (c Color) Code() string {
// return fmt.Sprintf("%d", c)
return strconv.Itoa(int(c))
}
// String convert to code string. eg "35"
func (c Color) String() string {
// return fmt.Sprintf("%d", c)
return strconv.Itoa(int(c))
}
// IsValid color value
func (c Color) IsValid() bool {
return c < 107
}
/*************************************************************
* basic color maps
*************************************************************/
// FgColors foreground colors map
var FgColors = map[string]Color{
"black": FgBlack,
"red": FgRed,
"green": FgGreen,
"yellow": FgYellow,
"blue": FgBlue,
"magenta": FgMagenta,
"cyan": FgCyan,
"white": FgWhite,
"default": FgDefault,
}
// BgColors background colors map
var BgColors = map[string]Color{
"black": BgBlack,
"red": BgRed,
"green": BgGreen,
"yellow": BgYellow,
"blue": BgBlue,
"magenta": BgMagenta,
"cyan": BgCyan,
"white": BgWhite,
"default": BgDefault,
}
// ExFgColors extra foreground colors map
var ExFgColors = map[string]Color{
"darkGray": FgDarkGray,
"lightRed": FgLightRed,
"lightGreen": FgLightGreen,
"lightYellow": FgLightYellow,
"lightBlue": FgLightBlue,
"lightMagenta": FgLightMagenta,
"lightCyan": FgLightCyan,
"lightWhite": FgLightWhite,
}
// ExBgColors extra background colors map
var ExBgColors = map[string]Color{
"darkGray": BgDarkGray,
"lightRed": BgLightRed,
"lightGreen": BgLightGreen,
"lightYellow": BgLightYellow,
"lightBlue": BgLightBlue,
"lightMagenta": BgLightMagenta,
"lightCyan": BgLightCyan,
"lightWhite": BgLightWhite,
}
// Options color options map
// Deprecated
// NOTICE: please use AllOptions instead.
var Options = AllOptions
// AllOptions color options map
var AllOptions = map[string]Color{
"reset": OpReset,
"bold": OpBold,
"fuzzy": OpFuzzy,
"italic": OpItalic,
"underscore": OpUnderscore,
"blink": OpBlink,
"reverse": OpReverse,
"concealed": OpConcealed,
}
var (
// TODO basic name alias
// basicNameAlias = map[string]string{}
// basic color name to code
name2basicMap = initName2basicMap()
// basic2nameMap basic color code to name
basic2nameMap = map[uint8]string{
30: "black",
31: "red",
32: "green",
33: "yellow",
34: "blue",
35: "magenta",
36: "cyan",
37: "white",
// hi color code
90: "lightBlack",
91: "lightRed",
92: "lightGreen",
93: "lightYellow",
94: "lightBlue",
95: "lightMagenta",
96: "lightCyan",
97: "lightWhite",
// options
0: "reset",
1: "bold",
2: "fuzzy",
3: "italic",
4: "underscore",
5: "blink",
7: "reverse",
8: "concealed",
}
)
// Basic2nameMap data
func Basic2nameMap() map[uint8]string {
return basic2nameMap
}
func initName2basicMap() map[string]uint8 {
n2b := make(map[string]uint8, len(basic2nameMap))
for u, s := range basic2nameMap {
n2b[s] = u
}
return n2b
}

View File

@@ -1,308 +0,0 @@
package color
import (
"fmt"
"strconv"
"strings"
)
/*
from wikipedia, 256 color:
ESC[ … 38;5;<n> … m选择前景色
ESC[ … 48;5;<n> … m选择背景色
0- 7标准颜色同 ESC[3037m
8- 15高强度颜色同 ESC[9097m
16-2316 × 6 × 6 立方216色: 16 + 36 × r + 6 × g + b (0 ≤ r, g, b ≤ 5)
232-255从黑到白的24阶灰度色
*/
// tpl for 8 bit 256 color(`2^8`)
//
// format:
// ESC[ … 38;5;<n> … m // 选择前景色
// ESC[ … 48;5;<n> … m // 选择背景色
//
// example:
// fg "\x1b[38;5;242m"
// bg "\x1b[48;5;208m"
// both "\x1b[38;5;242;48;5;208m"
//
// links:
// https://zh.wikipedia.org/wiki/ANSI%E8%BD%AC%E4%B9%89%E5%BA%8F%E5%88%97#8位
const (
TplFg256 = "38;5;%d"
TplBg256 = "48;5;%d"
Fg256Pfx = "38;5;"
Bg256Pfx = "48;5;"
)
/*************************************************************
* 8bit(256) Color: Bit8Color Color256
*************************************************************/
// Color256 256 color (8 bit), uint8 range at 0 - 255
//
// 颜色值使用10进制和16进制都可 0x98 = 152
//
// The color consists of two uint8:
// 0: color value
// 1: color type; Fg=0, Bg=1, >1: unset value
//
// example:
// fg color: [152, 0]
// bg color: [152, 1]
//
// NOTICE: now support 256 color on windows CMD, PowerShell
// lint warn - Name starts with package name
type Color256 [2]uint8
type Bit8Color = Color256 // alias
var emptyC256 = Color256{1: 99}
// Bit8 create a color256
func Bit8(val uint8, isBg ...bool) Color256 {
return C256(val, isBg...)
}
// C256 create a color256
func C256(val uint8, isBg ...bool) Color256 {
bc := Color256{val}
// mark is bg color
if len(isBg) > 0 && isBg[0] {
bc[1] = AsBg
}
return bc
}
// Set terminal by 256 color code
func (c Color256) Set() error {
return SetTerminal(c.String())
}
// Reset terminal. alias of the ResetTerminal()
func (c Color256) Reset() error {
return ResetTerminal()
}
// Print print message
func (c Color256) Print(a ...interface{}) {
doPrintV2(c.String(), fmt.Sprint(a...))
}
// Printf format and print message
func (c Color256) Printf(format string, a ...interface{}) {
doPrintV2(c.String(), fmt.Sprintf(format, a...))
}
// Println print message with newline
func (c Color256) Println(a ...interface{}) {
doPrintlnV2(c.String(), a)
}
// Sprint returns rendered message
func (c Color256) Sprint(a ...interface{}) string {
return RenderCode(c.String(), a...)
}
// Sprintf returns format and rendered message
func (c Color256) Sprintf(format string, a ...interface{}) string {
return RenderString(c.String(), fmt.Sprintf(format, a...))
}
// C16 convert color-256 to 16 color.
func (c Color256) C16() Color {
return c.Basic()
}
// Basic convert color-256 to basic 16 color.
func (c Color256) Basic() Color {
return Color(c[0]) // TODO
}
// RGB convert color-256 to RGB color.
func (c Color256) RGB() RGBColor {
return RGBFromSlice(C256ToRgb(c[0]), c[1] == AsBg)
}
// RGBColor convert color-256 to RGB color.
func (c Color256) RGBColor() RGBColor {
return c.RGB()
}
// Value return color value
func (c Color256) Value() uint8 {
return c[0]
}
// Code convert to color code string. eg: "12"
func (c Color256) Code() string {
return strconv.Itoa(int(c[0]))
}
// FullCode convert to color code string with prefix. eg: "38;5;12"
func (c Color256) FullCode() string {
return c.String()
}
// String convert to color code string with prefix. eg: "38;5;12"
func (c Color256) String() string {
if c[1] == AsFg { // 0 is Fg
// return fmt.Sprintf(TplFg256, c[0])
return Fg256Pfx + strconv.Itoa(int(c[0]))
}
if c[1] == AsBg { // 1 is Bg
// return fmt.Sprintf(TplBg256, c[0])
return Bg256Pfx + strconv.Itoa(int(c[0]))
}
return "" // empty
}
// IsFg color
func (c Color256) IsFg() bool {
return c[1] == AsFg
}
// ToFg 256 color
func (c Color256) ToFg() Color256 {
c[1] = AsFg
return c
}
// IsBg color
func (c Color256) IsBg() bool {
return c[1] == AsBg
}
// ToBg 256 color
func (c Color256) ToBg() Color256 {
c[1] = AsBg
return c
}
// IsEmpty value
func (c Color256) IsEmpty() bool {
return c[1] > 1
}
/*************************************************************
* 8bit(256) Style
*************************************************************/
// Style256 definition
//
// 前/背景色
// 都是由两位uint8组成, 第一位是色彩值;
// 第二位与 Bit8Color 不一样的是,在这里表示是否设置了值 0 未设置 !=0 已设置
type Style256 struct {
// p Printer
// Name of the style
Name string
// color options of the style
opts Opts
// fg and bg color
fg, bg Color256
}
// S256 create a color256 style
// Usage:
// s := color.S256()
// s := color.S256(132) // fg
// s := color.S256(132, 203) // fg and bg
func S256(fgAndBg ...uint8) *Style256 {
s := &Style256{}
vl := len(fgAndBg)
if vl > 0 { // with fg
s.fg = Color256{fgAndBg[0], 1}
if vl > 1 { // and with bg
s.bg = Color256{fgAndBg[1], 1}
}
}
return s
}
// Set fg and bg color value, can also with color options
func (s *Style256) Set(fgVal, bgVal uint8, opts ...Color) *Style256 {
s.fg = Color256{fgVal, 1}
s.bg = Color256{bgVal, 1}
s.opts.Add(opts...)
return s
}
// SetBg set bg color value
func (s *Style256) SetBg(bgVal uint8) *Style256 {
s.bg = Color256{bgVal, 1}
return s
}
// SetFg set fg color value
func (s *Style256) SetFg(fgVal uint8) *Style256 {
s.fg = Color256{fgVal, 1}
return s
}
// SetOpts set options
func (s *Style256) SetOpts(opts Opts) *Style256 {
s.opts = opts
return s
}
// AddOpts add options
func (s *Style256) AddOpts(opts ...Color) *Style256 {
s.opts.Add(opts...)
return s
}
// Print message
func (s *Style256) Print(a ...interface{}) {
doPrintV2(s.String(), fmt.Sprint(a...))
}
// Printf format and print message
func (s *Style256) Printf(format string, a ...interface{}) {
doPrintV2(s.String(), fmt.Sprintf(format, a...))
}
// Println print message with newline
func (s *Style256) Println(a ...interface{}) {
doPrintlnV2(s.String(), a)
}
// Sprint returns rendered message
func (s *Style256) Sprint(a ...interface{}) string {
return RenderCode(s.Code(), a...)
}
// Sprintf returns format and rendered message
func (s *Style256) Sprintf(format string, a ...interface{}) string {
return RenderString(s.Code(), fmt.Sprintf(format, a...))
}
// Code convert to color code string
func (s *Style256) Code() string {
return s.String()
}
// String convert to color code string
func (s *Style256) String() string {
var ss []string
if s.fg[1] > 0 {
ss = append(ss, fmt.Sprintf(TplFg256, s.fg[0]))
}
if s.bg[1] > 0 {
ss = append(ss, fmt.Sprintf(TplBg256, s.bg[0]))
}
if s.opts.IsValid() {
ss = append(ss, s.opts.String())
}
return strings.Join(ss, ";")
}

View File

@@ -1,391 +0,0 @@
package color
import (
"fmt"
"strconv"
"strings"
)
// 24 bit RGB color
// RGB:
// R 0-255 G 0-255 B 0-255
// R 00-FF G 00-FF B 00-FF (16进制)
//
// Format:
// ESC[ … 38;2;<r>;<g>;<b> … m // Select RGB foreground color
// ESC[ … 48;2;<r>;<g>;<b> … m // Choose RGB background color
//
// links:
// https://zh.wikipedia.org/wiki/ANSI%E8%BD%AC%E4%B9%89%E5%BA%8F%E5%88%97#24位
//
// example:
// fg: \x1b[38;2;30;144;255mMESSAGE\x1b[0m
// bg: \x1b[48;2;30;144;255mMESSAGE\x1b[0m
// both: \x1b[38;2;233;90;203;48;2;30;144;255mMESSAGE\x1b[0m
const (
TplFgRGB = "38;2;%d;%d;%d"
TplBgRGB = "48;2;%d;%d;%d"
FgRGBPfx = "38;2;"
BgRGBPfx = "48;2;"
)
// mark color is fg or bg.
const (
AsFg uint8 = iota
AsBg
)
// values from https://github.com/go-terminfo/terminfo
// var (
// RgbaBlack = image_color.RGBA{0, 0, 0, 255}
// Red = color.RGBA{205, 0, 0, 255}
// Green = color.RGBA{0, 205, 0, 255}
// Orange = color.RGBA{205, 205, 0, 255}
// Blue = color.RGBA{0, 0, 238, 255}
// Magenta = color.RGBA{205, 0, 205, 255}
// Cyan = color.RGBA{0, 205, 205, 255}
// LightGrey = color.RGBA{229, 229, 229, 255}
//
// DarkGrey = color.RGBA{127, 127, 127, 255}
// LightRed = color.RGBA{255, 0, 0, 255}
// LightGreen = color.RGBA{0, 255, 0, 255}
// Yellow = color.RGBA{255, 255, 0, 255}
// LightBlue = color.RGBA{92, 92, 255, 255}
// LightMagenta = color.RGBA{255, 0, 255, 255}
// LightCyan = color.RGBA{0, 255, 255, 255}
// White = color.RGBA{255, 255, 255, 255}
// )
/*************************************************************
* RGB Color(Bit24Color, TrueColor)
*************************************************************/
// RGBColor definition.
//
// The first to third digits represent the color value.
// The last digit represents the foreground(0), background(1), >1 is unset value
//
// Usage:
// // 0, 1, 2 is R,G,B.
// // 3rd: Fg=0, Bg=1, >1: unset value
// RGBColor{30,144,255, 0}
// RGBColor{30,144,255, 1}
//
// NOTICE: now support RGB color on windows CMD, PowerShell
type RGBColor [4]uint8
// create a empty RGBColor
var emptyRGBColor = RGBColor{3: 99}
// RGB color create.
// Usage:
// c := RGB(30,144,255)
// c := RGB(30,144,255, true)
// c.Print("message")
func RGB(r, g, b uint8, isBg ...bool) RGBColor {
rgb := RGBColor{r, g, b}
if len(isBg) > 0 && isBg[0] {
rgb[3] = AsBg
}
return rgb
}
// Rgb alias of the RGB()
func Rgb(r, g, b uint8, isBg ...bool) RGBColor { return RGB(r, g, b, isBg...) }
// Bit24 alias of the RGB()
func Bit24(r, g, b uint8, isBg ...bool) RGBColor { return RGB(r, g, b, isBg...) }
// RGBFromSlice quick RGBColor from slice
func RGBFromSlice(rgb []uint8, isBg ...bool) RGBColor {
return RGB(rgb[0], rgb[1], rgb[2], isBg...)
}
// HEX create RGB color from a HEX color string.
// Usage:
// c := HEX("ccc") // rgb: [204 204 204]
// c := HEX("aabbcc") // rgb: [170 187 204]
// c := HEX("#aabbcc")
// c := HEX("0xaabbcc")
// c.Print("message")
func HEX(hex string, isBg ...bool) RGBColor {
if rgb := HexToRgb(hex); len(rgb) > 0 {
return RGB(uint8(rgb[0]), uint8(rgb[1]), uint8(rgb[2]), isBg...)
}
// mark is empty
return emptyRGBColor
}
// Hex alias of the HEX()
func Hex(hex string, isBg ...bool) RGBColor { return HEX(hex, isBg...) }
// RGBFromString create RGB color from a string.
// Usage:
// c := RGBFromString("170,187,204")
// c.Print("message")
func RGBFromString(rgb string, isBg ...bool) RGBColor {
ss := stringToArr(rgb, ",")
if len(ss) != 3 {
return emptyRGBColor
}
var ar [3]int
for i, val := range ss {
iv, err := strconv.Atoi(val)
if err != nil {
return emptyRGBColor
}
ar[i] = iv
}
return RGB(uint8(ar[0]), uint8(ar[1]), uint8(ar[2]), isBg...)
}
// Set terminal by rgb/true color code
func (c RGBColor) Set() error {
return SetTerminal(c.String())
}
// Reset terminal. alias of the ResetTerminal()
func (c RGBColor) Reset() error {
return ResetTerminal()
}
// Print print message
func (c RGBColor) Print(a ...interface{}) {
doPrintV2(c.String(), fmt.Sprint(a...))
}
// Printf format and print message
func (c RGBColor) Printf(format string, a ...interface{}) {
doPrintV2(c.String(), fmt.Sprintf(format, a...))
}
// Println print message with newline
func (c RGBColor) Println(a ...interface{}) {
doPrintlnV2(c.String(), a)
}
// Sprint returns rendered message
func (c RGBColor) Sprint(a ...interface{}) string {
return RenderCode(c.String(), a...)
}
// Sprintf returns format and rendered message
func (c RGBColor) Sprintf(format string, a ...interface{}) string {
return RenderString(c.String(), fmt.Sprintf(format, a...))
}
// Values to RGB values
func (c RGBColor) Values() []int {
return []int{int(c[0]), int(c[1]), int(c[2])}
}
// Code to color code string without prefix. eg: "204;123;56"
func (c RGBColor) Code() string {
return fmt.Sprintf("%d;%d;%d", c[0], c[1], c[2])
}
// Hex color rgb to hex string. as in "ff0080".
func (c RGBColor) Hex() string {
return fmt.Sprintf("%02x%02x%02x", c[0], c[1], c[2])
}
// FullCode to color code string with prefix
func (c RGBColor) FullCode() string {
return c.String()
}
// String to color code string with prefix. eg: "38;2;204;123;56"
func (c RGBColor) String() string {
if c[3] == AsFg {
return fmt.Sprintf(TplFgRGB, c[0], c[1], c[2])
}
if c[3] == AsBg {
return fmt.Sprintf(TplBgRGB, c[0], c[1], c[2])
}
// c[3] > 1 is empty
return ""
}
// IsEmpty value
func (c RGBColor) IsEmpty() bool {
return c[3] > AsBg
}
// IsValid value
// func (c RGBColor) IsValid() bool {
// return c[3] <= AsBg
// }
// C256 returns the closest approximate 256 (8 bit) color
func (c RGBColor) C256() Color256 {
return C256(RgbTo256(c[0], c[1], c[2]), c[3] == AsBg)
}
// Basic returns the closest approximate 16 (4 bit) color
func (c RGBColor) Basic() Color {
// return Color(RgbToAnsi(c[0], c[1], c[2], c[3] == AsBg))
return Color(Rgb2basic(c[0], c[1], c[2], c[3] == AsBg))
}
// Color returns the closest approximate 16 (4 bit) color
func (c RGBColor) Color() Color { return c.Basic() }
// C16 returns the closest approximate 16 (4 bit) color
func (c RGBColor) C16() Color { return c.Basic() }
/*************************************************************
* RGB Style
*************************************************************/
// RGBStyle definition.
//
// Foreground/Background color
// All are composed of 4 digits uint8, the first three digits are the color value;
// The last bit is different from RGBColor, here it indicates whether the value is set.
// - 1 Has been set
// - ^1 Not set
type RGBStyle struct {
// Name of the style
Name string
// color options of the style
opts Opts
// fg and bg color
fg, bg RGBColor
}
// NewRGBStyle create a RGBStyle.
func NewRGBStyle(fg RGBColor, bg ...RGBColor) *RGBStyle {
s := &RGBStyle{}
if len(bg) > 0 {
s.SetBg(bg[0])
}
return s.SetFg(fg)
}
// HEXStyle create a RGBStyle from HEX color string.
// Usage:
// s := HEXStyle("aabbcc", "eee")
// s.Print("message")
func HEXStyle(fg string, bg ...string) *RGBStyle {
s := &RGBStyle{}
if len(bg) > 0 {
s.SetBg(HEX(bg[0]))
}
if len(fg) > 0 {
s.SetFg(HEX(fg))
}
return s
}
// RGBStyleFromString create a RGBStyle from color value string.
// Usage:
// s := RGBStyleFromString("170,187,204", "70,87,4")
// s.Print("message")
func RGBStyleFromString(fg string, bg ...string) *RGBStyle {
s := &RGBStyle{}
if len(bg) > 0 {
s.SetBg(RGBFromString(bg[0]))
}
return s.SetFg(RGBFromString(fg))
}
// Set fg and bg color, can also with color options
func (s *RGBStyle) Set(fg, bg RGBColor, opts ...Color) *RGBStyle {
return s.SetFg(fg).SetBg(bg).SetOpts(opts)
}
// SetFg set fg color
func (s *RGBStyle) SetFg(fg RGBColor) *RGBStyle {
fg[3] = 1 // add fixed value, mark is valid
s.fg = fg
return s
}
// SetBg set bg color
func (s *RGBStyle) SetBg(bg RGBColor) *RGBStyle {
bg[3] = 1 // add fixed value, mark is valid
s.bg = bg
return s
}
// SetOpts set color options
func (s *RGBStyle) SetOpts(opts Opts) *RGBStyle {
s.opts = opts
return s
}
// AddOpts add options
func (s *RGBStyle) AddOpts(opts ...Color) *RGBStyle {
s.opts.Add(opts...)
return s
}
// Print print message
func (s *RGBStyle) Print(a ...interface{}) {
doPrintV2(s.String(), fmt.Sprint(a...))
}
// Printf format and print message
func (s *RGBStyle) Printf(format string, a ...interface{}) {
doPrintV2(s.String(), fmt.Sprintf(format, a...))
}
// Println print message with newline
func (s *RGBStyle) Println(a ...interface{}) {
doPrintlnV2(s.String(), a)
}
// Sprint returns rendered message
func (s *RGBStyle) Sprint(a ...interface{}) string {
return RenderCode(s.String(), a...)
}
// Sprintf returns format and rendered message
func (s *RGBStyle) Sprintf(format string, a ...interface{}) string {
return RenderString(s.String(), fmt.Sprintf(format, a...))
}
// Code convert to color code string
func (s *RGBStyle) Code() string {
return s.String()
}
// FullCode convert to color code string
func (s *RGBStyle) FullCode() string {
return s.String()
}
// String convert to color code string
func (s *RGBStyle) String() string {
var ss []string
// last value ensure is enable.
if s.fg[3] == 1 {
ss = append(ss, fmt.Sprintf(TplFgRGB, s.fg[0], s.fg[1], s.fg[2]))
}
if s.bg[3] == 1 {
ss = append(ss, fmt.Sprintf(TplBgRGB, s.bg[0], s.bg[1], s.bg[2]))
}
if s.opts.IsValid() {
ss = append(ss, s.opts.String())
}
return strings.Join(ss, ";")
}
// IsEmpty style
func (s *RGBStyle) IsEmpty() bool {
return s.fg[3] != 1 && s.bg[3] != 1
}

View File

@@ -1,427 +0,0 @@
package color
import (
"fmt"
"regexp"
"strings"
)
// output colored text like use html tag. (not support windows cmd)
const (
// MatchExpr regex to match color tags
//
// Notice: golang 不支持反向引用. 即不支持使用 \1 引用第一个匹配 ([a-z=;]+)
// MatchExpr = `<([a-z=;]+)>(.*?)<\/\1>`
// 所以调整一下 统一使用 `</>` 来结束标签,例如 "<info>some text</>"
//
// allow custom attrs, eg: "<fg=white;bg=blue;op=bold>content</>"
// (?s:...) s - 让 "." 匹配换行
MatchExpr = `<([0-9a-zA-Z_=,;]+)>(?s:(.*?))<\/>`
// AttrExpr regex to match custom color attributes
// eg: "<fg=white;bg=blue;op=bold>content</>"
AttrExpr = `(fg|bg|op)[\s]*=[\s]*([0-9a-zA-Z,]+);?`
// StripExpr regex used for removing color tags
// StripExpr = `<[\/]?[a-zA-Z=;]+>`
// 随着上面的做一些调整
StripExpr = `<[\/]?[0-9a-zA-Z_=,;]*>`
)
var (
attrRegex = regexp.MustCompile(AttrExpr)
matchRegex = regexp.MustCompile(MatchExpr)
stripRegex = regexp.MustCompile(StripExpr)
)
/*************************************************************
* internal defined color tags
*************************************************************/
// There are internal defined color tags
// Usage: <tag>content text</>
// @notice 加 0 在前面是为了防止之前的影响到现在的设置
var colorTags = map[string]string{
// basic tags
"red": "0;31",
"red1": "1;31", // with bold
"redB": "1;31",
"red_b": "1;31",
"blue": "0;34",
"blue1": "1;34", // with bold
"blueB": "1;34",
"blue_b": "1;34",
"cyan": "0;36",
"cyan1": "1;36", // with bold
"cyanB": "1;36",
"cyan_b": "1;36",
"green": "0;32",
"green1": "1;32", // with bold
"greenB": "1;32",
"green_b": "1;32",
"black": "0;30",
"white": "1;37",
"default": "0;39", // no color
"normal": "0;39", // no color
"brown": "0;33", // #A52A2A
"yellow": "0;33",
"ylw0": "0;33",
"yellowB": "1;33", // with bold
"ylw1": "1;33",
"ylwB": "1;33",
"magenta": "0;35",
"mga": "0;35", // short name
"magentaB": "1;35", // with bold
"mgb": "1;35",
"mgaB": "1;35",
// light/hi tags
"gray": "0;90",
"darkGray": "0;90",
"dark_gray": "0;90",
"lightYellow": "0;93",
"light_yellow": "0;93",
"hiYellow": "0;93",
"hi_yellow": "0;93",
"hiYellowB": "1;93", // with bold
"hi_yellow_b": "1;93",
"lightMagenta": "0;95",
"light_magenta": "0;95",
"hiMagenta": "0;95",
"hi_magenta": "0;95",
"lightMagentaB": "1;95", // with bold
"hiMagentaB": "1;95", // with bold
"hi_magenta_b": "1;95",
"lightRed": "0;91",
"light_red": "0;91",
"hiRed": "0;91",
"hi_red": "0;91",
"lightRedB": "1;91", // with bold
"light_red_b": "1;91",
"hi_red_b": "1;91",
"lightGreen": "0;92",
"light_green": "0;92",
"hiGreen": "0;92",
"hi_green": "0;92",
"lightGreenB": "1;92",
"light_green_b": "1;92",
"hi_green_b": "1;92",
"lightBlue": "0;94",
"light_blue": "0;94",
"hiBlue": "0;94",
"hi_blue": "0;94",
"lightBlueB": "1;94",
"light_blue_b": "1;94",
"hi_blue_b": "1;94",
"lightCyan": "0;96",
"light_cyan": "0;96",
"hiCyan": "0;96",
"hi_cyan": "0;96",
"lightCyanB": "1;96",
"light_cyan_b": "1;96",
"hi_cyan_b": "1;96",
"lightWhite": "0;97;40",
"light_white": "0;97;40",
// option
"bold": "1",
"b": "1",
"underscore": "4",
"us": "4", // short name for 'underscore'
"reverse": "7",
// alert tags, like bootstrap's alert
"suc": "1;32", // same "green" and "bold"
"success": "1;32",
"info": "0;32", // same "green",
"comment": "0;33", // same "brown"
"note": "36;1",
"notice": "36;4",
"warn": "0;1;33",
"warning": "0;30;43",
"primary": "0;34",
"danger": "1;31", // same "red" but add bold
"err": "97;41",
"error": "97;41", // fg light white; bg red
}
/*************************************************************
* parse color tags
*************************************************************/
var (
tagParser = TagParser{}
rxNumStr = regexp.MustCompile("^[0-9]{1,3}$")
rxHexCode = regexp.MustCompile("^#?([0-9a-fA-F]{3}|[0-9a-fA-F]{6})$")
)
// TagParser struct
type TagParser struct {
disable bool
}
// NewTagParser create
func NewTagParser() *TagParser {
return &TagParser{}
}
// func (tp *TagParser) Disable() *TagParser {
// tp.disable = true
// return tp
// }
// ParseByEnv parse given string. will check package setting.
func (tp *TagParser) ParseByEnv(str string) string {
// disable handler TAG
if !RenderTag {
return str
}
// disable OR not support color
if !Enable || !SupportColor() {
return ClearTag(str)
}
return tp.Parse(str)
}
// Parse parse given string, replace color tag and return rendered string
func (tp *TagParser) Parse(str string) string {
// not contains color tag
if !strings.Contains(str, "</>") {
return str
}
// find color tags by regex. str eg: "<fg=white;bg=blue;op=bold>content</>"
matched := matchRegex.FindAllStringSubmatch(str, -1)
// item: 0 full text 1 tag name 2 tag content
for _, item := range matched {
full, tag, content := item[0], item[1], item[2]
// use defined tag name: "<info>content</>" -> tag: "info"
if !strings.ContainsRune(tag, '=') {
code := colorTags[tag]
if len(code) > 0 {
now := RenderString(code, content)
// old := WrapTag(content, tag) is equals to var 'full'
str = strings.Replace(str, full, now, 1)
}
continue
}
// custom color in tag
// - basic: "fg=white;bg=blue;op=bold"
if code := ParseCodeFromAttr(tag); len(code) > 0 {
now := RenderString(code, content)
str = strings.Replace(str, full, now, 1)
}
}
return str
}
// func (tp *TagParser) ParseAttr(attr string) (code string) {
// return
// }
// ReplaceTag parse string, replace color tag and return rendered string
func ReplaceTag(str string) string {
return tagParser.ParseByEnv(str)
}
// ParseCodeFromAttr parse color attributes.
//
// attr format:
// // VALUE please see var: FgColors, BgColors, AllOptions
// "fg=VALUE;bg=VALUE;op=VALUE"
// 16 color:
// "fg=yellow"
// "bg=red"
// "op=bold,underscore" option is allow multi value
// "fg=white;bg=blue;op=bold"
// "fg=white;op=bold,underscore"
// 256 color:
// "fg=167"
// "fg=167;bg=23"
// "fg=167;bg=23;op=bold"
// true color:
// // hex
// "fg=fc1cac"
// "fg=fc1cac;bg=c2c3c4"
// // r,g,b
// "fg=23,45,214"
// "fg=23,45,214;bg=109,99,88"
func ParseCodeFromAttr(attr string) (code string) {
if !strings.ContainsRune(attr, '=') {
return
}
attr = strings.Trim(attr, ";=,")
if len(attr) == 0 {
return
}
var codes []string
matched := attrRegex.FindAllStringSubmatch(attr, -1)
for _, item := range matched {
pos, val := item[1], item[2]
switch pos {
case "fg":
if c, ok := FgColors[val]; ok { // basic
codes = append(codes, c.String())
} else if c, ok := ExFgColors[val]; ok { // extra
codes = append(codes, c.String())
} else if code := rgbHex256toCode(val, false); code != "" {
codes = append(codes, code)
}
case "bg":
if c, ok := BgColors[val]; ok { // basic bg
codes = append(codes, c.String())
} else if c, ok := ExBgColors[val]; ok { // extra bg
codes = append(codes, c.String())
} else if code := rgbHex256toCode(val, true); code != "" {
codes = append(codes, code)
}
case "op": // options allow multi value
if strings.Contains(val, ",") {
ns := strings.Split(val, ",")
for _, n := range ns {
if c, ok := AllOptions[n]; ok {
codes = append(codes, c.String())
}
}
} else if c, ok := AllOptions[val]; ok {
codes = append(codes, c.String())
}
}
}
return strings.Join(codes, ";")
}
func rgbHex256toCode(val string, isBg bool) (code string) {
if len(val) == 6 && rxHexCode.MatchString(val) { // hex: "fc1cac"
code = HEX(val, isBg).String()
} else if strings.ContainsRune(val, ',') { // rgb: "231,178,161"
code = strings.Replace(val, ",", ";", -1)
if isBg {
code = BgRGBPfx + code
} else {
code = FgRGBPfx + code
}
} else if len(val) < 4 && rxNumStr.MatchString(val) { // 256 code
if isBg {
code = Bg256Pfx + val
} else {
code = Fg256Pfx + val
}
}
return
}
// ClearTag clear all tag for a string
func ClearTag(s string) string {
if !strings.Contains(s, "</>") {
return s
}
return stripRegex.ReplaceAllString(s, "")
}
/*************************************************************
* helper methods
*************************************************************/
// GetTagCode get color code by tag name
func GetTagCode(name string) string {
return colorTags[name]
}
// ApplyTag for messages
func ApplyTag(tag string, a ...interface{}) string {
return RenderCode(GetTagCode(tag), a...)
}
// WrapTag wrap a tag for a string "<tag>content</>"
func WrapTag(s string, tag string) string {
if s == "" || tag == "" {
return s
}
return fmt.Sprintf("<%s>%s</>", tag, s)
}
// GetColorTags get all internal color tags
func GetColorTags() map[string]string {
return colorTags
}
// IsDefinedTag is defined tag name
func IsDefinedTag(name string) bool {
_, ok := colorTags[name]
return ok
}
/*************************************************************
* Tag extra
*************************************************************/
// Tag value is a defined style name
// Usage:
// Tag("info").Println("message")
type Tag string
// Print messages
func (tg Tag) Print(a ...interface{}) {
name := string(tg)
str := fmt.Sprint(a...)
if stl := GetStyle(name); !stl.IsEmpty() {
stl.Print(str)
} else {
doPrintV2(GetTagCode(name), str)
}
}
// Printf format and print messages
func (tg Tag) Printf(format string, a ...interface{}) {
name := string(tg)
str := fmt.Sprintf(format, a...)
if stl := GetStyle(name); !stl.IsEmpty() {
stl.Print(str)
} else {
doPrintV2(GetTagCode(name), str)
}
}
// Println messages line
func (tg Tag) Println(a ...interface{}) {
name := string(tg)
if stl := GetStyle(name); !stl.IsEmpty() {
stl.Println(a...)
} else {
doPrintlnV2(GetTagCode(name), a)
}
}
// Sprint render messages
func (tg Tag) Sprint(a ...interface{}) string {
name := string(tg)
// if stl := GetStyle(name); !stl.IsEmpty() {
// return stl.Render(args...)
// }
return RenderCode(GetTagCode(name), a...)
}
// Sprintf format and render messages
func (tg Tag) Sprintf(format string, a ...interface{}) string {
tag := string(tg)
str := fmt.Sprintf(format, a...)
return RenderString(GetTagCode(tag), str)
}

View File

@@ -1,593 +0,0 @@
package color
import (
"fmt"
"math"
"strconv"
"strings"
)
var (
// ---------- basic(16) <=> 256 color convert ----------
basicTo256Map = map[uint8]uint8{
30: 0, // black 000000
31: 160, // red c51e14
32: 34, // green 1dc121
33: 184, // yellow c7c329
34: 20, // blue 0a2fc4
35: 170, // magenta c839c5
36: 44, // cyan 20c5c6
37: 188, // white c7c7c7
90: 59, // lightBlack 686868
91: 203, // lightRed fd6f6b
92: 83, // lightGreen 67f86f
93: 227, // lightYellow fffa72
94: 69, // lightBlue 6a76fb
95: 213, // lightMagenta fd7cfc
96: 87, // lightCyan 68fdfe
97: 15, // lightWhite ffffff
}
// ---------- basic(16) <=> RGB color convert ----------
// refer from Hyper app
basic2hexMap = map[uint8]string{
30: "000000", // black
31: "c51e14", // red
32: "1dc121", // green
33: "c7c329", // yellow
34: "0a2fc4", // blue
35: "c839c5", // magenta
36: "20c5c6", // cyan
37: "c7c7c7", // white
90: "686868", // lightBlack/darkGray
91: "fd6f6b", // lightRed
92: "67f86f", // lightGreen
93: "fffa72", // lightYellow
94: "6a76fb", // lightBlue
95: "fd7cfc", // lightMagenta
96: "68fdfe", // lightCyan
97: "ffffff", // lightWhite
}
// will convert data from basic2hexMap
hex2basicMap = initHex2basicMap()
// ---------- 256 <=> RGB color convert ----------
// adapted from https://gist.github.com/MicahElliott/719710
c256ToHexMap = init256ToHexMap()
// rgb to 256 color look-up table
// RGB hex => 256 code
hexTo256Table = map[string]uint8{
// Primary 3-bit (8 colors). Unique representation!
"000000": 0,
"800000": 1,
"008000": 2,
"808000": 3,
"000080": 4,
"800080": 5,
"008080": 6,
"c0c0c0": 7,
// Equivalent "bright" versions of original 8 colors.
"808080": 8,
"ff0000": 9,
"00ff00": 10,
"ffff00": 11,
"0000ff": 12,
"ff00ff": 13,
"00ffff": 14,
"ffffff": 15,
// values commented out below are duplicates from the prior sections
// Strictly ascending.
// "000000": 16,
"000001": 16, // up: avoid key conflicts, value + 1
"00005f": 17,
"000087": 18,
"0000af": 19,
"0000d7": 20,
// "0000ff": 21,
"0000fe": 21, // up: avoid key conflicts, value - 1
"005f00": 22,
"005f5f": 23,
"005f87": 24,
"005faf": 25,
"005fd7": 26,
"005fff": 27,
"008700": 28,
"00875f": 29,
"008787": 30,
"0087af": 31,
"0087d7": 32,
"0087ff": 33,
"00af00": 34,
"00af5f": 35,
"00af87": 36,
"00afaf": 37,
"00afd7": 38,
"00afff": 39,
"00d700": 40,
"00d75f": 41,
"00d787": 42,
"00d7af": 43,
"00d7d7": 44,
"00d7ff": 45,
// "00ff00": 46,
"00ff01": 46, // up: avoid key conflicts, value + 1
"00ff5f": 47,
"00ff87": 48,
"00ffaf": 49,
"00ffd7": 50,
// "00ffff": 51,
"00fffe": 51, // up: avoid key conflicts, value - 1
"5f0000": 52,
"5f005f": 53,
"5f0087": 54,
"5f00af": 55,
"5f00d7": 56,
"5f00ff": 57,
"5f5f00": 58,
"5f5f5f": 59,
"5f5f87": 60,
"5f5faf": 61,
"5f5fd7": 62,
"5f5fff": 63,
"5f8700": 64,
"5f875f": 65,
"5f8787": 66,
"5f87af": 67,
"5f87d7": 68,
"5f87ff": 69,
"5faf00": 70,
"5faf5f": 71,
"5faf87": 72,
"5fafaf": 73,
"5fafd7": 74,
"5fafff": 75,
"5fd700": 76,
"5fd75f": 77,
"5fd787": 78,
"5fd7af": 79,
"5fd7d7": 80,
"5fd7ff": 81,
"5fff00": 82,
"5fff5f": 83,
"5fff87": 84,
"5fffaf": 85,
"5fffd7": 86,
"5fffff": 87,
"870000": 88,
"87005f": 89,
"870087": 90,
"8700af": 91,
"8700d7": 92,
"8700ff": 93,
"875f00": 94,
"875f5f": 95,
"875f87": 96,
"875faf": 97,
"875fd7": 98,
"875fff": 99,
"878700": 100,
"87875f": 101,
"878787": 102,
"8787af": 103,
"8787d7": 104,
"8787ff": 105,
"87af00": 106,
"87af5f": 107,
"87af87": 108,
"87afaf": 109,
"87afd7": 110,
"87afff": 111,
"87d700": 112,
"87d75f": 113,
"87d787": 114,
"87d7af": 115,
"87d7d7": 116,
"87d7ff": 117,
"87ff00": 118,
"87ff5f": 119,
"87ff87": 120,
"87ffaf": 121,
"87ffd7": 122,
"87ffff": 123,
"af0000": 124,
"af005f": 125,
"af0087": 126,
"af00af": 127,
"af00d7": 128,
"af00ff": 129,
"af5f00": 130,
"af5f5f": 131,
"af5f87": 132,
"af5faf": 133,
"af5fd7": 134,
"af5fff": 135,
"af8700": 136,
"af875f": 137,
"af8787": 138,
"af87af": 139,
"af87d7": 140,
"af87ff": 141,
"afaf00": 142,
"afaf5f": 143,
"afaf87": 144,
"afafaf": 145,
"afafd7": 146,
"afafff": 147,
"afd700": 148,
"afd75f": 149,
"afd787": 150,
"afd7af": 151,
"afd7d7": 152,
"afd7ff": 153,
"afff00": 154,
"afff5f": 155,
"afff87": 156,
"afffaf": 157,
"afffd7": 158,
"afffff": 159,
"d70000": 160,
"d7005f": 161,
"d70087": 162,
"d700af": 163,
"d700d7": 164,
"d700ff": 165,
"d75f00": 166,
"d75f5f": 167,
"d75f87": 168,
"d75faf": 169,
"d75fd7": 170,
"d75fff": 171,
"d78700": 172,
"d7875f": 173,
"d78787": 174,
"d787af": 175,
"d787d7": 176,
"d787ff": 177,
"d7af00": 178,
"d7af5f": 179,
"d7af87": 180,
"d7afaf": 181,
"d7afd7": 182,
"d7afff": 183,
"d7d700": 184,
"d7d75f": 185,
"d7d787": 186,
"d7d7af": 187,
"d7d7d7": 188,
"d7d7ff": 189,
"d7ff00": 190,
"d7ff5f": 191,
"d7ff87": 192,
"d7ffaf": 193,
"d7ffd7": 194,
"d7ffff": 195,
// "ff0000": 196,
"ff0001": 196, // up: avoid key conflicts, value + 1
"ff005f": 197,
"ff0087": 198,
"ff00af": 199,
"ff00d7": 200,
// "ff00ff": 201,
"ff00fe": 201, // up: avoid key conflicts, value - 1
"ff5f00": 202,
"ff5f5f": 203,
"ff5f87": 204,
"ff5faf": 205,
"ff5fd7": 206,
"ff5fff": 207,
"ff8700": 208,
"ff875f": 209,
"ff8787": 210,
"ff87af": 211,
"ff87d7": 212,
"ff87ff": 213,
"ffaf00": 214,
"ffaf5f": 215,
"ffaf87": 216,
"ffafaf": 217,
"ffafd7": 218,
"ffafff": 219,
"ffd700": 220,
"ffd75f": 221,
"ffd787": 222,
"ffd7af": 223,
"ffd7d7": 224,
"ffd7ff": 225,
// "ffff00": 226,
"ffff01": 226, // up: avoid key conflicts, value + 1
"ffff5f": 227,
"ffff87": 228,
"ffffaf": 229,
"ffffd7": 230,
// "ffffff": 231,
"fffffe": 231, // up: avoid key conflicts, value - 1
// Gray-scale range.
"080808": 232,
"121212": 233,
"1c1c1c": 234,
"262626": 235,
"303030": 236,
"3a3a3a": 237,
"444444": 238,
"4e4e4e": 239,
"585858": 240,
"626262": 241,
"6c6c6c": 242,
"767676": 243,
// "808080": 244,
"808081": 244, // up: avoid key conflicts, value + 1
"8a8a8a": 245,
"949494": 246,
"9e9e9e": 247,
"a8a8a8": 248,
"b2b2b2": 249,
"bcbcbc": 250,
"c6c6c6": 251,
"d0d0d0": 252,
"dadada": 253,
"e4e4e4": 254,
"eeeeee": 255,
}
incs = []uint8{0x00, 0x5f, 0x87, 0xaf, 0xd7, 0xff}
)
func initHex2basicMap() map[string]uint8 {
h2b := make(map[string]uint8, len(basic2hexMap))
// ini data map
for u, s := range basic2hexMap {
h2b[s] = u
}
return h2b
}
func init256ToHexMap() map[uint8]string {
c256toh := make(map[uint8]string, len(hexTo256Table))
// ini data map
for hex, c256 := range hexTo256Table {
c256toh[c256] = hex
}
return c256toh
}
// RgbTo256Table mapping data
func RgbTo256Table() map[string]uint8 {
return hexTo256Table
}
// Colors2code convert colors to code. return like "32;45;3"
func Colors2code(colors ...Color) string {
if len(colors) == 0 {
return ""
}
var codes []string
for _, color := range colors {
codes = append(codes, color.String())
}
return strings.Join(codes, ";")
}
/*************************************************************
* HEX code <=> RGB/True color code
*************************************************************/
// Hex2rgb alias of the HexToRgb()
func Hex2rgb(hex string) []int { return HexToRgb(hex) }
// HexToRGB alias of the HexToRgb()
func HexToRGB(hex string) []int { return HexToRgb(hex) }
// HexToRgb convert hex color string to RGB numbers
//
// Usage:
// rgb := HexToRgb("ccc") // rgb: [204 204 204]
// rgb := HexToRgb("aabbcc") // rgb: [170 187 204]
// rgb := HexToRgb("#aabbcc") // rgb: [170 187 204]
// rgb := HexToRgb("0xad99c0") // rgb: [170 187 204]
func HexToRgb(hex string) (rgb []int) {
hex = strings.TrimSpace(hex)
if hex == "" {
return
}
// like from css. eg "#ccc" "#ad99c0"
if hex[0] == '#' {
hex = hex[1:]
}
hex = strings.ToLower(hex)
switch len(hex) {
case 3: // "ccc"
hex = string([]byte{hex[0], hex[0], hex[1], hex[1], hex[2], hex[2]})
case 8: // "0xad99c0"
hex = strings.TrimPrefix(hex, "0x")
}
// recheck
if len(hex) != 6 {
return
}
// convert string to int64
if i64, err := strconv.ParseInt(hex, 16, 32); err == nil {
color := int(i64)
// parse int
rgb = make([]int, 3)
rgb[0] = color >> 16
rgb[1] = (color & 0x00FF00) >> 8
rgb[2] = color & 0x0000FF
}
return
}
// Rgb2hex alias of the RgbToHex()
func Rgb2hex(rgb []int) string { return RgbToHex(rgb) }
// RgbToHex convert RGB-code to hex-code
//
// Usage:
// hex := RgbToHex([]int{170, 187, 204}) // hex: "aabbcc"
func RgbToHex(rgb []int) string {
hexNodes := make([]string, len(rgb))
for _, v := range rgb {
hexNodes = append(hexNodes, strconv.FormatInt(int64(v), 16))
}
return strings.Join(hexNodes, "")
}
/*************************************************************
* 4bit(16) color <=> RGB/True color
*************************************************************/
// Basic2hex convert basic color to hex string.
func Basic2hex(val uint8) string {
return basic2hexMap[val]
}
// Hex2basic convert hex string to basic color code.
func Hex2basic(hex string) uint8 {
return hex2basicMap[hex]
}
// Rgb2basic alias of the RgbToAnsi()
func Rgb2basic(r, g, b uint8, isBg bool) uint8 {
// is basic color, direct use static map data.
hex := RgbToHex([]int{int(r), int(g), int(b)})
if val, ok := hex2basicMap[hex]; ok {
if isBg {
return val + 10
}
return val
}
return RgbToAnsi(r, g, b, isBg)
}
// Rgb2ansi alias of the RgbToAnsi()
func Rgb2ansi(r, g, b uint8, isBg bool) uint8 {
return RgbToAnsi(r, g, b, isBg)
}
// RgbToAnsi convert RGB-code to 16-code
// refer https://github.com/radareorg/radare2/blob/master/libr/cons/rgb.c#L249-L271
func RgbToAnsi(r, g, b uint8, isBg bool) uint8 {
var bright, c, k uint8
base := compareVal(isBg, BgBase, FgBase)
// eco bright-specific
if r == 0x80 && g == 0x80 && b == 0x80 { // 0x80=128
bright = 53
} else if r == 0xff || g == 0xff || b == 0xff { // 0xff=255
bright = 60
} // else bright = 0
if r == g && g == b {
// 0x7f=127
// r = (r > 0x7f) ? 1 : 0;
r = compareVal(r > 0x7f, 1, 0)
g = compareVal(g > 0x7f, 1, 0)
b = compareVal(b > 0x7f, 1, 0)
} else {
k = (r + g + b) / 3
// r = (r >= k) ? 1 : 0;
r = compareVal(r >= k, 1, 0)
g = compareVal(g >= k, 1, 0)
b = compareVal(b >= k, 1, 0)
}
// c = (r ? 1 : 0) + (g ? (b ? 6 : 2) : (b ? 4 : 0))
c = compareVal(r > 0, 1, 0)
if g > 0 {
c += compareVal(b > 0, 6, 2)
} else {
c += compareVal(b > 0, 4, 0)
}
return base + bright + c
}
/*************************************************************
* 8bit(256) color <=> RGB/True color
*************************************************************/
// Rgb2short convert RGB-code to 256-code
func Rgb2short(r, g, b uint8) uint8 {
return RgbTo256(r, g, b)
}
// RgbTo256 convert RGB-code to 256-code
func RgbTo256(r, g, b uint8) uint8 {
res := make([]uint8, 3)
for partI, part := range [3]uint8{r, g, b} {
i := 0
for i < len(incs)-1 {
s, b := incs[i], incs[i+1] // smaller, bigger
if s <= part && part <= b {
s1 := math.Abs(float64(s) - float64(part))
b1 := math.Abs(float64(b) - float64(part))
var closest uint8
if s1 < b1 {
closest = s
} else {
closest = b
}
res[partI] = closest
break
}
i++
}
}
hex := fmt.Sprintf("%02x%02x%02x", res[0], res[1], res[2])
equiv := hexTo256Table[hex]
return equiv
}
// C256ToRgb convert an 256 color code to RGB numbers
func C256ToRgb(val uint8) (rgb []uint8) {
hex := c256ToHexMap[val]
// convert to rgb code
rgbInts := Hex2rgb(hex)
return []uint8{
uint8(rgbInts[0]),
uint8(rgbInts[1]),
uint8(rgbInts[2]),
}
}
// C256ToRgbV1 convert an 256 color code to RGB numbers
// refer https://github.com/torvalds/linux/commit/cec5b2a97a11ade56a701e83044d0a2a984c67b4
func C256ToRgbV1(val uint8) (rgb []uint8) {
var r, g, b uint8
if val < 8 { // Standard colours.
// r = val&1 ? 0xaa : 0x00;
r = compareVal(val&1 == 1, 0xaa, 0x00)
g = compareVal(val&2 == 2, 0xaa, 0x00)
b = compareVal(val&4 == 4, 0xaa, 0x00)
} else if val < 16 {
// r = val & 1 ? 0xff : 0x55;
r = compareVal(val&1 == 1, 0xff, 0x55)
g = compareVal(val&2 == 2, 0xff, 0x55)
b = compareVal(val&4 == 4, 0xff, 0x55)
} else if val < 232 { /* 6x6x6 colour cube. */
r = (val - 16) / 36 * 85 / 2
g = (val - 16) / 6 % 6 * 85 / 2
b = (val - 16) % 6 * 85 / 2
} else { /* Grayscale ramp. */
nv := uint8(int(val)*10 - 2312)
// set value
r, g, b = nv, nv, nv
}
return []uint8{r, g, b}
}

View File

@@ -1,281 +0,0 @@
package color
import (
"io"
"io/ioutil"
"os"
"runtime"
"strconv"
"strings"
"syscall"
"github.com/xo/terminfo"
)
/*************************************************************
* helper methods for detect color supports
*************************************************************/
// DetectColorLevel for current env
//
// NOTICE: The method will detect terminal info each times,
// if only want get current color level, please direct call SupportColor() or TermColorLevel()
func DetectColorLevel() terminfo.ColorLevel {
level, _ := detectTermColorLevel()
return level
}
// detect terminal color support level
//
// refer https://github.com/Delta456/box-cli-maker
func detectTermColorLevel() (level terminfo.ColorLevel, needVTP bool) {
// on windows WSL:
// - runtime.GOOS == "Linux"
// - support true-color
// env:
// WSL_DISTRO_NAME=Debian
if val := os.Getenv("WSL_DISTRO_NAME"); val != "" {
// detect WSL as it has True Color support
if detectWSL() {
debugf("True Color support on WSL environment")
return terminfo.ColorLevelMillions, false
}
}
isWin := runtime.GOOS == "windows"
termVal := os.Getenv("TERM")
// on TERM=screen: not support true-color
if termVal != "screen" {
// On JetBrains Terminal
// - support true-color
// env:
// TERMINAL_EMULATOR=JetBrains-JediTerm
val := os.Getenv("TERMINAL_EMULATOR")
if val == "JetBrains-JediTerm" {
debugf("True Color support on JetBrains-JediTerm, is win: %v", isWin)
return terminfo.ColorLevelMillions, isWin
}
}
// level, err = terminfo.ColorLevelFromEnv()
level = detectColorLevelFromEnv(termVal, isWin)
debugf("color level by detectColorLevelFromEnv: %s", level.String())
// fallback: simple detect by TERM value string.
if level == terminfo.ColorLevelNone {
debugf("level none - fallback check special term color support")
// on Windows: enable VTP as it has True Color support
level, needVTP = detectSpecialTermColor(termVal)
}
return
}
// detectColorFromEnv returns the color level COLORTERM, FORCE_COLOR,
// TERM_PROGRAM, or determined from the TERM environment variable.
//
// refer the terminfo.ColorLevelFromEnv()
// https://en.wikipedia.org/wiki/Terminfo
func detectColorLevelFromEnv(termVal string, isWin bool) terminfo.ColorLevel {
// check for overriding environment variables
colorTerm, termProg, forceColor := os.Getenv("COLORTERM"), os.Getenv("TERM_PROGRAM"), os.Getenv("FORCE_COLOR")
switch {
case strings.Contains(colorTerm, "truecolor") || strings.Contains(colorTerm, "24bit"):
if termVal == "screen" { // on TERM=screen: not support true-color
return terminfo.ColorLevelHundreds
}
return terminfo.ColorLevelMillions
case colorTerm != "" || forceColor != "":
return terminfo.ColorLevelBasic
case termProg == "Apple_Terminal":
return terminfo.ColorLevelHundreds
case termProg == "Terminus" || termProg == "Hyper":
if termVal == "screen" { // on TERM=screen: not support true-color
return terminfo.ColorLevelHundreds
}
return terminfo.ColorLevelMillions
case termProg == "iTerm.app":
if termVal == "screen" { // on TERM=screen: not support true-color
return terminfo.ColorLevelHundreds
}
// check iTerm version
ver := os.Getenv("TERM_PROGRAM_VERSION")
if ver != "" {
i, err := strconv.Atoi(strings.Split(ver, ".")[0])
if err != nil {
saveInternalError(terminfo.ErrInvalidTermProgramVersion)
// return terminfo.ColorLevelNone
return terminfo.ColorLevelHundreds
}
if i == 3 {
return terminfo.ColorLevelMillions
}
}
return terminfo.ColorLevelHundreds
}
// otherwise determine from TERM's max_colors capability
if !isWin && termVal != "" {
debugf("TERM=%s - check color level by load terminfo file", termVal)
ti, err := terminfo.Load(termVal)
if err != nil {
saveInternalError(err)
return terminfo.ColorLevelNone
}
debugf("the loaded term info file is: %s", ti.File)
v, ok := ti.Nums[terminfo.MaxColors]
switch {
case !ok || v <= 16:
return terminfo.ColorLevelNone
case ok && v >= 256:
return terminfo.ColorLevelHundreds
}
return terminfo.ColorLevelBasic
}
// no TERM env value. default return none level
return terminfo.ColorLevelNone
// return terminfo.ColorLevelBasic
}
var detectedWSL bool
var wslContents string
// https://github.com/Microsoft/WSL/issues/423#issuecomment-221627364
func detectWSL() bool {
if !detectedWSL {
b := make([]byte, 1024)
// `cat /proc/version`
// on mac:
// !not the file!
// on linux(debian,ubuntu,alpine):
// Linux version 4.19.121-linuxkit (root@18b3f92ade35) (gcc version 9.2.0 (Alpine 9.2.0)) #1 SMP Thu Jan 21 15:36:34 UTC 2021
// on win git bash, conEmu:
// MINGW64_NT-10.0-19042 version 3.1.7-340.x86_64 (@WIN-N0G619FD3UK) (gcc version 9.3.0 (GCC) ) 2020-10-23 13:08 UTC
// on WSL:
// Linux version 4.4.0-19041-Microsoft (Microsoft@Microsoft.com) (gcc version 5.4.0 (GCC) ) #488-Microsoft Mon Sep 01 13:43:00 PST 2020
f, err := os.Open("/proc/version")
if err == nil {
_, _ = f.Read(b) // ignore error
if err = f.Close(); err != nil {
saveInternalError(err)
}
wslContents = string(b)
}
detectedWSL = true
}
return strings.Contains(wslContents, "Microsoft")
}
// refer
// https://github.com/Delta456/box-cli-maker/blob/7b5a1ad8a016ce181e7d8b05e24b54ff60b4b38a/detect_unix.go#L27-L45
// detect WSL as it has True Color support
func isWSL() bool {
// on windows WSL:
// - runtime.GOOS == "Linux"
// - support true-color
// WSL_DISTRO_NAME=Debian
if val := os.Getenv("WSL_DISTRO_NAME"); val == "" {
return false
}
// `cat /proc/sys/kernel/osrelease`
// on mac:
// !not the file!
// on linux:
// 4.19.121-linuxkit
// on WSL Output:
// 4.4.0-19041-Microsoft
wsl, err := ioutil.ReadFile("/proc/sys/kernel/osrelease")
if err != nil {
saveInternalError(err)
return false
}
// it gives "Microsoft" for WSL and "microsoft" for WSL 2
// it support True-color
content := strings.ToLower(string(wsl))
return strings.Contains(content, "microsoft")
}
/*************************************************************
* helper methods for check env
*************************************************************/
// IsWindows OS env
func IsWindows() bool {
return runtime.GOOS == "windows"
}
// IsConsole Determine whether w is one of stderr, stdout, stdin
func IsConsole(w io.Writer) bool {
o, ok := w.(*os.File)
if !ok {
return false
}
fd := o.Fd()
// fix: cannot use 'o == os.Stdout' to compare
return fd == uintptr(syscall.Stdout) || fd == uintptr(syscall.Stdin) || fd == uintptr(syscall.Stderr)
}
// IsMSys msys(MINGW64) environment, does not necessarily support color
func IsMSys() bool {
// like "MSYSTEM=MINGW64"
if len(os.Getenv("MSYSTEM")) > 0 {
return true
}
return false
}
// IsSupportColor check current console is support color.
//
// NOTICE: The method will detect terminal info each times,
// if only want get current color level, please direct call SupportColor() or TermColorLevel()
func IsSupportColor() bool {
return IsSupport16Color()
}
// IsSupportColor check current console is support color.
//
// NOTICE: The method will detect terminal info each times,
// if only want get current color level, please direct call SupportColor() or TermColorLevel()
func IsSupport16Color() bool {
level, _ := detectTermColorLevel()
return level > terminfo.ColorLevelNone
}
// IsSupport256Color render check
//
// NOTICE: The method will detect terminal info each times,
// if only want get current color level, please direct call SupportColor() or TermColorLevel()
func IsSupport256Color() bool {
level, _ := detectTermColorLevel()
return level > terminfo.ColorLevelBasic
}
// IsSupportRGBColor check. alias of the IsSupportTrueColor()
//
// NOTICE: The method will detect terminal info each times,
// if only want get current color level, please direct call SupportColor() or TermColorLevel()
func IsSupportRGBColor() bool {
return IsSupportTrueColor()
}
// IsSupportTrueColor render check.
//
// NOTICE: The method will detect terminal info each times,
// if only want get current color level, please direct call SupportColor() or TermColorLevel()
//
// ENV:
// "COLORTERM=truecolor"
// "COLORTERM=24bit"
func IsSupportTrueColor() bool {
level, _ := detectTermColorLevel()
return level > terminfo.ColorLevelHundreds
}

View File

@@ -1,48 +0,0 @@
// +build !windows
// The method in the file has no effect
// Only for compatibility with non-Windows systems
package color
import (
"strings"
"syscall"
"github.com/xo/terminfo"
)
// detect special term color support
func detectSpecialTermColor(termVal string) (terminfo.ColorLevel, bool) {
if termVal == "" {
return terminfo.ColorLevelNone, false
}
debugf("terminfo check fail - fallback detect color by check TERM value")
// on TERM=screen:
// - support 256, not support true-color. test on macOS
if termVal == "screen" {
return terminfo.ColorLevelHundreds, false
}
if strings.Contains(termVal, "256color") {
return terminfo.ColorLevelHundreds, false
}
if strings.Contains(termVal, "xterm") {
return terminfo.ColorLevelHundreds, false
// return terminfo.ColorLevelBasic, false
}
// return terminfo.ColorLevelNone, nil
return terminfo.ColorLevelBasic, false
}
// IsTerminal returns true if the given file descriptor is a terminal.
//
// Usage:
// IsTerminal(os.Stdout.Fd())
func IsTerminal(fd uintptr) bool {
return fd == uintptr(syscall.Stdout) || fd == uintptr(syscall.Stdin) || fd == uintptr(syscall.Stderr)
}

View File

@@ -1,243 +0,0 @@
// +build windows
// Display color on windows
// refer:
// golang.org/x/sys/windows
// golang.org/x/crypto/ssh/terminal
// https://docs.microsoft.com/en-us/windows/console
package color
import (
"os"
"syscall"
"unsafe"
"github.com/xo/terminfo"
"golang.org/x/sys/windows"
)
// related docs
// https://docs.microsoft.com/zh-cn/windows/console/console-virtual-terminal-sequences
// https://docs.microsoft.com/zh-cn/windows/console/console-virtual-terminal-sequences#samples
var (
// isMSys bool
kernel32 *syscall.LazyDLL
procGetConsoleMode *syscall.LazyProc
procSetConsoleMode *syscall.LazyProc
)
func init() {
if !SupportColor() {
isLikeInCmd = true
return
}
// if disabled.
if !Enable {
return
}
// if at windows's ConEmu, Cmder, putty ... terminals not need VTP
// -------- try force enable colors on windows terminal -------
tryEnableVTP(needVTP)
// fetch console screen buffer info
// err := getConsoleScreenBufferInfo(uintptr(syscall.Stdout), &defScreenInfo)
}
// try force enable colors on windows terminal
func tryEnableVTP(enable bool) bool {
if !enable {
return false
}
debugf("True-Color by enable VirtualTerminalProcessing on windows")
initKernel32Proc()
// enable colors on windows terminal
if tryEnableOnCONOUT() {
return true
}
return tryEnableOnStdout()
}
func initKernel32Proc() {
if kernel32 != nil {
return
}
// load related windows dll
// https://docs.microsoft.com/en-us/windows/console/setconsolemode
kernel32 = syscall.NewLazyDLL("kernel32.dll")
procGetConsoleMode = kernel32.NewProc("GetConsoleMode")
procSetConsoleMode = kernel32.NewProc("SetConsoleMode")
}
func tryEnableOnCONOUT() bool {
outHandle, err := syscall.Open("CONOUT$", syscall.O_RDWR, 0)
if err != nil {
saveInternalError(err)
return false
}
err = EnableVirtualTerminalProcessing(outHandle, true)
if err != nil {
saveInternalError(err)
return false
}
return true
}
func tryEnableOnStdout() bool {
// try direct open syscall.Stdout
err := EnableVirtualTerminalProcessing(syscall.Stdout, true)
if err != nil {
saveInternalError(err)
return false
}
return true
}
// Get the Windows Version and Build Number
var (
winVersion, _, buildNumber = windows.RtlGetNtVersionNumbers()
)
// refer
// https://github.com/Delta456/box-cli-maker/blob/7b5a1ad8a016ce181e7d8b05e24b54ff60b4b38a/detect_windows.go#L30-L57
// https://github.com/gookit/color/issues/25#issuecomment-738727917
// detects the Color Level Supported on windows: cmd, powerShell
func detectSpecialTermColor(termVal string) (tl terminfo.ColorLevel, needVTP bool) {
if os.Getenv("ConEmuANSI") == "ON" {
debugf("support True Color by ConEmuANSI=ON")
// ConEmuANSI is "ON" for generic ANSI support
// but True Color option is enabled by default
// I am just assuming that people wouldn't have disabled it
// Even if it is not enabled then ConEmu will auto round off
// accordingly
return terminfo.ColorLevelMillions, false
}
// Before Windows 10 Build Number 10586, console never supported ANSI Colors
if buildNumber < 10586 || winVersion < 10 {
// Detect if using ANSICON on older systems
if os.Getenv("ANSICON") != "" {
conVersion := os.Getenv("ANSICON_VER")
// 8 bit Colors were only supported after v1.81 release
if conVersion >= "181" {
return terminfo.ColorLevelHundreds, false
}
return terminfo.ColorLevelBasic, false
}
return terminfo.ColorLevelNone, false
}
// True Color is not available before build 14931 so fallback to 8 bit color.
if buildNumber < 14931 {
return terminfo.ColorLevelHundreds, true
}
// Windows 10 build 14931 is the first release that supports 16m/TrueColor
debugf("support True Color on windows version is >= build 14931")
return terminfo.ColorLevelMillions, true
}
/*************************************************************
* render full color code on windows(8,16,24bit color)
*************************************************************/
// docs https://docs.microsoft.com/zh-cn/windows/console/getconsolemode#parameters
const (
// equals to docs page's ENABLE_VIRTUAL_TERMINAL_PROCESSING 0x0004
EnableVirtualTerminalProcessingMode uint32 = 0x4
)
// EnableVirtualTerminalProcessing Enable virtual terminal processing
//
// ref from github.com/konsorten/go-windows-terminal-sequences
// doc https://docs.microsoft.com/zh-cn/windows/console/console-virtual-terminal-sequences#samples
//
// Usage:
// err := EnableVirtualTerminalProcessing(syscall.Stdout, true)
// // support print color text
// err = EnableVirtualTerminalProcessing(syscall.Stdout, false)
func EnableVirtualTerminalProcessing(stream syscall.Handle, enable bool) error {
var mode uint32
// Check if it is currently in the terminal
// err := syscall.GetConsoleMode(syscall.Stdout, &mode)
err := syscall.GetConsoleMode(stream, &mode)
if err != nil {
// fmt.Println("EnableVirtualTerminalProcessing", err)
return err
}
if enable {
mode |= EnableVirtualTerminalProcessingMode
} else {
mode &^= EnableVirtualTerminalProcessingMode
}
ret, _, err := procSetConsoleMode.Call(uintptr(stream), uintptr(mode))
if ret == 0 {
return err
}
return nil
}
// renderColorCodeOnCmd enable cmd color render.
// func renderColorCodeOnCmd(fn func()) {
// err := EnableVirtualTerminalProcessing(syscall.Stdout, true)
// // if is not in terminal, will clear color tag.
// if err != nil {
// // panic(err)
// fn()
// return
// }
//
// // force open color render
// old := ForceOpenColor()
// fn()
// // revert color setting
// supportColor = old
//
// err = EnableVirtualTerminalProcessing(syscall.Stdout, false)
// if err != nil {
// panic(err)
// }
// }
/*************************************************************
* render simple color code on windows
*************************************************************/
// IsTty returns true if the given file descriptor is a terminal.
func IsTty(fd uintptr) bool {
initKernel32Proc()
var st uint32
r, _, e := syscall.Syscall(procGetConsoleMode.Addr(), 2, fd, uintptr(unsafe.Pointer(&st)), 0)
return r != 0 && e == 0
}
// IsTerminal returns true if the given file descriptor is a terminal.
//
// Usage:
// fd := os.Stdout.Fd()
// fd := uintptr(syscall.Stdout) // for windows
// IsTerminal(fd)
func IsTerminal(fd uintptr) bool {
initKernel32Proc()
var st uint32
r, _, e := syscall.Syscall(procGetConsoleMode.Addr(), 2, fd, uintptr(unsafe.Pointer(&st)), 0)
return r != 0 && e == 0
}

View File

@@ -1,9 +0,0 @@
module github.com/gookit/color
go 1.12
require (
github.com/stretchr/testify v1.6.1
github.com/xo/terminfo v0.0.0-20210125001918-ca9a967f8778
golang.org/x/sys v0.0.0-20210330210617-4fbd30eecc44
)

View File

@@ -1,15 +0,0 @@
github.com/davecgh/go-spew v1.1.0 h1:ZDRjVQ15GmhC3fiQ8ni8+OwkZQO4DARzQgrnXU1Liz8=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v1.6.1 h1:hDPOHmpOpP40lSULcqw7IrRb/u7w6RpDC9399XyoNd0=
github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/xo/terminfo v0.0.0-20210125001918-ca9a967f8778 h1:QldyIu/L63oPpyvQmHgvgickp1Yw510KJOqX7H24mg8=
github.com/xo/terminfo v0.0.0-20210125001918-ca9a967f8778/go.mod h1:2MuV+tbUrU1zIOPMxZ5EncGwgmMJsa+9ucAQZXxsObs=
golang.org/x/sys v0.0.0-20210330210617-4fbd30eecc44 h1:Bli41pIlzTzf3KEY06n+xnzK/BESIg2ze4Pgfh/aI8c=
golang.org/x/sys v0.0.0-20210330210617-4fbd30eecc44/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c h1:dUUwHk2QECo/6vqA44rthZ8ie2QXMNeKRTHCNY2nXvo=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=

View File

@@ -1,122 +0,0 @@
package color
import "fmt"
/*************************************************************
* colored message Printer
*************************************************************/
// PrinterFace interface
type PrinterFace interface {
fmt.Stringer
Sprint(a ...interface{}) string
Sprintf(format string, a ...interface{}) string
Print(a ...interface{})
Printf(format string, a ...interface{})
Println(a ...interface{})
}
// Printer a generic color message printer.
//
// Usage:
// p := &Printer{Code: "32;45;3"}
// p.Print("message")
type Printer struct {
// NoColor disable color.
NoColor bool
// Code color code string. eg "32;45;3"
Code string
}
// NewPrinter instance
func NewPrinter(colorCode string) *Printer {
return &Printer{Code: colorCode}
}
// String returns color code string. eg: "32;45;3"
func (p *Printer) String() string {
// panic("implement me")
return p.Code
}
// Sprint returns rendering colored messages
func (p *Printer) Sprint(a ...interface{}) string {
return RenderCode(p.String(), a...)
}
// Sprintf returns format and rendering colored messages
func (p *Printer) Sprintf(format string, a ...interface{}) string {
return RenderString(p.String(), fmt.Sprintf(format, a...))
}
// Print rendering colored messages
func (p *Printer) Print(a ...interface{}) {
doPrintV2(p.String(), fmt.Sprint(a...))
}
// Printf format and rendering colored messages
func (p *Printer) Printf(format string, a ...interface{}) {
doPrintV2(p.String(), fmt.Sprintf(format, a...))
}
// Println rendering colored messages with newline
func (p *Printer) Println(a ...interface{}) {
doPrintlnV2(p.Code, a)
}
// IsEmpty color code
func (p *Printer) IsEmpty() bool {
return p.Code == ""
}
/*************************************************************
* SimplePrinter struct
*************************************************************/
// SimplePrinter use for quick use color print on inject to struct
type SimplePrinter struct{}
// Print message
func (s *SimplePrinter) Print(v ...interface{}) {
Print(v...)
}
// Printf message
func (s *SimplePrinter) Printf(format string, v ...interface{}) {
Printf(format, v...)
}
// Println message
func (s *SimplePrinter) Println(v ...interface{}) {
Println(v...)
}
// Infof message
func (s *SimplePrinter) Infof(format string, a ...interface{}) {
Info.Printf(format, a...)
}
// Infoln message
func (s *SimplePrinter) Infoln(a ...interface{}) {
Info.Println(a...)
}
// Warnf message
func (s *SimplePrinter) Warnf(format string, a ...interface{}) {
Warn.Printf(format, a...)
}
// Warnln message
func (s *SimplePrinter) Warnln(a ...interface{}) {
Warn.Println(a...)
}
// Errorf message
func (s *SimplePrinter) Errorf(format string, a ...interface{}) {
Error.Printf(format, a...)
}
// Errorln message
func (s *SimplePrinter) Errorln(a ...interface{}) {
Error.Println(a...)
}

View File

@@ -1,109 +0,0 @@
package color
/*************************************************************
* quick use color print message
*************************************************************/
// Redp print message with Red color
func Redp(a ...interface{}) {
Red.Print(a...)
}
// Redln print message line with Red color
func Redln(a ...interface{}) {
Red.Println(a...)
}
// Bluep print message with Blue color
func Bluep(a ...interface{}) {
Blue.Print(a...)
}
// Blueln print message line with Blue color
func Blueln(a ...interface{}) {
Blue.Println(a...)
}
// Cyanp print message with Cyan color
func Cyanp(a ...interface{}) {
Cyan.Print(a...)
}
// Cyanln print message line with Cyan color
func Cyanln(a ...interface{}) {
Cyan.Println(a...)
}
// Grayp print message with Gray color
func Grayp(a ...interface{}) {
Gray.Print(a...)
}
// Grayln print message line with Gray color
func Grayln(a ...interface{}) {
Gray.Println(a...)
}
// Greenp print message with Green color
func Greenp(a ...interface{}) {
Green.Print(a...)
}
// Greenln print message line with Green color
func Greenln(a ...interface{}) {
Green.Println(a...)
}
// Yellowp print message with Yellow color
func Yellowp(a ...interface{}) {
Yellow.Print(a...)
}
// Yellowln print message line with Yellow color
func Yellowln(a ...interface{}) {
Yellow.Println(a...)
}
// Magentap print message with Magenta color
func Magentap(a ...interface{}) {
Magenta.Print(a...)
}
// Magentaln print message line with Magenta color
func Magentaln(a ...interface{}) {
Magenta.Println(a...)
}
/*************************************************************
* quick use style print message
*************************************************************/
// Infof print message with Info style
func Infof(format string, a ...interface{}) {
Info.Printf(format, a...)
}
// Infoln print message with Info style
func Infoln(a ...interface{}) {
Info.Println(a...)
}
// Errorf print message with Error style
func Errorf(format string, a ...interface{}) {
Error.Printf(format, a...)
}
// Errorln print message with Error style
func Errorln(a ...interface{}) {
Error.Println(a...)
}
// Warnf print message with Warn style
func Warnf(format string, a ...interface{}) {
Warn.Printf(format, a...)
}
// Warnln print message with Warn style
func Warnln(a ...interface{}) {
Warn.Println(a...)
}

View File

@@ -1,315 +0,0 @@
package color
import (
"fmt"
"strings"
)
/*************************************************************
* 16 color Style
*************************************************************/
// Style a 16 color style. can add: fg color, bg color, color options
//
// Example:
// color.Style{color.FgGreen}.Print("message")
type Style []Color
// New create a custom style
//
// Usage:
// color.New(color.FgGreen).Print("message")
// equals to:
// color.Style{color.FgGreen}.Print("message")
func New(colors ...Color) Style {
return colors
}
// Save to global styles map
func (s Style) Save(name string) {
AddStyle(name, s)
}
// Add to global styles map
func (s *Style) Add(cs ...Color) {
*s = append(*s, cs...)
}
// Render render text
// Usage:
// color.New(color.FgGreen).Render("text")
// color.New(color.FgGreen, color.BgBlack, color.OpBold).Render("text")
func (s Style) Render(a ...interface{}) string {
return RenderCode(s.String(), a...)
}
// Renderln render text line.
// like Println, will add spaces for each argument
// Usage:
// color.New(color.FgGreen).Renderln("text", "more")
// color.New(color.FgGreen, color.BgBlack, color.OpBold).Render("text", "more")
func (s Style) Renderln(a ...interface{}) string {
return RenderWithSpaces(s.String(), a...)
}
// Sprint is alias of the 'Render'
func (s Style) Sprint(a ...interface{}) string {
return RenderCode(s.String(), a...)
}
// Sprintf format and render message.
func (s Style) Sprintf(format string, a ...interface{}) string {
return RenderString(s.String(), fmt.Sprintf(format, a...))
}
// Print render and Print text
func (s Style) Print(a ...interface{}) {
doPrintV2(s.String(), fmt.Sprint(a...))
}
// Printf render and print text
func (s Style) Printf(format string, a ...interface{}) {
doPrintV2(s.Code(), fmt.Sprintf(format, a...))
}
// Println render and print text line
func (s Style) Println(a ...interface{}) {
doPrintlnV2(s.String(), a)
}
// Code convert to code string. returns like "32;45;3"
func (s Style) Code() string {
return s.String()
}
// String convert to code string. returns like "32;45;3"
func (s Style) String() string {
return Colors2code(s...)
}
// IsEmpty style
func (s Style) IsEmpty() bool {
return len(s) == 0
}
/*************************************************************
* Theme(extended Style)
*************************************************************/
// Theme definition. extends from Style
type Theme struct {
// Name theme name
Name string
// Style for the theme
Style
}
// NewTheme instance
func NewTheme(name string, style Style) *Theme {
return &Theme{name, style}
}
// Save to themes map
func (t *Theme) Save() {
AddTheme(t.Name, t.Style)
}
// Tips use name as title, only apply style for name
func (t *Theme) Tips(format string, a ...interface{}) {
// only apply style for name
t.Print(strings.ToUpper(t.Name) + ": ")
Printf(format+"\n", a...)
}
// Prompt use name as title, and apply style for message
func (t *Theme) Prompt(format string, a ...interface{}) {
title := strings.ToUpper(t.Name) + ":"
t.Println(title, fmt.Sprintf(format, a...))
}
// Block like Prompt, but will wrap a empty line
func (t *Theme) Block(format string, a ...interface{}) {
title := strings.ToUpper(t.Name) + ":\n"
t.Println(title, fmt.Sprintf(format, a...))
}
/*************************************************************
* Theme: internal themes
*************************************************************/
// internal themes(like bootstrap style)
// Usage:
// color.Info.Print("message")
// color.Info.Printf("a %s message", "test")
// color.Warn.Println("message")
// color.Error.Println("message")
var (
// Info color style
Info = &Theme{"info", Style{OpReset, FgGreen}}
// Note color style
Note = &Theme{"note", Style{OpBold, FgLightCyan}}
// Warn color style
Warn = &Theme{"warning", Style{OpBold, FgYellow}}
// Light color style
Light = &Theme{"light", Style{FgLightWhite, BgBlack}}
// Error color style
Error = &Theme{"error", Style{FgLightWhite, BgRed}}
// Danger color style
Danger = &Theme{"danger", Style{OpBold, FgRed}}
// Debug color style
Debug = &Theme{"debug", Style{OpReset, FgCyan}}
// Notice color style
Notice = &Theme{"notice", Style{OpBold, FgCyan}}
// Comment color style
Comment = &Theme{"comment", Style{OpReset, FgYellow}}
// Success color style
Success = &Theme{"success", Style{OpBold, FgGreen}}
// Primary color style
Primary = &Theme{"primary", Style{OpReset, FgBlue}}
// Question color style
Question = &Theme{"question", Style{OpReset, FgMagenta}}
// Secondary color style
Secondary = &Theme{"secondary", Style{FgDarkGray}}
)
// Themes internal defined themes.
// Usage:
// color.Themes["info"].Println("message")
var Themes = map[string]*Theme{
"info": Info,
"note": Note,
"light": Light,
"error": Error,
"debug": Debug,
"danger": Danger,
"notice": Notice,
"success": Success,
"comment": Comment,
"primary": Primary,
"warning": Warn,
"question": Question,
"secondary": Secondary,
}
// AddTheme add a theme and style
func AddTheme(name string, style Style) {
Themes[name] = NewTheme(name, style)
Styles[name] = style
}
// GetTheme get defined theme by name
func GetTheme(name string) *Theme {
return Themes[name]
}
/*************************************************************
* internal styles
*************************************************************/
// Styles internal defined styles, like bootstrap styles.
// Usage:
// color.Styles["info"].Println("message")
var Styles = map[string]Style{
"info": {OpReset, FgGreen},
"note": {OpBold, FgLightCyan},
"light": {FgLightWhite, BgRed},
"error": {FgLightWhite, BgRed},
"danger": {OpBold, FgRed},
"notice": {OpBold, FgCyan},
"success": {OpBold, FgGreen},
"comment": {OpReset, FgMagenta},
"primary": {OpReset, FgBlue},
"warning": {OpBold, FgYellow},
"question": {OpReset, FgMagenta},
"secondary": {FgDarkGray},
}
// some style name alias
var styleAliases = map[string]string{
"err": "error",
"suc": "success",
"warn": "warning",
}
// AddStyle add a style
func AddStyle(name string, s Style) {
Styles[name] = s
}
// GetStyle get defined style by name
func GetStyle(name string) Style {
if s, ok := Styles[name]; ok {
return s
}
if realName, ok := styleAliases[name]; ok {
return Styles[realName]
}
// empty style
return New()
}
/*************************************************************
* color scheme
*************************************************************/
// Scheme struct
type Scheme struct {
Name string
Styles map[string]Style
}
// NewScheme create new Scheme
func NewScheme(name string, styles map[string]Style) *Scheme {
return &Scheme{Name: name, Styles: styles}
}
// NewDefaultScheme create an defuault color Scheme
func NewDefaultScheme(name string) *Scheme {
return NewScheme(name, map[string]Style{
"info": {OpReset, FgGreen},
"warn": {OpBold, FgYellow},
"error": {FgLightWhite, BgRed},
})
}
// Style get by name
func (s *Scheme) Style(name string) Style {
return s.Styles[name]
}
// Infof message print
func (s *Scheme) Infof(format string, a ...interface{}) {
s.Styles["info"].Printf(format, a...)
}
// Infoln message print
func (s *Scheme) Infoln(v ...interface{}) {
s.Styles["info"].Println(v...)
}
// Warnf message print
func (s *Scheme) Warnf(format string, a ...interface{}) {
s.Styles["warn"].Printf(format, a...)
}
// Warnln message print
func (s *Scheme) Warnln(v ...interface{}) {
s.Styles["warn"].Println(v...)
}
// Errorf message print
func (s *Scheme) Errorf(format string, a ...interface{}) {
s.Styles["error"].Printf(format, a...)
}
// Errorln message print
func (s *Scheme) Errorln(v ...interface{}) {
s.Styles["error"].Println(v...)
}

View File

@@ -1,206 +0,0 @@
package color
import (
"fmt"
"io"
"log"
"strings"
)
// SetTerminal by given code.
func SetTerminal(code string) error {
if !Enable || !SupportColor() {
return nil
}
_, err := fmt.Fprintf(output, SettingTpl, code)
return err
}
// ResetTerminal terminal setting.
func ResetTerminal() error {
if !Enable || !SupportColor() {
return nil
}
_, err := fmt.Fprint(output, ResetSet)
return err
}
/*************************************************************
* print methods(will auto parse color tags)
*************************************************************/
// Print render color tag and print messages
func Print(a ...interface{}) {
Fprint(output, a...)
}
// Printf format and print messages
func Printf(format string, a ...interface{}) {
Fprintf(output, format, a...)
}
// Println messages with new line
func Println(a ...interface{}) {
Fprintln(output, a...)
}
// Fprint print rendered messages to writer
// Notice: will ignore print error
func Fprint(w io.Writer, a ...interface{}) {
_, err := fmt.Fprint(w, Render(a...))
saveInternalError(err)
// if isLikeInCmd {
// renderColorCodeOnCmd(func() {
// _, _ = fmt.Fprint(w, Render(a...))
// })
// } else {
// _, _ = fmt.Fprint(w, Render(a...))
// }
}
// Fprintf print format and rendered messages to writer.
// Notice: will ignore print error
func Fprintf(w io.Writer, format string, a ...interface{}) {
str := fmt.Sprintf(format, a...)
_, err := fmt.Fprint(w, ReplaceTag(str))
saveInternalError(err)
}
// Fprintln print rendered messages line to writer
// Notice: will ignore print error
func Fprintln(w io.Writer, a ...interface{}) {
str := formatArgsForPrintln(a)
_, err := fmt.Fprintln(w, ReplaceTag(str))
saveInternalError(err)
}
// Lprint passes colored messages to a log.Logger for printing.
// Notice: should be goroutine safe
func Lprint(l *log.Logger, a ...interface{}) {
l.Print(Render(a...))
}
// Render parse color tags, return rendered string.
// Usage:
// text := Render("<info>hello</> <cyan>world</>!")
// fmt.Println(text)
func Render(a ...interface{}) string {
if len(a) == 0 {
return ""
}
return ReplaceTag(fmt.Sprint(a...))
}
// Sprint parse color tags, return rendered string
func Sprint(a ...interface{}) string {
if len(a) == 0 {
return ""
}
return ReplaceTag(fmt.Sprint(a...))
}
// Sprintf format and return rendered string
func Sprintf(format string, a ...interface{}) string {
return ReplaceTag(fmt.Sprintf(format, a...))
}
// String alias of the ReplaceTag
func String(s string) string {
return ReplaceTag(s)
}
// Text alias of the ReplaceTag
func Text(s string) string {
return ReplaceTag(s)
}
/*************************************************************
* helper methods for print
*************************************************************/
// new implementation, support render full color code on pwsh.exe, cmd.exe
func doPrintV2(code, str string) {
_, err := fmt.Fprint(output, RenderString(code, str))
saveInternalError(err)
// if isLikeInCmd {
// renderColorCodeOnCmd(func() {
// _, _ = fmt.Fprint(output, RenderString(code, str))
// })
// } else {
// _, _ = fmt.Fprint(output, RenderString(code, str))
// }
}
// new implementation, support render full color code on pwsh.exe, cmd.exe
func doPrintlnV2(code string, args []interface{}) {
str := formatArgsForPrintln(args)
_, err := fmt.Fprintln(output, RenderString(code, str))
saveInternalError(err)
}
// if use Println, will add spaces for each arg
func formatArgsForPrintln(args []interface{}) (message string) {
if ln := len(args); ln == 0 {
message = ""
} else if ln == 1 {
message = fmt.Sprint(args[0])
} else {
message = fmt.Sprintln(args...)
// clear last "\n"
message = message[:len(message)-1]
}
return
}
/*************************************************************
* helper methods
*************************************************************/
// is on debug mode
// func isDebugMode() bool {
// return debugMode == "on"
// }
func debugf(f string, v ...interface{}) {
if debugMode {
fmt.Print("COLOR_DEBUG: ")
fmt.Printf(f, v...)
fmt.Println()
}
}
// equals: return ok ? val1 : val2
func compareVal(ok bool, val1, val2 uint8) uint8 {
if ok {
return val1
}
return val2
}
func saveInternalError(err error) {
if err != nil {
debugf("inner error: %s", err.Error())
innerErrs = append(innerErrs, err)
}
}
func stringToArr(str, sep string) (arr []string) {
str = strings.TrimSpace(str)
if str == "" {
return
}
ss := strings.Split(str, sep)
for _, val := range ss {
if val = strings.TrimSpace(val); val != "" {
arr = append(arr, val)
}
}
return
}

View File

@@ -1,16 +0,0 @@
language: go
sudo: false
go:
- 1.13.x
- tip
before_install:
- go get -t -v ./...
script:
- go generate
- git diff --cached --exit-code
- ./go.test.sh
after_success:
- bash <(curl -s https://codecov.io/bash)

View File

@@ -1,21 +0,0 @@
The MIT License (MIT)
Copyright (c) 2016 Yasuhiro Matsumoto
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

View File

@@ -1,27 +0,0 @@
go-runewidth
============
[![Build Status](https://travis-ci.org/mattn/go-runewidth.png?branch=master)](https://travis-ci.org/mattn/go-runewidth)
[![Codecov](https://codecov.io/gh/mattn/go-runewidth/branch/master/graph/badge.svg)](https://codecov.io/gh/mattn/go-runewidth)
[![GoDoc](https://godoc.org/github.com/mattn/go-runewidth?status.svg)](http://godoc.org/github.com/mattn/go-runewidth)
[![Go Report Card](https://goreportcard.com/badge/github.com/mattn/go-runewidth)](https://goreportcard.com/report/github.com/mattn/go-runewidth)
Provides functions to get fixed width of the character or string.
Usage
-----
```go
runewidth.StringWidth("つのだ☆HIRO") == 12
```
Author
------
Yasuhiro Matsumoto
License
-------
under the MIT License: http://mattn.mit-license.org/2013

View File

@@ -1,5 +0,0 @@
module github.com/mattn/go-runewidth
go 1.9
require github.com/rivo/uniseg v0.2.0

View File

@@ -1,2 +0,0 @@
github.com/rivo/uniseg v0.2.0 h1:S1pD9weZBuJdFmowNwbpi7BJ8TNftyUImj/0WQi72jY=
github.com/rivo/uniseg v0.2.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJtxc=

View File

@@ -1,12 +0,0 @@
#!/usr/bin/env bash
set -e
echo "" > coverage.txt
for d in $(go list ./... | grep -v vendor); do
go test -race -coverprofile=profile.out -covermode=atomic "$d"
if [ -f profile.out ]; then
cat profile.out >> coverage.txt
rm profile.out
fi
done

View File

@@ -1,273 +0,0 @@
package runewidth
import (
"os"
"github.com/rivo/uniseg"
)
//go:generate go run script/generate.go
var (
// EastAsianWidth will be set true if the current locale is CJK
EastAsianWidth bool
// StrictEmojiNeutral should be set false if handle broken fonts
StrictEmojiNeutral bool = true
// DefaultCondition is a condition in current locale
DefaultCondition = &Condition{
EastAsianWidth: false,
StrictEmojiNeutral: true,
}
)
func init() {
handleEnv()
}
func handleEnv() {
env := os.Getenv("RUNEWIDTH_EASTASIAN")
if env == "" {
EastAsianWidth = IsEastAsian()
} else {
EastAsianWidth = env == "1"
}
// update DefaultCondition
DefaultCondition.EastAsianWidth = EastAsianWidth
}
type interval struct {
first rune
last rune
}
type table []interval
func inTables(r rune, ts ...table) bool {
for _, t := range ts {
if inTable(r, t) {
return true
}
}
return false
}
func inTable(r rune, t table) bool {
if r < t[0].first {
return false
}
bot := 0
top := len(t) - 1
for top >= bot {
mid := (bot + top) >> 1
switch {
case t[mid].last < r:
bot = mid + 1
case t[mid].first > r:
top = mid - 1
default:
return true
}
}
return false
}
var private = table{
{0x00E000, 0x00F8FF}, {0x0F0000, 0x0FFFFD}, {0x100000, 0x10FFFD},
}
var nonprint = table{
{0x0000, 0x001F}, {0x007F, 0x009F}, {0x00AD, 0x00AD},
{0x070F, 0x070F}, {0x180B, 0x180E}, {0x200B, 0x200F},
{0x2028, 0x202E}, {0x206A, 0x206F}, {0xD800, 0xDFFF},
{0xFEFF, 0xFEFF}, {0xFFF9, 0xFFFB}, {0xFFFE, 0xFFFF},
}
// Condition have flag EastAsianWidth whether the current locale is CJK or not.
type Condition struct {
EastAsianWidth bool
StrictEmojiNeutral bool
}
// NewCondition return new instance of Condition which is current locale.
func NewCondition() *Condition {
return &Condition{
EastAsianWidth: EastAsianWidth,
StrictEmojiNeutral: StrictEmojiNeutral,
}
}
// RuneWidth returns the number of cells in r.
// See http://www.unicode.org/reports/tr11/
func (c *Condition) RuneWidth(r rune) int {
// optimized version, verified by TestRuneWidthChecksums()
if !c.EastAsianWidth {
switch {
case r < 0x20 || r > 0x10FFFF:
return 0
case (r >= 0x7F && r <= 0x9F) || r == 0xAD: // nonprint
return 0
case r < 0x300:
return 1
case inTable(r, narrow):
return 1
case inTables(r, nonprint, combining):
return 0
case inTable(r, doublewidth):
return 2
default:
return 1
}
} else {
switch {
case r < 0 || r > 0x10FFFF || inTables(r, nonprint, combining):
return 0
case inTable(r, narrow):
return 1
case inTables(r, ambiguous, doublewidth):
return 2
case !c.StrictEmojiNeutral && inTables(r, ambiguous, emoji, narrow):
return 2
default:
return 1
}
}
}
// StringWidth return width as you can see
func (c *Condition) StringWidth(s string) (width int) {
g := uniseg.NewGraphemes(s)
for g.Next() {
var chWidth int
for _, r := range g.Runes() {
chWidth = c.RuneWidth(r)
if chWidth > 0 {
break // Our best guess at this point is to use the width of the first non-zero-width rune.
}
}
width += chWidth
}
return
}
// Truncate return string truncated with w cells
func (c *Condition) Truncate(s string, w int, tail string) string {
if c.StringWidth(s) <= w {
return s
}
w -= c.StringWidth(tail)
var width int
pos := len(s)
g := uniseg.NewGraphemes(s)
for g.Next() {
var chWidth int
for _, r := range g.Runes() {
chWidth = c.RuneWidth(r)
if chWidth > 0 {
break // See StringWidth() for details.
}
}
if width+chWidth > w {
pos, _ = g.Positions()
break
}
width += chWidth
}
return s[:pos] + tail
}
// Wrap return string wrapped with w cells
func (c *Condition) Wrap(s string, w int) string {
width := 0
out := ""
for _, r := range []rune(s) {
cw := c.RuneWidth(r)
if r == '\n' {
out += string(r)
width = 0
continue
} else if width+cw > w {
out += "\n"
width = 0
out += string(r)
width += cw
continue
}
out += string(r)
width += cw
}
return out
}
// FillLeft return string filled in left by spaces in w cells
func (c *Condition) FillLeft(s string, w int) string {
width := c.StringWidth(s)
count := w - width
if count > 0 {
b := make([]byte, count)
for i := range b {
b[i] = ' '
}
return string(b) + s
}
return s
}
// FillRight return string filled in left by spaces in w cells
func (c *Condition) FillRight(s string, w int) string {
width := c.StringWidth(s)
count := w - width
if count > 0 {
b := make([]byte, count)
for i := range b {
b[i] = ' '
}
return s + string(b)
}
return s
}
// RuneWidth returns the number of cells in r.
// See http://www.unicode.org/reports/tr11/
func RuneWidth(r rune) int {
return DefaultCondition.RuneWidth(r)
}
// IsAmbiguousWidth returns whether is ambiguous width or not.
func IsAmbiguousWidth(r rune) bool {
return inTables(r, private, ambiguous)
}
// IsNeutralWidth returns whether is neutral width or not.
func IsNeutralWidth(r rune) bool {
return inTable(r, neutral)
}
// StringWidth return width as you can see
func StringWidth(s string) (width int) {
return DefaultCondition.StringWidth(s)
}
// Truncate return string truncated with w cells
func Truncate(s string, w int, tail string) string {
return DefaultCondition.Truncate(s, w, tail)
}
// Wrap return string wrapped with w cells
func Wrap(s string, w int) string {
return DefaultCondition.Wrap(s, w)
}
// FillLeft return string filled in left by spaces in w cells
func FillLeft(s string, w int) string {
return DefaultCondition.FillLeft(s, w)
}
// FillRight return string filled in left by spaces in w cells
func FillRight(s string, w int) string {
return DefaultCondition.FillRight(s, w)
}

View File

@@ -1,8 +0,0 @@
// +build appengine
package runewidth
// IsEastAsian return true if the current locale is CJK
func IsEastAsian() bool {
return false
}

View File

@@ -1,9 +0,0 @@
// +build js
// +build !appengine
package runewidth
func IsEastAsian() bool {
// TODO: Implement this for the web. Detect east asian in a compatible way, and return true.
return false
}

View File

@@ -1,82 +0,0 @@
// +build !windows
// +build !js
// +build !appengine
package runewidth
import (
"os"
"regexp"
"strings"
)
var reLoc = regexp.MustCompile(`^[a-z][a-z][a-z]?(?:_[A-Z][A-Z])?\.(.+)`)
var mblenTable = map[string]int{
"utf-8": 6,
"utf8": 6,
"jis": 8,
"eucjp": 3,
"euckr": 2,
"euccn": 2,
"sjis": 2,
"cp932": 2,
"cp51932": 2,
"cp936": 2,
"cp949": 2,
"cp950": 2,
"big5": 2,
"gbk": 2,
"gb2312": 2,
}
func isEastAsian(locale string) bool {
charset := strings.ToLower(locale)
r := reLoc.FindStringSubmatch(locale)
if len(r) == 2 {
charset = strings.ToLower(r[1])
}
if strings.HasSuffix(charset, "@cjk_narrow") {
return false
}
for pos, b := range []byte(charset) {
if b == '@' {
charset = charset[:pos]
break
}
}
max := 1
if m, ok := mblenTable[charset]; ok {
max = m
}
if max > 1 && (charset[0] != 'u' ||
strings.HasPrefix(locale, "ja") ||
strings.HasPrefix(locale, "ko") ||
strings.HasPrefix(locale, "zh")) {
return true
}
return false
}
// IsEastAsian return true if the current locale is CJK
func IsEastAsian() bool {
locale := os.Getenv("LC_ALL")
if locale == "" {
locale = os.Getenv("LC_CTYPE")
}
if locale == "" {
locale = os.Getenv("LANG")
}
// ignore C locale
if locale == "POSIX" || locale == "C" {
return false
}
if len(locale) > 1 && locale[0] == 'C' && (locale[1] == '.' || locale[1] == '-') {
return false
}
return isEastAsian(locale)
}

View File

@@ -1,439 +0,0 @@
// Code generated by script/generate.go. DO NOT EDIT.
package runewidth
var combining = table{
{0x0300, 0x036F}, {0x0483, 0x0489}, {0x07EB, 0x07F3},
{0x0C00, 0x0C00}, {0x0C04, 0x0C04}, {0x0D00, 0x0D01},
{0x135D, 0x135F}, {0x1A7F, 0x1A7F}, {0x1AB0, 0x1AC0},
{0x1B6B, 0x1B73}, {0x1DC0, 0x1DF9}, {0x1DFB, 0x1DFF},
{0x20D0, 0x20F0}, {0x2CEF, 0x2CF1}, {0x2DE0, 0x2DFF},
{0x3099, 0x309A}, {0xA66F, 0xA672}, {0xA674, 0xA67D},
{0xA69E, 0xA69F}, {0xA6F0, 0xA6F1}, {0xA8E0, 0xA8F1},
{0xFE20, 0xFE2F}, {0x101FD, 0x101FD}, {0x10376, 0x1037A},
{0x10EAB, 0x10EAC}, {0x10F46, 0x10F50}, {0x11300, 0x11301},
{0x1133B, 0x1133C}, {0x11366, 0x1136C}, {0x11370, 0x11374},
{0x16AF0, 0x16AF4}, {0x1D165, 0x1D169}, {0x1D16D, 0x1D172},
{0x1D17B, 0x1D182}, {0x1D185, 0x1D18B}, {0x1D1AA, 0x1D1AD},
{0x1D242, 0x1D244}, {0x1E000, 0x1E006}, {0x1E008, 0x1E018},
{0x1E01B, 0x1E021}, {0x1E023, 0x1E024}, {0x1E026, 0x1E02A},
{0x1E8D0, 0x1E8D6},
}
var doublewidth = table{
{0x1100, 0x115F}, {0x231A, 0x231B}, {0x2329, 0x232A},
{0x23E9, 0x23EC}, {0x23F0, 0x23F0}, {0x23F3, 0x23F3},
{0x25FD, 0x25FE}, {0x2614, 0x2615}, {0x2648, 0x2653},
{0x267F, 0x267F}, {0x2693, 0x2693}, {0x26A1, 0x26A1},
{0x26AA, 0x26AB}, {0x26BD, 0x26BE}, {0x26C4, 0x26C5},
{0x26CE, 0x26CE}, {0x26D4, 0x26D4}, {0x26EA, 0x26EA},
{0x26F2, 0x26F3}, {0x26F5, 0x26F5}, {0x26FA, 0x26FA},
{0x26FD, 0x26FD}, {0x2705, 0x2705}, {0x270A, 0x270B},
{0x2728, 0x2728}, {0x274C, 0x274C}, {0x274E, 0x274E},
{0x2753, 0x2755}, {0x2757, 0x2757}, {0x2795, 0x2797},
{0x27B0, 0x27B0}, {0x27BF, 0x27BF}, {0x2B1B, 0x2B1C},
{0x2B50, 0x2B50}, {0x2B55, 0x2B55}, {0x2E80, 0x2E99},
{0x2E9B, 0x2EF3}, {0x2F00, 0x2FD5}, {0x2FF0, 0x2FFB},
{0x3000, 0x303E}, {0x3041, 0x3096}, {0x3099, 0x30FF},
{0x3105, 0x312F}, {0x3131, 0x318E}, {0x3190, 0x31E3},
{0x31F0, 0x321E}, {0x3220, 0x3247}, {0x3250, 0x4DBF},
{0x4E00, 0xA48C}, {0xA490, 0xA4C6}, {0xA960, 0xA97C},
{0xAC00, 0xD7A3}, {0xF900, 0xFAFF}, {0xFE10, 0xFE19},
{0xFE30, 0xFE52}, {0xFE54, 0xFE66}, {0xFE68, 0xFE6B},
{0xFF01, 0xFF60}, {0xFFE0, 0xFFE6}, {0x16FE0, 0x16FE4},
{0x16FF0, 0x16FF1}, {0x17000, 0x187F7}, {0x18800, 0x18CD5},
{0x18D00, 0x18D08}, {0x1B000, 0x1B11E}, {0x1B150, 0x1B152},
{0x1B164, 0x1B167}, {0x1B170, 0x1B2FB}, {0x1F004, 0x1F004},
{0x1F0CF, 0x1F0CF}, {0x1F18E, 0x1F18E}, {0x1F191, 0x1F19A},
{0x1F200, 0x1F202}, {0x1F210, 0x1F23B}, {0x1F240, 0x1F248},
{0x1F250, 0x1F251}, {0x1F260, 0x1F265}, {0x1F300, 0x1F320},
{0x1F32D, 0x1F335}, {0x1F337, 0x1F37C}, {0x1F37E, 0x1F393},
{0x1F3A0, 0x1F3CA}, {0x1F3CF, 0x1F3D3}, {0x1F3E0, 0x1F3F0},
{0x1F3F4, 0x1F3F4}, {0x1F3F8, 0x1F43E}, {0x1F440, 0x1F440},
{0x1F442, 0x1F4FC}, {0x1F4FF, 0x1F53D}, {0x1F54B, 0x1F54E},
{0x1F550, 0x1F567}, {0x1F57A, 0x1F57A}, {0x1F595, 0x1F596},
{0x1F5A4, 0x1F5A4}, {0x1F5FB, 0x1F64F}, {0x1F680, 0x1F6C5},
{0x1F6CC, 0x1F6CC}, {0x1F6D0, 0x1F6D2}, {0x1F6D5, 0x1F6D7},
{0x1F6EB, 0x1F6EC}, {0x1F6F4, 0x1F6FC}, {0x1F7E0, 0x1F7EB},
{0x1F90C, 0x1F93A}, {0x1F93C, 0x1F945}, {0x1F947, 0x1F978},
{0x1F97A, 0x1F9CB}, {0x1F9CD, 0x1F9FF}, {0x1FA70, 0x1FA74},
{0x1FA78, 0x1FA7A}, {0x1FA80, 0x1FA86}, {0x1FA90, 0x1FAA8},
{0x1FAB0, 0x1FAB6}, {0x1FAC0, 0x1FAC2}, {0x1FAD0, 0x1FAD6},
{0x20000, 0x2FFFD}, {0x30000, 0x3FFFD},
}
var ambiguous = table{
{0x00A1, 0x00A1}, {0x00A4, 0x00A4}, {0x00A7, 0x00A8},
{0x00AA, 0x00AA}, {0x00AD, 0x00AE}, {0x00B0, 0x00B4},
{0x00B6, 0x00BA}, {0x00BC, 0x00BF}, {0x00C6, 0x00C6},
{0x00D0, 0x00D0}, {0x00D7, 0x00D8}, {0x00DE, 0x00E1},
{0x00E6, 0x00E6}, {0x00E8, 0x00EA}, {0x00EC, 0x00ED},
{0x00F0, 0x00F0}, {0x00F2, 0x00F3}, {0x00F7, 0x00FA},
{0x00FC, 0x00FC}, {0x00FE, 0x00FE}, {0x0101, 0x0101},
{0x0111, 0x0111}, {0x0113, 0x0113}, {0x011B, 0x011B},
{0x0126, 0x0127}, {0x012B, 0x012B}, {0x0131, 0x0133},
{0x0138, 0x0138}, {0x013F, 0x0142}, {0x0144, 0x0144},
{0x0148, 0x014B}, {0x014D, 0x014D}, {0x0152, 0x0153},
{0x0166, 0x0167}, {0x016B, 0x016B}, {0x01CE, 0x01CE},
{0x01D0, 0x01D0}, {0x01D2, 0x01D2}, {0x01D4, 0x01D4},
{0x01D6, 0x01D6}, {0x01D8, 0x01D8}, {0x01DA, 0x01DA},
{0x01DC, 0x01DC}, {0x0251, 0x0251}, {0x0261, 0x0261},
{0x02C4, 0x02C4}, {0x02C7, 0x02C7}, {0x02C9, 0x02CB},
{0x02CD, 0x02CD}, {0x02D0, 0x02D0}, {0x02D8, 0x02DB},
{0x02DD, 0x02DD}, {0x02DF, 0x02DF}, {0x0300, 0x036F},
{0x0391, 0x03A1}, {0x03A3, 0x03A9}, {0x03B1, 0x03C1},
{0x03C3, 0x03C9}, {0x0401, 0x0401}, {0x0410, 0x044F},
{0x0451, 0x0451}, {0x2010, 0x2010}, {0x2013, 0x2016},
{0x2018, 0x2019}, {0x201C, 0x201D}, {0x2020, 0x2022},
{0x2024, 0x2027}, {0x2030, 0x2030}, {0x2032, 0x2033},
{0x2035, 0x2035}, {0x203B, 0x203B}, {0x203E, 0x203E},
{0x2074, 0x2074}, {0x207F, 0x207F}, {0x2081, 0x2084},
{0x20AC, 0x20AC}, {0x2103, 0x2103}, {0x2105, 0x2105},
{0x2109, 0x2109}, {0x2113, 0x2113}, {0x2116, 0x2116},
{0x2121, 0x2122}, {0x2126, 0x2126}, {0x212B, 0x212B},
{0x2153, 0x2154}, {0x215B, 0x215E}, {0x2160, 0x216B},
{0x2170, 0x2179}, {0x2189, 0x2189}, {0x2190, 0x2199},
{0x21B8, 0x21B9}, {0x21D2, 0x21D2}, {0x21D4, 0x21D4},
{0x21E7, 0x21E7}, {0x2200, 0x2200}, {0x2202, 0x2203},
{0x2207, 0x2208}, {0x220B, 0x220B}, {0x220F, 0x220F},
{0x2211, 0x2211}, {0x2215, 0x2215}, {0x221A, 0x221A},
{0x221D, 0x2220}, {0x2223, 0x2223}, {0x2225, 0x2225},
{0x2227, 0x222C}, {0x222E, 0x222E}, {0x2234, 0x2237},
{0x223C, 0x223D}, {0x2248, 0x2248}, {0x224C, 0x224C},
{0x2252, 0x2252}, {0x2260, 0x2261}, {0x2264, 0x2267},
{0x226A, 0x226B}, {0x226E, 0x226F}, {0x2282, 0x2283},
{0x2286, 0x2287}, {0x2295, 0x2295}, {0x2299, 0x2299},
{0x22A5, 0x22A5}, {0x22BF, 0x22BF}, {0x2312, 0x2312},
{0x2460, 0x24E9}, {0x24EB, 0x254B}, {0x2550, 0x2573},
{0x2580, 0x258F}, {0x2592, 0x2595}, {0x25A0, 0x25A1},
{0x25A3, 0x25A9}, {0x25B2, 0x25B3}, {0x25B6, 0x25B7},
{0x25BC, 0x25BD}, {0x25C0, 0x25C1}, {0x25C6, 0x25C8},
{0x25CB, 0x25CB}, {0x25CE, 0x25D1}, {0x25E2, 0x25E5},
{0x25EF, 0x25EF}, {0x2605, 0x2606}, {0x2609, 0x2609},
{0x260E, 0x260F}, {0x261C, 0x261C}, {0x261E, 0x261E},
{0x2640, 0x2640}, {0x2642, 0x2642}, {0x2660, 0x2661},
{0x2663, 0x2665}, {0x2667, 0x266A}, {0x266C, 0x266D},
{0x266F, 0x266F}, {0x269E, 0x269F}, {0x26BF, 0x26BF},
{0x26C6, 0x26CD}, {0x26CF, 0x26D3}, {0x26D5, 0x26E1},
{0x26E3, 0x26E3}, {0x26E8, 0x26E9}, {0x26EB, 0x26F1},
{0x26F4, 0x26F4}, {0x26F6, 0x26F9}, {0x26FB, 0x26FC},
{0x26FE, 0x26FF}, {0x273D, 0x273D}, {0x2776, 0x277F},
{0x2B56, 0x2B59}, {0x3248, 0x324F}, {0xE000, 0xF8FF},
{0xFE00, 0xFE0F}, {0xFFFD, 0xFFFD}, {0x1F100, 0x1F10A},
{0x1F110, 0x1F12D}, {0x1F130, 0x1F169}, {0x1F170, 0x1F18D},
{0x1F18F, 0x1F190}, {0x1F19B, 0x1F1AC}, {0xE0100, 0xE01EF},
{0xF0000, 0xFFFFD}, {0x100000, 0x10FFFD},
}
var narrow = table{
{0x0020, 0x007E}, {0x00A2, 0x00A3}, {0x00A5, 0x00A6},
{0x00AC, 0x00AC}, {0x00AF, 0x00AF}, {0x27E6, 0x27ED},
{0x2985, 0x2986},
}
var neutral = table{
{0x0000, 0x001F}, {0x007F, 0x00A0}, {0x00A9, 0x00A9},
{0x00AB, 0x00AB}, {0x00B5, 0x00B5}, {0x00BB, 0x00BB},
{0x00C0, 0x00C5}, {0x00C7, 0x00CF}, {0x00D1, 0x00D6},
{0x00D9, 0x00DD}, {0x00E2, 0x00E5}, {0x00E7, 0x00E7},
{0x00EB, 0x00EB}, {0x00EE, 0x00EF}, {0x00F1, 0x00F1},
{0x00F4, 0x00F6}, {0x00FB, 0x00FB}, {0x00FD, 0x00FD},
{0x00FF, 0x0100}, {0x0102, 0x0110}, {0x0112, 0x0112},
{0x0114, 0x011A}, {0x011C, 0x0125}, {0x0128, 0x012A},
{0x012C, 0x0130}, {0x0134, 0x0137}, {0x0139, 0x013E},
{0x0143, 0x0143}, {0x0145, 0x0147}, {0x014C, 0x014C},
{0x014E, 0x0151}, {0x0154, 0x0165}, {0x0168, 0x016A},
{0x016C, 0x01CD}, {0x01CF, 0x01CF}, {0x01D1, 0x01D1},
{0x01D3, 0x01D3}, {0x01D5, 0x01D5}, {0x01D7, 0x01D7},
{0x01D9, 0x01D9}, {0x01DB, 0x01DB}, {0x01DD, 0x0250},
{0x0252, 0x0260}, {0x0262, 0x02C3}, {0x02C5, 0x02C6},
{0x02C8, 0x02C8}, {0x02CC, 0x02CC}, {0x02CE, 0x02CF},
{0x02D1, 0x02D7}, {0x02DC, 0x02DC}, {0x02DE, 0x02DE},
{0x02E0, 0x02FF}, {0x0370, 0x0377}, {0x037A, 0x037F},
{0x0384, 0x038A}, {0x038C, 0x038C}, {0x038E, 0x0390},
{0x03AA, 0x03B0}, {0x03C2, 0x03C2}, {0x03CA, 0x0400},
{0x0402, 0x040F}, {0x0450, 0x0450}, {0x0452, 0x052F},
{0x0531, 0x0556}, {0x0559, 0x058A}, {0x058D, 0x058F},
{0x0591, 0x05C7}, {0x05D0, 0x05EA}, {0x05EF, 0x05F4},
{0x0600, 0x061C}, {0x061E, 0x070D}, {0x070F, 0x074A},
{0x074D, 0x07B1}, {0x07C0, 0x07FA}, {0x07FD, 0x082D},
{0x0830, 0x083E}, {0x0840, 0x085B}, {0x085E, 0x085E},
{0x0860, 0x086A}, {0x08A0, 0x08B4}, {0x08B6, 0x08C7},
{0x08D3, 0x0983}, {0x0985, 0x098C}, {0x098F, 0x0990},
{0x0993, 0x09A8}, {0x09AA, 0x09B0}, {0x09B2, 0x09B2},
{0x09B6, 0x09B9}, {0x09BC, 0x09C4}, {0x09C7, 0x09C8},
{0x09CB, 0x09CE}, {0x09D7, 0x09D7}, {0x09DC, 0x09DD},
{0x09DF, 0x09E3}, {0x09E6, 0x09FE}, {0x0A01, 0x0A03},
{0x0A05, 0x0A0A}, {0x0A0F, 0x0A10}, {0x0A13, 0x0A28},
{0x0A2A, 0x0A30}, {0x0A32, 0x0A33}, {0x0A35, 0x0A36},
{0x0A38, 0x0A39}, {0x0A3C, 0x0A3C}, {0x0A3E, 0x0A42},
{0x0A47, 0x0A48}, {0x0A4B, 0x0A4D}, {0x0A51, 0x0A51},
{0x0A59, 0x0A5C}, {0x0A5E, 0x0A5E}, {0x0A66, 0x0A76},
{0x0A81, 0x0A83}, {0x0A85, 0x0A8D}, {0x0A8F, 0x0A91},
{0x0A93, 0x0AA8}, {0x0AAA, 0x0AB0}, {0x0AB2, 0x0AB3},
{0x0AB5, 0x0AB9}, {0x0ABC, 0x0AC5}, {0x0AC7, 0x0AC9},
{0x0ACB, 0x0ACD}, {0x0AD0, 0x0AD0}, {0x0AE0, 0x0AE3},
{0x0AE6, 0x0AF1}, {0x0AF9, 0x0AFF}, {0x0B01, 0x0B03},
{0x0B05, 0x0B0C}, {0x0B0F, 0x0B10}, {0x0B13, 0x0B28},
{0x0B2A, 0x0B30}, {0x0B32, 0x0B33}, {0x0B35, 0x0B39},
{0x0B3C, 0x0B44}, {0x0B47, 0x0B48}, {0x0B4B, 0x0B4D},
{0x0B55, 0x0B57}, {0x0B5C, 0x0B5D}, {0x0B5F, 0x0B63},
{0x0B66, 0x0B77}, {0x0B82, 0x0B83}, {0x0B85, 0x0B8A},
{0x0B8E, 0x0B90}, {0x0B92, 0x0B95}, {0x0B99, 0x0B9A},
{0x0B9C, 0x0B9C}, {0x0B9E, 0x0B9F}, {0x0BA3, 0x0BA4},
{0x0BA8, 0x0BAA}, {0x0BAE, 0x0BB9}, {0x0BBE, 0x0BC2},
{0x0BC6, 0x0BC8}, {0x0BCA, 0x0BCD}, {0x0BD0, 0x0BD0},
{0x0BD7, 0x0BD7}, {0x0BE6, 0x0BFA}, {0x0C00, 0x0C0C},
{0x0C0E, 0x0C10}, {0x0C12, 0x0C28}, {0x0C2A, 0x0C39},
{0x0C3D, 0x0C44}, {0x0C46, 0x0C48}, {0x0C4A, 0x0C4D},
{0x0C55, 0x0C56}, {0x0C58, 0x0C5A}, {0x0C60, 0x0C63},
{0x0C66, 0x0C6F}, {0x0C77, 0x0C8C}, {0x0C8E, 0x0C90},
{0x0C92, 0x0CA8}, {0x0CAA, 0x0CB3}, {0x0CB5, 0x0CB9},
{0x0CBC, 0x0CC4}, {0x0CC6, 0x0CC8}, {0x0CCA, 0x0CCD},
{0x0CD5, 0x0CD6}, {0x0CDE, 0x0CDE}, {0x0CE0, 0x0CE3},
{0x0CE6, 0x0CEF}, {0x0CF1, 0x0CF2}, {0x0D00, 0x0D0C},
{0x0D0E, 0x0D10}, {0x0D12, 0x0D44}, {0x0D46, 0x0D48},
{0x0D4A, 0x0D4F}, {0x0D54, 0x0D63}, {0x0D66, 0x0D7F},
{0x0D81, 0x0D83}, {0x0D85, 0x0D96}, {0x0D9A, 0x0DB1},
{0x0DB3, 0x0DBB}, {0x0DBD, 0x0DBD}, {0x0DC0, 0x0DC6},
{0x0DCA, 0x0DCA}, {0x0DCF, 0x0DD4}, {0x0DD6, 0x0DD6},
{0x0DD8, 0x0DDF}, {0x0DE6, 0x0DEF}, {0x0DF2, 0x0DF4},
{0x0E01, 0x0E3A}, {0x0E3F, 0x0E5B}, {0x0E81, 0x0E82},
{0x0E84, 0x0E84}, {0x0E86, 0x0E8A}, {0x0E8C, 0x0EA3},
{0x0EA5, 0x0EA5}, {0x0EA7, 0x0EBD}, {0x0EC0, 0x0EC4},
{0x0EC6, 0x0EC6}, {0x0EC8, 0x0ECD}, {0x0ED0, 0x0ED9},
{0x0EDC, 0x0EDF}, {0x0F00, 0x0F47}, {0x0F49, 0x0F6C},
{0x0F71, 0x0F97}, {0x0F99, 0x0FBC}, {0x0FBE, 0x0FCC},
{0x0FCE, 0x0FDA}, {0x1000, 0x10C5}, {0x10C7, 0x10C7},
{0x10CD, 0x10CD}, {0x10D0, 0x10FF}, {0x1160, 0x1248},
{0x124A, 0x124D}, {0x1250, 0x1256}, {0x1258, 0x1258},
{0x125A, 0x125D}, {0x1260, 0x1288}, {0x128A, 0x128D},
{0x1290, 0x12B0}, {0x12B2, 0x12B5}, {0x12B8, 0x12BE},
{0x12C0, 0x12C0}, {0x12C2, 0x12C5}, {0x12C8, 0x12D6},
{0x12D8, 0x1310}, {0x1312, 0x1315}, {0x1318, 0x135A},
{0x135D, 0x137C}, {0x1380, 0x1399}, {0x13A0, 0x13F5},
{0x13F8, 0x13FD}, {0x1400, 0x169C}, {0x16A0, 0x16F8},
{0x1700, 0x170C}, {0x170E, 0x1714}, {0x1720, 0x1736},
{0x1740, 0x1753}, {0x1760, 0x176C}, {0x176E, 0x1770},
{0x1772, 0x1773}, {0x1780, 0x17DD}, {0x17E0, 0x17E9},
{0x17F0, 0x17F9}, {0x1800, 0x180E}, {0x1810, 0x1819},
{0x1820, 0x1878}, {0x1880, 0x18AA}, {0x18B0, 0x18F5},
{0x1900, 0x191E}, {0x1920, 0x192B}, {0x1930, 0x193B},
{0x1940, 0x1940}, {0x1944, 0x196D}, {0x1970, 0x1974},
{0x1980, 0x19AB}, {0x19B0, 0x19C9}, {0x19D0, 0x19DA},
{0x19DE, 0x1A1B}, {0x1A1E, 0x1A5E}, {0x1A60, 0x1A7C},
{0x1A7F, 0x1A89}, {0x1A90, 0x1A99}, {0x1AA0, 0x1AAD},
{0x1AB0, 0x1AC0}, {0x1B00, 0x1B4B}, {0x1B50, 0x1B7C},
{0x1B80, 0x1BF3}, {0x1BFC, 0x1C37}, {0x1C3B, 0x1C49},
{0x1C4D, 0x1C88}, {0x1C90, 0x1CBA}, {0x1CBD, 0x1CC7},
{0x1CD0, 0x1CFA}, {0x1D00, 0x1DF9}, {0x1DFB, 0x1F15},
{0x1F18, 0x1F1D}, {0x1F20, 0x1F45}, {0x1F48, 0x1F4D},
{0x1F50, 0x1F57}, {0x1F59, 0x1F59}, {0x1F5B, 0x1F5B},
{0x1F5D, 0x1F5D}, {0x1F5F, 0x1F7D}, {0x1F80, 0x1FB4},
{0x1FB6, 0x1FC4}, {0x1FC6, 0x1FD3}, {0x1FD6, 0x1FDB},
{0x1FDD, 0x1FEF}, {0x1FF2, 0x1FF4}, {0x1FF6, 0x1FFE},
{0x2000, 0x200F}, {0x2011, 0x2012}, {0x2017, 0x2017},
{0x201A, 0x201B}, {0x201E, 0x201F}, {0x2023, 0x2023},
{0x2028, 0x202F}, {0x2031, 0x2031}, {0x2034, 0x2034},
{0x2036, 0x203A}, {0x203C, 0x203D}, {0x203F, 0x2064},
{0x2066, 0x2071}, {0x2075, 0x207E}, {0x2080, 0x2080},
{0x2085, 0x208E}, {0x2090, 0x209C}, {0x20A0, 0x20A8},
{0x20AA, 0x20AB}, {0x20AD, 0x20BF}, {0x20D0, 0x20F0},
{0x2100, 0x2102}, {0x2104, 0x2104}, {0x2106, 0x2108},
{0x210A, 0x2112}, {0x2114, 0x2115}, {0x2117, 0x2120},
{0x2123, 0x2125}, {0x2127, 0x212A}, {0x212C, 0x2152},
{0x2155, 0x215A}, {0x215F, 0x215F}, {0x216C, 0x216F},
{0x217A, 0x2188}, {0x218A, 0x218B}, {0x219A, 0x21B7},
{0x21BA, 0x21D1}, {0x21D3, 0x21D3}, {0x21D5, 0x21E6},
{0x21E8, 0x21FF}, {0x2201, 0x2201}, {0x2204, 0x2206},
{0x2209, 0x220A}, {0x220C, 0x220E}, {0x2210, 0x2210},
{0x2212, 0x2214}, {0x2216, 0x2219}, {0x221B, 0x221C},
{0x2221, 0x2222}, {0x2224, 0x2224}, {0x2226, 0x2226},
{0x222D, 0x222D}, {0x222F, 0x2233}, {0x2238, 0x223B},
{0x223E, 0x2247}, {0x2249, 0x224B}, {0x224D, 0x2251},
{0x2253, 0x225F}, {0x2262, 0x2263}, {0x2268, 0x2269},
{0x226C, 0x226D}, {0x2270, 0x2281}, {0x2284, 0x2285},
{0x2288, 0x2294}, {0x2296, 0x2298}, {0x229A, 0x22A4},
{0x22A6, 0x22BE}, {0x22C0, 0x2311}, {0x2313, 0x2319},
{0x231C, 0x2328}, {0x232B, 0x23E8}, {0x23ED, 0x23EF},
{0x23F1, 0x23F2}, {0x23F4, 0x2426}, {0x2440, 0x244A},
{0x24EA, 0x24EA}, {0x254C, 0x254F}, {0x2574, 0x257F},
{0x2590, 0x2591}, {0x2596, 0x259F}, {0x25A2, 0x25A2},
{0x25AA, 0x25B1}, {0x25B4, 0x25B5}, {0x25B8, 0x25BB},
{0x25BE, 0x25BF}, {0x25C2, 0x25C5}, {0x25C9, 0x25CA},
{0x25CC, 0x25CD}, {0x25D2, 0x25E1}, {0x25E6, 0x25EE},
{0x25F0, 0x25FC}, {0x25FF, 0x2604}, {0x2607, 0x2608},
{0x260A, 0x260D}, {0x2610, 0x2613}, {0x2616, 0x261B},
{0x261D, 0x261D}, {0x261F, 0x263F}, {0x2641, 0x2641},
{0x2643, 0x2647}, {0x2654, 0x265F}, {0x2662, 0x2662},
{0x2666, 0x2666}, {0x266B, 0x266B}, {0x266E, 0x266E},
{0x2670, 0x267E}, {0x2680, 0x2692}, {0x2694, 0x269D},
{0x26A0, 0x26A0}, {0x26A2, 0x26A9}, {0x26AC, 0x26BC},
{0x26C0, 0x26C3}, {0x26E2, 0x26E2}, {0x26E4, 0x26E7},
{0x2700, 0x2704}, {0x2706, 0x2709}, {0x270C, 0x2727},
{0x2729, 0x273C}, {0x273E, 0x274B}, {0x274D, 0x274D},
{0x274F, 0x2752}, {0x2756, 0x2756}, {0x2758, 0x2775},
{0x2780, 0x2794}, {0x2798, 0x27AF}, {0x27B1, 0x27BE},
{0x27C0, 0x27E5}, {0x27EE, 0x2984}, {0x2987, 0x2B1A},
{0x2B1D, 0x2B4F}, {0x2B51, 0x2B54}, {0x2B5A, 0x2B73},
{0x2B76, 0x2B95}, {0x2B97, 0x2C2E}, {0x2C30, 0x2C5E},
{0x2C60, 0x2CF3}, {0x2CF9, 0x2D25}, {0x2D27, 0x2D27},
{0x2D2D, 0x2D2D}, {0x2D30, 0x2D67}, {0x2D6F, 0x2D70},
{0x2D7F, 0x2D96}, {0x2DA0, 0x2DA6}, {0x2DA8, 0x2DAE},
{0x2DB0, 0x2DB6}, {0x2DB8, 0x2DBE}, {0x2DC0, 0x2DC6},
{0x2DC8, 0x2DCE}, {0x2DD0, 0x2DD6}, {0x2DD8, 0x2DDE},
{0x2DE0, 0x2E52}, {0x303F, 0x303F}, {0x4DC0, 0x4DFF},
{0xA4D0, 0xA62B}, {0xA640, 0xA6F7}, {0xA700, 0xA7BF},
{0xA7C2, 0xA7CA}, {0xA7F5, 0xA82C}, {0xA830, 0xA839},
{0xA840, 0xA877}, {0xA880, 0xA8C5}, {0xA8CE, 0xA8D9},
{0xA8E0, 0xA953}, {0xA95F, 0xA95F}, {0xA980, 0xA9CD},
{0xA9CF, 0xA9D9}, {0xA9DE, 0xA9FE}, {0xAA00, 0xAA36},
{0xAA40, 0xAA4D}, {0xAA50, 0xAA59}, {0xAA5C, 0xAAC2},
{0xAADB, 0xAAF6}, {0xAB01, 0xAB06}, {0xAB09, 0xAB0E},
{0xAB11, 0xAB16}, {0xAB20, 0xAB26}, {0xAB28, 0xAB2E},
{0xAB30, 0xAB6B}, {0xAB70, 0xABED}, {0xABF0, 0xABF9},
{0xD7B0, 0xD7C6}, {0xD7CB, 0xD7FB}, {0xD800, 0xDFFF},
{0xFB00, 0xFB06}, {0xFB13, 0xFB17}, {0xFB1D, 0xFB36},
{0xFB38, 0xFB3C}, {0xFB3E, 0xFB3E}, {0xFB40, 0xFB41},
{0xFB43, 0xFB44}, {0xFB46, 0xFBC1}, {0xFBD3, 0xFD3F},
{0xFD50, 0xFD8F}, {0xFD92, 0xFDC7}, {0xFDF0, 0xFDFD},
{0xFE20, 0xFE2F}, {0xFE70, 0xFE74}, {0xFE76, 0xFEFC},
{0xFEFF, 0xFEFF}, {0xFFF9, 0xFFFC}, {0x10000, 0x1000B},
{0x1000D, 0x10026}, {0x10028, 0x1003A}, {0x1003C, 0x1003D},
{0x1003F, 0x1004D}, {0x10050, 0x1005D}, {0x10080, 0x100FA},
{0x10100, 0x10102}, {0x10107, 0x10133}, {0x10137, 0x1018E},
{0x10190, 0x1019C}, {0x101A0, 0x101A0}, {0x101D0, 0x101FD},
{0x10280, 0x1029C}, {0x102A0, 0x102D0}, {0x102E0, 0x102FB},
{0x10300, 0x10323}, {0x1032D, 0x1034A}, {0x10350, 0x1037A},
{0x10380, 0x1039D}, {0x1039F, 0x103C3}, {0x103C8, 0x103D5},
{0x10400, 0x1049D}, {0x104A0, 0x104A9}, {0x104B0, 0x104D3},
{0x104D8, 0x104FB}, {0x10500, 0x10527}, {0x10530, 0x10563},
{0x1056F, 0x1056F}, {0x10600, 0x10736}, {0x10740, 0x10755},
{0x10760, 0x10767}, {0x10800, 0x10805}, {0x10808, 0x10808},
{0x1080A, 0x10835}, {0x10837, 0x10838}, {0x1083C, 0x1083C},
{0x1083F, 0x10855}, {0x10857, 0x1089E}, {0x108A7, 0x108AF},
{0x108E0, 0x108F2}, {0x108F4, 0x108F5}, {0x108FB, 0x1091B},
{0x1091F, 0x10939}, {0x1093F, 0x1093F}, {0x10980, 0x109B7},
{0x109BC, 0x109CF}, {0x109D2, 0x10A03}, {0x10A05, 0x10A06},
{0x10A0C, 0x10A13}, {0x10A15, 0x10A17}, {0x10A19, 0x10A35},
{0x10A38, 0x10A3A}, {0x10A3F, 0x10A48}, {0x10A50, 0x10A58},
{0x10A60, 0x10A9F}, {0x10AC0, 0x10AE6}, {0x10AEB, 0x10AF6},
{0x10B00, 0x10B35}, {0x10B39, 0x10B55}, {0x10B58, 0x10B72},
{0x10B78, 0x10B91}, {0x10B99, 0x10B9C}, {0x10BA9, 0x10BAF},
{0x10C00, 0x10C48}, {0x10C80, 0x10CB2}, {0x10CC0, 0x10CF2},
{0x10CFA, 0x10D27}, {0x10D30, 0x10D39}, {0x10E60, 0x10E7E},
{0x10E80, 0x10EA9}, {0x10EAB, 0x10EAD}, {0x10EB0, 0x10EB1},
{0x10F00, 0x10F27}, {0x10F30, 0x10F59}, {0x10FB0, 0x10FCB},
{0x10FE0, 0x10FF6}, {0x11000, 0x1104D}, {0x11052, 0x1106F},
{0x1107F, 0x110C1}, {0x110CD, 0x110CD}, {0x110D0, 0x110E8},
{0x110F0, 0x110F9}, {0x11100, 0x11134}, {0x11136, 0x11147},
{0x11150, 0x11176}, {0x11180, 0x111DF}, {0x111E1, 0x111F4},
{0x11200, 0x11211}, {0x11213, 0x1123E}, {0x11280, 0x11286},
{0x11288, 0x11288}, {0x1128A, 0x1128D}, {0x1128F, 0x1129D},
{0x1129F, 0x112A9}, {0x112B0, 0x112EA}, {0x112F0, 0x112F9},
{0x11300, 0x11303}, {0x11305, 0x1130C}, {0x1130F, 0x11310},
{0x11313, 0x11328}, {0x1132A, 0x11330}, {0x11332, 0x11333},
{0x11335, 0x11339}, {0x1133B, 0x11344}, {0x11347, 0x11348},
{0x1134B, 0x1134D}, {0x11350, 0x11350}, {0x11357, 0x11357},
{0x1135D, 0x11363}, {0x11366, 0x1136C}, {0x11370, 0x11374},
{0x11400, 0x1145B}, {0x1145D, 0x11461}, {0x11480, 0x114C7},
{0x114D0, 0x114D9}, {0x11580, 0x115B5}, {0x115B8, 0x115DD},
{0x11600, 0x11644}, {0x11650, 0x11659}, {0x11660, 0x1166C},
{0x11680, 0x116B8}, {0x116C0, 0x116C9}, {0x11700, 0x1171A},
{0x1171D, 0x1172B}, {0x11730, 0x1173F}, {0x11800, 0x1183B},
{0x118A0, 0x118F2}, {0x118FF, 0x11906}, {0x11909, 0x11909},
{0x1190C, 0x11913}, {0x11915, 0x11916}, {0x11918, 0x11935},
{0x11937, 0x11938}, {0x1193B, 0x11946}, {0x11950, 0x11959},
{0x119A0, 0x119A7}, {0x119AA, 0x119D7}, {0x119DA, 0x119E4},
{0x11A00, 0x11A47}, {0x11A50, 0x11AA2}, {0x11AC0, 0x11AF8},
{0x11C00, 0x11C08}, {0x11C0A, 0x11C36}, {0x11C38, 0x11C45},
{0x11C50, 0x11C6C}, {0x11C70, 0x11C8F}, {0x11C92, 0x11CA7},
{0x11CA9, 0x11CB6}, {0x11D00, 0x11D06}, {0x11D08, 0x11D09},
{0x11D0B, 0x11D36}, {0x11D3A, 0x11D3A}, {0x11D3C, 0x11D3D},
{0x11D3F, 0x11D47}, {0x11D50, 0x11D59}, {0x11D60, 0x11D65},
{0x11D67, 0x11D68}, {0x11D6A, 0x11D8E}, {0x11D90, 0x11D91},
{0x11D93, 0x11D98}, {0x11DA0, 0x11DA9}, {0x11EE0, 0x11EF8},
{0x11FB0, 0x11FB0}, {0x11FC0, 0x11FF1}, {0x11FFF, 0x12399},
{0x12400, 0x1246E}, {0x12470, 0x12474}, {0x12480, 0x12543},
{0x13000, 0x1342E}, {0x13430, 0x13438}, {0x14400, 0x14646},
{0x16800, 0x16A38}, {0x16A40, 0x16A5E}, {0x16A60, 0x16A69},
{0x16A6E, 0x16A6F}, {0x16AD0, 0x16AED}, {0x16AF0, 0x16AF5},
{0x16B00, 0x16B45}, {0x16B50, 0x16B59}, {0x16B5B, 0x16B61},
{0x16B63, 0x16B77}, {0x16B7D, 0x16B8F}, {0x16E40, 0x16E9A},
{0x16F00, 0x16F4A}, {0x16F4F, 0x16F87}, {0x16F8F, 0x16F9F},
{0x1BC00, 0x1BC6A}, {0x1BC70, 0x1BC7C}, {0x1BC80, 0x1BC88},
{0x1BC90, 0x1BC99}, {0x1BC9C, 0x1BCA3}, {0x1D000, 0x1D0F5},
{0x1D100, 0x1D126}, {0x1D129, 0x1D1E8}, {0x1D200, 0x1D245},
{0x1D2E0, 0x1D2F3}, {0x1D300, 0x1D356}, {0x1D360, 0x1D378},
{0x1D400, 0x1D454}, {0x1D456, 0x1D49C}, {0x1D49E, 0x1D49F},
{0x1D4A2, 0x1D4A2}, {0x1D4A5, 0x1D4A6}, {0x1D4A9, 0x1D4AC},
{0x1D4AE, 0x1D4B9}, {0x1D4BB, 0x1D4BB}, {0x1D4BD, 0x1D4C3},
{0x1D4C5, 0x1D505}, {0x1D507, 0x1D50A}, {0x1D50D, 0x1D514},
{0x1D516, 0x1D51C}, {0x1D51E, 0x1D539}, {0x1D53B, 0x1D53E},
{0x1D540, 0x1D544}, {0x1D546, 0x1D546}, {0x1D54A, 0x1D550},
{0x1D552, 0x1D6A5}, {0x1D6A8, 0x1D7CB}, {0x1D7CE, 0x1DA8B},
{0x1DA9B, 0x1DA9F}, {0x1DAA1, 0x1DAAF}, {0x1E000, 0x1E006},
{0x1E008, 0x1E018}, {0x1E01B, 0x1E021}, {0x1E023, 0x1E024},
{0x1E026, 0x1E02A}, {0x1E100, 0x1E12C}, {0x1E130, 0x1E13D},
{0x1E140, 0x1E149}, {0x1E14E, 0x1E14F}, {0x1E2C0, 0x1E2F9},
{0x1E2FF, 0x1E2FF}, {0x1E800, 0x1E8C4}, {0x1E8C7, 0x1E8D6},
{0x1E900, 0x1E94B}, {0x1E950, 0x1E959}, {0x1E95E, 0x1E95F},
{0x1EC71, 0x1ECB4}, {0x1ED01, 0x1ED3D}, {0x1EE00, 0x1EE03},
{0x1EE05, 0x1EE1F}, {0x1EE21, 0x1EE22}, {0x1EE24, 0x1EE24},
{0x1EE27, 0x1EE27}, {0x1EE29, 0x1EE32}, {0x1EE34, 0x1EE37},
{0x1EE39, 0x1EE39}, {0x1EE3B, 0x1EE3B}, {0x1EE42, 0x1EE42},
{0x1EE47, 0x1EE47}, {0x1EE49, 0x1EE49}, {0x1EE4B, 0x1EE4B},
{0x1EE4D, 0x1EE4F}, {0x1EE51, 0x1EE52}, {0x1EE54, 0x1EE54},
{0x1EE57, 0x1EE57}, {0x1EE59, 0x1EE59}, {0x1EE5B, 0x1EE5B},
{0x1EE5D, 0x1EE5D}, {0x1EE5F, 0x1EE5F}, {0x1EE61, 0x1EE62},
{0x1EE64, 0x1EE64}, {0x1EE67, 0x1EE6A}, {0x1EE6C, 0x1EE72},
{0x1EE74, 0x1EE77}, {0x1EE79, 0x1EE7C}, {0x1EE7E, 0x1EE7E},
{0x1EE80, 0x1EE89}, {0x1EE8B, 0x1EE9B}, {0x1EEA1, 0x1EEA3},
{0x1EEA5, 0x1EEA9}, {0x1EEAB, 0x1EEBB}, {0x1EEF0, 0x1EEF1},
{0x1F000, 0x1F003}, {0x1F005, 0x1F02B}, {0x1F030, 0x1F093},
{0x1F0A0, 0x1F0AE}, {0x1F0B1, 0x1F0BF}, {0x1F0C1, 0x1F0CE},
{0x1F0D1, 0x1F0F5}, {0x1F10B, 0x1F10F}, {0x1F12E, 0x1F12F},
{0x1F16A, 0x1F16F}, {0x1F1AD, 0x1F1AD}, {0x1F1E6, 0x1F1FF},
{0x1F321, 0x1F32C}, {0x1F336, 0x1F336}, {0x1F37D, 0x1F37D},
{0x1F394, 0x1F39F}, {0x1F3CB, 0x1F3CE}, {0x1F3D4, 0x1F3DF},
{0x1F3F1, 0x1F3F3}, {0x1F3F5, 0x1F3F7}, {0x1F43F, 0x1F43F},
{0x1F441, 0x1F441}, {0x1F4FD, 0x1F4FE}, {0x1F53E, 0x1F54A},
{0x1F54F, 0x1F54F}, {0x1F568, 0x1F579}, {0x1F57B, 0x1F594},
{0x1F597, 0x1F5A3}, {0x1F5A5, 0x1F5FA}, {0x1F650, 0x1F67F},
{0x1F6C6, 0x1F6CB}, {0x1F6CD, 0x1F6CF}, {0x1F6D3, 0x1F6D4},
{0x1F6E0, 0x1F6EA}, {0x1F6F0, 0x1F6F3}, {0x1F700, 0x1F773},
{0x1F780, 0x1F7D8}, {0x1F800, 0x1F80B}, {0x1F810, 0x1F847},
{0x1F850, 0x1F859}, {0x1F860, 0x1F887}, {0x1F890, 0x1F8AD},
{0x1F8B0, 0x1F8B1}, {0x1F900, 0x1F90B}, {0x1F93B, 0x1F93B},
{0x1F946, 0x1F946}, {0x1FA00, 0x1FA53}, {0x1FA60, 0x1FA6D},
{0x1FB00, 0x1FB92}, {0x1FB94, 0x1FBCA}, {0x1FBF0, 0x1FBF9},
{0xE0001, 0xE0001}, {0xE0020, 0xE007F},
}
var emoji = table{
{0x203C, 0x203C}, {0x2049, 0x2049}, {0x2122, 0x2122},
{0x2139, 0x2139}, {0x2194, 0x2199}, {0x21A9, 0x21AA},
{0x231A, 0x231B}, {0x2328, 0x2328}, {0x2388, 0x2388},
{0x23CF, 0x23CF}, {0x23E9, 0x23F3}, {0x23F8, 0x23FA},
{0x24C2, 0x24C2}, {0x25AA, 0x25AB}, {0x25B6, 0x25B6},
{0x25C0, 0x25C0}, {0x25FB, 0x25FE}, {0x2600, 0x2605},
{0x2607, 0x2612}, {0x2614, 0x2685}, {0x2690, 0x2705},
{0x2708, 0x2712}, {0x2714, 0x2714}, {0x2716, 0x2716},
{0x271D, 0x271D}, {0x2721, 0x2721}, {0x2728, 0x2728},
{0x2733, 0x2734}, {0x2744, 0x2744}, {0x2747, 0x2747},
{0x274C, 0x274C}, {0x274E, 0x274E}, {0x2753, 0x2755},
{0x2757, 0x2757}, {0x2763, 0x2767}, {0x2795, 0x2797},
{0x27A1, 0x27A1}, {0x27B0, 0x27B0}, {0x27BF, 0x27BF},
{0x2934, 0x2935}, {0x2B05, 0x2B07}, {0x2B1B, 0x2B1C},
{0x2B50, 0x2B50}, {0x2B55, 0x2B55}, {0x3030, 0x3030},
{0x303D, 0x303D}, {0x3297, 0x3297}, {0x3299, 0x3299},
{0x1F000, 0x1F0FF}, {0x1F10D, 0x1F10F}, {0x1F12F, 0x1F12F},
{0x1F16C, 0x1F171}, {0x1F17E, 0x1F17F}, {0x1F18E, 0x1F18E},
{0x1F191, 0x1F19A}, {0x1F1AD, 0x1F1E5}, {0x1F201, 0x1F20F},
{0x1F21A, 0x1F21A}, {0x1F22F, 0x1F22F}, {0x1F232, 0x1F23A},
{0x1F23C, 0x1F23F}, {0x1F249, 0x1F3FA}, {0x1F400, 0x1F53D},
{0x1F546, 0x1F64F}, {0x1F680, 0x1F6FF}, {0x1F774, 0x1F77F},
{0x1F7D5, 0x1F7FF}, {0x1F80C, 0x1F80F}, {0x1F848, 0x1F84F},
{0x1F85A, 0x1F85F}, {0x1F888, 0x1F88F}, {0x1F8AE, 0x1F8FF},
{0x1F90C, 0x1F93A}, {0x1F93C, 0x1F945}, {0x1F947, 0x1FAFF},
{0x1FC00, 0x1FFFD},
}

View File

@@ -1,28 +0,0 @@
// +build windows
// +build !appengine
package runewidth
import (
"syscall"
)
var (
kernel32 = syscall.NewLazyDLL("kernel32")
procGetConsoleOutputCP = kernel32.NewProc("GetConsoleOutputCP")
)
// IsEastAsian return true if the current locale is CJK
func IsEastAsian() bool {
r1, _, _ := procGetConsoleOutputCP.Call()
if r1 == 0 {
return false
}
switch int(r1) {
case 932, 51932, 936, 949, 950:
return true
}
return false
}

View File

@@ -1,21 +0,0 @@
# Binaries for programs and plugins
*.exe
*.exe~
*.dll
*.so
*.dylib
# Test binary, built with `go test -c`
*.test
# Output of the go coverage tool, specifically when used with LiteIDE
*.out
# Dependency directories (remove the comment below to include it)
vendor/
# This is where we test stuff
/experimenting/
/.history
/.vscode

View File

@@ -1,92 +0,0 @@
linters-settings:
gocritic:
enabled-tags:
- diagnostic
- experimental
- opinionated
- performance
- style
disabled-checks:
- dupImport
- ifElseChain
- octalLiteral
- whyNoLint
- wrapperFunc
- exitAfterDefer
- hugeParam
- ptrToRefParam
- paramTypeCombine
- unnamedResult
# maligned:
# suggest-new: true
misspell:
locale: US
linters:
disable-all: true
enable:
- gocritic
- gosec
- govet
- ineffassign
- interfacer
- unconvert
- gosimple
- godox
- whitespace
- staticcheck
# - bodyclose
# - maligned
# - godot
# - deadcode
# - depguard
# - dogsled
# - dupl
# - errcheck
# - exhaustive
# - funlen
# - gochecknoinits
# - goconst
# - gocyclo
# - gofmt
# - goimports
# - golint
# - gomnd
# - goprintffuncname
# - lll
# - misspell
# - nakedret
# - noctx
# - nolintlint
# - rowserrcheck
# - scopelint
# - structcheck
# - stylecheck
# - typecheck
# - unparam
# - unused
# - varcheck
# - whitespace
# - asciicheck
# - gochecknoglobals
# - gocognit
# - goerr113
# - nestif
# - prealloc
# - testpackage
# - wsl
issues:
# Excluding configuration per-path, per-linter, per-text and per-source
exclude-rules:
- path: _test\.go
linters:
- gocyclo
- errcheck
- dupl
- gosec
- gocritic
# https://github.com/go-critic/go-critic/issues/926
- linters:
- gocritic
text: "unnecessaryDefer:"
service:
golangci-lint-version: 1.31.x # use the fixed version to not introduce new linters unexpectedly

File diff suppressed because it is too large Load Diff

Some files were not shown because too many files have changed in this diff Show More