sneak/mfer

Files

check / check (push) Successful in 38s

Details

Configure prettier and make fmt-check cover markdown (closes #69 )

script/fmt ran prettier with default settings over root-level *.md and
*.json, swallowing every failure with `|| true`, while script/fmt-check
checked gofmt only. The formatter and the gate therefore disagreed
silently: `make fmt` rewrote markdown that `make check` never looked at,
including REPO_POLICIES.md, which is a verbatim copy of an authoritative
upstream document that local tooling must not touch.

Configuration:

- .prettierrc pins the two policy deviations from prettier defaults,
  four-space indents and proseWrap: always. Nothing else.
- .prettierignore excludes REPO_POLICIES.md so no local run can drift it
  from upstream again, plus .golangci.yml (user-owned, and listed even
  though the current file set does not reach it) and node_modules,
  vendor, bin.

One canonical file set:

- New script/prettier takes --write or --check and applies the same
  patterns in both modes, so script/fmt and script/fmt-check cannot
  drift apart by construction. The patterns are repo-wide (**/*.md,
  **/*.json) rather than root-only, so markdown in subdirectories such
  as a future docs/ is covered.
- No `|| true` anywhere, and no --no-error-on-unmatched-pattern: both
  patterns always match tracked files, so an empty match means the glob
  broke and prettier should say so instead of passing vacuously. A
  missing prettier is a hard error naming script/bootstrap, not a
  silent skip.

Pinned prettier:

- package.json/yarn.lock pin prettier 3.9.6; the lockfile carries the
  integrity hash, and --frozen-lockfile enforces it. script/prettier
  prefers node_modules/.bin/prettier and warns on stderr when it has to
  fall back to a PATH prettier of unknown version.
- script/bootstrap now installs node, yarn, and the locked JS deps. Its
  NODE_VERSION and YARN_VERSION pins already existed.

Docker gate:

- The golangci-lint image has no node, so the lint stage runs the new
  script/fmt-check-go (the Go half of fmt-check, extracted) instead of
  the whole thing.
- The markdown half gets its own stage on a digest-pinned node image
  shipping exactly the node and yarn versions bootstrap pins. The
  builder stage takes a COPY --from dependency on it, so BuildKit cannot
  skip it and a markdown violation fails `docker build .` rather than
  being skipped somewhere nobody looks.

Markdown files other than REPO_POLICIES.md are reformatted here for the
first time under the policy settings.

2026-08-10 14:08:33 +00:00

6.2 KiB

Raw Blame History

.mf File Format Specification

Version 1.0

Overview

An .mf file is a binary manifest that describes a directory tree of files, including their paths, sizes, and cryptographic checksums. It supports optional GPG signatures for integrity verification and optional timestamps for metadata preservation.

File Structure

An .mf file consists of two parts, concatenated:

Magic bytes (8 bytes): the ASCII string ZNAVSRFG
Outer message: a Protocol Buffers serialized MFFileOuter message

There is no length prefix or version byte between the magic and the protobuf message. The protobuf message extends to the end of the file.

See mfer/mf.proto for exact field numbers and types.

Outer Message (`MFFileOuter`)

The outer message contains:

Field	Number	Type	Description
`version`	101	enum	Must be `VERSION_ONE` (1)
`compressionType`	102	enum	Compression of `innerMessage`; must be `COMPRESSION_ZSTD` (1)
`size`	103	int64	Uncompressed size of `innerMessage` (corruption detection)
`sha256`	104	bytes	SHA-256 hash of the compressed `innerMessage` (corruption detection)
`uuid`	105	bytes	Random v4 UUID; must match the inner message UUID
`innerMessage`	199	bytes	Zstd-compressed serialized `MFFile` message
`signature`	201	bytes (optional)	GPG signature (ASCII-armored or binary)
`signer`	202	bytes (optional)	Full GPG key ID of the signer
`signingPubKey`	203	bytes (optional)	Full GPG signing public key

SHA-256 Hash

The sha256 field (104) covers the compressed innerMessage bytes. This allows verifying data integrity before decompression.

Compression

The innerMessage field is compressed with Zstandard (zstd). Implementations must enforce a decompression size limit to prevent decompression bombs. The reference implementation limits decompressed size to 256 MB.

Inner Message (`MFFile`)

After decompressing innerMessage, the result is a serialized MFFile (referred to as the manifest):

Field	Number	Type	Description
`version`	100	enum	Must be `VERSION_ONE` (1)
`files`	101	repeated `MFFilePath`	List of files in the manifest
`uuid`	102	bytes	Random v4 UUID; must match outer UUID
`createdAt`	201	Timestamp (optional)	When the manifest was created

File Entries (`MFFilePath`)

Each file entry contains:

Field	Number	Type	Description
`path`	1	string	Relative file path (see Path Rules)
`size`	2	int64	File size in bytes
`hashes`	3	repeated `MFFileChecksum`	At least one hash required
`mimeType`	301	string (optional)	MIME type
`mtime`	302	Timestamp (optional)	Modification time
`ctime`	303	Timestamp (optional)	Change time (inode metadata change)

Field 304 (atime) has been removed from the specification. Access time is volatile and non-deterministic; it is not useful for integrity verification.

Path Rules

All path values must satisfy these invariants:

UTF-8: paths must be valid UTF-8
Forward slashes: use / as the path separator (never \)
Relative only: no leading /
No parent traversal: no .. path segments
No empty segments: no // sequences
No trailing slash: paths refer to files, not directories

Implementations must validate these invariants when reading and writing manifests. Paths that violate these rules must be rejected.

Hash Format (`MFFileChecksum`)

Each checksum is a single bytes multiHash field containing a multihash-encoded value. Multihash is self-describing: the encoded bytes include a varint algorithm identifier followed by a varint digest length followed by the digest itself.

The 1.0 implementation writes SHA-256 multihashes (0x12 algorithm code). Implementations must be able to verify SHA-256 multihashes at minimum.

Signature Scheme

Signing is optional. When present, the signature covers a canonical string constructed as:

ZNAVSRFG-<UUID>-<SHA256>

Where:

ZNAVSRFG is the magic bytes string (literal ASCII)
<UUID> is the hex-encoded UUID from the outer message
<SHA256> is the hex-encoded SHA-256 hash from the outer message (covering compressed data)

Components are separated by hyphens. The signature is produced by GPG over this canonical string and stored in the signature field of the outer message.

Deterministic Serialization

By default, manifests are generated deterministically:

File entries are sorted by path in lexicographic byte order
createdAt is omitted unless explicitly requested
atime is never included (field removed from schema)

This ensures that two independent runs over the same directory tree produce byte-identical .mf files (assuming file contents and metadata have not changed).

MIME Type

The recommended MIME type for .mf files is application/octet-stream. The .mf file extension is the canonical identifier.

Reference

Proto definition: mfer/mf.proto
Reference implementation: git.eeqj.de/sneak/mfer

6.2 KiB Raw Blame History