Owner decisions required. These six questions are currently inlined in the
README's 1.0 roadmap with empty > _answer:_ slots. Moving them here so
answers land in one place.
Every one of these changes the on-disk format. After 1.0 they are frozen,
because .mf files written by 1.0 must stay readable forever. Answer them
now or accept them as-is permanently.
1. Simplify MFFileChecksum? It is a message wrapping a single bytes multiHash field. Multihash already self-describes its algorithm, so repeated bytes hashes directly on MFFilePath would be simpler and cut
per-file protobuf overhead. Is the extra message layer intentional — e.g.
reserved for per-hash metadata like verified_at?
Recommendation: keep the wrapper. The overhead is a few bytes per file
and the format's stated non-goal is small manifests. Removing an
extension point to save bytes you have explicitly said you do not care
about is the wrong trade.
2. Store Unix file mode? The format stores mtime/ctime but not
permissions. Irrelevant for pure archival, a real gap for software
distribution and filesystem restore.
Recommendation: reserve optional uint32 mode = 305 now without
populating it. Reserving a field number costs nothing; adding one after
1.0 to a format with deployed readers costs a version negotiation.
3. Remove atime? Access time is volatile, frequently disabled
(noatime), and non-deterministic — two manifests of an unchanged directory
will differ, which directly conflicts with the determinism goal.
Recommendation: remove it from the schema. It is actively harmful to the
stated goal and there is no use case in the README that needs it. If
removal is too aggressive, mark it reserved and never populate it — but do
not ship 1.0 writing a field that breaks reproducibility.
4. What are the path normalization rules?string path has no
specification: forward-slash always? relative always? .. forbidden? UTF-8
NFC vs NFD (macOS writes NFD, Linux NFC — the same filename produces
different bytes)? Maximum length? This is both a security question (#61) and
a cross-platform correctness question.
Recommendation: mandate relative, forward-slash, valid UTF-8, no ..,
no leading /, no empty segments — which is what ValidatePath already
enforces on write. The genuinely open part is Unicode normalization; NFC
is the usual choice, but it means macOS-generated manifests need
normalizing at write time, which is a real behavior change worth deciding
deliberately.
5. Add a version byte after the magic? Currently ZNAVSRFG is followed
immediately by protobuf. A version byte (ZNAVSRFG\x01) would allow future
framing changes without parsing protobuf first. MFFileOuter.Version exists
but requires successful deserialization to read.
Recommendation: add it. One byte, and it is the difference between being
able to change the framing later and not. This is the cheapest option
value in the list.
6. Add a length prefix after the magic? Protobuf is not
self-delimiting, so the current framing cannot support concatenating
manifests or appending data.
Recommendation: add a varint length prefix if there is any intent to
support concatenation or embedding; skip it if .mf is always a whole
file. This one genuinely depends on intent — worth stating the intent
either way so the spec can say so.
Definition of done
Each question is answered inline in a comment here.
Each answer becomes either a tracked implementation issue or an explicit
"no change" recorded in docs/FORMAT.md.
The format specification documents the final answers, including the ones
that resulted in no change — a spec that is silent on normalization is a
spec that will be implemented inconsistently by the planned JavaScript
library.
Please answer and reassign to clawbot.
Owner decisions required. These six questions are currently inlined in the
README's 1.0 roadmap with empty `> _answer:_` slots. Moving them here so
answers land in one place.
Every one of these changes the on-disk format. After 1.0 they are frozen,
because `.mf` files written by 1.0 must stay readable forever. Answer them
now or accept them as-is permanently.
---
**1. Simplify `MFFileChecksum`?** It is a message wrapping a single
`bytes multiHash` field. Multihash already self-describes its algorithm, so
`repeated bytes hashes` directly on `MFFilePath` would be simpler and cut
per-file protobuf overhead. Is the extra message layer intentional — e.g.
reserved for per-hash metadata like `verified_at`?
> _Recommendation:_ keep the wrapper. The overhead is a few bytes per file
> and the format's stated non-goal is small manifests. Removing an
> extension point to save bytes you have explicitly said you do not care
> about is the wrong trade.
**2. Store Unix file mode?** The format stores mtime/ctime but not
permissions. Irrelevant for pure archival, a real gap for software
distribution and filesystem restore.
> _Recommendation:_ reserve `optional uint32 mode = 305` now without
> populating it. Reserving a field number costs nothing; adding one after
> 1.0 to a format with deployed readers costs a version negotiation.
**3. Remove `atime`?** Access time is volatile, frequently disabled
(`noatime`), and non-deterministic — two manifests of an unchanged directory
will differ, which directly conflicts with the determinism goal.
> _Recommendation:_ remove it from the schema. It is actively harmful to the
> stated goal and there is no use case in the README that needs it. If
> removal is too aggressive, mark it reserved and never populate it — but do
> not ship 1.0 writing a field that breaks reproducibility.
**4. What are the path normalization rules?** `string path` has no
specification: forward-slash always? relative always? `..` forbidden? UTF-8
NFC vs NFD (macOS writes NFD, Linux NFC — the same filename produces
different bytes)? Maximum length? This is both a security question (#61) and
a cross-platform correctness question.
> _Recommendation:_ mandate relative, forward-slash, valid UTF-8, no `..`,
> no leading `/`, no empty segments — which is what `ValidatePath` already
> enforces on write. The genuinely open part is Unicode normalization; NFC
> is the usual choice, but it means macOS-generated manifests need
> normalizing at write time, which is a real behavior change worth deciding
> deliberately.
**5. Add a version byte after the magic?** Currently `ZNAVSRFG` is followed
immediately by protobuf. A version byte (`ZNAVSRFG\x01`) would allow future
framing changes without parsing protobuf first. `MFFileOuter.Version` exists
but requires successful deserialization to read.
> _Recommendation:_ add it. One byte, and it is the difference between being
> able to change the framing later and not. This is the cheapest option
> value in the list.
**6. Add a length prefix after the magic?** Protobuf is not
self-delimiting, so the current framing cannot support concatenating
manifests or appending data.
> _Recommendation:_ add a varint length prefix if there is any intent to
> support concatenation or embedding; skip it if `.mf` is always a whole
> file. This one genuinely depends on intent — worth stating the intent
> either way so the spec can say so.
---
## Definition of done
- Each question is answered inline in a comment here.
- Each answer becomes either a tracked implementation issue or an explicit
"no change" recorded in `docs/FORMAT.md`.
- The format specification documents the final answers, including the ones
that resulted in no change — a spec that is silent on normalization is a
spec that will be implemented inconsistently by the planned JavaScript
library.
Please answer and reassign to `clawbot`.
clawbot
added this to the 1.0.0 milestone 2026-08-09 03:44:03 +02:00
sneak
was assigned by clawbot2026-08-09 03:44:03 +02:00
One more question for this list, found while reviewing the tree for #99:
2b. ctime is declared but never written.MFFilePath.ctime (field 303) is in the proto and documented in FORMAT.md, but the reference implementation never populates it: mfer/scanner.go leaves it unset with a comment that platform-specific code would be needed. ctime also changes on chmod, chown, and rename, so populating it would make manifests of an unchanged tree differ, the same objection as atime.
> Recommendation: keep the field number reserved, state in the spec that writers may omit it and that the reference implementation does not write it. Implementing it per platform buys nothing the stated use cases need.
Model: fable-5-1
One more question for this list, found while reviewing the tree for https://git.eeqj.de/sneak/mfer/issues/99:
**2b. `ctime` is declared but never written.** `MFFilePath.ctime` (field 303) is in the proto and documented in `FORMAT.md`, but the reference implementation never populates it: `mfer/scanner.go` leaves it unset with a comment that platform-specific code would be needed. `ctime` also changes on `chmod`, `chown`, and rename, so populating it would make manifests of an unchanged tree differ, the same objection as `atime`.
> _Recommendation:_ keep the field number reserved, state in the spec that writers may omit it and that the reference implementation does not write it. Implementing it per platform buys nothing the stated use cases need.
Model: fable-5-1
One more question for this list, moved here from the README section Open Questions by #122, where it was on no issue:
Checksums for chunks of large files? Should the manifest carry checksums for chunks of each large file, or only for the whole file? If chunks, should the chunk size be fixed or variable? This changes the on-disk format, which has no chunk checksums today. The README's Non-Goals already say manifest size is no reason to leave per-chunk checksums out.
Model: opus-5-5
One more question for this list, moved here from the README section `Open Questions` by https://git.eeqj.de/sneak/mfer/pulls/122, where it was on no issue:
**Checksums for chunks of large files?** Should the manifest carry checksums for chunks of each large file, or only for the whole file? If chunks, should the chunk size be fixed or variable? This changes the on-disk format, which has no chunk checksums today. The README's Non-Goals already say manifest size is no reason to leave per-chunk checksums out.
Model: opus-5-5
Ruling from sneak on question 3 (chat, 2026-10-05 ~23:17 UTC): drop atime. Remove it from the schema before the 1.0 format freeze, as recommended above. The other questions here stay open with him.
Model: opus-5-5
**Ruling from sneak on question 3** (chat, 2026-10-05 ~23:17 UTC): drop `atime`. Remove it from the schema before the 1.0 format freeze, as recommended above. The other questions here stay open with him.
Model: opus-5-5
Ruling from sneak on question 2 (chat, 2026-10-05 ~23:42 UTC): yes, add the file-mode field, and use it in 1.0 rather than only reserving it. Writing permissions is part of 1.0. By default the manifest records the mode as 0000; it records each file's real Unix permissions only when whoever creates the manifest asks for them.
He also set a rule for the whole format: nothing goes in the 1.0 manifest that 1.0 does not read or write. This is a greenfield project, so there are no reserved-but-unused fields. That applies to the other questions here too.
Definition of done for question 2:
The schema has the mode field. The manifest writer fills it with 0000 by default and with each file's permission bits when the creator passes an option to include them (CLI flag and library option, named in the style of the existing ones).
The library and every command that reads manifests read the field and expose it. If what check should do with a recorded mode is a design choice not settled here, put it to sneak as one question with a recommendation.
The format docs state the field, its 0000 default and the "nothing unused" rule.
Tests cover the default, the opt-in and reading it back. Lands on next after an independent review.
Model: opus-5-5
**Ruling from sneak on question 2** (chat, 2026-10-05 ~23:42 UTC): yes, add the file-mode field, and use it in 1.0 rather than only reserving it. Writing permissions is part of 1.0. By default the manifest records the mode as `0000`; it records each file's real Unix permissions only when whoever creates the manifest asks for them.
He also set a rule for the whole format: **nothing goes in the 1.0 manifest that 1.0 does not read or write.** This is a greenfield project, so there are no reserved-but-unused fields. That applies to the other questions here too.
Definition of done for question 2:
- The schema has the mode field. The manifest writer fills it with `0000` by default and with each file's permission bits when the creator passes an option to include them (CLI flag and library option, named in the style of the existing ones).
- The library and every command that reads manifests read the field and expose it. If what `check` should do with a recorded mode is a design choice not settled here, put it to sneak as one question with a recommendation.
- The format docs state the field, its `0000` default and the "nothing unused" rule.
- Tests cover the default, the opt-in and reading it back. Lands on `next` after an independent review.
Model: opus-5-5
Two questions for sneak that follow from the rulings above. Nothing waits on them: work proceeds on the recommended readings, which you can reverse.
A. What do check and fetch do with a recorded file mode?
Recommended: 0000 means "not recorded" and is never checked or applied. A recorded mode holds only the nine rwx bits (0777); setuid, setgid and sticky are never written. check reports a mode that differs from the file on disk as a failure, like a hash mismatch. fetch sets the recorded mode on each file it writes. Keeping to 0777 stops a manifest from an untrusted server from making fetch create setuid files.
Alternative: check only warns on a mode mismatch, and fetch ignores modes.
B. mimeType (field 301) and ctime (field 303) are never written by mfer. Nothing fills mimeType, and nothing fills ctime either (export prints it if present). Under the rule that nothing goes in the 1.0 manifest that 1.0 does not read or write, each either goes or gets written.
Recommended: remove both. ctime changes on chmod and rename and is not portable, so it works against reproducible manifests the way atime did; a MIME type can be derived by whoever reads the file.
Alternative: write them: ctime from the file's status where the platform gives it, mimeType detected from content.
Model: opus-5-5
Two questions for sneak that follow from the rulings above. Nothing waits on them: work proceeds on the recommended readings, which you can reverse.
**A. What do `check` and `fetch` do with a recorded file mode?**
Recommended: `0000` means "not recorded" and is never checked or applied. A recorded mode holds only the nine `rwx` bits (`0777`); setuid, setgid and sticky are never written. `check` reports a mode that differs from the file on disk as a failure, like a hash mismatch. `fetch` sets the recorded mode on each file it writes. Keeping to `0777` stops a manifest from an untrusted server from making `fetch` create setuid files.
Alternative: `check` only warns on a mode mismatch, and `fetch` ignores modes.
**B. `mimeType` (field 301) and `ctime` (field 303) are never written by mfer.** Nothing fills `mimeType`, and nothing fills `ctime` either (`export` prints it if present). Under the rule that nothing goes in the 1.0 manifest that 1.0 does not read or write, each either goes or gets written.
Recommended: remove both. `ctime` changes on `chmod` and rename and is not portable, so it works against reproducible manifests the way `atime` did; a MIME type can be derived by whoever reads the file.
Alternative: write them: `ctime` from the file's status where the platform gives it, `mimeType` detected from content.
Model: opus-5-5
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Owner decisions required. These six questions are currently inlined in the
README's 1.0 roadmap with empty
> _answer:_slots. Moving them here soanswers land in one place.
Every one of these changes the on-disk format. After 1.0 they are frozen,
because
.mffiles written by 1.0 must stay readable forever. Answer themnow or accept them as-is permanently.
1. Simplify
MFFileChecksum? It is a message wrapping a singlebytes multiHashfield. Multihash already self-describes its algorithm, sorepeated bytes hashesdirectly onMFFilePathwould be simpler and cutper-file protobuf overhead. Is the extra message layer intentional — e.g.
reserved for per-hash metadata like
verified_at?2. Store Unix file mode? The format stores mtime/ctime but not
permissions. Irrelevant for pure archival, a real gap for software
distribution and filesystem restore.
3. Remove
atime? Access time is volatile, frequently disabled(
noatime), and non-deterministic — two manifests of an unchanged directorywill differ, which directly conflicts with the determinism goal.
4. What are the path normalization rules?
string pathhas nospecification: forward-slash always? relative always?
..forbidden? UTF-8NFC vs NFD (macOS writes NFD, Linux NFC — the same filename produces
different bytes)? Maximum length? This is both a security question (#61) and
a cross-platform correctness question.
5. Add a version byte after the magic? Currently
ZNAVSRFGis followedimmediately by protobuf. A version byte (
ZNAVSRFG\x01) would allow futureframing changes without parsing protobuf first.
MFFileOuter.Versionexistsbut requires successful deserialization to read.
6. Add a length prefix after the magic? Protobuf is not
self-delimiting, so the current framing cannot support concatenating
manifests or appending data.
Definition of done
"no change" recorded in
docs/FORMAT.md.that resulted in no change — a spec that is silent on normalization is a
spec that will be implemented inconsistently by the planned JavaScript
library.
Please answer and reassign to
clawbot.One more question for this list, found while reviewing the tree for #99:
2b.
ctimeis declared but never written.MFFilePath.ctime(field 303) is in the proto and documented inFORMAT.md, but the reference implementation never populates it:mfer/scanner.goleaves it unset with a comment that platform-specific code would be needed.ctimealso changes onchmod,chown, and rename, so populating it would make manifests of an unchanged tree differ, the same objection asatime.> Recommendation: keep the field number reserved, state in the spec that writers may omit it and that the reference implementation does not write it. Implementing it per platform buys nothing the stated use cases need.
Model: fable-5-1
One more question for this list, moved here from the README section
Open Questionsby #122, where it was on no issue:Checksums for chunks of large files? Should the manifest carry checksums for chunks of each large file, or only for the whole file? If chunks, should the chunk size be fixed or variable? This changes the on-disk format, which has no chunk checksums today. The README's Non-Goals already say manifest size is no reason to leave per-chunk checksums out.
Model: opus-5-5
Ruling from sneak on question 3 (chat, 2026-10-05 ~23:17 UTC): drop
atime. Remove it from the schema before the 1.0 format freeze, as recommended above. The other questions here stay open with him.Model: opus-5-5
Ruling from sneak on question 2 (chat, 2026-10-05 ~23:42 UTC): yes, add the file-mode field, and use it in 1.0 rather than only reserving it. Writing permissions is part of 1.0. By default the manifest records the mode as
0000; it records each file's real Unix permissions only when whoever creates the manifest asks for them.He also set a rule for the whole format: nothing goes in the 1.0 manifest that 1.0 does not read or write. This is a greenfield project, so there are no reserved-but-unused fields. That applies to the other questions here too.
Definition of done for question 2:
0000by default and with each file's permission bits when the creator passes an option to include them (CLI flag and library option, named in the style of the existing ones).checkshould do with a recorded mode is a design choice not settled here, put it to sneak as one question with a recommendation.0000default and the "nothing unused" rule.nextafter an independent review.Model: opus-5-5
Two questions for sneak that follow from the rulings above. Nothing waits on them: work proceeds on the recommended readings, which you can reverse.
A. What do
checkandfetchdo with a recorded file mode?Recommended:
0000means "not recorded" and is never checked or applied. A recorded mode holds only the ninerwxbits (0777); setuid, setgid and sticky are never written.checkreports a mode that differs from the file on disk as a failure, like a hash mismatch.fetchsets the recorded mode on each file it writes. Keeping to0777stops a manifest from an untrusted server from makingfetchcreate setuid files.Alternative:
checkonly warns on a mode mismatch, andfetchignores modes.B.
mimeType(field 301) andctime(field 303) are never written by mfer. Nothing fillsmimeType, and nothing fillsctimeeither (exportprints it if present). Under the rule that nothing goes in the 1.0 manifest that 1.0 does not read or write, each either goes or gets written.Recommended: remove both.
ctimechanges onchmodand rename and is not portable, so it works against reproducible manifests the wayatimedid; a MIME type can be derived by whoever reads the file.Alternative: write them:
ctimefrom the file's status where the platform gives it,mimeTypedetected from content.Model: opus-5-5