API cleanup: fs.FS-based store access, no *[]DCFStore, contexts on walks #4

Closed
opened 2026-08-30 13:14:49 +02:00 by clawbot · 6 comments
Collaborator

Current API returns *[]DCFStore (pointer to slice), hardcodes os filesystem access, and offers no cancellation. Rework: store construction takes an fs.FS (or a path convenience wrapper), walk/scan take context.Context, plain slice returns, errors wrapped with %w. Exact surface follows sneak's pick from the API proposals (tracked in chat 2026-08-30); this issue holds the mechanical cleanup that applies regardless.

Done: no pointer-to-slice in the public surface; store parsing testable against fstest.MapFS; walks cancelable.

Current API returns `*[]DCFStore` (pointer to slice), hardcodes `os` filesystem access, and offers no cancellation. Rework: store construction takes an `fs.FS` (or a path convenience wrapper), walk/scan take `context.Context`, plain slice returns, errors wrapped with `%w`. Exact surface follows sneak's pick from the API proposals (tracked in chat 2026-08-30); this issue holds the mechanical cleanup that applies regardless. Done: no pointer-to-slice in the public surface; store parsing testable against `fstest.MapFS`; walks cancelable.
Author
Collaborator

Note moved here from the project-management repo's TODO.md (sneak, 25 September: each repo's notes live on that repo). On 30 August sneak was offered options for this API's shape in chat, and his pick was to decide this issue and #5. Neither the options nor an answer were recorded here, and the chat is gone. When dcf resumes, its repo-manager works the options out again from the code and asks him on this issue before either is implemented.

Model: opus-5-5

Note moved here from the project-management repo's `TODO.md` (sneak, 25 September: each repo's notes live on that repo). On 30 August sneak was offered options for this API's shape in chat, and his pick was to decide this issue and https://git.eeqj.de/sneak/dcf/issues/5. Neither the options nor an answer were recorded here, and the chat is gone. When dcf resumes, its repo-manager works the options out again from the code and asks him on this issue before either is implemented. Model: opus-5-5
Author
Collaborator

Question for sneak, covering this issue and #5. Your 30 August pick was lost with the chat, so here are the options again, worked out from the code. Settled either way (this issue's body): cards are read through fs.FS, every walk takes a context.Context, plain slices, wrapped errors.

1. How a program gets a card.

  • A (recommended): find, then open. roots, err := dcf.FindCards(ctx) lists mounted volumes with DCIM or PRIVATE/M4ROOT; store, err := dcf.Open(ctx, os.DirFS(root)) reads one. A copied card folder or a test tree opens the same way as a mounted card. The count argument goes away.
  • B: one call, as now. stores, err := dcf.GetStores(ctx, n) finds and reads every mounted card. Fewer calls, but it only ever sees mounted volumes.

2. Where the import puts files. The call would be res, err := store.Import(ctx, dest, dcf.ImportOptions{SkipExisting: true}), reporting what was copied, skipped and in conflict, verifying each copy by SHA-256 and never deleting from the card.

  • A (recommended): the card's own layout under dest, e.g. dest/DCIM/100MSDCF/DSC00001.JPG. A different file already at that path is reported as a conflict and left alone, never overwritten. Two cards that reuse the same numbers conflict in one dest, so give each card its own.
  • B: one folder per day from the file's time, e.g. dest/2026-10-03/DSC00001.JPG, same conflict rule.
  • C: the program decides by passing a function from file group to destination path, with A as the default.

Reply with a letter for each, or "recommended". #1, #2, #3 and #6 go ahead meanwhile; none of them depends on this.

Model: opus-5-5

Question for sneak, covering this issue and https://git.eeqj.de/sneak/dcf/issues/5. Your 30 August pick was lost with the chat, so here are the options again, worked out from the code. Settled either way (this issue's body): cards are read through `fs.FS`, every walk takes a `context.Context`, plain slices, wrapped errors. **1. How a program gets a card.** - **A (recommended): find, then open.** `roots, err := dcf.FindCards(ctx)` lists mounted volumes with `DCIM` or `PRIVATE/M4ROOT`; `store, err := dcf.Open(ctx, os.DirFS(root))` reads one. A copied card folder or a test tree opens the same way as a mounted card. The count argument goes away. - **B: one call, as now.** `stores, err := dcf.GetStores(ctx, n)` finds and reads every mounted card. Fewer calls, but it only ever sees mounted volumes. **2. Where the import puts files.** The call would be `res, err := store.Import(ctx, dest, dcf.ImportOptions{SkipExisting: true})`, reporting what was copied, skipped and in conflict, verifying each copy by SHA-256 and never deleting from the card. - **A (recommended): the card's own layout under `dest`**, e.g. `dest/DCIM/100MSDCF/DSC00001.JPG`. A different file already at that path is reported as a conflict and left alone, never overwritten. Two cards that reuse the same numbers conflict in one `dest`, so give each card its own. - **B: one folder per day** from the file's time, e.g. `dest/2026-10-03/DSC00001.JPG`, same conflict rule. - **C: the program decides** by passing a function from file group to destination path, with A as the default. Reply with a letter for each, or "recommended". https://git.eeqj.de/sneak/dcf/issues/1, https://git.eeqj.de/sneak/dcf/issues/2, https://git.eeqj.de/sneak/dcf/issues/3 and https://git.eeqj.de/sneak/dcf/issues/6 go ahead meanwhile; none of them depends on this. Model: opus-5-5
sneak was assigned by clawbot 2026-10-03 14:29:51 +02:00
Owner

we’re gonna be renaming files when they come off the card eventually to YYYY-MM-DD.origfilename.sha256.ext so filename conflicts won’t be an issue. use the date-based subdir.

getting a card should be as simple as possible. use the one call form. add a helper that returns err if cards are >1 and one card if there is ==1 card.

we’re gonna be renaming files when they come off the card eventually to YYYY-MM-DD.origfilename.sha256.ext so filename conflicts won’t be an issue. use the date-based subdir. getting a card should be as simple as possible. use the one call form. add a helper that returns err if cards are >1 and one card if there is ==1 card.
Author
Collaborator

Ruling (sneak, 2026-10-17, #4 (comment)), covering this issue and #5:

  • Getting a card: 1B, the one-call form, as simple as possible: one call finds and reads every mounted card. Add a helper that returns the one card when exactly one is mounted, and an error when there is more than one (or none).
  • Where the import puts files: 2B, one folder per day from the file's time, e.g. dest/2026-10-03/DSC00001.JPG, with the conflict rule from the question above.
  • Later, not part of this work: files will eventually be renamed on import to YYYY-MM-DD.origfilename.sha256.ext, so name conflicts stop mattering. Nothing is built for that until sneak asks for it.

Next: dcf's repo-manager builds this when dcf reaches a manager slot on the priority list. Pushing needs write access to sneak/dcf (sneak/project-management#10); the work can be built and reviewed before that.

Model: opus-5-5

**Ruling (sneak, 2026-10-17, https://git.eeqj.de/sneak/dcf/issues/4#issuecomment-145366), covering this issue and https://git.eeqj.de/sneak/dcf/issues/5:** - **Getting a card: 1B, the one-call form**, as simple as possible: one call finds and reads every mounted card. Add a helper that returns the one card when exactly one is mounted, and an error when there is more than one (or none). - **Where the import puts files: 2B, one folder per day** from the file's time, e.g. `dest/2026-10-03/DSC00001.JPG`, with the conflict rule from the question above. - **Later, not part of this work:** files will eventually be renamed on import to `YYYY-MM-DD.origfilename.sha256.ext`, so name conflicts stop mattering. Nothing is built for that until sneak asks for it. **Next:** dcf's repo-manager builds this when dcf reaches a manager slot on the priority list. Pushing needs write access to `sneak/dcf` (https://git.eeqj.de/sneak/project-management/issues/10); the work can be built and reviewed before that. Model: opus-5-5
sneak was unassigned by clawbot 2026-10-17 23:54:14 +02:00
clawbot self-assigned this 2026-10-17 23:54:14 +02:00
Author
Collaborator

Plan, built to the ruling at #4 (comment).

  • GetDCFStores(n) becomes GetStores(ctx) ([]*Store, error): one call finds and reads every mounted card.
  • GetStore(ctx) (*Store, error) is the helper. It returns the one card, ErrNoStore when none is mounted, and ErrMultipleStores when more than one is.
  • A card is read through fs.FS inside the library. parseStore stays unexported and takes the context, fs.FS and root, so tests use fstest.MapFS. Each Object keeps that fs.FS, and Hash reads through it instead of os.Open.
  • GetStores uses disk.PartitionsWithContext. parseStore checks the context before each directory it reads, so a cancelled read returns an error matching context.Canceled.
  • Every error from the library is wrapped with %w.
  • The README's Getting Started example, Design section and TODO list follow the new API.

Judgement calls, built unless sneak says otherwise:

  • The count argument goes away. The helper covers the one-card case, and fewest options wins.
  • No public function takes an fs.FS. sneak picked the one-call form over find-then-open, so a public open call would bring back the option he passed over.

Tests: a cancelled context at each directory level, GetStore with zero, one and two cards through the unexported loop over mount points, and Hash on a fstest.MapFS card.

Model: opus-5-5

Plan, built to the ruling at https://git.eeqj.de/sneak/dcf/issues/4#issuecomment-145367. - `GetDCFStores(n)` becomes `GetStores(ctx) ([]*Store, error)`: one call finds and reads every mounted card. - `GetStore(ctx) (*Store, error)` is the helper. It returns the one card, `ErrNoStore` when none is mounted, and `ErrMultipleStores` when more than one is. - A card is read through `fs.FS` inside the library. `parseStore` stays unexported and takes the context, `fs.FS` and root, so tests use `fstest.MapFS`. Each `Object` keeps that `fs.FS`, and `Hash` reads through it instead of `os.Open`. - `GetStores` uses `disk.PartitionsWithContext`. `parseStore` checks the context before each directory it reads, so a cancelled read returns an error matching `context.Canceled`. - Every error from the library is wrapped with `%w`. - The README's Getting Started example, Design section and TODO list follow the new API. Judgement calls, built unless sneak says otherwise: - The count argument goes away. The helper covers the one-card case, and fewest options wins. - No public function takes an `fs.FS`. sneak picked the one-call form over find-then-open, so a public open call would bring back the option he passed over. Tests: a cancelled context at each directory level, `GetStore` with zero, one and two cards through the unexported loop over mount points, and `Hash` on a `fstest.MapFS` card. Model: opus-5-5
Author
Collaborator

Built as #13. The PR body has the details and disclosures.

Model: opus-5-5

Built as https://git.eeqj.de/sneak/dcf/pulls/13. The PR body has the details and disclosures. Model: opus-5-5
Sign in to join this conversation.
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sneak/dcf#4