API cleanup: fs.FS-based store access, no *[]DCFStore, contexts on walks #4

Open
opened 2026-08-30 13:14:49 +02:00 by clawbot · 4 comments
Collaborator

Current API returns *[]DCFStore (pointer to slice), hardcodes os filesystem access, and offers no cancellation. Rework: store construction takes an fs.FS (or a path convenience wrapper), walk/scan take context.Context, plain slice returns, errors wrapped with %w. Exact surface follows sneak's pick from the API proposals (tracked in chat 2026-08-30); this issue holds the mechanical cleanup that applies regardless.

Done: no pointer-to-slice in the public surface; store parsing testable against fstest.MapFS; walks cancelable.

Current API returns `*[]DCFStore` (pointer to slice), hardcodes `os` filesystem access, and offers no cancellation. Rework: store construction takes an `fs.FS` (or a path convenience wrapper), walk/scan take `context.Context`, plain slice returns, errors wrapped with `%w`. Exact surface follows sneak's pick from the API proposals (tracked in chat 2026-08-30); this issue holds the mechanical cleanup that applies regardless. Done: no pointer-to-slice in the public surface; store parsing testable against `fstest.MapFS`; walks cancelable.
Author
Collaborator

Note moved here from the project-management repo's TODO.md (sneak, 25 September: each repo's notes live on that repo). On 30 August sneak was offered options for this API's shape in chat, and his pick was to decide this issue and #5. Neither the options nor an answer were recorded here, and the chat is gone. When dcf resumes, its repo-manager works the options out again from the code and asks him on this issue before either is implemented.

Model: opus-5-5

Note moved here from the project-management repo's `TODO.md` (sneak, 25 September: each repo's notes live on that repo). On 30 August sneak was offered options for this API's shape in chat, and his pick was to decide this issue and https://git.eeqj.de/sneak/dcf/issues/5. Neither the options nor an answer were recorded here, and the chat is gone. When dcf resumes, its repo-manager works the options out again from the code and asks him on this issue before either is implemented. Model: opus-5-5
Author
Collaborator

Question for sneak, covering this issue and #5. Your 30 August pick was lost with the chat, so here are the options again, worked out from the code. Settled either way (this issue's body): cards are read through fs.FS, every walk takes a context.Context, plain slices, wrapped errors.

1. How a program gets a card.

  • A (recommended): find, then open. roots, err := dcf.FindCards(ctx) lists mounted volumes with DCIM or PRIVATE/M4ROOT; store, err := dcf.Open(ctx, os.DirFS(root)) reads one. A copied card folder or a test tree opens the same way as a mounted card. The count argument goes away.
  • B: one call, as now. stores, err := dcf.GetStores(ctx, n) finds and reads every mounted card. Fewer calls, but it only ever sees mounted volumes.

2. Where the import puts files. The call would be res, err := store.Import(ctx, dest, dcf.ImportOptions{SkipExisting: true}), reporting what was copied, skipped and in conflict, verifying each copy by SHA-256 and never deleting from the card.

  • A (recommended): the card's own layout under dest, e.g. dest/DCIM/100MSDCF/DSC00001.JPG. A different file already at that path is reported as a conflict and left alone, never overwritten. Two cards that reuse the same numbers conflict in one dest, so give each card its own.
  • B: one folder per day from the file's time, e.g. dest/2026-10-03/DSC00001.JPG, same conflict rule.
  • C: the program decides by passing a function from file group to destination path, with A as the default.

Reply with a letter for each, or "recommended". #1, #2, #3 and #6 go ahead meanwhile; none of them depends on this.

Model: opus-5-5

Question for sneak, covering this issue and https://git.eeqj.de/sneak/dcf/issues/5. Your 30 August pick was lost with the chat, so here are the options again, worked out from the code. Settled either way (this issue's body): cards are read through `fs.FS`, every walk takes a `context.Context`, plain slices, wrapped errors. **1. How a program gets a card.** - **A (recommended): find, then open.** `roots, err := dcf.FindCards(ctx)` lists mounted volumes with `DCIM` or `PRIVATE/M4ROOT`; `store, err := dcf.Open(ctx, os.DirFS(root))` reads one. A copied card folder or a test tree opens the same way as a mounted card. The count argument goes away. - **B: one call, as now.** `stores, err := dcf.GetStores(ctx, n)` finds and reads every mounted card. Fewer calls, but it only ever sees mounted volumes. **2. Where the import puts files.** The call would be `res, err := store.Import(ctx, dest, dcf.ImportOptions{SkipExisting: true})`, reporting what was copied, skipped and in conflict, verifying each copy by SHA-256 and never deleting from the card. - **A (recommended): the card's own layout under `dest`**, e.g. `dest/DCIM/100MSDCF/DSC00001.JPG`. A different file already at that path is reported as a conflict and left alone, never overwritten. Two cards that reuse the same numbers conflict in one `dest`, so give each card its own. - **B: one folder per day** from the file's time, e.g. `dest/2026-10-03/DSC00001.JPG`, same conflict rule. - **C: the program decides** by passing a function from file group to destination path, with A as the default. Reply with a letter for each, or "recommended". https://git.eeqj.de/sneak/dcf/issues/1, https://git.eeqj.de/sneak/dcf/issues/2, https://git.eeqj.de/sneak/dcf/issues/3 and https://git.eeqj.de/sneak/dcf/issues/6 go ahead meanwhile; none of them depends on this. Model: opus-5-5
sneak was assigned by clawbot 2026-10-03 14:29:51 +02:00
Owner

we’re gonna be renaming files when they come off the card eventually to YYYY-MM-DD.origfilename.sha256.ext so filename conflicts won’t be an issue. use the date-based subdir.

getting a card should be as simple as possible. use the one call form. add a helper that returns err if cards are >1 and one card if there is ==1 card.

we’re gonna be renaming files when they come off the card eventually to YYYY-MM-DD.origfilename.sha256.ext so filename conflicts won’t be an issue. use the date-based subdir. getting a card should be as simple as possible. use the one call form. add a helper that returns err if cards are >1 and one card if there is ==1 card.
Author
Collaborator

Ruling (sneak, 2026-10-17, #4 (comment)), covering this issue and #5:

  • Getting a card: 1B, the one-call form, as simple as possible: one call finds and reads every mounted card. Add a helper that returns the one card when exactly one is mounted, and an error when there is more than one (or none).
  • Where the import puts files: 2B, one folder per day from the file's time, e.g. dest/2026-10-03/DSC00001.JPG, with the conflict rule from the question above.
  • Later, not part of this work: files will eventually be renamed on import to YYYY-MM-DD.origfilename.sha256.ext, so name conflicts stop mattering. Nothing is built for that until sneak asks for it.

Next: dcf's repo-manager builds this when dcf reaches a manager slot on the priority list. Pushing needs write access to sneak/dcf (sneak/project-management#10); the work can be built and reviewed before that.

Model: opus-5-5

**Ruling (sneak, 2026-10-17, https://git.eeqj.de/sneak/dcf/issues/4#issuecomment-145366), covering this issue and https://git.eeqj.de/sneak/dcf/issues/5:** - **Getting a card: 1B, the one-call form**, as simple as possible: one call finds and reads every mounted card. Add a helper that returns the one card when exactly one is mounted, and an error when there is more than one (or none). - **Where the import puts files: 2B, one folder per day** from the file's time, e.g. `dest/2026-10-03/DSC00001.JPG`, with the conflict rule from the question above. - **Later, not part of this work:** files will eventually be renamed on import to `YYYY-MM-DD.origfilename.sha256.ext`, so name conflicts stop mattering. Nothing is built for that until sneak asks for it. **Next:** dcf's repo-manager builds this when dcf reaches a manager slot on the priority list. Pushing needs write access to `sneak/dcf` (https://git.eeqj.de/sneak/project-management/issues/10); the work can be built and reviewed before that. Model: opus-5-5
sneak was unassigned by clawbot 2026-10-17 23:54:14 +02:00
clawbot self-assigned this 2026-10-17 23:54:14 +02:00
Sign in to join this conversation.
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sneak/dcf#4