copy files off a card, verified by SHA256 (closes #5)

Store.Import copies every member of every file group and the images
and videos under PRIVATE/M4ROOT to dest/YYYY-MM-DD/<file name>. The day
comes from the new Object.ModTime. A file group stays in the folder of
its primary member.

Each copy is read back and compared by SHA256. A file with the same
hash already there is skipped, and anything else there is a conflict
and left alone. One failed file does not stop the others. A cancelled
context stops at once and removes the partial copy.

Object.Hash now shares its reading code with the import.

Model: opus-5-5
This commit is contained in:
clawbot committed 2026-11-03 20:06:00 +00:00
1 parent acb108a82e
commit d2728b1cea
6 files changed
+911 -28

No files matched your search

+82 -7
View File
@@ -3,7 +3,7 @@
## Description
`dcf` is a WTFPL-licensed Go library by [@sneak](https://sneak.berlin) that
reads DCF memory cards from digital cameras.
reads DCF memory cards from digital cameras and copies the files off them.
DCF is the "Design rule for Camera File system". It is also JEITA specification
number CP-3461. JEITA is the Japan Electronics and Information Technology
@@ -52,6 +52,38 @@ Call `dcf.GetStore` instead when you expect exactly one DCF store. It returns
`dcf.ErrNoStore` when none is mounted, and `dcf.ErrMultipleStores` when more
than one is.
To copy the files off a card, call `Import` with a destination directory:
```go
package main
import (
"context"
"fmt"
"log"
"sneak.berlin/go/dcf/pkg/dcf"
)
func main() {
ctx := context.Background()
store, err := dcf.GetStore(ctx)
if err != nil {
log.Fatal(err)
}
result, err := store.Import(ctx, "/path/to/photos")
if err != nil {
log.Fatal(err)
}
fmt.Println("copied:", result.Copied)
fmt.Println("skipped:", result.Skipped)
fmt.Println("conflicts:", result.Conflicts)
}
```
To work on the library itself, you need docker and git:
```sh
@@ -110,11 +142,13 @@ overengineering and reusable code.
All the code is in the `pkg/dcf` package. The repo has no commands.
- `dcf.go` holds the public API. `GetStores` finds the DCF stores and reads each
one. `GetStore` returns the one DCF store when exactly one is mounted.
- `dcf.go` holds the public API for reading a card. `GetStores` finds the DCF
stores and reads each one. `GetStore` returns the one DCF store when exactly
one is mounted.
- `helpers.go` finds the mount points that hold a DCF store.
- `parse.go` reads one DCF store through an `fs.FS` rooted at its mount point.
- `filegroup.go` sorts the files of one DCF directory into file groups.
- `import.go` holds `Store.Import`, which copies files off a card.
The main types are:
@@ -122,13 +156,14 @@ The main types are:
- `Directory` is one DCF directory. It has a directory number, a name, the
objects inside it and their file groups.
- `Object` is one file in a store. A file in a DCF directory also has a
directory number and a file number. `Object.Hash` reads the file through the
`fs.FS` of its store. An `Object` you build yourself has none, so its `Hash`
returns an error.
directory number and a file number. `Object.ModTime` is its modification time.
`Object.Hash` reads the file through the `fs.FS` of its store. An `Object` you
build yourself has none, so its `Hash` returns an error.
- `FileGroup` is one file group in a DCF directory.
- `Image` and `Video` embed `Object`. The file extension decides which one a
file becomes. A file with any other extension, such as a `THM` thumbnail, is
neither.
- `ImportResult` lists what `Store.Import` did with each file.
A mounted filesystem is a DCF store when its root holds a `DCIM` or
`PRIVATE/M4ROOT` directory. It needs only one of them. `dcf` never mounts a
@@ -191,9 +226,49 @@ The primary member is the first of these that the group holds:
`Store.Images` and `Store.Videos` list each file on its own, whatever its group.
`Store.Import` copies files off a card to a destination directory. It verifies
each copy against the SHA256 hash of its source. It only reads the card, so it
never deletes anything there.
It copies:
- every member of every file group
- the images and videos under `PRIVATE/M4ROOT`
It does not copy the files in `Store.NonConforming`.
Each file goes to `YYYY-MM-DD/<file name>` under the destination directory. The
date is the day of the file's modification time, in the computer's time zone.
Every member of a file group goes to the day of its primary member, so a group
never splits across two folders. Each copy keeps the modification time of its
source. Only the user who ran the import can read the copies.
`ImportResult` lists each destination path in one of three lists:
- `Copied` holds the copies this import wrote and verified.
- `Skipped` holds the paths that already held a file with the same SHA256 hash.
- `Conflicts` holds the paths that already held anything else. `dcf` leaves that
alone and does not copy the file.
A conflict is not an error, so a caller must check `Conflicts`. These are the
usual causes:
- Two different files have the same name and the same day. Two cards can reuse a
file number, and so can two DCF directories on one card. The first file is
copied, and the second is a conflict.
- A partial file remains because the process died during a copy. Delete it and
import again.
A copy that fails or does not match the hash of its source is removed. One file
that fails does not stop the others. The returned error joins the failures, each
naming its file. The result still lists what was done.
When the context is cancelled, the import stops at once, even inside a file, and
removes the partial copy. The returned error matches `context.Canceled`.
## TODO
- Copy files off a card, and verify each copy against its SHA256 hash.
No open items.
## License