copy files off a card, verified by SHA256 (closes #5)
Store.Import copies every member of every file group and the images and videos under PRIVATE/M4ROOT to dest/YYYY-MM-DD/<file name>. The day comes from the new Object.ModTime. A file group stays in the folder of its primary member. Each copy is read back and compared by SHA256. A file with the same hash already there is skipped, and anything else there is a conflict and left alone. One failed file does not stop the others. A cancelled context stops at once and removes the partial copy. Object.Hash now shares its reading code with the import. Model: opus-5-5
This commit is contained in:
1 parent
acb108a82e
commit
d2728b1cea
6 files changed
+911
-28
No files matched your search
@@ -3,7 +3,7 @@
|
||||
## Description
|
||||
|
||||
`dcf` is a WTFPL-licensed Go library by [@sneak](https://sneak.berlin) that
|
||||
reads DCF memory cards from digital cameras.
|
||||
reads DCF memory cards from digital cameras and copies the files off them.
|
||||
|
||||
DCF is the "Design rule for Camera File system". It is also JEITA specification
|
||||
number CP-3461. JEITA is the Japan Electronics and Information Technology
|
||||
@@ -52,6 +52,38 @@ Call `dcf.GetStore` instead when you expect exactly one DCF store. It returns
|
||||
`dcf.ErrNoStore` when none is mounted, and `dcf.ErrMultipleStores` when more
|
||||
than one is.
|
||||
|
||||
To copy the files off a card, call `Import` with a destination directory:
|
||||
|
||||
```go
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"log"
|
||||
|
||||
"sneak.berlin/go/dcf/pkg/dcf"
|
||||
)
|
||||
|
||||
func main() {
|
||||
ctx := context.Background()
|
||||
|
||||
store, err := dcf.GetStore(ctx)
|
||||
if err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
|
||||
result, err := store.Import(ctx, "/path/to/photos")
|
||||
if err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
|
||||
fmt.Println("copied:", result.Copied)
|
||||
fmt.Println("skipped:", result.Skipped)
|
||||
fmt.Println("conflicts:", result.Conflicts)
|
||||
}
|
||||
```
|
||||
|
||||
To work on the library itself, you need docker and git:
|
||||
|
||||
```sh
|
||||
@@ -110,11 +142,13 @@ overengineering and reusable code.
|
||||
|
||||
All the code is in the `pkg/dcf` package. The repo has no commands.
|
||||
|
||||
- `dcf.go` holds the public API. `GetStores` finds the DCF stores and reads each
|
||||
one. `GetStore` returns the one DCF store when exactly one is mounted.
|
||||
- `dcf.go` holds the public API for reading a card. `GetStores` finds the DCF
|
||||
stores and reads each one. `GetStore` returns the one DCF store when exactly
|
||||
one is mounted.
|
||||
- `helpers.go` finds the mount points that hold a DCF store.
|
||||
- `parse.go` reads one DCF store through an `fs.FS` rooted at its mount point.
|
||||
- `filegroup.go` sorts the files of one DCF directory into file groups.
|
||||
- `import.go` holds `Store.Import`, which copies files off a card.
|
||||
|
||||
The main types are:
|
||||
|
||||
@@ -122,13 +156,14 @@ The main types are:
|
||||
- `Directory` is one DCF directory. It has a directory number, a name, the
|
||||
objects inside it and their file groups.
|
||||
- `Object` is one file in a store. A file in a DCF directory also has a
|
||||
directory number and a file number. `Object.Hash` reads the file through the
|
||||
`fs.FS` of its store. An `Object` you build yourself has none, so its `Hash`
|
||||
returns an error.
|
||||
directory number and a file number. `Object.ModTime` is its modification time.
|
||||
`Object.Hash` reads the file through the `fs.FS` of its store. An `Object` you
|
||||
build yourself has none, so its `Hash` returns an error.
|
||||
- `FileGroup` is one file group in a DCF directory.
|
||||
- `Image` and `Video` embed `Object`. The file extension decides which one a
|
||||
file becomes. A file with any other extension, such as a `THM` thumbnail, is
|
||||
neither.
|
||||
- `ImportResult` lists what `Store.Import` did with each file.
|
||||
|
||||
A mounted filesystem is a DCF store when its root holds a `DCIM` or
|
||||
`PRIVATE/M4ROOT` directory. It needs only one of them. `dcf` never mounts a
|
||||
@@ -191,9 +226,49 @@ The primary member is the first of these that the group holds:
|
||||
|
||||
`Store.Images` and `Store.Videos` list each file on its own, whatever its group.
|
||||
|
||||
`Store.Import` copies files off a card to a destination directory. It verifies
|
||||
each copy against the SHA256 hash of its source. It only reads the card, so it
|
||||
never deletes anything there.
|
||||
|
||||
It copies:
|
||||
|
||||
- every member of every file group
|
||||
- the images and videos under `PRIVATE/M4ROOT`
|
||||
|
||||
It does not copy the files in `Store.NonConforming`.
|
||||
|
||||
Each file goes to `YYYY-MM-DD/<file name>` under the destination directory. The
|
||||
date is the day of the file's modification time, in the computer's time zone.
|
||||
Every member of a file group goes to the day of its primary member, so a group
|
||||
never splits across two folders. Each copy keeps the modification time of its
|
||||
source. Only the user who ran the import can read the copies.
|
||||
|
||||
`ImportResult` lists each destination path in one of three lists:
|
||||
|
||||
- `Copied` holds the copies this import wrote and verified.
|
||||
- `Skipped` holds the paths that already held a file with the same SHA256 hash.
|
||||
- `Conflicts` holds the paths that already held anything else. `dcf` leaves that
|
||||
alone and does not copy the file.
|
||||
|
||||
A conflict is not an error, so a caller must check `Conflicts`. These are the
|
||||
usual causes:
|
||||
|
||||
- Two different files have the same name and the same day. Two cards can reuse a
|
||||
file number, and so can two DCF directories on one card. The first file is
|
||||
copied, and the second is a conflict.
|
||||
- A partial file remains because the process died during a copy. Delete it and
|
||||
import again.
|
||||
|
||||
A copy that fails or does not match the hash of its source is removed. One file
|
||||
that fails does not stop the others. The returned error joins the failures, each
|
||||
naming its file. The result still lists what was done.
|
||||
|
||||
When the context is cancelled, the import stops at once, even inside a file, and
|
||||
removes the partial copy. The returned error matches `context.Canceled`.
|
||||
|
||||
## TODO
|
||||
|
||||
- Copy files off a card, and verify each copy against its SHA256 hash.
|
||||
No open items.
|
||||
|
||||
## License
|
||||
|
||||
|
||||
Reference in new issue
Block a user