dcf/README.md

# dcf

## Description

`dcf` is a WTFPL-licensed Go library by [@sneak](https://sneak.berlin) that
reads DCF memory cards from digital cameras.

DCF is the "Design rule for Camera File system". It is also JEITA specification
number CP-3461. JEITA is the Japan Electronics and Information Technology
Industries Association. Wikipedia describes it at
[Design rule for Camera File system](https://en.wikipedia.org/wiki/Design_rule_for_Camera_File_system).

The DCF specification is why your digital camera puts images and videos in
`DCIM` and `PRIVATE/M4ROOT` directories on the memory card.

`dcf` is incomplete and under development. It does not work yet.

## Getting Started

Add the library to your Go module:

```sh
go get sneak.berlin/go/dcf
```

Then list the DCF stores on the mounted filesystems:

```go
package main

import (
	"fmt"
	"log"

	"sneak.berlin/go/dcf/pkg/dcf"
)

func main() {
	stores, err := dcf.GetDCFStores(0)
	if err != nil {
		log.Fatal(err)
	}

	for _, store := range *stores {
		fmt.Println(store.String())
	}
}
```

To work on the library itself, you need docker and git:

```sh
git clone git@git.eeqj.de:sneak/dcf.git
cd dcf
script/setup
make check
```

## Entrypoints

This repo adheres to the
[Scripts to Rule Them All](https://github.com/github/scripts-to-rule-them-all)
standard. Each entrypoint is a POSIX sh script in `script/`. Use these instead
of running the underlying tools directly.

- `script/bootstrap` (`make bootstrap`) installs the tools the host needs. It
  does not install docker or the linter. It installs:
    - `make` and git
    - Go into `~/.local/go`, at the version in the `Dockerfile`, unless the `go`
      on the `PATH` is already that version
    - the Go modules
    - node and yarn
    - `prettier`
- `script/setup` (`make setup`) runs `script/bootstrap` and then installs the
  git pre-commit hook.
- `script/test` (`make test`) runs the tests by building the `test` stage of the
  `Dockerfile`.
- `script/lint` (`make lint`) checks the Go code by building the `lint` stage of
  the `Dockerfile`.
- `script/fmt` (`make fmt`) rewrites the Go code with `go fmt` and the Markdown
  files with `prettier`.
- `script/fmt-check` (`make fmt-check`) checks the same formatting and changes
  nothing.
- `script/check` (`make check`) runs the tests, then the linter, then the
  formatting check.
- `script/build` (`make build`) compiles every package. It needs the Go that
  `script/bootstrap` installs.
- `script/docker` (`make docker`) builds the development image, tagged `dcf`.
- `script/cibuild` runs the whole CI build. The Gitea workflow runs it on every
  push.
- `script/precommit` is what the git pre-commit hook runs. It runs
  `script/check`.
- `script/install-precommit` (`make hooks`) installs the git pre-commit hook.
- `script/projectname` prints the project name for the other scripts.

The development image cannot be built unless the `lint` and `test` stages pass.
Run `docker run --rm -it dcf` for a shell inside it, with Go and `make`.

## Rationale

I wanted to copy images off my memory cards for processing. I also like
overengineering and reusable code.

## Design

All the code is in the `pkg/dcf` package. The repo has no commands.

- `dcf.go` holds the public API. `GetDCFStores` finds the DCF stores and reads
  each one.
- `helpers.go` finds the mount points that hold a DCF store.
- `parse.go` reads one DCF store through an `fs.FS` rooted at its mount point.
- `filegroup.go` sorts the files of one DCF directory into file groups.

The main types are:

- `Store` is one DCF store on one mounted filesystem.
- `Directory` is one DCF directory. It has a directory number, a name, the
  objects inside it and their file groups.
- `Object` is one file in a store. A file in a DCF directory also has a
  directory number and a file number.
- `FileGroup` is one file group in a DCF directory.
- `Image` and `Video` embed `Object`. The file extension decides which one a
  file becomes. A file with any other extension, such as a `THM` thumbnail, is
  neither.

A mounted filesystem is a DCF store when its root holds a `DCIM` or
`PRIVATE/M4ROOT` directory. It needs only one of them. `dcf` never mounts a
filesystem. Media anywhere else on the filesystem is not part of the store.

`dcf` never follows a symlink.

`dcf` applies the naming rules of the DCF specification, CIPA DC-009-2010, under
`DCIM`:

- A DCF directory sits directly under `DCIM`. Its name is a directory number
  from 100 to 999, then five characters, such as `100MSDCF`.
- A DCF file sits directly in a DCF directory. Its name is four characters, then
  a file number from 0001 to 9999, then a dot and a three-character extension,
  such as `DSC00001.ARW`.
- The characters allowed are `A` to `Z`, `0` to `9` and `_`. Lower-case letters
  count as upper-case.
- Directories that share a directory number are not DCF directories.
- `JPG` files that share a file number in one directory are not DCF files.

`Store.NonConforming` lists the path of everything else under `DCIM`. It
includes everything inside a directory that breaks the rules. It also includes
every symlink, whatever its name. These entries have no DCF numbers, and they
are not counted as images or videos.

The naming rules do not apply under `PRIVATE/M4ROOT`. The images and videos
there are in `Store.Images` and `Store.Videos`, and their DCF numbers are 0.
`dcf` skips any symlink there.

A file group is made of the DCF files in one DCF directory that share a file
number. The specification calls it a DCF object. A RAW file and its JPEG form a
file group, and so do a video and its `THM` thumbnail.

- The first four characters and the extension can differ within a group.
- A DCF file that shares its number with no other file is a group of one.
- Files in different DCF directories are never in the same group.
- Only DCF files are grouped. Files in `Store.NonConforming` and under
  `PRIVATE/M4ROOT` are in no group.

`Directory.FileGroups` lists the groups in file number order. Each `FileGroup`
has:

- `DirectoryNumber` and `FileNumber`
- `Members`, which is every file in the group
- `Primary`, which is the member that stands for the whole group
- `MembersCount`, which returns the number of members
- `TotalSize`, which returns the total size of the members in bytes

The primary member is the first of these that the group holds:

1. a `JPG` file
2. any other file in `Store.Images`
3. a file in `Store.Videos`
4. any other file except a `THM`
5. a `THM`

`Store.Images` and `Store.Videos` list each file on its own, whatever its group.

## TODO

- Take a `context.Context` to cancel the walk of a store.
- Return a plain slice from `GetDCFStores`, not a pointer to one.
- Copy files off a card, and verify each copy against its SHA256 hash.

## License

WTFPL. See [LICENSE](LICENSE).

## Author

[@sneak](https://sneak.berlin) <[sneak@sneak.berlin](mailto:sneak@sneak.berlin)>