dcf/README.md

dcf

Description

dcf is a WTFPL-licensed Go library by @sneak that reads DCF memory cards from digital cameras.

DCF is the "Design rule for Camera File system". It is also JEITA specification number CP-3461. JEITA is the Japan Electronics and Information Technology Industries Association. Wikipedia describes it at Design rule for Camera File system.

The DCF specification is why your digital camera puts images and videos in DCIM and PRIVATE/M4ROOT directories on the memory card.

dcf is incomplete and under development. It does not work yet.

Getting Started

Add the library to your Go module:

go get sneak.berlin/go/dcf

Then list the DCF stores on the mounted filesystems:

package main

import (
	"fmt"
	"log"

	"sneak.berlin/go/dcf/pkg/dcf"
)

func main() {
	stores, err := dcf.GetDCFStores(0)
	if err != nil {
		log.Fatal(err)
	}

	for _, store := range *stores {
		fmt.Println(store.String())
	}
}

To work on the library itself, you need docker and git:

git clone git@git.eeqj.de:sneak/dcf.git
cd dcf
script/setup
make check

Entrypoints

This repo adheres to the Scripts to Rule Them All standard. Each entrypoint is a POSIX sh script in script/. Use these instead of running the underlying tools directly.

• script/bootstrap (make bootstrap) installs the tools the host needs. It does not install docker or the linter. It installs:
  • make and git
  • Go into ~/.local/go, at the version in the Dockerfile, unless the go on the PATH is already that version
  • the Go modules
  • node and yarn
  • prettier
• script/setup (make setup) runs script/bootstrap and then installs the git pre-commit hook.
• script/test (make test) runs the tests by building the test stage of the Dockerfile.
• script/lint (make lint) checks the Go code by building the lint stage of the Dockerfile.
• script/fmt (make fmt) rewrites the Go code with go fmt and the Markdown files with prettier.
• script/fmt-check (make fmt-check) checks the same formatting and changes nothing.
• script/check (make check) runs the tests, then the linter, then the formatting check.
• script/build (make build) compiles every package. It needs the Go that script/bootstrap installs.
• script/docker (make docker) builds the development image, tagged dcf.
• script/cibuild runs the whole CI build. The Gitea workflow runs it on every push.
• script/precommit is what the git pre-commit hook runs. It runs script/check.
• script/install-precommit (make hooks) installs the git pre-commit hook.
• script/projectname prints the project name for the other scripts.

The development image cannot be built unless the lint and test stages pass. Run docker run --rm -it dcf for a shell inside it, with Go and make.

Rationale

I wanted to copy images off my memory cards for processing. I also like overengineering and reusable code.

Design

All the code is in the pkg/dcf package. The repo has no commands.

• dcf.go holds the public API. GetDCFStores finds the DCF stores and reads each one.
• helpers.go finds the mount points that hold a DCF store.
• parse.go reads one DCF store through an fs.FS rooted at its mount point.

The main types are:

• Store is one DCF store on one mounted filesystem.
• Directory is one DCF directory. It has a directory number, a name and the objects inside it.
• Object is one file in a store. A file in a DCF directory also has a directory number and a file number.
• Image and Video embed Object. The file extension decides which one a file becomes. A file with any other extension, such as a THM thumbnail, is neither.

A mounted filesystem is a DCF store when its root holds a DCIM or PRIVATE/M4ROOT directory. It needs only one of them. dcf never mounts a filesystem. Media anywhere else on the filesystem is not part of the store.

dcf never follows a symlink.

dcf applies the naming rules of the DCF specification, CIPA DC-009-2010, under DCIM:

• A DCF directory sits directly under DCIM. Its name is a directory number from 100 to 999, then five characters, such as 100MSDCF.
• A DCF file sits directly in a DCF directory. Its name is four characters, then a file number from 0001 to 9999, then a dot and a three-character extension, such as DSC00001.ARW.
• The characters allowed are A to Z, 0 to 9 and _. Lower-case letters count as upper-case.
• Directories that share a directory number are not DCF directories.
• JPG files that share a file number in one directory are not DCF files.

Store.NonConforming lists the path of everything else under DCIM. It includes everything inside a directory that breaks the rules. It also includes every symlink, whatever its name. These entries have no DCF numbers, and they are not counted as images or videos.

The naming rules do not apply under PRIVATE/M4ROOT. The images and videos there are in Store.Images and Store.Videos, and their DCF numbers are 0. dcf skips any symlink there.

TODO

• Handle DCF file groups, where multiple objects share the same index number. dcf does not handle these yet. If you need this, send me a link to a zip or tar of a memory card that uses them, and I'll see what I can do.
• Take a context.Context to cancel the walk of a store.
• Return a plain slice from GetDCFStores, not a pointer to one.
• Copy files off a card, and verify each copy against its SHA256 hash.

License

WTFPL. See LICENSE.

Author

@sneak <sneak@sneak.berlin>