dcf

dcf is a WTFPL-licensed Go library by @sneak that reads the so-called "Design rule for Camera File system", or DCF, aka JEITA (Japan Electronics and Information Technology Industries Association) specification number CP-3461.

wikipedia.org/wiki/Design_rule_for_Camera_File_system

The DCF specification is why your digital camera puts images and videos in DCIM and PRIVATE/M4ROOT directories on the memory card.

Status

incomplete, under development, does not work yet

Getting started

go get git.eeqj.de/sneak/dcf

package main

import (
    "fmt"

    "git.eeqj.de/sneak/dcf/pkg/dcf"
)

func main() {
    stores, err := dcf.GetDCFStores(0)
    if err != nil {
        panic(err)
    }

    for _, store := range *stores {
        fmt.Println(store.String())
    }
}

To work on the library itself:

git clone git@git.eeqj.de:sneak/dcf.git
cd dcf
script/setup   # dependencies plus the git pre-commit hook
make check     # test, lint, fmt-check

Entrypoints

This repo adheres to the Scripts to Rule Them All standard. Each script/ entrypoint has a thin make shim; the scripts are where the project's knowledge about flags, timeouts and docker-only linting lives, so use them rather than invoking go or golangci-lint directly.

• script/bootstrap (make bootstrap) — install build dependencies (git, make, go) idempotently. Linting additionally needs docker; markdown formatting needs docker or a local prettier.
• script/setup (make setup) — bootstrap plus the git pre-commit hook.
• script/test (make test) — go test with the race detector and coverage, -count=1, 90s timeout; quiet on success, verbose rerun on failure.
• script/lint (make lint) — golangci-lint via docker only, against the digest-pinned image in Dockerfile.lint, then script/assert-step-ran and script/assert-context-complete over the build log. Nothing is installed locally and nothing runs on the host.
• script/fmt (make fmt) — format Go with gofmt and everything else with prettier (writes).
• script/fmt-check (make fmt-check) — the same scope, read-only.
• script/check (make check) — test + lint + fmt-check. What the pre-commit hook runs. Never modifies files.
• script/docker (make docker) — build the image, tagged with script/projectname.
• script/cibuild (make cibuild) — the CI gate: docker build --progress=plain --no-cache-filter=lint --no-cache-filter=builder . followed by assertions that the lint step and the test step really ran, and that both stages really received the whole repository. The Gitea workflow runs this on every push.
• script/precommit — the hook body: go mod tidy must not change go.mod/go.sum, then check.
• script/install-precommit (make hooks) — installs the hook.
• script/projectname — prints the project name; other scripts call it so they can stay identical across repos.
• script/prettier, script/assert-step-ran, script/assert-context-complete and script/repo-source-manifest are helpers, not entrypoints.

make build, make deps and make clean are the ordinary conveniences. make build compiles every package; this repo is a library and ships no binary, so there is no cmd/ and nothing to link.

Why the lint assertions exist

A green docker build is not evidence that the checks ran. On an unchanged tree every layer comes from cache and the build exits 0 having executed nothing; BuildKit silently ignores a --no-cache-filter naming a stage that no longer exists; and a .dockerignore entry can remove a package from the build context, after which the linter genuinely runs, genuinely examines what it was handed, and genuinely reports 0 issues. over a repository with a violation in it. script/assert-step-ran and script/assert-context-complete close both holes; read the comments in them before changing either.

Rationale

I wanted to copy images off my memory cards for processing and like overengineering and reusable code.

Design

Everything lives in pkg/dcf, a single library package with no command entrypoints.

• dcf.go — the public surface. Store is one DCF store (one mounted filesystem); Object is one file in it, embedded in Image and Video. GetDCFStores finds the stores on the system and walks each one, classifying files by extension.
• helpers.go — mountpoint discovery. findAllMountPoints enumerates the system's physical filesystems via gopsutil; findDCFMountPoints keeps only those whose root holds a marker directory.

Detection is by directory name at the filesystem root, so no filesystem is ever mounted and nothing outside a filesystem root is searched.

Bugs

does not yet handle "dcf file groups" where multiple objects share the same index number. if you need this, send me a link to a zip or tar of a memory card that uses it and i'll see what i can do.

TODO

The work in flight is on the tracker; this is the reading order for picking it up.

• #2 — parse the CP-3461 structure rather than grepping every filesystem root for media extensions.
• #3 — DCF file groups: objects sharing an index number are one logical object.
• #4 — API cleanup: fs.FS-based store access, no pointer-to-slice returns, contexts on walks. This is also what makes the filesystem walk testable; today it is reachable only through GetDCFStores, which needs real mounted media, so it has no test coverage.
• #5 — checksummed import off the card, which is what the library is for.
• #6 — mountpoint detection checks M4ROOT at the filesystem root rather than PRIVATE/M4ROOT, and findDCFMountPoints stops one store short of requestedCount.

License

WTFPL. See LICENSE.

Author

sneak <sneak@sneak.berlin>