clawbot
ed95e66f7d
Adopt the standard tooling: a Makefile of thin shims over a full scripts-to-rule-them-all `script/` set, the canonical `.golangci.yml` vendored byte-identical from `sneak/prompts`, docker-only linting via `Dockerfile.lint`, a `Dockerfile` and Gitea workflow that gate every push, and prettier/editorconfig/dockerignore config. `make build` pointed at a `cmd/dcfinfo` that is not in the tree and could never have succeeded; this repo is a library, so `build` is now the compile check over every package. Clear the 57 findings the canonical linter config reports on `pkg/dcf`. Two were real: `findDCFMountPoints` checked a never-assigned `erro` instead of the error from `findAllMountPoints`, discarding it, and `privatePath` was computed twice so the first computation was dead. Mountpoint selection otherwise behaves exactly as before; the defects that survive are filed as issue #6, not fixed here. Rename `DCFStore` to `Store` and `DCFObject` to `Object` (with its `DCFStoreRoot` field to `StoreRoot`), which revive's stutter rule requires and which the `fs.FS` rework in issue #4 will build on. Replace the placeholder test with tests over the exported surface. The filesystem walk stays uncovered: it is reachable only through `GetDCFStores`, which needs real mounted media. README keeps its content, reorganised into the required sections and gaining Entrypoints. (closes #1)
159 lines
6.1 KiB
Markdown
159 lines
6.1 KiB
Markdown
# dcf
|
|
|
|
`dcf` is a WTFPL-licensed Go library by [@sneak](https://sneak.berlin) that
|
|
reads the so-called "Design rule for Camera File system", or DCF, aka JEITA
|
|
(Japan Electronics and Information Technology Industries Association)
|
|
specification number CP-3461.
|
|
|
|
[wikipedia.org/wiki/Design_rule_for_Camera_File_system](https://en.wikipedia.org/wiki/Design_rule_for_Camera_File_system)
|
|
|
|
The DCF specification is why your digital camera puts images and videos in
|
|
`DCIM` and `PRIVATE/M4ROOT` directories on the memory card.
|
|
|
|
## Status
|
|
|
|
incomplete, under development, does not work yet
|
|
|
|
## Getting started
|
|
|
|
```sh
|
|
go get git.eeqj.de/sneak/dcf
|
|
```
|
|
|
|
```go
|
|
package main
|
|
|
|
import (
|
|
"fmt"
|
|
|
|
"git.eeqj.de/sneak/dcf/pkg/dcf"
|
|
)
|
|
|
|
func main() {
|
|
stores, err := dcf.GetDCFStores(0)
|
|
if err != nil {
|
|
panic(err)
|
|
}
|
|
|
|
for _, store := range *stores {
|
|
fmt.Println(store.String())
|
|
}
|
|
}
|
|
```
|
|
|
|
To work on the library itself:
|
|
|
|
```sh
|
|
git clone git@git.eeqj.de:sneak/dcf.git
|
|
cd dcf
|
|
script/setup # dependencies plus the git pre-commit hook
|
|
make check # test, lint, fmt-check
|
|
```
|
|
|
|
## Entrypoints
|
|
|
|
This repo adheres to the
|
|
[Scripts to Rule Them All](https://github.com/github/scripts-to-rule-them-all)
|
|
standard. Each `script/` entrypoint has a thin `make` shim; the scripts are
|
|
where the project's knowledge about flags, timeouts and docker-only linting
|
|
lives, so use them rather than invoking `go` or `golangci-lint` directly.
|
|
|
|
- `script/bootstrap` (`make bootstrap`) — install build dependencies (git, make,
|
|
go) idempotently. Linting additionally needs docker; markdown formatting needs
|
|
docker or a local `prettier`.
|
|
- `script/setup` (`make setup`) — `bootstrap` plus the git pre-commit hook.
|
|
- `script/test` (`make test`) — `go test` with the race detector and coverage,
|
|
`-count=1`, 90s timeout; quiet on success, verbose rerun on failure.
|
|
- `script/lint` (`make lint`) — `golangci-lint` via docker only, against the
|
|
digest-pinned image in `Dockerfile.lint`, then `script/assert-step-ran` and
|
|
`script/assert-context-complete` over the build log. Nothing is installed
|
|
locally and nothing runs on the host.
|
|
- `script/fmt` (`make fmt`) — format Go with `gofmt` and everything else with
|
|
`prettier` (writes).
|
|
- `script/fmt-check` (`make fmt-check`) — the same scope, read-only.
|
|
- `script/check` (`make check`) — `test` + `lint` + `fmt-check`. What the
|
|
pre-commit hook runs. Never modifies files.
|
|
- `script/docker` (`make docker`) — build the image, tagged with
|
|
`script/projectname`.
|
|
- `script/cibuild` (`make cibuild`) — the CI gate:
|
|
`docker build --progress=plain --no-cache-filter=lint --no-cache-filter=builder .`
|
|
followed by assertions that the lint step and the test step really ran, and
|
|
that both stages really received the whole repository. The Gitea workflow runs
|
|
this on every push.
|
|
- `script/precommit` — the hook body: `go mod tidy` must not change
|
|
`go.mod`/`go.sum`, then `check`.
|
|
- `script/install-precommit` (`make hooks`) — installs the hook.
|
|
- `script/projectname` — prints the project name; other scripts call it so they
|
|
can stay identical across repos.
|
|
- `script/prettier`, `script/assert-step-ran`, `script/assert-context-complete`
|
|
and `script/repo-source-manifest` are helpers, not entrypoints.
|
|
|
|
`make build`, `make deps` and `make clean` are the ordinary conveniences.
|
|
`make build` compiles every package; this repo is a library and ships no binary,
|
|
so there is no `cmd/` and nothing to link.
|
|
|
|
### Why the lint assertions exist
|
|
|
|
A green `docker build` is not evidence that the checks ran. On an unchanged tree
|
|
every layer comes from cache and the build exits 0 having executed nothing;
|
|
BuildKit silently ignores a `--no-cache-filter` naming a stage that no longer
|
|
exists; and a `.dockerignore` entry can remove a package from the build context,
|
|
after which the linter genuinely runs, genuinely examines what it was handed,
|
|
and genuinely reports `0 issues.` over a repository with a violation in it.
|
|
`script/assert-step-ran` and `script/assert-context-complete` close both holes;
|
|
read the comments in them before changing either.
|
|
|
|
## Rationale
|
|
|
|
I wanted to copy images off my memory cards for processing and like
|
|
overengineering and reusable code.
|
|
|
|
## Design
|
|
|
|
Everything lives in `pkg/dcf`, a single library package with no command
|
|
entrypoints.
|
|
|
|
- `dcf.go` — the public surface. `Store` is one DCF store (one mounted
|
|
filesystem); `Object` is one file in it, embedded in `Image` and `Video`.
|
|
`GetDCFStores` finds the stores on the system and walks each one, classifying
|
|
files by extension.
|
|
- `helpers.go` — mountpoint discovery. `findAllMountPoints` enumerates the
|
|
system's physical filesystems via `gopsutil`; `findDCFMountPoints` keeps only
|
|
those whose root holds a marker directory.
|
|
|
|
Detection is by directory name at the filesystem root, so no filesystem is ever
|
|
mounted and nothing outside a filesystem root is searched.
|
|
|
|
## Bugs
|
|
|
|
does not yet handle "dcf file groups" where multiple objects share the same
|
|
index number. if you need this, send me a link to a zip or tar of a memory card
|
|
that uses it and i'll see what i can do.
|
|
|
|
## TODO
|
|
|
|
The work in flight is on the tracker; this is the reading order for picking it
|
|
up.
|
|
|
|
- [#2](https://git.eeqj.de/sneak/dcf/issues/2) — parse the CP-3461 structure
|
|
rather than grepping every filesystem root for media extensions.
|
|
- [#3](https://git.eeqj.de/sneak/dcf/issues/3) — DCF file groups: objects
|
|
sharing an index number are one logical object.
|
|
- [#4](https://git.eeqj.de/sneak/dcf/issues/4) — API cleanup: `fs.FS`-based
|
|
store access, no pointer-to-slice returns, contexts on walks. This is also
|
|
what makes the filesystem walk testable; today it is reachable only through
|
|
`GetDCFStores`, which needs real mounted media, so it has no test coverage.
|
|
- [#5](https://git.eeqj.de/sneak/dcf/issues/5) — checksummed import off the
|
|
card, which is what the library is for.
|
|
- [#6](https://git.eeqj.de/sneak/dcf/issues/6) — mountpoint detection checks
|
|
`M4ROOT` at the filesystem root rather than `PRIVATE/M4ROOT`, and
|
|
`findDCFMountPoints` stops one store short of `requestedCount`.
|
|
|
|
## License
|
|
|
|
WTFPL. See [LICENSE](LICENSE).
|
|
|
|
## Author
|
|
|
|
sneak <[sneak@sneak.berlin](mailto:sneak@sneak.berlin)>
|