bring the repo up to org standards
Adopt the standard tooling: a Makefile of thin shims over a full scripts-to-rule-them-all `script/` set, the canonical `.golangci.yml` vendored byte-identical from `sneak/prompts`, docker-only linting via `Dockerfile.lint`, a `Dockerfile` and Gitea workflow that gate every push, and prettier/editorconfig/dockerignore config. `make build` pointed at a `cmd/dcfinfo` that is not in the tree and could never have succeeded; this repo is a library, so `build` is now the compile check over every package. Clear the 57 findings the canonical linter config reports on `pkg/dcf`. Two were real: `findDCFMountPoints` checked a never-assigned `erro` instead of the error from `findAllMountPoints`, discarding it, and `privatePath` was computed twice so the first computation was dead. Mountpoint selection otherwise behaves exactly as before; the defects that survive are filed as issue #6, not fixed here. Rename `DCFStore` to `Store` and `DCFObject` to `Object` (with its `DCFStoreRoot` field to `StoreRoot`), which revive's stutter rule requires and which the `fs.FS` rework in issue #4 will build on. Replace the placeholder test with tests over the exported surface. The filesystem walk stays uncovered: it is reachable only through `GetDCFStores`, which needs real mounted media. README keeps its content, reorganised into the required sections and gaining Entrypoints. (closes #1)
This commit is contained in:
151
README.md
151
README.md
@@ -1,33 +1,158 @@
|
||||
# dcf
|
||||
|
||||
Golang reader implementation of the so-called "Design rule for Camera File
|
||||
system" or DCF, aka JEITA (Japan Electronics and Information Technology
|
||||
Industries Association) specification number CP-3461.
|
||||
`dcf` is a WTFPL-licensed Go library by [@sneak](https://sneak.berlin) that
|
||||
reads the so-called "Design rule for Camera File system", or DCF, aka JEITA
|
||||
(Japan Electronics and Information Technology Industries Association)
|
||||
specification number CP-3461.
|
||||
|
||||
[wikipedia.org/wiki/Design_rule_for_Camera_File_system](https://en.wikipedia.org/wiki/Design_rule_for_Camera_File_system)
|
||||
|
||||
The DCF specification is why your digital camera puts images and videos in
|
||||
`DCIM` and `PRIVATE/M4ROOT` directories on the memory card.
|
||||
|
||||
# status
|
||||
## Status
|
||||
|
||||
incomplete, under development, does not work yet
|
||||
|
||||
# why
|
||||
## Getting started
|
||||
|
||||
```sh
|
||||
go get git.eeqj.de/sneak/dcf
|
||||
```
|
||||
|
||||
```go
|
||||
package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
|
||||
"git.eeqj.de/sneak/dcf/pkg/dcf"
|
||||
)
|
||||
|
||||
func main() {
|
||||
stores, err := dcf.GetDCFStores(0)
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
|
||||
for _, store := range *stores {
|
||||
fmt.Println(store.String())
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
To work on the library itself:
|
||||
|
||||
```sh
|
||||
git clone git@git.eeqj.de:sneak/dcf.git
|
||||
cd dcf
|
||||
script/setup # dependencies plus the git pre-commit hook
|
||||
make check # test, lint, fmt-check
|
||||
```
|
||||
|
||||
## Entrypoints
|
||||
|
||||
This repo adheres to the
|
||||
[Scripts to Rule Them All](https://github.com/github/scripts-to-rule-them-all)
|
||||
standard. Each `script/` entrypoint has a thin `make` shim; the scripts are
|
||||
where the project's knowledge about flags, timeouts and docker-only linting
|
||||
lives, so use them rather than invoking `go` or `golangci-lint` directly.
|
||||
|
||||
- `script/bootstrap` (`make bootstrap`) — install build dependencies (git, make,
|
||||
go) idempotently. Linting additionally needs docker; markdown formatting needs
|
||||
docker or a local `prettier`.
|
||||
- `script/setup` (`make setup`) — `bootstrap` plus the git pre-commit hook.
|
||||
- `script/test` (`make test`) — `go test` with the race detector and coverage,
|
||||
`-count=1`, 90s timeout; quiet on success, verbose rerun on failure.
|
||||
- `script/lint` (`make lint`) — `golangci-lint` via docker only, against the
|
||||
digest-pinned image in `Dockerfile.lint`, then `script/assert-step-ran` and
|
||||
`script/assert-context-complete` over the build log. Nothing is installed
|
||||
locally and nothing runs on the host.
|
||||
- `script/fmt` (`make fmt`) — format Go with `gofmt` and everything else with
|
||||
`prettier` (writes).
|
||||
- `script/fmt-check` (`make fmt-check`) — the same scope, read-only.
|
||||
- `script/check` (`make check`) — `test` + `lint` + `fmt-check`. What the
|
||||
pre-commit hook runs. Never modifies files.
|
||||
- `script/docker` (`make docker`) — build the image, tagged with
|
||||
`script/projectname`.
|
||||
- `script/cibuild` (`make cibuild`) — the CI gate:
|
||||
`docker build --progress=plain --no-cache-filter=lint --no-cache-filter=builder .`
|
||||
followed by assertions that the lint step and the test step really ran, and
|
||||
that both stages really received the whole repository. The Gitea workflow runs
|
||||
this on every push.
|
||||
- `script/precommit` — the hook body: `go mod tidy` must not change
|
||||
`go.mod`/`go.sum`, then `check`.
|
||||
- `script/install-precommit` (`make hooks`) — installs the hook.
|
||||
- `script/projectname` — prints the project name; other scripts call it so they
|
||||
can stay identical across repos.
|
||||
- `script/prettier`, `script/assert-step-ran`, `script/assert-context-complete`
|
||||
and `script/repo-source-manifest` are helpers, not entrypoints.
|
||||
|
||||
`make build`, `make deps` and `make clean` are the ordinary conveniences.
|
||||
`make build` compiles every package; this repo is a library and ships no binary,
|
||||
so there is no `cmd/` and nothing to link.
|
||||
|
||||
### Why the lint assertions exist
|
||||
|
||||
A green `docker build` is not evidence that the checks ran. On an unchanged tree
|
||||
every layer comes from cache and the build exits 0 having executed nothing;
|
||||
BuildKit silently ignores a `--no-cache-filter` naming a stage that no longer
|
||||
exists; and a `.dockerignore` entry can remove a package from the build context,
|
||||
after which the linter genuinely runs, genuinely examines what it was handed,
|
||||
and genuinely reports `0 issues.` over a repository with a violation in it.
|
||||
`script/assert-step-ran` and `script/assert-context-complete` close both holes;
|
||||
read the comments in them before changing either.
|
||||
|
||||
## Rationale
|
||||
|
||||
I wanted to copy images off my memory cards for processing and like
|
||||
overengineering and reusable code.
|
||||
|
||||
# bugs
|
||||
## Design
|
||||
|
||||
Everything lives in `pkg/dcf`, a single library package with no command
|
||||
entrypoints.
|
||||
|
||||
- `dcf.go` — the public surface. `Store` is one DCF store (one mounted
|
||||
filesystem); `Object` is one file in it, embedded in `Image` and `Video`.
|
||||
`GetDCFStores` finds the stores on the system and walks each one, classifying
|
||||
files by extension.
|
||||
- `helpers.go` — mountpoint discovery. `findAllMountPoints` enumerates the
|
||||
system's physical filesystems via `gopsutil`; `findDCFMountPoints` keeps only
|
||||
those whose root holds a marker directory.
|
||||
|
||||
Detection is by directory name at the filesystem root, so no filesystem is ever
|
||||
mounted and nothing outside a filesystem root is searched.
|
||||
|
||||
## Bugs
|
||||
|
||||
does not yet handle "dcf file groups" where multiple objects share the same
|
||||
index number. if you need this, send me a link to a zip or tar of a memory
|
||||
card that uses it and i'll see what i can do.
|
||||
index number. if you need this, send me a link to a zip or tar of a memory card
|
||||
that uses it and i'll see what i can do.
|
||||
|
||||
# author
|
||||
## TODO
|
||||
|
||||
The work in flight is on the tracker; this is the reading order for picking it
|
||||
up.
|
||||
|
||||
- [#2](https://git.eeqj.de/sneak/dcf/issues/2) — parse the CP-3461 structure
|
||||
rather than grepping every filesystem root for media extensions.
|
||||
- [#3](https://git.eeqj.de/sneak/dcf/issues/3) — DCF file groups: objects
|
||||
sharing an index number are one logical object.
|
||||
- [#4](https://git.eeqj.de/sneak/dcf/issues/4) — API cleanup: `fs.FS`-based
|
||||
store access, no pointer-to-slice returns, contexts on walks. This is also
|
||||
what makes the filesystem walk testable; today it is reachable only through
|
||||
`GetDCFStores`, which needs real mounted media, so it has no test coverage.
|
||||
- [#5](https://git.eeqj.de/sneak/dcf/issues/5) — checksummed import off the
|
||||
card, which is what the library is for.
|
||||
- [#6](https://git.eeqj.de/sneak/dcf/issues/6) — mountpoint detection checks
|
||||
`M4ROOT` at the filesystem root rather than `PRIVATE/M4ROOT`, and
|
||||
`findDCFMountPoints` stops one store short of `requestedCount`.
|
||||
|
||||
## License
|
||||
|
||||
WTFPL. See [LICENSE](LICENSE).
|
||||
|
||||
## Author
|
||||
|
||||
sneak <[sneak@sneak.berlin](mailto:sneak@sneak.berlin)>
|
||||
|
||||
# license
|
||||
|
||||
WTFPL
|
||||
|
||||
Reference in New Issue
Block a user