Compare commits
3
Commits
3ae49bd477
..
next
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
e6825abcdb | ||
|
|
c27e2cb629 | ||
|
|
e6a9e929c2 |
@@ -8,9 +8,9 @@ and downloads individual images while decrypting them on the way to disk.
|
|||||||
quak also includes a resilient backup command that downloads every file in the
|
quak also includes a resilient backup command that downloads every file in the
|
||||||
account into a deduplicated local directory tree, skipping files that already
|
account into a deduplicated local directory tree, skipping files that already
|
||||||
exist on disk and continuing past individual download failures instead of
|
exist on disk and continuing past individual download failures instead of
|
||||||
crashing. It decrypts and persists all three metadata layers (basic, private
|
crashing. For each file it persists the basic metadata fields quak keeps (title,
|
||||||
magic, public magic) per file, including camera info, GPS coordinates, captions,
|
file type, creation and modification time, latitude, longitude, content hash),
|
||||||
and any face/keyword labels the Ente clients have added. A helper subcommand can
|
and the private and public magic metadata in full. A helper subcommand can
|
||||||
detect and regenerate missing thumbnails, encrypting and uploading them back to
|
detect and regenerate missing thumbnails, encrypting and uploading them back to
|
||||||
the server.
|
the server.
|
||||||
|
|
||||||
@@ -171,16 +171,19 @@ local cache is reliable, and the UI is responsive on a five-year-old laptop.
|
|||||||
|
|
||||||
All work on quak is test-driven. No exceptions.
|
All work on quak is test-driven. No exceptions.
|
||||||
|
|
||||||
1. Every change starts on a feature branch off `main`.
|
1. Every change starts on a feature branch off `next`, and its pull request
|
||||||
|
targets `next`.
|
||||||
2. The first commit on the branch is the test suite for what is being added or
|
2. The first commit on the branch is the test suite for what is being added or
|
||||||
changed. Those tests must fail at that commit; the branch is red until the
|
changed. Those tests must fail at that commit; the branch is red until the
|
||||||
implementation lands.
|
implementation lands.
|
||||||
3. Subsequent commits add the implementation and any refactors needed to make
|
3. Subsequent commits add the implementation and any refactors needed to make
|
||||||
the tests pass.
|
the tests pass.
|
||||||
4. A feature branch can only be merged into `main` when `make check` is green.
|
4. A pull request can only be merged into `next` when `make check` is green.
|
||||||
`main` is always green. CI runs `script/cibuild`, which builds the
|
Once it has passed review, the repository manager squash-merges it into
|
||||||
`Dockerfile`: its `lint` and `test` phases, then the compile, so neither a
|
`next`. Only sneak merges `next` into `main`. `main` and `next` are always
|
||||||
red branch nor one that does not compile can pass CI.
|
green. CI runs `script/cibuild`, which builds the `Dockerfile`: its `lint`
|
||||||
|
and `test` phases, then the compile, so neither a red branch nor one that
|
||||||
|
does not compile can pass CI.
|
||||||
5. Tests are the canonical API documentation for this library. Every test file
|
5. Tests are the canonical API documentation for this library. Every test file
|
||||||
is commented thoroughly enough that a reader who has never seen quak can
|
is commented thoroughly enough that a reader who has never seen quak can
|
||||||
learn how to use it from the tests alone. Comments explain why a behavior
|
learn how to use it from the tests alone. Comments explain why a behavior
|
||||||
@@ -198,7 +201,7 @@ All work on quak is test-driven. No exceptions.
|
|||||||
not the tests, and so not the full `make check`. This is deliberate so the
|
not the tests, and so not the full `make check`. This is deliberate so the
|
||||||
TDD red-phase commit (failing tests, no implementation yet) can land. The
|
TDD red-phase commit (failing tests, no implementation yet) can land. The
|
||||||
`test` phase is part of the image build, which is what CI executes via
|
`test` phase is part of the image build, which is what CI executes via
|
||||||
`script/cibuild`, so a red branch still cannot reach `main`.
|
`script/cibuild`, so a red branch still cannot reach `next`.
|
||||||
|
|
||||||
## Design
|
## Design
|
||||||
|
|
||||||
@@ -220,7 +223,7 @@ quak/
|
|||||||
and search, request pools
|
and search, request pools
|
||||||
backup.ts resilient full-account backup with dedup
|
backup.ts resilient full-account backup with dedup
|
||||||
metadata-backup.ts
|
metadata-backup.ts
|
||||||
backup-metadata: all decrypted metadata as JSON
|
backup-metadata: the metadata quak keeps, as JSON
|
||||||
mldata-fetch.ts fetch + decrypt per-file ML data
|
mldata-fetch.ts fetch + decrypt per-file ML data
|
||||||
filename.ts safe file names from server metadata
|
filename.ts safe file names from server metadata
|
||||||
errors.ts error types shared across layers
|
errors.ts error types shared across layers
|
||||||
@@ -470,7 +473,7 @@ quak files --collection <id> [--json] list files in a collec
|
|||||||
quak get <fileID> [--out path] [--collection] download and decrypt a file
|
quak get <fileID> [--out path] [--collection] download and decrypt a file
|
||||||
quak get-thumb <fileID> [--out] [--collection] download and decrypt a thumbnail
|
quak get-thumb <fileID> [--out] [--collection] download and decrypt a thumbnail
|
||||||
quak backup <dir> [--json] full incremental backup
|
quak backup <dir> [--json] full incremental backup
|
||||||
quak backup-metadata <dir> [--exif] dump all decrypted metadata as JSON
|
quak backup-metadata <dir> [--exif] dump the metadata quak keeps as JSON
|
||||||
quak helper list-missing-thumbnails [--json] find files with missing thumbnails
|
quak helper list-missing-thumbnails [--json] find files with missing thumbnails
|
||||||
quak helper fix-missing-thumbnails [--file ids] [--json] generate + upload missing thumbnails
|
quak helper fix-missing-thumbnails [--file ids] [--json] generate + upload missing thumbnails
|
||||||
```
|
```
|
||||||
@@ -522,7 +525,9 @@ the smallest does not.
|
|||||||
originals/
|
originals/
|
||||||
<fileID>.<ext> actual file content (one per unique file,
|
<fileID>.<ext> actual file content (one per unique file,
|
||||||
two for a live photo: see below)
|
two for a live photo: see below)
|
||||||
<fileID>.json all decrypted metadata for that file
|
<fileID>.json the file's basic metadata fields quak
|
||||||
|
keeps, and its private and public magic
|
||||||
|
metadata
|
||||||
<fileID>.livephoto.json which of a live photo's two files is which
|
<fileID>.livephoto.json which of a live photo's two files is which
|
||||||
collections/
|
collections/
|
||||||
<name>/
|
<name>/
|
||||||
@@ -611,8 +616,7 @@ The library's primary surface is the cache-backed `Library`; the lower-level
|
|||||||
`Client` sits underneath it and is covered by the Design sections above. The
|
`Client` sits underneath it and is covered by the Design sections above. The
|
||||||
test suite is the canonical, executable documentation — `test/library/` and
|
test suite is the canonical, executable documentation — `test/library/` and
|
||||||
`test/client/usage.test.ts` walk most operations, `test/cli/backup.test.ts`
|
`test/client/usage.test.ts` walk most operations, `test/cli/backup.test.ts`
|
||||||
walks `lib.backup()`, and `yarn test` verifies them. No test calls
|
walks `lib.backup()`, and `yarn test` verifies them.
|
||||||
`getFileByID()`.
|
|
||||||
|
|
||||||
### Opening a library
|
### Opening a library
|
||||||
|
|
||||||
@@ -838,14 +842,15 @@ documents:
|
|||||||
`yarn.lock`. Never `git add -A`. Never force-push to main.
|
`yarn.lock`. Never `git add -A`. Never force-push to main.
|
||||||
|
|
||||||
- **The "Development workflow" section above.** All changes go on feature
|
- **The "Development workflow" section above.** All changes go on feature
|
||||||
branches. Tests are written first and committed in a failing state before the
|
branches off `next`, and every pull request targets `next`; only sneak merges
|
||||||
implementation. Tests are the canonical API documentation and must be
|
`next` into `main`. Tests are written first and committed in a failing state
|
||||||
commented thoroughly. `main` is always green.
|
before the implementation. Tests are the canonical API documentation and must
|
||||||
|
be commented thoroughly. `main` and `next` are always green.
|
||||||
|
|
||||||
- **Required checks before every commit:** `make lint` must pass — that is
|
- **Required checks before every commit:** `make lint` must pass — that is
|
||||||
eslint plus the prettier check, and it builds the `lint` phase of the
|
eslint plus the prettier check, and it builds the `lint` phase of the
|
||||||
`Dockerfile`, so it needs docker. The pre-commit hook enforces exactly that.
|
`Dockerfile`, so it needs docker. The pre-commit hook enforces exactly that.
|
||||||
`make check` (which also runs the tests) must pass before merging to `main`.
|
`make check` (which also runs the tests) must pass before merging into `next`.
|
||||||
`make fmt-check` is available for a host-side formatting check on its own, but
|
`make fmt-check` is available for a host-side formatting check on its own, but
|
||||||
it is not a separate requirement: `make lint` already covers it, and running
|
it is not a separate requirement: `make lint` already covers it, and running
|
||||||
both would check formatting twice. Never invoke eslint or prettier directly;
|
both would check formatting twice. Never invoke eslint or prettier directly;
|
||||||
|
|||||||
@@ -1,12 +1,15 @@
|
|||||||
# Workflow
|
# Workflow
|
||||||
|
|
||||||
- branch (from `main`)
|
- branch from `next`
|
||||||
- do the work in Next Step
|
- do the work in Next Step
|
||||||
- move Next Step to the top of Completed Steps
|
- move Next Step to the top of Completed Steps
|
||||||
- move the top item of Future Steps into Next Step
|
- move the top item of Future Steps into Next Step
|
||||||
- commit (`TODO.md` changes in the same commit as the work)
|
- commit (`TODO.md` changes in the same commit as the work)
|
||||||
- merge to `main` if the branch is not protected, otherwise open a PR
|
|
||||||
- push
|
- push
|
||||||
|
- open a pull request that targets `next`
|
||||||
|
- once the pull request has passed review, the repository manager squash-merges
|
||||||
|
it into `next`
|
||||||
|
- only sneak merges `next` into `main`
|
||||||
|
|
||||||
# Status
|
# Status
|
||||||
|
|
||||||
@@ -22,6 +25,16 @@ declares one.
|
|||||||
|
|
||||||
# Completed Steps
|
# Completed Steps
|
||||||
|
|
||||||
|
- 2026-09-29: The "Workflow" list at the top of this file now says to branch
|
||||||
|
from `next` and open a pull request that targets `next`, that the repository
|
||||||
|
manager squash-merges a reviewed pull request into `next`, and that only sneak
|
||||||
|
merges `next` into `main`, as the README does (issue 137).
|
||||||
|
|
||||||
|
- 2026-09-29: The README's "Development workflow" and "For LLMs" sections now
|
||||||
|
say work branches from `next`, every pull request targets `next`, the
|
||||||
|
repository manager squash-merges reviewed pull requests into `next`, and only
|
||||||
|
sneak merges `next` into `main` (issue 135).
|
||||||
|
|
||||||
- 2026-09-29: Brought the README and this file in line with `next` after the
|
- 2026-09-29: Brought the README and this file in line with `next` after the
|
||||||
milestone merge (issue 132). The Next Step says no implementation work is open
|
milestone merge (issue 132). The Next Step says no implementation work is open
|
||||||
and the cache design (issue 36) waits on sneak's review. README corrections:
|
and the cache design (issue 36) waits on sneak's review. README corrections:
|
||||||
@@ -41,12 +54,15 @@ declares one.
|
|||||||
`logout` open no library; `--exif` records XMP and, for a JPEG, EXIF, and no
|
`logout` open no library; `--exif` records XMP and, for a JPEG, EXIF, and no
|
||||||
IPTC; which commands take `--json`; a failed ML data request may not have been
|
IPTC; which commands take `--json`; a failed ML data request may not have been
|
||||||
retried; the thumbnail fixer is not limited to baseline JPEG; `quak backup`
|
retried; the thumbnail fixer is not limited to baseline JPEG; `quak backup`
|
||||||
still fetches ML data; the default cache directory is the per-user one, not an
|
still fetches ML data; a backup's JSON holds the basic metadata fields quak
|
||||||
XDG path on macOS; pinned originals can exceed `cacheOriginalsMaxBytes`; which
|
keeps and the private and public magic metadata, not every decrypted field,
|
||||||
tests cover which operations; a default read is only as current as the last
|
and the README no longer lists what the magic metadata holds; the default
|
||||||
refresh whose requests all succeeded; `fresh()` and `lib.backup()` join a
|
cache directory is the per-user one, not an XDG path on macOS; pinned
|
||||||
refresh already running; a `Photo` has no `thumbnailPath` or `originalPath`;
|
originals can exceed `cacheOriginalsMaxBytes`; which tests cover which
|
||||||
and `408` and `429` are retried.
|
operations; a default read is only as current as the last refresh whose
|
||||||
|
requests all succeeded; `fresh()` and `lib.backup()` join a refresh already
|
||||||
|
running; a `Photo` has no `thumbnailPath` or `originalPath`; and `408` and
|
||||||
|
`429` are retried.
|
||||||
|
|
||||||
- 2026-09-28: Tested the live-photo writer's fsyncs (issue 130). A test checks
|
- 2026-09-28: Tested the live-photo writer's fsyncs (issue 130). A test checks
|
||||||
that the image's and the video's temp files are fsynced before either is
|
that the image's and the video's temp files are fsynced before either is
|
||||||
|
|||||||
Reference in New Issue
Block a user