An album whose folder cannot be made stops the run before the other albums and before failures.json is saved #197
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: sneak/quak#197
Reference in New Issue
Block a user
Severity: medium
Where: the per-album loop at the end of
runLockedBackupinsrc/backup.ts:mkdirSync(colDir, ...)andwriteAlbumJSON(...)run outside anytry, unlike the symlink work beside them (rebuildSymlink,removeStaleLinks,removeStaleAlbumDirs), which is caught and logged.Problem: an album name the server sends can make either call throw, and the throw ends the run with
quak: ENAMETOOLONG: ...orquak: EEXIST: ...and exit 1. Every original is stored by then, but the album folders from that album on are not rebuilt, andfailures.jsonis never written, so the failures this run did record (a 404, say) are lost and their attempt counts never advance. The same happens on every later run, so for such an account the album folders and the ledger never work. Two names do it:x.jsonbeside an album namedx: the foldercollections/x.json/and the JSON filecollections/x.jsonare the same path.The README says each run rebuilds the album folders and removes stale links, and that per-file errors are logged and the run continues.
Reproduced: two backups through the fake Ente server of
test/ente-server.ts. Albumsx(one file) andx.json(one file): exit 1,quak: EEXIST: file already exists, mkdir '.../collections/x.json', both originals at their save paths, the same again on the second run. An album named with 300Ls (one file) before an albumokwhose file always answers 404:FAILED original b.jpg: HTTP 404is printed, thenquak: ENAMETOOLONG: name too long, mkdir '.../collections/LLL...', exit 1, nofailures.jsonand nocollections/ok.json.Done when: a failure to make or write one album's folder or JSON is logged on one line naming the album, counted as a failure (the run exits 1), and the other albums and
failures.jsonare still written; a folder name that would clash with another album's JSON name, or exceed the name length the file system allows, gets the album's ID added, as two albums with one name do (README, "Backup layout"). Tests intest/cli/backup.test.tsortest/cli/backup-end-to-end.test.tscover both names and check the ledger and the later album's folder.Model: fable-5-1
Plan, in
src/backup.tsunless another file is named.runLockedBackup(its folder, its links, its JSON) runs inside onetry. A failure logsFAILED album NAME: REASONon one line, the album's ID is listed in the result asundecryptableAlbumsis, andquak backupexits 1 while any is listed and shows it in--json(src/cli-commands.ts). The other albums are still rebuilt andfailures.jsonis still saved.(ID), as two albums with one name do..jsonfile) is shortened at a character boundary so that it fits with(ID)and.json, and gets the ID. Shortening a name to a length in bytes is the work ofsrc/filename.ts("safe file names from server metadata"), so that function goes there. The same limit applies to the link names made for the files in an album folder, since a title over 255 bytes fails its link the same way on every run.Model: opus-5-5
Built in #204, as the plan comment says. An album whose folder or JSON cannot be written is logged as
FAILED album <name>: <reason>, its ID goes in the newfailedAlbumsof the result and of--json, and the run goes on and exits 1. An album namedx.jsonbeside an albumxgets the folderx.json (ID)/. An album or link name over the 255-byte limit is cut short between two characters and gets the ID.Model: opus-5-5