Write rclone uploads under a temporary name and move them into place (closes #266)
check / check (push) Waiting to run
check / check (push) Waiting to run
The rclone backend wrote each object straight to its key, so killing an upload to a local or sftp remote left a truncated object there that the next backup trusted. On every remote with a server-side move, an object is now written under a name ending in `.partial` and moved onto its key with rclone's operations.Move, which removes an object already at the key first; drive, dropbox and others will not move onto one. Listings skip `.partial` names. Rclone's own copy also requires the PartialUploads flag; this does not, because hdfs shows a file while it is written without setting it. Remotes without a move are written in place. Model: opus-5-5
This commit is contained in:
@@ -50,9 +50,10 @@ const storageDirPerm = 0o755
|
||||
// temp file carrying this suffix and only renames it onto the real key once
|
||||
// the whole object is on disk, so an interrupted write can never leave a
|
||||
// truncated object at the key a later run would Stat and trust as a complete
|
||||
// blob. List and ListStream skip these files, so a leftover from an
|
||||
// interrupted write is never listed or trusted as a blob; it is otherwise
|
||||
// harmless and is overwritten when the same key is written again.
|
||||
// blob. The rclone backend's upload does the same on remotes with a
|
||||
// server-side move. List and ListStream skip these files, so a leftover from
|
||||
// an interrupted write is never listed or trusted as a blob; it is otherwise
|
||||
// harmless.
|
||||
const tempSuffix = ".partial"
|
||||
|
||||
// Put stores data at the specified key.
|
||||
|
||||
Reference in New Issue
Block a user