Write rclone uploads under a temporary name and move them into place (closes #266)
check / check (push) Waiting to run
check / check (push) Waiting to run
The rclone backend wrote each object straight to its key, so killing an upload to a local or sftp remote left a truncated object there that the next backup trusted. On every remote with a server-side move, an object is now written under a name ending in `.partial` and moved onto its key with rclone's operations.Move, which removes an object already at the key first; drive, dropbox and others will not move onto one. Listings skip `.partial` names. Rclone's own copy also requires the PartialUploads flag; this does not, because hdfs shows a file while it is written without setting it. Remotes without a move are written in place. Model: opus-5-5
This commit is contained in:
@@ -428,6 +428,16 @@ a local or mounted filesystem. Useful for testing or backing up to a NAS.
|
||||
**Rclone** (`rclone://remote/path`): Uses rclone's 70+ supported cloud
|
||||
providers. Requires rclone to be configured separately (`rclone config`).
|
||||
|
||||
An upload cut off part-way leaves nothing under the object's name on S3, which
|
||||
shows an object only once its upload has completed, and on the local filesystem
|
||||
backend, which writes a temporary file and renames it into place. Rclone
|
||||
remotes with a server-side move (local and sftp among them) are written under a
|
||||
temporary name ending in `.partial` and moved into place. Rclone remotes without
|
||||
one are written in place: where such a remote shows a file while it is still
|
||||
being written, a killed upload can leave a truncated object under its name,
|
||||
which a later backup takes for complete. A leftover `.partial` file is ignored
|
||||
and can be deleted.
|
||||
|
||||
Legacy S3 configuration via `s3.*` fields (endpoint, bucket, prefix, etc.) is
|
||||
still supported for backward compatibility. `storage_url` takes precedence if
|
||||
both are set.
|
||||
|
||||
Reference in New Issue
Block a user