Write rclone uploads under a temporary name, re-upload short blobs (closes #266)
check / check (push) Canceled after 0s
check / check (push) Canceled after 0s
The rclone backend wrote each object straight to its key, so killing an upload to a local or sftp remote left a truncated object there. The next backup found the key, skipped the upload and recorded a snapshot that could not be restored. On a remote where rclone says a file can be seen while it is still being written, an object is now written under a name ending in `.partial` and moved onto its key with the remote's server-side move, as rclone's own copy does; listings skip such names. A backup also uploads a blob again when the stored object's size differs from the blob's. Judgement call: the temporary name is used only where rclone sets its PartialUploads feature; other remotes already show an object only once complete. Model: opus-5-5
This commit is contained in:
@@ -428,6 +428,17 @@ a local or mounted filesystem. Useful for testing or backing up to a NAS.
|
||||
**Rclone** (`rclone://remote/path`): Uses rclone's 70+ supported cloud
|
||||
providers. Requires rclone to be configured separately (`rclone config`).
|
||||
|
||||
An upload cut off part-way leaves nothing under the object's name on S3, which
|
||||
shows an object only once its upload has completed, and on the local filesystem
|
||||
backend, which writes a temporary file and renames it into place. Rclone
|
||||
remotes that show a file while it is still being written (local, sftp, ftp and
|
||||
smb among them) are written under a temporary name ending in `.partial` and
|
||||
moved into place where the remote has a server-side move; without one they are
|
||||
written in place. Other rclone remotes show an object only once its upload has
|
||||
completed. A leftover `.partial` file is ignored and can be deleted. A backup
|
||||
that finds a blob stored at a size other than the one it packed uploads the
|
||||
blob again.
|
||||
|
||||
Legacy S3 configuration via `s3.*` fields (endpoint, bucket, prefix, etc.) is
|
||||
still supported for backward compatibility. `storage_url` takes precedence if
|
||||
both are set.
|
||||
|
||||
Reference in New Issue
Block a user