Pass s3.part_size to the multipart uploader (closes #232)
check / check (push) Waiting to run

s3.part_size was loaded and defaulted but never reached the S3 client,
whose uploader used a fixed 10MiB part. The client now takes the part
size from the config, for storage_url and for the s3.* fields, and
config load rejects a value below 5MiB or above 5GiB, the part sizes S3
accepts. The docs gave the default as 5MB, which the config file reads
as 5,000,000 bytes, below the minimum; they now say 5MiB.

Judgement call: the 5GiB maximum is enforced along with the 5MiB
minimum the issue names.
Trap: at the 5MiB default the uploader's 10,000-part limit caps one
upload at about 48.8GiB, down from about 97.7GiB; blob_size_limit is
not checked against it.

Model: opus-5-5
This commit is contained in:
2026-10-07 10:18:44 +00:00
parent 3fc8a8f2f4
commit 98b981203e
9 changed files with 199 additions and 13 deletions
+4 -4
View File
@@ -287,10 +287,10 @@ storage_url: "rclone://myremote/path/to/backups"
# #use_ssl: true
#
# # Part size for multipart uploads
# # Minimum 5MB, affects memory usage during upload
# # Supports: 5MB, 10M, 100MiB, etc.
# # Default: 5MB
# #part_size: 5MB
# # Minimum 5MiB, maximum 5GiB; affects memory usage during upload
# # Supports: 10MB, 16MiB, 100MiB, etc. (5MB is below the minimum)
# # Default: 5MiB
# #part_size: 5MiB
# Path to local SQLite index database
# This database tracks file state for incremental backups