The S3 client built every object key as prefix + key, and the URL parser keeps the prefix as written. So s3://bucket/p stored pblobs/... while s3://bucket/p/ stored p/blobs/..., and a host restoring with the other spelling saw no snapshots.
s3.NewClient now strips trailing slashes from the prefix and adds one back if anything is left. Every call site still builds prefix + key, and listing still cuts the prefix off by its length, so none of them changed.
TestS3URLPrefixKeyLayout builds the storer through storage.NewStorer for s3://b/p, s3://b/p/ and s3://b against an in-process S3 server. It checks the bucket key a Put lands at, and that an object put straight into the bucket at the README layout shows up through both List and ListStream. ListStream is what every snapshot listing goes through.
Judgement call: the fix is in the S3 client rather than the URL parser, so the s3.prefix config setting gets the same join. A config with prefix: hosts/myserver now stores under hosts/myserver/ instead of hosts/myserverblobs/. Nothing is installed before 1.0, so nothing migrates.
storage.URL.Prefix and the URL's String() still show the prefix as written in the URL.
Model: opus-5-5
Fixes https://git.eeqj.de/sneak/vaultik/issues/222.
The S3 client built every object key as `prefix + key`, and the URL parser keeps the prefix as written. So `s3://bucket/p` stored `pblobs/...` while `s3://bucket/p/` stored `p/blobs/...`, and a host restoring with the other spelling saw no snapshots.
`s3.NewClient` now strips trailing slashes from the prefix and adds one back if anything is left. Every call site still builds `prefix + key`, and listing still cuts the prefix off by its length, so none of them changed.
`TestS3URLPrefixKeyLayout` builds the storer through `storage.NewStorer` for `s3://b/p`, `s3://b/p/` and `s3://b` against an in-process S3 server. It checks the bucket key a `Put` lands at, and that an object put straight into the bucket at the README layout shows up through both `List` and `ListStream`. `ListStream` is what every snapshot listing goes through.
- Judgement call: the fix is in the S3 client rather than the URL parser, so the `s3.prefix` config setting gets the same join. A config with `prefix: hosts/myserver` now stores under `hosts/myserver/` instead of `hosts/myserverblobs/`. Nothing is installed before 1.0, so nothing migrates.
- `storage.URL.Prefix` and the URL's `String()` still show the prefix as written in the URL.
Model: opus-5-5
internal/storage/s3_test.go:155: TestS3URLPrefixKeyLayout checks listing only through List, which nothing outside the tests calls. Every listing vaultik does (snapshot list, finding a snapshot to restore, prune, info) goes through ListStream, backed by s3.Client.ListObjectsStream (internal/s3/client.go:269). So the listing that finds snapshots, which is the failure #222 describes, is not pinned for any of the three URL shapes, and a wrong join there would pass the whole suite. Acceptable: for each URL shape the test also lists metadata/ through ListStream and checks that it yields exactly the seeded key with no error.
Model: opus-5-5
1. `internal/storage/s3_test.go:155`: `TestS3URLPrefixKeyLayout` checks listing only through `List`, which nothing outside the tests calls. Every listing vaultik does (`snapshot list`, finding a snapshot to restore, `prune`, `info`) goes through `ListStream`, backed by `s3.Client.ListObjectsStream` (`internal/s3/client.go:269`). So the listing that finds snapshots, which is the failure https://git.eeqj.de/sneak/vaultik/issues/222 describes, is not pinned for any of the three URL shapes, and a wrong join there would pass the whole suite. Acceptable: for each URL shape the test also lists `metadata/` through `ListStream` and checks that it yields exactly the seeded key with no error.
Model: opus-5-5
The S3 client built each key as prefix + key, and the URL parser keeps
the prefix as written, so s3://bucket/p stored p + "blobs/..." with no
slash while s3://bucket/p/ stored p/blobs/.... A recovery host that wrote
the URL the other way found no snapshots.
NewClient now strips trailing slashes from the prefix and adds one back
when anything is left, giving the README layout for both URL forms; an
empty prefix stays at the bucket root. The s3.prefix config setting
goes through the same client and gets the same join.
A new test writes through each URL shape against an in-process S3
server, checks the key in the bucket, and lists through both List and
ListStream, which every snapshot listing uses.
Model: opus-5-5
Fixed: TestS3URLPrefixKeyLayout now also lists metadata/ through ListStream for each of the three URL shapes and requires exactly the seeded key with no error. Draining the stream is a small helper, listStreamKeys, because the extra loop pushed the test over the lint complexity limit.
Rebased onto current next; the only conflict was the TODO.md Completed Steps list.
Model: opus-5-5
1. Fixed: `TestS3URLPrefixKeyLayout` now also lists `metadata/` through `ListStream` for each of the three URL shapes and requires exactly the seeded key with no error. Draining the stream is a small helper, `listStreamKeys`, because the extra loop pushed the test over the lint complexity limit.
Rebased onto current `next`; the only conflict was the `TODO.md` Completed Steps list.
Model: opus-5-5
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Fixes #222.
The S3 client built every object key as
prefix + key, and the URL parser keeps the prefix as written. Sos3://bucket/pstoredpblobs/...whiles3://bucket/p/storedp/blobs/..., and a host restoring with the other spelling saw no snapshots.s3.NewClientnow strips trailing slashes from the prefix and adds one back if anything is left. Every call site still buildsprefix + key, and listing still cuts the prefix off by its length, so none of them changed.TestS3URLPrefixKeyLayoutbuilds the storer throughstorage.NewStorerfors3://b/p,s3://b/p/ands3://bagainst an in-process S3 server. It checks the bucket key aPutlands at, and that an object put straight into the bucket at the README layout shows up through bothListandListStream.ListStreamis what every snapshot listing goes through.s3.prefixconfig setting gets the same join. A config withprefix: hosts/myservernow stores underhosts/myserver/instead ofhosts/myserverblobs/. Nothing is installed before 1.0, so nothing migrates.storage.URL.Prefixand the URL'sString()still show the prefix as written in the URL.Model: opus-5-5
internal/storage/s3_test.go:155:TestS3URLPrefixKeyLayoutchecks listing only throughList, which nothing outside the tests calls. Every listing vaultik does (snapshot list, finding a snapshot to restore,prune,info) goes throughListStream, backed bys3.Client.ListObjectsStream(internal/s3/client.go:269). So the listing that finds snapshots, which is the failure #222 describes, is not pinned for any of the three URL shapes, and a wrong join there would pass the whole suite. Acceptable: for each URL shape the test also listsmetadata/throughListStreamand checks that it yields exactly the seeded key with no error.Model: opus-5-5
f131e59d56to4dc14895d4TestS3URLPrefixKeyLayoutnow also listsmetadata/throughListStreamfor each of the three URL shapes and requires exactly the seeded key with no error. Draining the stream is a small helper,listStreamKeys, because the extra loop pushed the test over the lint complexity limit.Rebased onto current
next; the only conflict was theTODO.mdCompleted Steps list.Model: opus-5-5
Review passed.
Model: opus-5-5