|
пре 1 месец | |
---|---|---|
LICENSE | пре 1 месец | |
README.md | пре 1 месец |
Manifest file generator and checker.
Given a plain URL, there is no standard way to safely and programmatically download everything “under” that URL path. wget -r
can traverse directory listings if they’re enabled, but every server has a different format, and this does not verify cryptographic integrity of the files, or enable them to be fetched using a different protocol other than HTTP/s.
Currently, the solution that people are using are sidecar files in the format of SHASUMS
checksum files, as well as a SHASUMS.asc
PGP detached signature. This is not checksum-algorithm-agnostic and the sidecar file is not always consistently named.
Real issues I face:
SHASUMS
/SHASUMS.asc
filesA standard, a manifest file format, and a tool for generating same.
The manifest file would be called index.mf
, and the tool for generating such would be called mfer
.
The manifest file would do several important things:
https://example.com/downloadpackage/
one could fetch https://example.com/downloadpackage/index.mf
to enumerate the full directory listing.index.mf
does not imply publishing on ipfs at that timemfer gen
/ mfer gen .
index.mf
mfer check
/ mfer check .
mfer fetch https://example.com/stuff/
/stuff/index.mf
and downloads all files listed in manifest, optionally resuming any that already exist locally, and assures cryptographic integrity of downloaded files.mfer
CLImfer.js
CLIaria2c https://example.com/manifestdirectory/
https://example.com/manifestdirectory/index.mf
, downloads and checksums all files, resumes any that exist locally already)mfer fetch https://example.com/manifestdirectory/
mfer fetch https://exmaple.com/manifestdirectory/?key=5539AD00DE4C42F3AFE11575052443F4DF2A55C2
to assert in the URL which PGP signing key should be used in the manifest, so that shared URLs have a cryptographic trust rootberlin.sneak.app.slideshow
key that maps to a json slideshow config listing what image paths to show, and for how long, and in what orderI’d like to be able to put a bunch of images into a directory, generate a manifest, and then point a slideshow client (such as an ambient display, or a react app with the target directory in a query string arg) at that statically hosted directory, and have it discover the full list of images available at that URL.
I’d like to be able to download a whole tree of files available via HTTP resumably by either HTTP or IPFS/BitTorrent without a .torrent file.
I use filesystems that don’t include data checksums, and I would like a cryptographically signed checksum file so that I can later verify that a set of archive files have not been modified, none are missing, and that the checksums have not been altered in storage by a second party.
I would like to be able to plug in a hard drive or flash drive and, if there is an index.mf
in the root, automatically detect missing/corrupted files, regardless of filesystem format.
Please email sneak@sneak.berlin
with your desired username for an account on this Gitea instance.
I am currently interested in hiring a contractor skilled with the Go standard library interfaces to specify this tool in full and develop a prototype implementation.