168 lines
4.0 KiB
Markdown
168 lines
4.0 KiB
Markdown
# vaultik
|
|
|
|
`vaultik` is a incremental backup daemon written in Go. It
|
|
encrypts data using an `age` public key and uploads each encrypted blob
|
|
directly to a remote S3-compatible object store. It requires no private
|
|
keys, secrets, or credentials stored on the backed-up system.
|
|
|
|
---
|
|
|
|
## what
|
|
|
|
`vaultik` walks a set of configured directories and builds a
|
|
content-addressable chunk map of changed files using deterministic chunking.
|
|
Each chunk is streamed into a blob packer. Blobs are compressed with `zstd`,
|
|
encrypted with `age`, and uploaded directly to remote storage under a
|
|
content-addressed S3 path.
|
|
|
|
No plaintext file contents ever hit disk. No private key is needed or stored
|
|
locally. All encrypted data is streaming-processed and immediately discarded
|
|
once uploaded. Metadata is encrypted and pushed with the same mechanism.
|
|
|
|
## why
|
|
|
|
Existing backup software fails under one or more of these conditions:
|
|
|
|
* Requires secrets (passwords, private keys) on the source system
|
|
* Depends on symmetric encryption unsuitable for zero-trust environments
|
|
* Stages temporary archives or repositories
|
|
* Writes plaintext metadata or plaintext file paths
|
|
|
|
`vaultik` addresses all of these by using:
|
|
|
|
* Public-key-only encryption (via `age`) requires no secrets (other than
|
|
bucket access key) on the source system
|
|
* Blob-level deduplication and batching
|
|
* Local state cache for incremental detection
|
|
* S3-native chunked upload interface
|
|
* Self-contained encrypted snapshot metadata
|
|
|
|
## how
|
|
|
|
1. **install**
|
|
|
|
```sh
|
|
go install git.eeqj.de/sneak/vaultik@latest
|
|
```
|
|
|
|
2. **generate keypair**
|
|
|
|
```sh
|
|
age-keygen -o agekey.txt
|
|
grep 'public key:' agekey.txt
|
|
```
|
|
|
|
3. **write config**
|
|
|
|
```yaml
|
|
source_dirs:
|
|
- /etc
|
|
- /home/user/data
|
|
exclude:
|
|
- '*.log'
|
|
- '*.tmp'
|
|
age_recipient: age1xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
|
|
s3:
|
|
endpoint: https://s3.example.com
|
|
bucket: vaultik-data
|
|
prefix: host1/
|
|
access_key_id: ...
|
|
secret_access_key: ...
|
|
region: us-east-1
|
|
backup_interval: 1h # only used in daemon mode, not for --cron mode
|
|
full_scan_interval: 24h # normally we use inotify to mark dirty, but
|
|
# every 24h we do a full stat() scan
|
|
min_time_between_run: 15m # again, only for daemon mode
|
|
index_path: /var/lib/vaultik/index.sqlite
|
|
chunk_size: 10MB
|
|
blob_size_limit: 10GB
|
|
index_prefix: index/
|
|
```
|
|
|
|
4. **run**
|
|
|
|
```sh
|
|
vaultik backup /etc/vaultik.yaml
|
|
```
|
|
|
|
```sh
|
|
vaultik backup /etc/vaultik.yaml --cron # silent unless error
|
|
```
|
|
|
|
```sh
|
|
vaultik backup /etc/vaultik.yaml --daemon # runs in background, uses inotify
|
|
```
|
|
|
|
---
|
|
|
|
## cli
|
|
|
|
```sh
|
|
vaultik backup /etc/vaultik.yaml
|
|
vaultik restore <bucket> <prefix> <snapshot_id> <target_dir>
|
|
vaultik prune <bucket> <prefix>
|
|
vaultik fetch <bucket> <prefix> <snapshot_id> <filepath> <target_fileordir>
|
|
```
|
|
|
|
* `VAULTIK_PRIVATE_KEY` must be available in environment for `restore` and `prune`
|
|
|
|
---
|
|
|
|
## does not
|
|
|
|
* Store any secrets on the backed-up machine
|
|
* Require mutable remote metadata
|
|
* Use tarballs, restic, rsync, or ssh
|
|
* Require a symmetric passphrase or password
|
|
* Trust the source system with anything
|
|
|
|
---
|
|
|
|
## does
|
|
|
|
* Incremental deduplicated backup
|
|
* Blob-packed chunk encryption
|
|
* Content-addressed immutable blobs
|
|
* Public-key encryption only
|
|
* SQLite-based local and snapshot metadata
|
|
* Fully stream-processed storage
|
|
|
|
---
|
|
|
|
## restore
|
|
|
|
`vaultik restore` downloads only the snapshot metadata and required blobs. It
|
|
never contacts the source system. All restore operations depend only on:
|
|
|
|
* `VAULTIK_PRIVATE_KEY`
|
|
* The bucket
|
|
|
|
The entire system is restore-only from object storage.
|
|
|
|
---
|
|
|
|
## prune
|
|
|
|
Run `vaultik prune` on a machine with the private key. It:
|
|
|
|
* Downloads the most recent snapshot
|
|
* Decrypts metadata
|
|
* Lists referenced blobs
|
|
* Deletes any blob in the bucket not referenced
|
|
|
|
This enables garbage collection from immutable storage.
|
|
|
|
---
|
|
|
|
## license
|
|
|
|
WTFPL — see LICENSE.
|
|
|
|
---
|
|
|
|
## author
|
|
|
|
sneak
|
|
[sneak@sneak.berlin](mailto:sneak@sneak.berlin)
|
|
[https://sneak.berlin](https://sneak.berlin)
|