feat: implement daemon mode with filesystem watching
All checks were successful
check / check (pull_request) Successful in 4m57s
All checks were successful
check / check (pull_request) Successful in 4m57s
Replace the daemon mode stub with a full implementation that: - Watches configured snapshot paths for filesystem changes using fsnotify (inotify on Linux, FSEvents on macOS, etc.) - Runs an initial full backup on startup - Triggers incremental backups at backup_interval when changes are detected, only for snapshots whose paths were affected - Performs full periodic scans at full_scan_interval regardless of detected changes - Respects min_time_between_run to prevent excessive backup runs - Handles SIGTERM/SIGINT for graceful shutdown, completing any in-progress backup before exiting - Automatically watches newly created subdirectories - Uses a backup semaphore to prevent concurrent backup runs New files: - internal/vaultik/daemon.go: RunDaemon(), changeTracker, watcher setup - internal/vaultik/daemon_test.go: Tests for changeTracker, isSubpath, concurrency safety, and daemon constants closes #3
This commit is contained in:
50
README.md
50
README.md
@@ -170,8 +170,9 @@ vaultik [--config <path>] store info
|
||||
* Config is located at `/etc/vaultik/config.yml` by default
|
||||
* Optional snapshot names argument to create specific snapshots (default: all)
|
||||
* `--cron`: Silent unless error (for crontab)
|
||||
* `--daemon`: Run continuously with inotify monitoring and periodic scans
|
||||
* `--daemon`: Run continuously with filesystem monitoring and periodic scans (see [daemon mode](#daemon-mode))
|
||||
* `--prune`: Delete old snapshots and orphaned blobs after backup
|
||||
* `--skip-errors`: Skip file read errors (log them loudly but continue)
|
||||
|
||||
**snapshot list**: List all snapshots with their timestamps and sizes
|
||||
* `--json`: Output in JSON format
|
||||
@@ -208,6 +209,53 @@ vaultik [--config <path>] store info
|
||||
|
||||
---
|
||||
|
||||
## daemon mode
|
||||
|
||||
When `--daemon` is passed to `snapshot create`, vaultik runs as a
|
||||
long-running process that continuously monitors configured directories for
|
||||
changes and creates backups automatically.
|
||||
|
||||
```sh
|
||||
vaultik --config /etc/vaultik.yaml snapshot create --daemon
|
||||
```
|
||||
|
||||
### how it works
|
||||
|
||||
1. **Initial backup**: On startup, a full backup of all configured snapshots
|
||||
runs immediately.
|
||||
2. **Filesystem watching**: All configured snapshot paths are monitored for
|
||||
file changes using OS-native filesystem notifications (inotify on Linux,
|
||||
FSEvents on macOS, ReadDirectoryChangesW on Windows) via the
|
||||
[fsnotify](https://github.com/fsnotify/fsnotify) library.
|
||||
3. **Periodic backups**: At each `backup_interval` tick, if filesystem
|
||||
changes have been detected and `min_time_between_run` has elapsed since
|
||||
the last backup, a backup runs for only the affected snapshots.
|
||||
4. **Full scans**: At each `full_scan_interval` tick, a full backup of all
|
||||
snapshots runs regardless of detected changes. This catches any changes
|
||||
that filesystem notifications may have missed.
|
||||
5. **Graceful shutdown**: On SIGTERM or SIGINT, the daemon completes any
|
||||
in-progress backup before exiting.
|
||||
|
||||
### configuration
|
||||
|
||||
These config fields control daemon behavior:
|
||||
|
||||
```yaml
|
||||
backup_interval: 1h # How often to check for changes and run backups
|
||||
full_scan_interval: 24h # How often to do a complete scan of all paths
|
||||
min_time_between_run: 15m # Minimum gap between consecutive backup runs
|
||||
```
|
||||
|
||||
### notes
|
||||
|
||||
* New directories created under watched paths are automatically picked up.
|
||||
* The daemon uses the same `CreateSnapshot` logic as one-shot mode — each
|
||||
backup run is a standard incremental snapshot.
|
||||
* The `--prune`, `--cron`, and `--skip-errors` flags work in daemon mode
|
||||
and apply to each individual backup run.
|
||||
|
||||
---
|
||||
|
||||
## architecture
|
||||
|
||||
### s3 bucket layout
|
||||
|
||||
Reference in New Issue
Block a user