Commit Graph

14 Commits

Author SHA1 Message Date
8845f0ade6 Add minimal compilation smoke test; tidy go.mod
Some checks failed
check / check (push) Failing after 59s
Add a smoke test that references the package's exported surface so go
test fails if it stops compiling. Run go mod tidy to promote cobra to
a direct dependency.
2026-06-28 10:09:56 +02:00
7316054b07 Add README, LICENSE, Makefile, Dockerfile, and CI
Add a detailed README, WTFPL LICENSE, and build/CI tooling modeled on
the vaultik repo (Makefile, multi-stage digest-pinned Dockerfile,
.gitea/workflows/check.yml). Bump Go to 1.26.4 and pin golangci-lint
to v2.12.2. gofmt existing sources so the new fmt-check gate passes.
2026-06-28 10:08:44 +02:00
9c4f8a7cc9 latest 2026-06-28 09:42:31 +02:00
05b5c2b528 fix SQLite database locking issues
- Use tx.QueryRow instead of db.QueryRow within transaction for media table check
- Switch to WAL mode instead of OFF for better concurrency handling
- Add busy_timeout pragma to wait up to 5 seconds if database is locked
- Clear transaction variable after commit to prevent defer rollback issues
2026-02-12 13:40:58 -08:00
8e3a868b7a remove parallel processing to fix SQLite locking issues
- Remove parallel.go entirely
- Process days strictly sequentially to avoid SQLITE_BUSY errors
- Complete each day fully (extract, dump, compress, verify, write) before starting next
- Clean up temp files immediately after each day to save disk space
- SQLite cannot handle multiple concurrent ATTACH operations on same database
2026-02-12 13:26:55 -08:00
56c3fe7804 fix Linux build: use golang.org/x/sys/unix for Fadvise
Replace syscall.Fadvise with unix.Fadvise from golang.org/x/sys/unix
which is the correct portable way to access fadvise on Linux
2026-02-12 13:15:39 -08:00
156217043e change temp directory to NVMe storage
Move temporary working directory from /srv/tmp to /srv/storage/tmp
to leverage NVMe storage for better I/O performance during extraction
2026-02-12 13:10:04 -08:00
f23db96922 optimize file copy performance for large database files
- Increase copy buffer from 4MB to 256MB for NVMe-speed transfers
- Add Linux-specific fadvise hints for sequential reads and prefetching
- Pre-allocate destination file to avoid fragmentation
- Add throughput monitoring to track copy performance
- Platform-specific build tags for Linux optimizations
2026-02-12 13:08:08 -08:00
cf79e008b5 speed optimizations 2026-02-12 13:01:52 -08:00
2d46dc70a9 date range 2026-02-11 03:01:48 -08:00
3c890f0b83 speed up database pruning for throwaway copies
Disable crash-safety pragmas (synchronous=OFF, journal_mode=MEMORY),
use a 200MB page cache, disable foreign keys to avoid per-row CASCADE
overhead, bulk-delete junction tables explicitly, and wrap all deletes
in a single transaction.
2026-02-09 01:48:42 -08:00
c31484f0cf support targeting specific date 2026-02-09 01:18:16 -08:00
136a090b6d change url 2026-02-09 00:48:03 -08:00
e6e85b6e11 initial 2026-02-09 00:42:06 -08:00