Commit Graph

18 Commits

Author SHA1 Message Date
8ee180e27c Adopt scripts-to-rule-them-all: script/ entrypoints, Makefile shims 2026-07-07 00:19:20 +02:00
84f8366c41 Add standard Workflow section to TODO.md
All checks were successful
check / check (push) Successful in 1m23s
2026-07-06 21:06:40 +02:00
2645f60536 Add TODO.md
All checks were successful
check / check (push) Successful in 1m20s
2026-07-06 20:35:45 +02:00
a256b83734 Fix errcheck lint failures with proper error handling
All checks were successful
check / check (push) Successful in 1m42s
Handle every error return flagged by golangci-lint's errcheck rather
than discarding it:

- run.go: add a cleanup() helper that logs a warning (and ignores
  ErrNotExist) when removing a temp file fails, so leaked scratch files
  are surfaced; use it for all best-effort removals.
- copy.go / extract.go: log a warning on deferred Close() failures for
  the source file, destination DB, and result-set rows.
- extract.go: on the rollback path, ignore the benign sql.ErrTxDone
  (already committed) and log any other rollback failure.
- verify.go: add killCat() which ignores os.ErrProcessDone (zstdcat
  already exited via SIGPIPE) and logs any unexpected kill failure.

make check is clean (0 lint issues, tests pass).
2026-06-28 10:25:47 +02:00
8845f0ade6 Add minimal compilation smoke test; tidy go.mod
Some checks failed
check / check (push) Failing after 59s
Add a smoke test that references the package's exported surface so go
test fails if it stops compiling. Run go mod tidy to promote cobra to
a direct dependency.
2026-06-28 10:09:56 +02:00
7316054b07 Add README, LICENSE, Makefile, Dockerfile, and CI
Add a detailed README, WTFPL LICENSE, and build/CI tooling modeled on
the vaultik repo (Makefile, multi-stage digest-pinned Dockerfile,
.gitea/workflows/check.yml). Bump Go to 1.26.4 and pin golangci-lint
to v2.12.2. gofmt existing sources so the new fmt-check gate passes.
2026-06-28 10:08:44 +02:00
9c4f8a7cc9 latest 2026-06-28 09:42:31 +02:00
05b5c2b528 fix SQLite database locking issues
- Use tx.QueryRow instead of db.QueryRow within transaction for media table check
- Switch to WAL mode instead of OFF for better concurrency handling
- Add busy_timeout pragma to wait up to 5 seconds if database is locked
- Clear transaction variable after commit to prevent defer rollback issues
2026-02-12 13:40:58 -08:00
8e3a868b7a remove parallel processing to fix SQLite locking issues
- Remove parallel.go entirely
- Process days strictly sequentially to avoid SQLITE_BUSY errors
- Complete each day fully (extract, dump, compress, verify, write) before starting next
- Clean up temp files immediately after each day to save disk space
- SQLite cannot handle multiple concurrent ATTACH operations on same database
2026-02-12 13:26:55 -08:00
56c3fe7804 fix Linux build: use golang.org/x/sys/unix for Fadvise
Replace syscall.Fadvise with unix.Fadvise from golang.org/x/sys/unix
which is the correct portable way to access fadvise on Linux
2026-02-12 13:15:39 -08:00
156217043e change temp directory to NVMe storage
Move temporary working directory from /srv/tmp to /srv/storage/tmp
to leverage NVMe storage for better I/O performance during extraction
2026-02-12 13:10:04 -08:00
f23db96922 optimize file copy performance for large database files
- Increase copy buffer from 4MB to 256MB for NVMe-speed transfers
- Add Linux-specific fadvise hints for sequential reads and prefetching
- Pre-allocate destination file to avoid fragmentation
- Add throughput monitoring to track copy performance
- Platform-specific build tags for Linux optimizations
2026-02-12 13:08:08 -08:00
cf79e008b5 speed optimizations 2026-02-12 13:01:52 -08:00
2d46dc70a9 date range 2026-02-11 03:01:48 -08:00
3c890f0b83 speed up database pruning for throwaway copies
Disable crash-safety pragmas (synchronous=OFF, journal_mode=MEMORY),
use a 200MB page cache, disable foreign keys to avoid per-row CASCADE
overhead, bulk-delete junction tables explicitly, and wrap all deletes
in a single transaction.
2026-02-09 01:48:42 -08:00
c31484f0cf support targeting specific date 2026-02-09 01:18:16 -08:00
136a090b6d change url 2026-02-09 00:48:03 -08:00
e6e85b6e11 initial 2026-02-09 00:42:06 -08:00