sneak/vaultik
1
0
Fork 0
Code Issues 2 Pull Requests 1 Actions Packages Projects Releases Wiki Activity
279 Commits 2 Branches 0 Tags
714eea086b8e1b031135dfe14856818d45af7a06
Commit Graph
5 Commits
Author SHA1 Message Date
sneak 714eea086b Document that migrations are supported and none are added before 1.0 (closes #68)
check / check (pull_request) Successful in 3m17s
Details 
#146 called the numbered schema
files a bootstrap and said vaultik has no upgrade path between versions.
That was wrong: they are migrations, and database.New applies any the
database has not recorded. None are added before 1.0 because nothing is
installed anywhere yet; after 1.0 each schema change is a new numbered
file and the local database is migrated on update.

docs/DATAMODEL.md gets a Schema Migrations section that owns the
explanation. The README caveat and roadmap entry and AGENTS.md policy 13
say the same and link to it. CLAUDE.md, the owner's file, gets a
one-line edit so it no longer says migrations are not needed.

Model: opus-5-5
 2026-09-28 18:09:48 +00:00
sneak 43a69c2cfb Fix FK constraint errors in batched file insertion 
Generate file UUIDs upfront in checkFileInMemory() rather than
deferring to Files.Create(). This ensures file_chunks and chunk_files
records have valid FileID values when constructed during file
processing, before the batch insert transaction.

Root cause: For new files, file.ID was empty when building the
fileChunks and chunkFiles slices. The ID was only generated later
in Files.Create(), but by then the slices already had empty FileID
values, causing FK constraint failures.

Also adds PROCESS.md documenting the snapshot creation lifecycle,
database transactions, and FK dependency ordering.
 2025-12-19 19:48:48 +07:00
sneak 78af626759 Major refactoring: UUID-based storage, streaming architecture, and CLI improvements 
This commit represents a significant architectural overhaul of vaultik:

Database Schema Changes:
- Switch files table to use UUID primary keys instead of path-based keys
- Add UUID primary keys to blobs table for immediate chunk association
- Update all foreign key relationships to use UUIDs
- Add comprehensive schema documentation in DATAMODEL.md
- Add SQLite busy timeout handling for concurrent operations

Streaming and Performance Improvements:
- Implement true streaming blob packing without intermediate storage
- Add streaming chunk processing to reduce memory usage
- Improve progress reporting with real-time metrics
- Add upload metrics tracking in new uploads table

CLI Refactoring:
- Restructure CLI to use subcommands: snapshot create/list/purge/verify
- Add store info command for S3 configuration display
- Add custom duration parser supporting days/weeks/months/years
- Remove old backup.go in favor of enhanced snapshot.go
- Add --cron flag for silent operation

Configuration Changes:
- Remove unused index_prefix configuration option
- Add support for snapshot pruning retention policies
- Improve configuration validation and error messages

Testing Improvements:
- Add comprehensive repository tests with edge cases
- Add cascade delete debugging tests
- Fix concurrent operation tests to use SQLite busy timeout
- Remove tolerance for SQLITE_BUSY errors in tests

Documentation:
- Add MIT LICENSE file
- Update README with new command structure
- Add comprehensive DATAMODEL.md explaining database schema
- Update DESIGN.md with UUID-based architecture

Other Changes:
- Add test-config.yml for testing
- Update Makefile with better test output formatting
- Fix various race conditions in concurrent operations
- Improve error handling throughout
 2025-07-22 14:56:44 +02:00
sneak 86b533d6ee Refactor blob storage to use UUID primary keys and implement streaming chunking 
- Changed blob table to use ID (UUID) as primary key instead of hash
- Blob records are now created at packing start, enabling immediate chunk associations
- Implemented streaming chunking to process large files without memory exhaustion
- Fixed blob manifest generation to include all referenced blobs
- Updated all foreign key references from blob_hash to blob_id
- Added progress reporting and improved error handling
- Enforced encryption requirement for all blob packing
- Updated tests to use test encryption keys
- Added Cyrillic transliteration to README
 2025-07-22 07:43:39 +02:00
sneak 67319a4699 initial design 2025-07-20 08:51:38 +02:00