sneak
9de439a0a4
- Extract implementation TODO from DESIGN.md to TODO.md - Remove completed Phase 1 tasks - Add --quick option to verify command for S3 hash checking - Update documentation to reflect deep verification as default
112 lines
3.7 KiB
Markdown
112 lines
3.7 KiB
Markdown
# Implementation TODO
|
|
|
|
## Local Index Database
|
|
1. Implement SQLite schema creation
|
|
1. Create Index type with all database operations
|
|
1. Add transaction support and proper locking
|
|
1. Implement file tracking (save, lookup, delete)
|
|
1. Implement chunk tracking and deduplication
|
|
1. Implement blob tracking and chunk-to-blob mapping
|
|
1. Write tests for all index operations
|
|
|
|
## Chunking and Hashing
|
|
1. Implement Rabin fingerprint chunker
|
|
1. Create streaming chunk processor
|
|
1. Implement SHA256 hashing for chunks
|
|
1. Add configurable chunk size parameters
|
|
1. Write tests for chunking consistency
|
|
|
|
## Compression and Encryption
|
|
1. Implement zstd compression wrapper
|
|
1. Integrate age encryption library
|
|
1. Create Encryptor type for public key encryption
|
|
1. Create Decryptor type for private key decryption
|
|
1. Implement streaming encrypt/decrypt pipelines
|
|
1. Write tests for compression and encryption
|
|
|
|
## Blob Packing
|
|
1. Implement BlobWriter with size limits
|
|
1. Add chunk accumulation and flushing
|
|
1. Create blob hash calculation
|
|
1. Implement proper error handling and rollback
|
|
1. Write tests for blob packing scenarios
|
|
|
|
## S3 Operations
|
|
1. Integrate MinIO client library
|
|
1. Implement S3Client wrapper type
|
|
1. Add multipart upload support for large blobs
|
|
1. Implement retry logic with exponential backoff
|
|
1. Add connection pooling and timeout handling
|
|
1. Write tests using MinIO container
|
|
|
|
## Backup Command - Basic
|
|
1. Implement directory walking with exclusion patterns
|
|
1. Add file change detection using index
|
|
1. Integrate chunking pipeline for changed files
|
|
1. Implement blob upload coordination
|
|
1. Add progress reporting to stderr
|
|
1. Write integration tests for backup
|
|
|
|
## Snapshot Metadata
|
|
1. Implement snapshot metadata extraction from index
|
|
1. Create SQLite snapshot database builder
|
|
1. Add metadata compression and encryption
|
|
1. Implement metadata chunking for large snapshots
|
|
1. Add hash calculation and verification
|
|
1. Implement metadata upload to S3
|
|
1. Write tests for metadata operations
|
|
|
|
## Restore Command
|
|
1. Implement snapshot listing and selection
|
|
1. Add metadata download and reconstruction
|
|
1. Implement hash verification for metadata
|
|
1. Create file restoration logic with chunk retrieval
|
|
1. Add blob caching for efficiency
|
|
1. Implement proper file permissions and mtime restoration
|
|
1. Write integration tests for restore
|
|
|
|
## Prune Command
|
|
1. Implement latest snapshot detection
|
|
1. Add referenced blob extraction from metadata
|
|
1. Create S3 blob listing and comparison
|
|
1. Implement safe deletion of unreferenced blobs
|
|
1. Add dry-run mode for safety
|
|
1. Write tests for prune scenarios
|
|
|
|
## Verify Command
|
|
1. Implement metadata integrity checking
|
|
1. Add blob existence verification
|
|
1. Implement quick mode (S3 hash checking)
|
|
1. Implement deep mode (download and verify chunks)
|
|
1. Add detailed error reporting
|
|
1. Write tests for verification
|
|
|
|
## Fetch Command
|
|
1. Implement single-file metadata query
|
|
1. Add minimal blob downloading for file
|
|
1. Create streaming file reconstruction
|
|
1. Add support for output redirection
|
|
1. Write tests for fetch command
|
|
|
|
## Daemon Mode
|
|
1. Implement inotify watcher for Linux
|
|
1. Add dirty path tracking in index
|
|
1. Create periodic full scan scheduler
|
|
1. Implement backup interval enforcement
|
|
1. Add proper signal handling and shutdown
|
|
1. Write tests for daemon behavior
|
|
|
|
## Cron Mode
|
|
1. Implement silent operation mode
|
|
1. Add proper exit codes for cron
|
|
1. Implement lock file to prevent concurrent runs
|
|
1. Add error summary reporting
|
|
1. Write tests for cron mode
|
|
|
|
## Finalization
|
|
1. Add comprehensive logging throughout
|
|
1. Implement proper error wrapping and context
|
|
1. Add performance metrics collection
|
|
1. Create end-to-end integration tests
|
|
1. Write documentation and examples
|
|
1. Set up CI/CD pipeline |