Document that migrations are supported and none are added before 1.0 (closes #68)
The docs now say vaultik supports migrations. The numbered files in `internal/database/schema/` are migrations: `schema_migrations` records which have run, and opening a database applies any that have not. None are added before 1.0 because nothing is installed anywhere yet, so a schema change edits `001.sql` directly. After 1.0 each change is a new numbered file, and an existing local database is migrated when vaultik is updated. `docs/DATAMODEL.md` owns the explanation. The README caveat and roadmap entry and `AGENTS.md` policy 13 link to it. This replaces the wording from #146, which said there was no upgrade path. Disclosure: `CLAUDE.md` line 33, the owner's file, changes from "do not need to support migrations" to "do not add migrations before 1.0". Model: opus-5-5
This commit was merged in pull request #205.
This commit is contained in:
@@ -102,14 +102,12 @@ Version: 2025-06-08
|
||||
build files are acceptable in the root, but source code and other files
|
||||
should be organized in appropriate subdirectories.
|
||||
|
||||
13. Pre-1.0: NEVER write database migrations. There are no live databases
|
||||
anywhere — every user's local index can be rebuilt from a fresh full
|
||||
backup. To change the schema, edit `internal/database/schema/001.sql`
|
||||
(and any code that touches the affected tables) directly; do not add new
|
||||
numbered schema files. Those numbered files and the `schema_migrations`
|
||||
table they populate only bootstrap a fresh database — they are not an
|
||||
upgrade path. The local index is disposable until 1.0 ships and is
|
||||
tagged; once 1.0 is tagged that clause expires and the question of
|
||||
upgrading existing indexes returns. See [`docs/DATAMODEL.md`](docs/DATAMODEL.md)
|
||||
for the full explanation.
|
||||
13. Pre-1.0: NEVER add a database migration. Migrations are supported, but
|
||||
nothing is installed anywhere yet, so there is nothing to migrate. To
|
||||
change the schema, edit `internal/database/schema/001.sql` (and any
|
||||
code that touches the affected tables) directly. After 1.0, each schema
|
||||
change is a new numbered file in that directory and a released file is
|
||||
never edited; an existing local database is then migrated when vaultik
|
||||
is updated. See
|
||||
[`docs/DATAMODEL.md`](docs/DATAMODEL.md#schema-migrations).
|
||||
|
||||
|
||||
@@ -30,7 +30,7 @@ Read the rules in AGENTS.md and follow them.
|
||||
done provided to you in the initial instruction. Don't do part or most of
|
||||
the work, do all of the work until the criteria for done are met.
|
||||
|
||||
* We do not need to support migrations; schema upgrades can be handled by
|
||||
* We do not add migrations before 1.0; schema upgrades can be handled by
|
||||
deleting the local state file and doing a full backup to re-create it.
|
||||
|
||||
* When testing on a 2.5Gbit/s ethernet to an s3 server backed by 2000MB/sec SSD,
|
||||
|
||||
@@ -559,13 +559,13 @@ complete annotated example also lives in
|
||||
sequentially. Restore speed is bound by single-stream throughput.
|
||||
* **Device nodes, named pipes, and sockets are silently skipped.** Only
|
||||
regular files, directories, and symlinks are backed up.
|
||||
* **No upgrade path between versions.** There is no supported way to carry
|
||||
an existing local index across a schema change; if the local SQLite
|
||||
schema changes between versions, delete the local database (`vaultik
|
||||
database delete`) and run a full backup. Remote storage is unaffected.
|
||||
(The binary does embed numbered schema files and a `schema_migrations`
|
||||
table to bootstrap a fresh database — see [`docs/DATAMODEL.md`](docs/DATAMODEL.md)
|
||||
— but that is not an upgrade path.)
|
||||
* **Before 1.0, an update can make the local index unusable.** Vaultik
|
||||
supports schema migrations, but none are added before 1.0 because
|
||||
there is no installed base yet. If an update leaves your local index
|
||||
unusable, run `vaultik database delete` and then a full backup; remote
|
||||
storage is unaffected. After 1.0, the local index is migrated when
|
||||
vaultik is updated. See
|
||||
[`docs/DATAMODEL.md`](docs/DATAMODEL.md#schema-migrations).
|
||||
* **Files that change during backup may be inconsistent.** There is no
|
||||
filesystem snapshot or freeze. If a file is modified between the scan
|
||||
and chunk phases, the backed-up copy may reflect a partial write.
|
||||
@@ -631,12 +631,11 @@ priority.
|
||||
|
||||
### infrastructure
|
||||
|
||||
* **Cross-version schema upgrades.** There is no upgrade path between
|
||||
released versions — pre-1.0 schema changes are handled by `vaultik
|
||||
database delete` plus a full re-scan (see
|
||||
[`docs/DATAMODEL.md`](docs/DATAMODEL.md)). Post-1.0 we'll need a
|
||||
migration story to keep existing index databases usable across
|
||||
upgrades.
|
||||
* **Schema migrations after 1.0.** Migrations are supported, but none
|
||||
are added before 1.0 because there is no installed base yet. After
|
||||
1.0, each schema change is a new migration, so an existing local
|
||||
index is migrated when vaultik is updated (see
|
||||
[`docs/DATAMODEL.md`](docs/DATAMODEL.md#schema-migrations)).
|
||||
* **Storage backend coverage tests.** S3, file://, and rclone://
|
||||
all share the Storer interface but the rclone path is the least
|
||||
exercised in CI.
|
||||
|
||||
+18
-23
@@ -6,33 +6,28 @@ Vaultik uses a local SQLite database to track file metadata, chunk mappings, and
|
||||
|
||||
**Important Notes:**
|
||||
|
||||
This section is the authoritative explanation of the schema/migration story;
|
||||
other documents (the README and `AGENTS.md`) link here.
|
||||
|
||||
- **No upgrade path between versions (pre-1.0)**: Vaultik has no supported way to
|
||||
carry an existing local index across a schema change. The index is disposable
|
||||
— if the on-disk schema changes between versions, delete the local SQLite
|
||||
database (`vaultik database delete`) and run a full backup. Remote storage is
|
||||
unaffected; the new index re-deduplicates against existing remote blobs. This
|
||||
is the standing project policy, and it is separate from the schema bootstrap
|
||||
described next.
|
||||
- **Schema bootstrap**: a fresh database is populated from numbered SQL files
|
||||
embedded in the binary under `internal/database/schema/`. `000.sql` creates the
|
||||
`schema_migrations` table; `001.sql` creates the application tables. On opening
|
||||
a database the code applies each numbered file that has not yet run and records
|
||||
its version in `schema_migrations`. This bootstraps a new database; it does not
|
||||
upgrade an existing one between released versions.
|
||||
- **Changing the schema (pre-1.0)**: edit `internal/database/schema/001.sql` (and
|
||||
the code that touches the affected tables) directly. Do not add new numbered
|
||||
files — there is no installed base to migrate.
|
||||
- **Disposability expires at 1.0**: the index is treated as disposable only until
|
||||
1.0 ships and is tagged. Once 1.0 is tagged that clause expires and the
|
||||
question of upgrading existing indexes returns. It is deliberately left open
|
||||
here.
|
||||
- **Version Compatibility**: In rare cases, you may need to use the same version
|
||||
of Vaultik to restore a backup as was used to create it. This ensures
|
||||
compatibility with the metadata format stored in S3.
|
||||
|
||||
## Schema Migrations
|
||||
|
||||
Vaultik supports schema migrations. They are the numbered SQL files in
|
||||
`internal/database/schema/`, embedded in the binary: `000.sql` creates the
|
||||
`schema_migrations` table, which records each migration that has run, and
|
||||
`001.sql` creates the application tables. `database.New` opens a database and
|
||||
applies, in order, every migration that database has not yet recorded.
|
||||
|
||||
**Before 1.0** no migrations are added, because nothing is installed anywhere
|
||||
yet. A schema change edits `001.sql` (and the code that uses the affected
|
||||
tables) directly. A local database created before the change has already
|
||||
recorded `001.sql` as run, so it keeps the old schema and can become unusable;
|
||||
`vaultik database delete` followed by a full backup rebuilds it.
|
||||
|
||||
**After 1.0** each schema change is a new numbered file, so an existing local
|
||||
database is migrated the first time an updated vaultik opens it. A file that
|
||||
has shipped in a release is never edited.
|
||||
|
||||
## Database Tables
|
||||
|
||||
### 1. `files`
|
||||
|
||||
Reference in New Issue
Block a user