Document that migrations are supported and none are added before 1.0 (closes #68)
check / check (pull_request) Successful in 3m17s

#146 called the numbered schema
files a bootstrap and said vaultik has no upgrade path between versions.
That was wrong: they are migrations, and database.New applies any the
database has not recorded. None are added before 1.0 because nothing is
installed anywhere yet; after 1.0 each schema change is a new numbered
file and the local database is migrated on update.

docs/DATAMODEL.md gets a Schema Migrations section that owns the
explanation. The README caveat and roadmap entry and AGENTS.md policy 13
say the same and link to it. CLAUDE.md, the owner's file, gets a
one-line edit so it no longer says migrations are not needed.

Model: opus-5-5
This commit is contained in:
2026-09-28 18:09:48 +00:00
parent d24f5dc33c
commit 714eea086b
4 changed files with 39 additions and 47 deletions
+8 -10
View File
@@ -102,14 +102,12 @@ Version: 2025-06-08
build files are acceptable in the root, but source code and other files build files are acceptable in the root, but source code and other files
should be organized in appropriate subdirectories. should be organized in appropriate subdirectories.
13. Pre-1.0: NEVER write database migrations. There are no live databases 13. Pre-1.0: NEVER add a database migration. Migrations are supported, but
anywhere — every user's local index can be rebuilt from a fresh full nothing is installed anywhere yet, so there is nothing to migrate. To
backup. To change the schema, edit `internal/database/schema/001.sql` change the schema, edit `internal/database/schema/001.sql` (and any
(and any code that touches the affected tables) directly; do not add new code that touches the affected tables) directly. After 1.0, each schema
numbered schema files. Those numbered files and the `schema_migrations` change is a new numbered file in that directory and a released file is
table they populate only bootstrap a fresh database — they are not an never edited; an existing local database is then migrated when vaultik
upgrade path. The local index is disposable until 1.0 ships and is is updated. See
tagged; once 1.0 is tagged that clause expires and the question of [`docs/DATAMODEL.md`](docs/DATAMODEL.md#schema-migrations).
upgrading existing indexes returns. See [`docs/DATAMODEL.md`](docs/DATAMODEL.md)
for the full explanation.
+1 -1
View File
@@ -30,7 +30,7 @@ Read the rules in AGENTS.md and follow them.
done provided to you in the initial instruction. Don't do part or most of done provided to you in the initial instruction. Don't do part or most of
the work, do all of the work until the criteria for done are met. the work, do all of the work until the criteria for done are met.
* We do not need to support migrations; schema upgrades can be handled by * We do not add migrations before 1.0; schema upgrades can be handled by
deleting the local state file and doing a full backup to re-create it. deleting the local state file and doing a full backup to re-create it.
* When testing on a 2.5Gbit/s ethernet to an s3 server backed by 2000MB/sec SSD, * When testing on a 2.5Gbit/s ethernet to an s3 server backed by 2000MB/sec SSD,
+12 -13
View File
@@ -559,13 +559,13 @@ complete annotated example also lives in
sequentially. Restore speed is bound by single-stream throughput. sequentially. Restore speed is bound by single-stream throughput.
* **Device nodes, named pipes, and sockets are silently skipped.** Only * **Device nodes, named pipes, and sockets are silently skipped.** Only
regular files, directories, and symlinks are backed up. regular files, directories, and symlinks are backed up.
* **No upgrade path between versions.** There is no supported way to carry * **Before 1.0, an update can make the local index unusable.** Vaultik
an existing local index across a schema change; if the local SQLite supports schema migrations, but none are added before 1.0 because
schema changes between versions, delete the local database (`vaultik there is no installed base yet. If an update leaves your local index
database delete`) and run a full backup. Remote storage is unaffected. unusable, run `vaultik database delete` and then a full backup; remote
(The binary does embed numbered schema files and a `schema_migrations` storage is unaffected. After 1.0, the local index is migrated when
table to bootstrap a fresh database — see [`docs/DATAMODEL.md`](docs/DATAMODEL.md) vaultik is updated. See
— but that is not an upgrade path.) [`docs/DATAMODEL.md`](docs/DATAMODEL.md#schema-migrations).
* **Files that change during backup may be inconsistent.** There is no * **Files that change during backup may be inconsistent.** There is no
filesystem snapshot or freeze. If a file is modified between the scan filesystem snapshot or freeze. If a file is modified between the scan
and chunk phases, the backed-up copy may reflect a partial write. and chunk phases, the backed-up copy may reflect a partial write.
@@ -631,12 +631,11 @@ priority.
### infrastructure ### infrastructure
* **Cross-version schema upgrades.** There is no upgrade path between * **Schema migrations after 1.0.** Migrations are supported, but none
released versions — pre-1.0 schema changes are handled by `vaultik are added before 1.0 because there is no installed base yet. After
database delete` plus a full re-scan (see 1.0, each schema change is a new migration, so an existing local
[`docs/DATAMODEL.md`](docs/DATAMODEL.md)). Post-1.0 we'll need a index is migrated when vaultik is updated (see
migration story to keep existing index databases usable across [`docs/DATAMODEL.md`](docs/DATAMODEL.md#schema-migrations)).
upgrades.
* **Storage backend coverage tests.** S3, file://, and rclone:// * **Storage backend coverage tests.** S3, file://, and rclone://
all share the Storer interface but the rclone path is the least all share the Storer interface but the rclone path is the least
exercised in CI. exercised in CI.
+18 -23
View File
@@ -6,33 +6,28 @@ Vaultik uses a local SQLite database to track file metadata, chunk mappings, and
**Important Notes:** **Important Notes:**
This section is the authoritative explanation of the schema/migration story;
other documents (the README and `AGENTS.md`) link here.
- **No upgrade path between versions (pre-1.0)**: Vaultik has no supported way to
carry an existing local index across a schema change. The index is disposable
— if the on-disk schema changes between versions, delete the local SQLite
database (`vaultik database delete`) and run a full backup. Remote storage is
unaffected; the new index re-deduplicates against existing remote blobs. This
is the standing project policy, and it is separate from the schema bootstrap
described next.
- **Schema bootstrap**: a fresh database is populated from numbered SQL files
embedded in the binary under `internal/database/schema/`. `000.sql` creates the
`schema_migrations` table; `001.sql` creates the application tables. On opening
a database the code applies each numbered file that has not yet run and records
its version in `schema_migrations`. This bootstraps a new database; it does not
upgrade an existing one between released versions.
- **Changing the schema (pre-1.0)**: edit `internal/database/schema/001.sql` (and
the code that touches the affected tables) directly. Do not add new numbered
files — there is no installed base to migrate.
- **Disposability expires at 1.0**: the index is treated as disposable only until
1.0 ships and is tagged. Once 1.0 is tagged that clause expires and the
question of upgrading existing indexes returns. It is deliberately left open
here.
- **Version Compatibility**: In rare cases, you may need to use the same version - **Version Compatibility**: In rare cases, you may need to use the same version
of Vaultik to restore a backup as was used to create it. This ensures of Vaultik to restore a backup as was used to create it. This ensures
compatibility with the metadata format stored in S3. compatibility with the metadata format stored in S3.
## Schema Migrations
Vaultik supports schema migrations. They are the numbered SQL files in
`internal/database/schema/`, embedded in the binary: `000.sql` creates the
`schema_migrations` table, which records each migration that has run, and
`001.sql` creates the application tables. `database.New` opens a database and
applies, in order, every migration that database has not yet recorded.
**Before 1.0** no migrations are added, because nothing is installed anywhere
yet. A schema change edits `001.sql` (and the code that uses the affected
tables) directly. A local database created before the change has already
recorded `001.sql` as run, so it keeps the old schema and can become unusable;
`vaultik database delete` followed by a full backup rebuilds it.
**After 1.0** each schema change is a new numbered file, so an existing local
database is migrated the first time an updated vaultik opens it. A file that
has shipped in a release is never edited.
## Database Tables ## Database Tables
### 1. `files` ### 1. `files`