Move schema_migrations table creation into 000.sql with INTEGER version column #172

Open
clawbot wants to merge 3 commits from feat/integer-schema-migrations into main
Collaborator

Closes #171

Refactors the migration system to follow the pixa pattern:

Changes

New file: internal/database/migrations/000.sql

  • Bootstrap migration that creates schema_migrations with version INTEGER PRIMARY KEY
  • Self-contained: includes both CREATE TABLE IF NOT EXISTS and INSERT OR IGNORE INTO schema_migrations (version) VALUES (0)

Refactored: internal/database/migrations.go

  • Go code no longer creates the migrations table inline — it reads and executes 000.sql as a bootstrap step before the normal migration loop
  • ParseMigrationVersion(filename) — exported function that extracts integer version from filenames like 001_initial.sql1
  • ApplyMigrations(ctx, db, log) — exported function for tests to apply schema without full fx lifecycle
  • bootstrapMigrationsTable — checks if table exists; if missing, runs 000.sql; if present, checks for legacy format
  • convertLegacyMigrations — one-time conversion for existing databases that stored versions as TEXT filenames (e.g. "001_initial.sql") to INTEGER versions (e.g. 1)
  • Transaction-wrapped migration application — each migration runs in a transaction for atomicity
  • Sentinel error ErrInvalidMigrationFilename for static error wrapping

New file: internal/database/migrations_test.go

  • TestParseMigrationVersion — valid/invalid filename parsing
  • TestApplyMigrationsFreshDatabase — verifies bootstrap creates table, all migrations apply, application tables exist
  • TestApplyMigrationsIdempotent — double-apply is a no-op
  • TestApplyMigrationsLegacyConversion — simulates old TEXT-based table with filename entries, verifies conversion to integer format

Backward Compatibility

Existing databases with the old TEXT-based schema_migrations table are automatically converted on first run. The conversion:

  1. Detects legacy entries (versions containing _)
  2. Parses integer version from each legacy filename
  3. Drops the old table
  4. Creates the new INTEGER-based table via 000.sql
  5. Re-inserts all parsed integer versions

All existing migrations (001-007) continue to work unchanged.

Closes #171 Refactors the migration system to follow the [pixa pattern](https://git.eeqj.de/sneak/pixa/pulls/36): ## Changes ### New file: `internal/database/migrations/000.sql` - Bootstrap migration that creates `schema_migrations` with `version INTEGER PRIMARY KEY` - Self-contained: includes both `CREATE TABLE IF NOT EXISTS` and `INSERT OR IGNORE INTO schema_migrations (version) VALUES (0)` ### Refactored: `internal/database/migrations.go` - **Go code no longer creates the migrations table inline** — it reads and executes `000.sql` as a bootstrap step before the normal migration loop - **`ParseMigrationVersion(filename)`** — exported function that extracts integer version from filenames like `001_initial.sql` → `1` - **`ApplyMigrations(ctx, db, log)`** — exported function for tests to apply schema without full fx lifecycle - **`bootstrapMigrationsTable`** — checks if table exists; if missing, runs `000.sql`; if present, checks for legacy format - **`convertLegacyMigrations`** — one-time conversion for existing databases that stored versions as TEXT filenames (e.g. `"001_initial.sql"`) to INTEGER versions (e.g. `1`) - **Transaction-wrapped migration application** — each migration runs in a transaction for atomicity - Sentinel error `ErrInvalidMigrationFilename` for static error wrapping ### New file: `internal/database/migrations_test.go` - `TestParseMigrationVersion` — valid/invalid filename parsing - `TestApplyMigrationsFreshDatabase` — verifies bootstrap creates table, all migrations apply, application tables exist - `TestApplyMigrationsIdempotent` — double-apply is a no-op - `TestApplyMigrationsLegacyConversion` — simulates old TEXT-based table with filename entries, verifies conversion to integer format ## Backward Compatibility Existing databases with the old TEXT-based `schema_migrations` table are automatically converted on first run. The conversion: 1. Detects legacy entries (versions containing `_`) 2. Parses integer version from each legacy filename 3. Drops the old table 4. Creates the new INTEGER-based table via `000.sql` 5. Re-inserts all parsed integer versions All existing migrations (001-007) continue to work unchanged.
clawbot added 1 commit 2026-03-26 14:44:18 +01:00
Move schema_migrations table creation into 000.sql with INTEGER version column
All checks were successful
Check / check (pull_request) Successful in 3m12s
31a6299ce7
Refactors the migration system to follow the pixa pattern:

- Add 000.sql bootstrap migration that creates schema_migrations with
  INTEGER PRIMARY KEY version column
- Go code no longer creates the migrations table inline; it reads and
  executes 000.sql as a bootstrap step before the normal migration loop
- Export ParseMigrationVersion and ApplyMigrations for test use
- Add legacy TEXT-to-INTEGER conversion for existing databases that
  stored migration versions as filenames (e.g. '001_initial.sql')
- Wrap individual migration application in transactions for safety
- Add comprehensive tests for version parsing, fresh database bootstrap,
  idempotent re-application, and legacy conversion
Author
Collaborator

Review: PR #172 — Move schema_migrations table creation into 000.sql

Policy Divergences

1. Bootstrap migration filename violates REPO_POLICIES.md naming convention

REPO_POLICIES.md states:

000_migration.sql — contains ONLY the creation of the migrations tracking table itself. Nothing else.

The PR creates internal/database/migrations/000.sql instead of 000_migration.sql. All other migrations in this repo follow the NNN_description.sql pattern (001_initial.sql, 002_remove_container_id.sql, etc.). The bootstrap migration should be 000_migration.sql to match both the policy and the existing naming convention.

Affected locations that hardcode the filename:

  • internal/database/migrations.goapplyBootstrapMigration(): migrationsFS.ReadFile("migrations/000.sql")
  • internal/database/migrations.gorebuildMigrationsTable(): migrationsFS.ReadFile("migrations/000.sql")
  • internal/database/migrations.gobootstrapMigrationsTable() doc comment references 000.sql

All must be updated to reference 000_migration.sql.

2. Legacy conversion is not transaction-safe

rebuildMigrationsTable() in internal/database/migrations.go performs a destructive sequence (DROP TABLE → CREATE TABLE → INSERT loop) without wrapping it in a transaction. If the process crashes between the DROP and the final INSERT, the user's migration tracking data is lost and the database enters an inconsistent state. The PR already wraps normal migration application in transactions via applyMigrationTx() — the same protection should apply to the legacy conversion path, which is arguably the most dangerous operation in this code.

Requirements Checklist (issue #171)

Requirement Status
schema_migrations table DDL lives in a .sql file, not inline Go code Met
version column is INTEGER PRIMARY KEY (not TEXT) Met
Bootstrap .sql is self-contained (CREATE TABLE + INSERT OR IGNORE) Met
Go code does zero INSERTs for the bootstrap — just reads and executes the SQL file Met

Test Coverage

Exported API Test Status
ParseMigrationVersion() TestParseMigrationVersion Covered
ApplyMigrations() TestApplyMigrationsFreshDatabase, TestApplyMigrationsIdempotent, TestApplyMigrationsLegacyConversion Covered
ErrInvalidMigrationFilename Tested via ParseMigrationVersion error cases Covered

Build Result

docker build .PASS

Cheating Check

No changes to Makefile, Dockerfile, .golangci.yml, or CI config. No weakened assertions.

README Consistency

README does not expose migration implementation details. No update needed.

Verdict: FAIL

  1. The bootstrap migration file must be renamed from 000.sql to 000_migration.sql per REPO_POLICIES.md, and all hardcoded references updated.
  2. rebuildMigrationsTable() must wrap the DROP/CREATE/INSERT sequence in a transaction for data safety.
## Review: [PR #172](https://git.eeqj.de/sneak/upaas/pulls/172) — Move schema_migrations table creation into 000.sql ### Policy Divergences **1. Bootstrap migration filename violates REPO_POLICIES.md naming convention** REPO_POLICIES.md states: > `000_migration.sql` — contains ONLY the creation of the migrations tracking table itself. Nothing else. The PR creates `internal/database/migrations/000.sql` instead of `000_migration.sql`. All other migrations in this repo follow the `NNN_description.sql` pattern (`001_initial.sql`, `002_remove_container_id.sql`, etc.). The bootstrap migration should be `000_migration.sql` to match both the policy and the existing naming convention. Affected locations that hardcode the filename: - `internal/database/migrations.go` — `applyBootstrapMigration()`: `migrationsFS.ReadFile("migrations/000.sql")` - `internal/database/migrations.go` — `rebuildMigrationsTable()`: `migrationsFS.ReadFile("migrations/000.sql")` - `internal/database/migrations.go` — `bootstrapMigrationsTable()` doc comment references `000.sql` All must be updated to reference `000_migration.sql`. **2. Legacy conversion is not transaction-safe** `rebuildMigrationsTable()` in `internal/database/migrations.go` performs a destructive sequence (DROP TABLE → CREATE TABLE → INSERT loop) without wrapping it in a transaction. If the process crashes between the DROP and the final INSERT, the user's migration tracking data is lost and the database enters an inconsistent state. The PR already wraps normal migration application in transactions via `applyMigrationTx()` — the same protection should apply to the legacy conversion path, which is arguably the most dangerous operation in this code. ### Requirements Checklist ([issue #171](https://git.eeqj.de/sneak/upaas/issues/171)) | Requirement | Status | |---|---| | `schema_migrations` table DDL lives in a `.sql` file, not inline Go code | ✅ Met | | `version` column is `INTEGER PRIMARY KEY` (not TEXT) | ✅ Met | | Bootstrap `.sql` is self-contained (CREATE TABLE + INSERT OR IGNORE) | ✅ Met | | Go code does zero INSERTs for the bootstrap — just reads and executes the SQL file | ✅ Met | ### Test Coverage | Exported API | Test | Status | |---|---|---| | `ParseMigrationVersion()` | `TestParseMigrationVersion` | ✅ Covered | | `ApplyMigrations()` | `TestApplyMigrationsFreshDatabase`, `TestApplyMigrationsIdempotent`, `TestApplyMigrationsLegacyConversion` | ✅ Covered | | `ErrInvalidMigrationFilename` | Tested via `ParseMigrationVersion` error cases | ✅ Covered | ### Build Result `docker build .` — **PASS** ✅ ### Cheating Check No changes to Makefile, Dockerfile, `.golangci.yml`, or CI config. No weakened assertions. ✅ ### README Consistency README does not expose migration implementation details. No update needed. ✅ ### Verdict: **FAIL** 1. The bootstrap migration file must be renamed from `000.sql` to `000_migration.sql` per REPO_POLICIES.md, and all hardcoded references updated. 2. `rebuildMigrationsTable()` must wrap the DROP/CREATE/INSERT sequence in a transaction for data safety.
clawbot added the needs-rework label 2026-03-26 14:51:59 +01:00
clawbot added 1 commit 2026-03-26 14:57:08 +01:00
fix: rename 000.sql to 000_migration.sql and wrap legacy conversion in transaction
All checks were successful
Check / check (pull_request) Successful in 3m14s
22cc1a7fb1
- Rename bootstrap migration from 000.sql to 000_migration.sql per
  REPO_POLICIES.md naming convention (NNN_description.sql pattern)
- Update all hardcoded references in migrations.go
- Wrap rebuildMigrationsTable() DROP/CREATE/INSERT sequence in a
  single transaction for crash safety
- Update test case filename to match renamed file
Author
Collaborator

Rework Summary

Fixed both review findings from PR #172:

1. Bootstrap migration renamed to 000_migration.sql

Renamed internal/database/migrations/000.sqlinternal/database/migrations/000_migration.sql per REPO_POLICIES.md naming convention (NNN_description.sql pattern). Updated all hardcoded references:

  • applyBootstrapMigration() — ReadFile path and error messages
  • rebuildMigrationsTable() — ReadFile path and error messages
  • bootstrapMigrationsTable() — doc comment
  • Test case in TestParseMigrationVersion updated from 000.sql to 000_migration.sql

2. Legacy conversion wrapped in transaction

rebuildMigrationsTable() now wraps the entire DROP TABLE → CREATE TABLE → INSERT loop in a single transaction (db.BeginTx / tx.Commit). If any step fails or the process crashes mid-conversion, the transaction rolls back and no data is lost. The file read (migrationsFS.ReadFile) is done before the transaction begins so the transaction window is minimal.

Build Result

docker build .PASS (all tests pass, lint clean, binary builds)

## Rework Summary Fixed both review findings from [PR #172](https://git.eeqj.de/sneak/upaas/pulls/172): ### 1. Bootstrap migration renamed to `000_migration.sql` Renamed `internal/database/migrations/000.sql` → `internal/database/migrations/000_migration.sql` per REPO_POLICIES.md naming convention (`NNN_description.sql` pattern). Updated all hardcoded references: - `applyBootstrapMigration()` — ReadFile path and error messages - `rebuildMigrationsTable()` — ReadFile path and error messages - `bootstrapMigrationsTable()` — doc comment - Test case in `TestParseMigrationVersion` updated from `000.sql` to `000_migration.sql` ### 2. Legacy conversion wrapped in transaction `rebuildMigrationsTable()` now wraps the entire DROP TABLE → CREATE TABLE → INSERT loop in a single transaction (`db.BeginTx` / `tx.Commit`). If any step fails or the process crashes mid-conversion, the transaction rolls back and no data is lost. The file read (`migrationsFS.ReadFile`) is done before the transaction begins so the transaction window is minimal. ### Build Result `docker build .` — **PASS** ✅ (all tests pass, lint clean, binary builds)
clawbot added needs-review and removed needs-rework labels 2026-03-26 14:57:52 +01:00
Owner

@clawbot remove the old-style table conversion - this is pre-1.0 software, no schema changes require any migrations. there is no installed base.

@clawbot remove the old-style table conversion - this is pre-1.0 software, no schema changes require any migrations. there is no installed base.
clawbot added needs-rework and removed needs-review labels 2026-03-26 16:34:58 +01:00
clawbot added 1 commit 2026-03-26 16:39:42 +01:00
remove legacy migration conversion code
All checks were successful
Check / check (pull_request) Successful in 3m15s
6591f77dac
Pre-1.0 software with no installed base — no need to handle
converting from old TEXT-based schema_migrations format.

Removed: convertLegacyMigrations, ensureBootstrapVersion,
readLegacyVersions, rebuildMigrationsTable, and
TestApplyMigrationsLegacyConversion.

Simplified bootstrapMigrationsTable to only check existence
and run 000_migration.sql if missing.
Author
Collaborator

Rework Summary — Remove Legacy Conversion

Per sneak's instruction, removed all legacy/old-style migration table conversion code. This is pre-1.0 software with no installed base — no need to handle converting from old TEXT-based format.

Removed

  • convertLegacyMigrations() — detected legacy TEXT filename entries and orchestrated conversion
  • ensureBootstrapVersion() — inserted version 0 if missing (only used in legacy path)
  • readLegacyVersions() — read and parsed legacy TEXT version entries
  • rebuildMigrationsTable() — transactional DROP/CREATE/INSERT for table rebuild
  • TestApplyMigrationsLegacyConversion — test that simulated old TEXT-based table and verified conversion

Simplified

  • bootstrapMigrationsTable() — now just checks if table exists; if missing, runs 000_migration.sql. If table exists, returns nil. No legacy format detection.

Net change: -213 lines, +2 lines

Build Result

docker build .PASS (all tests pass, lint clean, binary builds)

README

No references to legacy conversion in README. No update needed.

## Rework Summary — Remove Legacy Conversion Per [sneak's instruction](https://git.eeqj.de/sneak/upaas/pulls/172#issuecomment-1323), removed all legacy/old-style migration table conversion code. This is pre-1.0 software with no installed base — no need to handle converting from old TEXT-based format. ### Removed - **`convertLegacyMigrations()`** — detected legacy TEXT filename entries and orchestrated conversion - **`ensureBootstrapVersion()`** — inserted version 0 if missing (only used in legacy path) - **`readLegacyVersions()`** — read and parsed legacy TEXT version entries - **`rebuildMigrationsTable()`** — transactional DROP/CREATE/INSERT for table rebuild - **`TestApplyMigrationsLegacyConversion`** — test that simulated old TEXT-based table and verified conversion ### Simplified - **`bootstrapMigrationsTable()`** — now just checks if table exists; if missing, runs `000_migration.sql`. If table exists, returns nil. No legacy format detection. ### Net change: -213 lines, +2 lines ### Build Result `docker build .` — **PASS** ✅ (all tests pass, lint clean, binary builds) ### README No references to legacy conversion in README. No update needed. ✅
clawbot added needs-review and removed needs-rework labels 2026-03-26 16:40:22 +01:00
clawbot reviewed 2026-03-26 16:43:39 +01:00
clawbot left a comment
Author
Collaborator

Re-Review: PR #172 — Move schema_migrations table creation into 000.sql

Third review pass after legacy conversion code removal per sneak's instruction.

Requirements Checklist (issue #171)

Requirement Status Evidence
schema_migrations DDL in .sql file, not inline Go Met internal/database/migrations/000_migration.sql contains DDL; Go code just reads and executes it via applyBootstrapMigration()
version column is INTEGER PRIMARY KEY Met 000_migration.sql line 4: version INTEGER PRIMARY KEY
000_migration.sql is self-contained (CREATE TABLE + INSERT OR IGNORE) Met File contains both CREATE TABLE IF NOT EXISTS and INSERT OR IGNORE INTO schema_migrations (version) VALUES (0)
Go code does zero INSERTs for bootstrap Met applyBootstrapMigration() only does db.ExecContext(ctx, string(content)) — no Go-side INSERT statements

Human Comment Compliance

  • sneak requested: remove all legacy conversion code (pre-1.0, no installed base) → Fully addressed. All legacy functions (convertLegacyMigrations, ensureBootstrapVersion, readLegacyVersions, rebuildMigrationsTable) and TestApplyMigrationsLegacyConversion are gone. bootstrapMigrationsTable() is now a clean check-and-create.

Policy Compliance

No violations found in the PR diff.

  • Filename 000_migration.sql matches REPO_POLICIES NNN_description.sql pattern
  • No changes to .golangci.yml, Makefile, Dockerfile, or CI config
  • All external dependencies unchanged (no new imports beyond stdlib + existing deps)
  • Error wrapping uses %w with sentinel ErrInvalidMigrationFilename
  • Transaction safety: each migration applied via applyMigrationTx() with proper defer/rollback
  • Bootstrap uses idempotent SQL (IF NOT EXISTS + OR IGNORE), safe without explicit tx

Test Coverage

Exported Symbol Test(s) Status
ParseMigrationVersion() TestParseMigrationVersion (5 valid, 4 invalid cases)
ApplyMigrations() TestApplyMigrationsFreshDatabase, TestApplyMigrationsIdempotent
ErrInvalidMigrationFilename Tested via invalid parse cases (asserts error return)

Cheating Check

  • No modified linter config, Makefile, Dockerfile, or CI workflows
  • No weakened test assertions
  • Only 3 files changed, all in internal/database/

README Consistency

README does not expose migration internals. No update needed.

Build Result

docker build .PASS

Code Quality Notes

  • Clean separation: bootstrapMigrationsTablecollectMigrations → per-migration loop
  • ApplyMigrations exported for test use without full fx lifecycle — good design
  • ParseMigrationVersion handles edge cases (empty, no extension, non-numeric, underscore-prefixed)
  • migrate() method delegates cleanly to ApplyMigrations(ctx, d.database, d.log)
  • Net change is well-structured: -48 old lines, +272 new lines across 3 files

Verdict: PASS

All four issue requirements implemented correctly. Sneak's rework request fully addressed. Code is clean, tested, and builds successfully. No policy violations.

## Re-Review: [PR #172](https://git.eeqj.de/sneak/upaas/pulls/172) — Move schema_migrations table creation into 000.sql Third review pass after legacy conversion code removal per [sneak's instruction](https://git.eeqj.de/sneak/upaas/pulls/172#issuecomment-1323). ### Requirements Checklist ([issue #171](https://git.eeqj.de/sneak/upaas/issues/171)) | Requirement | Status | Evidence | |---|---|---| | `schema_migrations` DDL in `.sql` file, not inline Go | ✅ Met | `internal/database/migrations/000_migration.sql` contains DDL; Go code just reads and executes it via `applyBootstrapMigration()` | | `version` column is `INTEGER PRIMARY KEY` | ✅ Met | `000_migration.sql` line 4: `version INTEGER PRIMARY KEY` | | `000_migration.sql` is self-contained (CREATE TABLE + INSERT OR IGNORE) | ✅ Met | File contains both `CREATE TABLE IF NOT EXISTS` and `INSERT OR IGNORE INTO schema_migrations (version) VALUES (0)` | | Go code does zero INSERTs for bootstrap | ✅ Met | `applyBootstrapMigration()` only does `db.ExecContext(ctx, string(content))` — no Go-side INSERT statements | ### Human Comment Compliance - [sneak requested](https://git.eeqj.de/sneak/upaas/pulls/172#issuecomment-1323): remove all legacy conversion code (pre-1.0, no installed base) → ✅ **Fully addressed.** All legacy functions (`convertLegacyMigrations`, `ensureBootstrapVersion`, `readLegacyVersions`, `rebuildMigrationsTable`) and `TestApplyMigrationsLegacyConversion` are gone. `bootstrapMigrationsTable()` is now a clean check-and-create. ### Policy Compliance No violations found in the PR diff. - Filename `000_migration.sql` matches REPO_POLICIES `NNN_description.sql` pattern ✅ - No changes to `.golangci.yml`, `Makefile`, `Dockerfile`, or CI config ✅ - All external dependencies unchanged (no new imports beyond stdlib + existing deps) ✅ - Error wrapping uses `%w` with sentinel `ErrInvalidMigrationFilename` ✅ - Transaction safety: each migration applied via `applyMigrationTx()` with proper defer/rollback ✅ - Bootstrap uses idempotent SQL (`IF NOT EXISTS` + `OR IGNORE`), safe without explicit tx ✅ ### Test Coverage | Exported Symbol | Test(s) | Status | |---|---|---| | `ParseMigrationVersion()` | `TestParseMigrationVersion` (5 valid, 4 invalid cases) | ✅ | | `ApplyMigrations()` | `TestApplyMigrationsFreshDatabase`, `TestApplyMigrationsIdempotent` | ✅ | | `ErrInvalidMigrationFilename` | Tested via invalid parse cases (asserts `error` return) | ✅ | ### Cheating Check - No modified linter config, Makefile, Dockerfile, or CI workflows ✅ - No weakened test assertions ✅ - Only 3 files changed, all in `internal/database/` ✅ ### README Consistency README does not expose migration internals. No update needed. ✅ ### Build Result `docker build .` — **PASS** ✅ ### Code Quality Notes - Clean separation: `bootstrapMigrationsTable` → `collectMigrations` → per-migration loop - `ApplyMigrations` exported for test use without full fx lifecycle — good design - `ParseMigrationVersion` handles edge cases (empty, no extension, non-numeric, underscore-prefixed) - `migrate()` method delegates cleanly to `ApplyMigrations(ctx, d.database, d.log)` - Net change is well-structured: -48 old lines, +272 new lines across 3 files ### Verdict: **PASS** ✅ All four issue requirements implemented correctly. Sneak's rework request fully addressed. Code is clean, tested, and builds successfully. No policy violations.
clawbot added merge-ready and removed needs-review labels 2026-03-26 16:44:11 +01:00
sneak was assigned by clawbot 2026-03-26 16:44:11 +01:00
All checks were successful
Check / check (pull_request) Successful in 3m15s
Required
Details
This pull request can be merged automatically.
You are not authorized to merge this pull request.
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u origin feat/integer-schema-migrations:feat/integer-schema-migrations
git checkout feat/integer-schema-migrations
Sign in to join this conversation.