Archive database target: rotation setting puts year-month, day or hour in the filename #379

Open
opened 2026-10-01 21:25:25 +02:00 by clawbot · 1 comment
Collaborator

Owner's words (chat, 2026-10-01 ~19:25 UTC):

there should be a flag on an archive database that causes it to put either the year and month, or year and month and day, or year and month and day and hour in the filename (to support automatic rotation)

PRIORITY: an owner's direct request, in the tier of #367 to #377. Builds on #376 (one archive file per database target, named for webhook and target). Sequence it after 376.

Definition of done:

  • Each database (archive) target has a rotation setting with four choices: none, monthly, daily and hourly. Monthly puts YYYY-MM in the filename, daily YYYY-MM-DD and hourly YYYY-MM-DD-HH. When the period changes, new events go into a new file, so old files rotate out naturally. "None" keeps a single file.
  • The setting is on the target's add and edit forms (see #370 and #373) and is shown in the target list.
  • The plan comment on this issue states, before implementation:
    • which clock decides the period. Recommendation: UTC, by the event's receive time, so a file holds exactly its period's events;
    • how this interacts with archive pruning (373). Recommendation: pruning still removes rows by age, and a rotated file left empty is deleted;
    • what Download (#374) exports for a rotated target: the current file, a chosen period, or all files;
    • what happens when the setting is changed on an existing target.
  • Pre-1.0: no migration or compatibility for existing files.
  • Tests cover each choice across a period boundary, using an injected clock, not wall-clock waits. Lands on next with an independent review.

model: opus-5-5

Owner's words (chat, 2026-10-01 ~19:25 UTC): > there should be a flag on an archive database that causes it to put either the year and month, or year and month and day, or year and month and day and hour in the filename (to support automatic rotation) PRIORITY: an owner's direct request, in the tier of https://git.eeqj.de/sneak/webhooker/issues/367 to https://git.eeqj.de/sneak/webhooker/issues/377. Builds on https://git.eeqj.de/sneak/webhooker/issues/376 (one archive file per database target, named for webhook and target). Sequence it after 376. Definition of done: - Each database (archive) target has a rotation setting with four choices: none, monthly, daily and hourly. Monthly puts `YYYY-MM` in the filename, daily `YYYY-MM-DD` and hourly `YYYY-MM-DD-HH`. When the period changes, new events go into a new file, so old files rotate out naturally. "None" keeps a single file. - The setting is on the target's add and edit forms (see https://git.eeqj.de/sneak/webhooker/issues/370 and https://git.eeqj.de/sneak/webhooker/issues/373) and is shown in the target list. - The plan comment on this issue states, before implementation: - which clock decides the period. Recommendation: UTC, by the event's receive time, so a file holds exactly its period's events; - how this interacts with archive pruning (373). Recommendation: pruning still removes rows by age, and a rotated file left empty is deleted; - what Download (https://git.eeqj.de/sneak/webhooker/issues/374) exports for a rotated target: the current file, a chosen period, or all files; - what happens when the setting is changed on an existing target. - Pre-1.0: no migration or compatibility for existing files. - Tests cover each choice across a period boundary, using an injected clock, not wall-clock waits. Lands on `next` with an independent review. model: opus-5-5
clawbot self-assigned this 2026-10-01 21:25:25 +02:00
Author
Collaborator

Plan.

  • Setting: rotation in the database target's config, beside expiry: none (the default), monthly, daily or hourly. No schema change. It is on the target's add and edit forms and shown in the target list.
  • Filename: #376's name with the period appended: archive-WEBHOOKNAME-TARGETNAME-TARGETID-2026-10.db, -2026-10-01.db or -2026-10-01-19.db. none keeps the single file from 376.
  • Clock: UTC, by the event's receive time, so each file holds exactly its period's events.
  • Pruning: expiry still removes rows by age, in every one of the target's files; a rotated file that pruning leaves empty is deleted.
  • Download (#374): exports all of the target's files as one export, oldest period first, each row carrying the period it came from.
  • Changing the setting: from the next write, new events go to the file the new setting names. Existing files keep their names and stay, still pruned by age and included in Download. Renaming the webhook or the target renames all of its files.
  • Tests cross each period boundary with an injected clock.
  • Sequencing: after 376 (one file per target and its naming), #370 (the add form) and 374 (Download).

Model: opus-5-5

Plan. - **Setting:** `rotation` in the `database` target's config, beside `expiry`: `none` (the default), `monthly`, `daily` or `hourly`. No schema change. It is on the target's add and edit forms and shown in the target list. - **Filename:** https://git.eeqj.de/sneak/webhooker/issues/376's name with the period appended: `archive-WEBHOOKNAME-TARGETNAME-TARGETID-2026-10.db`, `-2026-10-01.db` or `-2026-10-01-19.db`. `none` keeps the single file from 376. - **Clock:** UTC, by the event's receive time, so each file holds exactly its period's events. - **Pruning:** `expiry` still removes rows by age, in every one of the target's files; a rotated file that pruning leaves empty is deleted. - **Download** (https://git.eeqj.de/sneak/webhooker/issues/374): exports all of the target's files as one export, oldest period first, each row carrying the period it came from. - **Changing the setting:** from the next write, new events go to the file the new setting names. Existing files keep their names and stay, still pruned by age and included in Download. Renaming the webhook or the target renames all of its files. - Tests cross each period boundary with an injected clock. - **Sequencing:** after 376 (one file per target and its naming), https://git.eeqj.de/sneak/webhooker/issues/370 (the add form) and 374 (Download). Model: opus-5-5
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sneak/webhooker#379