3 Commits

Author SHA1 Message Date
clawbot
d9dbadaf62 refactor: 30-day defaults, time-based message expiry
All checks were successful
check / check (push) Successful in 5s
- Change QUEUE_MAX_AGE default from 48h to 30 days (2592000s)
- Replace count-based MAX_HISTORY with time-based MESSAGE_MAX_AGE
  (default 30 days) — messages older than the configured age are
  pruned instead of keeping a fixed count per target
- Update README to reflect new defaults and time-based expiry
2026-03-10 03:47:37 -07:00
clawbot
5f05b70be7 fix: remove PruneOrphanedMessages to preserve history within MAX_HISTORY
PruneOrphanedMessages deleted messages that lost their client_queues
references after PruneOldQueueEntries ran, even when those messages
were within the MAX_HISTORY limit. This made MAX_HISTORY meaningless
for low-traffic channels.

RotateChannelMessages already caps messages per target. Queue pruning
handles client_queues growth. Orphan cleanup is redundant.

Closes #40
2026-03-10 03:47:37 -07:00
user
b452c915cc feat: implement queue pruning and message rotation
Enforce QUEUE_MAX_AGE and MAX_HISTORY config values that previously
existed but were not applied. The existing cleanup loop now also:

- Prunes client_queues entries older than QUEUE_MAX_AGE (default 48h)
- Rotates messages per target (channel/DM) beyond MAX_HISTORY (default 10000)
- Removes orphaned messages no longer referenced by any client queue

closes #40
2026-03-10 03:47:37 -07:00
13 changed files with 122 additions and 69 deletions

View File

@@ -249,8 +249,8 @@ Key properties:
- **Ordered**: Queue entries have monotonically increasing IDs. Messages are - **Ordered**: Queue entries have monotonically increasing IDs. Messages are
always delivered in order within a client's queue. always delivered in order within a client's queue.
- **No delivery/read receipts** for channel messages. DM receipts are planned. - **No delivery/read receipts** for channel messages. DM receipts are planned.
- **Queue depth**: Server-configurable via `QUEUE_MAX_AGE`. Default is 48 - **Queue depth**: Server-configurable via `QUEUE_MAX_AGE`. Default is 30
hours. Entries older than this are pruned. days. Entries older than this are pruned.
### Long-Polling ### Long-Polling
@@ -1788,10 +1788,10 @@ skew issues) and simpler than UUIDs (integer comparison vs. string comparison).
### Data Lifecycle ### Data Lifecycle
- **Messages**: Stored indefinitely in the current implementation. Rotation - **Messages**: Pruned automatically when older than `MESSAGE_MAX_AGE`
per `MAX_HISTORY` is planned. (default 30 days).
- **Queue entries**: Stored until pruned. Pruning by `QUEUE_MAX_AGE` is - **Queue entries**: Pruned automatically when older than `QUEUE_MAX_AGE`
planned. (default 30 days).
- **Channels**: Deleted when the last member leaves (ephemeral). - **Channels**: Deleted when the last member leaves (ephemeral).
- **Users/sessions**: Deleted on `QUIT` or `POST /api/v1/logout`. Idle - **Users/sessions**: Deleted on `QUIT` or `POST /api/v1/logout`. Idle
sessions are automatically expired after `SESSION_IDLE_TIMEOUT` (default sessions are automatically expired after `SESSION_IDLE_TIMEOUT` (default
@@ -1812,9 +1812,9 @@ directory is also loaded automatically via
| `PORT` | int | `8080` | HTTP listen port | | `PORT` | int | `8080` | HTTP listen port |
| `DBURL` | string | `file:///var/lib/neoirc/state.db?_journal_mode=WAL` | SQLite connection string. For file-based: `file:///path/to/db.db?_journal_mode=WAL`. For in-memory (testing): `file::memory:?cache=shared`. | | `DBURL` | string | `file:///var/lib/neoirc/state.db?_journal_mode=WAL` | SQLite connection string. For file-based: `file:///path/to/db.db?_journal_mode=WAL`. For in-memory (testing): `file::memory:?cache=shared`. |
| `DEBUG` | bool | `false` | Enable debug logging (verbose request/response logging) | | `DEBUG` | bool | `false` | Enable debug logging (verbose request/response logging) |
| `MAX_HISTORY` | int | `10000` | Maximum messages retained per channel before rotation (planned) | | `MESSAGE_MAX_AGE` | int | `2592000` | Maximum age of messages in seconds (30 days). Messages older than this are pruned. |
| `SESSION_IDLE_TIMEOUT` | string | `24h` | Session idle timeout as a Go duration string (e.g. `24h`, `30m`). Sessions with no activity for this long are expired and the nick is released. | | `SESSION_IDLE_TIMEOUT` | string | `24h` | Session idle timeout as a Go duration string (e.g. `24h`, `30m`). Sessions with no activity for this long are expired and the nick is released. |
| `QUEUE_MAX_AGE` | int | `172800` | Maximum age of client queue entries in seconds (48h). Entries older than this are pruned (planned). | | `QUEUE_MAX_AGE` | int | `2592000` | Maximum age of client queue entries in seconds (30 days). Entries older than this are pruned. |
| `MAX_MESSAGE_SIZE` | int | `4096` | Maximum message body size in bytes (planned enforcement) | | `MAX_MESSAGE_SIZE` | int | `4096` | Maximum message body size in bytes (planned enforcement) |
| `LONG_POLL_TIMEOUT`| int | `15` | Default long-poll timeout in seconds (client can override via query param, server caps at 30) | | `LONG_POLL_TIMEOUT`| int | `15` | Default long-poll timeout in seconds (client can override via query param, server caps at 30) |
| `MOTD` | string | `""` | Message of the day, shown to clients via `GET /api/v1/server` | | `MOTD` | string | `""` | Message of the day, shown to clients via `GET /api/v1/server` |
@@ -2228,8 +2228,8 @@ GET /api/v1/challenge
### Post-MVP (Planned) ### Post-MVP (Planned)
- [ ] **Hashcash proof-of-work** for session creation (abuse prevention) - [ ] **Hashcash proof-of-work** for session creation (abuse prevention)
- [ ] **Queue pruning** — delete old queue entries per `QUEUE_MAX_AGE` - [x] **Queue pruning** — delete old queue entries per `QUEUE_MAX_AGE`
- [ ] **Message rotation** — enforce `MAX_HISTORY` per channel - [x] **Message expiry** — delete messages older than `MESSAGE_MAX_AGE`
- [ ] **Channel modes** — enforce `+i`, `+m`, `+s`, `+t`, `+n` - [ ] **Channel modes** — enforce `+i`, `+m`, `+s`, `+t`, `+n`
- [ ] **User channel modes** — `+o` (operator), `+v` (voice) - [ ] **User channel modes** — `+o` (operator), `+v` (voice)
- [x] **MODE command** — query channel and user modes (set not yet implemented) - [x] **MODE command** — query channel and user modes (set not yet implemented)
@@ -2336,7 +2336,7 @@ neoirc/
| Purpose | Library | | Purpose | Library |
|------------|---------| |------------|---------|
| DI | `go.uber.org/fx` | | DI | `go.uber.org/fx` |
| Router | `github.com/go-chi/chi/v5` | | Router | `github.com/go-chi/chi` |
| Logging | `log/slog` (stdlib) | | Logging | `log/slog` (stdlib) |
| Config | `github.com/spf13/viper` | | Config | `github.com/spf13/viper` |
| Env | `github.com/joho/godotenv/autoload` | | Env | `github.com/joho/godotenv/autoload` |

2
go.mod
View File

@@ -6,7 +6,7 @@ require (
github.com/99designs/basicauth-go v0.0.0-20230316000542-bf6f9cbbf0f8 github.com/99designs/basicauth-go v0.0.0-20230316000542-bf6f9cbbf0f8
github.com/gdamore/tcell/v2 v2.13.8 github.com/gdamore/tcell/v2 v2.13.8
github.com/getsentry/sentry-go v0.42.0 github.com/getsentry/sentry-go v0.42.0
github.com/go-chi/chi/v5 v5.2.1 github.com/go-chi/chi v1.5.5
github.com/go-chi/cors v1.2.2 github.com/go-chi/cors v1.2.2
github.com/google/uuid v1.6.0 github.com/google/uuid v1.6.0
github.com/joho/godotenv v1.5.1 github.com/joho/godotenv v1.5.1

4
go.sum
View File

@@ -18,8 +18,8 @@ github.com/gdamore/tcell/v2 v2.13.8 h1:Mys/Kl5wfC/GcC5Cx4C2BIQH9dbnhnkPgS9/wF3Rl
github.com/gdamore/tcell/v2 v2.13.8/go.mod h1:+Wfe208WDdB7INEtCsNrAN6O2m+wsTPk1RAovjaILlo= github.com/gdamore/tcell/v2 v2.13.8/go.mod h1:+Wfe208WDdB7INEtCsNrAN6O2m+wsTPk1RAovjaILlo=
github.com/getsentry/sentry-go v0.42.0 h1:eeFMACuZTbUQf90RE8dE4tXeSe4CZyfvR1MBL7RLEt8= github.com/getsentry/sentry-go v0.42.0 h1:eeFMACuZTbUQf90RE8dE4tXeSe4CZyfvR1MBL7RLEt8=
github.com/getsentry/sentry-go v0.42.0/go.mod h1:eRXCoh3uvmjQLY6qu63BjUZnaBu5L5WhMV1RwYO8W5s= github.com/getsentry/sentry-go v0.42.0/go.mod h1:eRXCoh3uvmjQLY6qu63BjUZnaBu5L5WhMV1RwYO8W5s=
github.com/go-chi/chi/v5 v5.2.1 h1:KOIHODQj58PmL80G2Eak4WdvUzjSJSm0vG72crDCqb8= github.com/go-chi/chi v1.5.5 h1:vOB/HbEMt9QqBqErz07QehcOKHaWFtuj87tTDVz2qXE=
github.com/go-chi/chi/v5 v5.2.1/go.mod h1:L2yAIGWB3H+phAw1NxKwWM+7eUH/lU8pOMm5hHcoops= github.com/go-chi/chi v1.5.5/go.mod h1:C9JqLr3tIYjDOZpzn+BCuxY8z8vmca43EeMgyZt7irw=
github.com/go-chi/cors v1.2.2 h1:Jmey33TE+b+rB7fT8MUy1u0I4L+NARQlK6LhzKPSyQE= github.com/go-chi/cors v1.2.2 h1:Jmey33TE+b+rB7fT8MUy1u0I4L+NARQlK6LhzKPSyQE=
github.com/go-chi/cors v1.2.2/go.mod h1:sSbTewc+6wYHBBCW7ytsFSn836hqM7JxpglAy2Vzc58= github.com/go-chi/cors v1.2.2/go.mod h1:sSbTewc+6wYHBBCW7ytsFSn836hqM7JxpglAy2Vzc58=
github.com/go-errors/errors v1.4.2 h1:J6MZopCL4uSllY1OfXM374weqZFFItUbrImctkmUxIA= github.com/go-errors/errors v1.4.2 h1:J6MZopCL4uSllY1OfXM374weqZFFItUbrImctkmUxIA=

View File

@@ -38,8 +38,9 @@ type Config struct {
MetricsUsername string MetricsUsername string
Port int Port int
SentryDSN string SentryDSN string
MaxHistory int MessageMaxAge int
MaxMessageSize int MaxMessageSize int
QueueMaxAge int
MOTD string MOTD string
ServerName string ServerName string
FederationKey string FederationKey string
@@ -68,8 +69,9 @@ func New(
viper.SetDefault("SENTRY_DSN", "") viper.SetDefault("SENTRY_DSN", "")
viper.SetDefault("METRICS_USERNAME", "") viper.SetDefault("METRICS_USERNAME", "")
viper.SetDefault("METRICS_PASSWORD", "") viper.SetDefault("METRICS_PASSWORD", "")
viper.SetDefault("MAX_HISTORY", "10000") viper.SetDefault("MESSAGE_MAX_AGE", "2592000")
viper.SetDefault("MAX_MESSAGE_SIZE", "4096") viper.SetDefault("MAX_MESSAGE_SIZE", "4096")
viper.SetDefault("QUEUE_MAX_AGE", "2592000")
viper.SetDefault("MOTD", defaultMOTD) viper.SetDefault("MOTD", defaultMOTD)
viper.SetDefault("SERVER_NAME", "") viper.SetDefault("SERVER_NAME", "")
viper.SetDefault("FEDERATION_KEY", "") viper.SetDefault("FEDERATION_KEY", "")
@@ -92,8 +94,9 @@ func New(
MaintenanceMode: viper.GetBool("MAINTENANCE_MODE"), MaintenanceMode: viper.GetBool("MAINTENANCE_MODE"),
MetricsUsername: viper.GetString("METRICS_USERNAME"), MetricsUsername: viper.GetString("METRICS_USERNAME"),
MetricsPassword: viper.GetString("METRICS_PASSWORD"), MetricsPassword: viper.GetString("METRICS_PASSWORD"),
MaxHistory: viper.GetInt("MAX_HISTORY"), MessageMaxAge: viper.GetInt("MESSAGE_MAX_AGE"),
MaxMessageSize: viper.GetInt("MAX_MESSAGE_SIZE"), MaxMessageSize: viper.GetInt("MAX_MESSAGE_SIZE"),
QueueMaxAge: viper.GetInt("QUEUE_MAX_AGE"),
MOTD: viper.GetString("MOTD"), MOTD: viper.GetString("MOTD"),
ServerName: viper.GetString("SERVER_NAME"), ServerName: viper.GetString("SERVER_NAME"),
FederationKey: viper.GetString("FEDERATION_KEY"), FederationKey: viper.GetString("FEDERATION_KEY"),

View File

@@ -64,14 +64,12 @@ func (database *Database) RegisterUser(
sessionID, _ := res.LastInsertId() sessionID, _ := res.LastInsertId()
tokenHash := hashToken(token)
clientRes, err := transaction.ExecContext(ctx, clientRes, err := transaction.ExecContext(ctx,
`INSERT INTO clients `INSERT INTO clients
(uuid, session_id, token, (uuid, session_id, token,
created_at, last_seen) created_at, last_seen)
VALUES (?, ?, ?, ?, ?)`, VALUES (?, ?, ?, ?, ?)`,
clientUUID, sessionID, tokenHash, now, now) clientUUID, sessionID, token, now, now)
if err != nil { if err != nil {
_ = transaction.Rollback() _ = transaction.Rollback()
@@ -139,14 +137,12 @@ func (database *Database) LoginUser(
now := time.Now() now := time.Now()
tokenHash := hashToken(token)
res, err := database.conn.ExecContext(ctx, res, err := database.conn.ExecContext(ctx,
`INSERT INTO clients `INSERT INTO clients
(uuid, session_id, token, (uuid, session_id, token,
created_at, last_seen) created_at, last_seen)
VALUES (?, ?, ?, ?, ?)`, VALUES (?, ?, ?, ?, ?)`,
clientUUID, sessionID, tokenHash, now, now) clientUUID, sessionID, token, now, now)
if err != nil { if err != nil {
return 0, 0, "", fmt.Errorf( return 0, 0, "", fmt.Errorf(
"create login client: %w", err, "create login client: %w", err,

View File

@@ -1,20 +0,0 @@
// Package db provides database access and migration management.
package db
import (
"errors"
"modernc.org/sqlite"
sqlite3 "modernc.org/sqlite/lib"
)
// IsUniqueConstraintError reports whether err is a SQLite
// unique-constraint violation.
func IsUniqueConstraintError(err error) bool {
var sqliteErr *sqlite.Error
if !errors.As(err, &sqliteErr) {
return false
}
return sqliteErr.Code() == sqlite3.SQLITE_CONSTRAINT_UNIQUE
}

View File

@@ -3,7 +3,6 @@ package db
import ( import (
"context" "context"
"crypto/rand" "crypto/rand"
"crypto/sha256"
"database/sql" "database/sql"
"encoding/hex" "encoding/hex"
"encoding/json" "encoding/json"
@@ -32,14 +31,6 @@ func generateToken() (string, error) {
return hex.EncodeToString(buf), nil return hex.EncodeToString(buf), nil
} }
// hashToken returns the lowercase hex-encoded SHA-256
// digest of a plaintext token string.
func hashToken(token string) string {
sum := sha256.Sum256([]byte(token))
return hex.EncodeToString(sum[:])
}
// IRCMessage is the IRC envelope for all messages. // IRCMessage is the IRC envelope for all messages.
type IRCMessage struct { type IRCMessage struct {
ID string `json:"id"` ID string `json:"id"`
@@ -114,14 +105,12 @@ func (database *Database) CreateSession(
sessionID, _ := res.LastInsertId() sessionID, _ := res.LastInsertId()
tokenHash := hashToken(token)
clientRes, err := transaction.ExecContext(ctx, clientRes, err := transaction.ExecContext(ctx,
`INSERT INTO clients `INSERT INTO clients
(uuid, session_id, token, (uuid, session_id, token,
created_at, last_seen) created_at, last_seen)
VALUES (?, ?, ?, ?, ?)`, VALUES (?, ?, ?, ?, ?)`,
clientUUID, sessionID, tokenHash, now, now) clientUUID, sessionID, token, now, now)
if err != nil { if err != nil {
_ = transaction.Rollback() _ = transaction.Rollback()
@@ -154,8 +143,6 @@ func (database *Database) GetSessionByToken(
nick string nick string
) )
tokenHash := hashToken(token)
err := database.conn.QueryRowContext( err := database.conn.QueryRowContext(
ctx, ctx,
`SELECT s.id, c.id, s.nick `SELECT s.id, c.id, s.nick
@@ -163,7 +150,7 @@ func (database *Database) GetSessionByToken(
INNER JOIN sessions s INNER JOIN sessions s
ON s.id = c.session_id ON s.id = c.session_id
WHERE c.token = ?`, WHERE c.token = ?`,
tokenHash, token,
).Scan(&sessionID, &clientID, &nick) ).Scan(&sessionID, &clientID, &nick)
if err != nil { if err != nil {
return 0, 0, "", fmt.Errorf( return 0, 0, "", fmt.Errorf(
@@ -1109,3 +1096,45 @@ func (database *Database) GetSessionCreatedAt(
return createdAt, nil return createdAt, nil
} }
// PruneOldQueueEntries deletes client_queues rows older
// than cutoff and returns the number of rows removed.
func (database *Database) PruneOldQueueEntries(
ctx context.Context,
cutoff time.Time,
) (int64, error) {
res, err := database.conn.ExecContext(ctx,
"DELETE FROM client_queues WHERE created_at < ?",
cutoff,
)
if err != nil {
return 0, fmt.Errorf(
"prune old queue entries: %w", err,
)
}
deleted, _ := res.RowsAffected()
return deleted, nil
}
// PruneOldMessages deletes messages older than cutoff and
// returns the number of rows removed.
func (database *Database) PruneOldMessages(
ctx context.Context,
cutoff time.Time,
) (int64, error) {
res, err := database.conn.ExecContext(ctx,
"DELETE FROM messages WHERE created_at < ?",
cutoff,
)
if err != nil {
return 0, fmt.Errorf(
"prune old messages: %w", err,
)
}
deleted, _ := res.RowsAffected()
return deleted, nil
}

View File

@@ -10,9 +10,8 @@ import (
"strings" "strings"
"time" "time"
"git.eeqj.de/sneak/neoirc/internal/db"
"git.eeqj.de/sneak/neoirc/internal/irc" "git.eeqj.de/sneak/neoirc/internal/irc"
"github.com/go-chi/chi/v5" "github.com/go-chi/chi"
) )
var validNickRe = regexp.MustCompile( var validNickRe = regexp.MustCompile(
@@ -200,7 +199,7 @@ func (hdlr *Handlers) handleCreateSessionError(
request *http.Request, request *http.Request,
err error, err error,
) { ) {
if db.IsUniqueConstraintError(err) { if strings.Contains(err.Error(), "UNIQUE") {
hdlr.respondError( hdlr.respondError(
writer, request, writer, request,
"nick already taken", "nick already taken",
@@ -1428,7 +1427,7 @@ func (hdlr *Handlers) executeNickChange(
request.Context(), sessionID, newNick, request.Context(), sessionID, newNick,
) )
if err != nil { if err != nil {
if db.IsUniqueConstraintError(err) { if strings.Contains(err.Error(), "UNIQUE") {
hdlr.respondIRCError( hdlr.respondIRCError(
writer, request, clientID, sessionID, writer, request, clientID, sessionID,
irc.ErrNicknameInUse, nick, []string{newNick}, irc.ErrNicknameInUse, nick, []string{newNick},

View File

@@ -4,8 +4,6 @@ import (
"encoding/json" "encoding/json"
"net/http" "net/http"
"strings" "strings"
"git.eeqj.de/sneak/neoirc/internal/db"
) )
const minPasswordLength = 8 const minPasswordLength = 8
@@ -96,7 +94,7 @@ func (hdlr *Handlers) handleRegisterError(
request *http.Request, request *http.Request,
err error, err error,
) { ) {
if db.IsUniqueConstraintError(err) { if strings.Contains(err.Error(), "UNIQUE") {
hdlr.respondError( hdlr.respondError(
writer, request, writer, request,
"nick already taken", "nick already taken",

View File

@@ -200,4 +200,52 @@ func (hdlr *Handlers) runCleanup(
"deleted", deleted, "deleted", deleted,
) )
} }
hdlr.pruneQueuesAndMessages(ctx)
}
// pruneQueuesAndMessages removes old client_queues entries
// per QUEUE_MAX_AGE and old messages per MESSAGE_MAX_AGE.
func (hdlr *Handlers) pruneQueuesAndMessages(
ctx context.Context,
) {
queueMaxAge := hdlr.params.Config.QueueMaxAge
if queueMaxAge > 0 {
queueCutoff := time.Now().Add(
-time.Duration(queueMaxAge) * time.Second,
)
pruned, err := hdlr.params.Database.
PruneOldQueueEntries(ctx, queueCutoff)
if err != nil {
hdlr.log.Error(
"queue pruning failed", "error", err,
)
} else if pruned > 0 {
hdlr.log.Info(
"pruned old queue entries",
"deleted", pruned,
)
}
}
messageMaxAge := hdlr.params.Config.MessageMaxAge
if messageMaxAge > 0 {
msgCutoff := time.Now().Add(
-time.Duration(messageMaxAge) * time.Second,
)
pruned, err := hdlr.params.Database.
PruneOldMessages(ctx, msgCutoff)
if err != nil {
hdlr.log.Error(
"message pruning failed", "error", err,
)
} else if pruned > 0 {
hdlr.log.Info(
"pruned old messages",
"deleted", pruned,
)
}
}
} }

View File

@@ -11,7 +11,7 @@ import (
"git.eeqj.de/sneak/neoirc/internal/globals" "git.eeqj.de/sneak/neoirc/internal/globals"
"git.eeqj.de/sneak/neoirc/internal/logger" "git.eeqj.de/sneak/neoirc/internal/logger"
basicauth "github.com/99designs/basicauth-go" basicauth "github.com/99designs/basicauth-go"
chimw "github.com/go-chi/chi/v5/middleware" chimw "github.com/go-chi/chi/middleware"
"github.com/go-chi/cors" "github.com/go-chi/cors"
metrics "github.com/slok/go-http-metrics/metrics/prometheus" metrics "github.com/slok/go-http-metrics/metrics/prometheus"
ghmm "github.com/slok/go-http-metrics/middleware" ghmm "github.com/slok/go-http-metrics/middleware"

View File

@@ -8,8 +8,8 @@ import (
"git.eeqj.de/sneak/neoirc/web" "git.eeqj.de/sneak/neoirc/web"
sentryhttp "github.com/getsentry/sentry-go/http" sentryhttp "github.com/getsentry/sentry-go/http"
"github.com/go-chi/chi/v5" "github.com/go-chi/chi"
"github.com/go-chi/chi/v5/middleware" "github.com/go-chi/chi/middleware"
"github.com/prometheus/client_golang/prometheus/promhttp" "github.com/prometheus/client_golang/prometheus/promhttp"
"github.com/spf13/viper" "github.com/spf13/viper"
) )

View File

@@ -20,7 +20,7 @@ import (
"go.uber.org/fx" "go.uber.org/fx"
"github.com/getsentry/sentry-go" "github.com/getsentry/sentry-go"
"github.com/go-chi/chi/v5" "github.com/go-chi/chi"
_ "github.com/joho/godotenv/autoload" // loads .env file _ "github.com/joho/godotenv/autoload" // loads .env file
) )