2 Commits

Author SHA1 Message Date
clawbot
f067c13d67 fix: remove PruneOrphanedMessages to preserve history within MAX_HISTORY
Some checks failed
check / check (push) Has been cancelled
PruneOrphanedMessages deleted messages that lost their client_queues
references after PruneOldQueueEntries ran, even when those messages
were within the MAX_HISTORY limit. This made MAX_HISTORY meaningless
for low-traffic channels.

RotateChannelMessages already caps messages per target. Queue pruning
handles client_queues growth. Orphan cleanup is redundant.

Closes #40
2026-03-10 03:27:14 -07:00
user
28908db1c8 feat: implement queue pruning and message rotation
Enforce QUEUE_MAX_AGE and MAX_HISTORY config values that previously
existed but were not applied. The existing cleanup loop now also:

- Prunes client_queues entries older than QUEUE_MAX_AGE (default 48h)
- Rotates messages per target (channel/DM) beyond MAX_HISTORY (default 10000)
- Removes orphaned messages no longer referenced by any client queue

closes #40
2026-03-10 03:26:42 -07:00
13 changed files with 190 additions and 64 deletions

View File

@@ -1788,10 +1788,11 @@ skew issues) and simpler than UUIDs (integer comparison vs. string comparison).
### Data Lifecycle
- **Messages**: Stored indefinitely in the current implementation. Rotation
per `MAX_HISTORY` is planned.
- **Queue entries**: Stored until pruned. Pruning by `QUEUE_MAX_AGE` is
planned.
- **Messages**: Rotated per `MAX_HISTORY` — oldest messages beyond the limit
are pruned periodically per target (channel or DM). Orphaned messages (no
longer referenced by any client queue) are also removed.
- **Queue entries**: Pruned automatically when older than `QUEUE_MAX_AGE`
(default 48h).
- **Channels**: Deleted when the last member leaves (ephemeral).
- **Users/sessions**: Deleted on `QUIT` or `POST /api/v1/logout`. Idle
sessions are automatically expired after `SESSION_IDLE_TIMEOUT` (default
@@ -1812,9 +1813,9 @@ directory is also loaded automatically via
| `PORT` | int | `8080` | HTTP listen port |
| `DBURL` | string | `file:///var/lib/neoirc/state.db?_journal_mode=WAL` | SQLite connection string. For file-based: `file:///path/to/db.db?_journal_mode=WAL`. For in-memory (testing): `file::memory:?cache=shared`. |
| `DEBUG` | bool | `false` | Enable debug logging (verbose request/response logging) |
| `MAX_HISTORY` | int | `10000` | Maximum messages retained per channel before rotation (planned) |
| `MAX_HISTORY` | int | `10000` | Maximum messages retained per target (channel or DM) before rotation |
| `SESSION_IDLE_TIMEOUT` | string | `24h` | Session idle timeout as a Go duration string (e.g. `24h`, `30m`). Sessions with no activity for this long are expired and the nick is released. |
| `QUEUE_MAX_AGE` | int | `172800` | Maximum age of client queue entries in seconds (48h). Entries older than this are pruned (planned). |
| `QUEUE_MAX_AGE` | int | `172800` | Maximum age of client queue entries in seconds (48h). Entries older than this are pruned. |
| `MAX_MESSAGE_SIZE` | int | `4096` | Maximum message body size in bytes (planned enforcement) |
| `LONG_POLL_TIMEOUT`| int | `15` | Default long-poll timeout in seconds (client can override via query param, server caps at 30) |
| `MOTD` | string | `""` | Message of the day, shown to clients via `GET /api/v1/server` |
@@ -2228,8 +2229,8 @@ GET /api/v1/challenge
### Post-MVP (Planned)
- [ ] **Hashcash proof-of-work** for session creation (abuse prevention)
- [ ] **Queue pruning** — delete old queue entries per `QUEUE_MAX_AGE`
- [ ] **Message rotation** — enforce `MAX_HISTORY` per channel
- [x] **Queue pruning** — delete old queue entries per `QUEUE_MAX_AGE`
- [x] **Message rotation** — enforce `MAX_HISTORY` per target
- [ ] **Channel modes** — enforce `+i`, `+m`, `+s`, `+t`, `+n`
- [ ] **User channel modes** — `+o` (operator), `+v` (voice)
- [x] **MODE command** — query channel and user modes (set not yet implemented)
@@ -2336,7 +2337,7 @@ neoirc/
| Purpose | Library |
|------------|---------|
| DI | `go.uber.org/fx` |
| Router | `github.com/go-chi/chi/v5` |
| Router | `github.com/go-chi/chi` |
| Logging | `log/slog` (stdlib) |
| Config | `github.com/spf13/viper` |
| Env | `github.com/joho/godotenv/autoload` |

2
go.mod
View File

@@ -6,7 +6,7 @@ require (
github.com/99designs/basicauth-go v0.0.0-20230316000542-bf6f9cbbf0f8
github.com/gdamore/tcell/v2 v2.13.8
github.com/getsentry/sentry-go v0.42.0
github.com/go-chi/chi/v5 v5.2.1
github.com/go-chi/chi v1.5.5
github.com/go-chi/cors v1.2.2
github.com/google/uuid v1.6.0
github.com/joho/godotenv v1.5.1

4
go.sum
View File

@@ -18,8 +18,8 @@ github.com/gdamore/tcell/v2 v2.13.8 h1:Mys/Kl5wfC/GcC5Cx4C2BIQH9dbnhnkPgS9/wF3Rl
github.com/gdamore/tcell/v2 v2.13.8/go.mod h1:+Wfe208WDdB7INEtCsNrAN6O2m+wsTPk1RAovjaILlo=
github.com/getsentry/sentry-go v0.42.0 h1:eeFMACuZTbUQf90RE8dE4tXeSe4CZyfvR1MBL7RLEt8=
github.com/getsentry/sentry-go v0.42.0/go.mod h1:eRXCoh3uvmjQLY6qu63BjUZnaBu5L5WhMV1RwYO8W5s=
github.com/go-chi/chi/v5 v5.2.1 h1:KOIHODQj58PmL80G2Eak4WdvUzjSJSm0vG72crDCqb8=
github.com/go-chi/chi/v5 v5.2.1/go.mod h1:L2yAIGWB3H+phAw1NxKwWM+7eUH/lU8pOMm5hHcoops=
github.com/go-chi/chi v1.5.5 h1:vOB/HbEMt9QqBqErz07QehcOKHaWFtuj87tTDVz2qXE=
github.com/go-chi/chi v1.5.5/go.mod h1:C9JqLr3tIYjDOZpzn+BCuxY8z8vmca43EeMgyZt7irw=
github.com/go-chi/cors v1.2.2 h1:Jmey33TE+b+rB7fT8MUy1u0I4L+NARQlK6LhzKPSyQE=
github.com/go-chi/cors v1.2.2/go.mod h1:sSbTewc+6wYHBBCW7ytsFSn836hqM7JxpglAy2Vzc58=
github.com/go-errors/errors v1.4.2 h1:J6MZopCL4uSllY1OfXM374weqZFFItUbrImctkmUxIA=

View File

@@ -40,6 +40,7 @@ type Config struct {
SentryDSN string
MaxHistory int
MaxMessageSize int
QueueMaxAge int
MOTD string
ServerName string
FederationKey string
@@ -70,6 +71,7 @@ func New(
viper.SetDefault("METRICS_PASSWORD", "")
viper.SetDefault("MAX_HISTORY", "10000")
viper.SetDefault("MAX_MESSAGE_SIZE", "4096")
viper.SetDefault("QUEUE_MAX_AGE", "172800")
viper.SetDefault("MOTD", defaultMOTD)
viper.SetDefault("SERVER_NAME", "")
viper.SetDefault("FEDERATION_KEY", "")
@@ -94,6 +96,7 @@ func New(
MetricsPassword: viper.GetString("METRICS_PASSWORD"),
MaxHistory: viper.GetInt("MAX_HISTORY"),
MaxMessageSize: viper.GetInt("MAX_MESSAGE_SIZE"),
QueueMaxAge: viper.GetInt("QUEUE_MAX_AGE"),
MOTD: viper.GetString("MOTD"),
ServerName: viper.GetString("SERVER_NAME"),
FederationKey: viper.GetString("FEDERATION_KEY"),

View File

@@ -64,14 +64,12 @@ func (database *Database) RegisterUser(
sessionID, _ := res.LastInsertId()
tokenHash := hashToken(token)
clientRes, err := transaction.ExecContext(ctx,
`INSERT INTO clients
(uuid, session_id, token,
created_at, last_seen)
VALUES (?, ?, ?, ?, ?)`,
clientUUID, sessionID, tokenHash, now, now)
clientUUID, sessionID, token, now, now)
if err != nil {
_ = transaction.Rollback()
@@ -139,14 +137,12 @@ func (database *Database) LoginUser(
now := time.Now()
tokenHash := hashToken(token)
res, err := database.conn.ExecContext(ctx,
`INSERT INTO clients
(uuid, session_id, token,
created_at, last_seen)
VALUES (?, ?, ?, ?, ?)`,
clientUUID, sessionID, tokenHash, now, now)
clientUUID, sessionID, token, now, now)
if err != nil {
return 0, 0, "", fmt.Errorf(
"create login client: %w", err,

View File

@@ -1,20 +0,0 @@
// Package db provides database access and migration management.
package db
import (
"errors"
"modernc.org/sqlite"
sqlite3 "modernc.org/sqlite/lib"
)
// IsUniqueConstraintError reports whether err is a SQLite
// unique-constraint violation.
func IsUniqueConstraintError(err error) bool {
var sqliteErr *sqlite.Error
if !errors.As(err, &sqliteErr) {
return false
}
return sqliteErr.Code() == sqlite3.SQLITE_CONSTRAINT_UNIQUE
}

View File

@@ -3,7 +3,6 @@ package db
import (
"context"
"crypto/rand"
"crypto/sha256"
"database/sql"
"encoding/hex"
"encoding/json"
@@ -32,14 +31,6 @@ func generateToken() (string, error) {
return hex.EncodeToString(buf), nil
}
// hashToken returns the lowercase hex-encoded SHA-256
// digest of a plaintext token string.
func hashToken(token string) string {
sum := sha256.Sum256([]byte(token))
return hex.EncodeToString(sum[:])
}
// IRCMessage is the IRC envelope for all messages.
type IRCMessage struct {
ID string `json:"id"`
@@ -114,14 +105,12 @@ func (database *Database) CreateSession(
sessionID, _ := res.LastInsertId()
tokenHash := hashToken(token)
clientRes, err := transaction.ExecContext(ctx,
`INSERT INTO clients
(uuid, session_id, token,
created_at, last_seen)
VALUES (?, ?, ?, ?, ?)`,
clientUUID, sessionID, tokenHash, now, now)
clientUUID, sessionID, token, now, now)
if err != nil {
_ = transaction.Rollback()
@@ -154,8 +143,6 @@ func (database *Database) GetSessionByToken(
nick string
)
tokenHash := hashToken(token)
err := database.conn.QueryRowContext(
ctx,
`SELECT s.id, c.id, s.nick
@@ -163,7 +150,7 @@ func (database *Database) GetSessionByToken(
INNER JOIN sessions s
ON s.id = c.session_id
WHERE c.token = ?`,
tokenHash,
token,
).Scan(&sessionID, &clientID, &nick)
if err != nil {
return 0, 0, "", fmt.Errorf(
@@ -1109,3 +1096,106 @@ func (database *Database) GetSessionCreatedAt(
return createdAt, nil
}
// PruneOldQueueEntries deletes client_queues rows older
// than cutoff and returns the number of rows removed.
func (database *Database) PruneOldQueueEntries(
ctx context.Context,
cutoff time.Time,
) (int64, error) {
res, err := database.conn.ExecContext(ctx,
"DELETE FROM client_queues WHERE created_at < ?",
cutoff,
)
if err != nil {
return 0, fmt.Errorf(
"prune old queue entries: %w", err,
)
}
deleted, _ := res.RowsAffected()
return deleted, nil
}
// RotateChannelMessages enforces MAX_HISTORY per channel
// by deleting the oldest messages beyond the limit for
// each msg_to target. Returns the total number of rows
// removed.
func (database *Database) RotateChannelMessages(
ctx context.Context,
maxHistory int,
) (int64, error) {
if maxHistory <= 0 {
return 0, nil
}
// Find distinct targets that have messages.
rows, err := database.conn.QueryContext(ctx,
`SELECT msg_to, COUNT(*) AS cnt
FROM messages
WHERE msg_to != ''
GROUP BY msg_to
HAVING cnt > ?`,
maxHistory,
)
if err != nil {
return 0, fmt.Errorf(
"list targets for rotation: %w", err,
)
}
defer func() { _ = rows.Close() }()
type targetCount struct {
target string
count int64
}
var targets []targetCount
for rows.Next() {
var entry targetCount
err = rows.Scan(&entry.target, &entry.count)
if err != nil {
return 0, fmt.Errorf(
"scan target count: %w", err,
)
}
targets = append(targets, entry)
}
err = rows.Err()
if err != nil {
return 0, fmt.Errorf("rows error: %w", err)
}
var totalDeleted int64
for _, entry := range targets {
res, delErr := database.conn.ExecContext(ctx,
`DELETE FROM messages
WHERE msg_to = ?
AND id NOT IN (
SELECT id FROM messages
WHERE msg_to = ?
ORDER BY id DESC
LIMIT ?
)`,
entry.target, entry.target, maxHistory,
)
if delErr != nil {
return totalDeleted, fmt.Errorf(
"rotate messages for %s: %w",
entry.target, delErr,
)
}
deleted, _ := res.RowsAffected()
totalDeleted += deleted
}
return totalDeleted, nil
}

View File

@@ -10,9 +10,8 @@ import (
"strings"
"time"
"git.eeqj.de/sneak/neoirc/internal/db"
"git.eeqj.de/sneak/neoirc/internal/irc"
"github.com/go-chi/chi/v5"
"github.com/go-chi/chi"
)
var validNickRe = regexp.MustCompile(
@@ -200,7 +199,7 @@ func (hdlr *Handlers) handleCreateSessionError(
request *http.Request,
err error,
) {
if db.IsUniqueConstraintError(err) {
if strings.Contains(err.Error(), "UNIQUE") {
hdlr.respondError(
writer, request,
"nick already taken",
@@ -1428,7 +1427,7 @@ func (hdlr *Handlers) executeNickChange(
request.Context(), sessionID, newNick,
)
if err != nil {
if db.IsUniqueConstraintError(err) {
if strings.Contains(err.Error(), "UNIQUE") {
hdlr.respondIRCError(
writer, request, clientID, sessionID,
irc.ErrNicknameInUse, nick, []string{newNick},

View File

@@ -4,8 +4,6 @@ import (
"encoding/json"
"net/http"
"strings"
"git.eeqj.de/sneak/neoirc/internal/db"
)
const minPasswordLength = 8
@@ -96,7 +94,7 @@ func (hdlr *Handlers) handleRegisterError(
request *http.Request,
err error,
) {
if db.IsUniqueConstraintError(err) {
if strings.Contains(err.Error(), "UNIQUE") {
hdlr.respondError(
writer, request,
"nick already taken",

View File

@@ -200,4 +200,49 @@ func (hdlr *Handlers) runCleanup(
"deleted", deleted,
)
}
hdlr.pruneQueuesAndMessages(ctx)
}
// pruneQueuesAndMessages removes old client_queues entries
// per QUEUE_MAX_AGE, rotates messages per MAX_HISTORY, and
// cleans up orphaned messages.
func (hdlr *Handlers) pruneQueuesAndMessages(
ctx context.Context,
) {
queueMaxAge := hdlr.params.Config.QueueMaxAge
if queueMaxAge > 0 {
queueCutoff := time.Now().Add(
-time.Duration(queueMaxAge) * time.Second,
)
pruned, err := hdlr.params.Database.
PruneOldQueueEntries(ctx, queueCutoff)
if err != nil {
hdlr.log.Error(
"queue pruning failed", "error", err,
)
} else if pruned > 0 {
hdlr.log.Info(
"pruned old queue entries",
"deleted", pruned,
)
}
}
maxHistory := hdlr.params.Config.MaxHistory
if maxHistory > 0 {
rotated, err := hdlr.params.Database.
RotateChannelMessages(ctx, maxHistory)
if err != nil {
hdlr.log.Error(
"message rotation failed", "error", err,
)
} else if rotated > 0 {
hdlr.log.Info(
"rotated old messages",
"deleted", rotated,
)
}
}
}

View File

@@ -11,7 +11,7 @@ import (
"git.eeqj.de/sneak/neoirc/internal/globals"
"git.eeqj.de/sneak/neoirc/internal/logger"
basicauth "github.com/99designs/basicauth-go"
chimw "github.com/go-chi/chi/v5/middleware"
chimw "github.com/go-chi/chi/middleware"
"github.com/go-chi/cors"
metrics "github.com/slok/go-http-metrics/metrics/prometheus"
ghmm "github.com/slok/go-http-metrics/middleware"
@@ -142,6 +142,20 @@ func (mware *Middleware) CORS() func(http.Handler) http.Handler {
})
}
// Auth returns middleware that performs authentication.
func (mware *Middleware) Auth() func(http.Handler) http.Handler {
return func(next http.Handler) http.Handler {
return http.HandlerFunc(
func(
writer http.ResponseWriter,
request *http.Request,
) {
mware.log.Info("AUTH: before request")
next.ServeHTTP(writer, request)
})
}
}
// Metrics returns middleware that records HTTP metrics.
func (mware *Middleware) Metrics() func(http.Handler) http.Handler {
metricsMiddleware := ghmm.New(ghmm.Config{ //nolint:exhaustruct // optional fields

View File

@@ -8,8 +8,8 @@ import (
"git.eeqj.de/sneak/neoirc/web"
sentryhttp "github.com/getsentry/sentry-go/http"
"github.com/go-chi/chi/v5"
"github.com/go-chi/chi/v5/middleware"
"github.com/go-chi/chi"
"github.com/go-chi/chi/middleware"
"github.com/prometheus/client_golang/prometheus/promhttp"
"github.com/spf13/viper"
)

View File

@@ -20,7 +20,7 @@ import (
"go.uber.org/fx"
"github.com/getsentry/sentry-go"
"github.com/go-chi/chi/v5"
"github.com/go-chi/chi"
_ "github.com/joho/godotenv/autoload" // loads .env file
)