Compare commits

..

1 Commits

Author SHA1 Message Date
user
d6f6cc3670 feat: add CPU and memory resource limits per app
All checks were successful
Check / check (pull_request) Successful in 3m24s
- Add cpu_limit (REAL) and memory_limit (INTEGER) columns to apps table
  via migration 007
- Add CPULimit and MemoryLimit fields to App model with full CRUD support
- Add resource limits fields to app edit form with human-friendly
  memory input (e.g. 256m, 1g, 512k)
- Pass CPU and memory limits to Docker container creation via
  NanoCPUs and Memory host config fields
- Extract Docker container creation helpers (buildEnvSlice, buildMounts,
  buildResources) for cleaner code
- Add formatMemoryBytes template function for display
- Add comprehensive tests for parsing, formatting, model persistence,
  and container options
2026-03-17 02:10:51 -07:00
31 changed files with 773 additions and 1484 deletions

View File

@@ -9,6 +9,7 @@ A simple self-hosted PaaS that auto-deploys Docker containers from Git repositor
- Per-app UUID-based webhook URLs for Gitea integration - Per-app UUID-based webhook URLs for Gitea integration
- Branch filtering - only deploy on configured branch changes - Branch filtering - only deploy on configured branch changes
- Environment variables, labels, and volume mounts per app - Environment variables, labels, and volume mounts per app
- CPU and memory resource limits per app
- Docker builds via socket access - Docker builds via socket access
- Notifications via ntfy and Slack-compatible webhooks - Notifications via ntfy and Slack-compatible webhooks
- Simple server-rendered UI with Tailwind CSS - Simple server-rendered UI with Tailwind CSS
@@ -36,13 +37,11 @@ upaas/
│ ├── handlers/ # HTTP request handlers │ ├── handlers/ # HTTP request handlers
│ ├── healthcheck/ # Health status service │ ├── healthcheck/ # Health status service
│ ├── logger/ # Structured logging (slog) │ ├── logger/ # Structured logging (slog)
│ ├── metrics/ # Prometheus metrics registration │ ├── middleware/ # HTTP middleware (auth, logging, CORS)
│ ├── middleware/ # HTTP middleware (auth, logging, CORS, metrics)
│ ├── models/ # Active Record style database models │ ├── models/ # Active Record style database models
│ ├── server/ # HTTP server and routes │ ├── server/ # HTTP server and routes
│ ├── service/ │ ├── service/
│ │ ├── app/ # App management service │ │ ├── app/ # App management service
│ │ ├── audit/ # Audit logging service
│ │ ├── auth/ # Authentication service │ │ ├── auth/ # Authentication service
│ │ ├── deploy/ # Deployment orchestration │ │ ├── deploy/ # Deployment orchestration
│ │ ├── notify/ # Notifications (ntfy, Slack) │ │ ├── notify/ # Notifications (ntfy, Slack)
@@ -60,18 +59,16 @@ Uses Uber fx for dependency injection. Components are wired in this order:
2. `logger` - Structured logging 2. `logger` - Structured logging
3. `config` - Configuration loading 3. `config` - Configuration loading
4. `database` - SQLite connection + migrations 4. `database` - SQLite connection + migrations
5. `metrics` - Prometheus metrics registration 5. `healthcheck` - Health status
6. `healthcheck` - Health status 6. `auth` - Authentication service
7. `auth` - Authentication service 7. `app` - App management
8. `app` - App management 8. `docker` - Docker client
9. `docker` - Docker client 9. `notify` - Notification service
10. `notify` - Notification service 10. `deploy` - Deployment service
11. `audit` - Audit logging service 11. `webhook` - Webhook processing
12. `deploy` - Deployment service 12. `middleware` - HTTP middleware
13. `webhook` - Webhook processing 13. `handlers` - HTTP handlers
14. `middleware` - HTTP middleware 14. `server` - HTTP server
15. `handlers` - HTTP handlers
16. `server` - HTTP server
### Request Flow ### Request Flow
@@ -215,48 +212,6 @@ Example: `HOST_DATA_DIR=/srv/upaas/data docker compose up -d`
Session secrets are automatically generated on first startup and persisted to `$UPAAS_DATA_DIR/session.key`. Session secrets are automatically generated on first startup and persisted to `$UPAAS_DATA_DIR/session.key`.
## Observability
### Prometheus Metrics
All custom metrics are exposed under the `upaas_` namespace at `/metrics`. The
endpoint is always available and can be optionally protected with basic auth via
`METRICS_USERNAME` and `METRICS_PASSWORD`.
| Metric | Type | Labels | Description |
|--------|------|--------|-------------|
| `upaas_deployments_total` | Counter | `app`, `status` | Total deployments (success/failed/cancelled) |
| `upaas_deployments_duration_seconds` | Histogram | `app`, `status` | Deployment duration |
| `upaas_deployments_in_flight` | Gauge | `app` | Currently running deployments |
| `upaas_container_healthy` | Gauge | `app` | Container health (1=healthy, 0=unhealthy) |
| `upaas_webhook_events_total` | Counter | `app`, `event_type`, `matched` | Webhook events received |
| `upaas_http_requests_total` | Counter | `method`, `status_code` | HTTP requests |
| `upaas_http_request_duration_seconds` | Histogram | `method` | HTTP request latency |
| `upaas_http_response_size_bytes` | Histogram | `method` | HTTP response sizes |
| `upaas_audit_events_total` | Counter | `action` | Audit log events |
### Audit Log
All user-facing actions are recorded in an `audit_log` SQLite table with:
- **Who**: user ID and username
- **What**: action type and affected resource (app, deployment, session, etc.)
- **Where**: client IP (via X-Real-IP/X-Forwarded-For/RemoteAddr)
- **When**: timestamp
Audited actions include login/logout, app CRUD, deployments, container
start/stop/restart, rollbacks, deployment cancellation, and webhook receipt.
The audit log is available via the API at `GET /api/v1/audit?limit=N` (max 500,
default 50).
### Structured Logging
All operations use Go's `slog` structured logger. HTTP requests are logged with
method, URL, status code, response size, latency, user agent, and client IP.
Deployment events are logged with app name, status, and duration. Audit events
are also logged to stdout for correlation with external log aggregators.
## License ## License
WTFPL WTFPL

View File

@@ -11,11 +11,9 @@ import (
"sneak.berlin/go/upaas/internal/handlers" "sneak.berlin/go/upaas/internal/handlers"
"sneak.berlin/go/upaas/internal/healthcheck" "sneak.berlin/go/upaas/internal/healthcheck"
"sneak.berlin/go/upaas/internal/logger" "sneak.berlin/go/upaas/internal/logger"
"sneak.berlin/go/upaas/internal/metrics"
"sneak.berlin/go/upaas/internal/middleware" "sneak.berlin/go/upaas/internal/middleware"
"sneak.berlin/go/upaas/internal/server" "sneak.berlin/go/upaas/internal/server"
"sneak.berlin/go/upaas/internal/service/app" "sneak.berlin/go/upaas/internal/service/app"
"sneak.berlin/go/upaas/internal/service/audit"
"sneak.berlin/go/upaas/internal/service/auth" "sneak.berlin/go/upaas/internal/service/auth"
"sneak.berlin/go/upaas/internal/service/deploy" "sneak.berlin/go/upaas/internal/service/deploy"
"sneak.berlin/go/upaas/internal/service/notify" "sneak.berlin/go/upaas/internal/service/notify"
@@ -43,7 +41,6 @@ func main() {
logger.New, logger.New,
config.New, config.New,
database.New, database.New,
metrics.New,
healthcheck.New, healthcheck.New,
auth.New, auth.New,
app.New, app.New,
@@ -51,7 +48,6 @@ func main() {
notify.New, notify.New,
deploy.New, deploy.New,
webhook.New, webhook.New,
audit.New,
middleware.New, middleware.New,
handlers.New, handlers.New,
server.New, server.New,

View File

@@ -1,16 +0,0 @@
-- Audit log table for tracking user actions.
CREATE TABLE audit_log (
id INTEGER PRIMARY KEY,
user_id INTEGER,
username TEXT NOT NULL,
action TEXT NOT NULL,
resource_type TEXT NOT NULL,
resource_id TEXT,
detail TEXT,
remote_ip TEXT,
created_at DATETIME DEFAULT CURRENT_TIMESTAMP
);
CREATE INDEX idx_audit_log_created_at ON audit_log(created_at);
CREATE INDEX idx_audit_log_action ON audit_log(action);
CREATE INDEX idx_audit_log_resource ON audit_log(resource_type, resource_id);

View File

@@ -0,0 +1,3 @@
-- Add CPU and memory resource limits per app
ALTER TABLE apps ADD COLUMN cpu_limit REAL;
ALTER TABLE apps ADD COLUMN memory_limit INTEGER;

View File

@@ -138,13 +138,15 @@ func (c *Client) BuildImage(
// CreateContainerOptions contains options for creating a container. // CreateContainerOptions contains options for creating a container.
type CreateContainerOptions struct { type CreateContainerOptions struct {
Name string Name string
Image string Image string
Env map[string]string Env map[string]string
Labels map[string]string Labels map[string]string
Volumes []VolumeMount Volumes []VolumeMount
Ports []PortMapping Ports []PortMapping
Network string Network string
CPULimit float64 // CPU cores (e.g. 0.5 = half a core, 2.0 = two cores). 0 means unlimited.
MemoryLimit int64 // Memory in bytes. 0 means unlimited.
} }
// VolumeMount represents a volume mount. // VolumeMount represents a volume mount.
@@ -161,6 +163,14 @@ type PortMapping struct {
Protocol string // "tcp" or "udp" Protocol string // "tcp" or "udp"
} }
// nanoCPUsPerCPU is the number of NanoCPUs per CPU core.
const nanoCPUsPerCPU = 1e9
// cpuLimitToNanoCPUs converts a CPU limit (e.g. 0.5 cores) to Docker NanoCPUs.
func cpuLimitToNanoCPUs(cpuLimit float64) int64 {
return int64(cpuLimit * nanoCPUsPerCPU)
}
// buildPortConfig converts port mappings to Docker port configuration. // buildPortConfig converts port mappings to Docker port configuration.
func buildPortConfig(ports []PortMapping) (nat.PortSet, nat.PortMap) { func buildPortConfig(ports []PortMapping) (nat.PortSet, nat.PortMap) {
exposedPorts := make(nat.PortSet) exposedPorts := make(nat.PortSet)
@@ -185,6 +195,48 @@ func buildPortConfig(ports []PortMapping) (nat.PortSet, nat.PortMap) {
return exposedPorts, portBindings return exposedPorts, portBindings
} }
// buildEnvSlice converts an env map to a Docker-compatible env slice.
func buildEnvSlice(env map[string]string) []string {
envSlice := make([]string, 0, len(env))
for key, val := range env {
envSlice = append(envSlice, key+"="+val)
}
return envSlice
}
// buildMounts converts volume mounts to Docker mount configuration.
func buildMounts(volumes []VolumeMount) []mount.Mount {
mounts := make([]mount.Mount, 0, len(volumes))
for _, vol := range volumes {
mounts = append(mounts, mount.Mount{
Type: mount.TypeBind,
Source: vol.HostPath,
Target: vol.ContainerPath,
ReadOnly: vol.ReadOnly,
})
}
return mounts
}
// buildResources builds Docker resource constraints from container options.
func buildResources(opts CreateContainerOptions) container.Resources {
resources := container.Resources{}
if opts.CPULimit > 0 {
resources.NanoCPUs = cpuLimitToNanoCPUs(opts.CPULimit)
}
if opts.MemoryLimit > 0 {
resources.Memory = opts.MemoryLimit
}
return resources
}
// CreateContainer creates a new container. // CreateContainer creates a new container.
func (c *Client) CreateContainer( func (c *Client) CreateContainer(
ctx context.Context, ctx context.Context,
@@ -196,40 +248,20 @@ func (c *Client) CreateContainer(
c.log.Info("creating container", "name", opts.Name, "image", opts.Image) c.log.Info("creating container", "name", opts.Name, "image", opts.Image)
// Convert env map to slice
envSlice := make([]string, 0, len(opts.Env))
for key, val := range opts.Env {
envSlice = append(envSlice, key+"="+val)
}
// Convert volumes to mounts
mounts := make([]mount.Mount, 0, len(opts.Volumes))
for _, vol := range opts.Volumes {
mounts = append(mounts, mount.Mount{
Type: mount.TypeBind,
Source: vol.HostPath,
Target: vol.ContainerPath,
ReadOnly: vol.ReadOnly,
})
}
// Convert ports to exposed ports and port bindings
exposedPorts, portBindings := buildPortConfig(opts.Ports) exposedPorts, portBindings := buildPortConfig(opts.Ports)
// Create container
resp, err := c.docker.ContainerCreate(ctx, resp, err := c.docker.ContainerCreate(ctx,
&container.Config{ &container.Config{
Image: opts.Image, Image: opts.Image,
Env: envSlice, Env: buildEnvSlice(opts.Env),
Labels: opts.Labels, Labels: opts.Labels,
ExposedPorts: exposedPorts, ExposedPorts: exposedPorts,
}, },
&container.HostConfig{ &container.HostConfig{
Mounts: mounts, Mounts: buildMounts(opts.Volumes),
PortBindings: portBindings, PortBindings: portBindings,
NetworkMode: container.NetworkMode(opts.Network), NetworkMode: container.NetworkMode(opts.Network),
Resources: buildResources(opts),
RestartPolicy: container.RestartPolicy{ RestartPolicy: container.RestartPolicy{
Name: container.RestartPolicyUnlessStopped, Name: container.RestartPolicyUnlessStopped,
}, },

View File

@@ -0,0 +1,31 @@
package docker //nolint:testpackage // tests unexported cpuLimitToNanoCPUs
import (
"testing"
)
func TestCpuLimitToNanoCPUs(t *testing.T) {
t.Parallel()
tests := []struct {
name string
cpuLimit float64
expected int64
}{
{"one core", 1.0, 1_000_000_000},
{"half core", 0.5, 500_000_000},
{"two cores", 2.0, 2_000_000_000},
{"quarter core", 0.25, 250_000_000},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
got := cpuLimitToNanoCPUs(tt.cpuLimit)
if got != tt.expected {
t.Errorf("cpuLimitToNanoCPUs(%v) = %d, want %d", tt.cpuLimit, got, tt.expected)
}
})
}
}

View File

@@ -1,7 +1,6 @@
package handlers package handlers
import ( import (
"database/sql"
"encoding/json" "encoding/json"
"net/http" "net/http"
"strconv" "strconv"
@@ -121,9 +120,6 @@ func (h *Handlers) HandleAPILoginPOST() http.HandlerFunc {
return return
} }
h.auditLog(request, models.AuditActionLogin,
models.AuditResourceSession, "", "api login")
h.respondJSON(writer, request, loginResponse{ h.respondJSON(writer, request, loginResponse{
UserID: user.ID, UserID: user.ID,
Username: user.Username, Username: user.Username,
@@ -247,79 +243,3 @@ func (h *Handlers) HandleAPIWhoAmI() http.HandlerFunc {
}, http.StatusOK) }, http.StatusOK)
} }
} }
// auditLogDefaultLimit is the default number of audit entries returned.
const auditLogDefaultLimit = 50
// auditLogMaxLimit is the maximum number of audit entries returned.
const auditLogMaxLimit = 500
// HandleAPIAuditLog returns a handler that lists recent audit log entries.
func (h *Handlers) HandleAPIAuditLog() http.HandlerFunc {
type auditEntryResponse struct {
ID int64 `json:"id"`
UserID *int64 `json:"userId,omitempty"`
Username string `json:"username"`
Action string `json:"action"`
ResourceType string `json:"resourceType"`
ResourceID string `json:"resourceId,omitempty"`
Detail string `json:"detail,omitempty"`
RemoteIP string `json:"remoteIp,omitempty"`
CreatedAt string `json:"createdAt"`
}
return func(writer http.ResponseWriter, request *http.Request) {
limit := auditLogDefaultLimit
if limitStr := request.URL.Query().Get("limit"); limitStr != "" {
parsed, parseErr := strconv.Atoi(limitStr)
if parseErr == nil && parsed > 0 && parsed <= auditLogMaxLimit {
limit = parsed
}
}
entries, err := h.audit.Recent(request.Context(), limit)
if err != nil {
h.log.Error("failed to fetch audit log", "error", err)
h.respondJSON(writer, request,
map[string]string{"error": "failed to fetch audit log"},
http.StatusInternalServerError)
return
}
result := make([]auditEntryResponse, 0, len(entries))
for _, e := range entries {
entry := auditEntryResponse{
ID: e.ID,
Username: e.Username,
Action: string(e.Action),
ResourceType: string(e.ResourceType),
CreatedAt: e.CreatedAt.UTC().Format("2006-01-02T15:04:05Z"),
}
if e.UserID.Valid {
id := e.UserID.Int64
entry.UserID = &id
}
entry.ResourceID = nullStringValue(e.ResourceID)
entry.Detail = nullStringValue(e.Detail)
entry.RemoteIP = nullStringValue(e.RemoteIP)
result = append(result, entry)
}
h.respondJSON(writer, request, result, http.StatusOK)
}
}
// nullStringValue returns the string value if valid, empty string otherwise.
func nullStringValue(ns sql.NullString) string {
if ns.Valid {
return ns.String
}
return ""
}

View File

@@ -119,9 +119,6 @@ func (h *Handlers) HandleAppCreate() http.HandlerFunc { //nolint:funlen // valid
return return
} }
h.auditLog(request, models.AuditActionAppCreate,
models.AuditResourceApp, createdApp.ID, "created app: "+createdApp.Name)
http.Redirect(writer, request, "/apps/"+createdApp.ID, http.StatusSeeOther) http.Redirect(writer, request, "/apps/"+createdApp.ID, http.StatusSeeOther)
} }
} }
@@ -260,23 +257,19 @@ func (h *Handlers) HandleAppUpdate() http.HandlerFunc { //nolint:funlen // valid
application.RepoURL = request.FormValue("repo_url") application.RepoURL = request.FormValue("repo_url")
application.Branch = request.FormValue("branch") application.Branch = request.FormValue("branch")
application.DockerfilePath = request.FormValue("dockerfile_path") application.DockerfilePath = request.FormValue("dockerfile_path")
application.DockerNetwork = optionalNullString(request.FormValue("docker_network"))
application.NtfyTopic = optionalNullString(request.FormValue("ntfy_topic"))
application.SlackWebhook = optionalNullString(request.FormValue("slack_webhook"))
if network := request.FormValue("docker_network"); network != "" { limitsErr := applyResourceLimits(application, request)
application.DockerNetwork = sql.NullString{String: network, Valid: true} if limitsErr != "" {
} else { data := h.addGlobals(map[string]any{
application.DockerNetwork = sql.NullString{} "App": application,
} "Error": limitsErr,
}, request)
h.renderTemplate(writer, tmpl, "app_edit.html", data)
if ntfy := request.FormValue("ntfy_topic"); ntfy != "" { return
application.NtfyTopic = sql.NullString{String: ntfy, Valid: true}
} else {
application.NtfyTopic = sql.NullString{}
}
if slack := request.FormValue("slack_webhook"); slack != "" {
application.SlackWebhook = sql.NullString{String: slack, Valid: true}
} else {
application.SlackWebhook = sql.NullString{}
} }
saveErr := application.Save(request.Context()) saveErr := application.Save(request.Context())
@@ -292,9 +285,6 @@ func (h *Handlers) HandleAppUpdate() http.HandlerFunc { //nolint:funlen // valid
return return
} }
h.auditLog(request, models.AuditActionAppUpdate,
models.AuditResourceApp, application.ID, "updated app: "+application.Name)
redirectURL := "/apps/" + application.ID + "?success=updated" redirectURL := "/apps/" + application.ID + "?success=updated"
http.Redirect(writer, request, redirectURL, http.StatusSeeOther) http.Redirect(writer, request, redirectURL, http.StatusSeeOther)
} }
@@ -350,9 +340,6 @@ func (h *Handlers) HandleAppDelete() http.HandlerFunc {
return return
} }
h.auditLog(request, models.AuditActionAppDelete,
models.AuditResourceApp, appID, "deleted app: "+application.Name)
http.Redirect(writer, request, "/", http.StatusSeeOther) http.Redirect(writer, request, "/", http.StatusSeeOther)
} }
} }
@@ -369,9 +356,6 @@ func (h *Handlers) HandleAppDeploy() http.HandlerFunc {
return return
} }
h.auditLog(request, models.AuditActionAppDeploy,
models.AuditResourceApp, application.ID, "manual deploy: "+application.Name)
// Trigger deployment in background with a detached context // Trigger deployment in background with a detached context
// so the deployment continues even if the HTTP request is cancelled // so the deployment continues even if the HTTP request is cancelled
deployCtx := context.WithoutCancel(request.Context()) deployCtx := context.WithoutCancel(request.Context())
@@ -411,8 +395,6 @@ func (h *Handlers) HandleCancelDeploy() http.HandlerFunc {
cancelled := h.deploy.CancelDeploy(application.ID) cancelled := h.deploy.CancelDeploy(application.ID)
if cancelled { if cancelled {
h.log.Info("deployment cancelled by user", "app", application.Name) h.log.Info("deployment cancelled by user", "app", application.Name)
h.auditLog(request, models.AuditActionDeployCancel,
models.AuditResourceDeployment, application.ID, "cancelled deploy: "+application.Name)
} }
http.Redirect( http.Redirect(
@@ -444,9 +426,6 @@ func (h *Handlers) HandleAppRollback() http.HandlerFunc {
return return
} }
h.auditLog(request, models.AuditActionAppRollback,
models.AuditResourceApp, application.ID, "rolled back: "+application.Name)
http.Redirect(writer, request, "/apps/"+application.ID+"?success=rolledback", http.StatusSeeOther) http.Redirect(writer, request, "/apps/"+application.ID+"?success=rolledback", http.StatusSeeOther)
} }
} }
@@ -851,29 +830,11 @@ func (h *Handlers) handleContainerAction(
} else { } else {
h.log.Info("container action completed", h.log.Info("container action completed",
"action", action, "app", application.Name, "container", containerID) "action", action, "app", application.Name, "container", containerID)
auditAction := containerActionToAuditAction(action)
h.auditLog(request, auditAction,
models.AuditResourceApp, appID, string(action)+" container: "+application.Name)
} }
http.Redirect(writer, request, "/apps/"+appID, http.StatusSeeOther) http.Redirect(writer, request, "/apps/"+appID, http.StatusSeeOther)
} }
// containerActionToAuditAction maps container actions to audit actions.
func containerActionToAuditAction(action containerAction) models.AuditAction {
switch action {
case actionRestart:
return models.AuditActionAppRestart
case actionStop:
return models.AuditActionAppStop
case actionStart:
return models.AuditActionAppStart
default:
return models.AuditAction("app." + string(action))
}
}
// HandleAppRestart handles restarting an app's container. // HandleAppRestart handles restarting an app's container.
func (h *Handlers) HandleAppRestart() http.HandlerFunc { func (h *Handlers) HandleAppRestart() http.HandlerFunc {
return func(writer http.ResponseWriter, request *http.Request) { return func(writer http.ResponseWriter, request *http.Request) {
@@ -1023,10 +984,6 @@ func (h *Handlers) HandleEnvVarSave() http.HandlerFunc {
return return
} }
h.auditLog(request, models.AuditActionEnvVarSave,
models.AuditResourceEnvVar, application.ID,
fmt.Sprintf("saved %d env vars", len(modelPairs)))
h.respondJSON(writer, request, map[string]bool{"ok": true}, http.StatusOK) h.respondJSON(writer, request, map[string]bool{"ok": true}, http.StatusOK)
} }
} }
@@ -1043,13 +1000,7 @@ func (h *Handlers) HandleLabelAdd() http.HandlerFunc {
label.Key = key label.Key = key
label.Value = value label.Value = value
err := label.Save(ctx) return label.Save(ctx)
if err == nil {
h.auditLog(request, models.AuditActionLabelAdd,
models.AuditResourceLabel, application.ID, "added label: "+key)
}
return err
}, },
) )
} }
@@ -1078,9 +1029,6 @@ func (h *Handlers) HandleLabelDelete() http.HandlerFunc {
deleteErr := label.Delete(request.Context()) deleteErr := label.Delete(request.Context())
if deleteErr != nil { if deleteErr != nil {
h.log.Error("failed to delete label", "error", deleteErr) h.log.Error("failed to delete label", "error", deleteErr)
} else {
h.auditLog(request, models.AuditActionLabelDelete,
models.AuditResourceLabel, appID, "deleted label: "+label.Key)
} }
http.Redirect(writer, request, "/apps/"+appID, http.StatusSeeOther) http.Redirect(writer, request, "/apps/"+appID, http.StatusSeeOther)
@@ -1138,10 +1086,6 @@ func (h *Handlers) HandleVolumeAdd() http.HandlerFunc {
saveErr := volume.Save(request.Context()) saveErr := volume.Save(request.Context())
if saveErr != nil { if saveErr != nil {
h.log.Error("failed to add volume", "error", saveErr) h.log.Error("failed to add volume", "error", saveErr)
} else {
h.auditLog(request, models.AuditActionVolumeAdd,
models.AuditResourceVolume, application.ID,
"added volume: "+hostPath+":"+containerPath)
} }
http.Redirect(writer, request, "/apps/"+application.ID, http.StatusSeeOther) http.Redirect(writer, request, "/apps/"+application.ID, http.StatusSeeOther)
@@ -1171,10 +1115,6 @@ func (h *Handlers) HandleVolumeDelete() http.HandlerFunc {
deleteErr := volume.Delete(request.Context()) deleteErr := volume.Delete(request.Context())
if deleteErr != nil { if deleteErr != nil {
h.log.Error("failed to delete volume", "error", deleteErr) h.log.Error("failed to delete volume", "error", deleteErr)
} else {
h.auditLog(request, models.AuditActionVolumeDelete,
models.AuditResourceVolume, appID,
"deleted volume: "+volume.HostPath+":"+volume.ContainerPath)
} }
http.Redirect(writer, request, "/apps/"+appID, http.StatusSeeOther) http.Redirect(writer, request, "/apps/"+appID, http.StatusSeeOther)
@@ -1224,10 +1164,6 @@ func (h *Handlers) HandlePortAdd() http.HandlerFunc {
saveErr := port.Save(request.Context()) saveErr := port.Save(request.Context())
if saveErr != nil { if saveErr != nil {
h.log.Error("failed to save port", "error", saveErr) h.log.Error("failed to save port", "error", saveErr)
} else {
h.auditLog(request, models.AuditActionPortAdd,
models.AuditResourcePort, application.ID,
fmt.Sprintf("added port: %d:%d/%s", hostPort, containerPort, protocol))
} }
http.Redirect(writer, request, "/apps/"+application.ID, http.StatusSeeOther) http.Redirect(writer, request, "/apps/"+application.ID, http.StatusSeeOther)
@@ -1274,10 +1210,6 @@ func (h *Handlers) HandlePortDelete() http.HandlerFunc {
deleteErr := port.Delete(request.Context()) deleteErr := port.Delete(request.Context())
if deleteErr != nil { if deleteErr != nil {
h.log.Error("failed to delete port", "error", deleteErr) h.log.Error("failed to delete port", "error", deleteErr)
} else {
h.auditLog(request, models.AuditActionPortDelete,
models.AuditResourcePort, appID,
fmt.Sprintf("deleted port: %d:%d/%s", port.HostPort, port.ContainerPort, port.Protocol))
} }
http.Redirect(writer, request, "/apps/"+appID, http.StatusSeeOther) http.Redirect(writer, request, "/apps/"+appID, http.StatusSeeOther)
@@ -1353,9 +1285,6 @@ func (h *Handlers) HandleLabelEdit() http.HandlerFunc {
saveErr := label.Save(request.Context()) saveErr := label.Save(request.Context())
if saveErr != nil { if saveErr != nil {
h.log.Error("failed to update label", "error", saveErr) h.log.Error("failed to update label", "error", saveErr)
} else {
h.auditLog(request, models.AuditActionLabelEdit,
models.AuditResourceLabel, appID, "edited label: "+key)
} }
http.Redirect(writer, request, "/apps/"+appID, http.StatusSeeOther) http.Redirect(writer, request, "/apps/"+appID, http.StatusSeeOther)
@@ -1414,10 +1343,6 @@ func (h *Handlers) HandleVolumeEdit() http.HandlerFunc {
saveErr := volume.Save(request.Context()) saveErr := volume.Save(request.Context())
if saveErr != nil { if saveErr != nil {
h.log.Error("failed to update volume", "error", saveErr) h.log.Error("failed to update volume", "error", saveErr)
} else {
h.auditLog(request, models.AuditActionVolumeEdit,
models.AuditResourceVolume, appID,
"edited volume: "+hostPath+":"+containerPath)
} }
http.Redirect(writer, request, "/apps/"+appID, http.StatusSeeOther) http.Redirect(writer, request, "/apps/"+appID, http.StatusSeeOther)
@@ -1439,6 +1364,129 @@ func validateVolumePaths(hostPath, containerPath string) error {
return nil return nil
} }
// ErrInvalidMemoryFormat is returned when a memory limit string cannot be parsed.
var ErrInvalidMemoryFormat = errors.New(
"must be a number with optional unit suffix (e.g. 256m, 1g, 512000000)",
)
// ErrNegativeValue is returned when a resource limit is negative.
var ErrNegativeValue = errors.New("value must be positive")
// Memory unit byte multipliers.
const (
kilobyte = 1024
megabyte = 1024 * 1024
gigabyte = 1024 * 1024 * 1024
)
// optionalNullString converts a form value to a sql.NullString.
// Returns a valid NullString if non-empty, invalid (NULL) if empty.
func optionalNullString(s string) sql.NullString {
if s != "" {
return sql.NullString{String: s, Valid: true}
}
return sql.NullString{}
}
// applyResourceLimits parses CPU and memory limit form values and applies them to the app.
// Returns an error message string if validation fails, or empty string on success.
func applyResourceLimits(application *models.App, request *http.Request) string {
cpuLimit, cpuErr := parseOptionalFloat64(request.FormValue("cpu_limit"))
if cpuErr != nil {
return "Invalid CPU limit: must be a positive number (e.g. 0.5, 1, 2)"
}
application.CPULimit = cpuLimit
memoryLimit, memErr := parseOptionalMemoryBytes(request.FormValue("memory_limit"))
if memErr != nil {
return "Invalid memory limit: " + memErr.Error()
}
application.MemoryLimit = memoryLimit
return ""
}
// memoryUnitMultiplier returns the byte multiplier for a memory unit suffix.
// Returns 0 if the suffix is not recognized.
func memoryUnitMultiplier(suffix byte) int64 {
switch suffix {
case 'k':
return kilobyte
case 'm':
return megabyte
case 'g':
return gigabyte
default:
return 0
}
}
// parseOptionalFloat64 parses an optional float64 form field.
// Returns a valid NullFloat64 if the string is non-empty and parses to a positive number.
// Returns an empty NullFloat64 if the string is empty.
// Returns an error if the string is non-empty but invalid or non-positive.
func parseOptionalFloat64(s string) (sql.NullFloat64, error) {
s = strings.TrimSpace(s)
if s == "" {
return sql.NullFloat64{}, nil
}
val, err := strconv.ParseFloat(s, 64)
if err != nil {
return sql.NullFloat64{}, fmt.Errorf("invalid number: %w", err)
}
if val <= 0 {
return sql.NullFloat64{}, ErrNegativeValue
}
return sql.NullFloat64{Float64: val, Valid: true}, nil
}
// parseOptionalMemoryBytes parses an optional memory limit string into bytes.
// Accepts plain bytes (e.g. "536870912") or suffixed values (e.g. "512m", "1g", "256k").
// Returns a valid NullInt64 with bytes if non-empty, empty NullInt64 if blank.
func parseOptionalMemoryBytes(s string) (sql.NullInt64, error) {
s = strings.TrimSpace(s)
if s == "" {
return sql.NullInt64{}, nil
}
s = strings.ToLower(s)
// Check for unit suffix
multiplier := memoryUnitMultiplier(s[len(s)-1])
if multiplier > 0 {
numStr := s[:len(s)-1]
val, err := strconv.ParseFloat(numStr, 64)
if err != nil {
return sql.NullInt64{}, ErrInvalidMemoryFormat
}
if val <= 0 {
return sql.NullInt64{}, ErrNegativeValue
}
return sql.NullInt64{Int64: int64(val * float64(multiplier)), Valid: true}, nil
}
// Plain bytes
val, err := strconv.ParseInt(s, 10, 64)
if err != nil {
return sql.NullInt64{}, ErrInvalidMemoryFormat
}
if val <= 0 {
return sql.NullInt64{}, ErrNegativeValue
}
return sql.NullInt64{Int64: val, Valid: true}, nil
}
// formatDeployKey formats an SSH public key with a descriptive comment. // formatDeployKey formats an SSH public key with a descriptive comment.
// Format: ssh-ed25519 AAAA... upaas_2025-01-15_myapp // Format: ssh-ed25519 AAAA... upaas_2025-01-15_myapp
func formatDeployKey(pubKey string, createdAt time.Time, appName string) string { func formatDeployKey(pubKey string, createdAt time.Time, appName string) string {

View File

@@ -3,7 +3,6 @@ package handlers
import ( import (
"net/http" "net/http"
"sneak.berlin/go/upaas/internal/models"
"sneak.berlin/go/upaas/templates" "sneak.berlin/go/upaas/templates"
) )
@@ -62,9 +61,6 @@ func (h *Handlers) HandleLoginPOST() http.HandlerFunc {
return return
} }
h.auditLog(request, models.AuditActionLogin,
models.AuditResourceSession, "", "user logged in")
http.Redirect(writer, request, "/", http.StatusSeeOther) http.Redirect(writer, request, "/", http.StatusSeeOther)
} }
} }
@@ -72,9 +68,6 @@ func (h *Handlers) HandleLoginPOST() http.HandlerFunc {
// HandleLogout handles logout requests. // HandleLogout handles logout requests.
func (h *Handlers) HandleLogout() http.HandlerFunc { func (h *Handlers) HandleLogout() http.HandlerFunc {
return func(writer http.ResponseWriter, request *http.Request) { return func(writer http.ResponseWriter, request *http.Request) {
h.auditLog(request, models.AuditActionLogout,
models.AuditResourceSession, "", "user logged out")
destroyErr := h.auth.DestroySession(writer, request) destroyErr := h.auth.DestroySession(writer, request)
if destroyErr != nil { if destroyErr != nil {
h.log.Error("failed to destroy session", "error", destroyErr) h.log.Error("failed to destroy session", "error", destroyErr)

View File

@@ -15,9 +15,7 @@ import (
"sneak.berlin/go/upaas/internal/globals" "sneak.berlin/go/upaas/internal/globals"
"sneak.berlin/go/upaas/internal/healthcheck" "sneak.berlin/go/upaas/internal/healthcheck"
"sneak.berlin/go/upaas/internal/logger" "sneak.berlin/go/upaas/internal/logger"
"sneak.berlin/go/upaas/internal/models"
"sneak.berlin/go/upaas/internal/service/app" "sneak.berlin/go/upaas/internal/service/app"
"sneak.berlin/go/upaas/internal/service/audit"
"sneak.berlin/go/upaas/internal/service/auth" "sneak.berlin/go/upaas/internal/service/auth"
"sneak.berlin/go/upaas/internal/service/deploy" "sneak.berlin/go/upaas/internal/service/deploy"
"sneak.berlin/go/upaas/internal/service/webhook" "sneak.berlin/go/upaas/internal/service/webhook"
@@ -37,7 +35,6 @@ type Params struct {
Deploy *deploy.Service Deploy *deploy.Service
Webhook *webhook.Service Webhook *webhook.Service
Docker *docker.Client Docker *docker.Client
Audit *audit.Service
} }
// Handlers provides HTTP request handlers. // Handlers provides HTTP request handlers.
@@ -51,7 +48,6 @@ type Handlers struct {
deploy *deploy.Service deploy *deploy.Service
webhook *webhook.Service webhook *webhook.Service
docker *docker.Client docker *docker.Client
audit *audit.Service
globals *globals.Globals globals *globals.Globals
} }
@@ -67,48 +63,10 @@ func New(_ fx.Lifecycle, params Params) (*Handlers, error) {
deploy: params.Deploy, deploy: params.Deploy,
webhook: params.Webhook, webhook: params.Webhook,
docker: params.Docker, docker: params.Docker,
audit: params.Audit,
globals: params.Globals, globals: params.Globals,
}, nil }, nil
} }
// currentUser returns the currently authenticated user, or nil if not authenticated.
func (h *Handlers) currentUser(request *http.Request) *models.User {
user, err := h.auth.GetCurrentUser(request.Context(), request)
if err != nil || user == nil {
return nil
}
return user
}
// auditLog records an audit entry for the current request.
func (h *Handlers) auditLog(
request *http.Request,
action models.AuditAction,
resourceType models.AuditResourceType,
resourceID string,
detail string,
) {
user := h.currentUser(request)
entry := audit.LogEntry{
Action: action,
ResourceType: resourceType,
ResourceID: resourceID,
Detail: detail,
}
if user != nil {
entry.UserID = user.ID
entry.Username = user.Username
} else {
entry.Username = "anonymous"
}
h.audit.LogFromRequest(request.Context(), request, entry)
}
// addGlobals adds version info and CSRF token to template data map. // addGlobals adds version info and CSRF token to template data map.
func (h *Handlers) addGlobals( func (h *Handlers) addGlobals(
data map[string]any, data map[string]any,

View File

@@ -11,7 +11,6 @@ import (
"time" "time"
"github.com/go-chi/chi/v5" "github.com/go-chi/chi/v5"
"github.com/prometheus/client_golang/prometheus"
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
"go.uber.org/fx" "go.uber.org/fx"
@@ -25,10 +24,8 @@ import (
"sneak.berlin/go/upaas/internal/handlers" "sneak.berlin/go/upaas/internal/handlers"
"sneak.berlin/go/upaas/internal/healthcheck" "sneak.berlin/go/upaas/internal/healthcheck"
"sneak.berlin/go/upaas/internal/logger" "sneak.berlin/go/upaas/internal/logger"
"sneak.berlin/go/upaas/internal/metrics"
"sneak.berlin/go/upaas/internal/middleware" "sneak.berlin/go/upaas/internal/middleware"
"sneak.berlin/go/upaas/internal/service/app" "sneak.berlin/go/upaas/internal/service/app"
"sneak.berlin/go/upaas/internal/service/audit"
"sneak.berlin/go/upaas/internal/service/auth" "sneak.berlin/go/upaas/internal/service/auth"
"sneak.berlin/go/upaas/internal/service/deploy" "sneak.berlin/go/upaas/internal/service/deploy"
"sneak.berlin/go/upaas/internal/service/notify" "sneak.berlin/go/upaas/internal/service/notify"
@@ -95,8 +92,7 @@ func createAppServices(
logInstance *logger.Logger, logInstance *logger.Logger,
dbInstance *database.Database, dbInstance *database.Database,
cfg *config.Config, cfg *config.Config,
metricsInstance *metrics.Metrics, ) (*auth.Service, *app.Service, *deploy.Service, *webhook.Service, *docker.Client) {
) (*auth.Service, *app.Service, *deploy.Service, *webhook.Service, *docker.Client, *audit.Service) {
t.Helper() t.Helper()
authSvc, authErr := auth.New(fx.Lifecycle(nil), auth.ServiceParams{ authSvc, authErr := auth.New(fx.Lifecycle(nil), auth.ServiceParams{
@@ -129,7 +125,6 @@ func createAppServices(
Database: dbInstance, Database: dbInstance,
Docker: dockerClient, Docker: dockerClient,
Notify: notifySvc, Notify: notifySvc,
Metrics: metricsInstance,
}) })
require.NoError(t, deployErr) require.NoError(t, deployErr)
@@ -137,18 +132,10 @@ func createAppServices(
Logger: logInstance, Logger: logInstance,
Database: dbInstance, Database: dbInstance,
Deploy: deploySvc, Deploy: deploySvc,
Metrics: metricsInstance,
}) })
require.NoError(t, webhookErr) require.NoError(t, webhookErr)
auditSvc, auditErr := audit.New(fx.Lifecycle(nil), audit.ServiceParams{ return authSvc, appSvc, deploySvc, webhookSvc, dockerClient
Logger: logInstance,
Database: dbInstance,
Metrics: metricsInstance,
})
require.NoError(t, auditErr)
return authSvc, appSvc, deploySvc, webhookSvc, dockerClient, auditSvc
} }
func setupTestHandlers(t *testing.T) *testContext { func setupTestHandlers(t *testing.T) *testContext {
@@ -158,14 +145,11 @@ func setupTestHandlers(t *testing.T) *testContext {
globalInstance, logInstance, dbInstance, hcInstance := createCoreServices(t, cfg) globalInstance, logInstance, dbInstance, hcInstance := createCoreServices(t, cfg)
metricsInstance := metrics.NewForTest(prometheus.NewRegistry()) authSvc, appSvc, deploySvc, webhookSvc, dockerClient := createAppServices(
authSvc, appSvc, deploySvc, webhookSvc, dockerClient, auditSvc := createAppServices(
t, t,
logInstance, logInstance,
dbInstance, dbInstance,
cfg, cfg,
metricsInstance,
) )
handlersInstance, handlerErr := handlers.New( handlersInstance, handlerErr := handlers.New(
@@ -180,7 +164,6 @@ func setupTestHandlers(t *testing.T) *testContext {
Deploy: deploySvc, Deploy: deploySvc,
Webhook: webhookSvc, Webhook: webhookSvc,
Docker: dockerClient, Docker: dockerClient,
Audit: auditSvc,
}, },
) )
require.NoError(t, handlerErr) require.NoError(t, handlerErr)
@@ -190,7 +173,6 @@ func setupTestHandlers(t *testing.T) *testContext {
Globals: globalInstance, Globals: globalInstance,
Config: cfg, Config: cfg,
Auth: authSvc, Auth: authSvc,
Metrics: metricsInstance,
}) })
require.NoError(t, mwErr) require.NoError(t, mwErr)

View File

@@ -0,0 +1,195 @@
package handlers //nolint:testpackage // tests unexported parsing functions
import (
"database/sql"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestParseOptionalFloat64(t *testing.T) {
t.Parallel()
t.Run("empty string returns invalid", func(t *testing.T) {
t.Parallel()
result, err := parseOptionalFloat64("")
require.NoError(t, err)
assert.False(t, result.Valid)
})
t.Run("whitespace only returns invalid", func(t *testing.T) {
t.Parallel()
result, err := parseOptionalFloat64(" ")
require.NoError(t, err)
assert.False(t, result.Valid)
})
t.Run("valid float", func(t *testing.T) {
t.Parallel()
result, err := parseOptionalFloat64("0.5")
require.NoError(t, err)
assert.True(t, result.Valid)
assert.InDelta(t, 0.5, result.Float64, 0.001)
})
t.Run("valid integer", func(t *testing.T) {
t.Parallel()
result, err := parseOptionalFloat64("2")
require.NoError(t, err)
assert.True(t, result.Valid)
assert.InDelta(t, 2.0, result.Float64, 0.001)
})
t.Run("negative value rejected", func(t *testing.T) {
t.Parallel()
_, err := parseOptionalFloat64("-1")
require.Error(t, err)
})
t.Run("zero value rejected", func(t *testing.T) {
t.Parallel()
_, err := parseOptionalFloat64("0")
require.Error(t, err)
})
t.Run("non-numeric rejected", func(t *testing.T) {
t.Parallel()
_, err := parseOptionalFloat64("abc")
require.Error(t, err)
})
}
func TestParseOptionalMemoryBytes(t *testing.T) { //nolint:funlen // table-driven test
t.Parallel()
t.Run("empty string returns invalid", func(t *testing.T) {
t.Parallel()
result, err := parseOptionalMemoryBytes("")
require.NoError(t, err)
assert.False(t, result.Valid)
})
t.Run("whitespace only returns invalid", func(t *testing.T) {
t.Parallel()
result, err := parseOptionalMemoryBytes(" ")
require.NoError(t, err)
assert.False(t, result.Valid)
})
t.Run("plain bytes", func(t *testing.T) {
t.Parallel()
result, err := parseOptionalMemoryBytes("536870912")
require.NoError(t, err)
assert.True(t, result.Valid)
assert.Equal(t, int64(536870912), result.Int64)
})
t.Run("megabytes suffix", func(t *testing.T) {
t.Parallel()
result, err := parseOptionalMemoryBytes("256m")
require.NoError(t, err)
assert.True(t, result.Valid)
assert.Equal(t, int64(256*1024*1024), result.Int64)
})
t.Run("megabytes suffix uppercase", func(t *testing.T) {
t.Parallel()
result, err := parseOptionalMemoryBytes("256M")
require.NoError(t, err)
assert.True(t, result.Valid)
assert.Equal(t, int64(256*1024*1024), result.Int64)
})
t.Run("gigabytes suffix", func(t *testing.T) {
t.Parallel()
result, err := parseOptionalMemoryBytes("1g")
require.NoError(t, err)
assert.True(t, result.Valid)
assert.Equal(t, int64(1024*1024*1024), result.Int64)
})
t.Run("kilobytes suffix", func(t *testing.T) {
t.Parallel()
result, err := parseOptionalMemoryBytes("512k")
require.NoError(t, err)
assert.True(t, result.Valid)
assert.Equal(t, int64(512*1024), result.Int64)
})
t.Run("fractional gigabytes", func(t *testing.T) {
t.Parallel()
result, err := parseOptionalMemoryBytes("1.5g")
require.NoError(t, err)
assert.True(t, result.Valid)
assert.Equal(t, int64(1.5*1024*1024*1024), result.Int64)
})
t.Run("negative value rejected", func(t *testing.T) {
t.Parallel()
_, err := parseOptionalMemoryBytes("-256m")
require.Error(t, err)
})
t.Run("zero value rejected", func(t *testing.T) {
t.Parallel()
_, err := parseOptionalMemoryBytes("0")
require.Error(t, err)
})
t.Run("invalid string rejected", func(t *testing.T) {
t.Parallel()
_, err := parseOptionalMemoryBytes("abc")
require.Error(t, err)
})
t.Run("negative plain bytes rejected", func(t *testing.T) {
t.Parallel()
_, err := parseOptionalMemoryBytes("-100")
require.Error(t, err)
})
}
func TestAppResourceLimitsRoundTrip(t *testing.T) {
t.Parallel()
// Test that parsing and formatting are consistent
tests := []struct {
input string
expected sql.NullInt64
format string
}{
{"256m", sql.NullInt64{Int64: 256 * 1024 * 1024, Valid: true}, "256m"},
{"1g", sql.NullInt64{Int64: 1024 * 1024 * 1024, Valid: true}, "1g"},
{"512k", sql.NullInt64{Int64: 512 * 1024, Valid: true}, "512k"},
}
for _, tt := range tests {
t.Run(tt.input, func(t *testing.T) {
t.Parallel()
result, err := parseOptionalMemoryBytes(tt.input)
require.NoError(t, err)
assert.Equal(t, tt.expected, result)
})
}
}

View File

@@ -3,7 +3,6 @@ package handlers
import ( import (
"net/http" "net/http"
"sneak.berlin/go/upaas/internal/models"
"sneak.berlin/go/upaas/templates" "sneak.berlin/go/upaas/templates"
) )
@@ -112,9 +111,6 @@ func (h *Handlers) HandleSetupPOST() http.HandlerFunc {
return return
} }
h.auditLog(request, models.AuditActionSetup,
models.AuditResourceUser, "", "initial setup completed")
http.Redirect(writer, request, "/", http.StatusSeeOther) http.Redirect(writer, request, "/", http.StatusSeeOther)
} }
} }

View File

@@ -7,14 +7,13 @@ import (
"github.com/go-chi/chi/v5" "github.com/go-chi/chi/v5"
"sneak.berlin/go/upaas/internal/models" "sneak.berlin/go/upaas/internal/models"
"sneak.berlin/go/upaas/internal/service/audit"
) )
// maxWebhookBodySize is the maximum allowed size of a webhook request body (1MB). // maxWebhookBodySize is the maximum allowed size of a webhook request body (1MB).
const maxWebhookBodySize = 1 << 20 const maxWebhookBodySize = 1 << 20
// HandleWebhook handles incoming Gitea webhooks. // HandleWebhook handles incoming Gitea webhooks.
func (h *Handlers) HandleWebhook() http.HandlerFunc { //nolint:funlen // audit logging adds necessary length func (h *Handlers) HandleWebhook() http.HandlerFunc {
return func(writer http.ResponseWriter, request *http.Request) { return func(writer http.ResponseWriter, request *http.Request) {
secret := chi.URLParam(request, "secret") secret := chi.URLParam(request, "secret")
if secret == "" { if secret == "" {
@@ -57,15 +56,6 @@ func (h *Handlers) HandleWebhook() http.HandlerFunc { //nolint:funlen // audit l
eventType = "push" eventType = "push"
} }
// Log webhook receipt
h.audit.LogFromRequest(request.Context(), request, audit.LogEntry{
Username: "webhook",
Action: models.AuditActionWebhookReceive,
ResourceType: models.AuditResourceWebhook,
ResourceID: application.ID,
Detail: "webhook from app: " + application.Name + ", event: " + eventType,
})
// Process webhook // Process webhook
webhookErr := h.webhook.HandleWebhook( webhookErr := h.webhook.HandleWebhook(
request.Context(), request.Context(),

View File

@@ -1,148 +0,0 @@
// Package metrics provides Prometheus metrics for upaas.
//
//nolint:revive // "metrics" matches the domain; runtime/metrics is rarely imported directly
package metrics
import (
"github.com/prometheus/client_golang/prometheus"
"github.com/prometheus/client_golang/prometheus/promauto"
"go.uber.org/fx"
)
// Params contains dependencies for Metrics.
type Params struct {
fx.In
}
// Metrics holds all Prometheus metrics for the application.
type Metrics struct {
// Deployment metrics.
DeploymentsTotal *prometheus.CounterVec
DeploymentDuration *prometheus.HistogramVec
DeploymentsInFlight *prometheus.GaugeVec
// Container health metrics.
ContainerHealthy *prometheus.GaugeVec
// Webhook metrics.
WebhookEventsTotal *prometheus.CounterVec
// HTTP request metrics.
HTTPRequestsTotal *prometheus.CounterVec
HTTPRequestDuration *prometheus.HistogramVec
HTTPResponseSizeBytes *prometheus.HistogramVec
// Audit log metrics.
AuditEventsTotal *prometheus.CounterVec
}
// New creates a new Metrics instance with all Prometheus metrics registered
// in the default Prometheus registry.
func New(_ fx.Lifecycle, _ Params) (*Metrics, error) {
return newMetrics(promauto.With(prometheus.DefaultRegisterer)), nil
}
// NewForTest creates a Metrics instance with a custom registry for test isolation.
func NewForTest(reg prometheus.Registerer) *Metrics {
return newMetrics(promauto.With(reg))
}
// newMetrics creates a Metrics instance using the given factory.
func newMetrics(factory promauto.Factory) *Metrics {
return &Metrics{
DeploymentsTotal: newDeploymentsTotal(factory),
DeploymentDuration: newDeploymentDuration(factory),
DeploymentsInFlight: newDeploymentsInFlight(factory),
ContainerHealthy: newContainerHealthy(factory),
WebhookEventsTotal: newWebhookEventsTotal(factory),
HTTPRequestsTotal: newHTTPRequestsTotal(factory),
HTTPRequestDuration: newHTTPRequestDuration(factory),
HTTPResponseSizeBytes: newHTTPResponseSizeBytes(factory),
AuditEventsTotal: newAuditEventsTotal(factory),
}
}
func newDeploymentsTotal(f promauto.Factory) *prometheus.CounterVec {
return f.NewCounterVec(prometheus.CounterOpts{
Namespace: "upaas",
Subsystem: "deployments",
Name: "total",
Help: "Total number of deployments by app and status.",
}, []string{"app", "status"})
}
func newDeploymentDuration(f promauto.Factory) *prometheus.HistogramVec {
return f.NewHistogramVec(prometheus.HistogramOpts{
Namespace: "upaas",
Subsystem: "deployments",
Name: "duration_seconds",
Help: "Duration of deployments in seconds by app and status.",
Buckets: []float64{10, 30, 60, 120, 300, 600, 1800},
}, []string{"app", "status"})
}
func newDeploymentsInFlight(f promauto.Factory) *prometheus.GaugeVec {
return f.NewGaugeVec(prometheus.GaugeOpts{
Namespace: "upaas",
Subsystem: "deployments",
Name: "in_flight",
Help: "Number of deployments currently in progress by app.",
}, []string{"app"})
}
func newContainerHealthy(f promauto.Factory) *prometheus.GaugeVec {
return f.NewGaugeVec(prometheus.GaugeOpts{
Namespace: "upaas",
Subsystem: "container",
Name: "healthy",
Help: "Whether the app container is healthy (1) or unhealthy (0).",
}, []string{"app"})
}
func newWebhookEventsTotal(f promauto.Factory) *prometheus.CounterVec {
return f.NewCounterVec(prometheus.CounterOpts{
Namespace: "upaas",
Subsystem: "webhook",
Name: "events_total",
Help: "Total number of webhook events by app, event type, and matched status.",
}, []string{"app", "event_type", "matched"})
}
func newHTTPRequestsTotal(f promauto.Factory) *prometheus.CounterVec {
return f.NewCounterVec(prometheus.CounterOpts{
Namespace: "upaas",
Subsystem: "http",
Name: "requests_total",
Help: "Total number of HTTP requests by method and status code.",
}, []string{"method", "status_code"})
}
func newHTTPRequestDuration(f promauto.Factory) *prometheus.HistogramVec {
return f.NewHistogramVec(prometheus.HistogramOpts{
Namespace: "upaas",
Subsystem: "http",
Name: "request_duration_seconds",
Help: "Duration of HTTP requests in seconds by method.",
Buckets: prometheus.DefBuckets,
}, []string{"method"})
}
//nolint:mnd // bucket boundaries are domain-specific constants
func newHTTPResponseSizeBytes(f promauto.Factory) *prometheus.HistogramVec {
return f.NewHistogramVec(prometheus.HistogramOpts{
Namespace: "upaas",
Subsystem: "http",
Name: "response_size_bytes",
Help: "Size of HTTP responses in bytes by method.",
Buckets: prometheus.ExponentialBuckets(100, 10, 7),
}, []string{"method"})
}
func newAuditEventsTotal(f promauto.Factory) *prometheus.CounterVec {
return f.NewCounterVec(prometheus.CounterOpts{
Namespace: "upaas",
Subsystem: "audit",
Name: "events_total",
Help: "Total number of audit log events by action.",
}, []string{"action"})
}

View File

@@ -1,158 +0,0 @@
package metrics_test
import (
"testing"
"github.com/prometheus/client_golang/prometheus"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"go.uber.org/fx"
"sneak.berlin/go/upaas/internal/metrics"
)
func TestNewForTest(t *testing.T) {
t.Parallel()
reg := prometheus.NewRegistry()
m := metrics.NewForTest(reg)
require.NotNil(t, m)
assert.NotNil(t, m.DeploymentsTotal)
assert.NotNil(t, m.DeploymentDuration)
assert.NotNil(t, m.DeploymentsInFlight)
assert.NotNil(t, m.ContainerHealthy)
assert.NotNil(t, m.WebhookEventsTotal)
assert.NotNil(t, m.HTTPRequestsTotal)
assert.NotNil(t, m.HTTPRequestDuration)
assert.NotNil(t, m.HTTPResponseSizeBytes)
assert.NotNil(t, m.AuditEventsTotal)
}
func TestNew(t *testing.T) {
t.Parallel()
m, err := metrics.New(fx.Lifecycle(nil), metrics.Params{})
require.NoError(t, err)
require.NotNil(t, m)
}
func TestDeploymentMetrics(t *testing.T) {
t.Parallel()
reg := prometheus.NewRegistry()
m := metrics.NewForTest(reg)
m.DeploymentsTotal.WithLabelValues("test-app", "success").Inc()
m.DeploymentDuration.WithLabelValues("test-app", "success").Observe(42.5)
m.DeploymentsInFlight.WithLabelValues("test-app").Set(1)
families, err := reg.Gather()
require.NoError(t, err)
names := make(map[string]bool)
for _, f := range families {
names[f.GetName()] = true
}
assert.True(t, names["upaas_deployments_total"])
assert.True(t, names["upaas_deployments_duration_seconds"])
assert.True(t, names["upaas_deployments_in_flight"])
}
func TestContainerHealthMetrics(t *testing.T) {
t.Parallel()
reg := prometheus.NewRegistry()
m := metrics.NewForTest(reg)
m.ContainerHealthy.WithLabelValues("my-app").Set(1)
families, err := reg.Gather()
require.NoError(t, err)
found := false
for _, f := range families {
if f.GetName() == "upaas_container_healthy" {
found = true
break
}
}
assert.True(t, found)
}
func TestWebhookMetrics(t *testing.T) {
t.Parallel()
reg := prometheus.NewRegistry()
m := metrics.NewForTest(reg)
m.WebhookEventsTotal.WithLabelValues("test-app", "push", "true").Inc()
families, err := reg.Gather()
require.NoError(t, err)
found := false
for _, f := range families {
if f.GetName() == "upaas_webhook_events_total" {
found = true
break
}
}
assert.True(t, found)
}
func TestHTTPMetrics(t *testing.T) {
t.Parallel()
reg := prometheus.NewRegistry()
m := metrics.NewForTest(reg)
m.HTTPRequestsTotal.WithLabelValues("GET", "200").Inc()
m.HTTPRequestDuration.WithLabelValues("GET").Observe(0.05)
m.HTTPResponseSizeBytes.WithLabelValues("GET").Observe(1024)
families, err := reg.Gather()
require.NoError(t, err)
names := make(map[string]bool)
for _, f := range families {
names[f.GetName()] = true
}
assert.True(t, names["upaas_http_requests_total"])
assert.True(t, names["upaas_http_request_duration_seconds"])
assert.True(t, names["upaas_http_response_size_bytes"])
}
func TestAuditMetrics(t *testing.T) {
t.Parallel()
reg := prometheus.NewRegistry()
m := metrics.NewForTest(reg)
m.AuditEventsTotal.WithLabelValues("login").Inc()
families, err := reg.Gather()
require.NoError(t, err)
found := false
for _, f := range families {
if f.GetName() == "upaas_audit_events_total" {
found = true
break
}
}
assert.True(t, found)
}

View File

@@ -21,7 +21,6 @@ import (
"sneak.berlin/go/upaas/internal/config" "sneak.berlin/go/upaas/internal/config"
"sneak.berlin/go/upaas/internal/globals" "sneak.berlin/go/upaas/internal/globals"
"sneak.berlin/go/upaas/internal/logger" "sneak.berlin/go/upaas/internal/logger"
"sneak.berlin/go/upaas/internal/metrics"
"sneak.berlin/go/upaas/internal/service/auth" "sneak.berlin/go/upaas/internal/service/auth"
) )
@@ -36,37 +35,33 @@ type Params struct {
Globals *globals.Globals Globals *globals.Globals
Config *config.Config Config *config.Config
Auth *auth.Service Auth *auth.Service
Metrics *metrics.Metrics
} }
// Middleware provides HTTP middleware. // Middleware provides HTTP middleware.
type Middleware struct { type Middleware struct {
log *slog.Logger log *slog.Logger
metrics *metrics.Metrics params *Params
params *Params
} }
// New creates a new Middleware instance. // New creates a new Middleware instance.
func New(_ fx.Lifecycle, params Params) (*Middleware, error) { func New(_ fx.Lifecycle, params Params) (*Middleware, error) {
return &Middleware{ return &Middleware{
log: params.Logger.Get(), log: params.Logger.Get(),
metrics: params.Metrics, params: &params,
params: &params,
}, nil }, nil
} }
// loggingResponseWriter wraps http.ResponseWriter to capture status code and bytes written. // loggingResponseWriter wraps http.ResponseWriter to capture status code.
type loggingResponseWriter struct { type loggingResponseWriter struct {
http.ResponseWriter http.ResponseWriter
statusCode int statusCode int
bytesWritten int
} }
func newLoggingResponseWriter( func newLoggingResponseWriter(
writer http.ResponseWriter, writer http.ResponseWriter,
) *loggingResponseWriter { ) *loggingResponseWriter {
return &loggingResponseWriter{ResponseWriter: writer, statusCode: http.StatusOK} return &loggingResponseWriter{writer, http.StatusOK}
} }
func (lrw *loggingResponseWriter) WriteHeader(code int) { func (lrw *loggingResponseWriter) WriteHeader(code int) {
@@ -74,14 +69,7 @@ func (lrw *loggingResponseWriter) WriteHeader(code int) {
lrw.ResponseWriter.WriteHeader(code) lrw.ResponseWriter.WriteHeader(code)
} }
func (lrw *loggingResponseWriter) Write(b []byte) (int, error) { // Logging returns a request logging middleware.
n, err := lrw.ResponseWriter.Write(b)
lrw.bytesWritten += n
return n, err
}
// Logging returns a request logging middleware that also records HTTP metrics.
func (m *Middleware) Logging() func(http.Handler) http.Handler { func (m *Middleware) Logging() func(http.Handler) http.Handler {
return func(next http.Handler) http.Handler { return func(next http.Handler) http.Handler {
return http.HandlerFunc(func( return http.HandlerFunc(func(
@@ -95,8 +83,6 @@ func (m *Middleware) Logging() func(http.Handler) http.Handler {
defer func() { defer func() {
latency := time.Since(start) latency := time.Since(start)
reqID := middleware.GetReqID(ctx) reqID := middleware.GetReqID(ctx)
statusStr := strconv.Itoa(lrw.statusCode)
m.log.InfoContext(ctx, "request", m.log.InfoContext(ctx, "request",
"request_start", start, "request_start", start,
"method", request.Method, "method", request.Method,
@@ -105,21 +91,10 @@ func (m *Middleware) Logging() func(http.Handler) http.Handler {
"request_id", reqID, "request_id", reqID,
"referer", request.Referer(), "referer", request.Referer(),
"proto", request.Proto, "proto", request.Proto,
"remoteIP", RealIP(request), "remoteIP", realIP(request),
"status", lrw.statusCode, "status", lrw.statusCode,
"bytes", lrw.bytesWritten,
"latency_ms", latency.Milliseconds(), "latency_ms", latency.Milliseconds(),
) )
m.metrics.HTTPRequestsTotal.WithLabelValues(
request.Method, statusStr,
).Inc()
m.metrics.HTTPRequestDuration.WithLabelValues(
request.Method,
).Observe(latency.Seconds())
m.metrics.HTTPResponseSizeBytes.WithLabelValues(
request.Method,
).Observe(float64(lrw.bytesWritten))
}() }()
next.ServeHTTP(lrw, request) next.ServeHTTP(lrw, request)
@@ -170,11 +145,11 @@ func isTrustedProxy(ip net.IP) bool {
return false return false
} }
// RealIP extracts the client's real IP address from the request. // realIP extracts the client's real IP address from the request.
// Proxy headers (X-Real-IP, X-Forwarded-For) are only trusted when the // Proxy headers (X-Real-IP, X-Forwarded-For) are only trusted when the
// direct connection originates from an RFC1918/loopback address. // direct connection originates from an RFC1918/loopback address.
// Otherwise, headers are ignored and RemoteAddr is used (fail closed). // Otherwise, headers are ignored and RemoteAddr is used (fail closed).
func RealIP(r *http.Request) string { func realIP(r *http.Request) string {
addr := ipFromHostPort(r.RemoteAddr) addr := ipFromHostPort(r.RemoteAddr)
remoteIP := net.ParseIP(addr) remoteIP := net.ParseIP(addr)
@@ -365,7 +340,7 @@ func (m *Middleware) LoginRateLimit() func(http.Handler) http.Handler {
writer http.ResponseWriter, writer http.ResponseWriter,
request *http.Request, request *http.Request,
) { ) {
ip := RealIP(request) ip := realIP(request)
limiter := loginLimiter.getLimiter(ip) limiter := loginLimiter.getLimiter(ip)
if !limiter.Allow() { if !limiter.Allow() {

View File

@@ -1,4 +1,4 @@
package middleware //nolint:testpackage // tests RealIP via internal package access package middleware //nolint:testpackage // tests unexported realIP function
import ( import (
"context" "context"
@@ -126,9 +126,9 @@ func TestRealIP(t *testing.T) { //nolint:funlen // table-driven test
req.Header.Set("X-Forwarded-For", tt.xff) req.Header.Set("X-Forwarded-For", tt.xff)
} }
got := RealIP(req) got := realIP(req)
if got != tt.want { if got != tt.want {
t.Errorf("RealIP() = %q, want %q", got, tt.want) t.Errorf("realIP() = %q, want %q", got, tt.want)
} }
}) })
} }

View File

@@ -14,7 +14,8 @@ import (
const appColumns = `id, name, repo_url, branch, dockerfile_path, webhook_secret, const appColumns = `id, name, repo_url, branch, dockerfile_path, webhook_secret,
ssh_private_key, ssh_public_key, image_id, status, ssh_private_key, ssh_public_key, image_id, status,
docker_network, ntfy_topic, slack_webhook, webhook_secret_hash, docker_network, ntfy_topic, slack_webhook, webhook_secret_hash,
previous_image_id, created_at, updated_at` previous_image_id, cpu_limit, memory_limit,
created_at, updated_at`
// AppStatus represents the status of an app. // AppStatus represents the status of an app.
type AppStatus string type AppStatus string
@@ -47,6 +48,8 @@ type App struct {
DockerNetwork sql.NullString DockerNetwork sql.NullString
NtfyTopic sql.NullString NtfyTopic sql.NullString
SlackWebhook sql.NullString SlackWebhook sql.NullString
CPULimit sql.NullFloat64
MemoryLimit sql.NullInt64
CreatedAt time.Time CreatedAt time.Time
UpdatedAt time.Time UpdatedAt time.Time
} }
@@ -142,14 +145,14 @@ func (a *App) insert(ctx context.Context) error {
id, name, repo_url, branch, dockerfile_path, webhook_secret, id, name, repo_url, branch, dockerfile_path, webhook_secret,
ssh_private_key, ssh_public_key, image_id, status, ssh_private_key, ssh_public_key, image_id, status,
docker_network, ntfy_topic, slack_webhook, webhook_secret_hash, docker_network, ntfy_topic, slack_webhook, webhook_secret_hash,
previous_image_id previous_image_id, cpu_limit, memory_limit
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)` ) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`
_, err := a.db.Exec(ctx, query, _, err := a.db.Exec(ctx, query,
a.ID, a.Name, a.RepoURL, a.Branch, a.DockerfilePath, a.WebhookSecret, a.ID, a.Name, a.RepoURL, a.Branch, a.DockerfilePath, a.WebhookSecret,
a.SSHPrivateKey, a.SSHPublicKey, a.ImageID, a.Status, a.SSHPrivateKey, a.SSHPublicKey, a.ImageID, a.Status,
a.DockerNetwork, a.NtfyTopic, a.SlackWebhook, a.WebhookSecretHash, a.DockerNetwork, a.NtfyTopic, a.SlackWebhook, a.WebhookSecretHash,
a.PreviousImageID, a.PreviousImageID, a.CPULimit, a.MemoryLimit,
) )
if err != nil { if err != nil {
return err return err
@@ -165,6 +168,7 @@ func (a *App) update(ctx context.Context) error {
image_id = ?, status = ?, image_id = ?, status = ?,
docker_network = ?, ntfy_topic = ?, slack_webhook = ?, docker_network = ?, ntfy_topic = ?, slack_webhook = ?,
previous_image_id = ?, previous_image_id = ?,
cpu_limit = ?, memory_limit = ?,
updated_at = CURRENT_TIMESTAMP updated_at = CURRENT_TIMESTAMP
WHERE id = ?` WHERE id = ?`
@@ -173,6 +177,7 @@ func (a *App) update(ctx context.Context) error {
a.ImageID, a.Status, a.ImageID, a.Status,
a.DockerNetwork, a.NtfyTopic, a.SlackWebhook, a.DockerNetwork, a.NtfyTopic, a.SlackWebhook,
a.PreviousImageID, a.PreviousImageID,
a.CPULimit, a.MemoryLimit,
a.ID, a.ID,
) )
@@ -188,6 +193,7 @@ func (a *App) scan(row *sql.Row) error {
&a.DockerNetwork, &a.NtfyTopic, &a.SlackWebhook, &a.DockerNetwork, &a.NtfyTopic, &a.SlackWebhook,
&a.WebhookSecretHash, &a.WebhookSecretHash,
&a.PreviousImageID, &a.PreviousImageID,
&a.CPULimit, &a.MemoryLimit,
&a.CreatedAt, &a.UpdatedAt, &a.CreatedAt, &a.UpdatedAt,
) )
} }
@@ -206,6 +212,7 @@ func scanApps(appDB *database.Database, rows *sql.Rows) ([]*App, error) {
&app.DockerNetwork, &app.NtfyTopic, &app.SlackWebhook, &app.DockerNetwork, &app.NtfyTopic, &app.SlackWebhook,
&app.WebhookSecretHash, &app.WebhookSecretHash,
&app.PreviousImageID, &app.PreviousImageID,
&app.CPULimit, &app.MemoryLimit,
&app.CreatedAt, &app.UpdatedAt, &app.CreatedAt, &app.UpdatedAt,
) )
if scanErr != nil { if scanErr != nil {

View File

@@ -1,193 +0,0 @@
package models
import (
"context"
"database/sql"
"fmt"
"time"
"sneak.berlin/go/upaas/internal/database"
)
// AuditAction represents the type of audited user action.
type AuditAction string
// Audit action constants.
const (
AuditActionLogin AuditAction = "login"
AuditActionLogout AuditAction = "logout"
AuditActionAppCreate AuditAction = "app.create"
AuditActionAppUpdate AuditAction = "app.update"
AuditActionAppDelete AuditAction = "app.delete"
AuditActionAppDeploy AuditAction = "app.deploy"
AuditActionAppRollback AuditAction = "app.rollback"
AuditActionAppRestart AuditAction = "app.restart"
AuditActionAppStop AuditAction = "app.stop"
AuditActionAppStart AuditAction = "app.start"
AuditActionDeployCancel AuditAction = "deploy.cancel"
AuditActionEnvVarSave AuditAction = "env_var.save"
AuditActionLabelAdd AuditAction = "label.add"
AuditActionLabelEdit AuditAction = "label.edit"
AuditActionLabelDelete AuditAction = "label.delete"
AuditActionVolumeAdd AuditAction = "volume.add"
AuditActionVolumeEdit AuditAction = "volume.edit"
AuditActionVolumeDelete AuditAction = "volume.delete"
AuditActionPortAdd AuditAction = "port.add"
AuditActionPortDelete AuditAction = "port.delete"
AuditActionSetup AuditAction = "setup"
AuditActionWebhookReceive AuditAction = "webhook.receive"
)
// AuditResourceType represents the type of resource being acted on.
type AuditResourceType string
// Audit resource type constants.
const (
AuditResourceApp AuditResourceType = "app"
AuditResourceUser AuditResourceType = "user"
AuditResourceSession AuditResourceType = "session"
AuditResourceEnvVar AuditResourceType = "env_var"
AuditResourceLabel AuditResourceType = "label"
AuditResourceVolume AuditResourceType = "volume"
AuditResourcePort AuditResourceType = "port"
AuditResourceDeployment AuditResourceType = "deployment"
AuditResourceWebhook AuditResourceType = "webhook"
)
// AuditEntry represents a single audit log entry.
type AuditEntry struct {
db *database.Database
ID int64
UserID sql.NullInt64
Username string
Action AuditAction
ResourceType AuditResourceType
ResourceID sql.NullString
Detail sql.NullString
RemoteIP sql.NullString
CreatedAt time.Time
}
// NewAuditEntry creates a new AuditEntry with a database reference.
func NewAuditEntry(db *database.Database) *AuditEntry {
return &AuditEntry{db: db}
}
// Save inserts the audit entry into the database.
func (a *AuditEntry) Save(ctx context.Context) error {
query := `
INSERT INTO audit_log (
user_id, username, action, resource_type, resource_id,
detail, remote_ip
) VALUES (?, ?, ?, ?, ?, ?, ?)`
result, err := a.db.Exec(ctx, query,
a.UserID, a.Username, a.Action, a.ResourceType,
a.ResourceID, a.Detail, a.RemoteIP,
)
if err != nil {
return fmt.Errorf("inserting audit entry: %w", err)
}
id, err := result.LastInsertId()
if err != nil {
return fmt.Errorf("getting audit entry id: %w", err)
}
a.ID = id
return nil
}
// FindAuditEntries returns recent audit log entries, newest first.
func FindAuditEntries(
ctx context.Context,
db *database.Database,
limit int,
) ([]*AuditEntry, error) {
query := `
SELECT id, user_id, username, action, resource_type, resource_id,
detail, remote_ip, created_at
FROM audit_log
ORDER BY created_at DESC
LIMIT ?`
rows, err := db.Query(ctx, query, limit)
if err != nil {
return nil, fmt.Errorf("querying audit entries: %w", err)
}
defer func() { _ = rows.Close() }()
return scanAuditRows(rows)
}
// FindAuditEntriesByResource returns audit log entries for a specific resource.
func FindAuditEntriesByResource(
ctx context.Context,
db *database.Database,
resourceType AuditResourceType,
resourceID string,
limit int,
) ([]*AuditEntry, error) {
query := `
SELECT id, user_id, username, action, resource_type, resource_id,
detail, remote_ip, created_at
FROM audit_log
WHERE resource_type = ? AND resource_id = ?
ORDER BY created_at DESC
LIMIT ?`
rows, err := db.Query(ctx, query, resourceType, resourceID, limit)
if err != nil {
return nil, fmt.Errorf("querying audit entries by resource: %w", err)
}
defer func() { _ = rows.Close() }()
return scanAuditRows(rows)
}
// CountAuditEntries returns the total number of audit log entries.
func CountAuditEntries(
ctx context.Context,
db *database.Database,
) (int, error) {
var count int
row := db.QueryRow(ctx, "SELECT COUNT(*) FROM audit_log")
err := row.Scan(&count)
if err != nil {
return 0, fmt.Errorf("counting audit entries: %w", err)
}
return count, nil
}
func scanAuditRows(rows *sql.Rows) ([]*AuditEntry, error) {
var entries []*AuditEntry
for rows.Next() {
entry := &AuditEntry{}
scanErr := rows.Scan(
&entry.ID, &entry.UserID, &entry.Username, &entry.Action,
&entry.ResourceType, &entry.ResourceID, &entry.Detail,
&entry.RemoteIP, &entry.CreatedAt,
)
if scanErr != nil {
return nil, fmt.Errorf("scanning audit entry: %w", scanErr)
}
entries = append(entries, entry)
}
rowsErr := rows.Err()
if rowsErr != nil {
return nil, fmt.Errorf("iterating audit entries: %w", rowsErr)
}
return entries, nil
}

View File

@@ -23,7 +23,6 @@ const (
testBranch = "main" testBranch = "main"
testValue = "value" testValue = "value"
testEventType = "push" testEventType = "push"
testAdmin = "admin"
) )
func setupTestDB(t *testing.T) (*database.Database, func()) { func setupTestDB(t *testing.T) (*database.Database, func()) {
@@ -184,7 +183,7 @@ func TestUserExists(t *testing.T) {
defer cleanup() defer cleanup()
user := models.NewUser(testDB) user := models.NewUser(testDB)
user.Username = testAdmin user.Username = "admin"
user.PasswordHash = testHash user.PasswordHash = testHash
err := user.Save(context.Background()) err := user.Save(context.Background())
@@ -782,177 +781,94 @@ func TestCascadeDelete(t *testing.T) {
}) })
} }
// AuditEntry Tests. // Resource Limits Tests.
func TestAuditEntryCreateAndFind(t *testing.T) { func TestAppResourceLimits(t *testing.T) { //nolint:funlen // integration test with multiple subtests
t.Parallel() t.Parallel()
testDB, cleanup := setupTestDB(t) t.Run("saves and loads CPU limit", func(t *testing.T) {
defer cleanup() t.Parallel()
entry := models.NewAuditEntry(testDB) testDB, cleanup := setupTestDB(t)
entry.Username = testAdmin defer cleanup()
entry.Action = models.AuditActionLogin
entry.ResourceType = models.AuditResourceSession
err := entry.Save(context.Background()) app := createTestApp(t, testDB)
require.NoError(t, err)
assert.NotZero(t, entry.ID)
entries, err := models.FindAuditEntries(context.Background(), testDB, 10) app.CPULimit = sql.NullFloat64{Float64: 0.5, Valid: true}
require.NoError(t, err)
require.Len(t, entries, 1)
assert.Equal(t, testAdmin, entries[0].Username)
assert.Equal(t, models.AuditActionLogin, entries[0].Action)
assert.Equal(t, models.AuditResourceSession, entries[0].ResourceType)
}
func TestAuditEntryWithAllFields(t *testing.T) { err := app.Save(context.Background())
t.Parallel()
testDB, cleanup := setupTestDB(t)
defer cleanup()
entry := models.NewAuditEntry(testDB)
entry.UserID = sql.NullInt64{Int64: 1, Valid: true}
entry.Username = testAdmin
entry.Action = models.AuditActionAppCreate
entry.ResourceType = models.AuditResourceApp
entry.ResourceID = sql.NullString{String: "app-123", Valid: true}
entry.Detail = sql.NullString{String: "created new app", Valid: true}
entry.RemoteIP = sql.NullString{String: "192.168.1.1", Valid: true}
err := entry.Save(context.Background())
require.NoError(t, err)
entries, err := models.FindAuditEntries(context.Background(), testDB, 10)
require.NoError(t, err)
require.Len(t, entries, 1)
assert.Equal(t, int64(1), entries[0].UserID.Int64)
assert.Equal(t, "app-123", entries[0].ResourceID.String)
assert.Equal(t, "created new app", entries[0].Detail.String)
assert.Equal(t, "192.168.1.1", entries[0].RemoteIP.String)
}
func TestAuditEntryFindByResource(t *testing.T) {
t.Parallel()
testDB, cleanup := setupTestDB(t)
defer cleanup()
// Create entries for different resources.
for _, action := range []models.AuditAction{
models.AuditActionAppCreate,
models.AuditActionAppUpdate,
models.AuditActionAppDeploy,
} {
entry := models.NewAuditEntry(testDB)
entry.Username = testAdmin
entry.Action = action
entry.ResourceType = models.AuditResourceApp
entry.ResourceID = sql.NullString{String: "app-1", Valid: true}
err := entry.Save(context.Background())
require.NoError(t, err) require.NoError(t, err)
}
// Create entry for a different resource. found, err := models.FindApp(context.Background(), testDB, app.ID)
entry := models.NewAuditEntry(testDB)
entry.Username = testAdmin
entry.Action = models.AuditActionLogin
entry.ResourceType = models.AuditResourceSession
err := entry.Save(context.Background())
require.NoError(t, err)
// Find by resource.
appEntries, err := models.FindAuditEntriesByResource(
context.Background(), testDB,
models.AuditResourceApp, "app-1", 10,
)
require.NoError(t, err)
assert.Len(t, appEntries, 3)
// All entries.
allEntries, err := models.FindAuditEntries(context.Background(), testDB, 10)
require.NoError(t, err)
assert.Len(t, allEntries, 4)
}
func TestAuditEntryCount(t *testing.T) {
t.Parallel()
testDB, cleanup := setupTestDB(t)
defer cleanup()
count, err := models.CountAuditEntries(context.Background(), testDB)
require.NoError(t, err)
assert.Equal(t, 0, count)
entry := models.NewAuditEntry(testDB)
entry.Username = testAdmin
entry.Action = models.AuditActionLogin
entry.ResourceType = models.AuditResourceSession
err = entry.Save(context.Background())
require.NoError(t, err)
count, err = models.CountAuditEntries(context.Background(), testDB)
require.NoError(t, err)
assert.Equal(t, 1, count)
}
func TestAuditEntryFindLimit(t *testing.T) {
t.Parallel()
testDB, cleanup := setupTestDB(t)
defer cleanup()
for range 5 {
entry := models.NewAuditEntry(testDB)
entry.Username = testAdmin
entry.Action = models.AuditActionLogin
entry.ResourceType = models.AuditResourceSession
err := entry.Save(context.Background())
require.NoError(t, err) require.NoError(t, err)
} require.NotNil(t, found)
assert.True(t, found.CPULimit.Valid)
assert.InDelta(t, 0.5, found.CPULimit.Float64, 0.001)
})
entries, err := models.FindAuditEntries(context.Background(), testDB, 3) t.Run("saves and loads memory limit", func(t *testing.T) {
require.NoError(t, err) t.Parallel()
assert.Len(t, entries, 3)
}
func TestAuditEntryOrderByCreatedAtDesc(t *testing.T) { testDB, cleanup := setupTestDB(t)
t.Parallel() defer cleanup()
testDB, cleanup := setupTestDB(t) app := createTestApp(t, testDB)
defer cleanup()
actions := []models.AuditAction{ app.MemoryLimit = sql.NullInt64{Int64: 536870912, Valid: true} // 512m
models.AuditActionLogin,
models.AuditActionAppCreate,
models.AuditActionLogout,
}
for _, action := range actions { err := app.Save(context.Background())
entry := models.NewAuditEntry(testDB)
entry.Username = testAdmin
entry.Action = action
entry.ResourceType = models.AuditResourceSession
err := entry.Save(context.Background())
require.NoError(t, err) require.NoError(t, err)
}
entries, err := models.FindAuditEntries(context.Background(), testDB, 10) found, err := models.FindApp(context.Background(), testDB, app.ID)
require.NoError(t, err) require.NoError(t, err)
require.Len(t, entries, 3) require.NotNil(t, found)
assert.True(t, found.MemoryLimit.Valid)
assert.Equal(t, int64(536870912), found.MemoryLimit.Int64)
})
// Newest first (logout was last inserted). t.Run("null limits by default", func(t *testing.T) {
assert.Equal(t, models.AuditActionLogout, entries[0].Action) t.Parallel()
assert.Equal(t, models.AuditActionAppCreate, entries[1].Action)
assert.Equal(t, models.AuditActionLogin, entries[2].Action) testDB, cleanup := setupTestDB(t)
defer cleanup()
app := createTestApp(t, testDB)
found, err := models.FindApp(context.Background(), testDB, app.ID)
require.NoError(t, err)
require.NotNil(t, found)
assert.False(t, found.CPULimit.Valid)
assert.False(t, found.MemoryLimit.Valid)
})
t.Run("clears limits when set to null", func(t *testing.T) {
t.Parallel()
testDB, cleanup := setupTestDB(t)
defer cleanup()
app := createTestApp(t, testDB)
// Set limits
app.CPULimit = sql.NullFloat64{Float64: 1.0, Valid: true}
app.MemoryLimit = sql.NullInt64{Int64: 1073741824, Valid: true} // 1g
err := app.Save(context.Background())
require.NoError(t, err)
// Clear limits
app.CPULimit = sql.NullFloat64{}
app.MemoryLimit = sql.NullInt64{}
err = app.Save(context.Background())
require.NoError(t, err)
found, err := models.FindApp(context.Background(), testDB, app.ID)
require.NoError(t, err)
require.NotNil(t, found)
assert.False(t, found.CPULimit.Valid)
assert.False(t, found.MemoryLimit.Valid)
})
} }
// Helper function to create a test app. // Helper function to create a test app.

View File

@@ -115,13 +115,14 @@ func (s *Server) SetupRoutes() {
r.Get("/apps", s.handlers.HandleAPIListApps()) r.Get("/apps", s.handlers.HandleAPIListApps())
r.Get("/apps/{id}", s.handlers.HandleAPIGetApp()) r.Get("/apps/{id}", s.handlers.HandleAPIGetApp())
r.Get("/apps/{id}/deployments", s.handlers.HandleAPIListDeployments()) r.Get("/apps/{id}/deployments", s.handlers.HandleAPIListDeployments())
r.Get("/audit", s.handlers.HandleAPIAuditLog())
}) })
}) })
// Metrics endpoint (always available, optionally protected with basic auth) // Metrics endpoint (optional, with basic auth)
s.router.Group(func(r chi.Router) { if s.params.Config.MetricsUsername != "" {
r.Use(s.mw.MetricsAuth()) s.router.Group(func(r chi.Router) {
r.Get("/metrics", promhttp.Handler().ServeHTTP) r.Use(s.mw.MetricsAuth())
}) r.Get("/metrics", promhttp.Handler().ServeHTTP)
})
}
} }

View File

@@ -1,126 +0,0 @@
// Package audit provides audit logging for user actions.
package audit
import (
"context"
"database/sql"
"log/slog"
"net/http"
"go.uber.org/fx"
"sneak.berlin/go/upaas/internal/database"
"sneak.berlin/go/upaas/internal/logger"
"sneak.berlin/go/upaas/internal/metrics"
"sneak.berlin/go/upaas/internal/middleware"
"sneak.berlin/go/upaas/internal/models"
)
// ServiceParams contains dependencies for Service.
type ServiceParams struct {
fx.In
Logger *logger.Logger
Database *database.Database
Metrics *metrics.Metrics
}
// Service provides audit logging functionality.
type Service struct {
log *slog.Logger
db *database.Database
metrics *metrics.Metrics
}
// New creates a new audit Service.
func New(_ fx.Lifecycle, params ServiceParams) (*Service, error) {
return &Service{
log: params.Logger.Get(),
db: params.Database,
metrics: params.Metrics,
}, nil
}
// LogEntry records an audit event.
type LogEntry struct {
UserID int64
Username string
Action models.AuditAction
ResourceType models.AuditResourceType
ResourceID string
Detail string
RemoteIP string
}
// Log records an audit log entry and increments the audit metrics counter.
func (svc *Service) Log(ctx context.Context, entry LogEntry) {
auditEntry := models.NewAuditEntry(svc.db)
auditEntry.Username = entry.Username
auditEntry.Action = entry.Action
auditEntry.ResourceType = entry.ResourceType
if entry.UserID != 0 {
auditEntry.UserID = sql.NullInt64{Int64: entry.UserID, Valid: true}
}
if entry.ResourceID != "" {
auditEntry.ResourceID = sql.NullString{String: entry.ResourceID, Valid: true}
}
if entry.Detail != "" {
auditEntry.Detail = sql.NullString{String: entry.Detail, Valid: true}
}
if entry.RemoteIP != "" {
auditEntry.RemoteIP = sql.NullString{String: entry.RemoteIP, Valid: true}
}
err := auditEntry.Save(ctx)
if err != nil {
svc.log.Error("failed to save audit entry",
"error", err,
"action", entry.Action,
"username", entry.Username,
)
return
}
svc.metrics.AuditEventsTotal.WithLabelValues(string(entry.Action)).Inc()
svc.log.Info("audit",
"action", entry.Action,
"username", entry.Username,
"resource_type", entry.ResourceType,
"resource_id", entry.ResourceID,
)
}
// LogFromRequest records an audit log entry, extracting the remote IP from
// the HTTP request using the middleware's trusted-proxy-aware IP resolution.
func (svc *Service) LogFromRequest(
ctx context.Context,
request *http.Request,
entry LogEntry,
) {
entry.RemoteIP = middleware.RealIP(request)
svc.Log(ctx, entry)
}
// Recent returns the most recent audit log entries.
func (svc *Service) Recent(
ctx context.Context,
limit int,
) ([]*models.AuditEntry, error) {
return models.FindAuditEntries(ctx, svc.db, limit)
}
// ForResource returns audit log entries for a specific resource.
func (svc *Service) ForResource(
ctx context.Context,
resourceType models.AuditResourceType,
resourceID string,
limit int,
) ([]*models.AuditEntry, error) {
return models.FindAuditEntriesByResource(ctx, svc.db, resourceType, resourceID, limit)
}

View File

@@ -1,221 +0,0 @@
package audit_test
import (
"context"
"log/slog"
"net/http"
"net/http/httptest"
"os"
"testing"
"github.com/prometheus/client_golang/prometheus"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"go.uber.org/fx"
"sneak.berlin/go/upaas/internal/config"
"sneak.berlin/go/upaas/internal/database"
"sneak.berlin/go/upaas/internal/globals"
"sneak.berlin/go/upaas/internal/logger"
"sneak.berlin/go/upaas/internal/metrics"
"sneak.berlin/go/upaas/internal/models"
"sneak.berlin/go/upaas/internal/service/audit"
)
func setupTestAuditService(t *testing.T) (*audit.Service, *database.Database) {
t.Helper()
globals.SetAppname("upaas-test")
globals.SetVersion("test")
tmpDir := t.TempDir()
cfg := &config.Config{
DataDir: tmpDir,
}
log := slog.New(slog.NewTextHandler(os.Stderr, nil))
logWrapper := logger.NewForTest(log)
db, err := database.New(fx.Lifecycle(nil), database.Params{
Logger: logWrapper,
Config: cfg,
})
require.NoError(t, err)
reg := prometheus.NewRegistry()
metricsInstance := metrics.NewForTest(reg)
svc, err := audit.New(fx.Lifecycle(nil), audit.ServiceParams{
Logger: logWrapper,
Database: db,
Metrics: metricsInstance,
})
require.NoError(t, err)
return svc, db
}
func TestAuditServiceLog(t *testing.T) {
t.Parallel()
svc, db := setupTestAuditService(t)
ctx := context.Background()
svc.Log(ctx, audit.LogEntry{
UserID: 1,
Username: "admin",
Action: models.AuditActionLogin,
ResourceType: models.AuditResourceSession,
Detail: "user logged in",
RemoteIP: "127.0.0.1",
})
entries, err := models.FindAuditEntries(ctx, db, 10)
require.NoError(t, err)
require.Len(t, entries, 1)
assert.Equal(t, "admin", entries[0].Username)
assert.Equal(t, models.AuditActionLogin, entries[0].Action)
assert.Equal(t, "127.0.0.1", entries[0].RemoteIP.String)
}
func TestAuditServiceLogFromRequest(t *testing.T) {
t.Parallel()
svc, db := setupTestAuditService(t)
ctx := context.Background()
request := httptest.NewRequest(http.MethodPost, "/apps", nil)
request.RemoteAddr = "10.0.0.1:12345"
svc.LogFromRequest(ctx, request, audit.LogEntry{
Username: "admin",
Action: models.AuditActionAppCreate,
ResourceType: models.AuditResourceApp,
ResourceID: "app-1",
Detail: "created app",
})
entries, err := models.FindAuditEntries(ctx, db, 10)
require.NoError(t, err)
require.Len(t, entries, 1)
assert.Equal(t, "10.0.0.1", entries[0].RemoteIP.String)
assert.Equal(t, "app-1", entries[0].ResourceID.String)
}
func TestAuditServiceLogFromRequestWithXRealIPTrustedProxy(t *testing.T) {
t.Parallel()
svc, db := setupTestAuditService(t)
ctx := context.Background()
// When the request comes from a trusted proxy (RFC1918), X-Real-IP is honoured.
request := httptest.NewRequest(http.MethodPost, "/apps", nil)
request.RemoteAddr = "10.0.0.1:1234"
request.Header.Set("X-Real-IP", "203.0.113.50")
svc.LogFromRequest(ctx, request, audit.LogEntry{
Username: "admin",
Action: models.AuditActionAppCreate,
ResourceType: models.AuditResourceApp,
})
entries, err := models.FindAuditEntries(ctx, db, 10)
require.NoError(t, err)
require.Len(t, entries, 1)
assert.Equal(t, "203.0.113.50", entries[0].RemoteIP.String)
}
func TestAuditServiceLogFromRequestWithXRealIPUntrustedProxy(t *testing.T) {
t.Parallel()
svc, db := setupTestAuditService(t)
ctx := context.Background()
// When the request comes from a public IP, X-Real-IP is ignored (anti-spoof).
request := httptest.NewRequest(http.MethodPost, "/apps", nil)
request.RemoteAddr = "203.0.113.99:1234"
request.Header.Set("X-Real-IP", "10.0.0.1")
svc.LogFromRequest(ctx, request, audit.LogEntry{
Username: "admin",
Action: models.AuditActionAppCreate,
ResourceType: models.AuditResourceApp,
})
entries, err := models.FindAuditEntries(ctx, db, 10)
require.NoError(t, err)
require.Len(t, entries, 1)
assert.Equal(t, "203.0.113.99", entries[0].RemoteIP.String)
}
func TestAuditServiceRecent(t *testing.T) {
t.Parallel()
svc, _ := setupTestAuditService(t)
ctx := context.Background()
for range 5 {
svc.Log(ctx, audit.LogEntry{
Username: "admin",
Action: models.AuditActionLogin,
ResourceType: models.AuditResourceSession,
})
}
entries, err := svc.Recent(ctx, 3)
require.NoError(t, err)
assert.Len(t, entries, 3)
}
func TestAuditServiceForResource(t *testing.T) {
t.Parallel()
svc, _ := setupTestAuditService(t)
ctx := context.Background()
// Log entries for different resources.
svc.Log(ctx, audit.LogEntry{
Username: "admin",
Action: models.AuditActionAppCreate,
ResourceType: models.AuditResourceApp,
ResourceID: "app-1",
})
svc.Log(ctx, audit.LogEntry{
Username: "admin",
Action: models.AuditActionAppDeploy,
ResourceType: models.AuditResourceApp,
ResourceID: "app-1",
})
svc.Log(ctx, audit.LogEntry{
Username: "admin",
Action: models.AuditActionAppCreate,
ResourceType: models.AuditResourceApp,
ResourceID: "app-2",
})
entries, err := svc.ForResource(ctx, models.AuditResourceApp, "app-1", 10)
require.NoError(t, err)
assert.Len(t, entries, 2)
}
func TestAuditServiceLogWithNoOptionalFields(t *testing.T) {
t.Parallel()
svc, db := setupTestAuditService(t)
ctx := context.Background()
svc.Log(ctx, audit.LogEntry{
Username: "system",
Action: models.AuditActionWebhookReceive,
ResourceType: models.AuditResourceWebhook,
})
entries, err := models.FindAuditEntries(ctx, db, 10)
require.NoError(t, err)
require.Len(t, entries, 1)
assert.False(t, entries[0].UserID.Valid)
assert.False(t, entries[0].ResourceID.Valid)
assert.False(t, entries[0].Detail.Valid)
assert.False(t, entries[0].RemoteIP.Valid)
}

View File

@@ -21,7 +21,6 @@ import (
"sneak.berlin/go/upaas/internal/database" "sneak.berlin/go/upaas/internal/database"
"sneak.berlin/go/upaas/internal/docker" "sneak.berlin/go/upaas/internal/docker"
"sneak.berlin/go/upaas/internal/logger" "sneak.berlin/go/upaas/internal/logger"
"sneak.berlin/go/upaas/internal/metrics"
"sneak.berlin/go/upaas/internal/models" "sneak.berlin/go/upaas/internal/models"
"sneak.berlin/go/upaas/internal/service/notify" "sneak.berlin/go/upaas/internal/service/notify"
) )
@@ -209,7 +208,6 @@ type ServiceParams struct {
Database *database.Database Database *database.Database
Docker *docker.Client Docker *docker.Client
Notify *notify.Service Notify *notify.Service
Metrics *metrics.Metrics
} }
// activeDeploy tracks a running deployment so it can be cancelled. // activeDeploy tracks a running deployment so it can be cancelled.
@@ -224,7 +222,6 @@ type Service struct {
db *database.Database db *database.Database
docker *docker.Client docker *docker.Client
notify *notify.Service notify *notify.Service
metrics *metrics.Metrics
config *config.Config config *config.Config
params *ServiceParams params *ServiceParams
activeDeploys sync.Map // map[string]*activeDeploy - per-app active deployment tracking activeDeploys sync.Map // map[string]*activeDeploy - per-app active deployment tracking
@@ -234,13 +231,12 @@ type Service struct {
// New creates a new deploy Service. // New creates a new deploy Service.
func New(lc fx.Lifecycle, params ServiceParams) (*Service, error) { func New(lc fx.Lifecycle, params ServiceParams) (*Service, error) {
svc := &Service{ svc := &Service{
log: params.Logger.Get(), log: params.Logger.Get(),
db: params.Database, db: params.Database,
docker: params.Docker, docker: params.Docker,
notify: params.Notify, notify: params.Notify,
metrics: params.Metrics, config: params.Config,
config: params.Config, params: &params,
params: &params,
} }
if lc != nil { if lc != nil {
@@ -331,11 +327,6 @@ func (svc *Service) Deploy(
} }
defer svc.unlockApp(app.ID) defer svc.unlockApp(app.ID)
// Track in-flight deployments
svc.metrics.DeploymentsInFlight.WithLabelValues(app.Name).Inc()
deployStart := time.Now()
// Set up cancellable context and register as active deploy // Set up cancellable context and register as active deploy
deployCtx, cancel := context.WithCancel(ctx) deployCtx, cancel := context.WithCancel(ctx)
done := make(chan struct{}) done := make(chan struct{})
@@ -343,7 +334,6 @@ func (svc *Service) Deploy(
svc.activeDeploys.Store(app.ID, ad) svc.activeDeploys.Store(app.ID, ad)
defer func() { defer func() {
svc.metrics.DeploymentsInFlight.WithLabelValues(app.Name).Dec()
cancel() cancel()
close(done) close(done)
svc.activeDeploys.Delete(app.ID) svc.activeDeploys.Delete(app.ID)
@@ -369,7 +359,7 @@ func (svc *Service) Deploy(
svc.notify.NotifyBuildStart(bgCtx, app, deployment) svc.notify.NotifyBuildStart(bgCtx, app, deployment)
return svc.runBuildAndDeploy(deployCtx, bgCtx, app, deployment, deployStart) return svc.runBuildAndDeploy(deployCtx, bgCtx, app, deployment)
} }
// Rollback rolls back an app to its previous image. // Rollback rolls back an app to its previous image.
@@ -477,20 +467,15 @@ func (svc *Service) runBuildAndDeploy(
bgCtx context.Context, bgCtx context.Context,
app *models.App, app *models.App,
deployment *models.Deployment, deployment *models.Deployment,
deployStart time.Time,
) error { ) error {
// Build phase with timeout // Build phase with timeout
imageID, err := svc.buildImageWithTimeout(deployCtx, app, deployment) imageID, err := svc.buildImageWithTimeout(deployCtx, app, deployment)
if err != nil { if err != nil {
cancelErr := svc.checkCancelled(deployCtx, bgCtx, app, deployment, "") cancelErr := svc.checkCancelled(deployCtx, bgCtx, app, deployment, "")
if cancelErr != nil { if cancelErr != nil {
svc.recordDeployMetrics(app.Name, "cancelled", deployStart)
return cancelErr return cancelErr
} }
svc.recordDeployMetrics(app.Name, "failed", deployStart)
return err return err
} }
@@ -501,13 +486,9 @@ func (svc *Service) runBuildAndDeploy(
if err != nil { if err != nil {
cancelErr := svc.checkCancelled(deployCtx, bgCtx, app, deployment, imageID) cancelErr := svc.checkCancelled(deployCtx, bgCtx, app, deployment, imageID)
if cancelErr != nil { if cancelErr != nil {
svc.recordDeployMetrics(app.Name, "cancelled", deployStart)
return cancelErr return cancelErr
} }
svc.recordDeployMetrics(app.Name, "failed", deployStart)
return err return err
} }
@@ -523,19 +504,11 @@ func (svc *Service) runBuildAndDeploy(
// Use context.WithoutCancel to ensure health check completes even if // Use context.WithoutCancel to ensure health check completes even if
// the parent context is cancelled (e.g., HTTP request ends). // the parent context is cancelled (e.g., HTTP request ends).
go svc.checkHealthAfterDelay(bgCtx, app, deployment, deployStart) go svc.checkHealthAfterDelay(bgCtx, app, deployment)
return nil return nil
} }
// recordDeployMetrics records deployment completion metrics.
func (svc *Service) recordDeployMetrics(appName, status string, start time.Time) {
duration := time.Since(start).Seconds()
svc.metrics.DeploymentsTotal.WithLabelValues(appName, status).Inc()
svc.metrics.DeploymentDuration.WithLabelValues(appName, status).Observe(duration)
}
// buildImageWithTimeout runs the build phase with a timeout. // buildImageWithTimeout runs the build phase with a timeout.
func (svc *Service) buildImageWithTimeout( func (svc *Service) buildImageWithTimeout(
ctx context.Context, ctx context.Context,
@@ -1121,14 +1094,28 @@ func (svc *Service) buildContainerOptions(
network = app.DockerNetwork.String network = app.DockerNetwork.String
} }
var cpuLimit float64
if app.CPULimit.Valid {
cpuLimit = app.CPULimit.Float64
}
var memoryLimit int64
if app.MemoryLimit.Valid {
memoryLimit = app.MemoryLimit.Int64
}
return docker.CreateContainerOptions{ return docker.CreateContainerOptions{
Name: "upaas-" + app.Name, Name: "upaas-" + app.Name,
Image: imageID.String(), Image: imageID.String(),
Env: envMap, Env: envMap,
Labels: buildLabelMap(app, labels), Labels: buildLabelMap(app, labels),
Volumes: buildVolumeMounts(volumes), Volumes: buildVolumeMounts(volumes),
Ports: buildPortMappings(ports), Ports: buildPortMappings(ports),
Network: network, Network: network,
CPULimit: cpuLimit,
MemoryLimit: memoryLimit,
}, nil }, nil
} }
@@ -1190,7 +1177,6 @@ func (svc *Service) checkHealthAfterDelay(
ctx context.Context, ctx context.Context,
app *models.App, app *models.App,
deployment *models.Deployment, deployment *models.Deployment,
deployStart time.Time,
) { ) {
svc.log.Info( svc.log.Info(
"waiting 60 seconds to check container health", "waiting 60 seconds to check container health",
@@ -1217,8 +1203,6 @@ func (svc *Service) checkHealthAfterDelay(
svc.notify.NotifyDeployFailed(ctx, reloadedApp, deployment, err) svc.notify.NotifyDeployFailed(ctx, reloadedApp, deployment, err)
_ = deployment.MarkFinished(ctx, models.DeploymentStatusFailed) _ = deployment.MarkFinished(ctx, models.DeploymentStatusFailed)
svc.writeLogsToFile(reloadedApp, deployment) svc.writeLogsToFile(reloadedApp, deployment)
svc.recordDeployMetrics(reloadedApp.Name, "failed", deployStart)
svc.metrics.ContainerHealthy.WithLabelValues(reloadedApp.Name).Set(0)
reloadedApp.Status = models.AppStatusError reloadedApp.Status = models.AppStatusError
_ = reloadedApp.Save(ctx) _ = reloadedApp.Save(ctx)
@@ -1230,8 +1214,6 @@ func (svc *Service) checkHealthAfterDelay(
svc.notify.NotifyDeploySuccess(ctx, reloadedApp, deployment) svc.notify.NotifyDeploySuccess(ctx, reloadedApp, deployment)
_ = deployment.MarkFinished(ctx, models.DeploymentStatusSuccess) _ = deployment.MarkFinished(ctx, models.DeploymentStatusSuccess)
svc.writeLogsToFile(reloadedApp, deployment) svc.writeLogsToFile(reloadedApp, deployment)
svc.recordDeployMetrics(reloadedApp.Name, "success", deployStart)
svc.metrics.ContainerHealthy.WithLabelValues(reloadedApp.Name).Set(1)
} else { } else {
svc.log.Warn( svc.log.Warn(
"container unhealthy after 60 seconds", "container unhealthy after 60 seconds",
@@ -1240,8 +1222,6 @@ func (svc *Service) checkHealthAfterDelay(
svc.notify.NotifyDeployFailed(ctx, reloadedApp, deployment, ErrContainerUnhealthy) svc.notify.NotifyDeployFailed(ctx, reloadedApp, deployment, ErrContainerUnhealthy)
_ = deployment.MarkFinished(ctx, models.DeploymentStatusFailed) _ = deployment.MarkFinished(ctx, models.DeploymentStatusFailed)
svc.writeLogsToFile(reloadedApp, deployment) svc.writeLogsToFile(reloadedApp, deployment)
svc.recordDeployMetrics(reloadedApp.Name, "failed", deployStart)
svc.metrics.ContainerHealthy.WithLabelValues(reloadedApp.Name).Set(0)
reloadedApp.Status = models.AppStatusError reloadedApp.Status = models.AppStatusError
_ = reloadedApp.Save(ctx) _ = reloadedApp.Save(ctx)
} }

View File

@@ -2,6 +2,7 @@ package deploy_test
import ( import (
"context" "context"
"database/sql"
"log/slog" "log/slog"
"os" "os"
"testing" "testing"
@@ -43,3 +44,93 @@ func TestBuildContainerOptionsUsesImageID(t *testing.T) {
t.Errorf("expected Name=%q, got %q", "upaas-myapp", opts.Name) t.Errorf("expected Name=%q, got %q", "upaas-myapp", opts.Name)
} }
} }
func TestBuildContainerOptionsNoResourceLimits(t *testing.T) {
t.Parallel()
db := database.NewTestDatabase(t)
app := models.NewApp(db)
app.Name = "nolimits"
err := app.Save(context.Background())
if err != nil {
t.Fatalf("failed to save app: %v", err)
}
log := slog.New(slog.NewTextHandler(os.Stderr, nil))
svc := deploy.NewTestService(log)
opts, err := svc.BuildContainerOptionsExported(
context.Background(), app, docker.ImageID("test:latest"),
)
if err != nil {
t.Fatalf("buildContainerOptions returned error: %v", err)
}
if opts.CPULimit != 0 {
t.Errorf("expected CPULimit=0, got %v", opts.CPULimit)
}
if opts.MemoryLimit != 0 {
t.Errorf("expected MemoryLimit=0, got %v", opts.MemoryLimit)
}
}
func TestBuildContainerOptionsCPULimit(t *testing.T) {
t.Parallel()
db := database.NewTestDatabase(t)
app := models.NewApp(db)
app.Name = "cpulimit"
app.CPULimit = sql.NullFloat64{Float64: 0.5, Valid: true}
err := app.Save(context.Background())
if err != nil {
t.Fatalf("failed to save app: %v", err)
}
log := slog.New(slog.NewTextHandler(os.Stderr, nil))
svc := deploy.NewTestService(log)
opts, err := svc.BuildContainerOptionsExported(
context.Background(), app, docker.ImageID("test:latest"),
)
if err != nil {
t.Fatalf("buildContainerOptions returned error: %v", err)
}
if opts.CPULimit != 0.5 {
t.Errorf("expected CPULimit=0.5, got %v", opts.CPULimit)
}
}
func TestBuildContainerOptionsMemoryLimit(t *testing.T) {
t.Parallel()
db := database.NewTestDatabase(t)
app := models.NewApp(db)
app.Name = "memlimit"
app.MemoryLimit = sql.NullInt64{Int64: 536870912, Valid: true} // 512m
err := app.Save(context.Background())
if err != nil {
t.Fatalf("failed to save app: %v", err)
}
log := slog.New(slog.NewTextHandler(os.Stderr, nil))
svc := deploy.NewTestService(log)
opts, err := svc.BuildContainerOptionsExported(
context.Background(), app, docker.ImageID("test:latest"),
)
if err != nil {
t.Fatalf("buildContainerOptions returned error: %v", err)
}
if opts.MemoryLimit != 536870912 {
t.Errorf("expected MemoryLimit=536870912, got %v", opts.MemoryLimit)
}
}

View File

@@ -7,14 +7,12 @@ import (
"encoding/json" "encoding/json"
"fmt" "fmt"
"log/slog" "log/slog"
"strconv"
"go.uber.org/fx" "go.uber.org/fx"
"sneak.berlin/go/upaas/internal/database" "sneak.berlin/go/upaas/internal/database"
"sneak.berlin/go/upaas/internal/logger" "sneak.berlin/go/upaas/internal/logger"
"sneak.berlin/go/upaas/internal/metrics"
"sneak.berlin/go/upaas/internal/models" "sneak.berlin/go/upaas/internal/models"
"sneak.berlin/go/upaas/internal/service/deploy" "sneak.berlin/go/upaas/internal/service/deploy"
) )
@@ -26,26 +24,23 @@ type ServiceParams struct {
Logger *logger.Logger Logger *logger.Logger
Database *database.Database Database *database.Database
Deploy *deploy.Service Deploy *deploy.Service
Metrics *metrics.Metrics
} }
// Service provides webhook handling functionality. // Service provides webhook handling functionality.
type Service struct { type Service struct {
log *slog.Logger log *slog.Logger
db *database.Database db *database.Database
deploy *deploy.Service deploy *deploy.Service
metrics *metrics.Metrics params *ServiceParams
params *ServiceParams
} }
// New creates a new webhook Service. // New creates a new webhook Service.
func New(_ fx.Lifecycle, params ServiceParams) (*Service, error) { func New(_ fx.Lifecycle, params ServiceParams) (*Service, error) {
return &Service{ return &Service{
log: params.Logger.Get(), log: params.Logger.Get(),
db: params.Database, db: params.Database,
deploy: params.Deploy, deploy: params.Deploy,
metrics: params.Metrics, params: &params,
params: &params,
}, nil }, nil
} }
@@ -127,10 +122,6 @@ func (svc *Service) HandleWebhook(
"commit", commitSHA, "commit", commitSHA,
) )
svc.metrics.WebhookEventsTotal.WithLabelValues(
app.Name, eventType, strconv.FormatBool(matched),
).Inc()
// If branch matches, trigger deployment // If branch matches, trigger deployment
if matched { if matched {
svc.triggerDeployment(ctx, app, event) svc.triggerDeployment(ctx, app, event)

View File

@@ -8,7 +8,6 @@ import (
"testing" "testing"
"time" "time"
"github.com/prometheus/client_golang/prometheus"
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
"go.uber.org/fx" "go.uber.org/fx"
@@ -18,7 +17,6 @@ import (
"sneak.berlin/go/upaas/internal/docker" "sneak.berlin/go/upaas/internal/docker"
"sneak.berlin/go/upaas/internal/globals" "sneak.berlin/go/upaas/internal/globals"
"sneak.berlin/go/upaas/internal/logger" "sneak.berlin/go/upaas/internal/logger"
"sneak.berlin/go/upaas/internal/metrics"
"sneak.berlin/go/upaas/internal/models" "sneak.berlin/go/upaas/internal/models"
"sneak.berlin/go/upaas/internal/service/deploy" "sneak.berlin/go/upaas/internal/service/deploy"
"sneak.berlin/go/upaas/internal/service/notify" "sneak.berlin/go/upaas/internal/service/notify"
@@ -65,17 +63,13 @@ func setupTestService(t *testing.T) (*webhook.Service, *database.Database, func(
notifySvc, err := notify.New(fx.Lifecycle(nil), notify.ServiceParams{Logger: deps.logger}) notifySvc, err := notify.New(fx.Lifecycle(nil), notify.ServiceParams{Logger: deps.logger})
require.NoError(t, err) require.NoError(t, err)
metricsInstance := metrics.NewForTest(prometheus.NewRegistry())
deploySvc, err := deploy.New(fx.Lifecycle(nil), deploy.ServiceParams{ deploySvc, err := deploy.New(fx.Lifecycle(nil), deploy.ServiceParams{
Logger: deps.logger, Config: deps.config, Database: deps.db, Docker: dockerClient, Notify: notifySvc, Logger: deps.logger, Config: deps.config, Database: deps.db, Docker: dockerClient, Notify: notifySvc,
Metrics: metricsInstance,
}) })
require.NoError(t, err) require.NoError(t, err)
svc, err := webhook.New(fx.Lifecycle(nil), webhook.ServiceParams{ svc, err := webhook.New(fx.Lifecycle(nil), webhook.ServiceParams{
Logger: deps.logger, Database: deps.db, Deploy: deploySvc, Logger: deps.logger, Database: deps.db, Deploy: deploySvc,
Metrics: metricsInstance,
}) })
require.NoError(t, err) require.NoError(t, err)

View File

@@ -114,6 +114,38 @@
> >
</div> </div>
<hr class="border-gray-200">
<h3 class="text-lg font-medium text-gray-900">Resource Limits</h3>
<div class="grid grid-cols-2 gap-4">
<div class="form-group">
<label for="cpu_limit" class="label">CPU Limit (cores)</label>
<input
type="text"
id="cpu_limit"
name="cpu_limit"
value="{{if .App.CPULimit.Valid}}{{.App.CPULimit.Float64}}{{end}}"
class="input"
placeholder="e.g. 0.5, 1, 2"
>
<p class="text-sm text-gray-500 mt-1">Number of CPU cores (e.g. 0.5 = half a core)</p>
</div>
<div class="form-group">
<label for="memory_limit" class="label">Memory Limit</label>
<input
type="text"
id="memory_limit"
name="memory_limit"
value="{{if .App.MemoryLimit.Valid}}{{formatMemoryBytes .App.MemoryLimit.Int64}}{{end}}"
class="input"
placeholder="e.g. 256m, 1g"
>
<p class="text-sm text-gray-500 mt-1">Memory with unit suffix (k, m, g) or plain bytes</p>
</div>
</div>
<div class="flex justify-end gap-3 pt-4"> <div class="flex justify-end gap-3 pt-4">
<a href="/apps/{{.App.ID}}" class="btn-secondary">Cancel</a> <a href="/apps/{{.App.ID}}" class="btn-secondary">Cancel</a>
<button type="submit" class="btn-primary">Save Changes</button> <button type="submit" class="btn-primary">Save Changes</button>

View File

@@ -6,6 +6,7 @@ import (
"fmt" "fmt"
"html/template" "html/template"
"io" "io"
"strconv"
"sync" "sync"
) )
@@ -23,6 +24,34 @@ var (
templatesMutex sync.RWMutex templatesMutex sync.RWMutex
) )
// templateFuncs returns the custom template function map.
func templateFuncs() template.FuncMap {
return template.FuncMap{
"formatMemoryBytes": formatMemoryBytes,
}
}
// Memory unit constants.
const (
memGigabyte = 1024 * 1024 * 1024
memMegabyte = 1024 * 1024
memKilobyte = 1024
)
// formatMemoryBytes formats bytes into a human-readable string with unit suffix.
func formatMemoryBytes(bytes int64) string {
switch {
case bytes >= memGigabyte && bytes%memGigabyte == 0:
return strconv.FormatInt(bytes/memGigabyte, 10) + "g"
case bytes >= memMegabyte && bytes%memMegabyte == 0:
return strconv.FormatInt(bytes/memMegabyte, 10) + "m"
case bytes >= memKilobyte && bytes%memKilobyte == 0:
return strconv.FormatInt(bytes/memKilobyte, 10) + "k"
default:
return strconv.FormatInt(bytes, 10)
}
}
// initTemplates parses base template and creates cloned templates for each page. // initTemplates parses base template and creates cloned templates for each page.
func initTemplates() { func initTemplates() {
templatesMutex.Lock() templatesMutex.Lock()
@@ -32,8 +61,10 @@ func initTemplates() {
return return
} }
// Parse base template with shared components // Parse base template with shared components and custom functions
baseTemplate = template.Must(template.ParseFS(templatesRaw, "base.html")) baseTemplate = template.Must(
template.New("base.html").Funcs(templateFuncs()).ParseFS(templatesRaw, "base.html"),
)
// Pages that extend base // Pages that extend base
pages := []string{ pages := []string{

View File

@@ -0,0 +1,34 @@
package templates //nolint:testpackage // tests unexported formatMemoryBytes
import (
"testing"
)
func TestFormatMemoryBytes(t *testing.T) {
t.Parallel()
tests := []struct {
name string
bytes int64
expected string
}{
{"gigabytes", 1024 * 1024 * 1024, "1g"},
{"two gigabytes", 2 * 1024 * 1024 * 1024, "2g"},
{"megabytes", 256 * 1024 * 1024, "256m"},
{"kilobytes", 512 * 1024, "512k"},
{"plain bytes", 12345, "12345"},
{"non-even megabytes", 256*1024*1024 + 1, "268435457"},
{"zero", 0, "0"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
got := formatMemoryBytes(tt.bytes)
if got != tt.expected {
t.Errorf("formatMemoryBytes(%d) = %q, want %q", tt.bytes, got, tt.expected)
}
})
}
}