Files
upaas/README.md
clawbot f558e2cdd8
All checks were successful
Check / check (pull_request) Successful in 1m45s
feat: add observability improvements (metrics, audit log, structured logging)
- Add Prometheus metrics package (internal/metrics) with deployment,
  container health, webhook, HTTP request, and audit counters/histograms
- Add audit_log SQLite table via migration 007
- Add AuditEntry model with CRUD operations and query methods
- Add audit service (internal/service/audit) for recording user actions
- Instrument deploy service with deployment duration, count, and
  in-flight metrics; container health gauge updates on deploy completion
- Instrument webhook service with event counters by app/type/matched
- Instrument HTTP middleware with request count, duration, and response
  size metrics; also log response bytes in structured request logs
- Add audit logging to all key handler operations: login/logout, app
  CRUD, deploy, cancel, rollback, restart/stop/start, webhook receipt,
  and initial setup
- Add GET /api/audit endpoint for querying recent audit entries
- Make /metrics endpoint always available (optionally auth-protected)
- Add comprehensive tests for metrics, audit model, and audit service
- Update existing test infrastructure with metrics and audit dependencies
- Update README with Observability section documenting all metrics,
  audit log, and structured logging
2026-03-17 02:23:44 -07:00

263 lines
9.0 KiB
Markdown

# µPaaS by [@sneak](https://sneak.berlin)
A simple self-hosted PaaS that auto-deploys Docker containers from Git repositories via Gitea webhooks.
## Features
- Single admin user with argon2id password hashing
- Per-app SSH keypairs for read-only deploy keys
- Per-app UUID-based webhook URLs for Gitea integration
- Branch filtering - only deploy on configured branch changes
- Environment variables, labels, and volume mounts per app
- Docker builds via socket access
- Notifications via ntfy and Slack-compatible webhooks
- Simple server-rendered UI with Tailwind CSS
## Non-Goals
- Multi-user support
- Complex CI pipelines
- Multiple container orchestration
- SPA/API-first design
- Support for non-Gitea webhooks
## Architecture
### Project Structure
```
upaas/
├── cmd/upaasd/ # Application entry point
├── internal/
│ ├── config/ # Configuration via Viper
│ ├── database/ # SQLite database with migrations
│ ├── docker/ # Docker client for builds/deploys
│ ├── globals/ # Build-time variables (version, etc.)
│ ├── handlers/ # HTTP request handlers
│ ├── healthcheck/ # Health status service
│ ├── logger/ # Structured logging (slog)
│ ├── metrics/ # Prometheus metrics registration
│ ├── middleware/ # HTTP middleware (auth, logging, CORS, metrics)
│ ├── models/ # Active Record style database models
│ ├── server/ # HTTP server and routes
│ ├── service/
│ │ ├── app/ # App management service
│ │ ├── audit/ # Audit logging service
│ │ ├── auth/ # Authentication service
│ │ ├── deploy/ # Deployment orchestration
│ │ ├── notify/ # Notifications (ntfy, Slack)
│ │ └── webhook/ # Gitea webhook processing
│ └── ssh/ # SSH key generation
├── static/ # Embedded CSS/JS assets
└── templates/ # Embedded HTML templates
```
### Dependency Injection
Uses Uber fx for dependency injection. Components are wired in this order:
1. `globals` - Build-time variables
2. `logger` - Structured logging
3. `config` - Configuration loading
4. `database` - SQLite connection + migrations
5. `metrics` - Prometheus metrics registration
6. `healthcheck` - Health status
7. `auth` - Authentication service
8. `app` - App management
9. `docker` - Docker client
10. `notify` - Notification service
11. `audit` - Audit logging service
10. `deploy` - Deployment service
11. `webhook` - Webhook processing
12. `middleware` - HTTP middleware
13. `handlers` - HTTP handlers
14. `server` - HTTP server
### Request Flow
```
HTTP Request
chi Router ──► Middleware Stack ──► Handler
(Logging, Auth, CORS, etc.)
Handler Function
Service Layer (app, auth, deploy, etc.)
Models (Active Record)
Database
```
### Key Patterns
- **Closure-based handlers**: Handlers return `http.HandlerFunc` allowing one-time initialization
- **Active Record models**: Models encapsulate database operations (`Save()`, `Delete()`, `Reload()`)
- **Async deployments**: Webhook triggers deploy via goroutine with `context.WithoutCancel()`
- **Embedded assets**: Templates and static files embedded via `//go:embed`
## Development
### Prerequisites
- Go 1.25+
- golangci-lint
- Docker (for running)
### Commands
```bash
make fmt # Format code
make fmt-check # Check formatting (read-only, fails if unformatted)
make lint # Run comprehensive linting
make test # Run tests with race detection (30s timeout)
make check # Verify everything passes (fmt-check, lint, test)
make build # Build binary
make docker # Build Docker image
make hooks # Install pre-commit hook (runs make check)
```
### Commit Requirements
**All commits must pass `make check` before being committed.**
Before every commit:
1. **Format**: Run `make fmt` to format all code
2. **Lint**: Run `make lint` and fix all errors/warnings
- Do not disable linters or add nolint comments without good reason
- Fix the code, don't hide the problem
3. **Test**: Run `make test` and ensure all tests pass
- Fix failing tests by fixing the code, not by modifying tests to pass
- Add tests for new functionality
4. **Verify**: Run `make check` to confirm everything passes
```bash
# Standard workflow before commit:
make fmt
make lint # Fix any issues
make test # Fix any failures
make check # Final verification
git add .
git commit -m "Your message"
```
The Docker build runs `make check` and will fail if:
- Code is not formatted
- Linting errors exist
- Tests fail
- Code doesn't compile
This ensures the main branch always contains clean, tested, working code.
## Configuration
Environment variables:
| Variable | Description | Default |
|----------|-------------|---------|
| `PORT` | HTTP listen port | 8080 |
| `UPAAS_DATA_DIR` | Data directory for SQLite and keys | `./data` (local dev only — use absolute path for Docker) |
| `UPAAS_HOST_DATA_DIR` | Host path for DATA_DIR (when running in container) | *(none — must be set to an absolute path)* |
| `UPAAS_DOCKER_HOST` | Docker socket path | unix:///var/run/docker.sock |
| `DEBUG` | Enable debug logging | false |
| `SENTRY_DSN` | Sentry error reporting DSN | "" |
| `METRICS_USERNAME` | Basic auth for /metrics | "" |
| `METRICS_PASSWORD` | Basic auth for /metrics | "" |
## Running with Docker
```bash
docker run -d \
-p 8080:8080 \
-v /var/run/docker.sock:/var/run/docker.sock \
-v /path/on/host/upaas-data:/var/lib/upaas \
-e UPAAS_HOST_DATA_DIR=/path/on/host/upaas-data \
upaas
```
### Docker Compose
```yaml
services:
upaas:
build: .
restart: unless-stopped
ports:
- "8080:8080"
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- ${HOST_DATA_DIR}:/var/lib/upaas
environment:
- UPAAS_HOST_DATA_DIR=${HOST_DATA_DIR}
# Optional: uncomment to enable debug logging
# - DEBUG=true
# Optional: Sentry error reporting
# - SENTRY_DSN=https://...
# Optional: Prometheus metrics auth
# - METRICS_USERNAME=prometheus
# - METRICS_PASSWORD=secret
```
**Important**: You **must** set `HOST_DATA_DIR` to an **absolute path** on the host before running
`docker compose up`. This value is bind-mounted into the container and passed as `UPAAS_HOST_DATA_DIR`
so that Docker bind mounts during builds resolve correctly. Relative paths (e.g. `./data`) will break
container builds because the Docker daemon resolves paths relative to the host, not the container.
Example: `HOST_DATA_DIR=/srv/upaas/data docker compose up -d`
Session secrets are automatically generated on first startup and persisted to `$UPAAS_DATA_DIR/session.key`.
## Observability
### Prometheus Metrics
All custom metrics are exposed under the `upaas_` namespace at `/metrics`. The
endpoint is always available and can be optionally protected with basic auth via
`METRICS_USERNAME` and `METRICS_PASSWORD`.
| Metric | Type | Labels | Description |
|--------|------|--------|-------------|
| `upaas_deployments_total` | Counter | `app`, `status` | Total deployments (success/failed/cancelled) |
| `upaas_deployments_duration_seconds` | Histogram | `app`, `status` | Deployment duration |
| `upaas_deployments_in_flight` | Gauge | `app` | Currently running deployments |
| `upaas_container_healthy` | Gauge | `app` | Container health (1=healthy, 0=unhealthy) |
| `upaas_webhook_events_total` | Counter | `app`, `event_type`, `matched` | Webhook events received |
| `upaas_http_requests_total` | Counter | `method`, `status_code` | HTTP requests |
| `upaas_http_request_duration_seconds` | Histogram | `method` | HTTP request latency |
| `upaas_http_response_size_bytes` | Histogram | `method` | HTTP response sizes |
| `upaas_audit_events_total` | Counter | `action` | Audit log events |
### Audit Log
All user-facing actions are recorded in an `audit_log` SQLite table with:
- **Who**: user ID and username
- **What**: action type and affected resource (app, deployment, session, etc.)
- **Where**: client IP (via X-Real-IP/X-Forwarded-For/RemoteAddr)
- **When**: timestamp
Audited actions include login/logout, app CRUD, deployments, container
start/stop/restart, rollbacks, deployment cancellation, and webhook receipt.
The audit log is available via the API at `GET /api/audit?limit=N` (max 500,
default 50).
### Structured Logging
All operations use Go's `slog` structured logger. HTTP requests are logged with
method, URL, status code, response size, latency, user agent, and client IP.
Deployment events are logged with app name, status, and duration. Audit events
are also logged to stdout for correlation with external log aggregators.
## License
WTFPL