diff --git a/.dockerignore b/.dockerignore new file mode 100644 index 0000000..4878078 --- /dev/null +++ b/.dockerignore @@ -0,0 +1,6 @@ +.git +bin +data +.env +.DS_Store +*.exe diff --git a/.editorconfig b/.editorconfig new file mode 100644 index 0000000..f52b09b --- /dev/null +++ b/.editorconfig @@ -0,0 +1,15 @@ +root = true + +[*] +indent_style = space +indent_size = 4 +end_of_line = lf +charset = utf-8 +trim_trailing_whitespace = true +insert_final_newline = true + +[*.go] +indent_style = tab + +[Makefile] +indent_style = tab diff --git a/CLAUDE.md b/CLAUDE.md deleted file mode 100644 index 9c9ebe5..0000000 --- a/CLAUDE.md +++ /dev/null @@ -1,68 +0,0 @@ -# Repository Rules - -Last Updated 2026-01-08 - -These rules MUST be followed at all times, it is very important. - -* Never use `git add -A` - add specific changes to a deliberate commit. A - commit should contain one change. After each change, make a commit with a - good one-line summary. - -* NEVER modify the linter config without asking first. - -* NEVER modify tests to exclude special cases or otherwise get them to pass - without asking first. In almost all cases, the code should be changed, - NOT the tests. If you think the test needs to be changed, make your case - for that and ask for permission to proceed, then stop. You need explicit - user approval to modify existing tests. (You do not need user approval - for writing NEW tests.) - -* When linting, assume the linter config is CORRECT, and that each item - output by the linter is something that legitimately needs fixing in the - code. - -* When running tests, use `make test`. - -* Before commits, run `make check`. This runs `make lint` and `make test` - and `make check-fmt`. Any issues discovered MUST be resolved before - committing unless explicitly told otherwise. - -* When fixing a bug, write a failing test for the bug FIRST. Add - appropriate logging to the test to ensure it is written correctly. Commit - that. Then go about fixing the bug until the test passes (without - modifying the test further). Then commit that. - -* When adding a new feature, do the same - implement a test first (TDD). It - doesn't have to be super complex. Commit the test, then commit the - feature. - -* When adding a new feature, use a feature branch. When the feature is - completely finished and the code is up to standards (passes `make check`) - then and only then can the feature branch be merged into `main` and the - branch deleted. - -* Write godoc documentation comments for all exported types and functions as - you go along. - -* ALWAYS be consistent in naming. If you name something one thing in one - place, name it the EXACT SAME THING in another place. - -* Be descriptive and specific in naming. `wl` is bad; - `SourceHostWhitelist` is good. `ConnsPerHost` is bad; - `MaxConnectionsPerHost` is good. - -* This is not prototype or teaching code - this is designed for production. - Any security issues (such as denial of service) or other web - vulnerabilities are P1 bugs and must be added to TODO.md at the top. - -* As this is production code, no stubbing of implementations unless - specifically instructed. We need working implementations. - -* Avoid vendoring deps unless specifically instructed to. NEVER commit - the vendor directory, NEVER commit compiled binaries. If these - directories or files exist, add them to .gitignore (and commit the - .gitignore) if they are not already in there. Keep the entire git - repository (with history) small - under 20MiB, unless you specifically - must commit larger files (e.g. test fixture example media files). Only - OUR source code and immediately supporting files (such as test examples) - goes into the repo/history. diff --git a/CONVENTIONS.md b/CONVENTIONS.md deleted file mode 100644 index 6f7c217..0000000 --- a/CONVENTIONS.md +++ /dev/null @@ -1,1225 +0,0 @@ -# Go HTTP Server Conventions - -This document defines the architectural patterns, design decisions, and conventions for building Go HTTP servers. All new projects must follow these standards. - -## Table of Contents - -1. [Required Libraries](#1-required-libraries) -2. [Project Structure](#2-project-structure) -3. [Dependency Injection (Uber fx)](#3-dependency-injection-uber-fx) -4. [Server Architecture](#4-server-architecture) -5. [Routing (go-chi)](#5-routing-go-chi) -6. [Handler Conventions](#6-handler-conventions) -7. [Middleware Conventions](#7-middleware-conventions) -8. [Configuration (Viper)](#8-configuration-viper) -9. [Logging (slog)](#9-logging-slog) -10. [Database Wrapper](#10-database-wrapper) -11. [Globals Package](#11-globals-package) -12. [Static Assets & Templates](#12-static-assets--templates) -13. [Health Check](#13-health-check) -14. [External Integrations](#14-external-integrations) - ---- - -## 1. Required Libraries - -These libraries are **mandatory** for all new projects: - -| Purpose | Library | Import Path | -|---------|---------|-------------| -| Dependency Injection | Uber fx | `go.uber.org/fx` | -| HTTP Router | go-chi | `github.com/go-chi/chi` | -| Logging | slog (stdlib) | `log/slog` | -| Configuration | Viper | `github.com/spf13/viper` | -| Environment Loading | godotenv | `github.com/joho/godotenv/autoload` | -| CORS | go-chi/cors | `github.com/go-chi/cors` | -| Error Reporting | Sentry | `github.com/getsentry/sentry-go` | -| Metrics | Prometheus | `github.com/prometheus/client_golang` | -| Metrics Middleware | go-http-metrics | `github.com/slok/go-http-metrics` | -| Basic Auth | basicauth-go | `github.com/99designs/basicauth-go` | - ---- - -## 2. Project Structure - -``` -project-root/ -├── cmd/ -│ └── {appname}/ -│ └── main.go # Entry point -├── internal/ -│ ├── config/ -│ │ └── config.go # Configuration loading -│ ├── database/ -│ │ └── database.go # Database wrapper -│ ├── globals/ -│ │ └── globals.go # Build-time variables -│ ├── handlers/ -│ │ ├── handlers.go # Base handler struct and helpers -│ │ ├── index.go # Individual handlers... -│ │ ├── healthcheck.go -│ │ └── {feature}.go -│ ├── healthcheck/ -│ │ └── healthcheck.go # Health check service -│ ├── logger/ -│ │ └── logger.go # Logger setup -│ ├── middleware/ -│ │ └── middleware.go # All middleware definitions -│ └── server/ -│ ├── server.go # Server struct and lifecycle -│ ├── http.go # HTTP server setup -│ └── routes.go # Route definitions -├── static/ -│ ├── static.go # Embed directive -│ ├── css/ -│ └── js/ -├── templates/ -│ ├── templates.go # Embed and parse -│ └── *.html -├── go.mod -├── go.sum -├── Makefile -└── Dockerfile -``` - -### Key Principles - -- **`cmd/{appname}/`**: Only the entry point. Minimal logic, just bootstrapping. -- **`internal/`**: All application packages. Not importable by external projects. -- **One package per concern**: config, database, handlers, middleware, etc. -- **Flat handler files**: One file per handler or logical group of handlers. - ---- - -## 3. Dependency Injection (Uber fx) - -### Entry Point Pattern - -```go -// cmd/httpd/main.go -package main - -import ( - "yourproject/internal/config" - "yourproject/internal/database" - "yourproject/internal/globals" - "yourproject/internal/handlers" - "yourproject/internal/healthcheck" - "yourproject/internal/logger" - "yourproject/internal/middleware" - "yourproject/internal/server" - "go.uber.org/fx" -) - -var ( - Appname string = "CHANGEME" - Version string - Buildarch string -) - -func main() { - globals.Appname = Appname - globals.Version = Version - globals.Buildarch = Buildarch - - fx.New( - fx.Provide( - config.New, - database.New, - globals.New, - handlers.New, - logger.New, - server.New, - middleware.New, - healthcheck.New, - ), - fx.Invoke(func(*server.Server) {}), - ).Run() -} -``` - -### Params Struct Pattern - -Every component that receives dependencies uses a params struct with `fx.In`: - -```go -type HandlersParams struct { - fx.In - Logger *logger.Logger - Globals *globals.Globals - Database *database.Database - Healthcheck *healthcheck.Healthcheck -} - -type Handlers struct { - params *HandlersParams - log *slog.Logger - hc *healthcheck.Healthcheck -} -``` - -### Factory Function Pattern - -All components expose a `New` function with this signature: - -```go -func New(lc fx.Lifecycle, params SomeParams) (*Something, error) { - s := new(Something) - s.params = ¶ms - s.log = params.Logger.Get() - - lc.Append(fx.Hook{ - OnStart: func(ctx context.Context) error { - // Initialize resources - return nil - }, - OnStop: func(ctx context.Context) error { - // Cleanup resources - return nil - }, - }) - return s, nil -} -``` - -### Dependency Order - -Providers are resolved automatically by fx, but conceptually follow this order: - -1. `globals.New` - Build-time variables (no dependencies) -2. `logger.New` - Logger (depends on Globals) -3. `config.New` - Configuration (depends on Globals, Logger) -4. `database.New` - Database (depends on Logger, Config) -5. `healthcheck.New` - Health check (depends on Globals, Config, Logger, Database) -6. `middleware.New` - Middleware (depends on Logger, Globals, Config) -7. `handlers.New` - Handlers (depends on Logger, Globals, Database, Healthcheck) -8. `server.New` - Server (depends on all above) - ---- - -## 4. Server Architecture - -### Server Struct - -The Server struct is the central orchestrator: - -```go -// internal/server/server.go -type ServerParams struct { - fx.In - Logger *logger.Logger - Globals *globals.Globals - Config *config.Config - Middleware *middleware.Middleware - Handlers *handlers.Handlers -} - -type Server struct { - startupTime time.Time - port int - exitCode int - sentryEnabled bool - log *slog.Logger - ctx context.Context - cancelFunc context.CancelFunc - httpServer *http.Server - router *chi.Mux - params ServerParams - mw *middleware.Middleware - h *handlers.Handlers -} -``` - -### Server Factory - -```go -func New(lc fx.Lifecycle, params ServerParams) (*Server, error) { - s := new(Server) - s.params = params - s.mw = params.Middleware - s.h = params.Handlers - s.log = params.Logger.Get() - - lc.Append(fx.Hook{ - OnStart: func(ctx context.Context) error { - s.startupTime = time.Now() - go s.Run() - return nil - }, - OnStop: func(ctx context.Context) error { - // Server shutdown logic - return nil - }, - }) - return s, nil -} -``` - -### HTTP Server Setup - -```go -// internal/server/http.go -func (s *Server) serveUntilShutdown() { - listenAddr := fmt.Sprintf(":%d", s.params.Config.Port) - s.httpServer = &http.Server{ - Addr: listenAddr, - ReadTimeout: 10 * time.Second, - WriteTimeout: 10 * time.Second, - MaxHeaderBytes: 1 << 20, - Handler: s, - } - - s.SetupRoutes() - - s.log.Info("http begin listen", "listenaddr", listenAddr) - if err := s.httpServer.ListenAndServe(); err != nil && err != http.ErrServerClosed { - s.log.Error("listen error", "error", err) - if s.cancelFunc != nil { - s.cancelFunc() - } - } -} - -func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request) { - s.router.ServeHTTP(w, r) -} -``` - -### Signal Handling and Graceful Shutdown - -```go -func (s *Server) serve() int { - s.ctx, s.cancelFunc = context.WithCancel(context.Background()) - - // Signal watcher - go func() { - c := make(chan os.Signal, 1) - signal.Ignore(syscall.SIGPIPE) - signal.Notify(c, os.Interrupt, syscall.SIGTERM) - sig := <-c - s.log.Info("signal received", "signal", sig) - if s.cancelFunc != nil { - s.cancelFunc() - } - }() - - go s.serveUntilShutdown() - - for range s.ctx.Done() { - } - s.cleanShutdown() - return s.exitCode -} - -func (s *Server) cleanShutdown() { - s.exitCode = 0 - ctxShutdown, shutdownCancel := context.WithTimeout(context.Background(), 5*time.Second) - if err := s.httpServer.Shutdown(ctxShutdown); err != nil { - s.log.Error("server clean shutdown failed", "error", err) - } - if shutdownCancel != nil { - shutdownCancel() - } - s.cleanupForExit() - if s.sentryEnabled { - sentry.Flush(2 * time.Second) - } -} -``` - ---- - -## 5. Routing (go-chi) - -### Route Setup Pattern - -```go -// internal/server/routes.go -func (s *Server) SetupRoutes() { - s.router = chi.NewRouter() - - // Global middleware (applied to all routes) - s.router.Use(middleware.Recoverer) - s.router.Use(middleware.RequestID) - s.router.Use(s.mw.Logging()) - - // Conditional middleware - if viper.GetString("METRICS_USERNAME") != "" { - s.router.Use(s.mw.Metrics()) - } - - s.router.Use(s.mw.CORS()) - s.router.Use(middleware.Timeout(60 * time.Second)) - - if s.sentryEnabled { - sentryHandler := sentryhttp.New(sentryhttp.Options{ - Repanic: true, - }) - s.router.Use(sentryHandler.Handle) - } - - // Routes - s.router.Get("/", s.h.HandleIndex()) - - // Static files - s.router.Mount("/s", http.StripPrefix("/s", http.FileServer(http.FS(static.Static)))) - - // API versioning - s.router.Route("/api/v1", func(r chi.Router) { - r.Get("/now", s.h.HandleNow()) - }) - - // Routes with specific middleware - auth := s.mw.Auth() - s.router.Get("/login", auth(s.h.HandleLoginGET()).ServeHTTP) - - // Health check (standard path) - s.router.Get("/.well-known/healthcheck.json", s.h.HandleHealthCheck()) - - // Protected route groups - if viper.GetString("METRICS_USERNAME") != "" { - s.router.Group(func(r chi.Router) { - r.Use(s.mw.MetricsAuth()) - r.Get("/metrics", http.HandlerFunc(promhttp.Handler().ServeHTTP)) - }) - } -} -``` - -### Middleware Ordering (Critical) - -1. `middleware.Recoverer` - Panic recovery (must be first) -2. `middleware.RequestID` - Generate request IDs -3. `s.mw.Logging()` - Request logging -4. `s.mw.Metrics()` - Prometheus metrics (if enabled) -5. `s.mw.CORS()` - CORS headers -6. `middleware.Timeout(60s)` - Request timeout -7. `sentryhttp.Handler` - Sentry error reporting (if enabled) - -### API Versioning - -Use route groups for API versioning: - -```go -s.router.Route("/api/v1", func(r chi.Router) { - r.Get("/resource", s.h.HandleResource()) -}) -``` - -### Static File Serving - -Static files are served at `/s/` prefix: - -```go -s.router.Mount("/s", http.StripPrefix("/s", http.FileServer(http.FS(static.Static)))) -``` - ---- - -## 6. Handler Conventions - -### Handler Base Struct - -```go -// internal/handlers/handlers.go -type HandlersParams struct { - fx.In - Logger *logger.Logger - Globals *globals.Globals - Database *database.Database - Healthcheck *healthcheck.Healthcheck -} - -type Handlers struct { - params *HandlersParams - log *slog.Logger - hc *healthcheck.Healthcheck -} - -func New(lc fx.Lifecycle, params HandlersParams) (*Handlers, error) { - s := new(Handlers) - s.params = ¶ms - s.log = params.Logger.Get() - s.hc = params.Healthcheck - lc.Append(fx.Hook{ - OnStart: func(ctx context.Context) error { - // Compile templates or other initialization - return nil - }, - }) - return s, nil -} -``` - -### Closure-Based Handler Pattern - -All handlers return `http.HandlerFunc` using the closure pattern. This allows initialization logic to run once when the handler is created: - -```go -// internal/handlers/index.go -func (s *Handlers) HandleIndex() http.HandlerFunc { - // Initialization runs once - t := templates.GetParsed() - - // Handler runs per-request - return func(w http.ResponseWriter, r *http.Request) { - err := t.ExecuteTemplate(w, "index.html", nil) - if err != nil { - s.log.Error("template execution failed", "error", err) - http.Error(w, http.StatusText(500), 500) - } - } -} -``` - -### JSON Handler Pattern - -```go -// internal/handlers/now.go -func (s *Handlers) HandleNow() http.HandlerFunc { - // Response struct defined in closure scope - type response struct { - Now time.Time `json:"now"` - } - return func(w http.ResponseWriter, r *http.Request) { - s.respondJSON(w, r, &response{Now: time.Now()}, 200) - } -} -``` - -### Response Helpers - -```go -// internal/handlers/handlers.go -func (s *Handlers) respondJSON(w http.ResponseWriter, r *http.Request, data interface{}, status int) { - w.WriteHeader(status) - w.Header().Set("Content-Type", "application/json") - if data != nil { - err := json.NewEncoder(w).Encode(data) - if err != nil { - s.log.Error("json encode error", "error", err) - } - } -} - -func (s *Handlers) decodeJSON(w http.ResponseWriter, r *http.Request, v interface{}) error { - return json.NewDecoder(r.Body).Decode(v) -} -``` - -### Handler Naming Convention - -- `HandleIndex()` - Main page -- `HandleLoginGET()` / `HandleLoginPOST()` - Form handlers with HTTP method suffix -- `HandleNow()` - API endpoints -- `HandleHealthCheck()` - System endpoints - ---- - -## 7. Middleware Conventions - -### Middleware Struct - -```go -// internal/middleware/middleware.go -type MiddlewareParams struct { - fx.In - Logger *logger.Logger - Globals *globals.Globals - Config *config.Config -} - -type Middleware struct { - log *slog.Logger - params *MiddlewareParams -} - -func New(lc fx.Lifecycle, params MiddlewareParams) (*Middleware, error) { - s := new(Middleware) - s.params = ¶ms - s.log = params.Logger.Get() - return s, nil -} -``` - -### Middleware Signature - -All custom middleware methods return `func(http.Handler) http.Handler`: - -```go -func (s *Middleware) Auth() func(http.Handler) http.Handler { - return func(next http.Handler) http.Handler { - return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - // Before request - s.log.Info("AUTH: before request") - - next.ServeHTTP(w, r) - - // After request (optional) - }) - } -} -``` - -### Logging Middleware with Status Capture - -```go -type loggingResponseWriter struct { - http.ResponseWriter - statusCode int -} - -func NewLoggingResponseWriter(w http.ResponseWriter) *loggingResponseWriter { - return &loggingResponseWriter{w, http.StatusOK} -} - -func (lrw *loggingResponseWriter) WriteHeader(code int) { - lrw.statusCode = code - lrw.ResponseWriter.WriteHeader(code) -} - -func (s *Middleware) Logging() func(http.Handler) http.Handler { - return func(next http.Handler) http.Handler { - return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - start := time.Now() - lrw := NewLoggingResponseWriter(w) - ctx := r.Context() - defer func() { - latency := time.Since(start) - s.log.InfoContext(ctx, "request", - "request_start", start, - "method", r.Method, - "url", r.URL.String(), - "useragent", r.UserAgent(), - "request_id", ctx.Value(middleware.RequestIDKey).(string), - "referer", r.Referer(), - "proto", r.Proto, - "remoteIP", ipFromHostPort(r.RemoteAddr), - "status", lrw.statusCode, - "latency_ms", latency.Milliseconds(), - ) - }() - next.ServeHTTP(lrw, r) - }) - } -} -``` - -### CORS Middleware - -```go -func (s *Middleware) CORS() func(http.Handler) http.Handler { - return cors.Handler(cors.Options{ - AllowedOrigins: []string{"*"}, - AllowedMethods: []string{"GET", "POST", "PUT", "DELETE", "OPTIONS"}, - AllowedHeaders: []string{"Accept", "Authorization", "Content-Type", "X-CSRF-Token"}, - ExposedHeaders: []string{"Link"}, - AllowCredentials: false, - MaxAge: 300, - }) -} -``` - -### Metrics Middleware - -```go -func (s *Middleware) Metrics() func(http.Handler) http.Handler { - mdlw := ghmm.New(ghmm.Config{ - Recorder: metrics.NewRecorder(metrics.Config{}), - }) - return func(next http.Handler) http.Handler { - return std.Handler("", mdlw, next) - } -} - -func (s *Middleware) MetricsAuth() func(http.Handler) http.Handler { - return basicauth.New( - "metrics", - map[string][]string{ - viper.GetString("METRICS_USERNAME"): { - viper.GetString("METRICS_PASSWORD"), - }, - }, - ) -} -``` - ---- - -## 8. Configuration (Viper) - -### Config Struct - -```go -// internal/config/config.go -type ConfigParams struct { - fx.In - Globals *globals.Globals - Logger *logger.Logger -} - -type Config struct { - DBURL string - Debug bool - MaintenanceMode bool - MetricsPassword string - MetricsUsername string - Port int - SentryDSN string - params *ConfigParams - log *slog.Logger -} -``` - -### Configuration Loading - -```go -func New(lc fx.Lifecycle, params ConfigParams) (*Config, error) { - log := params.Logger.Get() - name := params.Globals.Appname - - // Config file settings - viper.SetConfigName(name) - viper.SetConfigType("yaml") - viper.AddConfigPath(fmt.Sprintf("/etc/%s", name)) - viper.AddConfigPath(fmt.Sprintf("$HOME/.config/%s", name)) - - // Environment variables override everything - viper.AutomaticEnv() - - // Defaults - viper.SetDefault("DEBUG", "false") - viper.SetDefault("MAINTENANCE_MODE", "false") - viper.SetDefault("PORT", "8080") - viper.SetDefault("DBURL", "") - viper.SetDefault("SENTRY_DSN", "") - viper.SetDefault("METRICS_USERNAME", "") - viper.SetDefault("METRICS_PASSWORD", "") - - // Read config file (optional) - if err := viper.ReadInConfig(); err != nil { - if _, ok := err.(viper.ConfigFileNotFoundError); ok { - // Config file not found is OK - } else { - log.Error("config file malformed", "error", err) - panic(err) - } - } - - // Build config struct - s := &Config{ - DBURL: viper.GetString("DBURL"), - Debug: viper.GetBool("debug"), - Port: viper.GetInt("PORT"), - SentryDSN: viper.GetString("SENTRY_DSN"), - MaintenanceMode: viper.GetBool("MAINTENANCE_MODE"), - MetricsUsername: viper.GetString("METRICS_USERNAME"), - MetricsPassword: viper.GetString("METRICS_PASSWORD"), - log: log, - params: ¶ms, - } - - // Enable debug logging if configured - if s.Debug { - params.Logger.EnableDebugLogging() - s.log = params.Logger.Get() - } - - return s, nil -} -``` - -### Configuration Precedence - -1. **Environment variables** (highest priority via `AutomaticEnv()`) -2. **`.env` file** (loaded via `godotenv/autoload` import) -3. **Config files**: `/etc/{appname}/{appname}.yaml`, `~/.config/{appname}/{appname}.yaml` -4. **Defaults** (lowest priority) - -### Environment Loading - -Import godotenv with autoload to automatically load `.env` files: - -```go -import ( - _ "github.com/joho/godotenv/autoload" -) -``` - ---- - -## 9. Logging (slog) - -### Logger Struct - -```go -// internal/logger/logger.go -type LoggerParams struct { - fx.In - Globals *globals.Globals -} - -type Logger struct { - log *slog.Logger - level *slog.LevelVar - params LoggerParams -} -``` - -### Logger Setup with TTY Detection - -```go -func New(lc fx.Lifecycle, params LoggerParams) (*Logger, error) { - l := new(Logger) - l.level = new(slog.LevelVar) - l.level.Set(slog.LevelInfo) - - // TTY detection for dev vs prod output - tty := false - if fileInfo, _ := os.Stdout.Stat(); (fileInfo.Mode() & os.ModeCharDevice) != 0 { - tty = true - } - - var handler slog.Handler - if tty { - // Text output for development - handler = slog.NewTextHandler(os.Stdout, &slog.HandlerOptions{ - Level: l.level, - AddSource: true, - }) - } else { - // JSON output for production - handler = slog.NewJSONHandler(os.Stdout, &slog.HandlerOptions{ - Level: l.level, - AddSource: true, - }) - } - - l.log = slog.New(handler) - return l, nil -} -``` - -### Logger Methods - -```go -func (l *Logger) EnableDebugLogging() { - l.level.Set(slog.LevelDebug) - l.log.Debug("debug logging enabled", "debug", true) -} - -func (l *Logger) Get() *slog.Logger { - return l.log -} - -func (l *Logger) Identify() { - l.log.Info("starting", - "appname", l.params.Globals.Appname, - "version", l.params.Globals.Version, - "buildarch", l.params.Globals.Buildarch, - ) -} -``` - -### Logging Patterns - -```go -// Info with fields -s.log.Info("message", "key", "value") - -// Error with error object -s.log.Error("operation failed", "error", err) - -// With context -s.log.InfoContext(ctx, "processing request", "request_id", reqID) - -// Structured request logging -s.log.Info("request completed", - "request_start", start, - "method", r.Method, - "url", r.URL.String(), - "status", statusCode, - "latency_ms", latency.Milliseconds(), -) - -// Using slog.Group for nested attributes -s.log.Info("request", - slog.Group("http", - "method", r.Method, - "url", r.URL.String(), - ), - slog.Group("timing", - "start", start, - "latency_ms", latency.Milliseconds(), - ), -) -``` - ---- - -## 10. Database Wrapper - -### Database Struct - -```go -// internal/database/database.go -type DatabaseParams struct { - fx.In - Logger *logger.Logger - Config *config.Config -} - -type Database struct { - URL string - log *slog.Logger - params *DatabaseParams -} -``` - -### Database Factory with Lifecycle - -```go -func New(lc fx.Lifecycle, params DatabaseParams) (*Database, error) { - s := new(Database) - s.params = ¶ms - s.log = params.Logger.Get() - - s.log.Info("Database instantiated") - - lc.Append(fx.Hook{ - OnStart: func(ctx context.Context) error { - s.log.Info("Database OnStart Hook") - // Connect to database here - // Example: s.db, err = sql.Open("postgres", s.params.Config.DBURL) - return nil - }, - OnStop: func(ctx context.Context) error { - // Disconnect from database here - // Example: s.db.Close() - return nil - }, - }) - return s, nil -} -``` - -### Usage Pattern - -The Database struct is injected into handlers and other services: - -```go -type HandlersParams struct { - fx.In - Database *database.Database - // ... -} - -// Access in handler -func (s *Handlers) HandleSomething() http.HandlerFunc { - return func(w http.ResponseWriter, r *http.Request) { - // Use s.params.Database - } -} -``` - ---- - -## 11. Globals Package - -### Package Variables and Struct - -```go -// internal/globals/globals.go -package globals - -import "go.uber.org/fx" - -// Package-level variables (set from main) -var ( - Appname string - Version string - Buildarch string -) - -// Struct for DI -type Globals struct { - Appname string - Version string - Buildarch string -} - -func New(lc fx.Lifecycle) (*Globals, error) { - n := &Globals{ - Appname: Appname, - Buildarch: Buildarch, - Version: Version, - } - return n, nil -} -``` - -### Setting Globals in Main - -```go -// cmd/httpd/main.go -var ( - Appname string = "CHANGEME" // Default, overridden by build - Version string // Set at build time - Buildarch string // Set at build time -) - -func main() { - globals.Appname = Appname - globals.Version = Version - globals.Buildarch = Buildarch - // ... -} -``` - -### Build-Time Variable Injection - -Use ldflags to inject version information at build time: - -```makefile -VERSION := $(shell git describe --tags --always) -BUILDARCH := $(shell go env GOARCH) - -build: - go build -ldflags "-X main.Version=$(VERSION) -X main.Buildarch=$(BUILDARCH)" ./cmd/httpd -``` - ---- - -## 12. Static Assets & Templates - -### Static File Embedding - -```go -// static/static.go -package static - -import "embed" - -//go:embed css js -var Static embed.FS -``` - -Directory structure: -``` -static/ -├── static.go -├── css/ -│ ├── bootstrap-4.5.3.min.css -│ └── style.css -└── js/ - ├── bootstrap-4.5.3.bundle.min.js - └── jquery-3.5.1.slim.min.js -``` - -### Template Embedding and Lazy Parsing - -```go -// templates/templates.go -package templates - -import ( - "embed" - "text/template" -) - -//go:embed *.html -var TemplatesRaw embed.FS -var TemplatesParsed *template.Template - -func GetParsed() *template.Template { - if TemplatesParsed == nil { - TemplatesParsed = template.Must(template.ParseFS(TemplatesRaw, "*")) - } - return TemplatesParsed -} -``` - -### Template Composition - -Templates use Go's template composition: - -```html - -{{ template "htmlheader.html" . }} -{{ template "navbar.html" . }} - -
- -
- -{{ template "pagefooter.html" . }} -{{ template "htmlfooter.html" . }} -``` - -### Static Asset References - -Reference static files with `/s/` prefix: - -```html - - - -``` - ---- - -## 13. Health Check - -### Health Check Service - -```go -// internal/healthcheck/healthcheck.go -type HealthcheckParams struct { - fx.In - Globals *globals.Globals - Config *config.Config - Logger *logger.Logger - Database *database.Database -} - -type Healthcheck struct { - StartupTime time.Time - log *slog.Logger - params *HealthcheckParams -} - -func New(lc fx.Lifecycle, params HealthcheckParams) (*Healthcheck, error) { - s := new(Healthcheck) - s.params = ¶ms - s.log = params.Logger.Get() - - lc.Append(fx.Hook{ - OnStart: func(ctx context.Context) error { - s.StartupTime = time.Now() - return nil - }, - OnStop: func(ctx context.Context) error { - return nil - }, - }) - return s, nil -} -``` - -### Health Check Response - -```go -type HealthcheckResponse struct { - Status string `json:"status"` - Now string `json:"now"` - UptimeSeconds int64 `json:"uptime_seconds"` - UptimeHuman string `json:"uptime_human"` - Version string `json:"version"` - Appname string `json:"appname"` - Maintenance bool `json:"maintenance_mode"` -} - -func (s *Healthcheck) uptime() time.Duration { - return time.Since(s.StartupTime) -} - -func (s *Healthcheck) Healthcheck() *HealthcheckResponse { - resp := &HealthcheckResponse{ - Status: "ok", - Now: time.Now().UTC().Format(time.RFC3339Nano), - UptimeSeconds: int64(s.uptime().Seconds()), - UptimeHuman: s.uptime().String(), - Appname: s.params.Globals.Appname, - Version: s.params.Globals.Version, - } - return resp -} -``` - -### Standard Endpoint - -Health check is served at the standard `.well-known` path: - -```go -s.router.Get("/.well-known/healthcheck.json", s.h.HandleHealthCheck()) -``` - ---- - -## 14. External Integrations - -### Sentry Error Reporting - -Sentry is conditionally enabled based on `SENTRY_DSN` environment variable: - -```go -func (s *Server) enableSentry() { - s.sentryEnabled = false - - if s.params.Config.SentryDSN == "" { - return - } - - err := sentry.Init(sentry.ClientOptions{ - Dsn: s.params.Config.SentryDSN, - Release: fmt.Sprintf("%s-%s", s.params.Globals.Appname, s.params.Globals.Version), - }) - if err != nil { - s.log.Error("sentry init failure", "error", err) - os.Exit(1) - return - } - s.log.Info("sentry error reporting activated") - s.sentryEnabled = true -} -``` - -Sentry middleware with repanic (bubbles panics to chi's Recoverer): - -```go -if s.sentryEnabled { - sentryHandler := sentryhttp.New(sentryhttp.Options{ - Repanic: true, - }) - s.router.Use(sentryHandler.Handle) -} -``` - -Flush Sentry on shutdown: - -```go -if s.sentryEnabled { - sentry.Flush(2 * time.Second) -} -``` - -### Prometheus Metrics - -Metrics are conditionally enabled and protected by basic auth: - -```go -// Only enable if credentials are configured -if viper.GetString("METRICS_USERNAME") != "" { - s.router.Use(s.mw.Metrics()) -} - -// Protected /metrics endpoint -if viper.GetString("METRICS_USERNAME") != "" { - s.router.Group(func(r chi.Router) { - r.Use(s.mw.MetricsAuth()) - r.Get("/metrics", http.HandlerFunc(promhttp.Handler().ServeHTTP)) - }) -} -``` - -### Environment Variables Summary - -| Variable | Description | Default | -|----------|-------------|---------| -| `PORT` | HTTP listen port | 8080 | -| `DEBUG` | Enable debug logging | false | -| `DBURL` | Database connection URL | "" | -| `SENTRY_DSN` | Sentry DSN for error reporting | "" | -| `MAINTENANCE_MODE` | Enable maintenance mode | false | -| `METRICS_USERNAME` | Basic auth username for /metrics | "" | -| `METRICS_PASSWORD` | Basic auth password for /metrics | "" | diff --git a/LICENSE b/LICENSE new file mode 100644 index 0000000..34edefe --- /dev/null +++ b/LICENSE @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2026 sneak + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/Makefile b/Makefile index 1da1f2d..1775f2a 100644 --- a/Makefile +++ b/Makefile @@ -1,4 +1,4 @@ -.PHONY: all build lint fmt test check clean +.PHONY: all build lint fmt fmt-check test check clean docker hooks BINARY := dnswatcher VERSION := $(shell git describe --tags --always --dirty 2>/dev/null || echo "dev") @@ -17,21 +17,26 @@ fmt: gofmt -s -w . goimports -w . +fmt-check: + @test -z "$$(gofmt -l .)" || (echo "Files not formatted:" && gofmt -l . && exit 1) + test: - go test -v -race -cover ./... + go test -v -race -cover -timeout 30s ./... # Check runs all validation without making changes # Used by CI and Docker build - fails if anything is wrong -check: - @echo "==> Checking formatting..." - @test -z "$$(gofmt -l .)" || (echo "Files not formatted:" && gofmt -l . && exit 1) - @echo "==> Running linter..." - golangci-lint run --config .golangci.yml ./... - @echo "==> Running tests..." - go test -v -race ./... +check: fmt-check lint test @echo "==> Building..." go build -ldflags "$(LDFLAGS)" -o /dev/null ./cmd/dnswatcher @echo "==> All checks passed!" +docker: + docker build . + +hooks: + @printf '#!/bin/sh\nset -e\nmake check\n' > .git/hooks/pre-commit + @chmod +x .git/hooks/pre-commit + @echo "Pre-commit hook installed." + clean: rm -rf bin/ diff --git a/README.md b/README.md index 0a9d555..3c16e9e 100644 --- a/README.md +++ b/README.md @@ -327,10 +327,13 @@ tracks reachability: ```sh make build # Build binary to bin/dnswatcher -make test # Run tests with race detector +make test # Run tests with race detector and 30s timeout make lint # Run golangci-lint -make fmt # Format code -make check # Run all checks (format, lint, test, build) +make fmt # Format code (writes) +make fmt-check # Read-only format check +make check # Run all checks (fmt-check, lint, test, build) +make docker # Build Docker image +make hooks # Install pre-commit hook make clean # Remove build artifacts ``` @@ -385,7 +388,17 @@ docker run -d \ ## Project Structure -Follows the conventions defined in `CONVENTIONS.md`, adapted from the +Follows the conventions defined in `REPO_POLICIES.md`, adapted from the [upaas](https://git.eeqj.de/sneak/upaas) project template. Uses uber/fx for dependency injection, go-chi for HTTP routing, slog for logging, and Viper for configuration. + +--- + +## License + +MIT — see [LICENSE](LICENSE). + +## Author + +[@sneak](https://sneak.berlin) diff --git a/REPO_POLICIES.md b/REPO_POLICIES.md new file mode 100644 index 0000000..a024cbd --- /dev/null +++ b/REPO_POLICIES.md @@ -0,0 +1,188 @@ +--- +title: Repository Policies +last_modified: 2026-02-22 +--- + +This document covers repository structure, tooling, and workflow standards. Code +style conventions are in separate documents: + +- [Code Styleguide](https://git.eeqj.de/sneak/prompts/raw/branch/main/prompts/CODE_STYLEGUIDE.md) + (general, bash, Docker) +- [Go](https://git.eeqj.de/sneak/prompts/raw/branch/main/prompts/CODE_STYLEGUIDE_GO.md) +- [JavaScript](https://git.eeqj.de/sneak/prompts/raw/branch/main/prompts/CODE_STYLEGUIDE_JS.md) +- [Python](https://git.eeqj.de/sneak/prompts/raw/branch/main/prompts/CODE_STYLEGUIDE_PYTHON.md) +- [Go HTTP Server Conventions](https://git.eeqj.de/sneak/prompts/raw/branch/main/prompts/GO_HTTP_SERVER_CONVENTIONS.md) + +--- + +- Cross-project documentation (such as this file) must include + `last_modified: YYYY-MM-DD` in the YAML front matter so it can be kept in sync + with the authoritative source as policies evolve. + +- **ALL external references must be pinned by cryptographic hash.** This + includes Docker base images, Go modules, npm packages, GitHub Actions, and + anything else fetched from a remote source. Version tags (`@v4`, `@latest`, + `:3.21`, etc.) are server-mutable and therefore remote code execution + vulnerabilities. The ONLY acceptable way to reference an external dependency + is by its content hash (Docker `@sha256:...`, Go module hash in `go.sum`, npm + integrity hash in lockfile, GitHub Actions `@`). No exceptions. + This also means never `curl | bash` to install tools like pyenv, nvm, rustup, + etc. Instead, download a specific release archive from GitHub, verify its hash + (hardcoded in the Dockerfile or script), and only then install. Unverified + install scripts are arbitrary remote code execution. This is the single most + important rule in this document. Double-check every external reference in + every file before committing. There are zero exceptions to this rule. + +- Every repo with software must have a root `Makefile` with these targets: + `make test`, `make lint`, `make fmt` (writes), `make fmt-check` (read-only), + `make check` (prereqs: `test`, `lint`, `fmt-check`), `make docker`, and + `make hooks` (installs pre-commit hook). A model Makefile is at + `https://git.eeqj.de/sneak/prompts/raw/branch/main/Makefile`. + +- Always use Makefile targets (`make fmt`, `make test`, `make lint`, etc.) + instead of invoking the underlying tools directly. The Makefile is the single + source of truth for how these operations are run. + +- The Makefile is authoritative documentation for how the repo is used. Beyond + the required targets above, it should have targets for every common operation: + running a local development server (`make run`, `make dev`), re-initializing + or migrating the database (`make db-reset`, `make migrate`), building + artifacts (`make build`), generating code, seeding data, or anything else a + developer would do regularly. If someone checks out the repo and types + `make`, they should see every meaningful operation available. A new + contributor should be able to understand the entire development workflow by + reading the Makefile. + +- Every repo should have a `Dockerfile`. All Dockerfiles must run `make check` + as a build step so the build fails if the branch is not green. For non-server + repos, the Dockerfile should bring up a development environment and run + `make check`. For server repos, `make check` should run as an early build + stage before the final image is assembled. + +- Every repo should have a Gitea Actions workflow (`.gitea/workflows/`) that + runs `docker build .` on push. Since the Dockerfile already runs `make check`, + a successful build implies all checks pass. + +- Use platform-standard formatters: `black` for Python, `prettier` for + JS/CSS/Markdown/HTML, `go fmt` for Go. Always use default configuration with + two exceptions: four-space indents (except Go), and `proseWrap: always` for + Markdown (hard-wrap at 80 columns). Documentation and writing repos (Markdown, + HTML, CSS) should also have `.prettierrc` and `.prettierignore`. + +- Pre-commit hook: `make check` if local testing is possible, otherwise + `make lint && make fmt-check`. The Makefile should provide a `make hooks` + target to install the pre-commit hook. + +- All repos with software must have tests that run via the platform-standard + test framework (`go test`, `pytest`, `jest`/`vitest`, etc.). If no meaningful + tests exist yet, add the most minimal test possible — e.g. importing the + module under test to verify it compiles/parses. There is no excuse for + `make test` to be a no-op. + +- `make test` must complete in under 20 seconds. Add a 30-second timeout in the + Makefile. + +- Docker builds must complete in under 5 minutes. + +- `make check` must not modify any files in the repo. Tests may use temporary + directories. + +- `main` must always pass `make check`, no exceptions. + +- Never commit secrets. `.env` files, credentials, API keys, and private keys + must be in `.gitignore`. No exceptions. + +- `.gitignore` should be comprehensive from the start: OS files (`.DS_Store`), + editor files (`.swp`, `*~`), language build artifacts, and `node_modules/`. + Fetch the standard `.gitignore` from + `https://git.eeqj.de/sneak/prompts/raw/branch/main/.gitignore` when setting up + a new repo. + +- Never use `git add -A` or `git add .`. Always stage files explicitly by name. + +- Never force-push to `main`. + +- Make all changes on a feature branch. You can do whatever you want on a + feature branch. + +- `.golangci.yml` is standardized and must _NEVER_ be modified by an agent, only + manually by the user. Fetch from + `https://git.eeqj.de/sneak/prompts/raw/branch/main/.golangci.yml`. + +- When pinning images or packages by hash, add a comment above the reference + with the version and date (YYYY-MM-DD). + +- Use `yarn`, not `npm`. + +- Write all dates as YYYY-MM-DD (ISO 8601). + +- Simple projects should be configured with environment variables. + +- Dockerized web services listen on port 8080 by default, overridable with + `PORT`. + +- `README.md` is the primary documentation. Required sections: + - **Description**: First line must include the project name, purpose, + category (web server, SPA, CLI tool, etc.), license, and author. Example: + "µPaaS is an MIT-licensed Go web application by @sneak that receives + git-frontend webhooks and deploys applications via Docker in realtime." + - **Getting Started**: Copy-pasteable install/usage code block. + - **Rationale**: Why does this exist? + - **Design**: How is the program structured? + - **TODO**: Update meticulously, even between commits. When planning, put + the todo list in the README so a new agent can pick up where the last one + left off. + - **License**: MIT, GPL, or WTFPL. Ask the user for new projects. Include a + `LICENSE` file in the repo root and a License section in the README. + - **Author**: [@sneak](https://sneak.berlin). + +- First commit of a new repo should contain only `README.md`. + +- Go module root: `sneak.berlin/go/`. Always run `go mod tidy` before + committing. + +- Use SemVer. + +- Database migrations live in `internal/db/migrations/` and must be embedded in + the binary. + - `000_migration.sql` — contains ONLY the creation of the migrations tracking + table itself. Nothing else. + - `001_schema.sql` — the full application schema. + - **Pre-1.0.0:** never add additional migration files (002, 003, etc.). There + is no installed base to migrate. Edit `001_schema.sql` directly. + - **Post-1.0.0:** add new numbered migration files for each schema change. + Never edit existing migrations after release. + +- All repos should have an `.editorconfig` enforcing the project's indentation + settings. + +- Avoid putting files in the repo root unless necessary. Root should contain + only project-level config files (`README.md`, `Makefile`, `Dockerfile`, + `LICENSE`, `.gitignore`, `.editorconfig`, `REPO_POLICIES.md`, and + language-specific config). Everything else goes in a subdirectory. Canonical + subdirectory names: + - `bin/` — executable scripts and tools + - `cmd/` — Go command entrypoints + - `configs/` — configuration templates and examples + - `deploy/` — deployment manifests (k8s, compose, terraform) + - `docs/` — documentation and markdown (README.md stays in root) + - `internal/` — Go internal packages + - `internal/db/migrations/` — database migrations + - `pkg/` — Go library packages + - `share/` — systemd units, data files + - `static/` — static assets (images, fonts, etc.) + - `web/` — web frontend source + +- When setting up a new repo, files from the `prompts` repo may be used as + templates. Fetch them from + `https://git.eeqj.de/sneak/prompts/raw/branch/main/`. + +- New repos must contain at minimum: + - `README.md`, `.git`, `.gitignore`, `.editorconfig` + - `LICENSE`, `REPO_POLICIES.md` (copy from the `prompts` repo) + - `Makefile` + - `Dockerfile`, `.dockerignore` + - `.gitea/workflows/check.yml` + - Go: `go.mod`, `go.sum`, `.golangci.yml` + - JS: `package.json`, `yarn.lock`, `.prettierrc`, `.prettierignore` + - Python: `pyproject.toml`