webhooker/pkg/config

Configuration Module (Go)

A simple, clean, and generic configuration management system that supports multiple environments and automatic value resolution. This module is completely standalone and can be used in any Go project.

Features

• Simple API: Just config.Get() and config.GetSecret()
• Type-safe helpers: config.GetString(), config.GetInt(), config.GetBool()
• Environment Support: Separate configs for different environments (dev/prod/staging/etc)
• Value Resolution: Automatic resolution of special values:
  • $ENV:VARIABLE - Read from environment variable
  • $GSM:secret-name - Read from Google Secret Manager
  • $ASM:secret-name - Read from AWS Secrets Manager
  • $FILE:/path/to/file - Read from file contents
• Hierarchical Defaults: Environment-specific values override defaults
• YAML-based: Easy to read and edit configuration files
• Thread-safe: Safe for concurrent use
• Testable: Uses afero filesystem abstraction for easy testing
• Minimal Dependencies: Only requires YAML parser and cloud SDKs (optional)

Installation

go get git.eeqj.de/sneak/webhooker/pkg/config

Usage

package main

import (
    "fmt"
    "git.eeqj.de/sneak/webhooker/pkg/config"
)

func main() {
    // Set the environment explicitly
    config.SetEnvironment("prod")
    
    // Get configuration values
    baseURL := config.GetString("baseURL")
    apiTimeout := config.GetInt("timeout", 30)
    debugMode := config.GetBool("debugMode", false)
    
    // Get secret values
    apiKey := config.GetSecretString("api_key")
    dbPassword := config.GetSecretString("db_password", "default")
    
    // Get all values (for debugging)
    allConfig := config.GetAllConfig()
    allSecrets := config.GetAllSecrets()
    
    // Reload configuration from file
    if err := config.Reload(); err != nil {
        fmt.Printf("Failed to reload config: %v\n", err)
    }
}

Configuration File Structure

Create a config.yaml file in your project root:

environments:
  dev:
    config:
      baseURL: https://dev.example.com
      debugMode: true
      timeout: 30
    secrets:
      api_key: dev-key-12345
      db_password: $ENV:DEV_DB_PASSWORD
  
  prod:
    config:
      baseURL: https://prod.example.com
      debugMode: false
      timeout: 10
      GCPProject: my-project-123
      AWSRegion: us-west-2
    secrets:
      api_key: $GSM:prod-api-key
      db_password: $ASM:prod/db/password

configDefaults:
  app_name: my-app
  timeout: 30
  log_level: INFO
  port: 8080

How It Works

1. Environment Selection: Call config.SetEnvironment("prod") to select which environment to use

2. Value Lookup: When you call config.Get("key"):

   • First checks environments.<env>.config.key
   • Falls back to configDefaults.key
   • Returns the default value if not found

3. Secret Lookup: When you call config.GetSecret("key"):

   • Looks in environments.<env>.secrets.key
   • Returns the default value if not found

4. Value Resolution: If a value starts with a special prefix:

   • $ENV: - Reads from environment variable
   • $GSM: - Fetches from Google Secret Manager (requires GCPProject to be set in config)
   • $ASM: - Fetches from AWS Secrets Manager (uses AWSRegion from config or defaults to us-east-1)
   • $FILE: - Reads from file (supports ~ expansion)

Type-Safe Access

The module provides type-safe helper functions:

// String values
baseURL := config.GetString("baseURL", "http://localhost")

// Integer values
port := config.GetInt("port", 8080)

// Boolean values
debug := config.GetBool("debug", false)

// Secret string values
apiKey := config.GetSecretString("api_key", "default-key")

Local Development

For local development, you can:

1. Use environment variables:

   secrets:
     api_key: $ENV:LOCAL_API_KEY
   

2. Use local files:

   secrets:
     api_key: $FILE:~/.secrets/api-key.txt
   

3. Create a config.local.yaml (gitignored) with literal values for testing

Cloud Provider Support

Google Secret Manager

To use GSM resolution ($GSM: prefix):

1. Set GCPProject in your config
2. Ensure proper authentication (e.g., GOOGLE_APPLICATION_CREDENTIALS environment variable)
3. The module will automatically initialize the GSM client when needed

AWS Secrets Manager

To use ASM resolution ($ASM: prefix):

1. Optionally set AWSRegion in your config (defaults to us-east-1)
2. Ensure proper authentication (e.g., AWS credentials in environment or IAM role)
3. The module will automatically initialize the ASM client when needed

Advanced Usage

Loading from a Specific File

// Load configuration from a specific file
if err := config.LoadFile("/path/to/config.yaml"); err != nil {
    log.Fatal(err)
}

Checking Configuration Values

// Get all configuration for current environment
allConfig := config.GetAllConfig()
for key, value := range allConfig {
    fmt.Printf("%s: %v\n", key, value)
}

// Get all secrets (be careful with logging!)
allSecrets := config.GetAllSecrets()

Testing

The module uses the afero filesystem abstraction, making it easy to test without real files:

package myapp_test

import (
    "testing"
    "github.com/spf13/afero"
    "git.eeqj.de/sneak/webhooker/pkg/config"
)

func TestMyApp(t *testing.T) {
    // Create an in-memory filesystem for testing
    fs := afero.NewMemMapFs()
    
    // Write a test config file
    testConfig := `
environments:
  test:
    config:
      apiURL: http://test.example.com
    secrets:
      apiKey: test-key-123
`
    afero.WriteFile(fs, "config.yaml", []byte(testConfig), 0644)
    
    // Use the test filesystem
    config.SetFs(fs)
    config.SetEnvironment("test")
    
    // Now your tests use the in-memory config
    if url := config.GetString("apiURL"); url != "http://test.example.com" {
        t.Errorf("Expected test URL, got %s", url)
    }
}

Unit Testing with Isolated Config

For unit tests, you can create isolated configuration managers:

func TestMyComponent(t *testing.T) {
    // Create a test-specific manager
    manager := config.NewManager()
    
    // Use in-memory filesystem
    fs := afero.NewMemMapFs()
    afero.WriteFile(fs, "config.yaml", []byte(testConfig), 0644)
    manager.SetFs(fs)
    
    // Test with isolated configuration
    manager.SetEnvironment("test")
    value := manager.Get("someKey", "default")
}

Error Handling

• If a config file is not found when using the default loader, an error is returned
• If a key is not found, the default value is returned
• If a special value cannot be resolved (e.g., env var not set, file not found), nil is returned
• Cloud provider errors are logged but return nil to allow graceful degradation

Thread Safety

All operations are thread-safe. The module uses read-write mutexes to ensure safe concurrent access to configuration data.

Example Integration

package main

import (
    "log"
    "os"
    "git.eeqj.de/sneak/webhooker/pkg/config"
)

func main() {
    // Read environment from your app-specific env var
    environment := os.Getenv("APP_ENV")
    if environment == "" {
        environment = "dev"
    }
    
    config.SetEnvironment(environment)
    
    // Now use configuration throughout your app
    databaseURL := config.GetString("database_url")
    apiKey := config.GetSecretString("api_key")
    
    log.Printf("Running in %s environment", environment)
    log.Printf("Database URL: %s", databaseURL)
}

Migration from Python Version

The Go version maintains API compatibility with the Python version where possible:

Python	Go
config.get('key')	config.Get("key") or config.GetString("key")
config.getSecret('key')	config.GetSecret("key") or config.GetSecretString("key")
config.set_environment('prod')	config.SetEnvironment("prod")
config.reload()	config.Reload()
config.get_all_config()	config.GetAllConfig()
config.get_all_secrets()	config.GetAllSecrets()

License

This module is designed to be standalone and can be extracted into its own repository with your preferred license.