4 Commits

Author SHA1 Message Date
c1ec038c99 docs: document cache_max_bytes, update TODO.md (closes #51)
All checks were successful
check / check (push) Successful in 1m40s
Add cache_max_bytes to config.example.yml and the README key settings
list. TODO.md: move cache size management and eviction to Completed
Steps, promote P1 blocked networks configuration into Next Step, and
note in Status that the unbounded disk growth DoS vector is closed.
2026-08-07 21:12:05 +00:00
bdd86a4c1e feat: DB-tracked cache size accounting with background LRU eviction
Migration 002 adds a variant_content table (processed variants were
untracked on disk) and an LRU timestamp on source_content. Total usage
is two SUMs, never a directory scan on the hot path; hits touch LRU
timestamps best-effort. A background goroutine evicts globally
least-recently-used entries (variants and source blobs merged) until
usage is under MaxBytes, woken by a periodic ticker and by non-blocking
write-pressure notifications from stores. Evicting a source blob
deletes all source_metadata rows referencing it plus its
source_content row in one transaction before the file is unlinked, so
multi-referenced blobs are removed only with all their references and
rows never point at deleted files; JSON sidecars are cleaned up too. A
one-time startup reconciliation walk adopts untracked variant files,
drops rows whose files are missing, removes unreachable source blobs,
and sweeps stale temp files. CacheConfig.DisableDiskCache turns the
disk cache off entirely (config maps cache_max_bytes: 0 to it): no
directories, lookups miss, stores no-op, no evictor. Handlers wire the
limit, start eviction on startup, and stop it on shutdown.
2026-08-07 21:06:03 +00:00
8cb09b6aaf feat: add cache_max_bytes config key with statfs-derived default
Strict int64 parsing via the startup validation framework: a SET but
invalid value (negative, float, null, non-numeric) aborts startup
naming the key and value. An omitted key resolves after state_dir
validation to max(75% of free bytes on the filesystem containing
<state_dir>/cache/, 500 MiB), measured via an injectable statfs probe;
the floor never applies to explicit values. Zero is valid and means
the disk cache is disabled. The effective limit is logged at startup.
2026-08-07 21:02:08 +00:00
3963ec31c1 test: add failing tests for cache_max_bytes config and cache eviction
Red phase for #51: covers strict cache_max_bytes parsing (invalid
explicit values abort naming key and value), the computed default of
max(75% of free space, 500 MiB) via an injectable free-space probe,
explicit-value-no-floor, zero-disables-cache, size accounting over
source blobs and variants, LRU eviction under the limit, the
multi-referenced blob case, write-pressure and periodic eviction
triggers, and startup reconciliation. Minimal API skeletons keep the
tree compiling and lint-clean; only the new tests fail.
2026-08-07 20:58:03 +00:00
63 changed files with 4296 additions and 3009 deletions

View File

@@ -1,34 +1,117 @@
version: "2"
# Config schema uses the golangci-lint v2 layout (settings live under
# linters.settings, not top-level linters-settings) so that the
# thresholds below are actually applied by golangci-lint >= v2.
run:
timeout: 5m
modules-download-mode: readonly
go: "1.24"
tests: false
linters:
default: all
disable:
# Genuinely incompatible with project patterns
- exhaustruct # Requires all struct fields
- depguard # Dependency allow/block lists
- godot # Requires comments to end with periods
- wsl # Deprecated, replaced by wsl_v5
- wrapcheck # Too verbose for internal packages
- varnamelen # Short names like db, id are idiomatic Go
settings:
enable:
# Additional linters requested
- testifylint # Checks usage of github.com/stretchr/testify
- usetesting # usetesting is an analyzer that detects using os.Setenv instead of t.Setenv since Go 1.17
# - tagliatelle # Disabled: we need snake_case for external API compatibility
- nlreturn # nlreturn checks for a new line before return and branch statements
- nilnil # Checks that there is no simultaneous return of nil error and an invalid value
- nestif # Reports deeply nested if statements
- mnd # An analyzer to detect magic numbers
- lll # Reports long lines
- intrange # intrange is a linter to find places where for loops could make use of an integer range
- gochecknoglobals # Check that no global variables exist
# Default/existing linters that are commonly useful
- govet
- errcheck
- staticcheck
- unused
- ineffassign
- misspell
- revive
- gosec
- unconvert
- unparam
linters-settings:
lll:
line-length: 88
funlen:
lines: 80
statements: 50
cyclop:
max-complexity: 15
dupl:
threshold: 100
line-length: 120
nestif:
min-complexity: 4
nlreturn:
block-size: 2
revive:
rules:
- name: var-naming
arguments:
- []
- []
- "upperCaseConst=true"
tagliatelle:
case:
rules:
json: snake
yaml: snake
xml: snake
bson: snake
testifylint:
enable-all: true
usetesting: {}
issues:
max-issues-per-linter: 0
max-same-issues: 0
exclude-rules:
# Exclude unused parameter warnings for cobra command signatures
- text: "parameter '(args|cmd)' seems to be unused"
linters:
- revive
# Allow ALL_CAPS constant names
- text: "don't use ALL_CAPS in Go names"
linters:
- revive
# Allow snake_case JSON tags for external API compatibility
- path: "internal/types/ris.go"
linters:
- tagliatelle
# Allow snake_case JSON tags for database models
- path: "internal/database/models.go"
linters:
- tagliatelle
# Allow generic package name for types that define data structures
- path: "internal/types/"
text: "avoid meaningless package names"
linters:
- revive
# Allow globals in the globals package (by design)
- path: "internal/globals/"
linters:
- gochecknoglobals
# Allow globals in main (Version/Buildarch set by ldflags)
- path: "cmd/"
linters:
- gochecknoglobals
# Allow blank imports for driver registration
- text: "blank-imports"
linters:
- revive
# Allow unused fx.Lifecycle parameters (required by fx signature)
- text: "parameter 'lc' seems to be unused"
linters:
- revive
# Allow unused context parameters in fx hooks
- text: "parameter 'ctx' seems to be unused"
linters:
- revive

View File

@@ -1,6 +1,6 @@
# Lint stage
# golangci/golangci-lint:v2.12.2-alpine, 2026-08-07
FROM golangci/golangci-lint:v2.12.2-alpine@sha256:91b27804074a0bacea298707f016911e60cf0cdbc6c7bf5ccacb5f0606d18d60 AS lint
# golangci/golangci-lint:v2.10.1-alpine, 2026-02-17
FROM golangci/golangci-lint:v2.10.1-alpine@sha256:33bc6b6156d4c7da87175f187090019769903d04dd408833b83083ed214b0ddf AS lint
RUN apk add --no-cache make build-base vips-dev libheif-dev pkgconfig

View File

@@ -115,6 +115,9 @@ Configured via YAML file (`--config`). Key settings:
- `upstream_max_response_size` — max origin response size
- `downstream_timeout` — client response timeout
- `signing_key` — HMAC secret for URL signatures
- `cache_max_bytes` — disk cache size limit in bytes; `0` disables the
disk cache entirely; omitted defaults to 75% of the free space on
the filesystem containing `<state_dir>/cache/` (minimum 500 MiB)
See `config.example.yml` for all options with defaults.

41
TODO.md
View File

@@ -12,28 +12,35 @@
pre-1.0. No git tags exist. Recent work extracted the internal/magic,
internal/allowlist, internal/httpfetcher, and internal/signature
packages. The gosec findings from the 2026-07-06 survey are resolved:
the last two open findings (G124, session cookie attributes in
internal/session) are fixed as of this change, so `make check` is green
on main.
packages. The gosec findings from the 2026-07-06 survey are resolved
and `make check` is green on main. The disk cache is now size-bounded
with LRU eviction (`cache_max_bytes`), closing the unbounded disk
growth DoS vector.
# Next Step
P0: implement cache size management and eviction so the disk cannot
fill up
P1: implement blocked networks configuration to extend SSRF protection
# Completed Steps
- 2026-08-07 update golangci-lint to v2.12.2 with the canonical
`.golangci.yml` (v2 schema, `default: all` minus six disabled
linters, `lll` 88, tests included): bumped the pinned
`golangci/golangci-lint:v2.12.2-alpine` image in `Dockerfile` and the
release-archive sha256 pins in `script/bootstrap`; fixed all 747
findings the stricter config surfaced (notably `paralleltest`,
`wsl_v5`, `goconst`, `lll`, `noinlineerr`, `err113`, `errcheck`,
`testpackage` — white-box test files renamed to
`*_internal_test.go`); three `//nolint:tagliatelle` directives keep
the snake_case JSON wire/disk formats unchanged; `make check` green
- 2026-08-07 implement cache size management and eviction (closes
#51): new `cache_max_bytes` config key validated by the startup
framework (explicit values used exactly with no floor, `0` disables
the disk cache entirely, omitted defaults to max(75% of free space
on the filesystem containing `<state_dir>/cache/`, 500 MiB), logged
at startup); processed variants are now tracked in the database
(migration 002 adds `variant_content` and an LRU timestamp on
`source_content`) so total usage is two SUMs, never a directory scan
on the hot path; a background goroutine evicts globally
least-recently-used entries (variants and source blobs merged) to
the limit, woken by a periodic ticker and by write-pressure
notifications from stores; a source blob and ALL of its
`source_metadata` references are deleted in one transaction before
the file is unlinked, so multi-referenced blobs are never removed
while referenced and rows never point at deleted files; a startup
reconciliation pass adopts untracked variant files, drops rows for
missing files, removes unreachable source blobs, and sweeps stale
temp files
- 2026-08-07 validate configuration on startup, fail fast on bad
config (closes #52): a config value that is set but unparseable or
invalid aborts startup naming the key and value (defaults apply only
@@ -89,8 +96,6 @@ fill up
# Future Steps
- P1: implement blocked networks configuration to extend SSRF
protection
- P1: rate limit global concurrent upstream fetches to prevent
resource exhaustion
- P1: strip EXIF and other metadata from processed images (privacy)

View File

@@ -30,8 +30,7 @@ func main() {
rootCmd.Flags().StringVarP(&configPath, "config", "c", "", "path to config file")
err := rootCmd.Execute()
if err != nil {
if err := rootCmd.Execute(); err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}

View File

@@ -28,6 +28,13 @@ allow_http: false
# Maximum concurrent connections per upstream host (default: 20)
upstream_connections_per_host: 20
# Maximum disk cache size in bytes. Explicit values are used exactly as
# given; 0 disables the disk cache entirely (every request fetches and
# processes uncached). When omitted, the default is 75% of the free
# space on the filesystem containing <state_dir>/cache/ at startup,
# with a minimum of 500 MiB.
# cache_max_bytes: 10737418240
# Sentry error reporting (optional)
sentry_dsn: ""

View File

@@ -10,8 +10,7 @@ import (
type HostAllowList struct {
// exactHosts contains hosts that must match exactly (e.g., "cdn.example.com")
exactHosts map[string]struct{}
// suffixHosts contains domain suffixes to match
// (e.g., ".example.com" matches "cdn.example.com")
// suffixHosts contains domain suffixes to match (e.g., ".example.com" matches "cdn.example.com")
suffixHosts []string
}

View File

@@ -7,37 +7,104 @@ import (
"sneak.berlin/go/pixa/internal/allowlist"
)
const (
testExactHost = "cdn.example.com"
testImageURL = "https://cdn.example.com/image.jpg"
testSuffix = ".example.com"
)
type isAllowedCase struct {
func TestHostAllowList_IsAllowed(t *testing.T) {
tests := []struct {
name string
patterns []string
testURL string
want bool
}
func runIsAllowedCases(t *testing.T, tests []isAllowedCase) {
t.Helper()
}{
{
name: "exact match",
patterns: []string{"cdn.example.com"},
testURL: "https://cdn.example.com/image.jpg",
want: true,
},
{
name: "exact match case insensitive",
patterns: []string{"CDN.Example.COM"},
testURL: "https://cdn.example.com/image.jpg",
want: true,
},
{
name: "exact match not found",
patterns: []string{"cdn.example.com"},
testURL: "https://other.example.com/image.jpg",
want: false,
},
{
name: "suffix match",
patterns: []string{".example.com"},
testURL: "https://cdn.example.com/image.jpg",
want: true,
},
{
name: "suffix match deep subdomain",
patterns: []string{".example.com"},
testURL: "https://cdn.images.example.com/image.jpg",
want: true,
},
{
name: "suffix match apex domain",
patterns: []string{".example.com"},
testURL: "https://example.com/image.jpg",
want: true,
},
{
name: "suffix match not found",
patterns: []string{".example.com"},
testURL: "https://notexample.com/image.jpg",
want: false,
},
{
name: "suffix match partial not allowed",
patterns: []string{".example.com"},
testURL: "https://fakeexample.com/image.jpg",
want: false,
},
{
name: "multiple patterns",
patterns: []string{"cdn.example.com", ".images.org", "static.test.net"},
testURL: "https://photos.images.org/image.jpg",
want: true,
},
{
name: "empty allow list",
patterns: []string{},
testURL: "https://cdn.example.com/image.jpg",
want: false,
},
{
name: "nil url",
patterns: []string{"cdn.example.com"},
testURL: "",
want: false,
},
{
name: "url with port",
patterns: []string{"cdn.example.com"},
testURL: "https://cdn.example.com:443/image.jpg",
want: true,
},
{
name: "whitespace in patterns",
patterns: []string{" cdn.example.com ", " .other.com "},
testURL: "https://cdn.example.com/image.jpg",
want: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
w := allowlist.New(tt.patterns)
var u *url.URL
if tt.testURL != "" {
parsed, err := url.Parse(tt.testURL)
var err error
u, err = url.Parse(tt.testURL)
if err != nil {
t.Fatalf("failed to parse test URL: %v", err)
}
u = parsed
}
got := w.IsAllowed(u)
@@ -48,101 +115,7 @@ func runIsAllowedCases(t *testing.T, tests []isAllowedCase) {
}
}
func TestHostAllowList_IsAllowed_ExactMatch(t *testing.T) {
t.Parallel()
runIsAllowedCases(t, []isAllowedCase{
{
name: "exact match",
patterns: []string{testExactHost},
testURL: testImageURL,
want: true,
},
{
name: "exact match case insensitive",
patterns: []string{"CDN.Example.COM"},
testURL: testImageURL,
want: true,
},
{
name: "exact match not found",
patterns: []string{testExactHost},
testURL: "https://other.example.com/image.jpg",
want: false,
},
{
name: "multiple patterns",
patterns: []string{testExactHost, ".images.org", "static.test.net"},
testURL: "https://photos.images.org/image.jpg",
want: true,
},
{
name: "empty allow list",
patterns: []string{},
testURL: testImageURL,
want: false,
},
{
name: "nil url",
patterns: []string{testExactHost},
testURL: "",
want: false,
},
{
name: "url with port",
patterns: []string{testExactHost},
testURL: "https://cdn.example.com:443/image.jpg",
want: true,
},
{
name: "whitespace in patterns",
patterns: []string{" cdn.example.com ", " .other.com "},
testURL: testImageURL,
want: true,
},
})
}
func TestHostAllowList_IsAllowed_SuffixMatch(t *testing.T) {
t.Parallel()
runIsAllowedCases(t, []isAllowedCase{
{
name: "suffix match",
patterns: []string{testSuffix},
testURL: testImageURL,
want: true,
},
{
name: "suffix match deep subdomain",
patterns: []string{testSuffix},
testURL: "https://cdn.images.example.com/image.jpg",
want: true,
},
{
name: "suffix match apex domain",
patterns: []string{testSuffix},
testURL: "https://example.com/image.jpg",
want: true,
},
{
name: "suffix match not found",
patterns: []string{testSuffix},
testURL: "https://notexample.com/image.jpg",
want: false,
},
{
name: "suffix match partial not allowed",
patterns: []string{testSuffix},
testURL: "https://fakeexample.com/image.jpg",
want: false,
},
})
}
func TestHostAllowList_IsEmpty(t *testing.T) {
t.Parallel()
tests := []struct {
name string
patterns []string
@@ -172,8 +145,6 @@ func TestHostAllowList_IsEmpty(t *testing.T) {
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
w := allowlist.New(tt.patterns)
if got := w.IsEmpty(); got != tt.want {
t.Errorf("IsEmpty() = %v, want %v", got, tt.want)
@@ -183,8 +154,6 @@ func TestHostAllowList_IsEmpty(t *testing.T) {
}
func TestHostAllowList_Count(t *testing.T) {
t.Parallel()
tests := []struct {
name string
patterns []string
@@ -214,8 +183,6 @@ func TestHostAllowList_Count(t *testing.T) {
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
w := allowlist.New(tt.patterns)
if got := w.Count(); got != tt.want {
t.Errorf("Count() = %v, want %v", got, tt.want)

View File

@@ -0,0 +1,271 @@
package config
import (
"errors"
"io"
"log/slog"
"os"
"path/filepath"
"strings"
"testing"
)
// discardLogger returns a logger that swallows all output, for tests
// that exercise code paths which log.
func discardLogger() *slog.Logger {
return slog.New(slog.NewTextHandler(io.Discard, nil))
}
// TestCacheMaxBytesExplicitValueUsedWithoutFloor verifies that an
// explicitly configured cache_max_bytes value is used exactly as
// given: the 500 MiB floor applies only to the computed default, never
// to explicit values.
func TestCacheMaxBytesExplicitValueUsedWithoutFloor(t *testing.T) {
yamlContent := "signing_key: " + validTestSigningKey + "\ncache_max_bytes: 1024\n"
c, err := configFromYAML(t, yamlContent)
if err != nil {
t.Fatalf("explicit cache_max_bytes must be accepted, got error: %v", err)
}
if c.CacheMaxBytes != 1024 {
t.Errorf("CacheMaxBytes = %d, want 1024 (no floor for explicit values)",
c.CacheMaxBytes)
}
}
// TestCacheMaxBytesZeroIsValidAndDisablesCache verifies that an
// explicit zero is a valid value (it disables the disk cache), not an
// error.
func TestCacheMaxBytesZeroIsValidAndDisablesCache(t *testing.T) {
yamlContent := "signing_key: " + validTestSigningKey + "\ncache_max_bytes: 0\n"
c, err := configFromYAML(t, yamlContent)
if err != nil {
t.Fatalf("cache_max_bytes: 0 must be accepted, got error: %v", err)
}
if c.CacheMaxBytes != 0 {
t.Errorf("CacheMaxBytes = %d, want 0", c.CacheMaxBytes)
}
}
// TestCacheMaxBytesLargeExplicitValueParses verifies that values above
// 32-bit range parse correctly (the field is an int64 byte count).
func TestCacheMaxBytesLargeExplicitValueParses(t *testing.T) {
yamlContent := "signing_key: " + validTestSigningKey + "\ncache_max_bytes: 10737418240\n"
c, err := configFromYAML(t, yamlContent)
if err != nil {
t.Fatalf("large cache_max_bytes must be accepted, got error: %v", err)
}
if c.CacheMaxBytes != 10737418240 {
t.Errorf("CacheMaxBytes = %d, want 10737418240", c.CacheMaxBytes)
}
}
// TestCacheMaxBytesInvalidValuesAbortStartup verifies that a SET but
// invalid cache_max_bytes value aborts startup naming the key and the
// offending value, per the no-silent-fallback rule: defaults apply
// only to omitted keys.
func TestCacheMaxBytesInvalidValuesAbortStartup(t *testing.T) {
signingKeyLine := "signing_key: " + validTestSigningKey + "\n"
cases := []struct {
name string
yaml string
// wantErrSubstrings must all appear in the error message.
wantErrSubstrings []string
}{
{
name: "negative",
yaml: signingKeyLine + "cache_max_bytes: -1024\n",
wantErrSubstrings: []string{"cache_max_bytes", "-1024"},
},
{
name: "float",
yaml: signingKeyLine + "cache_max_bytes: 3.5\n",
wantErrSubstrings: []string{"cache_max_bytes", "3.5"},
},
{
name: "non-numeric string",
yaml: signingKeyLine + "cache_max_bytes: banana\n",
wantErrSubstrings: []string{"cache_max_bytes", "banana"},
},
{
name: "explicit null",
yaml: signingKeyLine + "cache_max_bytes: null\n",
wantErrSubstrings: []string{"cache_max_bytes", "null"},
},
{
name: "bare key no value",
yaml: signingKeyLine + "cache_max_bytes:\n",
wantErrSubstrings: []string{"cache_max_bytes", "null"},
},
{
name: "boolean",
yaml: signingKeyLine + "cache_max_bytes: true\n",
wantErrSubstrings: []string{"cache_max_bytes", "true"},
},
{
name: "list",
yaml: signingKeyLine + "cache_max_bytes:\n - 1\n",
wantErrSubstrings: []string{"cache_max_bytes"},
},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
c, err := configFromYAML(t, tc.yaml)
if err == nil {
t.Fatalf("config with %s cache_max_bytes must abort startup, got config: %+v",
tc.name, c)
}
t.Logf("got expected error: %v", err)
for _, want := range tc.wantErrSubstrings {
if !strings.Contains(err.Error(), want) {
t.Errorf("error %q does not mention %q", err.Error(), want)
}
}
})
}
}
// TestComputeDefaultCacheMaxBytesUses75PercentOfFreeSpace verifies the
// computed default is 75% of the probed free space when that exceeds
// the floor.
func TestComputeDefaultCacheMaxBytesUses75PercentOfFreeSpace(t *testing.T) {
// 4 GiB free -> 3 GiB default.
probe := func(string) (uint64, error) { return 4294967296, nil }
got, err := ComputeDefaultCacheMaxBytes(t.TempDir(), probe)
if err != nil {
t.Fatalf("ComputeDefaultCacheMaxBytes returned error: %v", err)
}
if got != 3221225472 {
t.Errorf("ComputeDefaultCacheMaxBytes = %d, want 3221225472 (75%% of 4 GiB)", got)
}
}
// TestComputeDefaultCacheMaxBytesAppliesFloorToComputedDefault
// verifies that when 75% of free space is below 500 MiB, the computed
// default is floored at DefaultCacheMaxBytesFloor.
func TestComputeDefaultCacheMaxBytesAppliesFloorToComputedDefault(t *testing.T) {
cases := []struct {
name string
freeBytes uint64
}{
{name: "100 MiB free", freeBytes: 104857600},
{name: "zero free", freeBytes: 0},
{name: "just below floor threshold", freeBytes: 699050665},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
probe := func(string) (uint64, error) { return tc.freeBytes, nil }
got, err := ComputeDefaultCacheMaxBytes(t.TempDir(), probe)
if err != nil {
t.Fatalf("ComputeDefaultCacheMaxBytes returned error: %v", err)
}
if got != DefaultCacheMaxBytesFloor {
t.Errorf("ComputeDefaultCacheMaxBytes = %d, want floor %d",
got, DefaultCacheMaxBytesFloor)
}
})
}
}
// TestComputeDefaultCacheMaxBytesPropagatesProbeError verifies that a
// failing free-space probe produces an error naming the config key,
// instead of a silently wrong default.
func TestComputeDefaultCacheMaxBytesPropagatesProbeError(t *testing.T) {
probe := func(string) (uint64, error) { return 0, errors.New("statfs failed") }
_, err := ComputeDefaultCacheMaxBytes(t.TempDir(), probe)
if err == nil {
t.Fatal("probe failure must produce an error, got nil")
}
t.Logf("got expected error: %v", err)
if !strings.Contains(err.Error(), "cache_max_bytes") {
t.Errorf("error %q does not name the config key cache_max_bytes", err.Error())
}
}
// TestResolveCacheMaxBytesComputesDefaultWhenOmitted verifies that an
// omitted cache_max_bytes key resolves to the computed default, that
// the probe is pointed at <state_dir>/cache/ (which must be created
// first so statfs measures the right filesystem), and that the result
// lands on the Config.
func TestResolveCacheMaxBytesComputesDefaultWhenOmitted(t *testing.T) {
c, err := configFromYAML(t, "signing_key: "+validTestSigningKey+"\n")
if err != nil {
t.Fatalf("minimal config should be valid, got error: %v", err)
}
c.StateDir = t.TempDir()
wantCacheDir := filepath.Join(c.StateDir, "cache")
var probedPath string
// 4 GiB free -> 3 GiB default.
probe := func(path string) (uint64, error) {
probedPath = path
return 4294967296, nil
}
if err := c.resolveCacheMaxBytes(discardLogger(), probe); err != nil {
t.Fatalf("resolveCacheMaxBytes returned error: %v", err)
}
if c.CacheMaxBytes != 3221225472 {
t.Errorf("CacheMaxBytes = %d, want computed default 3221225472", c.CacheMaxBytes)
}
if probedPath != wantCacheDir {
t.Errorf("free space probed at %q, want cache directory %q", probedPath, wantCacheDir)
}
info, err := os.Stat(wantCacheDir)
if err != nil || !info.IsDir() {
t.Errorf("cache directory %q was not created before probing: info=%v err=%v",
wantCacheDir, info, err)
}
}
// TestResolveCacheMaxBytesDoesNotOverrideExplicitValue verifies that
// an explicitly configured value survives resolution untouched and
// that the free-space probe is never consulted for it.
func TestResolveCacheMaxBytesDoesNotOverrideExplicitValue(t *testing.T) {
yamlContent := "signing_key: " + validTestSigningKey + "\ncache_max_bytes: 1024\n"
c, err := configFromYAML(t, yamlContent)
if err != nil {
t.Fatalf("explicit cache_max_bytes must be accepted, got error: %v", err)
}
c.StateDir = t.TempDir()
probe := func(string) (uint64, error) {
t.Error("free-space probe must not be consulted for explicit values")
return 0, errors.New("probe must not be called")
}
if err := c.resolveCacheMaxBytes(discardLogger(), probe); err != nil {
t.Fatalf("resolveCacheMaxBytes returned error: %v", err)
}
if c.CacheMaxBytes != 1024 {
t.Errorf("CacheMaxBytes = %d, want explicit 1024 (no floor, no recompute)",
c.CacheMaxBytes)
}
}

View File

@@ -0,0 +1,110 @@
package config
import (
"fmt"
"log/slog"
"math"
"os"
"path/filepath"
"syscall"
)
// DefaultCacheMaxBytesFloor is the minimum computed default for the
// cache_max_bytes setting: 500 MiB. The floor applies only to the
// computed default (when the key is omitted from the configuration),
// never to explicitly configured values.
const DefaultCacheMaxBytesFloor int64 = 524288000
// cacheDirPerms is the permission mode for the cache directory created
// before probing free space, matching the state directory permissions.
const cacheDirPerms = 0o750
// freeSpaceFractionNumerator and freeSpaceFractionDenominator express
// the 75% share of free space used for the computed default limit as
// integer arithmetic (dividing before multiplying avoids overflow).
const (
freeSpaceFractionNumerator uint64 = 3
freeSpaceFractionDenominator uint64 = 4
)
// FreeSpaceProbeFunc reports the number of free bytes available on the
// filesystem containing path. It is a function type so tests can
// inject a fake probe instead of depending on the host disk.
type FreeSpaceProbeFunc func(path string) (uint64, error)
// defaultFreeSpaceProbe reports free filesystem bytes via statfs on
// the given path, as available to unprivileged processes.
func defaultFreeSpaceProbe(path string) (uint64, error) {
var stat syscall.Statfs_t
if err := syscall.Statfs(path, &stat); err != nil {
return 0, err
}
if stat.Bsize < 0 {
return 0, fmt.Errorf("statfs reported negative block size %d for %q", stat.Bsize, path)
}
blockSize := uint64(stat.Bsize) //nolint:gosec // G115: negative Bsize rejected above
return stat.Bavail * blockSize, nil
}
// ComputeDefaultCacheMaxBytes returns the default cache size limit for
// the filesystem containing cacheDir: 75% of the free bytes reported
// by probe, with a floor of DefaultCacheMaxBytesFloor.
func ComputeDefaultCacheMaxBytes(cacheDir string, probe FreeSpaceProbeFunc) (int64, error) {
freeBytes, err := probe(cacheDir)
if err != nil {
return 0, fmt.Errorf("config key %q: cannot determine free space for %q: %w",
"cache_max_bytes", cacheDir, err)
}
computed := freeBytes / freeSpaceFractionDenominator * freeSpaceFractionNumerator
if computed > math.MaxInt64 {
computed = math.MaxInt64
}
limit := int64(computed) //nolint:gosec // G115: clamped to MaxInt64 above
if limit < DefaultCacheMaxBytesFloor {
limit = DefaultCacheMaxBytesFloor
}
return limit, nil
}
// resolveCacheMaxBytes finalizes CacheMaxBytes after state_dir
// validation: an explicitly configured value is kept as-is (no floor
// applies), while an omitted key receives the computed default based
// on free space in <state_dir>/cache/. The cache directory is created
// first so statfs measures the filesystem that will actually hold the
// cache. The effective limit is logged either way.
func (c *Config) resolveCacheMaxBytes(log *slog.Logger, probe FreeSpaceProbeFunc) error {
if !c.cacheMaxBytesExplicit {
cacheDir := filepath.Join(c.StateDir, "cache")
if err := os.MkdirAll(cacheDir, cacheDirPerms); err != nil {
return fmt.Errorf("config key %q: cannot create cache directory %q: %w",
"cache_max_bytes", cacheDir, err)
}
limit, err := ComputeDefaultCacheMaxBytes(cacheDir, probe)
if err != nil {
return err
}
c.CacheMaxBytes = limit
log.Info("computed default cache size limit from free space",
"cache_max_bytes", limit,
"cache_dir", cacheDir,
)
}
log.Info("effective cache size limit",
"cache_max_bytes", c.CacheMaxBytes,
"cache_disabled", c.CacheMaxBytes == 0,
)
return nil
}

View File

@@ -2,7 +2,6 @@
package config
import (
"errors"
"fmt"
"log/slog"
"math"
@@ -26,54 +25,9 @@ const (
DefaultUpstreamConnectionsPerHost = 20
)
// Configuration key names.
const (
keyDebug = "debug"
keyMaintenanceMode = "maintenance_mode"
keyPort = "port"
keyStateDir = "state_dir"
keySentryDSN = "sentry_dsn"
keyDBURL = "db_url"
keyMetrics = "metrics"
keyMetricsUsername = "metrics.username"
keyMetricsPassword = "metrics.password"
keySigningKey = "signing_key"
keyAllowlistHosts = "allowlist_hosts"
keyAllowHTTP = "allow_http"
keyUpstreamConnectionsPerHost = "upstream_connections_per_host"
)
// Static validation errors. Each use site attaches the offending key
// and value by wrapping these with fmt.Errorf and %w.
var (
errValueRequired = errors.New("a value is required")
errValueEmpty = errors.New("value must not be empty")
errUnknownConfigKeys = errors.New("unknown config keys")
errNotAString = errors.New("not a string")
errNotAnInteger = errors.New("not an integer")
errNotABoolean = errors.New("not a boolean")
errNotAStringList = errors.New("not a list of strings")
errNotAMetricsMap = errors.New("not a map of metrics settings")
errEmptyListEntry = errors.New("list contains an empty entry")
errEmptyEntry = errors.New("contains an empty entry")
errNotAValidURL = errors.New("not a valid URL")
errPortOutOfRange = errors.New("outside the valid port range")
errTooFewConnections = errors.New("must be at least 1")
errValueTooShort = errors.New("value too short")
errMustBeSetTogether = errors.New("must be set together")
errValueNull = errors.New(
"value is null; omit the key entirely to use the default")
errValuesNull = errors.New(
"value is null; omit a key entirely to use its default")
errNotBareHostname = errors.New(
"must be a bare hostname without scheme, path, or whitespace")
errNoHostnameLabels = errors.New("contains no hostname labels")
)
// Params defines dependencies for Config.
type Params struct {
fx.In
Globals *globals.Globals
Logger *logger.Logger
}
@@ -94,6 +48,19 @@ type Config struct {
AllowlistHosts []string // Hosts that don't require signatures
AllowHTTP bool // Allow non-TLS upstream (testing only)
UpstreamConnectionsPerHost int // Max concurrent connections per upstream host
// CacheMaxBytes is the disk cache size limit in bytes. Zero
// disables the disk cache entirely. When cache_max_bytes is
// omitted from the configuration, this holds the computed default
// (75% of free space on the filesystem containing
// <state_dir>/cache/, floored at DefaultCacheMaxBytesFloor).
CacheMaxBytes int64
// cacheMaxBytesExplicit records whether cache_max_bytes was
// explicitly set in the configuration file. Explicit values are
// used exactly as given; only an omitted key gets the computed
// default (and its floor) in resolveCacheMaxBytes.
cacheMaxBytesExplicit bool
}
// New creates a new Config instance by loading configuration from file.
@@ -115,8 +82,11 @@ func New(_ fx.Lifecycle, params Params) (*Config, error) {
return nil, err
}
err = c.ensureStateDirWritable()
if err != nil {
if err := c.ensureStateDirWritable(); err != nil {
return nil, err
}
if err := c.resolveCacheMaxBytes(log, defaultFreeSpaceProbe); err != nil {
return nil, err
}
@@ -134,13 +104,11 @@ func New(_ fx.Lifecycle, params Params) (*Config, error) {
// to omitted keys, never to invalid explicit values.
func newFromSmartConfig(sc *smartconfig.Config) (*Config, error) {
if sc != nil {
err := validateKnownKeys(sc)
if err != nil {
if err := validateKnownKeys(sc); err != nil {
return nil, err
}
err = validateAllowlistHostsValue(sc)
if err != nil {
if err := validateAllowlistHostsValue(sc); err != nil {
return nil, err
}
}
@@ -148,30 +116,40 @@ func newFromSmartConfig(sc *smartconfig.Config) (*Config, error) {
loader := &strictLoader{sc: sc}
c := &Config{
Debug: loader.boolVal(keyDebug, false),
MaintenanceMode: loader.boolVal(keyMaintenanceMode, false),
Port: loader.intVal(keyPort, DefaultPort),
StateDir: loader.stringVal(keyStateDir, DefaultStateDir),
SentryDSN: loader.stringVal(keySentryDSN, ""),
MetricsUsername: loader.stringVal(keyMetricsUsername, ""),
MetricsPassword: loader.stringVal(keyMetricsPassword, ""),
SigningKey: loader.stringVal(keySigningKey, ""),
AllowlistHosts: getStringSlice(sc),
AllowHTTP: loader.boolVal(keyAllowHTTP, false),
Debug: loader.boolVal("debug", false),
MaintenanceMode: loader.boolVal("maintenance_mode", false),
Port: loader.intVal("port", DefaultPort),
StateDir: loader.stringVal("state_dir", DefaultStateDir),
SentryDSN: loader.stringVal("sentry_dsn", ""),
MetricsUsername: loader.stringVal("metrics.username", ""),
MetricsPassword: loader.stringVal("metrics.password", ""),
SigningKey: loader.stringVal("signing_key", ""),
AllowlistHosts: getStringSlice(sc, "allowlist_hosts"),
AllowHTTP: loader.boolVal("allow_http", false),
UpstreamConnectionsPerHost: loader.intVal(
keyUpstreamConnectionsPerHost, DefaultUpstreamConnectionsPerHost),
"upstream_connections_per_host", DefaultUpstreamConnectionsPerHost),
CacheMaxBytes: loader.int64Val("cache_max_bytes", 0),
}
// The computed default for cache_max_bytes needs a validated
// state_dir, so it is resolved later (resolveCacheMaxBytes); here
// we only record whether the operator set the key explicitly.
if sc != nil {
if _, present := sc.Get("cache_max_bytes"); present {
c.cacheMaxBytesExplicit = true
}
}
// Build DBURL from StateDir if not explicitly set. The derived URL
// is a default: it applies only when db_url is omitted, never to an
// explicitly empty value.
c.DBURL = loader.stringVal(keyDBURL, "")
c.DBURL = loader.stringVal("db_url", "")
if c.DBURL == "" && loader.err == nil {
if sc != nil {
if _, present := sc.Get(keyDBURL); present {
if _, present := sc.Get("db_url"); present {
return nil, fmt.Errorf(
"config key %q: %w; omit the key to derive it from state_dir",
keyDBURL, errValueEmpty)
"config key %q: value must not be empty; omit the key to derive it from state_dir",
"db_url")
}
}
@@ -182,8 +160,7 @@ func newFromSmartConfig(sc *smartconfig.Config) (*Config, error) {
return nil, loader.err
}
err := c.validate()
if err != nil {
if err := c.validate(); err != nil {
return nil, err
}
@@ -212,22 +189,23 @@ func validateKnownKeys(sc *smartconfig.Config) error {
continue
}
if key == keyMetrics {
metricsMap, ok := value.(map[string]any)
if key == "metrics" {
metricsMap, ok := value.(map[string]interface{})
if !ok {
return fmt.Errorf("config key %q: value %v is %w",
keyMetrics, value, errNotAMetricsMap)
return fmt.Errorf(
"config key %q: value %v is not a map of metrics settings",
"metrics", value)
}
for subkey, subvalue := range metricsMap {
if subkey != "username" && subkey != "password" {
unknown = append(unknown, keyMetrics+"."+subkey)
unknown = append(unknown, "metrics."+subkey)
continue
}
if subvalue == nil {
nullKeys = append(nullKeys, keyMetrics+"."+subkey)
nullKeys = append(nullKeys, "metrics."+subkey)
}
}
}
@@ -236,7 +214,7 @@ func validateKnownKeys(sc *smartconfig.Config) error {
if len(unknown) > 0 {
sort.Strings(unknown)
return fmt.Errorf("%w: %s", errUnknownConfigKeys, strings.Join(unknown, ", "))
return fmt.Errorf("unknown config keys: %s", strings.Join(unknown, ", "))
}
if len(nullKeys) > 0 {
@@ -246,8 +224,9 @@ func validateKnownKeys(sc *smartconfig.Config) error {
return errNullConfigValue(nullKeys[0])
}
return fmt.Errorf("config keys %s: %w",
strings.Join(nullKeys, ", "), errValuesNull)
return fmt.Errorf(
"config keys %s: value is null; omit a key entirely to use its default",
strings.Join(nullKeys, ", "))
}
return nil
@@ -257,16 +236,17 @@ func validateKnownKeys(sc *smartconfig.Config) error {
// null (including the bare "key:" form and the "~" alias). Silently
// applying the default would mask a truncated or typo'd config entry.
func errNullConfigValue(key string) error {
return fmt.Errorf("config key %q: %w", key, errValueNull)
return fmt.Errorf(
"config key %q: value is null; omit the key entirely to use the default", key)
}
// isKnownConfigKey reports whether key is a permitted top-level
// configuration key.
func isKnownConfigKey(key string) bool {
switch key {
case keyDebug, keyMaintenanceMode, keyPort, keyStateDir, keySentryDSN,
keyDBURL, keyMetrics, keySigningKey, keyAllowlistHosts, keyAllowHTTP,
keyUpstreamConnectionsPerHost, "env":
case "debug", "maintenance_mode", "port", "state_dir", "sentry_dsn",
"db_url", "metrics", "signing_key", "allowlist_hosts", "allow_http",
"upstream_connections_per_host", "cache_max_bytes", "env":
return true
}
@@ -279,30 +259,28 @@ func isKnownConfigKey(key string) bool {
func (c *Config) ensureStateDirWritable() error {
const stateDirPerms = 0o750
err := os.MkdirAll(c.StateDir, stateDirPerms)
if err != nil {
if err := os.MkdirAll(c.StateDir, stateDirPerms); err != nil {
return fmt.Errorf("config key %q: cannot create directory %q: %w",
keyStateDir, c.StateDir, err)
"state_dir", c.StateDir, err)
}
probe, err := os.CreateTemp(c.StateDir, ".startup-write-probe-*")
if err != nil {
return fmt.Errorf("config key %q: directory %q is not writable: %w",
keyStateDir, c.StateDir, err)
"state_dir", c.StateDir, err)
}
probePath := probe.Name()
err = probe.Close()
if err != nil {
if err := probe.Close(); err != nil {
return fmt.Errorf("config key %q: cannot close probe file %q: %w",
keyStateDir, probePath, err)
"state_dir", probePath, err)
}
err = os.Remove(probePath)
if err != nil {
//nolint:gosec // G703: probePath comes from os.CreateTemp inside the just-validated StateDir
if err := os.Remove(probePath); err != nil {
return fmt.Errorf("config key %q: cannot remove probe file %q: %w",
keyStateDir, probePath, err)
"state_dir", probePath, err)
}
return nil
@@ -313,35 +291,40 @@ func (c *Config) ensureStateDirWritable() error {
func (c *Config) validate() error {
// The signing key value is never echoed in error messages.
if c.SigningKey == "" {
return fmt.Errorf("config key %q: %w", keySigningKey, errValueRequired)
return fmt.Errorf("config key %q: a value is required", "signing_key")
}
// Minimum key length for security (32 bytes = 256 bits)
const minKeyLength = 32
if len(c.SigningKey) < minKeyLength {
return fmt.Errorf("config key %q: %w: must be at least %d characters, got %d",
keySigningKey, errValueTooShort, minKeyLength, len(c.SigningKey))
return fmt.Errorf("config key %q: value must be at least %d characters, got %d",
"signing_key", minKeyLength, len(c.SigningKey))
}
const maxPort = 65535
if c.Port < 1 || c.Port > maxPort {
return fmt.Errorf("config key %q: value %d is %w 1-%d",
keyPort, c.Port, errPortOutOfRange, maxPort)
return fmt.Errorf("config key %q: value %d is outside the valid port range 1-%d",
"port", c.Port, maxPort)
}
if c.UpstreamConnectionsPerHost < 1 {
return fmt.Errorf("config key %q: value %d %w",
keyUpstreamConnectionsPerHost, c.UpstreamConnectionsPerHost,
errTooFewConnections)
return fmt.Errorf("config key %q: value %d must be at least 1",
"upstream_connections_per_host", c.UpstreamConnectionsPerHost)
}
if c.StateDir == "" {
return fmt.Errorf("config key %q: %w", keyStateDir, errValueEmpty)
return fmt.Errorf("config key %q: value must not be empty", "state_dir")
}
// Zero is valid (it disables the disk cache); only negative
// values are rejected. No floor applies to explicit values.
if c.CacheMaxBytes < 0 {
return fmt.Errorf("config key %q: value %d must not be negative",
"cache_max_bytes", c.CacheMaxBytes)
}
for _, host := range c.AllowlistHosts {
err := validateAllowlistHost(host)
if err != nil {
if err := validateAllowlistHost(host); err != nil {
return err
}
}
@@ -349,14 +332,14 @@ func (c *Config) validate() error {
if c.SentryDSN != "" {
parsed, err := url.Parse(c.SentryDSN)
if err != nil || parsed.Scheme == "" || parsed.Host == "" {
return fmt.Errorf("config key %q: value %q is %w",
keySentryDSN, c.SentryDSN, errNotAValidURL)
return fmt.Errorf("config key %q: value %q is not a valid URL",
"sentry_dsn", c.SentryDSN)
}
}
if (c.MetricsUsername == "") != (c.MetricsPassword == "") {
return fmt.Errorf("config keys %q and %q %w",
keyMetricsUsername, keyMetricsPassword, errMustBeSetTogether)
return fmt.Errorf("config keys %q and %q must be set together",
"metrics.username", "metrics.password")
}
return nil
@@ -371,20 +354,21 @@ func (c *Config) validate() error {
// disable URL signing.
func validateAllowlistHost(host string) error {
if strings.Contains(host, "://") || strings.ContainsAny(host, "/ \t") {
return fmt.Errorf("config key %q: entry %q %w",
keyAllowlistHosts, host, errNotBareHostname)
return fmt.Errorf(
"config key %q: entry %q must be a bare hostname without scheme, path, or whitespace",
"allowlist_hosts", host)
}
if strings.Trim(host, ".") == "" {
return fmt.Errorf("config key %q: entry %q %w",
keyAllowlistHosts, host, errNoHostnameLabels)
return fmt.Errorf(
"config key %q: entry %q contains no hostname labels",
"allowlist_hosts", host)
}
return nil
}
// loadConfigFile loads configuration from the PIXA_CONFIG_PATH env var
// or standard locations.
// loadConfigFile loads configuration from PIXA_CONFIG_PATH env var or standard locations.
func loadConfigFile(log *slog.Logger, appName string) (*smartconfig.Config, error) {
// Check for explicit config path from environment
if envPath := os.Getenv("PIXA_CONFIG_PATH"); envPath != "" {
@@ -410,9 +394,8 @@ func loadConfigFile(log *slog.Logger, appName string) (*smartconfig.Config, erro
for _, path := range configPaths {
cleanPath := filepath.Clean(path)
_, statErr := os.Stat(cleanPath)
if statErr == nil {
//nolint:gosec // G703: paths are hardcoded config locations
if _, statErr := os.Stat(cleanPath); statErr == nil {
// A config file that exists but does not parse is a fatal
// startup error, never something to skip over.
sc, err := smartconfig.NewFromConfigPath(path)
@@ -463,6 +446,19 @@ func (l *strictLoader) intVal(key string, defaultVal int) int {
return val
}
func (l *strictLoader) int64Val(key string, defaultVal int64) int64 {
if l.err != nil {
return 0
}
val, err := getInt64(l.sc, key, defaultVal)
if err != nil {
l.err = err
}
return val
}
func (l *strictLoader) boolVal(key string, defaultVal bool) bool {
if l.err != nil {
return false
@@ -495,8 +491,8 @@ func getString(sc *smartconfig.Config, key, defaultVal string) (string, error) {
str, ok := raw.(string)
if !ok {
return "", fmt.Errorf("config key %q: value %v (%T) is %w",
key, raw, raw, errNotAString)
return "", fmt.Errorf("config key %q: value %v (%T) is not a string",
key, raw, raw)
}
return str, nil
@@ -526,22 +522,69 @@ func getInt(sc *smartconfig.Config, key string, defaultVal int) (int, error) {
return int(val), nil
case float64:
if val != math.Trunc(val) {
return 0, fmt.Errorf("config key %q: value %v is %w",
key, val, errNotAnInteger)
return 0, fmt.Errorf("config key %q: value %v is not an integer", key, val)
}
return int(val), nil
case string:
parsed, err := strconv.Atoi(strings.TrimSpace(val))
if err != nil {
return 0, fmt.Errorf("config key %q: value %q is %w",
key, val, errNotAnInteger)
return 0, fmt.Errorf("config key %q: value %q is not an integer", key, val)
}
return parsed, nil
default:
return 0, fmt.Errorf("config key %q: value %v (%T) is %w",
key, raw, raw, errNotAnInteger)
return 0, fmt.Errorf("config key %q: value %v (%T) is not an integer",
key, raw, raw)
}
}
// getInt64 returns the 64-bit integer value for key, or defaultVal if
// the key is omitted. A present value that is not a whole number, or
// is explicitly null, is an error; fractional values are never
// truncated and out-of-range values are never clamped.
func getInt64(sc *smartconfig.Config, key string, defaultVal int64) (int64, error) {
if sc == nil {
return defaultVal, nil
}
raw, ok := sc.Get(key)
if !ok {
return defaultVal, nil
}
if raw == nil {
return 0, errNullConfigValue(key)
}
switch val := raw.(type) {
case int:
return int64(val), nil
case int64:
return val, nil
case uint64:
if val > math.MaxInt64 {
return 0, fmt.Errorf("config key %q: value %d overflows a 64-bit integer",
key, val)
}
return int64(val), nil //nolint:gosec // G115: bounds checked above
case float64:
if val != math.Trunc(val) {
return 0, fmt.Errorf("config key %q: value %v is not an integer", key, val)
}
return int64(val), nil
case string:
parsed, err := strconv.ParseInt(strings.TrimSpace(val), 10, 64)
if err != nil {
return 0, fmt.Errorf("config key %q: value %q is not an integer", key, val)
}
return parsed, nil
default:
return 0, fmt.Errorf("config key %q: value %v (%T) is not an integer",
key, raw, raw)
}
}
@@ -569,14 +612,13 @@ func getBool(sc *smartconfig.Config, key string, defaultVal bool) (bool, error)
case string:
parsed, err := strconv.ParseBool(strings.TrimSpace(val))
if err != nil {
return false, fmt.Errorf("config key %q: value %q is %w",
key, val, errNotABoolean)
return false, fmt.Errorf("config key %q: value %q is not a boolean", key, val)
}
return parsed, nil
default:
return false, fmt.Errorf("config key %q: value %v (%T) is %w",
key, raw, raw, errNotABoolean)
return false, fmt.Errorf("config key %q: value %v (%T) is not a boolean",
key, raw, raw)
}
}
@@ -586,27 +628,28 @@ func getBool(sc *smartconfig.Config, key string, defaultVal bool) (bool, error)
// (or a comma-separated string), a non-string entry, or an empty entry
// is an error, never silently skipped.
func validateAllowlistHostsValue(sc *smartconfig.Config) error {
raw, ok := sc.Get(keyAllowlistHosts)
const key = "allowlist_hosts"
raw, ok := sc.Get(key)
if !ok {
return nil
}
if raw == nil {
return errNullConfigValue(keyAllowlistHosts)
return errNullConfigValue(key)
}
switch val := raw.(type) {
case []any:
case []interface{}:
for _, item := range val {
str, ok := item.(string)
if !ok {
return fmt.Errorf("config key %q: list entry %v (%T) is %w",
keyAllowlistHosts, item, item, errNotAString)
return fmt.Errorf(
"config key %q: list entry %v (%T) is not a string", key, item, item)
}
if strings.TrimSpace(str) == "" {
return fmt.Errorf("config key %q: %w",
keyAllowlistHosts, errEmptyListEntry)
return fmt.Errorf("config key %q: list contains an empty entry", key)
}
}
case string:
@@ -614,36 +657,36 @@ func validateAllowlistHostsValue(sc *smartconfig.Config) error {
return nil
}
for part := range strings.SplitSeq(val, ",") {
for _, part := range strings.Split(val, ",") {
if strings.TrimSpace(part) == "" {
return fmt.Errorf("config key %q: value %q %w",
keyAllowlistHosts, val, errEmptyEntry)
return fmt.Errorf(
"config key %q: value %q contains an empty entry", key, val)
}
}
default:
return fmt.Errorf("config key %q: value %v (%T) is %w",
keyAllowlistHosts, raw, raw, errNotAStringList)
return fmt.Errorf("config key %q: value %v (%T) is not a list of strings",
key, raw, raw)
}
return nil
}
// getStringSlice returns the allowlist_hosts list of strings, or nil if
// the key is omitted. It accepts a YAML list of strings or a
// comma-separated string (backwards compatibility). Malformed entries
// are rejected beforehand by validateAllowlistHostsValue.
func getStringSlice(sc *smartconfig.Config) []string {
// getStringSlice returns the list of strings for key, or nil if the key
// is omitted. It accepts a YAML list of strings or a comma-separated
// string (backwards compatibility). Malformed entries are rejected
// beforehand by validateAllowlistHostsValue.
func getStringSlice(sc *smartconfig.Config, key string) []string {
if sc == nil {
return nil
}
val, ok := sc.Get(keyAllowlistHosts)
val, ok := sc.Get(key)
if !ok || val == nil {
return nil
}
// Handle YAML list format
if slice, ok := val.([]any); ok {
if slice, ok := val.([]interface{}); ok {
result := make([]string, 0, len(slice))
for _, item := range slice {
if str, ok := item.(string); ok {

View File

@@ -1,98 +0,0 @@
package config
import (
"os"
"path/filepath"
"testing"
"git.eeqj.de/sneak/smartconfig"
)
// writeTestConfig writes yamlContent to a temp config file and returns
// the file path.
func writeTestConfig(t *testing.T, yamlContent string) string {
t.Helper()
tmpDir := t.TempDir()
configPath := filepath.Join(tmpDir, "config.yml")
err := os.WriteFile(configPath, []byte(yamlContent), 0o600)
if err != nil {
t.Fatalf("failed to write test config: %v", err)
}
return configPath
}
// checkAllowlistHosts loads the config at configPath and asserts that
// getStringSlice returns the three expected hosts.
func checkAllowlistHosts(t *testing.T, configPath string) {
t.Helper()
sc, err := loadTestConfig(configPath)
if err != nil {
t.Fatalf("failed to load config: %v", err)
}
hosts := getStringSlice(sc)
if len(hosts) != 3 {
t.Errorf("expected 3 hosts, got %d: %v", len(hosts), hosts)
}
expected := []string{"static.sneak.cloud", "sneak.berlin", testHostS3}
for i, want := range expected {
if i >= len(hosts) {
t.Errorf("missing host at index %d: want %q", i, want)
continue
}
if hosts[i] != want {
t.Errorf("host[%d] = %q, want %q", i, hosts[i], want)
}
}
}
func TestGetStringSlice_YAMLList(t *testing.T) {
t.Parallel()
yamlContent := `
allowlist_hosts:
- static.sneak.cloud
- sneak.berlin
- s3.sneak.cloud
`
checkAllowlistHosts(t, writeTestConfig(t, yamlContent))
}
func TestGetStringSlice_CommaSeparated(t *testing.T) {
t.Parallel()
// Backwards compatibility with comma-separated string values.
yamlContent := `allowlist_hosts: "static.sneak.cloud, sneak.berlin, s3.sneak.cloud"`
checkAllowlistHosts(t, writeTestConfig(t, yamlContent))
}
func TestGetStringSlice_Empty(t *testing.T) {
t.Parallel()
configPath := writeTestConfig(t, `port: 8080`)
sc, err := loadTestConfig(configPath)
if err != nil {
t.Fatalf("failed to load config: %v", err)
}
hosts := getStringSlice(sc)
if len(hosts) != 0 {
t.Errorf("expected nil or empty slice, got %v", hosts)
}
}
// loadTestConfig is a helper to load a config file for testing.
func loadTestConfig(path string) (*smartconfig.Config, error) {
return smartconfig.NewFromConfigPath(path)
}

View File

@@ -0,0 +1,113 @@
package config
import (
"os"
"path/filepath"
"testing"
"git.eeqj.de/sneak/smartconfig"
)
func TestGetStringSlice_YAMLList(t *testing.T) {
// Create a temp config file with YAML list format
tmpDir := t.TempDir()
configPath := filepath.Join(tmpDir, "config.yml")
yamlContent := `
allowlist_hosts:
- static.sneak.cloud
- sneak.berlin
- s3.sneak.cloud
`
err := os.WriteFile(configPath, []byte(yamlContent), 0644)
if err != nil {
t.Fatalf("failed to write test config: %v", err)
}
// Load config using smartconfig
sc, err := loadTestConfig(configPath)
if err != nil {
t.Fatalf("failed to load config: %v", err)
}
// Test that getStringSlice correctly parses YAML list
hosts := getStringSlice(sc, "allowlist_hosts")
if len(hosts) != 3 {
t.Errorf("expected 3 hosts, got %d: %v", len(hosts), hosts)
}
expected := []string{"static.sneak.cloud", "sneak.berlin", "s3.sneak.cloud"}
for i, want := range expected {
if i >= len(hosts) {
t.Errorf("missing host at index %d: want %q", i, want)
continue
}
if hosts[i] != want {
t.Errorf("host[%d] = %q, want %q", i, hosts[i], want)
}
}
}
func TestGetStringSlice_CommaSeparated(t *testing.T) {
// Test backwards compatibility with comma-separated string
tmpDir := t.TempDir()
configPath := filepath.Join(tmpDir, "config.yml")
yamlContent := `allowlist_hosts: "static.sneak.cloud, sneak.berlin, s3.sneak.cloud"`
err := os.WriteFile(configPath, []byte(yamlContent), 0644)
if err != nil {
t.Fatalf("failed to write test config: %v", err)
}
sc, err := loadTestConfig(configPath)
if err != nil {
t.Fatalf("failed to load config: %v", err)
}
hosts := getStringSlice(sc, "allowlist_hosts")
if len(hosts) != 3 {
t.Errorf("expected 3 hosts, got %d: %v", len(hosts), hosts)
}
expected := []string{"static.sneak.cloud", "sneak.berlin", "s3.sneak.cloud"}
for i, want := range expected {
if i >= len(hosts) {
t.Errorf("missing host at index %d: want %q", i, want)
continue
}
if hosts[i] != want {
t.Errorf("host[%d] = %q, want %q", i, hosts[i], want)
}
}
}
func TestGetStringSlice_Empty(t *testing.T) {
tmpDir := t.TempDir()
configPath := filepath.Join(tmpDir, "config.yml")
yamlContent := `port: 8080`
err := os.WriteFile(configPath, []byte(yamlContent), 0644)
if err != nil {
t.Fatalf("failed to write test config: %v", err)
}
sc, err := loadTestConfig(configPath)
if err != nil {
t.Fatalf("failed to load config: %v", err)
}
hosts := getStringSlice(sc, "allowlist_hosts")
if hosts != nil && len(hosts) != 0 {
t.Errorf("expected nil or empty slice, got %v", hosts)
}
}
// loadTestConfig is a helper to load a config file for testing
func loadTestConfig(path string) (*smartconfig.Config, error) {
return smartconfig.NewFromConfigPath(path)
}

View File

@@ -1,6 +1,7 @@
package config
import (
"io"
"log/slog"
"os"
"path/filepath"
@@ -14,26 +15,6 @@ import (
// minimum length requirement in validate().
const validTestSigningKey = "0123456789abcdef0123456789abcdef"
// signingKeyLine is a valid signing_key config line used as the base of
// test config files.
const signingKeyLine = "signing_key: " + validTestSigningKey + "\n"
// testHostS3 is an allowlist host entry used across the config tests.
const testHostS3 = "s3.sneak.cloud"
// nullValueText is the substring that error messages about explicitly
// null config values must contain.
const nullValueText = "null"
// abortCase describes a config file that must abort startup with an
// error mentioning every string in wantErrSubstrings.
type abortCase struct {
name string
yaml string
// wantErrSubstrings must all appear in the error message.
wantErrSubstrings []string
}
// configFromYAML writes yamlContent to a temporary config file, loads it
// via smartconfig, and constructs a Config from it using the same code
// path the server uses at startup.
@@ -43,8 +24,7 @@ func configFromYAML(t *testing.T, yamlContent string) (*Config, error) {
tmpDir := t.TempDir()
configPath := filepath.Join(tmpDir, "config.yml")
err := os.WriteFile(configPath, []byte(yamlContent), 0o600)
if err != nil {
if err := os.WriteFile(configPath, []byte(yamlContent), 0o600); err != nil {
t.Fatalf("failed to write test config: %v", err)
}
@@ -57,9 +37,7 @@ func configFromYAML(t *testing.T, yamlContent string) (*Config, error) {
}
func TestOmittedValuesUseDefaults(t *testing.T) {
t.Parallel()
c, err := configFromYAML(t, signingKeyLine)
c, err := configFromYAML(t, "signing_key: "+validTestSigningKey+"\n")
if err != nil {
t.Fatalf("minimal config should be valid, got error: %v", err)
}
@@ -100,8 +78,6 @@ func TestOmittedValuesUseDefaults(t *testing.T) {
}
func TestExplicitValidValuesAreUsed(t *testing.T) {
t.Parallel()
yamlContent := `
port: 9090
debug: true
@@ -142,10 +118,9 @@ metrics:
t.Errorf("DBURL = %q, want explicit value", c.DBURL)
}
if len(c.AllowlistHosts) != 2 || c.AllowlistHosts[0] != testHostS3 ||
if len(c.AllowlistHosts) != 2 || c.AllowlistHosts[0] != "s3.sneak.cloud" ||
c.AllowlistHosts[1] != ".example.com" {
t.Errorf("AllowlistHosts = %v, want [s3.sneak.cloud .example.com]",
c.AllowlistHosts)
t.Errorf("AllowlistHosts = %v, want [s3.sneak.cloud .example.com]", c.AllowlistHosts)
}
if c.UpstreamConnectionsPerHost != 5 {
@@ -163,10 +138,8 @@ metrics:
}
func TestCommaSeparatedAllowlistStillSupported(t *testing.T) {
t.Parallel()
yamlContent := signingKeyLine +
`allowlist_hosts: "s3.sneak.cloud, sneak.berlin"
yamlContent := `signing_key: ` + validTestSigningKey + `
allowlist_hosts: "s3.sneak.cloud, sneak.berlin"
`
c, err := configFromYAML(t, yamlContent)
@@ -174,22 +147,138 @@ func TestCommaSeparatedAllowlistStillSupported(t *testing.T) {
t.Fatalf("comma-separated allowlist should load, got error: %v", err)
}
if len(c.AllowlistHosts) != 2 || c.AllowlistHosts[0] != testHostS3 ||
if len(c.AllowlistHosts) != 2 || c.AllowlistHosts[0] != "s3.sneak.cloud" ||
c.AllowlistHosts[1] != "sneak.berlin" {
t.Errorf("AllowlistHosts = %v, want [s3.sneak.cloud sneak.berlin]",
c.AllowlistHosts)
t.Errorf("AllowlistHosts = %v, want [s3.sneak.cloud sneak.berlin]", c.AllowlistHosts)
}
}
// runAbortCases asserts that each case's config aborts startup with an
// error message mentioning every expected substring.
func runAbortCases(t *testing.T, cases []abortCase) {
t.Helper()
// TestSetButInvalidValueAbortsStartup verifies the no-silent-fallback
// rule: a key that is explicitly set to an unparseable or out-of-range
// value must produce a startup error naming the offending key, never
// silently fall back to the default.
func TestSetButInvalidValueAbortsStartup(t *testing.T) {
signingKeyLine := "signing_key: " + validTestSigningKey + "\n"
cases := []struct {
name string
yaml string
// wantErrSubstrings must all appear in the error message.
wantErrSubstrings []string
}{
{
name: "port not a number",
yaml: signingKeyLine + "port: banana\n",
wantErrSubstrings: []string{"port", "banana"},
},
{
name: "port zero",
yaml: signingKeyLine + "port: 0\n",
wantErrSubstrings: []string{"port", "0"},
},
{
name: "port above 65535",
yaml: signingKeyLine + "port: 99999\n",
wantErrSubstrings: []string{"port", "99999"},
},
{
name: "port fractional",
yaml: signingKeyLine + "port: 8080.5\n",
wantErrSubstrings: []string{"port", "8080.5"},
},
{
name: "debug not a bool",
yaml: signingKeyLine + "debug: notabool\n",
wantErrSubstrings: []string{"debug", "notabool"},
},
{
name: "maintenance_mode not a bool",
yaml: signingKeyLine + "maintenance_mode: sometimes\n",
wantErrSubstrings: []string{"maintenance_mode", "sometimes"},
},
{
name: "allow_http numeric",
yaml: signingKeyLine + "allow_http: 2\n",
wantErrSubstrings: []string{"allow_http", "2"},
},
{
name: "upstream_connections_per_host zero",
yaml: signingKeyLine + "upstream_connections_per_host: 0\n",
wantErrSubstrings: []string{"upstream_connections_per_host", "0"},
},
{
name: "upstream_connections_per_host negative",
yaml: signingKeyLine + "upstream_connections_per_host: -3\n",
wantErrSubstrings: []string{"upstream_connections_per_host", "-3"},
},
{
name: "upstream_connections_per_host not a number",
yaml: signingKeyLine + "upstream_connections_per_host: many\n",
wantErrSubstrings: []string{"upstream_connections_per_host", "many"},
},
{
name: "allowlist host with scheme",
yaml: signingKeyLine + "allowlist_hosts:\n - https://example.com\n",
wantErrSubstrings: []string{
"allowlist_hosts", "https://example.com",
},
},
{
name: "allowlist host with path",
yaml: signingKeyLine + "allowlist_hosts:\n - example.com/images\n",
wantErrSubstrings: []string{
"allowlist_hosts", "example.com/images",
},
},
{
name: "allowlist host with whitespace",
yaml: signingKeyLine + "allowlist_hosts:\n - \"exa mple.com\"\n",
wantErrSubstrings: []string{"allowlist_hosts", "exa mple.com"},
},
{
name: "allowlist entry not a string",
yaml: signingKeyLine + "allowlist_hosts:\n - 123\n",
wantErrSubstrings: []string{"allowlist_hosts", "123"},
},
{
name: "allowlist not a list",
yaml: signingKeyLine + "allowlist_hosts:\n key: value\n",
wantErrSubstrings: []string{"allowlist_hosts"},
},
{
name: "signing_key too short",
yaml: "signing_key: short\n",
wantErrSubstrings: []string{"signing_key"},
},
{
name: "signing_key missing",
yaml: "port: 8080\n",
wantErrSubstrings: []string{"signing_key"},
},
{
name: "state_dir explicitly empty",
yaml: signingKeyLine + "state_dir: \"\"\n",
wantErrSubstrings: []string{"state_dir"},
},
{
name: "sentry_dsn not a URL",
yaml: signingKeyLine + "sentry_dsn: \"not a url\"\n",
wantErrSubstrings: []string{"sentry_dsn", "not a url"},
},
{
name: "metrics username without password",
yaml: signingKeyLine + "metrics:\n username: bob\n",
wantErrSubstrings: []string{"metrics"},
},
{
name: "metrics password without username",
yaml: signingKeyLine + "metrics:\n password: hunter2\n",
wantErrSubstrings: []string{"metrics"},
},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
c, err := configFromYAML(t, tc.yaml)
if err == nil {
t.Fatalf("config with %s must abort startup, got config: %+v", tc.name, c)
@@ -206,225 +295,104 @@ func runAbortCases(t *testing.T, cases []abortCase) {
}
}
// invalidScalarValueCases are configs where a scalar key is explicitly
// set to an unparseable or out-of-range value; each must abort startup
// naming the offending key, never silently fall back to the default.
func invalidScalarValueCases() []abortCase {
return []abortCase{
{
name: "port not a number",
yaml: signingKeyLine + "port: banana\n",
wantErrSubstrings: []string{keyPort, "banana"},
},
{
name: "port zero",
yaml: signingKeyLine + "port: 0\n",
wantErrSubstrings: []string{keyPort, "0"},
},
{
name: "port above 65535",
yaml: signingKeyLine + "port: 99999\n",
wantErrSubstrings: []string{keyPort, "99999"},
},
{
name: "port fractional",
yaml: signingKeyLine + "port: 8080.5\n",
wantErrSubstrings: []string{keyPort, "8080.5"},
},
{
name: "debug not a bool",
yaml: signingKeyLine + "debug: notabool\n",
wantErrSubstrings: []string{keyDebug, "notabool"},
},
{
name: "maintenance_mode not a bool",
yaml: signingKeyLine + "maintenance_mode: sometimes\n",
wantErrSubstrings: []string{keyMaintenanceMode, "sometimes"},
},
{
name: "allow_http numeric",
yaml: signingKeyLine + "allow_http: 2\n",
wantErrSubstrings: []string{keyAllowHTTP, "2"},
},
{
name: "upstream_connections_per_host zero",
yaml: signingKeyLine + "upstream_connections_per_host: 0\n",
wantErrSubstrings: []string{keyUpstreamConnectionsPerHost, "0"},
},
{
name: "upstream_connections_per_host negative",
yaml: signingKeyLine + "upstream_connections_per_host: -3\n",
wantErrSubstrings: []string{keyUpstreamConnectionsPerHost, "-3"},
},
{
name: "upstream_connections_per_host not a number",
yaml: signingKeyLine + "upstream_connections_per_host: many\n",
wantErrSubstrings: []string{keyUpstreamConnectionsPerHost, "many"},
},
}
}
// invalidHostAndCredentialCases are configs where allowlist_hosts,
// signing_key, state_dir, sentry_dsn, or metrics is explicitly set to
// an invalid value; each must abort startup naming the offending key.
func invalidHostAndCredentialCases() []abortCase {
return []abortCase{
{
name: "allowlist host with scheme",
yaml: signingKeyLine + "allowlist_hosts:\n - https://example.com\n",
wantErrSubstrings: []string{
keyAllowlistHosts, "https://example.com",
},
},
{
name: "allowlist host with path",
yaml: signingKeyLine + "allowlist_hosts:\n - example.com/images\n",
wantErrSubstrings: []string{
keyAllowlistHosts, "example.com/images",
},
},
{
name: "allowlist host with whitespace",
yaml: signingKeyLine + "allowlist_hosts:\n - \"exa mple.com\"\n",
wantErrSubstrings: []string{keyAllowlistHosts, "exa mple.com"},
},
{
name: "allowlist entry not a string",
yaml: signingKeyLine + "allowlist_hosts:\n - 123\n",
wantErrSubstrings: []string{keyAllowlistHosts, "123"},
},
{
name: "allowlist not a list",
yaml: signingKeyLine + "allowlist_hosts:\n key: value\n",
wantErrSubstrings: []string{keyAllowlistHosts},
},
{
name: "signing_key too short",
yaml: "signing_key: short\n",
wantErrSubstrings: []string{keySigningKey},
},
{
name: "signing_key missing",
yaml: "port: 8080\n",
wantErrSubstrings: []string{keySigningKey},
},
{
name: "state_dir explicitly empty",
yaml: signingKeyLine + "state_dir: \"\"\n",
wantErrSubstrings: []string{keyStateDir},
},
{
name: "sentry_dsn not a URL",
yaml: signingKeyLine + "sentry_dsn: \"not a url\"\n",
wantErrSubstrings: []string{keySentryDSN, "not a url"},
},
{
name: "metrics username without password",
yaml: signingKeyLine + "metrics:\n username: bob\n",
wantErrSubstrings: []string{keyMetrics},
},
{
name: "metrics password without username",
yaml: signingKeyLine + "metrics:\n password: hunter2\n",
wantErrSubstrings: []string{keyMetrics},
},
}
}
// TestSetButInvalidValueAbortsStartup verifies the no-silent-fallback
// rule: a key that is explicitly set to an unparseable or out-of-range
// value must produce a startup error naming the offending key, never
// silently fall back to the default.
func TestSetButInvalidValueAbortsStartup(t *testing.T) {
t.Parallel()
runAbortCases(t, append(
invalidScalarValueCases(), invalidHostAndCredentialCases()...))
}
// explicitNullValueCases are configs where a key is explicitly set to
// null (including the bare "key:" form and the "~" alias); each must
// abort startup naming the key.
func explicitNullValueCases() []abortCase {
return []abortCase{
{
name: "port explicit null",
yaml: signingKeyLine + "port: null\n",
wantErrSubstrings: []string{keyPort, nullValueText},
},
{
name: "port bare key no value",
yaml: signingKeyLine + "port:\n",
wantErrSubstrings: []string{keyPort, nullValueText},
},
{
name: "debug tilde null",
yaml: signingKeyLine + "debug: ~\n",
wantErrSubstrings: []string{keyDebug, nullValueText},
},
{
name: "maintenance_mode null",
yaml: signingKeyLine + "maintenance_mode: null\n",
wantErrSubstrings: []string{keyMaintenanceMode, nullValueText},
},
{
name: "allow_http null",
yaml: signingKeyLine + "allow_http: null\n",
wantErrSubstrings: []string{keyAllowHTTP, nullValueText},
},
{
name: "state_dir null",
yaml: signingKeyLine + "state_dir: null\n",
wantErrSubstrings: []string{keyStateDir, nullValueText},
},
{
name: "db_url null",
yaml: signingKeyLine + "db_url: null\n",
wantErrSubstrings: []string{keyDBURL, nullValueText},
},
{
name: "sentry_dsn null",
yaml: signingKeyLine + "sentry_dsn: null\n",
wantErrSubstrings: []string{keySentryDSN, nullValueText},
},
{
name: "upstream_connections_per_host null",
yaml: signingKeyLine + "upstream_connections_per_host: null\n",
wantErrSubstrings: []string{keyUpstreamConnectionsPerHost, nullValueText},
},
{
name: "allowlist_hosts null",
yaml: signingKeyLine + "allowlist_hosts: null\n",
wantErrSubstrings: []string{keyAllowlistHosts, nullValueText},
},
{
name: "signing_key null",
yaml: "signing_key: null\n",
wantErrSubstrings: []string{keySigningKey, nullValueText},
},
{
name: "metrics null",
yaml: signingKeyLine + "metrics: null\n",
wantErrSubstrings: []string{keyMetrics, nullValueText},
},
{
name: "metrics subkeys null",
yaml: signingKeyLine + "metrics:\n username: null\n password: null\n",
wantErrSubstrings: []string{
keyMetricsUsername, keyMetricsPassword, nullValueText,
},
},
}
}
// TestExplicitNullValueAbortsStartup verifies that a key explicitly
// set to null (including the bare "key:" form and the "~" alias) aborts
// startup naming the key. An explicit null is a SET value: it must
// never silently fall back to the default the way an omitted key does.
func TestExplicitNullValueAbortsStartup(t *testing.T) {
t.Parallel()
signingKeyLine := "signing_key: " + validTestSigningKey + "\n"
runAbortCases(t, explicitNullValueCases())
cases := []struct {
name string
yaml string
// wantErrSubstrings must all appear in the error message.
wantErrSubstrings []string
}{
{
name: "port explicit null",
yaml: signingKeyLine + "port: null\n",
wantErrSubstrings: []string{"port", "null"},
},
{
name: "port bare key no value",
yaml: signingKeyLine + "port:\n",
wantErrSubstrings: []string{"port", "null"},
},
{
name: "debug tilde null",
yaml: signingKeyLine + "debug: ~\n",
wantErrSubstrings: []string{"debug", "null"},
},
{
name: "maintenance_mode null",
yaml: signingKeyLine + "maintenance_mode: null\n",
wantErrSubstrings: []string{"maintenance_mode", "null"},
},
{
name: "allow_http null",
yaml: signingKeyLine + "allow_http: null\n",
wantErrSubstrings: []string{"allow_http", "null"},
},
{
name: "state_dir null",
yaml: signingKeyLine + "state_dir: null\n",
wantErrSubstrings: []string{"state_dir", "null"},
},
{
name: "db_url null",
yaml: signingKeyLine + "db_url: null\n",
wantErrSubstrings: []string{"db_url", "null"},
},
{
name: "sentry_dsn null",
yaml: signingKeyLine + "sentry_dsn: null\n",
wantErrSubstrings: []string{"sentry_dsn", "null"},
},
{
name: "upstream_connections_per_host null",
yaml: signingKeyLine + "upstream_connections_per_host: null\n",
wantErrSubstrings: []string{"upstream_connections_per_host", "null"},
},
{
name: "allowlist_hosts null",
yaml: signingKeyLine + "allowlist_hosts: null\n",
wantErrSubstrings: []string{"allowlist_hosts", "null"},
},
{
name: "signing_key null",
yaml: "signing_key: null\n",
wantErrSubstrings: []string{"signing_key", "null"},
},
{
name: "metrics null",
yaml: signingKeyLine + "metrics: null\n",
wantErrSubstrings: []string{"metrics", "null"},
},
{
name: "metrics subkeys null",
yaml: signingKeyLine + "metrics:\n username: null\n password: null\n",
wantErrSubstrings: []string{
"metrics.username", "metrics.password", "null",
},
},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
c, err := configFromYAML(t, tc.yaml)
if err == nil {
t.Fatalf("config with %s must abort startup, got config: %+v", tc.name, c)
}
t.Logf("got expected error: %v", err)
for _, want := range tc.wantErrSubstrings {
if !strings.Contains(err.Error(), want) {
t.Errorf("error %q does not mention %q", err.Error(), want)
}
}
})
}
}
// TestExplicitlyEmptyDBURLAbortsStartup verifies that db_url set to an
@@ -432,9 +400,7 @@ func TestExplicitNullValueAbortsStartup(t *testing.T) {
// a default, and defaults apply only to omitted keys. This matches
// state_dir, where an explicitly empty value already aborts.
func TestExplicitlyEmptyDBURLAbortsStartup(t *testing.T) {
t.Parallel()
yamlContent := signingKeyLine + "db_url: \"\"\n"
yamlContent := "signing_key: " + validTestSigningKey + "\ndb_url: \"\"\n"
c, err := configFromYAML(t, yamlContent)
if err == nil {
@@ -443,7 +409,7 @@ func TestExplicitlyEmptyDBURLAbortsStartup(t *testing.T) {
t.Logf("got expected error: %v", err)
if !strings.Contains(err.Error(), keyDBURL) {
if !strings.Contains(err.Error(), "db_url") {
t.Errorf("error %q does not name the offending key db_url", err.Error())
}
}
@@ -454,12 +420,10 @@ func TestExplicitlyEmptyDBURLAbortsStartup(t *testing.T) {
// host written in FQDN trailing-dot form (e.g. evil.com.) and
// effectively disable URL signing with a single character.
func TestAllowlistHostsRejectsDotOnlyEntries(t *testing.T) {
t.Parallel()
signingKeyLine := "signing_key: " + validTestSigningKey + "\n"
for _, entry := range []string{".", ".."} {
t.Run(entry, func(t *testing.T) {
t.Parallel()
yamlContent := signingKeyLine +
"allowlist_hosts:\n - \"" + entry + "\"\n"
@@ -471,7 +435,7 @@ func TestAllowlistHostsRejectsDotOnlyEntries(t *testing.T) {
t.Logf("got expected error: %v", err)
if !strings.Contains(err.Error(), keyAllowlistHosts) {
if !strings.Contains(err.Error(), "allowlist_hosts") {
t.Errorf("error %q does not name the offending key allowlist_hosts",
err.Error())
}
@@ -480,9 +444,8 @@ func TestAllowlistHostsRejectsDotOnlyEntries(t *testing.T) {
}
func TestUnknownTopLevelKeyAbortsStartup(t *testing.T) {
t.Parallel()
yamlContent := signingKeyLine + `whitelist_hosts:
yamlContent := `signing_key: ` + validTestSigningKey + `
whitelist_hosts:
- example.com
`
@@ -499,9 +462,8 @@ func TestUnknownTopLevelKeyAbortsStartup(t *testing.T) {
}
func TestUnknownMetricsSubkeyAbortsStartup(t *testing.T) {
t.Parallel()
yamlContent := signingKeyLine + `metrics:
yamlContent := `signing_key: ` + validTestSigningKey + `
metrics:
username: bob
password: hunter2
port: 9100
@@ -520,17 +482,13 @@ func TestUnknownMetricsSubkeyAbortsStartup(t *testing.T) {
}
func TestEnvSectionIsPermitted(t *testing.T) {
t.Parallel()
yamlContent := signingKeyLine + `env:
yamlContent := `signing_key: ` + validTestSigningKey + `
env:
PIXA_TEST_ENV_INJECTION: injected
`
_, err := configFromYAML(t, yamlContent)
if err != nil {
t.Fatalf(
"env section must be permitted (smartconfig consumes it), got error: %v",
err)
if _, err := configFromYAML(t, yamlContent); err != nil {
t.Fatalf("env section must be permitted (smartconfig consumes it), got error: %v", err)
}
}
@@ -538,8 +496,7 @@ func TestMalformedConfigFileAbortsStartup(t *testing.T) {
tmpDir := t.TempDir()
configPath := filepath.Join(tmpDir, "config.yml")
err := os.WriteFile(configPath, []byte("port: [unclosed\n"), 0o600)
if err != nil {
if err := os.WriteFile(configPath, []byte("port: [unclosed\n"), 0o600); err != nil {
t.Fatalf("failed to write malformed config: %v", err)
}
@@ -548,7 +505,7 @@ func TestMalformedConfigFileAbortsStartup(t *testing.T) {
t.Setenv("PIXA_CONFIG_PATH", "")
t.Chdir(tmpDir)
log := slog.New(slog.DiscardHandler)
log := slog.New(slog.NewTextHandler(io.Discard, nil))
sc, err := loadConfigFile(log, "pixa-test-nonexistent-app")
if err == nil {
@@ -559,14 +516,10 @@ func TestMalformedConfigFileAbortsStartup(t *testing.T) {
}
func TestEnsureStateDirCreatesDirectory(t *testing.T) {
t.Parallel()
stateDir := filepath.Join(t.TempDir(), "nested", "state")
c := &Config{StateDir: stateDir}
err := c.ensureStateDirWritable()
if err != nil {
if err := c.ensureStateDirWritable(); err != nil {
t.Fatalf("creatable state_dir must validate, got error: %v", err)
}
@@ -577,8 +530,6 @@ func TestEnsureStateDirCreatesDirectory(t *testing.T) {
}
func TestEnsureStateDirFailsOnUncreatablePath(t *testing.T) {
t.Parallel()
// A path below /dev/null can never be created, even when running
// as root (as in the Docker build).
c := &Config{StateDir: "/dev/null/pixa-state"}
@@ -590,7 +541,7 @@ func TestEnsureStateDirFailsOnUncreatablePath(t *testing.T) {
t.Logf("got expected error: %v", err)
if !strings.Contains(err.Error(), keyStateDir) {
if !strings.Contains(err.Error(), "state_dir") {
t.Errorf("error %q does not name the offending key state_dir", err.Error())
}
}

View File

@@ -5,7 +5,6 @@ import (
"context"
"database/sql"
"embed"
"errors"
"fmt"
"log/slog"
"path/filepath"
@@ -30,15 +29,10 @@ const bootstrapVersion = 0
// Params defines dependencies for Database.
type Params struct {
fx.In
Logger *logger.Logger
Config *config.Config
}
// errInvalidMigrationFilename is returned when a migration filename does
// not match the "<version>[_<description>].sql" pattern.
var errInvalidMigrationFilename = errors.New("invalid migration filename")
// Database wraps the SQL database connection.
type Database struct {
db *sql.DB
@@ -54,31 +48,33 @@ type Database struct {
func ParseMigrationVersion(filename string) (int, error) {
name := strings.TrimSuffix(filename, filepath.Ext(filename))
if name == "" {
return 0, fmt.Errorf("%w %q: empty name", errInvalidMigrationFilename, filename)
return 0, fmt.Errorf("invalid migration filename %q: empty name", filename)
}
// Split on underscore to separate version from description.
// If there's no underscore, the entire stem is the version.
versionStr, _, _ := strings.Cut(name, "_")
versionStr := name
if idx := strings.IndexByte(name, '_'); idx >= 0 {
versionStr = name[:idx]
}
if versionStr == "" {
return 0, fmt.Errorf(
"%w %q: empty version prefix", errInvalidMigrationFilename, filename,
)
return 0, fmt.Errorf("invalid migration filename %q: empty version prefix", filename)
}
// Validate the version is purely numeric.
for _, ch := range versionStr {
if ch < '0' || ch > '9' {
return 0, fmt.Errorf(
"%w %q: version %q contains non-numeric character %q",
errInvalidMigrationFilename, filename, versionStr, string(ch),
"invalid migration filename %q: version %q contains non-numeric character %q",
filename, versionStr, string(ch),
)
}
}
version, err := strconv.Atoi(versionStr)
if err != nil {
return 0, fmt.Errorf("%w %q: %w", errInvalidMigrationFilename, filename, err)
return 0, fmt.Errorf("invalid migration filename %q: %w", filename, err)
}
return version, nil
@@ -101,7 +97,6 @@ func New(lc fx.Lifecycle, params Params) (*Database, error) {
},
OnStop: func(_ context.Context) error {
s.log.Info("Database OnStop Hook")
if s.db != nil {
return s.db.Close()
}
@@ -113,6 +108,30 @@ func New(lc fx.Lifecycle, params Params) (*Database, error) {
return s, nil
}
func (s *Database) connect(ctx context.Context) error {
dbURL := s.config.DBURL
s.log.Info("connecting to database", "url", dbURL)
db, err := sql.Open("sqlite", dbURL)
if err != nil {
s.log.Error("failed to open database", "error", err)
return err
}
if err := db.PingContext(ctx); err != nil {
s.log.Error("failed to ping database", "error", err)
return err
}
s.db = db
s.log.Info("database connected")
return ApplyMigrations(ctx, s.db, s.log)
}
// collectMigrations reads the embedded schema directory and returns
// migration filenames sorted lexicographically.
func collectMigrations() ([]string, error) {
@@ -172,8 +191,7 @@ func bootstrapMigrationsTable(ctx context.Context, db *sql.DB, log *slog.Logger)
// This is exported so tests can apply the real schema without the full fx
// lifecycle.
func ApplyMigrations(ctx context.Context, db *sql.DB, log *slog.Logger) error {
err := bootstrapMigrationsTable(ctx, db, log)
if err != nil {
if err := bootstrapMigrationsTable(ctx, db, log); err != nil {
return err
}
@@ -243,28 +261,3 @@ func ApplyMigrations(ctx context.Context, db *sql.DB, log *slog.Logger) error {
func (s *Database) DB() *sql.DB {
return s.db
}
func (s *Database) connect(ctx context.Context) error {
dbURL := s.config.DBURL
s.log.Info("connecting to database", "url", dbURL)
db, err := sql.Open("sqlite", dbURL)
if err != nil {
s.log.Error("failed to open database", "error", err)
return err
}
err = db.PingContext(ctx)
if err != nil {
s.log.Error("failed to ping database", "error", err)
return err
}
s.db = db
s.log.Info("database connected")
return ApplyMigrations(ctx, s.db, s.log)
}

View File

@@ -1,6 +1,7 @@
package database
import (
"context"
"database/sql"
"testing"
@@ -16,14 +17,12 @@ func openTestDB(t *testing.T) *sql.DB {
t.Fatalf("failed to open test db: %v", err)
}
t.Cleanup(func() { _ = db.Close() })
t.Cleanup(func() { db.Close() })
return db
}
func TestParseMigrationVersion(t *testing.T) {
t.Parallel()
tests := []struct {
name string
filename string
@@ -79,8 +78,6 @@ func TestParseMigrationVersion(t *testing.T) {
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
got, err := ParseMigrationVersion(tt.filename)
if tt.wantErr {
if err == nil {
@@ -104,50 +101,37 @@ func TestParseMigrationVersion(t *testing.T) {
}
func TestApplyMigrations_CreatesSchemaAndTables(t *testing.T) {
t.Parallel()
db := openTestDB(t)
ctx := t.Context()
ctx := context.Background()
err := ApplyMigrations(ctx, db, nil)
if err != nil {
if err := ApplyMigrations(ctx, db, nil); err != nil {
t.Fatalf("ApplyMigrations failed: %v", err)
}
// The schema_migrations table must exist and contain at least
// version 0 (the bootstrap) and 1 (the initial schema).
rows, err := db.QueryContext(
ctx, "SELECT version FROM schema_migrations ORDER BY version",
)
rows, err := db.Query("SELECT version FROM schema_migrations ORDER BY version")
if err != nil {
t.Fatalf("failed to query schema_migrations: %v", err)
}
defer func() { _ = rows.Close() }()
defer rows.Close()
var versions []int
for rows.Next() {
var v int
scanErr := rows.Scan(&v)
if scanErr != nil {
t.Fatalf("failed to scan version: %v", scanErr)
if err := rows.Scan(&v); err != nil {
t.Fatalf("failed to scan version: %v", err)
}
versions = append(versions, v)
}
err = rows.Err()
if err != nil {
if err := rows.Err(); err != nil {
t.Fatalf("row iteration error: %v", err)
}
if len(versions) < 2 {
t.Fatalf(
"expected at least 2 migrations recorded, got %d: %v",
len(versions), versions,
)
t.Fatalf("expected at least 2 migrations recorded, got %d: %v", len(versions), versions)
}
if versions[0] != 0 {
@@ -159,15 +143,10 @@ func TestApplyMigrations_CreatesSchemaAndTables(t *testing.T) {
}
// Verify that the application tables created by 001.sql exist.
tables := []string{
"source_content", "source_metadata", "output_content",
"request_cache", "negative_cache", "cache_stats",
}
for _, table := range tables {
for _, table := range []string{"source_content", "source_metadata", "output_content", "request_cache", "negative_cache", "cache_stats"} {
var count int
err := db.QueryRowContext(
ctx,
err := db.QueryRow(
"SELECT COUNT(*) FROM sqlite_master WHERE type='table' AND name=?",
table,
).Scan(&count)
@@ -182,28 +161,22 @@ func TestApplyMigrations_CreatesSchemaAndTables(t *testing.T) {
}
func TestApplyMigrations_Idempotent(t *testing.T) {
t.Parallel()
db := openTestDB(t)
ctx := t.Context()
ctx := context.Background()
err := ApplyMigrations(ctx, db, nil)
if err != nil {
if err := ApplyMigrations(ctx, db, nil); err != nil {
t.Fatalf("first ApplyMigrations failed: %v", err)
}
// Running a second time must succeed without errors.
err = ApplyMigrations(ctx, db, nil)
if err != nil {
if err := ApplyMigrations(ctx, db, nil); err != nil {
t.Fatalf("second ApplyMigrations failed: %v", err)
}
// Verify no duplicate rows in schema_migrations.
var count int
err = db.QueryRowContext(
ctx, "SELECT COUNT(*) FROM schema_migrations WHERE version = 0",
).Scan(&count)
err := db.QueryRow("SELECT COUNT(*) FROM schema_migrations WHERE version = 0").Scan(&count)
if err != nil {
t.Fatalf("failed to count version 0 rows: %v", err)
}
@@ -214,21 +187,17 @@ func TestApplyMigrations_Idempotent(t *testing.T) {
}
func TestBootstrapMigrationsTable_FreshDatabase(t *testing.T) {
t.Parallel()
db := openTestDB(t)
ctx := t.Context()
ctx := context.Background()
err := bootstrapMigrationsTable(ctx, db, nil)
if err != nil {
if err := bootstrapMigrationsTable(ctx, db, nil); err != nil {
t.Fatalf("bootstrapMigrationsTable failed: %v", err)
}
// schema_migrations table must exist.
var tableCount int
err = db.QueryRowContext(
ctx,
err := db.QueryRow(
"SELECT COUNT(*) FROM sqlite_master WHERE type='table' AND name='schema_migrations'",
).Scan(&tableCount)
if err != nil {
@@ -242,8 +211,8 @@ func TestBootstrapMigrationsTable_FreshDatabase(t *testing.T) {
// Version 0 must be recorded.
var recorded int
err = db.QueryRowContext(
ctx, "SELECT COUNT(*) FROM schema_migrations WHERE version = 0",
err = db.QueryRow(
"SELECT COUNT(*) FROM schema_migrations WHERE version = 0",
).Scan(&recorded)
if err != nil {
t.Fatalf("failed to check version: %v", err)

View File

@@ -0,0 +1,25 @@
-- Migration 002: cache size accounting and eviction
--
-- Tracks processed variants in the database (source content blobs are
-- already tracked in source_content) so total cache usage can be
-- computed without directory scans, and adds last-access timestamps
-- for LRU eviction ordering.
-- Processed variant blobs
-- Files stored at: cache/variants/<ab>/<cd>/<cache_key> (plus a
-- .meta sidecar with the content type)
CREATE TABLE IF NOT EXISTS variant_content (
cache_key TEXT PRIMARY KEY,
size_bytes INTEGER NOT NULL,
content_type TEXT NOT NULL,
created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
last_accessed_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP
);
CREATE INDEX IF NOT EXISTS idx_variant_content_last_accessed
ON variant_content(last_accessed_at);
-- LRU timestamp for source content blobs. Rows written before this
-- migration have NULL here; eviction falls back to fetched_at.
ALTER TABLE source_content ADD COLUMN last_accessed_at DATETIME;
CREATE INDEX IF NOT EXISTS idx_source_content_last_accessed
ON source_content(last_accessed_at);

View File

@@ -48,8 +48,7 @@ type Generator struct {
key [seal.KeySize]byte
}
// NewGenerator creates an encrypted URL generator with a key derived
// from the signing key.
// NewGenerator creates an encrypted URL generator with a key derived from the signing key.
func NewGenerator(signingKey string) (*Generator, error) {
key, err := seal.DeriveKey([]byte(signingKey), urlKeySalt)
if err != nil {
@@ -78,8 +77,7 @@ func (g *Generator) Parse(token string) (*Payload, error) {
// Decrypt
data, err := seal.Decrypt(g.key, token)
if err != nil {
if errors.Is(err, seal.ErrDecryptionFailed) ||
errors.Is(err, seal.ErrInvalidPayload) {
if errors.Is(err, seal.ErrDecryptionFailed) || errors.Is(err, seal.ErrInvalidPayload) {
return nil, ErrDecryptFailed
}
@@ -88,9 +86,7 @@ func (g *Generator) Parse(token string) (*Payload, error) {
// CBOR decode
var p Payload
err = cbor.Unmarshal(data, &p)
if err != nil {
if err := cbor.Unmarshal(data, &p); err != nil {
return nil, ErrInvalidFormat
}

View File

@@ -1,33 +1,22 @@
package encurl_test
package encurl
import (
"errors"
"testing"
"time"
"sneak.berlin/go/pixa/internal/encurl"
"sneak.berlin/go/pixa/internal/imgcache"
)
// Shared test fixture strings.
const (
testSourceHost = "cdn.example.com"
testSourcePath = "/images/photo.jpg"
testSourceQuery = "v=2"
)
func TestGenerator_GenerateAndParse(t *testing.T) {
t.Parallel()
gen, err := encurl.NewGenerator("test-signing-key-12345")
gen, err := NewGenerator("test-signing-key-12345")
if err != nil {
t.Fatalf("NewGenerator() error = %v", err)
}
payload := &encurl.Payload{
SourceHost: testSourceHost,
SourcePath: testSourcePath,
SourceQuery: testSourceQuery,
payload := &Payload{
SourceHost: "cdn.example.com",
SourcePath: "/images/photo.jpg",
SourceQuery: "v=2",
Width: 800,
Height: 600,
Format: imgcache.FormatWebP,
@@ -54,48 +43,38 @@ func TestGenerator_GenerateAndParse(t *testing.T) {
if parsed.SourceHost != payload.SourceHost {
t.Errorf("SourceHost = %q, want %q", parsed.SourceHost, payload.SourceHost)
}
if parsed.SourcePath != payload.SourcePath {
t.Errorf("SourcePath = %q, want %q", parsed.SourcePath, payload.SourcePath)
}
if parsed.SourceQuery != payload.SourceQuery {
t.Errorf("SourceQuery = %q, want %q", parsed.SourceQuery, payload.SourceQuery)
}
if parsed.Width != payload.Width {
t.Errorf("Width = %d, want %d", parsed.Width, payload.Width)
}
if parsed.Height != payload.Height {
t.Errorf("Height = %d, want %d", parsed.Height, payload.Height)
}
if parsed.Format != payload.Format {
t.Errorf("Format = %q, want %q", parsed.Format, payload.Format)
}
if parsed.Quality != payload.Quality {
t.Errorf("Quality = %d, want %d", parsed.Quality, payload.Quality)
}
if parsed.FitMode != payload.FitMode {
t.Errorf("FitMode = %q, want %q", parsed.FitMode, payload.FitMode)
}
if parsed.ExpiresAt != payload.ExpiresAt {
t.Errorf("ExpiresAt = %d, want %d", parsed.ExpiresAt, payload.ExpiresAt)
}
}
func TestGenerator_Parse_Expired(t *testing.T) {
t.Parallel()
gen, _ := NewGenerator("test-signing-key-12345")
gen, _ := encurl.NewGenerator("test-signing-key-12345")
payload := &encurl.Payload{
SourceHost: testSourceHost,
SourcePath: testSourcePath,
payload := &Payload{
SourceHost: "cdn.example.com",
SourcePath: "/images/photo.jpg",
ExpiresAt: time.Now().Add(-time.Hour).Unix(), // Already expired
}
@@ -109,15 +88,13 @@ func TestGenerator_Parse_Expired(t *testing.T) {
t.Error("Parse() should fail for expired token")
}
if !errors.Is(err, encurl.ErrExpired) {
t.Errorf("Parse() error = %v, want %v", err, encurl.ErrExpired)
if err != ErrExpired {
t.Errorf("Parse() error = %v, want %v", err, ErrExpired)
}
}
func TestGenerator_Parse_InvalidToken(t *testing.T) {
t.Parallel()
gen, _ := encurl.NewGenerator("test-signing-key-12345")
gen, _ := NewGenerator("test-signing-key-12345")
_, err := gen.Parse("not-a-valid-token")
if err == nil {
@@ -126,13 +103,11 @@ func TestGenerator_Parse_InvalidToken(t *testing.T) {
}
func TestGenerator_Parse_TamperedToken(t *testing.T) {
t.Parallel()
gen, _ := NewGenerator("test-signing-key-12345")
gen, _ := encurl.NewGenerator("test-signing-key-12345")
payload := &encurl.Payload{
SourceHost: testSourceHost,
SourcePath: testSourcePath,
payload := &Payload{
SourceHost: "cdn.example.com",
SourcePath: "/images/photo.jpg",
ExpiresAt: time.Now().Add(time.Hour).Unix(),
}
@@ -151,14 +126,12 @@ func TestGenerator_Parse_TamperedToken(t *testing.T) {
}
func TestGenerator_Parse_WrongKey(t *testing.T) {
t.Parallel()
gen1, _ := NewGenerator("signing-key-1")
gen2, _ := NewGenerator("signing-key-2")
gen1, _ := encurl.NewGenerator("signing-key-1")
gen2, _ := encurl.NewGenerator("signing-key-2")
payload := &encurl.Payload{
SourceHost: testSourceHost,
SourcePath: testSourcePath,
payload := &Payload{
SourceHost: "cdn.example.com",
SourcePath: "/images/photo.jpg",
ExpiresAt: time.Now().Add(time.Hour).Unix(),
}
@@ -171,12 +144,10 @@ func TestGenerator_Parse_WrongKey(t *testing.T) {
}
func TestPayload_ToImageRequest(t *testing.T) {
t.Parallel()
payload := &encurl.Payload{
SourceHost: testSourceHost,
SourcePath: testSourcePath,
SourceQuery: testSourceQuery,
payload := &Payload{
SourceHost: "cdn.example.com",
SourcePath: "/images/photo.jpg",
SourceQuery: "v=2",
Width: 800,
Height: 600,
Format: imgcache.FormatWebP,
@@ -190,68 +161,55 @@ func TestPayload_ToImageRequest(t *testing.T) {
if req.SourceHost != payload.SourceHost {
t.Errorf("SourceHost = %q, want %q", req.SourceHost, payload.SourceHost)
}
if req.SourcePath != payload.SourcePath {
t.Errorf("SourcePath = %q, want %q", req.SourcePath, payload.SourcePath)
}
if req.SourceQuery != payload.SourceQuery {
t.Errorf("SourceQuery = %q, want %q", req.SourceQuery, payload.SourceQuery)
}
if req.Size.Width != payload.Width {
t.Errorf("Width = %d, want %d", req.Size.Width, payload.Width)
}
if req.Size.Height != payload.Height {
t.Errorf("Height = %d, want %d", req.Size.Height, payload.Height)
}
if req.Format != payload.Format {
t.Errorf("Format = %q, want %q", req.Format, payload.Format)
}
if req.Quality != payload.Quality {
t.Errorf("Quality = %d, want %d", req.Quality, payload.Quality)
}
if req.FitMode != payload.FitMode {
t.Errorf("FitMode = %q, want %q", req.FitMode, payload.FitMode)
}
}
func TestPayload_ToImageRequest_Defaults(t *testing.T) {
t.Parallel()
// Payload with only required fields - should get defaults
payload := &encurl.Payload{
SourceHost: testSourceHost,
SourcePath: testSourcePath,
payload := &Payload{
SourceHost: "cdn.example.com",
SourcePath: "/images/photo.jpg",
ExpiresAt: time.Now().Add(time.Hour).Unix(),
}
req := payload.ToImageRequest()
if req.Format != encurl.DefaultFormat {
t.Errorf("Format = %q, want default %q", req.Format, encurl.DefaultFormat)
if req.Format != DefaultFormat {
t.Errorf("Format = %q, want default %q", req.Format, DefaultFormat)
}
if req.Quality != encurl.DefaultQuality {
t.Errorf("Quality = %d, want default %d", req.Quality, encurl.DefaultQuality)
if req.Quality != DefaultQuality {
t.Errorf("Quality = %d, want default %d", req.Quality, DefaultQuality)
}
if req.FitMode != encurl.DefaultFitMode {
t.Errorf("FitMode = %q, want default %q", req.FitMode, encurl.DefaultFitMode)
if req.FitMode != DefaultFitMode {
t.Errorf("FitMode = %q, want default %q", req.FitMode, DefaultFitMode)
}
}
func TestFromImageRequest(t *testing.T) {
t.Parallel()
req := &imgcache.ImageRequest{
SourceHost: testSourceHost,
SourcePath: testSourcePath,
SourceQuery: testSourceQuery,
SourceHost: "cdn.example.com",
SourcePath: "/images/photo.jpg",
SourceQuery: "v=2",
Size: imgcache.Size{Width: 800, Height: 600},
Format: imgcache.FormatWebP,
Quality: 90,
@@ -259,62 +217,52 @@ func TestFromImageRequest(t *testing.T) {
}
expiresAt := time.Now().Add(time.Hour)
payload := encurl.FromImageRequest(req, expiresAt)
payload := FromImageRequest(req, expiresAt)
if payload.SourceHost != req.SourceHost {
t.Errorf("SourceHost = %q, want %q", payload.SourceHost, req.SourceHost)
}
if payload.SourcePath != req.SourcePath {
t.Errorf("SourcePath = %q, want %q", payload.SourcePath, req.SourcePath)
}
if payload.Width != req.Size.Width {
t.Errorf("Width = %d, want %d", payload.Width, req.Size.Width)
}
if payload.ExpiresAt != expiresAt.Unix() {
t.Errorf("ExpiresAt = %d, want %d", payload.ExpiresAt, expiresAt.Unix())
}
}
func TestFromImageRequest_OmitsDefaults(t *testing.T) {
t.Parallel()
// Request with default values - payload should omit them for
// smaller encoding
// Request with default values - payload should omit them for smaller encoding
req := &imgcache.ImageRequest{
SourceHost: testSourceHost,
SourcePath: testSourcePath,
Format: encurl.DefaultFormat,
Quality: encurl.DefaultQuality,
FitMode: encurl.DefaultFitMode,
SourceHost: "cdn.example.com",
SourcePath: "/images/photo.jpg",
Format: DefaultFormat,
Quality: DefaultQuality,
FitMode: DefaultFitMode,
}
payload := encurl.FromImageRequest(req, time.Now().Add(time.Hour))
payload := FromImageRequest(req, time.Now().Add(time.Hour))
// These should be zero/empty because they match defaults
if payload.Format != "" {
t.Errorf("Format should be empty for default, got %q", payload.Format)
}
if payload.Quality != 0 {
t.Errorf("Quality should be 0 for default, got %d", payload.Quality)
}
if payload.FitMode != "" {
t.Errorf("FitMode should be empty for default, got %q", payload.FitMode)
}
}
func TestGenerator_TokenIsURLSafe(t *testing.T) {
t.Parallel()
gen, _ := NewGenerator("test-signing-key-12345")
gen, _ := encurl.NewGenerator("test-signing-key-12345")
payload := &encurl.Payload{
SourceHost: testSourceHost,
SourcePath: testSourcePath,
payload := &Payload{
SourceHost: "cdn.example.com",
SourcePath: "/images/photo.jpg",
ExpiresAt: time.Now().Add(time.Hour).Unix(),
}

View File

@@ -35,8 +35,7 @@ func (s *Handlers) HandleRoot() http.HandlerFunc {
// handleLoginPost handles login form submission.
func (s *Handlers) handleLoginPost(w http.ResponseWriter, r *http.Request) {
err := r.ParseForm()
if err != nil {
if err := r.ParseForm(); err != nil {
s.renderLogin(w, "Invalid form data")
return
@@ -53,8 +52,7 @@ func (s *Handlers) handleLoginPost(w http.ResponseWriter, r *http.Request) {
}
// Create session
err = s.sessMgr.CreateSession(w)
if err != nil {
if err := s.sessMgr.CreateSession(w); err != nil {
s.log.Error("failed to create session", "error", err)
s.renderLogin(w, "Failed to create session")
@@ -85,14 +83,20 @@ func (s *Handlers) HandleGenerateURL() http.HandlerFunc {
return
}
err := r.ParseForm()
if err != nil {
if err := r.ParseForm(); err != nil {
s.renderGenerator(w, &generatorData{Error: "Invalid form data"})
return
}
// Parse form values
sourceURL := r.FormValue("url")
widthStr := r.FormValue("width")
heightStr := r.FormValue("height")
format := r.FormValue("format")
qualityStr := r.FormValue("quality")
fit := r.FormValue("fit")
ttlStr := r.FormValue("ttl")
// Validate source URL
parsed, err := url.Parse(sourceURL)
@@ -102,7 +106,38 @@ func (s *Handlers) HandleGenerateURL() http.HandlerFunc {
return
}
payload, expiresAt, ttl := buildGeneratePayload(parsed, r.Form)
// Parse dimensions
width, _ := strconv.Atoi(widthStr)
height, _ := strconv.Atoi(heightStr)
quality, _ := strconv.Atoi(qualityStr)
ttl, _ := strconv.Atoi(ttlStr)
if quality <= 0 {
quality = 85
}
// Create payload
// ttl=0 means never expires
var expiresAt time.Time
var expiresAtUnix int64
if ttl > 0 {
expiresAt = time.Now().Add(time.Duration(ttl) * time.Second)
expiresAtUnix = expiresAt.Unix()
}
// else expiresAtUnix stays 0 (never expires)
payload := &encurl.Payload{
SourceHost: parsed.Host,
SourcePath: parsed.Path,
SourceQuery: parsed.RawQuery,
Width: width,
Height: height,
Format: imgcache.ImageFormat(format),
Quality: quality,
FitMode: imgcache.FitMode(fit),
ExpiresAt: expiresAtUnix,
}
// Generate encrypted token
token, err := s.encGen.Generate(payload)
@@ -113,7 +148,20 @@ func (s *Handlers) HandleGenerateURL() http.HandlerFunc {
return
}
generatedURL := s.buildGeneratedURL(r, token, r.FormValue("format"))
// Build full URL (URL-encode the token for safety)
scheme := "https"
if s.config.Debug {
scheme = "http"
}
// Determine file extension for the trailing filename
ext := format
if ext == "" || ext == "orig" {
ext = "jpg" // Default extension
}
host := r.Host
generatedURL := scheme + "://" + host + "/v1/e/" + url.PathEscape(token) + "/img." + ext
// Format expiry for display
expiresAtStr := "Never"
@@ -125,55 +173,16 @@ func (s *Handlers) HandleGenerateURL() http.HandlerFunc {
GeneratedURL: generatedURL,
ExpiresAt: expiresAtStr,
FormURL: sourceURL,
FormWidth: r.FormValue("width"),
FormHeight: r.FormValue("height"),
FormFormat: r.FormValue("format"),
FormQuality: r.FormValue("quality"),
FormFit: r.FormValue("fit"),
FormTTL: r.FormValue("ttl"),
FormWidth: widthStr,
FormHeight: heightStr,
FormFormat: format,
FormQuality: qualityStr,
FormFit: fit,
FormTTL: ttlStr,
})
}
}
// buildGeneratePayload parses the numeric form fields and assembles the
// encrypted URL payload. ttl=0 means never expires (ExpiresAt stays 0).
func buildGeneratePayload(
parsed *url.URL, form url.Values,
) (*encurl.Payload, time.Time, int) {
width, _ := strconv.Atoi(form.Get("width"))
height, _ := strconv.Atoi(form.Get("height"))
quality, _ := strconv.Atoi(form.Get("quality"))
ttl, _ := strconv.Atoi(form.Get("ttl"))
if quality <= 0 {
quality = 85
}
var (
expiresAt time.Time
expiresAtUnix int64
)
if ttl > 0 {
expiresAt = time.Now().Add(time.Duration(ttl) * time.Second)
expiresAtUnix = expiresAt.Unix()
}
payload := &encurl.Payload{
SourceHost: parsed.Host,
SourcePath: parsed.Path,
SourceQuery: parsed.RawQuery,
Width: width,
Height: height,
Format: imgcache.ImageFormat(form.Get("format")),
Quality: quality,
FitMode: imgcache.FitMode(form.Get("fit")),
ExpiresAt: expiresAtUnix,
}
return payload, expiresAt, ttl
}
// generatorData holds template data for the generator page.
type generatorData struct {
GeneratedURL string
@@ -197,8 +206,7 @@ func (s *Handlers) renderLogin(w http.ResponseWriter, errorMsg string) {
Error: errorMsg,
}
err := templates.Render(w, "login.html", data)
if err != nil {
if err := templates.Render(w, "login.html", data); err != nil {
s.log.Error("failed to render login template", "error", err)
http.Error(w, "Internal server error", http.StatusInternalServerError)
}
@@ -211,16 +219,13 @@ func (s *Handlers) renderGenerator(w http.ResponseWriter, data *generatorData) {
data = &generatorData{}
}
err := templates.Render(w, "generator.html", data)
if err != nil {
if err := templates.Render(w, "generator.html", data); err != nil {
s.log.Error("failed to render generator template", "error", err)
http.Error(w, "Internal server error", http.StatusInternalServerError)
}
}
func (s *Handlers) renderGeneratorWithForm(
w http.ResponseWriter, errorMsg string, form url.Values,
) {
func (s *Handlers) renderGeneratorWithForm(w http.ResponseWriter, errorMsg string, form url.Values) {
s.renderGenerator(w, &generatorData{
Error: errorMsg,
FormURL: form.Get("url"),
@@ -232,19 +237,3 @@ func (s *Handlers) renderGeneratorWithForm(
FormTTL: form.Get("ttl"),
})
}
func (s *Handlers) buildGeneratedURL(r *http.Request, token, format string) string {
// Build full URL (URL-encode the token for safety)
scheme := "https"
if s.config.Debug {
scheme = "http"
}
// Determine file extension for the trailing filename
ext := format
if ext == "" || ext == "orig" {
ext = "jpg" // Default extension
}
return scheme + "://" + r.Host + "/v1/e/" + url.PathEscape(token) + "/img." + ext
}

View File

@@ -22,7 +22,6 @@ import (
// Params defines dependencies for Handlers.
type Params struct {
fx.In
Logger *logger.Logger
Healthcheck *healthcheck.Healthcheck
Database *database.Database
@@ -54,6 +53,13 @@ func New(lc fx.Lifecycle, params Params) (*Handlers, error) {
OnStart: func(_ context.Context) error {
return s.initImageService()
},
OnStop: func(_ context.Context) error {
if s.imgCache != nil {
s.imgCache.StopEviction()
}
return nil
},
})
return s, nil
@@ -61,11 +67,15 @@ func New(lc fx.Lifecycle, params Params) (*Handlers, error) {
// initImageService initializes the image cache and service.
func (s *Handlers) initImageService() error {
// Create the cache
// Create the cache. cache_max_bytes: 0 disables the disk cache
// entirely; any other value is the eviction limit in bytes.
cache, err := imgcache.NewCache(s.db.DB(), imgcache.CacheConfig{
StateDir: s.config.StateDir,
CacheTTL: imgcache.DefaultCacheTTL,
NegativeTTL: imgcache.DefaultNegativeTTL,
MaxBytes: s.config.CacheMaxBytes,
DisableDiskCache: s.config.CacheMaxBytes == 0,
Logger: s.log,
})
if err != nil {
return err
@@ -73,10 +83,13 @@ func (s *Handlers) initImageService() error {
s.imgCache = cache
// Background eviction: startup reconciliation, then periodic and
// write-pressure passes. No-op when the disk cache is disabled.
cache.StartEviction(imgcache.DefaultEvictionInterval)
// Create the fetcher config
fetcherCfg := httpfetcher.DefaultConfig()
fetcherCfg.AllowHTTP = s.config.AllowHTTP
if s.config.UpstreamConnectionsPerHost > 0 {
fetcherCfg.MaxConnectionsPerHost = s.config.UpstreamConnectionsPerHost
}
@@ -102,7 +115,6 @@ func (s *Handlers) initImageService() error {
if err != nil {
return err
}
s.sessMgr = sessMgr
// Initialize encrypted URL generator
@@ -110,7 +122,6 @@ func (s *Handlers) initImageService() error {
if err != nil {
return err
}
s.encGen = encGen
s.log.Info("session manager and URL generator initialized")
@@ -118,10 +129,9 @@ func (s *Handlers) initImageService() error {
return nil
}
func (s *Handlers) respondJSON(w http.ResponseWriter, data any, status int) {
func (s *Handlers) respondJSON(w http.ResponseWriter, data interface{}, status int) {
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(status)
if data != nil {
err := json.NewEncoder(w).Encode(data)
if err != nil {
@@ -131,7 +141,7 @@ func (s *Handlers) respondJSON(w http.ResponseWriter, data any, status int) {
}
func (s *Handlers) respondError(w http.ResponseWriter, message string, status int) {
s.respondJSON(w, map[string]any{
s.respondJSON(w, map[string]interface{}{
"error": message,
"status": status,
"timestamp": time.Now().UTC().Format(time.RFC3339),

View File

@@ -83,8 +83,7 @@ func setupTestDB(t *testing.T) *sql.DB {
t.Fatalf("failed to open test db: %v", err)
}
err = database.ApplyMigrations(context.Background(), db, nil)
if err != nil {
if err := database.ApplyMigrations(context.Background(), db, nil); err != nil {
t.Fatalf("failed to apply migrations: %v", err)
}
@@ -95,16 +94,14 @@ func generateTestJPEG(t *testing.T, width, height int, c color.Color) []byte {
t.Helper()
img := image.NewRGBA(image.Rect(0, 0, width, height))
for y := range height {
for x := range width {
for y := 0; y < height; y++ {
for x := 0; x < width; x++ {
img.Set(x, y, c)
}
}
var buf bytes.Buffer
err := jpeg.Encode(&buf, img, &jpeg.Options{Quality: 85})
if err != nil {
if err := jpeg.Encode(&buf, img, &jpeg.Options{Quality: 85}); err != nil {
t.Fatalf("failed to encode test JPEG: %v", err)
}
@@ -120,9 +117,7 @@ func newMockFetcher(fs fs.FS) *mockFetcher {
return &mockFetcher{fs: fs}
}
func (f *mockFetcher) Fetch(
_ context.Context, url string,
) (*httpfetcher.FetchResult, error) {
func (f *mockFetcher) Fetch(ctx context.Context, url string) (*httpfetcher.FetchResult, error) {
// Remove https:// prefix
path := url[8:] // Remove "https://"
@@ -139,16 +134,13 @@ func (f *mockFetcher) Fetch(
}
func TestHandleImage_HEAD_ReturnsHeadersOnly(t *testing.T) {
t.Parallel()
fix := setupTestHandler(t)
// Create a chi router to properly handle wildcards
r := chi.NewRouter()
r.Head("/v1/image/*", fix.handler.HandleImage())
req := httptest.NewRequestWithContext(t.Context(), http.MethodHead,
"/v1/image/"+fix.goodHost+"/images/photo.jpg/50x50.jpeg", nil)
req := httptest.NewRequest(http.MethodHead, "/v1/image/"+fix.goodHost+"/images/photo.jpg/50x50.jpeg", nil)
rec := httptest.NewRecorder()
r.ServeHTTP(rec, req)
@@ -175,16 +167,13 @@ func TestHandleImage_HEAD_ReturnsHeadersOnly(t *testing.T) {
}
func TestHandleImage_ConditionalRequest_IfNoneMatch_Returns304(t *testing.T) {
t.Parallel()
fix := setupTestHandler(t)
r := chi.NewRouter()
r.Get("/v1/image/*", fix.handler.HandleImage())
// First request to get the ETag
req1 := httptest.NewRequestWithContext(t.Context(), http.MethodGet,
"/v1/image/"+fix.goodHost+"/images/photo.jpg/50x50.jpeg", nil)
req1 := httptest.NewRequest(http.MethodGet, "/v1/image/"+fix.goodHost+"/images/photo.jpg/50x50.jpeg", nil)
rec1 := httptest.NewRecorder()
r.ServeHTTP(rec1, req1)
@@ -199,18 +188,15 @@ func TestHandleImage_ConditionalRequest_IfNoneMatch_Returns304(t *testing.T) {
}
// Second request with If-None-Match header
req2 := httptest.NewRequestWithContext(t.Context(), http.MethodGet,
"/v1/image/"+fix.goodHost+"/images/photo.jpg/50x50.jpeg", nil)
req2 := httptest.NewRequest(http.MethodGet, "/v1/image/"+fix.goodHost+"/images/photo.jpg/50x50.jpeg", nil)
req2.Header.Set("If-None-Match", etag)
rec2 := httptest.NewRecorder()
r.ServeHTTP(rec2, req2)
// Should return 304 Not Modified
if rec2.Code != http.StatusNotModified {
t.Errorf("Conditional request status = %d, want %d",
rec2.Code, http.StatusNotModified)
t.Errorf("Conditional request status = %d, want %d", rec2.Code, http.StatusNotModified)
}
// Body should be empty for 304 response
@@ -220,26 +206,21 @@ func TestHandleImage_ConditionalRequest_IfNoneMatch_Returns304(t *testing.T) {
}
func TestHandleImage_ConditionalRequest_IfNoneMatch_DifferentETag(t *testing.T) {
t.Parallel()
fix := setupTestHandler(t)
r := chi.NewRouter()
r.Get("/v1/image/*", fix.handler.HandleImage())
// Request with non-matching ETag
req := httptest.NewRequestWithContext(t.Context(), http.MethodGet,
"/v1/image/"+fix.goodHost+"/images/photo.jpg/50x50.jpeg", nil)
req := httptest.NewRequest(http.MethodGet, "/v1/image/"+fix.goodHost+"/images/photo.jpg/50x50.jpeg", nil)
req.Header.Set("If-None-Match", `"different-etag"`)
rec := httptest.NewRecorder()
r.ServeHTTP(rec, req)
// Should return 200 OK with full response
if rec.Code != http.StatusOK {
t.Errorf("Request with non-matching ETag status = %d, want %d",
rec.Code, http.StatusOK)
t.Errorf("Request with non-matching ETag status = %d, want %d", rec.Code, http.StatusOK)
}
// Body should not be empty
@@ -249,15 +230,12 @@ func TestHandleImage_ConditionalRequest_IfNoneMatch_DifferentETag(t *testing.T)
}
func TestHandleImage_ETagHeader(t *testing.T) {
t.Parallel()
fix := setupTestHandler(t)
r := chi.NewRouter()
r.Get("/v1/image/*", fix.handler.HandleImage())
req := httptest.NewRequestWithContext(t.Context(), http.MethodGet,
"/v1/image/"+fix.goodHost+"/images/photo.jpg/50x50.jpeg", nil)
req := httptest.NewRequest(http.MethodGet, "/v1/image/"+fix.goodHost+"/images/photo.jpg/50x50.jpeg", nil)
rec := httptest.NewRecorder()
r.ServeHTTP(rec, req)

View File

@@ -16,14 +16,64 @@ import (
// /v1/image/<host>/<path>/<width>x<height>.<format>
func (s *Handlers) HandleImage() http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
req, ok := s.parseImageRequest(w, r)
if !ok {
ctx := r.Context()
// Get the wildcard path from chi
pathParam := chi.URLParam(r, "*")
// Parse the URL path
parsed, err := imgcache.ParseImagePath(pathParam)
if err != nil {
s.log.Warn("failed to parse image URL",
"path", pathParam,
"error", err,
)
s.respondError(w, "invalid image URL: "+err.Error(), http.StatusBadRequest)
return
}
// Convert to ImageRequest
req := parsed.ToImageRequest()
// Parse signature params from query string
query := r.URL.Query()
req.Signature = query.Get("sig")
if expStr := query.Get("exp"); expStr != "" {
if exp, err := strconv.ParseInt(expStr, 10, 64); err == nil {
req.Expires = time.Unix(exp, 0)
}
}
// Parse optional quality and fit params
if qStr := query.Get("q"); qStr != "" {
if q, err := strconv.Atoi(qStr); err == nil && q > 0 && q <= 100 {
req.Quality = q
}
}
if fit := query.Get("fit"); fit != "" {
req.FitMode = imgcache.FitMode(fit)
if err := imgcache.ValidateFitMode(req.FitMode); err != nil {
s.respondError(w, "invalid fit mode: "+fit, http.StatusBadRequest)
return
}
}
// Default quality if not set
if req.Quality == 0 {
req.Quality = 85
}
// Default fit mode if not set
if req.FitMode == "" {
req.FitMode = imgcache.FitCover
}
// Validate signature if required
err := s.imgSvc.ValidateRequest(req)
if err != nil {
if err := s.imgSvc.ValidateRequest(req); err != nil {
s.log.Warn("signature validation failed",
"host", req.SourceHost,
"path", req.SourcePath,
@@ -39,103 +89,8 @@ func (s *Handlers) HandleImage() http.HandlerFunc {
// Get the image (from cache or fetch/process)
startTime := time.Now()
resp, err := s.imgSvc.Get(r.Context(), req)
resp, err := s.imgSvc.Get(ctx, req)
if err != nil {
s.respondImageError(w, req, err)
return
}
defer func() { _ = resp.Content.Close() }()
s.writeImageResponse(w, r, req, resp, cacheKey, startTime)
}
}
// HandleRobotsTxt serves robots.txt to prevent search engine crawling.
func (s *Handlers) HandleRobotsTxt() http.HandlerFunc {
robotsTxt := []byte("User-agent: *\nDisallow: /\n")
return func(w http.ResponseWriter, _ *http.Request) {
w.Header().Set("Content-Type", "text/plain")
w.Header().Set("Content-Length", strconv.Itoa(len(robotsTxt)))
w.WriteHeader(http.StatusOK)
_, _ = w.Write(robotsTxt)
}
}
// parseImageRequest parses the wildcard path and query parameters into
// an ImageRequest. On invalid input it writes an error response and
// returns false.
func (s *Handlers) parseImageRequest(
w http.ResponseWriter, r *http.Request,
) (*imgcache.ImageRequest, bool) {
// Get the wildcard path from chi
pathParam := chi.URLParam(r, "*")
// Parse the URL path
parsed, err := imgcache.ParseImagePath(pathParam)
if err != nil {
s.log.Warn("failed to parse image URL",
"path", pathParam,
"error", err,
)
s.respondError(w, "invalid image URL: "+err.Error(), http.StatusBadRequest)
return nil, false
}
// Convert to ImageRequest
req := parsed.ToImageRequest()
// Parse signature params from query string
query := r.URL.Query()
req.Signature = query.Get("sig")
if expStr := query.Get("exp"); expStr != "" {
exp, parseErr := strconv.ParseInt(expStr, 10, 64)
if parseErr == nil {
req.Expires = time.Unix(exp, 0)
}
}
// Parse optional quality and fit params
if qStr := query.Get("q"); qStr != "" {
q, parseErr := strconv.Atoi(qStr)
if parseErr == nil && q > 0 && q <= 100 {
req.Quality = q
}
}
if fit := query.Get("fit"); fit != "" {
req.FitMode = imgcache.FitMode(fit)
fitErr := imgcache.ValidateFitMode(req.FitMode)
if fitErr != nil {
s.respondError(w, "invalid fit mode: "+fit, http.StatusBadRequest)
return nil, false
}
}
// Default quality if not set
if req.Quality == 0 {
req.Quality = 85
}
// Default fit mode if not set
if req.FitMode == "" {
req.FitMode = imgcache.FitCover
}
return req, true
}
// respondImageError maps image retrieval errors to HTTP responses.
func (s *Handlers) respondImageError(
w http.ResponseWriter, req *imgcache.ImageRequest, err error,
) {
s.log.Error("failed to get image",
"host", req.SourceHost,
"path", req.SourcePath,
@@ -156,18 +111,13 @@ func (s *Handlers) respondImageError(
}
s.respondError(w, "internal error", http.StatusInternalServerError)
}
// writeImageResponse writes headers and streams the image content,
// handling conditional and HEAD requests.
func (s *Handlers) writeImageResponse(
w http.ResponseWriter, r *http.Request,
req *imgcache.ImageRequest, resp *imgcache.ImageResponse,
cacheKey imgcache.VariantKey, startTime time.Time,
) {
return
}
defer func() { _ = resp.Content.Close() }()
// Set response headers
w.Header().Set("Content-Type", resp.ContentType)
if resp.ContentLength > 0 {
w.Header().Set("Content-Length", strconv.FormatInt(resp.ContentLength, 10))
}
@@ -216,4 +166,17 @@ func (s *Handlers) writeImageResponse(
"served_bytes", servedBytes,
"fetched_bytes", resp.FetchedBytes,
)
}
}
// HandleRobotsTxt serves robots.txt to prevent search engine crawling.
func (s *Handlers) HandleRobotsTxt() http.HandlerFunc {
robotsTxt := []byte("User-agent: *\nDisallow: /\n")
return func(w http.ResponseWriter, _ *http.Request) {
w.Header().Set("Content-Type", "text/plain")
w.Header().Set("Content-Length", strconv.Itoa(len(robotsTxt)))
w.WriteHeader(http.StatusOK)
_, _ = w.Write(robotsTxt)
}
}

View File

@@ -15,9 +15,8 @@ import (
"sneak.berlin/go/pixa/internal/imgcache"
)
// HandleImageEnc handles requests to /v1/e/{token}/* for encrypted
// image URLs. The trailing path (e.g., /img.jpg) is ignored but helps
// browsers identify the content type.
// HandleImageEnc handles requests to /v1/e/{token}/* for encrypted image URLs.
// The trailing path (e.g., /img.jpg) is ignored but helps browsers identify the content type.
func (s *Handlers) HandleImageEnc() http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
@@ -58,8 +57,7 @@ func (s *Handlers) HandleImageEnc() http.HandlerFunc {
"format", req.Format,
)
// Fetch and process the image (no signature validation
// needed - encrypted URL is trusted)
// Fetch and process the image (no signature validation needed - encrypted URL is trusted)
resp, err := s.imgSvc.Get(ctx, req)
if err != nil {
s.handleImageError(w, err)
@@ -70,7 +68,6 @@ func (s *Handlers) HandleImageEnc() http.HandlerFunc {
// Set response headers
w.Header().Set("Content-Type", resp.ContentType)
if resp.ContentLength > 0 {
w.Header().Set("Content-Length", strconv.FormatInt(resp.ContentLength, 10))
}

View File

@@ -16,7 +16,6 @@ import (
// Params defines dependencies for Healthcheck.
type Params struct {
fx.In
Globals *globals.Globals
Config *config.Config
Logger *logger.Logger
@@ -54,8 +53,6 @@ func New(lc fx.Lifecycle, params Params) (*Healthcheck, error) {
}
// Response is the JSON response for health checks.
//
//nolint:tagliatelle // health endpoint response format uses snake_case
type Response struct {
Status string `json:"status"`
Now string `json:"now"`
@@ -66,6 +63,10 @@ type Response struct {
Maintenance bool `json:"maintenance_mode"`
}
func (s *Healthcheck) uptime() time.Duration {
return time.Since(s.StartupTime)
}
// Healthcheck returns the current health status.
func (s *Healthcheck) Healthcheck() *Response {
resp := &Response{
@@ -80,7 +81,3 @@ func (s *Healthcheck) Healthcheck() *Response {
return resp
}
func (s *Healthcheck) uptime() time.Duration {
return time.Since(s.StartupTime)
}

View File

@@ -12,7 +12,6 @@ import (
"net/http"
"net/http/httptrace"
neturl "net/url"
"slices"
"strings"
"sync"
"time"
@@ -29,23 +28,6 @@ const (
DefaultMaxConnectionsPerHost = 20
)
// MIME content types.
const (
contentTypeJPEG = "image/jpeg"
contentTypePNG = "image/png"
contentTypeGIF = "image/gif"
contentTypeWebP = "image/webp"
contentTypeAVIF = "image/avif"
contentTypeSVG = "image/svg+xml"
contentTypeOctetStream = "application/octet-stream"
)
// Loopback addresses blocked by SSRF protection.
const (
localhostIPv4 = "127.0.0.1"
localhostIPv6 = "::1"
)
// Fetcher errors.
var (
ErrSSRFBlocked = errors.New("request blocked: private or internal IP")
@@ -57,12 +39,6 @@ var (
ErrUpstreamTimeout = errors.New("upstream request timeout")
)
// Internal fetcher errors.
var (
errTooManyRedirects = errors.New("too many redirects")
errConnectFailed = errors.New("failed to connect")
)
// Fetcher retrieves content from upstream origins.
type Fetcher interface {
// Fetch retrieves content from the given URL.
@@ -116,12 +92,12 @@ func DefaultConfig() *Config {
MaxResponseSize: DefaultMaxResponseSize,
UserAgent: "pixa/1.0",
AllowedContentTypes: []string{
contentTypeJPEG,
contentTypePNG,
contentTypeGIF,
contentTypeWebP,
contentTypeAVIF,
contentTypeSVG,
"image/jpeg",
"image/png",
"image/gif",
"image/webp",
"image/avif",
"image/svg+xml",
},
AllowHTTP: false,
MaxConnectionsPerHost: DefaultMaxConnectionsPerHost,
@@ -156,12 +132,10 @@ func New(config *Config) *HTTPFetcher {
// Don't follow redirects automatically - we need to validate each hop
CheckRedirect: func(req *http.Request, via []*http.Request) error {
if len(via) >= DefaultMaxRedirects {
return errTooManyRedirects
return errors.New("too many redirects")
}
// Validate the redirect target
err := validateURL(req.Context(), req.URL.String(), config.AllowHTTP)
if err != nil {
if err := validateURL(req.URL.String(), config.AllowHTTP); err != nil {
return fmt.Errorf("redirect blocked: %w", err)
}
@@ -176,11 +150,24 @@ func New(config *Config) *HTTPFetcher {
}
}
// getHostSemaphore returns the semaphore for a host, creating it if necessary.
func (f *HTTPFetcher) getHostSemaphore(host string) chan struct{} {
f.hostSemMu.Lock()
defer f.hostSemMu.Unlock()
sem, ok := f.hostSems[host]
if !ok {
sem = make(chan struct{}, f.config.MaxConnectionsPerHost)
f.hostSems[host] = sem
}
return sem
}
// Fetch retrieves content from the given URL with SSRF protection.
func (f *HTTPFetcher) Fetch(ctx context.Context, url string) (*FetchResult, error) {
// Validate URL before making request
err := validateURL(ctx, url, f.config.AllowHTTP)
if err != nil {
if err := validateURL(url, f.config.AllowHTTP); err != nil {
return nil, err
}
@@ -214,6 +201,7 @@ func (f *HTTPFetcher) Fetch(ctx context.Context, url string) (*FetchResult, erro
URL: parsedURL,
Header: make(http.Header),
}
req = req.WithContext(ctx)
req.Header.Set("User-Agent", f.config.UserAgent)
req.Header.Set("Accept", strings.Join(f.config.AllowedContentTypes, ", "))
@@ -228,10 +216,11 @@ func (f *HTTPFetcher) Fetch(ctx context.Context, url string) (*FetchResult, erro
}
},
}
req = req.WithContext(httptrace.WithClientTrace(ctx, trace))
req = req.WithContext(httptrace.WithClientTrace(req.Context(), trace))
startTime := time.Now()
//nolint:gosec // G704: URL validated by validateURL() above
resp, err := f.client.Do(req)
fetchDuration := time.Since(startTime)
@@ -244,39 +233,6 @@ func (f *HTTPFetcher) Fetch(ctx context.Context, url string) (*FetchResult, erro
return nil, fmt.Errorf("upstream request failed: %w", err)
}
result, err := f.buildResult(resp, remoteAddr, fetchDuration, sem)
if err != nil {
return nil, err
}
// Mark success so defer doesn't release the semaphore
success = true
return result, nil
}
// getHostSemaphore returns the semaphore for a host, creating it if necessary.
func (f *HTTPFetcher) getHostSemaphore(host string) chan struct{} {
f.hostSemMu.Lock()
defer f.hostSemMu.Unlock()
sem, ok := f.hostSems[host]
if !ok {
sem = make(chan struct{}, f.config.MaxConnectionsPerHost)
f.hostSems[host] = sem
}
return sem
}
// buildResult validates the upstream response and assembles a FetchResult
// whose Content releases the host semaphore slot when closed.
func (f *HTTPFetcher) buildResult(
resp *http.Response,
remoteAddr string,
fetchDuration time.Duration,
sem chan struct{},
) (*FetchResult, error) {
// Extract HTTP version (strip "HTTP/" prefix)
httpVersion := strings.TrimPrefix(resp.Proto, "HTTP/")
@@ -309,6 +265,9 @@ func (f *HTTPFetcher) buildResult(
remaining: f.config.MaxResponseSize,
}
// Mark success so defer doesn't release the semaphore
success = true
return &FetchResult{
Content: &semaphoreReleasingReadCloser{limitedBody, resp.Body, sem},
ContentLength: resp.ContentLength,
@@ -338,7 +297,7 @@ func (f *HTTPFetcher) isAllowedContentType(contentType string) bool {
}
// validateURL checks if a URL is safe to fetch (not internal/private).
func validateURL(ctx context.Context, rawURL string, allowHTTP bool) error {
func validateURL(rawURL string, allowHTTP bool) error {
if !allowHTTP && !strings.HasPrefix(rawURL, "https://") {
return ErrUnsupportedScheme
}
@@ -350,8 +309,7 @@ func validateURL(ctx context.Context, rawURL string, allowHTTP bool) error {
}
// Remove port if present
h, _, err := net.SplitHostPort(host)
if err == nil {
if h, _, err := net.SplitHostPort(host); err == nil {
host = h
}
@@ -361,17 +319,16 @@ func validateURL(ctx context.Context, rawURL string, allowHTTP bool) error {
}
// Resolve the host to check IP addresses
addrs, err := net.DefaultResolver.LookupIPAddr(ctx, host)
ips, err := net.LookupIP(host)
if err != nil {
return fmt.Errorf("%w: %s", ErrInvalidHost, host)
}
private := slices.ContainsFunc(addrs, func(addr net.IPAddr) bool {
return isPrivateIP(addr.IP)
})
if private {
for _, ip := range ips {
if isPrivateIP(ip) {
return ErrSSRFBlocked
}
}
return nil
}
@@ -383,11 +340,9 @@ func extractHost(rawURL string) string {
if idx := strings.Index(url, "://"); idx != -1 {
url = url[idx+3:]
}
if idx := strings.Index(url, "/"); idx != -1 {
url = url[:idx]
}
if idx := strings.Index(url, "?"); idx != -1 {
url = url[:idx]
}
@@ -400,8 +355,8 @@ func isLocalhost(host string) bool {
host = strings.ToLower(host)
return host == "localhost" ||
host == localhostIPv4 ||
host == localhostIPv6 ||
host == "127.0.0.1" ||
host == "::1" ||
host == "[::1]" ||
strings.HasSuffix(host, ".localhost") ||
strings.HasSuffix(host, ".local")
@@ -467,23 +422,23 @@ func ssrfSafeDialer(ctx context.Context, network, addr string) (net.Conn, error)
}
// Check all resolved IPs
if slices.ContainsFunc(ips, isPrivateIP) {
for _, ip := range ips {
if isPrivateIP(ip) {
return nil, ErrSSRFBlocked
}
}
// Connect using the first valid IP
var dialer net.Dialer
for _, ip := range ips {
addr := net.JoinHostPort(ip.String(), port)
conn, err := dialer.DialContext(ctx, network, addr)
if err == nil {
return conn, nil
}
}
return nil, fmt.Errorf("%w to %s", errConnectFailed, host)
return nil, fmt.Errorf("failed to connect to %s", host)
}
// limitedReader wraps a reader and limits the number of bytes read.
@@ -510,7 +465,6 @@ func (r *limitedReader) Read(p []byte) (int, error) {
// semaphoreReleasingReadCloser releases a semaphore slot when closed.
type semaphoreReleasingReadCloser struct {
*limitedReader
closer io.Closer
sem chan struct{}
}

View File

@@ -9,12 +9,7 @@ import (
"testing/fstest"
)
// testHost is the hostname used by mock fetch tests.
const testHost = "example.com"
func TestDefaultConfig(t *testing.T) {
t.Parallel()
cfg := DefaultConfig()
if cfg.Timeout != DefaultFetchTimeout {
@@ -40,8 +35,6 @@ func TestDefaultConfig(t *testing.T) {
}
func TestNewWithNilConfigUsesDefaults(t *testing.T) {
t.Parallel()
f := New(nil)
if f == nil {
@@ -58,28 +51,24 @@ func TestNewWithNilConfigUsesDefaults(t *testing.T) {
}
func TestIsAllowedContentType(t *testing.T) {
t.Parallel()
f := New(DefaultConfig())
tests := []struct {
contentType string
want bool
}{
{contentTypeJPEG, true},
{contentTypePNG, true},
{contentTypeWebP, true},
{"image/jpeg", true},
{"image/png", true},
{"image/webp", true},
{"image/jpeg; charset=utf-8", true},
{"IMAGE/JPEG", true},
{"text/html", false},
{contentTypeOctetStream, false},
{"application/octet-stream", false},
{"", false},
}
for _, tc := range tests {
t.Run(tc.contentType, func(t *testing.T) {
t.Parallel()
got := f.isAllowedContentType(tc.contentType)
if got != tc.want {
t.Errorf("isAllowedContentType(%q) = %v, want %v", tc.contentType, got, tc.want)
@@ -89,24 +78,20 @@ func TestIsAllowedContentType(t *testing.T) {
}
func TestExtractHost(t *testing.T) {
t.Parallel()
tests := []struct {
url string
want string
}{
{"https://example.com/path", testHost},
{"https://example.com/path", "example.com"},
{"http://example.com:8080/path", "example.com:8080"},
{"https://example.com", testHost},
{"https://example.com?q=1", testHost},
{"example.com/path", testHost},
{"https://example.com", "example.com"},
{"https://example.com?q=1", "example.com"},
{"example.com/path", "example.com"},
{"", ""},
}
for _, tc := range tests {
t.Run(tc.url, func(t *testing.T) {
t.Parallel()
got := extractHost(tc.url)
if got != tc.want {
t.Errorf("extractHost(%q) = %q, want %q", tc.url, got, tc.want)
@@ -116,27 +101,23 @@ func TestExtractHost(t *testing.T) {
}
func TestIsLocalhost(t *testing.T) {
t.Parallel()
tests := []struct {
host string
want bool
}{
{"localhost", true},
{"LOCALHOST", true},
{localhostIPv4, true},
{localhostIPv6, true},
{"127.0.0.1", true},
{"::1", true},
{"[::1]", true},
{"foo.localhost", true},
{"foo.local", true},
{testHost, false},
{"example.com", false},
{"127.0.0.2", false}, // Handled by isPrivateIP, not isLocalhost string match
}
for _, tc := range tests {
t.Run(tc.host, func(t *testing.T) {
t.Parallel()
got := isLocalhost(tc.host)
if got != tc.want {
t.Errorf("isLocalhost(%q) = %v, want %v", tc.host, got, tc.want)
@@ -146,20 +127,18 @@ func TestIsLocalhost(t *testing.T) {
}
func TestIsPrivateIP(t *testing.T) {
t.Parallel()
tests := []struct {
ip string
want bool
}{
{localhostIPv4, true}, // loopback
{"127.0.0.1", true}, // loopback
{"10.0.0.1", true}, // private
{"192.168.1.1", true}, // private
{"172.16.0.1", true}, // private
{"169.254.1.1", true}, // link-local
{"0.0.0.0", true}, // unspecified
{"224.0.0.1", true}, // multicast
{localhostIPv6, true}, // IPv6 loopback
{"::1", true}, // IPv6 loopback
{"fe80::1", true}, // IPv6 link-local
{"8.8.8.8", false}, // public
{"2001:4860:4860::8888", false}, // public IPv6
@@ -167,8 +146,6 @@ func TestIsPrivateIP(t *testing.T) {
for _, tc := range tests {
t.Run(tc.ip, func(t *testing.T) {
t.Parallel()
ip := net.ParseIP(tc.ip)
if ip == nil {
t.Fatalf("failed to parse IP %q", tc.ip)
@@ -187,19 +164,15 @@ func TestIsPrivateIP(t *testing.T) {
}
func TestValidateURL_RejectsNonHTTPS(t *testing.T) {
t.Parallel()
err := validateURL(t.Context(), "http://example.com/path", false)
err := validateURL("http://example.com/path", false)
if !errors.Is(err, ErrUnsupportedScheme) {
t.Errorf("validateURL http = %v, want ErrUnsupportedScheme", err)
}
}
func TestValidateURL_AllowsHTTPWhenConfigured(t *testing.T) {
t.Parallel()
// Use a host that won't resolve (explicit .invalid TLD) so we don't hit DNS.
err := validateURL(t.Context(), "http://nonexistent.invalid/path", true)
err := validateURL("http://nonexistent.invalid/path", true)
// We expect a host resolution error, not ErrUnsupportedScheme.
if errors.Is(err, ErrUnsupportedScheme) {
t.Error("validateURL with AllowHTTP should not return ErrUnsupportedScheme")
@@ -207,26 +180,20 @@ func TestValidateURL_AllowsHTTPWhenConfigured(t *testing.T) {
}
func TestValidateURL_RejectsLocalhost(t *testing.T) {
t.Parallel()
err := validateURL(t.Context(), "https://localhost/path", false)
err := validateURL("https://localhost/path", false)
if !errors.Is(err, ErrSSRFBlocked) {
t.Errorf("validateURL localhost = %v, want ErrSSRFBlocked", err)
}
}
func TestValidateURL_EmptyHost(t *testing.T) {
t.Parallel()
err := validateURL(t.Context(), "https:///path", false)
err := validateURL("https:///path", false)
if !errors.Is(err, ErrInvalidHost) {
t.Errorf("validateURL empty host = %v, want ErrInvalidHost", err)
}
}
func TestMockFetcher_FetchesFile(t *testing.T) {
t.Parallel()
mockFS := fstest.MapFS{
"example.com/images/photo.jpg": &fstest.MapFile{Data: []byte("fake-jpeg-data")},
}
@@ -239,7 +206,7 @@ func TestMockFetcher_FetchesFile(t *testing.T) {
}
defer func() { _ = result.Content.Close() }()
if result.ContentType != contentTypeJPEG {
if result.ContentType != "image/jpeg" {
t.Errorf("ContentType = %q, want image/jpeg", result.ContentType)
}
@@ -258,8 +225,6 @@ func TestMockFetcher_FetchesFile(t *testing.T) {
}
func TestMockFetcher_MissingFileReturnsUpstreamError(t *testing.T) {
t.Parallel()
mockFS := fstest.MapFS{}
m := NewMock(mockFS)
@@ -270,8 +235,6 @@ func TestMockFetcher_MissingFileReturnsUpstreamError(t *testing.T) {
}
func TestMockFetcher_RespectsContextCancellation(t *testing.T) {
t.Parallel()
mockFS := fstest.MapFS{
"example.com/photo.jpg": &fstest.MapFile{Data: []byte("data")},
}
@@ -287,28 +250,24 @@ func TestMockFetcher_RespectsContextCancellation(t *testing.T) {
}
func TestDetectContentTypeFromPath(t *testing.T) {
t.Parallel()
tests := []struct {
path string
want string
}{
{"foo/bar.jpg", contentTypeJPEG},
{"foo/bar.JPG", contentTypeJPEG},
{"foo/bar.jpeg", contentTypeJPEG},
{"foo/bar.png", contentTypePNG},
{"foo/bar.gif", contentTypeGIF},
{"foo/bar.webp", contentTypeWebP},
{"foo/bar.avif", contentTypeAVIF},
{"foo/bar.svg", contentTypeSVG},
{"foo/bar.bin", contentTypeOctetStream},
{"foo/bar", contentTypeOctetStream},
{"foo/bar.jpg", "image/jpeg"},
{"foo/bar.JPG", "image/jpeg"},
{"foo/bar.jpeg", "image/jpeg"},
{"foo/bar.png", "image/png"},
{"foo/bar.gif", "image/gif"},
{"foo/bar.webp", "image/webp"},
{"foo/bar.avif", "image/avif"},
{"foo/bar.svg", "image/svg+xml"},
{"foo/bar.bin", "application/octet-stream"},
{"foo/bar", "application/octet-stream"},
}
for _, tc := range tests {
t.Run(tc.path, func(t *testing.T) {
t.Parallel()
got := detectContentTypeFromPath(tc.path)
if got != tc.want {
t.Errorf("detectContentTypeFromPath(%q) = %q, want %q", tc.path, got, tc.want)
@@ -318,8 +277,6 @@ func TestDetectContentTypeFromPath(t *testing.T) {
}
func TestLimitedReader_EnforcesLimit(t *testing.T) {
t.Parallel()
src := make([]byte, 100)
r := &limitedReader{
reader: &byteReader{data: src},
@@ -341,11 +298,10 @@ func TestLimitedReader_EnforcesLimit(t *testing.T) {
total := n
for total < 50 {
nn, err := r.Read(buf)
total += nn
if err != nil {
t.Fatalf("during drain: %v", err)
}
total += nn
}
// Now the limit is exhausted — next read should error.

View File

@@ -4,14 +4,12 @@ import (
"context"
"errors"
"fmt"
"io"
"io/fs"
"net/http"
"strings"
)
// errEmptyURLPath is returned when a mock URL has no usable path.
var errEmptyURLPath = errors.New("empty URL path")
// MockFetcher implements Fetcher using an embedded filesystem.
// Files are organized as: hostname/path/to/file.ext
// URLs like https://example.com/images/photo.jpg map to example.com/images/photo.jpg.
@@ -61,7 +59,7 @@ func (m *MockFetcher) Fetch(ctx context.Context, url string) (*FetchResult, erro
contentType := detectContentTypeFromPath(path)
return &FetchResult{
Content: f,
Content: f.(io.ReadCloser),
ContentLength: stat.Size(),
ContentType: contentType,
Headers: make(http.Header),
@@ -88,7 +86,7 @@ func urlToFSPath(rawURL string) (string, error) {
}
if url == "" {
return "", errEmptyURLPath
return "", errors.New("empty URL path")
}
return url, nil
@@ -100,18 +98,18 @@ func detectContentTypeFromPath(path string) string {
switch {
case strings.HasSuffix(path, ".jpg"), strings.HasSuffix(path, ".jpeg"):
return contentTypeJPEG
return "image/jpeg"
case strings.HasSuffix(path, ".png"):
return contentTypePNG
return "image/png"
case strings.HasSuffix(path, ".gif"):
return contentTypeGIF
return "image/gif"
case strings.HasSuffix(path, ".webp"):
return contentTypeWebP
return "image/webp"
case strings.HasSuffix(path, ".avif"):
return contentTypeAVIF
return "image/avif"
case strings.HasSuffix(path, ".svg"):
return contentTypeSVG
return "image/svg+xml"
default:
return contentTypeOctetStream
return "application/octet-stream"
}
}

View File

@@ -13,9 +13,7 @@ import (
)
// vipsOnce ensures vips is initialized exactly once.
//
//nolint:gochecknoglobals // package-level sync.Once for one-time vips init
var vipsOnce sync.Once
var vipsOnce sync.Once //nolint:gochecknoglobals // package-level sync.Once for one-time vips init
// initVips initializes libvips with quiet logging.
func initVips() {
@@ -98,12 +96,10 @@ const DefaultMaxInputBytes = 50 << 20
// ErrInputTooLarge is returned when input image dimensions exceed MaxInputDimension.
var ErrInputTooLarge = errors.New("input image dimensions exceed maximum")
// ErrInputDataTooLarge is returned when the raw input data exceeds the
// configured byte limit.
// ErrInputDataTooLarge is returned when the raw input data exceeds the configured byte limit.
var ErrInputDataTooLarge = errors.New("input data exceeds maximum allowed size")
// ErrUnsupportedOutputFormat is returned when the requested output format is
// not supported.
// ErrUnsupportedOutputFormat is returned when the requested output format is not supported.
var ErrUnsupportedOutputFormat = errors.New("unsupported output format")
// ImageProcessor implements image transformation using libvips via govips.
@@ -174,12 +170,25 @@ func (p *ImageProcessor) Process(
}
// Determine target dimensions
targetWidth, targetHeight := targetDimensions(req.Size, origWidth, origHeight)
targetWidth := req.Size.Width
targetHeight := req.Size.Height
// Handle dimension calculation
if targetWidth == 0 && targetHeight == 0 {
// Both are 0: keep original size
targetWidth = origWidth
targetHeight = origHeight
} else if targetWidth == 0 {
// Only height specified: calculate width proportionally
targetWidth = origWidth * targetHeight / origHeight
} else if targetHeight == 0 {
// Only width specified: calculate height proportionally
targetHeight = origHeight * targetWidth / origWidth
}
// Resize if needed
if targetWidth != origWidth || targetHeight != origHeight {
err := p.resize(img, targetWidth, targetHeight, req.FitMode)
if err != nil {
if err := p.resize(img, targetWidth, targetHeight, req.FitMode); err != nil {
return nil, fmt.Errorf("failed to resize: %w", err)
}
}
@@ -208,42 +217,14 @@ func (p *ImageProcessor) Process(
}, nil
}
// targetDimensions calculates the output dimensions for a requested size,
// scaling proportionally when only one dimension is given and keeping the
// original dimensions when both are zero.
func targetDimensions(size Size, origWidth, origHeight int) (int, int) {
switch {
case size.Width == 0 && size.Height == 0:
// Both are 0: keep original size
return origWidth, origHeight
case size.Width == 0:
// Only height specified: calculate width proportionally
return origWidth * size.Height / origHeight, size.Height
case size.Height == 0:
// Only width specified: calculate height proportionally
return size.Width, origHeight * size.Width / origWidth
default:
return size.Width, size.Height
}
}
// MIME types for the supported image formats.
const (
mimeJPEG = "image/jpeg"
mimePNG = "image/png"
mimeGIF = "image/gif"
mimeWebP = "image/webp"
mimeAVIF = "image/avif"
)
// SupportedInputFormats returns MIME types this processor can read.
func (p *ImageProcessor) SupportedInputFormats() []string {
return []string{
mimeJPEG,
mimePNG,
mimeGIF,
mimeWebP,
mimeAVIF,
"image/jpeg",
"image/png",
"image/gif",
"image/webp",
"image/avif",
}
}
@@ -262,17 +243,15 @@ func (p *ImageProcessor) SupportedOutputFormats() []Format {
func FormatToMIME(format Format) string {
switch format {
case FormatJPEG:
return mimeJPEG
return "image/jpeg"
case FormatPNG:
return mimePNG
return "image/png"
case FormatWebP:
return mimeWebP
return "image/webp"
case FormatGIF:
return mimeGIF
return "image/gif"
case FormatAVIF:
return mimeAVIF
case FormatOriginal:
return "application/octet-stream"
return "image/avif"
default:
return "application/octet-stream"
}
@@ -291,20 +270,14 @@ func (p *ImageProcessor) detectFormat(img *vips.ImageRef) string {
case vips.ImageTypeWEBP:
return "webp"
case vips.ImageTypeAVIF, vips.ImageTypeHEIF:
return string(FormatAVIF)
case vips.ImageTypeUnknown, vips.ImageTypeMagick, vips.ImageTypePDF,
vips.ImageTypeSVG, vips.ImageTypeTIFF, vips.ImageTypeBMP,
vips.ImageTypeJP2K, vips.ImageTypeJXL:
return "unknown"
return "avif"
default:
return "unknown"
}
}
// resize resizes the image according to the fit mode.
func (p *ImageProcessor) resize(
img *vips.ImageRef, width, height int, fit FitMode,
) error {
func (p *ImageProcessor) resize(img *vips.ImageRef, width, height int, fit FitMode) error {
switch fit {
case FitCover, "":
// Resize and crop to fill exact dimensions (default)
@@ -330,7 +303,6 @@ func (p *ImageProcessor) resize(
if img.Width() <= width && img.Height() <= height {
return nil // Already fits
}
imgW, imgH := img.Width(), img.Height()
scaleW := float64(width) / float64(imgW)
scaleH := float64(height) / float64(imgH)
@@ -359,9 +331,7 @@ func (p *ImageProcessor) resize(
const defaultQuality = 85
// encode encodes an image to the specified format.
func (p *ImageProcessor) encode(
img *vips.ImageRef, format Format, quality int,
) ([]byte, error) {
func (p *ImageProcessor) encode(img *vips.ImageRef, format Format, quality int) ([]byte, error) {
if quality <= 0 {
quality = defaultQuality
}
@@ -397,11 +367,8 @@ func (p *ImageProcessor) encode(
Quality: quality,
}
case FormatOriginal:
return nil, fmt.Errorf("%w: %s", ErrUnsupportedOutputFormat, format)
default:
return nil, fmt.Errorf("%w: %s", ErrUnsupportedOutputFormat, format)
return nil, fmt.Errorf("unsupported output format: %s", format)
}
output, _, err := img.Export(&params)
@@ -423,7 +390,7 @@ func (p *ImageProcessor) formatFromString(format string) Format {
return FormatGIF
case "webp":
return FormatWebP
case string(FormatAVIF):
case "avif":
return FormatAVIF
default:
return FormatJPEG

View File

@@ -3,7 +3,6 @@ package imageprocessor
import (
"bytes"
"context"
"errors"
"image"
"image/color"
"image/jpeg"
@@ -17,9 +16,7 @@ import (
func TestMain(m *testing.M) {
initVips()
code := m.Run()
vips.Shutdown()
os.Exit(code)
}
@@ -30,11 +27,11 @@ func createTestJPEG(t *testing.T, width, height int) []byte {
img := image.NewRGBA(image.Rect(0, 0, width, height))
// Fill with a gradient
for y := range height {
for x := range width {
for y := 0; y < height; y++ {
for x := 0; x < width; x++ {
img.Set(x, y, color.RGBA{
R: uint8((x * 255 / width) & 0xff),
G: uint8((y * 255 / height) & 0xff),
R: uint8(x * 255 / width),
G: uint8(y * 255 / height),
B: 128,
A: 255,
})
@@ -42,9 +39,7 @@ func createTestJPEG(t *testing.T, width, height int) []byte {
}
var buf bytes.Buffer
err := jpeg.Encode(&buf, img, &jpeg.Options{Quality: 90})
if err != nil {
if err := jpeg.Encode(&buf, img, &jpeg.Options{Quality: 90}); err != nil {
t.Fatalf("failed to encode test JPEG: %v", err)
}
@@ -56,11 +51,11 @@ func createTestPNG(t *testing.T, width, height int) []byte {
t.Helper()
img := image.NewRGBA(image.Rect(0, 0, width, height))
for y := range height {
for x := range width {
for y := 0; y < height; y++ {
for x := 0; x < width; x++ {
img.Set(x, y, color.RGBA{
R: uint8((x * 255 / width) & 0xff),
G: uint8((y * 255 / height) & 0xff),
R: uint8(x * 255 / width),
G: uint8(y * 255 / height),
B: 128,
A: 255,
})
@@ -68,54 +63,37 @@ func createTestPNG(t *testing.T, width, height int) []byte {
}
var buf bytes.Buffer
err := png.Encode(&buf, img)
if err != nil {
if err := png.Encode(&buf, img); err != nil {
t.Fatalf("failed to encode test PNG: %v", err)
}
return buf.Bytes()
}
// isAVIF reports whether data starts with an AVIF ftyp box.
func isAVIF(data []byte) bool {
if len(data) < 12 || string(data[4:8]) != "ftyp" {
return false
}
brand := string(data[8:12])
return brand == string(FormatAVIF) || brand == "avis"
}
// detectMIME is a minimal magic-byte detector for test assertions.
func detectMIME(data []byte) string {
if len(data) >= 3 && data[0] == 0xFF && data[1] == 0xD8 && data[2] == 0xFF {
return mimeJPEG
return "image/jpeg"
}
if len(data) >= 8 && string(data[:8]) == "\x89PNG\r\n\x1a\n" {
return mimePNG
return "image/png"
}
if len(data) >= 4 && string(data[:4]) == "GIF8" {
return mimeGIF
return "image/gif"
}
if len(data) >= 12 && string(data[:4]) == "RIFF" && string(data[8:12]) == "WEBP" {
return mimeWebP
return "image/webp"
}
if len(data) >= 12 && string(data[4:8]) == "ftyp" {
brand := string(data[8:12])
if brand == "avif" || brand == "avis" {
return "image/avif"
}
if isAVIF(data) {
return mimeAVIF
}
return ""
}
func TestImageProcessor_ResizeJPEG(t *testing.T) {
t.Parallel()
proc := New(Params{})
ctx := context.Background()
@@ -132,8 +110,7 @@ func TestImageProcessor_ResizeJPEG(t *testing.T) {
if err != nil {
t.Fatalf("Process() error = %v", err)
}
defer func() { _ = result.Content.Close() }()
defer result.Content.Close()
if result.Width != 400 {
t.Errorf("Process() width = %d, want 400", result.Width)
@@ -154,14 +131,12 @@ func TestImageProcessor_ResizeJPEG(t *testing.T) {
}
mime := detectMIME(data)
if mime != mimeJPEG {
if mime != "image/jpeg" {
t.Errorf("Output format = %v, want image/jpeg", mime)
}
}
func TestImageProcessor_ConvertToPNG(t *testing.T) {
t.Parallel()
proc := New(Params{})
ctx := context.Background()
@@ -177,8 +152,7 @@ func TestImageProcessor_ConvertToPNG(t *testing.T) {
if err != nil {
t.Fatalf("Process() error = %v", err)
}
defer func() { _ = result.Content.Close() }()
defer result.Content.Close()
data, err := io.ReadAll(result.Content)
if err != nil {
@@ -186,25 +160,19 @@ func TestImageProcessor_ConvertToPNG(t *testing.T) {
}
mime := detectMIME(data)
if mime != mimePNG {
if mime != "image/png" {
t.Errorf("Output format = %v, want image/png", mime)
}
}
// processAndCheckSize processes a test JPEG of the given input dimensions
// with the requested size and asserts the resulting dimensions.
func processAndCheckSize(
t *testing.T, inputW, inputH int, size Size, wantW, wantH int,
) {
t.Helper()
func TestImageProcessor_OriginalSize(t *testing.T) {
proc := New(Params{})
ctx := context.Background()
input := createTestJPEG(t, inputW, inputH)
input := createTestJPEG(t, 640, 480)
req := &Request{
Size: size,
Size: Size{Width: 0, Height: 0}, // Original size
Format: FormatJPEG,
Quality: 85,
FitMode: FitCover,
@@ -214,28 +182,18 @@ func processAndCheckSize(
if err != nil {
t.Fatalf("Process() error = %v", err)
}
defer result.Content.Close()
defer func() { _ = result.Content.Close() }()
if result.Width != wantW {
t.Errorf("Process() width = %d, want %d", result.Width, wantW)
if result.Width != 640 {
t.Errorf("Process() width = %d, want 640", result.Width)
}
if result.Height != wantH {
t.Errorf("Process() height = %d, want %d", result.Height, wantH)
if result.Height != 480 {
t.Errorf("Process() height = %d, want 480", result.Height)
}
}
func TestImageProcessor_OriginalSize(t *testing.T) {
t.Parallel()
// Width and height 0: keep original size
processAndCheckSize(t, 640, 480, Size{Width: 0, Height: 0}, 640, 480)
}
func TestImageProcessor_FitContain(t *testing.T) {
t.Parallel()
proc := New(Params{})
ctx := context.Background()
@@ -254,8 +212,7 @@ func TestImageProcessor_FitContain(t *testing.T) {
if err != nil {
t.Fatalf("Process() error = %v", err)
}
defer func() { _ = result.Content.Close() }()
defer result.Content.Close()
// With contain, the image should fit within the box
if result.Width > 400 || result.Height > 400 {
@@ -264,24 +221,66 @@ func TestImageProcessor_FitContain(t *testing.T) {
}
func TestImageProcessor_ProportionalScale_WidthOnly(t *testing.T) {
t.Parallel()
proc := New(Params{})
ctx := context.Background()
// 800x600 image, request width=400 height=0
// Should scale proportionally to 400x300
processAndCheckSize(t, 800, 600, Size{Width: 400, Height: 0}, 400, 300)
input := createTestJPEG(t, 800, 600)
req := &Request{
Size: Size{Width: 400, Height: 0},
Format: FormatJPEG,
Quality: 85,
FitMode: FitCover,
}
result, err := proc.Process(ctx, bytes.NewReader(input), req)
if err != nil {
t.Fatalf("Process() error = %v", err)
}
defer result.Content.Close()
if result.Width != 400 {
t.Errorf("Process() width = %d, want 400", result.Width)
}
if result.Height != 300 {
t.Errorf("Process() height = %d, want 300", result.Height)
}
}
func TestImageProcessor_ProportionalScale_HeightOnly(t *testing.T) {
t.Parallel()
proc := New(Params{})
ctx := context.Background()
// 800x600 image, request width=0 height=300
// Should scale proportionally to 400x300
processAndCheckSize(t, 800, 600, Size{Width: 0, Height: 300}, 400, 300)
input := createTestJPEG(t, 800, 600)
req := &Request{
Size: Size{Width: 0, Height: 300},
Format: FormatJPEG,
Quality: 85,
FitMode: FitCover,
}
result, err := proc.Process(ctx, bytes.NewReader(input), req)
if err != nil {
t.Fatalf("Process() error = %v", err)
}
defer result.Content.Close()
if result.Width != 400 {
t.Errorf("Process() width = %d, want 400", result.Width)
}
if result.Height != 300 {
t.Errorf("Process() height = %d, want 300", result.Height)
}
}
func TestImageProcessor_ProcessPNG(t *testing.T) {
t.Parallel()
proc := New(Params{})
ctx := context.Background()
@@ -297,8 +296,7 @@ func TestImageProcessor_ProcessPNG(t *testing.T) {
if err != nil {
t.Fatalf("Process() error = %v", err)
}
defer func() { _ = result.Content.Close() }()
defer result.Content.Close()
if result.Width != 200 {
t.Errorf("Process() width = %d, want 200", result.Width)
@@ -310,8 +308,6 @@ func TestImageProcessor_ProcessPNG(t *testing.T) {
}
func TestImageProcessor_SupportedFormats(t *testing.T) {
t.Parallel()
proc := New(Params{})
inputFormats := proc.SupportedInputFormats()
@@ -326,26 +322,12 @@ func TestImageProcessor_SupportedFormats(t *testing.T) {
}
func TestImageProcessor_RejectsOversizedInput(t *testing.T) {
t.Parallel()
// Images exceeding MaxInputDimension in either dimension must be
// rejected before processing to prevent DoS.
tests := []struct {
name string
width int
height int
}{
{name: "oversized width", width: 10000, height: 100},
{name: "oversized height", width: 100, height: 10000},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
proc := New(Params{})
ctx := context.Background()
input := createTestJPEG(t, tt.width, tt.height)
// Create an image that exceeds MaxInputDimension (e.g., 10000x100)
// This should be rejected before processing to prevent DoS
input := createTestJPEG(t, 10000, 100)
req := &Request{
Size: Size{Width: 100, Height: 100},
@@ -359,16 +341,36 @@ func TestImageProcessor_RejectsOversizedInput(t *testing.T) {
t.Error("Process() should reject oversized input images")
}
if !errors.Is(err, ErrInputTooLarge) {
if err != ErrInputTooLarge {
t.Errorf("Process() error = %v, want ErrInputTooLarge", err)
}
})
}
func TestImageProcessor_RejectsOversizedInputHeight(t *testing.T) {
proc := New(Params{})
ctx := context.Background()
// Create an image with oversized height
input := createTestJPEG(t, 100, 10000)
req := &Request{
Size: Size{Width: 100, Height: 100},
Format: FormatJPEG,
Quality: 85,
FitMode: FitCover,
}
_, err := proc.Process(ctx, bytes.NewReader(input), req)
if err == nil {
t.Error("Process() should reject oversized input images")
}
if err != ErrInputTooLarge {
t.Errorf("Process() error = %v, want ErrInputTooLarge", err)
}
}
func TestImageProcessor_AcceptsMaxDimensionInput(t *testing.T) {
t.Parallel()
proc := New(Params{})
ctx := context.Background()
@@ -384,20 +386,12 @@ func TestImageProcessor_AcceptsMaxDimensionInput(t *testing.T) {
result, err := proc.Process(ctx, bytes.NewReader(input), req)
if err != nil {
t.Fatalf(
"Process() should accept images at MaxInputDimension, got error: %v",
err,
)
t.Fatalf("Process() should accept images at MaxInputDimension, got error: %v", err)
}
defer func() { _ = result.Content.Close() }()
defer result.Content.Close()
}
// encodeAndCheck processes a 200x150 test JPEG into a 100x75 output of the
// given format and asserts the output MIME type and dimensions.
func encodeAndCheck(t *testing.T, format Format, quality int, wantMIME string) {
t.Helper()
func TestImageProcessor_EncodeWebP(t *testing.T) {
proc := New(Params{})
ctx := context.Background()
@@ -405,8 +399,8 @@ func encodeAndCheck(t *testing.T, format Format, quality int, wantMIME string) {
req := &Request{
Size: Size{Width: 100, Height: 75},
Format: format,
Quality: quality,
Format: FormatWebP,
Quality: 80,
FitMode: FitCover,
}
@@ -414,39 +408,29 @@ func encodeAndCheck(t *testing.T, format Format, quality int, wantMIME string) {
if err != nil {
t.Fatalf("Process() error = %v, want nil", err)
}
defer result.Content.Close()
defer func() { _ = result.Content.Close() }()
// Verify output format
// Verify output is valid WebP
data, err := io.ReadAll(result.Content)
if err != nil {
t.Fatalf("failed to read result: %v", err)
}
mime := detectMIME(data)
if mime != wantMIME {
t.Errorf("Output format = %v, want %v", mime, wantMIME)
if mime != "image/webp" {
t.Errorf("Output format = %v, want image/webp", mime)
}
// Verify dimensions
if result.Width != 100 {
t.Errorf("Width = %d, want 100", result.Width)
}
if result.Height != 75 {
t.Errorf("Height = %d, want 75", result.Height)
}
}
func TestImageProcessor_EncodeWebP(t *testing.T) {
t.Parallel()
encodeAndCheck(t, FormatWebP, 80, mimeWebP)
}
func TestImageProcessor_DecodeAVIF(t *testing.T) {
t.Parallel()
proc := New(Params{})
ctx := context.Background()
@@ -468,8 +452,7 @@ func TestImageProcessor_DecodeAVIF(t *testing.T) {
if err != nil {
t.Fatalf("Process() error = %v, want nil (AVIF decoding should work)", err)
}
defer func() { _ = result.Content.Close() }()
defer result.Content.Close()
// Verify output is valid JPEG
data, err := io.ReadAll(result.Content)
@@ -478,17 +461,14 @@ func TestImageProcessor_DecodeAVIF(t *testing.T) {
}
mime := detectMIME(data)
if mime != mimeJPEG {
if mime != "image/jpeg" {
t.Errorf("Output format = %v, want image/jpeg", mime)
}
}
func TestImageProcessor_RejectsOversizedInputData(t *testing.T) {
t.Parallel()
// Create a processor with a very small byte limit
const limit = 1024
proc := New(Params{MaxInputBytes: limit})
ctx := context.Background()
@@ -510,14 +490,12 @@ func TestImageProcessor_RejectsOversizedInputData(t *testing.T) {
t.Fatal("Process() should reject input exceeding maxInputBytes")
}
if !errors.Is(err, ErrInputDataTooLarge) {
if err != ErrInputDataTooLarge {
t.Errorf("Process() error = %v, want ErrInputDataTooLarge", err)
}
}
func TestImageProcessor_AcceptsInputWithinLimit(t *testing.T) {
t.Parallel()
// Create a small image and set limit well above its size
input := createTestJPEG(t, 10, 10)
limit := int64(len(input)) * 10 // 10× headroom
@@ -536,13 +514,10 @@ func TestImageProcessor_AcceptsInputWithinLimit(t *testing.T) {
if err != nil {
t.Fatalf("Process() error = %v, want nil", err)
}
defer func() { _ = result.Content.Close() }()
defer result.Content.Close()
}
func TestImageProcessor_DefaultMaxInputBytes(t *testing.T) {
t.Parallel()
// Passing 0 should use the default
proc := New(Params{})
if proc.maxInputBytes != DefaultMaxInputBytes {
@@ -557,7 +532,40 @@ func TestImageProcessor_DefaultMaxInputBytes(t *testing.T) {
}
func TestImageProcessor_EncodeAVIF(t *testing.T) {
t.Parallel()
proc := New(Params{})
ctx := context.Background()
encodeAndCheck(t, FormatAVIF, 85, mimeAVIF)
input := createTestJPEG(t, 200, 150)
req := &Request{
Size: Size{Width: 100, Height: 75},
Format: FormatAVIF,
Quality: 85,
FitMode: FitCover,
}
result, err := proc.Process(ctx, bytes.NewReader(input), req)
if err != nil {
t.Fatalf("Process() error = %v, want nil (AVIF encoding should work)", err)
}
defer result.Content.Close()
// Verify output is valid AVIF
data, err := io.ReadAll(result.Content)
if err != nil {
t.Fatalf("failed to read result: %v", err)
}
mime := detectMIME(data)
if mime != "image/avif" {
t.Errorf("Output format = %v, want image/avif", mime)
}
// Verify dimensions
if result.Width != 100 {
t.Errorf("Width = %d, want 100", result.Width)
}
if result.Height != 75 {
t.Errorf("Height = %d, want 75", result.Height)
}
}

View File

@@ -7,7 +7,9 @@ import (
"errors"
"fmt"
"io"
"log/slog"
"path/filepath"
"sync"
"time"
"sneak.berlin/go/pixa/internal/httpfetcher"
@@ -27,6 +29,22 @@ type CacheConfig struct {
StateDir string
CacheTTL time.Duration
NegativeTTL time.Duration
// MaxBytes is the disk cache size limit in bytes that eviction
// enforces. Zero means no limit is enforced (no eviction). The
// config layer supplies the computed default when the operator
// omits cache_max_bytes.
MaxBytes int64
// DisableDiskCache turns the disk cache off entirely: no cache
// directories are created, lookups always miss, stores are
// no-ops, and no eviction machinery runs. The config layer sets
// this when the operator configures cache_max_bytes: 0.
DisableDiskCache bool
// Logger receives accounting and eviction log output. A nil
// Logger means slog.Default().
Logger *slog.Logger
}
// variantMeta stores content type for fast cache hits without reading .meta file.
@@ -42,43 +60,66 @@ type Cache struct {
variants *VariantStorage // processed variants by cache key
srcMetadata *MetadataStorage // source metadata by host/path
config CacheConfig
log *slog.Logger
// In-memory cache of variant metadata (content type, size) to avoid
// reading .meta files
// disabled means the disk cache is turned off entirely: lookups
// always miss, stores are no-ops, and no eviction runs.
disabled bool
// Eviction machinery. The channels are created in NewCache so
// stores can signal write pressure without racing StartEviction.
evictionPressure chan struct{}
evictionStop chan struct{}
evictionDone chan struct{}
evictionStarted bool
evictionStopOnce sync.Once
// In-memory cache of variant metadata (content type, size) to avoid reading .meta files
metaCache map[VariantKey]variantMeta
}
// NewCache creates a new cache instance.
func NewCache(db *sql.DB, config CacheConfig) (*Cache, error) {
srcContent, err := NewContentStorage(
filepath.Join(config.StateDir, "cache", "sources"),
)
log := config.Logger
if log == nil {
log = slog.Default()
}
c := &Cache{
db: db,
config: config,
log: log,
disabled: config.DisableDiskCache,
evictionPressure: make(chan struct{}, 1),
evictionStop: make(chan struct{}),
evictionDone: make(chan struct{}),
metaCache: make(map[VariantKey]variantMeta),
}
if c.disabled {
return c, nil
}
srcContent, err := NewContentStorage(filepath.Join(config.StateDir, "cache", "sources"))
if err != nil {
return nil, fmt.Errorf("failed to create source content storage: %w", err)
}
variants, err := NewVariantStorage(
filepath.Join(config.StateDir, "cache", "variants"),
)
variants, err := NewVariantStorage(filepath.Join(config.StateDir, "cache", "variants"))
if err != nil {
return nil, fmt.Errorf("failed to create variant storage: %w", err)
}
srcMetadata, err := NewMetadataStorage(
filepath.Join(config.StateDir, "cache", "metadata"),
)
srcMetadata, err := NewMetadataStorage(filepath.Join(config.StateDir, "cache", "metadata"))
if err != nil {
return nil, fmt.Errorf("failed to create source metadata storage: %w", err)
}
return &Cache{
db: db,
srcContent: srcContent,
variants: variants,
srcMetadata: srcMetadata,
config: config,
metaCache: make(map[VariantKey]variantMeta),
}, nil
c.srcContent = srcContent
c.variants = variants
c.srcMetadata = srcMetadata
return c, nil
}
// LookupResult contains the result of a cache lookup.
@@ -90,12 +131,15 @@ type LookupResult struct {
CacheStatus CacheStatus
}
// Lookup checks if a processed variant exists on disk (no DB access for hits).
func (c *Cache) Lookup(_ context.Context, req *ImageRequest) (*LookupResult, error) {
// Lookup checks if a processed variant exists on disk. Hits touch the
// variant's LRU timestamp; a disabled cache always misses.
func (c *Cache) Lookup(ctx context.Context, req *ImageRequest) (*LookupResult, error) {
cacheKey := CacheKey(req)
// Check variant storage directly - no DB needed for cache hits
if c.variants.Exists(cacheKey) {
if !c.disabled && c.variants.Exists(cacheKey) {
c.touchVariant(ctx, cacheKey)
return &LookupResult{
Hit: true,
CacheKey: cacheKey,
@@ -110,18 +154,53 @@ func (c *Cache) Lookup(_ context.Context, req *ImageRequest) (*LookupResult, err
}, nil
}
// touchVariant updates the LRU timestamp of a variant, best-effort:
// a failed touch only makes the entry look colder to eviction.
func (c *Cache) touchVariant(ctx context.Context, cacheKey VariantKey) {
_, err := c.db.ExecContext(ctx, `
UPDATE variant_content SET last_accessed_at = CURRENT_TIMESTAMP
WHERE cache_key = ?
`, string(cacheKey))
if err != nil {
c.log.Debug("failed to touch variant LRU timestamp",
"cache_key", cacheKey, "error", err)
}
}
// touchSourceContent updates the LRU timestamp of a source content
// blob, best-effort: a failed touch only makes the blob look colder.
func (c *Cache) touchSourceContent(ctx context.Context, contentHash ContentHash) {
_, err := c.db.ExecContext(ctx, `
UPDATE source_content SET last_accessed_at = CURRENT_TIMESTAMP
WHERE content_hash = ?
`, string(contentHash))
if err != nil {
c.log.Debug("failed to touch source content LRU timestamp",
"content_hash", contentHash, "error", err)
}
}
// GetVariant returns a reader, size, and content type for a cached variant.
func (c *Cache) GetVariant(cacheKey VariantKey) (io.ReadCloser, int64, string, error) {
if c.disabled {
return nil, 0, "", ErrNotFound
}
return c.variants.LoadWithMeta(cacheKey)
}
// StoreSource stores fetched source content and metadata.
// StoreSource stores fetched source content and metadata. On a
// disabled cache it is a no-op returning an empty hash.
func (c *Cache) StoreSource(
ctx context.Context,
req *ImageRequest,
content io.Reader,
result *httpfetcher.FetchResult,
) (ContentHash, error) {
if c.disabled {
return "", nil
}
// Store content
contentHash, size, err := c.srcContent.Store(content)
if err != nil {
@@ -130,11 +209,7 @@ func (c *Cache) StoreSource(
// Store in database
pathHash := HashPath(req.SourcePath + "?" + req.SourceQuery)
headersJSON, err := json.Marshal(result.Headers)
if err != nil {
return "", fmt.Errorf("failed to marshal response headers: %w", err)
}
headersJSON, _ := json.Marshal(result.Headers)
_, err = c.db.ExecContext(ctx, `
INSERT INTO source_content (content_hash, content_type, size_bytes)
@@ -177,26 +252,57 @@ func (c *Cache) StoreSource(
RemoteAddr: result.RemoteAddr,
}
// A failure here is non-fatal; the metadata is in the database.
_ = c.srcMetadata.Store(req.SourceHost, pathHash, meta)
if err := c.srcMetadata.Store(req.SourceHost, pathHash, meta); err != nil {
// Non-fatal, we have it in the database
_ = err
}
c.notifyWritePressure()
return contentHash, nil
}
// StoreVariant stores a processed variant by its cache key.
func (c *Cache) StoreVariant(
cacheKey VariantKey, content io.Reader, contentType string,
) error {
_, err := c.variants.Store(cacheKey, content, contentType)
// StoreVariant stores a processed variant by its cache key and records
// it in the size accounting. On a disabled cache it is a no-op. The
// accounting insert is best-effort (the startup reconciliation pass
// adopts any variant file that misses its accounting row).
func (c *Cache) StoreVariant(cacheKey VariantKey, content io.Reader, contentType string) error {
if c.disabled {
return nil
}
size, err := c.variants.Store(cacheKey, content, contentType)
if err != nil {
return err
}
_, err = c.db.Exec(`
INSERT INTO variant_content (cache_key, size_bytes, content_type)
VALUES (?, ?, ?)
ON CONFLICT(cache_key) DO UPDATE SET
size_bytes = excluded.size_bytes,
content_type = excluded.content_type,
last_accessed_at = CURRENT_TIMESTAMP
`, string(cacheKey), size, contentType)
if err != nil {
c.log.Warn("failed to record variant in size accounting",
"cache_key", cacheKey, "error", err)
}
c.notifyWritePressure()
return nil
}
// LookupSource checks if we have cached source content for a request.
// Returns the content hash and content type if found, or empty values if not.
func (c *Cache) LookupSource(
ctx context.Context, req *ImageRequest,
) (ContentHash, string, error) {
// Returns the content hash and content type if found, or empty values
// if not. Hits touch the blob's LRU timestamp; a disabled cache always
// reports no cached source.
func (c *Cache) LookupSource(ctx context.Context, req *ImageRequest) (ContentHash, string, error) {
if c.disabled {
return "", "", nil
}
var hashStr, contentType string
err := c.db.QueryRowContext(ctx, `
@@ -219,19 +325,17 @@ func (c *Cache) LookupSource(
return "", "", nil
}
c.touchSourceContent(ctx, contentHash)
return contentHash, contentType, nil
}
// StoreNegative stores a negative cache entry for a failed fetch.
func (c *Cache) StoreNegative(
ctx context.Context, req *ImageRequest, statusCode int, errMsg string,
) error {
func (c *Cache) StoreNegative(ctx context.Context, req *ImageRequest, statusCode int, errMsg string) error {
expiresAt := time.Now().UTC().Add(c.config.NegativeTTL)
_, err := c.db.ExecContext(ctx, `
INSERT INTO negative_cache
(source_host, source_path, source_query, status_code,
error_message, expires_at)
INSERT INTO negative_cache (source_host, source_path, source_query, status_code, error_message, expires_at)
VALUES (?, ?, ?, ?, ?, ?)
ON CONFLICT(source_host, source_path, source_query) DO UPDATE SET
status_code = excluded.status_code,
@@ -246,16 +350,46 @@ func (c *Cache) StoreNegative(
return nil
}
// checkNegativeCache checks if a request is in the negative cache.
func (c *Cache) checkNegativeCache(ctx context.Context, req *ImageRequest) (bool, error) {
var expiresAt time.Time
err := c.db.QueryRowContext(ctx, `
SELECT expires_at FROM negative_cache
WHERE source_host = ? AND source_path = ? AND source_query = ?
`, req.SourceHost, req.SourcePath, req.SourceQuery).Scan(&expiresAt)
if errors.Is(err, sql.ErrNoRows) {
return false, nil
}
if err != nil {
return false, fmt.Errorf("failed to check negative cache: %w", err)
}
// Check if expired
if time.Now().After(expiresAt) {
// Clean up expired entry
_, _ = c.db.ExecContext(ctx, `
DELETE FROM negative_cache
WHERE source_host = ? AND source_path = ? AND source_query = ?
`, req.SourceHost, req.SourcePath, req.SourceQuery)
return false, nil
}
return true, nil
}
// GetSourceMetadataID returns the source metadata ID for a request.
func (c *Cache) GetSourceMetadataID(
ctx context.Context, req *ImageRequest,
) (int64, error) {
func (c *Cache) GetSourceMetadataID(ctx context.Context, req *ImageRequest) (int64, error) {
var id int64
err := c.db.QueryRowContext(ctx, `
SELECT id FROM source_metadata
WHERE source_host = ? AND source_path = ? AND source_query = ?
`, req.SourceHost, req.SourcePath, req.SourceQuery).Scan(&id)
if err != nil {
return 0, fmt.Errorf("failed to get source metadata ID: %w", err)
}
@@ -265,6 +399,10 @@ func (c *Cache) GetSourceMetadataID(
// GetSourceContent returns a reader for cached source content by its hash.
func (c *Cache) GetSourceContent(contentHash ContentHash) (io.ReadCloser, error) {
if c.disabled {
return nil, ErrNotFound
}
return c.srcContent.Load(contentHash)
}
@@ -296,12 +434,8 @@ func (c *Cache) Stats(ctx context.Context) (*CacheStats, error) {
}
// Get actual item count and total size from content tables
_ = c.db.QueryRowContext(ctx,
`SELECT COUNT(*) FROM request_cache`,
).Scan(&stats.TotalItems)
_ = c.db.QueryRowContext(ctx,
`SELECT COALESCE(SUM(size_bytes), 0) FROM output_content`,
).Scan(&stats.TotalSizeBytes)
_ = c.db.QueryRowContext(ctx, `SELECT COUNT(*) FROM request_cache`).Scan(&stats.TotalItems)
_ = c.db.QueryRowContext(ctx, `SELECT COALESCE(SUM(size_bytes), 0) FROM output_content`).Scan(&stats.TotalSizeBytes)
// Compute hit rate as a ratio
if stats.HitCount+stats.MissCount > 0 {
@@ -315,17 +449,11 @@ func (c *Cache) Stats(ctx context.Context) (*CacheStats, error) {
func (c *Cache) IncrementStats(ctx context.Context, hit bool, fetchBytes int64) {
if hit {
_, _ = c.db.ExecContext(ctx, `
UPDATE cache_stats
SET hit_count = hit_count + 1,
last_updated_at = CURRENT_TIMESTAMP
WHERE id = 1
UPDATE cache_stats SET hit_count = hit_count + 1, last_updated_at = CURRENT_TIMESTAMP WHERE id = 1
`)
} else {
_, _ = c.db.ExecContext(ctx, `
UPDATE cache_stats
SET miss_count = miss_count + 1,
last_updated_at = CURRENT_TIMESTAMP
WHERE id = 1
UPDATE cache_stats SET miss_count = miss_count + 1, last_updated_at = CURRENT_TIMESTAMP WHERE id = 1
`)
}
@@ -339,36 +467,3 @@ func (c *Cache) IncrementStats(ctx context.Context, hit bool, fetchBytes int64)
`, fetchBytes)
}
}
// checkNegativeCache checks if a request is in the negative cache.
func (c *Cache) checkNegativeCache(
ctx context.Context, req *ImageRequest,
) (bool, error) {
var expiresAt time.Time
err := c.db.QueryRowContext(ctx, `
SELECT expires_at FROM negative_cache
WHERE source_host = ? AND source_path = ? AND source_query = ?
`, req.SourceHost, req.SourcePath, req.SourceQuery).Scan(&expiresAt)
if errors.Is(err, sql.ErrNoRows) {
return false, nil
}
if err != nil {
return false, fmt.Errorf("failed to check negative cache: %w", err)
}
// Check if expired
if time.Now().After(expiresAt) {
// Clean up expired entry
_, _ = c.db.ExecContext(ctx, `
DELETE FROM negative_cache
WHERE source_host = ? AND source_path = ? AND source_query = ?
`, req.SourceHost, req.SourcePath, req.SourceQuery)
return false, nil
}
return true, nil
}

View File

@@ -86,15 +86,14 @@ func setupTestDB(t *testing.T) *sql.DB {
INSERT INTO cache_stats (id) VALUES (1);
`
_, err = db.ExecContext(t.Context(), schema)
if err != nil {
if _, err := db.Exec(schema); err != nil {
t.Fatalf("failed to create schema: %v", err)
}
return db
}
func setupTestCache(t *testing.T) *Cache {
func setupTestCache(t *testing.T) (*Cache, string) {
t.Helper()
tmpDir := t.TempDir()
@@ -109,18 +108,16 @@ func setupTestCache(t *testing.T) *Cache {
t.Fatalf("failed to create cache: %v", err)
}
return cache
return cache, tmpDir
}
func TestCache_LookupMiss(t *testing.T) {
t.Parallel()
cache := setupTestCache(t)
cache, _ := setupTestCache(t)
ctx := context.Background()
req := &ImageRequest{
SourceHost: testHostCDN,
SourcePath: testPathCat,
SourceHost: "cdn.example.com",
SourcePath: "/photos/cat.jpg",
Size: Size{Width: 800, Height: 600},
Format: FormatWebP,
Quality: 85,
@@ -142,14 +139,12 @@ func TestCache_LookupMiss(t *testing.T) {
}
func TestCache_StoreAndLookup(t *testing.T) {
t.Parallel()
cache := setupTestCache(t)
cache, _ := setupTestCache(t)
ctx := context.Background()
req := &ImageRequest{
SourceHost: testHostCDN,
SourcePath: testPathCat,
SourceHost: "cdn.example.com",
SourcePath: "/photos/cat.jpg",
Size: Size{Width: 800, Height: 600},
Format: FormatWebP,
Quality: 85,
@@ -159,12 +154,11 @@ func TestCache_StoreAndLookup(t *testing.T) {
// Store source content
sourceContent := []byte("fake jpeg data")
fetchResult := &httpfetcher.FetchResult{
ContentType: testContentTypeJPEG,
Headers: map[string][]string{"Content-Type": {testContentTypeJPEG}},
ContentType: "image/jpeg",
Headers: map[string][]string{"Content-Type": {"image/jpeg"}},
}
contentHash, err := cache.StoreSource(
ctx, req, bytes.NewReader(sourceContent), fetchResult)
contentHash, err := cache.StoreSource(ctx, req, bytes.NewReader(sourceContent), fetchResult)
if err != nil {
t.Fatalf("StoreSource() error = %v", err)
}
@@ -176,7 +170,6 @@ func TestCache_StoreAndLookup(t *testing.T) {
// Store variant
cacheKey := CacheKey(req)
outputContent := []byte("fake webp data")
err = cache.StoreVariant(cacheKey, bytes.NewReader(outputContent), "image/webp")
if err != nil {
t.Fatalf("StoreVariant() error = %v", err)
@@ -202,13 +195,11 @@ func TestCache_StoreAndLookup(t *testing.T) {
}
func TestCache_NegativeCache(t *testing.T) {
t.Parallel()
cache := setupTestCache(t)
cache, _ := setupTestCache(t)
ctx := context.Background()
req := &ImageRequest{
SourceHost: testHostCDN,
SourceHost: "cdn.example.com",
SourcePath: "/photos/notfound.jpg",
Size: Size{Width: 800, Height: 600},
Format: FormatWebP,
@@ -232,8 +223,6 @@ func TestCache_NegativeCache(t *testing.T) {
}
func TestCache_NegativeCacheExpiry(t *testing.T) {
t.Parallel()
tmpDir := t.TempDir()
db := setupTestDB(t)
@@ -250,7 +239,7 @@ func TestCache_NegativeCacheExpiry(t *testing.T) {
ctx := context.Background()
req := &ImageRequest{
SourceHost: testHostCDN,
SourceHost: "cdn.example.com",
SourcePath: "/photos/expired.jpg",
Size: Size{Width: 800, Height: 600},
Format: FormatWebP,
@@ -277,13 +266,11 @@ func TestCache_NegativeCacheExpiry(t *testing.T) {
}
func TestCache_VariantLookup(t *testing.T) {
t.Parallel()
cache := setupTestCache(t)
cache, _ := setupTestCache(t)
ctx := context.Background()
req := &ImageRequest{
SourceHost: testHostCDN,
SourceHost: "cdn.example.com",
SourcePath: "/photos/variant.jpg",
Size: Size{Width: 800, Height: 600},
Format: FormatWebP,
@@ -294,7 +281,6 @@ func TestCache_VariantLookup(t *testing.T) {
// Store variant
cacheKey := CacheKey(req)
outputContent := []byte("output data")
err := cache.StoreVariant(cacheKey, bytes.NewReader(outputContent), "image/webp")
if err != nil {
t.Fatalf("StoreVariant() error = %v", err)
@@ -326,13 +312,11 @@ func TestCache_VariantLookup(t *testing.T) {
}
func TestCache_GetVariant_ReturnsContentType(t *testing.T) {
t.Parallel()
cache := setupTestCache(t)
cache, _ := setupTestCache(t)
ctx := context.Background()
req := &ImageRequest{
SourceHost: testHostCDN,
SourceHost: "cdn.example.com",
SourcePath: "/photos/variantct.jpg",
Size: Size{Width: 800, Height: 600},
Format: FormatWebP,
@@ -343,7 +327,6 @@ func TestCache_GetVariant_ReturnsContentType(t *testing.T) {
// Store variant
cacheKey := CacheKey(req)
outputContent := []byte("output webp data")
err := cache.StoreVariant(cacheKey, bytes.NewReader(outputContent), "image/webp")
if err != nil {
t.Fatalf("StoreVariant() error = %v", err)
@@ -364,8 +347,7 @@ func TestCache_GetVariant_ReturnsContentType(t *testing.T) {
if err != nil {
t.Fatalf("GetVariant() error = %v", err)
}
defer func() { _ = reader.Close() }()
defer reader.Close()
if contentType != "image/webp" {
t.Errorf("GetVariant() ContentType = %q, want %q", contentType, "image/webp")
@@ -377,13 +359,11 @@ func TestCache_GetVariant_ReturnsContentType(t *testing.T) {
}
func TestCache_GetVariant(t *testing.T) {
t.Parallel()
cache := setupTestCache(t)
cache, _ := setupTestCache(t)
ctx := context.Background()
req := &ImageRequest{
SourceHost: testHostCDN,
SourceHost: "cdn.example.com",
SourcePath: "/photos/output.jpg",
Size: Size{Width: 800, Height: 600},
Format: FormatWebP,
@@ -394,7 +374,6 @@ func TestCache_GetVariant(t *testing.T) {
// Store variant
cacheKey := CacheKey(req)
outputContent := []byte("the actual output content")
err := cache.StoreVariant(cacheKey, bytes.NewReader(outputContent), "image/webp")
if err != nil {
t.Fatalf("StoreVariant() error = %v", err)
@@ -411,8 +390,7 @@ func TestCache_GetVariant(t *testing.T) {
if err != nil {
t.Fatalf("GetVariant() error = %v", err)
}
defer func() { _ = reader.Close() }()
defer reader.Close()
buf := make([]byte, 100)
n, _ := reader.Read(buf)
@@ -423,9 +401,7 @@ func TestCache_GetVariant(t *testing.T) {
}
func TestCache_Stats(t *testing.T) {
t.Parallel()
cache := setupTestCache(t)
cache, _ := setupTestCache(t)
ctx := context.Background()
// Increment some stats
@@ -448,8 +424,6 @@ func TestCache_Stats(t *testing.T) {
}
func TestCache_CleanExpired(t *testing.T) {
t.Parallel()
tmpDir := t.TempDir()
db := setupTestDB(t)
@@ -462,8 +436,7 @@ func TestCache_CleanExpired(t *testing.T) {
// Insert expired negative cache entry directly
_, err := db.ExecContext(ctx, `
INSERT INTO negative_cache
(source_host, source_path, source_query, status_code, expires_at)
INSERT INTO negative_cache (source_host, source_path, source_query, status_code, expires_at)
VALUES ('example.com', '/old.jpg', '', 404, datetime('now', '-1 hour'))
`)
if err != nil {
@@ -472,12 +445,7 @@ func TestCache_CleanExpired(t *testing.T) {
// Verify it exists
var count int
err = db.QueryRowContext(ctx, `SELECT COUNT(*) FROM negative_cache`).Scan(&count)
if err != nil {
t.Fatalf("failed to count negative cache entries: %v", err)
}
db.QueryRowContext(ctx, `SELECT COUNT(*) FROM negative_cache`).Scan(&count)
if count != 1 {
t.Fatalf("expected 1 negative cache entry, got %d", count)
}
@@ -489,19 +457,13 @@ func TestCache_CleanExpired(t *testing.T) {
}
// Verify it's gone
err = db.QueryRowContext(ctx, `SELECT COUNT(*) FROM negative_cache`).Scan(&count)
if err != nil {
t.Fatalf("failed to count negative cache entries: %v", err)
}
db.QueryRowContext(ctx, `SELECT COUNT(*) FROM negative_cache`).Scan(&count)
if count != 0 {
t.Errorf("expected 0 negative cache entries after clean, got %d", count)
}
}
func TestCache_StorageDirectoriesCreated(t *testing.T) {
t.Parallel()
tmpDir := t.TempDir()
db := setupTestDB(t)
@@ -521,9 +483,7 @@ func TestCache_StorageDirectoriesCreated(t *testing.T) {
for _, dir := range dirs {
path := tmpDir + "/" + dir
_, err := os.Stat(path)
if os.IsNotExist(err) {
if _, err := os.Stat(path); os.IsNotExist(err) {
t.Errorf("directory %s was not created", dir)
}
}

View File

@@ -7,8 +7,6 @@ import (
)
func TestSizePercentSafeWithZeroFetchBytes(t *testing.T) {
t.Parallel()
// Simulate the calculation from processAndStore
fetchBytes := int64(0)
outputSize := int64(100)
@@ -31,8 +29,6 @@ func TestSizePercentSafeWithZeroFetchBytes(t *testing.T) {
}
func TestSizePercentNormalCase(t *testing.T) {
t.Parallel()
fetchBytes := int64(1000)
outputSize := int64(500)

View File

@@ -0,0 +1,741 @@
package imgcache
import (
"context"
"encoding/json"
"fmt"
"io/fs"
"os"
"path/filepath"
"strings"
"time"
)
// DefaultEvictionInterval is how often the background evictor checks
// cache usage against the configured limit, in addition to the
// write-pressure wakeups triggered by stores.
const DefaultEvictionInterval = 5 * time.Minute
// evictionBatchSize is how many LRU candidates of each class (variants
// and source blobs) one eviction pass fetches from the database.
const evictionBatchSize = 100
// staleTempFileAge is how old an orphaned temp file (left behind by a
// crashed write) must be before reconciliation removes it. Fresh temp
// files may still belong to an in-flight store.
const staleTempFileAge = time.Hour
// sqliteTimestampLayout matches SQLite's CURRENT_TIMESTAMP format, so
// timestamps written by reconciliation order correctly against ones
// written by the hot path.
const sqliteTimestampLayout = "2006-01-02 15:04:05"
// tempFilePrefix is the prefix os.CreateTemp uses for in-flight cache
// writes (".tmp-*" patterns in the storage layer).
const tempFilePrefix = ".tmp-"
// variantMetaSuffix is the sidecar suffix VariantStorage writes next
// to each variant file.
const variantMetaSuffix = ".meta"
// fallbackContentType is recorded when a reconciled variant file has
// no readable .meta sidecar.
const fallbackContentType = "application/octet-stream"
// UsageBytes returns the total number of bytes of cache content
// tracked in the database (source content blobs plus processed
// variants). It never scans the cache directories.
func (c *Cache) UsageBytes(ctx context.Context) (int64, error) {
if c.disabled {
return 0, nil
}
var total int64
err := c.db.QueryRowContext(ctx, `
SELECT (SELECT COALESCE(SUM(size_bytes), 0) FROM source_content)
+ (SELECT COALESCE(SUM(size_bytes), 0) FROM variant_content)
`).Scan(&total)
if err != nil {
return 0, fmt.Errorf("failed to compute cache usage: %w", err)
}
return total, nil
}
// evictionCandidate is one LRU eviction victim candidate: either a
// processed variant (isVariant true, identified by cacheKey) or a
// source content blob (identified by contentHash).
type evictionCandidate struct {
isVariant bool
cacheKey VariantKey
contentHash ContentHash
sizeBytes int64
lastAccessedAt string
}
// EvictToLimit evicts least-recently-used cache entries until total
// tracked usage is at or below the configured MaxBytes limit. It is a
// no-op when the cache is disabled or no limit is configured.
func (c *Cache) EvictToLimit(ctx context.Context) error {
if c.disabled || c.config.MaxBytes <= 0 {
return nil
}
for {
usage, err := c.UsageBytes(ctx)
if err != nil {
return err
}
if usage <= c.config.MaxBytes {
return nil
}
freed, err := c.evictBatch(ctx, usage-c.config.MaxBytes)
if err != nil {
return err
}
if freed == 0 {
c.log.Warn("cache eviction made no progress",
"usage_bytes", usage,
"cache_max_bytes", c.config.MaxBytes,
)
return nil
}
c.log.Info("evicted cache content",
"freed_bytes", freed,
"usage_bytes", usage-freed,
"cache_max_bytes", c.config.MaxBytes,
)
}
}
// evictBatch fetches one batch of LRU candidates across variants and
// source blobs and evicts them oldest-first until excessBytes are
// freed or the batch is exhausted. It returns the bytes freed.
func (c *Cache) evictBatch(ctx context.Context, excessBytes int64) (int64, error) {
candidates, err := c.evictionCandidates(ctx)
if err != nil {
return 0, err
}
var freed int64
for _, candidate := range candidates {
if freed >= excessBytes {
break
}
if err := c.evictCandidate(ctx, candidate); err != nil {
c.log.Warn("failed to evict cache entry",
"cache_key", candidate.cacheKey,
"content_hash", candidate.contentHash,
"error", err,
)
continue
}
freed += candidate.sizeBytes
}
return freed, nil
}
// evictCandidate removes a single eviction victim.
func (c *Cache) evictCandidate(ctx context.Context, candidate evictionCandidate) error {
if candidate.isVariant {
return c.evictVariant(ctx, candidate.cacheKey)
}
return c.evictSourceBlob(ctx, candidate.contentHash)
}
// evictionCandidates returns up to evictionBatchSize variants and
// evictionBatchSize source blobs, merged into a single list ordered by
// last access time (oldest first).
func (c *Cache) evictionCandidates(ctx context.Context) ([]evictionCandidate, error) {
variants, err := c.variantCandidates(ctx)
if err != nil {
return nil, err
}
sources, err := c.sourceCandidates(ctx)
if err != nil {
return nil, err
}
// Merge the two lists, each already sorted oldest-first. SQLite
// CURRENT_TIMESTAMP strings compare correctly lexicographically.
merged := make([]evictionCandidate, 0, len(variants)+len(sources))
for len(variants) > 0 && len(sources) > 0 {
if variants[0].lastAccessedAt <= sources[0].lastAccessedAt {
merged = append(merged, variants[0])
variants = variants[1:]
} else {
merged = append(merged, sources[0])
sources = sources[1:]
}
}
merged = append(merged, variants...)
merged = append(merged, sources...)
return merged, nil
}
// variantCandidates returns the least recently used variants.
func (c *Cache) variantCandidates(ctx context.Context) ([]evictionCandidate, error) {
rows, err := c.db.QueryContext(ctx, `
SELECT cache_key, size_bytes, last_accessed_at
FROM variant_content
ORDER BY last_accessed_at ASC, cache_key ASC
LIMIT ?
`, evictionBatchSize)
if err != nil {
return nil, fmt.Errorf("failed to query variant eviction candidates: %w", err)
}
defer func() { _ = rows.Close() }()
var candidates []evictionCandidate
for rows.Next() {
candidate := evictionCandidate{isVariant: true}
var key string
if err := rows.Scan(&key, &candidate.sizeBytes, &candidate.lastAccessedAt); err != nil {
return nil, fmt.Errorf("failed to scan variant candidate: %w", err)
}
candidate.cacheKey = VariantKey(key)
candidates = append(candidates, candidate)
}
if err := rows.Err(); err != nil {
return nil, fmt.Errorf("variant candidate iteration failed: %w", err)
}
return candidates, nil
}
// sourceCandidates returns the least recently used source blobs. Rows
// written before the LRU column existed fall back to fetched_at.
func (c *Cache) sourceCandidates(ctx context.Context) ([]evictionCandidate, error) {
rows, err := c.db.QueryContext(ctx, `
SELECT content_hash, size_bytes,
COALESCE(last_accessed_at, fetched_at, '1970-01-01 00:00:00') AS lru
FROM source_content
ORDER BY lru ASC, content_hash ASC
LIMIT ?
`, evictionBatchSize)
if err != nil {
return nil, fmt.Errorf("failed to query source eviction candidates: %w", err)
}
defer func() { _ = rows.Close() }()
var candidates []evictionCandidate
for rows.Next() {
var candidate evictionCandidate
var hash string
if err := rows.Scan(&hash, &candidate.sizeBytes, &candidate.lastAccessedAt); err != nil {
return nil, fmt.Errorf("failed to scan source candidate: %w", err)
}
candidate.contentHash = ContentHash(hash)
candidates = append(candidates, candidate)
}
if err := rows.Err(); err != nil {
return nil, fmt.Errorf("source candidate iteration failed: %w", err)
}
return candidates, nil
}
// evictVariant removes one variant: accounting row first, then the
// content and .meta files, so the database never references a deleted
// file.
func (c *Cache) evictVariant(ctx context.Context, cacheKey VariantKey) error {
_, err := c.db.ExecContext(ctx,
`DELETE FROM variant_content WHERE cache_key = ?`, string(cacheKey))
if err != nil {
return fmt.Errorf("failed to delete variant accounting row: %w", err)
}
if err := c.variants.DeleteWithMeta(cacheKey); err != nil {
return err
}
return nil
}
// sourceReference identifies one source_metadata row's JSON sidecar.
type sourceReference struct {
host string
pathHash PathHash
}
// evictSourceBlob removes one source content blob. All source_metadata
// rows referencing the blob are deleted together with its
// source_content row in a single transaction BEFORE the file is
// unlinked: a blob referenced by multiple source paths is only ever
// removed together with all of its references, and database rows never
// point at deleted files. The JSON metadata sidecars for the removed
// rows are deleted afterwards.
func (c *Cache) evictSourceBlob(ctx context.Context, contentHash ContentHash) error {
references, err := c.sourceReferences(ctx, contentHash)
if err != nil {
return err
}
tx, err := c.db.BeginTx(ctx, nil)
if err != nil {
return fmt.Errorf("failed to begin eviction transaction: %w", err)
}
defer func() { _ = tx.Rollback() }()
if _, err := tx.ExecContext(ctx,
`DELETE FROM source_metadata WHERE content_hash = ?`, string(contentHash)); err != nil {
return fmt.Errorf("failed to delete source metadata rows: %w", err)
}
if _, err := tx.ExecContext(ctx,
`DELETE FROM source_content WHERE content_hash = ?`, string(contentHash)); err != nil {
return fmt.Errorf("failed to delete source content row: %w", err)
}
if err := tx.Commit(); err != nil {
return fmt.Errorf("failed to commit eviction transaction: %w", err)
}
// Only after the rows are gone may the files be removed.
for _, reference := range references {
if err := c.srcMetadata.Delete(reference.host, reference.pathHash); err != nil {
c.log.Warn("failed to delete metadata sidecar",
"host", reference.host, "path_hash", reference.pathHash, "error", err)
}
}
if err := c.srcContent.Delete(contentHash); err != nil {
return err
}
return nil
}
// sourceReferences lists the metadata sidecar locations of every
// source_metadata row referencing the given blob.
func (c *Cache) sourceReferences(
ctx context.Context, contentHash ContentHash,
) ([]sourceReference, error) {
rows, err := c.db.QueryContext(ctx, `
SELECT source_host, path_hash FROM source_metadata WHERE content_hash = ?
`, string(contentHash))
if err != nil {
return nil, fmt.Errorf("failed to query source references: %w", err)
}
defer func() { _ = rows.Close() }()
var references []sourceReference
for rows.Next() {
var reference sourceReference
var pathHash string
if err := rows.Scan(&reference.host, &pathHash); err != nil {
return nil, fmt.Errorf("failed to scan source reference: %w", err)
}
reference.pathHash = PathHash(pathHash)
references = append(references, reference)
}
if err := rows.Err(); err != nil {
return nil, fmt.Errorf("source reference iteration failed: %w", err)
}
return references, nil
}
// notifyWritePressure wakes the background evictor after a store, so
// eviction under write pressure happens promptly without blocking the
// storing request. The notification channel has capacity one and drops
// when a wakeup is already pending.
func (c *Cache) notifyWritePressure() {
if c.disabled || c.config.MaxBytes <= 0 {
return
}
select {
case c.evictionPressure <- struct{}{}:
default:
}
}
// StartEviction launches the background eviction goroutine, which
// reconciles the database accounting with the cache directories once
// at startup and then evicts to the configured limit on the given
// periodic interval and on write-pressure notifications. It is a
// no-op on a disabled cache or when already started.
func (c *Cache) StartEviction(interval time.Duration) {
if c.disabled || c.evictionStarted {
return
}
c.evictionStarted = true
go c.evictionLoop(interval)
}
// StopEviction stops the background eviction goroutine and waits for
// it to exit. It is safe to call when eviction was never started, and
// safe to call more than once.
func (c *Cache) StopEviction() {
if !c.evictionStarted {
return
}
c.evictionStopOnce.Do(func() {
close(c.evictionStop)
<-c.evictionDone
})
}
// evictionLoop is the body of the background eviction goroutine.
func (c *Cache) evictionLoop(interval time.Duration) {
defer close(c.evictionDone)
ctx := context.Background()
if err := c.reconcileAccounting(ctx); err != nil {
c.log.Warn("cache accounting reconciliation failed", "error", err)
}
c.runEvictionPass(ctx)
ticker := time.NewTicker(interval)
defer ticker.Stop()
for {
select {
case <-c.evictionStop:
return
case <-ticker.C:
case <-c.evictionPressure:
}
c.runEvictionPass(ctx)
}
}
// runEvictionPass runs one eviction pass, logging failures instead of
// propagating them (the loop must keep running).
func (c *Cache) runEvictionPass(ctx context.Context) {
if err := c.EvictToLimit(ctx); err != nil {
c.log.Warn("cache eviction pass failed", "error", err)
}
}
// reconcileAccounting synchronizes the database size accounting with
// the actual contents of the cache directories. It runs once when the
// background evictor starts, off the request hot path: it adopts
// variant files that predate the accounting table, drops accounting
// rows whose files are missing, removes source blob files the database
// does not know (and rows whose files are gone), and sweeps stale temp
// files left behind by crashed writes.
func (c *Cache) reconcileAccounting(ctx context.Context) error {
if c.disabled {
return nil
}
if err := c.reconcileVariantFiles(ctx); err != nil {
return err
}
if err := c.reconcileVariantRows(ctx); err != nil {
return err
}
if err := c.reconcileSourceFiles(ctx); err != nil {
return err
}
if err := c.reconcileSourceRows(ctx); err != nil {
return err
}
return nil
}
// reconcileVariantFiles walks the variant storage directory, adopting
// files without accounting rows and sweeping stale temp files.
func (c *Cache) reconcileVariantFiles(ctx context.Context) error {
return filepath.WalkDir(c.variants.baseDir, func(path string, entry fs.DirEntry, err error) error {
if err != nil || entry.IsDir() {
return err
}
name := entry.Name()
if strings.HasPrefix(name, tempFilePrefix) {
c.sweepStaleTempFile(path, entry)
return nil
}
if strings.HasSuffix(name, variantMetaSuffix) {
return nil
}
return c.adoptVariantFile(ctx, path, entry, VariantKey(name))
})
}
// adoptVariantFile inserts an accounting row for a variant file that
// has none, using the file's size and modification time.
func (c *Cache) adoptVariantFile(
ctx context.Context, path string, entry fs.DirEntry, cacheKey VariantKey,
) error {
var rowExists int
err := c.db.QueryRowContext(ctx,
`SELECT COUNT(*) FROM variant_content WHERE cache_key = ?`, string(cacheKey),
).Scan(&rowExists)
if err != nil {
return fmt.Errorf("failed to check variant accounting row: %w", err)
}
if rowExists > 0 {
return nil
}
info, err := entry.Info()
if err != nil {
return fmt.Errorf("failed to stat variant file: %w", err)
}
modTime := info.ModTime().UTC().Format(sqliteTimestampLayout)
contentType := c.variantContentTypeFromSidecar(path)
_, err = c.db.ExecContext(ctx, `
INSERT INTO variant_content
(cache_key, size_bytes, content_type, created_at, last_accessed_at)
VALUES (?, ?, ?, ?, ?)
`, string(cacheKey), info.Size(), contentType, modTime, modTime)
if err != nil {
return fmt.Errorf("failed to adopt variant file into accounting: %w", err)
}
c.log.Info("adopted untracked variant file into size accounting",
"cache_key", cacheKey, "size_bytes", info.Size())
return nil
}
// variantContentTypeFromSidecar reads the content type from a variant
// .meta sidecar, falling back to application/octet-stream.
func (c *Cache) variantContentTypeFromSidecar(variantPath string) string {
metaData, err := os.ReadFile(variantPath + variantMetaSuffix) //nolint:gosec // path from cache walk
if err != nil {
return fallbackContentType
}
var meta VariantMeta
if json.Unmarshal(metaData, &meta) != nil || meta.ContentType == "" {
return fallbackContentType
}
return meta.ContentType
}
// reconcileVariantRows drops accounting rows whose variant files are
// missing, so the database never references deleted content.
func (c *Cache) reconcileVariantRows(ctx context.Context) error {
keys, err := c.allVariantKeys(ctx)
if err != nil {
return err
}
for _, key := range keys {
if c.variants.Exists(key) {
continue
}
if _, err := c.db.ExecContext(ctx,
`DELETE FROM variant_content WHERE cache_key = ?`, string(key)); err != nil {
return fmt.Errorf("failed to drop stale variant accounting row: %w", err)
}
c.log.Info("dropped accounting row for missing variant file", "cache_key", key)
}
return nil
}
// allVariantKeys returns every tracked variant cache key.
func (c *Cache) allVariantKeys(ctx context.Context) ([]VariantKey, error) {
rows, err := c.db.QueryContext(ctx, `SELECT cache_key FROM variant_content`)
if err != nil {
return nil, fmt.Errorf("failed to query variant keys: %w", err)
}
defer func() { _ = rows.Close() }()
var keys []VariantKey
for rows.Next() {
var key string
if err := rows.Scan(&key); err != nil {
return nil, fmt.Errorf("failed to scan variant key: %w", err)
}
keys = append(keys, VariantKey(key))
}
if err := rows.Err(); err != nil {
return nil, fmt.Errorf("variant key iteration failed: %w", err)
}
return keys, nil
}
// reconcileSourceFiles walks the source content directory, removing
// blob files the database does not track (they are unreachable: source
// lookups always go through source_metadata) and sweeping stale temp
// files.
func (c *Cache) reconcileSourceFiles(ctx context.Context) error {
return filepath.WalkDir(c.srcContent.baseDir, func(path string, entry fs.DirEntry, err error) error {
if err != nil || entry.IsDir() {
return err
}
name := entry.Name()
if strings.HasPrefix(name, tempFilePrefix) {
c.sweepStaleTempFile(path, entry)
return nil
}
return c.removeUntrackedSourceFile(ctx, path, ContentHash(name))
})
}
// removeUntrackedSourceFile deletes a source blob file that has no
// source_content row. Any source_metadata rows referencing the hash
// are removed first so no row ever points at a deleted file.
func (c *Cache) removeUntrackedSourceFile(
ctx context.Context, path string, contentHash ContentHash,
) error {
var rowExists int
err := c.db.QueryRowContext(ctx,
`SELECT COUNT(*) FROM source_content WHERE content_hash = ?`, string(contentHash),
).Scan(&rowExists)
if err != nil {
return fmt.Errorf("failed to check source content row: %w", err)
}
if rowExists > 0 {
return nil
}
if _, err := c.db.ExecContext(ctx,
`DELETE FROM source_metadata WHERE content_hash = ?`, string(contentHash)); err != nil {
return fmt.Errorf("failed to delete metadata rows for untracked blob: %w", err)
}
//nolint:gosec // G703: path comes from walking our own cache directory
if err := os.Remove(path); err != nil && !os.IsNotExist(err) {
return fmt.Errorf("failed to remove untracked source file: %w", err)
}
c.log.Info("removed untracked source content file", "content_hash", contentHash)
return nil
}
// reconcileSourceRows removes source_content rows (and their metadata
// references and sidecars) whose blob files are missing on disk.
func (c *Cache) reconcileSourceRows(ctx context.Context) error {
hashes, err := c.allSourceContentHashes(ctx)
if err != nil {
return err
}
for _, hash := range hashes {
if c.srcContent.Exists(hash) {
continue
}
// The blob file is already gone; evictSourceBlob removes the
// rows and sidecars and tolerates the missing file.
if err := c.evictSourceBlob(ctx, hash); err != nil {
return err
}
c.log.Info("dropped rows for missing source content file", "content_hash", hash)
}
return nil
}
// allSourceContentHashes returns every tracked source content hash.
func (c *Cache) allSourceContentHashes(ctx context.Context) ([]ContentHash, error) {
rows, err := c.db.QueryContext(ctx, `SELECT content_hash FROM source_content`)
if err != nil {
return nil, fmt.Errorf("failed to query source content hashes: %w", err)
}
defer func() { _ = rows.Close() }()
var hashes []ContentHash
for rows.Next() {
var hash string
if err := rows.Scan(&hash); err != nil {
return nil, fmt.Errorf("failed to scan content hash: %w", err)
}
hashes = append(hashes, ContentHash(hash))
}
if err := rows.Err(); err != nil {
return nil, fmt.Errorf("content hash iteration failed: %w", err)
}
return hashes, nil
}
// sweepStaleTempFile removes a temp file left behind by a crashed
// write once it is old enough that no in-flight store can own it.
func (c *Cache) sweepStaleTempFile(path string, entry fs.DirEntry) {
info, err := entry.Info()
if err != nil {
return
}
if time.Since(info.ModTime()) < staleTempFileAge {
return
}
//nolint:gosec // G703: path comes from walking our own cache directory
if err := os.Remove(path); err != nil && !os.IsNotExist(err) {
c.log.Warn("failed to remove stale temp file", "path", path, "error", err)
return
}
c.log.Info("removed stale temp file", "path", path)
}

View File

@@ -0,0 +1,669 @@
package imgcache
import (
"bytes"
"context"
"database/sql"
"io/fs"
"os"
"path/filepath"
"testing"
"time"
_ "modernc.org/sqlite"
"sneak.berlin/go/pixa/internal/database"
"sneak.berlin/go/pixa/internal/httpfetcher"
)
// sqliteTimestampFormat matches the format SQLite's CURRENT_TIMESTAMP
// produces, so injected timestamps compare correctly against ones the
// implementation writes.
const sqliteTimestampFormat = "2006-01-02 15:04:05"
// evictionTestDB creates an in-memory SQLite database with the real
// production schema, limited to a single connection so the background
// eviction goroutine shares the same in-memory database as the test.
func evictionTestDB(t *testing.T) *sql.DB {
t.Helper()
db, err := sql.Open("sqlite", ":memory:")
if err != nil {
t.Fatalf("failed to open test db: %v", err)
}
db.SetMaxOpenConns(1)
if err := database.ApplyMigrations(context.Background(), db, nil); err != nil {
t.Fatalf("failed to apply migrations: %v", err)
}
t.Cleanup(func() { _ = db.Close() })
return db
}
// newEvictionTestCache creates a Cache backed by a temp directory and
// an in-memory database, with the given size limit.
func newEvictionTestCache(t *testing.T, maxBytes int64) (*Cache, string) {
t.Helper()
tmpDir := t.TempDir()
db := evictionTestDB(t)
// maxBytes zero mirrors the production mapping of
// cache_max_bytes: 0 (handlers sets DisableDiskCache); at the
// CacheConfig layer itself a zero MaxBytes means "no limit" for
// backwards compatibility with existing fixtures.
cache, err := NewCache(db, CacheConfig{
StateDir: tmpDir,
CacheTTL: time.Hour,
NegativeTTL: 5 * time.Minute,
MaxBytes: maxBytes,
DisableDiskCache: maxBytes == 0,
})
if err != nil {
t.Fatalf("failed to create cache: %v", err)
}
return cache, tmpDir
}
// storeEvictionTestSource stores content as a fetched source for
// host/path and returns the resulting content hash.
func storeEvictionTestSource(
t *testing.T, cache *Cache, host, path string, content []byte,
) ContentHash {
t.Helper()
req := &ImageRequest{
SourceHost: host,
SourcePath: path,
Format: FormatJPEG,
Quality: 85,
FitMode: FitCover,
}
result := &httpfetcher.FetchResult{
StatusCode: 200,
ContentType: "image/jpeg",
ContentLength: int64(len(content)),
Headers: map[string][]string{"Content-Type": {"image/jpeg"}},
}
hash, err := cache.StoreSource(context.Background(), req, bytes.NewReader(content), result)
if err != nil {
t.Fatalf("StoreSource(%s%s) failed: %v", host, path, err)
}
return hash
}
// storeEvictionTestVariant stores content as a processed variant under
// the given cache key.
func storeEvictionTestVariant(t *testing.T, cache *Cache, key VariantKey, content []byte) {
t.Helper()
if err := cache.StoreVariant(key, bytes.NewReader(content), "image/webp"); err != nil {
t.Fatalf("StoreVariant(%s) failed: %v", key, err)
}
}
// setVariantLastAccessed backdates the last access time of a tracked
// variant, to make LRU ordering deterministic in tests.
func setVariantLastAccessed(t *testing.T, cache *Cache, key VariantKey, when time.Time) {
t.Helper()
res, err := cache.db.Exec(
`UPDATE variant_content SET last_accessed_at = ? WHERE cache_key = ?`,
when.UTC().Format(sqliteTimestampFormat), string(key),
)
if err != nil {
t.Fatalf("failed to set variant last_accessed_at: %v", err)
}
affected, err := res.RowsAffected()
if err != nil {
t.Fatalf("failed to read affected rows: %v", err)
}
if affected != 1 {
t.Fatalf("variant %s has no accounting row (affected=%d); "+
"stores must track variants in the database", key, affected)
}
}
// setSourceLastAccessed backdates the last access time of a tracked
// source content blob.
func setSourceLastAccessed(t *testing.T, cache *Cache, hash ContentHash, when time.Time) {
t.Helper()
res, err := cache.db.Exec(
`UPDATE source_content SET last_accessed_at = ? WHERE content_hash = ?`,
when.UTC().Format(sqliteTimestampFormat), string(hash),
)
if err != nil {
t.Fatalf("failed to set source last_accessed_at: %v", err)
}
affected, err := res.RowsAffected()
if err != nil {
t.Fatalf("failed to read affected rows: %v", err)
}
if affected != 1 {
t.Fatalf("source %s has no accounting row (affected=%d)", hash, affected)
}
}
// countRows returns the number of rows the given query yields.
func countRows(t *testing.T, cache *Cache, query string, args ...interface{}) int {
t.Helper()
var n int
if err := cache.db.QueryRow(query, args...).Scan(&n); err != nil {
t.Fatalf("count query %q failed: %v", query, err)
}
return n
}
// assertNoDanglingReferences verifies the core eviction invariant:
// every database row that references cache content on disk points at a
// file that actually exists.
func assertNoDanglingReferences(t *testing.T, cache *Cache) {
t.Helper()
rows, err := cache.db.Query(
`SELECT content_hash FROM source_metadata
WHERE content_hash IS NOT NULL AND content_hash != ''`,
)
if err != nil {
t.Fatalf("failed to query source_metadata: %v", err)
}
defer func() { _ = rows.Close() }()
for rows.Next() {
var hash string
if err := rows.Scan(&hash); err != nil {
t.Fatalf("failed to scan content_hash: %v", err)
}
if !cache.srcContent.Exists(ContentHash(hash)) {
t.Errorf("source_metadata references content %s but the file is missing", hash)
}
}
if err := rows.Err(); err != nil {
t.Fatalf("source_metadata iteration failed: %v", err)
}
variantRows, err := cache.db.Query(`SELECT cache_key FROM variant_content`)
if err != nil {
t.Fatalf("failed to query variant_content: %v", err)
}
defer func() { _ = variantRows.Close() }()
for variantRows.Next() {
var key string
if err := variantRows.Scan(&key); err != nil {
t.Fatalf("failed to scan cache_key: %v", err)
}
if !cache.variants.Exists(VariantKey(key)) {
t.Errorf("variant_content references key %s but the file is missing", key)
}
}
if err := variantRows.Err(); err != nil {
t.Fatalf("variant_content iteration failed: %v", err)
}
}
// waitForUsageAtOrBelow polls UsageBytes until it reaches limit or the
// timeout expires, returning the last observed usage.
func waitForUsageAtOrBelow(t *testing.T, cache *Cache, limit int64, timeout time.Duration) int64 {
t.Helper()
deadline := time.Now().Add(timeout)
var usage int64
for time.Now().Before(deadline) {
var err error
usage, err = cache.UsageBytes(context.Background())
if err != nil {
t.Fatalf("UsageBytes failed: %v", err)
}
if usage <= limit {
return usage
}
time.Sleep(25 * time.Millisecond)
}
return usage
}
func TestUsageBytesAccountsSourceAndVariantBytes(t *testing.T) {
cache, _ := newEvictionTestCache(t, 1<<30)
storeEvictionTestSource(t, cache, "src.example.com", "/a.jpg",
bytes.Repeat([]byte{0xAA}, 1000))
storeEvictionTestSource(t, cache, "src.example.com", "/b.jpg",
bytes.Repeat([]byte{0xAB}, 2000))
storeEvictionTestVariant(t, cache, "aabbccdd0001", bytes.Repeat([]byte{0xAC}, 500))
storeEvictionTestVariant(t, cache, "aabbccdd0002", bytes.Repeat([]byte{0xAD}, 250))
usage, err := cache.UsageBytes(context.Background())
if err != nil {
t.Fatalf("UsageBytes failed: %v", err)
}
if usage != 3750 {
t.Errorf("UsageBytes = %d, want 3750 (1000+2000+500+250)", usage)
}
}
func TestUsageBytesCountsMultiReferencedBlobOnce(t *testing.T) {
cache, _ := newEvictionTestCache(t, 1<<30)
content := bytes.Repeat([]byte{0xCC}, 1200)
hashOne := storeEvictionTestSource(t, cache, "src.example.com", "/one.jpg", content)
hashTwo := storeEvictionTestSource(t, cache, "src.example.com", "/two.jpg", content)
if hashOne != hashTwo {
t.Fatalf("identical content produced different hashes: %s vs %s", hashOne, hashTwo)
}
usage, err := cache.UsageBytes(context.Background())
if err != nil {
t.Fatalf("UsageBytes failed: %v", err)
}
if usage != 1200 {
t.Errorf("UsageBytes = %d, want 1200 (deduplicated blob counted once)", usage)
}
}
func TestEvictToLimitEvictsLeastRecentlyUsedFirst(t *testing.T) {
const limit = 3000
cache, _ := newEvictionTestCache(t, limit)
now := time.Now()
keys := []VariantKey{"aabbccdd0001", "aabbccdd0002", "aabbccdd0003", "aabbccdd0004"}
fills := []byte{0x01, 0x02, 0x03, 0x04}
ages := []time.Duration{4 * time.Hour, 3 * time.Hour, 2 * time.Hour, 1 * time.Hour}
for i, key := range keys {
storeEvictionTestVariant(t, cache, key, bytes.Repeat([]byte{fills[i]}, 1000))
setVariantLastAccessed(t, cache, key, now.Add(-ages[i]))
}
if err := cache.EvictToLimit(context.Background()); err != nil {
t.Fatalf("EvictToLimit failed: %v", err)
}
usage, err := cache.UsageBytes(context.Background())
if err != nil {
t.Fatalf("UsageBytes failed: %v", err)
}
if usage > limit {
t.Errorf("usage after eviction = %d, want <= %d", usage, limit)
}
if cache.variants.Exists(keys[0]) {
t.Errorf("least recently used variant %s must be evicted", keys[0])
}
if n := countRows(t, cache,
`SELECT COUNT(*) FROM variant_content WHERE cache_key = ?`, string(keys[0]),
); n != 0 {
t.Errorf("evicted variant %s still has %d accounting rows", keys[0], n)
}
for _, key := range keys[1:] {
if !cache.variants.Exists(key) {
t.Errorf("more recently used variant %s must survive eviction", key)
}
}
assertNoDanglingReferences(t, cache)
}
func TestEvictionRemovesMultiReferencedBlobTogetherWithAllReferences(t *testing.T) {
const limit = 1000
cache, _ := newEvictionTestCache(t, limit)
now := time.Now()
// One 800-byte blob referenced by two source paths.
sharedContent := bytes.Repeat([]byte{0xDD}, 800)
sharedHash := storeEvictionTestSource(t, cache, "src.example.com", "/a.jpg", sharedContent)
if h := storeEvictionTestSource(t, cache, "src.example.com", "/b.jpg", sharedContent); h != sharedHash {
t.Fatalf("identical content produced different hashes: %s vs %s", h, sharedHash)
}
// A newer 600-byte blob referenced by one source path.
recentHash := storeEvictionTestSource(t, cache, "src.example.com", "/c.jpg",
bytes.Repeat([]byte{0xEE}, 600))
setSourceLastAccessed(t, cache, sharedHash, now.Add(-2*time.Hour))
setSourceLastAccessed(t, cache, recentHash, now.Add(-time.Minute))
if err := cache.EvictToLimit(context.Background()); err != nil {
t.Fatalf("EvictToLimit failed: %v", err)
}
usage, err := cache.UsageBytes(context.Background())
if err != nil {
t.Fatalf("UsageBytes failed: %v", err)
}
if usage > limit {
t.Errorf("usage after eviction = %d, want <= %d", usage, limit)
}
// The multi-referenced blob must be gone from disk, from
// source_content, and from BOTH source_metadata rows: references
// are removed together with the blob, never left dangling.
if cache.srcContent.Exists(sharedHash) {
t.Errorf("evicted blob %s still exists on disk", sharedHash)
}
if n := countRows(t, cache,
`SELECT COUNT(*) FROM source_content WHERE content_hash = ?`, string(sharedHash),
); n != 0 {
t.Errorf("evicted blob %s still has %d source_content rows", sharedHash, n)
}
if n := countRows(t, cache,
`SELECT COUNT(*) FROM source_metadata WHERE content_hash = ?`, string(sharedHash),
); n != 0 {
t.Errorf("evicted blob %s still has %d source_metadata references", sharedHash, n)
}
// The JSON metadata sidecars for both referencing paths must be
// removed along with the rows.
for _, path := range []string{"/a.jpg", "/b.jpg"} {
pathHash := HashPath(path + "?")
if cache.srcMetadata.Exists("src.example.com", pathHash) {
t.Errorf("metadata sidecar for %s must be removed with its row", path)
}
}
// The more recently used blob survives fully intact.
if !cache.srcContent.Exists(recentHash) {
t.Errorf("recently used blob %s must survive eviction", recentHash)
}
if n := countRows(t, cache,
`SELECT COUNT(*) FROM source_metadata WHERE content_hash = ?`, string(recentHash),
); n != 1 {
t.Errorf("recently used blob %s has %d source_metadata rows, want 1", recentHash, n)
}
assertNoDanglingReferences(t, cache)
}
func TestEvictionKeepsEverythingWhenUnderLimit(t *testing.T) {
cache, _ := newEvictionTestCache(t, 1<<30)
content := bytes.Repeat([]byte{0xDF}, 800)
hash := storeEvictionTestSource(t, cache, "src.example.com", "/a.jpg", content)
if h := storeEvictionTestSource(t, cache, "src.example.com", "/b.jpg", content); h != hash {
t.Fatalf("identical content produced different hashes: %s vs %s", h, hash)
}
storeEvictionTestVariant(t, cache, "aabbccdd0001", bytes.Repeat([]byte{0xE0}, 500))
if err := cache.EvictToLimit(context.Background()); err != nil {
t.Fatalf("EvictToLimit failed: %v", err)
}
if !cache.srcContent.Exists(hash) {
t.Errorf("blob %s must not be evicted while usage is under the limit", hash)
}
if n := countRows(t, cache,
`SELECT COUNT(*) FROM source_metadata WHERE content_hash = ?`, string(hash),
); n != 2 {
t.Errorf("blob %s has %d source_metadata rows, want 2", hash, n)
}
if !cache.variants.Exists("aabbccdd0001") {
t.Error("variant must not be evicted while usage is under the limit")
}
assertNoDanglingReferences(t, cache)
}
func TestZeroMaxBytesDisablesDiskCache(t *testing.T) {
cache, tmpDir := newEvictionTestCache(t, 0)
ctx := context.Background()
req := &ImageRequest{
SourceHost: "src.example.com",
SourcePath: "/a.jpg",
Format: FormatJPEG,
Quality: 85,
FitMode: FitCover,
}
// Writes are no-ops that report success.
if err := cache.StoreVariant(CacheKey(req), bytes.NewReader([]byte("data")), "image/webp"); err != nil {
t.Fatalf("StoreVariant on disabled cache must be a no-op, got error: %v", err)
}
result := &httpfetcher.FetchResult{
StatusCode: 200,
ContentType: "image/jpeg",
ContentLength: 4,
Headers: map[string][]string{},
}
hash, err := cache.StoreSource(ctx, req, bytes.NewReader([]byte("data")), result)
if err != nil {
t.Fatalf("StoreSource on disabled cache must be a no-op, got error: %v", err)
}
if hash != "" {
t.Errorf("StoreSource on disabled cache returned hash %q, want empty", hash)
}
// Reads always miss.
lookup, err := cache.Lookup(ctx, req)
if err != nil {
t.Fatalf("Lookup on disabled cache failed: %v", err)
}
if lookup.Hit {
t.Error("Lookup on disabled cache must always miss")
}
srcHash, srcType, err := cache.LookupSource(ctx, req)
if err != nil {
t.Fatalf("LookupSource on disabled cache failed: %v", err)
}
if srcHash != "" || srcType != "" {
t.Errorf("LookupSource on disabled cache = (%q, %q), want empty", srcHash, srcType)
}
// Nothing is tracked and nothing is written to disk.
usage, err := cache.UsageBytes(ctx)
if err != nil {
t.Fatalf("UsageBytes failed: %v", err)
}
if usage != 0 {
t.Errorf("UsageBytes on disabled cache = %d, want 0", usage)
}
if n := countRows(t, cache, `SELECT COUNT(*) FROM source_content`); n != 0 {
t.Errorf("disabled cache wrote %d source_content rows, want 0", n)
}
if n := countRows(t, cache, `SELECT COUNT(*) FROM source_metadata`); n != 0 {
t.Errorf("disabled cache wrote %d source_metadata rows, want 0", n)
}
if _, err := os.Stat(filepath.Join(tmpDir, "cache")); !os.IsNotExist(err) {
t.Errorf("disabled cache must not create the cache directory tree (stat err=%v)", err)
}
var foundFiles []string
walkErr := filepath.WalkDir(tmpDir, func(path string, d fs.DirEntry, err error) error {
if err != nil {
return err
}
if !d.IsDir() {
foundFiles = append(foundFiles, path)
}
return nil
})
if walkErr != nil {
t.Fatalf("failed to walk state dir: %v", walkErr)
}
if len(foundFiles) != 0 {
t.Errorf("disabled cache wrote files to disk: %v", foundFiles)
}
}
func TestEvictionRunsUnderWritePressure(t *testing.T) {
const limit = 1500
cache, _ := newEvictionTestCache(t, limit)
// An interval far longer than the test ensures only write
// pressure can trigger eviction here.
cache.StartEviction(time.Hour)
defer cache.StopEviction()
keys := []VariantKey{"aabbccdd0001", "aabbccdd0002", "aabbccdd0003"}
fills := []byte{0x11, 0x12, 0x13}
for i, key := range keys {
storeEvictionTestVariant(t, cache, key, bytes.Repeat([]byte{fills[i]}, 1000))
}
usage := waitForUsageAtOrBelow(t, cache, limit, 5*time.Second)
if usage > limit {
t.Errorf("write pressure did not trigger eviction: usage = %d, want <= %d",
usage, limit)
}
assertNoDanglingReferences(t, cache)
}
func TestEvictionRunsOnPeriodicSchedule(t *testing.T) {
const limit = 1500
cache, _ := newEvictionTestCache(t, limit)
// Start the evictor while the cache is empty, then create tracked
// over-limit state WITHOUT going through the store methods, so no
// write-pressure notification fires and only the periodic ticker
// can trigger eviction.
cache.StartEviction(100 * time.Millisecond)
defer cache.StopEviction()
keys := []VariantKey{"aabbccdd0001", "aabbccdd0002", "aabbccdd0003"}
fills := []byte{0x21, 0x22, 0x23}
for i, key := range keys {
content := bytes.Repeat([]byte{fills[i]}, 1000)
if _, err := cache.variants.Store(key, bytes.NewReader(content), "image/webp"); err != nil {
t.Fatalf("failed to store variant file: %v", err)
}
if _, err := cache.db.Exec(
`INSERT INTO variant_content (cache_key, size_bytes, content_type)
VALUES (?, ?, ?)`,
string(key), len(content), "image/webp",
); err != nil {
t.Fatalf("failed to insert variant accounting row: %v", err)
}
}
usage := waitForUsageAtOrBelow(t, cache, limit, 5*time.Second)
if usage > limit {
t.Errorf("periodic schedule did not trigger eviction: usage = %d, want <= %d",
usage, limit)
}
assertNoDanglingReferences(t, cache)
}
func TestStartEvictionReconcilesAccountingWithDisk(t *testing.T) {
cache, _ := newEvictionTestCache(t, 1<<30)
// An untracked variant file on disk (e.g. written before this
// feature existed) must be adopted into the accounting.
untracked := bytes.Repeat([]byte{0x31}, 1000)
if _, err := cache.variants.Store("aabbccdd0001", bytes.NewReader(untracked), "image/webp"); err != nil {
t.Fatalf("failed to store untracked variant file: %v", err)
}
// An accounting row whose file is missing must be dropped.
if _, err := cache.db.Exec(
`INSERT INTO variant_content (cache_key, size_bytes, content_type)
VALUES (?, ?, ?)`,
"deadbeef0001", 700, "image/webp",
); err != nil {
t.Fatalf("failed to insert stale variant accounting row: %v", err)
}
cache.StartEviction(time.Hour)
defer cache.StopEviction()
deadline := time.Now().Add(5 * time.Second)
var usage int64
for time.Now().Before(deadline) {
var err error
usage, err = cache.UsageBytes(context.Background())
if err != nil {
t.Fatalf("UsageBytes failed: %v", err)
}
if usage == 1000 {
break
}
time.Sleep(25 * time.Millisecond)
}
if usage != 1000 {
t.Errorf("usage after reconciliation = %d, want 1000 "+
"(untracked file adopted, stale row dropped)", usage)
}
if n := countRows(t, cache,
`SELECT COUNT(*) FROM variant_content WHERE cache_key = ?`, "aabbccdd0001",
); n != 1 {
t.Errorf("untracked variant file was not adopted into accounting (rows=%d)", n)
}
if n := countRows(t, cache,
`SELECT COUNT(*) FROM variant_content WHERE cache_key = ?`, "deadbeef0001",
); n != 0 {
t.Errorf("stale accounting row without a file was not dropped (rows=%d)", n)
}
}

View File

@@ -90,7 +90,6 @@ func (r *ImageRequest) SourceURL() string {
if r.AllowHTTP {
scheme = "http"
}
url := scheme + "://" + r.SourceHost + r.SourcePath
if r.SourceQuery != "" {
url += "?" + r.SourceQuery

View File

@@ -8,8 +8,6 @@ import (
)
func TestNegativeCache_StoreAndCheck(t *testing.T) {
t.Parallel()
db := setupTestDB(t)
dir := t.TempDir()
@@ -24,7 +22,7 @@ func TestNegativeCache_StoreAndCheck(t *testing.T) {
ctx := context.Background()
req := &ImageRequest{
SourceHost: testHostExample,
SourceHost: "example.com",
SourcePath: "/missing.jpg",
}
@@ -33,7 +31,6 @@ func TestNegativeCache_StoreAndCheck(t *testing.T) {
if err != nil {
t.Fatal(err)
}
if hit {
t.Error("expected no negative cache hit initially")
}
@@ -49,15 +46,12 @@ func TestNegativeCache_StoreAndCheck(t *testing.T) {
if err != nil {
t.Fatal(err)
}
if !hit {
t.Error("expected negative cache hit after storing")
}
}
func TestNegativeCache_Expired(t *testing.T) {
t.Parallel()
db := setupTestDB(t)
dir := t.TempDir()
@@ -72,7 +66,7 @@ func TestNegativeCache_Expired(t *testing.T) {
ctx := context.Background()
req := &ImageRequest{
SourceHost: testHostExample,
SourceHost: "example.com",
SourcePath: "/expired.jpg",
}
@@ -90,15 +84,12 @@ func TestNegativeCache_Expired(t *testing.T) {
if err != nil {
t.Fatal(err)
}
if hit {
t.Error("expected expired negative cache entry to be a miss")
}
}
func TestService_Get_ReturnsErrorForNegativeCachedURL(t *testing.T) {
t.Parallel()
// This test verifies that Service.Get() checks the negative cache
// We can't easily test the full pipeline without vips, but we can
// verify the error type

View File

@@ -18,8 +18,7 @@ import (
"sneak.berlin/go/pixa/internal/signature"
)
// Service implements the ImageCache interface, orchestrating cache,
// fetcher, and processor.
// Service implements the ImageCache interface, orchestrating cache, fetcher, and processor.
type Service struct {
cache *Cache
fetcher httpfetcher.Fetcher
@@ -47,21 +46,14 @@ type ServiceConfig struct {
Logger *slog.Logger
}
// Static errors for service construction and unimplemented operations.
var (
errCacheRequired = errors.New("cache is required")
errSigningKeyRequired = errors.New("signing key is required")
errPurgeNotImplemented = errors.New("purge not implemented")
)
// NewService creates a new image service.
func NewService(cfg *ServiceConfig) (*Service, error) {
if cfg.Cache == nil {
return nil, errCacheRequired
return nil, errors.New("cache is required")
}
if cfg.SigningKey == "" {
return nil, errSigningKeyRequired
return nil, errors.New("signing key is required")
}
// Resolve fetcher config for defaults
@@ -91,14 +83,11 @@ func NewService(cfg *ServiceConfig) (*Service, error) {
}
maxResponseSize := fetcherCfg.MaxResponseSize
processor := imageprocessor.New(
imageprocessor.Params{MaxInputBytes: maxResponseSize},
)
return &Service{
cache: cfg.Cache,
fetcher: fetcher,
processor: processor,
processor: imageprocessor.New(imageprocessor.Params{MaxInputBytes: maxResponseSize}),
signer: signer,
allowlist: allowlist.New(cfg.Allowlist),
log: log,
@@ -120,7 +109,6 @@ func (s *Service) Get(ctx context.Context, req *ImageRequest) (*ImageResponse, e
if err != nil {
s.log.Warn("negative cache check failed", "error", err)
}
if negHit {
s.log.Debug("negative cache hit",
"host", req.SourceHost,
@@ -157,7 +145,6 @@ func (s *Service) Get(ctx context.Context, req *ImageRequest) (*ImageResponse, e
// Cache miss - check if we have source content cached
cacheKey := CacheKey(req)
s.cache.IncrementStats(ctx, false, 0)
response, err := s.processFromSourceOrFetch(ctx, req, cacheKey)
@@ -170,57 +157,6 @@ func (s *Service) Get(ctx context.Context, req *ImageRequest) (*ImageResponse, e
return response, nil
}
// Warm pre-fetches and caches an image without returning it.
func (s *Service) Warm(ctx context.Context, req *ImageRequest) error {
_, err := s.Get(ctx, req)
return err
}
// Purge removes a cached image. Purging is not implemented yet.
func (s *Service) Purge(_ context.Context, _ *ImageRequest) error {
return errPurgeNotImplemented
}
// Stats returns cache statistics.
func (s *Service) Stats(ctx context.Context) (*CacheStats, error) {
return s.cache.Stats(ctx)
}
// ValidateRequest validates the request signature if required.
func (s *Service) ValidateRequest(req *ImageRequest) error {
// Check if host is allowed (no signature required)
sourceURL := req.SourceURL()
parsedURL, err := url.Parse(sourceURL)
if err != nil {
return fmt.Errorf("invalid source URL: %w", err)
}
if s.allowlist.IsAllowed(parsedURL) {
return nil
}
// Signature required for non-allowed hosts
return s.signer.Verify(signatureRequest(req))
}
// GenerateSignedURL generates a signed URL for the given request.
func (s *Service) GenerateSignedURL(
baseURL string,
req *ImageRequest,
ttl time.Duration,
) (string, error) {
sigReq := signatureRequest(req)
path, sig, exp := s.signer.GenerateSignedURL(sigReq, ttl)
// Propagate the generated signature and expiration back onto the request.
req.Expires = sigReq.Expires
req.Signature = sigReq.Signature
return fmt.Sprintf("%s%s?sig=%s&exp=%d", baseURL, path, sig, exp), nil
}
// loadCachedSource attempts to load source content from cache, returning nil
// if the cached data is unavailable or exceeds maxResponseSize.
func (s *Service) loadCachedSource(contentHash ContentHash) []byte {
@@ -255,8 +191,7 @@ func (s *Service) loadCachedSource(contentHash ContentHash) []byte {
return data
}
// processFromSourceOrFetch processes an image, using cached source content
// if available.
// processFromSourceOrFetch processes an image, using cached source content if available.
func (s *Service) processFromSourceOrFetch(
ctx context.Context,
req *ImageRequest,
@@ -268,10 +203,8 @@ func (s *Service) processFromSourceOrFetch(
s.log.Warn("source lookup failed", "error", err)
}
var (
sourceData []byte
fetchBytes int64
)
var sourceData []byte
var fetchBytes int64
if contentHash != "" {
s.log.Debug("using cached source", "hash", contentHash)
@@ -325,7 +258,6 @@ func (s *Service) fetchAndProcess(
// Calculate download bitrate
fetchBytes := int64(len(sourceData))
var downloadRate string
if fetchResult.FetchDurationMs > 0 {
@@ -348,8 +280,7 @@ func (s *Service) fetchAndProcess(
)
// Validate magic bytes match content type
err = magic.ValidateMagicBytes(sourceData, fetchResult.ContentType)
if err != nil {
if err := magic.ValidateMagicBytes(sourceData, fetchResult.ContentType); err != nil {
return nil, fmt.Errorf("content validation failed: %w", err)
}
@@ -401,8 +332,7 @@ func (s *Service) processAndStore(
var sizePercent float64
if fetchBytes > 0 {
//nolint:mnd // percentage calculation
sizePercent = float64(outputSize) / float64(fetchBytes) * 100.0
sizePercent = float64(outputSize) / float64(fetchBytes) * 100.0 //nolint:mnd // percentage calculation
}
s.log.Info("image converted",
@@ -412,10 +342,8 @@ func (s *Service) processAndStore(
"dst_format", req.Format,
"src_bytes", fetchBytes,
"dst_bytes", outputSize,
"src_dimensions", fmt.Sprintf("%dx%d",
processResult.InputWidth, processResult.InputHeight),
"dst_dimensions", fmt.Sprintf("%dx%d",
processResult.Width, processResult.Height),
"src_dimensions", fmt.Sprintf("%dx%d", processResult.InputWidth, processResult.InputHeight),
"dst_dimensions", fmt.Sprintf("%dx%d", processResult.Width, processResult.Height),
"size_ratio", fmt.Sprintf("%.1f%%", sizePercent),
"convert_ms", processDuration.Milliseconds(),
"quality", req.Quality,
@@ -423,10 +351,7 @@ func (s *Service) processAndStore(
)
// Store variant to cache
err = s.cache.StoreVariant(
cacheKey, bytes.NewReader(processedData), processResult.ContentType,
)
if err != nil {
if err := s.cache.StoreVariant(cacheKey, bytes.NewReader(processedData), processResult.ContentType); err != nil {
s.log.Warn("failed to store variant", "error", err)
// Continue even if caching fails
}
@@ -440,6 +365,58 @@ func (s *Service) processAndStore(
}, nil
}
// Warm pre-fetches and caches an image without returning it.
func (s *Service) Warm(ctx context.Context, req *ImageRequest) error {
_, err := s.Get(ctx, req)
return err
}
// Purge removes a cached image.
func (s *Service) Purge(_ context.Context, _ *ImageRequest) error {
// TODO: Implement purge
return errors.New("purge not implemented")
}
// Stats returns cache statistics.
func (s *Service) Stats(ctx context.Context) (*CacheStats, error) {
return s.cache.Stats(ctx)
}
// ValidateRequest validates the request signature if required.
func (s *Service) ValidateRequest(req *ImageRequest) error {
// Check if host is allowed (no signature required)
sourceURL := req.SourceURL()
parsedURL, err := url.Parse(sourceURL)
if err != nil {
return fmt.Errorf("invalid source URL: %w", err)
}
if s.allowlist.IsAllowed(parsedURL) {
return nil
}
// Signature required for non-allowed hosts
return s.signer.Verify(signatureRequest(req))
}
// GenerateSignedURL generates a signed URL for the given request.
func (s *Service) GenerateSignedURL(
baseURL string,
req *ImageRequest,
ttl time.Duration,
) (string, error) {
sigReq := signatureRequest(req)
path, sig, exp := s.signer.GenerateSignedURL(sigReq, ttl)
// Propagate the generated signature and expiration back onto the request.
req.Expires = sigReq.Expires
req.Signature = sigReq.Signature
return fmt.Sprintf("%s%s?sig=%s&exp=%d", baseURL, path, sig, exp), nil
}
// signatureRequest projects an ImageRequest onto the standalone
// signature.Request type used by the signature package. This keeps the
// import edge one-way: imgcache depends on signature, never the reverse.

View File

@@ -10,22 +10,13 @@ import (
"sneak.berlin/go/pixa/internal/signature"
)
// Test data literals used repeatedly in this file (goconst).
const (
testPathPhoto = "/images/photo.jpg"
testPathUpload = "/uploads/image.jpg"
testSigningKey = "test-signing-key-12345"
)
func TestService_Get_AllowlistedHost(t *testing.T) {
t.Parallel()
svc, fixtures := SetupTestService(t)
ctx := context.Background()
req := &ImageRequest{
SourceHost: fixtures.GoodHost,
SourcePath: testPathPhoto,
SourcePath: "/images/photo.jpg",
Size: Size{Width: 50, Height: 50},
Format: FormatJPEG,
Quality: 85,
@@ -36,8 +27,7 @@ func TestService_Get_AllowlistedHost(t *testing.T) {
if err != nil {
t.Fatalf("Get() error = %v", err)
}
defer func() { _ = resp.Content.Close() }()
defer resp.Content.Close()
// Verify we got content
data, err := io.ReadAll(resp.Content)
@@ -49,19 +39,17 @@ func TestService_Get_AllowlistedHost(t *testing.T) {
t.Error("expected non-empty response")
}
if resp.ContentType != testContentTypeJPEG {
t.Errorf("ContentType = %q, want %q", resp.ContentType, testContentTypeJPEG)
if resp.ContentType != "image/jpeg" {
t.Errorf("ContentType = %q, want %q", resp.ContentType, "image/jpeg")
}
}
func TestService_Get_NonAllowlistedHost_NoSignature(t *testing.T) {
t.Parallel()
svc, fixtures := SetupTestService(t, WithSigningKey("test-key"))
req := &ImageRequest{
SourceHost: fixtures.OtherHost,
SourcePath: testPathUpload,
SourcePath: "/uploads/image.jpg",
Size: Size{Width: 50, Height: 50},
Format: FormatJPEG,
Quality: 85,
@@ -76,15 +64,13 @@ func TestService_Get_NonAllowlistedHost_NoSignature(t *testing.T) {
}
func TestService_Get_NonAllowlistedHost_ValidSignature(t *testing.T) {
t.Parallel()
signingKey := testSigningKey
signingKey := "test-signing-key-12345"
svc, fixtures := SetupTestService(t, WithSigningKey(signingKey))
ctx := context.Background()
req := &ImageRequest{
SourceHost: fixtures.OtherHost,
SourcePath: testPathUpload,
SourcePath: "/uploads/image.jpg",
Size: Size{Width: 50, Height: 50},
Format: FormatJPEG,
Quality: 85,
@@ -107,8 +93,7 @@ func TestService_Get_NonAllowlistedHost_ValidSignature(t *testing.T) {
if err != nil {
t.Fatalf("Get() error = %v", err)
}
defer func() { _ = resp.Content.Close() }()
defer resp.Content.Close()
data, err := io.ReadAll(resp.Content)
if err != nil {
@@ -121,14 +106,12 @@ func TestService_Get_NonAllowlistedHost_ValidSignature(t *testing.T) {
}
func TestService_Get_NonAllowlistedHost_ExpiredSignature(t *testing.T) {
t.Parallel()
signingKey := testSigningKey
signingKey := "test-signing-key-12345"
svc, fixtures := SetupTestService(t, WithSigningKey(signingKey))
req := &ImageRequest{
SourceHost: fixtures.OtherHost,
SourcePath: testPathUpload,
SourcePath: "/uploads/image.jpg",
Size: Size{Width: 50, Height: 50},
Format: FormatJPEG,
Quality: 85,
@@ -148,14 +131,12 @@ func TestService_Get_NonAllowlistedHost_ExpiredSignature(t *testing.T) {
}
func TestService_Get_NonAllowlistedHost_InvalidSignature(t *testing.T) {
t.Parallel()
signingKey := testSigningKey
signingKey := "test-signing-key-12345"
svc, fixtures := SetupTestService(t, WithSigningKey(signingKey))
req := &ImageRequest{
SourceHost: fixtures.OtherHost,
SourcePath: testPathUpload,
SourcePath: "/uploads/image.jpg",
Size: Size{Width: 50, Height: 50},
Format: FormatJPEG,
Quality: 85,
@@ -178,8 +159,6 @@ func TestService_Get_NonAllowlistedHost_InvalidSignature(t *testing.T) {
// signature for one host must not verify for a different host, even
// if they share a domain suffix.
func TestService_ValidateRequest_SignatureExactHostMatch(t *testing.T) {
t.Parallel()
signingKey := "test-signing-key-must-be-32-chars"
svc, _ := SetupTestService(t,
WithSigningKey(signingKey),
@@ -190,8 +169,8 @@ func TestService_ValidateRequest_SignatureExactHostMatch(t *testing.T) {
// Sign a request for "cdn.example.com"
signedReq := &ImageRequest{
SourceHost: testHostCDN,
SourcePath: testPathCat,
SourceHost: "cdn.example.com",
SourcePath: "/photos/cat.jpg",
Size: Size{Width: 50, Height: 50},
Format: FormatJPEG,
Quality: 85,
@@ -202,8 +181,6 @@ func TestService_ValidateRequest_SignatureExactHostMatch(t *testing.T) {
// The original request should pass validation
t.Run("exact host passes", func(t *testing.T) {
t.Parallel()
err := svc.ValidateRequest(signedReq)
if err != nil {
t.Errorf("ValidateRequest() exact host failed: %v", err)
@@ -215,7 +192,7 @@ func TestService_ValidateRequest_SignatureExactHostMatch(t *testing.T) {
name string
host string
}{
{"parent domain", testHostExample},
{"parent domain", "example.com"},
{"sibling subdomain", "images.example.com"},
{"deeper subdomain", "a.cdn.example.com"},
{"evil suffix domain", "cdn.example.com.evil.com"},
@@ -224,8 +201,6 @@ func TestService_ValidateRequest_SignatureExactHostMatch(t *testing.T) {
for _, tt := range tests {
t.Run(tt.name+" rejected", func(t *testing.T) {
t.Parallel()
req := &ImageRequest{
SourceHost: tt.host,
SourcePath: signedReq.SourcePath,
@@ -240,8 +215,7 @@ func TestService_ValidateRequest_SignatureExactHostMatch(t *testing.T) {
err := svc.ValidateRequest(req)
if err == nil {
t.Errorf(
"ValidateRequest() should reject signature for host %q (signed for %q)",
t.Errorf("ValidateRequest() should reject signature for host %q (signed for %q)",
tt.host, signedReq.SourceHost)
}
})
@@ -249,8 +223,6 @@ func TestService_ValidateRequest_SignatureExactHostMatch(t *testing.T) {
}
func TestService_Get_InvalidFile(t *testing.T) {
t.Parallel()
svc, fixtures := SetupTestService(t)
ctx := context.Background()
@@ -271,8 +243,6 @@ func TestService_Get_InvalidFile(t *testing.T) {
}
func TestService_Get_NotFound(t *testing.T) {
t.Parallel()
svc, fixtures := SetupTestService(t)
ctx := context.Background()
@@ -292,8 +262,6 @@ func TestService_Get_NotFound(t *testing.T) {
}
func TestService_Get_FormatConversion(t *testing.T) {
t.Parallel()
svc, fixtures := SetupTestService(t)
ctx := context.Background()
@@ -305,7 +273,7 @@ func TestService_Get_FormatConversion(t *testing.T) {
}{
{
name: "JPEG to PNG",
sourcePath: testPathPhoto,
sourcePath: "/images/photo.jpg",
outFormat: FormatPNG,
wantMIME: "image/png",
},
@@ -313,7 +281,7 @@ func TestService_Get_FormatConversion(t *testing.T) {
name: "PNG to JPEG",
sourcePath: "/images/logo.png",
outFormat: FormatJPEG,
wantMIME: testContentTypeJPEG,
wantMIME: "image/jpeg",
},
{
name: "GIF to PNG",
@@ -325,8 +293,6 @@ func TestService_Get_FormatConversion(t *testing.T) {
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
req := &ImageRequest{
SourceHost: fixtures.GoodHost,
SourcePath: tt.sourcePath,
@@ -340,8 +306,7 @@ func TestService_Get_FormatConversion(t *testing.T) {
if err != nil {
t.Fatalf("Get() error = %v", err)
}
defer func() { _ = resp.Content.Close() }()
defer resp.Content.Close()
if resp.ContentType != tt.wantMIME {
t.Errorf("ContentType = %q, want %q", resp.ContentType, tt.wantMIME)
@@ -376,14 +341,12 @@ func TestService_Get_FormatConversion(t *testing.T) {
}
func TestService_Get_Caching(t *testing.T) {
t.Parallel()
svc, fixtures := SetupTestService(t)
ctx := context.Background()
req := &ImageRequest{
SourceHost: fixtures.GoodHost,
SourcePath: testPathPhoto,
SourcePath: "/images/photo.jpg",
Size: Size{Width: 50, Height: 50},
Format: FormatJPEG,
Quality: 85,
@@ -404,8 +367,7 @@ func TestService_Get_Caching(t *testing.T) {
if err != nil {
t.Fatalf("failed to read first response: %v", err)
}
_ = resp1.Content.Close()
resp1.Content.Close()
// Second request - should be a cache hit
resp2, err := svc.Get(ctx, req)
@@ -421,8 +383,7 @@ func TestService_Get_Caching(t *testing.T) {
if err != nil {
t.Fatalf("failed to read second response: %v", err)
}
_ = resp2.Content.Close()
resp2.Content.Close()
// Content should be identical
if len(data1) != len(data2) {
@@ -431,8 +392,6 @@ func TestService_Get_Caching(t *testing.T) {
}
func TestService_Get_DifferentSizes(t *testing.T) {
t.Parallel()
svc, fixtures := SetupTestService(t)
ctx := context.Background()
@@ -443,12 +402,12 @@ func TestService_Get_DifferentSizes(t *testing.T) {
{Width: 75, Height: 75},
}
responses := make([][]byte, 0, len(sizes))
var responses [][]byte
for _, size := range sizes {
req := &ImageRequest{
SourceHost: fixtures.GoodHost,
SourcePath: testPathPhoto,
SourcePath: "/images/photo.jpg",
Size: size,
Format: FormatJPEG,
Quality: 85,
@@ -464,31 +423,27 @@ func TestService_Get_DifferentSizes(t *testing.T) {
if err != nil {
t.Fatalf("failed to read response: %v", err)
}
_ = resp.Content.Close()
resp.Content.Close()
responses = append(responses, data)
}
// All responses should be different sizes (different cache entries)
for i := range len(responses) - 1 {
for i := 0; i < len(responses)-1; i++ {
if len(responses[i]) == len(responses[i+1]) {
// Not necessarily an error, but worth noting
t.Logf("responses %d and %d have same size: %d bytes",
i, i+1, len(responses[i]))
t.Logf("responses %d and %d have same size: %d bytes", i, i+1, len(responses[i]))
}
}
}
func TestService_ValidateRequest_NoSigningKey(t *testing.T) {
t.Parallel()
// Service with no signing key - all non-allowlisted requests should fail
svc, fixtures := SetupTestService(t, WithNoAllowlist())
req := &ImageRequest{
SourceHost: fixtures.OtherHost,
SourcePath: testPathUpload,
SourcePath: "/uploads/image.jpg",
Size: Size{Width: 50, Height: 50},
Format: FormatJPEG,
Quality: 85,
@@ -497,15 +452,11 @@ func TestService_ValidateRequest_NoSigningKey(t *testing.T) {
err := svc.ValidateRequest(req)
if err == nil {
t.Error(
"ValidateRequest() expected error when no signing key and host not allowlisted",
)
t.Error("ValidateRequest() expected error when no signing key and host not allowlisted")
}
}
func TestService_Get_ContextCancellation(t *testing.T) {
t.Parallel()
svc, fixtures := SetupTestService(t)
ctx, cancel := context.WithCancel(context.Background())
@@ -513,7 +464,7 @@ func TestService_Get_ContextCancellation(t *testing.T) {
req := &ImageRequest{
SourceHost: fixtures.GoodHost,
SourcePath: testPathPhoto,
SourcePath: "/images/photo.jpg",
Size: Size{Width: 50, Height: 50},
Format: FormatJPEG,
Quality: 85,
@@ -527,14 +478,12 @@ func TestService_Get_ContextCancellation(t *testing.T) {
}
func TestService_Get_ReturnsETag(t *testing.T) {
t.Parallel()
svc, fixtures := SetupTestService(t)
ctx := context.Background()
req := &ImageRequest{
SourceHost: fixtures.GoodHost,
SourcePath: testPathPhoto,
SourcePath: "/images/photo.jpg",
Size: Size{Width: 50, Height: 50},
Format: FormatJPEG,
Quality: 85,
@@ -545,8 +494,7 @@ func TestService_Get_ReturnsETag(t *testing.T) {
if err != nil {
t.Fatalf("Get() error = %v", err)
}
defer func() { _ = resp.Content.Close() }()
defer resp.Content.Close()
// ETag should be set
if resp.ETag == "" {
@@ -560,14 +508,12 @@ func TestService_Get_ReturnsETag(t *testing.T) {
}
func TestService_Get_ETagConsistency(t *testing.T) {
t.Parallel()
svc, fixtures := SetupTestService(t)
ctx := context.Background()
req := &ImageRequest{
SourceHost: fixtures.GoodHost,
SourcePath: testPathPhoto,
SourcePath: "/images/photo.jpg",
Size: Size{Width: 50, Height: 50},
Format: FormatJPEG,
Quality: 85,
@@ -579,20 +525,16 @@ func TestService_Get_ETagConsistency(t *testing.T) {
if err != nil {
t.Fatalf("Get() first request error = %v", err)
}
etag1 := resp1.ETag
_ = resp1.Content.Close()
resp1.Content.Close()
// Second request (from cache)
resp2, err := svc.Get(ctx, req)
if err != nil {
t.Fatalf("Get() second request error = %v", err)
}
etag2 := resp2.ETag
_ = resp2.Content.Close()
resp2.Content.Close()
// ETags should be identical for the same content
if etag1 != etag2 {
@@ -601,15 +543,13 @@ func TestService_Get_ETagConsistency(t *testing.T) {
}
func TestService_Get_DifferentETagsForDifferentContent(t *testing.T) {
t.Parallel()
svc, fixtures := SetupTestService(t)
ctx := context.Background()
// Request same image at different sizes - should get different ETags
req1 := &ImageRequest{
SourceHost: fixtures.GoodHost,
SourcePath: testPathPhoto,
SourcePath: "/images/photo.jpg",
Size: Size{Width: 25, Height: 25},
Format: FormatJPEG,
Quality: 85,
@@ -618,7 +558,7 @@ func TestService_Get_DifferentETagsForDifferentContent(t *testing.T) {
req2 := &ImageRequest{
SourceHost: fixtures.GoodHost,
SourcePath: testPathPhoto,
SourcePath: "/images/photo.jpg",
Size: Size{Width: 50, Height: 50},
Format: FormatJPEG,
Quality: 85,
@@ -629,19 +569,15 @@ func TestService_Get_DifferentETagsForDifferentContent(t *testing.T) {
if err != nil {
t.Fatalf("Get() first request error = %v", err)
}
etag1 := resp1.ETag
_ = resp1.Content.Close()
resp1.Content.Close()
resp2, err := svc.Get(ctx, req2)
if err != nil {
t.Fatalf("Get() second request error = %v", err)
}
etag2 := resp2.ETag
_ = resp2.Content.Close()
resp2.Content.Close()
// ETags should be different for different content
if etag1 == etag2 {

View File

@@ -3,16 +3,13 @@ package imgcache
import "testing"
func TestImageRequest_SourceURL_DefaultHTTPS(t *testing.T) {
t.Parallel()
req := &ImageRequest{
SourceHost: testHostCDN,
SourcePath: testPathCat,
SourceHost: "cdn.example.com",
SourcePath: "/photos/cat.jpg",
SourceQuery: "v=2",
}
got := req.SourceURL()
want := "https://cdn.example.com/photos/cat.jpg?v=2"
if got != want {
t.Errorf("SourceURL() = %q, want %q", got, want)
@@ -20,16 +17,13 @@ func TestImageRequest_SourceURL_DefaultHTTPS(t *testing.T) {
}
func TestImageRequest_SourceURL_AllowHTTP(t *testing.T) {
t.Parallel()
req := &ImageRequest{
SourceHost: "localhost:8080",
SourcePath: testPathCat,
SourcePath: "/photos/cat.jpg",
AllowHTTP: true,
}
got := req.SourceURL()
want := "http://localhost:8080/photos/cat.jpg"
if got != want {
t.Errorf("SourceURL() = %q, want %q", got, want)
@@ -37,10 +31,8 @@ func TestImageRequest_SourceURL_AllowHTTP(t *testing.T) {
}
func TestImageRequest_SourceURL_AllowHTTPFalse(t *testing.T) {
t.Parallel()
req := &ImageRequest{
SourceHost: testHostCDN,
SourceHost: "cdn.example.com",
SourcePath: "/img.jpg",
AllowHTTP: false,
}

View File

@@ -12,25 +12,18 @@ import (
func setupStatsTestDB(t *testing.T) *sql.DB {
t.Helper()
db, err := sql.Open("sqlite", ":memory:")
if err != nil {
t.Fatal(err)
}
err = database.ApplyMigrations(context.Background(), db, nil)
if err != nil {
if err := database.ApplyMigrations(context.Background(), db, nil); err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = db.Close() })
t.Cleanup(func() { db.Close() })
return db
}
func TestStats_HitRateIsRatio(t *testing.T) {
t.Parallel()
db := setupStatsTestDB(t)
dir := t.TempDir()
@@ -47,9 +40,7 @@ func TestStats_HitRateIsRatio(t *testing.T) {
// Set some hit/miss counts and a transform_count
_, err = db.ExecContext(ctx, `
UPDATE cache_stats
SET hit_count = 75, miss_count = 25, transform_count = 9999
WHERE id = 1
UPDATE cache_stats SET hit_count = 75, miss_count = 25, transform_count = 9999 WHERE id = 1
`)
if err != nil {
t.Fatal(err)
@@ -63,7 +54,6 @@ func TestStats_HitRateIsRatio(t *testing.T) {
if stats.HitCount != 75 {
t.Errorf("HitCount = %d, want 75", stats.HitCount)
}
if stats.MissCount != 25 {
t.Errorf("MissCount = %d, want 25", stats.MissCount)
}
@@ -71,14 +61,11 @@ func TestStats_HitRateIsRatio(t *testing.T) {
// HitRate should be 0.75, NOT 9999 (transform_count)
expectedRate := 0.75
if math.Abs(stats.HitRate-expectedRate) > 0.001 {
t.Errorf("HitRate = %f, want %f (was it scanning transform_count?)",
stats.HitRate, expectedRate)
t.Errorf("HitRate = %f, want %f (was it scanning transform_count?)", stats.HitRate, expectedRate)
}
}
func TestStats_ZeroCounts(t *testing.T) {
t.Parallel()
db := setupStatsTestDB(t)
dir := t.TempDir()

View File

@@ -44,8 +44,7 @@ type ContentStorage struct {
// NewContentStorage creates a new content storage at the given base directory.
func NewContentStorage(baseDir string) (*ContentStorage, error) {
err := os.MkdirAll(baseDir, StorageDirPerm)
if err != nil {
if err := os.MkdirAll(baseDir, StorageDirPerm); err != nil {
return nil, fmt.Errorf("failed to create storage directory: %w", err)
}
@@ -54,7 +53,7 @@ func NewContentStorage(baseDir string) (*ContentStorage, error) {
// Store writes content to storage and returns its SHA256 hash.
// The content is read fully into memory to compute the hash before writing.
func (s *ContentStorage) Store(r io.Reader) (ContentHash, int64, error) {
func (s *ContentStorage) Store(r io.Reader) (hash ContentHash, size int64, err error) {
// Read all content to compute hash
data, err := io.ReadAll(r)
if err != nil {
@@ -63,23 +62,20 @@ func (s *ContentStorage) Store(r io.Reader) (ContentHash, int64, error) {
// Compute hash
h := sha256.Sum256(data)
hash := ContentHash(hex.EncodeToString(h[:]))
size := int64(len(data))
hash = ContentHash(hex.EncodeToString(h[:]))
size = int64(len(data))
// Build path: <basedir>/<ab>/<cd>/<hash>
path := s.hashToPath(hash)
// Check if already exists
_, err = os.Stat(path)
if err == nil {
if _, err := os.Stat(path); err == nil {
return hash, size, nil
}
// Create directory structure
dir := filepath.Dir(path)
err = os.MkdirAll(dir, StorageDirPerm)
if err != nil {
if err := os.MkdirAll(dir, StorageDirPerm); err != nil {
return "", 0, fmt.Errorf("failed to create directory: %w", err)
}
@@ -88,29 +84,27 @@ func (s *ContentStorage) Store(r io.Reader) (ContentHash, int64, error) {
if err != nil {
return "", 0, fmt.Errorf("failed to create temp file: %w", err)
}
tmpPath := tmpFile.Name()
_, err = tmpFile.Write(data)
defer func() {
if err != nil {
_ = tmpFile.Close()
_ = os.Remove(tmpPath)
}
}()
if _, err := tmpFile.Write(data); err != nil {
_ = tmpFile.Close()
return "", 0, fmt.Errorf("failed to write content: %w", err)
}
err = tmpFile.Close()
if err != nil {
_ = os.Remove(tmpPath)
if err := tmpFile.Close(); err != nil {
return "", 0, fmt.Errorf("failed to close temp file: %w", err)
}
// Atomic rename
err = os.Rename(filepath.Clean(tmpPath), filepath.Clean(path))
if err != nil {
_ = os.Remove(tmpPath)
//nolint:gosec // G703: paths from internal SHA256 hashes
if err := os.Rename(filepath.Clean(tmpPath), filepath.Clean(path)); err != nil {
return "", 0, fmt.Errorf("failed to rename temp file: %w", err)
}
@@ -194,8 +188,7 @@ type MetadataStorage struct {
// NewMetadataStorage creates a new metadata storage at the given base directory.
func NewMetadataStorage(baseDir string) (*MetadataStorage, error) {
err := os.MkdirAll(baseDir, StorageDirPerm)
if err != nil {
if err := os.MkdirAll(baseDir, StorageDirPerm); err != nil {
return nil, fmt.Errorf("failed to create metadata directory: %w", err)
}
@@ -203,8 +196,6 @@ func NewMetadataStorage(baseDir string) (*MetadataStorage, error) {
}
// SourceMetadata represents cached metadata about a source URL.
//
//nolint:tagliatelle // stored metadata format uses snake_case
type SourceMetadata struct {
Host string `json:"host"`
Path string `json:"path"`
@@ -223,16 +214,12 @@ type SourceMetadata struct {
}
// Store writes metadata to storage.
func (s *MetadataStorage) Store(
host string, pathHash PathHash, meta *SourceMetadata,
) error {
func (s *MetadataStorage) Store(host string, pathHash PathHash, meta *SourceMetadata) error {
path := s.metaPath(host, pathHash)
// Create directory structure
dir := filepath.Dir(path)
err := os.MkdirAll(dir, StorageDirPerm)
if err != nil {
if err := os.MkdirAll(dir, StorageDirPerm); err != nil {
return fmt.Errorf("failed to create directory: %w", err)
}
@@ -247,29 +234,27 @@ func (s *MetadataStorage) Store(
if err != nil {
return fmt.Errorf("failed to create temp file: %w", err)
}
tmpPath := tmpFile.Name()
_, err = tmpFile.Write(data)
defer func() {
if err != nil {
_ = tmpFile.Close()
_ = os.Remove(tmpPath)
}
}()
if _, err := tmpFile.Write(data); err != nil {
_ = tmpFile.Close()
return fmt.Errorf("failed to write metadata: %w", err)
}
err = tmpFile.Close()
if err != nil {
_ = os.Remove(tmpPath)
if err := tmpFile.Close(); err != nil {
return fmt.Errorf("failed to close temp file: %w", err)
}
// Atomic rename
err = os.Rename(filepath.Clean(tmpPath), filepath.Clean(path))
if err != nil {
_ = os.Remove(tmpPath)
//nolint:gosec // G703: paths from internal SHA256 hashes
if err := os.Rename(filepath.Clean(tmpPath), filepath.Clean(path)); err != nil {
return fmt.Errorf("failed to rename temp file: %w", err)
}
@@ -277,9 +262,7 @@ func (s *MetadataStorage) Store(
}
// Load reads metadata from storage.
func (s *MetadataStorage) Load(
host string, pathHash PathHash,
) (*SourceMetadata, error) {
func (s *MetadataStorage) Load(host string, pathHash PathHash) (*SourceMetadata, error) {
path := s.metaPath(host, pathHash)
data, err := os.ReadFile(path) //nolint:gosec // path derived from host+hash
@@ -292,9 +275,7 @@ func (s *MetadataStorage) Load(
}
var meta SourceMetadata
err = json.Unmarshal(data, &meta)
if err != nil {
if err := json.Unmarshal(data, &meta); err != nil {
return nil, fmt.Errorf("failed to unmarshal metadata: %w", err)
}
@@ -360,8 +341,6 @@ type VariantStorage struct {
}
// VariantMeta contains metadata about a cached variant.
//
//nolint:tagliatelle // stored metadata format uses snake_case
type VariantMeta struct {
ContentType string `json:"content_type"`
Size int64 `json:"size"`
@@ -370,8 +349,7 @@ type VariantMeta struct {
// NewVariantStorage creates a new variant storage at the given base directory.
func NewVariantStorage(baseDir string) (*VariantStorage, error) {
err := os.MkdirAll(baseDir, StorageDirPerm)
if err != nil {
if err := os.MkdirAll(baseDir, StorageDirPerm); err != nil {
return nil, fmt.Errorf("failed to create variant storage directory: %w", err)
}
@@ -379,23 +357,19 @@ func NewVariantStorage(baseDir string) (*VariantStorage, error) {
}
// Store writes content and metadata to storage at the given key.
func (s *VariantStorage) Store(
key VariantKey, r io.Reader, contentType string,
) (int64, error) {
func (s *VariantStorage) Store(key VariantKey, r io.Reader, contentType string) (size int64, err error) {
data, err := io.ReadAll(r)
if err != nil {
return 0, fmt.Errorf("failed to read content: %w", err)
}
size := int64(len(data))
size = int64(len(data))
path := s.keyToPath(key)
metaPath := path + ".meta"
// Create directory structure
dir := filepath.Dir(path)
err = os.MkdirAll(dir, StorageDirPerm)
if err != nil {
if err := os.MkdirAll(dir, StorageDirPerm); err != nil {
return 0, fmt.Errorf("failed to create directory: %w", err)
}
@@ -404,29 +378,27 @@ func (s *VariantStorage) Store(
if err != nil {
return 0, fmt.Errorf("failed to create temp file: %w", err)
}
tmpPath := tmpFile.Name()
_, err = tmpFile.Write(data)
defer func() {
if err != nil {
_ = tmpFile.Close()
_ = os.Remove(tmpPath)
}
}()
if _, err := tmpFile.Write(data); err != nil {
_ = tmpFile.Close()
return 0, fmt.Errorf("failed to write content: %w", err)
}
err = tmpFile.Close()
if err != nil {
_ = os.Remove(tmpPath)
if err := tmpFile.Close(); err != nil {
return 0, fmt.Errorf("failed to close temp file: %w", err)
}
// Atomic rename content
err = os.Rename(filepath.Clean(tmpPath), filepath.Clean(path))
if err != nil {
_ = os.Remove(tmpPath)
//nolint:gosec // G703: paths from internal SHA256 hashes
if err := os.Rename(filepath.Clean(tmpPath), filepath.Clean(path)); err != nil {
return 0, fmt.Errorf("failed to rename temp file: %w", err)
}
@@ -442,8 +414,10 @@ func (s *VariantStorage) Store(
return 0, fmt.Errorf("failed to marshal metadata: %w", err)
}
// Metadata write failure is non-fatal; content is already stored.
_ = os.WriteFile(metaPath, metaData, StorageFilePerm)
if err := os.WriteFile(metaPath, metaData, StorageFilePerm); err != nil {
// Non-fatal, content is stored
_ = err
}
return size, nil
}
@@ -464,11 +438,8 @@ func (s *VariantStorage) Load(key VariantKey) (io.ReadCloser, error) {
return f, nil
}
// LoadWithMeta returns a reader, size, and content type for the content at
// the given key.
func (s *VariantStorage) LoadWithMeta(
key VariantKey,
) (io.ReadCloser, int64, string, error) {
// LoadWithMeta returns a reader, size, and content type for the content at the given key.
func (s *VariantStorage) LoadWithMeta(key VariantKey) (io.ReadCloser, int64, string, error) {
path := s.keyToPath(key)
metaPath := path + ".meta"
@@ -522,6 +493,24 @@ func (s *VariantStorage) Delete(key VariantKey) error {
return nil
}
// DeleteWithMeta removes the content at the given key together with
// its .meta sidecar file. A missing file is not an error.
func (s *VariantStorage) DeleteWithMeta(key VariantKey) error {
if err := s.Delete(key); err != nil {
return err
}
metaPath := s.keyToPath(key) + ".meta"
//nolint:gosec // G703: path derived from cache key
err := os.Remove(metaPath)
if err != nil && !os.IsNotExist(err) {
return fmt.Errorf("failed to delete variant metadata: %w", err)
}
return nil
}
// keyToPath converts a key to a file path: <basedir>/<ab>/<cd>/<key>
func (s *VariantStorage) keyToPath(key VariantKey) string {
k := string(key)

View File

@@ -2,7 +2,6 @@ package imgcache
import (
"bytes"
"errors"
"io"
"os"
"path/filepath"
@@ -10,17 +9,13 @@ import (
)
func TestContentStorage_StoreAndLoad(t *testing.T) {
t.Parallel()
tmpDir := t.TempDir()
storage, err := NewContentStorage(tmpDir)
if err != nil {
t.Fatalf("NewContentStorage() error = %v", err)
}
content := []byte("hello world")
hash, size, err := storage.Store(bytes.NewReader(content))
if err != nil {
t.Fatalf("Store() error = %v", err)
@@ -36,11 +31,8 @@ func TestContentStorage_StoreAndLoad(t *testing.T) {
// Verify file exists at expected path
hashStr := string(hash)
expectedPath := filepath.Join(tmpDir, hashStr[0:2], hashStr[2:4], hashStr)
_, err = os.Stat(expectedPath)
if err != nil {
if _, err := os.Stat(expectedPath); err != nil {
t.Errorf("File not at expected path %s: %v", expectedPath, err)
}
@@ -49,8 +41,7 @@ func TestContentStorage_StoreAndLoad(t *testing.T) {
if err != nil {
t.Fatalf("Load() error = %v", err)
}
defer func() { _ = r.Close() }()
defer r.Close()
loaded, err := io.ReadAll(r)
if err != nil {
@@ -63,10 +54,7 @@ func TestContentStorage_StoreAndLoad(t *testing.T) {
}
func TestContentStorage_StoreIdempotent(t *testing.T) {
t.Parallel()
tmpDir := t.TempDir()
storage, err := NewContentStorage(tmpDir)
if err != nil {
t.Fatalf("NewContentStorage() error = %v", err)
@@ -90,33 +78,26 @@ func TestContentStorage_StoreIdempotent(t *testing.T) {
}
func TestContentStorage_LoadNotFound(t *testing.T) {
t.Parallel()
tmpDir := t.TempDir()
storage, err := NewContentStorage(tmpDir)
if err != nil {
t.Fatalf("NewContentStorage() error = %v", err)
}
_, err = storage.Load(ContentHash("nonexistent"))
if !errors.Is(err, ErrNotFound) {
if err != ErrNotFound {
t.Errorf("Load() error = %v, want ErrNotFound", err)
}
}
func TestContentStorage_Delete(t *testing.T) {
t.Parallel()
tmpDir := t.TempDir()
storage, err := NewContentStorage(tmpDir)
if err != nil {
t.Fatalf("NewContentStorage() error = %v", err)
}
content := []byte("to be deleted")
hash, _, err := storage.Store(bytes.NewReader(content))
if err != nil {
t.Fatalf("Store() error = %v", err)
@@ -126,8 +107,7 @@ func TestContentStorage_Delete(t *testing.T) {
t.Error("Exists() = false, want true")
}
err = storage.Delete(hash)
if err != nil {
if err := storage.Delete(hash); err != nil {
t.Fatalf("Delete() error = %v", err)
}
@@ -137,27 +117,20 @@ func TestContentStorage_Delete(t *testing.T) {
}
func TestContentStorage_DeleteNonexistent(t *testing.T) {
t.Parallel()
tmpDir := t.TempDir()
storage, err := NewContentStorage(tmpDir)
if err != nil {
t.Fatalf("NewContentStorage() error = %v", err)
}
// Should not error
err = storage.Delete(ContentHash("nonexistent"))
if err != nil {
if err := storage.Delete(ContentHash("nonexistent")); err != nil {
t.Errorf("Delete() error = %v, want nil", err)
}
}
func TestContentStorage_HashToPath(t *testing.T) {
t.Parallel()
tmpDir := t.TempDir()
storage, err := NewContentStorage(tmpDir)
if err != nil {
t.Fatalf("NewContentStorage() error = %v", err)
@@ -165,59 +138,50 @@ func TestContentStorage_HashToPath(t *testing.T) {
// Test by storing and verifying the resulting path structure
content := []byte("test content for path verification")
hash, _, err := storage.Store(bytes.NewReader(content))
if err != nil {
t.Fatalf("Store() error = %v", err)
}
hashStr := string(hash)
expectedPath := filepath.Join(tmpDir, hashStr[0:2], hashStr[2:4], hashStr)
_, err = os.Stat(expectedPath)
if err != nil {
if _, err := os.Stat(expectedPath); err != nil {
t.Errorf("File not at expected path %s: %v", expectedPath, err)
}
}
func TestMetadataStorage_StoreAndLoad(t *testing.T) {
t.Parallel()
tmpDir := t.TempDir()
storage, err := NewMetadataStorage(tmpDir)
if err != nil {
t.Fatalf("NewMetadataStorage() error = %v", err)
}
meta := &SourceMetadata{
Host: testHostCDN,
Path: testPathCat,
Host: "cdn.example.com",
Path: "/photos/cat.jpg",
ContentHash: "abc123",
StatusCode: 200,
ContentType: testContentTypeJPEG,
ContentType: "image/jpeg",
FetchedAt: 1704067200,
ETag: `"etag123"`,
}
pathHash := HashPath(testPathCat)
pathHash := HashPath("/photos/cat.jpg")
err = storage.Store(testHostCDN, pathHash, meta)
err = storage.Store("cdn.example.com", pathHash, meta)
if err != nil {
t.Fatalf("Store() error = %v", err)
}
// Verify file exists at expected path
expectedPath := filepath.Join(tmpDir, testHostCDN, string(pathHash)+".json")
_, err = os.Stat(expectedPath)
if err != nil {
expectedPath := filepath.Join(tmpDir, "cdn.example.com", string(pathHash)+".json")
if _, err := os.Stat(expectedPath); err != nil {
t.Errorf("File not at expected path %s: %v", expectedPath, err)
}
// Load and verify
loaded, err := storage.Load(testHostCDN, pathHash)
loaded, err := storage.Load("cdn.example.com", pathHash)
if err != nil {
t.Fatalf("Load() error = %v", err)
}
@@ -244,64 +208,55 @@ func TestMetadataStorage_StoreAndLoad(t *testing.T) {
}
func TestMetadataStorage_LoadNotFound(t *testing.T) {
t.Parallel()
tmpDir := t.TempDir()
storage, err := NewMetadataStorage(tmpDir)
if err != nil {
t.Fatalf("NewMetadataStorage() error = %v", err)
}
_, err = storage.Load(testHostExample, PathHash("nonexistent"))
if !errors.Is(err, ErrNotFound) {
_, err = storage.Load("example.com", PathHash("nonexistent"))
if err != ErrNotFound {
t.Errorf("Load() error = %v, want ErrNotFound", err)
}
}
func TestMetadataStorage_Delete(t *testing.T) {
t.Parallel()
tmpDir := t.TempDir()
storage, err := NewMetadataStorage(tmpDir)
if err != nil {
t.Fatalf("NewMetadataStorage() error = %v", err)
}
meta := &SourceMetadata{
Host: testHostExample,
Host: "example.com",
Path: "/test.jpg",
StatusCode: 200,
}
pathHash := HashPath("/test.jpg")
err = storage.Store(testHostExample, pathHash, meta)
err = storage.Store("example.com", pathHash, meta)
if err != nil {
t.Fatalf("Store() error = %v", err)
}
if !storage.Exists(testHostExample, pathHash) {
if !storage.Exists("example.com", pathHash) {
t.Error("Exists() = false, want true")
}
err = storage.Delete(testHostExample, pathHash)
if err != nil {
if err := storage.Delete("example.com", pathHash); err != nil {
t.Fatalf("Delete() error = %v", err)
}
if storage.Exists(testHostExample, pathHash) {
if storage.Exists("example.com", pathHash) {
t.Error("Exists() = true after delete, want false")
}
}
func TestHashPath(t *testing.T) {
t.Parallel()
// Same input should produce same hash
hash1 := HashPath(testPathCat)
hash2 := HashPath(testPathCat)
hash1 := HashPath("/photos/cat.jpg")
hash2 := HashPath("/photos/cat.jpg")
if hash1 != hash2 {
t.Errorf("HashPath() not deterministic: %s vs %s", hash1, hash2)
@@ -321,11 +276,9 @@ func TestHashPath(t *testing.T) {
}
func TestCacheKey(t *testing.T) {
t.Parallel()
req1 := &ImageRequest{
SourceHost: testHostCDN,
SourcePath: testPathCat,
SourceHost: "cdn.example.com",
SourcePath: "/photos/cat.jpg",
SourceQuery: "",
Size: Size{Width: 800, Height: 600},
Format: FormatWebP,
@@ -334,8 +287,8 @@ func TestCacheKey(t *testing.T) {
}
req2 := &ImageRequest{
SourceHost: testHostCDN,
SourcePath: testPathCat,
SourceHost: "cdn.example.com",
SourcePath: "/photos/cat.jpg",
SourceQuery: "",
Size: Size{Width: 800, Height: 600},
Format: FormatWebP,
@@ -358,8 +311,8 @@ func TestCacheKey(t *testing.T) {
// Different size should produce different key
req3 := &ImageRequest{
SourceHost: testHostCDN,
SourcePath: testPathCat,
SourceHost: "cdn.example.com",
SourcePath: "/photos/cat.jpg",
SourceQuery: "",
Size: Size{Width: 400, Height: 300}, // Different size
Format: FormatWebP,
@@ -374,8 +327,8 @@ func TestCacheKey(t *testing.T) {
// Different format should produce different key
req4 := &ImageRequest{
SourceHost: testHostCDN,
SourcePath: testPathCat,
SourceHost: "cdn.example.com",
SourcePath: "/photos/cat.jpg",
SourceQuery: "",
Size: Size{Width: 800, Height: 600},
Format: FormatPNG, // Different format
@@ -390,8 +343,8 @@ func TestCacheKey(t *testing.T) {
// Different quality should produce different key
req5 := &ImageRequest{
SourceHost: testHostCDN,
SourcePath: testPathCat,
SourceHost: "cdn.example.com",
SourcePath: "/photos/cat.jpg",
SourceQuery: "",
Size: Size{Width: 800, Height: 600},
Format: FormatWebP,

View File

@@ -18,14 +18,6 @@ import (
"sneak.berlin/go/pixa/internal/httpfetcher"
)
// Shared test data literals, extracted as constants for goconst.
const (
testHostCDN = "cdn.example.com"
testHostExample = "example.com"
testPathCat = "/photos/cat.jpg"
testContentTypeJPEG = "image/jpeg"
)
// TestFixtures contains paths to test files in the mock filesystem.
type TestFixtures struct {
// Valid image files
@@ -97,16 +89,14 @@ func generateTestJPEG(t *testing.T, width, height int, c color.Color) []byte {
t.Helper()
img := image.NewRGBA(image.Rect(0, 0, width, height))
for y := range height {
for x := range width {
for y := 0; y < height; y++ {
for x := 0; x < width; x++ {
img.Set(x, y, c)
}
}
var buf bytes.Buffer
err := jpeg.Encode(&buf, img, &jpeg.Options{Quality: 85})
if err != nil {
if err := jpeg.Encode(&buf, img, &jpeg.Options{Quality: 85}); err != nil {
t.Fatalf("failed to encode test JPEG: %v", err)
}
@@ -118,16 +108,14 @@ func generateTestPNG(t *testing.T, width, height int, c color.Color) []byte {
t.Helper()
img := image.NewRGBA(image.Rect(0, 0, width, height))
for y := range height {
for x := range width {
for y := 0; y < height; y++ {
for x := 0; x < width; x++ {
img.Set(x, y, c)
}
}
var buf bytes.Buffer
err := png.Encode(&buf, img)
if err != nil {
if err := png.Encode(&buf, img); err != nil {
t.Fatalf("failed to encode test PNG: %v", err)
}
@@ -138,20 +126,15 @@ func generateTestPNG(t *testing.T, width, height int, c color.Color) []byte {
func generateTestGIF(t *testing.T, width, height int, c color.Color) []byte {
t.Helper()
img := image.NewPaletted(
image.Rect(0, 0, width, height),
[]color.Color{c, color.White},
)
for y := range height {
for x := range width {
img := image.NewPaletted(image.Rect(0, 0, width, height), []color.Color{c, color.White})
for y := 0; y < height; y++ {
for x := 0; x < width; x++ {
img.SetColorIndex(x, y, 0)
}
}
var buf bytes.Buffer
err := gif.Encode(&buf, img, nil)
if err != nil {
if err := gif.Encode(&buf, img, nil); err != nil {
t.Fatalf("failed to encode test GIF: %v", err)
}
@@ -159,9 +142,7 @@ func generateTestGIF(t *testing.T, width, height int, c color.Color) []byte {
}
// SetupTestService creates a Service with mock fetcher for testing.
func SetupTestService(
t *testing.T, opts ...TestServiceOption,
) (*Service, *TestFixtures) {
func SetupTestService(t *testing.T, opts ...TestServiceOption) (*Service, *TestFixtures) {
t.Helper()
mockFS, fixtures := NewTestFS(t)
@@ -214,8 +195,7 @@ func setupServiceTestDB(t *testing.T) *sql.DB {
}
// Use the real production schema via migrations
err = database.ApplyMigrations(context.Background(), db, nil)
if err != nil {
if err := database.ApplyMigrations(context.Background(), db, nil); err != nil {
t.Fatalf("failed to apply migrations: %v", err)
}

View File

@@ -40,8 +40,7 @@ type ParsedURL struct {
Format ImageFormat
}
// ParseImagePath parses the path captured by chi's wildcard:
// <host>/<path>/<size>.<format>
// ParseImagePath parses the path captured by chi's wildcard: <host>/<path>/<size>.<format>
// This is the primary entry point when using chi routing.
// Examples:
// - cdn.example.com/photos/cat.jpg/800x600.webp
@@ -77,8 +76,7 @@ func ParseImageURL(urlPath string) (*ParsedURL, error) {
// parseImageComponents parses <host>/<path>/<size>.<format> structure.
func parseImageComponents(remainder string) (*ParsedURL, error) {
// Check for path traversal before any other processing
err := checkPathTraversal(remainder)
if err != nil {
if err := checkPathTraversal(remainder); err != nil {
return nil, err
}
@@ -104,7 +102,6 @@ func parseImageComponents(remainder string) (*ParsedURL, error) {
// Split host from path
// The first segment is the host, everything after is the path
firstSlash := strings.Index(hostAndPath, "/")
var host, path, query string
if firstSlash == -1 {
@@ -184,7 +181,8 @@ func checkPathTraversal(path string) error {
// Also check for ".." as a path segment in the original path
// This catches cases where the path hasn't been normalized
for seg := range strings.SplitSeq(path, "/") {
segments := strings.Split(path, "/")
for _, seg := range segments {
// URL decode the segment
decodedSeg, _ := url.PathUnescape(seg)
decodedSeg = strings.ReplaceAll(decodedSeg, "\\", "/")
@@ -204,10 +202,8 @@ func parseSizeFormat(s string) (Size, ImageFormat, error) {
return Size{}, "", ErrInvalidSize
}
var (
size Size
formatStr string
)
var size Size
var formatStr string
if matches[4] == "orig" {
// "orig.format" pattern

View File

@@ -1,124 +1,93 @@
package imgcache
import (
"errors"
"testing"
)
// assertParsedURL compares all fields of a parsed URL against the
// expected value.
func assertParsedURL(t *testing.T, got, want *ParsedURL) {
t.Helper()
if got.Host != want.Host {
t.Errorf("Host = %q, want %q", got.Host, want.Host)
}
if got.Path != want.Path {
t.Errorf("Path = %q, want %q", got.Path, want.Path)
}
if got.Query != want.Query {
t.Errorf("Query = %q, want %q", got.Query, want.Query)
}
if got.Size != want.Size {
t.Errorf("Size = %v, want %v", got.Size, want.Size)
}
if got.Format != want.Format {
t.Errorf("Format = %q, want %q", got.Format, want.Format)
}
}
func TestParseImageURL(t *testing.T) {
t.Parallel()
tests := []struct {
name string
input string
want *ParsedURL
wantErr error
}{
{
name: "basic path with size",
input: "/v1/image/cdn.example.com/photos/cat.jpg/800x600.webp",
want: &ParsedURL{
Host: testHostCDN, Path: testPathCat,
Size: Size{Width: 800, Height: 600}, Format: FormatWebP,
Host: "cdn.example.com",
Path: "/photos/cat.jpg",
Query: "",
Size: Size{Width: 800, Height: 600},
Format: FormatWebP,
},
},
{
name: "original size with 0x0",
input: "/v1/image/cdn.example.com/photos/cat.jpg/0x0.jpeg",
want: &ParsedURL{
Host: testHostCDN, Path: testPathCat,
Size: Size{Width: 0, Height: 0}, Format: FormatJPEG,
Host: "cdn.example.com",
Path: "/photos/cat.jpg",
Query: "",
Size: Size{Width: 0, Height: 0},
Format: FormatJPEG,
},
},
{
name: "original size with orig keyword",
input: "/v1/image/cdn.example.com/photos/cat.jpg/orig.png",
want: &ParsedURL{
Host: testHostCDN, Path: testPathCat,
Size: Size{Width: 0, Height: 0}, Format: FormatPNG,
Host: "cdn.example.com",
Path: "/photos/cat.jpg",
Query: "",
Size: Size{Width: 0, Height: 0},
Format: FormatPNG,
},
},
{
name: "path with query string",
input: "/v1/image/cdn.example.com/photos/cat.jpg?arg1=val1&arg2=val2/800x600.webp",
want: &ParsedURL{
Host: testHostCDN, Path: testPathCat, Query: "arg1=val1&arg2=val2",
Size: Size{Width: 800, Height: 600}, Format: FormatWebP,
Host: "cdn.example.com",
Path: "/photos/cat.jpg",
Query: "arg1=val1&arg2=val2",
Size: Size{Width: 800, Height: 600},
Format: FormatWebP,
},
},
{
name: "deep nested path",
input: "/v1/image/cdn.example.com/a/b/c/d/image.jpg/1920x1080.avif",
want: &ParsedURL{
Host: testHostCDN, Path: "/a/b/c/d/image.jpg",
Size: Size{Width: 1920, Height: 1080}, Format: FormatAVIF,
Host: "cdn.example.com",
Path: "/a/b/c/d/image.jpg",
Query: "",
Size: Size{Width: 1920, Height: 1080},
Format: FormatAVIF,
},
},
{
name: "jpg alias for jpeg",
input: "/v1/image/example.com/img.png/100x100.jpg",
want: &ParsedURL{
Host: testHostExample, Path: "/img.png",
Size: Size{Width: 100, Height: 100}, Format: FormatJPEG,
Host: "example.com",
Path: "/img.png",
Query: "",
Size: Size{Width: 100, Height: 100},
Format: FormatJPEG,
},
},
{
name: "gif format",
input: "/v1/image/example.com/animated.gif/200x200.gif",
want: &ParsedURL{
Host: testHostExample, Path: "/animated.gif",
Size: Size{Width: 200, Height: 200}, Format: FormatGIF,
Host: "example.com",
Path: "/animated.gif",
Query: "",
Size: Size{Width: 200, Height: 200},
Format: FormatGIF,
},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
got, err := ParseImageURL(tt.input)
if err != nil {
t.Fatalf("ParseImageURL() unexpected error = %v", err)
}
assertParsedURL(t, got, tt.want)
})
}
}
func TestParseImageURL_Errors(t *testing.T) {
t.Parallel()
tests := []struct {
name string
input string
wantErr error
}{
{
name: "missing prefix",
input: "/image/cdn.example.com/photo.jpg/800x600.webp",
@@ -153,23 +122,47 @@ func TestParseImageURL_Errors(t *testing.T) {
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
got, err := ParseImageURL(tt.input)
_, err := ParseImageURL(tt.input)
if tt.wantErr != nil {
if err == nil {
t.Fatalf("ParseImageURL() error = nil, wantErr %v", tt.wantErr)
}
t.Errorf("ParseImageURL() error = nil, wantErr %v", tt.wantErr)
return
}
if !errorIs(err, tt.wantErr) {
t.Errorf("ParseImageURL() error = %v, wantErr %v", err, tt.wantErr)
}
return
}
if err != nil {
t.Errorf("ParseImageURL() unexpected error = %v", err)
return
}
if got.Host != tt.want.Host {
t.Errorf("Host = %q, want %q", got.Host, tt.want.Host)
}
if got.Path != tt.want.Path {
t.Errorf("Path = %q, want %q", got.Path, tt.want.Path)
}
if got.Query != tt.want.Query {
t.Errorf("Query = %q, want %q", got.Query, tt.want.Query)
}
if got.Size != tt.want.Size {
t.Errorf("Size = %v, want %v", got.Size, tt.want.Size)
}
if got.Format != tt.want.Format {
t.Errorf("Format = %q, want %q", got.Format, tt.want.Format)
}
})
}
}
func TestParseImagePath(t *testing.T) {
t.Parallel()
// ParseImagePath is for chi wildcard capture (no /v1/image/ prefix)
tests := []struct {
name string
@@ -181,8 +174,8 @@ func TestParseImagePath(t *testing.T) {
name: "chi wildcard capture",
input: "cdn.example.com/photos/cat.jpg/800x600.webp",
want: &ParsedURL{
Host: testHostCDN,
Path: testPathCat,
Host: "cdn.example.com",
Path: "/photos/cat.jpg",
Size: Size{Width: 800, Height: 600},
Format: FormatWebP,
},
@@ -191,8 +184,8 @@ func TestParseImagePath(t *testing.T) {
name: "with leading slash from chi",
input: "/cdn.example.com/photos/cat.jpg/800x600.webp",
want: &ParsedURL{
Host: testHostCDN,
Path: testPathCat,
Host: "cdn.example.com",
Path: "/photos/cat.jpg",
Size: Size{Width: 800, Height: 600},
Format: FormatWebP,
},
@@ -201,30 +194,35 @@ func TestParseImagePath(t *testing.T) {
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
got, err := ParseImagePath(tt.input)
if (err != nil) != tt.wantErr {
t.Errorf("ParseImagePath() error = %v, wantErr %v", err, tt.wantErr)
return
}
if err != nil {
return
}
assertParsedURL(t, got, tt.want)
if got.Host != tt.want.Host {
t.Errorf("Host = %q, want %q", got.Host, tt.want.Host)
}
if got.Path != tt.want.Path {
t.Errorf("Path = %q, want %q", got.Path, tt.want.Path)
}
if got.Size != tt.want.Size {
t.Errorf("Size = %v, want %v", got.Size, tt.want.Size)
}
if got.Format != tt.want.Format {
t.Errorf("Format = %q, want %q", got.Format, tt.want.Format)
}
})
}
}
func TestParsedURL_ToImageRequest(t *testing.T) {
t.Parallel()
parsed := &ParsedURL{
Host: testHostCDN,
Path: testPathCat,
Host: "cdn.example.com",
Path: "/photos/cat.jpg",
Query: "version=2",
Size: Size{Width: 800, Height: 600},
Format: FormatWebP,
@@ -235,27 +233,21 @@ func TestParsedURL_ToImageRequest(t *testing.T) {
if req.SourceHost != parsed.Host {
t.Errorf("SourceHost = %q, want %q", req.SourceHost, parsed.Host)
}
if req.SourcePath != parsed.Path {
t.Errorf("SourcePath = %q, want %q", req.SourcePath, parsed.Path)
}
if req.SourceQuery != parsed.Query {
t.Errorf("SourceQuery = %q, want %q", req.SourceQuery, parsed.Query)
}
if req.Size != parsed.Size {
t.Errorf("Size = %v, want %v", req.Size, parsed.Size)
}
if req.Format != parsed.Format {
t.Errorf("Format = %q, want %q", req.Format, parsed.Format)
}
}
func TestParseImageURL_PathTraversal(t *testing.T) {
t.Parallel()
// All path traversal attempts should be rejected
tests := []struct {
name string
@@ -301,14 +293,12 @@ func TestParseImageURL_PathTraversal(t *testing.T) {
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
_, err := ParseImageURL(tt.input)
if err == nil {
t.Error("ParseImageURL() should reject path traversal attempts")
}
if !errors.Is(err, ErrPathTraversal) {
if err != ErrPathTraversal {
t.Errorf("ParseImageURL() error = %v, want ErrPathTraversal", err)
}
})
@@ -316,8 +306,6 @@ func TestParseImageURL_PathTraversal(t *testing.T) {
}
func TestParseImagePath_PathTraversal(t *testing.T) {
t.Parallel()
// Test path traversal via ParseImagePath (chi wildcard)
tests := []struct {
name string
@@ -335,14 +323,12 @@ func TestParseImagePath_PathTraversal(t *testing.T) {
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
_, err := ParseImagePath(tt.input)
if err == nil {
t.Error("ParseImagePath() should reject path traversal attempts")
}
if !errors.Is(err, ErrPathTraversal) {
if err != ErrPathTraversal {
t.Errorf("ParseImagePath() error = %v, want ErrPathTraversal", err)
}
})
@@ -351,7 +337,7 @@ func TestParseImagePath_PathTraversal(t *testing.T) {
// errorIs checks if err matches target (handles wrapped errors).
func errorIs(err, target error) bool {
if errors.Is(err, target) {
if err == target {
return true
}
// Check if error message contains target message for wrapped errors

View File

@@ -15,7 +15,6 @@ import (
// Params defines dependencies for Logger.
type Params struct {
fx.In
Globals *globals.Globals
}

View File

@@ -46,10 +46,6 @@ const (
// MinMagicBytes is the minimum number of bytes needed to detect format.
const MinMagicBytes = 12
// mimeOctetStream is the fallback MIME type for formats without a
// specific MIME type.
const mimeOctetStream = "application/octet-stream"
// Magic byte signatures for supported formats.
// These are effectively constants but Go doesn't support const slices.
//
@@ -194,17 +190,14 @@ func IsSupportedMIMEType(mimeType string) bool {
func PeekAndValidate(r io.Reader, declaredType string) (io.Reader, error) {
// Read minimum bytes for detection
buf := make([]byte, MinMagicBytes)
n, err := io.ReadFull(r, buf)
if err != nil && !errors.Is(err, io.ErrUnexpectedEOF) {
if err != nil && err != io.ErrUnexpectedEOF {
return nil, err
}
buf = buf[:n]
// Validate magic bytes
err = ValidateMagicBytes(buf, declaredType)
if err != nil {
if err := ValidateMagicBytes(buf, declaredType); err != nil {
return nil, err
}
@@ -226,9 +219,6 @@ func MIMEToImageFormat(mimeType string) (ImageFormat, bool) {
return FormatGIF, true
case MIMETypeAVIF:
return FormatAVIF, true
case MIMETypeSVG:
// SVG has no corresponding output format.
return "", false
default:
return "", false
}
@@ -247,10 +237,7 @@ func ImageFormatToMIME(format ImageFormat) string {
return string(MIMETypeGIF)
case FormatAVIF:
return string(MIMETypeAVIF)
case FormatOriginal:
// Original format passes content through unchanged.
return mimeOctetStream
default:
return mimeOctetStream
return "application/octet-stream"
}
}

View File

@@ -2,90 +2,121 @@ package magic
import (
"bytes"
"errors"
"io"
"slices"
"strings"
"testing"
)
// Shared test fixture strings.
const (
testNameEmpty = "empty"
testMIMEJPEG = "image/jpeg"
testMIMEJPEGParams = "image/jpeg; charset=utf-8"
testMIMEPNG = "image/png"
testMIMEWebP = "image/webp"
testMIMEGIF = "image/gif"
testMIMEAVIF = "image/avif"
)
// pad appends zero bytes so data is comfortably above MinMagicBytes.
func pad(b ...byte) []byte {
return append(b, make([]byte, 100)...)
}
func TestDetectFormat(t *testing.T) {
t.Parallel()
jpeg := pad(0xFF, 0xD8, 0xFF, 0xE0, 0x00, 0x10, 0x4A, 0x46, 0x49, 0x46, 0x00, 0x01)
png := pad(0x89, 0x50, 0x4E, 0x47, 0x0D, 0x0A, 0x1A, 0x0A, 0x00, 0x00, 0x00, 0x0D)
gif87a := pad(0x47, 0x49, 0x46, 0x38, 0x37, 0x61, 0, 0, 0, 0, 0, 0)
gif89a := pad(0x47, 0x49, 0x46, 0x38, 0x39, 0x61, 0, 0, 0, 0, 0, 0)
// RIFF + size placeholder + WEBP
webp := pad(0x52, 0x49, 0x46, 0x46, 0, 0, 0, 0, 0x57, 0x45, 0x42, 0x50)
// box size + ftyp + brand
avif := pad(0x00, 0x00, 0x00, 0x1C, 0x66, 0x74, 0x79, 0x70, 0x61, 0x76, 0x69, 0x66)
avis := pad(0x00, 0x00, 0x00, 0x1C, 0x66, 0x74, 0x79, 0x70, 0x61, 0x76, 0x69, 0x73)
tests := []struct {
name string
data []byte
wantMIME MIMEType
wantErr error
}{
{name: "JPEG", data: jpeg, wantMIME: MIMETypeJPEG},
{name: "PNG", data: png, wantMIME: MIMETypePNG},
{name: "GIF87a", data: gif87a, wantMIME: MIMETypeGIF},
{name: "GIF89a", data: gif89a, wantMIME: MIMETypeGIF},
{name: "WebP", data: webp, wantMIME: MIMETypeWebP},
{name: "AVIF", data: avif, wantMIME: MIMETypeAVIF},
{name: "AVIF sequence", data: avis, wantMIME: MIMETypeAVIF},
{
name: "JPEG",
data: append([]byte{0xFF, 0xD8, 0xFF, 0xE0, 0x00, 0x10, 0x4A, 0x46, 0x49, 0x46, 0x00, 0x01}, make([]byte, 100)...),
wantMIME: MIMETypeJPEG,
wantErr: nil,
},
{
name: "PNG",
data: append([]byte{0x89, 0x50, 0x4E, 0x47, 0x0D, 0x0A, 0x1A, 0x0A, 0x00, 0x00, 0x00, 0x0D}, make([]byte, 100)...),
wantMIME: MIMETypePNG,
wantErr: nil,
},
{
name: "GIF87a",
data: append([]byte{0x47, 0x49, 0x46, 0x38, 0x37, 0x61, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00}, make([]byte, 100)...),
wantMIME: MIMETypeGIF,
wantErr: nil,
},
{
name: "GIF89a",
data: append([]byte{0x47, 0x49, 0x46, 0x38, 0x39, 0x61, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00}, make([]byte, 100)...),
wantMIME: MIMETypeGIF,
wantErr: nil,
},
{
name: "WebP",
data: append([]byte{
0x52, 0x49, 0x46, 0x46, // RIFF
0x00, 0x00, 0x00, 0x00, // file size (placeholder)
0x57, 0x45, 0x42, 0x50, // WEBP
}, make([]byte, 100)...),
wantMIME: MIMETypeWebP,
wantErr: nil,
},
{
name: "AVIF",
data: append([]byte{
0x00, 0x00, 0x00, 0x1C, // box size
0x66, 0x74, 0x79, 0x70, // ftyp
0x61, 0x76, 0x69, 0x66, // avif brand
}, make([]byte, 100)...),
wantMIME: MIMETypeAVIF,
wantErr: nil,
},
{
name: "AVIF sequence",
data: append([]byte{
0x00, 0x00, 0x00, 0x1C, // box size
0x66, 0x74, 0x79, 0x70, // ftyp
0x61, 0x76, 0x69, 0x73, // avis brand
}, make([]byte, 100)...),
wantMIME: MIMETypeAVIF,
wantErr: nil,
},
{
name: "SVG with XML declaration",
data: []byte(`<?xml version="1.0"?><svg></svg>`),
wantMIME: MIMETypeSVG,
wantErr: nil,
},
{
name: "SVG without declaration",
data: []byte(`<svg xmlns="http://www.w3.org/2000/svg"></svg>`),
wantMIME: MIMETypeSVG,
wantErr: nil,
},
{
name: "SVG with whitespace",
data: []byte(` <?xml version="1.0"?><svg></svg>`),
wantMIME: MIMETypeSVG,
wantErr: nil,
},
{
name: "SVG with BOM",
data: append([]byte{0xEF, 0xBB, 0xBF}, []byte(`<svg></svg>`)...),
wantMIME: MIMETypeSVG,
wantErr: nil,
},
{
name: "unknown format",
data: make([]byte, MinMagicBytes),
data: []byte{0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00},
wantMIME: "",
wantErr: ErrUnknownFormat,
},
{name: "too short", data: []byte{0xFF, 0xD8}, wantErr: ErrNotEnoughData},
{name: testNameEmpty, data: []byte{}, wantErr: ErrNotEnoughData},
{
name: "too short",
data: []byte{0xFF, 0xD8},
wantMIME: "",
wantErr: ErrNotEnoughData,
},
{
name: "empty",
data: []byte{},
wantMIME: "",
wantErr: ErrNotEnoughData,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
got, err := DetectFormat(tt.data)
if !errors.Is(err, tt.wantErr) {
if err != tt.wantErr {
t.Errorf("DetectFormat() error = %v, wantErr %v", err, tt.wantErr)
return
@@ -99,10 +130,8 @@ func TestDetectFormat(t *testing.T) {
}
func TestValidateMagicBytes(t *testing.T) {
t.Parallel()
jpegData := pad(0xFF, 0xD8, 0xFF, 0xE0, 0x00, 0x10, 0x4A, 0x46, 0x49, 0x46, 0x00, 0x01)
pngData := pad(0x89, 0x50, 0x4E, 0x47, 0x0D, 0x0A, 0x1A, 0x0A, 0x00, 0x00, 0x00, 0x0D)
jpegData := append([]byte{0xFF, 0xD8, 0xFF, 0xE0, 0x00, 0x10, 0x4A, 0x46, 0x49, 0x46, 0x00, 0x01}, make([]byte, 100)...)
pngData := append([]byte{0x89, 0x50, 0x4E, 0x47, 0x0D, 0x0A, 0x1A, 0x0A, 0x00, 0x00, 0x00, 0x0D}, make([]byte, 100)...)
tests := []struct {
name string
@@ -113,42 +142,40 @@ func TestValidateMagicBytes(t *testing.T) {
{
name: "matching JPEG",
data: jpegData,
declaredType: testMIMEJPEG,
declaredType: "image/jpeg",
wantErr: nil,
},
{
name: "matching JPEG with params",
data: jpegData,
declaredType: testMIMEJPEGParams,
declaredType: "image/jpeg; charset=utf-8",
wantErr: nil,
},
{
name: "matching PNG",
data: pngData,
declaredType: testMIMEPNG,
declaredType: "image/png",
wantErr: nil,
},
{
name: "mismatched type",
data: jpegData,
declaredType: testMIMEPNG,
declaredType: "image/png",
wantErr: ErrMagicByteMismatch,
},
{
name: "unknown data",
data: make([]byte, MinMagicBytes),
declaredType: testMIMEJPEG,
data: []byte{0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00},
declaredType: "image/jpeg",
wantErr: ErrUnknownFormat,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
err := ValidateMagicBytes(tt.data, tt.declaredType)
if !errors.Is(err, tt.wantErr) {
if err != tt.wantErr {
t.Errorf("ValidateMagicBytes() error = %v, wantErr %v", err, tt.wantErr)
}
})
@@ -156,31 +183,27 @@ func TestValidateMagicBytes(t *testing.T) {
}
func TestIsSupportedMIMEType(t *testing.T) {
t.Parallel()
tests := []struct {
mimeType string
want bool
}{
{testMIMEJPEG, true},
{testMIMEPNG, true},
{testMIMEWebP, true},
{testMIMEGIF, true},
{testMIMEAVIF, true},
{"image/jpeg", true},
{"image/png", true},
{"image/webp", true},
{"image/gif", true},
{"image/avif", true},
{"image/svg+xml", true},
{"IMAGE/JPEG", true},
{testMIMEJPEGParams, true},
{"image/jpeg; charset=utf-8", true},
{"image/tiff", false},
{"image/bmp", false},
{mimeOctetStream, false},
{"application/octet-stream", false},
{"text/plain", false},
{"", false},
}
for _, tt := range tests {
t.Run(tt.mimeType, func(t *testing.T) {
t.Parallel()
if got := IsSupportedMIMEType(tt.mimeType); got != tt.want {
t.Errorf("IsSupportedMIMEType(%q) = %v, want %v", tt.mimeType, got, tt.want)
}
@@ -189,16 +212,8 @@ func TestIsSupportedMIMEType(t *testing.T) {
}
func TestPeekAndValidate(t *testing.T) {
t.Parallel()
jpegMagic := []byte{
0xFF, 0xD8, 0xFF, 0xE0, 0x00, 0x10, 0x4A, 0x46, 0x49, 0x46, 0x00, 0x01,
}
pngMagic := []byte{
0x89, 0x50, 0x4E, 0x47, 0x0D, 0x0A, 0x1A, 0x0A, 0x00, 0x00, 0x00, 0x0D,
}
jpegData := slices.Concat(jpegMagic, []byte("rest of jpeg data"))
pngData := slices.Concat(pngMagic, []byte("rest of png data"))
jpegData := append([]byte{0xFF, 0xD8, 0xFF, 0xE0, 0x00, 0x10, 0x4A, 0x46, 0x49, 0x46, 0x00, 0x01}, []byte("rest of jpeg data")...)
pngData := append([]byte{0x89, 0x50, 0x4E, 0x47, 0x0D, 0x0A, 0x1A, 0x0A, 0x00, 0x00, 0x00, 0x0D}, []byte("rest of png data")...)
tests := []struct {
name string
@@ -210,32 +225,30 @@ func TestPeekAndValidate(t *testing.T) {
{
name: "valid JPEG",
data: jpegData,
declaredType: testMIMEJPEG,
declaredType: "image/jpeg",
wantErr: false,
wantData: jpegData,
},
{
name: "valid PNG",
data: pngData,
declaredType: testMIMEPNG,
declaredType: "image/png",
wantErr: false,
wantData: pngData,
},
{
name: "mismatched type",
data: jpegData,
declaredType: testMIMEPNG,
declaredType: "image/png",
wantErr: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
r := bytes.NewReader(tt.data)
result, err := PeekAndValidate(r, tt.declaredType)
if tt.wantErr {
if err == nil {
t.Error("PeekAndValidate() expected error, got nil")
@@ -259,28 +272,23 @@ func TestPeekAndValidate(t *testing.T) {
}
if !bytes.Equal(got, tt.wantData) {
t.Errorf(
"PeekAndValidate() data mismatch: got %d bytes, want %d bytes",
len(got), len(tt.wantData),
)
t.Errorf("PeekAndValidate() data mismatch: got %d bytes, want %d bytes", len(got), len(tt.wantData))
}
})
}
}
func TestMIMEToImageFormat(t *testing.T) {
t.Parallel()
tests := []struct {
mimeType string
wantFormat ImageFormat
wantOk bool
}{
{testMIMEJPEG, FormatJPEG, true},
{testMIMEPNG, FormatPNG, true},
{testMIMEWebP, FormatWebP, true},
{testMIMEGIF, FormatGIF, true},
{testMIMEAVIF, FormatAVIF, true},
{"image/jpeg", FormatJPEG, true},
{"image/png", FormatPNG, true},
{"image/webp", FormatWebP, true},
{"image/gif", FormatGIF, true},
{"image/avif", FormatAVIF, true},
{"image/svg+xml", "", false}, // SVG doesn't convert to ImageFormat
{"image/tiff", "", false},
{"text/plain", "", false},
@@ -288,8 +296,6 @@ func TestMIMEToImageFormat(t *testing.T) {
for _, tt := range tests {
t.Run(tt.mimeType, func(t *testing.T) {
t.Parallel()
got, ok := MIMEToImageFormat(tt.mimeType)
if ok != tt.wantOk {
@@ -304,25 +310,21 @@ func TestMIMEToImageFormat(t *testing.T) {
}
func TestImageFormatToMIME(t *testing.T) {
t.Parallel()
tests := []struct {
format ImageFormat
wantMIME string
}{
{FormatJPEG, testMIMEJPEG},
{FormatPNG, testMIMEPNG},
{FormatWebP, testMIMEWebP},
{FormatGIF, testMIMEGIF},
{FormatAVIF, testMIMEAVIF},
{FormatOriginal, mimeOctetStream},
{"unknown", mimeOctetStream},
{FormatJPEG, "image/jpeg"},
{FormatPNG, "image/png"},
{FormatWebP, "image/webp"},
{FormatGIF, "image/gif"},
{FormatAVIF, "image/avif"},
{FormatOriginal, "application/octet-stream"},
{"unknown", "application/octet-stream"},
}
for _, tt := range tests {
t.Run(string(tt.format), func(t *testing.T) {
t.Parallel()
got := ImageFormatToMIME(tt.format)
if got != tt.wantMIME {
@@ -333,23 +335,19 @@ func TestImageFormatToMIME(t *testing.T) {
}
func TestNormalizeMIMEType(t *testing.T) {
t.Parallel()
tests := []struct {
input string
want string
}{
{testMIMEJPEG, testMIMEJPEG},
{"IMAGE/JPEG", testMIMEJPEG},
{testMIMEJPEGParams, testMIMEJPEG},
{" image/jpeg ", testMIMEJPEG},
{"image/jpeg; boundary=something", testMIMEJPEG},
{"image/jpeg", "image/jpeg"},
{"IMAGE/JPEG", "image/jpeg"},
{"image/jpeg; charset=utf-8", "image/jpeg"},
{" image/jpeg ", "image/jpeg"},
{"image/jpeg; boundary=something", "image/jpeg"},
}
for _, tt := range tests {
t.Run(tt.input, func(t *testing.T) {
t.Parallel()
got := normalizeMIMEType(tt.input)
if got != tt.want {
@@ -360,8 +358,6 @@ func TestNormalizeMIMEType(t *testing.T) {
}
func TestDetectSVG(t *testing.T) {
t.Parallel()
tests := []struct {
name string
data string
@@ -369,24 +365,17 @@ func TestDetectSVG(t *testing.T) {
}{
{"xml declaration", `<?xml version="1.0"?><svg></svg>`, true},
{"svg element", `<svg xmlns="http://www.w3.org/2000/svg"></svg>`, true},
{
"doctype",
`<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" ` +
`"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">`,
true,
},
{"doctype", `<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">`, true},
{"with whitespace", `
<?xml version="1.0"?><svg></svg>`, true},
{"uppercase", `<SVG></SVG>`, true},
{"not svg", `<html></html>`, false},
{"random text", `hello world`, false},
{testNameEmpty, ``, false},
{"empty", ``, false},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
got := detectSVG([]byte(tt.data))
if got != tt.want {
@@ -397,8 +386,6 @@ func TestDetectSVG(t *testing.T) {
}
func TestSkipBOM(t *testing.T) {
t.Parallel()
tests := []struct {
name string
data []byte
@@ -406,15 +393,13 @@ func TestSkipBOM(t *testing.T) {
}{
{"with BOM", []byte{0xEF, 0xBB, 0xBF, 'h', 'e', 'l', 'l', 'o'}, []byte("hello")},
{"without BOM", []byte("hello"), []byte("hello")},
{testNameEmpty, []byte{}, []byte{}},
{"empty", []byte{}, []byte{}},
{"only BOM", []byte{0xEF, 0xBB, 0xBF}, []byte{}},
{"partial BOM", []byte{0xEF, 0xBB}, []byte{0xEF, 0xBB}},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
got := skipBOM(tt.data)
if !bytes.Equal(got, tt.want) {
@@ -425,8 +410,6 @@ func TestSkipBOM(t *testing.T) {
}
func TestRealWorldSVGPatterns(t *testing.T) {
t.Parallel()
// Test various real-world SVG patterns
svgPatterns := []string{
`<?xml version="1.0" encoding="UTF-8"?>
@@ -436,8 +419,7 @@ func TestRealWorldSVGPatterns(t *testing.T) {
`<svg viewBox="0 0 24 24" fill="none" xmlns="http://www.w3.org/2000/svg">
<path d="M12 2L2 7l10 5 10-5-10-5z"/>
</svg>`,
`<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" ` +
`"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">` + `
`<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
<svg xmlns="http://www.w3.org/2000/svg">
</svg>`,
}
@@ -463,8 +445,6 @@ func TestRealWorldSVGPatterns(t *testing.T) {
}
func TestDetectFormatRIFFNotWebP(t *testing.T) {
t.Parallel()
// RIFF container but not WebP (e.g., WAV file)
wavData := []byte{
0x52, 0x49, 0x46, 0x46, // RIFF
@@ -473,14 +453,12 @@ func TestDetectFormatRIFFNotWebP(t *testing.T) {
}
_, err := DetectFormat(wavData)
if !errors.Is(err, ErrUnknownFormat) {
if err != ErrUnknownFormat {
t.Errorf("DetectFormat(WAV) error = %v, want %v", err, ErrUnknownFormat)
}
}
func TestDetectFormatFtypNotAVIF(t *testing.T) {
t.Parallel()
// ftyp container but not AVIF (e.g., MP4)
mp4Data := []byte{
0x00, 0x00, 0x00, 0x1C, // box size
@@ -489,24 +467,20 @@ func TestDetectFormatFtypNotAVIF(t *testing.T) {
}
_, err := DetectFormat(mp4Data)
if !errors.Is(err, ErrUnknownFormat) {
if err != ErrUnknownFormat {
t.Errorf("DetectFormat(MP4) error = %v, want %v", err, ErrUnknownFormat)
}
}
func TestPeekAndValidatePreservesReader(t *testing.T) {
t.Parallel()
// Ensure that after PeekAndValidate, we can read the complete
// original content
// Ensure that after PeekAndValidate, we can read the complete original content
originalContent := append(
[]byte{0x89, 0x50, 0x4E, 0x47, 0x0D, 0x0A, 0x1A, 0x0A, 0x00, 0x00, 0x00, 0x0D},
[]byte(strings.Repeat("PNG IDAT chunk data here ", 100))...,
)
r := bytes.NewReader(originalContent)
validated, err := PeekAndValidate(r, testMIMEPNG)
validated, err := PeekAndValidate(r, "image/png")
if err != nil {
t.Fatalf("PeekAndValidate() error = %v", err)
}
@@ -518,9 +492,6 @@ func TestPeekAndValidatePreservesReader(t *testing.T) {
}
if !bytes.Equal(got, originalContent) {
t.Errorf(
"Content mismatch: got %d bytes, want %d bytes",
len(got), len(originalContent),
)
t.Errorf("Content mismatch: got %d bytes, want %d bytes", len(got), len(originalContent))
}
}

View File

@@ -24,7 +24,6 @@ const CORSMaxAgeSeconds = 86400
// Params defines dependencies for Middleware.
type Params struct {
fx.In
Logger *logger.Logger
Config *config.Config
}
@@ -50,7 +49,6 @@ func ipFromHostPort(hp string) string {
if err != nil {
return ""
}
if len(h) > 0 && h[0] == '[' {
return h[1 : len(h)-1]
}
@@ -60,7 +58,6 @@ func ipFromHostPort(hp string) string {
type loggingResponseWriter struct {
http.ResponseWriter
statusCode int
bytesWritten int64
}
@@ -88,7 +85,6 @@ func (s *Middleware) Logging() func(http.Handler) http.Handler {
start := time.Now()
lrw := newLoggingResponseWriter(w)
ctx := r.Context()
defer func() {
latency := time.Since(start)
reqID, _ := ctx.Value(middleware.RequestIDKey).(string)

View File

@@ -10,8 +10,6 @@ import (
)
func TestSecurityHeaders(t *testing.T) {
t.Parallel()
// Create middleware instance
cfg := &config.Config{}
mw := &Middleware{
@@ -28,7 +26,7 @@ func TestSecurityHeaders(t *testing.T) {
handler := mw.SecurityHeaders()(testHandler)
// Make a test request
req := httptest.NewRequestWithContext(t.Context(), http.MethodGet, "/test", nil)
req := httptest.NewRequest(http.MethodGet, "/test", nil)
rec := httptest.NewRecorder()
handler.ServeHTTP(rec, req)
@@ -46,8 +44,6 @@ func TestSecurityHeaders(t *testing.T) {
for _, tt := range tests {
t.Run(tt.header, func(t *testing.T) {
t.Parallel()
got := rec.Header().Get(tt.header)
if got != tt.want {
t.Errorf("%s = %q, want %q", tt.header, got, tt.want)
@@ -57,8 +53,6 @@ func TestSecurityHeaders(t *testing.T) {
}
func TestSecurityHeaders_PreservesExistingHeaders(t *testing.T) {
t.Parallel()
cfg := &config.Config{}
mw := &Middleware{
log: slog.Default(),
@@ -74,7 +68,7 @@ func TestSecurityHeaders_PreservesExistingHeaders(t *testing.T) {
handler := mw.SecurityHeaders()(testHandler)
req := httptest.NewRequestWithContext(t.Context(), http.MethodGet, "/test", nil)
req := httptest.NewRequest(http.MethodGet, "/test", nil)
rec := httptest.NewRecorder()
handler.ServeHTTP(rec, req)

View File

@@ -34,8 +34,7 @@ func DeriveKey(masterKey []byte, salt string) ([KeySize]byte, error) {
hkdfReader := hkdf.New(sha256.New, masterKey, []byte(salt), nil)
_, err := io.ReadFull(hkdfReader, key[:])
if err != nil {
if _, err := io.ReadFull(hkdfReader, key[:]); err != nil {
return key, ErrKeyDerivation
}
@@ -47,9 +46,7 @@ func DeriveKey(masterKey []byte, salt string) ([KeySize]byte, error) {
func Encrypt(key [KeySize]byte, plaintext []byte) (string, error) {
// Generate random nonce
var nonce [NonceSize]byte
_, err := rand.Read(nonce[:])
if err != nil {
if _, err := rand.Read(nonce[:]); err != nil {
return "", err
}

View File

@@ -1,25 +1,20 @@
package seal_test
package seal
import (
"bytes"
"errors"
"testing"
"sneak.berlin/go/pixa/internal/seal"
)
func TestDeriveKey_Consistent(t *testing.T) {
t.Parallel()
masterKey := []byte("test-master-key-12345")
salt := "test-salt-v1"
key1, err := seal.DeriveKey(masterKey, salt)
key1, err := DeriveKey(masterKey, salt)
if err != nil {
t.Fatalf("DeriveKey() error = %v", err)
}
key2, err := seal.DeriveKey(masterKey, salt)
key2, err := DeriveKey(masterKey, salt)
if err != nil {
t.Fatalf("DeriveKey() error = %v", err)
}
@@ -30,16 +25,14 @@ func TestDeriveKey_Consistent(t *testing.T) {
}
func TestDeriveKey_DifferentSalts(t *testing.T) {
t.Parallel()
masterKey := []byte("test-master-key-12345")
key1, err := seal.DeriveKey(masterKey, "salt-1")
key1, err := DeriveKey(masterKey, "salt-1")
if err != nil {
t.Fatalf("DeriveKey() error = %v", err)
}
key2, err := seal.DeriveKey(masterKey, "salt-2")
key2, err := DeriveKey(masterKey, "salt-2")
if err != nil {
t.Fatalf("DeriveKey() error = %v", err)
}
@@ -50,16 +43,14 @@ func TestDeriveKey_DifferentSalts(t *testing.T) {
}
func TestDeriveKey_DifferentMasterKeys(t *testing.T) {
t.Parallel()
salt := "test-salt"
key1, err := seal.DeriveKey([]byte("master-key-1"), salt)
key1, err := DeriveKey([]byte("master-key-1"), salt)
if err != nil {
t.Fatalf("DeriveKey() error = %v", err)
}
key2, err := seal.DeriveKey([]byte("master-key-2"), salt)
key2, err := DeriveKey([]byte("master-key-2"), salt)
if err != nil {
t.Fatalf("DeriveKey() error = %v", err)
}
@@ -70,21 +61,19 @@ func TestDeriveKey_DifferentMasterKeys(t *testing.T) {
}
func TestEncryptDecrypt_RoundTrip(t *testing.T) {
t.Parallel()
key, err := seal.DeriveKey([]byte("test-key"), "test-salt")
key, err := DeriveKey([]byte("test-key"), "test-salt")
if err != nil {
t.Fatalf("DeriveKey() error = %v", err)
}
plaintext := []byte("hello, world! this is a test message.")
ciphertext, err := seal.Encrypt(key, plaintext)
ciphertext, err := Encrypt(key, plaintext)
if err != nil {
t.Fatalf("Encrypt() error = %v", err)
}
decrypted, err := seal.Decrypt(key, ciphertext)
decrypted, err := Decrypt(key, ciphertext)
if err != nil {
t.Fatalf("Decrypt() error = %v", err)
}
@@ -95,17 +84,15 @@ func TestEncryptDecrypt_RoundTrip(t *testing.T) {
}
func TestEncryptDecrypt_EmptyPlaintext(t *testing.T) {
t.Parallel()
key, _ := seal.DeriveKey([]byte("test-key"), "test-salt")
key, _ := DeriveKey([]byte("test-key"), "test-salt")
plaintext := []byte{}
ciphertext, err := seal.Encrypt(key, plaintext)
ciphertext, err := Encrypt(key, plaintext)
if err != nil {
t.Fatalf("Encrypt() error = %v", err)
}
decrypted, err := seal.Decrypt(key, ciphertext)
decrypted, err := Decrypt(key, ciphertext)
if err != nil {
t.Fatalf("Decrypt() error = %v", err)
}
@@ -116,35 +103,31 @@ func TestEncryptDecrypt_EmptyPlaintext(t *testing.T) {
}
func TestDecrypt_WrongKey(t *testing.T) {
t.Parallel()
key1, _ := seal.DeriveKey([]byte("key-1"), "salt")
key2, _ := seal.DeriveKey([]byte("key-2"), "salt")
key1, _ := DeriveKey([]byte("key-1"), "salt")
key2, _ := DeriveKey([]byte("key-2"), "salt")
plaintext := []byte("secret message")
ciphertext, err := seal.Encrypt(key1, plaintext)
ciphertext, err := Encrypt(key1, plaintext)
if err != nil {
t.Fatalf("Encrypt() error = %v", err)
}
_, err = seal.Decrypt(key2, ciphertext)
_, err = Decrypt(key2, ciphertext)
if err == nil {
t.Error("Decrypt() should fail with wrong key")
}
if !errors.Is(err, seal.ErrDecryptionFailed) {
t.Errorf("Decrypt() error = %v, want %v", err, seal.ErrDecryptionFailed)
if err != ErrDecryptionFailed {
t.Errorf("Decrypt() error = %v, want %v", err, ErrDecryptionFailed)
}
}
func TestDecrypt_TamperedCiphertext(t *testing.T) {
t.Parallel()
key, _ := seal.DeriveKey([]byte("test-key"), "test-salt")
key, _ := DeriveKey([]byte("test-key"), "test-salt")
plaintext := []byte("secret message")
ciphertext, err := seal.Encrypt(key, plaintext)
ciphertext, err := Encrypt(key, plaintext)
if err != nil {
t.Fatalf("Encrypt() error = %v", err)
}
@@ -155,51 +138,45 @@ func TestDecrypt_TamperedCiphertext(t *testing.T) {
tampered[10] ^= 0x01
}
_, err = seal.Decrypt(key, string(tampered))
_, err = Decrypt(key, string(tampered))
if err == nil {
t.Error("Decrypt() should fail with tampered ciphertext")
}
}
func TestDecrypt_InvalidBase64(t *testing.T) {
t.Parallel()
key, _ := DeriveKey([]byte("test-key"), "test-salt")
key, _ := seal.DeriveKey([]byte("test-key"), "test-salt")
_, err := seal.Decrypt(key, "not-valid-base64!!!")
_, err := Decrypt(key, "not-valid-base64!!!")
if err == nil {
t.Error("Decrypt() should fail with invalid base64")
}
if !errors.Is(err, seal.ErrInvalidPayload) {
t.Errorf("Decrypt() error = %v, want %v", err, seal.ErrInvalidPayload)
if err != ErrInvalidPayload {
t.Errorf("Decrypt() error = %v, want %v", err, ErrInvalidPayload)
}
}
func TestDecrypt_TooShort(t *testing.T) {
t.Parallel()
key, _ := seal.DeriveKey([]byte("test-key"), "test-salt")
key, _ := DeriveKey([]byte("test-key"), "test-salt")
// Create a base64 string that's too short to contain nonce + auth tag
_, err := seal.Decrypt(key, "dG9vLXNob3J0")
_, err := Decrypt(key, "dG9vLXNob3J0")
if err == nil {
t.Error("Decrypt() should fail with too-short ciphertext")
}
if !errors.Is(err, seal.ErrInvalidPayload) {
t.Errorf("Decrypt() error = %v, want %v", err, seal.ErrInvalidPayload)
if err != ErrInvalidPayload {
t.Errorf("Decrypt() error = %v, want %v", err, ErrInvalidPayload)
}
}
func TestEncrypt_ProducesDifferentCiphertexts(t *testing.T) {
t.Parallel()
key, _ := seal.DeriveKey([]byte("test-key"), "test-salt")
key, _ := DeriveKey([]byte("test-key"), "test-salt")
plaintext := []byte("same message")
ciphertext1, _ := seal.Encrypt(key, plaintext)
ciphertext2, _ := seal.Encrypt(key, plaintext)
ciphertext1, _ := Encrypt(key, plaintext)
ciphertext2, _ := Encrypt(key, plaintext)
if ciphertext1 == ciphertext2 {
t.Error("Encrypt() should produce different ciphertexts due to random nonce")

View File

@@ -1,7 +1,6 @@
package server
import (
"errors"
"fmt"
"net/http"
"time"
@@ -27,11 +26,8 @@ func (s *Server) serveUntilShutdown() {
s.SetupRoutes()
s.log.Info("http begin listen", "listenaddr", listenAddr)
err := s.httpServer.ListenAndServe()
if err != nil && !errors.Is(err, http.ErrServerClosed) {
if err := s.httpServer.ListenAndServe(); err != nil && err != http.ErrServerClosed {
s.log.Error("listen error", "error", err)
if s.cancelFunc != nil {
s.cancelFunc()
}

View File

@@ -56,8 +56,7 @@ func (s *Server) SetupRoutes() {
s.router.Head("/v1/image/*", s.h.HandleImage())
// Encrypted image URL route
// The trailing filename (e.g., /img.jpg) is ignored but helps
// browsers with content type
// The trailing filename (e.g., /img.jpg) is ignored but helps browsers with content type
s.router.Get("/v1/e/{token}/*", s.h.HandleImageEnc())
// Metrics endpoint with auth

View File

@@ -30,7 +30,6 @@ const (
// Params defines dependencies for Server.
type Params struct {
fx.In
Logger *logger.Logger
Globals *globals.Globals
Config *config.Config
@@ -48,6 +47,7 @@ type Server struct {
startupTime time.Time
exitCode int
sentryEnabled bool
ctx context.Context
cancelFunc context.CancelFunc
httpServer *http.Server
router *chi.Mux
@@ -64,9 +64,9 @@ func New(lc fx.Lifecycle, params Params) (*Server, error) {
}
lc.Append(fx.Hook{
OnStart: func(ctx context.Context) error {
OnStart: func(_ context.Context) error {
s.startupTime = time.Now()
go s.Run(context.WithoutCancel(ctx))
go s.Run()
return nil
},
@@ -83,14 +83,9 @@ func New(lc fx.Lifecycle, params Params) (*Server, error) {
}
// Run starts the server.
func (s *Server) Run(ctx context.Context) {
func (s *Server) Run() {
s.enableSentry()
s.serve(ctx)
}
// MaintenanceMode returns whether maintenance mode is enabled.
func (s *Server) MaintenanceMode() bool {
return s.config.MaintenanceMode
s.serve()
}
func (s *Server) enableSentry() {
@@ -108,24 +103,19 @@ func (s *Server) enableSentry() {
s.log.Error("sentry init failure", "error", err)
os.Exit(1)
}
s.log.Info("sentry error reporting activated")
s.sentryEnabled = true
}
func (s *Server) serve(ctx context.Context) int {
ctx, cancelFunc := context.WithCancel(ctx)
s.cancelFunc = cancelFunc
func (s *Server) serve() int {
s.ctx, s.cancelFunc = context.WithCancel(context.Background())
go func() {
c := make(chan os.Signal, 1)
signal.Ignore(syscall.SIGPIPE)
signal.Notify(c, os.Interrupt, syscall.SIGTERM)
sig := <-c
s.log.Info("signal received", "signal", sig)
if s.cancelFunc != nil {
s.cancelFunc()
}
@@ -133,22 +123,19 @@ func (s *Server) serve(ctx context.Context) int {
go s.serveUntilShutdown()
<-ctx.Done()
s.cleanShutdown(ctx)
<-s.ctx.Done()
s.cleanShutdown()
return s.exitCode
}
func (s *Server) cleanShutdown(ctx context.Context) {
func (s *Server) cleanShutdown() {
s.exitCode = 0
ctxShutdown, shutdownCancel := context.WithTimeout(
context.WithoutCancel(ctx), ShutdownTimeout)
ctxShutdown, shutdownCancel := context.WithTimeout(context.Background(), ShutdownTimeout)
defer shutdownCancel()
if s.httpServer != nil {
err := s.httpServer.Shutdown(ctxShutdown)
if err != nil {
if err := s.httpServer.Shutdown(ctxShutdown); err != nil {
s.log.Error("server clean shutdown failed", "error", err)
}
}
@@ -157,3 +144,8 @@ func (s *Server) cleanShutdown(ctx context.Context) {
sentry.Flush(SentryFlushTimeout)
}
}
// MaintenanceMode returns whether maintenance mode is enabled.
func (s *Server) MaintenanceMode() bool {
return s.config.MaintenanceMode
}

View File

@@ -107,9 +107,7 @@ func (m *Manager) ValidateSession(r *http.Request) (*Data, error) {
}
var data Data
err = m.sc.Decode(CookieName, cookie.Value, &data)
if err != nil {
if err := m.sc.Decode(CookieName, cookie.Value, &data); err != nil {
return nil, ErrInvalidSession
}

View File

@@ -1,11 +1,9 @@
package session_test
package session
import (
"net/http"
"net/http/httptest"
"testing"
"sneak.berlin/go/pixa/internal/session"
)
// TestSessionCookieAttributesAlwaysSecure verifies that every cookie
@@ -18,9 +16,7 @@ import (
// This covers both cookie-writing paths: CreateSession (the login
// set-cookie path) and ClearSession (the logout delete-cookie path).
func TestSessionCookieAttributesAlwaysSecure(t *testing.T) {
t.Parallel()
mgr, err := session.NewManager("test-signing-key-12345")
mgr, err := NewManager("test-signing-key-12345")
if err != nil {
t.Fatalf("NewManager() error = %v", err)
}
@@ -33,9 +29,7 @@ func TestSessionCookieAttributesAlwaysSecure(t *testing.T) {
name: "CreateSession",
setCookie: func(t *testing.T, w http.ResponseWriter) {
t.Helper()
err := mgr.CreateSession(w)
if err != nil {
if err := mgr.CreateSession(w); err != nil {
t.Fatalf("CreateSession() error = %v", err)
}
},
@@ -51,15 +45,12 @@ func TestSessionCookieAttributesAlwaysSecure(t *testing.T) {
for _, writePath := range writePaths {
t.Run(writePath.name, func(t *testing.T) {
t.Parallel()
w := httptest.NewRecorder()
writePath.setCookie(t, w)
var sessionCookie *http.Cookie
for _, c := range w.Result().Cookies() {
if c.Name == session.CookieName {
if c.Name == CookieName {
sessionCookie = c
break
@@ -67,7 +58,7 @@ func TestSessionCookieAttributesAlwaysSecure(t *testing.T) {
}
if sessionCookie == nil {
t.Fatalf("no cookie named %q was set", session.CookieName)
t.Fatalf("no cookie named %q was set", CookieName)
}
t.Logf("cookie attributes: HttpOnly=%v Secure=%v SameSite=%v",

View File

@@ -1,55 +1,45 @@
package session_test
package session
import (
"errors"
"net/http"
"net/http/httptest"
"testing"
"time"
"sneak.berlin/go/pixa/internal/session"
)
func TestManager_CreateAndValidate(t *testing.T) {
t.Parallel()
mgr, err := session.NewManager("test-signing-key-12345")
mgr, err := NewManager("test-signing-key-12345")
if err != nil {
t.Fatalf("NewManager() error = %v", err)
}
// Create a session
w := httptest.NewRecorder()
err = mgr.CreateSession(w)
if err != nil {
if err := mgr.CreateSession(w); err != nil {
t.Fatalf("CreateSession() error = %v", err)
}
// Extract the cookie from response
resp := w.Result()
cookies := resp.Cookies()
if len(cookies) == 0 {
t.Fatal("CreateSession() did not set a cookie")
}
var sessionCookie *http.Cookie
for _, c := range cookies {
if c.Name == session.CookieName {
if c.Name == CookieName {
sessionCookie = c
break
}
}
if sessionCookie == nil {
t.Fatalf("CreateSession() did not set cookie named %q", session.CookieName)
t.Fatalf("CreateSession() did not set cookie named %q", CookieName)
}
// Validate the session
req := httptest.NewRequestWithContext(t.Context(), http.MethodGet, "/", nil)
req := httptest.NewRequest(http.MethodGet, "/", nil)
req.AddCookie(sessionCookie)
data, err := mgr.ValidateSession(req)
@@ -67,34 +57,27 @@ func TestManager_CreateAndValidate(t *testing.T) {
}
func TestManager_ValidateSession_NoCookie(t *testing.T) {
t.Parallel()
mgr, _ := NewManager("test-signing-key-12345")
mgr, _ := session.NewManager("test-signing-key-12345")
req := httptest.NewRequestWithContext(t.Context(), http.MethodGet, "/", nil)
req := httptest.NewRequest(http.MethodGet, "/", nil)
_, err := mgr.ValidateSession(req)
if err == nil {
t.Error("ValidateSession() should fail with no cookie")
}
if !errors.Is(err, session.ErrNoSession) {
t.Errorf("ValidateSession() error = %v, want %v", err, session.ErrNoSession)
if err != ErrNoSession {
t.Errorf("ValidateSession() error = %v, want %v", err, ErrNoSession)
}
}
func TestManager_ValidateSession_TamperedCookie(t *testing.T) {
t.Parallel()
mgr, _ := NewManager("test-signing-key-12345")
mgr, _ := session.NewManager("test-signing-key-12345")
req := httptest.NewRequestWithContext(t.Context(), http.MethodGet, "/", nil)
req := httptest.NewRequest(http.MethodGet, "/", nil)
req.AddCookie(&http.Cookie{
Name: session.CookieName,
Name: CookieName,
Value: "tampered-invalid-cookie-value",
Secure: true,
HttpOnly: true,
SameSite: http.SameSiteStrictMode,
})
_, err := mgr.ValidateSession(req)
@@ -102,35 +85,30 @@ func TestManager_ValidateSession_TamperedCookie(t *testing.T) {
t.Error("ValidateSession() should fail with tampered cookie")
}
if !errors.Is(err, session.ErrInvalidSession) {
t.Errorf("ValidateSession() error = %v, want %v", err, session.ErrInvalidSession)
if err != ErrInvalidSession {
t.Errorf("ValidateSession() error = %v, want %v", err, ErrInvalidSession)
}
}
func TestManager_ValidateSession_WrongKey(t *testing.T) {
t.Parallel()
mgr1, _ := session.NewManager("signing-key-1")
mgr2, _ := session.NewManager("signing-key-2")
mgr1, _ := NewManager("signing-key-1")
mgr2, _ := NewManager("signing-key-2")
// Create session with mgr1
w := httptest.NewRecorder()
_ = mgr1.CreateSession(w)
resp := w.Result()
var sessionCookie *http.Cookie
for _, c := range resp.Cookies() {
if c.Name == session.CookieName {
if c.Name == CookieName {
sessionCookie = c
break
}
}
// Try to validate with mgr2 (different key)
req := httptest.NewRequestWithContext(t.Context(), http.MethodGet, "/", nil)
req := httptest.NewRequest(http.MethodGet, "/", nil)
req.AddCookie(sessionCookie)
_, err := mgr2.ValidateSession(req)
@@ -140,9 +118,7 @@ func TestManager_ValidateSession_WrongKey(t *testing.T) {
}
func TestManager_ClearSession(t *testing.T) {
t.Parallel()
mgr, _ := session.NewManager("test-signing-key-12345")
mgr, _ := NewManager("test-signing-key-12345")
w := httptest.NewRecorder()
mgr.ClearSession(w)
@@ -151,11 +127,9 @@ func TestManager_ClearSession(t *testing.T) {
cookies := resp.Cookies()
var sessionCookie *http.Cookie
for _, c := range cookies {
if c.Name == session.CookieName {
if c.Name == CookieName {
sessionCookie = c
break
}
}
@@ -170,12 +144,10 @@ func TestManager_ClearSession(t *testing.T) {
}
func TestManager_IsAuthenticated(t *testing.T) {
t.Parallel()
mgr, _ := session.NewManager("test-signing-key-12345")
mgr, _ := NewManager("test-signing-key-12345")
// No session - should return false
req := httptest.NewRequestWithContext(t.Context(), http.MethodGet, "/", nil)
req := httptest.NewRequest(http.MethodGet, "/", nil)
if mgr.IsAuthenticated(req) {
t.Error("IsAuthenticated() should return false with no session")
}
@@ -185,19 +157,16 @@ func TestManager_IsAuthenticated(t *testing.T) {
_ = mgr.CreateSession(w)
resp := w.Result()
var sessionCookie *http.Cookie
for _, c := range resp.Cookies() {
if c.Name == session.CookieName {
if c.Name == CookieName {
sessionCookie = c
break
}
}
// With valid session - should return true
req = httptest.NewRequestWithContext(t.Context(), http.MethodGet, "/", nil)
req = httptest.NewRequest(http.MethodGet, "/", nil)
req.AddCookie(sessionCookie)
if !mgr.IsAuthenticated(req) {
@@ -206,21 +175,16 @@ func TestManager_IsAuthenticated(t *testing.T) {
}
func TestManager_CookieAttributes(t *testing.T) {
t.Parallel()
mgr, _ := session.NewManager("test-key")
mgr, _ := NewManager("test-key")
w := httptest.NewRecorder()
_ = mgr.CreateSession(w)
resp := w.Result()
var sessionCookie *http.Cookie
for _, c := range resp.Cookies() {
if c.Name == session.CookieName {
if c.Name == CookieName {
sessionCookie = c
break
}
}
@@ -234,7 +198,6 @@ func TestManager_CookieAttributes(t *testing.T) {
}
if sessionCookie.SameSite != http.SameSiteStrictMode {
t.Errorf("Cookie SameSite = %v, want %v",
sessionCookie.SameSite, http.SameSiteStrictMode)
t.Errorf("Cookie SameSite = %v, want %v", sessionCookie.SameSite, http.SameSiteStrictMode)
}
}

View File

@@ -1,10 +1,8 @@
package signature_test
package signature
import (
"testing"
"time"
"sneak.berlin/go/pixa/internal/signature"
)
// goldenExpiresUnix is the fixed expiration timestamp used by all golden
@@ -14,9 +12,23 @@ const goldenExpiresUnix int64 = 1704067200
// goldenSigningKey is the fixed signing key used by all golden vectors.
const goldenSigningKey = "golden-test-key"
type goldenVector struct {
// TestSigner_GoldenVectors pins the exact HMAC-SHA256 signature output and
// the exact generated signed URL path for fully-specified requests with a
// hardcoded signing key. The expected values were computed once and are
// hardcoded here as known answers.
//
// If any of these assertions fail, the signed byte format
// ("host:path:query:width:height:format:expiration"), the base64url
// encoding, or the signed URL layout has changed. Such a change breaks
// every signature already issued to clients, so it must be made
// deliberately: update these constants only as part of an intentional,
// documented signature format migration.
func TestSigner_GoldenVectors(t *testing.T) {
signer := New(goldenSigningKey)
vectors := []struct {
name string
req signature.Request
req Request
// wantSignature is the exact base64url (RFC 4648 URL-safe,
// padded) HMAC-SHA256 signature for the request with Expires
// set to goldenExpiresUnix.
@@ -26,78 +38,53 @@ type goldenVector struct {
// expiration are returned separately by GenerateSignedURL and
// are not embedded in the path.
wantSignedPath string
}
// goldenVectors returns the known-answer vectors. The expected values
// were computed once and are hardcoded here.
func goldenVectors() []goldenVector {
return []goldenVector{
}{
{
name: "resized without query",
req: signature.Request{
SourceHost: testHost,
SourcePath: testPath,
req: Request{
SourceHost: "cdn.example.com",
SourcePath: "/photos/cat.jpg",
SourceQuery: "",
Width: 800,
Height: 600,
Format: testFormatWebP,
Format: "webp",
},
// Signed data: "cdn.example.com:/photos/cat.jpg::800:600:webp:1704067200"
wantSignature: "x5PfPp8QSDo0cJT96od-AEgrQyOVLfqifH5sst61_-w=",
wantSignedPath: testSignedPath,
wantSignedPath: "/v1/image/cdn.example.com/photos/cat.jpg/800x600.webp",
},
{
name: "resized with query string",
req: signature.Request{
SourceHost: testHost,
SourcePath: testPath,
req: Request{
SourceHost: "cdn.example.com",
SourcePath: "/photos/cat.jpg",
SourceQuery: "token=abc&v=2",
Width: 800,
Height: 600,
Format: testFormatWebP,
Format: "webp",
},
// Signed data:
// "cdn.example.com:/photos/cat.jpg:token=abc&v=2:800:600:webp:1704067200"
// Signed data: "cdn.example.com:/photos/cat.jpg:token=abc&v=2:800:600:webp:1704067200"
wantSignature: "394_Vf9TdQFkpQ3XKFDQSyxgqKq8N7mApf2S4QaHqyo=",
wantSignedPath: "/v1/image/cdn.example.com/photos/cat.jpg" +
"%3Ftoken=abc&v=2/800x600.webp",
wantSignedPath: "/v1/image/cdn.example.com/photos/cat.jpg%3Ftoken=abc&v=2/800x600.webp",
},
{
name: "original size without query",
req: signature.Request{
SourceHost: testHost,
SourcePath: testPath,
req: Request{
SourceHost: "cdn.example.com",
SourcePath: "/photos/cat.jpg",
SourceQuery: "",
Width: 0,
Height: 0,
Format: testFormatPNG,
Format: "png",
},
// Signed data: "cdn.example.com:/photos/cat.jpg::0:0:png:1704067200"
wantSignature: "7Be7oteeQwvnSPU4bchyQ4ZGYGsAGBKpeEtuQ02ox60=",
wantSignedPath: "/v1/image/cdn.example.com/photos/cat.jpg/orig.png",
},
}
}
// TestSigner_GoldenVectors pins the exact HMAC-SHA256 signature output and
// the exact generated signed URL path for fully-specified requests with a
// hardcoded signing key.
//
// If any of these assertions fail, the signed byte format
// ("host:path:query:width:height:format:expiration"), the base64url
// encoding, or the signed URL layout has changed. Such a change breaks
// every signature already issued to clients, so it must be made
// deliberately: update these constants only as part of an intentional,
// documented signature format migration.
func TestSigner_GoldenVectors(t *testing.T) {
t.Parallel()
signer := signature.New(goldenSigningKey)
for _, tt := range goldenVectors() {
for _, tt := range vectors {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
signReq := tt.req
signReq.Expires = time.Unix(goldenExpiresUnix, 0)
@@ -108,10 +95,9 @@ func TestSigner_GoldenVectors(t *testing.T) {
}
urlReq := tt.req
gotPath, _, _ := signer.GenerateSignedURL(&urlReq, time.Hour)
if gotPath != tt.wantSignedPath {
t.Errorf("GenerateSignedURL() path = %q, want %q (layout changed?)",
t.Errorf("GenerateSignedURL() path = %q, want %q (signed URL layout changed?)",
gotPath, tt.wantSignedPath)
}
})

View File

@@ -93,17 +93,31 @@ func (s *Signer) Verify(req *Request) error {
return nil
}
// buildSignatureData creates the string to be signed.
// Format: "host:path:query:width:height:format:expiration"
// All components are used verbatim (exact match). No normalization,
// suffix matching, or wildcard expansion is performed.
func (s *Signer) buildSignatureData(req *Request) string {
return fmt.Sprintf("%s:%s:%s:%d:%d:%s:%d",
req.SourceHost,
req.SourcePath,
req.SourceQuery,
req.Width,
req.Height,
req.Format,
req.Expires.Unix(),
)
}
// GenerateSignedURL creates a complete URL with signature and expiration.
// Returns the path portion that should be appended to the base URL.
func (s *Signer) GenerateSignedURL(
req *Request, ttl time.Duration,
) (string, string, int64) {
func (s *Signer) GenerateSignedURL(req *Request, ttl time.Duration) (path string, sig string, exp int64) {
// Set expiration
req.Expires = time.Now().Add(ttl)
exp := req.Expires.Unix()
exp = req.Expires.Unix()
// Generate signature
sig := s.Sign(req)
sig = s.Sign(req)
req.Signature = sig
// Build the size component
@@ -120,7 +134,6 @@ func (s *Signer) GenerateSignedURL(
// it from the last-slash split. The "?" inside a path segment is
// percent-encoded by clients but chi delivers it decoded, which is
// exactly what the URL parser expects.
var path string
if req.SourceQuery != "" {
path = fmt.Sprintf("/v1/image/%s%s%%3F%s/%s.%s",
req.SourceHost,
@@ -141,26 +154,12 @@ func (s *Signer) GenerateSignedURL(
return path, sig, exp
}
// buildSignatureData creates the string to be signed.
// Format: "host:path:query:width:height:format:expiration"
// All components are used verbatim (exact match). No normalization,
// suffix matching, or wildcard expansion is performed.
func (s *Signer) buildSignatureData(req *Request) string {
return fmt.Sprintf("%s:%s:%s:%d:%d:%s:%d",
req.SourceHost,
req.SourcePath,
req.SourceQuery,
req.Width,
req.Height,
req.Format,
req.Expires.Unix(),
)
}
// ParseParams extracts signature and expiration from query parameters.
func ParseParams(sig, expStr string) (string, time.Time, error) {
func ParseParams(sig, expStr string) (parsed string, expires time.Time, err error) {
parsed = sig
if expStr == "" {
return sig, time.Time{}, nil
return parsed, time.Time{}, nil
}
expUnix, err := strconv.ParseInt(expStr, 10, 64)
@@ -168,5 +167,7 @@ func ParseParams(sig, expStr string) (string, time.Time, error) {
return "", time.Time{}, fmt.Errorf("invalid expiration: %w", err)
}
return sig, time.Unix(expUnix, 0), nil
expires = time.Unix(expUnix, 0)
return parsed, expires, nil
}

View File

@@ -1,36 +1,21 @@
package signature_test
package signature
import (
"errors"
"strings"
"testing"
"time"
"sneak.berlin/go/pixa/internal/signature"
)
// Shared fixture values used across the signature tests.
const (
testHost = "cdn.example.com"
testPath = "/photos/cat.jpg"
testFormatWebP = "webp"
testFormatPNG = "png"
testSignedPath = "/v1/image/cdn.example.com/photos/cat.jpg/800x600.webp"
testSig = "abc123"
)
func TestSigner_Sign(t *testing.T) {
t.Parallel()
signer := New("test-secret-key")
signer := signature.New("test-secret-key")
req := &signature.Request{
SourceHost: testHost,
SourcePath: testPath,
req := &Request{
SourceHost: "cdn.example.com",
SourcePath: "/photos/cat.jpg",
SourceQuery: "",
Width: 800,
Height: 600,
Format: testFormatWebP,
Format: "webp",
Expires: time.Unix(1704067200, 0), // Fixed timestamp for reproducibility
}
@@ -39,8 +24,7 @@ func TestSigner_Sign(t *testing.T) {
// Same input should produce same signature
if sig1 != sig2 {
t.Errorf("Sign() produced different signatures for same input: %q vs %q",
sig1, sig2)
t.Errorf("Sign() produced different signatures for same input: %q vs %q", sig1, sig2)
}
// Signature should be non-empty
@@ -49,13 +33,13 @@ func TestSigner_Sign(t *testing.T) {
}
// Different input should produce different signature
req2 := &signature.Request{
SourceHost: testHost,
req2 := &Request{
SourceHost: "cdn.example.com",
SourcePath: "/photos/dog.jpg", // Different path
SourceQuery: "",
Width: 800,
Height: 600,
Format: testFormatWebP,
Format: "webp",
Expires: time.Unix(1704067200, 0),
}
@@ -65,31 +49,25 @@ func TestSigner_Sign(t *testing.T) {
}
}
// validVerifyRequest returns a fully-populated request that verifies
// successfully once signed.
func validVerifyRequest() *signature.Request {
return &signature.Request{
SourceHost: testHost,
SourcePath: testPath,
Width: 800,
Height: 600,
Format: testFormatWebP,
Expires: time.Now().Add(1 * time.Hour),
}
}
func TestSigner_Verify(t *testing.T) {
signer := New("test-secret-key")
type verifyCase struct {
tests := []struct {
name string
setup func() *signature.Request
setup func() *Request
wantErr error
}
func verifyCases(signer *signature.Signer) []verifyCase {
return []verifyCase{
}{
{
name: "valid signature",
setup: func() *signature.Request {
req := validVerifyRequest()
setup: func() *Request {
req := &Request{
SourceHost: "cdn.example.com",
SourcePath: "/photos/cat.jpg",
Width: 800,
Height: 600,
Format: "webp",
Expires: time.Now().Add(1 * time.Hour),
}
req.Signature = signer.Sign(req)
return req
@@ -98,59 +76,74 @@ func verifyCases(signer *signature.Signer) []verifyCase {
},
{
name: "expired signature",
setup: func() *signature.Request {
req := validVerifyRequest()
req.Expires = time.Now().Add(-1 * time.Hour)
setup: func() *Request {
req := &Request{
SourceHost: "cdn.example.com",
SourcePath: "/photos/cat.jpg",
Width: 800,
Height: 600,
Format: "webp",
Expires: time.Now().Add(-1 * time.Hour), // Expired
}
req.Signature = signer.Sign(req)
return req
},
wantErr: signature.ErrExpired,
wantErr: ErrExpired,
},
{
name: "invalid signature",
setup: func() *signature.Request {
req := validVerifyRequest()
req.Signature = "invalid-signature"
return req
setup: func() *Request {
return &Request{
SourceHost: "cdn.example.com",
SourcePath: "/photos/cat.jpg",
Width: 800,
Height: 600,
Format: "webp",
Expires: time.Now().Add(1 * time.Hour),
Signature: "invalid-signature",
}
},
wantErr: signature.ErrInvalid,
wantErr: ErrInvalid,
},
{
name: "missing expiration",
setup: func() *signature.Request {
req := validVerifyRequest()
req.Expires = time.Time{}
req.Signature = "some-signature"
return req
setup: func() *Request {
return &Request{
SourceHost: "cdn.example.com",
SourcePath: "/photos/cat.jpg",
Width: 800,
Height: 600,
Format: "webp",
Signature: "some-signature",
// Expires is zero
}
},
wantErr: signature.ErrMissingExpiration,
wantErr: ErrMissingExpiration,
},
{
name: "tampered request",
setup: func() *signature.Request {
req := validVerifyRequest()
setup: func() *Request {
req := &Request{
SourceHost: "cdn.example.com",
SourcePath: "/photos/cat.jpg",
Width: 800,
Height: 600,
Format: "webp",
Expires: time.Now().Add(1 * time.Hour),
}
req.Signature = signer.Sign(req)
// Tamper with the request
req.SourcePath = "/photos/secret.jpg"
return req
},
wantErr: signature.ErrInvalid,
wantErr: ErrInvalid,
},
}
}
func TestSigner_Verify(t *testing.T) {
t.Parallel()
signer := signature.New("test-secret-key")
for _, tt := range verifyCases(signer) {
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
req := tt.setup()
err := signer.Verify(req)
@@ -158,82 +151,12 @@ func TestSigner_Verify(t *testing.T) {
if err != nil {
t.Errorf("Verify() unexpected error = %v", err)
}
return
}
if !errors.Is(err, tt.wantErr) {
} else {
if err != tt.wantErr {
t.Errorf("Verify() error = %v, wantErr %v", err, tt.wantErr)
}
})
}
}
type tamperCase struct {
name string
tamper func(r *signature.Request)
}
// exactMatchTamperCases mutates one signed component per case; every
// mutation must cause verification to fail with ErrInvalid.
func exactMatchTamperCases() []tamperCase {
return []tamperCase{
{
name: "parent domain does not match subdomain",
tamper: func(r *signature.Request) { r.SourceHost = "example.com" },
},
{
name: "subdomain does not match parent domain",
tamper: func(r *signature.Request) { r.SourceHost = "images.cdn.example.com" },
},
{
name: "sibling subdomain does not match",
tamper: func(r *signature.Request) { r.SourceHost = "images.example.com" },
},
{
name: "host with suffix appended does not match",
tamper: func(r *signature.Request) { r.SourceHost = testHost + ".evil.com" },
},
{
name: "host with prefix does not match",
tamper: func(r *signature.Request) { r.SourceHost = "evilcdn.example.com" },
},
{
name: "different path does not match",
tamper: func(r *signature.Request) { r.SourcePath = "/photos/dog.jpg" },
},
{
name: "path suffix does not match",
tamper: func(r *signature.Request) { r.SourcePath = testPath + "/extra" },
},
{
name: "path prefix does not match",
tamper: func(r *signature.Request) { r.SourcePath = "/other" + testPath },
},
{
name: "different query does not match",
tamper: func(r *signature.Request) { r.SourceQuery = "token=xyz" },
},
{
name: "added query does not match empty query",
tamper: func(r *signature.Request) { r.SourceQuery = "extra=1" },
},
{
name: "removed query does not match",
tamper: func(r *signature.Request) { r.SourceQuery = "" },
},
{
name: "different width does not match",
tamper: func(r *signature.Request) { r.Width = 801 },
},
{
name: "different height does not match",
tamper: func(r *signature.Request) { r.Height = 601 },
},
{
name: "different format does not match",
tamper: func(r *signature.Request) { r.Format = testFormatPNG },
},
})
}
}
@@ -241,19 +164,17 @@ func exactMatchTamperCases() []tamperCase {
// matching on every URL component. No suffix matching, wildcard matching,
// or partial matching is supported.
func TestSigner_Verify_ExactMatchOnly(t *testing.T) {
t.Parallel()
signer := signature.New("test-secret-key")
signer := New("test-secret-key")
// Base request that we'll sign, then tamper with individual fields.
baseReq := func() *signature.Request {
req := &signature.Request{
SourceHost: testHost,
SourcePath: testPath,
baseReq := func() *Request {
req := &Request{
SourceHost: "cdn.example.com",
SourcePath: "/photos/cat.jpg",
SourceQuery: "token=abc",
Width: 800,
Height: 600,
Format: testFormatWebP,
Format: "webp",
Expires: time.Now().Add(1 * time.Hour),
}
req.Signature = signer.Sign(req)
@@ -261,28 +182,117 @@ func TestSigner_Verify_ExactMatchOnly(t *testing.T) {
return req
}
for _, tt := range exactMatchTamperCases() {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
tests := []struct {
name string
tamper func(req *Request)
}{
{
name: "parent domain does not match subdomain",
tamper: func(req *Request) {
// Signed for cdn.example.com, try example.com
req.SourceHost = "example.com"
},
},
{
name: "subdomain does not match parent domain",
tamper: func(req *Request) {
// Signed for cdn.example.com, try images.cdn.example.com
req.SourceHost = "images.cdn.example.com"
},
},
{
name: "sibling subdomain does not match",
tamper: func(req *Request) {
// Signed for cdn.example.com, try images.example.com
req.SourceHost = "images.example.com"
},
},
{
name: "host with suffix appended does not match",
tamper: func(req *Request) {
// Signed for cdn.example.com, try cdn.example.com.evil.com
req.SourceHost = "cdn.example.com.evil.com"
},
},
{
name: "host with prefix does not match",
tamper: func(req *Request) {
// Signed for cdn.example.com, try evilcdn.example.com
req.SourceHost = "evilcdn.example.com"
},
},
{
name: "different path does not match",
tamper: func(req *Request) {
req.SourcePath = "/photos/dog.jpg"
},
},
{
name: "path suffix does not match",
tamper: func(req *Request) {
req.SourcePath = "/photos/cat.jpg/extra"
},
},
{
name: "path prefix does not match",
tamper: func(req *Request) {
req.SourcePath = "/other/photos/cat.jpg"
},
},
{
name: "different query does not match",
tamper: func(req *Request) {
req.SourceQuery = "token=xyz"
},
},
{
name: "added query does not match empty query",
tamper: func(req *Request) {
req.SourceQuery = "extra=1"
},
},
{
name: "removed query does not match",
tamper: func(req *Request) {
req.SourceQuery = ""
},
},
{
name: "different width does not match",
tamper: func(req *Request) {
req.Width = 801
},
},
{
name: "different height does not match",
tamper: func(req *Request) {
req.Height = 601
},
},
{
name: "different format does not match",
tamper: func(req *Request) {
req.Format = "png"
},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
req := baseReq()
tt.tamper(req)
err := signer.Verify(req)
if !errors.Is(err, signature.ErrInvalid) {
t.Errorf("Verify() = %v, want %v", err, signature.ErrInvalid)
if err != ErrInvalid {
t.Errorf("Verify() = %v, want %v", err, ErrInvalid)
}
})
}
// Verify the unmodified base request still passes
t.Run("unmodified request passes", func(t *testing.T) {
t.Parallel()
req := baseReq()
err := signer.Verify(req)
if err != nil {
if err := signer.Verify(req); err != nil {
t.Errorf("Verify() unmodified request failed: %v", err)
}
})
@@ -292,12 +302,10 @@ func TestSigner_Verify_ExactMatchOnly(t *testing.T) {
// string in the signature data, producing different signatures for
// suffix-related hosts.
func TestSigner_Sign_ExactHostInData(t *testing.T) {
t.Parallel()
signer := signature.New("test-secret-key")
signer := New("test-secret-key")
hosts := []string{
testHost,
"cdn.example.com",
"example.com",
"images.example.com",
"images.cdn.example.com",
@@ -307,13 +315,13 @@ func TestSigner_Sign_ExactHostInData(t *testing.T) {
sigs := make(map[string]string)
for _, host := range hosts {
req := &signature.Request{
req := &Request{
SourceHost: host,
SourcePath: testPath,
SourcePath: "/photos/cat.jpg",
SourceQuery: "",
Width: 800,
Height: 600,
Format: testFormatWebP,
Format: "webp",
Expires: time.Unix(1704067200, 0),
}
@@ -327,17 +335,15 @@ func TestSigner_Sign_ExactHostInData(t *testing.T) {
}
func TestSigner_DifferentKeys(t *testing.T) {
t.Parallel()
signer1 := New("secret-key-1")
signer2 := New("secret-key-2")
signer1 := signature.New("secret-key-1")
signer2 := signature.New("secret-key-2")
req := &signature.Request{
SourceHost: testHost,
SourcePath: testPath,
req := &Request{
SourceHost: "cdn.example.com",
SourcePath: "/photos/cat.jpg",
Width: 800,
Height: 600,
Format: testFormatWebP,
Format: "webp",
Expires: time.Now().Add(1 * time.Hour),
}
@@ -345,38 +351,35 @@ func TestSigner_DifferentKeys(t *testing.T) {
req.Signature = signer1.Sign(req)
// Verify with key 1 should succeed
err := signer1.Verify(req)
if err != nil {
if err := signer1.Verify(req); err != nil {
t.Errorf("Verify() with same key failed: %v", err)
}
// Verify with key 2 should fail
err = signer2.Verify(req)
if !errors.Is(err, signature.ErrInvalid) {
if err := signer2.Verify(req); err != ErrInvalid {
t.Errorf("Verify() with different key should fail, got: %v", err)
}
}
func TestGenerateSignedURL(t *testing.T) {
t.Parallel()
signer := New("test-secret-key")
signer := signature.New("test-secret-key")
req := &signature.Request{
SourceHost: testHost,
SourcePath: testPath,
req := &Request{
SourceHost: "cdn.example.com",
SourcePath: "/photos/cat.jpg",
SourceQuery: "",
Width: 800,
Height: 600,
Format: testFormatWebP,
Format: "webp",
}
ttl := 1 * time.Hour
path, sig, exp := signer.GenerateSignedURL(req, ttl)
// Path should be correct format
if path != testSignedPath {
t.Errorf("GenerateSignedURL() path = %q, want %q", path, testSignedPath)
expectedPath := "/v1/image/cdn.example.com/photos/cat.jpg/800x600.webp"
if path != expectedPath {
t.Errorf("GenerateSignedURL() path = %q, want %q", path, expectedPath)
}
// Signature should be non-empty
@@ -386,7 +389,6 @@ func TestGenerateSignedURL(t *testing.T) {
// Expiration should be approximately now + TTL
expTime := time.Unix(exp, 0)
expectedExp := time.Now().Add(ttl)
if expTime.Sub(expectedExp) > time.Second {
t.Errorf("GenerateSignedURL() exp time off by too much")
@@ -399,16 +401,14 @@ func TestGenerateSignedURL(t *testing.T) {
}
func TestGenerateSignedURL_OrigSize(t *testing.T) {
t.Parallel()
signer := New("test-secret-key")
signer := signature.New("test-secret-key")
req := &signature.Request{
SourceHost: testHost,
SourcePath: testPath,
req := &Request{
SourceHost: "cdn.example.com",
SourcePath: "/photos/cat.jpg",
Width: 0, // Original size
Height: 0,
Format: testFormatPNG,
Format: "png",
}
path, _, _ := signer.GenerateSignedURL(req, time.Hour)
@@ -420,24 +420,21 @@ func TestGenerateSignedURL_OrigSize(t *testing.T) {
}
func TestGenerateSignedURL_WithQueryString(t *testing.T) {
t.Parallel()
signer := New("test-secret-key-for-testing!")
signer := signature.New("test-secret-key-for-testing!")
req := &signature.Request{
SourceHost: testHost,
SourcePath: testPath,
req := &Request{
SourceHost: "cdn.example.com",
SourcePath: "/photos/cat.jpg",
SourceQuery: "token=abc&v=2",
Width: 800,
Height: 600,
Format: testFormatWebP,
Format: "webp",
}
path, _, _ := signer.GenerateSignedURL(req, time.Hour)
// The path must NOT contain a bare "?" that would be interpreted as
// a query string delimiter. The size segment must appear as the last
// path component.
// The path must NOT contain a bare "?" that would be interpreted as a query string delimiter.
// The size segment must appear as the last path component.
if strings.Contains(path, "?token=abc") {
t.Errorf("GenerateSignedURL() produced bare query string in path: %q", path)
}
@@ -454,28 +451,25 @@ func TestGenerateSignedURL_WithQueryString(t *testing.T) {
}
func TestGenerateSignedURL_WithoutQueryString(t *testing.T) {
t.Parallel()
signer := New("test-secret-key-for-testing!")
signer := signature.New("test-secret-key-for-testing!")
req := &signature.Request{
SourceHost: testHost,
SourcePath: testPath,
req := &Request{
SourceHost: "cdn.example.com",
SourcePath: "/photos/cat.jpg",
Width: 800,
Height: 600,
Format: testFormatWebP,
Format: "webp",
}
path, _, _ := signer.GenerateSignedURL(req, time.Hour)
if path != testSignedPath {
t.Errorf("GenerateSignedURL() path = %q, want %q", path, testSignedPath)
expected := "/v1/image/cdn.example.com/photos/cat.jpg/800x600.webp"
if path != expected {
t.Errorf("GenerateSignedURL() path = %q, want %q", path, expected)
}
}
func TestParseParams(t *testing.T) {
t.Parallel()
tests := []struct {
name string
sig string
@@ -486,21 +480,21 @@ func TestParseParams(t *testing.T) {
}{
{
name: "valid params",
sig: testSig,
sig: "abc123",
expStr: "1704067200",
wantSig: testSig,
wantSig: "abc123",
wantErr: false,
},
{
name: "empty expiration",
sig: testSig,
sig: "abc123",
expStr: "",
wantSig: testSig,
wantSig: "abc123",
wantErr: false,
},
{
name: "invalid expiration",
sig: testSig,
sig: "abc123",
expStr: "not-a-number",
wantErr: true,
},
@@ -508,9 +502,7 @@ func TestParseParams(t *testing.T) {
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
sig, exp, err := signature.ParseParams(tt.sig, tt.expStr)
sig, exp, err := ParseParams(tt.sig, tt.expStr)
if tt.wantErr {
if err == nil {

View File

@@ -11,11 +11,11 @@ set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
# Pinned versions, 2026-08-07. Never "latest"; exact versions only.
GOLANGCI_LINT_VERSION="2.12.2"
# sha256 of golangci-lint-2.12.2-linux-<arch>.tar.gz release archives
GOLANGCI_LINT_SHA256_AMD64="8df580d2670fed8fa984aac0507099af8df275e665215f5c7a2ae3943893a553"
GOLANGCI_LINT_SHA256_ARM64="44cd40a8c76c86755375adfeea52cfd3533cb43d7bd647771e0ae065e166df3a"
# Pinned versions, 2026-07-07. Never "latest"; exact versions only.
GOLANGCI_LINT_VERSION="2.10.1"
# sha256 of golangci-lint-2.10.1-linux-<arch>.tar.gz release archives
GOLANGCI_LINT_SHA256_AMD64="dfa775874cf0561b404a02a8f4481fc69b28091da95aa697259820d429b09c99"
GOLANGCI_LINT_SHA256_ARM64="6652b42ae02915eb2f9cb2a2e0cac99514c8eded8388d88ae3e06e1a52c00de8"
PKGMGR=""
SUDO=""