Build the image pipeline in pixad's fx graph (closes #313)
check / check (push) Canceled after 0s
check / check (push) Canceled after 0s
The new package internal/providers builds the cache, the fetcher, the processor, the image service, the session manager and the encrypted URL generator from the config. pixad's fx graph provides them with providers.Constructors(), and handlers.New receives them built. The server tests build the same graph, with only the fetcher's dial function replaced. database.New opens the database, so these parts can read it while fx builds them; the cache's stop hook still runs before the database's. Each setting's default is declared in internal/config, and internal/providers works out the default cache_max_bytes. The cache's MaxBytes is the one value for the disk cache limit, and 0 turns the disk cache off. The fetcher and the processor take explicit values. Model: opus-5-5
This commit was merged in pull request #330.
This commit is contained in:
+39
-12
@@ -45,6 +45,37 @@ func DefaultMaxConcurrentProcessing() int {
|
||||
return max(runtime.GOMAXPROCS(0)/2, 1) //nolint:mnd // half the CPUs
|
||||
}
|
||||
|
||||
// DefaultCacheMaxBytesFloor is the least DefaultCacheMaxBytes returns:
|
||||
// 500 MiB. It applies only to the default, never to an explicit
|
||||
// cache_max_bytes.
|
||||
const DefaultCacheMaxBytesFloor int64 = 500 << 20
|
||||
|
||||
// freeSpaceFractionNumerator and freeSpaceFractionDenominator express
|
||||
// the 75% share of DefaultCacheMaxBytes as integer arithmetic (dividing
|
||||
// before multiplying avoids overflow).
|
||||
const (
|
||||
freeSpaceFractionNumerator uint64 = 3
|
||||
freeSpaceFractionDenominator uint64 = 4
|
||||
)
|
||||
|
||||
// DefaultCacheMaxBytes returns the disk cache limit when cache_max_bytes
|
||||
// is omitted: 75% of the sum of freeBytes, the space free on the
|
||||
// filesystem holding the cache, and heldBytes, the bytes the cache
|
||||
// already holds, and at least DefaultCacheMaxBytesFloor. Counting what
|
||||
// the cache holds keeps the limit from shrinking as the cache fills.
|
||||
// pixad works it out when it builds the cache, as both numbers are known
|
||||
// only then.
|
||||
func DefaultCacheMaxBytes(freeBytes uint64, heldBytes int64) int64 {
|
||||
// Both terms are at most math.MaxInt64, so the sum cannot overflow.
|
||||
//nolint:gosec // G115: heldBytes is the cache's total, never negative
|
||||
spaceBytes := min(freeBytes, math.MaxInt64) + uint64(heldBytes)
|
||||
|
||||
computed := spaceBytes / freeSpaceFractionDenominator * freeSpaceFractionNumerator
|
||||
computed = min(computed, math.MaxInt64)
|
||||
|
||||
return max(int64(computed), DefaultCacheMaxBytesFloor)
|
||||
}
|
||||
|
||||
// Configuration key names.
|
||||
const (
|
||||
keyDebug = "debug"
|
||||
@@ -190,18 +221,14 @@ type Config struct {
|
||||
// address, and an explicit list replaces the default.
|
||||
TrustedProxies []netip.Prefix
|
||||
|
||||
// CacheMaxBytes is the disk cache size limit in bytes. Only an
|
||||
// explicit zero (CacheMaxBytesExplicit true) disables the disk
|
||||
// cache. Zero with CacheMaxBytesExplicit false means
|
||||
// cache_max_bytes was omitted, and the cache works out the default
|
||||
// limit when it opens.
|
||||
// CacheMaxBytes is the disk cache size limit in bytes, as set; 0
|
||||
// turns the disk cache off.
|
||||
CacheMaxBytes int64
|
||||
|
||||
// CacheMaxBytesExplicit records whether cache_max_bytes was
|
||||
// explicitly set, in the environment or the configuration file.
|
||||
// Explicit values are used exactly as given; for an omitted key the
|
||||
// cache works out the default limit when it opens (see
|
||||
// imgcache.CacheConfig.UseDefaultMaxBytes).
|
||||
// Explicit values are used exactly as given. For an omitted key
|
||||
// pixad works out DefaultCacheMaxBytes when it builds the cache.
|
||||
CacheMaxBytesExplicit bool
|
||||
}
|
||||
|
||||
@@ -239,8 +266,8 @@ func New(_ fx.Lifecycle, params Params) (*Config, error) {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// An omitted cache_max_bytes is worked out and logged when the
|
||||
// cache opens.
|
||||
// An omitted cache_max_bytes is worked out and logged when pixad
|
||||
// builds the cache.
|
||||
if c.CacheMaxBytesExplicit {
|
||||
log.Info("effective cache size limit",
|
||||
"cache_max_bytes", c.CacheMaxBytes,
|
||||
@@ -325,8 +352,8 @@ func newFromSmartConfig(sc *smartconfig.Config) (*Config, error) {
|
||||
}
|
||||
|
||||
// The default for an omitted cache_max_bytes is worked out when
|
||||
// the cache opens; here we only record whether the operator set
|
||||
// the key explicitly.
|
||||
// pixad builds the cache; here we only record whether the operator
|
||||
// set the key explicitly.
|
||||
_, c.CacheMaxBytesExplicit = lookupValue(sc, keyCacheMaxBytes)
|
||||
|
||||
// Build DBURL from StateDir if not explicitly set. The derived URL
|
||||
|
||||
Reference in New Issue
Block a user