Default max_concurrent_processing to half the CPUs pixa sees (closes #242)
check / check (push) Canceled after 0s
check / check (push) Canceled after 0s
The default was one image per CPU Go uses, and a container with no CPU limit sees every CPU of the host. It is now half of runtime.GOMAXPROCS(0), rounded down and at least 1, from imageprocessor.DefaultMaxConcurrentProcessing, which both the config and imageprocessor.New use. The two existing tests of the default expect the new one, as the owner allowed on the issue. "Running under upaas" gives the worst-case memory, with a per-image peak of about 1.6 GiB measured on pixad for an 8192x8192 source in each output format, and tells the operator to set PIXA_MAX_CONCURRENT_PROCESSING and PIXA_UPSTREAM_CONNECTIONS to fit the container's memory. Model: opus-5-5
This commit is contained in:
@@ -12,7 +12,6 @@ import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"regexp"
|
||||
"runtime"
|
||||
"sort"
|
||||
"strconv"
|
||||
"strings"
|
||||
@@ -21,6 +20,7 @@ import (
|
||||
|
||||
"git.eeqj.de/sneak/smartconfig"
|
||||
"go.uber.org/fx"
|
||||
"sneak.berlin/go/pixa/internal/imageprocessor"
|
||||
"sneak.berlin/go/pixa/internal/logger"
|
||||
)
|
||||
|
||||
@@ -297,10 +297,8 @@ func newFromSmartConfig(sc *smartconfig.Config) (*Config, error) {
|
||||
keyUpstreamConnectionsPerHost, DefaultUpstreamConnectionsPerHost),
|
||||
UpstreamConnections: loader.intVal(
|
||||
keyUpstreamConnections, DefaultUpstreamConnections),
|
||||
// Decoding and encoding are CPU-bound, so the default is one image
|
||||
// per CPU Go uses, which follows a container's CPU limit.
|
||||
MaxConcurrentProcessing: loader.intVal(
|
||||
keyMaxConcurrentProcessing, runtime.GOMAXPROCS(0)),
|
||||
MaxConcurrentProcessing: loader.intVal(keyMaxConcurrentProcessing,
|
||||
imageprocessor.DefaultMaxConcurrentProcessing()),
|
||||
UpstreamFetchTimeout: loader.durationVal(
|
||||
keyUpstreamFetchTimeout, DefaultUpstreamFetchTimeout),
|
||||
UpstreamMaxResponseSize: loader.int64Val(
|
||||
|
||||
Reference in New Issue
Block a user