Compare commits

1 Commits
Author SHA1 Message Date
clawbot 0969cf43bb Install libvips JPEG XL support and require it at startup (part of #222)
check / check (push) Canceled after 0s
script/bootstrap --cgo installs vips-jxl on Alpine, where libvips' JPEG
XL loader and saver are a package of their own; the nix and brew
libvips include them, as does apt's from Debian 12 and Ubuntu 24.04 on.
The runtime stage of the Dockerfile installs vips-jxl too.

imageprocessor.New now returns an error when libvips cannot load and
save JPEG XL, and NewService passes it on, so pixad does not start
without that support. JPEG XL becomes the default output later in the
issue, which a pixad without it could not serve. A new test saves an
image as JPEG XL with govips and loads it back.

Model: opus-5-5
2026-10-08 01:06:54 +00:00
11 changed files with 195 additions and 149 deletions
+4 -7
View File
@@ -532,13 +532,10 @@ Key settings in more detail:
- `signing_key` — HMAC secret for URL signatures - `signing_key` — HMAC secret for URL signatures
- `db_url` — the SQLite database to open; omitted, it is - `db_url` — the SQLite database to open; omitted, it is
`file:<state_dir>/state.sqlite3?_pragma=journal_mode(WAL)`, which keeps the `file:<state_dir>/state.sqlite3?_pragma=journal_mode(WAL)`, which keeps the
database in WAL mode. pixa opens one connection to it, so its own reads and database in WAL mode. pixa adds `_pragma=busy_timeout(5000)` to any `db_url`,
writes run one at a time. Requests wait while eviction runs one of its so a write that finds another in progress waits up to five seconds for it
queries, some of which read a whole table. pixa adds instead of failing. WAL mode comes only from the URL: keep
`_pragma=busy_timeout(5000)` to any `db_url`, so a write that finds another `_pragma=journal_mode(WAL)` in one you set
program writing to the file waits up to five seconds for it instead of
failing. WAL mode comes only from the URL: keep `_pragma=journal_mode(WAL)` in
one you set
- `cache_max_bytes` — disk cache size limit in bytes; `0` disables the disk - `cache_max_bytes` — disk cache size limit in bytes; `0` disables the disk
cache entirely; omitted defaults to 75% of the sum of the free space on the cache entirely; omitted defaults to 75% of the sum of the free space on the
filesystem containing `<state_dir>/cache/` and the bytes of source and filesystem containing `<state_dir>/cache/` and the bytes of source and
+5 -14
View File
@@ -31,20 +31,11 @@ P2: security: per-IP rate limiting on the image routes
# Completed Steps # Completed Steps
- 2026-10-08 libvips' JPEG XL support is installed and required (part of #222): - 2026-10-08 libvips' JPEG XL support is installed and required (part of #222):
`script/bootstrap --cgo` installs `vips-jxl` when its package manager is apk, `script/bootstrap --cgo` installs `vips-jxl` on Alpine, whose `vips` package
as Alpine's `vips` package lacks the support, and the runtime stage of the lacks it, and the runtime stage of the `Dockerfile` installs it too.
`Dockerfile` installs it too. `imgcache.NewService` fails, naming the fix, `imageprocessor.New` fails when libvips cannot load and save JPEG XL, so pixad
when `imageprocessor.CheckJPEGXLSupport` finds that libvips cannot load and does not start without it. A test saves an image as JPEG XL with govips and
save JPEG XL, so pixad does not start without it. A test saves an image as loads it back. JPEG XL is not yet a format pixa serves.
JPEG XL with govips and loads it back. JPEG XL is not yet a format pixa
serves.
- 2026-10-08 SQLite writes no longer fail with "database is locked" under load
(closes #223): `internal/database` opens the database with one connection, so
pixa's own reads and writes run on it one at a time instead of competing for
SQLite's lock, where a write that kept losing could wait past the five-second
busy timeout and be lost. The busy timeout stays, for another program writing
to the same file. No code in pixa keeps rows or a transaction open while it
runs another query, which with one connection would wait forever.
- 2026-10-05 the format `auto` (closes #88): a format in the `/v1/image/` path, - 2026-10-05 the format `auto` (closes #88): a format in the `/v1/image/` path,
an encrypted URL's token and the generator page's format choice, chosen for an encrypted URL's token and the generator page's format choice, chosen for
each request from `Accept` once the signature or token is checked: AVIF when each request from `Accept` once the signature or token is checked: AVIF when
@@ -13,10 +13,11 @@ import (
) )
// TestConcurrentWritesAllSucceed opens a database the way pixad does and // TestConcurrentWritesAllSucceed opens a database the way pixad does and
// writes to it from several goroutines at once, as one request's writes and // writes to it from several goroutines at once, so the writes run on
// the background eviction pass do. Every write must succeed, none failing // separate connections, as one request's writes and the background eviction
// with "database is locked", whether or not db_url already has parameters, // pass do. Every write must succeed, none failing with "database is locked",
// and the parameters it has must still apply. // whether or not db_url already has parameters, and the parameters it has
// must still apply.
func TestConcurrentWritesAllSucceed(t *testing.T) { func TestConcurrentWritesAllSucceed(t *testing.T) {
t.Parallel() t.Parallel()
+6 -12
View File
@@ -237,18 +237,17 @@ func ApplyMigrations(ctx context.Context, db *sql.DB, log *slog.Logger) error {
return nil return nil
} }
// DB returns the underlying sql.DB. It has one connection, so close any // DB returns the underlying sql.DB.
// rows and end any transaction before running another query on it; a
// query run while they are open waits forever.
func (s *Database) DB() *sql.DB { func (s *Database) DB() *sql.DB {
return s.db return s.db
} }
func (s *Database) connect(ctx context.Context) error { func (s *Database) connect(ctx context.Context) error {
// With a busy timeout, a write that finds another program writing to // Requests and the eviction pass write on separate connections. With
// the same database file waits up to five seconds for it instead of // a busy timeout, a write that finds another one in progress waits up
// failing at once with "database is locked". The driver runs each // to five seconds for it instead of failing at once with "database is
// _pragma parameter on every connection it opens. // locked". The driver runs each _pragma parameter on every connection
// it opens.
separator := "?" separator := "?"
if strings.Contains(s.config.DBURL, "?") { if strings.Contains(s.config.DBURL, "?") {
separator = "&" separator = "&"
@@ -265,11 +264,6 @@ func (s *Database) connect(ctx context.Context) error {
return err return err
} }
// One connection: pixa's own reads and writes run on it one at a
// time instead of competing for SQLite's lock, where a write that
// keeps losing can wait past the busy timeout and be lost.
db.SetMaxOpenConns(1)
err = db.PingContext(ctx) err = db.PingContext(ctx)
if err != nil { if err != nil {
s.log.Error("failed to ping database", "error", err) s.log.Error("failed to ping database", "error", err)
+13 -15
View File
@@ -19,13 +19,17 @@ import (
//nolint:gochecknoglobals // package-level sync.Once for one-time vips init //nolint:gochecknoglobals // package-level sync.Once for one-time vips init
var vipsOnce sync.Once var vipsOnce sync.Once
// errNoJPEGXL is returned by New when libvips cannot load and save JPEG XL.
var errNoJPEGXL = errors.New("libvips lacks JPEG XL support")
// initVips initializes libvips with quiet logging, one worker thread per // initVips initializes libvips with quiet logging, one worker thread per
// image and no operation cache. Process already works on one image per CPU // image and no operation cache. Process already works on one image per CPU
// by default, so more threads per image would only compete for the CPUs. // by default, so more threads per image would only compete for the CPUs.
// Each request decodes different source bytes, so the operation cache // Each request decodes different source bytes, so the operation cache
// would rarely be hit and would hold memory outside MaxConcurrentProcessing; // would rarely be hit and would hold memory outside MaxConcurrentProcessing;
// repeated requests are served from pixa's disk cache instead. // repeated requests are served from pixa's disk cache instead.
func initVips() { // It returns errNoJPEGXL when libvips cannot load and save JPEG XL.
func initVips() error {
vipsOnce.Do(func() { vipsOnce.Do(func() {
vips.LoggingSettings(nil, vips.LogLevelError) vips.LoggingSettings(nil, vips.LogLevelError)
vips.Startup(&vips.Config{ vips.Startup(&vips.Config{
@@ -35,16 +39,6 @@ func initVips() {
MaxCacheFiles: 0, MaxCacheFiles: 0,
}) })
}) })
}
// errNoJPEGXL is returned by CheckJPEGXLSupport.
var errNoJPEGXL = errors.New("libvips lacks JPEG XL support: install " +
"vips-jxl on Alpine, or use a libvips built with libjxl")
// CheckJPEGXLSupport returns an error, naming the fix, when libvips
// cannot load and save JPEG XL.
func CheckJPEGXLSupport() error {
initVips()
// govips counts a format as supported when libvips has its loader; // govips counts a format as supported when libvips has its loader;
// libvips builds the JPEG XL loader and saver together. // libvips builds the JPEG XL loader and saver together.
@@ -167,9 +161,13 @@ type Params struct {
} }
// New creates a new image processor with the given parameters. // New creates a new image processor with the given parameters.
// A zero-value Params{} uses sensible defaults. // A zero-value Params{} uses sensible defaults. It fails when libvips
func New(params Params) *ImageProcessor { // cannot load and save JPEG XL.
initVips() func New(params Params) (*ImageProcessor, error) {
err := initVips()
if err != nil {
return nil, err
}
maxInputBytes := params.MaxInputBytes maxInputBytes := params.MaxInputBytes
if maxInputBytes <= 0 { if maxInputBytes <= 0 {
@@ -185,7 +183,7 @@ func New(params Params) *ImageProcessor {
maxInputBytes: maxInputBytes, maxInputBytes: maxInputBytes,
processingSemaphore: make(chan struct{}, maxConcurrentProcessing), processingSemaphore: make(chan struct{}, maxConcurrentProcessing),
processingWaitTimeout: ProcessingWaitTimeout, processingWaitTimeout: ProcessingWaitTimeout,
} }, nil
} }
// Process transforms an image according to the request. When // Process transforms an image according to the request. When
@@ -18,7 +18,10 @@ import (
) )
func TestMain(m *testing.M) { func TestMain(m *testing.M) {
initVips() err := initVips()
if err != nil {
panic(err)
}
code := m.Run() code := m.Run()
@@ -118,7 +121,11 @@ func detectMIME(data []byte) string {
func TestImageProcessor_ResizeJPEG(t *testing.T) { func TestImageProcessor_ResizeJPEG(t *testing.T) {
t.Parallel() t.Parallel()
proc := New(Params{}) proc, err := New(Params{})
if err != nil {
t.Fatalf("New() error = %v", err)
}
ctx := context.Background() ctx := context.Background()
input := createTestJPEG(t, 800, 600) input := createTestJPEG(t, 800, 600)
@@ -164,7 +171,11 @@ func TestImageProcessor_ResizeJPEG(t *testing.T) {
func TestImageProcessor_ConvertToPNG(t *testing.T) { func TestImageProcessor_ConvertToPNG(t *testing.T) {
t.Parallel() t.Parallel()
proc := New(Params{}) proc, err := New(Params{})
if err != nil {
t.Fatalf("New() error = %v", err)
}
ctx := context.Background() ctx := context.Background()
input := createTestJPEG(t, 200, 150) input := createTestJPEG(t, 200, 150)
@@ -200,7 +211,11 @@ func processAndCheckSize(
) { ) {
t.Helper() t.Helper()
proc := New(Params{}) proc, err := New(Params{})
if err != nil {
t.Fatalf("New() error = %v", err)
}
ctx := context.Background() ctx := context.Background()
input := createTestJPEG(t, inputW, inputH) input := createTestJPEG(t, inputW, inputH)
@@ -238,7 +253,11 @@ func TestImageProcessor_OriginalSize(t *testing.T) {
func TestImageProcessor_FitContain(t *testing.T) { func TestImageProcessor_FitContain(t *testing.T) {
t.Parallel() t.Parallel()
proc := New(Params{}) proc, err := New(Params{})
if err != nil {
t.Fatalf("New() error = %v", err)
}
ctx := context.Background() ctx := context.Background()
// 800x400 image (2:1 aspect) into 400x400 box with contain // 800x400 image (2:1 aspect) into 400x400 box with contain
@@ -284,7 +303,11 @@ func TestImageProcessor_ProportionalScale_HeightOnly(t *testing.T) {
func TestImageProcessor_ProcessPNG(t *testing.T) { func TestImageProcessor_ProcessPNG(t *testing.T) {
t.Parallel() t.Parallel()
proc := New(Params{}) proc, err := New(Params{})
if err != nil {
t.Fatalf("New() error = %v", err)
}
ctx := context.Background() ctx := context.Background()
input := createTestPNG(t, 400, 300) input := createTestPNG(t, 400, 300)
@@ -314,7 +337,10 @@ func TestImageProcessor_ProcessPNG(t *testing.T) {
func TestImageProcessor_SupportedFormats(t *testing.T) { func TestImageProcessor_SupportedFormats(t *testing.T) {
t.Parallel() t.Parallel()
proc := New(Params{}) proc, err := New(Params{})
if err != nil {
t.Fatalf("New() error = %v", err)
}
inputFormats := proc.SupportedInputFormats() inputFormats := proc.SupportedInputFormats()
if len(inputFormats) == 0 { if len(inputFormats) == 0 {
@@ -345,7 +371,11 @@ func TestImageProcessor_RejectsOversizedInput(t *testing.T) {
t.Run(tt.name, func(t *testing.T) { t.Run(tt.name, func(t *testing.T) {
t.Parallel() t.Parallel()
proc := New(Params{}) proc, err := New(Params{})
if err != nil {
t.Fatalf("New() error = %v", err)
}
ctx := context.Background() ctx := context.Background()
input := createTestJPEG(t, tt.width, tt.height) input := createTestJPEG(t, tt.width, tt.height)
@@ -356,7 +386,7 @@ func TestImageProcessor_RejectsOversizedInput(t *testing.T) {
FitMode: FitCover, FitMode: FitCover,
} }
_, err := proc.Process(ctx, bytes.NewReader(input), req) _, err = proc.Process(ctx, bytes.NewReader(input), req)
if err == nil { if err == nil {
t.Error("Process() should reject oversized input images") t.Error("Process() should reject oversized input images")
} }
@@ -371,7 +401,11 @@ func TestImageProcessor_RejectsOversizedInput(t *testing.T) {
func TestImageProcessor_AcceptsMaxDimensionInput(t *testing.T) { func TestImageProcessor_AcceptsMaxDimensionInput(t *testing.T) {
t.Parallel() t.Parallel()
proc := New(Params{}) proc, err := New(Params{})
if err != nil {
t.Fatalf("New() error = %v", err)
}
ctx := context.Background() ctx := context.Background()
// Create an image at exactly MaxInputDimension - should be accepted // Create an image at exactly MaxInputDimension - should be accepted
@@ -400,7 +434,11 @@ func TestImageProcessor_AcceptsMaxDimensionInput(t *testing.T) {
func encodeAndCheck(t *testing.T, format Format, quality int, wantMIME string) { func encodeAndCheck(t *testing.T, format Format, quality int, wantMIME string) {
t.Helper() t.Helper()
proc := New(Params{}) proc, err := New(Params{})
if err != nil {
t.Fatalf("New() error = %v", err)
}
ctx := context.Background() ctx := context.Background()
input := createTestJPEG(t, 200, 150) input := createTestJPEG(t, 200, 150)
@@ -449,7 +487,11 @@ func TestImageProcessor_EncodeWebP(t *testing.T) {
func TestImageProcessor_DecodeAVIF(t *testing.T) { func TestImageProcessor_DecodeAVIF(t *testing.T) {
t.Parallel() t.Parallel()
proc := New(Params{}) proc, err := New(Params{})
if err != nil {
t.Fatalf("New() error = %v", err)
}
ctx := context.Background() ctx := context.Background()
// Load test AVIF file // Load test AVIF file
@@ -491,7 +533,11 @@ func TestImageProcessor_RejectsOversizedInputData(t *testing.T) {
// Create a processor with a very small byte limit // Create a processor with a very small byte limit
const limit = 1024 const limit = 1024
proc := New(Params{MaxInputBytes: limit}) proc, err := New(Params{MaxInputBytes: limit})
if err != nil {
t.Fatalf("New() error = %v", err)
}
ctx := context.Background() ctx := context.Background()
// Create a valid JPEG that exceeds the byte limit // Create a valid JPEG that exceeds the byte limit
@@ -507,7 +553,7 @@ func TestImageProcessor_RejectsOversizedInputData(t *testing.T) {
FitMode: FitCover, FitMode: FitCover,
} }
_, err := proc.Process(ctx, bytes.NewReader(input), req) _, err = proc.Process(ctx, bytes.NewReader(input), req)
if err == nil { if err == nil {
t.Fatal("Process() should reject input exceeding maxInputBytes") t.Fatal("Process() should reject input exceeding maxInputBytes")
} }
@@ -524,7 +570,11 @@ func TestImageProcessor_AcceptsInputWithinLimit(t *testing.T) {
input := createTestJPEG(t, 10, 10) input := createTestJPEG(t, 10, 10)
limit := int64(len(input)) * 10 // 10× headroom limit := int64(len(input)) * 10 // 10× headroom
proc := New(Params{MaxInputBytes: limit}) proc, err := New(Params{MaxInputBytes: limit})
if err != nil {
t.Fatalf("New() error = %v", err)
}
ctx := context.Background() ctx := context.Background()
req := &Request{ req := &Request{
@@ -546,13 +596,21 @@ func TestImageProcessor_DefaultMaxInputBytes(t *testing.T) {
t.Parallel() t.Parallel()
// Passing 0 should use the default // Passing 0 should use the default
proc := New(Params{}) proc, err := New(Params{})
if err != nil {
t.Fatalf("New() error = %v", err)
}
if proc.maxInputBytes != DefaultMaxInputBytes { if proc.maxInputBytes != DefaultMaxInputBytes {
t.Errorf("maxInputBytes = %d, want %d", proc.maxInputBytes, DefaultMaxInputBytes) t.Errorf("maxInputBytes = %d, want %d", proc.maxInputBytes, DefaultMaxInputBytes)
} }
// Passing negative should also use the default // Passing negative should also use the default
proc = New(Params{MaxInputBytes: -1}) proc, err = New(Params{MaxInputBytes: -1})
if err != nil {
t.Fatalf("New() error = %v", err)
}
if proc.maxInputBytes != DefaultMaxInputBytes { if proc.maxInputBytes != DefaultMaxInputBytes {
t.Errorf("maxInputBytes = %d, want %d", proc.maxInputBytes, DefaultMaxInputBytes) t.Errorf("maxInputBytes = %d, want %d", proc.maxInputBytes, DefaultMaxInputBytes)
} }
@@ -564,14 +622,51 @@ func TestImageProcessor_EncodeAVIF(t *testing.T) {
encodeAndCheck(t, FormatAVIF, 85, mimeAVIF) encodeAndCheck(t, FormatAVIF, 85, mimeAVIF)
} }
// TestLibvipsSavesAndLoadsJPEGXL saves an image as JPEG XL with govips and
// loads it back. It fails when libvips lacks JPEG XL support, as on Alpine
// without the vips-jxl package.
func TestLibvipsSavesAndLoadsJPEGXL(t *testing.T) {
t.Parallel()
img, err := vips.NewImageFromBuffer(createTestJPEG(t, 64, 48))
if err != nil {
t.Fatalf("failed to load test JPEG: %v", err)
}
defer img.Close()
jxl, _, err := img.ExportJxl(vips.NewJxlExportParams())
if err != nil {
t.Fatalf("ExportJxl() error = %v", err)
}
loaded, err := vips.NewImageFromBuffer(jxl)
if err != nil {
t.Fatalf("failed to load the JPEG XL image: %v", err)
}
defer loaded.Close()
if loaded.Format() != vips.ImageTypeJXL {
t.Errorf("loaded format = %s, want jxl", vips.ImageTypes[loaded.Format()])
}
if loaded.Width() != 64 || loaded.Height() != 48 {
t.Errorf("loaded size = %dx%d, want 64x48", loaded.Width(), loaded.Height())
}
}
// processAndDecode runs input through Process and decodes the output with // processAndDecode runs input through Process and decodes the output with
// vips, so a test can inspect the image a client would receive. // vips, so a test can inspect the image a client would receive.
func processAndDecode(t *testing.T, input []byte, req *Request) *vips.ImageRef { func processAndDecode(t *testing.T, input []byte, req *Request) *vips.ImageRef {
t.Helper() t.Helper()
result, err := New(Params{}).Process( proc, err := New(Params{})
context.Background(), bytes.NewReader(input), req, if err != nil {
) t.Fatalf("New() error = %v", err)
}
result, err := proc.Process(context.Background(), bytes.NewReader(input), req)
if err != nil { if err != nil {
t.Fatalf("Process() error = %v", err) t.Fatalf("Process() error = %v", err)
} }
@@ -1,52 +0,0 @@
package imageprocessor
import (
"testing"
"github.com/davidbyttow/govips/v2/vips"
)
// TestCheckJPEGXLSupport fails when libvips lacks JPEG XL support, as on
// Alpine without the vips-jxl package.
func TestCheckJPEGXLSupport(t *testing.T) {
t.Parallel()
err := CheckJPEGXLSupport()
if err != nil {
t.Fatalf("CheckJPEGXLSupport() error = %v", err)
}
}
// TestLibvipsSavesAndLoadsJPEGXL saves an image as JPEG XL with govips and
// loads it back. It fails when libvips lacks JPEG XL support, as on Alpine
// without the vips-jxl package.
func TestLibvipsSavesAndLoadsJPEGXL(t *testing.T) {
t.Parallel()
img, err := vips.NewImageFromBuffer(createTestJPEG(t, 64, 48))
if err != nil {
t.Fatalf("failed to load test JPEG: %v", err)
}
defer img.Close()
jxl, _, err := img.ExportJxl(vips.NewJxlExportParams())
if err != nil {
t.Fatalf("ExportJxl() error = %v", err)
}
loaded, err := vips.NewImageFromBuffer(jxl)
if err != nil {
t.Fatalf("failed to load the JPEG XL image: %v", err)
}
defer loaded.Close()
if loaded.Format() != vips.ImageTypeJXL {
t.Errorf("loaded format = %s, want jxl", vips.ImageTypes[loaded.Format()])
}
if loaded.Width() != 64 || loaded.Height() != 48 {
t.Errorf("loaded size = %dx%d, want 64x48", loaded.Width(), loaded.Height())
}
}
@@ -119,14 +119,22 @@ func TestNewDefaultsMaxConcurrentProcessingToCPUs(t *testing.T) {
t.Parallel() t.Parallel()
for _, limit := range []int{0, -1} { for _, limit := range []int{0, -1} {
proc := New(Params{MaxConcurrentProcessing: limit}) proc, err := New(Params{MaxConcurrentProcessing: limit})
if err != nil {
t.Fatalf("New() error = %v", err)
}
if got := cap(proc.processingSemaphore); got != runtime.GOMAXPROCS(0) { if got := cap(proc.processingSemaphore); got != runtime.GOMAXPROCS(0) {
t.Errorf("MaxConcurrentProcessing %d: %d slots, want %d, one per CPU", t.Errorf("MaxConcurrentProcessing %d: %d slots, want %d, one per CPU",
limit, got, runtime.GOMAXPROCS(0)) limit, got, runtime.GOMAXPROCS(0))
} }
} }
proc := New(Params{MaxConcurrentProcessing: 3}) proc, err := New(Params{MaxConcurrentProcessing: 3})
if err != nil {
t.Fatalf("New() error = %v", err)
}
if got := cap(proc.processingSemaphore); got != 3 { if got := cap(proc.processingSemaphore); got != 3 {
t.Errorf("MaxConcurrentProcessing 3: %d slots, want 3", got) t.Errorf("MaxConcurrentProcessing 3: %d slots, want 3", got)
} }
@@ -145,7 +153,11 @@ func TestProcessNeverExceedsMaxConcurrentProcessing(t *testing.T) {
calls = 6 calls = 6
) )
proc := New(Params{MaxConcurrentProcessing: limit}) proc, err := New(Params{MaxConcurrentProcessing: limit})
if err != nil {
t.Fatalf("New() error = %v", err)
}
input := createTestJPEG(t, 50, 50) input := createTestJPEG(t, 50, 50)
counter := &readingCounter{} counter := &readingCounter{}
@@ -192,7 +204,11 @@ func TestProcessNeverExceedsMaxConcurrentProcessing(t *testing.T) {
func TestProcessWaitsThenFailsWhenNoSlotFrees(t *testing.T) { func TestProcessWaitsThenFailsWhenNoSlotFrees(t *testing.T) {
t.Parallel() t.Parallel()
proc := New(Params{MaxConcurrentProcessing: 1}) proc, err := New(Params{MaxConcurrentProcessing: 1})
if err != nil {
t.Fatalf("New() error = %v", err)
}
proc.processingWaitTimeout = 100 * time.Millisecond proc.processingWaitTimeout = 100 * time.Millisecond
input := createTestJPEG(t, 10, 10) input := createTestJPEG(t, 10, 10)
@@ -212,7 +228,7 @@ func TestProcessWaitsThenFailsWhenNoSlotFrees(t *testing.T) {
start := time.Now() start := time.Now()
_, err := proc.Process(context.Background(), bytes.NewReader(input), _, err = proc.Process(context.Background(), bytes.NewReader(input),
smallJPEGRequest()) smallJPEGRequest())
if !errors.Is(err, ErrTooManyImages) { if !errors.Is(err, ErrTooManyImages) {
t.Fatalf("Process() error = %v, want ErrTooManyImages", err) t.Fatalf("Process() error = %v, want ErrTooManyImages", err)
@@ -278,10 +294,14 @@ func TestProcessReleasesSlotOnError(t *testing.T) {
t.Run(tc.name, func(t *testing.T) { t.Run(tc.name, func(t *testing.T) {
t.Parallel() t.Parallel()
proc := New(Params{MaxInputBytes: 4096, MaxConcurrentProcessing: 1}) proc, err := New(Params{MaxInputBytes: 4096, MaxConcurrentProcessing: 1})
if err != nil {
t.Fatalf("New() error = %v", err)
}
proc.processingWaitTimeout = 100 * time.Millisecond proc.processingWaitTimeout = 100 * time.Millisecond
_, err := proc.Process(context.Background(), tc.input, tc.req) _, err = proc.Process(context.Background(), tc.input, tc.req)
if err == nil || (tc.want != nil && !errors.Is(err, tc.want)) { if err == nil || (tc.want != nil && !errors.Is(err, tc.want)) {
t.Fatalf("Process() error = %v, want %v", err, tc.want) t.Fatalf("Process() error = %v, want %v", err, tc.want)
} }
@@ -308,7 +328,10 @@ func TestProcessReleasesSlotOnError(t *testing.T) {
func TestWaitForProcessing(t *testing.T) { func TestWaitForProcessing(t *testing.T) {
t.Parallel() t.Parallel()
proc := New(Params{MaxConcurrentProcessing: 2}) proc, err := New(Params{MaxConcurrentProcessing: 2})
if err != nil {
t.Fatalf("New() error = %v", err)
}
gate := make(chan struct{}) gate := make(chan struct{})
entered := make(chan struct{}, 1) entered := make(chan struct{}, 1)
@@ -347,7 +370,7 @@ func TestWaitForProcessing(t *testing.T) {
openGate() openGate()
err := <-results err = <-results
if err != nil { if err != nil {
t.Errorf("Process() error = %v, want nil", err) t.Errorf("Process() error = %v, want nil", err)
} }
@@ -60,9 +60,15 @@ func TestService_Get_WaitsForSlotBeforeReadingCachedSource(t *testing.T) {
t.Parallel() t.Parallel()
svc, fixtures := SetupTestService(t) svc, fixtures := SetupTestService(t)
svc.processor = imageprocessor.New(
processor, err := imageprocessor.New(
imageprocessor.Params{MaxConcurrentProcessing: 1}, imageprocessor.Params{MaxConcurrentProcessing: 1},
) )
if err != nil {
t.Fatalf("imageprocessor.New() error = %v", err)
}
svc.processor = processor
// A first request caches the photo as a source. // A first request caches the photo as a source.
resp, err := svc.Get(t.Context(), widthOnlyRequest(fixtures, 50)) resp, err := svc.Get(t.Context(), widthOnlyRequest(fixtures, 50))
+5 -8
View File
@@ -100,18 +100,15 @@ func NewService(cfg *ServiceConfig) (*Service, error) {
allowHTTP = cfg.FetcherConfig.AllowHTTP allowHTTP = cfg.FetcherConfig.AllowHTTP
} }
// JPEG XL is to become the default output format, so pixad does not
// start without it.
err := imageprocessor.CheckJPEGXLSupport()
if err != nil {
return nil, err
}
maxResponseSize := fetcherCfg.MaxResponseSize maxResponseSize := fetcherCfg.MaxResponseSize
processor := imageprocessor.New(imageprocessor.Params{
processor, err := imageprocessor.New(imageprocessor.Params{
MaxInputBytes: maxResponseSize, MaxInputBytes: maxResponseSize,
MaxConcurrentProcessing: cfg.MaxConcurrentProcessing, MaxConcurrentProcessing: cfg.MaxConcurrentProcessing,
}) })
if err != nil {
return nil, err
}
return &Service{ return &Service{
cache: cfg.Cache, cache: cfg.Cache,
+3 -7
View File
@@ -152,13 +152,9 @@ ensure_cgo_deps() {
pkg_install vips libvips-dev vips vips-dev pkg_install vips libvips-dev vips vips-dev
fi fi
# libvips' JPEG XL loader and saver are in the nix and brew vips # libvips' JPEG XL loader and saver are in the nix and brew vips
# packages, and in apt's from Debian 12 and Ubuntu 24.04 on, but in # packages, and in apt's from Debian 12 and Ubuntu 24.04 on, but in a
# the package vips-jxl on Alpine. detect_pkgmgr is called only where # package of their own on Alpine.
# apk exists, as on apt it updates the package lists. if command -v apk >/dev/null 2>&1 && ! apk info -e vips-jxl >/dev/null; then
if ! missing apk; then
detect_pkgmgr
fi
if [ "$PKGMGR" = "apk" ] && ! apk info -e vips-jxl >/dev/null; then
apk add --no-cache vips-jxl apk add --no-cache vips-jxl
fi fi
if ! pkg-config --exists libheif; then if ! pkg-config --exists libheif; then