Compare commits
2
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
47a2c2360a | ||
|
|
84a86274cd |
@@ -336,6 +336,12 @@ either side may be; so is a size with `fit=outside`, which scales the image to
|
|||||||
cover the size and so past it on one side, when that side would come out
|
cover the size and so past it on one side, when that side would come out
|
||||||
above 8192.
|
above 8192.
|
||||||
|
|
||||||
|
An AVIF image of more than 16,777,216 pixels, the number 4096x4096 and 8192x2048
|
||||||
|
have, is refused with 400 before anything is decoded, as its encode could take
|
||||||
|
more than 40 seconds (see below). That includes `orig` of an AVIF source, and
|
||||||
|
`auto` when it chooses AVIF, so a client whose `Accept` names `image/avif` but
|
||||||
|
not `image/jxl` is refused such a size.
|
||||||
|
|
||||||
The source image may be JPEG, PNG, GIF, WebP, AVIF or JPEG XL. The upstream's
|
The source image may be JPEG, PNG, GIF, WebP, AVIF or JPEG XL. The upstream's
|
||||||
`Content-Type` must name its format, and the image's first bytes must match it.
|
`Content-Type` must name its format, and the image's first bytes must match it.
|
||||||
An upstream answering with any other `Content-Type`, SVG included, is refused
|
An upstream answering with any other `Content-Type`, SVG included, is refused
|
||||||
@@ -385,14 +391,18 @@ transcode that stops this way is answered 503 with the error
|
|||||||
ended. govips gives pixa no way to stop libvips once an encode has begun, so an
|
ended. govips gives pixa no way to stop libvips once an encode has begun, so an
|
||||||
encode keeps its slot until it finishes, which can be after
|
encode keeps its slot until it finishes, which can be after
|
||||||
`downstream_timeout`. The limit of 8192 on each side of the source and of the
|
`downstream_timeout`. The limit of 8192 on each side of the source and of the
|
||||||
output bounds how long that takes. With one libvips thread, as pixad runs it,
|
output, and for AVIF the limit of 16,777,216 pixels, bound how long that takes.
|
||||||
encoding 8192x8192 random pixels takes up to about 1 second as JPEG, 15 as PNG,
|
With one libvips thread, as pixad runs it, encoding 8192x8192 random pixels
|
||||||
16 as GIF, 63 as WebP, 110 as AVIF and 96 as JPEG XL, the slowest of 8 or 16
|
takes up to about 1 second as JPEG, 15 as PNG and 16 as GIF, the slowest of 8 or
|
||||||
bits per sample, with or without an alpha channel. WebP, AVIF and JPEG XL are
|
16 bits per sample, with or without an alpha channel. The WebP and JPEG XL
|
||||||
that slow only with an alpha channel: 8-bit RGB takes about 44, 53 and 28
|
encoders run below libvips' default effort, and AVIF at the lowest effort govips
|
||||||
seconds. Decoding such a source adds up to about 11 seconds. A WebP, AVIF or
|
can set, so that no encode takes more than 40 seconds of the default
|
||||||
JPEG XL encode can therefore hold its slot past the default `downstream_timeout`
|
`downstream_timeout` of 60: at any quality and 8 or 16 bits per sample,
|
||||||
of 60 seconds.
|
8192x8192 random pixels with an alpha channel take up to about 23 seconds as
|
||||||
|
WebP and 27 as JPEG XL, and 4096x4096, 8192x2048 or 2048x8192 up to about 34 as
|
||||||
|
AVIF, which at quality 85 would take about 96 at 8192x8192. Without an alpha
|
||||||
|
channel they take up to about 16, 13 and 22 seconds. Decoding such a source adds
|
||||||
|
up to about 11 seconds.
|
||||||
|
|
||||||
The login form (`POST /`) is limited to 5 attempts per minute per client
|
The login form (`POST /`) is limited to 5 attempts per minute per client
|
||||||
address, counting an IPv6 client by its /64; an attempt over the limit is
|
address, counting an IPv6 client by its /64; an attempt over the limit is
|
||||||
@@ -444,7 +454,8 @@ nor change what it asks for.
|
|||||||
sent with an empty format, or none, also makes a JPEG XL URL. Width and
|
sent with an empty format, or none, also makes a JPEG XL URL. Width and
|
||||||
height both empty or `0` keep the original size; if only one of them is empty
|
height both empty or `0` keep the original size; if only one of them is empty
|
||||||
or `0`, that side is scaled to keep the image's proportions. A URL whose
|
or `0`, that side is scaled to keep the image's proportions. A URL whose
|
||||||
scaled side would come out above 8192 answers 400.
|
scaled side would come out above 8192 answers 400, and so does an AVIF URL of
|
||||||
|
more than 16,777,216 pixels (see Routes).
|
||||||
3. The page shows the URL, `https://<host>/v1/e/<token>/img.<format>`, and when
|
3. The page shows the URL, `https://<host>/v1/e/<token>/img.<format>`, and when
|
||||||
it expires. `<host>` is the host the page was opened on, and the URL starts
|
it expires. `<host>` is the host the page was opened on, and the URL starts
|
||||||
with `http` instead while `debug` is on. The name after the token is ignored
|
with `http` instead while `debug` is on. The name after the token is ignored
|
||||||
|
|||||||
@@ -30,6 +30,18 @@ P2: security: per-origin rate limiting
|
|||||||
|
|
||||||
# Completed Steps
|
# Completed Steps
|
||||||
|
|
||||||
|
- 2026-10-15 a WebP, AVIF or JPEG XL encode at the limits takes at most about 34
|
||||||
|
seconds (closes #272): with one libvips thread, 8192x8192 random pixels with
|
||||||
|
an alpha channel took about 61 seconds to encode as WebP, 96 as AVIF and 81 as
|
||||||
|
JPEG XL, so with the default `downstream_timeout` of 60 seconds such an image
|
||||||
|
was never delivered, yet its encode held a processing slot to the end. WebP is
|
||||||
|
now saved at effort 2 and JPEG XL at effort 4, which take up to about 23 and
|
||||||
|
27 seconds. AVIF was already at the lowest effort govips can set, so an AVIF
|
||||||
|
output of more than 16,777,216 pixels is refused with 400 before anything is
|
||||||
|
decoded; 4096x4096, 8192x2048 or 2048x8192 takes up to about 34 seconds. Each
|
||||||
|
time is the slowest at any quality. Resizing with `fit=cover` already decodes
|
||||||
|
the input, so the output's dimensions are worked out before resizing.
|
||||||
|
`README.md` gives the limit and the new times.
|
||||||
- 2026-10-15 the README says how a source URL with a query is written into an
|
- 2026-10-15 the README says how a source URL with a query is written into an
|
||||||
image URL, and the image route accepts it (closes #255): the query follows the
|
image URL, and the image route accepts it (closes #255): the query follows the
|
||||||
source path after `%3F`. pixa now reads the image URL's path decoded once, so
|
source path after `%3F`. pixa now reads the image URL's path decoded once, so
|
||||||
|
|||||||
@@ -0,0 +1,49 @@
|
|||||||
|
package handlers
|
||||||
|
|
||||||
|
import (
|
||||||
|
"fmt"
|
||||||
|
"log/slog"
|
||||||
|
"net/http"
|
||||||
|
"net/http/httptest"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"sneak.berlin/go/pixa/internal/imageprocessor"
|
||||||
|
"sneak.berlin/go/pixa/internal/imgcache"
|
||||||
|
)
|
||||||
|
|
||||||
|
// TestAVIFOutputTooLargeAnswers400 checks that respondImageError and
|
||||||
|
// handleImageError, which answer the errors of the two image routes, answer
|
||||||
|
// 400 naming the AVIF pixel limit when the image processor refuses an AVIF
|
||||||
|
// output above it, wrapped as the service wraps it. The request's source only
|
||||||
|
// goes into a log line, so it is left empty.
|
||||||
|
func TestAVIFOutputTooLargeAnswers400(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
h := &Handlers{log: slog.New(slog.DiscardHandler)}
|
||||||
|
err := fmt.Errorf("image processing failed: %w: 4097x4096",
|
||||||
|
imageprocessor.ErrAVIFOutputTooLarge)
|
||||||
|
wantBody := fmt.Sprintf("above %d pixels", imageprocessor.MaxAVIFOutputPixels)
|
||||||
|
|
||||||
|
plain := httptest.NewRecorder()
|
||||||
|
h.respondImageError(plain, &imgcache.ImageRequest{}, err)
|
||||||
|
|
||||||
|
encrypted := httptest.NewRecorder()
|
||||||
|
h.handleImageError(encrypted, err)
|
||||||
|
|
||||||
|
for name, rec := range map[string]*httptest.ResponseRecorder{
|
||||||
|
"respondImageError": plain, "handleImageError": encrypted,
|
||||||
|
} {
|
||||||
|
t.Logf("%s: status = %d, body = %q", name, rec.Code, rec.Body.String())
|
||||||
|
|
||||||
|
if rec.Code != http.StatusBadRequest {
|
||||||
|
t.Errorf("%s: status = %d, want %d",
|
||||||
|
name, rec.Code, http.StatusBadRequest)
|
||||||
|
}
|
||||||
|
|
||||||
|
if !strings.Contains(rec.Body.String(), wantBody) {
|
||||||
|
t.Errorf("%s: body = %q, want it to say %q",
|
||||||
|
name, rec.Body.String(), wantBody)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -264,6 +264,14 @@ func (s *Handlers) respondImageError(
|
|||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
|
||||||
|
if errors.Is(err, imageprocessor.ErrAVIFOutputTooLarge) {
|
||||||
|
s.respondError(w, fmt.Sprintf("size too large for AVIF: the output "+
|
||||||
|
"would be above %d pixels", imageprocessor.MaxAVIFOutputPixels),
|
||||||
|
http.StatusBadRequest)
|
||||||
|
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
s.respondError(w, "internal error", http.StatusInternalServerError)
|
s.respondError(w, "internal error", http.StatusInternalServerError)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -170,6 +170,10 @@ func (s *Handlers) handleImageError(w http.ResponseWriter, err error) {
|
|||||||
s.respondError(w, fmt.Sprintf("size too large for the source: the side "+
|
s.respondError(w, fmt.Sprintf("size too large for the source: the side "+
|
||||||
"computed from its proportions is above %d",
|
"computed from its proportions is above %d",
|
||||||
imageprocessor.MaxOutputDimension), http.StatusBadRequest)
|
imageprocessor.MaxOutputDimension), http.StatusBadRequest)
|
||||||
|
case errors.Is(err, imageprocessor.ErrAVIFOutputTooLarge):
|
||||||
|
s.respondError(w, fmt.Sprintf("size too large for AVIF: the output "+
|
||||||
|
"would be above %d pixels", imageprocessor.MaxAVIFOutputPixels),
|
||||||
|
http.StatusBadRequest)
|
||||||
default:
|
default:
|
||||||
s.log.Error("image request failed", "error", err)
|
s.log.Error("image request failed", "error", err)
|
||||||
s.respondError(w, "internal error", http.StatusInternalServerError)
|
s.respondError(w, "internal error", http.StatusInternalServerError)
|
||||||
|
|||||||
@@ -0,0 +1,64 @@
|
|||||||
|
package imageprocessor
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"errors"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
// TestImageProcessor_RefusesAVIFAboveMaxPixels verifies that an AVIF output
|
||||||
|
// of more than MaxAVIFOutputPixels is refused with ErrAVIFOutputTooLarge
|
||||||
|
// before anything is decoded, whatever size and fit give it, and that an AVIF
|
||||||
|
// output of at most that many pixels, or one in another format, is not. The
|
||||||
|
// input is a 64x32 PNG cut off in its pixel data: libvips reads its size from
|
||||||
|
// its header but fails when it decodes it, so a request that gets that failure
|
||||||
|
// was not refused, and a refusal came before any decoding.
|
||||||
|
func TestImageProcessor_RefusesAVIFAboveMaxPixels(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
whole := createTestPNG(t, 64, 32)
|
||||||
|
cutOff := whole[:len(whole)/2]
|
||||||
|
|
||||||
|
tests := []struct {
|
||||||
|
name string
|
||||||
|
format Format
|
||||||
|
width, height int
|
||||||
|
fit FitMode
|
||||||
|
refused bool
|
||||||
|
}{
|
||||||
|
{"AVIF at the limit", FormatAVIF, 4096, 4096, FitCover, false},
|
||||||
|
{"AVIF above the limit", FormatAVIF, 4096, 4097, FitCover, true},
|
||||||
|
{"WebP above the AVIF limit", FormatWebP, 4096, 4097, FitCover, false},
|
||||||
|
// FitInside does not enlarge, so the output stays 64x32
|
||||||
|
{"AVIF inside a larger size", FormatAVIF, 8192, 8192, FitInside, false},
|
||||||
|
// FitOutside scales 64x32 by 128 to cover 8192x1: 8192x4096
|
||||||
|
{"AVIF outside a smaller size", FormatAVIF, 8192, 1, FitOutside, true},
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, tt := range tests {
|
||||||
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
_, err := New(Params{}).Process(t.Context(), bytes.NewReader(cutOff),
|
||||||
|
&Request{
|
||||||
|
Size: Size{Width: tt.width, Height: tt.height},
|
||||||
|
Format: tt.format,
|
||||||
|
FitMode: tt.fit,
|
||||||
|
})
|
||||||
|
t.Logf("Process() error = %v", err)
|
||||||
|
|
||||||
|
if tt.refused {
|
||||||
|
if !errors.Is(err, ErrAVIFOutputTooLarge) {
|
||||||
|
t.Errorf("Process() error = %v, want %v", err, ErrAVIFOutputTooLarge)
|
||||||
|
}
|
||||||
|
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
if err == nil || errors.Is(err, ErrAVIFOutputTooLarge) {
|
||||||
|
t.Errorf("Process() error = %v, want the failure to decode "+
|
||||||
|
"the cut-off input", err)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -77,10 +77,10 @@ func TestImageProcessor_PNGIsCompressed(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// TestImageProcessor_WebPAtDefaultEffort verifies that WebP is saved at
|
// TestImageProcessor_WebPAtEffort2 verifies that WebP is saved at effort 2,
|
||||||
// libvips' default effort, 4: the output is the size of the same image saved
|
// not libvips' default of 4: the output is the size of the same image saved
|
||||||
// at that effort.
|
// at effort 2.
|
||||||
func TestImageProcessor_WebPAtDefaultEffort(t *testing.T) {
|
func TestImageProcessor_WebPAtEffort2(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
input := createTestJPEG(t, 200, 150)
|
input := createTestJPEG(t, 200, 150)
|
||||||
@@ -90,14 +90,14 @@ func TestImageProcessor_WebPAtDefaultEffort(t *testing.T) {
|
|||||||
want, _, err := decode(t, input).ExportWebp(&vips.WebpExportParams{
|
want, _, err := decode(t, input).ExportWebp(&vips.WebpExportParams{
|
||||||
StripMetadata: true,
|
StripMetadata: true,
|
||||||
Quality: 85,
|
Quality: 85,
|
||||||
ReductionEffort: 4,
|
ReductionEffort: 2,
|
||||||
})
|
})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("ExportWebp() error = %v", err)
|
t.Fatalf("ExportWebp() error = %v", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
if size != int64(len(want)) {
|
if size != int64(len(want)) {
|
||||||
t.Errorf("WebP output is %d bytes, want %d, the size at effort 4",
|
t.Errorf("WebP output is %d bytes, want %d, the size at effort 2",
|
||||||
size, len(want))
|
size, len(want))
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -143,3 +143,29 @@ func TestImageProcessor_AVIFAtEffort1(t *testing.T) {
|
|||||||
"and 8 bits", size, len(want))
|
"and 8 bits", size, len(want))
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// TestImageProcessor_JPEGXLAtEffort4 verifies that JPEG XL is saved at
|
||||||
|
// effort 4, not libvips' default of 7: the output is the size of the same
|
||||||
|
// image saved at effort 4, with the distance exportJXL gives quality 85.
|
||||||
|
func TestImageProcessor_JPEGXLAtEffort4(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
input := createTestJPEG(t, 200, 150)
|
||||||
|
|
||||||
|
size := processedSize(t, input, &Request{Format: FormatJXL, Quality: 85})
|
||||||
|
|
||||||
|
params := vips.NewJxlExportParams()
|
||||||
|
params.Quality = 85
|
||||||
|
params.Distance = jxlDistance(85)
|
||||||
|
params.Effort = 4
|
||||||
|
|
||||||
|
want, _, err := decode(t, input).ExportJxl(params)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("ExportJxl() error = %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if size != int64(len(want)) {
|
||||||
|
t.Errorf("JPEG XL output is %d bytes, want %d, the size at effort 4",
|
||||||
|
size, len(want))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
@@ -142,6 +142,19 @@ const MaxOutputDimension = 8192
|
|||||||
// requested size.
|
// requested size.
|
||||||
var ErrOutputTooLarge = errors.New("output image dimensions exceed maximum")
|
var ErrOutputTooLarge = errors.New("output image dimensions exceed maximum")
|
||||||
|
|
||||||
|
// MaxAVIFOutputPixels is the most pixels an AVIF output may have, so that an
|
||||||
|
// encode takes at most 40 seconds of the default downstream_timeout of 60
|
||||||
|
// seconds; AVIF is already at the lowest effort govips can set (avifEffort).
|
||||||
|
// With one thread, as pixad runs libvips, saving 4096x4096, 8192x2048 or
|
||||||
|
// 2048x8192 random pixels with an alpha channel takes up to about 34
|
||||||
|
// seconds, the slowest at any quality and at 8 or 16 bits per sample, and
|
||||||
|
// 5120x4096 at 8 bits already takes about 36.
|
||||||
|
const MaxAVIFOutputPixels = 4096 * 4096
|
||||||
|
|
||||||
|
// ErrAVIFOutputTooLarge is returned, before anything is decoded, when an AVIF
|
||||||
|
// output would have more than MaxAVIFOutputPixels pixels.
|
||||||
|
var ErrAVIFOutputTooLarge = errors.New("AVIF output has too many pixels")
|
||||||
|
|
||||||
// ErrUnsupportedOutputFormat is returned when the requested output format is
|
// ErrUnsupportedOutputFormat is returned when the requested output format is
|
||||||
// not supported.
|
// not supported.
|
||||||
var ErrUnsupportedOutputFormat = errors.New("unsupported output format")
|
var ErrUnsupportedOutputFormat = errors.New("unsupported output format")
|
||||||
@@ -210,7 +223,8 @@ func New(params Params) *ImageProcessor {
|
|||||||
// ErrTooManyImages. When ctx has ended once Process has a slot, or before it
|
// ErrTooManyImages. When ctx has ended once Process has a slot, or before it
|
||||||
// encodes, it returns ctx's error and gives the slot back. govips gives pixa
|
// encodes, it returns ctx's error and gives the slot back. govips gives pixa
|
||||||
// no way to stop libvips once an encode has begun, so an encode keeps its
|
// no way to stop libvips once an encode has begun, so an encode keeps its
|
||||||
// slot until it finishes.
|
// slot until it finishes. An AVIF output of more than MaxAVIFOutputPixels is
|
||||||
|
// refused with ErrAVIFOutputTooLarge before anything is decoded.
|
||||||
func (p *ImageProcessor) Process(
|
func (p *ImageProcessor) Process(
|
||||||
ctx context.Context,
|
ctx context.Context,
|
||||||
input io.Reader,
|
input io.Reader,
|
||||||
@@ -223,18 +237,9 @@ func (p *ImageProcessor) Process(
|
|||||||
|
|
||||||
defer release()
|
defer release()
|
||||||
|
|
||||||
// Read input with a size limit to prevent unbounded memory consumption.
|
data, err := p.readInput(input)
|
||||||
// We read at most maxInputBytes+1 so we can detect if the input exceeds
|
|
||||||
// the limit without consuming additional memory.
|
|
||||||
limited := io.LimitReader(input, p.maxInputBytes+1)
|
|
||||||
|
|
||||||
data, err := io.ReadAll(limited)
|
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, fmt.Errorf("failed to read input: %w", err)
|
return nil, err
|
||||||
}
|
|
||||||
|
|
||||||
if int64(len(data)) > p.maxInputBytes {
|
|
||||||
return nil, ErrInputDataTooLarge
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// govips picks the libvips loader from the input's first bytes with
|
// govips picks the libvips loader from the input's first bytes with
|
||||||
@@ -277,20 +282,28 @@ func (p *ImageProcessor) Process(
|
|||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
|
|
||||||
// Resize if needed
|
|
||||||
if targetWidth != origWidth || targetHeight != origHeight {
|
|
||||||
err := p.resize(img, targetWidth, targetHeight, req.FitMode)
|
|
||||||
if err != nil {
|
|
||||||
return nil, fmt.Errorf("failed to resize: %w", err)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// orig is the source's own format; encode refuses an empty format
|
// orig is the source's own format; encode refuses an empty format
|
||||||
outputFormat := req.Format
|
outputFormat := req.Format
|
||||||
if outputFormat == FormatOriginal {
|
if outputFormat == FormatOriginal {
|
||||||
outputFormat = sourceFormat
|
outputFormat = sourceFormat
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Determine output dimensions; too large ones are refused before
|
||||||
|
// resizing, which with FitCover already decodes the input
|
||||||
|
outputWidth, outputHeight, err := outputDimensions(origWidth, origHeight,
|
||||||
|
targetWidth, targetHeight, req.FitMode, outputFormat)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
// Resize if needed
|
||||||
|
if targetWidth != origWidth || targetHeight != origHeight {
|
||||||
|
err := p.resize(img, outputWidth, outputHeight, req.FitMode)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("failed to resize: %w", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// Encode to target format
|
// Encode to target format
|
||||||
output, err := p.encode(ctx, img, outputFormat, req.Quality)
|
output, err := p.encode(ctx, img, outputFormat, req.Quality)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -443,6 +456,23 @@ func (p *ImageProcessor) acquireSlot(ctx context.Context) (func(), error) {
|
|||||||
return release, nil
|
return release, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// readInput reads input with a size limit to prevent unbounded memory
|
||||||
|
// consumption, refusing more than maxInputBytes with ErrInputDataTooLarge. It
|
||||||
|
// reads at most maxInputBytes+1 so it can detect if the input exceeds the
|
||||||
|
// limit without consuming additional memory.
|
||||||
|
func (p *ImageProcessor) readInput(input io.Reader) ([]byte, error) {
|
||||||
|
data, err := io.ReadAll(io.LimitReader(input, p.maxInputBytes+1))
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("failed to read input: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if int64(len(data)) > p.maxInputBytes {
|
||||||
|
return nil, ErrInputDataTooLarge
|
||||||
|
}
|
||||||
|
|
||||||
|
return data, nil
|
||||||
|
}
|
||||||
|
|
||||||
// detectFormat returns the format string for a govips image type.
|
// detectFormat returns the format string for a govips image type.
|
||||||
func (p *ImageProcessor) detectFormat(imageType vips.ImageType) string {
|
func (p *ImageProcessor) detectFormat(imageType vips.ImageType) string {
|
||||||
switch imageType {
|
switch imageType {
|
||||||
@@ -467,9 +497,61 @@ func (p *ImageProcessor) detectFormat(imageType vips.ImageType) string {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// resize resizes the image according to the fit mode. With FitOutside one
|
// outputDimensions returns the dimensions resize gives an image of imgWidth x
|
||||||
// side can come out larger than requested; when it would be above
|
// imgHeight for the target dimensions and fit mode, without resizing it, and
|
||||||
// MaxOutputDimension, resize returns ErrOutputTooLarge without resizing.
|
// refuses an output too large for format. With FitOutside one side can come
|
||||||
|
// out larger than the target; when it would be above MaxOutputDimension,
|
||||||
|
// outputDimensions returns ErrOutputTooLarge. An AVIF output of more than
|
||||||
|
// MaxAVIFOutputPixels is ErrAVIFOutputTooLarge.
|
||||||
|
func outputDimensions(
|
||||||
|
imgWidth, imgHeight, width, height int, fit FitMode, format Format,
|
||||||
|
) (int, int, error) {
|
||||||
|
scaleW := float64(width) / float64(imgWidth)
|
||||||
|
scaleH := float64(height) / float64(imgHeight)
|
||||||
|
outputWidth, outputHeight := width, height
|
||||||
|
|
||||||
|
switch fit {
|
||||||
|
case FitCover, FitFill, "":
|
||||||
|
// Cover crops, and fill stretches, to exactly the target dimensions
|
||||||
|
|
||||||
|
case FitContain:
|
||||||
|
// Fit within the target dimensions, maintaining aspect ratio
|
||||||
|
scale := min(scaleW, scaleH)
|
||||||
|
outputWidth = int(float64(imgWidth) * scale)
|
||||||
|
outputHeight = int(float64(imgHeight) * scale)
|
||||||
|
|
||||||
|
case FitInside:
|
||||||
|
// Same as contain, but only shrink
|
||||||
|
scale := min(scaleW, scaleH, 1)
|
||||||
|
outputWidth = int(float64(imgWidth) * scale)
|
||||||
|
outputHeight = int(float64(imgHeight) * scale)
|
||||||
|
|
||||||
|
case FitOutside:
|
||||||
|
// The smallest dimension fits, the other may exceed the target
|
||||||
|
scale := max(scaleW, scaleH)
|
||||||
|
outputWidth = int(float64(imgWidth) * scale)
|
||||||
|
outputHeight = int(float64(imgHeight) * scale)
|
||||||
|
|
||||||
|
default:
|
||||||
|
return 0, 0, fmt.Errorf("%w: %s", ErrInvalidFitMode, fit)
|
||||||
|
}
|
||||||
|
|
||||||
|
// The side FitOutside scales past the target is held to the maximum too
|
||||||
|
if max(outputWidth, outputHeight) > MaxOutputDimension {
|
||||||
|
return 0, 0, fmt.Errorf("%w: %dx%d is above %d", ErrOutputTooLarge,
|
||||||
|
outputWidth, outputHeight, MaxOutputDimension)
|
||||||
|
}
|
||||||
|
|
||||||
|
if format == FormatAVIF && outputWidth*outputHeight > MaxAVIFOutputPixels {
|
||||||
|
return 0, 0, fmt.Errorf("%w: %dx%d is above %d pixels",
|
||||||
|
ErrAVIFOutputTooLarge, outputWidth, outputHeight, MaxAVIFOutputPixels)
|
||||||
|
}
|
||||||
|
|
||||||
|
return outputWidth, outputHeight, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// resize resizes the image to width x height, the dimensions
|
||||||
|
// outputDimensions gives for the fit mode.
|
||||||
func (p *ImageProcessor) resize(
|
func (p *ImageProcessor) resize(
|
||||||
img *vips.ImageRef, width, height int, fit FitMode,
|
img *vips.ImageRef, width, height int, fit FitMode,
|
||||||
) error {
|
) error {
|
||||||
@@ -478,52 +560,20 @@ func (p *ImageProcessor) resize(
|
|||||||
// Resize and crop to fill exact dimensions (default)
|
// Resize and crop to fill exact dimensions (default)
|
||||||
return img.Thumbnail(width, height, vips.InterestingCentre)
|
return img.Thumbnail(width, height, vips.InterestingCentre)
|
||||||
|
|
||||||
case FitContain:
|
|
||||||
// Resize to fit within dimensions, maintaining aspect ratio
|
|
||||||
imgW, imgH := img.Width(), img.Height()
|
|
||||||
scaleW := float64(width) / float64(imgW)
|
|
||||||
scaleH := float64(height) / float64(imgH)
|
|
||||||
scale := min(scaleW, scaleH)
|
|
||||||
newW := int(float64(imgW) * scale)
|
|
||||||
newH := int(float64(imgH) * scale)
|
|
||||||
|
|
||||||
return img.Thumbnail(newW, newH, vips.InterestingNone)
|
|
||||||
|
|
||||||
case FitFill:
|
case FitFill:
|
||||||
// Resize to exact dimensions (may distort)
|
// Resize to exact dimensions (may distort)
|
||||||
return img.ThumbnailWithSize(width, height, vips.InterestingNone, vips.SizeForce)
|
return img.ThumbnailWithSize(width, height, vips.InterestingNone, vips.SizeForce)
|
||||||
|
|
||||||
case FitInside:
|
case FitInside:
|
||||||
// Same as contain, but only shrink
|
// outputDimensions keeps the dimensions of an image that already fits
|
||||||
if img.Width() <= width && img.Height() <= height {
|
if img.Width() == width && img.Height() == height {
|
||||||
return nil // Already fits
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
imgW, imgH := img.Width(), img.Height()
|
return img.Thumbnail(width, height, vips.InterestingNone)
|
||||||
scaleW := float64(width) / float64(imgW)
|
|
||||||
scaleH := float64(height) / float64(imgH)
|
|
||||||
scale := min(scaleW, scaleH)
|
|
||||||
newW := int(float64(imgW) * scale)
|
|
||||||
newH := int(float64(imgH) * scale)
|
|
||||||
|
|
||||||
return img.Thumbnail(newW, newH, vips.InterestingNone)
|
case FitContain, FitOutside:
|
||||||
|
return img.Thumbnail(width, height, vips.InterestingNone)
|
||||||
case FitOutside:
|
|
||||||
// Resize so smallest dimension fits, may exceed target on other dimension
|
|
||||||
imgW, imgH := img.Width(), img.Height()
|
|
||||||
scaleW := float64(width) / float64(imgW)
|
|
||||||
scaleH := float64(height) / float64(imgH)
|
|
||||||
scale := max(scaleW, scaleH)
|
|
||||||
newW := int(float64(imgW) * scale)
|
|
||||||
newH := int(float64(imgH) * scale)
|
|
||||||
|
|
||||||
// The dimension that exceeds the target is held to the maximum too
|
|
||||||
if newW > MaxOutputDimension || newH > MaxOutputDimension {
|
|
||||||
return fmt.Errorf("%w: %dx%d is above %d", ErrOutputTooLarge,
|
|
||||||
newW, newH, MaxOutputDimension)
|
|
||||||
}
|
|
||||||
|
|
||||||
return img.Thumbnail(newW, newH, vips.InterestingNone)
|
|
||||||
|
|
||||||
default:
|
default:
|
||||||
return fmt.Errorf("%w: %s", ErrInvalidFitMode, fit)
|
return fmt.Errorf("%w: %s", ErrInvalidFitMode, fit)
|
||||||
@@ -627,11 +677,16 @@ func exportGIF(img *vips.ImageRef) ([]byte, error) {
|
|||||||
return output, err
|
return output, err
|
||||||
}
|
}
|
||||||
|
|
||||||
// webpEffort is libvips' default WebP effort, from 0 (fastest) to 6.
|
// webpEffort is the WebP effort, from 0 (fastest) to 6. It is below libvips'
|
||||||
const webpEffort = 4
|
// default of 4 so that an encode takes at most 40 seconds of the default
|
||||||
|
// downstream_timeout of 60 seconds. With one thread, as pixad runs libvips,
|
||||||
|
// saving an 8192x8192 image of random pixels with an alpha channel takes up
|
||||||
|
// to about 23 seconds at 2, and up to about 16 without one, at any quality;
|
||||||
|
// with an alpha channel, 3 takes about 50 seconds and 4 about 61.
|
||||||
|
const webpEffort = 2
|
||||||
|
|
||||||
// exportWebP encodes img as lossy WebP at quality and libvips' default
|
// exportWebP encodes img as lossy WebP at quality and webpEffort, without
|
||||||
// effort, without metadata.
|
// metadata.
|
||||||
func exportWebP(img *vips.ImageRef, quality int) ([]byte, error) {
|
func exportWebP(img *vips.ImageRef, quality int) ([]byte, error) {
|
||||||
output, _, err := img.ExportWebp(&vips.WebpExportParams{
|
output, _, err := img.ExportWebp(&vips.WebpExportParams{
|
||||||
StripMetadata: true,
|
StripMetadata: true,
|
||||||
@@ -643,12 +698,13 @@ func exportWebP(img *vips.ImageRef, quality int) ([]byte, error) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// avifEffort is the AVIF effort, from 0 (fastest) to 9; 1 is the lowest
|
// avifEffort is the AVIF effort, from 0 (fastest) to 9; 1 is the lowest
|
||||||
// govips can set. With one thread, as pixad runs libvips, 1 takes about 51
|
// govips can set. With one thread, as pixad runs libvips, 1 takes about 52
|
||||||
// seconds to save an 8192x8192 RGB image of random pixels, and about 110
|
// seconds to save an 8192x8192 RGB image of random pixels at quality 85,
|
||||||
// with an alpha channel, the worst case, against the default
|
// and about 96 with an alpha channel, against the default
|
||||||
// downstream_timeout of 60 seconds. On an image of milder noise, which 1
|
// downstream_timeout of 60 seconds, so an AVIF output is held to
|
||||||
// saves in about 12 seconds, 2 takes nearly a minute and libvips' default,
|
// MaxAVIFOutputPixels. On an image of milder noise, which 1 saves in about
|
||||||
// 4, takes minutes.
|
// 12 seconds, 2 takes nearly a minute and libvips' default, 4, takes
|
||||||
|
// minutes.
|
||||||
const avifEffort = 1
|
const avifEffort = 1
|
||||||
|
|
||||||
// avifBitdepth is the AVIF bit depth, 8 bits per sample for every image.
|
// avifBitdepth is the AVIF bit depth, 8 bits per sample for every image.
|
||||||
@@ -675,10 +731,18 @@ func exportAVIF(img *vips.ImageRef, quality int) ([]byte, error) {
|
|||||||
// JPEG that names none.
|
// JPEG that names none.
|
||||||
const jxlResolution = 72 / 25.4
|
const jxlResolution = 72 / 25.4
|
||||||
|
|
||||||
// exportJXL encodes img as JPEG XL at quality, with libvips' default effort
|
// jxlEffort is the JPEG XL effort, from 1 (fastest) to 9. It is below
|
||||||
// and without metadata. govips sends libvips a distance, the JPEG XL
|
// libvips' default of 7 so that an encode takes at most 40 seconds of the
|
||||||
// encoder's own measure of quality, along with the quality, and libvips then
|
// default downstream_timeout of 60 seconds. With one thread, as pixad runs
|
||||||
// uses the distance alone, so the quality is also given as a distance.
|
// libvips, saving an 8192x8192 image of random pixels with an alpha channel
|
||||||
|
// takes up to about 27 seconds at 4, and up to about 13 without one, at any
|
||||||
|
// quality; with an alpha channel, 5 takes about 43 seconds and 7 about 81.
|
||||||
|
const jxlEffort = 4
|
||||||
|
|
||||||
|
// exportJXL encodes img as JPEG XL at quality and jxlEffort, without
|
||||||
|
// metadata. govips sends libvips a distance, the JPEG XL encoder's own
|
||||||
|
// measure of quality, along with the quality, and libvips then uses the
|
||||||
|
// distance alone, so the quality is also given as a distance.
|
||||||
func exportJXL(img *vips.ImageRef, quality int) ([]byte, error) {
|
func exportJXL(img *vips.ImageRef, quality int) ([]byte, error) {
|
||||||
// libvips converts a CMYK image to sRGB before it saves WebP, AVIF or
|
// libvips converts a CMYK image to sRGB before it saves WebP, AVIF or
|
||||||
// PNG, but cannot save one as JPEG XL. encode has already converted
|
// PNG, but cannot save one as JPEG XL. encode has already converted
|
||||||
@@ -717,6 +781,7 @@ func exportJXL(img *vips.ImageRef, quality int) ([]byte, error) {
|
|||||||
params := vips.NewJxlExportParams()
|
params := vips.NewJxlExportParams()
|
||||||
params.Quality = quality
|
params.Quality = quality
|
||||||
params.Distance = jxlDistance(quality)
|
params.Distance = jxlDistance(quality)
|
||||||
|
params.Effort = jxlEffort
|
||||||
|
|
||||||
output, _, err := toSave.ExportJxl(params)
|
output, _, err := toSave.ExportJxl(params)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
|
|||||||
Reference in New Issue
Block a user