The fetcher accepts image/svg+xml again and internal/magic detects SVG. A raster format renders the SVG at the least size that covers the output, held to 8192 on a side, through libvips' thumbnail, which renders at any scale, so an enlarged SVG stays sharp. An SVG's own size is not limited; one with a side more than 8192 times the other is ErrSVGTooThin, answered 415. orig of an SVG at the orig size is the SVG unchanged, cached as a variant; at any other size it is ErrSVGNotOriginalSize, answered 400. The cache reads up to 1024 first bytes of a file to find its format. Render time and memory grow with what the SVG draws; their bound is the owner's decision on #68. Tests that pinned SVG as refused now expect it to be read. Model: opus-5-5
This commit is contained in:
1 parent
1379497897
commit
98b79c2e27
19 files changed
+796
-82
No files matched your search
@@ -181,7 +181,8 @@ run under. Its app for pixa needs:
|
||||
is half the CPUs pixa sees times 4.2 GiB, plus 64 times 50 MiB. On a host
|
||||
where pixa sees 48 CPUs, that is 24 times 4.2 GiB plus 3.1 GiB, about 104 GiB.
|
||||
Set `PIXA_MAX_CONCURRENT_PROCESSING` and `PIXA_UPSTREAM_CONNECTIONS` so that
|
||||
this fits the memory the container may use.
|
||||
this fits the memory the container may use. Rendering an SVG source can need
|
||||
more, depending on what it draws.
|
||||
- **Temporary files:** while pixa processes a large source, libvips writes the
|
||||
decoded pixels to a file in `/tmp`. The file is removed from the directory
|
||||
once it is written, so a listing of `/tmp` does not show it. It keeps its
|
||||
@@ -281,6 +282,7 @@ to any method the list does not give for it.
|
||||
- 304 when `If-None-Match` matches the image's `ETag`.
|
||||
- 400 for a URL or parameter that is not valid.
|
||||
- 400 when the format is `auto` and the `Accept` header is not valid.
|
||||
- 400 for `orig` of an SVG source at a size other than `orig`.
|
||||
- 406 when the format is `auto` and `Accept` allows none of the formats it
|
||||
chooses from.
|
||||
- 401 for a missing or wrong signature.
|
||||
@@ -301,6 +303,8 @@ to any method the list does not give for it.
|
||||
format its `Content-Type` names.
|
||||
- 415 when the upstream sends an image whose first bytes libvips reads as
|
||||
another format.
|
||||
- 415 for an SVG source with one side more than 8192 times the other, in any
|
||||
format but `orig`.
|
||||
- 502 when the upstream answers with an error status.
|
||||
- 502 for 5 minutes after that for the same source URL, when pixa has
|
||||
neither the image nor its source in its cache.
|
||||
@@ -318,6 +322,8 @@ to any method the list does not give for it.
|
||||
- 304 when `If-None-Match` matches the image's `ETag`.
|
||||
- 400 for a token that does not decrypt.
|
||||
- 400 for a token that asks for a size or fit that is not valid.
|
||||
- 400 for a token that asks for `orig` of an SVG source at a size other than
|
||||
`orig`.
|
||||
- 410 once the token has expired.
|
||||
- 400 when the token's format is `auto` and the `Accept` header is not
|
||||
valid.
|
||||
@@ -479,9 +485,9 @@ applies to the format `orig` when the source is AVIF, and to `auto` when it
|
||||
chooses AVIF. So a client whose `Accept` names `image/avif` but not `image/jxl`
|
||||
is refused such a size.
|
||||
|
||||
**Source formats.** The source may be JPEG, PNG, GIF, WebP, AVIF or JPEG XL. The
|
||||
upstream's `Content-Type` must name its format, and the image's first bytes must
|
||||
match it. An upstream answering with any other `Content-Type`, SVG included, is
|
||||
**Source formats.** The source may be JPEG, PNG, GIF, WebP, AVIF, JPEG XL or
|
||||
SVG. The upstream's `Content-Type` must name its format, and the image's first
|
||||
bytes must match it. An upstream answering with any other `Content-Type` is
|
||||
refused with 415. So is a body whose first bytes are not those of the format its
|
||||
`Content-Type` names: a JPEG sent as `image/png`, text sent as an image, or an
|
||||
empty body. pixa then does not fetch that source URL again for 5 minutes, as
|
||||
@@ -490,6 +496,20 @@ need that fetch. An image whose first bytes libvips reads as another format is
|
||||
refused with 415 too. An SVG whose first bytes pass the check for AVIF is one
|
||||
example.
|
||||
|
||||
**SVG sources.** For any format but `orig`, pixa renders an SVG source at the
|
||||
requested size, so an enlarged SVG stays sharp. At the `orig` size it is
|
||||
rendered at its own size. The `orig` size of an SVG above 8192 on a side is
|
||||
refused with 400. An SVG with one side more than 8192 times the other is refused
|
||||
with 415 in those formats: rendered with no side above 8192 pixels, its shorter
|
||||
side would be under 1 pixel. Rendering loads no file or URL that the SVG refers
|
||||
to.
|
||||
|
||||
`orig` of an SVG source is the SVG itself, unchanged, and is cached like any
|
||||
other image. It is given only at the `orig` size, since pixa does not resize an
|
||||
SVG as SVG: `orig` of an SVG at any other size is refused with 400. A browser
|
||||
that opens an SVG from pixa runs none of its scripts, as pixa's
|
||||
`Content-Security-Policy` allows no inline script.
|
||||
|
||||
**The format `auto`.** With `auto`, pixa chooses the format for each request
|
||||
from its `Accept` header, in this order:
|
||||
|
||||
@@ -698,8 +718,9 @@ written percent-encoded.
|
||||
|
||||
### Image Metadata
|
||||
|
||||
pixa decodes and re-encodes every image it serves, and removes all metadata from
|
||||
the output. This cannot be turned off. The metadata removed is:
|
||||
pixa decodes and re-encodes every image it serves, except `orig` of an SVG
|
||||
source, which is the SVG unchanged. It removes all metadata from every image it
|
||||
re-encodes. This cannot be turned off. The metadata removed is:
|
||||
|
||||
- EXIF, which can hold the GPS position, the camera make, model and serial
|
||||
number, the capture time and an embedded thumbnail;
|
||||
@@ -708,14 +729,16 @@ the output. This cannot be turned off. The metadata removed is:
|
||||
|
||||
The rest of the rules:
|
||||
|
||||
- The `orig` format means the source's own format, not the source's bytes. An
|
||||
`orig` image is re-encoded and stripped like any other.
|
||||
- For a source in any format but SVG, the `orig` format means the source's own
|
||||
format, not the source's bytes. An `orig` image is re-encoded and stripped
|
||||
like any other. For an SVG source, `orig` is the SVG's own bytes, its metadata
|
||||
included.
|
||||
- An image with an EXIF orientation is turned upright first, so it displays the
|
||||
same without the tag. A requested size applies to the upright image.
|
||||
- An image with an ICC profile is first converted to sRGB, the standard colour
|
||||
space, since clients show an image with no profile as sRGB. Colours outside
|
||||
sRGB are clipped: the most saturated ones in a Display P3 photo, for example.
|
||||
- JPEG XL output is the one exception, with libvips 8.16 and later. libvips then
|
||||
- JPEG XL output is an exception, with libvips 8.16 and later. libvips then
|
||||
writes an EXIF block of its own, which pixa cannot stop. The block holds the
|
||||
image's size and fixed values: an orientation of 1 and a resolution of 72 dpi.
|
||||
Nothing in it comes from the source. The Docker image has libvips 8.16, so its
|
||||
@@ -767,7 +790,9 @@ about 34 seconds with an alpha channel, and 22 without. At 8192x8192 and quality
|
||||
85 it would take about 96 seconds with an alpha channel, and 52 without. The 34
|
||||
seconds hold at any quality and at 8 or 16 bits per sample. No encode within the
|
||||
limits takes more than about 40 seconds. The default `downstream_timeout` is 60
|
||||
seconds.
|
||||
seconds. These limits do not bound rendering an SVG source, which happens within
|
||||
the encode: its time and memory grow with what the SVG draws, such as filter
|
||||
effects.
|
||||
|
||||
**Processing and connections.** `max_concurrent_processing` images are decoded
|
||||
and encoded at once. A request that finds every slot in use waits up to 10
|
||||
|
||||
@@ -267,11 +267,19 @@ func (s *Handlers) respondImageError(
|
||||
errors.Is(err, imageprocessor.ErrUnsupportedInputFormat):
|
||||
status = http.StatusUnsupportedMediaType
|
||||
message = "unsupported source format"
|
||||
case errors.Is(err, imageprocessor.ErrSVGTooThin):
|
||||
status = http.StatusUnsupportedMediaType
|
||||
message = fmt.Sprintf("SVG source too thin to render: one side is "+
|
||||
"more than %d times the other", imageprocessor.MaxInputDimension)
|
||||
case errors.Is(err, imageprocessor.ErrOutputTooLarge):
|
||||
status = http.StatusBadRequest
|
||||
message = fmt.Sprintf("size too large for the source: the side "+
|
||||
"computed from its proportions is above %d",
|
||||
imageprocessor.MaxOutputDimension)
|
||||
case errors.Is(err, imageprocessor.ErrSVGNotOriginalSize):
|
||||
status = http.StatusBadRequest
|
||||
message = "an SVG source is returned as SVG only at the orig size; " +
|
||||
"ask for another format to resize it"
|
||||
case errors.Is(err, imageprocessor.ErrAVIFOutputTooLarge):
|
||||
status = http.StatusBadRequest
|
||||
message = fmt.Sprintf("size too large for AVIF: the output "+
|
||||
|
||||
@@ -467,7 +467,7 @@ func TestImageRoutes_SameErrorAnswers(t *testing.T) {
|
||||
{"no free upstream connection", httpfetcher.ErrTooManyConnections,
|
||||
http.StatusServiceUnavailable, slog.LevelWarn},
|
||||
{"Content-Type not a source format",
|
||||
fmt.Errorf("%w: image/svg+xml", httpfetcher.ErrInvalidContentType),
|
||||
fmt.Errorf("%w: image/tiff", httpfetcher.ErrInvalidContentType),
|
||||
http.StatusUnsupportedMediaType, slog.LevelWarn},
|
||||
{"any other failure", io.ErrUnexpectedEOF,
|
||||
http.StatusInternalServerError, slog.LevelError},
|
||||
|
||||
@@ -15,7 +15,7 @@ import (
|
||||
|
||||
// TestUnsupportedSourceFormatAnswers415 checks that respondImageError, which
|
||||
// answers the errors of both image routes, answers 415 with a clear error when
|
||||
// the upstream answers with a content type pixa does not accept, such as SVG,
|
||||
// the upstream answers with a content type pixa does not accept, such as TIFF,
|
||||
// and when libvips does not read a source as the format its first bytes
|
||||
// show, wrapped as the service wraps them.
|
||||
func TestUnsupportedSourceFormatAnswers415(t *testing.T) {
|
||||
@@ -23,11 +23,11 @@ func TestUnsupportedSourceFormatAnswers415(t *testing.T) {
|
||||
|
||||
h := &Handlers{log: slog.New(slog.DiscardHandler)}
|
||||
req := &imagerequest.Request{
|
||||
Source: imagerequest.Source{Host: "img.example.com", Path: "/a.svg"},
|
||||
Source: imagerequest.Source{Host: "img.example.com", Path: "/a.tiff"},
|
||||
}
|
||||
|
||||
for _, err := range []error{
|
||||
fmt.Errorf("upstream fetch failed: %w: image/svg+xml",
|
||||
fmt.Errorf("upstream fetch failed: %w: image/tiff",
|
||||
httpfetcher.ErrInvalidContentType),
|
||||
fmt.Errorf("image processing failed: %w: libvips does not read the input as avif",
|
||||
imageprocessor.ErrUnsupportedInputFormat),
|
||||
@@ -47,3 +47,30 @@ func TestUnsupportedSourceFormatAnswers415(t *testing.T) {
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestSVGTooThinAnswers415 checks that respondImageError answers 415 with a
|
||||
// clear error for an SVG source too thin to render, wrapped as the service
|
||||
// wraps it.
|
||||
func TestSVGTooThinAnswers415(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
h := &Handlers{log: slog.New(slog.DiscardHandler)}
|
||||
req := &imagerequest.Request{
|
||||
Source: imagerequest.Source{Host: "svg.example.com", Path: "/a.svg"},
|
||||
}
|
||||
err := fmt.Errorf("image processing failed: %w: 81930x10",
|
||||
imageprocessor.ErrSVGTooThin)
|
||||
|
||||
rec := httptest.NewRecorder()
|
||||
h.respondImageError(rec, req, err)
|
||||
t.Logf("for %v: %d %s", err, rec.Code, rec.Body.String())
|
||||
|
||||
if rec.Code != http.StatusUnsupportedMediaType {
|
||||
t.Errorf("status = %d, want %d", rec.Code,
|
||||
http.StatusUnsupportedMediaType)
|
||||
}
|
||||
|
||||
if !strings.Contains(rec.Body.String(), "SVG source too thin to render") {
|
||||
t.Errorf("body = %q, want the SVG too thin error", rec.Body.String())
|
||||
}
|
||||
}
|
||||
@@ -119,6 +119,7 @@ func allowedContentTypes() []string {
|
||||
imagerequest.FormatWebP.MIMEType(),
|
||||
imagerequest.FormatAVIF.MIMEType(),
|
||||
imagerequest.FormatJXL.MIMEType(),
|
||||
imagerequest.FormatSVG.MIMEType(),
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -279,7 +279,7 @@ func TestDetectContentTypeFromPath(t *testing.T) {
|
||||
{"foo/bar.gif", imagerequest.FormatGIF.MIMEType()},
|
||||
{"foo/bar.webp", imagerequest.FormatWebP.MIMEType()},
|
||||
{"foo/bar.avif", imagerequest.FormatAVIF.MIMEType()},
|
||||
{"foo/bar.svg", contentTypeSVG},
|
||||
{"foo/bar.svg", imagerequest.FormatSVG.MIMEType()},
|
||||
{"foo/bar.bin", contentTypeOctetStream},
|
||||
{"foo/bar", contentTypeOctetStream},
|
||||
}
|
||||
|
||||
@@ -66,11 +66,9 @@ func (m *MockFetcher) Fetch(
|
||||
}, nil
|
||||
}
|
||||
|
||||
// The content types the mock fetcher gives files in no format pixa reads.
|
||||
const (
|
||||
contentTypeSVG = "image/svg+xml"
|
||||
contentTypeOctetStream = "application/octet-stream"
|
||||
)
|
||||
// contentTypeOctetStream is the content type the mock fetcher gives files in
|
||||
// no format pixa reads.
|
||||
const contentTypeOctetStream = "application/octet-stream"
|
||||
|
||||
// detectContentTypeFromPath returns the MIME type based on file extension.
|
||||
func detectContentTypeFromPath(path string) string {
|
||||
@@ -90,7 +88,7 @@ func detectContentTypeFromPath(path string) string {
|
||||
case strings.HasSuffix(path, ".jxl"):
|
||||
return imagerequest.FormatJXL.MIMEType()
|
||||
case strings.HasSuffix(path, ".svg"):
|
||||
return contentTypeSVG
|
||||
return imagerequest.FormatSVG.MIMEType()
|
||||
default:
|
||||
return contentTypeOctetStream
|
||||
}
|
||||
|
||||
@@ -7,6 +7,7 @@ import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"math"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
@@ -113,10 +114,21 @@ var ErrUnsupportedOutputFormat = errors.New("unsupported output format")
|
||||
// ErrUnsupportedInputFormat is returned, before the input is decoded, when
|
||||
// libvips reads the input's first bytes as another format than its source
|
||||
// format, or as none. An SVG whose first bytes pass the first-bytes check for
|
||||
// AVIF is one example. A source format other than the six pixa reads is also
|
||||
// refused with it.
|
||||
// AVIF is one example. A source format other than the seven pixa reads is
|
||||
// also refused with it.
|
||||
var ErrUnsupportedInputFormat = errors.New("unsupported input format")
|
||||
|
||||
// ErrSVGNotOriginalSize is returned when orig is asked of an SVG source at a
|
||||
// size other than orig. pixa returns an SVG only as it is; a raster format
|
||||
// gives it a size.
|
||||
var ErrSVGNotOriginalSize = errors.New(
|
||||
"an SVG is returned as SVG only at the orig size")
|
||||
|
||||
// ErrSVGTooThin is returned, before rendering, when an SVG source has one
|
||||
// side more than MaxInputDimension times the other. Rendered with its longer
|
||||
// side at most MaxInputDimension, its shorter side would be under 1 pixel.
|
||||
var ErrSVGTooThin = errors.New("SVG source too thin to render")
|
||||
|
||||
// ErrTooManyImages is returned when MaxConcurrentProcessing images are being
|
||||
// processed and none finishes within ProcessingWaitTimeout.
|
||||
var ErrTooManyImages = errors.New("too many images being processed at once")
|
||||
@@ -164,7 +176,9 @@ func New(params Params) *ImageProcessor {
|
||||
// encodes, it returns ctx's error and gives the slot back. govips gives pixa
|
||||
// no way to stop libvips once an encode has begun, so an encode keeps its
|
||||
// slot until it finishes. An AVIF output of more than MaxAVIFOutputPixels is
|
||||
// refused with ErrAVIFOutputTooLarge before anything is decoded.
|
||||
// refused with ErrAVIFOutputTooLarge before anything is decoded. orig of an
|
||||
// SVG source is the SVG as it is, since libvips cannot save SVG; any other
|
||||
// format renders it at the output size.
|
||||
func (p *ImageProcessor) Process(
|
||||
ctx context.Context,
|
||||
input io.Reader,
|
||||
@@ -190,27 +204,17 @@ func (p *ImageProcessor) Process(
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// Decode image
|
||||
img, err := vips.NewImageFromBuffer(data)
|
||||
img, err := loadImage(data, sourceFormat)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to decode image: %w", err)
|
||||
return nil, err
|
||||
}
|
||||
defer img.Close()
|
||||
|
||||
// Turn the image upright now: encode strips the EXIF orientation tag,
|
||||
// and sizes below must be worked out on the upright image.
|
||||
err = img.AutoRotate()
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to auto-rotate: %w", err)
|
||||
}
|
||||
origWidth, origHeight := img.Width(), img.Height()
|
||||
outputFormat := resolveOutputFormat(transform.Format, sourceFormat)
|
||||
|
||||
// Get original dimensions
|
||||
origWidth := img.Width()
|
||||
origHeight := img.Height()
|
||||
|
||||
// Validate input dimensions to prevent DoS via decompression bombs
|
||||
if origWidth > MaxInputDimension || origHeight > MaxInputDimension {
|
||||
return nil, ErrInputTooLarge
|
||||
if outputFormat == imagerequest.FormatSVG && sourceFormat == imagerequest.FormatSVG {
|
||||
return svgAsItIs(data, transform.Size, origWidth, origHeight)
|
||||
}
|
||||
|
||||
// Determine target dimensions; too large ones are refused before resizing
|
||||
@@ -220,12 +224,6 @@ func (p *ImageProcessor) Process(
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// orig is the source's own format; encode refuses an empty format
|
||||
outputFormat := transform.Format
|
||||
if outputFormat == imagerequest.FormatOriginal {
|
||||
outputFormat = sourceFormat
|
||||
}
|
||||
|
||||
// Determine output dimensions; too large ones are refused before
|
||||
// resizing, which with FitCover already decodes the input
|
||||
outputWidth, outputHeight, err := outputDimensions(origWidth, origHeight,
|
||||
@@ -234,6 +232,19 @@ func (p *ImageProcessor) Process(
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// An SVG is rendered again at about the output size, to stay sharp
|
||||
if sourceFormat == imagerequest.FormatSVG {
|
||||
rendered, err := renderSVG(data, origWidth, origHeight,
|
||||
outputWidth, outputHeight)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
defer rendered.Close()
|
||||
|
||||
img = rendered
|
||||
}
|
||||
|
||||
// Resize if needed
|
||||
if targetWidth != origWidth || targetHeight != origHeight {
|
||||
err := p.resize(img, outputWidth, outputHeight, transform.FitMode)
|
||||
@@ -290,6 +301,110 @@ func targetDimensions(
|
||||
return width, height, nil
|
||||
}
|
||||
|
||||
// resolveOutputFormat returns the format of the output for the requested
|
||||
// format: sourceFormat for orig, and format itself for any other. encode
|
||||
// refuses an empty format.
|
||||
func resolveOutputFormat(
|
||||
format, sourceFormat imagerequest.Format,
|
||||
) imagerequest.Format {
|
||||
if format == imagerequest.FormatOriginal {
|
||||
return sourceFormat
|
||||
}
|
||||
|
||||
return format
|
||||
}
|
||||
|
||||
// loadImage decodes data and turns the image upright: encode strips the EXIF
|
||||
// orientation tag, and sizes must be worked out on the upright image. An
|
||||
// image with a side above MaxInputDimension is ErrInputTooLarge, which guards
|
||||
// against decompression bombs. An SVG is not checked: only its header has
|
||||
// been read, and the output checks limit the size it is rendered at.
|
||||
func loadImage(
|
||||
data []byte, sourceFormat imagerequest.Format,
|
||||
) (*vips.ImageRef, error) {
|
||||
img, err := vips.NewImageFromBuffer(data)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to decode image: %w", err)
|
||||
}
|
||||
|
||||
err = img.AutoRotate()
|
||||
if err != nil {
|
||||
img.Close()
|
||||
|
||||
return nil, fmt.Errorf("failed to auto-rotate: %w", err)
|
||||
}
|
||||
|
||||
if sourceFormat != imagerequest.FormatSVG &&
|
||||
(img.Width() > MaxInputDimension || img.Height() > MaxInputDimension) {
|
||||
img.Close()
|
||||
|
||||
return nil, ErrInputTooLarge
|
||||
}
|
||||
|
||||
return img, nil
|
||||
}
|
||||
|
||||
// svgAsItIs returns the SVG data unchanged, which is orig of an SVG source
|
||||
// at the orig size. width x height is the SVG's own size. Any other size is
|
||||
// ErrSVGNotOriginalSize.
|
||||
func svgAsItIs(
|
||||
data []byte, size imagerequest.Size, width, height int,
|
||||
) (*Result, error) {
|
||||
if !size.OriginalSize() {
|
||||
return nil, fmt.Errorf("%w: asked for %dx%d", ErrSVGNotOriginalSize,
|
||||
size.Width, size.Height)
|
||||
}
|
||||
|
||||
return &Result{
|
||||
Content: io.NopCloser(bytes.NewReader(data)),
|
||||
ContentLength: int64(len(data)),
|
||||
ContentType: imagerequest.FormatSVG.MIMEType(),
|
||||
Width: width,
|
||||
Height: height,
|
||||
InputWidth: width,
|
||||
InputHeight: height,
|
||||
}, nil
|
||||
}
|
||||
|
||||
// svgRenderSize returns the least size in the proportions of width x height,
|
||||
// an SVG's own size, that covers outputWidth x outputHeight, so that resize
|
||||
// hardly changes it. Neither side is above MaxInputDimension.
|
||||
func svgRenderSize(width, height, outputWidth, outputHeight int) (int, int) {
|
||||
scale := min(
|
||||
max(float64(outputWidth)/float64(width),
|
||||
float64(outputHeight)/float64(height)),
|
||||
float64(MaxInputDimension)/float64(max(width, height)))
|
||||
|
||||
return int(math.Round(float64(width) * scale)),
|
||||
int(math.Round(float64(height) * scale))
|
||||
}
|
||||
|
||||
// renderSVG renders the SVG data, whose own size is width x height, at
|
||||
// svgRenderSize for outputWidth x outputHeight. An SVG with one side more
|
||||
// than MaxInputDimension times the other is ErrSVGTooThin. libvips'
|
||||
// thumbnail has librsvg render the SVG at that size directly, at any scale:
|
||||
// govips takes a density only in whole dots per inch, which cannot render
|
||||
// an SVG below 1/72 of its own size. Given the SVG as bytes with no file
|
||||
// name, librsvg loads no file or URL the SVG refers to.
|
||||
func renderSVG(
|
||||
data []byte, width, height, outputWidth, outputHeight int,
|
||||
) (*vips.ImageRef, error) {
|
||||
if max(width, height) > MaxInputDimension*min(width, height) {
|
||||
return nil, fmt.Errorf("%w: %dx%d", ErrSVGTooThin, width, height)
|
||||
}
|
||||
|
||||
renderWidth, renderHeight := svgRenderSize(width, height,
|
||||
outputWidth, outputHeight)
|
||||
|
||||
img, err := vips.NewThumbnailWithSizeFromBuffer(data,
|
||||
renderWidth, renderHeight, vips.InterestingNone, vips.SizeBoth)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to render SVG: %w", err)
|
||||
}
|
||||
|
||||
return img, nil
|
||||
}
|
||||
|
||||
// WaitForProcessing waits until no image is being processed, or until ctx
|
||||
// ends, and returns how many images were still being processed then. It
|
||||
// waits by taking each slot in processingSemaphore as it frees up until it
|
||||
@@ -397,6 +512,8 @@ func vipsImageType(format imagerequest.Format) vips.ImageType {
|
||||
return vips.ImageTypeAVIF
|
||||
case imagerequest.FormatJXL:
|
||||
return vips.ImageTypeJXL
|
||||
case imagerequest.FormatSVG:
|
||||
return vips.ImageTypeSVG
|
||||
case imagerequest.FormatOriginal, imagerequest.FormatAuto:
|
||||
return vips.ImageTypeUnknown
|
||||
default:
|
||||
@@ -575,7 +692,9 @@ func (p *ImageProcessor) encode(
|
||||
case imagerequest.FormatJXL:
|
||||
return exportJXL(img, quality)
|
||||
|
||||
case imagerequest.FormatOriginal, imagerequest.FormatAuto:
|
||||
// libvips cannot save SVG; Process returns an SVG source as it is
|
||||
case imagerequest.FormatSVG, imagerequest.FormatOriginal,
|
||||
imagerequest.FormatAuto:
|
||||
return nil, fmt.Errorf("%w: %s", ErrUnsupportedOutputFormat, format)
|
||||
|
||||
default:
|
||||
|
||||
@@ -51,7 +51,7 @@ func TestImageProcessor_LoadsEachSourceFormat(t *testing.T) {
|
||||
}
|
||||
|
||||
// TestImageProcessor_NotASourceFormatRefused verifies that a source format
|
||||
// other than the six, such as orig, auto or none, is refused with
|
||||
// other than the seven, such as orig, auto or none, is refused with
|
||||
// ErrUnsupportedInputFormat. The input is text, which libvips reads as no
|
||||
// format.
|
||||
func TestImageProcessor_NotASourceFormatRefused(t *testing.T) {
|
||||
|
||||
@@ -0,0 +1,293 @@
|
||||
package imageprocessor
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"errors"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"sync/atomic"
|
||||
"testing"
|
||||
|
||||
"github.com/davidbyttow/govips/v2/vips"
|
||||
"sneak.berlin/go/pixa/internal/imagerequest"
|
||||
)
|
||||
|
||||
// testSVG is a 16x12 SVG, black on its left half and white on its right.
|
||||
const testSVG = `<svg xmlns="http://www.w3.org/2000/svg" width="16" height="12">` +
|
||||
`<rect width="16" height="12" fill="white"/>` +
|
||||
`<rect width="8" height="12" fill="black"/></svg>`
|
||||
|
||||
// newSVGTestProcessor returns a processor for the SVG tests.
|
||||
func newSVGTestProcessor() *ImageProcessor {
|
||||
return New(Params{MaxInputBytes: 1 << 20, MaxConcurrentProcessing: 1})
|
||||
}
|
||||
|
||||
// processSVG processes svg, as an SVG source, with transform.
|
||||
func processSVG(
|
||||
t *testing.T, svg string, transform imagerequest.Transform,
|
||||
) (*Result, []byte, error) {
|
||||
t.Helper()
|
||||
|
||||
result, err := newSVGTestProcessor().Process(t.Context(),
|
||||
bytes.NewReader([]byte(svg)), imagerequest.FormatSVG, &transform)
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
|
||||
output, err := io.ReadAll(result.Content)
|
||||
if err != nil {
|
||||
t.Fatalf("reading the output: %v", err)
|
||||
}
|
||||
|
||||
return result, output, nil
|
||||
}
|
||||
|
||||
// TestImageProcessor_SVGOrigIsTheSVG checks that orig of an SVG at the orig
|
||||
// size is the SVG itself, never another format, and that orig at another
|
||||
// size is ErrSVGNotOriginalSize.
|
||||
func TestImageProcessor_SVGOrigIsTheSVG(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
result, output, err := processSVG(t, testSVG,
|
||||
imagerequest.Transform{Format: imagerequest.FormatOriginal})
|
||||
if err != nil {
|
||||
t.Fatalf("orig of the SVG: %v", err)
|
||||
}
|
||||
|
||||
t.Logf("orig of the SVG: %s, %dx%d", result.ContentType, result.Width,
|
||||
result.Height)
|
||||
|
||||
if result.ContentType != "image/svg+xml" || string(output) != testSVG {
|
||||
t.Errorf("orig of the SVG = %s %q, want image/svg+xml %q",
|
||||
result.ContentType, output, testSVG)
|
||||
}
|
||||
|
||||
_, _, err = processSVG(t, testSVG, imagerequest.Transform{
|
||||
Size: imagerequest.Size{Width: 32, Height: 24},
|
||||
Format: imagerequest.FormatOriginal,
|
||||
})
|
||||
if !errors.Is(err, ErrSVGNotOriginalSize) {
|
||||
t.Errorf("orig of the SVG at 32x24: error = %v, want %v", err,
|
||||
ErrSVGNotOriginalSize)
|
||||
}
|
||||
}
|
||||
|
||||
// TestImageProcessor_RendersSVGToEachRasterFormat checks that an SVG is
|
||||
// rendered to each raster format at the requested size.
|
||||
func TestImageProcessor_RendersSVGToEachRasterFormat(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
for _, format := range []imagerequest.Format{
|
||||
imagerequest.FormatJPEG, imagerequest.FormatPNG, imagerequest.FormatGIF,
|
||||
imagerequest.FormatWebP, imagerequest.FormatAVIF, imagerequest.FormatJXL,
|
||||
} {
|
||||
result, output, err := processSVG(t, testSVG, imagerequest.Transform{
|
||||
Size: imagerequest.Size{Width: 40, Height: 30},
|
||||
Format: format,
|
||||
})
|
||||
if err != nil {
|
||||
t.Errorf("SVG as %s: %v", format, err)
|
||||
|
||||
continue
|
||||
}
|
||||
|
||||
t.Logf("SVG as %s: %s, %dx%d, %d bytes", format, result.ContentType,
|
||||
result.Width, result.Height, len(output))
|
||||
|
||||
if result.ContentType != format.MIMEType() ||
|
||||
vips.DetermineImageType(output) != vipsImageType(format) ||
|
||||
result.Width != 40 || result.Height != 30 {
|
||||
t.Errorf("SVG as %s = %s, %dx%d; want %s, 40x30", format,
|
||||
result.ContentType, result.Width, result.Height, format.MIMEType())
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestImageProcessor_RendersSVGSharpWhenEnlarged checks that a 16x12 SVG
|
||||
// asked for at 1600x1200 is rendered at that size, not rendered at 16x12 and
|
||||
// enlarged: 10 pixels either side of the edge between its black and white
|
||||
// halves are black and white, where an enlarged image is grey there.
|
||||
func TestImageProcessor_RendersSVGSharpWhenEnlarged(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
_, output, err := processSVG(t, testSVG, imagerequest.Transform{
|
||||
Size: imagerequest.Size{Width: 1600, Height: 1200},
|
||||
Format: imagerequest.FormatPNG,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("SVG as PNG at 1600x1200: %v", err)
|
||||
}
|
||||
|
||||
img, err := vips.NewImageFromBuffer(output)
|
||||
if err != nil {
|
||||
t.Fatalf("decoding the PNG: %v", err)
|
||||
}
|
||||
|
||||
t.Cleanup(img.Close)
|
||||
|
||||
for x, want := range map[int]float64{790: 0, 810: 255} {
|
||||
point, err := img.GetPoint(x, 600)
|
||||
if err != nil {
|
||||
t.Fatalf("GetPoint(%d, 600): %v", x, err)
|
||||
}
|
||||
|
||||
t.Logf("pixel at %d,600: %v", x, point)
|
||||
|
||||
if point[0] != want {
|
||||
t.Errorf("pixel at %d,600 = %v, want %v", x, point[0], want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestImageProcessor_SVGFetchesNothing checks that rendering an SVG that
|
||||
// refers to an image and an external entity on a local server makes no
|
||||
// request to that server.
|
||||
func TestImageProcessor_SVGFetchesNothing(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
var requests atomic.Int64
|
||||
|
||||
png := createTestPNG(t, 4, 4)
|
||||
|
||||
server := httptest.NewServer(http.HandlerFunc(
|
||||
func(w http.ResponseWriter, r *http.Request) {
|
||||
requests.Add(1)
|
||||
t.Logf("request for %s", r.URL)
|
||||
w.Header().Set("Content-Type", "image/png")
|
||||
_, _ = w.Write(png)
|
||||
}))
|
||||
t.Cleanup(server.Close)
|
||||
|
||||
svg := `<?xml version="1.0"?>` +
|
||||
`<!DOCTYPE svg [<!ENTITY outside SYSTEM "` + server.URL + `/entity">]>` +
|
||||
`<svg xmlns="http://www.w3.org/2000/svg" ` +
|
||||
`xmlns:xlink="http://www.w3.org/1999/xlink" width="16" height="12">` +
|
||||
`<image href="` + server.URL + `/image.png" width="16" height="12"/>` +
|
||||
`<image xlink:href="` + server.URL + `/xlink.png" width="16" height="12"/>` +
|
||||
`<text y="10">&outside;</text></svg>`
|
||||
|
||||
_, _, err := processSVG(t, svg, imagerequest.Transform{
|
||||
Format: imagerequest.FormatPNG,
|
||||
})
|
||||
t.Logf("SVG as PNG: error = %v", err)
|
||||
|
||||
if requests.Load() != 0 {
|
||||
t.Errorf("rendering the SVG made %d requests, want none", requests.Load())
|
||||
}
|
||||
}
|
||||
|
||||
// TestImageProcessor_RendersSVGLargerThanMaxInputDimension checks that an
|
||||
// SVG whose own size is above MaxInputDimension is limited only by the
|
||||
// output checks: it is rendered at a raster size, orig of it is the SVG
|
||||
// itself, and a raster format at its own size is ErrOutputTooLarge. The
|
||||
// 1000000x1000000 one is rendered at a ten-thousandth of its own size.
|
||||
func TestImageProcessor_RendersSVGLargerThanMaxInputDimension(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
for _, svg := range []string{
|
||||
`<svg xmlns="http://www.w3.org/2000/svg" width="10000" height="10">` +
|
||||
`<rect width="10000" height="10"/></svg>`,
|
||||
`<svg xmlns="http://www.w3.org/2000/svg" width="1000000" ` +
|
||||
`height="1000000"><rect width="1000000" height="1000000"/></svg>`,
|
||||
} {
|
||||
result, _, err := processSVG(t, svg, imagerequest.Transform{
|
||||
Size: imagerequest.Size{Width: 100, Height: 100},
|
||||
Format: imagerequest.FormatPNG,
|
||||
})
|
||||
if err != nil {
|
||||
t.Errorf("%s as PNG at 100x100: %v", svg, err)
|
||||
} else {
|
||||
t.Logf("%s as PNG at 100x100: %dx%d", svg, result.Width, result.Height)
|
||||
|
||||
if result.Width != 100 || result.Height != 100 {
|
||||
t.Errorf("%s as PNG at 100x100 = %dx%d, want 100x100", svg,
|
||||
result.Width, result.Height)
|
||||
}
|
||||
}
|
||||
|
||||
result, output, err := processSVG(t, svg,
|
||||
imagerequest.Transform{Format: imagerequest.FormatOriginal})
|
||||
if err != nil || result.ContentType != imagerequest.FormatSVG.MIMEType() ||
|
||||
string(output) != svg {
|
||||
t.Errorf("orig of %s: error = %v, want the SVG itself", svg, err)
|
||||
}
|
||||
|
||||
_, _, err = processSVG(t, svg,
|
||||
imagerequest.Transform{Format: imagerequest.FormatPNG})
|
||||
if !errors.Is(err, ErrOutputTooLarge) {
|
||||
t.Errorf("%s as PNG at its own size: error = %v, want %v", svg,
|
||||
err, ErrOutputTooLarge)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestImageProcessor_RefusesSVGTooThin checks that an SVG with one side more
|
||||
// than MaxInputDimension times the other is ErrSVGTooThin at a raster size,
|
||||
// that one at exactly that proportion is rendered, and that orig of either
|
||||
// is the SVG itself.
|
||||
func TestImageProcessor_RefusesSVGTooThin(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
tooThin := `<svg xmlns="http://www.w3.org/2000/svg" width="81930" ` +
|
||||
`height="10"><rect width="81930" height="10"/></svg>`
|
||||
atTheLimit := `<svg xmlns="http://www.w3.org/2000/svg" width="81920" ` +
|
||||
`height="10"><rect width="81920" height="10"/></svg>`
|
||||
size := imagerequest.Size{Width: 100, Height: 100}
|
||||
|
||||
_, _, err := processSVG(t, tooThin,
|
||||
imagerequest.Transform{Size: size, Format: imagerequest.FormatPNG})
|
||||
if !errors.Is(err, ErrSVGTooThin) {
|
||||
t.Errorf("81930x10 SVG as PNG at 100x100: error = %v, want %v", err,
|
||||
ErrSVGTooThin)
|
||||
}
|
||||
|
||||
result, _, err := processSVG(t, atTheLimit,
|
||||
imagerequest.Transform{Size: size, Format: imagerequest.FormatPNG})
|
||||
if err != nil {
|
||||
t.Errorf("81920x10 SVG as PNG at 100x100: %v", err)
|
||||
} else if result.Width != 100 || result.Height != 100 {
|
||||
t.Errorf("81920x10 SVG as PNG at 100x100 = %dx%d, want 100x100",
|
||||
result.Width, result.Height)
|
||||
}
|
||||
|
||||
for _, svg := range []string{tooThin, atTheLimit} {
|
||||
result, output, err := processSVG(t, svg,
|
||||
imagerequest.Transform{Format: imagerequest.FormatOriginal})
|
||||
if err != nil || result.ContentType != imagerequest.FormatSVG.MIMEType() ||
|
||||
string(output) != svg {
|
||||
t.Errorf("orig of %s: error = %v, want the SVG itself", svg, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestSVGRenderSize checks the size at which an SVG is rendered to cover an
|
||||
// output size, and that it is held to MaxInputDimension.
|
||||
func TestSVGRenderSize(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
tests := []struct {
|
||||
name string
|
||||
width, height int
|
||||
outputWidth, outputHeight int
|
||||
wantWidth, wantHeight int
|
||||
}{
|
||||
{"own size", 16, 12, 16, 12, 16, 12},
|
||||
{"100 times larger", 16, 12, 1600, 1200, 1600, 1200},
|
||||
{"wider than the SVG's proportions", 16, 12, 160, 12, 160, 120},
|
||||
{"10 times smaller", 100, 100, 10, 10, 10, 10},
|
||||
{"held to MaxInputDimension", 8192, 1, 8192, 8192, 8192, 1},
|
||||
{"held to MaxInputDimension when enlarged", 1024, 1, 8192, 8192, 8192, 8},
|
||||
{"far above MaxInputDimension", 1000000, 1000000, 100, 100, 100, 100},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
gotWidth, gotHeight := svgRenderSize(tt.width, tt.height,
|
||||
tt.outputWidth, tt.outputHeight)
|
||||
if gotWidth != tt.wantWidth || gotHeight != tt.wantHeight {
|
||||
t.Errorf("%s: svgRenderSize(%d, %d, %d, %d) = %dx%d, want %dx%d",
|
||||
tt.name, tt.width, tt.height, tt.outputWidth, tt.outputHeight,
|
||||
gotWidth, gotHeight, tt.wantWidth, tt.wantHeight)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -14,7 +14,8 @@ import (
|
||||
var ErrInvalidFitMode = errors.New("invalid fit mode")
|
||||
|
||||
// Format is an image format. pixa reads and writes the formats from JPEG to
|
||||
// GIF below. In a request, orig and auto stand for one of them.
|
||||
// GIF below. It reads SVG and writes it only as orig of an SVG source. In a
|
||||
// request, orig and auto stand for another format.
|
||||
type Format string
|
||||
|
||||
// Supported image formats.
|
||||
@@ -26,6 +27,7 @@ const (
|
||||
FormatAVIF Format = "avif"
|
||||
FormatJXL Format = "jxl"
|
||||
FormatGIF Format = "gif"
|
||||
FormatSVG Format = "svg"
|
||||
|
||||
// FormatAuto stands for JPEG XL, AVIF, WebP or JPEG, chosen for each
|
||||
// request from its Accept header once the URL's signature or token has
|
||||
@@ -50,6 +52,8 @@ func (f Format) MIMEType() string {
|
||||
return "image/jxl"
|
||||
case FormatGIF:
|
||||
return "image/gif"
|
||||
case FormatSVG:
|
||||
return "image/svg+xml"
|
||||
case FormatOriginal, FormatAuto:
|
||||
return ""
|
||||
default:
|
||||
|
||||
@@ -449,16 +449,18 @@ func (c *Cache) cleanExpiredBatch(ctx context.Context) (int64, error) {
|
||||
return deleted, nil
|
||||
}
|
||||
|
||||
// readFormat reads the first bytes of a source or variant file, without
|
||||
// moving its offset, and returns the format they show. Sources and variants
|
||||
// are in the same six formats.
|
||||
// readFormat reads up to magic.MaxMagicBytes first bytes of a source or
|
||||
// variant file, without moving its offset, and returns the format they show.
|
||||
// Sources and variants are in the same seven formats; a variant is SVG only
|
||||
// as orig of an SVG source. A file shorter than magic.MinMagicBytes is
|
||||
// io.EOF.
|
||||
func readFormat(content io.ReaderAt) (imagerequest.Format, error) {
|
||||
firstBytes := make([]byte, magic.MinMagicBytes)
|
||||
firstBytes := make([]byte, magic.MaxMagicBytes)
|
||||
|
||||
_, err := content.ReadAt(firstBytes, 0)
|
||||
if err != nil {
|
||||
n, err := content.ReadAt(firstBytes, 0)
|
||||
if err != nil && (!errors.Is(err, io.EOF) || n < magic.MinMagicBytes) {
|
||||
return "", fmt.Errorf("failed to read the first bytes: %w", err)
|
||||
}
|
||||
|
||||
return magic.DetectFormat(firstBytes)
|
||||
return magic.DetectFormat(firstBytes[:n])
|
||||
}
|
||||
@@ -95,8 +95,6 @@ func TestVariant_RefusesFileOfNoOutputFormat(t *testing.T) {
|
||||
wantErr error
|
||||
}{
|
||||
{"text", []byte("this is text, not an image"), magic.ErrUnknownFormat},
|
||||
{"SVG", []byte(`<svg xmlns="http://www.w3.org/2000/svg"></svg>`),
|
||||
magic.ErrUnknownFormat},
|
||||
{"JPEG cut to 10 bytes", jpegImage[:10], io.EOF},
|
||||
{"empty", nil, io.EOF},
|
||||
}
|
||||
|
||||
@@ -216,8 +216,6 @@ func TestSource_RefusesFileOfNoSourceFormat(t *testing.T) {
|
||||
wantErr error
|
||||
}{
|
||||
{"text", []byte("this is text, not an image"), magic.ErrUnknownFormat},
|
||||
{"SVG", []byte(`<svg xmlns="http://www.w3.org/2000/svg"></svg>`),
|
||||
magic.ErrUnknownFormat},
|
||||
{"JPEG cut to 10 bytes", jpegImage[:10], io.EOF},
|
||||
{"empty file", nil, io.EOF},
|
||||
}
|
||||
|
||||
@@ -0,0 +1,54 @@
|
||||
package imgcache
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"sneak.berlin/go/pixa/internal/imagerequest"
|
||||
)
|
||||
|
||||
// testSVG is an SVG with a comment before its root element, so that its
|
||||
// first magic.MinMagicBytes bytes do not show its format.
|
||||
const testSVG = `<!-- a drawing --><svg xmlns="http://www.w3.org/2000/svg" ` +
|
||||
`width="4" height="4"/>`
|
||||
|
||||
// TestSourceAndVariant_ReadSVG checks that a cached SVG source is read as
|
||||
// SVG, and a cached SVG variant (orig of an SVG source) as image/svg+xml.
|
||||
func TestSourceAndVariant_ReadSVG(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
cache, _ := newEvictionTestCache(t, 1<<30)
|
||||
|
||||
storeEvictionTestSource(t, cache, testHostCDN, "/drawing.svg", []byte(testSVG))
|
||||
|
||||
reader, _, format, err := cache.Source(t.Context(),
|
||||
imagerequest.Source{Host: testHostCDN, Path: "/drawing.svg"})
|
||||
t.Logf("Source() format = %q, error = %v", format, err)
|
||||
|
||||
if err != nil || format != imagerequest.FormatSVG {
|
||||
t.Errorf("Source() = format %q, error %v; want %q", format, err,
|
||||
imagerequest.FormatSVG)
|
||||
}
|
||||
|
||||
if reader != nil {
|
||||
_ = reader.Close()
|
||||
}
|
||||
|
||||
key := imagerequest.VariantKey("aabbccdd3001")
|
||||
|
||||
err = cache.variants.Write(string(key), []byte(testSVG))
|
||||
if err != nil {
|
||||
t.Fatalf("storing the variant file: %v", err)
|
||||
}
|
||||
|
||||
reader, _, contentType, err := cache.Variant(key)
|
||||
t.Logf("Variant() content type = %q, error = %v", contentType, err)
|
||||
|
||||
if err != nil || contentType != "image/svg+xml" {
|
||||
t.Errorf("Variant() = content type %q, error %v; want image/svg+xml",
|
||||
contentType, err)
|
||||
}
|
||||
|
||||
if reader != nil {
|
||||
_ = reader.Close()
|
||||
}
|
||||
}
|
||||
+27
-1
@@ -20,6 +20,10 @@ var (
|
||||
// MinMagicBytes is the minimum number of bytes needed to detect format.
|
||||
const MinMagicBytes = 12
|
||||
|
||||
// MaxMagicBytes is the most bytes DetectFormat looks at. An SVG's root
|
||||
// element must start within them, as libvips requires.
|
||||
const MaxMagicBytes = 1024
|
||||
|
||||
// Magic byte signatures for supported formats.
|
||||
// These are effectively constants but Go doesn't support const slices.
|
||||
//
|
||||
@@ -39,6 +43,12 @@ var (
|
||||
}
|
||||
)
|
||||
|
||||
// utf8BOM is the UTF-8 byte order mark, which a text file such as an SVG may
|
||||
// start with.
|
||||
//
|
||||
//nolint:gochecknoglobals // immutable lookup data
|
||||
var utf8BOM = []byte{0xEF, 0xBB, 0xBF}
|
||||
|
||||
// WebP identifier appears at offset 8 after RIFF header.
|
||||
//
|
||||
//nolint:gochecknoglobals // immutable lookup data
|
||||
@@ -53,7 +63,7 @@ var (
|
||||
)
|
||||
|
||||
// DetectFormat detects the image format from magic bytes. Data in none of
|
||||
// the formats pixa reads, SVG included, is ErrUnknownFormat.
|
||||
// the formats pixa reads is ErrUnknownFormat.
|
||||
func DetectFormat(data []byte) (imagerequest.Format, error) {
|
||||
if len(data) < MinMagicBytes {
|
||||
return "", ErrNotEnoughData
|
||||
@@ -96,9 +106,25 @@ func DetectFormat(data []byte) (imagerequest.Format, error) {
|
||||
return imagerequest.FormatJXL, nil
|
||||
}
|
||||
|
||||
if isSVG(data) {
|
||||
return imagerequest.FormatSVG, nil
|
||||
}
|
||||
|
||||
return "", ErrUnknownFormat
|
||||
}
|
||||
|
||||
// isSVG reports whether data starts as an SVG does: with a "<", after an
|
||||
// optional UTF-8 byte order mark and white space, and with "<svg" within the
|
||||
// first MaxMagicBytes bytes. Before the root element an SVG may have an XML
|
||||
// declaration, comments and a DOCTYPE.
|
||||
func isSVG(data []byte) bool {
|
||||
data = data[:min(len(data), MaxMagicBytes)]
|
||||
text := bytes.TrimLeft(bytes.TrimPrefix(data, utf8BOM), " \t\r\n")
|
||||
|
||||
return bytes.HasPrefix(text, []byte("<")) &&
|
||||
bytes.Contains(text, []byte("<svg"))
|
||||
}
|
||||
|
||||
// ValidateMagicBytes detects the format of data from its magic bytes, and
|
||||
// returns it when it is the format the declared MIME type names. Data in
|
||||
// another format is ErrMagicByteMismatch.
|
||||
|
||||
@@ -47,9 +47,9 @@ func TestDetectFormat(t *testing.T) {
|
||||
{name: "AVIF", data: avif, wantFormat: imagerequest.FormatAVIF},
|
||||
{name: "AVIF sequence", data: avis, wantFormat: imagerequest.FormatAVIF},
|
||||
{
|
||||
name: "SVG",
|
||||
data: []byte(`<svg xmlns="http://www.w3.org/2000/svg"></svg>`),
|
||||
wantErr: ErrUnknownFormat,
|
||||
name: "SVG",
|
||||
data: []byte(`<svg xmlns="http://www.w3.org/2000/svg"></svg>`),
|
||||
wantFormat: imagerequest.FormatSVG,
|
||||
},
|
||||
{
|
||||
name: "unknown format",
|
||||
|
||||
@@ -0,0 +1,85 @@
|
||||
package magic
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"sneak.berlin/go/pixa/internal/imagerequest"
|
||||
)
|
||||
|
||||
// TestDetectFormat_SVG checks that DetectFormat names SVG text with what may
|
||||
// come before the root element, and does not name as SVG text that does not
|
||||
// start with "<" or has no "<svg" within its first MaxMagicBytes bytes.
|
||||
func TestDetectFormat_SVG(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
const root = `<svg xmlns="http://www.w3.org/2000/svg" width="4" height="4"/>`
|
||||
|
||||
tests := []struct {
|
||||
name string
|
||||
data string
|
||||
wantErr error
|
||||
}{
|
||||
{name: "root element only", data: root},
|
||||
{name: "XML declaration", data: `<?xml version="1.0"?>` + "\n" + root},
|
||||
{name: "byte order mark and white space", data: "\xEF\xBB\xBF \n\t" + root},
|
||||
{name: "comment", data: "<!-- a drawing -->" + root},
|
||||
{
|
||||
name: "DOCTYPE",
|
||||
data: `<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" ` +
|
||||
`"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">` + root,
|
||||
},
|
||||
{
|
||||
name: "root element after the first bytes",
|
||||
data: "<!--" + strings.Repeat(" ", MaxMagicBytes) + "-->" + root,
|
||||
wantErr: ErrUnknownFormat,
|
||||
},
|
||||
{name: "text naming <svg", data: "draw it with " + root, wantErr: ErrUnknownFormat},
|
||||
{name: "HTML", data: "<!DOCTYPE html><html></html>", wantErr: ErrUnknownFormat},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
got, err := DetectFormat([]byte(tt.data))
|
||||
t.Logf("DetectFormat() = %q, %v", got, err)
|
||||
|
||||
if tt.wantErr != nil {
|
||||
if !errors.Is(err, tt.wantErr) {
|
||||
t.Errorf("DetectFormat() error = %v, want %v", err, tt.wantErr)
|
||||
}
|
||||
|
||||
return
|
||||
}
|
||||
|
||||
if err != nil || got != imagerequest.FormatSVG {
|
||||
t.Errorf("DetectFormat() = %q, %v; want %q", got, err,
|
||||
imagerequest.FormatSVG)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestValidateMagicBytes_SVG checks that SVG data is SVG for the
|
||||
// Content-Type image/svg+xml, with a charset too, and a mismatch for another.
|
||||
func TestValidateMagicBytes_SVG(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
data := []byte(`<svg xmlns="http://www.w3.org/2000/svg" width="4" height="4"/>`)
|
||||
|
||||
for _, declared := range []string{"image/svg+xml", "image/svg+xml; charset=utf-8"} {
|
||||
got, err := ValidateMagicBytes(data, declared)
|
||||
if err != nil || got != imagerequest.FormatSVG {
|
||||
t.Errorf("ValidateMagicBytes(%q) = %q, %v; want %q", declared, got,
|
||||
err, imagerequest.FormatSVG)
|
||||
}
|
||||
}
|
||||
|
||||
_, err := ValidateMagicBytes(data, "image/png")
|
||||
if !errors.Is(err, ErrMagicByteMismatch) {
|
||||
t.Errorf("ValidateMagicBytes(image/png) error = %v, want %v", err,
|
||||
ErrMagicByteMismatch)
|
||||
}
|
||||
}
|
||||
@@ -4,38 +4,114 @@ import (
|
||||
"io"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"sync/atomic"
|
||||
"testing"
|
||||
|
||||
"sneak.berlin/go/pixa/internal/imageproxy"
|
||||
"sneak.berlin/go/pixa/internal/imagerequest"
|
||||
"sneak.berlin/go/pixa/internal/magic"
|
||||
)
|
||||
|
||||
// TestImageProxyRefusesSVG requests an image the upstream answers as SVG,
|
||||
// in its own format (orig) and resized as PNG, through pixa's router,
|
||||
// upstream fetcher and image processor. pixa does not accept SVG as a
|
||||
// source, so each request is answered 415, never with an image.
|
||||
func TestImageProxyRefusesSVG(t *testing.T) {
|
||||
t.Parallel()
|
||||
// testSVG is the SVG the upstream serves in the SVG tests below.
|
||||
const testSVG = `<svg xmlns="http://www.w3.org/2000/svg" width="64" height="48">` +
|
||||
`<rect width="64" height="48"/></svg>`
|
||||
|
||||
// startSVGUpstream starts an upstream that answers every request with
|
||||
// testSVG as image/svg+xml, and returns it with its count of requests.
|
||||
func startSVGUpstream(t *testing.T) (*httptest.Server, *atomic.Int64) {
|
||||
t.Helper()
|
||||
|
||||
var requests atomic.Int64
|
||||
|
||||
upstream := httptest.NewServer(http.HandlerFunc(
|
||||
func(w http.ResponseWriter, _ *http.Request) {
|
||||
requests.Add(1)
|
||||
w.Header().Set("Content-Type", "image/svg+xml")
|
||||
_, _ = io.WriteString(w, `<svg xmlns="http://www.w3.org/2000/svg" `+
|
||||
`width="64" height="48"><rect width="64" height="48"/></svg>`)
|
||||
_, _ = io.WriteString(w, testSVG)
|
||||
}))
|
||||
t.Cleanup(upstream.Close)
|
||||
|
||||
return upstream, &requests
|
||||
}
|
||||
|
||||
// TestImageProxyServesSVGAsItIs requests an SVG source in its own format
|
||||
// (orig) at the orig size through pixa's router, upstream fetcher, image
|
||||
// processor and cache, twice. Both answers are the SVG itself, never another
|
||||
// format, the second from the cache; the upstream is asked once. Scripts in
|
||||
// the SVG cannot run, as the Content-Security-Policy allows no inline
|
||||
// script. orig at another size is refused with 400.
|
||||
func TestImageProxyServesSVGAsItIs(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
upstream, requests := startSVGUpstream(t)
|
||||
s, _, _ := startImageProxy(t, upstream)
|
||||
target := "/v1/image/" + upstreamHost + "/drawing.svg/orig.orig"
|
||||
|
||||
for _, wantCache := range []imageproxy.CacheStatus{
|
||||
imageproxy.CacheMiss, imageproxy.CacheHit,
|
||||
} {
|
||||
rec := getImage(t, s, target)
|
||||
|
||||
if rec.Header().Get("Content-Type") != "image/svg+xml" ||
|
||||
rec.Body.String() != testSVG {
|
||||
t.Errorf("GET %s: %s %q, want image/svg+xml %q", target,
|
||||
rec.Header().Get("Content-Type"), rec.Body.String(), testSVG)
|
||||
}
|
||||
|
||||
if got := rec.Header().Get("X-Pixa-Cache"); got != string(wantCache) {
|
||||
t.Errorf("GET %s: X-Pixa-Cache = %q, want %q", target, got, wantCache)
|
||||
}
|
||||
|
||||
policy := rec.Header().Get("Content-Security-Policy")
|
||||
if !strings.Contains(policy, "script-src 'self'") ||
|
||||
strings.Contains(policy, "unsafe-inline") {
|
||||
t.Errorf("GET %s: Content-Security-Policy = %q, want script-src "+
|
||||
"'self' and no unsafe-inline", target, policy)
|
||||
}
|
||||
}
|
||||
|
||||
if requests.Load() != 1 {
|
||||
t.Errorf("upstream requests = %d, want 1", requests.Load())
|
||||
}
|
||||
|
||||
resized := "/v1/image/" + upstreamHost + "/drawing.svg/32x24.orig"
|
||||
|
||||
rec := httptest.NewRecorder()
|
||||
s.ServeHTTP(rec, httptest.NewRequestWithContext(
|
||||
t.Context(), http.MethodGet, resized, nil))
|
||||
t.Logf("GET %s: %d %s", resized, rec.Code, rec.Body.String())
|
||||
|
||||
if rec.Code != http.StatusBadRequest {
|
||||
t.Errorf("GET %s: %d %s, want 400", resized, rec.Code, rec.Body.String())
|
||||
}
|
||||
}
|
||||
|
||||
// TestImageProxyRendersSVG requests an SVG source as each raster format
|
||||
// through pixa's router, upstream fetcher and image processor. Each answer
|
||||
// is an image in that format.
|
||||
func TestImageProxyRendersSVG(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
upstream, _ := startSVGUpstream(t)
|
||||
s, _, _ := startImageProxy(t, upstream)
|
||||
|
||||
for _, sizeFormat := range []string{"64x48.orig", "32x24.png"} {
|
||||
target := "/v1/image/" + upstreamHost + "/drawing.svg/" + sizeFormat
|
||||
for _, format := range []imagerequest.Format{
|
||||
imagerequest.FormatJPEG, imagerequest.FormatPNG, imagerequest.FormatGIF,
|
||||
imagerequest.FormatWebP, imagerequest.FormatAVIF, imagerequest.FormatJXL,
|
||||
} {
|
||||
target := "/v1/image/" + upstreamHost + "/drawing.svg/32x24." +
|
||||
string(format)
|
||||
|
||||
rec := httptest.NewRecorder()
|
||||
s.ServeHTTP(rec, httptest.NewRequestWithContext(
|
||||
t.Context(), http.MethodGet, target, nil))
|
||||
t.Logf("GET %s: %d, Content-Type %s, body %s",
|
||||
target, rec.Code, rec.Header().Get("Content-Type"), rec.Body.String())
|
||||
rec := getImage(t, s, target)
|
||||
detected, err := magic.DetectFormat(rec.Body.Bytes())
|
||||
t.Logf("GET %s: %s, body in %q (%v)", target,
|
||||
rec.Header().Get("Content-Type"), detected, err)
|
||||
|
||||
if rec.Code != http.StatusUnsupportedMediaType {
|
||||
t.Errorf("GET %s status = %d, want %d",
|
||||
target, rec.Code, http.StatusUnsupportedMediaType)
|
||||
if rec.Header().Get("Content-Type") != format.MIMEType() ||
|
||||
err != nil || detected != format {
|
||||
t.Errorf("GET %s: %s, body in %q (%v); want %s", target,
|
||||
rec.Header().Get("Content-Type"), detected, err, format.MIMEType())
|
||||
}
|
||||
}
|
||||
}
|
||||
Reference in new issue
Block a user