Support SVG sources (closes #68)
check / check (push) Canceled after 0s

The fetcher accepts image/svg+xml and internal/magic detects SVG. orig
of an SVG at the orig size is the SVG unchanged, cached as a variant; at
another size it is ErrSVGNotOriginalSize (400). Any other format renders
it with libvips' thumbnail at the least size covering the output, at
most 8192 on a side, so it stays sharp. An SVG's own size is not
limited; one with a side over 8192 times the other is ErrSVGTooThin
(415). The cache reads up to 1024 first bytes to find a file's format.
Tests that pinned the SVG refusal now expect SVG to be read. Render time
and memory are unbounded: P1 in TODO.md until the owner decides on
#68.

Model: opus-5-5
This commit is contained in:
clawbot committed 2026-11-11 15:43:16 +00:00
1 parent 7f87aa17ca
commit d96899eca6
20 files changed
+858 -87

No files matched your search

+40 -14
View File
@@ -182,7 +182,8 @@ run under. Its app for pixa needs:
is half the CPUs pixa sees times 4.2 GiB, plus 64 times 50 MiB. On a host is half the CPUs pixa sees times 4.2 GiB, plus 64 times 50 MiB. On a host
where pixa sees 48 CPUs, that is 24 times 4.2 GiB plus 3.1 GiB, about 104 GiB. where pixa sees 48 CPUs, that is 24 times 4.2 GiB plus 3.1 GiB, about 104 GiB.
Set `PIXA_MAX_CONCURRENT_PROCESSING` and `PIXA_UPSTREAM_CONNECTIONS` so that Set `PIXA_MAX_CONCURRENT_PROCESSING` and `PIXA_UPSTREAM_CONNECTIONS` so that
this fits the memory the container may use. this fits the memory the container may use. Rendering an SVG source can need
more, depending on what it draws.
- **Temporary files:** while pixa processes a large source, libvips writes the - **Temporary files:** while pixa processes a large source, libvips writes the
decoded pixels to a file in `/tmp`. The file is removed from the directory decoded pixels to a file in `/tmp`. The file is removed from the directory
once it is written, so a listing of `/tmp` does not show it. It keeps its once it is written, so a listing of `/tmp` does not show it. It keeps its
@@ -285,6 +286,7 @@ to any method the list does not give for it.
- 304 when `If-None-Match` matches the image's `ETag`. - 304 when `If-None-Match` matches the image's `ETag`.
- 400 for a URL or parameter that is not valid. - 400 for a URL or parameter that is not valid.
- 400 when the format is `auto` and the `Accept` header is not valid. - 400 when the format is `auto` and the `Accept` header is not valid.
- 400 for `orig` of an SVG source at a size other than `orig`.
- 406 when the format is `auto` and `Accept` allows none of the formats it - 406 when the format is `auto` and `Accept` allows none of the formats it
chooses from. chooses from.
- 401 for a missing or wrong signature. - 401 for a missing or wrong signature.
@@ -305,6 +307,8 @@ to any method the list does not give for it.
format its `Content-Type` names. format its `Content-Type` names.
- 415 when the upstream sends an image whose first bytes libvips reads as - 415 when the upstream sends an image whose first bytes libvips reads as
another format. another format.
- 415 for an SVG source with one side more than 8192 times the other, in any
format but `orig`.
- 502 when the upstream answers with an error status. - 502 when the upstream answers with an error status.
- 502 for 5 minutes after that for the same source URL, when pixa has - 502 for 5 minutes after that for the same source URL, when pixa has
neither the image nor its source in its cache. neither the image nor its source in its cache.
@@ -322,6 +326,8 @@ to any method the list does not give for it.
- 304 when `If-None-Match` matches the image's `ETag`. - 304 when `If-None-Match` matches the image's `ETag`.
- 400 for a token that does not decrypt. - 400 for a token that does not decrypt.
- 400 for a token that asks for a size or fit that is not valid. - 400 for a token that asks for a size or fit that is not valid.
- 400 for a token that asks for `orig` of an SVG source at a size other than
`orig`.
- 410 once the token has expired. - 410 once the token has expired.
- 400 when the token's format is `auto` and the `Accept` header is not - 400 when the token's format is `auto` and the `Accept` header is not
valid. valid.
@@ -460,10 +466,11 @@ outside ASCII. pixa signs and caches the query in that form. So
`img%3Fname=my%20cat` and `img%3Fname=my%2520cat` in the image URL are the same `img%3Fname=my%20cat` and `img%3Fname=my%2520cat` in the image URL are the same
source query, `name=my%20cat`. source query, `name=my%20cat`.
**Sizes.** The largest width or height is 8192, for the source and for the **Sizes.** The largest width or height is 8192, for the output and for a source
output. A size with one side `0` has that side worked out from the other side in any format but SVG. A size with one side `0` has that side worked out from
and the source's proportions. The side is rounded down, but is at least 1. When the other side and the source's proportions. The side is rounded down, but is at
it would come out above 8192, the request is refused with 400. The fit modes: least 1. When it would come out above 8192, the request is refused with 400. The
fit modes:
- `cover`: scale the image to cover the size, keeping its proportions, and cut - `cover`: scale the image to cover the size, keeping its proportions, and cut
out the centre. The output is exactly the size. out the centre. The output is exactly the size.
@@ -483,9 +490,9 @@ applies to the format `orig` when the source is AVIF, and to `auto` when it
chooses AVIF. So a client whose `Accept` names `image/avif` but not `image/jxl` chooses AVIF. So a client whose `Accept` names `image/avif` but not `image/jxl`
is refused such a size. is refused such a size.
**Source formats.** The source may be JPEG, PNG, GIF, WebP, AVIF or JPEG XL. The **Source formats.** The source may be JPEG, PNG, GIF, WebP, AVIF, JPEG XL or
upstream's `Content-Type` must name its format, and the image's first bytes must SVG. The upstream's `Content-Type` must name its format, and the image's first
match it. An upstream answering with any other `Content-Type`, SVG included, is bytes must match it. An upstream answering with any other `Content-Type` is
refused with 415. So is a body whose first bytes are not those of the format its refused with 415. So is a body whose first bytes are not those of the format its
`Content-Type` names: a JPEG sent as `image/png`, text sent as an image, or an `Content-Type` names: a JPEG sent as `image/png`, text sent as an image, or an
empty body. pixa then does not fetch that source URL again for 5 minutes, as empty body. pixa then does not fetch that source URL again for 5 minutes, as
@@ -494,6 +501,20 @@ need that fetch. An image whose first bytes libvips reads as another format is
refused with 415 too. An SVG whose first bytes pass the check for AVIF is one refused with 415 too. An SVG whose first bytes pass the check for AVIF is one
example. example.
**SVG sources.** `orig` of an SVG source is the SVG itself, unchanged, and is
cached like any other image. It is given only at the `orig` size, since pixa
does not resize an SVG as SVG. `orig` of an SVG at any other size is refused
with 400. A browser that opens an SVG from pixa runs none of its scripts, as
pixa's `Content-Security-Policy` allows no inline script.
In any format but `orig`, pixa renders an SVG source at the requested size, so
an enlarged SVG stays sharp. Rendering fetches no URL and reads no file that the
SVG refers to. In any format but `orig`, an SVG at the `orig` size is rendered
at its own size. In any format but `orig`, an SVG above 8192 on a side is
refused with 400 at the `orig` size. In any format but `orig`, an SVG with one
side more than 8192 times the other is refused with 415. Rendered with no side
above 8192, its shorter side would be under 1 pixel.
**The format `auto`.** With `auto`, pixa chooses the format for each request **The format `auto`.** With `auto`, pixa chooses the format for each request
from its `Accept` header, in this order: from its `Accept` header, in this order:
@@ -704,8 +725,9 @@ written percent-encoded.
### Image Metadata ### Image Metadata
pixa decodes and re-encodes every image it serves, and removes all metadata from pixa decodes and re-encodes every image it serves, except `orig` of an SVG
the output. This cannot be turned off. The metadata removed is: source, which is the SVG unchanged. It removes all metadata from every image it
re-encodes. This cannot be turned off. The metadata removed is:
- EXIF, which can hold the GPS position, the camera make, model and serial - EXIF, which can hold the GPS position, the camera make, model and serial
number, the capture time and an embedded thumbnail; number, the capture time and an embedded thumbnail;
@@ -714,14 +736,16 @@ the output. This cannot be turned off. The metadata removed is:
The rest of the rules: The rest of the rules:
- The `orig` format means the source's own format, not the source's bytes. An - For a source in any format but SVG, the `orig` format means the source's own
`orig` image is re-encoded and stripped like any other. format, not the source's bytes. An `orig` image is re-encoded and stripped
like any other. For an SVG source, `orig` is the SVG's own bytes, its metadata
included.
- An image with an EXIF orientation is turned upright first, so it displays the - An image with an EXIF orientation is turned upright first, so it displays the
same without the tag. A requested size applies to the upright image. same without the tag. A requested size applies to the upright image.
- An image with an ICC profile is first converted to sRGB, the standard colour - An image with an ICC profile is first converted to sRGB, the standard colour
space, since clients show an image with no profile as sRGB. Colours outside space, since clients show an image with no profile as sRGB. Colours outside
sRGB are clipped: the most saturated ones in a Display P3 photo, for example. sRGB are clipped: the most saturated ones in a Display P3 photo, for example.
- JPEG XL output is the one exception, with libvips 8.16 and later. libvips then - JPEG XL output is an exception, with libvips 8.16 and later. libvips then
writes an EXIF block of its own, which pixa cannot stop. The block holds the writes an EXIF block of its own, which pixa cannot stop. The block holds the
image's size and fixed values: an orientation of 1 and a resolution of 72 dpi. image's size and fixed values: an orientation of 1 and a resolution of 72 dpi.
Nothing in it comes from the source. The Docker image has libvips 8.16, so its Nothing in it comes from the source. The Docker image has libvips 8.16, so its
@@ -773,7 +797,9 @@ about 34 seconds with an alpha channel, and 22 without. At 8192x8192 and quality
85 it would take about 96 seconds with an alpha channel, and 52 without. The 34 85 it would take about 96 seconds with an alpha channel, and 52 without. The 34
seconds hold at any quality and at 8 or 16 bits per sample. No encode within the seconds hold at any quality and at 8 or 16 bits per sample. No encode within the
limits takes more than about 40 seconds. The default `downstream_timeout` is 60 limits takes more than about 40 seconds. The default `downstream_timeout` is 60
seconds. seconds. These limits do not bound rendering an SVG source, which happens within
the encode: its time and memory grow with what the SVG draws, such as filter
effects.
**Processing and connections.** `max_concurrent_processing` images are decoded **Processing and connections.** `max_concurrent_processing` images are decoded
and encoded at once. A request that finds every slot in use waits up to 10 and encoded at once. A request that finds every slot in use waits up to 10
+6 -1
View File
@@ -20,10 +20,15 @@ pre-1.0. The `1.0.0` milestone is in progress.
# Next Step # Next Step
P2: security: per-origin rate limiting P1: security: bound the time and memory of rendering an SVG source. Both grow
with what the SVG draws, such as filter effects. govips cannot stop or limit a
render. The bound is the owner's decision on
https://git.eeqj.de/sneak/pixa/issues/68.
# Future Steps # Future Steps
- P2: security
- per-origin rate limiting
- P2: HTTP response handling - P2: HTTP response handling
- Last-Modified headers - Last-Modified headers
- P2: operational - P2: operational
+8
View File
@@ -267,11 +267,19 @@ func (s *Handlers) respondImageError(
errors.Is(err, imageprocessor.ErrUnsupportedInputFormat): errors.Is(err, imageprocessor.ErrUnsupportedInputFormat):
status = http.StatusUnsupportedMediaType status = http.StatusUnsupportedMediaType
message = "unsupported source format" message = "unsupported source format"
case errors.Is(err, imageprocessor.ErrSVGTooThin):
status = http.StatusUnsupportedMediaType
message = fmt.Sprintf("SVG source too thin to render: one side is "+
"more than %d times the other", imageprocessor.MaxInputDimension)
case errors.Is(err, imageprocessor.ErrOutputTooLarge): case errors.Is(err, imageprocessor.ErrOutputTooLarge):
status = http.StatusBadRequest status = http.StatusBadRequest
message = fmt.Sprintf("size too large for the source: the side "+ message = fmt.Sprintf("size too large for the source: the side "+
"computed from its proportions is above %d", "computed from its proportions is above %d",
imageprocessor.MaxOutputDimension) imageprocessor.MaxOutputDimension)
case errors.Is(err, imageprocessor.ErrSVGNotOriginalSize):
status = http.StatusBadRequest
message = "an SVG source is returned as SVG only at the orig size; " +
"ask for another format to resize it"
case errors.Is(err, imageprocessor.ErrAVIFOutputTooLarge): case errors.Is(err, imageprocessor.ErrAVIFOutputTooLarge):
status = http.StatusBadRequest status = http.StatusBadRequest
message = fmt.Sprintf("size too large for AVIF: the output "+ message = fmt.Sprintf("size too large for AVIF: the output "+
@@ -467,7 +467,7 @@ func TestImageRoutes_SameErrorAnswers(t *testing.T) {
{"no free upstream connection", httpfetcher.ErrTooManyConnections, {"no free upstream connection", httpfetcher.ErrTooManyConnections,
http.StatusServiceUnavailable, slog.LevelWarn}, http.StatusServiceUnavailable, slog.LevelWarn},
{"Content-Type not a source format", {"Content-Type not a source format",
fmt.Errorf("%w: image/svg+xml", httpfetcher.ErrInvalidContentType), fmt.Errorf("%w: image/tiff", httpfetcher.ErrInvalidContentType),
http.StatusUnsupportedMediaType, slog.LevelWarn}, http.StatusUnsupportedMediaType, slog.LevelWarn},
{"any other failure", io.ErrUnexpectedEOF, {"any other failure", io.ErrUnexpectedEOF,
http.StatusInternalServerError, slog.LevelError}, http.StatusInternalServerError, slog.LevelError},
@@ -15,7 +15,7 @@ import (
// TestUnsupportedSourceFormatAnswers415 checks that respondImageError, which // TestUnsupportedSourceFormatAnswers415 checks that respondImageError, which
// answers the errors of both image routes, answers 415 with a clear error when // answers the errors of both image routes, answers 415 with a clear error when
// the upstream answers with a content type pixa does not accept, such as SVG, // the upstream answers with a content type pixa does not accept, such as TIFF,
// and when libvips does not read a source as the format its first bytes // and when libvips does not read a source as the format its first bytes
// show, wrapped as the service wraps them. // show, wrapped as the service wraps them.
func TestUnsupportedSourceFormatAnswers415(t *testing.T) { func TestUnsupportedSourceFormatAnswers415(t *testing.T) {
@@ -23,11 +23,11 @@ func TestUnsupportedSourceFormatAnswers415(t *testing.T) {
h := &Handlers{log: slog.New(slog.DiscardHandler)} h := &Handlers{log: slog.New(slog.DiscardHandler)}
req := &imagerequest.Request{ req := &imagerequest.Request{
Source: imagerequest.Source{Host: "img.example.com", Path: "/a.svg"}, Source: imagerequest.Source{Host: "img.example.com", Path: "/a.tiff"},
} }
for _, err := range []error{ for _, err := range []error{
fmt.Errorf("upstream fetch failed: %w: image/svg+xml", fmt.Errorf("upstream fetch failed: %w: image/tiff",
httpfetcher.ErrInvalidContentType), httpfetcher.ErrInvalidContentType),
fmt.Errorf("image processing failed: %w: libvips does not read the input as avif", fmt.Errorf("image processing failed: %w: libvips does not read the input as avif",
imageprocessor.ErrUnsupportedInputFormat), imageprocessor.ErrUnsupportedInputFormat),
@@ -47,3 +47,30 @@ func TestUnsupportedSourceFormatAnswers415(t *testing.T) {
} }
} }
} }
// TestSVGTooThinAnswers415 checks that respondImageError answers 415 with a
// clear error for an SVG source too thin to render, wrapped as the service
// wraps it.
func TestSVGTooThinAnswers415(t *testing.T) {
t.Parallel()
h := &Handlers{log: slog.New(slog.DiscardHandler)}
req := &imagerequest.Request{
Source: imagerequest.Source{Host: "svg.example.com", Path: "/a.svg"},
}
err := fmt.Errorf("image processing failed: %w: 81930x10",
imageprocessor.ErrSVGTooThin)
rec := httptest.NewRecorder()
h.respondImageError(rec, req, err)
t.Logf("for %v: %d %s", err, rec.Code, rec.Body.String())
if rec.Code != http.StatusUnsupportedMediaType {
t.Errorf("status = %d, want %d", rec.Code,
http.StatusUnsupportedMediaType)
}
if !strings.Contains(rec.Body.String(), "SVG source too thin to render") {
t.Errorf("body = %q, want the SVG too thin error", rec.Body.String())
}
}
+1
View File
@@ -124,6 +124,7 @@ func allowedContentTypes() []string {
imagerequest.FormatWebP.MIMEType(), imagerequest.FormatWebP.MIMEType(),
imagerequest.FormatAVIF.MIMEType(), imagerequest.FormatAVIF.MIMEType(),
imagerequest.FormatJXL.MIMEType(), imagerequest.FormatJXL.MIMEType(),
imagerequest.FormatSVG.MIMEType(),
} }
} }
@@ -279,7 +279,7 @@ func TestDetectContentTypeFromPath(t *testing.T) {
{"foo/bar.gif", imagerequest.FormatGIF.MIMEType()}, {"foo/bar.gif", imagerequest.FormatGIF.MIMEType()},
{"foo/bar.webp", imagerequest.FormatWebP.MIMEType()}, {"foo/bar.webp", imagerequest.FormatWebP.MIMEType()},
{"foo/bar.avif", imagerequest.FormatAVIF.MIMEType()}, {"foo/bar.avif", imagerequest.FormatAVIF.MIMEType()},
{"foo/bar.svg", contentTypeSVG}, {"foo/bar.svg", imagerequest.FormatSVG.MIMEType()},
{"foo/bar.bin", contentTypeOctetStream}, {"foo/bar.bin", contentTypeOctetStream},
{"foo/bar", contentTypeOctetStream}, {"foo/bar", contentTypeOctetStream},
} }
+4 -6
View File
@@ -67,11 +67,9 @@ func (m *MockFetcher) Fetch(
}, nil }, nil
} }
// The content types the mock fetcher gives files in no format pixa reads. // contentTypeOctetStream is the content type the mock fetcher gives files in
const ( // no format pixa reads.
contentTypeSVG = "image/svg+xml" const contentTypeOctetStream = "application/octet-stream"
contentTypeOctetStream = "application/octet-stream"
)
// detectContentTypeFromPath returns the MIME type based on file extension. // detectContentTypeFromPath returns the MIME type based on file extension.
func detectContentTypeFromPath(path string) string { func detectContentTypeFromPath(path string) string {
@@ -91,7 +89,7 @@ func detectContentTypeFromPath(path string) string {
case strings.HasSuffix(path, ".jxl"): case strings.HasSuffix(path, ".jxl"):
return imagerequest.FormatJXL.MIMEType() return imagerequest.FormatJXL.MIMEType()
case strings.HasSuffix(path, ".svg"): case strings.HasSuffix(path, ".svg"):
return contentTypeSVG return imagerequest.FormatSVG.MIMEType()
default: default:
return contentTypeOctetStream return contentTypeOctetStream
} }
+145 -26
View File
@@ -7,6 +7,7 @@ import (
"errors" "errors"
"fmt" "fmt"
"io" "io"
"math"
"sync" "sync"
"time" "time"
@@ -113,10 +114,21 @@ var ErrUnsupportedOutputFormat = errors.New("unsupported output format")
// ErrUnsupportedInputFormat is returned, before the input is decoded, when // ErrUnsupportedInputFormat is returned, before the input is decoded, when
// libvips reads the input's first bytes as another format than its source // libvips reads the input's first bytes as another format than its source
// format, or as none. An SVG whose first bytes pass the first-bytes check for // format, or as none. An SVG whose first bytes pass the first-bytes check for
// AVIF is one example. A source format other than the six pixa reads is also // AVIF is one example. A source format other than the seven pixa reads is
// refused with it. // also refused with it.
var ErrUnsupportedInputFormat = errors.New("unsupported input format") var ErrUnsupportedInputFormat = errors.New("unsupported input format")
// ErrSVGNotOriginalSize is returned when orig is asked of an SVG source at a
// size other than orig. pixa returns an SVG only as it is; a raster format
// gives it a size.
var ErrSVGNotOriginalSize = errors.New(
"an SVG is returned as SVG only at the orig size")
// ErrSVGTooThin is returned, before rendering, when an SVG source has one
// side more than MaxInputDimension times the other. Rendered with its longer
// side at most MaxInputDimension, its shorter side would be under 1 pixel.
var ErrSVGTooThin = errors.New("SVG source too thin to render")
// ErrTooManyImages is returned when MaxConcurrentProcessing images are being // ErrTooManyImages is returned when MaxConcurrentProcessing images are being
// processed and none finishes within ProcessingWaitTimeout. // processed and none finishes within ProcessingWaitTimeout.
var ErrTooManyImages = errors.New("too many images being processed at once") var ErrTooManyImages = errors.New("too many images being processed at once")
@@ -164,7 +176,9 @@ func New(params Params) *ImageProcessor {
// encodes, it returns ctx's error and gives the slot back. govips gives pixa // encodes, it returns ctx's error and gives the slot back. govips gives pixa
// no way to stop libvips once an encode has begun, so an encode keeps its // no way to stop libvips once an encode has begun, so an encode keeps its
// slot until it finishes. An AVIF output of more than MaxAVIFOutputPixels is // slot until it finishes. An AVIF output of more than MaxAVIFOutputPixels is
// refused with ErrAVIFOutputTooLarge before anything is decoded. // refused with ErrAVIFOutputTooLarge before anything is decoded. orig of an
// SVG source is the SVG as it is, since libvips cannot save SVG; any other
// format renders it at the output size.
func (p *ImageProcessor) Process( func (p *ImageProcessor) Process(
ctx context.Context, ctx context.Context,
input io.Reader, input io.Reader,
@@ -190,27 +204,17 @@ func (p *ImageProcessor) Process(
return nil, err return nil, err
} }
// Decode image img, err := loadImage(data, sourceFormat)
img, err := vips.NewImageFromBuffer(data)
if err != nil { if err != nil {
return nil, fmt.Errorf("failed to decode image: %w", err) return nil, err
} }
defer img.Close() defer img.Close()
// Turn the image upright now: encode strips the EXIF orientation tag, origWidth, origHeight := img.Width(), img.Height()
// and sizes below must be worked out on the upright image. outputFormat := resolveOutputFormat(transform.Format, sourceFormat)
err = img.AutoRotate()
if err != nil {
return nil, fmt.Errorf("failed to auto-rotate: %w", err)
}
// Get original dimensions if outputFormat == imagerequest.FormatSVG && sourceFormat == imagerequest.FormatSVG {
origWidth := img.Width() return svgAsItIs(data, transform.Size, origWidth, origHeight)
origHeight := img.Height()
// Validate input dimensions to prevent DoS via decompression bombs
if origWidth > MaxInputDimension || origHeight > MaxInputDimension {
return nil, ErrInputTooLarge
} }
// Determine target dimensions; too large ones are refused before resizing // Determine target dimensions; too large ones are refused before resizing
@@ -220,12 +224,6 @@ func (p *ImageProcessor) Process(
return nil, err return nil, err
} }
// orig is the source's own format; encode refuses an empty format
outputFormat := transform.Format
if outputFormat == imagerequest.FormatOriginal {
outputFormat = sourceFormat
}
// Determine output dimensions; too large ones are refused before // Determine output dimensions; too large ones are refused before
// resizing, which with FitCover already decodes the input // resizing, which with FitCover already decodes the input
outputWidth, outputHeight, err := outputDimensions(origWidth, origHeight, outputWidth, outputHeight, err := outputDimensions(origWidth, origHeight,
@@ -234,6 +232,19 @@ func (p *ImageProcessor) Process(
return nil, err return nil, err
} }
// An SVG is rendered again at about the output size, to stay sharp
if sourceFormat == imagerequest.FormatSVG {
rendered, err := renderSVG(data, origWidth, origHeight,
outputWidth, outputHeight)
if err != nil {
return nil, err
}
defer rendered.Close()
img = rendered
}
// Resize if needed // Resize if needed
if targetWidth != origWidth || targetHeight != origHeight { if targetWidth != origWidth || targetHeight != origHeight {
err := p.resize(img, outputWidth, outputHeight, transform.FitMode) err := p.resize(img, outputWidth, outputHeight, transform.FitMode)
@@ -290,6 +301,110 @@ func targetDimensions(
return width, height, nil return width, height, nil
} }
// resolveOutputFormat returns the format of the output for the requested
// format: sourceFormat for orig, and format itself for any other. encode
// refuses an empty format.
func resolveOutputFormat(
format, sourceFormat imagerequest.Format,
) imagerequest.Format {
if format == imagerequest.FormatOriginal {
return sourceFormat
}
return format
}
// loadImage decodes data and turns the image upright: encode strips the EXIF
// orientation tag, and sizes must be worked out on the upright image. An
// image with a side above MaxInputDimension is ErrInputTooLarge, which guards
// against decompression bombs. An SVG is not checked: only its header has
// been read, and the output checks limit the size it is rendered at.
func loadImage(
data []byte, sourceFormat imagerequest.Format,
) (*vips.ImageRef, error) {
img, err := vips.NewImageFromBuffer(data)
if err != nil {
return nil, fmt.Errorf("failed to decode image: %w", err)
}
err = img.AutoRotate()
if err != nil {
img.Close()
return nil, fmt.Errorf("failed to auto-rotate: %w", err)
}
if sourceFormat != imagerequest.FormatSVG &&
(img.Width() > MaxInputDimension || img.Height() > MaxInputDimension) {
img.Close()
return nil, ErrInputTooLarge
}
return img, nil
}
// svgAsItIs returns the SVG data unchanged, which is orig of an SVG source
// at the orig size. width x height is the SVG's own size. Any other size is
// ErrSVGNotOriginalSize.
func svgAsItIs(
data []byte, size imagerequest.Size, width, height int,
) (*Result, error) {
if !size.OriginalSize() {
return nil, fmt.Errorf("%w: asked for %dx%d", ErrSVGNotOriginalSize,
size.Width, size.Height)
}
return &Result{
Content: io.NopCloser(bytes.NewReader(data)),
ContentLength: int64(len(data)),
ContentType: imagerequest.FormatSVG.MIMEType(),
Width: width,
Height: height,
InputWidth: width,
InputHeight: height,
}, nil
}
// svgRenderSize returns the least size in the proportions of width x height,
// an SVG's own size, that covers outputWidth x outputHeight, so that resize
// hardly changes it. Neither side is above MaxInputDimension.
func svgRenderSize(width, height, outputWidth, outputHeight int) (int, int) {
scale := min(
max(float64(outputWidth)/float64(width),
float64(outputHeight)/float64(height)),
float64(MaxInputDimension)/float64(max(width, height)))
return int(math.Round(float64(width) * scale)),
int(math.Round(float64(height) * scale))
}
// renderSVG renders the SVG data, whose own size is width x height, at
// svgRenderSize for outputWidth x outputHeight. An SVG with one side more
// than MaxInputDimension times the other is ErrSVGTooThin. libvips'
// thumbnail has librsvg render the SVG at that size directly, at any scale:
// govips takes a density only in whole dots per inch, which cannot render
// an SVG below 1/72 of its own size. Given the SVG as bytes with no file
// name, librsvg loads no file or URL the SVG refers to.
func renderSVG(
data []byte, width, height, outputWidth, outputHeight int,
) (*vips.ImageRef, error) {
if max(width, height) > MaxInputDimension*min(width, height) {
return nil, fmt.Errorf("%w: %dx%d", ErrSVGTooThin, width, height)
}
renderWidth, renderHeight := svgRenderSize(width, height,
outputWidth, outputHeight)
img, err := vips.NewThumbnailWithSizeFromBuffer(data,
renderWidth, renderHeight, vips.InterestingNone, vips.SizeBoth)
if err != nil {
return nil, fmt.Errorf("failed to render SVG: %w", err)
}
return img, nil
}
// WaitForProcessing waits until no image is being processed, or until ctx // WaitForProcessing waits until no image is being processed, or until ctx
// ends, and returns how many images were still being processed then. It // ends, and returns how many images were still being processed then. It
// waits by taking each slot in processingSemaphore as it frees up until it // waits by taking each slot in processingSemaphore as it frees up until it
@@ -397,6 +512,8 @@ func vipsImageType(format imagerequest.Format) vips.ImageType {
return vips.ImageTypeAVIF return vips.ImageTypeAVIF
case imagerequest.FormatJXL: case imagerequest.FormatJXL:
return vips.ImageTypeJXL return vips.ImageTypeJXL
case imagerequest.FormatSVG:
return vips.ImageTypeSVG
case imagerequest.FormatOriginal, imagerequest.FormatAuto: case imagerequest.FormatOriginal, imagerequest.FormatAuto:
return vips.ImageTypeUnknown return vips.ImageTypeUnknown
default: default:
@@ -575,7 +692,9 @@ func (p *ImageProcessor) encode(
case imagerequest.FormatJXL: case imagerequest.FormatJXL:
return exportJXL(img, quality) return exportJXL(img, quality)
case imagerequest.FormatOriginal, imagerequest.FormatAuto: // libvips cannot save SVG; Process returns an SVG source as it is
case imagerequest.FormatSVG, imagerequest.FormatOriginal,
imagerequest.FormatAuto:
return nil, fmt.Errorf("%w: %s", ErrUnsupportedOutputFormat, format) return nil, fmt.Errorf("%w: %s", ErrUnsupportedOutputFormat, format)
default: default:
@@ -51,7 +51,7 @@ func TestImageProcessor_LoadsEachSourceFormat(t *testing.T) {
} }
// TestImageProcessor_NotASourceFormatRefused verifies that a source format // TestImageProcessor_NotASourceFormatRefused verifies that a source format
// other than the six, such as orig, auto or none, is refused with // other than the seven, such as orig, auto or none, is refused with
// ErrUnsupportedInputFormat. The input is text, which libvips reads as no // ErrUnsupportedInputFormat. The input is text, which libvips reads as no
// format. // format.
func TestImageProcessor_NotASourceFormatRefused(t *testing.T) { func TestImageProcessor_NotASourceFormatRefused(t *testing.T) {
@@ -0,0 +1,344 @@
package imageprocessor
import (
"bytes"
"errors"
"io"
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"sync/atomic"
"testing"
"github.com/davidbyttow/govips/v2/vips"
"sneak.berlin/go/pixa/internal/imagerequest"
)
// testSVG is a 16x12 SVG, black on its left half and white on its right.
const testSVG = `<svg xmlns="http://www.w3.org/2000/svg" width="16" height="12">` +
`<rect width="16" height="12" fill="white"/>` +
`<rect width="8" height="12" fill="black"/></svg>`
// newSVGTestProcessor returns a processor for the SVG tests.
func newSVGTestProcessor() *ImageProcessor {
return New(Params{MaxInputBytes: 1 << 20, MaxConcurrentProcessing: 1})
}
// processSVG processes svg, as an SVG source, with transform.
func processSVG(
t *testing.T, svg string, transform imagerequest.Transform,
) (*Result, []byte, error) {
t.Helper()
result, err := newSVGTestProcessor().Process(t.Context(),
bytes.NewReader([]byte(svg)), imagerequest.FormatSVG, &transform)
if err != nil {
return nil, nil, err
}
output, err := io.ReadAll(result.Content)
if err != nil {
t.Fatalf("reading the output: %v", err)
}
return result, output, nil
}
// TestImageProcessor_SVGOrigIsTheSVG checks that orig of an SVG at the orig
// size is the SVG itself, never another format, and that orig at another
// size is ErrSVGNotOriginalSize.
func TestImageProcessor_SVGOrigIsTheSVG(t *testing.T) {
t.Parallel()
result, output, err := processSVG(t, testSVG,
imagerequest.Transform{Format: imagerequest.FormatOriginal})
if err != nil {
t.Fatalf("orig of the SVG: %v", err)
}
t.Logf("orig of the SVG: %s, %dx%d", result.ContentType, result.Width,
result.Height)
if result.ContentType != "image/svg+xml" || string(output) != testSVG {
t.Errorf("orig of the SVG = %s %q, want image/svg+xml %q",
result.ContentType, output, testSVG)
}
_, _, err = processSVG(t, testSVG, imagerequest.Transform{
Size: imagerequest.Size{Width: 32, Height: 24},
Format: imagerequest.FormatOriginal,
})
if !errors.Is(err, ErrSVGNotOriginalSize) {
t.Errorf("orig of the SVG at 32x24: error = %v, want %v", err,
ErrSVGNotOriginalSize)
}
}
// TestImageProcessor_RendersSVGToEachRasterFormat checks that an SVG is
// rendered to each raster format at the requested size.
func TestImageProcessor_RendersSVGToEachRasterFormat(t *testing.T) {
t.Parallel()
for _, format := range []imagerequest.Format{
imagerequest.FormatJPEG, imagerequest.FormatPNG, imagerequest.FormatGIF,
imagerequest.FormatWebP, imagerequest.FormatAVIF, imagerequest.FormatJXL,
} {
result, output, err := processSVG(t, testSVG, imagerequest.Transform{
Size: imagerequest.Size{Width: 40, Height: 30},
Format: format,
})
if err != nil {
t.Errorf("SVG as %s: %v", format, err)
continue
}
t.Logf("SVG as %s: %s, %dx%d, %d bytes", format, result.ContentType,
result.Width, result.Height, len(output))
if result.ContentType != format.MIMEType() ||
vips.DetermineImageType(output) != vipsImageType(format) ||
result.Width != 40 || result.Height != 30 {
t.Errorf("SVG as %s = %s, %dx%d; want %s, 40x30", format,
result.ContentType, result.Width, result.Height, format.MIMEType())
}
}
}
// TestImageProcessor_RendersSVGSharpWhenEnlarged checks that a 16x12 SVG
// asked for at 1600x1200 is rendered at that size, not rendered at 16x12 and
// enlarged: 10 pixels either side of the edge between its black and white
// halves are black and white, where an enlarged image is grey there.
func TestImageProcessor_RendersSVGSharpWhenEnlarged(t *testing.T) {
t.Parallel()
_, output, err := processSVG(t, testSVG, imagerequest.Transform{
Size: imagerequest.Size{Width: 1600, Height: 1200},
Format: imagerequest.FormatPNG,
})
if err != nil {
t.Fatalf("SVG as PNG at 1600x1200: %v", err)
}
img, err := vips.NewImageFromBuffer(output)
if err != nil {
t.Fatalf("decoding the PNG: %v", err)
}
t.Cleanup(img.Close)
for x, want := range map[int]float64{790: 0, 810: 255} {
point, err := img.GetPoint(x, 600)
if err != nil {
t.Fatalf("GetPoint(%d, 600): %v", x, err)
}
t.Logf("pixel at %d,600: %v", x, point)
if point[0] != want {
t.Errorf("pixel at %d,600 = %v, want %v", x, point[0], want)
}
}
}
// TestImageProcessor_SVGFetchesNothing checks that rendering an SVG draws no
// image it refers to: one at a URL on a local server, by href or xlink:href,
// or a local file that exists. Each SVG renders without error, white where
// the image would be. An SVG that uses an external entity at that server is
// a separate case. No SVG makes a request to the server.
func TestImageProcessor_SVGFetchesNothing(t *testing.T) {
t.Parallel()
var requests atomic.Int64
// Blue is 128 in every pixel of this PNG.
png := createTestPNG(t, 4, 4)
server := httptest.NewServer(http.HandlerFunc(
func(w http.ResponseWriter, r *http.Request) {
requests.Add(1)
t.Logf("request for %s", r.URL)
w.Header().Set("Content-Type", "image/png")
_, _ = w.Write(png)
}))
t.Cleanup(server.Close)
pngPath := filepath.Join(t.TempDir(), "image.png")
err := os.WriteFile(pngPath, png, 0o600)
if err != nil {
t.Fatalf("writing the PNG file: %v", err)
}
for _, reference := range []string{
`href="` + server.URL + `/image.png"`,
`xlink:href="` + server.URL + `/xlink.png"`,
`href="file://` + pngPath + `"`,
} {
svg := `<svg xmlns="http://www.w3.org/2000/svg" ` +
`xmlns:xlink="http://www.w3.org/1999/xlink" width="16" height="12">` +
`<rect width="16" height="12" fill="white"/>` +
`<image ` + reference + ` width="16" height="12"/></svg>`
_, output, err := processSVG(t, svg, imagerequest.Transform{
Format: imagerequest.FormatPNG,
})
if err != nil {
t.Errorf("SVG with image %s as PNG: %v", reference, err)
continue
}
img, err := vips.NewImageFromBuffer(output)
if err != nil {
t.Fatalf("decoding the PNG: %v", err)
}
point, err := img.GetPoint(8, 6)
img.Close()
if err != nil {
t.Fatalf("GetPoint(8, 6): %v", err)
}
t.Logf("SVG with image %s: pixel at 8,6 = %v", reference, point)
if point[2] != 255 {
t.Errorf("SVG with image %s: blue at 8,6 = %v, want 255: the "+
"image is drawn", reference, point[2])
}
}
svg := `<?xml version="1.0"?>` +
`<!DOCTYPE svg [<!ENTITY outside SYSTEM "` + server.URL + `/entity">]>` +
`<svg xmlns="http://www.w3.org/2000/svg" width="16" height="12">` +
`<text y="10">&outside;</text></svg>`
// librsvg loads no external entity, so it refuses this SVG: outside is
// not defined.
_, _, err = processSVG(t, svg, imagerequest.Transform{
Format: imagerequest.FormatPNG,
})
t.Logf("SVG with an external entity as PNG: error = %v", err)
if requests.Load() != 0 {
t.Errorf("rendering the SVGs made %d requests, want none",
requests.Load())
}
}
// TestImageProcessor_RendersSVGLargerThanMaxInputDimension checks that an
// SVG whose own size is above MaxInputDimension is limited only by the
// output checks: it is rendered at a raster size, orig of it is the SVG
// itself, and a raster format at its own size is ErrOutputTooLarge. The
// 1000000x1000000 one is rendered at a ten-thousandth of its own size.
func TestImageProcessor_RendersSVGLargerThanMaxInputDimension(t *testing.T) {
t.Parallel()
for _, svg := range []string{
`<svg xmlns="http://www.w3.org/2000/svg" width="10000" height="10">` +
`<rect width="10000" height="10"/></svg>`,
`<svg xmlns="http://www.w3.org/2000/svg" width="1000000" ` +
`height="1000000"><rect width="1000000" height="1000000"/></svg>`,
} {
result, _, err := processSVG(t, svg, imagerequest.Transform{
Size: imagerequest.Size{Width: 100, Height: 100},
Format: imagerequest.FormatPNG,
})
if err != nil {
t.Errorf("%s as PNG at 100x100: %v", svg, err)
} else {
t.Logf("%s as PNG at 100x100: %dx%d", svg, result.Width, result.Height)
if result.Width != 100 || result.Height != 100 {
t.Errorf("%s as PNG at 100x100 = %dx%d, want 100x100", svg,
result.Width, result.Height)
}
}
result, output, err := processSVG(t, svg,
imagerequest.Transform{Format: imagerequest.FormatOriginal})
if err != nil || result.ContentType != imagerequest.FormatSVG.MIMEType() ||
string(output) != svg {
t.Errorf("orig of %s: error = %v, want the SVG itself", svg, err)
}
_, _, err = processSVG(t, svg,
imagerequest.Transform{Format: imagerequest.FormatPNG})
if !errors.Is(err, ErrOutputTooLarge) {
t.Errorf("%s as PNG at its own size: error = %v, want %v", svg,
err, ErrOutputTooLarge)
}
}
}
// TestImageProcessor_RefusesSVGTooThin checks that an SVG with one side more
// than MaxInputDimension times the other is ErrSVGTooThin at a raster size,
// that one at exactly that proportion is rendered, and that orig of either
// is the SVG itself.
func TestImageProcessor_RefusesSVGTooThin(t *testing.T) {
t.Parallel()
tooThin := `<svg xmlns="http://www.w3.org/2000/svg" width="81930" ` +
`height="10"><rect width="81930" height="10"/></svg>`
atTheLimit := `<svg xmlns="http://www.w3.org/2000/svg" width="81920" ` +
`height="10"><rect width="81920" height="10"/></svg>`
size := imagerequest.Size{Width: 100, Height: 100}
_, _, err := processSVG(t, tooThin,
imagerequest.Transform{Size: size, Format: imagerequest.FormatPNG})
if !errors.Is(err, ErrSVGTooThin) {
t.Errorf("81930x10 SVG as PNG at 100x100: error = %v, want %v", err,
ErrSVGTooThin)
}
result, _, err := processSVG(t, atTheLimit,
imagerequest.Transform{Size: size, Format: imagerequest.FormatPNG})
if err != nil {
t.Errorf("81920x10 SVG as PNG at 100x100: %v", err)
} else if result.Width != 100 || result.Height != 100 {
t.Errorf("81920x10 SVG as PNG at 100x100 = %dx%d, want 100x100",
result.Width, result.Height)
}
for _, svg := range []string{tooThin, atTheLimit} {
result, output, err := processSVG(t, svg,
imagerequest.Transform{Format: imagerequest.FormatOriginal})
if err != nil || result.ContentType != imagerequest.FormatSVG.MIMEType() ||
string(output) != svg {
t.Errorf("orig of %s: error = %v, want the SVG itself", svg, err)
}
}
}
// TestSVGRenderSize checks the size at which an SVG is rendered to cover an
// output size, and that it is held to MaxInputDimension.
func TestSVGRenderSize(t *testing.T) {
t.Parallel()
tests := []struct {
name string
width, height int
outputWidth, outputHeight int
wantWidth, wantHeight int
}{
{"own size", 16, 12, 16, 12, 16, 12},
{"100 times larger", 16, 12, 1600, 1200, 1600, 1200},
{"wider than the SVG's proportions", 16, 12, 160, 12, 160, 120},
{"10 times smaller", 100, 100, 10, 10, 10, 10},
{"held to MaxInputDimension", 8192, 1, 8192, 8192, 8192, 1},
{"held to MaxInputDimension when enlarged", 1024, 1, 8192, 8192, 8192, 8},
{"far above MaxInputDimension", 1000000, 1000000, 100, 100, 100, 100},
}
for _, tt := range tests {
gotWidth, gotHeight := svgRenderSize(tt.width, tt.height,
tt.outputWidth, tt.outputHeight)
if gotWidth != tt.wantWidth || gotHeight != tt.wantHeight {
t.Errorf("%s: svgRenderSize(%d, %d, %d, %d) = %dx%d, want %dx%d",
tt.name, tt.width, tt.height, tt.outputWidth, tt.outputHeight,
gotWidth, gotHeight, tt.wantWidth, tt.wantHeight)
}
}
}
+5 -1
View File
@@ -14,7 +14,8 @@ import (
var ErrInvalidFitMode = errors.New("invalid fit mode") var ErrInvalidFitMode = errors.New("invalid fit mode")
// Format is an image format. pixa reads and writes the formats from JPEG to // Format is an image format. pixa reads and writes the formats from JPEG to
// GIF below. In a request, orig and auto stand for one of them. // GIF below. It reads SVG and writes it only as orig of an SVG source. In a
// request, orig and auto stand for another format.
type Format string type Format string
// Supported image formats. // Supported image formats.
@@ -26,6 +27,7 @@ const (
FormatAVIF Format = "avif" FormatAVIF Format = "avif"
FormatJXL Format = "jxl" FormatJXL Format = "jxl"
FormatGIF Format = "gif" FormatGIF Format = "gif"
FormatSVG Format = "svg"
// FormatAuto stands for JPEG XL, AVIF, WebP or JPEG, chosen for each // FormatAuto stands for JPEG XL, AVIF, WebP or JPEG, chosen for each
// request from its Accept header once the URL's signature or token has // request from its Accept header once the URL's signature or token has
@@ -50,6 +52,8 @@ func (f Format) MIMEType() string {
return "image/jxl" return "image/jxl"
case FormatGIF: case FormatGIF:
return "image/gif" return "image/gif"
case FormatSVG:
return "image/svg+xml"
case FormatOriginal, FormatAuto: case FormatOriginal, FormatAuto:
return "" return ""
default: default:
+9 -7
View File
@@ -735,16 +735,18 @@ func (c *Cache) finishOpeningVariant(
}, nil }, nil
} }
// readFormat reads the first bytes of a source or variant file, without // readFormat reads up to magic.MaxMagicBytes first bytes of a source or
// moving its offset, and returns the format they show. Sources and variants // variant file, without moving its offset, and returns the format they show.
// are in the same six formats. // Sources and variants are in the same seven formats; a variant is SVG only
// as orig of an SVG source. A file shorter than magic.MinMagicBytes is
// io.EOF.
func readFormat(content io.ReaderAt) (imagerequest.Format, error) { func readFormat(content io.ReaderAt) (imagerequest.Format, error) {
firstBytes := make([]byte, magic.MinMagicBytes) firstBytes := make([]byte, magic.MaxMagicBytes)
_, err := content.ReadAt(firstBytes, 0) n, err := content.ReadAt(firstBytes, 0)
if err != nil { if err != nil && (!errors.Is(err, io.EOF) || n < magic.MinMagicBytes) {
return "", fmt.Errorf("failed to read the first bytes: %w", err) return "", fmt.Errorf("failed to read the first bytes: %w", err)
} }
return magic.DetectFormat(firstBytes) return magic.DetectFormat(firstBytes[:n])
} }
@@ -98,8 +98,6 @@ func TestVariant_RefusesFileOfNoOutputFormat(t *testing.T) {
wantErr error wantErr error
}{ }{
{"text", []byte("this is text, not an image"), magic.ErrUnknownFormat}, {"text", []byte("this is text, not an image"), magic.ErrUnknownFormat},
{"SVG", []byte(`<svg xmlns="http://www.w3.org/2000/svg"></svg>`),
magic.ErrUnknownFormat},
{"JPEG cut to 10 bytes", jpegImage[:10], io.EOF}, {"JPEG cut to 10 bytes", jpegImage[:10], io.EOF},
{"empty", nil, io.EOF}, {"empty", nil, io.EOF},
} }
@@ -216,8 +216,6 @@ func TestSource_RefusesFileOfNoSourceFormat(t *testing.T) {
wantErr error wantErr error
}{ }{
{"text", []byte("this is text, not an image"), magic.ErrUnknownFormat}, {"text", []byte("this is text, not an image"), magic.ErrUnknownFormat},
{"SVG", []byte(`<svg xmlns="http://www.w3.org/2000/svg"></svg>`),
magic.ErrUnknownFormat},
{"JPEG cut to 10 bytes", jpegImage[:10], io.EOF}, {"JPEG cut to 10 bytes", jpegImage[:10], io.EOF},
{"empty file", nil, io.EOF}, {"empty file", nil, io.EOF},
} }
+54
View File
@@ -0,0 +1,54 @@
package imgcache
import (
"testing"
"sneak.berlin/go/pixa/internal/imagerequest"
)
// testSVG is an SVG with a comment before its root element, so that its
// first magic.MinMagicBytes bytes do not show its format.
const testSVG = `<!-- a drawing --><svg xmlns="http://www.w3.org/2000/svg" ` +
`width="4" height="4"/>`
// TestSourceAndVariant_ReadSVG checks that a cached SVG source is read as
// SVG, and a cached SVG variant (orig of an SVG source) as image/svg+xml.
func TestSourceAndVariant_ReadSVG(t *testing.T) {
t.Parallel()
cache, _ := newEvictionTestCache(t, 1<<30)
storeEvictionTestSource(t, cache, testHostCDN, "/drawing.svg", []byte(testSVG))
reader, _, format, err := cache.Source(t.Context(),
imagerequest.Source{Host: testHostCDN, Path: "/drawing.svg"})
t.Logf("Source() format = %q, error = %v", format, err)
if err != nil || format != imagerequest.FormatSVG {
t.Errorf("Source() = format %q, error %v; want %q", format, err,
imagerequest.FormatSVG)
}
if reader != nil {
_ = reader.Close()
}
key := imagerequest.VariantKey("aabbccdd3001")
err = cache.variants.Write(string(key), []byte(testSVG))
if err != nil {
t.Fatalf("storing the variant file: %v", err)
}
variant, err := cache.Variant(key)
t.Logf("Variant() content type = %q, error = %v", variant.ContentType, err)
if err != nil || variant.ContentType != "image/svg+xml" {
t.Errorf("Variant() = content type %q, error %v; want image/svg+xml",
variant.ContentType, err)
}
if variant.Content != nil {
_ = variant.Content.Close()
}
}
+27 -1
View File
@@ -20,6 +20,10 @@ var (
// MinMagicBytes is the minimum number of bytes needed to detect format. // MinMagicBytes is the minimum number of bytes needed to detect format.
const MinMagicBytes = 12 const MinMagicBytes = 12
// MaxMagicBytes is the most bytes DetectFormat looks at. An SVG's root
// element must start within them, as libvips requires.
const MaxMagicBytes = 1024
// Magic byte signatures for supported formats. // Magic byte signatures for supported formats.
// These are effectively constants but Go doesn't support const slices. // These are effectively constants but Go doesn't support const slices.
// //
@@ -39,6 +43,12 @@ var (
} }
) )
// utf8BOM is the UTF-8 byte order mark, which a text file such as an SVG may
// start with.
//
//nolint:gochecknoglobals // immutable lookup data
var utf8BOM = []byte{0xEF, 0xBB, 0xBF}
// WebP identifier appears at offset 8 after RIFF header. // WebP identifier appears at offset 8 after RIFF header.
// //
//nolint:gochecknoglobals // immutable lookup data //nolint:gochecknoglobals // immutable lookup data
@@ -53,7 +63,7 @@ var (
) )
// DetectFormat detects the image format from magic bytes. Data in none of // DetectFormat detects the image format from magic bytes. Data in none of
// the formats pixa reads, SVG included, is ErrUnknownFormat. // the formats pixa reads is ErrUnknownFormat.
func DetectFormat(data []byte) (imagerequest.Format, error) { func DetectFormat(data []byte) (imagerequest.Format, error) {
if len(data) < MinMagicBytes { if len(data) < MinMagicBytes {
return "", ErrNotEnoughData return "", ErrNotEnoughData
@@ -96,9 +106,25 @@ func DetectFormat(data []byte) (imagerequest.Format, error) {
return imagerequest.FormatJXL, nil return imagerequest.FormatJXL, nil
} }
if isSVG(data) {
return imagerequest.FormatSVG, nil
}
return "", ErrUnknownFormat return "", ErrUnknownFormat
} }
// isSVG reports whether data starts as an SVG does: with a "<", after an
// optional UTF-8 byte order mark and white space, and with "<svg" within the
// first MaxMagicBytes bytes. Before the root element an SVG may have an XML
// declaration, comments and a DOCTYPE.
func isSVG(data []byte) bool {
data = data[:min(len(data), MaxMagicBytes)]
text := bytes.TrimLeft(bytes.TrimPrefix(data, utf8BOM), " \t\r\n")
return bytes.HasPrefix(text, []byte("<")) &&
bytes.Contains(text, []byte("<svg"))
}
// ValidateMagicBytes detects the format of data from its magic bytes, and // ValidateMagicBytes detects the format of data from its magic bytes, and
// returns it when it is the format the declared MIME type names. Data in // returns it when it is the format the declared MIME type names. Data in
// another format is ErrMagicByteMismatch. // another format is ErrMagicByteMismatch.
+3 -3
View File
@@ -47,9 +47,9 @@ func TestDetectFormat(t *testing.T) {
{name: "AVIF", data: avif, wantFormat: imagerequest.FormatAVIF}, {name: "AVIF", data: avif, wantFormat: imagerequest.FormatAVIF},
{name: "AVIF sequence", data: avis, wantFormat: imagerequest.FormatAVIF}, {name: "AVIF sequence", data: avis, wantFormat: imagerequest.FormatAVIF},
{ {
name: "SVG", name: "SVG",
data: []byte(`<svg xmlns="http://www.w3.org/2000/svg"></svg>`), data: []byte(`<svg xmlns="http://www.w3.org/2000/svg"></svg>`),
wantErr: ErrUnknownFormat, wantFormat: imagerequest.FormatSVG,
}, },
{ {
name: "unknown format", name: "unknown format",
+85
View File
@@ -0,0 +1,85 @@
package magic
import (
"errors"
"strings"
"testing"
"sneak.berlin/go/pixa/internal/imagerequest"
)
// TestDetectFormat_SVG checks that DetectFormat names SVG text with what may
// come before the root element, and does not name as SVG text that does not
// start with "<" or has no "<svg" within its first MaxMagicBytes bytes.
func TestDetectFormat_SVG(t *testing.T) {
t.Parallel()
const root = `<svg xmlns="http://www.w3.org/2000/svg" width="4" height="4"/>`
tests := []struct {
name string
data string
wantErr error
}{
{name: "root element only", data: root},
{name: "XML declaration", data: `<?xml version="1.0"?>` + "\n" + root},
{name: "byte order mark and white space", data: "\xEF\xBB\xBF \n\t" + root},
{name: "comment", data: "<!-- a drawing -->" + root},
{
name: "DOCTYPE",
data: `<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" ` +
`"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">` + root,
},
{
name: "root element after the first bytes",
data: "<!--" + strings.Repeat(" ", MaxMagicBytes) + "-->" + root,
wantErr: ErrUnknownFormat,
},
{name: "text naming <svg", data: "draw it with " + root, wantErr: ErrUnknownFormat},
{name: "HTML", data: "<!DOCTYPE html><html></html>", wantErr: ErrUnknownFormat},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
got, err := DetectFormat([]byte(tt.data))
t.Logf("DetectFormat() = %q, %v", got, err)
if tt.wantErr != nil {
if !errors.Is(err, tt.wantErr) {
t.Errorf("DetectFormat() error = %v, want %v", err, tt.wantErr)
}
return
}
if err != nil || got != imagerequest.FormatSVG {
t.Errorf("DetectFormat() = %q, %v; want %q", got, err,
imagerequest.FormatSVG)
}
})
}
}
// TestValidateMagicBytes_SVG checks that SVG data is SVG for the
// Content-Type image/svg+xml, with a charset too, and a mismatch for another.
func TestValidateMagicBytes_SVG(t *testing.T) {
t.Parallel()
data := []byte(`<svg xmlns="http://www.w3.org/2000/svg" width="4" height="4"/>`)
for _, declared := range []string{"image/svg+xml", "image/svg+xml; charset=utf-8"} {
got, err := ValidateMagicBytes(data, declared)
if err != nil || got != imagerequest.FormatSVG {
t.Errorf("ValidateMagicBytes(%q) = %q, %v; want %q", declared, got,
err, imagerequest.FormatSVG)
}
}
_, err := ValidateMagicBytes(data, "image/png")
if !errors.Is(err, ErrMagicByteMismatch) {
t.Errorf("ValidateMagicBytes(image/png) error = %v, want %v", err,
ErrMagicByteMismatch)
}
}
+94 -18
View File
@@ -4,38 +4,114 @@ import (
"io" "io"
"net/http" "net/http"
"net/http/httptest" "net/http/httptest"
"strings"
"sync/atomic"
"testing" "testing"
"sneak.berlin/go/pixa/internal/imageproxy"
"sneak.berlin/go/pixa/internal/imagerequest"
"sneak.berlin/go/pixa/internal/magic"
) )
// TestImageProxyRefusesSVG requests an image the upstream answers as SVG, // testSVG is the SVG the upstream serves in the SVG tests below.
// in its own format (orig) and resized as PNG, through pixa's router, const testSVG = `<svg xmlns="http://www.w3.org/2000/svg" width="64" height="48">` +
// upstream fetcher and image processor. pixa does not accept SVG as a `<rect width="64" height="48"/></svg>`
// source, so each request is answered 415, never with an image.
func TestImageProxyRefusesSVG(t *testing.T) { // startSVGUpstream starts an upstream that answers every request with
t.Parallel() // testSVG as image/svg+xml, and returns it with its count of requests.
func startSVGUpstream(t *testing.T) (*httptest.Server, *atomic.Int64) {
t.Helper()
var requests atomic.Int64
upstream := httptest.NewServer(http.HandlerFunc( upstream := httptest.NewServer(http.HandlerFunc(
func(w http.ResponseWriter, _ *http.Request) { func(w http.ResponseWriter, _ *http.Request) {
requests.Add(1)
w.Header().Set("Content-Type", "image/svg+xml") w.Header().Set("Content-Type", "image/svg+xml")
_, _ = io.WriteString(w, `<svg xmlns="http://www.w3.org/2000/svg" `+ _, _ = io.WriteString(w, testSVG)
`width="64" height="48"><rect width="64" height="48"/></svg>`)
})) }))
t.Cleanup(upstream.Close) t.Cleanup(upstream.Close)
return upstream, &requests
}
// TestImageProxyServesSVGAsItIs requests an SVG source in its own format
// (orig) at the orig size through pixa's router, upstream fetcher, image
// processor and cache, twice. Both answers are the SVG itself, never another
// format, the second from the cache; the upstream is asked once. Scripts in
// the SVG cannot run, as the Content-Security-Policy allows no inline
// script. orig at another size is refused with 400.
func TestImageProxyServesSVGAsItIs(t *testing.T) {
t.Parallel()
upstream, requests := startSVGUpstream(t)
s, _, _ := startImageProxy(t, upstream)
target := "/v1/image/" + upstreamHost + "/drawing.svg/orig.orig"
for _, wantCache := range []imageproxy.CacheStatus{
imageproxy.CacheMiss, imageproxy.CacheHit,
} {
rec := getImage(t, s, target)
if rec.Header().Get("Content-Type") != "image/svg+xml" ||
rec.Body.String() != testSVG {
t.Errorf("GET %s: %s %q, want image/svg+xml %q", target,
rec.Header().Get("Content-Type"), rec.Body.String(), testSVG)
}
if got := rec.Header().Get("X-Pixa-Cache"); got != string(wantCache) {
t.Errorf("GET %s: X-Pixa-Cache = %q, want %q", target, got, wantCache)
}
policy := rec.Header().Get("Content-Security-Policy")
if !strings.Contains(policy, "script-src 'self'") ||
strings.Contains(policy, "unsafe-inline") {
t.Errorf("GET %s: Content-Security-Policy = %q, want script-src "+
"'self' and no unsafe-inline", target, policy)
}
}
if requests.Load() != 1 {
t.Errorf("upstream requests = %d, want 1", requests.Load())
}
resized := "/v1/image/" + upstreamHost + "/drawing.svg/32x24.orig"
rec := httptest.NewRecorder()
s.ServeHTTP(rec, httptest.NewRequestWithContext(
t.Context(), http.MethodGet, resized, nil))
t.Logf("GET %s: %d %s", resized, rec.Code, rec.Body.String())
if rec.Code != http.StatusBadRequest {
t.Errorf("GET %s: %d %s, want 400", resized, rec.Code, rec.Body.String())
}
}
// TestImageProxyRendersSVG requests an SVG source as each raster format
// through pixa's router, upstream fetcher and image processor. Each answer
// is an image in that format.
func TestImageProxyRendersSVG(t *testing.T) {
t.Parallel()
upstream, _ := startSVGUpstream(t)
s, _, _ := startImageProxy(t, upstream) s, _, _ := startImageProxy(t, upstream)
for _, sizeFormat := range []string{"64x48.orig", "32x24.png"} { for _, format := range []imagerequest.Format{
target := "/v1/image/" + upstreamHost + "/drawing.svg/" + sizeFormat imagerequest.FormatJPEG, imagerequest.FormatPNG, imagerequest.FormatGIF,
imagerequest.FormatWebP, imagerequest.FormatAVIF, imagerequest.FormatJXL,
} {
target := "/v1/image/" + upstreamHost + "/drawing.svg/32x24." +
string(format)
rec := httptest.NewRecorder() rec := getImage(t, s, target)
s.ServeHTTP(rec, httptest.NewRequestWithContext( detected, err := magic.DetectFormat(rec.Body.Bytes())
t.Context(), http.MethodGet, target, nil)) t.Logf("GET %s: %s, body in %q (%v)", target,
t.Logf("GET %s: %d, Content-Type %s, body %s", rec.Header().Get("Content-Type"), detected, err)
target, rec.Code, rec.Header().Get("Content-Type"), rec.Body.String())
if rec.Code != http.StatusUnsupportedMediaType { if rec.Header().Get("Content-Type") != format.MIMEType() ||
t.Errorf("GET %s status = %d, want %d", err != nil || detected != format {
target, rec.Code, http.StatusUnsupportedMediaType) t.Errorf("GET %s: %s, body in %q (%v); want %s", target,
rec.Header().Get("Content-Type"), detected, err, format.MIMEType())
} }
} }
} }