Compare commits
1
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
6ff59fd5f6 |
@@ -190,12 +190,9 @@ path under `/v1/` answers 200, in maintenance mode too.
|
|||||||
the cache and the upstream fetch; 403 when the upstream host, or a host it
|
the cache and the upstream fetch; 403 when the upstream host, or a host it
|
||||||
redirects to, is `localhost`, ends in `.localhost` or `.local`, or has an
|
redirects to, is `localhost`, ends in `.localhost` or `.local`, or has an
|
||||||
address in a blocked network (see `blocked_networks`); 429 over
|
address in a blocked network (see `blocked_networks`); 429 over
|
||||||
`image_requests_per_minute` (below); 415 when the upstream answers with a
|
`image_requests_per_minute` (below); 502 when the upstream answered with an
|
||||||
`Content-Type` that is not one of the source formats (below), or with an image
|
error status, and for 5 minutes after that for the same source URL; 503 when
|
||||||
whose first bytes libvips reads as another format; 502 when the upstream
|
pixa is busy or in maintenance mode; 500 for any other failure.
|
||||||
answered with an error status, and for 5 minutes after that for the same
|
|
||||||
source URL; 503 when pixa is busy or in maintenance mode; 500 for any other
|
|
||||||
failure.
|
|
||||||
- `GET` or `HEAD` `/v1/e/<token>/<name>` — an image through an encrypted URL
|
- `GET` or `HEAD` `/v1/e/<token>/<name>` — an image through an encrypted URL
|
||||||
(see Encrypted URLs). Needs: nothing but the URL. Answers: 200; 304 when
|
(see Encrypted URLs). Needs: nothing but the URL. Answers: 200; 304 when
|
||||||
`If-None-Match` matches the image's `ETag`; 400 for a token that does not
|
`If-None-Match` matches the image's `ETag`; 400 for a token that does not
|
||||||
@@ -203,7 +200,7 @@ path under `/v1/` answers 200, in maintenance mode too.
|
|||||||
expired; 504 when the upstream has not sent its response headers within
|
expired; 504 when the upstream has not sent its response headers within
|
||||||
`upstream_fetch_timeout`, but 500 when that time runs out while the image
|
`upstream_fetch_timeout`, but 500 when that time runs out while the image
|
||||||
itself is still arriving; 400 for an `Accept` header that is not valid, and
|
itself is still arriving; 400 for an `Accept` header that is not valid, and
|
||||||
406, 403, 415, 429, 502, 503 and 500, as for `/v1/image/`.
|
406, 403, 429, 502, 503 and 500, as for `/v1/image/`.
|
||||||
- `GET /robots.txt` — asks every crawler to stay away (`Disallow: /`). Needs:
|
- `GET /robots.txt` — asks every crawler to stay away (`Disallow: /`). Needs:
|
||||||
nothing. Answers: 200.
|
nothing. Answers: 200.
|
||||||
- `GET /.well-known/healthcheck.json` — JSON with `status` (`ok`), `now`,
|
- `GET /.well-known/healthcheck.json` — JSON with `status` (`ok`), `now`,
|
||||||
@@ -263,11 +260,9 @@ once, is refused with 400.
|
|||||||
both optional (values under Signature Specification). Both are part of what is
|
both optional (values under Signature Specification). Both are part of what is
|
||||||
cached, so each value of either is a separate cached image.
|
cached, so each value of either is a separate cached image.
|
||||||
|
|
||||||
The source image may be JPEG, PNG, GIF, WebP, AVIF or JPEG XL. The upstream's
|
The source image may be JPEG, PNG, GIF, WebP, AVIF, JPEG XL or SVG. The
|
||||||
`Content-Type` must name its format, and the image's first bytes must match it.
|
upstream's `Content-Type` must name its format, and the image's first bytes must
|
||||||
An upstream answering with any other `Content-Type`, SVG included, is refused
|
match it.
|
||||||
with 415, and so is an image whose first bytes libvips reads as another format,
|
|
||||||
such as an SVG whose first bytes pass the check for AVIF.
|
|
||||||
|
|
||||||
With the format `auto`, pixa chooses the format for each request from its
|
With the format `auto`, pixa chooses the format for each request from its
|
||||||
`Accept` header, in this order:
|
`Accept` header, in this order:
|
||||||
|
|||||||
@@ -35,22 +35,6 @@ P2: security: per-origin rate limiting
|
|||||||
never read, so cached images never expired. The field and the constant are
|
never read, so cached images never expired. The field and the constant are
|
||||||
removed. Cached source and transformed images stay until eviction removes them
|
removed. Cached source and transformed images stay until eviction removes them
|
||||||
to keep the cache within `cache_max_bytes`.
|
to keep the cache within `cache_max_bytes`.
|
||||||
- 2026-10-13 SVG sources are refused, and `orig` is never saved as JPEG in place
|
|
||||||
of the source's own format (closes #68): the upstream fetch no longer accepts
|
|
||||||
`image/svg+xml`, which libvips would render. Both image routes answer 415 for
|
|
||||||
an upstream `Content-Type` that is not one of the source formats, SVG
|
|
||||||
included, where they answered 500. Before decoding, the image processor
|
|
||||||
refuses with `ErrUnsupportedInputFormat`, also answered 415, any input that
|
|
||||||
govips' `DetermineImageType`, which picks the libvips loader from the first
|
|
||||||
bytes, does not name as a source format. That catches an SVG whose first bytes
|
|
||||||
pass the first-bytes check for another type, as an XML comment can make them
|
|
||||||
do for AVIF; an SVG whose first bytes do not pass that check answers 500, as
|
|
||||||
before, and so does a damaged image in a source format, whose first bytes
|
|
||||||
`DetermineImageType` names. `orig` is always the source's own format, where a
|
|
||||||
source the processor could not name was saved as JPEG. `README.md` no longer
|
|
||||||
lists SVG. The SVG detection in `internal/magic` stays: the request path does
|
|
||||||
not call it, and it goes with the unused code of
|
|
||||||
https://git.eeqj.de/sneak/pixa/issues/168.
|
|
||||||
- 2026-10-12 per-client limit on the image routes (closes #236): the new setting
|
- 2026-10-12 per-client limit on the image routes (closes #236): the new setting
|
||||||
`image_requests_per_minute` (`PIXA_IMAGE_REQUESTS_PER_MINUTE`), default 600,
|
`image_requests_per_minute` (`PIXA_IMAGE_REQUESTS_PER_MINUTE`), default 600,
|
||||||
is how many `GET` and `HEAD` requests one client may send to `/v1/image/` and
|
is how many `GET` and `HEAD` requests one client may send to `/v1/image/` and
|
||||||
|
|||||||
@@ -247,14 +247,6 @@ func (s *Handlers) respondImageError(
|
|||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
|
||||||
if errors.Is(err, httpfetcher.ErrInvalidContentType) ||
|
|
||||||
errors.Is(err, imageprocessor.ErrUnsupportedInputFormat) {
|
|
||||||
s.respondError(w, "unsupported source format",
|
|
||||||
http.StatusUnsupportedMediaType)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
s.respondError(w, "internal error", http.StatusInternalServerError)
|
s.respondError(w, "internal error", http.StatusInternalServerError)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -161,10 +161,6 @@ func (s *Handlers) handleImageError(w http.ResponseWriter, err error) {
|
|||||||
errors.Is(err, imageprocessor.ErrTooManyImages):
|
errors.Is(err, imageprocessor.ErrTooManyImages):
|
||||||
s.respondError(w, "server busy, try again later",
|
s.respondError(w, "server busy, try again later",
|
||||||
http.StatusServiceUnavailable)
|
http.StatusServiceUnavailable)
|
||||||
case errors.Is(err, httpfetcher.ErrInvalidContentType),
|
|
||||||
errors.Is(err, imageprocessor.ErrUnsupportedInputFormat):
|
|
||||||
s.respondError(w, "unsupported source format",
|
|
||||||
http.StatusUnsupportedMediaType)
|
|
||||||
default:
|
default:
|
||||||
s.log.Error("image request failed", "error", err)
|
s.log.Error("image request failed", "error", err)
|
||||||
s.respondError(w, "internal error", http.StatusInternalServerError)
|
s.respondError(w, "internal error", http.StatusInternalServerError)
|
||||||
|
|||||||
@@ -1,54 +0,0 @@
|
|||||||
package handlers
|
|
||||||
|
|
||||||
import (
|
|
||||||
"fmt"
|
|
||||||
"log/slog"
|
|
||||||
"net/http"
|
|
||||||
"net/http/httptest"
|
|
||||||
"strings"
|
|
||||||
"testing"
|
|
||||||
|
|
||||||
"sneak.berlin/go/pixa/internal/httpfetcher"
|
|
||||||
"sneak.berlin/go/pixa/internal/imageprocessor"
|
|
||||||
"sneak.berlin/go/pixa/internal/imgcache"
|
|
||||||
)
|
|
||||||
|
|
||||||
// TestUnsupportedSourceFormatAnswers415 checks that both image routes answer
|
|
||||||
// 415 with a clear error when the upstream answers with a content type pixa
|
|
||||||
// does not accept, such as SVG, and when orig is asked of a source in a
|
|
||||||
// format the processor cannot name, wrapped as the service wraps them.
|
|
||||||
func TestUnsupportedSourceFormatAnswers415(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
h := &Handlers{log: slog.New(slog.DiscardHandler)}
|
|
||||||
req := &imgcache.ImageRequest{SourceHost: "img.example.com", SourcePath: "/a.svg"}
|
|
||||||
|
|
||||||
for _, err := range []error{
|
|
||||||
fmt.Errorf("upstream fetch failed: %w: image/svg+xml",
|
|
||||||
httpfetcher.ErrInvalidContentType),
|
|
||||||
fmt.Errorf("image processing failed: %w: unknown",
|
|
||||||
imageprocessor.ErrUnsupportedInputFormat),
|
|
||||||
} {
|
|
||||||
plain := httptest.NewRecorder()
|
|
||||||
h.respondImageError(plain, req, err)
|
|
||||||
|
|
||||||
encrypted := httptest.NewRecorder()
|
|
||||||
h.handleImageError(encrypted, err)
|
|
||||||
|
|
||||||
for route, rec := range map[string]*httptest.ResponseRecorder{
|
|
||||||
"HandleImage": plain, "HandleImageEnc": encrypted,
|
|
||||||
} {
|
|
||||||
t.Logf("%s for %v: %d %s", route, err, rec.Code, rec.Body.String())
|
|
||||||
|
|
||||||
if rec.Code != http.StatusUnsupportedMediaType {
|
|
||||||
t.Errorf("%s for %v: status = %d, want %d",
|
|
||||||
route, err, rec.Code, http.StatusUnsupportedMediaType)
|
|
||||||
}
|
|
||||||
|
|
||||||
if !strings.Contains(rec.Body.String(), "unsupported source format") {
|
|
||||||
t.Errorf("%s for %v: body = %q, want the unsupported source format error",
|
|
||||||
route, err, rec.Body.String())
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -158,6 +158,7 @@ func DefaultConfig() *Config {
|
|||||||
contentTypeWebP,
|
contentTypeWebP,
|
||||||
contentTypeAVIF,
|
contentTypeAVIF,
|
||||||
contentTypeJXL,
|
contentTypeJXL,
|
||||||
|
contentTypeSVG,
|
||||||
},
|
},
|
||||||
AllowHTTP: false,
|
AllowHTTP: false,
|
||||||
MaxConnectionsPerHost: DefaultMaxConnectionsPerHost,
|
MaxConnectionsPerHost: DefaultMaxConnectionsPerHost,
|
||||||
|
|||||||
@@ -137,12 +137,6 @@ var ErrInputDataTooLarge = errors.New("input data exceeds maximum allowed size")
|
|||||||
// not supported.
|
// not supported.
|
||||||
var ErrUnsupportedOutputFormat = errors.New("unsupported output format")
|
var ErrUnsupportedOutputFormat = errors.New("unsupported output format")
|
||||||
|
|
||||||
// ErrUnsupportedInputFormat is returned, before the input is decoded, when
|
|
||||||
// libvips reads the input's first bytes as another format than the source
|
|
||||||
// formats, or as none: for example an SVG whose first bytes pass the
|
|
||||||
// first-bytes check for AVIF.
|
|
||||||
var ErrUnsupportedInputFormat = errors.New("unsupported input format")
|
|
||||||
|
|
||||||
// ErrTooManyImages is returned when MaxConcurrentProcessing images are being
|
// ErrTooManyImages is returned when MaxConcurrentProcessing images are being
|
||||||
// processed and none finishes within ProcessingWaitTimeout.
|
// processed and none finishes within ProcessingWaitTimeout.
|
||||||
var ErrTooManyImages = errors.New("too many images being processed at once")
|
var ErrTooManyImages = errors.New("too many images being processed at once")
|
||||||
@@ -225,17 +219,6 @@ func (p *ImageProcessor) Process(
|
|||||||
return nil, ErrInputDataTooLarge
|
return nil, ErrInputDataTooLarge
|
||||||
}
|
}
|
||||||
|
|
||||||
// govips picks the libvips loader from the input's first bytes with
|
|
||||||
// DetermineImageType. Input whose first bytes it reads as no source
|
|
||||||
// format, such as SVG, which librsvg would render, is refused before
|
|
||||||
// libvips reads it.
|
|
||||||
inputFormat := p.detectFormat(vips.DetermineImageType(data))
|
|
||||||
|
|
||||||
sourceFormat, err := p.formatFromString(inputFormat)
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
|
|
||||||
// Decode image
|
// Decode image
|
||||||
img, err := vips.NewImageFromBuffer(data)
|
img, err := vips.NewImageFromBuffer(data)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -254,6 +237,9 @@ func (p *ImageProcessor) Process(
|
|||||||
origWidth := img.Width()
|
origWidth := img.Width()
|
||||||
origHeight := img.Height()
|
origHeight := img.Height()
|
||||||
|
|
||||||
|
// Detect input format
|
||||||
|
inputFormat := p.detectFormat(img)
|
||||||
|
|
||||||
// Validate input dimensions to prevent DoS via decompression bombs
|
// Validate input dimensions to prevent DoS via decompression bombs
|
||||||
if origWidth > MaxInputDimension || origHeight > MaxInputDimension {
|
if origWidth > MaxInputDimension || origHeight > MaxInputDimension {
|
||||||
return nil, ErrInputTooLarge
|
return nil, ErrInputTooLarge
|
||||||
@@ -273,7 +259,7 @@ func (p *ImageProcessor) Process(
|
|||||||
// orig is the source's own format; encode refuses an empty format
|
// orig is the source's own format; encode refuses an empty format
|
||||||
outputFormat := req.Format
|
outputFormat := req.Format
|
||||||
if outputFormat == FormatOriginal {
|
if outputFormat == FormatOriginal {
|
||||||
outputFormat = sourceFormat
|
outputFormat = p.formatFromString(inputFormat)
|
||||||
}
|
}
|
||||||
|
|
||||||
// Encode to target format
|
// Encode to target format
|
||||||
@@ -417,9 +403,10 @@ func (p *ImageProcessor) acquireSlot(ctx context.Context) (func(), error) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// detectFormat returns the format string for a govips image type.
|
// detectFormat returns the format string from a vips image.
|
||||||
func (p *ImageProcessor) detectFormat(imageType vips.ImageType) string {
|
func (p *ImageProcessor) detectFormat(img *vips.ImageRef) string {
|
||||||
switch imageType {
|
format := img.Format()
|
||||||
|
switch format {
|
||||||
case vips.ImageTypeJPEG:
|
case vips.ImageTypeJPEG:
|
||||||
return "jpeg"
|
return "jpeg"
|
||||||
case vips.ImageTypePNG:
|
case vips.ImageTypePNG:
|
||||||
@@ -700,23 +687,22 @@ func jxlDistance(quality int) float64 {
|
|||||||
return 53.0/3000.0*q*q - 23.0/20.0*q + 25.0
|
return 53.0/3000.0*q*q - 23.0/20.0*q + 25.0
|
||||||
}
|
}
|
||||||
|
|
||||||
// formatFromString converts a format string from detectFormat to Format. A
|
// formatFromString converts a format string to Format.
|
||||||
// format detectFormat could not name is ErrUnsupportedInputFormat.
|
func (p *ImageProcessor) formatFromString(format string) Format {
|
||||||
func (p *ImageProcessor) formatFromString(format string) (Format, error) {
|
|
||||||
switch format {
|
switch format {
|
||||||
case "jpeg":
|
case "jpeg":
|
||||||
return FormatJPEG, nil
|
return FormatJPEG
|
||||||
case "png":
|
case "png":
|
||||||
return FormatPNG, nil
|
return FormatPNG
|
||||||
case "gif":
|
case "gif":
|
||||||
return FormatGIF, nil
|
return FormatGIF
|
||||||
case "webp":
|
case "webp":
|
||||||
return FormatWebP, nil
|
return FormatWebP
|
||||||
case string(FormatAVIF):
|
case string(FormatAVIF):
|
||||||
return FormatAVIF, nil
|
return FormatAVIF
|
||||||
case string(FormatJXL):
|
case string(FormatJXL):
|
||||||
return FormatJXL, nil
|
return FormatJXL
|
||||||
default:
|
default:
|
||||||
return "", fmt.Errorf("%w: %s", ErrUnsupportedInputFormat, format)
|
return FormatJPEG
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1,39 +0,0 @@
|
|||||||
package imageprocessor
|
|
||||||
|
|
||||||
import (
|
|
||||||
"bytes"
|
|
||||||
"errors"
|
|
||||||
"testing"
|
|
||||||
|
|
||||||
"github.com/davidbyttow/govips/v2/vips"
|
|
||||||
)
|
|
||||||
|
|
||||||
// TestImageProcessor_OrigOfUnnamedFormatRefused verifies that orig of a TIFF
|
|
||||||
// input, which is not a source format, is refused with
|
|
||||||
// ErrUnsupportedInputFormat before it is decoded, and not saved as JPEG.
|
|
||||||
func TestImageProcessor_OrigOfUnnamedFormatRefused(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
img, err := vips.NewImageFromBuffer(createTestPNG(t, 20, 20))
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("failed to decode test PNG: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
t.Cleanup(img.Close)
|
|
||||||
|
|
||||||
tiff, _, err := img.ExportTiff(vips.NewTiffExportParams())
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("failed to save test TIFF: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
result, err := New(Params{}).Process(
|
|
||||||
t.Context(), bytes.NewReader(tiff), &Request{Format: FormatOriginal},
|
|
||||||
)
|
|
||||||
if result != nil {
|
|
||||||
t.Logf("Process() returned %s", result.ContentType)
|
|
||||||
}
|
|
||||||
|
|
||||||
if !errors.Is(err, ErrUnsupportedInputFormat) {
|
|
||||||
t.Errorf("Process() error = %v, want %v", err, ErrUnsupportedInputFormat)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,46 +0,0 @@
|
|||||||
package imageprocessor
|
|
||||||
|
|
||||||
import (
|
|
||||||
"bytes"
|
|
||||||
"io"
|
|
||||||
"testing"
|
|
||||||
)
|
|
||||||
|
|
||||||
// TestImageProcessor_LoadsEachSourceFormat verifies that an image in each of
|
|
||||||
// the six source formats, saved by the processor itself, is loaded and saved
|
|
||||||
// again in its own format (orig).
|
|
||||||
func TestImageProcessor_LoadsEachSourceFormat(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
proc := New(Params{})
|
|
||||||
|
|
||||||
for _, format := range []Format{
|
|
||||||
FormatJPEG, FormatPNG, FormatGIF, FormatWebP, FormatAVIF, FormatJXL,
|
|
||||||
} {
|
|
||||||
saved, err := proc.Process(t.Context(),
|
|
||||||
bytes.NewReader(createTestPNG(t, 20, 20)), &Request{Format: format})
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("saving the test image as %s: %v", format, err)
|
|
||||||
}
|
|
||||||
|
|
||||||
source, err := io.ReadAll(saved.Content)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("reading the %s image: %v", format, err)
|
|
||||||
}
|
|
||||||
|
|
||||||
result, err := proc.Process(t.Context(), bytes.NewReader(source),
|
|
||||||
&Request{Format: FormatOriginal})
|
|
||||||
if err != nil {
|
|
||||||
t.Errorf("orig of the %s image: %v", format, err)
|
|
||||||
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
|
|
||||||
t.Logf("orig of the %s image: %s", format, result.ContentType)
|
|
||||||
|
|
||||||
if result.ContentType != FormatToMIME(format) {
|
|
||||||
t.Errorf("orig of the %s image is %s, want %s",
|
|
||||||
format, result.ContentType, FormatToMIME(format))
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -433,22 +433,9 @@ func TestService_Get_Caching(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// TestService_Get_ServesSourceAndVariantCachedTwoDaysAgo caches a source and
|
// TestService_Get_ServesSourceAndVariantCachedTwoDaysAgo caches a source and
|
||||||
// a variant of it and sets back two days every time pixa stored for them.
|
// a variant of it, sets their stored fetch and creation times back two days,
|
||||||
// For the source: fetched_at in source_metadata, fetched_at and
|
// and checks that the variant is still served from the cache and that a new
|
||||||
// last_accessed_at in source_content, the fetched_at in its metadata file,
|
// variant is made from the cached source without fetching it again.
|
||||||
// and the access and modification times of its file and of its metadata
|
|
||||||
// file. For the variant: created_at and last_accessed_at in variant_content,
|
|
||||||
// the created_at in its .meta file, and the access and modification times of
|
|
||||||
// its file and of its .meta file. It then checks that the variant is still
|
|
||||||
// served from the cache and that a new variant is made from the cached source
|
|
||||||
// without fetching it again.
|
|
||||||
//
|
|
||||||
// Left at the present: the files' status change and birth times, which a
|
|
||||||
// program cannot set, and the times of the directories holding the files,
|
|
||||||
// which other entries share. The source's stored response headers are empty,
|
|
||||||
// as the mock fetcher sends none. expires_at, etag and last_modified in
|
|
||||||
// source_metadata and in the metadata file are never written, nor are the
|
|
||||||
// request_cache and output_content tables.
|
|
||||||
func TestService_Get_ServesSourceAndVariantCachedTwoDaysAgo(t *testing.T) {
|
func TestService_Get_ServesSourceAndVariantCachedTwoDaysAgo(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
@@ -462,9 +449,45 @@ func TestService_Get_ServesSourceAndVariantCachedTwoDaysAgo(t *testing.T) {
|
|||||||
|
|
||||||
_ = resp.Content.Close()
|
_ = resp.Content.Close()
|
||||||
|
|
||||||
|
for _, statement := range []string{
|
||||||
|
`UPDATE source_metadata SET fetched_at = datetime('now', '-2 days')`,
|
||||||
|
`UPDATE source_content SET fetched_at = datetime('now', '-2 days')`,
|
||||||
|
`UPDATE variant_content SET created_at = datetime('now', '-2 days')`,
|
||||||
|
} {
|
||||||
|
result, err := svc.cache.db.ExecContext(t.Context(), statement)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("%s: %v", statement, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
rows, err := result.RowsAffected()
|
||||||
|
if err != nil || rows != 1 {
|
||||||
|
t.Fatalf("%s: rows = %d, error = %v, want 1 row", statement, rows, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The variant's creation time is also its file's modification time
|
||||||
|
// and the created_at in its .meta file
|
||||||
twoDaysAgo := time.Now().AddDate(0, 0, -2)
|
twoDaysAgo := time.Now().AddDate(0, 0, -2)
|
||||||
backdateSource(t, svc.cache, variant, twoDaysAgo)
|
variantPath := svc.cache.variants.keyToPath(CacheKey(variant))
|
||||||
backdateVariant(t, svc.cache, variant, twoDaysAgo)
|
|
||||||
|
err = os.Chtimes(variantPath, twoDaysAgo, twoDaysAgo)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("setting the variant file's modification time back: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
meta, err := json.Marshal(VariantMeta{
|
||||||
|
ContentType: resp.ContentType,
|
||||||
|
Size: resp.ContentLength,
|
||||||
|
CreatedAt: twoDaysAgo.Unix(),
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("encoding .meta file: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
err = os.WriteFile(variantPath+".meta", meta, StorageFilePerm)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("writing .meta file: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
resp, err = svc.Get(t.Context(), variant)
|
resp, err = svc.Get(t.Context(), variant)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -492,112 +515,6 @@ func TestService_Get_ServesSourceAndVariantCachedTwoDaysAgo(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// backdateSource sets every time pixa stored for req's cached source to when,
|
|
||||||
// as listed on TestService_Get_ServesSourceAndVariantCachedTwoDaysAgo.
|
|
||||||
func backdateSource(
|
|
||||||
t *testing.T, cache *Cache, req *ImageRequest, when time.Time,
|
|
||||||
) {
|
|
||||||
t.Helper()
|
|
||||||
|
|
||||||
pathHash := HashPath(req.SourcePath + "?" + req.SourceQuery)
|
|
||||||
|
|
||||||
meta, err := cache.srcMetadata.Load(req.SourceHost, pathHash)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("reading the source's metadata file: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
meta.FetchedAt = when.Unix()
|
|
||||||
|
|
||||||
err = cache.srcMetadata.Store(req.SourceHost, pathHash, meta)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("writing the source's metadata file: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
dbTime := when.UTC().Format(sqliteTimestampFormat)
|
|
||||||
|
|
||||||
updateOneRow(t, cache, `
|
|
||||||
UPDATE source_metadata SET fetched_at = ?
|
|
||||||
WHERE source_host = ? AND source_path = ? AND source_query = ?
|
|
||||||
`, dbTime, req.SourceHost, req.SourcePath, req.SourceQuery)
|
|
||||||
updateOneRow(t, cache,
|
|
||||||
`UPDATE source_content SET fetched_at = ? WHERE content_hash = ?`,
|
|
||||||
dbTime, meta.ContentHash)
|
|
||||||
setSourceLastAccessed(t, cache, ContentHash(meta.ContentHash), when)
|
|
||||||
|
|
||||||
for _, path := range []string{
|
|
||||||
cache.srcContent.hashToPath(ContentHash(meta.ContentHash)),
|
|
||||||
cache.srcMetadata.metaPath(req.SourceHost, pathHash),
|
|
||||||
} {
|
|
||||||
err = os.Chtimes(path, when, when)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("setting back the times of %s: %v", path, err)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// backdateVariant sets every time pixa stored for the cached variant req asks
|
|
||||||
// for to when, as listed on
|
|
||||||
// TestService_Get_ServesSourceAndVariantCachedTwoDaysAgo.
|
|
||||||
func backdateVariant(
|
|
||||||
t *testing.T, cache *Cache, req *ImageRequest, when time.Time,
|
|
||||||
) {
|
|
||||||
t.Helper()
|
|
||||||
|
|
||||||
key := CacheKey(req)
|
|
||||||
|
|
||||||
content, size, contentType, err := cache.variants.LoadWithMeta(key)
|
|
||||||
if err != nil || contentType == "" {
|
|
||||||
t.Fatalf("reading the variant's .meta file: content type %q, error %v",
|
|
||||||
contentType, err)
|
|
||||||
}
|
|
||||||
|
|
||||||
_ = content.Close()
|
|
||||||
|
|
||||||
meta, err := json.Marshal(VariantMeta{
|
|
||||||
ContentType: contentType,
|
|
||||||
Size: size,
|
|
||||||
CreatedAt: when.Unix(),
|
|
||||||
})
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("encoding the variant's .meta file: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
path := cache.variants.keyToPath(key)
|
|
||||||
|
|
||||||
err = os.WriteFile(path+variantMetaSuffix, meta, StorageFilePerm)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("writing the variant's .meta file: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
updateOneRow(t, cache,
|
|
||||||
`UPDATE variant_content SET created_at = ? WHERE cache_key = ?`,
|
|
||||||
when.UTC().Format(sqliteTimestampFormat), string(key))
|
|
||||||
setVariantLastAccessed(t, cache, key, when)
|
|
||||||
|
|
||||||
for _, file := range []string{path, path + variantMetaSuffix} {
|
|
||||||
err = os.Chtimes(file, when, when)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("setting back the times of %s: %v", file, err)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// updateOneRow runs the UPDATE statement query and fails the test unless it
|
|
||||||
// changed exactly one row.
|
|
||||||
func updateOneRow(t *testing.T, cache *Cache, query string, args ...any) {
|
|
||||||
t.Helper()
|
|
||||||
|
|
||||||
result, err := cache.db.ExecContext(t.Context(), query, args...)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("%s: %v", query, err)
|
|
||||||
}
|
|
||||||
|
|
||||||
rows, err := result.RowsAffected()
|
|
||||||
if err != nil || rows != 1 {
|
|
||||||
t.Fatalf("%s: rows = %d, error = %v, want 1 row", query, rows, err)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestService_Get_DifferentSizes(t *testing.T) {
|
func TestService_Get_DifferentSizes(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
|
|||||||
@@ -1,71 +0,0 @@
|
|||||||
package server
|
|
||||||
|
|
||||||
import (
|
|
||||||
"io"
|
|
||||||
"net/http"
|
|
||||||
"net/http/httptest"
|
|
||||||
"strings"
|
|
||||||
"testing"
|
|
||||||
|
|
||||||
"sneak.berlin/go/pixa/internal/encurl"
|
|
||||||
"sneak.berlin/go/pixa/internal/imgcache"
|
|
||||||
)
|
|
||||||
|
|
||||||
// svgServedAsAVIF is an SVG whose first bytes pass the first-bytes check for
|
|
||||||
// AVIF: its XML comment puts "ftypavis" at bytes 4 to 11, where an AVIF file
|
|
||||||
// has its ftyp box and brand. libvips would load it as SVG.
|
|
||||||
const svgServedAsAVIF = `<!--ftypavis--><svg xmlns="http://www.w3.org/2000/svg" ` +
|
|
||||||
`width="64" height="48"><rect width="64" height="48"/></svg>`
|
|
||||||
|
|
||||||
// TestImageProxyRefusesSVGServedAsAVIF requests svgServedAsAVIF, which the
|
|
||||||
// upstream answers as image/avif, through both image routes, resized as PNG
|
|
||||||
// and in its own format (orig). Each request is answered 415, as for an SVG
|
|
||||||
// source, never with an image.
|
|
||||||
func TestImageProxyRefusesSVGServedAsAVIF(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
upstream := httptest.NewServer(http.HandlerFunc(
|
|
||||||
func(w http.ResponseWriter, _ *http.Request) {
|
|
||||||
w.Header().Set("Content-Type", "image/avif")
|
|
||||||
_, _ = io.WriteString(w, svgServedAsAVIF)
|
|
||||||
}))
|
|
||||||
t.Cleanup(upstream.Close)
|
|
||||||
|
|
||||||
s, _, _ := startImageProxy(t, upstream)
|
|
||||||
|
|
||||||
generator, err := encurl.NewGenerator(testSigningKey)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("encurl.NewGenerator: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
encryptedURL := func(format imgcache.ImageFormat) string {
|
|
||||||
token, err := generator.Generate(&encurl.Payload{
|
|
||||||
SourceHost: upstreamHost, SourcePath: "/drawing.avif",
|
|
||||||
Width: 32, Height: 24, Format: format,
|
|
||||||
})
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("encurl Generate: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
return "/v1/e/" + token + "/drawing." + string(format)
|
|
||||||
}
|
|
||||||
|
|
||||||
for _, target := range []string{
|
|
||||||
"/v1/image/" + upstreamHost + "/drawing.avif/32x24.png",
|
|
||||||
"/v1/image/" + upstreamHost + "/drawing.avif/64x48.orig",
|
|
||||||
encryptedURL(imgcache.FormatPNG),
|
|
||||||
encryptedURL(imgcache.FormatOriginal),
|
|
||||||
} {
|
|
||||||
rec := httptest.NewRecorder()
|
|
||||||
s.ServeHTTP(rec, httptest.NewRequestWithContext(
|
|
||||||
t.Context(), http.MethodGet, target, nil))
|
|
||||||
t.Logf("GET %s: %d, Content-Type %s, body %q",
|
|
||||||
target, rec.Code, rec.Header().Get("Content-Type"), rec.Body.String())
|
|
||||||
|
|
||||||
if rec.Code != http.StatusUnsupportedMediaType ||
|
|
||||||
!strings.Contains(rec.Body.String(), "unsupported source format") {
|
|
||||||
t.Errorf("GET %s: %d %q, want %d unsupported source format",
|
|
||||||
target, rec.Code, rec.Body.String(), http.StatusUnsupportedMediaType)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,41 +0,0 @@
|
|||||||
package server
|
|
||||||
|
|
||||||
import (
|
|
||||||
"io"
|
|
||||||
"net/http"
|
|
||||||
"net/http/httptest"
|
|
||||||
"testing"
|
|
||||||
)
|
|
||||||
|
|
||||||
// TestImageProxyRefusesSVG requests an image the upstream answers as SVG,
|
|
||||||
// in its own format (orig) and resized as PNG, through pixa's router,
|
|
||||||
// upstream fetcher and image processor. pixa does not accept SVG as a
|
|
||||||
// source, so each request is answered 415, never with an image.
|
|
||||||
func TestImageProxyRefusesSVG(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
upstream := httptest.NewServer(http.HandlerFunc(
|
|
||||||
func(w http.ResponseWriter, _ *http.Request) {
|
|
||||||
w.Header().Set("Content-Type", "image/svg+xml")
|
|
||||||
_, _ = io.WriteString(w, `<svg xmlns="http://www.w3.org/2000/svg" `+
|
|
||||||
`width="64" height="48"><rect width="64" height="48"/></svg>`)
|
|
||||||
}))
|
|
||||||
t.Cleanup(upstream.Close)
|
|
||||||
|
|
||||||
s, _, _ := startImageProxy(t, upstream)
|
|
||||||
|
|
||||||
for _, sizeFormat := range []string{"64x48.orig", "32x24.png"} {
|
|
||||||
target := "/v1/image/" + upstreamHost + "/drawing.svg/" + sizeFormat
|
|
||||||
|
|
||||||
rec := httptest.NewRecorder()
|
|
||||||
s.ServeHTTP(rec, httptest.NewRequestWithContext(
|
|
||||||
t.Context(), http.MethodGet, target, nil))
|
|
||||||
t.Logf("GET %s: %d, Content-Type %s, body %s",
|
|
||||||
target, rec.Code, rec.Header().Get("Content-Type"), rec.Body.String())
|
|
||||||
|
|
||||||
if rec.Code != http.StatusUnsupportedMediaType {
|
|
||||||
t.Errorf("GET %s status = %d, want %d",
|
|
||||||
target, rec.Code, http.StatusUnsupportedMediaType)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
Reference in New Issue
Block a user