Compare commits
2
Commits
3c8108dcd0
...
next
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
bdde021b45 | ||
|
|
db91ab29e6 |
@@ -175,20 +175,21 @@ path under `/v1/` answers 200, in maintenance mode too.
|
|||||||
with the page naming a field that is not valid; 500 when the URL cannot be
|
with the page naming a field that is not valid; 500 when the URL cannot be
|
||||||
made.
|
made.
|
||||||
- `GET /logout` — end the login session. Needs: nothing. Answers: 303 to `/`.
|
- `GET /logout` — end the login session. Needs: nothing. Answers: 303 to `/`.
|
||||||
- `GET` or `HEAD` `/v1/image/<host>/<path>/<size>.<format>` — an image, fetched,
|
- `GET` or `HEAD` `/v1/image/<host>/<path>/<size>.<format>`, or
|
||||||
resized and converted (below). Needs: a signature, unless the host is
|
`/v1/image/<host>/<path>/<size>` with no format — an image, fetched, resized
|
||||||
allowlisted (see Source Hosts). Answers: 200; 304 when `If-None-Match` matches
|
and converted (below). Needs: a signature, unless the host is allowlisted (see
|
||||||
the image's `ETag`; 400 for a URL or parameter that is not valid, or for the
|
Source Hosts). Answers: 200; 304 when `If-None-Match` matches the image's
|
||||||
format `auto` an `Accept` header that is not valid; 406 for the format `auto`
|
`ETag`; 400 for a URL or parameter that is not valid, or for the format `auto`
|
||||||
when `Accept` allows none of the formats it chooses from; 401 for a missing or
|
an `Accept` header that is not valid; 406 for the format `auto` when `Accept`
|
||||||
wrong signature, a missing `exp` or an `exp` in the past; 403 when the
|
allows none of the formats it chooses from; 401 for a missing or wrong
|
||||||
request's `Referer` names a host in `referer_blocklist`, checked before the
|
signature, a missing `exp` or an `exp` in the past; 403 when the request's
|
||||||
signature, the cache and the upstream fetch; 403 when the upstream host, or a
|
`Referer` names a host in `referer_blocklist`, checked before the signature,
|
||||||
host it redirects to, is `localhost`, ends in `.localhost` or `.local`, or has
|
the cache and the upstream fetch; 403 when the upstream host, or a host it
|
||||||
an address in a blocked network (see `blocked_networks`); 502 when the
|
redirects to, is `localhost`, ends in `.localhost` or `.local`, or has an
|
||||||
upstream answered with an error status, and for 5 minutes after that for the
|
address in a blocked network (see `blocked_networks`); 502 when the upstream
|
||||||
same source URL; 503 when pixa is busy or in maintenance mode; 500 for any
|
answered with an error status, and for 5 minutes after that for the same
|
||||||
other failure.
|
source URL; 503 when pixa is busy or in maintenance mode; 500 for any other
|
||||||
|
failure.
|
||||||
- `GET` or `HEAD` `/v1/e/<token>/<name>` — an image through an encrypted URL
|
- `GET` or `HEAD` `/v1/e/<token>/<name>` — an image through an encrypted URL
|
||||||
(see Encrypted URLs). Needs: nothing but the URL. Answers: 200; 304 when
|
(see Encrypted URLs). Needs: nothing but the URL. Answers: 200; 304 when
|
||||||
`If-None-Match` matches the image's `ETag`; 400 for a token that does not
|
`If-None-Match` matches the image's `ETag`; 400 for a token that does not
|
||||||
@@ -231,10 +232,11 @@ proxy in front of pixa must pass that header on unchanged. A form body over 1
|
|||||||
MiB is refused with 413. The image routes answer the errors listed for them with
|
MiB is refused with 413. The image routes answer the errors listed for them with
|
||||||
JSON holding `error`, `status` and `timestamp`.
|
JSON holding `error`, `status` and `timestamp`.
|
||||||
|
|
||||||
An image URL has this form:
|
An image URL has one of these forms, the second with no format:
|
||||||
|
|
||||||
```
|
```
|
||||||
/v1/image/<host>/<path>/<size>.<format>?sig=<signature>&exp=<expiration>&q=<quality>&fit=<fit>
|
/v1/image/<host>/<path>/<size>.<format>?sig=<signature>&exp=<expiration>&q=<quality>&fit=<fit>
|
||||||
|
/v1/image/<host>/<path>/<size>?sig=<signature>&exp=<expiration>&q=<quality>&fit=<fit>
|
||||||
```
|
```
|
||||||
|
|
||||||
Images are only fetched from origins using TLS with valid certificates, unless
|
Images are only fetched from origins using TLS with valid certificates, unless
|
||||||
@@ -245,7 +247,9 @@ A request whose query string cannot be decoded, or gives any parameter more than
|
|||||||
once, is refused with 400.
|
once, is refused with 400.
|
||||||
|
|
||||||
- `<format>`: one of `orig` (or `original`), `jpeg` (or `jpg`), `png`, `webp`,
|
- `<format>`: one of `orig` (or `original`), `jpeg` (or `jpg`), `png`, `webp`,
|
||||||
`avif`, `jxl` (JPEG XL), `gif`, or `auto` (below)
|
`avif`, `jxl` (JPEG XL), `gif`, or `auto` (below). A URL with no format (the
|
||||||
|
second form, with no dot after the size) is served as JPEG XL, the default, as
|
||||||
|
with `jxl`
|
||||||
- `<size>`: `orig` or `<width>x<height>` (e.g. `800x600`)
|
- `<size>`: `orig` or `<width>x<height>` (e.g. `800x600`)
|
||||||
- `sig` and `exp`: the signature and its expiry, needed unless the host is
|
- `sig` and `exp`: the signature and its expiry, needed unless the host is
|
||||||
allowlisted (see Signature Specification)
|
allowlisted (see Signature Specification)
|
||||||
@@ -321,13 +325,15 @@ nor change what it asks for.
|
|||||||
lasts 30 days, or until `/logout`.
|
lasts 30 days, or until `/logout`.
|
||||||
2. On the generator page, give the source image's URL, the width and height, the
|
2. On the generator page, give the source image's URL, the width and height, the
|
||||||
format, quality and fit, and how long the URL lasts, then submit the form
|
format, quality and fit, and how long the URL lasts, then submit the form
|
||||||
(`POST /generate`). Width and height both empty or `0` keep the original
|
(`POST /generate`). The format is JPEG XL unless another is chosen; a form
|
||||||
size; if only one of them is empty or `0`, that side is scaled to keep the
|
sent with an empty format, or none, also makes a JPEG XL URL. Width and
|
||||||
image's proportions.
|
height both empty or `0` keep the original size; if only one of them is empty
|
||||||
|
or `0`, that side is scaled to keep the image's proportions.
|
||||||
3. The page shows the URL, `https://<host>/v1/e/<token>/img.<format>`, and when
|
3. The page shows the URL, `https://<host>/v1/e/<token>/img.<format>`, and when
|
||||||
it expires. `<host>` is the host the page was opened on, and the URL starts
|
it expires. `<host>` is the host the page was opened on, and the URL starts
|
||||||
with `http` instead while `debug` is on. The name after the token is ignored
|
with `http` instead while `debug` is on. The name after the token is ignored
|
||||||
and only gives the URL a file extension, `jpg` for `orig` and `auto`.
|
and only gives the URL a file extension, `jpg` for `orig` and `auto`, and
|
||||||
|
`jxl` for a form with no format.
|
||||||
|
|
||||||
The token holds the source's host, path and query and the size, format, quality,
|
The token holds the source's host, path and query and the size, format, quality,
|
||||||
fit and expiry, encrypted with a key derived from `signing_key`. The source
|
fit and expiry, encrypted with a key derived from `signing_key`. The source
|
||||||
@@ -387,8 +393,9 @@ Where:
|
|||||||
- `width` — requested width in pixels, `0` for original
|
- `width` — requested width in pixels, `0` for original
|
||||||
- `height` — requested height in pixels, `0` for original
|
- `height` — requested height in pixels, `0` for original
|
||||||
- `format` — output format, one of those listed under Routes, with `original`
|
- `format` — output format, one of those listed under Routes, with `original`
|
||||||
signed as `orig` and `jpg` as `jpeg`; `auto` is signed as `auto`, not as the
|
signed as `orig`, `jpg` as `jpeg`, and no format as `jxl`, so a URL with no
|
||||||
format chosen for the request
|
format has the signature of the same URL ending in `.jxl`; `auto` is signed as
|
||||||
|
`auto`, not as the format chosen for the request
|
||||||
- `expiration` — the URL's `exp` query parameter, the Unix timestamp when the
|
- `expiration` — the URL's `exp` query parameter, the Unix timestamp when the
|
||||||
signature expires; a request whose `exp` is not a whole number, an empty
|
signature expires; a request whose `exp` is not a whole number, an empty
|
||||||
`exp=` included, is refused with 400
|
`exp=` included, is refused with 400
|
||||||
|
|||||||
@@ -30,6 +30,30 @@ P2: security: per-IP rate limiting on the image routes
|
|||||||
|
|
||||||
# Completed Steps
|
# Completed Steps
|
||||||
|
|
||||||
|
- 2026-10-08 every output format is saved with settings pixa sets on purpose
|
||||||
|
(closes #232): each format has its own govips export, as JPEG XL does, in
|
||||||
|
place of govips' generic `Export`, which sent libvips a zero for some settings
|
||||||
|
it was not given. PNG gets libvips' default compression, 6, where it had none,
|
||||||
|
and WebP libvips' default effort, 4, where it had 0. GIF was already at
|
||||||
|
libvips' default effort, 7, and JPEG output is unchanged. AVIF is saved at
|
||||||
|
effort 1, the lowest govips can set, where it had libvips' default, 4. With
|
||||||
|
one libvips thread, an 8192x8192 image of random pixels, the worst case, takes
|
||||||
|
about 51 seconds as AVIF at effort 1 and 43 as WebP at effort 4, against the
|
||||||
|
default `downstream_timeout` of 60 seconds. On an image of milder noise, which
|
||||||
|
AVIF at effort 1 saves in about 12 seconds, effort 4 takes minutes. AVIF is
|
||||||
|
also saved with 8 bits per sample from a 16-bit source, which libvips would
|
||||||
|
save with 12: a 16-bit 8192x8192 image of milder noise takes about 54 seconds
|
||||||
|
at effort 1 with 12 bits, nearly all of the default `downstream_timeout`, and
|
||||||
|
about 12 with 8.
|
||||||
|
- 2026-10-08 JPEG XL is the default output (closes #222): a `/v1/image/` URL
|
||||||
|
whose last segment is a size with no format, such as `800x600` or `orig`, is
|
||||||
|
served as JPEG XL and signed as `jxl`, so it has the signature of the same URL
|
||||||
|
ending in `.jxl`. An encrypted URL whose token holds no format is served as
|
||||||
|
JPEG XL (`encurl.DefaultFormat`). The generator page selects JPEG XL until
|
||||||
|
another format is chosen, and a form with an empty format, or none, makes a
|
||||||
|
JPEG XL URL whose name ends in `.jxl`. The image processor no longer takes an
|
||||||
|
empty format as `orig`: both routes give every request a format, and it
|
||||||
|
refuses a request with none. `auto` still ends with JPEG.
|
||||||
- 2026-10-08 JPEG XL as an input and output format (part of #222): a source
|
- 2026-10-08 JPEG XL as an input and output format (part of #222): a source
|
||||||
whose bytes start with either JPEG XL signature, the bare codestream's `FF 0A`
|
whose bytes start with either JPEG XL signature, the bare codestream's `FF 0A`
|
||||||
or the container's, is detected as `image/jxl`, which the upstream fetch
|
or the container's, is detected as `image/jxl`, which the upstream fetch
|
||||||
|
|||||||
@@ -14,7 +14,7 @@ import (
|
|||||||
// Default values for optional fields.
|
// Default values for optional fields.
|
||||||
const (
|
const (
|
||||||
DefaultQuality = 85
|
DefaultQuality = 85
|
||||||
DefaultFormat = imgcache.FormatOriginal
|
DefaultFormat = imgcache.FormatJXL
|
||||||
DefaultFitMode = imgcache.FitCover
|
DefaultFitMode = imgcache.FitCover
|
||||||
|
|
||||||
// HKDF salt for URL encryption key derivation
|
// HKDF salt for URL encryption key derivation
|
||||||
@@ -37,7 +37,7 @@ type Payload struct {
|
|||||||
SourceQuery string `cbor:"q,omitempty"` // optional
|
SourceQuery string `cbor:"q,omitempty"` // optional
|
||||||
Width int `cbor:"w,omitempty"` // 0 = original
|
Width int `cbor:"w,omitempty"` // 0 = original
|
||||||
Height int `cbor:"ht,omitempty"` // 0 = original
|
Height int `cbor:"ht,omitempty"` // 0 = original
|
||||||
Format imgcache.ImageFormat `cbor:"f,omitempty"` // default: orig
|
Format imgcache.ImageFormat `cbor:"f,omitempty"` // default: jxl
|
||||||
Quality int `cbor:"ql,omitempty"` // default: 85
|
Quality int `cbor:"ql,omitempty"` // default: 85
|
||||||
FitMode imgcache.FitMode `cbor:"fm,omitempty"` // default: cover
|
FitMode imgcache.FitMode `cbor:"fm,omitempty"` // default: cover
|
||||||
ExpiresAt int64 `cbor:"e,omitempty"` // 0 = never expires
|
ExpiresAt int64 `cbor:"e,omitempty"` // 0 = never expires
|
||||||
|
|||||||
@@ -369,10 +369,15 @@ func (s *Handlers) buildGeneratedURL(r *http.Request, token, format string) stri
|
|||||||
scheme = "http"
|
scheme = "http"
|
||||||
}
|
}
|
||||||
|
|
||||||
// Determine file extension for the trailing filename
|
// Determine file extension for the trailing filename. A form with no
|
||||||
|
// format makes a token with none, which is served as encurl.DefaultFormat.
|
||||||
ext := format
|
ext := format
|
||||||
if ext == "" || ext == "orig" || ext == "auto" {
|
|
||||||
ext = "jpg" // Default extension
|
switch format {
|
||||||
|
case "":
|
||||||
|
ext = string(encurl.DefaultFormat)
|
||||||
|
case "orig", "auto":
|
||||||
|
ext = "jpg"
|
||||||
}
|
}
|
||||||
|
|
||||||
return scheme + "://" + r.Host + "/v1/e/" + url.PathEscape(token) + "/img." + ext
|
return scheme + "://" + r.Host + "/v1/e/" + url.PathEscape(token) + "/img." + ext
|
||||||
|
|||||||
@@ -18,7 +18,8 @@ import (
|
|||||||
)
|
)
|
||||||
|
|
||||||
// HandleImage handles the main image proxy route:
|
// HandleImage handles the main image proxy route:
|
||||||
// /v1/image/<host>/<path>/<width>x<height>.<format>
|
// /v1/image/<host>/<path>/<width>x<height>.<format>, or with no format
|
||||||
|
// /v1/image/<host>/<path>/<width>x<height>
|
||||||
func (s *Handlers) HandleImage() http.HandlerFunc {
|
func (s *Handlers) HandleImage() http.HandlerFunc {
|
||||||
return func(w http.ResponseWriter, r *http.Request) {
|
return func(w http.ResponseWriter, r *http.Request) {
|
||||||
if s.refuseBlockedReferer(w, r) {
|
if s.refuseBlockedReferer(w, r) {
|
||||||
|
|||||||
@@ -0,0 +1,162 @@
|
|||||||
|
package handlers
|
||||||
|
|
||||||
|
import (
|
||||||
|
"fmt"
|
||||||
|
"log/slog"
|
||||||
|
"maps"
|
||||||
|
"net/http"
|
||||||
|
"net/http/httptest"
|
||||||
|
"net/url"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/davidbyttow/govips/v2/vips"
|
||||||
|
|
||||||
|
"sneak.berlin/go/pixa/internal/encurl"
|
||||||
|
"sneak.berlin/go/pixa/internal/imgcache"
|
||||||
|
"sneak.berlin/go/pixa/internal/signature"
|
||||||
|
)
|
||||||
|
|
||||||
|
// requireJPEGXL requires that rec answers 200 with a JPEG XL image.
|
||||||
|
func requireJPEGXL(t *testing.T, rec *httptest.ResponseRecorder) {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
if rec.Code != http.StatusOK {
|
||||||
|
t.Fatalf("status = %d, want %d; body %q",
|
||||||
|
rec.Code, http.StatusOK, rec.Body.String())
|
||||||
|
}
|
||||||
|
|
||||||
|
if got := rec.Header().Get("Content-Type"); got != jxlType {
|
||||||
|
t.Errorf("Content-Type = %q, want %s", got, jxlType)
|
||||||
|
}
|
||||||
|
|
||||||
|
if got := vips.DetermineImageType(rec.Body.Bytes()); got != vips.ImageTypeJXL {
|
||||||
|
t.Errorf("body is %s, want jxl", vips.ImageTypes[got])
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestImageWithoutFormat_ServesJPEGXL verifies that a /v1/image/ URL whose
|
||||||
|
// last segment is a size with no format, 50x50 or orig, answers JPEG XL.
|
||||||
|
func TestImageWithoutFormat_ServesJPEGXL(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
route := newImageRoute(t, newPhotoFetcher(t, allowlistedHost))
|
||||||
|
|
||||||
|
for _, size := range []string{"50x50", "orig"} {
|
||||||
|
target := "/v1/image/" + allowlistedHost + photoPath + "/" + size
|
||||||
|
requireJPEGXL(t, sendGet(t, route, target))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestImageWithoutFormat_SignedAsJXL verifies that a /v1/image/ URL with no
|
||||||
|
// format is signed as jxl: the signature made for the URL ending in .jxl is
|
||||||
|
// accepted for the same URL without .jxl.
|
||||||
|
func TestImageWithoutFormat_SignedAsJXL(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
route := newImageRoute(t, newPhotoFetcher(t, signedHost))
|
||||||
|
expires := time.Now().Add(time.Hour)
|
||||||
|
|
||||||
|
sig := signature.New(testSigningKey).Sign(&signature.Request{
|
||||||
|
SourceHost: signedHost,
|
||||||
|
SourcePath: photoPath,
|
||||||
|
Width: 50,
|
||||||
|
Height: 50,
|
||||||
|
Format: string(imgcache.FormatJXL),
|
||||||
|
Quality: encurl.DefaultQuality,
|
||||||
|
FitMode: string(imgcache.FitCover),
|
||||||
|
Expires: expires,
|
||||||
|
})
|
||||||
|
query := fmt.Sprintf("?sig=%s&exp=%d", sig, expires.Unix())
|
||||||
|
|
||||||
|
for _, size := range []string{"50x50.jxl", "50x50"} {
|
||||||
|
target := "/v1/image/" + signedHost + photoPath + "/" + size + query
|
||||||
|
requireJPEGXL(t, sendGet(t, route, target))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestImageEncWithoutFormat_ServesJPEGXL verifies that an encrypted URL whose
|
||||||
|
// token holds no format answers JPEG XL.
|
||||||
|
func TestImageEncWithoutFormat_ServesJPEGXL(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
h, srv := newSignedHostServer(t, slog.New(slog.DiscardHandler))
|
||||||
|
|
||||||
|
token, err := h.encGen.Generate(&encurl.Payload{
|
||||||
|
SourceHost: signedHost,
|
||||||
|
SourcePath: photoPath,
|
||||||
|
Width: 50,
|
||||||
|
Height: 50,
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Generate() error = %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
requireJPEGXL(t, getEncToken(srv, token))
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestGeneratorPage_SelectsJPEGXL verifies that the generator page's format
|
||||||
|
// choice is JPEG XL until another is chosen.
|
||||||
|
func TestGeneratorPage_SelectsJPEGXL(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
h, srv := newCSRFTestRouter(t)
|
||||||
|
|
||||||
|
req := httptest.NewRequestWithContext(t.Context(), http.MethodGet, "/", nil)
|
||||||
|
req.AddCookie(newSessionCookie(t, h))
|
||||||
|
|
||||||
|
rec := httptest.NewRecorder()
|
||||||
|
srv.ServeHTTP(rec, req)
|
||||||
|
|
||||||
|
if !strings.Contains(rec.Body.String(), `<option value="jxl" selected>`) {
|
||||||
|
t.Errorf("generator page does not select JPEG XL: %s", rec.Body.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestGeneratePost_NoFormat_MakesJPEGXLURL verifies that the generator form
|
||||||
|
// sent with an empty format field, or with none, makes a URL whose name ends
|
||||||
|
// in .jxl and which answers JPEG XL.
|
||||||
|
func TestGeneratePost_NoFormat_MakesJPEGXLURL(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
photo := url.Values{
|
||||||
|
sourceURLField: {"https://" + signedHost + photoPath},
|
||||||
|
widthField: {"50"},
|
||||||
|
heightField: {"50"},
|
||||||
|
}
|
||||||
|
|
||||||
|
emptyFormat := maps.Clone(photo)
|
||||||
|
emptyFormat.Set(formatField, "")
|
||||||
|
|
||||||
|
for name, form := range map[string]url.Values{
|
||||||
|
"empty format field": emptyFormat,
|
||||||
|
"no format field": photo,
|
||||||
|
} {
|
||||||
|
t.Run(name, func(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
_, imageSrv := newSignedHostServer(t, slog.New(slog.DiscardHandler))
|
||||||
|
|
||||||
|
rec := generatePost(t, form)
|
||||||
|
|
||||||
|
match := generatedURLPattern.FindStringSubmatch(rec.Body.String())
|
||||||
|
if match == nil {
|
||||||
|
t.Fatalf("generator page shows no URL: %d %s",
|
||||||
|
rec.Code, rec.Body.String())
|
||||||
|
}
|
||||||
|
|
||||||
|
t.Logf("generated URL path: %s", match[1])
|
||||||
|
|
||||||
|
if !strings.HasSuffix(match[1], "/img.jxl") {
|
||||||
|
t.Errorf("generated URL %s does not end in /img.jxl", match[1])
|
||||||
|
}
|
||||||
|
|
||||||
|
imageRec := httptest.NewRecorder()
|
||||||
|
imageSrv.ServeHTTP(imageRec, httptest.NewRequestWithContext(
|
||||||
|
t.Context(), http.MethodGet, match[1], nil))
|
||||||
|
|
||||||
|
requireJPEGXL(t, imageRec)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,21 @@
|
|||||||
|
package imageprocessor
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"errors"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
// TestImageProcessor_EmptyFormatRefused verifies that a request with no format
|
||||||
|
// is refused, as both image routes give every request a format before it is
|
||||||
|
// processed.
|
||||||
|
func TestImageProcessor_EmptyFormatRefused(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
_, err := New(Params{}).Process(
|
||||||
|
t.Context(), bytes.NewReader(createTestJPEG(t, 20, 20)), &Request{},
|
||||||
|
)
|
||||||
|
if !errors.Is(err, ErrUnsupportedOutputFormat) {
|
||||||
|
t.Errorf("Process() error = %v, want %v", err, ErrUnsupportedOutputFormat)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,145 @@
|
|||||||
|
package imageprocessor
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"image"
|
||||||
|
"image/color"
|
||||||
|
"image/png"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/davidbyttow/govips/v2/vips"
|
||||||
|
)
|
||||||
|
|
||||||
|
// processedSize runs input through Process and returns the output's size in
|
||||||
|
// bytes.
|
||||||
|
func processedSize(t *testing.T, input []byte, req *Request) int64 {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
result, err := New(Params{}).Process(t.Context(), bytes.NewReader(input), req)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Process() error = %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
_ = result.Content.Close()
|
||||||
|
|
||||||
|
return result.ContentLength
|
||||||
|
}
|
||||||
|
|
||||||
|
// encodePNG encodes img as PNG with Go's encoder.
|
||||||
|
func encodePNG(t *testing.T, img image.Image) []byte {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
var buf bytes.Buffer
|
||||||
|
|
||||||
|
err := png.Encode(&buf, img)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("failed to encode test PNG: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
return buf.Bytes()
|
||||||
|
}
|
||||||
|
|
||||||
|
// decode decodes input with vips, as Process does.
|
||||||
|
func decode(t *testing.T, input []byte) *vips.ImageRef {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
img, err := vips.NewImageFromBuffer(input)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("failed to decode input: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
t.Cleanup(img.Close)
|
||||||
|
|
||||||
|
return img
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestImageProcessor_PNGIsCompressed verifies that a PNG output is
|
||||||
|
// compressed: a flat 200x150 image, 90,000 bytes of raw pixels, comes out at
|
||||||
|
// a small fraction of that.
|
||||||
|
func TestImageProcessor_PNGIsCompressed(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
const width, height = 200, 150
|
||||||
|
|
||||||
|
flat := image.NewRGBA(image.Rect(0, 0, width, height))
|
||||||
|
for y := range height {
|
||||||
|
for x := range width {
|
||||||
|
flat.Set(x, y, color.RGBA{R: 40, G: 120, B: 200, A: 255})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
size := processedSize(t, encodePNG(t, flat), &Request{Format: FormatPNG})
|
||||||
|
|
||||||
|
const rawBytes = width * height * 3
|
||||||
|
if size > rawBytes/10 {
|
||||||
|
t.Errorf("PNG output is %d bytes, want under a tenth of its %d raw",
|
||||||
|
size, rawBytes)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestImageProcessor_WebPAtDefaultEffort verifies that WebP is saved at
|
||||||
|
// libvips' default effort, 4: the output is the size of the same image saved
|
||||||
|
// at that effort.
|
||||||
|
func TestImageProcessor_WebPAtDefaultEffort(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
input := createTestJPEG(t, 200, 150)
|
||||||
|
|
||||||
|
size := processedSize(t, input, &Request{Format: FormatWebP, Quality: 85})
|
||||||
|
|
||||||
|
want, _, err := decode(t, input).ExportWebp(&vips.WebpExportParams{
|
||||||
|
StripMetadata: true,
|
||||||
|
Quality: 85,
|
||||||
|
ReductionEffort: 4,
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("ExportWebp() error = %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if size != int64(len(want)) {
|
||||||
|
t.Errorf("WebP output is %d bytes, want %d, the size at effort 4",
|
||||||
|
size, len(want))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestImageProcessor_AVIFAtEffort1 verifies that AVIF is saved at effort 1
|
||||||
|
// with 8 bits per sample: the output is the size of the same image saved
|
||||||
|
// with those settings. The source is 16-bit, which libvips saves with 12
|
||||||
|
// bits when it is not given a bit depth, and 640x480: on a small image,
|
||||||
|
// such as 64x48, efforts 1 and 2 give the same output.
|
||||||
|
func TestImageProcessor_AVIFAtEffort1(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
const width, height = 640, 480
|
||||||
|
|
||||||
|
source := image.NewRGBA64(image.Rect(0, 0, width, height))
|
||||||
|
for y := range height {
|
||||||
|
for x := range width {
|
||||||
|
source.Set(x, y, color.RGBA64{
|
||||||
|
R: uint16((x * 65535 / width) & 0xffff),
|
||||||
|
G: uint16((y * 65535 / height) & 0xffff),
|
||||||
|
B: 32768,
|
||||||
|
A: 65535,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
input := encodePNG(t, source)
|
||||||
|
|
||||||
|
size := processedSize(t, input, &Request{Format: FormatAVIF, Quality: 85})
|
||||||
|
|
||||||
|
want, _, err := decode(t, input).ExportAvif(&vips.AvifExportParams{
|
||||||
|
StripMetadata: true,
|
||||||
|
Quality: 85,
|
||||||
|
Effort: 1,
|
||||||
|
Bitdepth: 8,
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("ExportAvif() error = %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if size != int64(len(want)) {
|
||||||
|
t.Errorf("AVIF output is %d bytes, want %d, the size at effort 1 "+
|
||||||
|
"and 8 bits", size, len(want))
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -256,9 +256,9 @@ func (p *ImageProcessor) Process(
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// Determine output format
|
// orig is the source's own format; encode refuses an empty format
|
||||||
outputFormat := req.Format
|
outputFormat := req.Format
|
||||||
if outputFormat == FormatOriginal || outputFormat == "" {
|
if outputFormat == FormatOriginal {
|
||||||
outputFormat = p.formatFromString(inputFormat)
|
outputFormat = p.formatFromString(inputFormat)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -504,36 +504,21 @@ func (p *ImageProcessor) encode(
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
var params vips.ExportParams
|
|
||||||
|
|
||||||
switch format {
|
switch format {
|
||||||
case FormatJPEG:
|
case FormatJPEG:
|
||||||
params = vips.ExportParams{
|
return exportJPEG(img, quality)
|
||||||
Format: vips.ImageTypeJPEG,
|
|
||||||
Quality: quality,
|
|
||||||
}
|
|
||||||
|
|
||||||
case FormatPNG:
|
case FormatPNG:
|
||||||
params = vips.ExportParams{
|
return exportPNG(img)
|
||||||
Format: vips.ImageTypePNG,
|
|
||||||
}
|
|
||||||
|
|
||||||
case FormatGIF:
|
case FormatGIF:
|
||||||
params = vips.ExportParams{
|
return exportGIF(img)
|
||||||
Format: vips.ImageTypeGIF,
|
|
||||||
}
|
|
||||||
|
|
||||||
case FormatWebP:
|
case FormatWebP:
|
||||||
params = vips.ExportParams{
|
return exportWebP(img, quality)
|
||||||
Format: vips.ImageTypeWEBP,
|
|
||||||
Quality: quality,
|
|
||||||
}
|
|
||||||
|
|
||||||
case FormatAVIF:
|
case FormatAVIF:
|
||||||
params = vips.ExportParams{
|
return exportAVIF(img, quality)
|
||||||
Format: vips.ImageTypeAVIF,
|
|
||||||
Quality: quality,
|
|
||||||
}
|
|
||||||
|
|
||||||
case FormatJXL:
|
case FormatJXL:
|
||||||
return exportJXL(img, quality)
|
return exportJXL(img, quality)
|
||||||
@@ -544,17 +529,90 @@ func (p *ImageProcessor) encode(
|
|||||||
default:
|
default:
|
||||||
return nil, fmt.Errorf("%w: %s", ErrUnsupportedOutputFormat, format)
|
return nil, fmt.Errorf("%w: %s", ErrUnsupportedOutputFormat, format)
|
||||||
}
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// Drop EXIF, XMP, IPTC and the ICC profile. govips ignores this for
|
// govips sends libvips Go's zero value for some settings an export leaves
|
||||||
// GIF, which carries none of them.
|
// out, such as no compression at all for PNG, so each export below sets
|
||||||
params.StripMetadata = true
|
// every setting whose zero value is not what pixa wants. Stripping metadata
|
||||||
|
// drops EXIF, XMP, IPTC and the ICC profile.
|
||||||
|
|
||||||
output, _, err := img.Export(¶ms)
|
// exportJPEG encodes img as JPEG at quality, without metadata. The settings
|
||||||
if err != nil {
|
// it leaves out are at libvips' defaults.
|
||||||
return nil, err
|
func exportJPEG(img *vips.ImageRef, quality int) ([]byte, error) {
|
||||||
}
|
output, _, err := img.ExportJpeg(&vips.JpegExportParams{
|
||||||
|
StripMetadata: true,
|
||||||
|
Quality: quality,
|
||||||
|
})
|
||||||
|
|
||||||
return output, nil
|
return output, err
|
||||||
|
}
|
||||||
|
|
||||||
|
// pngCompression is libvips' default PNG compression, from 0 (none) to 9.
|
||||||
|
const pngCompression = 6
|
||||||
|
|
||||||
|
// exportPNG encodes img as PNG at libvips' default compression and row
|
||||||
|
// filter, without metadata.
|
||||||
|
func exportPNG(img *vips.ImageRef) ([]byte, error) {
|
||||||
|
output, _, err := img.ExportPng(&vips.PngExportParams{
|
||||||
|
StripMetadata: true,
|
||||||
|
Compression: pngCompression,
|
||||||
|
Filter: vips.PngFilterNone,
|
||||||
|
})
|
||||||
|
|
||||||
|
return output, err
|
||||||
|
}
|
||||||
|
|
||||||
|
// gifEffort is libvips' default GIF effort, from 1 to 10.
|
||||||
|
const gifEffort = 7
|
||||||
|
|
||||||
|
// exportGIF encodes img as GIF at libvips' default effort. govips cannot
|
||||||
|
// have libvips strip metadata from GIF, which carries none.
|
||||||
|
func exportGIF(img *vips.ImageRef) ([]byte, error) {
|
||||||
|
output, _, err := img.ExportGIF(&vips.GifExportParams{Effort: gifEffort})
|
||||||
|
|
||||||
|
return output, err
|
||||||
|
}
|
||||||
|
|
||||||
|
// webpEffort is libvips' default WebP effort, from 0 (fastest) to 6.
|
||||||
|
const webpEffort = 4
|
||||||
|
|
||||||
|
// exportWebP encodes img as lossy WebP at quality and libvips' default
|
||||||
|
// effort, without metadata.
|
||||||
|
func exportWebP(img *vips.ImageRef, quality int) ([]byte, error) {
|
||||||
|
output, _, err := img.ExportWebp(&vips.WebpExportParams{
|
||||||
|
StripMetadata: true,
|
||||||
|
Quality: quality,
|
||||||
|
ReductionEffort: webpEffort,
|
||||||
|
})
|
||||||
|
|
||||||
|
return output, err
|
||||||
|
}
|
||||||
|
|
||||||
|
// avifEffort is the AVIF effort, from 0 (fastest) to 9; 1 is the lowest
|
||||||
|
// govips can set. With one thread, as pixad runs libvips, 1 takes about 51
|
||||||
|
// seconds to save an 8192x8192 image of random pixels, the worst case,
|
||||||
|
// against the default downstream_timeout of 60 seconds. On an image of
|
||||||
|
// milder noise, which 1 saves in about 12 seconds, 2 takes nearly a minute
|
||||||
|
// and libvips' default, 4, takes minutes.
|
||||||
|
const avifEffort = 1
|
||||||
|
|
||||||
|
// avifBitdepth is the AVIF bit depth, 8 bits per sample for every image.
|
||||||
|
// libvips would save a 16-bit image with 12, but at avifEffort that takes
|
||||||
|
// about 54 seconds for a 16-bit 8192x8192 image of milder noise, nearly all
|
||||||
|
// of the default downstream_timeout, and about 12 seconds with 8.
|
||||||
|
const avifBitdepth = 8
|
||||||
|
|
||||||
|
// exportAVIF encodes img as lossy AVIF at quality, avifEffort and
|
||||||
|
// avifBitdepth, without metadata.
|
||||||
|
func exportAVIF(img *vips.ImageRef, quality int) ([]byte, error) {
|
||||||
|
output, _, err := img.ExportAvif(&vips.AvifExportParams{
|
||||||
|
StripMetadata: true,
|
||||||
|
Quality: quality,
|
||||||
|
Effort: avifEffort,
|
||||||
|
Bitdepth: avifBitdepth,
|
||||||
|
})
|
||||||
|
|
||||||
|
return output, err
|
||||||
}
|
}
|
||||||
|
|
||||||
// jxlResolution is the resolution every JPEG XL image is saved with, in
|
// jxlResolution is the resolution every JPEG XL image is saved with, in
|
||||||
|
|||||||
@@ -100,8 +100,8 @@ func NewService(cfg *ServiceConfig) (*Service, error) {
|
|||||||
allowHTTP = cfg.FetcherConfig.AllowHTTP
|
allowHTTP = cfg.FetcherConfig.AllowHTTP
|
||||||
}
|
}
|
||||||
|
|
||||||
// JPEG XL is to become the default output format, so pixad does not
|
// JPEG XL is the default output format, so pixad does not start
|
||||||
// start without it.
|
// without it.
|
||||||
err := imageprocessor.CheckJPEGXLSupport()
|
err := imageprocessor.CheckJPEGXLSupport()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
|
|||||||
@@ -38,8 +38,10 @@ func ValidateDimension(name string, value int) error {
|
|||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
// sizeFormatRegex matches patterns like "800x600.webp", "0x0.jpeg", "orig.png"
|
// sizeFormatRegex matches patterns like "800x600.webp", "0x0.jpeg", "orig.png",
|
||||||
var sizeFormatRegex = regexp.MustCompile(`^(\d+)x(\d+)\.(\w+)$|^(orig)\.(\w+)$`)
|
// and a size with no format, such as "800x600" or "orig"
|
||||||
|
var sizeFormatRegex = regexp.MustCompile(
|
||||||
|
`^(\d+)x(\d+)(?:\.(\w+))?$|^(orig)(?:\.(\w+))?$`)
|
||||||
|
|
||||||
// ParsedURL contains the parsed components of an image proxy URL.
|
// ParsedURL contains the parsed components of an image proxy URL.
|
||||||
type ParsedURL struct {
|
type ParsedURL struct {
|
||||||
@@ -56,12 +58,13 @@ type ParsedURL struct {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// ParseImagePath parses the path captured by chi's wildcard:
|
// ParseImagePath parses the path captured by chi's wildcard:
|
||||||
// <host>/<path>/<size>.<format>
|
// <host>/<path>/<size>.<format>, or <host>/<path>/<size> for JPEG XL
|
||||||
// This is the primary entry point when using chi routing.
|
// This is the primary entry point when using chi routing.
|
||||||
// Examples:
|
// Examples:
|
||||||
// - cdn.example.com/photos/cat.jpg/800x600.webp
|
// - cdn.example.com/photos/cat.jpg/800x600.webp
|
||||||
// - cdn.example.com/photos/cat.jpg/0x0.jpeg
|
// - cdn.example.com/photos/cat.jpg/0x0.jpeg
|
||||||
// - cdn.example.com/photos/cat.jpg/orig.png
|
// - cdn.example.com/photos/cat.jpg/orig.png
|
||||||
|
// - cdn.example.com/photos/cat.jpg/800x600
|
||||||
func ParseImagePath(path string) (*ParsedURL, error) {
|
func ParseImagePath(path string) (*ParsedURL, error) {
|
||||||
// Strip leading slash if present (chi may include it)
|
// Strip leading slash if present (chi may include it)
|
||||||
path = strings.TrimPrefix(path, "/")
|
path = strings.TrimPrefix(path, "/")
|
||||||
@@ -72,7 +75,8 @@ func ParseImagePath(path string) (*ParsedURL, error) {
|
|||||||
return parseImageComponents(path)
|
return parseImageComponents(path)
|
||||||
}
|
}
|
||||||
|
|
||||||
// ParseImageURL parses a full URL path like /v1/image/<host>/<path>/<size>.<format>
|
// ParseImageURL parses a full URL path like /v1/image/<host>/<path>/<size>.<format>,
|
||||||
|
// or /v1/image/<host>/<path>/<size> for JPEG XL
|
||||||
// Use ParseImagePath instead when working with chi's wildcard capture.
|
// Use ParseImagePath instead when working with chi's wildcard capture.
|
||||||
func ParseImageURL(urlPath string) (*ParsedURL, error) {
|
func ParseImageURL(urlPath string) (*ParsedURL, error) {
|
||||||
// Remove the /v1/image/ prefix
|
// Remove the /v1/image/ prefix
|
||||||
@@ -89,7 +93,8 @@ func ParseImageURL(urlPath string) (*ParsedURL, error) {
|
|||||||
return parseImageComponents(remainder)
|
return parseImageComponents(remainder)
|
||||||
}
|
}
|
||||||
|
|
||||||
// parseImageComponents parses <host>/<path>/<size>.<format> structure.
|
// parseImageComponents parses <host>/<path>/<size>.<format>, or
|
||||||
|
// <host>/<path>/<size> for JPEG XL.
|
||||||
func parseImageComponents(remainder string) (*ParsedURL, error) {
|
func parseImageComponents(remainder string) (*ParsedURL, error) {
|
||||||
// Check for path traversal before any other processing
|
// Check for path traversal before any other processing
|
||||||
err := checkPathTraversal(remainder)
|
err := checkPathTraversal(remainder)
|
||||||
@@ -97,7 +102,7 @@ func parseImageComponents(remainder string) (*ParsedURL, error) {
|
|||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
|
|
||||||
// Find the last path segment which contains size.format
|
// Find the last path segment, which holds "size" or "size.format"
|
||||||
lastSlash := strings.LastIndex(remainder, "/")
|
lastSlash := strings.LastIndex(remainder, "/")
|
||||||
if lastSlash == -1 {
|
if lastSlash == -1 {
|
||||||
return nil, ErrMissingSize
|
return nil, ErrMissingSize
|
||||||
@@ -212,7 +217,7 @@ func checkPathTraversal(path string) error {
|
|||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
// parseSizeFormat parses strings like "800x600.webp" or "orig.png"
|
// parseSizeFormat parses strings like "800x600.webp", "orig.png" or "800x600"
|
||||||
func parseSizeFormat(s string) (Size, ImageFormat, error) {
|
func parseSizeFormat(s string) (Size, ImageFormat, error) {
|
||||||
matches := sizeFormatRegex.FindStringSubmatch(s)
|
matches := sizeFormatRegex.FindStringSubmatch(s)
|
||||||
if matches == nil {
|
if matches == nil {
|
||||||
@@ -225,11 +230,11 @@ func parseSizeFormat(s string) (Size, ImageFormat, error) {
|
|||||||
)
|
)
|
||||||
|
|
||||||
if matches[4] == "orig" {
|
if matches[4] == "orig" {
|
||||||
// "orig.format" pattern
|
// "orig" or "orig.format" pattern
|
||||||
size = Size{Width: 0, Height: 0}
|
size = Size{Width: 0, Height: 0}
|
||||||
formatStr = matches[5]
|
formatStr = matches[5]
|
||||||
} else {
|
} else {
|
||||||
// "WxH.format" pattern
|
// "WxH" or "WxH.format" pattern
|
||||||
width, err := strconv.Atoi(matches[1])
|
width, err := strconv.Atoi(matches[1])
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return Size{}, "", ErrInvalidSize
|
return Size{}, "", ErrInvalidSize
|
||||||
@@ -254,6 +259,11 @@ func parseSizeFormat(s string) (Size, ImageFormat, error) {
|
|||||||
return Size{}, "", err
|
return Size{}, "", err
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// A URL that names no format is served, and signed, as JPEG XL
|
||||||
|
if formatStr == "" {
|
||||||
|
return size, FormatJXL, nil
|
||||||
|
}
|
||||||
|
|
||||||
format, err := parseFormat(formatStr)
|
format, err := parseFormat(formatStr)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return Size{}, "", err
|
return Size{}, "", err
|
||||||
|
|||||||
@@ -89,7 +89,8 @@ func (s *Server) SetupRoutes() {
|
|||||||
r.Use(s.refuseDuringMaintenance)
|
r.Use(s.refuseDuringMaintenance)
|
||||||
|
|
||||||
// Main image proxy route
|
// Main image proxy route
|
||||||
// /v1/image/<host>/<path>/<width>x<height>.<format>
|
// /v1/image/<host>/<path>/<width>x<height>.<format>, or with no
|
||||||
|
// format /v1/image/<host>/<path>/<width>x<height>
|
||||||
r.Get("/image/*", s.h.HandleImage())
|
r.Get("/image/*", s.h.HandleImage())
|
||||||
r.Head("/image/*", s.h.HandleImage())
|
r.Head("/image/*", s.h.HandleImage())
|
||||||
|
|
||||||
|
|||||||
@@ -100,7 +100,7 @@
|
|||||||
<option value="png" {{if eq .FormFormat "png"}}selected{{end}}>PNG</option>
|
<option value="png" {{if eq .FormFormat "png"}}selected{{end}}>PNG</option>
|
||||||
<option value="webp" {{if eq .FormFormat "webp"}}selected{{end}}>WebP</option>
|
<option value="webp" {{if eq .FormFormat "webp"}}selected{{end}}>WebP</option>
|
||||||
<option value="avif" {{if eq .FormFormat "avif"}}selected{{end}}>AVIF</option>
|
<option value="avif" {{if eq .FormFormat "avif"}}selected{{end}}>AVIF</option>
|
||||||
<option value="jxl" {{if eq .FormFormat "jxl"}}selected{{end}}>JPEG XL</option>
|
<option value="jxl" {{if or (eq .FormFormat "jxl") (eq .FormFormat "")}}selected{{end}}>JPEG XL</option>
|
||||||
<option value="gif" {{if eq .FormFormat "gif"}}selected{{end}}>GIF</option>
|
<option value="gif" {{if eq .FormFormat "gif"}}selected{{end}}>GIF</option>
|
||||||
</select>
|
</select>
|
||||||
</div>
|
</div>
|
||||||
|
|||||||
Reference in New Issue
Block a user