Serve JPEG XL when a request names no format (closes #222) #233

Merged
clawbot merged 1 commits from issue-222-jxl-default into next 2026-10-08 12:49:57 +02:00
12 changed files with 259 additions and 43 deletions
+29 -22
View File
@@ -175,20 +175,21 @@ path under `/v1/` answers 200, in maintenance mode too.
with the page naming a field that is not valid; 500 when the URL cannot be
made.
- `GET /logout` — end the login session. Needs: nothing. Answers: 303 to `/`.
- `GET` or `HEAD` `/v1/image/<host>/<path>/<size>.<format>` — an image, fetched,
resized and converted (below). Needs: a signature, unless the host is
allowlisted (see Source Hosts). Answers: 200; 304 when `If-None-Match` matches
the image's `ETag`; 400 for a URL or parameter that is not valid, or for the
format `auto` an `Accept` header that is not valid; 406 for the format `auto`
when `Accept` allows none of the formats it chooses from; 401 for a missing or
wrong signature, a missing `exp` or an `exp` in the past; 403 when the
request's `Referer` names a host in `referer_blocklist`, checked before the
signature, the cache and the upstream fetch; 403 when the upstream host, or a
host it redirects to, is `localhost`, ends in `.localhost` or `.local`, or has
an address in a blocked network (see `blocked_networks`); 502 when the
upstream answered with an error status, and for 5 minutes after that for the
same source URL; 503 when pixa is busy or in maintenance mode; 500 for any
other failure.
- `GET` or `HEAD` `/v1/image/<host>/<path>/<size>.<format>`, or
`/v1/image/<host>/<path>/<size>` with no format — an image, fetched, resized
and converted (below). Needs: a signature, unless the host is allowlisted (see
Source Hosts). Answers: 200; 304 when `If-None-Match` matches the image's
`ETag`; 400 for a URL or parameter that is not valid, or for the format `auto`
an `Accept` header that is not valid; 406 for the format `auto` when `Accept`
allows none of the formats it chooses from; 401 for a missing or wrong
signature, a missing `exp` or an `exp` in the past; 403 when the request's
`Referer` names a host in `referer_blocklist`, checked before the signature,
the cache and the upstream fetch; 403 when the upstream host, or a host it
redirects to, is `localhost`, ends in `.localhost` or `.local`, or has an
address in a blocked network (see `blocked_networks`); 502 when the upstream
answered with an error status, and for 5 minutes after that for the same
source URL; 503 when pixa is busy or in maintenance mode; 500 for any other
failure.
- `GET` or `HEAD` `/v1/e/<token>/<name>` — an image through an encrypted URL
(see Encrypted URLs). Needs: nothing but the URL. Answers: 200; 304 when
`If-None-Match` matches the image's `ETag`; 400 for a token that does not
@@ -231,10 +232,11 @@ proxy in front of pixa must pass that header on unchanged. A form body over 1
MiB is refused with 413. The image routes answer the errors listed for them with
JSON holding `error`, `status` and `timestamp`.
An image URL has this form:
An image URL has one of these forms, the second with no format:
```
/v1/image/<host>/<path>/<size>.<format>?sig=<signature>&exp=<expiration>&q=<quality>&fit=<fit>
/v1/image/<host>/<path>/<size>?sig=<signature>&exp=<expiration>&q=<quality>&fit=<fit>
```
Images are only fetched from origins using TLS with valid certificates, unless
@@ -245,7 +247,9 @@ A request whose query string cannot be decoded, or gives any parameter more than
once, is refused with 400.
- `<format>`: one of `orig` (or `original`), `jpeg` (or `jpg`), `png`, `webp`,
`avif`, `jxl` (JPEG XL), `gif`, or `auto` (below)
`avif`, `jxl` (JPEG XL), `gif`, or `auto` (below). A URL with no format (the
second form, with no dot after the size) is served as JPEG XL, the default, as
with `jxl`
- `<size>`: `orig` or `<width>x<height>` (e.g. `800x600`)
- `sig` and `exp`: the signature and its expiry, needed unless the host is
allowlisted (see Signature Specification)
@@ -321,13 +325,15 @@ nor change what it asks for.
lasts 30 days, or until `/logout`.
2. On the generator page, give the source image's URL, the width and height, the
format, quality and fit, and how long the URL lasts, then submit the form
(`POST /generate`). Width and height both empty or `0` keep the original
size; if only one of them is empty or `0`, that side is scaled to keep the
image's proportions.
(`POST /generate`). The format is JPEG XL unless another is chosen; a form
sent with an empty format, or none, also makes a JPEG XL URL. Width and
height both empty or `0` keep the original size; if only one of them is empty
or `0`, that side is scaled to keep the image's proportions.
3. The page shows the URL, `https://<host>/v1/e/<token>/img.<format>`, and when
it expires. `<host>` is the host the page was opened on, and the URL starts
with `http` instead while `debug` is on. The name after the token is ignored
and only gives the URL a file extension, `jpg` for `orig` and `auto`.
and only gives the URL a file extension, `jpg` for `orig` and `auto`, and
`jxl` for a form with no format.
The token holds the source's host, path and query and the size, format, quality,
fit and expiry, encrypted with a key derived from `signing_key`. The source
@@ -387,8 +393,9 @@ Where:
- `width` — requested width in pixels, `0` for original
- `height` — requested height in pixels, `0` for original
- `format` — output format, one of those listed under Routes, with `original`
signed as `orig` and `jpg` as `jpeg`; `auto` is signed as `auto`, not as the
format chosen for the request
signed as `orig`, `jpg` as `jpeg`, and no format as `jxl`, so a URL with no
format has the signature of the same URL ending in `.jxl`; `auto` is signed as
`auto`, not as the format chosen for the request
- `expiration` — the URL's `exp` query parameter, the Unix timestamp when the
signature expires; a request whose `exp` is not a whole number, an empty
`exp=` included, is refused with 400
+9
View File
@@ -30,6 +30,15 @@ P2: security: per-IP rate limiting on the image routes
# Completed Steps
- 2026-10-08 JPEG XL is the default output (closes #222): a `/v1/image/` URL
whose last segment is a size with no format, such as `800x600` or `orig`, is
served as JPEG XL and signed as `jxl`, so it has the signature of the same URL
ending in `.jxl`. An encrypted URL whose token holds no format is served as
JPEG XL (`encurl.DefaultFormat`). The generator page selects JPEG XL until
another format is chosen, and a form with an empty format, or none, makes a
JPEG XL URL whose name ends in `.jxl`. The image processor no longer takes an
empty format as `orig`: both routes give every request a format, and it
refuses a request with none. `auto` still ends with JPEG.
- 2026-10-08 JPEG XL as an input and output format (part of #222): a source
whose bytes start with either JPEG XL signature, the bare codestream's `FF 0A`
or the container's, is detected as `image/jxl`, which the upstream fetch
+2 -2
View File
@@ -14,7 +14,7 @@ import (
// Default values for optional fields.
const (
DefaultQuality = 85
DefaultFormat = imgcache.FormatOriginal
DefaultFormat = imgcache.FormatJXL
DefaultFitMode = imgcache.FitCover
// HKDF salt for URL encryption key derivation
@@ -37,7 +37,7 @@ type Payload struct {
SourceQuery string `cbor:"q,omitempty"` // optional
Width int `cbor:"w,omitempty"` // 0 = original
Height int `cbor:"ht,omitempty"` // 0 = original
Format imgcache.ImageFormat `cbor:"f,omitempty"` // default: orig
Format imgcache.ImageFormat `cbor:"f,omitempty"` // default: jxl
Quality int `cbor:"ql,omitempty"` // default: 85
FitMode imgcache.FitMode `cbor:"fm,omitempty"` // default: cover
ExpiresAt int64 `cbor:"e,omitempty"` // 0 = never expires
+8 -3
View File
@@ -369,10 +369,15 @@ func (s *Handlers) buildGeneratedURL(r *http.Request, token, format string) stri
scheme = "http"
}
// Determine file extension for the trailing filename
// Determine file extension for the trailing filename. A form with no
// format makes a token with none, which is served as encurl.DefaultFormat.
ext := format
if ext == "" || ext == "orig" || ext == "auto" {
ext = "jpg" // Default extension
switch format {
case "":
ext = string(encurl.DefaultFormat)
case "orig", "auto":
ext = "jpg"
}
return scheme + "://" + r.Host + "/v1/e/" + url.PathEscape(token) + "/img." + ext
+2 -1
View File
@@ -18,7 +18,8 @@ import (
)
// HandleImage handles the main image proxy route:
// /v1/image/<host>/<path>/<width>x<height>.<format>
// /v1/image/<host>/<path>/<width>x<height>.<format>, or with no format
// /v1/image/<host>/<path>/<width>x<height>
func (s *Handlers) HandleImage() http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
if s.refuseBlockedReferer(w, r) {
@@ -0,0 +1,162 @@
package handlers
import (
"fmt"
"log/slog"
"maps"
"net/http"
"net/http/httptest"
"net/url"
"strings"
"testing"
"time"
"github.com/davidbyttow/govips/v2/vips"
"sneak.berlin/go/pixa/internal/encurl"
"sneak.berlin/go/pixa/internal/imgcache"
"sneak.berlin/go/pixa/internal/signature"
)
// requireJPEGXL requires that rec answers 200 with a JPEG XL image.
func requireJPEGXL(t *testing.T, rec *httptest.ResponseRecorder) {
t.Helper()
if rec.Code != http.StatusOK {
t.Fatalf("status = %d, want %d; body %q",
rec.Code, http.StatusOK, rec.Body.String())
}
if got := rec.Header().Get("Content-Type"); got != jxlType {
t.Errorf("Content-Type = %q, want %s", got, jxlType)
}
if got := vips.DetermineImageType(rec.Body.Bytes()); got != vips.ImageTypeJXL {
t.Errorf("body is %s, want jxl", vips.ImageTypes[got])
}
}
// TestImageWithoutFormat_ServesJPEGXL verifies that a /v1/image/ URL whose
// last segment is a size with no format, 50x50 or orig, answers JPEG XL.
func TestImageWithoutFormat_ServesJPEGXL(t *testing.T) {
t.Parallel()
route := newImageRoute(t, newPhotoFetcher(t, allowlistedHost))
for _, size := range []string{"50x50", "orig"} {
target := "/v1/image/" + allowlistedHost + photoPath + "/" + size
requireJPEGXL(t, sendGet(t, route, target))
}
}
// TestImageWithoutFormat_SignedAsJXL verifies that a /v1/image/ URL with no
// format is signed as jxl: the signature made for the URL ending in .jxl is
// accepted for the same URL without .jxl.
func TestImageWithoutFormat_SignedAsJXL(t *testing.T) {
t.Parallel()
route := newImageRoute(t, newPhotoFetcher(t, signedHost))
expires := time.Now().Add(time.Hour)
sig := signature.New(testSigningKey).Sign(&signature.Request{
SourceHost: signedHost,
SourcePath: photoPath,
Width: 50,
Height: 50,
Format: string(imgcache.FormatJXL),
Quality: encurl.DefaultQuality,
FitMode: string(imgcache.FitCover),
Expires: expires,
})
query := fmt.Sprintf("?sig=%s&exp=%d", sig, expires.Unix())
for _, size := range []string{"50x50.jxl", "50x50"} {
target := "/v1/image/" + signedHost + photoPath + "/" + size + query
requireJPEGXL(t, sendGet(t, route, target))
}
}
// TestImageEncWithoutFormat_ServesJPEGXL verifies that an encrypted URL whose
// token holds no format answers JPEG XL.
func TestImageEncWithoutFormat_ServesJPEGXL(t *testing.T) {
t.Parallel()
h, srv := newSignedHostServer(t, slog.New(slog.DiscardHandler))
token, err := h.encGen.Generate(&encurl.Payload{
SourceHost: signedHost,
SourcePath: photoPath,
Width: 50,
Height: 50,
})
if err != nil {
t.Fatalf("Generate() error = %v", err)
}
requireJPEGXL(t, getEncToken(srv, token))
}
// TestGeneratorPage_SelectsJPEGXL verifies that the generator page's format
// choice is JPEG XL until another is chosen.
func TestGeneratorPage_SelectsJPEGXL(t *testing.T) {
t.Parallel()
h, srv := newCSRFTestRouter(t)
req := httptest.NewRequestWithContext(t.Context(), http.MethodGet, "/", nil)
req.AddCookie(newSessionCookie(t, h))
rec := httptest.NewRecorder()
srv.ServeHTTP(rec, req)
if !strings.Contains(rec.Body.String(), `<option value="jxl" selected>`) {
t.Errorf("generator page does not select JPEG XL: %s", rec.Body.String())
}
}
// TestGeneratePost_NoFormat_MakesJPEGXLURL verifies that the generator form
// sent with an empty format field, or with none, makes a URL whose name ends
// in .jxl and which answers JPEG XL.
func TestGeneratePost_NoFormat_MakesJPEGXLURL(t *testing.T) {
t.Parallel()
photo := url.Values{
sourceURLField: {"https://" + signedHost + photoPath},
widthField: {"50"},
heightField: {"50"},
}
emptyFormat := maps.Clone(photo)
emptyFormat.Set(formatField, "")
for name, form := range map[string]url.Values{
"empty format field": emptyFormat,
"no format field": photo,
} {
t.Run(name, func(t *testing.T) {
t.Parallel()
_, imageSrv := newSignedHostServer(t, slog.New(slog.DiscardHandler))
rec := generatePost(t, form)
match := generatedURLPattern.FindStringSubmatch(rec.Body.String())
if match == nil {
t.Fatalf("generator page shows no URL: %d %s",
rec.Code, rec.Body.String())
}
t.Logf("generated URL path: %s", match[1])
if !strings.HasSuffix(match[1], "/img.jxl") {
t.Errorf("generated URL %s does not end in /img.jxl", match[1])
}
imageRec := httptest.NewRecorder()
imageSrv.ServeHTTP(imageRec, httptest.NewRequestWithContext(
t.Context(), http.MethodGet, match[1], nil))
requireJPEGXL(t, imageRec)
})
}
}
@@ -0,0 +1,21 @@
package imageprocessor
import (
"bytes"
"errors"
"testing"
)
// TestImageProcessor_EmptyFormatRefused verifies that a request with no format
// is refused, as both image routes give every request a format before it is
// processed.
func TestImageProcessor_EmptyFormatRefused(t *testing.T) {
t.Parallel()
_, err := New(Params{}).Process(
t.Context(), bytes.NewReader(createTestJPEG(t, 20, 20)), &Request{},
)
if !errors.Is(err, ErrUnsupportedOutputFormat) {
t.Errorf("Process() error = %v, want %v", err, ErrUnsupportedOutputFormat)
}
}
+2 -2
View File
@@ -256,9 +256,9 @@ func (p *ImageProcessor) Process(
}
}
// Determine output format
// orig is the source's own format; encode refuses an empty format
outputFormat := req.Format
if outputFormat == FormatOriginal || outputFormat == "" {
if outputFormat == FormatOriginal {
outputFormat = p.formatFromString(inputFormat)
}
+2 -2
View File
@@ -100,8 +100,8 @@ func NewService(cfg *ServiceConfig) (*Service, error) {
allowHTTP = cfg.FetcherConfig.AllowHTTP
}
// JPEG XL is to become the default output format, so pixad does not
// start without it.
// JPEG XL is the default output format, so pixad does not start
// without it.
err := imageprocessor.CheckJPEGXLSupport()
if err != nil {
return nil, err
+19 -9
View File
@@ -38,8 +38,10 @@ func ValidateDimension(name string, value int) error {
return nil
}
// sizeFormatRegex matches patterns like "800x600.webp", "0x0.jpeg", "orig.png"
var sizeFormatRegex = regexp.MustCompile(`^(\d+)x(\d+)\.(\w+)$|^(orig)\.(\w+)$`)
// sizeFormatRegex matches patterns like "800x600.webp", "0x0.jpeg", "orig.png",
// and a size with no format, such as "800x600" or "orig"
var sizeFormatRegex = regexp.MustCompile(
`^(\d+)x(\d+)(?:\.(\w+))?$|^(orig)(?:\.(\w+))?$`)
// ParsedURL contains the parsed components of an image proxy URL.
type ParsedURL struct {
@@ -56,12 +58,13 @@ type ParsedURL struct {
}
// ParseImagePath parses the path captured by chi's wildcard:
// <host>/<path>/<size>.<format>
// <host>/<path>/<size>.<format>, or <host>/<path>/<size> for JPEG XL
// This is the primary entry point when using chi routing.
// Examples:
// - cdn.example.com/photos/cat.jpg/800x600.webp
// - cdn.example.com/photos/cat.jpg/0x0.jpeg
// - cdn.example.com/photos/cat.jpg/orig.png
// - cdn.example.com/photos/cat.jpg/800x600
func ParseImagePath(path string) (*ParsedURL, error) {
// Strip leading slash if present (chi may include it)
path = strings.TrimPrefix(path, "/")
@@ -72,7 +75,8 @@ func ParseImagePath(path string) (*ParsedURL, error) {
return parseImageComponents(path)
}
// ParseImageURL parses a full URL path like /v1/image/<host>/<path>/<size>.<format>
// ParseImageURL parses a full URL path like /v1/image/<host>/<path>/<size>.<format>,
// or /v1/image/<host>/<path>/<size> for JPEG XL
// Use ParseImagePath instead when working with chi's wildcard capture.
func ParseImageURL(urlPath string) (*ParsedURL, error) {
// Remove the /v1/image/ prefix
@@ -89,7 +93,8 @@ func ParseImageURL(urlPath string) (*ParsedURL, error) {
return parseImageComponents(remainder)
}
// parseImageComponents parses <host>/<path>/<size>.<format> structure.
// parseImageComponents parses <host>/<path>/<size>.<format>, or
// <host>/<path>/<size> for JPEG XL.
func parseImageComponents(remainder string) (*ParsedURL, error) {
// Check for path traversal before any other processing
err := checkPathTraversal(remainder)
@@ -97,7 +102,7 @@ func parseImageComponents(remainder string) (*ParsedURL, error) {
return nil, err
}
// Find the last path segment which contains size.format
// Find the last path segment, which holds "size" or "size.format"
lastSlash := strings.LastIndex(remainder, "/")
if lastSlash == -1 {
return nil, ErrMissingSize
@@ -212,7 +217,7 @@ func checkPathTraversal(path string) error {
return nil
}
// parseSizeFormat parses strings like "800x600.webp" or "orig.png"
// parseSizeFormat parses strings like "800x600.webp", "orig.png" or "800x600"
func parseSizeFormat(s string) (Size, ImageFormat, error) {
matches := sizeFormatRegex.FindStringSubmatch(s)
if matches == nil {
@@ -225,11 +230,11 @@ func parseSizeFormat(s string) (Size, ImageFormat, error) {
)
if matches[4] == "orig" {
// "orig.format" pattern
// "orig" or "orig.format" pattern
size = Size{Width: 0, Height: 0}
formatStr = matches[5]
} else {
// "WxH.format" pattern
// "WxH" or "WxH.format" pattern
width, err := strconv.Atoi(matches[1])
if err != nil {
return Size{}, "", ErrInvalidSize
@@ -254,6 +259,11 @@ func parseSizeFormat(s string) (Size, ImageFormat, error) {
return Size{}, "", err
}
// A URL that names no format is served, and signed, as JPEG XL
if formatStr == "" {
return size, FormatJXL, nil
}
format, err := parseFormat(formatStr)
if err != nil {
return Size{}, "", err
+2 -1
View File
@@ -89,7 +89,8 @@ func (s *Server) SetupRoutes() {
r.Use(s.refuseDuringMaintenance)
// Main image proxy route
// /v1/image/<host>/<path>/<width>x<height>.<format>
// /v1/image/<host>/<path>/<width>x<height>.<format>, or with no
// format /v1/image/<host>/<path>/<width>x<height>
r.Get("/image/*", s.h.HandleImage())
r.Head("/image/*", s.h.HandleImage())
+1 -1
View File
@@ -100,7 +100,7 @@
<option value="png" {{if eq .FormFormat "png"}}selected{{end}}>PNG</option>
<option value="webp" {{if eq .FormFormat "webp"}}selected{{end}}>WebP</option>
<option value="avif" {{if eq .FormFormat "avif"}}selected{{end}}>AVIF</option>
<option value="jxl" {{if eq .FormFormat "jxl"}}selected{{end}}>JPEG XL</option>
<option value="jxl" {{if or (eq .FormFormat "jxl") (eq .FormFormat "")}}selected{{end}}>JPEG XL</option>
<option value="gif" {{if eq .FormFormat "gif"}}selected{{end}}>GIF</option>
</select>
</div>