Serve the format auto, chosen from the Accept header (closes #88) #220

Merged
clawbot merged 2 commits from issue-88-format-auto into next 2026-10-05 05:24:51 +02:00
9 changed files with 189 additions and 14 deletions
Showing only changes of commit 9a5ef06c80 - Show all commits
+32 -10
View File
@@ -175,12 +175,14 @@ path under `/v1/` answers 200, in maintenance mode too.
- `GET` or `HEAD` `/v1/image/<host>/<path>/<size>.<format>` — an image, fetched,
resized and converted (below). Needs: a signature, unless the host is
allowlisted (see Source Hosts). Answers: 200; 304 when `If-None-Match` matches
the image's `ETag`; 400 for a URL or parameter that is not valid; 401 for a
missing or wrong signature, a missing `exp` or an `exp` in the past; 403 when
the request's `Referer` names a host in `referer_blocklist`, checked before
the signature, the cache and the upstream fetch; 403 when the upstream host,
or a host it redirects to, is `localhost`, ends in `.localhost` or `.local`,
or has an address in a blocked network (see `blocked_networks`); 502 when the
the image's `ETag`; 400 for a URL or parameter that is not valid, or for the
format `auto` an `Accept` header that is not valid; 406 for the format `auto`
when `Accept` allows none of the formats it chooses from; 401 for a missing or
wrong signature, a missing `exp` or an `exp` in the past; 403 when the
request's `Referer` names a host in `referer_blocklist`, checked before the
signature, the cache and the upstream fetch; 403 when the upstream host, or a
host it redirects to, is `localhost`, ends in `.localhost` or `.local`, or has
an address in a blocked network (see `blocked_networks`); 502 when the
upstream answered with an error status, and for 5 minutes after that for the
same source URL; 503 when pixa is busy or in maintenance mode; 500 for any
other failure.
@@ -190,7 +192,8 @@ path under `/v1/` answers 200, in maintenance mode too.
decrypt, or that asks for a size or fit that is not valid; 410 once it has
expired; 504 when the upstream has not sent its response headers within
`upstream_fetch_timeout`, but 500 when that time runs out while the image
itself is still arriving; 403, 502, 503 and 500 as for `/v1/image/`.
itself is still arriving; 400 for an `Accept` header that is not valid, and
406, 403, 502, 503 and 500, as for `/v1/image/`.
- `GET /robots.txt` — asks every crawler to stay away (`Disallow: /`). Needs:
nothing. Answers: 200.
- `GET /.well-known/healthcheck.json` — JSON with `status` (`ok`), `now`,
@@ -239,7 +242,7 @@ A request whose query string cannot be decoded, or gives any parameter more than
once, is refused with 400.
- `<format>`: one of `orig` (or `original`), `jpeg` (or `jpg`), `png`, `webp`,
`avif`, `gif`
`avif`, `gif`, or `auto` (below)
- `<size>`: `orig` or `<width>x<height>` (e.g. `800x600`)
- `sig` and `exp`: the signature and its expiry, needed unless the host is
allowlisted (see Signature Specification)
@@ -247,6 +250,24 @@ once, is refused with 400.
both optional (values under Signature Specification). Both are part of what is
cached, so each value of either is a separate cached image.
With the format `auto`, pixa chooses the format for each request from its
`Accept` header, in this order:
1. AVIF, when the header names `image/avif`;
2. WebP, when it names `image/webp`;
3. JPEG, when the first of `image/jpeg`, `image/*` and `*/*` that it names
allows it, or when there is no `Accept` header or it is empty.
An entry with `q=0` refuses its format; other `q` values do not change the
order. AVIF and WebP must be named, as clients that cannot show them also send
`image/*` and `*/*`. pixa never sends a format the client refused: when the
header allows none of the three, the answer is 406, and a header that does not
parse, or has a `q` that is not a number from 0 to 1, is refused with 400. The
signature, or the token of an encrypted URL, covers `auto` itself, so one URL
serves every client. Each format chosen is cached as a separate image, and every
answer that depends on `Accept` (the image, a 304, and the 400 and 406 above)
carries `Vary: Accept`, so a shared cache keeps the formats apart too.
An image is served with `Cache-Control: public, max-age=<seconds>, immutable`.
When the URL has an expiry (an `exp`, or the TTL of an encrypted URL), `max-age`
is the whole seconds left until then, at most one year, so no browser or proxy
@@ -297,7 +318,7 @@ nor change what it asks for.
3. The page shows the URL, `https://<host>/v1/e/<token>/img.<format>`, and when
it expires. `<host>` is the host the page was opened on, and the URL starts
with `http` instead while `debug` is on. The name after the token is ignored
and only gives the URL a file extension, `jpg` for `orig`.
and only gives the URL a file extension, `jpg` for `orig` and `auto`.
The token holds the source's host, path and query and the size, format, quality,
fit and expiry, encrypted with a key derived from `signing_key`. The source
@@ -353,7 +374,8 @@ Where:
- `width` — requested width in pixels, `0` for original
- `height` — requested height in pixels, `0` for original
- `format` — output format, one of those listed under Routes, with `original`
signed as `orig` and `jpg` as `jpeg`
signed as `orig` and `jpg` as `jpeg`; `auto` is signed as `auto`, not as the
format chosen for the request
- `expiration` — the URL's `exp` query parameter, the Unix timestamp when the
signature expires; a request whose `exp` is not a whole number, an empty
`exp=` included, is refused with 400
+11 -2
View File
@@ -30,6 +30,17 @@ P2: security: per-IP rate limiting on the image routes
# Completed Steps
- 2026-10-05 the format `auto` (closes #88): a format in the `/v1/image/` path,
an encrypted URL's token and the generator page's format choice, chosen for
each request from `Accept` once the signature or token is checked: AVIF when
the header names `image/avif`, else WebP when it names `image/webp`, else JPEG
when the first of `image/jpeg`, `image/*` and `*/*` that it names allows it,
or when it names nothing; `q=0` refuses a format. AVIF and WebP must be named,
as clients that cannot show them send the wildcards too. A header that allows
none of the three answers 406, one that does not parse 400. The signature and
the token cover `auto` itself; the cache key and `ETag` use the format chosen.
Answers from the point the format is chosen carry `Vary: Accept`, next to the
CORS `Vary: Origin`; fixed-format answers do not.
- 2026-10-05 the markdown is formatted with prettier (closes #100): `script/fmt`
and `script/fmt-check` run prettier 3.8.1, pinned in `package.json` and
`yarn.lock`, on `**/*.md` after `gofmt`, with four-space tabs and
@@ -661,8 +672,6 @@ P2: security: per-IP rate limiting on the image routes
- per-origin rate limiting
- P2: HTTP response handling
- Last-Modified headers
- Vary header for content negotiation
- P2: auto format selection (format=auto based on Accept header)
- P2: configuration
- YAML config file support
- P2: operational
+1 -1
View File
@@ -371,7 +371,7 @@ func (s *Handlers) buildGeneratedURL(r *http.Request, token, format string) stri
// Determine file extension for the trailing filename
ext := format
if ext == "" || ext == "orig" {
if ext == "" || ext == "orig" || ext == "auto" {
ext = "jpg" // Default extension
}
+131
View File
@@ -0,0 +1,131 @@
package handlers
import (
"errors"
"fmt"
"mime"
"net/http"
"strconv"
"strings"
"sneak.berlin/go/pixa/internal/imgcache"
)
// Errors for an Accept header that an auto URL cannot be served for.
var (
errInvalidAccept = errors.New("invalid Accept header")
errNotAcceptable = errors.New(
"not acceptable: auto serves image/avif, image/webp or image/jpeg")
)
// chooseAutoFormat replaces the format auto in req with the format
// formatForAccept chooses from r's Accept header, and adds Vary: Accept to the
// response, which then depends on that header. It answers 400 for an Accept
// header that is not valid and 406 for one that allows none of the formats,
// and reports whether req can be served. Any other format is left as it is.
func (s *Handlers) chooseAutoFormat(
w http.ResponseWriter, r *http.Request, req *imgcache.ImageRequest,
) bool {
if req.Format != imgcache.FormatAuto {
return true
}
w.Header().Add("Vary", "Accept")
format, err := formatForAccept(strings.Join(r.Header.Values("Accept"), ","))
if errors.Is(err, errNotAcceptable) {
s.respondError(w, err.Error(), http.StatusNotAcceptable)
return false
}
if err != nil {
s.respondError(w, err.Error(), http.StatusBadRequest)
return false
}
req.Format = format
return true
}
// formatForAccept returns the format an auto URL is served in for the Accept
// header accept: AVIF when it names image/avif, else WebP when it names
// image/webp, else JPEG when its most specific entry of image/jpeg, image/*
// and */* allows it, or when it names nothing. A q of 0 refuses a format.
// AVIF and WebP must be named, as clients that cannot show them send image/*
// and */* too.
func formatForAccept(accept string) (imgcache.ImageFormat, error) {
qualities, err := parseAccept(accept)
if err != nil {
return "", err
}
if len(qualities) == 0 {
return imgcache.FormatJPEG, nil
}
if qualities["image/avif"] > 0 {
return imgcache.FormatAVIF, nil
}
if qualities["image/webp"] > 0 {
return imgcache.FormatWebP, nil
}
// For JPEG, the most specific entry the header has decides
quality, named := qualities["image/jpeg"]
if !named {
quality, named = qualities["image/*"]
}
if !named {
quality = qualities["*/*"]
}
if quality > 0 {
return imgcache.FormatJPEG, nil
}
return "", errNotAcceptable
}
// parseAccept returns the q of each media range the Accept header accept
// names, 1 where it gives none. A media range named more than once keeps its
// lowest q, so a refusal is never overridden. A media range that does not
// parse, or a q that is not a number from 0 to 1, is an error.
func parseAccept(accept string) (map[string]float64, error) {
qualities := make(map[string]float64)
for entry := range strings.SplitSeq(accept, ",") {
// A header field list may hold empty entries
if strings.TrimSpace(entry) == "" {
continue
}
mediaRange, params, err := mime.ParseMediaType(entry)
if err != nil {
return nil, fmt.Errorf("%w: %q: %w", errInvalidAccept, entry, err)
}
quality := 1.0
if qParam, given := params["q"]; given {
quality, err = strconv.ParseFloat(qParam, 64)
inRange := quality >= 0 && quality <= 1
if err != nil || !inRange {
return nil, fmt.Errorf("%w: %q: q is not a number from 0 to 1",
errInvalidAccept, entry)
}
}
previous, named := qualities[mediaRange]
if !named || quality < previous {
qualities[mediaRange] = quality
}
}
return qualities, nil
}
+5
View File
@@ -43,6 +43,11 @@ func (s *Handlers) HandleImage() http.HandlerFunc {
return
}
// The signature covers the format auto, not the format chosen
if !s.chooseAutoFormat(w, r, req) {
return
}
// Get cache key for logging
cacheKey := imgcache.CacheKey(req)
+1 -1
View File
@@ -30,7 +30,7 @@ func (s *Handlers) HandleImageEnc() http.HandlerFunc {
start := time.Now()
req, ok := s.parseImageEncRequest(w, r)
if !ok {
if !ok || !s.chooseAutoFormat(w, r, req) {
return
}
+5
View File
@@ -22,6 +22,11 @@ const (
FormatWebP ImageFormat = "webp"
FormatAVIF ImageFormat = "avif"
FormatGIF ImageFormat = "gif"
// FormatAuto stands for AVIF, WebP or JPEG, chosen for each request
// from its Accept header once the URL's signature or token has been
// checked; it is never processed or cached as itself.
FormatAuto ImageFormat = "auto"
)
// Size represents requested image dimensions
+2
View File
@@ -277,6 +277,8 @@ func parseFormat(s string) (ImageFormat, error) {
return FormatAVIF, nil
case "gif":
return FormatGIF, nil
case "auto":
return FormatAuto, nil
default:
return "", fmt.Errorf("%w: %s", ErrInvalidFormat, s)
}
+1
View File
@@ -95,6 +95,7 @@
</label>
<select id="format" name="format">
<option value="orig" {{if eq .FormFormat "orig"}}selected{{end}}>Original</option>
<option value="auto" {{if eq .FormFormat "auto"}}selected{{end}}>Auto (AVIF, WebP or JPEG)</option>
<option value="jpeg" {{if eq .FormFormat "jpeg"}}selected{{end}}>JPEG</option>
<option value="png" {{if eq .FormFormat "png"}}selected{{end}}>PNG</option>
<option value="webp" {{if eq .FormFormat "webp"}}selected{{end}}>WebP</option>