Document how a source query goes into an image URL, and accept it (closes #255)
check / check (push) Canceled after 0s

The query follows the source path after %3F. The image route now parses
the request's path decoded once instead of chi's wildcard, which holds
the path as the client wrote it whenever that is not Go's own encoding
of it, so a / ; or , written as %2F, %3B or %2C, as GenerateSignedURL
wrote them, hid the %3F and the request failed with 401. Now the ? and
every % in the source path or query must be percent-encoded and any
other character may be. GenerateSignedURL writes Go's own encoding of
the path. The README states the rule, with an example and its signature.

Model: opus-5-5
This commit is contained in:
2026-10-15 21:14:59 +00:00
parent 4634d2dc2f
commit f9c878161a
7 changed files with 74 additions and 48 deletions
+25
View File
@@ -316,6 +316,20 @@ once, is refused with 400.
both optional (values under Signature Specification). Both are part of what is both optional (values under Signature Specification). Both are part of what is
cached, so each value of either is a separate cached image. cached, so each value of either is a separate cached image.
A source URL's query follows its path after `%3F`, the encoded `?`:
`/v1/image/<host>/<path>%3F<query>/<size>.<format>`. pixa decodes the
percent-escapes in the image URL's path once, then reads the host up to the
first `/`, the size and format after the last `/`, and between them the source
path up to the first `?` and the source query after it. So the `?` before the
query, any `?` in the query, and every `%` in the source path or query must be
percent-encoded, as `%3F` and `%25`. Any other character may be written as
itself or percent-encoded: `/`, `;`, `,`, `=` and `&` work either way. For
example,
`https://cdn.example.com/img?url=https://origin.example.com/my%20cat.jpg` at
800x600 WebP is
`/v1/image/cdn.example.com/img%3Furl=https://origin.example.com/my%2520cat.jpg/800x600.webp`,
signed as shown under Signature Specification.
A size with one side `0` has that side scaled to keep the source's proportions, A size with one side `0` has that side scaled to keep the source's proportions,
and is refused with 400 when that side would come out above 8192, the most and is refused with 400 when that side would come out above 8192, the most
either side may be; so is a size with `fit=outside`, which scales the image to either side may be; so is a size with `fit=outside`, which scales the image to
@@ -535,6 +549,17 @@ For the same image at quality 40 with fit `contain`, the input ends in
and the URL is and the URL is
`/v1/image/cdn.example.com/photos/cat.jpg/800x600.webp?sig=5IwXUx6vf7yefhaUvFzgXZvG2o0Df4RJxPTK3pKq5VU=&exp=1704067200&q=40&fit=contain`. `/v1/image/cdn.example.com/photos/cat.jpg/800x600.webp?sig=5IwXUx6vf7yefhaUvFzgXZvG2o0Df4RJxPTK3pKq5VU=&exp=1704067200&q=40&fit=contain`.
For a source URL with a query,
`https://cdn.example.com/img?url=https://origin.example.com/my%20cat.jpg`, at
800x600 WebP with the same key and expiration and the default quality and fit:
1. Build input, with the query's `:` written as `%3A` and its `%` as `%25`:
`cdn.example.com:/img:url=https%3A//origin.example.com/my%2520cat.jpg:800:600:webp:1704067200:85:cover`
2. The signature is `37aFa9GSVCZM1kRY9SzMOk2fOg_M7Wt5gKNR4xM-FYA=`
3. URL, with the `?` before the query written as `%3F` and the query's `%` as
`%25` (see Routes):
`/v1/image/cdn.example.com/img%3Furl=https://origin.example.com/my%2520cat.jpg/800x600.webp?sig=37aFa9GSVCZM1kRY9SzMOk2fOg_M7Wt5gKNR4xM-FYA=&exp=1704067200`
**Allowlist patterns:** **Allowlist patterns:**
- **Exact match**: `cdn.example.com` — matches only that host - **Exact match**: `cdn.example.com` — matches only that host
+11
View File
@@ -30,6 +30,17 @@ P2: security: per-origin rate limiting
# Completed Steps # Completed Steps
- 2026-10-15 the README says how a source URL with a query is written into an
image URL, and the image route accepts it (closes #255): the query follows the
source path after `%3F`. pixa now reads the image URL's path decoded once, so
the `?` before the query, any `?` in it and every `%` in the source path or
query must be percent-encoded, and any other character may be. The route used
to parse chi's wildcard, which holds the path as the client wrote it whenever
that is not Go's own encoding of it; a `/`, `;` or `,` in the query written as
`%2F`, `%3B` or `%2C`, as `GenerateSignedURL` wrote them, kept the `%3F` from
being read as `?`, and the request failed with 401. `GenerateSignedURL` now
writes Go's own encoding of the path. The README has an example with a query
and its signature.
- 2026-10-15 `max_concurrent_processing` defaults to half the CPUs pixa sees, - 2026-10-15 `max_concurrent_processing` defaults to half the CPUs pixa sees,
rounded down, and at least 1 (closes #242): it defaulted to one image per CPU, rounded down, and at least 1 (closes #242): it defaulted to one image per CPU,
and a container with no CPU limit sees every CPU of the host, so on a 48-CPU and a container with no CPU limit sees every CPU of the host, so on a 48-CPU
+8 -8
View File
@@ -9,7 +9,6 @@ import (
"strconv" "strconv"
"time" "time"
"github.com/go-chi/chi/v5"
"github.com/go-chi/chi/v5/middleware" "github.com/go-chi/chi/v5/middleware"
"sneak.berlin/go/pixa/internal/encurl" "sneak.berlin/go/pixa/internal/encurl"
"sneak.berlin/go/pixa/internal/httpfetcher" "sneak.berlin/go/pixa/internal/httpfetcher"
@@ -80,20 +79,21 @@ func (s *Handlers) HandleRobotsTxt() http.HandlerFunc {
} }
} }
// parseImageRequest parses the wildcard path and query parameters into // parseImageRequest parses the image URL's path and query parameters into
// an ImageRequest. On invalid input it writes an error response and // an ImageRequest. On invalid input it writes an error response and
// returns false. // returns false.
func (s *Handlers) parseImageRequest( func (s *Handlers) parseImageRequest(
w http.ResponseWriter, r *http.Request, w http.ResponseWriter, r *http.Request,
) (*imgcache.ImageRequest, bool) { ) (*imgcache.ImageRequest, bool) {
// Get the wildcard path from chi // The path is parsed decoded once, as r.URL.Path holds it, and not from
pathParam := chi.URLParam(r, "*") // chi's wildcard: chi routes on the path as the client wrote it whenever
// that is not Go's own encoding of it, so its wildcard would still hold
// Parse the URL path // the %3F before a source query when, say, a "/" in the query is written
parsed, err := imgcache.ParseImagePath(pathParam) // as %2F
parsed, err := imgcache.ParseImageURL(r.URL.Path)
if err != nil { if err != nil {
s.log.Warn("failed to parse image URL", s.log.Warn("failed to parse image URL",
"path", pathParam, "path", r.URL.Path,
"error", err, "error", err,
) )
s.respondError(w, "invalid image URL: "+err.Error(), http.StatusBadRequest) s.respondError(w, "invalid image URL: "+err.Error(), http.StatusBadRequest)
@@ -198,7 +198,7 @@ func TestHandleImage_GeneratedSignedURLWithSourceQueryVerifies(t *testing.T) {
signedURL, err := fix.service.GenerateSignedURL("", &imgcache.ImageRequest{ signedURL, err := fix.service.GenerateSignedURL("", &imgcache.ImageRequest{
SourceHost: signedHost, SourceHost: signedHost,
SourcePath: "/images/photo.jpg", SourcePath: photoPath,
SourceQuery: tt.query, SourceQuery: tt.query,
Size: imgcache.Size{Width: 50, Height: 50}, Size: imgcache.Size{Width: 50, Height: 50},
Format: imgcache.FormatJPEG, Format: imgcache.FormatJPEG,
@@ -226,7 +226,7 @@ func TestHandleImage_SourceQueryEncodedAnyWay(t *testing.T) {
signer := signature.New("test-signing-key-must-be-32-chars") signer := signature.New("test-signing-key-must-be-32-chars")
sig := signer.Sign(&signature.Request{ sig := signer.Sign(&signature.Request{
SourceHost: signedHost, SourceHost: signedHost,
SourcePath: "/images/photo.jpg", SourcePath: photoPath,
SourceQuery: "url=https://origin.example.com/my%20cat.jpg", SourceQuery: "url=https://origin.example.com/my%20cat.jpg",
Width: 50, Width: 50,
Height: 50, Height: 50,
@@ -237,23 +237,23 @@ func TestHandleImage_SourceQueryEncodedAnyWay(t *testing.T) {
}) })
tests := []struct { tests := []struct {
name string name string
writtenSource string writtenQuery string
}{ }{
{"only ? and % encoded", {"only ? and % encoded",
"/images/photo.jpg%3Furl=https://origin.example.com/my%2520cat.jpg"}, "%3Furl=https://origin.example.com/my%2520cat.jpg"},
{"every reserved character encoded", {"every reserved character encoded",
"/images/photo.jpg%3Furl%3Dhttps%3A%2F%2Forigin.example.com%2Fmy%2520cat.jpg"}, "%3Furl%3Dhttps%3A%2F%2Forigin.example.com%2Fmy%2520cat.jpg"},
{"lower-case hex digits", {"lower-case hex digits",
"/images/photo.jpg%3furl=https://origin.example.com/my%2520cat.jpg"}, "%3furl=https://origin.example.com/my%2520cat.jpg"},
} }
for _, tt := range tests { for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) { t.Run(tt.name, func(t *testing.T) {
t.Parallel() t.Parallel()
target := fmt.Sprintf("/v1/image/%s%s/50x50.jpeg?sig=%s&exp=%d", target := fmt.Sprintf("/v1/image/%s%s%s/50x50.jpeg?sig=%s&exp=%d",
signedHost, tt.writtenSource, sig, expires.Unix()) signedHost, photoPath, tt.writtenQuery, sig, expires.Unix())
status := getImage(t, setupTestHandler(t), target) status := getImage(t, setupTestHandler(t), target)
if status != http.StatusBadGateway { if status != http.StatusBadGateway {
+6 -6
View File
@@ -61,9 +61,8 @@ type ParsedURL struct {
Format ImageFormat Format ImageFormat
} }
// ParseImagePath parses the path captured by chi's wildcard: // ParseImagePath parses an image URL's path after /v1/image/, decoded once:
// <host>/<path>/<size>.<format>, or <host>/<path>/<size> for JPEG XL // <host>/<path>/<size>.<format>, or <host>/<path>/<size> for JPEG XL.
// This is the primary entry point when using chi routing.
// Examples: // Examples:
// - cdn.example.com/photos/cat.jpg/800x600.webp // - cdn.example.com/photos/cat.jpg/800x600.webp
// - cdn.example.com/photos/cat.jpg/0x0.jpeg // - cdn.example.com/photos/cat.jpg/0x0.jpeg
@@ -79,9 +78,10 @@ func ParseImagePath(path string) (*ParsedURL, error) {
return parseImageComponents(path) return parseImageComponents(path)
} }
// ParseImageURL parses a full URL path like /v1/image/<host>/<path>/<size>.<format>, // ParseImageURL parses an image URL's path, decoded once as r.URL.Path holds
// or /v1/image/<host>/<path>/<size> for JPEG XL // it: /v1/image/<host>/<path>/<size>.<format>, or /v1/image/<host>/<path>/<size>
// Use ParseImagePath instead when working with chi's wildcard capture. // for JPEG XL. A source query follows the source path after a "?", which the
// URL writes as %3F.
func ParseImageURL(urlPath string) (*ParsedURL, error) { func ParseImageURL(urlPath string) (*ParsedURL, error) {
// Remove the /v1/image/ prefix // Remove the /v1/image/ prefix
const prefix = "/v1/image/" const prefix = "/v1/image/"
+11 -22
View File
@@ -126,31 +126,20 @@ func (s *Signer) GenerateSignedURL(
sizeStr = fmt.Sprintf("%dx%d", req.Width, req.Height) sizeStr = fmt.Sprintf("%dx%d", req.Width, req.Height)
} }
// Build the path. // Build the path. A source query follows the source path after a "?".
// When a source query is present, it is embedded as a path segment // The image route reads the path decoded once, so the whole path is
// (e.g. /host/path?query/size.fmt) so that the URL parser can extract // written in Go's own percent-encoding of a URL path, which writes "?"
// it from the last-slash split. The "?" inside a path segment is // as %3F and "%" as %25, and leaves "/", ";", ",", "=" and "&" as they
// percent-encoded by clients but chi delivers it decoded, which is // are.
// exactly what the URL parser expects. sourcePathAndQuery := req.SourcePath
var path string
if req.SourceQuery != "" { if req.SourceQuery != "" {
path = fmt.Sprintf("/v1/image/%s%s%%3F%s/%s.%s", sourcePathAndQuery += "?" + req.SourceQuery
req.SourceHost,
req.SourcePath,
url.PathEscape(req.SourceQuery),
sizeStr,
req.Format,
)
} else {
path = fmt.Sprintf("/v1/image/%s%s/%s.%s",
req.SourceHost,
req.SourcePath,
sizeStr,
req.Format,
)
} }
return path, sig, exp unescapedPath := fmt.Sprintf("/v1/image/%s%s/%s.%s",
req.SourceHost, sourcePathAndQuery, sizeStr, req.Format)
return (&url.URL{Path: unescapedPath}).EscapedPath(), sig, exp
} }
// buildSignatureData creates the string to be signed. // buildSignatureData creates the string to be signed.
+4 -3
View File
@@ -476,9 +476,10 @@ func TestGenerateSignedURL_WithoutQueryString(t *testing.T) {
} }
// TestGenerateSignedURL_ReadmeQueryExample checks the README's example of a // TestGenerateSignedURL_ReadmeQueryExample checks the README's example of a
// source URL with a query, https://cdn.example.com/img?url=https://origin.example.com/my%20cat.jpg: // source URL with a query, the image at
// its signature with the README's key, and the path GenerateSignedURL writes // https://cdn.example.com/img?url=https://origin.example.com/my%20cat.jpg: its
// for it, with the "?" written as %3F and the "%" as %25. // signature with the README's key, and the path GenerateSignedURL writes for
// it, with the "?" written as %3F and the "%" as %25.
func TestGenerateSignedURL_ReadmeQueryExample(t *testing.T) { func TestGenerateSignedURL_ReadmeQueryExample(t *testing.T) {
t.Parallel() t.Parallel()