Fetch the source path and query URL-encoded, not joined as text (closes #288)

The source path, decoded, is fetched percent-encoded again by net/url,
so 100%25.png fetches /100%25.png, and my%20cat.png and my%2520cat.png
fetch different paths. The source query stays as the source URL writes
it; the characters a browser percent-encodes in a query, such as a
space, are percent-encoded when the image URL is read, so the query is
signed and cached as it is sent. The generator page answers 400 for a
source URL its image URL would fetch written another way, such as with
%2F in the path, and shows the URL pixa would fetch. README.md says how
the source path and query are fetched.

Model: opus-5-5
This commit is contained in:
2026-10-16 00:47:13 +00:00
parent af5ce499b9
commit 545507e9d4
5 changed files with 94 additions and 16 deletions
+26 -6
View File
@@ -331,6 +331,20 @@ writes one as `%23` is refused with 400. For example,
`/v1/image/cdn.example.com/img%3Furl=https://origin.example.com/my%2520cat.jpg/800x600.webp`, `/v1/image/cdn.example.com/img%3Furl=https://origin.example.com/my%2520cat.jpg/800x600.webp`,
signed as shown under Signature Specification. signed as shown under Signature Specification.
The source path is the path with its percent-escapes decoded, and pixa fetches
it percent-encoded again: every character other than letters, digits, `-`, `.`,
`_`, `~`, `/`, `$`, `&`, `+`, `,`, `:`, `;`, `=` and `@` is sent as a
percent-escape. So `my%20cat.jpg` in the image URL is the source path
`/my cat.jpg`, fetched as `/my%20cat.jpg`; `my%2520cat.jpg` is the source path
`/my%20cat.jpg`, fetched as `/my%2520cat.jpg`; and `100%25.png` is fetched as
`/100%25.png`. The source query is the source URL's query as that URL writes it,
its percent-escapes included, and is sent as it is, except that the characters a
browser percent-encodes in a query (a space, `"`, `<`, `>`, control characters
and characters outside ASCII) and a `%` that does not begin a percent-escape are
percent-encoded. pixa signs and caches the query in that form, so
`img%3Fname=my%20cat` and `img%3Fname=my%2520cat` in the image URL are the same
source query, `name=my%20cat`.
A size with one side `0` has that side scaled to keep the source's proportions, A size with one side `0` has that side scaled to keep the source's proportions,
and is refused with 400 when that side would come out above 8192, the most and is refused with 400 when that side would come out above 8192, the most
either side may be; so is a size with `fit=outside`, which scales the image to either side may be; so is a size with `fit=outside`, which scales the image to
@@ -445,7 +459,11 @@ nor change what it asks for.
sent with an empty format, or none, also makes a JPEG XL URL. Width and sent with an empty format, or none, also makes a JPEG XL URL. Width and
height both empty or `0` keep the original size; if only one of them is empty height both empty or `0` keep the original size; if only one of them is empty
or `0`, that side is scaled to keep the image's proportions. A URL whose or `0`, that side is scaled to keep the image's proportions. A URL whose
scaled side would come out above 8192 answers 400. scaled side would come out above 8192 answers 400. So does a source URL that
the image URL would fetch written another way, such as one whose path holds
`%2F`, fetched as `/`, or `(`, fetched as `%28`, as pixa keeps the source
path decoded and fetches it percent-encoded again (see Routes); the page then
shows the URL pixa would fetch.
3. The page shows the URL, `https://<host>/v1/e/<token>/img.<format>`, and when 3. The page shows the URL, `https://<host>/v1/e/<token>/img.<format>`, and when
it expires. `<host>` is the host the page was opened on, and the URL starts it expires. `<host>` is the host the page was opened on, and the URL starts
with `http` instead while `debug` is on. The name after the token is ignored with `http` instead while `debug` is on. The name after the token is ignored
@@ -507,11 +525,13 @@ Where:
- `host` — source host as the URL writes it up to the first `/`, in lower case, - `host` — source host as the URL writes it up to the first `/`, in lower case,
keeping an IPv6 address's brackets and any `:` and port after it (e.g. keeping an IPv6 address's brackets and any `:` and port after it (e.g.
`cdn.example.com`, `cdn.example.com:8443`, `[2001:db8::1]:8443`) `cdn.example.com`, `cdn.example.com:8443`, `[2001:db8::1]:8443`)
- `path` — source path (e.g. `/photos/cat.jpg`), with each `%` written as `%25` - `path` — source path, with its percent-escapes decoded as under Routes (e.g.
and each `:` as `%3A`, so `/photos/a:b.jpg` is signed as `/photos/a%3Ab.jpg` `/photos/cat.jpg`, or `/photos/my cat.jpg` for `/photos/my%20cat.jpg`), with
- `query` — source query string, empty string if none, with `%` and `:` written each `%` written as `%25` and each `:` as `%3A`, so `/photos/a:b.jpg` is
as in `path`; neither then holds a `:`, so path `/a:b` with no query and path signed as `/photos/a%3Ab.jpg`
`/a` with query `b:` are signed differently - `query` — source query string as under Routes, empty string if none, with `%`
and `:` written as in `path`; neither then holds a `:`, so path `/a:b` with no
query and path `/a` with query `b:` are signed differently
- `width` — requested width in pixels, `0` for original - `width` — requested width in pixels, `0` for original
- `height` — requested height in pixels, `0` for original - `height` — requested height in pixels, `0` for original
- `format` — output format, one of those listed under Routes, with `original` - `format` — output format, one of those listed under Routes, with `original`
+11
View File
@@ -42,6 +42,17 @@ P2: security: per-origin rate limiting
resident memory peaked at about 242 MiB for the 8192x64 source and 431 MiB for resident memory peaked at about 242 MiB for the 8192x64 source and 431 MiB for
the 8192x1 one. The output can sit a fraction of an output pixel from where it the 8192x1 one. The output can sit a fraction of an output pixel from where it
was. was.
- 2026-10-16 the source URL a request fetches is built with `net/url` from the
source host, path and query, not joined as text (closes #288). The source
path, decoded, is fetched percent-encoded again, so `100%25.png` fetches
`/100%25.png` where it answered 401, and `my%20cat.png` and `my%2520cat.png`
fetch different paths where both fetched `/my%20cat.png` under two cache keys.
The source query stays as the source URL writes it; the characters a browser
percent-encodes in a query, such as a space, which was sent raw and answered
502, are percent-encoded when the image URL is read, so the query is signed
and cached as it is sent. The generator page answers 400, showing the URL pixa
would fetch, for a source URL its image URL would fetch written another way,
such as with `%2F` in the path.
- 2026-10-15 a `/v1/image/` URL whose source path or source query holds a `#`, - 2026-10-15 a `/v1/image/` URL whose source path or source query holds a `#`,
written as `%23`, is refused with 400 (closes #286): the source URL built from written as `%23`, is refused with 400 (closes #286): the source URL built from
it ended at the `#` when fetched, so each text after it fetched the same it ended at the `#` when fetched, so each text after it fetched the same
+16
View File
@@ -141,6 +141,22 @@ func (s *Handlers) HandleGenerateURL() http.HandlerFunc {
return return
} }
// The image URL must fetch the source URL as it was written. The
// token keeps the path decoded and pixa fetches it percent-encoded
// again, so a path written another way, such as with %2F, would fetch
// another URL. The scheme is not kept: only path and query are compared
sourceRequest := payload.ToImageRequest()
sourceRequest.AllowHTTP = s.config.AllowHTTP
fetchURL, err := url.Parse(sourceRequest.SourceURL())
if err != nil || fetchURL.RequestURI() != parsed.RequestURI() {
s.renderGeneratorWithForm(w, r, "Source URL cannot be fetched as "+
"written: pixa would fetch "+sourceRequest.SourceURL(),
r.Form, http.StatusBadRequest)
return
}
// Generate encrypted token // Generate encrypted token
token, err := s.encGen.Generate(payload) token, err := s.encGen.Generate(payload)
if err != nil { if err != nil {
+16 -8
View File
@@ -5,6 +5,7 @@ import (
"context" "context"
"errors" "errors"
"io" "io"
"net/url"
"time" "time"
) )
@@ -86,9 +87,11 @@ func ValidateImageRequest(req *ImageRequest) error {
type ImageRequest struct { type ImageRequest struct {
// SourceHost is the origin host (e.g., "cdn.example.com") // SourceHost is the origin host (e.g., "cdn.example.com")
SourceHost string SourceHost string
// SourcePath is the path on the origin (e.g., "/photos/cat.jpg") // SourcePath is the path on the origin with its percent-escapes decoded,
// as url.URL's Path holds it (e.g., "/photos/my cat.jpg")
SourcePath string SourcePath string
// SourceQuery is the optional query string for the origin URL // SourceQuery is the optional query string for the origin URL as the URL
// writes it, percent-escapes included, as url.URL's RawQuery holds it
SourceQuery string SourceQuery string
// Size is the requested output dimensions // Size is the requested output dimensions
Size Size Size Size
@@ -107,20 +110,25 @@ type ImageRequest struct {
AllowHTTP bool AllowHTTP bool
} }
// SourceURL returns the full upstream URL to fetch. // SourceURL returns the full upstream URL to fetch: the source path
// Uses http:// scheme when AllowHTTP is true, otherwise https://. // percent-encoded again, as net/url writes a path, so each source path is one
// upstream path, and the source query as it is, with the characters
// escapeQuery encodes percent-encoded. Uses http:// scheme when AllowHTTP is
// true, otherwise https://.
func (r *ImageRequest) SourceURL() string { func (r *ImageRequest) SourceURL() string {
scheme := "https" scheme := "https"
if r.AllowHTTP { if r.AllowHTTP {
scheme = "http" scheme = "http"
} }
url := scheme + "://" + r.SourceHost + r.SourcePath sourceURL := url.URL{
if r.SourceQuery != "" { Scheme: scheme,
url += "?" + r.SourceQuery Host: r.SourceHost,
Path: r.SourcePath,
RawQuery: escapeQuery(r.SourceQuery),
} }
return url return sourceURL.String()
} }
// ImageResponse represents a processed image ready to serve // ImageResponse represents a processed image ready to serve
+25 -2
View File
@@ -51,6 +51,12 @@ var sizeFormatRegex = regexp.MustCompile(
// percentEscapeRegex matches one percent-escape, such as "%2e" // percentEscapeRegex matches one percent-escape, such as "%2e"
var percentEscapeRegex = regexp.MustCompile(`%[0-9A-Fa-f]{2}`) var percentEscapeRegex = regexp.MustCompile(`%[0-9A-Fa-f]{2}`)
// queryEscapeRegex matches a percent-escape, or one character that a browser
// percent-encodes in a URL's query: one outside the printable ASCII characters
// "!" to "~", such as a space, a control character or a character outside
// ASCII, or one of `"#<>`. A "%" that does not begin a percent-escape matches too.
var queryEscapeRegex = regexp.MustCompile(`%[0-9A-Fa-f]{2}|[^!-~]|["#<>%]`)
// hostRegex matches the forms the host of an image URL may take: a host name // hostRegex matches the forms the host of an image URL may take: a host name
// or an IPv4 address (letters, digits, "-", "_" and "."), or an IPv6 address // or an IPv4 address (letters, digits, "-", "_" and "."), or an IPv6 address
// in brackets (hex digits, ":" and "."), either one optionally followed by ":" // in brackets (hex digits, ":" and "."), either one optionally followed by ":"
@@ -179,9 +185,12 @@ func parseImageComponents(remainder string) (*ParsedURL, error) {
`%w: the source path or query holds a "#"`, ErrInvalidPath) `%w: the source path or query holds a "#"`, ErrInvalidPath)
} }
// Extract query string if present in path // Extract query string if present in path. It is the source URL's query
// as that URL writes it. A character a URL's query cannot hold as it is,
// such as a space written %20 in the image URL, is percent-encoded here,
// so the query is signed and cached as it is sent upstream
if qIndex := strings.Index(path, "?"); qIndex != -1 { if qIndex := strings.Index(path, "?"); qIndex != -1 {
query = path[qIndex+1:] query = escapeQuery(path[qIndex+1:])
path = path[:qIndex] path = path[:qIndex]
} }
@@ -270,6 +279,20 @@ func decodePercentEscapes(s string) string {
}) })
} }
// escapeQuery returns a source query as a URL's query holds it: each
// character that queryEscapeRegex names, such as a space, percent-encoded,
// and the rest, the query's percent-escapes included, kept as they are. The
// result is unchanged by a second escapeQuery.
func escapeQuery(query string) string {
return queryEscapeRegex.ReplaceAllStringFunc(query, func(match string) string {
if percentEscapeRegex.MatchString(match) {
return match
}
return url.PathEscape(match)
})
}
// parseSizeFormat parses strings like "800x600.webp", "orig.png" or "800x600" // parseSizeFormat parses strings like "800x600.webp", "orig.png" or "800x600"
func parseSizeFormat(s string) (Size, ImageFormat, error) { func parseSizeFormat(s string) (Size, ImageFormat, error) {
matches := sizeFormatRegex.FindStringSubmatch(s) matches := sizeFormatRegex.FindStringSubmatch(s)