Fetch the source path and query URL-encoded, not joined as text (closes #288)
The source path, decoded, is fetched percent-encoded again by net/url, so 100%25.png fetches /100%25.png, and my%20cat.png and my%2520cat.png fetch different paths. The source query stays as the source URL writes it; the characters a browser percent-encodes in a query, such as a space, are percent-encoded when the image URL is read, so the query is signed and cached as it is sent. The generator page answers 400 for a source URL its image URL would fetch written another way, such as with %2F in the path, and shows the URL pixa would fetch. README.md says how the source path and query are fetched. Model: opus-5-5
This commit is contained in:
@@ -331,6 +331,20 @@ writes one as `%23` is refused with 400. For example,
|
|||||||
`/v1/image/cdn.example.com/img%3Furl=https://origin.example.com/my%2520cat.jpg/800x600.webp`,
|
`/v1/image/cdn.example.com/img%3Furl=https://origin.example.com/my%2520cat.jpg/800x600.webp`,
|
||||||
signed as shown under Signature Specification.
|
signed as shown under Signature Specification.
|
||||||
|
|
||||||
|
The source path is the path with its percent-escapes decoded, and pixa fetches
|
||||||
|
it percent-encoded again: every character other than letters, digits, `-`, `.`,
|
||||||
|
`_`, `~`, `/`, `$`, `&`, `+`, `,`, `:`, `;`, `=` and `@` is sent as a
|
||||||
|
percent-escape. So `my%20cat.jpg` in the image URL is the source path
|
||||||
|
`/my cat.jpg`, fetched as `/my%20cat.jpg`; `my%2520cat.jpg` is the source path
|
||||||
|
`/my%20cat.jpg`, fetched as `/my%2520cat.jpg`; and `100%25.png` is fetched as
|
||||||
|
`/100%25.png`. The source query is the source URL's query as that URL writes it,
|
||||||
|
its percent-escapes included, and is sent as it is, except that the characters a
|
||||||
|
browser percent-encodes in a query (a space, `"`, `<`, `>`, control characters
|
||||||
|
and characters outside ASCII) and a `%` that does not begin a percent-escape are
|
||||||
|
percent-encoded. pixa signs and caches the query in that form, so
|
||||||
|
`img%3Fname=my%20cat` and `img%3Fname=my%2520cat` in the image URL are the same
|
||||||
|
source query, `name=my%20cat`.
|
||||||
|
|
||||||
A size with one side `0` has that side scaled to keep the source's proportions,
|
A size with one side `0` has that side scaled to keep the source's proportions,
|
||||||
and is refused with 400 when that side would come out above 8192, the most
|
and is refused with 400 when that side would come out above 8192, the most
|
||||||
either side may be; so is a size with `fit=outside`, which scales the image to
|
either side may be; so is a size with `fit=outside`, which scales the image to
|
||||||
@@ -445,7 +459,11 @@ nor change what it asks for.
|
|||||||
sent with an empty format, or none, also makes a JPEG XL URL. Width and
|
sent with an empty format, or none, also makes a JPEG XL URL. Width and
|
||||||
height both empty or `0` keep the original size; if only one of them is empty
|
height both empty or `0` keep the original size; if only one of them is empty
|
||||||
or `0`, that side is scaled to keep the image's proportions. A URL whose
|
or `0`, that side is scaled to keep the image's proportions. A URL whose
|
||||||
scaled side would come out above 8192 answers 400.
|
scaled side would come out above 8192 answers 400. So does a source URL that
|
||||||
|
the image URL would fetch written another way, such as one whose path holds
|
||||||
|
`%2F`, fetched as `/`, or `(`, fetched as `%28`, as pixa keeps the source
|
||||||
|
path decoded and fetches it percent-encoded again (see Routes); the page then
|
||||||
|
shows the URL pixa would fetch.
|
||||||
3. The page shows the URL, `https://<host>/v1/e/<token>/img.<format>`, and when
|
3. The page shows the URL, `https://<host>/v1/e/<token>/img.<format>`, and when
|
||||||
it expires. `<host>` is the host the page was opened on, and the URL starts
|
it expires. `<host>` is the host the page was opened on, and the URL starts
|
||||||
with `http` instead while `debug` is on. The name after the token is ignored
|
with `http` instead while `debug` is on. The name after the token is ignored
|
||||||
@@ -507,11 +525,13 @@ Where:
|
|||||||
- `host` — source host as the URL writes it up to the first `/`, in lower case,
|
- `host` — source host as the URL writes it up to the first `/`, in lower case,
|
||||||
keeping an IPv6 address's brackets and any `:` and port after it (e.g.
|
keeping an IPv6 address's brackets and any `:` and port after it (e.g.
|
||||||
`cdn.example.com`, `cdn.example.com:8443`, `[2001:db8::1]:8443`)
|
`cdn.example.com`, `cdn.example.com:8443`, `[2001:db8::1]:8443`)
|
||||||
- `path` — source path (e.g. `/photos/cat.jpg`), with each `%` written as `%25`
|
- `path` — source path, with its percent-escapes decoded as under Routes (e.g.
|
||||||
and each `:` as `%3A`, so `/photos/a:b.jpg` is signed as `/photos/a%3Ab.jpg`
|
`/photos/cat.jpg`, or `/photos/my cat.jpg` for `/photos/my%20cat.jpg`), with
|
||||||
- `query` — source query string, empty string if none, with `%` and `:` written
|
each `%` written as `%25` and each `:` as `%3A`, so `/photos/a:b.jpg` is
|
||||||
as in `path`; neither then holds a `:`, so path `/a:b` with no query and path
|
signed as `/photos/a%3Ab.jpg`
|
||||||
`/a` with query `b:` are signed differently
|
- `query` — source query string as under Routes, empty string if none, with `%`
|
||||||
|
and `:` written as in `path`; neither then holds a `:`, so path `/a:b` with no
|
||||||
|
query and path `/a` with query `b:` are signed differently
|
||||||
- `width` — requested width in pixels, `0` for original
|
- `width` — requested width in pixels, `0` for original
|
||||||
- `height` — requested height in pixels, `0` for original
|
- `height` — requested height in pixels, `0` for original
|
||||||
- `format` — output format, one of those listed under Routes, with `original`
|
- `format` — output format, one of those listed under Routes, with `original`
|
||||||
|
|||||||
@@ -42,6 +42,17 @@ P2: security: per-origin rate limiting
|
|||||||
resident memory peaked at about 242 MiB for the 8192x64 source and 431 MiB for
|
resident memory peaked at about 242 MiB for the 8192x64 source and 431 MiB for
|
||||||
the 8192x1 one. The output can sit a fraction of an output pixel from where it
|
the 8192x1 one. The output can sit a fraction of an output pixel from where it
|
||||||
was.
|
was.
|
||||||
|
- 2026-10-16 the source URL a request fetches is built with `net/url` from the
|
||||||
|
source host, path and query, not joined as text (closes #288). The source
|
||||||
|
path, decoded, is fetched percent-encoded again, so `100%25.png` fetches
|
||||||
|
`/100%25.png` where it answered 401, and `my%20cat.png` and `my%2520cat.png`
|
||||||
|
fetch different paths where both fetched `/my%20cat.png` under two cache keys.
|
||||||
|
The source query stays as the source URL writes it; the characters a browser
|
||||||
|
percent-encodes in a query, such as a space, which was sent raw and answered
|
||||||
|
502, are percent-encoded when the image URL is read, so the query is signed
|
||||||
|
and cached as it is sent. The generator page answers 400, showing the URL pixa
|
||||||
|
would fetch, for a source URL its image URL would fetch written another way,
|
||||||
|
such as with `%2F` in the path.
|
||||||
- 2026-10-15 a `/v1/image/` URL whose source path or source query holds a `#`,
|
- 2026-10-15 a `/v1/image/` URL whose source path or source query holds a `#`,
|
||||||
written as `%23`, is refused with 400 (closes #286): the source URL built from
|
written as `%23`, is refused with 400 (closes #286): the source URL built from
|
||||||
it ended at the `#` when fetched, so each text after it fetched the same
|
it ended at the `#` when fetched, so each text after it fetched the same
|
||||||
|
|||||||
@@ -141,6 +141,22 @@ func (s *Handlers) HandleGenerateURL() http.HandlerFunc {
|
|||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// The image URL must fetch the source URL as it was written. The
|
||||||
|
// token keeps the path decoded and pixa fetches it percent-encoded
|
||||||
|
// again, so a path written another way, such as with %2F, would fetch
|
||||||
|
// another URL. The scheme is not kept: only path and query are compared
|
||||||
|
sourceRequest := payload.ToImageRequest()
|
||||||
|
sourceRequest.AllowHTTP = s.config.AllowHTTP
|
||||||
|
|
||||||
|
fetchURL, err := url.Parse(sourceRequest.SourceURL())
|
||||||
|
if err != nil || fetchURL.RequestURI() != parsed.RequestURI() {
|
||||||
|
s.renderGeneratorWithForm(w, r, "Source URL cannot be fetched as "+
|
||||||
|
"written: pixa would fetch "+sourceRequest.SourceURL(),
|
||||||
|
r.Form, http.StatusBadRequest)
|
||||||
|
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
// Generate encrypted token
|
// Generate encrypted token
|
||||||
token, err := s.encGen.Generate(payload)
|
token, err := s.encGen.Generate(payload)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
|
|||||||
@@ -5,6 +5,7 @@ import (
|
|||||||
"context"
|
"context"
|
||||||
"errors"
|
"errors"
|
||||||
"io"
|
"io"
|
||||||
|
"net/url"
|
||||||
"time"
|
"time"
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -86,9 +87,11 @@ func ValidateImageRequest(req *ImageRequest) error {
|
|||||||
type ImageRequest struct {
|
type ImageRequest struct {
|
||||||
// SourceHost is the origin host (e.g., "cdn.example.com")
|
// SourceHost is the origin host (e.g., "cdn.example.com")
|
||||||
SourceHost string
|
SourceHost string
|
||||||
// SourcePath is the path on the origin (e.g., "/photos/cat.jpg")
|
// SourcePath is the path on the origin with its percent-escapes decoded,
|
||||||
|
// as url.URL's Path holds it (e.g., "/photos/my cat.jpg")
|
||||||
SourcePath string
|
SourcePath string
|
||||||
// SourceQuery is the optional query string for the origin URL
|
// SourceQuery is the optional query string for the origin URL as the URL
|
||||||
|
// writes it, percent-escapes included, as url.URL's RawQuery holds it
|
||||||
SourceQuery string
|
SourceQuery string
|
||||||
// Size is the requested output dimensions
|
// Size is the requested output dimensions
|
||||||
Size Size
|
Size Size
|
||||||
@@ -107,20 +110,25 @@ type ImageRequest struct {
|
|||||||
AllowHTTP bool
|
AllowHTTP bool
|
||||||
}
|
}
|
||||||
|
|
||||||
// SourceURL returns the full upstream URL to fetch.
|
// SourceURL returns the full upstream URL to fetch: the source path
|
||||||
// Uses http:// scheme when AllowHTTP is true, otherwise https://.
|
// percent-encoded again, as net/url writes a path, so each source path is one
|
||||||
|
// upstream path, and the source query as it is, with the characters
|
||||||
|
// escapeQuery encodes percent-encoded. Uses http:// scheme when AllowHTTP is
|
||||||
|
// true, otherwise https://.
|
||||||
func (r *ImageRequest) SourceURL() string {
|
func (r *ImageRequest) SourceURL() string {
|
||||||
scheme := "https"
|
scheme := "https"
|
||||||
if r.AllowHTTP {
|
if r.AllowHTTP {
|
||||||
scheme = "http"
|
scheme = "http"
|
||||||
}
|
}
|
||||||
|
|
||||||
url := scheme + "://" + r.SourceHost + r.SourcePath
|
sourceURL := url.URL{
|
||||||
if r.SourceQuery != "" {
|
Scheme: scheme,
|
||||||
url += "?" + r.SourceQuery
|
Host: r.SourceHost,
|
||||||
|
Path: r.SourcePath,
|
||||||
|
RawQuery: escapeQuery(r.SourceQuery),
|
||||||
}
|
}
|
||||||
|
|
||||||
return url
|
return sourceURL.String()
|
||||||
}
|
}
|
||||||
|
|
||||||
// ImageResponse represents a processed image ready to serve
|
// ImageResponse represents a processed image ready to serve
|
||||||
|
|||||||
@@ -51,6 +51,12 @@ var sizeFormatRegex = regexp.MustCompile(
|
|||||||
// percentEscapeRegex matches one percent-escape, such as "%2e"
|
// percentEscapeRegex matches one percent-escape, such as "%2e"
|
||||||
var percentEscapeRegex = regexp.MustCompile(`%[0-9A-Fa-f]{2}`)
|
var percentEscapeRegex = regexp.MustCompile(`%[0-9A-Fa-f]{2}`)
|
||||||
|
|
||||||
|
// queryEscapeRegex matches a percent-escape, or one character that a browser
|
||||||
|
// percent-encodes in a URL's query: one outside the printable ASCII characters
|
||||||
|
// "!" to "~", such as a space, a control character or a character outside
|
||||||
|
// ASCII, or one of `"#<>`. A "%" that does not begin a percent-escape matches too.
|
||||||
|
var queryEscapeRegex = regexp.MustCompile(`%[0-9A-Fa-f]{2}|[^!-~]|["#<>%]`)
|
||||||
|
|
||||||
// hostRegex matches the forms the host of an image URL may take: a host name
|
// hostRegex matches the forms the host of an image URL may take: a host name
|
||||||
// or an IPv4 address (letters, digits, "-", "_" and "."), or an IPv6 address
|
// or an IPv4 address (letters, digits, "-", "_" and "."), or an IPv6 address
|
||||||
// in brackets (hex digits, ":" and "."), either one optionally followed by ":"
|
// in brackets (hex digits, ":" and "."), either one optionally followed by ":"
|
||||||
@@ -179,9 +185,12 @@ func parseImageComponents(remainder string) (*ParsedURL, error) {
|
|||||||
`%w: the source path or query holds a "#"`, ErrInvalidPath)
|
`%w: the source path or query holds a "#"`, ErrInvalidPath)
|
||||||
}
|
}
|
||||||
|
|
||||||
// Extract query string if present in path
|
// Extract query string if present in path. It is the source URL's query
|
||||||
|
// as that URL writes it. A character a URL's query cannot hold as it is,
|
||||||
|
// such as a space written %20 in the image URL, is percent-encoded here,
|
||||||
|
// so the query is signed and cached as it is sent upstream
|
||||||
if qIndex := strings.Index(path, "?"); qIndex != -1 {
|
if qIndex := strings.Index(path, "?"); qIndex != -1 {
|
||||||
query = path[qIndex+1:]
|
query = escapeQuery(path[qIndex+1:])
|
||||||
path = path[:qIndex]
|
path = path[:qIndex]
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -270,6 +279,20 @@ func decodePercentEscapes(s string) string {
|
|||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// escapeQuery returns a source query as a URL's query holds it: each
|
||||||
|
// character that queryEscapeRegex names, such as a space, percent-encoded,
|
||||||
|
// and the rest, the query's percent-escapes included, kept as they are. The
|
||||||
|
// result is unchanged by a second escapeQuery.
|
||||||
|
func escapeQuery(query string) string {
|
||||||
|
return queryEscapeRegex.ReplaceAllStringFunc(query, func(match string) string {
|
||||||
|
if percentEscapeRegex.MatchString(match) {
|
||||||
|
return match
|
||||||
|
}
|
||||||
|
|
||||||
|
return url.PathEscape(match)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
// parseSizeFormat parses strings like "800x600.webp", "orig.png" or "800x600"
|
// parseSizeFormat parses strings like "800x600.webp", "orig.png" or "800x600"
|
||||||
func parseSizeFormat(s string) (Size, ImageFormat, error) {
|
func parseSizeFormat(s string) (Size, ImageFormat, error) {
|
||||||
matches := sizeFormatRegex.FindStringSubmatch(s)
|
matches := sizeFormatRegex.FindStringSubmatch(s)
|
||||||
|
|||||||
Reference in New Issue
Block a user