6 Commits
Author SHA1 Message Date
sneak 0dac2c2120 Say what a 0 side does in the ValidateDimension doc comment
check / check (push) Canceled after 0s
It said 0 meant "original size"; the side is worked out from the
source's proportions, as README.md now says.

Model: opus-5-5
2026-10-16 01:59:24 +00:00
sneak dc1c069ad6 Keep each side at 1 or more with fit=outside too
fit=outside works both sides out again from the larger scale, and a side
scaled to 1 could come out just under 1 and round down to 0, which
libvips refuses: a 49x8192 source at 0x100 and a 49x49 source at 1x1
answered 500. README.md now says what fit=outside does, with the 8192x1
source at 100x0 that comes out at 8192x1, and how a 0 side is worked
out from the source's proportions.

Model: opus-5-5
2026-10-16 01:58:49 +00:00
sneak 5d30212352 Test that fit=outside keeps each side at 1 or more
A 49x8192 source at 0x100 and a 49x49 source at 1x1, both with
fit=outside. 49 times 1/49 is just under 1 in floating point, so the
width is rounded down to 0 and libvips refuses it. The test fails until
the fix for #283 covers fit=outside.

Model: opus-5-5
2026-10-16 01:58:13 +00:00
sneak 3126037a5d Keep a side computed for a size with one side 0 at 1 or more (closes #283)
The side was rounded down, so for a thin source it came out at 0, and
libvips made the size square: an 8192x1 source at 100x0 came out at
100x100, at 1300x0 with fit=cover libvips asked for about 39 GB, and an
8192x2 source at 100x0 answered 500. fit=contain and fit=inside work
both sides out again from the smaller scale and keep each at 1 or more
as well, as a 1x8192 source at 0x100 answered 500 there.

Model: opus-5-5
2026-10-16 01:58:13 +00:00
sneak 21fa87ddf6 Test that a side computed for a size with one side 0 is at least 1
An 8192x1 source at 100x0 and 1300x0 and an 8192x2 source at 100x0, and
the same turned on their side, in every fit mode. The test fails until
the fix for #283 lands; TODO.md
records the bug at the top as P1 until then.

Model: opus-5-5
2026-10-16 01:58:13 +00:00
clawbot 4a4bc9758f Fetch the source path and query URL-encoded, not joined as text (closes #288)
check / check (push) Canceled after 0s
The source URL was built by joining the decoded path and query as
text, so a space in the query reached the upstream raw and failed with
502, 100%25.png made a URL Go could not parse and answered 401, and
%20 and %2520 fetched the same path under different cache keys. The
source URL is now built with net/url, so each decoded path and query is
one upstream request matching its signature and cache key; the query
encodes the characters a browser encodes, ' included. The generator page
compares what pixa would fetch with the URL as entered, and answers 400
with the URL to use when they differ, or for a user name and password
or a .. segment.

Model: opus-5-5
2026-10-16 03:31:15 +02:00
9 changed files with 658 additions and 32 deletions

No files matched your search

+39 -9
View File
@@ -331,6 +331,20 @@ writes one as `%23` is refused with 400. For example,
`/v1/image/cdn.example.com/img%3Furl=https://origin.example.com/my%2520cat.jpg/800x600.webp`, `/v1/image/cdn.example.com/img%3Furl=https://origin.example.com/my%2520cat.jpg/800x600.webp`,
signed as shown under Signature Specification. signed as shown under Signature Specification.
The source path is the path with its percent-escapes decoded, and pixa fetches
it percent-encoded again: every character other than letters, digits, `-`, `.`,
`_`, `~`, `/`, `$`, `&`, `+`, `,`, `:`, `;`, `=` and `@` is sent as a
percent-escape. So `my%20cat.jpg` in the image URL is the source path
`/my cat.jpg`, fetched as `/my%20cat.jpg`; `my%2520cat.jpg` is the source path
`/my%20cat.jpg`, fetched as `/my%2520cat.jpg`; and `100%25.png` is fetched as
`/100%25.png`. The source query is the source URL's query as that URL writes it,
its percent-escapes included, and is sent as it is, except that the characters a
browser percent-encodes in a query (a space, `"`, `'`, `<`, `>`, control
characters and characters outside ASCII) and a `%` that does not begin a
percent-escape are percent-encoded. pixa signs and caches the query in that
form, so `img%3Fname=my%20cat` and `img%3Fname=my%2520cat` in the image URL are
the same source query, `name=my%20cat`.
A size with one side `0` has that side scaled to keep the source's proportions, A size with one side `0` has that side scaled to keep the source's proportions,
and is refused with 400 when that side would come out above 8192, the most and is refused with 400 when that side would come out above 8192, the most
either side may be; so is a size with `fit=outside`, which scales the image to either side may be; so is a size with `fit=outside`, which scales the image to
@@ -445,7 +459,13 @@ nor change what it asks for.
sent with an empty format, or none, also makes a JPEG XL URL. Width and sent with an empty format, or none, also makes a JPEG XL URL. Width and
height both empty or `0` keep the original size; if only one of them is empty height both empty or `0` keep the original size; if only one of them is empty
or `0`, that side is scaled to keep the image's proportions. A URL whose or `0`, that side is scaled to keep the image's proportions. A URL whose
scaled side would come out above 8192 answers 400. scaled side would come out above 8192 answers 400, and so does a source URL
whose path holds a `..` segment. So does a source URL that the image URL
would fetch written another way, and the page then shows the URL pixa would
fetch: one with a user name and password, which pixa does not send, or one
whose path holds `%2F`, fetched as `/`, or `(`, fetched as `%28`, as pixa
keeps the source path decoded and fetches it percent-encoded again (see
Routes).
3. The page shows the URL, `https://<host>/v1/e/<token>/img.<format>`, and when 3. The page shows the URL, `https://<host>/v1/e/<token>/img.<format>`, and when
it expires. `<host>` is the host the page was opened on, and the URL starts it expires. `<host>` is the host the page was opened on, and the URL starts
with `http` instead while `debug` is on. The name after the token is ignored with `http` instead while `debug` is on. The name after the token is ignored
@@ -507,13 +527,20 @@ Where:
- `host` — source host as the URL writes it up to the first `/`, in lower case, - `host` — source host as the URL writes it up to the first `/`, in lower case,
keeping an IPv6 address's brackets and any `:` and port after it (e.g. keeping an IPv6 address's brackets and any `:` and port after it (e.g.
`cdn.example.com`, `cdn.example.com:8443`, `[2001:db8::1]:8443`) `cdn.example.com`, `cdn.example.com:8443`, `[2001:db8::1]:8443`)
- `path` — source path (e.g. `/photos/cat.jpg`), with each `%` written as `%25` - `path` — source path, with its percent-escapes decoded as under Routes (e.g.
and each `:` as `%3A`, so `/photos/a:b.jpg` is signed as `/photos/a%3Ab.jpg` `/photos/cat.jpg`, or `/photos/my cat.jpg` for `/photos/my%20cat.jpg`), with
- `query` — source query string, empty string if none, with `%` and `:` written each `%` written as `%25` and each `:` as `%3A`, so `/photos/a:b.jpg` is
as in `path`; neither then holds a `:`, so path `/a:b` with no query and path signed as `/photos/a%3Ab.jpg`
`/a` with query `b:` are signed differently - `query` — source query string as under Routes, empty string if none, with `%`
- `width` — requested width in pixels, `0` for original and `:` written as in `path`; neither then holds a `:`, so path `/a:b` with no
- `height` — requested height in pixels, `0` for original query and path `/a` with query `b:` are signed differently
- `width` — requested width in pixels, or `0`: the width is then worked out from
the source's proportions and the requested height, rounded down but at least
1, or is the source's own width when the height is `0` too (the `orig` size)
- `height` — requested height in pixels, or `0`: the height is then worked out
from the source's proportions and the requested width, rounded down but at
least 1, or is the source's own height when the width is `0` too (the `orig`
size)
- `format` — output format, one of those listed under Routes, with `original` - `format` — output format, one of those listed under Routes, with `original`
signed as `orig`, `jpg` as `jpeg`, and no format as `jxl`, so a URL with no signed as `orig`, `jpg` as `jpeg`, and no format as `jxl`, so a URL with no
format has the signature of the same URL ending in `.jxl`; `auto` is signed as format has the signature of the same URL ending in `.jxl`; `auto` is signed as
@@ -526,7 +553,10 @@ Where:
with 400 with 400
- `fit` — the URL's `fit` query parameter (cover, contain, fill, inside, - `fit` — the URL's `fit` query parameter (cover, contain, fill, inside,
outside), or `cover` when the URL has no `fit`; a request whose `fit` is outside), or `cover` when the URL has no `fit`; a request whose `fit` is
anything else, an empty `fit=` included, is refused with 400 anything else, an empty `fit=` included, is refused with 400. `outside` scales
the image to cover the requested size, keeping its proportions, so it can come
out larger than that size on one side: an 8192x1 source at `100x0`, which is
100x1 by its proportions, comes out at 8192x1
The URL's `sig` is the HMAC-SHA256 result in base64url (the URL-safe alphabet of The URL's `sig` is the HMAC-SHA256 result in base64url (the URL-safe alphabet of
RFC 4648) with the trailing `=` padding kept, 44 characters in all. pixa RFC 4648) with the trailing `=` padding kept, 44 characters in all. pixa
+27
View File
@@ -30,6 +30,33 @@ P2: security: per-origin rate limiting
# Completed Steps # Completed Steps
- 2026-10-16 a size with one side `0` keeps the side computed from the source's
proportions at 1 or more (closes #283): it was rounded down, so for a thin
source it came out at 0, and libvips then made the size square. An 8192x1
source at `100x0` came out at 100x100, and at `1300x0` with `fit=cover` made
libvips ask for about 39 GB at once; an 8192x2 source at `100x0` answered 500.
`fit=contain` and `fit=inside` work both sides out again from the smaller of
the two scales and `fit=outside` from the larger, and each keeps both sides at
1 or more too, so a 1x8192 source at `0x100` with `fit=contain`, and a 49x8192
source at `0x100` or a 49x49 source at `1x1` with `fit=outside`, no longer
answer 500. An 8192x1 source at `1300x0` now comes out at 1300x1, and the
process running it alone peaked at about 34 MiB resident. With `fit=outside`
the image is scaled to cover the size, so that source at `100x0` comes out at
8192x1. `README.md` now says this under Signature Specification, and says how
a `0` side is worked out where it said `0` meant the original.
- 2026-10-16 the source URL a request fetches is built with `net/url` from the
source host, path and query, not joined as text (closes #288). The source
path, decoded, is fetched percent-encoded again, so `100%25.png` fetches
`/100%25.png` where it answered 401, and `my%20cat.png` and `my%2520cat.png`
fetch different paths where both fetched `/my%20cat.png` under two cache keys.
The source query stays as the source URL writes it; the characters a browser
percent-encodes in a query, such as `'` or a space, which was sent raw and
answered 502, are percent-encoded when the image URL is read, so the query is
signed and cached as it is sent. The generator page answers 400, showing the
URL pixa would fetch, for a source URL its image URL would fetch written
another way, such as with `%2F` in the path, whatever else the path holds, or
with a user name and password. It also answers 400 for a source path holding a
`..` segment, as `/v1/image/` refuses one.
- 2026-10-15 a thin source requested with `fit=cover` no longer makes libvips - 2026-10-15 a thin source requested with `fit=cover` no longer makes libvips
build a huge image (closes #282): libvips' thumbnail scaled the whole source build a huge image (closes #282): libvips' thumbnail scaled the whole source
to cover the requested size and copied that into memory before it cut out the to cover the requested size and copied that into memory before it cut out the
+48
View File
@@ -9,7 +9,9 @@ import (
"math" "math"
"net/http" "net/http"
"net/url" "net/url"
"slices"
"strconv" "strconv"
"strings"
"time" "time"
"sneak.berlin/go/pixa/internal/clientip" "sneak.berlin/go/pixa/internal/clientip"
@@ -133,6 +135,16 @@ func (s *Handlers) HandleGenerateURL() http.HandlerFunc {
return return
} }
// /v1/image/ refuses a ".." segment in the source path, and so does
// the generator. The path is decoded, so %2e%2e counts too
if slices.Contains(strings.Split(parsed.Path, "/"), "..") {
s.renderGeneratorWithForm(w, r,
"Source URL path cannot hold a .. segment", r.Form,
http.StatusBadRequest)
return
}
payload, expiresAt, ttl, err := buildGeneratePayload(parsed, r.Form) payload, expiresAt, ttl, err := buildGeneratePayload(parsed, r.Form)
if err != nil { if err != nil {
s.renderGeneratorWithForm(w, r, err.Error(), r.Form, s.renderGeneratorWithForm(w, r, err.Error(), r.Form,
@@ -141,6 +153,19 @@ func (s *Handlers) HandleGenerateURL() http.HandlerFunc {
return return
} }
// The image URL must fetch the source URL as it was written
sourceRequest := payload.ToImageRequest()
sourceRequest.AllowHTTP = s.config.AllowHTTP
fetchURL := sourceRequest.SourceURL()
if !fetchedAsWritten(parsed, fetchURL) {
s.renderGeneratorWithForm(w, r, "Source URL cannot be fetched as "+
"written: pixa would fetch "+fetchURL,
r.Form, http.StatusBadRequest)
return
}
// Generate encrypted token // Generate encrypted token
token, err := s.encGen.Generate(payload) token, err := s.encGen.Generate(payload)
if err != nil { if err != nil {
@@ -173,6 +198,29 @@ func (s *Handlers) HandleGenerateURL() http.HandlerFunc {
} }
} }
// fetchedAsWritten reports whether fetchURL, the URL pixa fetches for an
// image URL made from the source URL source, is source as it was written,
// apart from the scheme, which the token does not keep. It is not when source
// holds a user name and password, which the token does not keep either, or a
// "?" with no query after it. The token keeps the source path decoded and
// pixa fetches it percent-encoded again, so a path written another way, such
// as with %2F, is fetched as another path. url.URL keeps the path as written
// in RawPath when it is not the decoded path percent-encoded again.
// EscapedPath ignores RawPath when it also holds a character net/url encodes,
// such as a space, so RawPath is compared itself.
func fetchedAsWritten(source *url.URL, fetchURL string) bool {
writtenPath := source.RawPath
if writtenPath == "" {
writtenPath = source.EscapedPath()
}
fetched, err := url.Parse(fetchURL)
return err == nil && source.User == nil && !source.ForceQuery &&
fetched.EscapedPath() == writtenPath &&
fetched.RawQuery == source.RawQuery
}
// buildGeneratePayload parses the numeric form fields and assembles the // buildGeneratePayload parses the numeric form fields and assembles the
// encrypted URL payload. ttl=0 means never expires (ExpiresAt stays 0). A // encrypted URL payload. ttl=0 means never expires (ExpiresAt stays 0). A
// non-numeric or out-of-range width, height, quality or ttl, or an // non-numeric or out-of-range width, height, quality or ttl, or an
@@ -50,3 +50,69 @@ func TestImageProcessor_RefusesComputedSideAboveMaximum(t *testing.T) {
}) })
} }
} }
// TestImageProcessor_ComputedSideIsAtLeastOne verifies that, for a size with
// one side 0, the side computed from the input's proportions is 1 where it
// would round down to 0, and that Process makes that size in every fit mode
// but FitOutside, which scales the input to cover the size and so can make
// it longer. Each case checks the size targetDimensions gives before it
// calls Process: with that side at 0, an 8192x1 input at 1300x0 makes
// libvips ask for about 39 GB with FitCover.
func TestImageProcessor_ComputedSideIsAtLeastOne(t *testing.T) {
t.Parallel()
tests := []struct {
name string
inputWidth, inputHeight int
size Size
wantWidth, wantHeight int
}{
{"8192x1 at 100x0", 8192, 1, Size{Width: 100}, 100, 1},
{"8192x1 at 1300x0", 8192, 1, Size{Width: 1300}, 1300, 1},
{"8192x2 at 100x0", 8192, 2, Size{Width: 100}, 100, 1},
{"1x8192 at 0x100", 1, 8192, Size{Height: 100}, 1, 100},
{"1x8192 at 0x1300", 1, 8192, Size{Height: 1300}, 1, 1300},
{"2x8192 at 0x100", 2, 8192, Size{Height: 100}, 1, 100},
}
fits := []FitMode{FitCover, FitContain, FitFill, FitInside, FitOutside}
for _, tt := range tests {
for _, fit := range fits {
t.Run(tt.name+" "+string(fit), func(t *testing.T) {
t.Parallel()
width, height, err := targetDimensions(tt.size,
tt.inputWidth, tt.inputHeight)
if err != nil || width != tt.wantWidth || height != tt.wantHeight {
t.Fatalf("targetDimensions() = %dx%d, %v, want %dx%d",
width, height, err, tt.wantWidth, tt.wantHeight)
}
input := createTestJPEG(t, tt.inputWidth, tt.inputHeight)
result, err := New(Params{}).Process(t.Context(),
bytes.NewReader(input),
&Request{Size: tt.size, Format: FormatJPEG, FitMode: fit})
if err != nil {
t.Fatalf("Process() error = %v", err)
}
defer func() { _ = result.Content.Close() }()
t.Logf("Process() made %dx%d", result.Width, result.Height)
ok := result.Width == tt.wantWidth && result.Height == tt.wantHeight
if fit == FitOutside {
ok = result.Width >= tt.wantWidth && result.Height >= tt.wantHeight
}
if !ok {
t.Errorf("Process() made %dx%d, want %dx%d (at least, for %s)",
result.Width, result.Height, tt.wantWidth, tt.wantHeight,
FitOutside)
}
})
}
}
}
@@ -60,3 +60,45 @@ func TestImageProcessor_RefusesFitOutsideAboveMaximum(t *testing.T) {
}) })
} }
} }
// TestImageProcessor_FitOutsideSideIsAtLeastOne verifies that FitOutside
// keeps each side at 1 or more when it works both sides out again from the
// larger scale. A side of 49 scaled to 1 is 49 times 1/49, which in floating
// point is just under 1 and so rounded down to 0, a width libvips refuses.
func TestImageProcessor_FitOutsideSideIsAtLeastOne(t *testing.T) {
t.Parallel()
tests := []struct {
name string
inputWidth, inputHeight int
size Size
wantWidth, wantHeight int
}{
{"49x8192 at 0x100", 49, 8192, Size{Height: 100}, 1, 167},
{"49x49 at 1x1", 49, 49, Size{Width: 1, Height: 1}, 1, 1},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
input := createTestJPEG(t, tt.inputWidth, tt.inputHeight)
result, err := New(Params{}).Process(t.Context(),
bytes.NewReader(input),
&Request{Size: tt.size, Format: FormatJPEG, FitMode: FitOutside})
if err != nil {
t.Fatalf("Process() error = %v", err)
}
defer func() { _ = result.Content.Close() }()
t.Logf("Process() made %dx%d", result.Width, result.Height)
if result.Width != tt.wantWidth || result.Height != tt.wantHeight {
t.Errorf("Process() made %dx%d, want %dx%d",
result.Width, result.Height, tt.wantWidth, tt.wantHeight)
}
})
}
}
+20 -12
View File
@@ -311,8 +311,10 @@ func (p *ImageProcessor) Process(
// targetDimensions calculates the output dimensions for a requested size, // targetDimensions calculates the output dimensions for a requested size,
// scaling proportionally when only one dimension is given and keeping the // scaling proportionally when only one dimension is given and keeping the
// original dimensions when both are zero. A side scaled proportionally can // original dimensions when both are zero. A side scaled proportionally is
// come out above MaxOutputDimension, which is ErrOutputTooLarge. // rounded down but kept at 1 or more, as for a thin image it would round
// down to 0, a side of no pixels. It can come out above MaxOutputDimension,
// which is ErrOutputTooLarge.
func targetDimensions(size Size, origWidth, origHeight int) (int, int, error) { func targetDimensions(size Size, origWidth, origHeight int) (int, int, error) {
width, height := size.Width, size.Height width, height := size.Width, size.Height
@@ -322,10 +324,10 @@ func targetDimensions(size Size, origWidth, origHeight int) (int, int, error) {
return origWidth, origHeight, nil return origWidth, origHeight, nil
case width == 0: case width == 0:
// Only height specified: calculate width proportionally // Only height specified: calculate width proportionally
width = origWidth * height / origHeight width = max(1, origWidth*height/origHeight)
case height == 0: case height == 0:
// Only width specified: calculate height proportionally // Only width specified: calculate height proportionally
height = origHeight * width / origWidth height = max(1, origHeight*width/origWidth)
} }
if width > MaxOutputDimension || height > MaxOutputDimension { if width > MaxOutputDimension || height > MaxOutputDimension {
@@ -479,13 +481,16 @@ func (p *ImageProcessor) resize(
return cropCentreAndResize(img, width, height) return cropCentreAndResize(img, width, height)
case FitContain: case FitContain:
// Resize to fit within dimensions, maintaining aspect ratio // Resize to fit within dimensions, maintaining aspect ratio. Scaled
// by the smaller scale, the short side of a thin image can round
// down to 0, which libvips refuses as a width, so each side is at
// least 1.
imgW, imgH := img.Width(), img.Height() imgW, imgH := img.Width(), img.Height()
scaleW := float64(width) / float64(imgW) scaleW := float64(width) / float64(imgW)
scaleH := float64(height) / float64(imgH) scaleH := float64(height) / float64(imgH)
scale := min(scaleW, scaleH) scale := min(scaleW, scaleH)
newW := int(float64(imgW) * scale) newW := max(1, int(float64(imgW)*scale))
newH := int(float64(imgH) * scale) newH := max(1, int(float64(imgH)*scale))
return img.Thumbnail(newW, newH, vips.InterestingNone) return img.Thumbnail(newW, newH, vips.InterestingNone)
@@ -503,19 +508,22 @@ func (p *ImageProcessor) resize(
scaleW := float64(width) / float64(imgW) scaleW := float64(width) / float64(imgW)
scaleH := float64(height) / float64(imgH) scaleH := float64(height) / float64(imgH)
scale := min(scaleW, scaleH) scale := min(scaleW, scaleH)
newW := int(float64(imgW) * scale) newW := max(1, int(float64(imgW)*scale))
newH := int(float64(imgH) * scale) newH := max(1, int(float64(imgH)*scale))
return img.Thumbnail(newW, newH, vips.InterestingNone) return img.Thumbnail(newW, newH, vips.InterestingNone)
case FitOutside: case FitOutside:
// Resize so smallest dimension fits, may exceed target on other dimension // Resize so smallest dimension fits, may exceed target on other
// dimension. A side scaled to 1 can come out just under 1 in
// floating point, as 49 times 1/49 does, and round down to 0, which
// libvips refuses as a width, so each side is at least 1.
imgW, imgH := img.Width(), img.Height() imgW, imgH := img.Width(), img.Height()
scaleW := float64(width) / float64(imgW) scaleW := float64(width) / float64(imgW)
scaleH := float64(height) / float64(imgH) scaleH := float64(height) / float64(imgH)
scale := max(scaleW, scaleH) scale := max(scaleW, scaleH)
newW := int(float64(imgW) * scale) newW := max(1, int(float64(imgW)*scale))
newH := int(float64(imgH) * scale) newH := max(1, int(float64(imgH)*scale))
// The dimension that exceeds the target is held to the maximum too // The dimension that exceeds the target is held to the maximum too
if newW > MaxOutputDimension || newH > MaxOutputDimension { if newW > MaxOutputDimension || newH > MaxOutputDimension {
+16 -8
View File
@@ -5,6 +5,7 @@ import (
"context" "context"
"errors" "errors"
"io" "io"
"net/url"
"time" "time"
) )
@@ -86,9 +87,11 @@ func ValidateImageRequest(req *ImageRequest) error {
type ImageRequest struct { type ImageRequest struct {
// SourceHost is the origin host (e.g., "cdn.example.com") // SourceHost is the origin host (e.g., "cdn.example.com")
SourceHost string SourceHost string
// SourcePath is the path on the origin (e.g., "/photos/cat.jpg") // SourcePath is the path on the origin with its percent-escapes decoded,
// as url.URL's Path holds it (e.g., "/photos/my cat.jpg")
SourcePath string SourcePath string
// SourceQuery is the optional query string for the origin URL // SourceQuery is the optional query string for the origin URL as the URL
// writes it, percent-escapes included, as url.URL's RawQuery holds it
SourceQuery string SourceQuery string
// Size is the requested output dimensions // Size is the requested output dimensions
Size Size Size Size
@@ -107,20 +110,25 @@ type ImageRequest struct {
AllowHTTP bool AllowHTTP bool
} }
// SourceURL returns the full upstream URL to fetch. // SourceURL returns the full upstream URL to fetch: the source path
// Uses http:// scheme when AllowHTTP is true, otherwise https://. // percent-encoded again, as net/url writes a path, so each source path is one
// upstream path, and the source query as it is, with the characters
// escapeQuery encodes percent-encoded. Uses http:// scheme when AllowHTTP is
// true, otherwise https://.
func (r *ImageRequest) SourceURL() string { func (r *ImageRequest) SourceURL() string {
scheme := "https" scheme := "https"
if r.AllowHTTP { if r.AllowHTTP {
scheme = "http" scheme = "http"
} }
url := scheme + "://" + r.SourceHost + r.SourcePath sourceURL := url.URL{
if r.SourceQuery != "" { Scheme: scheme,
url += "?" + r.SourceQuery Host: r.SourceHost,
Path: r.SourcePath,
RawQuery: escapeQuery(r.SourceQuery),
} }
return url return sourceURL.String()
} }
// ImageResponse represents a processed image ready to serve // ImageResponse represents a processed image ready to serve
+29 -3
View File
@@ -29,7 +29,9 @@ var (
const MaxDimension = imageprocessor.MaxOutputDimension const MaxDimension = imageprocessor.MaxOutputDimension
// ValidateDimension checks one requested width or height; name ("width" or // ValidateDimension checks one requested width or height; name ("width" or
// "height") appears in the error. 0 means "original size" and is valid. // "height") appears in the error. 0 is valid: that side is then worked out from
// the source's proportions and the other requested side, rounded down but at
// least 1, or is the source's own side when the other side is 0 too.
func ValidateDimension(name string, value int) error { func ValidateDimension(name string, value int) error {
if value < 0 { if value < 0 {
return fmt.Errorf("%w: %s is negative", ErrInvalidSize, name) return fmt.Errorf("%w: %s is negative", ErrInvalidSize, name)
@@ -51,6 +53,13 @@ var sizeFormatRegex = regexp.MustCompile(
// percentEscapeRegex matches one percent-escape, such as "%2e" // percentEscapeRegex matches one percent-escape, such as "%2e"
var percentEscapeRegex = regexp.MustCompile(`%[0-9A-Fa-f]{2}`) var percentEscapeRegex = regexp.MustCompile(`%[0-9A-Fa-f]{2}`)
// queryEscapeRegex matches a percent-escape, or one character that a browser
// percent-encodes in the query of an http or https URL: one outside the
// printable ASCII characters "!" to "~", such as a space, a control character
// or a character outside ASCII, or one of `"#'<>`. A "%" that does not begin a
// percent-escape matches too.
var queryEscapeRegex = regexp.MustCompile(`%[0-9A-Fa-f]{2}|[^!-~]|["#'<>%]`)
// hostRegex matches the forms the host of an image URL may take: a host name // hostRegex matches the forms the host of an image URL may take: a host name
// or an IPv4 address (letters, digits, "-", "_" and "."), or an IPv6 address // or an IPv4 address (letters, digits, "-", "_" and "."), or an IPv6 address
// in brackets (hex digits, ":" and "."), either one optionally followed by ":" // in brackets (hex digits, ":" and "."), either one optionally followed by ":"
@@ -179,9 +188,12 @@ func parseImageComponents(remainder string) (*ParsedURL, error) {
`%w: the source path or query holds a "#"`, ErrInvalidPath) `%w: the source path or query holds a "#"`, ErrInvalidPath)
} }
// Extract query string if present in path // Extract query string if present in path. It is the source URL's query
// as that URL writes it. A character a URL's query cannot hold as it is,
// such as a space written %20 in the image URL, is percent-encoded here,
// so the query is signed and cached as it is sent upstream
if qIndex := strings.Index(path, "?"); qIndex != -1 { if qIndex := strings.Index(path, "?"); qIndex != -1 {
query = path[qIndex+1:] query = escapeQuery(path[qIndex+1:])
path = path[:qIndex] path = path[:qIndex]
} }
@@ -270,6 +282,20 @@ func decodePercentEscapes(s string) string {
}) })
} }
// escapeQuery returns a source query as a URL's query holds it: each
// character that queryEscapeRegex names, such as a space, percent-encoded,
// and the rest, the query's percent-escapes included, kept as they are. The
// result is unchanged by a second escapeQuery.
func escapeQuery(query string) string {
return queryEscapeRegex.ReplaceAllStringFunc(query, func(match string) string {
if percentEscapeRegex.MatchString(match) {
return match
}
return url.PathEscape(match)
})
}
// parseSizeFormat parses strings like "800x600.webp", "orig.png" or "800x600" // parseSizeFormat parses strings like "800x600.webp", "orig.png" or "800x600"
func parseSizeFormat(s string) (Size, ImageFormat, error) { func parseSizeFormat(s string) (Size, ImageFormat, error) {
matches := sizeFormatRegex.FindStringSubmatch(s) matches := sizeFormatRegex.FindStringSubmatch(s)
+371
View File
@@ -0,0 +1,371 @@
package server
import (
"net/http"
"net/http/httptest"
"net/url"
"regexp"
"slices"
"strings"
"sync"
"testing"
"sneak.berlin/go/pixa/internal/imgcache"
"sneak.berlin/go/pixa/internal/session"
)
// percentSignPath is a source path holding a "%", which a URL writes %25.
const percentSignPath = "/100%25.png"
// generateFormPattern extracts the CSRF token rendered into the generator
// page's form.
var generateFormPattern = regexp.MustCompile(
`(?s)action="/generate".*?name="gorilla\.csrf\.Token" value="([^"]+)"`)
// generatedURLPattern extracts the path of the encrypted URL the generator
// page shows.
var generatedURLPattern = regexp.MustCompile(`value="[^"]*(/v1/e/[^"]+)"`)
// recordingUpstream is a test upstream that answers every request with a PNG
// and records the request target, the path and query as sent, of each request
// it receives.
type recordingUpstream struct {
*httptest.Server
mu sync.Mutex
received []string
}
// startRecordingUpstream starts a recordingUpstream, which is closed when the
// test ends.
func startRecordingUpstream(t *testing.T) *recordingUpstream {
t.Helper()
source := encodeTestPNG(t, 64, 48)
upstream := &recordingUpstream{}
upstream.Server = httptest.NewServer(http.HandlerFunc(
func(w http.ResponseWriter, r *http.Request) {
upstream.mu.Lock()
upstream.received = append(upstream.received, r.RequestURI)
upstream.mu.Unlock()
w.Header().Set("Content-Type", "image/png")
_, _ = w.Write(source)
}))
t.Cleanup(upstream.Close)
return upstream
}
// requestTargets returns the request targets the upstream has received, in
// the order it received them.
func (u *recordingUpstream) requestTargets() []string {
u.mu.Lock()
defer u.mu.Unlock()
return slices.Clone(u.received)
}
// pageOrigin is the origin of the login and generator pages below, opened
// over HTTPS; httptest gives every request the host example.com.
const pageOrigin = "https://example.com"
// openRoot sends GET / to s over HTTPS, with cookie when it is not nil, and
// returns the page: the login page, or the generator page for a login session.
func openRoot(
t *testing.T, s *Server, cookie *http.Cookie,
) *httptest.ResponseRecorder {
t.Helper()
req := httptest.NewRequestWithContext(t.Context(), http.MethodGet,
pageOrigin+"/", nil)
if cookie != nil {
req.AddCookie(cookie)
}
page := httptest.NewRecorder()
s.ServeHTTP(page, req)
return page
}
// submitForm posts form to path on s as a browser submits it from page over
// HTTPS: with the CSRF token tokenPattern finds in page, the cookies page set,
// cookie when it is not nil, and a Referer naming the page, which pixa needs
// while debug is off, as it is in startImageProxy's config.
func submitForm(
t *testing.T, s *Server, page *httptest.ResponseRecorder,
tokenPattern *regexp.Regexp, path string, form url.Values,
cookie *http.Cookie,
) *httptest.ResponseRecorder {
t.Helper()
match := tokenPattern.FindStringSubmatch(page.Body.String())
if match == nil {
t.Fatalf("the page has no form for %s with a CSRF token", path)
}
form.Set("gorilla.csrf.Token", match[1])
req := httptest.NewRequestWithContext(t.Context(), http.MethodPost,
pageOrigin+path, strings.NewReader(form.Encode()))
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
req.Header.Set("Referer", pageOrigin+"/")
for _, c := range page.Result().Cookies() {
req.AddCookie(c)
}
if cookie != nil {
req.AddCookie(cookie)
}
rec := httptest.NewRecorder()
s.ServeHTTP(rec, req)
return rec
}
// postGenerate logs in to s with the signing key, submits the generator page's
// form with sourceURL, and returns the response.
func postGenerate(
t *testing.T, s *Server, sourceURL string,
) *httptest.ResponseRecorder {
t.Helper()
login := submitForm(t, s, openRoot(t, s, nil), csrfFieldPattern, "/",
url.Values{"key": {testSigningKey}}, nil)
var sessionCookie *http.Cookie
for _, c := range login.Result().Cookies() {
if c.Name == session.CookieName {
sessionCookie = c
}
}
if sessionCookie == nil {
t.Fatalf("logging in answered %d and set no session cookie", login.Code)
}
rec := submitForm(t, s, openRoot(t, s, sessionCookie), generateFormPattern,
"/generate", url.Values{"url": {sourceURL}}, sessionCookie)
t.Logf("POST /generate with url %s: %d", sourceURL, rec.Code)
return rec
}
// TestImageURLFetchesItsSourcePathAndQuery requests image URLs whose source
// query holds a space, written percent-encoded, or a "'", and whose source
// path holds a "%", written percent-encoded, and checks the one request the
// upstream receives: the source path percent-encoded again, and the source
// query with the space written %20 and the "'" written %27, as a browser
// writes them in a query.
func TestImageURLFetchesItsSourcePathAndQuery(t *testing.T) {
t.Parallel()
tests := []struct {
name string
// pathAndQuery is the source path and query as the image URL writes
// them
pathAndQuery string
// wantRequest is the path and query the upstream must receive
wantRequest string
}{
{"space in the query", "/photo.png%3Fname=my%20cat",
"/photo.png?name=my%20cat"},
{"apostrophe in the query", "/photo.png%3Fname=it's",
"/photo.png?name=it%27s"},
{"percent sign in the path", percentSignPath, percentSignPath},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
upstream := startRecordingUpstream(t)
s, _, _ := startImageProxy(t, upstream.Server)
getImage(t, s, "/v1/image/"+upstreamHost+tt.pathAndQuery+"/orig.png")
got := upstream.requestTargets()
t.Logf("upstream received %q", got)
if !slices.Equal(got, []string{tt.wantRequest}) {
t.Errorf("upstream received %q, want [%q]", got, tt.wantRequest)
}
})
}
}
// TestSourcePathsHoldingSpaceAndPercent20AreFetchedApart requests twice each
// of the source paths /my cat.png and /my%20cat.png, which the image URL
// writes my%20cat.png and my%2520cat.png. Each is fetched once, as a request
// of its own, and its second request is served from the cache.
func TestSourcePathsHoldingSpaceAndPercent20AreFetchedApart(t *testing.T) {
t.Parallel()
upstream := startRecordingUpstream(t)
s, _, _ := startImageProxy(t, upstream.Server)
imageURLs := []string{
"/v1/image/" + upstreamHost + "/my%20cat.png/orig.png",
"/v1/image/" + upstreamHost + "/my%2520cat.png/orig.png",
}
for _, imageURL := range imageURLs {
getImage(t, s, imageURL)
}
for _, imageURL := range imageURLs {
cacheStatus := getImage(t, s, imageURL).Header().Get("X-Pixa-Cache")
if cacheStatus != string(imgcache.CacheHit) {
t.Errorf("second GET %s: X-Pixa-Cache = %q, want HIT",
imageURL, cacheStatus)
}
}
got := upstream.requestTargets()
t.Logf("upstream received %q", got)
want := []string{"/my%20cat.png", "/my%2520cat.png"}
if !slices.Equal(got, want) {
t.Errorf("upstream received %q, want %q", got, want)
}
}
// TestSourceQueryWrittenTwoWaysIsOneUpstreamRequest requests the source query
// name=my%20cat written as the README says, name=my%2520cat, and with the
// space written %20, name=my%20cat. Both are one source query: the upstream
// receives one request, and the second image URL is served from the cache.
func TestSourceQueryWrittenTwoWaysIsOneUpstreamRequest(t *testing.T) {
t.Parallel()
upstream := startRecordingUpstream(t)
s, _, _ := startImageProxy(t, upstream.Server)
getImage(t, s, "/v1/image/"+upstreamHost+"/photo.png%3Fname=my%2520cat/orig.png")
second := getImage(t, s,
"/v1/image/"+upstreamHost+"/photo.png%3Fname=my%20cat/orig.png")
if got := second.Header().Get("X-Pixa-Cache"); got != string(imgcache.CacheHit) {
t.Errorf("second X-Pixa-Cache = %q, want HIT", got)
}
got := upstream.requestTargets()
t.Logf("upstream received %q", got)
want := []string{"/photo.png?name=my%20cat"}
if !slices.Equal(got, want) {
t.Errorf("upstream received %q, want %q", got, want)
}
}
// TestGeneratedURLFetchesTheSourceURLAsEntered makes an encrypted URL on the
// generator page for source URLs holding %20, %25 and %23 in the path or the
// query, requests it, and checks that the upstream receives the path and
// query exactly as entered.
func TestGeneratedURLFetchesTheSourceURLAsEntered(t *testing.T) {
t.Parallel()
// The names hold no "%": each test's state directory is named after it
// and goes into the database URL, which decodes percent-escapes
tests := []struct {
name string
pathAndQuery string
}{
{"space in the path", "/my%20cat.png"},
{"percent sign in the path", percentSignPath},
{"number sign in the path", "/a%23b.png"},
{"all three in the query", "/photo.png?name=my%20cat%25%23"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
upstream := startRecordingUpstream(t)
s, _, _ := startImageProxy(t, upstream.Server)
rec := postGenerate(t, s, "https://"+upstreamHost+tt.pathAndQuery)
match := generatedURLPattern.FindStringSubmatch(rec.Body.String())
if rec.Code != http.StatusOK || match == nil {
t.Fatalf("POST /generate: %d with no encrypted URL; body %s",
rec.Code, rec.Body.String())
}
getImage(t, s, match[1])
got := upstream.requestTargets()
t.Logf("upstream received %q", got)
if !slices.Equal(got, []string{tt.pathAndQuery}) {
t.Errorf("upstream received %q, want [%q]", got, tt.pathAndQuery)
}
})
}
}
// TestGeneratorRefusesSourceURLItCannotFetchAsWritten submits source URLs the
// generator page must make no encrypted URL for, and checks that it answers
// 400 and shows why. pixa keeps the source path decoded and fetches it
// percent-encoded again, so a path holding %2F would be fetched with a "/"
// there, whatever else the path holds. pixa sends no user name or password,
// and no "?" with no query after it. For these the page shows the URL pixa
// would fetch. A source path holding a ".." segment is refused, as
// /v1/image/ refuses it.
func TestGeneratorRefusesSourceURLItCannotFetchAsWritten(t *testing.T) {
t.Parallel()
// The test config sets allow_http, so pixa would fetch over plain HTTP
const fetchedFrom = "http://" + upstreamHost
tests := []struct {
name string
// sourceURL is the source URL after "https://"
sourceURL string
// wantShown is what the page must show: the URL pixa would fetch, or
// why it makes no encrypted URL
wantShown string
}{
{"encoded slash", upstreamHost + "/a%2Fb.png",
fetchedFrom + "/a/b.png"},
{"encoded slash and a space", upstreamHost + "/a%2Fb c.png",
fetchedFrom + "/a/b%20c.png"},
{"encoded slash and a bar", upstreamHost + "/a%2Fb|c.png",
fetchedFrom + "/a/b%7Cc.png"},
{"encoded slash and a letter outside ASCII", upstreamHost + "/café%2Fx.png",
fetchedFrom + "/caf%C3%A9/x.png"},
{"user name and password", "user:pass@" + upstreamHost + "/a.png",
fetchedFrom + "/a.png"},
{"question mark with no query", upstreamHost + "/a.png?",
fetchedFrom + "/a.png"},
{"two-dot segment", upstreamHost + "/a/../b.png",
"cannot hold a .. segment"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
s, _, _ := startImageProxy(t, startRecordingUpstream(t).Server)
rec := postGenerate(t, s, "https://"+tt.sourceURL)
body := rec.Body.String()
if rec.Code != http.StatusBadRequest {
t.Errorf("status = %d, want %d", rec.Code, http.StatusBadRequest)
}
if strings.Contains(body, "/v1/e/") {
t.Error("the page shows an encrypted URL")
}
if !strings.Contains(body, tt.wantShown) {
t.Errorf("the page does not show %s; body %s", tt.wantShown, body)
}
})
}
}