5 Commits
Author SHA1 Message Date
sneak 0dac2c2120 Say what a 0 side does in the ValidateDimension doc comment
check / check (push) Canceled after 0s
It said 0 meant "original size"; the side is worked out from the
source's proportions, as README.md now says.

Model: opus-5-5
2026-10-16 01:59:24 +00:00
sneak dc1c069ad6 Keep each side at 1 or more with fit=outside too
fit=outside works both sides out again from the larger scale, and a side
scaled to 1 could come out just under 1 and round down to 0, which
libvips refuses: a 49x8192 source at 0x100 and a 49x49 source at 1x1
answered 500. README.md now says what fit=outside does, with the 8192x1
source at 100x0 that comes out at 8192x1, and how a 0 side is worked
out from the source's proportions.

Model: opus-5-5
2026-10-16 01:58:49 +00:00
sneak 5d30212352 Test that fit=outside keeps each side at 1 or more
A 49x8192 source at 0x100 and a 49x49 source at 1x1, both with
fit=outside. 49 times 1/49 is just under 1 in floating point, so the
width is rounded down to 0 and libvips refuses it. The test fails until
the fix for #283 covers fit=outside.

Model: opus-5-5
2026-10-16 01:58:13 +00:00
sneak 3126037a5d Keep a side computed for a size with one side 0 at 1 or more (closes #283)
The side was rounded down, so for a thin source it came out at 0, and
libvips made the size square: an 8192x1 source at 100x0 came out at
100x100, at 1300x0 with fit=cover libvips asked for about 39 GB, and an
8192x2 source at 100x0 answered 500. fit=contain and fit=inside work
both sides out again from the smaller scale and keep each at 1 or more
as well, as a 1x8192 source at 0x100 answered 500 there.

Model: opus-5-5
2026-10-16 01:58:13 +00:00
sneak 21fa87ddf6 Test that a side computed for a size with one side 0 is at least 1
An 8192x1 source at 100x0 and 1300x0 and an 8192x2 source at 100x0, and
the same turned on their side, in every fit mode. The test fails until
the fix for #283 lands; TODO.md
records the bug at the top as P1 until then.

Model: opus-5-5
2026-10-16 01:58:13 +00:00
9 changed files with 28 additions and 627 deletions
+9 -17
View File
@@ -473,11 +473,9 @@ nor change what it asks for.
`jxl` for a form with no format. `jxl` for a form with no format.
The token holds the source's host, path and query and the size, format, quality, The token holds the source's host, path and query and the size, format, quality,
fit and expiry, encrypted with a key derived from `signing_key`. The host is fit and expiry, encrypted with a key derived from `signing_key`. The source
kept in lower case and an empty path as `/`, as for `/v1/image/`, so URL's scheme is not kept: the image is fetched like any other (see Routes), and
`https://CDN.Example.com` and `https://cdn.example.com/` are one source image, the blocked networks still apply.
fetched and cached once. The source URL's scheme is not kept: the image is
fetched like any other (see Routes), and the blocked networks still apply.
How long the URL lasts is chosen on the page, from 1 minute to 1 year, or never. How long the URL lasts is chosen on the page, from 1 minute to 1 year, or never.
The expiry is fixed in the token when the URL is made and cannot be changed or The expiry is fixed in the token when the URL is made and cannot be changed or
@@ -694,10 +692,7 @@ Key settings in more detail:
and generator pages. Default: empty and generator pages. Default: empty
- `blocked_networks` — list of CIDR ranges to refuse for SSRF protection, added - `blocked_networks` — list of CIDR ranges to refuse for SSRF protection, added
to the always-enforced built-in ranges (loopback, private, link-local, CGNAT, to the always-enforced built-in ranges (loopback, private, link-local, CGNAT,
benchmark, NAT64, and the like); an invalid CIDR aborts startup. An IPv6 benchmark, NAT64, and the like); an invalid CIDR aborts startup
address in the IPv4-mapped form (`::ffff:a.b.c.d`) is matched only as the IPv4
address it holds, so an IPv6 range such as `::/0` refuses neither it nor a
plain IPv4 address
- `trusted_proxies` — list of CIDR ranges of the reverse proxies in front of - `trusted_proxies` — list of CIDR ranges of the reverse proxies in front of
pixa. `X-Forwarded-For` is believed only when the direct peer falls inside one pixa. `X-Forwarded-For` is believed only when the direct peer falls inside one
of these ranges; the logged and login-recorded client address is then the of these ranges; the logged and login-recorded client address is then the
@@ -728,14 +723,11 @@ Key settings in more detail:
- `signing_key` — HMAC secret for URL signatures - `signing_key` — HMAC secret for URL signatures
- `db_url` — the SQLite database to open; omitted, it is - `db_url` — the SQLite database to open; omitted, it is
`file:<state_dir>/state.sqlite3?_pragma=journal_mode(WAL)`, which keeps the `file:<state_dir>/state.sqlite3?_pragma=journal_mode(WAL)`, which keeps the
database in WAL mode. `state_dir` is percent-encoded in it, so a `%`, `?` or database in WAL mode. pixa opens one connection to it, so its own reads and
`#` in the name is written as `%25`, `%3F` or `%23`. For a `state_dir` of `/` writes run one at a time. pixa adds `_pragma=busy_timeout(5000)` to any
or one starting with `//` it starts `file:////`, so SQLite does not read the `db_url`, so a write that finds another program writing to the file waits up
first name in the path as a host. pixa opens one connection to it, so its own to five seconds for it instead of failing. WAL mode comes only from the URL:
reads and writes run one at a time. pixa adds `_pragma=busy_timeout(5000)` to keep `_pragma=journal_mode(WAL)` in one you set
any `db_url`, so a write that finds another program writing to the file waits
up to five seconds for it instead of failing. WAL mode comes only from the
URL: keep `_pragma=journal_mode(WAL)` in one you set
- `cache_max_bytes` — disk cache size limit in bytes; `0` disables the disk - `cache_max_bytes` — disk cache size limit in bytes; `0` disables the disk
cache entirely; omitted defaults to 75% of the sum of the free space on the cache entirely; omitted defaults to 75% of the sum of the free space on the
filesystem containing `<state_dir>/cache/` and the bytes of source and filesystem containing `<state_dir>/cache/` and the bytes of source and
-39
View File
@@ -30,40 +30,6 @@ P2: security: per-origin rate limiting
# Completed Steps # Completed Steps
- 2026-10-16 P1 security bug: the check that keeps pixa from fetching internal
addresses refuses the local-use NAT64 range (`64:ff9b:1::/48`, RFC 8215) as it
refuses `64:ff9b::/96`, and refuses an IPv6 address in the IPv4-compatible
form (`::/96`) when `blocked_networks` contains either the address as written
or the IPv4 address it holds (closes #299). A source host such as
`[64:ff9b:1::a00:1]` was fetched, which on a network that runs NAT64 on that
range reaches the IPv4 address it holds, here `10.0.0.1`; and with
`203.0.113.0/24` in `blocked_networks`, `[::ffff:203.0.113.5]` was refused but
`[::203.0.113.5]` was fetched. Both now answer 403. The mapped form
(`::ffff:a.b.c.d`) is still matched only as the IPv4 address it holds: Go
keeps a plain IPv4 address in that form, and an IPv6 entry such as `::/0` must
not refuse every IPv4 address.
- 2026-10-16 P1 security bug: the check that keeps pixa from fetching internal
addresses judges an IPv6 address in the IPv4-compatible form (`::/96`), such
as `::127.0.0.1` or `::10.0.0.1`, by the IPv4 address in its last four bytes
(closes #297). It read only the mapped form (`::ffff:a.b.c.d`) as IPv4, so a
source host such as `[::127.0.0.1]` was checked as a public IPv6 address and
fetched, which on a network that routes it reaches the internal IPv4 host; it
is now refused with 403. The NAT64 form (`64:ff9b::/96`) was already refused
as a whole, as one of the built-in blocked ranges, and still is.
- 2026-10-16 a source host that is an IPv6 address in brackets with no port,
such as `[2001:db8::a]`, is fetched like one with a port, through `/v1/image/`
and through an encrypted URL (closes #295). The fetcher's check of the source
URL kept the brackets when no port followed them, could not look the host up,
and the request answered 500. It now checks the address without brackets, so a
private one is refused with 403, as with a port.
- 2026-10-16 a `state_dir` whose name holds `%`, `?` or `#` keeps its database
at `state.sqlite3` in that directory (closes #290): the `db_url` derived from
it put the name into a `file:` URL as it was, so SQLite read `%41` as `A`, a
`?` or `#` ended the path early, and the journal mode parameter was lost. The
derived `db_url` is now built with Go's `net/url`, which writes those
characters as `%25`, `%3F` and `%23`. For a `state_dir` of `/` or one starting
with `//` it starts `file:////`, so SQLite no longer reads the first name in
the path as a host.
- 2026-10-16 a size with one side `0` keeps the side computed from the source's - 2026-10-16 a size with one side `0` keeps the side computed from the source's
proportions at 1 or more (closes #283): it was rounded down, so for a thin proportions at 1 or more (closes #283): it was rounded down, so for a thin
source it came out at 0, and libvips then made the size square. An 8192x1 source it came out at 0, and libvips then made the size square. An 8192x1
@@ -78,11 +44,6 @@ P2: security: per-origin rate limiting
the image is scaled to cover the size, so that source at `100x0` comes out at the image is scaled to cover the size, so that source at `100x0` comes out at
8192x1. `README.md` now says this under Signature Specification, and says how 8192x1. `README.md` now says this under Signature Specification, and says how
a `0` side is worked out where it said `0` meant the original. a `0` side is worked out where it said `0` meant the original.
- 2026-10-16 the generator page lower-cases the source host and writes an empty
source path as `/` before it makes the encrypted URL, as `/v1/image/` does
(closes #293). A host written in upper and in lower case, or a source URL with
no path and with the path `/`, made two cache keys and two upstream requests
for one image.
- 2026-10-16 the source URL a request fetches is built with `net/url` from the - 2026-10-16 the source URL a request fetches is built with `net/url` from the
source host, path and query, not joined as text (closes #288). The source source host, path and query, not joined as text (closes #288). The source
path, decoded, is fetched percent-encoded again, so `100%25.png` fetches path, decoded, is fetched percent-encoded again, so `100%25.png` fetches
+2 -25
View File
@@ -338,7 +338,8 @@ func newFromSmartConfig(sc *smartconfig.Config) (*Config, error) {
settingName(keyDBURL), errValueEmpty) settingName(keyDBURL), errValueEmpty)
} }
c.DBURL = defaultDBURL(c.StateDir) // The driver sets the journal mode only through a _pragma parameter.
c.DBURL = fmt.Sprintf("file:%s/state.sqlite3?_pragma=journal_mode(WAL)", c.StateDir)
} }
if loader.err != nil { if loader.err != nil {
@@ -353,30 +354,6 @@ func newFromSmartConfig(sc *smartconfig.Config) (*Config, error) {
return c, nil return c, nil
} }
// defaultDBURL returns the db_url used when db_url is omitted: the file
// state.sqlite3 in stateDir, in WAL mode, which the driver sets only
// through a _pragma parameter. String() percent-encodes any %, ? or # in
// the directory name, which SQLite decodes back, so the URL names exactly
// this file. OmitHost writes file:/var/lib/pixa/... instead of
// file:///var/lib/pixa/..., and keeps a relative stateDir from being read
// as a host name. The path written into the URL is stateDir followed by
// /state.sqlite3; when it starts with //, as it does for a stateDir of /
// or one starting with //, the empty host is kept, so the URL starts
// file://// and SQLite does not read the first name in the path as a
// host.
func defaultDBURL(stateDir string) string {
dbPath := stateDir + "/state.sqlite3"
dbURL := url.URL{
Scheme: "file",
OmitHost: !strings.HasPrefix(dbPath, "//"),
Path: dbPath,
RawQuery: "_pragma=journal_mode(WAL)",
}
return dbURL.String()
}
// validateKnownKeys rejects configuration files containing keys the // validateKnownKeys rejects configuration files containing keys the
// application does not understand, so typos fail at startup instead of // application does not understand, so typos fail at startup instead of
// being silently ignored, and rejects keys that are explicitly set to // being silently ignored, and rejects keys that are explicitly set to
@@ -135,104 +135,6 @@ func TestDefaultDBURLOpensTheDatabaseInWALMode(t *testing.T) {
} }
} }
// TestDerivedDBURLOpensStateDirHoldingURLCharacters opens the db_url
// derived from a state_dir whose name holds %41, ? and #, and checks that
// the database is state.sqlite3 in that directory, in WAL mode. Put into
// a file: URL as it is, the name would lose its meaning: %41 is read as
// A, and the path ends at the ?, which also loses the journal mode.
func TestDerivedDBURLOpensStateDirHoldingURLCharacters(t *testing.T) {
t.Parallel()
stateDir := filepath.Join(t.TempDir(), "a%41b?c#d")
err := os.Mkdir(stateDir, 0o750)
if err != nil {
t.Fatalf("failed to create %q: %v", stateDir, err)
}
// Quoted so YAML takes the whole name as it is.
c, err := configFromYAML(t, signingKeyLine+"state_dir: '"+stateDir+"'\n")
if err != nil {
t.Fatalf("config with state_dir %q should be valid, got: %v", stateDir, err)
}
t.Logf("state_dir %q gives db_url %q", stateDir, c.DBURL)
checkDBURLOpensFileInWALMode(t, c.DBURL, filepath.Join(stateDir, "state.sqlite3"))
}
// TestDerivedDBURLOpensStateDirStartingWithTwoSlashes opens the db_url
// derived from a state_dir that starts with //, and checks that the
// database is state.sqlite3 in that directory, in WAL mode. Written right
// after file:, such a name gives a URL starting file://, and SQLite reads
// the first directory name as a host and refuses to open it.
func TestDerivedDBURLOpensStateDirStartingWithTwoSlashes(t *testing.T) {
t.Parallel()
dir := t.TempDir()
stateDir := "/" + dir
c, err := configFromYAML(t, signingKeyLine+"state_dir: "+stateDir+"\n")
if err != nil {
t.Fatalf("config with state_dir %q should be valid, got: %v", stateDir, err)
}
t.Logf("state_dir %q gives db_url %q", stateDir, c.DBURL)
checkDBURLOpensFileInWALMode(t, c.DBURL, filepath.Join(dir, "state.sqlite3"))
}
// TestDerivedDBURLForRootStateDir checks the db_url derived from a
// state_dir of /. The path written into the URL is then
// //state.sqlite3, and written right after file: it gives a URL starting
// file://, where SQLite reads state.sqlite3 as a host and refuses to open
// it. The test only checks the URL, since it cannot write to /.
func TestDerivedDBURLForRootStateDir(t *testing.T) {
t.Parallel()
c, err := configFromYAML(t, signingKeyLine+"state_dir: /\n")
if err != nil {
t.Fatalf("config with state_dir / should be valid, got: %v", err)
}
t.Logf("state_dir / gives db_url %q", c.DBURL)
wantDBURL := "file:////state.sqlite3?_pragma=journal_mode(WAL)"
if c.DBURL != wantDBURL {
t.Errorf("DBURL = %q, want %q", c.DBURL, wantDBURL)
}
}
// checkDBURLOpensFileInWALMode opens dbURL with the SQLite driver pixad
// uses, and checks that the database is in WAL mode and is the file
// dbFile.
func checkDBURLOpensFileInWALMode(t *testing.T, dbURL, dbFile string) {
t.Helper()
db, err := sql.Open("sqlite", dbURL)
if err != nil {
t.Fatalf("failed to open %q: %v", dbURL, err)
}
t.Cleanup(func() { _ = db.Close() })
var journalMode string
err = db.QueryRowContext(t.Context(), "PRAGMA journal_mode").Scan(&journalMode)
if err != nil {
t.Fatalf("failed to read the journal mode of %q: %v", dbURL, err)
}
if journalMode != "wal" {
t.Errorf("journal mode of %q = %q, want wal", dbURL, journalMode)
}
_, err = os.Stat(dbFile)
if err != nil {
t.Errorf("opening %q did not create %q: %v", dbURL, dbFile, err)
}
}
func TestExplicitValidValuesAreUsed(t *testing.T) { func TestExplicitValidValuesAreUsed(t *testing.T) {
t.Parallel() t.Parallel()
+1 -20
View File
@@ -127,7 +127,7 @@ func (s *Handlers) HandleGenerateURL() http.HandlerFunc {
sourceURL := r.FormValue("url") sourceURL := r.FormValue("url")
// Validate source URL // Validate source URL
parsed, err := parseSourceURL(sourceURL) parsed, err := url.Parse(sourceURL)
if err != nil || parsed.Host == "" { if err != nil || parsed.Host == "" {
s.renderGeneratorWithForm(w, r, "Invalid source URL", r.Form, s.renderGeneratorWithForm(w, r, "Invalid source URL", r.Form,
http.StatusBadRequest) http.StatusBadRequest)
@@ -198,25 +198,6 @@ func (s *Handlers) HandleGenerateURL() http.HandlerFunc {
} }
} }
// parseSourceURL parses the source URL entered on the generator page with its
// host lower-cased and an empty path written as "/", as /v1/image/ reads a
// source URL, so that each spelling of one source URL makes one cache key and
// one upstream request.
func parseSourceURL(sourceURL string) (*url.URL, error) {
parsed, err := url.Parse(sourceURL)
if err != nil {
return nil, err
}
parsed.Host = strings.ToLower(parsed.Host)
if parsed.Path == "" {
parsed.Path = "/"
}
return parsed, nil
}
// fetchedAsWritten reports whether fetchURL, the URL pixa fetches for an // fetchedAsWritten reports whether fetchURL, the URL pixa fetches for an
// image URL made from the source URL source, is source as it was written, // image URL made from the source URL source, is source as it was written,
// apart from the scheme, which the token does not keep. It is not when source // apart from the scheme, which the token does not keep. It is not when source
+12 -39
View File
@@ -59,9 +59,8 @@ const (
// builtinBlockedPrefixes are internal or special-use ranges that Go's // builtinBlockedPrefixes are internal or special-use ranges that Go's
// net.IP predicates (IsPrivate, IsLinkLocalUnicast, and the like) do not // net.IP predicates (IsPrivate, IsLinkLocalUnicast, and the like) do not
// already cover. They are always blocked, in addition to any // already cover. They are always blocked, in addition to any
// operator-supplied networks. IPv4-mapped and IPv4-compatible IPv6 // operator-supplied networks. IPv4-mapped IPv6 addresses are unmapped
// addresses are read as the IPv4 address they hold before matching, so // before matching, so these IPv4 ranges are caught in both forms.
// these IPv4 ranges are caught in all three forms.
// //
//nolint:gochecknoglobals // immutable built-in blocklist //nolint:gochecknoglobals // immutable built-in blocklist
var builtinBlockedPrefixes = []netip.Prefix{ var builtinBlockedPrefixes = []netip.Prefix{
@@ -69,7 +68,6 @@ var builtinBlockedPrefixes = []netip.Prefix{
netip.MustParsePrefix("192.0.0.0/24"), // RFC 6890 IETF protocol assignments netip.MustParsePrefix("192.0.0.0/24"), // RFC 6890 IETF protocol assignments
netip.MustParsePrefix("198.18.0.0/15"), // RFC 2544 benchmarking range netip.MustParsePrefix("198.18.0.0/15"), // RFC 2544 benchmarking range
netip.MustParsePrefix("64:ff9b::/96"), // RFC 6052 NAT64 (maps onto IPv4) netip.MustParsePrefix("64:ff9b::/96"), // RFC 6052 NAT64 (maps onto IPv4)
netip.MustParsePrefix("64:ff9b:1::/48"), // RFC 8215 local-use NAT64 (maps onto IPv4)
} }
// Fetcher errors. // Fetcher errors.
@@ -475,18 +473,18 @@ func validateURL(ctx context.Context, rawURL string, allowHTTP bool) error {
return ErrUnsupportedScheme return ErrUnsupportedScheme
} }
parsedURL, err := neturl.Parse(rawURL) // Parse to extract host
if err != nil { host := extractHost(rawURL)
return fmt.Errorf("%w: %w", ErrInvalidHost, err)
}
// Hostname drops any port, and the brackets of an IPv6 address whether
// or not a port follows them
host := parsedURL.Hostname()
if host == "" { if host == "" {
return ErrInvalidHost return ErrInvalidHost
} }
// Remove port if present
h, _, err := net.SplitHostPort(host)
if err == nil {
host = h
}
// Block obvious localhost patterns // Block obvious localhost patterns
if isLocalhost(host) { if isLocalhost(host) {
return ErrSSRFBlocked return ErrSSRFBlocked
@@ -545,9 +543,6 @@ func isPrivateIP(ip net.IP) bool {
return true return true
} }
// An IPv4-compatible IPv6 address is judged as the IPv4 address it holds
ip = ipv4CompatibleToIPv4(ip)
// Check for loopback // Check for loopback
if ip.IsLoopback() { if ip.IsLoopback() {
return true return true
@@ -599,12 +594,7 @@ func isPrivateIP(ip net.IP) bool {
} }
// isBlockedIP reports whether ip is refused, either by the built-in // isBlockedIP reports whether ip is refused, either by the built-in
// internal-range check or by one of the operator-supplied prefixes. An // internal-range check or by one of the operator-supplied prefixes.
// IPv4-compatible address (in ::/96) is refused when a prefix contains the
// address as written or the IPv4 address it holds. An IPv4-mapped address (in
// ::ffff:0:0/96) is matched only as the IPv4 address it holds: Go keeps a
// plain IPv4 address in that form too, and an IPv6 prefix such as ::/0 must
// not refuse every IPv4 address.
func isBlockedIP(ip net.IP, blocked []netip.Prefix) bool { func isBlockedIP(ip net.IP, blocked []netip.Prefix) bool {
if isPrivateIP(ip) { if isPrivateIP(ip) {
return true return true
@@ -617,28 +607,11 @@ func isBlockedIP(ip net.IP, blocked []netip.Prefix) bool {
addr = addr.Unmap() addr = addr.Unmap()
// ip read as IPv4 where it is or holds an IPv4 address, otherwise the
// zero netip.Addr, which no prefix contains. Matching asIPv4 as well as
// addr refuses more only for an IPv4-compatible address.
asIPv4, _ := netip.AddrFromSlice(ipv4CompatibleToIPv4(ip).To4())
return slices.ContainsFunc(blocked, func(prefix netip.Prefix) bool { return slices.ContainsFunc(blocked, func(prefix netip.Prefix) bool {
return prefix.Contains(addr) || prefix.Contains(asIPv4) return prefix.Contains(addr)
}) })
} }
// ipv4CompatibleToIPv4 returns the IPv4 address that an IPv4-compatible IPv6
// address (in ::/96, such as ::127.0.0.1) holds in its last four bytes, and
// any other address unchanged.
func ipv4CompatibleToIPv4(ip net.IP) net.IP {
addr, ok := netip.AddrFromSlice(ip)
if ok && netip.MustParsePrefix("::/96").Contains(addr) {
return ip[12:]
}
return ip
}
// ssrfSafeDialer validates IP addresses against the built-in blocked ranges // ssrfSafeDialer validates IP addresses against the built-in blocked ranges
// before connecting. New wraps dialSSRFSafe with the operator-supplied // before connecting. New wraps dialSSRFSafe with the operator-supplied
// blocklist; this entry point enforces the built-in ranges alone. // blocklist; this entry point enforces the built-in ranges alone.
@@ -186,44 +186,6 @@ func TestIsPrivateIP(t *testing.T) {
} }
} }
// TestIsPrivateIPJudgesEmbeddedIPv4Address checks IPv6 addresses that hold an
// IPv4 address in their last four bytes: the IPv4-compatible form (::/96) and
// the NAT64 form (64:ff9b::/96). One holding a loopback, private or
// link-local IPv4 address is refused as that address is, and one holding a
// public IPv4 address in the IPv4-compatible form passes.
func TestIsPrivateIPJudgesEmbeddedIPv4Address(t *testing.T) {
t.Parallel()
tests := []struct {
ip string
want bool
}{
{"::127.0.0.1", true}, // loopback
{"::10.0.0.1", true}, // private
{"::169.254.169.254", true}, // link-local
{"64:ff9b::7f00:1", true}, // NAT64 form of 127.0.0.1
{"::8.8.8.8", false}, // public
}
for _, tc := range tests {
t.Run(tc.ip, func(t *testing.T) {
t.Parallel()
ip := net.ParseIP(tc.ip)
if ip == nil {
t.Fatalf("failed to parse IP %q", tc.ip)
}
got := isPrivateIP(ip)
t.Logf("isPrivateIP(%s) = %v", ip, got)
if got != tc.want {
t.Errorf("isPrivateIP(%q) = %v, want %v", tc.ip, got, tc.want)
}
})
}
}
func TestValidateURL_RejectsNonHTTPS(t *testing.T) { func TestValidateURL_RejectsNonHTTPS(t *testing.T) {
t.Parallel() t.Parallel()
@@ -14,12 +14,10 @@ import (
"net" "net"
"net/http" "net/http"
"net/http/httptest" "net/http/httptest"
"net/netip"
"os" "os"
"path/filepath" "path/filepath"
"sync/atomic" "sync/atomic"
"testing" "testing"
"time"
"go.uber.org/fx" "go.uber.org/fx"
"go.uber.org/fx/fxtest" "go.uber.org/fx/fxtest"
@@ -185,57 +183,6 @@ func startImageProxy(
return s, db.DB(), stateDir return s, db.DB(), stateDir
} }
// startImageProxyWithBlockedNetworks starts the components pixad's fx app
// builds, from a config with a fresh state directory and blockedNetworks as
// blocked_networks. Unlike startImageProxy it provides no upstream fetcher, so
// the handlers build the one pixad uses, which checks every address it
// connects to against the built-in blocked ranges and blocked_networks. It
// returns the server with its routes.
func startImageProxyWithBlockedNetworks(
t *testing.T, blockedNetworks []netip.Prefix,
) *Server {
t.Helper()
stateDir := t.TempDir()
cfg := &config.Config{
SigningKey: testSigningKey,
StateDir: stateDir,
DBURL: "file:" + filepath.Join(stateDir, "state.sqlite3"),
BlockedNetworks: blockedNetworks,
// A fetch that gets past the checks and finds nothing at the address
// fails within this time at the latest.
UpstreamFetchTimeout: 2 * time.Second,
// With no connection slots, the fetch would fail before connecting.
UpstreamConnections: config.DefaultUpstreamConnections,
DownstreamTimeout: config.DefaultDownstreamTimeout,
}
var (
h *handlers.Handlers
mw *middleware.Middleware
)
app := fxtest.New(t,
fx.Supply(cfg),
fx.Provide(
globals.New,
logger.New,
database.New,
healthcheck.New,
handlers.New,
middleware.New,
),
fx.Populate(&h, &mw),
)
app.RequireStart()
t.Cleanup(app.RequireStop)
s := &Server{config: cfg, mw: mw, h: h}
s.SetupRoutes()
return s
}
// getImage sends a GET for target to s and fails unless it answers 200. // getImage sends a GET for target to s and fails unless it answers 200.
func getImage(t *testing.T, s *Server, target string) *httptest.ResponseRecorder { func getImage(t *testing.T, s *Server, target string) *httptest.ResponseRecorder {
t.Helper() t.Helper()
-294
View File
@@ -3,20 +3,15 @@ package server
import ( import (
"net/http" "net/http"
"net/http/httptest" "net/http/httptest"
"net/netip"
"net/url" "net/url"
"regexp" "regexp"
"slices" "slices"
"strconv"
"strings" "strings"
"sync" "sync"
"testing" "testing"
"time"
"sneak.berlin/go/pixa/internal/encurl"
"sneak.berlin/go/pixa/internal/imgcache" "sneak.berlin/go/pixa/internal/imgcache"
"sneak.berlin/go/pixa/internal/session" "sneak.berlin/go/pixa/internal/session"
"sneak.berlin/go/pixa/internal/signature"
) )
// percentSignPath is a source path holding a "%", which a URL writes %25. // percentSignPath is a source path holding a "%", which a URL writes %25.
@@ -313,70 +308,6 @@ func TestGeneratedURLFetchesTheSourceURLAsEntered(t *testing.T) {
} }
} }
// TestGeneratorMakesOneImageURLForTwoSpellingsOfASourceURL enters two
// spellings of one source URL on the generator page, requests the encrypted
// URL made for each, and checks that the upstream receives one request and
// that the second encrypted URL is served from the cache. One pair writes the
// host in upper and in lower case, which /v1/image/ lower-cases. The host is
// an IPv6 address, whose hex digits have letter case, because the fetcher
// would look a host name up in DNS. It has a port, as the fetcher refuses an
// IPv6 address written without one. The other pair is a source URL with no
// path and with the path "/", both of which fetch "/".
func TestGeneratorMakesOneImageURLForTwoSpellingsOfASourceURL(t *testing.T) {
t.Parallel()
tests := []struct {
name string
// sourceURLs are the two spellings, after "https://"
sourceURLs []string
// wantRequest is the path the upstream must receive
wantRequest string
}{
{"host in upper and in lower case",
[]string{"[2001:DB8::A]:8443/a.png", "[2001:db8::a]:8443/a.png"},
"/a.png"},
{"no path and the path written as a slash",
[]string{upstreamHost, upstreamHost + "/"}, "/"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
upstream := startRecordingUpstream(t)
s, _, _ := startImageProxy(t, upstream.Server)
var imageURLs []string
for _, sourceURL := range tt.sourceURLs {
rec := postGenerate(t, s, "https://"+sourceURL)
match := generatedURLPattern.FindStringSubmatch(rec.Body.String())
if rec.Code != http.StatusOK || match == nil {
t.Fatalf("POST /generate: %d with no encrypted URL; body %s",
rec.Code, rec.Body.String())
}
imageURLs = append(imageURLs, match[1])
}
getImage(t, s, imageURLs[0])
second := getImage(t, s, imageURLs[1])
if got := second.Header().Get("X-Pixa-Cache"); got != string(imgcache.CacheHit) {
t.Errorf("second X-Pixa-Cache = %q, want HIT", got)
}
got := upstream.requestTargets()
t.Logf("upstream received %q", got)
if !slices.Equal(got, []string{tt.wantRequest}) {
t.Errorf("upstream received %q, want [%q]", got, tt.wantRequest)
}
})
}
}
// TestGeneratorRefusesSourceURLItCannotFetchAsWritten submits source URLs the // TestGeneratorRefusesSourceURLItCannotFetchAsWritten submits source URLs the
// generator page must make no encrypted URL for, and checks that it answers // generator page must make no encrypted URL for, and checks that it answers
// 400 and shows why. pixa keeps the source path decoded and fetches it // 400 and shows why. pixa keeps the source path decoded and fetches it
@@ -438,228 +369,3 @@ func TestGeneratorRefusesSourceURLItCannotFetchAsWritten(t *testing.T) {
}) })
} }
} }
// ipv6SourcePath is the source path of the image URLs signedImageURL and
// encryptedImageURL make.
const ipv6SourcePath = "/ipv6.png"
// TestImageRoutesFetchIPv6SourceHostWithOrWithoutPort requests ipv6SourcePath
// on an IPv6 source host, written with and without a port, through a
// /v1/image/ URL with a signature and through an encrypted URL. The public
// address 2001:db8::a, a documentation address (RFC 3849) the fetcher accepts
// as public, answers 200, and the upstream receives the request for
// ipv6SourcePath. The private address fd00::1 answers 403, and the upstream
// receives no request: the test's fetcher connects every address to the test
// upstream, so only the fetcher's check of the source URL can refuse it.
func TestImageRoutesFetchIPv6SourceHostWithOrWithoutPort(t *testing.T) {
t.Parallel()
tests := []struct {
name string
sourceHost string
wantStatus int
// wantRequests are the request targets the upstream must receive
wantRequests []string
}{
{"public address without a port", "[2001:db8::a]",
http.StatusOK, []string{ipv6SourcePath}},
{"public address with a port", "[2001:db8::a]:8443",
http.StatusOK, []string{ipv6SourcePath}},
{"private address without a port", "[fd00::1]",
http.StatusForbidden, nil},
{"private address with a port", "[fd00::1]:8443",
http.StatusForbidden, nil},
}
for _, tt := range tests {
imageURLs := map[string]string{
"image URL with a signature": signedImageURL(t, tt.sourceHost),
"encrypted URL": encryptedImageURL(t, tt.sourceHost),
}
for route, imageURL := range imageURLs {
t.Run(tt.name+" "+route, func(t *testing.T) {
t.Parallel()
upstream := startRecordingUpstream(t)
s, _, _ := startImageProxy(t, upstream.Server)
rec := httptest.NewRecorder()
s.ServeHTTP(rec, httptest.NewRequestWithContext(
t.Context(), http.MethodGet, imageURL, nil))
t.Logf("GET %s: %d", imageURL, rec.Code)
if rec.Code != tt.wantStatus {
t.Errorf("status = %d, want %d; body %s",
rec.Code, tt.wantStatus, rec.Body.String())
}
got := upstream.requestTargets()
t.Logf("upstream received %q", got)
if !slices.Equal(got, tt.wantRequests) {
t.Errorf("upstream received %q, want %q", got, tt.wantRequests)
}
})
}
}
}
// signedImageURL returns the /v1/image/ URL of ipv6SourcePath on sourceHost at
// its original size as PNG, with a signature made with the test signing key,
// which a host not on the allowlist needs. The URL has no q or fit, so it is
// signed with the quality 85 and the fit cover.
func signedImageURL(t *testing.T, sourceHost string) string {
t.Helper()
path, sig, exp := signature.New(testSigningKey).GenerateSignedURL(
&signature.Request{
SourceHost: sourceHost,
SourcePath: ipv6SourcePath,
Format: string(imgcache.FormatPNG),
Quality: 85,
FitMode: string(imgcache.FitCover),
}, time.Hour)
return path + "?sig=" + sig + "&exp=" + strconv.FormatInt(exp, 10)
}
// encryptedImageURL returns the encrypted URL of ipv6SourcePath on sourceHost
// at its original size as PNG, made with the test signing key.
func encryptedImageURL(t *testing.T, sourceHost string) string {
t.Helper()
generator, err := encurl.NewGenerator(testSigningKey)
if err != nil {
t.Fatalf("encurl.NewGenerator: %v", err)
}
token, err := generator.Generate(&encurl.Payload{
SourceHost: sourceHost, SourcePath: ipv6SourcePath,
Format: imgcache.FormatPNG,
})
if err != nil {
t.Fatalf("encurl Generate: %v", err)
}
return "/v1/e/" + token + "/img.png"
}
// TestImageRoutesRefuseIPv6SourceHostHoldingInternalIPv4Address requests
// ipv6SourcePath on IPv6 source hosts that hold a loopback or private IPv4
// address in their last four bytes, in the IPv4-compatible form (::/96) or the
// NAT64 form (64:ff9b::/96), through a /v1/image/ URL with a signature and
// through an encrypted URL. Each answers 403, and the upstream receives no
// request: the test's fetcher connects every address to the test upstream, so
// only the fetcher's check of the source URL can refuse it.
func TestImageRoutesRefuseIPv6SourceHostHoldingInternalIPv4Address(t *testing.T) {
t.Parallel()
upstream := startRecordingUpstream(t)
s, _, _ := startImageProxy(t, upstream.Server)
sourceHosts := []string{
"[::127.0.0.1]", // IPv4-compatible form of a loopback address
"[::10.0.0.1]", // IPv4-compatible form of a private address
"[64:ff9b::7f00:1]", // NAT64 form of 127.0.0.1
}
for _, sourceHost := range sourceHosts {
imageURLs := []string{
signedImageURL(t, sourceHost),
encryptedImageURL(t, sourceHost),
}
for _, imageURL := range imageURLs {
rec := httptest.NewRecorder()
s.ServeHTTP(rec, httptest.NewRequestWithContext(
t.Context(), http.MethodGet, imageURL, nil))
t.Logf("GET %s (source host %s): %d", imageURL, sourceHost, rec.Code)
if rec.Code != http.StatusForbidden {
t.Errorf("source host %s: status = %d, want %d; body %s",
sourceHost, rec.Code, http.StatusForbidden, rec.Body.String())
}
}
}
if got := upstream.requestTargets(); len(got) != 0 {
t.Errorf("upstream received %q, want no request", got)
}
}
// TestImageRoutesCheckIPv4AddressInIPv6SourceHost requests ipv6SourcePath on
// source hosts that hold an IPv4 address, through a /v1/image/ URL with a
// signature and through an encrypted URL, with the fetcher pixad uses.
// [64:ff9b:1::a00:1], in the local-use NAT64 range (64:ff9b:1::/48, RFC 8215),
// answers 403. [::203.0.113.5] holds the public address 203.0.113.5 in the
// IPv4-compatible form (::/96) and answers 403 when blocked_networks lists
// 203.0.113.0/24 or ::/96. [::ffff:203.0.113.5] holds it in the IPv4-mapped
// form (::ffff:0:0/96) and answers 403 when blocked_networks lists
// 203.0.113.0/24. Go keeps the plain address 203.0.113.5 in the mapped form
// too, and an IPv6 range must not refuse it: with ::/0 listed it is not
// refused. Neither is [::203.0.113.5] with blocked_networks empty. Where a
// host is not refused, the fetcher tries to connect to it, finds nothing
// there, and the request fails with another status.
func TestImageRoutesCheckIPv4AddressInIPv6SourceHost(t *testing.T) {
t.Parallel()
const (
ipv4CompatibleHost = "[::203.0.113.5]"
ipv4MappedHost = "[::ffff:203.0.113.5]"
)
tests := []struct {
name string
sourceHost string
blockedNetworks []netip.Prefix
wantForbidden bool
}{
{"local-use NAT64 form of 10.0.0.1", "[64:ff9b:1::a00:1]", nil, true},
{"IPv4-compatible form of an address in blocked_networks",
ipv4CompatibleHost,
[]netip.Prefix{netip.MustParsePrefix("203.0.113.0/24")}, true},
{"IPv4-compatible form with ::/96 in blocked_networks",
ipv4CompatibleHost,
[]netip.Prefix{netip.MustParsePrefix("::/96")}, true},
{"IPv4-mapped form of an address in blocked_networks",
ipv4MappedHost,
[]netip.Prefix{netip.MustParsePrefix("203.0.113.0/24")}, true},
{"plain IPv4 address with ::/0 in blocked_networks",
"203.0.113.5",
[]netip.Prefix{netip.MustParsePrefix("::/0")}, false},
{"IPv4-compatible form of an address with blocked_networks empty",
ipv4CompatibleHost, nil, false},
}
for _, tt := range tests {
imageURLs := map[string]string{
"image URL with a signature": signedImageURL(t, tt.sourceHost),
"encrypted URL": encryptedImageURL(t, tt.sourceHost),
}
for route, imageURL := range imageURLs {
t.Run(tt.name+" "+route, func(t *testing.T) {
t.Parallel()
s := startImageProxyWithBlockedNetworks(t, tt.blockedNetworks)
rec := httptest.NewRecorder()
s.ServeHTTP(rec, httptest.NewRequestWithContext(
t.Context(), http.MethodGet, imageURL, nil))
t.Logf("GET %s: %d", imageURL, rec.Code)
forbidden := rec.Code == http.StatusForbidden
if tt.wantForbidden && !forbidden {
t.Errorf("status = %d, want %d; body %s",
rec.Code, http.StatusForbidden, rec.Body.String())
}
if !tt.wantForbidden && forbidden {
t.Errorf("status = %d, want any other status; body %s",
rec.Code, rec.Body.String())
}
})
}
}
}