6 Commits
Author SHA1 Message Date
clawbot faad959bea Refuse local-use NAT64 addresses, and IPv4-compatible ones in blocked_networks (closes #299)
check / check (push) Waiting to run
The check that keeps pixa from fetching internal addresses let through
64:ff9b:1::/48, the local-use NAT64 range, and matched an address in the
IPv4-compatible form (::/96) against blocked_networks only as IPv6, so
::203.0.113.5 passed with 203.0.113.0/24 listed. The local-use NAT64
range is now blocked next to 64:ff9b::/96, and an address in ::/96 is
refused when blocked_networks holds it as written or the IPv4 address it
holds. The mapped form stays matched as IPv4 only, as Go keeps plain
IPv4 addresses in that form; README.md says so.

Model: opus-5-5
2026-10-16 06:52:17 +02:00
clawbot 3a6b3bc202 Refuse an IPv6 address that holds an internal IPv4 address (closes #297)
check / check (push) Canceled after 0s
The check that keeps pixa from fetching internal addresses read an
IPv6 address as IPv4 only in the mapped form (::ffff:a.b.c.d), so the
IPv4-compatible form, such as ::127.0.0.1, was judged as IPv6 and
passed. An address in ::/96 is now judged by the IPv4 address it holds,
so one holding a private, loopback or link-local address is refused on
both image routes before any upstream request. The NAT64 range
64:ff9b::/96 stays blocked whole, as it was on next.

Model: opus-5-5
2026-10-16 05:31:55 +02:00
clawbot 08d6c3c459 Fetch an IPv6 source host written without a port (closes #295)
check / check (push) Canceled after 0s
The fetcher took the host from the source URL and stripped a port with
net.SplitHostPort, which fails when there is no port, so a bracketed
IPv6 host such as [2001:db8::a] kept its brackets, its name lookup
failed, and the request answered 500. validateURL now takes the host
with url.URL.Hostname(), which drops the brackets and any port, so such
a host is fetched like one with a port, through /v1/image/ and through
an encrypted URL, and a private IPv6 address is still refused.

Model: opus-5-5
2026-10-16 05:00:47 +02:00
clawbot efa431ba18 Escape state_dir in the db_url derived from it (closes #290)
check / check (push) Canceled after 0s
When db_url is omitted, pixa derives it from state_dir, and it put the
directory name into a file: URL as it was, so a state_dir holding %, ?
or # opened another file and lost the journal mode. The derived URL is
now built with net/url, which percent-encodes the directory name; an
ordinary path gives the same URL as before. When the path written into
the URL starts with //, as for a state_dir of / or one starting with //,
the empty host is kept so SQLite does not read the first name in the
path as a host.

Model: opus-5-5
2026-10-16 04:37:54 +02:00
clawbot 8b4dcda549 Keep a side computed for a size with one side 0 at 1 or more (closes #283)
check / check (push) Canceled after 0s
A size with one side 0 has that side worked out from the source's
proportions by integer division, and nothing kept it at 1 or more. When
it rounded down to 0, libvips was asked for a square: an 8192x1 source
at 100x0 came out 100x100, at 1300x0 it asked for one block of about
39 GB, and some shapes answered 500. Each computed side, and each side
contain, inside and outside work out again from the scale, is now at
least 1. README.md now says what fit=outside does and what a 0 side
means; with fit=outside an 8192x1 source at 100x0 comes out 8192x1, as
covering the size requires.

Model: opus-5-5
2026-10-16 04:17:59 +02:00
clawbot 6d271d4617 Make one image URL per source on the generator page for host case and an empty path (closes #293)
check / check (push) Canceled after 0s
The generator page passed the source host and path into the encrypted
URL as entered, so a host spelled in another letter case, or a source
with no path against the same source with /, made separate cache keys
and fetched one upstream image twice. The page now lower-cases the
source host, as /v1/image/ does, and writes an empty source path as /,
before it builds the encrypted URL. The checks the page gained for
#288 still hold.

Model: opus-5-5
2026-10-16 04:04:19 +02:00
13 changed files with 783 additions and 44 deletions
+28 -12
View File
@@ -473,9 +473,11 @@ nor change what it asks for.
`jxl` for a form with no format.
The token holds the source's host, path and query and the size, format, quality,
fit and expiry, encrypted with a key derived from `signing_key`. The source
URL's scheme is not kept: the image is fetched like any other (see Routes), and
the blocked networks still apply.
fit and expiry, encrypted with a key derived from `signing_key`. The host is
kept in lower case and an empty path as `/`, as for `/v1/image/`, so
`https://CDN.Example.com` and `https://cdn.example.com/` are one source image,
fetched and cached once. The source URL's scheme is not kept: the image is
fetched like any other (see Routes), and the blocked networks still apply.
How long the URL lasts is chosen on the page, from 1 minute to 1 year, or never.
The expiry is fixed in the token when the URL is made and cannot be changed or
@@ -534,8 +536,13 @@ Where:
- `query` — source query string as under Routes, empty string if none, with `%`
and `:` written as in `path`; neither then holds a `:`, so path `/a:b` with no
query and path `/a` with query `b:` are signed differently
- `width` — requested width in pixels, `0` for original
- `height` — requested height in pixels, `0` for original
- `width` — requested width in pixels, or `0`: the width is then worked out from
the source's proportions and the requested height, rounded down but at least
1, or is the source's own width when the height is `0` too (the `orig` size)
- `height` — requested height in pixels, or `0`: the height is then worked out
from the source's proportions and the requested width, rounded down but at
least 1, or is the source's own height when the width is `0` too (the `orig`
size)
- `format` — output format, one of those listed under Routes, with `original`
signed as `orig`, `jpg` as `jpeg`, and no format as `jxl`, so a URL with no
format has the signature of the same URL ending in `.jxl`; `auto` is signed as
@@ -548,7 +555,10 @@ Where:
with 400
- `fit` — the URL's `fit` query parameter (cover, contain, fill, inside,
outside), or `cover` when the URL has no `fit`; a request whose `fit` is
anything else, an empty `fit=` included, is refused with 400
anything else, an empty `fit=` included, is refused with 400. `outside` scales
the image to cover the requested size, keeping its proportions, so it can come
out larger than that size on one side: an 8192x1 source at `100x0`, which is
100x1 by its proportions, comes out at 8192x1
The URL's `sig` is the HMAC-SHA256 result in base64url (the URL-safe alphabet of
RFC 4648) with the trailing `=` padding kept, 44 characters in all. pixa
@@ -684,7 +694,10 @@ Key settings in more detail:
and generator pages. Default: empty
- `blocked_networks` — list of CIDR ranges to refuse for SSRF protection, added
to the always-enforced built-in ranges (loopback, private, link-local, CGNAT,
benchmark, NAT64, and the like); an invalid CIDR aborts startup
benchmark, NAT64, and the like); an invalid CIDR aborts startup. An IPv6
address in the IPv4-mapped form (`::ffff:a.b.c.d`) is matched only as the IPv4
address it holds, so an IPv6 range such as `::/0` refuses neither it nor a
plain IPv4 address
- `trusted_proxies` — list of CIDR ranges of the reverse proxies in front of
pixa. `X-Forwarded-For` is believed only when the direct peer falls inside one
of these ranges; the logged and login-recorded client address is then the
@@ -715,11 +728,14 @@ Key settings in more detail:
- `signing_key` — HMAC secret for URL signatures
- `db_url` — the SQLite database to open; omitted, it is
`file:<state_dir>/state.sqlite3?_pragma=journal_mode(WAL)`, which keeps the
database in WAL mode. pixa opens one connection to it, so its own reads and
writes run one at a time. pixa adds `_pragma=busy_timeout(5000)` to any
`db_url`, so a write that finds another program writing to the file waits up
to five seconds for it instead of failing. WAL mode comes only from the URL:
keep `_pragma=journal_mode(WAL)` in one you set
database in WAL mode. `state_dir` is percent-encoded in it, so a `%`, `?` or
`#` in the name is written as `%25`, `%3F` or `%23`. For a `state_dir` of `/`
or one starting with `//` it starts `file:////`, so SQLite does not read the
first name in the path as a host. pixa opens one connection to it, so its own
reads and writes run one at a time. pixa adds `_pragma=busy_timeout(5000)` to
any `db_url`, so a write that finds another program writing to the file waits
up to five seconds for it instead of failing. WAL mode comes only from the
URL: keep `_pragma=journal_mode(WAL)` in one you set
- `cache_max_bytes` — disk cache size limit in bytes; `0` disables the disk
cache entirely; omitted defaults to 75% of the sum of the free space on the
filesystem containing `<state_dir>/cache/` and the bytes of source and
+53
View File
@@ -30,6 +30,59 @@ P2: security: per-origin rate limiting
# Completed Steps
- 2026-10-16 P1 security bug: the check that keeps pixa from fetching internal
addresses refuses the local-use NAT64 range (`64:ff9b:1::/48`, RFC 8215) as it
refuses `64:ff9b::/96`, and refuses an IPv6 address in the IPv4-compatible
form (`::/96`) when `blocked_networks` contains either the address as written
or the IPv4 address it holds (closes #299). A source host such as
`[64:ff9b:1::a00:1]` was fetched, which on a network that runs NAT64 on that
range reaches the IPv4 address it holds, here `10.0.0.1`; and with
`203.0.113.0/24` in `blocked_networks`, `[::ffff:203.0.113.5]` was refused but
`[::203.0.113.5]` was fetched. Both now answer 403. The mapped form
(`::ffff:a.b.c.d`) is still matched only as the IPv4 address it holds: Go
keeps a plain IPv4 address in that form, and an IPv6 entry such as `::/0` must
not refuse every IPv4 address.
- 2026-10-16 P1 security bug: the check that keeps pixa from fetching internal
addresses judges an IPv6 address in the IPv4-compatible form (`::/96`), such
as `::127.0.0.1` or `::10.0.0.1`, by the IPv4 address in its last four bytes
(closes #297). It read only the mapped form (`::ffff:a.b.c.d`) as IPv4, so a
source host such as `[::127.0.0.1]` was checked as a public IPv6 address and
fetched, which on a network that routes it reaches the internal IPv4 host; it
is now refused with 403. The NAT64 form (`64:ff9b::/96`) was already refused
as a whole, as one of the built-in blocked ranges, and still is.
- 2026-10-16 a source host that is an IPv6 address in brackets with no port,
such as `[2001:db8::a]`, is fetched like one with a port, through `/v1/image/`
and through an encrypted URL (closes #295). The fetcher's check of the source
URL kept the brackets when no port followed them, could not look the host up,
and the request answered 500. It now checks the address without brackets, so a
private one is refused with 403, as with a port.
- 2026-10-16 a `state_dir` whose name holds `%`, `?` or `#` keeps its database
at `state.sqlite3` in that directory (closes #290): the `db_url` derived from
it put the name into a `file:` URL as it was, so SQLite read `%41` as `A`, a
`?` or `#` ended the path early, and the journal mode parameter was lost. The
derived `db_url` is now built with Go's `net/url`, which writes those
characters as `%25`, `%3F` and `%23`. For a `state_dir` of `/` or one starting
with `//` it starts `file:////`, so SQLite no longer reads the first name in
the path as a host.
- 2026-10-16 a size with one side `0` keeps the side computed from the source's
proportions at 1 or more (closes #283): it was rounded down, so for a thin
source it came out at 0, and libvips then made the size square. An 8192x1
source at `100x0` came out at 100x100, and at `1300x0` with `fit=cover` made
libvips ask for about 39 GB at once; an 8192x2 source at `100x0` answered 500.
`fit=contain` and `fit=inside` work both sides out again from the smaller of
the two scales and `fit=outside` from the larger, and each keeps both sides at
1 or more too, so a 1x8192 source at `0x100` with `fit=contain`, and a 49x8192
source at `0x100` or a 49x49 source at `1x1` with `fit=outside`, no longer
answer 500. An 8192x1 source at `1300x0` now comes out at 1300x1, and the
process running it alone peaked at about 34 MiB resident. With `fit=outside`
the image is scaled to cover the size, so that source at `100x0` comes out at
8192x1. `README.md` now says this under Signature Specification, and says how
a `0` side is worked out where it said `0` meant the original.
- 2026-10-16 the generator page lower-cases the source host and writes an empty
source path as `/` before it makes the encrypted URL, as `/v1/image/` does
(closes #293). A host written in upper and in lower case, or a source URL with
no path and with the path `/`, made two cache keys and two upstream requests
for one image.
- 2026-10-16 the source URL a request fetches is built with `net/url` from the
source host, path and query, not joined as text (closes #288). The source
path, decoded, is fetched percent-encoded again, so `100%25.png` fetches
+25 -2
View File
@@ -338,8 +338,7 @@ func newFromSmartConfig(sc *smartconfig.Config) (*Config, error) {
settingName(keyDBURL), errValueEmpty)
}
// The driver sets the journal mode only through a _pragma parameter.
c.DBURL = fmt.Sprintf("file:%s/state.sqlite3?_pragma=journal_mode(WAL)", c.StateDir)
c.DBURL = defaultDBURL(c.StateDir)
}
if loader.err != nil {
@@ -354,6 +353,30 @@ func newFromSmartConfig(sc *smartconfig.Config) (*Config, error) {
return c, nil
}
// defaultDBURL returns the db_url used when db_url is omitted: the file
// state.sqlite3 in stateDir, in WAL mode, which the driver sets only
// through a _pragma parameter. String() percent-encodes any %, ? or # in
// the directory name, which SQLite decodes back, so the URL names exactly
// this file. OmitHost writes file:/var/lib/pixa/... instead of
// file:///var/lib/pixa/..., and keeps a relative stateDir from being read
// as a host name. The path written into the URL is stateDir followed by
// /state.sqlite3; when it starts with //, as it does for a stateDir of /
// or one starting with //, the empty host is kept, so the URL starts
// file://// and SQLite does not read the first name in the path as a
// host.
func defaultDBURL(stateDir string) string {
dbPath := stateDir + "/state.sqlite3"
dbURL := url.URL{
Scheme: "file",
OmitHost: !strings.HasPrefix(dbPath, "//"),
Path: dbPath,
RawQuery: "_pragma=journal_mode(WAL)",
}
return dbURL.String()
}
// validateKnownKeys rejects configuration files containing keys the
// application does not understand, so typos fail at startup instead of
// being silently ignored, and rejects keys that are explicitly set to
@@ -135,6 +135,104 @@ func TestDefaultDBURLOpensTheDatabaseInWALMode(t *testing.T) {
}
}
// TestDerivedDBURLOpensStateDirHoldingURLCharacters opens the db_url
// derived from a state_dir whose name holds %41, ? and #, and checks that
// the database is state.sqlite3 in that directory, in WAL mode. Put into
// a file: URL as it is, the name would lose its meaning: %41 is read as
// A, and the path ends at the ?, which also loses the journal mode.
func TestDerivedDBURLOpensStateDirHoldingURLCharacters(t *testing.T) {
t.Parallel()
stateDir := filepath.Join(t.TempDir(), "a%41b?c#d")
err := os.Mkdir(stateDir, 0o750)
if err != nil {
t.Fatalf("failed to create %q: %v", stateDir, err)
}
// Quoted so YAML takes the whole name as it is.
c, err := configFromYAML(t, signingKeyLine+"state_dir: '"+stateDir+"'\n")
if err != nil {
t.Fatalf("config with state_dir %q should be valid, got: %v", stateDir, err)
}
t.Logf("state_dir %q gives db_url %q", stateDir, c.DBURL)
checkDBURLOpensFileInWALMode(t, c.DBURL, filepath.Join(stateDir, "state.sqlite3"))
}
// TestDerivedDBURLOpensStateDirStartingWithTwoSlashes opens the db_url
// derived from a state_dir that starts with //, and checks that the
// database is state.sqlite3 in that directory, in WAL mode. Written right
// after file:, such a name gives a URL starting file://, and SQLite reads
// the first directory name as a host and refuses to open it.
func TestDerivedDBURLOpensStateDirStartingWithTwoSlashes(t *testing.T) {
t.Parallel()
dir := t.TempDir()
stateDir := "/" + dir
c, err := configFromYAML(t, signingKeyLine+"state_dir: "+stateDir+"\n")
if err != nil {
t.Fatalf("config with state_dir %q should be valid, got: %v", stateDir, err)
}
t.Logf("state_dir %q gives db_url %q", stateDir, c.DBURL)
checkDBURLOpensFileInWALMode(t, c.DBURL, filepath.Join(dir, "state.sqlite3"))
}
// TestDerivedDBURLForRootStateDir checks the db_url derived from a
// state_dir of /. The path written into the URL is then
// //state.sqlite3, and written right after file: it gives a URL starting
// file://, where SQLite reads state.sqlite3 as a host and refuses to open
// it. The test only checks the URL, since it cannot write to /.
func TestDerivedDBURLForRootStateDir(t *testing.T) {
t.Parallel()
c, err := configFromYAML(t, signingKeyLine+"state_dir: /\n")
if err != nil {
t.Fatalf("config with state_dir / should be valid, got: %v", err)
}
t.Logf("state_dir / gives db_url %q", c.DBURL)
wantDBURL := "file:////state.sqlite3?_pragma=journal_mode(WAL)"
if c.DBURL != wantDBURL {
t.Errorf("DBURL = %q, want %q", c.DBURL, wantDBURL)
}
}
// checkDBURLOpensFileInWALMode opens dbURL with the SQLite driver pixad
// uses, and checks that the database is in WAL mode and is the file
// dbFile.
func checkDBURLOpensFileInWALMode(t *testing.T, dbURL, dbFile string) {
t.Helper()
db, err := sql.Open("sqlite", dbURL)
if err != nil {
t.Fatalf("failed to open %q: %v", dbURL, err)
}
t.Cleanup(func() { _ = db.Close() })
var journalMode string
err = db.QueryRowContext(t.Context(), "PRAGMA journal_mode").Scan(&journalMode)
if err != nil {
t.Fatalf("failed to read the journal mode of %q: %v", dbURL, err)
}
if journalMode != "wal" {
t.Errorf("journal mode of %q = %q, want wal", dbURL, journalMode)
}
_, err = os.Stat(dbFile)
if err != nil {
t.Errorf("opening %q did not create %q: %v", dbURL, dbFile, err)
}
}
func TestExplicitValidValuesAreUsed(t *testing.T) {
t.Parallel()
+20 -1
View File
@@ -127,7 +127,7 @@ func (s *Handlers) HandleGenerateURL() http.HandlerFunc {
sourceURL := r.FormValue("url")
// Validate source URL
parsed, err := url.Parse(sourceURL)
parsed, err := parseSourceURL(sourceURL)
if err != nil || parsed.Host == "" {
s.renderGeneratorWithForm(w, r, "Invalid source URL", r.Form,
http.StatusBadRequest)
@@ -198,6 +198,25 @@ func (s *Handlers) HandleGenerateURL() http.HandlerFunc {
}
}
// parseSourceURL parses the source URL entered on the generator page with its
// host lower-cased and an empty path written as "/", as /v1/image/ reads a
// source URL, so that each spelling of one source URL makes one cache key and
// one upstream request.
func parseSourceURL(sourceURL string) (*url.URL, error) {
parsed, err := url.Parse(sourceURL)
if err != nil {
return nil, err
}
parsed.Host = strings.ToLower(parsed.Host)
if parsed.Path == "" {
parsed.Path = "/"
}
return parsed, nil
}
// fetchedAsWritten reports whether fetchURL, the URL pixa fetches for an
// image URL made from the source URL source, is source as it was written,
// apart from the scheme, which the token does not keep. It is not when source
+43 -16
View File
@@ -59,15 +59,17 @@ const (
// builtinBlockedPrefixes are internal or special-use ranges that Go's
// net.IP predicates (IsPrivate, IsLinkLocalUnicast, and the like) do not
// already cover. They are always blocked, in addition to any
// operator-supplied networks. IPv4-mapped IPv6 addresses are unmapped
// before matching, so these IPv4 ranges are caught in both forms.
// operator-supplied networks. IPv4-mapped and IPv4-compatible IPv6
// addresses are read as the IPv4 address they hold before matching, so
// these IPv4 ranges are caught in all three forms.
//
//nolint:gochecknoglobals // immutable built-in blocklist
var builtinBlockedPrefixes = []netip.Prefix{
netip.MustParsePrefix("100.64.0.0/10"), // RFC 6598 CGNAT / carrier-grade NAT
netip.MustParsePrefix("192.0.0.0/24"), // RFC 6890 IETF protocol assignments
netip.MustParsePrefix("198.18.0.0/15"), // RFC 2544 benchmarking range
netip.MustParsePrefix("64:ff9b::/96"), // RFC 6052 NAT64 (maps onto IPv4)
netip.MustParsePrefix("100.64.0.0/10"), // RFC 6598 CGNAT / carrier-grade NAT
netip.MustParsePrefix("192.0.0.0/24"), // RFC 6890 IETF protocol assignments
netip.MustParsePrefix("198.18.0.0/15"), // RFC 2544 benchmarking range
netip.MustParsePrefix("64:ff9b::/96"), // RFC 6052 NAT64 (maps onto IPv4)
netip.MustParsePrefix("64:ff9b:1::/48"), // RFC 8215 local-use NAT64 (maps onto IPv4)
}
// Fetcher errors.
@@ -473,16 +475,16 @@ func validateURL(ctx context.Context, rawURL string, allowHTTP bool) error {
return ErrUnsupportedScheme
}
// Parse to extract host
host := extractHost(rawURL)
if host == "" {
return ErrInvalidHost
parsedURL, err := neturl.Parse(rawURL)
if err != nil {
return fmt.Errorf("%w: %w", ErrInvalidHost, err)
}
// Remove port if present
h, _, err := net.SplitHostPort(host)
if err == nil {
host = h
// Hostname drops any port, and the brackets of an IPv6 address whether
// or not a port follows them
host := parsedURL.Hostname()
if host == "" {
return ErrInvalidHost
}
// Block obvious localhost patterns
@@ -543,6 +545,9 @@ func isPrivateIP(ip net.IP) bool {
return true
}
// An IPv4-compatible IPv6 address is judged as the IPv4 address it holds
ip = ipv4CompatibleToIPv4(ip)
// Check for loopback
if ip.IsLoopback() {
return true
@@ -594,7 +599,12 @@ func isPrivateIP(ip net.IP) bool {
}
// isBlockedIP reports whether ip is refused, either by the built-in
// internal-range check or by one of the operator-supplied prefixes.
// internal-range check or by one of the operator-supplied prefixes. An
// IPv4-compatible address (in ::/96) is refused when a prefix contains the
// address as written or the IPv4 address it holds. An IPv4-mapped address (in
// ::ffff:0:0/96) is matched only as the IPv4 address it holds: Go keeps a
// plain IPv4 address in that form too, and an IPv6 prefix such as ::/0 must
// not refuse every IPv4 address.
func isBlockedIP(ip net.IP, blocked []netip.Prefix) bool {
if isPrivateIP(ip) {
return true
@@ -607,11 +617,28 @@ func isBlockedIP(ip net.IP, blocked []netip.Prefix) bool {
addr = addr.Unmap()
// ip read as IPv4 where it is or holds an IPv4 address, otherwise the
// zero netip.Addr, which no prefix contains. Matching asIPv4 as well as
// addr refuses more only for an IPv4-compatible address.
asIPv4, _ := netip.AddrFromSlice(ipv4CompatibleToIPv4(ip).To4())
return slices.ContainsFunc(blocked, func(prefix netip.Prefix) bool {
return prefix.Contains(addr)
return prefix.Contains(addr) || prefix.Contains(asIPv4)
})
}
// ipv4CompatibleToIPv4 returns the IPv4 address that an IPv4-compatible IPv6
// address (in ::/96, such as ::127.0.0.1) holds in its last four bytes, and
// any other address unchanged.
func ipv4CompatibleToIPv4(ip net.IP) net.IP {
addr, ok := netip.AddrFromSlice(ip)
if ok && netip.MustParsePrefix("::/96").Contains(addr) {
return ip[12:]
}
return ip
}
// ssrfSafeDialer validates IP addresses against the built-in blocked ranges
// before connecting. New wraps dialSSRFSafe with the operator-supplied
// blocklist; this entry point enforces the built-in ranges alone.
@@ -186,6 +186,44 @@ func TestIsPrivateIP(t *testing.T) {
}
}
// TestIsPrivateIPJudgesEmbeddedIPv4Address checks IPv6 addresses that hold an
// IPv4 address in their last four bytes: the IPv4-compatible form (::/96) and
// the NAT64 form (64:ff9b::/96). One holding a loopback, private or
// link-local IPv4 address is refused as that address is, and one holding a
// public IPv4 address in the IPv4-compatible form passes.
func TestIsPrivateIPJudgesEmbeddedIPv4Address(t *testing.T) {
t.Parallel()
tests := []struct {
ip string
want bool
}{
{"::127.0.0.1", true}, // loopback
{"::10.0.0.1", true}, // private
{"::169.254.169.254", true}, // link-local
{"64:ff9b::7f00:1", true}, // NAT64 form of 127.0.0.1
{"::8.8.8.8", false}, // public
}
for _, tc := range tests {
t.Run(tc.ip, func(t *testing.T) {
t.Parallel()
ip := net.ParseIP(tc.ip)
if ip == nil {
t.Fatalf("failed to parse IP %q", tc.ip)
}
got := isPrivateIP(ip)
t.Logf("isPrivateIP(%s) = %v", ip, got)
if got != tc.want {
t.Errorf("isPrivateIP(%q) = %v, want %v", tc.ip, got, tc.want)
}
})
}
}
func TestValidateURL_RejectsNonHTTPS(t *testing.T) {
t.Parallel()
@@ -50,3 +50,69 @@ func TestImageProcessor_RefusesComputedSideAboveMaximum(t *testing.T) {
})
}
}
// TestImageProcessor_ComputedSideIsAtLeastOne verifies that, for a size with
// one side 0, the side computed from the input's proportions is 1 where it
// would round down to 0, and that Process makes that size in every fit mode
// but FitOutside, which scales the input to cover the size and so can make
// it longer. Each case checks the size targetDimensions gives before it
// calls Process: with that side at 0, an 8192x1 input at 1300x0 makes
// libvips ask for about 39 GB with FitCover.
func TestImageProcessor_ComputedSideIsAtLeastOne(t *testing.T) {
t.Parallel()
tests := []struct {
name string
inputWidth, inputHeight int
size Size
wantWidth, wantHeight int
}{
{"8192x1 at 100x0", 8192, 1, Size{Width: 100}, 100, 1},
{"8192x1 at 1300x0", 8192, 1, Size{Width: 1300}, 1300, 1},
{"8192x2 at 100x0", 8192, 2, Size{Width: 100}, 100, 1},
{"1x8192 at 0x100", 1, 8192, Size{Height: 100}, 1, 100},
{"1x8192 at 0x1300", 1, 8192, Size{Height: 1300}, 1, 1300},
{"2x8192 at 0x100", 2, 8192, Size{Height: 100}, 1, 100},
}
fits := []FitMode{FitCover, FitContain, FitFill, FitInside, FitOutside}
for _, tt := range tests {
for _, fit := range fits {
t.Run(tt.name+" "+string(fit), func(t *testing.T) {
t.Parallel()
width, height, err := targetDimensions(tt.size,
tt.inputWidth, tt.inputHeight)
if err != nil || width != tt.wantWidth || height != tt.wantHeight {
t.Fatalf("targetDimensions() = %dx%d, %v, want %dx%d",
width, height, err, tt.wantWidth, tt.wantHeight)
}
input := createTestJPEG(t, tt.inputWidth, tt.inputHeight)
result, err := New(Params{}).Process(t.Context(),
bytes.NewReader(input),
&Request{Size: tt.size, Format: FormatJPEG, FitMode: fit})
if err != nil {
t.Fatalf("Process() error = %v", err)
}
defer func() { _ = result.Content.Close() }()
t.Logf("Process() made %dx%d", result.Width, result.Height)
ok := result.Width == tt.wantWidth && result.Height == tt.wantHeight
if fit == FitOutside {
ok = result.Width >= tt.wantWidth && result.Height >= tt.wantHeight
}
if !ok {
t.Errorf("Process() made %dx%d, want %dx%d (at least, for %s)",
result.Width, result.Height, tt.wantWidth, tt.wantHeight,
FitOutside)
}
})
}
}
}
@@ -60,3 +60,45 @@ func TestImageProcessor_RefusesFitOutsideAboveMaximum(t *testing.T) {
})
}
}
// TestImageProcessor_FitOutsideSideIsAtLeastOne verifies that FitOutside
// keeps each side at 1 or more when it works both sides out again from the
// larger scale. A side of 49 scaled to 1 is 49 times 1/49, which in floating
// point is just under 1 and so rounded down to 0, a width libvips refuses.
func TestImageProcessor_FitOutsideSideIsAtLeastOne(t *testing.T) {
t.Parallel()
tests := []struct {
name string
inputWidth, inputHeight int
size Size
wantWidth, wantHeight int
}{
{"49x8192 at 0x100", 49, 8192, Size{Height: 100}, 1, 167},
{"49x49 at 1x1", 49, 49, Size{Width: 1, Height: 1}, 1, 1},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
input := createTestJPEG(t, tt.inputWidth, tt.inputHeight)
result, err := New(Params{}).Process(t.Context(),
bytes.NewReader(input),
&Request{Size: tt.size, Format: FormatJPEG, FitMode: FitOutside})
if err != nil {
t.Fatalf("Process() error = %v", err)
}
defer func() { _ = result.Content.Close() }()
t.Logf("Process() made %dx%d", result.Width, result.Height)
if result.Width != tt.wantWidth || result.Height != tt.wantHeight {
t.Errorf("Process() made %dx%d, want %dx%d",
result.Width, result.Height, tt.wantWidth, tt.wantHeight)
}
})
}
}
+20 -12
View File
@@ -311,8 +311,10 @@ func (p *ImageProcessor) Process(
// targetDimensions calculates the output dimensions for a requested size,
// scaling proportionally when only one dimension is given and keeping the
// original dimensions when both are zero. A side scaled proportionally can
// come out above MaxOutputDimension, which is ErrOutputTooLarge.
// original dimensions when both are zero. A side scaled proportionally is
// rounded down but kept at 1 or more, as for a thin image it would round
// down to 0, a side of no pixels. It can come out above MaxOutputDimension,
// which is ErrOutputTooLarge.
func targetDimensions(size Size, origWidth, origHeight int) (int, int, error) {
width, height := size.Width, size.Height
@@ -322,10 +324,10 @@ func targetDimensions(size Size, origWidth, origHeight int) (int, int, error) {
return origWidth, origHeight, nil
case width == 0:
// Only height specified: calculate width proportionally
width = origWidth * height / origHeight
width = max(1, origWidth*height/origHeight)
case height == 0:
// Only width specified: calculate height proportionally
height = origHeight * width / origWidth
height = max(1, origHeight*width/origWidth)
}
if width > MaxOutputDimension || height > MaxOutputDimension {
@@ -479,13 +481,16 @@ func (p *ImageProcessor) resize(
return cropCentreAndResize(img, width, height)
case FitContain:
// Resize to fit within dimensions, maintaining aspect ratio
// Resize to fit within dimensions, maintaining aspect ratio. Scaled
// by the smaller scale, the short side of a thin image can round
// down to 0, which libvips refuses as a width, so each side is at
// least 1.
imgW, imgH := img.Width(), img.Height()
scaleW := float64(width) / float64(imgW)
scaleH := float64(height) / float64(imgH)
scale := min(scaleW, scaleH)
newW := int(float64(imgW) * scale)
newH := int(float64(imgH) * scale)
newW := max(1, int(float64(imgW)*scale))
newH := max(1, int(float64(imgH)*scale))
return img.Thumbnail(newW, newH, vips.InterestingNone)
@@ -503,19 +508,22 @@ func (p *ImageProcessor) resize(
scaleW := float64(width) / float64(imgW)
scaleH := float64(height) / float64(imgH)
scale := min(scaleW, scaleH)
newW := int(float64(imgW) * scale)
newH := int(float64(imgH) * scale)
newW := max(1, int(float64(imgW)*scale))
newH := max(1, int(float64(imgH)*scale))
return img.Thumbnail(newW, newH, vips.InterestingNone)
case FitOutside:
// Resize so smallest dimension fits, may exceed target on other dimension
// Resize so smallest dimension fits, may exceed target on other
// dimension. A side scaled to 1 can come out just under 1 in
// floating point, as 49 times 1/49 does, and round down to 0, which
// libvips refuses as a width, so each side is at least 1.
imgW, imgH := img.Width(), img.Height()
scaleW := float64(width) / float64(imgW)
scaleH := float64(height) / float64(imgH)
scale := max(scaleW, scaleH)
newW := int(float64(imgW) * scale)
newH := int(float64(imgH) * scale)
newW := max(1, int(float64(imgW)*scale))
newH := max(1, int(float64(imgH)*scale))
// The dimension that exceeds the target is held to the maximum too
if newW > MaxOutputDimension || newH > MaxOutputDimension {
+3 -1
View File
@@ -29,7 +29,9 @@ var (
const MaxDimension = imageprocessor.MaxOutputDimension
// ValidateDimension checks one requested width or height; name ("width" or
// "height") appears in the error. 0 means "original size" and is valid.
// "height") appears in the error. 0 is valid: that side is then worked out from
// the source's proportions and the other requested side, rounded down but at
// least 1, or is the source's own side when the other side is 0 too.
func ValidateDimension(name string, value int) error {
if value < 0 {
return fmt.Errorf("%w: %s is negative", ErrInvalidSize, name)
@@ -14,10 +14,12 @@ import (
"net"
"net/http"
"net/http/httptest"
"net/netip"
"os"
"path/filepath"
"sync/atomic"
"testing"
"time"
"go.uber.org/fx"
"go.uber.org/fx/fxtest"
@@ -183,6 +185,57 @@ func startImageProxy(
return s, db.DB(), stateDir
}
// startImageProxyWithBlockedNetworks starts the components pixad's fx app
// builds, from a config with a fresh state directory and blockedNetworks as
// blocked_networks. Unlike startImageProxy it provides no upstream fetcher, so
// the handlers build the one pixad uses, which checks every address it
// connects to against the built-in blocked ranges and blocked_networks. It
// returns the server with its routes.
func startImageProxyWithBlockedNetworks(
t *testing.T, blockedNetworks []netip.Prefix,
) *Server {
t.Helper()
stateDir := t.TempDir()
cfg := &config.Config{
SigningKey: testSigningKey,
StateDir: stateDir,
DBURL: "file:" + filepath.Join(stateDir, "state.sqlite3"),
BlockedNetworks: blockedNetworks,
// A fetch that gets past the checks and finds nothing at the address
// fails within this time at the latest.
UpstreamFetchTimeout: 2 * time.Second,
// With no connection slots, the fetch would fail before connecting.
UpstreamConnections: config.DefaultUpstreamConnections,
DownstreamTimeout: config.DefaultDownstreamTimeout,
}
var (
h *handlers.Handlers
mw *middleware.Middleware
)
app := fxtest.New(t,
fx.Supply(cfg),
fx.Provide(
globals.New,
logger.New,
database.New,
healthcheck.New,
handlers.New,
middleware.New,
),
fx.Populate(&h, &mw),
)
app.RequireStart()
t.Cleanup(app.RequireStop)
s := &Server{config: cfg, mw: mw, h: h}
s.SetupRoutes()
return s
}
// getImage sends a GET for target to s and fails unless it answers 200.
func getImage(t *testing.T, s *Server, target string) *httptest.ResponseRecorder {
t.Helper()
+294
View File
@@ -3,15 +3,20 @@ package server
import (
"net/http"
"net/http/httptest"
"net/netip"
"net/url"
"regexp"
"slices"
"strconv"
"strings"
"sync"
"testing"
"time"
"sneak.berlin/go/pixa/internal/encurl"
"sneak.berlin/go/pixa/internal/imgcache"
"sneak.berlin/go/pixa/internal/session"
"sneak.berlin/go/pixa/internal/signature"
)
// percentSignPath is a source path holding a "%", which a URL writes %25.
@@ -308,6 +313,70 @@ func TestGeneratedURLFetchesTheSourceURLAsEntered(t *testing.T) {
}
}
// TestGeneratorMakesOneImageURLForTwoSpellingsOfASourceURL enters two
// spellings of one source URL on the generator page, requests the encrypted
// URL made for each, and checks that the upstream receives one request and
// that the second encrypted URL is served from the cache. One pair writes the
// host in upper and in lower case, which /v1/image/ lower-cases. The host is
// an IPv6 address, whose hex digits have letter case, because the fetcher
// would look a host name up in DNS. It has a port, as the fetcher refuses an
// IPv6 address written without one. The other pair is a source URL with no
// path and with the path "/", both of which fetch "/".
func TestGeneratorMakesOneImageURLForTwoSpellingsOfASourceURL(t *testing.T) {
t.Parallel()
tests := []struct {
name string
// sourceURLs are the two spellings, after "https://"
sourceURLs []string
// wantRequest is the path the upstream must receive
wantRequest string
}{
{"host in upper and in lower case",
[]string{"[2001:DB8::A]:8443/a.png", "[2001:db8::a]:8443/a.png"},
"/a.png"},
{"no path and the path written as a slash",
[]string{upstreamHost, upstreamHost + "/"}, "/"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
upstream := startRecordingUpstream(t)
s, _, _ := startImageProxy(t, upstream.Server)
var imageURLs []string
for _, sourceURL := range tt.sourceURLs {
rec := postGenerate(t, s, "https://"+sourceURL)
match := generatedURLPattern.FindStringSubmatch(rec.Body.String())
if rec.Code != http.StatusOK || match == nil {
t.Fatalf("POST /generate: %d with no encrypted URL; body %s",
rec.Code, rec.Body.String())
}
imageURLs = append(imageURLs, match[1])
}
getImage(t, s, imageURLs[0])
second := getImage(t, s, imageURLs[1])
if got := second.Header().Get("X-Pixa-Cache"); got != string(imgcache.CacheHit) {
t.Errorf("second X-Pixa-Cache = %q, want HIT", got)
}
got := upstream.requestTargets()
t.Logf("upstream received %q", got)
if !slices.Equal(got, []string{tt.wantRequest}) {
t.Errorf("upstream received %q, want [%q]", got, tt.wantRequest)
}
})
}
}
// TestGeneratorRefusesSourceURLItCannotFetchAsWritten submits source URLs the
// generator page must make no encrypted URL for, and checks that it answers
// 400 and shows why. pixa keeps the source path decoded and fetches it
@@ -369,3 +438,228 @@ func TestGeneratorRefusesSourceURLItCannotFetchAsWritten(t *testing.T) {
})
}
}
// ipv6SourcePath is the source path of the image URLs signedImageURL and
// encryptedImageURL make.
const ipv6SourcePath = "/ipv6.png"
// TestImageRoutesFetchIPv6SourceHostWithOrWithoutPort requests ipv6SourcePath
// on an IPv6 source host, written with and without a port, through a
// /v1/image/ URL with a signature and through an encrypted URL. The public
// address 2001:db8::a, a documentation address (RFC 3849) the fetcher accepts
// as public, answers 200, and the upstream receives the request for
// ipv6SourcePath. The private address fd00::1 answers 403, and the upstream
// receives no request: the test's fetcher connects every address to the test
// upstream, so only the fetcher's check of the source URL can refuse it.
func TestImageRoutesFetchIPv6SourceHostWithOrWithoutPort(t *testing.T) {
t.Parallel()
tests := []struct {
name string
sourceHost string
wantStatus int
// wantRequests are the request targets the upstream must receive
wantRequests []string
}{
{"public address without a port", "[2001:db8::a]",
http.StatusOK, []string{ipv6SourcePath}},
{"public address with a port", "[2001:db8::a]:8443",
http.StatusOK, []string{ipv6SourcePath}},
{"private address without a port", "[fd00::1]",
http.StatusForbidden, nil},
{"private address with a port", "[fd00::1]:8443",
http.StatusForbidden, nil},
}
for _, tt := range tests {
imageURLs := map[string]string{
"image URL with a signature": signedImageURL(t, tt.sourceHost),
"encrypted URL": encryptedImageURL(t, tt.sourceHost),
}
for route, imageURL := range imageURLs {
t.Run(tt.name+" "+route, func(t *testing.T) {
t.Parallel()
upstream := startRecordingUpstream(t)
s, _, _ := startImageProxy(t, upstream.Server)
rec := httptest.NewRecorder()
s.ServeHTTP(rec, httptest.NewRequestWithContext(
t.Context(), http.MethodGet, imageURL, nil))
t.Logf("GET %s: %d", imageURL, rec.Code)
if rec.Code != tt.wantStatus {
t.Errorf("status = %d, want %d; body %s",
rec.Code, tt.wantStatus, rec.Body.String())
}
got := upstream.requestTargets()
t.Logf("upstream received %q", got)
if !slices.Equal(got, tt.wantRequests) {
t.Errorf("upstream received %q, want %q", got, tt.wantRequests)
}
})
}
}
}
// signedImageURL returns the /v1/image/ URL of ipv6SourcePath on sourceHost at
// its original size as PNG, with a signature made with the test signing key,
// which a host not on the allowlist needs. The URL has no q or fit, so it is
// signed with the quality 85 and the fit cover.
func signedImageURL(t *testing.T, sourceHost string) string {
t.Helper()
path, sig, exp := signature.New(testSigningKey).GenerateSignedURL(
&signature.Request{
SourceHost: sourceHost,
SourcePath: ipv6SourcePath,
Format: string(imgcache.FormatPNG),
Quality: 85,
FitMode: string(imgcache.FitCover),
}, time.Hour)
return path + "?sig=" + sig + "&exp=" + strconv.FormatInt(exp, 10)
}
// encryptedImageURL returns the encrypted URL of ipv6SourcePath on sourceHost
// at its original size as PNG, made with the test signing key.
func encryptedImageURL(t *testing.T, sourceHost string) string {
t.Helper()
generator, err := encurl.NewGenerator(testSigningKey)
if err != nil {
t.Fatalf("encurl.NewGenerator: %v", err)
}
token, err := generator.Generate(&encurl.Payload{
SourceHost: sourceHost, SourcePath: ipv6SourcePath,
Format: imgcache.FormatPNG,
})
if err != nil {
t.Fatalf("encurl Generate: %v", err)
}
return "/v1/e/" + token + "/img.png"
}
// TestImageRoutesRefuseIPv6SourceHostHoldingInternalIPv4Address requests
// ipv6SourcePath on IPv6 source hosts that hold a loopback or private IPv4
// address in their last four bytes, in the IPv4-compatible form (::/96) or the
// NAT64 form (64:ff9b::/96), through a /v1/image/ URL with a signature and
// through an encrypted URL. Each answers 403, and the upstream receives no
// request: the test's fetcher connects every address to the test upstream, so
// only the fetcher's check of the source URL can refuse it.
func TestImageRoutesRefuseIPv6SourceHostHoldingInternalIPv4Address(t *testing.T) {
t.Parallel()
upstream := startRecordingUpstream(t)
s, _, _ := startImageProxy(t, upstream.Server)
sourceHosts := []string{
"[::127.0.0.1]", // IPv4-compatible form of a loopback address
"[::10.0.0.1]", // IPv4-compatible form of a private address
"[64:ff9b::7f00:1]", // NAT64 form of 127.0.0.1
}
for _, sourceHost := range sourceHosts {
imageURLs := []string{
signedImageURL(t, sourceHost),
encryptedImageURL(t, sourceHost),
}
for _, imageURL := range imageURLs {
rec := httptest.NewRecorder()
s.ServeHTTP(rec, httptest.NewRequestWithContext(
t.Context(), http.MethodGet, imageURL, nil))
t.Logf("GET %s (source host %s): %d", imageURL, sourceHost, rec.Code)
if rec.Code != http.StatusForbidden {
t.Errorf("source host %s: status = %d, want %d; body %s",
sourceHost, rec.Code, http.StatusForbidden, rec.Body.String())
}
}
}
if got := upstream.requestTargets(); len(got) != 0 {
t.Errorf("upstream received %q, want no request", got)
}
}
// TestImageRoutesCheckIPv4AddressInIPv6SourceHost requests ipv6SourcePath on
// source hosts that hold an IPv4 address, through a /v1/image/ URL with a
// signature and through an encrypted URL, with the fetcher pixad uses.
// [64:ff9b:1::a00:1], in the local-use NAT64 range (64:ff9b:1::/48, RFC 8215),
// answers 403. [::203.0.113.5] holds the public address 203.0.113.5 in the
// IPv4-compatible form (::/96) and answers 403 when blocked_networks lists
// 203.0.113.0/24 or ::/96. [::ffff:203.0.113.5] holds it in the IPv4-mapped
// form (::ffff:0:0/96) and answers 403 when blocked_networks lists
// 203.0.113.0/24. Go keeps the plain address 203.0.113.5 in the mapped form
// too, and an IPv6 range must not refuse it: with ::/0 listed it is not
// refused. Neither is [::203.0.113.5] with blocked_networks empty. Where a
// host is not refused, the fetcher tries to connect to it, finds nothing
// there, and the request fails with another status.
func TestImageRoutesCheckIPv4AddressInIPv6SourceHost(t *testing.T) {
t.Parallel()
const (
ipv4CompatibleHost = "[::203.0.113.5]"
ipv4MappedHost = "[::ffff:203.0.113.5]"
)
tests := []struct {
name string
sourceHost string
blockedNetworks []netip.Prefix
wantForbidden bool
}{
{"local-use NAT64 form of 10.0.0.1", "[64:ff9b:1::a00:1]", nil, true},
{"IPv4-compatible form of an address in blocked_networks",
ipv4CompatibleHost,
[]netip.Prefix{netip.MustParsePrefix("203.0.113.0/24")}, true},
{"IPv4-compatible form with ::/96 in blocked_networks",
ipv4CompatibleHost,
[]netip.Prefix{netip.MustParsePrefix("::/96")}, true},
{"IPv4-mapped form of an address in blocked_networks",
ipv4MappedHost,
[]netip.Prefix{netip.MustParsePrefix("203.0.113.0/24")}, true},
{"plain IPv4 address with ::/0 in blocked_networks",
"203.0.113.5",
[]netip.Prefix{netip.MustParsePrefix("::/0")}, false},
{"IPv4-compatible form of an address with blocked_networks empty",
ipv4CompatibleHost, nil, false},
}
for _, tt := range tests {
imageURLs := map[string]string{
"image URL with a signature": signedImageURL(t, tt.sourceHost),
"encrypted URL": encryptedImageURL(t, tt.sourceHost),
}
for route, imageURL := range imageURLs {
t.Run(tt.name+" "+route, func(t *testing.T) {
t.Parallel()
s := startImageProxyWithBlockedNetworks(t, tt.blockedNetworks)
rec := httptest.NewRecorder()
s.ServeHTTP(rec, httptest.NewRequestWithContext(
t.Context(), http.MethodGet, imageURL, nil))
t.Logf("GET %s: %d", imageURL, rec.Code)
forbidden := rec.Code == http.StatusForbidden
if tt.wantForbidden && !forbidden {
t.Errorf("status = %d, want %d; body %s",
rec.Code, http.StatusForbidden, rec.Body.String())
}
if !tt.wantForbidden && forbidden {
t.Errorf("status = %d, want any other status; body %s",
rec.Code, rec.Body.String())
}
})
}
}
}