Auteur SHA1 Message Date
sneak c3a176358f feat: blocked_networks config and extended SSRF ranges (closes #67)
check / check (push) Failing after 0s
Add a blocked_networks config key: a list of CIDRs parsed with net/netip,
added to (not replacing) the built-in SSRF blocklist. An invalid CIDR
aborts startup naming the key and the offending value.

Extend the built-in blocklist to CGNAT 100.64.0.0/10, IETF protocol
assignments 192.0.0.0/24, benchmark 198.18.0.0/15, and NAT64 64:ff9b::/96,
unmapping IPv4-mapped IPv6 so the IPv4 ranges are caught in both forms.
Enforcement stays in the dial-time re-resolution (dialSSRFSafe), which now
also consults the operator-supplied prefixes, so the DNS-rebinding window
remains closed.

Model: opus-4-8
2026-09-21 18:30:53 +00:00
sneak 6ce9d266de test: failing tests for blocked_networks config and extended SSRF ranges
Table-driven coverage for the CGNAT, IETF-protocol, benchmark, and NAT64
ranges (IPv4, IPv6, and IPv4-mapped forms), an operator-supplied
blocked_networks entry enforced by the dialer, and strict parsing that
aborts startup naming the key and the offending value.

Model: opus-4-8
2026-09-21 18:30:53 +00:00
clawbot 4f95cb6a37 docs: update TODO.md Workflow and Status for the next branching model (closes #106)
check / check (push) Failing after 0s
The Workflow section of TODO.md still told contributors to branch from main and merge there. It now describes the current model: one branch per issue cut from next, a PR based on next, an independent reviewer, a squash-merge into next by the manager, and only the owner merging next into main through the milestone PR. The Status paragraph no longer claims work is green on main.

Disclosure: only the wrong lines are touched; reflowing the whole file is left to #100.

Model: opus-4-8 (implementation, review); fable-5-1 (landing message)
2026-09-21 19:59:51 +02:00
clawbot 6f416eac31 test: cover redirect SSRF and semaphore release in httpfetcher (closes #78)
check / check (push) Failing after 0s
internal/httpfetcher had only helper-level tests. This adds tests of the full Fetch path, with no non-test code changed: a redirect to a private address is refused and never dialed while public redirects and a two-hop chain still work; the per-host semaphore is released on error, after a full read and after a partial read; an oversized body yields ErrResponseTooLarge; non-2xx and disallowed content types are rejected; the dialer blocks private, link-local and loopback targets.

What a reader would trip over: the upstream host in the tests is the TEST-NET-1 literal 192.0.2.10, which the private-IP check treats as public; a recording dialer routes it to the local test server and records every dial.

Disclosure: DNS rebinding is not simulated end to end (it would mean changing the global resolver under parallel race tests); the dial-time re-resolution is tested directly instead.

Model: opus-4-8 (implementation, review); fable-5-1 (landing message)
2026-09-21 19:59:26 +02:00
clawbot b95ef1eb69 fix: script/test conditional-verbose-rerun with -cover (closes #59)
check / check (push) Failing after 2s
script/test now runs the suite quietly first (with -race and -cover, 30s timeout) and re-runs it with -v only when that run fails, then exits non-zero. This is the pattern REPO_POLICIES.md mandates; before, every green run printed full per-test output.

What a reader would trip over: the whole compound command is passed as one string to run_with_cgo_deps, so it behaves the same on the host path and under the nix-shell fallback. -cover is on the first run only; the verbose rerun exists for diagnostics.

Disclosure: no test was written for the wrapper script itself; the failure path was exercised by hand by author and reviewer.
Disclosure: the nix-shell fallback is kept; moving tests into Docker belongs to #101 and #104.

Model: opus-4-8 (implementation, review); fable-5-1 (landing message)
2026-09-21 19:43:21 +02:00
clawbot a96eba8083 CSRF protection on the login and URL-generator forms (closes #93)
check / check (push) Failing after 1s
Adds CSRF protection to the two cookie-authenticated form posts, POST / (login) and POST /generate, using github.com/gorilla/csrf, the recorded default for this job.

The token key is derived from signing_key with its own HKDF salt, so it needs no new config and survives restarts. The token cookie is separate from the session cookie, which also covers login CSRF, where no session exists yet. Both templates carry the hidden token field.

What a reader would trip over: outside debug mode the library enforces its https Referer origin check, so the TLS-terminating proxy must preserve the Host and Referer headers from the browser or form posts are rejected.

Disclosure: one nolint:gosec on a test constant holding the library field name (G101 false positive).

Model: opus-4-8 (implementation, review); fable-5-1 (landing message)
2026-09-21 19:26:18 +02:00
22 fichiers modifiés avec 1246 ajouts et 275 suppressions
+9 -12
Voir le fichier
@@ -75,7 +75,7 @@ or partial matching is supported.
**Signed data format** (colon-separated):
```
HMAC-SHA256(secret, "host:path:query:width:height:format:expiration:quality:fit")
HMAC-SHA256(secret, "host:path:query:width:height:format:expiration")
```
Where:
@@ -87,20 +87,13 @@ Where:
- `height` — requested height in pixels, `0` for original
- `format` — output format (jpeg, png, webp, avif, gif, orig)
- `expiration` — Unix timestamp when signature expires
- `quality` — output quality 1-100; sign `85` (the default) when the URL
omits the `q` parameter
- `fit` — fit mode (cover, contain, fill, inside, outside); sign `cover`
(the default) when the URL omits the `fit` parameter
The `q` and `fit` query parameters are covered by the signature. A URL
signed for one quality or fit value will not verify when replayed with a
different value; the effective (post-default) value is what is signed.
**Example:** resize `https://cdn.example.com/photos/cat.jpg` to 800x600
WebP with expiration 1704067200, default quality and fit:
**Example:** resize
`https://cdn.example.com/photos/cat.jpg` to 800x600 WebP with
expiration 1704067200:
1. Build input:
`cdn.example.com:/photos/cat.jpg::800:600:webp:1704067200:85:cover`
`cdn.example.com:/photos/cat.jpg::800:600:webp:1704067200`
2. Compute HMAC-SHA256 with your secret key
3. Base64URL-encode the result
4. URL:
@@ -118,6 +111,10 @@ Configured via YAML file (`--config`). Key settings:
- `access_control_allow_origin` — CORS origin
- `allowlist_hosts` — list of allowed upstream hosts
- `blocked_networks` — list of CIDR ranges to refuse for SSRF protection,
added to the always-enforced built-in ranges (loopback, private,
link-local, CGNAT, benchmark, NAT64, and the like); an invalid CIDR
aborts startup
- `upstream_fetch_timeout` — timeout for origin requests
- `upstream_max_response_size` — max origin response size
- `downstream_timeout` — client response timeout
+24 -10
Voir le fichier
@@ -1,28 +1,44 @@
# Workflow
* branch (from `main`)
* branch per issue from `next`
* do the work in Next Step
* move Next Step to the top of Completed Steps
* move the top item of Future Steps into Next Step
* commit (`TODO.md` changes in the same commit as the work)
* merge to `main` if the branch is not protected, otherwise open a PR
* open a PR based on `next`
* an independent reviewer who did not write the change gates it
* the manager squash-merges the PR into `next` once review passes
* `next` stays green and mergeable to `main` at any time; only the owner
merges `next` into `main`, via the single milestone PR
* push
# Status
pre-1.0. No git tags exist. Recent work extracted the internal/magic,
pre-1.0. No git tags exist. The `1.0.0` milestone is in progress; work
lands on `next`, and `main` receives only the milestone PR that the
owner merges. `next` is at the canonical `golangci-lint` v2.12.2 config
and is green. Recent work extracted the internal/magic,
internal/allowlist, internal/httpfetcher, and internal/signature
packages. The gosec findings from the 2026-07-06 survey are resolved
and `make check` is green on main. The disk cache is now size-bounded
with LRU eviction (`cache_max_bytes`), closing the unbounded disk
growth DoS vector.
packages. The gosec findings from the 2026-07-06 survey are resolved.
The disk cache is now size-bounded with LRU eviction
(`cache_max_bytes`), closing the unbounded disk growth DoS vector.
# Next Step
P1: implement blocked networks configuration to extend SSRF protection
P1: rate limit global concurrent upstream fetches to prevent resource
exhaustion
# Completed Steps
- 2026-09-21 blocked networks configuration extending SSRF protection: a
`blocked_networks` config key taking a list of CIDRs (parsed with
`net/netip`, an invalid entry aborts startup naming the key and value),
added to the built-in blocklist rather than replacing it; the built-in
ranges extended to CGNAT `100.64.0.0/10`, IETF protocol assignments
`192.0.0.0/24`, benchmark `198.18.0.0/15`, and NAT64 `64:ff9b::/96`
(IPv4-mapped forms covered); enforcement stays in the dial-time
re-resolution so the DNS-rebinding window remains closed; documented in
`README.md` and `config.example.yml`.
- 2026-08-07 update golangci-lint to v2.12.2 with the canonical
`.golangci.yml` (v2 schema, `default: all` minus six disabled
linters, `lll` 88, tests included): bumped the pinned
@@ -116,8 +132,6 @@ P1: implement blocked networks configuration to extend SSRF protection
# Future Steps
- P1: rate limit global concurrent upstream fetches to prevent
resource exhaustion
- P1: strip EXIF and other metadata from processed images (privacy)
- P2: security
- referer blacklist
+9
Voir le fichier
@@ -22,6 +22,15 @@ allowlist_hosts:
- github.com
- user-images.githubusercontent.com
# Additional CIDR ranges to refuse when fetching upstream, extending the
# SSRF protection. These are added to the always-enforced built-in ranges
# (loopback, RFC 1918 private, link-local, CGNAT, benchmark, NAT64, and
# similar), never replacing them. Each entry must be a valid CIDR in IPv4
# or IPv6 form; an invalid entry aborts startup.
# blocked_networks:
# - 100.64.0.0/10
# - 2001:db8::/32
# Allow HTTP upstream (only for testing, always use HTTPS in production)
allow_http: false
+1
Voir le fichier
@@ -11,6 +11,7 @@ require (
github.com/getsentry/sentry-go v0.40.0
github.com/go-chi/chi/v5 v5.2.3
github.com/go-chi/cors v1.2.2
github.com/gorilla/csrf v1.7.3
github.com/gorilla/securecookie v1.1.2
github.com/prometheus/client_golang v1.23.2
github.com/slok/go-http-metrics v0.13.0
+2
Voir le fichier
@@ -175,6 +175,8 @@ github.com/googleapis/enterprise-certificate-proxy v0.3.6 h1:GW/XbdyBFQ8Qe+YAmFU
github.com/googleapis/enterprise-certificate-proxy v0.3.6/go.mod h1:MkHOF77EYAE7qfSuSS9PU6g4Nt4e11cnsDUowfwewLA=
github.com/googleapis/gax-go/v2 v2.14.2 h1:eBLnkZ9635krYIPD+ag1USrOAI0Nr0QYF3+/3GqO0k0=
github.com/googleapis/gax-go/v2 v2.14.2/go.mod h1:ON64QhlJkhVtSqp4v1uaK92VyZ2gmvDQsweuyLV+8+w=
github.com/gorilla/csrf v1.7.3 h1:BHWt6FTLZAb2HtWT5KDBf6qgpZzvtbp9QWDRKZMXJC0=
github.com/gorilla/csrf v1.7.3/go.mod h1:F1Fj3KG23WYHE6gozCmBAezKookxbIvUJT+121wTuLk=
github.com/gorilla/securecookie v1.1.2 h1:YCIWL56dvtr73r6715mJs5ZvhtnY73hBvEF8kXD8ePA=
github.com/gorilla/securecookie v1.1.2/go.mod h1:NfCASbcHqRSY+3a8tlWJwsQap2VX5pwzwo4h3eOamfo=
github.com/grpc-ecosystem/grpc-gateway/v2 v2.26.3 h1:5ZPtiqj0JL5oKWmcsq4VMaAW5ukBEgSGXEN89zeH1Jo=
+90
Voir le fichier
@@ -0,0 +1,90 @@
package config
import (
"testing"
)
// TestBlockedNetworksParsed loads a valid blocked_networks list and checks
// each CIDR is parsed into the resolved prefixes in order.
func TestBlockedNetworksParsed(t *testing.T) {
t.Parallel()
yamlContent := signingKeyLine + `blocked_networks:
- 203.0.113.0/24
- 2001:db8::/32
`
c, err := configFromYAML(t, yamlContent)
if err != nil {
t.Fatalf("valid blocked_networks should load, got error: %v", err)
}
want := []string{"203.0.113.0/24", "2001:db8::/32"}
if len(c.BlockedNetworks) != len(want) {
t.Fatalf("BlockedNetworks = %v, want %d entries", c.BlockedNetworks, len(want))
}
for i, w := range want {
if got := c.BlockedNetworks[i].String(); got != w {
t.Errorf("BlockedNetworks[%d] = %q, want %q", i, got, w)
}
}
}
// TestBlockedNetworksOmittedIsEmpty confirms an omitted key leaves the
// operator list empty; the built-in defaults still apply in the fetcher.
func TestBlockedNetworksOmittedIsEmpty(t *testing.T) {
t.Parallel()
c, err := configFromYAML(t, signingKeyLine)
if err != nil {
t.Fatalf("minimal config should be valid, got error: %v", err)
}
if len(c.BlockedNetworks) != 0 {
t.Errorf("BlockedNetworks = %v, want empty", c.BlockedNetworks)
}
}
// TestBlockedNetworksInvalidAbortsStartup checks that malformed values abort
// startup with an error naming the key and the offending value.
func TestBlockedNetworksInvalidAbortsStartup(t *testing.T) {
t.Parallel()
runAbortCases(t, []abortCase{
{
name: "not-a-cidr",
yaml: signingKeyLine + `blocked_networks:
- not-a-cidr
`,
wantErrSubstrings: []string{keyBlockedNetworks, "not-a-cidr"},
},
{
name: "bare-address-without-prefix",
yaml: signingKeyLine + `blocked_networks:
- 10.0.0.1
`,
wantErrSubstrings: []string{keyBlockedNetworks, "10.0.0.1"},
},
{
name: "empty-entry",
yaml: signingKeyLine + `blocked_networks:
- ""
`,
wantErrSubstrings: []string{keyBlockedNetworks},
},
{
name: "non-string-entry",
yaml: signingKeyLine + `blocked_networks:
- 42
`,
wantErrSubstrings: []string{keyBlockedNetworks},
},
{
name: "null-value",
yaml: signingKeyLine + `blocked_networks:
`,
wantErrSubstrings: []string{keyBlockedNetworks, nullValueText},
},
})
}
+100 -2
Voir le fichier
@@ -6,6 +6,7 @@ import (
"fmt"
"log/slog"
"math"
"net/netip"
"net/url"
"os"
"path/filepath"
@@ -42,6 +43,7 @@ const (
keyAllowHTTP = "allow_http"
keyUpstreamConnectionsPerHost = "upstream_connections_per_host"
keyCacheMaxBytes = "cache_max_bytes"
keyBlockedNetworks = "blocked_networks"
)
// Static validation errors. Each use site attaches the offending key
@@ -54,6 +56,7 @@ var (
errNotAnInteger = errors.New("not an integer")
errNotABoolean = errors.New("not a boolean")
errNotAStringList = errors.New("not a list of strings")
errNotAValidCIDR = errors.New("not a valid CIDR network")
errNotAMetricsMap = errors.New("not a map of metrics settings")
errEmptyListEntry = errors.New("list contains an empty entry")
errEmptyEntry = errors.New("contains an empty entry")
@@ -100,6 +103,11 @@ type Config struct {
AllowHTTP bool // Allow non-TLS upstream (testing only)
UpstreamConnectionsPerHost int // Max concurrent connections per upstream host
// BlockedNetworks are operator-supplied CIDR ranges to refuse in
// addition to the built-in SSRF blocklist. Enforced by the upstream
// fetcher's dialer; the built-in ranges always apply.
BlockedNetworks []netip.Prefix
// CacheMaxBytes is the disk cache size limit in bytes. Zero
// disables the disk cache entirely. When cache_max_bytes is
// omitted from the configuration, this holds the computed default
@@ -168,6 +176,11 @@ func newFromSmartConfig(sc *smartconfig.Config) (*Config, error) {
}
}
blockedNetworks, err := getBlockedNetworks(sc)
if err != nil {
return nil, err
}
loader := &strictLoader{sc: sc}
c := &Config{
@@ -184,6 +197,7 @@ func newFromSmartConfig(sc *smartconfig.Config) (*Config, error) {
UpstreamConnectionsPerHost: loader.intVal(
keyUpstreamConnectionsPerHost, DefaultUpstreamConnectionsPerHost),
CacheMaxBytes: loader.int64Val(keyCacheMaxBytes, 0),
BlockedNetworks: blockedNetworks,
}
// The computed default for cache_max_bytes needs a validated
@@ -215,7 +229,7 @@ func newFromSmartConfig(sc *smartconfig.Config) (*Config, error) {
return nil, loader.err
}
err := c.validate()
err = c.validate()
if err != nil {
return nil, err
}
@@ -299,7 +313,7 @@ func isKnownConfigKey(key string) bool {
switch key {
case keyDebug, keyMaintenanceMode, keyPort, keyStateDir, keySentryDSN,
keyDBURL, keyMetrics, keySigningKey, keyAllowlistHosts, keyAllowHTTP,
keyUpstreamConnectionsPerHost, keyCacheMaxBytes, "env":
keyUpstreamConnectionsPerHost, keyCacheMaxBytes, keyBlockedNetworks, "env":
return true
}
@@ -778,3 +792,87 @@ func getStringSlice(sc *smartconfig.Config) []string {
return nil
}
// getBlockedNetworks parses the blocked_networks value into CIDR prefixes,
// or returns nil if the key is omitted. It accepts a YAML list of strings
// or a comma-separated string. An explicitly null value, a wrong type, an
// empty entry, a non-string entry, or an unparseable CIDR aborts startup
// naming the key and the offending value; a default (the built-in
// blocklist alone) applies only to an omitted key.
func getBlockedNetworks(sc *smartconfig.Config) ([]netip.Prefix, error) {
if sc == nil {
return nil, nil
}
raw, ok := sc.Get(keyBlockedNetworks)
if !ok {
return nil, nil
}
if raw == nil {
return nil, errNullConfigValue(keyBlockedNetworks)
}
entries, err := blockedNetworkEntries(raw)
if err != nil {
return nil, err
}
prefixes := make([]netip.Prefix, 0, len(entries))
for _, entry := range entries {
prefix, err := netip.ParsePrefix(entry)
if err != nil {
return nil, fmt.Errorf("config key %q: value %q is %w",
keyBlockedNetworks, entry, errNotAValidCIDR)
}
prefixes = append(prefixes, prefix)
}
return prefixes, nil
}
// blockedNetworkEntries extracts the raw blocked_networks entries as
// trimmed, non-empty strings, from either a YAML list of strings or a
// comma-separated string. Any other shape is a configuration error.
func blockedNetworkEntries(raw any) ([]string, error) {
switch val := raw.(type) {
case []any:
entries := make([]string, 0, len(val))
for _, item := range val {
str, ok := item.(string)
if !ok {
return nil, fmt.Errorf("config key %q: list entry %v (%T) is %w",
keyBlockedNetworks, item, item, errNotAString)
}
if strings.TrimSpace(str) == "" {
return nil, fmt.Errorf("config key %q: %w",
keyBlockedNetworks, errEmptyListEntry)
}
entries = append(entries, strings.TrimSpace(str))
}
return entries, nil
case string:
entries := make([]string, 0)
for part := range strings.SplitSeq(val, ",") {
trimmed := strings.TrimSpace(part)
if trimmed == "" {
return nil, fmt.Errorf("config key %q: value %q %w",
keyBlockedNetworks, val, errEmptyEntry)
}
entries = append(entries, trimmed)
}
return entries, nil
default:
return nil, fmt.Errorf("config key %q: value %v (%T) is %w",
keyBlockedNetworks, raw, raw, errNotAStringList)
}
}
+23 -13
Voir le fichier
@@ -2,6 +2,7 @@ package handlers
import (
"crypto/subtle"
"html/template"
"net/http"
"net/url"
"strconv"
@@ -23,13 +24,13 @@ func (s *Handlers) HandleRoot() http.HandlerFunc {
// Check if authenticated
if s.sessMgr.IsAuthenticated(r) {
s.renderGenerator(w, nil)
s.renderGenerator(w, r, nil)
return
}
// Show login page
s.renderLogin(w, "")
s.renderLogin(w, r, "")
}
}
@@ -37,7 +38,7 @@ func (s *Handlers) HandleRoot() http.HandlerFunc {
func (s *Handlers) handleLoginPost(w http.ResponseWriter, r *http.Request) {
err := r.ParseForm()
if err != nil {
s.renderLogin(w, "Invalid form data")
s.renderLogin(w, r, "Invalid form data")
return
}
@@ -47,7 +48,7 @@ func (s *Handlers) handleLoginPost(w http.ResponseWriter, r *http.Request) {
// Constant-time comparison to prevent timing attacks
if subtle.ConstantTimeCompare([]byte(submittedKey), []byte(s.config.SigningKey)) != 1 {
s.log.Warn("failed login attempt", "remote_addr", r.RemoteAddr)
s.renderLogin(w, "Invalid signing key")
s.renderLogin(w, r, "Invalid signing key")
return
}
@@ -56,7 +57,7 @@ func (s *Handlers) handleLoginPost(w http.ResponseWriter, r *http.Request) {
err = s.sessMgr.CreateSession(w)
if err != nil {
s.log.Error("failed to create session", "error", err)
s.renderLogin(w, "Failed to create session")
s.renderLogin(w, r, "Failed to create session")
return
}
@@ -87,7 +88,7 @@ func (s *Handlers) HandleGenerateURL() http.HandlerFunc {
err := r.ParseForm()
if err != nil {
s.renderGenerator(w, &generatorData{Error: "Invalid form data"})
s.renderGenerator(w, r, &generatorData{Error: "Invalid form data"})
return
}
@@ -97,7 +98,7 @@ func (s *Handlers) HandleGenerateURL() http.HandlerFunc {
// Validate source URL
parsed, err := url.Parse(sourceURL)
if err != nil || parsed.Host == "" {
s.renderGeneratorWithForm(w, "Invalid source URL", r.Form)
s.renderGeneratorWithForm(w, r, "Invalid source URL", r.Form)
return
}
@@ -108,7 +109,7 @@ func (s *Handlers) HandleGenerateURL() http.HandlerFunc {
token, err := s.encGen.Generate(payload)
if err != nil {
s.log.Error("failed to generate encrypted URL", "error", err)
s.renderGeneratorWithForm(w, "Failed to generate URL", r.Form)
s.renderGeneratorWithForm(w, r, "Failed to generate URL", r.Form)
return
}
@@ -121,7 +122,7 @@ func (s *Handlers) HandleGenerateURL() http.HandlerFunc {
expiresAtStr = expiresAt.Format(time.RFC3339)
}
s.renderGenerator(w, &generatorData{
s.renderGenerator(w, r, &generatorData{
GeneratedURL: generatedURL,
ExpiresAt: expiresAtStr,
FormURL: sourceURL,
@@ -186,15 +187,20 @@ type generatorData struct {
FormQuality string
FormFit string
FormTTL string
CSRFField template.HTML
}
func (s *Handlers) renderLogin(w http.ResponseWriter, errorMsg string) {
func (s *Handlers) renderLogin(
w http.ResponseWriter, r *http.Request, errorMsg string,
) {
w.Header().Set("Content-Type", "text/html; charset=utf-8")
data := struct {
Error string
CSRFField template.HTML
}{
Error: errorMsg,
CSRFField: csrfField(r),
}
err := templates.Render(w, "login.html", data)
@@ -204,13 +210,17 @@ func (s *Handlers) renderLogin(w http.ResponseWriter, errorMsg string) {
}
}
func (s *Handlers) renderGenerator(w http.ResponseWriter, data *generatorData) {
func (s *Handlers) renderGenerator(
w http.ResponseWriter, r *http.Request, data *generatorData,
) {
w.Header().Set("Content-Type", "text/html; charset=utf-8")
if data == nil {
data = &generatorData{}
}
data.CSRFField = csrfField(r)
err := templates.Render(w, "generator.html", data)
if err != nil {
s.log.Error("failed to render generator template", "error", err)
@@ -219,9 +229,9 @@ func (s *Handlers) renderGenerator(w http.ResponseWriter, data *generatorData) {
}
func (s *Handlers) renderGeneratorWithForm(
w http.ResponseWriter, errorMsg string, form url.Values,
w http.ResponseWriter, r *http.Request, errorMsg string, form url.Values,
) {
s.renderGenerator(w, &generatorData{
s.renderGenerator(w, r, &generatorData{
Error: errorMsg,
FormURL: form.Get("url"),
FormWidth: form.Get("width"),
+273
Voir le fichier
@@ -0,0 +1,273 @@
package handlers
import (
"context"
"log/slog"
"net/http"
"net/http/httptest"
"net/url"
"regexp"
"strings"
"testing"
"github.com/go-chi/chi/v5"
"sneak.berlin/go/pixa/internal/config"
"sneak.berlin/go/pixa/internal/encurl"
"sneak.berlin/go/pixa/internal/session"
)
// testSigningKey is a throwaway signing key for the CSRF flow tests. It
// seeds the session manager, the encrypted-URL generator, and the CSRF
// token key, exactly as the real signing key does in production.
const testSigningKey = "test-signing-key-0123456789abcdef"
// Form field names used in the CSRF flow tests.
const (
loginKeyField = "key"
// gorilla/csrf's default form field name, not a credential.
csrfTokenField = "gorilla.csrf.Token" //nolint:gosec // G101 false positive
)
// csrfFieldPattern extracts the token rendered by csrf.TemplateField into
// the form. The field name is gorilla/csrf's default.
var csrfFieldPattern = regexp.MustCompile(
`name="gorilla\.csrf\.Token" value="([^"]+)"`)
// newCSRFTestRouter builds a router that mirrors the production wiring for
// the CSRF-protected UI routes (see server.SetupRoutes): the login and
// generator forms and their POST targets sit behind the real CSRF
// middleware. Requests are marked plaintext (Debug: true) so the flow runs
// over httptest's http transport without an https Referer.
func newCSRFTestRouter(t *testing.T) (*Handlers, http.Handler) {
t.Helper()
cfg := &config.Config{SigningKey: testSigningKey, Debug: true}
sessMgr, err := session.NewManager(testSigningKey)
if err != nil {
t.Fatalf("session.NewManager() error = %v", err)
}
encGen, err := encurl.NewGenerator(testSigningKey)
if err != nil {
t.Fatalf("encurl.NewGenerator() error = %v", err)
}
protect, err := newCSRFProtect(testSigningKey, cfg.Debug)
if err != nil {
t.Fatalf("newCSRFProtect() error = %v", err)
}
h := &Handlers{
log: slog.New(slog.DiscardHandler),
config: cfg,
sessMgr: sessMgr,
encGen: encGen,
csrfProtect: protect,
}
r := chi.NewRouter()
r.Group(func(r chi.Router) {
r.Use(h.CSRF())
r.Get("/", h.HandleRoot())
r.Post("/", h.HandleRoot())
r.Post("/generate", h.HandleGenerateURL())
})
return h, r
}
// csrfCredentials performs a GET that renders a form and returns the CSRF
// cookies the middleware set and the token embedded in the form. Passing
// the authenticated session cookie renders the generator form instead of
// the login form.
func csrfCredentials(
t *testing.T, srv http.Handler, reqCookies []*http.Cookie,
) ([]*http.Cookie, string) {
t.Helper()
req := httptest.NewRequestWithContext(
context.Background(), http.MethodGet, "/", nil)
for _, c := range reqCookies {
req.AddCookie(c)
}
rec := httptest.NewRecorder()
srv.ServeHTTP(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("GET / status = %d, want %d", rec.Code, http.StatusOK)
}
match := csrfFieldPattern.FindStringSubmatch(rec.Body.String())
if match == nil {
t.Fatalf("no CSRF token field found in rendered form")
}
return rec.Result().Cookies(), match[1]
}
// postForm submits form values with the given cookies and returns the
// recorder.
func postForm(
srv http.Handler, path string,
cookies []*http.Cookie, form url.Values,
) *httptest.ResponseRecorder {
req := httptest.NewRequestWithContext(
context.Background(), http.MethodPost, path,
strings.NewReader(form.Encode()))
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
for _, c := range cookies {
req.AddCookie(c)
}
rec := httptest.NewRecorder()
srv.ServeHTTP(rec, req)
return rec
}
// TestLoginPostRejectedWithoutToken verifies that POST / with no CSRF token
// is rejected. This is login CSRF: no session cookie exists yet, so the
// protection must rest on a token bound to a pre-session cookie.
func TestLoginPostRejectedWithoutToken(t *testing.T) {
t.Parallel()
_, srv := newCSRFTestRouter(t)
rec := postForm(srv, "/", nil, url.Values{loginKeyField: {testSigningKey}})
if rec.Code != http.StatusForbidden {
t.Errorf("POST / without token status = %d, want %d",
rec.Code, http.StatusForbidden)
}
}
// TestLoginPostRejectedWithForeignToken verifies that a token that does not
// match the request's CSRF cookie is rejected: a token minted for one
// cookie cannot authorize a request carrying a different cookie.
func TestLoginPostRejectedWithForeignToken(t *testing.T) {
t.Parallel()
_, srv := newCSRFTestRouter(t)
cookiesA, _ := csrfCredentials(t, srv, nil)
_, tokenB := csrfCredentials(t, srv, nil)
rec := postForm(srv, "/", cookiesA, url.Values{
loginKeyField: {testSigningKey},
csrfTokenField: {tokenB},
})
if rec.Code != http.StatusForbidden {
t.Errorf("POST / with foreign token status = %d, want %d",
rec.Code, http.StatusForbidden)
}
}
// TestLoginPostAcceptedWithValidToken verifies that POST / with a matching
// cookie and token succeeds: the login is processed and a session is
// established (303 redirect).
func TestLoginPostAcceptedWithValidToken(t *testing.T) {
t.Parallel()
_, srv := newCSRFTestRouter(t)
cookies, token := csrfCredentials(t, srv, nil)
rec := postForm(srv, "/", cookies, url.Values{
loginKeyField: {testSigningKey},
csrfTokenField: {token},
})
if rec.Code != http.StatusSeeOther {
t.Fatalf("POST / with valid token status = %d, want %d",
rec.Code, http.StatusSeeOther)
}
var authed bool
for _, c := range rec.Result().Cookies() {
if c.Name == session.CookieName && c.Value != "" {
authed = true
}
}
if !authed {
t.Error("valid login did not set a session cookie")
}
}
// TestGeneratePostRejectedWithoutToken verifies that POST /generate is
// rejected without a CSRF token even when the request carries a valid
// authenticated session. The session cookie is not sufficient; the policy
// requires a CSRF token on this cookie-authenticated form.
func TestGeneratePostRejectedWithoutToken(t *testing.T) {
t.Parallel()
h, srv := newCSRFTestRouter(t)
sessionCookie := newSessionCookie(t, h)
rec := postForm(srv, "/generate",
[]*http.Cookie{sessionCookie},
url.Values{"url": {"https://example.com/a.jpg"}})
if rec.Code != http.StatusForbidden {
t.Errorf("POST /generate without token status = %d, want %d",
rec.Code, http.StatusForbidden)
}
}
// TestGeneratePostAcceptedWithValidToken verifies that POST /generate
// succeeds with a valid session and a matching CSRF cookie and token.
func TestGeneratePostAcceptedWithValidToken(t *testing.T) {
t.Parallel()
h, srv := newCSRFTestRouter(t)
sessionCookie := newSessionCookie(t, h)
cookies, token := csrfCredentials(t, srv, []*http.Cookie{sessionCookie})
cookies = append(cookies, sessionCookie)
rec := postForm(srv, "/generate", cookies, url.Values{
"url": {"https://example.com/a.jpg"},
"format": {"jpeg"},
csrfTokenField: {token},
})
if rec.Code != http.StatusOK {
t.Fatalf("POST /generate with valid token status = %d, want %d",
rec.Code, http.StatusOK)
}
if !strings.Contains(rec.Body.String(), "/v1/e/") {
t.Error("generator response did not contain a generated URL")
}
}
// newSessionCookie creates an authenticated session cookie via the
// handler's session manager.
func newSessionCookie(t *testing.T, h *Handlers) *http.Cookie {
t.Helper()
rec := httptest.NewRecorder()
err := h.sessMgr.CreateSession(rec)
if err != nil {
t.Fatalf("CreateSession() error = %v", err)
}
for _, c := range rec.Result().Cookies() {
if c.Name == session.CookieName {
return c
}
}
t.Fatalf("session manager did not set a %q cookie", session.CookieName)
return nil
}
+65
Voir le fichier
@@ -0,0 +1,65 @@
package handlers
import (
"html/template"
"net/http"
"github.com/gorilla/csrf"
"sneak.berlin/go/pixa/internal/seal"
)
// csrfKeySalt provides domain separation for the CSRF authentication key,
// derived from the signing key so tokens survive restarts without extra
// configuration and never reuse the session or encrypted-URL key material.
const csrfKeySalt = "pixa-csrf-v1"
// newCSRFProtect builds the CSRF-protection middleware for the
// state-mutating HTML form routes. The token lives in its own cookie,
// independent of the session cookie, so it also protects the login POST
// where no session exists yet (login CSRF).
//
// When plaintext is true (local HTTP development), requests are marked
// plaintext so the library neither demands an https Referer nor sets a
// Secure cookie the browser would withhold over http. In production the
// service runs behind a TLS-terminating proxy, so plaintext is false and
// the library enforces its https Referer origin check.
func newCSRFProtect(
signingKey string, plaintext bool,
) (func(http.Handler) http.Handler, error) {
key, err := seal.DeriveKey([]byte(signingKey), csrfKeySalt)
if err != nil {
return nil, err
}
protect := csrf.Protect(
key[:],
csrf.Path("/"),
csrf.Secure(!plaintext),
csrf.SameSite(csrf.SameSiteStrictMode),
)
if !plaintext {
return protect, nil
}
return func(next http.Handler) http.Handler {
protected := protect(next)
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
protected.ServeHTTP(w, csrf.PlaintextHTTPRequest(r))
})
}, nil
}
// CSRF returns the CSRF-protection middleware for the login and generator
// form routes.
func (s *Handlers) CSRF() func(http.Handler) http.Handler {
return s.csrfProtect
}
// csrfField returns the hidden form input carrying the CSRF token for the
// given request, to be embedded verbatim in a rendered form.
func csrfField(r *http.Request) template.HTML {
return csrf.TemplateField(r)
}
+9
Voir le fichier
@@ -39,15 +39,22 @@ type Handlers struct {
imgCache *imgcache.Cache
sessMgr *session.Manager
encGen *encurl.Generator
csrfProtect func(http.Handler) http.Handler
}
// New creates a new Handlers instance.
func New(lc fx.Lifecycle, params Params) (*Handlers, error) {
csrfProtect, err := newCSRFProtect(params.Config.SigningKey, params.Config.Debug)
if err != nil {
return nil, err
}
s := &Handlers{
log: params.Logger.Get(),
hc: params.Healthcheck,
db: params.Database,
config: params.Config,
csrfProtect: csrfProtect,
}
lc.Append(fx.Hook{
@@ -104,6 +111,8 @@ func (s *Handlers) initImageService() error {
fetcherCfg.MaxConnectionsPerHost = s.config.UpstreamConnectionsPerHost
}
fetcherCfg.BlockedNetworks = s.config.BlockedNetworks
// Create the service
svc, err := imgcache.NewService(&imgcache.ServiceConfig{
Cache: cache,
+111
Voir le fichier
@@ -0,0 +1,111 @@
package httpfetcher
import (
"context"
"errors"
"net"
"net/http"
"net/netip"
"testing"
)
// TestIsPrivateIPBlocksSpecialRanges covers the internal and special-use
// ranges added to the built-in blocklist, in IPv4, IPv6, and IPv4-mapped
// forms, alongside public controls that must stay reachable.
func TestIsPrivateIPBlocksSpecialRanges(t *testing.T) {
t.Parallel()
tests := []struct {
name string
ip string
want bool
}{
{"cgnat-low", "100.64.0.1", true},
{"cgnat-high", "100.127.255.254", true},
{"ietf-protocol", "192.0.0.1", true},
{"benchmark-low", "198.18.0.1", true},
{"benchmark-high", "198.19.255.254", true},
{"nat64", "64:ff9b::1", true},
{"nat64-embeds-private", "64:ff9b::a00:1", true}, // maps 10.0.0.1
{"ipv4-mapped-private", "::ffff:10.0.0.1", true},
{"cloud-metadata", "169.254.169.254", true},
{"public-v4", "8.8.8.8", false},
{"test-net-1-public", testPublicHost, false}, // TEST-NET-1, stays public
{"public-v6", "2001:4860:4860::8888", false},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
ip := net.ParseIP(tc.ip)
if ip == nil {
t.Fatalf("failed to parse IP %q", tc.ip)
}
got := isPrivateIP(ip)
if got != tc.want {
t.Errorf("isPrivateIP(%q) = %v, want %v", tc.ip, got, tc.want)
}
})
}
}
// transportOf returns the *http.Transport backing a fetcher, so a test can
// exercise the SSRF-safe dialer New installed with the operator blocklist.
func transportOf(t *testing.T, f *HTTPFetcher) *http.Transport {
t.Helper()
transport, ok := f.client.Transport.(*http.Transport)
if !ok {
t.Fatalf("transport is %T, want *http.Transport", f.client.Transport)
}
return transport
}
// TestDialerEnforcesBlockedNetworks proves an operator-supplied
// blocked_networks entry is enforced by the dialer, in addition to the
// built-in ranges, while an address outside both stays dialable.
func TestDialerEnforcesBlockedNetworks(t *testing.T) {
t.Parallel()
cfg := DefaultConfig()
// TEST-NET-2 (198.51.100.0/24) is public to the built-in check, so
// blocking it can only come from the operator-supplied list.
cfg.BlockedNetworks = []netip.Prefix{netip.MustParsePrefix("198.51.100.0/24")}
transport := transportOf(t, New(cfg))
blocked := []string{
"198.51.100.5:80", // operator-supplied range
"10.0.0.5:80", // built-in RFC 1918, still enforced
"100.64.0.1:80", // built-in CGNAT range
}
for _, addr := range blocked {
t.Run("blocked/"+addr, func(t *testing.T) {
t.Parallel()
_, err := transport.DialContext(context.Background(), "tcp", addr)
if !errors.Is(err, ErrSSRFBlocked) {
t.Errorf("DialContext(%q) = %v, want ErrSSRFBlocked", addr, err)
}
})
}
t.Run("public-not-blocked", func(t *testing.T) {
t.Parallel()
// A cancelled context makes the dial fail without touching the
// network; the point is only that a public literal outside every
// blocked range is not SSRF-blocked.
ctx, cancel := context.WithCancel(context.Background())
cancel()
_, err := transport.DialContext(ctx, "tcp", testPublicHost+":80")
if errors.Is(err, ErrSSRFBlocked) {
t.Errorf("public target SSRF-blocked with operator list set: %v", err)
}
})
}
+421
Voir le fichier
@@ -0,0 +1,421 @@
package httpfetcher
import (
"context"
"errors"
"fmt"
"io"
"net"
"net/http"
"net/http/httptest"
"slices"
"strings"
"sync"
"testing"
"time"
)
// testPublicHost is a TEST-NET-1 (RFC 5737) literal. isPrivateIP treats it as
// public, so validateURL and the redirect check accept it with no DNS lookup,
// while the recording dialer routes it to the local httptest server. The
// address is reserved for documentation and is never routed on the network.
const testPublicHost = "192.0.2.10"
// imagePayload is the body served by the fake upstream's image route.
const imagePayload = "fake-jpeg-bytes"
// errUnexpectedDial reports a dial to any host other than testPublicHost, which
// would mean SSRF protection let a forbidden target reach the transport.
var errUnexpectedDial = errors.New("unexpected dial target")
// upstreamURL builds a fetch URL on the fake public host for the given path.
func upstreamURL(path string) string {
return "http://" + testPublicHost + path
}
// recordingDialer records every address the transport asks it to dial and
// routes connections for testPublicHost to a real local server, so the SSRF
// checks run against a public-looking host while bytes go to httptest.
type recordingDialer struct {
target string
mu sync.Mutex
dialed []string
}
func (d *recordingDialer) dialContext(
ctx context.Context,
network, addr string,
) (net.Conn, error) {
d.mu.Lock()
d.dialed = append(d.dialed, addr)
d.mu.Unlock()
host, _, err := net.SplitHostPort(addr)
if err != nil {
return nil, err
}
if host != testPublicHost {
return nil, fmt.Errorf("%w: %s", errUnexpectedDial, addr)
}
var dialer net.Dialer
return dialer.DialContext(ctx, network, d.target)
}
// dialedAddrs returns a copy of the addresses the dialer was asked to reach.
func (d *recordingDialer) dialedAddrs() []string {
d.mu.Lock()
defer d.mu.Unlock()
return slices.Clone(d.dialed)
}
// startUpstream launches a fake upstream with the routes the fetch tests
// exercise and stops it when the test finishes.
func startUpstream(t *testing.T) *httptest.Server {
t.Helper()
mux := http.NewServeMux()
mux.HandleFunc("/image", func(w http.ResponseWriter, _ *http.Request) {
w.Header().Set("Content-Type", contentTypeJPEG)
_, _ = io.WriteString(w, imagePayload)
})
mux.HandleFunc("/status/500", func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusInternalServerError)
})
mux.HandleFunc("/html", func(w http.ResponseWriter, _ *http.Request) {
w.Header().Set("Content-Type", "text/html; charset=utf-8")
_, _ = io.WriteString(w, "<html></html>")
})
mux.HandleFunc("/redirect/private", func(w http.ResponseWriter, r *http.Request) {
http.Redirect(w, r, "http://169.254.169.254/latest/meta-data/", http.StatusFound)
})
mux.HandleFunc("/redirect/public", func(w http.ResponseWriter, r *http.Request) {
http.Redirect(w, r, "/image", http.StatusFound)
})
mux.HandleFunc("/redirect/chain", func(w http.ResponseWriter, r *http.Request) {
http.Redirect(w, r, "/redirect/hop", http.StatusFound)
})
mux.HandleFunc("/redirect/hop", func(w http.ResponseWriter, r *http.Request) {
http.Redirect(w, r, "/image", http.StatusFound)
})
srv := httptest.NewServer(mux)
t.Cleanup(srv.Close)
return srv
}
// newServerFetcher builds a fetcher whose transport routes testPublicHost to
// srv, leaving the real SSRF validation and redirect checks in place.
func newServerFetcher(
t *testing.T,
srv *httptest.Server,
cfg *Config,
) (*HTTPFetcher, *recordingDialer) {
t.Helper()
if cfg == nil {
cfg = DefaultConfig()
}
cfg.AllowHTTP = true
f := New(cfg)
transport, ok := f.client.Transport.(*http.Transport)
if !ok {
t.Fatalf("transport is %T, want *http.Transport", f.client.Transport)
}
dialer := &recordingDialer{target: srv.Listener.Addr().String()}
transport.DialContext = dialer.dialContext
return f, dialer
}
// testContext returns a context cancelled when the test ends, bounding any
// fetch that would otherwise block on a leaked semaphore slot.
func testContext(t *testing.T) context.Context {
t.Helper()
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
t.Cleanup(cancel)
return ctx
}
// fetchImage fetches path from the fake upstream and fails on error.
func fetchImage(t *testing.T, f *HTTPFetcher, path string) *FetchResult {
t.Helper()
res, err := f.Fetch(testContext(t), upstreamURL(path))
if err != nil {
t.Fatalf("Fetch(%s) error = %v", path, err)
}
return res
}
// fetchExpectError fetches path and fails unless Fetch returns an error.
func fetchExpectError(t *testing.T, f *HTTPFetcher, path string) error {
t.Helper()
res, err := f.Fetch(testContext(t), upstreamURL(path))
if err == nil {
_ = res.Content.Close()
t.Fatalf("Fetch(%s) = nil error, want an error", path)
}
return err
}
// fetchBody fetches path and returns the fully read, closed response body.
func fetchBody(t *testing.T, f *HTTPFetcher, path string) string {
t.Helper()
res := fetchImage(t, f, path)
defer func() { _ = res.Content.Close() }()
data, err := io.ReadAll(res.Content)
if err != nil {
t.Fatalf("read body: %v", err)
}
return string(data)
}
// semLen reports how many per-host semaphore slots are currently held.
func semLen(f *HTTPFetcher, host string) int {
return len(f.getHostSemaphore(host))
}
func TestFetchRedirectToPrivateIPBlocked(t *testing.T) {
t.Parallel()
srv := startUpstream(t)
f, dialer := newServerFetcher(t, srv, nil)
_, err := f.Fetch(testContext(t), upstreamURL("/redirect/private"))
if !errors.Is(err, ErrSSRFBlocked) {
t.Fatalf("Fetch() error = %v, want ErrSSRFBlocked", err)
}
for _, addr := range dialer.dialedAddrs() {
if strings.Contains(addr, "169.254.169.254") {
t.Errorf("dialer connected to the private redirect target: %s", addr)
}
}
}
func TestFetchRedirectToPublicSucceeds(t *testing.T) {
t.Parallel()
srv := startUpstream(t)
f, _ := newServerFetcher(t, srv, nil)
if body := fetchBody(t, f, "/redirect/public"); body != imagePayload {
t.Errorf("body = %q, want %q", body, imagePayload)
}
}
func TestFetchRedirectChainSucceeds(t *testing.T) {
t.Parallel()
srv := startUpstream(t)
f, _ := newServerFetcher(t, srv, nil)
if body := fetchBody(t, f, "/redirect/chain"); body != imagePayload {
t.Errorf("body = %q, want %q", body, imagePayload)
}
}
func TestFetchRejectsNon2xx(t *testing.T) {
t.Parallel()
srv := startUpstream(t)
f, _ := newServerFetcher(t, srv, nil)
err := fetchExpectError(t, f, "/status/500")
if !errors.Is(err, ErrUpstreamError) {
t.Fatalf("Fetch() error = %v, want ErrUpstreamError", err)
}
}
func TestFetchRejectsDisallowedContentType(t *testing.T) {
t.Parallel()
srv := startUpstream(t)
f, _ := newServerFetcher(t, srv, nil)
err := fetchExpectError(t, f, "/html")
if !errors.Is(err, ErrInvalidContentType) {
t.Fatalf("Fetch() error = %v, want ErrInvalidContentType", err)
}
}
func TestFetchMaxResponseSizeEnforced(t *testing.T) {
t.Parallel()
srv := startUpstream(t)
cfg := DefaultConfig()
cfg.MaxResponseSize = 8
f, _ := newServerFetcher(t, srv, cfg)
res := fetchImage(t, f, "/image")
defer func() { _ = res.Content.Close() }()
data, err := io.ReadAll(res.Content)
if !errors.Is(err, ErrResponseTooLarge) {
t.Fatalf("read error = %v, want ErrResponseTooLarge", err)
}
if int64(len(data)) > cfg.MaxResponseSize {
t.Errorf("read %d bytes, exceeds limit %d", len(data), cfg.MaxResponseSize)
}
}
func TestFetchSemaphoreReleasedOnError(t *testing.T) {
t.Parallel()
srv := startUpstream(t)
cfg := DefaultConfig()
cfg.MaxConnectionsPerHost = 1
f, _ := newServerFetcher(t, srv, cfg)
err := fetchExpectError(t, f, "/status/500")
if !errors.Is(err, ErrUpstreamError) {
t.Fatalf("Fetch() error = %v, want ErrUpstreamError", err)
}
if held := semLen(f, testPublicHost); held != 0 {
t.Fatalf("semaphore slot leaked after error: %d held", held)
}
// One slot per host: this fetch proceeds only if the slot was released.
res := fetchImage(t, f, "/image")
_ = res.Content.Close()
}
// assertSlotReleasedByClose fetches an image over a one-slot host, hands the
// open result to consume, and asserts the slot is held before and freed after,
// then that a follow-up fetch can still acquire it.
func assertSlotReleasedByClose(
t *testing.T,
consume func(*testing.T, *FetchResult),
) {
t.Helper()
srv := startUpstream(t)
cfg := DefaultConfig()
cfg.MaxConnectionsPerHost = 1
f, _ := newServerFetcher(t, srv, cfg)
res := fetchImage(t, f, "/image")
if held := semLen(f, testPublicHost); held != 1 {
t.Fatalf("slot not held while body is open: %d held", held)
}
consume(t, res)
if held := semLen(f, testPublicHost); held != 0 {
t.Fatalf("slot not released after close: %d held", held)
}
next := fetchImage(t, f, "/image")
_ = next.Content.Close()
}
func TestFetchSemaphoreReleasedOnBodyClose(t *testing.T) {
t.Parallel()
assertSlotReleasedByClose(t, func(t *testing.T, res *FetchResult) {
t.Helper()
_, err := io.ReadAll(res.Content)
if err != nil {
t.Fatalf("read body: %v", err)
}
err = res.Content.Close()
if err != nil {
t.Fatalf("close body: %v", err)
}
})
}
func TestFetchSemaphoreReleasedOnPartialReadClose(t *testing.T) {
t.Parallel()
assertSlotReleasedByClose(t, func(t *testing.T, res *FetchResult) {
t.Helper()
buf := make([]byte, 1)
_, err := res.Content.Read(buf)
if err != nil {
t.Fatalf("partial read: %v", err)
}
err = res.Content.Close()
if err != nil {
t.Fatalf("close body: %v", err)
}
})
}
// The dial-time re-resolution in ssrfSafeDialer is what closes the DNS
// rebinding window: even if validateURL saw a public answer earlier, the
// dialer independently re-checks the address it is about to connect to. A full
// rebinding simulation (a resolver returning public, then private) would mean
// replacing the global net.DefaultResolver with a fake DNS server, which is
// heavyweight and unsafe to mutate under parallel -race tests. The property is
// proven directly here instead: the dialer rejects a private target outright,
// which is exactly the check that fires when a validated host later resolves
// to a private address.
func TestSSRFSafeDialerBlocksPrivateTarget(t *testing.T) {
t.Parallel()
for _, addr := range []string{
"169.254.169.254:80", // link-local (cloud metadata)
"127.0.0.1:80", // loopback
"10.0.0.5:80", // RFC 1918 private
} {
t.Run(addr, func(t *testing.T) {
t.Parallel()
_, err := ssrfSafeDialer(context.Background(), "tcp", addr)
if !errors.Is(err, ErrSSRFBlocked) {
t.Errorf("ssrfSafeDialer(%q) = %v, want ErrSSRFBlocked", addr, err)
}
})
}
}
func TestSSRFSafeDialerAllowsPublicTarget(t *testing.T) {
t.Parallel()
// A cancelled context makes the dial fail immediately without touching the
// network; the point is only that a public literal is not SSRF-blocked.
ctx, cancel := context.WithCancel(context.Background())
cancel()
_, err := ssrfSafeDialer(ctx, "tcp", testPublicHost+":80")
if err == nil {
t.Fatal("expected a dial error for an unreachable public target")
}
if errors.Is(err, ErrSSRFBlocked) {
t.Errorf("public target was SSRF-blocked: %v", err)
}
}
+71 -5
Voir le fichier
@@ -11,6 +11,7 @@ import (
"net"
"net/http"
"net/http/httptrace"
"net/netip"
neturl "net/url"
"slices"
"strings"
@@ -46,6 +47,20 @@ const (
localhostIPv6 = "::1"
)
// builtinBlockedPrefixes are internal or special-use ranges that Go's
// net.IP predicates (IsPrivate, IsLinkLocalUnicast, and the like) do not
// already cover. They are always blocked, in addition to any
// operator-supplied networks. IPv4-mapped IPv6 addresses are unmapped
// before matching, so these IPv4 ranges are caught in both forms.
//
//nolint:gochecknoglobals // immutable built-in blocklist
var builtinBlockedPrefixes = []netip.Prefix{
netip.MustParsePrefix("100.64.0.0/10"), // RFC 6598 CGNAT / carrier-grade NAT
netip.MustParsePrefix("192.0.0.0/24"), // RFC 6890 IETF protocol assignments
netip.MustParsePrefix("198.18.0.0/15"), // RFC 2544 benchmarking range
netip.MustParsePrefix("64:ff9b::/96"), // RFC 6052 NAT64 (maps onto IPv4)
}
// Fetcher errors.
var (
ErrSSRFBlocked = errors.New("request blocked: private or internal IP")
@@ -107,6 +122,9 @@ type Config struct {
AllowHTTP bool
// MaxConnectionsPerHost limits concurrent connections to each upstream host.
MaxConnectionsPerHost int
// BlockedNetworks are operator-supplied CIDR ranges refused by the
// dialer, in addition to the always-enforced built-in ranges.
BlockedNetworks []netip.Prefix
}
// DefaultConfig returns a Config with sensible defaults.
@@ -142,9 +160,13 @@ func New(config *Config) *HTTPFetcher {
config = DefaultConfig()
}
// Create transport with SSRF-safe dialer
// Create transport with SSRF-safe dialer. The dialer re-resolves and
// re-checks at connect time (closing the DNS-rebinding window) against
// both the built-in ranges and the operator-supplied blocklist.
transport := &http.Transport{
DialContext: ssrfSafeDialer,
DialContext: func(ctx context.Context, network, addr string) (net.Conn, error) {
return dialSSRFSafe(ctx, network, addr, config.BlockedNetworks)
},
TLSHandshakeTimeout: DefaultTLSTimeout,
MaxIdleConns: DefaultMaxIdleConns,
IdleConnTimeout: DefaultIdleConnTimeout,
@@ -451,11 +473,53 @@ func isPrivateIP(ip net.IP) bool {
}
}
return false
// Special-use ranges the net.IP predicates above do not cover.
addr, ok := netip.AddrFromSlice(ip)
if !ok {
return true
}
// ssrfSafeDialer is a custom dialer that validates IP addresses before connecting.
addr = addr.Unmap()
return slices.ContainsFunc(builtinBlockedPrefixes, func(prefix netip.Prefix) bool {
return prefix.Contains(addr)
})
}
// isBlockedIP reports whether ip is refused, either by the built-in
// internal-range check or by one of the operator-supplied prefixes.
func isBlockedIP(ip net.IP, blocked []netip.Prefix) bool {
if isPrivateIP(ip) {
return true
}
addr, ok := netip.AddrFromSlice(ip)
if !ok {
return true
}
addr = addr.Unmap()
return slices.ContainsFunc(blocked, func(prefix netip.Prefix) bool {
return prefix.Contains(addr)
})
}
// ssrfSafeDialer validates IP addresses against the built-in blocked ranges
// before connecting. New wraps dialSSRFSafe with the operator-supplied
// blocklist; this entry point enforces the built-in ranges alone.
func ssrfSafeDialer(ctx context.Context, network, addr string) (net.Conn, error) {
return dialSSRFSafe(ctx, network, addr, nil)
}
// dialSSRFSafe re-resolves addr and refuses to connect to any built-in
// internal range or operator-supplied blocked prefix, closing the
// DNS-rebinding window at connect time.
func dialSSRFSafe(
ctx context.Context,
network, addr string,
blocked []netip.Prefix,
) (net.Conn, error) {
host, port, err := net.SplitHostPort(addr)
if err != nil {
return nil, err
@@ -468,9 +532,11 @@ func ssrfSafeDialer(ctx context.Context, network, addr string) (net.Conn, error)
}
// Check all resolved IPs
if slices.ContainsFunc(ips, isPrivateIP) {
for _, ip := range ips {
if isBlockedIP(ip, blocked) {
return nil, ErrSSRFBlocked
}
}
// Connect using the first valid IP
var dialer net.Dialer
-2
Voir le fichier
@@ -452,8 +452,6 @@ func signatureRequest(req *ImageRequest) *signature.Request {
Width: req.Size.Width,
Height: req.Size.Height,
Format: string(req.Format),
Quality: req.Quality,
FitMode: string(req.FitMode),
Signature: req.Signature,
Expires: req.Expires,
}
+10 -4
Voir le fichier
@@ -44,11 +44,17 @@ func (s *Server) SetupRoutes() {
// Static files (Tailwind CSS, etc.)
s.router.Handle("/static/*", http.StripPrefix("/static/", static.Handler()))
// Login/generator UI
s.router.Get("/", s.h.HandleRoot())
s.router.Post("/", s.h.HandleRoot())
// Login/generator UI. The form routes carry CSRF protection; the
// token cookie is independent of the session cookie, so it also
// covers the login POST, where no session exists yet.
s.router.Group(func(r chi.Router) {
r.Use(s.h.CSRF())
r.Get("/", s.h.HandleRoot())
r.Post("/", s.h.HandleRoot())
r.Post("/generate", s.h.HandleGenerateURL())
})
s.router.Get("/logout", s.h.HandleLogout())
s.router.Post("/generate", s.h.HandleGenerateURL())
// Main image proxy route
// /v1/image/<host>/<path>/<width>x<height>.<format>
-122
Voir le fichier
@@ -1,122 +0,0 @@
package signature_test
import (
"testing"
"time"
"sneak.berlin/go/pixa/internal/signature"
)
// Fixed inputs for the quality/fit golden vectors. They are independent of
// the constants in golden_test.go so this file pins the current signed
// format on its own.
const (
qfSigningKey = "golden-test-key"
qfExpiresUnix int64 = 1704067200 // 2024-01-01T00:00:00Z
qfFitCover = "cover"
qfFitContain = "contain"
)
type qualityFitGoldenVector struct {
name string
req signature.Request
// wantSignature is the exact base64url (RFC 4648 URL-safe, padded)
// HMAC-SHA256 signature for the request with Expires set to
// qfExpiresUnix, under the signed format
// "host:path:query:width:height:format:expiration:quality:fit".
wantSignature string
}
// qualityFitGoldenVectors returns the known-answer vectors that pin quality
// and fit as signed components. The three default-value vectors use the
// effective quality (85) and fit ("cover") the handler applies when a URL
// omits q and fit, so they are the signatures real signed URLs must carry.
func qualityFitGoldenVectors() []qualityFitGoldenVector {
return []qualityFitGoldenVector{
{
name: "resized, default quality and fit",
req: signature.Request{
SourceHost: testHost,
SourcePath: testPath,
Width: 800,
Height: 600,
Format: testFormatWebP,
Quality: 85,
FitMode: qfFitCover,
},
// "cdn.example.com:/photos/cat.jpg::800:600:webp:1704067200:85:cover"
wantSignature: "kdqeGoW2SX7qnaYtoB970wEnLydn0UnIgQYQLfAnjXQ=",
},
{
name: "resized with query, default quality and fit",
req: signature.Request{
SourceHost: testHost,
SourcePath: testPath,
SourceQuery: "token=abc&v=2",
Width: 800,
Height: 600,
Format: testFormatWebP,
Quality: 85,
FitMode: qfFitCover,
},
// "cdn.example.com:/photos/cat.jpg:token=abc&v=2:800:600:webp:1704067200:85:cover"
wantSignature: "pKgVBOTd_Q_EikI7MNQLC9Q8Hurdxzyv3EIYvVhqc2I=",
},
{
name: "original size, default quality and fit",
req: signature.Request{
SourceHost: testHost,
SourcePath: testPath,
Width: 0,
Height: 0,
Format: testFormatPNG,
Quality: 85,
FitMode: qfFitCover,
},
// "cdn.example.com:/photos/cat.jpg::0:0:png:1704067200:85:cover"
wantSignature: "6_rZ0yyVbGZRs8kG7n7HLgLi5Jt8vjiWQljIEL1jbIs=",
},
{
name: "non-default quality and fit",
req: signature.Request{
SourceHost: testHost,
SourcePath: testPath,
Width: 800,
Height: 600,
Format: testFormatWebP,
Quality: 40,
FitMode: qfFitContain,
},
// "cdn.example.com:/photos/cat.jpg::800:600:webp:1704067200:40:contain"
wantSignature: "pGaXpPUbI3A7nMx-4T9bfq9bYWBNL0kY4bxlcv3g1F8=",
},
}
}
// TestSigner_GoldenVectors_QualityFit pins the exact HMAC-SHA256 signature
// output for requests that carry quality and fit as signed components. If
// these assertions fail, the signed byte format
// ("host:path:query:width:height:format:expiration:quality:fit") or the
// base64url encoding has changed, breaking every signature already issued.
// Update these constants only as part of a deliberate, documented signature
// format migration.
func TestSigner_GoldenVectors_QualityFit(t *testing.T) {
t.Parallel()
signer := signature.New(qfSigningKey)
for _, tt := range qualityFitGoldenVectors() {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
signReq := tt.req
signReq.Expires = time.Unix(qfExpiresUnix, 0)
gotSignature := signer.Sign(&signReq)
if gotSignature != tt.wantSignature {
t.Errorf("Sign() = %q, want %q (signed byte format changed?)",
gotSignature, tt.wantSignature)
}
})
}
}
-68
Voir le fichier
@@ -1,68 +0,0 @@
package signature_test
import (
"errors"
"testing"
"time"
"sneak.berlin/go/pixa/internal/signature"
)
// signedQualityFitRequest returns a request signed for quality 85 and fit
// mode "cover", the effective defaults the handler applies before
// verification.
func signedQualityFitRequest(signer *signature.Signer) *signature.Request {
req := &signature.Request{
SourceHost: testHost,
SourcePath: testPath,
Width: 800,
Height: 600,
Format: testFormatWebP,
Quality: 85,
FitMode: "cover",
Expires: time.Now().Add(1 * time.Hour),
}
req.Signature = signer.Sign(req)
return req
}
// TestSigner_Verify_QualityAndFitAreSigned proves that quality and fit are
// covered by the signature: a URL signed for one quality or fit mode must
// not verify when replayed with a different quality or fit mode. This is the
// amplification vector from the issue — one signed URL replayed across many
// quality and fit values yields many unauthorized cache entries and
// transcodes — so it must be rejected.
func TestSigner_Verify_QualityAndFitAreSigned(t *testing.T) {
t.Parallel()
signer := signature.New("test-secret-key")
cases := []struct {
name string
tamper func(r *signature.Request)
}{
{
name: "replayed with different quality",
tamper: func(r *signature.Request) { r.Quality = 40 },
},
{
name: "replayed with different fit mode",
tamper: func(r *signature.Request) { r.FitMode = "contain" },
},
}
for _, tt := range cases {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
req := signedQualityFitRequest(signer)
tt.tamper(req)
err := signer.Verify(req)
if !errors.Is(err, signature.ErrInvalid) {
t.Errorf("Verify() = %v, want %v", err, signature.ErrInvalid)
}
})
}
}
+5 -20
Voir le fichier
@@ -37,15 +37,6 @@ type Request struct {
Height int
// Format is the requested output format (e.g. "webp").
Format string
// Quality is the requested output quality (1-100) for lossy formats.
// It is the effective value the request resolves to: callers pass the
// default quality when the request omits the parameter, so an omitted
// quality signs identically to that same value stated explicitly.
Quality int
// FitMode is how the image is fit into the requested dimensions
// (e.g. "cover"). Like Quality it is the effective value: callers pass
// the default fit mode when the request omits the parameter.
FitMode string
// Signature is the HMAC signature to verify.
Signature string
// Expires is the signature expiration timestamp.
@@ -65,8 +56,7 @@ func New(secretKey string) *Signer {
}
// Sign generates an HMAC-SHA256 signature for the given request.
// The signature covers: host + path + query + width + height + format +
// expiration + quality + fit.
// The signature covers: host + path + query + width + height + format + expiration.
func (s *Signer) Sign(req *Request) string {
data := s.buildSignatureData(req)
mac := hmac.New(sha256.New, s.secretKey)
@@ -78,8 +68,7 @@ func (s *Signer) Sign(req *Request) string {
// Verify checks if the signature on the request is valid and not expired.
// Signatures are exact-match only: every component of the signed data
// (host, path, query, dimensions, format, expiration, quality, fit) must
// match exactly.
// (host, path, query, dimensions, format, expiration) must match exactly.
// No suffix matching, wildcard matching, or partial matching is supported.
// A signature for "cdn.example.com" will NOT verify for "example.com" or
// "other.cdn.example.com", and vice versa.
@@ -153,13 +142,11 @@ func (s *Signer) GenerateSignedURL(
}
// buildSignatureData creates the string to be signed.
// Format: "host:path:query:width:height:format:expiration:quality:fit"
// Format: "host:path:query:width:height:format:expiration"
// All components are used verbatim (exact match). No normalization,
// suffix matching, or wildcard expansion is performed. Quality and fit
// are the effective transform values, so replaying a signed URL with a
// different quality or fit mode fails verification.
// suffix matching, or wildcard expansion is performed.
func (s *Signer) buildSignatureData(req *Request) string {
return fmt.Sprintf("%s:%s:%s:%d:%d:%s:%d:%d:%s",
return fmt.Sprintf("%s:%s:%s:%d:%d:%s:%d",
req.SourceHost,
req.SourcePath,
req.SourceQuery,
@@ -167,8 +154,6 @@ func (s *Signer) buildSignatureData(req *Request) string {
req.Height,
req.Format,
req.Expires.Unix(),
req.Quality,
req.FitMode,
)
}
+1
Voir le fichier
@@ -47,6 +47,7 @@
{{end}}
<form method="POST" action="/generate" class="bg-white rounded-lg shadow-md p-6 space-y-4">
{{ .CSRFField }}
<div>
<label for="url" class="block text-sm font-medium text-gray-700 mb-1">
Source URL
+1
Voir le fichier
@@ -17,6 +17,7 @@
{{end}}
<form method="POST" action="/" class="space-y-4">
{{ .CSRFField }}
<div>
<label for="key" class="block text-sm font-medium text-gray-700 mb-1">
Signing Key
+5 -1
Voir le fichier
@@ -17,7 +17,11 @@ run_with_cgo_deps() {
main() {
cd "$ROOT"
echo "Running tests..."
run_with_cgo_deps "CGO_ENABLED=1 go test -timeout 30s -race -v ./..."
# Run without -v first for clean output on success; on failure rerun
# with -v for full diagnostics, then exit non-zero (REPO_POLICIES.md
# conditional-verbose-rerun pattern). The first run already proved the
# tests broken, so the build fails even if the rerun happens to pass.
run_with_cgo_deps "CGO_ENABLED=1 go test -timeout 30s -race -cover ./... || { echo '--- Rerunning with -v for details ---'; CGO_ENABLED=1 go test -timeout 30s -race -v ./...; exit 1; }"
}
main "$@"