Compare commits
2
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
ba5a716223 | ||
|
|
c7173c47d8 |
@@ -120,8 +120,9 @@ path under `/v1/` answers 200, in maintenance mode too.
|
||||
`blocked_networks`); 502 when the upstream answered with an error status, and
|
||||
for 5 minutes after that for the same source URL; 503 when pixa is busy or in
|
||||
maintenance mode; 500 for any other failure.
|
||||
- `GET /v1/e/<token>/<name>` — an image through an encrypted URL (see Encrypted
|
||||
URLs). Needs: nothing but the URL. Answers: 200; 400 for a token that does not
|
||||
- `GET` or `HEAD` `/v1/e/<token>/<name>` — an image through an encrypted URL
|
||||
(see Encrypted URLs). Needs: nothing but the URL. Answers: 200; 304 when
|
||||
`If-None-Match` matches the image's `ETag`; 400 for a token that does not
|
||||
decrypt, or that asks for a size or fit that is not valid; 410 once it has
|
||||
expired; 504 when the upstream has not sent its response headers within
|
||||
`upstream_fetch_timeout`, but 500 when that time runs out while the image
|
||||
@@ -137,6 +138,15 @@ path under `/v1/` answers 200, in maintenance mode too.
|
||||
authentication with `metrics.username` and `metrics.password`. Answers: 200;
|
||||
401 without them; 404 when they are not set, as the route then does not exist.
|
||||
|
||||
Every response carries an `X-Request-ID` header holding the request's ID, which
|
||||
a client can quote when reporting a problem: the request's own `X-Request-ID`,
|
||||
as a reverse proxy in front of pixa may send, when it is at most 64 letters,
|
||||
digits, `-`, `_` or `.`; otherwise a random one pixa makes for the request,
|
||||
which tells nothing about the machine or the other requests. pixa's log line for
|
||||
the request carries the same ID as `request_id`, and so do the lines it logs
|
||||
when it fetches, converts and serves an image; the fetch sends it to the
|
||||
upstream host as `X-Request-ID`.
|
||||
|
||||
Both `POST` routes accept only a form that pixa's own page served: the page puts
|
||||
a token in the form and sets a cookie to match, and a request without both is
|
||||
refused with 403, so another site cannot submit the form from a visitor's
|
||||
@@ -186,7 +196,9 @@ source) and one transcode: the first request does the work, and the others wait
|
||||
for its image or its error, holding no upstream connection or processing slot
|
||||
of their own. A waiting request stops waiting when its own client goes away.
|
||||
The work goes on for the others even if the first request's client goes away,
|
||||
until that request's `downstream_timeout` ends.
|
||||
until that request's `downstream_timeout` ends. The shared fetch sends the first
|
||||
request's ID upstream, and the lines logged for the fetch and the transcode
|
||||
carry that ID.
|
||||
|
||||
The login form (`POST /`) is limited to 5 attempts per minute per client
|
||||
address, counting an IPv6 client by its /64; an attempt over the limit is
|
||||
|
||||
@@ -29,21 +29,29 @@ P2: security: referer blacklist
|
||||
|
||||
# Completed Steps
|
||||
|
||||
- 2026-10-04 the metrics basic auth, CORS preflight, request logging and
|
||||
metrics recording have tests (closes #79): `MetricsAuth` on its own answers
|
||||
401 with a challenge without credentials or with a wrong username or password
|
||||
and lets the configured ones through; a preflight request gets `*` for any
|
||||
origin when `access_control_allow_origin` is `*` and no
|
||||
`Access-Control-Allow-Origin` from another origin than the configured one; a
|
||||
`POST /` carrying the signing key leaves no trace of it in the request log
|
||||
line, and the login handler's own log lines leave out the submitted key; the
|
||||
metrics middleware on its own records a request it served, and the router
|
||||
records nothing while no metrics username is set. Not tested: that the router
|
||||
puts the basic auth in front of `/metrics` and records requests when a
|
||||
metrics username is set. Only one test per package can set up `/metrics`, and
|
||||
in `internal/server` that is `TestMaintenanceModeKeepsOtherRoutes`, which
|
||||
needs the owner's approval to change; #180 holds it. Tests only; the basic
|
||||
auth library already compares the password in constant time.
|
||||
- 2026-10-04 the image route's signature check and error answers are tested
|
||||
(closes #76): new tests in `internal/handlers`, with no network, check the
|
||||
status and JSON error body for a missing, wrong, unpadded, upper-case or
|
||||
expired signature on a host not on the allowlist, or a valid one sent for
|
||||
its parent domain, a sibling host, a subdomain or the host with another
|
||||
domain appended (401), an unparseable path (400), `localhost` as the
|
||||
upstream host (403) and an upstream error (502); that an allowlisted host is
|
||||
served without a signature, another host only with a valid one; and the
|
||||
answers of `/robots.txt` and the health check. No code changes.
|
||||
- 2026-10-04 request IDs returned and passed on, and `/v1/e/` revalidates
|
||||
(closes #84): pixa's own `RequestID` middleware, in place of chi's, gives each
|
||||
request an ID, its own `X-Request-ID` when that is at most 64 letters, digits,
|
||||
`-`, `_` or `.` and a random one otherwise, stores it where chi's did and
|
||||
sends it back as `X-Request-ID` on every response; the upstream fetch sends
|
||||
that ID, and the "upstream fetched", "image converted" and "image served" log
|
||||
lines carry it as `request_id`, a fetch shared by several requests carrying
|
||||
the first request's; `/v1/e/` sets `ETag`, answers a matching `If-None-Match`
|
||||
with 304 and is routed for `HEAD`, the `ETag` and 304 code being
|
||||
`notModified`, which `/v1/image/` calls too; its token checks moved unchanged
|
||||
into `parseImageEncRequest` to keep `HandleImageEnc` within the line limit; no
|
||||
`Vary` is added, as no response depends on a request header except the image
|
||||
routes' CORS headers, for which `go-chi/cors` already sends `Vary: Origin`;
|
||||
`Vary: Accept` is left to #88.
|
||||
- 2026-10-04 routes, encrypted URLs and config file documented (closes #75):
|
||||
"Routes" in `README.md` lists every route with its method, purpose, what it
|
||||
needs and the status codes it answers with, and says `q` and `fit` are part
|
||||
@@ -453,7 +461,6 @@ P2: security: referer blacklist
|
||||
- P2: HTTP response handling
|
||||
- Last-Modified headers
|
||||
- Vary header for content negotiation
|
||||
- X-Request-ID propagation
|
||||
- P2: auto format selection (format=auto based on Accept header)
|
||||
- P2: configuration
|
||||
- YAML config file support
|
||||
|
||||
@@ -1,59 +0,0 @@
|
||||
package handlers
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"log/slog"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"net/url"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"sneak.berlin/go/pixa/internal/config"
|
||||
"sneak.berlin/go/pixa/internal/session"
|
||||
)
|
||||
|
||||
// TestLoginLogLeavesOutSubmittedKey verifies that the log lines for a
|
||||
// failed and for a successful login do not contain the submitted key.
|
||||
func TestLoginLogLeavesOutSubmittedKey(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
const wrongKey = "wrong-signing-key-fedcba9876543210"
|
||||
|
||||
var buf bytes.Buffer
|
||||
|
||||
sessMgr, err := session.NewManager(testSigningKey)
|
||||
if err != nil {
|
||||
t.Fatalf("session.NewManager() error = %v", err)
|
||||
}
|
||||
|
||||
h := &Handlers{
|
||||
log: slog.New(slog.NewJSONHandler(&buf, nil)),
|
||||
config: &config.Config{SigningKey: testSigningKey},
|
||||
sessMgr: sessMgr,
|
||||
}
|
||||
|
||||
submittedKeys := []string{wrongKey, testSigningKey}
|
||||
|
||||
for _, key := range submittedKeys {
|
||||
form := url.Values{loginKeyField: {key}}
|
||||
req := httptest.NewRequestWithContext(
|
||||
t.Context(), http.MethodPost, "/",
|
||||
strings.NewReader(form.Encode()))
|
||||
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
|
||||
|
||||
h.handleLoginPost(httptest.NewRecorder(), req)
|
||||
}
|
||||
|
||||
for _, msg := range []string{"failed login attempt", "successful login"} {
|
||||
if !strings.Contains(buf.String(), msg) {
|
||||
t.Fatalf("log missing %q; got %q", msg, buf.String())
|
||||
}
|
||||
}
|
||||
|
||||
for _, key := range submittedKeys {
|
||||
if strings.Contains(buf.String(), key) {
|
||||
t.Errorf("log contains submitted key %q; got %q", key, buf.String())
|
||||
}
|
||||
}
|
||||
}
|
||||
+23
-11
@@ -10,6 +10,7 @@ import (
|
||||
"time"
|
||||
|
||||
"github.com/go-chi/chi/v5"
|
||||
"github.com/go-chi/chi/v5/middleware"
|
||||
"sneak.berlin/go/pixa/internal/encurl"
|
||||
"sneak.berlin/go/pixa/internal/httpfetcher"
|
||||
"sneak.berlin/go/pixa/internal/imageprocessor"
|
||||
@@ -247,6 +248,25 @@ func cacheControl(expires time.Time) string {
|
||||
return fmt.Sprintf("public, max-age=%d, immutable", int64(maxAge/time.Second))
|
||||
}
|
||||
|
||||
// notModified sets the ETag header to etag and, when the request's
|
||||
// If-None-Match is that ETag, answers 304 Not Modified. It reports whether it
|
||||
// answered. An empty etag sets no header and never answers.
|
||||
func notModified(w http.ResponseWriter, r *http.Request, etag string) bool {
|
||||
if etag == "" {
|
||||
return false
|
||||
}
|
||||
|
||||
w.Header().Set("ETag", etag)
|
||||
|
||||
if r.Header.Get("If-None-Match") != etag {
|
||||
return false
|
||||
}
|
||||
|
||||
w.WriteHeader(http.StatusNotModified)
|
||||
|
||||
return true
|
||||
}
|
||||
|
||||
// writeImageResponse writes headers and streams the image content,
|
||||
// handling conditional and HEAD requests.
|
||||
func (s *Handlers) writeImageResponse(
|
||||
@@ -265,17 +285,8 @@ func (s *Handlers) writeImageResponse(
|
||||
w.Header().Set("Cache-Control", cacheControl(req.Expires))
|
||||
w.Header().Set("X-Pixa-Cache", string(resp.CacheStatus))
|
||||
|
||||
if resp.ETag != "" {
|
||||
w.Header().Set("ETag", resp.ETag)
|
||||
|
||||
// Check for conditional request (If-None-Match)
|
||||
if ifNoneMatch := r.Header.Get("If-None-Match"); ifNoneMatch != "" {
|
||||
if ifNoneMatch == resp.ETag {
|
||||
w.WriteHeader(http.StatusNotModified)
|
||||
|
||||
return
|
||||
}
|
||||
}
|
||||
if notModified(w, r, resp.ETag) {
|
||||
return
|
||||
}
|
||||
|
||||
// Handle HEAD request - return headers only
|
||||
@@ -298,6 +309,7 @@ func (s *Handlers) writeImageResponse(
|
||||
// Log cache status and timing after serving
|
||||
duration := time.Since(startTime)
|
||||
s.log.Info("image served",
|
||||
"request_id", middleware.GetReqID(r.Context()),
|
||||
"cache_key", cacheKey,
|
||||
"cache_status", resp.CacheStatus,
|
||||
"duration_ms", duration.Milliseconds(),
|
||||
|
||||
@@ -23,8 +23,10 @@ const photoPath = "/images/photo.jpg"
|
||||
// newSignedHostServer returns a router for both image routes, and the Handlers
|
||||
// behind it, whose fetcher serves a JPEG at photoPath on signedHost. signedHost
|
||||
// is not on the allowlist, so a /v1/image/ URL for it is served only with a
|
||||
// valid signature.
|
||||
func newSignedHostServer(t *testing.T) (*Handlers, http.Handler) {
|
||||
// valid signature. The handlers and the image service log to log.
|
||||
func newSignedHostServer(
|
||||
t *testing.T, log *slog.Logger,
|
||||
) (*Handlers, http.Handler) {
|
||||
t.Helper()
|
||||
|
||||
cache, err := imgcache.NewCache(setupTestDB(t), imgcache.CacheConfig{
|
||||
@@ -44,6 +46,7 @@ func newSignedHostServer(t *testing.T) (*Handlers, http.Handler) {
|
||||
signedHost + photoPath: &fstest.MapFile{Data: jpegData},
|
||||
}),
|
||||
SigningKey: testSigningKey,
|
||||
Logger: log,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("imgcache.NewService() error = %v", err)
|
||||
@@ -55,7 +58,7 @@ func newSignedHostServer(t *testing.T) (*Handlers, http.Handler) {
|
||||
}
|
||||
|
||||
h := &Handlers{
|
||||
log: slog.New(slog.DiscardHandler),
|
||||
log: log,
|
||||
imgSvc: svc,
|
||||
encGen: encGen,
|
||||
}
|
||||
@@ -103,7 +106,7 @@ func getMaxAge(t *testing.T, srv http.Handler, target string) int {
|
||||
func TestHandleImage_SignedURL_MaxAgeEndsAtExp(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
h, srv := newSignedHostServer(t)
|
||||
h, srv := newSignedHostServer(t, slog.New(slog.DiscardHandler))
|
||||
|
||||
signedURL, err := h.imgSvc.GenerateSignedURL("", &imgcache.ImageRequest{
|
||||
SourceHost: signedHost,
|
||||
@@ -179,7 +182,7 @@ func TestHandleImageEnc_MaxAge(t *testing.T) {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
h, srv := newSignedHostServer(t)
|
||||
h, srv := newSignedHostServer(t, slog.New(slog.DiscardHandler))
|
||||
|
||||
token, err := h.encGen.Generate(&encurl.Payload{
|
||||
SourceHost: signedHost,
|
||||
|
||||
@@ -0,0 +1,267 @@
|
||||
package handlers
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"image/color"
|
||||
"log/slog"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"testing"
|
||||
"testing/fstest"
|
||||
"time"
|
||||
|
||||
"github.com/go-chi/chi/v5"
|
||||
"sneak.berlin/go/pixa/internal/httpfetcher"
|
||||
"sneak.berlin/go/pixa/internal/imgcache"
|
||||
"sneak.berlin/go/pixa/internal/signature"
|
||||
)
|
||||
|
||||
// allowlistedHost is the only host on the allowlist of the image route
|
||||
// newImageRoute builds.
|
||||
const allowlistedHost = "allowed.example.com"
|
||||
|
||||
// newImageRoute returns the image route of a Handlers whose service fetches
|
||||
// with fetcher and checks signatures with testSigningKey.
|
||||
func newImageRoute(t *testing.T, fetcher httpfetcher.Fetcher) http.Handler {
|
||||
t.Helper()
|
||||
|
||||
cache, err := imgcache.NewCache(setupTestDB(t), imgcache.CacheConfig{
|
||||
StateDir: t.TempDir(),
|
||||
CacheTTL: time.Hour,
|
||||
NegativeTTL: 5 * time.Minute,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("failed to create cache: %v", err)
|
||||
}
|
||||
|
||||
svc, err := imgcache.NewService(&imgcache.ServiceConfig{
|
||||
Cache: cache,
|
||||
Fetcher: fetcher,
|
||||
SigningKey: testSigningKey,
|
||||
Allowlist: []string{allowlistedHost},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("failed to create service: %v", err)
|
||||
}
|
||||
|
||||
h := &Handlers{imgSvc: svc, log: slog.New(slog.DiscardHandler)}
|
||||
|
||||
r := chi.NewRouter()
|
||||
r.Get("/v1/image/*", h.HandleImage())
|
||||
|
||||
return r
|
||||
}
|
||||
|
||||
// newPhotoFetcher returns a mock fetcher that serves a JPEG at photoPath on
|
||||
// each of hosts, and answers any other URL with an upstream error.
|
||||
func newPhotoFetcher(t *testing.T, hosts ...string) *httpfetcher.MockFetcher {
|
||||
t.Helper()
|
||||
|
||||
photo := &fstest.MapFile{
|
||||
Data: generateTestJPEG(t, 100, 100, color.RGBA{255, 0, 0, 255}),
|
||||
}
|
||||
|
||||
files := fstest.MapFS{}
|
||||
for _, host := range hosts {
|
||||
files[host+photoPath] = photo
|
||||
}
|
||||
|
||||
return httpfetcher.NewMock(files)
|
||||
}
|
||||
|
||||
// photoURL returns the image route URL of photoPath on host, as a 50x50 JPEG.
|
||||
func photoURL(host string) string {
|
||||
return "/v1/image/" + host + photoPath + "/50x50.jpeg"
|
||||
}
|
||||
|
||||
// photoURLWithSig returns photoURL(host) with sig and expires as its sig and
|
||||
// exp.
|
||||
func photoURLWithSig(host, sig string, expires time.Time) string {
|
||||
return fmt.Sprintf("%s?sig=%s&exp=%d", photoURL(host), sig, expires.Unix())
|
||||
}
|
||||
|
||||
// photoSignature returns the signature of photoURL(host) at the default
|
||||
// quality and fit, made with key and expiring at expires.
|
||||
func photoSignature(key, host string, expires time.Time) string {
|
||||
return signature.New(key).Sign(&signature.Request{
|
||||
SourceHost: host,
|
||||
SourcePath: photoPath,
|
||||
Width: 50,
|
||||
Height: 50,
|
||||
Format: string(imgcache.FormatJPEG),
|
||||
Quality: 85,
|
||||
FitMode: string(imgcache.FitCover),
|
||||
Expires: expires,
|
||||
})
|
||||
}
|
||||
|
||||
// sendGet sends a GET for target to route and returns the response.
|
||||
func sendGet(
|
||||
t *testing.T, route http.Handler, target string,
|
||||
) *httptest.ResponseRecorder {
|
||||
t.Helper()
|
||||
|
||||
req := httptest.NewRequestWithContext(t.Context(), http.MethodGet, target, nil)
|
||||
rec := httptest.NewRecorder()
|
||||
|
||||
route.ServeHTTP(rec, req)
|
||||
t.Logf("GET %s: %d", target, rec.Code)
|
||||
|
||||
return rec
|
||||
}
|
||||
|
||||
// checkErrorBody checks that rec has status wantStatus and the JSON error body
|
||||
// the image route sends: wantError, wantStatus and the time in RFC 3339.
|
||||
func checkErrorBody(
|
||||
t *testing.T, rec *httptest.ResponseRecorder, wantStatus int, wantError string,
|
||||
) {
|
||||
t.Helper()
|
||||
|
||||
if rec.Code != wantStatus {
|
||||
t.Errorf("status = %d, want %d", rec.Code, wantStatus)
|
||||
}
|
||||
|
||||
if ct := rec.Header().Get("Content-Type"); ct != "application/json" {
|
||||
t.Errorf("Content-Type = %q, want application/json", ct)
|
||||
}
|
||||
|
||||
var body struct {
|
||||
Error string `json:"error"`
|
||||
Status int `json:"status"`
|
||||
Timestamp string `json:"timestamp"`
|
||||
}
|
||||
|
||||
err := json.NewDecoder(rec.Body).Decode(&body)
|
||||
if err != nil {
|
||||
t.Fatalf("decoding response body: %v", err)
|
||||
}
|
||||
|
||||
if body.Error != wantError || body.Status != wantStatus {
|
||||
t.Errorf("body error and status = %q %d, want %q %d",
|
||||
body.Error, body.Status, wantError, wantStatus)
|
||||
}
|
||||
|
||||
_, err = time.Parse(time.RFC3339, body.Timestamp)
|
||||
if err != nil {
|
||||
t.Errorf("body timestamp: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// TestHandleImage_ErrorAnswers checks the status and the JSON error body the
|
||||
// image route answers each request below with. The JPEG at photoPath exists on
|
||||
// signedHost and on each host below that differs from it, so a request refused
|
||||
// with 401 would otherwise be served.
|
||||
func TestHandleImage_ErrorAnswers(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
// A signature for signedHost must not verify for any of these.
|
||||
parentHost := "example.com"
|
||||
siblingHost := "other.example.com"
|
||||
subdomainHost := "img." + signedHost
|
||||
appendedHost := signedHost + ".example.net"
|
||||
|
||||
photos := newPhotoFetcher(t,
|
||||
signedHost, parentHost, siblingHost, subdomainHost, appendedHost)
|
||||
// The real fetcher refuses localhost before any lookup or connection.
|
||||
realFetcher := httpfetcher.New(httpfetcher.DefaultConfig())
|
||||
|
||||
exp := time.Now().Add(time.Hour)
|
||||
expired := time.Now().Add(-time.Hour)
|
||||
sig := photoSignature(testSigningKey, signedHost, exp)
|
||||
otherKeySig := photoSignature("another-signing-key", signedHost, exp)
|
||||
expiredSig := photoSignature(testSigningKey, signedHost, expired)
|
||||
localhostSig := photoSignature(testSigningKey, "localhost", exp)
|
||||
|
||||
// The error every request refused for its signature gets.
|
||||
const unauthorized = "unauthorized"
|
||||
|
||||
tests := []struct {
|
||||
name string
|
||||
fetcher httpfetcher.Fetcher
|
||||
target string
|
||||
wantStatus int
|
||||
wantError string
|
||||
}{
|
||||
{"no sig or exp", photos, photoURL(signedHost),
|
||||
http.StatusUnauthorized, unauthorized},
|
||||
{"exp but no sig", photos,
|
||||
fmt.Sprintf("%s?exp=%d", photoURL(signedHost), exp.Unix()),
|
||||
http.StatusUnauthorized, unauthorized},
|
||||
{"sig made with another key", photos,
|
||||
photoURLWithSig(signedHost, otherKeySig, exp),
|
||||
http.StatusUnauthorized, unauthorized},
|
||||
{"sig without its = padding", photos,
|
||||
photoURLWithSig(signedHost, strings.TrimRight(sig, "="), exp),
|
||||
http.StatusUnauthorized, unauthorized},
|
||||
{"sig in upper case", photos,
|
||||
photoURLWithSig(signedHost, strings.ToUpper(sig), exp),
|
||||
http.StatusUnauthorized, unauthorized},
|
||||
{"expired sig", photos, photoURLWithSig(signedHost, expiredSig, expired),
|
||||
http.StatusUnauthorized, unauthorized},
|
||||
{"sig sent for the parent domain", photos,
|
||||
photoURLWithSig(parentHost, sig, exp),
|
||||
http.StatusUnauthorized, unauthorized},
|
||||
{"sig sent for a sibling host", photos,
|
||||
photoURLWithSig(siblingHost, sig, exp),
|
||||
http.StatusUnauthorized, unauthorized},
|
||||
{"sig sent for a subdomain", photos,
|
||||
photoURLWithSig(subdomainHost, sig, exp),
|
||||
http.StatusUnauthorized, unauthorized},
|
||||
{"sig sent with another domain appended", photos,
|
||||
photoURLWithSig(appendedHost, sig, exp),
|
||||
http.StatusUnauthorized, unauthorized},
|
||||
{"unparseable path", photos,
|
||||
"/v1/image/" + allowlistedHost + photoPath + "/big.jpeg",
|
||||
http.StatusBadRequest, "invalid image URL: invalid size format"},
|
||||
{"blocked upstream address", realFetcher,
|
||||
photoURLWithSig("localhost", localhostSig, exp),
|
||||
http.StatusForbidden, "forbidden"},
|
||||
{"upstream error", photos,
|
||||
"/v1/image/" + allowlistedHost + "/images/missing.jpg/50x50.jpeg",
|
||||
http.StatusBadGateway, "upstream error"},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
rec := sendGet(t, newImageRoute(t, tt.fetcher), tt.target)
|
||||
checkErrorBody(t, rec, tt.wantStatus, tt.wantError)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestHandleImage_AllowlistOrSignature checks that the image route serves an
|
||||
// image without a signature for a host on the allowlist only, and for another
|
||||
// host only with a valid signature.
|
||||
func TestHandleImage_AllowlistOrSignature(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
photos := newPhotoFetcher(t, allowlistedHost, signedHost)
|
||||
exp := time.Now().Add(time.Hour)
|
||||
sig := photoSignature(testSigningKey, signedHost, exp)
|
||||
|
||||
tests := []struct {
|
||||
name string
|
||||
target string
|
||||
wantStatus int
|
||||
}{
|
||||
{"allowlisted host, no sig", photoURL(allowlistedHost), http.StatusOK},
|
||||
{"other host, no sig", photoURL(signedHost), http.StatusUnauthorized},
|
||||
{"other host, valid sig", photoURLWithSig(signedHost, sig, exp),
|
||||
http.StatusOK},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
rec := sendGet(t, newImageRoute(t, photos), tt.target)
|
||||
if rec.Code != tt.wantStatus {
|
||||
t.Errorf("status = %d, want %d", rec.Code, tt.wantStatus)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -9,6 +9,7 @@ import (
|
||||
"time"
|
||||
|
||||
"github.com/go-chi/chi/v5"
|
||||
"github.com/go-chi/chi/v5/middleware"
|
||||
|
||||
"sneak.berlin/go/pixa/internal/encurl"
|
||||
"sneak.berlin/go/pixa/internal/httpfetcher"
|
||||
@@ -24,43 +25,8 @@ func (s *Handlers) HandleImageEnc() http.HandlerFunc {
|
||||
ctx := r.Context()
|
||||
start := time.Now()
|
||||
|
||||
// Extract token from URL
|
||||
token := chi.URLParam(r, "token")
|
||||
if token == "" {
|
||||
s.respondError(w, "missing token", http.StatusBadRequest)
|
||||
|
||||
return
|
||||
}
|
||||
|
||||
// Decrypt and validate the payload
|
||||
payload, err := s.encGen.Parse(token)
|
||||
if err != nil {
|
||||
if errors.Is(err, encurl.ErrExpired) {
|
||||
s.log.Debug("encrypted URL expired", "error", err)
|
||||
s.respondError(w, "URL has expired", http.StatusGone)
|
||||
|
||||
return
|
||||
}
|
||||
|
||||
s.log.Debug("failed to decrypt URL", "error", err)
|
||||
s.respondError(w, "invalid encrypted URL", http.StatusBadRequest)
|
||||
|
||||
return
|
||||
}
|
||||
|
||||
// Convert payload to ImageRequest
|
||||
req := payload.ToImageRequest()
|
||||
|
||||
// Apply the same dimension and fit-mode bounds as the plain image
|
||||
// route: a sealed payload is trusted for its origin, not for staying
|
||||
// within limits, so an over-limit size or unknown fit mode is a 400
|
||||
// here rather than an out-of-memory or a 500 from the processor.
|
||||
err = imgcache.ValidateImageRequest(req)
|
||||
if err != nil {
|
||||
s.log.Debug("encrypted URL failed validation", "error", err)
|
||||
s.respondError(w, "invalid encrypted URL: "+err.Error(),
|
||||
http.StatusBadRequest)
|
||||
|
||||
req, ok := s.parseImageEncRequest(w, r)
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
|
||||
@@ -94,6 +60,17 @@ func (s *Handlers) HandleImageEnc() http.HandlerFunc {
|
||||
w.Header().Set("Cache-Control", cacheControl(req.Expires))
|
||||
w.Header().Set("X-Pixa-Cache", string(resp.CacheStatus))
|
||||
|
||||
if notModified(w, r, resp.ETag) {
|
||||
return
|
||||
}
|
||||
|
||||
// A HEAD request gets the headers only
|
||||
if r.Method == http.MethodHead {
|
||||
w.WriteHeader(http.StatusOK)
|
||||
|
||||
return
|
||||
}
|
||||
|
||||
// Stream the response
|
||||
written, err := io.Copy(w, resp.Content)
|
||||
if err != nil {
|
||||
@@ -105,6 +82,7 @@ func (s *Handlers) HandleImageEnc() http.HandlerFunc {
|
||||
// Log completion
|
||||
duration := time.Since(start)
|
||||
s.log.Info("image served",
|
||||
"request_id", middleware.GetReqID(ctx),
|
||||
"cache_key", imgcache.CacheKey(req),
|
||||
"host", req.SourceHost,
|
||||
"path", req.SourcePath,
|
||||
@@ -116,6 +94,56 @@ func (s *Handlers) HandleImageEnc() http.HandlerFunc {
|
||||
}
|
||||
}
|
||||
|
||||
// parseImageEncRequest decrypts the token of an encrypted image URL into an
|
||||
// ImageRequest and checks it. On a token that is missing, does not decrypt,
|
||||
// has expired or asks for something not valid, it writes an error response
|
||||
// and returns false.
|
||||
func (s *Handlers) parseImageEncRequest(
|
||||
w http.ResponseWriter, r *http.Request,
|
||||
) (*imgcache.ImageRequest, bool) {
|
||||
// Extract token from URL
|
||||
token := chi.URLParam(r, "token")
|
||||
if token == "" {
|
||||
s.respondError(w, "missing token", http.StatusBadRequest)
|
||||
|
||||
return nil, false
|
||||
}
|
||||
|
||||
// Decrypt and validate the payload
|
||||
payload, err := s.encGen.Parse(token)
|
||||
if err != nil {
|
||||
if errors.Is(err, encurl.ErrExpired) {
|
||||
s.log.Debug("encrypted URL expired", "error", err)
|
||||
s.respondError(w, "URL has expired", http.StatusGone)
|
||||
|
||||
return nil, false
|
||||
}
|
||||
|
||||
s.log.Debug("failed to decrypt URL", "error", err)
|
||||
s.respondError(w, "invalid encrypted URL", http.StatusBadRequest)
|
||||
|
||||
return nil, false
|
||||
}
|
||||
|
||||
// Convert payload to ImageRequest
|
||||
req := payload.ToImageRequest()
|
||||
|
||||
// Apply the same dimension and fit-mode bounds as the plain image
|
||||
// route: a sealed payload is trusted for its origin, not for staying
|
||||
// within limits, so an over-limit size or unknown fit mode is a 400
|
||||
// here rather than an out-of-memory or a 500 from the processor.
|
||||
err = imgcache.ValidateImageRequest(req)
|
||||
if err != nil {
|
||||
s.log.Debug("encrypted URL failed validation", "error", err)
|
||||
s.respondError(w, "invalid encrypted URL: "+err.Error(),
|
||||
http.StatusBadRequest)
|
||||
|
||||
return nil, false
|
||||
}
|
||||
|
||||
return req, true
|
||||
}
|
||||
|
||||
// handleImageError converts image service errors to HTTP responses.
|
||||
func (s *Handlers) handleImageError(w http.ResponseWriter, err error) {
|
||||
switch {
|
||||
|
||||
@@ -96,3 +96,70 @@ func TestHandleImageEnc_InvalidFitMode_Returns400(t *testing.T) {
|
||||
t.Fatalf("status = %d, want %d", rec.Code, http.StatusBadRequest)
|
||||
}
|
||||
}
|
||||
|
||||
// TestHandleImageEnc_IfNoneMatch_Returns304 verifies that an image served
|
||||
// through an encrypted URL carries an ETag, and that a request whose
|
||||
// If-None-Match is that ETag is answered 304 Not Modified with no body.
|
||||
func TestHandleImageEnc_IfNoneMatch_Returns304(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
h, srv := newSignedHostServer(t, slog.New(slog.DiscardHandler))
|
||||
target := encPhotoURL(t, h)
|
||||
|
||||
rec := httptest.NewRecorder()
|
||||
srv.ServeHTTP(rec, httptest.NewRequestWithContext(
|
||||
t.Context(), http.MethodGet, target, nil))
|
||||
|
||||
etag := rec.Header().Get("ETag")
|
||||
t.Logf("GET: %d, ETag %q", rec.Code, etag)
|
||||
|
||||
if rec.Code != http.StatusOK || etag == "" {
|
||||
t.Fatalf("GET: status = %d, ETag = %q, want %d and an ETag",
|
||||
rec.Code, etag, http.StatusOK)
|
||||
}
|
||||
|
||||
req := httptest.NewRequestWithContext(t.Context(), http.MethodGet, target, nil)
|
||||
req.Header.Set("If-None-Match", etag)
|
||||
|
||||
rec = httptest.NewRecorder()
|
||||
srv.ServeHTTP(rec, req)
|
||||
t.Logf("GET with If-None-Match: %d, %d body bytes", rec.Code, rec.Body.Len())
|
||||
|
||||
if rec.Code != http.StatusNotModified || rec.Body.Len() != 0 {
|
||||
t.Errorf("status = %d with %d body bytes, want %d with none",
|
||||
rec.Code, rec.Body.Len(), http.StatusNotModified)
|
||||
}
|
||||
}
|
||||
|
||||
// TestHandleImageEnc_HEAD_ReturnsHeadersOnly verifies that HEAD on an
|
||||
// encrypted URL is answered 200 with the headers GET sends and no body.
|
||||
func TestHandleImageEnc_HEAD_ReturnsHeadersOnly(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
h, _ := newSignedHostServer(t, slog.New(slog.DiscardHandler))
|
||||
|
||||
r := chi.NewRouter()
|
||||
r.Head("/v1/e/{token}/*", h.HandleImageEnc())
|
||||
|
||||
rec := httptest.NewRecorder()
|
||||
r.ServeHTTP(rec, httptest.NewRequestWithContext(
|
||||
t.Context(), http.MethodHead, encPhotoURL(t, h), nil))
|
||||
t.Logf("HEAD: %d, headers %v, %d body bytes",
|
||||
rec.Code, rec.Header(), rec.Body.Len())
|
||||
|
||||
if rec.Code != http.StatusOK {
|
||||
t.Fatalf("status = %d, want %d", rec.Code, http.StatusOK)
|
||||
}
|
||||
|
||||
for _, name := range []string{
|
||||
"Content-Type", "Content-Length", "Cache-Control", "ETag",
|
||||
} {
|
||||
if rec.Header().Get(name) == "" {
|
||||
t.Errorf("HEAD response has no %s", name)
|
||||
}
|
||||
}
|
||||
|
||||
if rec.Body.Len() != 0 {
|
||||
t.Errorf("HEAD response body has %d bytes, want none", rec.Body.Len())
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,131 @@
|
||||
package handlers
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"io"
|
||||
"log/slog"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/go-chi/chi/v5/middleware"
|
||||
|
||||
"sneak.berlin/go/pixa/internal/encurl"
|
||||
"sneak.berlin/go/pixa/internal/imgcache"
|
||||
)
|
||||
|
||||
// signedPhotoURL returns a signed /v1/image/ URL, valid for a minute, for the
|
||||
// JPEG at photoPath on signedHost at 50x50, made with h's image service.
|
||||
func signedPhotoURL(t *testing.T, h *Handlers) string {
|
||||
t.Helper()
|
||||
|
||||
signedURL, err := h.imgSvc.GenerateSignedURL("", &imgcache.ImageRequest{
|
||||
SourceHost: signedHost,
|
||||
SourcePath: photoPath,
|
||||
Size: imgcache.Size{Width: 50, Height: 50},
|
||||
Format: imgcache.FormatJPEG,
|
||||
}, time.Minute)
|
||||
if err != nil {
|
||||
t.Fatalf("GenerateSignedURL() error = %v", err)
|
||||
}
|
||||
|
||||
return signedURL
|
||||
}
|
||||
|
||||
// encPhotoURL returns an encrypted /v1/e/ URL, which never expires, for the
|
||||
// JPEG at photoPath on signedHost at 50x50, made with h's generator.
|
||||
func encPhotoURL(t *testing.T, h *Handlers) string {
|
||||
t.Helper()
|
||||
|
||||
token, err := h.encGen.Generate(&encurl.Payload{
|
||||
SourceHost: signedHost,
|
||||
SourcePath: photoPath,
|
||||
Width: 50,
|
||||
Height: 50,
|
||||
Format: imgcache.FormatJPEG,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("Generate() error = %v", err)
|
||||
}
|
||||
|
||||
return "/v1/e/" + token + "/img.jpg"
|
||||
}
|
||||
|
||||
// requestIDByMessage reads the JSON log lines in logs and returns the
|
||||
// request_id of each line, by its message.
|
||||
func requestIDByMessage(t *testing.T, logs io.Reader) map[string]string {
|
||||
t.Helper()
|
||||
|
||||
logged := make(map[string]string)
|
||||
|
||||
dec := json.NewDecoder(logs)
|
||||
for dec.More() {
|
||||
var line map[string]any
|
||||
|
||||
err := dec.Decode(&line)
|
||||
if err != nil {
|
||||
t.Fatalf("decoding log line: %v", err)
|
||||
}
|
||||
|
||||
msg, _ := line["msg"].(string)
|
||||
requestID, _ := line["request_id"].(string)
|
||||
logged[msg] = requestID
|
||||
}
|
||||
|
||||
return logged
|
||||
}
|
||||
|
||||
// TestImageLogLinesCarryRequestID verifies that the lines logged when an image
|
||||
// is fetched, converted and served through either image route carry the
|
||||
// request's ID as request_id, as the request log line does, so they can be
|
||||
// found from it.
|
||||
func TestImageLogLinesCarryRequestID(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
const requestID = "test-request-id"
|
||||
|
||||
imageURLs := map[string]func(*testing.T, *Handlers) string{
|
||||
"/v1/image/": signedPhotoURL,
|
||||
"/v1/e/": encPhotoURL,
|
||||
}
|
||||
|
||||
for route, imageURL := range imageURLs {
|
||||
t.Run(route, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
var logs bytes.Buffer
|
||||
|
||||
h, srv := newSignedHostServer(t,
|
||||
slog.New(slog.NewJSONHandler(&logs, nil)))
|
||||
|
||||
ctx := context.WithValue(t.Context(),
|
||||
middleware.RequestIDKey, requestID)
|
||||
rec := httptest.NewRecorder()
|
||||
srv.ServeHTTP(rec, httptest.NewRequestWithContext(
|
||||
ctx, http.MethodGet, imageURL(t, h), nil))
|
||||
|
||||
if rec.Code != http.StatusOK {
|
||||
t.Fatalf("status = %d, want %d", rec.Code, http.StatusOK)
|
||||
}
|
||||
|
||||
t.Logf("logged:\n%s", logs.String())
|
||||
|
||||
logged := requestIDByMessage(t, &logs)
|
||||
|
||||
for _, msg := range []string{
|
||||
"upstream fetched", "image converted", "image served",
|
||||
} {
|
||||
got, ok := logged[msg]
|
||||
if !ok {
|
||||
t.Errorf("no %q line logged", msg)
|
||||
} else if got != requestID {
|
||||
t.Errorf("%q line has request_id %q, want %q",
|
||||
msg, got, requestID)
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,90 @@
|
||||
package handlers
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"log/slog"
|
||||
"net/http"
|
||||
"testing"
|
||||
|
||||
"go.uber.org/fx/fxtest"
|
||||
"sneak.berlin/go/pixa/internal/config"
|
||||
"sneak.berlin/go/pixa/internal/globals"
|
||||
"sneak.berlin/go/pixa/internal/healthcheck"
|
||||
"sneak.berlin/go/pixa/internal/logger"
|
||||
)
|
||||
|
||||
// TestHandleRobotsTxt checks that /robots.txt asks every crawler to stay off
|
||||
// the whole site.
|
||||
func TestHandleRobotsTxt(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
h := &Handlers{log: slog.New(slog.DiscardHandler)}
|
||||
rec := sendGet(t, h.HandleRobotsTxt(), "/robots.txt")
|
||||
|
||||
if rec.Code != http.StatusOK {
|
||||
t.Errorf("status = %d, want %d", rec.Code, http.StatusOK)
|
||||
}
|
||||
|
||||
if ct := rec.Header().Get("Content-Type"); ct != "text/plain" {
|
||||
t.Errorf("Content-Type = %q, want text/plain", ct)
|
||||
}
|
||||
|
||||
want := "User-agent: *\nDisallow: /\n"
|
||||
if rec.Body.String() != want {
|
||||
t.Errorf("body = %q, want %q", rec.Body.String(), want)
|
||||
}
|
||||
}
|
||||
|
||||
// TestHandleHealthCheck checks that the health check answers 200 with status
|
||||
// ok, the app's name and version, now, uptime_seconds, uptime_human and
|
||||
// maintenance_mode, which is true here: the health check stays 200 while
|
||||
// maintenance mode is on.
|
||||
func TestHandleHealthCheck(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
lc := fxtest.NewLifecycle(t)
|
||||
|
||||
log, err := logger.New(lc, logger.Params{Globals: &globals.Globals{}})
|
||||
if err != nil {
|
||||
t.Fatalf("logger.New() error = %v", err)
|
||||
}
|
||||
|
||||
hc, err := healthcheck.New(lc, healthcheck.Params{
|
||||
Globals: &globals.Globals{Appname: "pixad", Version: "v1.2.3"},
|
||||
Config: &config.Config{MaintenanceMode: true},
|
||||
Logger: log,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("healthcheck.New() error = %v", err)
|
||||
}
|
||||
|
||||
h := &Handlers{hc: hc, log: slog.New(slog.DiscardHandler)}
|
||||
rec := sendGet(t, h.HandleHealthCheck(), "/.well-known/healthcheck.json")
|
||||
|
||||
if rec.Code != http.StatusOK {
|
||||
t.Errorf("status = %d, want %d", rec.Code, http.StatusOK)
|
||||
}
|
||||
|
||||
if ct := rec.Header().Get("Content-Type"); ct != "application/json" {
|
||||
t.Errorf("Content-Type = %q, want application/json", ct)
|
||||
}
|
||||
|
||||
var body map[string]any
|
||||
|
||||
err = json.NewDecoder(rec.Body).Decode(&body)
|
||||
if err != nil {
|
||||
t.Fatalf("decoding response body: %v", err)
|
||||
}
|
||||
|
||||
if body["status"] != "ok" || body["appname"] != "pixad" ||
|
||||
body["version"] != "v1.2.3" || body["maintenance_mode"] != true {
|
||||
t.Errorf("body = %v, want status ok, appname pixad, version v1.2.3 "+
|
||||
"and maintenance_mode true", body)
|
||||
}
|
||||
|
||||
for _, key := range []string{"now", "uptime_seconds", "uptime_human"} {
|
||||
if _, ok := body[key]; !ok {
|
||||
t.Errorf("body = %v, has no %s", body, key)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -18,6 +18,8 @@ import (
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/go-chi/chi/v5/middleware"
|
||||
)
|
||||
|
||||
// Fetcher configuration constants.
|
||||
@@ -267,6 +269,13 @@ func (f *HTTPFetcher) Fetch(ctx context.Context, url string) (*FetchResult, erro
|
||||
req.Header.Set("User-Agent", f.config.UserAgent)
|
||||
req.Header.Set("Accept", strings.Join(f.config.AllowedContentTypes, ", "))
|
||||
|
||||
// The ID of the request this fetch serves, so the fetch can be found in
|
||||
// the upstream host's logs
|
||||
requestID := middleware.GetReqID(ctx)
|
||||
if requestID != "" {
|
||||
req.Header.Set(middleware.RequestIDHeader, requestID)
|
||||
}
|
||||
|
||||
// Use httptrace to capture connection details
|
||||
var remoteAddr string
|
||||
|
||||
|
||||
@@ -0,0 +1,50 @@
|
||||
package httpfetcher
|
||||
|
||||
import (
|
||||
"context"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"testing"
|
||||
|
||||
"github.com/go-chi/chi/v5/middleware"
|
||||
)
|
||||
|
||||
// TestFetchSendsRequestID verifies that a fetch sends the ID of the request
|
||||
// it serves, which the RequestID middleware stores in the request context,
|
||||
// to the upstream host as X-Request-Id, so the fetch can be found in that
|
||||
// host's logs.
|
||||
func TestFetchSendsRequestID(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
const requestID = "test-request-id"
|
||||
|
||||
received := make(chan string, 1)
|
||||
|
||||
srv := httptest.NewServer(http.HandlerFunc(
|
||||
func(w http.ResponseWriter, r *http.Request) {
|
||||
received <- r.Header.Get("X-Request-Id")
|
||||
|
||||
w.Header().Set("Content-Type", contentTypeJPEG)
|
||||
_, _ = io.WriteString(w, imagePayload)
|
||||
}))
|
||||
t.Cleanup(srv.Close)
|
||||
|
||||
f, _ := newServerFetcher(t, srv, nil)
|
||||
|
||||
ctx := context.WithValue(testContext(t), middleware.RequestIDKey, requestID)
|
||||
|
||||
res, err := f.Fetch(ctx, upstreamURL("/image"))
|
||||
if err != nil {
|
||||
t.Fatalf("Fetch() error = %v", err)
|
||||
}
|
||||
|
||||
_ = res.Content.Close()
|
||||
|
||||
got := <-received
|
||||
t.Logf("upstream received X-Request-Id %q", got)
|
||||
|
||||
if got != requestID {
|
||||
t.Errorf("upstream X-Request-Id = %q, want %q", got, requestID)
|
||||
}
|
||||
}
|
||||
@@ -13,6 +13,7 @@ import (
|
||||
|
||||
"github.com/dustin/go-humanize"
|
||||
"github.com/getsentry/sentry-go"
|
||||
"github.com/go-chi/chi/v5/middleware"
|
||||
"golang.org/x/sync/singleflight"
|
||||
"sneak.berlin/go/pixa/internal/allowlist"
|
||||
"sneak.berlin/go/pixa/internal/httpfetcher"
|
||||
@@ -461,6 +462,7 @@ func (s *Service) fetchAndProcess(
|
||||
|
||||
// Log upstream fetch details
|
||||
s.log.Info("upstream fetched",
|
||||
"request_id", middleware.GetReqID(ctx),
|
||||
"host", req.SourceHost,
|
||||
"path", req.SourcePath,
|
||||
"bytes", fetchBytes,
|
||||
@@ -545,6 +547,7 @@ func (s *Service) processAndStore(
|
||||
}
|
||||
|
||||
s.log.Info("image converted",
|
||||
"request_id", middleware.GetReqID(ctx),
|
||||
"host", req.SourceHost,
|
||||
"path", req.SourcePath,
|
||||
"src_format", processResult.InputFormat,
|
||||
|
||||
@@ -2,9 +2,12 @@
|
||||
package middleware
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/rand"
|
||||
"log/slog"
|
||||
"net/http"
|
||||
"net/netip"
|
||||
"regexp"
|
||||
"time"
|
||||
|
||||
basicauth "github.com/99designs/basicauth-go"
|
||||
@@ -115,6 +118,32 @@ func (s *Middleware) RateLimit(
|
||||
})
|
||||
}
|
||||
|
||||
// requestIDPattern is what a request's own X-Request-Id must look like to be
|
||||
// kept as its ID: 1 to 64 letters, digits, '-', '_' or '.'.
|
||||
var requestIDPattern = regexp.MustCompile(`^[A-Za-z0-9._-]{1,64}$`)
|
||||
|
||||
// RequestID returns a middleware that gives each request an ID and sends it as
|
||||
// the X-Request-Id response header, so a client can quote it when reporting a
|
||||
// problem. The ID is the request's own X-Request-Id when that matches
|
||||
// requestIDPattern, and otherwise a random one, which tells nothing about the
|
||||
// machine or the traffic. It is stored in the request context under chi's
|
||||
// RequestIDKey, where the logging middleware, the handlers and the upstream
|
||||
// fetch read it.
|
||||
func (s *Middleware) RequestID() func(http.Handler) http.Handler {
|
||||
return func(next http.Handler) http.Handler {
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
id := r.Header.Get(middleware.RequestIDHeader)
|
||||
if !requestIDPattern.MatchString(id) {
|
||||
id = rand.Text()
|
||||
}
|
||||
|
||||
w.Header().Set(middleware.RequestIDHeader, id)
|
||||
ctx := context.WithValue(r.Context(), middleware.RequestIDKey, id)
|
||||
next.ServeHTTP(w, r.WithContext(ctx))
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
type loggingResponseWriter struct {
|
||||
http.ResponseWriter
|
||||
|
||||
|
||||
@@ -1,16 +1,11 @@
|
||||
package middleware
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"log/slog"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"net/url"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/prometheus/client_golang/prometheus/promhttp"
|
||||
|
||||
"sneak.berlin/go/pixa/internal/config"
|
||||
)
|
||||
|
||||
@@ -61,203 +56,6 @@ func TestCORSAnswersWithConfiguredOrigin(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// TestCORSAnswersPreflightWithConfiguredOrigin checks that the CORS
|
||||
// middleware answers a preflight request, which the CORS library handles
|
||||
// apart from other requests, the same way: "*" lets any origin read
|
||||
// responses and a single origin lets only that origin read them.
|
||||
func TestCORSAnswersPreflightWithConfiguredOrigin(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
const appOrigin = "https://app.example.com"
|
||||
|
||||
cases := []struct {
|
||||
configured string
|
||||
requestOrigin string
|
||||
want string
|
||||
}{
|
||||
{"*", "https://any.example.com", "*"},
|
||||
{appOrigin, appOrigin, appOrigin},
|
||||
{appOrigin, "https://other.example.com", ""},
|
||||
}
|
||||
|
||||
testHandler := http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
|
||||
w.WriteHeader(http.StatusOK)
|
||||
})
|
||||
|
||||
for _, tc := range cases {
|
||||
mw := &Middleware{
|
||||
log: slog.Default(),
|
||||
config: &config.Config{AccessControlAllowOrigin: tc.configured},
|
||||
}
|
||||
|
||||
handler := mw.CORS()(testHandler)
|
||||
|
||||
// An OPTIONS request naming the method it asks about is the
|
||||
// preflight a browser sends before some cross-origin requests.
|
||||
req := httptest.NewRequestWithContext(
|
||||
t.Context(), http.MethodOptions, "/v1/image/example.com/a.jpg/1x1.png", nil)
|
||||
req.Header.Set("Origin", tc.requestOrigin)
|
||||
req.Header.Set("Access-Control-Request-Method", http.MethodGet)
|
||||
|
||||
rec := httptest.NewRecorder()
|
||||
|
||||
handler.ServeHTTP(rec, req)
|
||||
|
||||
got := rec.Header().Get("Access-Control-Allow-Origin")
|
||||
if got != tc.want {
|
||||
t.Errorf("configured %q, preflight from %q: "+
|
||||
"Access-Control-Allow-Origin = %q, want %q",
|
||||
tc.configured, tc.requestOrigin, got, tc.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestMetricsAuthRequiresConfiguredCredentials checks that MetricsAuth on
|
||||
// its own answers 401 with a challenge to a request without credentials or
|
||||
// with a wrong username or password, and lets a request with the configured
|
||||
// username and password through. That the router puts it in front of
|
||||
// /metrics is not tested.
|
||||
func TestMetricsAuthRequiresConfiguredCredentials(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
const (
|
||||
username = "metricsuser"
|
||||
password = "metricspass"
|
||||
challenge = `Basic realm="metrics"`
|
||||
)
|
||||
|
||||
// An empty username stands for a request sent without credentials.
|
||||
cases := []struct {
|
||||
name string
|
||||
username string
|
||||
password string
|
||||
wantReached bool
|
||||
}{
|
||||
{"no credentials", "", "", false},
|
||||
{"wrong username", "someone", password, false},
|
||||
{"wrong password", username, "wrongpass", false},
|
||||
{"configured credentials", username, password, true},
|
||||
}
|
||||
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
mw := &Middleware{
|
||||
log: slog.Default(),
|
||||
config: &config.Config{
|
||||
MetricsUsername: username,
|
||||
MetricsPassword: password,
|
||||
},
|
||||
}
|
||||
|
||||
reached := false
|
||||
handler := mw.MetricsAuth()(http.HandlerFunc(
|
||||
func(http.ResponseWriter, *http.Request) {
|
||||
reached = true
|
||||
}))
|
||||
|
||||
req := httptest.NewRequestWithContext(
|
||||
t.Context(), http.MethodGet, "/metrics", nil)
|
||||
|
||||
if tc.username != "" {
|
||||
req.SetBasicAuth(tc.username, tc.password)
|
||||
}
|
||||
|
||||
rec := httptest.NewRecorder()
|
||||
|
||||
handler.ServeHTTP(rec, req)
|
||||
|
||||
if reached != tc.wantReached {
|
||||
t.Fatalf("request reached /metrics = %v, want %v",
|
||||
reached, tc.wantReached)
|
||||
}
|
||||
|
||||
if tc.wantReached {
|
||||
return
|
||||
}
|
||||
|
||||
if rec.Code != http.StatusUnauthorized {
|
||||
t.Errorf("status = %d, want %d",
|
||||
rec.Code, http.StatusUnauthorized)
|
||||
}
|
||||
|
||||
if got := rec.Header().Get("WWW-Authenticate"); got != challenge {
|
||||
t.Errorf("WWW-Authenticate = %q, want %q", got, challenge)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestMetricsRecordsServedRequest checks that the metrics middleware
|
||||
// records a request it served, so /metrics reports it. It is the only test
|
||||
// in this package that sets up the metrics middleware, which registers with
|
||||
// the process-wide Prometheus registry and can do so only once.
|
||||
func TestMetricsRecordsServedRequest(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
// The line /metrics shows once one GET /test has been served.
|
||||
const want = `http_request_duration_seconds_count{` +
|
||||
`code="200",handler="/test",method="GET",service=""} 1`
|
||||
|
||||
mw := &Middleware{log: slog.Default(), config: &config.Config{}}
|
||||
|
||||
handler := mw.Metrics()(http.HandlerFunc(
|
||||
func(w http.ResponseWriter, _ *http.Request) {
|
||||
w.WriteHeader(http.StatusOK)
|
||||
}))
|
||||
|
||||
handler.ServeHTTP(httptest.NewRecorder(), httptest.NewRequestWithContext(
|
||||
t.Context(), http.MethodGet, "/test", nil))
|
||||
|
||||
rec := httptest.NewRecorder()
|
||||
promhttp.Handler().ServeHTTP(rec, httptest.NewRequestWithContext(
|
||||
t.Context(), http.MethodGet, "/metrics", nil))
|
||||
|
||||
if !strings.Contains(rec.Body.String(), want) {
|
||||
t.Errorf("/metrics does not report the GET /test served; "+
|
||||
"want the line %q in:\n%s", want, rec.Body.String())
|
||||
}
|
||||
}
|
||||
|
||||
// TestLoggingLeavesOutSubmittedSigningKey checks that a login, a POST /
|
||||
// whose form carries the signing key, leaves no trace of the key in the
|
||||
// request's log line.
|
||||
func TestLoggingLeavesOutSubmittedSigningKey(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
const signingKey = "test-signing-key-0123456789abcdef"
|
||||
|
||||
var buf bytes.Buffer
|
||||
|
||||
mw := newTestMiddleware(t, &buf)
|
||||
|
||||
// The handler reads the key from the form, as the login handler does.
|
||||
handler := mw.Logging()(http.HandlerFunc(
|
||||
func(w http.ResponseWriter, r *http.Request) {
|
||||
if got := r.FormValue("key"); got != signingKey {
|
||||
t.Errorf("key in form = %q, want %q", got, signingKey)
|
||||
}
|
||||
|
||||
w.WriteHeader(http.StatusSeeOther)
|
||||
}))
|
||||
|
||||
form := url.Values{"key": {signingKey}}
|
||||
req := httptest.NewRequestWithContext(t.Context(), http.MethodPost, "/",
|
||||
strings.NewReader(form.Encode()))
|
||||
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
|
||||
|
||||
handler.ServeHTTP(httptest.NewRecorder(), req)
|
||||
|
||||
if !strings.Contains(buf.String(), `"method":"POST"`) {
|
||||
t.Fatalf("no log line for the request; got %q", buf.String())
|
||||
}
|
||||
|
||||
if strings.Contains(buf.String(), signingKey) {
|
||||
t.Errorf("log output contains the signing key; got %q", buf.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestSecurityHeaders(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
|
||||
@@ -0,0 +1,118 @@
|
||||
package middleware
|
||||
|
||||
import (
|
||||
"log/slog"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"os"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/go-chi/chi/v5/middleware"
|
||||
|
||||
"sneak.berlin/go/pixa/internal/config"
|
||||
)
|
||||
|
||||
// sendRequestID sends a request through the RequestID middleware, carrying
|
||||
// incoming as its X-Request-Id unless that is empty. It returns the ID the
|
||||
// next handler found in the request context, which the upstream fetch sends
|
||||
// and the log lines carry, and the X-Request-Id of the response.
|
||||
func sendRequestID(t *testing.T, incoming string) (string, string) {
|
||||
t.Helper()
|
||||
|
||||
mw := &Middleware{log: slog.Default(), config: &config.Config{}}
|
||||
|
||||
var inContext string
|
||||
|
||||
handler := mw.RequestID()(http.HandlerFunc(
|
||||
func(_ http.ResponseWriter, r *http.Request) {
|
||||
inContext = middleware.GetReqID(r.Context())
|
||||
}))
|
||||
|
||||
req := httptest.NewRequestWithContext(t.Context(), http.MethodGet, "/", nil)
|
||||
if incoming != "" {
|
||||
req.Header.Set("X-Request-Id", incoming)
|
||||
}
|
||||
|
||||
rec := httptest.NewRecorder()
|
||||
handler.ServeHTTP(rec, req)
|
||||
|
||||
return inContext, rec.Header().Get("X-Request-Id")
|
||||
}
|
||||
|
||||
// TestRequestIDKeepsShortPlainID verifies that a request's own X-Request-Id of
|
||||
// at most 64 letters, digits, '-', '_' or '.' is kept as its ID.
|
||||
func TestRequestIDKeepsShortPlainID(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
for _, incoming := range []string{
|
||||
"client-request-id",
|
||||
"A1_b2.c3-d4",
|
||||
strings.Repeat("a", 64),
|
||||
} {
|
||||
inContext, inResponse := sendRequestID(t, incoming)
|
||||
|
||||
if inContext != incoming || inResponse != incoming {
|
||||
t.Errorf("incoming %q: context has %q, response %q, want both %q",
|
||||
incoming, inContext, inResponse, incoming)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestRequestIDReplacesLongOrUnusualID verifies that a request's own
|
||||
// X-Request-Id that is over 64 characters or holds anything but letters,
|
||||
// digits, '-', '_' or '.' is neither sent back nor sent upstream: the request
|
||||
// gets a fresh ID instead.
|
||||
func TestRequestIDReplacesLongOrUnusualID(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
for _, incoming := range []string{
|
||||
strings.Repeat("a", 65),
|
||||
strings.Repeat("a", 9000),
|
||||
"has space",
|
||||
"a/b",
|
||||
"a,b",
|
||||
"<script>",
|
||||
"ünicode",
|
||||
} {
|
||||
inContext, inResponse := sendRequestID(t, incoming)
|
||||
t.Logf("incoming %.20q: made up %q", incoming, inResponse)
|
||||
|
||||
if inResponse == "" || inResponse == incoming {
|
||||
t.Errorf("incoming %.20q: response has %q, want a fresh ID",
|
||||
incoming, inResponse)
|
||||
}
|
||||
|
||||
if inContext != inResponse {
|
||||
t.Errorf("incoming %.20q: context has %q, want the response's %q",
|
||||
incoming, inContext, inResponse)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestRequestIDMadeUpTellsNothing verifies that the ID made up for a request
|
||||
// that sent none differs for every request and does not hold the host name.
|
||||
func TestRequestIDMadeUpTellsNothing(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
hostname, err := os.Hostname()
|
||||
if err != nil {
|
||||
t.Fatalf("os.Hostname() error = %v", err)
|
||||
}
|
||||
|
||||
firstInContext, first := sendRequestID(t, "")
|
||||
_, second := sendRequestID(t, "")
|
||||
t.Logf("host %q, made up %q and %q", hostname, first, second)
|
||||
|
||||
if first == "" || first == second {
|
||||
t.Errorf("made up %q and %q, want two different IDs", first, second)
|
||||
}
|
||||
|
||||
if firstInContext != first {
|
||||
t.Errorf("context has %q, want the response's %q", firstInContext, first)
|
||||
}
|
||||
|
||||
if strings.Contains(first, hostname) {
|
||||
t.Errorf("made-up ID %q holds the host name %q", first, hostname)
|
||||
}
|
||||
}
|
||||
@@ -1,32 +0,0 @@
|
||||
package server
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/prometheus/client_golang/prometheus/promhttp"
|
||||
)
|
||||
|
||||
// TestNoMetricsRecordedWithoutMetricsUsername checks that with no metrics
|
||||
// username set the router records nothing about the requests it serves.
|
||||
// /metrics is not served then, so the process-wide Prometheus registry is
|
||||
// read directly. TestMaintenanceModeKeepsOtherRoutes records into the same
|
||||
// registry, but never a GET /robots.txt.
|
||||
func TestNoMetricsRecordedWithoutMetricsUsername(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
s := newTestServer(t)
|
||||
s.ServeHTTP(httptest.NewRecorder(), httptest.NewRequestWithContext(
|
||||
t.Context(), http.MethodGet, "/robots.txt", nil))
|
||||
|
||||
rec := httptest.NewRecorder()
|
||||
promhttp.Handler().ServeHTTP(rec, httptest.NewRequestWithContext(
|
||||
t.Context(), http.MethodGet, "/metrics", nil))
|
||||
|
||||
if strings.Contains(rec.Body.String(), `handler="/robots.txt"`) {
|
||||
t.Errorf("with no metrics username GET /robots.txt was recorded:\n%s",
|
||||
rec.Body.String())
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,58 @@
|
||||
package server
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// requestIDHeader is the header that carries a request's ID.
|
||||
const requestIDHeader = "X-Request-Id"
|
||||
|
||||
// TestResponsesCarryRequestID verifies that every response, whatever its route
|
||||
// and status, carries the request's ID as X-Request-Id, so a client can quote
|
||||
// it when reporting a problem: one pixa made up when the request brought none,
|
||||
// and the request's own X-Request-Id when it brought one.
|
||||
func TestResponsesCarryRequestID(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
const clientRequestID = "client-request-id"
|
||||
|
||||
s := newTestServer(t)
|
||||
|
||||
paths := []string{
|
||||
"/robots.txt",
|
||||
"/no-such-path",
|
||||
unsignedImagePath,
|
||||
encryptedImagePath,
|
||||
}
|
||||
|
||||
for _, path := range paths {
|
||||
t.Run(path, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
rec := httptest.NewRecorder()
|
||||
s.ServeHTTP(rec, httptest.NewRequestWithContext(
|
||||
t.Context(), http.MethodGet, path, nil))
|
||||
t.Logf("status %d, %s %q",
|
||||
rec.Code, requestIDHeader, rec.Header().Get(requestIDHeader))
|
||||
|
||||
if rec.Header().Get(requestIDHeader) == "" {
|
||||
t.Errorf("response has no %s", requestIDHeader)
|
||||
}
|
||||
|
||||
req := httptest.NewRequestWithContext(
|
||||
t.Context(), http.MethodGet, path, nil)
|
||||
req.Header.Set(requestIDHeader, clientRequestID)
|
||||
|
||||
rec = httptest.NewRecorder()
|
||||
s.ServeHTTP(rec, req)
|
||||
|
||||
got := rec.Header().Get(requestIDHeader)
|
||||
if got != clientRequestID {
|
||||
t.Errorf("%s = %q, want the request's own %q",
|
||||
requestIDHeader, got, clientRequestID)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -28,7 +28,7 @@ func (s *Server) SetupRoutes() {
|
||||
s.router = chi.NewRouter()
|
||||
|
||||
s.router.Use(middleware.Recoverer)
|
||||
s.router.Use(middleware.RequestID)
|
||||
s.router.Use(s.mw.RequestID())
|
||||
s.router.Use(s.mw.ClientIP())
|
||||
s.router.Use(s.mw.SecurityHeaders())
|
||||
s.router.Use(s.mw.Logging())
|
||||
@@ -97,6 +97,7 @@ func (s *Server) SetupRoutes() {
|
||||
// The trailing filename (e.g., /img.jpg) is ignored but helps
|
||||
// browsers with content type
|
||||
r.Get("/e/{token}/*", s.h.HandleImageEnc())
|
||||
r.Head("/e/{token}/*", s.h.HandleImageEnc())
|
||||
})
|
||||
})
|
||||
|
||||
|
||||
@@ -0,0 +1,27 @@
|
||||
package server
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// TestEncryptedImageRouteAnswersHEAD verifies that HEAD on the encrypted image
|
||||
// route reaches its handler, as GET does, instead of being answered 405 Method
|
||||
// Not Allowed. The handler refuses a token it cannot decrypt with 400, so that
|
||||
// status shows the request got through.
|
||||
func TestEncryptedImageRouteAnswersHEAD(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
s := newTestServer(t)
|
||||
|
||||
rec := httptest.NewRecorder()
|
||||
s.ServeHTTP(rec, httptest.NewRequestWithContext(
|
||||
t.Context(), http.MethodHead, encryptedImagePath, nil))
|
||||
t.Logf("status %d", rec.Code)
|
||||
|
||||
if rec.Code != http.StatusBadRequest {
|
||||
t.Errorf("status = %d, want %d from the encrypted image handler",
|
||||
rec.Code, http.StatusBadRequest)
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user