Author SHA1 Message Date
clawbot 6ebb6ed618 Answer 415 for a source whose first bytes do not match its Content-Type (closes #253)
check / check (push) Waiting to run
A body whose first bytes are not those of the format its Content-Type
names answered 500 on both image routes, was logged as pixa's own
error, and was fetched again on every request. Both routes now answer
415 for each error the first-bytes check returns, and the image service
logs the refused source at warning level. The failure also gets a
negative cache entry, as an upstream error status does, so the source
URL is not fetched for 5 minutes and answers 502 meanwhile. README.md
says so.

TestService_Get_ConcurrentMissesShareAFailure still expects such a
source to be fetched again and fails until its case is changed.

Model: opus-5-5
2026-10-15 08:13:18 +00:00
clawbot 40387e90bb Test that a source not matching its Content-Type answers 415
Both image routes must answer 415 for an upstream body whose first
bytes are not those of the format its Content-Type names, log it as a
refused source and not at error level, and answer the next request for
that source 502 without fetching it again. The test fails until the
fix for #253.

Model: opus-5-5
2026-10-15 08:12:58 +00:00
clawbot 5d1e3ea871 Escape : and % in the signed and cached path and query (closes #247)
check / check (push) Waiting to run
The signed data and the cache key joined host, path and query with ":" and no escaping, so path /a:b with no query and path /a with query b: gave the same string: a URL signed for https://h/a:b verified for https://h/a?b:, and the two shared a cached image. Both now write each % in the path and the query as %25 and each : as %3A before joining them. The host never holds / and the path always starts with /, so the joined string reads back one way only. A path and query holding neither character are signed and keyed as before, so the existing known-answer vectors and the README example are unchanged; new vectors cover the escaped cases. Signature Specification in README.md states the escaping.

Model: opus-5-5
2026-10-15 09:37:01 +02:00
17 changed files with 464 additions and 521 deletions
+21 -20
View File
@@ -71,10 +71,9 @@ The state directory (`state_dir`, `/var/lib/pixa` in the container) holds the
database and the disk cache:
- It needs a persistent volume: without one, every restart starts with an empty
cache and forgets the latest logout, so the login sessions that logout ended
work again (see Encrypted URLs). In the container, the startup script gives
the directory to the user pixa runs as (uid 65532) and sets its mode to `750`;
outside it, that user must be able to write the directory.
cache. In the container, the startup script gives the directory to the user
pixa runs as (uid 65532) and sets its mode to `750`; outside it, that user
must be able to write the directory.
- `cache_max_bytes` limits the source and transformed images together. The
database, the metadata files, the `.meta` file beside each transformed image
and files still being written come on top, and eviction runs in the
@@ -185,10 +184,7 @@ path under `/v1/` answers 200, in maintenance mode too.
session it answers 303 to `/`. Answers: 200 with the page showing the URL; 400
with the page naming a field that is not valid; 500 when the URL cannot be
made.
- `GET /logout` — end every login session, in every browser (see Encrypted
URLs), and remove the session cookie. Needs: a login session; without one it
only removes the cookie. Answers: 303 to `/`; 500 when the logout cannot be
saved in the database.
- `GET /logout` — end the login session. Needs: nothing. Answers: 303 to `/`.
- `GET` or `HEAD` `/v1/image/<host>/<path>/<size>.<format>`, or
`/v1/image/<host>/<path>/<size>` with no format — an image, fetched, resized
and converted (below). Needs: a signature, unless the host is allowlisted (see
@@ -202,9 +198,11 @@ path under `/v1/` answers 200, in maintenance mode too.
redirects to, is `localhost`, ends in `.localhost` or `.local`, or has an
address in a blocked network (see `blocked_networks`); 429 over
`image_requests_per_minute` (below); 415 when the upstream answers with a
`Content-Type` that is not one of the source formats (below), or with an image
whose first bytes libvips reads as another format; 502 when the upstream
answered with an error status, and for 5 minutes after that for the same
`Content-Type` that is not one of the source formats (below), with a body
whose first bytes are not those of the format its `Content-Type` names, or
with an image whose first bytes libvips reads as another format; 502 when the
upstream answered with an error status, and for 5 minutes after that, or after
a body whose first bytes are not those of its `Content-Type`, for the same
source URL; 503 when pixa is busy or in maintenance mode; 500 for any other
failure.
- `GET` or `HEAD` `/v1/e/<token>/<name>` — an image through an encrypted URL
@@ -283,8 +281,12 @@ above 8192.
The source image may be JPEG, PNG, GIF, WebP, AVIF or JPEG XL. The upstream's
`Content-Type` must name its format, and the image's first bytes must match it.
An upstream answering with any other `Content-Type`, SVG included, is refused
with 415, and so is an image whose first bytes libvips reads as another format,
such as an SVG whose first bytes pass the check for AVIF.
with 415. So is a body whose first bytes are not those of the format its
`Content-Type` names, such as a JPEG sent as `image/png`, text sent as an image,
or an empty body. pixa then does not fetch that source URL again for 5 minutes,
and answers 502 for it meanwhile, as after an upstream error status. An image
whose first bytes libvips reads as another format, such as an SVG whose first
bytes pass the check for AVIF, is refused with 415 too.
With the format `auto`, pixa chooses the format for each request from its
`Accept` header, in this order:
@@ -356,11 +358,7 @@ nor change what it asks for.
1. Open `/` in a browser over HTTPS (or over plain HTTP while `debug` is on, see
Routes) and log in with the signing key (`signing_key`). The login session
lasts 30 days, or until someone logs out at `/logout`. Every login is made
with the one signing key, so logging out ends every login session, in every
browser: pixa saves the time of the latest logout in its database and refuses
a login session made before it, a copy of the session cookie included, also
after a restart that keeps the state directory (see Deployment).
lasts 30 days, or until `/logout`.
2. On the generator page, give the source image's URL, the width and height, the
format, quality and fit, and how long the URL lasts, then submit the form
(`POST /generate`). The format is JPEG XL unless another is chosen; a form
@@ -427,8 +425,11 @@ HMAC-SHA256(secret, "host:path:query:width:height:format:expiration:quality:fit"
Where:
- `host` — source origin hostname (e.g. `cdn.example.com`)
- `path` — source path (e.g. `/photos/cat.jpg`)
- `query` — source query string, empty string if none
- `path` — source path (e.g. `/photos/cat.jpg`), with each `%` written as `%25`
and each `:` as `%3A`, so `/photos/a:b.jpg` is signed as `/photos/a%3Ab.jpg`
- `query` — source query string, empty string if none, with `%` and `:` written
as in `path`; neither then holds a `:`, so path `/a:b` with no query and path
`/a` with query `b:` are signed differently
- `width` — requested width in pixels, `0` for original
- `height` — requested height in pixels, `0` for original
- `format` — output format, one of those listed under Routes, with `original`
+17 -12
View File
@@ -30,18 +30,23 @@ P2: security: per-origin rate limiting
# Completed Steps
- 2026-10-15 logging out ends every login session (closes #256): a login session
was a cookie with nothing kept on the server, so `/logout` removed it from one
browser and a copy of it kept working for its 30 days. Every login is made
with the one signing key, so `/logout` now ends them all: the session manager
keeps the time of the latest logout, saves it in the new `logout` table so
that it outlasts a restart, and refuses a session made before it. A restart
without a persistent volume for the state directory forgets it. A `/logout`
without a valid login session, such as one another site makes a browser send
without the `SameSite=Strict` cookie, ends none and only removes the cookie. A
logout that cannot be saved answers 500. `session.NewManager`, which the tests
use, keeps logouts in memory only; pixad uses
`session.NewManagerWithDatabase`.
- 2026-10-15 a source whose first bytes do not match its `Content-Type` answers
415 (closes #253): a body whose first bytes are not those of the format its
`Content-Type` names, such as a JPEG sent as `image/png`, text, or a body
under 12 bytes, answered 500 on both image routes and was logged as an error
of pixa's, and each request for it fetched the whole body again. Both routes
now answer 415, and the image service logs it at warning level as a refused
source. It also gets a negative cache entry, as an upstream error status does,
so the source URL is not fetched again for 5 minutes and answers 502
meanwhile.
- 2026-10-15 a `:` in the source path or query no longer moves where the path
ends and the query begins (closes #247): the signed data and the cache key
joined host, path and query with `:` and no escaping, so a URL signed for
`https://h/a:b` verified for `https://h/a?b:`; the two also shared a cache
key, and whichever was fetched first was served for both. In both, the path
and the query now have each `%` written as `%25` and each `:` as `%3A`; a path
and query holding neither are signed and keyed as before. Signature
Specification in `README.md` says so.
- 2026-10-15 the example config allowlists no host (closes #249): it allowlisted
`github.com` and `user-images.githubusercontent.com`, where anyone can put an
image, so a deployment set up as Getting Started says served those images to
-7
View File
@@ -176,10 +176,3 @@ CREATE TABLE IF NOT EXISTS cache_stats (
last_updated_at DATETIME DEFAULT CURRENT_TIMESTAMP
);
INSERT OR IGNORE INTO cache_stats (id) VALUES (1);
-- Time of the latest logout. A login session made before it is refused,
-- so logging out ends every login session. No row: nobody has logged out.
CREATE TABLE IF NOT EXISTS logout (
id INTEGER PRIMARY KEY CHECK (id = 1),
logged_out_at DATETIME NOT NULL
);
+2 -10
View File
@@ -89,18 +89,10 @@ func (s *Handlers) handleLoginPost(w http.ResponseWriter, r *http.Request) {
http.Redirect(w, r, "/", http.StatusSeeOther)
}
// HandleLogout logs out, which ends every login session (see
// session.Manager.Logout), and redirects to login.
// HandleLogout clears the session and redirects to login.
func (s *Handlers) HandleLogout() http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
err := s.sessMgr.Logout(w, r)
if err != nil {
s.log.Error("failed to log out", "error", err)
http.Error(w, "Failed to log out", http.StatusInternalServerError)
return
}
s.sessMgr.ClearSession(w)
http.Redirect(w, r, "/", http.StatusSeeOther)
}
}
@@ -185,98 +185,6 @@ func TestHandleLogout_ClearsSessionCookie(t *testing.T) {
}
}
// TestHandleLogout_EndsCopiedSession verifies that after GET /logout a copy
// of the session cookie taken before it shows the login page, not the
// generator page.
func TestHandleLogout_EndsCopiedSession(t *testing.T) {
t.Parallel()
h, srv := newCSRFTestRouter(t)
copied := newSessionCookie(t, h)
req := httptest.NewRequestWithContext(
t.Context(), http.MethodGet, "/logout", nil)
req.AddCookie(copied)
rec := httptest.NewRecorder()
h.HandleLogout().ServeHTTP(rec, req)
if rec.Code != http.StatusSeeOther {
t.Fatalf("GET /logout status = %d, want %d",
rec.Code, http.StatusSeeOther)
}
req = httptest.NewRequestWithContext(t.Context(), http.MethodGet, "/", nil)
req.AddCookie(copied)
rec = httptest.NewRecorder()
srv.ServeHTTP(rec, req)
body := rec.Body.String()
t.Logf("GET / with the copied cookie: status %d, login page %v",
rec.Code, strings.Contains(body, loginForm))
if !strings.Contains(body, loginForm) || strings.Contains(body, generatorForm) {
t.Error("a cookie copied before /logout still opens the generator page")
}
}
// TestHandleLogout_SaveFails_Answers500 verifies that when the logout cannot
// be saved in the database, GET /logout answers 500, leaves the session
// cookie in place and ends no session.
func TestHandleLogout_SaveFails_Answers500(t *testing.T) {
t.Parallel()
h, srv := newCSRFTestRouter(t)
db := setupTestDB(t)
sessMgr, err := session.NewManagerWithDatabase(
t.Context(), testSigningKey, db)
if err != nil {
t.Fatalf("session.NewManagerWithDatabase() error = %v", err)
}
h.sessMgr = sessMgr
cookie := newSessionCookie(t, h)
// Saving the logout fails on a closed database.
err = db.Close()
if err != nil {
t.Fatalf("closing the database: %v", err)
}
req := httptest.NewRequestWithContext(
t.Context(), http.MethodGet, "/logout", nil)
req.AddCookie(cookie)
rec := httptest.NewRecorder()
h.HandleLogout().ServeHTTP(rec, req)
t.Logf("GET /logout: status %d, Set-Cookie %q",
rec.Code, rec.Header().Get("Set-Cookie"))
if rec.Code != http.StatusInternalServerError {
t.Errorf("GET /logout status = %d, want %d",
rec.Code, http.StatusInternalServerError)
}
if c := findSessionCookie(rec); c != nil {
t.Errorf("a logout that failed set the session cookie: %s", c)
}
req = httptest.NewRequestWithContext(t.Context(), http.MethodGet, "/", nil)
req.AddCookie(cookie)
rec = httptest.NewRecorder()
srv.ServeHTTP(rec, req)
if !strings.Contains(rec.Body.String(), generatorForm) {
t.Error("a logout that failed ended the session")
}
}
// TestGeneratePost_NoSession_RedirectsToLogin verifies that POST /generate
// with a valid CSRF token but no login session answers 303 to / and makes no
// URL.
@@ -0,0 +1,180 @@
package handlers
import (
"bytes"
"encoding/json"
"image/color"
"log/slog"
"net/http"
"slices"
"testing"
"testing/fstest"
"time"
"github.com/go-chi/chi/v5"
"sneak.berlin/go/pixa/internal/encurl"
"sneak.berlin/go/pixa/internal/httpfetcher"
"sneak.berlin/go/pixa/internal/imgcache"
)
// mismatchedSourcePath is the path of the source that
// TestFirstBytesNotMatchingContentType serves on allowlistedHost. The mock
// fetcher names a file's Content-Type from its extension, so the source is
// sent as image/png.
const mismatchedSourcePath = "/images/source.png"
// newRoutesFetchingThrough returns both image routes, and the Handlers behind
// them, whose image service fetches through fetcher and has allowlistedHost on
// its allowlist. The handlers and the image service log to log.
func newRoutesFetchingThrough(
t *testing.T, fetcher httpfetcher.Fetcher, log *slog.Logger,
) (*Handlers, http.Handler) {
t.Helper()
cache, err := imgcache.NewCache(setupTestDB(t), imgcache.CacheConfig{
StateDir: t.TempDir(),
NegativeTTL: 5 * time.Minute,
})
if err != nil {
t.Fatalf("imgcache.NewCache() error = %v", err)
}
svc, err := imgcache.NewService(&imgcache.ServiceConfig{
Cache: cache,
Fetcher: fetcher,
SigningKey: testSigningKey,
Allowlist: []string{allowlistedHost},
Logger: log,
})
if err != nil {
t.Fatalf("imgcache.NewService() error = %v", err)
}
encGen, err := encurl.NewGenerator(testSigningKey)
if err != nil {
t.Fatalf("encurl.NewGenerator() error = %v", err)
}
h := &Handlers{log: log, imgSvc: svc, encGen: encGen}
r := chi.NewRouter()
r.Get("/v1/image/*", h.HandleImage())
r.Get("/v1/e/{token}/*", h.HandleImageEnc())
return h, r
}
// mismatchedSourcePlainURL returns the /v1/image/ URL of mismatchedSourcePath
// on allowlistedHost at 50x50 as a JPEG.
func mismatchedSourcePlainURL(*testing.T, *Handlers) string {
return "/v1/image/" + allowlistedHost + mismatchedSourcePath + "/50x50.jpeg"
}
// mismatchedSourceEncURL returns an encrypted /v1/e/ URL, made with h's
// generator, of mismatchedSourcePath on allowlistedHost at 50x50 as a JPEG.
func mismatchedSourceEncURL(t *testing.T, h *Handlers) string {
t.Helper()
token, err := h.encGen.Generate(&encurl.Payload{
SourceHost: allowlistedHost,
SourcePath: mismatchedSourcePath,
Width: 50,
Height: 50,
Format: imgcache.FormatJPEG,
})
if err != nil {
t.Fatalf("Generate() error = %v", err)
}
return "/v1/e/" + token + "/img.jpg"
}
// loggedLevels reads the JSON log lines in logs and returns the messages
// logged at each level.
func loggedLevels(t *testing.T, logs *bytes.Buffer) map[string][]string {
t.Helper()
t.Logf("logged:\n%s", logs.String())
messages := make(map[string][]string)
dec := json.NewDecoder(logs)
for dec.More() {
var line struct {
Level string `json:"level"`
Msg string `json:"msg"`
}
err := dec.Decode(&line)
if err != nil {
t.Fatalf("decoding log line: %v", err)
}
messages[line.Level] = append(messages[line.Level], line.Msg)
}
return messages
}
// TestFirstBytesNotMatchingContentType checks that both image routes answer
// 415 for an upstream body whose first bytes are not those of the format its
// Content-Type names, and log it as a refused source, not as an error of
// pixa's. The next request for that source must answer 502 without fetching
// it again, as after an upstream error status.
func TestFirstBytesNotMatchingContentType(t *testing.T) {
t.Parallel()
bodies := map[string][]byte{
"JPEG sent as image/png": generateTestJPEG(t, 10, 10, color.Black),
"text sent as image/png": []byte("This is just text, not a PNG"),
"body too short to name a format": []byte("short"),
}
// The source's URL on each image route.
sourceURLs := map[string]func(*testing.T, *Handlers) string{
"plain URL": mismatchedSourcePlainURL,
"encrypted URL": mismatchedSourceEncURL,
}
for bodyName, body := range bodies {
for urlName, sourceURL := range sourceURLs {
t.Run(urlName+", "+bodyName, func(t *testing.T) {
t.Parallel()
fetcher := &countingFetcher{
Fetcher: httpfetcher.NewMock(fstest.MapFS{
allowlistedHost + mismatchedSourcePath: &fstest.MapFile{
Data: body,
},
}),
}
var logs bytes.Buffer
h, routes := newRoutesFetchingThrough(t, fetcher,
slog.New(slog.NewJSONHandler(&logs, nil)))
target := sourceURL(t, h)
checkErrorBody(t, sendGet(t, routes, target),
http.StatusUnsupportedMediaType, "unsupported source format")
levels := loggedLevels(t, &logs)
if len(levels[slog.LevelError.String()]) != 0 {
t.Errorf("logged at error level: %q",
levels[slog.LevelError.String()])
}
if !slices.Contains(levels[slog.LevelWarn.String()], "source refused") {
t.Error(`no "source refused" line logged at warning level`)
}
checkErrorBody(t, sendGet(t, routes, target),
http.StatusBadGateway, "upstream error")
if n := fetcher.fetches.Load(); n != 1 {
t.Errorf("upstream fetched %d times, want 1", n)
}
})
}
}
}
+1 -3
View File
@@ -163,9 +163,7 @@ func (s *Handlers) initImageService() error {
// Initialize session manager (signing key is validated at config load
// time). Session cookies are always Secure/HttpOnly/SameSite=Strict.
// Logouts are saved in the database, so they outlast a restart.
sessMgr, err := session.NewManagerWithDatabase(
context.Background(), s.config.SigningKey, s.db.DB())
sessMgr, err := session.NewManager(s.config.SigningKey)
if err != nil {
return err
}
+13
View File
@@ -15,6 +15,7 @@ import (
"sneak.berlin/go/pixa/internal/httpfetcher"
"sneak.berlin/go/pixa/internal/imageprocessor"
"sneak.berlin/go/pixa/internal/imgcache"
"sneak.berlin/go/pixa/internal/magic"
)
// HandleImage handles the main image proxy route:
@@ -220,6 +221,18 @@ func (s *Handlers) HandleTooManyRequests() http.HandlerFunc {
func (s *Handlers) respondImageError(
w http.ResponseWriter, req *imgcache.ImageRequest, err error,
) {
// An upstream body whose first bytes are not those of the format its
// Content-Type names is the upstream's error, not pixa's: the image
// service has logged it as a refused source.
if errors.Is(err, magic.ErrMagicByteMismatch) ||
errors.Is(err, magic.ErrUnknownFormat) ||
errors.Is(err, magic.ErrNotEnoughData) {
s.respondError(w, "unsupported source format",
http.StatusUnsupportedMediaType)
return
}
s.log.Error("failed to get image",
"host", req.SourceHost,
"path", req.SourcePath,
+5 -1
View File
@@ -15,6 +15,7 @@ import (
"sneak.berlin/go/pixa/internal/httpfetcher"
"sneak.berlin/go/pixa/internal/imageprocessor"
"sneak.berlin/go/pixa/internal/imgcache"
"sneak.berlin/go/pixa/internal/magic"
)
// HandleImageEnc handles requests to /v1/e/{token}/* for encrypted
@@ -162,7 +163,10 @@ func (s *Handlers) handleImageError(w http.ResponseWriter, err error) {
s.respondError(w, "server busy, try again later",
http.StatusServiceUnavailable)
case errors.Is(err, httpfetcher.ErrInvalidContentType),
errors.Is(err, imageprocessor.ErrUnsupportedInputFormat):
errors.Is(err, imageprocessor.ErrUnsupportedInputFormat),
errors.Is(err, magic.ErrMagicByteMismatch),
errors.Is(err, magic.ErrUnknownFormat),
errors.Is(err, magic.ErrNotEnoughData):
s.respondError(w, "unsupported source format",
http.StatusUnsupportedMediaType)
case errors.Is(err, imageprocessor.ErrOutputTooLarge):
+25 -10
View File
@@ -438,13 +438,7 @@ func (s *Service) fetchAndProcess(
if err != nil {
// Store negative cache for certain errors
if isNegativeCacheable(err) {
statusCode := extractStatusCode(err)
storeErr := s.cache.StoreNegative(ctx, req, statusCode, err.Error())
if storeErr != nil {
s.log.Warn("failed to store negative cache entry",
"host", req.SourceHost, "path", req.SourcePath, "error", storeErr)
}
s.storeNegative(ctx, req, extractStatusCode(err), err)
}
return nil, fmt.Errorf("upstream fetch failed: %w", err)
@@ -490,9 +484,16 @@ func (s *Service) fetchAndProcess(
"cipher", fetchResult.TLSCipherSuite,
)
// Validate magic bytes match content type
// Validate magic bytes match content type. A body that does not is the
// upstream's error, so it gets a negative cache entry, as an upstream
// error status does.
err = magic.ValidateMagicBytes(sourceData, fetchResult.ContentType)
if err != nil {
s.log.Warn("source refused", "request_id", middleware.GetReqID(ctx),
"host", req.SourceHost, "path", req.SourcePath,
"content_type", fetchResult.ContentType, "error", err)
s.storeNegative(ctx, req, httpStatusUnsupportedMediaType, err)
return nil, fmt.Errorf("content validation failed: %w", err)
}
@@ -508,6 +509,19 @@ func (s *Service) fetchAndProcess(
)
}
// storeNegative stores a negative cache entry for req's source, whose fetch
// failed with err, so the source is not fetched again until the entry
// expires. A failure to store the entry is only logged.
func (s *Service) storeNegative(
ctx context.Context, req *ImageRequest, statusCode int, err error,
) {
storeErr := s.cache.StoreNegative(ctx, req, statusCode, err.Error())
if storeErr != nil {
s.log.Warn("failed to store negative cache entry",
"host", req.SourceHost, "path", req.SourcePath, "error", storeErr)
}
}
// processedVariant is a variant as processAndStore made it. Each request that
// shared its processing serves it through a reader of its own.
type processedVariant struct {
@@ -616,8 +630,9 @@ func signatureRequest(req *ImageRequest) *signature.Request {
// HTTP status codes for error responses.
const (
httpStatusBadGateway = 502
httpStatusInternalError = 500
httpStatusBadGateway = 502
httpStatusInternalError = 500
httpStatusUnsupportedMediaType = 415
)
// isNegativeCacheable returns true if the error should be cached.
+13 -2
View File
@@ -10,6 +10,7 @@ import (
"log/slog"
"os"
"path/filepath"
"strings"
"time"
)
@@ -371,11 +372,13 @@ func HashPath(path string) PathHash {
// CacheKey generates a unique key for a request variant.
// Format: sha256(host:path:query:width:height:format:quality:fit_mode)
// The path and query go through escapePercentAndColon, so that a ":" in
// either cannot be taken for the one between them.
func CacheKey(req *ImageRequest) VariantKey {
data := fmt.Sprintf("%s:%s:%s:%d:%d:%s:%d:%s",
req.SourceHost,
req.SourcePath,
req.SourceQuery,
escapePercentAndColon(req.SourcePath),
escapePercentAndColon(req.SourceQuery),
req.Size.Width,
req.Size.Height,
req.Format,
@@ -387,6 +390,14 @@ func CacheKey(req *ImageRequest) VariantKey {
return VariantKey(hex.EncodeToString(h[:]))
}
// escapePercentAndColon returns s with each "%" written as "%25" and each
// ":" as "%3A", so that the result holds no ":".
func escapePercentAndColon(s string) string {
s = strings.ReplaceAll(s, "%", "%25")
return strings.ReplaceAll(s, ":", "%3A")
}
// VariantStorage handles key-based file storage for processed image variants.
// Files are stored at: <basedir>/<ab>/<cd>/<cache_key>
// Metadata is stored at: <basedir>/<ab>/<cd>/<cache_key>.meta
@@ -407,6 +407,62 @@ func TestCacheKey(t *testing.T) {
}
}
// TestCacheKey_ColonOrPercentInPathOrQuery checks that two different source
// URLs whose path or query holds ":" or "%" get different keys.
func TestCacheKey_ColonOrPercentInPathOrQuery(t *testing.T) {
t.Parallel()
cases := []struct {
name string
path1, query1 string
path2, query2 string
}{
{
name: "colon moved from path to query",
path1: "/a:b", query1: "",
path2: "/a", query2: "b:",
},
{
name: "colon in path and its percent-encoding",
path1: "/a:b", query1: "",
path2: "/a%3Ab", query2: "",
},
}
for _, tt := range cases {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
key1 := CacheKey(&ImageRequest{
SourceHost: testHostCDN,
SourcePath: tt.path1,
SourceQuery: tt.query1,
Size: Size{Width: 800, Height: 600},
Format: FormatWebP,
Quality: 85,
FitMode: FitCover,
})
key2 := CacheKey(&ImageRequest{
SourceHost: testHostCDN,
SourcePath: tt.path2,
SourceQuery: tt.query2,
Size: Size{Width: 800, Height: 600},
Format: FormatWebP,
Quality: 85,
FitMode: FitCover,
})
t.Logf("CacheKey() of path %q query %q = %s", tt.path1, tt.query1, key1)
t.Logf("CacheKey() of path %q query %q = %s", tt.path2, tt.query2, key2)
if key1 == key2 {
t.Errorf("CacheKey() gave path %q query %q and path %q query %q "+
"the same key", tt.path1, tt.query1, tt.path2, tt.query2)
}
})
}
}
// TestVariantStorage_StoreLogsFailedMetaWrite verifies that a .meta write
// that fails is logged at warn and the store still succeeds.
func TestVariantStorage_StoreLogsFailedMetaWrite(t *testing.T) {
-112
View File
@@ -1,112 +0,0 @@
package server
import (
"net/http"
"net/http/httptest"
"strings"
"testing"
"go.uber.org/fx"
"go.uber.org/fx/fxtest"
"sneak.berlin/go/pixa/internal/config"
"sneak.berlin/go/pixa/internal/database"
"sneak.berlin/go/pixa/internal/globals"
"sneak.berlin/go/pixa/internal/handlers"
"sneak.berlin/go/pixa/internal/healthcheck"
"sneak.berlin/go/pixa/internal/logger"
"sneak.berlin/go/pixa/internal/middleware"
"sneak.berlin/go/pixa/internal/session"
)
// startPixa starts the components pixad's fx app builds, from cfg, and
// returns the server with its routes, as newTestServer does.
func startPixa(t *testing.T, cfg *config.Config) *Server {
t.Helper()
var (
h *handlers.Handlers
mw *middleware.Middleware
)
app := fxtest.New(t,
fx.Supply(cfg),
fx.Provide(
globals.New,
logger.New,
database.New,
healthcheck.New,
handlers.New,
middleware.New,
),
fx.Populate(&h, &mw),
)
app.RequireStart()
t.Cleanup(app.RequireStop)
s := &Server{config: cfg, mw: mw, h: h}
s.SetupRoutes()
return s
}
// opensGeneratorPage reports whether GET / with cookie shows s's generator
// page, which only a valid login session opens.
func opensGeneratorPage(t *testing.T, s *Server, cookie *http.Cookie) bool {
t.Helper()
req := httptest.NewRequestWithContext(t.Context(), http.MethodGet, "/", nil)
req.AddCookie(cookie)
rec := httptest.NewRecorder()
s.ServeHTTP(rec, req)
return rec.Code == http.StatusOK &&
strings.Contains(rec.Body.String(), `action="/generate"`)
}
// TestLogoutOutlastsRestart verifies that a copy of the session cookie taken
// before /logout is still refused by a pixa started again on the same state
// directory, as pixad is after a restart.
func TestLogoutOutlastsRestart(t *testing.T) {
t.Parallel()
s := newTestServer(t)
login := postLogin(t, s, firstClient, "", testSigningKey)
var copied *http.Cookie
for _, c := range login.Result().Cookies() {
if c.Name == session.CookieName {
copied = c
}
}
if copied == nil || !opensGeneratorPage(t, s, copied) {
t.Fatal("logging in gave no session cookie that opens the generator page")
}
req := httptest.NewRequestWithContext(
t.Context(), http.MethodGet, "/logout", nil)
req.AddCookie(copied)
logout := httptest.NewRecorder()
s.ServeHTTP(logout, req)
if logout.Code != http.StatusSeeOther {
t.Fatalf("GET /logout status = %d, want %d",
logout.Code, http.StatusSeeOther)
}
restarted := startPixa(t, s.config)
opens := opensGeneratorPage(t, restarted, copied)
t.Logf("after the restart, the copied cookie opens the generator page: %v",
opens)
if opens {
t.Error("after a restart, a cookie copied before /logout " +
"still opens the generator page")
}
}
+1 -97
View File
@@ -1,15 +1,9 @@
// Package session provides encrypted session cookie management, and keeps the
// time of the latest logout in the database so that a logout ends every
// session made before it.
// Package session provides encrypted session cookie management.
package session
import (
"context"
"database/sql"
"errors"
"fmt"
"net/http"
"sync"
"time"
"github.com/gorilla/securecookie"
@@ -41,25 +35,11 @@ type Data struct {
}
// Manager handles session creation and validation using encrypted cookies.
//
// Every login is made with the one signing key, so a logout ends every
// session made before it, in every browser. The Manager keeps the time of
// the latest logout in memory, so checking a session reads no database; a
// Manager from NewManagerWithDatabase also saves it in the database, so that
// it outlasts a restart.
type Manager struct {
sc *securecookie.SecureCookie
db *sql.DB
// mu guards latestLogout, and is held while a logout is saved, so that
// the database and latestLogout agree.
mu sync.Mutex
latestLogout time.Time
}
// NewManager creates a session manager with keys derived from the signing key.
// It keeps logouts in memory only, so a restart forgets them;
// NewManagerWithDatabase keeps them in the database.
//
// Session cookies always carry the Secure, HttpOnly, and SameSite=Strict
// attributes; this cannot be configured. Browsers treat http://localhost as a
@@ -87,29 +67,6 @@ func NewManager(signingKey string) (*Manager, error) {
}, nil
}
// NewManagerWithDatabase creates a session manager like NewManager that saves
// each logout in db, which must have pixa's schema, and starts from the
// latest logout saved there.
func NewManagerWithDatabase(
ctx context.Context, signingKey string, db *sql.DB,
) (*Manager, error) {
m, err := NewManager(signingKey)
if err != nil {
return nil, err
}
err = db.QueryRowContext(ctx,
"SELECT logged_out_at FROM logout WHERE id = 1",
).Scan(&m.latestLogout)
if err != nil && !errors.Is(err, sql.ErrNoRows) {
return nil, fmt.Errorf("failed to read the latest logout: %w", err)
}
m.db = db
return m, nil
}
// CreateSession creates a new authenticated session and sets the cookie.
func (m *Manager) CreateSession(w http.ResponseWriter) error {
now := time.Now()
@@ -165,31 +122,9 @@ func (m *Manager) ValidateSession(r *http.Request) (*Data, error) {
return nil, ErrInvalidSession
}
// A session made before the latest logout has ended.
if !data.CreatedAt.After(m.latestLogoutTime()) {
return nil, ErrInvalidSession
}
return &data, nil
}
// Logout removes the session cookie and, when r has a valid session, ends
// every session made until now. A request without one ends no session:
// another site can make a browser request /logout, but the browser leaves
// the SameSite=Strict session cookie out of that request.
func (m *Manager) Logout(w http.ResponseWriter, r *http.Request) error {
if m.IsAuthenticated(r) {
err := m.saveLogout(r.Context())
if err != nil {
return err
}
}
m.ClearSession(w)
return nil
}
// ClearSession removes the session cookie.
func (m *Manager) ClearSession(w http.ResponseWriter) {
http.SetCookie(w, &http.Cookie{
@@ -210,34 +145,3 @@ func (m *Manager) IsAuthenticated(r *http.Request) bool {
return err == nil && data != nil && data.Authenticated
}
// saveLogout makes now the time of the latest logout, saving it in the
// database first when the Manager has one.
func (m *Manager) saveLogout(ctx context.Context) error {
m.mu.Lock()
defer m.mu.Unlock()
now := time.Now().UTC()
if m.db != nil {
_, err := m.db.ExecContext(ctx,
"INSERT OR REPLACE INTO logout (id, logged_out_at) VALUES (1, ?)",
now)
if err != nil {
return fmt.Errorf("failed to save the logout: %w", err)
}
}
m.latestLogout = now
return nil
}
// latestLogoutTime returns the time of the latest logout, or the zero time
// when there has been none.
func (m *Manager) latestLogoutTime() time.Time {
m.mu.Lock()
defer m.mu.Unlock()
return m.latestLogout
}
-149
View File
@@ -1,149 +0,0 @@
package session_test
import (
"database/sql"
"errors"
"net/http"
"net/http/httptest"
"testing"
"sneak.berlin/go/pixa/internal/database"
"sneak.berlin/go/pixa/internal/session"
)
const logoutTestSigningKey = "test-signing-key-12345"
// newTestDatabase opens an empty in-memory database with pixa's schema. It
// has one connection, as pixad's has: each connection to ":memory:" would
// open a database of its own.
func newTestDatabase(t *testing.T) *sql.DB {
t.Helper()
db, err := sql.Open("sqlite", ":memory:")
if err != nil {
t.Fatalf("sql.Open() error = %v", err)
}
db.SetMaxOpenConns(1)
t.Cleanup(func() { _ = db.Close() })
err = database.ApplyMigrations(t.Context(), db, nil)
if err != nil {
t.Fatalf("ApplyMigrations() error = %v", err)
}
return db
}
// createSessionCookie logs in with mgr and returns the session cookie it
// sets.
func createSessionCookie(t *testing.T, mgr *session.Manager) *http.Cookie {
t.Helper()
rec := httptest.NewRecorder()
err := mgr.CreateSession(rec)
if err != nil {
t.Fatalf("CreateSession() error = %v", err)
}
for _, c := range rec.Result().Cookies() {
if c.Name == session.CookieName {
return c
}
}
t.Fatalf("CreateSession() did not set a %q cookie", session.CookieName)
return nil
}
// requestWithCookie returns a GET request carrying cookie, or no cookie when
// cookie is nil.
func requestWithCookie(t *testing.T, cookie *http.Cookie) *http.Request {
t.Helper()
req := httptest.NewRequestWithContext(t.Context(), http.MethodGet, "/", nil)
if cookie != nil {
req.AddCookie(cookie)
}
return req
}
// TestLogout_EndsEverySessionMadeBeforeIt verifies that after a logout a
// copy of the session cookie taken before it is refused and a login after it
// works, and that both still hold for a new Manager on the same database, as
// after a restart.
func TestLogout_EndsEverySessionMadeBeforeIt(t *testing.T) {
t.Parallel()
db := newTestDatabase(t)
mgr, err := session.NewManagerWithDatabase(
t.Context(), logoutTestSigningKey, db)
if err != nil {
t.Fatalf("NewManagerWithDatabase() error = %v", err)
}
copied := createSessionCookie(t, mgr)
err = mgr.Logout(httptest.NewRecorder(), requestWithCookie(t, copied))
if err != nil {
t.Fatalf("Logout() error = %v", err)
}
_, err = mgr.ValidateSession(requestWithCookie(t, copied))
t.Logf("cookie copied before the logout: ValidateSession() error = %v", err)
if !errors.Is(err, session.ErrInvalidSession) {
t.Errorf("cookie copied before the logout: error = %v, want %v",
err, session.ErrInvalidSession)
}
loginAfter := createSessionCookie(t, mgr)
_, err = mgr.ValidateSession(requestWithCookie(t, loginAfter))
if err != nil {
t.Errorf("login after the logout: ValidateSession() error = %v", err)
}
restarted, err := session.NewManagerWithDatabase(
t.Context(), logoutTestSigningKey, db)
if err != nil {
t.Fatalf("NewManagerWithDatabase() after restart error = %v", err)
}
if restarted.IsAuthenticated(requestWithCookie(t, copied)) {
t.Error("after a restart, a cookie copied before the logout is accepted")
}
if !restarted.IsAuthenticated(requestWithCookie(t, loginAfter)) {
t.Error("after a restart, a login made after the logout is refused")
}
}
// TestLogout_WithoutSession_EndsNoSession verifies that a logout from a
// request without a session cookie ends no session. Another site can make a
// browser request /logout, but the browser leaves the SameSite=Strict
// session cookie out of that request.
func TestLogout_WithoutSession_EndsNoSession(t *testing.T) {
t.Parallel()
mgr, err := session.NewManagerWithDatabase(
t.Context(), logoutTestSigningKey, newTestDatabase(t))
if err != nil {
t.Fatalf("NewManagerWithDatabase() error = %v", err)
}
cookie := createSessionCookie(t, mgr)
err = mgr.Logout(httptest.NewRecorder(), requestWithCookie(t, nil))
if err != nil {
t.Fatalf("Logout() error = %v", err)
}
if !mgr.IsAuthenticated(requestWithCookie(t, cookie)) {
t.Error("a logout without a session cookie ended a session")
}
}
+113
View File
@@ -0,0 +1,113 @@
package signature_test
import (
"errors"
"testing"
"time"
"sneak.berlin/go/pixa/internal/signature"
)
// TestSigner_Verify_ColonCannotMovePathQueryBoundary checks that a URL
// signed for path "/a:b" with no query does not verify for path "/a" with
// query "b:", a different resource on the same host.
func TestSigner_Verify_ColonCannotMovePathQueryBoundary(t *testing.T) {
t.Parallel()
signer := signature.New(goldenSigningKey)
signed := &signature.Request{
SourceHost: testHost,
SourcePath: "/a:b",
Width: 800,
Height: 600,
Format: testFormatWebP,
Quality: 85,
FitMode: testFitCover,
Expires: time.Now().Add(1 * time.Hour),
}
signed.Signature = signer.Sign(signed)
replayed := *signed
replayed.SourcePath = "/a"
replayed.SourceQuery = "b:"
t.Logf("signature for path %q query %q: %s",
signed.SourcePath, signed.SourceQuery, signed.Signature)
t.Logf("signature for path %q query %q: %s",
replayed.SourcePath, replayed.SourceQuery, signer.Sign(&replayed))
err := signer.Verify(&replayed)
if !errors.Is(err, signature.ErrInvalid) {
t.Errorf("Verify() of path %q query %q with the signature for "+
"path %q = %v, want %v",
replayed.SourcePath, replayed.SourceQuery, signed.SourcePath,
err, signature.ErrInvalid)
}
}
// TestSigner_EscapedGoldenVectors pins the signature of requests whose path
// or query holds ":" or "%", which are signed with "%" written as "%25" and
// ":" as "%3A". The expected values were computed once, outside pixa, from
// the signed data in each comment.
func TestSigner_EscapedGoldenVectors(t *testing.T) {
t.Parallel()
signer := signature.New(goldenSigningKey)
vectors := []struct {
name string
path string
query string
wantSignature string
}{
{
name: "colon in path",
path: "/photos/a:b.jpg",
// Signed data:
// "cdn.example.com:/photos/a%3Ab.jpg::800:600:webp:1704067200:85:cover"
wantSignature: "WnEpZ7bQp1Zvw8WDatlj2vu-4wYKWE3BFBizV8m__6k=",
},
{
name: "colon in query",
path: testPath,
query: "t=1:2",
// Signed data:
// "cdn.example.com:/photos/cat.jpg:t=1%3A2:800:600:webp:1704067200:85:cover"
wantSignature: "85kRoN8hhr4cCUkEUjNvUsvVYKjYYfWdeffx5TjJjqA=",
},
{
name: "percent in path and query",
path: "/photos/a%20b.jpg",
query: "q=50%",
// Signed data:
// "cdn.example.com:/photos/a%2520b.jpg:q=50%25:800:600:webp:1704067200:85:cover"
wantSignature: "MjDiiRdgXMrsjqPMsENtD4aG1hp1e-a6zy67vKTX4CQ=",
},
}
for _, tt := range vectors {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
req := &signature.Request{
SourceHost: testHost,
SourcePath: tt.path,
SourceQuery: tt.query,
Width: 800,
Height: 600,
Format: testFormatWebP,
Quality: 85,
FitMode: testFitCover,
Expires: time.Unix(goldenExpiresUnix, 0),
}
gotSignature := signer.Sign(req)
t.Logf("Sign() of path %q query %q = %s", tt.path, tt.query, gotSignature)
if gotSignature != tt.wantSignature {
t.Errorf("Sign() = %q, want %q", gotSignature, tt.wantSignature)
}
})
}
}
+17 -6
View File
@@ -10,6 +10,7 @@ import (
"fmt"
"net/url"
"strconv"
"strings"
"time"
)
@@ -154,15 +155,17 @@ func (s *Signer) GenerateSignedURL(
// buildSignatureData creates the string to be signed.
// Format: "host:path:query:width:height:format:expiration:quality:fit"
// All components are used verbatim (exact match). No normalization,
// suffix matching, or wildcard expansion is performed. Quality and fit
// are the effective transform values, so replaying a signed URL with a
// different quality or fit mode fails verification.
// The path and query go through escapePercentAndColon, so that a ":" in
// either cannot be taken for the one between them; every other component
// is used verbatim. No normalization, suffix matching, or wildcard
// expansion is performed. Quality and fit are the effective transform
// values, so replaying a signed URL with a different quality or fit mode
// fails verification.
func (s *Signer) buildSignatureData(req *Request) string {
return fmt.Sprintf("%s:%s:%s:%d:%d:%s:%d:%d:%s",
req.SourceHost,
req.SourcePath,
req.SourceQuery,
escapePercentAndColon(req.SourcePath),
escapePercentAndColon(req.SourceQuery),
req.Width,
req.Height,
req.Format,
@@ -172,6 +175,14 @@ func (s *Signer) buildSignatureData(req *Request) string {
)
}
// escapePercentAndColon returns s with each "%" written as "%25" and each
// ":" as "%3A", so that the result holds no ":".
func escapePercentAndColon(s string) string {
s = strings.ReplaceAll(s, "%", "%25")
return strings.ReplaceAll(s, ":", "%3A")
}
// ParseParams extracts signature and expiration from query parameters.
func ParseParams(sig, expStr string) (string, time.Time, error) {
if expStr == "" {