Compare commits
5
Commits
4637026bc4
...
7f52b5bb3f
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
7f52b5bb3f | ||
|
|
a306160952 | ||
|
|
adfc5d4abe | ||
|
|
8432d99815 | ||
|
|
be6c715b36 |
@@ -34,6 +34,65 @@ else has a built-in default. A config file mounted at `/etc/pixa/config.yml`
|
|||||||
is optional: it is read when present, and an environment variable wins over
|
is optional: it is read when present, and an environment variable wins over
|
||||||
the same setting in it.
|
the same setting in it.
|
||||||
|
|
||||||
|
## Deployment
|
||||||
|
|
||||||
|
pixa listens on plain HTTP and runs behind a reverse proxy that terminates TLS.
|
||||||
|
[`configs/Caddyfile`](configs/Caddyfile) is an example for Caddy, chosen because
|
||||||
|
it is the smallest correct one: Caddy gets the TLS certificate itself and does
|
||||||
|
everything in this list without further settings. The reverse proxy must:
|
||||||
|
|
||||||
|
- terminate TLS, as the login and generator pages work only over HTTPS (see
|
||||||
|
Routes);
|
||||||
|
- pass the `Host`, `Origin` and `Referer` headers on unchanged, as pixa refuses
|
||||||
|
a form from those pages unless `Origin` or `Referer` names the host in `Host`,
|
||||||
|
and builds encrypted URLs from `Host`;
|
||||||
|
- set `X-Forwarded-For` to the client's address, with `trusted_proxies` set to
|
||||||
|
the address pixa sees the proxy's requests come from, so the login limit
|
||||||
|
counts each client by its own address (see `trusted_proxies` under
|
||||||
|
Configuration);
|
||||||
|
- wait for pixa's answer for at least `downstream_timeout` (default `60s`), the
|
||||||
|
longest pixa takes to fetch, convert and send an image.
|
||||||
|
|
||||||
|
It may also refuse `/metrics`, as the example does, so that only a scraper that
|
||||||
|
reaches pixa directly can read it; pixa itself asks for the metrics username and
|
||||||
|
password there.
|
||||||
|
|
||||||
|
pixa does the rest itself: it checks signatures and encrypted URLs, applies the
|
||||||
|
allowlist, refuses upstream hosts with private or local addresses, limits login
|
||||||
|
attempts, upstream response size and image dimensions, and sends the security
|
||||||
|
headers, `Strict-Transport-Security` included, with every response.
|
||||||
|
|
||||||
|
The state directory (`state_dir`, `/var/lib/pixa` in the container) holds the
|
||||||
|
database and the disk cache:
|
||||||
|
|
||||||
|
- It needs a persistent volume: without one, every restart starts with an empty
|
||||||
|
cache. In the container, the startup script gives the directory to the user
|
||||||
|
pixa runs as (uid 65532) and sets its mode to `750`; outside it, that user
|
||||||
|
must be able to write the directory.
|
||||||
|
- `cache_max_bytes` limits the source and transformed images together. The
|
||||||
|
database, the metadata files, the `.meta` file beside each transformed image
|
||||||
|
and files still being written come on top, and eviction runs in the
|
||||||
|
background, so the cache can pass the limit for a while: leave room on the
|
||||||
|
volume beyond it.
|
||||||
|
- Set `cache_max_bytes` for a lasting deployment. Its default is 75% of the
|
||||||
|
space free when pixa starts, which the cache's own files reduce, so a fuller
|
||||||
|
cache gives a smaller limit after a restart.
|
||||||
|
|
||||||
|
A load balancer's health check can request `/.well-known/healthcheck.json`,
|
||||||
|
which answers 200 whenever pixa is running, in maintenance mode too (see
|
||||||
|
`maintenance_mode`).
|
||||||
|
|
||||||
|
On SIGTERM or SIGINT pixa stops accepting connections, gives the requests in
|
||||||
|
progress and the images being processed 5 seconds to finish, and exits: with 0,
|
||||||
|
or with 1 when images were still being processed after those 5 seconds or
|
||||||
|
another part of pixa failed to stop. A request not finished by then is cut off.
|
||||||
|
`docker stop` waits 10 seconds before it kills the container.
|
||||||
|
|
||||||
|
Outside Docker, pixa needs libvips (the image has 8.15) and libheif to run, as
|
||||||
|
it uses libvips through CGO; building it also needs their development files,
|
||||||
|
`pkg-config` and a C compiler. `script/bootstrap` installs all of these with
|
||||||
|
nix, apt, brew or apk.
|
||||||
|
|
||||||
## Running under upaas
|
## Running under upaas
|
||||||
|
|
||||||
What the [upaas](https://git.eeqj.de/sneak/upaas) app for pixa needs:
|
What the [upaas](https://git.eeqj.de/sneak/upaas) app for pixa needs:
|
||||||
@@ -354,10 +413,10 @@ pixa finds: `/etc/pixa/config.yml`, `/etc/pixa/config.yaml`,
|
|||||||
`~/.config/pixa/config.yml`, `~/.config/pixa/config.yaml`, then `config.yml`
|
`~/.config/pixa/config.yml`, `~/.config/pixa/config.yaml`, then `config.yml`
|
||||||
and `config.yaml` in the working directory. A named file that does not exist,
|
and `config.yaml` in the working directory. A named file that does not exist,
|
||||||
cannot be read or does not parse aborts startup. Of the files pixa looks for on
|
cannot be read or does not parse aborts startup. Of the files pixa looks for on
|
||||||
its own, one it finds but cannot read or parse aborts startup; one it cannot
|
its own, only one that does not exist is passed over, without a message. One
|
||||||
find, for any reason, is passed over without a message, even when the file is
|
that pixa cannot read or parse aborts startup, naming the file. So does one in a
|
||||||
there in a directory pixa may not enter. With no file, pixa uses the environment
|
directory pixa may not enter, whether or not it is there, since pixa cannot
|
||||||
and the defaults.
|
tell. With no file, pixa uses the environment and the defaults.
|
||||||
|
|
||||||
| Variable | Config key | Meaning |
|
| Variable | Config key | Meaning |
|
||||||
| ------------------------------------ | ------------------------------- | ---------------------------------------------------------------------------- |
|
| ------------------------------------ | ------------------------------- | ---------------------------------------------------------------------------- |
|
||||||
|
|||||||
@@ -29,6 +29,20 @@ P2: security: referer blacklist
|
|||||||
|
|
||||||
# Completed Steps
|
# Completed Steps
|
||||||
|
|
||||||
|
- 2026-10-04 a config file pixa cannot read aborts startup (closes #176): of the
|
||||||
|
places pixa looks for its config file on its own, only one where the file does
|
||||||
|
not exist is passed over; any other error, such as a directory on the path
|
||||||
|
that pixa may not enter, aborts startup naming the file, as a file that does
|
||||||
|
not parse already did.
|
||||||
|
- 2026-10-04 deployment guide and example Caddy config (closes #89):
|
||||||
|
"Deployment" in `README.md` says what the reverse proxy in front of pixa must
|
||||||
|
do (terminate TLS; pass `Host`, `Origin` and `Referer` on unchanged; set
|
||||||
|
`X-Forwarded-For`, with `trusted_proxies` to match; wait at least
|
||||||
|
`downstream_timeout`; optionally refuse `/metrics`) and what pixa does itself,
|
||||||
|
that the state directory needs a persistent volume and what `cache_max_bytes`
|
||||||
|
counts, the health check for a load balancer, what a stop does and its exit
|
||||||
|
codes, and what running outside Docker needs; `configs/Caddyfile` is the
|
||||||
|
example, checked with `caddy validate`.
|
||||||
- 2026-10-04 the metrics basic auth, CORS preflight, request logging and
|
- 2026-10-04 the metrics basic auth, CORS preflight, request logging and
|
||||||
metrics recording have tests (closes #79): `MetricsAuth` on its own answers
|
metrics recording have tests (closes #79): `MetricsAuth` on its own answers
|
||||||
401 with a challenge without credentials or with a wrong username or password
|
401 with a challenge without credentials or with a wrong username or password
|
||||||
@@ -485,6 +499,3 @@ P2: security: referer blacklist
|
|||||||
- Prometheus performance metrics
|
- Prometheus performance metrics
|
||||||
- integration tests for the image proxy flow
|
- integration tests for the image proxy flow
|
||||||
- load tests to verify the 1k to 5k req/s target
|
- load tests to verify the 1k to 5k req/s target
|
||||||
- P2: documentation
|
|
||||||
- deployment guide
|
|
||||||
- example nginx or caddy reverse proxy config
|
|
||||||
|
|||||||
@@ -0,0 +1,17 @@
|
|||||||
|
# Example Caddy config for running pixa behind Caddy; see "Deployment" in
|
||||||
|
# README.md. Replace images.example.com with pixa's public host name, and
|
||||||
|
# 127.0.0.1:8080 with the address Caddy reaches pixa on.
|
||||||
|
#
|
||||||
|
# Caddy gets and renews the TLS certificate for the host name, passes the
|
||||||
|
# Host, Origin and Referer headers on unchanged, sets X-Forwarded-For to the
|
||||||
|
# client's address, and waits for pixa's answer with no time limit of its
|
||||||
|
# own, so pixa's downstream_timeout is what ends a slow request.
|
||||||
|
|
||||||
|
images.example.com
|
||||||
|
|
||||||
|
# pixa asks for metrics.username and metrics.password on /metrics. This
|
||||||
|
# line also keeps it off the public address, for a scraper that reaches
|
||||||
|
# pixa directly; remove it to read /metrics through Caddy.
|
||||||
|
respond /metrics 404
|
||||||
|
|
||||||
|
reverse_proxy 127.0.0.1:8080
|
||||||
+21
-11
@@ -4,6 +4,7 @@ package config
|
|||||||
import (
|
import (
|
||||||
"errors"
|
"errors"
|
||||||
"fmt"
|
"fmt"
|
||||||
|
"io/fs"
|
||||||
"log/slog"
|
"log/slog"
|
||||||
"math"
|
"math"
|
||||||
"net/netip"
|
"net/netip"
|
||||||
@@ -14,6 +15,7 @@ import (
|
|||||||
"sort"
|
"sort"
|
||||||
"strconv"
|
"strconv"
|
||||||
"strings"
|
"strings"
|
||||||
|
"syscall"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
"git.eeqj.de/sneak/smartconfig"
|
"git.eeqj.de/sneak/smartconfig"
|
||||||
@@ -778,19 +780,27 @@ func loadConfigFile(log *slog.Logger, appName string) (*smartconfig.Config, erro
|
|||||||
for _, path := range configPaths {
|
for _, path := range configPaths {
|
||||||
cleanPath := filepath.Clean(path)
|
cleanPath := filepath.Clean(path)
|
||||||
|
|
||||||
|
// Only a config file that does not exist is skipped, including
|
||||||
|
// one whose path runs through a file, such as under a HOME of
|
||||||
|
// /dev/null. One that cannot be read or does not parse is a
|
||||||
|
// fatal startup error.
|
||||||
_, statErr := os.Stat(cleanPath)
|
_, statErr := os.Stat(cleanPath)
|
||||||
if statErr == nil {
|
if errors.Is(statErr, fs.ErrNotExist) || errors.Is(statErr, syscall.ENOTDIR) {
|
||||||
// A config file that exists but does not parse is a fatal
|
continue
|
||||||
// startup error, never something to skip over.
|
|
||||||
sc, err := smartconfig.NewFromConfigPath(path)
|
|
||||||
if err != nil {
|
|
||||||
return nil, fmt.Errorf("failed to parse config file %s: %w", path, err)
|
|
||||||
}
|
|
||||||
|
|
||||||
log.Info("loaded config file", "path", path)
|
|
||||||
|
|
||||||
return sc, nil
|
|
||||||
}
|
}
|
||||||
|
|
||||||
|
if statErr != nil {
|
||||||
|
return nil, fmt.Errorf("failed to read config file %s: %w", path, statErr)
|
||||||
|
}
|
||||||
|
|
||||||
|
sc, err := smartconfig.NewFromConfigPath(path)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("failed to parse config file %s: %w", path, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
log.Info("loaded config file", "path", path)
|
||||||
|
|
||||||
|
return sc, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
return nil, nil //nolint:nilnil // nil config is valid (use defaults)
|
return nil, nil //nolint:nilnil // nil config is valid (use defaults)
|
||||||
|
|||||||
@@ -564,6 +564,116 @@ func TestMalformedConfigFileAbortsStartup(t *testing.T) {
|
|||||||
t.Logf("got expected error: %v", err)
|
t.Logf("got expected error: %v", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// TestConfigFileInDirectoryPixaMayNotEnterAbortsStartup checks that a
|
||||||
|
// config file pixa cannot read because it may not enter its directory
|
||||||
|
// aborts startup instead of being passed over.
|
||||||
|
func TestConfigFileInDirectoryPixaMayNotEnterAbortsStartup(t *testing.T) {
|
||||||
|
if os.Geteuid() == 0 {
|
||||||
|
t.Skip("root may enter any directory")
|
||||||
|
}
|
||||||
|
|
||||||
|
home := t.TempDir()
|
||||||
|
configDir := filepath.Join(home, ".config", "pixa-test-nonexistent-app")
|
||||||
|
configPath := filepath.Join(configDir, "config.yml")
|
||||||
|
|
||||||
|
err := os.MkdirAll(configDir, 0o700)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("failed to create config directory: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
err = os.WriteFile(configPath, []byte(signingKeyLine), 0o600)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("failed to write config: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
err = os.Chmod(configDir, 0)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("failed to remove the config directory's permissions: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Give the directory back its permissions so t.TempDir can remove it.
|
||||||
|
t.Cleanup(func() {
|
||||||
|
//nolint:gosec // G302: a directory needs its execute bit to be removed
|
||||||
|
_ = os.Chmod(configDir, 0o700)
|
||||||
|
})
|
||||||
|
|
||||||
|
// The ~/.config candidate is the only one that exists: the appname
|
||||||
|
// rules out /etc, and the working directory is empty.
|
||||||
|
t.Setenv("PIXA_CONFIG_PATH", "")
|
||||||
|
t.Setenv("HOME", home)
|
||||||
|
t.Chdir(t.TempDir())
|
||||||
|
|
||||||
|
log := slog.New(slog.DiscardHandler)
|
||||||
|
|
||||||
|
sc, err := loadConfigFile(log, "pixa-test-nonexistent-app")
|
||||||
|
if err == nil {
|
||||||
|
t.Fatalf("config file pixa cannot read must abort startup, got config: %v",
|
||||||
|
sc)
|
||||||
|
}
|
||||||
|
|
||||||
|
t.Logf("got expected error: %v", err)
|
||||||
|
|
||||||
|
if !strings.Contains(err.Error(), configPath) {
|
||||||
|
t.Errorf("error %q does not name the config file %s", err.Error(), configPath)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestConfigFileLinkingToItselfAbortsStartup checks that a config file
|
||||||
|
// pixa cannot read for a reason other than not existing aborts startup,
|
||||||
|
// as root too: a symbolic link to itself fails with "too many levels of
|
||||||
|
// symbolic links".
|
||||||
|
func TestConfigFileLinkingToItselfAbortsStartup(t *testing.T) {
|
||||||
|
workDir := t.TempDir()
|
||||||
|
|
||||||
|
err := os.Symlink("config.yml", filepath.Join(workDir, "config.yml"))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("failed to create symbolic link: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Only the working directory's config.yml is there: the appname rules
|
||||||
|
// out /etc, and HOME is empty.
|
||||||
|
t.Setenv("PIXA_CONFIG_PATH", "")
|
||||||
|
t.Setenv("HOME", t.TempDir())
|
||||||
|
t.Chdir(workDir)
|
||||||
|
|
||||||
|
log := slog.New(slog.DiscardHandler)
|
||||||
|
|
||||||
|
sc, err := loadConfigFile(log, "pixa-test-nonexistent-app")
|
||||||
|
if err == nil {
|
||||||
|
t.Fatalf("config file pixa cannot read must abort startup, got config: %v",
|
||||||
|
sc)
|
||||||
|
}
|
||||||
|
|
||||||
|
t.Logf("got expected error: %v", err)
|
||||||
|
|
||||||
|
if !strings.Contains(err.Error(), "config.yml") {
|
||||||
|
t.Errorf("error %q does not name the config file config.yml", err.Error())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestConfigPathThroughFileIsPassedOver checks that a config file path
|
||||||
|
// that runs through a file, such as one under a HOME of /dev/null, is
|
||||||
|
// passed over like one that does not exist, since no file can be there.
|
||||||
|
func TestConfigPathThroughFileIsPassedOver(t *testing.T) {
|
||||||
|
// No config file is there: the appname rules out /etc, HOME is
|
||||||
|
// /dev/null, and the working directory is empty.
|
||||||
|
t.Setenv("PIXA_CONFIG_PATH", "")
|
||||||
|
t.Setenv("HOME", os.DevNull)
|
||||||
|
t.Chdir(t.TempDir())
|
||||||
|
|
||||||
|
log := slog.New(slog.DiscardHandler)
|
||||||
|
|
||||||
|
sc, err := loadConfigFile(log, "pixa-test-nonexistent-app")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("a config path through a file must be passed over, got error: %v",
|
||||||
|
err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if sc != nil {
|
||||||
|
t.Errorf("expected no config file, got config: %v", sc)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func TestEnsureStateDirCreatesDirectory(t *testing.T) {
|
func TestEnsureStateDirCreatesDirectory(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user