diff --git a/Dockerfile b/Dockerfile index 46e3169..4112847 100644 --- a/Dockerfile +++ b/Dockerfile @@ -83,6 +83,14 @@ ENV XDG_DATA_HOME=/var/lib # XDG_DATA_HOME above. ENV GOMEMLIMIT=1536MiB +# Cap glibc's malloc arenas. The SQLite C library allocates and frees millions +# of small page-cache chunks from many threads; glibc otherwise creates up to +# eight arenas per core (hundreds on a large host) and keeps each arena's freed +# chunks resident, so process RSS climbs far above SQLite's live heap and never +# comes back down. Two arenas keep that retained memory bounded; database writes +# are already serialized, so the lost allocator concurrency costs nothing here. +ENV MALLOC_ARENA_MAX=2 + # Expose HTTP port EXPOSE 8080 diff --git a/README.md b/README.md index e856eb4..aaf5f9a 100644 --- a/README.md +++ b/README.md @@ -169,6 +169,7 @@ Configuration is handled via environment variables and OS-specific paths: | `PORT` | `8080` | HTTP server port | | `DEBUG` | (empty) | Set to `routewatch` for debug logging | | `GOMEMLIMIT` | `1536MiB` (in the Docker image) | Go soft memory limit; see Memory | +| `MALLOC_ARENA_MAX` | `2` (in the Docker image) | glibc malloc arena cap; see Memory | State directory (database location): - macOS: `~/Library/Application Support/routewatch/` @@ -183,6 +184,13 @@ Budget: - Go heap: a 1.5 GiB soft limit (`GOMEMLIMIT=1536MiB`, set in the image). - SQLite: at most 640 MiB of page cache across the connection pool (64 MiB per connection, 10 connections) and a 1.5 GiB hard heap limit for the C library. +- glibc allocator: the SQLite C library runs on glibc `malloc`, which frees + page-cache chunks back to per-arena free lists rather than to the kernel, so + process RSS tracks the high-water mark of those arenas, not SQLite's live + heap. glibc creates up to eight arenas per core, so on a many-core host the + retained memory — and thus RSS — grows with the core count. The image sets + `MALLOC_ARENA_MAX=2` to bound it; the two-arena cap costs nothing here because + database writes are already serialized. - About 0.2 GiB for everything else in the runtime. Run the container with a memory limit of 5 GiB and swap disabled: @@ -195,7 +203,9 @@ or the equivalent in your deployment tool. This leaves headroom above the ceilings for spikes and the kernel page cache. Override the Go soft limit by setting `GOMEMLIMIT` in the environment (for -example `-e GOMEMLIMIT=1GiB`); this replaces the image default. +example `-e GOMEMLIMIT=1GiB`); this replaces the image default. `MALLOC_ARENA_MAX` +can be overridden the same way, but raising it lets RSS climb again on a +many-core host. What happens at each limit: - Go soft limit: as the heap approaches `GOMEMLIMIT`, the runtime runs garbage