Sets MALLOC_ARENA_MAX=2 in the runtime image so process memory stops climbing
on the production feed.
After the earlier memory work (#3),
SQLite's live heap is bounded but RSS still grew about 70 MiB/min, nearly all
anonymous, outside both the Go runtime and SQLite's own accounting, with no SQLITE_NOMEM. Cause: glibc. SQLite frees millions of small page-cache chunks
from many threads; with arenas uncapped glibc makes up to eight per core
(hundreds on this 48-core host) and keeps each arena's freed chunks resident.
Capping to two arenas bounds the retained memory; writes are already serialized,
so the cap costs no throughput.
Only the Dockerfile and the README Memory section change; no code.
Verification (disclosure): two containers from the same 658aadb build on the
live feed for 36 minutes, identical but for MALLOC_ARENA_MAX. Non-Go RSS
(RssAnon minus the Go runtime's Sys minus HeapReleased): uncapped rose
531 to 1152 MiB and kept climbing with the database; the fixed image levelled at
196-285 MiB for the whole run, under 1.2 GiB with wide margin. Full figures on #23.
Disclosures:
Ran without --memory (host refuses it); RSS from /proc with a 6 GiB
watchdog, so behaviour at the container limit is untested.
The baseline arm reused the existing 658aadb image; the two arms differ only
by the arena env.
database is locked still occurs; cause noted on the issue, not fixed here per
its instruction.
Model: opus-4-8
Sets `MALLOC_ARENA_MAX=2` in the runtime image so process memory stops climbing
on the production feed.
After the earlier memory work (https://git.eeqj.de/sneak/routewatch/issues/3),
SQLite's live heap is bounded but RSS still grew about 70 MiB/min, nearly all
anonymous, outside both the Go runtime and SQLite's own accounting, with no
`SQLITE_NOMEM`. Cause: glibc. SQLite frees millions of small page-cache chunks
from many threads; with arenas uncapped glibc makes up to eight per core
(hundreds on this 48-core host) and keeps each arena's freed chunks resident.
Capping to two arenas bounds the retained memory; writes are already serialized,
so the cap costs no throughput.
Only the `Dockerfile` and the README Memory section change; no code.
Verification (disclosure): two containers from the same `658aadb` build on the
live feed for 36 minutes, identical but for `MALLOC_ARENA_MAX`. Non-Go RSS
(RssAnon minus the Go runtime's `Sys` minus `HeapReleased`): uncapped rose
531 to 1152 MiB and kept climbing with the database; the fixed image levelled at
196-285 MiB for the whole run, under 1.2 GiB with wide margin. Full figures on
https://git.eeqj.de/sneak/routewatch/issues/23.
Disclosures:
- Ran without `--memory` (host refuses it); RSS from `/proc` with a 6 GiB
watchdog, so behaviour at the container limit is untested.
- The baseline arm reused the existing `658aadb` image; the two arms differ only
by the arena env.
- `database is locked` still occurs; cause noted on the issue, not fixed here per
its instruction.
Model: opus-4-8
After the earlier memory work, SQLite's live heap is bounded but process
RSS still climbed about 70 MiB/min, nearly all anonymous and outside the
Go runtime and outside SQLite's own accounting, without any SQLITE_NOMEM.
The cause is glibc: the SQLite C library allocates and frees millions of
small page-cache chunks from many threads, and glibc keeps each arena's
freed chunks resident. With arenas uncapped it creates up to eight per
core, so on a large host RSS grows with the core count.
Set MALLOC_ARENA_MAX=2 in the image to bound the retained memory. Writes
are already serialized, so the two-arena cap costs no throughput. Update
the README Memory section.
Model: opus-4-8
PASS: MALLOC_ARENA_MAX=2 is set in the glibc runtime stage and reaches the routewatch process through the entrypoint's runuser, the diagnosis on #23 supports the fix, every changed README sentence holds, and the Docker build gate is green.
Model: opus-4-8
PASS: `MALLOC_ARENA_MAX=2` is set in the glibc runtime stage and reaches the routewatch process through the entrypoint's `runuser`, the diagnosis on https://git.eeqj.de/sneak/routewatch/issues/23 supports the fix, every changed README sentence holds, and the Docker build gate is green.
Model: opus-4-8
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Sets
MALLOC_ARENA_MAX=2in the runtime image so process memory stops climbingon the production feed.
After the earlier memory work (#3),
SQLite's live heap is bounded but RSS still grew about 70 MiB/min, nearly all
anonymous, outside both the Go runtime and SQLite's own accounting, with no
SQLITE_NOMEM. Cause: glibc. SQLite frees millions of small page-cache chunksfrom many threads; with arenas uncapped glibc makes up to eight per core
(hundreds on this 48-core host) and keeps each arena's freed chunks resident.
Capping to two arenas bounds the retained memory; writes are already serialized,
so the cap costs no throughput.
Only the
Dockerfileand the README Memory section change; no code.Verification (disclosure): two containers from the same
658aadbbuild on thelive feed for 36 minutes, identical but for
MALLOC_ARENA_MAX. Non-Go RSS(RssAnon minus the Go runtime's
SysminusHeapReleased): uncapped rose531 to 1152 MiB and kept climbing with the database; the fixed image levelled at
196-285 MiB for the whole run, under 1.2 GiB with wide margin. Full figures on
#23.
Disclosures:
--memory(host refuses it); RSS from/procwith a 6 GiBwatchdog, so behaviour at the container limit is untested.
658aadbimage; the two arms differ onlyby the arena env.
database is lockedstill occurs; cause noted on the issue, not fixed here perits instruction.
Model: opus-4-8
PASS:
MALLOC_ARENA_MAX=2is set in the glibc runtime stage and reaches the routewatch process through the entrypoint'srunuser, the diagnosis on #23 supports the fix, every changed README sentence holds, and the Docker build gate is green.Model: opus-4-8