sneak
ce924fd762
make release builds, smoke-tests and pushes amd64 (on fsn1app1 via DOCKER_HOST) and arm64 (local), joins them into latest and dated multi-arch manifests, and commits the day's logs. Logic lives in bin/; the Makefile only wires dependencies so -j2 runs the arches in parallel.
195 lines
8.5 KiB
Markdown
195 lines
8.5 KiB
Markdown
# baseimage
|
|
|
|
A fat Docker base image that trades size for build speed. Everything a
|
|
typical Go, Node, Python, or Hugo project needs is preinstalled, so child
|
|
images only have to copy their source and build, instead of spending the
|
|
first few minutes of every build on `apt-get install` and toolchain
|
|
downloads.
|
|
|
|
Image: `git.eeqj.de/sneak/baseimage`, built for `linux/amd64` and
|
|
`linux/arm64`.
|
|
|
|
## What is in it
|
|
|
|
| Component | Source | Location |
|
|
| --------------- | ------------------------------ | ------------------------------ |
|
|
| Ubuntu 26.04 | `ubuntu:26.04`, fully upgraded | `/` |
|
|
| init | `runsvinit` (built from Go) | `/usr/local/sbin/runsvinit` |
|
|
| services | `runit` (apt) | `/etc/sv`, `/etc/service` |
|
|
| build-essential | apt | `gcc`, `make`, `pkg-config` |
|
|
| Go (distro) | apt `golang-go` | `/usr/lib/go-*` |
|
|
| Go (latest) | go.dev tarball | `/usr/local/go` |
|
|
| golangci-lint | GitHub release | `/usr/local/bin/golangci-lint` |
|
|
| Hugo (extended) | GitHub release `.deb` | `/usr/local/bin/hugo` |
|
|
| Node (distro) | apt `nodejs`, `npm` | `/usr/bin/node` |
|
|
| nvm | git tag | `/usr/local/nvm` |
|
|
| Node (LTS) | `nvm install --lts` | `/usr/local/nvm/current` |
|
|
| corepack | `npm install -g corepack` | nvm LTS global bin |
|
|
| yarn, pnpm | corepack, cache pre-populated | `/usr/local/corepack` |
|
|
| wrangler | `npm install -g wrangler` | nvm LTS global bin |
|
|
| Python (distro) | apt `python3`, `pip`, `venv` | `/usr/bin/python3` |
|
|
| pyenv | git tag | `/usr/local/pyenv` |
|
|
| Python (latest) | `pyenv install` | `/usr/local/pyenv/versions/*` |
|
|
| virtualenv | apt, and pip in pyenv Python | `virtualenv` on `PATH` |
|
|
|
|
Plus the usual odds and ends: `git`, `curl`, `wget`, `jq`, `rsync`,
|
|
`openssh-client`, `zip`/`unzip`, `xz-utils`, `bzip2`, and the `-dev`
|
|
headers needed to compile CPython and common native extensions.
|
|
|
|
The "latest" toolchains win on `PATH`. `go`, `node`, `npm`, `yarn`,
|
|
`python3`, and `pip` all resolve to the manually installed current
|
|
versions; the distro packages are there as a fallback and to satisfy apt
|
|
dependencies in child images.
|
|
|
|
`PATH` order:
|
|
|
|
```
|
|
/usr/local/go/bin
|
|
/root/go/bin
|
|
/usr/local/nvm/current/bin
|
|
/usr/local/pyenv/shims
|
|
/usr/local/pyenv/bin
|
|
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
|
|
```
|
|
|
|
Environment variables set in the image: `NVM_DIR`, `NVM_SYMLINK_CURRENT`,
|
|
`COREPACK_HOME`, `COREPACK_ENABLE_DOWNLOAD_PROMPT=0`,
|
|
`COREPACK_ENABLE_AUTO_PIN=0`, `PYENV_ROOT`, `GOPATH`, `LANG=C.UTF-8`.
|
|
|
|
### Yarn and corepack
|
|
|
|
`yarn` is provided by corepack, which is how the Yarn project ships Yarn
|
|
4. Corepack reads the `packageManager` field in a project's `package.json`
|
|
and runs exactly that version. The corepack cache in `/usr/local/corepack`
|
|
is pre-populated with current Yarn stable (the default when a project does
|
|
not pin one), Yarn 1 classic, and current pnpm, so projects pinning any of
|
|
those do not download anything at run time. A project pinning some other
|
|
version will be fetched on first use.
|
|
|
|
### Pinned versions
|
|
|
|
Only the sources that have no "latest" lookup are pinned, as build args in
|
|
the `Dockerfile`:
|
|
|
|
| Arg | What |
|
|
| ------------------ | -------------------------- |
|
|
| `RUNSVINIT_COMMIT` | `peterbourgon/runsvinit` |
|
|
| `NVM_GIT_TAG` | `nvm-sh/nvm` release tag |
|
|
| `PYENV_GIT_TAG` | `pyenv/pyenv` release tag |
|
|
|
|
Everything else resolves to its current release at build time: Ubuntu
|
|
packages, Go, Node LTS, Python, Hugo, golangci-lint, npm, yarn, pnpm,
|
|
wrangler.
|
|
|
|
## Services and init
|
|
|
|
PID 1 is [`runsvinit`](https://github.com/peterbourgon/runsvinit), a small
|
|
Go program written for exactly this job. It starts `runsvdir` on
|
|
`/etc/service`, reaps orphaned zombies, and on `SIGTERM` or `SIGINT` runs
|
|
`sv stop` on every service before letting `runsvdir` exit. `docker stop`
|
|
therefore stops services gracefully. The project is archived upstream, so
|
|
it is built from a pinned commit at image build time.
|
|
|
|
To add a service in a child image, create a runit service directory and
|
|
link it into `/etc/service`:
|
|
|
|
```dockerfile
|
|
FROM git.eeqj.de/sneak/baseimage:latest
|
|
COPY sv/myapp/run /etc/sv/myapp/run
|
|
RUN chmod +x /etc/sv/myapp/run && ln -s /etc/sv/myapp /etc/service/myapp
|
|
```
|
|
|
|
A minimal `run` script:
|
|
|
|
```sh
|
|
#!/bin/sh
|
|
exec 2>&1
|
|
exec /usr/local/bin/myapp
|
|
```
|
|
|
|
Services should log to stdout/stderr; the container's log is the combined
|
|
output of all services.
|
|
|
|
The image sets `CMD ["/usr/local/sbin/runsvinit"]` and no `ENTRYPOINT`, so
|
|
a child image that only needs one unsupervised process can override `CMD`
|
|
and ignore runit entirely, and `docker run -it image bash` works as
|
|
expected.
|
|
|
|
## Cleanup
|
|
|
|
Installation leaves nothing behind that production does not need:
|
|
|
|
- apt lists and cache are removed, and `dpkg` is configured (see
|
|
`rootfs/etc/dpkg/dpkg.cfg.d/01_nodoc`) to exclude `/usr/share/doc`
|
|
(except copyright files), `/usr/share/man`, `/usr/share/info`, and
|
|
locales other than English. Package installs in child images inherit
|
|
that config.
|
|
- npm, nvm, pip, and Go build/module caches are emptied.
|
|
- Python test suites and the Go toolchain's test tree are stripped.
|
|
- `/root/go` is removed; it is recreated on first use.
|
|
|
|
Development headers (`-dev` packages) are deliberately kept: they are
|
|
what child images need to build things.
|
|
|
|
## Building and releasing
|
|
|
|
```sh
|
|
make release # build, test, push both arches, tag, commit logs
|
|
make -j2 release # same, with the two arches in parallel
|
|
make build # build for this machine's arch only
|
|
make test # smoke-test this machine's build
|
|
```
|
|
|
|
Each architecture is built natively on a daemon of that architecture: arm64
|
|
on the local daemon, amd64 on `fsn1app1` via `DOCKER_HOST`. The daemon per
|
|
arch is set by `DOCKER_HOST_amd64` and `DOCKER_HOST_arm64` in the
|
|
`Makefile` and can be overridden on the command line. The release date
|
|
defaults to today and can be overridden with `DATE=YYYY-MM-DD`.
|
|
|
|
`make release` runs, per arch, `bin/build`, `bin/test`, and `bin/push`,
|
|
then `bin/manifest` to join the per-arch tags into `latest` and the dated
|
|
tag, then `bin/commit-logs` to commit `logs/YYYY-MM-DD/` and push the repo.
|
|
`bin/test` checks that the toolchains resolve, yarn and pnpm work offline,
|
|
nothing is left in `/tmp`, and a runit service is stopped cleanly by
|
|
`docker stop` with exit code 0.
|
|
|
|
Registry login is done once on the local docker client with
|
|
`docker login git.eeqj.de`; the client forwards the credentials to the
|
|
remote daemon on push.
|
|
|
|
Tags:
|
|
|
|
| Tag | Meaning |
|
|
| -------------------- | ----------------------------------------- |
|
|
| `latest` | multi-arch manifest of the newest release |
|
|
| `YYYY-MM-DD` | multi-arch manifest of that day's release |
|
|
| `amd64`, `arm64` | per-arch images behind the newest release |
|
|
|
|
The full `--progress=plain` build log for each arch of each release is
|
|
committed under `logs/YYYY-MM-DD/`.
|
|
|
|
Every unpinned version is resolved at build time, so a rebuild picks up
|
|
current packages and tool releases. Rebuild and retag periodically; child
|
|
images get everything new on their next build.
|
|
|
|
The registry sits behind Traefik. The large `RUN` layer (about 650MB
|
|
compressed, over 2GB uncompressed) needs the entrypoint
|
|
`respondingTimeouts` raised from Traefik's 60 second default, or the final
|
|
blob commit fails with a 499 or 500 and docker re-uploads the layer.
|
|
|
|
Interactive shells (login shells, or anything that sources
|
|
`/etc/profile.d`) also get the `nvm` and `pyenv` shell functions from
|
|
`rootfs/etc/profile.d/baseimage.sh`.
|
|
|
|
## Layout
|
|
|
|
- `Makefile` — target wiring and configuration only; no logic.
|
|
- `bin/` — one script per step: `build`, `test`, `push`, `manifest`,
|
|
`commit-logs`, `clean`, plus `lib.sh` (shared setup) and `native-arch`.
|
|
- `Dockerfile` — base, environment, one `COPY`, one `ADD`, one `RUN`.
|
|
- `build.sh` — everything the `RUN` does, in order: apt, Go, runsvinit,
|
|
golangci-lint, Hugo, nvm/Node/corepack/wrangler, pyenv/Python, cleanup,
|
|
sanity checks. Removes itself when done.
|
|
- `rootfs/` — files copied verbatim onto `/` before `build.sh` runs.
|
|
- `logs/YYYY-MM-DD/` — build logs for each released image, per arch.
|