diff --git a/Dockerfile b/Dockerfile index 345ddfd..1c338cc 100644 --- a/Dockerfile +++ b/Dockerfile @@ -20,6 +20,7 @@ ENV LANG=C.UTF-8 \ COREPACK_ENABLE_DOWNLOAD_PROMPT=0 \ COREPACK_ENABLE_AUTO_PIN=0 \ PYENV_ROOT=/usr/local/pyenv \ + PLAYWRIGHT_BROWSERS_PATH=/ms-playwright \ GOPATH=/root/go \ PATH=/usr/local/go/bin:/root/go/bin:/usr/local/nvm/current/bin:/usr/local/pyenv/shims:/usr/local/pyenv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin diff --git a/README.md b/README.md index b729601..0d3a182 100644 --- a/README.md +++ b/README.md @@ -11,26 +11,46 @@ Image: `git.eeqj.de/sneak/baseimage`, built for `linux/amd64` and ## What is in it -| Component | Source | Location | -| --------------- | ------------------------------ | ------------------------------ | -| Ubuntu 26.04 | `ubuntu:26.04`, fully upgraded | `/` | -| init | `runsvinit` (built from Go) | `/usr/local/sbin/runsvinit` | -| services | `runit` (apt) | `/etc/sv`, `/etc/service` | -| build-essential | apt | `gcc`, `make`, `pkg-config` | -| Go (distro) | apt `golang-go` | `/usr/lib/go-*` | -| Go (latest) | go.dev tarball | `/usr/local/go` | -| golangci-lint | GitHub release | `/usr/local/bin/golangci-lint` | -| Hugo (extended) | GitHub release `.deb` | `/usr/local/bin/hugo` | -| Node (distro) | apt `nodejs`, `npm` | `/usr/bin/node` | -| nvm | git tag | `/usr/local/nvm` | -| Node (LTS) | `nvm install --lts` | `/usr/local/nvm/current` | -| corepack | `npm install -g corepack` | nvm LTS global bin | -| yarn, pnpm | corepack, cache pre-populated | `/usr/local/corepack` | -| wrangler | `npm install -g wrangler` | nvm LTS global bin | -| Python (distro) | apt `python3`, `pip`, `venv` | `/usr/bin/python3` | -| pyenv | git tag | `/usr/local/pyenv` | -| Python (latest) | `pyenv install` | `/usr/local/pyenv/versions/*` | -| virtualenv | apt, and pip in pyenv Python | `virtualenv` on `PATH` | +| Component | Source | Location | +| ------------------- | ------------------------------------------ | ------------------------------------------------ | +| Ubuntu 26.04 | `ubuntu:26.04`, fully upgraded | `/` | +| init | `runsvinit` (built from Go) | `/usr/local/sbin/runsvinit` | +| services | `runit` (apt) | `/etc/sv`, `/etc/service` | +| build-essential | apt | `gcc`, `make`, `pkg-config` | +| Go (distro) | apt `golang-go` | `/usr/lib/go-*` | +| Go (latest) | go.dev tarball | `/usr/local/go` | +| golangci-lint | GitHub release | `/usr/local/bin/golangci-lint` | +| shellcheck | apt | `/usr/bin/shellcheck` | +| govulncheck | `go install` (latest) | `/usr/local/bin/govulncheck` | +| Hugo (extended) | GitHub release `.deb` | `/usr/local/bin/hugo` | +| Node (distro) | apt `nodejs`, `npm` | `/usr/bin/node` | +| nvm | git tag | `/usr/local/nvm` | +| Node (LTS) | `nvm install --lts` | `/usr/local/nvm/current` | +| corepack | `npm install -g corepack` | nvm LTS global bin | +| yarn, pnpm | corepack, cache pre-populated | `/usr/local/corepack` | +| wrangler | `npm install -g wrangler` | nvm LTS global bin | +| prettier | `npm install -g prettier` | nvm LTS global bin | +| Playwright | `npm install -g playwright` | nvm LTS global bin | +| Playwright browsers | `playwright install` (chromium, firefox) | `/ms-playwright` | +| Firefox | Mozilla apt repo | `/usr/bin/firefox` | +| geckodriver | GitHub release | `/usr/local/bin/geckodriver` | +| headless Chrome | Playwright Chromium | `chromium` on `PATH` | +| Python (distro) | apt `python3`, `pip`, `venv` | `/usr/bin/python3` | +| pyenv | git tag | `/usr/local/pyenv` | +| Python (latest) | `pyenv install` | `/usr/local/pyenv/versions/*` | +| virtualenv | apt, and pip in pyenv Python | `virtualenv` on `PATH` | +| tzdata | apt | `/usr/share/zoneinfo` | +| zstd | apt | `/usr/bin/zstd` | +| sqlite3 (CLI) | apt | `/usr/bin/sqlite3` | +| libvips | apt `libvips-dev`, `libvips-tools` | `vips` on `PATH` | +| libheif | apt `libheif-dev`, `libheif-plugin-aomenc` | HEIF/AVIF codec | +| ImageMagick | apt | `magick` | +| Ghostscript | apt | `gs` | +| Ruby | apt `ruby`, `ruby-dev` | `/usr/bin/ruby` | +| Bundler | apt `bundler` | `bundle` on `PATH` | +| Valkey | apt `valkey-server`, `valkey-tools` | `valkey-server`, `valkey-cli` | +| PostgreSQL | apt `postgresql` | `psql`, `postgres`, `initdb`, `pg_ctl` on `PATH` | +| nginx | apt | `/usr/sbin/nginx` | Plus the usual odds and ends: `git`, `curl`, `wget`, `jq`, `rsync`, `openssh-client`, `zip`/`unzip`, `xz-utils`, `bzip2`, and the `-dev` @@ -54,7 +74,8 @@ dependencies in child images. Environment variables set in the image: `NVM_DIR`, `NVM_SYMLINK_CURRENT`, `COREPACK_HOME`, `COREPACK_ENABLE_DOWNLOAD_PROMPT=0`, -`COREPACK_ENABLE_AUTO_PIN=0`, `PYENV_ROOT`, `GOPATH`, `LANG=C.UTF-8`. +`COREPACK_ENABLE_AUTO_PIN=0`, `PYENV_ROOT`, `GOPATH`, `LANG=C.UTF-8`, +`PLAYWRIGHT_BROWSERS_PATH`. ### Yarn and corepack @@ -77,9 +98,18 @@ the `Dockerfile`: | `NVM_GIT_TAG` | `nvm-sh/nvm` release tag | | `PYENV_GIT_TAG` | `pyenv/pyenv` release tag | -Everything else resolves to its current release at build time: Ubuntu -packages, Go, Node LTS, Python, Hugo, golangci-lint, npm, yarn, pnpm, -wrangler. +Everything else resolves to its current release at build time: + +- Ubuntu packages +- Go, Node LTS, and Python +- Hugo and golangci-lint +- npm, yarn, pnpm, wrangler, and prettier +- Playwright and its browsers +- Firefox and geckodriver + +Playwright installs its browsers for the image's Playwright version only. A +child image that pins a different Playwright version downloads its own +browsers. ## Services and init @@ -172,8 +202,8 @@ Every unpinned version is resolved at build time, so a rebuild picks up current packages and tool releases. Rebuild and retag periodically; child images get everything new on their next build. -The registry sits behind Traefik. The large `RUN` layer (about 650MB -compressed, over 2GB uncompressed) needs the entrypoint +The registry sits behind Traefik. The large `RUN` layer (about 1.63 GB +compressed, 4.3 GB on disk) needs the entrypoint `respondingTimeouts` raised from Traefik's 60 second default, or the final blob commit fails with a 499 or 500 and docker re-uploads the layer. @@ -188,7 +218,7 @@ Interactive shells (login shells, or anything that sources `commit-logs`, `clean`, plus `lib.sh` (shared setup) and `native-arch`. - `Dockerfile` — base, environment, one `COPY`, one `ADD`, one `RUN`. - `build.sh` — everything the `RUN` does, in order: apt, Go, runsvinit, - golangci-lint, Hugo, nvm/Node/corepack/wrangler, pyenv/Python, cleanup, - sanity checks. Removes itself when done. + golangci-lint, govulncheck, Hugo, nvm/Node/corepack/wrangler, browsers, + pyenv/Python, cleanup, sanity checks. Removes itself when done. - `rootfs/` — files copied verbatim onto `/` before `build.sh` runs. - `logs/YYYY-MM-DD/` — build logs for each released image, per arch. diff --git a/bin/test b/bin/test index a00d8c7..a94ea8d 100755 --- a/bin/test +++ b/bin/test @@ -20,6 +20,30 @@ docker run --rm "$img" bash -ec ' go version; node --version; python3 --version; hugo version | head -1 COREPACK_ENABLE_NETWORK=0 yarn --version COREPACK_ENABLE_NETWORK=0 pnpm --version + prettier --version + shellcheck --version + govulncheck -version + test -e /usr/share/zoneinfo/America/New_York + zstd --version + sqlite3 --version + vips --version + vips -l | grep -i heif > /dev/null + magick --version + gs --version + ruby --version + bundle --version + valkey-server --version + valkey-cli --version + valkey-server /etc/valkey/valkey.conf + ok=; for i in $(seq 1 50); do if valkey-cli ping 2>/dev/null | grep -q PONG; then ok=1; break; fi; sleep 0.1; done; [ -n "$ok" ] + valkey-cli shutdown nosave + psql --version + postgres --version + nginx -t + playwright --version + firefox --version + geckodriver --version + chromium --version ' echo "== $img: runit service lifecycle" diff --git a/build.sh b/build.sh index 3825f33..5c57002 100755 --- a/build.sh +++ b/build.sh @@ -25,7 +25,21 @@ apt-get -y install --no-install-recommends \ python3 python3-dev python3-pip python3-venv virtualenv \ libbz2-dev libffi-dev libgdbm-dev liblzma-dev libncurses-dev \ libreadline-dev libsqlite3-dev libssl-dev libxml2-dev libxmlsec1-dev \ - tk-dev uuid-dev zlib1g-dev + tk-dev uuid-dev zlib1g-dev \ + shellcheck \ + tzdata \ + zstd sqlite3 \ + libvips-dev libvips-tools libheif-dev libheif-plugin-aomenc \ + imagemagick ghostscript \ + ruby ruby-dev bundler \ + valkey-server valkey-tools postgresql \ + nginx + +# Link the PostgreSQL server binaries onto PATH. They live under a versioned +# directory (/usr/lib/postgresql/*/bin); the client psql is already on PATH. +for p in postgres initdb pg_ctl; do + ln -s /usr/lib/postgresql/*/bin/"$p" /usr/local/bin/"$p" +done # --- Go (latest) ------------------------------------------------------------- gover="$(curl -fsSL 'https://go.dev/VERSION?m=text' | head -n1)" @@ -46,6 +60,9 @@ gl="$(latest_tag golangci/golangci-lint)" curl -fsSL "https://github.com/golangci/golangci-lint/releases/download/${gl}/golangci-lint-${gl#v}-linux-${arch}.tar.gz" | tar -C /usr/local/bin -xz --strip-components=1 "golangci-lint-${gl#v}-linux-${arch}/golangci-lint" +# --- govulncheck (Go tool) --------------------------------------------------- +GOBIN=/usr/local/bin go install golang.org/x/vuln/cmd/govulncheck@latest + # --- Hugo (extended) --------------------------------------------------------- hv="$(latest_tag gohugoio/hugo)" curl -fsSL -o /tmp/hugo.deb \ @@ -61,7 +78,7 @@ nvm install --lts nvm alias default "lts/*" nvm use default set -ux -npm install -g npm@latest corepack wrangler +npm install -g npm@latest corepack wrangler prettier playwright corepack enable # Pre-populate the corepack cache so projects that pin a package manager # do not download at run time. The last yarn installed becomes the default. @@ -74,6 +91,42 @@ pnpm --version yarn --version nvm cache clear +# --- browsers for tests: Firefox, geckodriver, Playwright -------------------- +# Ubuntu's firefox is a snap that does not run in a container, so use +# Mozilla's own apt repository for the real binary. +install -d -m 0755 /etc/apt/keyrings +curl -fsSL https://packages.mozilla.org/apt/repo-signing-key.gpg \ + -o /etc/apt/keyrings/packages.mozilla.org.asc +echo "deb [signed-by=/etc/apt/keyrings/packages.mozilla.org.asc] https://packages.mozilla.org/apt mozilla main" \ + > /etc/apt/sources.list.d/mozilla.list +cat > /etc/apt/preferences.d/mozilla <<'EOF' +Package: * +Pin: origin packages.mozilla.org +Pin-Priority: 1000 +EOF +apt-get update +apt-get -y install --no-install-recommends -t mozilla firefox + +# geckodriver, at its latest release (independent of the Firefox version). +gd="$(latest_tag mozilla/geckodriver)" +case "$arch" in + arm64) gd_arch=linux-aarch64 ;; + *) gd_arch=linux64 ;; +esac +curl -fsSL "https://github.com/mozilla/geckodriver/releases/download/${gd}/geckodriver-${gd}-${gd_arch}.tar.gz" | + tar -C /usr/local/bin -xz geckodriver + +# Playwright's Chromium and Firefox. WebKit is skipped because no repository +# uses it today. Playwright's Chromium is also the headless Chrome for +# chromedp-style clients, exposed as `chromium` on PATH. +playwright install --with-deps chromium firefox +chromiums=( "$PLAYWRIGHT_BROWSERS_PATH"/chromium-*/chrome-linux*/chrome ) +if [ "${#chromiums[@]}" -ne 1 ] || [ ! -x "${chromiums[0]}" ]; then + echo "error: expected exactly one chromium binary, found ${#chromiums[@]}" >&2 + exit 1 +fi +ln -s "${chromiums[0]}" /usr/local/bin/chromium + # --- pyenv, Python (latest), pip, virtualenv --------------------------------- git clone -q --depth 1 --branch "$PYENV_GIT_TAG" https://github.com/pyenv/pyenv.git "$PYENV_ROOT" rm -rf "$PYENV_ROOT/.git" @@ -93,21 +146,53 @@ rm -rf /usr/share/doc /usr/share/man /usr/share/info /usr/share/lintian find /usr/share/locale -mindepth 1 -maxdepth 1 ! -name 'en*' ! -name 'locale.alias' -exec rm -rf {} + rm -rf /root/.cache /root/.npm /root/.config /root/go /tmp/* /var/tmp/* +# nginx and Valkey open their log files at startup; recreate the log +# directories the cleanup above removed, with their package owners. +install -d -o root -g adm /var/log/nginx +install -d -o valkey -g adm -m 2750 /var/log/valkey + # --- sanity ------------------------------------------------------------------ go version +govulncheck -version golangci-lint version +shellcheck --version +test -e /usr/share/zoneinfo/America/New_York +zstd --version +sqlite3 --version +vips --version +vips -l | grep -i heif > /dev/null +magick --version +gs --version +ruby --version +bundle --version +valkey-server --version +valkey-cli --version +valkey-server /etc/valkey/valkey.conf +ok=; for i in $(seq 1 50); do if valkey-cli ping 2>/dev/null | grep -q PONG; then ok=1; break; fi; sleep 0.1; done; [ -n "$ok" ] +valkey-cli shutdown nosave +psql --version +postgres --version +nginx -t hugo version node --version npm --version COREPACK_ENABLE_NETWORK=0 yarn --version COREPACK_ENABLE_NETWORK=0 pnpm --version wrangler --version +prettier --version +playwright --version +firefox --version +geckodriver --version +chromium --version python3 --version pip --version virtualenv --version test -x /usr/local/sbin/runsvinit test -d /etc/service -# The checks above leave caches behind (node compile cache etc). +# The checks above leave caches and log files behind. rm -rf /tmp/* /var/tmp/* /root/.cache /root/.npm /root/.config +rm -f /var/log/valkey/* +rm -f /var/log/nginx/* +install -d -o valkey -g adm -m 2750 /var/log/valkey rm -f "$0"