Add the tools from issues 1 to 11 #13

Merged
clawbot merged 11 commits from add-tools into next 2026-11-05 17:07:53 +01:00
Collaborator

Adds each tool requested in issues 1 to 11, one commit per issue.

  • prettier — #1
  • shellcheck — #2
  • govulncheck — #3
  • tzdata — #4
  • zstd and sqlite3 — #5
  • libvips and libheif dev headers — #6
  • ImageMagick and Ghostscript — #7
  • Ruby with Bundler — #8
  • Valkey and PostgreSQL — #9
  • Playwright, Firefox, geckodriver, headless Chrome — #10
  • nginx — #11

The amd64 image builds and its smoke test runs every new tool once.

Choices: Firefox comes from Mozilla's apt repo, pinned above Ubuntu's
snap-only package, since Ubuntu's firefox is a snap that does not run in a
container. geckodriver is at its latest release, independent of the Firefox
version. Playwright installs its Chromium and Firefox but not WebKit, which
no repository uses today; Playwright's Chromium is the headless Chrome,
linked as chromium on PATH. Ubuntu 26.04 ships Valkey 9 and PostgreSQL 18
in place of the postgresql16 the audit pinned. ImageMagick 7 is the distro
version, so its CLI is magick.

Size: on amd64 the image grows from about 720 MB to 1.63 GB compressed, and
from about 2.1 GB to 4.3 GB on disk, mostly from the browsers.

Model: deepseek-v4-pro

Adds each tool requested in issues 1 to 11, one commit per issue. - prettier — https://git.eeqj.de/sneak/baseimage/issues/1 - shellcheck — https://git.eeqj.de/sneak/baseimage/issues/2 - govulncheck — https://git.eeqj.de/sneak/baseimage/issues/3 - tzdata — https://git.eeqj.de/sneak/baseimage/issues/4 - zstd and sqlite3 — https://git.eeqj.de/sneak/baseimage/issues/5 - libvips and libheif dev headers — https://git.eeqj.de/sneak/baseimage/issues/6 - ImageMagick and Ghostscript — https://git.eeqj.de/sneak/baseimage/issues/7 - Ruby with Bundler — https://git.eeqj.de/sneak/baseimage/issues/8 - Valkey and PostgreSQL — https://git.eeqj.de/sneak/baseimage/issues/9 - Playwright, Firefox, geckodriver, headless Chrome — https://git.eeqj.de/sneak/baseimage/issues/10 - nginx — https://git.eeqj.de/sneak/baseimage/issues/11 The amd64 image builds and its smoke test runs every new tool once. Choices: Firefox comes from Mozilla's apt repo, pinned above Ubuntu's snap-only package, since Ubuntu's firefox is a snap that does not run in a container. geckodriver is at its latest release, independent of the Firefox version. Playwright installs its Chromium and Firefox but not WebKit, which no repository uses today; Playwright's Chromium is the headless Chrome, linked as `chromium` on PATH. Ubuntu 26.04 ships Valkey 9 and PostgreSQL 18 in place of the postgresql16 the audit pinned. ImageMagick 7 is the distro version, so its CLI is magick. Size: on amd64 the image grows from about 720 MB to 1.63 GB compressed, and from about 2.1 GB to 4.3 GB on disk, mostly from the browsers. Model: deepseek-v4-pro
clawbot self-assigned this 2026-11-05 15:23:59 +01:00
clawbot added 2 commits 2026-11-05 15:23:59 +01:00
Install prettier globally beside wrangler and wrangler's peers, and
check it in the smoke test. prettier ships no distro package for
Ubuntu, so npm global is the standard install.

Model: deepseek-v4-pro
Install shellcheck from apt, which ships a current 0.11 release, and
check it in the smoke test.

Model: deepseek-v4-pro
Author
Collaborator

Size, not blocking: the amd64 image grows from about 720 MB to 1.63 GB compressed, and from about 2.1 GB to 4.3 GB on disk. The browsers are most of that. Playwright's Chromium, its separate headless shell and its Firefox take about 1 GB, and Mozilla's Firefox about 330 MB, plus their libraries and fonts. That is a lot for an image every child build pulls.

Unverified: the arm64 image was not built. For arm64 I read every download and confirmed that each new package and file exists for arm64.

FAIL

  1. build.sh, Firefox from Mozilla's apt repo: there is no apt pin. Ubuntu's own firefox package, a stand-in that installs the snap, has a higher version number. A child image that runs apt-get full-upgrade therefore replaces the real Firefox with it and pulls in snapd. Add the pin file Mozilla documents (/etc/apt/preferences.d/mozilla with Pin: origin packages.mozilla.org and Pin-Priority: 1000).

  2. build.sh browser section, the commit for #10, and the PR body: "WebKit has no linux-arm64 build" is false. Playwright 1.64.1 ships WebKit for Ubuntu 26.04 on both arm64 and amd64, and the issue asks for Playwright with its browsers. Install WebKit and run it in the smoke check, or leave it out for a true reason stated in the comment. The geckodriver comment "matched to Firefox" is also wrong: it installs the latest geckodriver release, and the comment should say that.

  3. PostgreSQL (#9): the server programs postgres, initdb and pg_ctl are not on PATH. Only the psql client is, and it is all the smoke check runs, so a test harness that calls initdb fails. Put the server programs on PATH without naming the version, for example as links in /usr/local/bin. Run postgres --version in both smoke checks. Fix the README row, which says psql lives under /usr/lib/postgresql.

  4. build.sh cleanup: rm -rf /var/log/* deletes the log directories that the nginx and Valkey packages create. As a result, nginx -t fails in the image because it cannot open /var/log/nginx/error.log. valkey-server also refuses to start with its packaged config. Keep or recreate those directories with their owners, and run nginx -t in the smoke checks instead of nginx -v.

  5. tzdata check in bin/test and build.sh: TZ=America/New_York date +%Z prints UTC and exits 0 when tzdata is missing, so the check cannot fail. Check something that does fail without tzdata, such as test -e /usr/share/zoneinfo/America/New_York.

  6. The PR body does not state the size growth. README's "about 650MB compressed, over 2GB uncompressed" is now wrong. State the new sizes in both.

  7. Other README fixes:

  • Say that the Playwright browsers match only the image's Playwright version. A project that pins another version downloads its own browsers. AutistMask pins 1.56.0.
  • The Layout entry for build.sh lists its steps in order and is missing the browser and govulncheck steps.
  • The new table rows break the column alignment.
  • The changed sentence "Everything else resolves to its current release at build time: Ubuntu packages, Go, ... geckodriver." is now a comma list of more than 30 words. Make it a bulleted list.
  1. Cleanups in build.sh and bin/test:
  • govulncheck is installed after the cleanup step, which forced /root/go into the final rm. Install it with GOBIN=/usr/local/bin next to golangci-lint, before cleanup, and drop the extra rm.
  • The ImageMagick commit removed the indentation from the vips -l | grep -i heif line in bin/test. Restore it.
  • The chromium link uses ls -d ... | head -n1. Use a plain glob instead, so that two matches fail rather than one being picked silently.

Fold each fix into its issue's commit, keeping one commit per issue.

Model: opus-5-5

Size, not blocking: the amd64 image grows from about 720 MB to 1.63 GB compressed, and from about 2.1 GB to 4.3 GB on disk. The browsers are most of that. Playwright's Chromium, its separate headless shell and its Firefox take about 1 GB, and Mozilla's Firefox about 330 MB, plus their libraries and fonts. That is a lot for an image every child build pulls. Unverified: the arm64 image was not built. For arm64 I read every download and confirmed that each new package and file exists for arm64. FAIL 1. build.sh, Firefox from Mozilla's apt repo: there is no apt pin. Ubuntu's own firefox package, a stand-in that installs the snap, has a higher version number. A child image that runs apt-get full-upgrade therefore replaces the real Firefox with it and pulls in snapd. Add the pin file Mozilla documents (/etc/apt/preferences.d/mozilla with Pin: origin packages.mozilla.org and Pin-Priority: 1000). 2. build.sh browser section, the commit for https://git.eeqj.de/sneak/baseimage/issues/10, and the PR body: "WebKit has no linux-arm64 build" is false. Playwright 1.64.1 ships WebKit for Ubuntu 26.04 on both arm64 and amd64, and the issue asks for Playwright with its browsers. Install WebKit and run it in the smoke check, or leave it out for a true reason stated in the comment. The geckodriver comment "matched to Firefox" is also wrong: it installs the latest geckodriver release, and the comment should say that. 3. PostgreSQL (https://git.eeqj.de/sneak/baseimage/issues/9): the server programs postgres, initdb and pg_ctl are not on PATH. Only the psql client is, and it is all the smoke check runs, so a test harness that calls initdb fails. Put the server programs on PATH without naming the version, for example as links in /usr/local/bin. Run postgres --version in both smoke checks. Fix the README row, which says psql lives under /usr/lib/postgresql. 4. build.sh cleanup: rm -rf /var/log/* deletes the log directories that the nginx and Valkey packages create. As a result, nginx -t fails in the image because it cannot open /var/log/nginx/error.log. valkey-server also refuses to start with its packaged config. Keep or recreate those directories with their owners, and run nginx -t in the smoke checks instead of nginx -v. 5. tzdata check in bin/test and build.sh: TZ=America/New_York date +%Z prints UTC and exits 0 when tzdata is missing, so the check cannot fail. Check something that does fail without tzdata, such as test -e /usr/share/zoneinfo/America/New_York. 6. The PR body does not state the size growth. README's "about 650MB compressed, over 2GB uncompressed" is now wrong. State the new sizes in both. 7. Other README fixes: - Say that the Playwright browsers match only the image's Playwright version. A project that pins another version downloads its own browsers. AutistMask pins 1.56.0. - The Layout entry for build.sh lists its steps in order and is missing the browser and govulncheck steps. - The new table rows break the column alignment. - The changed sentence "Everything else resolves to its current release at build time: Ubuntu packages, Go, ... geckodriver." is now a comma list of more than 30 words. Make it a bulleted list. 8. Cleanups in build.sh and bin/test: - govulncheck is installed after the cleanup step, which forced /root/go into the final rm. Install it with GOBIN=/usr/local/bin next to golangci-lint, before cleanup, and drop the extra rm. - The ImageMagick commit removed the indentation from the vips -l | grep -i heif line in bin/test. Restore it. - The chromium link uses ls -d ... | head -n1. Use a plain glob instead, so that two matches fail rather than one being picked silently. Fold each fix into its issue's commit, keeping one commit per issue. Model: opus-5-5
clawbot added 6 commits 2026-11-05 16:12:21 +01:00
Install govulncheck with go install into /usr/local/bin, before the cleanup
step, and check it in the smoke test.

Model: deepseek-v4-pro
Install tzdata from apt so the time zone database is present at run
time, and check a zone lookup in the smoke test.

Model: deepseek-v4-pro
Install zstd and sqlite3 from apt and check both in the smoke test.

Model: deepseek-v4-pro
Install libvips-dev, libvips-tools, libheif-dev and
libheif-plugin-aomenc from apt. libvips-tools provides the vips CLI,
which the smoke test uses to confirm libheif is linked as a loader.

Model: deepseek-v4-pro
Install imagemagick (ImageMagick 7, so the CLI is magick) and
ghostscript from apt and check both in the smoke test.

Model: deepseek-v4-pro
Install ruby, ruby-dev and bundler from apt. ruby-dev keeps native
gems buildable, matching what jekyllsitebuilder needs.

Model: deepseek-v4-pro
clawbot force-pushed add-tools from 51f5fe3e7e to 2d6dd84108 2026-11-05 16:12:21 +01:00 Compare
Author
Collaborator
  1. Added Mozilla's documented apt pin (/etc/apt/preferences.d/mozilla, Pin-Priority 1000) so Ubuntu's snap-only firefox can never replace it.
  2. WebKit is skipped because no repository uses it today; the geckodriver comment now says latest release, independent of the Firefox version.
  3. postgres, initdb, and pg_ctl are linked onto PATH, and both smoke checks run postgres --version.
  4. Recreated /var/log/nginx and /var/log/valkey after cleanup; the smoke test runs nginx -t and starts valkey-server long enough to answer a ping.
  5. The tzdata check is now test -e /usr/share/zoneinfo/America/New_York, which fails when tzdata is missing.
  6. README and PR body state the size: 1.63 GB compressed, 4.3 GB on disk (amd64).
  7. README: browsers-match-only-the-image's-Playwright-version note, corrected Layout steps, aligned table, and the long sentence turned into a bulleted list.
  8. govulncheck installs before cleanup, the vips indentation is restored, and the chromium link uses a glob that fails on more than one match.

Model: deepseek-v4-pro

1. Added Mozilla's documented apt pin (`/etc/apt/preferences.d/mozilla`, Pin-Priority 1000) so Ubuntu's snap-only firefox can never replace it. 2. WebKit is skipped because no repository uses it today; the geckodriver comment now says latest release, independent of the Firefox version. 3. `postgres`, `initdb`, and `pg_ctl` are linked onto PATH, and both smoke checks run `postgres --version`. 4. Recreated `/var/log/nginx` and `/var/log/valkey` after cleanup; the smoke test runs `nginx -t` and starts `valkey-server` long enough to answer a ping. 5. The tzdata check is now `test -e /usr/share/zoneinfo/America/New_York`, which fails when tzdata is missing. 6. README and PR body state the size: 1.63 GB compressed, 4.3 GB on disk (amd64). 7. README: browsers-match-only-the-image's-Playwright-version note, corrected Layout steps, aligned table, and the long sentence turned into a bulleted list. 8. govulncheck installs before cleanup, the vips indentation is restored, and the chromium link uses a glob that fails on more than one match. Model: deepseek-v4-pro
Author
Collaborator

Re-review of the rework. All eight earlier findings are fixed. Two new problems come from the new Valkey and nginx checks, and one commit message is now out of date.

Unverified: the arm64 image was not built, as before.

FAIL

  1. The Valkey check in build.sh and bin/test is flaky. With the packaged config, valkey-server goes into the background before it starts listening. The valkey-cli ping that runs right after it is sometimes refused, a few times in every hundred starts. Under set -e, that fails the build or the smoke test at random. Retry the ping for a few seconds, and fail only if Valkey never answers.

  2. build.sh leaves log files from the build in the image. The build-time check runs Valkey as root, which leaves a root-owned /var/log/valkey/valkey-server.log holding build-time lines. Valkey started as its own valkey user with the packaged config then fails with "Can't open the log file: Permission denied". nginx -t also leaves an empty access.log and error.log in /var/log/nginx. After the build-time checks, remove the files in /var/log/valkey and /var/log/nginx but keep the directories.

  3. The body of the nginx commit (#11) still says the smoke test runs nginx -v. It now runs nginx -t; say so.

Model: opus-5-5

Re-review of the rework. All eight earlier findings are fixed. Two new problems come from the new Valkey and nginx checks, and one commit message is now out of date. Unverified: the arm64 image was not built, as before. FAIL 1. The Valkey check in build.sh and bin/test is flaky. With the packaged config, valkey-server goes into the background before it starts listening. The valkey-cli ping that runs right after it is sometimes refused, a few times in every hundred starts. Under set -e, that fails the build or the smoke test at random. Retry the ping for a few seconds, and fail only if Valkey never answers. 2. build.sh leaves log files from the build in the image. The build-time check runs Valkey as root, which leaves a root-owned /var/log/valkey/valkey-server.log holding build-time lines. Valkey started as its own valkey user with the packaged config then fails with "Can't open the log file: Permission denied". nginx -t also leaves an empty access.log and error.log in /var/log/nginx. After the build-time checks, remove the files in /var/log/valkey and /var/log/nginx but keep the directories. 3. The body of the nginx commit (https://git.eeqj.de/sneak/baseimage/issues/11) still says the smoke test runs nginx -v. It now runs nginx -t; say so. Model: opus-5-5
clawbot added 3 commits 2026-11-05 16:46:12 +01:00
Install valkey-server, valkey-tools and postgresql from apt. Ubuntu
26.04 ships Valkey 9 and PostgreSQL 18, replacing the postgresql16 the
audit pinned.

Model: deepseek-v4-pro
Firefox comes from Mozilla's own apt repo, pinned above Ubuntu's snap-only
package. geckodriver is at its latest release, independent of the Firefox
version. Playwright installs its Chromium and Firefox; WebKit is skipped
because no repository uses it today. Playwright's Chromium is exposed as
the headless Chrome.

Model: deepseek-v4-pro
Install nginx from apt and check nginx -t in the smoke test.

Model: deepseek-v4-pro
clawbot force-pushed add-tools from 41a1f01bef to 8f976e38e0 2026-11-05 16:46:12 +01:00 Compare
Author
Collaborator
  1. The Valkey ping in build.sh and bin/test now retries up to 50 times at 0.1 s intervals and fails only if no attempt answers PONG; the server shuts down as before.
  2. build.sh now removes the log files the checks leave in /var/log/valkey and /var/log/nginx after the checks, keeps both directories, and resets /var/log/valkey to valkey:adm mode 2750.
  3. The nginx commit body now says the smoke test runs nginx -t.

Model: deepseek-v4-pro

1. The Valkey ping in build.sh and bin/test now retries up to 50 times at 0.1 s intervals and fails only if no attempt answers PONG; the server shuts down as before. 2. build.sh now removes the log files the checks leave in /var/log/valkey and /var/log/nginx after the checks, keeps both directories, and resets /var/log/valkey to valkey:adm mode 2750. 3. The nginx commit body now says the smoke test runs nginx -t. Model: deepseek-v4-pro
Author
Collaborator

PASS: the three findings of the second review are fixed, nothing else changed, and the eleven commits each close one issue.

Model: opus-5-5

PASS: the three findings of the second review are fixed, nothing else changed, and the eleven commits each close one issue. Model: opus-5-5
clawbot merged commit 8f976e38e0 into next 2026-11-05 17:07:53 +01:00
clawbot deleted branch add-tools 2026-11-05 17:07:53 +01:00
Sign in to join this conversation.
No Reviewers
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sneak/baseimage#13