Reported by the owner in chat, 2026-10-01 ~18:3x UTC, deploying webhooker from upaas on fsn1app1 in production: "building webhooker from upaas after rm'ing the webhooker container". His log, verbatim:
Repository cloned (branch: prod)
Cloning into '/repo'...
Warning: Permanently added 'git.eeqj.de' (ED25519) to the list of known hosts.
COMMIT:1647b43aa6b211686719313bc6372c3693c54ca9
ERROR: failed to build image: failed to inspect image: Error response from daemon: No such image: upaas-webhook:146
#1 [internal] load remote build context
#1 DONE 0.2s
#2 copy /context /
#2 DONE 0.1s
#3 [internal] load metadata for docker.io/library/alpine:3.21@sha256:c3f8e73fdb79deaebaa2037150150191b9dcbfba68b4a46d70103204c53f4709
#3 DONE 0.0s
#4 [internal] load metadata for docker.io/golangci/golangci-lint:v2.12.2@sha256:5cceeef04e53efe1470638d4b4b4f5ceefd574955ab3941b2d9a68a8c9ad5240
#4 DONE 0.0s
#5 [internal] load metadata for docker.io/library/golang:1.26.1-bookworm@sha256:4465644228bc2857a954b092167e12aa59c006a3492282a6c820bf4755fd64a4
#5 ERROR: no active sessions
ERROR: golang:1.26.1-bookworm@sha256:4465644228bc2857a954b092167e12aa59c006a3492282a6c820bf4755fd64a4: failed to resolve source metadata for docker.io/library/golang:1.26.1-bookworm@sha256:4465644228bc2857a954b092167e12aa59c006a3492282a6c820bf4755fd64a4: no active sessions
------
> [internal] load metadata for docker.io/library/golang:1.26.1-bookworm@sha256:4465644228bc2857a954b092167e12aa59c006a3492282a6c820bf4755fd64a4:
------
(Stray control bytes from the log stream are dropped above. They show a second defect: the build log carries the stream framing bytes.)
Likely cause, to be confirmed: since the BuildKit switch for #234, builds go through the Docker API without an attached BuildKit session. Base images already on the host resolve (#3, #4). One that must be pulled (#5) needs the session for registry access, and fails with "no active sessions". The first error line then reports the follow-on image inspect instead of the build failure.
Definition of done:
A build whose base images are not yet on the host pulls them and succeeds. A test reproduces this: the build fails before the fix and passes after it.
When a build fails, upaas reports the build's own error first, not a follow-on "No such image".
The build log shown in the UI carries no stream framing or control bytes.
The fix lands on next with an independent review. A next to main PR goes to the owner as soon as it is on next, since he is deploying in production now.
Workaround for the owner until then (stated in chat): pull the missing base image on the host by its digest, then deploy again.
model: opus-5-5
Reported by the owner in chat, 2026-10-01 ~18:3x UTC, deploying webhooker from upaas on fsn1app1 in production: "building webhooker from upaas after rm'ing the webhooker container". His log, verbatim:
```
Repository cloned (branch: prod)
Cloning into '/repo'...
Warning: Permanently added 'git.eeqj.de' (ED25519) to the list of known hosts.
COMMIT:1647b43aa6b211686719313bc6372c3693c54ca9
ERROR: failed to build image: failed to inspect image: Error response from daemon: No such image: upaas-webhook:146
#1 [internal] load remote build context
#1 DONE 0.2s
#2 copy /context /
#2 DONE 0.1s
#3 [internal] load metadata for docker.io/library/alpine:3.21@sha256:c3f8e73fdb79deaebaa2037150150191b9dcbfba68b4a46d70103204c53f4709
#3 DONE 0.0s
#4 [internal] load metadata for docker.io/golangci/golangci-lint:v2.12.2@sha256:5cceeef04e53efe1470638d4b4b4f5ceefd574955ab3941b2d9a68a8c9ad5240
#4 DONE 0.0s
#5 [internal] load metadata for docker.io/library/golang:1.26.1-bookworm@sha256:4465644228bc2857a954b092167e12aa59c006a3492282a6c820bf4755fd64a4
#5 ERROR: no active sessions
ERROR: golang:1.26.1-bookworm@sha256:4465644228bc2857a954b092167e12aa59c006a3492282a6c820bf4755fd64a4: failed to resolve source metadata for docker.io/library/golang:1.26.1-bookworm@sha256:4465644228bc2857a954b092167e12aa59c006a3492282a6c820bf4755fd64a4: no active sessions
------
> [internal] load metadata for docker.io/library/golang:1.26.1-bookworm@sha256:4465644228bc2857a954b092167e12aa59c006a3492282a6c820bf4755fd64a4:
------
```
(Stray control bytes from the log stream are dropped above. They show a second defect: the build log carries the stream framing bytes.)
Likely cause, to be confirmed: since the BuildKit switch for https://git.eeqj.de/sneak/upaas/issues/234, builds go through the Docker API without an attached BuildKit session. Base images already on the host resolve (#3, #4). One that must be pulled (#5) needs the session for registry access, and fails with "no active sessions". The first error line then reports the follow-on image inspect instead of the build failure.
Definition of done:
- A build whose base images are not yet on the host pulls them and succeeds. A test reproduces this: the build fails before the fix and passes after it.
- When a build fails, upaas reports the build's own error first, not a follow-on "No such image".
- The build log shown in the UI carries no stream framing or control bytes.
- The fix lands on `next` with an independent review. A `next` to `main` PR goes to the owner as soon as it is on `next`, since he is deploying in production now.
- Workaround for the owner until then (stated in chat): pull the missing base image on the host by its digest, then deploy again.
model: opus-5-5
clawbot
self-assigned this 2026-10-01 20:32:23 +02:00
Cause.performBuild in internal/docker/client.go calls ImageBuild with Version: BuilderBuildKit and no SessionID. The daemon's BuildKit asks the client's session for registry access when it resolves a base image that is not on the host, and with no session attached it fails with "no active sessions". Images already on the host need no registry access, which is why #3 and #4 in the log passed.
Fix. Attach a BuildKit session to every build the way the docker command line does it: github.com/moby/buildkit/session (already a dependency at v0.16.0), run over the Docker client's DialHijack, its ID passed as SessionID, closed when the build ends. No parsing of FROM lines to pull images ahead of the build. Private registry credentials are out of scope.
"No such image" first. That is the code on main, which has the BuildKit switch (#220) but not the error reporting from #234, which is on next. The worker confirms with a test that on next a failed build reports the build's own error and never a follow-on image inspect.
Framing bytes.ContainerLogs reads the container log stream with io.ReadAll; a container without a terminal sends it in frames with an 8-byte header each. Demultiplex it with stdcopy from github.com/docker/docker. This covers the clone output in the build log and the app container logs.
Tests. A fake Docker daemon in the tests that, like the real one, fails a BuildKit build with "no active sessions" unless a session is attached: fails before the fix, passes after. A test that framed log output comes back clean. The worker also checks the fix by hand against a real daemon with a base image not on the host.
model: opus-5-5
Plan, confirmed from the code on `next`:
- **Cause.** `performBuild` in `internal/docker/client.go` calls `ImageBuild` with `Version: BuilderBuildKit` and no `SessionID`. The daemon's BuildKit asks the client's session for registry access when it resolves a base image that is not on the host, and with no session attached it fails with "no active sessions". Images already on the host need no registry access, which is why #3 and #4 in the log passed.
- **Fix.** Attach a BuildKit session to every build the way the `docker` command line does it: `github.com/moby/buildkit/session` (already a dependency at v0.16.0), run over the Docker client's `DialHijack`, its ID passed as `SessionID`, closed when the build ends. No parsing of `FROM` lines to pull images ahead of the build. Private registry credentials are out of scope.
- **"No such image" first.** That is the code on `main`, which has the BuildKit switch (https://git.eeqj.de/sneak/upaas/issues/220) but not the error reporting from https://git.eeqj.de/sneak/upaas/issues/234, which is on `next`. The worker confirms with a test that on `next` a failed build reports the build's own error and never a follow-on image inspect.
- **Framing bytes.** `ContainerLogs` reads the container log stream with `io.ReadAll`; a container without a terminal sends it in frames with an 8-byte header each. Demultiplex it with `stdcopy` from `github.com/docker/docker`. This covers the clone output in the build log and the app container logs.
- **Tests.** A fake Docker daemon in the tests that, like the real one, fails a BuildKit build with "no active sessions" unless a session is attached: fails before the fix, passes after. A test that framed log output comes back clean. The worker also checks the fix by hand against a real daemon with a base image not on the host.
model: opus-5-5
Built in #256: every build now attaches a BuildKit session, so a base image that is not on the host is pulled instead of failing with "no active sessions". Container logs are read without the stream frame headers, so the clone output in the build log is plain text and the deploy now records the commit it cloned.
Model: opus-5-5
Built in https://git.eeqj.de/sneak/upaas/pulls/256: every build now attaches a BuildKit session, so a base image that is not on the host is pulled instead of failing with "no active sessions". Container logs are read without the stream frame headers, so the clone output in the build log is plain text and the deploy now records the commit it cloned.
Model: opus-5-5
The fix is on next (#256, squashed as 5f9948d): a build now attaches a BuildKit session, so a base image not yet on the host is pulled, and the build and app logs no longer carry stream framing bytes. The next to main PR #244 carries it and is mergeable. To deploy after merging: docker compose up -d --build from an up-to-date clone.
Model: opus-5-5
The fix is on `next` (https://git.eeqj.de/sneak/upaas/pulls/256, squashed as `5f9948d`): a build now attaches a BuildKit session, so a base image not yet on the host is pulled, and the build and app logs no longer carry stream framing bytes. The `next` to `main` PR https://git.eeqj.de/sneak/upaas/pulls/244 carries it and is mergeable. To deploy after merging: `docker compose up -d --build` from an up-to-date clone.
Model: opus-5-5
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Reported by the owner in chat, 2026-10-01 ~18:3x UTC, deploying webhooker from upaas on fsn1app1 in production: "building webhooker from upaas after rm'ing the webhooker container". His log, verbatim:
(Stray control bytes from the log stream are dropped above. They show a second defect: the build log carries the stream framing bytes.)
Likely cause, to be confirmed: since the BuildKit switch for #234, builds go through the Docker API without an attached BuildKit session. Base images already on the host resolve (#3, #4). One that must be pulled (#5) needs the session for registry access, and fails with "no active sessions". The first error line then reports the follow-on image inspect instead of the build failure.
Definition of done:
nextwith an independent review. AnexttomainPR goes to the owner as soon as it is onnext, since he is deploying in production now.model: opus-5-5
Plan, confirmed from the code on
next:performBuildininternal/docker/client.gocallsImageBuildwithVersion: BuilderBuildKitand noSessionID. The daemon's BuildKit asks the client's session for registry access when it resolves a base image that is not on the host, and with no session attached it fails with "no active sessions". Images already on the host need no registry access, which is why #3 and #4 in the log passed.dockercommand line does it:github.com/moby/buildkit/session(already a dependency at v0.16.0), run over the Docker client'sDialHijack, its ID passed asSessionID, closed when the build ends. No parsing ofFROMlines to pull images ahead of the build. Private registry credentials are out of scope.main, which has the BuildKit switch (#220) but not the error reporting from #234, which is onnext. The worker confirms with a test that onnexta failed build reports the build's own error and never a follow-on image inspect.ContainerLogsreads the container log stream withio.ReadAll; a container without a terminal sends it in frames with an 8-byte header each. Demultiplex it withstdcopyfromgithub.com/docker/docker. This covers the clone output in the build log and the app container logs.model: opus-5-5
Built in #256: every build now attaches a BuildKit session, so a base image that is not on the host is pulled instead of failing with "no active sessions". Container logs are read without the stream frame headers, so the clone output in the build log is plain text and the deploy now records the commit it cloned.
Model: opus-5-5
The fix is on
next(#256, squashed as5f9948d): a build now attaches a BuildKit session, so a base image not yet on the host is pulled, and the build and app logs no longer carry stream framing bytes. ThenexttomainPR #244 carries it and is mergeable. To deploy after merging:docker compose up -d --buildfrom an up-to-date clone.Model: opus-5-5