17 Commits
Author SHA1 Message Date
clawbot 563156ab76 Keep the Applications list's table inside its card (closes #262)
Check / check (pull_request) Skipped
Table cells never wrap, so a long repository URL made the table wider
than its card, which hides what does not fit and so cut off the Actions
column and the Deploy buttons. The repository URL now wraps within its
column, and the card scrolls sideways when the table still does not
fit. A handler test checks both. Every class used was already in the
committed static/css/tailwind.css.

Model: opus-5-5
2026-10-02 03:49:16 +00:00
clawbot b101bc1a80 Keep git-ignored files and data/ out of the Docker build context (closes #266)
Check / check (pull_request) Successful in 4m12s
.dockerignore now lists every .gitignore pattern, each with **/ so Docker
matches it in every directory as git does, plus the top-level data/
directory. git-ignored secrets such as .env.local, *.key files and
data/session.key no longer reach the build stages or the build cache. A last
!.git/** line sends all of .git again, since git never applies these patterns
inside it, so a branch named like fix/session.key still resolves. No tracked
file is listed, so the version still comes from git describe without -dirty.

data/, where upaasd keeps its database and session key when run from the
checkout, is now git-ignored.

Model: opus-5-5
2026-10-02 05:43:40 +02:00
clawbot 5c836c085d Keep .git/config out of the Docker build context (closes #269)
Check / check (pull_request) Successful in 5m3s
.git goes into the build so `make build` can stamp the version, and with
it went .git/config, where a remote URL can carry a credential that then
stays in the builder stage's layers on the build host. `git describe`
does not need it, so .dockerignore now leaves it out.

Model: opus-5-5
2026-10-02 04:43:03 +02:00
clawbot 5168db69d9 Read the architecture at run time instead of a Buildarch ldflag (closes #259)
Check / check (pull_request) Successful in 5m19s
The Makefile no longer passes the architecture to the linker. The
Buildarch variable in main and the field and setter in globals are
gone; the startup log line reports runtime.GOARCH as `arch`.
CONVENTIONS.md drops Buildarch from its main, globals, logger and
Makefile examples, as the sneak/prompts conventions do.

Model: opus-5-5
2026-10-02 03:21:51 +02:00
clawbot 76858126e2 Revert "Say that a deploy keeps only an app's volumes when the app has none"
Check / check (pull_request) Successful in 4m29s
The owner asked for this commit to be reverted in full
(#254 (comment)). It
removes the no-volume-mounts sentence from the app page and the deploy
log, the constant and tests behind it, the README sentence, and its
TODO.md entry. Later TODO.md entries stay. #248
stays closed as rejected.

Model: opus-5-5
2026-10-02 01:43:28 +02:00
sneak 5ca1bce6fa Merge branch 'main' into next
Check / check (pull_request) Successful in 5m22s
2026-10-02 00:58:03 +02:00
clawbot 9488e2faef Tag built images with the commit's short hash (closes #239)
Check / check (pull_request) Successful in 5m31s
Builds are tagged upaas-<app>:<short hash>, git's short form of
the commit checked out, instead of the deployment number. The clone
prints the short hash and fails without one.

A redeploy of a commit gives the tag to the new image. The cleanup
after a deploy now also finds the app's untagged images among those
its deployments recorded, and removes them by ID once the app neither
runs them nor would roll back to them. It keeps every image any app
uses, since apps that build the same commit can share one.

commit_sha is now saved on update so manual deploys keep the commit
read from the clone.

Model: opus-5-5
2026-10-01 22:33:20 +02:00
clawbot 047945cabf Make concurrent database writes wait instead of failing (closes #253)
Check / check (pull_request) Successful in 4m25s
SQLite transactions started deferred, so two that both read and then
wrote could not both proceed, and SQLite refused the second at once with
"database is locked" without waiting. TestCreateUserRaceCondition hit
this now and then.

The database is now opened with _txlock=immediate, so each transaction
takes the write lock when it begins and a second one waits for it, and
with _busy_timeout=5000, so that wait lasts up to 5 seconds. The driver
already defaults to 5000; it is now stated in the code.

Model: opus-5-5
2026-10-01 22:16:14 +02:00
clawbot b16a5c9724 Say env var changes take effect at the next deploy or rollback (closes #255)
Check / check (pull_request) Successful in 4m35s
The hint under the app page's environment variable editor asked for a
container restart. Environment variables are set when a container is
created, and upaas's restart stops and starts the same container, so it
keeps the old values; deploy and rollback each create a new container.
The hint now says so in one sentence, styled with alert-warning like the
page's Volume Mounts note: its old text-amber-600 class is not in the
built stylesheet. A handler test renders the app page and checks the
hint.

Model: opus-5-5
2026-10-01 21:51:14 +02:00
clawbot cc786a1e84 Say that a deploy keeps only an app's volumes when the app has none (closes #248)
Check / check (pull_request) Successful in 3m50s
An app with no volume mounts now shows, in its Volume Mounts section,
that the files it writes are lost whenever a deploy or rollback replaces
its container and that a restart keeps them. Each deploy of such an app
writes the same sentence into its log, after the webhook payload and
before the clone. The README says in one sentence that a deploy or
rollback starts a new container that keeps only the files in the app's
volume mounts.

Model: opus-5-5
2026-10-01 21:34:29 +02:00
clawbot 5f9948d7e2 Attach a BuildKit session to builds and demultiplex container logs (closes #251)
Check / check (pull_request) Successful in 3m59s
Builds now attach a BuildKit session over the Docker API, as the docker
command line does, and pass its ID with the build. BuildKit fetches a base
image that is not on the host through that session; without one, Docker
Engine 27 failed the build with "no active sessions". The session is
closed when the build ends.

Container logs are now read with stdcopy, so the clone output in the build
log and the app logs no longer carry Docker's 8-byte frame headers, and the
commit is read from the clone output; the header in front of the COMMIT
line kept it from being found.

Model: opus-5-5
Co-authored-by: clawbot <sneak+clawbot@sneak.cloud>
2026-10-01 21:15:06 +02:00
clawbot d69f11f74c Have Docker create a missing volume host path (closes #235)
Check / check (pull_request) Successful in 4m54s
Docker refused to bind-mount a host path that did not exist, so an app's
first deploy failed until someone ran mkdir on the host. buildMounts now
sets BindOptions.CreateMountpoint on every bind mount, so the Docker
daemon creates a missing host path when the container starts and leaves
an existing one alone. upaas itself cannot create it: it runs in a
container and does not see the host's filesystem. The README gains a
Volume mounts section saying so. Needs Docker Engine 23.0 or later.

Model: opus-5-5
2026-10-01 20:48:13 +02:00
clawbot 9754b73f27 Widen the app page, double its log heights and move the logs (closes #246)
Check / check (pull_request) Successful in 5m29s
The app page's content column is now at most 84rem wide instead of
56rem (max-w-4xl), set inline because the committed Tailwind CSS has no
class for that width. The build log and container log boxes are 800px
tall instead of 400px. The build log section moves to between the
webhook URL and the environment variables, and the container log
section moves to directly above the deploy key; nothing else moves. A
new handler test renders the app page and checks the width, the section
order and both log heights.
Disclosure: merged after a rebase that changed only TODO.md; the review gated this change on the next before #234, which touches no template.

Model: opus-5-5
Co-authored-by: clawbot <sneak+clawbot@sneak.cloud>
2026-09-29 12:46:47 +02:00
clawbot 679c80700f Report the build's own error and refuse daemons too old for BuildKit (closes #234)
Check / check (pull_request) Successful in 4m23s
A build whose output ends in Docker's error line now fails with that
error, instead of going on to inspect a tag that was never created. The
build output is written to the deployment log before the failure is
recorded, so the log ends in order. Before building, upaas compares the
daemon's API version with 1.39 (Docker Engine 18.09), the first that
builds with BuildKit without experimental mode, and fails the deploy on
an older daemon instead of letting it use the legacy builder. The
README's Compose section gives the update command and the Docker Engine
versions builds need.
Disclosure: merged after a rebase that changed only TODO.md; the review gated this tree on the current next.

Model: opus-5-5
Co-authored-by: clawbot <sneak+clawbot@sneak.cloud>
2026-09-29 12:42:55 +02:00
clawbot a48d90f5ea Stamp the git commit into images built from the Dockerfile (closes #236)
Check / check (pull_request) Successful in 3m31s
.dockerignore left out .git, so `make build` in the image found no git
metadata and stamped `dev`. It now sends .git, and no longer leaves out
tracked files (LICENSE, README.md, ...), which git would see as deleted
and mark the version -dirty. The footer and /health already read the
same version. The startup log line that reports it, the logger's
Identify(), was never called; main now calls it. The README says to
build from a git clone.
Side effect: image layers after `COPY . .` now rebuild whenever `.git` changes.
Disclosure: merged after a rebase that changed only TODO.md; the second review gated this same tree on this base.

Model: opus-5-5
2026-09-29 12:35:45 +02:00
clawbot 211e2a4a5a Show the deploy branch in the app page title (closes #240)
Check / check (pull_request) Successful in 3m44s
The app page shows the app's configured branch as a neutral label next
to the status badge, so it can be read without opening the edit page.
The line under the title now shows only the repository. A new test
renders the app page for an app on a non-main branch and checks the
branch is in the title row.

Model: opus-5-5
Co-authored-by: clawbot <sneak+clawbot@sneak.cloud>
2026-09-29 12:21:35 +02:00
clawbot a836bc5f80 Show the 10 most recent deployments on the deployments page (closes #238)
Check / check (pull_request) Successful in 4m6s
An app's deployments page listed up to 50 deployments; it now lists the
10 most recent, newest first. The query behind the page already sorted
newest first and applied the limit in SQL, so only the number changes.
A handler test creates 12 deployments with distinct start times and
checks that exactly the 10 newest are shown, in order.

Model: opus-5-5
Co-authored-by: clawbot <sneak+clawbot@sneak.cloud>
2026-09-29 12:02:23 +02:00
30 changed files with 1681 additions and 213 deletions
+35 -11
View File
@@ -1,11 +1,35 @@
.git # .git is sent so that `make build` in the Dockerfile can stamp the commit into
.env # upaas. List no tracked file here: git would see it as deleted in the build and
bin/ # the version would end in -dirty.
.editorconfig
.vscode/ # The patterns of .gitignore; **/ makes Docker match them in every directory.
.idea/ **/.DS_Store
*.test **/Thumbs.db
LICENSE **/*.swp
CONVENTIONS.md **/*.swo
REPO_POLICIES.md **/*~
README.md **/*.bak
**/.idea/
**/.vscode/
**/*.sublime-*
**/node_modules/
**/.env
**/.env.*
**/*.pem
**/*.key
**/bin/
**/*.exe
**/*.exe~
**/*.dll
**/*.so
**/*.dylib
**/*.test
**/*.out
/data/
# Git never applies its ignore patterns inside .git; send all of it again.
!.git/**
# .git is sent without its config, because a remote URL there can carry a
# credential; `git describe` does not need it. Keep this after !.git/**.
.git/config
+5
View File
@@ -1,3 +1,5 @@
# .dockerignore repeats these patterns; change both together.
# OS # OS
.DS_Store .DS_Store
Thumbs.db Thumbs.db
@@ -29,3 +31,6 @@ bin/
*.dylib *.dylib
*.test *.test
*.out *.out
# upaasd's data directory when it runs from the checkout (UPAAS_DATA_DIR default)
/data/
+2 -10
View File
@@ -117,13 +117,11 @@ import (
var ( var (
Appname string = "CHANGEME" Appname string = "CHANGEME"
Version string Version string
Buildarch string
) )
func main() { func main() {
globals.Appname = Appname globals.Appname = Appname
globals.Version = Version globals.Version = Version
globals.Buildarch = Buildarch
fx.New( fx.New(
fx.Provide( fx.Provide(
@@ -823,7 +821,7 @@ func (l *Logger) Identify() {
l.log.Info("starting", l.log.Info("starting",
"appname", l.params.Globals.Appname, "appname", l.params.Globals.Appname,
"version", l.params.Globals.Version, "version", l.params.Globals.Version,
"buildarch", l.params.Globals.Buildarch, "arch", runtime.GOARCH,
) )
} }
``` ```
@@ -945,20 +943,17 @@ import "go.uber.org/fx"
var ( var (
Appname string Appname string
Version string Version string
Buildarch string
) )
// Struct for DI // Struct for DI
type Globals struct { type Globals struct {
Appname string Appname string
Version string Version string
Buildarch string
} }
func New(lc fx.Lifecycle) (*Globals, error) { func New(lc fx.Lifecycle) (*Globals, error) {
n := &Globals{ n := &Globals{
Appname: Appname, Appname: Appname,
Buildarch: Buildarch,
Version: Version, Version: Version,
} }
return n, nil return n, nil
@@ -972,13 +967,11 @@ func New(lc fx.Lifecycle) (*Globals, error) {
var ( var (
Appname string = "CHANGEME" // Default, overridden by build Appname string = "CHANGEME" // Default, overridden by build
Version string // Set at build time Version string // Set at build time
Buildarch string // Set at build time
) )
func main() { func main() {
globals.Appname = Appname globals.Appname = Appname
globals.Version = Version globals.Version = Version
globals.Buildarch = Buildarch
// ... // ...
} }
``` ```
@@ -989,10 +982,9 @@ Use ldflags to inject version information at build time:
```makefile ```makefile
VERSION := $(shell git describe --tags --always) VERSION := $(shell git describe --tags --always)
BUILDARCH := $(shell go env GOARCH)
build: build:
go build -ldflags "-X main.Version=$(VERSION) -X main.Buildarch=$(BUILDARCH)" ./cmd/httpd go build -ldflags "-X main.Version=$(VERSION)" ./cmd/httpd
``` ```
--- ---
+1
View File
@@ -31,6 +31,7 @@ RUN go mod download
COPY . . COPY . .
RUN make test RUN make test
# Takes the version from `git describe` on the .git copied in above.
RUN make build RUN make build
# Runtime stage # Runtime stage
+1 -2
View File
@@ -2,8 +2,7 @@
BINARY := upaasd BINARY := upaasd
VERSION := $(shell git describe --tags --always --dirty 2>/dev/null || echo "dev") VERSION := $(shell git describe --tags --always --dirty 2>/dev/null || echo "dev")
BUILDARCH := $(shell go env GOARCH) LDFLAGS := -X main.Version=$(VERSION)
LDFLAGS := -X main.Version=$(VERSION) -X main.Buildarch=$(BUILDARCH)
all: check build all: check build
+19 -1
View File
@@ -226,6 +226,10 @@ This recipe serves plain HTTP, so `UPAAS_PLAINTEXT_HTTP=true` is required for
setup and every other form to pass the CSRF origin check. Behind a setup and every other form to pass the CSRF origin check. Behind a
TLS-terminating reverse proxy, drop that line. TLS-terminating reverse proxy, drop that line.
The image shows the commit it was built from (the `git describe` output) in the
page footer, the startup log and `/health`. Build it from a git clone: without
the `.git` directory it shows `dev`.
### Deploying with Docker Compose ### Deploying with Docker Compose
[`docker-compose.yml`](docker-compose.yml) builds the image from this repo and [`docker-compose.yml`](docker-compose.yml) builds the image from this repo and
@@ -241,7 +245,9 @@ Other settings from [Configuration](#configuration) go in the same file, except
settings to 8080 and `UPAAS_DATA_DIR` to `/var/lib/upaas`, overriding `.env`, to settings to 8080 and `UPAAS_DATA_DIR` to `/var/lib/upaas`, overriding `.env`, to
match its port mapping, healthcheck and data directory mount. Then run match its port mapping, healthcheck and data directory mount. Then run
`docker compose up -d` from the repo root; `docker compose ps` shows the `docker compose up -d` from the repo root; `docker compose ps` shows the
container as healthy once `/health` answers. container as healthy once `/health` answers. To update, run `git pull` and then
`docker compose up -d --build`: without `--build`, Compose keeps running the
image built from the old checkout.
**Important**: `HOST_DATA_DIR` **must** be an **absolute path** on the host. It **Important**: `HOST_DATA_DIR` **must** be an **absolute path** on the host. It
is bind-mounted into the container and passed as `UPAAS_HOST_DATA_DIR` so that is bind-mounted into the container and passed as `UPAAS_HOST_DATA_DIR` so that
@@ -257,9 +263,21 @@ Docker's build cache rather than as untagged images. Docker Engine 28.2 and
later keeps that cache under a size limit by default; on older engines, set later keeps that cache under a size limit by default; on older engines, set
`"builder": {"gc": {"enabled": true}}` in the host's `daemon.json`. `"builder": {"gc": {"enabled": true}}` in the host's `daemon.json`.
Building with BuildKit needs Docker Engine 18.09 or later; on an older engine,
upaas fails the deploy instead of building. A Dockerfile that uses
`RUN --network` needs Docker Engine 23.0 or later unless its `# syntax=` line
names Dockerfile frontend 1.3 or later, such as `docker/dockerfile:1`.
Session secrets are automatically generated on first startup and persisted to Session secrets are automatically generated on first startup and persisted to
`$UPAAS_DATA_DIR/session.key`. `$UPAAS_DATA_DIR/session.key`.
### Volume mounts
An app's volume mounts are bind mounts of host paths. When a host path does not
exist yet, upaas has Docker create it as an empty directory, owned by root, when
the app's container starts; there is no need to create it first. An existing
host path is left as it is. This needs Docker Engine 23.0 or later.
## License ## License
WTFPL WTFPL
+73
View File
@@ -20,6 +20,79 @@ regress.
# Completed Steps # Completed Steps
- 2026-10-02: On the Applications list, a long repository URL now wraps within
its column instead of making the table wider than its card, which cut off the
Actions column and the Deploy buttons. In a window too narrow for the table,
the card scrolls sideways instead of cutting the table off (#262).
- 2026-10-02: `docker build .` no longer sends git-ignored files, such as
`.env.local`, `*.key` files or upaasd's `data/` directory with its session
key, into the build stages and the build cache: `.dockerignore` now leaves out
everything `.gitignore` does, and `data/` is git-ignored (#266).
- 2026-10-02: `.dockerignore` leaves out `.git/config`, so a remote URL there
that carries a credential no longer goes into the Docker build; the image
still shows the commit it was built from (#269).
- 2026-10-02: The build no longer passes the CPU architecture in: upaas reads it
from Go's `runtime.GOARCH` when it runs, and the startup log line reports it
as `arch`. `CONVENTIONS.md` follows the updated conventions in `sneak/prompts`
(#259).
- 2026-10-01: Built images are tagged `upaas-<app>:<short hash>`, git's short
form of the commit built, instead of the deployment number. A redeploy of a
commit gives its tag to the new image; the old one is kept while the app runs
it or Rollback would start it, then removed by its ID, found among the images
the app's deployments recorded. The removal of old images now keeps every
image any app runs or would roll back to. A deployment's commit is now saved
when it is updated, so manual deploys keep the commit read from the clone
(#239).
- 2026-10-01: Two database writes at the same moment no longer fail with
"database is locked": each transaction now takes the write lock when it begins
and waits up to 5 seconds for another writer to finish (#253).
- 2026-10-01: The hint under the app page's environment variable editor now says
changes take effect at the next deploy or rollback, in the page's warning
style, instead of asking for a container restart, which keeps the old values
(#255).
- 2026-10-01: Builds attach a BuildKit session, as the docker command line does,
so a base image that is not on the host is pulled instead of the build failing
with "no active sessions" on Docker Engine 27. Container logs, and so the
clone output in the build log and the app logs, no longer carry Docker's
stream frame headers, and the commit is now read from the clone output (#251).
- 2026-10-01: An app's first deploy no longer fails when a volume's host path
does not exist yet: upaas asks Docker to create a missing host path when the
app's container starts and to leave an existing one alone, so nobody has to
create it on the host first. The README has a Volume mounts section saying so
(#235).
- 2026-09-29: The app page is 50% wider on large screens (84rem instead of
56rem), its build log and container log boxes are twice as tall, the build log
sits between the webhook URL and the environment variables, and the container
log sits above the deploy key (#246).
- 2026-09-29: A failed build now fails the deploy with the build's own error
instead of a later "failed to inspect image", and the deployment log shows the
end of the build output before that error. upaas refuses to build on a Docker
Engine older than 18.09, which cannot build with BuildKit. The README's
Compose section says to update with `docker compose up -d --build` and names
the Docker Engine versions builds need (#234).
- 2026-09-29: An image built from the `Dockerfile` now shows the commit it was
built from (the `git describe` output) in the footer and `/health` instead of
`dev`: `.dockerignore` no longer leaves out `.git`, nor any tracked file,
which git would count as deleted and mark `-dirty`. upaas now also logs its
version at startup; the logger's `Identify()` was never called (#236).
- 2026-09-29: The app page shows the app's branch as a label in its title, next
to the status badge, instead of after the repository under it (#240).
- 2026-09-29: An app's deployments page now lists only its 10 most recent
deployments, newest first, instead of 50 (#238).
- 2026-09-29: `docker-compose.yml` now sets `UPAAS_PORT` to 8080 as well as - 2026-09-29: `docker-compose.yml` now sets `UPAAS_PORT` to 8080 as well as
`PORT`, since upaas reads `UPAAS_PORT` first and a `UPAAS_PORT` in `.env` made `PORT`, since upaas reads `UPAAS_PORT` first and a `UPAAS_PORT` in `.env` made
it listen away from the port mapping and healthcheck; the README's Compose it listen away from the port mapping and healthcheck; the README's Compose
+3 -3
View File
@@ -27,13 +27,11 @@ import (
var ( var (
Appname = "upaas" //nolint:gochecknoglobals // build-time variable Appname = "upaas" //nolint:gochecknoglobals // build-time variable
Version string //nolint:gochecknoglobals // build-time variable Version string //nolint:gochecknoglobals // build-time variable
Buildarch string //nolint:gochecknoglobals // build-time variable
) )
func main() { func main() {
globals.SetAppname(Appname) globals.SetAppname(Appname)
globals.SetVersion(Version) globals.SetVersion(Version)
globals.SetBuildarch(Buildarch)
fx.New( fx.New(
fx.Provide( fx.Provide(
@@ -52,6 +50,8 @@ func main() {
handlers.New, handlers.New,
server.New, server.New,
), ),
fx.Invoke(func(*server.Server) {}), fx.Invoke(func(log *logger.Logger, _ *server.Server) {
log.Identify()
}),
).Run() ).Run()
} }
+5 -2
View File
@@ -137,8 +137,11 @@ func (d *Database) connect(ctx context.Context) error {
return fmt.Errorf("failed to create data directory: %w", err) return fmt.Errorf("failed to create data directory: %w", err)
} }
// Open database with WAL mode and foreign keys // Open database with WAL mode and foreign keys. Transactions take the
dsn := dbPath + "?_journal_mode=WAL&_foreign_keys=on" // write lock when they begin and wait up to 5s for another writer,
// instead of failing with "database is locked" when both write.
dsn := dbPath + "?_journal_mode=WAL&_foreign_keys=on" +
"&_txlock=immediate&_busy_timeout=5000"
database, err := sql.Open("sqlite3", dsn) database, err := sql.Open("sqlite3", dsn)
if err != nil { if err != nil {
+151 -23
View File
@@ -3,12 +3,14 @@ package docker
import ( import (
"bufio" "bufio"
"bytes"
"context" "context"
"encoding/json" "encoding/json"
"errors" "errors"
"fmt" "fmt"
"io" "io"
"log/slog" "log/slog"
"net"
"os" "os"
"path/filepath" "path/filepath"
"regexp" "regexp"
@@ -21,12 +23,15 @@ import (
"github.com/docker/docker/api/types/image" "github.com/docker/docker/api/types/image"
"github.com/docker/docker/api/types/mount" "github.com/docker/docker/api/types/mount"
"github.com/docker/docker/api/types/network" "github.com/docker/docker/api/types/network"
"github.com/docker/docker/api/types/versions"
"github.com/docker/docker/client" "github.com/docker/docker/client"
"github.com/docker/docker/pkg/archive" "github.com/docker/docker/pkg/archive"
"github.com/docker/docker/pkg/jsonmessage" "github.com/docker/docker/pkg/jsonmessage"
"github.com/docker/docker/pkg/stdcopy"
"github.com/docker/go-connections/nat" "github.com/docker/go-connections/nat"
controlapi "github.com/moby/buildkit/api/services/control" controlapi "github.com/moby/buildkit/api/services/control"
buildkitclient "github.com/moby/buildkit/client" buildkitclient "github.com/moby/buildkit/client"
"github.com/moby/buildkit/session"
"github.com/moby/buildkit/util/progress/progressui" "github.com/moby/buildkit/util/progress/progressui"
"go.uber.org/fx" "go.uber.org/fx"
@@ -61,6 +66,15 @@ var ErrInvalidBranch = errors.New("invalid branch name")
// ErrInvalidCommitSHA is returned when a commit SHA is not a valid hex string. // ErrInvalidCommitSHA is returned when a commit SHA is not a valid hex string.
var ErrInvalidCommitSHA = errors.New("invalid commit SHA") var ErrInvalidCommitSHA = errors.New("invalid commit SHA")
// ErrBuildKitUnavailable is returned when the Docker daemon is too old to
// build with BuildKit.
var ErrBuildKitUnavailable = errors.New("BuildKit is unavailable on the Docker daemon")
// minBuildKitAPIVersion is the API version of Docker Engine 18.09, the first
// that builds with BuildKit when asked to without experimental mode. Older
// daemons refuse the request or silently use the legacy builder.
const minBuildKitAPIVersion = "1.39"
// validBranchRe matches safe git branch names. // validBranchRe matches safe git branch names.
var validBranchRe = regexp.MustCompile(`^[a-zA-Z0-9._/\-]+$`) var validBranchRe = regexp.MustCompile(`^[a-zA-Z0-9._/\-]+$`)
@@ -212,7 +226,9 @@ func buildEnvSlice(env map[string]string) []string {
return envSlice return envSlice
} }
// buildMounts converts volume mounts to Docker mount configuration. // buildMounts converts volume mounts to Docker mount configuration. Docker,
// not upaas, creates a missing host path, because upaas runs in a container
// and cannot see the host's filesystem. An existing path is left alone.
func buildMounts(volumes []VolumeMount) []mount.Mount { func buildMounts(volumes []VolumeMount) []mount.Mount {
mounts := make([]mount.Mount, 0, len(volumes)) mounts := make([]mount.Mount, 0, len(volumes))
@@ -222,6 +238,9 @@ func buildMounts(volumes []VolumeMount) []mount.Mount {
Source: vol.HostPath, Source: vol.HostPath,
Target: vol.ContainerPath, Target: vol.ContainerPath,
ReadOnly: vol.ReadOnly, ReadOnly: vol.ReadOnly,
BindOptions: &mount.BindOptions{
CreateMountpoint: true,
},
}) })
} }
@@ -373,12 +392,17 @@ func (c *Client) ContainerLogs(
} }
}() }()
logs, err := io.ReadAll(reader) // A container without a terminal, as all of upaas's are, sends its
// output in frames, each with a header naming stdout or stderr. Both
// go to one buffer, in the order they were written.
var logs bytes.Buffer
_, err = stdcopy.StdCopy(&logs, &logs, reader)
if err != nil { if err != nil {
return "", fmt.Errorf("failed to read container logs: %w", err) return "", fmt.Errorf("failed to read container logs: %w", err)
} }
return string(logs), nil return logs.String(), nil
} }
// IsContainerRunning checks if a container is running. // IsContainerRunning checks if a container is running.
@@ -481,6 +505,7 @@ type cloneConfig struct {
type CloneResult struct { type CloneResult struct {
Output string // Combined stdout/stderr from git clone Output string // Combined stdout/stderr from git clone
CommitSHA string // The HEAD commit SHA after clone/checkout CommitSHA string // The HEAD commit SHA after clone/checkout
ShortSHA string // git's short form of CommitSHA (git rev-parse --short)
} }
// CloneRepo clones a git repository using SSH and optionally checks out a // CloneRepo clones a git repository using SSH and optionally checks out a
@@ -543,7 +568,7 @@ func (c *Client) RemoveImage(ctx context.Context, imageID ImageID) error {
} }
// ListImageTags returns the tags in the given repository, such as // ListImageTags returns the tags in the given repository, such as
// "upaas-myapp:12" in "upaas-myapp", each with the ID of its image. // "upaas-myapp:1a2b3c4" in "upaas-myapp", each with the ID of its image.
// Tags the same image has in other repositories are left out. // Tags the same image has in other repositories are left out.
func (c *Client) ListImageTags( func (c *Client) ListImageTags(
ctx context.Context, ctx context.Context,
@@ -573,19 +598,44 @@ func (c *Client) ListImageTags(
return tags, nil return tags, nil
} }
// RemoveImageTag removes a tag such as "upaas-myapp:12", without force. // ListUntaggedImages returns the IDs of the images that have no tag, such
// Docker then deletes the image, and the untagged images it was built on, // as one whose tag a later build gave to the image it built.
// only if no other tag and no container still uses it. func (c *Client) ListUntaggedImages(ctx context.Context) ([]ImageID, error) {
func (c *Client) RemoveImageTag(ctx context.Context, tag string) error { if c.docker == nil {
return nil, ErrNotConnected
}
images, err := c.docker.ImageList(ctx, image.ListOptions{
Filters: filters.NewArgs(filters.Arg("dangling", "true")),
})
if err != nil {
return nil, fmt.Errorf("failed to list untagged images: %w", err)
}
imageIDs := make([]ImageID, 0, len(images))
for _, img := range images {
imageIDs = append(imageIDs, ImageID(img.ID))
}
return imageIDs, nil
}
// RemoveImageTag removes the tag name, such as "upaas-myapp:1a2b3c4",
// without force. Docker then deletes the image, and the untagged images it
// was built on, only if no other tag and no container still uses it. If name
// is instead the ID of an untagged image, it removes that image unless a
// container uses it.
func (c *Client) RemoveImageTag(ctx context.Context, name string) error {
if c.docker == nil { if c.docker == nil {
return ErrNotConnected return ErrNotConnected
} }
_, err := c.docker.ImageRemove(ctx, tag, image.RemoveOptions{ _, err := c.docker.ImageRemove(ctx, name, image.RemoveOptions{
PruneChildren: true, PruneChildren: true,
}) })
if err != nil && !client.IsErrNotFound(err) { if err != nil && !client.IsErrNotFound(err) {
return fmt.Errorf("failed to remove image tag %s: %w", tag, err) return fmt.Errorf("failed to remove image %s: %w", name, err)
} }
return nil return nil
@@ -595,6 +645,20 @@ func (c *Client) performBuild(
ctx context.Context, ctx context.Context,
opts BuildImageOptions, opts BuildImageOptions,
) (ImageID, error) { ) (ImageID, error) {
server, err := c.docker.ServerVersion(ctx)
if err != nil {
return "", fmt.Errorf("failed to get Docker version: %w", err)
}
if versions.LessThan(server.APIVersion, minBuildKitAPIVersion) {
return "", fmt.Errorf(
"%w: Docker Engine %s (API %s) is older than 18.09 (API %s); "+
"upgrade Docker Engine",
ErrBuildKitUnavailable, server.Version, server.APIVersion,
minBuildKitAPIVersion,
)
}
// Create tar archive of build context // Create tar archive of build context
tarArchive, err := archive.TarWithOptions(opts.ContextDir, &archive.TarOptions{}) tarArchive, err := archive.TarWithOptions(opts.ContextDir, &archive.TarOptions{})
if err != nil { if err != nil {
@@ -608,11 +672,24 @@ func (c *Client) performBuild(
} }
}() }()
buildSession, err := c.startBuildSession(ctx)
if err != nil {
return "", err
}
defer func() {
closeErr := buildSession.Close()
if closeErr != nil {
c.log.Error("failed to close build session", "error", closeErr)
}
}()
// Build with BuildKit: the stages of a multi-stage build are kept in // Build with BuildKit: the stages of a multi-stage build are kept in
// its build cache, which Docker limits on its own, instead of being // its build cache, which Docker limits on its own, instead of being
// left behind as untagged images. // left behind as untagged images.
resp, err := c.docker.ImageBuild(ctx, tarArchive, dockertypes.ImageBuildOptions{ resp, err := c.docker.ImageBuild(ctx, tarArchive, dockertypes.ImageBuildOptions{
Version: dockertypes.BuilderBuildKit, Version: dockertypes.BuilderBuildKit,
SessionID: buildSession.ID(),
Dockerfile: opts.DockerfilePath, Dockerfile: opts.DockerfilePath,
Tags: opts.Tags, Tags: opts.Tags,
Remove: true, Remove: true,
@@ -648,6 +725,34 @@ func (c *Client) performBuild(
return "", nil return "", nil
} }
// startBuildSession attaches a BuildKit session to the daemon, as the docker
// command line does for a build. BuildKit asks the client, over the session,
// for registry access to fetch a base image that is not on the host; without
// a session, Docker Engine 27 fails the build with "no active sessions". The
// shared key is only used for a build context sent over the session; upaas
// sends the context with the build request. The caller closes the session.
func (c *Client) startBuildSession(ctx context.Context) (*session.Session, error) {
buildSession, err := session.NewSession(ctx, "")
if err != nil {
return nil, fmt.Errorf("failed to create build session: %w", err)
}
go func() {
runErr := buildSession.Run(ctx, func(
ctx context.Context,
proto string,
meta map[string][]string,
) (net.Conn, error) {
return c.docker.DialHijack(ctx, "/session", proto, meta)
})
if runErr != nil {
c.log.Error("build session failed", "error", runErr)
}
}()
return buildSession, nil
}
// scannerInitialBufferSize is the initial buffer size for the build log scanner. // scannerInitialBufferSize is the initial buffer size for the build log scanner.
const scannerInitialBufferSize = 64 * 1024 // 64KB const scannerInitialBufferSize = 64 * 1024 // 64KB
@@ -660,7 +765,8 @@ const scannerMaxBufferSize = 1024 * 1024 // 1MB
// newline-delimited JSON. BuildKit's progress arrives encoded in // newline-delimited JSON. BuildKit's progress arrives encoded in
// "moby.buildkit.trace" messages; these are decoded and written as plain // "moby.buildkit.trace" messages; these are decoded and written as plain
// text, as "docker build --progress=plain" shows it. Other lines, such as // text, as "docker build --progress=plain" shows it. Other lines, such as
// build errors, are written unchanged. // build errors, are written unchanged. Docker ends a failed build with a line
// carrying the error; it is returned once the output is written.
func (c *Client) streamBuildOutput( func (c *Client) streamBuildOutput(
ctx context.Context, ctx context.Context,
body io.Reader, body io.Reader,
@@ -690,6 +796,8 @@ func (c *Client) streamBuildOutput(
buf := make([]byte, 0, scannerInitialBufferSize) buf := make([]byte, 0, scannerInitialBufferSize)
scanner.Buffer(buf, scannerMaxBufferSize) scanner.Buffer(buf, scannerMaxBufferSize)
var buildErr error
for scanner.Scan() { for scanner.Scan() {
line := scanner.Bytes() line := scanner.Bytes()
@@ -708,6 +816,10 @@ func (c *Client) streamBuildOutput(
continue continue
} }
if err == nil && msg.Error != nil {
buildErr = msg.Error
}
// One write per line, so it is not split by the display's output. // One write per line, so it is not split by the display's output.
_, _ = fmt.Fprintf(out, "%s\n", line) _, _ = fmt.Fprintf(out, "%s\n", line)
} }
@@ -720,7 +832,7 @@ func (c *Client) streamBuildOutput(
return fmt.Errorf("failed to read build output: %w", scanErr) return fmt.Errorf("failed to read build output: %w", scanErr)
} }
return nil return buildErr
} }
func (c *Client) performClone( func (c *Client) performClone(
@@ -819,11 +931,13 @@ func (c *Client) createGitContainer(
// Clone without depth limit so we can checkout any commit, then checkout specific SHA // Clone without depth limit so we can checkout any commit, then checkout specific SHA
script = `git clone --branch "$CLONE_BRANCH" "$CLONE_URL" /repo` + script = `git clone --branch "$CLONE_BRANCH" "$CLONE_URL" /repo` +
` && cd /repo && git checkout "$CLONE_SHA"` + ` && cd /repo && git checkout "$CLONE_SHA"` +
` && echo COMMIT:$(git rev-parse HEAD)` ` && echo COMMIT:$(git rev-parse HEAD)` +
` && echo SHORT_SHA:$(git rev-parse --short HEAD)`
} else { } else {
// Shallow clone of branch HEAD, then output commit SHA // Shallow clone of branch HEAD, then output commit SHA
script = `git clone --depth 1 --branch "$CLONE_BRANCH" "$CLONE_URL" /repo` + script = `git clone --depth 1 --branch "$CLONE_BRANCH" "$CLONE_URL" /repo` +
` && cd /repo && echo COMMIT:$(git rev-parse HEAD)` ` && cd /repo && echo COMMIT:$(git rev-parse HEAD)` +
` && echo SHORT_SHA:$(git rev-parse --short HEAD)`
} }
env := []string{ env := []string{
@@ -901,23 +1015,37 @@ func (c *Client) runGitClone(
) )
} }
// Parse commit SHA from output (looks for "COMMIT:<sha>" line) // Parse the commit from the "COMMIT:" and "SHORT_SHA:" lines.
commitSHA := parseCommitSHA(logs) result := &CloneResult{
Output: logs,
CommitSHA: parseCommitSHA(logs, commitMarker),
ShortSHA: parseCommitSHA(logs, shortSHAMarker),
}
return &CloneResult{Output: logs, CommitSHA: commitSHA}, nil // The short hash names the image the deploy builds.
if result.ShortSHA == "" {
return nil, fmt.Errorf("%w: no short commit hash in its output: %s",
ErrGitCloneFailed, logs)
}
return result, nil
} }
} }
// commitMarker is the prefix used to identify commit SHA in clone output. // Prefixes of the lines in the clone output that carry the commit checked
const commitMarker = "COMMIT:" // out, in full and in git's short form.
const (
commitMarker = "COMMIT:"
shortSHAMarker = "SHORT_SHA:"
)
// parseCommitSHA extracts the commit SHA from git clone output. // parseCommitSHA extracts a commit SHA from git clone output.
// It looks for a line starting with "COMMIT:" and returns the SHA after it. // It looks for a line starting with marker and returns the SHA after it.
func parseCommitSHA(output string) string { func parseCommitSHA(output, marker string) string {
for line := range strings.SplitSeq(output, "\n") { for line := range strings.SplitSeq(output, "\n") {
line = strings.TrimSpace(line) line = strings.TrimSpace(line)
sha, found := strings.CutPrefix(line, commitMarker) sha, found := strings.CutPrefix(line, marker)
if found { if found {
return strings.TrimSpace(sha) return strings.TrimSpace(sha)
} }
+39
View File
@@ -0,0 +1,39 @@
package docker //nolint:testpackage // tests unexported buildMounts
import (
"testing"
"github.com/docker/docker/api/types/mount"
"github.com/stretchr/testify/assert"
)
// TestBuildMountsCreatesMissingHostPath checks that every mount asks Docker
// to create its host path if it is missing, and keeps the rest of the volume
// as configured.
func TestBuildMountsCreatesMissingHostPath(t *testing.T) {
t.Parallel()
volumes := []VolumeMount{
{HostPath: "/srv/app/data", ContainerPath: "/data", ReadOnly: false},
{HostPath: "/srv/app/config", ContainerPath: "/etc/app", ReadOnly: true},
}
want := []mount.Mount{
{
Type: mount.TypeBind,
Source: "/srv/app/data",
Target: "/data",
ReadOnly: false,
BindOptions: &mount.BindOptions{CreateMountpoint: true},
},
{
Type: mount.TypeBind,
Source: "/srv/app/config",
Target: "/etc/app",
ReadOnly: true,
BindOptions: &mount.BindOptions{CreateMountpoint: true},
},
}
assert.Equal(t, want, buildMounts(volumes))
}
+378 -12
View File
@@ -6,6 +6,7 @@ import (
"encoding/json" "encoding/json"
"errors" "errors"
"fmt" "fmt"
"io"
"log/slog" "log/slog"
"net/http" "net/http"
"net/http/httptest" "net/http/httptest"
@@ -15,7 +16,9 @@ import (
"testing" "testing"
"time" "time"
"github.com/docker/docker/api/types/container"
"github.com/docker/docker/client" "github.com/docker/docker/client"
"github.com/docker/docker/pkg/stdcopy"
controlapi "github.com/moby/buildkit/api/services/control" controlapi "github.com/moby/buildkit/api/services/control"
) )
@@ -203,6 +206,8 @@ func TestPerformCloneRemovesContainerVolumes(t *testing.T) {
<-r.Context().Done() <-r.Context().Done()
case strings.HasSuffix(r.URL.Path, "/wait"): case strings.HasSuffix(r.URL.Path, "/wait"):
_, _ = fmt.Fprintf(w, `{"StatusCode":%d}`, tt.exitCode) _, _ = fmt.Fprintf(w, `{"StatusCode":%d}`, tt.exitCode)
case strings.HasSuffix(r.URL.Path, "/logs"):
writeCloneOutput(w)
default: default:
_, _ = w.Write([]byte(`{}`)) _, _ = w.Write([]byte(`{}`))
} }
@@ -219,18 +224,7 @@ func TestPerformCloneRemovesContainerVolumes(t *testing.T) {
c := &Client{docker: dockerAPI, log: slog.Default()} c := &Client{docker: dockerAPI, log: slog.Default()}
dir := t.TempDir() _, _ = c.performClone(ctx, testCloneConfig(t))
cfg := &cloneConfig{
repoURL: "git@example.com:repo.git",
branch: mainBranch,
sshPrivateKey: "fake-key",
containerDir: filepath.Join(dir, "repo"),
hostDir: filepath.Join(dir, "repo"),
keyFile: filepath.Join(dir, "deploy_key"),
hostKeyFile: filepath.Join(dir, "deploy_key"),
}
_, _ = c.performClone(ctx, cfg)
select { select {
case query := <-removeQuery: case query := <-removeQuery:
@@ -244,6 +238,38 @@ func TestPerformCloneRemovesContainerVolumes(t *testing.T) {
} }
} }
// testCloneConfig returns the settings of a clone in these tests, with its
// files in a temporary directory.
func testCloneConfig(t *testing.T) *cloneConfig {
t.Helper()
dir := t.TempDir()
return &cloneConfig{
repoURL: "git@example.com:repo.git",
branch: mainBranch,
sshPrivateKey: "fake-key",
containerDir: filepath.Join(dir, "repo"),
hostDir: filepath.Join(dir, "repo"),
keyFile: filepath.Join(dir, "deploy_key"),
hostKeyFile: filepath.Join(dir, "deploy_key"),
}
}
// cloneCommit is the commit the fake clones in these tests check out.
const cloneCommit = "1a2b3c4d5e6f7a8b9c0d1e2f3a4b5c6d7e8f9a0b"
// writeCloneOutput writes the output of a git clone of cloneCommit as
// Docker sends a container's log: each line after a header.
func writeCloneOutput(w io.Writer) {
stdout := stdcopy.NewStdWriter(w, stdcopy.Stdout)
stderr := stdcopy.NewStdWriter(w, stdcopy.Stderr)
_, _ = stderr.Write([]byte("Cloning into '/repo'...\n"))
_, _ = stdout.Write([]byte("COMMIT:" + cloneCommit + "\n"))
_, _ = stdout.Write([]byte("SHORT_SHA:1a2b3c4\n"))
}
// TestPerformBuildUsesBuildKit runs a build against a fake Docker API and // TestPerformBuildUsesBuildKit runs a build against a fake Docker API and
// checks that it asks for BuildKit and that BuildKit's progress reaches the // checks that it asks for BuildKit and that BuildKit's progress reaches the
// build log as plain text. // build log as plain text.
@@ -271,6 +297,10 @@ func TestPerformBuildUsesBuildKit(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc( srv := httptest.NewServer(http.HandlerFunc(
func(w http.ResponseWriter, r *http.Request) { func(w http.ResponseWriter, r *http.Request) {
switch { switch {
case strings.HasSuffix(r.URL.Path, "/version"):
_, _ = w.Write([]byte(`{"Version":"27.3.1","ApiVersion":"1.47"}`))
case strings.HasSuffix(r.URL.Path, "/session"):
serveSession(t, w, r, make(chan string, 1))
case strings.HasSuffix(r.URL.Path, "/build"): case strings.HasSuffix(r.URL.Path, "/build"):
if r.URL.Query().Get("version") != "2" { if r.URL.Query().Get("version") != "2" {
http.Error(w, "not a BuildKit build", http.StatusBadRequest) http.Error(w, "not a BuildKit build", http.StatusBadRequest)
@@ -314,3 +344,339 @@ func TestPerformBuildUsesBuildKit(t *testing.T) {
t.Errorf("build log is missing the build step:\n%s", buildLog.String()) t.Errorf("build log is missing the build step:\n%s", buildLog.String())
} }
} }
// TestPerformBuildFails runs builds that fail against a fake Docker API and
// checks that each returns its own error and that no image is inspected
// afterwards.
func TestPerformBuildFails(t *testing.T) {
t.Parallel()
tests := []struct {
name string
engine string // Docker Engine version the fake daemon reports
apiVersion string // API version the fake daemon reports
buildOutput string
wantErr string
}{
{
name: "build step fails",
engine: "27.3.1",
apiVersion: "1.47",
buildOutput: `{"stream":"Step 1/1 : RUN false\n"}` + "\n" +
`{"errorDetail":{"message":"exit code: 1"},"error":"exit code: 1"}`,
wantErr: "exit code: 1",
},
{
name: "daemon too old for BuildKit",
engine: "18.06.3-ce",
apiVersion: "1.38",
wantErr: "BuildKit is unavailable on the Docker daemon: " +
"Docker Engine 18.06.3-ce (API 1.38) is older than 18.09 (API 1.39); " +
"upgrade Docker Engine",
},
{
// The build step's own error shows the build went ahead.
name: "daemon at API 1.39 builds",
engine: "18.09.9",
apiVersion: "1.39",
buildOutput: `{"stream":"Step 1/1 : RUN false\n"}` + "\n" +
`{"errorDetail":{"message":"exit code: 1"},"error":"exit code: 1"}`,
wantErr: "exit code: 1",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
srv := httptest.NewServer(http.HandlerFunc(
func(w http.ResponseWriter, r *http.Request) {
switch {
case strings.HasSuffix(r.URL.Path, "/version"):
_, _ = fmt.Fprintf(w, `{"Version":%q,"ApiVersion":%q}`,
tt.engine, tt.apiVersion)
case strings.HasSuffix(r.URL.Path, "/session"):
serveSession(t, w, r, make(chan string, 1))
case strings.HasSuffix(r.URL.Path, "/build"):
_, _ = w.Write([]byte(tt.buildOutput))
default:
t.Errorf("unexpected request to %s", r.URL.Path)
}
},
))
t.Cleanup(srv.Close)
dockerAPI, err := client.NewClientWithOpts(
client.WithHost("tcp://" + srv.Listener.Addr().String()),
)
if err != nil {
t.Fatal(err)
}
c := &Client{docker: dockerAPI, log: slog.Default()}
_, err = c.performBuild(t.Context(), BuildImageOptions{
ContextDir: t.TempDir(),
Tags: []string{"upaas-test:1"},
})
if err == nil || err.Error() != tt.wantErr {
t.Errorf("got error %v, want %q", err, tt.wantErr)
}
})
}
}
// TestPerformBuildAttachesSession runs a build against a fake Docker API
// that, like the real daemon, fails the build with "no active sessions"
// unless the build names a session the client attached over the session
// endpoint. It also checks that the session is closed when the build ends.
func TestPerformBuildAttachesSession(t *testing.T) {
t.Parallel()
attached := make(chan string, 1)
sessionClosed := make(chan struct{})
srv := httptest.NewServer(http.HandlerFunc(
func(w http.ResponseWriter, r *http.Request) {
switch {
case strings.HasSuffix(r.URL.Path, "/version"):
_, _ = w.Write([]byte(`{"Version":"27.3.1","ApiVersion":"1.47"}`))
case strings.HasSuffix(r.URL.Path, "/session"):
serveSession(t, w, r, attached)
close(sessionClosed)
case strings.HasSuffix(r.URL.Path, "/build"):
id := r.URL.Query().Get("session")
attachedID := ""
// The daemon waits a few seconds for the build's session
// to attach.
if id != "" {
select {
case attachedID = <-attached:
case <-time.After(5 * time.Second):
}
}
if id == "" || attachedID != id {
_, _ = w.Write([]byte(`{"errorDetail":{"message":"no active sessions"},` +
`"error":"no active sessions"}`))
}
default:
t.Errorf("unexpected request to %s", r.URL.Path)
}
},
))
t.Cleanup(srv.Close)
dockerAPI, err := client.NewClientWithOpts(
client.WithHost("tcp://" + srv.Listener.Addr().String()),
)
if err != nil {
t.Fatal(err)
}
c := &Client{docker: dockerAPI, log: slog.Default()}
_, err = c.performBuild(t.Context(), BuildImageOptions{ContextDir: t.TempDir()})
if err != nil {
t.Fatal(err)
}
select {
case <-sessionClosed:
case <-time.After(5 * time.Second):
t.Error("the build's session was not closed when the build ended")
}
}
// serveSession answers a request to attach a session as the Docker daemon
// does: it switches the connection over to the session, sends the session's
// ID on attached, and holds the connection until the client closes it.
func serveSession(
t *testing.T,
w http.ResponseWriter,
r *http.Request,
attached chan<- string,
) {
t.Helper()
conn, _, err := http.NewResponseController(w).Hijack()
if err != nil {
t.Error(err)
return
}
defer func() { _ = conn.Close() }()
_, err = io.WriteString(conn, "HTTP/1.1 101 Switching Protocols\r\n"+
"Connection: Upgrade\r\nUpgrade: h2c\r\n\r\n")
if err != nil {
t.Error(err)
return
}
attached <- r.Header.Get("X-Docker-Expose-Session-Uuid")
_, _ = io.Copy(io.Discard, conn)
}
// TestPerformCloneReadsFramedLogs runs a clone against a fake Docker API that
// sends the clone container's output in frames, as Docker does for a
// container without a terminal, and checks that the output comes back as
// plain text and that the commit, in full and in git's short form, is read
// from it.
func TestPerformCloneReadsFramedLogs(t *testing.T) {
t.Parallel()
srv := httptest.NewServer(http.HandlerFunc(
func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
switch {
case strings.HasSuffix(r.URL.Path, "/containers/create"):
_, _ = w.Write([]byte(`{"Id":"gitcontainer"}`))
case strings.HasSuffix(r.URL.Path, "/wait"):
_, _ = w.Write([]byte(`{"StatusCode":0}`))
case strings.HasSuffix(r.URL.Path, "/logs"):
writeCloneOutput(w)
default:
_, _ = w.Write([]byte(`{}`))
}
},
))
t.Cleanup(srv.Close)
dockerAPI, err := client.NewClientWithOpts(
client.WithHost("tcp://" + srv.Listener.Addr().String()),
)
if err != nil {
t.Fatal(err)
}
c := &Client{docker: dockerAPI, log: slog.Default()}
result, err := c.performClone(t.Context(), testCloneConfig(t))
if err != nil {
t.Fatal(err)
}
want := "Cloning into '/repo'...\nCOMMIT:" + cloneCommit + "\nSHORT_SHA:1a2b3c4\n"
if result.Output != want {
t.Errorf("got clone output %q, want %q", result.Output, want)
}
if result.CommitSHA != cloneCommit || result.ShortSHA != "1a2b3c4" {
t.Errorf("got commit %q, short %q", result.CommitSHA, result.ShortSHA)
}
}
// TestPerformCloneFailsWithoutShortSHA runs a clone against a fake Docker API
// whose clone succeeds but prints no "SHORT_SHA:" line, and checks that the
// clone fails, since the short hash names the image the deploy builds.
func TestPerformCloneFailsWithoutShortSHA(t *testing.T) {
t.Parallel()
srv := httptest.NewServer(http.HandlerFunc(
func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
switch {
case strings.HasSuffix(r.URL.Path, "/containers/create"):
_, _ = w.Write([]byte(`{"Id":"gitcontainer"}`))
case strings.HasSuffix(r.URL.Path, "/wait"):
_, _ = w.Write([]byte(`{"StatusCode":0}`))
case strings.HasSuffix(r.URL.Path, "/logs"):
_, _ = stdcopy.NewStdWriter(w, stdcopy.Stdout).
Write([]byte("COMMIT:" + cloneCommit + "\n"))
default:
_, _ = w.Write([]byte(`{}`))
}
},
))
t.Cleanup(srv.Close)
dockerAPI, err := client.NewClientWithOpts(
client.WithHost("tcp://" + srv.Listener.Addr().String()),
)
if err != nil {
t.Fatal(err)
}
c := &Client{docker: dockerAPI, log: slog.Default()}
_, err = c.performClone(t.Context(), testCloneConfig(t))
if !errors.Is(err, ErrGitCloneFailed) {
t.Errorf("got error %v, want %v", err, ErrGitCloneFailed)
}
}
// TestPerformCloneAsksGitForShortSHA runs a clone of a branch's last commit
// and a clone of a given commit against a fake Docker API, and checks that
// the command each clone container is created with prints git's own short
// form of the commit checked out.
func TestPerformCloneAsksGitForShortSHA(t *testing.T) {
t.Parallel()
tests := []struct {
name string
commitSHA string
}{
{name: "branch", commitSHA: ""},
{name: "commit", commitSHA: cloneCommit},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
created := make(chan container.Config, 1)
srv := httptest.NewServer(http.HandlerFunc(
func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
switch {
case strings.HasSuffix(r.URL.Path, "/containers/create"):
var cfg container.Config
_ = json.NewDecoder(r.Body).Decode(&cfg)
created <- cfg
_, _ = w.Write([]byte(`{"Id":"gitcontainer"}`))
case strings.HasSuffix(r.URL.Path, "/wait"):
_, _ = w.Write([]byte(`{"StatusCode":0}`))
case strings.HasSuffix(r.URL.Path, "/logs"):
writeCloneOutput(w)
default:
_, _ = w.Write([]byte(`{}`))
}
},
))
t.Cleanup(srv.Close)
dockerAPI, err := client.NewClientWithOpts(
client.WithHost("tcp://" + srv.Listener.Addr().String()),
)
if err != nil {
t.Fatal(err)
}
c := &Client{docker: dockerAPI, log: slog.Default()}
cfg := testCloneConfig(t)
cfg.commitSHA = tt.commitSHA
_, err = c.performClone(t.Context(), cfg)
if err != nil {
t.Fatal(err)
}
cmd := strings.Join((<-created).Cmd, " ")
if !strings.Contains(cmd, "echo SHORT_SHA:$(git rev-parse --short HEAD)") {
t.Errorf("clone command %q does not print git's short hash", cmd)
}
})
}
}
-11
View File
@@ -15,14 +15,12 @@ var (
mu sync.RWMutex mu sync.RWMutex
appname string appname string
version string version string
buildarch string
) )
// Globals holds build-time variables for dependency injection. // Globals holds build-time variables for dependency injection.
type Globals struct { type Globals struct {
Appname string Appname string
Version string Version string
Buildarch string
} }
// New creates a new Globals instance from package-level variables. // New creates a new Globals instance from package-level variables.
@@ -33,7 +31,6 @@ func New(_ fx.Lifecycle) (*Globals, error) {
return &Globals{ return &Globals{
Appname: appname, Appname: appname,
Version: version, Version: version,
Buildarch: buildarch,
}, nil }, nil
} }
@@ -52,11 +49,3 @@ func SetVersion(ver string) {
version = ver version = ver
} }
// SetBuildarch sets the build architecture (used for testing and main init).
func SetBuildarch(arch string) {
mu.Lock()
defer mu.Unlock()
buildarch = arch
}
+1 -1
View File
@@ -28,7 +28,7 @@ const (
// recentDeploymentsLimit is the number of recent deployments to show. // recentDeploymentsLimit is the number of recent deployments to show.
recentDeploymentsLimit = 5 recentDeploymentsLimit = 5
// deploymentsHistoryLimit is the number of deployments to show in history. // deploymentsHistoryLimit is the number of deployments to show in history.
deploymentsHistoryLimit = 50 deploymentsHistoryLimit = 10
) )
// redirectToApp issues a SeeOther redirect to the page for the given // redirectToApp issues a SeeOther redirect to the page for the given
+46
View File
@@ -0,0 +1,46 @@
package handlers_test
import (
"net/http"
"net/http/httptest"
"strings"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"sneak.berlin/go/upaas/internal/service/app"
)
// TestAppPageTitleShowsBranch checks that an app's branch can be read from
// the app page title, next to the status badge, without opening the edit page.
func TestAppPageTitleShowsBranch(t *testing.T) {
t.Parallel()
testCtx := setupTestHandlers(t)
createdApp, err := testCtx.appSvc.CreateApp(t.Context(), app.CreateAppInput{
Name: "branch-shown-app",
RepoURL: "git@example.com:user/branch-shown-app.git",
Branch: "staging",
})
require.NoError(t, err)
request := httptest.NewRequestWithContext(
t.Context(), http.MethodGet, "/apps/"+createdApp.ID, nil,
)
request = addChiURLParams(request, map[string]string{"id": createdApp.ID})
recorder := httptest.NewRecorder()
testCtx.handlers.HandleAppDetail().ServeHTTP(recorder, request)
require.Equal(t, http.StatusOK, recorder.Code)
// The title row runs from the app name heading to the end of its div.
_, afterHeading, found := strings.Cut(recorder.Body.String(), "<h1")
require.True(t, found, "app page has no heading")
titleRow, _, _ := strings.Cut(afterHeading, "</div>")
assert.Contains(t, titleRow, `x-text="statusLabel"`)
assert.Contains(t, titleRow, ">staging</span>")
}
@@ -0,0 +1,39 @@
package handlers_test
import (
"net/http"
"net/http/httptest"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"sneak.berlin/go/upaas/internal/service/app"
)
// TestAppPageEnvVarHint checks that the environment variable editor says
// changes take effect at the next deploy or rollback, in the warning style.
func TestAppPageEnvVarHint(t *testing.T) {
t.Parallel()
testCtx := setupTestHandlers(t)
createdApp, err := testCtx.appSvc.CreateApp(t.Context(), app.CreateAppInput{
Name: "env-hint-app",
RepoURL: "git@example.com:user/env-hint-app.git",
})
require.NoError(t, err)
request := httptest.NewRequestWithContext(
t.Context(), http.MethodGet, "/apps/"+createdApp.ID, nil,
)
request = addChiURLParams(request, map[string]string{"id": createdApp.ID})
recorder := httptest.NewRecorder()
testCtx.handlers.HandleAppDetail().ServeHTTP(recorder, request)
require.Equal(t, http.StatusOK, recorder.Code)
assert.Contains(t, recorder.Body.String(),
`<p class="alert-warning mt-1">`+
"Environment variable changes take effect at the next deploy or rollback.</p>")
}
+76
View File
@@ -0,0 +1,76 @@
package handlers_test
import (
"net/http"
"net/http/httptest"
"strings"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"sneak.berlin/go/upaas/internal/service/app"
)
// TestAppPageLayout checks the app page's width, the order of its sections,
// and the height of its two log boxes.
func TestAppPageLayout(t *testing.T) {
t.Parallel()
testCtx := setupTestHandlers(t)
createdApp, err := testCtx.appSvc.CreateApp(t.Context(), app.CreateAppInput{
Name: "layout-app",
RepoURL: "git@example.com:user/layout-app.git",
})
require.NoError(t, err)
request := httptest.NewRequestWithContext(
t.Context(), http.MethodGet, "/apps/"+createdApp.ID, nil,
)
request = addChiURLParams(request, map[string]string{"id": createdApp.ID})
recorder := httptest.NewRecorder()
testCtx.handlers.HandleAppDetail().ServeHTTP(recorder, request)
require.Equal(t, http.StatusOK, recorder.Code)
body := recorder.Body.String()
_, afterMain, found := strings.Cut(body, "<main")
require.True(t, found, "app page has no main element")
mainTag, _, _ := strings.Cut(afterMain, ">")
assert.Contains(t, mainTag, "max-width: 84rem;")
sectionTitles := []string{
"Container Logs",
"Deploy Key",
"Webhook URL",
"Last Deployment Build Logs",
"Environment Variables",
"Docker Labels",
"Volume Mounts",
"Port Mappings",
"Recent Deployments",
"Danger Zone",
}
previousIndex := -1
for _, title := range sectionTitles {
index := strings.Index(body, ">"+title+"</h2>")
require.NotEqual(t, -1, index, "app page has no %q section", title)
assert.Greater(t, index, previousIndex, "%q section is out of order", title)
previousIndex = index
}
for _, logBox := range []string{"containerLogsWrapper", "buildLogsWrapper"} {
_, afterRef, found := strings.Cut(body, `x-ref="`+logBox+`"`)
require.True(t, found, "app page has no %s", logBox)
logBoxTag, _, _ := strings.Cut(afterRef, ">")
assert.Contains(t, logBoxTag, "max-height: 800px;", logBox)
}
}
@@ -0,0 +1,54 @@
package handlers_test
import (
"net/http"
"net/http/httptest"
"strings"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"sneak.berlin/go/upaas/internal/service/app"
)
// TestDashboardTableFitsCard checks that the card around the app table
// scrolls sideways instead of hiding what does not fit, and that a long
// repository URL wraps instead of pushing the action buttons out.
func TestDashboardTableFitsCard(t *testing.T) {
t.Parallel()
testCtx := setupTestHandlers(t)
repoURL := "https://git.example.com/user/" +
"a-repository-name-long-enough-to-push-the-action-buttons-out.git"
_, err := testCtx.appSvc.CreateApp(t.Context(), app.CreateAppInput{
Name: "long-url-app",
RepoURL: repoURL,
})
require.NoError(t, err)
request := httptest.NewRequestWithContext(t.Context(), http.MethodGet, "/", nil)
recorder := httptest.NewRecorder()
testCtx.handlers.HandleDashboard().ServeHTTP(recorder, request)
require.Equal(t, http.StatusOK, recorder.Code)
body := recorder.Body.String()
beforeTable, _, found := strings.Cut(body, `<table class="table">`)
require.True(t, found, "dashboard has no app table")
cardTag := beforeTable[strings.LastIndex(beforeTable, "<div"):]
assert.Contains(t, cardTag, "overflow-x-auto")
assert.NotContains(t, cardTag, "overflow-hidden")
beforeURL, _, found := strings.Cut(body, ">"+repoURL+"</td>")
require.True(t, found, "dashboard has no repository cell")
cellTag := beforeURL[strings.LastIndex(beforeURL, "<td"):]
assert.Contains(t, cellTag, "whitespace-normal")
assert.Contains(t, cellTag, "break-all")
}
+68
View File
@@ -8,6 +8,7 @@ import (
"strconv" "strconv"
"strings" "strings"
"testing" "testing"
"time"
"github.com/go-chi/chi/v5" "github.com/go-chi/chi/v5"
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
@@ -1148,6 +1149,73 @@ func TestHandleCancelDeployReturns404ForUnknownApp(t *testing.T) {
assert.Equal(t, http.StatusNotFound, recorder.Code) assert.Equal(t, http.StatusNotFound, recorder.Code)
} }
// TestHandleAppDeploymentsShowsTenNewest verifies the deployments page
// lists only the 10 most recent deployments, newest first.
func TestHandleAppDeploymentsShowsTenNewest(t *testing.T) {
t.Parallel()
testCtx := setupTestHandlers(t)
createdApp := createTestApp(t, testCtx, "deployments-page-app")
// Create 12 deployments, each started one minute after the one before.
firstStart := time.Date(2026, 1, 1, 0, 0, 0, 0, time.UTC)
ids := make([]int64, 0, 12)
for idx := range 12 {
deployment := models.NewDeployment(testCtx.database)
deployment.AppID = createdApp.ID
deployment.Status = models.DeploymentStatusSuccess
require.NoError(t, deployment.Save(context.Background()))
_, err := testCtx.database.Exec(
context.Background(),
"UPDATE deployments SET started_at = ? WHERE id = ?",
firstStart.Add(time.Duration(idx)*time.Minute),
deployment.ID,
)
require.NoError(t, err)
ids = append(ids, deployment.ID)
}
request := httptest.NewRequestWithContext(
t.Context(),
http.MethodGet,
"/apps/"+createdApp.ID+"/deployments",
nil,
)
request = addChiURLParams(request, map[string]string{"id": createdApp.ID})
recorder := httptest.NewRecorder()
handler := testCtx.handlers.HandleAppDeployments()
handler.ServeHTTP(recorder, request)
require.Equal(t, http.StatusOK, recorder.Code)
body := recorder.Body.String()
card := func(id int64) string {
return `data-deployment-id="` + strconv.FormatInt(id, 10) + `"`
}
assert.Equal(t, 10, strings.Count(body, `data-deployment-id="`))
// The two oldest are left out.
assert.NotContains(t, body, card(ids[0]))
assert.NotContains(t, body, card(ids[1]))
// The ten newest are shown, newest first.
previous := -1
for idx := len(ids) - 1; idx >= 2; idx-- {
position := strings.Index(body, card(ids[idx]))
require.Greater(t, position, previous,
"deployment %d missing or out of order", ids[idx])
previous = position
}
}
func TestHandleWebhookReturns404ForUnknownSecret(t *testing.T) { func TestHandleWebhookReturns404ForUnknownSecret(t *testing.T) {
t.Parallel() t.Parallel()
+2 -1
View File
@@ -4,6 +4,7 @@ package logger
import ( import (
"log/slog" "log/slog"
"os" "os"
"runtime"
"go.uber.org/fx" "go.uber.org/fx"
@@ -81,6 +82,6 @@ func (l *Logger) Identify() {
l.log.Info("starting", l.log.Info("starting",
"appname", l.params.Globals.Appname, "appname", l.params.Globals.Appname,
"version", l.params.Globals.Version, "version", l.params.Globals.Version,
"buildarch", l.params.Globals.Buildarch, "arch", runtime.GOARCH,
) )
} }
+34
View File
@@ -0,0 +1,34 @@
package logger //nolint:testpackage // sets the unexported log and params fields
import (
"bytes"
"encoding/json"
"log/slog"
"runtime"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"sneak.berlin/go/upaas/internal/globals"
)
func TestIdentifyLogsArchitectureFromRuntime(t *testing.T) {
t.Parallel()
var buf bytes.Buffer
l := &Logger{
log: slog.New(slog.NewJSONHandler(&buf, nil)),
params: Params{
Globals: &globals.Globals{Appname: "upaas-test", Version: "test"},
},
}
l.Identify()
var line map[string]any
require.NoError(t, json.Unmarshal(buf.Bytes(), &line))
assert.Equal(t, runtime.GOARCH, line["arch"])
}
+42 -2
View File
@@ -203,11 +203,12 @@ func (d *Deployment) insert(ctx context.Context) error {
func (d *Deployment) update(ctx context.Context) error { func (d *Deployment) update(ctx context.Context) error {
query := ` query := `
UPDATE deployments SET UPDATE deployments SET
image_id = ?, container_id = ?, status = ?, logs = ?, finished_at = ? commit_sha = ?, image_id = ?, container_id = ?, status = ?, logs = ?,
finished_at = ?
WHERE id = ?` WHERE id = ?`
_, err := d.db.Exec(ctx, query, _, err := d.db.Exec(ctx, query,
d.ImageID, d.ContainerID, d.Status, d.Logs, d.FinishedAt, d.ID, d.CommitSHA, d.ImageID, d.ContainerID, d.Status, d.Logs, d.FinishedAt, d.ID,
) )
return err return err
@@ -295,6 +296,45 @@ func FindDeploymentsByAppID(
return deployments, nil return deployments, nil
} }
// FindDeploymentImageIDs returns the IDs of the images an app's
// deployments built or rolled back to.
func FindDeploymentImageIDs(
ctx context.Context,
deployDB *database.Database,
appID string,
) ([]string, error) {
rows, err := deployDB.Query(ctx, `
SELECT DISTINCT image_id FROM deployments
WHERE app_id = ? AND image_id IS NOT NULL`,
appID,
)
if err != nil {
return nil, fmt.Errorf("querying deployment image IDs: %w", err)
}
defer func() { _ = rows.Close() }()
var imageIDs []string
for rows.Next() {
var imageID string
scanErr := rows.Scan(&imageID)
if scanErr != nil {
return nil, fmt.Errorf("scanning deployment image ID: %w", scanErr)
}
imageIDs = append(imageIDs, imageID)
}
rowsErr := rows.Err()
if rowsErr != nil {
return nil, fmt.Errorf("iterating deployment image IDs: %w", rowsErr)
}
return imageIDs, nil
}
// LatestDeploymentForApp finds the most recent deployment for an app. // LatestDeploymentForApp finds the most recent deployment for an app.
// //
//nolint:nilnil // returning nil,nil is idiomatic for "not found" in Active Record //nolint:nilnil // returning nil,nil is idiomatic for "not found" in Active Record
+27
View File
@@ -564,6 +564,33 @@ func TestDeploymentMarkFinished(t *testing.T) {
assert.True(t, found.FinishedAt.Valid) assert.True(t, found.FinishedAt.Valid)
} }
// TestDeploymentSaveStoresCommitSetAfterInsert checks that a commit set on a
// deployment after it was first saved, as a manual deploy does once the clone
// reads it, is stored by the next save.
func TestDeploymentSaveStoresCommitSetAfterInsert(t *testing.T) {
t.Parallel()
testDB, cleanup := setupTestDB(t)
defer cleanup()
app := createTestApp(t, testDB)
deployment := models.NewDeployment(testDB)
deployment.AppID = app.ID
err := deployment.Save(context.Background())
require.NoError(t, err)
deployment.CommitSHA = sql.NullString{String: "abc123def456", Valid: true}
err = deployment.Save(context.Background())
require.NoError(t, err)
found, err := models.FindDeployment(context.Background(), testDB, deployment.ID)
require.NoError(t, err)
assert.Equal(t, "abc123def456", found.CommitSHA.String)
}
func TestDeploymentFindByAppID(t *testing.T) { func TestDeploymentFindByAppID(t *testing.T) {
t.Parallel() t.Parallel()
+85 -20
View File
@@ -735,44 +735,103 @@ func (svc *Service) recordDeployedImage(
return nil return nil
} }
// removeUnusedImages removes the app's tags (upaas-<app>:<deployment>, set by // removeUnusedImages removes the app's images except those an app's running
// buildImage) except those of the image the running container uses and the // container uses or its Rollback would start. Docker deletes a tagged image
// one Rollback would start. Docker deletes an image only once no other tag, // only once no other tag, such as another app's, and no container still
// such as another app's, and no container still uses it. // uses it.
func (svc *Service) removeUnusedImages( func (svc *Service) removeUnusedImages(
ctx context.Context, ctx context.Context,
app *models.App, app *models.App,
deployment *models.Deployment, deployment *models.Deployment,
) { ) {
tags, err := svc.docker.ListImageTags(ctx, "upaas-"+app.Name) images, err := svc.findAppImages(ctx, app)
if err != nil { if err != nil {
svc.log.Error("failed to list app images", "error", err, "app", app.Name) svc.log.Error("failed to list app images", "error", err, "app", app.Name)
return return
} }
for _, tag := range slices.Sorted(maps.Keys(tags)) { keep, err := svc.imagesToKeep(ctx)
imageID := tags[tag].String() if err != nil {
if imageID == app.ImageID.String || imageID == app.PreviousImageID.String { svc.log.Error("failed to list the images apps use", "error", err, "app", app.Name)
return
}
for _, name := range slices.Sorted(maps.Keys(images)) {
if keep[images[name].String()] {
continue continue
} }
removeErr := svc.docker.RemoveImageTag(ctx, tag) removeErr := svc.docker.RemoveImageTag(ctx, name)
if removeErr != nil { if removeErr != nil {
svc.log.Error("failed to remove old image", svc.log.Error("failed to remove old image",
"error", removeErr, "app", app.Name, "tag", tag) "error", removeErr, "app", app.Name, "image", name)
_ = deployment.AppendLog( _ = deployment.AppendLog(
ctx, ctx,
"WARNING: failed to remove old image "+tag+": "+removeErr.Error(), "WARNING: failed to remove old image "+name+": "+removeErr.Error(),
) )
continue continue
} }
_ = deployment.AppendLog(ctx, "Removed old image: "+tag) _ = deployment.AppendLog(ctx, "Removed old image: "+name)
} }
} }
// findAppImages returns the app's images, each under the name it is removed
// by: its tag, upaas-<app>:<short hash> as set by buildImage, or its ID if
// it has none. A redeploy of a commit gives the commit's tag to the new
// image, so the old one is found among the images the app's deployments
// recorded.
func (svc *Service) findAppImages(
ctx context.Context,
app *models.App,
) (map[string]docker.ImageID, error) {
images, err := svc.docker.ListImageTags(ctx, "upaas-"+app.Name)
if err != nil {
return nil, fmt.Errorf("failed to list image tags: %w", err)
}
untagged, err := svc.docker.ListUntaggedImages(ctx)
if err != nil {
return nil, fmt.Errorf("failed to list untagged images: %w", err)
}
recorded, err := models.FindDeploymentImageIDs(ctx, svc.db, app.ID)
if err != nil {
return nil, fmt.Errorf("failed to find deployment images: %w", err)
}
for _, imageID := range untagged {
if slices.Contains(recorded, imageID.String()) {
images[imageID.String()] = imageID
}
}
return images, nil
}
// imagesToKeep returns the IDs of the image each app's running container
// uses and the one its Rollback would start. It covers every app because
// apps that build the same commit can share an image, and a redeploy can
// take the tag that kept the image for one of them.
func (svc *Service) imagesToKeep(ctx context.Context) (map[string]bool, error) {
apps, err := models.AllApps(ctx, svc.db)
if err != nil {
return nil, fmt.Errorf("failed to list apps: %w", err)
}
keep := make(map[string]bool)
for _, app := range apps {
keep[app.ImageID.String] = true
keep[app.PreviousImageID.String] = true
}
return keep, nil
}
// cleanupCancelledDeploy removes orphan resources left by a cancelled deployment. // cleanupCancelledDeploy removes orphan resources left by a cancelled deployment.
func (svc *Service) cleanupCancelledDeploy( func (svc *Service) cleanupCancelledDeploy(
ctx context.Context, ctx context.Context,
@@ -912,18 +971,17 @@ func (svc *Service) buildImage(
app *models.App, app *models.App,
deployment *models.Deployment, deployment *models.Deployment,
) (docker.ImageID, error) { ) (docker.ImageID, error) {
workDir, cleanup, err := svc.cloneRepository(ctx, app, deployment) workDir, shortSHA, cleanup, err := svc.cloneRepository(ctx, app, deployment)
if err != nil { if err != nil {
return "", err return "", err
} }
defer cleanup() defer cleanup()
imageTag := fmt.Sprintf("upaas-%s:%d", app.Name, deployment.ID) imageTag := "upaas-" + app.Name + ":" + shortSHA
// Create log writer that flushes build output to deployment logs every second // Create log writer that flushes build output to deployment logs every second
logWriter := newDeploymentLogWriter(ctx, deployment) logWriter := newDeploymentLogWriter(ctx, deployment)
defer logWriter.Close()
// BuildImage creates a tar archive from the local filesystem, // BuildImage creates a tar archive from the local filesystem,
// so it needs the container path where files exist, not the host path. // so it needs the container path where files exist, not the host path.
@@ -933,6 +991,10 @@ func (svc *Service) buildImage(
Tags: []string{imageTag}, Tags: []string{imageTag},
LogWriter: logWriter, LogWriter: logWriter,
}) })
// Write the rest of the build output to the log before the result.
logWriter.Close()
if err != nil { if err != nil {
svc.notify.NotifyBuildFailed(ctx, app, deployment, err) svc.notify.NotifyBuildFailed(ctx, app, deployment, err)
svc.failDeployment( svc.failDeployment(
@@ -951,11 +1013,14 @@ func (svc *Service) buildImage(
return imageID, nil return imageID, nil
} }
// cloneRepository clones the app's repository for a build. It returns the
// directory of the clone, git's short form of the commit checked out, and a
// function that removes the clone.
func (svc *Service) cloneRepository( func (svc *Service) cloneRepository(
ctx context.Context, ctx context.Context,
app *models.App, app *models.App,
deployment *models.Deployment, deployment *models.Deployment,
) (string, func(), error) { ) (string, string, func(), error) {
// Use a subdirectory of DataDir for builds since it's mounted from the host // Use a subdirectory of DataDir for builds since it's mounted from the host
// and accessible to Docker for bind mounts (unlike /tmp inside the container). // and accessible to Docker for bind mounts (unlike /tmp inside the container).
// Structure: builds/<appname>/<deployment-id>-<random>/ // Structure: builds/<appname>/<deployment-id>-<random>/
@@ -972,7 +1037,7 @@ func (svc *Service) cloneRepository(
fmt.Errorf("failed to create builds dir: %w", err), fmt.Errorf("failed to create builds dir: %w", err),
) )
return "", nil, fmt.Errorf("failed to create builds dir: %w", err) return "", "", nil, fmt.Errorf("failed to create builds dir: %w", err)
} }
buildDir, err := os.MkdirTemp(appBuildsDir, fmt.Sprintf("%d-*", deployment.ID)) buildDir, err := os.MkdirTemp(appBuildsDir, fmt.Sprintf("%d-*", deployment.ID))
@@ -984,7 +1049,7 @@ func (svc *Service) cloneRepository(
fmt.Errorf("failed to create temp dir: %w", err), fmt.Errorf("failed to create temp dir: %w", err),
) )
return "", nil, fmt.Errorf("failed to create temp dir: %w", err) return "", "", nil, fmt.Errorf("failed to create temp dir: %w", err)
} }
cleanup := func() { _ = os.RemoveAll(buildDir) } cleanup := func() { _ = os.RemoveAll(buildDir) }
@@ -1020,7 +1085,7 @@ func (svc *Service) cloneRepository(
fmt.Errorf("failed to clone repo: %w", cloneErr), fmt.Errorf("failed to clone repo: %w", cloneErr),
) )
return "", nil, fmt.Errorf("failed to clone repo: %w", cloneErr) return "", "", nil, fmt.Errorf("failed to clone repo: %w", cloneErr)
} }
svc.processCloneResult(ctx, app, deployment, cloneResult, commitSHA) svc.processCloneResult(ctx, app, deployment, cloneResult, commitSHA)
@@ -1028,7 +1093,7 @@ func (svc *Service) cloneRepository(
// Return the 'work' subdirectory where the repo was cloned // Return the 'work' subdirectory where the repo was cloned
workDir := filepath.Join(buildDir, "work") workDir := filepath.Join(buildDir, "work")
return workDir, cleanup, nil return workDir, cloneResult.ShortSHA, cleanup, nil
} }
// processCloneResult handles the result of a git clone operation. // processCloneResult handles the result of a git clone operation.
@@ -0,0 +1,93 @@
package deploy_test
import (
"context"
"log/slog"
"net/http"
"net/http/httptest"
"os"
"strings"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"go.uber.org/fx/fxtest"
"sneak.berlin/go/upaas/internal/config"
"sneak.berlin/go/upaas/internal/database"
"sneak.berlin/go/upaas/internal/docker"
"sneak.berlin/go/upaas/internal/logger"
"sneak.berlin/go/upaas/internal/models"
"sneak.berlin/go/upaas/internal/service/deploy"
)
// TestBuildImageLogsBuildErrorBeforeDeployError runs a build that fails
// against a fake Docker API and checks that the deploy fails with the
// build's own error, which the deployment log shows before the deploy's.
func TestBuildImageLogsBuildErrorBeforeDeployError(t *testing.T) {
t.Parallel()
srv := httptest.NewServer(http.HandlerFunc(
func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
switch {
case strings.HasSuffix(r.URL.Path, "/containers/create"):
_, _ = w.Write([]byte(`{"Id":"gitcontainer"}`))
case strings.HasSuffix(r.URL.Path, "/logs"):
writeCloneOutput(w, "abc1234")
case strings.HasSuffix(r.URL.Path, "/version"):
_, _ = w.Write([]byte(`{"Version":"27.3.1","ApiVersion":"1.47"}`))
case strings.HasSuffix(r.URL.Path, "/build"):
_, _ = w.Write([]byte(`{"stream":"Step 1/1 : RUN false\n"}` + "\n" +
`{"errorDetail":{"message":"exit code: 1"},"error":"exit code: 1"}`))
default:
// The steps of the git clone, which succeeds.
_, _ = w.Write([]byte(`{}`))
}
},
))
t.Cleanup(srv.Close)
log := slog.New(slog.NewTextHandler(os.Stderr, nil))
lifecycle := fxtest.NewLifecycle(t)
dockerClient, err := docker.New(lifecycle, docker.Params{
Logger: logger.NewForTest(log),
Config: &config.Config{DockerHost: "tcp://" + srv.Listener.Addr().String()},
})
require.NoError(t, err)
lifecycle.RequireStart()
t.Cleanup(lifecycle.RequireStop)
db := database.NewTestDatabase(t)
ctx := context.Background()
app := models.NewApp(db)
app.ID = "buildapp-id"
app.Name = "buildapp"
app.Branch = "main"
require.NoError(t, app.Save(ctx))
deployment := models.NewDeployment(db)
deployment.AppID = app.ID
require.NoError(t, deployment.Save(ctx))
dataDir := t.TempDir()
cfg := &config.Config{DataDir: dataDir, HostDataDir: dataDir}
// The service has no notify service: the app has no ntfy topic and no
// Slack webhook, so the build failure notification sends nothing.
svc := deploy.NewTestServiceWithConfig(log, cfg, db, dockerClient)
_, err = svc.BuildImage(ctx, app, deployment)
require.EqualError(t, err, "failed to build image: exit code: 1")
logs := deployment.Logs.String
buildError := strings.Index(logs, "ERROR: exit code: 1")
deployError := strings.Index(logs, "ERROR: failed to build image: exit code: 1")
require.NotEqual(t, -1, buildError, logs)
assert.Less(t, buildError, deployError, logs)
}
@@ -20,7 +20,7 @@ func TestCleanupCancelledDeploy_RemovesBuildDir(t *testing.T) {
tmpDir := t.TempDir() tmpDir := t.TempDir()
cfg := &config.Config{DataDir: tmpDir} cfg := &config.Config{DataDir: tmpDir}
svc := deploy.NewTestServiceWithConfig(slog.Default(), cfg, nil) svc := deploy.NewTestServiceWithConfig(slog.Default(), cfg, nil, nil)
// Create a fake build directory matching the deployment pattern // Create a fake build directory matching the deployment pattern
appName := "test-app" appName := "test-app"
@@ -59,7 +59,7 @@ func TestCleanupCancelledDeploy_NoBuildDir(t *testing.T) {
tmpDir := t.TempDir() tmpDir := t.TempDir()
cfg := &config.Config{DataDir: tmpDir} cfg := &config.Config{DataDir: tmpDir}
svc := deploy.NewTestServiceWithConfig(slog.Default(), cfg, nil) svc := deploy.NewTestServiceWithConfig(slog.Default(), cfg, nil, nil)
// Should not panic when build dir doesn't exist // Should not panic when build dir doesn't exist
svc.CleanupCancelledDeploy(context.Background(), "nonexistent-app", 1, "") svc.CleanupCancelledDeploy(context.Background(), "nonexistent-app", 1, "")
+317 -43
View File
@@ -3,14 +3,21 @@ package deploy_test
import ( import (
"context" "context"
"database/sql" "database/sql"
"encoding/json"
"fmt"
"io"
"log/slog" "log/slog"
"maps"
"net/http" "net/http"
"net/http/httptest" "net/http/httptest"
"os" "os"
"slices"
"strings" "strings"
"sync" "sync"
"testing" "testing"
"github.com/docker/docker/api/types/image"
"github.com/docker/docker/pkg/stdcopy"
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
"go.uber.org/fx/fxtest" "go.uber.org/fx/fxtest"
@@ -23,45 +30,139 @@ import (
"sneak.berlin/go/upaas/internal/service/deploy" "sneak.berlin/go/upaas/internal/service/deploy"
) )
// TestRecordDeployedImageRemovesOldImages runs the step after a deploy // fakeImageAPI is a fake Docker API that keeps images and their tags as
// against a fake Docker API. Image one is also tagged for another app, // Docker does: a build gives its tag to the image it builds, and removing
// image two was the previous image, three the current one, four is new. // an image's last tag, or an untagged image by its ID, deletes the image.
func TestRecordDeployedImageRemovesOldImages(t *testing.T) { // It also answers the steps of a git clone that reports shortSHA.
t.Parallel() type fakeImageAPI struct {
var (
mu sync.Mutex mu sync.Mutex
removed []string images map[string][]string // image ID -> tags
forced bool shortSHA string // the commit's short hash the clone reports
) nextID string // ID of the image the next build creates
removed []string // each tag or ID removed
forced bool // whether a removal was forced
}
func (api *fakeImageAPI) ServeHTTP(w http.ResponseWriter, r *http.Request) {
api.mu.Lock()
defer api.mu.Unlock()
srv := httptest.NewServer(http.HandlerFunc(
func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json") w.Header().Set("Content-Type", "application/json")
_, name, isImage := strings.Cut(r.URL.Path, "/images/")
switch { switch {
case r.Method == http.MethodDelete:
_, name, _ := strings.Cut(r.URL.Path, "/images/")
mu.Lock()
removed = append(removed, name)
forced = forced || r.URL.Query().Get("force") != ""
mu.Unlock()
_, _ = w.Write([]byte(`[]`))
case strings.HasSuffix(r.URL.Path, "/images/json"): case strings.HasSuffix(r.URL.Path, "/images/json"):
_, _ = w.Write([]byte(`[ dangling := strings.Contains(r.URL.Query().Get("filters"), "dangling")
{"Id":"sha256:one","RepoTags":["upaas-myapp:1","upaas-otherapp:7"]}, api.listImages(w, dangling)
{"Id":"sha256:two","RepoTags":["upaas-myapp:2"]}, case isImage && r.Method == http.MethodDelete:
{"Id":"sha256:three","RepoTags":["upaas-myapp:3"]}, api.forced = api.forced || r.URL.Query().Get("force") != ""
{"Id":"sha256:four","RepoTags":["upaas-myapp:4"]} api.removeImage(w, name)
]`)) case isImage && strings.HasSuffix(name, "/json"):
api.inspectImage(w, strings.TrimSuffix(name, "/json"))
case strings.HasSuffix(r.URL.Path, "/build"):
tag := r.URL.Query().Get("t")
api.untag(tag)
api.images[api.nextID] = append(api.images[api.nextID], tag)
case strings.HasSuffix(r.URL.Path, "/version"):
_, _ = w.Write([]byte(`{"Version":"27.3.1","ApiVersion":"1.47"}`))
case strings.HasSuffix(r.URL.Path, "/containers/create"):
_, _ = w.Write([]byte(`{"Id":"gitcontainer"}`))
case strings.HasSuffix(r.URL.Path, "/logs"):
writeCloneOutput(w, api.shortSHA)
default: default:
// The other steps of the git clone, which succeeds.
_, _ = w.Write([]byte(`{}`)) _, _ = w.Write([]byte(`{}`))
} }
}, }
))
// listImages lists the untagged images, or else the tagged ones.
func (api *fakeImageAPI) listImages(w http.ResponseWriter, dangling bool) {
list := []image.Summary{}
for _, id := range slices.Sorted(maps.Keys(api.images)) {
if (len(api.images[id]) == 0) == dangling {
list = append(list, image.Summary{ID: id, RepoTags: api.images[id]})
}
}
data, err := json.Marshal(list)
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
_, _ = w.Write(data)
}
func (api *fakeImageAPI) inspectImage(w http.ResponseWriter, name string) {
for id, tags := range api.images {
if id == name || slices.Contains(tags, name) {
_, _ = fmt.Fprintf(w, `{"Id":%q}`, id)
return
}
}
w.WriteHeader(http.StatusNotFound)
_, _ = w.Write([]byte(`{"message":"No such image"}`))
}
func (api *fakeImageAPI) removeImage(w http.ResponseWriter, name string) {
api.removed = append(api.removed, name)
id := name
if _, isID := api.images[name]; !isID {
id = api.untag(name)
}
if len(api.images[id]) == 0 {
delete(api.images, id)
}
_, _ = w.Write([]byte(`[]`))
}
// untag removes tag from the image that has it, leaving the image, and
// returns the image's ID.
func (api *fakeImageAPI) untag(tag string) string {
for id, tags := range api.images {
if slices.Contains(tags, tag) {
api.images[id] = slices.DeleteFunc(tags, func(t string) bool { return t == tag })
return id
}
}
return ""
}
// state returns each image's tags and each tag or ID removed.
func (api *fakeImageAPI) state() (map[string][]string, []string) {
api.mu.Lock()
defer api.mu.Unlock()
return maps.Clone(api.images), slices.Clone(api.removed)
}
// writeCloneOutput writes the line of a git clone's output that gives the
// commit's short hash, as Docker sends a container's log: after a header.
func writeCloneOutput(w io.Writer, shortSHA string) {
out := stdcopy.NewStdWriter(w, stdcopy.Stdout)
_, _ = fmt.Fprintf(out, "SHORT_SHA:%s\n", shortSHA)
}
// newImageTestService returns a deploy service that uses api as its
// Docker API, and its database.
func newImageTestService(
t *testing.T,
api *fakeImageAPI,
) (*deploy.Service, *database.Database) {
t.Helper()
srv := httptest.NewServer(api)
t.Cleanup(srv.Close) t.Cleanup(srv.Close)
log := slog.New(slog.NewTextHandler(os.Stderr, nil)) log := slog.New(slog.NewTextHandler(os.Stderr, nil))
@@ -77,30 +178,203 @@ func TestRecordDeployedImageRemovesOldImages(t *testing.T) {
t.Cleanup(lifecycle.RequireStop) t.Cleanup(lifecycle.RequireStop)
db := database.NewTestDatabase(t) db := database.NewTestDatabase(t)
ctx := context.Background() dataDir := t.TempDir()
cfg := &config.Config{DataDir: dataDir, HostDataDir: dataDir}
return deploy.NewTestServiceWithConfig(log, cfg, db, dockerClient), db
}
// saveApp saves an app with the given current and previous image.
func saveApp(
t *testing.T,
db *database.Database,
name, imageID, previousImageID string,
) *models.App {
t.Helper()
app := models.NewApp(db) app := models.NewApp(db)
app.ID = "myapp-id" app.ID = name + "-id"
app.Name = "myapp" app.Name = name
app.ImageID = sql.NullString{String: "sha256:three", Valid: true} app.ImageID = sql.NullString{String: imageID, Valid: true}
app.PreviousImageID = sql.NullString{String: "sha256:two", Valid: true} app.PreviousImageID = sql.NullString{String: previousImageID, Valid: true}
require.NoError(t, app.Save(ctx)) require.NoError(t, app.Save(context.Background()))
return app
}
// TestRecordDeployedImageRemovesOldImages runs the step after a deploy
// against a fake Docker API. Image one has a tag from before images were
// tagged with their commit, and is also tagged for another app. Image two
// was the previous image, three the current one, four is new. Image five is
// the other app's previous image, which a redeploy of its commit left
// without the other app's tag.
func TestRecordDeployedImageRemovesOldImages(t *testing.T) {
t.Parallel()
api := &fakeImageAPI{images: map[string][]string{
"sha256:one": {"upaas-myapp:140", "upaas-otherapp:1a2b3c4"},
"sha256:two": {"upaas-myapp:2b3c4d5"},
"sha256:three": {"upaas-myapp:3c4d5e6"},
"sha256:four": {"upaas-myapp:4d5e6f7"},
"sha256:five": {"upaas-myapp:5e6f7a8"},
}}
svc, db := newImageTestService(t, api)
ctx := context.Background()
app := saveApp(t, db, "myapp", "sha256:three", "sha256:two")
saveApp(t, db, "otherapp", "sha256:other", "sha256:five")
deployment := models.NewDeployment(db) deployment := models.NewDeployment(db)
deployment.AppID = app.ID deployment.AppID = app.ID
require.NoError(t, deployment.Save(ctx)) require.NoError(t, deployment.Save(ctx))
svc := deploy.NewTestServiceWithConfig(log, &config.Config{}, dockerClient) err := svc.RecordDeployedImage(ctx, app, deployment, "sha256:four")
err = svc.RecordDeployedImage(ctx, app, deployment, "sha256:four")
require.NoError(t, err) require.NoError(t, err)
assert.Equal(t, "sha256:four", app.ImageID.String) assert.Equal(t, "sha256:four", app.ImageID.String)
assert.Equal(t, "sha256:three", app.PreviousImageID.String) assert.Equal(t, "sha256:three", app.PreviousImageID.String)
mu.Lock() images, removed := api.state()
defer mu.Unlock()
assert.Equal(t, []string{"upaas-myapp:1", "upaas-myapp:2"}, removed) assert.Equal(t, []string{"upaas-myapp:140", "upaas-myapp:2b3c4d5"}, removed)
assert.False(t, forced, "old image tags must be removed without force") assert.Equal(t, map[string][]string{
"sha256:one": {"upaas-otherapp:1a2b3c4"},
"sha256:three": {"upaas-myapp:3c4d5e6"},
"sha256:four": {"upaas-myapp:4d5e6f7"},
"sha256:five": {"upaas-myapp:5e6f7a8"},
}, images)
assert.False(t, api.forced, "old images must be removed without force")
}
// TestRecordDeployedImageRemovesOnlyTheAppsUntaggedImages runs the step after
// a deploy against a fake Docker API that holds three untagged images: one an
// earlier deployment of the app recorded, one a deployment of another app
// recorded, and one no deployment recorded, such as an image upaas never
// built. Only the app's own is removed.
func TestRecordDeployedImageRemovesOnlyTheAppsUntaggedImages(t *testing.T) {
t.Parallel()
api := &fakeImageAPI{images: map[string][]string{
"sha256:new": {"upaas-myapp:1a2b3c4"},
"sha256:myapp": {},
"sha256:otherapp": {},
"sha256:unknown": {},
}}
svc, db := newImageTestService(t, api)
ctx := context.Background()
app := saveApp(t, db, "myapp", "", "")
otherApp := saveApp(t, db, "otherapp", "", "")
saveDeployment := func(appID, imageID string) *models.Deployment {
t.Helper()
deployment := models.NewDeployment(db)
deployment.AppID = appID
deployment.ImageID = sql.NullString{String: imageID, Valid: true}
require.NoError(t, deployment.Save(ctx))
return deployment
}
saveDeployment(app.ID, "sha256:myapp")
saveDeployment(otherApp.ID, "sha256:otherapp")
deployment := saveDeployment(app.ID, "sha256:new")
err := svc.RecordDeployedImage(ctx, app, deployment, "sha256:new")
require.NoError(t, err)
images, removed := api.state()
assert.Equal(t, []string{"sha256:myapp"}, removed)
assert.Equal(t, map[string][]string{
"sha256:new": {"upaas-myapp:1a2b3c4"},
"sha256:otherapp": {},
"sha256:unknown": {},
}, images)
}
// TestRedeployRemovesImagesLeftWithoutTag deploys commits against a fake
// Docker API, some of them again. A build takes the commit's tag from the
// image an earlier build of it made. That image is kept, without a tag,
// while the app runs it or Rollback would start it, and is removed by its
// ID once neither does.
func TestRedeployRemovesImagesLeftWithoutTag(t *testing.T) {
t.Parallel()
const (
tagABC1234 = "upaas-myapp:abc1234"
tag0123ABC = "upaas-myapp:0123abc"
retriedImage = "sha256:retried-0123abc"
)
// The app runs commit abc1234 and would roll back to def5678. The last
// deploy, of commit 0123abc, failed after its build.
api := &fakeImageAPI{images: map[string][]string{
"sha256:built-abc1234": {tagABC1234},
"sha256:built-def5678": {"upaas-myapp:def5678"},
"sha256:failed-0123abc": {tag0123ABC},
}}
svc, db := newImageTestService(t, api)
ctx := context.Background()
app := saveApp(t, db, "myapp", "sha256:built-abc1234", "sha256:built-def5678")
for imageID := range api.images {
deployment := models.NewDeployment(db)
deployment.AppID = app.ID
deployment.ImageID = sql.NullString{String: imageID, Valid: true}
require.NoError(t, deployment.Save(ctx))
}
deployCommit := func(shortSHA, imageID string) {
t.Helper()
api.mu.Lock()
api.shortSHA = shortSHA
api.nextID = imageID
api.mu.Unlock()
deployment := models.NewDeployment(db)
deployment.AppID = app.ID
require.NoError(t, deployment.Save(ctx))
built, err := svc.BuildImage(ctx, app, deployment)
require.NoError(t, err)
require.NoError(t, svc.RecordDeployedImage(ctx, app, deployment, built))
}
// The failed deploy's image loses its tag, and nothing uses it.
deployCommit("0123abc", retriedImage)
images, _ := api.state()
assert.Equal(t, map[string][]string{
"sha256:built-abc1234": {tagABC1234},
retriedImage: {tag0123ABC},
}, images)
// The running image loses its tag and becomes the one Rollback starts.
deployCommit("0123abc", "sha256:rebuilt-0123abc")
images, _ = api.state()
assert.Equal(t, map[string][]string{
"sha256:rebuilt-0123abc": {tag0123ABC},
retriedImage: {},
}, images)
assert.Equal(t, retriedImage, app.PreviousImageID.String)
// Once Rollback no longer needs it, the untagged image is removed.
deployCommit("4567def", "sha256:built-4567def")
images, removed := api.state()
assert.Equal(t, map[string][]string{
"sha256:built-4567def": {"upaas-myapp:4567def"},
"sha256:rebuilt-0123abc": {tag0123ABC},
}, images)
assert.Equal(t, []string{
"sha256:failed-0123abc", "upaas-myapp:def5678", // first deploy
tagABC1234, // second deploy
retriedImage, // third deploy
}, removed)
assert.False(t, api.forced, "old images must be removed without force")
} }
+14 -1
View File
@@ -9,6 +9,7 @@ import (
"strings" "strings"
"sneak.berlin/go/upaas/internal/config" "sneak.berlin/go/upaas/internal/config"
"sneak.berlin/go/upaas/internal/database"
"sneak.berlin/go/upaas/internal/docker" "sneak.berlin/go/upaas/internal/docker"
"sneak.berlin/go/upaas/internal/models" "sneak.berlin/go/upaas/internal/models"
) )
@@ -44,15 +45,18 @@ func (svc *Service) UnlockApp(appID string) {
svc.unlockApp(appID) svc.unlockApp(appID)
} }
// NewTestServiceWithConfig creates a Service with config and docker client for testing. // NewTestServiceWithConfig creates a Service with config, database and
// docker client for testing.
func NewTestServiceWithConfig( func NewTestServiceWithConfig(
log *slog.Logger, log *slog.Logger,
cfg *config.Config, cfg *config.Config,
db *database.Database,
dockerClient *docker.Client, dockerClient *docker.Client,
) *Service { ) *Service {
return &Service{ return &Service{
log: log, log: log,
config: cfg, config: cfg,
db: db,
docker: dockerClient, docker: dockerClient,
} }
} }
@@ -100,6 +104,15 @@ func (svc *Service) RecordDeployedImage(
return svc.recordDeployedImage(ctx, app, deployment, imageID) return svc.recordDeployedImage(ctx, app, deployment, imageID)
} }
// BuildImage exposes buildImage for testing.
func (svc *Service) BuildImage(
ctx context.Context,
app *models.App,
deployment *models.Deployment,
) (docker.ImageID, error) {
return svc.buildImage(ctx, app, deployment)
}
// BuildContainerOptionsExported exposes buildContainerOptions for testing. // BuildContainerOptionsExported exposes buildContainerOptions for testing.
func (svc *Service) BuildContainerOptionsExported( func (svc *Service) BuildContainerOptionsExported(
ctx context.Context, ctx context.Context,
+45 -44
View File
@@ -5,7 +5,7 @@
{{define "content"}} {{define "content"}}
{{template "nav" .}} {{template "nav" .}}
<main class="max-w-4xl mx-auto px-4 py-8" x-data="appDetail({ <main class="mx-auto px-4 py-8" style="max-width: 84rem;" x-data="appDetail({
appId: '{{.App.ID}}', appId: '{{.App.ID}}',
initialDeploymentId: {{if .LatestDeployment}}{{.LatestDeployment.ID}}{{else}}null{{end}}, initialDeploymentId: {{if .LatestDeployment}}{{.LatestDeployment.ID}}{{else}}null{{end}},
initialStatus: '{{.App.Status}}', initialStatus: '{{.App.Status}}',
@@ -26,11 +26,12 @@
<!-- Header --> <!-- Header -->
<div class="flex flex-col sm:flex-row sm:items-center sm:justify-between gap-4 mb-8"> <div class="flex flex-col sm:flex-row sm:items-center sm:justify-between gap-4 mb-8">
<div> <div>
<div class="flex items-center gap-3"> <div class="flex flex-wrap items-center gap-3">
<h1 class="text-2xl font-medium text-gray-900">{{.App.Name}}</h1> <h1 class="text-2xl font-medium text-gray-900">{{.App.Name}}</h1>
<span x-bind:class="statusBadgeClass" x-text="statusLabel"></span> <span x-bind:class="statusBadgeClass" x-text="statusLabel"></span>
<span class="badge-neutral font-mono break-all" title="Branch">{{.App.Branch}}</span>
</div> </div>
<p class="text-gray-500 font-mono text-sm mt-1">{{.App.RepoURL}}@{{.App.Branch}}</p> <p class="text-gray-500 font-mono text-sm mt-1">{{.App.RepoURL}}</p>
</div> </div>
<div class="flex gap-3"> <div class="flex gap-3">
<a href="/apps/{{.App.ID}}/edit" class="btn-secondary">Edit</a> <a href="/apps/{{.App.ID}}/edit" class="btn-secondary">Edit</a>
@@ -53,6 +54,26 @@
</div> </div>
</div> </div>
<!-- Container Logs -->
<div class="card p-6 mb-6">
<div class="flex items-center justify-between mb-4">
<h2 class="section-title">Container Logs</h2>
<span x-bind:class="containerStatusBadgeClass" x-text="containerStatusLabel"></span>
</div>
<div class="relative">
<div x-ref="containerLogsWrapper" class="bg-gray-900 rounded-lg p-4 overflow-y-auto" style="max-height: 800px;">
<pre class="text-gray-100 text-xs font-mono whitespace-pre-wrap break-words m-0" x-text="containerLogs"></pre>
</div>
<button
x-show="!_containerAutoScroll"
x-transition
@click="_containerAutoScroll = true; Alpine.store('utils').scrollToBottom($refs.containerLogsWrapper)"
class="absolute bottom-2 right-4 bg-primary-600 hover:bg-primary-700 text-white text-xs px-3 py-1 rounded-full shadow-lg opacity-90 hover:opacity-100 transition"
title="Scroll to bottom"
>↓ Follow</button>
</div>
</div>
<!-- Deploy Key --> <!-- Deploy Key -->
<div class="card p-6 mb-6"> <div class="card p-6 mb-6">
<h2 class="section-title mb-4">Deploy Key</h2> <h2 class="section-title mb-4">Deploy Key</h2>
@@ -100,6 +121,26 @@
</div> </div>
</div> </div>
<!-- Last Deployment Build Logs -->
<div class="card p-6 mb-6" x-show="showBuildLogs" x-cloak>
<div class="flex items-center justify-between mb-4">
<h2 class="section-title">Last Deployment Build Logs</h2>
<span x-bind:class="buildStatusBadgeClass" x-text="buildStatusLabel"></span>
</div>
<div class="relative">
<div x-ref="buildLogsWrapper" class="bg-gray-900 rounded-lg p-4 overflow-y-auto" style="max-height: 800px;">
<pre class="text-gray-100 text-xs font-mono whitespace-pre-wrap break-words m-0" x-text="buildLogs"></pre>
</div>
<button
x-show="!_buildAutoScroll"
x-transition
@click="_buildAutoScroll = true; Alpine.store('utils').scrollToBottom($refs.buildLogsWrapper)"
class="absolute bottom-2 right-4 bg-primary-600 hover:bg-primary-700 text-white text-xs px-3 py-1 rounded-full shadow-lg opacity-90 hover:opacity-100 transition"
title="Scroll to bottom"
>↓ Follow</button>
</div>
</div>
<!-- Environment Variables --> <!-- Environment Variables -->
<div class="card p-6 mb-6" x-data="envVarEditor('{{.App.ID}}')"> <div class="card p-6 mb-6" x-data="envVarEditor('{{.App.ID}}')">
<h2 class="section-title mb-4">Environment Variables</h2> <h2 class="section-title mb-4">Environment Variables</h2>
@@ -137,7 +178,7 @@
<button type="submit" class="btn-primary text-sm">Save</button> <button type="submit" class="btn-primary text-sm">Save</button>
<button type="button" @click="editIdx = -1" class="text-gray-500 hover:text-gray-700 text-sm">Cancel</button> <button type="button" @click="editIdx = -1" class="text-gray-500 hover:text-gray-700 text-sm">Cancel</button>
</form> </form>
<p class="text-xs text-amber-600 mt-1">⚠ Container restart needed after env var changes.</p> <p class="alert-warning mt-1">Environment variable changes take effect at the next deploy or rollback.</p>
</td> </td>
</template> </template>
</tr> </tr>
@@ -353,26 +394,6 @@
</form> </form>
</div> </div>
<!-- Container Logs -->
<div class="card p-6 mb-6">
<div class="flex items-center justify-between mb-4">
<h2 class="section-title">Container Logs</h2>
<span x-bind:class="containerStatusBadgeClass" x-text="containerStatusLabel"></span>
</div>
<div class="relative">
<div x-ref="containerLogsWrapper" class="bg-gray-900 rounded-lg p-4 overflow-y-auto" style="max-height: 400px;">
<pre class="text-gray-100 text-xs font-mono whitespace-pre-wrap break-words m-0" x-text="containerLogs"></pre>
</div>
<button
x-show="!_containerAutoScroll"
x-transition
@click="_containerAutoScroll = true; Alpine.store('utils').scrollToBottom($refs.containerLogsWrapper)"
class="absolute bottom-2 right-4 bg-primary-600 hover:bg-primary-700 text-white text-xs px-3 py-1 rounded-full shadow-lg opacity-90 hover:opacity-100 transition"
title="Scroll to bottom"
>↓ Follow</button>
</div>
</div>
<!-- Recent Deployments --> <!-- Recent Deployments -->
<div class="card p-6 mb-6"> <div class="card p-6 mb-6">
<div class="flex items-center justify-between mb-4"> <div class="flex items-center justify-between mb-4">
@@ -412,26 +433,6 @@
</template> </template>
</div> </div>
<!-- Last Deployment Build Logs -->
<div class="card p-6 mb-6" x-show="showBuildLogs" x-cloak>
<div class="flex items-center justify-between mb-4">
<h2 class="section-title">Last Deployment Build Logs</h2>
<span x-bind:class="buildStatusBadgeClass" x-text="buildStatusLabel"></span>
</div>
<div class="relative">
<div x-ref="buildLogsWrapper" class="bg-gray-900 rounded-lg p-4 overflow-y-auto" style="max-height: 400px;">
<pre class="text-gray-100 text-xs font-mono whitespace-pre-wrap break-words m-0" x-text="buildLogs"></pre>
</div>
<button
x-show="!_buildAutoScroll"
x-transition
@click="_buildAutoScroll = true; Alpine.store('utils').scrollToBottom($refs.buildLogsWrapper)"
class="absolute bottom-2 right-4 bg-primary-600 hover:bg-primary-700 text-white text-xs px-3 py-1 rounded-full shadow-lg opacity-90 hover:opacity-100 transition"
title="Scroll to bottom"
>↓ Follow</button>
</div>
</div>
<!-- Danger Zone --> <!-- Danger Zone -->
<div class="card border-2 border-error-500/20 bg-error-50/50 p-6"> <div class="card border-2 border-error-500/20 bg-error-50/50 p-6">
<h2 class="text-lg font-medium text-error-700 mb-4">Danger Zone</h2> <h2 class="text-lg font-medium text-error-700 mb-4">Danger Zone</h2>
+2 -2
View File
@@ -20,7 +20,7 @@
</div> </div>
{{if .AppStats}} {{if .AppStats}}
<div class="card overflow-hidden"> <div class="card overflow-x-auto">
<table class="table"> <table class="table">
<thead class="table-header"> <thead class="table-header">
<tr> <tr>
@@ -41,7 +41,7 @@
{{.App.Name}} {{.App.Name}}
</a> </a>
</td> </td>
<td class="text-gray-500 font-mono text-xs">{{.App.RepoURL}}</td> <td class="text-gray-500 font-mono text-xs whitespace-normal break-all">{{.App.RepoURL}}</td>
<td class="text-gray-500">{{.App.Branch}}</td> <td class="text-gray-500">{{.App.Branch}}</td>
<td> <td>
{{if eq .App.Status "running"}} {{if eq .App.Status "running"}}