Author SHA1 Message Date
clawbot dfa88eef2d List every setting upaas reads in the README Configuration table (closes #229)
Check / check (pull_request) Skipped
The table left out UPAAS_MAINTENANCE_MODE, UPAAS_SESSION_SECRET and
UPAAS_CORS_ORIGINS, and several rows gave the wrong default or effect.
Each row now matches internal/config/config.go and the code that uses
the value: UPAAS_PORT is also read and wins over PORT,
UPAAS_HOST_DATA_DIR falls back to UPAAS_DATA_DIR and must be absolute
when set, UPAAS_DEBUG also drops the session cookie's Secure flag,
UPAAS_SENTRY_DSN is not used, and /metrics exists only when
UPAAS_METRICS_USERNAME is set. A sentence under the table names the
standard Docker client variables. TODO.md records the step.

Model: opus-5-5
2026-09-29 01:31:58 +00:00
19 changed files with 150 additions and 1070 deletions
+6 -3
View File
@@ -1,8 +1,11 @@
# .git is sent so that `make build` in the Dockerfile can stamp the commit into
# upaas. List no tracked file here: git would see it as deleted in the build and
# the version would end in -dirty.
.git
.env
bin/
.editorconfig
.vscode/
.idea/
*.test
LICENSE
CONVENTIONS.md
REPO_POLICIES.md
README.md
-1
View File
@@ -31,7 +31,6 @@ RUN go mod download
COPY . .
RUN make test
# Takes the version from `git describe` on the .git copied in above.
RUN make build
# Runtime stage
+5 -17
View File
@@ -194,7 +194,7 @@ Environment variables:
| Variable | Description | Default |
| ------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------- |
| `PORT` | HTTP listen port. `UPAAS_PORT` is also read and wins when both are set. | 8080 |
| `UPAAS_DATA_DIR` | Directory for the SQLite database, session key, builds and deployment logs. Deploys need it to be an absolute path unless `UPAAS_HOST_DATA_DIR` is set. | `./data` (the Docker image sets `/var/lib/upaas`) |
| `UPAAS_DATA_DIR` | Directory for the SQLite database, session key, builds and deployment logs | `./data` (the Docker image sets `/var/lib/upaas`) |
| `UPAAS_HOST_DATA_DIR` | Host path of `UPAAS_DATA_DIR`, needed when upaas runs in a container so the bind mounts it passes to Docker point at the right host directory. When set, it must be absolute, or upaas refuses to start. | the value of `UPAAS_DATA_DIR` |
| `UPAAS_DOCKER_HOST` | Docker daemon address | unix:///var/run/docker.sock |
| `UPAAS_PLAINTEXT_HTTP` | Set when µPaaS is reached over plain HTTP (no TLS-terminating proxy in front) so CSRF origin checks use `http://`. Leave unset behind a TLS-terminating reverse proxy. | false |
@@ -226,10 +226,6 @@ This recipe serves plain HTTP, so `UPAAS_PLAINTEXT_HTTP=true` is required for
setup and every other form to pass the CSRF origin check. Behind a
TLS-terminating reverse proxy, drop that line.
The image shows the commit it was built from (the `git describe` output) in the
page footer, the startup log and `/health`. Build it from a git clone: without
the `.git` directory it shows `dev`.
### Deploying with Docker Compose
[`docker-compose.yml`](docker-compose.yml) builds the image from this repo and
@@ -241,13 +237,10 @@ HOST_DATA_DIR=/srv/upaas/data
```
Other settings from [Configuration](#configuration) go in the same file, except
`PORT`, `UPAAS_PORT` and `UPAAS_DATA_DIR`: the compose file sets both port
settings to 8080 and `UPAAS_DATA_DIR` to `/var/lib/upaas`, overriding `.env`, to
match its port mapping, healthcheck and data directory mount. Then run
`docker compose up -d` from the repo root; `docker compose ps` shows the
container as healthy once `/health` answers. To update, run `git pull` and then
`docker compose up -d --build`: without `--build`, Compose keeps running the
image built from the old checkout.
`PORT` and `UPAAS_DATA_DIR`: the compose file sets them to 8080 and
`/var/lib/upaas`, overriding `.env`, to match its port mapping, healthcheck and
data directory mount. Then run `docker compose up -d` from the repo root;
`docker compose ps` shows the container as healthy once `/health` answers.
**Important**: `HOST_DATA_DIR` **must** be an **absolute path** on the host. It
is bind-mounted into the container and passed as `UPAAS_HOST_DATA_DIR` so that
@@ -263,11 +256,6 @@ Docker's build cache rather than as untagged images. Docker Engine 28.2 and
later keeps that cache under a size limit by default; on older engines, set
`"builder": {"gc": {"enabled": true}}` in the host's `daemon.json`.
Building with BuildKit needs Docker Engine 18.09 or later; on an older engine,
upaas fails the deploy instead of building. A Dockerfile that uses
`RUN --network` needs Docker Engine 23.0 or later unless its `# syntax=` line
names Dockerfile frontend 1.3 or later, such as `docker/dockerfile:1`.
Session secrets are automatically generated on first startup and persisted to
`$UPAAS_DATA_DIR/session.key`.
-38
View File
@@ -20,44 +20,6 @@ regress.
# Completed Steps
- 2026-09-29: Built images are tagged `upaas-<app>:<short hash>`, git's short
form of the commit built, instead of the deployment number. A redeploy of a
commit gives its tag to the new image; the old one is kept while the app runs
it or Rollback would start it, then removed by its ID, found among the images
the app's deployments recorded. The removal of old images now keeps every
image any app runs or would roll back to. The clone now reads the commit from
git's output, which Docker's log headers had hidden, so manual deploys record
their commit too (#239).
- 2026-09-29: The app page is 50% wider on large screens (84rem instead of
56rem), its build log and container log boxes are twice as tall, the build log
sits between the webhook URL and the environment variables, and the container
log sits above the deploy key (#246).
- 2026-09-29: A failed build now fails the deploy with the build's own error
instead of a later "failed to inspect image", and the deployment log shows the
end of the build output before that error. upaas refuses to build on a Docker
Engine older than 18.09, which cannot build with BuildKit. The README's
Compose section says to update with `docker compose up -d --build` and names
the Docker Engine versions builds need (#234).
- 2026-09-29: An image built from the `Dockerfile` now shows the commit it was
built from (the `git describe` output) in the footer and `/health` instead of
`dev`: `.dockerignore` no longer leaves out `.git`, nor any tracked file,
which git would count as deleted and mark `-dirty`. upaas now also logs its
version at startup; the logger's `Identify()` was never called (#236).
- 2026-09-29: The app page shows the app's branch as a label in its title, next
to the status badge, instead of after the repository under it (#240).
- 2026-09-29: An app's deployments page now lists only its 10 most recent
deployments, newest first, instead of 50 (#238).
- 2026-09-29: `docker-compose.yml` now sets `UPAAS_PORT` to 8080 as well as
`PORT`, since upaas reads `UPAAS_PORT` first and a `UPAAS_PORT` in `.env` made
it listen away from the port mapping and healthcheck; the README's Compose
section names both (#230).
- 2026-09-29: The README Configuration table now lists every setting upaas
reads, adding `UPAAS_MAINTENANCE_MODE`, `UPAAS_SESSION_SECRET` and
`UPAAS_CORS_ORIGINS`, and gives the real default and effect of each:
+1 -3
View File
@@ -52,8 +52,6 @@ func main() {
handlers.New,
server.New,
),
fx.Invoke(func(log *logger.Logger, _ *server.Server) {
log.Identify()
}),
fx.Invoke(func(*server.Server) {}),
).Run()
}
+2 -4
View File
@@ -7,11 +7,9 @@ services:
# Every line of .env is passed to upaas as an environment variable.
env_file: .env
environment:
# Override any PORT or UPAAS_PORT in .env, so upaas listens where the
# port mapping and healthcheck below expect it. Both are set because
# upaas reads UPAAS_PORT first.
# Overrides any PORT in .env, so upaas listens where the port mapping
# and healthcheck below expect it.
PORT: "8080"
UPAAS_PORT: "8080"
# Overrides any UPAAS_DATA_DIR in .env, so the database stays on the
# HOST_DATA_DIR mount below instead of inside the container.
UPAAS_DATA_DIR: /var/lib/upaas
+21 -120
View File
@@ -21,11 +21,9 @@ import (
"github.com/docker/docker/api/types/image"
"github.com/docker/docker/api/types/mount"
"github.com/docker/docker/api/types/network"
"github.com/docker/docker/api/types/versions"
"github.com/docker/docker/client"
"github.com/docker/docker/pkg/archive"
"github.com/docker/docker/pkg/jsonmessage"
"github.com/docker/docker/pkg/stdcopy"
"github.com/docker/go-connections/nat"
controlapi "github.com/moby/buildkit/api/services/control"
buildkitclient "github.com/moby/buildkit/client"
@@ -63,15 +61,6 @@ var ErrInvalidBranch = errors.New("invalid branch name")
// ErrInvalidCommitSHA is returned when a commit SHA is not a valid hex string.
var ErrInvalidCommitSHA = errors.New("invalid commit SHA")
// ErrBuildKitUnavailable is returned when the Docker daemon is too old to
// build with BuildKit.
var ErrBuildKitUnavailable = errors.New("BuildKit is unavailable on the Docker daemon")
// minBuildKitAPIVersion is the API version of Docker Engine 18.09, the first
// that builds with BuildKit when asked to without experimental mode. Older
// daemons refuse the request or silently use the legacy builder.
const minBuildKitAPIVersion = "1.39"
// validBranchRe matches safe git branch names.
var validBranchRe = regexp.MustCompile(`^[a-zA-Z0-9._/\-]+$`)
@@ -492,7 +481,6 @@ type cloneConfig struct {
type CloneResult struct {
Output string // Combined stdout/stderr from git clone
CommitSHA string // The HEAD commit SHA after clone/checkout
ShortSHA string // git's short form of CommitSHA (git rev-parse --short)
}
// CloneRepo clones a git repository using SSH and optionally checks out a
@@ -555,7 +543,7 @@ func (c *Client) RemoveImage(ctx context.Context, imageID ImageID) error {
}
// ListImageTags returns the tags in the given repository, such as
// "upaas-myapp:1a2b3c4" in "upaas-myapp", each with the ID of its image.
// "upaas-myapp:12" in "upaas-myapp", each with the ID of its image.
// Tags the same image has in other repositories are left out.
func (c *Client) ListImageTags(
ctx context.Context,
@@ -585,44 +573,19 @@ func (c *Client) ListImageTags(
return tags, nil
}
// ListUntaggedImages returns the IDs of the images that have no tag, such
// as one whose tag a later build gave to the image it built.
func (c *Client) ListUntaggedImages(ctx context.Context) ([]ImageID, error) {
if c.docker == nil {
return nil, ErrNotConnected
}
images, err := c.docker.ImageList(ctx, image.ListOptions{
Filters: filters.NewArgs(filters.Arg("dangling", "true")),
})
if err != nil {
return nil, fmt.Errorf("failed to list untagged images: %w", err)
}
imageIDs := make([]ImageID, 0, len(images))
for _, img := range images {
imageIDs = append(imageIDs, ImageID(img.ID))
}
return imageIDs, nil
}
// RemoveImageTag removes the tag name, such as "upaas-myapp:1a2b3c4",
// without force. Docker then deletes the image, and the untagged images it
// was built on, only if no other tag and no container still uses it. If name
// is instead the ID of an untagged image, it removes that image unless a
// container uses it.
func (c *Client) RemoveImageTag(ctx context.Context, name string) error {
// RemoveImageTag removes a tag such as "upaas-myapp:12", without force.
// Docker then deletes the image, and the untagged images it was built on,
// only if no other tag and no container still uses it.
func (c *Client) RemoveImageTag(ctx context.Context, tag string) error {
if c.docker == nil {
return ErrNotConnected
}
_, err := c.docker.ImageRemove(ctx, name, image.RemoveOptions{
_, err := c.docker.ImageRemove(ctx, tag, image.RemoveOptions{
PruneChildren: true,
})
if err != nil && !client.IsErrNotFound(err) {
return fmt.Errorf("failed to remove image %s: %w", name, err)
return fmt.Errorf("failed to remove image tag %s: %w", tag, err)
}
return nil
@@ -632,20 +595,6 @@ func (c *Client) performBuild(
ctx context.Context,
opts BuildImageOptions,
) (ImageID, error) {
server, err := c.docker.ServerVersion(ctx)
if err != nil {
return "", fmt.Errorf("failed to get Docker version: %w", err)
}
if versions.LessThan(server.APIVersion, minBuildKitAPIVersion) {
return "", fmt.Errorf(
"%w: Docker Engine %s (API %s) is older than 18.09 (API %s); "+
"upgrade Docker Engine",
ErrBuildKitUnavailable, server.Version, server.APIVersion,
minBuildKitAPIVersion,
)
}
// Create tar archive of build context
tarArchive, err := archive.TarWithOptions(opts.ContextDir, &archive.TarOptions{})
if err != nil {
@@ -711,8 +660,7 @@ const scannerMaxBufferSize = 1024 * 1024 // 1MB
// newline-delimited JSON. BuildKit's progress arrives encoded in
// "moby.buildkit.trace" messages; these are decoded and written as plain
// text, as "docker build --progress=plain" shows it. Other lines, such as
// build errors, are written unchanged. Docker ends a failed build with a line
// carrying the error; it is returned once the output is written.
// build errors, are written unchanged.
func (c *Client) streamBuildOutput(
ctx context.Context,
body io.Reader,
@@ -742,8 +690,6 @@ func (c *Client) streamBuildOutput(
buf := make([]byte, 0, scannerInitialBufferSize)
scanner.Buffer(buf, scannerMaxBufferSize)
var buildErr error
for scanner.Scan() {
line := scanner.Bytes()
@@ -762,10 +708,6 @@ func (c *Client) streamBuildOutput(
continue
}
if err == nil && msg.Error != nil {
buildErr = msg.Error
}
// One write per line, so it is not split by the display's output.
_, _ = fmt.Fprintf(out, "%s\n", line)
}
@@ -778,7 +720,7 @@ func (c *Client) streamBuildOutput(
return fmt.Errorf("failed to read build output: %w", scanErr)
}
return buildErr
return nil
}
func (c *Client) performClone(
@@ -877,13 +819,11 @@ func (c *Client) createGitContainer(
// Clone without depth limit so we can checkout any commit, then checkout specific SHA
script = `git clone --branch "$CLONE_BRANCH" "$CLONE_URL" /repo` +
` && cd /repo && git checkout "$CLONE_SHA"` +
` && echo COMMIT:$(git rev-parse HEAD)` +
` && echo SHORT_SHA:$(git rev-parse --short HEAD)`
` && echo COMMIT:$(git rev-parse HEAD)`
} else {
// Shallow clone of branch HEAD, then output commit SHA
script = `git clone --depth 1 --branch "$CLONE_BRANCH" "$CLONE_URL" /repo` +
` && cd /repo && echo COMMIT:$(git rev-parse HEAD)` +
` && echo SHORT_SHA:$(git rev-parse --short HEAD)`
` && cd /repo && echo COMMIT:$(git rev-parse HEAD)`
}
env := []string{
@@ -950,7 +890,7 @@ func (c *Client) runGitClone(
return nil, fmt.Errorf("error waiting for git container: %w", err)
case status := <-statusCh:
// Always capture logs for the result
logs := c.gitContainerOutput(ctx, containerID)
logs, _ := c.ContainerLogs(ctx, containerID, "100")
if status.StatusCode != 0 {
return nil, fmt.Errorf(
@@ -961,62 +901,23 @@ func (c *Client) runGitClone(
)
}
// Parse the commit from the "COMMIT:" and "SHORT_SHA:" lines.
result := &CloneResult{
Output: logs,
CommitSHA: parseCommitSHA(logs, commitMarker),
ShortSHA: parseCommitSHA(logs, shortSHAMarker),
}
// Parse commit SHA from output (looks for "COMMIT:<sha>" line)
commitSHA := parseCommitSHA(logs)
// The short hash names the image the deploy builds.
if result.ShortSHA == "" {
return nil, fmt.Errorf("%w: no short commit hash in its output: %s",
ErrGitCloneFailed, logs)
}
return result, nil
return &CloneResult{Output: logs, CommitSHA: commitSHA}, nil
}
}
// gitContainerOutput returns the last 100 lines the git container wrote.
// Docker puts a header before each line a container without a terminal
// writes; stdcopy removes them so that the lines can be parsed.
func (c *Client) gitContainerOutput(
ctx context.Context,
containerID ContainerID,
) string {
reader, err := c.docker.ContainerLogs(ctx, containerID.String(), container.LogsOptions{
ShowStdout: true,
ShowStderr: true,
Tail: "100",
})
if err != nil {
return ""
}
// commitMarker is the prefix used to identify commit SHA in clone output.
const commitMarker = "COMMIT:"
defer func() { _ = reader.Close() }()
var output strings.Builder
_, _ = stdcopy.StdCopy(&output, &output, reader)
return output.String()
}
// Prefixes of the lines in the clone output that carry the commit checked
// out, in full and in git's short form.
const (
commitMarker = "COMMIT:"
shortSHAMarker = "SHORT_SHA:"
)
// parseCommitSHA extracts a commit SHA from git clone output.
// It looks for a line starting with marker and returns the SHA after it.
func parseCommitSHA(output, marker string) string {
// parseCommitSHA extracts the commit SHA from git clone output.
// It looks for a line starting with "COMMIT:" and returns the SHA after it.
func parseCommitSHA(output string) string {
for line := range strings.SplitSeq(output, "\n") {
line = strings.TrimSpace(line)
sha, found := strings.CutPrefix(line, marker)
sha, found := strings.CutPrefix(line, commitMarker)
if found {
return strings.TrimSpace(sha)
}
-141
View File
@@ -6,7 +6,6 @@ import (
"encoding/json"
"errors"
"fmt"
"io"
"log/slog"
"net/http"
"net/http/httptest"
@@ -17,7 +16,6 @@ import (
"time"
"github.com/docker/docker/client"
"github.com/docker/docker/pkg/stdcopy"
controlapi "github.com/moby/buildkit/api/services/control"
)
@@ -205,8 +203,6 @@ func TestPerformCloneRemovesContainerVolumes(t *testing.T) {
<-r.Context().Done()
case strings.HasSuffix(r.URL.Path, "/wait"):
_, _ = fmt.Fprintf(w, `{"StatusCode":%d}`, tt.exitCode)
case strings.HasSuffix(r.URL.Path, "/logs"):
writeCloneOutput(w)
default:
_, _ = w.Write([]byte(`{}`))
}
@@ -248,62 +244,6 @@ func TestPerformCloneRemovesContainerVolumes(t *testing.T) {
}
}
// cloneCommit is the commit the fake clones in these tests check out.
const cloneCommit = "1a2b3c4d5e6f7a8b9c0d1e2f3a4b5c6d7e8f9a0b"
// writeCloneOutput writes the output of a git clone of cloneCommit as
// Docker sends a container's log: each line after a header.
func writeCloneOutput(w io.Writer) {
stdout := stdcopy.NewStdWriter(w, stdcopy.Stdout)
stderr := stdcopy.NewStdWriter(w, stdcopy.Stderr)
_, _ = stderr.Write([]byte("Cloning into '/repo'...\n"))
_, _ = stdout.Write([]byte("COMMIT:" + cloneCommit + "\n"))
_, _ = stdout.Write([]byte("SHORT_SHA:1a2b3c4\n"))
}
// TestCloneRepoReadsCommit runs a clone against a fake Docker API and
// checks that the commit checked out is read from the clone's output, in
// full and in git's short form.
func TestCloneRepoReadsCommit(t *testing.T) {
t.Parallel()
srv := httptest.NewServer(http.HandlerFunc(
func(w http.ResponseWriter, r *http.Request) {
switch {
case strings.HasSuffix(r.URL.Path, "/containers/create"):
_, _ = w.Write([]byte(`{"Id":"gitcontainer"}`))
case strings.HasSuffix(r.URL.Path, "/logs"):
writeCloneOutput(w)
default:
_, _ = w.Write([]byte(`{}`))
}
},
))
t.Cleanup(srv.Close)
dockerAPI, err := client.NewClientWithOpts(
client.WithHost("tcp://" + srv.Listener.Addr().String()),
)
if err != nil {
t.Fatal(err)
}
c := &Client{docker: dockerAPI, log: slog.Default()}
result, err := c.CloneRepo(
t.Context(), "git@example.com:repo.git", mainBranch, "", "fake-key",
t.TempDir(), t.TempDir(),
)
if err != nil {
t.Fatal(err)
}
if result.CommitSHA != cloneCommit || result.ShortSHA != "1a2b3c4" {
t.Errorf("got commit %q, short %q", result.CommitSHA, result.ShortSHA)
}
}
// TestPerformBuildUsesBuildKit runs a build against a fake Docker API and
// checks that it asks for BuildKit and that BuildKit's progress reaches the
// build log as plain text.
@@ -331,8 +271,6 @@ func TestPerformBuildUsesBuildKit(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(
func(w http.ResponseWriter, r *http.Request) {
switch {
case strings.HasSuffix(r.URL.Path, "/version"):
_, _ = w.Write([]byte(`{"Version":"27.3.1","ApiVersion":"1.47"}`))
case strings.HasSuffix(r.URL.Path, "/build"):
if r.URL.Query().Get("version") != "2" {
http.Error(w, "not a BuildKit build", http.StatusBadRequest)
@@ -376,82 +314,3 @@ func TestPerformBuildUsesBuildKit(t *testing.T) {
t.Errorf("build log is missing the build step:\n%s", buildLog.String())
}
}
// TestPerformBuildFails runs builds that fail against a fake Docker API and
// checks that each returns its own error and that no image is inspected
// afterwards.
func TestPerformBuildFails(t *testing.T) {
t.Parallel()
tests := []struct {
name string
engine string // Docker Engine version the fake daemon reports
apiVersion string // API version the fake daemon reports
buildOutput string
wantErr string
}{
{
name: "build step fails",
engine: "27.3.1",
apiVersion: "1.47",
buildOutput: `{"stream":"Step 1/1 : RUN false\n"}` + "\n" +
`{"errorDetail":{"message":"exit code: 1"},"error":"exit code: 1"}`,
wantErr: "exit code: 1",
},
{
name: "daemon too old for BuildKit",
engine: "18.06.3-ce",
apiVersion: "1.38",
wantErr: "BuildKit is unavailable on the Docker daemon: " +
"Docker Engine 18.06.3-ce (API 1.38) is older than 18.09 (API 1.39); " +
"upgrade Docker Engine",
},
{
// The build step's own error shows the build went ahead.
name: "daemon at API 1.39 builds",
engine: "18.09.9",
apiVersion: "1.39",
buildOutput: `{"stream":"Step 1/1 : RUN false\n"}` + "\n" +
`{"errorDetail":{"message":"exit code: 1"},"error":"exit code: 1"}`,
wantErr: "exit code: 1",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
srv := httptest.NewServer(http.HandlerFunc(
func(w http.ResponseWriter, r *http.Request) {
switch {
case strings.HasSuffix(r.URL.Path, "/version"):
_, _ = fmt.Fprintf(w, `{"Version":%q,"ApiVersion":%q}`,
tt.engine, tt.apiVersion)
case strings.HasSuffix(r.URL.Path, "/build"):
_, _ = w.Write([]byte(tt.buildOutput))
default:
t.Errorf("unexpected request to %s", r.URL.Path)
}
},
))
t.Cleanup(srv.Close)
dockerAPI, err := client.NewClientWithOpts(
client.WithHost("tcp://" + srv.Listener.Addr().String()),
)
if err != nil {
t.Fatal(err)
}
c := &Client{docker: dockerAPI, log: slog.Default()}
_, err = c.performBuild(t.Context(), BuildImageOptions{
ContextDir: t.TempDir(),
Tags: []string{"upaas-test:1"},
})
if err == nil || err.Error() != tt.wantErr {
t.Errorf("got error %v, want %q", err, tt.wantErr)
}
})
}
}
+1 -1
View File
@@ -28,7 +28,7 @@ const (
// recentDeploymentsLimit is the number of recent deployments to show.
recentDeploymentsLimit = 5
// deploymentsHistoryLimit is the number of deployments to show in history.
deploymentsHistoryLimit = 10
deploymentsHistoryLimit = 50
)
// redirectToApp issues a SeeOther redirect to the page for the given
-46
View File
@@ -1,46 +0,0 @@
package handlers_test
import (
"net/http"
"net/http/httptest"
"strings"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"sneak.berlin/go/upaas/internal/service/app"
)
// TestAppPageTitleShowsBranch checks that an app's branch can be read from
// the app page title, next to the status badge, without opening the edit page.
func TestAppPageTitleShowsBranch(t *testing.T) {
t.Parallel()
testCtx := setupTestHandlers(t)
createdApp, err := testCtx.appSvc.CreateApp(t.Context(), app.CreateAppInput{
Name: "branch-shown-app",
RepoURL: "git@example.com:user/branch-shown-app.git",
Branch: "staging",
})
require.NoError(t, err)
request := httptest.NewRequestWithContext(
t.Context(), http.MethodGet, "/apps/"+createdApp.ID, nil,
)
request = addChiURLParams(request, map[string]string{"id": createdApp.ID})
recorder := httptest.NewRecorder()
testCtx.handlers.HandleAppDetail().ServeHTTP(recorder, request)
require.Equal(t, http.StatusOK, recorder.Code)
// The title row runs from the app name heading to the end of its div.
_, afterHeading, found := strings.Cut(recorder.Body.String(), "<h1")
require.True(t, found, "app page has no heading")
titleRow, _, _ := strings.Cut(afterHeading, "</div>")
assert.Contains(t, titleRow, `x-text="statusLabel"`)
assert.Contains(t, titleRow, ">staging</span>")
}
-76
View File
@@ -1,76 +0,0 @@
package handlers_test
import (
"net/http"
"net/http/httptest"
"strings"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"sneak.berlin/go/upaas/internal/service/app"
)
// TestAppPageLayout checks the app page's width, the order of its sections,
// and the height of its two log boxes.
func TestAppPageLayout(t *testing.T) {
t.Parallel()
testCtx := setupTestHandlers(t)
createdApp, err := testCtx.appSvc.CreateApp(t.Context(), app.CreateAppInput{
Name: "layout-app",
RepoURL: "git@example.com:user/layout-app.git",
})
require.NoError(t, err)
request := httptest.NewRequestWithContext(
t.Context(), http.MethodGet, "/apps/"+createdApp.ID, nil,
)
request = addChiURLParams(request, map[string]string{"id": createdApp.ID})
recorder := httptest.NewRecorder()
testCtx.handlers.HandleAppDetail().ServeHTTP(recorder, request)
require.Equal(t, http.StatusOK, recorder.Code)
body := recorder.Body.String()
_, afterMain, found := strings.Cut(body, "<main")
require.True(t, found, "app page has no main element")
mainTag, _, _ := strings.Cut(afterMain, ">")
assert.Contains(t, mainTag, "max-width: 84rem;")
sectionTitles := []string{
"Container Logs",
"Deploy Key",
"Webhook URL",
"Last Deployment Build Logs",
"Environment Variables",
"Docker Labels",
"Volume Mounts",
"Port Mappings",
"Recent Deployments",
"Danger Zone",
}
previousIndex := -1
for _, title := range sectionTitles {
index := strings.Index(body, ">"+title+"</h2>")
require.NotEqual(t, -1, index, "app page has no %q section", title)
assert.Greater(t, index, previousIndex, "%q section is out of order", title)
previousIndex = index
}
for _, logBox := range []string{"containerLogsWrapper", "buildLogsWrapper"} {
_, afterRef, found := strings.Cut(body, `x-ref="`+logBox+`"`)
require.True(t, found, "app page has no %s", logBox)
logBoxTag, _, _ := strings.Cut(afterRef, ">")
assert.Contains(t, logBoxTag, "max-height: 800px;", logBox)
}
}
-68
View File
@@ -8,7 +8,6 @@ import (
"strconv"
"strings"
"testing"
"time"
"github.com/go-chi/chi/v5"
"github.com/stretchr/testify/assert"
@@ -1149,73 +1148,6 @@ func TestHandleCancelDeployReturns404ForUnknownApp(t *testing.T) {
assert.Equal(t, http.StatusNotFound, recorder.Code)
}
// TestHandleAppDeploymentsShowsTenNewest verifies the deployments page
// lists only the 10 most recent deployments, newest first.
func TestHandleAppDeploymentsShowsTenNewest(t *testing.T) {
t.Parallel()
testCtx := setupTestHandlers(t)
createdApp := createTestApp(t, testCtx, "deployments-page-app")
// Create 12 deployments, each started one minute after the one before.
firstStart := time.Date(2026, 1, 1, 0, 0, 0, 0, time.UTC)
ids := make([]int64, 0, 12)
for idx := range 12 {
deployment := models.NewDeployment(testCtx.database)
deployment.AppID = createdApp.ID
deployment.Status = models.DeploymentStatusSuccess
require.NoError(t, deployment.Save(context.Background()))
_, err := testCtx.database.Exec(
context.Background(),
"UPDATE deployments SET started_at = ? WHERE id = ?",
firstStart.Add(time.Duration(idx)*time.Minute),
deployment.ID,
)
require.NoError(t, err)
ids = append(ids, deployment.ID)
}
request := httptest.NewRequestWithContext(
t.Context(),
http.MethodGet,
"/apps/"+createdApp.ID+"/deployments",
nil,
)
request = addChiURLParams(request, map[string]string{"id": createdApp.ID})
recorder := httptest.NewRecorder()
handler := testCtx.handlers.HandleAppDeployments()
handler.ServeHTTP(recorder, request)
require.Equal(t, http.StatusOK, recorder.Code)
body := recorder.Body.String()
card := func(id int64) string {
return `data-deployment-id="` + strconv.FormatInt(id, 10) + `"`
}
assert.Equal(t, 10, strings.Count(body, `data-deployment-id="`))
// The two oldest are left out.
assert.NotContains(t, body, card(ids[0]))
assert.NotContains(t, body, card(ids[1]))
// The ten newest are shown, newest first.
previous := -1
for idx := len(ids) - 1; idx >= 2; idx-- {
position := strings.Index(body, card(ids[idx]))
require.Greater(t, position, previous,
"deployment %d missing or out of order", ids[idx])
previous = position
}
}
func TestHandleWebhookReturns404ForUnknownSecret(t *testing.T) {
t.Parallel()
+2 -42
View File
@@ -203,12 +203,11 @@ func (d *Deployment) insert(ctx context.Context) error {
func (d *Deployment) update(ctx context.Context) error {
query := `
UPDATE deployments SET
commit_sha = ?, image_id = ?, container_id = ?, status = ?, logs = ?,
finished_at = ?
image_id = ?, container_id = ?, status = ?, logs = ?, finished_at = ?
WHERE id = ?`
_, err := d.db.Exec(ctx, query,
d.CommitSHA, d.ImageID, d.ContainerID, d.Status, d.Logs, d.FinishedAt, d.ID,
d.ImageID, d.ContainerID, d.Status, d.Logs, d.FinishedAt, d.ID,
)
return err
@@ -296,45 +295,6 @@ func FindDeploymentsByAppID(
return deployments, nil
}
// FindDeploymentImageIDs returns the IDs of the images an app's
// deployments built or rolled back to.
func FindDeploymentImageIDs(
ctx context.Context,
deployDB *database.Database,
appID string,
) ([]string, error) {
rows, err := deployDB.Query(ctx, `
SELECT DISTINCT image_id FROM deployments
WHERE app_id = ? AND image_id IS NOT NULL`,
appID,
)
if err != nil {
return nil, fmt.Errorf("querying deployment image IDs: %w", err)
}
defer func() { _ = rows.Close() }()
var imageIDs []string
for rows.Next() {
var imageID string
scanErr := rows.Scan(&imageID)
if scanErr != nil {
return nil, fmt.Errorf("scanning deployment image ID: %w", scanErr)
}
imageIDs = append(imageIDs, imageID)
}
rowsErr := rows.Err()
if rowsErr != nil {
return nil, fmt.Errorf("iterating deployment image IDs: %w", rowsErr)
}
return imageIDs, nil
}
// LatestDeploymentForApp finds the most recent deployment for an app.
//
//nolint:nilnil // returning nil,nil is idiomatic for "not found" in Active Record
+20 -85
View File
@@ -735,103 +735,44 @@ func (svc *Service) recordDeployedImage(
return nil
}
// removeUnusedImages removes the app's images except those an app's running
// container uses or its Rollback would start. Docker deletes a tagged image
// only once no other tag, such as another app's, and no container still
// uses it.
// removeUnusedImages removes the app's tags (upaas-<app>:<deployment>, set by
// buildImage) except those of the image the running container uses and the
// one Rollback would start. Docker deletes an image only once no other tag,
// such as another app's, and no container still uses it.
func (svc *Service) removeUnusedImages(
ctx context.Context,
app *models.App,
deployment *models.Deployment,
) {
images, err := svc.findAppImages(ctx, app)
tags, err := svc.docker.ListImageTags(ctx, "upaas-"+app.Name)
if err != nil {
svc.log.Error("failed to list app images", "error", err, "app", app.Name)
return
}
keep, err := svc.imagesToKeep(ctx)
if err != nil {
svc.log.Error("failed to list the images apps use", "error", err, "app", app.Name)
return
}
for _, name := range slices.Sorted(maps.Keys(images)) {
if keep[images[name].String()] {
for _, tag := range slices.Sorted(maps.Keys(tags)) {
imageID := tags[tag].String()
if imageID == app.ImageID.String || imageID == app.PreviousImageID.String {
continue
}
removeErr := svc.docker.RemoveImageTag(ctx, name)
removeErr := svc.docker.RemoveImageTag(ctx, tag)
if removeErr != nil {
svc.log.Error("failed to remove old image",
"error", removeErr, "app", app.Name, "image", name)
"error", removeErr, "app", app.Name, "tag", tag)
_ = deployment.AppendLog(
ctx,
"WARNING: failed to remove old image "+name+": "+removeErr.Error(),
"WARNING: failed to remove old image "+tag+": "+removeErr.Error(),
)
continue
}
_ = deployment.AppendLog(ctx, "Removed old image: "+name)
_ = deployment.AppendLog(ctx, "Removed old image: "+tag)
}
}
// findAppImages returns the app's images, each under the name it is removed
// by: its tag, upaas-<app>:<short hash> as set by buildImage, or its ID if
// it has none. A redeploy of a commit gives the commit's tag to the new
// image, so the old one is found among the images the app's deployments
// recorded.
func (svc *Service) findAppImages(
ctx context.Context,
app *models.App,
) (map[string]docker.ImageID, error) {
images, err := svc.docker.ListImageTags(ctx, "upaas-"+app.Name)
if err != nil {
return nil, fmt.Errorf("failed to list image tags: %w", err)
}
untagged, err := svc.docker.ListUntaggedImages(ctx)
if err != nil {
return nil, fmt.Errorf("failed to list untagged images: %w", err)
}
recorded, err := models.FindDeploymentImageIDs(ctx, svc.db, app.ID)
if err != nil {
return nil, fmt.Errorf("failed to find deployment images: %w", err)
}
for _, imageID := range untagged {
if slices.Contains(recorded, imageID.String()) {
images[imageID.String()] = imageID
}
}
return images, nil
}
// imagesToKeep returns the IDs of the image each app's running container
// uses and the one its Rollback would start. It covers every app because
// apps that build the same commit can share an image, and a redeploy can
// take the tag that kept the image for one of them.
func (svc *Service) imagesToKeep(ctx context.Context) (map[string]bool, error) {
apps, err := models.AllApps(ctx, svc.db)
if err != nil {
return nil, fmt.Errorf("failed to list apps: %w", err)
}
keep := make(map[string]bool)
for _, app := range apps {
keep[app.ImageID.String] = true
keep[app.PreviousImageID.String] = true
}
return keep, nil
}
// cleanupCancelledDeploy removes orphan resources left by a cancelled deployment.
func (svc *Service) cleanupCancelledDeploy(
ctx context.Context,
@@ -971,17 +912,18 @@ func (svc *Service) buildImage(
app *models.App,
deployment *models.Deployment,
) (docker.ImageID, error) {
workDir, shortSHA, cleanup, err := svc.cloneRepository(ctx, app, deployment)
workDir, cleanup, err := svc.cloneRepository(ctx, app, deployment)
if err != nil {
return "", err
}
defer cleanup()
imageTag := "upaas-" + app.Name + ":" + shortSHA
imageTag := fmt.Sprintf("upaas-%s:%d", app.Name, deployment.ID)
// Create log writer that flushes build output to deployment logs every second
logWriter := newDeploymentLogWriter(ctx, deployment)
defer logWriter.Close()
// BuildImage creates a tar archive from the local filesystem,
// so it needs the container path where files exist, not the host path.
@@ -991,10 +933,6 @@ func (svc *Service) buildImage(
Tags: []string{imageTag},
LogWriter: logWriter,
})
// Write the rest of the build output to the log before the result.
logWriter.Close()
if err != nil {
svc.notify.NotifyBuildFailed(ctx, app, deployment, err)
svc.failDeployment(
@@ -1013,14 +951,11 @@ func (svc *Service) buildImage(
return imageID, nil
}
// cloneRepository clones the app's repository for a build. It returns the
// directory of the clone, git's short form of the commit checked out, and a
// function that removes the clone.
func (svc *Service) cloneRepository(
ctx context.Context,
app *models.App,
deployment *models.Deployment,
) (string, string, func(), error) {
) (string, func(), error) {
// Use a subdirectory of DataDir for builds since it's mounted from the host
// and accessible to Docker for bind mounts (unlike /tmp inside the container).
// Structure: builds/<appname>/<deployment-id>-<random>/
@@ -1037,7 +972,7 @@ func (svc *Service) cloneRepository(
fmt.Errorf("failed to create builds dir: %w", err),
)
return "", "", nil, fmt.Errorf("failed to create builds dir: %w", err)
return "", nil, fmt.Errorf("failed to create builds dir: %w", err)
}
buildDir, err := os.MkdirTemp(appBuildsDir, fmt.Sprintf("%d-*", deployment.ID))
@@ -1049,7 +984,7 @@ func (svc *Service) cloneRepository(
fmt.Errorf("failed to create temp dir: %w", err),
)
return "", "", nil, fmt.Errorf("failed to create temp dir: %w", err)
return "", nil, fmt.Errorf("failed to create temp dir: %w", err)
}
cleanup := func() { _ = os.RemoveAll(buildDir) }
@@ -1085,7 +1020,7 @@ func (svc *Service) cloneRepository(
fmt.Errorf("failed to clone repo: %w", cloneErr),
)
return "", "", nil, fmt.Errorf("failed to clone repo: %w", cloneErr)
return "", nil, fmt.Errorf("failed to clone repo: %w", cloneErr)
}
svc.processCloneResult(ctx, app, deployment, cloneResult, commitSHA)
@@ -1093,7 +1028,7 @@ func (svc *Service) cloneRepository(
// Return the 'work' subdirectory where the repo was cloned
workDir := filepath.Join(buildDir, "work")
return workDir, cloneResult.ShortSHA, cleanup, nil
return workDir, cleanup, nil
}
// processCloneResult handles the result of a git clone operation.
@@ -1,93 +0,0 @@
package deploy_test
import (
"context"
"log/slog"
"net/http"
"net/http/httptest"
"os"
"strings"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"go.uber.org/fx/fxtest"
"sneak.berlin/go/upaas/internal/config"
"sneak.berlin/go/upaas/internal/database"
"sneak.berlin/go/upaas/internal/docker"
"sneak.berlin/go/upaas/internal/logger"
"sneak.berlin/go/upaas/internal/models"
"sneak.berlin/go/upaas/internal/service/deploy"
)
// TestBuildImageLogsBuildErrorBeforeDeployError runs a build that fails
// against a fake Docker API and checks that the deploy fails with the
// build's own error, which the deployment log shows before the deploy's.
func TestBuildImageLogsBuildErrorBeforeDeployError(t *testing.T) {
t.Parallel()
srv := httptest.NewServer(http.HandlerFunc(
func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
switch {
case strings.HasSuffix(r.URL.Path, "/containers/create"):
_, _ = w.Write([]byte(`{"Id":"gitcontainer"}`))
case strings.HasSuffix(r.URL.Path, "/logs"):
writeCloneOutput(w, "abc1234")
case strings.HasSuffix(r.URL.Path, "/version"):
_, _ = w.Write([]byte(`{"Version":"27.3.1","ApiVersion":"1.47"}`))
case strings.HasSuffix(r.URL.Path, "/build"):
_, _ = w.Write([]byte(`{"stream":"Step 1/1 : RUN false\n"}` + "\n" +
`{"errorDetail":{"message":"exit code: 1"},"error":"exit code: 1"}`))
default:
// The steps of the git clone, which succeeds.
_, _ = w.Write([]byte(`{}`))
}
},
))
t.Cleanup(srv.Close)
log := slog.New(slog.NewTextHandler(os.Stderr, nil))
lifecycle := fxtest.NewLifecycle(t)
dockerClient, err := docker.New(lifecycle, docker.Params{
Logger: logger.NewForTest(log),
Config: &config.Config{DockerHost: "tcp://" + srv.Listener.Addr().String()},
})
require.NoError(t, err)
lifecycle.RequireStart()
t.Cleanup(lifecycle.RequireStop)
db := database.NewTestDatabase(t)
ctx := context.Background()
app := models.NewApp(db)
app.ID = "buildapp-id"
app.Name = "buildapp"
app.Branch = "main"
require.NoError(t, app.Save(ctx))
deployment := models.NewDeployment(db)
deployment.AppID = app.ID
require.NoError(t, deployment.Save(ctx))
dataDir := t.TempDir()
cfg := &config.Config{DataDir: dataDir, HostDataDir: dataDir}
// The service has no notify service: the app has no ntfy topic and no
// Slack webhook, so the build failure notification sends nothing.
svc := deploy.NewTestServiceWithConfig(log, cfg, db, dockerClient)
_, err = svc.BuildImage(ctx, app, deployment)
require.EqualError(t, err, "failed to build image: exit code: 1")
logs := deployment.Logs.String
buildError := strings.Index(logs, "ERROR: exit code: 1")
deployError := strings.Index(logs, "ERROR: failed to build image: exit code: 1")
require.NotEqual(t, -1, buildError, logs)
assert.Less(t, buildError, deployError, logs)
}
@@ -20,7 +20,7 @@ func TestCleanupCancelledDeploy_RemovesBuildDir(t *testing.T) {
tmpDir := t.TempDir()
cfg := &config.Config{DataDir: tmpDir}
svc := deploy.NewTestServiceWithConfig(slog.Default(), cfg, nil, nil)
svc := deploy.NewTestServiceWithConfig(slog.Default(), cfg, nil)
// Create a fake build directory matching the deployment pattern
appName := "test-app"
@@ -59,7 +59,7 @@ func TestCleanupCancelledDeploy_NoBuildDir(t *testing.T) {
tmpDir := t.TempDir()
cfg := &config.Config{DataDir: tmpDir}
svc := deploy.NewTestServiceWithConfig(slog.Default(), cfg, nil, nil)
svc := deploy.NewTestServiceWithConfig(slog.Default(), cfg, nil)
// Should not panic when build dir doesn't exist
svc.CleanupCancelledDeploy(context.Background(), "nonexistent-app", 1, "")
+46 -272
View File
@@ -3,21 +3,14 @@ package deploy_test
import (
"context"
"database/sql"
"encoding/json"
"fmt"
"io"
"log/slog"
"maps"
"net/http"
"net/http/httptest"
"os"
"slices"
"strings"
"sync"
"testing"
"github.com/docker/docker/api/types/image"
"github.com/docker/docker/pkg/stdcopy"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"go.uber.org/fx/fxtest"
@@ -30,139 +23,45 @@ import (
"sneak.berlin/go/upaas/internal/service/deploy"
)
// fakeImageAPI is a fake Docker API that keeps images and their tags as
// Docker does: a build gives its tag to the image it builds, and removing
// an image's last tag, or an untagged image by its ID, deletes the image.
// It also answers the steps of a git clone that reports shortSHA.
type fakeImageAPI struct {
mu sync.Mutex
images map[string][]string // image ID -> tags
shortSHA string // the commit's short hash the clone reports
nextID string // ID of the image the next build creates
removed []string // each tag or ID removed
forced bool // whether a removal was forced
}
// TestRecordDeployedImageRemovesOldImages runs the step after a deploy
// against a fake Docker API. Image one is also tagged for another app,
// image two was the previous image, three the current one, four is new.
func TestRecordDeployedImageRemovesOldImages(t *testing.T) {
t.Parallel()
func (api *fakeImageAPI) ServeHTTP(w http.ResponseWriter, r *http.Request) {
api.mu.Lock()
defer api.mu.Unlock()
var (
mu sync.Mutex
removed []string
forced bool
)
w.Header().Set("Content-Type", "application/json")
srv := httptest.NewServer(http.HandlerFunc(
func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
_, name, isImage := strings.Cut(r.URL.Path, "/images/")
switch {
case r.Method == http.MethodDelete:
_, name, _ := strings.Cut(r.URL.Path, "/images/")
switch {
case strings.HasSuffix(r.URL.Path, "/images/json"):
dangling := strings.Contains(r.URL.Query().Get("filters"), "dangling")
api.listImages(w, dangling)
case isImage && r.Method == http.MethodDelete:
api.forced = api.forced || r.URL.Query().Get("force") != ""
api.removeImage(w, name)
case isImage && strings.HasSuffix(name, "/json"):
api.inspectImage(w, strings.TrimSuffix(name, "/json"))
case strings.HasSuffix(r.URL.Path, "/build"):
tag := r.URL.Query().Get("t")
api.untag(tag)
api.images[api.nextID] = append(api.images[api.nextID], tag)
case strings.HasSuffix(r.URL.Path, "/version"):
_, _ = w.Write([]byte(`{"Version":"27.3.1","ApiVersion":"1.47"}`))
case strings.HasSuffix(r.URL.Path, "/containers/create"):
_, _ = w.Write([]byte(`{"Id":"gitcontainer"}`))
case strings.HasSuffix(r.URL.Path, "/logs"):
writeCloneOutput(w, api.shortSHA)
default:
// The other steps of the git clone, which succeeds.
_, _ = w.Write([]byte(`{}`))
}
}
mu.Lock()
// listImages lists the untagged images, or else the tagged ones.
func (api *fakeImageAPI) listImages(w http.ResponseWriter, dangling bool) {
list := []image.Summary{}
removed = append(removed, name)
forced = forced || r.URL.Query().Get("force") != ""
mu.Unlock()
for _, id := range slices.Sorted(maps.Keys(api.images)) {
if (len(api.images[id]) == 0) == dangling {
list = append(list, image.Summary{ID: id, RepoTags: api.images[id]})
}
}
data, err := json.Marshal(list)
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
_, _ = w.Write(data)
}
func (api *fakeImageAPI) inspectImage(w http.ResponseWriter, name string) {
for id, tags := range api.images {
if id == name || slices.Contains(tags, name) {
_, _ = fmt.Fprintf(w, `{"Id":%q}`, id)
return
}
}
w.WriteHeader(http.StatusNotFound)
_, _ = w.Write([]byte(`{"message":"No such image"}`))
}
func (api *fakeImageAPI) removeImage(w http.ResponseWriter, name string) {
api.removed = append(api.removed, name)
id := name
if _, isID := api.images[name]; !isID {
id = api.untag(name)
}
if len(api.images[id]) == 0 {
delete(api.images, id)
}
_, _ = w.Write([]byte(`[]`))
}
// untag removes tag from the image that has it, leaving the image, and
// returns the image's ID.
func (api *fakeImageAPI) untag(tag string) string {
for id, tags := range api.images {
if slices.Contains(tags, tag) {
api.images[id] = slices.DeleteFunc(tags, func(t string) bool { return t == tag })
return id
}
}
return ""
}
// state returns each image's tags and each tag or ID removed.
func (api *fakeImageAPI) state() (map[string][]string, []string) {
api.mu.Lock()
defer api.mu.Unlock()
return maps.Clone(api.images), slices.Clone(api.removed)
}
// writeCloneOutput writes the line of a git clone's output that gives the
// commit's short hash, as Docker sends a container's log: after a header.
func writeCloneOutput(w io.Writer, shortSHA string) {
out := stdcopy.NewStdWriter(w, stdcopy.Stdout)
_, _ = fmt.Fprintf(out, "SHORT_SHA:%s\n", shortSHA)
}
// newImageTestService returns a deploy service that uses api as its
// Docker API, and its database.
func newImageTestService(
t *testing.T,
api *fakeImageAPI,
) (*deploy.Service, *database.Database) {
t.Helper()
srv := httptest.NewServer(api)
_, _ = w.Write([]byte(`[]`))
case strings.HasSuffix(r.URL.Path, "/images/json"):
_, _ = w.Write([]byte(`[
{"Id":"sha256:one","RepoTags":["upaas-myapp:1","upaas-otherapp:7"]},
{"Id":"sha256:two","RepoTags":["upaas-myapp:2"]},
{"Id":"sha256:three","RepoTags":["upaas-myapp:3"]},
{"Id":"sha256:four","RepoTags":["upaas-myapp:4"]}
]`))
default:
_, _ = w.Write([]byte(`{}`))
}
},
))
t.Cleanup(srv.Close)
log := slog.New(slog.NewTextHandler(os.Stderr, nil))
@@ -178,155 +77,30 @@ func newImageTestService(
t.Cleanup(lifecycle.RequireStop)
db := database.NewTestDatabase(t)
dataDir := t.TempDir()
cfg := &config.Config{DataDir: dataDir, HostDataDir: dataDir}
return deploy.NewTestServiceWithConfig(log, cfg, db, dockerClient), db
}
// saveApp saves an app with the given current and previous image.
func saveApp(
t *testing.T,
db *database.Database,
name, imageID, previousImageID string,
) *models.App {
t.Helper()
app := models.NewApp(db)
app.ID = name + "-id"
app.Name = name
app.ImageID = sql.NullString{String: imageID, Valid: true}
app.PreviousImageID = sql.NullString{String: previousImageID, Valid: true}
require.NoError(t, app.Save(context.Background()))
return app
}
// TestRecordDeployedImageRemovesOldImages runs the step after a deploy
// against a fake Docker API. Image one has a tag from before images were
// tagged with their commit, and is also tagged for another app. Image two
// was the previous image, three the current one, four is new. Image five is
// the other app's previous image, which a redeploy of its commit left
// without the other app's tag.
func TestRecordDeployedImageRemovesOldImages(t *testing.T) {
t.Parallel()
api := &fakeImageAPI{images: map[string][]string{
"sha256:one": {"upaas-myapp:140", "upaas-otherapp:1a2b3c4"},
"sha256:two": {"upaas-myapp:2b3c4d5"},
"sha256:three": {"upaas-myapp:3c4d5e6"},
"sha256:four": {"upaas-myapp:4d5e6f7"},
"sha256:five": {"upaas-myapp:5e6f7a8"},
}}
svc, db := newImageTestService(t, api)
ctx := context.Background()
app := saveApp(t, db, "myapp", "sha256:three", "sha256:two")
saveApp(t, db, "otherapp", "sha256:other", "sha256:five")
app := models.NewApp(db)
app.ID = "myapp-id"
app.Name = "myapp"
app.ImageID = sql.NullString{String: "sha256:three", Valid: true}
app.PreviousImageID = sql.NullString{String: "sha256:two", Valid: true}
require.NoError(t, app.Save(ctx))
deployment := models.NewDeployment(db)
deployment.AppID = app.ID
require.NoError(t, deployment.Save(ctx))
err := svc.RecordDeployedImage(ctx, app, deployment, "sha256:four")
svc := deploy.NewTestServiceWithConfig(log, &config.Config{}, dockerClient)
err = svc.RecordDeployedImage(ctx, app, deployment, "sha256:four")
require.NoError(t, err)
assert.Equal(t, "sha256:four", app.ImageID.String)
assert.Equal(t, "sha256:three", app.PreviousImageID.String)
images, removed := api.state()
mu.Lock()
defer mu.Unlock()
assert.Equal(t, []string{"upaas-myapp:140", "upaas-myapp:2b3c4d5"}, removed)
assert.Equal(t, map[string][]string{
"sha256:one": {"upaas-otherapp:1a2b3c4"},
"sha256:three": {"upaas-myapp:3c4d5e6"},
"sha256:four": {"upaas-myapp:4d5e6f7"},
"sha256:five": {"upaas-myapp:5e6f7a8"},
}, images)
assert.False(t, api.forced, "old images must be removed without force")
}
// TestRedeployRemovesImagesLeftWithoutTag deploys commits against a fake
// Docker API, some of them again. A build takes the commit's tag from the
// image an earlier build of it made. That image is kept, without a tag,
// while the app runs it or Rollback would start it, and is removed by its
// ID once neither does.
func TestRedeployRemovesImagesLeftWithoutTag(t *testing.T) {
t.Parallel()
const (
tagABC1234 = "upaas-myapp:abc1234"
tag0123ABC = "upaas-myapp:0123abc"
retriedImage = "sha256:retried-0123abc"
)
// The app runs commit abc1234 and would roll back to def5678. The last
// deploy, of commit 0123abc, failed after its build.
api := &fakeImageAPI{images: map[string][]string{
"sha256:built-abc1234": {tagABC1234},
"sha256:built-def5678": {"upaas-myapp:def5678"},
"sha256:failed-0123abc": {tag0123ABC},
}}
svc, db := newImageTestService(t, api)
ctx := context.Background()
app := saveApp(t, db, "myapp", "sha256:built-abc1234", "sha256:built-def5678")
for imageID := range api.images {
deployment := models.NewDeployment(db)
deployment.AppID = app.ID
deployment.ImageID = sql.NullString{String: imageID, Valid: true}
require.NoError(t, deployment.Save(ctx))
}
deployCommit := func(shortSHA, imageID string) {
t.Helper()
api.mu.Lock()
api.shortSHA = shortSHA
api.nextID = imageID
api.mu.Unlock()
deployment := models.NewDeployment(db)
deployment.AppID = app.ID
require.NoError(t, deployment.Save(ctx))
built, err := svc.BuildImage(ctx, app, deployment)
require.NoError(t, err)
require.NoError(t, svc.RecordDeployedImage(ctx, app, deployment, built))
}
// The failed deploy's image loses its tag, and nothing uses it.
deployCommit("0123abc", retriedImage)
images, _ := api.state()
assert.Equal(t, map[string][]string{
"sha256:built-abc1234": {tagABC1234},
retriedImage: {tag0123ABC},
}, images)
// The running image loses its tag and becomes the one Rollback starts.
deployCommit("0123abc", "sha256:rebuilt-0123abc")
images, _ = api.state()
assert.Equal(t, map[string][]string{
"sha256:rebuilt-0123abc": {tag0123ABC},
retriedImage: {},
}, images)
assert.Equal(t, retriedImage, app.PreviousImageID.String)
// Once Rollback no longer needs it, the untagged image is removed.
deployCommit("4567def", "sha256:built-4567def")
images, removed := api.state()
assert.Equal(t, map[string][]string{
"sha256:built-4567def": {"upaas-myapp:4567def"},
"sha256:rebuilt-0123abc": {tag0123ABC},
}, images)
assert.Equal(t, []string{
"sha256:failed-0123abc", "upaas-myapp:def5678", // first deploy
tagABC1234, // second deploy
retriedImage, // third deploy
}, removed)
assert.False(t, api.forced, "old images must be removed without force")
assert.Equal(t, []string{"upaas-myapp:1", "upaas-myapp:2"}, removed)
assert.False(t, forced, "old image tags must be removed without force")
}
+1 -14
View File
@@ -9,7 +9,6 @@ import (
"strings"
"sneak.berlin/go/upaas/internal/config"
"sneak.berlin/go/upaas/internal/database"
"sneak.berlin/go/upaas/internal/docker"
"sneak.berlin/go/upaas/internal/models"
)
@@ -45,18 +44,15 @@ func (svc *Service) UnlockApp(appID string) {
svc.unlockApp(appID)
}
// NewTestServiceWithConfig creates a Service with config, database and
// docker client for testing.
// NewTestServiceWithConfig creates a Service with config and docker client for testing.
func NewTestServiceWithConfig(
log *slog.Logger,
cfg *config.Config,
db *database.Database,
dockerClient *docker.Client,
) *Service {
return &Service{
log: log,
config: cfg,
db: db,
docker: dockerClient,
}
}
@@ -104,15 +100,6 @@ func (svc *Service) RecordDeployedImage(
return svc.recordDeployedImage(ctx, app, deployment, imageID)
}
// BuildImage exposes buildImage for testing.
func (svc *Service) BuildImage(
ctx context.Context,
app *models.App,
deployment *models.Deployment,
) (docker.ImageID, error) {
return svc.buildImage(ctx, app, deployment)
}
// BuildContainerOptionsExported exposes buildContainerOptions for testing.
func (svc *Service) BuildContainerOptionsExported(
ctx context.Context,
+43 -44
View File
@@ -5,7 +5,7 @@
{{define "content"}}
{{template "nav" .}}
<main class="mx-auto px-4 py-8" style="max-width: 84rem;" x-data="appDetail({
<main class="max-w-4xl mx-auto px-4 py-8" x-data="appDetail({
appId: '{{.App.ID}}',
initialDeploymentId: {{if .LatestDeployment}}{{.LatestDeployment.ID}}{{else}}null{{end}},
initialStatus: '{{.App.Status}}',
@@ -26,12 +26,11 @@
<!-- Header -->
<div class="flex flex-col sm:flex-row sm:items-center sm:justify-between gap-4 mb-8">
<div>
<div class="flex flex-wrap items-center gap-3">
<div class="flex items-center gap-3">
<h1 class="text-2xl font-medium text-gray-900">{{.App.Name}}</h1>
<span x-bind:class="statusBadgeClass" x-text="statusLabel"></span>
<span class="badge-neutral font-mono break-all" title="Branch">{{.App.Branch}}</span>
</div>
<p class="text-gray-500 font-mono text-sm mt-1">{{.App.RepoURL}}</p>
<p class="text-gray-500 font-mono text-sm mt-1">{{.App.RepoURL}}@{{.App.Branch}}</p>
</div>
<div class="flex gap-3">
<a href="/apps/{{.App.ID}}/edit" class="btn-secondary">Edit</a>
@@ -54,26 +53,6 @@
</div>
</div>
<!-- Container Logs -->
<div class="card p-6 mb-6">
<div class="flex items-center justify-between mb-4">
<h2 class="section-title">Container Logs</h2>
<span x-bind:class="containerStatusBadgeClass" x-text="containerStatusLabel"></span>
</div>
<div class="relative">
<div x-ref="containerLogsWrapper" class="bg-gray-900 rounded-lg p-4 overflow-y-auto" style="max-height: 800px;">
<pre class="text-gray-100 text-xs font-mono whitespace-pre-wrap break-words m-0" x-text="containerLogs"></pre>
</div>
<button
x-show="!_containerAutoScroll"
x-transition
@click="_containerAutoScroll = true; Alpine.store('utils').scrollToBottom($refs.containerLogsWrapper)"
class="absolute bottom-2 right-4 bg-primary-600 hover:bg-primary-700 text-white text-xs px-3 py-1 rounded-full shadow-lg opacity-90 hover:opacity-100 transition"
title="Scroll to bottom"
>↓ Follow</button>
</div>
</div>
<!-- Deploy Key -->
<div class="card p-6 mb-6">
<h2 class="section-title mb-4">Deploy Key</h2>
@@ -121,26 +100,6 @@
</div>
</div>
<!-- Last Deployment Build Logs -->
<div class="card p-6 mb-6" x-show="showBuildLogs" x-cloak>
<div class="flex items-center justify-between mb-4">
<h2 class="section-title">Last Deployment Build Logs</h2>
<span x-bind:class="buildStatusBadgeClass" x-text="buildStatusLabel"></span>
</div>
<div class="relative">
<div x-ref="buildLogsWrapper" class="bg-gray-900 rounded-lg p-4 overflow-y-auto" style="max-height: 800px;">
<pre class="text-gray-100 text-xs font-mono whitespace-pre-wrap break-words m-0" x-text="buildLogs"></pre>
</div>
<button
x-show="!_buildAutoScroll"
x-transition
@click="_buildAutoScroll = true; Alpine.store('utils').scrollToBottom($refs.buildLogsWrapper)"
class="absolute bottom-2 right-4 bg-primary-600 hover:bg-primary-700 text-white text-xs px-3 py-1 rounded-full shadow-lg opacity-90 hover:opacity-100 transition"
title="Scroll to bottom"
>↓ Follow</button>
</div>
</div>
<!-- Environment Variables -->
<div class="card p-6 mb-6" x-data="envVarEditor('{{.App.ID}}')">
<h2 class="section-title mb-4">Environment Variables</h2>
@@ -394,6 +353,26 @@
</form>
</div>
<!-- Container Logs -->
<div class="card p-6 mb-6">
<div class="flex items-center justify-between mb-4">
<h2 class="section-title">Container Logs</h2>
<span x-bind:class="containerStatusBadgeClass" x-text="containerStatusLabel"></span>
</div>
<div class="relative">
<div x-ref="containerLogsWrapper" class="bg-gray-900 rounded-lg p-4 overflow-y-auto" style="max-height: 400px;">
<pre class="text-gray-100 text-xs font-mono whitespace-pre-wrap break-words m-0" x-text="containerLogs"></pre>
</div>
<button
x-show="!_containerAutoScroll"
x-transition
@click="_containerAutoScroll = true; Alpine.store('utils').scrollToBottom($refs.containerLogsWrapper)"
class="absolute bottom-2 right-4 bg-primary-600 hover:bg-primary-700 text-white text-xs px-3 py-1 rounded-full shadow-lg opacity-90 hover:opacity-100 transition"
title="Scroll to bottom"
>↓ Follow</button>
</div>
</div>
<!-- Recent Deployments -->
<div class="card p-6 mb-6">
<div class="flex items-center justify-between mb-4">
@@ -433,6 +412,26 @@
</template>
</div>
<!-- Last Deployment Build Logs -->
<div class="card p-6 mb-6" x-show="showBuildLogs" x-cloak>
<div class="flex items-center justify-between mb-4">
<h2 class="section-title">Last Deployment Build Logs</h2>
<span x-bind:class="buildStatusBadgeClass" x-text="buildStatusLabel"></span>
</div>
<div class="relative">
<div x-ref="buildLogsWrapper" class="bg-gray-900 rounded-lg p-4 overflow-y-auto" style="max-height: 400px;">
<pre class="text-gray-100 text-xs font-mono whitespace-pre-wrap break-words m-0" x-text="buildLogs"></pre>
</div>
<button
x-show="!_buildAutoScroll"
x-transition
@click="_buildAutoScroll = true; Alpine.store('utils').scrollToBottom($refs.buildLogsWrapper)"
class="absolute bottom-2 right-4 bg-primary-600 hover:bg-primary-700 text-white text-xs px-3 py-1 rounded-full shadow-lg opacity-90 hover:opacity-100 transition"
title="Scroll to bottom"
>↓ Follow</button>
</div>
</div>
<!-- Danger Zone -->
<div class="card border-2 border-error-500/20 bg-error-50/50 p-6">
<h2 class="text-lg font-medium text-error-700 mb-4">Danger Zone</h2>