clawpub/AUTOMATED_DEV.md

# Automated Development Systems

How we connect self-hosted Gitea, Mattermost, CI, and a lightweight PaaS into a
continuous development pipeline where code goes from PR to production with
minimal human intervention.

---

## The Hub: Self-Hosted Gitea

[Gitea](https://gitea.io) is the coordination center. Every repo, issue, PR, and
code review lives here. We self-host it because:

- Full API access for automation (the agent has its own Gitea user with API
  token)
- Gitea Actions for CI (compatible with GitHub Actions syntax)
- Webhooks on every repo event
- No vendor lock-in, no rate limits, no surprise pricing changes
- Complete control over visibility (public/private per-repo)

The agent (OpenClaw) has its own Gitea account (`clawbot`) separate from the
human user. This matters because:

- The agent's commits, comments, and PR reviews are clearly attributed
- The human can @-mention the agent in issues to assign work
- Assignment is unambiguous — issues assigned to `clawbot` are the agent's
  queue, issues assigned to the human are theirs
- The agent can be given write access per-repo (some repos it can push to
  directly, others it must fork and PR)
- API rate limits and permissions are independent

### Issues as the To-Do List

Gitea issues aren't just bug reports — they're the universal task queue. Every
piece of work, from feature requests to personal errands, lives as an issue:

- **Issue = task.** If it needs doing, it's an issue. No separate task manager,
  no Notion boards, no sticky notes.
- **Assignment = ownership.** An issue assigned to `clawbot` means the agent is
  responsible for the next step. An issue assigned to the human means it's in
  their queue. An unassigned issue is unclaimed work.
- **Assignment as coordination flag.** When the agent finishes its part of a
  task but needs human input, it unassigns itself and assigns the human. When
  the human delegates work, they assign the agent. The assignment field IS the
  handoff mechanism — no need for separate status updates or "hey, this is ready
  for you" messages.
- **Close = done.** When an issue is closed, the work is complete. PRs reference
  issues with "(closes #N)" so merging the PR automatically closes the issue.

This means you can answer "what's on my plate?" with a single API call:
`GET /repos/{owner}/{repo}/issues?assignee=me&state=open`. The agent does this
during every sitrep and heartbeat.

For personal task management, we even have a dedicated issues-only repo (no
code) that serves as a general to-do list. The agent reviews open issues there,
suggests ways to complete tasks with minimal effort, and proactively comments
with research or offers to handle items directly.

### The Issue → PR → Deploy Lifecycle

Issues drive the entire pipeline:

1. Issue filed (by human or agent)
2. Assigned to whoever should work on it
3. Work happens on a feature branch
4. PR created with "(closes #N)" in the title
5. PR reviewed, checked, merged
6. Issue auto-closes on merge
7. Code auto-deploys to production (if applicable)

The issue tracker is the single source of truth for what needs doing, who's
doing it, and what's done. Everything else (PRs, branches, deployments) links
back to issues.

### PR State Machine

Once a PR exists, it enters a finite state machine tracked by Gitea labels and
issue assignments. Labels represent the current state; the assignment field
represents who's responsible for the next action.

#### States (Gitea Labels)

| Label          | Color  | Meaning                                           |
| -------------- | ------ | ------------------------------------------------- |
| `needs-rebase` | red    | PR has merge conflicts or is behind main          |
| `needs-checks` | orange | `make check` does not pass cleanly                |
| `needs-review` | yellow | Code review not yet done                          |
| `needs-rework` | purple | Code review found issues that need fixing         |
| `merge-ready`  | green  | All checks pass, reviewed, rebased, conflict-free |

#### Transitions

```
New PR created
  │
  ▼
[needs-rebase] ──rebase onto main──▶ [needs-checks]
  ▲                                        │
  │                                   run make check
  │ (main updated,                         │
  │  conflicts)              ┌─────────────┴──────────────┐
  │                          │                              │
  │                       passes                          fails
  │                          │                              │
  │                          ▼                              ▼
  │                   [needs-review]                  [needs-checks]
  │                          │                        (fix code, re-run)
  │                     code review
  │                          │
  │                ┌─────────┴──────────┐
  │                │                      │
  │             approved             issues found
  │                │                      │
  │                ▼                      ▼
  │          [merge-ready]          [needs-rework]
  │                │                      │
  │           assign human           fix issues
  │                                       │
  │                                       ▼
  └───────────────────────────── [needs-rebase]
                                 (restart cycle)
```

The cycle can repeat multiple times: rebase → check → review → rework → rebase →
check → review → rework → ... until the PR is clean. Each iteration typically
addresses a smaller set of issues until everything converges.

#### Assignment Rules

- **PR in any state except `merge-ready`** → assigned to the agent. It's the
  agent's job to drive it forward through the state machine.
- **PR reaches `merge-ready`** → assigned to the human. This is the ONLY time a
  PR should land in the human's queue.
- **Human requests changes during review** → PR moves back to `needs-rework`,
  reassigned to agent.

This means the human's PR inbox contains only PRs that are genuinely ready to
merge — no half-finished work, no failing CI, no merge conflicts. Everything
else is the agent's problem.

#### The Loop in Practice

A typical PR might go through this cycle:

1. Agent creates PR, labels `needs-rebase`
2. Agent rebases onto main → labels `needs-checks`
3. Agent runs `make check` — lint fails → fixes lint, pushes → back to
   `needs-rebase` (new commit)
4. Agent rebases → `needs-checks` → runs checks → passes → `needs-review`
5. Agent does code review — finds a missing error check → `needs-rework`
6. Agent fixes the error check, pushes → `needs-rebase`
7. Agent rebases → `needs-checks` → passes → `needs-review`
8. Agent reviews — looks good → `merge-ready`
9. Agent assigns to human
10. Human reviews, merges

Steps 1-9 happen without human involvement. The human sees a clean, reviewed,
passing PR ready for a final look.

#### Automated Sweep

A periodic cron job (every 4 hours) scans all open PRs across all repos:

- **No label** → classify into the correct state
- **`needs-rebase`** → spawn agent to rebase
- **`needs-checks`** → spawn agent to run checks and fix failures
- **`needs-review`** → spawn agent to do code review
- **`needs-rework`** → spawn agent to fix review feedback
- **`merge-ready`** → verify still true (main may have updated since), ensure
  assigned to human

This catches PRs that fell through the cracks — an agent session that timed out
mid-rework, a rebase that became necessary when main moved forward, etc.

#### Why Labels + Assignments

You could track PR state in a file, a database, or just in the agent's memory.
Labels and assignments are better because:

- **Visible in the Gitea UI.** Anyone can glance at the PR list and see what
  state each PR is in without reading comments.
- **Queryable via API.** "Show me all PRs that need review" is a single API call
  with a label filter.
- **Durable.** Labels survive agent restarts, session timeouts, and context
  loss. The state is in Gitea, not in the agent's head.
- **Human-readable.** Color-coded labels in a PR list give an instant dashboard:
  lots of red = rebase debt, lots of orange = CI problems, lots of green = ready
  for review.

#### Branch Protection

The state machine assumes nobody can bypass it by pushing directly to main.
Branch protection rules on Gitea (or GitHub) enforce this at the server level:

- **Require pull request reviews** before merging — no direct pushes to main
- **Require CI to pass** — the Docker build (which runs `make check`) must
  succeed before a PR can be merged
- **Block force-pushes** — history is immutable on protected branches
- **Require branches to be up-to-date** — PRs must be rebased before merge

This is the server-side interlock that makes the entire state machine
trustworthy. Without branch protection, an agent could skip the review/check
cycle and push directly to main. With it, the only path to main is through a PR
that passes all gates. A tired human at 3am, or an overconfident agent,
physically cannot bypass the review and CI gates — the server won't allow it.

Branch protection completes the interlocking chain:

```
Branch protection (server-side)
  └── Requires CI pass
        └── CI runs docker build
              └── Dockerfile runs make check
                    ├── make fmt-check
                    ├── make lint
                    └── make test
```

Every layer enforces the one below it. The developer (human or agent) can't skip
any step because each gate is enforced by a different system: the Makefile
enforces test/lint/fmt, the Dockerfile enforces the Makefile, CI enforces the
Docker build, and branch protection enforces CI. No single point of failure.

## Real-Time Activity Feed: Gitea → Mattermost

Every repo has a Gitea webhook that sends all activity (pushes, PRs, issues,
comments, reviews, CI status) to a channel in our self-hosted
[Mattermost](https://mattermost.com) instance. This creates a real-time feed
where everyone — human and agent — can see what's happening across all projects
without cross-communication overhead.

The agent also has its own Mattermost bot user (`@claw`), separate from the
human. This means:

- The agent posts status updates to dedicated channels (`#git` for Gitea work,
  `#claw` for general work narration)
- The human's DMs stay clean — only direct alerts and responses
- In group channels, it's clear who said what
- The agent can be @-mentioned in any channel

### Channel Architecture

A practical setup:

- **#git** — Real-time Gitea webhook feed (all repos) + agent's work status
  updates. Everyone sees commits, PRs, reviews, CI results as they happen.
- **#claw** — Agent's internal work narration. Useful for debugging what the
  agent is doing, but notifications muted so it doesn't disturb anyone.
- **DM with agent** — Private conversation, sitreps, sensitive commands
- **Project-specific channels** — For coordination with external collaborators

The webhook feed in #git means nobody needs to check Gitea's notification inbox
manually. PRs, reviews, and CI results flow into a channel that's always open.
The agent monitors the same feed (via its notification poller) and can react to
events in near-realtime.

## CI: Gitea Actions

Every repo has a CI workflow in `.gitea/workflows/` that runs on push. The
standard workflow is one line:

```yaml
- name: Build and check
  run: docker build .
```

Because the Dockerfile runs `make check` (which runs tests, linting, and
formatting checks), a successful Docker build means everything passes. A failed
build means something is broken. Binary signal, no ambiguity.

### PR Preview Deployments

For web projects (like a blog or documentation site), CI can go further than
pass/fail. When a PR is opened against a site repo, CI can:

1. Build the site from the PR branch
2. Deploy it to a preview URL
3. Post the preview URL as a comment on the PR or drop it into a Mattermost
   channel

This lets reviewers see the actual rendered result before merging, not just the
code diff. For a Jekyll/Hugo blog, this means you can see how a new post looks
on the real site layout, on mobile, with real CSS — before it goes live.

## Deployment: µPaaS

[µPaaS](https://git.eeqj.de/sneak/upaas) is a lightweight, self-hosted
platform-as-a-service that auto-deploys Docker containers when code changes. It
exists because:

- Full PaaS platforms (Kubernetes, Nomad, etc.) are massive overkill for a small
  fleet of services
- Heroku/Render/Fly.io mean vendor dependency and recurring costs
- We wanted: push to main → live in production, automatically, with zero human
  intervention

### How It Works

µPaaS is a single Go binary that:

1. **Receives Gitea webhooks** on push/merge events
2. **Clones the repo** using deploy keys (read-only SSH keys per-repo)
3. **Runs `docker build`** to build the new image
4. **Swaps the running container** with the new image
5. **Routes traffic** via Traefik reverse proxy with automatic TLS

The deploy flow:

```
Developer merges PR to main/prod
  → Gitea fires webhook to µPaaS
    → µPaaS clones repo, builds Docker image
      → µPaaS stops old container, starts new one
        → Traefik routes traffic to new container
          → Site is live on its production URL with TLS
```

Time from merge to live: typically under 2 minutes (dominated by Docker build
time).

### Deploy Keys

Each repo that deploys via µPaaS has a read-only SSH deploy key. This means:

- µPaaS can clone the repo to build it, but can't push to it
- Each key is scoped to one repo — compromise of one key doesn't affect others
- No shared credentials, no broad API tokens

### What's Deployed This Way

Any Docker-based service or site:

- Static sites (Jekyll, Hugo) — Dockerfile builds the site, nginx serves it
- Go services — Dockerfile builds the binary, runs it
- Web applications — same pattern

Everything gets a production URL with automatic TLS via Traefik.

## The Full Pipeline

Putting it all together, the development lifecycle looks like this:

```
1. Issue filed in Gitea (by human or agent)
     ↓
2. Agent picks up the issue (via notification poller)
     ↓
3. Agent posts "starting work on #N" to Mattermost #git
     ↓
4. Agent (or sub-agent) creates branch, writes code, pushes
     ↓
5. Gitea webhook fires → #git shows the push
     ↓
6. CI runs docker build → passes or fails
     ↓
7. Agent creates PR "(closes #N)"
     ↓
8. Gitea webhook fires → #git shows the PR
     ↓
9. Agent reviews code, runs make check locally, verifies
     ↓
10. Agent assigns PR to human when all checks pass
     ↓
11. Human reviews, requests changes or approves
     ↓
12. If changes requested → agent reworks, back to step 6
     ↓
13. Human merges PR
     ↓
14. Gitea webhook fires → µPaaS deploys to production
     ↓
15. Gitea webhook fires → #git shows the merge
     ↓
16. Site/service is live on production URL
```

Steps 2-10 can happen without any human involvement. The human's role is reduced
to: review the PR, approve or request changes, merge. Everything else is
automated.

### Observability

Because everything flows through Mattermost channels:

- The human can glance at #git to see the current state of all projects
- CI failures are immediately visible (Gitea Actions posts status)
- Deployments are immediately visible (µPaaS can log to the same channel)
- The agent's work narration in #claw shows what it's currently doing
- No need to check multiple dashboards — one chat client shows everything

## Identity Separation

A key architectural decision: the agent has its own identity everywhere.

| System     | Human Account | Agent Account |
| ---------- | ------------- | ------------- |
| Gitea      | @sneak        | @clawbot      |
| Mattermost | @sneak        | @claw         |

This separation means:

- **Clear attribution.** Every commit, comment, and message shows who did it.
  When reviewing git history, you know which commits were human and which were
  agent.
- **Independent permissions.** The agent can have write access to repos where
  it's trusted, fork-and-PR access where it's not. The human can have admin
  access without the agent inheriting it.
- **Mentionability.** The human can @-mention the agent in an issue comment to
  assign work. The agent can @-mention the human when it needs review. This
  works exactly like human-to-human collaboration.
- **Separate notification streams.** The agent's notification poller watches
  `@clawbot`'s inbox. The human's notifications are separate. No cross-talk.

## Why Self-Host Everything

The stack — Gitea, Mattermost, µPaaS, OpenClaw — is entirely self-hosted. This
isn't ideological; it's practical:

- **No API rate limits.** The agent makes dozens of API calls per hour to Gitea.
  GitHub's API limits would throttle it.
- **No surprise costs.** CI minutes, seat licenses, storage — all free when
  self-hosted.
- **Full API access.** Every feature of every tool is available via API. No
  "enterprise only" gates.
- **Custom webhooks.** We can wire up any event to any action. Gitea push →
  Mattermost notification → µPaaS deploy → agent notification, all custom.
- **Data sovereignty.** Code, issues, conversations, and deployment
  infrastructure all live on machines we control.
- **Offline resilience.** If GitHub/Slack/Vercel have an outage, our pipeline
  keeps running.

The trade-off is maintenance burden, but with an AI agent handling most of the
operational work (monitoring, updates, issue triage), the maintenance cost is
surprisingly low.

---

_This document describes a production system that's been running since
early 2026. The specific tools (Gitea, Mattermost, µPaaS) are interchangeable —
the patterns (webhook-driven deployment, real-time activity feeds, identity
separation, automated CI gates) apply to any self-hosted stack._