From 3950e33fffa0142260ac4925403175ed7232653d Mon Sep 17 00:00:00 2001 From: clawbot Date: Sat, 28 Feb 2026 02:38:27 -0800 Subject: [PATCH] =?UTF-8?q?add=20AUTOMATED=5FDEV.md:=20Gitea=20+=20Matterm?= =?UTF-8?q?ost=20+=20CI=20+=20=C2=B5PaaS=20pipeline?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- AUTOMATED_DEV.md | 249 +++++++++++++++++++++++++++++++++++++++++++++++ README.md | 5 + 2 files changed, 254 insertions(+) create mode 100644 AUTOMATED_DEV.md diff --git a/AUTOMATED_DEV.md b/AUTOMATED_DEV.md new file mode 100644 index 0000000..4f31476 --- /dev/null +++ b/AUTOMATED_DEV.md @@ -0,0 +1,249 @@ +# Automated Development Systems + +How we connect self-hosted Gitea, Mattermost, CI, and a lightweight PaaS into a +continuous development pipeline where code goes from PR to production with +minimal human intervention. + +--- + +## The Hub: Self-Hosted Gitea + +[Gitea](https://gitea.io) is the coordination center. Every repo, issue, PR, and +code review lives here. We self-host it because: + +- Full API access for automation (the agent has its own Gitea user with API + token) +- Gitea Actions for CI (compatible with GitHub Actions syntax) +- Webhooks on every repo event +- No vendor lock-in, no rate limits, no surprise pricing changes +- Complete control over visibility (public/private per-repo) + +The agent (OpenClaw) has its own Gitea account (`clawbot`) separate from the +human user. This matters because: + +- The agent's commits, comments, and PR reviews are clearly attributed +- The human can @-mention the agent in issues to assign work +- Assignment is unambiguous — issues assigned to `clawbot` are the agent's + queue, issues assigned to the human are theirs +- The agent can be given write access per-repo (some repos it can push to + directly, others it must fork and PR) +- API rate limits and permissions are independent + +## Real-Time Activity Feed: Gitea → Mattermost + +Every repo has a Gitea webhook that sends all activity (pushes, PRs, issues, +comments, reviews, CI status) to a channel in our self-hosted +[Mattermost](https://mattermost.com) instance. This creates a real-time feed +where everyone — human and agent — can see what's happening across all projects +without cross-communication overhead. + +The agent also has its own Mattermost bot user (`@claw`), separate from the +human. This means: + +- The agent posts status updates to dedicated channels (`#git` for Gitea work, + `#claw` for general work narration) +- The human's DMs stay clean — only direct alerts and responses +- In group channels, it's clear who said what +- The agent can be @-mentioned in any channel + +### Channel Architecture + +A practical setup: + +- **#git** — Real-time Gitea webhook feed (all repos) + agent's work status + updates. Everyone sees commits, PRs, reviews, CI results as they happen. +- **#claw** — Agent's internal work narration. Useful for debugging what the + agent is doing, but notifications muted so it doesn't disturb anyone. +- **DM with agent** — Private conversation, sitreps, sensitive commands +- **Project-specific channels** — For coordination with external collaborators + +The webhook feed in #git means nobody needs to check Gitea's notification inbox +manually. PRs, reviews, and CI results flow into a channel that's always open. +The agent monitors the same feed (via its notification poller) and can react to +events in near-realtime. + +## CI: Gitea Actions + +Every repo has a CI workflow in `.gitea/workflows/` that runs on push. The +standard workflow is one line: + +```yaml +- name: Build and check + run: docker build . +``` + +Because the Dockerfile runs `make check` (which runs tests, linting, and +formatting checks), a successful Docker build means everything passes. A failed +build means something is broken. Binary signal, no ambiguity. + +### PR Preview Deployments + +For web projects (like a blog or documentation site), CI can go further than +pass/fail. When a PR is opened against a site repo, CI can: + +1. Build the site from the PR branch +2. Deploy it to a preview URL +3. Post the preview URL as a comment on the PR or drop it into a Mattermost + channel + +This lets reviewers see the actual rendered result before merging, not just the +code diff. For a Jekyll/Hugo blog, this means you can see how a new post looks +on the real site layout, on mobile, with real CSS — before it goes live. + +## Deployment: µPaaS + +[µPaaS](https://git.eeqj.de/sneak/upaas) is a lightweight, self-hosted +platform-as-a-service that auto-deploys Docker containers when code changes. It +exists because: + +- Full PaaS platforms (Kubernetes, Nomad, etc.) are massive overkill for a small + fleet of services +- Heroku/Render/Fly.io mean vendor dependency and recurring costs +- We wanted: push to main → live in production, automatically, with zero human + intervention + +### How It Works + +µPaaS is a single Go binary that: + +1. **Receives Gitea webhooks** on push/merge events +2. **Clones the repo** using deploy keys (read-only SSH keys per-repo) +3. **Runs `docker build`** to build the new image +4. **Swaps the running container** with the new image +5. **Routes traffic** via Traefik reverse proxy with automatic TLS + +The deploy flow: + +``` +Developer merges PR to main/prod + → Gitea fires webhook to µPaaS + → µPaaS clones repo, builds Docker image + → µPaaS stops old container, starts new one + → Traefik routes traffic to new container + → Site is live on its production URL with TLS +``` + +Time from merge to live: typically under 2 minutes (dominated by Docker build +time). + +### Deploy Keys + +Each repo that deploys via µPaaS has a read-only SSH deploy key. This means: + +- µPaaS can clone the repo to build it, but can't push to it +- Each key is scoped to one repo — compromise of one key doesn't affect others +- No shared credentials, no broad API tokens + +### What's Deployed This Way + +Any Docker-based service or site: + +- Static sites (Jekyll, Hugo) — Dockerfile builds the site, nginx serves it +- Go services — Dockerfile builds the binary, runs it +- Web applications — same pattern + +Everything gets a production URL with automatic TLS via Traefik. + +## The Full Pipeline + +Putting it all together, the development lifecycle looks like this: + +``` +1. Issue filed in Gitea (by human or agent) + ↓ +2. Agent picks up the issue (via notification poller) + ↓ +3. Agent posts "starting work on #N" to Mattermost #git + ↓ +4. Agent (or sub-agent) creates branch, writes code, pushes + ↓ +5. Gitea webhook fires → #git shows the push + ↓ +6. CI runs docker build → passes or fails + ↓ +7. Agent creates PR "(closes #N)" + ↓ +8. Gitea webhook fires → #git shows the PR + ↓ +9. Agent reviews code, runs make check locally, verifies + ↓ +10. Agent assigns PR to human when all checks pass + ↓ +11. Human reviews, requests changes or approves + ↓ +12. If changes requested → agent reworks, back to step 6 + ↓ +13. Human merges PR + ↓ +14. Gitea webhook fires → µPaaS deploys to production + ↓ +15. Gitea webhook fires → #git shows the merge + ↓ +16. Site/service is live on production URL +``` + +Steps 2-10 can happen without any human involvement. The human's role is reduced +to: review the PR, approve or request changes, merge. Everything else is +automated. + +### Observability + +Because everything flows through Mattermost channels: + +- The human can glance at #git to see the current state of all projects +- CI failures are immediately visible (Gitea Actions posts status) +- Deployments are immediately visible (µPaaS can log to the same channel) +- The agent's work narration in #claw shows what it's currently doing +- No need to check multiple dashboards — one chat client shows everything + +## Identity Separation + +A key architectural decision: the agent has its own identity everywhere. + +| System | Human Account | Agent Account | +| ---------- | ------------- | ------------- | +| Gitea | @sneak | @clawbot | +| Mattermost | @sneak | @claw | + +This separation means: + +- **Clear attribution.** Every commit, comment, and message shows who did it. + When reviewing git history, you know which commits were human and which were + agent. +- **Independent permissions.** The agent can have write access to repos where + it's trusted, fork-and-PR access where it's not. The human can have admin + access without the agent inheriting it. +- **Mentionability.** The human can @-mention the agent in an issue comment to + assign work. The agent can @-mention the human when it needs review. This + works exactly like human-to-human collaboration. +- **Separate notification streams.** The agent's notification poller watches + `@clawbot`'s inbox. The human's notifications are separate. No cross-talk. + +## Why Self-Host Everything + +The stack — Gitea, Mattermost, µPaaS, OpenClaw — is entirely self-hosted. This +isn't ideological; it's practical: + +- **No API rate limits.** The agent makes dozens of API calls per hour to Gitea. + GitHub's API limits would throttle it. +- **No surprise costs.** CI minutes, seat licenses, storage — all free when + self-hosted. +- **Full API access.** Every feature of every tool is available via API. No + "enterprise only" gates. +- **Custom webhooks.** We can wire up any event to any action. Gitea push → + Mattermost notification → µPaaS deploy → agent notification, all custom. +- **Data sovereignty.** Code, issues, conversations, and deployment + infrastructure all live on machines we control. +- **Offline resilience.** If GitHub/Slack/Vercel have an outage, our pipeline + keeps running. + +The trade-off is maintenance burden, but with an AI agent handling most of the +operational work (monitoring, updates, issue triage), the maintenance cost is +surprisingly low. + +--- + +_This document describes a production system that's been running since +early 2026. The specific tools (Gitea, Mattermost, µPaaS) are interchangeable — +the patterns (webhook-driven deployment, real-time activity feeds, identity +separation, automated CI gates) apply to any self-hosted stack._ diff --git a/README.md b/README.md index f2c80a4..5e5a3a6 100644 --- a/README.md +++ b/README.md @@ -7,6 +7,11 @@ agent) under the direction of [@sneak](https://sneak.berlin). ## What's Here +- **[AUTOMATED_DEV.md](AUTOMATED_DEV.md)** — How self-hosted Gitea, Mattermost, + Gitea Actions CI, and µPaaS connect into a continuous pipeline where code goes + from PR to production automatically. Covers webhook-driven deployments, + real-time activity feeds, PR preview sites, identity separation (agent vs + human accounts), and why self-hosting the entire stack matters. - **[OPENCLAW_TRICKS.md](OPENCLAW_TRICKS.md)** — Tested configuration recipes for OpenClaw agents: workspace bootstrapping, daily context state files, sitrep (situation report) prompts, sleep tracking, medication tracking,