Compare commits
1 Commits
update-pol
...
add-notifi
| Author | SHA1 | Date | |
|---|---|---|---|
|
73f31c75be |
244
AUTOMATED_DEV.md
244
AUTOMATED_DEV.md
@@ -74,108 +74,103 @@ back to issues.
|
||||
|
||||
### PR State Machine
|
||||
|
||||
Once a PR exists, it enters a finite state machine tracked by Gitea labels. Each
|
||||
PR has exactly one state label at a time, plus a `bot` label indicating it's the
|
||||
agent's turn to act.
|
||||
Once a PR exists, it enters a finite state machine tracked by Gitea labels and
|
||||
issue assignments. Labels represent the current state; the assignment field
|
||||
represents who's responsible for the next action.
|
||||
|
||||
#### States (Gitea Labels)
|
||||
|
||||
| Label | Color | Meaning |
|
||||
| -------------- | ------ | --------------------------------------------- |
|
||||
| `needs-review` | yellow | Code pushed, `docker build .` passes, awaiting review |
|
||||
| -------------- | ------ | ------------------------------------------------- |
|
||||
| `needs-rebase` | red | PR has merge conflicts or is behind main |
|
||||
| `needs-checks` | orange | `make check` does not pass cleanly |
|
||||
| `needs-review` | yellow | Code review not yet done |
|
||||
| `needs-rework` | purple | Code review found issues that need fixing |
|
||||
| `merge-ready` | green | Reviewed clean, build passes, ready for human |
|
||||
| `merge-ready` | green | All checks pass, reviewed, rebased, conflict-free |
|
||||
|
||||
Earlier iterations included `needs-rebase` and `needs-checks` states, but we
|
||||
eliminated them. Rebasing is handled inline by workers and reviewers (they
|
||||
rebase onto the target branch as part of their normal work). And `docker build .`
|
||||
is the only check — it's run by workers before pushing and by reviewers before
|
||||
approving. There's no separate "checks" phase.
|
||||
|
||||
#### The `bot` Label + Assignment Model
|
||||
|
||||
The `bot` label signals that an issue or PR is the agent's turn to act. The
|
||||
assignment field tracks who is actively working on it:
|
||||
|
||||
- **`bot` label + unassigned** = work available, poller dispatches an agent
|
||||
- **`bot` label + assigned to agent** = actively being worked
|
||||
- **No `bot` label** = not the agent's turn (either human's turn or done)
|
||||
|
||||
The notification poller assigns the agent account to the issue at dispatch time,
|
||||
before the agent session even starts. This prevents race conditions — by the
|
||||
time a second poller scan runs, the issue is already assigned and gets skipped.
|
||||
|
||||
When the agent finishes its step and spawns the next agent, it unassigns itself
|
||||
first (releasing the lock). The next agent's first action is to verify it's the
|
||||
only one working on the issue by checking comments for duplicate work.
|
||||
|
||||
At chain-end (`merge-ready`): the agent assigns the human and removes the `bot`
|
||||
label. The human's PR inbox contains only PRs that are genuinely ready to merge.
|
||||
|
||||
#### Agent Chaining — No Self-Review
|
||||
|
||||
Each step in the pipeline is handled by a separate, isolated agent session.
|
||||
Agents spawn the next agent in the chain via `openclaw cron add --session
|
||||
isolated`. This enforces a critical rule: **the agent that wrote the code never
|
||||
reviews it.**
|
||||
|
||||
The chain looks like this:
|
||||
#### Transitions
|
||||
|
||||
```
|
||||
Worker agent (writes/fixes code)
|
||||
→ docker build . → push → label needs-review
|
||||
→ unassign self → spawn reviewer agent → STOP
|
||||
|
||||
Reviewer agent (reviews code it didn't write)
|
||||
→ read diff + referenced issues → review
|
||||
→ PASS: rebase if needed → docker build . → label merge-ready
|
||||
→ assign human → remove bot label → STOP
|
||||
→ FAIL: comment findings → label needs-rework
|
||||
→ unassign self → spawn worker agent → STOP
|
||||
New PR created
|
||||
│
|
||||
▼
|
||||
[needs-rebase] ──rebase onto main──▶ [needs-checks]
|
||||
▲ │
|
||||
│ run make check
|
||||
│ (main updated, │
|
||||
│ conflicts) ┌─────────────┴──────────────┐
|
||||
│ │ │
|
||||
│ passes fails
|
||||
│ │ │
|
||||
│ ▼ ▼
|
||||
│ [needs-review] [needs-checks]
|
||||
│ │ (fix code, re-run)
|
||||
│ code review
|
||||
│ │
|
||||
│ ┌─────────┴──────────┐
|
||||
│ │ │
|
||||
│ approved issues found
|
||||
│ │ │
|
||||
│ ▼ ▼
|
||||
│ [merge-ready] [needs-rework]
|
||||
│ │ │
|
||||
│ assign human fix issues
|
||||
│ │
|
||||
│ ▼
|
||||
└───────────────────────────── [needs-rebase]
|
||||
(restart cycle)
|
||||
```
|
||||
|
||||
The cycle repeats (worker → reviewer → worker → reviewer → ...) until the
|
||||
reviewer approves. Each agent is a fresh session with no memory of previous
|
||||
iterations — it reads the issue comments and PR diff to understand context.
|
||||
The cycle can repeat multiple times: rebase → check → review → rework → rebase →
|
||||
check → review → rework → ... until the PR is clean. Each iteration typically
|
||||
addresses a smaller set of issues until everything converges.
|
||||
|
||||
#### TOCTOU Protection
|
||||
#### Assignment Rules
|
||||
|
||||
Just before changing labels or assignments, agents re-read all comments and
|
||||
current labels via the API. If the state changed since they started (another
|
||||
agent already acted), they report the conflict and stop. This prevents stale
|
||||
agents from overwriting fresh state.
|
||||
- **PR in any state except `merge-ready`** → assigned to the agent. It's the
|
||||
agent's job to drive it forward through the state machine.
|
||||
- **PR reaches `merge-ready`** → assigned to the human. This is the ONLY time a
|
||||
PR should land in the human's queue.
|
||||
- **Human requests changes during review** → PR moves back to `needs-rework`,
|
||||
reassigned to agent.
|
||||
|
||||
#### Race Detection
|
||||
|
||||
If an agent starts and finds its work was already done (e.g., a reviewer sees a
|
||||
review was already posted, or a worker sees a PR was already created), it
|
||||
reports to the status channel and stops.
|
||||
This means the human's PR inbox contains only PRs that are genuinely ready to
|
||||
merge — no half-finished work, no failing CI, no merge conflicts. Everything
|
||||
else is the agent's problem.
|
||||
|
||||
#### The Loop in Practice
|
||||
|
||||
A typical PR goes through this cycle:
|
||||
A typical PR might go through this cycle:
|
||||
|
||||
1. Worker agent creates PR, runs `docker build .`, labels `needs-review`
|
||||
2. Worker spawns reviewer agent
|
||||
3. Reviewer reads diff — finds a missing error check → labels `needs-rework`
|
||||
4. Reviewer spawns worker agent
|
||||
5. Worker fixes the error check, rebases, runs `docker build .`, labels
|
||||
`needs-review`
|
||||
6. Worker spawns reviewer agent
|
||||
7. Reviewer reads diff — looks good → rebases → `docker build .` → labels
|
||||
`merge-ready`, assigns human
|
||||
8. Human reviews, merges
|
||||
1. Agent creates PR, labels `needs-rebase`
|
||||
2. Agent rebases onto main → labels `needs-checks`
|
||||
3. Agent runs `make check` — lint fails → fixes lint, pushes → back to
|
||||
`needs-rebase` (new commit)
|
||||
4. Agent rebases → `needs-checks` → runs checks → passes → `needs-review`
|
||||
5. Agent does code review — finds a missing error check → `needs-rework`
|
||||
6. Agent fixes the error check, pushes → `needs-rebase`
|
||||
7. Agent rebases → `needs-checks` → passes → `needs-review`
|
||||
8. Agent reviews — looks good → `merge-ready`
|
||||
9. Agent assigns to human
|
||||
10. Human reviews, merges
|
||||
|
||||
Steps 1-7 happen without human involvement. Each step is a separate agent
|
||||
session that spawns the next one.
|
||||
Steps 1-9 happen without human involvement. The human sees a clean, reviewed,
|
||||
passing PR ready for a final look.
|
||||
|
||||
#### Safety Net
|
||||
#### Automated Sweep
|
||||
|
||||
The notification poller runs a periodic scan (every 2 minutes) of all watched
|
||||
repos for issues/PRs with the `bot` label that are unassigned. This catches
|
||||
broken chains — if an agent crashes or times out without spawning the next agent,
|
||||
the poller will eventually re-dispatch. A 30-minute cooldown prevents duplicate
|
||||
dispatches during normal operation.
|
||||
A periodic cron job (every 4 hours) scans all open PRs across all repos:
|
||||
|
||||
- **No label** → classify into the correct state
|
||||
- **`needs-rebase`** → spawn agent to rebase
|
||||
- **`needs-checks`** → spawn agent to run checks and fix failures
|
||||
- **`needs-review`** → spawn agent to do code review
|
||||
- **`needs-rework`** → spawn agent to fix review feedback
|
||||
- **`merge-ready`** → verify still true (main may have updated since), ensure
|
||||
assigned to human
|
||||
|
||||
This catches PRs that fell through the cracks — an agent session that timed out
|
||||
mid-rework, a rebase that became necessary when main moved forward, etc.
|
||||
|
||||
#### Why Labels + Assignments
|
||||
|
||||
@@ -268,45 +263,26 @@ A practical setup:
|
||||
- **DM with agent** — Private conversation, sitreps, sensitive commands
|
||||
- **Project-specific channels** — For coordination with external collaborators
|
||||
|
||||
### The Notification Poller + Dispatcher
|
||||
### The Notification Poller
|
||||
|
||||
Because the agent can't see Gitea webhooks in Mattermost (bot-to-bot visibility
|
||||
issue), we built a Python script that both polls and dispatches. It polls the
|
||||
Gitea notifications API every 15 seconds, triages each notification (checking
|
||||
@-mentions and assignment), marks them as read, and spawns one isolated agent
|
||||
session per actionable item via `openclaw cron add --session isolated`.
|
||||
|
||||
The poller also runs a secondary **label scan** every 2 minutes, checking all
|
||||
watched repos for open issues/PRs with the `bot` label that are unassigned
|
||||
(meaning they need work but no agent has claimed them yet). This catches cases
|
||||
where the agent chain broke — an agent timed out or crashed without spawning the
|
||||
next one.
|
||||
issue), we built a lightweight Python script that polls the Gitea notifications
|
||||
API every 2 seconds and wakes the agent via OpenClaw's `/hooks/wake` endpoint
|
||||
when new notifications arrive.
|
||||
|
||||
Key design decisions:
|
||||
|
||||
- **The poller IS the dispatcher.** No flag files, no heartbeat dependency. The
|
||||
poller triages notifications and spawns agents directly.
|
||||
- **Marks notifications as read immediately.** Prevents re-dispatch on the next
|
||||
poll cycle.
|
||||
- **Assigns the agent account at dispatch time.** Before spawning the agent
|
||||
session, the poller assigns the bot user to the issue via API. This prevents
|
||||
race conditions — subsequent scans skip assigned issues.
|
||||
- **Dispatched issues are tracked in a persistent JSON file.** Survives poller
|
||||
restarts. Entries auto-prune after 1 hour.
|
||||
- **30-minute re-dispatch cooldown.** The poller won't re-dispatch for the same
|
||||
issue within 30 minutes, even if it appears unassigned again.
|
||||
- **Concurrency cap.** The poller checks how many agents are currently running
|
||||
and defers dispatch if the cap is reached.
|
||||
- **Stale agent reaper.** Kills agent sessions that have been running longer
|
||||
than 10 minutes (the `--timeout-seconds` flag isn't always enforced).
|
||||
- **`bot` label + `merge-ready` skip.** The label scan skips issues that are
|
||||
already labeled `merge-ready` — those are in the human's court.
|
||||
- **Zero dependencies.** Python stdlib only. Runs anywhere.
|
||||
|
||||
Response time: ~15-30 seconds from notification to agent starting work.
|
||||
- **The poller never marks notifications as read.** That's the agent's job after
|
||||
processing. Prevents the poller and agent from racing.
|
||||
- **Tracks notification IDs, not counts.** Only fires on genuinely new
|
||||
notifications, not re-reads of existing ones.
|
||||
- **The wake message tells the agent to route output to Gitea/Mattermost, not
|
||||
DM.** Prevents chatty notification processing from disturbing the human.
|
||||
- **Zero dependencies.** Python stdlib only (`urllib`, `json`, `time`). Runs
|
||||
anywhere.
|
||||
|
||||
Full source code is available in
|
||||
[OPENCLAW_TRICKS.md](OPENCLAW_TRICKS.md#gitea-integration--notification-polling).
|
||||
[OPENCLAW_TRICKS.md](OPENCLAW_TRICKS.md#the-gitea-notification-poller).
|
||||
|
||||
## CI: Gitea Actions
|
||||
|
||||
@@ -395,34 +371,42 @@ Everything gets a production URL with automatic TLS via Traefik.
|
||||
Putting it all together, the development lifecycle looks like this:
|
||||
|
||||
```
|
||||
1. Human labels issue with `bot` (or agent files issue)
|
||||
1. Issue filed in Gitea (by human or agent)
|
||||
↓
|
||||
2. Poller detects `bot` label + unassigned → assigns agent → spawns worker
|
||||
2. Agent picks up the issue (via notification poller)
|
||||
↓
|
||||
3. Worker agent clones repo, writes code, runs `docker build .`
|
||||
3. Agent posts "starting work on #N" to Mattermost #git
|
||||
↓
|
||||
4. Worker creates PR "(closes #N)", labels `needs-review`
|
||||
4. Agent (or sub-agent) creates branch, writes code, pushes
|
||||
↓
|
||||
5. Worker spawns reviewer agent → stops
|
||||
5. Gitea webhook fires → #git shows the push
|
||||
↓
|
||||
6. Reviewer agent reads diff + referenced issues → reviews
|
||||
6. CI runs docker build → passes or fails
|
||||
↓
|
||||
7a. Review PASS → reviewer rebases if needed → `docker build .`
|
||||
→ labels `merge-ready` → assigns human → removes `bot`
|
||||
7. Agent creates PR "(closes #N)"
|
||||
↓
|
||||
7b. Review FAIL → reviewer labels `needs-rework`
|
||||
→ spawns worker agent → back to step 3
|
||||
8. Gitea webhook fires → #git shows the PR
|
||||
↓
|
||||
8. Human reviews, merges
|
||||
9. Agent reviews code, runs make check locally, verifies
|
||||
↓
|
||||
9. Gitea webhook fires → µPaaS deploys to production
|
||||
10. Agent assigns PR to human when all checks pass
|
||||
↓
|
||||
10. Site/service is live
|
||||
11. Human reviews, requests changes or approves
|
||||
↓
|
||||
12. If changes requested → agent reworks, back to step 6
|
||||
↓
|
||||
13. Human merges PR
|
||||
↓
|
||||
14. Gitea webhook fires → µPaaS deploys to production
|
||||
↓
|
||||
15. Gitea webhook fires → #git shows the merge
|
||||
↓
|
||||
16. Site/service is live on production URL
|
||||
```
|
||||
|
||||
Steps 2-7 happen without any human involvement, driven by agent-to-agent
|
||||
chaining. The human's role is reduced to: label the issue, review the final PR,
|
||||
merge. Everything else is automated.
|
||||
Steps 2-10 can happen without any human involvement. The human's role is reduced
|
||||
to: review the PR, approve or request changes, merge. Everything else is
|
||||
automated.
|
||||
|
||||
### Observability
|
||||
|
||||
|
||||
@@ -173,102 +173,46 @@ The landing checklist (triggered automatically after every flight) updates
|
||||
location, timezone, nearest airport, and lodging in the daily context file. It
|
||||
also checks if any cron jobs have hardcoded timezones that need updating.
|
||||
|
||||
### Gitea Notification Delivery
|
||||
### The Gitea Notification Poller
|
||||
|
||||
There are two approaches for getting Gitea notifications to your agent,
|
||||
depending on your network setup.
|
||||
|
||||
#### Option A: Direct Webhooks (VPS / Public Server)
|
||||
|
||||
If your OpenClaw instance runs on a VPS or other publicly reachable server, the
|
||||
simplest approach is direct webhooks. Run Traefik (or any reverse proxy with
|
||||
automatic TLS) on the same server and configure Gitea webhooks to POST directly
|
||||
to OpenClaw's webhook endpoint. This is push-based and realtime — notifications
|
||||
arrive instantly.
|
||||
|
||||
Setup: add a webhook on each Gitea repo (or use an organization-level webhook)
|
||||
pointing to `https://your-openclaw-host/hooks/gitea`. OpenClaw handles the rest.
|
||||
|
||||
#### Option B: Notification Poller + Dispatcher (Local Machine Behind NAT)
|
||||
|
||||
If your OpenClaw runs on a dedicated local machine behind NAT (like a home Mac
|
||||
or Linux workstation), Gitea can't reach it directly. This is our setup —
|
||||
OpenClaw runs on a Mac Studio on a home LAN.
|
||||
|
||||
The solution: a Python script that both polls and dispatches. It polls the Gitea
|
||||
notifications API every 15 seconds, triages each notification (checking
|
||||
@-mentions and assignments), marks them as read, and spawns one isolated agent
|
||||
session per actionable item via `openclaw cron add --session isolated`.
|
||||
|
||||
The poller also runs a secondary **label scan** every 2 minutes, checking all
|
||||
watched repos for open issues/PRs with the `bot` label that are unassigned. This
|
||||
catches cases where the agent chain broke — an agent timed out or crashed
|
||||
without spawning the next agent. It also picks up newly-labeled issues that
|
||||
didn't trigger a notification.
|
||||
OpenClaw has heartbeats, but those are periodic (every ~30min). For Gitea issues
|
||||
and PRs, we wanted near-realtime response. The solution: a tiny Python script
|
||||
that polls the Gitea notifications API every 2 seconds and wakes the agent via
|
||||
OpenClaw's `/hooks/wake` endpoint when new notifications arrive.
|
||||
|
||||
Key design decisions:
|
||||
|
||||
- **The poller IS the dispatcher.** No flag files, no heartbeat dependency. The
|
||||
poller triages notifications and spawns agents directly.
|
||||
- **Marks notifications as read immediately.** Prevents re-dispatch on the next
|
||||
poll cycle.
|
||||
- **Assigns the bot user at dispatch time.** Before spawning the agent, the
|
||||
poller assigns the bot account to the issue via API. This prevents race
|
||||
conditions — subsequent scans skip assigned issues. The spawned agent doesn't
|
||||
need to claim ownership; it's already claimed.
|
||||
- **Persistent dispatch tracking.** Dispatched issues are tracked in a JSON
|
||||
file on disk (not just in memory), surviving poller restarts. Entries
|
||||
auto-prune after 1 hour.
|
||||
- **30-minute re-dispatch cooldown.** Safety net for broken agent chains. Normal
|
||||
operation uses agent-to-agent chaining (each agent spawns the next), so the
|
||||
poller only re-dispatches if the chain breaks.
|
||||
- **Concurrency cap.** The poller checks how many agents are currently running
|
||||
(`openclaw cron list`) and defers dispatch if the cap is reached.
|
||||
- **Stale agent reaper.** Each scan cycle, kills agent sessions running longer
|
||||
than 10 minutes. The `--timeout-seconds` flag isn't always enforced by
|
||||
OpenClaw, so the poller handles cleanup itself.
|
||||
- **`merge-ready` skip.** The label scan skips issues already labeled
|
||||
`merge-ready` — those are in the human's court.
|
||||
- **Template-based prompts.** The poller reads two workspace files (a dispatch
|
||||
header with `{{variable}}` placeholders, and a workflow rules document),
|
||||
concatenates them, substitutes variables, and passes the result as the
|
||||
agent's `--message`. This keeps all instructions in version-controlled
|
||||
workspace files with a single source of truth.
|
||||
- **Zero dependencies.** Python stdlib only. Runs anywhere.
|
||||
- **The poller never marks notifications as read.** That's the agent's job after
|
||||
it processes them. This prevents the poller and agent from racing.
|
||||
- **It tracks notification IDs, not counts.** This way it only fires on
|
||||
genuinely new notifications, not re-reads of existing ones.
|
||||
- **The wake message tells the agent to route output to Gitea/Mattermost, not to
|
||||
DM.** This prevents chatty notification processing from disturbing the human.
|
||||
- **Zero dependencies.** Just Python stdlib (`urllib`, `json`, `time`). Runs
|
||||
anywhere.
|
||||
|
||||
Response time: ~15–30s from notification to agent starting work.
|
||||
Here's the full source:
|
||||
|
||||
```python
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Gitea notification poller + dispatcher.
|
||||
|
||||
Two polling loops:
|
||||
1. Notification-based: detects new @-mentions and assignments, dispatches
|
||||
agents for actionable notifications.
|
||||
2. Label-based: periodically scans for issues/PRs with the 'bot' label
|
||||
that are unassigned (available for work). Catches broken agent chains
|
||||
and newly-labeled issues.
|
||||
|
||||
The poller assigns the bot user to the issue BEFORE spawning the agent,
|
||||
preventing race conditions where multiple scans dispatch for the same issue.
|
||||
Gitea notification poller.
|
||||
Polls for unread notifications and wakes OpenClaw when the count
|
||||
changes. The AGENT marks notifications as read after processing —
|
||||
the poller never marks anything as read.
|
||||
|
||||
Required env vars:
|
||||
GITEA_URL - Gitea instance URL
|
||||
GITEA_TOKEN - Gitea API token
|
||||
HOOK_TOKEN - OpenClaw hooks auth token
|
||||
|
||||
Optional env vars:
|
||||
POLL_DELAY - Seconds between notification polls (default: 15)
|
||||
COOLDOWN - Seconds between dispatch batches (default: 30)
|
||||
BOT_SCAN_INTERVAL - Seconds between label scans (default: 120)
|
||||
MAX_CONCURRENT_AGENTS - Max simultaneous agents (default: 10)
|
||||
REAP_AGE_SECONDS - Kill agents older than this (default: 600)
|
||||
OPENCLAW_BIN - Path to openclaw binary
|
||||
GATEWAY_URL - OpenClaw gateway URL (default: http://127.0.0.1:18789)
|
||||
POLL_DELAY - Delay between polls in seconds (default: 2)
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
import urllib.request
|
||||
@@ -276,270 +220,109 @@ import urllib.error
|
||||
|
||||
GITEA_URL = os.environ.get("GITEA_URL", "").rstrip("/")
|
||||
GITEA_TOKEN = os.environ.get("GITEA_TOKEN", "")
|
||||
POLL_DELAY = int(os.environ.get("POLL_DELAY", "15"))
|
||||
COOLDOWN = int(os.environ.get("COOLDOWN", "30"))
|
||||
BOT_SCAN_INTERVAL = int(os.environ.get("BOT_SCAN_INTERVAL", "120"))
|
||||
MAX_CONCURRENT_AGENTS = int(os.environ.get("MAX_CONCURRENT_AGENTS", "10"))
|
||||
REAP_AGE_SECONDS = int(os.environ.get("REAP_AGE_SECONDS", "600"))
|
||||
REDISPATCH_COOLDOWN = 1800 # 30 min safety net for broken agent chains
|
||||
OPENCLAW_BIN = os.environ.get("OPENCLAW_BIN", "openclaw")
|
||||
BOT_USER = os.environ.get("BOT_USER", "clawbot")
|
||||
|
||||
WORKSPACE = os.path.expanduser("~/.openclaw/workspace")
|
||||
DISPATCH_HEADER = os.path.join(
|
||||
WORKSPACE, "taskprompts", "how-to-handle-gitea-notifications.md"
|
||||
GATEWAY_URL = os.environ.get("GATEWAY_URL", "http://127.0.0.1:18789").rstrip(
|
||||
"/"
|
||||
)
|
||||
WORKFLOW_DOC = os.path.join(
|
||||
WORKSPACE, "taskprompts", "how-to-work-on-a-gitea-issue-or-pr.md"
|
||||
)
|
||||
DISPATCH_STATE_PATH = os.path.join(
|
||||
os.path.dirname(os.path.abspath(__file__)), ".dispatch-state.json"
|
||||
)
|
||||
|
||||
# Repos to watch for bot-labeled issues
|
||||
WATCHED_REPOS = [
|
||||
# "org/repo1",
|
||||
# "org/repo2",
|
||||
]
|
||||
|
||||
# Dispatch tracking (persisted to disk)
|
||||
dispatched_issues: dict[str, float] = {}
|
||||
HOOK_TOKEN = os.environ.get("HOOK_TOKEN", "")
|
||||
POLL_DELAY = int(os.environ.get("POLL_DELAY", "2"))
|
||||
|
||||
|
||||
def _load_dispatch_state() -> dict[str, float]:
|
||||
try:
|
||||
with open(DISPATCH_STATE_PATH) as f:
|
||||
state = json.load(f)
|
||||
now = time.time()
|
||||
return {k: v for k, v in state.items() if now - v < 3600}
|
||||
except (FileNotFoundError, json.JSONDecodeError):
|
||||
return {}
|
||||
|
||||
|
||||
def _save_dispatch_state():
|
||||
try:
|
||||
with open(DISPATCH_STATE_PATH, "w") as f:
|
||||
json.dump(dispatched_issues, f)
|
||||
except OSError as e:
|
||||
print(f"WARN: Could not save dispatch state: {e}", file=sys.stderr)
|
||||
|
||||
|
||||
def gitea_api(method, path, data=None):
|
||||
url = f"{GITEA_URL}/api/v1{path}"
|
||||
body = json.dumps(data).encode() if data else None
|
||||
headers = {"Authorization": f"token {GITEA_TOKEN}"}
|
||||
if body:
|
||||
headers["Content-Type"] = "application/json"
|
||||
req = urllib.request.Request(url, headers=headers, method=method, data=body)
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=15) as resp:
|
||||
raw = resp.read()
|
||||
return json.loads(raw) if raw else None
|
||||
except Exception as e:
|
||||
print(f"WARN: {method} {path}: {e}", file=sys.stderr, flush=True)
|
||||
return None
|
||||
|
||||
|
||||
def load_template() -> str:
|
||||
"""Load dispatch header + workflow doc, concatenated."""
|
||||
parts = []
|
||||
for path in [DISPATCH_HEADER, WORKFLOW_DOC]:
|
||||
try:
|
||||
with open(path) as f:
|
||||
parts.append(f.read())
|
||||
except FileNotFoundError:
|
||||
print(f"ERROR: File not found: {path}", file=sys.stderr)
|
||||
def check_config():
|
||||
missing = []
|
||||
if not GITEA_URL:
|
||||
missing.append("GITEA_URL")
|
||||
if not GITEA_TOKEN:
|
||||
missing.append("GITEA_TOKEN")
|
||||
if not HOOK_TOKEN:
|
||||
missing.append("HOOK_TOKEN")
|
||||
if missing:
|
||||
print(
|
||||
f"ERROR: Missing required env vars: {', '.join(missing)}",
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(1)
|
||||
return "\n\n---\n\n".join(parts)
|
||||
|
||||
|
||||
def render_template(template, repo_full, issue_number, title,
|
||||
subject_type, reason):
|
||||
return (
|
||||
template
|
||||
.replace("{{repo_full}}", repo_full)
|
||||
.replace("{{issue_number}}", str(issue_number))
|
||||
.replace("{{title}}", title)
|
||||
.replace("{{subject_type}}", subject_type)
|
||||
.replace("{{reason}}", reason)
|
||||
.replace("{{gitea_url}}", GITEA_URL)
|
||||
.replace("{{gitea_token}}", GITEA_TOKEN)
|
||||
.replace("{{openclaw_bin}}", OPENCLAW_BIN)
|
||||
.replace("{{bot_user}}", BOT_USER)
|
||||
# Add your own variables here (e.g. git_channel)
|
||||
def gitea_unread_ids():
|
||||
"""Return set of unread notification IDs."""
|
||||
req = urllib.request.Request(
|
||||
f"{GITEA_URL}/api/v1/notifications?status-types=unread",
|
||||
headers={"Authorization": f"token {GITEA_TOKEN}"},
|
||||
)
|
||||
|
||||
|
||||
def count_running_agents() -> int:
|
||||
try:
|
||||
result = subprocess.run(
|
||||
[OPENCLAW_BIN, "cron", "list"],
|
||||
capture_output=True, text=True, timeout=10,
|
||||
)
|
||||
return sum(1 for line in result.stdout.splitlines()
|
||||
if "running" in line or "idle" in line)
|
||||
except Exception:
|
||||
return 0
|
||||
|
||||
|
||||
def spawn_agent(template, repo_full, issue_number, title,
|
||||
subject_type, reason):
|
||||
dispatch_key = f"{repo_full}#{issue_number}"
|
||||
last = dispatched_issues.get(dispatch_key)
|
||||
if last and (time.time() - last) < REDISPATCH_COOLDOWN:
|
||||
return
|
||||
|
||||
if count_running_agents() >= MAX_CONCURRENT_AGENTS:
|
||||
print(f" → Concurrency limit reached, deferring {dispatch_key}",
|
||||
flush=True)
|
||||
return
|
||||
|
||||
dispatched_issues[dispatch_key] = time.time()
|
||||
|
||||
# Assign bot user immediately to prevent races
|
||||
gitea_api("PATCH", f"/repos/{repo_full}/issues/{issue_number}",
|
||||
{"assignees": [BOT_USER]})
|
||||
|
||||
repo_short = repo_full.split("/")[-1]
|
||||
job_name = f"gitea-{repo_short}-{issue_number}-{int(time.time())}"
|
||||
msg = render_template(template, repo_full, issue_number, title,
|
||||
subject_type, reason)
|
||||
|
||||
try:
|
||||
result = subprocess.run(
|
||||
[OPENCLAW_BIN, "cron", "add",
|
||||
"--name", job_name, "--at", "1s",
|
||||
"--message", msg, "--delete-after-run",
|
||||
"--session", "isolated", "--no-deliver",
|
||||
"--thinking", "low", "--timeout-seconds", "300"],
|
||||
capture_output=True, text=True, timeout=15,
|
||||
)
|
||||
if result.returncode == 0:
|
||||
_save_dispatch_state()
|
||||
else:
|
||||
dispatched_issues.pop(dispatch_key, None)
|
||||
with urllib.request.urlopen(req, timeout=10) as resp:
|
||||
notifs = json.loads(resp.read())
|
||||
return {n["id"] for n in notifs}
|
||||
except Exception as e:
|
||||
print(f"Spawn error: {e}", file=sys.stderr, flush=True)
|
||||
dispatched_issues.pop(dispatch_key, None)
|
||||
|
||||
|
||||
def is_actionable(notif):
|
||||
"""Check if a notification warrants spawning an agent."""
|
||||
subject = notif.get("subject", {})
|
||||
repo = notif.get("repository", {})
|
||||
repo_full = repo.get("full_name", "")
|
||||
url = subject.get("url", "")
|
||||
number = url.rstrip("/").split("/")[-1] if url else ""
|
||||
if not number or not number.isdigit():
|
||||
return False, "no issue number", None
|
||||
|
||||
issue = gitea_api("GET", f"/repos/{repo_full}/issues/{number}")
|
||||
if not issue:
|
||||
return False, "couldn't fetch issue", number
|
||||
|
||||
# Check for @-mentions in the latest comment
|
||||
comments = gitea_api(
|
||||
"GET", f"/repos/{repo_full}/issues/{number}/comments"
|
||||
print(
|
||||
f"WARN: Gitea API failed: {e}", file=sys.stderr, flush=True
|
||||
)
|
||||
if comments:
|
||||
last = comments[-1]
|
||||
if last.get("user", {}).get("login") == BOT_USER:
|
||||
return False, "own comment is latest", number
|
||||
if f"@{BOT_USER}" in (last.get("body") or ""):
|
||||
return True, "@-mentioned in comment", number
|
||||
|
||||
# Check for @-mention in issue body
|
||||
body = issue.get("body", "") or ""
|
||||
if f"@{BOT_USER}" in body:
|
||||
return True, "@-mentioned in body", number
|
||||
|
||||
return False, "not mentioned", number
|
||||
return set()
|
||||
|
||||
|
||||
def scan_bot_labeled(template):
|
||||
"""Scan for issues/PRs with 'bot' label that are unassigned."""
|
||||
for repo_full in WATCHED_REPOS:
|
||||
for issue_type in ["issues", "pulls"]:
|
||||
items = gitea_api(
|
||||
"GET",
|
||||
f"/repos/{repo_full}/issues?state=open&type={issue_type}"
|
||||
f"&labels=bot&sort=updated&limit=10",
|
||||
) or []
|
||||
for item in items:
|
||||
number = str(item["number"])
|
||||
dispatch_key = f"{repo_full}#{number}"
|
||||
|
||||
last = dispatched_issues.get(dispatch_key)
|
||||
if last and (time.time() - last) < REDISPATCH_COOLDOWN:
|
||||
continue
|
||||
|
||||
assignees = [
|
||||
a.get("login", "") for a in item.get("assignees") or []
|
||||
]
|
||||
if BOT_USER in assignees:
|
||||
continue
|
||||
|
||||
labels = [
|
||||
l.get("name", "") for l in item.get("labels") or []
|
||||
]
|
||||
if "merge-ready" in labels:
|
||||
continue
|
||||
|
||||
kind = "PR" if issue_type == "pulls" else "issue"
|
||||
spawn_agent(
|
||||
template, repo_full, number,
|
||||
item.get("title", "")[:60],
|
||||
"pull" if issue_type == "pulls" else "issue",
|
||||
"bot label, unassigned",
|
||||
def wake_openclaw(count):
|
||||
text = (
|
||||
f"[Gitea Notification] {count} new notification(s). "
|
||||
"Check your Gitea notification inbox via API, process them, "
|
||||
"and mark as read when done. "
|
||||
"Route all output to Gitea comments or Mattermost #git/#claw. "
|
||||
"Do NOT reply to this session — respond with NO_REPLY."
|
||||
)
|
||||
payload = json.dumps({"text": text, "mode": "now"}).encode()
|
||||
req = urllib.request.Request(
|
||||
f"{GATEWAY_URL}/hooks/wake",
|
||||
data=payload,
|
||||
headers={
|
||||
"Authorization": f"Bearer {HOOK_TOKEN}",
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
method="POST",
|
||||
)
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=5) as resp:
|
||||
status = resp.status
|
||||
print(f" Wake responded: {status}", flush=True)
|
||||
return True
|
||||
except Exception as e:
|
||||
print(
|
||||
f"WARN: Failed to wake OpenClaw: {e}",
|
||||
file=sys.stderr,
|
||||
flush=True,
|
||||
)
|
||||
return False
|
||||
|
||||
|
||||
def main():
|
||||
global dispatched_issues
|
||||
dispatched_issues = _load_dispatch_state()
|
||||
|
||||
if not GITEA_URL or not GITEA_TOKEN:
|
||||
print("ERROR: GITEA_URL and GITEA_TOKEN required", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
template = load_template()
|
||||
print(f"Poller started (poll={POLL_DELAY}s, cooldown={COOLDOWN}s, "
|
||||
f"bot_scan={BOT_SCAN_INTERVAL}s, repos={len(WATCHED_REPOS)})",
|
||||
flush=True)
|
||||
|
||||
seen_ids = set(
|
||||
n["id"] for n in
|
||||
(gitea_api("GET", "/notifications?status-types=unread") or [])
|
||||
check_config()
|
||||
print(
|
||||
f"Gitea notification poller started (delay={POLL_DELAY}s)",
|
||||
flush=True,
|
||||
)
|
||||
|
||||
last_seen_ids = gitea_unread_ids()
|
||||
print(
|
||||
f"Initial unread: {len(last_seen_ids)} notification(s)", flush=True
|
||||
)
|
||||
last_dispatch = 0
|
||||
last_bot_scan = 0
|
||||
|
||||
while True:
|
||||
time.sleep(POLL_DELAY)
|
||||
now = time.time()
|
||||
|
||||
# --- Notification polling ---
|
||||
notifs = gitea_api("GET", "/notifications?status-types=unread") or []
|
||||
current_ids = {n["id"] for n in notifs}
|
||||
new_ids = current_ids - seen_ids
|
||||
if new_ids and now - last_dispatch >= COOLDOWN:
|
||||
for n in [n for n in notifs if n["id"] in new_ids]:
|
||||
nid = n.get("id")
|
||||
if nid:
|
||||
gitea_api("PATCH", f"/notifications/threads/{nid}")
|
||||
is_act, reason, num = is_actionable(n)
|
||||
if is_act:
|
||||
repo = n["repository"]["full_name"]
|
||||
title = n["subject"]["title"][:60]
|
||||
stype = n["subject"].get("type", "").lower()
|
||||
spawn_agent(template, repo, num, title, stype, reason)
|
||||
last_dispatch = now
|
||||
seen_ids = current_ids
|
||||
current_ids = gitea_unread_ids()
|
||||
new_ids = current_ids - last_seen_ids
|
||||
|
||||
# --- Bot label scan (less frequent) ---
|
||||
if now - last_bot_scan >= BOT_SCAN_INTERVAL:
|
||||
scan_bot_labeled(template)
|
||||
last_bot_scan = now
|
||||
if not new_ids:
|
||||
last_seen_ids = current_ids
|
||||
continue
|
||||
|
||||
ts = time.strftime("%H:%M:%S")
|
||||
print(
|
||||
f"[{ts}] {len(new_ids)} new notification(s) "
|
||||
f"({len(current_ids)} total unread), waking agent",
|
||||
flush=True,
|
||||
)
|
||||
|
||||
wake_openclaw(len(new_ids))
|
||||
last_seen_ids = current_ids
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
@@ -585,15 +368,13 @@ This applies to everything: project rules ("no mocks in tests"), workflow
|
||||
preferences ("fewer PRs, don't over-split"), corrections, new policies.
|
||||
Immediate write to the daily file, and to MEMORY.md if it's a standing rule.
|
||||
|
||||
### Sensitive Output Routing
|
||||
### PII-Aware Output Routing
|
||||
|
||||
A lesson learned the hard way: **the audience determines what you can say, not
|
||||
who asked.** If the human asks for a medication status report in a group
|
||||
channel, the agent can't just dump it there — other people can read it. The
|
||||
rule: if the output would contain sensitive information (PII, secrets,
|
||||
credentials, API keys, operational details like flight numbers, locations,
|
||||
travel plans, medical info, etc.) and the channel isn't private, redirect to DM
|
||||
and reply in-channel with "sent privately."
|
||||
rule: if the output would contain PII and the channel isn't private, redirect to
|
||||
DM and reply in-channel with "sent privately."
|
||||
|
||||
This is enforced at multiple levels:
|
||||
|
||||
@@ -624,7 +405,7 @@ The heartbeat handles:
|
||||
- Periodic memory maintenance
|
||||
|
||||
State tracking in `memory/heartbeat-state.json` prevents redundant checks (e.g.,
|
||||
don't re-check notifications if you checked 10 minutes ago).
|
||||
don't re-check email if you checked 10 minutes ago).
|
||||
|
||||
The key output rule: heartbeats should either be `HEARTBEAT_OK` (nothing to do)
|
||||
or a direct alert. Work narration goes to a designated status channel, never to
|
||||
@@ -884,27 +665,25 @@ From REPO_POLICIES.md and our operational experience:
|
||||
|
||||
#### The PR Pipeline
|
||||
|
||||
Our agent follows a strict PR lifecycle using agent-to-agent chaining. Each step
|
||||
is handled by a separate, isolated agent session — the agent that writes code
|
||||
never reviews it:
|
||||
Our agent follows a strict PR lifecycle:
|
||||
|
||||
```markdown
|
||||
## PR pipeline (every PR, no exceptions)
|
||||
|
||||
Worker agent → docker build . → push → label needs-review → spawn reviewer
|
||||
Reviewer agent → review diff → PASS: docker build . → label merge-ready
|
||||
→ FAIL: label needs-rework → spawn worker
|
||||
Repeat until reviewer approves.
|
||||
1. **Review/rework loop**: code review → rework → re-review → repeat until clean
|
||||
2. **Check/rework loop**: `make check` + `docker build .` → rework → re-check →
|
||||
repeat until clean
|
||||
3. Only after BOTH loops pass with zero issues: assign to human
|
||||
|
||||
- docker build . is the ONLY authoritative check (runs make check inside)
|
||||
- "Passes checks" ≠ "ready for human"
|
||||
- Never weaken tests/linters. Fix the code.
|
||||
- Pre-existing failures are YOUR problem. Fix them as part of your PR.
|
||||
```
|
||||
|
||||
The agent chain doesn't just create a PR and hand it off — it drives the PR
|
||||
through review, rework, and verification until it's genuinely ready. A PR
|
||||
assigned to the human means: build passes, code reviewed by a separate agent,
|
||||
review feedback addressed, rebased. Anything less is still in the agent chain.
|
||||
The agent doesn't just create a PR and hand it off — it drives the PR through
|
||||
review, rework, and verification until it's genuinely ready. A PR assigned to
|
||||
the human means: all checks pass, code reviewed, review feedback addressed,
|
||||
rebased against main, no conflicts. Anything less is the agent's open task.
|
||||
|
||||
#### New Repo Bootstrap
|
||||
|
||||
@@ -1638,8 +1417,7 @@ stay quiet.
|
||||
|
||||
## Inbox Check (PRIORITY)
|
||||
|
||||
(check whatever notification sources apply to your setup — e.g. Gitea
|
||||
notifications, emails, issue trackers)
|
||||
(check notifications, issues, emails — whatever applies)
|
||||
|
||||
## Flight Prep Blocks (daily)
|
||||
|
||||
@@ -1673,9 +1451,10 @@ Never send internal thinking or status narration to user's DM. Output should be:
|
||||
```json
|
||||
{
|
||||
"lastChecks": {
|
||||
"gitea": 1703280000,
|
||||
"email": 1703275200,
|
||||
"calendar": 1703260800,
|
||||
"weather": null
|
||||
"weather": null,
|
||||
"gitea": 1703280000
|
||||
},
|
||||
"lastWeeklyDocsReview": "2026-02-24"
|
||||
}
|
||||
@@ -1703,9 +1482,51 @@ Never send internal thinking or status narration to user's DM. Output should be:
|
||||
|
||||
## Gitea Integration & Notification Polling
|
||||
|
||||
For self-hosted Gitea instances, you can set up a notification poller that
|
||||
injects Gitea events (issue assignments, PR reviews, @-mentions) into the
|
||||
agent's session.
|
||||
For self-hosted Gitea instances, you need a way to deliver notifications (issue
|
||||
assignments, PR reviews, @-mentions) to your agent. There are two approaches,
|
||||
depending on your network setup.
|
||||
|
||||
### Notification Delivery: Webhooks vs Polling
|
||||
|
||||
#### 1. Direct webhooks (VPS / public server)
|
||||
|
||||
If your OpenClaw instance runs on a VPS or other publicly reachable server, you
|
||||
can run Traefik (or any reverse proxy) on the same server and configure Gitea
|
||||
webhooks to POST directly to OpenClaw's webhook endpoint. This is push-based and
|
||||
realtime — notifications arrive instantly.
|
||||
|
||||
Set up a Gitea webhook (per-repo or org-wide) pointing at your OpenClaw
|
||||
instance's `/hooks/wake` endpoint. Gitea sends a POST on every event, and the
|
||||
agent wakes immediately to process it.
|
||||
|
||||
#### 2. Notification poller (local machine behind NAT)
|
||||
|
||||
If your OpenClaw instance runs on a dedicated local machine behind NAT (like a
|
||||
home Mac or Linux box), Gitea can't reach it directly. In this case, use a
|
||||
lightweight polling script that checks the Gitea notifications API every few
|
||||
seconds and signals the agent when new notifications arrive.
|
||||
|
||||
This is the approach we use — OpenClaw runs on a dedicated Mac Studio on a home
|
||||
LAN, so we poll Gitea's notification API and wake the agent via the local
|
||||
`/hooks/wake` endpoint when new notifications appear. The poller script is
|
||||
included below in the [Notification poller](#notification-poller) section.
|
||||
|
||||
The poller approach trades ~30 seconds of latency (polling interval) for
|
||||
simplicity and no NAT/firewall configuration. For most workflows this is
|
||||
perfectly fine — code review and issue triage don't need sub-second response
|
||||
times. If no new notifications arrive between heartbeats, the effective latency
|
||||
is bounded by the heartbeat interval (~30 minutes), but in practice the poller
|
||||
catches most events within seconds.
|
||||
|
||||
#### Which should you choose?
|
||||
|
||||
| Factor | Webhooks | Poller |
|
||||
| ------------------- | ------------------- | ------------------------- |
|
||||
| Network requirement | Public IP / domain | None (outbound-only) |
|
||||
| Latency | Instant | ~2-30s (polling interval) |
|
||||
| Setup complexity | Reverse proxy + TLS | Single background script |
|
||||
| Dependencies | Traefik/nginx/Caddy | Python stdlib only |
|
||||
| Best for | VPS / cloud deploys | Home LAN / NAT setups |
|
||||
|
||||
### Workflow rules (HEARTBEAT.md / AGENTS.md):
|
||||
|
||||
@@ -1756,12 +1577,12 @@ For complex coding tasks, spawn isolated sub-agents.
|
||||
|
||||
### Sub-Agent PR Quality Gate (MANDATORY)
|
||||
|
||||
- `docker build .` must pass. This is identical to CI and the only
|
||||
authoritative check. No exceptions.
|
||||
- `make check` must pass with ZERO failures. No exceptions.
|
||||
- Pre-existing failures are YOUR problem. Fix them as part of your PR.
|
||||
- NEVER modify linter config to make checks pass. Fix the code.
|
||||
- Every PR must include full `make check` output
|
||||
- Rebase before and after committing
|
||||
- Never self-review — each agent spawns a separate agent for review
|
||||
- Never self-review
|
||||
```
|
||||
|
||||
---
|
||||
@@ -1844,24 +1665,21 @@ Never lose a rule or preference your human states:
|
||||
|
||||
---
|
||||
|
||||
## Sensitive Output Routing — Audience-Aware Responses
|
||||
## PII Output Routing — Audience-Aware Responses
|
||||
|
||||
A critical security pattern: **the audience determines what you can say, not who
|
||||
asked.** If your human asks for a sitrep (or any sensitive info) in a group
|
||||
asked.** If your human asks for a sitrep (or any PII-containing info) in a group
|
||||
channel, you can't just dump it there — other people can read it.
|
||||
|
||||
### AGENTS.md / checklist prompt:
|
||||
|
||||
```markdown
|
||||
## Sensitive Output Routing (CRITICAL)
|
||||
## PII Output Routing (CRITICAL)
|
||||
|
||||
- NEVER output sensitive information in any non-private channel, even if your
|
||||
human asks for it
|
||||
- This includes: PII, secrets, credentials, API keys, and sensitive operational
|
||||
information (flight numbers/times/dates, locations, travel plans, medical
|
||||
info, financial details, etc.)
|
||||
- If a request would produce any of the above in a shared channel: send the
|
||||
response via DM instead, and reply in-channel with "sent privately"
|
||||
- NEVER output PII in any non-private channel, even if your human asks for it
|
||||
- If a request would produce PII (medication status, travel details, financial
|
||||
info, etc.) in a shared channel: send the response via DM instead, and reply
|
||||
in-channel with "sent privately"
|
||||
- The rule is: the audience determines what you can say, not who asked
|
||||
- This applies to: group chats, public issue trackers, shared Mattermost
|
||||
channels, Discord servers — anywhere that isn't a 1:1 DM
|
||||
@@ -1870,10 +1688,10 @@ channel, you can't just dump it there — other people can read it.
|
||||
### Why this matters:
|
||||
|
||||
This is a real failure mode. If someone asks "sitrep" in a group channel and you
|
||||
respond with medication names, partner details, travel dates, hotel names, or
|
||||
API credentials — you just leaked all of that to everyone in the channel. The
|
||||
human asking is authorized to see it; the channel audience is not. Always check
|
||||
WHERE you're responding, not just WHO asked.
|
||||
respond with medication names, partner details, travel dates, and hotel names —
|
||||
you just leaked all of that to everyone in the channel. The human asking is
|
||||
authorized to see it; the channel audience is not. Always check WHERE you're
|
||||
responding, not just WHO asked.
|
||||
|
||||
---
|
||||
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user