docs: update poller to dispatcher architecture (closes #4) #5

Merged
clawbot merged 1 commits from fix/update-poller-docs into main 2026-02-28 16:31:43 +01:00

View File

@ -189,50 +189,68 @@ arrive instantly.
Setup: add a webhook on each Gitea repo (or use an organization-level webhook)
pointing to `https://your-openclaw-host/hooks/gitea`. OpenClaw handles the rest.
#### Option B: Notification Poller (Local Machine Behind NAT)
#### Option B: Notification Poller + Dispatcher (Local Machine Behind NAT)
If your OpenClaw runs on a dedicated local machine behind NAT (like a home Mac
or Linux workstation), Gitea can't reach it directly. This is our setup —
OpenClaw runs on a Mac Studio on a home LAN.
The solution: a lightweight Python script that polls the Gitea notifications API
every few seconds. When new notifications appear, it writes a flag file that the
agent checks during heartbeats.
The solution: a Python script that both polls and dispatches. It polls the Gitea
notifications API every 15 seconds, triages each notification (checking
assignment and @-mentions), marks them as read, and spawns one isolated agent
session per actionable item via `openclaw cron add --session isolated`.
The poller also runs a secondary **assignment scan** every 2 minutes, checking
all watched repos for open issues/PRs assigned to the bot that were recently
updated and still need a response. This catches cases where notifications aren't
generated (e.g. self-assignment, API-created issues).
Key design decisions:
- **The poller never marks notifications as read.** The agent does that after
processing. This prevents lost notifications if the agent fails to process.
- **It tracks notification IDs, not counts.** Only fires on genuinely new
notifications, not re-reads of existing ones.
- **Flag file instead of wake events.** We initially used OpenClaw's
`/hooks/wake` endpoint, but wake events target the main (DM) session — any
model response during processing leaked to DM as a notification. The flag file
approach is processed during heartbeats, where output routing is controlled.
- **The poller IS the dispatcher.** No flag files, no heartbeat dependency. The
poller triages notifications and spawns agents directly.
- **Marks notifications as read immediately.** Each notification is marked read
as it's processed, preventing re-dispatch on the next poll.
- **One agent per issue.** Each spawned agent gets a `SCOPE` instruction
limiting it to one specific issue/PR. Agents post results as Gitea comments,
not DMs.
- **Dedup tracking.** An in-memory `dispatched_issues` set prevents spawning
multiple agents for the same issue within one poller lifetime.
- **`--no-deliver` instead of `--announce`.** Agents report via Gitea API
directly. The `--announce` flag on isolated sessions had delivery failures.
- **Assignment scan filters by recency.** Only issues updated in the last 5
minutes are considered, preventing re-dispatch for stale assigned issues.
- **Zero dependencies.** Just Python stdlib. Runs anywhere.
Tradeoff: notifications are processed at heartbeat cadence (~30 min) instead of
realtime. For code review and issue triage, this is fine.
Response time: ~1560s from notification to agent comment (vs ~30 min with the
old heartbeat approach).
```python
#!/usr/bin/env python3
"""
Gitea notification poller (flag-file approach).
Polls for unread notifications and writes a flag file when new ones
appear. The agent checks this flag during heartbeats and processes
notifications via the Gitea API directly.
Gitea notification poller + dispatcher.
Two polling loops:
1. Notification-based: detects new notifications (mentions, assignments by
other users) and dispatches agents for actionable ones.
2. Assignment-based: periodically checks for open issues/PRs assigned to
the bot that have no recent bot comment. Catches cases where
notifications aren't generated (e.g. self-assignment, API-created issues).
Required env vars:
GITEA_URL - Gitea instance URL
GITEA_TOKEN - Gitea API token
Optional env vars:
FLAG_PATH - Path to flag file (default: workspace/memory/gitea-notify-flag)
POLL_DELAY - Delay between polls in seconds (default: 5)
POLL_DELAY - Delay between polls in seconds (default: 15)
COOLDOWN - Minimum seconds between dispatches (default: 30)
ASSIGNMENT_INTERVAL - Seconds between assignment scans (default: 120)
OPENCLAW_BIN - Path to openclaw binary
"""
import json
import os
import subprocess
import sys
import time
import urllib.request
@ -240,62 +258,158 @@ import urllib.error
GITEA_URL = os.environ.get("GITEA_URL", "").rstrip("/")
GITEA_TOKEN = os.environ.get("GITEA_TOKEN", "")
POLL_DELAY = int(os.environ.get("POLL_DELAY", "5"))
FLAG_PATH = os.environ.get(
"FLAG_PATH",
os.path.join(
os.path.dirname(os.path.dirname(os.path.abspath(__file__))),
"memory",
"gitea-notify-flag",
),
)
POLL_DELAY = int(os.environ.get("POLL_DELAY", "15"))
COOLDOWN = int(os.environ.get("COOLDOWN", "30"))
ASSIGNMENT_INTERVAL = int(os.environ.get("ASSIGNMENT_INTERVAL", "120"))
OPENCLAW_BIN = os.environ.get("OPENCLAW_BIN", "/opt/homebrew/bin/openclaw")
BOT_USER = "clawbot" # Change to your bot's Gitea username
# Repos to scan for assigned issues
WATCHED_REPOS = [
# "org/repo1",
# "org/repo2",
]
# Track dispatched issues to prevent duplicates
dispatched_issues = set()
def check_config():
if not GITEA_URL or not GITEA_TOKEN:
print("ERROR: GITEA_URL and GITEA_TOKEN required", file=sys.stderr)
sys.exit(1)
def gitea_api(method, path, data=None):
url = f"{GITEA_URL}/api/v1{path}"
body = json.dumps(data).encode() if data else None
headers = {"Authorization": f"token {GITEA_TOKEN}"}
if body:
headers["Content-Type"] = "application/json"
req = urllib.request.Request(url, headers=headers, method=method, data=body)
try:
with urllib.request.urlopen(req, timeout=15) as resp:
raw = resp.read()
return json.loads(raw) if raw else None
except Exception as e:
print(f"WARN: {method} {path}: {e}", file=sys.stderr, flush=True)
return None
def gitea_unread_ids():
req = urllib.request.Request(
f"{GITEA_URL}/api/v1/notifications?status-types=unread",
headers={"Authorization": f"token {GITEA_TOKEN}"},
def needs_bot_response(repo_full, issue_number):
"""True if the bot is NOT the author of the most recent comment."""
comments = gitea_api("GET", f"/repos/{repo_full}/issues/{issue_number}/comments")
if comments and len(comments) > 0:
if comments[-1].get("user", {}).get("login") == BOT_USER:
return False
return True
def is_actionable(notif):
"""Returns (actionable, reason, issue_number)."""
subject = notif.get("subject", {})
repo = notif.get("repository", {})
repo_full = repo.get("full_name", "")
url = subject.get("url", "")
number = url.rstrip("/").split("/")[-1] if url else ""
if not number or not number.isdigit():
return False, "no issue number", None
issue = gitea_api("GET", f"/repos/{repo_full}/issues/{number}")
if not issue:
return False, "couldn't fetch issue", number
assignees = [a.get("login") for a in (issue.get("assignees") or [])]
if BOT_USER in assignees:
if needs_bot_response(repo_full, number):
return True, f"assigned to {BOT_USER}", number
return False, "assigned but already responded", number
issue_body = issue.get("body", "") or ""
if f"@{BOT_USER}" in issue_body and issue.get("user", {}).get("login") != BOT_USER:
if needs_bot_response(repo_full, number):
return True, f"@-mentioned in body", number
comments = gitea_api("GET", f"/repos/{repo_full}/issues/{number}/comments")
if comments:
last = comments[-1]
if last.get("user", {}).get("login") == BOT_USER:
return False, "own comment is latest", number
if f"@{BOT_USER}" in (last.get("body") or ""):
return True, f"@-mentioned in comment", number
return False, "not mentioned or assigned", number
def spawn_agent(repo_full, issue_number, title, subject_type, reason):
dispatch_key = f"{repo_full}#{issue_number}"
if dispatch_key in dispatched_issues:
return
dispatched_issues.add(dispatch_key)
repo_short = repo_full.split("/")[-1]
job_name = f"gitea-{repo_short}-{issue_number}-{int(time.time())}"
msg = (
f"Gitea: {reason} on {subject_type} #{issue_number} "
f"'{title}' in {repo_full}.\n"
f"API: {GITEA_URL}/api/v1 | Token: {GITEA_TOKEN}\n"
f"SCOPE: Only {subject_type} #{issue_number} in {repo_full}.\n"
f"Read all comments, do the work, post results as Gitea comments."
)
try:
with urllib.request.urlopen(req, timeout=10) as resp:
return {n["id"] for n in json.loads(resp.read())}
subprocess.run(
[OPENCLAW_BIN, "cron", "add",
"--name", job_name, "--at", "1s",
"--message", msg, "--delete-after-run",
"--session", "isolated", "--no-deliver",
"--thinking", "low", "--timeout-seconds", "300"],
capture_output=True, text=True, timeout=15,
)
except Exception as e:
print(f"WARN: Gitea API failed: {e}", file=sys.stderr, flush=True)
return set()
def write_flag(count):
os.makedirs(os.path.dirname(FLAG_PATH), exist_ok=True)
with open(FLAG_PATH, "w") as f:
f.write(json.dumps({
"ts": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
"count": count,
}))
print(f"Spawn error: {e}", file=sys.stderr, flush=True)
dispatched_issues.discard(dispatch_key)
def main():
check_config()
print(f"Gitea poller started (delay={POLL_DELAY}s, flag={FLAG_PATH})", flush=True)
last_seen_ids = gitea_unread_ids()
print(f"Initial unread: {len(last_seen_ids)}", flush=True)
print(f"Poller started (poll={POLL_DELAY}s, cooldown={COOLDOWN}s)", flush=True)
seen_ids = set(n["id"] for n in (gitea_api("GET", "/notifications?status-types=unread") or []))
last_dispatch = 0
last_assign_scan = 0
while True:
time.sleep(POLL_DELAY)
current_ids = gitea_unread_ids()
new_ids = current_ids - last_seen_ids
if not new_ids:
last_seen_ids = current_ids
now = time.time()
# Notification polling
notifs = gitea_api("GET", "/notifications?status-types=unread") or []
current_ids = {n["id"] for n in notifs}
new_ids = current_ids - seen_ids
if new_ids and now - last_dispatch >= COOLDOWN:
for n in [n for n in notifs if n["id"] in new_ids]:
nid = n.get("id")
if nid:
gitea_api("PATCH", f"/notifications/threads/{nid}")
is_act, reason, num = is_actionable(n)
if is_act:
repo = n["repository"]["full_name"]
title = n["subject"]["title"][:60]
stype = n["subject"].get("type", "").lower()
spawn_agent(repo, num, title, stype, reason)
last_dispatch = now
seen_ids = current_ids
# Assignment scan (less frequent)
if now - last_assign_scan >= ASSIGNMENT_INTERVAL:
for repo in WATCHED_REPOS:
for itype in ["issues", "pulls"]:
items = gitea_api("GET",
f"/repos/{repo}/issues?state=open&type={itype}"
f"&assignee={BOT_USER}&sort=updated&limit=10") or []
for item in items:
num = str(item["number"])
if f"{repo}#{num}" in dispatched_issues:
continue
ts = time.strftime("%H:%M:%S")
print(f"[{ts}] {len(new_ids)} new ({len(current_ids)} total), flag written", flush=True)
write_flag(len(new_ids))
last_seen_ids = current_ids
# Only recently updated items (5 min)
# ... add is_recently_updated() check here
if needs_bot_response(repo, num):
spawn_agent(repo, num, item["title"][:60],
"pull" if itype == "pulls" else "issue",
f"assigned to {BOT_USER}")
last_assign_scan = now
if __name__ == "__main__":