Merge pull request 'docs: update poller to dispatcher architecture (closes #4)' (#5) from fix/update-poller-docs into main

OPENCLAW_TRICKS.md

@@ -189,50 +189,68 @@ arrive instantly.
 Setup: add a webhook on each Gitea repo (or use an organization-level webhook)
 pointing to `https://your-openclaw-host/hooks/gitea`. OpenClaw handles the rest.
 
-#### Option B: Notification Poller (Local Machine Behind NAT)
+#### Option B: Notification Poller + Dispatcher (Local Machine Behind NAT)
 
 If your OpenClaw runs on a dedicated local machine behind NAT (like a home Mac
 or Linux workstation), Gitea can't reach it directly. This is our setup —
 OpenClaw runs on a Mac Studio on a home LAN.
 
-The solution: a lightweight Python script that polls the Gitea notifications API
-every few seconds. When new notifications appear, it writes a flag file that the
-agent checks during heartbeats.
+The solution: a Python script that both polls and dispatches. It polls the Gitea
+notifications API every 15 seconds, triages each notification (checking
+assignment and @-mentions), marks them as read, and spawns one isolated agent
+session per actionable item via `openclaw cron add --session isolated`.
+
+The poller also runs a secondary **assignment scan** every 2 minutes, checking
+all watched repos for open issues/PRs assigned to the bot that were recently
+updated and still need a response. This catches cases where notifications aren't
+generated (e.g. self-assignment, API-created issues).
 
 Key design decisions:
 
-- **The poller never marks notifications as read.** The agent does that after
-  processing. This prevents lost notifications if the agent fails to process.
-- **It tracks notification IDs, not counts.** Only fires on genuinely new
-  notifications, not re-reads of existing ones.
-- **Flag file instead of wake events.** We initially used OpenClaw's
-  `/hooks/wake` endpoint, but wake events target the main (DM) session — any
-  model response during processing leaked to DM as a notification. The flag file
-  approach is processed during heartbeats, where output routing is controlled.
+- **The poller IS the dispatcher.** No flag files, no heartbeat dependency. The
+  poller triages notifications and spawns agents directly.
+- **Marks notifications as read immediately.** Each notification is marked read
+  as it's processed, preventing re-dispatch on the next poll.
+- **One agent per issue.** Each spawned agent gets a `SCOPE` instruction
+  limiting it to one specific issue/PR. Agents post results as Gitea comments,
+  not DMs.
+- **Dedup tracking.** An in-memory `dispatched_issues` set prevents spawning
+  multiple agents for the same issue within one poller lifetime.
+- **`--no-deliver` instead of `--announce`.** Agents report via Gitea API
+  directly. The `--announce` flag on isolated sessions had delivery failures.
+- **Assignment scan filters by recency.** Only issues updated in the last 5
+  minutes are considered, preventing re-dispatch for stale assigned issues.
 - **Zero dependencies.** Just Python stdlib. Runs anywhere.
 
-Tradeoff: notifications are processed at heartbeat cadence (~30 min) instead of
-realtime. For code review and issue triage, this is fine.
+Response time: ~15–60s from notification to agent comment (vs ~30 min with the
+old heartbeat approach).
 
 ```python
 #!/usr/bin/env python3
 """
-Gitea notification poller (flag-file approach).
-Polls for unread notifications and writes a flag file when new ones
-appear. The agent checks this flag during heartbeats and processes
-notifications via the Gitea API directly.
+Gitea notification poller + dispatcher.
+
+Two polling loops:
+1. Notification-based: detects new notifications (mentions, assignments by
+   other users) and dispatches agents for actionable ones.
+2. Assignment-based: periodically checks for open issues/PRs assigned to
+   the bot that have no recent bot comment. Catches cases where
+   notifications aren't generated (e.g. self-assignment, API-created issues).
 
 Required env vars:
   GITEA_URL        - Gitea instance URL
   GITEA_TOKEN      - Gitea API token
 
 Optional env vars:
-  FLAG_PATH   - Path to flag file (default: workspace/memory/gitea-notify-flag)
-  POLL_DELAY  - Delay between polls in seconds (default: 5)
+  POLL_DELAY       - Delay between polls in seconds (default: 15)
+  COOLDOWN         - Minimum seconds between dispatches (default: 30)
+  ASSIGNMENT_INTERVAL - Seconds between assignment scans (default: 120)
+  OPENCLAW_BIN     - Path to openclaw binary
 """
 
 import json
 import os
+import subprocess
 import sys
 import time
 import urllib.request
@@ -240,62 +258,158 @@ import urllib.error
 
 GITEA_URL = os.environ.get("GITEA_URL", "").rstrip("/")
 GITEA_TOKEN = os.environ.get("GITEA_TOKEN", "")
-POLL_DELAY = int(os.environ.get("POLL_DELAY", "5"))
-FLAG_PATH = os.environ.get(
-    "FLAG_PATH",
-    os.path.join(
-        os.path.dirname(os.path.dirname(os.path.abspath(__file__))),
-        "memory",
-        "gitea-notify-flag",
-    ),
-)
+POLL_DELAY = int(os.environ.get("POLL_DELAY", "15"))
+COOLDOWN = int(os.environ.get("COOLDOWN", "30"))
+ASSIGNMENT_INTERVAL = int(os.environ.get("ASSIGNMENT_INTERVAL", "120"))
+OPENCLAW_BIN = os.environ.get("OPENCLAW_BIN", "/opt/homebrew/bin/openclaw")
+BOT_USER = "clawbot"  # Change to your bot's Gitea username
+
+# Repos to scan for assigned issues
+WATCHED_REPOS = [
+    # "org/repo1",
+    # "org/repo2",
+]
+
+# Track dispatched issues to prevent duplicates
+dispatched_issues = set()
 
 
-def check_config():
-    if not GITEA_URL or not GITEA_TOKEN:
-        print("ERROR: GITEA_URL and GITEA_TOKEN required", file=sys.stderr)
-        sys.exit(1)
+def gitea_api(method, path, data=None):
+    url = f"{GITEA_URL}/api/v1{path}"
+    body = json.dumps(data).encode() if data else None
+    headers = {"Authorization": f"token {GITEA_TOKEN}"}
+    if body:
+        headers["Content-Type"] = "application/json"
+    req = urllib.request.Request(url, headers=headers, method=method, data=body)
+    try:
+        with urllib.request.urlopen(req, timeout=15) as resp:
+            raw = resp.read()
+            return json.loads(raw) if raw else None
+    except Exception as e:
+        print(f"WARN: {method} {path}: {e}", file=sys.stderr, flush=True)
+        return None
 
 
-def gitea_unread_ids():
-    req = urllib.request.Request(
-        f"{GITEA_URL}/api/v1/notifications?status-types=unread",
-        headers={"Authorization": f"token {GITEA_TOKEN}"},
+def needs_bot_response(repo_full, issue_number):
+    """True if the bot is NOT the author of the most recent comment."""
+    comments = gitea_api("GET", f"/repos/{repo_full}/issues/{issue_number}/comments")
+    if comments and len(comments) > 0:
+        if comments[-1].get("user", {}).get("login") == BOT_USER:
+            return False
+    return True
+
+
+def is_actionable(notif):
+    """Returns (actionable, reason, issue_number)."""
+    subject = notif.get("subject", {})
+    repo = notif.get("repository", {})
+    repo_full = repo.get("full_name", "")
+    url = subject.get("url", "")
+    number = url.rstrip("/").split("/")[-1] if url else ""
+    if not number or not number.isdigit():
+        return False, "no issue number", None
+
+    issue = gitea_api("GET", f"/repos/{repo_full}/issues/{number}")
+    if not issue:
+        return False, "couldn't fetch issue", number
+
+    assignees = [a.get("login") for a in (issue.get("assignees") or [])]
+    if BOT_USER in assignees:
+        if needs_bot_response(repo_full, number):
+            return True, f"assigned to {BOT_USER}", number
+        return False, "assigned but already responded", number
+
+    issue_body = issue.get("body", "") or ""
+    if f"@{BOT_USER}" in issue_body and issue.get("user", {}).get("login") != BOT_USER:
+        if needs_bot_response(repo_full, number):
+            return True, f"@-mentioned in body", number
+
+    comments = gitea_api("GET", f"/repos/{repo_full}/issues/{number}/comments")
+    if comments:
+        last = comments[-1]
+        if last.get("user", {}).get("login") == BOT_USER:
+            return False, "own comment is latest", number
+        if f"@{BOT_USER}" in (last.get("body") or ""):
+            return True, f"@-mentioned in comment", number
+
+    return False, "not mentioned or assigned", number
+
+
+def spawn_agent(repo_full, issue_number, title, subject_type, reason):
+    dispatch_key = f"{repo_full}#{issue_number}"
+    if dispatch_key in dispatched_issues:
+        return
+    dispatched_issues.add(dispatch_key)
+
+    repo_short = repo_full.split("/")[-1]
+    job_name = f"gitea-{repo_short}-{issue_number}-{int(time.time())}"
+    msg = (
+        f"Gitea: {reason} on {subject_type} #{issue_number} "
+        f"'{title}' in {repo_full}.\n"
+        f"API: {GITEA_URL}/api/v1 | Token: {GITEA_TOKEN}\n"
+        f"SCOPE: Only {subject_type} #{issue_number} in {repo_full}.\n"
+        f"Read all comments, do the work, post results as Gitea comments."
     )
     try:
-        with urllib.request.urlopen(req, timeout=10) as resp:
-            return {n["id"] for n in json.loads(resp.read())}
+        subprocess.run(
+            [OPENCLAW_BIN, "cron", "add",
+             "--name", job_name, "--at", "1s",
+             "--message", msg, "--delete-after-run",
+             "--session", "isolated", "--no-deliver",
+             "--thinking", "low", "--timeout-seconds", "300"],
+            capture_output=True, text=True, timeout=15,
+        )
     except Exception as e:
-        print(f"WARN: Gitea API failed: {e}", file=sys.stderr, flush=True)
-        return set()
-
-
-def write_flag(count):
-    os.makedirs(os.path.dirname(FLAG_PATH), exist_ok=True)
-    with open(FLAG_PATH, "w") as f:
-        f.write(json.dumps({
-            "ts": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
-            "count": count,
-        }))
+        print(f"Spawn error: {e}", file=sys.stderr, flush=True)
+        dispatched_issues.discard(dispatch_key)
 
 
 def main():
-    check_config()
-    print(f"Gitea poller started (delay={POLL_DELAY}s, flag={FLAG_PATH})", flush=True)
-    last_seen_ids = gitea_unread_ids()
-    print(f"Initial unread: {len(last_seen_ids)}", flush=True)
+    print(f"Poller started (poll={POLL_DELAY}s, cooldown={COOLDOWN}s)", flush=True)
+    seen_ids = set(n["id"] for n in (gitea_api("GET", "/notifications?status-types=unread") or []))
+    last_dispatch = 0
+    last_assign_scan = 0
 
     while True:
         time.sleep(POLL_DELAY)
-        current_ids = gitea_unread_ids()
-        new_ids = current_ids - last_seen_ids
-        if not new_ids:
-            last_seen_ids = current_ids
+        now = time.time()
+
+        # Notification polling
+        notifs = gitea_api("GET", "/notifications?status-types=unread") or []
+        current_ids = {n["id"] for n in notifs}
+        new_ids = current_ids - seen_ids
+        if new_ids and now - last_dispatch >= COOLDOWN:
+            for n in [n for n in notifs if n["id"] in new_ids]:
+                nid = n.get("id")
+                if nid:
+                    gitea_api("PATCH", f"/notifications/threads/{nid}")
+                is_act, reason, num = is_actionable(n)
+                if is_act:
+                    repo = n["repository"]["full_name"]
+                    title = n["subject"]["title"][:60]
+                    stype = n["subject"].get("type", "").lower()
+                    spawn_agent(repo, num, title, stype, reason)
+            last_dispatch = now
+        seen_ids = current_ids
+
+        # Assignment scan (less frequent)
+        if now - last_assign_scan >= ASSIGNMENT_INTERVAL:
+            for repo in WATCHED_REPOS:
+                for itype in ["issues", "pulls"]:
+                    items = gitea_api("GET",
+                        f"/repos/{repo}/issues?state=open&type={itype}"
+                        f"&assignee={BOT_USER}&sort=updated&limit=10") or []
+                    for item in items:
+                        num = str(item["number"])
+                        if f"{repo}#{num}" in dispatched_issues:
                             continue
-        ts = time.strftime("%H:%M:%S")
-        print(f"[{ts}] {len(new_ids)} new ({len(current_ids)} total), flag written", flush=True)
-        write_flag(len(new_ids))
-        last_seen_ids = current_ids
+                        # Only recently updated items (5 min)
+                        # ... add is_recently_updated() check here
+                        if needs_bot_response(repo, num):
+                            spawn_agent(repo, num, item["title"][:60],
+                                "pull" if itype == "pulls" else "issue",
+                                f"assigned to {BOT_USER}")
+            last_assign_scan = now
 
 
 if __name__ == "__main__":