phase1: FEATURE-PARITY-TESTS.md (74 tests) + DEPENDENCIES.md (28 items)
This commit is contained in:
222
DEPENDENCIES.md
Normal file
222
DEPENDENCIES.md
Normal file
@@ -0,0 +1,222 @@
|
||||
# Openclaw dependency matrix
|
||||
|
||||
Every openclaw-provided capability that the current gitea-webhooks pipeline uses, and what Caret's replacement does with it. Source citations: R01 = RESEARCH-01, R02 = RESEARCH-02, R03 = RESEARCH-03, PLAN = PLAN.md.
|
||||
|
||||
Categories:
|
||||
- **REMOVE** — Caret's replacement doesn't need this at all
|
||||
- **REPLACE** — Caret builds a standalone equivalent
|
||||
- **KEEP** — Caret continues depending on openclaw for this (and why that's safe)
|
||||
- **BLOCKING** — Caret cannot move forward without resolving this first (Rooh action required)
|
||||
|
||||
## By capability
|
||||
|
||||
### 1. HTTP webhook ingress (port 18789, `/hooks/agent`)
|
||||
**Category:** REPLACE
|
||||
**What Caret does:** Stand up a bun HTTP listener at a Caret-owned port (e.g. 18790 or unix socket), with its own path. Mirrors openclaw's `/hooks/agent` shape so existing scripts can be reused. [R01 §Ingress path, R02 §HTTP endpoints]
|
||||
**Risk if kept:** Hard-couples Caret's lifecycle to the openclaw gateway container. Defeats the entire migration.
|
||||
**Effort:** 1 day. Stateless request/response, well-understood. [R02 difficulty matrix: Easy]
|
||||
|
||||
### 2. HMAC verification (currently NOT done by openclaw)
|
||||
**Category:** REPLACE (net-new — openclaw never had it)
|
||||
**What Caret does:** Real HMAC-SHA256 of the raw body using `X-Gitea-Signature` header, timing-safe compare, raw body preserved before JSON parse. [R01 §HMAC recipe, R03 confirms all 4 known webhooks have `Secret: NOT SET`]
|
||||
**Risk if kept:** No HMAC at all today; protection is bearer + nginx ACL only. Caret should do better.
|
||||
**Effort:** 0.5 day. ~30 lines.
|
||||
**Depends on BLOCKING #1 (webhook secret provisioning).**
|
||||
|
||||
### 3. Bearer token authentication (`OPENCLAW_HOOKS_TOKEN`)
|
||||
**Category:** REPLACE
|
||||
**What Caret does:** Own bearer token (`CARET_HOOKS_TOKEN`) stored in Caret's credentials dir, validated on each request as a second factor alongside HMAC. [R01 §Ingress path layer 3]
|
||||
**Risk if kept:** Sharing the openclaw token cross-couples secret rotation.
|
||||
**Effort:** Trivial.
|
||||
|
||||
### 4. nginx TLS termination and path rewriting
|
||||
**Category:** KEEP (with new location block)
|
||||
**What Caret does:** Adds a new nginx `location /hooks/caret` block that proxies to the Caret listener, same TLS cert. Reuses existing Let's Encrypt setup. [R01 §Ingress path]
|
||||
**Risk if kept:** None — nginx is host-level infra, not openclaw-specific.
|
||||
**Effort:** 1 hour, but requires host root access.
|
||||
**See BLOCKING #2.**
|
||||
|
||||
### 5. Event dedup cache (`X-Gitea-Delivery` 24h)
|
||||
**Category:** REPLACE
|
||||
**What Caret does:** Own JSON-on-disk dedup cache at `/host/root/.caret/state/dedup-cache.json`, identical 24h TTL semantics. [R01 §Ingress path layer 4, R01 gotcha 4]
|
||||
**Risk if kept:** Sharing openclaw's cache file is fragile and couples crash recovery.
|
||||
**Effort:** 0.5 day.
|
||||
|
||||
### 6. Rate limiting (5 concurrent per agent)
|
||||
**Category:** REPLACE
|
||||
**What Caret does:** Per-route concurrency limiter in the listener. Caret's routing is simpler (one "agent") so this is mostly a global semaphore. [R01 gotcha 7]
|
||||
**Risk if kept:** N/A.
|
||||
**Effort:** Trivial.
|
||||
|
||||
### 7. Session lock manager (`hooks/locks/`)
|
||||
**Category:** REPLACE
|
||||
**What Caret does:** Own lock dir at `/host/root/.caret/state/locks/`, same 2h TTL, same file naming `{owner}-{repo}-{issue}`, same IS_DONE 5min grace. [R01 §Session lock]
|
||||
**Risk if kept:** Two writers to one lock dir = race.
|
||||
**Effort:** 0.5 day.
|
||||
|
||||
### 8. Queue daemon (`queue-inbox/` + `queue-daemon.js`)
|
||||
**Category:** REPLACE
|
||||
**What Caret does:** In-process queue inside the listener (or a sidecar bun script reading from `/host/root/.caret/state/queue/`). Re-verifies spawn signatures off the hot path. [R01 gotcha 12, R03 §queue-daemon.js]
|
||||
**Risk if kept:** Couples Caret's spawn pipeline to openclaw container restart.
|
||||
**Effort:** 1 day.
|
||||
|
||||
### 9. `sessions_spawn` agent orchestration primitive
|
||||
**Category:** REPLACE (with Claude Code Channels plugin)
|
||||
**What Caret does:** Build a Channels plugin at `/host/root/.caret/channels/gitea-judgment/` that receives an HTTP POST and starts a Claude Code session with the payload as initial prompt. [R02 §Caret's conclusion, PLAN §Phase 3 J3.1]
|
||||
**Risk if kept:** Openclaw's `sessions_spawn` is the deepest coupling — see R02 difficulty matrix "Hard" rating. Keeping it means Caret can never fully cut over.
|
||||
**Effort:** 2-3 days for the minimum viable plugin. Full parity (wakeMode, thinking, model selection) is more like 1 week.
|
||||
|
||||
### 10. Tool allowlist enforcement per agent
|
||||
**Category:** KEEP (judgment path runs inside Claude Code, which has its own tool config)
|
||||
**What Caret does:** Caret's deterministic path runs scripts directly with no tools concept. The judgment path is a Claude Code session whose tool allowlist is configured in Claude Code's settings.json (not openclaw's). [R02 §Tool policy resolution]
|
||||
**Risk if kept:** None — Caret never touches openclaw's tool policy resolver.
|
||||
**Effort:** 0 (already separated).
|
||||
|
||||
### 11. Plugin SDK (`src/plugin-sdk/`)
|
||||
**Category:** REMOVE
|
||||
**What Caret does:** Doesn't load openclaw plugins. Caret's listener is a plain bun script with no plugin loader. Channels plugin uses Claude Code's plugin mechanism, not openclaw's. [R02 §Don't reinvent #2]
|
||||
**Risk if kept:** N/A.
|
||||
**Effort:** 0.
|
||||
|
||||
### 12. Delivery system (Telegram, Mattermost, Discord)
|
||||
**Category:** REPLACE (use tg-stream + Mattermost direct calls)
|
||||
**What Caret does:** Calls tg-stream HTTP API for Telegram alerts and Mattermost webhook URL directly for incident posts. No multi-channel abstraction layer. [R02 §Don't reinvent #4 warns it's tightly coupled — we don't replicate, we use simpler primitives]
|
||||
**Risk if kept:** Hard coupling to openclaw outbound system.
|
||||
**Effort:** 0.5 day (just curl calls).
|
||||
|
||||
### 13. Session storage (JSONL transcripts at `~/.openclaw/sessions/`)
|
||||
**Category:** KEEP (judgment path only — Claude Code's own session store)
|
||||
**What Caret does:** Deterministic path has no sessions. Judgment path uses Claude Code's native session JSONL under Claude Code's config dir, NOT openclaw's. [R02 §Don't reinvent #1]
|
||||
**Risk if kept:** None — Caret never reads/writes openclaw's session files.
|
||||
**Effort:** 0.
|
||||
|
||||
### 14. Heartbeat scheduler (`heartbeat-runner.ts`)
|
||||
**Category:** REPLACE (use Claude Code `schedule` skill / cron)
|
||||
**What Caret does:** Use systemd timer in the Caret container OR Claude Code's schedule mechanism for the 6h policy sweep and 15min webhook audit. No 28-item checklist — Caret's heartbeat is much smaller. [R02 §Heartbeat loop, R03 §Heartbeat checklists]
|
||||
**Risk if kept:** Couples to openclaw gateway uptime, which R03 shows is currently degraded.
|
||||
**Effort:** 0.5 day for the timer setup.
|
||||
|
||||
### 15. Cron service (`CronService` in gateway)
|
||||
**Category:** REPLACE (same as #14 — single mechanism)
|
||||
**What Caret does:** Same as heartbeat — systemd timers or Claude Code schedule. [R02 §Cron service, R03 §Active cron jobs shows 8 of 12 currently failing]
|
||||
**Risk if kept:** R03 shows openclaw cron is currently degraded; depending on it inherits the breakage.
|
||||
**Effort:** Combined with #14.
|
||||
|
||||
### 16. Gitea API integration (token + curl wrappers)
|
||||
**Category:** KEEP
|
||||
**What Caret does:** Continue using the same `GITEA_TOKEN` and stateless curl calls. Wrapper scripts ported into `/host/root/.caret/tools/`. [R01 §Tools fan-out, PLAN B2.3]
|
||||
**Risk if kept:** None — Gitea API is a third-party service, openclaw is not in the path.
|
||||
**Effort:** Path-strip and copy, 1 hour.
|
||||
**Depends on BLOCKING #3 (admin scope).**
|
||||
|
||||
### 17. Workspace directory (`/root/.openclaw/workspace/`)
|
||||
**Category:** KEEP (155 projects stay owned by openclaw/Xen)
|
||||
**What Caret does:** Caret's migration scope is the Gitea-facing slice ONLY. The 155-project workspace, the PROJ-XXX-* hierarchy, and Xen's project ownership stay as-is. Caret has its own `/host/root/.caret/workspace/` for migration artifacts only. [R03 §What the migration actually has to replace, PLAN §Goal]
|
||||
**Risk if kept:** None for the in-scope migration.
|
||||
**Effort:** 0.
|
||||
|
||||
### 18. Project registry (`projects/registry.json`)
|
||||
**Category:** KEEP
|
||||
**What Caret does:** Read-only access if needed (e.g., to look up which manager owns a project for a given issue). No writes. [R03 §Shared state]
|
||||
**Risk if kept:** Stale reads possible but acceptable.
|
||||
**Effort:** 0.
|
||||
|
||||
### 19. Memory system (`memory/`)
|
||||
**Category:** REMOVE (from migration scope)
|
||||
**What Caret does:** Caret's own memory is in `/root/.claude/projects/-app/memory/`. Doesn't touch openclaw's memory dir. [R03 §Shared state]
|
||||
**Risk if kept:** N/A.
|
||||
**Effort:** 0.
|
||||
|
||||
### 20. Credentials store (`credentials/`)
|
||||
**Category:** KEEP (read-only for shared tokens like GITEA_TOKEN)
|
||||
**What Caret does:** Read GITEA_TOKEN from openclaw credentials OR copy to Caret credentials store. Caret-owned secrets (HMAC webhook secret, CARET_HOOKS_TOKEN) live in `/host/root/.caret/credentials/`. [R03 §Shared state credentials]
|
||||
**Risk if kept:** Cross-coupling on token rotation. Acceptable short-term.
|
||||
**Effort:** Trivial.
|
||||
**See BLOCKING #4 (secret rotation story).**
|
||||
|
||||
### 21. `post-repo-audit.sh`
|
||||
**Category:** REPLACE (port the script verbatim)
|
||||
**What Caret does:** Copy to `/host/root/.caret/tools/post-repo-audit.sh`, strip openclaw path prefixes, ship as-is. Pure script, zero tokens. [R01 §Tools fan-out, PLAN B2.3]
|
||||
**Risk if kept:** Couples to openclaw tools dir lifecycle.
|
||||
**Effort:** 1 hour.
|
||||
|
||||
### 22. `audit-repo-policies.sh`
|
||||
**Category:** REPLACE (port verbatim)
|
||||
**What Caret does:** Copy to Caret tools dir, point at sol/repo-policies template repo, run with `--fix` from Caret's heartbeat. [R01 §Tools fan-out]
|
||||
**Risk if kept:** Same as #21.
|
||||
**Effort:** 1 hour.
|
||||
|
||||
### 23. `spawn-manager.sh`
|
||||
**Category:** REPLACE (port verbatim, but only after #9 is solved)
|
||||
**What Caret does:** Copy script. Generates Manager spawn JSON from issue body, creates project workspace. Caret's listener calls it via execSync with same 30s timeout pattern. [R01 §Tools fan-out, R01 gotcha 9]
|
||||
**Risk if kept:** Tightly coupled to openclaw workspace path conventions — depends on whether Caret keeps openclaw's PROJ-XXX scheme (R01 advises yes).
|
||||
**Effort:** 0.5 day.
|
||||
|
||||
### 24. `create-implement-issue.sh`
|
||||
**Category:** REPLACE (port verbatim)
|
||||
**What Caret does:** Copy script. Creates signed `[IMPLEMENT]` issue with HMAC spawn signature using `/host/root/.caret/credentials/spawn-secret`. [R01 §Tools fan-out]
|
||||
**Risk if kept:** Spawn secret coupling.
|
||||
**Effort:** 1 hour.
|
||||
**Depends on BLOCKING #5 (spawn secret ownership).**
|
||||
|
||||
### 25. `secret-scan.sh`
|
||||
**Category:** REPLACE (port verbatim)
|
||||
**What Caret does:** Copy script, used by both CI (`make check`) inside repos and Caret's push handler. [R01 §Tools fan-out]
|
||||
**Risk if kept:** N/A.
|
||||
**Effort:** 1 hour.
|
||||
|
||||
### 26. `check-implement-orphans.sh`
|
||||
**Category:** REPLACE (port verbatim)
|
||||
**What Caret does:** Copy script, run from Caret's 15min heartbeat to detect stale pending spawns and orphaned managers. [R01 §Tools fan-out]
|
||||
**Risk if kept:** N/A.
|
||||
**Effort:** 1 hour.
|
||||
|
||||
### 27. `auditLog → logs/audit.jsonl`
|
||||
**Category:** REPLACE
|
||||
**What Caret does:** Caret's listener writes to `/host/root/.caret/log/audit.jsonl`, same JSONL line schema. Rotation by line count. [R01 §Logging, PLAN B2.5]
|
||||
**Risk if kept:** Two writers to one file = corruption.
|
||||
**Effort:** 0.5 day.
|
||||
|
||||
### 28. Incident flagging (`logs/incidents.jsonl`)
|
||||
**Category:** REPLACE
|
||||
**What Caret does:** Caret writes to `/host/root/.caret/log/incidents.jsonl`. Critical incidents also fan out to Telegram via tg-stream. [R01 §Logging, T2.23]
|
||||
**Risk if kept:** Same as #27.
|
||||
**Effort:** 0.5 day (combined with #27).
|
||||
|
||||
## Summary by category
|
||||
|
||||
| Category | Count |
|
||||
|---|---|
|
||||
| REMOVE | 2 (#11, #19) |
|
||||
| REPLACE | 19 (#1, #2, #3, #5, #6, #7, #8, #9, #12, #14, #15, #21, #22, #23, #24, #25, #26, #27, #28) |
|
||||
| KEEP | 6 (#4, #10, #13, #16, #17, #18, #20) — note #20 partially |
|
||||
| BLOCKING | 5 (see below) |
|
||||
|
||||
## BLOCKING items — Rooh action required
|
||||
|
||||
These five items must be resolved by Rooh before Caret can write the production code. They are not engineering decisions; they are policy / access decisions only Rooh can make.
|
||||
|
||||
### BLOCKING #1 — Webhook secret provisioning and storage location
|
||||
**Question:** Where is the new HMAC webhook secret stored? `/host/root/.caret/credentials/webhook-secret`? A vault entry? Reused from an existing openclaw secret?
|
||||
**Why it blocks:** Cannot ship HMAC verification (T1.06–T1.11) without the secret being authoritative somewhere. Cannot register webhooks on sol/* repos without knowing what secret to set on the Gitea side.
|
||||
**Reference:** R01 §HMAC recipe, R03 confirms all current webhooks have `Secret: NOT SET`.
|
||||
|
||||
### BLOCKING #2 — nginx config write access for `/hooks/caret` location
|
||||
**Question:** Does Caret have permission to edit `/host/etc/nginx/...` and reload nginx, or does Rooh do that one-time setup?
|
||||
**Why it blocks:** Without an nginx route, Gitea cannot reach the Caret listener via HTTPS. Direct port exposure is the only alternative and that needs firewall changes.
|
||||
**Reference:** R01 §Ingress path nginx termination, dependency #4.
|
||||
|
||||
### BLOCKING #3 — Gitea token admin scope OR manual system-webhook registration
|
||||
**Question:** Will Rooh elevate the sol token to include `read:admin` + `write:admin`, or will Rooh manually register the system-level webhook one time?
|
||||
**Why it blocks:** Without admin scope Caret cannot list system-level webhooks (R03 confirmed) and cannot create them. The sol/* per-repo webhooks can be registered with the existing token, but for new repo bootstrap to register Caret's webhook automatically (T1.02), the admin scope or manual setup is required.
|
||||
**Reference:** R03 §Registered Gitea webhooks, PLAN §Dependencies token scope.
|
||||
|
||||
### BLOCKING #4 — Secret rotation story across openclaw + Caret
|
||||
**Question:** When `GITEA_TOKEN` or the webhook secret rotates, what's the rollout? Does Caret read from openclaw's credentials dir live, or get its own copy that needs separate rotation? Who is responsible for rotation drills?
|
||||
**Why it blocks:** PLAN §Risks #1 explicitly calls this out as a first-class deliverable. Without a documented rotation procedure, T3.13 cannot pass and the migration leaves a security debt.
|
||||
**Reference:** PLAN §Risks #1, dependency #20.
|
||||
|
||||
### BLOCKING #5 — Spawn signature secret ownership (`/root/.openclaw/hooks/spawn-secret`)
|
||||
**Question:** Does Caret get its own spawn secret (and `create-implement-issue.sh` is updated to use it), or does Caret read openclaw's existing secret? If the latter, what happens when openclaw rotates it?
|
||||
**Why it blocks:** T1.15–T1.17 (spawn signature verification) require Caret and `create-implement-issue.sh` to share a secret. Cross-system sharing is the cleaner short-term answer but locks Caret to openclaw's lifecycle until cut-over completes.
|
||||
**Reference:** R01 §Ingress path "spawn signatures", dependency #24.
|
||||
126
FEATURE-PARITY-TESTS.md
Normal file
126
FEATURE-PARITY-TESTS.md
Normal file
@@ -0,0 +1,126 @@
|
||||
# Feature parity test list
|
||||
|
||||
Every test must pass on the Caret replacement before cut-over (Phase 5). Tests are ordered by criticality — tier 1 is "must pass", tier 2 is "should pass", tier 3 is "nice to have". Sources cited inline: R01 = RESEARCH-01-gitea-webhooks-deep-read.md, R02 = RESEARCH-02-gateway-internals.md, R03 = RESEARCH-03-live-state-audit.md.
|
||||
|
||||
## Tier 1 — mandatory
|
||||
|
||||
### Repo creation / bootstrap
|
||||
|
||||
- [ ] T1.01 new sol/* repo created → Rooh (user id 29) added as admin collaborator within 10s [R01 §Tools fan-out, post-repo-audit.sh]
|
||||
- [ ] T1.02 new sol/* repo created → Caret webhook registered on the repo within 10s, URL points to Caret listener, content-type application/json [R01 §Tools fan-out, R03 §Registered Gitea webhooks]
|
||||
- [ ] T1.03 new sol/* repo created → required policy files (Makefile, .editorconfig, .prettierrc, .prettierignore, .dockerignore, .gitignore, tools/secret-scan.sh) present on default branch within 30s [R01 audit-repo-policies.sh]
|
||||
- [ ] T1.04 repo creation handler is idempotent — re-firing the same `repository.create` event does NOT create duplicate collaborator entries or duplicate webhooks [PLAN §Risks idempotency]
|
||||
- [ ] T1.05 repo creation handler runs as a pure script with zero LLM tokens consumed (asserted by zero entries in any token-spend log for that delivery id) [R01 post-repo-audit.sh "zero tokens"]
|
||||
|
||||
### Authentication / ingress
|
||||
|
||||
- [ ] T1.06 HTTPS POST to listener with valid HMAC-SHA256 over the raw body using the configured webhook secret → 200, processed [R01 §HMAC recipe]
|
||||
- [ ] T1.07 POST with wrong HMAC signature → rejected 403 within 50ms, body parsing skipped, "hmac_failed" line in log [R01 §HMAC recipe]
|
||||
- [ ] T1.08 POST missing `X-Gitea-Signature` header → rejected 403 with "missing_signature" log line [R01 §HMAC recipe]
|
||||
- [ ] T1.09 POST with valid HMAC but non-Gitea content-type → rejected 415 [R03 webhook content-type]
|
||||
- [ ] T1.10 raw body must be available to verifier — listener does NOT JSON-parse before HMAC verification (asserted by sending malformed JSON with valid HMAC and observing 200 + "unparseable_body" log) [R01 §HMAC recipe gotcha]
|
||||
- [ ] T1.11 timing-safe HMAC compare — sending one-byte-off signatures over 1000 requests shows constant-time response (variance < 5ms) [R01 §HMAC recipe `timingSafeEqual`]
|
||||
- [ ] T1.12 listener bound only to localhost OR fronted by nginx ACL — direct connection from non-allowlisted host refused [R01 §Ingress path]
|
||||
|
||||
### Event routing
|
||||
|
||||
- [ ] T1.13 `push` event to non-default branch → recorded in log, no enforcement scripts fired [R01 §Transform phases push]
|
||||
- [ ] T1.14 `push` to main/master → triggers secret-scan and policy re-check [R01 §Transform phases]
|
||||
- [ ] T1.15 `issues.opened` with title prefix `[IMPLEMENT]` from sol with valid `<!-- xen-spawn-sig:HMAC:TIMESTAMP -->` → spawn signature verified, dispatch enqueued [R01 §Transform phases issues.opened]
|
||||
- [ ] T1.16 `issues.opened` `[IMPLEMENT]` from sol with stale (>2h) signature → rejected with "spawn_sig_expired" [R01 gotcha 3]
|
||||
- [ ] T1.17 `issues.opened` `[IMPLEMENT]` from sol with invalid HMAC → rejected with "spawn_sig_invalid", incident written to incidents.jsonl [R01 §Transform phases]
|
||||
- [ ] T1.18 `issue_comment` containing approval word from Rooh (id 29) → lock acquired, EXECUTE_PLAN dispatched [R01 §Transform phases issue_comment]
|
||||
- [ ] T1.19 `issue_comment` approval word from non-Rooh → ignored, log line "approval_ignored_non_owner" [R01 §Trust level]
|
||||
- [ ] T1.20 `issue_comment` approval word from sol account → ignored (sol is contributor only) [R01 gotcha 6]
|
||||
- [ ] T1.21 events from `clawbot` sender → silently dropped (loop prevention) [R01 §Validation gates]
|
||||
- [ ] T1.22 issue body containing `<!-- openclaw-agent -->` (or Caret equivalent) → silently dropped (echo suppression) [R01 §Validation gates]
|
||||
|
||||
### Idempotency / dedup
|
||||
|
||||
- [ ] T1.23 duplicate delivery (same `X-Gitea-Delivery` id within 24h) → returns 200 but skips processing, log line "dedup_hit" [R01 §Ingress dedup, R01 gotcha 4]
|
||||
- [ ] T1.24 dedup cache persisted to disk and survives listener restart [R01 §Ingress dedup]
|
||||
- [ ] T1.25 dedup cache trims entries older than 24h on each write [R01 §Ingress dedup]
|
||||
|
||||
### Locking / concurrency
|
||||
|
||||
- [ ] T1.26 concurrent `issue_comment` events on the same `(owner, repo, issue)` → only one acquires the lock; second sees "lock_held" and is queued or dropped per policy [R01 §Session lock]
|
||||
- [ ] T1.27 lock file TTL is 2h; expired lock is reclaimable [R01 §Session lock]
|
||||
- [ ] T1.28 issue closed while locked → transitions to IS_DONE with 5min grace before lock release [R01 §Transform phases, R01 gotcha 2]
|
||||
- [ ] T1.29 rate limit: 6th concurrent agent dispatch for the same agent id → rejected with "rate_limited" [R01 gotcha 7]
|
||||
|
||||
### Observability / audit
|
||||
|
||||
- [ ] T1.30 every accepted event produces exactly one line in `audit.jsonl` containing `{ts, delivery_id, event, repo, sender, decision}` [R01 §Logging]
|
||||
- [ ] T1.31 every rejected event also produces an audit line with `decision: rejected` and reason [R01 §Logging, PLAN §Risks silent drops]
|
||||
- [ ] T1.32 incidents (HMAC fail, spawn sig fail, script error) appended to `incidents.jsonl` [R01 §Logging]
|
||||
|
||||
### Rollback
|
||||
|
||||
- [ ] T1.33 disabling Caret listener and re-enabling openclaw gateway restores end-to-end pipeline within 60s, verified by canary repo create [PLAN §Phase 5 C5.3]
|
||||
- [ ] T1.34 Caret listener can be stopped with one command and pending in-flight requests drained or 503'd cleanly [PLAN §Phase 5]
|
||||
|
||||
## Tier 2 — should pass
|
||||
|
||||
### Push / scan / policy
|
||||
|
||||
- [ ] T2.01 push to main triggers `secret-scan.sh`; a planted private-key blob in the diff creates a Gitea issue labeled `security` within 30s [R01 secret-scan.sh]
|
||||
- [ ] T2.02 push to main with no findings → exit 0, audit line "scan_clean", no issue created [R01 secret-scan.sh]
|
||||
- [ ] T2.03 secret-scan respects `.secret-scan-allowlist` — allowlisted hash is not flagged [R01 secret-scan.sh]
|
||||
- [ ] T2.04 `audit-repo-policies.sh --fix` re-applies missing files on a repo where one was deleted, within next 6h heartbeat [R01 audit-repo-policies.sh]
|
||||
- [ ] T2.05 `audit-webhooks.sh --fix` recreates a deleted webhook within next 15min check [R01 audit-webhooks.sh]
|
||||
- [ ] T2.06 push that touches `tools/secret-scan.sh` itself → policy re-check still passes (file is part of policy template) [R01 audit-repo-policies.sh]
|
||||
|
||||
### Spawn pipeline
|
||||
|
||||
- [ ] T2.07 `[IMPLEMENT]` issue with valid sig → `spawn-manager.sh` invoked, project workspace `PROJ-XXX-*` created [R01 spawn-manager.sh]
|
||||
- [ ] T2.08 `precomputeSpawnParams` execSync timeout (30s) → falls back to text-directive without crashing the listener [R01 gotcha 9]
|
||||
- [ ] T2.09 `asyncDispatchToSpawner` failure (spawner down) → dispatch failure recorded in incidents.jsonl, NOT a 500 to Gitea [R01 gotcha 8]
|
||||
- [ ] T2.10 `check-implement-orphans.sh` equivalent runs every 15min and detects stale pending spawn files older than 2h [R01 check-implement-orphans.sh]
|
||||
|
||||
### Trust levels
|
||||
|
||||
- [ ] T2.11 sender id 29 → trust=owner; collaborator → trust=contributor; unknown → trust=readonly [R01 §Trust level detection]
|
||||
- [ ] T2.12 readonly sender events → no script fan-out, only audit log [R01 §Trust level detection]
|
||||
|
||||
### Heartbeat / cron
|
||||
|
||||
- [ ] T2.13 6h policy sweep job runs on schedule and exits 0 [PLAN §Phase 4, R03 webhook-verify]
|
||||
- [ ] T2.14 15min webhook-audit job runs on schedule and exits 0 [R01 audit-webhooks.sh]
|
||||
- [ ] T2.15 cron job failure increments a consecutive-failure counter; ≥3 consecutive failures posts a Telegram alert [R03 §Critical finding cron failures]
|
||||
- [ ] T2.16 webhook-verify E2E canary (synthetic event roundtrip) succeeds every 6h [R03 webhook-verify]
|
||||
|
||||
### Gitea API hygiene
|
||||
|
||||
- [ ] T2.17 Gitea API call failure (5xx) → retried with exponential backoff up to 3 attempts before recording incident [PLAN §Risks]
|
||||
- [ ] T2.18 Gitea API rate-limit response (429) → backs off per Retry-After header, no incident on first occurrence [R02 §Replacement difficulty]
|
||||
- [ ] T2.19 token rotation: changing the Gitea token in config and SIGHUP'ing the listener takes effect without restart [PLAN §Risks HMAC secret management]
|
||||
|
||||
### Concurrency edges
|
||||
|
||||
- [ ] T2.20 two `repository.create` events for the same repo arriving within 100ms → exactly one bootstrap run, second deduped [T1.04 + dedup]
|
||||
- [ ] T2.21 listener under 50 req/s sustained for 60s → no dropped events, p99 latency < 500ms [R02 §Hook API]
|
||||
- [ ] T2.22 lock acquisition under contention is fair-ish — no event waits >5min behind a single 2h lock when policy says to drop, not queue [R01 §Session lock]
|
||||
|
||||
### Delivery / alerting
|
||||
|
||||
- [ ] T2.23 critical incident (HMAC failure storm: >10 in 1min) → Telegram alert posted via tg-stream within 30s [PLAN §Phase 3]
|
||||
- [ ] T2.24 alert delivery failure (Telegram down) → fallback to Mattermost, then to local incidents.jsonl [R02 §Delivery system]
|
||||
- [ ] T2.25 listener exposes a metrics counter for events_total, events_rejected_total, hmac_failures_total [R02 §Hook API observability]
|
||||
|
||||
## Tier 3 — nice to have
|
||||
|
||||
- [ ] T3.01 listener exposes `/health` returning `{"ok":true,"uptime_s":N,"last_event_ms":N}` [R02 §HTTP endpoints health]
|
||||
- [ ] T3.02 listener exposes `/ready` returning 503 until dedup cache loaded and Gitea token validated [R02 §HTTP endpoints]
|
||||
- [ ] T3.03 audit.jsonl rotated by line count (configurable, default 100k lines) — rotation happens without losing in-flight writes [PLAN B2.5]
|
||||
- [ ] T3.04 audit.jsonl old segments gzipped after rotation [PLAN B2.5]
|
||||
- [ ] T3.05 listener restart replays no events (dedup cache prevents) and reports start time in /health [R01 §Ingress dedup persisted]
|
||||
- [ ] T3.06 structured log lines parseable as single-line JSON, no multi-line stack traces [PLAN B2.5]
|
||||
- [ ] T3.07 listener handles SIGTERM gracefully — finishes in-flight, refuses new, exits within 10s [PLAN §Phase 5]
|
||||
- [ ] T3.08 dry-run mode (`CARET_DRYRUN=1`) logs the script that *would* run without executing [R02 §Tools invoke dryRun]
|
||||
- [ ] T3.09 admin API endpoint to list registered Gitea webhooks across all sol/* repos in one call (requires admin token) [R03 §Registered Gitea webhooks token scope]
|
||||
- [ ] T3.10 manual replay endpoint: POST `/replay/{delivery_id}` re-processes a stored event bypassing dedup [PLAN §Risks idempotency]
|
||||
- [ ] T3.11 listener config hot-reload on SIGHUP without restart [R02 §Hot-reload config]
|
||||
- [ ] T3.12 PR (`pull_request`) events trigger same policy checks as push to main [R01 §Transform phases]
|
||||
- [ ] T3.13 webhook secret rotation runbook: rotate, deploy, verify, rollback documented and tested [PLAN §Risks HMAC secret management]
|
||||
- [ ] T3.14 listener supports both `Authorization: Bearer` AND `X-Gitea-Signature` simultaneously for migration window [R01 §Ingress path layered auth]
|
||||
- [ ] T3.15 cost report: weekly summary of LLM tokens spent by judgment-path wakeups vs deterministic-path zero-cost runs [PLAN §Phase 3 cost hygiene]
|
||||
Reference in New Issue
Block a user