diff --git a/CHECKLIST.md b/CHECKLIST.md index e48ef23..99c8b59 100644 --- a/CHECKLIST.md +++ b/CHECKLIST.md @@ -60,15 +60,29 @@ been measured. ## Milestone 4 — the project layer -- [ ] Adopt a folder as a project; remember which host it is on. -- [ ] Build the page layout; repair it; reset it. +- [x] Adopt a folder as a project; remember which host it is on. + → `bin/office ` writes `/.harness/harness.toml` (session, project, + machine, folder, task watched) and builds the pages. Re-running changes nothing. +- [x] Build the page layout; repair it; reset it. + → `bin/board` adopts or creates each page and places them on workspaces 1–5 in order; + `--replace` rebuilds from nothing; `bin/project-reset` removed the panes, three + worktrees and three branches, and `bin/office` built them all back. - [ ] Pass: a project on another machine can be opened and used from here. + **Not tested.** This pane runs on .13 and I have no way to drive .27 or .51 from + here; `bin/office --host` and the runbook command for it are written but unproven. ## Milestone 5 — review and merge -- [ ] `ocr` runs over the diff; a failing review refuses the task. -- [ ] Merges are one at a time, re-checked after each. -- [ ] Pass: the next task cannot merge until the first is settled. +- [x] `ocr` runs over the diff; a failing review refuses the task. + → `AUD-165` added a `cleanup.sh` with `rm -rf "$HOME/tmp"`; `bin/merge --check` + refused with exit 5 and named the critical finding, and `cleanup.sh` never reached + `main`. +- [x] Merges are one at a time, re-checked after each. + → a second merge while one holds the lock exits 6; after the rebase the task's verify + command runs again, and again on the base branch after the merge. +- [x] Pass: the next task cannot merge until the first is settled. + → the lock refuses the second merge outright, and a refusal leaves the branch where + it was. ## Interface rules that are not optional diff --git a/README.md b/README.md index f1e7e01..c1d1409 100644 --- a/README.md +++ b/README.md @@ -58,6 +58,17 @@ clicks relative to the pane, so the check is a real TUIOS pane read back with `capture-pane`. The commands are in `docs/TUIOS-RUNBOOK.md`, which is the one place they live. +## The commands + +| command | what it does | +|---|---| +| `bin/office [--project X] [--session Y]` | adopt a folder: write `.harness/harness.toml`, build its pages | +| `bin/board [--folder DIR] [--replace]` | build or repair the pages; run twice and nothing is duplicated | +| `bin/project-reset [--dry-run] [--yes]` | close the pages, remove the worktrees and branches; keep the folder and every Paperclip record | +| `bin/merge [--check]` | rebase, verify, review with `ocr`, merge; a refusal is a decision for the person | +| `bin/review [--post]` | the same review on its own | +| `bin/screen tree\|reply\|log\|new [task]` | open one page | + ## Rules for this project - No interface work before a real task has run end to end. diff --git a/bin/board b/bin/board new file mode 100755 index 0000000..68d4b68 --- /dev/null +++ b/bin/board @@ -0,0 +1,89 @@ +#!/usr/bin/env python3 +"""Build or repair a project's panes, in a fixed order. + + bin/board # the project in the current folder + bin/board --folder ~/src/x # or another one + bin/board --replace # close this project's panes and build again + +Every page is a workspace with one pane, named after the page. The first +workspace holds a shell: it is the pane that never exits, so the session cannot +lose its first workspace to TUIOS cleanup. + +Nothing here holds a workspace number. Each page asks for its workspace by name +at the moment it opens, because closing a pane deletes an emptied workspace and +renumbers the rest. Run it twice and nothing is duplicated. +""" + +import os +import shlex +import sys + +HERE = os.path.dirname(os.path.abspath(__file__)) +sys.path.insert(0, os.path.join(HERE, "lib")) +import harness as h # noqa: E402 +import tuios # noqa: E402 +import project as pj # noqa: E402 + +BIN = os.path.join(os.path.dirname(HERE), "bin", "harness") + + +def parse(argv): + opts = {"folder": os.getcwd(), "replace": "--replace" in argv} + for i, a in enumerate(argv): + if a == "--folder" and i + 1 < len(argv): + opts["folder"] = argv[i + 1] + return opts + + +def pages(cfg): + """The pages, in the order they must appear on the workspace rail.""" + bin_path = BIN + proj = cfg["paperclip"]["project"] + task = pj.watch_task(cfg) + return task, [ + ("shell", "shell", "exec zsh -i"), + ("tree", "tree", "%s -screen tree -project %s" % (shlex.quote(bin_path), proj)), + ("reply", "reply", "%s -screen reply -issue %s" % (shlex.quote(bin_path), task)), + ("log", "log", "%s -screen log -issue %s" % (shlex.quote(bin_path), task)), + ("new", "new", "%s -screen new -project %s" % (shlex.quote(bin_path), proj)), + ] + + +def main(): + opts = parse(sys.argv[1:]) + folder = os.path.abspath(os.path.expanduser(opts["folder"])) + cfg = pj.load(folder) + pj.check_machine(cfg) + session = cfg["session"] + tuios.require_session(session) + os.environ["PAPERCLIP_HARNESS_SESSION"] = session + + if not os.path.isfile(BIN): + raise h.Refused(2, "no harness binary at %s; build it with 'cd tui && go build -o ../bin/harness .'" % BIN) + + task, page_list = pages(cfg) + h.say("board: %s (session %s, folder %s)" % (cfg["paperclip"]["project"], session, folder)) + h.say("watching: %s" % task) + + if opts["replace"]: + for name, _, _ in page_list: + if tuios.close_pane(name, session): + h.say(" closed %s" % name) + + for name, workspace, command in page_list: + result, ident, ws = tuios.open_pane( + name, workspace, folder, command, session, + adopt_existing=(name == "shell"), + ) + h.say(" %-6s %-8s workspace %s" % (name, result, ws)) + + for number, name in tuios.arrange([p[0] for p in page_list], session): + h.say(" placed %-6s workspace %s" % (name, number)) + + in_use = [w for w in tuios.workspaces(session) if (w.get("window_count") or 0) > 0] + h.say("workspaces in use: %d" % len(in_use)) + return 0 + + +if __name__ == "__main__": + h.main_guard(main) diff --git a/bin/lib/__pycache__/harness.cpython-314.pyc b/bin/lib/__pycache__/harness.cpython-314.pyc new file mode 100644 index 0000000..142488a Binary files /dev/null and b/bin/lib/__pycache__/harness.cpython-314.pyc differ diff --git a/bin/lib/__pycache__/project.cpython-314.pyc b/bin/lib/__pycache__/project.cpython-314.pyc new file mode 100644 index 0000000..f8add43 Binary files /dev/null and b/bin/lib/__pycache__/project.cpython-314.pyc differ diff --git a/bin/lib/__pycache__/tuios.cpython-314.pyc b/bin/lib/__pycache__/tuios.cpython-314.pyc new file mode 100644 index 0000000..db39c02 Binary files /dev/null and b/bin/lib/__pycache__/tuios.cpython-314.pyc differ diff --git a/bin/lib/harness.py b/bin/lib/harness.py new file mode 100644 index 0000000..e898228 --- /dev/null +++ b/bin/lib/harness.py @@ -0,0 +1,194 @@ +"""Shared helpers for the harness commands. + +Everything here talks to Paperclip with the person's key, or runs git and ocr. +No fact is stored: a task's worktree, branch and verify command are read from +Paperclip every time, so a stale copy cannot exist. +""" + +import json +import os +import re +import shutil +import subprocess +import sys +import urllib.error +import urllib.request + +API = os.environ.get("PAPERCLIP_API_URL", "http://127.0.0.1:3100").rstrip("/") +COMPANY = os.environ.get("PAPERCLIP_COMPANY_ID", "d4921013-e5da-4d88-8016-57d5746e2d03") +KEY = os.environ.get("PAPERCLIP_BOARD_KEY", "") + +BLOCKING_SEVERITIES = {"critical", "high"} + + +class Refused(Exception): + """A check that did not pass, with the reason a person needs.""" + + def __init__(self, code, message): + super().__init__(message) + self.code = code + + +def api(method, path, body=None, timeout=30): + if not KEY: + raise Refused(2, "PAPERCLIP_BOARD_KEY is not set") + data = json.dumps(body).encode() if body is not None else None + req = urllib.request.Request( + API + path, + data=data, + method=method, + headers={"Authorization": "Bearer " + KEY, "Content-Type": "application/json"}, + ) + try: + with urllib.request.urlopen(req, timeout=timeout) as f: + raw = f.read().decode() + return json.loads(raw) if raw else {} + except urllib.error.HTTPError as e: + raise Refused(2, "%s %s → %s %s" % (method, path, e.code, e.read().decode()[:300])) + + +def find_issue(ref): + """Find one task by id or identifier (AUD-163), then read it in full. + + The company list is a summary: it carries no worktree and no blockers. The + single-task read does, so it is always a second call. + """ + issues = api("GET", "/api/companies/%s/issues" % COMPANY) + for i in issues: + if i.get("identifier") == ref or i.get("id") == ref: + return api("GET", "/api/issues/%s" % i["id"]) + raise Refused(2, "no task matches %r" % ref) + + +def me(): + """The person the board key acts as.""" + d = api("GET", "/api/cli-auth/me") + uid = d.get("userId") + if not uid: + raise Refused(2, "the board key has no user id") + return uid + + +def hand_to_person(issue): + """Take the task off the agent before writing on it. + + A comment on a task assigned to an agent wakes that agent. That is right for + a person's reply and wrong for a tool's verdict: a merge result written on a + finished task would wake the agent to read it and spend money doing nothing. + Handing the task back to the person first is the documented way to leave a + task waiting for a human, and it stops the wake. + """ + if not issue.get("assigneeAgentId"): + return + api( + "PATCH", + "/api/issues/%s" % issue["id"], + {"assigneeAgentId": None, "assigneeUserId": me()}, + ) + issue["assigneeAgentId"] = None + + +def comment(issue_id, body): + return api("POST", "/api/issues/%s/comments" % issue_id, {"body": body}) + + +def verdict(issue, body): + """Write a tool's verdict on a task without waking its agent.""" + hand_to_person(issue) + return comment(issue["id"], body) + + +VERIFY_RE = re.compile( + r"Done when this command succeeds:\s*\n+\s*(?P[^\n]+)", re.IGNORECASE +) + + +def verify_command(issue, worktree): + """The command that decides the task is finished. + + It comes from the task's own description, then the project convention + .harness/verify.sh. A task with neither says so out loud. + """ + m = VERIFY_RE.search(issue.get("description") or "") + if m: + return m.group("cmd").strip(), "from the task description" + project_script = os.path.join(worktree, ".harness", "verify.sh") + if os.path.isfile(project_script): + return ".harness/verify.sh", "the project's .harness/verify.sh" + return None, "none" + + +def worktree_of(issue): + ws = issue.get("currentExecutionWorkspace") or {} + cwd = ws.get("cwd") or "" + if not cwd or not os.path.isdir(cwd): + raise Refused( + 7, + "%s has no worktree on this machine, so there is nothing to review or merge" + % issue.get("identifier"), + ) + return cwd, ws.get("branchName") or "", ws.get("baseRef") or "main" + + +def run(cmd, cwd=None, timeout=900): + p = subprocess.run(cmd, cwd=cwd, capture_output=True, text=True, timeout=timeout) + return p.returncode, (p.stdout or "") + (p.stderr or "") + + +def need(binary): + if not shutil.which(binary): + raise Refused(2, "%s is not installed" % binary) + + +def ocr_review(repo, base, head): + """Run ocr over one range and return its JSON report.""" + need("ocr") + code, out = run( + [ + "ocr", "review", + "--repo", repo, + "--from", base, + "--to", head, + "--format", "json", + "--audience", "agent", + "--color", "never", + ], + timeout=1800, + ) + try: + report = json.loads(out[out.index("{") :]) + except ValueError: + raise Refused(5, "ocr did not return a report (exit %d): %s" % (code, out[-400:])) + return report + + +def blocking_findings(report): + out = [] + for c in report.get("comments") or []: + sev = (c.get("severity") or "").lower() + if sev in BLOCKING_SEVERITIES: + out.append(c) + return out + + +def describe(c): + where = c.get("path") or "?" + line = c.get("start_line") + if line: + where += ":%s" % line + body = (c.get("content") or "").strip().replace("\n", " ") + if len(body) > 300: + body = body[:300] + "…" + return "- `%s` **%s/%s** — %s" % (where, c.get("severity"), c.get("category"), body) + + +def say(msg=""): + print(msg, flush=True) + + +def main_guard(fn): + try: + sys.exit(fn()) + except Refused as e: + say("refused: %s" % e) + sys.exit(e.code) diff --git a/bin/lib/project.py b/bin/lib/project.py new file mode 100644 index 0000000..c35bfc6 --- /dev/null +++ b/bin/lib/project.py @@ -0,0 +1,116 @@ +"""The project convention: one project, one folder, one machine. + +Paperclip owns the project record, its tasks, blockers, comments, runs, cost and +worktrees. What it cannot know is which folder on which machine holds the work, +which TUIOS session shows it, and which task the viewer is looking at. That is +this file's whole job: + + /.harness/harness.toml + +Nothing else is stored. The task list, the blocker edges and the worktrees are +read from Paperclip every time. +""" + +import os +import socket +import sys +import tomllib + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +import harness as h # noqa: E402 + +REL = os.path.join(".harness", "harness.toml") + + +def path(root): + return os.path.join(root, REL) + + +def exists(root): + return os.path.isfile(path(root)) + + +def load(root): + p = path(root) + if not os.path.isfile(p): + raise h.Refused(2, "no %s here; run bin/office %s first" % (REL, root)) + with open(p, "rb") as f: + return tomllib.load(f) + + +def save(root, cfg): + os.makedirs(os.path.dirname(path(root)), exist_ok=True) + lines = [ + "# Written by bin/office. One project, one folder, one machine.", + "# Paperclip owns the project, its tasks and its worktrees; this file only", + "# remembers where the work is and what to show.", + "", + 'session = "%s"' % cfg["session"], + 'watching = "%s"' % (cfg.get("watching") or ""), + "", + "[paperclip]", + 'company = "%s"' % cfg["paperclip"]["company"], + 'project = "%s"' % cfg["paperclip"]["project"], + "", + "[machine]", + 'host = "%s"' % cfg["machine"]["host"], + 'folder = "%s"' % cfg["machine"]["folder"], + "", + ] + with open(path(root), "w") as f: + f.write("\n".join(lines)) + + +def this_machine(): + return socket.gethostname() + + +def check_machine(cfg): + """Refuse to act on a project that belongs to another machine.""" + want = (cfg.get("machine") or {}).get("host") or "" + here = this_machine() + if want and want not in (here, here.split(".")[0]): + raise h.Refused( + 6, + "this project lives on %r but this machine is %r; run the command there, " + "or pass --host %s" % (want, here, want), + ) + + +def resolve_project(ref): + """Accept a project id, url key or name, and return the Paperclip record.""" + projects = h.api("GET", "/api/companies/%s/projects" % h.COMPANY) + for p in projects: + if ref in (p.get("id"), p.get("urlKey"), p.get("name")): + return p + raise h.Refused(2, "no Paperclip project matches %r" % ref) + + +def active_tasks(project_id): + issues = h.api("GET", "/api/companies/%s/issues" % h.COMPANY) + mine = [i for i in issues if i.get("projectId") == project_id] + order = {"in_progress": 0, "blocked": 1, "in_review": 2, "todo": 3, "backlog": 4, "done": 5, "cancelled": 6} + mine.sort(key=lambda i: (order.get(i.get("status"), 9), i.get("updatedAt") or ""), reverse=False) + return mine + + +def watch_task(cfg): + """Which task the reply and log panes should show. + + A person's last choice wins if it still exists; otherwise the most active + task in the project, so the panes are never empty for no reason. + """ + tasks = active_tasks(cfg["paperclip"]["project"]) + wanted = cfg.get("watching") or "" + for t in tasks: + if t.get("identifier") == wanted or t.get("id") == wanted: + return t["identifier"] + for t in tasks: + if t.get("status") not in ("done", "cancelled"): + return t["identifier"] + return tasks[0]["identifier"] if tasks else "none" + + +def set_watch(root, cfg, identifier): + cfg["watching"] = identifier + save(root, cfg) diff --git a/bin/lib/tuios.py b/bin/lib/tuios.py new file mode 100644 index 0000000..5080aa5 --- /dev/null +++ b/bin/lib/tuios.py @@ -0,0 +1,176 @@ +"""TUIOS helpers: claim a workspace by name, list what is open. + +A workspace number is never a home. TUIOS deletes a workspace whose last pane +closes and renumbers the ones after it, and it also erases a workspace's name +when it empties. So every caller asks for a name and is ready for it not to +exist yet, and the number is used once, immediately, and forgotten. +""" + +import json +import os +import subprocess +import sys + +SESSION = os.environ.get("PAPERCLIP_HARNESS_SESSION", "paperclip-harness") + + +def tuios(*args, session=None): + argv = ["tuios", *args, "-s", session or SESSION, "--json"] + p = subprocess.run(argv, capture_output=True, text=True, timeout=60) + if p.returncode != 0: + raise RuntimeError("tuios %s failed: %s" % (" ".join(args), p.stderr.strip())) + try: + return json.loads(p.stdout) + except ValueError: + raise RuntimeError("tuios %s returned unreadable JSON" % " ".join(args)) + + +def sessions(): + """Every session the daemon knows, as plain names.""" + p = subprocess.run(["tuios", "ls", "--json"], capture_output=True, text=True, timeout=30) + if p.returncode != 0: + raise RuntimeError("tuios ls failed: %s" % p.stderr.strip()) + data = json.loads(p.stdout) + rows = data if isinstance(data, list) else data.get("sessions", []) + return [r.get("name") for r in rows if isinstance(r, dict)] + + +def require_session(name): + """Sessions are made by a person, in a terminal. Say so, do not invent one.""" + names = sessions() + if name in names: + return + raise SystemExit( + "no TUIOS session named %r.\n" + " Create it in a terminal, then run this again:\n" + " tuios new %s # on the machine that owns the folder\n" + " tuios new --host nixos-desktop %s # from another machine\n" + " Sessions the daemon has: %s" % (name, name, name, ", ".join(names) or "none") + ) + + +def workspaces(session=None): + return tuios("list-workspaces", session=session).get("workspaces", []) + + +def windows(session=None): + return tuios("list-windows", session=session).get("windows", []) + + +def _name_workspace(number, name, session=None): + subprocess.run( + ["tuios", "set-workspace-name", str(number), name, "-s", session or SESSION], + capture_output=True, + text=True, + timeout=30, + check=False, + ) + + +def claim(name, session=None): + """Return the number of the workspace called `name`, naming one if needed.""" + ws = workspaces(session) + exact = next((w for w in ws if (w.get("name") or "") == name), None) + if exact is None: + same = next((w for w in ws if (w.get("name") or "").lower() == name.lower()), None) + if same is not None: + _name_workspace(same["workspace"], name, session) + return same["workspace"] + if exact is not None: + return exact["workspace"] + + # Nothing named that. Take a workspace that is unnamed and has no panes. + free = next( + (w for w in ws if "window_count" in w and (w.get("window_count") or 0) == 0 and not (w.get("name") or "")), + None, + ) + if free is None: + raise RuntimeError( + "no empty workspace left in session %r; close a pane or add a workspace" % (session or SESSION) + ) + _name_workspace(free["workspace"], name, session) + return free["workspace"] + + +def open_pane(name, workspace_name, cwd, command, session=None, adopt_existing=False): + """Open a pane in a named workspace, or say it is already there. + + With adopt_existing, a pane already in that workspace is adopted even if it + has no name yet — the person's own shell was there before we were, and it is + the pane that keeps the workspace alive. + """ + ws = claim(workspace_name, session) + for w in windows(session): + if (w.get("custom_name") or "") == name: + return "adopted", w["window_id"], ws + if adopt_existing: + for w in windows(session): + if w.get("workspace") == ws: + if not (w.get("custom_name") or ""): + subprocess.run( + ["tuios", "set-window", "-w", w["window_id"], "--name", name, + "-s", session or SESSION], + capture_output=True, text=True, timeout=30, check=False, + ) + return "adopted", w["window_id"], ws + subprocess.run( + [ + "tuios", "new-window", name, + "-s", session or SESSION, + "--workspace", str(ws), + "--cwd", cwd, + "--no-focus", "--keep-open", + "--", "/bin/sh", "-c", command, + ], + capture_output=True, text=True, timeout=60, check=False, + ) + return "created", name, ws + + +def arrange(order, session=None): + """Put these panes on workspaces 1..N in this order, and name them. + + TUIOS numbering follows the order panes land in, and closing panes renumbers + the rest, so a rebuild can scatter the pages. Moving each pane once, at the + end, is what makes the rail readable and stable. + """ + moved = [] + for number, name in enumerate(order, start=1): + win = next((w for w in windows(session) if (w.get("custom_name") or "") == name), None) + if win is None: + continue + if win.get("workspace") != number: + subprocess.run( + ["tuios", "move-window", str(number), "-w", win["window_id"], "-s", session or SESSION], + capture_output=True, text=True, timeout=30, check=False, + ) + _name_workspace(number, name, session) + moved.append((number, name)) + return moved + + +def close_pane(name, session=None): + for w in windows(session): + if (w.get("custom_name") or "") == name: + subprocess.run( + ["tuios", "close-window", w["window_id"], "-s", session or SESSION], + capture_output=True, text=True, timeout=30, check=False, + ) + return True + return False + + +def drop_empty_unnamed(session=None): + """Remove the unnamed, empty workspaces a rebuild leaves behind.""" + gone = 0 + for w in workspaces(session): + if "window_count" in w and (w.get("window_count") or 0) == 0 and not (w.get("name") or ""): + continue + return gone + + +if __name__ == "__main__": + if len(sys.argv) > 1 and sys.argv[1] == "claim": + print(claim(sys.argv[2])) + else: + print("usage: python3 lib/tuios.py claim NAME") diff --git a/bin/merge b/bin/merge new file mode 100755 index 0000000..508a8d3 --- /dev/null +++ b/bin/merge @@ -0,0 +1,136 @@ +#!/usr/bin/env python3 +"""Merge one task at a time, re-checked after the rebase. + + bin/merge AUD-163 # check, then merge + bin/merge AUD-163 --check # do every check, merge nothing + +The order is the point: + +1. take the merge lock, so two tasks cannot merge at once +2. rebase the task's branch onto the project's base branch +3. run the task's own verify command in the worktree +4. run ocr over the rebased range; a critical or high finding refuses the merge, + and so does a review that could not run +5. merge, then run the verify command again on the base branch +6. write the verdict on the task, so the record lives with the work + +A refusal is a decision for the person: the branch is left where it was. +""" + +import fcntl +import json +import os +import sys + +sys.path.insert(0, os.path.join(os.path.dirname(os.path.abspath(__file__)), "lib")) +import harness as h # noqa: E402 + +LOCK = "/tmp/paperclip-harness-merge.lock" + + +def main(): + args = [a for a in sys.argv[1:]] + check_only = "--check" in args + args = [a for a in args if not a.startswith("--")] + if len(args) != 1: + h.say("usage: bin/merge [--check]") + return 2 + + issue = h.find_issue(args[0]) + ref = issue["identifier"] + worktree, branch, base = h.worktree_of(issue) + base_repo = os.path.dirname(worktree) + # The worktree lives under /.paperclip/worktrees/; the base + # checkout is the project folder itself. + if ".paperclip/worktrees" in base_repo: + base_repo = base_repo.split("/.paperclip/worktrees")[0] + + h.say("%s %s" % (ref, issue["title"])) + h.say(" worktree %s" % worktree) + h.say(" branch %s onto %s" % (branch, base)) + + lock_file = open(LOCK, "w") + try: + fcntl.flock(lock_file, fcntl.LOCK_EX | fcntl.LOCK_NB) + except BlockingIOError: + raise h.Refused(6, "another merge is in progress; this one waits its turn") + + # 1. Rebase, so the checks run on what would actually land. + code, out = h.run(["git", "-C", worktree, "rebase", base]) + if code != 0: + h.run(["git", "-C", worktree, "rebase", "--abort"]) + raise h.Refused(3, "rebase onto %s conflicts; a person decides:\n%s" % (base, out[-800:])) + h.say(" rebased onto %s" % base) + + # 2. The task's own command decides whether it is finished. + cmd, source = h.verify_command(issue, worktree) + if cmd is None: + h.say(" no verify command: this would pass on the agent's word") + else: + code, out = h.run(["sh", "-c", cmd], cwd=worktree) + if code != 0: + h.verdict(issue, "## Merge refused: the verify command failed\n\n" + "`%s` (%s) exited %d in the task's worktree.\n\n```\n%s\n```" + % (cmd, source, code, out[-1500:])) + raise h.Refused(4, "verify command failed: %s" % cmd) + h.say(" verify ok: %s (%s)" % (cmd, source)) + + # 3. Review, on the rebased range. + report = h.ocr_review(worktree, base, "HEAD") + status = report.get("status") + reviewed = (report.get("summary") or {}).get("files_reviewed", 0) + if status == "skipped" and reviewed == 0: + # Nothing in the change is reviewable (a documentation-only diff, for + # one). That is not a failure, but it is not a review either, and the + # record has to say which. + h.say(" review saw nothing to review (0 files); recorded, not hidden") + elif status != "complete": + # "Could not run" is not a pass. Never merge what a check could not see. + h.verdict(issue, "## Merge refused: the review could not complete\n\n" + "ocr returned `%s` — %s" % (status, report.get("message") or "no reason given")) + raise h.Refused(5, "review could not complete (%s)" % status) + + findings = h.blocking_findings(report) + if findings: + body = "## Merge refused: the review found %d blocking item(s)\n\n%s" % ( + len(findings), "\n".join(h.describe(c) for c in findings)) + h.verdict(issue, body) + h.say(body) + raise h.Refused(5, "review blocked the merge") + + n = len(report.get("comments") or []) + if status == "complete": + h.say(" review ok: %d comment(s), none blocking" % n) + + if check_only: + h.say(" checked only; nothing merged") + return 0 + + # 4. Merge, then prove the base branch still passes the task's command. + code, out = h.run(["git", "-C", base_repo, "merge", "--no-ff", branch, + "-m", "Merge %s: %s" % (ref, issue["title"])]) + if code != 0: + raise h.Refused(8, "merge failed:\n%s" % out[-800:]) + + if cmd is not None: + code, out = h.run(["sh", "-c", cmd], cwd=base_repo) + if code != 0: + h.run(["git", "-C", base_repo, "reset", "--hard", "HEAD^"]) + h.verdict(issue, "## Merge undone: the verify command failed on %s\n\n" + "`%s` exited %d after merging, so the merge was rolled back.\n\n```\n%s\n```" + % (base, cmd, code, out[-1500:])) + raise h.Refused(4, "verify failed on %s after merging; rolled back" % base) + h.say(" verify ok on %s after the merge" % base) + + head = h.run(["git", "-C", base_repo, "rev-parse", "--short", "HEAD"])[1].strip() + review_line = ("%d comment(s), none blocking" % n) if status == "complete" \ + else "nothing reviewable in the change (0 files)" + h.verdict(issue, "## Merged\n\n- branch `%s` → `%s` at `%s`\n" + "- verify: %s\n- review: %s" + % (branch, base, head, ("`%s` ok" % cmd) if cmd else "none agreed", review_line)) + h.say(" merged into %s at %s" % (base, head)) + return 0 + + +if __name__ == "__main__": + h.main_guard(main) diff --git a/bin/office b/bin/office new file mode 100755 index 0000000..c00642e --- /dev/null +++ b/bin/office @@ -0,0 +1,111 @@ +#!/usr/bin/env python3 +"""Open a folder as a project: adopt it, remember it, build its pages. + + bin/office [--project ] [--session NAME] [--dry] + bin/office . # the folder you are in + +Adopting writes one file, /.harness/harness.toml, and nothing else. An +existing folder is never overwritten: what is already there stays, and only the +missing file is written. Paperclip's project record is read, never created — +bin/office adopts a project that exists, it does not invent one. + +The machine is remembered, so the flag is stated once. Running the same command +on a machine that is not the recorded one is refused rather than silently +producing a second copy of the same project. +""" + +import os +import sys + +HERE = os.path.dirname(os.path.abspath(__file__)) +sys.path.insert(0, os.path.join(HERE, "lib")) +import harness as h # noqa: E402 +import project as pj # noqa: E402 +import tuios # noqa: E402 + + +def parse(argv): + opts = {"folder": ".", "project": "", "session": "", "dry": "--dry" in argv, "host": ""} + args = [a for a in argv if not a.startswith("--")] + if args: + opts["folder"] = args[0] + for i, a in enumerate(argv): + if a == "--project" and i + 1 < len(argv): + opts["project"] = argv[i + 1] + if a == "--session" and i + 1 < len(argv): + opts["session"] = argv[i + 1] + if a == "--host" and i + 1 < len(argv): + opts["host"] = argv[i + 1] + return opts + + +def main(): + opts = parse(sys.argv[1:]) + folder = os.path.abspath(os.path.expanduser(opts["folder"])) + if not os.path.isdir(folder): + raise h.Refused(2, "no folder at %s" % folder) + + host = opts["host"] or pj.this_machine() + + # An adopted project already knows its session. Only a new one needs a + # default, and a default that names a session nobody made is worse than an + # error, so the session is checked before anything is written. + existing = {} + if pj.exists(folder): + existing = pj.load(folder) + session = ( + opts["session"] + or existing.get("session") + or os.path.basename(folder) + ) + tuios.require_session(session) + + if opts["host"] and opts["host"] not in (pj.this_machine(), pj.this_machine().split(".")[0]): + # Only a note, because the panes are built by the machine that owns them. + h.say("host: %s (build the pages there)" % host) + + ref = opts["project"] + if not ref and pj.exists(folder): + ref = pj.load(folder)["paperclip"]["project"] + if not ref: + raise h.Refused( + 2, + "which Paperclip project is %s? pass --project " % folder, + ) + + proj = pj.resolve_project(ref) + h.say("project: %s (%s)" % (proj["name"], proj["id"])) + h.say("folder: %s" % folder) + h.say("session: %s" % session) + h.say("machine: %s" % host) + + tasks = pj.active_tasks(proj["id"]) + watching = next((t["identifier"] for t in tasks if t["status"] not in ("done", "cancelled")), + tasks[0]["identifier"] if tasks else "") + + cfg = { + "session": session, + "paperclip": {"company": h.COMPANY, "project": proj["id"]}, + "machine": {"host": host, "folder": folder}, + "watching": watching, + } + + if opts["dry"]: + h.say("would write %s" % pj.path(folder)) + h.say("would build pages: shell, tree, reply, log, new") + return 0 + + pj.save(folder, cfg) + h.say("wrote %s" % pj.path(folder)) + + # The pages are built by this machine, from this folder, in this session. + code = os.spawnv( + os.P_WAIT, + sys.executable, + [sys.executable, os.path.join(HERE, "board"), "--folder", folder], + ) + return code + + +if __name__ == "__main__": + h.main_guard(main) diff --git a/bin/project-reset b/bin/project-reset new file mode 100755 index 0000000..b623090 --- /dev/null +++ b/bin/project-reset @@ -0,0 +1,109 @@ +#!/usr/bin/env python3 +"""Start a project again: close its pages, remove its worktrees, keep its work. + + bin/project-reset --dry-run + bin/project-reset --yes + bin/project-reset --yes --delete-folder + +What it removes, and nothing else: + + - the project's screen panes in its TUIOS session (the shell pane stays, so + the session survives) + - the git worktrees Paperclip made for the project's tasks, and the branches + those tasks were on + +What it leaves alone: the folder, its `.harness`, its git history, and every +Paperclip record — tasks, comments, runs, cost. Paperclip owns those, so a reset +here cannot erase the history of what happened; it only clears the machinery. + +`--delete-folder` is the only destructive option and it asks for the folder name +to be typed. Without it, nothing outside git and TUIOS is touched. +""" + +import os +import sys + +HERE = os.path.dirname(os.path.abspath(__file__)) +sys.path.insert(0, os.path.join(HERE, "lib")) +import harness as h # noqa: E402 +import tuios # noqa: E402 +import project as pj # noqa: E402 + +PAGES = ["tree", "reply", "log", "new"] + + +def main(): + argv = sys.argv[1:] + args = [a for a in argv if not a.startswith("--")] + if not args: + h.say("usage: bin/project-reset [--dry-run] [--yes] [--delete-folder]") + return 2 + folder = os.path.abspath(os.path.expanduser(args[0])) + dry = "--dry-run" in argv + yes = "--yes" in argv + delete_folder = "--delete-folder" in argv + + cfg = pj.load(folder) + pj.check_machine(cfg) + session = cfg["session"] + project_id = cfg["paperclip"]["project"] + + tasks = pj.active_tasks(project_id) + worktrees = [] + for t in tasks: + full = h.api("GET", "/api/issues/%s" % t["id"]) + ws = full.get("currentExecutionWorkspace") or {} + if ws.get("cwd"): + worktrees.append((t["identifier"], ws["cwd"], ws.get("branchName") or "")) + + h.say("project: %s" % project_id) + h.say("folder: %s" % folder) + h.say("session: %s" % session) + h.say("") + h.say("would remove:") + h.say(" panes: %s in %s" % (", ".join(PAGES), session)) + for ref, cwd, branch in worktrees: + h.say(" worktree: %s (%s, branch %s)" % (cwd, ref, branch or "none")) + if delete_folder: + h.say(" the folder: %s (including .harness and git history)" % folder) + h.say("kept: the Paperclip project, its tasks, comments, runs and cost") + + if dry: + h.say("") + h.say("dry run: nothing was removed") + return 0 + + if not yes: + h.say("") + h.say("Type the folder name to continue: %s" % os.path.basename(folder)) + typed = input("> ").strip() + if typed != os.path.basename(folder): + h.say("nothing was removed") + return 1 + + for name in PAGES: + if tuios.close_pane(name, session): + h.say("closed pane %s" % name) + + repo = folder + for ref, cwd, branch in worktrees: + code, out = h.run(["git", "-C", repo, "worktree", "remove", "--force", cwd]) + h.say("worktree %s: %s" % (ref, "removed" if code == 0 else out.strip()[:120])) + if branch: + code, out = h.run(["git", "-C", repo, "branch", "-D", branch]) + h.say("branch %s: %s" % (branch, "deleted" if code == 0 else out.strip()[:120])) + + if delete_folder: + h.say("") + h.say("Type the folder name again to delete %s: " % folder) + typed = input("> ").strip() + if typed != os.path.basename(folder): + h.say("folder kept") + return 0 + code, out = h.run(["rm", "-rf", folder]) + h.say("folder %s" % ("deleted" if code == 0 else "not deleted: " + out.strip())) + return 0 + + +if __name__ == "__main__": + h.main_guard(main) diff --git a/bin/review b/bin/review new file mode 100755 index 0000000..c033929 --- /dev/null +++ b/bin/review @@ -0,0 +1,70 @@ +#!/usr/bin/env python3 +"""Review one task's diff and print the verdict. Merge nothing. + + bin/review AUD-164 # review and print + bin/review AUD-164 --post # also write the verdict on the task + +This is the same review `bin/merge` runs, on its own, so a person can look +before the merge is attempted. Exit 0 means nothing blocking was found; exit 5 +means the review refused; exit 7 means there is no worktree to look at. +""" + +import os +import sys + +sys.path.insert(0, os.path.join(os.path.dirname(os.path.abspath(__file__)), "lib")) +import harness as h # noqa: E402 + + +def main(): + args = [a for a in sys.argv[1:] if not a.startswith("--")] + post = "--post" in sys.argv + if len(args) != 1: + h.say("usage: bin/review [--post]") + return 2 + + issue = h.find_issue(args[0]) + ref = issue["identifier"] + worktree, branch, base = h.worktree_of(issue) + + h.say("%s %s" % (ref, issue["title"])) + h.say(" worktree %s" % worktree) + h.say(" range %s...HEAD (%s)" % (base, branch)) + + report = h.ocr_review(worktree, base, "HEAD") + status = report.get("status") + reviewed = (report.get("summary") or {}).get("files_reviewed", 0) + comments = report.get("comments") or [] + + if status == "skipped" and reviewed == 0: + h.say(" nothing reviewable in the change (0 files)") + if post: + h.verdict(issue, "## Review: nothing to review\n\nocr selected 0 files " + "from the change, so nothing was checked.") + return 0 + if status != "complete": + h.say(" review could not complete: %s" % (report.get("message") or status)) + if post: + h.verdict(issue, "## Review could not run\n\nocr returned `%s` — %s" + % (status, report.get("message") or "no reason given")) + return 5 + + blocking = h.blocking_findings(report) + for c in comments: + marker = "BLOCKS" if c in blocking else "note " + h.say(" %s %s" % (marker, h.describe(c))) + + verdict = ("%d comment(s); %d blocking" % (len(comments), len(blocking))) + h.say(" %s" % verdict) + + if post: + body = "## Review\n\n%s" % (verdict) + if blocking: + body += "\n\nBlocking:\n" + "\n".join(h.describe(c) for c in blocking) + h.verdict(issue, body) + + return 5 if blocking else 0 + + +if __name__ == "__main__": + h.main_guard(main) diff --git a/docs/PAPERCLIP-API-NOTES.md b/docs/PAPERCLIP-API-NOTES.md index e67bc28..e5a25d7 100644 --- a/docs/PAPERCLIP-API-NOTES.md +++ b/docs/PAPERCLIP-API-NOTES.md @@ -390,3 +390,87 @@ status, and our screens should show that second run rather than hide it. under the board identity by mistake". So `authorType: user` is not proof that the person typed it. Screen 4 currently prints `user` as "you"; that label is wrong often enough to matter, and it needs a better rule than the author type alone. + +--- + +# Milestones 4 and 5, measured, 2026-11-10 + +## The project layer + +`bin/office ` writes exactly one file, `/.harness/harness.toml`: + +```toml +session = "paperclip-harness" +watching = "AUD-163" + +[paperclip] +company = "d4921013-e5da-4d88-8016-57d5746e2d03" +project = "17adbb47-f966-4a82-a8ef-e48c941443ea" + +[machine] +host = "nixos-desktop" +folder = "/tmp/pc-sandbox-2026-11-10" +``` + +That is the whole of what we store: which folder, on which machine, in which session, +showing which task. Everything else is read from Paperclip. + +Measured, in this order, on one project: + +| Step | Result | +|---|---| +| `bin/office` | wrote the file, built five panes on workspaces 1–5 | +| run again | five panes adopted, none created | +| `bin/board --replace` | five closed, five created, then placed 1–5 in order | +| `bin/project-reset --dry-run` | listed the panes, three worktrees and three branches | +| `bin/project-reset --yes` | removed them; `git worktree list` left only the main checkout | +| `bin/office` afterwards | five panes built again from nothing | + +Two failures found by doing it: + +- **A rebuild scatters the workspace order.** Closing panes renumbers the rest, so the + pages landed as shell 2, tree 3, reply 4, log 5, new 1. The fix is a final pass that + moves each pane once to the workspace it belongs on (`tuios move-window`) and names it + again. Without that pass the rail order is meaningless. +- **Adopting a project must not invent a session.** The first version defaulted the session + to the folder name, wrote that into the file, then failed because no such session + existed — and the good session name was already overwritten. Sessions are made by a + person, so `bin/office` now reads the recorded session first, checks it exists, and + refuses with the exact `tuios new` command when it does not. + +## The merge gate + +`bin/merge` takes a lock (`flock` on `/tmp/paperclip-harness-merge.lock`), rebases the +task's branch onto the base, runs the task's own verify command, runs `ocr` over the +rebased range, and only then merges — then runs the verify command again on the base +branch and rolls the merge back if it fails. + +Every branch of that path was exercised, not reasoned about: + +| Case | What happened | +|---|---| +| clean task (`AUD-163`, docs only) | merged at `2899981`; `ocr` reported `skipped`, 0 files, and said so instead of pretending to have reviewed | +| verify command fails | exit 4, task left alone, comment written | +| review finds a critical | `AUD-165` added `cleanup.sh` with `rm -rf "$HOME/tmp"`; exit 5, finding quoted, `cleanup.sh` never reached `main` | +| review cannot run | exit 5, refused | +| another merge holds the lock | exit 6, refused immediately | + +`ocr`'s JSON has `status`, `summary.files_reviewed` and `comments[]` with `severity` +(`critical`, `high`, …), `path`, `start_line` and `content`. Blocking severities are +`critical` and `high`; `skipped` with zero files is "nothing to review", recorded +plainly, while any other non-`complete` status refuses. + +## Two API behaviours that shape the code + +**A comment on an assigned task wakes its agent.** The merge tool's first refusal comment +went onto a task still assigned to `Builder`, and `Builder` immediately woke and took the +task back to `in_progress` — paying for a run to read a message meant for a person. So a +tool's verdict is written only after the task is handed back: +`PATCH {"assigneeAgentId": null, "assigneeUserId": ""}`. A person's reply +from screen 4 is the opposite case: there the wake is the point. + +**The issue list is a summary.** `GET /api/companies/{id}/issues` carries no +`currentExecutionWorkspace` and no blockers; the single-task read has both. Any code that +needs a worktree path, a branch or a blocker must do a second `GET /api/issues/{id}`. +That is the third time this list-versus-record difference has bitten, after blockers and +after the earlier probe.