sam/paperclip-harness
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Wrap overlay text instead of cutting it; the project check is a regression check, renamed check.sh; concrete Headroom steps; Mouse-1 measured

Browse Source
This commit is contained in:
Sam
2026-11-11 16:35:44 +11:00
parent c50c1efa22
commit 70e1f55486
7 changed files with 150 additions and 56 deletions
Download Patch File Download Diff File Expand all files Collapse all files

BIN
bin/harness
View File

Binary file not shown.

8
bin/lib/harness.py
View File

@@ -126,9 +126,11 @@ def verify_command(issue, worktree):
m = VERIFY_RE.search(issue.get("description") or "")
if m:
return m.group("cmd").strip(), "from the task description"
project_script = os.path.join(worktree, ".harness", "verify.sh")
if os.path.isfile(project_script):
return ".harness/verify.sh", "the project's .harness/verify.sh"
# The project's own check. check.sh is the current name; verify.sh is what
# the earlier project wrote, and is still honoured.
for name in (".harness/check.sh", ".harness/verify.sh"):
if os.path.isfile(os.path.join(worktree, name)):
return name, "the project's " + name
return None, "none"

24
bin/office
View File

@@ -49,23 +49,27 @@ def scaffold(folder, name):
harness = os.path.join(folder, ".harness")
os.makedirs(harness, exist_ok=True)
verify = os.path.join(harness, "verify.sh")
if not os.path.exists(verify):
with open(verify, "w") as f:
check = os.path.join(harness, "check.sh")
if not os.path.exists(check):
with open(check, "w") as f:
f.write("""#!/bin/sh
# The command that decides a task in this project is finished.
# This project's own check, run by bin/merge before and after it puts a task's
# work on the base branch. Exit 0 means "the project still passes".
#
# Leave it as it is to accept that there is no agreed command: bin/merge will
# say so and let the review decide. Replace the body with the real one when the
# project has tests, for example:
# It does not decide that a task is finished: an agent sets a task's status, and
# the review decides whether the change is good. This is the regression check,
# the same thing a CI pipeline would run.
#
# Leave it as it is while the project has no tests. bin/merge will say there is
# no check and let the review decide. Replace the body when tests exist:
#
# exec go test ./...
# exec make test
echo "no verify command agreed for this project"
echo "this project has no check yet"
exit 0
""")
os.chmod(verify, 0o755)
written.append(verify)
os.chmod(check, 0o755)
written.append(check)
notes = os.path.join(harness, "NOTES.md")
if not os.path.exists(notes):

18
docs/NEXT.md
View File

@@ -104,9 +104,21 @@ step; a switch is only worth adding after one agent has been run both ways and m
## 9. MOUSE-1
Corrected: Mouse-1 is a hosted API, not software to install. `MOUSE_API_KEY` is present; what is
missing is a client call from our side. The old project's policy stands: triage only, never a
gate, never a permit — it could order `Board>needs`, never decide a merge.
It is a hosted API, not software: `POST https://api.mouse.dev/v1/decisions` with the key already
in the environment. Measured facts are in `docs/MOUSE-1-NOTES.md` and summarised in
`docs/PLAN-INTERFACE.md`: cold start of minutes, ~490 ms warm, stable answers at high
confidence, and four proven uses — failure routing first among them.
The one place it would earn its keep here: a failed run in `Agents>agents` offers `f`, which
asks "what does this failure need" with four options and records the answer, the confidence and
the options on the task. A person still decides, and it never decides a merge.
## Done: Headroom
A provider entry in `~/.pi/agent/settings.json` pointing at `http://192.168.20.13:8787/v1`, and
an agent's model set to `headroom/<model>`. Per agent, because the model is per agent; per
project is not possible. The exact steps and the one unknown to measure are in
`docs/PLAN-INTERFACE.md`.
## Unproven, and worth a test

106
docs/PLAN-INTERFACE.md
View File

@@ -576,48 +576,92 @@ A transcript tab inside `Agents>agents` is the integration option: read the JSON
messages, no second process. Worth doing only if the file browser feels like a detour after a
week of use. Recommendation: do the cheap one now, decide later.
## verify.sh, plainly
## The project's check: `.harness/check.sh`
`.harness/verify.sh` is a shell script in the project folder. Its only job is to answer one
question: **is this project's work finished?** It exits 0 when the answer is yes.
This is a script in the project folder that exits 0 when the project still passes its own
tests. It answers "did this change break the project", not "is this task finished".
- It is **not** Paperclip's. Paperclip has no such concept; an agent sets a task's status.
- It is **our** convention, from the old project, and `bin/office` now writes a template:
**What decides a task is finished**, in order of authority:
```sh
#!/bin/sh
echo "no verify command agreed for this project"
exit 0
```
1. the agent, by setting the task's status — this is Paperclip's own model and nothing in our
software can overrule it;
2. a reviewer, if a review stage is configured on the task (Paperclip's execution policy);
3. `ocr`, at merge time, which can refuse the change.
As it stands it says yes to everything, which is honest: there is no agreed check. Replace
the body with the project's real tests when they exist:
The project's check is a **regression check** at the same point as a CI pipeline: `bin/merge`
runs it in the task's worktree before merging, and again on the base branch after merging, and
rolls the merge back if it fails. It never decides that a task is finished.
```sh
exec go test ./...
```
Why keep it at all: an agent says it is done, a review says the diff looks right, and neither
proves the build still works. One command does. If a project has no check, `bin/merge` says so
and the review decides alone — that is the current state of this sandbox.
- `bin/merge` runs it in the task's worktree, and again on the base branch after merging.
- A task may carry its own command in its description; that one wins for that task only.
Naming: `check.sh` is the current name. `.harness/verify.sh` was the earlier project's name and
is still honoured if present, so an adopted folder keeps working. `bin/office` writes
`check.sh` with a comment explaining what to replace.
So: a per-project default, a per-task override, both optional. If neither exists, the merge
says so and lets the `ocr` review decide.
## Headroom, concretely
## Headroom, and whether it is good for a project
Headroom is a proxy that accepts the same HTTP API as the model router and shortens long
prompts before forwarding. Measured here:
Headroom is a **compression proxy**: requests pass through it and long prompts are shortened
before they reach the model. It is running here at `192.168.20.13:8787` (v0.24.0, healthy,
memory backend disabled, compression on).
```
GET http://192.168.20.13:8787/v1/models → 200, 486 models (auto/best-coding, deepseek/…)
GET http://192.168.20.13:8787/health → 200, healthy, compression on, memory off
upstream (config `openai_api_url`) → http://192.168.20.13:20129/v1
```
It is good where prompts are long and repetitive — a big repository, a long thread, many tool
results. It is pointless for a small task whose prompt is already short, and it adds a hop and a
failure mode.
The agents' provider `omni` already points at `http://192.168.20.13:20129/v1`, which is the
same upstream headroom forwards to. So headroom can sit in front of it.
The lever is per **agent**, not per project, because a Paperclip agent's provider is configured
on the agent. `adapterConfig.env` accepts variables, so an agent can be pointed at the proxy
without touching anything else. That is why the interface offers a read-out first: until one
agent has run both ways and you have compared cost and quality, "good for some projects" stays
a hypothesis rather than a setting.
**How to use it for one agent.** Two steps, and only the second is per agent:
1. Add a provider to `~/.pi/agent/settings.json` (global, once):
```json
"headroom": {
"baseUrl": "http://192.168.20.13:8787/v1",
"apiKey": "<the same key the omni provider uses>",
"api": "openai-completions"
}
```
2. Set that agent's model in `Agents>settings` to `headroom/auto/best-coding` (or any model id
from the list above). Paperclip stores the model per agent, so exactly one agent changes.
**Would it work.** The endpoint answers the OpenAI-style API that Pi's `openai-completions`
provider speaks, and lists the models Pi would ask for, so the pieces line up. It is still a
hypothesis until one run is made through it: the unknowns are whether the proxy passes Pi's
tool-calling requests unchanged, and what compression does to a prompt with large tool results.
The measurement is one run: compare cost, duration and result quality for the same task with and
without the provider.
**Per project is not possible.** The provider is chosen by the agent's model, and agents belong
to the company, not to a project. The finest granularity is one agent, which is what the
`Agents>settings` model field already gives.
## Mouse-1, measured
The person supplied `docs/MOUSE-1-NOTES.md`, written from measurements on 2026-11-08/09. The
facts that shape the design:
- `POST https://api.mouse.dev/v1/decisions`, `Authorization: Bearer $MOUSE_API_KEY`, body
`{model, state, questions:{id:{type:"choice", instructions, criteria}}}`; the answer returns
`choice`, `confidence`, `probabilities`.
- **Cold start is minutes.** The first call after idle returned `503 engine_timeout` after 30 s;
the retry returned 200 in 13 s; warm calls were 1044 ms then ~489 ms.
- The same state and options gave the same choice at ~0.98 confidence. It behaves as a
classifier, not a writer.
- Proven useful: **failure routing** (redo / stronger model / environment / person), unblock
triage, intake clarity, task sizing. Measured in the old project as `bin/route-failure`.
- Ruled out: the merge decision, judging test results, anything on a keystroke path, vague
options, and being the only record of a decision.
Where it would go in this interface, if added: `Agents>agents` → a failure on a run offers
`f`, which asks Mouse-1 one question with four options and writes the recommendation, the
confidence and the options as a comment on the task. A person still decides. Expected cost of
the first call after idle is 30 seconds, so `f` must show "asking, this can take half a minute"
rather than appear to hang.
## Mouse-1, corrected

44
tui/frame.go
View File

@@ -278,15 +278,47 @@ func wrap(s string, w int) []string {
// screen. It replaces the whole view rather than floating over it, so nothing
// behind it can be misread as still active.
func helpOverlay(title string, entries [][2]string, w, h int) string {
boxW := min(64, max(30, w-8))
boxH := min(len(entries)+4, max(8, h-4))
lines := []string{""}
// The box is as wide as the screen allows, and every line of text is wrapped
// inside it. Cutting a sentence in half hides the reason a key exists.
boxW := min(92, max(40, w-6))
if boxW > w {
boxW = w
}
keyCol := 16
inner := boxW - 4 - keyCol
if inner < 20 {
keyCol = 8
inner = boxW - 4 - keyCol
}
var lines []string
lines = append(lines, "")
for _, e := range entries {
key := lipgloss.NewStyle().Foreground(lipgloss.Color(active.Accent)).Bold(true).
Width(16).Render(e[0])
lines = append(lines, " "+key+lipgloss.NewStyle().Foreground(lipgloss.Color(active.Text)).Render(e[1]))
key := e[0]
if key == "" {
for _, cont := range wrap(e[1], max(12, inner)) {
lines = append(lines, " "+strings.Repeat(" ", keyCol)+
lipgloss.NewStyle().Foreground(lipgloss.Color(active.Text)).Render(cont))
}
continue
}
wrapped := wrap(e[1], max(12, inner))
styled := lipgloss.NewStyle().Foreground(lipgloss.Color(active.Accent)).Bold(true).
Render(padTo(key, keyCol))
text := lipgloss.NewStyle().Foreground(lipgloss.Color(active.Text)).Render(wrapped[0])
lines = append(lines, " "+styled+text)
for _, cont := range wrapped[1:] {
lines = append(lines, " "+strings.Repeat(" ", keyCol)+
lipgloss.NewStyle().Foreground(lipgloss.Color(active.Text)).Render(cont))
}
}
lines = append(lines, "", dimStyle().Render(" esc or ? closes this"))
boxH := len(lines) + 2
if boxH > h-2 {
boxH = h - 2
lines = lines[:max(1, boxH-2)]
}
p := panel{title: title, width: boxW, height: boxH, focused: true}
return lipgloss.Place(w, h, lipgloss.Center, lipgloss.Center, p.render(lines))
}

6
tui/work.go
View File

@@ -467,9 +467,9 @@ func (m workModel) View() string {
}
if m.confirm {
verify := "no verify command: the review decides"
check := "no project check: the review decides"
if cmd := verifyFromDescription(m.task.Description); cmd != "" {
verify = "verify command: " + cmd
check = "project check: " + cmd
}
branch := "(no branch yet)"
if m.task.Workspace != nil && m.task.Workspace.Branch != "" {
@@ -478,7 +478,7 @@ func (m workModel) View() string {
return helpOverlay("merge "+m.task.Identifier+" into main?", [][2]string{
{"what is merged", branch + " → " + baseRefOf(m.task)},
{"what happens", "rebase the branch onto the base, then merge it"},
{"", verify},
{"", check},
{"", "ocr reviews the change; a critical finding refuses the merge"},
{"", "merge, then re-run the checks on the base branch"},
{"y", "do it"},