Wrap overlay text instead of cutting it; the project check is a regression check, renamed check.sh; concrete Headroom steps; Mouse-1 measured

This commit is contained in:
Sam
2026-11-11 16:35:44 +11:00
parent c50c1efa22
commit 70e1f55486
7 changed files with 150 additions and 56 deletions

View File

@@ -104,9 +104,21 @@ step; a switch is only worth adding after one agent has been run both ways and m
## 9. MOUSE-1
Corrected: Mouse-1 is a hosted API, not software to install. `MOUSE_API_KEY` is present; what is
missing is a client call from our side. The old project's policy stands: triage only, never a
gate, never a permit — it could order `Board>needs`, never decide a merge.
It is a hosted API, not software: `POST https://api.mouse.dev/v1/decisions` with the key already
in the environment. Measured facts are in `docs/MOUSE-1-NOTES.md` and summarised in
`docs/PLAN-INTERFACE.md`: cold start of minutes, ~490 ms warm, stable answers at high
confidence, and four proven uses — failure routing first among them.
The one place it would earn its keep here: a failed run in `Agents>agents` offers `f`, which
asks "what does this failure need" with four options and records the answer, the confidence and
the options on the task. A person still decides, and it never decides a merge.
## Done: Headroom
A provider entry in `~/.pi/agent/settings.json` pointing at `http://192.168.20.13:8787/v1`, and
an agent's model set to `headroom/<model>`. Per agent, because the model is per agent; per
project is not possible. The exact steps and the one unknown to measure are in
`docs/PLAN-INTERFACE.md`.
## Unproven, and worth a test

View File

@@ -576,48 +576,92 @@ A transcript tab inside `Agents>agents` is the integration option: read the JSON
messages, no second process. Worth doing only if the file browser feels like a detour after a
week of use. Recommendation: do the cheap one now, decide later.
## verify.sh, plainly
## The project's check: `.harness/check.sh`
`.harness/verify.sh` is a shell script in the project folder. Its only job is to answer one
question: **is this project's work finished?** It exits 0 when the answer is yes.
This is a script in the project folder that exits 0 when the project still passes its own
tests. It answers "did this change break the project", not "is this task finished".
- It is **not** Paperclip's. Paperclip has no such concept; an agent sets a task's status.
- It is **our** convention, from the old project, and `bin/office` now writes a template:
**What decides a task is finished**, in order of authority:
```sh
#!/bin/sh
echo "no verify command agreed for this project"
exit 0
```
1. the agent, by setting the task's status — this is Paperclip's own model and nothing in our
software can overrule it;
2. a reviewer, if a review stage is configured on the task (Paperclip's execution policy);
3. `ocr`, at merge time, which can refuse the change.
As it stands it says yes to everything, which is honest: there is no agreed check. Replace
the body with the project's real tests when they exist:
The project's check is a **regression check** at the same point as a CI pipeline: `bin/merge`
runs it in the task's worktree before merging, and again on the base branch after merging, and
rolls the merge back if it fails. It never decides that a task is finished.
```sh
exec go test ./...
```
Why keep it at all: an agent says it is done, a review says the diff looks right, and neither
proves the build still works. One command does. If a project has no check, `bin/merge` says so
and the review decides alone — that is the current state of this sandbox.
- `bin/merge` runs it in the task's worktree, and again on the base branch after merging.
- A task may carry its own command in its description; that one wins for that task only.
Naming: `check.sh` is the current name. `.harness/verify.sh` was the earlier project's name and
is still honoured if present, so an adopted folder keeps working. `bin/office` writes
`check.sh` with a comment explaining what to replace.
So: a per-project default, a per-task override, both optional. If neither exists, the merge
says so and lets the `ocr` review decide.
## Headroom, concretely
## Headroom, and whether it is good for a project
Headroom is a proxy that accepts the same HTTP API as the model router and shortens long
prompts before forwarding. Measured here:
Headroom is a **compression proxy**: requests pass through it and long prompts are shortened
before they reach the model. It is running here at `192.168.20.13:8787` (v0.24.0, healthy,
memory backend disabled, compression on).
```
GET http://192.168.20.13:8787/v1/models → 200, 486 models (auto/best-coding, deepseek/…)
GET http://192.168.20.13:8787/health → 200, healthy, compression on, memory off
upstream (config `openai_api_url`) → http://192.168.20.13:20129/v1
```
It is good where prompts are long and repetitive — a big repository, a long thread, many tool
results. It is pointless for a small task whose prompt is already short, and it adds a hop and a
failure mode.
The agents' provider `omni` already points at `http://192.168.20.13:20129/v1`, which is the
same upstream headroom forwards to. So headroom can sit in front of it.
The lever is per **agent**, not per project, because a Paperclip agent's provider is configured
on the agent. `adapterConfig.env` accepts variables, so an agent can be pointed at the proxy
without touching anything else. That is why the interface offers a read-out first: until one
agent has run both ways and you have compared cost and quality, "good for some projects" stays
a hypothesis rather than a setting.
**How to use it for one agent.** Two steps, and only the second is per agent:
1. Add a provider to `~/.pi/agent/settings.json` (global, once):
```json
"headroom": {
"baseUrl": "http://192.168.20.13:8787/v1",
"apiKey": "<the same key the omni provider uses>",
"api": "openai-completions"
}
```
2. Set that agent's model in `Agents>settings` to `headroom/auto/best-coding` (or any model id
from the list above). Paperclip stores the model per agent, so exactly one agent changes.
**Would it work.** The endpoint answers the OpenAI-style API that Pi's `openai-completions`
provider speaks, and lists the models Pi would ask for, so the pieces line up. It is still a
hypothesis until one run is made through it: the unknowns are whether the proxy passes Pi's
tool-calling requests unchanged, and what compression does to a prompt with large tool results.
The measurement is one run: compare cost, duration and result quality for the same task with and
without the provider.
**Per project is not possible.** The provider is chosen by the agent's model, and agents belong
to the company, not to a project. The finest granularity is one agent, which is what the
`Agents>settings` model field already gives.
## Mouse-1, measured
The person supplied `docs/MOUSE-1-NOTES.md`, written from measurements on 2026-11-08/09. The
facts that shape the design:
- `POST https://api.mouse.dev/v1/decisions`, `Authorization: Bearer $MOUSE_API_KEY`, body
`{model, state, questions:{id:{type:"choice", instructions, criteria}}}`; the answer returns
`choice`, `confidence`, `probabilities`.
- **Cold start is minutes.** The first call after idle returned `503 engine_timeout` after 30 s;
the retry returned 200 in 13 s; warm calls were 1044 ms then ~489 ms.
- The same state and options gave the same choice at ~0.98 confidence. It behaves as a
classifier, not a writer.
- Proven useful: **failure routing** (redo / stronger model / environment / person), unblock
triage, intake clarity, task sizing. Measured in the old project as `bin/route-failure`.
- Ruled out: the merge decision, judging test results, anything on a keystroke path, vague
options, and being the only record of a decision.
Where it would go in this interface, if added: `Agents>agents` → a failure on a run offers
`f`, which asks Mouse-1 one question with four options and writes the recommendation, the
confidence and the options as a comment on the task. A person still decides. Expected cost of
the first call after idle is 30 seconds, so `f` must show "asking, this can take half a minute"
rather than appear to hang.
## Mouse-1, corrected