Wrap overlay text instead of cutting it; the project check is a regression check, renamed check.sh; concrete Headroom steps; Mouse-1 measured
This commit is contained in:
@@ -576,48 +576,92 @@ A transcript tab inside `Agents>agents` is the integration option: read the JSON
|
||||
messages, no second process. Worth doing only if the file browser feels like a detour after a
|
||||
week of use. Recommendation: do the cheap one now, decide later.
|
||||
|
||||
## verify.sh, plainly
|
||||
## The project's check: `.harness/check.sh`
|
||||
|
||||
`.harness/verify.sh` is a shell script in the project folder. Its only job is to answer one
|
||||
question: **is this project's work finished?** It exits 0 when the answer is yes.
|
||||
This is a script in the project folder that exits 0 when the project still passes its own
|
||||
tests. It answers "did this change break the project", not "is this task finished".
|
||||
|
||||
- It is **not** Paperclip's. Paperclip has no such concept; an agent sets a task's status.
|
||||
- It is **our** convention, from the old project, and `bin/office` now writes a template:
|
||||
**What decides a task is finished**, in order of authority:
|
||||
|
||||
```sh
|
||||
#!/bin/sh
|
||||
echo "no verify command agreed for this project"
|
||||
exit 0
|
||||
```
|
||||
1. the agent, by setting the task's status — this is Paperclip's own model and nothing in our
|
||||
software can overrule it;
|
||||
2. a reviewer, if a review stage is configured on the task (Paperclip's execution policy);
|
||||
3. `ocr`, at merge time, which can refuse the change.
|
||||
|
||||
As it stands it says yes to everything, which is honest: there is no agreed check. Replace
|
||||
the body with the project's real tests when they exist:
|
||||
The project's check is a **regression check** at the same point as a CI pipeline: `bin/merge`
|
||||
runs it in the task's worktree before merging, and again on the base branch after merging, and
|
||||
rolls the merge back if it fails. It never decides that a task is finished.
|
||||
|
||||
```sh
|
||||
exec go test ./...
|
||||
```
|
||||
Why keep it at all: an agent says it is done, a review says the diff looks right, and neither
|
||||
proves the build still works. One command does. If a project has no check, `bin/merge` says so
|
||||
and the review decides alone — that is the current state of this sandbox.
|
||||
|
||||
- `bin/merge` runs it in the task's worktree, and again on the base branch after merging.
|
||||
- A task may carry its own command in its description; that one wins for that task only.
|
||||
Naming: `check.sh` is the current name. `.harness/verify.sh` was the earlier project's name and
|
||||
is still honoured if present, so an adopted folder keeps working. `bin/office` writes
|
||||
`check.sh` with a comment explaining what to replace.
|
||||
|
||||
So: a per-project default, a per-task override, both optional. If neither exists, the merge
|
||||
says so and lets the `ocr` review decide.
|
||||
## Headroom, concretely
|
||||
|
||||
## Headroom, and whether it is good for a project
|
||||
Headroom is a proxy that accepts the same HTTP API as the model router and shortens long
|
||||
prompts before forwarding. Measured here:
|
||||
|
||||
Headroom is a **compression proxy**: requests pass through it and long prompts are shortened
|
||||
before they reach the model. It is running here at `192.168.20.13:8787` (v0.24.0, healthy,
|
||||
memory backend disabled, compression on).
|
||||
```
|
||||
GET http://192.168.20.13:8787/v1/models → 200, 486 models (auto/best-coding, deepseek/…)
|
||||
GET http://192.168.20.13:8787/health → 200, healthy, compression on, memory off
|
||||
upstream (config `openai_api_url`) → http://192.168.20.13:20129/v1
|
||||
```
|
||||
|
||||
It is good where prompts are long and repetitive — a big repository, a long thread, many tool
|
||||
results. It is pointless for a small task whose prompt is already short, and it adds a hop and a
|
||||
failure mode.
|
||||
The agents' provider `omni` already points at `http://192.168.20.13:20129/v1`, which is the
|
||||
same upstream headroom forwards to. So headroom can sit in front of it.
|
||||
|
||||
The lever is per **agent**, not per project, because a Paperclip agent's provider is configured
|
||||
on the agent. `adapterConfig.env` accepts variables, so an agent can be pointed at the proxy
|
||||
without touching anything else. That is why the interface offers a read-out first: until one
|
||||
agent has run both ways and you have compared cost and quality, "good for some projects" stays
|
||||
a hypothesis rather than a setting.
|
||||
**How to use it for one agent.** Two steps, and only the second is per agent:
|
||||
|
||||
1. Add a provider to `~/.pi/agent/settings.json` (global, once):
|
||||
|
||||
```json
|
||||
"headroom": {
|
||||
"baseUrl": "http://192.168.20.13:8787/v1",
|
||||
"apiKey": "<the same key the omni provider uses>",
|
||||
"api": "openai-completions"
|
||||
}
|
||||
```
|
||||
|
||||
2. Set that agent's model in `Agents>settings` to `headroom/auto/best-coding` (or any model id
|
||||
from the list above). Paperclip stores the model per agent, so exactly one agent changes.
|
||||
|
||||
**Would it work.** The endpoint answers the OpenAI-style API that Pi's `openai-completions`
|
||||
provider speaks, and lists the models Pi would ask for, so the pieces line up. It is still a
|
||||
hypothesis until one run is made through it: the unknowns are whether the proxy passes Pi's
|
||||
tool-calling requests unchanged, and what compression does to a prompt with large tool results.
|
||||
The measurement is one run: compare cost, duration and result quality for the same task with and
|
||||
without the provider.
|
||||
|
||||
**Per project is not possible.** The provider is chosen by the agent's model, and agents belong
|
||||
to the company, not to a project. The finest granularity is one agent, which is what the
|
||||
`Agents>settings` model field already gives.
|
||||
|
||||
## Mouse-1, measured
|
||||
|
||||
The person supplied `docs/MOUSE-1-NOTES.md`, written from measurements on 2026-11-08/09. The
|
||||
facts that shape the design:
|
||||
|
||||
- `POST https://api.mouse.dev/v1/decisions`, `Authorization: Bearer $MOUSE_API_KEY`, body
|
||||
`{model, state, questions:{id:{type:"choice", instructions, criteria}}}`; the answer returns
|
||||
`choice`, `confidence`, `probabilities`.
|
||||
- **Cold start is minutes.** The first call after idle returned `503 engine_timeout` after 30 s;
|
||||
the retry returned 200 in 13 s; warm calls were 1044 ms then ~489 ms.
|
||||
- The same state and options gave the same choice at ~0.98 confidence. It behaves as a
|
||||
classifier, not a writer.
|
||||
- Proven useful: **failure routing** (redo / stronger model / environment / person), unblock
|
||||
triage, intake clarity, task sizing. Measured in the old project as `bin/route-failure`.
|
||||
- Ruled out: the merge decision, judging test results, anything on a keystroke path, vague
|
||||
options, and being the only record of a decision.
|
||||
|
||||
Where it would go in this interface, if added: `Agents>agents` → a failure on a run offers
|
||||
`f`, which asks Mouse-1 one question with four options and writes the recommendation, the
|
||||
confidence and the options as a comment on the task. A person still decides. Expected cost of
|
||||
the first call after idle is 30 seconds, so `f` must show "asking, this can take half a minute"
|
||||
rather than appear to hang.
|
||||
|
||||
## Mouse-1, corrected
|
||||
|
||||
|
||||
Reference in New Issue
Block a user