clean tests run: pi 1.5.0 model-list shim, panes checked by process, board r refreshes detail, office . accepted
This commit is contained in:
@@ -127,3 +127,102 @@ only `README.md`, and cancelling removes it from the board without touching `mai
|
||||
|
||||
For each step: what you pressed, what appeared, and anything that did not match. That is worth
|
||||
more than a verdict, because it says where the interface and Paperclip disagree.
|
||||
|
||||
---
|
||||
|
||||
# Results, 2026-11-12
|
||||
|
||||
Both tests were run on `.13`, driven with `tuios send-text`, `tuios send-keys` and
|
||||
`tuios capture-pane`, so every step below is what the screen actually showed.
|
||||
|
||||
## Test 1 — passed, after one real defect was found and fixed
|
||||
|
||||
| Step | What happened |
|
||||
|---|---|
|
||||
| folder + `git init` | ok |
|
||||
| `office --new "test-alpha"` | project, session and eight panes created |
|
||||
| `New task>new` → **AUD-172** | created and handed to Builder; the Assign field takes the word `yes` |
|
||||
| first three runs | **all failed in 4–5 s**, `errorCode: adapter_failed`, `logBytes: 0` |
|
||||
| the fix | see "Pi 1.5.0 and the model list" below |
|
||||
| the run after the fix | **succeeded in 69 s** |
|
||||
| `Work>change` | worktree, branch, base, commits, `STATUS.md \| 1 +`, and the diff |
|
||||
| `Work>review` (`v`) | `ocr` **selected 0 files** — it does not review Markdown |
|
||||
| `Work>merge` (`m`, `y`) | rebased, no verify command, review recorded as "nothing to review", `merged into main at 36acb8d` |
|
||||
| `main` | `STATUS.md` present, contents `alpha is running.` |
|
||||
|
||||
**What did not work as the test expected**
|
||||
|
||||
- **Cost is always zero.** `Project>costs` and `Board>the chosen task` report `0 cent(s)` and
|
||||
`0 tokens out` over four runs. `usageJson` on the run is `null`: the `pi_local` adapter does not
|
||||
capture token usage from Pi, so cost cannot be shown for these runs. Open item.
|
||||
- **`ocr` reviews code, not Markdown.** A documentation-only change gets `status: skipped`,
|
||||
`selected: []`. `bin/merge` already treats this as "nothing to review, recorded, not hidden",
|
||||
so it does not block a merge — but a reviewer reading the tab sees "nothing reviewable" while
|
||||
the `change` tab lists one file.
|
||||
|
||||
## Test 2 — passed, after one real defect was found and fixed
|
||||
|
||||
| Step | What happened |
|
||||
|---|---|
|
||||
| `office . --project harness-repo --session harness-repo` | adopted the project and the panes; `.harness/` was not rewritten |
|
||||
| panes after the reboot | **restored shells**, and `office` called them "adopted" |
|
||||
| the fix | a pane is adopted only if its own process is the harness |
|
||||
| `New task>new` → **AUD-173** | created and handed to Builder; the run succeeded in 24 s |
|
||||
| the diff | `README.md \| 1 +`, nothing else, committed on the branch |
|
||||
| merge | **not merged on purpose** |
|
||||
| `c`, then `y` in `Board>board` | task `cancelled`; status becomes cancelled, thread and runs stay |
|
||||
| `main` | untouched — `README.md` has no new line, last commit still `b81cb91` |
|
||||
|
||||
**What did not work as the test expected**
|
||||
|
||||
- **After a reboot, every pane is a bare shell and the pane keeps its name.** `board` matched the
|
||||
panes by name and reported them "adopted" while nothing was running in them. A page that looks
|
||||
built but does nothing is the worst kind of wrong. Now `open_pane` reads
|
||||
`/proc/<pid>/cmdline` and adopts a pane only when the harness is what is running; otherwise it
|
||||
closes the pane and opens a fresh one. `shell` is exempt: it is a shell either way.
|
||||
- **`Board>board` `r` refreshed the task list but not the selected task's detail**, so
|
||||
"last run running" stayed on screen minutes after the run finished. `r` now reloads both.
|
||||
- A cancelled task is grouped under **Done** and its row reads `gone`. That is deliberate
|
||||
(Paperclip records it as finished) but it is worth knowing before you look for it.
|
||||
|
||||
## Pi 1.5.0 and the model list — where names live, and what changed
|
||||
|
||||
Measured on `.13`:
|
||||
|
||||
| File | What it holds |
|
||||
|---|---|
|
||||
| `~/.agents/models.json` (symlinked as `~/.pi/agent/models.json`) | provider `omni` with **476 explicit model names**, and `openrouter` overrides |
|
||||
| `~/.pi/agent/settings.json` | five providers with `baseUrl`, `api`, `apiKey`: `omni`, `deepseek`, `openrouter`, `opencode-go`, `google` |
|
||||
| `~/.pi/agent/models-store.json` | **discovered** catalogues per provider with `etag` and `checkedAt`, e.g. `deepseek` → 2 models |
|
||||
| a project's `.pi/` | no model names at all; for an agent workspace it holds only `.pi/tasks` |
|
||||
|
||||
So model names are global, not per project. An agent workspace's `.pi/` cannot hold a stale copy.
|
||||
|
||||
The names are there: `deepseek deepseek-flash`, `deepseek deepseek-v4-pro`, and through the router
|
||||
`omni deepseek/deepseek-v4-flash`, `omni deepseek/deepseek-v4-pro`, `omni ds/…`,
|
||||
`omni oc/deepseek-v4-flash-free`, `omni opencode-go/…`. The configured agent model
|
||||
`omni/deepseek/deepseek-v4-flash` resolves.
|
||||
|
||||
**The change that broke Paperclip is not the names, it is the stream.** Pi 1.5.0 prints
|
||||
`--list-models` on **stdout**; Pi 0.8x printed it on **stderr**. Paperclip's `pi_local` adapter
|
||||
reads stderr first:
|
||||
|
||||
```js
|
||||
// Pi outputs model list to stderr, but fall back to stdout for older versions
|
||||
const output = result.stderr || result.stdout;
|
||||
```
|
||||
|
||||
Five extension warnings now live on stderr, so the adapter parsed those, found no `provider model`
|
||||
rows, and refused every run with `Pi returned no models`. The shim that fixes it,
|
||||
`~/.local/share/paperclip-harness/wrappers/pi-paperclip.sh`, puts the table back on stderr for
|
||||
that one flag and passes everything else through unchanged; the Builder agent's
|
||||
`adapterConfig.command` points at it.
|
||||
|
||||
**Consequences to keep in mind**
|
||||
|
||||
- **omniroute** is the source of the `omni` names. A name that is missing from the router's
|
||||
`/v1/models` will not appear in Pi, and `models-store.json` caches with an `etag`, so a stale
|
||||
entry can hide a new name until the catalogue is refreshed.
|
||||
- **headroom** keeps its own model registry (LiteLLM) and still has no entries for the router's
|
||||
ids, which is why it answers `502`. It is unrelated to Pi's storage, but when it is fixed it
|
||||
will need the same names.
|
||||
|
||||
Reference in New Issue
Block a user