Mouse-1 notes: measured behaviour, the cold start trap, and where it is worth using

Cold call 503 after ~30s, retry 200 after ~13s, warm 489-1044ms, stable answers at ~0.98
confidence. Effective for failure routing, unblock triage, intake clarity and task sizing.
Never for merge decisions, judging tests, keystroke paths, vague options, or as the only
record. Not needed before Milestone 1.
This commit is contained in:
sam
2026-11-11 16:28:48 +11:00
parent e02b39943d
commit c50c1efa22
5 changed files with 362 additions and 0 deletions

89
docs/MOUSE-1-NOTES.md Normal file
View File

@@ -0,0 +1,89 @@
# Mouse-1: what it is, how to call it, and where it actually helps
Written from measurements on this machine, 2026-11-08/09. Nothing here is guesswork; the
numbers are the ones we saw.
## What it is
A small decision model, called over HTTPS, that answers a **typed multiple-choice
question** about a **state you describe**. You do not ask it to write anything. You give it
up to a handful of options with a sentence defining each, and it returns the option it
thinks fits, with a confidence and a probability per option.
- Endpoint: `POST https://api.mouse.dev/v1/decisions`
- Auth: `Authorization: Bearer $MOUSE_API_KEY` (already in your environment)
- Body: `{"model":"mouse-1","state":{...},"questions":{"<id>":{"type":"choice","instructions":"...","criteria":{"<option>":"what this option means"}}}}`
- Response: `answers.<id>.{choice, confidence, probabilities}`
- It also offers `noul` and `score` modes. We have only used `decisions`.
About 100 ms is claimed. We measured differently (below), so plan for the measured numbers.
## The cold start, which is the only real trap
The model is idle until something calls it, and then takes minutes to be ready.
- **Cold:** first call returned `503 engine_timeout` after ~30 seconds of waiting. The body
says: "The model did not answer in time. After an idle period it takes a few minutes to
start; retry after the Retry-After header."
- **Retry once:** the second call returned `200` in ~13 seconds.
- **Warm:** 1044 ms on the first warm call, then ~489 ms steady.
- We saw this again today: `503 engine_timeout`, then `200` with `choice: "wire"`,
`confidence: 0.982`.
So the rule is: **retry once on 503, never block a person's keystroke on it, and never call
it on a path that has to be fast.** If it must be warm, call it once at the start of a run
and throw the answer away.
Determinism: the same state and options produced the same choice with confidence around
0.98 across calls. Treat it as a classifier, not a creative step.
## Where it is effective — the use cases we proved
**1. Failure routing.** A task failed. The useful question is not "fix it" but "what kind of
failure is this": redo the task, use a stronger model, fix the environment, or ask the
person. Options with one line each:
```json
{"model":"mouse-1",
"state":{"task":"build skeleton","test":"npm test","output":"...tail of the failure..."},
"questions":{"next":{"type":"choice",
"instructions":"This task failed. What does it need next?",
"criteria":{
"redo":"the work is incomplete or wrong; another attempt with the same agent is likely to succeed",
"stronger_model":"the task needs more capability than this model has",
"environment":"something outside the code is wrong: a missing tool, a service, a path",
"person":"a decision only the person can make"}}}}
```
This was built and measured in the old project (`bin/route-failure`): a cold call returned
503 after 30.5 s, the retry 200 after 12.8 s, and warm calls ranged 489–1044 ms with a
stable answer. It is the best use we found.
**2. Unblock triage.** A task is blocked and the thread has several messages: is it waiting
on the person, on another task, on a missing tool, or on a decision that was already made?
Same shape, four options, one paragraph of state.
**3. Intake clarity.** Before a goal becomes a plan: is this goal specific enough to plan,
or does it need one more question? That was designed and never built.
**4. Task sizing.** Given a task description, is it one task or does it need splitting?
Also designed, never built.
## Where it is not effective, and must not be used
- **Never as the merge decision.** Whether code is correct is decided by tests and review,
not by a model with no repository.
- **Never as a judge of test results.** If a test failed, it failed.
- **Never on a keystroke path.** The cold start alone rules it out.
- **Never with vague options.** "good" and "bad" give you nothing. Every option needs a
sentence that says what choosing it means.
- **Never as the only record.** Keep the answer, the confidence and the options with the
task, so a person can see why a machine decided something.
## How to use it in this project
Not at Milestone 1. Nothing in "one task, really run" needs it.
Add it when a task first fails for a reason that is not obvious, and use it only for
routing: which kind of failure is this, and therefore what should happen next. That is the
point where it saves a person's attention, which is the whole reason it is here.