Files
paperclip-harness/docs/TEST-PLAN.md

7.0 KiB

The test plan, with every key and every expected result

Sit at the keyboard and follow this in order. Each step says what to press, what you should see, and when to stop. Nothing here needs a shell inside the factory; where a shell is needed it is Shell>shell.

Before you start: the eight workspaces are already built. R in Repair>repair rebuilds them at any time. If a pane looks wrong, press R and wait ten seconds.

The words, once, plainly

Word Means
task one unit of work in Paperclip, with its own thread, its own copy of the code, its own cost
blocker another task that must reach done before this one may start. A link between two tasks, stored by Paperclip.
verify command (optional) a shell command that exits 0 only when the task is finished, when you want a hard gate no agent can talk past. Most tasks need none: the agent runs the project's own tests and reports what it did.
assign give the task to one agent. This is what wakes the agent and makes it start.
worktree the task's own copy of the repository, made by Paperclip, so two tasks cannot overwrite each other
run one execution of one agent. A task usually has one or two.

The eight workspaces are not agents. Workspaces are TUIOS pages. The agents are six records inside Paperclip: researcher, System, Hub, Chief of Staff, Documenter, Builder. You see them all in Agents>agents.

Step 0 — check the ground

  1. Go to Repair>repair. The health tab should say everything checked passes.
  2. Go to Agents>agents. The roster should list six agents, all idle except any that are paused. If Builder says paused, select it with ↑/↓ and press P to resume.

If either is wrong, stop and say what the screen shows.

Step 1 — create a task

Go to New task>new. The form has five fields. Type into the first, press Enter, and the next field takes the cursor. Tab also moves forward, Shift-Tab back.

Fill it in exactly like this:

Field Type
Title Add a file named STATUS.md with one line
What it must do Create STATUS.md at the repository root containing exactly one line: the factory is running. Change no other file. Commit it.
Must finish first leave empty — press Enter
A command that proves it (optional) leave empty — the agent decides how to check its own work
Assign to yes

Enter on the last field creates it. The form stays open and shows:

created AUD-1xx, handed to an agent which will start on its own

Expected: within a second or two the task exists and Builder has picked it up. Go to Board>board: your task is at the top under IN PROGRESS, assigned to Builder.

Stop if: an error line appears. The text is Paperclip's own message; copy it.

Step 2 — watch it work

Stay in Board>board. The row for your task should move from todo to in progress and then to done within a minute or two.

Then go to Agents>agents:

  1. The roster tab: Builder should show a run in progress, or idle with the money increased.
  2. Press ] once to the runs tab. The newest run is on top, with agent, state, what woke it (assignment), duration and the task identifier.
  3. Select the top run with ↑ and press Enter. The log tab opens with what Pi actually did: the prompt, its reasoning, its tool calls.
  4. Press [ twice to come back.

Expected: one run, succeeded, woken by assignment.

Step 3 — read what changed

Go to Work>work. It follows the task you chose on the board.

  1. runs tab: the run timeline, then the activity.
  2. ] to change: the worktree path, the branch, the commit, STATUS.md, and the diff (green + lines).
  3. ] to facts: the folder, branch, base and cost.
  4. ] to review: either a recorded verdict or "no review recorded".

Expected: the diff shows exactly one file, STATUS.md, one line added.

Step 4 — merge it

This is the one step that needs a shell, because a merge takes a lock and re-runs the checks.

  1. Go to Shell>shell. You are in the project folder.
  2. Type: bin/merge AUD-1xx --check (your identifier), press Enter.

Expected: it rebases, runs the review, and prints either checked only; nothing merged or no verify command: this would pass on the agent's word followed by the same. Both are normal: nothing is merged yet.

  1. Type: bin/merge AUD-1xx, press Enter.

Expected: the same checks, then merged into main at <commit>. A review that finds something serious refuses the merge and says so; that is the gate working, not a failure.

  1. Type git log --oneline -3, press Enter. Your task's merge commit is at the top.

Stop if: it says refused. The reason is printed and also written on the task, where Work>review shows it.

Step 5 — a task that must wait for another

  1. New task>new again. Title Read STATUS.md and quote it.
  2. What it must do: Read STATUS.md at the repository root and quote its contents in a comment on this task. Change no file.
  3. Must finish first: the identifier of the task you just merged, for example AUD-171.
  4. A command that proves it: leave empty.
  5. Assign to: yes.

Expected: Board>board shows it under TODO with waits on AUD-171 beside it, and it does not start. That is the blocker: Paperclip holds it until the other task is done — which it is, so the agent may start. If the blocker were unfinished, it would sit there.

To see the waiting case, do the same but block the new task on a task that is still todo.

Step 6 — change an agent

Go to Agents>agents, then ] four times to settings.

  1. ↑/↓ to model, press Enter, edit the text, press Enter to save.
  2. ↑/↓ to monthly budget, press Enter, type a number of cents, press Enter.
  3. p pauses the chosen agent, P resumes it, w wakes it now.

Expected: the row shows the new value immediately and Paperclip has it. A model change applies to that agent's next run, never one already running.

Note: you can set an agent at any time — before a run, between runs, while paused. What you cannot do is change the model of a run that has already started; that is why the next run picks it up.

Step 7 — skills and extensions per agent

Still in Agents>agents, ] to skills.

  1. ↑/↓ to a skill marked not in Paperclip, press space.

Expected: the line changes to on this agent. That skill — and no other — is now in that agent's list. space again removes it.

] once more to extensions. space on one writes a wrapper for that agent only and sets its command; space again removes it.

Step 8 — rebuild

Go to Repair>repair and press R.

Expected: every pane closes and comes back within about ten seconds, in the same order: Shell, Board, Work, Agents, Reply, New task, Project, Repair. The third tab shows the rebuild log.

What to write down

For each step: what you pressed, what appeared, and anything that did not match. That is more useful than a verdict, because it says where the interface and Paperclip disagree.