Files
paperclip-harness/docs/TEST-PLAN.md

8.0 KiB

The test plan, with every key and every expected result

Sit at the keyboard and follow this in order. Each step says what to press, what you should see, and when to stop. Nothing here needs a shell inside the factory; where a shell is needed it is Shell>shell.

Before you start: the eight workspaces are already built. R in Repair>repair rebuilds them at any time. If a pane looks wrong, press R and wait ten seconds.

The words, once, plainly

Word Means
task one unit of work in Paperclip, with its own thread, its own copy of the code, its own cost
blocker another task that must reach done before this one may start. A link between two tasks, stored by Paperclip.
verify command (optional) a shell command that exits 0 only when the task is finished, when you want a hard gate no agent can talk past. Most tasks need none: the agent runs the project's own tests and reports what it did.
assign give the task to one agent. This is what wakes the agent and makes it start.
worktree the task's own copy of the repository, made by Paperclip, so two tasks cannot overwrite each other
run one execution of one agent. A task usually has one or two.

The eight workspaces are not agents. Workspaces are TUIOS pages. The agents are six records inside Paperclip: researcher, System, Hub, Chief of Staff, Documenter, Builder. You see them all in Agents>agents.

Step 0 — check the ground

  1. Go to Repair>repair. The health tab should say everything checked passes.
  2. Go to Agents>agents. The roster should list six agents, all idle except any that are paused. If Builder says paused, select it with ↑/↓ and press P to resume.

If either is wrong, stop and say what the screen shows.

Step 1 — create a task

Go to New task>new. The form has five fields. Type into the first, press Enter, and the next field takes the cursor. Tab also moves forward, Shift-Tab back.

Fill it in exactly like this:

Field Type
Title Add a file named STATUS.md with one line
What it must do Create STATUS.md at the repository root containing exactly one line: the factory is running. Change no other file. Commit it.
Must finish first leave empty — press Enter
A command that proves it (optional) leave empty — the agent decides how to check its own work
Assign to yes

Enter on the last field creates it. The form stays open and shows:

created AUD-1xx, handed to an agent which will start on its own

Expected: within a second or two the task exists and Builder has picked it up. Go to Board>board: your task is at the top under IN PROGRESS, assigned to Builder.

Stop if: an error line appears. The text is Paperclip's own message; copy it.

Step 2 — watch it work

Stay in Board>board. The row for your task should move from todo to in progress and then to done within a minute or two.

Then go to Agents>agents:

  1. The roster tab: Builder should show a run in progress, or idle with the money increased.
  2. Press ] once to the runs tab. The newest run is on top, with agent, state, what woke it (assignment), duration and the task identifier.
  3. Select the top run with ↑ and press Enter. The log tab opens with what Pi actually did: the prompt, its reasoning, its tool calls.
  4. Press [ twice to come back.

Expected: one run, succeeded, woken by assignment.

Step 3 — read what changed

Go to Work>work. It follows the task you chose on the board.

  1. runs tab: the run timeline, then the activity.
  2. ] to change: the worktree path, the branch, the commit, STATUS.md, and the diff (green + lines).
  3. ] to facts: the folder, branch, base and cost.
  4. ] to review: either a recorded verdict or "no review recorded".

Expected: the diff shows exactly one file, STATUS.md, one line added.

Step 4 — merge it

No terminal. Stay in Work>work.

  1. Press m. It asks first, and tells you what it is about to do:

    merge AUD-1xx?
      what happens   rebase onto the base branch
                     no verify command: the review decides
                     ocr reviews the change; a critical finding refuses the merge
                     merge, then re-run the checks on the base branch
      y              do it
      esc or n       do nothing
    

    That is the approval step. Nothing merges until you press y.

  2. Press y. The merge tab shows what is happening:

    merge of AUD-1xx started 12:01:39
      rebased onto main
      no verify command: this would pass on the agent's word
      review saw nothing to review (0 files); recorded, not hidden
      merged into main at 4a44473
    merge finished 12:01:43
    
  3. Wait for merged into main at <commit>. That commit is the task's work, on the project's base branch.

Expected: a green refusal line, not a red one. If it says refused: the review found 1 blocking item(s), the gate worked — the findings are on the task and in Work>review.

Starting a merge twice by accident is safe: the merge takes a lock and the second one is refused with another merge is in progress.

Step 4b — look at one run, without leaving Work

In Work>work the runs tab lists every run of the chosen task. ↑/↓ to one and press Enter: the log tab opens with what Pi actually did — the wake payload, its reasoning, its tool calls. [ goes back.

Step 4c — choose a task without leaving Work

In Work>work, press o. A list of this project's tasks appears. ↑/↓ to one, Enter, and the runs, change, facts, review and merge tabs all switch to it. You never have to go to Board>board first.

Step 5 — a task that must wait for another

  1. New task>new again. Title Read STATUS.md and quote it.
  2. What it must do: Read STATUS.md at the repository root and quote its contents in a comment on this task. Change no file.
  3. Must finish first: the identifier of the task you just merged, for example AUD-171.
  4. A command that proves it: leave empty.
  5. Assign to: yes.

Expected: Board>board shows it under TODO with waits on AUD-171 beside it, and it does not start. That is the blocker: Paperclip holds it until the other task is done — which it is, so the agent may start. If the blocker were unfinished, it would sit there.

To see the waiting case, do the same but block the new task on a task that is still todo.

Step 6 — change an agent

Go to Agents>agents, then ] four times to settings.

  1. ↑/↓ to model, press Enter, edit the text, press Enter to save.
  2. ↑/↓ to monthly budget, press Enter, type a number of cents, press Enter.
  3. p pauses the chosen agent, P resumes it, w wakes it now.

Expected: the row shows the new value immediately and Paperclip has it. A model change applies to that agent's next run, never one already running.

Note: you can set an agent at any time — before a run, between runs, while paused. What you cannot do is change the model of a run that has already started; that is why the next run picks it up.

Step 7 — skills and extensions per agent

Still in Agents>agents, ] to skills.

  1. ↑/↓ to a skill marked not in Paperclip, press space.

Expected: the line changes to on this agent. That skill — and no other — is now in that agent's list. space again removes it.

] once more to extensions. space on one writes a wrapper for that agent only and sets its command; space again removes it.

Step 8 — rebuild

Go to Repair>repair and press R.

Expected: every pane closes and comes back within about ten seconds, in the same order: Shell, Board, Work, Agents, Reply, New task, Project, Repair. The third tab shows the rebuild log.

What to write down

For each step: what you pressed, what appeared, and anything that did not match. That is more useful than a verdict, because it says where the interface and Paperclip disagree.