# Phase 6 — Guardrails, cost and kill switch

**Goal:** safe to leave running in public. This is a **release gate**, not a follow-up.

**Depends on:** Phase 5.

**Policy source:** `agents/security.md` (authoritative) and `agents/persona.md`. Both are runtime files, versioned in this repo.

---

## Work

### 6.1 any-guardrail is the mechanism

Input and output validation is performed by **[`mozilla-ai/any-guardrail`](https://github.com/mozilla-ai/any-guardrail)**, pinned to an exact version.

- **Inbound:** prompt injection · jailbreak · scope violation · personal-data requests · system-prompt extraction.
- **Outbound:** policy violation · disclosure of persona/security files · personal or infrastructure detail · claims outside the corpus.
- **Fail closed.** Error, timeout or ambiguous verdict ⇒ refuse.
- Rules live in this repo, not in library defaults.
- **If a rule cannot be enforced with this library, report it — do not quietly drop the rule.**

### 6.2 The architectural guardrail

The public agent has **no tools**: no shell, no filesystem, no network fetch, no write path, no memory between visitors. This removes tool abuse, SSRF, exfiltration-by-callback and cross-visitor poisoning as attack classes rather than defending them. Add an assertion to the test suite that no tool or function definitions are registered.

### 6.3 Output safety
- Model output never reaches the DOM as raw HTML; strict allow-list.
- Citation URLs allow-listed.
- Canary string in the system prompt; any appearance ⇒ discard answer, log incident, terminate session.

### 6.4 Cost and abuse

| Control | Value | Enforced at |
|---|---|---|
| Requests per IP | 10/min · 100/day | Caddy + app |
| Questions per session | 20 | App |
| Max output tokens | 600 | Request + gateway |
| Monthly ceiling | hard stop, alert at 60% | **OmniRoute** |
| Kill switch | Mode B off, Mode A unaffected | App config |

Refusals state that the visitor is rate-limited. Never silently drop a request.

### 6.5 Red team — **release gate**

Build and run a payload suite of **≥ 25** attacks, including **≥ 5 indirect** (malicious instructions embedded in ingested content):

- direct injection · persona override · system-prompt extraction · roleplay jailbreaks · encoding tricks · cost-exhaustion loops · citation forgery · off-topic misuse
- and specifically: **overstate Sam's experience** and **ask about a technology not in the corpus**

Pass = 0 disclosures, 0 bypasses, 0 unbounded-cost responses. The honesty payloads must produce a refusal and an honest alternative, not a hedge.

---

## Definition of Done

- [ ] All four guards (limits, classify, retrieve, compose/sanitise) individually testable and fail closed.
- [ ] ≥ 25 payloads executed; results tabulated with actual responses; 0 bypasses.
- [ ] ≥ 5 indirect injections from inside ingested content; untrusted-data framing holds.
- [ ] Canary detected test passes.
- [ ] Monthly ceiling demonstrably enforced — a deliberate over-budget request is refused.
- [ ] Kill switch stops Mode B and **leaves Mode A fully working** (re-verify A1).
- [ ] Test proves the agent has no tool definitions.
- [ ] `agents/persona.md` and `agents/security.md` loaded at runtime, not inlined in code.

---

## Out of scope

- Changing any existing system's guardrails or limits.
- Voice input, if not yet added.

---

## Risks

| Risk | Mitigation |
|---|---|
| Prompt-only defence | Architecture first (no tools), classification second, prompt third |
| "Just for testing" cap removal on a live endpoint | Any change to a cap or guard requires Sam's approval, recorded |
| Red team run after launch | This phase **is** the release gate; nothing goes public without it |
| Guardrail makes the demo timid | Publish the security design and the passing results as site content — "25 payloads, 0 bypasses" is a stronger claim than an unprotected demo |