Lab Specification — Module 12: Capstone

Module: 12 · Duration: 90+ min (this IS the capstone build) · Environment: Codespace, Docker, Node 18+ / Python 3.11+, OpenAI key.

This module IS the lab. Build a production harness from scratch, then audit and attack it.

Phase 1 — Design document (20 min)

Write design-doc.md BEFORE coding. Fill all 12 decisions + threat model:

# Harness Design Document

## Architecture choice: [thin Pi-inspired | thick Claude Code-inspired]
## Use case: [personal coding assistant | enterprise multi-tenant | ...]

## 12 Decisions
1. Loop: [ReAct | Plan-then-Execute | Graph] — because [tradeoff]
2. Tools: [list, count] — because [Vercel finding | use case]
3. Context: [compaction threshold N | masking] — because [use case]
4. Memory: [working files + episodic log | + semantic store]
5. Sandbox: [inside | outside] [Docker | E2B] — because [credential question]
6. Permission: [risk-tiered + capabilities] — because [Module 6]
7. Errors: [4-category taxonomy applied]
8. Checkpointing: [git | atomic file | DB]
9. Prompt: [thin <1k | dense ~40k] — cache_control? [yes/no]
10. Subagents: [none for v1 | agents-as-tools]
11. Verification: [computed (pytest/lint)]
12. Observability: [8-field per-turn payload]

## Security threat model (Module 11)
Defenses in v1: [untrusted-tagging, capability perms, fs/network scoping, risk-tiered approval]
Deferred: [signed manifests (C2/S12), model-judged verification]

Phase 2 — Build (40 min)

Implement the minimum viable production harness:

ReAct loop with 5 stop conditions (Module 1)
4 tools with Zod/Pydantic schemas + structured errors (Module 2)
Compaction at ~50% context threshold (Module 3)
Working files + episodic log (Module 4)
Docker sandbox with fs/network scope (Module 5)
Risk-tiered permission gate + capability checks (Module 6)
Error taxonomy classifier (Module 7)
Atomic checkpoint writes (Module 8)
Computed verification gate (pytest) (Module 9)
8-field per-turn logging to trace.jsonl (Module 10)
Untrusted-tagging on all tool outputs (Module 11)

Phase 3 — Audit + attack (30 min)

Run the 6-phase methodology on your harness (Module 0.3).
Fill the 12-module scoring sheet (/60).
Execute 3 attacks:
- Indirect injection (forge README → agent reads → executes injection)
- Memory poisoning (sleeper: inject S1, activate S2)
- Tool abuse (legitimate tool for unintended purpose)
Fix every vulnerability found.
Write the Architect's Verdict (3 sentences) + MLSecOps note (1 sentence).

Deliverables (the portfolio artifact)

design-doc.md — 12 decisions + threat model, stated BEFORE code
The harness code (all 11 components integrated)
scoring-sheet.md — 12-module self-score (/60)
Attack results: 3 attacks attempted; vulnerabilities found + fixed
verdict.md — Architect's Verdict (3 sentences) + MLSecOps Relevance (1 sentence)

Solution key (instructor)

Design doc: must state all 12 decisions + threat model BEFORE code. If it reads like a post-hoc rationalization (decisions after the build), reject — redo as design-first.
Build: must include all 5 stop conditions; all 4 tools must have schemas + structured errors; compaction must fire at threshold; observability must emit all 8 fields per turn.
Audit: the 6-phase self-audit should find at least 2 vulnerabilities the builder didn't notice during construction (this is the point of the methodology — objectivity via distance).
Attacks: indirect injection should succeed WITHOUT untrusted-tagging and fail WITH it. Memory poisoning should succeed without the write-gate and fail with it. Tool abuse should be caught by capability permissions.
Verdict: must address all 3 questions (optimize for / sacrifice / who builds on it) in exactly 3 sentences. MLSecOps note must name the single most important security property in 1 sentence.

What's next

Course 2 — take this harness into offensive/defensive security: bug bounty, SDLC gates, red teaming.
The 21 deep-dives — apply the 6-phase method to Pi, Aider, OpenCode, Codex CLI, Gemini CLI, oh-my-opencode, OpenClaw, Hermes, NemoClaw, LangGraph, OpenAI Agents SDK, CrewAI, OpenHarness, Mastra, Command Code, ZeroClaw, PicoClaw, MetaClaw, CrabTrap, IronCurtain, Tau.
Course 3 (Fleet) — the system around many harnesses: control plane, routing, cost governance, multi-tenancy.

The capstone harness is the foundation for all three.

# Lab Specification — Module 12: Capstone **Module**: 12 · **Duration**: 90+ min (this IS the capstone build) · **Environment**: Codespace, Docker, Node 18+ / Python 3.11+, OpenAI key. > This module IS the lab. Build a production harness from scratch, then audit and attack it. --- ## Phase 1 — Design document (20 min) Write `design-doc.md` BEFORE coding. Fill all 12 decisions + threat model: ```markdown # Harness Design Document ## Architecture choice: [thin Pi-inspired | thick Claude Code-inspired] ## Use case: [personal coding assistant | enterprise multi-tenant | ...] ## 12 Decisions 1. Loop: [ReAct | Plan-then-Execute | Graph] — because [tradeoff] 2. Tools: [list, count] — because [Vercel finding | use case] 3. Context: [compaction threshold N | masking] — because [use case] 4. Memory: [working files + episodic log | + semantic store] 5. Sandbox: [inside | outside] [Docker | E2B] — because [credential question] 6. Permission: [risk-tiered + capabilities] — because [Module 6] 7. Errors: [4-category taxonomy applied] 8. Checkpointing: [git | atomic file | DB] 9. Prompt: [thin <1k | dense ~40k] — cache_control? [yes/no] 10. Subagents: [none for v1 | agents-as-tools] 11. Verification: [computed (pytest/lint)] 12. Observability: [8-field per-turn payload] ## Security threat model (Module 11) Defenses in v1: [untrusted-tagging, capability perms, fs/network scoping, risk-tiered approval] Deferred: [signed manifests (C2/S12), model-judged verification] ``` --- ## Phase 2 — Build (40 min) Implement the minimum viable production harness: - ReAct loop with 5 stop conditions (Module 1) - 4 tools with Zod/Pydantic schemas + structured errors (Module 2) - Compaction at ~50% context threshold (Module 3) - Working files + episodic log (Module 4) - Docker sandbox with fs/network scope (Module 5) - Risk-tiered permission gate + capability checks (Module 6) - Error taxonomy classifier (Module 7) - Atomic checkpoint writes (Module 8) - Computed verification gate (pytest) (Module 9) - 8-field per-turn logging to trace.jsonl (Module 10) - Untrusted-tagging on all tool outputs (Module 11) --- ## Phase 3 — Audit + attack (30 min) 1. Run the 6-phase methodology on your harness (Module 0.3). 2. Fill the 12-module scoring sheet (/60). 3. Execute 3 attacks: - Indirect injection (forge README → agent reads → executes injection) - Memory poisoning (sleeper: inject S1, activate S2) - Tool abuse (legitimate tool for unintended purpose) 4. Fix every vulnerability found. 5. Write the Architect's Verdict (3 sentences) + MLSecOps note (1 sentence). --- ## Deliverables (the portfolio artifact) - [ ] `design-doc.md` — 12 decisions + threat model, stated BEFORE code - [ ] The harness code (all 11 components integrated) - [ ] `scoring-sheet.md` — 12-module self-score (/60) - [ ] Attack results: 3 attacks attempted; vulnerabilities found + fixed - [ ] `verdict.md` — Architect's Verdict (3 sentences) + MLSecOps Relevance (1 sentence) --- ## Solution key (instructor) - **Design doc**: must state all 12 decisions + threat model BEFORE code. If it reads like a post-hoc rationalization (decisions after the build), reject — redo as design-first. - **Build**: must include all 5 stop conditions; all 4 tools must have schemas + structured errors; compaction must fire at threshold; observability must emit all 8 fields per turn. - **Audit**: the 6-phase self-audit should find at least 2 vulnerabilities the builder didn't notice during construction (this is the point of the methodology — objectivity via distance). - **Attacks**: indirect injection should succeed WITHOUT untrusted-tagging and fail WITH it. Memory poisoning should succeed without the write-gate and fail with it. Tool abuse should be caught by capability permissions. - **Verdict**: must address all 3 questions (optimize for / sacrifice / who builds on it) in exactly 3 sentences. MLSecOps note must name the single most important security property in 1 sentence. --- ## What's next - **Course 2** — take this harness into offensive/defensive security: bug bounty, SDLC gates, red teaming. - **The 21 deep-dives** — apply the 6-phase method to Pi, Aider, OpenCode, Codex CLI, Gemini CLI, oh-my-opencode, OpenClaw, Hermes, NemoClaw, LangGraph, OpenAI Agents SDK, CrewAI, OpenHarness, Mastra, Command Code, ZeroClaw, PicoClaw, MetaClaw, CrabTrap, IronCurtain, Tau. - **Course 3 (Fleet)** — the system around many harnesses: control plane, routing, cost governance, multi-tenancy. The capstone harness is the foundation for all three.