The agent harness

ii. Pillar I — Memory

Stop re-explaining yourself

Every time you correct an agent and the correction evaporates at session end, you pay for the same mistake twice. Memory is the fix: a durable place where corrections graduate into rules the agent reads at the start of every session.

What we already do — strong. We run a three-file memory system more sophisticated than the article's: PROFILE (how Esteban works), LEARNINGS (dated mistakes + the rule going forward), TEACH-ME (plain-language teaching log). A SessionStart hook surfaces them automatically; a /prune-memory skill keeps them sharp.

The hard number to know High — frontier models reliably follow ~150–200 instructions before compliance degrades, and Claude Code's own system prompt already spends ~50. The heuristic: keep each memory file under ~200 lines. Past that you hit "context rot" — Anthropic's term for the measured effect that, as the window fills, the model's ability to recall and obey buried instructions drops. The failure is silent: it doesn't error, it just quietly ignores the rule.

The rule every memory system needs High — a stated conflict-resolution order. Layered memory makes it easy to encode two rules that disagree. Our CLAUDE.md does this well in one place ("where they conflict, Tech-Minimal wins") — and also contains a live example of the trap: a superseded "never touch main" rule a later rule overrides.

iii. Pillar II — Hooks

Mechanical checks while the work happens

A hook is a shell command (or HTTP call, MCP tool, even a sub-agent) that Claude Code runs automatically at a fixed point in its lifecycle — before a tool runs, after a file is written, when a session starts, when it tries to stop. It's how you make a check deterministic instead of hoping the model remembers.

How big this surface got High — the most surprising finding. Hooks grew from a handful of events to ~30+ lifecycle events by mid-2026: not just PreToolUse / PostToolUse / Stop, but SubagentStart/Stop, PostToolBatch, ConfigChange, FileChanged, PreCompact, Notification, even MCP elicitation. Nearly every step of the agent loop is now instrumentable.

The rules that actually matter

• PreToolUse is the only true gate. It fires before execution and can cancel a tool call — and before the permission check, so it blocks even in "skip-permissions" mode. This is where you stop dangerous things.
• PostToolUse is observability-only. By the time it runs, the action already happened. You can format, log, annotate — you cannot undo. Don't rely on it for prevention.
• Hooks tighten, never loosen. A hook saying "allow" can't override a "deny" from settings. Deny always wins — so a hook can't accidentally open a hole.
• Exit 0 = proceed; exit 2 = block and feed the reason back to Claude as something it can act on.

What we already do. We run four hooks: SessionStart (surface memory), UserPromptSubmit (a nudge-style skill router that flags matching skills without ever hijacking the message), PreToolUse-Bash (block destructive commands), and a conditional Stop (wrap-up only when there's uncommitted work). The router and the conditional Stop are genuinely more nuanced than most public examples.

The security model, in one line High — a hook is pre-authorized arbitrary code execution with your full shell privileges, run automatically every time the trigger fires. That's the entire risk. Sanitize inputs as untrusted; scope matchers narrowly; and for hard guarantees use the permission system, not hooks (which can fail open).

iv. Pillar III — Orchestration

One command, many agents

Instead of you being the operator who copies output from step one into step two, a single command runs the whole workflow — often in parallel. The article's signature example: one newsletter → three agents at once → a LinkedIn post, a Twitter thread, and Substack Notes.

• Slash command vs sub-agent. A slash command is a macro — a saved prompt that expands inline in your main context. A sub-agent is a worker — a separate context window that does a scoped job and returns only its conclusion, with the messy intermediate work garbage-collected. The benefit is context isolation: a clean main thread.
• The economics are steep High. Anthropic's multi-agent research system beat a single agent by 90.2% — but burned ~15× the tokens, and token usage alone explained ~80% of that gain. A lot of the "win" is just spending more compute.
• The domain warning that reshapes our plan High. Both Anthropic and Cognition ("Don't Build Multi-Agents") agree: parallel multi-agent is a poor fit for coding and any task with shared context or dependencies — split the work and each agent makes unstated decisions the others can't see, so the pieces don't compose. To build one coherent thing: stay single-threaded.

What we already do — strong. Our /council and /llm-council already fan out to parallel critics. We use parallel research agents — this paper was researched by several agents at once. We have a documented /parallel-session protocol for two Claudes on one repo.

v. Pillar IV — Evaluation loops · where the research bites hardest

Grounded, or theatre

Don't trust first-draft output: generate → evaluate → repair → re-check before a human sees it. The deep research changed my view here, and the split is the single most important thing in this paper.

Grounded verification works High

When the evaluator is an external oracle — unit tests, a compiler, a type-checker, a linter, a runtime error, an accessibility audit — the loop is reliable, predictable, auditable. It's why coding agents self-correct well in practice: the tests tell the truth. Reflexion hit 91% on HumanEval (beating GPT-4's 80%) precisely because the loop was closed by test execution, not opinion.

Self-grading is largely theatre High

When a model grades its own output with no external check, the wheels come off. The load-bearing paper — Huang et al. (DeepMind, ICLR 2024): "LLMs struggle to self-correct their responses without external feedback, and at times, their performance even degrades." Earlier optimistic results were quietly leaking the correct answer into the loop; remove that oracle and the gains vanish. GPT-4 drops from 95.5% → 91.5% on GSM8K when it reviews its own reasoning unaided.

And the judge is biased toward itself High

LLM-as-judge shows documented self-preference bias — models score their own output higher, and the bias grows with capability. Add position bias, verbosity bias, the fact that self-refinement scores aren't even monotonic, and the fact that judges are gameable by surface features — and pure self-grading looks less like quality control and more like confidence-laundering.

A model grading its own work, alone, is the weakest signal in the system — and the most dangerous, because it feels like rigour.

What we already do — better than expected. Our councils are deliberately built as independent critics; /brand-check against the design-system Definition of Done is a genuine grounded check (a checklist, not a vibe). The instinct — separate the maker from the judge — is exactly right. Where we lag: the loops are manual, and a council of LLM personas is still LLM judgement unless anchored to something external. Our most grounded checks (brand DoD, contrast, performance) are the ones we should automate and run every time.

vi. Challenge & defend

I tried to kill the idea. Here's what survived.

The brief was to challenge these ideas, then defend what holds. An honest adversarial pass:

vii. The honest scorecard

Where we actually stand

We're not behind the article. We're ahead of it on orchestration and memory architecture — and the gaps are specific and cheap to close.

viii. Next steps · ranked by leverage

The four upgrades we're shipping

Do now — high leverage, low effort, low risk

Do next — high leverage, moderate effort

Consider — worth a look, handle with care

ix. Method & honesty note