Summary

A comprehensive technical deep-dive into agent harness engineering, synthesizing how Anthropic, OpenAI, LangChain, and CrewAI implement the infrastructure that transforms a stateless LLM into a capable agent. The post defines 12 harness components, traces a step-by-step execution cycle, compares real framework implementations, and identifies seven key architectural decisions every harness designer must make. The central finding: harness design (not model quality) is the primary performance differentiator in production agents.

這篇技術文章深度分析代理人 Harness 工程,整合 Anthropic、OpenAI、LangChain、CrewAI 的設計哲學。定義 12 個 harness 元件,逐步說明執行迴圈,比較各框架實作,並提出七個架構關鍵決策。核心結論:harness 設計(而非模型品質)是生產代理人效能的主要決定因素。

Key Points

  • Agent harness = orchestration loop + tools + memory + context management + prompt construction + output parsing + state management + error handling + guardrails + verification loops + subagent orchestration + scaffolding lifecycle
  • “If you’re not the model, you’re the harness” (LangChain’s Vivek Trivedy)
  • Context rot: model performance degrades 30%+ when key content falls in mid-window positions (“Lost in the Middle”)
  • Verification loops improve quality 2-3x (Boris Cherny, Claude Code creator)
  • 10-step process at 99% per-step success = only 90.4% end-to-end success; errors compound fast
  • Seven decisions: single vs. multi-agent, ReAct vs. plan-and-execute (3.6x speedup for LLMCompiler), context management strategy (26-54% token reduction), verification design, permission architecture, tool scoping (Vercel removed 80% of tools for better results), harness thickness
  • Claude Code: 95% context reduction via lazy loading; git commits as checkpoints; Ralph Loop for long-running tasks

Insights

The Von Neumann computer analogy (context window = RAM, databases = disk, tools = device drivers, harness = OS) is the clearest architectural frame for explaining why harness engineering matters. The tool scoping finding — removing 80% of tools improved performance — is counterintuitive and important: more tools create decision overhead and context pollution. The “co-evolution principle” (models post-trained with specific harnesses, changing harness degrades performance) has significant implications for framework lock-in and upgrade paths.

Connections

Raw Excerpt

LangChain proved this when they changed only the infrastructure wrapping their LLM (same model, same weights) and jumped from outside the top 30 to rank 5 on TerminalBench 2.0. The harness is not a solved problem or a commodity layer. It’s where the hard engineering lives.