loop-engineering

Loop Engineering for AI Agents

Sources: Session notes, 2026-06-25; @h100envy, 2026-06-22; @thealexker, 2026-04-18; @phosphenq, 2026-06-21; @0xwhrrari, 2026-06-13 Raw: Loop Engineering Overview; [h100envy Technical Roadmap](../../raw/clippings/Loop Engineering A Technical Roadmap for an Autonomous Loop.md); [thealexker Harness Engineering](../../raw/clippings/Harnesses are everything. Here’s how to optimize yours..md); [phosphenq Build an AI That Codes While You Sleep](../../raw/clippings/Loop Engineering Build an AI That Codes While You Sleep.md); [0xwhrrari AI Skill Every Builder Needs](../../raw/clippings/Loop Engineering The AI skill every builder needs in 2026.md) Updated: 2026-08-12

Overview

Loop engineering is the discipline of designing automated agentic workflows rather than crafting individual prompts. The core shift: “a prompt is a question, a loop is a job.” In a loop, the agent discovers what needs doing, plans, executes, verifies against an objective criterion, and iterates — without requiring the human to be present for each step. The human’s job shifts from prompting to designing loops.

“I do not prompt Claude anymore. I have loops running that prompt Claude and figure out what to do. My job is to write loops.” — Boris Cherny (Claude Code lead)

Five-Stage Loop Structure

Almost all loop engineering frameworks use the same five stages:

DISCOVER → PLAN → EXECUTE → VERIFY → ITERATE

Verify is the core of the loop. Without an objective verification mechanism, the loop just has an agent agreeing with itself — not progress, just repetition.

4-Condition Test: Is This Worth a Loop?

Before building a loop, check all four conditions:

1. Task repeats (at least weekly) — for one-off tasks, a good prompt is more economical
2. Verification can be automated — tests, type checks, builds, linters; something that fails objectively, not an agent self-grading
3. Token budget can absorb waste — loops re-read context, retry, and explore; each round is more expensive as context grows
4. Agent has senior-engineer-grade tooling — can read logs, has a reproducible environment, can execute and observe its own code

Any missing condition → stay with manual prompting. Do not build a loop.

Six Building Blocks

Block	Role	Claude Code Equivalent
Automation	Heartbeat — triggers the loop	/loop, Routines (cloud-hosted, min 1 hour, ~30 min scheduling jitter)
Worktree	Parallel without conflict	--worktree flag, isolation: worktree
Skill	Reusable knowledge/conventions	.claude/skills/ SKILL.md
Connector/Plugin	Access real tools	MCP (GitHub, Slack, Linear, Jira)
Subagent	Separate maker from checker	.claude/agents/ — adversarial review agent
Memory/State	Cross-run persistence	STATE.md, log.md, GitHub Issues, Linear tickets

The subagent principle: the agent that writes code is too lenient on its own output (self-preferential bias). A separate reviewer agent — or a different model — that does not see the maker’s reasoning chain is the single highest-leverage quality improvement.

Open Loop vs. Closed Loop

Closed loop: human pre-designs the path — explicit goal, fixed steps, verification at each step, stop condition, handoff on failure. Cheap, reliable, clean output. Start here.

Open loop: give the agent a broad goal and let it explore its own path. Finds unexpected solutions, but easily burns tokens on wrong paths, drifts from the goal, and is hard to control.

Recommended: start closed, move to open only after verification gates are strong.

Correct Build Order

All major loop engineering sources agree: sequence matters more than tool choice.

1. Make ONE manual run reliable
2. Write that run as a Skill
3. Wrap the Skill in a Loop (add gate + stop condition)
4. Only then schedule

Skipping to scheduling before the earlier steps are stable is the most common cause of “loop explodes while you sleep.”

Common Failure Modes

Ralph Wiggum Loop (named by Geoffrey Huntley): agent declares “done” too early, loop exits with half-finished work, continues silently burning tokens. Fix: require an objective gate (test/build/lint result), not another agent’s judgment that it’s complete.

Goal drift: long conversations lose constraint information across summarizations. “Never do X” may disappear by round 47. Mitigation: a fixed-re-read VISION.md or AGENTS.md file.

Self-preferential bias: the maker agent grades its own output too leniently. Mitigation: independent reviewer subagent, separate from the maker’s reasoning.

Agentic laziness: loop declares “good enough” before truly finishing. Mitigation: /goal with an independent checker model’s objective stop condition.

Comprehension debt + cognitive surrender: the better the loop runs, the wider the gap between the codebase and what the team actually understands; resisting the urge to blindly accept loop output grows harder. Mitigation: read diffs, periodically check whether gates still catch real problems, restrict loops to small, machine-verifiable work — never architectural decisions.

No token cap: every retry, every subagent, every verification round burns money; uncapped loops easily cost 5-10× budget. Always set explicit iteration limits.

Vague scheduling instructions: “check notifications hourly” without “only notify me when X” creates information flooding. A report nobody can skim is equivalent to no report.

Real-World Cost Data

Production loop costs vary by several orders of magnitude depending on whether brakes are in place:

Case	Cost
Boris Cherny, 259 PRs in 30 days	one engineer, 100% written by Claude
Runaway loop, no cap, nobody watching	$47,000 in 11 days
Peter Steinberger’s 100-agent fleet	~$1.3M/month (employer-funded)

Token cost ranges (from production deployments):

• One medium coding loop: 50K–200K tokens
• Fleet loop (orchestrator + specialist agents): 500K–2M tokens
• Scheduled daily loop: can reach millions of tokens per week

The same technology produces all three outcomes. “The difference is the brakes.”

The frontier fleet is sponsored. Steinberger’s loop runs on ~$1.3M/monthbecauseOpenAInowemployshimandpaysthebill.Ona$20–$200/month plan, the loop that pays off is small, capped, and pointed at one dull job.

Metrics: What to Track

Not token count or loop run count — track cost per accepted change. If the loop produces 10 outputs and you discard 6, you’re doing the review work it was supposed to save. Acceptance rate below 50% means the loop is unprofitable.

Scale: Single-Agent vs. Fleet Loop

Single-agent loop: one agent runs the complete five-stage cycle. Appropriate for small-scope, single-goal tasks (bug fix, research summary, content draft).

Fleet loop: one orchestrator agent decomposes the task, distributes to specialist agents, each with their own subagent for review. Boris Cherny’s Tier 3 (/batch, dynamic JavaScript workflow with agent() / parallel() / pipeline()) is the concrete implementation — used to rename 4,000 files across a monorepo with 800 agents, each with its own worktree and adversarial reviewer.

Vault Application

The vault’s existing skills are closed-loop candidates:

• /lint-vault — clear verification standard: YAML validity, frontmatter completeness
• /analyze-vault — can add objective gate: did interest-map update?
• /arxiv-digest — already scheduled; upgrade path is adding seen-IDs deduplication as objective gate

The minimum viable vault loop: Routine (automation) + skill (SKILL.md) + log.md (state) + structural lint check (gate).

Technical Roadmap: Seven Steps (h100envy)

@h100envy’s technical roadmap (2026-06-22) adds concrete mechanics to the conceptual five-stage structure above. The order is strict — skipping a step is where the loop blows up later.

Step 0: Machine-verifiable check first. The check must be a deterministic external oracle — a test exit code, type check, linter, build result. An agent grading its own output is statistically biased (its own answer is already raised in probability by the sampling distribution). “An agent’s self-assessment is not a check, it is an echo.” The check must also be idempotent — run it ten times on one state; if the result is not stable, fix the check before the loop.

Step 1: One reliable manual run with measurement. Do not automate what does not work by hand. Record: number of model calls, tokens used, most frequent agent error type. This becomes the baseline for detecting loop degradation.

Step 2: Minimal stateless loop. Each iteration launches the agent from clean context. State lives on the filesystem and git, not in the conversation window. This cures context rot — the measurable loss of quality as the context window fills (instructions given at the start are lost by the middle; the “lost-in-the-middle” effect). Stateless iteration keeps cost linear; stateful accumulation makes cost quadratic.

MAX_ITER=20
while [ $i -lt $MAX_ITER ]; do
  if npm test --silent; then exit 0; fi
  claude -p "Tests fail. Fix the first failure minimally." --permission-mode acceptEdits
done

Step 2.5: Narrow context with an explicit token budget. Build the iteration context from three things only: current state, the specific open failure, and only the files relevant to that failure (from the stack trace + last diff). Use a TOKEN_BUDGET=8000 ceiling — without it, contexts grow silently across iterations and context rot returns via file accumulation instead of conversation history.

Step 3: Anti-reward-hacking. Three layers: (1) prompt prohibition (“do not weaken the tests”) — weakest, breaks under pressure; (2) a physical gate that checks the test files did not change in the diff (git diff --quiet -- test/); (3) an adversarial judge agent on a different model that verifies the task is solved in substance, not just that the test is green. The cost of a judge on a strong model doubles the per-turn bill, so use the cheap deterministic gate always and the judge only on expensive errors.

Step 4: State on disk at two levels. STATUS.md (human-readable, for your morning glance) + .loop_state.json (machine-parsable, for the loop’s logic). Free text in STATUS.md can be re-read differently across runs; fields critical to loop logic must be in structured JSON: phase, iteration, last green commit, blocked paths, open failures, budget spent.

Step 5: Isolation — worktree + container. git worktree add ../loop-sandbox -b loop/branch-name gives the loop a separate working copy. For real isolation: docker run --network none --read-only --tmpfs /tmp -v $(pwd):/work:rw. --network none is not paranoia — it limits the blast radius of prompt injection attacks. “Define the loop by what it can destroy, not by what you want it to do.”

Step 6: Brakes and observability. Minimum required: iteration limit (MAX_ITER), budget cap per turn (--max-budget-usd), repeat detector (same failure 3 times in a row → stuck, exit with code 2), liveness heartbeat (write iter=$i ts=$(date +%s) to .loop_heartbeat each iteration), structured JSONL log (one JSON line per event with timestamp, iteration, event type, detail). The log is what lets you diagnose post-mortem: runaway (many agent_call, no green), silent death (heartbeat stopped), random walk (failure changes every iteration without progress), reward hacking (tests modified event).

Step 7: Count cost nonlinearly. Rough estimate before launch: cost ≈ MAX_ITER × (state tokens + work tokens per iteration) × price. Measure one iteration at Step 1, multiply by MAX_ITER, get the upper bound. “The same approach in braked hands closes a contract for hundreds of dollars of API, unbraked it burns tens of thousands.”

Four loop failure modes by log pattern:

• Runaway: many agent_call events, no green. Cure: iteration limit + budget cap.
• Silent death: heartbeat stopped updating. Cause: full context window. Cure: fresh context per phase.
• Random walk: failure changes every iteration, no convergence. Cause: no hard stop condition. Cure: deterministic fixpoint check.
• Understanding debt: the loop ships code faster than you read it. Not visible in the log. “No code fixes it” — only mandatory human diff review.

Harness Engineering: Context and Prompting (thealexker)

@thealexker’s harness engineering guide (2026-04-18) addresses the layer above loop mechanics — how to keep configuration files, prompts, and subagent delegation from degrading agent quality.

The instruction budget problem. Frontier LLMs can only follow a few hundred instructions before entering the “dumb zone” — missing relevant instructions among bloat, functionally encouraged to hallucinate. For global system prompts (CLAUDE.md / AGENTS.md), human-written outperforms LLM-generated: ETH research found LLM-generated system prompts degrade performance while costing ~20% more in inference. Every token in a global prompt is injected on every session — it must fight for its place.

Progressive Disclosure. Instead of front-loading all context, let the agent pull context when needed:

• CLIs: agent runs --help to discover commands as needed, same as a human engineer would. Most useful for internal CLIs with no training data.
• Skills: at startup, only the name and description of each skill are loaded into context. Full SKILL.md instructions are read only when the agent decides a skill is relevant. Claude Code, Codex, and OpenCode all implement this pattern. Claude Code reports this reduces context usage by over 85% for MCP tools.
• MCP tools: Claude Code loads a lightweight index of tool names, then searches and pulls full schemas on demand (over 85% context reduction per Anthropic). Codex and OpenCode load all MCP tool definitions at session start — users must be selective about which servers to enable.

R.P.I. framework (from HumanLayer): every prompt should do exactly one of three things, each in its own context window:

1. Research: give the agent the problem statement, let it explore the codebase. No action taken at this step.
2. Plan: agent writes a step-by-step execution plan. Human reviews and verifies before proceeding. “Outsourcing thinking or being lazy at this step will cost you dearly later on.”
3. Implement: execute the approved plan in a new context window. If the plan is long, use subagents in their own sessions so intermediate states don’t pollute the main context window.

Subagent patterns — use a subagent when a summary of the work is sufficient for the main agent:

• Parallel fan-out: best for investigation. Spin up one subagent per theory/candidate simultaneously; each gets back a summary. The main agent synthesizes without holding all the detail in context. Speed and context isolation simultaneously.
• Pipelines: enforce depth where fan-out explores breadth. Push work through sequential roles (UX evaluator → technical architect → devil’s advocate); each stage receives the previous stage’s output. Useful for non-deterministic systems where multi-perspective evaluation with a frontier judge consolidates responses.

Config file discipline: treat every harness failure as a data point — what broke, at which step, under what conditions — and add it to .md files. The best harness is the one customized through usage, not the newest one.

Open Questions

• What is the right way to compute “cost per accepted change” in practice? What tooling exists for tracking this metric across Claude Code sessions?
• The five sources describing loop engineering are from the same community wave and largely cite each other. What independent validation exists from production-scale deployments over 6+ months?
• For vault skills specifically: what constitutes an objective “gate” for a knowledge quality check that can’t be reduced to structural validation?

See Also

• Claude Code Skills System
• Vault Architecture