本文由 AI 分析生成
建立時間： 2026-05-30 來源： https://x.com/MinLiBuilds/status/2034533228162187444
Summary
Ole Lehmann’s article (Chinese translation) applies Karpathy’s autoresearch methodology to Claude skill optimization: instead of manually tuning prompts, an agent loop automatically tests skill output against a binary checklist, retains improvements, reverts degradations, and repeats. The author raised a landing page copywriting skill from 56% to 92% pass rate in 4 automated iterations.
Ole Lehmann 文章的中文譯版，將 Karpathy 的 autoresearch 方法論應用到 Claude Skill 優化：用 Agent 循環自動評測 Skill 輸出是否通過是/否 Checklist，保留改善、回滾退步。作者的落地頁文案 Skill 在 4 輪自動迭代後從 56% 提升至 92% 通過率。
Key Points
- Core loop: run skill → score against checklist → if improved keep, if worse revert → repeat
- Binary (yes/no) checklist questions are essential: they prevent score drift that 1-10 ratings suffer from
- 3-6 checklist items is the sweet spot — more items causes the skill to game the checklist rather than improve quality
- The agent automatically identifies the weakest checklist items and targets them first
- Skill prompt changes are versioned: changelog documents what worked, what failed, and why
- Author’s concrete improvements: added ban list (“revolutionary”, “cutting-edge”, “synergy”), added concrete number requirement in headlines, added high-quality example with annotations
Insights
The key insight is that most people never systematically validate their AI workflows — they notice individual failures and fix them in isolation. The autoresearch loop turns this ad hoc process into a systematic optimization with measurable convergence.
The 3-6 item limit on checklist is a counter-intuitive finding: more criteria causes the AI to optimize narrowly for the checklist rather than for actual quality. This is an instance of Goodhart’s Law in prompt engineering — once a measure becomes a target, it ceases to be a good measure.
The changelog produced by the optimization loop is described as “the most valuable artifact” — it gives future agents a complete history of what was tried, what worked, and what to avoid when a more powerful model is available.
Connections
Raw Excerpt
“從靠運気，變成靠系統。這才是最值錢的東西。Before: hoping each delivery works. After: knowing exactly when it works, when it fails, and how to fix it.”