本文由 AI 分析生成
建立時間： 2026-03-17 來源： https://x.com/MinLiBuilds/status/2034533228162187444
Summary
A Chinese translation and adaptation of Ole Lehmann’s guide on applying Karpathy’s autoresearch method to Claude skills. The method runs an agent in a continuous loop: generate output with a skill, score against a yes/no checklist, keep changes that improve score, revert changes that hurt it, repeat until convergence or 95% pass rate. The author raised a landing-page copywriting skill from 56% to 92% pass rate across four automated iterations with zero manual intervention.
Ole Lehmann 關於將 Karpathy 的 autoresearch 方法應用於 Claude skills 的中文翻譯與改編。該方法讓 agent 在持續循環中運作：用 skill 生成輸出、根據是/否 checklist 打分、保留提高分數的改動、還原降低分數的改動，直到收斂或達到 95% 通過率。作者透過四次自動迭代將落地頁文案 skill 的通過率從 56% 提升至 92%，全程零手動干預。
Key Points
- The autoresearch loop: try one small change → measure → keep if better, revert if not → repeat
- Yes/no checklists are essential — 1-10 scoring drifts; binary questions about concrete properties do not
- 3-6 checklist items is the sweet spot; more than 10 causes the skill to “teach to the test” rather than improve quality
- The changelog is the most valuable artifact — it is the skill’s accumulated “experience” that transfers to the next model version
- Applicable to any measurable artifact: ad copy, cold emails, newsletter openers, system prompts
- The autoresearch skill itself is open-source on GitHub (olelehmann100kMRR/autoresearch-skill)
Insights
The most practically transferable insight: skills degrade silently through “prompt drift” — the model gravitates toward safe, generic outputs over time, and without systematic measurement you cannot detect when or how often this happens. The checklist plus automated loop is the minimal viable quality assurance system for any production Claude skill. This pattern applies directly to vault analysis skills like clip, analyze-vault, and arxiv-digest.
Connections
Raw Excerpt
“我现在有个自己的规则：任何 skill，没跑过 autoresearch 的，我不拿出去用。不是完美主义，是因为我踩过那个坑。我知道那种自信其实只是无知。”