Summary

Sean Goedecke, a staff engineer, documents how his LLM usage shifted dramatically in one year: from occasional Copilot autocomplete to using AI agents for essentially every pull request. The key meta-skill is calibration—knowing when agents add value versus when they create more review overhead than they save. The balance continues shifting toward more agentic use, while certain high-context tasks (PR descriptions, ADRs, Slack) remain human-written.

一位 Staff Engineer 記錄過去一年中 LLM 使用方式的巨大轉變:從偶爾使用 Copilot 補全,到幾乎每個 PR 都依賴 AI Agent。核心技能是校準——判斷哪些工作交給 Agent 真的更有效率,哪些反而增加審查負擔。PR 說明文件、ADR、架構 Slack 溝通等需要組織情境的工作仍由人工完成。

Key Points

  • Agents used for ~100% of PRs now (up from occasional tactical use), with ~80% success rate on bug diagnosis
  • AI handles: writing and running tests, environment setup, boilerplate, short tactical changes in unfamiliar areas
  • Still hand-written: PR descriptions, ADRs, architecture Slack messages—anything requiring organizational context or political framing
  • Never uses AI for UI testing—review overhead exceeds time saved
  • The key calibration question has shifted from “can the AI do this?” to “what is the blast radius if this agent does it wrong, and how hard is it for me to catch that?”
  • Under-delegation (not trusting the agent enough) is now as costly a mistake as over-delegation

Insights

The “blast radius of being wrong” framing makes delegation decisions tractable in a way that capability-based reasoning does not. An agent might be technically capable of writing a PR description but the failure mode—a misframed narrative—is politically costly and hard to catch in review. This risk-calibration lens explains why organizational-context tasks remain human even as technical tasks move to agents. The observation that under-delegation is now expensive is also notable: the cost of not using AI has become comparable to the cost of using it badly.

Connections

Raw Excerpt

“A bit over a year ago I wrote How I use LLMs as a staff engineer. Here’s a brief summary of what I used AI for last year: Smart autocomplete with Copilot, short tactical changes in areas I don’t know well (always reviewed by a SME), writing lots of use-once-and-throwaway research code.”