本文由 AI 分析生成
建立時間： 2026-05-30 來源： https://x.com/junfanzhu98/status/2050095287352635652
Summary
A structured summary of Yann LeCun’s theoretical framework for why next-token prediction is insufficient for grounded intelligence, and how JEPA (Joint Embedding Predictive Architecture) addresses this by predicting in learned representation spaces conditioned on actions. LeCun argues that LLMs are powerful token distribution estimators but lack the four components of genuine agency: grounded state representation, hypothetical actions, a world model, and a planner. Intelligence requires hierarchical world models operating across multiple temporal and abstraction scales.
LeCun 主張下一詞預測不足以實現具身智能，JEPA 透過在學習表示空間中進行動作條件預測來解決這個問題。真正的 Agent 需要四個組件：接地狀態表示、假設動作、世界模型和規劃器，而 LLM 原生不具備這些結構。
Key Points
- LLMs approximate P(token|context) over finite vocabulary — this breaks for continuous, high-entropy sensory streams like video
- JEPA trains z_{t+1} = f_θ(z_t, a_t) where collapse is prevented via variance/covariance regularization on embeddings
- Self-supervised learning is the “bulk of compute” layer; RL is “the cherry” on top — not the foundation
- Chain-of-Thought increases compute but does not constitute causal reasoning; most LLM successes are sophisticated information retrieval
- Hierarchical world models across temporal scales are the central unsolved problem — lower layers handle geometry/motion, higher layers handle intent/plans
- A 4-year-old accumulates ~16,000 hours of visual input at ~2GB/s; language is a thin abstraction layer built atop this
Insights
LeCun’s framing of consciousness as a consequence of having a single “engine cortex” that can run one complex simulation at a time is an unusual but mechanistically coherent perspective. It maps onto the System 1 (RL policies, parallel, subconscious) vs. System 2 (serial world-model planning) distinction in a concrete neural architecture claim. The prediction that foundation models will commoditize like Linux — with long-term value shifting to applications and specialized agents — has significant implications for which companies will capture value in the AI stack.
Connections
Raw Excerpt
LLMs are powerful estimators of token distributions over language abstractions, but grounded intelligence requires action-conditioned latent world models that learn equivalence classes of future states under physical invariances, organized hierarchically, and trained to minimize objective-driven prediction error.