本文由 AI 分析生成
建立時間： 2026-05-30 來源： https://x.com/junfanzhu98/status/2040716119259164673
Summary
Recap of Reading Club Session 02 (April 2026), featuring a “JEPA Zoo” keynote surveying the family of Joint Embedding Predictive Architecture variants. The session reached consensus that reconstruction and planning are inherently conflicting objectives in latent space design, and that autonomous on-policy data collection will eventually dwarf third-party dataset purchasing. V-JEPA 2 achieves zero-shot deployment on Franka arms from 62 hours of robot data after internet-scale video pretraining.
第二期讀書會聚焦 JEPA 家族模型比較，核心結論是重建目標與規劃目標在潛在空間設計中天然衝突，應將重建僅用作後訓練診斷探針。自主 on-policy 數據採集將取代第三方數據集購買，成為機器人學習的未來正規範式。
Key Points
- V-JEPA 2: web-scale pretraining + 62h robot data → zero-shot pick-and-place on Franka (no reward, no env-specific training)
- Causal-JEPA introduces object masking in latent space to learn counterfactual “what if” states without explicit causal graphs
- LeWorldModel (15M params, single GPU) achieves latent prediction via only next-embedding prediction + SIGReg; criticized as “too extreme” — trajectory straightening degrades temporal semantics
- Reconstruction should not be a training objective but a post-training probe to verify latent semantics
- Decoupled world model proposal: split latent into static (geometry/scene) and dynamic (motion/forces) components
- Reward can emerge from latent manifold geometry (proportional distance over 100-step horizon) rather than trained reward heads
Insights
The “reconstruction vs planning conflict” is a fundamental architectural insight: the same latent space cannot simultaneously be ideal for recoverability (needs rich semantics) and planning (needs minimal sufficient dynamics). The practical implication is that most current world models are doing both poorly. The shift toward autonomous on-policy data echoes the AV (autonomous vehicle) data engine pattern — only by deploying robots and collecting real-world interaction can distribution shift be eliminated.
Connections
Raw Excerpt
World models > VLA because prediction forces physical understanding and enables better action generation (VLA is merely a context machine).