本文由 AI 分析生成
建立時間： 2026-05-30 來源： https://x.com/junfanzhu98/status/2045770010979905862
Summary
Session 04 of the Robotics World Model Reading Club (April 2026) examined world models as abstractions formalized as state transition dynamics p(s_{t+1}|s_t,a_t), then analyzed concrete systems including π0.7 VLA, model-based RL for drone flight (DreamerV3-style), tactile sensing, GaussGym for sim2real via 3D Gaussian Splatting, and the RMA two-phase locomotion adaptation framework. The session concluded with a hypothesis that mature robotics systems evolve toward modularity (MoE, separate high-level/low-level policies) rather than end-to-end monolithic architectures.
第四期讀書會以形式化的世界模型定義出發，分析了 π0.7 VLA 的模組化架構、基於潛在空間的無人機控制、觸覺感知噪音問題，以及 GaussGym 的 3DGS→mesh 轉換管道。核心假設是成熟的機器人系統最終走向模組化，而非端對端夢想。
Key Points
- World model formally: s_{t+1} ~ p(s_{t+1}|s_t, a_t); the key design question is what lives in s_t
- π0.7 uses SigLIP (400M) + Gemma (4B) for high-level policy, BAGEL (14B) as world model branch, and an 860M action expert
- Tactile sensing via microphones in grippers is cheaper than gel cameras but suffers from ambient noise; long-horizon tasks remain hard
- GaussGym: 3DGS → mesh conversion is mesh-precision sensitive — too coarse = inaccurate sim, too fine = real-time physics impossible
- RMA (Rapid Motor Adaptation): Phase 1 learns base policy with environment factor encoder; Phase 2 trains adaptation module running at 10Hz while base policy runs at 100Hz
- End-to-end architectures may be the “dream” — modularity (MoE, separate planners) is where production systems converge
Insights
The contrast between the photorealistic sleeping cat and the cartoon dangling cat is a useful framing: both are valid world models, but they preserve different structure for different downstream tasks. This reframes the debate about which representation (pixels, 3D, physics) is “correct” — the answer is always relative to the task. The dual-frequency control loop in RMA (10Hz adaptation + 100Hz base) is an elegant engineering solution to the latency-generalization tradeoff that the community often ignores.
Connections
Raw Excerpt
Robotics may need multiple world models, each preserving the structure required by its downstream task — whether pixel-level for semantics, 3D for kinematics, physics-augmented for control, or multimodal for rich interaction.