arXiv: https://arxiv.org/abs/2604.02029 | cs.AI | 2026-04-02
Abstract
Latent space is emerging as a fundamental substrate for language-based models, yet modern systems continue relying on explicit token-level generation. The survey argues that continuous latent space enables critical internal processes more effectively than human-readable verbal traces. Key limitations of explicit-space computation include linguistic redundancy, discretization bottlenecks, sequential inefficiency, and semantic loss.
The research landscape has expanded from early latent reasoning into planning, modeling, perception, memory, collaboration, and embodiment. This comprehensive survey organizes knowledge across five dimensions:
- Foundation: Defining latent space, distinguishing it from explicit/verbal space and visual generative model latent spaces
- Evolution: Tracing development from prototype exploration through formation, expansion, to current outbreak stage
- Mechanism: Analyzing architecture, representation, computation, and optimization approaches
- Ability: Examining reasoning, planning, modeling, perception, memory, collaboration, and embodiment capabilities
- Outlook: Discussing challenges and future research directions
Foundation (Section 2)
Latent space represents continuous, learned representations where models encode information not explicitly verbalized. Unlike explicit space’s discrete tokens, latent representations provide continuous, flexible, efficient substrates that preserve high-fidelity semantic information. Critical differences include:
- Representational: Machine-native vectorized form versus human-readable tokens
- Computational: Continuous and flexible versus discrete and symbolic
- Efficiency: Bypasses linguistic redundancy and sequential decoding overhead
- Information: Preserves high-fidelity content without discretization losses
Evolution (Section 3)
Four developmental stages characterize the field’s trajectory:
- Prototype (Pre-Mar 2025): Early validation and exploration establishing feasibility
- Formation (Apr–Jul 2025): Theoretical systematization and technical foundations with text focus
- Expansion (Aug–Nov 2025): Rapid diversification into vision, embodiment, and multi-agent systems
- Outbreak (Dec 2025–Present): Architectural specialization, optimization sophistication, and unified frameworks
Representative works progressed from COCONUT’s continuous thought loops through domain-specific innovations like MemGen (latent memory), Mirage (visual thinking), and UniVLA (embodied actions).
Mechanism (Section 4)
Four complementary mechanisms explain how latent space operates:
Architecture: How latent space integrates into models through backbone modifications, component augmentation, or auxiliary models. Backbone approaches embed computation natively; component-based methods preserve original architecture while adding latent functionality; auxiliary models provide supervision signals.
Representation: Forms include internal (hidden states), external (learned embeddings), learnable (trained parameters), and hybrid combinations.
Computation: Patterns span compressed (efficient), expanded (more capacity), adaptive (dynamic allocation), and interleaved (alternating between spaces) approaches.
Optimization: Mechanisms operating across pre-training, post-training, and inference stages to induce, align, or refine latent representations.
Ability (Section 5)
Seven capability domains emerge where latent computation excels:
- Reasoning: Internal thought processes without verbalization
- Planning: Multi-step trajectory generation
- Modeling: Environment and world representation
- Perception: Visual and multimodal understanding
- Memory: Experience retention and retrieval
- Collaboration: Multi-agent communication and coordination
- Embodiment: Robotics and physical action generation
Outlook (Section 6)
Key challenges include evaluability constraints, interpretability difficulties, standardization needs across modalities, and integrating latent systems into broader agentic architectures. Future directions emphasize consolidating fragmented techniques into coherent frameworks, establishing principled evaluation criteria, and developing unified latent interfaces.
Contributions
The survey clarifies conceptual scope, provides unified evolution review, introduces a two-dimensional taxonomy (Mechanism × Ability), and offers comprehensive resources for continued research.