Survey: Human-Robot Collaboration through Vision-and-Language Navigation
Authors: Nivedan Yakolli, Avinash Gautam, Abhijit Das, Yuankai Qi, Virendra Singh Shekhawat Reviewed: ~200 articles
Overview
VLN (Vision-and-Language Navigation) is a multimodal task where agents interpret human language instructions to navigate 3D environments. This survey focuses on how VLN advances human-robot collaboration.
Key Gaps Identified
Current VLN/HRC systems struggle with:
- Bidirectional communication — most systems only receive instructions, can’t ask clarifying questions
- Ambiguity resolution — unclear instructions cause navigation failures with no recovery mechanism
- Collaborative decision-making in multi-agent systems — multiple robots lack coordination frameworks
Recommendations
Systems should incorporate:
- Proactive clarification mechanisms (robot asks when uncertain)
- Real-time feedback capabilities
- Contextual reasoning through advanced natural language understanding
- Decentralized decision-making with dynamic role assignment
Application Domains
Healthcare, logistics, disaster response — environments where HRC failures have high stakes and bidirectional communication is essential.