Multimodal Perception-Driven Decision-Making for HRI: A Survey
Authors: Wenzheng Zhao, Kruthika Gangaraju, Fengpei Yuan (Worcester Polytechnic Institute) Published: Frontiers in Robotics and AI, August 2025 Coverage: 2004–2024
Overview
Reviews how robots integrate diverse sensory data — vision, language, and tactile — to understand environments and interact with humans. Focuses on Multimodal Perception-Driven Decision-Making (MPDDM) frameworks.
Application Domains
- Social and Assistive Robotics — emotion recognition, companion robots, healthcare assistance, rehabilitation
- Navigation and Mobile Robotics — obstacle avoidance, socially-aware autonomous movement
- Industrial Collaborative Robotics — worker safety, object manipulation
- General-Purpose Robotics — complex task understanding, adaptive planning
Major Challenges
- Sensor fusion complexity — noise and integration across modalities
- Generalization — learned behaviors don’t transfer to unseen environments
- Safety and robustness — reliable performance in dynamic human spaces
- Computational demands — real-time multimodal processing
- Human variability — unpredictable behavior, cultural differences in social interaction norms
Future Directions
- Adaptive multimodal fusion techniques
- More efficient learning paradigms
- Human-trusted decision-making frameworks