VAM-HRI 2025 Research Report
8th Workshop on Virtual, Augmented, and Mixed Reality for Human-Robot Interaction HRI 2025 · Melbourne, Australia · March 2025 官方頁面：https://vam-hri.github.io/program/
Workshop Overview
VAM-HRI 是 HRI 大會的旗艦附屬工作坊，專注於 VR/AR/MR 技術在人機互動中的應用研究。2025 年共收錄 11 篇論文，主題涵蓋遠端操控、示範學習資料收集、工業維護、醫療復健，以及新型 AR 顯示技術。
VAM-HRI is the flagship workshop at the ACM/IEEE HRI conference, focusing on immersive technology for human-robot interaction. The 2025 edition features 11 papers spanning teleoperation, learning-from-demonstration data collection, industrial maintenance, medical rehabilitation, and novel AR display methods.
Paper Summaries
#1 MiXR-Interact: Mixed Reality Interaction Dataset for Gaze, Hand, and Body
Authors: Takele, Delehelle, Kim, Tefera, Deshpande, Caldwell, Ortiz, Recchiuto Institution: Italian Institute of Technology (IIT) Source: https://hal.science/hal-05028351
中文摘要： 基於 Meta Quest Pro 建立的 MR 互動資料集，記錄使用者的視線、上半身動作與手勢。定義三種核心互動類型（推、指、抓），跨六個方向執行，標注 17 個關鍵接觸點。目標是為 MR 環境中的 HCI 研究提供高品質的動作追蹤基準資料。
English: A multimodal MR interaction dataset using Meta Quest Pro, capturing gaze, upper-body, and hand gesture data. Defines three interaction primitives (push, point, grasp) across six spatial directions with 17 annotated contact keypoints. Targets benchmarking needs in MR-based HCI research.
Key Contribution: Open dataset + standardized MR interaction evaluation framework
#2 Enhancing Shared Control for Telepresence in Dynamic Environments using Large Language Models
Authors: Jain, Kanade, Sebastian, Muniyandi
中文摘要： 將大型語言模型整合進共享控制遠端臨場（telepresence）系統，使機器人能在動態環境中更好地輔助操作者決策。透過 LLM 的場景理解能力提升系統在非結構化環境中的適應性。
English: Integrates LLMs into a shared-control telepresence pipeline to assist operators navigating dynamic, unstructured environments. Leverages language model scene understanding to improve adaptability beyond classical shared-control heuristics.
Key Contribution: LLM × shared control × telepresence three-way integration
#3 Enhancing Robotic Manipulation: AR-Powered Data Collection for Learning from Demonstration
Authors: Malagalage Don, Asadi Source: https://openreview.net/forum?id=XkZwSCP1vd
中文摘要： 比較五種示範資料收集方式（兩種傳統方法、兩種 AR 輔助傳統方法、一種純 AR 方法）對四種機器學習模型的訓練效果。結果顯示 AR 輔助方法全面優於純傳統方法，純 AR 方法效能接近最佳 AR 輔助方法，且無傳統方法的操作限制。
English: Comparative study of five data collection strategies for LfD: two traditional, two AR-augmented traditional, and one pure AR. AR-augmented methods consistently outperformed traditional baselines across four ML models; pure AR nearly matched the best AR-augmented method with fewer ergonomic constraints.
Key Contribution: Quantitative evidence that AR augmentation improves LfD data quality
#4 SoftBiT: Soft Bimanual Teleoperation with Proprioceptive Visual Augmentation
Authors: Inui, Yoo, Ichnowski, Oh Institution: Waseda University / CMU Source: https://openreview.net/forum?id=Eq5n24wGqO
中文摘要： 針對軟體機器人手指在抓取時常被物體遮擋的視覺反饋問題，提出在 Meta Quest 2 上即時視覺化手指形變的 XR 系統。透過 sim-to-real pipeline 估計軟體手指形狀，疊加到使用者視野中，提升雙手臂遙操作的本體感知。驗證任務：取放、組裝、物件形變。
English: Addresses occlusion-driven loss of proprioceptive feedback in soft robotic teleoperation. Uses Meta Quest 2 to overlay real-time estimated soft finger deformations onto the operator’s view via a sim-to-real shape estimation pipeline. Demonstrated on pick-and-place, assembly, and deformation tasks.
Key Contribution: Novel combination of soft robotics + XR proprioceptive visualization
#5 Mixed Reality Meets Robotic Systems: A HoloLens2-Enabled Waypoint Navigation Interface
Authors: Fang, Xu, Lin, Gilbert, Panagou
中文摘要： 以 HoloLens 2 為平台設計機器人路徑點導航介面，使操作者能直接在真實環境中以 MR 互動方式設定、調整機器人導航目標，提升空間直覺性，減少傳統 2D 介面的座標系心智負擔。
English: Develops a waypoint navigation interface using HoloLens 2, allowing operators to place and adjust robot navigation goals directly within the physical environment through MR interaction, improving spatial intuitiveness over screen-based interfaces.
Key Contribution: In-situ MR navigation goal input, reducing coordinate frame cognitive overhead
#6 Usability Study of VR Interfaces for Learning from Demonstrations in Bimanual Tasks
Authors: Saber, Baraka, Hou
中文摘要： 針對雙手臂任務示範收集的 VR 介面進行使用性研究，評估不同 VR 介面設計對示範資料品質與使用者體驗的影響。
English: User study evaluating VR-based interfaces for bimanual LfD data collection, assessing how different interface designs affect demonstration quality and operator experience in two-arm manipulation tasks.
Key Contribution: Human factors evaluation data for VR teleoperation interfaces
#7 LLM-Supported Safety Annotation in High-Risk Environments
Authors: Eskandari, Indukuri, Lukin, Matuszek Institution: UMBC / U.S. Army Research Laboratory
中文摘要： 探索使用 LLM 協助在高風險環境中自動生成安全標注，減少人工標注成本。針對機器人在危險場景（如爆炸物處置、工業危場）的感知資料做安全相關語義標注。
English: Explores using LLMs to automate safety-relevant annotation of robot perception data in high-risk environments (e.g., EOD, industrial hazard zones), reducing costly human labeling effort. Targets safety-aware robot learning applications.
Key Contribution: LLM-assisted safety annotation, lowering data collection barriers in high-risk scenarios
#8 A Controller for Robots to Autonomously Control Fog Machines
Authors: Lozada, Keth, Tijani, Klein, Han Institution: RARE Lab Source: https://openreview.net/forum?id=DCcX6S0s5n
中文摘要： 解決傳統霧機只能手動操作、機器人無法介接的問題。以自製 PCB（Arduino + 閂鎖繼電器 + 整流電路）取代原廠手持遙控器，實作 ROS 介面讓機器人能自主控制霧機噴霧。目標應用是以霧幕作為 AR 投影媒介（mid-air fog screen）實現空中浮空顯示。
English: Replaces a fog machine’s manual remote with a custom PCB (Arduino + latching relay + rectifier) exposing a ROS programming interface. Enables robots to autonomously trigger fog emission for mid-air fog screen AR displays — projecting AR content onto airborne fog particles.
Key Contribution: First ROS-enabled fog machine controller, opening new direction for mid-air AR displays
#9 User-Centric Mixed Reality Interventions for Parkinson’s Tremor Management: A Path Toward Digital Therapeutics
Authors: Li, Lei Source: https://openreview.net/pdf?id=jy8FfbQGey
中文摘要： 提出 MR 輔助帕金森氏症顫抖復健系統，整合人體工學手部支撐機構與互動式 MR 練習。系統設計透過醫療專業人員與患者的回饋迭代優化。研究顯示 77% 的帕金森患者對傳統復健方式感到挫折，MR 環境顯著提升參與動機。
English: Presents an MR-assisted tremor rehabilitation system for Parkinson’s disease, combining an ergonomic hand-support mechanism with interactive MR exercises. Designed with clinician and patient feedback. Addresses that 77% of PD patients express frustration with traditional rehabilitation monotony; prior VR/AI therapy exposure correlated with higher motivation.
Key Contribution: Human-centered design methodology for medical XR rehabilitation
#10 Augmented Reality for Human Decision Making and Human-Robot Collaboration: A Case Study in Gasket Room in Manufacturing
Author: Liu
中文摘要： 以製造業墊圈室（Gasket Room）為案例，研究 AR 如何輔助人類決策與人機協作，評估 AR 視覺化資訊對工人在複雜製造環境中操作效率的影響。
English: Case study examining how AR visualization aids human decision-making and HRC in a manufacturing gasket room. Evaluates the impact of AR-overlaid operational information on worker efficiency in a complex industrial environment.
Key Contribution: Field evaluation of AR-HRC in industrial manufacturing settings
#11 To Err is Humanoid; to Collaborate, Divine: A Transitional Reality Interface for Error Replay and Correction in Industrial Robotics
Authors: Marshall, Allspaw, Yanco Institution: UMass Lowell Robotics Lab
中文摘要： 提出「漸進現實介面」（Transitional Reality Interface），讓操作者能在 AR 環境中重播機器人錯誤動作並直接修正，無需重新操作真實機器人。透過 AR 的錯誤可視化與互動式修正，降低工業機器人維護成本與停機時間。
English: Proposes a Transitional Reality interface allowing operators to replay robot errors in AR and apply corrections without re-executing on physical hardware. Targets industrial robotics maintenance and fault correction workflows, reducing downtime by enabling AR-mediated error diagnosis and remediation.
Key Contribution: AR error replay + correction interface, a new paradigm for industrial robot maintenance
Thematic Analysis
Five Research Axes
| Theme | Papers |
|---|---|
| Teleoperation & XR Interfaces | #2, #4, #5, #6 |
| LfD Data Collection | #3, #4, #6 |
| LLM × XR Integration | #2, #7 |
| Industrial / Manufacturing Applications | #10, #11 |
| Medical / Novel Applications | #9, #8 |
Observed Trends
- LLMs entering XR-HRI: Papers #2 and #7 both integrate language models as core components, marking the VAM-HRI community’s adoption of foundation models.
- LfD data quality as a central concern: Papers #3, #4, #6 all address “how to collect better demonstration data with VR/AR,” directly responding to the robot learning data bottleneck.
- Shift toward industrial deployment: Papers #10, #11 are set in real factory environments, signaling a move from lab demos to field validation.
- Meta Quest as de facto research hardware: #4 uses Quest 2, #1 uses Quest Pro — consumer-grade VR headsets have become the standard VAM-HRI research platform.
- Novel AR display media: Paper #8’s fog screen AR is a rare non-traditional display research direction with potential for outdoor or large-space applications.