Quartz 5

Survey of Recent Reviews: Human-in-the-Loop and Human-Robot Interaction

Properties1
tagsrobotics, hri, human-in-the-loop, vla, lfd, survey

13 min read

Survey of Recent Reviews: Human-in-the-Loop and Human-Robot Interaction

Research Question

What survey and review papers from 2023–2026 cover human-in-the-loop (HITL) learning and human-robot interaction (HRI), particularly at the intersection of Vision-Language-Action (VLA) models, Learning from Demonstration (LfD), and robot learning with human feedback?

Knowledge Map

  • Behavior cloning / imitation learning — the foundational learning paradigm for all LfD systems; understanding its failure mode (covariate shift / distribution mismatch) explains why HITL methods exist
  • DAgger and interactive IL — the theoretical bridge between passive LfD and HITL; DAgger (Ross et al., 2011) is the canonical algorithm; all HITL systems are variants
  • VLA architectures — vision-language models extended with action heads; understanding how VLAs encode and decode actions (autoregressive vs. diffusion) is required to interpret survey findings about human data integration
  • Human feedback signals — the variety of feedback humans provide: kinesthetic correction, preference comparisons (RLHF), real-time teleoperation, natural language instruction — each carries different information and requires different learning machinery
  • Cross-embodiment transfer — why human demonstrations don’t trivially map to robot execution; the morphology gap is a structural challenge for all human-centric robot learning
  • Multimodal perception in HRI — robots need to perceive human intent from vision, language, gesture, and touch; this is the sensing substrate that makes bidirectional HRI possible
  • Trust and autonomy in HRI — the human side of the equation: how trust forms, at what autonomy level robots should operate, and how bidirectional communication changes human behavior and teaching quality

Sources Gathered

New sources clipped and analyzed during this research:

Existing vault notes referenced:

Key Findings

1. The HITL-LfD literature is converging on a shared diagnosis: covariate shift, solved by closing the loop

All recent survey papers identify the same root problem: passive behavior cloning fails because the trained policy visits states not covered by the demonstration distribution. The solution space is now well-mapped — DAgger and variants (iterative on-policy correction), diffusion policies (full distribution modeling instead of averaging), and real-time HITL systems (RoboCopilot: humans intervene mid-task, every intervention is training data). The interactive IL survey (arXiv:2506.00098) provides the most complete taxonomy of this solution space specifically for high-DOF dexterous manipulation.

2. VLA surveys are growing fast but treat human data as a scaling input, not an interactive signal

The pure VLA survey (arXiv:2509.19012, 300+ papers) and the human video survey (arXiv:2606.00054) both frame human data primarily as a training input to scale VLAs — not as an ongoing interactive feedback mechanism. This is a meaningful gap: the HITL robotics literature shows that interactive feedback during deployment is qualitatively different from passive demonstration data, yet VLA surveys don’t integrate this insight. The five VLA paradigms (autoregression, diffusion, RL, hybrid, specialized) map out how to model actions but say little about how to keep humans in the loop after deployment.

3. LLMs have restructured the HRI research agenda without solving its core human-centered challenges

The CHI 2026 systematic review (arXiv:2602.15063, 86 papers) documents that LLMs are now the dominant enabling technology for HRI, replacing scripted interaction patterns with contextual sensing, generative dialogue, and continuous alignment. The proposed SIA (Sense-Interaction-Alignment) framework captures this shift. But the review also finds that the research is fragmented, lacks longitudinal studies, has no standardized evaluation metrics, and underinvests in user modeling. LLMs solved the language interface problem; they did not solve trust, appropriate autonomy, or bidirectionality.

4. Bidirectionality is the persistent design gap across all HRI survey threads

Every major survey identifies the same structural gap from a different angle:

  • VLN survey (arXiv:2512.00027): robots cannot ask clarifying questions or signal uncertainty — HRI is currently a command-execution pipeline, not true interaction
  • CHI review (arXiv:2602.15063): almost no longitudinal studies; human adaptation to robots over time is not studied
  • Multimodal perception survey (Frontiers 2025): generalization from lab to real deployment fails; human variability is not modeled
  • Communicating robot learning survey (arXiv:2312.00948): the robot-to-human feedback channel is systematically underbuilt compared to the human-to-robot channel

5. The representation alignment problem underlies all these gaps

Bobu et al. (arXiv:2302.01928) provide the most principled theoretical account of why HITL and HRI remain hard: human and robot representations are misaligned at the feature level. Even a perfect learning algorithm will produce wrong behavior if the robot’s features don’t capture what the human cares about. This reframes human feedback not as a training signal for task values but as a probe for representation gaps. VLA pretraining on internet data may produce representations that are partially aligned with human concepts — but the surveys don’t directly address this.

Open Questions

  • When VLAs are fine-tuned with RL (the emerging reinforcement-based paradigm), what is the right role for human feedback vs. environment reward? Is RLHF for robot VLAs the next major paradigm?
  • The human video survey identifies 3D reconstruction approaches as promising but fragile. As reconstruction quality improves, will the embodiment gap close, or are there fundamental limits from kinematic incompatibility?
  • How do you build bidirectionality (robot asks clarifying questions, signals uncertainty) into VLA systems without disrupting task flow? The VLN survey identifies this as critical but proposes no solution.
  • The CHI review finds almost no longitudinal HRI studies. What does human-robot collaboration look like after 6 months of daily use? Trust and mental model formation over time are unexplored.
  • Can representation alignment (Bobu et al.) be made practical for VLAs? Probing a 7B-parameter VLA for feature alignment is computationally and methodologically challenging.

Report

The HITL and HRI survey literature in 2023–2026 reflects a field that has achieved substantial technical capability while remaining structurally limited by its treatment of humans. The surveys divide cleanly into two communities that rarely cite each other: robot learning researchers who study how humans provide training data (LfD, interactive IL, HITL), and HRI/CHI researchers who study how humans experience and interact with robots at deployment time. Recent survey papers are beginning to bridge this divide, but the synthesis is incomplete.

The robot learning side is technically mature. The interactive IL survey (arXiv:2506.00098) shows that the core DAgger framework from 2011 has been extended, hardened, and deployed on dexterous manipulation hardware (RoboCopilot, 2025). Diffusion policies are now the standard action model for contact-rich tasks. The remaining questions are practical: when is human intervention economically justified, how do you scale interactive correction to diverse robot hardware, and how do you filter low-quality interventions from operators who don’t understand the task physics.

The VLA side is technically ambitious but human-data-naive. Both VLA surveys treat human data as a scaling resource. The human video survey (arXiv:2606.00054) goes furthest by asking how to extract action supervision from passive human video — a compelling direction because it decouples robot learning from active teleoperation infrastructure. But neither survey integrates insights from the interactive IL literature about what makes human feedback valuable beyond its volume: on-policy correction, embodied contact information, and real-time interventions in states the autonomous policy fails to handle.

The HRI/CHI side has a clear diagnosis but incomplete prescriptions. The CHI systematic review (arXiv:2602.15063) proposes the SIA framework (Sense-Interaction-Alignment) as a replacement for the classical Sense-Plan-Act model. This is accurate as a description of what LLMs enable — contextual sensing, generative interaction, continuous alignment — but does not explain how to build systems that achieve the alignment layer in practice. The bidirectionality gap identified across multiple surveys (VLN: arXiv:2512.00027; multimodal: Frontiers 2025) is well-documented but solutions remain research proposals, not deployed systems.

Representation alignment is the deepest theoretical thread. Bobu et al.’s framework (arXiv:2302.01928) argues that the human-to-robot teaching problem is fundamentally a representation problem: the robot must learn to represent the world in terms that match what humans care about, before task learning can succeed. This reframes all HITL systems as implicit alignment mechanisms and suggests that future VLA training should include explicit representation alignment objectives — probing what concepts the model has learned and correcting gaps before fine-tuning on task demonstrations.

Where the field is going. The convergence of interactive IL, VLA scaling, and LLM-enabled HRI suggests a near-term synthesis: VLAs that are pre-trained on human video data (no active data collection), fine-tuned interactively via real-time human interventions during deployment (DAgger-style on-policy correction), and communicate their uncertainty and learned state back to human partners (closing the robot-to-human feedback loop). No current deployed system achieves all three. The survey literature documents the components; the engineering integration is the open research problem.

中文版

研究問題

2023–2026 年有哪些 HITL 與 HRI 的調查和回顧論文，特別是在 VLA 模型、示範學習（LfD）和人類回饋機器人學習交叉點上？

知識地圖

  • 行為克隆 / 模仿學習 — LfD 的基礎學習範式；理解其失敗模式（協方差漂移）解釋了 HITL 方法存在的原因
  • DAgger 與互動式 IL — 被動 LfD 和 HITL 之間的理論橋樑；所有 HITL 系統都是 DAgger 的變體
  • VLA 架構 — 具有動作頭的視覺語言模型；理解 VLA 如何編碼和解碼動作（自回歸 vs. 擴散）
  • 人類回饋信號 — 人類提供的各種回饋：動覺糾正、偏好比較（RLHF）、即時遠程操控、自然語言指令
  • 跨體態遷移 — 人類示範為何不能直接映射到機器人執行；形態差距是所有人類中心機器人學習的結構性挑戰
  • HRI 中的多模態感知 — 機器人需要從視覺、語言、手勢和觸覺感知人類意圖
  • HRI 中的信任與自主性 — 信任如何形成、機器人應在何種自主程度下運作

關鍵發現

1. HITL-LfD 文獻在共同診斷上趨於一致：協方差漂移，通過閉合反饋迴路解決

所有近期調查論文都指向同一根本問題：被動行為克隆失敗，因為訓練後的策略訪問示範分佈未覆蓋的狀態。解決方案空間現在已被清晰映射——DAgger 及其變體、擴散策略、即時 HITL 系統（RoboCopilot）。

2. VLA 調查快速增長，但將人類資料視為擴展輸入而非互動信號

純 VLA 調查（arXiv:2509.19012，300+ 篇）和人類影片調查（arXiv:2606.00054）都主要將人類資料視為訓練輸入來擴展 VLA，而非持續的互動回饋機制。這是一個有意義的缺口。

3. LLM 重構了 HRI 研究議程但未解決其核心以人為本的挑戰

CHI 2026 系統性回顧（arXiv:2602.15063，86 篇）記錄了 LLM 現在是 HRI 的主導使能技術。提出的 SIA 框架捕捉了這一轉變——但研究碎片化，缺乏縱向研究，沒有標準化評估指標。

4. 雙向性是所有 HRI 調查線索中持續存在的設計缺口

每個主要調查從不同角度識別出同一結構性缺口：機器人無法提問澄清或傳達不確定性——HRI 目前是指令執行管道，而非真正的互動。

5. 表徵對齊問題是所有這些缺口的底層根因

Bobu et al. 提供了最有原則性的理論解釋：人類和機器人的表徵在特徵層面是不對齊的。即使是完美的學習算法，如果機器人特徵不捕捉人類真正在意的，也會產生錯誤行為。

未解問題

  • 當 VLA 以 RL 微調時，人類回饋 vs. 環境獎勵的正確角色是什麼？機器人 VLA 的 RLHF 是否是下一個主要範式？
  • 如何在不干擾任務流的情況下將雙向性（機器人提問、傳達不確定性）內建到 VLA 系統中？
  • 沒有縱向 HRI 研究：6 個月日常使用後人機協作是什麼樣子？

報告

HITL 和 HRI 調查文獻在 2023–2026 年反映了一個在技術能力上取得重大進展，但在對人類的處理上仍存在結構性限制的領域。調查明顯分為兩個很少互相引用的社群：研究人類如何提供訓練資料的機器人學習研究者（LfD、互動式 IL、HITL），以及研究人類在部署時如何體驗和互動機器人的 HRI/CHI 研究者。

近期調查的趨勢指向合成的方向：以人類影片資料預訓練的 VLA（無需主動資料收集），通過部署時即時人類干預互動式微調（DAgger 式策略上修正），以及將其不確定性和學習狀態傳達給人類夥伴（閉合機器人到人類的回饋迴路）。目前沒有任何部署系統實現所有三者。調查文獻記錄了各個組件；工程整合是開放的研究問題。

Graph View

Backlinks