HITL and VLA Surveys: Human-in-the-Loop Robot Learning
Sources: Research synthesis, 2026-06-15 Raw: Survey of Recent Reviews: HITL and HRI Updated: 2026-08-12
Overview
The HITL and HRI survey literature from 2023-2026 reflects a field split between two communities that rarely cite each other: robot learning researchers who study how humans provide training data (LfD, interactive IL, HITL), and HRI/CHI researchers who study how humans experience and interact with robots at deployment time. All major surveys converge on the same root cause (covariate shift → need to close the loop), the same architectural gap (bidirectionality), and the same theoretical problem (representation alignment). The engineering integration that connects these components has not been built.
Five Key Surveys
Interactive IL for Dexterous Manipulation (arXiv:2506.00098, 2025): most complete taxonomy of the DAgger solution space for high-DOF dexterous manipulation hardware. Shows that RoboCopilot (2025) deploys hardware DAgger with bilateral teleoperation in real settings.
Pure VLA Models (arXiv:2509.19012, 2025): 300+ papers, five-paradigm taxonomy. Treats human data as training input for scaling VLAs — not as interactive feedback. The gap: doesn’t integrate insights from HITL literature about what makes feedback valuable (on-policy correction, embodied contact, real-time intervention in failure states).
Human Video → Robot VLA (arXiv:2606.00054, 2026): four-family taxonomy for approaches that extract action supervision from passive human video — decoupling robot learning from active teleoperation infrastructure.
LLM+HRI Systematic Review (arXiv:2602.15063, CHI 2026): PRISMA review of 86 papers, proposes SIA (Sense-Interaction-Alignment) framework. LLMs are the dominant enabling technology. Identifies: fragmented research, no standardized evaluation, almost no longitudinal studies, underinvestment in user modeling.
VLN for Human-Robot Collaboration (arXiv:2512.00027, 2025): ~200 papers on vision-language navigation for HRC. Critical finding: bidirectionality gap — robots cannot ask clarifying questions or signal uncertainty. HRI is currently a command-execution pipeline, not genuine interaction.
The Shared Diagnosis: Covariate Shift and Closing the Loop
Every major survey identifies the same root problem: passive behavioral cloning fails because the trained policy visits states not covered by the demonstration distribution. The solution space is well-mapped:
- DAgger and variants (iterative on-policy correction): the canonical approach since 2011; still the standard
- Diffusion policies (full distribution modeling instead of point-estimate averaging): standard action model for contact-rich tasks
- Real-time HITL systems (RoboCopilot): humans intervene mid-task; every intervention is training data
The VLA Community’s Blind Spot
VLA surveys frame human data as a scaling resource — more data → bigger pretraining corpus → better generalization. This misses the insight from interactive IL: on-policy correction during deployment is qualitatively different from passive demonstration data. A policy that sees corrections in the states it actually fails in learns faster and more robustly than one that only sees expert success demonstrations.
The practical gap: no VLA training pipeline currently incorporates deployment-time HITL corrections into the pretraining or fine-tuning cycle.
The Bidirectionality Gap
Every major survey identifies the same structural gap from different angles:
- VLN survey: robots cannot ask clarifying questions or signal uncertainty
- CHI review: almost no longitudinal studies of human adaptation to robots over time
- Multimodal perception survey: generalization from lab to real deployment fails; human variability is not modeled
- Communicating robot learning survey: the robot-to-human feedback channel is systematically underbuilt compared to the human-to-robot channel
Building bidirectionality requires: uncertainty quantification (when does the robot not know?), proactive communication design (how to ask without being annoying?), and dialogue turn-taking coherence.
The Representation Alignment Problem
Bobu et al. (arXiv:2302.01928, 2023) reframe the HITL problem at a theoretical level: human and robot representations are misaligned at the feature level. Even a perfect learning algorithm produces wrong behavior if the robot’s features don’t capture what the human cares about.
This means:
- Human feedback is better understood as a probe for representation gaps than as a training signal for task values
- VLA pretraining on internet data may produce partially aligned representations — but explicit alignment objectives are likely needed
- Correction data during deployment should target which features the robot uses, not just which actions it takes
The Near-Term Synthesis
The convergence of interactive IL, VLA scaling, and LLM-enabled HRI suggests a near-term architecture:
- VLA pretrained on passive human video data (no active teleoperation required for base pretraining)
- Fine-tuned via real-time human interventions during deployment (DAgger-style on-policy correction)
- Closes the robot-to-human feedback loop (uncertainty signaling, proactive clarification)
No current deployed system achieves all three. The survey literature documents the components; the integration is the open research problem.
Open Questions
- When VLAs are fine-tuned with RL, what is the correct role for human feedback vs. environment reward? Is RLHF for robot VLAs the next major paradigm?
- How do you build bidirectionality into VLA systems without disrupting task flow?
- Can representation alignment (Bobu et al.) be made practical for 7B-parameter VLAs? Methodologically challenging.
- What does longitudinal human-robot collaboration look like after 6 months? No data exists.