RoboCopilot: Human-in-the-Loop Interactive Imitation Learning

Summary

RoboCopilot (Abbeel lab, Berkeley, 2025) is a bilateral teleoperation system for bimanual robots that enables seamless real-time switching between human control and autonomous policy execution. By allowing humans to intervene mid-task and providing those interventions as on-policy training data, it implements hardware DAgger — solving covariate shift without offline retraining cycles.

RoboCopilot（Abbeel 實驗室，柏克萊，2025）是一個雙臂機器人的雙邊遠程操控系統，能在人類控制和自主策略執行之間無縫即時切換。通過允許人類在任務中途介入並將這些介入作為策略上訓練資料，它實現了硬體 DAgger，無需離線重訓練週期即可解決協方差漂移。

Key Points

• Bilateral (force-reflecting) teleoperation: human feels the robot’s contact forces — critical for dexterous tasks
• Seamless human ↔ policy control switching mid-task, no interruption
• Each human intervention = new on-policy training data (DAgger in hardware)
• Targets bimanual manipulation — higher complexity than single-arm setups
• Validated in simulation and hardware; faster skill acquisition than passive collection

Insights

• This is a hardware implementation of the insight behind DAgger: the distribution mismatch between demonstration-time and execution-time states is the core failure mode of behavior cloning, and the only real fix is to collect data at the states the policy actually visits
• Force reflection (bilateral teleoperation) is underutilized in most robot learning systems — most VR-based collection is kinematic only (position tracking). Transmitting forces back to the human gives the teacher information that pure position tracking cannot: whether the robot is encountering resistance, slipping, or making hard contact
• The “copilot” framing is meaningful: a copilot doesn’t take over the whole flight, just the moments where the pilot would otherwise crash. This selective intervention model is much more data-efficient than complete re-demonstrations
• The Abbeel lab context is significant: this is the team that pioneered imitation learning and DAgger; RoboCopilot is their production answer to the question “how do you deploy a partially-trained policy in the real world without catastrophic failures?”

Connections

• GR-Dexter: VLA for Bimanual Dexterous Robot Control
• How to Train Your Robots Demonstration Modality
• Imitation Learning
• DAgger
• Learning from Demonstration
• Bimanual Manipulation
• Teleoperation

Raw Excerpt

A compliant, bilateral teleoperation interface supports dynamic transitions between human control and autonomous robot execution, enabling interactive human teaching of complex bi-manual manipulation skills.