OPEN TEACH: A Versatile Teleoperation System for Robotic Manipulation

本文由 AI 分析生成

建立時間： 2026-03-26 來源： https://arxiv.org/abs/2403.07870

Summary

OPEN TEACH is an open-source teleoperation platform built on the Meta Quest 3 VR headset ($500 consumer hardware) enabling natural hand-gesture control of diverse robot configurations at 90Hz. Supports Franka/xArm/Jaco arms, Allegro dexterous hands, single-arm, dual-arm (bimanual), and mobile manipulator setups. Validated on 38 tasks with 10 contact-rich manipulation challenges for imitation learning policy evaluation. User studies show significant improvements over existing frameworks.

OPEN TEACH 是一個基於 Meta Quest 3 VR 頭戴顯示器（500 美元消費級硬件）的開源遙操作平台，以 90Hz 通過自然手勢控制各種機器人配置。支持 Franka/xArm/Jaco 手臂、Allegro 靈巧手，以及單臂、雙臂（雙手操作）和移動機械臂設置。在 38 個任務上驗證，包括 10 個接觸豐富的操作挑戰。

Prerequisites

• VR headset hand tracking — Meta Quest 3 uses optical hand tracking; understanding the tracking accuracy and latency characteristics matters for evaluating demonstration quality
• Robot arm teleoperation — end-effector control vs. joint control, workspace mapping, latency requirements
• Dexterous hand control — Allegro hand has 16 DOF; mapping human finger poses to robot finger joints is non-trivial
• Bimanual manipulation — coordinating two arms requires managing inter-arm collisions and coordination; the cognitive load on the operator increases substantially

Core Idea

The $500MetaQuest3consumerVRheadsethassufficienthandtrackingaccuracyfordexterousrobotteleoperation.OPENTEAC{H}^{\prime }scoreargument:professionalmotioncapturesetups($10k-50k) are not necessary for high-quality demonstration collection. By building on consumer hardware and open-sourcing everything, the system democratizes access to dexterous data collection. The 90Hz control rate is high enough for contact-rich manipulation; natural hand gesture interface reduces operator training time vs. joystick or spacemouse interfaces.

Results

Metric	Value
Supported platforms	Franka, xArm, Jaco + Allegro
Control frequency	90Hz
Tasks demonstrated	38 total
Contact-rich IL benchmarks	10 validated
Hardware cost	~$500 (Quest 3)

• User studies: significant improvement over existing frameworks on task success and operator comfort
• Bimanual tasks demonstrated with single operator wearing headset
• Mobile manipulator support (robot arm on wheeled base)

Limitations

• Author-stated: hand tracking accuracy of Quest 3 degrades under heavy occlusion (fingers hidden behind object)
• Author-stated: 90Hz control is not sufficient for all fine manipulation tasks requiring higher frequency feedback
• Unstated: $500 is consumer price; research labs face institutional procurement overhead; “democratization” claim is relative

Reproducibility

• Code: fully open-source (GitHub); MIT license
• Datasets: demonstration datasets for the 38 tasks
• Compute: real-time control (minimal); policy training requires GPU

Insights

The $500 price point is the core contribution, not the system design. By showing that consumer VR hardware is “good enough” for imitation learning data collection, OPEN TEACH changes the cost structure of dexterous robot learning research. The open-source release amplifies this: every lab that adopts OPEN TEACH extends the validated task library rather than rebuilding infrastructure. The bimanual support is particularly significant — most real-world tasks (cooking, assembly, caregiving) require two hands, and bimanual data collection was previously especially expensive.

Connections

• AnyTeleop: Vision-Based Dexterous Teleoperation
• Open-TeleVision: Immersive Active Visual Feedback
• DexCap: Scalable and Portable Mocap Data Collection
• bimanual manipulation
• contact-rich manipulation

Raw Excerpt

Open-source teleoperation platform built on the Meta Quest 3 VR headset ($500consumerhardware).Enablesintuitivecontrolofvariousroboticplatformsvianaturalhandgesturesatupto90Hzwithvisualfeedback.The$500 hardware cost democratizes dexterous teleoperation data collection.