本文由 AI 分析生成
建立時間： 2026-03-28
Summary
EN: RH20T is a large-scale robotic manipulation dataset introduced at NeurIPS 2023 workshop, containing 110K+ robot demonstration sequences and 110K+ human demonstrations across 140+ tasks. The dataset is distinguished by its multi-modal richness: synchronized RGB/depth video, 6-axis force/torque sensing, spatial audio, joint angles/velocities/torques, and tactile feedback. Seven different robot configurations are included. Data was collected via a novel haptic teleoperation system designed to capture natural human manipulation skills.
ZH: RH20T 是在 NeurIPS 2023 工作坊發表的大規模機器人操作資料集，包含超過 110K 個機器人示範與 110K 個人類示範，覆蓋 140+ 種任務。資料集以多模態豐富性著稱：同步 RGB/深度影像、六軸力/力矩感測、空間音訊、關節角度/速度/力矩及觸覺反饋，涵蓋七種機器人配置，透過觸覺遠端操作系統採集資料。
Prerequisites
- Familiarity with robotic manipulation and teleoperation
- Basic understanding of imitation learning / behavior cloning
- Knowledge of multi-modal sensor fusion concepts
Core Idea
Current robot learning datasets lack the sensor diversity and scale needed for one-shot/few-shot skill generalization. RH20T addresses this by providing paired human+robot demonstrations with rich multi-modal sensing, enabling models to learn from both human and robot perspectives. The haptic teleoperation system preserves force feedback during data collection, capturing subtle manipulation skills (grasping fragile objects, precise insertion) that visual-only datasets miss.
Results
| Metric | Value |
|---|---|
| Robot demonstrations | 110K+ sequences |
| Human demonstrations | 110K+ sequences |
| Total frames | 50M+ |
| Tasks covered | 140+ |
| Robot configurations | 7 |
| Modalities | RGB, depth, force/torque, audio, joint states, tactile |
Limitations
- Haptic teleoperation hardware required for data collection — not accessible to most labs
- 7 robot configurations may still be insufficient for broad sim-to-real transfer
- No standardized train/test splits reported in the workshop version
- Human demonstrations may not generalize to robot embodiment without alignment
Reproducibility
- Dataset available for download from project page
- Data format documented (ROS bag or custom format)
- Collection hardware specifications disclosed
- NeurIPS 2023 workshop paper (not full proceedings)
Connections
- Connects to the practical robotics journey article: both address the challenge of acquiring quality demonstration data
- ReMEmbR in this vault addresses the complementary problem: once the robot has skills, how does it remember what it experienced?
- The Sunday.ai Memo robot’s Skill Capture Glove is conceptually similar to RH20T’s haptic teleoperation approach
Raw Excerpt
“RH20T contains over 110,000 robot demonstration sequences across 140+ manipulation tasks, synchronized with force/torque sensors, spatial audio, and tactile feedback. This multi-modal richness captures the subtle contact dynamics that pure visual datasets miss — enabling models to learn skills like inserting a USB connector or grasping a delicate object without crushing it.”