Humanoid Policy ~ Human Policy

Authors: Ri-Zhao Qiu, Shiqi Yang, Xuxin Cheng, Chaitanya Chawla, Jialong Li, Tairan He, Ge Yan, David J. Yoon, Ryan Hoque, Lars Paulsen, Ge Yang, Jian Zhang, Sha Yi, Guanya Shi, Xiaolong Wang Submitted: March 17, 2025 (revised Oct 2025) Project page: https://human-as-robot.github.io/

Overview

Addresses scalable data collection for humanoid robot manipulation by treating egocentric human demonstrations as cross-embodiment training data. Key insight: human hand manipulation data is abundant, cheap to collect, and shares enough structure with humanoid hand manipulation to transfer — if the embodiment gap is handled correctly.

PH2D Dataset

Hardware options:

• Apple Vision Pro (built-in cameras + ARKit 3D pose tracking)
• Meta Quest 3 + ZED Mini stereo cameras (3D-printed mount, <$700)

Collection: Demonstrators perform manipulation tasks wearing VR headsets. No robot required. Anywhere, anytime.

Scale: 50,000+ processed frames with 3D hand-finger poses and language annotations.

Embodiment gap mitigations:

• Collectors maintain upright seated position (avoids whole-body motion absent in robots)
• Actions time-stretched 4x (α_slow = 4) to match slower robot execution speed

HAT Architecture (Human Action Transformer)

Unified state-action space: 54-dim vector — 6D rotations of head + wrists + 3D coordinates of wrists + fingertips. Same representation for humans and robots (different embodiments, same state space).

Visual processing: Frozen DINOv2 ViT-S encoder. Color jitter + Gaussian blur augmentation handles camera/appearance differences between human and robot data.

Retargeting: Human hand poses → robot joint angles via inverse kinematics.

Loss: Dual-component — end-effector positions (λ=2 weight) + finger trajectories.

Results

Setting	With Human Data	Without	Improvement
In-Distribution	49/60	45/60	+9%
Out-Of-Distribution	101/170	59/170	+71%

Data collection speed:

• Human (no headset): 3.79s per task
• Human (VR headset): 4.09s
• Robot teleoperation: 19.72s

Human data collection is ~5x faster than robot teleoperation.

Few-shot transfer: 20 robot demos + human data significantly outperforms robot-only baselines.

Key Claims

• Human data improves OOD generalization (novel backgrounds, object appearances, spatial arrangements) far more than in-distribution performance
• Treating humans and robots as different embodiments in a unified framework is sufficient — no separate affordance representations needed
• Scalable: no robot hardware required during collection