arXiv: https://arxiv.org/abs/2512.08920 | cs.RO | 2025-12-09
Abstract
OSMO is a wearable tactile glove enabling humans to collect manipulation demonstration data with rich contact feedback. A robot policy trained exclusively on human demonstrations collected with OSMO, without any real robot data, is capable of executing a challenging contact-rich manipulation task. The system captures continuous shear and normal forces while maintaining compatibility with hand-tracking systems for in-the-wild data collection.
Hardware Design
Sensor Configuration
- 12 three-axis magnetic tactile sensors distributed across fingertips and palm
- Sensing range: 0.3 N to 80 N
- Soft magnetic elastomers with embedded magnetometers (BMM350, 3-axis)
- Each sensor PCB: two 3-axis magnetometers + one 6-axis IMU (BHI360)
- Silicone (EcoFlex 00-30) with magnetic microparticles cast in 3D-printed molds
Sensor Placement
Five sensors on fingertips (distal phalanges) + seven on palm regions, covering primary contact areas while accommodating varying hand sizes.
Crosstalk Mitigation
- MuMetal magnetic shielding + dual magnetometer differential sensing
- Combined approach reduces average noise by 57% vs. single magnetometer
Data Collection Pipeline
System Architecture
- RGB + stereo infrared cameras capture 140 human demonstrations (~2 hours)
- Hand pose estimation: SAM2 (segmentation) → HaMeR (3D keypoints)
- Depth refinement using FoundationStereo stereo matching
- Savitzky-Golay filtering for trajectory smoothing
Data Format
Each frame: RGB imagery + infrared stereo pairs + magnetic flux measurements (3×2×5 tensor format). Retargeting: Hand positions → IK → robot joint commands via MuJoCo. Output: Synchronized triplets of RGB, robot joint positions (13-DOF: 7-DOF arm + 6-DOF hand), tactile readings.
Robot Compatibility
Deployable on both human hands and Psyonic Ability Hand due to stretchable base layer. Compatible with Meta Quest 3, Aria Gen 2 smart glasses, Manus Quantum glove, and HaMeR vision model.
Key Results
Wiping Task Performance
- Tactile-aware policy: 71.69% ± 27.43% success
- Vision-only baseline: 55.75%
- Proprioception-only: 27.12%
Policy Architecture
Diffusion-based policy: DINOv2 (RGB) + MLP (state) + MLP (tactile) → 16-step action chunks at 2 Hz.
Key Advantage
Zero real robot data required — policy trained entirely on human demonstrations with wearable glove.