Quartz 5

Feasibility-aware Imitation Learning from Observations through a Hand-mounted Demonstration Interface

Properties2
descriptionFABCO framework addresses the mismatch between human and robot movement capabilities in imitation learning by assessing demonstration feasibility via pre-trained dynamics models and weighting training data accordingly.
tagsclippings, research, robotics, ai

3 min read

Feasibility-aware Imitation Learning from Observations through a Hand-mounted Demonstration Interface

Authors: Kei Takahashi, Hikaru Sasaki, Takamitsu Matsubara (Nara Institute of Science and Technology)

arXiv: 2503.09018 | Submitted: March 12, 2025

Categories: cs.RO (Robotics), cs.LG (Machine Learning)

License: CC BY 4.0

Abstract

This paper introduces FABCO (Feasibility-Aware Behavior Cloning from Observation), a framework that addresses a critical challenge in robot imitation learning: humans naturally demonstrate actions that robots cannot physically execute due to differing movement capabilities.

FABCO assesses the feasibility of each demonstration using the robot’s pre-trained forward and inverse dynamics models and provides visual feedback to demonstrators, encouraging them to refine their demonstrations. The system weights demonstration data by feasibility during policy learning, improving both efficiency and robustness.

Four human participants tested the approach on a pipette insertion task. Results showed significantly higher success rates compared to standard behavior cloning methods. Workload was measured using NASA-TLX metrics.

Key Contributions

  • Hand-mounted demonstration interface: Allows human demonstrators to provide robot demonstrations while receiving real-time feasibility feedback
  • Feasibility assessment: Uses pre-trained forward and inverse dynamics models to evaluate whether each demonstrated motion is executable by the robot
  • Feasibility-weighted policy learning: Weights training data by feasibility scores to improve sample efficiency and robustness
  • Human study: Validated with 4 participants on a pipette insertion task with NASA-TLX workload measurement

Problem Context

In Learning from Demonstration (LfD) and Imitation Learning from Observations (ILfO), a fundamental mismatch exists: human demonstrators may unintentionally perform movements that fall outside the robot’s physical capabilities (joint limits, dynamics constraints, workspace boundaries). Standard behavior cloning treats all demonstrations equally, leading to failed reproductions when the robot attempts infeasible motions.

Method: FABCO

  1. Pre-training: Train forward dynamics model (state → next state) and inverse dynamics model (state, next state → action) on robot interaction data
  2. Demonstration collection: Human uses hand-mounted interface; system computes feasibility score per demonstration segment using the dynamics models
  3. Visual feedback loop: Demonstrator sees real-time feasibility indicators and can refine motions accordingly
  4. Weighted behavior cloning: Policy trained with demonstration samples weighted by their feasibility scores — infeasible demonstrations contribute less to the gradient

Experimental Results

  • Task: Pipette insertion (precision manipulation requiring fine motor control)
  • Participants: 4 human demonstrators
  • Baseline: Standard behavior cloning from observations
  • Outcome: FABCO achieved significantly higher task success rates
  • Workload: NASA-TLX scores evaluated demonstrator burden of the feedback interface

Relation to Prior Work

  • Builds on Imitation Learning from Observations (ILfO) — learning robot policies without requiring action labels from demonstrations
  • Extends behavior cloning (BC) with feasibility-aware weighting
  • Related to interactive imitation learning (IIL) where demonstrator feedback is incorporated
  • Hand-mounted interface design relates to wearable/egocentric demonstration systems in HRI

Graph View

Backlinks