Grasp Taxonomy and Type-Conditional Generation
Sources: Session notes, 2026-07-02 Raw: Dexonomy Summary Updated: 2026-08-12
Overview
Most dexterous grasping research optimizes “can the robot hold it?” while ignoring “which grasp type is appropriate?” Dexonomy (2025) closes this gap: a pipeline that synthesizes diverse, type-labeled grasps for the full GRASP taxonomy (31 human grasp types) using only one hand-labeled template per type, then trains a type-conditional generative model achieving 82.3% real-robot success on unseen objects.
The GRASP Taxonomy
The GRASP taxonomy categorizes all human daily grasps into 33 types (Cutkosky 1989, refined later), split between power grasps (whole-hand contact for force) and precision grasps (fingertip contact for control). Most prior robotic grasping work produces only 2–5 types implicitly; Dexonomy explicitly models 31 of the 33.
Why grasp type matters: Grasping an apple requires a power grasp (whole-hand wrap); picking up a thin card requires a precision grasp (fingertip). Using the wrong type risks dropping, crushing, or failing to pick up. Task-appropriate grasping is a prerequisite for truly dexterous manipulation.
Dexonomy Pipeline
Input: One human-labeled grasp template per hand × grasp type (contact points + normal vectors + joint pose).
Stage 1 — Global alignment (GPU-parallel): Fix the hand pose, sample object positions/rotations/scales, optimize for surface contact alignment with template contacts using a single energy function. Avoids local optima by operating globally.
Stage 2 — Local refinement (MuJoCo): Fix the object, use transposed Jacobian control to adjust hand pose. Achieves contact-rich results (power grasps: >10 finger links within 2mm) while remaining penetration-free.
Stage 3 — Contact-aware simulation verification: Compute contact forces, simulate resistance to 6-direction external loads for 2 seconds. Only force-closure grasps pass.
Stage 4 — Template library self-expansion: Successful grasps are added back to the template pool for subsequent iterations, reducing dependence on initial human labels and improving noise robustness.
Dataset output: Dexonomy Dataset — 10.7k objects, 9.5M grasps, 31 GRASP types (Shadow Hand), generated on 8× RTX 3090 in under 3 days.
Type-Conditional Generative Model
Architecture: point cloud input + grasp type codebook conditioning + normalizing flow.
Inference: Single-view point cloud + requested grasp type → grasp pose distribution → sample.
Real-robot results: 13 unseen objects, ~200 trials: 82.3% success rate. Primary failure mode: mismatched grasp type for the object’s geometry (e.g. selecting precision grasp for a large sphere).
Simulation benchmark: Substantially outperforms type-unaware baselines (DexGraspNet, FRoGGeR, SpringGrasp, BODex) especially under low friction and Objaverse object diversity.
Annotation Tool
A click-twice UI: the user specifies contact points and selects a grasp type; the system synthesizes semantically labeled grasp data in real time. This reduces the cost of building task-specific dexterous datasets.
Limitations
- Generates static grasp poses; does not handle dynamic grasp trajectories
- Does not handle cluttered scenes (multiple objects competing for grasp)
- Fails when grasp type is selected inappropriately for object geometry — selecting the right type is a separate perception problem
Open Questions
- Can type selection itself be automated? A scene-understanding model that predicts the appropriate GRASP type before generating the pose would close the remaining gap.
- Dexonomy generates static poses. Extending to full grasp approach trajectories is the natural next step for integration with manipulation policies.
- How does grasp type interact with downstream manipulation? Power vs. precision grasps likely require different control strategies after contact.