Learning Dexterous Manipulation Skills from Imperfect Simulations
Elvis Hsieh∗ , Wen-Han Hsieh∗ , Yen-Jen Wang∗ , Toru Lin, Jitendra Malik, Koushil Sreenath† , Haozhi Qi†
UC Berkeley
Screwdrivin
Nut-Bolt Fastenin
arXiv:2512.02011v2 [cs.RO] 25 Feb 2026
Generalizatio
Fig. 1: We propose DexScrew, a sim-to-real framework for learning dexterous manipulation skills when the environment
cannot be accurately simulated. In simulation, we use simplified objects to learn transferable rotational skills, which are then
used to collect data and train tactile policies in the real world. We demonstrate the framework on contact-rich screwdriving
(top row) and nut-bolt fastening (middle row). We also show generalization across different objects (bottom row). More
videos and code are available on https://dexscrew.github.io.
Abstract— Reinforcement learning and sim-to-real transfer However, in practice, sim-to-real transfer faces two major
have made significant progress in dexterous manipulation. How- limitations. First, due to the complexity of physics simula-
ever, progress remains limited by the difficulty of simulating tion, only a limited range of tasks can be accurately modeled.
complex contact dynamics and multisensory signals, especially
tactile feedback. In this work, we propose DexScrew, a sim- Prior work either relies on specialized techniques for high-
to-real framework that addresses these limitations and demon- fidelity simulation [8], [9] or seeks generalization through
strates its effectiveness on nut-bolt fastening and screwdriving domain randomization [1], [10]–[12]. However, as the task
with multi-fingered hands. The framework has three stages. becomes more dynamic, the sim-to-real gap grows [13],
First, we train reinforcement learning policies in simulation and simulation alone becomes insufficient. Second, existing
using simplified object models that lead to the emergence
of correct finger gaits. We then use the learned policy as a sensing modalities have an intrinsic sim-to-real gap. While
skill primitive within a teleoperation system to collect real- vision can be partially mitigated through domain random-
world demonstrations that contain tactile and proprioceptive ization [14], tactile sensing remains difficult to approximate
information. Finally, we train a behavior cloning policy that reliably. Although some work aims to improve tactile sim-
incorporates tactile sensing and show that it generalizes to nuts ulation [15], [16] or trains policies using alternative proxy
and screwdrivers with diverse geometries. Experiments across
both tasks show high task progress ratios compared to direct representations [17]–[19], these approaches cannot leverage
sim-to-real transfer and robust performance even on unseen the full power of tactile sensing. These limitations remain
object shapes and under external perturbations. widely viewed as major constraints on the complexity of
tasks that can be achieved.
I. I NTRODUCTION On the other hand, teleoperation and imitation learn-
Reinforcement learning (RL) paired with sim-to-real trans- ing [20], [21] remove the need for simulation entirely. In this
fer has recently delivered a number of promising results in setting, policies can learn directly from real-world interac-
dexterous manipulation [1]–[5]. Policies trained in massively tions and sensorimotor signals, which avoids the challenges
parallel simulation [6] with domain randomization [7] have introduced by sim-to-real transfer. However, teleoperating
demonstrated strong robustness and generalization capabili- dexterous hands is challenging because of the intrinsic mor-
ties in the real world. phology differences between human and robot hands [22]–
[24]. As a result, it is difficult to collect datasets that are
∗ Equal contribution (listed in alphabetical order). large and diverse enough to achieve the desired behavior
† Equal advising. and generalization.
Motivated by these observations, we introduce DexScrew, Both directions, however, face notable limitations. Sim- a framework that combines the strengths of both approaches to-real methods benefit from large-scale simulated data and to expand the capability of sim-to-real reinforcement learning can generalize across diverse objects, yet they remain lim- under imperfect simulation. The key idea is that the motion ited by inaccuracies in modeling complex contact dynamics primitives underlying contact-rich dexterous manipulation do and sensing, a challenge that becomes more significant not need to be learned from a perfect physics model. A as task complexity increases [13], [35]. Imitation learning simplified simulator is sufficient to induce the core rotational benefits from multisensory real-world data, yet collecting behaviors required for these tasks. Once this motion is high-quality dexterous demonstrations is considerably more learned, the resulting policy can be used as a skill prim- difficult than collecting data for simpler end-effectors. Our itive to collect real-world demonstrations, from which a work seeks to combine the strengths of both approaches. new policy can be learned. In this way, sensing modalities We use large-scale simulation to learn motion primitives and physical interactions that are difficult to simulate can while leveraging real-world data to close the dynamics and be obtained directly from real-world data, while the fine- sensing gaps. Moreover, our skill-based framework enables grained motions that are hard to teleoperate are provided efficient collection of dexterous real-world data by using the by the simulation-trained policy. We demonstrate this idea simulation-trained policy itself as a reusable skill primitive. through two tasks, nut-bolt fastening and screwdriving with In the context of nut fastening and screwdriving, there has a multifingered hand. Both tasks are traditionally viewed been recent work combining sim-to-real transfer with teleop- as requiring complex contact dynamics understanding and eration. For example, Liu et al. [36] build a residual model tactile sensing. We instead show that effective policies can from real-world interactions to compensate for the sim-to- be learned without relying on high-fidelity simulation. real gap and achieve robust in-hand manipulation. Yin et More specifically, our framework consists of three stages. al. [37] use simulation-trained policies as stability controllers First, we train reinforcement learning (RL) policies in simu- to enable complex manipulation skills. Both approaches can lation using a simplified physics model. Instead of modeling be integrated with teleoperated arm control to complete these the thread structure of the nut and screw, we approximate tasks. However, they do not produce autonomous policies their interaction with a revolute joint that connects two sim- that incorporate tactile sensing. Another line of work is ple geometric shapes, which allows the policy to efficiently Kumar et al. [38], who demonstrate screwdriver turning by learn rotational behavior. Second, we use this learned skill combining learning with trajectory optimization. Noseworthy as a primitive within a teleoperation system to collect real- et al. [39] present an autonomous sim-to-real policy but world demonstrations. The operator controls the arm motion only show results with a parallel-jaw gripper and do not and triggers the finger rotation skill rather than issuing low- demonstrate regrasping. level joint commands, which enables efficient collection of Another way to address the sim-to-real gap is to refine the tactile data during teleoperated execution. Finally, using the policies in the real world. For example, Transic [40] shows resulting multisensory dataset, we train a behavior cloning that sim-to-real policies can adapt to complex real-world policy that coordinates arm and finger motions while lever- dynamics with only a few human interventions as demonstra- aging tactile feedback. tions, although the demonstrations are primarily performed We evaluate our framework on two tasks: nut-bolt fasten- with simple end-effectors. In contrast, we apply this idea ing and screwdriving. Policies trained with simplified dynam- to dexterous hands. Maddukuri et al. [41] show that co- ics can generate reasonable rotational behavior but cannot training with both simulation and real-world data can reduce complete the tasks. By learning from real-world multisensory the gap and improve manipulation performance. RLPD [42] demonstrations, our method overcomes these limitations and and Sparsh-X [43] share a similar philosophy of augmenting achieves stable and reliable performance under challenging modalities through human-guided data collection; however, contact conditions. These results show that complex contact- they are not motivated by the possibility of learning skills rich manipulation skills can be bootstrapped from simplified from imperfect simulations, as their policies are designed to simulators and that real-world tactile feedback is essential. be directly transferable. Our framework provides a scalable path toward dexterous manipulation and supports broader deployment of general- III. D EXTERITY FROM I MPERFECT S IMULATION purpose robot hands in unstructured environments. An overview of our method is shown in Figure 2. It II. R ELATED W ORK consists of three stages. First, we train a reinforcement Dexterous manipulation has been a long-standing chal- learning (RL) policy in simulation using a simplified object lenge in robotics [25], [26]. Early work focused on classical model (Section III-A). The resulting policy learns the desired model-based control and analytic grasp planning [27]–[31]. finger motions but does not experience real-world dynamics Recent years have seen rapid progress in learning-based ap- and lacks tactile feedback. To address this, we collect real- proaches, which can be grouped into two primary directions: world trajectories using the learned policy as a skill primitive sim-to-real learning paired with reinforcement learning [1], for teleoperation (Section III-B). Finally, using this dataset, [4], [5] and imitation learning from teleoperation [23], [24], we train a new multisensory policy using behavior cloning [32] or human data [33], [34]. (Section III-C). (A) Train RL Policy in Simulation (B) Real World Data Collection using Learned Policy (C) BC with Multi-sensory Data
Human Arm Movement
Teleoperator
Activation of
RL Policy
Policy Learned with Imperfect Simulation
Fig. 2: An overview of our approach. We first train a reinforcement learning policy in simulation using a simplified object model, which serves as a motion prior for nut-bolt fastening and screwdriving. We then collect real-world trajectories by using the learned policy as a skill primitive during teleoperation. Finally, we train a behavior cloning policy on the collected data to obtain coordinated behavior between the arm and the fingers.
A. Training a Reinforcement Learning Policy in Simulation For the Nut-Bolt Task For the Screwdriving Task
Simplified Object Modeling. Our goal is to design a Simplified
simulation environment that enables fast training and en- Nut / Handle courages the emergence of desired finger gaits for rotation. To achieve this, we construct a simplified simulated object (Figure 3) that captures the essence of rotational motion. Revolute Joint The object consists of a fixed cylindrical base with a nut or handle attached via a revolute joint. This setup allows the Simplified Bolt / Screw policy to learn rotational motion efficiently without relying on expensive contact-rich simulations. A similar idea was explored in [44] to model bottle caps using a heuristic friction design. In contrast, we further simplify the model, since we Fig. 3: Simplified Object Models. Each nut or handle is can leverage real-world data to compensate for the resulting modeled as a rigid body attached to a fixed base through a dynamics mismatch. revolute joint. This abstraction ignores thread-level mechan- Specifically, for the nut-bolt task, we use a thick triangular ics while retaining the essential rotational dynamics needed shape as the training object (Figure 3A). The extra thickness for learning. is used to prevent the policy from learning suboptimal strate- gies that apply a large force from the bottom. The learned policy also discovers a high-clearance gait that transfers Actions. At each step, the policy outputs a relative target well to diverse real-world nuts such as hexagonal and cube- position. The position command is computed as aHand t = shaped nuts. For the screwdriver task, where the primary ηf (oRL ) + aHand t−1 , where η is the action scale. This command difficulty arises from slippage around the handle, we use is sent to the robot and converted into torque via a low-level spherical primitives to keep the learned behavior conservative PD controller. Here, oRL contains the robot’s proprioceptive (Figure 3B). This observation, that different training shapes state, including joint positions and previous target positions lead to different rotational gaits, is also consistent with the from a sliding window of recent 3 timesteps. findings in [5]. Note that these objects do not need to be Reward. The goal of the policy in simulation is to rotate visually aligned with real world objects, as they are only the simplified object around the revolute joint. The reward used to learn the coarse motions used for real world data consists of a task reward, energy penalties, and stability collection, as we discussed in Section III-B. penalties (time index t omitted for simplicity). Each com- Training Pipeline. Following [13], [17], we first train an ponent includes several terms defined in the appendix: oracle policy and then distill it into a sensorimotor policy. rt = λtask rtask + λenergy rtenergy + λstability rtstability . The oracle policy f is trained with access to an embedding of privileged information [45] zt . The sensorimotor policy Oracle Policy Training. We train the oracle policy using operates without privileged sensing and instead conditions proximal policy optimization (PPO) [46] with the reward on a predicted embedding ẑt = ϕ(ht ) inferred from propri- described above. The robot state and privileged information oceptive history ht by a prediction module ϕ. are each encoded with separate MLPs. These embeddings Privileged Information. The oracle policy has access to are concatenated and passed through an MLP to produce the ground-truth environment and object properties, including final action and value predictions. We train the policy for object attributes (e.g., position, scale, mass, center of mass, 1.5×109 environment steps. friction coefficients), hand pose and finger configurations, Sensorimotor Policy Training. The sensorimotor policy and low-level controller parameters. The full set of privileged receives proprioceptive states and a latent code ẑt = ϕ(ht ) inputs is documented in the appendix. inferred from a 30-timestep history. We train the policy Joystick: Horizontal Translation for finger motion control. Instead of commanding every joint individually, the human operator controls only the wrist movement and decides when to activate the skill primitive (Figure 4). Wrist position is specified using the Quest VR1 controller’s joystick. This approach is inspired by [32], but Wrist Up we use a much finer-grained skill for data collection. This framework offers several advantages. First, it dele- gates complex finger motions to a robust simulation-trained policy that generalizes across different objects, eliminating the need for humans to learn finger coordination under Wrist Down Policy Activation morphological differences. Second, using a joystick for arm control enables precise and intuitive wrist positioning. While Fig. 4: Teleoperation Interface. The human operator con- arm motion is relatively straightforward to simulate, we trols the wrist position using the VR controller buttons and intentionally exclude it from the initial simulation training adjusts yaw and pitch through the joystick. This setup allows stage, as learning the specific downward progression required the operator to guide the arm motion while relying on the for nut-bolt fastening would necessitate simulating thread- learned finger-rotation skill during data collection. level mechanics. Finally, collecting data in the physical envi- ronment provides the multisensory observations, particularly tactile signals, that are essential for reliable task completion using DAgger [47]: at each step, the sensorimotor policy acts but remain difficult to approximate reliably in simulation. in the environment, while the oracle policy provides target Concretely, at each timestep we record two actions: (1) the actions and ground-truth privileged embeddings. The training action generated by the RL policy πRL , which defaults to the objective is current hand joint positions when the policy is not activated, and (2) the arm action generated by human teleoperation. L = ∥aHand t − âHand t ∥22 + ∥zt − ẑt ∥22 , Formally, we define at = [aHand t , aArm t ], where at Hand ∈ R12 denotes the hand target joint positions, and at ∈ R6 de- Arm where âHand t denotes the actions produced by the sensorimo- notes the arm joint positions. We also record the multisensory tor policy. The embedding predictor ϕ is optimized using observation qt , ct , where qt = [qtHand , qtArm ] contains all Adam [48] until convergence. joint positions, and ct represents the raw tactile signals from Randomization. We apply domain randomization during all five fingers. training to improve the robustness of the RL policy [7]. Tactile Signal. In this work, we use the XHand’s built-in Following [13], we randomize the nut/handle mass, center tactile sensors to capture contact information. Each fingertip of mass, friction coefficient, size, and PD gains, and we also is equipped with a pressure-based tactile array comprising add observation noise. Detailed parameters are provided in 120 sensing elements, each measuring three-axis forces with the appendix. a minimum detectable force of 0.05 N. At each timestep, we Termination Conditions. To prevent the policy from getting record the tactile signal as ct ∈ R5×120×3 , which includes stuck in unrecoverable states, we terminate an episode when the signals from all five fingers across three axes. any of the following conditions are met: (1) the distance be- tween the thumb or index finger and the nut/handle exceeds a C. Behavior Cloning with Multisensory Data reset threshold (7 cm for nuts, 10 cm for handles); (2) the nut or handle remains stagnant over a sliding time window (3.5 s With the dataset DReal collected using the skill-based for nuts, 3 s for handles); or (3) near-zero contact forces are assisted teleoperation, we can train a behavior cloning (BC) detected for the same duration. These conditions accelerate policy πBC using the paired multisensory observations and training by penalizing failure modes such as drifting away expert actions. Vision is not used in our work. from the object or failing to maintain contact. Neural Network Architecture. We use a feedforward net- work as the policy. The past K timesteps of observations B. Real World Data Collection with Learned Policy (qt−K+1:t , ct−K+1:t ) are concatenated into a single feature The policy trained in simulation with the simplified object vector. Tactile signals are first flattened and passed through model learns the desired rotational behavior; however, it an MLP. The fused feature vector is then processed by an inevitably misses key physical dynamics such as thread hourglass encoder [49], which outputs the action predictions. interactions. These effects are difficult to model but are We also apply an action chunking strategy [20], [21], where crucial for reliable real-world fastening. It also lacks tactile the policy predicts a sequence of future actions ât:t+H rather information, which is hard to simulate but crucial for fine- than a single-step action. We use K = 5 and H = 16 unless grained wrist adjustments. otherwise noted. To bridge this gap, we introduce a skill-based assisted Training. The policy is trained with supervised learning teleoperation for real-world data collection. The core idea is to reuse the simulation-trained policy as a skill primitive 1 https://www.meta.com/quest/products/quest-2 TABLE I: Real-world fastening performance on square, triangular, hexagonal, and cross-shaped nuts. We report progress ratio and rotation time (mean ± standard deviation over 10 trials) for different observation modalities. Tactile sensing and temporal history both improve performance, and their combination yields the highest accuracy and fastest execution. * indicates that only one completely successful trial was recorded, so no standard deviation is reported. Square Nuts Triangular Nuts Hexagonal Nuts Cross-Shaped Nuts Tactile Hist. Prog. Ratio (%) ↑ Time (s) ↓ Prog. Ratio (%) ↑ Time (s) ↓ Prog. Ratio (%) ↑ Time (s) ↓ Prog. Ratio (%) ↑ Time (s) ↓ 63.75±33.05 148.76±30.57 30.00±39.62 229.39* 75.00±29.46 102.36±25.00 63.75±33.05 101.07±0.51 ✓ 62.50±33.85 202.21±59.69 66.25±27.67 134.19±46.72 75.00±16.67 205.02±82.44 82.50±10.54 127.17±47.98 ✓ 87.50±20.41 129.88±48.06 80.00±32.91 111.47±55.10 85.00±32.17 68.97±13.99 100.00±00.00 91.49±44.89 ✓ ✓ 97.50±7.91 124.20±33.22 96.25±8.44 117.79±52.13 95.00±10.54 75.07±17.41 98.75±3.95 84.21±57.19
using the loss not achieve a single successful fastening or screwing attempt T X H across ten trials. In such cases, we cannot report the standard deviation or the completion-time metric. X 2 LBC = ∥ ât+h − at+h ∥2 , t=1 h=0 B. Nut-Bolt Experiments where ât:t+H is the action chunk predicted by πBC , and at:t+H is the corresponding expert sequence. This objec- We first evaluate the system on the nut-bolt task. This tive encourages consistent predictions over the full horizon. task requires the fingers to establish correct contact patterns, We train the policy using Adam [48] for 200 epochs and sense progress through tactile feedback, and adjust the arm normalize observations following [50]. position accordingly. We choose this task because nut-bolt interactions are difficult to simulate efficiently, and complet- IV. E XPERIMENTS ing the task relies heavily on tactile sensing, making it a In this section, we first introduce the experiment setup strong testbed for our method. (Section IV-A). We then evaluate the performance of our We note that direct sim-to-real transfer can rotate the nut, policies on two challenging tasks, nut-bolt fastening (Sec- but it cannot drive the nut downward because the arm does tion IV-B) and screwdriving (Section IV-C). We conclude not move. Since thread interactions are not simulated, the with qualitative experiments that provide additional analysis nut remains at the same height even after completing full and design ablations in simulation. revolutions in simulation. Setting. In simulation, we use the thick triangular nut shown A. Experiment Setup in Figure 3 (left). Training with this shape produces high- Hardware Setup. Our system consists of a UR5e robot clearance gaits that transfer well and can rotate both square arm (6 DoF) and a 12-DoF XHand. The XHand has five and triangular nuts in the real world. fingers: the thumb and index finger each have 3 DoF, and During skill-based assisted teleoperation, we collect 50 the remaining fingers have 2 DoF. Only the thumb and index trajectories each for the square and triangular nuts. Each provide abduction/adduction. trajectory lasts about 80 seconds. We then train a behavior Simulation. We train our policies in IsaacGym [6] using cloning policy using the combined dataset. We evaluate 8,192 parallel environments. Each environment contains a performance on four types of nuts, which include square simulated XHand and the simplified object model described and triangular nuts as well as two unseen shapes, namely in Section III-A. The simulation runs at 200 Hz, with control hexagonal nuts and cross-shaped nuts. Our main results are applied at 20 Hz. Each episode lasts up to 800 simulation shown in Table I. steps (40 s). Observation History. We first study the effect of providing Object Set. For the nut-bolt task, we train on triangular a short temporal history in the observation. Adding history nuts. For the screwdriver task, we approximate handles by significantly improves progress ratio and reduces execution octagon- and dodecagon-shaped nuts. This multi-geometry time across all modalities and nut geometries. Temporal cues training in simulation helps the policy generalize to diverse help the policy track fine-grained rotational progress and real-world shapes. distinguish local geometric features. The benefit is especially Metrics. In simulation, we report the episode reward and clear for non-tactile policies, where history stabilizes perfor- episode length during training. In real-world evaluation, mance and narrows much of the gap to tactile-based policies we measure the progress ratio, defined as the number of on easier geometries. When combined with tactile sensing, successful rotations divided by the total number of rotations history yields the strongest overall performance, achieving required for full fastening. We also report the time for trials the highest accuracy and fastest completion times across all that fully complete the fastening process (progress ratio = nut types, including unseen shapes. 100), defined as the time needed to fully fasten the nut or Tactile Information. Across all nut geometries, adding fully tighten the screw. Note that some baseline methods do tactile input generally improves progress ratio. The effect TABLE II: Real-world screwdriving performance. We Policy with Tactile Info
report progress ratio and rotation time (mean ± standard deviation over 10 trials) for direct sim-to-real, expert replay, and our behavior cloning (BC) ablations. Tactile sensing Thumb Index Middle Thumb Index Middle Thumb Index Middle Thumb Index Middle Thumb Index Middle and temporal history each improve performance, and their combination achieves the highest progress ratio and fastest Policy without Tactile Info execution. Here * indicates that the policy never fully com- pleted the task, so no rotation time is reported.
Screwdriver Task Thumb
Index Middle Thumb
Index Middle Thumb Index Middle Thumb Index Middle Thumb Index Middle
Method Tactile Hist. Prog. Ratio (%) ↑ Time (s) ↓
Direct Sim2Real 41.60±26.21 N.A.* Fig. 5: Top: The policy with tactile information maintains Expert Replay 50.80±19.27 N.A.* a consistent alternating pattern of thumb and index finger 69.20±35.25 266.33±91.60 contact, which supports stable engagement as the nut is ✓ 67.63±35.65 264.06±76.57 rotated downward. Bottom: The policy without tactile infor- Ours ✓ 87.50±18.61 195.15±44.83 ✓ ✓ 95.00±13.24 187.87±24.87 mation does not maintain a clear contact pattern. This leads to unsuccessful engagement and prevents proper downward wrist motion. The resulting pattern reflects the index finger pressing against the bolt after losing stable contact. is most pronounced on challenging shapes such as triangu- lar and cross-shaped nuts, where progress ratios rise from roughly 30 to 65 percent for triangular nuts and from about Figure 3 (right). This curated set encourages the learned rota- 60 to 80 percent for cross-shaped nuts. This underscores tional gaits to remain conservative in clearance and maintain the importance of tactile feedback for maintaining stable stability. In the real world, we collect 72 trajectories, each contact and detecting effective rotation. We also observe lasting between 120 and 180 seconds. that certain non-tactile settings achieve high performance, even on unseen shapes, particularly on well-constrained Sim-to-Real Policy. Unlike the nut-bolt experiments, the geometries such as hexagonal and cross-shaped nuts. In these screwdriving task can still make progress even when the cases, the geometry provides strong passive guidance, and wrist does not move, since downward motion is not required. the finger gait learned in simulation is often sufficient to We first evaluate a direct sim-to-real policy that uses only maintain contact without relying on tactile cues. proprioception. The results are shown in Table II. The Failure Modes. We observe two main failure modes. First, direct sim-to-real policy achieves a 41.60% progress ratio, the policy without observation history struggles to infer indicating that it can produce meaningful behavior, which object shape from single-step proprioception. As a result, it is a prerequisite for collecting expert trajectories. However, fails to generalize across nut geometries, since different nuts because it inevitably makes mistakes, it never completes the require different rotational gaits. The policy cannot adjust its task even once, and therefore we cannot report statistics gait because the limited sensing information prevents it from for completion time. In contrast, during data collection, the identifying the correct motion pattern. Second, non-tactile human operator can adjust the wrist position to recover from policies frequently drift into unstable contact states and lose such mistakes. alignment with the bolt. Once misalignment occurs, the nut As a result, when we replay the expert data from the cannot sustain continuous rotation. In contrast, tactile poli- data we collected, it can achieve higher success rate to be cies can recover by adjusting wrist orientation or applying 50.80%. However, because it cannot adapt changes during corrective downward force to re-establish contact. deployment. It also fails at completely finishing the task. Next, we show the results of our policies. C. Screwdriving Experiments Main Results. Our behavioral cloning policies show clear We also demonstrate that our method extends to the improvements over the direct sim-to-real and expert-replay more challenging screwdriving task. Compared to nut-bolt baselines. Adding tactile sensing or temporal history indi- fastening, screwdriving is inherently less stable: the shaft vidually improves progress ratio. Combining both modalities is not kinematically constrained along the screw axis, and gives the strongest performance. even small tilts or misalignments can lead to slipping or The baseline behavior cloning model already achieves loss of contact. As a result, the task requires more fine- a 69.20% progress ratio, which substantially outperforms grained control to maintain continuous rotation. The complex expert replay. This phenomenon, where a behavior cloning interaction between the screwdriver and the screw is also policy surpasses the policy that generated the data, is con- difficult to simulate accurately, which is why we include this sistent with filtered behavior cloning [51]. In this approach, task in our study. only successful trials are used for training. A similar effect Setting. In simulation, we use a mix of octagonal and has also been reported in [13]. dodecagonal handles as the training objects, as shown in Using history alone produces a comparable improvement Finger Dragging Recover Episode Reward Episode Length (s) 2000 40
1500 30
1000 20
500 10
Reverse Rotation Recover
0 0
0 600 1200 1800 2400 3000 0 600 1200 1800 2400 3000
Environment Steps (M) Environment Steps (M)
Ours (Oracle) Ours (Sensorimotor) No Priv. Info Asym. Actor-Critic
Fig. 7: Simulation ablations of screwdriving policy training.
Fig. 6: Top row: The policy recovers back to the nut- We compare our privileged-information oracle policy, its bolt fastening motion when the fingers are dragged by an sensorimotor policy, an asymmetric actor–critic variant, and external force. Bottom row: The policy recovers back to a policy trained without privileged information. Providing the screwdriving motion when the screwdriver is rotated privileged information during training leads to significantly counterclockwise during the clockwise rotation by the policy. higher reward and more stable episode lengths. Each curve shows the mean and standard deviation over 5 seeds.
of 67.63%. This indicates that temporal information helps the policy track rotational progress and recover from partial fail- emerge when the nut or screwdriver handle is correctly ures. When history is not used, tactile information provides engaged. The policy learns to preserve these patterns by only limited benefit. However, once history is included, the adjusting wrist orientation and contact force, effectively two modalities become complementary. The progress ratio using tactile feedback as a local reference for alignment. increases to 95.00%, and the average rotation time decreases When these patterns deviate, corrective actions such as re- substantially. These results show that tactile feedback and engagement or downward pressing are triggered. temporal history work together to produce stable, consistent, E. Simulation Ablations and efficient screwdriving behaviors. In simulation, we verify the design choices used to Failure Modes. We observe that open-loop baselines fre- train the screwdriving policies. We evaluate our two-stage quently fail due to gradual handle slipping and accumulated training procedure against two alternatives: training without orientation drift. Without feedback, small misalignments any privileged information, and using an asymmetric actor- grow over time. Among behavioral cloning methods, the critic [52] architecture where the critic has access to full policy trained without observation history struggles to stabi- tactile information while the actor does not. The actor in this lize the screwdriver since it lacks the temporal information setting is directly deployable in the real world. The results needed to infer handle pose and orientation. Non-tactile are shown in Figure 7. policies fail to detect subtle torque imbalances and often lose We compare episode reward and episode length across stable contact under slight perturbations. With both tactile these training strategies. When both the actor and critic feedback and temporal history, BC with tactile and history have access to privileged information, our method (Ours, compensates for these effects by adjusting wrist orientation Oracle) achieves the highest performance. Removing privi- and applying appropriate forces. leged information from the actor, as in the asymmetric actor- D. Qualitative Experiments critic variant, leads to a noticeable drop in performance. Removing privileged information entirely results in a further Out-of-distribution Robustness. We study the robustness decline. These results show that privileged information plays of the learned policy under external perturbations that are an important role during policy learning. We also observe not encountered during training. These disturbances include that a sensorimotor policy can approach similar performance (1) dragging the fingers away from the object or rotating when proprioceptive history is provided as input. We also ex- the screwdriver in the opposite direction of the intended perimented with training the asymmetric and non-privileged direction. In Figure 6, we show that despite these pertur- models for longer horizons but have not observed further bations, the policy consistently recovers to stable fastening improvement. This suggests that the performance gap is not behavior. Specifically, it recovers from reverse rotation of the due to insufficient training but instead reflects the importance fastening object by re-establishing contact and restoring the of privileged information during learning. correct rotational direction. When finger contact is disrupted or temporarily blocked, the policy repositions the fingers and V. C ONCLUSION AND F UTURE W ORK wrist to regain stable engagement. We present a framework for learning dexterous manipula- Tactile Visualization. Tactile signals provide structured spa- tion skills for contact-rich tasks using imperfect simulation. tiotemporal patterns that the policy uses to infer contact The approach first learns transferable rotational skills through phases. As shown in Fig. 5, stable activation signatures reinforcement learning with simplified object modeling. It then uses these skills for skill-based teleoperation to col- [10] J. Lee, J. Hwangbo, L. Wellhausen, V. Koltun, and M. Hutter, lect real-world trajectories, and finally incorporates tactile “Learning quadrupedal locomotion over challenging terrain,” Science robotics, 2020. feedback and learns a sensorimotor policy through behavior [11] A. Kumar, Z. Fu, D. Pathak, and J. Malik, “Rma: Rapid motor cloning. Experiments on nut-bolt fastening and screwdriver adaptation for legged robots,” in RSS, 2021. usage show that simulation alone cannot capture the complex [12] A. Loquercio, E. Kaufmann, R. Ranftl, M. Müller, V. Koltun, and D. Scaramuzza, “Learning high-speed flight in the wild,” Science dynamics required for reliable task execution. However, Robotics, p. eabg5810, 2021. when behavior cloning is combined with tactile sensing and [13] J. Wang, Y. Yuan, H. Che, H. Qi, Y. Ma, J. Malik, and X. Wang, temporal history, the resulting policies become robust and “Lessons from learning to spin ‘pens’,” in CoRL, 2024. [14] R. Singh, A. Allshire, A. Handa, N. Ratliff, and K. Van Wyk, “Dextrah- reliable across diverse and unseen object geometries. This rgb: Visuomotor policies to grasp anything with dexterous hands,” staged pipeline provides a practical and scalable solution for arXiv:2412.01791, 2024. contact-rich manipulation and highlights the value of tactile [15] S. Wang, M. Lambeta, P.-W. Chou, and R. Calandra, “Tacto: A fast, flexible, and open-source simulator for high-resolution vision-based sensing and skill-based teleoperation as effective bridges tactile sensors,” RA-L, 2022. between simulation and real-world deployment. [16] I. Akinola, J. Xu, J. Carius, D. Fox, and Y. Narang, “Tacsl: A library for visuotactile sensor simulation and learning,” T-RO, 2025. Limitations. Although our skill-based teleoperation reduces [17] H. Qi, B. Yi, S. Suresh, M. Lambeta, Y. Ma, R. Calandra, and J. Malik, the burden on the human operator, it remains a constraint for “General in-hand object rotation with vision and touch,” in CoRL, scalable data acquisition. Fully autonomous data collection 2023. or learning skill-level guidance from human videos would [18] M. Yang, C. Lu, A. Church, Y. Lin, C. Ford, H. Li, E. Psomopoulou, D. A. Barton, and N. F. Lepora, “Anyrotate: Gravity-invariant in-hand further improve efficiency. In our current tasks, the nut is object rotation with sim-to-real touch,” in CoRL, 2024. already installed and the screwdriver is already inserted. [19] J. Yin, H. Qi, J. Malik, J. Pikul, M. Yim, and T. Hellebrekers, Extending the approach to fully long-horizon assembly will “Learning in-hand translation using tactile skin with shear and normal force sensing,” in ICRA, 2025. require vision sensing and possibly high-accuracy force- [20] C. Chi, Z. Xu, S. Feng, E. Cousineau, Y. Du, B. Burchfiel, R. Tedrake, torque sensing. A broader evaluation across more contact- and S. Song, “Diffusion policy: Visuomotor policy learning via action rich manipulation tasks is also needed to assess generality. diffusion,” IJRR, 2024. [21] T. Z. Zhao, V. Kumar, S. Levine, and C. Finn, “Learning fine-grained bimanual manipulation with low-cost hardware,” in RSS, 2023. ACKNOWLEDGMENT [22] S. P. Arunachalam, S. Silwal, B. Evans, and L. Pinto, “Dexterous im- itation made easy: A learning-based framework for efficient dexterous This work is supported in part by the program “Design manipulation,” in ICRA, 2023. of Robustly Implementable Autonomous and Intelligent Ma- [23] S. P. Arunachalam, I. Güzey, S. Chintala, and L. Pinto, “Holo-dex: chines (TIAMAT)”, Defense Advanced Research Projects Teaching dexterity with immersive mixed reality,” in ICRA, 2023. [24] Y. Qin, W. Yang, B. Huang, K. Van Wyk, H. Su, X. Wang, Y.-W. Chao, Agency award number HR00112490425. We thank Mengda and D. Fox, “Anyteleop: A general vision-based dexterous robot arm- Xu for his valuable feedback. hand teleoperation system,” in RSS, 2023. [25] A. Bicchi, “Hands for dexterous manipulation and robust grasping: A R EFERENCES difficult road toward simplicity,” IEEE Transactions on robotics and automation, 2002. [1] OpenAI, M. Andrychowicz, B. Baker, M. Chociej, R. Józefowicz, [26] A. M. Okamura, N. Smaby, and M. R. Cutkosky, “An overview of B. McGrew, J. Pachocki, A. Petron, M. Plappert, G. Powell, A. Ray, dexterous manipulation,” in ICRA, 2000. J. Schneider, S. Sidor, J. Tobin, P. Welinder, L. Weng, and W. Zaremba, [27] D. Rus, “In-hand dexterous manipulation of piecewise-smooth 3-d “Learning dexterous in-hand manipulation,” IJRR, 2019. objects,” IJRR, 1999. [2] OpenAI, I. Akkaya, M. Andrychowicz, M. Chociej, M. Litwin, [28] I. Mordatch, Z. Popović, and E. Todorov, “Contact-invariant optimiza- B. McGrew, A. Petron, A. Paino, M. Plappert, G. Powell, R. Ribas, tion for hand manipulation,” in Eurographics, 2012. J. Schneider, N. Tezak, J. Tworek, P. Welinder, L. Weng, Q. Yuan, [29] R. Fearing, “Implementing a force strategy for object re-orientation,” W. Zaremba, and L. Zhang, “Solving rubik’s cube with a robot hand,” in ICRA, 1986. arXiv:1910.07113, 2019. [30] A. S. Morgan, K. Hang, B. Wen, K. Bekris, and A. M. Dollar, [3] A. Handa, A. Allshire, V. Makoviychuk, A. Petrenko, R. Singh, J. Liu, “Complex in-hand manipulation via compliance-enabled finger gaiting D. Makoviichuk, K. Van Wyk, A. Zhurkevich, B. Sundaralingam, and multi-modal planning,” RA-L, 2022. Y. Narang, J.-F. Lafleche, D. Fox, and G. State, “Dextreme: Transfer [31] S. Patidar, A. Sieler, and O. Brock, “In-hand cube reconfiguration: of agile in-hand manipulation from simulation to reality,” in ICRA, Simplified,” in IROS, 2023. 2023. [32] T. Lin, Y. Zhang, Q. Li, H. Qi, B. Yi, S. Levine, and J. Malik, [4] T. Chen, M. Tippur, S. Wu, V. Kumar, E. Adelson, and P. Agrawal, “Learning visuotactile skills with two multifingered hands,” in ICRA, “Visual dexterity: In-hand reorientation of novel and complex object 2025. shapes,” Science Robotics, 2023. [33] C. Wang, H. Shi, W. Wang, R. Zhang, L. Fei-Fei, and C. K. Liu, [5] H. Qi, A. Kumar, R. Calandra, Y. Ma, and J. Malik, “In-hand object “Dexcap: Scalable and portable mocap data collection system for rotation via rapid motor adaptation,” in CoRL, 2022. dexterous manipulation,” in RSS, 2024. [6] V. Makoviychuk, L. Wawrzyniak, Y. Guo, M. Lu, K. Storey, M. Mack- [34] M. Xu, H. Zhang, Y. Hou, Z. Xu, L. Fan, M. Veloso, and S. Song, lin, D. Hoeller, N. Rudin, A. Allshire, A. Handa, and G. State, “Dexumi: Using human hand as the universal manipulation interface “Isaac gym: High performance gpu-based physics simulation for robot for dexterous manipulation,” in CoRL, 2025. learning,” in NeurIPS Datasets and Benchmarks, 2021. [35] B. Huang, Y. Chen, T. Wang, Y. Qin, Y. Yang, N. Atanasov, and [7] X. B. Peng, M. Andrychowicz, W. Zaremba, and P. Abbeel, “Sim- X. Wang, “Dynamic handover: Throw and catch with bimanual hands,” to-real transfer of robotic control with dynamics randomization,” in in CoRL, 2023. ICRA, 2018. [36] X. Liu, H. Wang, and L. Yi, “Dexndm: Closing the reality gap for [8] Y. Narang, K. Storey, I. Akinola, M. Macklin, P. Reist, L. Wawrzyniak, dexterous in-hand rotation via joint-wise neural dynamics model,” Y. Guo, A. Moravanszky, G. State, M. Lu, A. Handa, and D. Fox, arXiv:2510.08556, 2025. “Factory: Fast contact for robotic assembly,” in RSS, 2022. [37] Z.-H. Yin, C. Wang, L. Pineda, F. Hogan, K. Bodduluri, A. Sharma, [9] E. Heiden, M. Macklin, Y. Narang, D. Fox, A. Garg, and F. Ramos, P. Lancaster, I. Prasad, M. Kalakrishnan, J. Malik, M. Lambeta, T. Wu, “Disect: A differentiable simulation engine for autonomous robotic P. Abbeel, and M. Mukadam, “Dexteritygen: Foundation controller for cutting,” in RSS, 2021. unprecedented dexterity,” arXiv:2502.04307, 2025. [38] A. Kumar, T. Power, F. Yang, S. A. Marinovic, S. Iba, R. S. Zarrin, and D. Berenson, “Diffusion-informed probabilistic contact search for multi-finger manipulation,” in ICRA, 2025. [39] M. Noseworthy, B. Tang, B. Wen, A. Handa, C. Kessens, N. Roy, D. Fox, F. Ramos, Y. Narang, and I. Akinola, “Forge: Force-guided exploration for robust contact-rich manipulation under uncertainty,” RA-L, 2025. [40] Y. Jiang, C. Wang, R. Zhang, J. Wu, and L. Fei-Fei, “Transic: Sim- to-real policy transfer by learning from online correction,” in CoRL, 2024. [41] A. Maddukuri, Z. Jiang, L. Y. Chen, S. Nasiriany, Y. Xie, Y. Fang, W. Huang, Z. Wang, Z. Xu, N. Chernyadev, S. Reed, K. Goldberg, A. Mandlekar, L. Fan, and Y. Zhu, “Sim-and-real co-training: A simple recipe for vision-based robotic manipulation,” RSS, 2025. [42] H. Zhang, Z. Li, X. Zeng, L. Smith, K. Stachowicz, D. Shah, L. Yue, Z. Song, W. Xia, S. Levine, et al., “Traversability-aware legged navigation by learning from real-world visual data,” T-RO, 2025. [43] C. Higuera, A. Sharma, T. Fan, C. K. Bodduluri, B. Boots, M. Kaess, M. Lambeta, T. Wu, Z. Liu, F. R. Hogan, and M. Mukadam, “Tactile beyond pixels: Multisensory touch representations for robot manipu- lation,” in CoRL, 2025. [44] T. Lin, Z.-H. Yin, H. Qi, P. Abbeel, and J. Malik, “Twisting lids off with two hands,” in CoRL, 2024. [45] D. Chen, B. Zhou, V. Koltun, and P. Krähenbühl, “Learning by cheating,” in CoRL, 2020. [46] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,” arXiv:1707.06347, 2017. [47] S. Ross, G. Gordon, and D. Bagnell, “A reduction of imitation learning and structured prediction to no-regret online learning,” in AISTATS, 2011. [48] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimiza- tion,” in ICLR, 2015. [49] A. Newell, K. Yang, and J. Deng, “Stacked hourglass networks for human pose estimation,” in ECCV, 2016. [50] T. L. Team, J. Barreiros, A. Beaulieu, A. Bhat, R. Cory, E. Cousineau, H. Dai, C.-H. Fang, K. Hashimoto, M. Z. Irshad, M. Itkina, N. Kuppuswamy, K.-H. Lee, K. Liu, D. McConachie, I. McMahon, H. Nishimura, C. Phillips-Grafflin, C. Richter, P. Shah, K. Srinivasan, B. Wulfe, C. Xu, M. Zhang, A. Alspach, M. Angeles, K. Arora, V. C. Guizilini, A. Castro, D. Chen, T.-S. Chu, S. Creasey, S. Curtis, R. Denitto, E. Dixon, E. Dusel, M. Ferreira, A. Goncalves, G. Gould, D. Guoy, S. Gupta, X. Han, K. Hatch, B. Hathaway, A. Henry, H. Hochsztein, P. Horgan, S. Iwase, D. Jackson, S. Karamcheti, S. Keh, J. Masterjohn, J. Mercat, P. Miller, P. Mitiguy, T. Nguyen, J. Nimmer, Y. Noguchi, R. Ong, A. Onol, O. Pfannenstiehl, R. Poyner, L. P. M. Rocha, G. Richardson, C. Rodriguez, D. Seale, M. Sherman, M. Smith-Jones, D. Tago, P. Tokmakov, M. Tran, B. V. Hoorick, I. Vasiljevic, S. Zakharov, M. Zolotas, R. Ambrus, K. Fetzer-Borelli, B. Burchfiel, H. Kress-Gazit, S. Feng, S. Ford, and R. Tedrake, “A careful examination of large behavior models for multitask dexterous manipulation,” arXiv:2507.05331, 2025. [51] A. Kumar, J. Hong, A. Singh, and S. Levine, “When should we prefer offline reinforcement learning over behavioral cloning?” in ICLR, 2022. [52] L. Pinto, M. Andrychowicz, P. Welinder, W. Zaremba, and P. Abbeel, “Asymmetric actor critic for image-based robot learning,” arXiv:1710.06542, 2017. A PPENDIX TABLE III: Privileged information for nut-bolt task. The screwdriver task uses a subset of these features (excludes A. Privileged Information for Oracle Policy hand pose and PD gains). Nut-Bolt Task. The oracle policy receives privileged infor- Privileged Information Dimension mation about object properties, fingertip states, nut-specific dynamics, hand states, and PD controller gains. The full list Object position 3 of privileged inputs is provided in Table III. Object scale 1 Object mass 1 Screwdriver Task. The screwdriver task uses a subset of the Object friction coefficient 1 privileged information from the nut-bolt task. Specifically, Object center of mass 3 it excludes hand base position, hand orientation, hand joint Object orientation (quaternion) 4 positions, and PD controller gains. All other privileged inputs Object linear velocity 3 remain the same. Object angular velocity 3 Object restitution 1 B. Reward Fingertip positions (2 fingers) 6 Fingertip orientations (2 fingers) 8 Our reward function is a weighted combination of task Fingertip linear velocities (2 fingers) 6 rewards, energy penalties, and stability penalties: Fingertip angular velocities (2 fingers) 6 • Task Rewards encourage successful rotation: Nut contact indicator 1 Nut position 3 ◦ Rotation Rewards: rtrot = clip(ωt , ωmin , ωmax ). It Nut joint velocity 1 encourages positive angular velocity ω along the fas- Nut joint position 1 tening axis, clipped to [−4.0, 4.0] rad/s. Screw joint friction 1 ◦ Proximity Rewards: rtprox = max(0, 1−dt /dthresh ). It Hand scale 1 encourages fingers to remain close to the object, where Hand position 3 dt is the mean distance from thumb and index finger Hand orientation (quaternion) 4 Hand joint positions 12 to the nut/handle. PD controller gains (kp ) 12 • Energy Penalties discourage inefficient motions: PD controller gains (kd ) 12 ◦ Torque Penalty: rttorque = −∥τt ∥2 penalizes large joint Total 97 torques. ◦ Work Penalty: rtwork = −(|τt |⊤ |q̇t |)2 penalizes exces- TABLE IV: Reward function hyper-parameters for nut-bolt sive joint power. and screwdriver tasks. • Stability penalties maintain stable behavior: ◦ Pose Difference Penalty: rtpose = −∥qt − q 0 ∥2 pe- Reward Component nut-bolt Screwdriver nalizes deviations from the initial finger configuration Task Rewards (thumb joints masked). λrotate 6.0 2.5 ◦ Large Rotation Penalty: rtrp = − max(0, ωt − ωthresh ) λproximity 2.0 2.0 penalizes excessive angular velocity above threshold Energy Penalties ωthresh (10.0 rad/s for nut-bolt, curriculum from 7.5 to λtorque -0.1 -3.0 15.0 rad/s for screwdriver). λwork -0.01 -0.01 We sum the above rewards with weights listed in Table IV. Stability Penalties λpose -0.5 -0.1 C. Training Hyperparameters λrotate-penalty -0.3 -0.3 The inputs to our oracle policy contains the proprioceptive λpc-z -1.0 -1.0 observations consist of qt (joint positions over the last 3 timesteps) and at−1 (previous joint targets over the last 3 timesteps). The privileged information includes pt (5 finger- train 1.5 billion environment steps in total, which takes less tip positions), wt (object state with 3D position, quaternion than one day on a single GPU. We train our sensorimotor orientation, and angular velocity), and additional features policy with on-policy behavioral cloning, and the training detailed in Table III. We follow the domain randomization hyperparameters are shown in Table VII. parameters in Table V. We train our oracle policy with PPO, and the training hyperparameters are shown in Table VI. Specifically, we train with 8,192 parallel environments. Each environment gathers 12 steps of data to train in each epoch of PPO. The data is split into minibatches of size 16,384 and optimized with PPO loss. γ and λ are used for computing generalized advantage estimate (GAE) returns. We use the Adam optimizer to train PPO and adopt gradient clipping to stabilize training. We TABLE V: Domain Randomization Parameters. Object scale is discretely sampled from the specified set and mul- tiplied by the base scale. Mass, center of mass, friction, restitution, and PD controller gains are uniformly sampled at environment initialization. Observation and action noise are sampled i.i.d. from Gaussian distributions at each timestep. Following [1], we apply a random disturbance force with magnitude 2.0m (where m is object mass) with probability 0.25 at each timestep.
Parameter Range
Object Scale (nut-bolt) ×[0.95, 1.05]
Object Scale (Screwdriver) ×[0.85, 1.25]
Mass [0.04, 0.06] kg
Center of Mass [-0.001, 0.001] m
Coefficient of Friction [0.5, 8.0]
Object Restitution [0.0, 1.0]
PD Controller Stiffness (kp ) [2.7, 3.3]
PD Controller Damping (kd ) [0.009, 0.011]
Observation Noise (rotation) N (0, 0.01) (rad)
Observation Noise (translation) N (0, 0.005) (m)
Action Noise (rotation) N (0, 0.01) (rad)
Action Noise (translation) N (0, 0.005) (m)
External Force Scale 2.0m
External Force Probability 0.25 per timestep
TABLE VI: Hyperparameters for training the oracle policy.
Hyperparameter Value
# environments 8192
# steps 12
# minibatch size 16384
# Environment steps 3×109
discount factor (γ) 0.99
GAE (λ) 0.95
learning rate 5e-3
clip range 0.2
entropy coefficient 0.0
kl threshold 0.02
max gradient norm 1.0
TABLE VII: Hyperparameters for training the sensorimotor policy in simulation.
Hyperparameter Value
# environments 48
# steps 512
# minibatches 4096
learning rate 1e-3