Learning Dexterous Manipulation Skills from Imperfect Simulations
Elvis Hsieh∗ , Wen-Han Hsieh∗ , Yen-Jen Wang∗ , Toru Lin, Jitendra Malik, Koushil Sreenath† , Haozhi Qi†
UC Berkeley
Screwdrivin
Nut-Bolt Fastenin
arXiv:2512.02011v2 [cs.RO] 25 Feb 2026
Generalizatio
Fig. 1: We propose DexScrew, a sim-to-real framework for learning dexterous manipulation skills when the environment
cannot be accurately simulated. In simulation, we use simplified objects to learn transferable rotational skills, which are then
used to collect data and train tactile policies in the real world. We demonstrate the framework on contact-rich screwdriving
(top row) and nut-bolt fastening (middle row). We also show generalization across different objects (bottom row). More
videos and code are available on https://dexscrew.github.io.
Abstract— Reinforcement learning and sim-to-real transfer However, in practice, sim-to-real transfer faces two major
have made significant progress in dexterous manipulation. How- limitations. First, due to the complexity of physics simula-
ever, progress remains limited by the difficulty of simulating tion, only a limited range of tasks can be accurately modeled.
complex contact dynamics and multisensory signals, especially
tactile feedback. In this work, we propose DexScrew, a sim- Prior work either relies on specialized techniques for high-
to-real framework that addresses these limitations and demon- fidelity simulation [8], [9] or seeks generalization through
strates its effectiveness on nut-bolt fastening and screwdriving domain randomization [1], [10]–[12]. However, as the task
with multi-fingered hands. The framework has three stages. becomes more dynamic, the sim-to-real gap grows [13],
First, we train reinforcement learning policies in simulation and simulation alone becomes insufficient. Second, existing
using simplified object models that lead to the emergence
of correct finger gaits. We then use the learned policy as a sensing modalities have an intrinsic sim-to-real gap. While
skill primitive within a teleoperation system to collect real- vision can be partially mitigated through domain random-
world demonstrations that contain tactile and proprioceptive ization [14], tactile sensing remains difficult to approximate
information. Finally, we train a behavior cloning policy that reliably. Although some work aims to improve tactile sim-
incorporates tactile sensing and show that it generalizes to nuts ulation [15], [16] or trains policies using alternative proxy
and screwdrivers with diverse geometries. Experiments across
both tasks show high task progress ratios compared to direct representations [17]–[19], these approaches cannot leverage
sim-to-real transfer and robust performance even on unseen the full power of tactile sensing. These limitations remain
object shapes and under external perturbations. widely viewed as major constraints on the complexity of
tasks that can be achieved.
I. I NTRODUCTION On the other hand, teleoperation and imitation learn-
Reinforcement learning (RL) paired with sim-to-real trans- ing [20], [21] remove the need for simulation entirely. In this
fer has recently delivered a number of promising results in setting, policies can learn directly from real-world interac-
dexterous manipulation [1]–[5]. Policies trained in massively tions and sensorimotor signals, which avoids the challenges
parallel simulation [6] with domain randomization [7] have introduced by sim-to-real transfer. However, teleoperating
demonstrated strong robustness and generalization capabili- dexterous hands is challenging because of the intrinsic mor-
ties in the real world. phology differences between human and robot hands [22]–
[24]. As a result, it is difficult to collect datasets that are
∗ Equal contribution (listed in alphabetical order). large and diverse enough to achieve the desired behavior
† Equal advising. and generalization.
Motivated by these observations, we introduce DexScrew, Both directions, however, face notable limitations. Sim-
a framework that combines the strengths of both approaches to-real methods benefit from large-scale simulated data and
to expand the capability of sim-to-real reinforcement learning can generalize across diverse objects, yet they remain lim-
under imperfect simulation. The key idea is that the motion ited by inaccuracies in modeling complex contact dynamics
primitives underlying contact-rich dexterous manipulation do and sensing, a challenge that becomes more significant
not need to be learned from a perfect physics model. A as task complexity increases [13], [35]. Imitation learning
simplified simulator is sufficient to induce the core rotational benefits from multisensory real-world data, yet collecting
behaviors required for these tasks. Once this motion is high-quality dexterous demonstrations is considerably more
learned, the resulting policy can be used as a skill prim- difficult than collecting data for simpler end-effectors. Our
itive to collect real-world demonstrations, from which a work seeks to combine the strengths of both approaches.
new policy can be learned. In this way, sensing modalities We use large-scale simulation to learn motion primitives
and physical interactions that are difficult to simulate can while leveraging real-world data to close the dynamics and
be obtained directly from real-world data, while the fine- sensing gaps. Moreover, our skill-based framework enables
grained motions that are hard to teleoperate are provided efficient collection of dexterous real-world data by using the
by the simulation-trained policy. We demonstrate this idea simulation-trained policy itself as a reusable skill primitive.
through two tasks, nut-bolt fastening and screwdriving with In the context of nut fastening and screwdriving, there has
a multifingered hand. Both tasks are traditionally viewed been recent work combining sim-to-real transfer with teleop-
as requiring complex contact dynamics understanding and eration. For example, Liu et al. [36] build a residual model
tactile sensing. We instead show that effective policies can from real-world interactions to compensate for the sim-to-
be learned without relying on high-fidelity simulation. real gap and achieve robust in-hand manipulation. Yin et
More specifically, our framework consists of three stages. al. [37] use simulation-trained policies as stability controllers
First, we train reinforcement learning (RL) policies in simu- to enable complex manipulation skills. Both approaches can
lation using a simplified physics model. Instead of modeling be integrated with teleoperated arm control to complete these
the thread structure of the nut and screw, we approximate tasks. However, they do not produce autonomous policies
their interaction with a revolute joint that connects two sim- that incorporate tactile sensing. Another line of work is
ple geometric shapes, which allows the policy to efficiently Kumar et al. [38], who demonstrate screwdriver turning by
learn rotational behavior. Second, we use this learned skill combining learning with trajectory optimization. Noseworthy
as a primitive within a teleoperation system to collect real- et al. [39] present an autonomous sim-to-real policy but
world demonstrations. The operator controls the arm motion only show results with a parallel-jaw gripper and do not
and triggers the finger rotation skill rather than issuing low- demonstrate regrasping.
level joint commands, which enables efficient collection of Another way to address the sim-to-real gap is to refine the
tactile data during teleoperated execution. Finally, using the policies in the real world. For example, Transic [40] shows
resulting multisensory dataset, we train a behavior cloning that sim-to-real policies can adapt to complex real-world
policy that coordinates arm and finger motions while lever- dynamics with only a few human interventions as demonstra-
aging tactile feedback. tions, although the demonstrations are primarily performed
We evaluate our framework on two tasks: nut-bolt fasten- with simple end-effectors. In contrast, we apply this idea
ing and screwdriving. Policies trained with simplified dynam- to dexterous hands. Maddukuri et al. [41] show that co-
ics can generate reasonable rotational behavior but cannot training with both simulation and real-world data can reduce
complete the tasks. By learning from real-world multisensory the gap and improve manipulation performance. RLPD [42]
demonstrations, our method overcomes these limitations and and Sparsh-X [43] share a similar philosophy of augmenting
achieves stable and reliable performance under challenging modalities through human-guided data collection; however,
contact conditions. These results show that complex contact- they are not motivated by the possibility of learning skills
rich manipulation skills can be bootstrapped from simplified from imperfect simulations, as their policies are designed to
simulators and that real-world tactile feedback is essential. be directly transferable.
Our framework provides a scalable path toward dexterous
manipulation and supports broader deployment of general- III. D EXTERITY FROM I MPERFECT S IMULATION
purpose robot hands in unstructured environments.
An overview of our method is shown in Figure 2. It
II. R ELATED W ORK consists of three stages. First, we train a reinforcement
Dexterous manipulation has been a long-standing chal- learning (RL) policy in simulation using a simplified object
lenge in robotics [25], [26]. Early work focused on classical model (Section III-A). The resulting policy learns the desired
model-based control and analytic grasp planning [27]–[31]. finger motions but does not experience real-world dynamics
Recent years have seen rapid progress in learning-based ap- and lacks tactile feedback. To address this, we collect real-
proaches, which can be grouped into two primary directions: world trajectories using the learned policy as a skill primitive
sim-to-real learning paired with reinforcement learning [1], for teleoperation (Section III-B). Finally, using this dataset,
[4], [5] and imitation learning from teleoperation [23], [24], we train a new multisensory policy using behavior cloning
[32] or human data [33], [34]. (Section III-C).
(A) Train RL Policy in Simulation (B) Real World Data Collection using Learned Policy (C) BC with Multi-sensory Data
Human Arm Movement
Teleoperator
Activation of
RL Policy
Policy Learned with Imperfect Simulation
Fig. 2: An overview of our approach. We first train a reinforcement learning policy in simulation using a simplified object
model, which serves as a motion prior for nut-bolt fastening and screwdriving. We then collect real-world trajectories by
using the learned policy as a skill primitive during teleoperation. Finally, we train a behavior cloning policy on the collected
data to obtain coordinated behavior between the arm and the fingers.
A. Training a Reinforcement Learning Policy in Simulation For the Nut-Bolt Task For the Screwdriving Task
Simplified Object Modeling. Our goal is to design a Simplified
simulation environment that enables fast training and en- Nut / Handle
courages the emergence of desired finger gaits for rotation.
To achieve this, we construct a simplified simulated object
(Figure 3) that captures the essence of rotational motion. Revolute Joint
The object consists of a fixed cylindrical base with a nut or
handle attached via a revolute joint. This setup allows the Simplified Bolt / Screw
policy to learn rotational motion efficiently without relying
on expensive contact-rich simulations. A similar idea was
explored in [44] to model bottle caps using a heuristic friction
design. In contrast, we further simplify the model, since we Fig. 3: Simplified Object Models. Each nut or handle is
can leverage real-world data to compensate for the resulting modeled as a rigid body attached to a fixed base through a
dynamics mismatch. revolute joint. This abstraction ignores thread-level mechan-
Specifically, for the nut-bolt task, we use a thick triangular ics while retaining the essential rotational dynamics needed
shape as the training object (Figure 3A). The extra thickness for learning.
is used to prevent the policy from learning suboptimal strate-
gies that apply a large force from the bottom. The learned
policy also discovers a high-clearance gait that transfers Actions. At each step, the policy outputs a relative target
well to diverse real-world nuts such as hexagonal and cube- position. The position command is computed as aHand t =
shaped nuts. For the screwdriver task, where the primary ηf (oRL ) + aHand
t−1 , where η is the action scale. This command
difficulty arises from slippage around the handle, we use is sent to the robot and converted into torque via a low-level
spherical primitives to keep the learned behavior conservative PD controller. Here, oRL contains the robot’s proprioceptive
(Figure 3B). This observation, that different training shapes state, including joint positions and previous target positions
lead to different rotational gaits, is also consistent with the from a sliding window of recent 3 timesteps.
findings in [5]. Note that these objects do not need to be Reward. The goal of the policy in simulation is to rotate
visually aligned with real world objects, as they are only the simplified object around the revolute joint. The reward
used to learn the coarse motions used for real world data consists of a task reward, energy penalties, and stability
collection, as we discussed in Section III-B. penalties (time index t omitted for simplicity). Each com-
Training Pipeline. Following [13], [17], we first train an ponent includes several terms defined in the appendix:
oracle policy and then distill it into a sensorimotor policy. rt = λtask rtask + λenergy rtenergy + λstability rtstability .
The oracle policy f is trained with access to an embedding
of privileged information [45] zt . The sensorimotor policy Oracle Policy Training. We train the oracle policy using
operates without privileged sensing and instead conditions proximal policy optimization (PPO) [46] with the reward
on a predicted embedding ẑt = ϕ(ht ) inferred from propri- described above. The robot state and privileged information
oceptive history ht by a prediction module ϕ. are each encoded with separate MLPs. These embeddings
Privileged Information. The oracle policy has access to are concatenated and passed through an MLP to produce the
ground-truth environment and object properties, including final action and value predictions. We train the policy for
object attributes (e.g., position, scale, mass, center of mass, 1.5×109 environment steps.
friction coefficients), hand pose and finger configurations, Sensorimotor Policy Training. The sensorimotor policy
and low-level controller parameters. The full set of privileged receives proprioceptive states and a latent code ẑt = ϕ(ht )
inputs is documented in the appendix. inferred from a 30-timestep history. We train the policy
Joystick: Horizontal Translation for finger motion control. Instead of commanding every
joint individually, the human operator controls only the wrist
movement and decides when to activate the skill primitive
(Figure 4). Wrist position is specified using the Quest VR1
controller’s joystick. This approach is inspired by [32], but
Wrist Up we use a much finer-grained skill for data collection.
This framework offers several advantages. First, it dele-
gates complex finger motions to a robust simulation-trained
policy that generalizes across different objects, eliminating
the need for humans to learn finger coordination under
Wrist Down Policy Activation morphological differences. Second, using a joystick for arm
control enables precise and intuitive wrist positioning. While
Fig. 4: Teleoperation Interface. The human operator con- arm motion is relatively straightforward to simulate, we
trols the wrist position using the VR controller buttons and intentionally exclude it from the initial simulation training
adjusts yaw and pitch through the joystick. This setup allows stage, as learning the specific downward progression required
the operator to guide the arm motion while relying on the for nut-bolt fastening would necessitate simulating thread-
learned finger-rotation skill during data collection. level mechanics. Finally, collecting data in the physical envi-
ronment provides the multisensory observations, particularly
tactile signals, that are essential for reliable task completion
using DAgger [47]: at each step, the sensorimotor policy acts but remain difficult to approximate reliably in simulation.
in the environment, while the oracle policy provides target Concretely, at each timestep we record two actions: (1) the
actions and ground-truth privileged embeddings. The training action generated by the RL policy πRL , which defaults to the
objective is current hand joint positions when the policy is not activated,
and (2) the arm action generated by human teleoperation.
L = ∥aHand
t − âHand
t ∥22 + ∥zt − ẑt ∥22 , Formally, we define at = [aHand t , aArm
t ], where at
Hand
∈ R12
denotes the hand target joint positions, and at ∈ R6 de-
Arm
where âHand
t denotes the actions produced by the sensorimo-
notes the arm joint positions. We also record the multisensory
tor policy. The embedding predictor ϕ is optimized using
observation qt , ct , where qt = [qtHand , qtArm ] contains all
Adam [48] until convergence.
joint positions, and ct represents the raw tactile signals from
Randomization. We apply domain randomization during all five fingers.
training to improve the robustness of the RL policy [7].
Tactile Signal. In this work, we use the XHand’s built-in
Following [13], we randomize the nut/handle mass, center
tactile sensors to capture contact information. Each fingertip
of mass, friction coefficient, size, and PD gains, and we also
is equipped with a pressure-based tactile array comprising
add observation noise. Detailed parameters are provided in
120 sensing elements, each measuring three-axis forces with
the appendix.
a minimum detectable force of 0.05 N. At each timestep, we
Termination Conditions. To prevent the policy from getting record the tactile signal as ct ∈ R5×120×3 , which includes
stuck in unrecoverable states, we terminate an episode when the signals from all five fingers across three axes.
any of the following conditions are met: (1) the distance be-
tween the thumb or index finger and the nut/handle exceeds a C. Behavior Cloning with Multisensory Data
reset threshold (7 cm for nuts, 10 cm for handles); (2) the nut
or handle remains stagnant over a sliding time window (3.5 s With the dataset DReal collected using the skill-based
for nuts, 3 s for handles); or (3) near-zero contact forces are assisted teleoperation, we can train a behavior cloning (BC)
detected for the same duration. These conditions accelerate policy πBC using the paired multisensory observations and
training by penalizing failure modes such as drifting away expert actions. Vision is not used in our work.
from the object or failing to maintain contact. Neural Network Architecture. We use a feedforward net-
work as the policy. The past K timesteps of observations
B. Real World Data Collection with Learned Policy (qt−K+1:t , ct−K+1:t ) are concatenated into a single feature
The policy trained in simulation with the simplified object vector. Tactile signals are first flattened and passed through
model learns the desired rotational behavior; however, it an MLP. The fused feature vector is then processed by an
inevitably misses key physical dynamics such as thread hourglass encoder [49], which outputs the action predictions.
interactions. These effects are difficult to model but are We also apply an action chunking strategy [20], [21], where
crucial for reliable real-world fastening. It also lacks tactile the policy predicts a sequence of future actions ât:t+H rather
information, which is hard to simulate but crucial for fine- than a single-step action. We use K = 5 and H = 16 unless
grained wrist adjustments. otherwise noted.
To bridge this gap, we introduce a skill-based assisted Training. The policy is trained with supervised learning
teleoperation for real-world data collection. The core idea
is to reuse the simulation-trained policy as a skill primitive 1 https://www.meta.com/quest/products/quest-2
TABLE I: Real-world fastening performance on square, triangular, hexagonal, and cross-shaped nuts. We report progress
ratio and rotation time (mean ± standard deviation over 10 trials) for different observation modalities. Tactile sensing and
temporal history both improve performance, and their combination yields the highest accuracy and fastest execution.
indicates that only one completely successful trial was recorded, so no standard deviation is reported.
Square Nuts Triangular Nuts Hexagonal Nuts Cross-Shaped Nuts
Tactile Hist. Prog. Ratio (%) ↑ Time (s) ↓ Prog. Ratio (%) ↑ Time (s) ↓ Prog. Ratio (%) ↑ Time (s) ↓ Prog. Ratio (%) ↑ Time (s) ↓
63.75±33.05 148.76±30.57 30.00±39.62 229.39 75.00±29.46 102.36±25.00 63.75±33.05 101.07±0.51
✓ 62.50±33.85 202.21±59.69 66.25±27.67 134.19±46.72 75.00±16.67 205.02±82.44 82.50±10.54 127.17±47.98
✓ 87.50±20.41 129.88±48.06 80.00±32.91 111.47±55.10 85.00±32.17 68.97±13.99 100.00±00.00 91.49±44.89
✓ ✓ 97.50±7.91 124.20±33.22 96.25±8.44 117.79±52.13 95.00±10.54 75.07±17.41 98.75±3.95 84.21±57.19
using the loss not achieve a single successful fastening or screwing attempt
T X
H across ten trials. In such cases, we cannot report the standard
deviation or the completion-time metric.
X 2
LBC = ∥ ât+h − at+h ∥2 ,
t=1 h=0
B. Nut-Bolt Experiments
where ât:t+H is the action chunk predicted by πBC , and
at:t+H is the corresponding expert sequence. This objec- We first evaluate the system on the nut-bolt task. This
tive encourages consistent predictions over the full horizon. task requires the fingers to establish correct contact patterns,
We train the policy using Adam [48] for 200 epochs and sense progress through tactile feedback, and adjust the arm
normalize observations following [50]. position accordingly. We choose this task because nut-bolt
interactions are difficult to simulate efficiently, and complet-
IV. E XPERIMENTS ing the task relies heavily on tactile sensing, making it a
In this section, we first introduce the experiment setup strong testbed for our method.
(Section IV-A). We then evaluate the performance of our We note that direct sim-to-real transfer can rotate the nut,
policies on two challenging tasks, nut-bolt fastening (Sec- but it cannot drive the nut downward because the arm does
tion IV-B) and screwdriving (Section IV-C). We conclude not move. Since thread interactions are not simulated, the
with qualitative experiments that provide additional analysis nut remains at the same height even after completing full
and design ablations in simulation. revolutions in simulation.
Setting. In simulation, we use the thick triangular nut shown
A. Experiment Setup
in Figure 3 (left). Training with this shape produces high-
Hardware Setup. Our system consists of a UR5e robot clearance gaits that transfer well and can rotate both square
arm (6 DoF) and a 12-DoF XHand. The XHand has five and triangular nuts in the real world.
fingers: the thumb and index finger each have 3 DoF, and During skill-based assisted teleoperation, we collect 50
the remaining fingers have 2 DoF. Only the thumb and index trajectories each for the square and triangular nuts. Each
provide abduction/adduction. trajectory lasts about 80 seconds. We then train a behavior
Simulation. We train our policies in IsaacGym [6] using cloning policy using the combined dataset. We evaluate
8,192 parallel environments. Each environment contains a performance on four types of nuts, which include square
simulated XHand and the simplified object model described and triangular nuts as well as two unseen shapes, namely
in Section III-A. The simulation runs at 200 Hz, with control hexagonal nuts and cross-shaped nuts. Our main results are
applied at 20 Hz. Each episode lasts up to 800 simulation shown in Table I.
steps (40 s). Observation History. We first study the effect of providing
Object Set. For the nut-bolt task, we train on triangular a short temporal history in the observation. Adding history
nuts. For the screwdriver task, we approximate handles by significantly improves progress ratio and reduces execution
octagon- and dodecagon-shaped nuts. This multi-geometry time across all modalities and nut geometries. Temporal cues
training in simulation helps the policy generalize to diverse help the policy track fine-grained rotational progress and
real-world shapes. distinguish local geometric features. The benefit is especially
Metrics. In simulation, we report the episode reward and clear for non-tactile policies, where history stabilizes perfor-
episode length during training. In real-world evaluation, mance and narrows much of the gap to tactile-based policies
we measure the progress ratio, defined as the number of on easier geometries. When combined with tactile sensing,
successful rotations divided by the total number of rotations history yields the strongest overall performance, achieving
required for full fastening. We also report the time for trials the highest accuracy and fastest completion times across all
that fully complete the fastening process (progress ratio = nut types, including unseen shapes.
100), defined as the time needed to fully fasten the nut or Tactile Information. Across all nut geometries, adding
fully tighten the screw. Note that some baseline methods do tactile input generally improves progress ratio. The effect
TABLE II: Real-world screwdriving performance. We Policy with Tactile Info
report progress ratio and rotation time (mean ± standard
deviation over 10 trials) for direct sim-to-real, expert replay,
and our behavior cloning (BC) ablations. Tactile sensing Thumb
Index Middle Thumb
Index Middle Thumb
Index Middle Thumb
Index Middle Thumb
Index Middle
and temporal history each improve performance, and their
combination achieves the highest progress ratio and fastest Policy without Tactile Info
execution. Here * indicates that the policy never fully com-
pleted the task, so no rotation time is reported.
Screwdriver Task Thumb
Index Middle Thumb
Index Middle Thumb
Index Middle Thumb
Index Middle Thumb
Index Middle
Method Tactile Hist. Prog. Ratio (%) ↑ Time (s) ↓
Direct Sim2Real 41.60±26.21 N.A.
Fig. 5: Top: The policy with tactile information maintains
Expert Replay 50.80±19.27 N.A.
a consistent alternating pattern of thumb and index finger
69.20±35.25 266.33±91.60 contact, which supports stable engagement as the nut is
✓ 67.63±35.65 264.06±76.57 rotated downward. Bottom: The policy without tactile infor-
Ours
✓ 87.50±18.61 195.15±44.83
✓ ✓ 95.00±13.24 187.87±24.87
mation does not maintain a clear contact pattern. This leads
to unsuccessful engagement and prevents proper downward
wrist motion. The resulting pattern reflects the index finger
pressing against the bolt after losing stable contact.
is most pronounced on challenging shapes such as triangu-
lar and cross-shaped nuts, where progress ratios rise from
roughly 30 to 65 percent for triangular nuts and from about
Figure 3 (right). This curated set encourages the learned rota-
60 to 80 percent for cross-shaped nuts. This underscores
tional gaits to remain conservative in clearance and maintain
the importance of tactile feedback for maintaining stable
stability. In the real world, we collect 72 trajectories, each
contact and detecting effective rotation. We also observe
lasting between 120 and 180 seconds.
that certain non-tactile settings achieve high performance,
even on unseen shapes, particularly on well-constrained Sim-to-Real Policy. Unlike the nut-bolt experiments, the
geometries such as hexagonal and cross-shaped nuts. In these screwdriving task can still make progress even when the
cases, the geometry provides strong passive guidance, and wrist does not move, since downward motion is not required.
the finger gait learned in simulation is often sufficient to We first evaluate a direct sim-to-real policy that uses only
maintain contact without relying on tactile cues. proprioception. The results are shown in Table II. The
Failure Modes. We observe two main failure modes. First, direct sim-to-real policy achieves a 41.60% progress ratio,
the policy without observation history struggles to infer indicating that it can produce meaningful behavior, which
object shape from single-step proprioception. As a result, it is a prerequisite for collecting expert trajectories. However,
fails to generalize across nut geometries, since different nuts because it inevitably makes mistakes, it never completes the
require different rotational gaits. The policy cannot adjust its task even once, and therefore we cannot report statistics
gait because the limited sensing information prevents it from for completion time. In contrast, during data collection, the
identifying the correct motion pattern. Second, non-tactile human operator can adjust the wrist position to recover from
policies frequently drift into unstable contact states and lose such mistakes.
alignment with the bolt. Once misalignment occurs, the nut As a result, when we replay the expert data from the
cannot sustain continuous rotation. In contrast, tactile poli- data we collected, it can achieve higher success rate to be
cies can recover by adjusting wrist orientation or applying 50.80%. However, because it cannot adapt changes during
corrective downward force to re-establish contact. deployment. It also fails at completely finishing the task.
Next, we show the results of our policies.
C. Screwdriving Experiments Main Results. Our behavioral cloning policies show clear
We also demonstrate that our method extends to the improvements over the direct sim-to-real and expert-replay
more challenging screwdriving task. Compared to nut-bolt baselines. Adding tactile sensing or temporal history indi-
fastening, screwdriving is inherently less stable: the shaft vidually improves progress ratio. Combining both modalities
is not kinematically constrained along the screw axis, and gives the strongest performance.
even small tilts or misalignments can lead to slipping or The baseline behavior cloning model already achieves
loss of contact. As a result, the task requires more fine- a 69.20% progress ratio, which substantially outperforms
grained control to maintain continuous rotation. The complex expert replay. This phenomenon, where a behavior cloning
interaction between the screwdriver and the screw is also policy surpasses the policy that generated the data, is con-
difficult to simulate accurately, which is why we include this sistent with filtered behavior cloning [51]. In this approach,
task in our study. only successful trials are used for training. A similar effect
Setting. In simulation, we use a mix of octagonal and has also been reported in [13].
dodecagonal handles as the training objects, as shown in Using history alone produces a comparable improvement
Finger Dragging Recover Episode Reward Episode Length (s)
2000 40
1500 30
1000 20
500 10
Reverse Rotation Recover
0 0
0 600 1200 1800 2400 3000 0 600 1200 1800 2400 3000
Environment Steps (M) Environment Steps (M)
Ours (Oracle) Ours (Sensorimotor) No Priv. Info Asym. Actor-Critic
Fig. 7: Simulation ablations of screwdriving policy training.
Fig. 6: Top row: The policy recovers back to the nut- We compare our privileged-information oracle policy, its
bolt fastening motion when the fingers are dragged by an sensorimotor policy, an asymmetric actor–critic variant, and
external force. Bottom row: The policy recovers back to a policy trained without privileged information. Providing
the screwdriving motion when the screwdriver is rotated privileged information during training leads to significantly
counterclockwise during the clockwise rotation by the policy. higher reward and more stable episode lengths. Each curve
shows the mean and standard deviation over 5 seeds.
of 67.63%. This indicates that temporal information helps the
policy track rotational progress and recover from partial fail- emerge when the nut or screwdriver handle is correctly
ures. When history is not used, tactile information provides engaged. The policy learns to preserve these patterns by
only limited benefit. However, once history is included, the adjusting wrist orientation and contact force, effectively
two modalities become complementary. The progress ratio using tactile feedback as a local reference for alignment.
increases to 95.00%, and the average rotation time decreases When these patterns deviate, corrective actions such as re-
substantially. These results show that tactile feedback and engagement or downward pressing are triggered.
temporal history work together to produce stable, consistent, E. Simulation Ablations
and efficient screwdriving behaviors.
In simulation, we verify the design choices used to
Failure Modes. We observe that open-loop baselines fre- train the screwdriving policies. We evaluate our two-stage
quently fail due to gradual handle slipping and accumulated training procedure against two alternatives: training without
orientation drift. Without feedback, small misalignments any privileged information, and using an asymmetric actor-
grow over time. Among behavioral cloning methods, the critic [52] architecture where the critic has access to full
policy trained without observation history struggles to stabi- tactile information while the actor does not. The actor in this
lize the screwdriver since it lacks the temporal information setting is directly deployable in the real world. The results
needed to infer handle pose and orientation. Non-tactile are shown in Figure 7.
policies fail to detect subtle torque imbalances and often lose We compare episode reward and episode length across
stable contact under slight perturbations. With both tactile these training strategies. When both the actor and critic
feedback and temporal history, BC with tactile and history have access to privileged information, our method (Ours,
compensates for these effects by adjusting wrist orientation Oracle) achieves the highest performance. Removing privi-
and applying appropriate forces. leged information from the actor, as in the asymmetric actor-
D. Qualitative Experiments critic variant, leads to a noticeable drop in performance.
Removing privileged information entirely results in a further
Out-of-distribution Robustness. We study the robustness decline. These results show that privileged information plays
of the learned policy under external perturbations that are an important role during policy learning. We also observe
not encountered during training. These disturbances include that a sensorimotor policy can approach similar performance
(1) dragging the fingers away from the object or rotating when proprioceptive history is provided as input. We also ex-
the screwdriver in the opposite direction of the intended perimented with training the asymmetric and non-privileged
direction. In Figure 6, we show that despite these pertur- models for longer horizons but have not observed further
bations, the policy consistently recovers to stable fastening improvement. This suggests that the performance gap is not
behavior. Specifically, it recovers from reverse rotation of the due to insufficient training but instead reflects the importance
fastening object by re-establishing contact and restoring the of privileged information during learning.
correct rotational direction. When finger contact is disrupted
or temporarily blocked, the policy repositions the fingers and V. C ONCLUSION AND F UTURE W ORK
wrist to regain stable engagement. We present a framework for learning dexterous manipula-
Tactile Visualization. Tactile signals provide structured spa- tion skills for contact-rich tasks using imperfect simulation.
tiotemporal patterns that the policy uses to infer contact The approach first learns transferable rotational skills through
phases. As shown in Fig. 5, stable activation signatures reinforcement learning with simplified object modeling. It
then uses these skills for skill-based teleoperation to col- [10] J. Lee, J. Hwangbo, L. Wellhausen, V. Koltun, and M. Hutter,
lect real-world trajectories, and finally incorporates tactile “Learning quadrupedal locomotion over challenging terrain,” Science
robotics, 2020.
feedback and learns a sensorimotor policy through behavior [11] A. Kumar, Z. Fu, D. Pathak, and J. Malik, “Rma: Rapid motor
cloning. Experiments on nut-bolt fastening and screwdriver adaptation for legged robots,” in RSS, 2021.
usage show that simulation alone cannot capture the complex [12] A. Loquercio, E. Kaufmann, R. Ranftl, M. Müller, V. Koltun, and
D. Scaramuzza, “Learning high-speed flight in the wild,” Science
dynamics required for reliable task execution. However, Robotics, p. eabg5810, 2021.
when behavior cloning is combined with tactile sensing and [13] J. Wang, Y. Yuan, H. Che, H. Qi, Y. Ma, J. Malik, and X. Wang,
temporal history, the resulting policies become robust and “Lessons from learning to spin ‘pens’,” in CoRL, 2024.
[14] R. Singh, A. Allshire, A. Handa, N. Ratliff, and K. Van Wyk, “Dextrah-
reliable across diverse and unseen object geometries. This rgb: Visuomotor policies to grasp anything with dexterous hands,”
staged pipeline provides a practical and scalable solution for arXiv:2412.01791, 2024.
contact-rich manipulation and highlights the value of tactile [15] S. Wang, M. Lambeta, P.-W. Chou, and R. Calandra, “Tacto: A fast,
flexible, and open-source simulator for high-resolution vision-based
sensing and skill-based teleoperation as effective bridges tactile sensors,” RA-L, 2022.
between simulation and real-world deployment. [16] I. Akinola, J. Xu, J. Carius, D. Fox, and Y. Narang, “Tacsl: A library
for visuotactile sensor simulation and learning,” T-RO, 2025.
Limitations. Although our skill-based teleoperation reduces
[17] H. Qi, B. Yi, S. Suresh, M. Lambeta, Y. Ma, R. Calandra, and J. Malik,
the burden on the human operator, it remains a constraint for “General in-hand object rotation with vision and touch,” in CoRL,
scalable data acquisition. Fully autonomous data collection 2023.
or learning skill-level guidance from human videos would [18] M. Yang, C. Lu, A. Church, Y. Lin, C. Ford, H. Li, E. Psomopoulou,
D. A. Barton, and N. F. Lepora, “Anyrotate: Gravity-invariant in-hand
further improve efficiency. In our current tasks, the nut is object rotation with sim-to-real touch,” in CoRL, 2024.
already installed and the screwdriver is already inserted. [19] J. Yin, H. Qi, J. Malik, J. Pikul, M. Yim, and T. Hellebrekers,
Extending the approach to fully long-horizon assembly will “Learning in-hand translation using tactile skin with shear and normal
force sensing,” in ICRA, 2025.
require vision sensing and possibly high-accuracy force- [20] C. Chi, Z. Xu, S. Feng, E. Cousineau, Y. Du, B. Burchfiel, R. Tedrake,
torque sensing. A broader evaluation across more contact- and S. Song, “Diffusion policy: Visuomotor policy learning via action
rich manipulation tasks is also needed to assess generality. diffusion,” IJRR, 2024.
[21] T. Z. Zhao, V. Kumar, S. Levine, and C. Finn, “Learning fine-grained
bimanual manipulation with low-cost hardware,” in RSS, 2023.
ACKNOWLEDGMENT [22] S. P. Arunachalam, S. Silwal, B. Evans, and L. Pinto, “Dexterous im-
itation made easy: A learning-based framework for efficient dexterous
This work is supported in part by the program “Design manipulation,” in ICRA, 2023.
of Robustly Implementable Autonomous and Intelligent Ma- [23] S. P. Arunachalam, I. Güzey, S. Chintala, and L. Pinto, “Holo-dex:
chines (TIAMAT)”, Defense Advanced Research Projects Teaching dexterity with immersive mixed reality,” in ICRA, 2023.
[24] Y. Qin, W. Yang, B. Huang, K. Van Wyk, H. Su, X. Wang, Y.-W. Chao,
Agency award number HR00112490425. We thank Mengda and D. Fox, “Anyteleop: A general vision-based dexterous robot arm-
Xu for his valuable feedback. hand teleoperation system,” in RSS, 2023.
[25] A. Bicchi, “Hands for dexterous manipulation and robust grasping: A
R EFERENCES difficult road toward simplicity,” IEEE Transactions on robotics and
automation, 2002.
[1] OpenAI, M. Andrychowicz, B. Baker, M. Chociej, R. Józefowicz, [26] A. M. Okamura, N. Smaby, and M. R. Cutkosky, “An overview of
B. McGrew, J. Pachocki, A. Petron, M. Plappert, G. Powell, A. Ray, dexterous manipulation,” in ICRA, 2000.
J. Schneider, S. Sidor, J. Tobin, P. Welinder, L. Weng, and W. Zaremba, [27] D. Rus, “In-hand dexterous manipulation of piecewise-smooth 3-d
“Learning dexterous in-hand manipulation,” IJRR, 2019. objects,” IJRR, 1999.
[2] OpenAI, I. Akkaya, M. Andrychowicz, M. Chociej, M. Litwin, [28] I. Mordatch, Z. Popović, and E. Todorov, “Contact-invariant optimiza-
B. McGrew, A. Petron, A. Paino, M. Plappert, G. Powell, R. Ribas, tion for hand manipulation,” in Eurographics, 2012.
J. Schneider, N. Tezak, J. Tworek, P. Welinder, L. Weng, Q. Yuan, [29] R. Fearing, “Implementing a force strategy for object re-orientation,”
W. Zaremba, and L. Zhang, “Solving rubik’s cube with a robot hand,” in ICRA, 1986.
arXiv:1910.07113, 2019. [30] A. S. Morgan, K. Hang, B. Wen, K. Bekris, and A. M. Dollar,
[3] A. Handa, A. Allshire, V. Makoviychuk, A. Petrenko, R. Singh, J. Liu, “Complex in-hand manipulation via compliance-enabled finger gaiting
D. Makoviichuk, K. Van Wyk, A. Zhurkevich, B. Sundaralingam, and multi-modal planning,” RA-L, 2022.
Y. Narang, J.-F. Lafleche, D. Fox, and G. State, “Dextreme: Transfer [31] S. Patidar, A. Sieler, and O. Brock, “In-hand cube reconfiguration:
of agile in-hand manipulation from simulation to reality,” in ICRA, Simplified,” in IROS, 2023.
2023. [32] T. Lin, Y. Zhang, Q. Li, H. Qi, B. Yi, S. Levine, and J. Malik,
[4] T. Chen, M. Tippur, S. Wu, V. Kumar, E. Adelson, and P. Agrawal, “Learning visuotactile skills with two multifingered hands,” in ICRA,
“Visual dexterity: In-hand reorientation of novel and complex object 2025.
shapes,” Science Robotics, 2023. [33] C. Wang, H. Shi, W. Wang, R. Zhang, L. Fei-Fei, and C. K. Liu,
[5] H. Qi, A. Kumar, R. Calandra, Y. Ma, and J. Malik, “In-hand object “Dexcap: Scalable and portable mocap data collection system for
rotation via rapid motor adaptation,” in CoRL, 2022. dexterous manipulation,” in RSS, 2024.
[6] V. Makoviychuk, L. Wawrzyniak, Y. Guo, M. Lu, K. Storey, M. Mack- [34] M. Xu, H. Zhang, Y. Hou, Z. Xu, L. Fan, M. Veloso, and S. Song,
lin, D. Hoeller, N. Rudin, A. Allshire, A. Handa, and G. State, “Dexumi: Using human hand as the universal manipulation interface
“Isaac gym: High performance gpu-based physics simulation for robot for dexterous manipulation,” in CoRL, 2025.
learning,” in NeurIPS Datasets and Benchmarks, 2021. [35] B. Huang, Y. Chen, T. Wang, Y. Qin, Y. Yang, N. Atanasov, and
[7] X. B. Peng, M. Andrychowicz, W. Zaremba, and P. Abbeel, “Sim- X. Wang, “Dynamic handover: Throw and catch with bimanual hands,”
to-real transfer of robotic control with dynamics randomization,” in in CoRL, 2023.
ICRA, 2018. [36] X. Liu, H. Wang, and L. Yi, “Dexndm: Closing the reality gap for
[8] Y. Narang, K. Storey, I. Akinola, M. Macklin, P. Reist, L. Wawrzyniak, dexterous in-hand rotation via joint-wise neural dynamics model,”
Y. Guo, A. Moravanszky, G. State, M. Lu, A. Handa, and D. Fox, arXiv:2510.08556, 2025.
“Factory: Fast contact for robotic assembly,” in RSS, 2022. [37] Z.-H. Yin, C. Wang, L. Pineda, F. Hogan, K. Bodduluri, A. Sharma,
[9] E. Heiden, M. Macklin, Y. Narang, D. Fox, A. Garg, and F. Ramos, P. Lancaster, I. Prasad, M. Kalakrishnan, J. Malik, M. Lambeta, T. Wu,
“Disect: A differentiable simulation engine for autonomous robotic P. Abbeel, and M. Mukadam, “Dexteritygen: Foundation controller for
cutting,” in RSS, 2021. unprecedented dexterity,” arXiv:2502.04307, 2025.
[38] A. Kumar, T. Power, F. Yang, S. A. Marinovic, S. Iba, R. S. Zarrin,
and D. Berenson, “Diffusion-informed probabilistic contact search for
multi-finger manipulation,” in ICRA, 2025.
[39] M. Noseworthy, B. Tang, B. Wen, A. Handa, C. Kessens, N. Roy,
D. Fox, F. Ramos, Y. Narang, and I. Akinola, “Forge: Force-guided
exploration for robust contact-rich manipulation under uncertainty,”
RA-L, 2025.
[40] Y. Jiang, C. Wang, R. Zhang, J. Wu, and L. Fei-Fei, “Transic: Sim-
to-real policy transfer by learning from online correction,” in CoRL,
2024.
[41] A. Maddukuri, Z. Jiang, L. Y. Chen, S. Nasiriany, Y. Xie, Y. Fang,
W. Huang, Z. Wang, Z. Xu, N. Chernyadev, S. Reed, K. Goldberg,
A. Mandlekar, L. Fan, and Y. Zhu, “Sim-and-real co-training: A simple
recipe for vision-based robotic manipulation,” RSS, 2025.
[42] H. Zhang, Z. Li, X. Zeng, L. Smith, K. Stachowicz, D. Shah, L. Yue,
Z. Song, W. Xia, S. Levine, et al., “Traversability-aware legged
navigation by learning from real-world visual data,” T-RO, 2025.
[43] C. Higuera, A. Sharma, T. Fan, C. K. Bodduluri, B. Boots, M. Kaess,
M. Lambeta, T. Wu, Z. Liu, F. R. Hogan, and M. Mukadam, “Tactile
beyond pixels: Multisensory touch representations for robot manipu-
lation,” in CoRL, 2025.
[44] T. Lin, Z.-H. Yin, H. Qi, P. Abbeel, and J. Malik, “Twisting lids off
with two hands,” in CoRL, 2024.
[45] D. Chen, B. Zhou, V. Koltun, and P. Krähenbühl, “Learning by
cheating,” in CoRL, 2020.
[46] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov,
“Proximal policy optimization algorithms,” arXiv:1707.06347, 2017.
[47] S. Ross, G. Gordon, and D. Bagnell, “A reduction of imitation learning
and structured prediction to no-regret online learning,” in AISTATS,
2011.
[48] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimiza-
tion,” in ICLR, 2015.
[49] A. Newell, K. Yang, and J. Deng, “Stacked hourglass networks for
human pose estimation,” in ECCV, 2016.
[50] T. L. Team, J. Barreiros, A. Beaulieu, A. Bhat, R. Cory, E. Cousineau,
H. Dai, C.-H. Fang, K. Hashimoto, M. Z. Irshad, M. Itkina,
N. Kuppuswamy, K.-H. Lee, K. Liu, D. McConachie, I. McMahon,
H. Nishimura, C. Phillips-Grafflin, C. Richter, P. Shah, K. Srinivasan,
B. Wulfe, C. Xu, M. Zhang, A. Alspach, M. Angeles, K. Arora,
V. C. Guizilini, A. Castro, D. Chen, T.-S. Chu, S. Creasey, S. Curtis,
R. Denitto, E. Dixon, E. Dusel, M. Ferreira, A. Goncalves, G. Gould,
D. Guoy, S. Gupta, X. Han, K. Hatch, B. Hathaway, A. Henry,
H. Hochsztein, P. Horgan, S. Iwase, D. Jackson, S. Karamcheti,
S. Keh, J. Masterjohn, J. Mercat, P. Miller, P. Mitiguy, T. Nguyen,
J. Nimmer, Y. Noguchi, R. Ong, A. Onol, O. Pfannenstiehl, R. Poyner,
L. P. M. Rocha, G. Richardson, C. Rodriguez, D. Seale, M. Sherman,
M. Smith-Jones, D. Tago, P. Tokmakov, M. Tran, B. V. Hoorick,
I. Vasiljevic, S. Zakharov, M. Zolotas, R. Ambrus, K. Fetzer-Borelli,
B. Burchfiel, H. Kress-Gazit, S. Feng, S. Ford, and R. Tedrake, “A
careful examination of large behavior models for multitask dexterous
manipulation,” arXiv:2507.05331, 2025.
[51] A. Kumar, J. Hong, A. Singh, and S. Levine, “When should we prefer
offline reinforcement learning over behavioral cloning?” in ICLR,
2022.
[52] L. Pinto, M. Andrychowicz, P. Welinder, W. Zaremba, and
P. Abbeel, “Asymmetric actor critic for image-based robot learning,”
arXiv:1710.06542, 2017.
A PPENDIX TABLE III: Privileged information for nut-bolt task. The
screwdriver task uses a subset of these features (excludes
A. Privileged Information for Oracle Policy hand pose and PD gains).
Nut-Bolt Task. The oracle policy receives privileged infor-
Privileged Information Dimension
mation about object properties, fingertip states, nut-specific
dynamics, hand states, and PD controller gains. The full list Object position 3
of privileged inputs is provided in Table III. Object scale 1
Object mass 1
Screwdriver Task. The screwdriver task uses a subset of the Object friction coefficient 1
privileged information from the nut-bolt task. Specifically, Object center of mass 3
it excludes hand base position, hand orientation, hand joint Object orientation (quaternion) 4
positions, and PD controller gains. All other privileged inputs Object linear velocity 3
remain the same. Object angular velocity 3
Object restitution 1
B. Reward Fingertip positions (2 fingers) 6
Fingertip orientations (2 fingers) 8
Our reward function is a weighted combination of task Fingertip linear velocities (2 fingers) 6
rewards, energy penalties, and stability penalties: Fingertip angular velocities (2 fingers) 6
• Task Rewards encourage successful rotation: Nut contact indicator 1
Nut position 3
◦ Rotation Rewards: rtrot = clip(ωt , ωmin , ωmax ). It
Nut joint velocity 1
encourages positive angular velocity ω along the fas- Nut joint position 1
tening axis, clipped to [−4.0, 4.0] rad/s. Screw joint friction 1
◦ Proximity Rewards: rtprox = max(0, 1−dt /dthresh ). It Hand scale 1
encourages fingers to remain close to the object, where Hand position 3
dt is the mean distance from thumb and index finger Hand orientation (quaternion) 4
Hand joint positions 12
to the nut/handle.
PD controller gains (kp ) 12
• Energy Penalties discourage inefficient motions: PD controller gains (kd ) 12
◦ Torque Penalty: rttorque = −∥τt ∥2 penalizes large joint Total 97
torques.
◦ Work Penalty: rtwork = −(|τt |⊤ |q̇t |)2 penalizes exces-
TABLE IV: Reward function hyper-parameters for nut-bolt
sive joint power.
and screwdriver tasks.
• Stability penalties maintain stable behavior:
◦ Pose Difference Penalty: rtpose = −∥qt − q 0 ∥2 pe- Reward Component nut-bolt Screwdriver
nalizes deviations from the initial finger configuration Task Rewards
(thumb joints masked). λrotate 6.0 2.5
◦ Large Rotation Penalty: rtrp = − max(0, ωt − ωthresh ) λproximity 2.0 2.0
penalizes excessive angular velocity above threshold Energy Penalties
ωthresh (10.0 rad/s for nut-bolt, curriculum from 7.5 to λtorque -0.1 -3.0
15.0 rad/s for screwdriver). λwork -0.01 -0.01
We sum the above rewards with weights listed in Table IV. Stability Penalties
λpose -0.5 -0.1
C. Training Hyperparameters λrotate-penalty -0.3 -0.3
The inputs to our oracle policy contains the proprioceptive λpc-z -1.0 -1.0
observations consist of qt (joint positions over the last 3
timesteps) and at−1 (previous joint targets over the last 3
timesteps). The privileged information includes pt (5 finger- train 1.5 billion environment steps in total, which takes less
tip positions), wt (object state with 3D position, quaternion than one day on a single GPU. We train our sensorimotor
orientation, and angular velocity), and additional features policy with on-policy behavioral cloning, and the training
detailed in Table III. We follow the domain randomization hyperparameters are shown in Table VII.
parameters in Table V.
We train our oracle policy with PPO, and the training
hyperparameters are shown in Table VI. Specifically, we train
with 8,192 parallel environments. Each environment gathers
12 steps of data to train in each epoch of PPO. The data is
split into minibatches of size 16,384 and optimized with PPO
loss. γ and λ are used for computing generalized advantage
estimate (GAE) returns. We use the Adam optimizer to train
PPO and adopt gradient clipping to stabilize training. We
TABLE V: Domain Randomization Parameters. Object
scale is discretely sampled from the specified set and mul-
tiplied by the base scale. Mass, center of mass, friction,
restitution, and PD controller gains are uniformly sampled at
environment initialization. Observation and action noise are
sampled i.i.d. from Gaussian distributions at each timestep.
Following [1], we apply a random disturbance force with
magnitude 2.0m (where m is object mass) with probability
0.25 at each timestep.
Parameter Range
Object Scale (nut-bolt) ×[0.95, 1.05]
Object Scale (Screwdriver) ×[0.85, 1.25]
Mass [0.04, 0.06] kg
Center of Mass [-0.001, 0.001] m
Coefficient of Friction [0.5, 8.0]
Object Restitution [0.0, 1.0]
PD Controller Stiffness (kp ) [2.7, 3.3]
PD Controller Damping (kd ) [0.009, 0.011]
Observation Noise (rotation) N (0, 0.01) (rad)
Observation Noise (translation) N (0, 0.005) (m)
Action Noise (rotation) N (0, 0.01) (rad)
Action Noise (translation) N (0, 0.005) (m)
External Force Scale 2.0m
External Force Probability 0.25 per timestep
TABLE VI: Hyperparameters for training the oracle policy.
Hyperparameter Value
# environments 8192
# steps 12
# minibatch size 16384
# Environment steps 3×109
discount factor (γ) 0.99
GAE (λ) 0.95
learning rate 5e-3
clip range 0.2
entropy coefficient 0.0
kl threshold 0.02
max gradient norm 1.0
TABLE VII: Hyperparameters for training the sensorimotor
policy in simulation.
Hyperparameter Value
# environments 48
# steps 512
# minibatches 4096
learning rate 1e-3