Achieving Unprecedented Dexterity

We introduce DexerityGen (DexGen), a foundational controller that empowers unprecedented dexterous manipulation skills, ranging from reorientation and assembly to the use of pen, screwdriver, and syringe.

Idea: Building a Skillful Prior

DexGen learns diverse dexterous manipulation behavior in simulation. It refines a coarse motion by a high-level policy such as teleoperation to fine dexterous actions during deployment.

Scaling up Low-Level Dexterous Manipulation Dataset

We pretrain DexGen on a huge multi-task dexterous manipulation dataset. The core of our dataset is a large Anygrasp-to-Anygrasp dataset generated by reinforcement learning agents.

Application: Robust Teleoperation with DexGen.

Using teleoperation to perform dexterous manipulation is extremely hard.
DexGen makes it possible, in diverse challenging setups, for the first time.

Videos below demonstrate a human practicing teleop over a teleoperation system enhanced by DexGen. DexGen can reject dangerous actions in diverse scenarios.

More Basic Skill Evaluations

DexGen assists a clumsy user to perform sophisticated dexterous in-hand object manipulation skills. Small objects, rough edges, and gravity...
DexGen handles them all.
Videos are in 2x.

Reorienting diverse objects.

Pinch grasp to Power grasp.

Concluding Remark: Looking Forward

Dexterity Gen unlocks endless possibilities. It enables dexterous tool use data collection in the real world. It stabilizes dexterous grasping and manipulation and allows more flexible high-level policy designs. We believe it will be a foundational building block in future robotic agents.