LeRobot Cloth Folding

Two SO-101 arms fold a cloth in half on their own. Nothing is scripted: a SmolVLA policy I trained on demonstrations decides every movement.

Getting two arms to work on the same fold is its own problem. I recorded the demonstrations myself, steering both follower arms with leader arms while three cameras watched: one on each wrist for the close-up work, and one at the front for an overview of the cloth.

episodes

15

frames

13,133

camera views

3

fps

30

The dataset is public in LeRobot v3.0 format: 12-dimensional joint states and actions (six per arm, grippers included) with video from all three views. The policy in the video was trained on these 15 episodes.

pick and place

Before the fold, I taught a single arm to pick up a red cube and drop it into a box. I collected 80 demonstrations, ten at each of eight cube positions, with a wrist camera and a front camera giving two views of the block and the box.

SmolVLA is a 450M-parameter vision-language-action model. It takes the camera images, the task and the robot's state, and produces actions through a flow-matching action decoder, instead of following a hand-written sequence of coordinates. Asynchronous inference connects the camera observations to the arm's action stream, so the arm keeps moving while the next chunk of actions is computed.

For both tasks I collected the teleoperation data, trained the policy and ran it on the physical arms.