MIT researchers are training a robot arm to play Jenga

Turns out training a robotic arm to play Jenga is a surprisingly complex task. There are, so to speak, a lot of moving parts. Researchers at MIT are putting a modified ABB IRB 120 to work with the familiar tabletop game, utilizing a soft gripper, force-sensing wrist joint and external camera to design a bot that can remove a block without toppling the tower.

The robot was trained with 300 attempts, rather than the thousands it would traditionally take, learning to cluster different attempts into groups as a kind of shorthand similar to how humans teach themselves. With each attempt the robot is pushing against the block, testing for tactile feedback to determine whether it’s a safe bet.

“Unlike in more purely cognitive tasks or games such as chess or Go, playing the game of Jenga also requires mastery of physical skills such as probing, pushing, pulling, placing and aligning pieces. It requires interactive perception and manipulation, where you have to go and touch the tower to learn how and when to move blocks,” says MIT assistant professor Alberto Rodriguez. “This is very difficult to simulate, so the robot has to learn in the real world, by interacting with the real Jenga tower. The key challenge is to learn from a relatively small number of experiments by exploiting common sense about objects and physics.”

The robot has gotten pretty good at making attempts, but the team is quick to note that it’s not quite ready to take on experienced players. Among other things, the robot isn’t able to determine strategic blocks that can sabotage the tower’s strength for upcoming turns.