Watching a hand walk away on its fingers brings to mind Thing from The Addams Family. Heidi Klum brought a gruesome version to life this Halloween with her giant severed-hand costume.
In a Swiss robotics lab, a mechanical hand is taking its own steps toward something more useful.
Researchers at ETH Zurich taught it to walk on its fingertips, get upright after a fall, and press keyboard keys while balancing on its other fingers.
The idea is to give a robot’s hand a way to work somewhere its body can’t go.
“A future robot could send its hand into a space its body cannot enter, to inspect something, operate a control or move an object, then retrieve it afterward,” roboticist Robert Katzschmann told ScienceAlert.
The team’s study, posted as a preprint on arXiv and not yet peer-reviewed, explores how to build those abilities into a commercially available hand without redesigning its fingers.
The fingers themselves presented the first difficulty. Put your hand on a table with only your fingertips touching, then lift one finger. The others keep it steady. To walk, a robotic hand has to keep shifting that support while carrying its entire weight.
“The fingers have different lengths and a thumb off to one side, which makes walking awkward,” first author and doctoral researcher Amirhossein Kazemipour told ScienceAlert.
“They also have to keep the hand balanced while interacting with objects.”
Robotic hands can already perform delicate tasks, but an arm usually carries them into position. Here, the same fingers needed to provide both movement and support.
Kazemipour, Hehui Zheng, and Katzschmann started with a WUJI robotic hand, adding a battery, a small computer, and a motion sensor. The complete setup weighs 818 grams, or about 1.8 pounds, and carries its power and computing hardware on its back.
Before the physical tests, the researchers trained its control software in simulation using reinforcement learning. Useful movements, such as progressing in the requested direction, earned numerical rewards.
Unwanted movements, such as excessive rocking, incurred penalties that lowered the score. Through repeated attempts, the system learned which actions scored highest.
frameborder=”0″ allow=”accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share” referrerpolicy=”strict-origin-when-cross-origin” allowfullscreen>
To account for the unequal fingers, each fingertip received a reference position based on the hand’s resting stance. Moving farther from that position reduced the score, with more freedom along the direction of travel. This guided the hand toward a workable gait without prescribing every step.
Transferring those movements to the real hand required measurements of how its joints responded to commands and how its fingertips slid against a surface.
“The key was tailoring the training to the hand’s shape and carefully matching the simulation to the real hardware,” Kazemipour said.
The resulting hand crawled over 14 surfaces in demonstrations, including wooden floors, metal grating, grass, and gravel. An operator gave movement commands through a gamepad, while the onboard software coordinated the fingers and balance without camera guidance.
During steering tests on a mat, it averaged 9.3 centimeters (3.7 inches) per second across 21 runs. The team also trained it to right itself, and it succeeded in 21 of 25 physical trials. In the other four, its fingers caught on one another.
With movement and balance established, the researchers tested whether a finger could leave the ground to do a job. Using a separately trained controller, the hand pressed the correct keyboard arrow key in 29 of 32 consecutive commands.
That ability let it enter moves in Sokoban, a puzzle game involving pushing boxes into position. A human chose the moves; the hand operated the keyboard without visual feedback.
Pushing a real object needed a different setup. An overhead camera tracked the hand and a cube, supplying information that let the hand approach the cube and push it toward a target while staying balanced.
Each skill has its own trained controller. The next challenge is to combine them into longer tasks, improve reliability, and give the hand onboard vision. The outdoor demonstrations don’t yet establish how consistently it would handle unfamiliar terrain.
Automatic detachment from a larger robot and reconnection afterward also remain future goals. Katzschmann said muscle-like mechanisms could eventually help hands adjust their stiffness for gentler contact.
For now, the hand has learned some of the movements it would need to venture beyond its arm. The longer-term goal is to send it into a confined space with a job to do.
The research has been posted as a preprint on arXiv.
This article was fact-checked by Fiona MacDonald and edited by Fiona MacDonald. While we pride ourselves on our process, we are only human. If you spot a mistake, please let us know.
