Thin slices and dices of salmon are no problem for the Sashimi-Bot

Sashimi-Bot straightens a loin of salmon on the cutting board, then changes tools to grasp, slice and stabilise the raw fish. Finally, it arranges the sashimi slices on a platter, using three arms that ‘talk to each other’ — this is called physical intelligence in robotics.

Getting a robot to make sashimi may sound like a mere curiosity. In reality, it involves one of the most difficult challenges in modern robotics. Namely, teaching autonomous robot systems to understand and handle delicate objects without destroying them. A soft, slippery, sticky and deformable salmon loin is wonderful to eat, but it is a nightmarish object for robots to work with.

“There are hardly any objects in nature that are more challenging to work with,” said Ekrem Misimi, chief scientist for SINTEF Ocean, who headed the research team behind this work.

The study recently published in Nature Magazine and Nature Robotics explains how the Sashimi-Bot combines 3D shape manipulation, tool use, precise cutting and manipulating thin, fragile slices.

Sashimi-Bot is a new autonomous tri-manual robot system developed by researchers at SINTEF Ocean, NTNU, QUT, Inria, MIT and NMBU, within the projects GentleMAN (IKTPLUSS) and BIFROST (FRIPRO), funded by the Research Council of Norway. The system uses three robotic arms that work together to straighten a salmon loin, change tools, grasp and guide a knife, stabilise the fish during cutting and finally, pick up thin sashimi slices and place them on a platter.

Humans do not think of these as difficult and complicated tasks, but for robots they are extremely challenging. Salmon is soft, slippery and naturally varies in shape and size. The raw fish can deform, slip, be limp and change behaviour during contact with the robot.

“This isn’t just a demonstration of advanced manipulation and sashimi slicing, but of autonomous physical intelligence. Quite simply, this refers to robots that need to see, feel, adapt and act in the face of uncertainty,” Misimi said.

Misimi convened some of the world’s leading tactile sensing research groups. They have worked on developing visual control of robots, robot manipulation and autonomous systems to address the most fundamental challenges in today’s robotics.

The goal was not to create a robot that could slice sashimi, but to demonstrate how robots can develop greater physical intelligence, enabling them to interact with the real world as it actually is: unpredictable and constantly changing.

“We’ve managed to create a robot that combines several of the most advanced robotics solutions in one robot,” Misimi said.

How researchers taught the robot to feel

The robot first learns how to handle and shape a soft salmon loin in a simulation — much like practising in a virtual training room. It then takes this knowledge straight into reality, without having to retrain. As the robot works, it uses its vision to adjust its movements along the way, so that the fish has the right shape before it is cut. At the same time, a robot hand controls the knife like a tool, almost like a human hand, and makes it possible to cut precisely controlled slices.

The robot not only ‘sees’ what it is doing, it also feels it. A sensor on the knife registers how it feels when it cuts, a bit like when we feel the resistance in a knife. This allows the robot to detect when it hits the cutting board and to automatically adjust its movement.

Finally, the robot uses its vision to pick up the thin, fragile sashimi slices with chopsticks, even when they are stuck on the knife and difficult to grasp.

Fine motor skills at their best when the robot arm removes the piece of salmon from the knife. Image credit: SINTEF

These individual robotic skills are not the only results of the project. They also include an entire work process where learning, vision, touch and precise control are coordinated from start to finished product.

“This has been incredibly fun to work on, perhaps the most fun project I’ve been involved with so far,” said Sverre Herland, a research scientist at SINTEF Ocean.

He explained that the project also shows when the robots manage to combine vision, touch, learning and control in a good way — how a lot of the properties needed for advanced manipulation can be achieved using relatively simple tools.

“The work not only shows what is possible within robotics, but also highlights the challenges. Only when the robot actually has to grasp, cut and lift a soft and smooth object can we observe where the shoe still pinches,” Herland said.

Future applications

Salmon is not the easiest test object and was chosen precisely because it is so demanding to handle. Salmon consists of natural biological materials and varies in its geometry. In addition, seafood has poorly defined material properties and is difficult to hold without damaging it. This is precisely why seafood also makes an excellent test area for future robotics.

“If we can develop robots that handle objects like this, it opens up possibilities for far more diverse applications. Robots can contribute to automating tasks that today still have to be done manually because the objects are too variable, deformable or fragile for traditional robotic technology,” Misimi said.

The technology has significance far beyond handling salmon because it addresses a fundamental problem within robotics. How can robots understand and physically interact with objects that are challenging to model?

“This is relevant across different robot types and operating environments on land, underwater and in the air, wherever the robot has to combine the skills of perception, contact understanding and action in real time,” Misimi said.

In the seafood industry, these methods could contribute to more flexible processing, better raw material utilisation, reduced food waste and less dependence on heavy manual labour. The food industry could apply similar methods for handling meat, fruit, vegetables and other biological raw materials. In the fields of agriculture, health, textiles and recycling, there are similar challenges where robots need to understand and handle objects that do not behave the same way every time.

Work on this project has been developed collaboratively by several leading research environments that have been partners in the GentleMAN and BIFROST projects. Among the co-authors are internationally renowned research scientists in the fields of tactile perception, visual servoing and robot manipulation, including Edward H. Adelson from MIT, François Chaumette and Alexandre Krupa from Inria, and Peter Corke from QUT.

Image credits: SINTEF

Originally published here.