What are the tips to teach robots stir-fry cooking techniques?

Robots that can play various roles on behalf of humans still have many challenges in the kitchen. Researchers at the Chinese University of Hong Kong, Wuhan University, and the Idiap Institute in Switzerland have taken an approach to make robots 'stir-fry' Chinese food, and have succeeded in actually reproducing the cooking movement. ..

Robot Cooking With Stir-Fry: Bimanual Non-Prehensile Manipulation of Semi-Fluid Objects | IEEE Journals & Magazine | IEEE Xplore

A technique to teach bimanual robots stir-fry cooking
https://techxplore.com/news/2022-06-technique-bimanual-robots-stir-fry-cooking.html

The movement to make stir-fry requires a dynamic cooperative movement of the right arm and the left arm, which is difficult for cooks to learn, let alone robots.

To train the robot to learn this difficult movement, the research team first used a human demonstration to train a two-handed collaborative model called a 'Structured- Transformer .'

According to the lead author of the paper, Junjia Liu, this mechanism considers adjustment as a problem of sequence transformation between arm movements, and to achieve this, a combination model of deep learning model transformers and GNN (graph neural network) . It is said that it has adopted.

The point is to treat the right arm and the left arm separately. The movement of the robot's left arm is adjusted by visual feedback, and the corresponding movement of the right arm is generated by a pre-trained structured transformer model based on the movement of the left arm.

The research team evaluated the performance of the model using Panda's two-armed robot, known for simulation and robotic arms. When tested, the two-armed robot was able to successfully and realistically reproduce the movement of cooking stir-fry.

Liu said, 'The main contribution of this paper is to explicitly examine the adjustment mechanism of a two-armed robot in the form of sequence transformation.' 'Classical learning from demonstrations and deep learning / reinforcement learning bases. Compared to the method of, our separation framework is a clever combination of both methods. It can have both the generalization of the former and the expressiveness of the latter. '

In the future, this model may allow the development of a decent cooking robot, as well as the same approach that will allow the robot to be trained in other tasks that use two arms. increase. Liu and colleagues say they plan to continue development to expand the performance and general possibilities of the model.