Making AI “Curiosity” Can Improve Performance

In a comparative study of AI and mice, it was found that mice noticed 'new objects' quickly, while AI did not notice them at all. Researchers are using this challenge to improve performance by making AI 'curiosity'.

[2306.15934] Curious Replay for Model-based Adaptation

AI Agents that “Self-Reflect” Perform Better in Changing Environments

Animals adapt effectively -- why can't AI agents?

Introducing Curious Replay: improve AI adaptivity by focusing learning on interesting experiences. #ICML2023
Animal inspiration got us there. How? ????→???? A thread: pic.twitter.com/2BZ6nHjeka

—Isaac Kauvar (@ikauvar) July 6, 2023

Isaac Kauber of the Wu Tsai Neuroscience Institute at Stanford University and Chris Doyle of the Machine Learning Research Institute explored and adapted animals to their surroundings in the lab of Associate Professor Nick Haber of the School of Education. Based on my long research experience, I conducted an experiment comparing state-of-the-art AI and mice.

It involved placing a mouse in a small empty box and an AI agent in an empty 3D virtual arena, placing red balls in both environments and measuring which one explored new objects faster.

As a result of the experiment, the mouse immediately approached the ball and played with it for several minutes, but the AI did not notice the ball.

'It was unexpected. Even with state-of-the-art AI, I realized that there was a gap in performance,' said Cowber.

In response to the experiment, Mr. Cowver and Mr. Doyle considered whether the simple behavior of the mouse could be used as a source of inspiration to improve AI. Together with Associate Professor Haber and graduate student Lin-Chi Zhou, we designed a training method called 'curiosity replay' that allows AI to reflect on the most novel and interesting things it has encountered recently.

The AI, who became 'intrigued' by this training, quickly approached the red ball in the experiment and became involved. Also, the Minecraft-based game 'Crafter' has dramatically improved performance.

Detailed results will be announced at the 40th International Conference on Machine Learning (ICML2023) on July 25, 2023.