Research results show that 'indoor training,' which was thought to make AI incompetent, actually made AI smarter
Because a test environment without extraneous noise is different from the cluttered real world, many engineers train their AI in settings that are close to the production environment in which it operates. However, researchers at the Massachusetts Institute of Technology (MIT) have discovered an 'indoor training effect' in which AI agents trained in a noise-free simulated environment often perform better than AI agents trained in noisy settings.
[2401.15856] The Indoor-Training Effect: unexpected gains from distribution shifts in the transition function
New training approach could help AI agents perform better in uncertain conditions | MIT News | Massachusetts Institute of Technology
https://news.mit.edu/2025/new-training-approach-could-help-ai-perform-better-0129
A team of researchers from MIT, Harvard and Yale universities first trained an AI agent to play Atari games such as Pac-Man, Pong and Breakout.
The AI agent played two games: a 'clean' version without any extraneous elements and a 'noisy' version. For example, in the case of Pac-Man, the enemy ghost (monster) always moves in the same direction in the clean environment, but in the noisy version, it moves up, down, left and right.
The researchers developed a method to add a certain amount of noise to the 'transition function,' one of the elements of a reinforcement learning problem, and when they added noise to the game environment of an AI agent, the AI's performance decreased as expected. However, when an AI trained on a clean version played the noisy version, it was better at the game than an AI trained on the noisy version from the beginning.
This goes against conventional wisdom that AI accuracy is improved when it is trained in an environment that is closer to production. 'A rule of thumb is that when training, you want to maximize the effectiveness of your training by replicating the production deployment environment as closely as possible,' said Spandan Madan, a graduate student at Harvard University and co-author of the study. 'We didn't believe any results to the contrary, so we tested them thoroughly.'
Through repeated testing, the research team found that there are several rules regarding the relationship between the performance of AI agents and the training environment. First, when an AI trained in a clean environment and an AI trained in a noisy environment explore the same area, the former performed better. This is thought to be because it is easier for the AI to understand the rules of the game when there is no noise.
'You might be able to learn a lot of different shots by practicing indoors, when there's no wind,' said Serena Bono, a research assistant at MIT and lead author of the paper. 'If you then practice on a windy tennis court, you might be more likely to improve than someone who starts out learning tennis in a windy environment.'
On the other hand, when the two AIs explored different areas, the agent trained in the noisy environment tended to perform better, presumably because the AI agent trained in the noisy environment had to learn patterns that it could not learn in a clean environment.
'If you've been practicing just your forehand in still water and you're now being told you have to use your backhand in the wind, it's likely you won't play very well,' Bono said.
The researchers hope that their findings may lead to the development of better ways to train AI agents, and they plan to investigate how the 'indoor training effect' manifests itself in more complex reinforcement learning environments and in non-game technologies such as computer vision and natural language processing.
Related Posts:
