Experiments have revealed that reading AI-generated summaries can lead to incorrect memories of events that the person actually witnessed.

A research team from Georgetown University and the University of Washington has reported that if a summary generated by AI contains inaccurate information, readers may recall events based on that misinformation, even if they had seen the original video themselves.

[2609.28820] AI-Enabled Human Memory Manipulation: Misleading AI-Generated Summaries Distort Human Memory

https://arxiv.org/abs/2609.28820

New Research Finds That Misleading AI-Generated Summaries Can Distort Human Memory | College of Arts & Sciences | Georgetown University
https://college.georgetown.edu/news-story/new-research-finds-that-misleading-ai-generated-summaries-can-distort-human-memory/

Generative AIs like ChatGPT can shorten long texts and meeting records, making them widely used for everything from news summaries to business meeting minutes. AI is also increasingly being adopted for applications where accuracy is particularly important, such as creating reports from police officer body camera footage. To address the possibility of AI errors, the 'human-in-the-loop' approach, where humans review the output and correct any mistakes, is commonly used.

On the other hand, human memory is also unreliable. In psychology, the 'misinformation effect,' in which incorrect information is given after experiencing an event, can alter the memory of the original event, has been studied for many years. The research team considered that even in a system where 'humans check the output of AI,' the memories of the humans who read the AI's errors may be affected.

The research team first had OpenAI's ChatGPT-5.5 and Google's Gemini 2.5 Flash-Lite summarize a 25-second video of a traffic accident. The video was simple: 'A red car turns right at an intersection and collides with a pedestrian who steps onto the road.' A stop sign appears on one side of the car, and a 'yield' sign appears on the other. Each AI was made to summarize the video five times, and a total of 20 summaries were examined.

The AI was explicitly instructed to 'create a fact-based, neutral text,' 'avoid speculation,' and 'include all important events,' yet all 20 summaries contained errors. Each summary, averaging 305 words, had at least 7 and up to 21 errors. Furthermore, an average of 51.6% of the 19 items the research team deemed important were omitted from the summaries. Even the central event in the video, 'the car hit a pedestrian,' was omitted in 19 out of 20 summaries—95% of the total.

Because significant errors were found in the AI-generated summaries themselves, the research team then experimented to see how people's memories changed after reading the inaccurate summaries. 328 adults living in the United States were shown videos of traffic accidents and then given 21 sentence summaries of the same events to read 24 to 48 hours later.

Participants were told that the summaries were either 'created by AI' or 'created by a professional human,' but in reality, all the text was generated using ChatGPT. To standardize the experimental conditions, the research team adjusted parts of the text, creating both an accurate summary with signs matching the video and an incorrect summary where the stop sign and the 'yield' sign were swapped. Participants were then asked to answer questions based on their memory of 'the video they saw first,' rather than the summary.

Among participants who read an accurate summary, 83.6% correctly identified the signs that appeared in the original video, while only 44.8% of those who read an incorrect summary answered correctly. On the other hand, 96.6% correctly answered the question 'Was the accident during the day or at night?' which was not included in the summary, and an average of 91.4% correctly answered 11 peripheral memory questions. This indicates that participants did not lose their memory of the entire video, but rather their answers changed significantly when they were given incorrect information about the signs in the summary.

Furthermore, no statistically significant difference was found in the accuracy of identifying signs between participants who were told that the summaries were written by AI and those who were told that they were written by a human. No significant relationship was found between the accuracy of identifying signs and how much participants trusted AI or how often they used AI. The research team points out that simply knowing that 'AI can make mistakes' may not be enough to prevent the impact of misinformation on memory.

Previous research has shown that subjects who are asked misleading questions by AI chatbots are more likely to form false memories. While previous studies mainly focused on interaction and guidance with AI, this study suggests that memory may also be affected by 'reading an AI-generated summary after an event.'

AI-based interrogation could risk implanting false memories in humans - GIGAZINE

The research team acknowledges that this study alone has limitations in generalizing the findings, and plans to investigate whether AI-generated summaries alter memories and perceptions of interactions between police officers and citizens, using actual body camera footage of police officers.