Research suggests that AI assistance can reduce 'perseverance in tackling problems' and worsen academic performance. What are some ways to use AI to minimize these negative effects?
A study investigating the impact of AI revealed that when AI assists with studying, the accuracy and response rate on actual tests decrease.
AI Assistance Reduces Persistence
The research team conducted an experiment with 354 participants, having them perform fraction calculation problems. Participants were randomly divided into two groups: 'with AI assistance' and 'without AI assistance.' Participants in the 'with AI assistance' group were able to use AI alongside the test questions.
The experiment consisted of a total of 15 questions. Participants solved 12 questions that gradually increased in difficulty, followed by a 'test' of 3 questions. Participants could skip questions at any time, and the group receiving AI assistance was automatically deprived of access to the AI without prior notice once the 'test' began.
The results of the experiment are shown in the figure below, with the upper line graph showing the correct answer rate for each question and the lower line showing the skip rate for each question. The group that received AI assistance, shown by the orange line, performed well while receiving AI assistance, but their correct answer rate dropped sharply and the rate of skipping questions increased once they entered the 'test.' On the other hand, the group that did not receive AI assistance, shown by the green line, did not show much difference in correct answer rates or skip rates between the test and the other questions.
Based on the results of the first experiment, the research group conducted a second experiment to further improve accuracy. In the second experiment, 667 participants were given a total of 17 questions. They eliminated 'differences due to original ability' in the first three questions, and also eliminated differences in UI between the 'no AI assistance' group and the 'AI assistance' group by displaying a sidebar where the AI would normally appear.
The results of Experiment 2 are shown in the figure below. Similar to Experiment 1, the results showed that 'AI support improves performance during the learning phase but worsens test scores.'
The research team also investigated 'how participants used AI.' The figure below shows the differences based on AI usage trends: light blue represents the performance of the 'no AI assistance' group for comparison, yellow represents the performance of the group that 'directly asked the AI for answers,' brown represents the performance of the group that 'sought hints from the AI,' and black represents the performance of the group that did not use AI.
According to the research team, 61% of the 'AI-assisted' group directly asked the AI for answers. The group that directly asked the AI for answers experienced the biggest drop in test scores, while using the AI to ask for hints or explanations did not significantly worsen test scores.
To see if the same trend could be observed in areas other than mathematics, the research team conducted a third experiment in which reading comprehension questions were presented in the format of the SAT, a standardized test for prospective college students in the United States.
The third experiment involved 201 participants. The first question was a preliminary question designed to eliminate the influence of ability differences, questions 2 through 6 were practice questions, and questions 7 through 9 were the test. As in previous experiments, participants were randomly assigned to either a group that received AI support or a group that did not.
The results are shown in the figure below. Even in reading comprehension questions, the group that used AI showed a sharp decline in performance during the test, and a clear tendency for the skip rate to increase.
Based on the results of the three experiments, the research team summarized that 'when AI assistance is received, participants tend to give up on problems or their performance deteriorates when the AI assistance is removed,' and concluded that 'AI assistance reduces perseverance.'
Related Posts:
