What is the content of the alleged research paper in which the interactive AI ``ChatGPT'' was named as a co-author?
While the interactive AI 'ChatGPT', which can respond like a human being, can be used in various ways for the benefit of humans, voices that consider it a problem to take exams and write papers instead of humans There is also The academic journal Nature has already
Performance of ChatGPT on USMLE: Potential for AI-Assisted Medical Education Using Large Language Models | medRxiv
https://doi.org/10.1101/2022.12.19.22283643
One of the papers in which ChatGPT participated as a co-author was written by Tiffany Kung et al., who belong to Massachusetts General Hospital and Harvard Medical School Department of Anesthesiology. It was to make ChatGPT take the medical license examination.
The USMLE is a human exam that covers all of the physician's knowledge base, including basic science, clinical reasoning, medical management, and bioethics. Although it includes the problem of writing sentences based on complex clinical data, Mr. Kung and others thought that ChatGPT could answer because ChatGPT is capable of rich linguistic expressions.
Mr. Kun and his colleagues eliminated problems that may have already been learned by ChatGPT from the 376 test questions that were publicly available, and finally picked up 305 items. They were entered into ChatGPT, answered, and the answers were scored by two doctors.
As a result, ChatGPT output answers in line with the content of the problem with a matching rate of 94.6%, and said that the correct answer rate was over 50% for all items. It seems that most of them had a correct answer rate of over 60%, but since the USMLE passing criteria is about 60% correct answer rate every year, ChatGPT will be barely within the passing range.
In addition, after careful examination of whether the content of the answers satisfies the novelty, originality, and validity, it is said that 88.9% of the answers provided at least one significant insight. Based on this, Kung and others argue that ChatGPT has various advantages, such as the possibility of supporting people studying medicine, and the possibility of obtaining new findings and improvement measures from the answers. I'm here.
In addition, the accuracy of ChatGPT was the lowest score for 'Step 1', which is considered to be the most difficult in USMLE. This may be due to the ``under-learning of the model,'' which occurred because humans perceive medicine as difficult and opaque, and lack representation on specific subjects, said Kung and colleagues. It points out that AI's ability is vulnerable to human ability.
In addition, Step 1 is taken by students who have completed a two-year curriculum at the medical school, and it seems that about 300 to 400 hours will be spent on studying in general.
“Inspired by ChatGPT’s impressive performance, clinicians at Ansible Health, a virtual chronic lung disease clinic, have begun experimenting with ChatGPT as part of their workflows. , it is used to present records with many technical terms in an easy-to-understand manner to patients, and to conduct brainstorming when faced with ambiguous and difficult-to-diagnose cases. % decrease, which is an important early signal that natural language processing models like ChatGPT are maturing and will soon impact the entirety of clinical care to truly bring compassion to people. We believe it will enhance our ability to deliver scalable healthcare.'
As co-authors of this study, Mr. Kung and Mr. Morgan Cheatham of Brown University are officially listed next to ChatGPT.
Related Posts:
