How ChatGPT got a passing score in the law school exam, but could not be a good student



The interactive AI ` ` ChatGPT '' announced by OpenAI has a passing score in the university-level open-ended exam and the final exam of the Master of Business Administration (MBA), and researchers need to see the gist of the paper written by ChatGPT. It has been pointed out that the high ability to generate sentences has become a hot topic. Newly reported content in January 2023 shows that ChatGPT repeatedly challenged multiple-choice and essay tests for law school students, and that ChatGPT achieved passing scores on all tests. increase.

ChatGPT Goes to Law School by Jonathan H. Choi, Kristin E. Hickman, Amy Monahan, Daniel Schwarcz :: SSRN
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4335905




ChatGPT bot passes US law school exam
https://techxplore.com/news/2023-01-chatgpt-bot-law-school-exam.html

The conversational AI ``ChatGPT'', which can create natural sentences, has very high ability, and there were reports that it could pass the doctor's license examination and the examination for the master's degree in business administration, so various opinions were received about ChatGPT. I'm here. Especially in the field of education, about 17% of Stanford University students answered that they 'use ChatGPT for assignments or exams,' and discussions are becoming more active.

17% of students answered that they use ChatGPT for assignments or exams - GIGAZINE



According to the white paper 'ChatGPT goes to law school' published on January 25, 2023 in the Social Science Research Network (SSRN) , Professor Jonathan Choi of the University of Minnesota Law School asked 95 multiple-choice questions and 12 questions. We are conducting a survey to have ChatGPT answer four exams for law school students consisting of short essay questions. The exam covered a wide range of legal themes, from constitutional law to taxation and tort, but as a result of scoring, ChatGPT was reported to have earned a comprehensive passing score of 'C+' on all exams.

According to the report, ChatGPT's answer was sufficient as a passing score, but in most subjects it was close to the bottom of the class, and in multiple-choice questions including math problems, it was 'exploding'. In the study that earned a passing score on the United States Medical Licensing Exam (USMLE), which is an American medical license examination, it recorded 60% of the passing score, and on the final exam of the MBA program, it received a grade of B to B minus. So, although it is at the passing level, it does not get a high score, and Professor Choi also says, 'It is enough to pass, but it is not an excellent student.'

In response to ChatGPT, Professor Choi said, ``When writing essays, ChatGPT grasped the basic legal rules well, and the structure and composition were consistently solid. I was not good at finding problems in sentences, which is a skill, and I was not able to proceed with deep reasoning and analysis.




There are various opinions about ChatGPT, and there are movements to ban its use in educational settings and writing scientific papers with ChatGPT is prohibited at international conferences, but Professor Choi said, ``ChatGPT is a valuable teaching material. It could be,” he suggests. ChatGPT alone cannot create a ``good law student answer'', but by reading the answers you created yourself into ChatGPT, you can brush up on a ``better answer'', Or you can expect it to be very useful when opening a lawyer, ”said Choi.




Professor Choi also reports that two out of three graders found out that they were ``bots written essays''. According to Mr. Choi, ``ChatGPT's grammar was perfect and it was somewhat repetitive, so I felt uncomfortable, so the scorers who saw it seemed to intuitively think it was a bot sentence.''

Related Posts:

in Software, Posted by log1e_dh