Dec 05, 2022 11:20:00

Dialogue AI ``ChatGPT'' passes the free description problem of the college student level exam

When I entered free-form computer science questions for fun into

ChatGPT , a language model for dialogue that has become a hot topic for being able to answer questions from humans very naturally, I got answers that passed with high scores. was reported to have been

ChatGPT passes the 2022 APCSA free response section GitHub
https://gist.github.com/Gaelan/cf5ae4a1e9d8d64cb0b732cf3a38e04a

This time, ChatGPT solved the free description problem included in the 2022 edition of ' (PDF file) AP Computer Science A ' created with the aim of providing college-level education to high school students. In this exam, which mainly tests Java knowledge, questions were asked on the theme of a program that tallies video game scores.

Gaelan Steele, a student studying computer science at the University of St. It was 32 points in the middle, 'he reported.

The method is as follows. Mr. Steele first copied the full text of the four question sentences, pasted it once into the ChatGPT input field, and adopted the first answer obtained. However, figures and tables included in the question text were excluded because they could not be pasted as plain text. We also excluded instructions, such as asking them to write their answers in a separate booklet.

If the question is divided into part (a) and part (b), first paste the question text in (a) and output the answer, then paste the question text in (b) and have the answer output. I was. In one question, ChatGPT output up to the answer of (b) just by inputting the question text of (a), so there were cases where it was not necessary to ask the rest of the questions.

Mr. Steele edited the ChatGPT answers by inserting

indentation and question numbers as necessary to make them easier to read. We didn't do 'picking'.

Mr. Steel scored the resulting ChatGPT answer according to the (PDF file) scoring guidelines , and as mentioned above, the score was 32 out of 36 points, which was a passing score.

In response to Mr. Steele's post, there are comments such as `` Well done '' and `` I wonder if there will be a difference in the score if the information in the table is converted to text and solved '' .

Also, in a thread on the social news site Hacker News that covered this report, ``If you raise a truck problem on ChatGPT, ``In a situation where a truck is heading for two tracks with one and two people tied up, , The correct answer is to operate the lever to direct one person.As a result, one person may die, but you can save the lives of two people.The act of pulling the lever is one person's life. It can be considered the least of the two evils because it prevents the death of two humans instead of ``returned'', such as the post ``Returned'' and a discussion of reactions to ethical issues raised against AI. was also expanded.

On the other hand, there are also posts pointing out that ``I can answer certain questions brilliantly, but other questions are incoherent'', and in the above-mentioned trolley problem, ``2 to help one old man'' It has been pointed out that ChatGPT still has room for development, such as some posts claiming that it was instructed to sacrifice human children.