ChatGPT can have high-performance dialogue, but why do you sometimes give random answers?

OpenAI's interactive AI ' ChatGPT ' can not only answer questions from humans very naturally, but can also be used in various ways, such as passing a written test and completing a program that actually works in a few minutes. It is considered On the other hand, ChatGPT only gives ``like-for-like answers'' and the contents are often nonsense, and the academic journal Science and the international conference ICML prohibit writing papers using ChatGPT. David Smerdon, an assistant professor of economics at the University of Queensland and a chess grandmaster, explains why ChatGPT creates random papers.

Why does chatGPT make up fake academic papers?

By now, we know that the chatbot notoriously invents fake academic references. Eg's answer to the most cited economics paper is completely made-up (see image).

But why? And how does it make them? A THREAD (1/n) ???? pic.twitter.com/kyWuc915ZJ

— David Smerdon (@dsmerdon) January 27, 2023

According to Smerdon, ChatGPT is based on a language model that 'assigns a probability distribution to a series of words'. For example, given the sentence ``An apple a day,'' it will extract the most likely next word or sentence from its vast library, and use the Welsh proverb ` `An apple a day is I will explain the sentence 'Keep the doctor away '. In fact, due to a more sophisticated mechanism, ChatGPT has the function of ``predicting the beginning of a sentence and maintaining consistency throughout the sentence'', but roughly speaking, it is ``predicting the beginning of the sentence, It seems that it will be the idea of predicting the word with the highest probability.

A simplistic example: Give it “An apple a day…” and it will scan its immense library and come up with the most likely continuation: “…keep the doctor away.” (3/n) pic.twitter.com/Dg5IKlVYjR

— David Smerdon (@dsmerdon) January 27, 2023

To see ChatGPT in action, Smerdon entered the prompt, 'What is the most cited economics paper of all time?' The most common starting sentence for this question is 'What is the most cited economics paper of all time?', and ChatGPT will consider the following sentences.

Now consider the prompt “What is the most cited economics paper of all time”. The most 'likely' beginning to a language-based answer to this question is “The most cited economics paper of all time is”, which is what chatGPT spits out. (5/n) pic.twitter.com/olfbeYtrof

— David Smerdon (@dsmerdon) January 27, 2023

However, ChatGPT cannot find papers that are actually cited many times, and extracts ``frequent words'' from the library in the titles of economics papers that are frequently cited. ChatGPT understands that the words 'economy' and 'theory' appear frequently from the titles of economics papers that have been cited many times in the past 70 years, and named it 'A Theory of Economics.' produce words. Then, quoting 'history' as the word most likely to follow 'economic', I output the title of a nonexistent paper 'A Theory of Economic History'. And ChatGPT lists as its author Douglas North, the Nobel laureate in economics who is the most cited in relation to economic history.

Now we have the title of our fake paper: “A Theory of Economic History”.
The most highly-cited author associated with economic history is Nobel laureate Douglass North. https://t.co/CvtqwZodnO
(9/n)

— David Smerdon (@dsmerdon) January 27, 2023

In addition, as a co-author of the paper, Mr. Robert Thomas, whose co-authorship with Mr. North was the most cited in the past, was listed together, and Mr. North's most cited paper was published as the journal in which the fake paper was published. Select journal and ChatGPT answer is complete.

In this way, Smerdon points out that ChatGPT always selects the ``most likely answer'', resulting in non-existent titles and combinations of works and authors.