AI ``Galactica'' that automatically generates scientific articles will be stopped publishing in just 3 days, and it turns out that ``lie articles'' can be generated depending on the input content



On November 15, 2022, Meta AI , an artificial intelligence (AI) laboratory belonging to Meta, which operates Facebook, will launch a natural language processing model `` Galactica '' that can `` store, fuse, and shape scientific knowledge ''. Opened to the public. However, it became clear that this Galactica could also output scientifically incorrect content, and Meta AI made the Galactica demo private in just three days from the release.

New Meta AI demo writes racist and inaccurate scientific literature, gets pulled | Ars Technica
https://arstechnica.com/information-technology/2022/11/after-controversy-meta-pulls-demo-of-ai-model-that-writes-scientific-papers/

Galactica is an AI trained on a 'database curated from the vast amount of human scientific knowledge' that includes over 48 million papers, textbooks, lecture notes, scientific websites, and encyclopedias. You can automatically generate scientific articles and explanations for questions while citing literature and formulas.

Meta researchers open to the public AI 'Galactica' that creates scientific articles and lecture notes that answer questions from enormous scientific knowledge - GIGAZINE



Galactica has been able to automatically generate scientific articles with a very high degree of accuracy, but it does not scrutinize whether the questions asked are scientifically correct. Therefore, there have been reports of cases where sentences are generated even for clearly unscientific questions.

For example, when I asked Galactica about the 'benefits of eating crushed glass', it seems that eating crushed glass was affirmed in various ways, such as details of previous research, animal experiments, and chemical explanations.



Also, when Galactica wrote an article about 'linguistic prejudice', it was reported that it contained discriminatory content against blacks and immigrants.



Even if the scientific knowledge of Galactica is accurate, if there is a mistake in the content entered by the human as a prompt, Galactica cannot distinguish between correct and incorrect answers, and the answer sought in natural sentences while skillfully connecting scientific knowledge. This means that the

Michael Black, director of the Max Planck Institute for Intelligent Systems in Germany and researcher of deep learning, said, ``I tried using Galactica, but I was in trouble. But it sounded right and authoritative. I think it's dangerous.'



In response to this situation, Meta AI stopped releasing the demo version of Galactica on November 18, 2022.



Yann LeCun, a professor at New York University and head of AI research at Meta AI, tweeted, ``Galactica's demo is offline. increase.

Related Posts:

in Software,   Science, Posted by log1i_yk