Research results show that when AI is told human-like words such as ``Let's take a deep breath,'' the rate of correct answers to questions increases
AI can solve a variety of problems and generate natural responses according to input prompts, but we know that its accuracy is affected by how the prompts are written.
[2309.03409] Large Language Models as Optimizers
https://arxiv.org/abs/2309.03409
Uh oh, now AI is better than you at prompt engineering | ZDNET
https://www.zdnet.com/article/uh-oh-now-ai-is-better-than-you-at-prompt-engineering/
Telling AI model to “take a deep breath” causes math scores to soar in study | Ars Technica
https://arstechnica.com/information-technology/2023/09/telling-ai-model-to-take-a-deep-breath-causes-math-scores-to-soar-in-study/
Generally, to improve the performance of AI models, optimization algorithms and optimizers based on the mathematical definition of the task are used. In response, the DeepMind research team devised a method called Optimization by PROmpting (OPRO), which uses large-scale language models to improve meta-prompts written in natural language and perform an optimization process. Did.
The paper ``Large Language Models as Optimizers'' published on the paper preprint server arXiv explains how OPRO works.
The diagram below is a simple representation of how OPRO works. At the heart of OPRO is an algorithm called ``meta-prompt,'' which evaluates how effective past prompts were in solving a particular task. Based on this, a large-scale language model generates improved prompts, the responses are evaluated and scored, and meta-prompts are used to make further improvements.
The way OPRO works is that a large-scale language model is responsible for the process by which humans try and error through various prompts and gradually improve them. The research team says, 'Instead of formally defining an optimization problem and having a programmed solver derive update steps, we describe the optimization problem in natural language. Then, based on the problem description and previous solutions, we 'We instruct large-scale language models to iteratively generate new solutions.'
The research team tested the performance of OPRO on
The technology media Ars Technica points out that this research is particularly interesting: ``Take a deep breath and work on this problem step by step.'' '', the correct answer rate for GSM8K in Google's large-scale current model PaLM 2 was the highest. In PaLM 2, the correct answer rate for GSM8K was only 34% when there was no special prompt, but when 'Let's think step by step' was added, the correct answer rate increased to 71.8%. When they added, ``Let's take a deep breath and take this problem one step at a time,'' the correct answer rate reached 80.2%.
Naturally, AI does not have a body like humans, so it will not take a deep breath even if it is advised to take a deep breath. Nevertheless, a 2022 study published by the University of Tokyo and Google also reported that such human-like advice improves AI accuracy.
Ars Technica notes that books and web Q&A forums on training AI models often include phrases like 'Let's take it one step at a time' or 'Let's take a deep breath' before offering more prudent solutions. It is pointed out that there are often Therefore, they argued that inputting these phrases could help AI models arrive at high-quality answers in datasets or generate good examples of inference and problem-solving.
A forum related to this article has been set up on the GIGAZINE official Discord server. Anyone can write freely, so please feel free to comment!
• Discord | 'Do you use honorific language when talking to chat AI?' | GIGAZINE
https://discord.com/channels/1037961069903216680/1154344513192607804
Related Posts:
in Software, Web Service, Science, Posted by log1h_ik