'Prompt engineers' who operate AI are being eliminated by AI

'Prompt engineering' is the act of adjusting prompts, which are instructions to AI, in order to obtain the intended output. Until now, prompt engineering has been performed by human engineers, but research results have shown that performance can be improved by using AI to create prompts themselves.

[2402.10949] The Unreasonable Effectiveness of Eccentric Automatic Prompts


AI Prompt Engineering Is Dead - IEEE Spectrum

Since the introduction of ChatGPT in the fall of 2022, there have been many attempts to improve the AI's output by devising prompts. Along with this, prompt engineers have emerged as experts in finding 'appropriate prompts' that vary depending on models, datasets, etc.

However, a paper by Rick Battle and Teja Gollapudi published on February 20, 2024 suggested that prompt engineering using AI is extremely effective, and the profession of prompt engineer will be replaced by AI. The possibility has surfaced.

Mr. Battle and Mr. Gollapudi prepared 60 different prompts each and compared the results in the task of having three AI models solve elementary school math problems. Techniques called chain of thought, such as ``explaining reasoning step-by-step,'' are sometimes effective, but at other times, performance worsens, and both authors say ``the only trend is that there is no trend.'' They point out that 'the optimal prompt may differ for each model and dataset.'

In this way, there is no consistency regarding the optimal method for prompt engineering, and trial and error is required. Therefore, Mr. Battle and Mr. Gollapudi created a tool to automatically generate prompts, evaluated the output result of the prompt, and created a prompt again with the tool based on the evaluation. As a result, the time required for prompt engineering was shortened from several days to several hours compared to when humans perform prompt engineering, and at the same time, the final AI output results were also improved.

Some of the prompts generated by the algorithm read, ``Commander, plan the course of this turbulence and determine the source of the anomaly. Use all available data and expertise to overcome this difficult situation.'' It also included strange things like 'Please.'

Other efforts to automatically generate such prompts include Shachar Rosenman, Vasudev Lal, and Philip Howard at Intel Labs, who created a prompt reinforcement tool called

NeuroPrompts .

This tool automatically adds an appropriate prompt to StableDiffusion, an image generation AI, before inputting a prompt created by a human. Below is an example of when trying to execute the prompt 'A robot is playing the piano'. The two images lined up at the bottom right are the output result of the prompt added by AI on the left, and the output result of the prompt input by a human on the right, which has a high quality with a solid feel due to the addition by AI. You can see that an image is generated.

The example 'A boy on a horse' is as shown below. If you input it into StableDiffusion as is, it will output an image that looks like the old photo shown on the left, but if you enhance the prompt with NeuroPrompts, it will output a 'more beautiful' image like the one on the right.

'It will be important to explore optimizing these prompts and incorporate them into the base model itself, eliminating the need for complex prompt engineering steps,' said Lal, one of the researchers who developed NeuroPrompts. I mentioned my idea that there should be no need for a prompt engineer.

in Software, Posted by log1d_ts