The code generated by ChatGPT is not necessarily safe, and ChatGPT itself is aware of its vulnerabilities

AI chatbots like ChatGPT can not only process text like humans do, but they can also write code. However, when I verified whether the code was safe or not, I found that it sometimes output code with security problems, and that ChatGPT itself understood its vulnerability. .

[2304.09655] How Secure is Code Generated by ChatGPT?

ChatGPT creates mostly insecure code, but won't tell you • The Register

This problem was investigated by a team of computer scientists Raphaël Khoury and others at the University of Quebec, and the results were published on the dissertation publishing site arXiv as a formal pre-peer-reviewed manuscript.

Khoury et al. Let ChatGPT generate 21 different programs and scripts in five languages: C, C ++, Python, HTML, and JAVA, and as a result, ``ChatGPT also outputs unsafe code''. I was.

GitHub - RaphaelKhoury/ProgramsGeneratedByChatGPT: Programs generated by ChatGPT
https://github.com/RaphaelKhoury/ProgramsGeneratedByChatGPT

According to Khoury et al., 5 of the 21 programs and scripts originally created by ChatGPT were safe, and the others were vulnerable to memory corruption, denial of service, deserialization, and improper implementation of encryption. It was said that there was a problem with

After that, when we urged them to fix it, we succeeded in generating 7 safe programs. It seems that it could not be said that there is no exploitable vulnerability.

Khoury et al. point out that one of the reasons for the occurrence of these problems is that they did not assume an adversarial model of code execution.

On the other hand, it is also pointed out that ChatGPT itself recognizes that the generated code has important vulnerabilities, but does not say anything unless asked to evaluate its security.

For this reason, Khoury et al. said that there is an ethical contradiction in the fact that ChatGPT generates vulnerable code while refusing to generate offensive code.