'AI-developed biological weapons' emerge as a national security concern, prompting the U.S. government and AI companies to consider regulating them

The development of AI technology in recent years has been remarkable, and it is now possible not only to generate natural sentences that look like they were written by humans with a high degree of accuracy, but also to automatically write program source code and predict the three-dimensional structure of proteins. Experts are sounding the alarm, saying that 'the use of AI is lowering the hurdle to developing new biological weapons.'

Threats From AI: Easy Recipes for Bioweapons Are New Global Security Concern - Bloomberg
https://www.bloomberg.com/news/features/2024-08-02/national-security-threat-from-ai-made-bioweapons-grips-us-government

Rocco Casagrande, a biochemist and former UN weapons inspector, warned White House officials in the spring of 2023 about the possibility that AI chatbots could teach people how to create dangerous viruses.

In his demonstration, Casagrande showed how he actually purchased synthetic DNA for a pathogen suggested by the AI, put it in a test tube, and delivered it to the White House, proving that it is relatively easy to use AI to obtain materials for biological weapons.

Casagrande's warning spread through national security networks, with senior officials at the Defense, State and Homeland Security departments calling for similar demonstrations, raising awareness of how vulnerable the US is to the possibility that AI could facilitate the development of biological weapons.

The background to this issue is the fusion of synthetic biology and AI. Advances in gene editing technology, developments in computational biomechanics , and the emergence of cloud labs that can simulate experiments online have made it possible to create dangerous organisms quickly and cheaply without traditional research facilities. Casagrande warns that the addition of AI to the mix has made it even easier to access information about harmful organisms.

The Biden Administration issued a comprehensive executive order in late October 2023 to address this threat.

FACT SHEET: President Biden Issues Executive Order on Safe, Secure, and Trustworthy Artificial Intelligence | The White House
https://www.whitehouse.gov/briefing-room/statements-releases/2023/10/30/fact-sheet-president-biden-issues-executive-order-on-safe-secure-and-trustworthy-artificial-intelligence/

The order includes measures such as increased oversight of government-funded research using synthetic DNA, creating an agency to review AI safety, and creating guidelines and reporting requirements for AI tools that handle biological data.

Additionally, in March 2024, more than 130 scientists signed a pledge to prevent the use of AI in the development of biological weapons.

More than 130 scientists sign pledge to prevent AI from being used in biological weapons development, maximizing the benefits and minimizing the risks of AI protein design - GIGAZINE

Tom Inglesby, director of the Johns Hopkins Center for Health Security, argued that governments should 'regularly audit AI companies and consider ways to hold them accountable if they cause harm.'

Meanwhile, AI companies are also taking measures. Anthropic and OpenAI are adjusting their models to prevent AI from providing information about biological weapons. Casagrande has also formed a team of experts to test the safety of AI with the cooperation of Anthropic and OpenA.

However, it is difficult to identify and remove all harmful information, and to distinguish between 'useful scientific information' and 'potentially dangerous information.' Furthermore, for security reasons, AI companies tend not to disclose the details of specific countermeasures, and new vulnerabilities may emerge as AI capabilities improve, so the countermeasures are not sufficient.

Anthropic co-founder Jack Clark likened AI companies to 'financial institutions without accountants,' and pointed out the need for third-party organizations to test, evaluate and certify AI systems.