Jul 08, 2024 10:58:00

Anthropic CEO predicts that the cost of training AI could rise to $100 billion in just three years

Dario Amodei , CEO of Anthropic, an AI company that develops chat AI such as 'Claude,' claims that 'AI training costs could rise to $10 billion (about 1.6 trillion yen) to $100 billion (about 16 trillion yen) over the three years from 2025 to 2027.'

Dario Amodei - CEO of Anthropic | Podcast | In Good Company | Norges Bank Investment Management - YouTube


AI models that cost $1 billion to train are underway, $100 billion models coming — largest current models take 'only' $100 million to train: Anthropic CEO | Tom's Hardware
https://www.tomshardware.com/tech-industry/artificial-intelligence/ai-models-that-cost-dollar1-billion-to-train-are-in-development-dollar100-billion-models-coming-soon-largest-current-models-take-only-dollar100-million-to-train-anthropic-ceo

In a conversation with

Norges Bank Investment Management , Amodei said that we won't suddenly reach artificial general intelligence (AGI) from generative AI like ChatGPT. Instead, we'll 'progress toward AGI in stages, based on the evolution of past models, just like a human child learns gradually.'

Therefore, Amodei predicted, 'If AI models become 10 times more powerful every year, the hardware required to train them will also need to be more than 10 times stronger. Hardware will be the largest cost factor in AI training.' In fact, it has been reported that as of 2023, ChatGPT training will require as many as 30,000 GPUs, and Elon Musk's AI startup ' xAI ' has revealed that it plans to purchase approximately 300,000 AI GPUs ' B200 ', which cost approximately $30,000 (approximately 4.81 million yen) to $40,000 (approximately 6.42 million yen) per unit.

Given the pace of technology improvement, it's not worth sinking 1GW of power into H100s.

The @xAI 100k H100 liquid-cooled training cluster will be online in a few months.

The next big step would probably be ~300k B200s with CX8 networking next summer.

— Elon Musk (@elonmusk) June 2, 2024

It has also been reported that OpenAI and Microsoft are planning to open a $115 billion AI data center.

According to Amodei, the cost of training AI is already rising: It costs about $100 million to train current AI models, including OpenAI's GPT-4o, and the cost of training the AI model Anthropic is developing is estimated at about $1 billion.

'The cost of training AI will rise to $10 billion over the three-year period from 2025 to 2027, and possibly even $100 billion,' Amodei said. 'If algorithms and chips continue to improve at this rate, I think there's a good chance we'll have AI models that are better than humans by around 2027.'