Spending over tens of billions of yen, Microsoft built a supercomputer for AI development for OpenAI, the developer of ChatGPT
Since 2019, Microsoft has invested
How Microsoft's bet on Azure unlocked an AI revolution - Source
https://news.microsoft.com/source/features/ai/how-microsofts-bet-on-azure-unlocked-an-ai-revolution/
Microsoft Built an Expensive Supercomputer to Power OpenAI's ChatGPT - Bloomberg
https://www.bloomberg.com/news/articles/2023-03-13/microsoft-built-an-expensive-supercomputer-to-power-openai-s-chatgpt
Microsoft spent hundreds of millions of dollars on a ChatGPT supercomputer - The Verge
https://www.theverge.com/2023/3/13/23637675/microsoft-chatgpt-bing-millions-dollars-supercomputer-openai
According to overseas media Bloomberg, when Microsoft invested $ 1 billion (about 110 billion yen at that rate) in OpenAI in 2019, it also agreed to build a large-scale state-of-the-art supercomputer for OpenAI. He said he did. AI development companies such as OpenAI must have access to powerful computing resources as they need to ingest and train large amounts of data to develop large-scale AI models.
'One of the things we've learned from the research is that the larger the model, the more data, and the longer the training time, the more accurate the model,' said Nidhi Chappell, head of Azure high-performance computing and AI products at Microsoft. So there was a strong push to train larger models for a longer period of time, which means not only having the largest infrastructure, but also having to run it stably for a long period of time.” said.
Microsoft worked closely with OpenAI to build a supercomputer capable of training powerful AI models and to provide computing resources through cloud services. In building a supercomputer, Microsoft has faced various challenges such as cable tray design, increasing server capacity in data centers, optimizing communication between GPU units, devising power supply arrangements to prevent power outages in data centers, and cooling machines. Said it was dealt with.
Scott Guthrie, executive vice president of Microsoft, did not disclose the specific amount of money spent on the project, but commented in an interview with Bloomberg that it was 'probably larger (more than hundreds of millions of dollars).' increase.
In 2020, Microsoft developed a supercomputer connecting 10,000 NVIDIA A100 GPUs for AI and data centers made by NVIDIA and provided it with Microsoft Azure , a cloud computing service.
Microsoft develops a supercomputer with 285,000 CPU cores and 10,000 GPUs in cooperation with OpenAI - GIGAZINE
``We have built a system architecture that can be operated at a very large scale and is highly reliable. It shows the view that it will be born from a supercomputer built by.
Microsoft also sells supercomputing resources to customers outside of OpenAI. Guthrie told Bloomberg, ``The supercomputer started as a custom-made one, but it was built to be generalized so that people who want to train large-scale language models can use it. , could be a good cloud for AI more broadly.'
Since then, Microsoft has continued to evolve its infrastructure for AI, and the blog on March 13, 2023 uses Quantum-2 InfiniBand , an architecture that provides 8 to thousands of NVIDIA H100 GPUs and high-speed networks. announced the AI virtual machine ' ND H100 v5 VM '.
Azure previews powerful and scalable virtual machine series to accelerate generative AI | Azure Blog and Updates | Microsoft Azure
https://azure.microsoft.com/en-us/blog/azure-previews-powerful-and-scalable-virtual-machine-to-help-customers-accelerate-ai/
Microsoft continues to tackle issues such as customized server and chip design, supply chain optimization, and cost reduction. “The AI models that are taking the world by storm today are being developed on supercomputers that started being built two or three years ago,” Guthrie said. I plan to do so,” he said.
Related Posts:
in Hardware, Posted by log1h_ik