Intel announces AI accelerator 'Gaudi 3', which is faster and consumes less power than NVIDIA's H100 and even beats the H200 in some tests
Intel announced its AI accelerator ' Gaudi 3 ' on April 9, 2024. Gaudi 3 is said to have higher performance than NVIDIA's AI-specialized GPU 'H100' in terms of both AI learning performance and AI inference performance.
Intel Gaudi 3 AI Accelerator
Intel Breaks Down Proprietary Walls to Bring Choice to Enterprise GenAI Market
https://www.intel.com/content/www/us/en/newsroom/news/vision-2024-gaudi-3-ai-accelerator.html#gs.7pyw4a
Intel Unleashes Enterprise AI with Gaudi 3, AI Open Systems Strategy and New Customer Wins :: Intel Corporation (INTC)
https://www.intc.com/news-events/press-releases/detail/1689/intel-unleashes-enterprise-ai-with-gaudi-3-ai-open-systems
Gaudi 3 is equipped with 64 cores of Intel's 5th generation Tensor Processor Core (TPC) and 8 cores of matrix multiplication engine (MME) capable of 64,000 parallel processes. The computational performance reaches 1835 TFLOPS. It also has 128GB of HBM2e memory with a memory bandwidth of 3.7TB/s and 96MB of SRAM. This ensures sufficient memory for learning and inference of large-scale language models and multimodal AI models. In addition, it is equipped with 24 ports of 200 Gigabit Ethernet, which supports the construction of large-scale computing clusters.
According to Intel's test results, Gaudi 3 can reduce the learning time of large-scale language models such as 'Llama2 7B', 'Llama2 13B', and 'GPT-3 175B' by 50% compared to NVIDIA's H100. It has also been confirmed that the inference speed of 'Llama 7B', 'Llama 70B', and 'Falcon 180B' is 50% faster than H100 and power efficiency is 40% improved. In addition, it has been confirmed that 'Llama 7B', 'Llama 70B', and 'Falcon 180B' can perform inference 30% faster than H200, which NVIDIA plans to release in 2024.
Gaudi 3 is scheduled to be supplied to OEMs such as Dell, Hewlett Packard Enterprise, Lenovo and Supermicro in the second quarter of 2024, with general availability beginning in the third quarter of 2024. Gaudi 3 PCIe add-in cards are also scheduled to begin shipping in the fourth quarter of 2024.
Related Posts:
in Hardware, Posted by log1o_hf