63% of the cost of AI chips goes to memory; we're entering an era where memory costs more than the GPU.
Epoch AI, a research firm that studies AI advancements and infrastructure, has investigated AI chips designed by NVIDIA, AMD, Google, and Amazon and reported that the proportion of memory in the component cost of AI chips rose from 52% in the first quarter of 2024 to 63% in the fourth quarter of 2025.
AI Chip Component Costs: Memory at 63% | Epoch AI | Epoch AI
In semiconductors for AI, not only the GPU or accelerator itself that handles calculations is important, but also the memory that can quickly input and output large amounts of data. AI model training and inference involve massive matrix calculations, so even if the computing chip is high-performance, it cannot perform to its full potential if the data supply is slow. Therefore, AI chips utilize large amounts of high-bandwidth memory (HBM), a type of high-speed memory.
The image below shows the breakdown of memory, logic, packaging, and auxiliary components in the component cost of an AI chip. You can see that the light blue area at the bottom of the graph, which shows the proportion of memory, expanded from the first quarter of 2024 to the fourth quarter of 2025.
Epoch AI broadly categorized the component costs of AI chips into four types: 'memory,' 'logic,' 'packaging,' and 'auxiliary components.' Memory refers to HBM stacks such as HBM3 and HBM3e; logic refers to the computing semiconductor itself, manufactured using advanced processes from 3nm to 5nm; packaging refers to technologies that integrate multiple chips, such as TSMC's CoWoS; and auxiliary components refer to substrates and power supply components.
In the first quarter of 2024, the component cost breakdown was 52% for memory, 14% for logic, 19% for packaging, and 15% for auxiliary components. However, by the fourth quarter of 2025, memory is projected to increase to 63%, logic will remain relatively stable at around 13%, packaging will decrease to 15%, and auxiliary components to 10%. While the name 'AI chip' might suggest that the computational logic is the most expensive component, in reality, memory accounts for the majority of the component cost.
Memory spending has also grown significantly in monetary terms. According to Epoch AI, memory-related spending on AI chips designed by NVIDIA, AMD, Google, and Amazon increased from approximately $12 billion (approximately 1.9 trillion yen) in 2024 to approximately $32 billion (approximately 5 trillion yen) in 2025. Overall component spending on AI chips also expanded from approximately $22 billion (approximately 3.5 trillion yen) in 2024 to approximately $52 billion (approximately 8.26 trillion yen) in 2025, with approximately $20 billion (approximately 3.18 trillion yen) of that increase attributed to memory-related spending.
One reason for the increasing proportion of memory is that AI chips require a high level of memory bandwidth. Unlike typical DRAM modules, the high-bandwidth memory used in AI chips stacks multiple memory chips vertically and connects them with wide data paths, allowing for high-speed reading and writing of large amounts of data. In processing large-scale language models and image generation AI, parameters and intermediate data are frequently exchanged, making memory capacity and bandwidth components that affect both performance and cost.
However, Epoch AI's figures are estimates, not entirely measured values. Component unit prices vary depending on contract terms, suppliers, and purchase timing, and there are uncertainties in the production volume and product configuration for each chip. Epoch AI models the range of component costs using a 90% confidence interval, and explains that the 63% memory ratio for Q4 2025 is between 60% and 67% when considering only the cost range of memory alone, and between 54% and 73% when considering the uncertainties of all components simultaneously.
Epoch AI states that 'as memory supply becomes tighter and prices rise, memory may account for an even larger proportion of AI chip component costs in 2026.' Rising component prices are also said to be influencing Microsoft's capital expenditure forecast for fiscal year 2026 and Meta's upward revision of its capital expenditure range for 2026.
When considering the cost of AI chips, not only the computational performance of GPUs and accelerators, but also the supply and price of memory are important factors. Epoch AI stated that 'memory was the biggest driver of increased AI chip component spending from 2024 to 2025, and memory prices are becoming increasingly important in understanding the cost structure of AI infrastructure investments.'
Related Posts:
