A guide has been released for running the Claude Opus-class GLM-5.2 locally; the minimum configuration is estimated to be 223GB of memory.
Unsloth, a company specializing in AI model quantization and local execution environments, has released a guide for running the high-performance AI model 'GLM-5.2' in a local environment.
GLM-5.2 - How to Run Locally | Unsloth Documentation
In recent years, high-performance AI has increasingly been used not just as a chat partner, but as a development support agent that reads entire codebases and creates plans. However, when using cloud-based AI like Claude or ChatGPT, it is necessary to send source code and internal documents to an external service, which itself becomes a hurdle when dealing with 'code for unreleased products,' 'logs containing customer data,' or 'design documents for internal use.' This is where local AI, which runs the AI model locally, comes into play.
GLM-5.2 is a large-scale language model officially announced by the Chinese AI company Z.ai on June 17, 2026 (Japan time). It prioritizes performance for 'long-term tasks' such as modifying code over long periods or autonomously carrying out multi-step tasks. According to Z.ai, GLM-5.2 can handle long contexts of up to 1 million tokens, and demonstrates high performance in FrontierSWE, a measure of long-term coding ability, where it is 1% lower than Claude Opus 4.8 but 1% higher than GPT-5.5 and 11% higher than Claude Opus 4.7.
GLM-5.2, a Chinese model surpassing Claude Opus 4.7, has been officially announced. It has even outperformed Claude Fable 5 in some tests and is available for download as an open model - GIGAZINE
GLM-5.2 is an open model, meaning anyone can download and use it locally. However, running such a high-performance model locally requires a huge amount of memory. While lighter models may run on laptops, large-scale models comparable to Claude Opus will require far more memory than a typical gaming PC or development laptop. GLM-5.2 is no exception, requiring a massive 1.51TB of memory for its standard 16-bit version.
The solution presented in Unsloth's documentation is 'quantization.' Quantization is a technique that reduces the amount of memory and storage space required by making the numerical representation within a model lighter. Just as image files are compressed while maintaining a certain level of image quality, AI models can be converted into a more manageable format while balancing accuracy and size. Unsloth has released GLM-5.2 in GGUF format and provides multiple quantization versions: 1-bit, 2-bit, 3-bit, 4-bit, 5-bit, 6-bit, 8-bit, and 16-bit.
The required memory capacity is 810GB for the 8-bit version, 372GB to 475GB for the 4-bit version, 245GB for the 2-bit version, and 223GB for the 1-bit version. It is also stated that having even more memory available will improve performance.
To investigate the impact of quantization on the accuracy of GLM-5.2-GGUF, Unsloth also measured KLD, an index that shows the difference in output distribution from the original model. The following is a graph showing various quantization models with top-1% accuracy on the vertical axis and model capacity on the horizontal axis. The 4-bit quantization model has a top-1% accuracy of approximately 97.5%, which is quite close to the quality of the original model. Even smaller quantization versions are highly practical, with the 1-bit quantization model achieving a top-1% accuracy of approximately 76.2% while reducing the size by 86%. Note that since top-1% accuracy is a measure of 'whether the same word as the original model was output,' even if a natural response is given but a different word from the original model is selected, it is counted as 'did not match,' meaning that the actual output quality does not decrease as much as the numbers suggest.
The documentation states that the 2-bit quantization model can run smoothly by using a Mac with 256GB of integrated memory, or by enabling 'MoE offloading,' which offloads some processing to memory other than the GPU, on a workstation with a graphics card with 24GB of VRAM and 256GB of memory.
Related Posts:
in AI, Posted by log1d_ts