A QAD (Quick Adapter) version of the 'LFM2.5,' a compact AI model that runs on smartphones, has been released, featuring reduced memory usage while minimizing performance degradation.
AI company Liquid AI offers the LFM2.5 series as a compact AI model that can run on low-spec devices such as laptops and smartphones. A new version of the LFM2.5 series has been released as an open model that applies quantized recognition distillation (QAD), a technology that reduces memory usage while minimizing performance degradation.
LFM2.5 Q4_0: Quantization-Aware Distillation for Edge Deployment — Blog — Liquid AI
A widely used technique for reducing the memory usage of AI models is 'quantization.' However, because quantization reduces the size by 'reducing the computational precision of the model,' the quantized model performs worse than the base model. QAD is a technique that recovers the performance lost due to quantization by distilling the quantized model using a high-precision model as a training model. It is attracting attention as a method that can reduce memory usage while maintaining performance.
The LFM2.5 series offers multiple models with varying numbers of parameters and intended uses. The QAD versions released this time are 'LFM2.5-230M,' 'LFM2.5-350M,' 'LFM2.5-1.2B-Instruct,' and 'LFM2.5-2.6B,' which improve performance by distilling the models quantized with Q4_0.
The following graph shows the benchmark scores for each quantization precision when running the LFM2.5-230M on a 'MacBook Pro with M5 Max,' a 'mini PC with AMD Ryzen AI Max+ 395 (NucBox EVO-X2),' a 'Samsung Galaxy S26 Ultra,' and a 'Raspberry Pi 5.' The QAD version shows a significant performance improvement over the Q4_0 version, approaching the performance of the Q5_K_M version. The file size is 172MB for the Q5_K_M version and 149MB for the QAD version, indicating that the QAD version is more memory-efficient while minimizing performance degradation.
Similarly, the LFM2.5-350M also succeeded in reducing memory usage while minimizing performance degradation.
The same effect is observed in LFM2.5-1.2B-Instruct.
In the case of LFM2.5-2.6B, performance improved compared to the Q4_0 version, but it still fell short of the Q4_K_M version.
The LFM2.5 series is available free of charge as an open model, and QAD versions of each model are also available at the following link.
LiquidAI/LFM2.5-230M-GGUF · Hugging Face
https://huggingface.co/LiquidAI/LFM2.5-230M-GGUF
LiquidAI/LFM2.5-350M-GGUF · Hugging Face
https://huggingface.co/LiquidAI/LFM2.5-350M-GGUF
LiquidAI/LFM2.5-1.2B-Instruct-GGUF · Hugging Face
https://huggingface.co/LiquidAI/LFM2.5-1.2B-Instruct-GGUF
LiquidAI/LFM2.5-2.6B-GGUF · Hugging Face
https://huggingface.co/LiquidAI/LFM2.5-2.6B-GGUF
Related Posts:
in AI, Posted by log1o_hf