A version of the compact model 'LFM2.5' with DSpark, a speculative decoding technology that makes it more than twice as fast, has been released.



AI company Liquid AI has announced that it has applied its high-speed technology '

DSpark ' to its compact LFM2.5 series. The DSpark-enabled models have achieved more than double the speed while maintaining the same performance.

LFM2.5-DSpark: Up to 3.2x Faster Inference from H100 to MacBook — Blog — Liquid AI
https://www.liquid.ai/blog/lfm2.5-dspark




LFM2.5 is a compact AI model designed to run on devices with limited processing power, such as smartphones and laptops. It is available with multiple parameter counts to suit different devices, and a model specifically for the Japanese language has also been released.

The 'LFM2.5' series, an open model focused on on-device development, has been released, including the 'LFM2.5-1.2B-JP,' a compact AI model specifically designed for Japanese language use - GIGAZINE



Dspark is a type of AI acceleration technology called 'speculative decoding.' The AI model repeatedly performs the action of 'predicting the next token' and outputs text or code. Dspark accelerates the generation process by using a 'draft model' separate from the main model, and 'the draft model rapidly generates candidates for the next token, and the main model decides whether to accept or reject them.' One of Dspark's strengths is that it is not the type of method that 'degrades accuracy in exchange for speed,' but rather 'speeds up while maintaining perfect accuracy.'

Liquid AI has created draft models for three types: 'LFM2.5-1.2B-Instruct,' 'LFM2.5-2.6B,' and 'LFM2.5-8B-A1B.' The number of parameters for the draft models is 295.7 million for LFM2.5-1.2B-Instruct and 327.7 million for LFM2.5-2.6B and LFM2.5-8B-A1B.



Applying Dspark to LFM2.5-2.6B resulted in a 2.67x speed increase on H100 processors and a 2.27x speed increase on M4 Max-equipped MacBook Pros. For M4 Max-equipped MacBook Pros, LFM2.5-1.2B-Instruct achieved a 2.54x speed increase, and LFM2.5-8B-A1B achieved a 1.18x speed increase.



The Dspark-enabled model is available at the following link.

LiquidAI/LFM2.5-2.6B-DSpark · Hugging Face
https://huggingface.co/LiquidAI/LFM2.5-2.6B-DSpark

LiquidAI/LFM2.5-1.2B-Instruct-DSpark · Hugging Face
https://huggingface.co/LiquidAI/LFM2.5-1.2B-Instruct-DSpark

LiquidAI/LFM2.5-8B-A1B-DSpark · Hugging Face
https://huggingface.co/LiquidAI/LFM2.5-8B-A1B-DSpark

Piotr Mazurek of Liquid AI stated, 'Speculative decoding is a crucial component in compressing Fable-level intelligence to make it runnable on smartphones. This achievement is just the first step,' expressing optimism for future performance improvements.




Related Posts:

in AI, Posted by log1o_hf