Oct 01, 2024 15:00:00

MIT spinoff Liquid AI releases non-Transformer AI models LFM 1B, 3B, and 40B MoE

As the word

Transformer is used in the official name of OpenAI's ChatGPT, 'Chat Generative Pre-trained Transformer,' at the time of writing, many of the major AI models used Transformer, a deep learning model announced by Google. Liquid AI, an AI company that aims to build a foundation model that surpasses GPT without using Transformer, has announced the first series of Liquid Foundation Models (LFM), which demonstrate excellent performance while minimizing memory footprint .

Liquid Foundation Models: Our First Series of Generative AI Models
https://www.liquid.ai/liquid-foundation-models

The generative AI LFM announced this time is available in three sizes: LFM 1.3B, which is ideal for resource-constrained environments; LFM 3B, a medium-sized model of experts; and LFM 40B MoE, a mixed expert model (MoE) designed to tackle more complex tasks. According to Liquid AI, a spinoff company of the Massachusetts Institute of Technology (MIT), LFM has excellent performance relative to its size.

Liquid AI explained, 'LFM is a large-scale neural network built with computational units deeply rooted in the theory of dynamical systems, signal processing, and numerical linear algebra. This unique combination leverages decades of theoretical advances in these fields to make intelligence at any scale possible. In this way, LFM has become a general-purpose AI model that can be used to model any kind of sequential data, including video, audio, text, time series, and signals.'

LFM is more memory efficient than Transformer-based models, and by efficiently compressing the input, it is possible to process longer sequences on the same hardware.

Below is a graph of the memory required for inference on a 3B class model, showing how small the memory footprint of the LFM 3B is.