Jul 17, 2024 11:05:00

Mistral AI releases code generation AI 'Codestral Mamba' under open source license

Mistral AI , an AI development company founded by former employees of Meta and Google DeepMind, has announced a coding AI model called Codestral Mamba . It has been released under an open source license and is available for commercial use.

Codestral Mamba | Mistral AI | Frontier AI in your hands
https://mistral.ai/news/codestral-mamba/

Mistral AI released 'Codestral' in May 2024 as the first generative AI model for coding, but Codestral was prohibited for commercial use.

Mistral AI releases first generative AI model for coding, 'Codestral,' trained in over 80 programming languages - GIGAZINE

The newly released Codestral Mamba uses the ' Mamba architecture ' instead of the Transformer architecture that is widely used in conventional models. It has features such as linear processing time with respect to sequence length, high-speed processing even for long sequences, and no limit on sequence length.

The benchmark results are shown in the figure below. Codestral Mamba has 7 billion (7B) parameters, and we can see that it performs at the top of the class among models of similar size. Although it loses overall performance to the Codestral model with 22 billion (22B) parameters, it outperforms it in some indicators, demonstrating the high potential of the Mamba architecture.

The Mistral AI team has tested Codestral Mamba's in-context search feature on up to 256,000 tokens and hopes that it will perform well as a local code assistant.

The Codestral Mamba model is available for download from Hugging Face , as well as from Mistral Inference , the official Mistral library.