Jul 25, 2024 11:13:00

Mistral AI releases 'Mistral Large 2' with significantly improved code generation, mathematics and inference capabilities

French AI development company Mistral AI announced its new generation flagship model, Mistral Large 2 , on July 24, 2024. Mistral Large 2 has significantly improved code generation, mathematics, and inference capabilities, as well as a 128k context window and support for dozens of languages and programming languages.

Large Enough | Mistral AI | Frontier AI in your hands

https://mistral.ai/news/mistral-large-2407/

Mistral Large 2 has a model size of 123 billion parameters and is designed to achieve high throughput on a single node. It also has a 128k context window and supports many languages other than English, including French, German, Spanish, Italian, Portuguese, Arabic, Hindi, Russian, Chinese, Japanese, and Korean. It also supports more than 80 programming languages, including Python, Java, C, C++, JavaScript, and Bash.

Mistral AI claims that 'Mistral Large 2 achieved 84.0% accuracy in the MMLU (Massive Multitask Language) benchmark, a common performance evaluation index.' In particular, in terms of code generation and inference capabilities, it significantly outperforms the previous generation Mistral Large, and is said to perform on par with models such as GPT-4, Claude3 Opus, and Llama 3 405B.

Mathematical reasoning capabilities have also improved, achieving high accuracy in math benchmarks such as MultiPL-E , GSM8K , and MathInstruct . Mistral AI claims that efforts have been made to minimize hallucinations in Mistral Large 2, which is reflected in the improved performance in math benchmarks.

Below is a table summarizing the accuracy of code generation by language, which shows that it is comparable to OpenAI's GPT-4o.

In terms of its ability to follow instructions and its conversational skills, the model has achieved high scores in benchmarks such as

MTBench , Wild Bench , and Arena Hard . Notably, Mistral AI emphasizes that the model's answers are concise, which enables quick dialogue and keeps inference costs low.

It also shows excellent performance in language diversity, and in the

Multilingual MMUL benchmark, Mistral Large 2 achieved high scores in languages other than English. In particular, it is reported that it performed well in French, German, Spanish, Italian, Portuguese, Dutch, Russian, Chinese, Japanese, Korean, Arabic, and Hindi. Below are the results of the Multilingual MMUL benchmark, which shows an accuracy of over 80%, almost the same as Llama 3.1 405B with 405 billion parameters.

In addition, Mistral Large 2 has enhanced function call and information retrieval skills, allowing it to efficiently execute parallel and sequential function calls, making it the core engine for complex business applications.

Mistral Large 2 is available on La Plateforme as 'mistral-large-2407', the model is distributed on Hugging Face and can be accessed through an API, and is also available through major cloud service providers such as Google Cloud Platform's Vertex AI, Azure AI Studio, Amazon Bedrock, and IBM watsonx.ai. With Mistral Large 2, Mistral AI aims to make high-performance AI models available to a wider range of users.

Mistral Large 2 has been released under the Mistral Research License for research and non-commercial use. For commercial use, you must contact Mistral AI to obtain a Mistral Commercial License.