Meta releases commercial large-scale language model 'Llama 2' for free, and collaborates with Microsoft and Qualcomm to optimize it for smartphones and PCs
Meta, the company behind Facebook and Instagram, is also focusing on AI research and development. Recently, Meta announced Llama 2, the next generation of its large-scale language model,
Llama 2 - Meta AI
https://ai.meta.com/llama/
Meta and Microsoft Introduce the Next Generation of Llama | Meta
https://about.fb.com/news/2023/07/llama-2/
Microsoft and Meta Expand AI Partnership with Llama 2 on Azure and Windows - Official Microsoft Blog
https://blogs.microsoft.com/en/blog/2023/07/18/microsoft-and-meta-expand-their-ai-partnership-with-llama-2-on-azure-and-windows/
Qualcomm Works with Meta to Enable On-device AI Applications Using Llama 2 | Qualcomm
https://www.qualcomm.com/news/releases/2023/07/qualcomm-works-with-meta-to-enable-on-device-ai-applications-usi
Llama 2 has been trained with 2 trillion tokens and can handle twice the context length of the previous generation Llama (Llama 1). Llama 2 also offers three model sizes for each parameter: 7B (7 billion), 13B (13 billion), and 70B (70 billion).
Below is a table listing benchmark results for Llama 2 and open source language models such as '
| benchmark | MPT(7B) | Falcon(7B) | Llama 2(7B) | Llama 2(13B) | MPT(30B) | Falcon(40B) | Llama 1(65B) | Llama 2(70B) |
|---|---|---|---|---|---|---|---|---|
| MMLU | 26.8 | 26.2 | 45.3 | 54.8 | 46.9 | 55.4 | 63.4 | 68.9 |
| TriviaQA | 59.6 | 56.8 | 68.9 | 77.2 | 71.3 | 78.6 | 84.5 | 85.0 |
| Natural Questions | 17.8 | 18.1 | 22.7 | 28.0 | 23.0 | 29.5 | 31.0 | 33.0 |
| GSM8K | 6.8 | 6.8 | 14.6 | 28.7 | 15.2 | 19.6 | 50.9 | 56.8 |
| HumanEval | 18.3 | N/A | 12.8 | 18.3 | 25.0 | N/A | 23.7 | 29.9 |
| AGIEval | 23.5 | 21.2 | 29.3 | 39.1 | 33.8 | 37.0 | 47.6 | 54.2 |
| BoolQ | 75.0 | 67.5 | 77.4 | 81.7 | 79.0 | 83.1 | 85.3 | 85.0 |
| HellaSwag | 76.4 | 74.1 | 77.2 | 80.7 | 79.9 | 83.6 | 84.2 | 85.3 |
| OpenBookQA | 51.4 | 51.6 | 58.6 | 57.0 | 52.0 | 56.6 | 60.2 | 60.2 |
| QuAC | 37.7 | 18.8 | 39.7 | 44.8 | 41.1 | 43.3 | 39.8 | 49.3 |
| Winogrande | 68.3 | 66.3 | 69.2 | 72.8 | 71.0 | 76.9 | 77.0 | 80.2 |
Meta also announced that it will collaborate with technology companies such as Microsoft, Amazon, IBM, NVIDIA, and Qualcomm to develop Llama 2. Microsoft has already announced that it will enable fine-tuning and deployment of Llama 2 on Aure and that it will optimize Llama 2 to run locally on Windows. Qualcomm has also announced that it will enable Llama 2 to run on Snapdragon-powered devices by 2024.
The Llama 2 model data is available for free, and you can request a download by entering your name and affiliation on the following page. Llama 2 can also be used for research purposes and for commercial purposes.
Llama access request form - Meta AI
https://ai.meta.com/resources/models-and-libraries/llama-downloads/
Related Posts:
