Meta releases a commercially available large-scale language model 'Llama 2' for free, and cooperates with Microsoft and Qualcomm to optimize for smartphones and PCs



Meta, which operates Facebook and Instagram, is also focusing on research and development of AI. Newly, Meta has announced the next generation model ' Llama 2 ' of the large language model '

LLaMA (Large Language Model Meta AI) '. Llama 2 is open to the public for free and can be used for commercial purposes other than research purposes, and it has also been announced that it will work with Microsoft and Qualcomm to optimize for Windows and smartphones.

Llama 2 - Meta AI
https://ai.meta.com/llama/

Meta and Microsoft Introduce the Next Generation of Llama | Meta
https://about.fb.com/news/2023/07/llama-2/

Microsoft and Meta Expand AI Partnership with Llama 2 on Azure and Windows - Official Microsoft Blog
https://blogs.microsoft.com/en/blog/2023/07/18/microsoft-and-meta-expand-their-ai-partnership-with-llama-2-on-azure-and-windows/

Qualcomm Works with Meta to Enable On-device AI Applications Using Llama 2 | Qualcomm
https://www.qualcomm.com/news/releases/2023/07/qualcomm-works-with-meta-to-enable-on-device-ai-applications-usi

Llama 2 has been trained with 2 trillion tokens and can handle twice the context length compared to the previous generation Llama (Llama 1). In addition, Llama 2 has three model sizes, '7B (7 billion)', '13B (13 billion)' and '70B (70 billion)', for each number of parameters.



Below is a table listing the benchmark results of Llama 2 and open source language models such as '

MPT ' and ' Falcon '. Llama 2 greatly exceeds the scores of other language models with the same model size, and Llama 2 with a model size of 70B (70 billion) has a higher score than Llama 1 with a model size of 65B (65 billion).
benchmark MPT(7B) Falcon (7B) Llama 2(7B) Llama 2 (13B) MPT(30B) Falcon (40B) Llama 1 (65B) Llama 2 (70B)
MMLU 26.8 26.2 45.3 54.8 46.9 55.4 63.4 68.9
Trivia QA 59.6 56.8 68.9 77.2 71.3 78.6 84.5 85.0
Natural Questions 17.8 18.1 22.7 28.0 23.0 29.5 31.0 33.0
GSM8K 6.8 6.8 14.6 28.7 15.2 19.6 50.9 56.8
Human Eval 18.3 N/A 12.8 18.3 25.0 N/A 23.7 29.9
AGIEval 23.5 21.2 29.3 39.1 33.8 37.0 47.6 54.2
Bool Q 75.0 67.5 77.4 81.7 79.0 83.1 85.3 85.0
Hella Swag 76.4 74.1 77.2 80.7 79.9 83.6 84.2 85.3
OpenBook QA 51.4 51.6 58.6 57.0 52.0 56.6 60.2 60.2
QuAC 37.7 18.8 39.7 44.8 41.1 43.3 39.8 49.3
Winogrande 68.3 66.3 69.2 72.8 71.0 76.9 77.0 80.2


In addition, Meta also reveals that it will work with technology companies such as Microsoft, Amazon, IBM, NVIDIA, and Qualcomm to develop Llama 2. Microsoft has already announced that it will be possible to fine-tune and deploy Llama 2 on Aure, and that it will be optimized for local operation on Windows. In addition, Qualcomm has also announced that it will be able to operate Llama 2 on devices equipped with Snapdragon by 2024.



The model data of Llama 2 is open to the public for free, and you can request a download by entering your name and affiliation on the following page. In addition, Llama 2 can be used not only for research purposes but also for commercial use.

Llama access request form - Meta AI
https://ai.meta.com/resources/models-and-libraries/llama-downloads/



Related Posts:

in Software, Posted by log1o_hf