Meta releases new version of code generation AI model 'Code Llama 70B', improves code accuracy and also provides Python-optimized variant



Meta, which operates Facebook and Instagram, announced that it has released a 70 billion parameter model of the AI ``Code Llama'' that generates program code based on text input. The model is released under the same '

Llama 2 Community License ' as Llama 2, and can be used commercially for free if the number of monthly active users is 700 million or less.

Introducing Code Llama, a state-of-the-art large language model for coding
https://ai.meta.com/blog/code-llama-large-language-model-coding/



Code Llama is Llama 2, released by Meta in July 2023, further trained on code-specific datasets. It supports Python, C++, Java, PHP, Typescript & Javascript, C#, and Bash, and in addition to the function to generate continuation of code, it can also generate code based on input in natural language and generate explanations about code. Models with 7 billion, 13 billion, and 34 billion parameters were released in August 2023 .

On January 29, 2024, Meta released the “Code Llama 70B” model, which is based on the 70 billion parameter “Llama 2 70B” model and additionally trained with code-specific datasets. While the training data for other sized models is 500 billion tokens, the 70 billion parameter model was trained with 1 trillion tokens. Furthermore, in addition to the basic 'Code Llama' that is fine-tuned for each purpose, 'Code Llama - Instruct' specialized for the task of programming from natural language and 'Code Llama - Instruct' specialized for handling Python. A variant called ``Python'' is available.



The results of a performance comparison between the 70 billion parameter model measured using '

HumanEval ' and ' MBPP ' and other models are shown in the figure below. In addition to the Instruct model recording better performance than GPT-4, we can see that the performance improved steadily for all three variants.



Code Llama models are available by requesting on Meta and are also hosted on Hugging Face .

◆Forum now open
A forum related to this article has been set up on the GIGAZINE official Discord server . Anyone can write freely, so please feel free to comment! If you do not have a Discord account, please create one by referring to the article explaining how to create an account!

Discord | 'Have you tried the Llama language model? What do you think it can do?' | GIGAZINE
https://discord.com/channels/1037961069903216680/1201822215050641428

Related Posts:

in Software, Posted by log1d_ts