Meta releases AI model 'Llama 3.3', 70B model achieves performance comparable to Llama 3.1's 405B model

Meta released the large-scale language model ' Llama 3.3 ' on Saturday, December 6, 2024. At the time of writing, a model with 70B parameters had been released, and it is being promoted as being lower cost and more powerful than Amazon's base model '

As we continue to explore new post-training techniques, today we're releasing Llama 3.3 — a new open source model that delivers leading performance and quality across text-based use cases such as synthetic data generation at a fraction of the inference cost. pic.twitter.com/BNoV2czGKL

— AI at Meta (@AIatMeta) December 6, 2024

Llama 3.3 70B is trained on over 15 trillion tokens collected by December 2023. It has a context length of 128,000 and supports the following languages: English, German, French, Italian, Portuguese, Hindi, Spanish, and Thai.

Below is a table listing the benchmark results of Llama 3.3 70B and various AI models. Llama 3.3 70B outperforms Llama 3.1 70B in most tests, and exceeds the Amazon Nova Pro's score in 'MMLU' which evaluates overall performance and 'MATH' which evaluates mathematical performance. In addition, some benchmarks such as coding performance and mathematical performance have recorded scores that exceed those of the previous generation top model Llama 3.1 405B.

Meta claims that the Llama 3.3 70B delivers 'cost-efficient inference that can run locally on a typical developer workstation while delivering similar performance to the Llama 3.1 405B.'

Improvements in Llama 3.3 were driven by a new alignment process and progress in online RL techniques. This model delivers similar performance to Llama 3.1 405B with cost effective inference that's feasible to run locally on common developer workstations.

— AI at Meta (@AIatMeta) December 6, 2024

Llama 3.3 70B is available on the Meta official website and is also distributed by Hugging Face. The license is ' Llama 3.3 Community License ', and it can be used commercially if the number of monthly active users is less than 700 million.

meta-llama/Llama-3.3-70B-Instruct · Hugging Face
https://huggingface.co/meta-llama/Llama-3.3-70B-Instruct