Meta has officially open-sourced the Llama 3.1 series of large language models, including a 405B parameter version. This model series shows performance comparable to closed-source models like GPT-4o and Claude 3.5 Sonnet on multiple benchmarks, becoming the first frontier-level open-source AI model. This release supports multilingual applications and an extended context length, further driving the development of the open-source AI ecosystem.
Model Specifications and Release Information
Meta launched the Llama 3.1 model series on July 23, 2024. The series includes upgraded 8B and 70B models, as well as an all-new 405B parameter model. These are all pre-trained and instruction-fine-tuned generative text models, with the 405B model being Meta’s largest and most powerful open-source foundational model to date.
Open-Source License and Platform Support
The Llama 3.1 models are available under the Llama 3.1 Community License Agreement. Users can download the model weights from llama.meta.com and Hugging Face. The license permits the use of model outputs to improve other models. The models are supported for immediate development and deployment on partner platforms such as AWS, NVIDIA, and Databricks.
Technical Features and Capability Enhancements
The context length of the Llama 3.1 models has been extended to 128K tokens, with newly added support for eight languages. The new models excel in areas such as tool use, reasoning, general knowledge, controllability, math, and multilingual translation. They also support advanced capabilities like function calling, Retrieval-Augmented Generation (RAG), synthetic data generation, and model distillation.
Performance Benchmark Evaluation
Meta evaluated Llama 3.1 on over 150 benchmark datasets. The 405B model’s performance in key capabilities like general knowledge, controllability, math, tool use, and multilingual translation is comparable to top-tier closed-source models. The 8B and 70B models also show strong competitiveness within their respective parameter scales.
Real-World Availability and Ecosystem Advancement
Users can currently try the 405B model on the Meta AI platform and on WhatsApp (in the US). This open-source release provides global AI researchers and developers with frontier-level tools, poised to accelerate the development of new applications and innovation in modeling paradigms.
