Meta Open-Sources Llama 3.1 Model Series: 405B Version Outperforms Closed-Source Rivals on Multiple Benchmarks
On July 23, 2024, Meta released the Llama 3.1 series of open-source large language models, featuring 8B, 70B, and 405B parameter versions. The flagship 405B parameter model has surpassed closed-source competitors like OpenAI’s GPT-4o and Anthropic’s Claude 3.5 Sonnet on several authoritative benchmarks. This release marks a significant milestone, with open-source AI model performance approaching top-tier commercial levels.
Model Performance Surpasses Multiple Benchmarks
The Llama 3.1 405B model achieved leading scores on key benchmarks such as MMLU-Pro (86.5%), GPQA (51.1%), and MATH (73.8%), outperforming GPT-4o’s respective scores. It also scored 89.0% on the code generation task HumanEval and 84.0% on the instruction-following Arena Hard benchmark. These results are based on evaluation data released by Meta and have been preliminarily verified by platforms like Hugging Face and LMSYS. The model also supports a 128K token context length, making it suitable for processing complex, long texts.
Enhanced Multilingual and Tool-Use Capabilities
The new models support 8 languages, including English, Spanish, German, Italian, Portuguese, French, Thai, and Hindi, demonstrating excellent performance on multilingual benchmarks. For example, the 405B model achieved a 90.4% accuracy rate on the MGSM mathematical reasoning task. Additionally, Llama 3.1 introduces tool-calling capabilities, enabling it to interact with external APIs to perform tasks like mathematical calculations and file retrieval. Meta stated that the model series was optimized with 15 trillion tokens of training data, significantly improving its reasoning and safety.
Open-Source Strategy Promotes AI Democratization
The Llama 3.1 model series is now available for free download with a commercial license on the Meta AI website and the Hugging Face platform. Training the 405B model required 30.8×1024 FLOPs, equivalent to millions of GPU hours, but Meta’s open-source approach allows developers to access top-tier performance without incurring massive costs. Media outlets such as VentureBeat and TechCrunch reported that this move will accelerate global AI innovation and reduce dependency on closed-source models.
Community Reaction and Future Outlook
Following the release, the AI community responded rapidly, with Hugging Face reporting a surge in downloads. Meta emphasized that the models have built-in mechanisms to refuse harmful queries and has provided safety reports. In the future, Meta plans to expand the Llama series to support more languages and modalities. This open-source initiative is seen as a crucial step forward for open collaboration in the AI field.
