The newly released Visual Reasoning Benchmark (VRB) demonstrates significant progress by Multimodal Large Language Models (MLLMs) on vision-language tasks. By evaluating models on real-world visual problems from primary school classrooms, the benchmark reveals improvements in their spatial and relational reasoning abilities. This evaluation provides a crucial reference for the application of AI in cross-domain fields like education.
VRB Benchmark Design and Test Content
The Visual Reasoning Benchmark (VRB) is a novel dataset designed to evaluate the ability of Multimodal Large Language Models (MLLMs) to solve real-world visual problems from classrooms. The benchmark consists of 701 multiple-choice questions sourced from primary school exams in Zambia and India, emphasizing minimal-text images to test spatial and relational reasoning. Tasks include analogical reasoning, spot the difference, completing linear and 2D patterns, matching processes, and diagrams, involving skills such as counting, scaling, coloring, line types, shapes, rotation, reflection, folding, shape reconstruction, and layering.
Model Performance Evaluation Results
Researchers evaluated 45 multimodal models, with accuracies ranging from 23% to 78%. Among them, Gemini-3.0 Flash ranked first with 78% accuracy, followed by Gemini-3.0 Pro at 76%. Proprietary models generally outperformed open-weight models. The open-weight model Kimi-K2.5 achieved a 60% score, while GLM-4.6V and QWEN3-VL scored around 45%. Smaller open models performed near the random guess level of 25%.
Analysis of Performance Improvement Over Time
The benchmark tracks progress through different time-stamped snapshots. From late 2024 to late 2025, the value frontier is shown to be continuously advancing. For example, at a price point of approximately $0.10 per million tokens, accuracy improves from about 30% to 38-39%, while higher-cost models reach about 60%. This indicates that visual reasoning capabilities are rapidly improving, reflecting the continuous optimization of multimodal AI in vision-language understanding.
Task Discrepancies and Practical Limitations
Models perform well on static skills like counting and scaling but face a ‘spatial ceiling’ on dynamic operations such as folding, reflection, and rotation, where accuracy drops significantly. The benchmark also notes that accuracy on noisy images decreases from 38% to 30%, highlighting their fragility in real-world classroom applications. This underscores the need for human supervision to prevent incorrect labeling and misleading results.
Prospects for Cross-Domain Applications
The release of VRB provides a minimum capability threshold for evaluation in the education sector and serves as a reference for other cross-domain visual reasoning applications. Although the current top scores are still below the threshold for practical classroom utility, the continuous progress suggests that multimodal AI holds promise for supporting learning and teaching in the future, driving the broader adoption of artificial intelligence technologies.
