OpenAI has introduced the next-generation reasoning model series, o1, trained using reinforcement learning techniques. It can engage in extended chain-of-thought before answering, significantly improving its ability to solve complex problems. In various benchmarks, o1 has excelled in fields like science, mathematics, and programming, particularly approaching the level of a human PhD on graduate-level scientific problems while achieving breakthroughs in math competition performance.
Innovative Training Paradigm and Reasoning Mechanism
The o1 model differs from traditional large language models by introducing reinforcement learning to reward the correct reasoning process, rather than just the final answer. The model internally generates long chain-of-thought steps, simulating the human thought process for solving difficult problems. This approach gives o1 an advantage in handling multi-step reasoning tasks. Currently, available versions include o1-preview and the lower-cost o1-mini.
PhD-Level Performance in Science
On the GPQA graduate-level science benchmark, o1’s accuracy significantly surpasses previous models, with performance nearing or exceeding the average level of human PhD students on physics, chemistry, and biology problems. The tests demonstrate that o1 can effectively handle highly difficult scientific questions, and scientists have expressed that they are impressed with its performance.
Math Competitions and Benchmark Performance Boost
In the 2024 American Invitational Mathematics Examination (AIME), o1 correctly solved 83% of the problems, compared to only about 13% for GPT-4o. It also achieved high scores on math benchmarks like MATH, showing significant progress. Furthermore, o1 demonstrates strong performance on tasks related to the International Mathematical Olympiad qualification tests.
Programming and General Reasoning Abilities
o1 reached the 89th percentile in Codeforces programming competitions and set new records on general intelligence benchmarks like ARC-AGI. Compared to previous models, o1 has a clear advantage in coding tasks that require deep reasoning.
Model Release and Current Availability
The o1 series was officially released for preview on September 12, 2024, and is currently available to users via the ChatGPT and the API. The o1-mini focuses on STEM fields, reducing computational costs while maintaining high performance, thereby expanding its range of applications.
