As Large Language Model (LLM) applications enter an era of multi-model collaboration, efficiently orchestrating different models to solve complex tasks has become a central challenge. Existing routing systems primarily focus on selecting the most suitable single model for a given query, lacking explicit modeling of the collaborative process even in multi-turn interactions. To address this, researchers at the University of Illinois Urbana-Champaign (UIUC) released a new framework in April 2024 called GraphPlanner, aiming to advance LLM routing from “model selection” to a new stage of “agentic workflow generation.”
From Model Selection to Workflow Generation: A New Paradigm for LLM Routing
Traditional LLM routing techniques, whether single-turn or multi-turn, have the core task of “model selection”—determining which LLM should handle a given query. This approach reveals significant limitations when dealing with complex problems that require multi-step, multi-role collaboration, as it cannot proactively decompose tasks or plan the collaborative structure between different models.
Complex real-world tasks, such as multi-hop reasoning or code generation, often require a process akin to an “agentic workflow”: a Planner first decomposes the problem, multiple Executors then process sub-tasks in parallel, and finally, a Summarizer integrates the results. GraphPlanner is built on this insight, proposing its core idea: a routing decision should not just be about selecting a model (LLM Backbone), but rather selecting a pair of (Action = Agent Role + LLM Backbone).
By pre-defining basic agent roles like Planner, Executor, and Summarizer, GraphPlanner upgrades the router’s function from a “model selector” to a “multi-agent planner.” It can dynamically generate customized workflows based on the query’s complexity, for example, by directly calling an Executor for simple questions or constructing a complete “Plan-Execute-Summarize” chain for complex ones.
GARNet: The Decision-Making Core Fusing Graph Memory and Role Division
Another key innovation of GraphPlanner is the introduction of a heterogeneous graph memory network called GARNet (Graph-based Agentic Routing Network). This network is designed to fully leverage the valuable data generated during the interaction process of the multi-agent system.
Every model call contains rich contextual information: the query content, the model being called, its assigned role, the output result, the computational cost, and the final success or failure of the task. GraphPlanner structures this information to build two types of graph memory:
- Workflow Memory Graph: Records the sub-problems, role calls, and intermediate results generated during the inference process for the current query, providing context for the current decision.
- Historical Memory Graph: Stores interaction records from all past tasks, including queries, responses, LLM-role pairings, accuracy rates, and costs, forming an experience repository.
GARNet uses a heterogeneous graph structure to link query nodes, response nodes, and LLM-role nodes, connecting the current workflow with historical memory through shared “role hub nodes.” This design allows GraphPlanner to not only perceive the current task state when making routing decisions but also to draw upon model capability profiles and efficient collaboration patterns accumulated in historical data, leading to better judgments.
Reinforcement Learning-Driven Dynamic Policies and Cost Optimization
Since the workflow generation process involves discrete decisions (such as selecting roles, calling models, and decomposing problems), traditional end-to-end gradient optimization methods are not applicable. Therefore, the research team modeled this problem as a Markov Decision Process (MDP) and used Proximal Policy Optimization (PPO), a reinforcement learning algorithm, for training.
The reward function is ingeniously designed to balance task effectiveness and computational cost. The system receives “task utility” for correct task completion, while each model call incurs a corresponding “computational cost.” A hyperparameter α allows for flexible adjustment of the trade-off between pursuing high accuracy and controlling costs. This mechanism trains not a set of fixed workflow templates, but a dynamic policy that can adaptively decide the planning depth, collaboration mode, and resource allocation for different queries.
Comprehensive Experimental Validation: Performance, Efficiency, and Generalization
GraphPlanner was systematically evaluated on 14 tasks across 6 domains, including mathematics, code, and commonsense reasoning. The experiments were conducted in two stages: the first stage focused on optimizing model selection within predefined workflows, while the second stage involved simultaneous workflow generation and model selection.
The results showed that GraphPlanner significantly outperformed various baseline methods, including Router-R1, in both stages. Particularly in the second stage, when the system was given the freedom to generate workflows, its advantage widened, with an average accuracy improvement of about 9.3% over the strongest baseline model. This demonstrates the superiority of dynamically generating workflows tailored to specific queries.
In terms of efficiency, GraphPlanner also excelled. Compared to other reinforcement learning-based multi-turn routing methods, it significantly reduced GPU computational requirements during training and achieved faster inference decisions through a lightweight graph policy network.
More importantly, GraphPlanner demonstrated strong generalization capabilities. It achieved an average accuracy of 78% on out-of-domain tasks unseen during training, far surpassing other compared models. Furthermore, when new, untrained LLMs were introduced into the system, GraphPlanner maintained robust performance. This indicates that it learns transferable collaboration patterns rather than just memorizing specifics for certain models or tasks. The framework also supports two deployment modes: a lightweight “Inductive Setting” and a higher-performance “Transductive Setting” to suit different application scenarios.
