Single-Process, Multi-Agent Concurrent Scheduling: A Lightweight Implementation Using Coroutines and Streams
As AI Agent technology advances, efficiently and securely running multiple agents within limited resources has become a key challenge. A recent technical exploration demonstrates how to build a lightweight framework for concurrent multi-agent scheduling within a single Deno/Node.js process, drawing inspiration from operating system principles. This method avoids the security vulnerabilities and resource overhead of multi-process models, offering a new approach to agent collaboration.
Transforming Agent Execution into Schedulable Units
Traditionally, an agent runs in a continuous loop (Agent Loop), which monopolizes the execution thread and prevents concurrency. While using the operating system’s multi-process (fork) model can directly leverage OS scheduling capabilities, granting a process the permission to start new processes (e.g., --allow-run) in modern JavaScript runtimes like Deno undermines its core sandboxing security mechanism, introducing potential risks.
To solve this, the core idea of this solution is to “fragment” the agent’s execution process. Specifically, it uses asynchronous generators (AsyncGenerator) to refactor the agent’s core loop. An AsyncGenerator allows a function to pause itself during execution via the yield keyword and cede control. By inserting yield after an agent performs an asynchronous operation (like calling a large model API or executing a tool), the originally continuous execution flow is broken down into a series of discrete, schedulable task units. This mechanism essentially implements coroutines or lightweight threads, making task switching possible within a single thread.
A Stream-Based Concurrent Scheduling Mechanism
To manage and execute the task units produced by AsyncGenerator, this solution cleverly uses Node.js’s Stream technology to build a scheduler. The specific steps are as follows:
- Create a Central Scheduling Stream: A
PassThrough stream is used as a central hub for all agent events. This stream runs in object mode (objectMode: true) to handle the state objects yielded by the agents.
- Dynamically Attach Agents: Each agent’s
AsyncGenerator instance is treated as a readable stream and connected to the central PassThrough stream via the .pipe() method. The key is to set the pipe option end to false. This prevents any single agent’s completion from closing the entire scheduling stream, thus allowing agents to be dynamically added (pipe) or removed (unpipe) at runtime.
- Event-Driven Scheduling Loop: The main program consumes events from the central stream using a
for await...of loop. Whenever an agent yields a state, that event appears in the stream, and the scheduler can process it. Thanks to Node.js’s event loop and asynchronous I/O mechanism, when one agent is paused waiting for a network request, events from other agents can be processed, thus achieving macroscopic concurrent execution. This scheduling strategy is essentially a First Come First Serve (FCFS) algorithm.
System-Level Abstractions Inspired by Operating System Design
To make the entire framework more structured and scalable, the solution introduces several OS-like abstract concepts:
init Agent: Similar to the init process in an operating system, the system first starts a root agent. This agent is responsible for receiving external commands (like user input) and forking new child agents as needed to handle specific tasks.
Terminal Interface: To enable user interaction, the system abstracts a terminal interface that includes methods for handling input and output. The terminal implementation, based on standard input/output streams, is also designed as an AsyncGenerator, allowing it to be seamlessly integrated into the unified scheduling loop. This abstraction paves the way for future expansion of interaction to other channels like instant messaging (IM).
System Calls: An agent’s fork operation is implemented as a special kind of “system call.” When an agent needs to create a child agent, it yields a specific system call event. The scheduler intercepts this event, pauses the parent agent, creates and starts the child agent, and handles the context passing between them. When the child agent completes its execution, the scheduler is responsible for returning its result to the parent agent and resuming the parent’s execution. This mechanism provides precise control over the agent lifecycle.
Practical Verification and Future Outlook
A concrete demonstration showed the system successfully receiving a command via the init agent to create a pdf_reader agent. This agent then used a specific tool to summarize a PDF document, validating the functional viability of the entire scheduling framework.
Despite this, this single-process concurrency model faces practical challenges. When multiple agents need to call a locally deployed large model simultaneously, they may still have to queue up if the model service does not support concurrent requests. Furthermore, if multiple agents rely on different large models, running them locally will lead to a linear increase in memory requirements. This raises a question worth further exploration: should the future agent ecosystem rely on a single, all-powerful, high-concurrency mega-model, or a network of lightweight, domain-specific expert models?
