Agensh: Scaling Organizational Intelligence to 1,024 Agents
Open accessAbstract
A multi-agent system can reduce latency on complex tasks by executing work concurrently. Several pioneering harness frameworks support multi-agent systems. However, the scalability of current multi-agent harnesses is often constrained by a central orchestrator's capacity to allocate tasks and coordinate workers. To address this limitation, we introduce Agensh, a scalable self-organized multi-agent harness without a central orchestrator: concurrent workers execute a multi-agent cooperation loop, continuously gathering context, claiming and self-assigning sub-tasks, taking action and sharing findings, verifying results, and merging progress in an asynchronous manner. The loop is supported by the agentic organization infrastructure comprising three components: a shared workspace holds proposed, ongoing, and completed work; a message interface lets workers communicate; and shared context retains reusable findings and work intentions. To test the scalability of Agensh, we evaluate it on the five hardest ProgramBench tasks with GPT-5.6-sol (high). Scaling from 1 to 128 agents raises the mean final test-pass rate from 19.31% to 28.78%, an approximately 49% relative improvement. Larger organizations reach comparable test-pass rates earlier. On pandoc, scaling from 1 to 1,024 agents raises the final test-pass rate from 33.89% to 55.06%. Worker trajectories further show that different forms of self-organized cooperation gradually emerges and standardizes as the organization grows. These results reveal the number of agents as a new scaling dimension for multi-agent organizations to expand the frontier of general intelligence, offering a practical solution for complex tasks under hard latency constraints or time budgets.
AI snapshot
Written from the paper’s full text.
TL;DR
Agensh is a self-organized multi-agent harness without a central orchestrator, using a shared workspace, message interface, and shared context to let concurrent workers claim, execute, verify, and merge sub-tasks. On the five hardest ProgramBench tasks, scaling from 1 to 128 agents raised mean final test-pass rate from 19.31% to 28.78%, and on pandoc scaling to 1,024 agents raised it from 33.89% to 55.06%.
Key findings
- Scaling from 1 to 128 agents on the five hardest ProgramBench tasks increased mean final test-pass rate from 19.31% to 28.78%, an approximately 49% relative improvement; intermediate means were 20.68% at 8 agents and 26.52% at 32 agents.
- On pandoc, scaling from 1 to 1,024 agents raised final test-pass rate from 33.89% to 55.06%; 128 agents reached 50.94%, so the 1,024-agent result improved on 128 agents by 4.12 percentage points and on 1 agent by 21.17 percentage points.
- Larger organizations reached comparable test-pass rates earlier: on pandoc, 128 agents exceeded a 30% test-pass rate at the 30-minute checkpoint, while 32 and 8 agents first exceeded it at 60 and 90 minutes, respectively, and the single-agent run remained below that threshold throughout the first two hours.
- Worker trajectories showed self-organized cooperation broadening with scale: peer coordination at 8 agents, multi-worker integration at 32 agents, workflow standardization and specialization at 128 agents, and role specialization at organization scale with 1,024 agents.
Methods
- The paper evaluates Agensh on ProgramBench, selecting the five hardest tasks by state-of-the-art model pass rate: FFmpeg, gromacs, pandoc, PHP-src, and ctags. Tasks require rebuilding reference software behavior from scratch under a 6-hour budget with Internet access disabled.
- All reported configurations use GPT-5.6-sol(high) with high reasoning effort and Copilot as the underlying single-agent harness; the main scaling experiment compares 1, 8, 32, and 128 agents across the five tasks, with an extension to 1,024 agents on pandoc.
- For the 1,024-worker experiments, workers are distributed across 16 nodes with 64 agents per node; agent activation is staggered, launching one agent every 30 seconds during the first hour and one every 3 seconds thereafter. The main reported metric is the canonical-kept final test-pass rate, with trajectories also analyzed for emergent cooperation.
Limitations
- The paper does not state a dedicated limitations section.
- Evident from the text: the main scaling comparison is limited to five ProgramBench tasks, and the 1,024-agent result is reported only on pandoc, not across all five tasks.
- Evident from the text: all reported configurations use one agent model, GPT-5.6-sol(high), and one underlying single-agent harness, Copilot, so generality across models and harnesses is not empirically established.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
0 of 31 cited sources checked · 0 supported · 0 flagged
- Findingown data
Worker trajectories reveal that the forms of self-organized cooperation broaden with scale, from peer coordination and multi-worker integration to standardized workflows and organization-scale role specialization.
“Worker trajectories further reveal the forms of selforganized cooperation broadening with scale, from peer coordination and multi-worker integration to standardized workflows and organization-scale role specialization.”
Quote in the paperNot a substantial claim - Findingown data
Workers can discover and resolve overlapping claims themselves.
“Workers can also discover and resolve overlapping claims themselves.”
- Also stated in: § 3.2 Self-Organized Cooperation Emerges at Scale · plain
“In FFmpeg, one worker changed its scope and took on complementary work after discussing the overlap issue with a peer.”
Quote in the paperSubstantial claim - Also stated in: § 3.2 Self-Organized Cooperation Emerges at Scale · plain
- Findingown data
Larger organizations reach comparable test-pass rates earlier.
“Larger organizations reach comparable test-pass rates earlier.”
Quote in the paperSubstantial claim - Findingown data
At 8 agents, workers can agree on a concrete technical interface and independently implement components that conform to it.
“8 agents: coordinating implementation with peers.Workers can agree on a concrete technical interface and independently implement components that conform to it.”
- Also stated in: § 3.2 Self-Organized Cooperation Emerges at Scale · plain
“In gromacs, workers announced a module interface and independently implemented command modules that followed it.”
Quote in the paperSubstantial claim - Also stated in: § 3.2 Self-Organized Cooperation Emerges at Scale · plain
- Findingown data
In pandoc, multiple workers served as integrators.
“In pandoc, multiple workers served as integrators.”
Quote in the paperNot a substantial claim
- Findingown data
Scaling from 1 to 128 agents raises the mean final test-pass rate across the five hardest ProgramBench tasks from 19.31% to 28.78%.
“Scaling from 1 to 128 agents raises the mean final test-pass rate from 19.31% to 28.78%”
- Also stated in: § 3.1 Main Results · plain
“the mean final test-pass rate across the five tasks increases from 19.31% with 1 agent to 20.68%, 26.52%, and 28.78% with 8, 32, and 128 agents, respectively.”
Quote in the paperSubstantial claim - Also stated in: § 3.1 Main Results · plain
- Findingown data
Different forms of self-organized cooperation gradually emerge and standardize as the organization grows.
“Worker trajectories further show that different forms of self-organized cooperation gradually emerges and standardizes as the organization grows.”
- Also stated in: § Concurrent write Asynchronous read · plain
“Recorded trajectories further reveal progressively broader forms of self-organized cooperation as the organization grows, from peer coordination and multi-worker integration to standardized workflows and specialized roles at organization scale.”
Quote in the paperNot a substantial claim - Also stated in: § Concurrent write Asynchronous read · plain
- Findingnone given
Workers specializing in the same technical area can recover and take over work after another worker's attempt fails.
“Workers specializing in the same technical area can also recover and take over the work after another worker's attempt fails.”
Quote in the paperSubstantial claim - Findingown data
With the same agent model GPT-5.6-sol(high) and a 6-hour budget, scaling from 1 to 128 agents on the five hardest ProgramBench tasks raises the mean final test-pass rate from 19.31% to 28.78%, an approximately 49% relative improvement.
“In our experiments, with the same agent model GPT-5.6-sol(high) and 6h budget, scaling from 1 to 128 agents on the five hardest ProgramBench tasks raises the mean final test-pass rate from 19.31% to 28.78%, an approximately 49% relative improvement.”
Quote in the paperSubstantial claim- Rests on an assumption: A single-agent run can rarely sustain the full 6-hour runtime.
- Findingown data
On pandoc, scaling from 1 to 1,024 agents raises the final test-pass rate from 33.89% to 55.06%.
“On pandoc, scaling from 1 to 1,024 agents raises the final test-pass rate from 33.89% to 55.06%.”
- Also stated in: § Abstract · plain
“Final test-pass rate rises from 33.89% for 1 agent to 55.06% for 1,024 agents.”
Quote in the paperSubstantial claim - Also stated in: § Abstract · plain
- Findingown data
The reference repositories of the five tasks contain thousands of files and hundreds of thousands to millions of lines of code.
“Their reference repositories contain thousands of files and hundreds of thousands to millions of lines of code, as collected in Table 1”
Quote in the paperSubstantial claim - Findingown data
The relative improvement in mean final test-pass rate from 1 to 128 agents is approximately 49%.
“an approximately 49% relative improvement”
- Also stated in: § 3.1 Main Results · plain
“Scaling from 1 to 128 agents yields a gain of 9.47 percentage points, or an approximately 49% relative improvement in the average score.”
Quote in the paperSubstantial claim - Also stated in: § 3.1 Main Results · plain
- Findingown data
In pandoc, scaling to 1,024 agents raises the final test-pass rate from 33.89% with 1 agent to 55.06%.
“In pandoc, scaling to 1,024 agents raises the final testpass rate from 33.89% with 1 agent to 55.06%.”
- Also stated in: § Agent configuration. · plain
“Under the same 6h budget, the final test-pass rate rises from 33.89% with 1 agent to 50.94% with 128 agents and 55.06% with 1,024 agents.”
Quote in the paperSubstantial claim- Rests on an assumption: A single-agent run can rarely sustain the full 6-hour runtime.
- Also stated in: § Agent configuration. · plain
- Findingown data
Across the five tasks, final scores show a generally increasing trend as the agent organization grows.
“Across the five tasks, final scores show a generally increasing trend as the agent organization grows”
Quote in the paperSubstantial claim - Findingown data
At 128 agents, workers can transfer responsibility for integrating a contribution to a peer who resolves conflicts, validates the combined work, and merges it.
“They can also transfer responsibility for integrating a contribution to a peer who resolves conflicts, validates the combined work, and merges it.”
Quote in the paperSubstantial claim - Findingown data
The single-agent run remains below a 30% test-pass rate throughout the first two hours on pandoc.
“The single-agent run remains below this threshold throughout the first two hours.”
Quote in the paperSubstantial claim- Rests on an assumption: A single-agent run can rarely sustain the full 6-hour runtime.
- Interpretationown data
As the organization grows, cooperation becomes broader in scope, extending from coordinating implementation to managing integration, standardizing workflows, and developing specialized roles at organization scale.
“As the organization grows, cooperation becomes broader in scope, extending from coordinating implementation to managing integration, standardizing workflows, and developing specialized roles at organization scale.”
Quote in the paperSubstantial claim - Interpretationown data
Continuously increasing the number of cooperating workers can consistently and substantially improve the quality of complex software reproduction over long time horizons.
“indicating that continuously increasing the number of cooperating workers can consistently and substantially improve the quality of complex software reproduction over long time horizons.”
Quote in the paperSubstantial claim - Premisecitation
To overcome the constraints of single-agent systems, we need to scale the number of agents in multi-agent systems.
“When we want to overcome these constraints, we need to scale the number of agents in multi-agent systems [4].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [4]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Interpretationown data
The gains across five demanding tasks, together with the extension to 1,024 agents on pandoc, support agent count as a new scaling dimension for multi-agent organizations.
“The gains across five demanding tasks, together with the extension to 1,024 agents on pandoc, support agent count as a new scaling dimension for multi-agent organizations.”
Quote in the paperSubstantial claim - Interpretationreasoning
The Agensh design enables individual progress to accumulate into shared work and knowledge without relying on a central orchestrator to allocate tasks and manage workers.
“This design enables individual progress to accumulate into shared work and knowledge without relying on a central orchestrator to allocate tasks and manage workers.”
Quote in the paperSubstantial claim - Interpretationown data
The number of agents is a new scaling dimension for multi-agent organizations.
“These results demonstrate the number of agents as a new scaling dimension for multi-agent organizations.”
Quote in the paperSubstantial claim- Rests on an assumption: A single-agent run can rarely sustain the full 6-hour runtime.
- Interpretationreasoning
This recovery behavior strengthens the organization's robustness by avoiding dependence on any individual worker.
“This strengthens the organization's robustness by avoiding dependence on any individual worker.”
Quote in the paperSubstantial claim - Interpretationown data
Organizational intelligence can grow through scaling self-organized cooperation on complex, long-horizon work.
“They provide evidence that organizational intelligence can grow through scaling self-organized cooperation on complex, long-horizon work.”
Quote in the paperSubstantial claim- Rests on an assumption: Executing work concurrently in a multi-agent system can reduce latency on complex tasks.
- Premisecitation
Current multi-agent harnesses' scalability is often constrained by a central orchestrator's capacity to allocate tasks and coordinate workers.
“However, the scalability of current multi-agent harnesses is often constrained by a central orchestrator's capacity to allocate tasks and coordinate workers.”
- Also stated in: § 1 Introduction · plain
“However, the scalability of the overall system remains fundamentally limited by the orchestrator's capacity to manage and coordinate its workers and integrate their contributions [10,11,12].”
Quote in the paperSubstantial claim0 of 3 sources checked- cites [10]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [11]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [12]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Also stated in: § 1 Introduction · plain
- Interpretationreasoning
Agensh can connect to different underlying harnesses, such as Claude Code and Copilot, through lightweight harness-specific adapters without changing the cooperation loop, the shared services, or the agentic loop of the underlying harness.
“Therefore, Agens can connect to different underlying harnesses, such as Claude Code and Copilot, through lightweight harness-specific adapters without changing the cooperation loop, the shared services, or the agentic loop of the underlying harness.”
Quote in the paperSubstantial claim - Interpretationreasoning
This further illustrates the scaling potential of self-organized multi-agent systems.
“This further illustrates the scaling potential of self-organized multi-agent systems.”
Quote in the paperSubstantial claim - Interpretationreasoning
These tasks are a demanding test of long-horizon multi-agent cooperation.
“making them a demanding test of long-horizon multi-agent cooperation.”
Quote in the paperSubstantial claim - Interpretationown data
Agent count is a new scaling dimension for multi-agent organizations.
“These results reveal the number of agents as a new scaling dimension for multi-agent organizations to expand the frontier of general intelligence”
- Also stated in: § 3.1 Main Results · boosted
“Figure 5 demonstrates that agent count is a scaling dimension for multi-agent organizations.”
Quote in the paperSubstantial claim- Rests on an assumption: Executing work concurrently in a multi-agent system can reduce latency on complex tasks.
- Also stated in: § 3.1 Main Results · boosted
- Premisecitation
The five selected ProgramBench tasks are the five most difficult among ProgramBench's 200 instances, as measured by the mean test-pass rate of state-of-the-art models.
“These are the five most difficult tasks among ProgramBench's 200 instances, as measured by the mean testpass rate of state-of-the-art models.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [13]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Interpretationown data
Agensh continuously scales self-organized cooperation among more than a thousand agents.
“These results show that Agensh continuously scales self-organized cooperation among more than a thousand agents.”
Quote in the paperSubstantial claim- Rests on an assumption: A single-agent run can rarely sustain the full 6-hour runtime.
- May combine two claims. 26
- The quote only partly states this claim. 2
- Classified as probably not a substantial claim. 25
May combine two claims.
Findingown dataWorkers can negotiate, follow, and reuse standardized self-organized workflows.
“Workers can negotiate, follow, and reuse standardized self-organized workflows.”
- Also stated in: § 3.2 Self-Organized Cooperation Emerges at Scale · plain
“In pandoc, two workers established a standardized integration protocol: the author updated and tested a branch, then sent its commit hash for peer validation and merge.This protocol was later reused by other workers.”
Quote in the paperSubstantial claim- Also stated in: § 3.2 Self-Organized Cooperation Emerges at Scale · plain
May combine two claims.
Findingown dataAt 1,024 agents, multiple workers take on the same specialized roles or develop expertise in the same technical area.
“1,024 agents: role specialization at organization scale.At this scale, multiple workers take on the same specialized roles or develop expertise in the same technical area.”
Quote in the paperSubstantial claimMay combine two claims.
Findingnone givenA worker could contact several candidate integrators, select the first valid responder, cancel the other requests, and hand over the code to be integrated only after this selection.
“A worker could contact several candidate integrators, select the first valid responder, cancel the other requests, and hand over the code to be integrated only after this selection.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataAt 128 agents, self-organized specialization emerges, with workers choosing reviewers based on relevant prior experience and reusing review relationships over time.
“128 agents: self-organized specialization and workflow standardization.Specialization emerges at this scale.Workers are choosing reviewers based on relevant prior experience and reuse these review relationships over time.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataLarger organizations can reach comparable test-pass rates earlier, reducing the latency to a given level of performance.
“Larger organizations can reach comparable test-pass rates earlier, reducing the latency to a given level of performance.”
- Also stated in: § Agent configuration. · plain
“During the first two hours, larger organizations reach comparable test-pass rates earlier.”
Quote in the paperSubstantial claim- Rests on an assumption: A single-agent run can rarely sustain the full 6-hour runtime.
- Also stated in: § Agent configuration. · plain
May combine two claims. The quote only partly states this claim.
Findingown dataAt 32 agents, multiple workers can jointly work on a technical contribution, with peers reviewing and merging it.
“32 agents: managing integration across multiple workers.Multiple workers can jointly work on a technical contribution.”
- Also stated in: § 3.2 Self-Organized Cooperation Emerges at Scale · plain
“In PHP-src, several peers initially approved a contribution.Another worker found a concrete counterexample, so the earlier approval was withdrawn, and the author fixed the problem.Peers then reviewed the contribution again, and a peer merged it.”
Quote in the paperSubstantial claim- Also stated in: § 3.2 Self-Organized Cooperation Emerges at Scale · plain
May combine two claims.
Findingown dataAt 32 agents, workers became more active in managing integration, and a broader set of peers participated in the communication.
“Workers also became more active in managing integration, and a broader set of peers participated in the communication.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataAfter some failures, workers revised the cooperation protocol to give a peer permission to perform the entire update, test, check, and merge cycle, and peers accepted and carried out this revised procedure for new pull requests.
“After some failures, workers further revised the cooperation protocol: they agreed to give the peer permission to perform the entire update, test, check, and merge cycle.The peers explicitly accepted and carried out this revised procedure for new pull requests.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataThe largest organization improves on the 128-agent result by 4.12 percentage points and the single-agent result by 21.17 percentage points.
“The largest organization improves on the 128-agent result by 4.12 percentage points and the single-agent result by 21.17 percentage points.”
Quote in the paperSubstantial claim- Rests on an assumption: A single-agent run can rarely sustain the full 6-hour runtime.
May combine two claims.
Findingown dataOn pandoc, 128 agents exceed a 30% test-pass rate at the 30-minute checkpoint, while 32 and 8 agents first exceed that threshold at the 60-and 90-minute checkpoints, respectively.
“For example, on pandoc, 128 agents exceed a 30% test-pass rate at the 30-minute checkpoint, while 32 and 8 agents first exceed that threshold at the 60-and 90-minute checkpoints, respectively.”
Quote in the paperSubstantial claim- Rests on an assumption: A single-agent run can rarely sustain the full 6-hour runtime.
Probably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductcitationThe five hardest tasks were selected based on the official ProgramBench extended results, where task difficulty is determined by the average pass rate of state-of-the-art models.
“Task selection.We select the five hardest tasks based on the official ProgramBench extended results at https://programbench.com/extended/, where task difficulty is determined by the average pass rate of state-of-the-art models.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites https://programbench.com/extended/
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationThe shared context idea is adopted from DeLM.
“The idea is adopted from DeLM [14].”
- Also stated in: § 2.3 Implementation · plain
“The shared context follows the core idea of DeLM [14], but we adapt its tool formats and worker instructions.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [14]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Also stated in: § 2.3 Implementation · plain
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationSTORM reframes multi-agent coding as a state-management problem, replacing isolated multi-agent worktrees with a shared workspace that enforces local state consistency and detects stale or conflicting edits at write time.
“STORM reframes multi-agent coding as a state-management problem, replacing isolated multi-agent worktrees with a shared workspace that enforces local state consistency and detects stale or conflicting edits at write time [24].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [24]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Classified as probably not a substantial claim.
PremisecitationThese frameworks widely adopt an orchestrator-worker structure, in which a main agent plans, decomposes tasks, assigns them to concurrent workers, and manages them.
“These frameworks widely adopt an orchestrator-worker structure, in which a main agent plans, decomposes tasks, assigns them to concurrent workers, and manages them.”
Quote in the paperSubstantial claim0 of 5 sources checked- cites [5]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [6]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [7]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [8]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [9]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Classified as probably not a substantial claim.
PremisecitationProgramBench requires agents to reconstruct reference software's behavior from scratch given a 6h budget, with Internet access disabled.
“ProgramBench requires agents to reconstruct reference software's behavior from scratch given a 6h budget, with Internet access disabled to prevent retrieval of existing implementations.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [13]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
May combine two claims.
Interpretationown dataThese forms of cooperation are entirely self-organized: workers themselves choose collaborators, divide responsibilities, and establish and revise their workflows through peer interactions.
“Surprisingly, these forms of cooperation are entirely self-organized: workers themselves choose collaborators, divide responsibilities, and establish and revise their workflows through peer interactions.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationCarlini's compiler-building experiment coordinates 16 Claude agents through a Git repository and task-lock files, with workers selecting tasks and merging changes without an orchestration agent.
“For the shared workspace, Carlini's compiler-building experiment coordinates 16 Claude agents through a Git repository and task-lock files, with workers selecting tasks and merging changes without an orchestration agent [23].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [23]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
May combine two claims.
Interpretationnone givenThrough a shared workspace, a message interface, and shared context, self-organized workers discover and claim sub-tasks, exchange findings, and integrate contributions while proceeding concurrently and asynchronously.
“Through a shared workspace, a message interface, and shared context, self-organized workers discover and claim sub-tasks, exchange findings, and integrate contributions while proceeding concurrently and asynchronously.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationClaude Code supports persistent sub-agents and asynchronous agent teams, but the collaboration is still organized around a fixed lead session that spawns other agents, manages shared tasks, and coordinates with inter-agent messages.
“Claude Code supports persistent sub-agents and asynchronous agent teams, but the collaboration is still organized around a fixed lead session that spawns other agents, manages shared tasks, and coordinates with inter-agent messages [6,7].”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites [6]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [7]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Classified as probably not a substantial claim.
InterpretationreasoningThese observations complement the scaling results by showing how new forms of cooperation emerge while the number of agents scales.
“These observations complement the scaling results by showing how new forms of cooperation emerge while the number of agents scales.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S4 (based).
InterpretationreasoningThe message interface allows workers to resolve overlapping claims, negotiate dependencies, and request assistance in a timely manner while their peers continue to work.
“These functions allow workers to resolve overlapping claims, negotiate dependencies, and request assistance in a timely manner while their peers continue to work.”
Quote in the paperNot a substantial claim- Rests on an assumption: Workers in a multi-agent organization need to access one another's work, communicate about their sub-tasks, and retain findings to guide their next steps.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationSlackAgents embeds AI agents into Slack workspaces for communication and task orchestration.
“SlackAgents embeds AI agents into Slack workspaces for communication and task orchestration [22].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [22]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Classified as probably not a substantial claim.
PremisecitationSingle-agent systems allow large language models to perceive context, take actions through tools, update state, and iteratively progress toward a goal.
“Single-agent systems allow large language models to perceive context, take actions through tools, update state, and iteratively progress toward a goal [2,3].”
Quote in the paperSubstantial claim0 of 2 sources checked- cites [2]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [3]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
May combine two claims. The quote only partly states this claim.
InterpretationreasoningAgensh's multi-agent cooperation loop and agentic organization infrastructure enable workers to self-organize sub-task discovery and allocation.
“By proposing and claiming their own sub-tasks, workers self-organize sub-task discovery and allocation across the organization.”
- Also stated in: § 1 Introduction · plain
“Overall, the multi-agent cooperation loop enables workers to turn individual progress into collected contributions, while the infrastructure makes accumulated work, findings, and messages accessible throughout the organization.”
Quote in the paperSubstantial claim- Rests on an assumption: Workers in a multi-agent organization need to access one another's work, communicate about their sub-tasks, and retain findings to guide their next steps.
- Also stated in: § 1 Introduction · plain
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationKimi Agent Swarm trains an orchestrator to decompose tasks and schedule sub-agents concurrently for lower latency, but the harness remains an orchestrator-worker parallelization scheme.
“Kimi Agent Swarm trains an orchestrator to decompose tasks and schedule sub-agents concurrently for lower latency, but the harness remains an orchestrator-worker parallelization scheme [9].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [9]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Classified as probably not a substantial claim.
PremisecitationThis orchestrator-worker structure usually requires a trained orchestrator to plan and decompose tasks effectively.
“This structure usually requires a trained orchestrator to plan and decompose tasks effectively [9].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [9]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationChatCollab places human and AI participants in Slack as peers that coordinate roles, requests, and progress.
“ChatCollab places human and AI participants in Slack as peers that coordinate roles, requests, and progress [21].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [21]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationCodex supports sub-agent workflows with delegated parallel workers whose results are routed back to the parent for consolidation.
“Codex supports sub-agent workflows with delegated parallel workers whose results are routed back to the parent for consolidation [5].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [5]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductreasoningInternet access was disabled during ProgramBench runs to prevent retrieval of existing implementations.
“ProgramBench requires agents to reconstruct reference software's behavior from scratch given a 6h budget, with Internet access disabled to prevent retrieval of existing implementations.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim. May combine two claims.
InterpretationreasoningThe single-agent harness determines how one worker reasons and acts, while Agensh determines how many workers receive events, share state, coordinate, and repeatedly contribute to a shared goal.
“Thus, the single-agent harness determines how one worker reasons and acts, while Agensh determines how many workers receive events, share state, coordinate, and repeatedly contribute to a shared goal.”
Quote in the paperSubstantial claimMay combine two claims.
Interpretationnone givenAgensh is a scalable self-organized multi-agent harness without a central orchestrator.
“we introduce Agensh, a scalable self-organized multi-agent harness without a central orchestrator”
- Also stated in: § 1 Introduction · plain
“we introduce Agensh, a scalable multi-agent organization harness without a central orchestrator.”
Quote in the paperSubstantial claim- Also stated in: § 1 Introduction · plain
May combine two claims.
InterpretationreasoningA worker can reuse a peer's findings, address sub-task overlapping, and make an integrated contribution within its own cooperation loop.
“A worker can reuse a peer's findings, address sub-task overlapping, and make an integrated contribution within its own cooperation loop.”
Quote in the paperSubstantial claim- Rests on an assumption: Workers in a multi-agent organization need to access one another's work, communicate about their sub-tasks, and retain findings to guide their next steps.
May combine two claims.
Interpretationown dataIncreasing the number of workers can improve both the quality and speed of software reproduction, with the underlying model and harness held fixed.
“With the underlying model and harness held fixed, these experiments show that increasing the number of workers can improve both the quality and speed of software reproduction.”
Quote in the paperSubstantial claim- Rests on an assumption: Executing work concurrently in a multi-agent system can reduce latency on complex tasks.
Classified as probably not a substantial claim.
InterpretationreasoningThe interface between the single-agent agentic loop and the multi-agent cooperation loop is intentionally kept minimal and plug-and-play.
“The interface between the single-agent agentic loop and the multi-agent cooperation loop is intentionally kept minimal and plug-and-play.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationSeveral pioneering harness frameworks support multi-agent systems, including Codex sub-agent, Claude Code sub-agent and agent teams, Copilot fleet, and Kimi Agent Swarm.
“Several pioneering harness frameworks support multi-agent systems, including Codex sub-agent, Claude Code sub-agent and agent teams, Copilot fleet, and Kimi Agent Swarm [5,6,7,8,9].”
Quote in the paperNot a substantial claim0 of 5 sources checked- cites [5]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [6]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [7]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [8]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [9]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
May combine two claims.
InterpretationreasoningThe multi-agent cooperation loop enables workers to turn individual progress into collected contributions, while the infrastructure makes accumulated work, findings, and messages accessible throughout the organization.
“Overall, the multi-agent cooperation loop enables workers to turn individual progress into collected contributions, while the infrastructure makes accumulated work, findings, and messages accessible throughout the organization.”
Quote in the paperSubstantial claim- Rests on an assumption: Workers in a multi-agent organization need to access one another's work, communicate about their sub-tasks, and retain findings to guide their next steps.
Classified as probably not a substantial claim.
PremisecitationDeLM combines asynchronous workers, a task queue, and shared context so that agents can build on accumulated progress without routing every update through a central controller.
“DeLM combines asynchronous workers, a task queue, and shared context so that agents can build on accumulated progress without routing every update through a central controller [14].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [14]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Classified as probably not a substantial claim.
Interpretationnone givenAgensh is a scalable multi-agent organization harness that couples a multi-agent cooperation loop with lightweight agentic organization infrastructure.
“We introduced Agensh, a scalable multi-agent organization harness that couples a multi-agent cooperation loop with lightweight agentic organization infrastructure.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim. May combine two claims.
Study design and conductreasoningAll workers follow the same loop and receive the same prompt except for worker IDs.
“All workers follow the same loop and receive the same prompt except for worker IDs”
- Also stated in: § 2.3 Implementation · plain
“The complete worker prompt is presented in Appendix A, which is exactly the same for each worker except for the worker ID.”
Quote in the paperSubstantial claim- Also stated in: § 2.3 Implementation · plain
May combine two claims.
Study design and conductown dataThe scaling experiments held the underlying agent model (GPT-5.6-sol(high)), single-agent harness (Copilot), and 6h budget fixed across agent counts.
“With the same agent model, GPT-5.6-sol(high), underlying single-agent harness, Copilot, and 6h budget, the mean final test-pass rate across the five tasks increases from 19.31% with 1 agent to 20.68%, 26.52%, and 28.78% with 8, 32, and 128 agents, respectively.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim. May combine two claims.
Study design and conductnone givenAll reported configurations use gpt-5.6-sol with high reasoning effort and the same harness, worker protocol, and evaluation configuration.
“All reported configurations use gpt-5.6-solwith high reasoning effort and the same harness, worker protocol, and evaluation configuration.”
Quote in the paperSubstantial claimMay combine two claims.
Interpretationown dataIncreasing agent count can shorten the time needed to reach a given level of task performance, in addition to improving the final score.
“Increasing agent count can therefore shorten the time needed to reach a given level of task performance, in addition to improving the final score.”
Quote in the paperSubstantial claim- Rests on an assumption: A single-agent run can rarely sustain the full 6-hour runtime.
Classified as probably not a substantial claim. May combine two claims.
Study design and conductcitationThe experiment setup follows ProgramBench to prepare agent containers, set the total time budget to 6 hours, run the official evaluation, and report the canonical-kept pass rate.
“Experiment setup.We follow ProgramBench to prepare the agent containers, set the total time budget to 6 hours, run the official evaluation, and report the canonical-kept pass rate.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites ProgramBench
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Classified as probably not a substantial claim.
Study design and conductnone givenTo reduce scope contention, agent activation is staggered in all experiments, launching one agent every 30 seconds during the first hour and one every 3 seconds thereafter.
“Large-scale deployment.To reduce scope contention, we stagger agent activation in all experiments, launching one agent every 30 seconds during the first hour and one every 3 seconds thereafter.”
Quote in the paperSubstantial claim
The paper relies on this without support.
Executing work concurrently in a multi-agent system can reduce latency on complex tasks.
“A multi-agent system can reduce latency on complex tasks by executing work concurrently.”
The paper relies on this without support.
Single-agent systems' capability is bounded by one context window, one action stream, and one memory stream.
“However, their capability is bounded by one context window, one action stream, and one memory stream.”
The paper relies on this without support.
The sequential execution constraint imposes high latency on complex real-world tasks.
“The sequential execution constraint imposes high latency on complex real-world tasks.”
The paper relies on this without support.
Workers in a multi-agent organization need to access one another's work, communicate about their sub-tasks, and retain findings to guide their next steps.
“Workers in a multi-agent organization need to access one another's work, communicate about their sub-tasks, and retain findings to guide their next steps.”
The paper relies on this without support.
A single-agent run can rarely sustain the full 6-hour runtime.
“Since in practice, a single-agent run can rarely sustain the full 6-hour runtime, we add a stop hook to the single-agent baseline with the following prompt:”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Ctags citation points to PHP
MinorThe task list cites [20] for ctags, but reference 20 is the PHP source repository.
“We test Agensh on five particularly demanding systems: FFmpeg [17], gromacs [18], pandoc [1], PHP-src [19], and ctags [20].”
“20. The PHP Group. PHP source repository.”
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 4
- Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: Language agents with verbal reinforcement learning. In Advances in Neural Information Processing Systems, volume 36, pages 8634-8652.
- Curran Associates, Inc., 2023. doi: 10.52202/075280-0377. URL https://papers.nips.cc/paper_files/paper/ 2023/hash/1b44b878bb782e6954cd888628510e90-Abstract-Conference.html. · DOI · Paper page
- Yingxuan Yang, Huacan Chai, Shuai Shao, Yuanyi Song, Siyuan Qi, Renting Rui, and Weinan Zhang. AgentNet: Decentralized evolutionary coordination for LLM- based multi-agent systems. In Advances in Neural Information Processing Sys- tems, volume 38, pages 107309-107336.
- Curran Associates, Inc., 2025. doi: 10. 52202/085713-3578. URL https://papers.nips.cc/paper_files/paper/2025/hash/ 9a379c1b05793d1c42dc832269834515-Abstract-Conference.html. · DOI
- Authors
- Zhan, Z., Song, T., Dong, L., Huang, S., Lian, J., Xia, Y., & Wei, F.
- Affiliations
- Microsoft Research (United Kingdom) (GB)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2026
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2609.26781
- OpenAlex
- W7214087482
- Cited by
- 0
- References
- 0
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Multi-Agent Systems and Negotiation
- Physical Sciences › Computer Science › Hardware and Architecture › Parallel Computing and Optimization Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
- Keywords
- multi-agent systems, self-organization, agent harnesses, scalability, decentralized orchestration, task allocation, coworking spaces, message passing, shared context, pandoc, GPT-5, large language models, general intelligence, worker trajectories
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-29.
Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-29.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
