SE-Blackboard: A Shared-State Architecture for Multi-Agent Software Engineering Pipelines
Open accessThe DOI matches its Crossref record; 27 of the 35 references it cites were found in a registry or an index.
Abstract
Multi-agent large language model (LLM) systems are increasingly used for software engineering tasks, yet the role of communication architecture in these systems remains poorly understood. Most existing approaches assume message passing, where agents exchange sequential natural-language summaries, without examining the information loss this assumption may introduce. This paper makes a methodological and diagnostic contribution to that question. We present SE-Blackboard, a modular framework that places message passing, a shared-state blackboard, and a hybrid design within an otherwise identical multi-agent pipeline, and we introduce the Information Fidelity Score (IFS), a stage-level metric that measures how well key technical entities from the original issue survive successive agent handoffs. On 50 SWE-bench Lite issues, the blackboard configuration improves Coder-stage IFS by 62% over message passing (0.586 vs. 0.362) and correct file targeting by 27.5 percentage points (82.6% vs. 55.1%). Once the correct file is identified, however, all configurations converge to a ~22% conditional resolve rate, revealing a downstream bottleneck in LLM diff generation. Building on these decompositions, we present an information-flow bottleneck model that predicts how upstream gains attenuate at the patch-generation stage; the model accounts for the modest end-to-end resolve-rate improvement observed (16% vs. 12%). These findings identify communication architecture as a measurable, separable design dimension and supply tools—IFS and the bottleneck model—for diagnosing where future improvements in multi-agent SE pipelines should be targeted.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Individual Comparisons by Ranking Methods
Wilcoxon, F. (1945) · Biometrics Bulletin · doi:10.2307/3001968 · Cited by 15,354
Note on the Sampling Error of the Difference Between Correlated Proportions or Percentages
McNemar, Q. (1947) · Psychometrika · doi:10.1007/bf02295996 · Cited by 4,754 · cited 1×
“Given the paired design (each issue under all three configurations), we use McNemar’s exact test [26] for resolve rate comparisons, Wilcoxon signed-rank test [27] for continuous measurements (tokens, latency, IFS), Cohen’s h for effect size, and odds ratio with 95% CI.”
Chain-Of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q. V., & Zhou, D. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1800 · Cited by 4,505 · cited 1×
“Chain-of-Thought prompting [19] supports stepwise reasoning, while ReAct [20] integrates reasoning with tool use, grounding agent decisions in environmental feedback.”
Generative Agents: Interactive Simulacra of Human Behavior
Park, J. S., O'Brien, J., Cai, C. J., Morris, M. R., Liang, P., & Bernstein, M. S. (2023) · Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3586183.3606763 · Cited by 2,187 · cited 1×
“Park et al. [22] further demonstrate that generative agents with memory and reflective mechanisms can display coherent collaborative behavior, providing a firm basis for more advanced coordination strategies.”
Lost in the Middle: How Language Models Use Long Contexts
Liu, N. F., Lin, K., Hewitt, J., Paranjape, A., Bevilacqua, M., Petroni, F., & Liang, P. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00638 · Cited by 1,612 · cited 2×
“When presented with this larger context—consistent with prior findings that LLMs struggle to make effective use of very long inputs [28]—the Coder often produces no diff output.”
Evaluating Large Language Models Trained on Code
Chen, M., Tworek, J., Jun, H., Yuan, Q., Pinto, H. P. D. O., Kaplan, J., Edwards, H., Burda, Y., Joseph, N., Brockman, G., et al. (2021) · arXiv · doi:10.48550/arxiv.2107.03374 · Cited by 1,436
The Hearsay-II Speech-Understanding System: Integrating Knowledge to Resolve Uncertainty
Erman, L. D., Hayes-Roth, F., Lesser, V. R., & Reddy, D. R. (1980) · ACM Computing Surveys · doi:10.1145/356810.356816 · Cited by 1,357 · cited 1×
“The Blackboard architecture, first introduced by Nii [9], describes a framework in which multiple knowledge sources coordinate through a shared workspace, which was initially applied to speech understanding and planning systems [23].”
Toolformer: Language Models Can Teach Themselves to Use Tools
Schick, T., Dwivedi-Yu, J., Dessì, R., Raileanu, R., Lomeli, M., Zettlemoyer, L., Cancedda, N., & Scialom, T. (2023) · arXiv · doi:10.48550/arxiv.2302.04761 · Cited by 1,150
A Survey on Software Fault Localization
Wong, W. E., Gao, R., Li, Y., Abreu, R., & Wotawa, F. (2016) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2016.2521368 · Cited by 1,110 · cited 1×
“Among non-empty patches, the BB Coder correctly identifies the file(s) that require modification—closely related software fault localization [29], [30]—in 82.6% of cases, compared with 55.1% for MP, corresponding to a 27.5 percentage-point improvement (Figure 2c).”
InfluentialReflexion: Language Agents with Verbal Reinforcement Learning
Shinn, N., Cassano, F., Berman, E., Gopinath, A., Narasimhan, K., & Yao, S. (2023) · arXiv · doi:10.48550/arxiv.2303.11366 · Cited by 1,087
Demystifying LLM-Based Software Engineering Agents
Xia, C. S., Deng, Y., Dunn, S., & Zhang, L. (2025) · Proceedings of the ACM on Software Engineering · doi:10.1145/3715754 · Cited by 849
Cognitive Architectures for Language Agents
Sumers, T. R., Yao, S., Narasimhan, K., & Griffiths, T. L. (2023) · arXiv · doi:10.48550/arxiv.2309.02427 · Cited by 798 · cited 1×
“…of a structured shared state with role-bound write permissions is consistent with recent cognitive-architecture treatments of language agents [24], which advocate explicit separation between working memory, reasoning state, and action production—a separation our S = ( I , A , P , R , T )…”
Code Llama: Open Foundation Models for Code
Rozière, B., Gehring, J., Gloeckle, F., Sootla, S., Gat, I., Tan, X. E., Adi, Y., Liu, J., Sauvestre, R., Remez, T., et al. (2023) · arXiv · doi:10.48550/arxiv.2308.12950 · Cited by 531 · cited 1×
“Recent studies have introduced a range of multi-agent software engineering (SE) systems that differ in architectural design and interaction style, which typically rely on the code-generation strengths of foundation models such as Codex [6] and Code Llama [12].”
ChatDev: Communicative Agents for Software Development
Qian, C., Liu, W., Liu, H., Chen, N., Dang, Y., Li, J., Yang, C., Chen, W., Su, Y., Cong, X., et al. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.810 · Cited by 490 · cited 2×
“Systems such as MetaGPT [2], ChatDev [3], SWE-Agent [4], and AgentCoder [5] demonstrate that multiple AI agents can collaboratively solve complex tasks, including code generation [6], bug fixing, and code review, representing a shift from monolithic tools toward modular, role-based architectures.”
SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering
Yang, J., Jimenez, C., Wettig, A., Lieret, K., Yao, S., Narasimhan, K., & Press, O. (2024) · Advances in Neural Information Processing Systems 37 · doi:10.52202/079017-1601 · Cited by 264
MAGIS: LLM-Based Multi-Agent Framework for GitHub Issue Resolution
Tao, W., Zhou, Y., Wang, Y., Zhang, W., Zhang, H., & Cheng, Y. (2024) · neural information processing systems · doi:10.52202/079017-1647 · Cited by 255
MemoryBank: Enhancing Large Language Models with Long-Term Memory
Zhong, W., Guo, L., Gao, Q., Ye, H., & Wang, Y. (2023) · arXiv · doi:10.48550/arxiv.2305.10250 · Cited by 253
He, J., Treude, C., & Lo, D. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3712003 · Cited by 246
LLM-Based Multi-Agent Systems for Software Engineering: Literature Review, Vision and the Road Ahead
He, J., Treude, C., & Lo, D. (2024) · arXiv · doi:10.48550/arxiv.2404.04834 · Cited by 246
AutoCodeRover: Autonomous Program Improvement
Zhang, Y., Ruan, H., Fan, Z., & Roychoudhury, A. (2024) · Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis · doi:10.1145/3650212.3680384 · Cited by 202 · cited 2×
“In contrast, tool-augmented approaches such as SWE-Agent [4] and AutoCodeRover [14] focus on enhancing a single agent with repository navigation and execution tools, achieving strong performance without explicit multi-agent coordination.”
AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation
Wu, Q., Bansal, G., Zhang, J., Wu, Y., Li, B., Zhu, E., Jiang, L., Zhang, X., Zhang, S., Liu, J., et al. (2023) · arXiv · doi:10.48550/arxiv.2308.08155 · Cited by 193
Huo, J., Yan, Y., Xu, Z., Lyu, Y., Zou, X. G., Wei, Z., & Hu, X. (2025) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2025.findings-acl.375 · Cited by 68
AgentCoder: Multi-Agent-based Code Generation with Iterative Testing and Optimisation
Huang, D., Zhang, J. M., Luck, M., Bu, Q., Qing, Y., & Cui, H. (2023) · arXiv · doi:10.48550/arxiv.2312.13010 · Cited by 34
RTADev: Intention Aligned Multi-Agent Framework for Software Development
Liu, J., Wang, G., Yang, R., Zeng, J., Zhao, M., & Cai, Y. (2025) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2025.findings-acl.80 · Cited by 18 · cited 2×
“More recently, RTADev [8] introduces intention alignment as a reactive mechanism that mitigates information loss after it occurs.”
CodeR: Issue Resolving with Multi-Agent and Task Graphs
Chen, D., Lin, S., Zeng, M., Zan, D., Wang, J.-G., Cheshkov, A., Sun, J., Yu, H., Dong, G., Aliev, A., et al. (2024) · arXiv · doi:10.48550/arxiv.2406.01304 · Cited by 8
LLM-Based Multi-Agent Blackboard System for Information Discovery in Data Science
Salemi, A., Parmar, M., Goyal, P., Song, Y., Yoon, J., Zamani, H., Pfister, T., & Palangi, H. (2025) · arXiv · doi:10.48550/arxiv.2510.01285 · Cited by 2
(1986) · doi:10.1609/aimag.v7i2.537
Language models are few-shot learners
Brown (n.d.)
UnresolvedMetaGPT: Meta programming for a multi-agent collaborative framework
Hong (n.d.)
UnresolvedWhy do multi-agent llm systems fail?
Cemri (n.d.)
UnresolvedReAct: Synergizing reasoning and acting in language models
Yao (n.d.)
UnresolvedSWE-bench: Can language models resolve real-world GitHub issues?
Jimenez (n.d.)
UnresolvedBoosting coverage-based fault localization via graph-based representation of test failures
Lou (n.d.)
UnresolvedRetrieval-augmented generation for knowledge-intensive NLP tasks
Lewis (n.d.)
UnresolvedImproving factuality and reasoning in language models through multiagent debate
Du (n.d.)
Unresolved
Metadata
- Authors
- Liu, E., Zhu, Q., & Dong, X.
- Affiliations
- New York University (US)
- National Institute of Metrology, China
- National Institute of Metrology (CN)
- Venue
- IEEE Access
- Published
- 2026
- Volume
- 14
- Pages
- 106477-106491
- ISSN
- 2169-3536 (ISSN-L)
- OpenAlex
- W7161804021
- Cited by
- 0
- References
- 34
- FWCI
- 0.00
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- IEEE Access (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Physical Sciences › Computer Science › Software › Advanced Software Engineering Methodologies
- Keywords
- multi-agent systems, large language models, message passing, information fidelity, information flow, patch generation, code generation, SWE-bench, communication architecture
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.