CoChat

MeshAgent: Enabling Reliable Network Management with Large Language Models

Open access
Zhou, Yajie, Hsieh, Kevin, Mani, Sathiya Kumaran, Kandula, Srikanth, Liu, Zaoxing · 2025 · Proceedings of the ACM on Measurement and Analysis of Computing Systems

The DOI matches its Crossref record; 63 of the 63 references it cites were found in a registry or an index.

Abstract

The emergence of large language models (LLMs) offers great promise for building domain-specific agents, but adapting them for network management remains challenging. To understand why, we conduct a case study on network management tasks and find that state-of-the-art specialization techniques rely heavily on extensive, high-quality task-specific data to produce precise solutions. However, real-world network queries are often diverse and unpredictable, making such techniques difficult to scale. Motivated by this gap, we propose MeshAgent, a new workflow that improves precision by extracting domain-specific invariants from sample queries and encoding them as constraints. These constraints guide LLM's generation and validation process, narrowing the search space and enabling low-effort adaptation. We evaluate our method across three network management applications and a user study involving industrial network professionals, showing that it complements existing techniques and consistently improves accuracy. We also introduce reliability metrics and demonstrate that our system is more dependable, with the ability to abstain when confidence is low. Overall, our results show that MeshAgent achieves over 95% accuracy, reaching 100% when paired with fine-tuned agents, and improves accuracy by up to 26% compared to baseline methods. The extraction of reusable invariants provides a practical and scalable alternative to traditional LLM specialization, enabling the development of more reliable agents for real-world network management.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3771567

Cites

63 of 63 references verified0 citation errors · 0 not checkable

Statistics

Figures

Fig. 1 Example input and output for a datacenter capacity planning query.
Fig. 1 Example input and output for a datacenter capacity planning query.CC-BY · From the PDF
Image 2
CC-BY · From the PDF
Fig. 2 Fine-tuned models perform poorly on dissimilar queries in the MALT application, revealing limited generalization beyond training examples.
Fig. 2 Fine-tuned models perform poorly on dissimilar queries in the MALT application, revealing limited generalization beyond training examples.CC-BY · From the PDF
Fig. 3 Semantic similarity between 200 user-submitted and 30 pre-sampled traffic analysis queries shows a long tail of dissimilar cases, highlighting the unpredictability of real-world tasks.
Fig. 3 Semantic similarity between 200 user-submitted and 30 pre-sampled traffic analysis queries shows a long tail of dissimilar cases, highlighting the unpredictability of real-world tasks.CC-BY · From the PDF
Fig. 4 Instead of relying on prompting or fine-tuning with numerous examples, our framework specializes LLM agents using reusable constraints. This approach offers greater generalization, improved reliability, and reduced human effort for real-world network management tasks.
Fig. 4 Instead of relying on prompting or fine-tuning with numerous examples, our framework specializes LLM agents using reusable constraints. This approach offers greater generalization, improved reliability, and reduced human effort for real-world network management tasks.CC-BY · From the PDF
Fig. 6 Graph structure constraints are directly extracted from the graph data and translated into code by an LLM. Their explicit structure makes them easy for LLMs to process accurately.
Fig. 6 Graph structure constraints are directly extracted from the graph data and translated into code by an LLM. Their explicit structure makes them easy for LLMs to process accurately.CC-BY · From the PDF
Fig. 7 LLM-generated examples from MALT: one correct output and one failure case.
Fig. 7 LLM-generated examples from MALT: one correct output and one failure case.CC-BY · From the PDF
Fig. 8 Average total accuracy across three apps.
Fig. 8 Average total accuracy across three apps.CC-BY · From the PDF
Fig. 10 MeshAgent adapts to applications with fewer input samples compared to baseline agents.
Fig. 10 MeshAgent adapts to applications with fewer input samples compared to baseline agents.CC-BY · From the PDF
Fig. 11 Constraint # Effect
Fig. 11 Constraint # EffectCC-BY · From the PDF
Image 11
CC-BY · From the PDF
Image 12
CC-BY · From the PDF
Fig. 13 MeshAgent's constraints creation offers an effective balance between accuracy and effort.
Fig. 13 MeshAgent's constraints creation offers an effective balance between accuracy and effort.CC-BY · From the PDF
Fig. 14 Constraint-guided error reduction analysis
Fig. 14 Constraint-guided error reduction analysisCC-BY · From the PDF
Image 15
CC-BY · From the PDF
Fig. 15 MeshAgent's workflow and pre-built constraints shows strong generalization on openended user-submitted queries.
Fig. 15 MeshAgent's workflow and pre-built constraints shows strong generalization on openended user-submitted queries.CC-BY · From the PDF
Fig. 17 An example of how a natural-language-based network management system generates and executes a program in response to a network operator's query: 'Assign a unique color for each /16 IP address prefix'. The system displays the LLM-generated code and the updated communication graph.
Fig. 17 An example of how a natural-language-based network management system generates and executes a program in response to a network operator's query: 'Assign a unique color for each /16 IP address prefix'. The system displays the LLM-generated code and the updated communication graph.CC-BY · From the PDF

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 63 references

Research works printed but not linked: 14

  1. Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan Prafulla Dhariwal Arvind Neelakantan Pranav Shyam Girish Sastry Amanda Askell Sandhini Agarwal Ariel Herbert-Voss Gretchen Krueger Tom Henighan Rewon Child Aditya Ramesh Daniel M. Ziegler Jeffrey Wu Clemens Winter Christopher Hesse Mark Chen Eric Sigler Mateusz Litwin Scott Gray Benjamin Chess Jack Clark Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever and Dario Amodei. 2020. Language Models are Few-Shot Learners. In Advances in Neural Information Processing Systems (NeurIPS).
  2. Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. LoRA: Low-Rank Adaptation of Large Language Models. In The Tenth International Conference on Learning Representations (ICLR).
  3. Yudong Huang, Hongyang Du, Xinyuan Zhang, Dusit Niyato, Jiawen Kang, Zehui Xiong, Shuo Wang, and Tao Huang. 2024. Large language models for networking: Applications, enabling techniques, and challenges. IEEE Network (2024).
  4. Arthur S Jacobs, Ricardo J Pfitscher, Rafael H Ribeiro, Ronaldo A Ferreira, Lisandro Z Granville, Walter Willinger, and Sanjay G Rao. 2021. Hey, lumi! using natural language for {intent-based} network management. In 2021 USENIX Annual Technical Conference (USENIX ATC 21). 625-639.
  5. Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al., 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in Neural Information Processing Systems, Vol. 33 (2020), 9459-9474.
  6. Jeffrey C. Mogul, Drago Goricanec, Martin Pool, Anees Shaikh, Douglas Turk, Bikash Koley, and Xiaoxue Zhao. 2020. Experiences with Modeling Network Topologies at Multiple Levels of Abstraction. In USENIX Symposium on Networked Systems Design and Implementation (NSDI). https://www.usenix.org/conference/nsdi20/presentation/mogul
  7. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F. Christiano, Jan Leike, and Ryan Lowe. 2022. Training language models to follow instructions with human feedback. In Annual Conference on Neural Information Processing Systems (NeurIPS).
  8. Gartner Research. 2017. Innovation Insight: Intent-Based Networking Systems. https://www.gartner.com/en/documents/3599617.
  9. Haopei Wang, Anubhavnidhi Abhashkumar, Changyu Lin, Tianrong Zhang, Xiaoming Gu, Ning Ma, Chang Wu, Songlin Liu, Wei Zhou, Yongbin Dong, et al., 2024a. {NetAssistant}: Dialogue Based Network Diagnosis in Data Center Networks. In 21st USENIX Symposium on Networked Systems Design and Implementation (NSDI 24). 2011-2024.
  10. Jason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoc V. Le. 2022a. Finetuned Language Models are Zero-Shot Learners. In The Tenth International Conference on Learning Representations (ICLR).
  11. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. 2022b. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. In Advances in Neural Information Processing Systems (NeurIPS). http://papers.nips.cc/paper_files/paper/2022/hash/9d5609613524ecf4f15af0f7b31abca4-Abstract-Conference.html
  12. Zihao Xu, Yi Liu, Gelei Deng, Yuekang Li, and Stjepan Picek. 2024b. LLM Jailbreak Attack versus Defense Techniques-A Comprehensive Study. arXiv preprint arXiv:2402.13457 (2024). · arXiv
  13. Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. 2024. Tree of thoughts: Deliberate problem solving with large language models. Advances in Neural Information Processing Systems, Vol. 36 (2024).
  14. Yajie Zhou, Jiajun Ruan, Eric S Wang, Sadjad Fouladi, Francis Y Yan, Kevin Hsieh, and Zaoxing Liu. 2025. NetPress: Dynamically Generated LLM Benchmarks for Network Applications. arXiv preprint arXiv:2506.03231 (2025). · arXiv

Cited by

Metadata

Authors
Zhou, Y., Hsieh, K., Mani, S. K., Kandula, S., & Liu, Z.
Affiliations
  • University of Maryland, College Park (US)
  • Microsoft (United States) (US)
  • Amazon (United States) (US)
Venue
Proceedings of the ACM on Measurement and Analysis of Computing Systems
Published
2025-12
Volume
9
Issue
3
Pages
1-36
ISSN
2476-1249 (ISSN-L)
OpenAlex
W4416927413
Cited by
7
References
32
Citations by year
2026: 7
FWCI
2.94
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • Proceedings of the ACM on Measurement and Analysis of Computing Systems (publishedVersion, cc-by)
  • ACM SIGMETRICS International Conference on Measurement and Modeling of Computer Systems (SIGMETRICS) (publishedVersion, cc-by)
  • ACM SIGMETRICS Performance Evaluation Review (publishedVersion, cc-by)
Funders
National Science Foundation (2431093, 2415754, 2415758)
Topics
  • Physical Sciences › Computer Science › Computer Networks and Communications › Mobile Agent-Based Network Management
  • Physical Sciences › Computer Science › Computer Networks and Communications › Software-Defined Networks and 5G
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
Keywords
large language models, network management, LLM agents, query validation, abstention

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.