The DOI matches its Crossref record; 103 of the 103 references it cites were found in a registry or an index; 1 of the 3 sources its claims cite were checked for support.
LUMEN: Enhancing IoT System Observability with Multi-Agent Large Language Models and Knowledge Graphs
Open accessAbstract
This paper introduces LUMEN, which combines multi-agent LLMs, knowledge graphs, and heterogeneous databases to enable cognitive digital twins for IoT observability.
Overview
CoChat Summary
TL;DR
LUMEN combines multi-agent LLMs, knowledge graphs, and heterogeneous databases to enable cognitive digital twins for IoT observability. In two industrial case studies it achieved a 67.7% average success rate across queries, with most tasks completed under 60 seconds and at $0.002–$0.05 per run.
Key findings
- Across all evaluated tasks, LUMEN achieved a 67.7% average success rate; seven queries reached 100% success, while three queries produced no correct responses. Failures mostly stemmed from ambiguous queries, missing data schemas, or misinterpretation during complex task decomposition.
- Of 220 total runs, 124 (56%) involved no agent loops; among these, 99 were successful, giving a 79.8% first-attempt correctness rate. Of the 96 runs involving loops, 50 also resulted in correct answers.
- Most responses were returned within 60 seconds, and task resolution costs ranged from $0.002 to $0.05 per run. The paper reports no clear tradeoff between accuracy and execution time, token usage, or monetary cost; cost was driven primarily by token usage and message exchanges.
- Agent loops averaged 1.10 per query for TaskPlanner, 0.87 for CodeGenerator, 0.80 for FilePathDriver, and 0.25 for GraphExplorer. Q-DS-UC2-3 had over 10 loops per run (116 total) and a 0% success rate; Q-SD-UC1-1 had an average execution time of 470.93 seconds.
Methods
- LUMEN models the IoT system as a knowledge graph in Neo4j, stores static data in MinIO and time-series data in InfluxDB, and uses specialized LLM agents—Human Proxy, Task Planner, Graph Operator, File Path Driver, Code Generator, Code Executor, Output Repeater, and Group Manager—orchestrated via AutoGen.
- Evaluation used two industrial use cases: electric vehicle charging station infrastructure (UC1) and healthcare information technology infrastructure (UC2), with data from Metricbeat, OCPPFlowMeter, CICFlowMeter, and network traffic PCAPs, plus knowledge-graph creation and analysis queries.
- Experiments used GPT-4o via API with temperature 0.1, 10 runs per query, and a Docker-based environment on a Windows machine with an 8-core Intel i7-11800H CPU, 32 GB RAM, and an NVIDIA RTX A3000 GPU; metrics included success rate, execution time, token usage, cost, and agent loops.
Limitations
- Stated limitations include scalability for extremely large IoT systems, dependence on knowledge graph accuracy and completeness, energy consumption and sustainability concerns from LLM inference, limited domain-specific explanations for non-expert users, and the need for future probabilistic inference.
- Stated validity threats include internal validity concerns (subjective success/failure classification, LLM stochasticity, prompt sensitivity), external validity concerns (only two industrial scenarios, reliance on GPT-4o/OpenAI API, predefined queries, simulated runtime), construct validity concerns (metrics may overlook user satisfaction or explainability), reliability concerns (stochasticity, proprietary API, prompt sensitivity), and human interaction bias from manual query design, output interpretation, labeling, and metadata supplementation.
- Evident from the text: the evaluation is limited to two use cases and a fixed GPT-4o setup; some queries had 0% success due to ambiguous phrasing or schema-less data; agent loops can add inefficiency without consistently improving success.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
1 Valid · 2 Not valid · 0 Not checked · 110 Own results and interpretations
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These case studies demonstrated that LUMEN effectively automates IoT system analysis through multi-agent collaboration, but its performance highly depends on accurate task decomposition and good code generation abilities.”
- Also stated in: § Our Case Studies. · plain
“but its performance highly depends on accurate task decomposition and good code generation abilities.”
§ LUMEN: Enhancing IoT System Observability with Multi-Agent Large Language Models and Knowledge Graphs > Our Case Studies.
plainstates a causen = 2- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Our Case Studies. · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The approach is evaluated through two industrial case studies that demonstrate its effectiveness in IoT environments, covering multiple assessment dimensions.”
§ LUMEN: Enhancing IoT System Observability with Multi-Agent Large Language Models and Knowledge Graphs > Our Contributions.
boostedstates a causen = 2- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“One outlier is Q-DS-UC1-1, which has a 20% success rate.”
§ LUMEN: Enhancing IoT System Observability with Multi-Agent Large Language Models and Knowledge Graphs > RQ2: How efficiently do LUMEN agents collaborate to perform semantic reasoning, task decomposition, and dynamic code generation, and to what extent do iterative refinements and error corrections impact execution efficiency?
plainSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“achieving an average success rate of 67.7% across diverse queries”
§ LUMEN: Enhancing IoT System Observability with Multi-Agent Large Language Models and Knowledge Graphs > Our Case Studies.
plainn = 2- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“while maintaining execution times under 60 seconds for most tasks.”
§ LUMEN: Enhancing IoT System Observability with Multi-Agent Large Language Models and Knowledge Graphs > Our Case Studies.
plainn = 2- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
35 more claims
Create a free account to see all 113 claims and every source verdict.
- It may state two separate claims as one. 49
- Probably not a substantial claim. 37
- The quote only partly states this claim. 2
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ LUMEN: Enhancing IoT System Observability with Multi-Agent Large Language Models and Knowledge Graphs > 9.1.1 Use Cases.
“We assessed LUMEN in two use cases: (i) electric vehicle charging station infrastructure and (ii) healthcare information technology infrastructure.”
§ LUMEN: Enhancing IoT System Observability with Multi-Agent Large Language Models and Knowledge Graphs > 9.1.1 Use Cases.
“The healthcare system is our third critical infrastructure use case, distinguished by stringent trust, confidentiality requirements, and regulatory compliance.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ LUMEN: Enhancing IoT System Observability with Multi-Agent Large Language Models and Knowledge Graphs > 9.2 Results
“To assess timeliness, we measured the execution time for each query by timing all agent interactions.Figure 5 presents the results.”
§ LUMEN: Enhancing IoT System Observability with Multi-Agent Large Language Models and Knowledge Graphs > MERGE (n:ASSET name: 'SCADA') -[:DeployOn]->(m:ASSET name: 'n3') .
“Fig. 5. Agents facilitating the creation and maintenance of the knowledge graph in LUMEN.”
Citations
Cites
Internet of Things (IoT): A vision, architectural elements, and future directions
Gubbi, J., Buyya, R., Marusic, S., & Palaniswami, M. (2013) · Future Generation Computer Systems · doi:10.1016/j.future.2013.01.010 · Cited by 12,293
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929
Digital twin: Enabling technologies, challenges and open research
Fuller, A., Fan, Z., Day, C., & Barlow, C. (2020) · IEEE Access · doi:10.1109/access.2020.2998358 · Cited by 2,965
A Survey on Knowledge Graphs: Representation, Acquisition, and Applications
Ji, S., Pan, S., Cambria, E., Marttinen, P., & Yu, P. S. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3070843 · Cited by 2,947
MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework
Hong, S., Zhuge, M., Chen, J., Zheng, X., Cheng, Y., Zhang, C., Wang, J., Wang, Z., Yau, S. K. S., Lin, Z., et al. (2023) · arXiv · doi:10.48550/arxiv.2308.00352 · Cited by 2,825
Knowledge Graph Embedding: A Survey of Approaches and Applications
Wang, Q., Mao, Z., Wang, B., & Guo, L. (2017) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2017.2754499 · Cited by 2,654
A survey on large language model based autonomous agents
Wang, L., Ma, C., Feng, X., Zhang, Z., Yang, H., Zhang, J., Chen, Z., Tang, J., Chen, X., Lin, Y., et al. (2024) · Frontiers of Computer Science · doi:10.1007/s11704-024-40231-1 · Cited by 1,941
A Survey on Large Language Model based Autonomous Agents
Wang, L., Ma, C., Feng, X., Zhang, Z., Yang, H., Zhang, J., Chen, Z., Tang, J., Chen, X., Lin, Y., et al. (2023) · arXiv · doi:10.48550/arxiv.2308.11432 · Cited by 1,925
Hogan, A., Blomqvist, E., Cochez, M., D’amato, C., Melo, G. D., Gutierrez, C., Kirrane, S., Gayo, J. E. L., Navigli, R., Neumaier, S., et al. (2022) · ACM Computing Surveys · doi:10.1145/3447772 · Cited by 1,904 · cited 1×
“A knowledge graph organizes data as nodes (entities) and edges (relationships), enriched with metadata and domain-specific ontologies [17, 19, 20, 27].”
Large Language Model based Multi-Agents: A Survey of Progress and Challenges
Guo, T., Chen, X., Wang, Y., Chang, R., Pei, S., Chawla, N. V., Wiest, O., & Zhang, X. (2024) · arXiv · doi:10.48550/arxiv.2402.01680 · Cited by 1,713
Unifying Large Language Models and Knowledge Graphs: A Roadmap
Pan, S., Luo, L., Wang, Y., Chen, C., Wang, J., & Wu, X. (2024) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2024.3352100 · Cited by 1,376
A review: Knowledge reasoning over knowledge graph
Chen, X., Jia, S., & Xiang, Y. (2020) · Expert Systems with Applications · doi:10.1016/j.eswa.2019.112948 · Cited by 1,035
Big IoT data analytics: Architecture, opportunities, and open research challenges
(2017) · IEEE Access · doi:10.1109/access.2017.2689040 · Cited by 980
Knowledge graph refinement: A survey of approaches and evaluation methods
Paulheim, H. (2016) · Semantic Web · doi:10.3233/sw-160218 · Cited by 849
Internet of Things (IoT): A review of enabling technologies, challenges, and open research issues
Čolaković, A., & Hadžialić, M. (2018) · Computer Networks · doi:10.1016/j.comnet.2018.07.017 · Cited by 825
Digital Twin in the IoT Context: A Survey on Technical Features, Scenarios, and Architectural Models
Minerva, R., Lee, G. M., & Crespi, N. (2020) · Proceedings of the IEEE · doi:10.1109/jproc.2020.2998530 · Cited by 812
Knowledge Graphs: Opportunities and Challenges
Peng, C., Xia, F., Naseriparsa, M., & Osborne, F. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-023-10465-9 · Cited by 717
Intelligent Edge Computing for IoT-Based Energy Management in Smart Cities
Liu, Y., Yang, C., Jiang, L., Xie, S., & Zhang, Y. (2019) · IEEE Network · doi:10.1109/mnet.2019.1800254 · Cited by 595
Mistry, I., Tanwar, S., Tyagi, S., & Kumar, N. (2020) · Mechanical Systems and Signal Processing · doi:10.1016/j.ymssp.2019.106382 · Cited by 585
Cypher: An Evolving Query Language for Property Graphs
Francis, N., Green, A., Guagliardo, P., Libkin, L., Lindaaker, T., Marsault, V., Plantikow, S., Rydberg, M., Selmer, P., & Taylor, A. (2018) · Proceedings of the 2018 International Conference on Management of Data · doi:10.1145/3183713.3190657 · Cited by 545
A systematic review of IoT in healthcare: Applications, techniques, and trends
Haghi Kashani, M., Madanipour, M., Nikravan, M., Asghari, P., & Mahdipour, E. (2021) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2021.103164 · Cited by 491 · cited 1×
“These systems play a crucial role in domains such as smart cities [41], industrial automation [61], healthcare [39], and energy management [48], where continuous monitoring and timely insights are essential for decision-making and operational efficiency.”
Kim, T.-H., Ramos, C., & Mohammed, S. (2017) · Future Generation Computer Systems · doi:10.1016/j.future.2017.03.034 · Cited by 412
A Comprehensive Survey on Automatic Knowledge Graph Construction
Zhong, L., Wu, J., Li, Q., Peng, H., & Wu, X. (2024) · ACM Computing Surveys · doi:10.1145/3618295 · Cited by 382
MapCoder: Multi-Agent Code Generation for Competitive Problem SolvingPreprint
Islam, M. A., Ali, M. E., & Parvez, M. R. (2024) · Annual Meeting of the Association for Computational Linguistics · doi:10.48550/arxiv.2405.11403 · Cited by 309
A Survey on Digital Twin for Industrial Internet of Things: Applications, Technologies and Tools
Xu, H., Wu, J., Pan, Q., Guan, X., & Guizani, M. (2023) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2023.3297395 · Cited by 304
A Survey on Architecture, Protocols and Challenges in IoT
Sobin, C. C. (2020) · Wireless Personal Communications · doi:10.1007/s11277-020-07108-5 · Cited by 295 · cited 1×
“Traditional techniques and tools are fragmented and often designed for specific tasks or use cases [80, 82].”
LLMs for knowledge graph construction and reasoning: recent capabilities and future opportunities
Zhu, Y., Wang, X., Chen, J., Qiao, S., Ou, Y., Yao, Y., Deng, S., Chen, H., & Zhang, N. (2024) · World Wide Web · doi:10.1007/s11280-024-01297-w · Cited by 294
A survey on large language models for code generation
Jiang, J., Wang, F., Shen, J., Kim, S., & Kim, S. (2024) · arXiv · doi:10.48550/arxiv.2406.00515 · Cited by 257
The Convergence of Digital Twin, IoT, and Machine Learning: Transforming Data into Action
Kaur, M. J., Mishra, V. P., & Maheshwari, P. (2020) · Internet of Things · doi:10.1007/978-3-030-18732-3_1 · Cited by 252 · cited 1×
“These approaches decompose the complex process of software synthesis into specialized sub-tasks, such as planning, coding, testing, and debugging, handled by collaborating agents, thereby addressing limitations in coherence, correctness, and scalability inherent to single-agent LLM systems.”
A Survey on Large Language Models for Code Generation
Jiang, J., Wang, F., Shen, J., Kim, S., & Kim, S. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3747588 · Cited by 251
He, J., Treude, C., & Lo, D. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3712003 · Cited by 246
Introduction: What is a knowledge graph?
Fensel, D., Şimşek, U., Angele, K., Huaman, E., Kärle, E., Panasiuk, O., Toma, I., Umbrich, J., & Wahler, A. (2020) · Knowledge Graphs · doi:10.1007/978-3-030-37439-6_1 · Cited by 244
Simulating Classroom Education with LLM-Empowered Agents
Zhang, Z., Zhang-Li, D., Yu, J.-F., et al. (2024) · North American Chapter of the Association for Computational Linguistics · doi:10.48550/arxiv.2406.19226 · Cited by 219
Knowledge Graphs: Methodology, Tools and Selected Use Cases
Fensel, D., Şimşek, U., Angele, K., Huaman, E., Kärle, E., Panasiuk, O., Toma, I., Umbrich, J., & Wahler, A. (2020) · doi:10.1007/978-3-030-37439-6 · Cited by 205
AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation
Wu, Q., Bansal, G., Zhang, J., Wu, Y., Li, B., Zhu, E., Jiang, L., Zhang, X., Zhang, S., Liu, J., et al. (2023) · arXiv · doi:10.48550/arxiv.2308.08155 · Cited by 193
Analytics for the Internet of Things: A Survey
Siow, E., Tiropanis, T., & Hall, W. (2019) · ACM Computing Surveys · doi:10.1145/3204947 · Cited by 182
A Survey of IoT Management Protocols and Frameworks
Sinche, S., Raposo, D., Armando, N., Rodrigues, A., Boavida, F., Pereira, V., & Silva, J. S. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2019.2943087 · Cited by 158
Toward Smart-Building Digital Twins: BIM and IoT Data Integration
Eneyew, D. D., Capretz, M. A. M., & Bitsuamlak, G. T. (2022) · IEEE Access · doi:10.1109/access.2022.3229370 · Cited by 158
Gutierrez, C., & Sequeda, J. F. (2021) · Communications of the ACM · doi:10.1145/3418294 · Cited by 148
Semantic reasoning for context-aware internet of things applications
Maarala, A. I., Su, X., & Riekki, J. (2017) · IEEE Internet of Things Journal · doi:10.1109/jiot.2016.2587060 · Cited by 141
Test-Driven Development and LLM-based Code Generation
Mathews, N. S., & Nagappan, M. (2024) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3691620.3695527 · Cited by 135
Knowledge graph fusion for smart systems: A Survey
Nguyen, H. L., Vu, D. T., & Jung, J. J. (2020) · Information Fusion · doi:10.1016/j.inffus.2020.03.014 · Cited by 131 · cited 1×
“The ability of knowledge graphs to handle heterogeneous and dynamic data makes them particularly well-suited for IoT environments, where diverse devices and systems interact continuously [64].”
Rahman, H., & Hussain, M. I. (2020) · Transactions on Emerging Telecommunications Technologies · doi:10.1002/ett.3902 · Cited by 119 · cited 1×
“This diverse data must be semantically integrated to provide actionable, context-aware information about IoT systems [51, 75].”
Hogan, A., Gutiérrez, C., Cochez, M., de Melo, G., Kirrane, S., Polleres, A., Roberto Navigli, A., Ngomo, A.-C. N., Rashid, S. M., Schmelzeisen, L., et al. (2022) · Synthesis lectures on data, semantics and knowledge · doi:10.1007/978-3-031-01918-0 · Cited by 117
Husom, E. J., Göknil, A., Astekin, M., Shar, L. K., Kåsen, A., Sen, S., Mithassel, B. A., & Soylu, A. (2025) · ACM Transactions on Internet of Things · doi:10.1145/3767742 · Cited by 103 · cited 1×
“While LUMEN offers powerful capabilities for natural language-based monitoring and analysis of IoT networks, its reliance on LLMs introduces significant computational overhead [33, 34].”
Self-citationIoT-Based Framework for Digital Twins in the Industry 5.0 Era
Awouda, A. M. M., Traini, E., Bruno, G., & Chiabert, P. (2024) · Sensors · doi:10.3390/s24020594 · Cited by 98
PiVe: Prompting with Iterative Verification Improving Graph-based Generative Capability of LLMsPreprint
Han, J., Collier, N. H., Buntine, W. L., & Shareghi, E. (2023) · Annual Meeting of the Association for Computational Linguistics · doi:10.48550/arxiv.2305.12392 · Cited by 82
ChatDev: Communicative Agents for Software Development
Qian, C., Liu, W., Liu, H., Chen, N., Dang, Y., Li, J., Yang, C., Chen, W., Su, Y., Cong, X., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.07924 · Cited by 81
AgentBench: Evaluating LLMs as Agents
Liu, X., Yu, H., Zhang, H., Xu, Y., Lei, X., Lai, H., Gu, Y., Ding, H., Men, K., Yang, K., et al. (2023) · arXiv · doi:10.48550/arxiv.2308.03688 · Cited by 79
Knowledge Graphs as Enhancers of Intelligent Digital Twins
Sahlab, N., Kamm, S., Müller, T., Jazdi, N., & Weyrich, M. (2021) · IEEE International Conference on Industrial Cyber-Physical Systems (ICPS) · doi:10.1109/icps49255.2021.9468219 · Cited by 72 · cited 1×
“Zhong et al. [107] provide a comprehensive survey on automatic knowledge graph construction, categorizing the process into three key stages: knowledge acquisition, refinement, and evolution.”
Knowledge graphs: Introduction, history, and perspectives
Chaudhri, V. K., Baru, C., Chittar, N., Dong, X. L., Genesereth, M., Hendler, J., Kalyanpur, A., Lenat, D. B., Sequeda, J., Vrandečić, D., et al. (2022) · AI Magazine · doi:10.1002/aaai.12033 · Cited by 71
Chen, H.-Z., Shen, X., Lv, Q., Wang, J., Ni, X.-Q., & Ye, J.-P. (2024) · Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2024.acl-long.238 · Cited by 70
Multi-Agent Collaboration: Harnessing the Power of Intelligent LLM Agents
Talebirad, Y., & Nadiri, A. (2023) · arXiv · doi:10.48550/arxiv.2306.03314 · Cited by 65
Multilayer Internet-of-Things Middleware Based on Knowledge Graph
Xie, C., Yu, B., Zeng, Z., Yang, Y., & Liu, Q. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3019707 · Cited by 64
Multi-Agent Collaboration via Cross-Team OrchestrationPreprint
Du, Z., Qian, C., Liu, W., Xie, Z., Wang, Y., Qiu, R., Dang, Y., Chen, W., Yang, C. H., Tian, Y., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2406.08979 · Cited by 56
The Price of Prompting: Profiling Energy Use in Large Language Models InferencePreprint
Husom, E. J., Göknil, A., Shar, L. K., & Sen, S. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2407.16893 · Cited by 54
CodeTree: Agent-guided Tree Search for Code Generation with Large Language ModelsPreprint
Li, J.-R., Le, H., Zhou, Y.-B., Xiong, C.-M., Savarese, S., & Sahoo, D. (2024) · North American Chapter of the Association for Computational Linguistics · doi:10.48550/arxiv.2411.04329 · Cited by 49
Experimenting a New Programming Practice with LLMsPreprint
Zhang, S., Wang, J., Dong, G., Sun, J., Zhang, Y., & Pu, G. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2401.01062 · Cited by 44
AgentCoder: Multi-Agent-based Code Generation with Iterative Testing and Optimisation
Huang, D., Zhang, J. M., Luck, M., Bu, Q., Qing, Y., & Cui, H. (2023) · arXiv · doi:10.48550/arxiv.2312.13010 · Cited by 34
Zhang, H., Cheng, W., Wu, Y., & Hu, W. P. (2024) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3691620.3695506 · Cited by 33 · cited 1×
“PairCoder [100] introduces a multi-agent framework comprising a Navigator and a Driver agent.”
Building Knowledge Graph using Pre-trained Language Model for Learning Entity-aware Relationships
Kumar, A., Pandey, A., Gadia, R., & Mishra, M. (2020) · IEEE International Conference on Computing, Power and Communication Technologies (IC2PCT) · doi:10.1109/gucon48875.2020.9231227 · Cited by 32 · cited 1×
“The majority of tasks are completed within 60 seconds and at minimal cost, highlighting the framework’s efficiency and feasibility for real-world applications.”
Experimenting with Multi-Agent Software Development: Towards a Unified PlatformPreprint
Sami, M. A., Waseem, M., Rasheed, Z., Saari, M., Systä, K., & Abrahamsson, P. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2406.05381 · Cited by 30
A Flexible Context Aware Reasoning Approach for IoT Applications
Barbero, C., Zovo, P. D., & Gobbi, B. (2011) · Proceedings/IEEE International Conference on Mobile Data Management · doi:10.1109/mdm.2011.55 · Cited by 27
Analytics for the Internet of Things: A SurveyPreprint
Siow, E., Tiropanis, T., & Hall, W. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.00971 · Cited by 26
The MASSIF platform: a modular and semantic platform for the development of flexible IoT services
Bonte, P., Ongenae, F., Backere, F. D., Schaballie, J., Arndt, D., Verstichel, S., Mannens, E., Rik Van De Walle, B., & Turck, F. D. (2016) · Knowledge and Information Systems · doi:10.1007/s10115-016-0969-1 · Cited by 25
Scalable Knowledge Graph Construction over Text using Deep Learning based Predicate Mapping
Mehta, A., Singhal, A., & Karlapalem, K. (2019) · International World Wide Web Conference (WWW) · doi:10.1145/3308560.3317708 · Cited by 25
Designing a Semantic Digital Twin model for IoT
Muralidharan, S., Yoo, B., & Ko, H. (2020) · 2020 IEEE International Conference on Consumer Electronics (ICCE) · doi:10.1109/icce46568.2020.9043088 · Cited by 24
Executable Code Actions Elicit Better LLM Agents
Wang, X., Chen, Y., Yuan, L., Zhang, Y., Li, Y., Peng, H., & Ji, H. (2024) · arXiv · doi:10.48550/arxiv.2402.01030 · Cited by 23
CodePori: Large Scale Model for Autonomous Software Development by Using Multi-Agents
Rasheed, Z., Waseem, M., Saari, M., Systä, K., & Abrahamsson, P. (2024) · arXiv.org · doi:10.48550/arxiv.2402.01411 · Cited by 22
AgileCoder: Dynamic Collaborative Agents for Software Development based on Agile Methodology
Nguyen, M. H., Chau, T. P., Nguyen, P. X., & Bui, N. D. Q. (2024) · arXiv · doi:10.48550/arxiv.2406.11912 · Cited by 21
Liu, Z., Zeng, R., Wang, D., Peng, G., Liu, X., Liu, Q., Liu, P., Wang, W., & Wang, J. (2026) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2026.3667895 · Cited by 20
Tian, C., & Zhang, Y. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2408.13406 · Cited by 19
Ni, Z., Li, Y., Yang, N., Shen, D., Lv, P., & Dong, D. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2412.15305 · Cited by 19
Multi-Agent Consensus Seeking via Large Language Models
Chen, H., Ji, W., Xu, L., & Zhao, S. (2023) · arXiv · doi:10.48550/arxiv.2310.20151 · Cited by 16
Chen, H., Shen, X., Lv, Q., Wang, J., Ni, X., & Ye, J. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2410.02811 · Cited by 16
Wang, H., Liu, Z., Wang, S., Cui, G., Ding, N., Liu, Z.-Y., & Yu, G. (2023) · arXiv.org · doi:10.48550/arxiv.2311.09868 · Cited by 14
Constructing Chinese Historical Literature Knowledge Graph Based on BERT
Guo, Q., Sun, Y., Liu, G., Wang, Z., Ji, Z., Shen, Y., & Wang, X. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-87571-8_28 · Cited by 13 · cited 1×
“The study further demonstrates that LLM-based agents, when properly orchestrated and bounded by structured plans and termination criteria, can contribute to digital twin operability–reducing reliance on human experts while enabling explainable and actionable insights.”
Is Self-Repair a Silver Bullet for Code Generation?
Olausson, T. X., Inala, J. P., Wang, C., Gao, J., & Solar-Lezama, A. (2023) · arXiv · doi:10.48550/arxiv.2306.09896 · Cited by 13
MAS4POI: a Multi-Agents Collaboration System for Next POI Recommendation
Wu, Y., Peng, Y., Yu, J.-P., et al. (2024) · arXiv.org · doi:10.48550/arxiv.2409.13700 · Cited by 10
Context-aware query derivation for IoT data streams with DIVIDE enabling privacy by design
Brouwer, M. D., Steenwinckel, B., Fang, Z., Stojchevska, M., Bonte, P., Turck, F. D., Hoecke, S. V., & Ongenae, F. (2023) · Semantic Web · doi:10.3233/sw-223281 · Cited by 10
Federated detection of open charge point protocol 1.6 cyberattacks
Dalamagkas, C., Radoglou-Grammatikis, P., Bouzinis, P., Papadopoulos, I., Lagkas, T., Argyriou, V., Goudos, S., Margounakis, D., Fountoukidis, E., & Sarigiannidis, P. (2025) · Complex Engineering Systems · doi:10.20517/ces.2025.04 · Cited by 8
Ishibashi, Y., & Nishimura, Y. (2024) · arXiv · doi:10.48550/arxiv.2404.02183 · Cited by 8
gBuilder: A Scalable Knowledge Graph Construction System for Unstructured CorpusPreprint
Li, Y., & Zou, L. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2208.09705 · Cited by 8
Nunez, A., Islam, N. T., Jha, S. K., & Najafirad, P. (2024) · arXiv · doi:10.48550/arxiv.2409.10737 · Cited by 7
Semantic Reasoning for Context-aware Internet of Things Applications
Maarala, A. I., Su, X. J., & Riekki, J. P. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1604.08340 · Cited by 5
Intelligent Monitoring and Scheduling of Real-time Dynamic Data Based on Digital Twin
Zhao, B., Dang, Q., Zhang, Q., Du, C., & Li, C. (2023) · International Conference on Frontiers of Electronics, Information and Computation Technologies · doi:10.1109/icfeict59519.2023.00094 · Cited by 3
Developing Multi-Agent LLM Applications Through Continuous Human-LLM Co-Programming
Song, H., Goknil, A., Jiang, X., Melum, E., Joe, H., Gazzotti, C., Frascolla, V., Videsjorden, A. N., & Nguyen, P. (2025) · 2025 IEEE/ACM 4th International Conference on AI Engineering – Software Engineering for AI (CAIN) · doi:10.1109/cain66642.2025.00013 · Cited by 3 · cited 1×
“The evolution of LLMs from single-agent models to multi-agent frameworks has enabled collaborative problem-solving [26, 83].”
Self-citationZeeshan, T., Kumar, A., Pirttikangas, S., & Tarkoma, S. (2025) · arXiv · doi:10.48550/arxiv.2501.00906 · Cited by 3
Synergistic Simulations: Multi-Agent Problem Solving with Large Language ModelsPreprint
Sprigler, A., Drobek, A., Weinstock, K., Tapsoba, W. A., Childress, G., Dao, A., & Gral, L. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2409.13753 · Cited by 3
An IoT Architecture to Provide Hybrid Context Reasoning
Machado, R. D. S., Almeida, R. B., Albandes, R. D. C., Pernas, A. M., & Yamin, A. (2020) · IFIP advances in information and communication technology · doi:10.1007/978-3-030-43605-6_6 · Cited by 2 · cited 1×
“For hybrid reasoning, Machado et al. [52] propose an IoT architecture enabling a dynamic composition of reasoning strategies to address context-awareness challenges in healthcare environments.”
CodeS: Natural Language to Code Repository via Multi-Layer Sketch
Zan, D., Yu, A., Liu, W., Chen, D., Shen, B., Li, W., Yao, Y., Gong, Y., Chen, X., Guan, B., et al. (2024) · arXiv · doi:10.48550/arxiv.2403.16443 · Cited by 2
MedAide: Information Fusion and Anatomy of Medical Intents via LLM-based Agent CollaborationPreprint
Yang, D., Wei, J., Li, M., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2410.12532 · Cited by 2
Liu, Z., Zeng, R., Wang, D., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2410.14209 · Cited by 2
Large Language Model-Enabled Multi-Agent Manufacturing SystemsPreprint
Lim, J., Vogel‐Heuser, B., & Kovalenko, I. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2406.01893 · Cited by 1
Empowering Agile-Based Generative Software Development through Human-AI TeamworkPreprint
Zhang, S. M., Xing, Z., Guo, R., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2407.15568 · Cited by 1
CodexGraph: Bridging Large Language Models and Code Repositories via Code Graph Databases
Liu, X., Lan, B., Hu, Z., Liu, Y., Zhang, Z., Wang, F., Shieh, M., & Zhou, W. (2024) · arXiv · doi:10.48550/arxiv.2408.03910 · Cited by 1
OpenAgents: An Open Platform for Language Agents in the Wild
Xie, T., Zhou, F., Cheng, Z., Shi, P., Weng, L., Liu, Y., Hua, T. J., Zhao, J., Liu, Q., Liu, C., et al. (2023) · arXiv · doi:10.48550/arxiv.2310.10634
Test-Driven Development for Code Generation
Mathews, N. S., & Nagappan, M. (2024) · arXiv · doi:10.48550/arxiv.2402.13521
Federated Detection of Open Charge Point Protocol 1.6 Cyberattacks
Dalamagkas, C., Radoglou-Grammatikis, P., Bouzinis, P., et al. (2025) · arXiv · doi:10.48550/arxiv.2502.01569
Gutierrez, C., & Sequeda, J. F. (2020) · Proceedings of the 29th ACM International Conference on Information & Knowledge Management · doi:10.1145/3340531.3412176
Attack prediction in Internet of Things using knowledge graph
Zhang, S., Zhao, C., Wang, S., et al. (2023) · 3rd International Conference on Internet of Things and Smart City (IoTSC 2023) · doi:10.1117/12.2683915
Knowledge Graphs: Introduction, History and, Perspectives
Chaudhri, V., Baru, C., Chittar, N., et al. (2022) · AI Magazine · doi:10.1609/aimag.v43i1.19119
An inference sharing architecture for a more efficient context reasoning
Almeida, A., & Lopez-de-Ipina, D. (2012) · 2012 IEEE International Conference on Pervasive Computing and Communications Workshops · doi:10.1109/percomw.2012.6197630
Research works printed but not linked: 9
- AgentGPT. Visited in 2024. AgentGPT: a tool to configure and deploy Autonomous AI agents. Retrieved from https://agentgpt.reworkd.ai/
- Ryan Clancy, Ihab F. Ilyas, and Jimmy Lin. 2019. Scalable knowledge graph construction from text collections. In Proceedings of the 2nd Workshop on Fact Extraction and VERification. 39–46. · DOI
- Meng Ma, Ping Wang, and Chao-Hsien Chu. 2013. Data management for internet of things: Challenges, approaches and opportunities. In Proceedings of the 2013 IEEE International Conference on Green Computing and Communications and IEEE Internet of Things and IEEE Cyber, Physical and Social Computing. IEEE, 1144–1151.
- Igor Melnyk, Pierre Dognin, and Payel Das. 2021. Grapher: Multi-stage knowledge graph construction using pretrained language models. In Proceedings of the NeurIPS 2021 Workshop on Deep Generative Models and Downstream Applications.
- Jindřich Mynarz Kateřina Haniková and Vojtěch Svátek. 2023. Test-driven knowledge graph construction. https://kg-construct.github.io/workshop/2023/resources/paper4.pdf
- Ruwei Pan Hongyu Zhang and Chao Liu. 2025. CodeCoR: An LLM-based self-reflective multi-agent framework for code generation. arXiv:1409.0473. Retrieved from https://arxiv.org/abs/1701.00133 · arXiv
- 10.5555/3692070.3694124 · doi:10.5555/3692070.3694124
- Frank Wawrzik, Rami Dhouib, Khushnood Adil Rafique, and Christoph Grimm. 2023. Hybrid AI approach for knowledge graph construction. In Proceedings of the IJCKG (Workshop/Poster/Demo). 48–63.
- Zhanfang Zhao, Sung-Kook Han, and In-Mi So. 2018. Architecture of knowledge graph construction techniques. International Journal of Pure and Applied Mathematics 118, 19 (2018), 1869–1883.
Cited by
LLMs and IoT: A Comprehensive Survey on Large Language Models and the Internet of Things
Sarhaddi, F., Nguyen, N. T., Zuniga, A., Khan, M., Hui, P., Tarkoma, S., Flores, H., & Nurmi, P. (2026) · IEEE Open Journal of the Communications Society · doi:10.1109/ojcoms.2026.3680064 · Cited by 32
“Agentic AI has been used in various applications such as automated prescriptive maintenance in smart manufacturing [112], distributed IoT automation sys-tem (LLMind 2.0) [113], spatiotemporal urban data analysis (CityGPT) [114], and industrial IoT system enhancement (LUMEN) [115].”
InfluentialEmotionally Adaptive IoT Ecosystems for Neurodivergent Cognitive Regulation
Aarthi, S., & Ravikumar, R. N. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-2600-0462-3.ch007
Becattini, M., Biondino, E. G., Vicario, E., & Iera, A. (2026) · IEEE Internet of Things Journal · doi:10.1109/jiot.2026.3716300
Metadata
- Authors
- Nedisan Videsjorden, A., Song, H., Goknil, A., Roman, D., & Soylu, A.
- Affiliations
- SINTEF
- SINTEF (NO)
- Høyskolen Kristiania (NO)
- Venue
- ACM Transactions on Internet of Things
- Published
- 2026-04-21
- Volume
- 7
- Issue
- 2
- Pages
- 1-40
- ISSN
- 2577-6207 (ISSN-L)
- 2691-1914
- OpenAlex
- W4415689625
- Cited by
- 5
- References
- 28
- Citations by year
- 2026: 5
- FWCI
- 5.20
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- ACM Transactions on Internet of Things (publishedVersion, cc-by)
- Funders
- HORIZON EUROPE Framework Programme (101095634)
- Topics
- Physical Sciences › Computer Science › Information Systems › Internet of Things and AI
- Physical Sciences › Computer Science › Computer Networks and Communications › IoT and Edge/Fog Computing
- Physical Sciences › Engineering › Industrial and Manufacturing Engineering › Digital Transformation in Industry
- Keywords
- multi-agent large language models, knowledge graphs, cognitive digital twin, heterogeneous databases, time series, semantic reasoning, explainable analytics, LLM agents, Industrial Internet of Things
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-07, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-07 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.