Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 103 of the 103 references it cites were found in a registry or an index; 1 of the 3 sources its claims cite were checked for support.

LUMEN: Enhancing IoT System Observability with Multi-Agent Large Language Models and Knowledge Graphs

Open access
Nedisan Videsjorden, Adela, Song, Hui, Goknil, Arda, Roman, Dumitru, Soylu, Ahmet · 2026 · ACM Transactions on Internet of Things

Abstract

This paper introduces LUMEN, which combines multi-agent LLMs, knowledge graphs, and heterogeneous databases to enable cognitive digital twins for IoT observability.

Overview

CoChat Summary

TL;DR

LUMEN combines multi-agent LLMs, knowledge graphs, and heterogeneous databases to enable cognitive digital twins for IoT observability. In two industrial case studies it achieved a 67.7% average success rate across queries, with most tasks completed under 60 seconds and at $0.002–$0.05 per run.

Key findings

  • Across all evaluated tasks, LUMEN achieved a 67.7% average success rate; seven queries reached 100% success, while three queries produced no correct responses. Failures mostly stemmed from ambiguous queries, missing data schemas, or misinterpretation during complex task decomposition.
  • Of 220 total runs, 124 (56%) involved no agent loops; among these, 99 were successful, giving a 79.8% first-attempt correctness rate. Of the 96 runs involving loops, 50 also resulted in correct answers.
  • Most responses were returned within 60 seconds, and task resolution costs ranged from $0.002 to $0.05 per run. The paper reports no clear tradeoff between accuracy and execution time, token usage, or monetary cost; cost was driven primarily by token usage and message exchanges.
  • Agent loops averaged 1.10 per query for TaskPlanner, 0.87 for CodeGenerator, 0.80 for FilePathDriver, and 0.25 for GraphExplorer. Q-DS-UC2-3 had over 10 loops per run (116 total) and a 0% success rate; Q-SD-UC1-1 had an average execution time of 470.93 seconds.

Methods

  • LUMEN models the IoT system as a knowledge graph in Neo4j, stores static data in MinIO and time-series data in InfluxDB, and uses specialized LLM agents—Human Proxy, Task Planner, Graph Operator, File Path Driver, Code Generator, Code Executor, Output Repeater, and Group Manager—orchestrated via AutoGen.
  • Evaluation used two industrial use cases: electric vehicle charging station infrastructure (UC1) and healthcare information technology infrastructure (UC2), with data from Metricbeat, OCPPFlowMeter, CICFlowMeter, and network traffic PCAPs, plus knowledge-graph creation and analysis queries.
  • Experiments used GPT-4o via API with temperature 0.1, 10 runs per query, and a Docker-based environment on a Windows machine with an 8-core Intel i7-11800H CPU, 32 GB RAM, and an NVIDIA RTX A3000 GPU; metrics included success rate, execution time, token usage, cost, and agent loops.

Limitations

  • Stated limitations include scalability for extremely large IoT systems, dependence on knowledge graph accuracy and completeness, energy consumption and sustainability concerns from LLM inference, limited domain-specific explanations for non-expert users, and the need for future probabilistic inference.
  • Stated validity threats include internal validity concerns (subjective success/failure classification, LLM stochasticity, prompt sensitivity), external validity concerns (only two industrial scenarios, reliance on GPT-4o/OpenAI API, predefined queries, simulated runtime), construct validity concerns (metrics may overlook user satisfaction or explainability), reliability concerns (stochasticity, proprietary API, prompt sensitivity), and human interaction bias from manual query design, output interpretation, labeling, and metadata supplementation.
  • Evident from the text: the evaluation is limited to two use cases and a fixed GPT-4o setup; some queries had 0% success due to ambiguous phrasing or schema-less data; agent loops can add inefficiency without consistently improving success.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1145/3772077

Cites

103 of 103 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

1 Valid · 2 Not valid · 0 Not checked · 110 Own results and interpretations

35 more claims

Create a free account to see all 113 claims and every source verdict.

See all claims — free account
  • It may state two separate claims as one. 49
  • Probably not a substantial claim. 37
  • The quote only partly states this claim. 2

Create a free account to see these claims.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

2 of 2

Citations

Cites

Research works printed but not linked: 9

  1. AgentGPT. Visited in 2024. AgentGPT: a tool to configure and deploy Autonomous AI agents. Retrieved from https://agentgpt.reworkd.ai/
  2. Ryan Clancy, Ihab F. Ilyas, and Jimmy Lin. 2019. Scalable knowledge graph construction from text collections. In Proceedings of the 2nd Workshop on Fact Extraction and VERification. 39–46. · DOI
  3. Meng Ma, Ping Wang, and Chao-Hsien Chu. 2013. Data management for internet of things: Challenges, approaches and opportunities. In Proceedings of the 2013 IEEE International Conference on Green Computing and Communications and IEEE Internet of Things and IEEE Cyber, Physical and Social Computing. IEEE, 1144–1151.
  4. Igor Melnyk, Pierre Dognin, and Payel Das. 2021. Grapher: Multi-stage knowledge graph construction using pretrained language models. In Proceedings of the NeurIPS 2021 Workshop on Deep Generative Models and Downstream Applications.
  5. Jindřich Mynarz Kateřina Haniková and Vojtěch Svátek. 2023. Test-driven knowledge graph construction. https://kg-construct.github.io/workshop/2023/resources/paper4.pdf
  6. Ruwei Pan Hongyu Zhang and Chao Liu. 2025. CodeCoR: An LLM-based self-reflective multi-agent framework for code generation. arXiv:1409.0473. Retrieved from https://arxiv.org/abs/1701.00133 · arXiv
  7. 10.5555/3692070.3694124 · doi:10.5555/3692070.3694124
  8. Frank Wawrzik, Rami Dhouib, Khushnood Adil Rafique, and Christoph Grimm. 2023. Hybrid AI approach for knowledge graph construction. In Proceedings of the IJCKG (Workshop/Poster/Demo). 48–63.
  9. Zhanfang Zhao, Sung-Kook Han, and In-Mi So. 2018. Architecture of knowledge graph construction techniques. International Journal of Pure and Applied Mathematics 118, 19 (2018), 1869–1883.

Cited by

See all 5

Metadata

Authors
Nedisan Videsjorden, A., Song, H., Goknil, A., Roman, D., & Soylu, A.
Affiliations
  • SINTEF
  • SINTEF (NO)
  • Høyskolen Kristiania (NO)
Venue
ACM Transactions on Internet of Things
Published
2026-04-21
Volume
7
Issue
2
Pages
1-40
ISSN
  • 2577-6207 (ISSN-L)
  • 2691-1914
OpenAlex
W4415689625
Cited by
5
References
28
Citations by year
2026: 5
FWCI
5.20
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
ACM Transactions on Internet of Things (publishedVersion, cc-by)
Funders
HORIZON EUROPE Framework Programme (101095634)
Topics
  • Physical Sciences › Computer Science › Information Systems › Internet of Things and AI
  • Physical Sciences › Computer Science › Computer Networks and Communications › IoT and Edge/Fog Computing
  • Physical Sciences › Engineering › Industrial and Manufacturing Engineering › Digital Transformation in Industry
Keywords
multi-agent large language models, knowledge graphs, cognitive digital twin, heterogeneous databases, time series, semantic reasoning, explainable analytics, LLM agents, Industrial Internet of Things

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-10-07, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-07 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.