CoChat

LUMEN: Enhancing IoT System Observability with Multi-Agent Large Language Models and Knowledge Graphs

Open access
Nedisan Videsjorden, Adela, Song, Hui, Goknil, Arda, Roman, Dumitru, Soylu, Ahmet · 2026 · ACM Transactions on Internet of Things

The DOI matches its Crossref record; 103 of the 103 references it cites were found in a registry or an index; 1 of the 3 sources its claims cite were checked for support.

Abstract

This paper introduces LUMEN, which combines multi-agent LLMs, knowledge graphs, and heterogeneous databases to enable cognitive digital twins for IoT observability.

Overview

CoChat Summary

TL;DR

LUMEN combines multi-agent LLMs, knowledge graphs, and heterogeneous databases to enable cognitive digital twins for IoT observability. In two industrial case studies it achieved a 67.7% average success rate across queries, with most tasks completed under 60 seconds and at $0.002–$0.05 per run.

Key findings

  • Across all evaluated tasks, LUMEN achieved a 67.7% average success rate; seven queries reached 100% success, while three queries produced no correct responses. Failures mostly stemmed from ambiguous queries, missing data schemas, or misinterpretation during complex task decomposition.
  • Of 220 total runs, 124 (56%) involved no agent loops; among these, 99 were successful, giving a 79.8% first-attempt correctness rate. Of the 96 runs involving loops, 50 also resulted in correct answers.
  • Most responses were returned within 60 seconds, and task resolution costs ranged from $0.002 to $0.05 per run. The paper reports no clear tradeoff between accuracy and execution time, token usage, or monetary cost; cost was driven primarily by token usage and message exchanges.
  • Agent loops averaged 1.10 per query for TaskPlanner, 0.87 for CodeGenerator, 0.80 for FilePathDriver, and 0.25 for GraphExplorer. Q-DS-UC2-3 had over 10 loops per run (116 total) and a 0% success rate; Q-SD-UC1-1 had an average execution time of 470.93 seconds.

Methods

  • LUMEN models the IoT system as a knowledge graph in Neo4j, stores static data in MinIO and time-series data in InfluxDB, and uses specialized LLM agents—Human Proxy, Task Planner, Graph Operator, File Path Driver, Code Generator, Code Executor, Output Repeater, and Group Manager—orchestrated via AutoGen.
  • Evaluation used two industrial use cases: electric vehicle charging station infrastructure (UC1) and healthcare information technology infrastructure (UC2), with data from Metricbeat, OCPPFlowMeter, CICFlowMeter, and network traffic PCAPs, plus knowledge-graph creation and analysis queries.
  • Experiments used GPT-4o via API with temperature 0.1, 10 runs per query, and a Docker-based environment on a Windows machine with an 8-core Intel i7-11800H CPU, 32 GB RAM, and an NVIDIA RTX A3000 GPU; metrics included success rate, execution time, token usage, cost, and agent loops.

Limitations

  • Stated limitations include scalability for extremely large IoT systems, dependence on knowledge graph accuracy and completeness, energy consumption and sustainability concerns from LLM inference, limited domain-specific explanations for non-expert users, and the need for future probabilistic inference.
  • Stated validity threats include internal validity concerns (subjective success/failure classification, LLM stochasticity, prompt sensitivity), external validity concerns (only two industrial scenarios, reliance on GPT-4o/OpenAI API, predefined queries, simulated runtime), construct validity concerns (metrics may overlook user satisfaction or explainability), reliability concerns (stochasticity, proprietary API, prompt sensitivity), and human interaction bias from manual query design, output interpretation, labeling, and metadata supplementation.
  • Evident from the text: the evaluation is limited to two use cases and a fixed GPT-4o setup; some queries had 0% success due to ambiguous phrasing or schema-less data; agent loops can add inefficiency without consistently improving success.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1145/3772077

Cites

103 of 103 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Fig. 1 The LUMEN approach as a black box. Inputs include task description from the human operator, the system specification of the IoT system under monitoring (the entire or a subgraph of the IoT system which includes the mapping between assets and their corresponding data sources), and system metrics and raw data streams provided by monitoring tools. The operator interacts with LUMEN by submitting natural language queries. LUMEN processes these requests through its multi-agent architecture, which interprets the queries and generate and execute code to run analytical tasks. LUMEN outputs include actionable insights, natural language explanations, observations, and system diagnostics, depending on the task.
Fig. 1 The LUMEN approach as a black box. Inputs include task description from the human operator, the system specification of the IoT system under monitoring (the entire or a subgraph of the IoT system which includes the mapping between assets and their corresponding data sources), and system metrics and raw data streams provided by monitoring tools. The operator interacts with LUMEN by submitting natural language queries. LUMEN processes these requests through its multi-agent architecture, which interprets the queries and generate and execute code to run analytical tasks. LUMEN outputs include actionable insights, natural language explanations, observations, and system diagnostics, depending on the task.CC-BY · From the PDF
Fig. 2 Overview of the knowledge and data analysis process performed by LUMEN, illustrating its multi-step approach. In Step 1, one dedicated agent interprets user queries, initializes specialized agents for execution, including the initiation of an agent that performs task planning. Step 2 showcases the collaborative efforts of the agents with access to databases. These agents retrieve knowledge about the system and fetch relevant data, if required. Step 3 is reserved for code generation and execution, which is necessary to analyze data and produce actionable insights tailored to the operator's request. In step 4, the output is checked for errors or inaccuracies. If not satisfied, the agent calls for a revised plan, otherwise returns the response to the user. The figure highlights all of LUMEN's intermediary artifacts generated for a given task.
Fig. 2 Overview of the knowledge and data analysis process performed by LUMEN, illustrating its multi-step approach. In Step 1, one dedicated agent interprets user queries, initializes specialized agents for execution, including the initiation of an agent that performs task planning. Step 2 showcases the collaborative efforts of the agents with access to databases. These agents retrieve knowledge about the system and fetch relevant data, if required. Step 3 is reserved for code generation and execution, which is necessary to analyze data and produce actionable insights tailored to the operator's request. In step 4, the output is checked for errors or inaccuracies. If not satisfied, the agent calls for a revised plan, otherwise returns the response to the user. The figure highlights all of LUMEN's intermediary artifacts generated for a given task.CC-BY · From the PDF
Fig. 3 Sequence diagram illustrating the workflow of LUMEN for addressing user queries in the example.
Fig. 3 Sequence diagram illustrating the workflow of LUMEN for addressing user queries in the example.CC-BY · From the PDF
Fig. 4 Part of sample knowledge graph for the motivating example, representing the IoT system's structure and data relationships. The graph models assets such as SCADA, network nodes, and routers, along with their functional roles and connections. It also captures data sources. Relationships such as DeployOn, DataSourceOf, and DataOf define how assets interact and where data is generated or stored, enabling semantic reasoning and context-aware analysis.
Fig. 4 Part of sample knowledge graph for the motivating example, representing the IoT system's structure and data relationships. The graph models assets such as SCADA, network nodes, and routers, along with their functional roles and connections. It also captures data sources. Relationships such as DeployOn, DataSourceOf, and DataOf define how assets interact and where data is generated or stored, enabling semantic reasoning and context-aware analysis.CC-BY · From the PDF
Fig. 5 Agents facilitating the creation and maintenance of the knowledge graph in LUMEN. The Group Manager agent takes the user request from the Human Proxy agent and decides which agents will be involved in solving the task. The user request can be related to new assets, changes in existing assets, and the addition of static and real-time data sources. In a typical run, the Task Planner will create a plan, which leads the Graph Operator agent to translate the user request into Cypher queries, executed via the Neo4j API wrapper. This process updates the knowledge graph. The Output Repeater obtains the execution result and returns it to the user.
Fig. 5 Agents facilitating the creation and maintenance of the knowledge graph in LUMEN. The Group Manager agent takes the user request from the Human Proxy agent and decides which agents will be involved in solving the task. The user request can be related to new assets, changes in existing assets, and the addition of static and real-time data sources. In a typical run, the Task Planner will create a plan, which leads the Graph Operator agent to translate the user request into Cypher queries, executed via the Neo4j API wrapper. This process updates the knowledge graph. The Output Repeater obtains the execution result and returns it to the user.CC-BY · From the PDF
Fig. 6 Agents for task planning in LUMEN. The Human Proxy agent receives user requests and forwards them to the Task Planner agent, which is responsible for task decomposition and agent initialization. The Planner leverages an LLM model to interpret the request, refine it into actionable subtasks, and generate a structured execution plan. The planning process is guided by a system prompt and a list of registered agents.
Fig. 6 Agents for task planning in LUMEN. The Human Proxy agent receives user requests and forwards them to the Task Planner agent, which is responsible for task decomposition and agent initialization. The Planner leverages an LLM model to interpret the request, refine it into actionable subtasks, and generate a structured execution plan. The planning process is guided by a system prompt and a list of registered agents.CC-BY · From the PDF
Fig. 8 Agents facilitating collaborative data analysis in LUMEN. The Human Proxy wraps the user request with the current date and initiates the chat. The Group Manager agent coordinates interactions among specialized agents to process user requests. The Task Planner agent formulates collaboration plans, while the Graph Operator agent retrieves relevant knowledge from the Neo4j knowledge graph via the Neo4j API wrapper. The File Path Driver agent extracts data from MinIO and InfluxDB, using API invocations to retrieve and cache the required datasets, returning their file paths. The Code Generator agent produces Python scripts for data analysis, which are then executed by the Code Executor agent in an isolated environment. The Output Repeater checks the output of the execution and, if satisfactory, repeats it to the user and terminates the chat; otherwise, it will explain the problem, leading to a new round of interactions.
Fig. 8 Agents facilitating collaborative data analysis in LUMEN. The Human Proxy wraps the user request with the current date and initiates the chat. The Group Manager agent coordinates interactions among specialized agents to process user requests. The Task Planner agent formulates collaboration plans, while the Graph Operator agent retrieves relevant knowledge from the Neo4j knowledge graph via the Neo4j API wrapper. The File Path Driver agent extracts data from MinIO and InfluxDB, using API invocations to retrieve and cache the required datasets, returning their file paths. The Code Generator agent produces Python scripts for data analysis, which are then executed by the Code Executor agent in an isolated environment. The Output Repeater checks the output of the execution and, if satisfactory, repeats it to the user and terminates the chat; otherwise, it will explain the problem, leading to a new round of interactions.CC-BY · From the PDF
Fig. 13 Overview of the knowledge graph semiautomatically generated for UC1, representing the underlying infrastructure and network topology. Green nodes represent the assets, while the yellow and orange nodes represent the DataSource and StaticData nodes, respectively.
Fig. 13 Overview of the knowledge graph semiautomatically generated for UC1, representing the underlying infrastructure and network topology. Green nodes represent the assets, while the yellow and orange nodes represent the DataSource and StaticData nodes, respectively.CC-BY · From the PDF
Fig. 14 Overview of the knowledge graph semiautomatically generated for UC2, representing the underlying infrastructure. Green nodes represent the assets. The yellow and orange nodes represent the DataSource and StaticData nodes, respectively.
Fig. 14 Overview of the knowledge graph semiautomatically generated for UC2, representing the underlying infrastructure. Green nodes represent the assets. The yellow and orange nodes represent the DataSource and StaticData nodes, respectively.CC-BY · From the PDF
Fig. 15 Scatter plot over total cost and total tokens measured for all runs.
Fig. 15 Scatter plot over total cost and total tokens measured for all runs.CC-BY · From the PDF
Fig. 16 Scatter plot over total cost and number of exchanged messages measured for all runs.
Fig. 16 Scatter plot over total cost and number of exchanged messages measured for all runs.CC-BY · From the PDF
Fig. 17 Three scatter plots illustrating the relationships between the success rates and average execution time (in seconds), average total model cost (in American dollars) and average over total number of generated tokens, as computed for each query, over multiple runs. Note that Success Rate here is measured as a proportion, not percentage (i.e., 0.1 corresponds to a 10% success rate).
Fig. 17 Three scatter plots illustrating the relationships between the success rates and average execution time (in seconds), average total model cost (in American dollars) and average over total number of generated tokens, as computed for each query, over multiple runs. Note that Success Rate here is measured as a proportion, not percentage (i.e., 0.1 corresponds to a 10% success rate).CC-BY · From the PDF

Verification

Claims the paper makes

1 Valid · 2 Not valid · 0 Not checked · 110 Own results and interpretations

35 more claims

Create a free account to see all 113 claims and every source verdict.

See all claims — free account
  • It may state two separate claims as one. 49
  • Probably not a substantial claim. 37
  • The quote only partly states this claim. 2

Create a free account to see these claims.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

2 of 2

Citations

Cites

Show all 103 references

Research works printed but not linked: 9

  1. AgentGPT. Visited in 2024. AgentGPT: a tool to configure and deploy Autonomous AI agents. Retrieved from https://agentgpt.reworkd.ai/
  2. Ryan Clancy, Ihab F. Ilyas, and Jimmy Lin. 2019. Scalable knowledge graph construction from text collections. In Proceedings of the 2nd Workshop on Fact Extraction and VERification. 39–46. · DOI
  3. Meng Ma, Ping Wang, and Chao-Hsien Chu. 2013. Data management for internet of things: Challenges, approaches and opportunities. In Proceedings of the 2013 IEEE International Conference on Green Computing and Communications and IEEE Internet of Things and IEEE Cyber, Physical and Social Computing. IEEE, 1144–1151.
  4. Igor Melnyk, Pierre Dognin, and Payel Das. 2021. Grapher: Multi-stage knowledge graph construction using pretrained language models. In Proceedings of the NeurIPS 2021 Workshop on Deep Generative Models and Downstream Applications.
  5. Jindřich Mynarz Kateřina Haniková and Vojtěch Svátek. 2023. Test-driven knowledge graph construction. https://kg-construct.github.io/workshop/2023/resources/paper4.pdf
  6. Ruwei Pan Hongyu Zhang and Chao Liu. 2025. CodeCoR: An LLM-based self-reflective multi-agent framework for code generation. arXiv:1409.0473. Retrieved from https://arxiv.org/abs/1701.00133 · arXiv
  7. 10.5555/3692070.3694124 · doi:10.5555/3692070.3694124
  8. Frank Wawrzik, Rami Dhouib, Khushnood Adil Rafique, and Christoph Grimm. 2023. Hybrid AI approach for knowledge graph construction. In Proceedings of the IJCKG (Workshop/Poster/Demo). 48–63.
  9. Zhanfang Zhao, Sung-Kook Han, and In-Mi So. 2018. Architecture of knowledge graph construction techniques. International Journal of Pure and Applied Mathematics 118, 19 (2018), 1869–1883.

Cited by

Metadata

Authors
Nedisan Videsjorden, A., Song, H., Goknil, A., Roman, D., & Soylu, A.
Affiliations
  • SINTEF
  • SINTEF (NO)
  • Høyskolen Kristiania (NO)
Venue
ACM Transactions on Internet of Things
Published
2026-04-21
Volume
7
Issue
2
Pages
1-40
ISSN
  • 2577-6207 (ISSN-L)
  • 2691-1914
OpenAlex
W4415689625
Cited by
5
References
28
Citations by year
2026: 5
FWCI
5.20
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
ACM Transactions on Internet of Things (publishedVersion, cc-by)
Funders
HORIZON EUROPE Framework Programme (101095634)
Topics
  • Physical Sciences › Computer Science › Information Systems › Internet of Things and AI
  • Physical Sciences › Computer Science › Computer Networks and Communications › IoT and Edge/Fog Computing
  • Physical Sciences › Engineering › Industrial and Manufacturing Engineering › Digital Transformation in Industry
Keywords
multi-agent large language models, knowledge graphs, cognitive digital twin, heterogeneous databases, time series, semantic reasoning, explainable analytics, LLM agents, Industrial Internet of Things

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-10-07, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-07 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.