CoChat
Sign Up
Cited by 107IoT and Edge/Fog ComputingImage and Video Quality AssessmentInternet of Things and AI

QoE in IoT: a vision, survey and future directions

Open access
Fizza, Kaneez, Banerjee, Abhik, Mitra, Karan, Jayaraman, Prem Prakash, Ranjan, Rajiv, Patel, Pankesh, Georgakopoulos, Dimitrios·2021·Discover Internet of Things
DOI 10.1007/s43926-021-00006-7
Open this paper in CoChat — free accountOpen PDF

Abstract

The rapid evolution of the Internet of Things (IoT) is making way for the development of several IoT applications that require minimal or no human involvement in the data collection, transformation, knowledge extraction, and decision-making (actuation) process. To ensure that such IoT applications (we term them autonomic) function as expected, it is necessary to measure and evaluate their quality, which is challenging in the absence of any human involvement or feedback. Existing Quality of Experience (QoE) literature and most QoE definitions focuses on evaluating application quality from the lens of human receiving application services. However, in autonomic IoT applications, poor quality of decisions and resulting actions can degrade the application quality leading to economic and social losses. In this paper, we present a vision, survey and future directions for QoE research in IoT. We review existing QoE definitions followed by a survey of techniques and approaches in the literature used to evaluate QoE in IoT. We identify and review the role of data from the perspective of IoT architectures, which is a critical factor when evaluating the QoE of IoT applications. We conclude the paper by identifying and presenting our vision for future research in evaluating the QoE of autonomic IoT applications.

AI snapshot

Written from the paper’s full text.

TL;DR

This review surveys QoE definitions and models for IoT, arguing that existing QoE is human-centric and insufficient for autonomic IoT applications where devices sense, analyze, and actuate without human feedback. It presents a vision and future directions for objective, data-aware QoE evaluation across the IoT application lifecycle.

Key findings

  • Existing QoE definitions from ITU-T, ETSI, Qualinet, Nokia, Ericsson, and others focus on end-user subjective perception; the paper states there is no widely generic QoE definition for IoT, and only Minovski et al.’s QoIoT/QoME work addresses machine-focused quality alongside user-centric QoE.
  • In the surveyed literature, 77.77% of work involves human subjective evaluation for QoE in IoT, while 22.22% involves no human feedback for evaluation.
  • Reviewed QoE models either decompose application QoE into submetrics such as Quality of Data (QoD), Quality of Information (QoI), and QoS, or evaluate QoE without decomposition using subjective user feedback and IoT architectural layers; one QoD/QoI-based model reported a Pearson correlation of 0.81 with subjective test results.
  • Data quality is identified as crucial for IoT application quality, characterized by metrics including accuracy, completeness, usefulness, and timeliness.

Methods

  • Survey of research published from 2007 to the present (2020) referring to QoE in IoT, using Google Scholar, Springer, Science Direct, and IEEE Xplore, with search terms including “Quality of experience” and “IoT”, “QoE definition” and “IoT”, “QoE” and “data”, and “QoE models” in Abstract/Title/Keywords.
  • The paper reports selecting 50 papers in total, while also stating that 53 papers were surveyed; papers were classified by reading abstracts and full papers when necessary, with duplicates across databases eliminated.
  • Narrative review and synthesis of QoE definitions, QoE models, metrics, datasets, and the role of data; no primary empirical experiment is conducted.

Limitations

  • The paper states that no widely generic QoE definition exists for IoT, and existing definitions rely on human feedback and do not cover sensing, analytics, and actuation or machine experiences.
  • The paper notes a lack of general IoT architecture in the literature, and reviewed QoE models often use synthetic data or human feedback, limiting their applicability to autonomic IoT applications.
  • Evident from the text: the selection criteria list appears incomplete after “were the following,” the survey count is inconsistent (50 vs. 53 papers), and many reviewed works are summarized without detailed extraction.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1007/s43926-021-00006-7

Cites

49 of 49 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

13 of 55 cited sources checked · 6 supported · 1 flagged · accuracy not yet measured

  • Premisecitation

    In [36] authors developed a predictive model for quality evaluation in IoT based solely on data gathered from a survey performed on the users of the application.

    “In [36] authors developed a predictive model for quality evaluation in IoT.The model is based on user experience from the IoT application.This model is solely based on data gathered from the survey performed on the users of the application.”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [36] Robitza W, Ahmad A, Kara PA, Atzori L, Martini MG, Raake A, et al. Challenges of future multimedia QoE monitoring for internet service providers. Multimedia Tools Appl. 2017;76(21):22243–66. source

      Section 3.4, “Virtual objects” (page unavailable), describes creating quality models for virtualized IoT services as a challenge, not a model the authors developed. The Conclusion (page unavailable) likewise says the major challenge lies in developing appropriate models. Neither passage describes a model built solely from a user survey. (LLM, conf=0.84)

      Checked by a reasoning model — may be wrong.

      Experimental — accuracy not yet measured
  • Findingnone given

    Some other existing literature [30,37,38] do not assess how the application QoE relates to the individual components of IoT applications.

    “The same is true with some of the other existing literature [30,37,38], as they do not assess how the application QoE relates to the individual components of IoT applications.”
    Quote in the paperNot a substantial claim1 of 3 sources checked
    • cites [30] Shin D-H. Conceptualizing and measuring quality of experience of the internet of things: Exploring how quality is perceived by users. InfManag. 2017;54(8):998–1011.

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [37] Ikeda Y, Kouno S, Shiozu A, Noritake K. A framework of scalable QoE modeling for application explosion in the Internet of Things; 2016. p. 425–9. source
    • cites [38] Suryanegara M, Prasetyo DA, Andriyanto F, Hayati N. A 5-step framework for measuring the quality of experience (QoE) of Internet of Things (IoT) services. IEEE Access. 2019;7:175779–92. source
  • Premisenone given

    Using MOS is the most widely used method for subjective ratings.

    “Even though using MOS is the most widely used method, its validity is recently questioned by several researchers [35].”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [35] Hoßfeld T, Heegaard PE, Varela M, Möller S. QoE beyond the MOS: an in-depth look at QoE via better metrics and their relation to MOS. Qual User Exp. 2016;1(1):2. https://doi.org/10.1007/s41233-016-0002-1. source
  • Premisecitation

    The distributed architecture of an IoT application significantly impacts its performance.

    “the distributed architecture of an IoT application also significantly impacts its performance [52].”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [52] Anand P, Singh Y, Selwal A, Singh PK, Felseghi RA, Raboaca MS. IoVT: internet of vulnerable things? Threat architecture, attack surfaces, and vulnerabilities in Internet of Things And Its Applications Towards Smart Grids. Energies. 2020;13(18):1–23. source

      In the section labeled “X X X” (page not given), the paper reports that particular IoT architectures can reduce energy consumption or detect attacks with fewer overheads. It does not establish the broader claim that an IoT application's distributed architecture significantly affects its overall performance. (LLM, conf=0.76)

      Experimental — accuracy not yet measured
  • Premisecitation

    Data transfer between different networks in autonomic IoT applications renders a threat to the security aspects of the applications.

    “This necessitates data transfer between different networks rendering threat to the security aspects of the applications [52].”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [52] Anand P, Singh Y, Selwal A, Singh PK, Felseghi RA, Raboaca MS. IoVT: internet of vulnerable things? Threat architecture, attack surfaces, and vulnerabilities in Internet of Things And Its Applications Towards Smart Grids. Energies. 2020;13(18):1–23. source
  • Classified as probably not a substantial claim. 54
  • May combine two claims. 26
  • The quote only partly states this claim. 1

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The validity of MOS is recently questioned by several researchers.

    “Even though using MOS is the most widely used method, its validity is recently questioned by several researchers [35].”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [35] Hoßfeld T, Heegaard PE, Varela M, Möller S. QoE beyond the MOS: an in-depth look at QoE via better metrics and their relation to MOS. Qual User Exp. 2016;1(1):2. https://doi.org/10.1007/s41233-016-0002-1. source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    In [41], authors proposed a QoE model to measure QoE of a teleconsultation app using five factors: integrality, retainability, availability, usability and instantaneousness.

    “In [41], authors proposed a QoE model to measure QoE of a teleconsultation app.First, they identified five factors which impact QoE.These factors are integrality, retainability, availability, usability and instantaneousness.”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [41] de la Torre DI, Alonso SG, Cruz EM, Franco MA. Measuring QoE of a teleconsultation app in mental health using a pentagram model. J Med Syst. 2019;43(7):213. source

      The Methods section describes a five-factor pentagram model, and Table 1 names integrality, retainability, availability, usability, and instantaneousness. But the Introduction says the authors used an existing model-based approach [12], while the Conclusion says proposing a new model is future work; this paper did not propose the model as claimed. Page numbers are unavailable in our copy. (LLM, conf=0.90)

      Experimental — accuracy not yet measured

  • May combine two claims.

    Premisecitation

    Surveys [14,15] focused on QoE in multimedia IoT applications, which is determined based on how such applications are consumed by users.

    “In surveys such as [14,15], authors have focussed on QoE in multimedia IoT applications, which is determined based on how such applications are consumed by users.”
    Quote in the paperNot a substantial claim1 of 2 sources checked
    • cites [14] Barakabitze AA, Barman N, Ahmad A, Zadtootaghaj S, Sun L, Martini MG, et al. QoE management of multimedia streaming services in future networks: a tutorial and survey. IEEE CommunSurv Tutorials. 2020;22(1):526–65. https://doi.org/10.1109/COMST.2019.2958784. source

      The survey discusses QoE management for multimedia IoT applications in “Multimedia Communications in Internet of Things” (p. 29), and defines QoE in terms of users’ experience of an application or service in “Quality of Experience: Definition” (p. 4). But multimedia IoT is one subsection of a broader survey of multimedia streaming services, not the survey’s central focus. (LLM, conf=0.85)

      Experimental — accuracy not yet measured
    • cites [15] Skorin-Kapov L, Varela M, Hoßfeld T, Chen K-T. A survey of emerging concepts and challenges for QoE management of multimedia services. ACM Trans Multimedia Comput CommunAppl. 2018. https://doi.org/10.1145/3176648. source

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    In [43], they identified quality requirements of a smart city, defined users, computed a mean opinion score based on surveys before and after implementing the IoT application, and performed the difference between after and before implementing the application.

    “In this use case, authors first identified the quality requirements of a smart city.Secondly, they defined the users of this IoT application.Thirdly, they compute a mean opinion score (MOS) based on surveys performed before and after implementing the IoT application.Lastly, the difference between after implementing and before implementing the application is performed.”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [43] Pal D, Vanijja V, Arpnikanondt C, Zhang X, Papasratorn B. A quantitative approach for evaluating the quality of experience of smart-wearables from the quality of data and quality of information: an end user perspective. IEEE Access. 2019;7:64266–78. source

      Split into 2 atoms — 0 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 1 unverifiable. Expand the claim to see per-atom verdicts.

      Experimental — accuracy not yet measured

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    One recently evolving method for measuring the quality of multimedia content involves both subjective and objective evaluation to obtain the best of two methods.

    “One of the recently evolving methods for measuring the quality of multimedia content involves both subjective and objective evaluation to obtain the best of two methods [36].”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [36] Robitza W, Ahmad A, Kara PA, Atzori L, Martini MG, Raake A, et al. Challenges of future multimedia QoE monitoring for internet service providers. Multimedia Tools Appl. 2017;76(21):22243–66. source

      Section 2.2 (page unavailable) describes automated multimedia-quality models that estimate a value akin to a human observer’s rating, and §2.1 describes a shift toward predicting user-perceived quality. Neither passage says that a method combines subjective and objective evaluations to obtain the benefits of both. (LLM, conf=0.78)

      Experimental — accuracy not yet measured

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    In [43], authors measure the QoE of a smart city application using the smart city of Jakarta as a use case.

    “In [43], the authors measure the QoE of a smart city application using the smart city of Jakarta as a use case.”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [43] Pal D, Vanijja V, Arpnikanondt C, Zhang X, Papasratorn B. A quantitative approach for evaluating the quality of experience of smart-wearables from the quality of data and quality of information: an end user perspective. IEEE Access. 2019;7:64266–78. source

      The Introduction and Discussion describe a QoE evaluation of smart-wearables and their companion applications, so the source supports the measurement of QoE. They do not describe a smart-city application or a Jakarta use case; the paper mentions smart cities only as one example of the many kinds of IoT applications. (LLM, conf=0.88)

      Experimental — accuracy not yet measured

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    They model the application QoE and measure the users experience through subjective evaluations.

    “They model the application QoE and measure the users experience through subjective evaluations.”
    Quote in the paperNot a substantial claim1 of 3 sources checked
    • cites [30] Shin D-H. Conceptualizing and measuring quality of experience of the internet of things: Exploring how quality is perceived by users. InfManag. 2017;54(8):998–1011.

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [37] Ikeda Y, Kouno S, Shiozu A, Noritake K. A framework of scalable QoE modeling for application explosion in the Internet of Things; 2016. p. 425–9. source
    • cites [38] Suryanegara M, Prasetyo DA, Andriyanto F, Hayati N. A 5-step framework for measuring the quality of experience (QoE) of Internet of Things (IoT) services. IEEE Access. 2019;7:175779–92. source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    In [31] on QoE in multimedia IoT, authors measure QoE objectively.

    “For example, in one of such works [31] on QoE in multimedia IoT, authors measure QoE objectively.”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [31] Floris A, Atzori L. Managing the quality of experience in the multimedia internet of things: a layered-based approach. Sensors. 2016;16(12):2057. source

  • Classified as probably not a substantial claim.

    Findingnone given

    The role of data is a critical factor when evaluating the QoE of IoT applications.

    “We identify and review the role of data from the perspective of IoT architectures, which is a critical factor when evaluating the QoE of IoT applications.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    The authors in [40] do not highlight how these metrics impact the quality of IoT applications.

    “However, the authors do not highlight how these metrics impact the quality of IoT applications.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [40] Li L, Rong M, Zhang G. An Internet of Things QoE evaluation method based on multiple linear regression analysis; 2015. p. 925–8. source

  • May combine two claims.

    Findingnone given

    Existing QoE literature and most QoE definitions focus on evaluating application quality from the perspective of a human receiving application services, typically in multimedia applications.

    “Existing Quality of Experience (QoE) literature and most QoE definitions focuses on evaluating application quality from the lens of human receiving application services.”
    • Also stated in: § Organizations like International Telecommunication Union ITU-T (ITU-T) and · plain
      “However, all the existing definitions of QoE focus on the understanding and measurement of the quality of an application or a service received by the endusers (humans), and typically from multimedia applications perspective.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Existing QoE definitions are focused on human as users and QoE evaluation is based on their feedback.

    “We reviewed the existing literature including QoE definitions, all of which are focused on human as users and QoE evaluation is based on their feedback.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Findingnone given

    Currently, QoE models are based on human feedback or experience.

    “Currently, QoE models are based on human feedback or experience.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    To the best of the authors' knowledge, Minovski et al. [11] provided the first comprehensive definition of QoE from an IoT perspective.

    “To the best of our knowledge, Minovski et al. [11] provided the first comprehensive definition of QoE from IoT perspective.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [11] Satyanarayanan M. The emergence of edge computing. Computer. 2017;50(1):30–9. https://doi.org/10.1109/MC.2017.9. source

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Findingnone given

    The notion of QoE in IoT is relatively new.

    “As discussed till now, the notion of QoE in IoT is relatively new”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingreasoning

    In a food paste processing application, the IoT network is outside the corporate machine network, and security issues render delays in sensed data from the machine being delivered for analysis.

    “For example, in food paste processing application, the IoT network is outside the corporate machine network.Hence, there are security issues that render delays in sensed data from machine being delivered for analysis.”
    Quote in the paperNot a substantial claim

  • May combine two claims. The quote only partly states this claim.

    Findingown data

    The QoE models reviewed can be classified broadly into two categories: methods that break application QoE into sub-metrics such as QoD, QoI and QoS, and methods that evaluate application QoE without breaking it down using subjective user feedback.

    “The QoE models we reviewed in this section can be classified broadly into two categories as follows:”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    QoE in IoT is a well-explored area.

    “QoE in IoT showed is a well explored area.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    Table 3 lists existing papers that characterized IoT data based on metrics such as accuracy, completeness, usefulness and timeliness.

    “Table 3 lists such existing papers which has characterized IoT data based on the metrics such as accuracy, completeness, usefulness and timeliness.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    The introduction of edge computing introduces additional complexities due to its highly distributed nature and need to support distributed processing of IoT application.

    “However, the introduction of edge computing introduces additional complexities due to its highly distributed nature and need to support distributed processing of IoT application.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    Definitions of QoE given by ETSI and the Qualinet White Paper cannot be mixed with IoT in its purest form because IoT applications involve device-to-device interaction and humans may not be present as end users to provide feedback.

    “Similarly, definitions of QoE given by the European Telecommunications Standard Institute and Qualinet White Paper cannot be mixed with IoT in its purest form because IoT application involves IoT device to device interaction and humans may not be present as end users to provide his/her feedback about the application.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    This distributed processing of IoT applications further necessitates continuous evaluation of quality of IoT applications.

    “This distributed processing of IoT applications further necessitates continuous evaluation of quality of IoT applications.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    QoE considers only the end-user's objective and subjective experience but does not measure the overall machine performance.

    “QoE considers only the end-user's objective and subjective experience but does not measure the overall machine performance.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [19] Minovski D, Hlund C, Mitra K. Modeling quality of IoT experience in autonomous vehicles. IEEE Internet of Things J. 2020;7:3833–49. source

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    In IoT applications, quality is not driven by network metrics alone but also by other metrics such as data.

    “However, in IoT applications, quality is not driven by network metrics alone but also by other metrics such as data.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Machine performance is measured in terms of decision-making, reasoning, and optimized machine operations.

    “Machine performance is measured in terms of decision-making, reasoning, and optimized machine operations.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [19] Minovski D, Hlund C, Mitra K. Modeling quality of IoT experience in autonomous vehicles. IEEE Internet of Things J. 2020;7:3833–49. source

  • May combine two claims.

    Premisecitation

    Subjective evaluation is a complex method that fails for real-time applications.

    “However, it is a complex method that fails for real-time applications [23].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [23] Wubin P, Gaung C, Hua W, Yongning T, editors. Towards QoE assessment of encrypted YouTube adaptive video streaming in mobile networks. In: 2016 IEEE/ACM 24th international symposium on quality of service (IWQoS); 2016; 20–21 June 2016. source

  • May combine two claims.

    Interpretationreasoning

    Existing QoE definitions involve human feedback to evaluate application quality and do not include machine experiences; hence quality measurement of autonomic IoT applications remains uncovered.

    “Also, these definitions involve human feedback to evaluate the application quality and do not include machine experiences; hence., quality measurement of such IoT applications remains uncovered.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    In [40], authors aim to ensure QoE through existing QoS parameters and develop a functional model between QoE and QoS using principal component analysis and regression.

    “In [40], authors aim to ensure QoE through existing QoS parameters.In this direction, they develop a functional model between QoE and QoS.The model uses principal component analysis (PCA) to identify the principal components.Next, they deduce regression equation between QoE and principal components.Based on this regression equation, they obtain a functional model which connects QoE and QoS.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [40] Li L, Rong M, Zhang G. An Internet of Things QoE evaluation method based on multiple linear regression analysis; 2015. p. 925–8. source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    QoME measures machine performance objectively.

    “QoME measures machine performance objectively.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [19] Minovski D, Hlund C, Mitra K. Modeling quality of IoT experience in autonomous vehicles. IEEE Internet of Things J. 2020;7:3833–49. source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    In [39], the proposed model's results show a strong correlation with subjective test results through a Pearson correlation value of 0.81.

    “The results from the proposed model shows a strong correlation with the results obtained from subjective test through a Pearson correlation value of 0.81.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [39] Pal D, Triyason T, Varadarajan V, Zhang X, editors. Quality of experience evaluation of smart-wearables: a mathematical modelling approach. In: 2019 IEEE 35th international conference on data engineering workshops (ICDEW); 2019 8–12 April; 2019. source

  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationnone given

    QoS metrics such as throughput, delay, jitter, packet loss, availability and reliability are precise in definition but are rather crude.

    “QoS metrics such as throughput, delay, jitter, packet loss, availability and reliability are precise in definition but are rather crude.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    Evaluation of how individual components impact the final application quality is a grand challenge.

    “Further, since the result produced by the IoT application i.e., paste consistency, is not just from actuation (i.e., applying specific settings to the evaporator machine), but is a result of other components of IoT, which includes the sensor data collected in real-time as well as the data analytics performed using the collected sensor data, evaluation of how individual components impact the final”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationcitation

    In an IoT ecosystem, there is a need to consider device-to-device interactions that can impact the quality of the application and hence the need for more objective measures to assess QoE.

    “However, as mentioned above, in an IoT ecosystem, there is a need to consider device to device interactions that can impact the quality of the application and hence the need for more objective measures to assess QoE [19,22].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [19] Minovski D, Hlund C, Mitra K. Modeling quality of IoT experience in autonomous vehicles. IEEE Internet of Things J. 2020;7:3833–49. source
    • cites [22] Han B, Zhang X, Qi Y, Gao Y, Yang D. QoE model based optimization for streaming media service considering equipment and environment factors. Wireless PersCommun. 2012;66(3):595–612. https://doi.org/10.1007/s11277-012-0739-7. source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Subjective evaluation is considered the most accurate method of getting a user's opinion.

    “Subjective evaluation is considered as the most accurate method of getting a user's opinion.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [23] Wubin P, Gaung C, Hua W, Yongning T, editors. Towards QoE assessment of encrypted YouTube adaptive video streaming in mobile networks. In: 2016 IEEE/ACM 24th international symposium on quality of service (IWQoS); 2016; 20–21 June 2016. source

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    In [13], the proposed MQRP scheme leads to improved QoS performance metrics such as packet delivery ratio, memory utilization, control message overhead, residual energy, network lifetime, and throughput.

    “The experimental results in the paper show that the proposed scheme leads to improved QoS performance metrics, such as packet delivery ratio, memory utilization, control message overhead, residual energy, network lifetime, and throughput.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [13] Faheem M, Gungor VC. MQRP: Mobile sinks-based QoS-aware data gathering protocol for wireless sensor networks-based smart grid applications in the context of industry 4.0-based on internet of things. Future Gen Comput Syst. 2018;82:358–74. https://doi.org/10.1016/j.future.2017.10.009. source

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationreasoning

    The distributed and heterogeneous nature of IoT applications implies that the quality of IoT applications is impacted by multiple factors, which make them distinct from other QoE measurement methods in the literature.

    “Furthermore, the distributed and heterogeneous nature of IoT applications implies that the quality of IoT applications is impacted by multiple factors, which make them distinct from other QoE measurement methods in the literature.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    The quality of multimedia IoT applications can be evaluated in terms of how accurate the actuation command is.

    “The quality of such multimedia IoT applications can be evaluated in terms of how accurate the actuation command is.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    Recently Minovski et al. [19,20] and Mitra et al. [12] have argued for the need to understand quality in an IoT ecosystem; within an IoT ecosystem a user can be both human and/or other systems and IoT devices.

    “Recently Minovski et al. [19,20] and Mitra et al. [12] have argued for the need to understand quality in an IoT ecosystem; within an IoT ecosystem a user can be both human and/or other systems and IoT devices.”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [19] Minovski D, Hlund C, Mitra K. Modeling quality of IoT experience in autonomous vehicles. IEEE Internet of Things J. 2020;7:3833–49. source
    • cites [20] Minovski D, hlund C, Mitra K, Zhohov R. Quality of experience for the Internet of Things. IT Professional Magazine; 2020. p. 1–9. source
    • cites [12] Mitra K, Zaslavsky A, Åhlund C. Context-aware QoE modelling, measurement, and prediction in mobile computing systems. IEEE Trans Mob Comput. 2015;14(5):920–36. https://doi.org/10.1109/TMC.2013.155. source

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    The need to assess QoE objectively is enforced more strongly with the rapid increase and adoption of analytics-driven actuations in autonomic IoT applications where there is no human feedback.

    “The need to assess QoE objectively is enforced more strongly with the rapid increase and adoption of analytics-driven actuations in autonomic IoT applications where there is no human feedback.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: In autonomic IoT applications, poor quality of decisions and resulting actions can degrade application quality, leading to economic and social losses.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Premisenone given

    Distributed processing of IoT applications on the network edge and the centralized cloud is suitable, especially for latency-sensitive IoT applications.

    “IoT applications' distributed processing on the network edge and the centralized cloud is suitable, especially for latency sensitive IoT applications.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

    Interpretationnone given

    Measuring QoE provided by an application is an important metric for system designers and developers as it provides them with a mechanism to quantify the outcomes provided by the application they are developing.

    “Measuring QoE provided by an application is an important metric for system designers and developers as it provides them with a mechanism to quantify the outcomes provided by the application they are developing.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    In the device layer, metrics related to data producing IoT devices are considered, such as GPS accuracy of sensor devices or resolution of camera devices.

    “In the device layer, metrics related to data producing IoT devices are considered such as, GPS accuracy of sensor devices or the resolution of camera devices.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [40] Li L, Rong M, Zhang G. An Internet of Things QoE evaluation method based on multiple linear regression analysis; 2015. p. 925–8. source

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    The outcomes of an IoT application may be consumed by another device or even another IoT application.

    “This implies that the outcomes of an IoT application may be consumed by another device or even another IoT application.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: IoT applications are often systems of systems, in which multiple autonomous IoT applications interact with each other.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    In [30], they do not directly measure the quality of content, service, and system but measure these metrics through the utility and hedonicity the end-user receives.

    “However, in their work, they do not directly measure the quality of content, service, and system.But they measure these metrics through the utility and hedonicity the end-user (humans) receives.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [30] Shin D-H. Conceptualizing and measuring quality of experience of the internet of things: Exploring how quality is perceived by users. InfManag. 2017;54(8):998–1011.

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisereasoning

    Layer based model is another approach for modelling quality of IoT applications.

    “Layer based model is another approach for modelling quality of IoT applications.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    In [39], authors proposed a QoE model based on a generic QoE equation.

    “In [39], the authors proposed a QoE model based on a generic QoE equation.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [39] Pal D, Triyason T, Varadarajan V, Zhang X, editors. Quality of experience evaluation of smart-wearables: a mathematical modelling approach. In: 2019 IEEE 35th international conference on data engineering workshops (ICDEW); 2019 8–12 April; 2019. source

  • May combine two claims.

    Interpretationreasoning

    While existing definitions of QoE cover the satisfaction of end-users, they may not be sufficient for IoT applications where an actuation performed by an IoT device can impact another IoT device or a set of connected IoT devices (and not always a human).

    “While existing definitions of QoE cover the satisfaction of end-users, they may not be sufficient for IoT applications where an actuation performed by IoT device (e.g., machine) can impact another IoT device or a set of connected IoT devices (and not always a human).”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Physical metrics include factors that impact application quality and are mapped to IoT architecture; metaphysical metrics correspond to factors that impact application quality and are mapped to application requirements.

    “Physical metrics include the factors that impact the application quality and are mapped to IoT architecture.Metaphysical metrics correspond to factors which impact the application quality and are mapped to application requirements.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [40] Li L, Rong M, Zhang G. An Internet of Things QoE evaluation method based on multiple linear regression analysis; 2015. p. 925–8. source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    To organize quality metrics corresponding to IoT architecture, functions of IoT architecture are virtually divided into four layers: device, network, computing, and user interface.

    “To organize the quality metrics corresponding to IoT architecture (physical metrics), functions of IoT architecture are virtually divided into four different layers and the quality metrics are defined for each layer.These layers are device, network, computing, and user interface.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [40] Li L, Rong M, Zhang G. An Internet of Things QoE evaluation method based on multiple linear regression analysis; 2015. p. 925–8. source

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    In [30], authors used the ITU-T QoE definition and describe the IoT network as an aggregate of three factors: system, content, and service.

    “For example, in [30], authors have used the QoE definition given by ITU-T and they describe the IoT network as an aggregate of three factors i.e. system, content, and service.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [30] Shin D-H. Conceptualizing and measuring quality of experience of the internet of things: Exploring how quality is perceived by users. InfManag. 2017;54(8):998–1011.

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    In [40], authors categorized metrics impacting application QoE into physical metrics and metaphysical metrics.

    “In [40], the authors categorized the metrics which impact application QoE into two broad categories: physical metrics and metaphysical metrics.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [40] Li L, Rong M, Zhang G. An Internet of Things QoE evaluation method based on multiple linear regression analysis; 2015. p. 925–8. source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    In [42], authors measure the QoE of a teleconsultation application called Psiconnect using an approach based on the pentagram model.

    “One such work is [42], in which authors measure the QoE of teleconsultation application called Psiconnect using an approach that is based on pentagram model.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [42] Floris A, Atzori L. Quality of experience in the multimedia Internet of Things: definition and practical use-cases; 2015. p. 1747–52. source

  • May combine two claims.

    Interpretationreasoning

    Subjective tests cannot be used for fog/edge enabled IoT applications, and objective metrics are necessitated.

    “Thus, first, we cannot use the subjective tests in such cases, and objective metrics are necessitated.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationreasoning

    The outputs of an autonomic IoT application may be consumed by another IoT device, implying the absence of any user feedback.

    “However, the outputs of an autonomic IoT application may be consumed by another IoT device (e.g.machines in case of Industry 4.0), implying the absence of any user feedback.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The pentagram model is based on five factors: integrality, retainability, availability, usability, and instantaneousness.

    “This pentagram model is based on five factors (integrality, retainability, availability, usability, and instantaneousness).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [42] Floris A, Atzori L. Quality of experience in the multimedia Internet of Things: definition and practical use-cases; 2015. p. 1747–52. source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    They modeled QoE as a weighted linear combination of Quality of Data (QoD) and Quality of Information (QoI).

    “They modeled QoE as a weighted linear combination of Quality of Data (QoD) and Quality of Information (QoI).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [39] Pal D, Triyason T, Varadarajan V, Zhang X, editors. Quality of experience evaluation of smart-wearables: a mathematical modelling approach. In: 2019 IEEE 35th international conference on data engineering workshops (ICDEW); 2019 8–12 April; 2019. source

  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationnone given

    IoT applications may operate automatically without human intervention and may produce outcomes that are consumed later.

    “This is because IoT applications may operate automatically without human intervention and even produce outcomes that are consumed later.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The authors in [47] identified that poor quality data likely leads to unsound decisions and actions in IoT.

    “In [47], authors identified that a poor quality data likely leads to unsound decisions and actions in IoT.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [47] Karkouch A, Mousannif H, Al Moatassime H, Noel T. Data quality in internet of things: a state-of-the-art survey. J Netw Comput Appl. 2016;73:57–81. source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The authors in [45] identified that quality of data is crucial not only at device or network level but also at application level.

    “The authors in [45] identified that quality of data is crucial not only at device or network level but also at application level.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [45] Banerjee T, Sheth A. Iot quality control for data and application needs. IEEE Intell Syst. 2017;32(2):68–73. source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The papers were classified by reading their abstracts as well as the full paper when necessary.

    “We have classified the papers by reading their abstracts as well as the full paper when necessary”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Minovski et al. in [19] evaluated the quality of autonomous vehicles based on metrics such as quality of data, network and context.

    “Minovski et al. in [19] evaluated the quality of autonomous vehicles based on metrics such as quality of data, network and context.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [19] Minovski D, Hlund C, Mitra K. Modeling quality of IoT experience in autonomous vehicles. IEEE Internet of Things J. 2020;7:3833–49. source

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    In [30], the authors evaluate the quality of experience in IoT based on quality of network, which in turn is driven by the generated IoT data.

    “In [30], authors evaluate the quality of experience in IoT based on quality of network which in turn is driven by the generated IoT data.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [30] Shin D-H. Conceptualizing and measuring quality of experience of the internet of things: Exploring how quality is perceived by users. InfManag. 2017;54(8):998–1011.

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductown data

    The survey selected 50 papers, most of which are from the past 10 years.

    “In total, we selected 50 papers, most of which are from past 10 years.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    All papers repeated in more than one database were eliminated.

    “all the papers repeated in more than one database are eliminated.”
    Quote in the paperNot a substantial claim
64 of 64

  • The paper relies on this without support.

    In autonomic IoT applications, poor quality of decisions and resulting actions can degrade application quality, leading to economic and social losses.

    “However, in autonomic IoT applications, poor quality of decisions and resulting actions can degrade the application quality leading to economic and social losses.”

  • The paper relies on this without support.

    Manufacturing KPIs are typically not measured during production.

    “Firstly, there is typically no human feedback available based on which the QoE of the application can be estimated in real-time, since the manufacturing KPIs are typically not measured during production.”

  • The paper relies on this without support.

    IoT applications increasingly require minimal or no human involvement in data collection, transformation, knowledge extraction, and decision-making (actuation).

    “The rapid evolution of the Internet of Things (IoT) is making way for the development of several IoT applications that require minimal or no human involvement in the data collection, transformation, knowledge extraction, and decisionmaking (actuation) process.”

  • The paper relies on this without support.

    IoT applications are often systems of systems, in which multiple autonomous IoT applications interact with each other.

    “IoT applications are often systems of systems, in which multiple autonomous IoT applications interact with each other.”

  • The paper relies on this without support.

    IoT applications are inherently distributed, and this distribution results from both computation requirements and the distinct roles of application components that contribute differently to the application outcome.

    “IoT applications are distinct from other applications in the fact that not only are they inherently distributed, but the distribution is result of both computation requirements and the distinct roles of application components that contribute differently to the application outcome.”

  • The paper relies on this without support.

    The 53 surveyed papers are representative of the literature on QoE in IoT.

    “The plethora of the paper we have surveyed (53 papers) have considered human involvement in QoE evaluation for IoT as shown in Fig. 3.”

  • The paper relies on this without support.

    It is possible to evaluate application quality without human feedback by identifying factors that impact it.

    “Suppose the designer of the application can identify the factors and evaluate their impact on the application quality.In that case, such QoE models can be used for applications where a human is not present to provide his subjective feedback.”
7 of 7

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Reference [43] is described as two different application studies

    Minor

    One passage describes [43] as a study of Jakarta's smart city application; another describes it as a smart-wearable application.

  • Reference [11] is attributed to the wrong authors and topic

    Minor

    The paper credits Minovski et al. [11] with a definition of IoT QoE, but reference [11] is an edge-computing paper by Satyanarayanan.

  • Reference [42] is assigned two different models

    Minor

    The text attributes a pentagram model for a teleconsultation application to [42], then attributes a five-layer multimedia IoT model to the same reference.

  • Survey paper count changes from 50 to 53

    Minor

    The paper says it selected 50 papers, but later describes the surveyed set as 53 papers without explaining the different totals.

4 of 4

Cites

Flagged first, then by citations

1 of the 50 references couldn’t be matched to a known paper.

Research works printed but not linked: 2

  1. Ericsson. Video QoE: leveraging standards to meet rising user expectations; 2017. https://www.ericsson.com/en/reports-and-papers/ericsson-technology-review/articles/video-qoe-leveraging-standards-to-meet-rising-user-expectations. Accessed 12 Dec 2020.
  2. Floris A, Atzori L. Towards the evaluation of quality of experience of internet of things applications. IEEE Internet Things J. 2017.

Cited by

Sorted by citations

Related

Authors
Fizza, K., Banerjee, A., Mitra, K., Jayaraman, P. P., Ranjan, R., Patel, P., & Georgakopoulos, D.
Affiliations
  • Swinburne University of Technology (AU)
  • Luleå University of Technology (SE)
  • Newcastle University (GB)
  • Ollscoil na Gaillimhe – University of Galway (IE)
Corresponding author
  • Fizza, Kaneez
  • Jayaraman, Prem Prakash
  • Ranjan, Rajiv
ORCID
Venue
Discover Internet of Things
Publisher
Springer Science and Business Media LLC
Published
2021-02-24
Volume
1
Issue
1
ISSN
2730-7239 (ISSN-L)
DOI
10.1007/s43926-021-00006-7
OpenAlex
W3164531790
Semantic Scholar
f19003c6d80ec6d67ce01eec841b7959686413fc
Cited by
107
References
50
Citations by year
2026: 1 · 2025: 14 · 2024: 17 · 2023: 30 · 2022: 24 · 2021: 21
FWCI
10.41
Type
journal article
Language
en
Open access
Licence
Locations
  • Discover Internet of Things (publishedVersion, cc-by)
  • Diva portal (Dalarna University Library) (submittedVersion, other-oa)
Funders
  • National Natural Science Foundation of China (2019-2021, 2019-2022)
  • Engineering and Physical Sciences Research Council (EP/T021985/1)
  • Research England
Topics
  • Physical Sciences › Computer Science › Computer Networks and Communications › IoT and Edge/Fog Computing
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image and Video Quality Assessment
  • Physical Sciences › Computer Science › Information Systems › Internet of Things and AI
Keywords
quality of experience, Internet of Things, QoE evaluation, IoT architecture, decision quality, actuation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-09.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-09.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review