CoChat
Sign Up
Cited by 39Innovative Human-Technology InteractionImage and Video Quality AssessmentMobile Crowdsensing and Crowdsourcing

A Quantitative Approach for Evaluating the Quality of Experience of Smart-Wearables From the Quality of Data and Quality of Information: An End User Perspective

Open access
Pal, Debajyoti, Vanijja, Vajirasak, Arpnikanondt, Chonlameth, Zhang, Xiangmin, Papasratorn, Borworn·2019·IEEE Access
DOI 10.1109/access.2019.2917061
Open this paper in CoChat — free accountOpen PDF

Abstract

A rapid growth in the smart-wearable industry is making it increasingly important to cater to the quality of experience (QoE) requirements of the end users. In this paper, we try to model the relationship between human experience and quality perception in relation to the smart-wearable segment. For this, the concepts of quality of data (QoD) and quality of information (QoI) are used. While QoD is concerned with the accuracy and precision of the data collected by the smart-wearables, QoI relates to the useful information that is obtained from the raw data captured by the devices via the companion applications of each wearable installed on a smartphone. A subjective experiment comprising of 40 participants and 5 wearable devices is performed in a free-living condition in order to create the QoE model. Four different approaches for weight determination are presented in this paper: balanced weight distribution, correlation-based distribution, hybrid distribution, and priority-based distribution while proposing the QoE model. Our results show that the priority-based distribution approach performs slightly better than the rest of the techniques and has the best correlation to the subjective QoE when compared against others. Based on the results, the appropriate recommendations are provided to the different smart-wearable vendors for improving their products, thereby ensuring greater user adoption.

AI snapshot

Written from the paper’s full text.

TL;DR

The paper proposes a quantitative QoE model for smart wearables using quality of data (QoD: step count, heart rate) and quality of information (QoI: perceived ease of use, perceived usefulness, richness of information), tested with 40 participants and 5 devices in free-living conditions. The priority-based weight distribution performed best, with R² = 0.75 and adjusted R² = 0.73, while correlation-based distribution had the lowest accuracy (R² = 0.42, adjusted R² = 0.38).

Key findings

  • Priority-based weight distribution had the highest modeling accuracy (R² = 0.75, adjusted R² = 0.73) and the lowest RMSE; correlation-based distribution had the lowest accuracy (R² = 0.42, adjusted R² = 0.38) and the highest RMSE.
  • For QoD, heart-rate monitoring accuracy was more important to users than step counts, receiving higher weightage; priority-based weights were A = 0.210, B = 0.790, C = 0.080, D = 0.60, E = 0.320, and µ = 0.280.
  • For QoI, users gave higher preference to perceived usefulness and richness of information, while perceived ease of use received very little importance; balanced and hybrid weight distributions yielded almost the same results.
  • Accuracy testing used Bland-Altman plots and one-sample T-tests, which found no significant fixed bias for step counts or heart rate monitoring.

Methods

  • A subjective experiment used 40 participants evenly distributed by gender, aged 18–34, and 5 wearables: Xiaomi Mi Band 2, Lenovo HW01, Samsung Gear Fit2 Pro, Apple Watch Series 3, and Samsung Gear S3 Frontier, over a three-month free-living period.
  • Step-count and heart-rate procedures were repeated four times per device, worn evenly on dominant and non-dominant wrists; baselines were a Yamax Digi-Walker SW-701 pedometer and Onyx Vantage 9590 pulse oximeter, with heart rate recorded every 4 hours from 09:00 to 16:00 daily.
  • QoD was rated on a 5-point MOS scale after showing averaged Bland-Altman results; QoI was measured via questionnaire on PEOU, PU, and RI using a 5-point Likert scale; overall QoE was rated on a 5-point MOS scale. The model used weighted linear combinations, with a 50:50 data split for model building and validation, analyzed in SPSS version 17.0.

Limitations

  • Stated: the sample may not represent the whole wearable-user population, and different results may be obtained with elderly users; averaging results across devices to generalize the model overlooks possible differences in accuracy and companion-app features between cheap and expensive devices.
  • Stated: future work should compare additive versus multiplicative QoE approaches, use more subjects, and analyze data with machine learning techniques such as ANN and ANFIS.
  • Evident: the study is scoped to fitness trackers and smartwatches only, uses only step counts and heart rate for QoD, and uses only PEOU, PU, and RI for QoI; the young, tech-savvy sample may explain why perceived ease of use was given little importance.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1109/access.2019.2917061

Cites

52 of 52 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 8 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    The QoE values obtained by the different techniques vary in terms of modeling accuracy.

    “The QoE values obtained by the different techniques vary in terms of the modeling accuracy.”
    Quote in the paperSubstantial claim
  • Findingown data

    A one-sample T-test indicated no significant fixed bias between the smart-wearables and the reference methods for both step-counts and heart rate.

    “Before carrying out the Bland-Altman plots on the respective datasets, a one-sample T-test is conducted to indicate the presence/absence of a fixed bias and found to be non-significant for both cases.”
    Quote in the paperNot a substantial claim
  • Findingreasoning

    Using the priority based distribution technique, the weights obtained are A = 0.210, B = 0.790, C = 0.080, D = 0.60, E = 0.320, and μ = 0.280.

    “Following this technique, the corresponding weights obtained are A = 0.210, B = 0.790, C = 0.080, D = 0.60, E = 0.320, andµ = 0.280.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: The weightage of different parameters towards the final QoE are different.
  • Findingreasoning

    Using the correlation based distribution technique, the weights obtained are A = 0.403, B = 0.597, C = 0.155, D = 0.517, E = 0.328, and μ = 0.299.

    “Using the above technique, the following weights are obtained: A = 0.403, B = 0.597, C = 0.155, D = 0.517, E = 0.328, µ = 0.299.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: QoD and QoI attributes are more relevant in the wearable context and the IoT paradigm in general.
  • Findingreasoning

    For the different approaches, there is only a slight variation in the mean calculated QoE values along with their standard deviations.

    “For the different approaches, there is only a slight variation in the mean values along with their standard deviations.”
    Quote in the paperNot a substantial claim
  • May combine two claims. 10
  • Classified as probably not a substantial claim. 11

  • May combine two claims.

    Findingown data

    The priority-based distribution approach performs slightly better than the other weight-determination techniques and has the best correlation to subjective QoE.

    “Our results show that the priority-based distribution approach performs slightly better than the rest of the techniques and has the best correlation to the subjective QoE when compared against others.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: QoE is a weighted linear combination of the parameters from the QoD and QoI categories.
    • The authors state this as an assumption: QoE is a function of QoD and QoI.

  • May combine two claims.

    Findingown data

    The QoE ratings obtained by the priority based technique and the subjective evaluation from the twenty validation participants follow the same general trend and are in close proximity, with a relatively large Pearson correlation coefficient.

    “It is observed that the values follow the same general trend, and are in close proximity of each other (thereby having a relatively large Pearson Correlation Coefficient value).”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingreasoning

    In the balanced weight distribution technique, for equation (2) the QoD parameters A and B are each weighted 0.5, and for equation (3) the QoI parameters C, D, and E are each weighted 0.33.

    “In equation (2), the QoD category has two parameters, and hence the weights are distributed evenly between A and B with A = B = 0.5 and satisfying constraint (4).Similarly, for equation (3) the weights are evenly distributed between the three parameters and hence C = D = E = 0.33 and satisfying constraint (5).”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: QoD and QoI attributes are more relevant in the wearable context and the IoT paradigm in general.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingnone given

    Although the accuracy of the four proposed methods varies to a certain extent, there is a common trend.

    “Although the accuracy of the four proposed methods vary to a certain extent, yet there is common trend.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingreasoning

    In the hybrid distribution technique, the weights obtained are A = B = 0.5, C = D = E = 0.33, and μ = 0.299.

    “Hence, the weights obtained are A = B = 0.5 and C = D = E = 0.33.For obtaining the weight of equation ( 1), the correlation-based approach is followed using normalized values to add up to one (µ = 0.299).”
    Quote in the paperSubstantial claim
    • Rests on an assumption: QoD and QoI attributes are more relevant in the wearable context and the IoT paradigm in general.

  • May combine two claims.

    Findingown data

    The correlation-based method has the highest RMSE value, and the priority-based approach has the lowest RMSE value.

    “The correlation-based method has the lowest accuracy since it has the highest RMSE value.Similarly, the priority-based approach has the lowest RMSE value.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationcitation

    Different results can be obtained if more elderly people are included, because their acceptance and use of technology differ from the younger generation.

    “For example, different results can be obtained if more people that are elderly are taken into account, as their acceptance and use of technology are different from the younger generation [59].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [59] source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Study design and conductown data

    Accuracy was measured by assessing criterion validity and level of agreement between the wearables and reference methods using Bland-Altman plots with mean bias and upper and lower limits of agreement.

    “The accuracy is measured by assessing the criterion validity and level of agreement between the wearables and the respective reference methods.Bland-Altman plots with mean bias and upper and lower limits of agreement (LoA) are used for assessing the accuracy.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Study design and conductnone given

    The experiments were conducted in a free-living environment over three months, which is expected to simulate real-life usage more accurately than laboratory-based setups.

    “Second, unlike most of the previous works our experiments have been conducted in a free-living environment over a span of three months, which is expected to simulate a real-life usage scenario more accurately when compared to laboratory-based setups.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    The accuracy results were averaged across all smart-wearables to create a sense of generalization.

    “Instead of focusing on each of the individual devices, the results are averaged out across all the smart-wearables used in order to create a sense of generalization.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisenone given

    Neither QoD nor QoI concepts have been tested for the wearable category, nor has a methodology been provided to translate them to QoE for measuring human satisfaction.

    “However, neither these concepts have been tested for the wearable category, nor any methodology is provided as to how these concepts can be translated back to QoE for measuring the human satisfaction level.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    For model building and testing the accuracy of the different techniques, a 50:50 data split ratio was used, with randomly drawn data from twenty participants used for model building and the remaining data used for validation.

    “For the purpose of model building and testing the accuracy of the different techniques, we have used a 50:50 data split ratio.Thus, randomly drawn data from twenty different participants are used for model building, while the remaining ones are used for validating the respective models.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Study design and conductnone given

    Using latest generation wearables across a variety of price points minimizes the chances of bias for specific devices.

    “Third, for building the QoE model and minimizing the chances of bias for specific devices, we use the latest generation wearables across a variety of price points that support different features.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    The balanced weight distribution and hybrid weight distribution approaches almost yield the same results because the parameter correlation values are very close to each other in magnitude.

    “For example, the balanced weight distribution and the hybrid weight distribution approaches almost yield the same results.This is because that the parameter correlation values are very close to each other in magnitude as well.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationreasoning

    The finding that users give little importance to perceived ease of use is attributed to the relatively young age of the participants (18-34 years), who are generally more tech-savvy and curious to adopt new technology.

    “We attribute this finding to the relatively young age of our participants (18-34 years) who are generally more tech-savvy, and curious to adopt any new technology.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductreasoning

    Each procedure was repeated four times with the smart-wearable worn evenly between the dominant and non-dominant wrist to remove location dependency on the results.

    “Each procedure is repeated four times with the smart-wearable being worn evenly between the dominant and the non-dominant wrist (in order to remove any location dependency on the results).”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    If pre-expectations and post-expectations do not match, the adoption rate will be low.

    “If the preexpectations (before purchasing) and the post-expectations (after purchasing and using a product) do not match, the adoption rate will be low [20], [21].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [20], [21]

      Not checked in the free run.

17 of 17

  • The paper relies on this without support.

    The effect of application features and user-interface analysis on end-user adoption is unknown.

    “Similarly, the effect of application features and the user-interface (UI) analysis on the end-user adoption scenario is unknown.”

  • The paper relies on this without support.

    Subjective tests are the most accurate means of quality estimation.

    “Subjective tests are the most accurate means of quality estimation.”

  • The authors state this as an assumption.

    QoE is a weighted linear combination of the parameters from the QoD and QoI categories.

    “For computing the QoE, an assumption is made that it is a weighted linear combination of the parameters from the QoD and QoI categories.”

  • The authors state this as an assumption.

    QoD is a weighted linear combination of QoD_SC and QoD_HRM.

    “The QoD is in turn calculated as a weighted linear combination of all the parameters in the QoD category (QoD SC and QoD HRM for the present case)”

  • The authors state this as an assumption.

    QoI is a weighted linear combination of PEOU, PU, and RI.

    “whereas the QoI category is also adjusted by its own weights covering all the three parameters (QoD PEOU , QoD PU , and QoD RI ).”

  • The paper relies on this without support.

    SC and HRM are universally available across all fitness trackers or smartwatches irrespective of their price point.

    “First, these two features are universally available across all fitness trackers or smartwatches irrespective of their price point.”

  • The paper relies on this without support.

    Distance travelled and calories burned cannot be measured directly by the sensors and depend on algorithms for their calculation.

    “On the contrary, other features like distance travelled or calories burned cannot be measured directly by the sensors, instead depend on certain algorithms for their calculation.”

  • The authors state this as an assumption.

    QoE is a function of QoD and QoI.

    “As mentioned previously, for this work we hypothesize QoE to be a function of QoD and QoI.”

  • The paper relies on this without support.

    There is no standard measurement method for evaluating the QoE of wearable computing services.

    “we observe that there is no standard measurement method for evaluating the QoE of the wearable computing services.”

  • The paper relies on this without support.

    QoD and QoI attributes are more relevant in the wearable context and the IoT paradigm in general.

    “we propose a generic equation that includes the QoD and QoI attributes, as they are more relevant in the wearable context and the IoT paradigm in general.”

  • The authors state this as an assumption.

    The weightage of different parameters towards the final QoE are different.

    “In this technique, we use a more realistic approach because it is assumed that the weightage of the different parameters towards the final QoE are different.”
11 of 11

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • In [43], authors measure the QoE of a smart city application using the smart city of Jakarta as a use case.

    “In [43], the authors measure the QoE of a smart city application using the smart city of Jakarta as a use case.”

    From QoE in IoT: a vision, survey and future directions (2021) · cites this paper as [43]

    The Introduction and Discussion describe a QoE evaluation of smart-wearables and their companion applications, so the source supports the measurement of QoE. They do not describe a smart-city application or a Jakarta use case; the paper mentions smart cities only as one example of the many kinds of IoT applications. (LLM, conf=0.88)

    Experimental — accuracy not yet measured

  • In [43], they identified quality requirements of a smart city, defined users, computed a mean opinion score based on surveys before and after implementing the IoT application, and performed the difference between after and before implementing the application.

    “In this use case, authors first identified the quality requirements of a smart city.Secondly, they defined the users of this IoT application.Thirdly, they compute a mean opinion score (MOS) based on surveys performed before and after implementing the IoT application.Lastly, the difference between after implementing and before implementing the application is performed.”

    From QoE in IoT: a vision, survey and future directions (2021) · cites this paper as [43]

    Split into 2 atoms — 0 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 1 unverifiable. Expand the claim to see per-atom verdicts.

    Experimental — accuracy not yet measured

  • The quality of data can be expressed in many different ways, and data quality measures may also depend on the aims of the related analysis and project.

    “the quality of data can be expressed in many different ways, and data quality measures may also depend on the aims of the related analysis and project8–13”

    From Data quality evaluation in wearable monitoring (2022) · cites this paper as [11]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Setting C to zero gives the wrong normalization constraint

    Minor

    Equation (5) states that C + D + E = 1. Setting C to 0 therefore gives D + E = 1, not 0.

1 of 1

Cites

Flagged first, then by citations

Research works printed but not linked: 7

  1. 2019, Wearables Worldwide
  2. 2016, Reference Guide to Quality of Experience Assessment Methodologies
  3. 2012, Methodology for the Subjective Assessment of the Quality of Television Pictures
  4. 2008, Subjective video quality assessment methods for multimedia applications
  5. 1998, Subjective Audiovisual Quality Assessment Methods for Multimedia Applications
  6. 2018, The Wearable Database
  7. 10.1016/j.jretconser.2018.03.017 · DOI

Cited by

Sorted by citations

Related

Authors
Pal, D., Vanijja, V., Arpnikanondt, C., Zhang, X., & Papasratorn, B.
Affiliations
  • King Mongkut's University of Technology Thonburi (TH)
  • Wayne State University (US)
ORCID
Venue
IEEE Access
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Published
2019
Volume
7
Pages
64266-64278
ISSN
2169-3536 (ISSN-L)
DOI
10.1109/access.2019.2917061
OpenAlex
W2946543414
Semantic Scholar
ddcd0b953f84179cb0a47920f927e2b7627d4024
Cited by
39
References
52
Citations by year
2026: 3 · 2025: 1 · 2024: 5 · 2023: 6 · 2022: 10 · 2021: 8 · 2020: 4 · 2019: 2
FWCI
5.67
Type
journal article
Language
en
Open access
Licence
ieeexplore.ieee.org/Xplorehelp/downloads/license-information/OAPA.html (vor)
Locations
  • IEEE Access (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
KMUTT Research Fund
Topics
  • Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image and Video Quality Assessment
  • Physical Sciences › Computer Science › Computer Science Applications › Mobile Crowdsensing and Crowdsourcing
Keywords
wearable devices, quality of experience, data quality, information quality, wearable sensors, sensor data accuracy, QoE model, weight distribution, user adoption

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-09.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-09.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review