CoChat
Sign Up
Cited by 675Mobile Health and mHealth ApplicationsUsability and User Interface DesignHealth, Technology, Consumer Behavior

The mHealth App Usability Questionnaire (MAUQ): Development and Validation Study

Open access
Zhou, Leming, Bao, Jie, Setiawan, I Made Agus, Saptono, Andi, Parmanto, Bambang·2019·JMIR mHealth and uHealth
DOI 10.2196/11500PMC PMC6482399

Published version of a preprint

Open this paper in CoChat — free account

Abstract

BACKGROUND: After a mobile health (mHealth) app is created, an important step is to evaluate the usability of the app before it is released to the public. There are multiple ways of conducting a usability study, one of which is collecting target users' feedback with a usability questionnaire. Different groups have used different questionnaires for mHealth app usability evaluation: The commonly used questionnaires are the System Usability Scale (SUS) and Post-Study System Usability Questionnaire (PSSUQ). However, the SUS and PSSUQ were not designed to evaluate the usability of mHealth apps. Self-written questionnaires are also commonly used for evaluation of mHealth app usability but they have not been validated. OBJECTIVE: The goal of this project was to develop and validate a new mHealth app usability questionnaire. METHODS: An mHealth app usability questionnaire (MAUQ) was designed by the research team based on a number of existing questionnaires used in previous mobile app usability studies, especially the well-validated questionnaires. MAUQ, SUS, and PSSUQ were then used to evaluate the usability of two mHealth apps: an interactive mHealth app and a standalone mHealth app. The reliability and validity of the new questionnaire were evaluated. The correlation coefficients among MAUQ, SUS, and PSSUQ were calculated. RESULTS: In this study, 128 study participants provided responses to the questionnaire statements. Psychometric analysis indicated that the MAUQ has three subscales and their internal consistency reliability is high. The relevant subscales correlated well with the subscales of the PSSUQ. The overall scale also strongly correlated with the PSSUQ and SUS. Four versions of the MAUQ were created in relation to the type of app (interactive or standalone) and target user of the app (patient or provider). A website has been created to make it convenient for mHealth app developers to use this new questionnaire in order to assess the usability of their mHealth apps. CONCLUSIONS: The newly created mHealth app usability questionnaire-MAUQ-has the reliability and validity required to assess mHealth app usability.

Verified 2026-10-06 · 1 reference not checked yet

AI snapshot

Written from the paper’s full text.

TL;DR

The MAUQ was developed as an mHealth app–specific usability questionnaire with four versions based on app type (interactive/standalone) and target user (patient/provider). In a validation study with 128 participants and two mHealth apps, it showed high internal consistency and good criterion/construct validity against the PSSUQ and SUS.

Key findings

  • The MAUQ has three subscales. For the 21-item interactive patient version, Cronbach alpha was 0.932 overall; subscales were ease of use and satisfaction (8 items, alpha=0.895), system information arrangement (6 items, alpha=0.829), and usefulness (7 items, alpha=0.900).
  • For the 18-item standalone patient version, Cronbach alpha was 0.914 overall; subscales were ease of use (5 items, alpha=0.847), interface and satisfaction (7 items, alpha=0.908), and usefulness (6 items, alpha=0.717).
  • Overall MAUQ scores correlated strongly with PSSUQ (r=0.8448) and correlated with SUS (r=0.6425); PSSUQ and SUS correlated at r=0.6703.
  • None of the demographic factors examined (age, gender, race, education, income, marital status, living place, occupation, employment status, and smart device/mobile app experience) had a statistically significant impact on individual MAUQ statements or overall MAUQ score (P>.05 in all cases).

Methods

  • Questionnaire development drew from 1271 articles, 125 questionnaire-based mHealth app usability studies, and 38 questionnaires, yielding 312 unique statements; after preprocessing and expert review, 53 statements remained, then were reduced to 21 statements for interactive apps and 18 for standalone apps.
  • A usability study recruited 128 participants from the Greater Pittsburgh area who used an interactive app (iMHere 2.0) and a standalone app (Fitbit app version 2.36 on iPad Air 2), completed tasks, and responded to MAUQ, PSSUQ, and SUS via Qualtrics.
  • Analysis included exploratory factor analysis with maximum likelihood extraction and quartimin oblique rotation, a 0.32 factor-loading guideline, Cronbach alpha for internal consistency, and correlation coefficients among MAUQ, PSSUQ, and SUS using R 3.3 and IBM SPSS version 24.

Limitations

  • Stated in the paper: the study was performed in the greater Pittsburgh area, had more female participants, and was not an exact reflection of the US population; the N=128 sample was sufficient for EFA but larger samples would provide more reliable results.
  • Stated in the paper: selection criteria excluded some potential participants, such as people older than 65 years or with low education; only two mHealth apps were tested; and only patient versions of the MAUQ were tested, so provider versions were not explicitly evaluated.
  • Stated in the paper and evident from the text: some items were retained despite low factor loadings or cross-loading, and future studies with larger samples and other mHealth apps were recommended to further evaluate these statements and the provider versions.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.2196/11500

Cites

32 of 33 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 22 cited sources checked · 0 supported · 0 flagged

  • Findingreasoning

    The ease of learning statement was retained in the first construct of the interactive MAUQ despite its low factor loadings.

    “Therefore, we decided to retain this statement in the first construct of the questionnaire.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: The ease of learning statement is very important for a mobile app usability study and is closely related to ease of use and interface design.
  • Findingown data

    The distribution of MAUQ data was not normal according to both the Shapiro-Wilk test and the Kolmogorov-Smirnov test (P<.01).

    “The distribution of data from the MAUQ-based usability study was not normal according to both the Shapiro-Wilk test and the Kolmogorov-Smirnov test, where P<.01.”
    Quote in the paperSubstantial claim
  • Findingown data

    The interactive MAUQ's ease of use and satisfaction subscale had a Cronbach alpha of 0.895 (8 items).

    “Ease of use and satisfaction (alpha=0.895), 8 items (MAUQ_E)”
    Quote in the paperSubstantial claim
  • Findingown data

    None of the demographic factors had a statistically significant impact on the answers to individual MAUQ statements or the overall MAUQ score (P>.05 in all cases).

    “The test results indicated that none of the demographic factors (age, gender, race, education, income, marital status, living place, occupation, employment status, and experience using smart device and mobile apps) had a statistically significant impact on the answers to the individual statements or the overall score on the MAUQ (P>.05 in all cases).”
    Quote in the paperNot a substantial claim
  • Findingown data

    The MAUQ's internal consistency reliability is high.

    “Psychometric analysis indicated that the MAUQ has three subscales and their internal consistency reliability is high.”
    • Also stated in: § Psychometric Analysis Results · plain
      “Table 2 Exploratory factor analysis results for the 21 items on the mHealth App Usability Questionnaire designed for interactive mHealth apps (overall Cronbach alpha=0.932, 21 items).”
    Quote in the paperSubstantial claim
  • Classified as probably not a substantial claim. 32
  • May combine two claims. 27

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    A website has been created to make it convenient for mHealth app developers to use the MAUQ to assess the usability of their mHealth apps.

    “A website has been created to make it convenient for mHealth app developers to use this new questionnaire in order to assess the usability of their mHealth apps.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    After expert review, 53 statements remained in the questionnaire, 19 of which needed wording adjusted.

    “After this step, 53 statements remained in the questionnaire, 19 of which needed the wording to be adjusted.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    312 unique questionnaire statements were collected from the 38 questionnaires.

    “We collected 312 unique questionnaire statements (including similar statements with different wording) from these 38 questionnaires and arranged them into categories according to the general guidelines for usability assessment and the unique features of mobile apps.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Findingown data

    In this study, 128 study participants provided responses to the questionnaire statements.

    “In this study, 128 study participants provided responses to the questionnaire statements.”
    • Also stated in: § Participants · plain
      “In total, 128 participants were recruited from the Greater Pittsburgh area for the study.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingnone given

    The research team decided to reduce the number of statements to 21 for interactive mHealth apps and 18 for standalone mHealth apps.

    “At the end of the discussion, the entire research team decided to reduce the number of statements for interactive mHealth apps to 21 and the number of statements for standalone mHealth apps to 18.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingreasoning

    After preprocessing, 140 statements remained.

    “The remaining 140 statements were then distributed to seven researchers with extensive experience in mobile app usability studies (referred to as “usability experts” in the following descriptions) to collect their feedback on the relevance and clarity of these statements on a scale of 1-4, where 1 indicates no relevance or clarity and 4 indicates high relevance or clarity.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    Four versions of the MAUQ were created in relation to the type of app (interactive or standalone) and target user of the app (patient or provider).

    “Four versions of the MAUQ were created in relation to the type of app (interactive or standalone) and target user of the app (patient or provider).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The Cronbach alpha values for MAUQ_E, MAUQ_S, and MAUQ_U were 0.895, 0.829, and 0.900, respectively.

    “Their Cronbach alpha values were 0.895, 0.829, and 0.900, respectively, which indicate strong internal consistency.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingreasoning

    The Fitbit app is considered a standalone mHealth app.

    “Therefore, this Fitbit app is considered a standalone mHealth app.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingreasoning

    The iMHere 2.0 mobile app is considered an interactive mHealth app.

    “Therefore, this mobile app in iMHere 2.0 is considered an interactive mHealth app.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingreasoning

    MAUQ_U is correlated with the three subscales of the PSSUQ, but not as strongly as the other two MAUQ subscales.

    “MAUQ_U is correlated with the three subscales of the PSSUQ, but not as strongly as the other two MAUQ subscales.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingreasoning

    The MAUQ provides four versions for two types of target users (patients and providers) and two major types of mHealth apps (standalone and interactive), allowing users to choose the version that fits their needs.

    “In the MAUQ, we provide four versions for two types of target users (patients and providers) and two major types of mHealth apps (standalone and interactive), which allows MAUQ users to choose the version that fits their needs.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Of the 21 statements in the final MAUQ for interactive mobile apps, 5 were highly similar to statements in multiple other questionnaires; 10 were similar to statements in a few other questionnaires but more specific to mobile app-based health care activities; and 6 were unique to the MAUQ.

    “The result indicated that 5 statements in the MAUQ were highly similar to the ones in multiple other questionnaires; 10 statements in the MAUQ were similar to the ones in a few other questionnaires, but more specific to mobile app–based health care activities; and the remaining 6 statements were unique to the MAUQ.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingnone given

    The MAUQ subscale with features unique to mHealth apps had a weaker correlation with PSSUQ subscales, which is expected.

    “For the MAUQ subscale, with features unique to mHealth apps, the correlation was not that strong with any of the PSSUQ subscales, which is expected.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    High data-entry burden is a usability issue.

    “Among these factors, high data-entry burden is clearly a usability issue, while loss of interest may also be triggered by poor usability of an app.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    In criterion validity evaluation, a correlation coefficient as low as 0.30 or 0.40 is sufficient.

    “In the criterion validity evaluation, a correlation coefficient as low as 0.30 or 0.40 is sufficient.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [31] Sauro J Lewis J Quantifying the User Experience: Practical Statistics for User Research 2019 Burlington, MA Morgan Kaufmann

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Well-designed mHealth apps can empower patients, improve medication adherence, and reduce the cost of health care.

    “A number of research studies have been performed on mHealth apps, and the results have indicated that well-designed mHealth apps can empower patients, improve medication adherence, and reduce the cost of health care [3-6].”
    Quote in the paperNot a substantial claim0 of 4 sources checked
    • cites [3] Parmanto B Pramana G Yu DX Fairman AD Dicianno BE McCue MP iMHere: A Novel mHealth System for Supporting Self-Care in Management of Complex and Chronic Conditions JMIR Mhealth Uhealth 2013 07 11 1 2 e10 10.2196/mhealth.2391 25100682 v1i2e10 25100682 PMC4114477

      Not checked in the free run.

    • cites [4] Fairman AD Dicianno BE Datt N Garver A Parmanto B McCue M Outcomes of Clinicians, Caregivers, Family Members and Adults with Spina Bifida Regarding Receptivity to use of the iMHere mHealth Solution to Promote Wellness Int J Telerehabil 2013 06 5 1 3 16 10.5195/ijt.2013.6116 25945209 v5n1-art-10.5195-ijt.2013.6116 PMC4296836 25945209

      Not checked in the free run.

    • cites [5] Seto E Leonard KJ Cafazzo JA Barnsley J Masino C Ross HJ Perceptions and experiences of heart failure patients and clinicians on the use of mobile phone-based telemonitoring J Med Internet Res 2012 14 1 e25 10.2196/jmir.1912 22328237 v14i1e25 22328237 PMC3374534

      Not checked in the free run.

    • cites [6] Seto E Leonard KJ Cafazzo JA Barnsley J Masino C Ross HJ Mobile phone-based telemonitoring for heart failure management: a randomized controlled trial J Med Internet Res 2012 14 1 e31 10.2196/jmir.1909 22356799 v14i1e31 22356799 PMC3374537 source

      Not checked in the free run.

  • May combine two claims.

    Premisereasoning

    In published questionnaire-based mobile app usability studies, authors who created their own questionnaires typically recruited only a small number of participants, so the data were not sufficient for reliable psychometric analysis.

    “because the major focus of those usability studies was not on validation of their self-written questionnaires, the authors typically only recruited a small number of study participants; therefore, the obtained data were not sufficient for a reliable psychometric analysis.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationreasoning

    Loss of interest may be triggered by poor usability of an app.

    “Among these factors, high data-entry burden is clearly a usability issue, while loss of interest may also be triggered by poor usability of an app.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    In construct validity evaluation, a low correlation is good for divergent validity and a high correlation is good for convergent validity.

    “In the construct validity evaluation, a low correlation is good for divergent validity and a high correlation is good for convergent validity.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [31] Sauro J Lewis J Quantifying the User Experience: Practical Statistics for User Research 2019 Burlington, MA Morgan Kaufmann

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisecitation

    Roughly half of mHealth app users stop using some mHealth apps for various reasons such as hidden costs, high data-entry burden, and loss of interest.

    “However, a previous study pointed out that roughly half of mHealth app users stop using some mHealth apps for various reasons such as hidden costs, high data-entry burden, and loss of interest [7].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [7] Krebs P Duncan DT Health App Use Among US Mobile Phone Owners: A National Survey JMIR Mhealth Uhealth 2015 11 3 4 e101 10.2196/mhealth.4924 26537656 v3i4e101 26537656 PMC4704953

      Not checked in the free run.

  • May combine two claims.

    Interpretationown data

    The MAUQ has the reliability and validity required to assess mHealth app usability.

    “The newly created mHealth app usability questionnaire—MAUQ—has the reliability and validity required to assess mHealth app usability.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    Four different versions of the mHealth app usability questionnaire are needed to evaluate the usability of mHealth apps designed for different users (patients or health care providers) and different interaction modes (interactive or standalone).

    “Hence, four different versions of the mHealth app usability questionnaire are needed to evaluate the usability of mHealth apps designed for different users (patients or health care providers) and different interaction modes (interactive or standalone).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation

    People who meet the selection criteria are the majority of smart device owners and mobile app users and are good candidates for providing valuable information in this usability study.

    “People who meet these selection criteria are the majority of smart device owners and mobile app users [30], and therefore, they are good candidates for providing valuable information in this usability study.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [30] Pew Research Center Pew Research Center 2018 2018-02-12 Mobile Fact Sheet http://www.pewinternet.org/fact-sheet/mobile/

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    There is a need to create a reliable usability questionnaire specifically for mobile apps.

    “Therefore, there is a need to create a reliable usability questionnaire specifically for mobile apps.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    The correlation coefficient values show the criterion validity and construct validity of the MAUQ.

    “These correlation coefficient values show the criterion validity and construct validity of the MAUQ.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    One recently published work adjusted an existing usability questionnaire to apply it for mHealth apps.

    “One recently published work adjusted an existing usability questionnaire to apply it for mHealth apps [12].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [12] Schnall R Cho H Liu J Health Information Technology Usability Evaluation Scale (Health-ITUES) for Usability Assessment of Mobile Health Technology: Validation Study JMIR Mhealth Uhealth 2018 01 05 6 1 e4 10.2196/mhealth.8851 29305343 v6i1e4 29305343 PMC5775483

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    A simpler user version of MARS (uMARS) was created for the general population for the same purpose.

    “Shortly thereafter, a simpler user version of MARS (uMARS) was created for the general population for the same purpose [37].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [37] Stoyanov SR Hides L Kavanagh DJ Wilson H Development and Validation of the User Version of the Mobile Application Rating Scale (uMARS) JMIR Mhealth Uhealth 2016 06 10 4 2 e72 10.2196/mhealth.5849 27287964 v4i2e72 27287964 PMC4920963 source

      Not checked in the free run.

  • May combine two claims.

    Study design and conductown data

    All participants were randomly selected from eligible candidates, but significantly more female mobile app users expressed interest in this study.

    “All the participants were randomly selected from eligible candidates, but there were significantly more female mobile app users who expressed interest in this study.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    The PSSUQ version 3 used in this study had 16 items, with items 1-6 as PSSUQ1, items 7-12 as PSSUQ2, and items 13-15 as PSSUQ3.

    “The PSSUQ version 3 used in this study had 16 items, where items 1-6 were the first subscale (PSSUQ1), items 7-12 were the second subscale (PSSUQ2), and items 13-15 were the third subscale (PSSUQ3) [14,31].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [14] Lewis JR Psychometric Evaluation of the PSSUQ Using Data from Five Years of Usability Studies International Journal of Human-Computer Interaction 2002 09 14 3-4 463 488 10.1080/10447318.2002.9669130

      Not checked in the free run.

    • cites [31] Sauro J Lewis J Quantifying the User Experience: Practical Statistics for User Research 2019 Burlington, MA Morgan Kaufmann

      Not checked in the free run.

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    Cronbach alpha values of 0.7-0.8 are acceptable, while a value of around 0.9 is excellent.

    “For research and exploratory studies, Cronbach alpha values of 0.7-0.8 are acceptable, while a value of around 0.9 is excellent [34].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [34] Landauer T Helander M Landauer TK Prabhu P Behavioral research methods in human-computer interaction Handbook of Human-Computer Interaction. 2nd Edition 1997 Elsevier, Inc Amsterdam, Netherlands 203 227

      Not checked in the free run.

  • May combine two claims.

    Study design and conductreasoning

    The sample size was much bigger than typical usability studies, making it feasible to perform an exploratory factor analysis.

    “This sample size is much bigger than typical usability studies; hence, it was feasible for an EFA.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    A mobile app rating scale (MARS) was created for assessing the quality of mHealth apps, which is broader than usability.

    “A few years ago, a mobile app rating scale (MARS) was created for assessing the quality of mHealth apps [36], which is broader than usability.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [36] Stoyanov SR Hides L Kavanagh DJ Zelenko O Tjondronegoro D Mani M Mobile app rating scale: a new tool for assessing the quality of health mobile apps JMIR Mhealth Uhealth 2015 03 3 1 e27 10.2196/mhealth.3422 25760773 v3i1e27 25760773 PMC4376132

      Not checked in the free run.

  • May combine two claims.

    Study design and conductreasoning

    The study population was not an exact reflection of the US population, but it did indicate the distribution of mobile app users who were interested in using mHealth apps to take care of their health.

    “In other words, this study population was not an exact reflection of the US population; however, it did indicate the distribution of mobile app users who were interested in using mHealth apps to take care of their health.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    Customization of statements in a usability questionnaire creates challenges for questionnaire users, especially inexperienced ones, and makes the reliability and validity of the customized usability questionnaire questionable.

    “Although customization makes it convenient to measure unique features offered by an individual health information technology system or mHealth app, it also creates challenges for questionnaire users, especially ones who are not experienced in questionnaire development, and this, in turn, will make the reliability and validity of the customized usability questionnaire questionable.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given

    The website created for the MAUQ makes administration of the usability questionnaire easy.

    “Moreover, the website created for the MAUQ makes the administration of the usability questionnaire easy.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationreasoning

    The participants selected were representative of the majority of mHealth app users and ones who could provide the most reliable assessment on the questionnaire.

    “However, since the purpose of this study was to evaluate the newly created usability questionnaire, the participants selected were representative of the majority of mHealth app users and ones who could provide the most reliable assessment on the questionnaire.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The study protocol was approved by the institutional review board office at the University of Pittsburgh.

    “This study protocol was approved by the institutional review board office at the University of Pittsburgh.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    The actual study participants were randomly selected from the list of eligible candidates.

    “All data of the eligible candidates were stored in one Excel file, and the actual study participants were randomly selected from this list.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    Both MARS and uMARS included some usability components such as engagement, functionality, and information quality, but they were not specifically designed to study usability with end users.

    “Both MARS and uMARS included some usability components such as engagement, functionality, and information quality; however, they were not specifically designed to study usability with end users.”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [36] Stoyanov SR Hides L Kavanagh DJ Zelenko O Tjondronegoro D Mani M Mobile app rating scale: a new tool for assessing the quality of health mobile apps JMIR Mhealth Uhealth 2015 03 3 1 e27 10.2196/mhealth.3422 25760773 v3i1e27 25760773 PMC4376132 source

      Not checked in the free run.

    • cites [37] Stoyanov SR Hides L Kavanagh DJ Wilson H Development and Validation of the User Version of the Mobile Application Rating Scale (uMARS) JMIR Mhealth Uhealth 2016 06 10 4 2 e72 10.2196/mhealth.5849 27287964 v4i2e72 27287964 PMC4920963 source

      Not checked in the free run.

  • May combine two claims.

    Interpretationown data

    The data analysis results indicated that the MAUQ has strong construct validity and criterion validity, and the internal consistency of the three subscales and the entire questionnaire is high.

    “The data analysis results indicated that the MAUQ has strong construct validity and criterion validity, and the internal consistency of the three subscales and the entire questionnaire is high.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    In that study, the authors evaluated the psychometric properties of the existing usability questionnaire designed for health information technology systems by analyzing usability study data obtained from a group of patients with HIV.

    “In that study, the authors evaluated the psychometric properties of the existing usability questionnaire designed for health information technology systems by analyzing usability study data obtained from a group of patients with HIV.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [12] Schnall R Cho H Liu J Health Information Technology Usability Evaluation Scale (Health-ITUES) for Usability Assessment of Mobile Health Technology: Validation Study JMIR Mhealth Uhealth 2018 01 05 6 1 e4 10.2196/mhealth.8851 29305343 v6i1e4 29305343 PMC5775483

      Not checked in the free run.

  • May combine two claims.

    Study design and conductreasoning

    The study was performed in the greater Pittsburgh area, but approximately one-third of the study participants were undergraduate and graduate students from different states, making the conclusions generalizable within certain limitations.

    “The study was performed in the greater Pittsburgh area, but approximately one-third of the study participants were undergraduate and graduate students from different states, making the conclusions obtained in this study generalizable within certain limitations.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    The correlation coefficients among the MAUQ, PSSUQ, SUS, and their subscales indicate the criterion validity and construct validity of the MAUQ for standalone apps.

    “The correlation coefficients among MAUQ, PSSUQ, SUS, and their subscales also indicate the criterion validity and construct validity of the MAUQ.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    The target users of MARS were experts in one field (researchers, clinicians, and other professionals) who wanted to identify high-quality mobile apps in that field.

    “The target users of MARS were experts in one field (researchers, clinicians, and other professionals) who wanted to identify high-quality mobile apps in that field.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [36] Stoyanov SR Hides L Kavanagh DJ Zelenko O Tjondronegoro D Mani M Mobile app rating scale: a new tool for assessing the quality of health mobile apps JMIR Mhealth Uhealth 2015 03 3 1 e27 10.2196/mhealth.3422 25760773 v3i1e27 25760773 PMC4376132

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    The study had more female participants.

    “In this study, we had more female participants.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    0.32 was used as a guiding value for evaluating factor loadings.

    “Here, 0.32 was used as a guiding value for the evaluation [33].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [33] Tabachnick B Fidell L Using Multivariate Statistics 2001 Boston Allyn and Bacon

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    The four versions make it feasible for others to easily choose one version for their usability study according to the desired context.

    “This makes it feasible for others to easily choose one version for their usability study according to the desired context.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Study design and conductreasoning

    In this study, participants were required to have certain characteristics (high school or higher education, age 18-65, some experience using mHealth apps), which excluded some potential participants such as people older than 65 or with low education.

    “In this study, we required that the study participants had certain characteristics, such as a high school or higher education, age between 18 and 65 years, and some experience using mHealth apps. This excluded some potential participants, for instance, people older than 65 years or people with a low education level.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The SUS standard score conversion procedure converts each participant's answers to one score between 0 and 100.

    “For the SUS, we used the standard score conversion procedure to convert each study participant’s answers to one score between 0 and 100 [31].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [31] Sauro J Lewis J Quantifying the User Experience: Practical Statistics for User Research 2019 Burlington, MA Morgan Kaufmann

      Not checked in the free run.

50 of 50

  • The authors state this as an assumption.

    The MAUQ factors are not totally independent.

    “We expected multiple factors for the MAUQ and that these factors would not be totally independent.”

  • The authors state this as an assumption.

    The ease of learning statement is very important for a mobile app usability study and is closely related to ease of use and interface design.

    “however, according to our experience working with usability studies and evidence from numerous other previous studies, we believe this statement is very important for a mobile app usability study and is closely related to the ease of use statement and the design of the interface.”

  • The paper relies on this without support.

    There is no single highly reliable usability questionnaire specifically designed for mHealth apps.

    “According to our knowledge, there is no single highly reliable usability questionnaire specifically designed for mHealth apps.”

  • The paper relies on this without support.

    The differences between the patient versions and the provider versions of the MAUQ are not significant.

    “although we believe the differences in the versions for patients and the versions for providers are not significant”
4 of 4

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

  • Percentage reported for not-employed participants is incorrect

    Minor

    With 128 participants, 29 not-employed participants represent 22.65625%, not 21.9%.

    Reported 21.9 · recomputed 22.65625

  • Occupation categories account for only 126 participants

    Minor

    The five occupation counts sum to 126, whereas Table 1 states N=128; no missing-response category is shown.

    Reported 128 · recomputed 126

2 of 2

Cites

Flagged first, then by citations

Research works printed but not linked: 5

  1. Ntawanga, F, 2015, Afr J Inf Syst, 7, 30
  2. Lund, A, 2001, Usability Interface, 8, 3
  3. Coleman, N, 1993, SUMI (Software Usability Measurement Inventory) as a knowledge elicitation tool for improving usability
  4. Hart, S, 1988, Human Mental Workload
  5. Costello, AB, 2005, Practical Assessment Research & Evaluation, 10, 1

Cited by

Sorted by citations

Related

Authors
Zhou, L., Bao, J., Setiawan, I. M. A., Saptono, A., & Parmanto, B.
Affiliations
  • University of Pittsburgh (US)
  • Udayana University (ID)
Corresponding author
Zhou, Leming
ORCID
Venue
JMIR mHealth and uHealth
Publisher
JMIR Publications Inc.
Published
2019-04-11
Volume
7
Issue
4
Pages
e11500
ISSN
2291-5222 (ISSN-L)
DOI
10.2196/11500
PMC
PMC6482399
OpenAlex
W2929198708
Semantic Scholar
67055536ec909217aaeb87633d6f50fc4c3f718a
Cited by
675
References
33
Citations by year
2026: 112 · 2025: 160 · 2024: 117 · 2023: 122 · 2022: 92 · 2021: 42 · 2020: 26 · 2019: 4
FWCI
55.90
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • JMIR mhealth and uhealth (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
National Center for Chronic Disease Prevention and Health Promotion (U48 DP000064)
Topics
  • Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
  • Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
  • Health Sciences › Health Professions › Health Information Management › Health, Technology, Consumer Behavior
Keywords
usability questionnaires, questionnaire development, questionnaire validation, System Usability Scale, Post-Study System Usability Questionnaire, psychometric properties, usability evaluation
MeSH terms
Adolescent, Adult, Aged, Female, Humans, Male, Middle Aged, Pennsylvania, Psychometrics, Surveys and Questionnaires, Technology Assessment, Biomedical, Reproducibility of Results, Telemedicine, Validation Studies as Topic

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-06.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-06.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review