Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index.

An analysis of usability evaluation practices and contexts of use in wearable robotics

Open access
Meyer, Jan Thomas, Gassert, Roger, Lambercy, Olivier · 2021 · Journal of NeuroEngineering and Rehabilitation

Published version of a preprint

Abstract

Background: User-centered design approaches have gained attention over the past decade, aiming to tackle the technology acceptance issues of wearable robotic devices to assist, support or augment human capabilities. While there is a consensus that usability is key to user-centered design, dedicated usability evaluation studies are scarce and clear evaluation guidelines are missing. However, the careful consideration and integration of user needs appears to be essential to successfully develop an effective, efficient, and satisfactory human-robot interaction. It is primarily the responsibility of the developer, to ensure that this users involvement takes place throughout the design process. Methods: Through an online survey for developers of wearable robotics, we wanted to understand how the design and evaluation in actual daily practice compares to what is reported in literature. With a total of 31 questions, we analyzed the most common wearable robotic device applications and their technology maturity, and how these influence usability evaluation practices. Results: A total of 158 responses from a heterogeneous population were collected and analyzed. The dataset representing contexts of use for augmentation (16.5%), assistance (38.0%), therapy (39.8%), as well as few other specific applications (5.7%), allowed for an insightful analysis of the influence of technology maturity on user involvement and usability evaluation. We identified functionality, ease of use, and performance as the most evaluated usability attributes and could specify which measures are used to assess them. Also, we could underline the frequent use of qualitative measures alongside the expected high prevalence of performance-metrics. In conclusion of the analysis, we derived evaluation recommendations to foster user-centered design and usability evaluation. Conclusion: This analysis might serve as state-of-the-art comparison and recommendation for usability studies in wearable robotics. We believe that by motivating for more balanced, comparable and user-oriented evaluation practices, we may support the wearable robotics field in tackling the technology acceptance limitations.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1186/s12984-021-00963-8

Cites

38 of 38 references verified0 citation errors · 0 not checkable

Statistics

Figures

Fig. 1 Context of Use of Wearable Robotic Devices: A General usage purposes: augmentation, assistance, therapy and other. *Other usage purposes reported were: all of the above (n = 3), brain computer interfaces (n = 2), fitness and sports tracking (n =1), training and assistance for surgery (n = 1), benchmarking (n = 1) and user research (n = 1), B Supervision form: We can differentiate between fully supervised, partially-supervised (aided) and unsupervised use of WRD, C Target body areas: The relative frequencies of the four general usage purposes are reported for each body area D Usage environment: The intended use ranges from rather controlled (laboratory, clinic, home) to more dynamic (leisure, military, work) environments E Target population of medical applications: For the medical applications (therapy and assistance) respondents reported the specific target groups. **Other target groups where as: visually impaired, essential tremor, trauma, rhabdomyolysis, peripheral artery disease, first responders (emergency), physical training and exercise, and sports injuries
Fig. 1 Context of Use of Wearable Robotic Devices: A General usage purposes: augmentation, assistance, therapy and other. *Other usage purposes reported were: all of the above (n = 3), brain computer interfaces (n = 2), fitness and sports tracking (n =1), training and assistance for surgery (n = 1), benchmarking (n = 1) and user research (n = 1), B Supervision form: We can differentiate between fully supervised, partially-supervised (aided) and unsupervised use of WRD, C Target body areas: The relative frequencies of the four general usage purposes are reported for each body area D Usage environment: The intended use ranges from rather controlled (laboratory, clinic, home) to more dynamic (leisure, military, work) environments E Target population of medical applications: For the medical applications (therapy and assistance) respondents reported the specific target groups. **Other target groups where as: visually impaired, essential tremor, trauma, rhabdomyolysis, peripheral artery disease, first responders (emergency), physical training and exercise, and sports injuriesCC BY · PubMed Central
Fig. 2 Technology Maturity of Current Wearable Robotic Devices: The Technology Readiness Levels (TRL) of the four general usage purposes, visualized as stacked histogram (n = 143). The TRL were grouped in three Technology Readiness Phases (TRP): “Concept” (TRL 1–3), “Prototype” (TRL 4–7) and “Product” (TRL 8–9)
Fig. 2 Technology Maturity of Current Wearable Robotic Devices: The Technology Readiness Levels (TRL) of the four general usage purposes, visualized as stacked histogram (n = 143). The TRL were grouped in three Technology Readiness Phases (TRP): “Concept” (TRL 1–3), “Prototype” (TRL 4–7) and “Product” (TRL 8–9)CC BY · PubMed Central
Fig. 3 Progression of User Involvement with Technology Maturation. The number of users (target and mock) per Technology Readiness Level (TRL) are shown (n = 143). The size of the circles represents the number of responses within each TRL. For all respondents that only indicated testing with the one specific user group (target or mock), 0 users are shown in the opposite group. The Loess regression lines were fitted to smooth the scatter plot visualization
Fig. 3 Progression of User Involvement with Technology Maturation. The number of users (target and mock) per Technology Readiness Level (TRL) are shown (n = 143). The size of the circles represents the number of responses within each TRL. For all respondents that only indicated testing with the one specific user group (target or mock), 0 users are shown in the opposite group. The Loess regression lines were fitted to smooth the scatter plot visualizationCC BY · PubMed Central
Fig. 4 Analysis of Usability Evaluation Practice: A Evaluation efforts allocation per Technology Readiness Phase (TRP, n = 117): The allocated total of 100 evaluation-effort-points among the usability dimensions effectiveness, efficiency, and satisfaction, per TRP are shown. Paired comparison between the three dimension were analyzed within each TRP, while unpaired, two-sample comparisons between the TRP were calculated. Levels of significance indicated as: * = p < 0.05, ** = p < 0.01, *** = p < 0.001. B List of reported performance-related measurements (PRM, n = 88): *Other PRM described as spatiotemporal metric analysis, eye-tracking recording, number of steps. **Standardized functional measures (SFM) specified: upper limb SFM Box and Block Test (BBT, n = 6), Jebsen-Taylor Hand Function Test (JTHFT, n = 5), Action Research Arm Test (ARAT, n = 4), Chedoke Arm and Hand Activity Inventory (CAHAI, n = 2), Southampton Hand Assessment Procedure (SHAP), Assisting Hand Assessment (AHA), Smart Pegboard, Frenchay Arm Test (FAT), lower limb SFM 10 Meter Walk Test (10MWT, n = 2), 2 Minute Walk Test (2MWT, n = 2), 6 min Walk Test (6MWT, n = 2), Timed-up-and-Go (TUG), general SFM Human-Robot Fluency Metrics, Assessment of Capacity for Myoelectric Control (ACMC), Thermography, Failure Mode and Effects Analysis (FMEA), ISO regulation, fit and tolerance assemblies
Fig. 4 Analysis of Usability Evaluation Practice: A Evaluation efforts allocation per Technology Readiness Phase (TRP, n = 117): The allocated total of 100 evaluation-effort-points among the usability dimensions effectiveness, efficiency, and satisfaction, per TRP are shown. Paired comparison between the three dimension were analyzed within each TRP, while unpaired, two-sample comparisons between the TRP were calculated. Levels of significance indicated as: * = p < 0.05, ** = p < 0.01, *** = p < 0.001. B List of reported performance-related measurements (PRM, n = 88): *Other PRM described as spatiotemporal metric analysis, eye-tracking recording, number of steps. **Standardized functional measures (SFM) specified: upper limb SFM Box and Block Test (BBT, n = 6), Jebsen-Taylor Hand Function Test (JTHFT, n = 5), Action Research Arm Test (ARAT, n = 4), Chedoke Arm and Hand Activity Inventory (CAHAI, n = 2), Southampton Hand Assessment Procedure (SHAP), Assisting Hand Assessment (AHA), Smart Pegboard, Frenchay Arm Test (FAT), lower limb SFM 10 Meter Walk Test (10MWT, n = 2), 2 Minute Walk Test (2MWT, n = 2), 6 min Walk Test (6MWT, n = 2), Timed-up-and-Go (TUG), general SFM Human-Robot Fluency Metrics, Assessment of Capacity for Myoelectric Control (ACMC), Thermography, Failure Mode and Effects Analysis (FMEA), ISO regulation, fit and tolerance assembliesCC BY · PubMed Central
Fig. 5 Questionnaires and Scales Used in Wearable Robotics Usability Evaluation: The 20 most frequently selected questionnaires and scales are displayed (n = 74). Custom-made forms are marked in dark red, modified questionnaires are marked in light red. Abbreviations: LS = Likert Scale, VAS = Visual Analogue Scale, SUS = System Usability Scale, NRS = Numeric Rating Scale, NASA TLX = NASA Task Load Index, QUEST 2.0 = Quebec User Evaluation of Satisfaction with Assistive Technology, mSUS = modified System Usability Scale, RTLX = Raw Task Load Index, ASQ = After Scenario Questionnaire, PUEU = Perceived Usefulness, Perceived Ease of Use, ATD PA = Assistive Technology Device Predisposition Assessment, PSSUQ = Post-Study Usability Questionnaire, COPM = Canadian Occupational Performance Measure, USAT = Usability Scale for Assistive Technology, IMI = Intrinsic Motivation Inventory, USEQ = Usefulness, Satisfaction, Ease of Use Questionnaire, PYTHEIA = Psychometric Scale to Assess the Satisfaction of Users with Assistive Technology, QUIS = The Questionnaire for User interaction Satisfaction. *Other = Borg Scale of Perceived Exertion, Michigan Hand Outcomes Questionnaire, Prosthesis Evaluation Questionnaire, Psychosocial Impact of Assistive Device, Questionnaire to Explore Human Factors and their Technical potential, QuickDASH, SF-36, Telehealthcare Satisfaction Questionnaire - Wearable Technology, Trinity Amputation and Prosthesis Experience Scale, Usability Metric for User Experience, and Embodiment Questionnaire
Fig. 5 Questionnaires and Scales Used in Wearable Robotics Usability Evaluation: The 20 most frequently selected questionnaires and scales are displayed (n = 74). Custom-made forms are marked in dark red, modified questionnaires are marked in light red. Abbreviations: LS = Likert Scale, VAS = Visual Analogue Scale, SUS = System Usability Scale, NRS = Numeric Rating Scale, NASA TLX = NASA Task Load Index, QUEST 2.0 = Quebec User Evaluation of Satisfaction with Assistive Technology, mSUS = modified System Usability Scale, RTLX = Raw Task Load Index, ASQ = After Scenario Questionnaire, PUEU = Perceived Usefulness, Perceived Ease of Use, ATD PA = Assistive Technology Device Predisposition Assessment, PSSUQ = Post-Study Usability Questionnaire, COPM = Canadian Occupational Performance Measure, USAT = Usability Scale for Assistive Technology, IMI = Intrinsic Motivation Inventory, USEQ = Usefulness, Satisfaction, Ease of Use Questionnaire, PYTHEIA = Psychometric Scale to Assess the Satisfaction of Users with Assistive Technology, QUIS = The Questionnaire for User interaction Satisfaction. *Other = Borg Scale of Perceived Exertion, Michigan Hand Outcomes Questionnaire, Prosthesis Evaluation Questionnaire, Psychosocial Impact of Assistive Device, Questionnaire to Explore Human Factors and their Technical potential, QuickDASH, SF-36, Telehealthcare Satisfaction Questionnaire - Wearable Technology, Trinity Amputation and Prosthesis Experience Scale, Usability Metric for User Experience, and Embodiment QuestionnaireCC BY · PubMed Central
Fig. 6 Reflection of Usability Evaluation Practise: The level of agreement for each statement is shown. Disagreement is red, agreement is green. The more the entire bar (100%) is shifted to the left or right, the clearer a trend of agree- or disagreement is observable among the responses
Fig. 6 Reflection of Usability Evaluation Practise: The level of agreement for each statement is shown. Disagreement is red, agreement is green. The more the entire bar (100%) is shifted to the left or right, the clearer a trend of agree- or disagreement is observable among the responsesCC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 37 references

Cited by

See all 60

Metadata

Authors
Meyer, J. T., Gassert, R., & Lambercy, O.
Affiliations
ETH Zurich (CH)
Corresponding author
Meyer, Jan Thomas
Venue
Journal of NeuroEngineering and Rehabilitation
Published
2021-12
Volume
18
Issue
1
Pages
170-170
ISSN
1743-0003 (ISSN-L)
OpenAlex
W4226019611
Cited by
58
References
37
Citations by year
2026: 12 · 2025: 10 · 2024: 18 · 2023: 8 · 2022: 10
FWCI
5.82
Type
journal article
Language
en
Open access
Licence
Locations
  • Journal of NeuroEngineering and Rehabilitation (publishedVersion, cc-by)
  • Research Square (acceptedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • Repository for Publications and Research Data (ETH Zurich) (submittedVersion, cc-by)
  • ETH Zürich Research Collection (cc-by)
Funders
  • vontobel-stiftung
  • national centre of competence in research robotics
  • eth zürich foundation
  • national research foundation, prime minister’s office, singapore under its campus for research excellence and technological enterprise (create) program
Topics
  • Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
  • Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
  • Physical Sciences › Engineering › Biomedical Engineering › Prosthetics and Rehabilitation Robotics
Keywords
wearable robotics, usability evaluation, user-centered design, context of use, technology maturity, human-robot interaction, assistive robotics, rehabilitation robotics, evaluation guidelines, user participation, technology acceptance
MeSH terms
Wearable Electronic Devices, Humans, Surveys and Questionnaires, Robotics

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: An analysis of usability evaluation practices and contexts of use in wearable robotics. https://app.cochat.ai/papers/10.1186/s12984-021-00963-8
The paper
Meyer, J. T., Gassert, R., & Lambercy, O. (2021). An analysis of usability evaluation practices and contexts of use in wearable robotics. Journal of NeuroEngineering and Rehabilitation. https://doi.org/10.1186/s12984-021-00963-8

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.