CoChat

Development and usability testing of a web-based and therapist-assisted coping skills program for managing psychosocial problems in individuals with hand and upper limb injuries: Mixed methods study

Open access

Published version of a preprint

The DOI matches its Crossref record; 116 of the 116 references it cites were found in a registry or an index; 37 of the 74 sources its claims cite were checked for support.

Abstract

Background: Ineffective coping has been linked to prolonged pain, distress, anxiety, and depression after a hand and upper limb injury. Evidence shows that interventions based on cognitive behavioral therapy (CBT) may be effective in improving treatment outcomes, but traditional psychological interventions are resource intensive and unrealistic in busy hand therapy practices. Developing web-based, evidence-based psychological interventions specifically for hand therapy may be feasible in clinical practice and at home with reduced training and travel costs. Hand Therapy Online Coping Skills (HOCOS) is a program developed to supplement traditional hand therapy with therapist-assisted coping skills training based on principles from CBT and the Technology Acceptance Model. Objective: This study aimed to describe the development and assess the usability of HOCOS to support hand therapists in the management of psychosocial problems. Methods: The ADDIE model (Analysis, Design, Development, Implementation, and Evaluation) of system design was applied to create HOCOS. The usability testing of HOCOS involved a 2-stage process. In the first step, heuristic testing with information and communications technology (ICT) experts was completed using two sets of heuristics: Monkman heuristics and the Health Literacy Online (HLO) checklist. The second step involved user testing with hand therapists performing a series of online and face-to-face activities, completing 12 tasks on the website using the think-aloud protocol, completing the system usability scale (SUS) questionnaire, and a semistructured feedback interview in 2 iterative cycles. Descriptive statistics and content analyses were used to organize the data. Results: In total, 4 ICT experts and 12 therapists completed usability testing. The heuristic evaluation revealed 15 of 35 violations on the HLO checklist and 5 of 11 violations on the Monkman heuristics. Initially, hand therapists found 5 tasks to be difficult but were able to complete all 12 tasks after the second cycle of testing. The cognitive interview findings were organized into 6 themes: Task performance, navigation, design esthetics, content, functionality and features, and desire for future use. Usability issues identified were addressed in two iterative cycles. There was good agreement on all items of the SUS. Overall, therapists found that HOCOS was a detailed and helpful learning resource for therapists and patients. Conclusions: We describe the development and usability testing of HOCOS; a new web-based psychosocial intervention for individuals with a hand and upper limb injuries. HOCOS targets psychosocial problems linked to prolonged pain and disability by increasing access to therapist-guided coping skills training. We actively involved target users in the development and usability evaluation of the website. The final website was modified to meet the needs and preferences of the participants. © 2020 Folarin Omoniyi Babatunde, Joy MacDermid, Ruby Grewal, Luciana Macedo, Mike Szekeres.

Overview

CoChat Summary

## TL;DR

The paper describes the development and mixed-methods usability testing of HOCOS, a web-based, therapist-assisted coping skills program for people with hand and upper limb injuries. Usability testing with 4 ICT experts and 12 therapists led to iterative revisions, and the mean SUS score improved from 62.5 to 84.

## Key findings

  • Heuristic testing with 4 ICT experts found 15/35 violations on the Health Literacy Online checklist and 5/11 violations on the Monkman heuristics.
  • In user testing, therapists initially found 5 tasks difficult, but after the second testing cycle they were able to complete all 12 tasks.
  • The mean System Usability Scale (SUS) score improved from 62.5 (SD 8.5) to 84 (SD 8.2), which the authors interpret as moving from good to excellent usability.
  • Cognitive interview findings were organized into 6 themes: task performance, navigation, design aesthetics, content, functionality and features, and desire for future use.

## Methods

  • Mixed-methods usability study using the ADDIE model; this paper reports on step 3 of ADDIE (development/usability evaluation of the prototype).
  • Phase 1: heuristic evaluation by 4 ICT experts using the Monkman heuristics and Health Literacy Online (HLO) checklist.
  • Phase 2: user testing with 12 hand therapists in Ontario, Canada, using 12 tasks, the think-aloud protocol, SUS, and semistructured interviews; data were analyzed with descriptive statistics and content analysis.

## Limitations

  • The authors state the study was conducted among hand therapists in Ontario, and most participants were already comfortable using computers and the internet, which limits generalizability and may not represent patient end users.
  • They also state that snowball sampling was used, increasing the risk of a nonrepresentative sample.
  • The paper explicitly notes they could not test HOCOS with patients because of financial and time constraints; this is a stated limitation and also evident from the sample being clinicians rather than end users.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.2196/17088

Cites

116 of 116 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Figure 1 Diagram depicting the first 3 steps of Assessment or analysis, Design, Development, of the ADDIE Model. HOCOS: Hand Therapy Online Coping Skills; ICT: information and communications technology.
Figure 1 Diagram depicting the first 3 steps of Assessment or analysis, Design, Development, of the ADDIE Model. HOCOS: Hand Therapy Online Coping Skills; ICT: information and communications technology.CC BY · PubMed Central
Figure 2 Navigation workflow of the Hand Therapy Online Coping Skills (HOCOS) training program. HCP: health care professional.
Figure 2 Navigation workflow of the Hand Therapy Online Coping Skills (HOCOS) training program. HCP: health care professional.CC BY · PubMed Central
Figure 3 Screenshot of webpages showing the architecture of the website. FAQs: frequently answered questions; HOCOS: Hand Therapy Online Coping Skills.
Figure 3 Screenshot of webpages showing the architecture of the website. FAQs: frequently answered questions; HOCOS: Hand Therapy Online Coping Skills.CC BY · PubMed Central

Verification

Claims the paper makes

22 Valid · 6 Not valid · 18 Not checked · 67 Own results and interpretations

36 more claims · 3 with their cited sources checked

Create a free account to see all 113 claims and to verify the 7 references still open.

See all claims — free account
  • It may state two separate claims as one. 36
  • Probably not a substantial claim. 46
  • The quote only partly states this claim. 1

Create a free account to see these claims.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Self-check results appear here once an independent check has confirmed them.

Cited works that were retracted

Works this paper cites that were later retracted, from the retraction notices publishers deposit. Major when the notice came out before this paper was published.

1 of 1

Citations

Cites

1 retracted

Show all 116 references

Research works printed but not linked: 3

  1. Monkman, H, 2015, Stud Health Technol Inform, 216, 358
  2. 10.5555/2817912.2817913 · doi:10.5555/2817912.2817913
  3. Chad-Friedman, E, 2017, Arch Bone Jt Surg, 5, 400

Cited by

Metadata

Authors
Babatunde, F. O., MacDermid, J., Grewal, R., Macedo, L., & Szekeres, M.
Affiliations
  • McMaster University (CA)
  • Western University (CA)
Corresponding author
Babatunde, Folarin Omoniyi
Venue
JMIR Human Factors
Published
2020-05-06
Volume
7
Issue
2
Pages
e17088
ISSN
2292-9495 (ISSN-L)
OpenAlex
W3007738972
Cited by
12
References
97
Citations by year
2026: 1 · 2025: 1 · 2024: 2 · 2023: 2 · 2022: 5 · 2021: 1
FWCI
1.81
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • JMIR Human Factors (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Scholarship@Western (Western University) (submittedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • Scholarship@Western (Western University) (submittedVersion)
Funders
Canadian Institutes of Health Research
Topics
  • Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
  • Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
  • Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
Keywords
hand therapy, cognitive behavioral therapy, psychosocial problems, usability evaluation, heuristic evaluation, System Usability Scale, cognitive interviewing, hand therapists, technology acceptance model, ADDIE model

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.