CoChat

Exploring Think-Alouds in Usability Testing: An International Survey

McDonald, Sharon, Edwards, Helen M., Zhao, Tingting · 2012 · IEEE Transactions on Professional Communication
Cited by 122

The DOI matches its Crossref record; 3 of the 3 references it cites were found in a registry or an index.

Abstract

Research problem: The study explored think-aloud methods usage within usability testing by examining the following questions: How, and why is the think-aloud method used? What is the gap between theory and practice? Where does this gap occur? Literature review: The review informed the survey design. Usability research based on field studies and empirical tests indicates that variations in think-aloud procedures may reduce test reliability. The guidance offered on think-aloud procedures within a number of handbooks on usability testing is also mixed. This indicates potential variability in practice, but how much and for what reasons is unknown. Methodology: An exploratory, qualitative survey was conducted using a web-based questionnaire (during November-December 2010). Usability evaluators were sought via emails (sent to personal contacts, usability companies, conference attendees, and special interest groups) to be cascaded to the international community. As a result we received 207 full responses. Descriptive statistics and thematic coding were used to analyze the data sets. Results: Respondents found the concurrent technique particularly suited usability testing as it was fast, easy for users to relate to, and requires limited resources. Divergent practice was reported in terms of think-aloud instructions, practice, interventions, and the use of demonstrations. A range of interventions was used to better understand participant actions and verbalizations, however, respondents were aware of potential threats to test reliability, and took steps to reduce this impact. Implications: The reliability considerations underpinning the classic think-aloud approach are pragmatically balanced against the need to capture useful data in the time available. A limitation of the study is the focus on the concurrent method; other methods were explored but the differences in application were not considered. Future work is needed to explore the impact of divergent use of think-aloud instructions, practice tasks, and the use of demonstrations on test reliability.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/tpc.2011.2182569

Cites

3 of 3 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Cited by

See all 122

Metadata

Authors
McDonald, S., Edwards, H. M., & Zhao, T.
Venue
IEEE Transactions on Professional Communication
Published
2012-03
Volume
55
Issue
1
Pages
2-19
ISSN
  • 0361-1434 (ISSN-L)
  • 1558-1500
OpenAlex
W1978632524
Cited by
122
References
3
FWCI
14.17
Type
journal article
Language
en
Open access
closed

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.