The DOI matches its Crossref record; 202 of the 202 references it cites were found in a registry or an index.
Do procedures for verbal reporting of thinking have to be reactive? A meta-analysis and recommendations for best reporting methods.
Abstract
Since its establishment, psychology has struggled to find valid methods for studying thoughts and subjective experiences. Thirty years ago, Ericsson and Simon (1980) proposed that participants can give concurrent verbal expression to their thoughts (think aloud) while completing tasks without changing objectively measurable performance (accuracy). In contrast, directed requests for concurrent verbal reports, such as explanations or directions to describe particular kinds of information, were predicted to change thought processes as a consequence of the need to generate this information, thus altering performance. By comparing performance of concurrent verbal reporting conditions with their matching silent control condition, Ericsson and Simon found several studies demonstrating that directed verbalization was associated with changes in performance. In contrast, the lack of effects of thinking aloud was merely suggested by a handful of experimental studies. In this article, Ericsson and Simon's model is tested by a meta-analysis of 94 studies comparing performance while giving concurrent verbalizations to a matching condition without verbalization. Findings based on nearly 3,500 participants show that the "think-aloud" effect size is indistinguishable from zero (r = -.03) and that this procedure remains nonreactive even after statistically controlling additional factors such as task type (primarily visual or nonvisual). In contrast, procedures that entail describing or explaining thoughts and actions are significantly reactive, leading to higher performance than silent control conditions. All verbal reporting procedures tend to increase times to complete tasks. These results suggest that think-aloud should be distinguished from other methods in future studies. Theoretical and practical implications are discussed.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Items 201–202 of 202
Verbal Compared with Manipulative Solution of an Apparatus-Problem
Ray, W. S. (1957) · The American Journal of Psychology · doi:10.2307/1419339 · Cited by 13
Assessment of Student Problem-Solving on Ill-Defined Tasks
Leighton, J. P., Rogers, W. T., & Maguire, T. O. (1999) · Alberta Journal of Educational Research · doi:10.55016/ojs/ajer.v45i4.54712 · Cited by 11
Cited by
Handbook of Self-Regulation of Learning and Performance
(2017) · Routledge eBooks · doi:10.4324/9781315697048 · Cited by 1,816
Measuring Risk Literacy: The Berlin Numeracy Test
Cokely, E. T., Galesic, M., Schulz, E., Ghazal, S., & Garcia-Retamero, R. (2012) · Judgment and Decision Making · doi:10.1017/s1930297500001819 · Cited by 745
Navigating Into the Future or Driven by the Past
Seligman, M. E. P., Railton, P., Baumeister, R. F., & Sripada, C. (2013) · Perspectives on Psychological Science · doi:10.1177/1745691612474317 · Cited by 720
Keystroke Logging in Writing Research: Using Inputlog to Analyze and Visualize Writing Processes
Leijten, M., & Van Waes, L. (2013) · Written Communication · doi:10.1177/0741088313491692 · Cited by 513
The think aloud method: what is it and how do I use it?
Eccles, D. W., & Arsal, G. (2017) · Qualitative Research in Sport, Exercise and Health · doi:10.1080/2159676x.2017.1331501 · Cited by 421
Metadata
- Authors
- Fox, M. C., Ericsson, K. A., & Best, R.
- Venue
- Psychological Bulletin
- Published
- 2011-03
- Volume
- 137
- Issue
- 2
- Pages
- 316-344
- ISSN
- 0033-2909 (ISSN-L)
- 1939-1455
- OpenAlex
- W2077979711
- Cited by
- 424
- References
- 202
- FWCI
- 10.86
- Type
- journal article
- Language
- en
- Open access
- closed
- Funders
- Florida State University Foundation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.