A Comparison of Concurrent and Retrospective Verbal Protocol Analysis
Abstract
In verbal protocol analysis, verbalization can occur either during decision making (concurrent data) or after (retrospective data). Although both methods have advantages and disadvantages, no empirical research has focused on a direct comparison. This study compared the effectiveness of concurrent and retrospective data for revealing the human decision making process. In general, the concurrent protocol analysis method outperformed the retrospective method. Not only was the number of concurrent protocol segments elicited higher than that of retrospective protocol segments, but concurrent data provided more insights into the decision-making steps occurring between stimulus introduction and the final choice outcome. However, retrospective protocols offer an interesting advantage: More statements about the final choice are provided in retrospective protocols than in concurrent protocols.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Writers in the with-suggestions condition were randomly assigned to a system trained on a positive or negative review corpus.
“In the with-suggestions condition, the writers were randomly assigned to a suggestion system trained on a corpus of movie reviews with a mean rating of either 2.5 or 8.5 (the Corpus Rating) [44].”
From Interacting with Next-Phrase Suggestions: How Suggestion Systems Aid and Influence the Cognitive Processes of Writing (2023) · cites this paper as [44]
Concurrent think-aloud yields more protocol segments and insight into steps before the final decision, while retrospective protocols give more insight into the final decision.
“While the concurrent think-aloud method elicits more protocol segments and insight into the steps leading to the final decision, retrospective protocols provide more insight into the final decision [44].”
From Interacting with Next-Phrase Suggestions: How Suggestion Systems Aid and Influence the Cognitive Processes of Writing (2023) · cites this paper as [44]
Cited by
Sorted by citations
Servant, friend or master? The relationships users build with voice-controlled smart devices
Schweitzer, F., Belk, R., Jordan, W., & Ortner, M. (2019) · Journal of Marketing Management · doi:10.1080/0267257x.2019.1596970 · Cited by 246 · see citing works
Cheng, V. W. S., Davenport, T. A., Johnson, D., Vella, K., Mitchell, J., & Hickie, I. B. (2018) · JMIR Mental Health · doi:10.2196/11068 · Cited by 62 · see citing works
“Although there are weaknesses with think-aloud protocols, such as their reliance on participant subjectivity and their inability to capture subconscious cognitive processes, concurrent think-aloud protocols nonetheless have the ability to capture crucial insights at a low cost [52].”
BackgroundNelms, A. A., & Segura-Totten, M. (2019) · CBE—Life Sciences Education · doi:10.1187/cbe.18-05-0077 · Cited by 50 · see citing works
Bhat, A., Agashe, S., Oberoi, P., Mohile, N., Jangir, R., & Joshi, A. (2023) · Proceedings of the 28th International Conference on Intelligent User Interfaces · doi:10.1145/3581641.3584060 · Cited by 47 · see citing works
- Authors
- Kuusela, H., & Paul, P.
- Affiliations
- University of Tampere (FI)
- Venue
- The American Journal of Psychology
- Publisher
- University of Illinois Press
- Published
- 2000
- Volume
- 113
- Issue
- 3
- Pages
- 387
- ISSN
- 0002-9556 (ISSN-L)
- OpenAlex
- W2079247256
- Cited by
- 408
- References
- 0
- Citations by year
- 2026: 13 · 2025: 41 · 2024: 17 · 2023: 26 · 2022: 17 · 2021: 25 · 2020: 23 · 2019: 18 · 2018: 29 · 2017: 18 · 2016: 25 · 2015: 22 · 2014: 17 · 2013: 22 · 2012: 20
- FWCI
- 0.68
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- The American Journal of Psychology (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Memory Processes and Influences
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Categorization, perception, and language
- Keywords
- verbal protocol analysis, decision-making, choice process
- MeSH terms
- Data Collection, Decision Making, Humans, Research Design, Retrospective Studies, Time Factors
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
