How to show that 9 > 221: Collect judgments in a between-subjects design.
Abstract
In between-subjects (BS) designs, different groups may be asked to make judgments on numerical rating scales.According to judgment theory, judgments obtained BS are not an ordinal scale of subjective value.This article illustrates how BS designs can lead to strange conclusions: When different groups judge the subjective size of numbers, 9 is judged significantly larger than 221.The theory is that 9 brings to mind a context of small numbers, among which 9 seems "average" or even "large"; however, 221 invokes a context of 3-digit numbers, among which 221 seems relatively "small."Within-subjects, however, judges would not have said 9 > 221.Implications of this problem and suggestions for dealing with it are discussed.
Cites (2)
- On the psychology of prediction.Kahneman, Daniel, Tversky, Amos · 1973 · Psychological Review
- Perception and the Representative Design of Psychological ExperimentsBrunswik, Egon · 1956
Cited by (16)
- Representativeness revisited: Attribute substitution in intuitive judgment.Kahneman, Daniel, Frederick, Shane · 2002 · Heuristics and Biases
- The Affect HeuristicSlovic, Paul, Finucane, Melissa, Peters, Ellen, MacGregor, Donald G. · 2002 · Heuristics and Biases
- Heuristics and Biases: The Psychology of Intuitive Judgment2002 · Cambridge University Press eBooks
- Research Methods in Psychology2016
- Human Research and Data Collection via the InternetBirnbaum, Michael H. · 2004 · Annual Review of Psychology
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: How to show that 9 > 221: Collect judgments in a between-subjects design. https://app.cochat.ai/papers/10.1037//1082-989x.4.3.243
- The paper
- Birnbaum, M. H. (1999). How to show that 9 > 221: Collect judgments in a between-subjects design. Psychological Methods. https://doi.org/10.1037//1082-989x.4.3.243
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.