How to show that 9 > 221: Collect judgments in a between-subjects design.

Birnbaum, Michael H.·1999·Psychological Methods

Abstract

In between-subjects (BS) designs, different groups may be asked to make judgments on numerical rating scales.According to judgment theory, judgments obtained BS are not an ordinal scale of subjective value.This article illustrates how BS designs can lead to strange conclusions: When different groups judge the subjective size of numbers, 9 is judged significantly larger than 221.The theory is that 9 brings to mind a context of small numbers, among which 9 seems "average" or even "large"; however, 221 invokes a context of 3-digit numbers, among which 221 seems relatively "small."Within-subjects, however, judges would not have said 9 > 221.Implications of this problem and suggestions for dealing with it are discussed.

Cites (2)

  1. On the psychology of prediction.
    Kahneman, Daniel, Tversky, Amos · 1973 · Psychological Review
  2. Perception and the Representative Design of Psychological Experiments
    Brunswik, Egon · 1956

Show all 2 references

Cited by (16)

  1. Representativeness revisited: Attribute substitution in intuitive judgment.
    Kahneman, Daniel, Frederick, Shane · 2002 · Heuristics and Biases
  2. The Affect Heuristic
    Slovic, Paul, Finucane, Melissa, Peters, Ellen, MacGregor, Donald G. · 2002 · Heuristics and Biases
  3. Heuristics and Biases: The Psychology of Intuitive Judgment
    2002 · Cambridge University Press eBooks
  4. Research Methods in Psychology
    2016
  5. Human Research and Data Collection via the Internet
    Birnbaum, Michael H. · 2004 · Annual Review of Psychology

See all 228

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: How to show that 9 > 221: Collect judgments in a between-subjects design. https://app.cochat.ai/papers/10.1037//1082-989x.4.3.243
The paper
Birnbaum, M. H. (1999). How to show that 9 > 221: Collect judgments in a between-subjects design. Psychological Methods. https://doi.org/10.1037//1082-989x.4.3.243

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.