CoChat
Sign Up
Cited by 173Psychometric Methodologies and TestingReliability and Agreement in MeasurementMemory Processes and Influences

Sources of bias in the Goodman–Kruskal gamma coefficient measure of association: Implications for studies of metacognitive processes.

Masson, Michael E. J., Rotello, Caren M.·2009·Journal of Experimental Psychology: Learning, Memory, and Cognition
DOI 10.1037/a0014876
Open this paper in CoChat — free account

Abstract

In many cognitive, metacognitive, and perceptual tasks, measurement of performance or prediction accuracy may be influenced by response bias. Signal detection theory provides a means of assessing discrimination accuracy independent of such bias, but its application crucially depends on distributional assumptions. The Goodman-Kruskal gamma coefficient, G, has been proposed as an alternative means of measuring accuracy that is free of distributional assumptions. This measure is widely used with tasks that assess metamemory or metacognition performance. The authors demonstrate that the empirically determined value of G systematically deviates from its actual value under realistic conditions. A distribution-specific variant of G, called G-sub(c), is introduced to show why this bias arises. The findings imply that caution is needed when using G as a measure of accuracy, and alternative measures are recommended.

Integrity signals

Registry

Crossref · 10.1037/a0014876

Cites

57 of 57 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Masson and Rotello (2009) showed in an extensive simulation study that G is sensitive to the tendency to use higher or lower confidence ratings (bias), and this may lead to erroneous conclusions, such as interpreting a difference in G between groups as reflecting a true underlying difference in metacognitive sensitivity despite possible differences in bias.

    “In an extensive simulation study, Masson and Rotello (2009) showed that G was similarly sensitive to the tendency to use higher or lower confidence ratings (bias), and that this may lead to erroneous conclusions, such as interpreting a difference in G between groups as reflecting a true underlying difference in metacognitive sensitivity despite possible differences in bias.”

    From How to measure metacognition (2014) · cites this paper as Masson and Rotello (2009)

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Related

Authors
Masson, M. E. J., & Rotello, C. M.
Affiliations
  • University of Victoria (CA)
  • University of Massachusetts Amherst (US)
ORCID
Venue
Journal of Experimental Psychology: Learning, Memory, and Cognition
Publisher
American Psychological Association (APA)
Published
2009
Volume
35
Issue
2
Pages
509-527
ISSN
  • 0278-7393 (ISSN-L)
  • 1939-1285
DOI
10.1037/a0014876
OpenAlex
W2008182492
Cited by
173
References
57
Citations by year
2026: 5 · 2025: 9 · 2024: 6 · 2023: 14 · 2022: 6 · 2021: 13 · 2020: 9 · 2019: 11 · 2018: 8 · 2017: 12 · 2016: 11 · 2015: 15 · 2014: 18 · 2013: 9 · 2012: 7
FWCI
12.20
Type
journal article
Language
en
Open access
closed
Locations
  • Journal of Experimental Psychology Learning Memory and Cognition (publishedVersion)
  • PubMed (publishedVersion)
Funders
  • Natural Sciences and Engineering Research Council (7910-03)
  • National Institutes of Health (MH60274)
Topics
  • Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Memory Processes and Influences
Keywords
response bias, metacognition, metamemory, signal detection theory, discriminatory accuracy, distributional assumptions, bias correction
MeSH terms
Association, Culture, Data Interpretation, Statistical, Decision Making, Discrimination, Psychological, Humans, Judgment, Paired-Associate Learning, Psychometrics, Mental Recall, ROC Curve, Reproducibility of Results, Models, Statistical, Bias, Signal Detection, Psychological, Recognition, Psychology

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review