Type 2 tasks in the theory of signal detectability: Discrimination between correct and incorrect decisions
Open accessThe DOI matches its Crossref record; 44 of the 46 references it cites were found in a registry or an index.
Abstract
It has been known for over 40 years that there are two fundamentally different kinds of detection tasks in the theory of signal detectability. The Type 1 task is to distinguish between events defined independently of the observer; the Type 2 task is to distinguish between one's own correct and incorrect decisions about those Type 1 events. For the Type 1 task, the behavior of the detector can be summarized by the traditional receiver operating characteristic (ROC) curve. This curve can be compared with a theoretical ROC curve, which can be generated from overlapping probability functions conditional on the Type 1 events on an appropriate decision axis. We show how to derive the probability functions underlying Type 2 decisions from those for the Type 1 task. ROC curves and the usual measures of performance are readily obtained from those Type 2 functions, and some relationships among various Type 1 and Type 2 performance measures are presented. We discuss the relationship between Type 1 and Type 2 confidence ratings and caution against the practice of presenting transformed Type 2 ratings as empirical Type 1 ratings.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“Specifically, given a particular type 1 variance structure and bias, the form of the type 2 ROC is completely determined (Galvin et al., 2003).”
From How to measure metacognition (2014) · cites this paper as Galvin et al., 2003
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe cited source doesn’t support the claim as written.
In the citing paper
“Because such a family of type 2 ROC curves are all non-overlapping (Galvin et al., 2003), we can determine the curve from this family with just a single point, i.e., a single criterion.”
From How to measure metacognition (2014) · cites this paper as Galvin et al., 2003
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“However, type 2 d′ turns out to be problematic because SDT assumes that the distribution of internal signals for “correct” and “incorrect” trials are Gaussian with equal variances. While this assumption is usually more or less acceptable at the type 1 level (especially for 2-alternative forced-choice tasks), it is highly problematic for type 2 analysis.”
From How to measure metacognition (2014) · cites this paper as Galvin et al. (2003)
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Galvin et al. (2003) showed that these distributions are of different variance and highly non-Gaussian if the equal variance assumption holds at the type 1 level.”
From How to measure metacognition (2014) · cites this paper as Galvin et al. (2003)
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“The recently developed meta-d′ measure (Maniscalco and Lau, 2012, 2014) exploits the fact that given Gaussian variance assumptions at the type 1 level, the shapes of the type 2 distributions are known even if they are not themselves Gaussian (Galvin et al., 2003).”
From How to measure metacognition (2014) · cites this paper as Galvin et al., 2003
2 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(1966) · Medical Entomology and Zoology · Cited by 10,293
Signal detection theory and psychophysics
Ingleby, J. D. (1967) · Journal of Sound and Vibration · doi:10.1016/0022-460x(67)90197-6 · Cited by 8,509
Detection Theory: A User's Guide
Macmillan, N. A., & Creelman, C. D. (1991) · Cited by 5,415
An Introduction to the Theory of Statistics
(1911) · Nature · doi:10.1038/087345a0 · Cited by 4,394
Calibration of probabilities: the state of the art to 1980
Lichtenstein, S., Fischhoff, B., & Phillips, L. D. (1982) · Judgment under Uncertainty · doi:10.1017/cbo9780511809477.023 · Cited by 1,967
Signal detection theory and ROC analysis
Egan, J. P. (1975) · Academic Press eBooks · Cited by 1,659
Probabilistic mental models: a Brunswikian theory of confidence
Gigerenzer, G., Hoffrage, U., & Kleinbölting, H. (1991) · Psychological Review · doi:10.1037/0033-295x.98.4.506 · Cited by 1,506
An Introduction to the Theory of Statistics
Dempster, A. P., & Plackett, R. L. (1973) · International Statistical Review / Revue Internationale de Statistique · doi:10.2307/1402858 · Cited by 1,447
Decision Processes In Perception.
Swets, J. A., Tanner, W. P., Jr., & Birdsall, T. G. (1961) · Psychological Review · doi:10.1037/0033-295x.68.5.301 · Cited by 1,329
Introduction to the Theory of Statistics, 3rd ed.
Kass, G. V., Mood, A. M., Graybill, F. A., & Boes, D. C. (1974) · Journal of the American Statistical Association · doi:10.2307/2286195 · Cited by 975
The theory of signal detectability
Peterson, W., Birdsall, T., & Fox, W. (1954) · Transactions of the IRE Professional Group on Information Theory · doi:10.1109/tit.1954.1057460 · Cited by 867
Wood, G., & Murdock, B. B. (1975) · The American Journal of Psychology · doi:10.2307/1421679 · Cited by 659
Calibration and probability judgements: Conceptual and methodological issues
Keren, G. (1991) · Acta Psychologica · doi:10.1016/0001-6918(91)90036-y · Cited by 549
Perception without awareness: perspectives from cognitive psychology
Merikle, P. (2001) · Cognition · doi:10.1016/s0010-0277(00)00126-8 · Cited by 540
Decision Processes in Visual Perception
(1979) · Elsevier eBooks · doi:10.1016/c2013-0-11654-6 · Cited by 502
Information theory in psychology
(1955) · Acta Psychologica · doi:10.1016/s0001-6918(55)80084-4 · Cited by 475
Signal detection theory and human memory.
Banks, W. P. (1970) · Psychological Bulletin · doi:10.1037/h0029531 · Cited by 473
Detection theory and psychophysics
Marill, T. (1956) · Cited by 376
On the relationship between implicit and explicit modes in the learning of a complex rule structure.
Reber, A. S., Kassin, S. M., Lewis, S., & Cantor, G. (1980) · Journal of Experimental Psychology: Human Learning and Memory · doi:10.1037/0278-7393.6.5.492 · Cited by 367
On small differences in sensation
Peirce, C. S., & Jastrow, J. (1884) · PhilPapers (PhilPapers Foundation) · Cited by 362
The Wave Theory of Difference and Similarity
Link, S. W. (2020) · Routledge eBooks · doi:10.1201/9780429054709 · Cited by 347
Confidence and accuracy of near-threshold discrimination responses
Kunimoto, C., Miller, J., & Pashler, H. (2001) · Consciousness and Cognition · doi:10.1006/ccog.2000.0494 · Cited by 291
Accounts of the confidence-accuracy relation in recognition memory
Busey, T. A., Tunnicliff, J., Loftus, G. R., & Loftus, E. F. (2000) · Psychonomic Bulletin & Review · doi:10.3758/bf03210724 · Cited by 277
ROC curves and confidence judgments in recognition memory.
Van Zandt, T. (2000) · Journal of Experimental Psychology: Learning, Memory, and Cognition · doi:10.1037//0278-7393.26.3.582 · Cited by 247
Memoirs of the National Academy of Sciences
(1885) · The American Naturalist · doi:10.1086/274024 · Cited by 236
Blindsight in normal observers
Kolb, F. C., & Braun, J. (1995) · Nature · doi:10.1038/377336a0 · Cited by 198 · cited 2×
“Kolb and Braun (1995) used Type 2 tasks to demonstrate perception without awareness of regions of the visual field with different visual texture from that of their surrounds.”
BackgroundMethodA model of calibration for subjective probabilities
Ferrell, W. R., & McGoey, P. J. (1980) · Organizational Behavior and Human Performance · doi:10.1016/0030-5073(80)90045-8 · Cited by 192
Two Types of ROC Curves and Definitions of Parameters
Clarke, F. R., Birdsall, T. G., & Tanner, W. P. (1959) · The Journal of the Acoustical Society of America · doi:10.1121/1.1907764 · Cited by 168
Probing the locus of confidence judgments: Experiments on the time to determine confidence.
Baranski, J. V., & Petrusic, W. M. (1998) · Journal of Experimental Psychology: Human Perception and Performance · doi:10.1037/0096-1523.24.3.929 · Cited by 168
Keren, G. (1988) · Acta Psychologica · doi:10.1016/0001-6918(88)90007-8 · Cited by 163
Strength theory of decision rules and latency in retrieval from short-term memory
Norman, D., & Wickelgren, W. A. (1969) · Journal of Mathematical Psychology · doi:10.1016/0022-2496(69)90002-9 · Cited by 162
The criterion problem in short-term memory.
Murdock, B. B. (1966) · Journal of Experimental Psychology · doi:10.1037/h0023646 · Cited by 103
Several Unresolved Conceptual Problems of Mathematical Psychology
Luce, R. D. (1997) · Journal of Mathematical Psychology · doi:10.1006/jmps.1997.1150 · Cited by 97
Confidence Ratings, Message Reception, and the Receiver Operating Characteristic
Pollack, I. W., & Decker, L. R. (1958) · The Journal of the Acoustical Society of America · doi:10.1121/1.1909571 · Cited by 88
Consciousness and Commentaries
Weiskrantz, L. (1998) · International Journal of Psychology · doi:10.1080/002075998400411 · Cited by 68
Higham, P. A. (2002) · Memory & Cognition · doi:10.3758/bf03195266 · Cited by 60
SIGNAL‐DETECTION PROPERTIES OF VERBAL SELF‐REPORTS
Critchfield, T. S. (1993) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.1993.60-495 · Cited by 28
Healy, A. F., & Jones, C. (1973) · Psychological Bulletin · doi:10.1037/h0034378 · Cited by 25
On the Transmission and Confirmation of Messages in Noise
Egan, J. P., Clarke, F. R., & Carterette, E. C. (1956) · The Journal of the Acoustical Society of America · doi:10.1121/1.1908387 · Cited by 20
Source and Receiver Behavior in the Use of a Criterion
Egan, J. P., & Clarke, F. R. (1956) · The Journal of the Acoustical Society of America · doi:10.1121/1.1908615 · Cited by 18
On Indices of Signal and Response Discriminability
Pollack, I. (1959) · The Journal of the Acoustical Society of America · doi:10.1121/1.1907802 · Cited by 17
Detectability of correctness: A measure of knowing that one knows
Hosseini, J., & Ferrell, W. R. (1982) · Instructional Science · doi:10.1007/bf00154881 · Cited by 11
Taylor, A., Boven, R., & Whitmore, J. (1991) · Psychological Bulletin · doi:10.1037/0033-2909.109.1.133 · Cited by 8
Self-citation
Research works printed but not linked: 5
- Galvin, S. J. (1988).The theory of Type II ROC analysis. Unpublished master’s thesis, Victoria University, Wellington, New Zealand.
- Galvin, S. J., Beuke, C. J., Keogh, D., &Robinson, T. (2000). Can utrocular discrimination be subliminal?Investigative Ophthalmology & Visual Science,41(Suppl.), S732.
- Macmillan, N. A., &Creelman, C. D. (1991).Detection theory: A user9s guide. Cambridge: Cambridge University Press.
- Podd, J. V. (1975).Type I and Type II ROC analysis of change in human decision axis. Unpublished master’s thesis, Victoria University, Wellington, New Zealand.
- Weiskrantz, L. (1986).Blindsight: A case study and its implications. Oxford: Oxford University Press.
Cited by
Knowing your own heart: Distinguishing interoceptive accuracy from interoceptive awareness
Garfinkel, S. N., Seth, A. K., Barrett, A. B., Suzuki, K., & Critchley, H. D. (2015) · Biological Psychology · doi:10.1016/j.biopsycho.2014.11.004 · Cited by 1,624
Fleming, S. M., & Lau, H. C. (2014) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2014.00443 · Cited by 1,408
The structural basis of inter-individual differences in human behaviour and cognition
Kanai, R., & Rees, G. (2011) · Nature reviews. Neuroscience · doi:10.1038/nrn3000 · Cited by 1,055
Maniscalco, B., & Lau, H. (2012) · Consciousness and Cognition · doi:10.1016/j.concog.2011.09.021 · Cited by 883
Relating Introspective Accuracy to Individual Differences in Brain Structure
Fleming, S. M., Weil, R. S., Nagy, Z., Dolan, R. J., & Rees, G. (2010) · Science · doi:10.1126/science.1191883 · Cited by 809
Metadata
- Authors
- Galvin, S. J., Podd, J. V., Drga, V., & Whitmore, J.
- Affiliations
- University of Otago (NZ)
- Massey University (NZ)
- Victoria University of Wellington (NZ)
- Corresponding author
- Galvin, Susan J.
- ORCID
- Drga, Vit
- Venue
- Psychonomic Bulletin & Review
- Published
- 2003-12
- Volume
- 10
- Issue
- 4
- Pages
- 843-876
- ISSN
- 1069-9384 (ISSN-L)
- 1531-5320
- OpenAlex
- W1965187591
- Cited by
- 405
- References
- 46
- Citations by year
- 2026: 35 · 2025: 19 · 2024: 25 · 2023: 30 · 2022: 32 · 2021: 27 · 2020: 22 · 2019: 22 · 2018: 24 · 2017: 24 · 2016: 24 · 2015: 32 · 2014: 23 · 2013: 12 · 2012: 14
- FWCI
- 0.65
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Psychonomic Bulletin & Review (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Visual perception and processing mechanisms
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
- Keywords
- receiver operating characteristic, confidence ratings, decision errors, probability functions
- MeSH terms
- Cues, Decision Making, Discrimination Learning, Humans, Probability Learning, Problem Solving, ROC Curve, Models, Statistical, Observer Variation, Likelihood Functions, Signal Detection, Psychological
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.