The DOI matches its Crossref record; 54 of the 54 references it cites were found in a registry or an index.
The Relative Operating Characteristic in Psychology: A technique for isolating effects of response bias finds wide use in the study of perception and cognition.
Abstract
The clinician looking, listening, or feeling for signs of a disease may far prefer a false alarm to a miss, particularly if the disease is serious and contagious. On the other hand, he may believe that the available therapy is marginally effective, expensive, and debilitating. The pilot seeing the landing lights only when they are a few yards away may decide that his plane is adequately aligned with the runway if he is alone and familiar with that plight. He may be more inclined to circle the field before another try at landing if he has many passengers and recent memory of another plane crashing under those circumstances. The Food and Drug administrator suspecting botulism in a canned food may not want to accept even a remote threat to the public health. But he may be less clearly biased if a recent false alarm has cost a canning company millions of dollars and left some damaged reputations. The making of almost any fine discrimination is beset with such considerations of probability and utility, which are extraneous and potentially confounding when one is attempting to measure the acuity of discrimination per se. The ROC is an analytical technique, with origins in statistical decision theory and electronic detection theory, that quite effectively isolates the effects of the observer's response bias, or decision criterion, in the study of discrimination behavior. This capability, pursued through a century of psychological testing, provides a relatively pure measure of the discriminability of different stimuli and of the capacity of organisms to discriminate. The ROC also treats quantitatively the response, or decision, aspects of choice behavior. The decision parameter can then be functionally related to the probabilities of the stimulus alternatives and to the utilities of the various stimulus-response pairs, or to the observer's expectations and motivations. In separating and quantifying discrimination and decision processes, the ROC promises a more reliable and valid solution to some practical problems and enhances our understanding of the perceptual and cognitive phenomena that depend directly on these fundamental processes. In several problem areas in psychology, effects that were supposed to reflect properties of the discrimination process have been shown by the ROC analysis to reflect instead properties of the decision process.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A law of comparative judgment.
Thurstone, L. L. (1927) · Psychological Review · doi:10.1037/h0070288 · Cited by 5,210
Concreteness, imagery, and meaningfulness values for 925 nouns.
Paivio, A., Yuille, J. C., & Madigan, S. A. (1968) · Journal of Experimental Psychology · doi:10.1037/h0025327 · Cited by 2,827
IX. On the problem of the most efficient tests of statistical hypotheses
Neyman, J., & Pearson, E. S. (1933) · Philosophical Transactions of the Royal Society of London. Series A, Containing Papers of a Mathematical or Physical Character · doi:10.1098/rsta.1933.0009 · Cited by 2,341
ON THE PROBLEM OF THE MOST EFFICIENT TESTS OF STATISTICAL HYPOTHESES
(1967) · Joint Statistical Papers · doi:10.1525/9780520339897-008 · Cited by 1,504
A decision-making theory of visual detection.
Tanner, W. P., & Swets, J. A. (1954) · Psychological Review · doi:10.1037/h0058700 · Cited by 1,317
Self-citationNonparametric indexes for sensitivity and bias: Computing formulas.
Grier, J. B. (1971) · Psychological Bulletin · doi:10.1037/h0031246 · Cited by 1,039
Dorfman, D. D., & Alf, E. (1969) · Journal of Mathematical Psychology · doi:10.1016/0022-2496(69)90019-4 · Cited by 874
The theory of signal detectability
Peterson, W., Birdsall, T., & Fox, W. (1954) · Transactions of the IRE Professional Group on Information Theory · doi:10.1109/tit.1954.1057460 · Cited by 867
Thurstone, L. L. (1927) · The American Journal of Psychology · doi:10.2307/1415006 · Cited by 556
Signal detection theory and human memory.
Banks, W. P. (1970) · Psychological Bulletin · doi:10.1037/h0029531 · Cited by 473
Cited by
pROC: an open-source package for R and S+ to analyze and compare ROC curves
Robin, X., Turck, N., Hainard, A., Tiberti, N., Lisacek, F., Sanchez, J.-C., & Müller, M. (2011) · BMC Bioinformatics · doi:10.1186/1471-2105-12-77 · Cited by 13,697
Measuring the Accuracy of Diagnostic Systems
Swets, J. A. (1988) · Science · doi:10.1126/science.3287615 · Cited by 10,058
Self-citationBasic principles of ROC analysis
Metz, C. E. (1978) · Seminars in Nuclear Medicine · doi:10.1016/s0001-2998(78)80014-2 · Cited by 6,026
Calculation of signal detection theory measures
Stanislaw, H., & Todorov, N. (1999) · Behavior Research Methods, Instruments, & Computers · doi:10.3758/bf03207704 · Cited by 2,673
The diagnostic odds ratio: a single indicator of test performance
Glas, A. S., Lijmer, J. G., Prins, M. H., Bonsel, G. J., & Bossuyt, P. M. M. (2003) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(03)00177-x · Cited by 2,261
The DET curve in assessment of detection task performance
Martin, A., Doddington, G., Kamm, T., Ordowski, M., & Przybocki, M. (1997) · 5th European Conference on Speech Communication and Technology (Eurospeech 1997) · doi:10.21437/eurospeech.1997-504 · Cited by 1,384
Forecast Verification: A Practitioner's Guide in Atmospheric Science
(2011) · Wiley eBooks · doi:10.1002/9781119960003 · Cited by 1,267
Genuine memory deficits in dementia
Grober, E., & Buschke, H. (1987) · Developmental Neuropsychology · doi:10.1080/87565648709540361 · Cited by 1,047
A comparison of current measures of the accuracy of feeling-of-knowing predictions
Nelson, T. O. (1984) · Psychological Bulletin · doi:10.1037/0033-2909.95.1.109 · Cited by 976
Evaluating intrusion detection systems: the 1998 DARPA off-line intrusion detection evaluation
Lippmann, R. P., Fried, D. J., Graf, I., Haines, J. W., Kendall, K. R., McClung, D., Weber, D., Webster, S. E., Wyschogrod, D., Cunningham, R. K., et al. (2002) · Proceedings DARPA Information Survivability Conference and Exposition. DISCEX'00 · doi:10.1109/discex.2000.821506 · Cited by 917
Interpretation of Rank Histograms for Verifying Ensemble Forecasts
Hamill, T. M. (2001) · Monthly Weather Review · doi:10.1175/1520-0493(2001)129<0550:iorhfv>2.0.co;2 · Cited by 892
Mason, S. J., & Graham, N. E. (2002) · Quarterly Journal of the Royal Meteorological Society · doi:10.1256/003590002320603584 · Cited by 868
Tonic and burst firing: dual modes of thalamocortical relay
Sherman, S. M. (2001) · Trends in Neurosciences · doi:10.1016/s0166-2236(00)01714-8 · Cited by 743
Wilks, D. S. (2011) · International Geophysics · doi:10.1016/b978-0-12-385022-5.00008-7 · Cited by 727
Journal of Statistical Planning and Inference, the
(2005) · Encyclopedia of Statistical Sciences · doi:10.1002/0471667196.ess1316 · Cited by 720
Sensitivity to grammatical structure in so-called agrammatic aphasics
Linebarger, M. C., Schwartz, M. F., & Saffran, E. M. (1983) · Cognition · doi:10.1016/0010-0277(83)90015-x · Cited by 697
Cornblatt, B. A., Risch, N. J., Faris, G., Friedman, D., & Erlenmeyer-Kimling, L. (1988) · Psychiatry Research · doi:10.1016/0165-1781(88)90076-5 · Cited by 682
Financial Crises, Credit Booms, and External Imbalances: 140 Years of Lessons
Jordà, Ò., Schularick, M., & Taylor, A. M. (2011) · IMF Economic Review · doi:10.1057/imfer.2011.8 · Cited by 681
Scale and contour: Two components of a theory of memory for melodies.
Dowling, W. J. (1978) · Psychological Review · doi:10.1037//0033-295x.85.4.341 · Cited by 678
Triballeau, N., Acher, F., Brabet, I., Pin, J.-P., & Bertrand, H.-O. (2005) · Journal of Medicinal Chemistry · doi:10.1021/jm049092j · Cited by 631
A Hybrid Ensemble Kalman Filter–3D Variational Analysis Scheme
Hamill, T. M., & Snyder, C. (2000) · Monthly Weather Review · doi:10.1175/1520-0493(2000)128<2905:ahekfv>2.0.co;2 · Cited by 619
Risk interpretation and action: A conceptual framework for responses to natural hazards
Richard Eiser, J., Bostrom, A., Burton, I., Johnston, D. M., McClure, J., Paton, D., van der Pligt, J., & White, M. P. (2012) · International Journal of Disaster Risk Reduction · doi:10.1016/j.ijdrr.2012.05.002 · Cited by 599
Current approaches to seasonal to interannual climate predictions
Goddard, L., Mason, S. J., Zebiak, S. E., Ropelewski, C. F., Basher, R., & Cane, M. A. (2001) · International Journal of Climatology · doi:10.1002/joc.636 · Cited by 588
Indices of discrimination or diagnostic accuracy: Their ROCs and implied models.
Swets, J. A. (1986) · Psychological Bulletin · doi:10.1037//0033-2909.99.1.100 · Cited by 582
Self-citationSwets, J. A. (1986) · Psychological Bulletin · doi:10.1037//0033-2909.99.2.181 · Cited by 508
Self-citationRelation of heart rate control to heartbeat perception
Whitehead, W. E., Drescher, V. M., Heiman, P., & Blackwell, B. (1977) · Biofeedback and Self-Regulation · doi:10.1007/bf00998623 · Cited by 480
Galvin, S. J., Podd, J. V., Drga, V., & Whitmore, J. (2003) · Psychonomic Bulletin & Review · doi:10.3758/bf03196546 · Cited by 405
Predicting uncertainty in forecasts of weather and climate
Palmer, T. N. (2000) · Reports on Progress in Physics · doi:10.1088/0034-4885/63/2/201 · Cited by 446
Meta-analysis of the sensitivity decrement in vigilance.
See, J. E., Howe, S. R., Warm, J. S., & Dember, W. N. (1995) · Psychological Bulletin · doi:10.1037//0033-2909.117.2.230 · Cited by 410
Nuechterlein, K. H. (1983) · Journal of Abnormal Psychology · doi:10.1037//0021-843x.92.1.4 · Cited by 409
The psychological representation of musical pitch in a tonal context
Krumhansl, C. L. (1979) · Cognitive Psychology · doi:10.1016/0010-0285(79)90016-1 · Cited by 404
Lin, W.-Y., Lee, L.-T., Chen, C.-Y., Lo, H., Hsia, H.-H., Liu, I.-L., Lin, R.-S., Shau, W.-Y., & Huang, K.-C. (2002) · International Journal of Obesity · doi:10.1038/sj.ijo.0802040 · Cited by 396
Garmezy, N. (1987) · American Journal of Orthopsychiatry · doi:10.1111/j.1939-0025.1987.tb03526.x · Cited by 387
Memory Load and Event Rate Control Sensitivity Decrements in Sustained Attention
Parasuraman, R. (1979) · Science · doi:10.1126/science.472714 · Cited by 386
Metz, C. E., Wang, P.-L., & Kronman, H. B. (1984) · Information Processing in Medical Imaging · doi:10.1007/978-94-009-6045-9_25 · Cited by 356
Evaluating Probabilistic Forecasts Using Information Theory
Roulston, M. S., & Smith, L. A. (2002) · Monthly Weather Review · doi:10.1175/1520-0493(2002)130<1653:epfuit>2.0.co;2 · Cited by 350
Doblas-Reyes, F. J., Hagedorn, R., & Palmer, T. N. (2005) · Tellus A: Dynamic Meteorology and Oceanography · doi:10.3402/tellusa.v57i3.14658 · Cited by 340
Shinn, M., Weitzman, B. C., Stojanovic, D., Knickman, J. R., Jiménez, L., Duchon, L., James, S., & Krantz, D. H. (1998) · American Journal of Public Health · doi:10.2105/ajph.88.11.1651 · Cited by 334
Conditional Probabilities, Relative Operating Characteristics, and Relative Operating Levels
Mason, S. J., & Graham, N. E. (1999) · Weather and Forecasting · doi:10.1175/1520-0434(1999)014<0713:cproca>2.0.co;2 · Cited by 331
Use of the “Odds Ratio” for Diagnosing Forecast Skill
Stephenson, D. B. (2000) · Weather and Forecasting · doi:10.1175/1520-0434(2000)015<0221:uotorf>2.0.co;2 · Cited by 318
Fetsch, C. R., DeAngelis, G. C., & Angelaki, D. E. (2013) · Nature Reviews Neuroscience · doi:10.1038/nrn3503 · Cited by 302
GREENES, R. A., & BEGG, C. B. (1985) · Investigative Radiology · doi:10.1097/00004424-198510000-00018 · Cited by 292
Signal detection theory and generalized linear models.
DeCarlo, L. T. (1998) · Psychological Methods · doi:10.1037/1082-989x.3.2.186 · Cited by 276
Measuring forecast skill: is it real skill or is it the varying climatology?
Hamill, T. M., & Juras, J. (2006) · Quarterly Journal of the Royal Meteorological Society · doi:10.1256/qj.06.25 · Cited by 264
Interpreting Chest Radiographs without Visual Search
Kundel, H. L., & Nodine, C. F. (1975) · Radiology · doi:10.1148/116.3.527 · Cited by 253
Perception of gastric contractions and self-control of gastric motility
Whitehead, W. E., & Drescher, V. M. (1980) · Psychophysiology · doi:10.1111/j.1469-8986.1980.tb02296.x · Cited by 208
Lin, C. C. H., Hsiao, C. K., & Chen, W. J. (1999) · Journal of Abnormal Child Psychology · doi:10.1023/a:1021932119311 · Cited by 158
Behavioral Analysis of Insider Threat: A Survey and Bootstrapped Prediction in Imbalanced Data
Azaria, A., Richardson, A., Kraus, S., & Subrahmanian, V. S. (2014) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2014.2377811 · Cited by 133
HARVER, A., KATKIN, E. S., & BLOCH, E. (1993) · Psychophysiology · doi:10.1111/j.1469-8986.1993.tb03347.x · Cited by 113
Beer, A. L., & Watanabe, T. (2009) · Experimental Brain Research · doi:10.1007/s00221-009-1769-6 · Cited by 25
Metadata
- Authors
- Swets, J. A.
- Venue
- Science
- Published
- 1973-12-07
- Volume
- 182
- Issue
- 4116
- Pages
- 990-1000
- ISSN
- 0036-8075 (ISSN-L)
- 1095-9203
- OpenAlex
- W1970611288
- Cited by
- 731
- References
- 54
- FWCI
- 6.77
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.