The DOI matches its Crossref record; 54 of the 54 references it cites were found in a registry or an index.
The Relative Operating Characteristic in Psychology: A technique for isolating effects of response bias finds wide use in the study of perception and cognition.
Abstract
The clinician looking, listening, or feeling for signs of a disease may far prefer a false alarm to a miss, particularly if the disease is serious and contagious. On the other hand, he may believe that the available therapy is marginally effective, expensive, and debilitating. The pilot seeing the landing lights only when they are a few yards away may decide that his plane is adequately aligned with the runway if he is alone and familiar with that plight. He may be more inclined to circle the field before another try at landing if he has many passengers and recent memory of another plane crashing under those circumstances. The Food and Drug administrator suspecting botulism in a canned food may not want to accept even a remote threat to the public health. But he may be less clearly biased if a recent false alarm has cost a canning company millions of dollars and left some damaged reputations. The making of almost any fine discrimination is beset with such considerations of probability and utility, which are extraneous and potentially confounding when one is attempting to measure the acuity of discrimination per se. The ROC is an analytical technique, with origins in statistical decision theory and electronic detection theory, that quite effectively isolates the effects of the observer's response bias, or decision criterion, in the study of discrimination behavior. This capability, pursued through a century of psychological testing, provides a relatively pure measure of the discriminability of different stimuli and of the capacity of organisms to discriminate. The ROC also treats quantitatively the response, or decision, aspects of choice behavior. The decision parameter can then be functionally related to the probabilities of the stimulus alternatives and to the utilities of the various stimulus-response pairs, or to the observer's expectations and motivations. In separating and quantifying discrimination and decision processes, the ROC promises a more reliable and valid solution to some practical problems and enhances our understanding of the perceptual and cognitive phenomena that depend directly on these fundamental processes. In several problem areas in psychology, effects that were supposed to reflect properties of the discrimination process have been shown by the ROC analysis to reflect instead properties of the decision process.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A law of comparative judgment.
Thurstone, L. L. (1927) · Psychological Review · doi:10.1037/h0070288 · Cited by 5,210
Concreteness, imagery, and meaningfulness values for 925 nouns.
Paivio, A., Yuille, J. C., & Madigan, S. A. (1968) · Journal of Experimental Psychology · doi:10.1037/h0025327 · Cited by 2,827
IX. On the problem of the most efficient tests of statistical hypotheses
Neyman, J., & Pearson, E. S. (1933) · Philosophical Transactions of the Royal Society of London. Series A, Containing Papers of a Mathematical or Physical Character · doi:10.1098/rsta.1933.0009 · Cited by 2,341
ON THE PROBLEM OF THE MOST EFFICIENT TESTS OF STATISTICAL HYPOTHESES
(1967) · Joint Statistical Papers · doi:10.1525/9780520339897-008 · Cited by 1,504
A decision-making theory of visual detection.
Tanner, W. P., & Swets, J. A. (1954) · Psychological Review · doi:10.1037/h0058700 · Cited by 1,317
Self-citationNonparametric indexes for sensitivity and bias: Computing formulas.
Grier, J. B. (1971) · Psychological Bulletin · doi:10.1037/h0031246 · Cited by 1,039
Dorfman, D. D., & Alf, E. (1969) · Journal of Mathematical Psychology · doi:10.1016/0022-2496(69)90019-4 · Cited by 874
The theory of signal detectability
Peterson, W., Birdsall, T., & Fox, W. (1954) · Transactions of the IRE Professional Group on Information Theory · doi:10.1109/tit.1954.1057460 · Cited by 867
Thurstone, L. L. (1927) · The American Journal of Psychology · doi:10.2307/1415006 · Cited by 556
Signal detection theory and human memory.
Banks, W. P. (1970) · Psychological Bulletin · doi:10.1037/h0029531 · Cited by 473
Influence of imaged pictures and sounds on detection of visual and auditory signals.
Segal, S. J., & Fusella, V. (1970) · Journal of Experimental Psychology · doi:10.1037/h0028840 · Cited by 379
Word-frequency effect and response bias.
Broadbent, D. E. (1967) · Psychological Review · doi:10.1037/h0024206 · Cited by 358
Lusted, L. B. (1971) · Science · doi:10.1126/science.171.3977.1217 · Cited by 304
Neural Theories of Simple Visual Discriminations*†
Blackwell, H. R. (1963) · Journal of the Optical Society of America · doi:10.1364/josa.53.000129 · Cited by 303
Stimulus intensity and response evocation.
Grice, G. R. (1968) · Psychological Review · doi:10.1037/h0026287 · Cited by 292
A Significance Test for One Parameter Isosensitivity Functions
Gourevitch, V., & Galanter, E. (1967) · Psychometrika · doi:10.1007/bf02289402 · Cited by 249
Threshold theories of signal detection.
Krantz, D. H. (1969) · Psychological Review · doi:10.1037/h0027238 · Cited by 225
Swets, J. A. (1963) · Science · doi:10.1126/science.141.3577.245 · Cited by 219
Self-citationOperating Characteristics, Signal Detectability, and the Method of Free Response
Egan, J. P., Greenberg, G. Z., & Schulman, A. I. (1961) · The Journal of the Acoustical Society of America · doi:10.1121/1.1908935 · Cited by 217
Frankmann, J. P., & Adams, J. A. (1962) · Psychological Bulletin · doi:10.1037/h0046142 · Cited by 201
Goldiamond, I. (1958) · Psychological Bulletin · doi:10.1037/h0046992 · Cited by 189
Maximum Likelihood Estimation of Parameters of Signal Detection Theory–A Direct Solution
Dorfman, D. D., & Alf, E. (1968) · Psychometrika · doi:10.1007/bf02289677 · Cited by 177
The elusive tradeoff: Speed vs accuracy in visual discrimination tasks
Swensson, R. G. (1972) · Perception & Psychophysics · doi:10.3758/bf03212837 · Cited by 171
Effects of organization and semantic similarity on recall and recognition
Mandler, G., Pearlstone, Z., & Koopmans, H. S. (1969) · Journal of Verbal Learning and Verbal Behavior · doi:10.1016/s0022-5371(69)80134-9 · Cited by 170
Nonparametric index of response bias for use in detection and recognition experiments.
Hodos, W. (1970) · Psychological Bulletin · doi:10.1037/h0029933 · Cited by 162
A table for the calculation of d' and BETA.
Hochhaus, L. (1972) · Psychological Bulletin · doi:10.1037/h0032459 · Cited by 152
Effectiveness of information retrieval methods
Swets, J. A. (1969) · American Documentation · doi:10.1002/asi.4630200110 · Cited by 152
Self-citationResponse latency models for signal detection.
Pike, R. (1973) · Psychological Review · doi:10.1037/h0033871 · Cited by 149
Boring, E. G. (1921) · The American Journal of Psychology · doi:10.2307/1413768 · Cited by 147
Psychometric and psychophysical hue discrimination functions for the pigeon
Wright, A. A. (1972) · Vision Research · doi:10.1016/0042-6989(72)90171-x · Cited by 137
Unidimensional strength theory and component analysis of noise in absolute and comparative judgments
Wickelgren, W. A. (1968) · Journal of Mathematical Psychology · doi:10.1016/0022-2496(68)90059-x · Cited by 135
Factors affecting the slope of empirical ROC curves: Comparison of binary and rating responses
Markowitz, J., & Swets, J. A. (1967) · Perception & Psychophysics · doi:10.3758/bf03210301 · Cited by 129
Self-citationSome aspects of ROC curve-fitting: Normal and logistic models
Grey, D. R., & Morgan, B. J. T. (1972) · Journal of Mathematical Psychology · doi:10.1016/0022-2496(72)90009-0 · Cited by 129
Evoked Potential Correlates of Response Criterion in Auditory Signal Detection
Paul, D. D., & Sutton, S. (1972) · Science · doi:10.1126/science.177.4046.362 · Cited by 125
Squires, K. C., Hillyard, S. A., & Lindsay, P. H. (1973) · Perception & Psychophysics · doi:10.3758/bf03207230 · Cited by 112
Extensions of the Significance Test for One-Parameter Signal Detection Hypotheses
Marascuilo, L. A. (1970) · Psychometrika · doi:10.1007/bf02291265 · Cited by 102
Sampling variability of the area under the ROC-curve and of d'e.
Pollack, I. W., & Hsieh, R. (1969) · Psychological Bulletin · doi:10.1037/h0026862 · Cited by 100
Modern statistical approaches to reception in communication theory
Meter, D. V., & Middleton, D. (1954) · Transactions of the IRE Professional Group on Information Theory · doi:10.1109/tit.1954.1057471 · Cited by 95
Swets, J. A., & Kristofferson, A. B. (1970) · Annual Review of Psychology · doi:10.1146/annurev.ps.21.020170.002011 · Cited by 84
Self-citationCriterion change in continuous recognition memory.
Donaldson, W., & Murdock, B. B. (1968) · Journal of Experimental Psychology · doi:10.1037/h0025510 · Cited by 71
Signal detection by human observers.
Swets, J. A., & Green, D. M. (1965) · PsycEXTRA Dataset · doi:10.1037/e444572004-001 · Cited by 54
Self-citationOptimal Policies for the Prisoner’s Dilemma
Rapoport, A. (1990) · Theory and decision library. Series C, Game theory, mathematical programming and operations research · doi:10.1007/978-94-009-1992-1_1 · Cited by 53
THE PARAMETRIC DESCRIPTION OF RETRIEVAL TESTS: PART II: OVERALL MEASURES
Robertson, S. E. (1969) · Journal of Documentation · doi:10.1108/eb026466 · Cited by 52
Concurrent variation of response bias and sensitivity in an operant-psychophysical test
Terman, M., & Terman, J. S. (1972) · Perception & Psychophysics · doi:10.3758/bf03206285 · Cited by 52
Clark, W. C. (1966) · Psychological Bulletin · doi:10.1037/h0023313 · Cited by 43
Operating Characteristics from Yes-No and Forced-Choice Procedures
Schulman, A. I., & Mitchell, R. R. (1966) · The Journal of the Acoustical Society of America · doi:10.1121/1.1910098 · Cited by 42
Catlin, J. (1969) · Psychological Review · doi:10.1037/h0028127 · Cited by 40
Effect of images in six sense modalities on detection of visual signal from noise
Segal, S. J., & Fusella, V. (1971) · Psychonomic Science · doi:10.3758/bf03337889 · Cited by 37
THE MEASURES OF INFORMATION RETRIEVAL EFFECTIVENESS PROPOSED BY SWETS
Brookes, B. C. (1968) · Journal of Documentation · doi:10.1108/eb026442 · Cited by 37
A model of the auditory threshold and its application to the problem of the multiple observer.
Smith, M. H., & Wilson, E. A. (1953) · The Psychological Monographs · doi:10.1037/h0093654 · Cited by 32
Comments on Broadbent's response bias model for stimulus recognition.
Nakatani, L. H. (1970) · Psychological Review · doi:10.1037/h0029929 · Cited by 15
On the evaluation of models for the word-frequency effect.
Nakatani, L. H. (1973) · Psychological Review · doi:10.1037/h0034298 · Cited by 13
In defense of sophisticated-guessing theory.
Catlin, J. (1973) · Psychological Review · doi:10.1037/h0035438 · Cited by 12
Sex differences in recognition memory
Barr-Brown, M., & White, M. J. (1971) · Psychonomic Science · doi:10.3758/bf03335852 · Cited by 4
Cited by
pROC: an open-source package for R and S+ to analyze and compare ROC curves
Robin, X., Turck, N., Hainard, A., Tiberti, N., Lisacek, F., Sanchez, J.-C., & Müller, M. (2011) · BMC Bioinformatics · doi:10.1186/1471-2105-12-77 · Cited by 13,697
Measuring the Accuracy of Diagnostic Systems
Swets, J. A. (1988) · Science · doi:10.1126/science.3287615 · Cited by 10,058
Self-citationBasic principles of ROC analysis
Metz, C. E. (1978) · Seminars in Nuclear Medicine · doi:10.1016/s0001-2998(78)80014-2 · Cited by 6,026
Calculation of signal detection theory measures
Stanislaw, H., & Todorov, N. (1999) · Behavior Research Methods, Instruments, & Computers · doi:10.3758/bf03207704 · Cited by 2,673
The diagnostic odds ratio: a single indicator of test performance
Glas, A. S., Lijmer, J. G., Prins, M. H., Bonsel, G. J., & Bossuyt, P. M. M. (2003) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(03)00177-x · Cited by 2,261
Metadata
- Authors
- Swets, J. A.
- Venue
- Science
- Published
- 1973-12-07
- Volume
- 182
- Issue
- 4116
- Pages
- 990-1000
- ISSN
- 0036-8075 (ISSN-L)
- 1095-9203
- OpenAlex
- W1970611288
- Cited by
- 731
- References
- 54
- FWCI
- 6.77
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.