The DOI matches its Crossref record; 110 of the 110 references it cites were found in a registry or an index.
Confirmation, disconfirmation, and information in hypothesis testing
Abstract
Strategies for hypothesis testing in scientific investigation and everyday reasoning have interested both psychologists and philosophers. A number of these scholars stress the importance of disconnrmation in reasoning and suggest that people are instead prone to a general deleterious "confirmation bias." In particular, it is suggested that people tend to test those cases that have the best chance of verifying current beliefs rather than those that have the best chance of falsifying them. We show, however; that many phenomena labeled "confirmation bias" are better understood in terms of a general positive test strategy. With this strategy, there is a tendency to test cases that are expected (or known) to have the property of interest rather than those expected (or known) to lack that property. This strategy is not equivalent to confirmation bias in the first sense; we show that the positive test strategy can be a very good heuristic for determining the truth or falsity of a hypothesis under realistic conditions. It can, however, lead to systematic errors or inefficiencies. The appropriateness of human hypothesis-testing strategies and prescriptions about optimal strategies must be understood in terms of the interaction between the strategy and the task at hand.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“A hypothesis can create bias as it often entails confirmation and other biases that generate evidence, confirming that hypothesis while ignoring (or dismissing) conflicting evidence (Baldwin, 1993; Klayman & Ha, 1987; Wagenaar et al., 1993).”
From Biased and biasing: the hidden bias cascade and bias snowball effects (2025) · cites this paper as Klayman & Ha, 1987
In the citing paper
“In this sense,selection of evidence based on relevance is related to the confirmation bias in cognitive science, where learners have been observed to (arguably maladaptively) select data which they believe to be true (i.e. they fail to attempt to falsify their hypotheses) [133].”
From Evolution and impact of bias in human and machine learning algorithm interaction (2020) · cites this paper as [133]
Citations
Cites
Tversky, A., & Kahneman, D. (1974) · Science · doi:10.1126/science.185.4157.1124 · Cited by 28,810
Diagnostic and Statistical Manual of Mental Disorders
American Psychiatric Association (2022) · doi:10.1176/appi.books.9780890425787 · Cited by 23,476
Judgment under Uncertainty: Heuristics and Biases
Tversky, A., & Kahneman, D. (1975) · Utility, Probability, and Human Decision Making · doi:10.1007/978-94-010-1834-0_8 · Cited by 22,650
(1966) · Medical Entomology and Zoology · Cited by 10,293
Signal detection theory and psychophysics
Ingleby, D. (1967) · Journal of Sound and Vibration · doi:10.1016/0022-460x(67)90197-6 · Cited by 8,616
The Logic of Scientific Discovery
Bhaskar, R. (2013) · Routledge eBooks · doi:10.4324/9780203090732-9 · Cited by 7,765
Diagnostic and Statistical Manual of Mental Disorders, 3rd ed
Cooper, A. M., & Michels, R. (1981) · American Journal of Psychiatry · doi:10.1176/ajp.138.1.128 · Cited by 7,419
Falsification and the Methodology of Scientific Research Programmes
Lakatos, I. (1970) · Criticism and the Growth of Knowledge · doi:10.1017/cbo9781139171434.009 · Cited by 4,649
Lord, C. G., Ross, L., & Lepper, M. R. (1979) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.37.11.2098 · Cited by 4,583
Human Inference: Strategies and Shortcomings of Social Judgment.
Cherniak, C., Nisbett, R. E., & Ross, L. D. (1983) · The Philosophical Review · doi:10.2307/2184495 · Cited by 4,196
Falsification and the Methodology of Scientific Research Programmes
Lakatos, I. (1976) · Can Theories be Refuted? · doi:10.1007/978-94-010-1863-0_14 · Cited by 3,915
The Logic of Scientific Discovery.
Kendall, M. G., & Popper, K. R. (1959) · Biometrika · doi:10.2307/2332835 · Cited by 2,896
Miller, G. A., & Cantor, N. E. (1982) · Social Cognition · doi:10.1521/soco.1982.1.1.83 · Cited by 2,862
Human Inference: Strategies and Shortcomings of Social Judgment.
Connolly, T., Nisbett, R. E., & Ross, L. D. (1981) · Administrative Science Quarterly · doi:10.2307/2392481 · Cited by 2,690
On the Failure to Eliminate Hypotheses in a Conceptual Task
Wason, P. C. (1960) · Quarterly Journal of Experimental Psychology · doi:10.1080/17470216008416717 · Cited by 2,090
Kintsch, W. (1977) · Medical Entomology and Zoology · Cited by 2,051
Psychology of Reasoning: Structure and Content
Wason, P. C., & Johnson-Laird, P. N. (1972) · Medical Entomology and Zoology · Cited by 1,425
Confidence in judgment: persistence of the illusion of validity
Einhorn, H. J., & Hogarth, R. M. (1978) · Psychological Review · doi:10.1037/0033-295x.85.5.395 · Cited by 1,209
A hypothesis-confirming bias in labeling effects
Darley, J. M., & Gross, P. H. (1983) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.44.1.20 · Cited by 1,173
Foss, B. (1966) · Medical Entomology and Zoology · Cited by 1,157
Conservatism in human information processing
Edwards, W. (1982) · Cambridge University Press eBooks · doi:10.1017/cbo9780511809477.026 · Cited by 1,026
Hypothesis-testing processes in social interaction.
Snyder, M., & Swann, W. B. (1978) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.36.11.1202 · Cited by 937
Einhorn, H. J., & Hogarth, R. M. (1986) · Psychological Bulletin · doi:10.1037/0033-2909.99.1.3 · Cited by 928
Consumer Learning: Advertising and the Ambiguity of Product Experience
Hoch, S. J., & Ha, Y.-W. (1986) · Journal of Consumer Research · doi:10.1086/209062 · Cited by 908
Conservatism in a simple probability inference task.
Phillips, L. D., & Edwards, W. (1966) · Journal of Experimental Psychology · doi:10.1037/h0023653 · Cited by 880
The elusive thematic‐materials effect in Wason's selection task
Griggs, R. A., & Cox, J. R. (1982) · British Journal of Psychology · doi:10.1111/j.2044-8295.1982.tb01823.x · Cited by 859
Fishbein, M. (1980) · Lawrence Erlbaum Associates eBooks · Cited by 760
Theories in Cognitive Psychology: The Loyola Symposium
Wallace, W. P., Laughlin, P. R., & Solso, R. L. (1975) · The American Journal of Psychology · doi:10.2307/1421683 · Cited by 726
Cognitive Processes in Choice and Decision Behavior
Wallsten, T. S., Einhorn, H. J., Ebbesen, E. Β., Konečni, V. J., & Corbin, R. M. (1979) · doi:10.21236/ada078481 · Cited by 667
Alloy, L. B., & Kayne, N. T. (1984) · Psychological Review · doi:10.1037/0033-295x.91.1.112 · Cited by 616
Beyond discrete biases: Functional and dysfunctional aspects of judgmental heuristics.
Hogarth, R. M. (1981) · Psychological Bulletin · doi:10.1037/0033-2909.90.2.197 · Cited by 611
Causal schemas in judgments under uncertainty
Tversky, A., & Kahneman, D. J. (1982) · Cambridge University Press eBooks · doi:10.1017/cbo9780511809477.009 · Cited by 605
Judgment of contingency between responses and outcomes.
Jenkins, H. M., & Ward, W. C. (1965) · The Psychological Monographs · doi:10.1037/h0093874 · Cited by 593
Hypothesis evaluation from a Bayesian perspective.
Fischhoff, B., & Marom, T. (1983) · Psychological Review · doi:10.1037/0033-295x.90.3.239 · Cited by 539
d’Arcais, G. B. F., & Levelt, W. J. M. (1970) · Cited by 525
Confirmation Bias in a Simulated Research Environment: An Experimental Study of Scientific Inference
Mynatt, C. R., Doherty, M. E., & Tweney, R. D. (1977) · Quarterly Journal of Experimental Psychology · doi:10.1080/00335557743000053 · Cited by 521
Quest for accuracy in person perception: A matter of pragmatics.
Swann, W. B. (1984) · Psychological Review · doi:10.1037/0033-295x.91.4.457 · Cited by 475
Judgment of covariation by social perceivers.
Crocker, J. (1981) · Psychological Bulletin · doi:10.1037/0033-2909.90.2.272 · Cited by 445
Scientist as Subject: The Psychological Imperative.
Lewis, L. S., & Mahoney, M. J. (1978) · Contemporary Sociology A Journal of Reviews · doi:10.2307/2064480 · Cited by 415
Duda, R. O., & Shortliffe, E. H. (1983) · Science · doi:10.1126/science.6340198 · Cited by 399
Hypothesis behavior by humans during discrimination learning.
Levine, M. (1966) · Journal of Experimental Psychology · doi:10.1037/h0023006 · Cited by 387
THE CONCEPT OF CORRELATION IN ADULTS
Smedslund, J. (1963) · Scandinavian Journal of Psychology · doi:10.1111/j.1467-9450.1963.tb01324.x · Cited by 365
Edwards, W. (1965) · Journal of Mathematical Psychology · doi:10.1016/0022-2496(65)90007-6 · Cited by 335
Thinking and Reasoning: Psychological Approaches
Dominowski, R. L., & Evans, J. S. B. T. (1985) · The American Journal of Psychology · doi:10.2307/1422779 · Cited by 324
Falsification and the Methodology of Scientific Research Programmes
LAKATOS, I. (2014) · Philosophy, Science, and History · doi:10.4324/9780203802458-7 · Cited by 318
The display of information and the judgment of contingency.
Ward, W. C., & Jenkins, H. M. (1965) · Canadian Journal of Psychology/Revue Canadienne de Psychologie · doi:10.1037/h0082908 · Cited by 318
Sensible Reasoning: A Hypothesis about Hypotheses
Tschirgi, J. E. (1980) · Child Development · doi:10.2307/1129583 · Cited by 314
MATCHING BIAS IN THE SELECTION TASK
Evans, J. S. B. T., & Lynch, J. (1973) · British Journal of Psychology · doi:10.1111/j.2044-8295.1973.tb01365.x · Cited by 311
Skov, R. B., & Sherman, S. J. (1986) · Journal of Experimental Social Psychology · doi:10.1016/0022-1031(86)90031-4 · Cited by 310
Evaluation of evidence in causal inference.
Schustack, M. W., & Sternberg, R. J. (1981) · Journal of Experimental Psychology General · doi:10.1037/0096-3445.110.1.101 · Cited by 300
The subjective side of science
Martino, J. P., & Editor, O. (1976) · Technological Forecasting and Social Change · doi:10.1016/0040-1625(76)90016-0 · Cited by 292
Confirmatory and diagnosing strategies in social information gathering.
Trope, Y., & Bassok, M. (1982) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.43.1.22 · Cited by 287
Perception: An approach to personality.
(1951) · doi:10.1037/11505-000 · Cited by 283
The subjective side of science
(1975) · Futures · doi:10.1016/0016-3287(75)90055-5 · Cited by 276
Personality dynamics and the process of perceiving.
Bruner, J. S. (1951) · doi:10.1037/11505-005 · Cited by 254
Consequences of Confirmation and Disconfirmation in a Simulated Research Environment
Mynatt, C. R., Doherty, M. E., & Tweney, R. D. (1978) · Quarterly Journal of Experimental Psychology · doi:10.1080/00335557843000007 · Cited by 233
Learning from experience and suboptimal rules in decision making
Einhorn, H. J. (1982) · Cambridge University Press eBooks · doi:10.1017/cbo9780511809477.020 · Cited by 227
Rumain, B., Connell, J. W., & Braine, M. D. S. (1983) · Developmental Psychology · doi:10.1037//0012-1649.19.4.471 · Cited by 213
Reinforcement-induced behavioral stereotypy: How not to teach people to discover rules.
Schwartz, B. N. (1982) · Journal of Experimental Psychology General · doi:10.1037/0096-3445.111.1.23 · Cited by 195
Estimates of contingency between two dichotomous variables.
Arkes, H. R., & Harkness, A. R. (1983) · Journal of Experimental Psychology General · doi:10.1037/0096-3445.112.1.117 · Cited by 191
Strategies of Rule Discovery in an Inference Task
Tweney, R. D., Doherty, M. E., Worner, W. J., Pliske, D. B., Mynatt, C. R., Gross, K. A., & Arkkelin, D. L. (1980) · Quarterly Journal of Experimental Psychology · doi:10.1080/00335558008248237 · Cited by 183
Does Scientific Discovery Have a Logic?
Simon, H. A. (1973) · Philosophy of Science · doi:10.1086/288559 · Cited by 175
Biased Questions in Judgment of Covariation Studies
Crocker, J. (1982) · Personality and Social Psychology Bulletin · doi:10.1177/0146167282082005 · Cited by 166
Amano, S. (1923) · The Japanese journal of psychology · doi:10.4992/jjpsy1919.t1.356 · Cited by 162
The effects of experience on performance in Wason’s selection task
Cox, J. R., & Griggs, R. A. (1982) · Memory & Cognition · doi:10.3758/bf03197653 · Cited by 150
Hypothesis plausibility and hypothesis generation
Gettys, C. F., & Fisher, S. D. (1979) · Organizational Behavior and Human Performance · doi:10.1016/0030-5073(79)90018-7 · Cited by 148
Does Scientific Discovery Have a Logic?
Simon, H. A. (1977) · Boston studies in the philosophy of science · doi:10.1007/978-94-010-9521-1_18 · Cited by 145
The Effects of Problem Content and Negation on Wason's Selection Task
Griggs, R. A., & Cox, J. R. (1983) · The Quarterly Journal of Experimental Psychology Section A · doi:10.1080/14640748308402486 · Cited by 141
Information-gathering strategies in hypothesis-testing
Trope, Y., & Bassok, M. (1983) · Journal of Experimental Social Psychology · doi:10.1016/0022-1031(83)90016-1 · Cited by 136
Sources of error in judging event covariations: Effects of memory demands.
Shaklee, H., & Mims, M. (1982) · Journal of Experimental Psychology Learning Memory and Cognition · doi:10.1037/0278-7393.8.3.208 · Cited by 111
A rule analysis of judgments of covariation between events
Shaklee, H., & Tucker, D. C. (1980) · Memory & Cognition · doi:10.3758/bf03211142 · Cited by 107
A Comparison of Disconfirmatory, Confirmatory and Control Strategies on Wason's 2–4–6 Task
Gorman, M. E., & Gorman, M. E. (1984) · The Quarterly Journal of Experimental Psychology Section A · doi:10.1080/14640748408402183 · Cited by 105
How the possibility of error affects falsification on a task that models scientific problem solving
Gorman, M. E. (1986) · British Journal of Psychology · doi:10.1111/j.2044-8295.1986.tb01984.x · Cited by 104
>On the Failure to Eliminate Hypotheses...< – A Second Look
Wason, P. C. (1968) · Cited by 104
Faraday’s Discovery of Induction: A Cognitive Approach
Tweney, R. D. (1985) · doi:10.1007/978-1-349-11139-8_11 · Cited by 103
Laws of language use and formal logic
Politzer, G. (1986) · Journal of Psycholinguistic Research · doi:10.1007/bf01067391 · Cited by 99
Testing Hypotheses about Other People
Snyder, M., & Campbell, B. (1980) · Personality and Social Psychology Bulletin · doi:10.1177/014616728063015 · Cited by 93
Eliminative and Enumerative Behaviour in a Conceptual Task
Wetherick, N. E. (1962) · Quarterly Journal of Experimental Psychology · doi:10.1080/17470216208416542 · Cited by 93
Making decisions under the influence of memory.
Fox, J. (1980) · Psychological Review · doi:10.1037/0033-295x.87.2.190 · Cited by 76
Development of Rule Use in Judgments of Covariation between Events
Shaklee, H., & Mims, M. (1981) · Child Development · doi:10.1111/j.1467-8624.1981.tb03047.x · Cited by 68
The effect of thematic content on cognitive strategies in the four-card selection task
Yachanin, S. A., & Tweney, R. D. (1982) · Bulletin of the Psychonomic Society · doi:10.3758/bf03330048 · Cited by 67
Where leading questions can lead: The power of conjecture in social interaction.
Swann, W. B., Giuliano, T., & Wegner, D. M. (1982) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.42.6.1025 · Cited by 67
Learning conceptual rules: II. The role of positive and negative instances.
Bourne, L. E., & Guy, D. E. (1968) · Journal of Experimental Psychology · doi:10.1037/h0025952 · Cited by 63
Development of Rule Use in Judgments of Covariation between Events
Shaklee, H., & Mims, M. (1981) · Child Development · doi:10.2307/1129245 · Cited by 63
AN OPERANT DISCRIMINATION TASK ALLOWING VARIABILITY OF REINFORCED RESPONSE PATTERNING1
Vogel, R., & Annau, Z. (1973) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.1973.20-1 · Cited by 55
The questions lay interviewers ask
Trope, Y., Bassok, M., & Alon, E. (1984) · Journal of Personality · doi:10.1111/j.1467-6494.1984.tb00552.x · Cited by 53
Confirmatory Search Strategies in Social Interaction: How, When, Why, and With What Consequences
Swann, W. B., & Giuliano, T. (1987) · Journal of Social and Clinical Psychology · doi:10.1521/jscp.1987.5.4.511 · Cited by 52
Counselor hypothesis-testing strategies.
Strohmer, D. C., & Newman, L. J. (1983) · Journal of Counseling Psychology · doi:10.1037/0022-0167.30.4.557 · Cited by 49
Human discrimination learning: The subset-sampling assumption.
Levine, M. (1970) · Psychological Bulletin · doi:10.1037/h0032951 · Cited by 49
A Hypothesis-Confirming Bias in Labeling Effects
Darley, J. M., & Gross, P. H. (2004) · Social Cognition · doi:10.4324/9780203496398-27 · Cited by 35
Evaluation of evidence in causal inference.
Schustack, M. W., & Sternberg, R. J. (1981) · Journal of Experimental Psychology General · doi:10.1037//0096-3445.110.1.101 · Cited by 32
Logical knowledge and cue redundancy in deductive reasoning
Hoch, S. J., & Tschirgi, J. E. (1985) · Memory & Cognition · doi:10.3758/bf03198458 · Cited by 31
Rationality and the psychology of inference
Tweney, R. D., & Doherty, M. E. (1983) · Synthese · doi:10.1007/bf01063999 · Cited by 30
An Inference Model for Conceptual Rule Learning 1
Bourne, L. E. (2024) · Routledge eBooks · doi:10.4324/9781032722375-11 · Cited by 29
The direct measurement of hypothesis-sampling strategies
Millward, R. B., & Spoehr, K. T. (1973) · Cognitive Psychology · doi:10.1016/0010-0285(73)90002-9 · Cited by 29
Rationality and Disconfirmation: Further Evidence
Tweney, R. D., Doherty, M. E., & Mynatt, C. R. (1982) · Social Studies of Science · doi:10.1177/030631282012003004 · Cited by 29
Control of complex, sequential operants by systematic visual information in pigeons.
Schwartz, B. N. (1981) · Journal of Experimental Psychology Animal Behavior Processes · doi:10.1037/0097-7403.7.1.31 · Cited by 27
Sensible Reasoning: A Hypothesis about Hypotheses
Tschirgi, J. E. (1980) · Child Development · doi:10.1111/j.1467-8624.1980.tb02502.x · Cited by 24
Cue redundancy and extra logical inferences in a deductive reasoning task
Hoch, S. J., & Tschirgi, J. E. (1983) · Memory & Cognition · doi:10.3758/bf03213475 · Cited by 23
Wason, P. C. (1962) · Quarterly Journal of Experimental Psychology · doi:10.1080/17470216208416543 · Cited by 23
Kern, L. H. (1982) · OhioLink ETD Center (Ohio Library and Information Network) · Cited by 16
Studies in the History of Psychology and the Social Sciences 3
Bem, S., Rappard, H. V., & van Horn, W. (1985) · Cited by 15
Evaluation of hypotheses in concept identification
Taplin, J. E. (1975) · Memory & Cognition · doi:10.3758/bf03198215 · Cited by 15
Hypothesis Generation and Plausibility Assessment.
Gettys, C. F., Fisher, S. D., & Mehle, T. (1978) · Defense Technical Information Center (DTIC) · Cited by 10
Sources of error in judging event covariations: Effects of memory demands.
Shaklee, H., & Mims, M. (1982) · Journal of Experimental Psychology Learning Memory and Cognition · doi:10.1037//0278-7393.8.3.208 · Cited by 10
Making decisions under the influence of memory.
Fox, J. (1980) · Psychological Review · doi:10.1037//0033-295x.87.2.190 · Cited by 7
Rationality and Disconfirmation
Hardin, C. L. (1980) · Social Studies of Science · doi:10.1177/030631278001000407 · Cited by 6
Control of complex, sequential operants by systematic visual information in pigeons.
Schwartz, B. N. (1981) · Journal of Experimental Psychology Animal Behavior Processes · doi:10.1037//0097-7403.7.1.31 · Cited by 4
A Cross-Study Analysis of Covariation Judgments.
Lipe, M. G. (1982) · Defense Technical Information Center (DTIC) · Cited by 3
Rationality and Authority: On the Confusion of Justification and Permission
Mahoney, M. J. (1980) · Social Studies of Science · doi:10.1177/030631278001000408 · Cited by 3
Cited by
Sorted by citations
Strategic assets and organizational rent
Amit, R., & Schoemaker, P. J. H. (1993) · Strategic Management Journal · doi:10.1002/smj.4250140105 · Cited by 8,425
The case for motivated reasoning.
Kunda, Z. (1990) · Psychological Bulletin · doi:10.1037/0033-2909.108.3.480 · Cited by 9,022
Confirmation Bias: A Ubiquitous Phenomenon in Many Guises
Nickerson, R. S. (1998) · Review of General Psychology · doi:10.1037/1089-2680.2.2.175 · Cited by 7,267
Individual differences in reasoning: Implications for the rationality debate?
Stanovich, K. E., & West, R. F. (2000) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00003435 · Cited by 4,481
Heuristics and Biases: The Psychology of Intuitive Judgment
Bottom, W. P., Gilovich, T. D., Griffin, D., & Kahneman, D. J. (2004) · Academy of Management Review · doi:10.2307/20159081 · Cited by 3,583
Metadata
- Authors
- Klayman, J., & Ha, Y.-W.
- Affiliations
- University of Chicago (US)
- ORCID
- Klayman, Joshua
- Venue
- Psychological Review
- Published
- 1987-04
- Volume
- 94
- Issue
- 2
- Pages
- 211-228
- ISSN
- 0033-295X (ISSN-L)
- 1939-1471
- OpenAlex
- W2124371667
- Cited by
- 2,094
- References
- 107
- Citations by year
- 2026: 28 · 2025: 35 · 2024: 53 · 2023: 51 · 2022: 44 · 2021: 71 · 2020: 60 · 2019: 60 · 2018: 70 · 2017: 72 · 2016: 47 · 2015: 57 · 2014: 53 · 2013: 61 · 2012: 71
- FWCI
- 22.99
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Psychological Review (publishedVersion)
- Funders
- National Science Foundation (8309586)
- Topics
- Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
- Social Sciences › Psychology › General Psychology › Academic and Historical Perspectives in Psychology
- Social Sciences › Arts and Humanities › Philosophy › Epistemology, Ethics, and Metaphysics
- Keywords
- hypothesis testing, confirmation bias, disconfirmation, falsification, heuristics, scientific reasoning, everyday reasoning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
