Theory and data interactions of the scientific mind: Evidence from the molecular and the cognitive laboratory.
The DOI matches its Crossref record; 3 of the 3 references it cites were found in a registry or an index.
Abstract
A number of researchers and scholars have stressed the importance of disconfirmation in the quest for the development of scientific knowledge (e.g., Popper, 1959). Paradoxically, studies examining human reasoning in the laboratory have typically found that people display a confirmation bias in that they are more likely to seek out and attend to data consistent rather than data inconsistent with their initial theory (Wason, 1968). We examine the strategies that scientists and students use to evaluate data that are either consistent or inconsistent with their expectations. First, we present findings from scientists reasoning "live" in their laboratory meetings. We show that scientists often show an initial reluctance to consider inconsistent data as "real." However, this initial reluctance is often overcome with repeated observations of the inconsistent data such that they modify their theories to account for the new data. We further examine these issues in a controlled scientific causal thinking simulation specifically developed to examine the reasoning strategies we observed in the natural scientific environment. Like the scientists, we found that participants in our simulation initially displayed a propensity to discount data inconsistent with a theory provided. However, with repeated observations of the inconsistent data, the students, like the scientists, began to see the once anomalous data as "real" and the initial bias to discount that data was significantly diminished.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Confirmation, disconfirmation, and information in hypothesis testing
Klayman, J., & Ha, Y.-W. (1987) · Psychological Review · doi:10.1037/0033-295x.94.2.211 · Cited by 2,094
Koriat, A., Lichtenstein, S., & Fischhoff, B. (1980) · Journal of Experimental Psychology: Human Learning and Memory · doi:10.1037/0278-7393.6.2.107 · Cited by 1,477
Wason, P. C. (1968) · Quarterly Journal of Experimental Psychology · doi:10.1080/14640746808400161 · Cited by 1,476
Cited by
The Oxford Handbook of Thinking and Reasoning
(2012) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780199734689.001.0001 · Cited by 697
The heuristic-analytic theory of reasoning: Extension and evaluation
Evans, J. S. B. T. (2006) · Psychonomic Bulletin & Review · doi:10.3758/bf03193858 · Cited by 643
The (mis)reporting of statistical results in psychology journals
Bakker, M., & Wicherts, J. M. (2011) · Behavior Research Methods · doi:10.3758/s13428-011-0089-5 · Cited by 356
“…Several studies have shown that scientists are subject to confirmation bias in analyzing their data; that is, their reaction to empirical results depends on whether these results support their hypotheses (Edwards & Smith, 1996; Fugelsang et al. 2004; Koehler, 1993; Mynatt et al. 1977).”
BackgroundResultHow scientists fool themselves – and how they can stop
Nuzzo, R. (2015) · Nature · doi:10.1038/526182a · Cited by 296
Scientific Thinking and Reasoning
Dunbar, K. N., & Klahr, D. (2012) · The Oxford Handbook of Thinking and Reasoning · doi:10.1093/oxfordhb/9780199734689.013.0035 · Cited by 173
Self-citation
Metadata
- Authors
- Fugelsang, J. A., Stein, C. B., Green, A. E., & Dunbar, K. N.
- Venue
- Canadian Journal of Experimental Psychology / Revue canadienne de psychologie expérimentale
- Published
- 2004-06
- Volume
- 58
- Issue
- 2
- Pages
- 86-95
- ISSN
- 1196-1961 (ISSN-L)
- 1878-7290
- OpenAlex
- W2124008828
- Cited by
- 115
- References
- 3
- FWCI
- 24.04
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.