CoChat

Improving the accuracy of medical diagnosis with causal machine learning

Open access
Richens, Jonathan G., Lee, Ciarán M., Johri, Saurabh · 2020 · Nature Communications

The DOI matches its Crossref record; 92 of the 92 references it cites were found in a registry or an index.

Abstract

Machine learning promises to revolutionize clinical decision making and diagnosis. In medical diagnosis a doctor aims to explain a patient’s symptoms by determining the diseases causing them. However, existing machine learning approaches to diagnosis are purely associative, identifying diseases that are strongly correlated with a patients symptoms. We show that this inability to disentangle correlation from causation can result in sub-optimal or dangerous diagnoses. To overcome this, we reformulate diagnosis as a counterfactual inference task and derive counterfactual diagnostic algorithms. We compare our counterfactual algorithms to the standard associative algorithm and 44 doctors using a test set of clinical vignettes. While the associative algorithm achieves an accuracy placing in the top 48% of doctors in our cohort, our counterfactual algorithm places in the top 25% of doctors, achieving expert clinical accuracy. Our results show that causal reasoning is a vital missing ingredient for applying machine learning to medical diagnosis.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1038/s41467-020-17419-7

Cites

92 of 92 references verified0 citation errors · 0 not checkable

Statistics

Figures

Fig. 1 Three distinct causal structures for diseases and symptoms. a Disease D is a direct cause of symptom S, b D does not cause symptom S, but they are correlated by a latent common cause R, c D is a direct cause of S and a latent common cause R is present.
Fig. 1 Three distinct causal structures for diseases and symptoms. a Disease D is a direct cause of symptom S, b D does not cause symptom S, but they are correlated by a latent common cause R, c D is a direct cause of S and a latent common cause R is present.CC BY · PubMed Central
Fig. 2 Generative structure of our diagnostic Bayesian networks. a Three-layer Bayesian network representing risk factors Ri, diseases Dj and symptoms Sk. b noisy-OR CPT. S is the Boolean OR function of its parents, each with an independent probability λi of being ignored, removing them from the OR function.
Fig. 2 Generative structure of our diagnostic Bayesian networks. a Three-layer Bayesian network representing risk factors Ri, diseases Dj and symptoms Sk. b noisy-OR CPT. S is the Boolean OR function of its parents, each with an independent probability λi of being ignored, removing them from the OR function.CC BY · PubMed Central
Fig. 3 Top k accuracy of Bayesian and counterfactual algorithms. Figure shows the top k error (1—accuracy) of the counterfactual (green line) and associative (blue line) algorithms over all 1671 vignettes vs k. Shaded regions give 95% confidence intervals. The black dashed line shows the relative reduction in error when switching from the associative to counterfactual algorithm, given by 1 − ec/ea where ea is the error rate of the associative algorithm, and ec is the error rate of the counterfactual algorithm. Results shown for k = 1, …15, for complete results see the supplementary table 1.
Fig. 3 Top k accuracy of Bayesian and counterfactual algorithms. Figure shows the top k error (1—accuracy) of the counterfactual (green line) and associative (blue line) algorithms over all 1671 vignettes vs k. Shaded regions give 95% confidence intervals. The black dashed line shows the relative reduction in error when switching from the associative to counterfactual algorithm, given by 1 − ec/ea where ea is the error rate of the associative algorithm, and ec is the error rate of the counterfactual algorithm. Results shown for k = 1, …15, for complete results see the supplementary table 1.CC BY · PubMed Central
Fig. 4 Mean accuracy of each doctor compared to Bayesian and counterfactual algorithms. Figure shows the mean accuracy for each of the 44 doctors, compared to the posterior ranking (top) and expected sufficiency ranking (bottom) algorithms. The line y = x gives a reference for comparing the accuracy of each doctor to the algorithm shadowing them. Points above the line correspond to doctors who achieved a lower accuracy than the algorithm (blue), points on the line are doctors that achieved the same accuracy as the algorithm (red), and below the line are doctors that achieved higher accuracy than the algorithm (green). The linear correlation can be explained by the variation in the difficulty of the sets of vignettes diagnosed by each doctor. Sets of easier/harder vignettes results in higher/lower doctor and algorithm accuracy scores. As the results for the expected disablement and expected sufficiency are almost identical, we show only the results for the expected sufficiency. Complete results are listed in the Supplementary Table 3. All figures generated using matplotlib version 3.2.1.
Fig. 4 Mean accuracy of each doctor compared to Bayesian and counterfactual algorithms. Figure shows the mean accuracy for each of the 44 doctors, compared to the posterior ranking (top) and expected sufficiency ranking (bottom) algorithms. The line y = x gives a reference for comparing the accuracy of each doctor to the algorithm shadowing them. Points above the line correspond to doctors who achieved a lower accuracy than the algorithm (blue), points on the line are doctors that achieved the same accuracy as the algorithm (red), and below the line are doctors that achieved higher accuracy than the algorithm (green). The linear correlation can be explained by the variation in the difficulty of the sets of vignettes diagnosed by each doctor. Sets of easier/harder vignettes results in higher/lower doctor and algorithm accuracy scores. As the results for the expected disablement and expected sufficiency are almost identical, we show only the results for the expected sufficiency. Complete results are listed in the Supplementary Table 3. All figures generated using matplotlib version 3.2.1.CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 92 references

Research works printed but not linked: 4

  1. Heckerman, D. E., Horvitz, E. J. & Nathwani, B. N. Toward normative expert systems: Part i the pathfinder project. Methods Inf. Med. 31, 90–105 (1992). · DOI · Paper page
  2. Benzi, M. Medical diagnosis and actual causation. LPS-Log. Philos. Sci. 9, 365–372 (2011).
  3. Peters, J., Janzing, D. & Schölkopf, B. Elements of Causal Inference-Foundations and Learning Algorithms (The MIT Press, 2017).
  4. Higgs, J., Jones, M. A., Loftus, S. & Christensen, N. Clinical Reasoning in the Health Professions E-Book (Elsevier Health Sciences, 2008).

Cited by

See all 777

Metadata

Authors
Richens, J. G., Lee, C. M., & Johri, S.
Affiliations
Babylon Health (GB)
Corresponding author
Richens, Jonathan G.
Venue
Nature Communications
Published
2020-08-11
Volume
11
Issue
1
Pages
3923-3923
ISSN
2041-1723 (ISSN-L)
OpenAlex
W3048817558
Cited by
598
References
84
Citations by year
2026: 59 · 2025: 128 · 2024: 126 · 2023: 124 · 2022: 90 · 2021: 68 · 2020: 3
FWCI
62.43
Type
journal article
Language
en
Open access
Licence
Locations
  • Nature Communications (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • RePEc: Research Papers in Economics (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Topics
  • Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
Keywords
diagnosis, causal inference, counterfactual reasoning, correlation versus causation, clinical decision support, clinical vignettes, clinical reasoning
MeSH terms
Data Accuracy, Machine Learning, Algorithms, Bayes Theorem, Data Collection, Decision Making, Diagnosis, Diagnosis, Computer-Assisted, Disease, Humans, Models, Statistical

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Improving the accuracy of medical diagnosis with causal machine learning. https://app.cochat.ai/papers/10.1038/s41467-020-17419-7
The paper
Richens, J. G., Lee, C. M., & Johri, S. (2020). Improving the accuracy of medical diagnosis with causal machine learning. Nature Communications. https://doi.org/10.1038/s41467-020-17419-7

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.