CoChat

Superhuman artificial intelligence can improve human decision-making by increasing novelty

Open access
Shin, Minkyu, Kim, Jin, van Opheusden, Bas, Griffiths, Thomas L. · 2023 · Proceedings of the National Academy of Sciences

Published version of a preprint

The DOI matches its Crossref record; 31 of the 33 references it cites were found in a registry or an index, and 1 are not checked yet.

Abstract

How will superhuman artificial intelligence (AI) affect human decision-making? And what will be the mechanisms behind this effect? We address these questions in a domain where AI already exceeds human performance, analyzing more than 5.8 million move decisions made by professional Go players over the past 71 y (1950 to 2021). To address the first question, we use a superhuman AI program to estimate the quality of human decisions across time, generating 58 billion counterfactual game patterns and comparing the win rates of actual human decisions with those of counterfactual AI decisions. We find that humans began to make significantly better decisions following the advent of superhuman AI. We then examine human players' strategies across time and find that novel decisions (i.e., previously unobserved moves) occurred more frequently and became associated with higher decision quality after the advent of superhuman AI. Our findings suggest that the development of superhuman AI programs may have prompted human players to break away from traditional strategies and induced them to explore novel moves, which in turn may have improved their decision-making.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1073/pnas.2214840120

Cites

31 of 32 references verified0 citation errors · 0 not checkable

Statistics

Figures

Fig. 1. Historical changes in quality and novelty of human decisions in Go. Panel A (Panel B) shows the fixed effect of each year (month) on decision quality along with its 95% CI, estimated using the median of Decision Quality Indices of all decisions made by each player in each year (month). Similarly, Panel C (Panel D) shows the fixed effect of each year (month) on novelty as measured with the Novelty Index, along with its 95% CI, estimated using the median of Novelty Indices of all games for each player in each year (month).
Fig. 1. Historical changes in quality and novelty of human decisions in Go. Panel A (Panel B) shows the fixed effect of each year (month) on decision quality along with its 95% CI, estimated using the median of Decision Quality Indices of all decisions made by each player in each year (month). Similarly, Panel C (Panel D) shows the fixed effect of each year (month) on novelty as measured with the Novelty Index, along with its 95% CI, estimated using the median of Novelty Indices of all games for each player in each year (month).CC BY-NC-ND · PubMed Central
Fig. 2. Yearly fixed effects on DQI, constructed from a dataset that excludes the move decisions that matched the counterfactual (optimal) decisions of the AI program. We find a similar increase in human decision quality following the advent of superhuman AI. The upward time trends of DQI after 2016 are still evident even when analyzing the restricted set of move decisions. This suggests that memorization hypothesis cannot fully explain the post-AI increase in decision quality.
Fig. 2. Yearly fixed effects on DQI, constructed from a dataset that excludes the move decisions that matched the counterfactual (optimal) decisions of the AI program. We find a similar increase in human decision quality following the advent of superhuman AI. The upward time trends of DQI after 2016 are still evident even when analyzing the restricted set of move decisions. This suggests that memorization hypothesis cannot fully explain the post-AI increase in decision quality.CC BY-NC-ND · PubMed Central
Fig. 3. Yearly fixed effects on Novelty Index, constructed from a dataset that excludes the move decisions that matched the counterfactual (optimal) decisions of the AI program. Even when this restricted set of move decisions were analyzed, we still find a similar increase in novelty following the advent of superhuman AI.
Fig. 3. Yearly fixed effects on Novelty Index, constructed from a dataset that excludes the move decisions that matched the counterfactual (optimal) decisions of the AI program. Even when this restricted set of move decisions were analyzed, we still find a similar increase in novelty following the advent of superhuman AI.CC BY-NC-ND · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 32 references

Research works printed but not linked: 2

  1. G. Burgess The Mammoth Book of Chess (Constable& Robinson Ltd. 2009) pp. 924–929.
  2. M. Shin J. Kim M. Kim “Human learning from artificial intelligence: Evidence from human Go players’ decisions after AlphaGo” in Proceedings of the Annual Meeting of the Cognitive Science Society (2021) vol. 43 .

Cited by

See all 125

Metadata

Authors
Shin, M., Kim, J., van Opheusden, B., & Griffiths, T. L.
Affiliations
  • City University of Hong Kong (HK)
  • Tongji University (CN)
  • Department of Psychology, Princeton University, Princeton, NJ 08540
  • Princeton University (US)
Corresponding author
  • Shin, Minkyu
  • Kim, Jin
Venue
Proceedings of the National Academy of Sciences
Published
2023-03-13
Volume
120
Issue
12
Pages
e2214840120-e2214840120
ISSN
  • 0027-8424 (ISSN-L)
  • 1091-6490
OpenAlex
W4324028585
Cited by
106
References
33
Citations by year
2026: 30 · 2025: 37 · 2024: 25 · 2023: 14
FWCI
9.99
Type
journal article
Language
en
Open access
Locations
  • Proceedings of the National Academy of Sciences (publishedVersion, cc-by-nc-nd)
  • PubMed Central (publishedVersion, cc-by-nc-nd)
  • PubMed (publishedVersion)
  • RePEc: Research Papers in Economics (submittedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • CityU Scholars (submittedVersion, cc-by-nc-nd)
  • PsyArXiv (OSF Preprints) (submittedVersion, cc-by)
  • PsyArXiv (OSF Preprints) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence in Games
  • Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
  • Social Sciences › Psychology › Developmental and Educational Psychology › Sport Psychology and Performance
Keywords
human decision-making, Go, novelty, counterfactual analysis, human-AI interaction, professional Go players, game strategy
MeSH terms
Artificial Intelligence, Decision Making, Humans

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Superhuman artificial intelligence can improve human decision-making by increasing novelty. https://app.cochat.ai/papers/10.1073/pnas.2214840120
The paper
Shin, M., Kim, J., van Opheusden, B., & Griffiths, T. L. (2023). Superhuman artificial intelligence can improve human decision-making by increasing novelty. Proceedings of the National Academy of Sciences. https://doi.org/10.1073/pnas.2214840120

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.