Large language models surpass human experts in predicting neuroscience results
Open accessThe DOI matches its Crossref record; 22 of the 22 references it cites were found in a registry or an index.
Abstract
Scientific discoveries often hinge on synthesizing decades of research, a task that potentially outstrips human information processing capacities. Large language models (LLMs) offer a solution. LLMs trained on the vast scientific literature could potentially integrate noisy yet interrelated findings to forecast novel results better than human experts. Here, to evaluate this possibility, we created BrainBench, a forward-looking benchmark for predicting neuroscience results. We find that LLMs surpass experts in predicting experimental outcomes. BrainGPT, an LLM we tuned on the neuroscience literature, performed better yet. Like human experts, when LLMs indicated high confidence in their predictions, their responses were more likely to be correct, which presages a future where LLMs assist humans in making discoveries. Our approach is not neuroscience specific and is transferable to other knowledge-intensive endeavours.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Decoupled Weight Decay Regularization
Loshchilov, I., & Hutter, F. (2017) · arXiv · doi:10.48550/arxiv.1711.05101 · Cited by 12,522
Chain-Of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q. V., & Zhou, D. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1800 · Cited by 4,505
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.-T., Rocktäschel, T., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.11401 · Cited by 3,101 · cited 2×
“Unlike knowledge graphs that store verified facts, LLMs may not be trustworthy for backward-looking tasks such as summarizing research papers or providing accurate citations ( 17 ).”
Highly accurate protein structure prediction for the human proteome
Tunyasuvunakool, K., Adler, J., Wu, Z., Green, T., Zielinski, M., Žídek, A., Bridgland, A., Cowie, A., Meyer, C., Laydon, A., et al. (2021) · Nature · doi:10.1038/s41586-021-03828-1 · Cited by 3,025 · cited 1×
“This approach could take several forms, including specialist solutions that address specific challenges, such as in protein folding 3 , drug discovery 4 and materials science 5 .”
Llama 2: Open Foundation and Fine-Tuned Chat Models
Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.09288 · Cited by 2,604
Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena
Zheng, L., Chiang, W.-L., Sheng, Y., Zhuang, S., Wu, Z., Zhuang, Y., Lin, Z., Li, Z., Li, D., Xing, E. P., et al. (2023) · arXiv · doi:10.48550/arxiv.2306.05685 · Cited by 1,888 · cited 1×
“Prompting in natural language may yield less reliable judgments and degrade model competency compared to using model probability scores or training separate classifiers directly from internal representations ( 22–25 ).”
Bornmann, L., & Mutz, R. (2015) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23329 · Cited by 1,576
Deep learning enables rapid identification of potent DDR1 kinase inhibitors
Zhavoronkov, A., Ivanenkov, Y. A., Aliper, A., Veselov, M. S., Aladinskiy, V. A., Aladinskaya, A. V., Terentiev, V. A., Polykovskiy, D. A., Kuznetsov, M. D., Asadulaev, A., et al. (2019) · Nature Biotechnology · doi:10.1038/s41587-019-0224-x · Cited by 1,573 · cited 1×
“This approach could take several forms, including specialist solutions that address specific challenges, such as in protein folding 3 , drug discovery 4 and materials science 5 .”
Summary of ChatGPT-Related Research and Perspective Towards the Future of Large Language Models
Liu, Y., Han, T., Ma, S., Zhang, J., Yang, Y., Tian, J., He, H., Li, A., He, M., Liu, Z., et al. (2023) · arXiv · doi:10.48550/arxiv.2304.01852 · Cited by 1,172
Unsupervised word embeddings capture latent knowledge from materials science literature
Tshitoyan, V., Dagdelen, J., Weston, L., Dunn, A., Rong, Z., Kononova, O., Persson, K. A., Ceder, G., & Jain, A. (2019) · Nature · doi:10.1038/s41586-019-1335-8 · Cited by 1,123 · cited 1×
“This approach could take several forms, including specialist solutions that address specific challenges, such as in protein folding 3 , drug discovery 4 and materials science 5 .”
Cited by
Vending-Bench: A Benchmark for Long-Term Coherence of Autonomous Agents
Backlund, A., & Petersson, L. (2025) · arXiv · doi:10.48550/arxiv.2502.15840 · Cited by 101
How should the advancement of large language models affect the practice of science?
Binz, M., Alaniz, S., Roskies, A., Aczel, B., Bergstrom, C. T., Allen, C., Schad, D., Wulff, D., West, J. D., Zhang, Q., et al. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2401227121 · Cited by 74
Generative Artificial Intelligence in Anatomic Pathology
Brodsky, V., Ullah, E., Bychkov, A., Song, A. H., Walk, E. E., Louis, P., Rasool, G., Singh, R. S., Mahmood, F., Bui, M. M., et al. (2025) · Archives of Pathology & Laboratory Medicine · doi:10.5858/arpa.2024-0215-ra · Cited by 48
Hila, A. (2026) · AI & SOCIETY · doi:10.1007/s00146-025-02426-3 · Cited by 14
“…not trivial in light of the remarkable capacities of current LLMs to learn from data and surpass human capacities in speed of linguistic generation on the one hand, and a range of cognitive tasks on the other (Luo et al. 2024 (Dayan et al. 1995; LeCun et al. 2015; Smolensky 1987; Sun et al. 2001).”
Hermann, D. M., Bacigaluppi, M., Bassetti, C. L., Bassotti, G., Boltze, J., Chan, A., Dalkara, T., Denes, A., Diez-Tejedor, E., Dodel, R., et al. (2025) · Exploration of Neuroscience · doi:10.37349/en.2025.1006106 · Cited by 13
“The processing of research data is substantially enhanced by artificial intelligence (AI; e.g., deep learning) strategies, allowing the extraction of features from large data sets that hitherto escaped the attention of scientists [25].”
Self-citation
Metadata
- Authors
- Luo, X., Rechardt, A., Sun, G., Nejad, K. K., Yáñez, F., Yilmaz, B., Lee, K., Cohen, A. O., Borghesani, V., Pashkov, A., Marinazzo, D., Nicholas, J., Salatiello, A., Sucholutsky, I., Minervini, P., Razavi, S., Rocca, R., Yusifov, E., Okalova, T., Gu, N., Ferianc, M., Khona, M., Patil, K. R., Lee, P.-S., Mata, R., Myers, N. E., Bizley, J. K., Musslick, S., Bilgin, I. P., Niso, G., Ales, J. M., Gaebler, M., Ratan Murty, N. A., Loued-Khenissi, L., Behler, A., Hall, C. M., Dafflon, J., Bao, S. D., & Love, B. C.
- Venue
- Nature Human Behaviour
- Published
- 2024-11-27
- Volume
- 9
- Issue
- 2
- Pages
- 305-315
- ISSN
- 2397-3374 (ISSN-L)
- OpenAlex
- W4404764563
- Cited by
- 139
- References
- 22
- FWCI
- 20.69
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Large language models surpass human experts in predicting neuroscience results. https://app.cochat.ai/papers/10.1038/s41562-024-02046-9
- The paper
- Luo, X., Rechardt, A., Sun, G., Nejad, K. K., Yáñez, F., Yilmaz, B., Lee, K., Cohen, A. O., Borghesani, V., Pashkov, A., Marinazzo, D., Nicholas, J., Salatiello, A., Sucholutsky, I., Minervini, P., Razavi, S., Rocca, R., Yusifov, E., Okalova, T., . . . Love, B. C. (2024). Large language models surpass human experts in predicting neuroscience results. Nature Human Behaviour. https://doi.org/10.1038/s41562-024-02046-9
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.