GeneBench-Pro: Evaluating Multistage Statistical Reasoning in Genomics, Quantitative Biology, and Translational Biomedicine
Open accessAbstract
We introduce GeneBench-Pro, an expanded and improved version of GeneBench that comprises harder problems across a wider breadth of domains. GeneBench-Pro is a benchmark for AI agents performing realistic multi-stage scientific analyses in genomics, quantitative biology, and translational biomedicine which seeks to capture the complexity of real-world problems that computational life scientists face when tasked with producing a conclusion upon which a downstream scientific or translational decision is contingent. The benchmark comprises 129 evaluations targeting quantities of direct practical relevance across 10 primary domains and 21 terminal subdomains, with a genomics-centered core. Similarly to GeneBench, each problem provides the agent with brief context, a target estimand, and minimal guidance otherwise; the agent must then navigate multiple dependent decision points; i.e. , substantive inferential forks where a plausible wrong choice changes the downstream analysis, to identify and execute the correct analysis workflow and arrive at the correct answer. Relative to GeneBench, GeneBench-Pro adds 29 new problems, drops three, and introduces significantly redesigned versions of 54 of the remaining 100 overlapping problems. 82 of the 129 problems were reviewed by external domain experts, whose findings led to prompt/data modifications and redesign of those problems whose targets were not sufficiently identifiable. Ten externally reviewed problems are released publicly, 50 held-out problems were provided to Artificial Analysis for independent third-party model benchmarking, and the remainder are retained as an internal holdout. In evaluations over the full 129-problem suite, GPT-5.6 Sol reaches an eval-level pass rate of 28.7% at the max reasoning level, and GPT-5.6 Sol Pro reaches 31.5% in separately reported GPT Pro runs. GPT-5.5 reaches 12.0%, GPT-5.4 reaches 8.9%, and the strongest non-GPT baseline, Claude Opus 4.8, reaches 16.0%. As with GeneBench, models often complete substantial portions of the workflow but exhibit a consistent gap between noticing and acting by identifying local diagnostic signals but failing to propagate the implications to the corresponding analysis decision. As a result, models often select wrong estimators or persist on initially plausible but incorrect analysis paths. GeneBench-Pro therefore measures an emerging capability of long-horizon biological reasoning that remains unreliable.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Cites
The UK Biobank resource with deep phenotyping and genomic data
Bycroft, C., Freeman, C., Petkova, D., Band, G., Elliott, L. T., Sharp, K., Motyer, A., Vukcevic, D., Delaneau, O., O’Connell, J., et al. (2018) · Nature · doi:10.1038/s41586-018-0579-z · Cited by 8,876 · see citing works
Categorization and Representation of Physics Problems by Experts and Novices*
Chi, M. T. H., Feltovich, P. J., & Glaser, R. (1981) · Cognitive Science · doi:10.1207/s15516709cog0502_2 · Cited by 5,248 · see citing works
The support of human genetic evidence for approved drug indications
Nelson, M. R., Tipney, H., Painter, J. L., Shen, J., Nicoletti, P., Shen, Y., Floratos, A., Sham, P. C., Li, M. J., Wang, J., et al. (2015) · Nature Genetics · doi:10.1038/ng.3314 · Cited by 1,666 · see citing works
Simulating 500 million years of evolution with a language model
Hayes, T., Rao, R., Akin, H., Sofroniew, N. J., Oktay, D., Lin, Z., Verkuil, R., Tran, V. Q., Deaton, J., Wiggert, M., et al. (2025) · Science · doi:10.1126/science.ads0018 · Cited by 937 · see citing works
Genomic data in the all of us research program
The All of Us Research Program Genomics Investigators, Manuscript Writing Group, Bick, A. G., Metcalf, G. A., Mayo, K. R., Lichtenstein, L., Rura, S., Carroll, R. J., Musick, A., Linder, J. E., et al. (2024) · Nature · doi:10.1038/s41586-023-06957-x · Cited by 829 · see citing works
Many Analysts, One Data Set: Making Transparent How Variations in Analytic Choices Affect Results
Silberzahn, R., Uhlmann, E. L., Martin, D. P., Anselmi, P., Aust, F., Awtrey, E. C., Bahník, Š., Bai, F., Bannard, C., Bonnier, E., et al. (2018) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245917747646 · Cited by 773 · see citing works
Validating therapeutic targets through human genetics
Plenge, R. M., Scolnick, E. M., & Altshuler, D. (2013) · Nature Reviews Drug Discovery · doi:10.1038/nrd4051 · Cited by 712 · see citing works
King, E. A., Davis, J. W., & Degner, J. F. (2019) · PLOS Genetics · doi:10.1371/journal.pgen.1008489 · Cited by 649 · see citing works
Genome modelling and design across all domains of life with Evo 2
Brixi, G., Durrant, M. G., Ku, J., Naghipourfar, M., Poli, M., Sun, G., Brockman, G., Chang, D., Fanton, A., Gonzalez, G. A., et al. (2026) · Nature · doi:10.1038/s41586-026-10176-5 · Cited by 431 · see citing works
The real cost of sequencing: scaling computation to keep pace with data generation
Muir, P., Li, S., Lou, S., Wang, D., Spakowicz, D. J., Salichos, L., Zhang, J., Weinstock, G. M., Isaacs, F. J., Rozowsky, J. S., et al. (2016) · Genome biology · doi:10.1186/s13059-016-0917-0 · Cited by 369 · see citing works
Refining the impact of genetic evidence on clinical success
Minikel, E. V., Painter, J. L., Dong, C. C., & Nelson, M. R. (2024) · Nature · doi:10.1038/s41586-024-07316-0 · Cited by 338 · see citing works
What Do Experts and Novices “See” in Evolutionary Problems?
Nehm, R. H., & Ridgway, J. S. (2011) · Evolution Education and Outreach · doi:10.1007/s12052-011-0369-7 · Cited by 84 · see citing works
FrontierMath: A Benchmark for Evaluating Advanced Mathematical Reasoning in AIPreprint
Glazer, E., Erdil, E., Besiroglu, T., Chicharro, D., Chen, E., Gunning, A., Olsson, C. F., Denain, J.-S., Ho, A., Santos, T. D. O., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2411.04872 · Cited by 66 · see citing works
Self-citation(2024) · doi:10.52202/079017-1650 · Cited by 64 · see citing works
OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments
Xie, T., Zhang, D., Chen, J., Li, X., Zhao, S., Cao, R., Hua, T. J., Cheng, Z., Shin, D., Lei, F., et al. (2024) · arXiv · doi:10.48550/arxiv.2404.07972 · Cited by 64 · see citing works
Navigating bottlenecks and trade-offs in genomic data analysis
Berger, B., & Yu, Y. W. (2022) · Nature Reviews Genetics · doi:10.1038/s41576-022-00551-z · Cited by 61 · see citing works
Principles for data analysis workflows
Stoudt, S., Vásquez, V., & Martinez, C. C. (2021) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1008770 · Cited by 53 · see citing works
Benchmarking large language models for genomic knowledge with GeneTuring
Shang, X., Liao, X., Ji, Z., & Hou, W. (2025) · Briefings in Bioinformatics · doi:10.1093/bib/bbaf492 · Cited by 53 · see citing works
BixBench: a Comprehensive Benchmark for LLM-based Agents in Computational BiologyPreprint
Mitchener, L., Laurent, J. M., Andonian, A., Tenmann, B., Narayanan, S., Wellawatte, G. P., White, A., Sani, L., & Rodriques, S. G. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2503.00096 · Cited by 41 · see citing works
A benchmark of expert-level academic questions to assess AI capabilities
Center for AI Safety, Phan, L., Gatti, A., Li, N., Khoja, A., Kim, R., Ren, R., Hausenloy, J., Zhang, O., Mazeika, M., et al. (2026) · Nature · doi:10.1038/s41586-025-09962-4 · Cited by 23 · cited 2× · see citing works
“…evaluation efforts such as FrontierScience, FrontierMath, and Humanity’s Last Exam target difficult expert-level and novel-problem settings, 5–7 and METR’s recent time-horizon analyses likewise suggest that the duration of tasks frontier agents can complete autonomously is increasing rapidly.”
LAB-Bench: Measuring Capabilities of Language Models for Biology Research
Laurent, J. M., Janizek, J. D., Ruzo, M., Hinks, M. M., Hammerling, M. J., Narayanan, S., Ponnapati, M., White, A. D., & Rodriques, S. G. (2024) · arXiv · doi:10.48550/arxiv.2407.10362 · Cited by 21 · see citing works
Agentic Uncertainty QuantificationPreprint
Zhang, J.-X., Choubey, P. K., Huang, K.-H., Xiong, C.-M., & Wu, C.-S. (2026) · arXiv.org · doi:10.48550/arxiv.2601.15703 · Cited by 19 · see citing works
Measuring AI Ability to Complete Long Software Tasks
Kwa, T., West, B., Becker, J., Deng, A., Garcia, K., Hasin, M., Jawhar, S., Kinniment, M., Rush, N., Von Arx, S., et al. (2025) · arXiv · doi:10.48550/arxiv.2503.14499 · Cited by 14 · see citing works
Shared data science infrastructure for genomics data
Bagheri, H., Muppirala, U. K., Masonbrink, R. E., Severin, A. J., & Rajan, H. (2019) · BMC Bioinformatics · doi:10.1186/s12859-019-2967-2 · Cited by 13 · see citing works
Agentic systems are adept at solving well-scoped, verifiable problems in computational biologyPreprint
Nair, S., Gunsalus, L., Orcutt-Jahns, B. T., Rossen, J., Lal, A., de Donno, C., Çelik, M. H., Fletez-Brant, K., Xie, X.-M., Bravo, H. C., et al. (2026) · bioRxiv · doi:10.64898/2026.04.06.716850 · Cited by 13 · see citing works
ABC-Bench: An Agentic Bio-Capabilities Benchmark for BiosecurityPreprint
Liu, A. B., Nedungadi, S., Cai, B., Kleinman, A., Bhasin, H., & Donoughe, S. (2026) · arXiv.org · doi:10.48550/arxiv.2606.11150 · Cited by 10 · see citing works
Benchmarking AI scientists for omics data–driven biological discovery
Luo, E., Jia, J., Xiong, Y., Li, X., Guo, X., Yu, B., Hao, M., Wei, L., & Zhang, X. (2026) · Bioinformatics · doi:10.1093/bioinformatics/btag227 · Cited by 10 · see citing works
SpatialBench: Can Agents Analyze Real-World Spatial Biology Data?Preprint
Workman, K., Yang, Z., Muralidharan, H. S., & Le, H. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2512.21907 · Cited by 8 · see citing works
scBench: Evaluating AI Agents on Single-Cell RNA-seq AnalysisPreprint
Workman, K., Yang, Z., Muralidharan, H. S., Abdulali, A., & Le, H. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2602.09063 · Cited by 8 · see citing works
SWE-Lancer: Can Frontier LLMs Earn $1 Million from Real-World Freelance Software Engineering?
Miserendino, S., Wang, M., Patwardhan, T., & Heidecke, J. (2025) · arXiv · doi:10.48550/arxiv.2502.12115 · Cited by 5 · see citing works
SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?
Deng, X., Da, J., Pan, E., He, Y. Y., Ide, C., Garg, K., Lauffer, N., Park, A., Pasari, N., Rane, C., et al. (2025) · arXiv · doi:10.48550/arxiv.2509.16941 · Cited by 4 · cited 1× · see citing works
“…Pro, SWE-Lancer, DeepSWE, and FrontierSWE have shifted measurement toward longer-horizon, economically grounded, and deeper repository-level work; 1–4 broader evaluation efforts such as FrontierScience, FrontierMath, and Humanity’s Last Exam target difficult expert-level and novel-problem…”
LABBench2: An Improved Benchmark for AI Systems Performing Biology ResearchPreprint
Laurent, J. M., Bou, A., Pieler, M., Igoe, C., Andonian, A., Narayanan, S., Braza, J. D., Vassopoulos, A. S., Steenwyk, J. L., Lash, B., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2604.09554 · Cited by 4 · see citing works
Verifiable Benchmarking of Long-Horizon Spatial BiologyPreprint
Diks, I. F., Muralidharan, H. S., Proctor, T., & Workman, K. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.28065 · Cited by 3 · see citing works
FrontierScience: Evaluating AI's Ability to Perform Expert-Level Scientific TasksPreprint
Wang, M., Lin, R., Hu, K., Jiao, J., Chowdhury, N., Chang, E., & Patwardhan, T. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2601.21165 · Cited by 2 · see citing works
Open Source Planning & Control System with Language Agents for Autonomous Scientific DiscoveryPreprint
Xu, L., Sarkar, M., Lonappan, A. I., Zubeldía, Í., Villanueva-Domingo, P., Casas, S., Fidler, C., Amancharla, C., Tiwari, U., Bayer, A., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2507.07257 · Cited by 2 · see citing works
Benchmarking AI scientists for omics data driven biological discoveryPreprint
Luo, E., Jia, J., Xiong, Y., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2505.08341 · Cited by 1 · see citing works
REVERE: Reflective Evolving Research EngineerPreprint
Gangireddi, B. D., Garikaparthi, A., Patwardhan, M., & Cohan, A. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2603.20667 · Cited by 1 · see citing works
BioXArena: Benchmarking LLM Agents on Multi-Modal Biomedical Machine Learning TasksPreprint
Li, L., Zhang, D., Du, X., Song, L., Wang, Z., Aukenov, A., Thomas, N., Sailaukan, S., Yang, Y., Chen, F., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.15766 · Cited by 1 · see citing works
Frontier LLM-based agents can overcome the ontology curation bottleneck for natural phenotypesPreprint
Balhoff, J. P., & Lapp, H. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.28965
LifeSciBench: Evaluating Language Models on Realistic, Expert-Level Tasks in the Life SciencesPreprint
Liu, A., Ho, A., Droste, A. M., Martin, D., Wong, E., Zhou, E., Zhou, I., Park, J., Jiao, J., Skelly, K.-R., et al. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.08.13.744657
Self-citation
Flagged first, then by citations
Research works printed but not linked: 1
- Jeremy Li and Andrew Ho . GeneBench: Assessing AI Agents for Multi-Stage Inference Problems in Genomics and Quantitative Biology. bioRxiv, 2026.
Cited by
Sorted by citations
PG-LLM: Benchmarking General-Purpose Language Models for Protein Variant RankingPreprint
Arora, R. K., Chen, L. T., Du, M., Marks, D. S., & Church, G. M. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.07.27.741045 · Cited by 3 · see citing works
MetagenomicsBench: A Verifiable Benchmark for Agentic Metagenomics AnalysisPreprint
Yang, Z., Connolly, J., Banerjee, A., & Madad, S. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.10.05.756819
BioResearchBench: 50 Long Horizon Biological Research TasksPreprint
Abedrabbo, S., Pryhoda, A., Goodman, K., Taavola, J., Milam, E., Albright, A., Jabal, R., Yang, M., Burrese, C., Tubby, J., et al. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.10.28.762290
- Authors
- Li, J., Shringarpure, S., Wong, E., & Ho, A.
- Affiliations
- OpenAI
- OpenAI (United States) (US)
- Venue
- SSRN Electronic Journal
- Publisher
- openRxiv
- Published
- 2026-06-30
- ISSN
- 1556-5068 (ISSN-L)
- OpenAlex
- W7166736473
- Cited by
- 0
- References
- 37
- Type
- posted content
- Open access
- green
- doi.org
- Locations
- SSRN Electronic Journal (acceptedVersion)
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nc)
- Topics
- Social Sciences › Decision Sciences › Information Systems and Management › Scientific Computing and Data Management
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Genetics, Bioinformatics, and Biomedical Research
- Keywords
- quantitative biology, LLM evaluation, biological reasoning, long-horizon reasoning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
