Empirical assessment suggests that existing evidence could be used more fully in designing randomized controlled trials
Abstract
Background Meta-analyses of randomized controlled trials (RCTs) provide the highest level of evidence regarding the effectiveness of interventions. Less is known about how they are used to inform the design and reporting of RCTs.Methods A sample of RCTs published in leading medical journals in 2007 was assessed to establish whether authors considered previous trials in the design of their trial. An approach to calculate the sample size required for a significant pooled effect in an updated meta-analysis was applied to a subsample of the RCTs to illustrate the ways in which the results of an existing meta-analysis can be incorporated into the planning and reporting of new RCTs.Results Six of the 27 trials assessed (22%) reported the use of previous trial(s) for sample size calculations. Meta-analyses relating the results of the trial to previous research were cited in 37% (10 out of 27) of the report discussion sections. Previous evidence is formally incorporated into retrospective sample size calculations for three of the trials.Discussion/conclusion Consulting previous research before embarking on a new trial and basing decisions about future research on the impact on an updated meta-analysis will make the reporting of research more coherent and the design of new RCTs more efficient.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Evidence from systematic review can help researchers optimise usefulness of new trial by informing design aspects such as outcome definition and sample size.
“This evidence can also help researchers to optimise the usefulness of the new trial by informing design aspects such as outcome definition and sample size.143 144 145”
From SPIRIT 2025 explanation and elaboration: updated guideline for protocols of randomised trials (2025) · cites this paper as [145]
Cites
Evidence based medicine: what it is and what it isn't
Sackett, D. L., Rosenberg, W. M. C., Gray, J. A. M., Haynes, R. B., & Richardson, W. S. (1996) · BMJ · doi:10.1136/bmj.312.7023.71 · Cited by 15,769 · see citing works
Simultaneous comparison of multiple treatments: Combining direct and indirect evidence.
Caldwell, D. M., Ades, A. E., & Higgins, J. P. T. (2005) · BMJ · doi:10.1136/bmj.331.7521.897 · Cited by 1,600 · cited 1× · see citing works
“A mixed treatment comparison meta-analysis [14] would facilitate simultaneous comparisons among the four diet types and allow for the differences in treatments given in the trials combined in the meta-analysis and allow the inclusion of all four arms of the new trial.”
MethodSterne, J. A. C. (2001) · BMJ · doi:10.1136/bmj.322.7280.226 · Cited by 1,466 · see citing works
Uncertainty in heterogeneity estimates in meta-analyses
Ioannidis, J. P. A., Patsopoulos, N. A., & Evangelou, E. (2007) · BMJ · doi:10.1136/bmj.39343.408449.80 · Cited by 1,254 · see citing works
BORROWING STRENGTH FROM EXTERNAL TRIALS IN A META-ANALYSIS
Higgins, J. P. T., & WHITEHEAD, A. (1996) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19961230)15:24 · Cited by 553 · see citing works
Absence of evidence is not evidence of absence
Alderson, P. (2004) · BMJ · doi:10.1136/bmj.328.7438.476 · Cited by 253 · see citing works
Simmonds, M. C., & Higgins, J. P. T. (2006) · Statistics in Medicine · doi:10.1002/sim.2768 · Cited by 194 · see citing works
Putting clinical trials into context
Young, C., & Horton, R. (2005) · The Lancet · doi:10.1016/s0140-6736(05)66846-8 · Cited by 186 · see citing works
Discussion Sections in Reports of Controlled Trials Published in General Medical Journals
Clarke, M. J., & Chalmers, I. (1998) · JAMA · doi:10.1001/jama.280.3.280 · Cited by 167 · see citing works
Fergusson, D. A., Glass, K. C., Hutton, B., & Shapiro, S. (2005) · Clinical Trials · doi:10.1191/1740774505cn085oa · Cited by 165 · see citing works
Determining the sample size in a clinical trial
Kirby, A. C., Gebski, V., & KEECH, A. C. (2002) · The Medical Journal of Australia · doi:10.5694/j.1326-5377.2002.tb04759.x · Cited by 160 · see citing works
Multiparameter Evidence Synthesis in Epidemiology and Medical Decision-Making: Current Approaches
Ades, A. E., & Sutton, A. J. (2005) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.1111/j.1467-985x.2005.00377.x · Cited by 159 · see citing works
Self-citationEvidence‐based sample size calculations based upon updated meta‐analysis
Sutton, A. J., Cooper, N. J., Jones, D. R., Lambert, P. C., Thompson, J. R., & Abrams, K. R. (2006) · Statistics in Medicine · doi:10.1002/sim.2704 · Cited by 147 · see citing works
Self-citationClarke, M. J., Hopewell, S., & Chalmers, L. (2007) · Journal of the Royal Society of Medicine · doi:10.1177/014107680710011415 · Cited by 145 · see citing works
Discussion Sections in Reports of Controlled Trials Published in General Medical Journals
Clarke, M. J. (2002) · JAMA · doi:10.1001/jama.287.21.2799 · Cited by 123 · see citing works
Dose-Response to Exercise in Women Aged 45???75 yr (DREW): Design and Rationale
Morss, G. M., Jordan, A., Skinner, J. S., Dunn, A. L., Church, T. S., Earnest, C. P., Kampert, J. B., Jurča, R., & Blair, S. N. (2004) · Medicine & Science in Sports & Exercise · doi:10.1249/01.mss.0000113738.06267.e5 · Cited by 110 · see citing works
The use of systematic reviews when designing studies
Cooper, N. J., Jones, D. R., & Sutton, A. J. (2005) · Clinical Trials · doi:10.1191/1740774505cn090oa · Cited by 100 · see citing works
Self-citationEvidence synthesis as the key to more coherent and efficient research
Sutton, A. J., Cooper, N. J., & Jones, D. R. (2009) · BMC Medical Research Methodology · doi:10.1186/1471-2288-9-29 · Cited by 94 · see citing works
Self-citationClarke, M., Hopewell, S., & Chalmers, L. (2007) · Journal of the Royal Society of Medicine · doi:10.1258/jrsm.100.4.187 · Cited by 59 · see citing works
How useful are Cochrane reviews in identifying research needs?
Clarke, L., Clarke, M. J., & Clarke, T. T. (2007) · Journal of Health Services Research & Policy · doi:10.1258/135581907780279648 · Cited by 56 · see citing works
Are systematic reviews more cost-effective than randomised trials?
Glasziou, P., Djulbegovic, B., & Burls, A. (2006) · The Lancet · doi:10.1016/s0140-6736(06)68919-8 · Cited by 30 · see citing works
An encouraging assessment of methods to inform priorities for updating systematic reviews
Sutton, A. J., Donegan, S., Takwoingi, Y., Garner, P., Gamble, C. L., & Donald, A. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2008.04.005 · Cited by 29 · see citing works
Self-citationChalmers, I., Augoustides, J. G., & Fleisher, L. A. (2005) · Clinical Trials · Cited by 26 · see citing works
Detecting and describing heterogeneity in meta-analysis
Hardy, R. J., & Thompson, S. G. (1998) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19980430)17:8
Flagged first, then by citations
Cited by
Sorted by citations
SPIRIT 2013 explanation and elaboration: guidance for protocols of clinical trials
Chan, A.-W., Tetzlaff, J. M., Gotzsche, P. C., Altman, D. G., Mann, H., Berlin, J. A., Dickersin, K., Hrobjartsson, A., Schulz, K. F., Parulekar, W. R., et al. (2013) · BMJ · doi:10.1136/bmj.e7586 · Cited by 6,251 · see citing works
How to increase value and reduce waste when research priorities are set
Chalmers, I., Bracken, M. B., Djulbegovic, B., Garattini, S., Grant, J., Gülmezoglu, A. M., Howells, D. W., Ioannidis, J. P. A., & Oliver, S. (2014) · The Lancet · doi:10.1016/s0140-6736(13)62229-1 · Cited by 1,436 · see citing works
Ivers, N. M., Grimshaw, J., Jamtvedt, G., Flottorp, S. A., O’Brien, M. A., French, S. D., Young, J. M., & Odgaard‐Jensen, J. (2014) · Journal of General Internal Medicine · doi:10.1007/s11606-014-2913-y · Cited by 378 · see citing works
Cook, J., Hislop, J., Adewuyi, T., Harrild, K., Altman, D., Ramsay, C., Fraser, C., Buckley, B., Fayers, P., Harvey, I., et al. (2014) · Health Technology Assessment · doi:10.3310/hta18280 · Cited by 194 · see citing works
SPIRIT 2025 explanation and elaboration: updated guideline for protocols of randomised trials
Hróbjartsson, A., Boutron, I., Hopewell, S., Moher, D., Schulz, K. F., Collins, G. S., Tunn, R., Aggarwal, R., Berkwits, M., Berlin, J. A., et al. (2025) · BMJ · doi:10.1136/bmj-2024-081660 · Cited by 171 · see citing works
Evidence-Based Research Series-Paper 1: What Evidence-Based Research is and why is it important?
Robinson, K. A., Brunnhuber, K., Ciliska, D., Juhl, C. B., Christensen, R., & Lund, H. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2020.07.020 · Cited by 146 · see citing works
Nikolakopoulou, A., Mavridis, D., Furukawa, T. A., Cipriani, A., Tricco, A. C., Straus, S. E., Siontis, G. C. M., Egger, M., & Salanti, G. (2018) · BMJ · doi:10.1136/bmj.k585 · Cited by 106 · see citing works
Raftery, J. P., Hanney, S. R., Greenhalgh, T., Glover, M. J., & Blatch-Jones, A. (2016) · Health Technology Assessment · doi:10.3310/hta20760 · Cited by 105 · see citing works
Lund, H., Juhl, C. B., Nørgaard, B., Draborg, E., Henriksen, M., Andreasen, J., Christensen, R., Nasser, M., Ciliska, D. K., Clarke, M. J., et al. (2020) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2020.07.019 · Cited by 98 · see citing works
Langan, D., Higgins, J. P. T., Gregory, W. M., & Sutton, A. J. (2012) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.10.009 · Cited by 89 · see citing works
Self-citationEfficacy of psychosocial interventions for Autism spectrum disorder: an umbrella review
Gosling, C. J., Cartigny, A., Mellier, B. C., Solanes, A., Raduà, J., & Delorme, R. (2022) · Molecular Psychiatry · doi:10.1038/s41380-022-01670-z · Cited by 88 · see citing works
Peinemann, F., & Labeit, A. M. (2018) · Journal of Evidence-Based Medicine · doi:10.1111/jebm.12324 · Cited by 76 · see citing works
Trial sequential methods for meta‐analysis
Kulinskaya, E., & Wood, J. (2013) · Research Synthesis Methods · doi:10.1002/jrsm.1104 · Cited by 73 · see citing works
Jones, A. P., Conroy, E., Williamson, P. R., Clarke, M., & Gamble, C. (2013) · BMC Medical Research Methodology · doi:10.1186/1471-2288-13-50 · Cited by 72 · see citing works
Naci, H., & Ioannidis, J. P. A. (2014) · The Annual Review of Pharmacology and Toxicology · doi:10.1146/annurev-pharmtox-010814-124614 · Cited by 68 · see citing works
An increasing problem in publication ethics: Publication bias and editors’ role in avoiding it
Ekmekçi, P. E. (2017) · Medicine Health Care and Philosophy · doi:10.1007/s11019-017-9767-0 · Cited by 61 · see citing works
Using the bayesmeta R package for Bayesian random-effects meta-regression
Röver, C., & Friede, T. (2022) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2022.107303 · Cited by 59 · see citing works
Planning a future randomized clinical trial based on a network of relevant past trials
Salanti, G., Nikolakopoulou, A., Sutton, A. J., Reichenbach, S., Trelle, S., Naci, H., & Egger, M. (2018) · Trials · doi:10.1186/s13063-018-2740-2 · Cited by 53 · see citing works
Self-citationRedundant trials can be prevented, if the EU clinical trial regulation is applied duly
Kim, D., & Hasford, J. (2020) · BMC Medical Ethics · doi:10.1186/s12910-020-00536-9 · Cited by 30 · see citing works
Combining Statistical Evidence
Kulinskaya, E., Morgenthaler, S., & Staudte, R. G. (2014) · International Statistical Review · doi:10.1111/insr.12037 · Cited by 30 · see citing works
Systematic reviews are rarely used to inform study design - a systematic review and meta-analysis
Nørgaard, B., Draborg, E., Andreasen, J., Juhl, C. B., Yost, J., Brunnhuber, K., Robinson, K. A., & Lund, H. (2022) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2022.01.007 · Cited by 29 · see citing works
A new network to promote evidence-based research
Chalmers, I., & Nylenna, M. (2014) · The Lancet · doi:10.1016/s0140-6736(14)62252-2 · Cited by 24 · see citing works
Improving the effectiveness of clinical medicine: the need for better science
Scott, I. A., & Glasziou, P. (2012) · The Medical Journal of Australia · doi:10.5694/mja11.10364 · Cited by 24 · see citing works
Using systematic reviews to inform NIHR HTA trial planning and design: a retrospective cohort
Bhurke, S., Cook, A., Tallant, A., Young, A. J., Williams, E., & Raftery, J. P. (2015) · BMC Medical Research Methodology · doi:10.1186/s12874-015-0102-2 · Cited by 24 · see citing works
McLennan, S., Nußbaumer-Streit, B., Hemkens, L. G., & Briel, M. (2021) · JAMA Network Open · doi:10.1001/jamanetworkopen.2021.36577 · Cited by 21 · see citing works
Andreasen, J., Nørgaard, B., Draborg, E., Juhl, C. B., Yost, J., Brunnhuber, K., Robinson, K. A., & Lund, H. (2022) · PLoS ONE · doi:10.1371/journal.pone.0276955 · Cited by 19 · see citing works
Use of a random effects meta‐analysis in the design and analysis of a new clinical trial
Jones, H. E., Ades, A. E., Sutton, A. J., & Welton, N. J. (2018) · Statistics in Medicine · doi:10.1002/sim.7948 · Cited by 16 · see citing works
Self-citationSequential biases in accumulating evidence
Kulinskaya, E., Huggins, R., & Dogo, S. H. (2015) · Research Synthesis Methods · doi:10.1002/jrsm.1185 · Cited by 18 · see citing works
Draborg, E., Andreasen, J., Nørgaard, B., Juhl, C. B., Yost, J., Brunnhuber, K., Robinson, K. A., & Lund, H. (2022) · Systematic Reviews · doi:10.1186/s13643-022-02062-8 · Cited by 17 · see citing works
Citation behavior: A large‐scale test of the persuasion by name‐dropping hypothesis
Frandsen, T. F., & Nicolaisen, J. (2016) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23746 · Cited by 15 · see citing works
Crowther, M. J., Hinchliffe, S. R., Donald, A., & Sutton, A. J. (2013) · The Stata Journal Promoting communications on statistics and Stata · doi:10.1177/1536867x1301300302 · Cited by 12 · see citing works
Self-citationALL-IN meta-analysis: breathing life into living systematic reviews
ter Schure, J., & Grünwald, P. (2022) · F1000Research · doi:10.12688/f1000research.74223.1 · Cited by 12 · see citing works
Nørgaard, B., Briel, M., Chrysostomou, S., Ristić-Medić, D., Buttiġieġ, S. C., Kiisk, E., Puljak, L., Bała, M. M., Peričić, T. P., Leśniak, W., et al. (2022) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2022.06.021 · Cited by 11 · see citing works
Cook, J. A., Hislop, J., Altman, D. G., Briggs, A. H., Fayers, P. M., Norrie, J. D., Ramsay, C. R., Harvey, I. M., Vale, L. D., for the DELTA group, et al. (2014) · Clinical Trials · doi:10.1177/1740774514521907 · Cited by 10 · see citing works
The pathway to RCTs: how many roads are there? Examining the homogeneity of RCT justification
Chow, J., Lam, K. F. Y., Naeem, A. H., Akanda, Z. Z., Si, F. F., & Hodge, W. (2017) · Trials · doi:10.1186/s13063-017-1804-z · Cited by 10 · see citing works
Biau, D. J., Boulezaz, S., Casabianca, L., Hamadouche, M., Anract, P., & Chevret, S. (2017) · BMC Medical Research Methodology · doi:10.1186/s12874-017-0401-x · Cited by 9 · see citing works
Learning from the past: refining the way we study treatments
Viechtbauer, W. (2010) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.04.004 · Cited by 8 · see citing works
Completeness of clinical evidence citation in trial protocols: A cross-sectional analysis
Sheng, J., Feldhake, E., Zarin, D. A., & Kimmelman, J. (2022) · Med · doi:10.1016/j.medj.2022.03.002 · Cited by 7 · see citing works
Gamble, C. L., WOLF, A. R., Sinha, I. P., Spowart, C., & Williamson, P. R. (2013) · PLoS ONE · doi:10.1371/journal.pone.0051787 · Cited by 7 · see citing works
Satheeshkumar, P. S., & Sonis, S. T. (2021) · Clinical Oral Investigations · doi:10.1007/s00784-021-03802-1 · Cited by 7 · see citing works
A straightforward meta‐analysis approach for oncology phase I dose‐finding studies
Röver, C., Ursino, M., Friede, T., & Zohar, S. (2022) · Statistics in Medicine · doi:10.1002/sim.9484 · Cited by 5 · see citing works
Pędziwiatr, M., Mizera, M., Wysocki, M., Małczak, P., Stefura, T., Gajewska, N., Torbicz, G., Droś, J., Kacprzyk, A., Major, P., et al. (2019) · Clinical Nutrition · doi:10.1016/j.clnu.2019.05.024 · Cited by 5 · see citing works
Identifying prior evidence for new trials (REVEAL): guidance for clinical researchers
Griebler, U., Ledinger, D., Klerings, I., Schandelmaier, S., Dobrescu, A. I., Deschodt, M., McLennan, S., Hemkens, L. G., Nußbaumer-Streit, B., & Briel, M. (2026) · BMJ · doi:10.1136/bmj-2024-083718 · Cited by 4 · see citing works
Clayton, G., Elliott, D., Higgins, J. P. T., & Jones, H. E. (2021) · Trials · doi:10.1186/s13063-021-05759-8 · Cited by 4 · see citing works
Eerdenbrugh, S. V., Pingani, L., Prevendar, T., Lantta, T., Zając, J., Prokop‐Dorner, A., Brandão, M. P., Peričić, T. P., van Hoof, J., Lund, H., et al. (2024) · BMJ Open · doi:10.1136/bmjopen-2023-083676 · Cited by 3 · see citing works
Lund, H. (2014) · European Journal of Physiotherapy · doi:10.3109/21679169.2014.917838 · Cited by 2 · see citing works
ALL-IN meta-analysis: breathing life into living systematic reviews and prospective meta-analyses
ter Schure, J., & Grünwald, P. (2025) · F1000Research · doi:10.12688/f1000research.74223.2 · Cited by 2 · see citing works
Estimating the sample size of sham-controlled randomized controlled trials using existing evidence
Siontis, G. C. M., Nikolakopoulou, A., Sweda, R., Mavridis, D., & Salanti, G. (2022) · F1000Research · doi:10.12688/f1000research.108554.1 · Cited by 2 · see citing works
Clinical Trial Design in Prevention and Treatment of Acute Respiratory Distress Syndrome
Curley, G. F., & McAuley, D. F. (2014) · Clinics in Chest Medicine · doi:10.1016/j.ccm.2014.08.009 · Cited by 2 · see citing works
What discontinued trials teach us about trial registration?
Vellinga, A., Lambe, K. A., O'Connor, P. E., & O’Dea, A. (2021) · BMC Research Notes · doi:10.1186/s13104-020-05391-w · Cited by 2 · see citing works
Egesa, I. J., Bonnett, L. J., Emsley, R., Marson, A. G., & Smith, C. T. (2025) · Clinical Trials · doi:10.1177/17407745251385535 · Cited by 1 · see citing works
IPD as a Research Resource: Exclusively Controlled or Readily Accessible?
Kim, D. (2021) · Munich studies on innovation and competition · doi:10.1007/978-3-030-86778-2_8 · Cited by 1 · see citing works
Incorporating external evidence syntheses in the design and analysis of trials
Clayton, G. L. (2019) · Bristol Research (University of Bristol)
Making the most of a few small population clinical trials
Pateras, K. (2020) · doi:10.33540/53
A simple meta-analysis approach for exploratory phase I dose-finding studies.Preprint
Röver, C., Ursino, M., Friede, T., & Zohar, S. (2021) · arXiv (Cornell University)
Aschmann, H. E., McNeil, J. J., & Puhan, M. A. (2022) · Clinical Trials · doi:10.1177/17407745211068548
Estimating the sample size of sham-controlled randomized controlled trials using existing evidence
Siontis, G. C. M., Nikolakopoulou, A., Sweda, R., Mavridis, D., & Salanti, G. (2022) · F1000Research · doi:10.12688/f1000research.108554.2
Draborg, E., Andreasen, J., Nørgaard, B., Juhl, C. B., Yost, J., Brunnhuber, K., Robinson, K. A., & Lund, H. (2022) · Figshare · doi:10.6084/m9.figshare.20962291
Draborg, E., Andreasen, J., Nørgaard, B., Juhl, C. B., Yost, J., Brunnhuber, K., Robinson, K. A., & Lund, H. (2022) · Figshare · doi:10.6084/m9.figshare.20962288
Draborg, E., Andreasen, J., Nørgaard, B., Juhl, C. B., Yost, J., Brunnhuber, K., Robinson, K. A., & Lund, H. (2022) · Figshare · doi:10.6084/m9.figshare.20962282
Draborg, E., Andreasen, J., Nørgaard, B., Juhl, C. B., Yost, J., Brunnhuber, K., Robinson, K. A., & Lund, H. (2022) · Figshare · doi:10.6084/m9.figshare.20962297
Clayton, G. L., Elliott, D., Higgins, J. P. T., & Jones, H. E. (2021) · Figshare · doi:10.6084/m9.figshare.16960411
Draborg, E., Andreasen, J., Nørgaard, B., Juhl, C. B., Yost, J., Brunnhuber, K., Robinson, K. A., & Lund, H. (2022) · Figshare · doi:10.6084/m9.figshare.20962285
- Authors
- Goudie, A. C., Sutton, A. J., Jones, D. R., & Donald, A.
- Affiliations
- University of Leicester (GB)
- ORCID
- Jones, David R.
- Venue
- Journal of Clinical Epidemiology
- Publisher
- Elsevier BV
- Published
- 2010-09
- Volume
- 63
- Issue
- 9
- Pages
- 983-991
- ISSN
- 0895-4356 (ISSN-L)
- OpenAlex
- W2171634280
- Cited by
- 63
- References
- 24
- Citations by year
- 2026: 1 · 2025: 3 · 2024: 1 · 2022: 17 · 2021: 7 · 2020: 4 · 2019: 2 · 2018: 4 · 2017: 3 · 2016: 2 · 2015: 2 · 2014: 9 · 2013: 5 · 2012: 2
- FWCI
- 2.66
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of Clinical Epidemiology (publishedVersion)
- PubMed (publishedVersion)
- Leicester Research Archive (University of Leicester) (submittedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Keywords
- randomized controlled trials, meta-analysis, trial design, sample size determination, trial reporting, research planning, evidence synthesis
- MeSH terms
- Humans, Medical Informatics Applications, Research Design, Meta-Analysis as Topic, Randomized Controlled Trials as Topic, Sample Size, Evidence-Based Medicine
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
