Complexity of MRI White Matter Hyperintensity Assessments in Relation to Cognition in Aging and Dementia from the Sunnybrook Dementia Study
The DOI matches its Crossref record; 35 of the 36 references it cites were found in a registry or an index.
Abstract
PURPOSE: Quantification methods for white matter hyperintensities (WMH) on Magnetic Resonance Imaging are heterogeneous, deterring their application. This study compared three WMH rating scales, varying in complexity, and a volumetric method, to evaluate trade-offs between complexity and clinical utility in differentiating dementia subgroups and in correlating with cognition. METHODS: WMH were rated using the Fazekas, Age-Related White Matter Changes (ARWMC) and Scheltens scales, and segmented by computational volumetry in 108 patients with Alzheimer's Disease (AD), 23 with Mild Cognitive Impairment (MCI) and 34 normal controls (NC). Global and hippocampal atrophy, age and education, were accounted for in correlations of WMH with cognitive domains. RESULTS: Intra- and inter-rater reliability were high (intraclass correlation coefficients = 0.88-0.97) across rating scales. WMH scores of all scales were highly correlated with volumes (Spearman r = 0.78-0.90, Ps < 0.001), as well as with each other (Spearman r = 0.86-0.91, Ps < 0.001). The Fazekas scale showed significant separation between AD, MCI and NC using non-parametric analysis, while the ARWMC and Scheltens' scales, and WMH volumes demonstrated significant correlations (standardized β = -0.19 to -0.24, Ps < 0.05) with cognitive domain scores using multivariate regression analysis, controlling for age, education, global and hippocampal atrophy in patients with AD. CONCLUSIONS: This study suggests that the degree of complexity of WMH rating scales did not affect validation against WMH volumes, but did vary in validation against cognition. The simplest scale performed best in separating cognitive subgroups, but the more complex scales and quantification correlated better with cognitive measures, especially executive function. Hence the best choice of scale depends on the particular application.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“For some of the lesions, qualitative and quantitative approaches are closely related,82,83 and both have advantages and disadvantages that should be considered in the design of any study of SVD (appendix).”
From Neuroimaging standards for research into small vessel disease and its contribution to ageing and neurodegeneration (2013) · cites this paper as [83]
Citations
Cites
Diagnostic and Statistical Manual of Mental Disorders
Williams, J. B. W., & First, M. (2013) · Encyclopedia of Social Work · doi:10.1093/acrefore/9780199975839.013.104 · Cited by 109,519
Folstein, M. F., Folstein, S. E., & McHugh, P. R. (1975) · Journal of Psychiatric Research · doi:10.1016/0022-3956(75)90026-6 · Cited by 88,282 · cited 1×
“Mini-Mental State Examination (MMSE) [18] and the Mattis Dementia Rating Scale (DRS) total score [19] were used to measure global cognitive function.”
MethodMcKhann, G., Drachman, D., Folstein, M., Katzman, R., Price, D., & Stadlan, E. M. (1984) · Neurology · doi:10.1212/wnl.34.7.939 · Cited by 27,915
Mild Cognitive Impairment: Clinical Characterization and Outcome
Petersen, R. C., Smith, G. E., Waring, S. C., Ivnik, R. J., Tangalos, E. G., & Kokmen, E. (1999) · Archives of Neurology · doi:10.1001/archneur.56.3.303 · Cited by 8,980
MR signal abnormalities at 1.5 T in Alzheimer's dementia and normal aging
Fazekas, F., Chawluk, J., Alavi, A., Hurtig, H., & Zimmerman, R. (1987) · American Journal of Roentgenology · doi:10.2214/ajr.149.2.351 · Cited by 5,334 · cited 1×
“[24] B) Age-related White Matter Change (ARWMC) [25] and C) Scheltens et al.”
BackgroundThe Alzheimer’s disease neuroimaging initiative (ADNI): MRI methods
Jack, C. R., Bernstein, M. A., Fox, N. C., Thompson, P., Alexander, G., Harvey, D., Borowski, B., Britson, P. J., L. Whitwell, J., Ward, C., et al. (2008) · Journal of Magnetic Resonance Imaging · doi:10.1002/jmri.21049 · Cited by 4,347 · cited 1×
“The advantage of the rating scales is that they can be performed by trained observers on scans obtained clinically, as long as they include a T2 weighted or FLAIR sequence, using standardized protocols, preferably compliant with the Vascular Cognitive Impairment Harmonization Criteria [35], and exemplified in the prot…”
MethodA New Rating Scale for Age-Related White Matter Changes Applicable to MRI and CT
Wahlund, L. O., Barkhof, F., Fazekas, F., Bronge, L., Augustin, M., Sjögren, M., Wallin, A., Ader, H., Leys, D., Pantoni, L., et al. (2001) · Stroke · doi:10.1161/01.str.32.6.1318 · Cited by 1,797 · cited 1×
“[24] B) Age-related White Matter Change (ARWMC) [25] and C) Scheltens et al.”
BackgroundSelf-citationHachinski, V., Iadecola, C., Petersen, R. C., Breteler, M. M., Nyenhuis, D. L., Black, S. E., Powers, W. J., DeCarli, C., Merino, J. G., Kalaria, R. N., et al. (2006) · Stroke · doi:10.1161/01.str.0000237236.88823.47 · Cited by 1,722 · cited 1×
“The advantage of the rating scales is that they can be performed by trained observers on scans obtained clinically, as long as they include a T2 weighted or FLAIR sequence, using standardized protocols, preferably compliant with the Vascular Cognitive Impairment Harmonization Criteria [35], and exemplified in the prot…”
MethodSelf-citationde Leeuw, F.-E. (2001) · Journal of Neurology, Neurosurgery & Psychiatry · doi:10.1136/jnnp.70.1.9 · Cited by 1,291
Scheltens, P., Barkhof, F., Leys, D., Pruvo, J. P., Nauta, J. J. P., Vermersch, P., Steinling, M., & Valk, J. (1993) · Journal of the Neurological Sciences · doi:10.1016/0022-510x(93)90041-v · Cited by 923 · cited 1×
“[26] on axial T2/PD-weighted MR images.”
Self-citation
Cited by
Wardlaw, J. M., Smith, E. E., Biessels, G. J., Cordonnier, C., Fazekas, F., Frayne, R., Lindley, R. I., O'Brien, J. T., Barkhof, F., Benavente, O. R., et al. (2013) · The Lancet Neurology · doi:10.1016/s1474-4422(13)70124-8 · Cited by 5,544
Self-citationRamirez, J., Berezuk, C., McNeely, A. A., Scott, C. J. M., Gao, F., & Black, S. E. (2014) · Journal of Alzheimer's Disease · doi:10.3233/jad-132528 · Cited by 177
Self-citationRamirez, J., Berezuk, C., McNeely, A. A., Gao, F., McLaurin, J., & Black, S. E. (2016) · Cellular and Molecular Neurobiology · doi:10.1007/s10571-016-0343-6 · Cited by 161
Self-citationCaligiuri, M. E., Perrotta, P., Augimeri, A., Rocca, F., Quattrone, A., & Cherubini, A. (2015) · Neuroinformatics · doi:10.1007/s12021-015-9260-y · Cited by 151
“2006) and significant ceiling/floor effects (van Straaten et al. 2006; Gao et al. 2011), therefore leading to inconsistencies between WMH studies.”
Backgroundvan den Berg, E., Geerlings, M. I., Biessels, G. J., Nederkoorn, P. J., & Kloppenborg, R. P. (2018) · Journal of Alzheimer’s Disease · doi:10.3233/jad-170573 · Cited by 132
Kynast, J., Lampe, L., Luck, T., Frisch, S., Arélin, K., Hoffmann, K., Loeffler, M., Riedel-Heller, S. G., Villringer, A., & Schroeter, M. L. (2017) · Journal of Cerebral Blood Flow & Metabolism · doi:10.1177/0271678x17719380 · Cited by 121
“Compared to WMH volume, ceiling effects of the Fazekas scale may limit definite conclusions about the association of WMH and cognition.(47) The results of the actual study shall be verified with volumetric measures.”
Severity of white matter hyperintensities: Lesion patterns, cognition, and microstructural changes
Zeng, W., Chen, Y., Zhu, Z., Gao, S., Xia, J., Chen, X., Jia, J., & Zhang, Z. (2019) · Journal of Cerebral Blood Flow & Metabolism · doi:10.1177/0271678x19893600 · Cited by 101
“High correlation between Fazekas scale scores and volumetric WML assessment has previously been demonstrated in the Leukoaraiosis and Disability study (LADIS) and the Sunnybrook Dementia Study.(15,16) Therefore, we adopted both Fazekas scale and WMH volume as criteria for evaluating the severity of white matter hyperi…”
MethodBoespflug, E. L., Schwartz, D. L., Lahna, D. L., Pollock, J. M., Iliff, J. J., Kaye, J. A., Rooney, W. D., & Silbert, L. C. (2017) · Radiology · doi:10.1148/radiol.2017170205 · Cited by 79
Predicting Fazekas scores from automatic segmentations of white matter signal abnormalities
Cedrés, N., Ferreira, D., Machado, A., Shams, S., Sacuiu, S. F., Wærn, M., Wahlund, L., Zettergren, A., Kern, S., Skoog, I. G., et al. (2020) · Aging · doi:10.18632/aging.102662 · Cited by 73
“However, inter- and intra-rater reliability is usually higher for the Fazekas scale than for other visual rating scales [18].”
BackgroundDefrancesco, M., Marksteiner, J., Deisenhammer, E., Kemmler, G., Djurdjevic, T., & Schocke, M. (2013) · Journal of Alzheimer's Disease · doi:10.3233/jad-122095 · Cited by 61
“WML are usually measured by visual rating scales such as the Fazekas scale and the Scheltens scale in healthy aging, MCI, and AD [4].”
BackgroundMethodSchwartz, D. L., Boespflug, E. L., Lahna, D. L., Pollock, J. M., Roese, N. E., & Silbert, L. C. (2019) · NeuroImage · doi:10.1016/j.neuroimage.2019.116126 · Cited by 57
“Acquisition time of all four sequences may be prohibitive in certain clinical or research settings and the requirement of four separate sequences may preclude the use of the multimodal algorithm in some legacy longitudinal datasets, which are of particular interest in the context of neurodegenerative disease and norma…”
BackgroundCoutu, J., (ADNI), T. A. D. N. I., Goldblatt, A. E. A., Rosas, H. D., & Salat, D. H. (2015) · Journal of Alzheimer s Disease · doi:10.3233/jad-150306 · Cited by 57
Olsson, E., Klasson, N., Berge, J., Eckerström, C., Edman, Å., Malmgren, H., & Wallin, A. (2013) · Journal of Aging Research · doi:10.1155/2013/198471 · Cited by 53
“Manual volumetrical assessments have shown higher reliability than visual rating scales [11, 15] and would be valuable in clinical settings if made less labour intensive.”
BackgroundMethodSong, X., Mitnitski, A., Zhang, N., Chen, W., & Rockwood, K. (2013) · Journal of Neurology, Neurosurgery & Psychiatry · doi:10.1136/jnnp-2012-303579 · Cited by 40
White matter hyperintensities, cerebrospinal amyloid-β and dementia in Parkinson's disease
Compta, Y., Buongiorno, M., Bargalló, N., Valldeoriola, F., Muñoz, E., TOLOSA, E. S., Ríos, J., Cámara, A., Fernández, M., & Martí, M. J. (2016) · Journal of the Neurological Sciences · doi:10.1016/j.jns.2016.06.009 · Cited by 37
Topographical differences in white matter hyperintensity burden and cognition in aging, MCI, and AD
Kamal, F., Morrison, C., Maranzano, J., Zeighami, Y., Dadar, M., & Initiative, A. D. N. (2022) · GeroScience · doi:10.1007/s11357-022-00665-6 · Cited by 33
“However, the results of these studies have been inconsistent and most examine total instead of regional WMH burden [12, 17, 19–21, 23, 24].”
The effect of age on cognitive performance of frontal patients
Cipolotti, L., Healy, C., Chan, E., MacPherson, S. E., White, M. J., Woollett, K., Turner, M., Robinson, G. A., Spanò, B., Bozzali, M., et al. (2015) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2015.06.011 · Cited by 34
“Our results also demonstrated that age significantly predicted the degree of global brain atrophy and WMA, a finding consistent with previous studies in the literature (e.g., Raz et al., 2005; Elderkin-Thompson et al., 2008; Gao et al., 2011).”
ResultHemodynamic and structural brain measures in high and low sedentary older adults
Maasakkers, C. M., Thijssen, D. H. J., Knight, S. P., Newman, L., O’Connor, J. D., Scarlett, S., Carey, D., Buckley, A., McMorrow, J. P., Leidhin, C. N., et al. (2021) · Journal of Cerebral Blood Flow & Metabolism · doi:10.1177/0271678x211009382 · Cited by 29
“The Scheltens visual rating scale is a semi-quantitative method to calculate the extent of white matter disease that is well validated and commonly used.(23) Participants under 65 years of age were excluded from this scoring system.”
MethodHasan, T. F., Barrett, K. M., Brott, T. G., Badi, M. K., Lesser, E. R., Hodge, D. O., & Meschia, J. F. (2019) · Mayo Clinic Proceedings · doi:10.1016/j.mayocp.2018.10.024 · Cited by 30
“inappropriate in assessing longitudinal progression of WMH,(36) less sensitive to clinical differences,(37) and to have high intrarater and interrater variability(38) and ceiling/floor effects.(35,39)”
BackgroundPolyakova, M., Mueller, K., Arélin, K., Lampe, L., Rodriguez, F. S., Luck, T., Kratzsch, J., Hoffmann, K., Riedel-Heller, S. G., Villringer, A., et al. (2022) · Frontiers in Cellular Neuroscience · doi:10.3389/fncel.2022.788150 · Cited by 29
“However, literature shows a high correlation between Fazekas scores and volumetric assessment of WMH (Gao et al., 2011).”
BackgroundOgama, N., Sakurai, T., Nakai, T., Niida, S., Saji, N., Toba, K., Umegaki, H., & Kuzuya, M. (2017) · PLoS ONE · doi:10.1371/journal.pone.0172484 · Cited by 27
Zhang, Z., Ding, Z., Chen, F., Hua, R., Wu, J., Shen, Z., Shi, F., & Xu, X. (2024) · International Journal of General Medicine · doi:10.2147/ijgm.s446531 · Cited by 25
Zhu, H., Lu, H., Wang, F., Liu, S., Shi, Z., Gan, J., Du, X., Yang, Y., Li, D., Wang, L., et al. (2022) · Frontiers in Neurology · doi:10.3389/fneur.2021.779344 · Cited by 23
“Most studies believed that the relationship between the Fazekas scale and the WMH automatic measurement was consistent, and the Fazekas scale showed significant differences in distinguishing ADpatients from others (46, 47).”
ResultAshford, J. W., Salehi, A., Furst, A. J., Bayley, P. J., Frisoni, G. B., Jack, C. R., Sabri, O., Adamson, M. M., Coburn, K. L., Olichney, J., et al. (2011) · Journal of Alzheimer s Disease · doi:10.3233/jad-2011-0073 · Cited by 23
Self-citationMcNeely, A. A., Ramirez, J., Nestor, S. M., Zhao, J., Gao, F., Kiss, A., Stuss, D. T., & Black, S. E. (2014) · Journal of Alzheimer s Disease · doi:10.3233/jad-140588 · Cited by 25
“Additionally, various studies have demonstrated SVD to be associated with cognitive dysfunction and decline [11–16], stroke [17], and gait disturbances [18, 19].”
Self-citationZhang, N., Song, X., & Zhang, Y. (2012) · Dementia and Geriatric Cognitive Disorders · doi:10.1159/000339364 · Cited by 20
Zhang, L., Sun, W., Xing, M., Wang, Y., Zhang, Y., Sun, Q., Cheng, Y., Shi, C., & Zhang, N. (2019) · Journal of the International Neuropsychological Society · doi:10.1017/s1355617719000353 · Cited by 18
“…visual rating scales are easy to perform in clinical practice, the reliability of the MTA (interrater: 0.72–0.84, intrarater: 0.83–0.94, weighted Cohen’s κ) and Fazekas scales (interrater: 0.70–0.72, intrarater: 0.79–0.82) remains modest (Gao et al., 2011; Harper, Barkhof, Fox, & Schott, 2015).”
BackgroundInfluentialWhite Matter Hyperintensities in Mild Cognitive Impairment and Lower Risk of Cognitive Decline
Nolze-Charron, G., Mouiha, A., Duchesne, S., Bocti, C., & Initiative, A. D. N. (2015) · Journal of Alzheimer s Disease · doi:10.3233/jad-140618 · Cited by 16
Lam, C. L. M., Yiend, J., & Lee, T. M. C. (2017) · Neurorehabilitation · doi:10.3233/nre-171471 · Cited by 17
“Their results suggested that all three rating scales have good face validity and correlate well with the WML volumetry, but the more complex rating scales correlate better with cognitive measures (Gao et al., 2011).”
Reuck, J. D., Auger, F., Durieux, N., Cordonnier, C., Deramecourt, V., Pasquier, F., Maurage, C., Leys, D., & Bordet, R. (2016) · European Stroke Journal · doi:10.1177/2396987316650780 · Cited by 13
“The quantification methods for in-vivo used MRI are heterogeneous and vary in validation against cognition.(33,34) The progression of cognitive decline is mainly attributed to the WMCs.”
MethodRating total cerebral small-vessel disease
Jickling, G. C., & Chen, C. (2014) · Neurology · doi:10.1212/wnl.0000000000000843 · Cited by 10
Yue, J., Zhang, Q., Xu, Y., Wang, P., Sun, X.-C., Yan, S., Li, A., Zhao, W.-W., Cao, D., Wang, Y., et al. (2023) · Frontiers in Neuroscience · doi:10.3389/fnins.2023.1163809 · Cited by 10
“, 2013); the quantitative method of highWM intensity on MRI (Gao et al., 2011); and theWM change scale (Wahlund et al.”
MethodHu, Y., Yang, Y., Hou, X., Zhou, Y., & Nie, S. (2022) · Journal of X-Ray Science and Technology · doi:10.3233/xst-221218 · Cited by 8
“Some research has confirmed the high correlation between Fazekas score and WMH volume previously [15, 16].”
Mossa‐Basha, M., Andre, J. B., Yuh, E. L., Hunt, D., LaPiana, N., Howlett, B., Krakauer, C. A., Crane, P. K., Nelson, J. C., Dezelar, M., et al. (2024) · British Journal of Radiology · doi:10.1093/bjr/tqae004 · Cited by 6
Isolated, Subtle Neurological Abnormalities in Mild Cognitive Impairment Types
Camarda, C., Camarda, R., Pipia, C., Azzarello, D. M., Grassedonio, E., Sottile, G., Cilluffo, G., & Torelli, P. (2019) · Canadian Journal of Neurological Sciences / Journal Canadien des Sciences Neurologiques · doi:10.1017/cjn.2019.293 · Cited by 6
“However, it has been shown that WMH evaluated with visual scales correlates well with WMH volumetry.(62) Fourth, perhaps, we underestimated the magnitude of cerebral SVD in our cohort because we evaluated WMHs and lacunes only.”
Huang, V., Hogan, D. B., Ismail, Z., Maxwell, C. J., Smith, E. E., & Callahan, B. L. (2020) · Canadian Geriatrics Journal · doi:10.5770/cgj.23.416 · Cited by 5
Miller, T., Bittner, N., Moebus, S., & Caspers, S. (2024) · GeroScience · doi:10.1007/s11357-024-01306-w · Cited by 3
“The assessment of WML for a clinical diagnosis, such as mild cognitive impairment or dementia [7], is typically performed by visual inspections of the MR scans using rating scales (e.g.: [8–11]).”
Camarda, C., Torelli, P., Pipia, C., Sottile, G., Cilluffo, G., & Camarda, R. (2020) · Current Alzheimer Research · doi:10.2174/1567205017666201109093314 · Cited by 4
Changes in brain structure and function: under-recognized complications of type 1 diabetes
Nunley, K. A. (2014) · D-Scholarship@Pitt (University of Pittsburgh) · Cited by 2
Gregoire, S. M. (2014) · UCL Discovery (University College London) · Cited by 3
Brain MRI correlates of depression and vascular risk: Whitehall Imaging sub-study
Allan, C. (2014) · UCL Discovery (University College London) · Cited by 1
Volumetric assessment of hippocampus and cerebral white matter lesions in structural MRI
Olsson, E. (2013) · Gothenburg University Publications Electronic Archive (Gothenburg University)
MRI of White Matter in Dementia in Relation to Cardiovascular Risks
Sławek, J. W., & Białecka, M. A. (2014) · Elsevier eBooks · doi:10.1016/b978-0-12-407824-6.00038-0 · Cited by 1
Raya, S. P. A. B., & José, J. (2016) · Dipòsit Digital de la Universitat de Barcelona (Universitat de Barcelona)
Boulos, M. I., Muir, R. T., Gao, F., Lim, A. S. P., Swartz, R. H., Black, S. E., Walters, A. S., & Murray, B. J. (2017) · SLEEP · doi:10.1093/sleep/zsx060
“In comparison, we used the Age Related White Matter Change (ARWMC) Scale, a well-validated rating scale,(6) which is strongly correlated to WMH volume and furthermore to cognitive function.(3) Second, details surrounding the scoring of PLMs are not provided by Manconi et al.”
MethodSelf-citationAljondi, R. (2017) · Minerva Access (University of Melbourne)
Kwon, J.-W., Kim, H., & Lee, K. J. (2018) · 精神身體醫學 = Korean journal of psychosomatic medicine · doi:10.22722/kjpm.2018.26.2.119 · Cited by 1
Jaffar, A. S., & Latif, M. S. (2025) · Research Square · doi:10.21203/rs.3.rs-7431094/v1
Burrows, E. J., D’Antona, L., Booker, J., Islam, S., Shariff, F., Deehan, M., Monticelli, M., Rehman, H., Marcus, H. J., & Toma, A. K. (2026) · Neurosurgery Open · doi:10.1227/neuprac.0000000000000257
“Patients were followed for a median duration of 28 months [16-59].”
Influentialvan Dijk, G. G. C. E. A., Box, C. V. J., Faghir, F., Martens, J., Papma, J. M., Mattace-Raso, F. U. S., van den Berg, E., Vernooij, M. W., & Wolters, F. J. (2026) · Cerebral Circulation - Cognition and Behavior · doi:10.1016/j.cccb.2026.100564
Metadata
- Authors
- Gao, F.-Q., Swartz, R. H., Scheltens, P., Leibovitch, F. S., Kiss, A., Honjo, K., & Black, S. E.
- Affiliations
- LC Campbell Cognitive Neurology Research Unit, Heart and Stroke Foundation Centre for Stroke Recovery, University of Toronto, Toronto, ON, Canada
- University of Toronto (CA)
- Amsterdam UMC Location Vrije Universiteit Amsterdam (NL)
- Venue
- Journal of Alzheimer's Disease
- Publisher
- SAGE Publications
- Published
- 2011-10-04
- Volume
- 26
- Issue
- s3
- Pages
- 379-388
- ISSN
- 1387-2877 (ISSN-L)
- 1875-8908
- OpenAlex
- W1534377073
- Cited by
- 50
- References
- 36
- Citations by year
- 2026: 2 · 2025: 1 · 2024: 3 · 2023: 1 · 2022: 4 · 2021: 1 · 2020: 3 · 2019: 5 · 2018: 2 · 2017: 6 · 2016: 4 · 2015: 4 · 2014: 7 · 2013: 4 · 2012: 2
- FWCI
- 3.06
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of Alzheimer s Disease (publishedVersion)
- PubMed (publishedVersion)
- Pure Amsterdam UMC (publishedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- Funders
- Canadian Institutes of Health Research (13129, MT13129)
- Topics
- Health Sciences › Medicine › Psychiatry and Mental health › Dementia and Cognitive Impairment Research
- Health Sciences › Medicine › Clinical Neurology › Cerebrovascular and genetic disorders
- Life Sciences › Neuroscience › Neurology › Neurological Disease Mechanisms and Treatments
- Keywords
- white matter hyperintensities
- MeSH terms
- Aged, Aged, 80 and over, Aging, Alzheimer Disease, Analysis of Variance, Brain, Female, Humans, Magnetic Resonance Imaging, Male, Middle Aged, Nerve Fibers, Myelinated, Statistics, Nonparametric, Cognitive Dysfunction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.