The performance of digital microscopy for primary diagnosis in human pathology: a systematic review
Abstract
Validation studies of whole slide imaging (WSI) systems produce evidence regarding digital microscopy (DM). This systematic review aimed to provide information about the performance of WSI devices by evaluating intraobserver agreement reported in previously published studies as the best evidence to elucidate whether DM is reliable for primary diagnostic purposes. In addition, this review delineates the reasons for the occurrence of discordant diagnoses. Scopus, MEDLINE/PubMed, and Embase were searched electronically. A total of 13 articles were included. The total sample of 2145 had a majority of 695 (32.4%) cases from dermatopathology, followed by 200 (9.3%) cases from gastrointestinal pathology. Intraobserver agreements showed an excellent concordance, with values ranging from 87% to 98.3% (κ coefficient range 0.8-0.98). Ten studies (77%) reported a total of 128 disagreements. The remaining three studies (23%) did not report the exact number and nature of disagreements. Borderline/challenging cases were the most frequently reported reason for disagreements (53.8%). Six authors reported limitations of the equipment and/or limited image resolution as reasons for the discordant diagnoses. Within these articles, the reported pitfalls were as follows: difficulties in the identification of eosinophilic granular bodies in brain biopsies; eosinophils and nucleated red blood cells; and mitotic figures, nuclear details, and chromatin patterns in neuropathology specimens. The lack of image clarity was reported to be associated with difficulties in the identification of microorganisms (e.g., Candida albicans, Helicobacter pylori, and Giardia lamblia). However, authors stated that the intraobserver variances do not derive from technical limitations of WSI. A lack of clinical information was reported by four authors as a source for disagreements. Two studies (15.4%) reported poor quality of the biopsies, specifically small size of the biopsy material or inadequate routine laboratory processes as reasons for disagreements. One author (7.7%) indicated the lack of immunohistochemistry and special stains as a source for discordance. Furthermore, nine studies (69.2%) did not consider the performance of the digital method-limitations of the equipment, insufficient magnification/limited image resolution-as reasons for disagreements. To summarize the pitfalls of digital pathology practice and better address the root cause of the diagnostic discordance, we suggest a Categorization for Digital Pathology Discrepancies to be used in further validations studies. Among 99 discordances, only 37 (37.3%) had preferred diagnosis rendered by means of WSI. The risk of bias and applicability concerns were judged with the QUADAS-2. Two studies (15.4%) presented an unclear risk of bias in the sample selection domain and 2 (15.4%) presented a high risk of bias in the index test domain. Regarding applicability, all studies included were classified as a low concern in all domains. The included studies were optimally designed to validate WSI for general clinical use, providing evidence with confidence. In general, this systematic review showed a high concordance between diagnoses achieved by using WSI and conventional light microscope (CLM), summarizes difficulties related to specific findings of certain areas of pathology-including dermatopathology, pediatric pathology, neuropathology, and gastrointestinal pathology-and demonstrated that WSI can be used to render primary diagnoses in several subspecialties of human pathology.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Cites
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & for the PRISMA Group (2009) · BMJ · doi:10.1136/bmj.b2535 · Cited by 83,823 · see citing works
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & The PRISMA Group (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000097 · Cited by 64,042 · see citing works
Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement.
(2009) · PubMed Central · Cited by 44,859 · see citing works
Rayyan-a web and mobile app for systematic reviews.
Ouzzani, M., Hammady, H., Fedorowicz, Z., & Elmagarmid, A. (2016) · Systematic Reviews · doi:10.1186/s13643-016-0384-4 · Cited by 26,092 · see citing works
Wang, G., Ma, R., Qiao, G., Wada, K., Aizawa, Y., & Satoh, T. (2015) · Integrative Medicine International · doi:10.1159/000368408 · Cited by 18,759 · see citing works
Model Assisted Survey Sampling
Särndal, C., Swensson, B., & Wretman, J. H. (1992) · Springer series in statistics · doi:10.1007/978-1-4612-4378-6 · Cited by 3,417 · see citing works
Pantanowitz, L., Sinard, J. H., Henricks, W. H., Fatheree, L. A., Carter, A. B., Contis, L. C., Beckwith, B. A., Evans, A. J., Lal, A., & Parwani, A. V. (2013) · Archives of Pathology & Laboratory Medicine · doi:10.5858/arpa.2013-0093-cp · Cited by 622 · see citing works
Review of the current state of whole slide imaging in pathology
Pantanowitz, L., Valenstein, P. N., Evans, A. J., Kaplan, K. J., Pfeifer, J. D., Wilbur, D. C., Collins, L. C., & Colgan, T. J. (2011) · Journal of Pathology Informatics · doi:10.4103/2153-3539.83746 · Cited by 435 · see citing works
Mukhopadhyay, S., Feldman, M. D., Abels, E., Ashfaq, R., Beltaifa, S., Cacciabeve, N. G., Cathro, H. P., Cheng, L., Cooper, K., Dickey, G. E., et al. (2018) · American Journal of Surgical Pathology · doi:10.1097/pas.0000000000000948 · Cited by 399 · see citing works
Leeflang, M., Moons, K. G. M., Reitsma, J. B., & Zwinderman, A. H. (2008) · Clinical Chemistry · doi:10.1373/clinchem.2007.096032 · Cited by 351 · see citing works
No role for quality scores in systematic reviews of diagnostic accuracy studies
Whiting, P. F., Harbord, R. M., & Kleijnen, J. (2005) · BMC Medical Research Methodology · doi:10.1186/1471-2288-5-19 · Cited by 333 · see citing works
Validation of digital pathology imaging for primary histopathological diagnosis
Snead, D. R. J., Tsang, Y., Meskiri, A., Kimani, P. K., Crossman, R., Rajpoot, N. M., Blessing, E., Chen, K., Gopalakrishnan, K., Matthews, P., et al. (2016) · Histopathology · doi:10.1111/his.12879 · Cited by 238 · see citing works
The Diagnostic Concordance of Whole Slide Imaging and Light Microscopy: A Systematic Review
Goacher, E., Randell, R., Williams, B. J., & Treanor, D. E. (2016) · Archives of Pathology & Laboratory Medicine · doi:10.5858/arpa.2016-0025-ra · Cited by 173 · see citing works
Validation of whole slide imaging for primary diagnosis in surgical pathology
Bauer, T. W., Schoenfield, L., Slaw, R. J., Yerian, L., Sun, Z., & Henricks, W. H. (2013) · Archives of Pathology & Laboratory Medicine · doi:10.5858/arpa.2011-0678-oa · Cited by 155 · see citing works
Cornish, T. C., Swapp, R. E., & Kaplan, K. J. (2012) · Advances in Anatomic Pathology · doi:10.1097/pap.0b013e318253459e · Cited by 136 · see citing works
Koch, L. H., Lampros, J. N., DeLong, L. K., Chen, S. C., Woosley, J. T., & Hood, A. F. (2009) · Human Pathology · doi:10.1016/j.humpath.2008.10.009 · Cited by 101 · see citing works
Whole slide images for primary diagnostics of gastrointestinal tract pathology: a feasibility study
Al‐Janabi, S., Huisman, A., Vink, A., Leguit, R. J., Offerhaus, G. J. A., ten Kate, F. J. W., & van Diest, P. J. (2011) · Human Pathology · doi:10.1016/j.humpath.2011.06.017 · Cited by 92 · see citing works
Nielsen, P. S., Lindebjerg, J., Rasmussen, J., Starklint, H., Waldstrøm, M., & Nielsen, B. (2010) · Human Pathology · doi:10.1016/j.humpath.2010.05.015 · Cited by 91 · see citing works
Digital Pathology Consultations—a New Era in Digital Imaging, Challenges and Practical Applications
Lauro, G. R., Cable, W., Lesniak, A. J., et al. (2013) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-013-9572-0 · Cited by 88 · see citing works
Concordance between whole-slide imaging and light microscopy for routine surgical pathology
Campbell, W. S., Lele, S. M., West, W. W., Lazenby, A. J., Smith, L. M., & Hinrichs, S. H. (2012) · Human Pathology · doi:10.1016/j.humpath.2011.12.023 · Cited by 86 · see citing works
Whole slide images for primary diagnostics in dermatopathology: A feasibility study
Al-Janabi, S., Huisman, A., Vink, A., Leguit, R. J., Offerhaus, G. J. A., ten Kate, F. J. W., van Dijk, M. R., & van Diest, P. J. (2012) · Journal of Clinical Pathology · doi:10.1136/jclinpath-2011-200277 · Cited by 81 · see citing works
Fallon, M. A., Wilbur, D. C., & Prasad, M. L. (2010) · Archives of Pathology & Laboratory Medicine · doi:10.5858/2009-0320-oa.1 · Cited by 79 · see citing works
Williams, B. J., Hanby, A. M., Millican-Slater, R. A., Nijhawan, A., Verghese, E. T., & Treanor, D. E. (2017) · Histopathology · doi:10.1111/his.13403 · Cited by 78 · see citing works
Validating Whole-Slide Imaging for Consultation Diagnoses in Surgical Pathology
Bauer, T. W., & Slaw, R. J. (2014) · Archives of Pathology & Laboratory Medicine · doi:10.5858/arpa.2013-0541-oa · Cited by 77 · see citing works
Diagnostic time in digital pathology: A comparative study on 400 cases
Vodovnik, A. (2016) · Journal of Pathology Informatics · doi:10.4103/2153-3539.175377 · Cited by 74 · see citing works
Krishnamurthy, S., Mathews, K., McClure, S., Murray, M. P., Gilcrease, M. Z., Albarracin, C. T., Spinosa, J. C., Chang, B., Ho, J., Holt, J., et al. (2013) · Archives of Pathology & Laboratory Medicine · doi:10.5858/arpa.2012-0437-oa · Cited by 71 · see citing works
A Systematic Analysis of Discordant Diagnoses in Digital Pathology Compared With Light Microscopy
Williams, B. J., DaCosta, P., Goacher, E., & Treanor, D. E. (2017) · Archives of Pathology & Laboratory Medicine · doi:10.5858/arpa.2016-0494-oa · Cited by 71 · see citing works
Digital slide images for primary diagnostics in breast pathology: a feasibility study
Al‐Janabi, S., Huisman, A., Willems, S. M., & van Diest, P. J. (2012) · Human Pathology · doi:10.1016/j.humpath.2012.03.027 · Cited by 65 · see citing works
Rodríguez-Urrego, P. A., Cronin, A. M., Al‐Ahmadie, H. A., Gopalan, A., Tickoo, S. K., Reuter, V. E., & Fine, S. W. (2010) · Human Pathology · doi:10.1016/j.humpath.2010.07.001 · Cited by 65 · see citing works
Whole Slide Imaging and Its Applications to Histopathological Studies of Liver Disorders
Melo, R. C. N., Raas, M. W. D., Palazzi, C., Neves, V. H., Malta, K. K., & Silva, T. P. D. (2020) · Frontiers in Medicine · doi:10.3389/fmed.2019.00310 · Cited by 64 · see citing works
Whole slide imaging diagnostic concordance with light microscopy for breast needle biopsies
Campbell, W. S., Hinrichs, S. H., Lele, S. M., Baker, J. J., Lazenby, A. J., Talmon, G. A., Smith, L. M., & West, W. W. (2014) · Human Pathology · doi:10.1016/j.humpath.2014.04.007 · Cited by 63 · see citing works
Whole slide imaging: uses and limitations for surgical pathology and teaching
Boyce, B. F. (2015) · Biotechnic & Histochemistry · doi:10.3109/10520295.2015.1033463 · Cited by 60 · see citing works
Tabata, K., Mori, I., Sasaki, T., Itoh, T., Shiraishi, T., Yoshimi, N., Maeda, I., Harada, O., Taniyama, K., Taniyama, D., et al. (2017) · Pathology International · doi:10.1111/pin.12590 · Cited by 60 · see citing works
Buck, T. P., Dilorio, R., Havrilla, L., & O’Neill, D. G. (2014) · Journal of Pathology Informatics · doi:10.4103/2153-3539.145731 · Cited by 59 · see citing works
Diagnostic Accuracy of Virtual Pathology vs Traditional Microscopy in a Large Dermatopathology Study
Kent, M. N., Olsen, T. G., Feeser, T. A., Tesno, K. C., Moad, J. C., Conroy, M. P., Kendrick, M. J. J., Stephenson, S. R., Murchland, M. R., Khan, A. U., et al. (2017) · JAMA Dermatology · doi:10.1001/jamadermatol.2017.3284 · Cited by 58 · see citing works
Ordï, J., Castillo, P., Saco, A., Pino, M. L. D., Ordi, O., Rodríguez‐Carunchio, L., & Ramírez, J. (2014) · Journal of Clinical Pathology · doi:10.1136/jclinpath-2014-202524 · Cited by 57 · see citing works
Validation of Whole-Slide Imaging for Histolopathogical Diagnosis: Current State
Saco, A., Ramírez, J., Rakislova, N., et al. (2016) · Pathobiology · doi:10.1159/000442823 · Cited by 52 · see citing works
Feng, M., Deng, Y., Yang, L., Jing, Q., Zhang, Z., Xu, L., Wei, X., Zhou, Y., Wu, D., Fei, X., et al. (2020) · Diagnostic Pathology · doi:10.1186/s13000-020-00957-5 · Cited by 52 · see citing works
Brunelli, M., Beccari, S., Colombari, R., Gobbo, S., Giobelli, L., Pellegrini, A., Chilosi, M., Lunardi, M., Martignoni, G., Scarpa, A., et al. (2014) · Diagnostic Pathology · doi:10.1186/1746-1596-9-s1-s12 · Cited by 50 · see citing works
Diagnostic Efficiency in Digital Pathology
Mills, A. M., Gradecki, S. E., Horton, B. J., Blackwell, R., Moskaluk, C. A., Mandell, J. W., Mills, S. E., & Cathro, H. P. (2017) · The American Journal of Surgical Pathology · doi:10.1097/pas.0000000000000930 · Cited by 47 · see citing works
Thrall, M. J., Wimmer, J. L., & Schwartz, M. R. (2015) · Archives of Pathology & Laboratory Medicine · doi:10.5858/arpa.2014-0073-oa · Cited by 45 · see citing works
Analysis of T1 Bladder Cancer on Biopsy and Transurethral Resection Specimens
Leivo, M. Z., Sahoo, D., Hamilton, Z. A., et al. (2017) · The American Journal of Surgical Pathology · doi:10.1097/pas.0000000000000964 · Cited by 43 · see citing works
Randell, R., Ruddle, R. A., Mello-Thoms, C. R., et al. (2012) · Histopathology · doi:10.1111/j.1365-2559.2012.04323.x · Cited by 42 · see citing works
Whole slide images for primary diagnostics of paediatric pathology specimens: a feasibility study
Al‐Janabi, S., Huisman, A., Nikkels, P. G. J., ten Kate, F. J. W., & van Diest, P. J. (2012) · Journal of Clinical Pathology · doi:10.1136/jclinpath-2012-201104 · Cited by 42 · see citing works
Validation of diagnostic accuracy using digital slides in routine histopathology
Fónyad, L., Krenács, T., Nagy, P., Zalatnai, A., Csomor, J., Sápi, Z., Pápay, J., Schönléber, J., Diczházi, C., & Molnar, B. A. (2012) · Diagnostic Pathology · doi:10.1186/1746-1596-7-35 · Cited by 40 · see citing works
Shah, K., Lehman, J. S., Gibson, L. E., Lohse, C. M., Comfere, N. I., & Wieland, C. N. (2016) · Journal of the American Academy of Dermatology · doi:10.1016/j.jaad.2016.08.024 · Cited by 38 · see citing works
Elmore, J. G., Longton, G., PEPE, M. S., Carney, P. A., Nelson, H. D., Allison, K. H., Geller, B. M., Onega, T., Tosteson, A. N. A., Mercan, E., et al. (2017) · Journal of Pathology Informatics · doi:10.4103/2153-3539.201920 · Cited by 35 · see citing works
Arnold, M., Chenever, E., Baker, P. B., Boué, D. R., Fung, B., Hammond, S., Hendrickson, B. W., Kahwash, S. B., Pierson, C. R., Prasad, V., et al. (2014) · Pediatric and Developmental Pathology · doi:10.2350/14-07-1523-oa.1 · Cited by 34 · see citing works
Saco, A., Díaz, A., Hernandez, M., et al. (2017) · Digestive and Liver Disease · doi:10.1016/j.dld.2017.07.002 · Cited by 34 · see citing works
Lee, J. J., Jedrych, J. J., Pantanowitz, L., & Ho, J. (2017) · American Journal of Dermatopathology · doi:10.1097/dad.0000000000000888 · Cited by 34 · see citing works
Whole slide images for primary diagnostics of urinary system pathology: a feasibility study.
Al‐Janabi, S., Huisman, A., Jonges, G. N., ten Kate, F. J. W., Goldschmeding, R., & van Diest, P. J. (2014) · PubMed Central · doi:10.12861/jrip.2014.26 · Cited by 32 · see citing works
Sanders, D. S. A., Grabsch, H. I., Harrison, R. F., et al. (2012) · Histopathology · doi:10.1111/j.1365-2559.2012.04288.x · Cited by 27 · see citing works
Digital slide viewing for primary reporting in gastrointestinal pathology: a validation study
Loughrey, M. B., Kelly, P. J., Houghton, O. P., Coleman, H. G., Houghton, J., Carson, A., Salto‐Tellez, M., & Hamilton, P. W. (2015) · Archiv für Pathologische Anatomie und Physiologie und für Klinische Medicin · doi:10.1007/s00428-015-1780-1 · Cited by 27 · see citing works
Sakuma, Y., Nishikiori, H., Hirai, S., Yamaguchi, M., Yamada, G., Watanabe, A., Hasegawa, T., Kojima, T., Niki, T., & Takahashi, H. (2016) · Laboratory Investigation · doi:10.1038/labinvest.2015.155 · Cited by 26 · see citing works
Rodrigues‐Fernandes, C. I., Bonan, R. F., Bonan, P. R. F., et al. (2018) · Journal of Dental Education · doi:10.21815/jde.018.084 · Cited by 24 · see citing works
Foad, A. F. A. (2016) · Journal of Taibah University Medical Sciences · doi:10.1016/j.jtumed.2016.10.015 · Cited by 22 · see citing works
Gage, J. C., Joste, N. E., Ronnett, B. M., Stoler, M. H., Hunt, W. C., Schiffman, M., & Wheeler, C. M. (2013) · Human Pathology · doi:10.1016/j.humpath.2013.06.015 · Cited by 18 · see citing works
Validation of digital microscopy in the histopathological diagnoses of oral diseases
Araújo, A. L. D., do Amaral‐Silva, G. K., Fonseca, F. P., et al. (2018) · Archiv für Pathologische Anatomie und Physiologie und für Klinische Medicin · doi:10.1007/s00428-018-2382-5 · Cited by 18 · see citing works
Self-citationPekmezci, M., Uysal, S. P., Orhan, Y. C., et al. (2016) · Journal of Pathology Informatics · doi:10.4103/2153-3539.181769 · Cited by 14 · see citing works
FAVIA, G., Franco, S., Miccoli, S., et al. (2013) · Archiv für Pathologische Anatomie und Physiologie und für Klinische Medicin · doi:10.1007/s00428-013-1444-y · Cited by 11 · see citing works
Abstracts of the Singapore General Hospital 19th Annual Scientific Meeting 2011
(2011) · Proceedings of Singapore Healthcare · doi:10.1177/20101058110200s101 · Cited by 7 · see citing works
Sturm, B., Wj, M., Creytens, D., et al. (2017) · Ghent University Academic Bibliography (Ghent University) · Cited by 4 · see citing works
Torlakovic, E., Nielsen, S. M., Taylor, C. R., et al. (2015) · Laboratory Investigation · doi:10.1038/labinvest.2015.25 · Cited by 3 · see citing works
Validation of a Digital Pathology Whole Slide Imaging System
Maleeff, B. E. (2014) · Microscopy and Microanalysis · doi:10.1017/s1431927614008782 · Cited by 2 · see citing works
Comparing digital and optical microscopy diagnoses of breast and prostate core biopsies
Jara‐Lazaro, A. R., & Tan, P. H. (2011) · Pathology · doi:10.1097/pat.0b013e32834e4254 · Cited by 2 · see citing works
Making Primary Diagnosis on Liver Allograft Biopsies With Whole Slide Images - A Validation Study
Wang, M., Liu, S., Xie, C., et al. (2015) · American Journal of Clinical Pathology · doi:10.1093/ajcp/144.suppl2.168 · Cited by 2 · see citing works
(2013) · Laboratory Investigation · doi:10.1038/labinvest.2013.36 · Cited by 2 · see citing works
Westaby, J., Miles, C., Nadarajasundaram, A., et al. (2017) · The Journal of Pathology · doi:10.1002/path.4984 · Cited by 2 · see citing works
(2014) · Laboratory Investigation · doi:10.1038/labinvest.2014.28 · Cited by 1 · see citing works
(2012) · Laboratory Investigation · doi:10.1038/labinvest.2012.24 · Cited by 1 · see citing works
(2012) · Laboratory Investigation · doi:10.1038/labinvest.2012.23 · Cited by 1 · see citing works
(2016) · Laboratory Investigation · doi:10.1038/labinvest.2016.15 · Cited by 1 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
AI in Medical Imaging Informatics: Current Challenges and Future Directions
Panayides, A. S., Amini, A., Filipovic, N. D., Sharma, A., Tsaftaris, S. A., Young, A., Foran, D., Do, N., Golemati, S., Kurc, T., et al. (2020) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2020.2991043 · Cited by 713 · see citing works
- Authors
- Araújo, A. L. D., Arboleda, L. P. A., Palmier, N. R., Fonsêca, J. M., de Pauli Paglioni, M., Gomes-Silva, W., Ribeiro, A. C. P., Brandão, T. B., Simonato, L. E., Speight, P. M., Fonseca, F. P., Lopes, M. A., de Almeida, O. P., Vargas, P. A., Madrid Troconis, C. C., & Santos-Silva, A. R.
- Affiliations
- Universidade Estadual de Campinas (UNICAMP) (BR)
- Universidade de São Paulo (BR)
- Universidade Brasil (BR)
- University of Sheffield (GB)
- Universidade Federal de Minas Gerais (BR)
- ORCID
- Venue
- Virchows Archiv
- Published
- 2019-01-26
- Volume
- 474
- Issue
- 3
- Pages
- 269-287
- ISSN
- 0945-6317 (ISSN-L)
- 1432-2307
- OpenAlex
- W2914038321
- Cited by
- 64
- References
- 72
- Citations by year
- 2026: 3 · 2025: 3 · 2024: 8 · 2023: 6 · 2022: 11 · 2021: 18 · 2020: 11 · 2019: 4
- FWCI
- 0.72
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Archiv für Pathologische Anatomie und Physiologie und für Klinische Medicin (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Conselho Nacional de Desenvolvimento Cient?fico e Tecnol?gico (306699/2017-6)
- Funda??o de Amparo ? Pesquisa do Estado de S?o Paulo (2009/53839-2)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
- Health Sciences › Medicine › Physiology › Clinical Laboratory Practices and Quality Control
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Keywords
- whole slide imaging, digital pathology, intraobserver agreement, diagnostic discordance, primary diagnosis, dermatopathology, gastrointestinal pathology, neuropathology, image resolution, biopsy quality, diagnostic pitfalls, immunohistochemistry limitations, special stains, brain biopsy
- MeSH terms
- Biopsy, Humans, Image Interpretation, Computer-Assisted, Microscopy, Pathology, Predictive Value of Tests, Reproducibility of Results, Observer Variation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
