Cited by 94Clinical Laboratory Practices and Quality ControlClinical Reasoning and Diagnostic SkillsRadiology practices and education

Interpretive Diagnostic Error Reduction in Surgical Pathology and Cytology: Guideline From the College of American Pathologists Pathology and Laboratory Quality Center and the Association of Directors of Anatomic and Surgical Pathology

Open access
Nakhleh, Raouf E., Nosé, Vania, Colasacco, Carol, Fatheree, Lisa A., Lillemoe, Tamera J., McCrory, Douglas C., Meier, Frederick A., Otis, Christopher N. +5 more·2016·Archives of Pathology & Laboratory Medicine

Abstract

Context: Additional reviews of diagnostic surgical and cytology cases have been shown to detect diagnostic discrepancies. Objective: To develop, through a systematic review of the literature, recommendations for the review of pathology cases to detect or prevent interpretive diagnostic errors. Design: The College of American Pathologists Pathology and Laboratory Quality Center in association with the Association of Directors of Anatomic and Surgical Pathology convened an expert panel to develop an evidence-based guideline to help define the role of case reviews in surgical pathology and cytology. A literature search was conducted to gather data on the review of cases in surgical pathology and cytology. Results: The panel drafted 5 recommendations, with strong agreement from open comment period participants ranging from 87% to 93%. The recommendations are: (1) anatomic pathologists should develop procedures for the review of selected pathology cases to detect disagreements and potential interpretive errors; (2) anatomic pathologists should perform case reviews in a timely manner to avoid having a negative impact on patient care; (3) anatomic pathologists should have documented case review procedures that are relevant to their practice setting; (4) anatomic pathologists should continuously monitor and document the results of case reviews; and (5) if pathology case reviews show poor agreement within a defined case type, anatomic pathologists should take steps to improve agreement. Conclusions: Evidence exists that case reviews detect errors; therefore, the expert panel recommends that anatomic pathologists develop procedures for the review of pathology cases to detect disagreements and potential interpretive errors, in order to improve the quality of patient care.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.5858/arpa.2014-0511-sa

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • CAP guidelines prioritize error detection through secondary reviews without providing quantifiable metrics for measuring diagnostic accuracy.

    “Furthermore, the CAP guidelines prioritize error detection through secondary reviews without providing quantifiable metrics for measuring diagnostic accuracy.”

    From Toward Safer Diagnoses: A SEIPS-Based Narrative Review of Diagnostic Errors (2026) · cites this paper as [147]

    The Conclusion recommends secondary case reviews to detect or prevent interpretive errors. But the Outcomes of Interest section describes calculating discrepancy rates, and Limitations says those rates may measure quality within a group. The guideline cautions against using them to compare laboratories; it does not omit quantifiable measures altogether. Page numbers are not provided in our copy. (LLM, conf=0.87)

    Experimental — accuracy not yet measured
  • CAP guidelines recommend reviewing patient cases to reduce interpretive diagnostic errors in surgical pathology and cytology.

    “For example, the College of American Pathologists (CAP) guidelines recommend reviewing patient cases to reduce interpretive diagnostic errors in surgical pathology and cytology [147].”
  • CAP guidelines did not provide clear benchmarks for timeliness.

    “However, clear benchmarks for timeliness were not provided.”

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The citation lists haven’t been fetched yet

References 149 · Cited by 94

Part of this paper’s reference list could not be split into single entries.

Research works printed but not linked: 93

  1. Howanitz PJ. Errors in laboratory medicine: practical lessons to improve patient safety. Arch Pathol Lab Med. 2005;129(10):1252-1261.
  2. Meier FA, Varney RC, Zarbo RJ. Study of amended reports to evaluate and improve surgical pathology processes. Adv Anat Pathol. 2011;18(5):406-413.
  3. Nakhleh RE. Diagnostic error in surgical pathology. Diagn Histopathol. 2013;19(12):433-437.
  4. Renshaw AA, Gould EW. Increasing agreement over time in interlaboratory anatomic pathology consultation material. Am J Clin Pathol. 2013;140(2):215- 218.
  5. Raab SS, Meier FA, Zarbo RJ, et al. The ''Big Dog'' effect: variability assessing the causes of error in diagnoses of patients with lung cancer. J Clin Oncol. 2006;24(18):2808-2814.
  6. Raab SS, Nakhleh RE, Ruby SG. Patient safety in anatomic pathology: measuring discrepancy frequencies and causes. Arch Pathol Lab Med. 2005; 129(4):459-466.
  7. Al-Maghrabi JA, Sayadi HH. The importance of second opinion in surgical pathology referral material of lymphoma. Saudi Med J. 2012;33(4):399-405.
  8. Arbiser ZK, Folpe AL, Weiss SW. Consultative (expert) second opinions in soft tissue pathology: analysis of problem-prone diagnostic situations. Am J Clin Pathol. 2001;116(4):473-476.
  9. Coffin CS, Burak KW, Hart J, Gao ZH. The impact of pathologist experience on liver transplant biopsy interpretation. Mod Pathol. 2006;19(6): 832-838.
  10. Rakovitch E, Mihai A, Pignol JP, et al. Is expert breast pathology assessment necessary for the management of ductal carcinoma in situ? Breast Cancer Res Treat. 2004;87(3):265-272.
  11. Swapp RE, Aubry MC, Salomao DR, Cheville JC. Outside case review of surgical pathology for referred patients: the impact on patient care. Arch Pathol Lab Med. 2013;137(2):233-240.
  12. Ray-Coquard I, Montesco MC, Coindre JM, et al. Sarcoma: concordance between initial diagnosis and centralized expert review in a population-based study within three European regions. Ann Oncol. 2012;23(9):2442-2449.
  13. Fryback DG, Thornbury JR. The efficacy of diagnostic imaging. Med Decis Making. 1991;11(2):88-94.
  14. Lind AC, Bewtra C, Healy JC, Sims KL. Prospective peer review in surgical pathology. Am J Clin Pathol. 1995;104(5):560-566.
  15. Novis D. Routine review of surgical pathology cases as a method by which to reduce diagnostic errors in a community hospital. Pathol Case Rev. 2005;10(2):63-67.
  16. Owens SR, Wiehagen LT, Kelly SM, et al. Initial experience with a novel pre-sign-out quality assurance tool for review of random surgical pathology diagnoses in a subspecialty-based university practice. Am J Surg Pathol. 2010; 34(9):1319-1323.
  17. Renshaw AA, Gould EW. Measuring the value of review of pathology material by a second pathologist. Am J Clin Pathol. 2006;125(5):737-739.
  18. Nakhleh RE, Zarbo RJ. Amended reports in surgical pathology and implications for diagnostic error detection and avoidance: a College of American Pathologists q-probes study of 1,667,547 accessioned cases in 359 laboratories. Arch Pathol Lab Med. 1998;122(4):303-309.
  19. Raab SS, Stone CH, Jensen CS, et al. Double slide viewing as a cytology quality improvement initiative. Am J Clin Pathol. 2006;125(4):526-533.
  20. Renshaw AA, Cartagena N, Granter SR, Gould EW. Agreement and error rates using blinded review to evaluate surgical pathology of biopsy material. Am J Clin Pathol. 2003;119(6):797-800.
  21. Renshaw AA, Gould EW. Comparison of disagreement and amendment rates by tissue type and diagnosis: identifying cases for directed blinded review. Am J Clin Pathol. 2006;126(5):736-739.
  22. Troxel DB. Medicolegal aspects of error in pathology. Arch Pathol Lab Med. 2006;130(5):617-619.
  23. Kornstein MJ, Byrne SP. The medicolegal aspect of error in pathology: a search of jury verdicts and settlements. Arch Pathol Lab Med. 2007;131(4):615- 618.
  24. Raab SS, Grzybicki DM, Mahood LK, Parwani AV, Kuan SF, Rao UN. Effectiveness of random and focused review in detecting surgical pathology error. Am J Clin Pathol. 2008;130(6):905-912.
  25. Nakhleh RE, Bekeris LG, Souers RJ, Meier FA, Tworek JA. Surgical pathology case reviews before sign-out: a College of American Pathologists Q- Probes study of 45 laboratories. Arch Pathol Lab Med. 2010;134(5):740-743.
  26. The Bethesda System for reporting cervical/vaginal cytologic diagnoses: report of the 1991 Bethesda Workshop. Am J Surg Pathol. 1992;16(9):914- 916.
  27. Cibas ES, Ali SZ. The Bethesda System for reporting thyroid cytopathol- ogy. Am J Clin Pathol. 2009;132(5):658-665.
  28. Reid BJ, Haggitt RC, Rubin CE, et al. Observer variation in the diagnosis of dysplasia in Barrett's esophagus. Hum Pathol. 1988;19(2):166-178.
  29. Teutsch SM, Bradley LA, Palomaki GE, et al. The evaluation of genomic applications in practice and prevention (EGAPP) initiative: methods of the EGAPP Working Group. Genet Med. 2009;11(1):3-14.
  30. Abt AB, Abt LG, Olt GJ. The effect of interinstitution anatomic pathology consultation on patient care. Arch Pathol Lab Med. 1995;119(6):514-517.
  31. Agarwal S, Wadhwa N. Revisiting old slides-how worthwhile is it? Pathol Res Pract. 2010;206(6):368-371.
  32. Allsbrook WCJ, Mangold KA, Johnson MH, Lane RB, Lane CG, Epstein JI. Interobserver reproducibility of Gleason grading of prostatic carcinoma: general pathologist. Hum Pathol. 2001;32(1):81-88.
  33. Ascoli V, Bosco D, Carnovale Scalzo C. Cytologic re-evaluation of negative effusions from patients with malignant mesothelioma. Pathologica. 2011;103(6):318-324.
  34. Berney DM, Fisher G, Kattan MW, et al. Pitfalls in the diagnosis of prostatic cancer: retrospective review of 1791 cases with clinical outcome. Histopathology. 2007;51(4):452-457.
  35. Boiko PE, Piepkorn MW. Reliability of skin biopsy pathology. J Am Board Fam Pract. 1994;7(5):371-374.
  36. Castanon A, Ferryman S, Patnick J, Sasieni P. Review of cytology and histopathology as part of the NHS Cervical Screening Programme audit of invasive cervical cancers. Cytopathology. 2012;23(1):13-22.
  37. Clary KM, Silverman JF, Liu Y, et al. Cytohistologic discrepancies: a means to improve pathology practice and patient outcomes. Am J Clin Pathol. 2002; 117(4):567-573.
  38. Coblentz TR, Mills SE, Theodorescu D. Impact of second opinion pathology in the definitive management of patients with bladder carcinoma. Cancer. 2001;91(7):1284-1290.
  39. Cook IS, McCormick D, Poller DN. Referrals for second opinion in surgical pathology: implications for management of cancer patients in the UK. Eur J Surg Oncol. 2001;27(6):589-594.
  40. Golfier F, Clerc J, Hajri T, et al. Contribution of referent pathologists to the quality of trophoblastic diseases diagnosis. Hum Reprod. 2011;26(10):2651- 2657.
  41. Jacques SM, Qureshi F, Munkarah A, Lawrence WD. Interinstitutional surgical pathology review in gynecologic oncology: II. Endometrial cancer in hysterectomy specimens. Int J Gynecol Pathol. 1998;17(1):42-45.
  42. Jones K, Jordan RC. Patterns of second-opinion diagnosis in oral and maxillofacial pathology. Oral Surg Oral Med Oral Path Oral Radiol Endod. 2010; 109(6):865-869.
  43. Kennecke HF, Speers CH, Ennis CA, Gelmon K, Olivotto IA, Hayes M. Impact of routine pathology review on treatment for node-negative breast cancer. J Clin Oncol. 2012;30(18):2227-2231.
  44. Kwon JS, Francis JA, Qiu F, Weir MM, Ettler HC. When is a pathology review indicated in endometrial cancer? Obstet Gynecol. 2007;110(6):1224- 1230.
  45. Layfield LJ, Jones C, Rowe L, Gopez EV. Institutional review of outside cytology materials: a retrospective analysis of two institutions' experiences. Diagn Cytopathol. 2002;26(1):45-48.
  46. Lester JF, Dojcinov SD, Attanoos RL, et al. The clinical impact of expert pathological review on lymphoma management: a regional experience. Br J Haematol. 2003;123(3):463-468.
  47. Lueck N, Jensen C, Cohen MB, Weydert JA. Mandatory second opinion in cytopathology. Cancer. 2009;117(2):82-91.
  48. Lytwyn A, Salit IE, Raboud J, et al. Interobserver agreement in the interpretation of anal intraepithelial neoplasia. Cancer. 2005;103(7):1447- 1456.
  49. Manion E, Cohen MB, Weydert J. Mandatory second opinion in surgical pathology referral material: clinical consequences of major disagreements. Am J Surg Pathol. 2008;32(5):732-737.
  50. Matasar MJ, Shi W, Silberstien J, et al. Expert second-opinion pathology review of lymphoma in the era of the World Health Organization classification. Ann Oncol. 2012;23(1):159-166.
  51. McBroom HM, Ramsay AD. The clinicopathological meeting: a means of auditing diagnostic performance. Am J Surg Pathol. 1993;17(1):75-80.
  52. McGinnis KS, Lessin SR, Elder DE, et al. Pathology review of cases presenting to a multidisciplinary pigmented lesion clinic. Arch Dermatol. 2002; 138(5):617-621.
  53. Murali R, Hughes MT, Fitzgerald P, Thompson JF, Scolyer RA. Interobserver variation in the histopathologic reporting of key prognostic parameters, particularly Clark level, affects pathologic staging of primary cutaneous melanoma. Ann Surg. 2009;249(4):641-647.
  54. Owens SR, Dhir R, Yousem SA, et al. The development and testing of a laboratory information system-driven tool for pre-sign-out quality assurance of random surgical pathology reports. Am J Clin Pathol. 2010;133(6):836-841.
  55. Pinto Sanchez MI, Smecuol E, Vazquez H, Mazure R, Maurino E, Bai JC. Very high rate of misdiagnosis of celiac disease in clinical practice. Acta Gastroenterol Latinoam. 2009;39(4):250-253.
  56. Pongpruttipan T, Sitthinamsuwan P, Rungkaew P, Ruangchira-urai R, Vongjirad A, Sukpanichnant S. Pitfalls in classifying lymphomas. J Med Assoc Thai. 2007;90(6):1129-1136.
  57. Prayson RA, Agamanolis DP, Cohen ML, et al. Interobserver reproduc- ibility among neuropathologists and surgical pathologists in fibrillary astrocytoma grading. J Neurol Sci. 2000;175(1):33-39.
  58. Proctor IE, McNamara C, Rodriguez-Justo M, Isaacson PG, Ramsay A. Importance of expert central review in the diagnosis of lymphoid malignancies in a regional cancer network. J Clin Oncol. 2011;29(11):1431-1435.
  59. Qureshi A, Loya A, Azam M, Hussain M, Mushtaq S, Mahmood T. Study of parameters to ensure quality control in histopathology reporting: a meta- analysis at a tertiary care center. Indian J Pathol Microbiol. 2012;55(2):180-182.
  60. Raab SS, Grzybicki DM, Janosky JE, et al. Clinical impact and frequency of anatomic pathology errors in cancer diagnoses. Cancer. 2005;104(10):2205- 2213.
  61. Ramsay AD, Gallagher PJ. Local audit of surgical pathology: 18 month's experience of peer review-based quality assessment in an English teaching hospital. Am J Surg Pathol. 1992;16(5):476-482.
  62. Renshaw AA, Gould EW. Correlation of workload with disagreement and amendment rates in surgical pathology and nongynecologic cytology. Am J Clin Pathol. 2006;125(6):820-822.
  63. Renshaw AA, Pinnar NE, Jiroutek MR, Young ML. Blinded review as a method for quality improvement in surgical pathology. Arch Pathol Lab Med. 2002;126(8):961-963.
  64. Renshaw AA, Pinnar NE, Jiroutek MR, Young ML. Quantifying the value of in-house consultation in surgical pathology. Am J Clin Pathol. 2002;117(5):751- 754.
  65. Santoso JT, Coleman RL, Voet RL, Bernstein SG, Lifshitz S, Miller D. Pathology slide review in gynecologic oncology. Obstet Gynecol. 1998;91(5, pt 1):730-734.
  66. Sharif MA, Hamdani SN. Second opinion and discrepancy in the diagnosis of soft tissue lesions at surgical pathology. Indian J Pathol Microbiol. 2010;53(3):460-464.
  67. Shoo BA, Sagebiel RW, Kashani-Sabet M. Discordance in the histopath- ologic diagnosis of melanoma at a melanoma referral center. J Am Acad Dermatol. 2010;62(5):751-756.
  68. Thomas CW, Bainbridge TC, Thomson TA, McGahan CE, Morris WJ. Clinical impact of second pathology opinion: a longitudinal study of central genitourinary pathology review before prostate brachytherapy. Brachytherapy. 2007;6(2):135-141.
  69. Trotter MJ, Bruecks AK. Interpretation of skin biopsies by general pathologists: diagnostic discrepancy rate measured by blinded review. Arch Pathol Lab Med. 2003;127(11):1489-1492.
  70. Tsuda H, Akiyama F, Kurosumi M, Sakamoto G, Watanabe T. Monitoring of interobserver agreement in nuclear atypia scoring of node-negative breast carcinomas judged at individual collaborating hospitals in the National Surgical Adjuvant Study of Breast Cancer (NSAS-BC) protocol. Jpn J Clin Oncol. 1999; 29(9):413-420.
  71. Wayment RO, Bourne A, Kay P, Tarter TH. Second opinion pathology in tertiary care of patients with urologic malignancies. Urol Oncol. 2011;29(2):194- 198.
  72. Wechsler J, Bastuji-Garin S, Spatz A, et al. Reliability of the histopath- ologic diagnosis of malignant melanoma in childhood. Arch Dermatol. 2002; 138(5):625-628.
  73. Westra WH, Kronz JD, Eisele DW. The impact of second opinion surgical pathology on the practice of head and neck surgery: a decade experience at a large referral hospital. Head Neck. 2002;24(7):684-693.
  74. Zaino RJ, Kauderer J, Trimble CL, et al. Reproducibility of the diagnosis of atypical endometrial hyperplasia: a Gynecologic Oncology Group study. Cancer. 2006;106(4):804-811.
  75. Hahm GK, Niemann TH, Lucas JG, Frankel WL. The value of second opinion in gastrointestinal and liver pathology. Arch Pathol Lab Med. 2001; 125(6):736-739.
  76. Ahmed Z, Yaqoob N, Muzaffar S, Kayani N, Pervez S, Hasan SH. Diagnostic surgical pathology: the importance of second opinion in a developing country. J Pak Med Assoc. 2004;54(6):306-311.
  77. Bomeisl PEJ, Alam S, Wakely PEJ. Interinstitutional consultation in fine- needle aspiration cytopathology: a study of 742 cases. Cancer. 2009;117(4):237- 246.
  78. Davidov T, Trooskin SZ, Shanker BA, et al. Routine second-opinion cytopathology review of thyroid fine needle aspiration biopsies reduces diagnostic thyroidectomy. Surgery. 2010;148(6):1294-1301.
  79. Aldape K, Simmons ML, Davis RL, et al. Discrepancies in diagnoses of neuroepithelial neoplasms: the San Francisco Bay Area Adult Glioma Study. Cancer. 2000;88(10):2342-2349.
  80. Bajaj J, Morgenstern N, Sugrue C, Wasserman J, Wasserman P. Clinical impact of second opinion in thyroid fine needle aspiration cytology (FNAC): a study of 922 interinstitutional consultations. Diagn Cytopathol. 2012;40(5):422- 429.
  81. Baloch ZW, Hendreen S, Gupta PK, et al. Interinstitutional review of thyroid fine-needle aspirations: impact on clinical management of thyroid nodules. Diagn Cytopathol. 2001;25(4):231-234.
  82. Bejarano PA, Koehler A, Sherman KE. Second opinion pathology in liver biopsy interpretation. Am J Gastroenterol. 2001;96(11):3158-3164.
  83. Bruner JM, Inouye L, Fuller GN, Langford LA. Diagnostic discrepancies and their clinical impact in a neuropathology referral practice. Cancer. 1997; 79(4):796-803.
  84. Butler ST, Youker SR, Mandrell J, Flanagan KH, Fosko SW. The importance of reviewing pathology specimens before Mohs surgery. Dermatol Surg. 2009; 35(3):407-412.
  85. Chafe S, Honore L, Pearcey R, Capstick V. An analysis of the impact of pathology review in gynecologic cancer. Int J Radiat Oncol Biol Phys. 2000;48(5): 1433-1438.
  86. Chan YM, Cheung AN, Cheng DK, Ng TY, Ngan HY, Wong LC. Pathology slide review in gynecologic oncology: routine or selective? Gynecol Oncol. 1999;75(2):267-271.
  87. Corley DA, Kubo A, DeBoer J, Rumore GJ. Diagnosing Barrett's esophagus: reliability of clinical and pathologic diagnoses. Gastrointest Endosc. 2009;69(6):1004-1010.
  88. Epstein JI, Walsh PC, Sanfilippo F. Clinical and cost impact of second- opinion pathology: review of prostate biopsies prior to radical prostatectomy. Am J Surg Pathol. 1996;20(7):851-857.
  89. Jara-Lazaro AR, Thike AA, Tan PH. Diagnostic issues in second opinion consultations in prostate pathology. Pathology. 2010;42(1):6-14.
  90. Kamat S, Parwani AV, Khalbuss WE, et al. Use of a laboratory information system driven tool for pre-signout quality assurance of random cytopathology reports. J Pathol Inform. 2011;2:42.
  91. Kishimoto R, Saika T, Bekku K, et al. The clinical impact of pathological review on selection the treatment modality for localized prostate cancer in candidates for brachytherapy monotherapy. World J Urol. 2012;30(3):375-378.
  92. Kronz JD, Milord R, Wilentz R, Weir EG, Schreiner SR, Epstein JI. Lesions missed on prostate biopsies in cases sent in for consultation. Prostate. 2003;54(4): 310-314.
  93. Lehnhardt M, Daigeler A, Hauser J, et al. The value of expert second opinion in diagnosis of soft tissue sarcomas. J Surg Oncol. 2008;97(1):40-43.

Authors
Nakhleh, R. E., Nosé, V., Colasacco, C., Fatheree, L. A., Lillemoe, T. J., McCrory, D. C., Meier, F. A., Otis, C. N., Owens, S. R., Raab, S. S., Turner, R. R., Ventura, C. B., & Renshaw, A. A.
Affiliations
  • Abbott Northwestern Hospital (US)
  • College of American Pathologists (US)
  • Mayo Clinic (US)
  • Baystate Medical Center (US)
  • University of Michigan (US)
  • Memorial University of Newfoundland (CA)
  • Saint John's Health Center (US)
  • Massachusetts General Hospital (US)
Venue
Archives of Pathology & Laboratory Medicine
Published
2015-05-12
Volume
140
Issue
1
Pages
29-40
ISSN
  • 0003-9985 (ISSN-L)
  • 1543-2165
OpenAlex
W2129439022
Cited by
94
References
149
Citations by year
2026: 7 · 2025: 8 · 2024: 6 · 2023: 7 · 2022: 8 · 2021: 5 · 2020: 10 · 2019: 7 · 2018: 9 · 2017: 8 · 2016: 12 · 2015: 4
FWCI
8.12
Type
journal article
Language
en
Open access
Locations
  • Archives of Pathology & Laboratory Medicine (publishedVersion)
  • PubMed (publishedVersion)
Topics
  • Health Sciences › Medicine › Physiology › Clinical Laboratory Practices and Quality Control
  • Health Sciences › Medicine › Family Practice › Clinical Reasoning and Diagnostic Skills
  • Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiology practices and education
Keywords
diagnostic errors, case review, surgical pathology, cytopathology, pathological anatomy, diagnostic agreement, quality improvement, patient safety, expert panel recommendations
MeSH terms
Systematic Reviews as Topic, Cytodiagnosis, Diagnostic Errors, Humans, Laboratories, Pathology, Surgical

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review