Cited by 132Cancer Immunotherapy and BiomarkersHepatocellular Carcinoma Treatment and PrognosisHepatitis B Virus Studies

Monitoring Immune Checkpoint Regulators as Predictive Biomarkers in Hepatocellular Carcinoma

Open access
Shrestha, Ritu, Prithviraj, Prashanth, Anaka, Matthew, Bridle, Kim R., Crawford, Darrell H. G., Dhungel, Bijay, Steel, Jason C., Jayachandran, Aparna·2018·Frontiers in Oncology

Abstract

The global burden of Hepatocellular carcinoma (HCC), one of the frequent causes of cancer-related deaths worldwide, is rapidly increasing partly due to the limited treatment options available for this disease and recurrence due to therapy resistance. Immune checkpoint inhibitors that are proved to be beneficial in the treatment of advanced melanoma and other cancer types are currently in clinical trials in HCC. These ongoing trials are testing the efficacy and safety of a few select checkpoints in HCC. Similar to observations in other cancers, these immune checkpoint blockade treatments as monotherapy may benefit only a fraction of HCC patients. Studies that assess the prevalence and distribution of other immune checkpoints/modulatory molecules in HCC have been limited. Moreover, robust predictors to identify which HCC patients will respond to immunotherapy are currently lacking. The objective of this study is to perform a comprehensive evaluation on different immune modulators as predictive biomarkers to monitor HCC patients at high risk for poor prognosis. We screened publically available HCC patient databases for the expression of previously well described immune checkpoint regulators and evaluated the usefulness of these immune modulators to predict high risk, patient overall survival and recurrence. We also identified the immune modulators that synergized with known immune evasion molecules PD-L1, PD-1 and CTLA-4 and correlated with worse patient outcomes. We evaluated the association between the expression of epithelial-to-mesenchymal (EMT) markers and PD-L1 in HCC patient tumors. We also examined the relationship of tumor mutational burden with HCC patient survival. Notably, expression of immune modulators B7-H4, PD-L2, TIM-3, and VISTA were independently associated with worse prognosis, while B7-H4, CD73 and VISTA predicted low recurrence free survival. Moreover, the prognosis of patients expressing high PD-L1 with high B7-H4, TIM-3, VISTA, CD73 and PD-L2 expression was significantly worse. Interestingly, PD-L1 expression in HCC patients in the high risk group was closely associated with EMT marker expression and prognosticates poor survival. In HCC patients, high TMB predicted worse patient outcomes than those with low TMB.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.3389/fonc.2018.00269

Cites

5 of 5 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • B7H4, PDL1, and TIM3 are all associated with mortality in HCC.

    “which are all associated with mortality in HCC (53)”

    From HIF inhibitor 32-134D eradicates murine hepatocellular carcinoma in combination with anti-PD1 therapy (2022) · cites this paper as [53]

    The Abstract and Discussion associate B7-H4 and TIM-3 with worse prognosis or overall survival in HCC. The Results section on coordinated immune-regulator expression reports that PD-L1 expression alone did not show poor survival; worse survival was associated with PD-L1 in combination with other markers. The claim omits that qualification for PD-L1. Page numbers are not available in our copy. (LLM, conf=0.90)

    Experimental — accuracy not yet measured
  • B7H4 and TIM3 mRNA showed hypoxia-induced expression in vehicle-treated but not in 32-134D-treated Hepa1-6 cells (P < 0.05).

    “B7H4 and TIM3 mRNA, which are encoded by the Vtcn1 and Havcr2 genes, respectively, and like PDL1, encode immune checkpoint receptors that are HIF regulated (51, 52) and associated with patient mortality in HCC (53), showed hypoxia-induced expression in vehicle-treated but not in 32-134D–treated cells (P < 0.05; Figure 3B).”

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 62

  1. [1]Ferlay J Soerjomataram I Dikshit R Eser S Mathers C Rebelo M Cancer incidence and mortality worldwide: sources, methods and major patterns in GLOBOCAN 2012. Int J Cancer (2015) 136(5): E359– 86. 10.1002/ijc.29210 25220842 · DOI · Paper page
  2. [2]Montagnana M Danese E Lippi G. Squamous cell carcinoma antigen in hepatocellular carcinoma: ready for the prime time? Clin Chim Acta (2015) 445: 161– 6. 10.1016/j.cca.2015.03.031 25840050 · DOI
  3. [3]Chacko S Samanta S Hepatocellular carcinoma: a life-threatening disease. Biomed Pharmacother (2016) 84: 1679– 88. 10.1016/j.biopha.2016.10.078 27823920 · DOI
  4. [4]Balogh J Victor D Asham EH Burroughs SG Boktour M Saharia A Hepatocellular carcinoma: a review. J Hepatocell Carcinoma (2016) 3: 41– 53. 10.2147/JHC.S61146 27785449 PMC5063561 · DOI
  5. [5]Parikh S Hyman D. Hepatocellular cancer: a guide for the internist. Am J Med (2007) 120(3): 194– 202. 10.1016/j.amjmed.2006.11.020 17349437 · DOI
  6. [6]European Association For The Study Of The Liver, European Organisation For Research And Treatment Of Cancer. EASL-EORTC clinical practice guidelines: management of hepatocellular carcinoma. Eur J Cancer (2012) 48(5): 599– 641. 10.1016/j.ejca.2011.12.021 22424278 · DOI
  7. [7]Dhungel B Jayachandran A Layton CJ Steel JC. Seek and destroy: targeted adeno-associated viruses for gene delivery to hepatocellular carcinoma. Drug Deliv (2017) 24(1): 289– 99. 10.1080/10717544.2016.1247926 28165834 PMC8241004 · DOI
  8. [8]Llovet JM Burroughs A Bruix J. Hepatocellular carcinoma. Lancet (2003) 362(9399): 1907– 17. 10.1016/S0140-6736(03)14964-1 14667750 · DOI
  9. [9]Mazzaferro V Regalia E Doci R Andreola S Pulvirenti A Bozzetti F Liver transplantation for the treatment of small hepatocellular carcinomas in patients with cirrhosis. N Engl J Med (1996) 334(11): 693– 9. 10.1056/NEJM199603143341104 8594428 · DOI
  10. [10]Chen Z Shen S Peng B Tao J. Intratumoural GM-CSF microspheres and CTLA-4 blockade enhance the antitumour immunity induced by thermal ablation in a subcutaneous murine hepatoma model. Int J Hyperthermia (2009) 25(5): 374– 82. 10.1080/02656730902976807 19479598 · DOI
  11. [12]Mir N Jayachandran A Dhungel B Shrestha R Steel JC. Epithelial-to-mesenchymal transition: a mediator of sorafenib resistance in advanced hepatocellular carcinoma. Curr Cancer Drug Targets (2017) 17(8): 698– 706. 10.2174/1568009617666170427104356 28460616 · DOI
  12. [13]Hato T Goyal L Greten TF Duda DG Zhu AX. Immune checkpoint blockade in hepatocellular carcinoma: current progress and future directions. Hepatology (2014) 60(5): 1776– 82. 10.1002/hep.27246 24912948 PMC4211962 · DOI
  13. [14]Nemeth E Baird AW O’Farrelly C. Microanatomy of the liver immune system. Semin Immunopathol (2009) 31(3): 333– 43. 10.1007/s00281-009-0173-4 19639317 · DOI
  14. [15]Thomson AW Knolle PA. Antigen-presenting cell function in the tolerogenic liver environment. Nat Rev Immunol (2010) 10(11): 753– 66. 10.1038/nri2858 20972472 · DOI
  15. [16]Hodi FS O’Day SJ McDermott DF Weber RW Sosman JA Haanen JB Improved survival with ipilimumab in patients with metastatic melanoma. N Engl J Med (2010) 363(8): 711– 23. 10.1056/NEJMoa1003466 20525992 PMC3549297 · DOI · Paper page
  16. [17]Topalian SL Drake CG Pardoll DM. Immune checkpoint blockade: a common denominator approach to cancer therapy. Cancer Cell (2015) 27(4): 450– 61. 10.1016/j.ccell.2015.03.001 25858804 PMC4400238 · DOI · Paper page
  17. [18]Pardoll DM. The blockade of immune checkpoints in cancer immunotherapy. Nat Rev Cancer (2012) 12(4): 252– 64. 10.1038/nrc3239 22437870 PMC4856023 · DOI · Paper page
  18. [19]Kudo M. Immune checkpoint inhibition in hepatocellular carcinoma: basics and ongoing clinical trials. Oncology (2017) 92(Suppl 1): 50– 62. 10.1159/000451016 28147363 · DOI
  19. [20]Sangro B Gomez-Martin C de la Mata M Iñarrairaegui M Garralda E Barrera P A clinical trial of CTLA-4 blockade with tremelimumab in patients with hepatocellular carcinoma and chronic hepatitis C. J Hepatol (2013) 59(1): 81– 8. 10.1016/j.jhep.2013.02.022 23466307 · DOI
  20. [21]Zaretsky JM Garcia-Diaz A Shin DS Escuin-Ordinas H Hugo W Hu-Lieskovan S Mutations associated with acquired resistance to PD-1 blockade in melanoma. N Engl J Med (2016) 375(9): 819– 29. 10.1056/NEJMoa1604958 27433843 PMC5007206 · DOI · Paper page
  21. [22]Kalluri R Weinberg RA. The basics of epithelial-mesenchymal transition. J Clin Invest (2009) 119(6): 1420– 8. 10.1172/JCI39104 19487818 PMC2689101 · DOI · Paper page
  22. [23]Kudo-Saito C Shirako H Takeuchi T Kawakami Y. Cancer metastasis is accelerated through immunosuppression during Snail-induced EMT of cancer cells. Cancer Cell (2009) 15(3): 195– 206. 10.1016/j.ccr.2009.01.023 19249678 · DOI
  23. [24]Mak MP Tong P Diao L Cardnell RJ Gibbons DL William WN A patient-derived, pan-cancer EMT signature identifies global molecular alterations and immune target enrichment following epithelial-to-mesenchymal transition. Clin Cancer Res (2016) 22(3): 609– 20. 10.1158/1078-0432.CCR-15-0876 26420858 PMC4737991 · DOI
  24. [25]Bouillez A Rajabi H Jin C Samur M Tagde A Alam M MUC1-C integrates PD-L1 induction with repression of immune effectors in non-small-cell lung cancer. Oncogene (2017) 36(28): 4037– 46. 10.1038/onc.2017.47 28288138 PMC5509481 · DOI
  25. [26]Aguirre-Gamboa R Gomez-Rueda H Martínez-Ledesma E Martínez-Torteya A Chacolla-Huaringa R Rodriguez-Barrientos A SurvExpress: an online biomarker validation tool and database for cancer gene expression data using survival analysis. PLoS One (2013) 8(9): e74250. 10.1371/journal.pone.0074250 24066126 PMC3774754 · DOI
  26. [27]Gao J Aksoy BA Dogrusoz U Dresdner G Gross B Sumer SO Integrative analysis of complex cancer genomics and clinical profiles using the cBioPortal. Sci Signal (2013) 6(269): l1. 10.1126/scisignal.2004088 23550210 PMC4160307 · DOI · Paper page
  27. [28]Hoshida Y Villanueva A Kobayashi M Peix J Chiang DY Camargo A Gene expression in fixed tissues and outcome in hepatocellular carcinoma. N Engl J Med (2008) 359(19): 1995– 2004. 10.1056/NEJMoa0804525 18923165 PMC2963075 · DOI · Paper page
  28. [29]Hoshida Y Nijman SM Kobayashi M Chan JA Brunet JP Chiang DY Integrative transcriptome analysis reveals common molecular subclasses of human hepatocellular carcinoma. Cancer Res (2009) 69(18): 7385– 92. 10.1158/0008-5472.CAN-09-1089 19723656 PMC3549578 · DOI · Paper page
  29. [30]Tsuchiya M Parker JS Kono H Matsuda M Fujii H Rusyn I. Gene expression in nontumoral liver tissue and recurrence-free survival in hepatitis C virus-positive hepatocellular carcinoma. Mol Cancer (2010) 9: 74. 10.1186/1476-4598-9-74 20380719 PMC2856554 · DOI
  30. [31]Stelzer G Rosen N Plaschkes I Zimmerman S Twik M Fishilevich S The genecards suite: from gene data mining to disease genome sequence analyses. Curr Protoc Bioinformatics (2016) 54: 1.30.1– 1.30.33. 10.1002/cpbi.5 27322403 · DOI · Paper page
  31. [32]Jayachandran A Anaka M Prithviraj P Hudson C McKeown SJ Lo PH Thrombospondin 1 promotes an aggressive phenotype through epithelial-to-mesenchymal transition in human melanoma. Oncotarget (2014) 5(14): 5782– 97. 10.18632/oncotarget.2164 25051363 PMC4170613 · DOI
  32. [33]Awan FM Naz A Obaid A Ali A Ahmad J Anjum S Identification of circulating biomarker candidates for hepatocellular carcinoma (HCC): an integrated prioritization approach. PLoS One (2015) 10(9): e0138913. 10.1371/journal.pone.0138913 26414287 PMC4586137 · DOI
  33. [34]Zheng X Gai X Wu Z Liu Q Yao Y. Metastasin leads to poor prognosis of hepatocellular carcinoma through partly inducing EMT. Oncol Rep (2013) 29(5): 1811– 8. 10.3892/or.2013.2341 23483190 · DOI
  34. [35]Wan T Zhang T Si X Zhou Y. Overexpression of EMT-inducing transcription factors as a potential poor prognostic factor for hepatocellular carcinoma in Asian populations: a meta-analysis. Oncotarget (2017) 8(35): 59500– 8. 10.18632/oncotarget.18352 28938653 PMC5601749 · DOI
  35. [36]Lou Y Diao L Cuentas ER Denning WL Chen L Fan YH Epithelial-mesenchymal transition is associated with a distinct tumor microenvironment including elevation of inflammatory signals and multiple immune checkpoints in lung adenocarcinoma. Clin Cancer Res (2016) 22(14): 3630– 42. 10.1158/1078-0432.CCR-15-1434 26851185 PMC4947453 · DOI
  36. [38]Rizvi NA Hellmann MD Snyder A Kvistborg P Makarov V Havel JJ Cancer immunology. Mutational landscape determines sensitivity to PD-1 blockade in non-small cell lung cancer. Science (2015) 348(6230): 124– 8. 10.1126/science.aaa1348 25765070 PMC4993154 · DOI · Paper page
  37. [40]Chen CL Pan QZ Zhao JJ Wang Y Li YQ Wang QJ PD-L1 expression as a predictive biomarker for cytokine-induced killer cell immunotherapy in patients with hepatocellular carcinoma. Oncoimmunology (2016) 5(7): e1176653. 10.1080/2162402X.2016.1176653 27622026 PMC5006896 · DOI
  38. [41]Passiglia F Bronte G Bazan V Natoli C Rizzo S Galvano A PD-L1 expression as predictive biomarker in patients with NSCLC: a pooled analysis. Oncotarget (2016) 7(15): 19738– 47. 10.18632/oncotarget.7582 26918451 PMC4991415 · DOI
  39. [42]Yuan L Dong L Yu G Fan W Zhang L Wang P Aberrant expression of B7-H4 may contribute to the development of hepatocellular carcinoma. Mol Med Rep (2016) 14(6): 5015– 24. 10.3892/mmr.2016.5887 27840912 PMC5355751 · DOI
  40. [43]Kang FB Wang L Sun DX Li HJ Li D Wang Y B7-H4 overexpression is essential for early hepatocellular carcinoma progression and recurrence. Oncotarget (2017) 8(46): 80878– 88. 10.18632/oncotarget.20718 29113351 PMC5655246 · DOI
  41. [44]Zhang C Li Y Wang Y Diagnostic value of serum B7-H4 for hepatocellular carcinoma. J Surg Res (2015) 197(2): 301– 6. 10.1016/j.jss.2015.04.034 25963168 · DOI
  42. [45]Zhang SA Wu ZX Zhang X Zeng ZY Li DL Circulating B7-H4 in serum predicts prognosis in patients with hepatocellular carcinoma. Genet Mol Res (2015) 14(4): 13041– 8. 10.4238/2015.October.21.25 26505457 · DOI
  43. [46]Carreno BM Collins M. BTLA: a new inhibitory receptor with a B7-like ligand. Trends Immunol (2003) 24(10): 524– 7. 10.1016/j.it.2003.08.005 14552835 · DOI
  44. [47]Li N Xu W Yuan Y Ayithan N Imai Y Wu X Immune-checkpoint protein VISTA critically regulates the IL-23/IL-17 inflammatory axis. Sci Rep (2017) 7(1): 1485 10.1038/s41598-017-01411-1 28469254 PMC5431161 · DOI
  45. [48]Wu L Deng WW Huang CF Bu LL Yu GT Mao L Expression of VISTA correlated with immunosuppression and synergized with CD8 to predict survival in human oral squamous cell carcinoma. Cancer Immunol Immunother (2017) 66(5): 627– 36. 10.1007/s00262-017-1968-0 28236118 PMC11028774 · DOI
  46. [49]Hay CM Sult E Huang Q Mulgrew K Fuhrmann SR McGlinchey KA Targeting CD73 in the tumor microenvironment with MEDI9447. Oncoimmunology (2016) 5(8): e1208875. 10.1080/2162402X.2016.1208875 27622077 PMC5007986 · DOI
  47. [50]Loi S Pommey S Haibe-Kains B Beavis PA Darcy PK Smyth MJ CD73 promotes anthracycline resistance and poor prognosis in triple negative breast cancer. Proc Natl Acad Sci U S A (2013) 110(27): 11091– 6. 10.1073/pnas.1222251110 23776241 PMC3704029 · DOI
  48. [51]Lu XX Chen YT Feng B Mao XB Yu B Chu XY. Expression and clinical significance of CD73 and hypoxia-inducible factor-1α in gastric carcinoma. World J Gastroenterol (2013) 19(12): 1912– 8. 10.3748/wjg.v19.i12.1912 23569336 PMC3613106 · DOI
  49. [52]Turcotte M Spring K Pommey S Chouinard G Cousineau I George J CD73 is associated with poor prognosis in high-grade serous ovarian cancer. Cancer Res (2015) 75(21): 4494– 503. 10.1158/0008-5472.CAN-14-3569 26363007 · DOI
  50. [53]Ren ZH Lin CZ Cao W Yang R Lu W Liu ZQ CD73 is associated with poor prognosis in HNSCC. Oncotarget (2016) 7(38): 61690– 702. 10.18632/oncotarget.11435 27557512 PMC5308683 · DOI
  51. [54]Whiteside TL. Targeting adenosine in cancer immunotherapy: a review of recent progress. Expert Rev Anticancer Ther (2017) 17(6): 527– 35. 10.1080/14737140.2017.1316197 28399672 PMC6702668 · DOI
  52. [55]Anderson AC. Tim-3: an emerging target in the cancer immunotherapy landscape. Cancer Immunol Res (2014) 2(5): 393– 8. 10.1158/2326-6066.CIR-14-0039 24795351 · DOI
  53. [56]Yan W Liu X Ma H Zhang H Song X Gao L Tim-3 fosters HCC development by enhancing TGF-β-mediated alternative activation of macrophages. Gut (2015) 64(10): 1593– 604. 10.1136/gutjnl-2014-307671 25608525 · DOI
  54. [57]Cheng G Li M Wu J Ji M Fang C Shi H Expression of Tim-3 in gastric cancer tissue and its relationship with prognosis. Int J Clin Exp Pathol (2015) 8(8): 9452– 7. 26464703 PMC4583935
  55. [58]Suzuki K Kadota K Sima CS Sadelain M Rusch VW Travis WD Chronic inflammation in tumor stroma is an independent predictor of prolonged survival in epithelioid malignant pleural mesothelioma patients. Cancer Immunol Immunother (2011) 60(12): 1721– 8. 10.1007/s00262-011-1073-8 21769693 PMC6250069 · DOI
  56. [60]Wu Y Shi H Jiang M Qiu M Jia K Cao T The clinical value of combination of immune checkpoint inhibitors in cancer patients: a meta-analysis of efficacy and safety. Int J Cancer (2017) 141(12): 2562– 70. 10.1002/ijc.31012 28833119 · DOI
  57. [62]Gao J Ward JF Pettaway CA Shi LZ Subudhi SK Vence LM VISTA is an inhibitory immune checkpoint that is increased after ipilimumab therapy in patients with prostate cancer. Nat Med (2017) 23(5): 551– 5. 10.1038/nm.4308 28346412 PMC5466900 · DOI · Paper page
  58. [63]Chen L Gibbons DL Goswami S Cortez MA Ahn YH Byers LA Metastasis is regulated via microRNA-200/ZEB1 axis control of tumour cell PD-L1 expression and intratumoral immunosuppression. Nat Commun (2014) 5: 5241. 10.1038/ncomms6241 25348003 PMC4212319 · DOI
  59. [64]Alsuliman A Colak D Al-Harazi O Fitwi H Tulbah A Al-Tweigeri T Bidirectional crosstalk between PD-L1 expression and epithelial to mesenchymal transition: significance in claudin-low breast cancer cells. Mol Cancer (2015) 14: 149. 10.1186/s12943-015-0421-2 26245467 PMC4527106 · DOI
  60. [65]Alexandrov LB Nik-Zainal S Wedge DC Aparicio SA Behjati S Biankin AV Signatures of mutational processes in human cancer. Nature (2013) 500(7463): 415– 21. 10.1038/nature12477 23945592 PMC3776390 · DOI · Paper page
  61. [66]Goodman AM Kato S Bazhenova L Patel SP Frampton GM Miller V Tumor mutational burden as an independent predictor of response to immunotherapy in diverse cancers. Mol Cancer Ther (2017) 16(11): 2598– 608. 10.1158/1535-7163.MCT-17-0386 28835386 PMC5670009 · DOI · Paper page
  62. [67]Rosenberg JE Hoffman-Censits J Powles T van der Heijden MS Balar AV Necchi A Atezolizumab in patients with locally advanced and metastatic urothelial carcinoma who have progressed following treatment with platinum-based chemotherapy: a single-arm, multicentre, phase 2 trial. Lancet (2016) 387(10031): 1909– 20. 10.1016/S0140-6736(16)00561-4 26952546 PMC5480242 · DOI · Paper page

Cited by

Sorted by citations

Authors
Shrestha, R., Prithviraj, P., Anaka, M., Bridle, K. R., Crawford, D. H. G., Dhungel, B., Steel, J. C., & Jayachandran, A.
Affiliations
  • The University of Queensland (AU)
  • University of Alberta (CA)
Corresponding author
Jayachandran, Aparna
Venue
Frontiers in Oncology
Published
2018-07-13
Volume
8
Pages
269-269
ISSN
2234-943X (ISSN-L)
OpenAlex
W2884098295
Cited by
132
References
5
Citations by year
2026: 3 · 2025: 4 · 2024: 11 · 2023: 16 · 2022: 33 · 2021: 34 · 2020: 19 · 2019: 10 · 2018: 2
FWCI
6.11
Type
journal article
Language
en
Open access
Locations
  • Frontiers in Oncology (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Acquire (CQUniversity) (submittedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • Figshare (submittedVersion, cc-by)
Topics
  • Health Sciences › Medicine › Oncology › Cancer Immunotherapy and Biomarkers
  • Health Sciences › Medicine › Hepatology › Hepatocellular Carcinoma Treatment and Prognosis
  • Health Sciences › Medicine › Epidemiology › Hepatitis B Virus Studies
Keywords
hepatocellular carcinoma, predictive biomarkers, immune checkpoint inhibitors, PD-L1, PD-1, CTLA-4, B7-H4, PD-L2, TIM-3, VISTA, CD73, epithelial-mesenchymal transition, tumor mutational burden, prognosis, recurrence, overall survival, immune evasion

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review