The DOI matches its Crossref record; 88 of the 88 references it cites were found in a registry or an index.
Statistical Evaluation of a Biomarker
Abstract
A biomarker may provide a diagnosis, assess disease severity or risk, or guide other clinical interventions such as the use of drugs. Although considerable progress has been made in standardizing the methodology and reporting of randomized trials, less has been accomplished concerning the assessment of biomarkers. Biomarker studies are often presented with poor biostatistics and methodologic flaws that precludes them from providing a reliable and reproducible scientific message. A host of issues are discussed that can improve the statistical evaluation and reporting of biomarker studies. Investigators should be aware of these issues when designing their studies, editors and reviewers when analyzing a manuscript, and readers when interpreting results.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & for the PRISMA Group (2009) · BMJ · doi:10.1136/bmj.b2535 · Cited by 83,823
Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement.
(2009) · PubMed Central · Cited by 44,859
DeLong, E. R., DeLong, D. M., & Clarke-Pearson, D. L. (1988) · Biometrics · doi:10.2307/2531595 · Cited by 22,649
Measuring the Accuracy of Diagnostic Systems
Swets, J. A. (1988) · Science · doi:10.1126/science.3287615 · Cited by 10,058
Hanley, J. A., & McNeil, B. J. (1983) · Radiology · doi:10.1148/radiology.148.3.6878708 · Cited by 7,016
Receiver-operating characteristic (ROC) plots: a fundamental evaluation tool in clinical medicine
Zweig, M. H., & Campbell, G. (1993) · Clinical Chemistry · doi:10.1093/clinchem/39.4.561 · Cited by 6,569
Pencina, M. J., D' Agostino, R. B., D' Agostino, R. B., & Vasan, R. S. (2008) · Statistics in Medicine · doi:10.1002/sim.2929 · Cited by 6,493 · cited 3×
“Pencina et al.54 have recently purposed two ways of assessing improvement in model performance using reclassification tables: Net Reclassification Index (NRI) and Integrated Discrimination Improvement.”
BackgroundMethodInfluentialBasic principles of ROC analysis
Metz, C. E. (1978) · Seminars in Nuclear Medicine · doi:10.1016/s0001-2998(78)80014-2 · Cited by 6,026
2001 SCCM/ESICM/ACCP/ATS/SIS International Sepsis Definitions Conference
Levy, M. M., Fink, M. P., Marshall, J. C., Abraham, E., Angus, D., Cook, D., Cohen, J., Opal, S. M., Vincent, J.-L., & Ramsay, G. (2003) · Critical Care Medicine · doi:10.1097/01.ccm.0000050454.01978.3b · Cited by 5,995
The Revised CONSORT Statement for Reporting Randomized Trials: Explanation and Elaboration
Altman, D. G., Schulz, K. F., Moher, D., Egger, M., Davidoff, F., Elbourne, D., Gøtzsche, P. C., Lang, T., & for the CONSORT Group (2001) · Annals of Internal Medicine · doi:10.7326/0003-4819-134-8-200104170-00012 · Cited by 3,835
Rapid Measurement of B-Type Natriuretic Peptide in the Emergency Diagnosis of Heart Failure
Maisel, A. S., Krishnaswamy, P., Nowak, R. M., McCord, J., Hollander, J. E., Duc, P., Omland, T., Storrow, A. B., Abraham, W. T., Wu, A. H. B., et al. (2002) · New England Journal of Medicine · doi:10.1056/nejmoa020233 · Cited by 3,362
ROCR: visualizing classifier performance in R
Sing, T., Sander, O., Beerenwinkel, N., & Lengauer, T. (2005) · Bioinformatics · doi:10.1093/bioinformatics/bti623 · Cited by 3,200
Deeks, J. J., Altman, D. G., & Bradburn, M. J. (2001) · Systematic Reviews in Health Care · doi:10.1002/9780470693926.ch15 · Cited by 2,163 · cited 1×
“tistics are first computed for individually considered studies, and then a weighted average is computed across studies.(89) In”
MethodUse and Misuse of the Receiver Operating Characteristic Curve in Risk Prediction
Cook, N. R. (2007) · Circulation · doi:10.1161/circulationaha.106.672402 · Cited by 2,112
Empirical evidence of design-related bias in studies of diagnostic tests.
Lijmer, J. G. (1999) · JAMA · doi:10.1001/jama.282.11.1061 · Cited by 1,861 · cited 6×
“This bias (also called spectrum bias) may be associated with the largest bias effect.(3) Selection bias occurs when nonconsecutive patients or not randomly selected patients are included.”
BackgroundInfluentialPerkins, N. J., & Schisterman, E. F. (2006) · American Journal of Epidemiology · doi:10.1093/aje/kwj063 · Cited by 1,759
Genetic Determinants of Response to Clopidogrel and Cardiovascular Events
Simon, T., Verstuyft, C., Mary-Krause, M., Quteineh, L., Drouet, E., Méneveau, N., Steg, P. G., Ferrières, J., Danchin, N., & Becquemont, L. (2009) · New England Journal of Medicine · doi:10.1056/nejmoa0808227 · Cited by 1,728
A hierarchical regression approach to meta‐analysis of diagnostic test accuracy evaluations
Rutter, C. M., & Gatsonis, C. A. (2001) · Statistics in Medicine · doi:10.1002/sim.942 · Cited by 1,505
Systematic reviews of evaluations of diagnostic and screening tests
Deeks, J. J. (2001) · BMJ · doi:10.1136/bmj.323.7305.157 · Cited by 1,340
Prediction error estimation: a comparison of resampling methods
Molinaro, A. M., Simon, R., & Pfeiffer, R. M. (2005) · Bioinformatics · doi:10.1093/bioinformatics/bti499 · Cited by 1,339
Pepe, M. S. (2004) · American Journal of Epidemiology · doi:10.1093/aje/kwh101 · Cited by 1,276
Schisterman, E. F., Perkins, N. J., Liu, A., & Bondell, H. (2005) · Epidemiology · doi:10.1097/01.ede.0000147512.81966.ba · Cited by 1,185
Primer on Certain Elements of Medical Decision Making
McNeil, B. J., Keeler, E., & Adelstein, S. J. (1975) · New England Journal of Medicine · doi:10.1056/nejm197507312930501 · Cited by 1,140
The STARD Statement for Reporting Studies of Diagnostic Accuracy: Explanation and Elaboration
Bossuyt, P. M., Reitsma, J. B., Bruns, D. E., Gatsonis, C. A., Glasziou, P. P., Irwig, L. M., Moher, D., Rennie, D., de Vet, H. C. W., & Lijmer, J. G. (2003) · Annals of Internal Medicine · doi:10.7326/0003-4819-138-1-200301070-00012-w1 · Cited by 1,124 · cited 2×
“To address this gap in methodologic quality, some recommendations have been produced recently, but even they did not cover the entire biomarker domain.(4) Two main reasons may explain this situation.”
BackgroundMethodChrist-Crain, M., Jaccard-Stolz, D., Bingisser, R., Gencay, M. M., Huber, P. R., Tamm, M., & Müller, B. (2004) · The Lancet · doi:10.1016/s0140-6736(04)15591-8 · Cited by 1,096
(1975) · New England Journal of Medicine · doi:10.1056/nejm197507312930513 · Cited by 1,017
A readers' guide to the interpretation of diagnostic test properties: clinical example of sepsis
Fischer, J. E., Bachmann, L. M., & Jaeschke, R. (2003) · Intensive Care Medicine · doi:10.1007/s00134-003-1761-8 · Cited by 890
Use of Multiple Biomarkers to Improve the Prediction of Death from Cardiovascular Causes
Zethelius, B., Berglund, L., Sundström, J., Ingelsson, E., Basu, S., Larsson, A., Venge, P., & Ärnlöv, J. (2008) · New England Journal of Medicine · doi:10.1056/nejmoa0707064 · Cited by 889
Evaluation of QUADAS, a tool for the quality assessment of diagnostic accuracy studies
Whiting, P. F., Weswood, M. E., Rutjes, A. W., Reitsma, J. B., Bossuyt, P. N., & Kleijnen, J. (2006) · BMC Medical Research Methodology · doi:10.1186/1471-2288-6-9 · Cited by 769
A unification of models for meta-analysis of diagnostic accuracy studies
Harbord, R. M., Deeks, J. J., Egger, M., Whiting, P., & Sterne, J. A. C. (2007) · Biostatistics · doi:10.1093/biostatistics/kxl004 · Cited by 751
Use of methodological standards in diagnostic test research. Getting better but still not good
Reid, M. C. (1995) · JAMA · doi:10.1001/jama.274.8.645 · Cited by 712
Analyzing a Portion of the ROC Curve
McClish, D. K. (1989) · Medical Decision Making · doi:10.1177/0272989x8900900307 · Cited by 707
The use of relative operating characteristic (ROC) curves in test performance evaluation.
Beck, & Shultz, E. K. (1986) · PubMed · Cited by 621
Use of Procalcitonin to Shorten Antibiotic Treatment Duration in Septic Patients
Nobre, V., Harbarth, S., Graf, J.-D., Rohner, P., & Pugin, J. (2008) · American Journal of Respiratory and Critical Care Medicine · doi:10.1164/rccm.200708-1238oc · Cited by 605
Biases in the assessment of diagnostic tests
Begg, C. B. (1987) · Statistics in Medicine · doi:10.1002/sim.4780060402 · Cited by 496
BRENNER, H., & GEFELLER, O. (1997) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19970515)16:9<981::aid-sim510>3.0.co;2-n · Cited by 488
Sample size calculation should be performed for design accuracy in diagnostic test studies
Flahault, A., Cadilhac, M., & Thomas, G. (2005) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2004.12.009 · Cited by 468
The Limitations of Risk Factors as Prognostic Tools
Ware, J. H. (2006) · New England Journal of Medicine · doi:10.1056/nejmp068249 · Cited by 466
Cook, N. R., & Ridker, P. M. (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-150-11-200906020-00007 · Cited by 458
Konstantinides, S., Geibel, A., Olschewski, M., Kasper, W., Hruska, N., Jäckle, S., & Binder, L. (2002) · Circulation · doi:10.1161/01.cir.0000028422.51668.a2 · Cited by 447
Acute respiratory failure in the elderly: etiology, emergency diagnosis and prognosis
Ray, P., Birolleau, S., Lefort, Y., Becquemin, M.-H., Beigelman, C., Isnard, R., Teixeira, A., Arthaud, M., Riou, B., & Boddaert, J. (2006) · Critical Care · doi:10.1186/cc4926 · Cited by 411
Self-citationGibot, S., Kolopp-Sarda, M.-N., Béné, M. C., Cravoisy, A., Levy, B., Faure, G. C., & Bollaert, P.-E. (2004) · Annals of Internal Medicine · doi:10.7326/0003-4819-141-1-200407060-00009 · Cited by 407
Multivariable Analysis: A Primer for Readers of Medical Research
Katz, M. H. (2003) · Annals of Internal Medicine · doi:10.7326/0003-4819-138-8-200304150-00012 · Cited by 403
ROC Curves in Clinical Chemistry: Uses, Misuses, and Possible Solutions
Obuchowski, N. A., Lieber, M. L., & Wians, F. H. (2004) · Clinical Chemistry · doi:10.1373/clinchem.2004.031823 · Cited by 358
Occult hypoperfusion and mortality in patients with suspected infection
Howell, M. D., Donnino, M., Clardy, P., Talmor, D., & Shapiro, N. I. (2007) · Intensive Care Medicine · doi:10.1007/s00134-007-0680-5 · Cited by 356
SAMPLE SIZE DETERMINATION FOR DIAGNOSTIC ACCURACY STUDIES INVOLVING BINORMAL ROC CURVE INDICES
OBUCHOWSKI, N. A., & McCLISH, D. K. (1997) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19970715)16:13<1529::aid-sim565>3.0.co;2-h · Cited by 332
de Winter, R. J., Koster, R. W., Sturk, A., & Sanders, G. T. (1995) · Circulation · doi:10.1161/01.cir.92.12.3401 · Cited by 317
The use of “overall accuracy” to evaluate the validity of screening or diagnostic tests
Alberg, A. J., Park, J. W., Hager, B. W., Brock, M. V., & Diener-West, M. (2004) · Journal of General Internal Medicine · doi:10.1111/j.1525-1497.2004.30091.x · Cited by 295
Marshall, J. C., & Reinhart, K. (2009) · Critical Care Medicine · doi:10.1097/ccm.0b013e3181a02afc · Cited by 294
Saah, A. J., & Hoover, D. R. (1997) · Annals of Internal Medicine · doi:10.7326/0003-4819-126-1-199701010-00026 · Cited by 269
Sample size calculations in studies of test accuracy
Obuchowski, N. A. (1998) · Statistical Methods in Medical Research · doi:10.1177/096228029800700405 · Cited by 221
Evaluating diagnostic tests with imperfect standards
Valenstein, P. N. (1990) · American Journal of Clinical Pathology · doi:10.1093/ajcp/93.2.252 · Cited by 220 · cited 1×
“When an imperfect reference test must be used, it should be recognized that measures of test performance can be distorted.(79) Glueck et al.”
MethodA Comparison of C/B Ratios from Studies Using Receiver Operating Characteristic Curve Analysis
Cantor, S. B., Sun, C. C., Tortolero-Luna, G., Richards-Kortum, R., & Follen, M. (1999) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(99)00075-x · Cited by 218
Janes, H., & Pepe, M. S. (2008) · American Journal of Epidemiology · doi:10.1093/aje/kwn099 · Cited by 212
Prognostic Value of Brain Natriuretic Peptide in Noncardiac Surgery: A Meta-analysis
Ryding, A. D. S., Kumar, S., Worthington, A. M., & Burgess, D. (2009) · Anesthesiology · doi:10.1097/aln.0b013e3181aaeb11 · Cited by 187
Sample sizes of studies on diagnostic accuracy: literature survey.
Bachmann, L. M., Puhan, M. A., Riet, G. T., & Bossuyt, P. M. (2006) · BMJ · doi:10.1136/bmj.38793.637789.2f · Cited by 184
REGRET GRAPHS, DIAGNOSTIC UNCERTAINTY AND YOUDEN'S INDEX
HILDEN, J., & GLASZIOU, P. (1996) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19960530)15:10<969::aid-sim211>3.0.co;2-9 · Cited by 181
Perioperative Drug Therapy in Elderly Patients
Rivera, R., Antognini, J. F., & Riou, B. (2009) · Anesthesiology · doi:10.1097/aln.0b013e3181a10207 · Cited by 178
Self-citationBaker, M. (2005) · Nature Biotechnology · doi:10.1038/nbt0305-297 · Cited by 171
Hajian‐Tilaki, K., Hanley, J. A., Joseph, L., & Collet, J. (1997) · Medical Decision Making · doi:10.1177/0272989x9701700111 · Cited by 144
Zhang, D. D., Zhou, X., Freeman, D. H., & Freeman, J. L. (2002) · Statistics in Medicine · doi:10.1002/sim.1011 · Cited by 137
Multivariate random effects meta-analysis of diagnostic tests with multiple thresholds
Hamza, T. H., Arends, L. R., van Houwelingen, H. C., & Stijnen, T. (2009) · BMC Medical Research Methodology · doi:10.1186/1471-2288-9-73 · Cited by 131
An ROC-type measure of diagnostic accuracy when the gold standard is continuous-scale
Obuchowski, N. A. (2005) · Statistics in Medicine · doi:10.1002/sim.2228 · Cited by 128
Evaluating Bias and Variability in Diagnostic Test Reports
Mower, W. R. (1999) · Annals of Emergency Medicine · doi:10.1016/s0196-0644(99)70422-1 · Cited by 123
Hausfater, P., Juillien, G., Madonna-Py, B., Haroche, J., Bernard, M., & Riou, B. (2007) · Critical Care · doi:10.1186/cc5926 · Cited by 121
Self-citationInterval likelihood ratios: Another advantage for the evidence-based diagnostician
Brown, M. D., & Reeves, M. J. (2003) · Annals of Emergency Medicine · doi:10.1067/mem.2003.274 · Cited by 109
Amour, J., Birenbaum, A., Langeron, O., Manach, Y. L., Bertrand, M., Coriat, P., Riou, B., Bernard, M., & Hausfater, P. (2008) · Critical Care Medicine · doi:10.1097/ccm.0b013e3181692966 · Cited by 103
Self-citationWhen Should a New Test Become the Current Reference Standard?
Glasziou, P., Irwig, L., & Deeks, J. J. (2008) · Annals of Internal Medicine · doi:10.7326/0003-4819-149-11-200812020-00009 · Cited by 98
Modeling in Medical Decision Making: A Bayesian Approach
Parmigiani, G. (2002) · Cited by 97
Post hoc choice of cut points introduced bias to diagnostic research
Ewald, B. D. (2006) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2005.11.025 · Cited by 94
Prognostic factors in non-exertional heatstroke
Hausfater, P., Mégarbane, B., Dautheville, S., Patzak, A., Andronikof, M., Santin, A., André, S., Korchia, L., Terbaoui, N., Kierzek, G., et al. (2009) · Intensive Care Medicine · doi:10.1007/s00134-009-1694-y · Cited by 83
Self-citationDistribution-Free Confidence Bounds for ROC Curves
Hilgers, R. (1991) · Methods of Information in Medicine · doi:10.1055/s-0038-1634822 · Cited by 82
Greenland, S. (2007) · Statistics in Medicine · doi:10.1002/sim.2995 · Cited by 81
Ray, P., Arthaud, M., Birolleau, S., Isnard, R., Lefort, Y., Boddaert, J., Riou, B., & the EPIDASA Study Group (2005) · Journal of the American Geriatrics Society · doi:10.1111/j.1532-5415.2005.53213.x · Cited by 79
Self-citationSchneider, H. G., Lam, L., Lokuge, A., Krum, H., Naughton, M. T., Smit, P. D. V., Bystrzycki, A., Eccleston, D., Federman, J., Flannery, G., et al. (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-150-6-200903170-00004 · Cited by 75
Usefulness of B-type natriuretic peptide in elderly patients with acute dyspnea
Ray, P., Arthaud, M., Lefort, Y., Birolleau, S., Beigelman‐Aubry, C., Riou, B., & the EPIDASA Study Group (2004) · Intensive Care Medicine · doi:10.1007/s00134-004-2469-0 · Cited by 73
Self-citationJebali, M. A., Hausfater, P., Abbes, Z., Aouni, Z., Riou, B., & Ferjani, M. (2007) · Anesthesiology · doi:10.1097/01.anes.0000271871.07395.ad · Cited by 72
Self-citationLiu, J., Ma, M.-C., Wu, C.-Y., & Tai, J.-Y. (2005) · Statistics in Medicine · doi:10.1002/sim.2358 · Cited by 66
A non‐inferiority test for diagnostic accuracy based on the paired partial areas under ROC curves
Li, C., Liao, C., & Liu, J. (2007) · Statistics in Medicine · doi:10.1002/sim.3121 · Cited by 52
Fellahi, J.-L., Hédoire, F., Manach, Y. L., Monier, E., Guillou, L., & Riou, B. (2007) · Critical Care · doi:10.1186/cc6126 · Cited by 49
Self-citationIs procalcitonin a marker of critical illness in heatstroke?
Hausfater, P., Hurtado, M., Pease, S., Juillien, G., Lvovschi, V.-E., SALEHABADI, S., Lidove, O., Wolff, M., Bernard, M., Chollet‐Martin, S., et al. (2008) · Intensive Care Medicine · doi:10.1007/s00134-008-1083-y · Cited by 48
Self-citationFellahi, J.-L., Hanouz, J., Manach, Y. L., Gué, X., Monier, E., Guillou, L., & Riou, B. (2009) · Anesthesiology · doi:10.1097/aln.0b013e3181a1f720 · Cited by 44
Self-citationHausfater, P., Fillet, A., Rozenberg, F., Arthaud, M., Trystram, D., Huraux, J., Lebon, P., & Riou, B. (2004) · Journal of Medical Virology · doi:10.1002/jmv.20068 · Cited by 30
Self-citationA Single Procalcitonin Level Does Not Predict Adverse Outcomes of Women with Pyelonephritis
Lemiale, V., Renaud, B., Moutereau, S., Ngako, A., Salloum, M., Calmettes, M., Hervé, J., Boraud, C., Santin, A., Grego, J. C., et al. (2006) · European Urology · doi:10.1016/j.eururo.2006.12.018 · Cited by 21
Foxcroft, D., Kypri, K., & Simonite, V. (2009) · Addiction · doi:10.1111/j.1360-0443.2009.02574.x · Cited by 13
Glueck, D. H., Lamb, M. M., O’Donnell, C. I., Ringham, B. M., Brinton, J. T., Muller, K. E., Lewin, J., Alonzo, T. A., & Pisano, E. D. (2009) · BMC Medical Research Methodology · doi:10.1186/1471-2288-9-4 · Cited by 9
(1950) · doi:10.1002/1097-0142(1950)3:1<32::aid-cncr2820030106>3.0.co;2-3
Bootstrap confidence intervals: when, which, what? A practical guide for medical statisticians
Carpenter (2000) · doi:10.1002/(sici)1097-0258(20000515)19:9<1141::aid-sim479>3.0.co;2-f
Cited by
Hatib, F., Jian, Z., Buddi, S., Lee, C., Settels, J., Sibert, K., Rinehart, J., & Cannesson, M. (2018) · Anesthesiology · doi:10.1097/aln.0000000000002300 · Cited by 614
Cannesson, M., Le Manach, Y., Hofer, C. K., Goarin, J. P., Lehot, J.-J., Vallet, B., & Tavernier, B. (2011) · Anesthesiology · doi:10.1097/aln.0b013e318225b80a · Cited by 554
Soummer, A., Perbet, S., Brisson, H., Arbelot, C., Constantin, J.-M., Lu, Q., & Rouby, J.-J. (2012) · Critical Care Medicine · doi:10.1097/ccm.0b013e31824e68ae · Cited by 519
Molecular Biomarkers in Cancer
Sarhadi, V. K., & Armengol, G. (2022) · Biomolecules · doi:10.3390/biom12081021 · Cited by 467
Ahmad, A., Imran, M., & Ahsan, H. (2023) · Pharmaceutics · doi:10.3390/pharmaceutics15061630 · Cited by 386
Metadata
- Authors
- Ray, P., Manach, Y. L., Riou, B., Houle, T. T., & Warner, D. S.
- Venue
- Anesthesiology
- Published
- 2010-04-01
- Volume
- 112
- Issue
- 4
- Pages
- 1023-1040
- ISSN
- 0003-3022 (ISSN-L)
- OpenAlex
- W2080564894
- Cited by
- 503
- References
- 88
- FWCI
- 22.62
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.