Emerging Use of Early Health Technology Assessment in Medical Product Development: A Scoping Review of the Literature
Open accessThe DOI matches its Crossref record; 67 of the 67 references it cites were found in a registry or an index.
Abstract
Early health technology assessment is increasingly being used to support health economic evidence development during early stages of clinical research. Such early models can be used to inform research and development about the design and management of new medical technologies to mitigate the risks, perceived by industry and the public sector, associated with market access and reimbursement. Over the past 25 years it has been suggested that health economic evaluation in the early stages may benefit the development and diffusion of medical products. Early health technology assessment has been suggested in the context of iterative economic evaluation alongside phase I and II clinical research to inform clinical trial design, market access, and pricing. In addition, performing early health technology assessment was also proposed at an even earlier stage for managing technology portfolios. This scoping review suggests a generally accepted definition of early health technology assessment to be "all methods used to inform industry and other stakeholders about the potential value of new medical products in development, including methods to quantify and manage uncertainty". The present review also aimed to identify recent published empirical studies employing an early-stage assessment of a medical product. With most included studies carried out to support a market launch, the dominant methodology was early health economic modeling. Further methodological development is required, in particular, by combining systems engineering and health economics to manage uncertainty in medical product portfolios.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Thokala, P., Devlin, N., Marsh, K., Baltussen, R., Boysen, M., Kalo, Z., Longrenn, T., Mussen, F., Peacock, S., Watkins, J., et al. (2016) · Value in Health · doi:10.1016/j.jval.2015.12.003 · Cited by 739
Self-citationThe Effects of Cold and Lower Body Negative Pressure on Cardiovascular Homeostasis
Kean, D. J., Peacock, C. A., Sanders, G. J., McDaniel, J., Colvin, L. A. C., & Glickman, E. L. (2015) · BioMed Research International · doi:10.1155/2015/728145 · Cited by 492
Characterizing treatment pathways at scale using the OHDSI network
Hripcsak, G., Ryan, P. B., Duke, J. D., Shah, N. H., Park, R. W., Huser, V., Suchard, M. A., Schuemie, M. J., DeFalco, F. J., Perotte, A., et al. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1510502113 · Cited by 326 · cited 1×
“For instance, a recent paper analyzing approximately 250 million patients identified that between 10 and 25% of the patients follow a completely individualized drug treatment pathway [59].”
BackgroundO'Hagan, A. (1998) · Journal of the Royal Statistical Society: Series D (The Statistician) · doi:10.1111/1467-9884.00114 · Cited by 309
Development of a Glucocorticoid Toxicity Index (GTI) using multicriteria decision analysis
Miloslavsky, E. M., Naden, R. P., Bijlsma, J. W. J., Brogan, P. A., Brown, E. S., Brunetta, P., Buttgereit, F., Choi, H. K., DiCaire, J.-F., Gelfand, J. M., et al. (2017) · Annals of the Rheumatic Diseases · doi:10.1136/annrheumdis-2016-210002 · Cited by 230 · cited 1×
“Therefore, alternative methods to measure value have been explored and used such as discrete-choice experiments and MCDA and it is quite easy to see how such preference elicitation models could be used to estimate the relative value of (hypothetical) new technologies [56], or how they can be used to support R&D teams…”
MethodMarshall, D. A., Burgos-Liz, L., IJzerman, M. J., Osgood, N. D., Padula, W. V., Higashi, M. K., Wong, P. K., Pasupathy, K. S., & Crown, W. (2015) · Value in Health · doi:10.1016/j.jval.2014.12.001 · Cited by 221
Self-citationThe Iterative Use of Economic Evaluation as Part of the Process of Health Technology Assessment
Sculpher, M., Drummond, M., & Buxton, M. (1997) · Journal of Health Services Research & Policy · doi:10.1177/135581969700200107 · Cited by 208 · cited 3×
“based healthcare, [2] the dynamics of healthcare pathways, and [3] the personalization of treatment.”
BackgroundLinley, W. G., & Hughes, D. A. (2013) · Health Economics · doi:10.1002/hec.2872 · Cited by 208
(2016) · ecancermedicalscience · doi:10.3332/ecancer.2014.464 · Cited by 192
IJzerman, M. J., & Steuten, L. M. G. (2011) · Applied Health Economics and Health Policy · doi:10.2165/11593380-000000000-00000 · Cited by 184
Self-citationHartmann, M., & Hassan, A. (2006) · Research Policy · doi:10.1016/j.respol.2005.12.005 · Cited by 163 · cited 1×
“Incorporation of uncertainties over future outcomes and developmental uncertainty is close to real-options analysis, where future developmental scenarios are both identified and quantified as options-value [28].”
BackgroundMarshall, D. A., Burgos-Liz, L., IJzerman, M. J., Crown, W. H., Padula, W. V., Wong, P. K., Pasupathy, K. S., Higashi, M. K., & Osgood, N. D. (2015) · Value in Health · doi:10.1016/j.jval.2015.01.006 · Cited by 160
Self-citationPotential Application of Machine Learning in Health Outcomes Research and Some Statistical Cautions
Crown, W. H. (2015) · Value in Health · doi:10.1016/j.jval.2014.12.005 · Cited by 125 · cited 1×
“In this regard, several computational approaches are used and explored to mine large data that may also be used in early HTA [58].”
MethodAngelis, A., & Kanavos, P. G. (2016) · PharmacoEconomics · doi:10.1007/s40273-015-0370-z · Cited by 120
Vallejo‐Torres, L., Steuten, L., Buxton, M. J., Girling, A. J., Lilford, R. J., & Young, T. P. (2008) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462308080604 · Cited by 112 · cited 1×
“[11], Pietzsch and PatéCornell [12], and IJzerman and Steuten [13].”
BackgroundHartz, S., & John, J. (2007) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462308080616 · Cited by 110 · cited 7×
“Sculpher, 1997 [2], (CI: 21,05) Terrés, 1998 [4], (CI: 0,89) 20102000 Pharma R&D Devices R&D 20051995 30 23 103 13 26 Grabowski, 1997 [3], (CI: 2,25) Mauskopf, 1996 [1], (CI: 1,52) Annemans, 2000 [16], (CI: 0,60) 24 Hartz&John, 2008 [8], (CI: 1,00) 20 Dong, 2006 [5], (CI: 3,64) 50 IJzerman, 2011 [13], (CI: 2,45) 35 Ma…”
BackgroundInfluentialDong, H., & Buxton, M. J. (2006) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462306051014 · Cited by 105 · cited 2×
“Dong and Buxton published an early-stage Markov model with a probabilistic sensitivity analysis for a knee replacement procedure [5], van Til et al.”
BackgroundReporting Guidelines for the Use of Expert Judgement in Model-Based Economic Evaluations
Iglesias, C. P., Thompson, A., Rogowski, W. H., & Payne, K. (2016) · PharmacoEconomics · doi:10.1007/s40273-016-0425-9 · Cited by 96
McAteer, H., Cosh, E., Freeman, G. H., Pandit, A. S., Wood, P., & Lilford, R. J. (2007) · Journal of Tissue Engineering and Regenerative Medicine · doi:10.1002/term.36 · Cited by 92 · cited 1×
“in the case of tissue engineering, and three more recent publications presented different cases using headroom [23, 27].”
BackgroundMEDICAL DEVICES EARLY ASSESSMENT METHODS: SYSTEMATIC LITERATURE REVIEW
Markiewicz, K., van Til, J. A., & IJzerman, M. J. (2014) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462314000026 · Cited by 87 · cited 3×
“Markiewicz, 2014 [21], (CI: 1,24) 12 Miller, 2005 [10], (CI: 1,16) 27”
BackgroundSelf-citationInvesting in new medical technologies: A decision framework
Cosh, E., Girling, A. J., Lilford, R. J., McAteer, H., & Young, T. P. (2007) · Journal of Commercial Biotechnology · doi:10.1057/palgrave.jcb.3050062 · Cited by 86 · cited 3×
“The method was first introduced by Cosh et al. and McAteer et al. in the case of tissue engineering, and three more recent publications presented different cases using headroom [23, 27].”
BackgroundMethodOrphan drugs policies: a suitable case for treatment
Drummond, M., & Towse, A. (2014) · The European Journal of Health Economics · doi:10.1007/s10198-014-0560-1 · Cited by 84
Campos, K. D. P., Norman, C. D., & Jadad, A. R. (2011) · Social Science & Medicine · doi:10.1016/j.socscimed.2011.06.059 · Cited by 81 · cited 1×
“Although early assessment may support revenue maximization decisions in companies, it clearly emerged from a societal need to justify public investments in R&Das a result of the growing number of public-private partnerships [19].”
BackgroundRole of Pharmacoeconomic Analysis in R&D Decision Making: When, Where, How?
Miller, P. (2005) · PharmacoEconomics · doi:10.2165/00019053-200523010-00001 · Cited by 80 · cited 1×
“…0,60) 24 Hartz&John, 2008 [8], (CI: 1,00) 20 Dong, 2006 [5], (CI: 3,64) 50 IJzerman, 2011 [13], (CI: 2,45) 35 Markiewicz, 2014 [21], (CI: 1,24) 12 Miller, 2005 [10], (CI: 1,16) 27 Fenwick, 2006 [18], (CI: 2,71) DiMasi, 2001 [17], (CI: 2,72) 42 Van Til, 2006 [6], (CI: 2,65) 15 Conceptual /…”
Early technology assessment of new medical devices
Pietzsch, J. B., & Paté-Cornell, M. E. (2008) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462307080051 · Cited by 78 · cited 4×
“According to Pietzsch and Paté-Cornell, early HTA is used to inform manufacturers and investors about the design and management of a technology, as well as regulatory and reimbursement strategy [12].”
BackgroundMethodInfluentialEmerging Role of Pharmacoeconomics in the Research and Development Decision-Making Process
DiMasi, J. A., Caglarcan, E., & Wood-Armany, M. (2001) · PharmacoEconomics · doi:10.2165/00019053-200119070-00004 · Cited by 67
Fenwick, E. A. L., Palmer, S., Claxton, K., Sculpher, M., Abrams, K. R., & Sutton, A. J. (2006) · Medical Decision Making · doi:10.1177/0272989x06290493 · Cited by 66 · cited 3×
“…(CI: 1,00) 20 Dong, 2006 [5], (CI: 3,64) 50 IJzerman, 2011 [13], (CI: 2,45) 35 Markiewicz, 2014 [21], (CI: 1,24) 12 Miller, 2005 [10], (CI: 1,16) 27 Fenwick, 2006 [18], (CI: 2,71) DiMasi, 2001 [17], (CI: 2,72) 42 Van Til, 2006 [6], (CI: 2,65) 15 Conceptual / editorials / opinion Early empirical…”
BackgroundSelf-citationEarly Modelling for Assessing Health and Economic Outcomes of Drug Therapy
Annemans, L., Genesté, B., & Jolain, B. (2000) · Value in Health · doi:10.1046/j.1524-4733.2000.36007.x · Cited by 63 · cited 1×
“The figure shows that the initial papers were published from the mid-1990s until 2001 and proposed the collection of health economic evidence alongside clinical trials in stage I and II [1, 3, 4, 9, 16, 17].”
BackgroundThe role of health technology assessment bodies in shaping drug development
Ciani, O., & Jommi, C. (2014) · Drug Design Development and Therapy · doi:10.2147/dddt.s49935 · Cited by 61 · cited 1×
“The search was meant to be an update of previous reviews and thus limited to the time frame between September 2013 and April 2017 because the previous 2006–13 time window was well covered by other reviews [8, 20, 21].”
BackgroundHEADROOM APPROACH TO DEVICE DEVELOPMENT: CURRENT AND FUTURE DIRECTIONS
Girling, A. J., Lilford, R. J., Cole, A., & Young, T. P. (2015) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462315000501 · Cited by 57
Big Data and Health Economics: Strengths, Weaknesses, Opportunities and Threats
Collins, B. (2015) · PharmacoEconomics · doi:10.1007/s40273-015-0306-7 · Cited by 53
A Strategy for Collecting Pharmacoeconomic Data During Phase II/III Clinical Trials
Mauskopf, J., Schulman, K. A., Bell, L., & Glick, H. A. (1996) · PharmacoEconomics · doi:10.2165/00019053-199609030-00007 · Cited by 51 · cited 5×
“[1, 5, 9], Miller was the first to broaden the scope [10], followed by Vallejo-Torres et al.”
BackgroundInfluentialTechnology foresight for medical device development through hybrid simulation: The ProHTA Project
Kolominsky‐Rabas, P. L., Djanatliev, A., Wahlster, P., Gantner-Bär, M., Hofmann, B., German, R., Sedlmayr, M., Reinhardt, E. R., Schüttler, J., & Kriza, C. (2014) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2013.12.005 · Cited by 50
Ryan, M., Kinghorn, P., Entwistle, V. A., & Francis, J. (2014) · Social Science & Medicine · doi:10.1016/j.socscimed.2014.01.013 · Cited by 49
de Windt, T. S., Sorel, J. C., Vonk, L. A., Kip, M. M. A., IJzerman, M. J., & Saris, D. B. F. (2016) · Journal of Tissue Engineering and Regenerative Medicine · doi:10.1002/term.2197 · Cited by 49
Self-citationDegeling, K., Koffijberg, H., & IJzerman, M. J. (2016) · Expert Review of Pharmacoeconomics & Outcomes Research · doi:10.1080/14737167.2017.1273110 · Cited by 43 · cited 1×
“, changes in response rates over time), often an important feature of personalized medicine [64, 65].”
BackgroundSelf-citationCommercial viability of medical devices using Headroom and return on investment calculation
Markiewicz, K., van Til, J. A., Steuten, L., & IJzerman, M. J. (2016) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2016.07.041 · Cited by 40
Self-citationProspective cost-effectiveness analysis of genomic profiling in breast cancer
Retèl, V. P., Joore, M. A., Drukker, C. A., Bueno-de-Mesquita, J. M., Knauer, M., van Tinteren, H., Linn, S., & van Harten, W. H. (2013) · European Journal of Cancer · doi:10.1016/j.ejca.2013.08.001 · Cited by 39
Public stated preferences and predicted uptake for genome-based colorectal cancer screening
Groothuis‐Oudshoorn, C. G. M., Fermont, J. M., van Til, J. A., & IJzerman, M. J. (2014) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-14-18 · Cited by 36
Self-citationZibo, L., Zeng, Z.-J., Chen, Q., Luo, S., & Hu, W. (2006) · BMC Cancer · doi:10.1186/1471-2407-6-66 · Cited by 34
Cao, Q., Postmus, D., Hillege, H. L., & Buskens, E. (2013) · Value in Health · doi:10.1016/j.jval.2013.02.008 · Cited by 34 · cited 3×
“[47] Point-of-care-test at home/cardiology Market launch Yes, probability distribution Markov model Yes, probability elicitation (N = 3) Monte Carlo simulation, distributions”
BackgroundMethodInfluentialReview on early technology assessments of nanotechnologies in oncology
Retèl, V. P., Hummel, M. J. M., & van Harten, W. H. (2009) · Molecular Oncology · doi:10.1016/j.molonc.2009.05.001 · Cited by 33 · cited 3×
“Conceptual / editorials / opinion Early empirical studies (systema c) Reviews Retèl, 2009 [22], (CI: 0,54) 21”
BackgroundInfluentialStrategic Use of Pharmacoeconomic Research in Early Drug Development and Global Pricing
Clemens, K., Garrison, L. P., Jones, A., & Macdonald, F. (1993) · PharmacoEconomics · doi:10.2165/00019053-199304050-00002 · Cited by 32
Early economic evaluation of emerging health technologies: protocol of a systematic review
Pham, B., Tu, H., Han, D., Pechlivanoglou, P., Miller, F. A., Rac, V. E., Chin, W., Tricco, A. C., Paulden, M., BIELECKI, J. M., et al. (2014) · Systematic Reviews · doi:10.1186/2046-4053-3-81 · Cited by 32
Self-citation(Very) Early technology assessment and translation of predictive biomarkers in breast cancer
Miquel-Cases, A., Schouten, P. C., Steuten, L., Retèl, V. P., Linn, S. C., & van Harten, W. H. (2016) · Cancer Treatment Reviews · doi:10.1016/j.ctrv.2016.11.008 · Cited by 32 · cited 1×
“For instance, the speedy discovery of new biomarkers and the appropriate targeting of treatment poses a challenge for evidence generation and thus an interesting field for early HTA [63].”
BackgroundStewart, D. J., Batist, G., Kantarjian, H. M., Bradford, J., Schiller, J. H., & Kurzrock, R. (2015) · Clinical Cancer Research · doi:10.1158/1078-0432.ccr-14-3246 · Cited by 32
Belief Elicitation to Populate Health Economic Models of Medical Diagnostic Devices in Development
Haakma, W., Steuten, L., Bojke, L., & IJzerman, M. J. (2014) · Applied Health Economics and Health Policy · doi:10.1007/s40258-014-0092-y · Cited by 31 · cited 1×
“used belief elicitation to estimate the likely diagnostic performance of a new imaging device [44].”
MethodSelf-citationHuygens, S. A., Mölken, M. R., Bekkers, J. A., Bogers, A. J. J. C., Bouten, C. V. C., Chamuleau, S. A. J., de Jaegere, P. P. T., Kappetein, A. P., Kluin, J., Mieghem, N. M. V., et al. (2016) · Open Heart · doi:10.1136/openhrt-2016-000500 · Cited by 29
Jönsson, B. (2014) · Molecular Oncology · doi:10.1016/j.molonc.2014.10.009 · Cited by 29
Hjelmgren, J., Ghatnekar, O., Reimer, J., Grabowski, M., Lindvall, O., Persson, U., & Hagell, P. (2006) · Parkinsonism & Related Disorders · doi:10.1016/j.parkreldis.2006.04.006 · Cited by 28
Scenario drafting for early technology assessment of next generation sequencing in clinical oncology
Joosten, S. E. P., Retèl, V. P., Coupé, V. M. H., van den Heuvel, M. M., & van Harten, W. H. (2016) · BMC Cancer · doi:10.1186/s12885-016-2100-0 · Cited by 28
Middelkamp, H. H. T., van der Meer, A. D., Hummel, J. M., Stamatialis, D. F., Mummery, C. L., Passier, R. C., & IJzerman, M. J. (2016) · Applied In Vitro Toxicology · doi:10.1089/aivt.2015.0029 · Cited by 27
Self-citationAre current ICER thresholds outdated? Valuing medicines in the era of personalized healthcare
Carrera, P. M., & IJzerman, M. J. (2016) · Expert Review of Pharmacoeconomics & Outcomes Research · doi:10.1080/14737167.2016.1180980 · Cited by 27
Self-citationKip, M. M. A., Steuten, L., Koffijberg, H., IJzerman, M. J., & Kusters, R. (2016) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.12626 · Cited by 27 · cited 1×
“[45] Triple biomarker to exclude nSTEMI leading to early discharge Technology available but not used Yes Decision tree Yes, elicitation of clinical utility of test (N = 10) One-way (probabilistic) sensitivity analysis with Tornado diagram”
MethodSelf-citationValue of Research and Value of Development in Early Assessments of New Medical Technologies
Retèl, V. P., Grutters, J. P. C., van Harten, W. H., & Joore, M. A. (2013) · Value in Health · doi:10.1016/j.jval.2013.04.013 · Cited by 25 · cited 1×
“[29] 70-gene expression profile: development of paraffin based test R&D decision to develop 70GPAR No Markov model No, assumptions made based on reference data Simulations and VOI analysis to determine value of further development and research”
MethodImproving early cycle economic evaluation of diagnostic technologies
Steuten, L., & Ramsey, S. D. (2014) · Expert Review of Pharmacoeconomics & Outcomes Research · doi:10.1586/14737167.2014.914435 · Cited by 24
van Til, J. A., Renzenbrink, G. J., Groothuis‐Oudshoorn, C. G. M., & IJzerman, M. J. (2006) · Disability and Rehabilitation · doi:10.1080/09638280500277057 · Cited by 22 · cited 1×
“published an early-stage model on the use a neural prosthesis for post-stroke shoulder pain [6], and Hjelmgren et al.”
BackgroundSelf-citationEarly Evaluation and Value-Based Pricing of Regenerative Medicine Technologies
Koerber, F., Rolauffs, B., & Rogowski, W. H. (2013) · Regenerative Medicine · doi:10.2217/rme.13.69 · Cited by 22
Kolominsky‐Rabas, P. L., Kriza, C., Djanatliev, A., Meier, F., Uffenorde, S., Radeleff, J., Baumgärtel, P., Leb, I., Sedlmayr, M., Gaiser, S., et al. (2016) · Telemedicine Journal and e-Health · doi:10.1089/tmj.2015.0226 · Cited by 16
Levin, L. (2015) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462315000422 · Cited by 14 · cited 2×
“[15] Emerging Use of Early Health Technology Assessment in Medical Product Development 737”
BackgroundIJzerman, M. J., Manca, A., Keizer, J., & Ramsey, S. D. (2015) · Comparative Effectiveness Research · doi:10.2147/cer.s92212 · Cited by 13
Self-citationCraven, M. P., Allsop, M. J., Morgan, S. P., & Martin, J. L. (2012) · Health Research Policy and Systems · doi:10.1186/1478-4505-10-29 · Cited by 13 · cited 1×
“participants (mostly company directors or managers, including product or project managers) rated themselves as having either low or no knowledge of health economics prior to the workshops with the remainder professing at least average knowledge [50].”
BackgroundRanking medical innovations according to perceived health benefit
Fermont, J. M., Douw, K., Vondeling, H., & IJzerman, M. J. (2016) · Health Policy and Technology · doi:10.1016/j.hlpt.2016.02.009 · Cited by 13
Self-citationEarly decision-analytic modeling – a case study on vascular closure devices
Brandes, A., Sinner, M. F., Kääb, S., & Rogowski, W. H. (2015) · BMC Health Services Research · doi:10.1186/s12913-015-1118-3 · Cited by 12
Miquel-Cases, A., Steuten, L., Retèl, V. P., & van Harten, W. H. (2015) · The Breast · doi:10.1016/j.breast.2015.03.002 · Cited by 12 · cited 1×
“[32] BRCAl-like test to detect triple negative breast cancer Inform clinicians and R&D about effect of BRCA1 testing No Markov model No, based on clinical trial data One-way deterministic sensitivity, threshold and probabilistic analysis (CEAC)”
Methodde Graaf, G., Postmus, D., & Buskens, E. (2015) · BioMed Research International · doi:10.1155/2015/191809 · Cited by 11
van Harten, W., & Retèl, V. P. (2016) · ecancermedicalscience · doi:10.3332/ecancer.2016.683 · Cited by 10
Cited by
Biomedical application of optical fibre sensors
Correia, R., James, S., Lee, S.-W., Morgan, S. P., & Korposh, S. (2018) · Journal of Optics · doi:10.1088/2040-8986/aac68d · Cited by 206
Metadata
- Authors
- IJzerman, M. J., Koffijberg, H., Fenwick, E., & Krahn, M.
- Venue
- PharmacoEconomics
- Published
- 2017-04-21
- Volume
- 35
- Issue
- 7
- Pages
- 727-740
- ISSN
- 1170-7690 (ISSN-L)
- 1179-2027
- OpenAlex
- W2607056793
- Cited by
- 189
- References
- 67
- FWCI
- 25.88
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.