The economics of ‘more research is needed’
The DOI matches its Crossref record; 2 of the 2 references it cites were found in a registry or an index.
Abstract
BACKGROUND: Results from epidemiology and other health research affect millions of life-years and billions of dollars, and the research directly consumes millions of dollars. Yet we do little to assess the value of research projects for future policy, even amid the ubiquitous assertions that 'more research is necessary' on a given topic. This methodological proposal outlines the arguments for why and how ex ante assessments can inform us about the value of a particular piece of further research on a topic. METHODS: Economics and decision theory concepts-cost-benefit analysis and probability-weighted predictions of outcomes-allow us to calculate the payoff from applied health research based on resulting decisions. Starting with our probability distribution for the parameters of interest, a Monte Carlo simulation generates the distribution of outcomes from a particular new study. Each true value and outcome are associated with a policy decision, and improved decisions are valued to give us the study's contribution as applied research. RESULTS: The analysis demonstrates how to calculate the expected value of further research, for a simplified case, and assess whether it is really warranted. Perhaps more important, it points out what the measure of the value of a further study ought to be. CONCLUSIONS: It is quite possible to improve our technology for assessing the value of particular pieces of further research on a topic. However, this will only happen if the need and possibility are recognized by methodologists and applied researchers.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Probabilistic Sensitivity Analysis Using Monte Carlo Simulation: A Practical Approach
Doubilet, P., Begg, C. B., Weinstein, M. C., Braun, P., & McNeil, B. J. (1985) · Medical Decision Making · doi:10.1177/0272989x8500500205 · Cited by 860
Berger, J. O., & Wolpert, R. L. (1988) · doi:10.1214/lnms/1215466210 · Cited by 616
Research works printed but not linked: 7
- Gold MR, Siegel JE, Russell LB, Weinstein MC (eds). Cost-Effectiveness in Health and Medicine. New York: Oxford University Press, 1996. · DOI · Paper page
- Backhouse ME. An investment appraisal approach to clinical trial design. Health Econ1998;7:605–19. · doi:10.1002/(sici)1099-1050(1998110)7:7<605::aid-hec375>3.0.co;2-9
- Claxton K, Posnett J. An economic approach to clinical trial design and research priority-setting. Health Econ1996;5:513–24. · DOI · Paper page
- Arrow KJ, Fisher AC. Preservation, uncertainty and irreversibility. Quart J Econ1974;87:312–19. · DOI · Paper page
- Phillips CV, Maldonado G. Using Monte Carlo methods to quantify the multiple sources of error in studies. Am J Epidemiol1999; 149:S17.
- Manning WG, Fryback DG, Weinstein MC. Reflecting uncertainty in cost-effectiveness analysis. In: Gold MR, Siegel JE, Russell LB, Weinstein MC (eds). Cost-Effectiveness in Health and Medicine. New York: Oxford University Press, 1996.
- Salmon WC. The Foundations of Scientific Inference. Pittsburgh: University of Pittsburgh Press, 1966.
Cited by
Vandenbroucke, J. P., von Elm, E., Altman, D. G., Gøtzsche, P. C., Mulrow, C. D., Pocock, S. J., Poole, C., Schlesselman, J. J., Egger, M., & Initiative, F. T. S. (2007) · PLoS Medicine · doi:10.1371/journal.pmed.0040297 · Cited by 8,613
“Investigators should suggest ways in which subsequent research can improve on their studies rather than blandly stating ‘ more research is needed ’ [197,198].”
Vandenbroucke, J. P., Elm, E. V., Altman, D. G., Gøtzsche, P. C., Mulrow, C. D., Pocock, S. J., Poole, C., Schlesselman, J. J., Egger, M., & initiative, F. T. S. (2007) · Annals of Internal Medicine · doi:10.7326/0003-4819-147-8-200710160-00010-w1 · Cited by 3,543
Vandenbroucke, J. P., von Elm, E., Altman, D. G., Gøtzsche, P. C., Mulrow, C. D., Pocock, S. J., Poole, C., Schlesselman, J. J., & Egger, M. (2014) · International Journal of Surgery · doi:10.1016/j.ijsu.2014.07.014 · Cited by 2,969
Vandenbroucke, J. P., von Elm, E., Altman, D. G., Gøtzsche, P. C., Mulrow, C. D., Pocock, S. J., Poole, C., Schlesselman, J. J., & Egger, M. (2007) · Epidemiology · doi:10.1097/ede.0b013e3181577511 · Cited by 2,779
Multiple-bias modelling for analysis of observational data
Greenland, S. (2005) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.1111/j.1467-985x.2004.00349.x · Cited by 497
Metadata
- Authors
- Phillips, C. V.
- Venue
- International Journal of Epidemiology
- Publisher
- Oxford University Press (OUP)
- Published
- 2001-08
- Volume
- 30
- Issue
- 4
- Pages
- 771-776
- ISSN
- 0300-5771 (ISSN-L)
- 1464-3685
- OpenAlex
- W2118310001
- Cited by
- 66
- References
- 2
- FWCI
- 8.05
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.