Publication bias in clinical trials of electronic health records
Open accessAbstract
Objective To measure the rate of non-publication and assess possible publication bias in clinical trials of electronic health records.Methods We searched ClinicalTrials.gov to identify registered clinical trials of electronic health records and searched the biomedical literature and contacted trial investigators to determine whether the results of the trials were published. Publications were judged as positive, negative, or neutral according to the primary outcome.Results Seventy-six percent of trials had publications describing trial results; of these, 74% were positive, 21% were neutral, and 4% were negative (harmful). Of unpublished studies for which the investigator responded, 43% were positive, 57% were neutral, and none were negative; the lower rate of positive results was significant (p<0.001).Conclusion The rate of non-publication in electronic health record studies is similar to that in other biomedical studies. There appears to be a bias toward publication of positive trials in this domain.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Clinical trial registries have been used to measure publication bias and outcome reporting bias.
“Clinical trial registries are an important source of information for tracking clinical trials from their inception through to their reporting, and have been used to measure publication bias and outcome reporting bias.”
From Systematic review protocol assessing the processes for linking clinical trial registries and their published results (2016) · cites this paper as [10]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Selective publication of antidepressant trials and its influence on apparent efficacy
Turner, E. H., Matthews, A. M., Linardatos, E., Tell, R. A., & Rosenthal, R. (2008) · New England Journal of Medicine · doi:10.1056/nejmsa065779 · Cited by 2,435 · see citing works
Role of Computerized Physician Order Entry Systems in Facilitating Medication Errors
Koppel, R. (2005) · JAMA · doi:10.1001/jama.293.10.1197 · Cited by 2,126 · see citing works
Buntin, M. B., Burke, M. F., Hoaglin, M. C., & Blumenthal, D. (2011) · Health Affairs · doi:10.1377/hlthaff.2011.0178 · Cited by 1,311 · see citing works
Clinical Trial Registration: A Statement from the International Committee of Medical Journal Editors
De Angelis, C., Drazen, J. M., Frizelle, F. A., Haug, C., Hoey, J., Horton, R., Kotzin, S., Laine, C., Marusic, A., Overbeke, A. J. P. M., et al. (2004) · New England Journal of Medicine · doi:10.1056/nejme048225 · Cited by 1,241 · see citing works
Han, Y. Y., Carcillo, J. A., Venkataraman, S. T., Clark, R. S. B., Watson, R. S., Nguyen, T. C., Bayir, H., & Orr, R. A. (2005) · Pediatrics · doi:10.1542/peds.2005-1287 · Cited by 1,149 · see citing works
Publication bias: evidence of delayed publication in a cohort study of clinical research projects
Stern, J. M., & Simes, R. J. (1997) · BMJ · doi:10.1136/bmj.315.7109.640 · Cited by 818 · see citing works
Comparison of Registered and Published Primary Outcomes in Randomized Controlled Trials
Mathieu, S. (2009) · JAMA · doi:10.1001/jama.2009.1242 · Cited by 730 · see citing works
Evaluation Methods in Medical Informatics
Friedman, C. P., & Wyatt, J. C. (1997) · Computers and medicine · doi:10.1007/978-1-4757-2685-5 · Cited by 450 · see citing works
Munhoz, E. (2006) · Epidemiologia e Serviços de Saúde · doi:10.5123/s1679-49742006000100002 · Cited by 417 · see citing works
International Committee of Medical Journal Editors.
Riis, P. J. (1990) · PubMed · Cited by 415 · see citing works
Clinical trial registration: a statement from the International Committee of Medical Journal Editors
De Angelis, C., Drazen, J. M., Frizelle, F. A., Haug, C., Hoey, J., Horton, R., Kotzin, S., Laine, C., Marusic, A., Overbeke, A. J. P., et al. (2004) · The Lancet · doi:10.1016/s0140-6736(04)17034-7 · Cited by 408 · see citing works
Trial Publication after Registration in ClinicalTrials.Gov: A Cross-Sectional Analysis
Ross, J. S., Mulvey, G. K., Hines, E. M., Nissen, S. E., & Krumholz, H. M. (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000144 · Cited by 401 · see citing works
Publication Bias in Editorial Decision Making
Olson, C. M. (2002) · JAMA · doi:10.1001/jama.287.21.2825 · Cited by 395 · see citing works
Outcome Reporting Among Drug Trials Registered in ClinicalTrials.gov
Bourgeois, F. T., Murthy, S., & Mandl, K. D. (2010) · Annals of Internal Medicine · doi:10.7326/0003-4819-153-3-201008030-00006 · Cited by 332 · see citing works
Frequency and reasons for outcome reporting bias in clinical trials: interviews with trialists
Smyth, R. M. D., Kirkham, J. J., Jacoby, A., Altman, D. G., Gamble, C., & Williamson, P. R. (2011) · BMJ · doi:10.1136/bmj.c7153 · Cited by 220 · see citing works
Bloomrosen, M., Starren, J. B., Lorenzi, N. M., Ash, J. S., Patel, V. L., & Shortliffe, E. H. (2010) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.2010.007567 · Cited by 181 · see citing works
Reporting “Basic Results” in ClinicalTrials.gov
Tse, T., Williams, R. J., & Zarin, D. A. (2009) · CHEST Journal · doi:10.1378/chest.08-3022 · Cited by 140 · see citing works
The International Committee of Medical Journal Editors
van der Weyden, M. V. (2006) · doi:10.2349/biij.2.4.e54-13 · Cited by 110 · see citing works
Rasmussen, N., Lee, K. P., & Bero, L. A. (2009) · Trials · doi:10.1186/1745-6215-10-116 · Cited by 62 · see citing works
A clinical trials register for Europe
Tonks, A. (2002) · BMJ · doi:10.1136/bmj.325.7376.1314 · Cited by 46 · see citing works
Ammenwerth, E., & de Keizer, N. F. (2007) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2276 · Cited by 31 · see citing works
Evaluation Methods in Biomedical Informatics (Health Informatics)
Friedman, C. P., & Wyatt, J. C. (2005) · Springer eBooks · Cited by 28 · see citing works
Testing Informatics Innovations: The Value of Negative Trials
Tierney, W. M., & McDonald, C. J. (1996) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.1996.97035027 · Cited by 19 · see citing works
Publication Bias in Medical Informatics
Friedman, C. P., & Wyatt, J. C. (2001) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.2001.0080189 · Cited by 18 · see citing works
Publication bias in medical informatics evaluation research: is it an issue or not?
Machan, C., Ammenwerth, E., & Bodner, T. (2006) · PubMed · Cited by 12 · see citing works
Clinical trial registries and clinical trial result posting: new paradigm for medical writers
Foote, M. (2006) · Biotechnology annual review · doi:10.1016/s1387-2656(06)12011-6 · Cited by 1 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Schmucker, C., Schell, L. K., Portalupi, S., Oeller, P., Cabrera, L., Bassler, D., Schwarzer, G., Scherer, R. W., Antes, G., von Elm, E., et al. (2014) · PLoS ONE · doi:10.1371/journal.pone.0114023 · Cited by 269 · see citing works
Patient engagement in the inpatient setting: a systematic review
Prey, J. E., Woollen, J., Wilcox, L. G., Sackeim, A., Hripcsak, G. M., Henry, S. B., Restaino, S. W., Feiner, S., & Vawdrey, D. K. (2013) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2013-002141 · Cited by 265 · see citing works
Self-citationBirth month affects lifetime disease risk: a phenome-wide method
Boland, M. R., Shahn, Z., Madigan, D., Hripcsak, G. M., & Tatonetti, N. P. (2015) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocv046 · Cited by 139 · see citing works
Self-citationEffectiveness of Digital Technologies to Support Nursing Care: Results of a Scoping Review
Hüter, K., Krick, T., Domhoff, D., Seibert, K., Wolf‐Ostermann, K., & Rothgang, H. (2020) · Journal of Multidisciplinary Healthcare · doi:10.2147/jmdh.s286193 · Cited by 125 · see citing works
Song, F., Loke, Y. K., & Hooper, L. (2014) · PLoS ONE · doi:10.1371/journal.pone.0110418 · Cited by 117 · see citing works
Linking ClinicalTrials.gov and PubMed to Track Results of Interventional Human Clinical Trials
Huser, V., & Cimino, J. J. (2013) · PLoS ONE · doi:10.1371/journal.pone.0068409 · Cited by 90 · see citing works
Fung, M. E., Yuan, Y., Atkins, H. L., Shi, Q., & Bubela, T. M. (2017) · Stem Cell Reports · doi:10.1016/j.stemcr.2017.03.013 · Cited by 78 · see citing works
Bashir, R., Bourgeois, F. T., & Dunn, A. G. (2017) · Systematic Reviews · doi:10.1186/s13643-017-0518-3 · Cited by 47 · see citing works
Ayorinde, A., Williams, I., Mannion, R., Song, F., Skrybant, M., Lilford, R. J., & Chen, Y. (2020) · BMC Medical Research Methodology · doi:10.1186/s12874-020-01010-1 · Cited by 45 · see citing works
Schmidtke, K. A., Nightingale, P., Reeves, K., Gallier, S., Vlaev, I., Watson, S., & Lilford, R. J. (2019) · BMJ Quality & Safety · doi:10.1136/bmjqs-2019-009775 · Cited by 33 · see citing works
Time to publication for results of clinical trials
Showell, M. G., Cole, S., Clarke, M. J., DeVito, N., Farquhar, C. M., & Jordan, V. (2024) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000011.pub3 · Cited by 33 · see citing works
Canestaro, W. J., Hendrix, N., Bansal, A., Sullivan, S. D., Devine, E. B., & Carlson, J. J. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.08.004 · Cited by 27 · see citing works
Martin, P. B., Surian, D., Bashir, R., Bourgeois, F. T., & Dunn, A. G. (2019) · JAMIA Open · doi:10.1093/jamiaopen/ooy062 · Cited by 24 · see citing works
Asynchronous automated electronic laboratory result notifications: a systematic review
Slovis, B. H., NAHASS, T. A., Salmasian, H., Kuperman, G. J., & Vawdrey, D. K. (2017) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocx047 · Cited by 24 · see citing works
Self-citationElectronic Health Record: A review
Graña, M., & Jackwoski, K. (2015) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm.2015.7359879 · Cited by 19 · see citing works
Thomsen, P. T. (2015) · Journal of Dairy Science · doi:10.3168/jds.2014-9135 · Cited by 15 · see citing works
Massoudi, B. L., & Chester, K. (2017) · Yearbook of Medical Informatics · doi:10.1055/s-0037-1606510 · Cited by 15 · see citing works
Effectiveness of digital technologies to support nursing care: results of a scoping reviewPreprint
Hüter, K., Krick, T., Domhoff, D., Seibert, K., Wolf‐Ostermann, K., & Rothgang, H. (2020) · Research Square · doi:10.21203/rs.2.24344/v1 · Cited by 14 · see citing works
Bashir, R., & Dunn, A. G. (2016) · BMJ Open · doi:10.1136/bmjopen-2016-013048 · Cited by 10 · see citing works
Wittich, L., Rödiger, H., Rombey, T., Brecher, A.-L., Kraft, L., Sgraja, S., Stein, V. K., & Henschke, C. (2026) · Implementation Science Communications · doi:10.1186/s43058-026-00892-4 · Cited by 11 · see citing works
Asiimwe, I. G., & Rumona, D. (2016) · Research Integrity and Peer Review · doi:10.1186/s41073-016-0017-4 · Cited by 7 · see citing works
Boland, M. R., Kashyap, A., Xiong, J., Holmes, J. H., & Lorch, S. A. (2018) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocy119 · Cited by 7 · see citing works
Tempini, N., & Teira, D. (2020) · Springer eBooks · doi:10.1007/978-3-030-37177-7_11 · Cited by 4 · see citing works
Ayorinde, A., Williams, I., Mannion, R., Song, F., Skrybant, M., Lilford, R. J., & Chen, Y. (2020) · Health Services and Delivery Research · doi:10.3310/hsdr08330 · Cited by 4 · see citing works
Olejniczak, M. J., & Pliszka, P. (2019) · Przeglad Epidemiologiczny · doi:10.32394/pe.73.32 · Cited by 3 · see citing works
Commercial Interests in Continuing Medical Education: Where Do Electronic Health Record Vendors Fit?
Rubinstein, P., Middleton, B., Goodman, K., & Lehmann, C. U. (2020) · Academic medicine : journal of the Association of American Medical Colleges · doi:10.1097/acm.0000000000003190 · Cited by 2 · see citing works
Hersh, W. (2020) · Health informatics · doi:10.1007/978-3-030-47686-1_2 · Cited by 2 · see citing works
Publication of Lung Cancer Clinical Trials in Japan
Ochi, N., Honda, Y., Yamane, H., Umemura, S., Hotta, K., Tanimoto, M., Kiura, K., & Takigawa, N. (2015) · Haigan · doi:10.2482/haigan.55.1070 · Cited by 1 · see citing works
Scott, E. R., & Wallsten, R. L. (2019) · Elsevier eBooks · doi:10.1016/b978-0-12-812626-4.00010-3 · Cited by 1 · see citing works
The Value of a Negative Antinuclear Antibody (ANA) Test: An Often Forgotten Result
Lee, A. Y. S. (2021) · Reumatología Clínica · doi:10.1016/j.reuma.2021.04.002 · Cited by 1 · see citing works
Views on Electronic Health Record
Graña, M., & Echaniz, O. (2016) · Smart innovation, systems and technologies · doi:10.1007/978-3-319-39687-3_31
Patient Engagement to Improve Medication Safety in the Hospital
Prey, J. E. (2016) · doi:10.7916/d8f18zx4
Publication Bias: Assessment and Impact
Canestaro, W. J. (2017) · ResearchWorks at the University of Washington (University of Washington)
The Value of a Negative Antinuclear Antibody (ANA) Test: An Often Forgotten Result
Lee, A. Y. S. (2022) · Reumatología Clínica (English Edition) · doi:10.1016/j.reumae.2021.04.016
Ayorinde, A., Williams, I., Mannion, R., Song, F., Skrybant, M., Lilford, R. J., & Chen, Y.-F. (2020) · Figshare · doi:10.6084/m9.figshare.12413774
Ayorinde, A., Williams, I., Mannion, R., Song, F., Skrybant, M., Lilford, R. J., & Chen, Y.-F. (2020) · Figshare · doi:10.6084/m9.figshare.12413768
- Authors
- Vawdrey, D. K., & Hripcsak, G.
- Affiliations
- Columbia University (US)
- Corresponding author
- Vawdrey, David K.
- ORCID
- Hripcsak, George
- Venue
- Journal of Biomedical Informatics
- Publisher
- Elsevier BV
- Published
- 2013-02
- Volume
- 46
- Issue
- 1
- Pages
- 139-141
- ISSN
- 1532-0464 (ISSN-L)
- PMC
- PMC3662474
- OpenAlex
- W2142189912
- Cited by
- 38
- References
- 27
- Citations by year
- 2026: 1 · 2024: 1 · 2022: 1 · 2021: 1 · 2020: 9 · 2019: 4 · 2018: 2 · 2017: 6 · 2016: 4 · 2015: 5 · 2014: 2 · 2013: 2
- FWCI
- 3.28
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- Locations
- Journal of Biomedical Informatics (publishedVersion)
- PubMed (publishedVersion)
- Europe PMC (PubMed Central) (acceptedVersion)
- PubMed Central (acceptedVersion)
- Funders
- Agency for Healthcare Research and Quality (R03HS018250)
- U.S. National Library of Medicine (R01:LM06910, R01LM006910)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Life Sciences › Pharmacology, Toxicology and Pharmaceutics › Pharmacology › Pharmaceutical industry and healthcare
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Keywords
- publication bias, electronic health records, clinical trials, non-publication, ClinicalTrials.gov, clinical trial registration, positive results
- MeSH terms
- Clinical Trials as Topic, Publication Bias, Electronic Health Records
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
