Toward assessing clinical trial publications for reporting transparency
Open accessPublished version of a preprint
Abstract
Objective To annotate a corpus of randomized controlled trial (RCT) publications with the checklist items of CONSORT reporting guidelines and using the corpus to develop text mining methods for RCT appraisal.Methods We annotated a corpus of 50 RCT articles at the sentence level using 37 fine-grained CONSORT checklist items. A subset (31 articles) was double-annotated and adjudicated, while 19 were annotated by a single annotator and reconciled by another. We calculated inter-annotator agreement at the article and section level using MASI (Measuring Agreement on Set-Valued Items) and at the CONSORT item level using Krippendorff's α. We experimented with two rule-based methods (phrase-based and section header-based) and two supervised learning approaches (support vector machine and BioBERT-based neural network classifiers), for recognizing 17 methodology-related items in the RCT Methods sections.Results We created CONSORT-TM consisting of 10,709 sentences, 4,845 (45%) of which were annotated with 5,246 labels. A median of 28 CONSORT items (out of possible 37) were annotated per article. Agreement was moderate at the article and section levels (average MASI: 0.60 and 0.64, respectively). Agreement varied considerably among individual checklist items (Krippendorff's α= 0.06-0.96). The model based on BioBERT performed best overall for recognizing methodology-related items (micro-precision: 0.82, micro-recall: 0.63, micro-F1: 0.71). Combining models using majority vote and label aggregation further improved precision and recall, respectively.Conclusion Our annotated corpus, CONSORT-TM, contains more fine-grained information than earlier RCT corpora. Low frequency of some CONSORT items made it difficult to train effective text mining models to recognize them. For the items commonly reported, CONSORT-TM can serve as a testbed for text mining methods that assess RCT transparency, rigor, and reliability, and support methods for peer review and authoring assistance. Minor modifications to the annotation scheme and a larger corpus could facilitate improved text mining models. CONSORT-TM is publicly available at https://github.com/kilicogluh/CONSORT-TM.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
CONSORT-TM is relatively small and some checklist items are infrequently reported, which has led to poor performance for rare items in previous work.
“CONSORT-TM is relatively small and some checklist items are infrequently reported (e.g., 6b (Changes to Outcomes)). This has led to poor performance for rare items in previous work21,22.”
From Text classification models for assessing the completeness of randomized controlled trial publications based on CONSORT reporting guidelines (2024) · cites this paper as [21]
Inter-annotator agreement for these labels was lower, as annotators often confused them as well.
“We note that inter-annotator agreement for these labels was lower, as annotators often confused them as well21.”
From Text classification models for assessing the completeness of randomized controlled trial publications based on CONSORT reporting guidelines (2024) · cites this paper as [21]
Both micro and macro scores were reported by Singh et al. (2021), Kilicoglu et al. (2021), Kiritchenko et al. (2010), Fiszman et al. (2007), and Zhang et al. (2024).
“Both micro and macro scores were reported by Singh et al. (2021), 45 Kilicoglu et al. (2021), 38 Kiritchenko et al. (2010), 46 Fiszman et al. (2007), 47 Zhang et al. (2024), 179 Karystianis et al. (2014, 2017) 48 , 49 reported micro across documents, and macro across the classes.”
From Data extraction methods for systematic review (semi)automation: a living systematic review (2021) · cites this paper as [38]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · see citing works
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & for the PRISMA Group (2009) · BMJ · doi:10.1136/bmj.b2535 · Cited by 83,823 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · see citing works
Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement.
(2009) · PubMed Central · Cited by 44,859 · see citing works
Content Analysis: An Introduction to its Methodology.
Shelley, M., & Krippendorff, K. (1984) · Journal of the American Statistical Association · doi:10.2307/2288384 · Cited by 24,305 · see citing works
The FAIR Guiding Principles for scientific data management and stewardship
Wilkinson, M. D., Dumontier, M., Aalbersberg, I. J., Appleton, G., Axton, M., Baak, A., Blomberg, N., Boiten, J.-W., da Silva Santos, L. B., Bourne, P. E., et al. (2016) · Scientific Data · doi:10.1038/sdata.2016.18 · Cited by 18,423 · see citing works
Evidence based medicine: what it is and what it isn't
Sackett, D. L., Rosenberg, W. M. C., Gray, J. A. M., Haynes, R. B., & Richardson, W. S. (1996) · BMJ · doi:10.1136/bmj.312.7023.71 · Cited by 15,769 · see citing works
Content Analysis: An Introduction to Its Methodology
Krippendorff, K. (2019) · doi:10.4135/9781071878781 · Cited by 15,448 · see citing works
CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials
the CONSORT Group, Schulz, K. F., Altman, D. G., & Moher, D. (2010) · BMC Medicine · doi:10.1186/1741-7015-8-18 · Cited by 13,371 · see citing works
CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials
Schulz, K. F., Altman, D. G., Moher, D., & for the CONSORT Group (2010) · BMJ · doi:10.1136/bmj.c332 · Cited by 9,329 · see citing works
Moher, D., Hopewell, S., Schulz, K. F., Montori, V., Gotzsche, P. C., Devereaux, P. J., Elbourne, D., Egger, M., & Altman, D. G. (2010) · BMJ · doi:10.1136/bmj.c869 · Cited by 8,883 · see citing works
Improving Bioscience Research Reporting: The ARRIVE Guidelines for Reporting Animal Research
Kilkenny, C., Browne, W. J., Cuthill, I. C., Emerson, M., & Altman, D. G. (2010) · PLoS Biology · doi:10.1371/journal.pbio.1000412 · Cited by 7,300 · see citing works
BioBERT: a pre-trained biomedical language representation model for biomedical text mining
Lee, J., Yoon, W., Kim, S., Kim, D., Kim, S., So, C. H., & Kang, J. (2020) · Bioinformatics · doi:10.1093/bioinformatics/btz682 · Cited by 7,218 · see citing works
Improving bioscience research reporting: The ARRIVE guidelines for reporting animal research
Kilkenny, C., Browne, W. J., Cuthill, I. C., Emerson, M., & Altman, D. G. (2010) · Journal of Pharmacology and Pharmacotherapeutics · doi:10.4103/0976-500x.72351 · Cited by 6,277 · see citing works
1,500 scientists lift the lid on reproducibility
Baker, M. (2016) · Nature · doi:10.1038/533452a · Cited by 4,209 · see citing works
Yeh, M. (2014) · Journal of Nursing Research · doi:10.1097/jnr.0000000000000065 · Cited by 3,836 · see citing works
Consort 2010 statement: extension to cluster randomised trials
Campbell, M. K., Piaggio, G., Elbourne, D. R., Altman, D. G., & for the CONSORT Group (2012) · BMJ · doi:10.1136/bmj.e5661 · Cited by 3,510 · see citing works
Improving the Quality of Reporting of Randomized Controlled Trials: The CONSORT Statement
Begg, C. (1996) · JAMA · doi:10.1001/jama.1996.03540080059030 · Cited by 3,127 · see citing works
Promoting an open research culture
Nosek, B. A., Alter, G., Banks, G. C., Borsboom, D., Bowman, S. D., Breckler, S. J., Buck, S., Chambers, C. D., Chin, G., Christensen, G., et al. (2015) · Science · doi:10.1126/science.aab2374 · Cited by 2,708 · see citing works
EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks
Wei, J., & Zou, K. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1670 · Cited by 2,242 · see citing works
Avoidable waste in the production and reporting of research evidence
Chalmers, I., & Glasziou, P. (2009) · The Lancet · doi:10.1016/s0140-6736(09)60329-9 · Cited by 2,208 · see citing works
Inter-Coder Agreement for Computational Linguistics
Artstein, R., & Poesio, M. (2008) · Computational Linguistics · doi:10.1162/coli.07-034-r2 · Cited by 1,665 · see citing works
Boutron, I., Altman, D. G., Moher, D., Schulz, K. F., Ravaud, P., & for the CONSORT NPT Group* (2017) · Annals of Internal Medicine · doi:10.7326/m17-0046 · Cited by 1,538 · see citing works
An overview of MetaMap: historical perspective and recent advances
Aronson, A. R., & Lang, F.-M. (2010) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.2009.002733 · Cited by 1,519 · see citing works
Murad, M. H., Asi, N., Alsawas, M., & Alahdab, F. (2016) · Evidence Based Medicine · doi:10.1136/ebmed-2016-110401 · Cited by 1,388 · see citing works
Policy: NIH plans to enhance reproducibility
Collins, F. S., & Tabak, L. A. (2014) · Nature · doi:10.1038/505612a · Cited by 1,247 · see citing works
A call for transparent reporting to optimize the predictive value of preclinical research
Landis, S. C., Amara, S. G., Asadullah, K., Austin, C. P., Blumenstein, R., Bradley, E. W., Crystal, R. G., Darnell, R. B., Ferrante, R. J., Fillit, H., et al. (2012) · Nature · doi:10.1038/nature11556 · Cited by 1,196 · see citing works
Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing
Braud, C., Lacroix, O., & Søgaard, A. (2017) · doi:10.18653/v1/d17-1 · Cited by 910 · see citing works
Turner, L., Shamseer, L., Altman, D. G., Weeks, L., Peters, J., Kober, T., Dias, S., Schulz, K. F., Plint, A. C., & Moher, D. (2012) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000030.pub2 · Cited by 740 · see citing works
O’Mara-Eves, A., Thomas, J., McNaught, J., Miwa, M., & Ananiadou, S. (2015) · Systematic Reviews · doi:10.1186/2046-4053-4-5 · Cited by 681 · see citing works
Avoidable Waste in the Production and Reporting of Research Evidence
Chalmers, I., & Glasziou, P. (2009) · Obstetrics and Gynecology · doi:10.1097/aog.0b013e3181c3020d · Cited by 608 · see citing works
Simera, I., Moher, D., Hirst, A., Hoey, J., Schulz, K. F., & Altman, D. G. (2010) · BMC Medicine · doi:10.1186/1741-7015-8-24 · Cited by 590 · see citing works
The past, present and future of Registered Reports
Chambers, C. D., & Tzavella, L. (2021) · Nature Human Behaviour · doi:10.1038/s41562-021-01193-7 · Cited by 499 · see citing works
CONSORT for reporting randomised trials in journal and conference abstracts
Hopewell, S., Clarke, M., Moher, D., Wager, E., Middleton, P., Altman, D. G., & Schulz, K. F. (2008) · The Lancet · doi:10.1016/s0140-6736(07)61835-2 · Cited by 458 · see citing works
The Ontology for Biomedical Investigations
Bandrowski, A. E., Brinkman, R. R., Brochhausen, M., Brush, M., Bug, B., Chibucos, M. C., Clancy, K., Courtot, M., Derom, D., Dumontier, M., et al. (2016) · PLoS ONE · doi:10.1371/journal.pone.0154556 · Cited by 387 · see citing works
O’Mara-Eves, A., Thomas, J., McNaught, J., Miwa, M., & Ananiadou, S. (2015) · Systematic Reviews · doi:10.1186/s13643-015-0031-5 · Cited by 351 · see citing works
RobotReviewer: evaluation of a system for automatically assessing bias in clinical trials
Marshall, I. J., Kuiper, J., & Wallace, B. C. (2016) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocv044 · Cited by 317 · see citing works
Reproducible Research Practices and Transparency across the Biomedical Literature
Iqbal, S. A., Wallach, J. D., Khoury, M. J., Schully, S. D., & Ioannidis, J. P. A. (2016) · PLOS Biology · doi:10.1371/journal.pbio.1002333 · Cited by 310 · see citing works
Answering Clinical Questions with Knowledge-Based and Statistical Techniques
Demner‐Fushman, D., & Lin, J. (2007) · Computational Linguistics · doi:10.1162/coli.2007.33.1.63 · Cited by 297 · see citing works
Automating data extraction in systematic reviews: a systematic review
Jonnalagadda, S. R., Goyal, P., & Huffman, M. D. (2015) · Systematic Reviews · doi:10.1186/s13643-015-0066-7 · Cited by 287 · see citing works
Nye, B., Li, J. J., Patel, R., Yang, Y., Marshall, I., Nenkova, A., & Wallace, B. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1019 · Cited by 224 · see citing works
Automatic classification of sentences to support Evidence Based Medicine.
Kim, S. N., Martinez, D., Cavedon, L., & Yencken, L. (2011) · BMC Bioinformatics · doi:10.1186/1471-2105-12-s2-s5 · Cited by 214 · see citing works
A systematic scoping review of adherence to reporting guidelines in health care literature
Thabane, L., Samaan, Mbuagbaw, L. C. E., Kosa, S. D., Debono, V. B., Dillenburg, Zhang, Fruci, V., Dennis, B. B., & Bawor, M. (2013) · Journal of Multidisciplinary Healthcare · doi:10.2147/jmdh.s43952 · Cited by 194 · see citing works
ExaCT: automatic extraction of clinical trial characteristics from journal publications
Kiritchenko, S., de Bruijn, B., Carini, S., Martin, J., & Sim, I. (2010) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-10-56 · Cited by 188 · see citing works
Shamseer, L., Hopewell, S., Altman, D. G., Moher, D., & Schulz, K. F. (2016) · Trials · doi:10.1186/s13063-016-1408-z · Cited by 168 · see citing works
Measuring Agreement on Set-valued Items (MASI) for Semantic and Pragmatic Annotation
Passonneau, R. J. (2006) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/4nuo6thi27ax · Cited by 142 · see citing works
(2016) · PubMed · Cited by 134 · see citing works
(2013) · Dove Medical Press (Taylor and Francis Group) · Cited by 122 · see citing works
The Rigor and Transparency Index Quality Metric for Assessing Biological and Medical Science Methods
Menke, J., Roelandse, M., Ozyurt, B., Martone, M., & Bandrowski, A. (2020) · iScience · doi:10.1016/j.isci.2020.101698 · Cited by 90 · see citing works
Hierarchical Neural Networks for Sequential Sentence Classification in Medical Scientific Abstracts
Jin, D., & Szolovits, P. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1349 · Cited by 76 · see citing works
Neural Networks for Joint Sentence Classification in Medical Paper Abstracts
Dernoncourt, F., Lee, J. Y., & Peter Szolovits, B. (2017) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/e17-2110 · Cited by 70 · see citing works
The past, present, and future of Registered ReportsPreprint
Chambers, C., & Tzavella, L. (2020) · doi:10.31222/osf.io/43298 · Cited by 63 · see citing works
Identifying scientific artefacts in biomedical literature: The Evidence Based Medicine use case.
Hassanzadeh, H., Groza, T., & Hunter, J. (2014) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2014.02.006 · Cited by 62 · see citing works
Improving reference prioritisation with PICO recognition.
Brockmeier, A. J., Ju, M., Przybyła, P., & Ananiadou, S. (2019) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-019-0992-8 · Cited by 62 · see citing works
Biomedical text mining for research rigor and integrity: tasks, challenges, directions
Kilicoglu, H. (2017) · Briefings in Bioinformatics · doi:10.1093/bib/bbx057 · Cited by 61 · see citing works
Checklists work to improve science
(2018) · Nature · doi:10.1038/d41586-018-04590-7 · Cited by 38 · see citing works
Automatic integration of drug indications from multiple health resources
Névéol, A., & Lu, Z. (2010) · doi:10.1145/1882992.1883096 · Cited by 31 · see citing works
Integrative analysis workflow for the structural and functional classification of C-type lectins
Koh, G., Low, A., Poh, D. K. H., Yao, Y., Ng, S. K., Wong, V. V. T., Vagenende, V., Lam, K., & Lee, D. (2011) · BMC Bioinformatics · doi:10.1186/1471-2105-12-s14-s5 · Cited by 24 · see citing works
Hsu, W. W. Y., Speier, W. F., & Taira, R. K. (2012) · PubMed Central · Cited by 21 · see citing works
Menke, J., Roelandse, M., Özyurt, İ. B., et al. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.01.15.908111 · Cited by 16 · see citing works
Automatic recognition of self-acknowledged limitations in clinical research literature
Halil Kilicoglu, B., Rosemblat, G., Malički, M., & ter Riet, G. (2018) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocy038 · Cited by 15 · see citing works
Self-citationMenagerie: A text-mining tool to support animal-human translation in neurodegeneration research
Zeiss, C. J., Shin, D., Wyk, B. V., Beck, A. P., Zatz, N., Sneiderman, C. A., & Halil Kilicoglu, B. (2019) · PLoS ONE · doi:10.1371/journal.pone.0226176 · Cited by 12 · see citing works
A manual corpus of annotated main findings of clinical case reports
Smalheiser, N. R., Luo, M., Addepalli, S., & Cui, X. (2018) · Database · doi:10.1093/database/bay143 · Cited by 10 · see citing works
Identifying Sample Size Characteristics in Randomized Controlled Trial Publications.
Kilicoglu, H., Hoang, L., & Wadhwa, S. (2020) · AMIA · Cited by 2 · see citing works
Self-citationUpdate on the endorsement of CONSORT by high impact factor journals: a survey of journal
Moher, D., Schulz, K. F., Hopewell, S., Shamseer, L., & Altman, D. G. (2020) · Carolina Digital Repository (University of North Carolina at Chapel Hill) · doi:10.17615/tyxz-9y57 · Cited by 1 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Data extraction methods for systematic review (semi)automation: a living systematic review
Schmidt, L., Olorisade, B. K., McGuinness, L. A., Thomas, J., & Higgins, J. P. T. (2021) · F1000Research · doi:10.12688/f1000research.51117.1 · Cited by 106 · see citing works
Data extraction methods for systematic review (semi)automation: Update of a living systematic review
Schmidt, L., Finnerty, A. N., Elmore, R., Olorisade, B. K., Thomas, J., & Higgins, J. P. T. (2023) · F1000Research · doi:10.12688/f1000research.51117.2 · Cited by 49 · see citing works
Data extraction methods for systematic review (semi)automation: Update of a living systematic review
Schmidt, L., Finnerty Mutlu, A. N., Elmore, R., Olorisade, B. K., Thomas, J., & Higgins, J. P. T. (2025) · F1000Research · doi:10.12688/f1000research.51117.3 · Cited by 27 · see citing works
García-Méndez, S., de Arriba-Pérez, F., Barros-Vila, A., González-Castaño, F. J., & Costa‐Montenegro, E. (2023) · Applied Intelligence · doi:10.1007/s10489-023-04452-4 · Cited by 25 · see citing works
Schulz, R., Langen, G., Prill, R., Cassel, M., & Weissgerber, T. L. (2022) · BMJ Open · doi:10.1136/bmjopen-2021-059347 · Cited by 23 · see citing works
Large Language Model Analysis of Reporting Quality of Randomized Clinical Trial Articles
Srinivasan, A., Berkowitz, J. S., Friedrich, N. A., Kivelson, S., & Tatonetti, N. P. (2025) · JAMA Network Open · doi:10.1001/jamanetworkopen.2025.29418 · Cited by 21 · see citing works
Methodology reporting improved over time in 176,469 randomized controlled trials
Kilicoglu, H., Jiang, L., Hoang, L., Mayo-Wilson, E., Vinkers, C. H., & Otte, W. M. (2023) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2023.08.004 · Cited by 19 · see citing works
Self-citationGarcía-Méndez, S., de Arriba-Pérez, F., Barros-Vila, A., & González-Castaño, F. J. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.119611 · Cited by 19 · see citing works
Windisch, P. Y., Dennstädt, F., Koechli, C., Schröder, C., Aebersold, D. M., Forster, R. E., & Zwahlen, D. R. (2024) · Cureus · doi:10.7759/cureus.75748 · Cited by 20 · see citing works
Jiang, L., Lan, M., Menke, J. D., Vorland, C. J., & Kilicoglu, H. (2024) · Scientific Reports · doi:10.1038/s41598-024-72130-7 · Cited by 18 · see citing works
Dennstädt, F., Windisch, P. Y., Filchenko, I., Zink, J., Putora, P. M., Shaheen, A., Gaio, R., Čihorić, N., Wosny, M., Aeppli, S., et al. (2025) · Oncology · doi:10.1159/000546946 · Cited by 11 · see citing works
Retrieval augmented scientific claim verification
Liu, H., Soroush, A., Nestor, J. G., Park, E., Idnay, B., Fang, Y., Pan, J. Y., Liao, S., Bernard, M., Peng, Y., et al. (2024) · JAMIA Open · doi:10.1093/jamiaopen/ooae021 · Cited by 9 · see citing works
Lan, M., Cheng, M., Hoang, L., ter Riet, G., & Halil Kilicoglu, B. (2024) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2024.104628 · Cited by 8 · see citing works
Self-citationMethodological information extraction from randomized controlled trial publications: a pilot study.
Hoang, L., Guan, Y., & Halil Kilicoglu, B. (2022) · PubMed Central · Cited by 7 · see citing works
Self-citationChen, D., Li, P., Khoshkish, E., Lee, S., Ning, T., Tahir, U., Wong, H. C. Y., Lee, M. S. F., & Raman, S. (2026) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaf223 · Cited by 6 · see citing works
Windisch, P. Y., Dennstädt, F., Koechli, C., Foerster, R., Schröder, C., Aebersold, D. M., & Zwahlen, D. R. (2024) · JCO Clinical Cancer Informatics · doi:10.1200/cci-24-00150 · Cited by 5 · see citing works
Hoanga, L., Jiang, L., & Halil Kilicoglu, B. (2022) · PubMed Central · Cited by 6 · see citing works
Jiang, L., Vorland, C. J., Ying, X., Brown, A. W., Menke, J. D., Hong, G., Lan, M., Mayo‐Wilson, E., & Halil Kilicoglu, B. (2025) · Scientific Data · doi:10.1038/s41597-025-04629-1 · Cited by 6 · see citing works
Hoang, L., Jiang, L., & Kilicoglu, H. (2021) · doi:10.1101/2021.09.14.21263586 · Cited by 5 · see citing works
Self-citationWindisch, P. Y., Dennstädt, F., Koechli, C., Foerster, R., Schröder, C., Aebersold, D. M., & Zwahlen, D. R. (2025) · Oncology · doi:10.1159/000546970 · Cited by 5 · see citing works
Schulz, R., Langen, G., Prill, R., Cassel, M., & Weissgerber, T. (2021) · doi:10.1101/2021.07.20.21260565 · Cited by 23 · see citing works
Clinical trial transparency regulations: Implications to various scholarly publishing stakeholders
Abdul Razack, H. I., Aranjani, J. M., & Mathew, S. T. (2022) · Science and Public Policy · doi:10.1093/scipol/scac041 · Cited by 3 · see citing works
Buljan, I., García-Costa, D., Grimaldo, F., Klein, R. A., Bakker, M., & Marušić, A. (2024) · Journal of Informetrics · doi:10.1016/j.joi.2024.101555 · Cited by 3 · see citing works
Windisch, P. Y., Dennstädt, F., Koechli, C., Schroeder, C., Aebersold, D. M., Foerster, R., & Zwahlen, D. R. (2024) · medRxiv · doi:10.1101/2024.10.23.24316005 · Cited by 3 · see citing works
Evaluating the Reporting Quality of 21,041 Randomized Controlled Trial ArticlesPreprint
Srinivasan, A., Berkowitz, J., Kivelson, S., Friendrich, N., & Tatonetti, N. P. (2025) · medRxiv · doi:10.1101/2025.03.06.25323528 · Cited by 3 · see citing works
Windisch, P., Dennstädt, F., Koechli, C., Förster, R., Schröder, C., Aebersold, D. M., & Zwahlen, D. R. (2024) · doi:10.1101/2024.07.09.24310155 · Cited by 2 · see citing works
Predicting the sample size of randomized controlled trials using natural language processing
Windisch, P. Y., Dennstädt, F., Koechli, C., Forster, R. E., Schröder, C., Aebersold, D. M., & Zwahlen, D. R. (2024) · JAMIA Open · doi:10.1093/jamiaopen/ooae116 · Cited by 2 · see citing works
Eckmann, P., Barnett, A. G., Bannach‐Brown, A., Atria, E. B., Cabanac, G., Franzen, L. D. O., Gazda, M. A., Hair, K., Howison, J., Halil Kilicoglu, B., et al. (2026) · PLoS ONE · doi:10.1371/journal.pone.0342225 · Cited by 4 · see citing works
Windisch, P. Y., Dennstädt, F., Koechli, C., Forster, R. E., Schröder, C., Aebersold, D. M., & Zwahlen, D. R. (2024) · medRxiv · doi:10.1101/2024.06.17.24309020 · Cited by 1 · see citing works
Jiang, L., Vorland, C. J., Ying, X., Brown, A. W., Menke, J. D., Hong, G., Lan, M., Mayo‐Wilson, E., & Halil Kilicoglu, B. (2025) · medRxiv · doi:10.1101/2025.01.14.25320543
Wintjens, M. S. J. N., Aydeniz, E., van Rosmalen, F., Driessen, R., Hulshof, A., Bergmans, D. C. J. J., van Kuijk, S. M. J., van der Horst, I. C. C., van Bussel, B. C. T., & collaborators, M. I. C. C. (. (2025) · Critical Care Explorations · doi:10.1097/cce.0000000000001211
Srinivasan, A., Kivelson, S., Friedrich, N. A., Berkowitz, J. S., & Tatonetti, N. P. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-95838-0_42
Dobrescu, A., Cohen‐Boulakia, S., & Naderi, N. (2025) · ACM Conference on Reproducibility and Replicability (ACM REP) · doi:10.1145/3736731.3746133
Xing, Q., Jiang, Y., Xu, D., Hou, Z., & Xu, J. (2025) · International Journal of General Medicine · doi:10.2147/ijgm.s546037
Zuluaga, J. S. F., Centeno, J. N., Velásquez, J. M. D., Madera, S. S., Mugno, M. A. B., Cervantes, G. A. D., Ocampo, C. R., Orozco, L. M. M., Alamo-Anich, M. S., Rubio, O. P., et al. (2026) · European journal of health research · doi:10.32457/ejhr.v12.3624
Spiero, I., Leeuwenberg, A. M., Kamperman, J., Hooft, L., Moons, K. G. M., & Damen, J. A. (2026) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2026.112320
- Authors
- Kilicoglu, H., Rosemblat, G., Hoang, L., Wadhwa, S., Peng, Z., Malički, M., Schneider, J., & ter Riet, G.
- Affiliations
- National Institutes of Health (US)
- University of Illinois Urbana-Champaign (US)
- Stanford University (US)
- Venue
- Journal of Biomedical Informatics
- Publisher
- Elsevier BV
- Published
- 2021-04
- Volume
- 116
- Pages
- 103717
- ISSN
- 1532-0464 (ISSN-L)
- PMC
- PMC8112250
- OpenAlex
- W3133622905
- Cited by
- 34
- References
- 66
- Citations by year
- 2026: 3 · 2025: 12 · 2024: 10 · 2023: 4 · 2022: 4 · 2021: 1
- FWCI
- 2.75
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- www.elsevier.com/tdm/userlicense/1.0/ (tdm)
- www.elsevier.com/legal/tdmrep-license (tdm)
- www.elsevier.com/open-access/userlicense/1.0/ (vor, 365 d)
- doi.org/10.15223/policy-017 (stm-asf)
- doi.org/10.15223/policy-037 (stm-asf)
- doi.org/10.15223/policy-012 (stm-asf)
- doi.org/10.15223/policy-029 (stm-asf)
- doi.org/10.15223/policy-004 (stm-asf)
- Locations
- Journal of Biomedical Informatics (publishedVersion)
- medRxiv (acceptedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
- Data Archiving and Networked Services (DANS) (submittedVersion, cc-by-nc-nd)
- Pure Amsterdam UMC (submittedVersion, cc-by)
- Funders
- National Library of Medicine
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Biomedical Text Mining and Ontologies
- Social Sciences › General Social Sciences › Computational and Text Analysis Methods
- Keywords
- randomized controlled trials, reporting transparency, annotated corpus, text mining, inter-annotator agreement, BioBERT, sentence-level annotation, peer review
- MeSH terms
- Humans, Serial Publications, Randomized Controlled Trials as Topic, Checklist, Support Vector Machine
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
