Aggregator: A machine learning approach to identifying MEDLINE articles that derive from the same underlying clinical trial
Open accessThe DOI matches its Crossref record; 7 of the 7 references it cites were found in a registry or an index.
Abstract
OBJECTIVE: It is important to identify separate publications that report outcomes from the same underlying clinical trial, in order to avoid over-counting these as independent pieces of evidence. METHODS: We created positive and negative training sets (comprised of pairs of articles reporting on the same condition and intervention) that were, or were not, linked to the same clinicaltrials.gov trial registry number. Features were extracted from MEDLINE and PubMed metadata; pairwise similarity scores were modeled using logistic regression. RESULTS: Article pairs from the same trial were identified with high accuracy (F1 score=0.843). We also created a clustering tool, Aggregator, that takes as input a PubMed user query for RCTs on a given topic, and returns article clusters predicted to arise from the same clinical trial. DISCUSSION: Although painstaking examination of full-text may be needed to be conclusive, metadata are surprisingly accurate in predicting when two articles derive from the same underlying clinical trial.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Publication bias in meta-analysis its causes and consequences
Thornton, A. (2000) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(99)00161-4 · Cited by 1,291 · cited 2×
“Multiple publications that report clinical outcomes from the same clinical trial can bias the apparent effect size calculated in systematic reviews and meta-analyses [3, 4].”
BackgroundImpact of covert duplicate publication on meta-analysis: a case study
Tramèr, M. R., Reynolds, D. J. M., Moore, R. A., & McQuay, H. J. (1997) · BMJ · doi:10.1136/bmj.315.7109.635 · Cited by 702 · cited 2×
“Multiple publications that report clinical outcomes from the same clinical trial can bias the apparent effect size calculated in systematic reviews and meta-analyses [3, 4].”
BackgroundCharacteristics of Clinical Trials Registered in ClinicalTrials.gov, 2007-2010
Califf, R. M. (2012) · JAMA · doi:10.1001/jama.2012.3424 · Cited by 516
Publication of NIH funded trials registered in ClinicalTrials.gov: cross sectional analysis
Ross, J. S., Tse, T., Zarin, D. A., Xu, H., Zhou, L., & Krumholz, H. M. (2012) · BMJ · doi:10.1136/bmj.d7292 · Cited by 371 · cited 1×
“New treatments are tested in clinical trials (of varying size and design), which may be registered in formal clinical trial registries (or not), and which may be published in the peer-reviewed literature (or not) [1].”
BackgroundPubMed related articles: a probabilistic topic-based model for content similarity
Lin, J., & Wilbur, W. J. (2007) · BMC Bioinformatics · doi:10.1186/1471-2105-8-423 · Cited by 243 · cited 2×
“A pairwise similarity model incorporating 12 features had surprisingly good accuracy, of which shared author names, and high similarity in the PubMed Related Articles function [13] were the most important.”
MethodSystematic reviews can be produced and published faster
Sampson, M., Shojania, K. G., Garritty, C., Horsley, T., Ocampo, M., & Moher, D. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2008.02.004 · Cited by 103 · cited 1×
“We plan to implement Aggregator as one piece of an overall pipeline of informatics tools that can accelerate the process of writing systematic reviews [22–25].”
MethodCohen, A. M., Smalheiser, N. R., McDonagh, M. S., Yu, C., Adams, C. E., Davis, J. M., & Yu, P. S. (2015) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocu025 · Cited by 78
Self-citation
Cited by
O’Connor, A. M., Tsafnat, G., Gilbert, S. B., Thayer, K. A., Shemilt, I., Thomas, J., Glasziou, P., & Wolfe, M. S. (2019) · Systematic Reviews · doi:10.1186/s13643-019-0975-y · Cited by 110
Cohen, A. M., Smalheiser, N. R., McDonagh, M. S., Yu, C., Adams, C. E., Davis, J. M., & Yu, P. S. (2015) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocu025 · Cited by 78
Self-citationIsojarvi, J., Wood, H., Lefebvre, C., & Glanville, J. (2018) · Research Synthesis Methods · doi:10.1002/jrsm.1294 · Cited by 38
“TABLE 1 (Continued) Topic 1 Topic 2 Topic 3 Topic 4 Topic 5 Schroll et al 2015 x Shanahan 2016 x Shao et al 2015 x Smail‐Faugeron 2015 x Stegeman et al 2014 x Tai et al 2012 x Tang et al 2015 x Tse 2017 x Viergever et al 2015 x Wieseler et al 2012 x Wieseler et al 2013 x Wolfe et al 2013 x x…”
BackgroundA web-based tool for automatically linking clinical trials to their publications
Smalheiser, N. R., & Holt, A. W. (2022) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocab290 · Cited by 20
Self-citationMS2: Multi-Document Summarization of Medical Studies
DeYoung, J., Beltagy, I., van Zuylen, M., Kuehl, B., & Wang, L. L. (2021) · arXiv · doi:10.48550/arxiv.2104.06486 · Cited by 10
Metadata
- Authors
- Shao, W., Adams, C. E., Cohen, A. M., Davis, J. M., McDonagh, M. S., Thakurta, S., Yu, P. S., & Smalheiser, N. R.
- Venue
- Methods
- Publisher
- Elsevier BV
- Published
- 2015-03
- Volume
- 74
- Pages
- 65-70
- ISSN
- 1046-2023 (ISSN-L)
- OpenAlex
- W1993894371
- Cited by
- 19
- References
- 7
- FWCI
- 1.28
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Aggregator: A machine learning approach to identifying MEDLINE articles that derive from the same underlying clinical trial. https://app.cochat.ai/papers/10.1016/j.ymeth.2014.11.006
- The paper
- Shao, W., Adams, C. E., Cohen, A. M., Davis, J. M., McDonagh, M. S., Thakurta, S., Yu, P. S., & Smalheiser, N. R. (2015). Aggregator: A machine learning approach to identifying MEDLINE articles that derive from the same underlying clinical trial. Methods. https://doi.org/10.1016/j.ymeth.2014.11.006
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.