Automatic summarization of MEDLINE citations for evidence-based medical treatment: A topic-oriented evaluation
Open accessThe DOI matches its Crossref record; 4 of the 4 references it cites were found in a registry or an index.
Abstract
As the number of electronic biomedical textual resources increases, it becomes harder for physicians to find useful answers at the point of care. Information retrieval applications provide access to databases; however, little research has been done on using automatic summarization to help navigate the documents returned by these systems. After presenting a semantic abstraction automatic summarization system for MEDLINE citations, we concentrate on evaluating its ability to identify useful drug interventions for fifty-three diseases. The evaluation methodology uses existing sources of evidence-based medicine as surrogates for a physician-annotated reference standard. Mean average precision (MAP) and a clinical usefulness score developed for this study were computed as performance metrics. The automatic summarization system significantly outperformed the baseline in both metrics. The MAP gain was 0.17 (p < 0.01) and the increase in the overall score of clinical usefulness was 0.39 (p < 0.05).
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The quality of health care delivered to adults in the United States
McGlynn, E. A., Asch, S. M., Adams, J., Keesey, J., Hicks, J., DeCristofaro, A., & Kerr, E. A. (2003) · New England Journal of Medicine · doi:10.1056/nejmsa022615 · Cited by 5,079
The Pyramid Method: Incorporating human content selection variation in summarization evaluation
Nenkova, A., Passonneau, R., & McKeown, K. (2007) · ACM Transactions on Speech and Language Processing · doi:10.1145/1233912.1233913 · Cited by 307 · cited 1×
“Because of this, research is being pursued to produce reference standards automatically [35].”
Text Categorization Models for High-Quality Article Retrieval in Internal Medicine
Aphinyanaphongs, Y. (2004) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1641 · Cited by 177 · cited 1×
“Research is being pursued on automatic determination of quality of evidence [47,48]; such processing could be incorporated into the summarization system.”
Towards Automatic Recognition of Scientifically Rigorous Clinical Research Evidence
Kilicoglu, H., Demner-Fushman, D., Rindflesch, T. C., Wilczynski, N. L., & Haynes, R. B. (2009) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2996 · Cited by 101 · cited 1×
“Research is being pursued on automatic determination of quality of evidence [47,48]; such processing could be incorporated into the summarization system.”
Self-citation
Cited by
The structure and dynamics of cocitation clusters: A multiple‐perspective cocitation analysis
Chen, C., Ibekwe‐SanJuan, F., & Hou, J. (2010) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.21309 · Cited by 1,585
What can natural language processing do for clinical decision support?
Demner-Fushman, D., Chapman, W. W., & McDonald, C. J. (2009) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2009.08.007 · Cited by 688
Self-citationInformation retrieval and text mining technologies for chemistry
Krallinger, M., Rabal, O., Lourenço, A., Oyarzabal, J., & Valencia, A. (2017) · Chemical Reviews · doi:10.1021/acs.chemrev.6b00851 · Cited by 281
Bravo, À., Piñero, J., Queralt-Rosinach, N., Rautschka, M., & Furlong, L. I. (2015) · BMC Bioinformatics · doi:10.1186/s12859-015-0472-9 · Cited by 264
Cohen, A. M., Smalheiser, N. R., McDonagh, M. S., Yu, C., Adams, C. E., Davis, J. M., & Yu, P. S. (2015) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocu025 · Cited by 78
Metadata
- Authors
- Fiszman, M., Demner-Fushman, D., Kilicoglu, H., & Rindflesch, T. C.
- Venue
- Journal of Biomedical Informatics
- Publisher
- Elsevier BV
- Published
- 2009-10
- Volume
- 42
- Issue
- 5
- Pages
- 801-813
- ISSN
- 1532-0464 (ISSN-L)
- OpenAlex
- W1989217037
- Cited by
- 77
- References
- 4
- FWCI
- 2.85
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Automatic summarization of MEDLINE citations for evidence-based medical treatment: A topic-oriented evaluation. https://app.cochat.ai/papers/10.1016/j.jbi.2008.10.002
- The paper
- Fiszman, M., Demner-Fushman, D., Kilicoglu, H., & Rindflesch, T. C. (2009). Automatic summarization of MEDLINE citations for evidence-based medical treatment: A topic-oriented evaluation. Journal of Biomedical Informatics. https://doi.org/10.1016/j.jbi.2008.10.002
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.