The DOI matches its Crossref record; 65 of the 65 references it cites were found in a registry or an index.
MedSTS: a resource for clinical semantic textual similarity
Open accessOverview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Leveraging the research in corpus analysis and deidentification, the Open Health Natural Language Processing (OHNLP) consortium has created multiple deidentified data sets for a couple of NLP tasks based on real clinical sentences [14-16].”
From Family History Extraction From Synthetic Clinical Narratives Using Natural Language Processing: Overview and Evaluation of a Challenge Data Set and Solutions for the 2019 National NLP Clinical Challenges (n2c2)/Open Health Natural Language Processing (OHNLP) Competition (2021) · cites this paper as [15]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
WordNet: a lexical database for English
Miller, G. A. (1995) · Communications of the ACM · doi:10.1145/219717.219748 · Cited by 14,015
A vector space model for automatic indexing
Salton, G., Wong, A., & Yang, C. S. (1975) · Communications of the ACM · doi:10.1145/361219.361220 · Cited by 7,521
VII. Note on regression and inheritance in the case of two parents
Pearson, K. (1895) · Proceedings of the Royal Society of London · doi:10.1098/rspl.1895.0041 · Cited by 3,726
Savova, G. K., Masanz, J. J., Ogren, P. V., Zheng, J., Sohn, S., Kipper-Schuler, K. C., & Chute, C. G. (2010) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.2009.001560 · Cited by 1,967
Development and application of a metric on semantic nets
Rada, R., Mili, H., Bicknell, E. J., & Blettner, M. (1989) · IEEE Transactions on Systems Man and Cybernetics · doi:10.1109/21.24528 · Cited by 1,835
Clinical information extraction applications: A literature review
Wang, Y., Wang, L., Rastegar-Mojarad, M., Moon, S., Shen, F., Afzal, N., Liu, S., Zeng, Y., Mehrabi, S., Sohn, S., et al. (2018) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2017.11.011 · Cited by 910
Self-citationSavova, G. K., Kipper-Schuler, K. C., Hurdle, J. F., & Meystre, S. M. (2008) · Yearbook of Medical Informatics · doi:10.1055/s-0038-1638592 · Cited by 895
WordNet: similarity - measuring the relatedness of concepts
Pedersen, T., Patwardhan, S., & Michelizzi, J. (2004) · Cited by 854
Ted Pedersen, A., Patwardhan, S., & Michelizzi, J. (2004) · doi:10.3115/1614025.1614037 · Cited by 841
Sentence similarity based on semantic nets and corpus statistics
Li, Y., McLean, D., Bandar, Z. A., O'Shea, J. D., & Crockett, K. (2006) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2006.130 · Cited by 819
Hovy, E., Marcus, M., Palmer, M., Ramshaw, L., & Weischedel, R. (2006) · Proceedings of the Human Language Technology Conference of the NAACL, Companion Volume: Short Papers on XX - NAACL '06 · doi:10.3115/1614049.1614064 · Cited by 806
The Oxford handbook of computational linguistics
Mitkov, R. (2003) · Choice Reviews Online · doi:10.5860/choice.41-0758 · Cited by 718
SemEval-2012 Task 6: A Pilot on Semantic Textual Similarity
Agirre, E., Cer, D. M., Diab, M., & González-Agirre, A. (2012) · Cited by 653
Measures of semantic similarity and relatedness in the biomedical domain
Ted Pedersen, A., Pakhomov, S., Patwardhan, S., & Chute, C. G. (2006) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2006.06.004 · Cited by 566
Types and Origins of Diagnostic Errors in Primary Care Settings
Singh, H., Giardina, T. D., Meyer, A. N. D., Forjuoh, S. N., Reis, M. D., & Thomas, E. J. (2013) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2013.2777 · Cited by 559
SemEval-2016 Task 1: Semantic Textual Similarity, Monolingual and Cross-Lingual Evaluation
Agirre, E., Banea, C., Cer, D. M., Diab, M., González-Agirre, A., Mihalcea, R. F., Rigau, G., & Wiebe, J. (2016) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s16-1081 · Cited by 549
Collecting Image Annotations Using Amazon's Mechanical Turk
Rashtchian, C., Young, P., Hodosh, M., & Hockenmaier, J. (2010) · Cited by 541
SemEval-2014 Task 10: Multilingual Semantic Textual Similarity
Agirre, E., Banea, C., Cardie, C., Cer, D. M., Diab, M., González-Agirre, A., Guo, W., Mihalcea, R. F., Rigau, G., & Wiebe, J. (2014) · International Workshop on Semantic Evaluation (SemEval) · doi:10.3115/v1/s14-2010 · Cited by 511
Using Measures of Semantic Relatedness for Word Sense Disambiguation
Patwardhan, S., Banerjee, S., & Ted Pedersen, A. (2003) · Lecture notes in computer science · doi:10.1007/3-540-36456-0_24 · Cited by 506
Semantic text similarity using corpus-based word similarity and string similarity
Islam, A., & Inkpen, D. Z. (2008) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/1376815.1376819 · Cited by 492
SemEval-2015 Task 2: Semantic Textual Similarity, English, Spanish and Pilot on Interpretability
Agirre, E., Banea, C., Cardie, C., Cer, D. M., Diab, M., González-Agirre, A., Guo, W., López-Gazpio, I., Maritxalar, M., Mihalcea, R. F., et al. (2015) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s15-2045 · Cited by 488
Sentence Fusion for Multidocument News Summarization
Barzilay, R., & McKeown, K. R. (2005) · Computational Linguistics · doi:10.1162/089120105774321091 · Cited by 429
A comparison of word embeddings for the biomedical natural language processing
Wang, Y., Liu, S., Afzal, N., Rastegar-Mojarad, M., Wang, L., Shen, F., Kingsbury, P., & Liu, H. (2018) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2018.09.008 · Cited by 370
Self-citation*SEM 2013 shared task: Semantic Textual Similarity
Agirre, E., Cer, D. M., Diab, M., González-Agirre, A., & Guo, W. (2013) · Joint Conference on Lexical and Computational Semantics · Cited by 369
Measuring the semantic similarity of texts
Corley, C. D., & Mihalcea, R. F. (2005) · doi:10.3115/1631862.1631865 · Cited by 357
Kuhn, T., Basch, P., Barr, M., & Yackel, T. R. (2015) · Annals of Internal Medicine · doi:10.7326/m14-2128 · Cited by 272
Aliguliyev, R. M. (2008) · Expert Systems with Applications · doi:10.1016/j.eswa.2008.11.022 · Cited by 266
Can Electronic Clinical Documentation Help Prevent Diagnostic Errors?
Schiff, G. D., & Bates, D. W. (2010) · New England Journal of Medicine · doi:10.1056/nejmp0911734 · Cited by 239
Yearbook of Medical Informatics
(2013) · Yearbook of Medical Informatics · doi:10.1055/s-0038-1638865 · Cited by 238
Automated methods for the summarization of electronic health records
Perotte, R., & Elhadad, N. (2015) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocv032 · Cited by 227
Paraphrasing for automatic evaluation
Kauchak, D., & Barzilay, R. (2006) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.3115/1220835.1220893 · Cited by 222
SemEval-2014 Task 7: Analysis of Clinical Text
Pradhan, S., Elhadad, N., Chapman, W. W., Manandhar, S., & Savova, G. (2014) · International Workshop on Semantic Evaluation (SemEval) · doi:10.3115/v1/s14-2007 · Cited by 212
UKP: Computing Semantic Textual Similarity by Combining Multiple Content Similarity Measures
Bär, D., Biemann, C., Gurevych, I., & Zesch, T. (2012) · TUbilio (Technical University of Darmstadt) · doi:10.17185/duepublico/72186 · Cited by 200
TakeLab: Systems for Measuring Semantic Text Similarity
Šarić, F., Glavaš, G., Karan, M., Šnajder, J., & Bašić, B. D. (2012) · MADOC (University of Mannheim) · Cited by 200
Williams, C., Mostashari, F., Mertz, K., Hogin, E., & Atwal, P. M. S. (2012) · Health Affairs · doi:10.1377/hlthaff.2011.1314 · Cited by 183
Implementation of the Federal Health Information Technology Initiative
Blumenthal, D. (2011) · New England Journal of Medicine · doi:10.1056/nejmsr1112158 · Cited by 177
Quantifying clinical narrative redundancy in an electronic health record
Wrenn, J. O., Stein, D. M., Bakken, S., & Stetson, P. D. (2010) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m3390 · Cited by 153
Developing a corpus of plagiarised short answers
Clough, P. D., & Stevenson, M. (2010) · Language Resources and Evaluation · doi:10.1007/s10579-009-9112-1 · Cited by 139
HARVEST, a longitudinal patient record summarizer
Hirsch, J. S., Tanenbaum, J. S., Lipsky Gorman, S., Liu, C., Schmitz, E., Hashorva, D., Ervits, A., Vawdrey, D., Sturm, M., & Elhadad, N. (2015) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2014-002945 · Cited by 135
Intelligent Indexing and Semantic Retrieval of Multimodal Documents
Srihari, R. K., Zhang, Z. M., & Rao, A. (2000) · Information Retrieval · doi:10.1023/a:1009962928226 · Cited by 120
Blanco‐Fernández, Y., Pazos-Arias, J. J., Gil‐Solla, A., Ramos‐Cabrer, M., López‐Nores, M., García‐Duque, J., Vilas, A. F., Redondo, R. P. D., & Muñoz, J. B. (2007) · Knowledge-Based Systems · doi:10.1016/j.knosys.2007.07.004 · Cited by 111
Linking Tweets to News: A Framework to Enrich Short Text Data in Social Media
Guo, W., Li, H., Ji, H., & Diab, M. (2013) · Cited by 102
Semantic Textual Similarity Methods, Tools, and Applications: A Survey
Majumder, G., Pakray, P., Gelbukh, A. F., & Pinto, D. (2016) · Computación y Sistemas · doi:10.13053/cys-20-4-2506 · Cited by 99
Characterizing the Source of Text in Electronic Health Record Progress Notes
Wang, M. D., Khanna, R. R., & Najafi, N. (2017) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2017.1548 · Cited by 85
Unified Medical Language System term occurrences in clinical notes: a large-scale corpus analysis
Wu, S. T., Liu, H., Li, D., Tao, C., Musen, M. A., Chute, C. G., & Shah, N. H. (2012) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000744 · Cited by 80
Self-citationEmbi, P. J., Weir, C., Efthimiadis, E. N., Thielke, S. M., Hedeen, A. N., & Hammond, K. W. (2013) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2012-000946 · Cited by 74
Assessing sentence similarity through lexical, syntactic and semantic analysis
Ferreira, R., Lins, R. D., Simske, S. J., Freitas, F., & Riss, M. (2016) · Computer Speech & Language · doi:10.1016/j.csl.2016.01.003 · Cited by 60
Natural Language Processing in Health Care and Biomedicine
Friedman, C., & Elhadad, N. (2013) · Springer eBooks · doi:10.1007/978-1-4471-4474-8_8 · Cited by 60
Evaluating measures of redundancy in clinical texts.
Zhang, R., Pakhomov, S., McInnes, B. T., & Melton, G. B. (2011) · PubMed Central · Cited by 47
Beetle II: A System for Tutoring and Computational Linguistics Experimentation
Dzikovska, M. O., Moore, J. D., Steinhauser, N. B., Campbell, G. E., Farrow, E., & Callaway, C. B. (2010) · Edinburgh Research Explorer (University of Edinburgh) · Cited by 46
A knowledge-based question answering system for B2C eCommerce
Tapeh, A. G., & Rahgozar, M. (2008) · Knowledge-Based Systems · doi:10.1016/j.knosys.2008.04.005 · Cited by 43
The Oxford Handbook of Computational Linguistics
Sampson, G. (2003) · Literary and Linguistic Computing · doi:10.1093/llc/18.3.333 · Cited by 33
Discovering implicit intention-level knowledge from natural-language texts
Atkinson, J., Cabrera, A. F., & Aravena, E. (2009) · Knowledge-Based Systems · doi:10.1016/j.knosys.2008.10.007 · Cited by 29
A Comparison of Word Embeddings for the Biomedical Natural Language ProcessingPreprint
Wang, Y., Liu, S., Afzal, N., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.00400 · Cited by 28
Self-citationAfzal, N., Wang, Y., & Liu, H. (2016) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s16-1103 · Cited by 26
Self-citationBioCreative/OHNLP Challenge 2018
Rastegar-Mojarad, M., Liu, S., Wang, Y., Afzal, N., Wang, L., Shen, F., Fu, S., & Liu, H. (2018) · ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics (ACM-BCB) · doi:10.1145/3233547.3233672 · Cited by 26
Self-citationWang, Y., Rastegar-Mojarad, M., Komandur-Elayavilli, R., & Liu, H. (2017) · Database · doi:10.1093/database/bax091 · Cited by 18
Self-citationLearning Document Semantic Representation with Hybrid Deep Belief Network
Yan, Y., Yin, X.-C., Li, S., Yang, M., & Hao, H. (2015) · Computational Intelligence and Neuroscience · doi:10.1155/2015/650527 · Cited by 16
Using language models to identify relevant new information in inpatient clinical notes.
Zhang, R., Pakhomov, S., Lee, J. T., & Melton, G. B. (2014) · PubMed Central · Cited by 13
A frequency-filtering strategy of obtaining PHI-free sentences from clinical data repository
Li, D., Rastegar-Mojarad, M., Elayavilli, R. K., Wang, Y., Mehrabi, S., Yu, Y., Sohn, S., Li, Y., Afzal, N., & Liu, H. (2015) · ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics (ACM-BCB) · doi:10.1145/2808719.2808752 · Cited by 9
Self-citationExploiting Sentence Similarities for Better Alignments
Li, T., & Srikumar, V. (2016) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d16-1237 · Cited by 8
Moon, S., Liu, S., Kingsbury, P., Chen, D. J., Wang, Y., Shen, F., Chaudhry, R., & Liu, H. (2017) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm.2017.8217883 · Cited by 5
Self-citationSemantic Indexing of Multilingual Corpora and its Application on the History Domain
Raganato, A., Camacho-Collados, J., Raganato, A., & Joung, Y. (2016) · BOA (University of Milano-Bicocca) · Cited by 2
Analysis of the Impact of Machine Translation Evaluation Metrics for Semantic Textual Similarity
Magnolini, S., Vo, N. P. A., & Popescu, O. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-49130-1_33 · Cited by 1
Revisiting Recurrent Networks for Paraphrastic Sentence Embeddings
Wieting, J., & Gimpel, K. (2017) · arXiv · doi:10.48550/arxiv.1705.00364
Research works printed but not linked: 2
- Black, P. E. (2004). Ratcliff/Obershelp pattern recognition. In V. Pieterse & P. E. Black, (Eds.), Dictionary of algorithms and data structures (Vol. 17).
- Corley, C. (2007). A knowledge-based approach to text-to-text similarity CoUrTney Corley, Andras Csomai & Rada Mihalcea Dept. of Computer Science, University of North Texas. In Recent advances in natural language processing IV: Selected Papers from RANLP 2005 (Vol. 292, p. 197).
Cited by
Sorted by citations
A large language model for electronic health records
Yang, X., Chen, A., PourNejatian, N., Shin, H. C., Smith, K. E., Parisien, C., Compas, C., Martin, C., Costa, A. B., Flores, M. G., et al. (2022) · npj Digital Medicine · doi:10.1038/s41746-022-00742-2 · Cited by 847
Peng, Y., Yan, S., & Lu, Z. (2019) · Proceedings of the 18th BioNLP Workshop and Shared Task · doi:10.18653/v1/w19-5006 · Cited by 899
Recent Advancements in Emerging Technologies for Healthcare Management Systems: A Survey
Junaid, S. B., Imam, A. A., Balogun, A. O., Silva, L. C. D., Surakat, Y. A., Kumar, G., Abdulkarim, M., Shuaibu, A. N., Garba, A., Sahalu, Y., et al. (2022) · Healthcare · doi:10.3390/healthcare10101940 · Cited by 398
Large language models in medical and healthcare fields: applications, advances, and challenges
Wang, D., & Zhang, S. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10921-0 · Cited by 246
BioSentVec: creating sentence embeddings for biomedical texts
Chen, Q.-Y., Peng, Y., & Lu, Z.-Y. (2018) · IEEE International Conference on Healthcare Informatics · doi:10.1109/ichi.2019.8904728 · Cited by 241
Metadata
- Authors
- Wang, Y., Afzal, N., Fu, S., Wang, L., Shen, F., Rastegar-Mojarad, M., & Liu, H.
- Affiliations
- Mayo Clinic (US)
- Corresponding author
- Liu, Hongfang
- Venue
- Language Resources and Evaluation
- Published
- 2018-10-24
- Volume
- 54
- Issue
- 1
- Pages
- 57-72
- ISSN
- 1574-020X (ISSN-L)
- 1574-0218
- OpenAlex
- W2889272240
- Cited by
- 124
- References
- 65
- Citations by year
- 2026: 5 · 2025: 11 · 2024: 12 · 2023: 18 · 2022: 20 · 2021: 27 · 2020: 22 · 2019: 9
- FWCI
- 8.16
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Language Resources and Evaluation (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Institute of General Medical Sciences (R01GM102282)
- U.S. National Library of Medicine (R01LM11934)
- National Institute of Biomedical Imaging and Bioengineering (R01EB19403)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Health Sciences › Medicine › Health Informatics › Biomedical Text Mining and Ontologies
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Keywords
- semantic textual similarity, clinical text, shared task, electronic health records, clinical information extraction, redundancy reduction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
