The DOI matches its Crossref record; 29 of the 29 references it cites were found in a registry or an index.
Use of Sentiment Analysis for Capturing Patient Experience From Free-Text Comments Posted Online
Open accessAbstract
BACKGROUND: There are large amounts of unstructured, free-text information about quality of health care available on the Internet in blogs, social networks, and on physician rating websites that are not captured in a systematic way. New analytical techniques, such as sentiment analysis, may allow us to understand and use this information more effectively to improve the quality of health care. OBJECTIVE: We attempted to use machine learning to understand patients' unstructured comments about their care. We used sentiment analysis techniques to categorize online free-text comments by patients as either positive or negative descriptions of their health care. We tried to automatically predict whether a patient would recommend a hospital, whether the hospital was clean, and whether they were treated with dignity from their free-text description, compared to the patient's own quantitative rating of their care. METHODS: We applied machine learning techniques to all 6412 online comments about hospitals on the English National Health Service website in 2010 using Weka data-mining software. We also compared the results obtained from sentiment analysis with the paper-based national inpatient survey results at the hospital level using Spearman rank correlation for all 161 acute adult hospital trusts in England. RESULTS: There was 81%, 84%, and 89% agreement between quantitative ratings of care and those derived from free-text comments using sentiment analysis for cleanliness, being treated with dignity, and overall recommendation of hospital respectively (kappa scores: .40-.74, P<.001 for all). We observed mild to moderate associations between our machine learning predictions and responses to the large patient survey for the three categories examined (Spearman rho 0.37-0.51, P<.001 for all). CONCLUSIONS: The prediction accuracy that we have achieved using this machine learning process suggests that we are able to predict, from free-text, a reasonably accurate assessment of patients' opinion about different performance aspects of a hospital and that these machine learning predictions are associated with results of more conventional surveys.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Cortes, C., & Vapnik, V. (1995) · Machine Learning · doi:10.1007/bf00994018 · Cited by 59,421
Quinlan, J. R. (1986) · Machine Learning · doi:10.1007/bf00116251 · Cited by 21,439
Breiman, L. (1996) · Machine Learning · doi:10.1007/bf00058655 · Cited by 16,325
Opinion Mining and Sentiment Analysis
Pang, B., & Lee, L. (2008) · now publishers, Inc. eBooks · doi:10.1561/9781601981516 · Cited by 6,720
Opinion mining and sentiment analysis
Pang, B., & Lee, L. (2008) · Foundations and Trends® in Information Retrieval · doi:10.1561/1500000011 · Cited by 6,372
A Comparative Study on Feature Selection in Text Categorization
Yang, Y., & Pedersen, J. (1997) · Cited by 4,790
A comparison of event models for naive bayes text classification
McCallum, A., & Nigam, K. (1998) · Cited by 3,289
Sentiment Analysis and Opinion Mining
Liu, B. (2012) · Synthesis Lectures on Human Language Technologies · doi:10.1007/978-3-031-02145-9 · Cited by 3,281
Sentiment Analysis and Opinion Mining
Zhang, L., & Liu, B. (2017) · Encyclopedia of Machine Learning and Data Mining · doi:10.1007/978-1-4899-7687-1_907 · Cited by 3,260
SentiWordNet 3.0: An Enhanced Lexical Resource for Sentiment Analysis and Opinion Mining
Baccianella, S., Esuli, A., & Sebastiani, F. (2010) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/2jktnrykui9s · Cited by 2,666
Predicting Elections with Twitter: What 140 Characters Reveal about Political Sentiment
Tumasjan, A., Sprenger, T., Sandner, P., & Welpe, I. (2010) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v4i1.14009 · Cited by 2,649
SENTIWORDNET: A Publicly Available Lexical Resource for Opinion Mining
Esuli, A., & Sebastiani, F. (2006) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/54vo5wjya4cw · Cited by 2,480
Understanding bag-of-words model: a statistical framework
Zhang, Y., Jin, R., & Zhou, Z.-H. (2010) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-010-0001-0 · Cited by 1,384
WordNet Affect: an Affective Extension of WordNet
Strapparava, C., & Valitutti, A. (2004) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/2yctm9q26ar4 · Cited by 1,384
Statistical Comparisons of Classifiers over Multiple Data Sets
DemšarJanez (2006) · Journal of Machine Learning Research · doi:10.5555/1248547.1248548 · Cited by 1,074
Data mining in bioinformatics using Weka
Frank, E., Hall, M., Trigg, L., Holmes, G., & Witten, I. H. (2004) · Bioinformatics · doi:10.1093/bioinformatics/bth261 · Cited by 882
Holland, K. (2008) · Nurse Education in Practice · doi:10.1016/j.nepr.2008.07.004 · Cited by 588
Gao, G. G., McCullough, J. S., Agarwal, R., & Jha, A. K. (2012) · Journal of Medical Internet Research · doi:10.2196/jmir.2003 · Cited by 373
Harnessing the cloud of patient experience: using social media to detect poor quality healthcare
Greaves, F., Ramirez-Cano, D., Millett, C., Darzi, A., & Donaldson, L. (2013) · BMJ Quality & Safety · doi:10.1136/bmjqs-2012-001527 · Cited by 272
Self-citationPatient-centred healthcare, social media and the internet: the perfect storm?
Rozenblum, R., & Bates, D. W. (2013) · BMJ Quality & Safety · doi:10.1136/bmjqs-2012-001744 · Cited by 161
Greaves, F., Pape, U. J., King, D., Darzi, A. W., Majeed, A., Wachter, R. M., & Millett, C. J. (2012) · BMJ Quality & Safety · doi:10.1136/bmjqs-2012-000906 · Cited by 124
Self-citationOutcomes of educational interventions in type 2 diabetes: WEKA data-mining analysis
Sigurðardóttir, Á. K., Jónsdóttir, H., & Benediktsson, R. (2007) · Patient Education and Counseling · doi:10.1016/j.pec.2007.03.007 · Cited by 97
Greaves, F., Pape, U. J., Lee, H., Smith, D. M., Darzi, A. W., Majeed, A., & Millett, C. J. (2012) · Journal of Medical Internet Research · doi:10.2196/jmir.2280 · Cited by 79
Self-citationLagu, T. C., Goff, S. L., Hannon, N. S., Shatz, A., & Lindenauer, P. K. (2013) · The Joint Commission Journal on Quality and Patient Safety · doi:10.1016/s1553-7250(13)39003-5 · Cited by 74
Weka Machine Learning for Predicting the Phospholipidosis Inducing Potential
Ivanciuc, O. (2008) · Current Topics in Medicinal Chemistry · doi:10.2174/156802608786786589 · Cited by 71
Alemi, F., Torii, M., Clementz, L., & Aron, D. C. (2012) · Quality Management in Health Care · doi:10.1097/qmh.0b013e3182417fc4 · Cited by 67
Consistently Increasing Numbers of Online Ratings of Healthcare in England
Greaves, F., & Millett, C. J. (2012) · Journal of Medical Internet Research · doi:10.2196/jmir.2157 · Cited by 56
Self-citationQuality-Based Knowledge Discovery from Medical Text on the Web
Holzinger, A., Yildirim, P., Geier, M., & Simonic, K.-M. (2013) · Intelligent systems reference library · doi:10.1007/978-3-642-37688-7_7 · Cited by 34
Detecting salient aspects in online reviews of health providers.
Brody, S., & Elhadad, N. (2010) · PubMed Central · Cited by 31
Cited by
Machine learning in medicine: a practical introduction
Sidey-Gibbons, J. A. M., & Sidey-Gibbons, C. J. (2019) · BMC Medical Research Methodology · doi:10.1186/s12874-019-0681-4 · Cited by 1,271
Artificial intelligence for diabetes management and decision support: Literature review
Contreras, I., & Vehi, J. (2018) · Journal of Medical Internet Research · doi:10.2196/10775 · Cited by 560
A Systematic Review on Healthcare Analytics: Application and Theoretical Perspective of Data Mining
Islam, M. S., Hasan, M. M., Wang, X., Germack, H. D., & Noor-E-Alam, M. (2018) · Healthcare · doi:10.3390/healthcare6020054 · Cited by 267
Social Media and Rating Sites as Tools to Understanding Quality of Care: A Scoping Review
Verhoef, L. M., Van de Belt, T. H., Engelen, L. J., Schoonhoven, L., & Kool, R. B. (2014) · Journal of Medical Internet Research · doi:10.2196/jmir.3024 · Cited by 130
Clinical text classification research trends: Systematic literature review and open issues
Mujtaba, G., Shuib, L., Idris, N., Hoo, W. L., Raj, R. G., Khowaja, K., Shaikh, K., & Nweke, H. F. (2019) · Expert Systems with Applications · doi:10.1016/j.eswa.2018.09.034 · Cited by 123
Metadata
- Authors
- Greaves, F., Ramirez-Cano, D., Millett, C., Darzi, A., & Donaldson, L.
- Venue
- Journal of Medical Internet Research
- Publisher
- JMIR Publications Inc.
- Published
- 2013-11-01
- Volume
- 15
- Issue
- 11
- Pages
- e239
- ISSN
- 1438-8871 (ISSN-L)
- OpenAlex
- W2054290230
- Cited by
- 321
- References
- 29
- FWCI
- 13.62
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.