Redundancy-Aware Topic Modeling for Patient Record Notes

“While in many natural language processing applications, redundancy is considered useful and its presence in a document collection is leveraged [24], in bioinformatics, for instance, it is standard practice to remove redundant sequences from databases [25] or to adjust for such sequences when using Gibbs sampling [26].”

“Topic modeling has been shown to help in large number of tasks, including document classification and clustering, multi-document summarization [5], search [6], document labeling [7,8], and information extraction [9].”

“This research contributes primarily to the emerging field of understanding the characteristics of patient records, their impact on mining techniques, and how to correct for them [27].”

“In the news domain, for instance, the same story authored by a news agency is reported by a number of sources, resulting in clusters of documents likely to be rich in redundancy [28].”

“While these initial results are promising on EHR data, initial research suggests that the specific characteristics of the clinical language affect the methods and results of these techniques [3,22,23].”

“In the clinical community there is some discontent with copying and pasting [1,2] and its negative impact on quality of clinical documentation.”

“In the biomedical domain, probabilistic graphical models like LDA are an emerging technology only recently applied to EHR data [13–17], biomedical corpora [18–20], and consumer health resources [21].”

“While in many natural language processing applications, redundancy is considered useful and its presence in a document collection is leveraged [24], in bioinformatics, for instance, it is standard practice to remove redundant sequences from databases [25] or to adjust for such sequences when using Gibbs sampling [26].”

“Topic modeling has been shown to help in large number of tasks, including document classification and clustering, multi-document summarization [5], search [6], document labeling [7,8], and information extraction [9].”

“In the biomedical domain, probabilistic graphical models like LDA are an emerging technology only recently applied to EHR data [13–17], biomedical corpora [18–20], and consumer health resources [21].”

“While these initial results are promising on EHR data, initial research suggests that the specific characteristics of the clinical language affect the methods and results of these techniques [3,22,23].”

“In the biomedical domain, probabilistic graphical models like LDA are an emerging technology only recently applied to EHR data [13–17], biomedical corpora [18–20], and consumer health resources [21].”

“ithm analysis [33] (MSA-COSRs 2013 Sampling? A analysis of ws [89] opicSpam 2013 sampling A detection [95] A 2013 sampling A Clustering [51] A) 2014 sampling A ws [48] TWILITE 2014 algorithm A witter [98] A 2014 Gibbs-Samplin A in Notes 97] opic-Modeling(BTM) 2014 sampling A xt [47] A) 2014 sampling A (nDCG) platform…”

“Besides, it may be shaped in a precise way to learn them correctly [23].”

“Recently, a topic modeling approach was proposed to identify and control for such redundancy across patient notes.(56) In the general NLP community, identifying statement level similarity has been studied through the tasks of paraphrasing identification and textual entailment.”

“…in Natural Language Processing (NLP) and have been successfully applied: e.g. Support Vector Machines and Latent Dirichlet Allocation have been used for some tasks like classification on patient record notes [1] or other documents in diseases like diabetes showing satisfying results [2,3].”

“Recently, different data representation methods have been reported to help in classifying clinical documents, for example by using lexical features, such as bag-of-words and n-grams [10, 15], adopting topic modeling methods, for example, latent Dirichlet allocation (LDA) algorithm [16], or integrating knowledge in med…”

“Researchers have also investigated novel probabilistic graphical models, also used in a variety of tasks, including ICU illness severity scoring [25], diagnosis code prediction [26], redundancy-aware topic modeling [27], disease progression modeling [28], and disease subtypes identification [11].”

“More detailed discussion of this model and methods for inference can be found in the literature.(30,31) Example topics can be seen in Tables 2 and 3, where they are represented by their most highly associated words.”

“Redundancyaware LDA (Red-LDA) is one of the versions of LDA for handling redundancy issue in medical documents and has shown better performance than LDA [9].”

“These methods have been successfully applied to medical text classification tasks on patient record notes [15] and other text documents in diseases like diabetes and cancers [16, 17], but the assumption is an oversimplification that brings some shortcomings.”

“The frequency of redundant text in clinical notes, created by copy and paste or auto-generation, undermines machine learning training and evaluation due to over representation of an SDH single occurrence.(32) To derive an unbiased estimate of likely SDH documentation, we removed auto-generated and copy and paste entri…”

“Problems such as classifying cancer [8], patient record notes [9], intensive care unit (ICU) procedures and diagnosis [10], and other text documents [11] are solved by prevailing machine learning approaches with satisfactory results.”

“For example, clas-sification of patient record notes were successfully executed by applying Latent Dirichlet Allocation and Support Vector Machines [22], while [23] addresses the classification task of documents in diabetes diseases, also reporting satisfying results.”

“Some other researches propose new variant of LDA to improve its performance for example Cohen et al. (2014) propose a topic model based on LDA to take into account the problem of redundancy in clinical notes [23].”

“In the biomedical field, RedLDA topic modeling technique is utilized for identification of redundancy in the patient records [32].”

“In the clinical domain, work has investigated the use of topic models in cased-based retrieval [1], characterizing clinical concepts over time [9], and the impact of copy and pasted text on topic learning [10].”

“Topic modeling is often used in health care to generalize clinical notes, improve the automatic processing of patient data, and explore clinical datasets [52-55].”

“The over representation of certain SDoH factors because of the duplication impairs the training and evaluation of machine learning and deep learning models 81,82 .”

“0 suggesting more similar topics 19,50,51 .”

“One version of LDA for redundancy issue in medical documents is Redundancy-aware LDA (RedLDA) that shows higher performance than LDA [31].”

“The RedLDA topic model is used in the biomedical field to determine redundancy in patient information data [25].”

“Second, themes identified in clinical notes may reflect the use of copy and paste functions in clinical documentation (35).”

“2011), redundancy-aware topic modeling (Cohen et al. 2014), disease progression modeling (Wang, Sontag, and Wang 2014), and disease subtypes identification (Schulam, Wigley, and Saria 2015).”

“…domain, LVMs have also been applied to various analytic applications, including topic model for clinical notes analysis (Arnold and Speier, 2012; Cohen et al., 2014), restricted Boltzmann machine (Hinton, 2010) for patient profile modeling (Nguyen et al., 2013; Tran et al., 2015), distance…”

“LDA topic modeling has been applied to the unstructured notes of EHRs to describe clinical groups [104-108] and predicting outcomes [109-116].”

“Topic Models are utilized to discover topics from vast text collection of some particular domains [10, 16, 26, 34, 40].”

“The over representation of certain SDoH factors because of the duplication impairs the training and evaluation of machine learning and deep learning models [46 47].”

“Because of its ability to find hidden meaning in text, LDA has been used for topic extraction in various fields such as software engineering [21], political science [22], medicine [23] and linguistic science.”

“A total of ten extracted topics from the user tweets were found sufficient for all the sponsors, as further extracted topics were similar-in-meaning to one of the previous ten topics, implying semantic saturation and redundancy (Cohen et al., 2014).”

“Recently, topic modeling methods, particularly LDA, have been used for both structured and unstructured clinical data [21,26,54-56].”

“Other existing research for EMR text pre-processing has demonstrated the possibility of extracting temporal expressions [32], [33]; correcting misspelled words [34]; resolving existing co-references [35]; eliminating redundancy [36] and generating summaries [37].”

“Its applicability has garnered interests in variety of fields including, but not limited to, history [15], marketing[2], and medicine[6, 10].”

“Topic models are used in different applications to explore, categorize, summarize and analyze textual data [22, 23].”

“Those kind of redundancies may have a negative impact on the quality of clinical documentation and decision making [9,10].”

“…used methods, and we therefore decided to focus on comparing these three techniques: Latent Dirichlet Allocation (LDA) is extensively used in the literature due to its robustness and flexibility in identifying hidden thematic structures within large text corpora (e.g., [23, 39, 49, 54, 102, 103]).”

“Gibbs sampling has been applied to discover the relationship between authors and their research interest [8], a Topic structure for the patient record [9] and to learn topics from scientific research articles [17].”

“The assumption oversimplifies the matter, even though it has been successful in sorting patient record notes [ 11] and other documents related to disease such as diabetes and cancers [12].”

“For example, [7] effectively classified patient record entries using Latent Dirichlet Allocation and Support Vector Machines, while [17] handles the classification of documents in diabetic disorders, also obtaining satisfactory outcomes.”

“For instance, tasks such as classifying patient record notes have seen successful applications of algorithms like Support Vector Machines and Latent Dirichlet Allocation [8, 9].”

“By using this method, [vector 1] is represented as [[1, 2], [1, 12]] [[1,12]] [[3, 2], [3, 4]] [[4, 4], [6, 7, 8, 11]] [[5 ,2], [9, 10]], and [vector2] is represented as [[1, 2], [1, 6]] [[2, 2], [2, 3]] [[3, 0], [0]] [[4, 2], [4, 5]] [[5, 0], [0]].”

“A recent work tackles this problem for the modelling of medical case notes [Cohen et al. 2014].”