A Data Quality Assessment Guideline for Electronic Health Record Data Reuse
Open accessAbstract
Introduction: We describe the formulation, development, and initial expert review of 3x3 Data Quality Assessment (DQA), a dynamic, evidence-based guideline to enable electronic health record (EHR) data quality assessment and reporting for clinical research.Methods: 3x3 DQA was developed through the triangulation results from three studies: a review of the literature on EHR data quality assessment, a quantitative study of EHR data completeness, and a set of interviews with clinical researchers. Following initial development, the guideline was reviewed by a panel of EHR data quality experts.Results: The guideline embraces the task-dependent nature of data quality and data quality assessment. The core framework includes three constructs of data quality: complete, correct, and current data. These constructs are operationalized according to the three primary dimensions of EHR data: patients, variables, and time. Each of the nine operationalized constructs maps to a methodological recommendation for EHR data quality assessment. The initial expert response to the framework was positive, but improvements are required.Discussion: The initial version of 3x3 DQA promises to enable explicit guideline-based best practices for EHR data quality assessment and reporting. Future work will focus on increasing clarity on how and when 3x3 DQA should be used during the research process, improving the feasibility and ease-of-use of recommendation execution, and clarifying the process for users to determine which operationalized constructs and recommendations are relevant for a given dataset and study.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
A quality assessment framework was applied to all variables to address well-known issues when reusing EHR data for secondary analyses.
“Fifth, there are well-known issues when reusing EHR data for secondary analyses49. To address this, we applied a quality assessment framework to all variables50.”
From Real-time surveillance system for patient deterioration: a pragmatic cluster-randomized controlled trial (2025) · cites this paper as [50]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
1 retracted
WITHDRAWN - Mixed Methods Sampling: A Typology With Examples
Teddlie, C., & Yu, F. (2007) · Journal of Mixed Methods Research · doi:10.1177/2345678906292430 · Cited by 2,250 · see citing works
RetractedHow Many Interviews Are Enough?: An Experiment with Data Saturation and Variability
Guest, G., Bunce, A., & Johnson, L. (2006) · Field Methods · doi:10.1177/1525822x05279903 · Cited by 20,808 · see citing works
Fereday, J., & Muir-Cochrane, E. (2006) · International Journal of Qualitative Methods · doi:10.1177/160940690600500107 · Cited by 11,829 · see citing works
A General Inductive Approach for Analyzing Qualitative Evaluation Data
Thomas, D. R. (2006) · American Journal of Evaluation · doi:10.1177/1098214005283748 · Cited by 10,858 · see citing works
Beyond Accuracy: What Data Quality Means to Data Consumers
Wang, R. Y., & Strong, D. M. (1996) · Journal of Management Information Systems · doi:10.1080/07421222.1996.11518099 · Cited by 4,672 · see citing works
What is an adequate sample size? Operationalising data saturation for theory-based interview studies
Francis, J. J., Johnston, M., Robertson, C., Glidewell, L., Entwistle, V., Eccles, M. P., & Grimshaw, J. M. (2010) · Psychology & Health · doi:10.1080/08870440903194015 · Cited by 3,429 · see citing works
Effect of clinical guidelines on medical practice: a systematic review of rigorous evaluations
Grimshaw, J., & Russell, I. T. (1993) · The Lancet · doi:10.1016/0140-6736(93)92244-n · Cited by 2,791 · see citing works
External validity of randomised controlled trials: “To whom do the results of this trial apply?”
Rothwell, P. M. (2005) · The Lancet · doi:10.1016/s0140-6736(04)17670-8 · Cited by 2,544 · see citing works
Pipino, L. L., Lee, Y. W., & Wang, R. Y. (2002) · Communications of the ACM · doi:10.1145/505248.506010 · Cited by 1,724 · see citing works
(1999) · Cited by 1,648 · see citing works
Weiskopf, N. G., & Weng, C. (2013) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000681 · Cited by 1,216 · cited 3× · see citing works
“The details of this study can be found in Weiskopf and Weng, 2013.17 Our inclusion criteria were that the papers must describe methods of data quality assessment, that they focus on EHR- derived data, and that they were peer-reviewed.”
BackgroundMethodInfluentialSelf-citationKahn, M. G., Callahan, T. J., Barnard, J., Bauck, A. E., Brown, J., Davidson, B. N., Estiri, H., Goerg, C., Holve, E., Johnson, S. G., et al. (2016) · eGEMs (Generating Evidence & Methods to improve patient outcomes) · doi:10.13063/2327-9214.1244 · Cited by 636 · see citing works
Self-citationHersh, W. R., Weiner, M. G., Embi, P. J., Logan, J. R., Payne, P. R. O., Bernstam, E. V., Lehmann, H. P., Hripcsak, G., Hartzog, T. H., Cimino, J. J., et al. (2013) · Medical Care · doi:10.1097/mlr.0b013e31829b1dbd · Cited by 567 · see citing works
Self-citationExpressiveness and tractability in knowledge representation and reasoning 1
Levesque, H., & Brachman, R. (1987) · International Conference on Climate Informatics · doi:10.1111/j.1467-8640.1987.tb00176.x · Cited by 409 · see citing works
Accuracy of Data in Computer-based Patient Records
Hogan, W. R., & Wagner, M. M. (1997) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.1997.0040342 · Cited by 389 · see citing works
Defining and measuring completeness of electronic health records for secondary use
Weiskopf, N. G., Hripcsak, G., Swaminathan, S., & Weng, C. (2013) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2013.06.010 · Cited by 367 · see citing works
Self-citationChan, K. S., Fowles, J. B., & Weiner, J. P. (2010) · Medical Care Research and Review · doi:10.1177/1077558709359007 · Cited by 365 · see citing works
Systematic review of scope and quality of electronic patient record data in primary care
Thiru, K., Hassey, A., & Sullivan, F. (2003) · BMJ · doi:10.1136/bmj.326.7398.1070 · Cited by 342 · see citing works
Kahn, M. G., Raebel, M. A., Glanz, J. M., Riedlinger, K., & Steiner, J. F. (2012) · Medical Care · doi:10.1097/mlr.0b013e318257dd67 · Cited by 260 · see citing works
Rusanov, A., Weiskopf, N. G., Wang, S., & Weng, C. (2014) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-14-51 · Cited by 137 · see citing works
Self-citationQuick, Simple and Reliable: Forced Binary Survey Questions
Dolničar, S., Grün, B., & Leisch, F. (2011) · International Journal of Market Research · doi:10.2501/ijmr-53-2-231-252 · Cited by 124 · see citing works
Quality and correlates of medical record documentation in the ambulatory care setting
Soto, C. M., KLEINMAN, K. P., & Simon, S. R. (2002) · BMC Health Services Research · doi:10.1186/1472-6963-2-22 · Cited by 99 · see citing works
Bias Associated with Mining Electronic Health Records
Hripcsak, G., Knirsch, C., Zhou, L., Wilcox, A., & Melton, G. (2011) · Journal of Biomedical Discovery and Collaboration · doi:10.5210/disco.v6i0.3581 · Cited by 83 · see citing works
Self-citationData Quality Assessment Framework to Assess Electronic Medical Record Data for use in Research
Reimer, A. P., Milinovich, A., & Madigan, E. (2016) · Int. J. Medical Informatics · doi:10.1016/j.ijmedinf.2016.03.006 · Cited by 78 · see citing works
Application of an Ontology for Characterizing Data Quality for a Secondary Use of EHR Data
Speedie, S. M., Simon, G., Kumar, V., Westra, B. L., & Johnson, S. (2016) · Applied Clinical Informatics · doi:10.4338/aci-2015-08-ra-0107 · Cited by 63 · see citing works
Multisite Evaluation of a Data Quality Tool for Patient-Level Clinical Datasets
Huser, V., DeFalco, F., Schuemie, M. J., Ryan, P. B., Shang, N., Velez, M., Park, R. W., Boyce, R. D., Duke, J. D., Khare, R., et al. (2016) · eGEMs (Generating Evidence & Methods to improve patient outcomes) · doi:10.13063/2327-9214.1239 · Cited by 61 · see citing works
(2006) · PubMed Central · Cited by 53 · see citing works
Johnson, N., Mant, D., Jones, L., & Randall, T. (1991) · BMJ · doi:10.1136/bmj.302.6779.763 · Cited by 52 · see citing works
Iyen-Omofoman, B., Hubbard, R. B., Smith, C. J., Sparks, E., Bradley, E., Bourke, A., & Tata, L. J. (2011) · BMC Public Health · doi:10.1186/1471-2458-11-857 · Cited by 44 · see citing works
Faulconer, E., & DeLusignan, S. (2004) · Journal of Innovation in Health Informatics · doi:10.14236/jhi.v12i4.132 · Cited by 41 · see citing works
Brown, P. (2002) · International Journal of Medical Informatics · doi:10.1016/s1386-5056(02)00068-0 · Cited by 36 · cited 1× · see citing works
“Also consider the concordance between different variables (Cell 2B).(36,37) Concordance checks can be conceptualized as if-then rules.”
BackgroundStaes, C. J., Bennett, S. T., Evans, R. S., Narus, S. P., Huff, S. M., & Sorensen, J. B. (2006) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1813 · Cited by 24 · see citing works
Validating Emergency Department Vital Signs Using a Data Quality Engine for Data Warehouse
Genes, N. G., Chandra, D., Ellis, S. C., & Baumlin, K. M. (2013) · The Open Medical Informatics Journal · doi:10.2174/1874431101307010034 · Cited by 20 · see citing works
Quantifying the longitudinal value of healthcare record collections for pharmacoepidemiology.
Sperrin, M., Thew, S., Weatherall, J., Dixon, W. G., & Buchan, I. E. (2011) · PubMed Central · Cited by 15 · see citing works
Info-tsunami: surviving the storm with data quality probes
Brown, P., & Warmington, V. (2003) · Journal of Innovation in Health Informatics · doi:10.14236/jhi.v11i4.569 · Cited by 15 · see citing works
Data integrity in a general practice computer system (Clinics)
Basden, A., & Clark, E. M. (1980) · International Journal of Bio-Medical Computing · doi:10.1016/0020-7101(80)90017-3 · Cited by 8 · see citing works
Flagged first, then by citations
Research works printed but not linked: 2
- [5]Juran JM, Gryna FM. Juran’s quality control handbook 4th ed New York: McGraw-Hill; 1988.
- [26]Grant J, Davis L. Selection and use of content experts for instrument development. Research in Nursing & Health. 1997; 20(3): 269– 74. 9179180
Cited by
Sorted by citations
Electronic Health Record Data Quality Assessment and Tools: A Systematic Review
Lewis, A. E., Weiskopf, N., Abrams, Z. B., Foraker, R., Lai, A. M., Payne, P. R. O., & Gupta, A. (2023) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocad120 · Cited by 171 · see citing works
Self-citationIntegrating multi-omics data with EHR for precision medicine using advanced artificial intelligence
Tong, L., Shi, W., Isgut, M., Zhong, Y., Lais, P., Gloster, L., Sun, J., Swain, A., Giuste, F., & Wang, M. D. (2024) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2023.3324264 · Cited by 130 · see citing works
Schmidt, C. O., Struckmann, S., Enzenbach, C., Reineke, A., Stausberg, J., Damerow, S., Huebner, M., Schmidt, B., Sauerbrei, W., & Richter, A. (2021) · BMC Medical Research Methodology · doi:10.1186/s12874-021-01252-7 · Cited by 130 · see citing works
Digital Health Data Quality Issues: Systematic Review
Syed, R., Eden, R., Makasi, T., Chukwudi, I., Mamudu, A., Kamalpour, M., Geeganage, D. K., Sadeghianasl, S., Leemans, S. J. J., Goel, K., et al. (2023) · Journal of Medical Internet Research · doi:10.2196/42615 · Cited by 145 · see citing works
Carrigan, G. J., Whipple, S., Capra, W. B., Taylor, M. D., Brown, J. S., Lu, M. W., Arnieri, B., Copping, R., & Rothman, K. J. (2019) · Clinical Pharmacology & Therapeutics · doi:10.1002/cpt.1586 · Cited by 100 · see citing works
Bian, J., Lyu, T., Loiacono, A., Viramontes, T. M., Lipori, G., Guo, Y., Wu, Y., Prosperi, M., George, T. J., Harle, C. A., et al. (2020) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaa245 · Cited by 92 · see citing works
Automating Electronic Health Record Data Quality Assessment
Ozonze, O., Scott, P., & Hopgood, A. A. (2023) · Journal of Medical Systems · doi:10.1007/s10916-022-01892-2 · Cited by 76 · see citing works
Lynch, K. E., Deppen, S. A., DuVall, S. L., Viernes, B., Cao, A., Park, D., Hanchrow, E. E., Hewa, K., Greaves, P., & Matheny, M. E. (2019) · Applied Clinical Informatics · doi:10.1055/s-0039-1697598 · Cited by 62 · see citing works
Mitra, A., Pradhan, R., Melamed, R., Chen, K., Hoaglin, D. C., Tucker, K. L., Reisman, J. I., Yang, Z., Liu, W., Tsai, J., et al. (2023) · JAMA Network Open · doi:10.1001/jamanetworkopen.2023.3079 · Cited by 57 · see citing works
Rossetti, S. C., Dykes, P. C., Knaplund, C., Cho, S., Withall, J., Lowenthal, G., Albers, D., Lee, R. Y., Jia, H., Bakken, S., et al. (2025) · Nature Medicine · doi:10.1038/s41591-025-03609-7 · Cited by 54 · see citing works
Ehsani-Moghaddam, B., Martin, K., & Queenan, J. A. (2021) · Health Information Management Journal · doi:10.1177/1833358319887743 · Cited by 84 · see citing works
Edmondson, M. E., & Reimer, A. P. (2020) · CIN Computers Informatics Nursing · doi:10.1097/cin.0000000000000609 · Cited by 43 · see citing works
Leveraging Electronic Dental Record Data for Clinical Research in the National Dental PBRN Practices
Thyvalikakath, T. P., Duncan, W. D., Siddiqui, Z. A., LaPradd, M. L., Eckert, G. J., Schleyer, T., Rindal, D. B., Jurkovich, M. W., Shea, T. L., Gilbert, G. H., et al. (2020) · Applied Clinical Informatics · doi:10.1055/s-0040-1709506 · Cited by 45 · see citing works
Mind the clinical-analytic gap: Electronic health records and COVID-19 pandemic response
Sudat, S. E. K., Robinson, S. C., Mudiganti, S., Mani, A., & Pressman, A. R. (2021) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2021.103715 · Cited by 43 · see citing works
Challenges and recommendations for high quality research using electronic health records
Honeyford, K., Expert, P., Mendelsohn, E., Post, B., Faisal, A. A., Glampson, B., Mayer, E. K., & Costelloe, C. (2022) · Frontiers in Digital Health · doi:10.3389/fdgth.2022.940330 · Cited by 45 · see citing works
The ClearPath Method for Care Pathway Process Mining and Simulation
Johnson, O. A., Dhafari, T. B., Kurniati, A. P., Fox, F., & Rojas, E. (2019) · Lecture notes in business information processing · doi:10.1007/978-3-030-11641-5_19 · Cited by 39 · see citing works
Guidelines for Data Acquisition, Quality and Curation for Observational Research Designs (DAQCORD)
Ercole, A., Brinck, V., George, P., Hicks, R. R., Huijben, J. A., Jarrett, M., Vassar, M. J., Wilson, L., & collaborators, T. D. (2020) · Journal of Clinical and Translational Science · doi:10.1017/cts.2020.24 · Cited by 37 · see citing works
An automated data cleaning method for Electronic Health Records by incorporating clinical knowledge
Shi, X., Prins, C., Pottelbergh, G. V., Mamouris, P., Vaes, B., & Moor, B. D. (2021) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-021-01630-7 · Cited by 35 · see citing works
Data Quality in Health Care: Main Concepts and Assessment Methodologies
Mashoufi, M., Ayatollahi, H., Khorasani-Zavareh, D., & Talebi Azad Boni, T. (2023) · Methods of Information in Medicine · doi:10.1055/s-0043-1761500 · Cited by 32 · see citing works
Yin, A. L., Guo, W. L., Sholle, E., Rajan, M., Alshak, M. N., Choi, J. J., Goyal, P., Jabri, A., Li, H. A., Pinheiro, L. C., et al. (2021) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2021.104622 · Cited by 32 · see citing works
Madandola, O. O., Bjarnadóttir, R. I., Yao, Y., Ansell, M., Santos, F. C. D., Cho, H., Lopez, K. D., Macieira, T. G. R., & Keenan, G. M. (2023) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocad188 · Cited by 30 · see citing works
Risk of Suicidal Self-directed Violence Among US Veteran Survivors of Head and Neck Cancer
Nugent, S. M., Morasco, B. J., Handley, R., Clayburgh, D. R., Hooker, E. R., Ganzini, L. K., Knight, S. J., Chen, J. I., Sullivan, D. R., & Slatore, C. G. (2021) · JAMA Otolaryngology–Head & Neck Surgery · doi:10.1001/jamaoto.2021.2625 · Cited by 28 · see citing works
Dixon, B. E., Chen, W., French, T., Williams, J. L., Duke, J. D., & Grannis, S. J. (2020) · BMJ Health & Care Informatics · doi:10.1136/bmjhci-2019-100054 · Cited by 28 · see citing works
A method for interoperable knowledge-based data quality assessment
Tute, E., Scheffner, I., & Marschollek, M. (2021) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-021-01458-1 · Cited by 27 · see citing works
Mitra, A., Ahsan, H., Li, W., Liu, W., Kerns, R. D., Tsai, J., Becker, W. C., Smelson, D. A., & Yu, H. Q. (2021) · JMIR Medical Informatics · doi:10.2196/32851 · Cited by 26 · see citing works
McInerney, C. D., Kotzé, A., Bacon, S., Cutting, J., Fisher, L., Goldacre, B. M., Johnson, O. A., Kua, J., McGuckin, D., Mehrkar, A., et al. (2023) · Anaesthesia · doi:10.1111/anae.16001 · Cited by 26 · see citing works
Diaz‐Garelli, F., Bernstam, E. V., Lee, M., Hwang, K. O., Rahbar, M. H., & Johnson, T. R. (2019) · eGEMs (Generating Evidence & Methods to improve patient outcomes) · doi:10.5334/egems.286 · Cited by 26 · see citing works
Feasibility analysis of conducting observational studies with the electronic health record
von Lucadou, M., Ganslandt, T., Prokosch, H., & Toddenroth, D. (2019) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-019-0939-0 · Cited by 25 · see citing works
Razzaghi, H., Greenberg, J., & Bailey, L. C. (2021) · Learning Health Systems · doi:10.1002/lrh2.10264 · Cited by 25 · see citing works
TAQIH, a tool for tabular data quality assessment and improvement in the context of health data
Álvarez, R., Beristain, A., Epelde, G., & Carlin, P. F. (2018) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2018.12.029 · Cited by 24 · see citing works
Nagamine, T., Gillette, B. M., Kahoun, J., Burghaus, R., Lippert, J., & Saxena, M. (2022) · Scientific Reports · doi:10.1038/s41598-022-22398-4 · Cited by 23 · see citing works
Lin, J. S., Murad, M. H., Leas, B. F., Treadwell, J. R., Chou, R., Ivlev, I., & Kansagara, D. L. (2020) · Journal of General Internal Medicine · doi:10.1007/s11606-020-05783-5 · Cited by 23 · see citing works
Lifting Hospital Electronic Health Record Data Treasures: Challenges and Opportunities
Maletzky, A., Böck, C., Tschoellitsch, T., Roland, T., Ludwig, H., Thumfart, S., Giretzlehner, M., Hochreiter, S., & Meier, J. M. (2022) · JMIR Medical Informatics · doi:10.2196/38557 · Cited by 23 · see citing works
Miao, Z., Sealey, M. D., Sathyanarayanan, S., Delen, D., Zhu, L., & Shepherd, S. (2022) · Information Systems · doi:10.1016/j.is.2022.102130 · Cited by 20 · see citing works
Park, J., Zhong, X., Miley, E. N., Rutledge, R. S., Kakalecik, J., Johnson, M. C., & Gray, C. F. (2023) · Arthroplasty Today · doi:10.1016/j.artd.2023.101308 · Cited by 20 · see citing works
Electronic Health Record Data Quality and Performance Assessments: Scoping Review
Penev, Y. P., Buchanan, T. R., Ruppert, M. M., Liu, M., Shekouhi, R., Guan, Z., Balch, J. A., Ozrazgat‐Baslanti, T., Shickel, B., Loftus, T. J., et al. (2024) · JMIR Medical Informatics · doi:10.2196/58130 · Cited by 27 · see citing works
Addressing Note Bloat: Solutions for Effective Clinical Documentation
Genes, N. G., Sills, J., Heaton, H. A., Shy, B. D., & Scofi, J. (2025) · Journal of the American College of Emergency Physicians Open · doi:10.1016/j.acepjo.2024.100031 · Cited by 19 · see citing works
Patel, J. S., Shin, D., Willis, L., Zai, A., Kumar, K., & Thyvalikakath, T. P. (2023) · Scientific Reports · doi:10.1038/s41598-023-44307-z · Cited by 18 · see citing works
Ruan, X., Zhang, N., Wang, D., Huang, J., Huang, J., Huang, D., Chun, T. T. S., Ho, B., Ng, A. T., Tsu, J. H., et al. (2024) · JMIR Public Health and Surveillance · doi:10.2196/47161 · Cited by 17 · see citing works
McInerney, C. D., McCrorie, C., Benn, J. J., Habli, I., Lawton, T., Mebrahtu, T. F., Randell, R., Sheikh, N., & Johnson, O. A. (2022) · BMJ Open · doi:10.1136/bmjopen-2021-054090 · Cited by 17 · see citing works
Mariño, J., Kasbohm, E., Struckmann, S., Kapsner, L. A., & Schmidt, C. O. (2022) · Applied Sciences · doi:10.3390/app12094238 · Cited by 20 · see citing works
Patel, J. S., Brandon, R. G., Téllez, M., Albandar, J. M., Rao, R., Krois, J., & Wu, H. (2022) · Methods of Information in Medicine · doi:10.1055/s-0042-1757880 · Cited by 16 · see citing works
Looten, V., Chang, L. K. W., Neuraz, A., Loriot, M., Vedie, B., Paul, J., Mauge, L., Rivet, N., Bonifati, A., Châtellier, G., et al. (2018) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2018.12.030 · Cited by 16 · see citing works
Data quality assessment in healthcare, dimensions, methods and tools: a systematic review
Hosseinzadeh, E., Afkanpour, M., Momeni, M., & Tabesh, H. (2025) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-025-03136-y · Cited by 22 · see citing works
Developing a Data Quality Evaluation Framework for Sewer Inspection Data
Khaleghian, H., & Shan, Y. (2023) · Water · doi:10.3390/w15112043 · Cited by 15 · see citing works
Abdolkhani, R., Gray, K. M., Borda, A., & DeSouza, R. (2022) · JMIR mhealth and uhealth · doi:10.2196/35917 · Cited by 14 · see citing works
Razzaghi, H., Davies, A. G., Boss, S., Bunnell, H. T., Chen, Y., Chrischilles, E. A., Dickinson, K., Hanauer, D. A., Huang, Y., Ilunga, K. T. S., et al. (2024) · PLOS Digital Health · doi:10.1371/journal.pdig.0000527 · Cited by 14 · see citing works
Douthit, B. J., Fiol, G. D., Staes, C. J., Docherty, S. L., & Richesson, R. (2021) · Applied Clinical Informatics · doi:10.1055/s-0041-1732423 · Cited by 14 · see citing works
Quality Informatics: The Convergence of Healthcare Data, Analytics, and Clinical Excellence
Coppersmith, N. A., Sarkar, I. N., & Chen, E. S. (2019) · Applied Clinical Informatics · doi:10.1055/s-0039-1685221 · Cited by 15 · see citing works
Sexual orientation-related disparities in health conditions that elevate COVID-19 severity
Lynch, K. E., Shipherd, J. C., Gatsby, E., Viernes, B., DuVall, S. L., & Blosnich, J. R. (2021) · Annals of Epidemiology · doi:10.1016/j.annepidem.2021.11.006 · Cited by 14 · see citing works
Increasing Nursing Documentation Efficiency With Wearable Sensors for Pressure Injury Prevention
Rose, A., Cooley, A., Yap, T. L., Alderden, J., Sabol, V. K., Lin, J.-R., Brooks, K., & Kennerly, S. M. (2022) · Critical Care Nurse · doi:10.4037/ccn2022116 · Cited by 13 · see citing works
Privacy-protecting, reliable response data discovery using COVID-19 patient observations
Kim, J., Neumann, L., Paul, P., Day, M. E., Aratow, M., Bell, D. S., Doctor, J. N., Hinske, L. C., Jiang, X., Kim, K. K., et al. (2021) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocab054 · Cited by 13 · see citing works
Effective data quality management for electronic medical record data using SMART DATA
Lee, S., Roh, G.-H., Kim, J., Lee, Y. H., Woo, H., & Lee, S. (2023) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2023.105262 · Cited by 12 · see citing works
Zhang, Y., Callaghan‐Koru, J. A., & Koru, G. (2024) · JAMIA Open · doi:10.1093/jamiaopen/ooae058 · Cited by 15 · see citing works
The Venus score for the assessment of the quality and trustworthiness of biomedical datasets
Chicco, D., Fabris, A., & Jurman, G. (2025) · BioData Mining · doi:10.1186/s13040-024-00412-x · Cited by 14 · see citing works
Tian, Q., Han, Z., Yu, P., An, J.-Y., Lu, X.-D., & Duan, H. (2021) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-021-01481-2 · Cited by 11 · see citing works
Wang, H., Belitskaya‐Lévy, I., Wu, F., Lee, J. S., Shih, M., Tsao, P. S., Lü, Y., & Program, O. B. O. V. M. V. (2021) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-021-01643-2 · Cited by 12 · see citing works
Diaz‐Garelli, F., Strowd, R. E., III, Ahmed, T., Wells, B. J., Merrill, R., Laurini, J. A., Pasche, B., & Topaloĝlu, Ü. (2019) · JAMIA Open · doi:10.1093/jamiaopen/ooz020 · Cited by 11 · see citing works
Multi-Center Healthcare Data Quality Measurement Model and Assessment Using OMOP CDM
Kim, K., Choi, W., Ko, S., Chang, D. J., Chung, Y. W., Chang, S., Kim, J. K., Kim, D., & Choi, I. (2021) · Applied Sciences · doi:10.3390/app11199188 · Cited by 11 · see citing works
Integration of FHIR to Facilitate Electronic Case Reporting: Results from a Pilot Study
Dixon, B. E., Taylor David, E., Choi, M., Riley, M. J., Schneider, T., & Duke, J. D. (2019) · Studies in health technology and informatics · doi:10.3233/shti190362 · Cited by 11 · see citing works
Lamer, A., Fruchart, M., Paris, N., Popoff, B., Payen, A., Balcaen, T., Gacquer, W., Bouzillé, G., Marc Cuggia, A., Doutreligne, M., et al. (2022) · JMIR Medical Informatics · doi:10.2196/38936 · Cited by 10 · see citing works
DQAgui: a graphical user interface for the MIRACUM data quality assessment tool
Mang, J. M., Seuchter, S. A., Gülden, C., Schild, S., Kraska, D., Prokosch, H., & Kapsner, L. A. (2022) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-022-01961-z · Cited by 10 · see citing works
Ho, J. C., Staimez, L. R., Narayan, K. M. V., Lucila Ohno-machado, A., Simpson, R. L., & Hertzberg, V. (2022) · Computer Methods and Programs in Biomedicine Update · doi:10.1016/j.cmpbup.2022.100087 · Cited by 8 · see citing works
Strategies to Address Current Challenges in Real-World Evidence Generation in Japan
Laurent, T., Lambrelli, D., Wakabayashi, R., Hirano, T., & Kuwatsuru, R. (2023) · Drugs - Real World Outcomes · doi:10.1007/s40801-023-00371-5 · Cited by 10 · see citing works
An Expert Lens on Data Quality in Process Mining
Andrews, R., Emamjome, F., ter Hofstede, A. H. M., & Reijers, H. A. (2020) · International Conference on Process Mining (ICPM) · doi:10.1109/icpm49681.2020.00018 · Cited by 10 · see citing works
Data Quality in health records: A literature review
Iturry, M. D., Alves-Souza, S. N., Ito, M., & da Silva, S. A. (2021) · Iberian Conference on Information Systems and Technologies (CISTI) · doi:10.23919/cisti52073.2021.9476536 · Cited by 11 · see citing works
A data driven learning approach for the assessment of data quality
Tute, E., Ganapathy, N., & Wulff, A. (2021) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-021-01656-x · Cited by 10 · see citing works
An electronic health record cohort of Veterans with amyotrophic lateral sclerosis
Reimer, R. J., Gonçalves, A. R., Soper, B., Cadena, J., Wilson, J. L., Gryshuk, A. L., Suarez, P., Osborne, T. F., Grimes, K., & Ray, P. (2023) · Amyotrophic Lateral Sclerosis and Frontotemporal Degeneration · doi:10.1080/21678421.2023.2239300 · Cited by 9 · see citing works
Zhou, J., Hao, J., Tang, M., Sun, H., Wang, J., Li, J., & Qian, Q. (2024) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-024-02533-z · Cited by 10 · see citing works
Assessing EHR Data for Use in Clinical Improvement and Research
Lyons, A., Dimas, J. B., Richardson, S., & Sward, K. A. (2022) · AJN American Journal of Nursing · doi:10.1097/01.naj.0000832728.09164.3f · Cited by 9 · see citing works
A Standard Framework for Evaluating Large Health Care Data and Related Resources
Félix, S. E. B., Yusuf, H. R., Ritchey, M. D., Romano, S. D., Namulanda, G., Wilkins, N. J., & Boehmer, T. K. (2024) · MMWR Supplements · doi:10.15585/mmwr.su7303a1 · Cited by 10 · see citing works
Karch, A., Schindler, D., Kühn-Steven, A., Blaser, R., Kuhn, K. A., Sandmann, L., Sommerer, C., Guba, M. O., Heemann, U. W., Strohäker, J., et al. (2021) · European Journal of Epidemiology · doi:10.1007/s10654-020-00715-3 · Cited by 9 · see citing works
Tan, H. X., Teo, D. C. H., Lee, D. Y., Kim, C., Neo, J. W., Sung, C., Chahed, H., Ang, P. S., Tan, D. S., Park, R. W., et al. (2022) · Healthcare Informatics Research · doi:10.4258/hir.2022.28.2.112 · Cited by 9 · see citing works
Definition of a Practical Taxonomy for Referencing Data Quality Problems in Health Care Databases
Quindroit, P., Fruchart, M., Degoul, S., Périchon, R., Martignène, N., Soula, J., Marcilly, R., & Lamer, A. (2022) · Methods of Information in Medicine · doi:10.1055/a-1976-2371 · Cited by 10 · see citing works
A Data Element-Function Conceptual Model for Data Quality Checks
Rogers, J. R., Callahan, T. J., Kang, T., Bauck, A. E., Khare, R., Brown, J. S., Kahn, M. G., & Weng, C. (2019) · eGEMs (Generating Evidence & Methods to improve patient outcomes) · doi:10.5334/egems.289 · Cited by 8 · see citing works
Self-citationTa, C., Zucker, J. E., Chiu, P.-H., Fang, Y., Natarajan, K., & Weng, C. (2022) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocac208 · Cited by 7 · see citing works
Self-citationMohamed, Y., Song, X., McMahon, T. M., Sahil, S., Zozus, M. N., Wang, Z., & Waitman, L. R. (2022) · PubMed Central · Cited by 7 · see citing works
Wabo, G. K., Praßer, F., Gierend, K., Siegel, F. P., & Ganslandt, T. (2023) · JMIR Research Protocols · doi:10.2196/46471 · Cited by 5 · see citing works
Zaccaria, G. M., Ferrero, S., Rosati, S., Ghislieri, M., Genuardi, E., Evangelista, A., Sandrone, R., Castagneri, C., Barbero, D., Schirico, M. L., et al. (2019) · JCO Clinical Cancer Informatics · doi:10.1200/cci.19.00049 · Cited by 7 · see citing works
Data Quality of Chemotherapy-Induced Nausea and Vomiting Documentation
Beauchemin, M., Weng, C., Sung, L., Pichon, A., Abbott, M., Hershman, D. L., & Schnall, R. (2021) · Applied Clinical Informatics · doi:10.1055/s-0041-1728698 · Cited by 7 · see citing works
Self-citationKookal, K. K., Walji, M. F., Brandon, R. G., Kivanc, F., Mertz, E., Kottek, A. M., Mullins, J. M., Liang, S., Jenson, L. E., & White, J. M. (2024) · Journal of Public Health Dentistry · doi:10.1111/jphd.12618 · Cited by 6 · see citing works
Manias, G., Kouremenou, E., Azqueta-Alzúaz, A., Kranas, P., Melillo, F., & Kyriazis, D. (2023) · International Conference on Applied Mathematics & Computer Science (ICAMCS) · doi:10.1109/icamcs59110.2023.00015 · Cited by 6 · see citing works
Wassell, M., Vitiello, A., Butler-Henderson, K. A., Verspoor, K. M., McCann, P., & Pollard, H. P. (2024) · Journal of Occupational Rehabilitation · doi:10.1007/s10926-024-10175-1 · Cited by 6 · see citing works
Data Completeness and Concordance in the FeverApp Registry: Comparative Study
Rathjens, L., Fingerhut, I., Martin, D. D., Kerdar, S. H., Gwiasda, M., Schwarz, S., & Jenetzky, E. (2022) · JMIR Pediatrics and Parenting · doi:10.2196/35510 · Cited by 6 · see citing works
Wabo, G. K., Moorthy, P., Siegel, F. P., Seuchter, S. A., & Ganslandt, T. (2024) · JMIR Medical Informatics · doi:10.2196/57153 · Cited by 5 · see citing works
Implementing an artificial intelligence command centre in the NHS: a mixed-methods study
Johnson, O. A., McCrorie, C., McInerney, C. D., Mebrahtu, T. F., Granger, J., Sheikh, N., Lawton, T., Habli, I., Randell, R., & Benn, J. J. (2024) · Health and Social Care Delivery Research · doi:10.3310/tatm3277 · Cited by 9 · see citing works
Digital Health and Healthcare Quality: A Primer on the Evolving 4th Industrial Revolution
Otokiti, A. U. (2020) · IntechOpen eBooks · doi:10.5772/intechopen.94054 · Cited by 5 · see citing works
Tute, E., Mast, M., & Wulff, A. (2023) · Methods of Information in Medicine · doi:10.1055/s-0042-1760238 · Cited by 5 · see citing works
Prevalence and correlates of high‐dose opioid use among survivors of head and neck cancer
Nugent, S. M., Slatore, C. G., Winchell, K. A., Handley, R., Clayburgh, D. R., Chandra, R. A., Hooker, E. R., Knight, S. J., & Morasco, B. J. (2023) · Head & Neck · doi:10.1002/hed.27432 · Cited by 4 · see citing works
Review of Clinical Research Informatics
Solomonides, A. E. (2020) · Yearbook of Medical Informatics · doi:10.1055/s-0040-1701988 · Cited by 4 · see citing works
Zhang, W., Sotoodeh, M., Ho, J. C., Simpson, R. L., & Hertzberg, V. (2021) · Applied Clinical Informatics · doi:10.1055/s-0041-1735179 · Cited by 4 · see citing works
Nair, S. S., Li, C., Doijad, R., Nagy, P. G., Lehmann, H. P., & Kharrazi, H. H. K. (2021) · JAMIA Open · doi:10.1093/jamiaopen/ooab106 · Cited by 4 · see citing works
Real-World Data for Interdisciplinary Health Care Research
Nahm, E., Zhu, S., Seidl, K. L., Chen, L., Day, J., & Seong, H. (2023) · Advances in Nursing Science · doi:10.1097/ans.0000000000000496 · Cited by 3 · see citing works
Nakikj, D., Kreda, D., & Gehlenborg, N. (2023) · JMIR Human Factors · doi:10.2196/41552 · Cited by 4 · see citing works
Big data: challenges and opportunities within respiratory care
Markus, A. F., Arinze, J. T., & Verhamme, K. M. C. (2023) · European Respiratory Society eBooks · doi:10.1183/2312508x.10000723 · Cited by 4 · see citing works
Yang, L., Ren, M., Sun, S., Lu, J., & Wu, Y. (2024) · JAMIA Open · doi:10.1093/jamiaopen/ooae142 · Cited by 3 · see citing works
Understanding Clinician EHR Data Quality for Reuse in Predictive Modelling
Wassell, M., Murray, J. L., Kumar, C., Verspoor, K. M., & Butler-Henderson, K. A. (2024) · Studies in health technology and informatics · doi:10.3233/shti230949 · Cited by 3 · see citing works
Rödle, W., Prokosch, H., Neumann, E., Toni, I., Haering-Zahn, J., Neubert, A. C., & Eberl, S. (2024) · Applied Clinical Informatics · doi:10.1055/s-0043-1777741 · Cited by 3 · see citing works
Public Health Informatics: The Path Forward
Magnuson, J. A., & Dixon, B. E. (2020) · Health informatics · doi:10.1007/978-3-030-41215-9_27 · Cited by 3 · see citing works
Utilizing Electronic Dental Record Data to Track Periodontal Disease Change
Patel, J. S. (2020) · IUScholarWorks (Indiana University) · doi:10.7912/c2/958 · Cited by 3 · see citing works
- Authors
- Weiskopf, N. G., Bakken, S., Hripcsak, G., & Weng, C.
- Affiliations
- Oregon Health & Science University (US)
- Columbia University (US)
- Venue
- eGEMs (Generating Evidence & Methods to improve patient outcomes)
- Publisher
- Ubiquity Press, Ltd.
- Published
- 2017-09-04
- Volume
- 5
- Issue
- 1
- Pages
- 14
- ISSN
- 2327-9214 (ISSN-L)
- PMC
- PMC5983018
- OpenAlex
- W2752570039
- Cited by
- 152
- References
- 37
- Citations by year
- 2026: 10 · 2025: 16 · 2024: 20 · 2023: 24 · 2022: 26 · 2021: 28 · 2020: 14 · 2019: 12 · 2018: 2
- FWCI
- 10.51
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- cc-by-nc-nd
- Locations
- eGEMs (Generating Evidence & Methods to improve patient outcomes) (publishedVersion, cc-by-nc-nd)
- Europe PMC (PubMed Central) (publishedVersion, cc-by-nc-nd)
- PubMed Central (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Topics
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Health Sciences › Health Professions › Health Information Management › Medical Coding and Health Information
- Social Sciences › Decision Sciences › Management Science and Operations Research › Data Quality and Management
- Keywords
- electronic health records, data quality assessment, data reuse, clinical research data, data completeness, data correctness, data timeliness, data quality reporting, research guidelines, expert review
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
