Data quality probes—exploiting and improving the quality of electronic patient record data and patient care
Abstract
Increasing reliance is being placed on electronic medical records to support clinical care and achieve improved quality standards. In order for clinical information systems (CIS) to deliver excellence the data within it needs to be complete, consistent and accurate. This capture of data is critical but forms only part of the procedure in delivering quality health care during the clinician-patient encounter. A number of processes are involved in this encounter, each of which has to be performed flawlessly to deliver a perfect outcome. This paper outlines a method of assessing the quality of these processes involved in healthcare provision and data quality within a CIS. It proposes the principle of Data Quality Probes (DQP) to assess the performance of the whole encounter system. The main feature of this is the generation of a query which clinical knowledge predicts should not retrieve any cases in a system performing flawlessly. Any cases retrieved (which fail the DQP) indicate an error in either data quality or clinical judgment. This approach is applied practically within the paradigm of a UK family practice testing the hypothesis that a series DQPs can provide a valuable method for monitoring both the data accuracy of a CIS and the provision of quality patient care.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Brown and Warmington's data quality probes could be extended to various data elements, although they require detailed clinical knowledge to implement.
“Brown and Warmington's66 67 data quality probes, for example, could be extended to various data elements, although they require detailed clinical knowledge to implement.”
From Methods and dimensions of electronic health record data quality assessment: enabling reuse for clinical research (2013) · cites this paper as [66]
Cited by
Sorted by citations
Weiskopf, N. G., & Weng, C. (2013) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000681 · Cited by 1,216 · see citing works
Richesson, R. L., Hammond, W., Nahm, M., Wixted, D., Simon, G., Robinson, J., Bauck, A. E., Cifelli, D., Smerek, M. M., Dickerson, J. F., et al. (2013) · JAMIA Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2013-001926 · Cited by 204 · see citing works
A Data Quality Assessment Guideline for Electronic Health Record Data Reuse
Weiskopf, N. G., Bakken, S., Hripcsak, G., & Weng, C. (2017) · eGEMs (Generating Evidence & Methods to improve patient outcomes) · doi:10.5334/egems.218 · Cited by 152 · see citing works
“Also consider the concordance between different variables (Cell 2B).(36,37) Concordance checks can be conceptualized as if-then rules.”
BackgroundData Quality Assessment for Comparative Effectiveness Research in Distributed Data Networks
Brown, J. S., Kahn, M., & Toh, S. (2013) · Medical Care · doi:10.1097/mlr.0b013e31829b1e2c · Cited by 131 · see citing works
A large French prospective cohort of HIV‐infected patients: the Nadis Cohort
Puglièse, P., Cuzin, L., Cabié, A., Poizot-Martin, I., Allavena, C., Duvivier, C., Guedj, M. E., de la Tribonniére, X., Valantin, M., & Dellamonica, P. (2009) · HIV Medicine · doi:10.1111/j.1468-1293.2009.00719.x · Cited by 127 · see citing works
Data Quality Assessment Framework to Assess Electronic Medical Record Data for use in Research
Reimer, A. P., Milinovich, A., & Madigan, E. (2016) · Int. J. Medical Informatics · doi:10.1016/j.ijmedinf.2016.03.006 · Cited by 78 · see citing works
“Another approach is the Data Quality Probe method proposed by Brown and Warmington [10] that identifies cases in an EMR system that are not successfully matched (concordance), or contain errors between one item and another, that when applied longitudinally can improve data entry practice and…”
MethodAdoption factors of electronic health record systems
Spatar, D., Kok, O. M., Basoglu, N., & Daim, T. (2019) · Technology and Society · doi:10.1016/j.techsoc.2019.101144 · Cited by 68 · see citing works
Ethical Issues and the Electronic Health Record
Layman, E. (2008) · Health Care Supervisor · doi:10.1097/01.hcm.0000285044.19666.a8 · Cited by 57 · see citing works
“Little research has been reported on routine monitoring of data accuracy.(56,57) Moreover, no standard methodology for the study of data accuracy had been established.”
MethodCommon data quality elements for health information systems: a systematic review
Ghalavand, H., Shirshahi, S., Rahimi, A., Zarrinabadi, Z., & Amani, F. (2024) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-024-02644-7 · Cited by 44 · see citing works
“The quality of healthcare depends on the existence of quality data, which ultimately leads to a significant impact on customer satisfaction [13, 19].”
BackgroundA Rule-Based Data Quality Assessment System for Electronic Health Record Data
Wang, Z., Talburt, J., Wu, N., Dagtas, S., & Zozus, M. (2020) · Applied Clinical Informatics · doi:10.1055/s-0040-1715567 · Cited by 42 · see citing works
“23 Data quality probes detected errors, tracked the prevalence of these cases, monitored the EHR quality, and gave feedback to clinicians.”
BackgroundMethodMa, S., Lawpoolsri, S., Soonthornworasiri, N., Khamsiriwatchara, A., Jandee, K., Taweeseneepitch, K., Pawarana, R., Jaiklaew, S., Kijsanayotin, B., & Kaewkungwal, J. (2016) · JMIR Public Health and Surveillance · doi:10.2196/publichealth.5347 · Cited by 32 · see citing works
TEMPEST: An integrative model for health technology assessment
Currie, W. (2012) · Health Policy and Technology · doi:10.1016/j.hlpt.2012.01.004 · Cited by 30 · see citing works
“Evaluation studies from the health informatics community focus on benefits and risks from health technology to the user community, which includes, clinicians, nurses, administrators and patients [42–44].”
BackgroundRule-Based Data Quality Assessment and Monitoring System in Healthcare Facilities
Wang, Z., Dagtas, S., Talburt, J., Baghal, A., & Zozus, M. (2019) · Itch · doi:10.3233/978-1-61499-951-5-460 · Cited by 20 · see citing works
“Inspired by the rule abstraction in Brown’s work [14], we sorted the rules according to patterns in the rule logic.”
BackgroundMethodBenefits of the DICOM Modality Performed Procedure Step
Noumeir, R. (2005) · Journal of digital imaging · doi:10.1007/s10278-005-6702-3 · Cited by 19
“Access to relevant clinical information as soon as possible, information accuracy, and information completeness will turn out to be much more critical with wider EPR deployment and adoption after solutions for sharing information become readily available.(2) But accuracy and completeness of information cannot be achie…”
BackgroundInfo-tsunami: surviving the storm with data quality probes
Brown, P., & Warmington, V. (2003) · Journal of Innovation in Health Informatics · doi:10.14236/jhi.v11i4.569 · Cited by 15 · see citing works
Self-citationTian, Q., Han, Z., Yu, P., An, J.-Y., Lu, X.-D., & Duan, H. (2021) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-021-01481-2 · Cited by 11 · see citing works
An overview of electronic health information management systems quality assessment
Bouamrane, M.-M., Mair, F. S., & Tao, C. (2012) · International Workshop on Managing Interoperability and Complexity in Health Systems (MIXHS) · doi:10.1145/2389672.2389680 · Cited by 9 · see citing works
“We use the model of the patient / clinician encounter of Brown & Warmington (2002) to describe how issues of poor data quality and information mismanagement impact on the clinical encounter.”
BackgroundMethodInfluentialData Quality in Traditional Chinese Medicine
Feng, Y., Wu, Z., Chen, H., Yu, T., Mao, Y.-X., & Jiang, X.-H. (2008) · International Conference on BioMedical Engineering and Informatics · doi:10.1109/bmei.2008.268 · Cited by 8
“978-0-7695-3118-2/08 $25.00 © 2008 IEEE DOI 10.1109/BMEI.2008.268 255 In the past few years, there are a large number of researches discussing data quality in medical informatics [6][7][8][9].”
BackgroundInfluentialPuentes, J., Batrancourt, B., Atif, J., Angelini, E., Lecornu, L., Zemirline, A., Bloch, I., Coatrieux, G., & Roux, C. (2008) · Comput. Biol. Medicine · doi:10.1016/j.compbiomed.2008.01.009 · Cited by 8
Xie, S., Cai, H.-L., Sun, Y., & Lv, X. (2025) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2025.104951 · Cited by 7 · see citing works
Factors impacting physician use of information charted by others
Zozus, M., Penning, M. L., & Hammond, W. (2018) · JAMIA Open · doi:10.1093/jamiaopen/ooy041 · Cited by 7 · see citing works
Ethical Issues and the Electronic Health Record
Layman, E. (2020) · Health Care Supervisor · doi:10.1097/hcm.0000000000000302 · Cited by 6 · see citing works
“Little research has been reported on routine monitoring of data accuracy.(56,57) Moreover, no standard methodology for the study of data accuracy had been established.”
MethodRules Based Data Quality Assessment on Claims Database
Gadde, M., Wang, Z., Zozus, M., Talburt, J., & Greer, M. L. (2020) · Studies in Health Technology and Informatics · doi:10.3233/shti200567 · Cited by 2 · see citing works
Secured Electronic Patient Records Content Exploitation
Puentes, J., Coatrieux, G., & Lecornu, L. (2007) · Computers in health care · doi:10.1007/978-0-387-49009-0_12 · Cited by 2 · see citing works
“However, significant changes are still necessary to facilitate the adoption of an EPR [7-9].”
Integrating electronic health records
Currie, W., Finnegan, D., & Hamid, K. A. A. (2018) · CRC Press eBooks · doi:10.4324/b21767-7 · Cited by 1 · see citing works
Enhancing Electronic Patient Record Functionality Through Information Extraction from Images
Puentes, J., Batrancourt, B., Lecornu, L., Atif, J., Zemirline, A., Coatrieux, G., Angelini, E. D., Bloch, I., & Roux, C. (2006) · 2006 2nd International Conference on Information & Communication Technologies · doi:10.1109/ictta.2006.1684507 · Cited by 1
“Despite the incremental use of EPRs in hospital infornation systems, and various efforts to conceptualize models adapted to clinical practice [1] [2] [3] [4], significant progress is still necessary to use EPRs beyond the stage of data listing tool [5].”
BackgroundData Quality for Knowledge Discovery in Traditional Chinese Medicine
Wu, Z.-H., Chen, H.-J., & Jiang, X.-H. (2012) · doi:10.1016/b978-0-12-398510-1.00004-2
- Authors
- Brown, P.
- Affiliations
- St Thomas' Hospital (GB)
- Corresponding author
- Brown, P
- ORCID
- Brown, P
- Venue
- International Journal of Medical Informatics
- Publisher
- Elsevier BV
- Published
- 2002-12-18
- Volume
- 68
- Issue
- 1-3
- Pages
- 91-98
- ISSN
- 1386-5056 (ISSN-L)
- OpenAlex
- W2080635269
- Cited by
- 36
- References
- 0
- Citations by year
- 2025: 1 · 2024: 1 · 2021: 5 · 2020: 2 · 2019: 2 · 2018: 2 · 2017: 1 · 2016: 3 · 2014: 1 · 2013: 2 · 2012: 4
- FWCI
- 1.76
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- International Journal of Medical Informatics (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Health Sciences › Health Professions › Health Information Management › Medical Coding and Health Information
- Social Sciences › Decision Sciences › Management Science and Operations Research › Data Quality and Management
- Keywords
- electronic health records, clinical information systems, data quality, clinical judgment, quality of care, general practice, query generation, data accuracy, quality monitoring
- MeSH terms
- Delivery of Health Care, Family Practice, Female, United Kingdom, Humans, Information Systems, Male, Terminology as Topic, Physician-Patient Relations, Quality Assurance, Health Care, Research Design, Information Storage and Retrieval, Medical Records Systems, Computerized
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
