CoChat

Factors Influencing Data Quality in Electronic Health Record Systems in 50 Health Facilities in Rwanda and the Role of Clinical Alerts: Cross-Sectional Observational Study

Open access

Published version of a preprint

The DOI matches its Crossref record; 34 of the 35 references it cites were found in a registry or an index; 23 of the 46 sources its claims cite were checked for support.

Abstract

BACKGROUND: Electronic health records (EHRs) play an increasingly important role in delivering HIV care in low- and middle-income countries. The data collected are used for direct clinical care, quality improvement, program monitoring, public health interventions, and research. Despite widespread EHR use for HIV care in African countries, challenges remain, especially in collecting high-quality data. OBJECTIVE: We aimed to assess data completeness, accuracy, and timeliness compared to paper-based records, and factors influencing data quality in a large-scale EHR deployment in Rwanda. METHODS: We randomly selected 50 health facilities (HFs) using OpenMRS, an EHR system that supports HIV care in Rwanda, and performed a data quality evaluation. All HFs were part of a larger randomized controlled trial, with 25 HFs receiving an enhanced EHR with clinical decision support systems. Trained data collectors visited the 50 HFs to collect 28 variables from the paper charts and the EHR system using the Open Data Kit app. We measured data completeness, timeliness, and the degree of matching of the data in paper and EHR records, and calculated concordance scores. Factors potentially affecting data quality were drawn from a previous survey of users in the 50 HFs. RESULTS: We randomly selected 3467 patient records, reviewing both paper and EHR copies (194,152 total data items). Data completeness was >85% threshold for all data elements except viral load (VL) results, second-line, and third-line drug regimens. Matching scores for data values were close to or >85% threshold, except for dates, particularly for drug pickups and VL. The mean data concordance was 10.2 (SD 1.28) for 15 (68%) variables. HF and user factors (eg, years of EHR use, technology experience, EHR availability and uptime, and intervention status) were tested for correlation with data quality measures. EHR system availability and uptime was positively correlated with concordance, whereas users' experience with technology was negatively correlated with concordance. The alerts for missing VL results implemented at 11 intervention HFs showed clear evidence of improving timeliness and completeness of initially low matching of VL results in the EHRs and paper records (11.9%-26.7%; P<.001). Similar effects were seen on the completeness of the recording of medication pickups (18.7%-32.6%; P<.001). CONCLUSIONS: The EHR records in the 50 HFs generally had high levels of completeness except for VL results. Matching results were close to or >85% threshold for nondate variables. Higher EHR stability and uptime, and alerts for entering VL both strongly improved data quality. Most data were considered fit for purpose, but more regular data quality assessments, training, and technical improvements in EHR forms, data reports, and alerts are recommended. The application of quality improvement techniques described in this study should benefit a wide range of HFs and data uses for clinical care, public health, and disease surveillance.

Overview

CoChat Summary

## TL;DR

This cross-sectional study assessed data quality in OpenMRS HIV records across 50 health facilities in Rwanda by comparing paper charts with EHR data for completeness, matching, and concordance. Data were generally complete, but viral load fields and date fields were weaker, and clinical alerts for missing viral load results improved completeness and matching.

## Key findings

  • The study reviewed 3,467 patient records and 194,152 total data items across 50 health facilities; the mean was 69.3 (SD 8.8) records per facility.
  • Data completeness was >85% for all data elements except viral load results, second-line, and third-line drug regimens; paper viral load records were 20% to 30% more complete than EHR records.
  • The mean concordance score was 10.2 (SD 1.28) out of 15 variables, and only 2 health facilities met the 85% concordance threshold.
  • EHR availability/uptime was positively correlated with concordance (r=0.16; P<.001), while higher user technology experience was negatively correlated (r=−0.11; P<.001).
  • In facilities with viral load alerts, viral load records matched in 26.7% (85/318) of cases versus 11.9% (120/1008) without alerts (P<.001); drug pickup matches were 36.2% (280/774) with alerts versus 18.7% (183/981) without alerts (P<.001).

## Methods

  • Cross-sectional observational study using an LQAS approach in 50 of 112 health facilities from a larger randomized controlled trial of OpenMRS in Rwanda; facilities included both intervention and control sites, but not hospitals.
  • Trained data collectors extracted 28 variables from both paper charts and EHRs for randomly selected HIV patient records; the sample included 48 to 76 charts per facility depending on patient volume.
  • Main analyses measured completeness, matching, and concordance; correlations with facility characteristics used Pearson correlation with Bonferroni correction, and alert effects were tested with chi-square tests and Pearson contingency coefficients.

## Limitations

  • The paper states the study was limited to a subset of MOH-run OpenMRS EHR systems for HIV care, and the original inclusion criteria favored facilities with better IT hardware and successful server monitoring tool implementation.
  • The authors note that the apparent benefit of alerts in intervention facilities might change over time, and that differences in training emphasis on viral load could have contributed to the observed effects.
  • The text makes evident that this is a cross-sectional assessment with only a 4- to 5-month window after intervention rollout for some analyses; therefore, long-term causal effects are not established.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.2196/49127

Cites

34 of 34 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Figure 1 Correlation between concordance scores and the reported electronic health record (EHR) availability or uptime (ranging from 1 to 5 Likert scale responses of EHR users).
Figure 1 Correlation between concordance scores and the reported electronic health record (EHR) availability or uptime (ranging from 1 to 5 Likert scale responses of EHR users).CC BY · PubMed Central
Figure 2 Mean concordance scores for all health facilities color coded by intervention status (maximum score=15).
Figure 2 Mean concordance scores for all health facilities color coded by intervention status (maximum score=15).CC BY · PubMed Central
Figure 3 Graph showing percentage data matches for viral load results between paper records and electronic health records for each of the 25 health care centers in the intervention arm, ranked by percentage of matches.
Figure 3 Graph showing percentage data matches for viral load results between paper records and electronic health records for each of the 25 health care centers in the intervention arm, ranked by percentage of matches.CC BY · PubMed Central
Figure 4 Graph showing percentage data matches for drug pickups between paper records and electronic health records among the same 25 intervention health facilities (HFs) as in Figure 3. The overall scores were higher than in Figure 3 but only 1 (intervention) HF score was more than the data quality threshold.
Figure 4 Graph showing percentage data matches for drug pickups between paper records and electronic health records among the same 25 intervention health facilities (HFs) as in Figure 3. The overall scores were higher than in Figure 3 but only 1 (intervention) HF score was more than the data quality threshold.CC BY · PubMed Central

Verification

Claims the paper makes

19 Valid · 1 Not valid · 13 Not checked · 52 Own results and interpretations

27 more claims

Create a free account to see all 85 claims and to verify the 11 references still open.

See all claims — free account
  • It may state two separate claims as one. 39
  • Probably not a substantial claim. 34

Create a free account to see these claims.

Citations

Cites

Show all 34 references

Research works printed but not linked: 6

  1. Amoroso, CL, 2010, Stud Health Technol Inform, 160, 337
  2. Fraser, HS, 2010, AMIA Annu Symp Proc, 2010, 232
  3. Blaya, JA, 2010, Int J Tuberc Lung Dis, 14, 1009
  4. Wolfe, BA, 2006, AMIA Annu Symp Proc, 2006, 1146
  5. Allen, C, 2007, Stud Health Technol Inform, 129, 382
  6. Fraser, HS, 2021, ICT in Health: Survey on the Use of Information and Communication Technologies in Brazil Healthcare Facilities, 273

Cited by

See all 20

Metadata

Authors
Fraser, H. S. F., Mugisha, M., Bacher, I., Ngenzi, J. L., Seebregts, C., Umubyeyi, A., & Condo, J.
Affiliations
  • Brown University (US)
  • University of Rwanda (RW)
  • University of Cape Town (ZA)
  • Tulane University (US)
Corresponding author
Fraser, Hamish S F
Venue
JMIR Public Health and Surveillance
Published
2024-07-03
Volume
10
Pages
e49127
ISSN
2369-2960 (ISSN-L)
OpenAlex
W4388467758
Cited by
20
References
35
Citations by year
2026: 12 · 2025: 7 · 2024: 1
FWCI
4.48
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • JMIR Public Health and Surveillance (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
Funders
  • Brown University
  • U.S. President’s Emergency Plan for AIDS Relief
  • University of Rwanda
  • European Commission (661289)
  • Rwanda Biomedical Centre
  • Centers for Disease Control and Prevention (U01GH000782)
Topics
  • Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
  • Health Sciences › Health Professions › Health Information Management › Medical Coding and Health Information
  • Physical Sciences › Computer Science › Information Systems › Data Quality and Management
Keywords
electronic health records, data quality, clinical decision support, clinical alerts, HIV care, OpenMRS, Open Data Kit, data completeness, data timeliness, data concordance, paper records, Rwanda, low- and middle-income countries, health information system evaluation
MeSH terms
Data Accuracy, Cross-Sectional Studies, Health Facilities, Humans, Rwanda, HIV Infections, Electronic Health Records

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.