Factors Influencing Data Quality in Electronic Health Record Systems in 50 Health Facilities in Rwanda and the Role of Clinical Alerts: Cross-Sectional Observational Study
Open accessPublished version of a preprint
The DOI matches its Crossref record; 34 of the 35 references it cites were found in a registry or an index; 23 of the 46 sources its claims cite were checked for support.
Abstract
BACKGROUND: Electronic health records (EHRs) play an increasingly important role in delivering HIV care in low- and middle-income countries. The data collected are used for direct clinical care, quality improvement, program monitoring, public health interventions, and research. Despite widespread EHR use for HIV care in African countries, challenges remain, especially in collecting high-quality data. OBJECTIVE: We aimed to assess data completeness, accuracy, and timeliness compared to paper-based records, and factors influencing data quality in a large-scale EHR deployment in Rwanda. METHODS: We randomly selected 50 health facilities (HFs) using OpenMRS, an EHR system that supports HIV care in Rwanda, and performed a data quality evaluation. All HFs were part of a larger randomized controlled trial, with 25 HFs receiving an enhanced EHR with clinical decision support systems. Trained data collectors visited the 50 HFs to collect 28 variables from the paper charts and the EHR system using the Open Data Kit app. We measured data completeness, timeliness, and the degree of matching of the data in paper and EHR records, and calculated concordance scores. Factors potentially affecting data quality were drawn from a previous survey of users in the 50 HFs. RESULTS: We randomly selected 3467 patient records, reviewing both paper and EHR copies (194,152 total data items). Data completeness was >85% threshold for all data elements except viral load (VL) results, second-line, and third-line drug regimens. Matching scores for data values were close to or >85% threshold, except for dates, particularly for drug pickups and VL. The mean data concordance was 10.2 (SD 1.28) for 15 (68%) variables. HF and user factors (eg, years of EHR use, technology experience, EHR availability and uptime, and intervention status) were tested for correlation with data quality measures. EHR system availability and uptime was positively correlated with concordance, whereas users' experience with technology was negatively correlated with concordance. The alerts for missing VL results implemented at 11 intervention HFs showed clear evidence of improving timeliness and completeness of initially low matching of VL results in the EHRs and paper records (11.9%-26.7%; P<.001). Similar effects were seen on the completeness of the recording of medication pickups (18.7%-32.6%; P<.001). CONCLUSIONS: The EHR records in the 50 HFs generally had high levels of completeness except for VL results. Matching results were close to or >85% threshold for nondate variables. Higher EHR stability and uptime, and alerts for entering VL both strongly improved data quality. Most data were considered fit for purpose, but more regular data quality assessments, training, and technical improvements in EHR forms, data reports, and alerts are recommended. The application of quality improvement techniques described in this study should benefit a wide range of HFs and data uses for clinical care, public health, and disease surveillance.
Overview
CoChat Summary
## TL;DR
This cross-sectional study assessed data quality in OpenMRS HIV records across 50 health facilities in Rwanda by comparing paper charts with EHR data for completeness, matching, and concordance. Data were generally complete, but viral load fields and date fields were weaker, and clinical alerts for missing viral load results improved completeness and matching.
## Key findings
- The study reviewed 3,467 patient records and 194,152 total data items across 50 health facilities; the mean was 69.3 (SD 8.8) records per facility.
- Data completeness was >85% for all data elements except viral load results, second-line, and third-line drug regimens; paper viral load records were 20% to 30% more complete than EHR records.
- The mean concordance score was 10.2 (SD 1.28) out of 15 variables, and only 2 health facilities met the 85% concordance threshold.
- EHR availability/uptime was positively correlated with concordance (r=0.16; P<.001), while higher user technology experience was negatively correlated (r=−0.11; P<.001).
- In facilities with viral load alerts, viral load records matched in 26.7% (85/318) of cases versus 11.9% (120/1008) without alerts (P<.001); drug pickup matches were 36.2% (280/774) with alerts versus 18.7% (183/981) without alerts (P<.001).
## Methods
- Cross-sectional observational study using an LQAS approach in 50 of 112 health facilities from a larger randomized controlled trial of OpenMRS in Rwanda; facilities included both intervention and control sites, but not hospitals.
- Trained data collectors extracted 28 variables from both paper charts and EHRs for randomly selected HIV patient records; the sample included 48 to 76 charts per facility depending on patient volume.
- Main analyses measured completeness, matching, and concordance; correlations with facility characteristics used Pearson correlation with Bonferroni correction, and alert effects were tested with chi-square tests and Pearson contingency coefficients.
## Limitations
- The paper states the study was limited to a subset of MOH-run OpenMRS EHR systems for HIV care, and the original inclusion criteria favored facilities with better IT hardware and successful server monitoring tool implementation.
- The authors note that the apparent benefit of alerts in intervention facilities might change over time, and that differences in training emphasis on viral load could have contributed to the observed effects.
- The text makes evident that this is a cross-sectional assessment with only a 4- to 5-month window after intervention rollout for some analyses; therefore, long-term causal effects are not established.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures




Verification
Claims the paper makes
19 Valid · 1 Not valid · 13 Not checked · 52 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“Previous studies suggest that poor completion or accuracy of key data items required for clinical care is likely to limit the benefit of EHR systems in day-to-day patient management, such as the functioning of clinical decision support system in this study and that reported by Haskew et al [41].”
§ Factors Influencing Data Quality in Electronic Health Record Systems in 50 Health Facilities in Rwanda and the Role of Clinical Alerts: Cross-Sectional Observational Study > Discussion > Comparison to Previous Work
What the sources say
- Supportedcites [41] source
The Methods section ('Data verification and clinical decision support'; page not available) describes the clinical decision-support system reported by Haskew et al. It says the system flagged missing data and displayed reminders to clinicians until corrective action was taken, supporting the citation's use as an example of such a system. (LLM, conf=0.88)
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“The inclusion criteria for the 112 HFs in the larger study favored HFs with better information technology hardware and successful implementation of a server monitoring tool [4].”
§ Factors Influencing Data Quality in Electronic Health Record Systems in 50 Health Facilities in Rwanda and the Role of Clinical Alerts: Cross-Sectional Observational Study > Discussion > Limitations
What the sources say
- Supportedcites [4] source
The Methods section (“The EHR Implementation Science Study”) lists minimum computer, printer, and network requirements and successful installation of the Server Monitoring Tool among the criteria used to select the 112 sites. The Discussion (“Limitations”) also states that the selected sites had better hardware than other sites. The supplied passages do not provide page numbers. (LLM, conf=0.94)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“EHR system availability and uptime was positively correlated with concordance, whereas users’ experience with technology was negatively correlated with concordance.”
§ Factors Influencing Data Quality in Electronic Health Record Systems in 50 Health Facilities in Rwanda and the Role of Clinical Alerts: Cross-Sectional Observational Study > Abstract > Results
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Data completeness was >85% threshold for all data elements except viral load (VL) results, second-line, and third-line drug regimens.”
- Also stated in: § Conclusions · plain
“The EHR records in the 50 HFs generally had high levels of completeness except for VL results.”
§ Factors Influencing Data Quality in Electronic Health Record Systems in 50 Health Facilities in Rwanda and the Role of Clinical Alerts: Cross-Sectional Observational Study > Abstract > Results
plainn = 3,467- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
- Also stated in: § Conclusions · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“EHR system availability and uptime was positively correlated with concordance, whereas users’ experience with technology was negatively correlated with concordance.”
§ Factors Influencing Data Quality in Electronic Health Record Systems in 50 Health Facilities in Rwanda and the Role of Clinical Alerts: Cross-Sectional Observational Study > Abstract > Results
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
27 more claims
Create a free account to see all 85 claims and to verify the 11 references still open.
- It may state two separate claims as one. 39
- Probably not a substantial claim. 34
Citations
Cites
Addressing bias in artificial intelligence in health care
Parikh, R. B., Teeple, S., & Navathe, A. S. (2019) · JAMA · doi:10.1001/jama.2019.18058 · Cited by 764
Middleton, B., Bloomrosen, M., Dente, M. A., Hashmat, B., Koppel, R., Overhage, J. M., Payne, T. H., Rosenbloom, S. T., Weaver, C., & Zhang, J. (2013) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2012-001458 · Cited by 618
Uses of Electronic Health Records for Public Health Surveillance to Advance Public Health
Birkhead, G. S., Klompas, M., & Shah, N. R. (2015) · Annual Review of Public Health · doi:10.1146/annurev-publhealth-031914-122747 · Cited by 345 · cited 1×
“Routine clinical data, such as those collected from EHRs, are an important component of public health systems in high-income countries [20].”
BackgroundMobile Health (mHealth) in Low- and Middle-Income Countries
McCool, J., Dobson, R., Whittaker, R., & Paton, C. (2022) · Annual Review of Public Health · doi:10.1146/annurev-publhealth-052620-093850 · Cited by 269 · cited 2×
“FHIR-based Application Programming Interfaces and toolkits are also being built into the Apple OS and Android operating systems [36].”
MethodA review of PHR, EMR and EHR integration: A more personalized healthcare and public health policy
Heart, T., Ben-Assuli, O., & Shabtai, I. (2017) · Health Policy and Technology · doi:10.1016/j.hlpt.2016.08.002 · Cited by 251 · cited 1×
“To maximize the benefits of EHRs for clinical and public health functions, they increasingly allow data collection on behavioral, social, and environmental factors that may affect patient and population health [26].”
BackgroundEgger, M. (2008) · Bulletin of the World Health Organization · doi:10.2471/blt.07.049908 · Cited by 177
Rich, M. L., Miller, A. C., Niyigena, P., Franke, M. F., Niyonzima, J. B., Socci, A., Drobac, P. C., Hakizamungu, M., Mayfield, A., Ruhayisha, R., et al. (2012) · JAIDS Journal of Acquired Immune Deficiency Syndromes · doi:10.1097/qai.0b013e31824476c4 · Cited by 172
The OpenMRS Implementers Network
Seebregts, C. J., Mamlin, B. W., Biondich, P., Fraser, H., Wolfe, B. A., Jazayeri, D., Allen, C., Miranda, J., Baker, E., Musinguzi, N., et al. (2009) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2008.09.005 · Cited by 146
Self-citationFritz, F., Tilahun, B., & Dugas, M. (2015) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocu038 · Cited by 131 · cited 2×
“This typically leads to staff questioning the value of using the systems and keeping data up to date, which can lead to a downward cycle of deterioration and sometimes abandoning the EHR [9].”
BackgroundAn information system and medical record to support HIV treatment in rural Haiti
Fraser, H. S. F., Jazayeri, D., Nevil, P., Karacaoglu, Y., Farmer, P. E., Lyon, E., Fawzi, M. K. C. S., Leandre, F., Choi, S. S., & Mukherjee, J. S. (2004) · BMJ · doi:10.1136/bmj.329.7475.1142 · Cited by 125
Self-citation
Research works printed but not linked: 6
- Amoroso, CL, 2010, Stud Health Technol Inform, 160, 337
- Fraser, HS, 2010, AMIA Annu Symp Proc, 2010, 232
- Blaya, JA, 2010, Int J Tuberc Lung Dis, 14, 1009
- Wolfe, BA, 2006, AMIA Annu Symp Proc, 2006, 1146
- Allen, C, 2007, Stud Health Technol Inform, 129, 382
- Fraser, HS, 2021, ICT in Health: Survey on the Use of Information and Communication Technologies in Brazil Healthcare Facilities, 273
Cited by
Common data quality elements for health information systems: a systematic review
Ghalavand, H., Shirshahi, S., Rahimi, A., Zarrinabadi, Z., & Amani, F. (2024) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-024-02644-7 · Cited by 76
Sylvain, M. H., Nyabyenda, E. C., Uwase, M., Komezusenge, I., Ndikumana, F., & Ngaruye, I. (2025) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-025-02921-z · Cited by 28
Incompleteness of Electronic Health Records: An Impending Process Problem Within Healthcare
Gurupur, V. P., Hooshmand, S., Prabhu, D. F., Trader, E. A., & Salvi, S. (2025) · Healthcare · doi:10.3390/healthcare13222900 · Cited by 29
Petit, P., & Vuillerme, N. (2025) · JMIR Public Health and Surveillance · doi:10.2196/62939 · Cited by 22
Adoption of electronic health records by nurses in Africa: A scoping review
Harerimana, A., Wicking, K., Biedermann, N., Yates, K., Pillay, J. D., & Mchunu, G. G. (2025) · Digital Health · doi:10.1177/20552076251357401 · Cited by 18
Metadata
- Authors
- Fraser, H. S. F., Mugisha, M., Bacher, I., Ngenzi, J. L., Seebregts, C., Umubyeyi, A., & Condo, J.
- Affiliations
- Brown University (US)
- University of Rwanda (RW)
- University of Cape Town (ZA)
- Tulane University (US)
- Corresponding author
- Fraser, Hamish S F
- ORCID
- Venue
- JMIR Public Health and Surveillance
- Publisher
- JMIR Publications Inc.
- Published
- 2024-07-03
- Volume
- 10
- Pages
- e49127
- ISSN
- 2369-2960 (ISSN-L)
- PMC
- PMC11255528
- OpenAlex
- W4388467758
- Semantic Scholar
- 870bf0fac2a05ac519f6e719fcd62eec6fe3cc4c
- Cited by
- 20
- References
- 35
- Citations by year
- 2026: 12 · 2025: 7 · 2024: 1
- FWCI
- 4.48
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- JMIR Public Health and Surveillance (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Funders
- Brown University
- U.S. President’s Emergency Plan for AIDS Relief
- University of Rwanda
- European Commission (661289)
- Rwanda Biomedical Centre
- Centers for Disease Control and Prevention (U01GH000782)
- Topics
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Health Sciences › Health Professions › Health Information Management › Medical Coding and Health Information
- Physical Sciences › Computer Science › Information Systems › Data Quality and Management
- Keywords
- electronic health records, data quality, clinical decision support, clinical alerts, HIV care, OpenMRS, Open Data Kit, data completeness, data timeliness, data concordance, paper records, Rwanda, low- and middle-income countries, health information system evaluation
- MeSH terms
- Data Accuracy, Cross-Sectional Studies, Health Facilities, Humans, Rwanda, HIV Infections, Electronic Health Records
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.