Evaluation of a comprehensive EHR based on the DeLone and McLean model for IS success: Approach, results, and success factors
Open accessThe DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.
Abstract
OBJECTIVE: The article describes the methodological approach to, and results of an evaluation of a comprehensive electronic health record (EHR) in the shake down phase, shortly after its implementation at a regional hospital in Denmark. DESIGN: A formative evaluation based on a mixed-methods case study, designed to be interactive and concurrent was conducted at two hospital departments based on the updated DeLone and McLean framework for evaluating information systems success. METHODS: To ascertain user assessments of the EHR, we distributed a questionnaire two months after implementation to four groups of staff (physicians, nurses, medical secretaries, and physiotherapists; n=244), and at the same time we conducted thirteen individual, semi-structured interviews with representatives from these four groups. Subsequently, seven follow-up focus group interviews were conducted with the four above-mentioned groups, in order to go deeper into specific user assessments. Simultaneously, focus group interviews with two IT departments and the implementation team were conducted, to gain insight into system provider assessments of the implementation process and the EHR. Before, during, and after implementation, 88 h of ethnographic observation were carried out, to give the researchers an understanding of the daily routine of staff, and their use of health records. Finally, daily system performance data were obtained, to gather factual information on system response and downtime. RESULTS: Overall, staff had positive experiences with the EHR and its operational reliability, response time, login and support. Performance was acceptable. Medical secretaries found the use of the patient administration module cumbersome, and physicians found the establishment of the overview of professionally relevant data challenging. There were demands for improvements to these and other functionalities, and for the EHR to be integrated with other systems and databases. LIMITATIONS: Evaluations immediately following implementation are inherently difficult, but was required because a key role was to inform decision-making upon enrollment at other hospitals and systematically identify barriers in this respect. The strength of the evaluation is the mixed-methods approach. Further, the evaluation was based on assessments from staff in two departments that comprise around 50% of hospital staff. A weakness may be that staff assessment plays a major role in interviews and survey. These though are supplemented by performance data and observation. Also, the evaluation primarily reports upon the dimension 'user satisfaction', since use of the EHR is mandatory. Finally, generalizability may be low, since the evaluation was not based on a validated survey. All in all, however, the evaluation proposes an evaluation design in constrained circumstances. CONCLUSIONS: Despite inherent limitations, evaluation of a comprehensive EHR shortly after implementation may be necessary, can be conducted, and may inform political decision making. The updated DeLone and McLean framework was constructive in the overall design of the evaluation of the EHR implementation, and allowed the model to be adapted to the health care domain by being methodological flexible. The mixed-methods case study produced valid and reliable results, and was accepted by staff, system providers, and political decision makers. The successful implementation may be attributed to the configurability of the EHR and to factors such as an experienced, competent implementation organization at the hospital, upgraded soft- and hardware, and a high degree of user involvement.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hartley, J. (2004) · Essential Guide to Qualitative Methods in Organizational Research · doi:10.4135/9781446280119.n26 · Cited by 15,461
The DeLone and McLean Model of Information Systems Success: A Ten-Year Update
(2003) · Journal of Management Information Systems · doi:10.1080/07421222.2003.11045748 · Cited by 11,791 · cited 7×
“However, earlier studies have shown that successful implementation of EHRs also depends on a wide range of contextual and organizational factors [13,16,51], and hence the positive evaluation results merit further discussion.”
BackgroundMethodInfluentialTask-Technology Fit and Individual Performance
Goodhue, D. L., & Thompson, R. L. (1995) · MIS Quarterly · doi:10.2307/249689 · Cited by 6,006 · cited 1×
“[17–20].”
The Case Research Strategy in Studies of Information Systems
Benbasat, I., Goldstein, D. K., & Mead, M. (1987) · MIS Quarterly · doi:10.2307/248684 · Cited by 4,716
Assessing the Validity of IS Success Models: An Empirical Test and Theoretical Analysis
Rai, A., Lang, S. S., & Welker, R. B. (2002) · Information Systems Research · doi:10.1287/isre.13.1.50.96 · Cited by 1,611
Use of Electronic Health Records in U.S. Hospitals
Jha, A. K., DesRoches, C. M., Campbell, E. G., Donelan, K., Rao, S. R., Ferris, T. G., Shields, A., Rosenbaum, S., & Blumenthal, D. (2009) · New England Journal of Medicine · doi:10.1056/nejmsa0900592 · Cited by 1,473
Yusof, M. M., Kuljis, J., Papazafeiropoulou, A., & Stergioulas, L. K. (2008) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2007.08.011 · Cited by 740
LUDWICK, D., & DOUCETTE, J. (2009) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2008.06.005 · Cited by 668 · cited 2×
“However, earlier studies have shown that successful implementation of EHRs also depends on a wide range of contextual and organizational factors [13,16,51], and hence the positive evaluation results merit further discussion.”
BackgroundGREENHALGH, T., POTTS, H. W. W., WONG, G., BARK, P., & SWINGLEHURST, D. (2009) · The Milbank Quarterly · doi:10.1111/j.1468-0009.2009.00578.x · Cited by 541
Kaplan, B. (2001) · International Journal of Medical Informatics · doi:10.1016/s1386-5056(01)00184-8 · Cited by 380
Cited by
Understanding the impact of m-banking on individual performance: DeLone & McLean and TTF perspective
Tam, C., & Oliveira, T. (2016) · Computers in Human Behavior · doi:10.1016/j.chb.2016.03.016 · Cited by 383
Literature review of mobile banking and individual performance
Tam, C., & Oliveira, T. (2017) · International Journal of Bank Marketing · doi:10.1108/ijbm-09-2015-0143 · Cited by 202
“(Bossen et al., 2013) Evaluation of a comprehensive electronic health record D&M 244 professionals, ANOVA The results produced using the D&M framework are valid and reliable, and were accepted by staff, system providers, and political decision makers.”
BackgroundMethodInfluentialSharma, S. K., Gaur, A., Saddikuti, V., & Rastogi, A. (2017) · Behaviour & Information Technology · doi:10.1080/0144929x.2017.1340973 · Cited by 172
“Bossen, Jensen, and Udsen (2013) examined the user acceptance of the electronic health records in the healthcare domain.”
BackgroundUnlocking AI Chatbot Potential in Healthcare: Trust-Enhanced DeLone & McLean IS Success Model
Sarhan, M. Y., Alarify, M., & Khojah, M. (2026) · Healthcare · doi:10.3390/healthcare14101324 · Cited by 2
“The D&M ISS Model provides a practical framework for evaluating the effectiveness of health IS [70].”
Metadata
- Authors
- Bossen, C., Jensen, L. G., & Udsen, F. W.
- Venue
- International Journal of Medical Informatics
- Publisher
- Elsevier BV
- Published
- 2013-10
- Volume
- 82
- Issue
- 10
- Pages
- 940-953
- ISSN
- 1386-5056 (ISSN-L)
- OpenAlex
- W1977488000
- Cited by
- 183
- References
- 15
- FWCI
- 13.59
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Evaluation of a comprehensive EHR based on the DeLone and McLean model for IS success: Approach, results, and success factors. https://app.cochat.ai/papers/10.1016/j.ijmedinf.2013.05.010
- The paper
- Bossen, C., Jensen, L. G., & Udsen, F. W. (2013). Evaluation of a comprehensive EHR based on the DeLone and McLean model for IS success: Approach, results, and success factors. International Journal of Medical Informatics. https://doi.org/10.1016/j.ijmedinf.2013.05.010
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.