Applying an Evaluation Framework for Health Information System Design, Development, and Implementation
The DOI matches its Crossref record.
Abstract
BACKGROUND: Dramatic advances in health information technologies necessitate a comprehensive evaluation framework covering all phases of development, from conception to routine operational use. OBJECTIVES: The purpose of this article was to illustrate the application of an informatics evaluation framework that provides a heuristic for matching the stage of system design and the level of evaluation. METHODS: An evaluation framework is illustrated in the context of five studies in different stages of system design. RESULTS: The studies discussed in this article represent the various stages of the framework. In addition, they are examples of how evaluation research studies not only contribute to the assessment of system design but also constitute distinct contributions to knowledge. DISCUSSION: The ability to engineer advanced information systems has exceeded the understanding of how to deploy them effectively in complex settings and to adapt them to a range of user populations. A systematic, continuous evaluation increases the likelihood that the conditions for success will be understood, including how to tailor a system, how to maintain the target population's use of the system, and how to innovate continually to enhance the functionality of the system and the users' experience. Without sound evaluation methodologies throughout the stages of system development, information systems have limited potential to influence healthcare processes positively.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cited by
Greenhalgh, T., Wherton, J., Papoutsi, C., Lynch, J., Hughes, G., A'Court, C., Hinder, S., Fahy, N., Procter, R., & Shaw, S. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.8775 · Cited by 2,887
“Our search for evidence-based approaches to guide our empirical work on the above case studies identified 28 technology implementation frameworks, informed by several theoretical perspectives, which we sorted into a simple taxonomy [13,43,47-72]; see Multimedia Appendix 1.”
A Holistic Framework to Improve the Uptake and Impact of eHealth Technologies
van Gemert-Pijnen, J. E., Nijland, N., van Limburg, M., Ossebaard, H. C., Kelders, S. M., Eysenbach, G., & Seydel, E. R. (2011) · Journal of Medical Internet Research · doi:10.2196/jmir.1672 · Cited by 1,126
Review of health information technology usability study methodologies
Yen, P.-Y., & Bakken, S. (2012) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2010-000020 · Cited by 390
“6 To ensure the best utilization of health IT, it is essential to be attentive to health IT usability, keeping in mind its intended users (eg, physicians, nurses, or pharmacists), task (eg, medication management, free-text data entry, or patient record search), and environment (eg, operation room,…”
Brown, W., Yen, P.-Y., Rojas, M., & Schnall, R. (2013) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2013.08.001 · Cited by 299
“[16] While the promise of mHealth is that we can leverage the power and ubiquity of mobile and cloud technologies to monitor and manage side effects and treatment outside the clinical setting, it is essential to be attentive to usability, keeping in mind its intended users, task and environment.”
BackgroundEvaluating artificial intelligence in medicine: phases of clinical research
Park, Y., Jackson, G. P., Foreman, M. A., Gruen, D., Hu, J., & Das, A. K. (2020) · JAMIA Open · doi:10.1093/jamiaopen/ooaa033 · Cited by 150
Metadata
- Authors
- Kaufman, D., Roberts, W. D., Merrill, J., Lai, T.-Y., & Bakken, S.
- Venue
- Nursing Research
- Published
- 2006-03
- Volume
- 55
- Issue
- Supplement 1
- Pages
- S37-S42
- ISSN
- 0029-6562 (ISSN-L)
- OpenAlex
- W2053058732
- Cited by
- 81
- References
- 0
- FWCI
- 6.45
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Applying an Evaluation Framework for Health Information System Design, Development, and Implementation. https://app.cochat.ai/papers/10.1097/00006199-200603001-00007
- The paper
- Kaufman, D., Roberts, W. D., Merrill, J., Lai, T.-Y., & Bakken, S. (2006). Applying an Evaluation Framework for Health Information System Design, Development, and Implementation. Nursing Research. https://doi.org/10.1097/00006199-200603001-00007
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.