Review: evaluating information systems in nursing
Open accessAbstract
Aims. To review existing nursing research on inpatient hospitals’ information technology (IT) systems in order to explore new approaches for evaluation research on nursing informatics to guide further design and implementation of effective IT systems. Background. There has been an increase in the use of IT and information systems in nursing in recent years. However, there has been little evaluation of these systems and little guidance on how they might be evaluated. Methods. A literature review was conducted between 1995 and 2005 inclusive using CINAHL and Medline and the search terms ‘nursing information systems’, ‘clinical information systems’, ‘hospital information systems’, ‘documentation’, ‘nursing records’, ‘charting’. Results. Research in nursing information systems was analysed and some deficiencies and contradictory results were identified which impede a comprehensive understanding of effective implementation. There is a need for IT systems to be understood from a wider perspective that includes aspects related to the context where they are implemented. Conclusions. Social and organizational aspects need to be considered in evaluation studies and realistic evaluation can provide a framework for the evaluation of information systems in nursing. Relevance to clinical practice. The rapid introduction of IT systems for clinical practice urges evaluation of already implemented systems examining how and in what circumstances they work to guide effective further development and implementation of IT systems to enhance clinical practice. Evaluation involves more factors than just involving technologies such as changing attitudes, cultures and healthcare practices. Realistic evaluation could provide configurations of context‐mechanism‐outcomes that explain the underlying relationships to understand why and how a programme or intervention works.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Demographic factors such as age, education, sex, nationality, and clinical experience may influence healthcare professionals' attitudes towards e-health systems.
“Demographic factors such as age, education, sex, nationality, and clinical experience may influence healthcare professionals’ attitudes towards e-health systems [23, 28, 46, 56, 57];”
From Factors that influence the implementation of e-health: a systematic review of systematic reviews (an update) (2016) · cites this paper as [46]
Most studies conclude that no clear relationships between demographic characteristics and attitudes could be established.
“however, most studies conclude that no clear relationships between these characteristics and attitudes could be established [46, 56, 57].”
From Factors that influence the implementation of e-health: a systematic review of systematic reviews (an update) (2016) · cites this paper as [46]
Cites
Implementing information systems in health care organizations: myths and challenges
Berg, M. (2001) · International Journal of Medical Informatics · doi:10.1016/s1386-5056(01)00200-3 · Cited by 715 · see citing works
The Phenomenology of Knowing the Patient
Tanner, C. A., Benner, P., Chesla, C. A., et al. (1993) · Journal of Nursing Scholarship · doi:10.1111/j.1547-5069.1993.tb00259.x · Cited by 419 · see citing works
Evaluation of health information systems—problems and challenges
Ammenwerth, E., Gräber, S., Herrmann, G., Bürkle, T., & König, J. (2003) · International Journal of Medical Informatics · doi:10.1016/s1386-5056(03)00131-x · Cited by 414 · see citing works
Accuracy of Data in Computer-based Patient Records
Hogan, W. R., & Wagner, M. M. (1997) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.1997.0040342 · Cited by 389 · see citing works
Determinants of Success of Inpatient Clinical Information Systems: A Literature Review
van der Meijden, M. J. (2003) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1094 · Cited by 363 · see citing works
Evaluation Research: An Introduction to Principles, Methods and Practice
Clarke, A. W. (1999) · Medical Entomology and Zoology · Cited by 334 · see citing works
Ammenwerth, E., Mansmann, U., Iller, C., & Ronald Eichstädter, D. (2003) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1118 · Cited by 238 · see citing works
Visions and strategies to improve evaluation of health information systems
Ammenwerth, E., Brender, J., Nykänen, P., Prokosch, H., Rigby, M., & Talmon, J. L. (2004) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2004.04.004 · Cited by 233 · see citing works
United Kingdom central council for nursing, midwifery and health visiting
Burke, S. (1983) · Representation · doi:10.1080/00344898308459341 · Cited by 233 · see citing works
Patel, V. L., Kushniruk, A. W., Yang, S., & Yale, J. (2000) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.2000.0070569 · Cited by 232 · see citing works
Critical realism: a way forward for evaluation research in nursing?
McEvoy, P., & Richards, D. A. (2003) · Journal of Advanced Nursing · doi:10.1046/j.1365-2648.2003.02730.x · Cited by 203 · see citing works
Byng, R., Norman, I. J., & Redfern, S. (2005) · Evaluation · doi:10.1177/1356389005053198 · Cited by 187 · see citing works
Darbyshire, P. (2004) · Journal of Clinical Nursing · doi:10.1046/j.1365-2702.2003.00823.x · Cited by 183 · see citing works
The computerized patient record: balancing effort and benefit
van Ginneken, A. M. (2002) · International Journal of Medical Informatics · doi:10.1016/s1386-5056(02)00007-2 · Cited by 153 · see citing works
Improving clinical decisions and outcomes with information: a review
Tierney, W. M. (2001) · International Journal of Medical Informatics · doi:10.1016/s1386-5056(01)00127-7 · Cited by 139 · see citing works
Radwin, L. E. (1995) · Nursing Research · doi:10.1097/00006199-199511000-00008 · Cited by 135 · see citing works
A Randomized Evaluation of a Computer-Based Nursing Documentation System
Ronald Eichstädter, D., Haux, R., Pohl, U., Rebel, S., Ziegler, S., & Ammenwerth, E. (2001) · Methods of Information in Medicine · doi:10.1055/s-0038-1634465 · Cited by 128 · see citing works
Opportunities for and challenges of computerisation
Powsner, S. M., Wyatt, J. C., & Wright, P. (1998) · The Lancet · doi:10.1016/s0140-6736(98)08309-3 · Cited by 119 · see citing works
Nursing documentation and nursing practice: a discourse analysis
Heartfield, M. (1996) · Journal of Advanced Nursing · doi:10.1046/j.1365-2648.1996.15113.x · Cited by 115 · see citing works
Development of an audit instrument for nursing care plans in the patient record
Björvell, C., Thorell-Ekstrand, I., & Wredling, R. (2000) · BMJ Quality & Safety · doi:10.1136/qhc.9.1.6 · Cited by 112 · see citing works
Forbes, A., & Griffiths, P. (2002) · Nursing Inquiry · doi:10.1046/j.1440-1800.2002.00146.x · Cited by 98 · see citing works
Elske, A., & Nicolette, D. K. (2004) · Studies in health technology and informatics · doi:10.3233/978-1-60750-949-3-1289 · Cited by 97 · see citing works
Nahm, R., & Poston, I. (2000) · PubMed · Cited by 96 · see citing works
Enabling technologies promise to revitalize the role of nursing in an era of patient safety
Ball, M. J., Weaver, C. A., & Abbott, P. A. (2003) · International Journal of Medical Informatics · doi:10.1016/s1386-5056(02)00063-1 · Cited by 92 · see citing works
Health information systems challenges: the Heidelberg conference and the future
Giuse, D. A. (2003) · International Journal of Medical Informatics · doi:10.1016/s1386-5056(02)00182-x · Cited by 89 · see citing works
Nursing Attitudes and Images of Electronic Patient Record Systems
Dillon, T. W., Blankenship, R. J., & Crews, T. (2005) · CIN Computers Informatics Nursing · doi:10.1097/00024665-200505000-00009 · Cited by 87 · see citing works
Coiera, E. W. (2003) · International Journal of Medical Informatics · doi:10.1016/s1386-5056(02)00106-5 · Cited by 86 · see citing works
Design should help use of patients' data
Wyatt, J. C., & Wright, P. (1998) · The Lancet · doi:10.1016/s0140-6736(97)08306-2 · Cited by 83 · see citing works
Hospital information systems: perspectives on problems and prospects, 1979 and 2002
Ball, M. (2003) · International Journal of Medical Informatics · doi:10.1016/s1386-5056(02)00098-9 · Cited by 82 · see citing works
Helping clinicians to find data and avoid delays
Nygren, E., Wyatt, J. C., & Wright, P. (1998) · The Lancet · doi:10.1016/s0140-6736(97)08307-4 · Cited by 82 · see citing works
Attitudes of Australian nurses toward the implementation of a clinical information system.
Marasovic, C., Kenney, C., Elliott, D., & Sindhusake, D. P. (1997) · PubMed · Cited by 81 · see citing works
Thinking Processes Used by Nurses in Clinical Decision Making
Higuchi, K. S., & Donald, J. G. (2002) · Journal of Nursing Education · doi:10.3928/0148-4834-20020401-04 · Cited by 79 · see citing works
Lee, T. (2005) · Journal of Clinical Nursing · doi:10.1111/j.1365-2702.2004.01060.x · Cited by 76 · see citing works
Written and Computerized Care Plans
Daly, J. M., Buckwalter, K., & Maas, M. (2002) · Journal of Gerontological Nursing · doi:10.3928/0098-9134-20020901-05 · Cited by 72 · see citing works
Developing a module for nursing documentation integrated in the electronic patient record
Hellesø, R., & Ruland, C. M. (2001) · Journal of Clinical Nursing · doi:10.1046/j.1365-2702.2001.00557.x · Cited by 59 · see citing works
Moloney, R., & Maggs, C. J. (1999) · Journal of Advanced Nursing · doi:10.1046/j.1365-2648.1999.01048.x · Cited by 59 · see citing works
Evaluation frameworks for nursing informatics
Currie, L. M. (2005) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2005.07.007 · Cited by 58 · see citing works
Andersson, A., Hallberg, N., & Timpka, T. (2003) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2003.09.001 · Cited by 55 · see citing works
An Inventory of Evaluation Studies of Information Technology in Health Care
de Keizer, N. F., & Ammenwerth, E. (2005) · Methods of Information in Medicine · doi:10.1055/s-0038-1633922 · Cited by 54 · see citing works
Documentation practices of nurses in long‐term care
MARTIN, A., Hinds, C., & FELIX, M. (1999) · Journal of Clinical Nursing · doi:10.1046/j.1365-2702.1999.00281.x · Cited by 47 · see citing works
Nursing process documentation systems in clinical routine—prerequisites and experiences
Ammenwerth, E., Kutscha, U., Kutscha, A., Mahler, C., Ronald Eichstädter, D., & Haux, R. (2001) · International Journal of Medical Informatics · doi:10.1016/s1386-5056(01)00216-7 · Cited by 46 · see citing works
The Challenge of Ubiquitous Computing in Health Care: Technology, Concepts and Solutions
Ammenwerth, E., Brigl, B., Knaup, P., Lang, E., Pilgram, R., Pfeifer, B. E., Ruderich, F., Wolff, A. C., Haux, R., Kulikowski, C., et al. (2005) · Methods of Information in Medicine · doi:10.1055/s-0038-1633996 · Cited by 42 · see citing works
Structure and Content in Norwegian Nursing Care Documentation
Stokke, T. A., & Kalfoss, M. H. (1999) · Scandinavian Journal of Caring Sciences · doi:10.1111/j.1471-6712.1999.tb00510.x · Cited by 41 · see citing works
Nursing informatics: state of the science*
Henry, S. B. (1995) · Journal of Advanced Nursing · doi:10.1111/j.1365-2648.1995.tb03121.x · Cited by 39 · see citing works
Friedman, C. P. (2003) · International Journal of Medical Informatics · doi:10.1016/s1386-5056(02)00109-0 · Cited by 38 · see citing works
Evaluation of patient records as part of developing a nursing care classification
Kärkkäinen, O., & Eriksson, K. (2003) · Journal of Clinical Nursing · doi:10.1046/j.1365-2702.2003.00727.x · Cited by 36 · see citing works
Policies that drive the nursing practice of postoperative observations
Zeitz, K., & McCutcheon, H. (2002) · International Journal of Nursing Studies · doi:10.1016/s0020-7489(02)00024-x · Cited by 28 · see citing works
Reviewing the evidence on nursing record systems
Urquhart, C. J., & Currell, R. (2005) · Health Informatics Journal · doi:10.1177/1460458205050683 · Cited by 27 · see citing works
As good as it gets? Going online with a clinical information system.
Sleutel, M. R., & Guinn, M. A. F. (1999) · PubMed · Cited by 26 · see citing works
Epping, P. J. M. M., Abraham, I. L., & Goossen, W. T. F. (1996) · Methods of Information in Medicine · doi:10.1055/s-0038-1634633 · Cited by 20 · see citing works
Thompson, C. A. (2001) · Journal of Advanced Nursing · doi:10.1046/j.1365-2648.2001.01829.x · Cited by 16 · see citing works
The complexity of developing a nursing information system: a Brazilian experience.
Filho, J. R. (2001) · PubMed · Cited by 16 · see citing works
Developing a module for nursing documentation integrated in the electronic patient record
Hellesø, R., & Ruland, C. M. (2001) · Journal of Clinical Nursing · doi:10.1111/j.1365-2702.2001.00557.x · Cited by 16 · see citing works
Concept Development of Nursing Information
Erdley, W. S. (2005) · CIN Computers Informatics Nursing · doi:10.1097/00024665-200503000-00009 · Cited by 9 · see citing works
Problems encountered in integrated care records research
Jenkings, K. N. (2004) · Health Informatics Journal · doi:10.1177/1460458204048512 · Cited by 9 · see citing works
Informed decisions by clinicians are fundamental for EPR implementation
Nikula, R. E., Elberg, P. B., & Svedberg, H. B. (2000) · International Journal of Medical Informatics · doi:10.1016/s1386-5056(00)00082-4 · Cited by 7 · see citing works
Nursing Informatics: Where Caring and Technology Meet
Christman, L. P. (1996) · Nursing Administration Quarterly · doi:10.1097/00006216-199602110-00015 · Cited by 7 · see citing works
Evaluating clinical systems: the social programme perspective
Heathfield, H. (2001) · Health Informatics Journal · doi:10.1177/146045820100700103 · Cited by 6 · see citing works
Developing and testing a prototype patients care database.
McDaniel, A. M. (1997) · PubMed · Cited by 5 · see citing works
New Roles for Nurses in Healthcare Information Systems
Hersher, B. S. (2000) · Computers in health care · doi:10.1007/978-1-4757-3252-8_6 · Cited by 5 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Ross, J., Stevenson, F., Lau, R., & Murray, E. (2016) · Implementation Science · doi:10.1186/s13012-016-0510-7 · Cited by 1,011 · see citing works
Mair, F. S., May, C., O’Donnell, C., Finch, T., Sullivan, F., & Murray, E. (2012) · Bulletin of the World Health Organization · doi:10.2471/blt.11.099424 · Cited by 553 · see citing works
An investigation on physicians’ acceptance of hospital information systems: A case study
Chen, R.-F., & Hsiao, J.-L. (2012) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2012.05.003 · Cited by 150 · see citing works
Hospital Information Technology Systems' Impact on Nurses and Nursing Care
Waneka, R., & Spetz, J. (2010) · JONA: The Journal of Nursing Administration · doi:10.1097/nna.0b013e3181fc1a1c · Cited by 69 · see citing works
- Authors
- Oroviogoicoechea, C., Elliott, B., & Watson, R.
- Affiliations
- Clinica Universidad de Navarra (ES)
- Venue
- Journal of Clinical Nursing
- Publisher
- Wiley
- Published
- 2008-02-11
- Volume
- 17
- Issue
- 5
- Pages
- 567-575
- ISSN
- 0962-1067 (ISSN-L)
- 1365-2702
- OpenAlex
- W2081085022
- Cited by
- 82
- References
- 60
- Citations by year
- 2026: 1 · 2024: 1 · 2023: 2 · 2021: 3 · 2019: 3 · 2018: 5 · 2017: 3 · 2016: 5 · 2015: 6 · 2014: 8 · 2013: 8 · 2012: 14
- FWCI
- 16.83
- Type
- journal article
- Language
- en
- Open access
- Locations
- Journal of Clinical Nursing (publishedVersion)
- PubMed (publishedVersion)
- Repository@Hull (Worktribe) (University of Hull) (submittedVersion, cc-by)
- Repository@Hull (Worktribe) (University of Hull) (submittedVersion)
- Topics
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Health Sciences › Nursing › Issues, ethics and legal aspects › Nursing Diagnosis and Documentation
- Social Sciences › Sociology and Political Science › Information Systems Theories and Implementation
- Keywords
- nursing information systems, clinical information systems, hospital information systems, information system evaluation, nursing informatics, realist evaluation, context-mechanism-outcome, nursing documentation, nursing records, electronic charting
- MeSH terms
- Humans, Information Systems, Job Satisfaction, Nursing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
