Addressing organizational issues into the evaluation of medical systems
Open accessAbstract
New system design and evaluation methodologies are being developed to address social, organizational, political, and other non-technological issues in medical informatics. This paper describes a social interactionist framework for researching these kinds of organizational issues, based on research within medical informatics and other disciplines over the past 20 years. It discusses how effective evaluation strategies may be undertaken to address organizational issues concerning computer information systems in medicine and health care. The paper begins with a theoretical framework for evaluation. It then describes the 4Cs of evaluation: communication, care, control, and context. Five methodological guidelines are given for conducting comprehensive evaluations that address these 4Cs. An example of an evaluation research design that fits the guidelines and was used in an evaluation of an on-line clinical imaging system is discussed. Results of the evaluation study illustrate how this approach addresses organizational concerns and the 4Cs.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
An interactive sociotechnical approach using Kaplan's 4Cs of communication, care, control, and context could investigate the dynamic interactions among these issues.
“An interactive sociotechnical approach incorporating Kaplan's 4Cs of communication, care, control, and context that draw attention to ELSI could investigate the dynamic interactions between these issues [53,[57], [58]].”
From REVISITING HEALTH INFORMATION TECHNOLOGY ETHICAL, LEGAL, and SOCIAL ISSUES and EVALUATION: TELEHEALTH/TELEMEDICINE and COVID-19 (2020) · cites this paper as [57]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Power, politics, and MIS implementation
Markus, M. L. (1983) · Communications of the ACM · doi:10.1145/358141.358148 · Cited by 2,224 · see citing works
Information Technology and Organizational Change: Causal Structure in Theory and Research
Markus, M. L., & Robey, D. (1988) · Management Science · doi:10.1287/mnsc.34.5.583 · Cited by 2,108 · see citing works
Multimethod Research: A Synthesis of Styles
Brewer, J., & Hunter, A. (1989) · Cited by 1,189 · see citing works
Organizations and Organization Theory.
McKelvey, B., & Pfeffer, J. (1984) · Administrative Science Quarterly · doi:10.2307/2392948 · Cited by 1,144 · see citing works
Combining Qualitative and Quantitative Methods in Information Systems Research: A Case Study1
Kaplan, B., & Duchon, D. (1988) · MIS Quarterly · doi:10.2307/249133 · Cited by 1,074 · see citing works
Self-citationThe Computer-Based Patient Record: An Essential Technology for Health Care
(1992) · Annals of Internal Medicine · doi:10.7326/0003-4819-117-11-976_1 · Cited by 999 · see citing works
Four paradigms of information systems development
Hirschheim, R., & Klein, H. K. (1989) · Communications of the ACM · doi:10.1145/67933.67937 · Cited by 912 · see citing works
Qualitative Research Methods for Evaluating Computer Information Systems
Kaplan, B., & Maxwell, J. A. (2006) · Springer eBooks · doi:10.1007/0-387-30329-4_2 · Cited by 816 · see citing works
Self-citationSocial Analyses of Computing: Theoretical Perspectives in Recent Empirical Research
Kling, R. (1980) · ACM Computing Surveys · doi:10.1145/356802.356806 · Cited by 775 · see citing works
Information systems research: contemporary approaches and emergent traditions
Nissen, H.-E., Klein, H. K., & Hirschheim, R. (1991) · Elsevier eBooks · Cited by 736 · see citing works
Nimmo, D. D. (1980) · Cited by 702 · see citing works
The Web of Computing: Computer Technology as Social Organization
Kling, R., & Scacchi, W. (1982) · Advances in computers · doi:10.1016/s0065-2458(08)60567-7 · Cited by 586 · see citing works
Evaluation Methods in Medical Informatics
Friedman, C. P., & Wyatt, J. C. (1997) · Computers and medicine · doi:10.1007/978-1-4757-2685-5 · Cited by 450 · see citing works
Different perspectives on information systems: problems and solutions
Lyytinen, K. (1987) · ACM Computing Surveys · doi:10.1145/28865.28867 · Cited by 288 · see citing works
Massaro, T. A. (1993) · Academic Medicine · doi:10.1097/00001888-199301000-00003 · Cited by 287 · see citing works
National Research Council (1994) · National Academies Press eBooks · doi:10.17226/21440 · Cited by 234 · see citing works
Evaluating Health Care Information Systems: Methods and Applications
Anderson, J. G., Aydin, C. E., & Jay, S. J. (1993) · Cited by 180 · see citing works
Introducing physician order entry at a major academic medical center
Massaro, T. A. (1993) · Academic Medicine · doi:10.1097/00001888-199301000-00004 · Cited by 138 · see citing works
Organizational Aspects of Health Informatics: Managing Technological Change
Lorenzi, N. M., & Riley, R. T. (1994) · Cited by 115 · see citing works
Use and Impact of Computers in Clinical Medicine
Anderson, J. G., & Jay, S. J. (1987) · Computers and medicine · doi:10.1007/978-1-4613-8674-2 · Cited by 80 · see citing works
Social, Ergonomic and Stress Aspects of Work with Computers
Salvendy, G., Sauter, S. L., & Hurrell, J. J. (1987) · Elsevier eBooks · Cited by 69 · see citing works
Computers and clinical judgement: The role of physician networks
Anderson, J. G., & Jay, S. J. (1985) · Social Science & Medicine · doi:10.1016/0277-9536(85)90253-9 · Cited by 66 · see citing works
The Information Systems Research Challenge: Experimental Research Methods
Nunamaker, J. F., & Benbasat, I. (1990) · Harvard Business School Press eBooks · Cited by 64 · see citing works
Why Doctors Don't Use Computers: Some Empirical Findings
Anderson, J. G., Jay, S. J., Schweer, H. M., et al. (1986) · Journal of the Royal Society of Medicine · doi:10.1177/014107688607900305 · Cited by 62 · see citing works
Broadening our approach to evaluating medical information systems.
Forsythe, D. E., & Buchanan, B. G. (1991) · PubMed · Cited by 60 · see citing works
The Medical Computing “lag”: Perceptions of Barriers to the Application of Computers to Medicine
Kaplan, B. (1987) · International Journal of Technology Assessment in Health Care · doi:10.1017/s026646230001179x · Cited by 52 · see citing works
Self-citationMicrochips versus stethoscopes: Calgary hospital, MDs face off over controversial computer system.
Ls, W. (1992) · PubMed · Cited by 44 · see citing works
Slack, J. D. (1984) · Medical Entomology and Zoology · Cited by 43 · see citing works
Modelling work practices: input to the design of a physician's workstation.
Fafchamps, D., Young, C. Y., & Tang, P. C. (1991) · PubMed · Cited by 41 · see citing works
Information synthesis: a practical guide.
Pg, G. (1986) · PubMed · Cited by 41 · see citing works
The Influence of Medical Values and Practices on Medical Computer Applications
Kaplan, B. (1982) · Computers and medicine · doi:10.1007/978-1-4613-8674-2_4 · Cited by 37 · see citing works
Self-citationReducing barriers to physician data entry for computer-based patient records.
Kaplan, B. (1994) · PubMed · Cited by 37 · see citing works
Self-citationKaplan, B. (1995) · Artificial Intelligence in Medicine · doi:10.1016/0933-3657(95)00014-w · Cited by 36 · see citing works
Self-citationTransforming Health Care Through Information: Case Studies
Einbinder, L., Einbinder, J. S., Gadd, C. S., Ash, J. S., & Lorenzi, N. M. (2009) · Computers in health care · doi:10.1007/978-1-4419-0269-6 · Cited by 34 · see citing works
Toward an Evaluation of an Integrated Clinical Imaging System: Identifying Clinical Benefits
Lundsgaarde, H. P., & Kaplan, B. (1996) · Methods of Information in Medicine · doi:10.1055/s-0038-1634674 · Cited by 34 · see citing works
Self-citationInitial impact of a clinical laboratory computer system
Kaplan, B. (1987) · Journal of Medical Systems · doi:10.1007/bf00992348 · Cited by 31 · see citing works
Self-citationIntegrating medical images into hospital information systems
Dayhoff, R. E., Maloney, D. L., Kuzmak, P. M., et al. (1991) · Journal of Imaging Informatics in Medicine · doi:10.1007/bf03170416 · Cited by 29 · see citing works
Anderson, J. G., Jay, S. J., Schweer, H. M., Anderson, M. M., & Kassing, D. (1987) · Computers and medicine · doi:10.1007/978-1-4613-8674-2_12 · Cited by 23 · see citing works
Wetherbe, J. C. (1988) · Medical Entomology and Zoology · Cited by 22 · see citing works
Anderson, J. G., AYDIN, C., & Kaplan, B. (2002) · Hawaii International Conference on System Sciences (HICSS) · doi:10.1109/hicss.1995.375671 · Cited by 22 · see citing works
Self-citationThe construction of knowledge in neurology: implications for hypermedia system development
Nyce, J. M., & Graves, W. (1990) · Artificial Intelligence in Medicine · doi:10.1016/0933-3657(90)90006-d · Cited by 21 · see citing works
Nyce, J. M., & Timpka, T. (1993) · Medical & Biological Engineering & Computing · doi:10.1007/bf02446887 · Cited by 15 · see citing works
Information technology and three studies of clinical work
Kaplan, B. (1995) · ACM SIGBIO Newsletter · doi:10.1145/216200.216204 · Cited by 15 · see citing works
Self-citationUsing Attitudinal Questionnaires to Achieve Benefits Optimization.
Hp, L., Rm, G., & Rl, M. (1989) · Annual Symposium on Computer Application in Medical Care · Cited by 12 · see citing works
Some Considerations in the Diffusion of Medical Technologies: Medical Information Systems
Brenner, D. J., & Logan, R. A. (1980) · Annals of the International Communication Association · doi:10.1080/23808985.1980.11923828 · Cited by 12 · see citing works
Evaluating the impact of MEDLINE using the Critical Incident Technique.
Siegel, E. R., Rapp, B. A., & Lindberg, D. A. B. (1991) · PubMed · Cited by 11 · see citing works
Barriers to Medical Computing: History, Diagnosis, and Therapy for the Medical Computing ‘Lag’
Kaplan, B. (1985) · PubMed Central · Cited by 11 · see citing works
Self-citationEvaluation of a hospital-wide PACS: costs and benefits of the Hammersmith PACS installation
Bryan, S., Keen, J., Buxton, M. J., et al. (1992) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.60317 · Cited by 11 · see citing works
The diffusion of computer applications in medical settings
Anderson, J. G., & Jay, S. J. (1984) · Medical Informatics · doi:10.3109/14639238409015198 · Cited by 10 · see citing works
Fitting System Design to Work Practice:Using Observation in Evaluating A Clinical Imaging System
Kaplan, B. (1995) · Journal of the Association for Information Systems · Cited by 7 · see citing works
Self-citationDiffusion and Impact of Computer in Organizational Settings: Empirical Findings form a Hospital.
Anderson, J. G., Jay, S. J., Schweer, H. M., et al. (1987) · International Conference on Human-Computer Interaction · Cited by 6 · see citing works
Experience with an architecture for integrating images into a hospital information system
Dayhoff, R. E., Kuzmak, P. M., Maloney, D. L., et al. (2002) · doi:10.1109/cbms.1991.128953 · Cited by 5 · see citing works
Kaplan, B., & Duchon, D. (1989) · PubMed Central · Cited by 5 · see citing works
Self-citationAn integrated multidepartmental hospital imaging system: usage of data across specialties.
Dayhoff, R. E., & Maloney, D. L. (1992) · PubMed · Cited by 3 · see citing works
Conformational statistics of the nitrogen linkage in glycopeptides using umbrella sampling
Duben, A. J. (1995) · ACM SIGBIO Newsletter · doi:10.1145/228243.228244 · Cited by 1 · see citing works
Flagged first, then by citations
Research works printed but not linked: 12
- Anderson JG, Aydin CE, Jay SJ, eds. Evaluating Health Care Information Systems: Approaches and Applications. Thou- sand Oaks, CA: Sage Publications, 1994.
- Nyce JM, Graves W III. The construction of neurology: im- plications for hypermedia system development. Artif Intell Med. 1990;2:315 -22.
- Anderson JG, Aydin CE. Theoretical perspectives and meth- odologies for the evaluation of health care information sys- tems. In: Anderson JG, Aydin CE, and Jay SJ (eds). Evalu- ating Health Care Information Systems: Methods and Applications. Thousand Oaks, CA: Sage Publications, 1994; 5 -29.
- Kaplan B. A model comprehensive evaluation plan for com- plex information systems: clinical imaging systems as an example, European Conference on Information Technology Investment Evaluation Proceedings, 1995;174 -81.
- Anderson JG, Aydin CE. Theoretical perspectives and meth- odologies for the evaluation of health care information sys- tems. In: Anderson JG, Aydin CE, Jay SJ (eds). Evaluating Health Care Information Systems: Methods and Applica- tions. Thousand Oaks, CA: Sage Publications, 1994;5 -29.
- Kolb DA, Frohman AL. An organization development ap- proach to consulting. Sloan Manage Rev. 1970;12:51 -65.
- Rogers EM. Diffusion of Innovations, 3rd ed. New York: The Free Press, 1983.
- Kaplan B, Duchon D. Combining qualitative and quantita- tive approaches in information systems research: a case study. MIS Quarterly. 1988;4:571 -86.
- Kaplan B. Organizational evaluation of medical information systems. In: Friedman CP, Wyatt JC (eds). Evaluation Meth- ods in Medical Informatics. New York: Springer-Verlag, 1996.
- Keen J, Stirling B, Muris N, Weatherburn G, Buxton M. A model for the evaluation of PACS. SCAR Proceedings, 1994; 22 -9.
- Dayhoff RE, Maloney DL, Kuzmak PM, Shepard BM. Inte- grating medical images into hospital information systems. J Digit Imaging. 1991;4:87 -93.
- Dayhoff RE, Kuzmak P, Maloney D. Medical images as an integral part of the patient's automated record. Symp Com- puter-Assisted Radiology Proceedings. Baltimore, MD, 1991.
Cited by
Sorted by citations
The Technology Acceptance Model: Its past and its future in health care
Holden, R. J., & Karsh, B.-T. (2010) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2009.07.002 · Cited by 2,669 · see citing works
Patient care information systems and health care work: a sociotechnical approach
Berg, M. (1999) · International Journal of Medical Informatics · doi:10.1016/s1386-5056(99)00011-8 · Cited by 786 · see citing works
Implementing information systems in health care organizations: myths and challenges
Berg, M. (2001) · International Journal of Medical Informatics · doi:10.1016/s1386-5056(01)00200-3 · Cited by 715 · see citing works
Kaplan, B. (2001) · International Journal of Medical Informatics · doi:10.1016/s1386-5056(01)00184-8 · Cited by 380 · see citing works
Self-citationSchaper, L. K., & Pervan, G. P. (2007) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2006.05.028 · Cited by 371 · see citing works
Determinants of Success of Inpatient Clinical Information Systems: A Literature Review
van der Meijden, M. J. (2003) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1094 · Cited by 363 · see citing works
Kaplan, B. (2020) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2020.104239 · Cited by 303 · see citing works
Self-citationHolden, R. J., & Karsh, B.-T. (2009) · Behaviour & Information Technology · doi:10.1080/01449290601138245 · Cited by 128 · see citing works
The alignment of information systems with organizational objectives and strategies in health care
Bush, M., Lederer, A. L., Li, X., Palmisano, J., & Rao, S. (2009) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2009.02.004 · Cited by 99 · see citing works
A methodology for shifting the focus of e-health support design onto user needs
De Rouck, S., Jacobs, A., & Leys, M. (2008) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2007.11.004 · Cited by 70 · see citing works
Kazanjian, A., & Green, C. J. (2002) · Computers in Biology and Medicine · doi:10.1016/s0010-4825(02)00013-6 · Cited by 60 · see citing works
Horan, T. A., Tulu, B., Hilton, B., & Burton, J. (2004) · 37th Annual Hawaii International Conference on System Sciences, 2004. Proceedings of the · doi:10.1109/hicss.2004.1265364 · Cited by 38 · see citing works
From expert-driven to user-oriented communication of infection control guidelines
Verhoeven, F., Steehouder, M. F., Hendrix, R. M. G., & Van Gemert-Pijnen, J. E. W. C. (2010) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2009.07.003 · Cited by 24 · see citing works
- Authors
- Kaplan, B.
- Affiliations
- Quinnipiac University (US)
- Corresponding author
- Kaplan, B.
- ORCID
- Kaplan, B.
- Venue
- Journal of the American Medical Informatics Association
- Publisher
- Oxford University Press (OUP)
- Published
- 1997-03-01
- Volume
- 4
- Issue
- 2
- Pages
- 94-101
- ISSN
- 1067-5027 (ISSN-L)
- 1527-974X
- PMC
- PMC61498
- OpenAlex
- W2147164934
- Cited by
- 175
- References
- 55
- Citations by year
- 2026: 1 · 2025: 2 · 2023: 1 · 2022: 2 · 2020: 3 · 2019: 4 · 2018: 3 · 2017: 2 · 2016: 4 · 2015: 2 · 2014: 3 · 2013: 7 · 2012: 8
- FWCI
- 14.66
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Journal of the American Medical Informatics Association (publishedVersion)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Social Sciences › Sociology and Political Science › Information Systems Theories and Implementation
- Social Sciences › Decision Sciences › Management Science and Operations Research › Evaluation and Performance Assessment
- Keywords
- organizational challenges, communication, care, control, context, medical informatics
- MeSH terms
- Communication, Documentation, Humans, Interprofessional Relations, Management Information Systems, Organizational Innovation, Research Design, Program Evaluation, Models, Organizational
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
