The DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index.
Multimethod Evaluation of Information and Communication Technologies in Health in the Context of Wicked Problems and Sociotechnical Theory
Open accessAbstract
OBJECTIVE: Few research designs look at the deep structure of complex social systems. We report the design and implementation of a multimethod evaluation model to assess the impact of computerized order entry systems on both the technical and social systems within a health care organization. DESIGN: We designed a multimethod evaluation model informed by sociotechnical theory and an appreciation of the nature of wicked problems. We mobilized this model to assess the impact of an electronic medication management system via a three-year program of research at a major academic hospital. MEASUREMENTS: Model components include measurements relating to three dimensions of system impact: safety and quality, organizational culture, and work and communication patterns. RESULTS: Application of the evaluation model required the development and testing of purpose-built measurement tools such as software to collect multidimensional work measurement data. The model applied established research methods including medication error audits and social network analysis. Design features of these tools and techniques are described, along with the practical challenges of their implementation. The distinctiveness of doing research within a unique paradigm of complex systems, explicating the wickedness and the dimensionality of sociotechnical theory, is articulated. CONCLUSION: Designing an effective evaluation model requires a deep understanding of the nature and complexity of the problems that information technology interventions in health care are trying to address. Adopting a sociotechnical perspective for model generation improves our ability to develop evaluation models that are adaptive and sensitive to the characteristics of wicked problems and provides a strong theoretical basis from which to analyze and interpret findings.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Dilemmas in a general theory of planning
Rittel, H. W. J., & Webber, M. M. (1973) · Policy Sciences · doi:10.1007/bf01405730 · Cited by 16,338 · cited 1×
“Information and communication technologies are being targeted as a possible way of addressing what is in essence a “wicked problem”,”
Trist, E. L., & Bamforth, K. W. (1951) · Human Relations · doi:10.1177/001872675100400101 · Cited by 3,379
Incidence of Adverse Drug Events and Potential Adverse Drug Events: Implications for Prevention
Bates, D. W. (1995) · JAMA · doi:10.1001/jama.1995.03530010043033 · Cited by 2,127
Bates, D. W. (1998) · JAMA · doi:10.1001/jama.280.15.1311 · Cited by 2,107
The Impact of Computerized Physician Order Entry on Medication Error Prevention
Bates, D. W., Teich, J. M., Lee, J., Seger, D., Kuperman, G. J., Ma'Luf, N., Boyle, D., & Leape, L. (1999) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.1999.00660313 · Cited by 1,260
Relationship between medication errors and adverse drug events
Bates, D. W., Boyle, D. L., Vliet, M. B. V., Schneider, J., & Leape, L. (1995) · Journal of General Internal Medicine · doi:10.1007/bf02600255 · Cited by 981
Reducing the Frequency of Errors in Medicine Using Information Technology
Bates, D. W., Cohen, M., Leape, L. L., Overhage, J. M., Shabot, M. M., & Sheridan, T. (2001) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.2001.0080299 · Cited by 959
Patient care information systems and health care work: a sociotechnical approach
Berg, M. (1999) · International Journal of Medical Informatics · doi:10.1016/s1386-5056(99)00011-8 · Cited by 786
Does automation bias decision-making?
SKITKA, L. J., MOSIER, K. L., & BURDICK, M. (1999) · International Journal of Human-Computer Studies · doi:10.1006/ijhc.1999.0252 · Cited by 694 · cited 1×
“In the post–system implementation period, the electronic medication charts will be reviewed using the same approach as in the preintervention stage. A focus also will be the detection of new types of errors related to both system design features and automation bias.”
Chisholm, C. D., Collison, E. K., Nelson, D. R., & Cordell, W. H. (2000) · Academic Emergency Medicine · doi:10.1111/j.1553-2712.2000.tb00469.x · Cited by 610
Cited by
Lluch, M. (2011) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2011.09.005 · Cited by 358
The impact of clinical leadership on health information technology adoption: systematic review
Ingebrigtsen, T., Georgiou, A., Clay-Williams, R., Magrabi, F., Hordern, A., Prgomet, M., Li, J., Westbrook, J., & Braithwaite, J. (2014) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2014.02.005 · Cited by 165
Self-citationGuideline for good evaluation practice in health informatics (GEP-HI)
Nykänen, P., Brender, J., Talmon, J., de Keizer, N., Rigby, M., Beuscart-Zephir, M.-C., & Ammenwerth, E. (2011) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2011.08.004 · Cited by 105
Saleem, J. J., Russ, A. L., Sanderson, P., Johnson, T. R., Zhang, J., & Sittig, D. F. (2009) · Yearbook of Medical Informatics · doi:10.1055/s-0038-1638638 · Cited by 72
“Finding the right time to perform such evaluations, and making sure that the results influence development and rollout decisions, are key problems for large-scale CIS projects [52, 53].”
Background
Metadata
- Authors
- Westbrook, J. I., Braithwaite, J., Georgiou, A., Ampt, A., Creswick, N., Coiera, E., & Iedema, R.
- Venue
- Journal of the American Medical Informatics Association
- Publisher
- Oxford University Press (OUP)
- Published
- 2007-11-01
- Volume
- 14
- Issue
- 6
- Pages
- 746-755
- ISSN
- 1067-5027 (ISSN-L)
- 1527-974X
- OpenAlex
- W2155143189
- Cited by
- 174
- References
- 13
- FWCI
- 26.46
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Multimethod Evaluation of Information and Communication Technologies in Health in the Context of Wicked Problems and Sociotechnical Theory. https://app.cochat.ai/papers/10.1197/jamia.m2462
- The paper
- Westbrook, J. I., Braithwaite, J., Georgiou, A., Ampt, A., Creswick, N., Coiera, E., & Iedema, R. (2007). Multimethod Evaluation of Information and Communication Technologies in Health in the Context of Wicked Problems and Sociotechnical Theory. Journal of the American Medical Informatics Association. https://doi.org/10.1197/jamia.m2462
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.