Using FDA reports to inform a classification for health information technology safety problems
Open accessAbstract
OBJECTIVE: To expand an emerging classification for problems with health information technology (HIT) using reports submitted to the US Food and Drug Administration Manufacturer and User Facility Device Experience (MAUDE) database. DESIGN: HIT events submitted to MAUDE were retrieved using a standardized search strategy. Using an emerging classification with 32 categories of HIT problems, a subset of relevant events were iteratively analyzed to identify new categories. Two coders then independently classified the remaining events into one or more categories. Free-text descriptions were analyzed to identify the consequences of events. MEASUREMENTS: Descriptive statistics by number of reported problems per category and by consequence; inter-rater reliability analysis using the κ statistic for the major categories and consequences. RESULTS: A search of 899 768 reports from January 2008 to July 2010 yielded 1100 reports about HIT. After removing duplicate and unrelated reports, 678 reports describing 436 events remained. The authors identified four new categories to describe problems with software functionality, system configuration, interface with devices, and network configuration; the authors' classification with 32 categories of HIT problems was expanded by the addition of these four categories. Examination of the 436 events revealed 712 problems, 96% were machine-related, and 4% were problems at the human-computer interface. Almost half (46%) of the events related to hazardous circumstances. Of the 46 events (11%) associated with patient harm, four deaths were linked to HIT problems (0.9% of 436 events). CONCLUSIONS: Only 0.1% of the MAUDE reports searched were related to HIT. Nevertheless, Food and Drug Administration reports did prove to be a useful new source of information about the nature of software problems and their safety implications with potential to inform strategies for safe design and implementation.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
A few studies have attempted to quantify and classify EHR-related safety concerns by mining patient safety incident reporting databases.
“A few studies have attempted to quantify and classify EHR-related safety concerns by mining patient safety incident reporting databases.18 29–31”
From An analysis of electronic health record-related patient safety concerns (2014) · cites this paper as [29]
Previous studies have largely used isolated event reports without benefit of an independent human factors and informatics investigation to analyze safety concerns in the EHR.
“Previous studies have largely used isolated event reports without benefit of an independent human factors and informatics investigation to analyze safety concerns in the EHR.”
From An analysis of electronic health record-related patient safety concerns (2014) · cites this paper as [29]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The Measurement of Observer Agreement for Categorical Data
Landis, J. R., & Koch, G. G. (1977) · Biometrics · doi:10.2307/2529310 · Cited by 81,148 · see citing works
(1993) · Cited by 9,879 · see citing works
Reason, J. (1990) · doi:10.1017/cbo9781139062367 · Cited by 5,108 · see citing works
The Usability Engineering Lifecycle
NIELSEN, J. (1993) · Usability Engineering · doi:10.1016/b978-0-08-052029-2.50007-3 · Cited by 2,657 · see citing works
Ash, J. S. (2003) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1471 · Cited by 1,789 · see citing works
Jacko, J. A., & Sears, A. (2013) · Choice Reviews Online · doi:10.5860/choice.50-3307 · Cited by 1,360 · see citing works
Ammenwerth, E., Schnell‐Inderst, P., Machan, C., & Siebert, U. (2008) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2667 · Cited by 669 · see citing works
Towards an International Classification for Patient Safety: key concepts and terms
Runciman, W., Hibbert, P., Thomson, R., Van Der Schaaf, T., Sherman, H., & Lewalle, P. (2009) · International Journal for Quality in Health Care · doi:10.1093/intqhc/mzn057 · Cited by 647 · see citing works
Self-citationHealth information technology: fallacies and sober realities
Karsh, B.-T., Weinger, M. B., Abbott, P. A., & Wears, R. L. (2010) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.2010.005637 · Cited by 365 · cited 1× · see citing works
“CPOE (1) did not present procedures and tests as a standardized list; (2) did not adequately distinguish the range of preparations available for a medication in the list presented to users; (3) were unable to handle variable dose medications (eg, when venlafaxine was ordered as 150 mg daily at 09:00 and 75 mg daily at…”
Advances in Patient Safety: From Research to Implementation (Volume 2: Concepts and Methodology)
Henriksen, K., Battles, J. B., Marks, E. S., & Lewin, D. I. (2005) · Cited by 274 · see citing works
Hauser, R. G., Katsiyiannis, W. T., Gornick, C. C., Almquist, A. K., & Kallinen, L. M. (2010) · Europace · doi:10.1093/europace/eup375 · Cited by 264 · see citing works
Bloomrosen, M., Starren, J. B., Lorenzi, N. M., Ash, J. S., Patel, V. L., & Shortliffe, E. H. (2010) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.2010.007567 · Cited by 181 · see citing works
Magrabi, F., Ong, M.-S., Runciman, W., & Coiera, E. (2010) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.2009.002444 · Cited by 162 · see citing works
Self-citationRunciman, W. B., Williamson, J. A. H., Deakin, A., Benveniste, K. A., Bannon, K., & Hibbert, P. D. (2006) · Quality in Health Care · doi:10.1136/qshc.2005.017467 · Cited by 156 · see citing works
Self-citationJacko, J. A. (2003) · CRC Press eBooks · doi:10.1201/9781482269512 · Cited by 143 · see citing works
Safety and Ethics in Healthcare: A Guide to Getting it Right
Runciman, B., Merry, A., & Walton, M. (2017) · CRC Press eBooks · doi:10.1201/9781315607443 · Cited by 141 · see citing works
EHR Safety: The Way Forward to Safe and Effective Systems
Walker, J. M., Carayon, P., Leveson, N., Paulus, R. A., Tooker, J., Chin, H., Bothe, A., & Stewart, W. F. (2008) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2618 · Cited by 113 · see citing works
Crisis management during anaesthesia: the development of an anaesthetic crisis management manual
Runciman, W. B. (2005) · Quality and Safety in Health Care · doi:10.1136/qshc.2002.004101 · Cited by 92 · see citing works
Self-citationThe safety and quality of decision support systems.
Coiera, E. W., Westbrook, J. I., & Wyatt, J. C. (2006) · PubMed · Cited by 92 · see citing works
Self-citationSetting priorities for patient safety
Runciman, W. B. (2002) · Quality and Safety in Health Care · doi:10.1136/qhc.11.3.224 · Cited by 77 · see citing works
Self-citationSafe Electronic Health Record Use Requires a Comprehensive Monitoring and Evaluation Framework
Sittig, D. F. (2010) · JAMA · doi:10.1001/jama.2010.61 · Cited by 76 · see citing works
Advances in Patient Safety: From Research to Implementation (Volume 1: Research Findings)
Henriksen, K., Battles, J. B., Marks, E. S., & Lewin, D. I. (2005) · Cited by 72 · see citing works
Myers, R. B., Jones, S. L., & Sittig, D. F. (2011) · Applied Clinical Informatics · doi:10.4338/aci-2010-11-ra-0064 · Cited by 67 · see citing works
Goodman, K. W., Berner, E. S., Dente, M. A., Kaplan, B., Koppel, R., Rucker, D., Sands, D. Z., Winkelstein, P., & for the AMIA Board of Directors (2011) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.2010.008946 · Cited by 62 · see citing works
Symposium—The Australian Incident Monitoring Study
Holland, R., & Hains, J. (1993) · Anaesthesia and Intensive Care · doi:10.1177/0310057x9302100502 · Cited by 53 · see citing works
Magrabi, F., Ong, M.-S., Runciman, W. B., & Coiera, E. W. (2011) · PubMed Central · Cited by 48 · see citing works
Self-citationSection 1: Health and Clinical Mangement: The Safety and Quality of Decision Support Systems
Westbrook, J. I., Wyatt, J. C., & Coiera, E. W. (2006) · Yearbook of Medical Informatics · doi:10.1055/s-0038-1638469 · Cited by 20 · see citing works
Self-citationMaking Information Technology a Team Player in Safety: The Case of Infusion Devices
Nemeth, C., Nunnally, M., O’Connor, M. F., et al. (2005) · Cited by 11 · see citing works
Henriksen, K., Jb, B., Es, M., & Di, L. (2005) · Cited by 1 · see citing works
Flagged first, then by citations
Research works printed but not linked: 1
- Reason J. Human Error. Cambridge: Cambridge University Press, 1990.
Cited by
Sorted by citations
Kim, M. O., Coiera, E., & Magrabi, F. (2017) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocw154 · Cited by 263 · see citing works
Self-citationAn analysis of electronic health record-related patient safety concerns
Meeks, D. W., Smith, M. W., Taylor, L., Sittig, D. F., Scott, J. M., & Singh, H. (2014) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2013-002578 · Cited by 207 · see citing works
Electronic health records and national patient-safety goals
Sittig, D. F., & Singh, H. (2012) · New England Journal of Medicine · doi:10.1056/nejmsb1205420 · Cited by 204 · see citing works
Singh, H., & Sittig, D. F. (2016) · BMJ Quality & Safety · doi:10.1136/bmjqs-2015-004486 · Cited by 167 · see citing works
The unintended consequences of health information technology revisited
Ash, J., Berg, M., & Coiera, E. (2016) · Yearbook of Medical Informatics · doi:10.15265/iy-2016-014 · Cited by 101 · see citing works
Self-citationMagrabi, F., Baker, M., Sinha, I., Ong, M.-S., Harrison, S., Kidd, M. R., Runciman, W. B., & Coiera, E. (2015) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2014.12.003 · Cited by 100 · see citing works
Self-citationTsou, A., Lehmann, C., Michel, J., Solomon, R., Possanza, L., & Gandhi, T. (2017) · Applied Clinical Informatics · doi:10.4338/aci-2016-09-r-0150 · Cited by 100 · see citing works
Meeks, D. W., Takian, A., Sittig, D. F., Singh, H., & Barber, N. (2014) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2013-001762 · Cited by 91 · see citing works
The contribution of sociotechnical factors to health information technology-related sentinel events
Castro, G. M., Buczkowski, L., & Hafner, J. M. (2016) · The Joint Commission Journal on Quality and Patient Safety · doi:10.1016/s1553-7250(16)42008-8 · Cited by 48 · see citing works
Rahman Jabin, M. S., Pan, D., & Nilsson, E. (2022) · Health Informatics Journal · doi:10.1177/14604582221105440 · Cited by 40 · see citing works
Edwards, P., & Warm, D. (2012) · Applied Clinical Informatics · doi:10.4338/aci-2012-03-ra-0010 · Cited by 38 · see citing works
Electronic health record–related safety concerns: A cross‐sectional survey
Menon, S., Singh, H., Meyer, A. N. D., Belmont, E., & Sittig, D. F. (2014) · Journal of Healthcare Risk Management · doi:10.1002/jhrm.21146 · Cited by 29 · see citing works
Resilient practices in maintaining safety of health information technologies
Smith, M. W., Ash, J. S., Sittig, D. F., & Singh, H. (2014) · Journal of Cognitive Engineering and Decision Making · doi:10.1177/1555343414534242 · Cited by 26 · see citing works
Health technology, quality and safety in a learning health system
Borycki, E. M., & Kushniruk, A. W. (2023) · Healthcare Management Forum · doi:10.1177/08404704221139383 · Cited by 23 · see citing works
Syndromic surveillance for health information system failures: a feasibility study
Ong, M.-S., Magrabi, F., & Coiera, E. (2013) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2012-001144 · Cited by 22 · see citing works
Self-citationJabin, M. S. R., Magrabi, F., Hibbert, P., Schultz, T., & Runciman, W. (2019) · 2019 IEEE International Conference on Imaging Systems and Techniques (IST) · doi:10.1109/ist48021.2019.9010280 · Cited by 20 · see citing works
Self-citationEssential activities for electronic health record safety: A qualitative study
Ash, J. S., Singh, H., Wright, A., Chase, D., & Sittig, D. F. (2020) · Health Informatics Journal · doi:10.1177/1460458219833109 · Cited by 12 · see citing works
- Authors
- Magrabi, F., Ong, M.-S., Runciman, W., & Coiera, E.
- Affiliations
- UNSW Sydney (AU)
- University of South Australia (AU)
- Corresponding author
- Magrabi, Farah
- Venue
- Journal of the American Medical Informatics Association
- Publisher
- Oxford University Press (OUP)
- Published
- 2012-01
- Volume
- 19
- Issue
- 1
- Pages
- 45-53
- ISSN
- 1067-5027 (ISSN-L)
- 1527-974X
- PMC
- PMC3240763
- OpenAlex
- W2241024151
- Cited by
- 193
- References
- 29
- Citations by year
- 2026: 8 · 2025: 4 · 2024: 3 · 2023: 5 · 2022: 9 · 2021: 8 · 2020: 7 · 2019: 20 · 2018: 7 · 2017: 25 · 2016: 23 · 2015: 27 · 2014: 15 · 2013: 17 · 2012: 11
- FWCI
- 23.65
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Journal of the American Medical Informatics Association (publishedVersion)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- UniSA Research Outputs Repository (University of South Australia) (submittedVersion)
- Adelaide Research & Scholarship (AR&S) (University of Adelaide) (submittedVersion)
- Funders
- National Health and Medical Research Council (630583, 568612)
- Topics
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Health Sciences › Health Professions › Emergency Medical Services › Patient Safety and Medication Errors
- Life Sciences › Pharmacology, Toxicology and Pharmaceutics › Toxicology › Pharmacovigilance and Adverse Drug Reactions
- Keywords
- MAUDE database, system configuration, device interfaces, network configuration, patient harm, human-machine interaction, software safety
- MeSH terms
- Medical Informatics, Safety, Software, United States, United States Food and Drug Administration, Databases, Factual, Research Report
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
