Notification of Abnormal Lab Test Results in an Electronic Medical Record: Do Any Safety Concerns Remain?
Open accessAbstract
Background: Follow-up of abnormal outpatient laboratory test results is a major patient safety concern. Electronic medical records can potentially address this concern through automated notification. We examined whether automated notifications of abnormal laboratory results (alerts) in an integrated electronic medical record resulted in timely follow-up actions. Methods: We studied four alerts: hemoglobin A1c (HbA1c) ≥15%, positive hepatitis C antibody (HCV), prostate specific antigen (PSA) ≥15 ng/mL, and thyroid stimulating hormone (TSH) ≥ 15 mIU/L. An alert tracking system determined whether the alert was acknowledged (i.e. provider clicked on and opened the message) within two weeks of transmission; acknowledged alerts were considered read. Within 30 days of result transmission, record review and provider contact determined follow-up actions (e.g. patient contact, treatment etc.). Multivariable logistic regression models analyzed predictors for lack of timely follow-up. Results: Between May 2008 and December 2008, 78,158 tests (HbA1c, HCV, TSH and PSA) were performed, of which 1163 (1.48%) were transmitted as alerts; 10.2% of these (119/1163) were unacknowledged. Timely follow-up was lacking in 79 (6.8%) and was statistically not different for acknowledged and unacknowledged alerts (6.4% vs. 10.1%; p =.13). Two-hundred two alerts (17.4% of 1163) arose from unnecessarily ordered (redundant) tests. Alerts for a new versus known diagnosis were more likely to lack timely follow-up (OR: 7.35; 95% CI: 4.16-12.97) whereas alerts related to redundant tests were less likely to lack timely follow-up (OR: 0.24; 95% CI: 0.07-0.84). Conclusions: Safety concerns related to timely patient follow-up remain despite automated notification of non-life threatening abnormal laboratory results in the outpatient setting.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
0 of 24 cited sources checked · 0 supported · 0 flagged
- Findingown data
202 alerts (17.4% of 1,163) were for tests judged redundant or unnecessarily ordered.
“Two-hundred two alerts (17.4% of 1163) arose from unnecessarily ordered (redundant) tests.”
- Also stated in: § RESULTS · plain
“Two-hundred two alerts (17.4% of 1163) were considered redundant”
Quote in the paperNot a substantial claim - Also stated in: § RESULTS · plain
- Findingown data
The proportion of alerts lacking timely follow-up was not significantly different between acknowledged and unacknowledged alerts (6.4% vs 10.1%; p=.13).
“Timely follow-up was lacking in 79 (6.8%) and was statistically not different for acknowledged and unacknowledged alerts (6.4% vs. 10.1%; p =.13).”
- Also stated in: § Lack of Timely Follow-up · plain
“Statistically, there was no significant difference in rates of lack of timely follow-up between acknowledged and unacknowledged laboratory alerts (6.4% vs. 10.1%; p=.13).”
Quote in the paperSubstantial claim - Also stated in: § Lack of Timely Follow-up · plain
- Findingown data
Of 159 alerts deemed redundant because of repetition, only 11 (7%) would have triggered a computerized reminder about potential redundancy.
“in only 11 (7%) of these 159 cases would the provider have received a computerized reminder prompting them of potential redundancy based on repetition.”
Quote in the paperSubstantial claim - Findingown data
Timely follow-up was absent for 79 of the 1,163 alerts (6.8%).
“Timely follow-up was lacking in 79 (6.8%)”
- Also stated in: § RESULTS · plain
“Of these, 79 alerts (6.8% of total) were determined to lack timely follow-up.”
Quote in the paperSubstantial claim - Also stated in: § RESULTS · plain
- Findingown data
In the study period from May to December 2008, 78,158 HbA1c, PSA, HCV, and TSH tests were performed at the participating VA facility.
“Between May 2008 and December 2008, 27,092 HbA1c, 22,837 PSA, 6271 HCV and 21,958 TSH tests were performed.”
- Also stated in: § Results: · plain
“Between May 2008 and December 2008, 78,158 tests (HbA1c, HCV, TSH and PSA) were performed”
Quote in the paperNot a substantial claim - Also stated in: § Results: · plain
- Findingown data
In multivariable analysis, allied health providers and trainees had higher odds of not acknowledging alerts than attending physicians (OR 4.32, 95% CI 1.21-15.52; OR 8.39, 95% CI 2.97-23.68).
“allied health care providers as ordering providers (OR, 4.32; 95% CI: 1.21-15.52) and trainees as ordering providers (OR, 8.39, 95% CI: 2.97-23.68).”
Quote in the paperSubstantial claim - Findingown data
Among the 1,163 alerts, 119 (10.2%) were not acknowledged.
“10.2% of these (119/1163) were unacknowledged.”
- Also stated in: § RESULTS · plain
“Acknowledged alerts constituted 89.8% of the total high-priority alerts (1163).”
Quote in the paperSubstantial claim - Also stated in: § RESULTS · plain
- Findingown data
Of the 78,158 tests, 1,163 (about 1.5%) were sent as mandatory high-priority alerts.
“A total of 1163 (1.49%) results were electronically transmitted as mandatory high-priority alerts”
- Also stated in: § Results: · plain
“of which 1163 (1.48%) were transmitted as alerts”
Quote in the paperNot a substantial claim - Also stated in: § Results: · plain
- Findingown data
Alerts for redundant tests were less likely to lack timely follow-up than alerts for appropriately ordered tests (OR 0.24; 95% CI 0.07-0.84).
“alerts related to redundant tests were less likely to lack timely follow-up (OR: 0.24; 95% CI: 0.07-0.84).”
- Also stated in: § Lack of Timely Follow-up · plain
“Redundant tests were more likely to receive follow-up (p<.01).”
- Also stated in: § Lack of Timely Follow-up · plain
“redundant tests (OR: 0.24; 95% CI: 0.074-0.84) with appropriately ordered tests as referent”
Quote in the paperSubstantial claim - Also stated in: § Lack of Timely Follow-up · plain
- Findingown data
Mental health specialty was associated with a higher odds of lacking timely follow-up compared with primary care (OR 2.82; 95% CI 1.06-7.54).
“mental health as a specialty (OR: 2.82; 95% CI: 1.06-7.54) with primary care as referent.”
Quote in the paperSubstantial claim - Findingown data
Alerts for a newly diagnosed condition were more likely to lack timely follow-up than alerts for a previously known condition (OR 7.35; 95% CI 4.16-12.97).
“Alerts for a new versus known diagnosis were more likely to lack timely follow-up (OR: 7.35; 95% CI: 4.16-12.97)”
- Also stated in: § Lack of Timely Follow-up · plain
“Alerts for conditions signifying new diagnoses were more likely to lack timely follow-up than alerts for pre-existing conditions (p<.0001).”
- Also stated in: § Lack of Timely Follow-up · plain
“alerts for conditions newly diagnosed i.e., diagnosis was not made until the lab was done (OR: 7.35; 95% CI: 4.16-12.97) with previously known diagnosis as referent”
Quote in the paperSubstantial claim - Also stated in: § Lack of Timely Follow-up · plain
- Interpretationown data
Safety concerns about timely follow-up remain even when non-life-threatening abnormal lab results are automatically notified in outpatient care.
“Safety concerns related to timely patient follow-up remain despite automated notification of non-life threatening abnormal laboratory results in the outpatient setting.”
- Also stated in: § DISCUSSION · plain
“current systems of mandatory automated notification of abnormal lab results do not guarantee timely follow-up on the abnormality in the outpatient setting.”
Quote in the paperSubstantial claim - Also stated in: § DISCUSSION · plain
- Interpretationreasoning
The pattern of incomplete follow-up may extend to all alerts of abnormal diagnostic test results, not only laboratory alerts.
“These findings are similar to our previous findings of follow-up on abnormal imaging alerts in the same electronic medical record suggesting that this phenomenon may exist for all alerts of abnormal diagnostic test results.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [19] Singh, 2009, Timely follow-up of abnormal diagnostic imaging test results in an outpatient setting: are electronic medical records achieving their potential?, Arch Intern Med source
Not checked in the free run.
- Premisecitation
Missed laboratory results or delayed recognition of results creates a significant potential for outpatient diagnostic errors, adverse events, and liability claims.
“missed laboratory results or delayed recognition of results leads to a significant potential for outpatient diagnostic errors, adverse events and liability claims.”
Quote in the paperSubstantial claim0 of 9 sources checked- cites [4] Poon, 2004, “I wish I had seen this test result earlier!”: dissatisfaction with test result management systems in primary care, Arch Intern Med source
Not checked in the free run.
- cites [9] Schiff, 2005, Introduction: communicating critical test results, Jt Comm J Qual Patient Saf
Not checked in the free run.
- cites [11] Gandhi, 2005, Fumbled handoffs: one dropped ball after another, Ann Intern Med source
Not checked in the free run.
- cites [12] Poon, 2004, Communication factors in the follow-up of abnormal mammograms, J Gen Intern Med source
Not checked in the free run.
- cites [13] Gandhi, 2006, Missed and delayed diagnoses in the ambulatory setting: a study of closed malpractice claims, Ann Intern Med source
Not checked in the free run.
- cites [14] Graber, 2005, Diagnostic error in internal medicine, Arch Intern Med source
Not checked in the free run.
- cites [15] Singh, 2007, Communication outcomes of critical imaging results in a computerized notification system, J Am Med Inform Assoc source
Not checked in the free run.
- cites [16] Singh, 2008, Reducing diagnostic errors through effective communication: harnessing the power of information technology, J Gen Intern Med source
Not checked in the free run.
- cites [17] Singh, 2007, Errors in cancer diagnosis: current understanding and future directions, J Clin Oncol source
Not checked in the free run.
- Interpretationown data
Even when electronic alerts reach providers, follow-up of abnormal outpatient test results is not fail-safe.
“suggesting that follow-up of abnormal outpatient test results is not fail-safe even when providers are alerted about abnormal results through the electronic medical record.”
Quote in the paperSubstantial claim - Premisecitation
Integrated electronic medical record systems that automatically notify providers of abnormal test results may provide a solution.
“Automated systems notifying providers about abnormal test results in integrated electronic medical record systems offer a potential solution.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [18] Poon, 2003, Design and implementation of a comprehensive outpatient Results Manager, J Biomed Inform source
Not checked in the free run.
- Interpretationreasoning
The findings may not be generalizable outside the VA because the study population was predominantly male veterans and the setting was unique to VA.
“Because of the study population (e.g. predominantly male veterans) and the unique VA setting, our findings may not be generalizable outside the VA.”
Quote in the paperNot a substantial claim - Interpretationown data
Acknowledgment of an abnormal test result by a provider does not by itself ensure that timely follow-up occurs.
“provider acknowledgment of receipt of the test result also does not automatically result in timely follow-up.”
- Also stated in: § DISCUSSION · plain
“it cannot be assumed that automated notification of abnormal lab results within an electronic medical record and the resultant acknowledgement will translate into timely actions to address these alerts.”
Quote in the paperSubstantial claim- Rests on an assumption: Follow-up of an abnormal result depends on providers reviewing and acting on the electronic communication of the result.
- Also stated in: § DISCUSSION · plain
- Premisecitation
In inpatient settings, computerized reminders have been shown to cut down on redundant testing.
“Computerized reminders have been shown to reduce redundant tests in the inpatient setting.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [24] Bates, 1999, A randomized trial of a computer-based intervention to reduce utilization of redundant laboratory tests, Am J Med source
Not checked in the free run.
- May combine two claims. 8
- Classified as probably not a substantial claim. 5
May combine two claims.
Findingown dataTrainees and specialty services were less likely to acknowledge alerts than attending/staff physicians and primary care providers, respectively (p<.0001 for both).
“Trainees were less likely to acknowledge alerts compared to attending/staff physicians and allied health providers (physician assistants and nurse practitioners) whereas specialty services (including mental health) were less likely to acknowledge alerts compared to primary care providers (p<.0001 for both).”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataAmong the 159 repetition-redundant alerts, 28 involved a test ordered too soon after the previous PSA, TSH, or HbA1c, and 131 involved a repeat HCV ELISA despite an earlier positive result.
“we found 28 cases where the provider ordered the test too soon after the last PSA, TSH or HbA1c and in 131 cases a second HCV ELISA test was ordered despite the presence of a previous positive test.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataNo documented follow-up action was found for 307 alerts (26.4%), and 213 of the 1,163 alerts (18.3%) were classified as no-impact.
“No evidence of documented follow-up action was found in 307 (26.4%) of the alerts, however 213 (18.3% of 1163) cases were categorized as no-impact alerts.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductreasoningThe study's strengths included a large sample size, multiple clinics, more than 500 providers from different specialties, rigorous follow-up determination, explicit redundancy criteria, various abnormal lab alert types, and an advanced integrated EMR used across VA facilities.
“many factors including a large sample size, multiple clinics, large number of providers (over 500 from different specialties), rigorous methods to determine follow-up, explicit criteria for determination of redundant tests, various types of abnormal lab alerts and an advanced integrated electronic medical record used in VA facilities throughout the US, all add several unique strengths to our study”
Quote in the paperNot a substantial claimMay combine two claims.
Study design and conductown dataThe study lacked comparable data from non-electronic medical record systems and therefore could not assess the effectiveness of automated notification relative to those systems.
“We also lack comparable data from non-electronic medical record based systems and cannot comment on the effectiveness of automated notification compared to these systems.”
Quote in the paperNot a substantial claimMay combine two claims.
Interpretationown dataAlerts for redundant tests appear to distract providers and may contribute to missed alerts for newly diagnosed conditions.
“notifications of abnormal redundant tests appear to be a distracting influence on providers who are missing essential alerts for newly diagnosed conditions.”
Quote in the paperSubstantial claimMay combine two claims.
PremisecitationOutpatient care is often busy and fragmented, making follow-up of abnormal laboratory test results prone to error.
“Outpatient care is often busy and fragmented, and therefore follow-up of abnormal laboratory test results is prone to error.”
Quote in the paperSubstantial claim0 of 10 sources checked- cites [1] Edelman, 2002, Outpatient diagnostic errors: unrecognized hyperglycemia, Eff Clin Pract
Not checked in the free run.
- cites [2] Hickner, 2008, Testing process errors and their harms and consequences reported from family medicine practices: a study of the American Academy of Family Physicians National Research Network, Qual Saf Health Care source
Not checked in the free run.
- cites [3] Hickner, 2005, Issues and initiatives in the testing process in primary care physician offices, Jt Comm J Qual Patient Saf
Not checked in the free run.
- cites [4] Poon, 2004, “I wish I had seen this test result earlier!”: dissatisfaction with test result management systems in primary care, Arch Intern Med source
Not checked in the free run.
- cites [5] Schiff, 2005, Missed hypothyroidism diagnosis uncovered by linking laboratory and pharmacy data, Arch Intern Med source
Not checked in the free run.
- cites [6] Wahls, 2007, The continuing problem of missed test results in an integrated health system with an advanced electronic medical record, Jt Comm J Qual Patient Saf
Not checked in the free run.
- cites [7] Wahls, 2007, Diagnostic errors and abnormal diagnostic tests lost to follow-up: a source of needless waste and delay to treatment, J Ambul Care Manage source
Not checked in the free run.
- cites [8] Bates, 2005, Doing better with critical test results, Jt Comm J Qual Patient Saf
Not checked in the free run.
- cites [9] Schiff, 2005, Introduction: communicating critical test results, Jt Comm J Qual Patient Saf
Not checked in the free run.
- cites [10] Nepple, 2008, Prevalence of delayed clinician response to elevated prostate-specific antigen values, Mayo Clin Proc source
Not checked in the free run.
Classified as probably not a substantial claim.
Study design and conductnone givenWhen no follow-up was documented, a second investigator reviewed the record and contacted the provider by telephone to seek additional evidence of undocumented follow-up or a decision not to pursue follow-up.
“a second study investigator evaluated the medical record to confirm the findings and contacted the provider by telephone to obtain any additional evidence of undocumented follow-up or a decision to not pursue follow-up.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductcitationObjective pre-defined explicit criteria were used to determine whether tests were redundant.
“We used objective pre-defined explicit criteria to determine redundancy of tests (criteria specified in Table 3).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [20] van Walraven, 1998, Do we know what inappropriate laboratory utilization is?, JAMA source
Not checked in the free run.
Classified as probably not a substantial claim.
Study design and conductown dataHierarchical multivariable logistic regression models accounted for clustering of laboratory tests by providers.
“Hierarchical multivariable logistic regression models accounting for clustering of laboratory tests by providers were used to identify factors associated with the outcome variables”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
Study design and conductown dataA reviewer blinded to acknowledgment status used a standardized pre-tested data collection instrument to assess follow-up actions in the medical record.
“a reviewer blinded to acknowledgement status evaluated the medical record using a standardized pre-tested data collection instrument to determine any follow-up actions”
Quote in the paperSubstantial claimMay combine two claims.
PremisecitationEarlier research found that ordering practitioners may fail to review abnormal diagnostic imaging alerts, and even when they review them, they may not act promptly on the results.
“In previous work, we found abnormal diagnostic imaging alerts may not always be reviewed by ordering practitioners, and practitioners who review them may not always act upon the transmitted results in a timely manner.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [19] Singh, 2009, Timely follow-up of abnormal diagnostic imaging test results in an outpatient setting: are electronic medical records achieving their potential?, Arch Intern Med source
Not checked in the free run.
The paper relies on this without support.
Follow-up of an abnormal result depends on providers reviewing and acting on the electronic communication of the result.
“Abnormal result follow-up, however, will occur only if electronic communication of test results (either through alerts or direct access of test result) is reviewed and acted upon by providers.”
The paper relies on this without support.
A substantial share of outpatient lab results are not urgent enough to require verbal reporting to the ordering clinician.
“many laboratory test results in the outpatient setting may not be immediately life threatening and hence not verbally reported to the ordering provider.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
A sample size of 30 participants, each reviewing 35 test results, was required to achieve 80% power to detect a statistically significant difference in cognitive workload and performance.
“On the basis of the anticipated rate of appropriately identified abnormal test results in the literature12,13,14,15,16,17,18,19,20,21 and the anticipated magnitude of the association of the enhanced EHR, we required a sample size of 30 participants, each reviewing 35 test results, to achieve 80% power to detect a statistically significant difference in cognitive workload and performance.”
From Association of the Usability of Electronic Health Records With Cognitive Workload and Performance Levels Among Physicians (2019) · cites this paper as [21]
The citation is attached to the anticipated rate in the literature, not to the citing authors’ sample-size and power calculation. The Results (page unavailable) report rates of acknowledgment and timely follow-up for abnormal-test alerts, but not a rate at which participants appropriately identified abnormal results while reviewing tests. The paper therefore provides related background rates, not the specific rate described. (LLM, conf=0.75)
Experimental — accuracy not yet measuredEven in the EHR environment, with alerts and reminders in place, physicians continue to often inappropriately manage abnormal test results.
“Even in the EHR environment, with alerts and reminders in place, physicians continue to often inappropriately manage abnormal test results.12,13,14,15,16,17,18,19,20,21”
From Association of the Usability of Electronic Health Records With Cognitive Workload and Performance Levels Among Physicians (2019) · cites this paper as [21]
The sociotechnical model accounts for complexities of technology, users, workflow, and larger external or organizational policies and context in assessment of EHR-related safety concerns.
“The model accounts for the complexities of technology, its users, the involved workflow, and the larger external or organizational policies and context in assessment of EHR-related safety concerns.35 36”
From An analysis of electronic health record-related patient safety concerns (2014) · cites this paper as [36]
Efforts to improve EHR-related patient safety depend on identifying underlying risks and appreciating contributing areas of vulnerability.
“Efforts to improve EHR-related patient safety rely on identification of underlying risks as well as an appreciation of contributing areas of vulnerability”
From Exploring the sociotechnical intersection of patient safety and electronic health record implementation (2014) · cites this paper as [23]
Cites
Work system design for patient safety: the SEIPS model
Carayon, P., Schoofs Hundt, A., Karsh, B.-T., Gurses, A. P., Alvarado, C. J., Smith, M., & Flatley Brennan, P. (2006) · Quality in Health Care · doi:10.1136/qshc.2005.015842 · Cited by 1,795 · see citing works
Diagnostic error in internal medicine
Graber, M. L., Franklin, N., & Gordon, R. (2005) · Archives of Internal Medicine · doi:10.1001/archinte.165.13.1493 · Cited by 1,549 · see citing works
A cost-benefit analysis of electronic medical records in primary care
Wang, S. J., Middleton, B., Prosser, L. A., Bardon, C. G., Spurr, C. D., Carchidi, P. J., Kittler, A. F., Goldszer, R. C., Fairchild, D. G., Sussman, A. J., et al. (2003) · The American Journal of Medicine · doi:10.1016/s0002-9343(03)00057-3 · Cited by 740 · see citing works
Missed and delayed diagnoses in the ambulatory setting: a study of closed malpractice claims
Gandhi, T. K., Kachalia, A., Thomas, E. J., Puopolo, A. L., Yoon, C., Brennan, T. A., & Studdert, D. M. (2006) · Annals of Internal Medicine · doi:10.7326/0003-4819-145-7-200610030-00006 · Cited by 662 · cited 1× · see citing works
“Arch Intern Med 2005;165 (13):1493–1499.”
Self-citationvan Walraven, C., & Naylor, C. D. (1998) · JAMA · doi:10.1001/jama.280.6.550 · Cited by 373 · see citing works
Poon, E. G., Gandhi, T. K., Sequist, T. D., Murff, H. J., Karson, A. S., & Bates, D. W. (2004) · Archives of Internal Medicine · doi:10.1001/archinte.164.20.2223 · Cited by 301 · cited 4× · see citing works
“J Am Med Inform Assoc 2007;14(4):459– 466.”
InfluentialBates, D. W., Kuperman, G. J., Rittenberg, E., Teich, J. M., Fiskio, J., Ma’luf, N., Onderdonk, A., Wybenga, D., Winkelman, J., Brennan, T. A., et al. (1999) · The American Journal of Medicine · doi:10.1016/s0002-9343(98)00410-0 · Cited by 292 · see citing works
Fumbled handoffs: one dropped ball after another
Gandhi, T. K. (2005) · Annals of Internal Medicine · doi:10.7326/0003-4819-142-5-200503010-00010 · Cited by 246 · see citing works
Singh, H., Thomas, E. J., Mani, S., Sittig, D., Arora, H., Espadas, D., Khan, M. M., & Petersen, L. A. (2009) · Archives of Internal Medicine · doi:10.1001/archinternmed.2009.263 · Cited by 212 · see citing works
Self-citationSingh, H., Naik, A. D., Rao, R., & Petersen, L. A. (2008) · Journal of General Internal Medicine · doi:10.1007/s11606-007-0393-z · Cited by 162 · see citing works
Self-citationHickner, J., Graham, D. G., Elder, N. C., Brandt, E., Emsermann, C. B., Dovey, S., & Phillips, R. (2008) · Quality and Safety in Health Care · doi:10.1136/qshc.2006.021915 · Cited by 158 · cited 4× · see citing works
“Jt Comm J Qual Patient Saf 2005;31(2): 63–65.”
InfluentialCommunication outcomes of critical imaging results in a computerized notification system
Singh, H., Arora, H. S., Vij, M. S., Rao, R., Khan, M. M., & Petersen, L. A. (2007) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2280 · Cited by 140 · see citing works
Self-citationTo err is not entirely human: Complex technology and user cognition
Horsky, J., Zhang, J., & Patel, V. L. (2005) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2005.05.002 · Cited by 138 · see citing works
Communication factors in the follow-up of abnormal mammograms
Poon, E. G., Haas, J. S., Puopolo, A. L., Gandhi, T. K., Burdick, E., Bates, D. W., & Brennan, T. A. (2004) · Journal of General Internal Medicine · doi:10.1111/j.1525-1497.2004.30357.x · Cited by 138 · see citing works
Population-based study of repeat laboratory testing.
van Walraven, C., & Raymond, M. (2003) · Clinical Chemistry · doi:10.1373/clinchem.2003.021220 · Cited by 137 · see citing works
Errors in cancer diagnosis: current understanding and future directions
Singh, H., Sethi, S., Raber, M., & Petersen, L. A. (2007) · Journal of Clinical Oncology · doi:10.1200/jco.2007.13.2142 · Cited by 121 · see citing works
Self-citationDesign and implementation of a comprehensive outpatient Results Manager
Poon, E. G., Wang, S. J., Gandhi, T. K., Bates, D. W., & Kuperman, G. J. (2003) · Journal of Biomedical Informatics · doi:10.1016/s1532-0464(03)00061-3 · Cited by 111 · see citing works
Issues and Initiatives in the Testing Process in Primary Care Physician Offices
Hickner, J. M., Fernald, D. H., Harris, D. M., Poon, E. G., Elder, N. C., & Mold, J. W. (2005) · The Joint Commission Journal on Quality and Patient Safety · doi:10.1016/s1553-7250(05)31012-9 · Cited by 74 · cited 1× · see citing works
“Qual Saf Health Care 2008;17(3):194– 200.”
Kushniruk, A. W., & Patel, V. L. (1998) · International Journal of Medical Informatics · doi:10.1016/s1386-5056(98)00106-3 · Cited by 70 · see citing works
Missed hypothyroidism diagnosis uncovered by linking laboratory and pharmacy data
Schiff, G. D. (2005) · Archives of Internal Medicine · doi:10.1001/archinte.165.5.574 · Cited by 47 · cited 1× · see citing works
“The American Journal of Medicine 2003;114(5):397– 403.”
Wahls, T. L., Haugen, T. H., & Cram, P. M. (2007) · The Joint Commission Journal on Quality and Patient Safety · doi:10.1016/s1553-7250(07)33052-3 · Cited by 46 · see citing works
Outpatient diagnostic errors: unrecognized hyperglycemia.
Edelman, D. E. (2002) · PubMed · Cited by 28 · see citing works
Editorial: Doing Better with Critical Test Results
Bates, D. W., & Leape, L. L. (2005) · The Joint Commission Journal on Quality and Patient Safety · doi:10.1016/s1553-7250(05)31010-5 · Cited by 25 · see citing works
Prevalence of delayed clinician response to elevated prostate-specific antigen values
Nepple, K. G., Joudi, F. N., Hillis, S. L., & Wahls, T. L. (2008) · Mayo Clinic Proceedings · doi:10.4065/83.4.439 · Cited by 22 · see citing works
Wahls, T. (2007) · Journal of Ambulatory Care Management · doi:10.1097/01.jac.0000290402.89284.a9 · Cited by 21 · see citing works
Introduction: Communicating Critical Test Results
Schiff, G. D. (2005) · The Joint Commission Journal on Quality and Patient Safety · doi:10.1016/s1553-7250(05)31009-9 · Cited by 19 · see citing works
Flagged first, then by citations
4 of the 30 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Sittig, D. F., & Singh, H. (2010) · Quality and Safety in Health Care · doi:10.1136/qshc.2010.042085 · Cited by 897 · see citing works
Self-citationFailure to Follow-Up Test Results for Ambulatory Patients: A Systematic Review
Callen, J. L., Westbrook, J. I., Georgiou, A., & Li, J. (2012) · Journal of General Internal Medicine · doi:10.1007/s11606-011-1949-5 · Cited by 229 · see citing works
“This contributes to the variability of responsibility and the diversity of work practices around each of the steps in the test management process.2,29,33,34,39,40 Singh et al.39 concluded that if policies and procedures were in place, combined with the use of an electronic alert notification of abnormal radiology, the…”
BackgroundResultInfluentialInformation Overload and Missed Test Results in Electronic Health Record–Based Settings
Singh, H., Spitzmueller, C., Petersen, N. J., Sawhney, M. K., & Sittig, D. F. (2013) · JAMA Internal Medicine · doi:10.1001/2013.jamainternmed.61 · Cited by 209 · see citing works
Self-citationAn analysis of electronic health record-related patient safety concerns
Meeks, D. W., Smith, M. W., Taylor, L., Sittig, D. F., Scott, J. M., & Singh, H. (2014) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2013-002578 · Cited by 207 · see citing works
Self-citationAdvancing the science of measurement of diagnostic errors in healthcare: the Safer Dx framework
Singh, H., & Sittig, D. F. (2015) · BMJ Quality & Safety · doi:10.1136/bmjqs-2014-003675 · Cited by 201 · see citing works
“52 In a different study, we found that the same technical intervention to improve test result follow-up in two different VA facilities led to improved outcomes in only one facility.”
BackgroundSelf-citationThe economics of patient safety
Slawomirski, L., & Klazinga, N. (2022) · OECD health working papers · doi:10.1787/761f2da8-en · Cited by 199 · see citing works
Current challenges in health information technology–related patient safety
Sittig, D. F., Wright, A. T., Coiera, E. W., Magrabi, F., Ratwani, R. M., Bates, D. W., & Singh, H. (2018) · Health Informatics Journal · doi:10.1177/1460458218814893 · Cited by 173 · see citing works
Self-citationDefining Health Information Technology–Related Errors
Sittig, D. F. (2011) · Archives of Internal Medicine · doi:10.1001/archinternmed.2011.327 · Cited by 167 · see citing works
Self-citationMurphy, D. R., Reis, B., Sittig, D. F., & Singh, H. (2012) · The American Journal of Medicine · doi:10.1016/j.amjmed.2011.07.029 · Cited by 142 · see citing works
“We previously found that about 7% of abnormal laboratory and imaging test result alerts were missed in EHR systems.(8,9) Information overload from too any alerts might be a potential reason, given that practiioners must sort through a large number of alerts daily.”
BackgroundSelf-citationMazur, L. M., Mosaly, P. R., Moore, C., & Marks, L. (2019) · JAMA Network Open · doi:10.1001/jamanetworkopen.2019.1709 · Cited by 127 · see citing works
System-related interventions to reduce diagnostic errors: a narrative review
Singh, H., Graber, M. L., KISSAM, S. M., Sorensen, A. V., Lenfestey, N. F., Tant, E. M., Henriksen, K., & LaBresh, K. A. (2011) · BMJ Quality & Safety · doi:10.1136/bmjqs-2011-000150 · Cited by 134 · see citing works
Self-citationDirect Release of Test Results to Patients Increases Patient Engagement and Utilization of Care
Pillemer, F. M., Price, R. A., Paone, S., Martich, G. D., Albert, S. M., Haidari, L. A., Updike, G. M., Rudin, R. S., Liu, D., & Mehrotra, A. (2016) · PLoS ONE · doi:10.1371/journal.pone.0154743 · Cited by 126 · see citing works
“[1] To reach this goal, meaningful use criteria provide specific objectives for EHRs, such as providing patients the ability to message providers electronically and to view their health information online.”
BackgroundElectronic Health Record-Based Surveillance of Diagnostic Errors in Primary Care
Singh, H., Giardina, T. D., Forjuoh, S. N., Reis, M. D., Kosmach, S., Khan, M. M., & Thomas, E. J. (2012) · BMJ Quality & Safety · doi:10.1136/bmjqs-2011-000304 · Cited by 131 · see citing works
Self-citationWhen diagnostic testing leads to harm: a new outcomes-based approach for laboratory medicine
Epner, P. L., Gans, J. E., & Graber, M. L. (2013) · BMJ Quality & Safety · doi:10.1136/bmjqs-2012-001621 · Cited by 126 · see citing works
Electronic health record-based triggers to detect potential delays in cancer diagnosis
Murphy, D. R., Laxmisan, A., Reis, B., Thomas, E. J., Esquivel, A., Forjuoh, S. N., Parikh, R. C., Khan, M. M., & Singh, H. (2013) · BMJ Quality & Safety · doi:10.1136/bmjqs-2013-001874 · Cited by 125 · see citing works
Self-citationThe Burden of Inbox Notifications in Commercial Electronic Health Records
Murphy, D. R., Meyer, A. N. D., Russo, E. M., Sittig, D. F., Wei, L., & Singh, H. (2016) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2016.0209 · Cited by 116 · see citing works
Self-citationThe Impact of Electronic Health Records on Diagnosis
Graber, M. L., Byrne, C., & Johnston, D. (2017) · Diagnosis · doi:10.1515/dx-2017-0012 · Cited by 108 · see citing works
“In one study, over 10% of critical alerts were never acknowledged within 30 days [53].”
BackgroundExploring situational awareness in diagnostic errors in primary care
Singh, H., Giardina, T. D., Petersen, L. A., Smith, M. W., Paul, L. W., Dismukes, K., Bhagwath, G., & Thomas, E. J. (2011) · BMJ Quality & Safety · doi:10.1136/bmjqs-2011-000310 · Cited by 103 · see citing works
Self-citationLegal, Ethical, and Financial Dilemmas in Electronic Health Record Adoption and Use
Sittig, D. F., & Singh, H. (2011) · Pediatrics · doi:10.1542/peds.2010-2184 · Cited by 100 · see citing works
Self-citationMurphy, D. R., Wu, L., Thomas, E. J., Forjuoh, S. N., Meyer, A. N. D., & Singh, H. (2015) · Journal of Clinical Oncology · doi:10.1200/jco.2015.61.1301 · Cited by 100 · see citing works
“These delays often result from a variety of cognitive and system factors that lead to lack of timely evaluation of red flag signs and symptoms or abnormal test results.(9-21) Despite increasing adoption of electronic health records (EHRs) and greater access to diagnostic information, delays persist(10,18,19,22-26) in…”
BackgroundSelf-citationMeeks, D. W., Takian, A., Sittig, D. F., Singh, H., & Barber, N. (2014) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2013-001762 · Cited by 91 · see citing works
Self-citationStrategies to reduce diagnostic errors: a systematic review
Abimanyi-Ochom, J., Mudiyanselage, S. B., Catchpool, M., Firipis, M., Dona, S. W. A., & Watts, J. J. (2019) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-019-0901-1 · Cited by 92 · see citing works
Singh, H., Spitzmueller, C., Petersen, N. J., Sawhney, M. K., Smith, M. W., Murphy, D. R., Espadas, D., Laxmisan, A., & Sittig, D. F. (2013) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2012-001267 · Cited by 92 · see citing works
Self-citationGiardina, T. D., King, B. J., Ignaczak, A. P., Paull, D. E., Hoeksema, L. J., Mills, P. D., Neily, J. B., Hemphill, R. R., & Singh, H. (2013) · Health Affairs · doi:10.1377/hlthaff.2013.0130 · Cited by 85 · see citing works
Self-citationEight Recommendations for Policies for Communicating Abnormal Test Results
Singh, H., & Vij, M. S. (2010) · The Joint Commission Journal on Quality and Patient Safety · doi:10.1016/s1553-7250(10)36037-5 · Cited by 83 · see citing works
Self-citationThe patient portal and abnormal test results: An exploratory study of patient experiences
Giardina, T. D., Modi, V., Parrish, D. E., et al. (2015) · Patient Experience Journal · doi:10.35680/2372-0247.1055 · Cited by 74 · see citing works
Self-citationHarmonization of critical result management in laboratory medicine
Campbell, C., & Horvath, A. R. (2013) · Clinica Chimica Acta · doi:10.1016/j.cca.2013.11.004 · Cited by 71 · see citing works
Understanding the management of electronic test result notifications in the outpatient setting
Hysong, S. J., Sawhney, M. K., Wilson, L., Sittig, D. F., Esquivel, A., Singh, S., & Singh, H. (2011) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-11-22 · Cited by 68 · see citing works
Self-citationMeasures to Improve Diagnostic Safety in Clinical Practice
Singh, H., Graber, M. L., & Hofer, T. P. (2016) · Journal of Patient Safety · doi:10.1097/pts.0000000000000338 · Cited by 66 · see citing works
“42 Measurement criteria are better defined for missed test results than other types of missed opportunities.(42,43)”
BackgroundSelf-citationSingh, H., Wilson, L., Petersen, L. A., Sawhney, M. K., Reis, B., Espadas, D., & Sittig, D. F. (2009) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-9-49 · Cited by 64 · see citing works
Self-citationFollow-up Actions on Electronic Referral Communication in a Multispecialty Outpatient Setting
Singh, H., Esquivel, A., Sittig, D. F., Murphy, D., Kadiyala, H., Schiesser, R., Espadas, D., & Petersen, L. A. (2011) · Journal of General Internal Medicine · doi:10.1007/s11606-010-1501-z · Cited by 57 · see citing works
Self-citationClinical quality indicators in laboratory medicine
Barth, J. H. (2011) · Annals of Clinical Biochemistry International Journal of Laboratory Medicine · doi:10.1258/acb.2011.011126 · Cited by 55 · see citing works
Reducing Delay in Diagnosis: Multistage Recommendation Tracking
Wandtke, B., & Gallagher, S. (2017) · American Journal of Roentgenology · doi:10.2214/ajr.17.18332 · Cited by 53 · see citing works
“A study of 1196 alerts found no statistically significant difference in the rate of proper follow-up between acknowledged and unacknowledged alerts for abnormal test results [14].”
BackgroundMurphy, D. R. (2012) · Archives of Internal Medicine · doi:10.1001/archinternmed.2011.740 · Cited by 52 · see citing works
Sarkar, U., Bonacum, D., Strull, W. M., Spitzmueller, C., Jin, N., López, A., Giardina, T. D., Meyer, A. N. D., & Singh, H. (2012) · BMJ Quality & Safety · doi:10.1136/bmjqs-2011-000541 · Cited by 53 · see citing works
Self-citationCommunication breakdowns and diagnostic errors: a radiology perspective
Murphy, D. R., Singh, H., & Berlin, L. (2014) · Diagnosis · doi:10.1515/dx-2014-0035 · Cited by 52 · see citing works
“It is estimated that approximately 4%–8% of electronically delivered abnormal test results fail to receive timely follow up [20, 32, 33].”
BackgroundSelf-citationCallen, J. L., Giardina, T. D., Singh, H., Li, L., Paoloni, R., Georgiou, A., Runciman, W. B., & Westbrook, J. I. (2015) · Journal of Medical Internet Research · doi:10.2196/jmir.3721 · Cited by 43 · see citing works
“More systematic processes for test result follow-up are needed, including the use of electronic information systems [13,17-20].”
BackgroundSelf-citationHow context affects electronic health record-based test result follow-up: a mixed-methods evaluation
Menon, S., Smith, M. W., Sittig, D. F., Petersen, N. J., Hysong, S. J., Espadas, D., Modi, V., & Singh, H. (2014) · BMJ Open · doi:10.1136/bmjopen-2014-005985 · Cited by 45 · see citing works
“Although EHRs appear to reduce the risk of missed test results,(2) 4 5 they do not eliminate the problem.(2) 3 6 Lack of timely follow-up of test results remains a major patient safety concern in most healthcare organisations.”
BackgroundSelf-citationBuilding a bridge to safe diagnosis in health care. The role of the clinical laboratory
Lippi, G., Plebani, M., & Graber, M. L. (2016) · Clinical Chemistry and Laboratory Medicine (CCLM) · doi:10.1515/cclm-2015-1135 · Cited by 43 · see citing works
Ten Strategies to Improve Management of Abnormal Test Result Alerts in the Electronic Health Record
Singh, H., Wilson, L., Reis, B., Sawhney, M. K., Espadas, D., & Sittig, D. F. (2010) · Journal of Patient Safety · doi:10.1097/pts.0b013e3181ddf652 · Cited by 43 · see citing works
Self-citationWorkarounds and Test Results Follow-up in Electronic Health Record-Based Primary Care
Menon, S., Murphy, D. R., Singh, H., Meyer, A. N. D., & Sittig, D. F. (2016) · Applied Clinical Informatics · doi:10.4338/aci-2015-10-ra-0135 · Cited by 40 · see citing works
“In our previous work we found that almost 8% of abnormal imaging results and 7% of laboratory test notifications sent to providers through EHRs lacked timely follow–up [9, 12].”
BackgroundSelf-citationPatients as diagnostic collaborators: sharing visit notes to promote accuracy and safety
Blease, C., & Bell, S. K. (2019) · Diagnosis · doi:10.1515/dx-2018-0106 · Cited by 34 · see citing works
Processing of discharge summaries in general practice: a retrospective record review
Spencer, R. A., Spencer, S. E. F., Rodgers, S., Campbell, S., & Avery, A. J. (2018) · British Journal of General Practice · doi:10.3399/bjgp18x697877 · Cited by 36 · see citing works
Georgiou, A., Li, J., Thomas, J., Dahm, M. R., & Westbrook, J. I. (2019) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocz032 · Cited by 40 · see citing works
“The authors noted that the procedures for management of results were not uniform, with only 55% of respondents reporting use of the electronic notification system for results management.38 Observational studies by Singh et al.32,34 and Bhise et al.57 reported a range of between 0.2% and 16.7% failure to follow up test…”
BackgroundInfluentialLaboratory testing in primary care: A systematic review of health IT impacts
Maillet, É., Paré, G., Currie, L. M., Raymond, L., de Guinea, A. O., Trudel, M.-C., & Marsan, J. (2018) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2018.05.009 · Cited by 30 · see citing works
Improving outpatient safety through effective electronic communication: a study protocol
Hysong, S. J., Sawhney, M. K., Wilson, L., Sittig, D. F., Esquivel, A., Watford, M., Giardina, T. D., Espadas, D., & Singh, H. (2009) · Implementation Science · doi:10.1186/1748-5908-4-62 · Cited by 34 · see citing works
Self-citationPowell, L., Sittig, D. F., Chrouser, K. L., & Singh, H. (2020) · JAMA Network Open · doi:10.1001/jamanetworkopen.2020.6752 · Cited by 34 · see citing works
Self-citationGeorgiou, A., Lymer, S. J., Forster, M., Strachan, M., Graham, S. M., HIRST, G. H. L., Callen, J. L., & Westbrook, J. I. (2014) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2013-002466 · Cited by 30 · see citing works
Murphy, D. R., Meyer, A. N. D., Vaghani, V., Russo, E. M., Sittig, D. F., Wei, L., Wu, L., & Singh, H. (2017) · Clinical Gastroenterology and Hepatology · doi:10.1016/j.cgh.2017.08.007 · Cited by 31 · see citing works
Self-citationGosselin, R. C., Adcock, D. M., Dorgalaleh, A., et al. (2019) · Seminars in Thrombosis and Hemostasis · doi:10.1055/s-0039-1697677 · Cited by 32 · see citing works
Electronic Health Record Use Issues and Diagnostic Error: A Scoping Review and Framework
Dixit, R. A., Boxley, C. L., Samuel, S., Mohan, V., Ratwani, R. M., & Gold, J. A. (2022) · Journal of Patient Safety · doi:10.1097/pts.0000000000001081 · Cited by 32 · see citing works
Smith, M., Murphy, D., Laxmisan, A., Sittig, D., Reis, B., Esquivel, A., & Singh, H. (2013) · Applied Clinical Informatics · doi:10.4338/aci-2013-04-ra-0019 · Cited by 31 · see citing works
“In order to address this in a multifaceted sociotechnical context, we intended the functional prototype design to meet the following goals, many of which were conceived based on previous research findings: [1, 3, 4, 27–29, 33–37] 1.”
BackgroundMethodSelf-citationAmroze, A., Field, T. S., Fouayzi, H., Sundaresan, D., Burns, L., Garber, L., Sadasivam, R. S., Mazor, K. M., Gurwitz, J. H., & Cutrona, S. L. (2019) · JMIR Medical Informatics · doi:10.2196/12650 · Cited by 25 · see citing works
“This is especially relevant for noninterruptive EHR alerts, where assessments of the effectiveness of these alerts have found mixed results [9-12].”
BackgroundSaw, S., Loh, T. P., Ang, S. B. L., Yip, J., & Sethi, S. K. (2011) · American Journal of Clinical Pathology · doi:10.1309/ajcpuz53xzwqfyis · Cited by 31 · see citing works
Electronic health record–related safety concerns: A cross‐sectional survey
Menon, S., Singh, H., Meyer, A. N. D., Belmont, E., & Sittig, D. F. (2014) · Journal of Healthcare Risk Management · doi:10.1002/jhrm.21146 · Cited by 29 · see citing works
Self-citationAn Automated Clinical Alert System for Newly-Diagnosed Atrial Fibrillation
Cook, D. A., Enders, F., Caraballo, P. J., Nishimura, R. A., & Lloyd, F. J. (2015) · PLoS ONE · doi:10.1371/journal.pone.0122153 · Cited by 27 · see citing works
“[1–5] Such clinical alerts have been used to identify potential drug-drug interactions or dose adjustments when prescribing[1] and to notify providers of critical test results[4, 6] or unexpected changes in clinical status.”
BackgroundEstablishing an Evidence Base for Critical Laboratory Value Thresholds
Doering, T. A., Plapp, F., & Crawford, J. M. (2014) · American Journal of Clinical Pathology · doi:10.1309/ajcpdi0fyz4unweq · Cited by 27 · see citing works
“3) 17 3 18-21 12,095 (11) 2,480 (8) 60.”
Giardina, T. D., Hunte, H. E. R., Hill, M. A., Heimlich, S. L., Singh, H., & Smith, K. M. (2022) · Journal of Patient Safety · doi:10.1097/pts.0000000000000999 · Cited by 32 · see citing works
Self-citationShneiderman, B. (2011) · interactions · doi:10.1145/2029976.2029992 · Cited by 27 · see citing works
Why Test Results Are Still Getting “Lost” to Follow-up: a Qualitative Study of Implementation Gaps
Zimolzak, A. J., Shahid, U., Giardina, T. D., Memon, S. A., Mushtaq, U., Zubkoff, L., Murphy, D. R., Bradford, A. M., & Singh, H. (2021) · Journal of General Internal Medicine · doi:10.1007/s11606-021-06772-y · Cited by 27 · see citing works
Self-citationLaxmisan, A., Sittig, D. F., Pietz, K., Espadas, D., Krishnan, B., & Singh, H. (2012) · Medical Care · doi:10.1097/mlr.0b013e31825f6619 · Cited by 26 · see citing works
Self-citationLaxmisan, A., McCoy, A. B., Wright, A. T., & Sittig, D. F. (2012) · Applied Clinical Informatics · doi:10.4338/aci-2011-11-ra-0066 · Cited by 26 · see citing works
Self-citationColicchio, T. K., Facelli, J. C., Fiol, G. D., et al. (2016) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2016.07.018 · Cited by 25 · see citing works
Asynchronous automated electronic laboratory result notifications: a systematic review
Slovis, B. H., NAHASS, T. A., Salmasian, H., Kuperman, G. J., & Vawdrey, D. K. (2017) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocx047 · Cited by 24 · see citing works
“Other studies have demonstrated a lack of improvement or a reduction in adverse events,(42) and still others have demonstrated continued safety concerns associated with automated notifications, even increased rates of adverse events.(44,46) Yet in many instances, noteworthy clinical improvements have been demonstrated.”
BackgroundCutrona, S. L., Fouayzi, H., Burns, L., Sadasivam, R. S., Mazor, K. M., Gurwitz, J. H., Garber, L., Sundaresan, D., Houston, T. K., & Field, T. S. (2017) · Journal of General Internal Medicine · doi:10.1007/s11606-017-4146-3 · Cited by 24 · see citing works
“new diagnosis.(8) The type of report(7) and the manner in which the alert appears in the InBasket are also influential.”
BackgroundMurphy, D. R., Satterly, T., Rogith, D., Sittig, D. F., & Singh, H. (2019) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2019.04.004 · Cited by 21 · see citing works
Self-citationDanforth, K. N., Hahn, E. E., Slezak, J. M., Chen, L. H., Li, B. H., Munoz‐Plaza, C. E., Luong, T. Q., Harrison, T. N., Mittman, B. S., Sim, J. J., et al. (2019) · American Journal of Kidney Diseases · doi:10.1053/j.ajkd.2019.05.003 · Cited by 24 · see citing works
Self-citationPerrotta, P. L., & Karcher, D. S. (2016) · Archives of Pathology & Laboratory Medicine · doi:10.5858/arpa.2015-0320-cp · Cited by 23 · see citing works
Sarkar, U., Simchowitz, B., Bonacum, D., Strull, W. M., López, A., Rotteau, L., & Shojania, K. G. (2014) · The Joint Commission Journal on Quality and Patient Safety · doi:10.1016/s1553-7250(14)40059-x · Cited by 20 · see citing works
Beyond the threshold: real-time use of evidence in practice
Jones, J. B., Stewart, W. F., Darer, J. D., & Sittig, D. F. (2013) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-13-47 · Cited by 22 · see citing works
“Observational data may also be valuable in providing guidance on what forms of CDS do and do not work [77].”
BackgroundSelf-citationAdoption of electronic health records in Saudi Arabia hospitals: Knowledge and usage
Alharbi, R. (2022) · Journal of King Saud University - Science · doi:10.1016/j.jksus.2022.102470 · Cited by 21 · see citing works
The burden of inbox-messaging systems and its effect on work-life balance in dermatology
Bittar, P. G., & Nicholas, M. W. (2017) · Journal of the American Academy of Dermatology · doi:10.1016/j.jaad.2017.12.026 · Cited by 20 · see citing works
“However, increasing use raises concerns for their potential to contribute to burnout.(1-3) While prior studies have assessed the burden these systems cause on primary care providers, little is known about the factors driving use or the impact on physician burnout, and scant data about use by specialists is available.”
BackgroundDalal, A. K., Schaffer, A. C., Gershanik, E., Papanna, R., Eibensteiner, K., Nolido, N. V., Yoon, C. S., Williams, D. H., Lipsitz, S. R., Roy, C. L., et al. (2018) · Journal of General Internal Medicine · doi:10.1007/s11606-018-4393-y · Cited by 23 · see citing works
Veinot, T. C., Senteio, C. R., Hanauer, D. A., & Lowery, J. C. (2017) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocx085 · Cited by 19 · see citing works
Barbieri, A. L., Fadare, O., Fan, L. L., Singh, H., & Parkash, V. (2018) · Journal of Pathology Informatics · doi:10.4103/jpi.jpi_70_17 · Cited by 19 · see citing works
Self-citationFarzandipour, M., Meidani, Z., Jabali, M. S., et al. (2018) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.13074 · Cited by 18 · see citing works
Duplicate patient records--implication for missed laboratory results.
Joffe, E., Bearden, C. F., Byrne, M. J., & Bernstam, E. V. (2012) · PubMed Central · Cited by 18 · see citing works
Implementation of Electronic Triggers to Identify Diagnostic Errors in Emergency Departments
Vaghani, V., Gupta, A., Mir, U., Wei, L., Murphy, D. R., Mushtaq, U., Sittig, D. F., Zimolzak, A. J., & Singh, H. (2024) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2024.6214 · Cited by 19 · see citing works
Self-citationDalal, A. K., Pesterev, B. M., Eibensteiner, K., Newmark, L. P., Samal, L., & Rothschild, J. M. (2015) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocv007 · Cited by 17 · see citing works
Electronic Detection of Delayed Test Result Follow-Up in Patients with Hypothyroidism
Meyer, A. N. D., Murphy, D. R., Al-Mutairi, A., Sittig, D. F., Wei, L., Russo, E. M., & Singh, H. (2017) · Journal of General Internal Medicine · doi:10.1007/s11606-017-3988-z · Cited by 14 · see citing works
“However, in our prior work on determining delayed follow-up of abnormal laboratory (including TSH) and imaging results, we found that most instances without documented follow-up were truly instances where no follow-up occurred, as confirmed with the providers.(12,13,15) Second, due to limited resources, our study did…”
BackgroundResultSelf-citationDo not assume that no news is good news: test result management and communication in primary care
Kwan, J. L., & Cram, P. M. (2015) · BMJ Quality & Safety · doi:10.1136/bmjqs-2015-004645 · Cited by 16 · see citing works
“11 Furthermore, most EHRs currently lack even rudimentary sophistication in test result interpretation, such as the ability to differentiate an elevated serum creatinine in a patient with previously normal renal function (an urgent/emergent result) from an elevated creatinine in a patient on…”
BackgroundSittig, D. F., Salimi, M., Aiyagari, R. M., Banas, C. A., Clay, B. J., Gibson, K., Goel, A., Hines, R. B., Longhurst, C., Mishra, V. K., et al. (2018) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocy033 · Cited by 16 · see citing works
Self-citationEffects of laboratory data exchange in the care of patients with HIV
Bell, D. S., Cima, L., Seiden, D. S., Nakazono, T. T., Alcouloumre, M. S., & Cunningham, W. E. (2012) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2012.07.012 · Cited by 11 · see citing works
“Recent efforts to decompose the tasks of laboratory result management represent an important step toward the broader goal of improving EHR support for these tasks [3, 23] These efforts will need to be paired with ongoing evaluation and monitoring of delays in test result follow-up [24].”
BackgroundTarkan, S., Catherine Plaisant, B., Shneiderman, B., & Hettinger, A. Z. (2011) · PubMed Central · Cited by 14 · see citing works
Application of Human Factors Methods to Understand Missed Follow-up of Abnormal Test Results
Rogith, D., Satterly, T., Singh, H., Sittig, D. F., Russo, E. M., Smith, M. W., Roosan, D., Bhise, V., & Murphy, D. R. (2020) · Applied Clinical Informatics · doi:10.1055/s-0040-1716537 · Cited by 13 · see citing works
Self-citationImproving patient safety via automated laboratory-based adverse event grading
Niland, J. C., Stiller, T., Neat, J., Londrc, A., Johnson, D., & Pannoni, S. (2011) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000513 · Cited by 13 · see citing works
“Deployment will require an appropriate workflow in clinic, and avoidance of ‘alert fatigue’ among caregivers.(25) 26 Adding a configurable rules engine interface to incorporate protocol-specific rules to ‘fine tune’ the algorithms will provide additional efficiency in future.”
BackgroundBernard, R., Benhamou, D., & Beloeil, H. (2010) · Annales Françaises d Anesthésie et de Réanimation · doi:10.1016/j.annfar.2010.09.010 · Cited by 12 · see citing works
Were My Diagnosis and Treatment Correct? No News is Not Necessarily Good News
Singh, H., & Sittig, D. F. (2014) · Journal of General Internal Medicine · doi:10.1007/s11606-014-2890-1 · Cited by 12 · see citing works
Self-citationEvidence that electronic health records can promote physician counseling for healthy behaviors
Bae, J., Hockenberry, J. M., Rask, K. J., & Becker, E. R. (2016) · Health Care Management Review · doi:10.1097/hmr.0000000000000108 · Cited by 12 · see citing works
Lau, B., Overby, C. L., Wirtz, H. S., & Devine, E. B. (2013) · Applied Clinical Informatics · doi:10.4338/aci-2013-06-ra-0041 · Cited by 13 · see citing works
A Sociotechnical Approach to Electronic Health Record Related Safety
Sittig, D. F., & Singh, H. (2017) · Elsevier eBooks · doi:10.1016/b978-0-12-809523-2.00014-5 · Cited by 11 · see citing works
Self-citationMazur, L. M., Marks, L. B., McLeod, R., Karwowski, W., Mosaly, P. R., Tracton, G. S., Adams, R., Hoyle, L., Das, S. K., & Chera, B. S. (2018) · Advances in Radiation Oncology · doi:10.1016/j.adro.2018.01.002 · Cited by 11 · see citing works
“(and unnecessary) cognitive effort and confusion on health care providers, which increases the potential for patient harm.(6-10) Indeed, there are reports that link patient harm to the suboptimal design and use of health IT,(11-15) including in radiation oncology.”
BackgroundStelling, B. E. V., Halvorsen, A. J., Dupras, D. M., Kearns, L., Kisielewski, M., Martin, S. K., Uthlaut, B. S., & Leasure, E. L. (2023) · Journal of Graduate Medical Education · doi:10.4300/jgme-d-23-00165.1 · Cited by 10 · see citing works
Thomas, J., Dahm, M. R., Li, J., Smith, P., Irvine, J., Westbrook, J. I., & Georgiou, A. (2020) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaa093 · Cited by 12 · see citing works
Are Amended Surgical Pathology Reports Getting to the Correct Responsible Care Provider?
Parkash, V., Domfeh, A. B., Cohen, P., et al. (2014) · American Journal of Clinical Pathology · doi:10.1309/ajcpy55evinjclgx · Cited by 9 · see citing works
Li, J., Paoloni, R., Li, L., Callen, J. L., Westbrook, J. I., Runciman, W. B., & Georgiou, A. (2020) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-020-01135-9 · Cited by 9 · see citing works
Failure to review STAT clinical laboratory requests and its economical impact
Borja, E. R., Villalba-Martinez, C., Barba-Serrano, E., et al. (2016) · Biochemia Medica · doi:10.11613/bm.2016.005 · Cited by 7 · see citing works
“In addition to this, new information technology need to be developed in order to reduce errors in follow up requests and improve patient safety, given the volume of information that clinicians both generate and review (17,18).”
BackgroundContributing Factors for Pediatric Ambulatory Diagnostic Process Errors: Project RedDE
Dadlez, N. M., Adelman, J. S., Bundy, D. G., Singh, H., Applebaum, J. R., & Rinke, M. L. (2020) · Pediatric Quality and Safety · doi:10.1097/pq9.0000000000000299 · Cited by 7 · see citing works
Self-citationQian, Y., Ni, Z., Gui, W., & Liu, Y. (2021) · International Journal of Computational Intelligence Systems · doi:10.2991/ijcis.d.210203.006 · Cited by 7 · see citing works
“[37, 38] and Sittig and Singh [39] have made important contributions to patient safety.”
BackgroundMefford, M. T., Zhou, M. M., Zhou, H. C., Derakhshan, H., Harrison, T. N., Zia, M., Kanter, M. H., Scott, R. D., Imley, T. M., Sanders, M. A., et al. (2023) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2023.04.009 · Cited by 6 · see citing works
- Authors
- Singh, H., Thomas, E. J., Sittig, D. F., Wilson, L., Espadas, D., Khan, M. M., & Petersen, L. A.
- Affiliations
- Michael E. DeBakey VA Medical Center (US)
- Memorial Hermann (US)
- Corresponding author
- Singh, Hardeep
- Venue
- The American Journal of Medicine
- Publisher
- Elsevier BV
- Published
- 2010-03
- Volume
- 123
- Issue
- 3
- Pages
- 238-244
- ISSN
- 0002-9343 (ISSN-L)
- PMC
- PMC2878665
- OpenAlex
- W1981106602
- Semantic Scholar
- 6cb4dbf1bbfb77ebcb84cea2013870878498f97c
- Cited by
- 160
- References
- 30
- Citations by year
- 2026: 3 · 2025: 4 · 2024: 2 · 2023: 4 · 2022: 9 · 2021: 6 · 2020: 10 · 2019: 13 · 2018: 12 · 2017: 14 · 2016: 11 · 2015: 12 · 2014: 11 · 2013: 16 · 2012: 10
- FWCI
- 28.08
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- The American Journal of Medicine (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- Digital Commons-TMC (Texas Medical Center) (submittedVersion)
- Funders
- National Institutes of Health (K23CA125585)
- VA National Center for Patient Safety (R18 HS17244-02)
- Topics
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Health Sciences › Medicine › Physiology › Clinical Laboratory Practices and Quality Control
- Health Sciences › Health Professions › Emergency Medical Services › Patient Safety and Medication Errors
- Keywords
- laboratory abnormalities, electronic health records, patient safety, HbA1c, anti-HCV antibodies, prostate-specific antigen, thyroid-stimulating hormone, outpatient care
- MeSH terms
- Communication, Diagnostic Errors, Female, Follow-Up Studies, Hepatitis C, Humans, Male, Practice Patterns, Physicians', Primary Health Care, Retrospective Studies, United States, Medical Records Systems, Computerized
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-07.
Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-07.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
