Timely Follow-up of Abnormal Diagnostic Imaging Test Results in an Outpatient Setting: Are Electronic Medical Records Achieving Their Potential?
Open accessAbstract
BACKGROUND: Given the fragmentation of outpatient care, timely follow-up of abnormal diagnostic imaging results remains a challenge. We hypothesized that an electronic medical record (EMR) that facilitates the transmission and availability of critical imaging results through either automated notification (alerting) or direct access to the primary report would eliminate this problem. METHODS: We studied critical imaging alert notifications in the outpatient setting of a tertiary care Department of Veterans Affairs facility from November 2007 to June 2008. Tracking software determined whether the alert was acknowledged (ie, health care practitioner/provider [HCP] opened the message for viewing) within 2 weeks of transmission; acknowledged alerts were considered read. We reviewed medical records and contacted HCPs to determine timely follow-up actions (eg, ordering a follow-up test or consultation) within 4 weeks of transmission. Multivariable logistic regression models accounting for clustering effect by HCPs analyzed predictors for 2 outcomes: lack of acknowledgment and lack of timely follow-up. RESULTS: Of 123 638 studies (including radiographs, computed tomographic scans, ultrasonograms, magnetic resonance images, and mammograms), 1196 images (0.97%) generated alerts; 217 (18.1%) of these were unacknowledged. Alerts had a higher risk of being unacknowledged when the ordering HCPs were trainees (odds ratio [OR], 5.58; 95% confidence interval [CI], 2.86-10.89) and when dual-alert (>1 HCP alerted) as opposed to single-alert communication was used (OR, 2.02; 95% CI, 1.22-3.36). Timely follow-up was lacking in 92 (7.7% of all alerts) and was similar for acknowledged and unacknowledged alerts (7.3% vs 9.7%; P = .22). Risk for lack of timely follow-up was higher with dual-alert communication (OR, 1.99; 95% CI, 1.06-3.48) but lower when additional verbal communication was used by the radiologist (OR, 0.12; 95% CI, 0.04-0.38). Nearly all abnormal results lacking timely follow-up at 4 weeks were eventually found to have measurable clinical impact in terms of further diagnostic testing or treatment. CONCLUSIONS: Critical imaging results may not receive timely follow-up actions even when HCPs receive and read results in an advanced, integrated electronic medical record system. A multidisciplinary approach is needed to improve patient safety in this area.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
0 of 27 cited sources checked · 0 supported · 0 flagged
- Findingown data
Among the 1196 alerts, 73 (6.1%) had both verbal communication and subsequent admission.
“For 206 (17.2%) alerts, the radiologist also communicated verbally with a provider; in 172 (14.4%) the patient was subsequently admitted after alert transmission; and in 73 (6.1%) there was evidence of both.”
Quote in the paperNot a substantial claim - Findingown data
Hospitalization within 2 weeks of alert transmission was not significantly associated with acknowledgment.
“Additional verbal communication by the radiologist and hospitalization of the patient within 2-weeks of alert transmission was not significantly associated with acknowledgement.”
Quote in the paperNot a substantial claim - Findingown data
Half (50.0%) of the tests lacking timely follow-up were abnormal chest x-rays.
“Half (50.0%) of the tests lacking timely follow-up were abnormal chest x-rays.”
Quote in the paperNot a substantial claim - Findingown data
Cancer was determined to be a new diagnosis in 11 of those cases.
“Cancer was determined to be a new diagnosis in 11 of these cases.”
Quote in the paperNot a substantial claim - Findingown data
Between November 2007 and June 2008, 123,638 outpatient studies were performed, of which 1196 (0.97%) generated alerts.
“Between November 2007 and June 2008, 123,638 outpatient studies (including X-rays, CT scans, ultrasounds, MRI and mammography) were performed, of which 1196 (0.97%) images generated alerts.”
Quote in the paperNot a substantial claim
- Findingown data
In 5 of the 92 cases lacking timely follow-up, outcomes could not be determined because the patient did not return to the institution.
“Although in 5 cases we could not determine outcomes because the patient did not return to the institution”
Quote in the paperNot a substantial claim - Findingown data
In 46 (50%) of the 92 alerts lacking timely follow-up, providers executed appropriate action within a week after the initial call.
“We found that in 46 (50%) cases providers had executed the appropriate action.”
Quote in the paperNot a substantial claim - Findingown data
Among the 1196 alerts, 206 (17.2%) had additional verbal communication by the radiologist.
“For 206 (17.2%) alerts, the radiologist also communicated verbally with a provider; in 172 (14.4%) the patient was subsequently admitted after alert transmission; and in 73 (6.1%) there was evidence of both.”
Quote in the paperNot a substantial claim - Findingown data
In the 92 cases lacking timely follow-up, more than a fourth (n=26; 28%) led to a new diagnosis being established.
“In these 92 cases, more than a fourth (n=26; 28%) led to a new diagnosis being established.”
Quote in the paperSubstantial claim - Findingown data
Among the 1196 alerts, 172 (14.4%) had subsequent patient admission after alert transmission.
“For 206 (17.2%) alerts, the radiologist also communicated verbally with a provider; in 172 (14.4%) the patient was subsequently admitted after alert transmission; and in 73 (6.1%) there was evidence of both.”
Quote in the paperNot a substantial claim - Findingown data
Additional verbal communication was significantly more likely to be associated with timely follow-up (p<0.0001).
“Additional verbal communication and instances where the patient was hospitalized within 2 weeks of alert transmission were significantly more likely to be associated with timely follow-up (p< 0.0001).”
Quote in the paperSubstantial claim - Findingown data
Of the 1196 alerts, 979 (81.9%) were acknowledged.
“Nine hundred seventy nine alerts (81.9%) were tracked as acknowledged and 217 (18.1%) were tracked as unacknowledged.”
Quote in the paperSubstantial claim - Findingown data
There was no significant difference in rates of lack of timely follow-up between acknowledged and unacknowledged alerts (7.3% vs 9.7%; p=0.2).
“There was no significant difference in rates of lack of timely follow-up between the acknowledged and unacknowledged alerts (7.3% vs. 9.7%; p=0.2).”
Quote in the paperNot a substantial claim - Findingown data
Additional verbal communication by the radiologist was not significantly associated with acknowledgment.
“Additional verbal communication by the radiologist and hospitalization of the patient within 2-weeks of alert transmission was not significantly associated with acknowledgement.”
Quote in the paperNot a substantial claim - Findingown data
Chest imaging showing a non-specific density was more likely to be associated with lack of timely follow-up (p=0.04).
“Chest imaging showing a non-specific density was more likely to be associated with lack of timely follow-up (p=0.04).”
Quote in the paperNot a substantial claim - Findingown data
In the multivariable model, additional verbal communication was associated with lower odds of lack of timely follow-up (OR 0.12; 95% CI 0.04-0.38).
“In a logistic regression model for lack of timely follow-up, the following variables were significant predictors (Table 3): additional verbal communication (OR: 0.12; 95%CI: 0.04-0.38), a subsequent hospital admission after alert transmission (OR: 0.22; 95%CI: 0.08-0.38), alerts of CT scans (OR: 0.49; 95%CI: 0.29-0.85) and when dual as opposed to single communication was used (OR: 1.99; 95%CI: 1.”
Quote in the paperSubstantial claim - Findingown data
In the multivariable model, trainees as ordering providers had higher odds of lack of acknowledgment compared to attending physicians (OR 5.58; 95% CI 2.86-10.89).
“In a logistic regression model for lack of acknowledgment, the following variables were significant (data not shown in tables): physician assistants as ordering providers as compared to attending physicians (OR: 0.46; 95%CI: 0.22-0.98), trainees as ordering providers (OR: 5.58; 95%CI: 2.86-10.89), and when dual as opposed to single communication was used (OR: 2.02; 95%CI: 1.22-3.36).”
Quote in the paperSubstantial claim - Findingown data
Trainees were less likely to acknowledge alerts compared to other provider types (p<0.0001).
“For instance, trainees and surgical specialties were less likely to acknowledge alerts compared to other types and specialties of ordering providers (p<0.0001 for both).”
Quote in the paperNot a substantial claim - Findingown data
In the multivariable model, CT scan alerts were associated with lower odds of lack of timely follow-up (OR 0.49; 95% CI 0.29-0.85).
“In a logistic regression model for lack of timely follow-up, the following variables were significant predictors (Table 3): additional verbal communication (OR: 0.12; 95%CI: 0.04-0.38), a subsequent hospital admission after alert transmission (OR: 0.22; 95%CI: 0.08-0.38), alerts of CT scans (OR: 0.49; 95%CI: 0.29-0.85) and when dual as opposed to single communication was used (OR: 1.99; 95%CI: 1.”
Quote in the paperSubstantial claim - Findingown data
Seven providers who were called volunteered that someone other than them was responsible for follow-up.
“seven providers we called volunteered that someone else other than them (e.g. the PCP) was the provider responsible for follow-up.”
Quote in the paperNot a substantial claim - Findingown data
In the multivariable model, dual communication was associated with higher odds of lack of timely follow-up (OR 1.99; 95% CI 1.06-3.48).
“In a logistic regression model for lack of timely follow-up, the following variables were significant predictors (Table 3): additional verbal communication (OR: 0.12; 95%CI: 0.04-0.38), a subsequent hospital admission after alert transmission (OR: 0.22; 95%CI: 0.08-0.38), alerts of CT scans (OR: 0.49; 95%CI: 0.29-0.85) and when dual as opposed to single communication was used (OR: 1.99; 95%CI: 1.”
Quote in the paperSubstantial claim - Findingown data
Overall, 92 alerts (7.7%) lacked timely follow-up at 4 weeks.
“Overall, 92 alerts (7.7%) lacked timely follow-up at 4-weeks; thus the estimated rate of lack of timely follow-up was 0.07% of all diagnostic imaging tests performed.”
Quote in the paperSubstantial claim - Findingown data
In 20 (1.7%) alerts, calling the provider was deemed unnecessary or without impact because the diagnosis was not new, the patient was already receiving appropriate care, or the patient had died.
“In 20 (1.7%) of these cases, we determined calling the provider would be unnecessary or make no impact on outcome either because the diagnosis was not new, the patient was already receiving appropriate care for the condition, or the patient had died.”
Quote in the paperNot a substantial claim - Findingown data
Of the 1196 alerts, 217 (18.1%) were unacknowledged.
“Nine hundred seventy nine alerts (81.9%) were tracked as acknowledged and 217 (18.1%) were tracked as unacknowledged.”
Quote in the paperSubstantial claim - Findingown data
Hospitalization within 2 weeks of alert transmission was significantly more likely to be associated with timely follow-up (p<0.0001).
“Additional verbal communication and instances where the patient was hospitalized within 2 weeks of alert transmission were significantly more likely to be associated with timely follow-up (p< 0.0001).”
Quote in the paperSubstantial claim - Findingown data
Surgical specialties were less likely to acknowledge alerts compared to other specialties (p<0.0001).
“For instance, trainees and surgical specialties were less likely to acknowledge alerts compared to other types and specialties of ordering providers (p<0.0001 for both).”
Quote in the paperNot a substantial claim - Findingown data
All other tests among the 92 lacking timely follow-up had a measurable clinical impact in terms of diagnostic testing or treatment.
“all other tests had a measurable clinical impact in terms of either diagnostic testing or treatment.”
Quote in the paperSubstantial claim - Findingown data
In the multivariable model, subsequent hospital admission after alert transmission was associated with lower odds of lack of timely follow-up (OR 0.22; 95% CI 0.08-0.38).
“In a logistic regression model for lack of timely follow-up, the following variables were significant predictors (Table 3): additional verbal communication (OR: 0.12; 95%CI: 0.04-0.38), a subsequent hospital admission after alert transmission (OR: 0.22; 95%CI: 0.08-0.38), alerts of CT scans (OR: 0.49; 95%CI: 0.29-0.85) and when dual as opposed to single communication was used (OR: 1.99; 95%CI: 1.”
Quote in the paperSubstantial claim - Findingown data
No statistically significant differences in provider type or specialty were seen for timely follow-up.
“No statistically significant differences in provider type or specialty was seen”
Quote in the paperNot a substantial claim - Findingown data
The majority of near-misses lacking timely follow-up (62 of 92; 67.4%) were related to a suspected new malignancy.
“the majority of these (62 of 92; 67.4%) were related to some form of a suspected new malignancy.”
Quote in the paperNot a substantial claim - Findingown data
In the multivariable model, dual communication as opposed to single communication was associated with higher odds of lack of acknowledgment (OR 2.02; 95% CI 1.22-3.36).
“In a logistic regression model for lack of acknowledgment, the following variables were significant (data not shown in tables): physician assistants as ordering providers as compared to attending physicians (OR: 0.46; 95%CI: 0.22-0.98), trainees as ordering providers (OR: 5.58; 95%CI: 2.86-10.89), and when dual as opposed to single communication was used (OR: 2.02; 95%CI: 1.22-3.36).”
Quote in the paperNot a substantial claim - Findingown data
No evidence of documented follow-up action was found in 131 (11%) alerts.
“No evidence of documented follow-up action was found in 131 (11%) of the alerts.”
Quote in the paperSubstantial claim - Findingreasoning
The estimated rate of lack of timely follow-up was 0.07% of all diagnostic imaging tests performed.
“Overall, 92 alerts (7.7%) lacked timely follow-up at 4-weeks; thus the estimated rate of lack of timely follow-up was 0.07% of all diagnostic imaging tests performed.”
Quote in the paperSubstantial claim - Findingown data
In the multivariable model, physician assistants as ordering providers had lower odds of lack of acknowledgment compared to attending physicians (OR 0.46; 95% CI 0.22-0.98).
“In a logistic regression model for lack of acknowledgment, the following variables were significant (data not shown in tables): physician assistants as ordering providers as compared to attending physicians (OR: 0.46; 95%CI: 0.22-0.98), trainees as ordering providers (OR: 5.58; 95%CI: 2.86-10.89), and when dual as opposed to single communication was used (OR: 2.02; 95%CI: 1.22-3.36).”
Quote in the paperNot a substantial claim - Findingown data
X-rays were more likely to be acknowledged than other imaging tests (p=0.0005).
“X-rays were more likely to be acknowledged (p=0.0005) than other imaging tests.”
Quote in the paperNot a substantial claim - Findingown data
CT scans and MRIs were more likely to be associated with timely follow-up (p=0.002).
“but certain tests (CT scans and MRIs) were more likely associated with timely follow-up (p= 0.002).”
Quote in the paperNot a substantial claim - Premisecitation
The volume and high-risk transitions in outpatient care make timely communication particularly challenging.
“The volume of outpatient care and nature of high-risk transitions between practitioners, settings and systems of care makes timely communication particularly challenging. 13”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [13] Singh H Naik A Rao R Petersen L Reducing Diagnostic Errors Through Effective Communication: Harnessing the Power of Information Technology. Journal of General Internal Medicine April 2008 23 4 489 94 18373151 PMC2359508
Not checked in the free run.
- Interpretationown data
An EMR that facilitates transmission and availability of critical imaging results through automated notification or direct access does not eliminate missed test results even when one or more providers read the results.
“Our findings suggest that an EMR that facilitates transmission and availability of critical imaging results to the provider through either automated notification or direct access of primary report does not eliminate the problem of missed test results even when one or more providers read the results.”
- Also stated in: § Conclusions · hedged
“Critical imaging results may not receive timely follow-up actions even when providers receive and read results in an advanced, integrated EMR system.”
Quote in the paperSubstantial claim - Also stated in: § Conclusions · hedged
- Study design and conductreasoning
Lack of a comparison group was a limitation of the study.
“Because sophisticated tracking such as the one we used is not easily possible in the non-EMR environment and similar data from the pre-EMR era in the VA does not exist, lack of a comparison group was a limitation.”
Quote in the paperNot a substantial claim - Interpretationown data
Preliminary evidence from the study suggests providers may believe responsibility for follow-up belonged to someone else.
“Preliminary evidence from our study suggests providers may believe responsibility for follow-up belonged to someone else.”
Quote in the paperSubstantial claim - Premisecitation
In previous work, about 4% of critical imaging results lacked follow-up, based only on unacknowledged alerts.
“In our previous work, we calculated this to be about 4% of critical imaging results, but this figure was based only on unacknowledged alerts.14”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [14] Singh H Arora HS Vij MS Rao R Khan M Petersen LA Communication outcomes of critical imaging results in a computerized notification system. J Am Med Inform Assoc 2007 14 4 459 66 17460135 PMC2244901
Not checked in the free run.
- Interpretationown data
The study's findings suggest providers may perceive a lack of urgency for test results with less immediate consequential implications.
“However, our findings suggest that providers may perceive a lack of urgency for test results that have less immediate consequential implications (e.g. suspicious for a new cancer).”
Quote in the paperSubstantial claim - Interpretationnone given
All near-misses had potential to lead to missed or delayed diagnosis had the study not intervened.
“All near-misses had potential to lead to missed or delayed diagnosis had we not intervened”
Quote in the paperNot a substantial claim - Premisecitation
Providers often receive several hundred normal diagnostic test results and many abnormal alerts weekly.
“providers often receive several hundred normal diagnostic test results and many abnormal alerts every week, suggesting that a similar problem could exist here.24”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [24] Poon EG Wang SJ Gandhi TK Bates DW Kuperman GJ Design and implementation of a comprehensive outpatient Results Manager. J Biomed Inform February 2003 36 1-2 80 91 14552849
Not checked in the free run.
- Study design and conductnone given
The study did not account for alert transmission failures, such as when a radiologist inadvertently did not code a report as abnormal.
“In addition, we did not account for any problems with alert transmission failures such as situations when the radiologist inadvertently did not code the report as an abnormality.”
Quote in the paperNot a substantial claim - Interpretationreasoning
The findings may not be generalizable outside the VA setting, especially to free-standing clinics without integrated EMR systems.
“Because of the study population and site our findings may not be generalizable outside the VA setting; especially to free standing clinics that do not use integrated EMR systems.”
Quote in the paperNot a substantial claim
- May combine two claims. 11
- Classified as probably not a substantial claim. 10
May combine two claims.
Findingown dataTypes and specialties of ordering providers and types of imaging tests differed significantly between acknowledged and unacknowledged alerts.
“Types and specialties of ordering providers and types of imaging tests were significantly different across the two groups.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataOf 74 alerts unacknowledged by trainees that had a documented response, the ordering trainee directly responded without opening the alert in 34 cases, 9 of which were prompted by direct radiologist notification.
“Of 74 alerts unacknowledged by trainees that were followed by a documented response in the medical record at the time of review, the ordering trainee directly responded to the test result (without opening the alert) in 34 cases (9 of which were prompted by a direct notification by the radiologist).”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataIn the remaining 40 of those 74 cases, another provider documented a response, 4 of which were prompted by direct radiologist notification.
“In the remaining 40 cases, another provider documented a response to the report (4 of which were prompted by a direct notification by the radiologist).”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
PremisecitationPrevious studies raise concern that providers may not read all their alerts.
“Previous studies raise concern that providers may not read all their alerts.14, 16-19”
Quote in the paperSubstantial claim0 of 5 sources checked- cites [14] Singh H Arora HS Vij MS Rao R Khan M Petersen LA Communication outcomes of critical imaging results in a computerized notification system. J Am Med Inform Assoc 2007 14 4 459 66 17460135 PMC2244901
Not checked in the free run.
- cites [16] Choksi VR Marn CS Bell Y Carlos R Efficiency of a semiautomated coding and review process for notification of critical findings in diagnostic imaging. AJR Am J Roentgenol April 2006 186 4 933 6 16554559
Not checked in the free run.
- cites [17] Wahls TL Haugen T Cram P Diagnostic errors associated with failure to follow up on abnormal results. Joint Commission Journal on Quality and Patient Safety/Joint Commission Resources 2007
Not checked in the free run.
- cites [18] Wahls TL Cram PM The frequency of missed test results and associated treatment delays in a highly computerized health system. BMC Fam Pract 2007 8 32 17519017 PMC1891295
Not checked in the free run.
- cites [19] Kuperman GJ Teich JM Bates DW Detecting alerts, notifying the physician, and offering action items: a comprehensive alerting system. Proc AMIA Annu Fall Symp 1996 704 8 8947756 PMC2233134
Not checked in the free run.
May combine two claims.
PremisecitationElectronic alerts can facilitate transmission and potentially response and follow-up by an ordering practitioner, an advantage over paper-based reporting.
“Electronic communication using alerts (computerized notifications of critical information such as abnormal diagnostic test results) can facilitate transmission and potentially, a response and follow-up action by an ordering practitioner – an advantage over paper based reporting.14”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [14] Singh H Arora HS Vij MS Rao R Khan M Petersen LA Communication outcomes of critical imaging results in a computerized notification system. J Am Med Inform Assoc 2007 14 4 459 66 17460135 PMC2244901
Not checked in the free run.
Probably not a substantial claim: may not pass S2 (checkable).
PremisecitationEffective communication requires a response from the recipient, such as follow-up action and acknowledgment, beyond information transfer.
“However, effective communication involves more than just information transfer; it requires a response from the recipient, such as taking a follow-up action and acknowledging receipt of the information to the sender.15”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [15] Ash JS Berg M Coiera E Some unintended consequences of information technology in health care: the nature of patient care information system-related errors. J Am Med Inform Assoc March 2004 11 2 104 12 14633936 PMC353015
Not checked in the free run.
May combine two claims.
PremisecitationCommunication breakdown is a preventable factor in adverse events and contributes significantly to outpatient diagnostic errors caused by lack of follow-up of abnormal test results.
“Communication breakdown is consistently identified as a preventable factor in studies of adverse events1-6 and a significant contributor to outpatient diagnostic errors from lack of follow-up of abnormal test results.7-12”
Quote in the paperSubstantial claim0 of 10 sources checked- cites [1] Joint Commission on Accreditation of Healthcare Organizations Sentinel event statistics, March 31, 2006. Joint Commission on Accreditation of Healthcare Organizations 2005 Available at: URL: http://www.jointcommission.org/NR/rdonlyres/FA465646-5F5F-4543-AC8F-E8AF6571E372/0/root_cause_se.jpg 19343995
Not checked in the free run.
- cites [2] Arora V Johnson J Lovinger D Humphrey HJ Meltzer DO Communication failures in patient sign-out and suggestions for improvement: a critical incident analysis. Qual Saf Health Care 1 December 2005 14 6 401 7 16326783 PMC1744089
Not checked in the free run.
- cites [3] Graber ML Franklin N Gordon R Diagnostic error in internal medicine. Arch Intern Med 11 July 2005 165 13 1493 9 16009864
Not checked in the free run.
- cites [4] Greenberg CC Regenbogen SE Studdert DM Patterns of Communication Breakdowns Resulting in Injury to Surgical Patients. Journal of the American College of Surgeons April 2007 204 4 533 40 17382211
Not checked in the free run.
- cites [5] Kripalani S LeFevre F Phillips CO Williams MV Basaviah P Baker DW Deficits in Communication and Information Transfer Between Hospital-Based and Primary Care Physicians: Implications for Patient Safety and Continuity of Care. JAMA 28 February 2007 297 8 831 41 17327525
Not checked in the free run.
- cites [6] Singh H Thomas EJ Petersen LA Studdert DM Medical Errors Involving Trainees: A Study of Closed Malpractice Claims From 5 Insurers. Arch Intern Med 22 October 2007 167 19 2030 6 17954795
Not checked in the free run.
- cites [7] Gandhi TK Fumbled handoffs: one dropped ball after another. Ann Intern Med 1 March 2005 142 5 352 8 15738454
Not checked in the free run.
- cites [8] Poon EG Gandhi TK Sequist TD Murff HJ Karson AS Bates DW “I wish I had seen this test result earlier!”: Dissatisfaction with test result management systems in primary care. Arch Intern Med 8 November 2004 164 20 2223 8 15534158
Not checked in the free run.
- cites [9] Poon EG Haas JS Louise PA Communication factors in the follow-up of abnormal mammograms. J Gen Intern Med April 2004 19 4 316 23 15061740 PMC1492194
Not checked in the free run.
- cites [10] Schiff GD Introduction: Communicating critical test results. Jt Comm J Qual Patient Saf February 2005 31 2 63 5 15791764
Not checked in the free run.
May combine two claims.
PremisecitationThe VA EMR mostly relies on a View Alert notification system, with radiologist verbal communication only in selected life-threatening cases.
“the electronic medical record (EMR) used by the Department of Veterans Affairs (VA) mostly relies on a notification system (the “View Alert” window) to alert clinicians about critical test results, whereas only in selected life-threatening cases does the radiologist verbally communicate the abnormality to a provider.14”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [14] Singh H Arora HS Vij MS Rao R Khan M Petersen LA Communication outcomes of critical imaging results in a computerized notification system. J Am Med Inform Assoc 2007 14 4 459 66 17460135 PMC2244901
Not checked in the free run.
May combine two claims.
Interpretationown dataThe current study shows substantial lack of timely follow-up even in alerts read by providers, increasing the estimate from previous work.
“Unexpectedly, our current study shows substantial lack of timely follow-up even in alerts read by providers; hence the increase from our previous estimate.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim. May combine two claims.
Study design and conductnone givenIf no response was documented, a second investigator confirmed findings and contacted the ordering provider by telephone.
“If no response was documented, a second study investigator confirmed the findings in the medical record and then contacted the ordering provider by telephone.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
Study design and conductown dataMultivariable logistic regression models accounted for clustering by provider to analyze predictors of lack of acknowledgment and lack of timely follow-up.
“Multivariable logistic regression models accounting for clustering effect by providers were used to identify factors associated with the two outcome variables (i.e. lack of acknowledgment, and lack of timely follow-up).”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
PremisecitationFour weeks was allowed as the maximum time to follow-up on non-life-threatening findings based on previous work.
“We allowed 4 weeks as the maximum time to follow-up on non-life threatening findings based on our previous work.14”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [14] Singh H Arora HS Vij MS Rao R Khan M Petersen LA Communication outcomes of critical imaging results in a computerized notification system. J Am Med Inform Assoc 2007 14 4 459 66 17460135 PMC2244901
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationThe CPRS EMR is used at all VA facilities and maintained within VISTA.
“The Computerized Patient Record System (CPRS) is the EMR used at all VA facilities. It is maintained within the Veterans Health Information Systems and Technology Architecture (VISTA), an automated information system used to support ambulatory and inpatient care.25”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [25] VA Information Resource Center Veterans Health Information Systems and Technology Architecture (VistA). Veterans Health Information Systems and Technology Architecture (VistA) - Description 2007 Available at: URL: http://www.virec.research.va.gov/DataSourcesName/VISTA/VISTA.htm. January 25, 2007
Not checked in the free run.
Classified as probably not a substantial claim.
Study design and conductreasoningVariables with small numbers of patients were removed to ensure stable and precise odds ratio estimates.
“To ensure the model had stable and precise estimates of odds ratios, variables with small numbers of patients with a given characteristic (such as mammography) were removed, and a final model was run on the remaining predictor variables.”
Quote in the paperSubstantial claimMay combine two claims.
PremisecitationOutpatient drug-related alert data show providers ignore alerts due to information overload, often failing to act on important and unimportant alerts.
“Data on outpatient drug related alerts has shown providers ignore alerts because of information overload, 20-23 often resulting in failure to act upon important as well as clinically unimportant alerts.20”
Quote in the paperSubstantial claim0 of 4 sources checked- cites [20] van der Sijs H Aarts J Vulto A Berg M Overriding of Drug Safety Alerts in Computerized Physician Order Entry. J Am Med Inform Assoc 1 March 2006 13 2 138 47 16357358 PMC1447540
Not checked in the free run.
- cites [21] Taylor LK Tamblyn R Reasons for physician non-adherence to electronic drug alerts. Stud Health Technol Inform 2004 107 Pt 2 1101 5 15360983
Not checked in the free run.
- cites [22] Magnus D Rodgers S Avery AJ GPs’ views on computerized drug interaction alerts: questionnaire survey. J Clin Pharm Ther October 2002 27 5 377 82 12383140
Not checked in the free run.
- cites [23] Feldstein A Simon SR Schneider J How to design computerized alerts to safe prescribing practices. Jt Comm J Qual Saf November 2004 30 11 602 13 15565759
Not checked in the free run.
Classified as probably not a substantial claim.
Study design and conductnone givenA reviewer blinded to acknowledgment status evaluated medical records for documented response.
“A reviewer blinded to acknowledgement status evaluated the medical records of patients to determine if there was any documented response to the alert.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenThe local Institutional Review Boards approved the study.
“The local Institutional Review Boards approved the study.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim. May combine two claims.
Study design and conductreasoningThe study's large sample size, multiple clinics, over 120,000 patients, over 500 providers, rigorous methods, 9-month outcome assessment, varied imaging types, and advanced integrated EMR add strength to the findings.
“Conversely, many factors including a large sample size, involvement of several clinics, large number of patients (over 120,000 veterans in southeast Texas) and providers (over 500 from different specialties), rigorous methods to determine timely follow-up and outcomes at 9 months of alerts without timely follow-up, various types of diagnostic imaging tests and an advanced integrated EMR used in VA”
Quote in the paperSubstantial claim
The paper relies on this without support.
No data exists to substantiate a similar problem for diagnostic test result alerts.
“While no data exists to substantiate a similar problem for diagnostic test result alerts”
The paper relies on this without support.
Because of computerized provider order entry, the ordering clinician is always known and notified.
“Because of computerized provider order entry, the ordering clinician is always known and notified.”
The paper relies on this without support.
Institutional protocol requires documentation of verbal communication in the imaging report.
“Institutional protocol requires that documentation of verbal communication is always included in the imaging report.”
The paper relies on this without support.
In our system, residents spend only ½ to 1 day in a two-week period in an outpatient setting usually in their own continuity clinics.
“residents spend only ½ to 1 day in a two-week period in an outpatient setting usually in their own continuity clinics.”
The paper relies on this without support.
Duty hour rules impact resident clinic schedule.
“In addition, duty hour rules also impact their clinic schedule.”
The paper relies on this without support.
Providers in the VA are highly dependent on the View Alert system for their abnormal test results.
“Being their only “inbox” for all types of notifications, providers in the VA are highly dependent on the View Alert system for their abnormal test results.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The result from the baseline EHR group is in line with findings from real clinical settings that indicated physicians did not acknowledge abnormal test results in approximately 4% of cases.
“The result from the baseline EHR group is in line with findings from real clinical settings that indicated physicians did not acknowledge abnormal test results in approximately 4% of cases.19,20”
From Association of the Usability of Electronic Health Records With Cognitive Workload and Performance Levels Among Physicians (2019) · cites this paper as [20]
The Discussion reports an earlier estimate of about 4% for critical imaging results, based only on unacknowledged alerts. That is not this study’s acknowledgment rate: the Results report that 217 of 1,196 alerts (18.1%) were unacknowledged. The claim also broadens the finding from critical imaging results to abnormal test results generally. (LLM, conf=0.87)
Experimental — accuracy not yet measuredEven in the EHR environment, with alerts and reminders in place, physicians continue to often inappropriately manage abnormal test results.
“Even in the EHR environment, with alerts and reminders in place, physicians continue to often inappropriately manage abnormal test results.12,13,14,15,16,17,18,19,20,21”
From Association of the Usability of Electronic Health Records With Cognitive Workload and Performance Levels Among Physicians (2019) · cites this paper as [20]
The Results report that 92 critical imaging alerts (7.7%) lacked timely follow-up at four weeks, despite the EMR alert system described in Methods. The Discussion proposes EMR reminders as a way to improve follow-up; it does not establish that reminders were already in place. The study also addresses abnormal imaging results, rather than abnormal test results generally. (LLM, conf=0.86)
Experimental — accuracy not yet measuredA sample size of 30 participants, each reviewing 35 test results, was required to achieve 80% power to detect a statistically significant difference in cognitive workload and performance.
“On the basis of the anticipated rate of appropriately identified abnormal test results in the literature12,13,14,15,16,17,18,19,20,21 and the anticipated magnitude of the association of the enhanced EHR, we required a sample size of 30 participants, each reviewing 35 test results, to achieve 80% power to detect a statistically significant difference in cognitive workload and performance.”
From Association of the Usability of Electronic Health Records With Cognitive Workload and Performance Levels Among Physicians (2019) · cites this paper as [20]
In previous work, the sociotechnical model was applicable in specific clinical contexts such as test results and referral communication.
“For instance, in our previous work, we found this sociotechnical model applicable in specific clinical contexts (eg, test results and referral communication), 43 -48”
From Exploring the sociotechnical intersection of patient safety and electronic health record implementation (2014) · cites this paper as [46]
The pattern of incomplete follow-up may extend to all alerts of abnormal diagnostic test results, not only laboratory alerts.
“These findings are similar to our previous findings of follow-up on abnormal imaging alerts in the same electronic medical record suggesting that this phenomenon may exist for all alerts of abnormal diagnostic test results.”
From Notification of Abnormal Lab Test Results in an Electronic Medical Record: Do Any Safety Concerns Remain? (2010) · cites this paper as [19]
Earlier research found that ordering practitioners may fail to review abnormal diagnostic imaging alerts, and even when they review them, they may not act promptly on the results.
“In previous work, we found abnormal diagnostic imaging alerts may not always be reviewed by ordering practitioners, and practitioners who review them may not always act upon the transmitted results in a timely manner.”
From Notification of Abnormal Lab Test Results in an Electronic Medical Record: Do Any Safety Concerns Remain? (2010) · cites this paper as [19]
Cites
Kawamoto, K., Houlihan, C. A., Balas, E. A., & Lobach, D. F. (2005) · BMJ · doi:10.1136/bmj.38398.500764.8f · Cited by 2,703 · see citing works
Kripalani, S., LeFevre, F., Phillips, C. O., Williams, M. V., Basaviah, P., & Baker, D. W. (2007) · JAMA · doi:10.1001/jama.297.8.831 · Cited by 2,103 · see citing works
Work system design for patient safety: the SEIPS model
Carayon, P., Schoofs Hundt, A., Karsh, B.-T., Gurses, A. P., Alvarado, C. J., Smith, M., & Flatley Brennan, P. (2006) · Quality in Health Care · doi:10.1136/qshc.2005.015842 · Cited by 1,795 · cited 1× · see citing works
“The impact of several factors such as time pressures, organizational characteristics, factors related to the task of processing alerts, technological factors, and HCP factors should be considered in the design of interventions to improve timely follow-up.(29,30) Future studies should address process-of-care issues lea…”
BackgroundAsh, J. S. (2003) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1471 · Cited by 1,789 · cited 1× · see citing works
“just information transfer—it requires a response from the recipient, such as taking follow-up action and acknowledging receipt of the information to the sender.(15) Previous studies raise concern that HCPs may not read all their alerts.”
BackgroundDiagnostic error in internal medicine
Graber, M. L., Franklin, N., & Gordon, R. (2005) · Archives of Internal Medicine · doi:10.1001/archinte.165.13.1493 · Cited by 1,549 · see citing works
Overriding of Drug Safety Alerts in Computerized Physician Order Entry
van der Sijs, H., Aarts, J., Vulto, A., & Berg, M. (2006) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1809 · Cited by 1,117 · cited 1× · see citing works
“Data on outpatient drug-related alerts have shown that HCPs ignore alerts because of information overload,(20-23) often resulting in failure to act on important as well as clinically unimportant alerts.(20) While no data exist to substantiate a similar problem for diagnostic test result alerts, HCPs often receive seve…”
BackgroundPatterns of Communication Breakdowns Resulting in Injury to Surgical Patients
Greenberg, C. C., Regenbogen, S. E., Studdert, D. M., Lipsitz, S. R., Rogers, S. O., Zinner, M. J., & Gawande, A. A. (2007) · Journal of the American College of Surgeons · doi:10.1016/j.jamcollsurg.2007.01.010 · Cited by 856 · see citing works
Arora, V. M. (2005) · BMJ Quality & Safety · doi:10.1136/qshc.2005.015107 · Cited by 592 · see citing works
Institute of Medicine (2004) · National Academies Press eBooks · doi:10.17226/10863 · Cited by 510 · see citing works
Expanding access to cancer screening and clinical follow-up among the medically underserved.
Freeman, H. P., Muth, B., & Kerner, J. F. (1995) · PubMed · Cited by 449 · see citing works
Medical Errors Involving Trainees: A Study of Closed Malpractice Claims From 5 Insurers
Singh, H., Thomas, E. J., Petersen, L. A., & Studdert, D. M. (2007) · Archives of Internal Medicine · doi:10.1001/archinte.167.19.2030 · Cited by 407 · see citing works
Self-citationPoon, E. G., Gandhi, T. K., Sequist, T. D., Murff, H. J., Karson, A. S., & Bates, D. W. (2004) · Archives of Internal Medicine · doi:10.1001/archinte.164.20.2223 · Cited by 301 · see citing works
Improving follow-up to abnormal breast cancer screening in an urban population
Battaglia, T. A., Roloff, K. S., Posner, M. A., & Freund, K. M. (2006) · Cancer · doi:10.1002/cncr.22354 · Cited by 263 · see citing works
Fumbled handoffs: one dropped ball after another
Gandhi, T. K. (2005) · Annals of Internal Medicine · doi:10.7326/0003-4819-142-5-200503010-00010 · Cited by 246 · see citing works
Patient navigation and case management following an abnormal mammogram: A randomized clinical trial
Ell, K., Vourlekis, B. S., Lee, P.-J., & Xie, B. (2006) · Preventive Medicine · doi:10.1016/j.ypmed.2006.08.001 · Cited by 218 · see citing works
Patient Navigation: A Community Centered Approach to Reducing Cancer Mortality
Freeman, H. P. (2006) · Journal of Cancer Education · doi:10.1207/s15430154jce2101s_4 · Cited by 199 · see citing works
Singh, H., Naik, A. D., Rao, R., & Petersen, L. A. (2008) · Journal of General Internal Medicine · doi:10.1007/s11606-007-0393-z · Cited by 162 · cited 2× · see citing works
“The volume of outpatient care and nature of high-risk transitions between health care practitioners/ providers (HCPs), settings, and systems of care makes timely communication particularly challenging.(13) For example, a patient referred for diagnostic workup for respiratory symptoms by a primary care provider (PCP) m…”
BackgroundSelf-citationHickner, J., Graham, D. G., Elder, N. C., Brandt, E., Emsermann, C. B., Dovey, S., & Phillips, R. (2008) · Quality and Safety in Health Care · doi:10.1136/qshc.2006.021915 · Cited by 158 · see citing works
GPs' views on computerized drug interaction alerts: questionnaire survey
Magnus, D., Rodgers, S., & Avery, A. J. (2002) · Journal of Clinical Pharmacy and Therapeutics · doi:10.1046/j.1365-2710.2002.00434.x · Cited by 157 · see citing works
Understanding diagnostic errors in medicine: a lesson from aviation
Singh, H. (2006) · Quality and Safety in Health Care · doi:10.1136/qshc.2005.016444 · Cited by 156 · see citing works
Self-citationCommunication outcomes of critical imaging results in a computerized notification system
Singh, H., Arora, H. S., Vij, M. S., Rao, R., Khan, M. M., & Petersen, L. A. (2007) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2280 · Cited by 140 · cited 7× · see citing works
“We allowed 4 weeks as the maximum time to follow up on non–life-threatening findings based on our previous work.(14) When we called HCPs and lack of timely follow-up was confirmed, we encouraged them to initiate appropriate follow-up action and rereviewed their records a week later to check for action.”
BackgroundMethodResultInfluentialSelf-citationTo err is not entirely human: Complex technology and user cognition
Horsky, J., Zhang, J., & Patel, V. L. (2005) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2005.05.002 · Cited by 138 · see citing works
Communication factors in the follow-up of abnormal mammograms
Poon, E. G., Haas, J. S., Puopolo, A. L., Gandhi, T. K., Burdick, E., Bates, D. W., & Brennan, T. A. (2004) · Journal of General Internal Medicine · doi:10.1111/j.1525-1497.2004.30357.x · Cited by 138 · cited 1× · see citing works
“However, we believe that our estimates of lack of timely follow-up are likely better than in systems that do not use computerized notification, although accurate and comparable data from such systems is lacking.(9,31) Trainees had a higher likelihood of not reading their alerts but were not associated with a lack of t…”
BackgroundErrors in cancer diagnosis: current understanding and future directions
Singh, H., Sethi, S., Raber, M., & Petersen, L. A. (2007) · Journal of Clinical Oncology · doi:10.1200/jco.2007.13.2142 · Cited by 121 · cited 1× · see citing works
“Policies and procedures regarding the use of verbal communication for such results may need to be reevaluated.(39,40) While the next generation of EMR-based notification systems are being designed, we propose several potential interventions based on our findings that can be used immediately to improve timely follow-up…”
BackgroundSelf-citationWahls, T. L., & Cram, P. M. (2007) · BMC Family Practice · doi:10.1186/1471-2296-8-32 · Cited by 118 · see citing works
Design and implementation of a comprehensive outpatient Results Manager
Poon, E. G., Wang, S. J., Gandhi, T. K., Bates, D. W., & Kuperman, G. J. (2003) · Journal of Biomedical Informatics · doi:10.1016/s1532-0464(03)00061-3 · Cited by 111 · cited 1× · see citing works
“While no data exist to substantiate a similar problem for diagnostic test result alerts, HCPs often receive several hundred normal diagnostic test results and many abnormal alerts every week, suggesting that a similar problem could exist.(24) Given that research in this area has thus far focused on critical results th…”
BackgroundKuperman, G. J., Teich, J. M., Bates, D. W., Hiltz, F. L., Hurley, J. M., Lee, R. Y., & Paterno, M. D. (1996) · Europe PMC (PubMed Central) · Cited by 87 · see citing works
Reasons for Physician Non-Adherence to Electronic Drug Alerts
Taylor, L. K., & Tamblyn, R. M. (2004) · Studies in health technology and informatics · doi:10.3233/978-1-60750-949-3-1101 · Cited by 86 · see citing works
Murff, H. J., Gandhi, T. K., Karson, A. K., Mort, E. A., Poon, E. G., Wang, S. J., Fairchild, D. G., & Bates, D. W. (2003) · International Journal of Medical Informatics · doi:10.1016/s1386-5056(03)00133-3 · Cited by 83 · see citing works
How to Design Computerized Alerts to Ensure Safe Prescribing Practices
Feldstein, A. C., Simon, S. R., Schneider, J. P., Krall, M., Laferriere, D. S., Smith, D. H., Sittig, D. F., & Soumerai, S. B. (2004) · The Joint Commission Journal on Quality and Safety · doi:10.1016/s1549-3741(04)30071-7 · Cited by 81 · see citing works
Self-citationThe Patterns of Communication Breakdowns Resulting in Injury to Surgical Patients
Greenberg, C. C., Regenbogen, S. E., & Studdert, D. N. (2007) · Journal of Vascular Surgery · doi:10.1016/j.jvs.2007.06.019 · Cited by 79 · see citing works
Issues and Initiatives in the Testing Process in Primary Care Physician Offices
Hickner, J. M., Fernald, D. H., Harris, D. M., Poon, E. G., Elder, N. C., & Mold, J. W. (2005) · The Joint Commission Journal on Quality and Patient Safety · doi:10.1016/s1553-7250(05)31012-9 · Cited by 74 · cited 1× · see citing works
“While institutional practices may determine whether the PCP or another ordering HCP (for instance the subspecialist) follows up an abnormality, this problem may be improved by better tracking systems,(14,32-34) using reminders built into the EMR, and care-coordination programs.(27,35-38) Second, direct (verbal) commun…”
BackgroundKushniruk, A. W., & Patel, V. L. (1998) · International Journal of Medical Informatics · doi:10.1016/s1386-5056(98)00106-3 · Cited by 70 · cited 1× · see citing works
“The impact of several factors such as time pressures, organizational characteristics, factors related to the task of processing alerts, technological factors, and HCP factors should be considered in the design of interventions to improve timely follow-up.(29,30) Future studies should address process-of-care issues lea…”
BackgroundChoksi, V. R., Marn, C. S., Bell, Y., & Carlos, R. C. (2006) · American Journal of Roentgenology · doi:10.2214/ajr.04.1913 · Cited by 53 · see citing works
Failure to Recognize and Act on Abnormal Test Results: The Case of Screening Bone Densitometry
Cram, P. M., Rosenthal, G. E., Ohsfeldt, R. L., Wallace, R. B., Schlechte, J., & Schiff, G. D. (2005) · The Joint Commission Journal on Quality and Patient Safety · doi:10.1016/s1553-7250(05)31013-0 · Cited by 51 · cited 1× · see citing works
“However, we believe that our estimates of lack of timely follow-up are likely better than in systems that do not use computerized notification, although accurate and comparable data from such systems is lacking.(9,31) Trainees had a higher likelihood of not reading their alerts but were not associated with a lack of t…”
BackgroundWahls, T. L., Haugen, T. H., & Cram, P. M. (2007) · The Joint Commission Journal on Quality and Patient Safety · doi:10.1016/s1553-7250(07)33052-3 · Cited by 46 · see citing works
Improving Colorectal Cancer Screening by Targeting Office Systems in Primary Care Practices
Wei, E. K., Ryan, C. T., Dietrich, A. J., & Colditz, G. A. (2005) · Archives of Internal Medicine · doi:10.1001/archinte.165.6.661 · Cited by 44 · see citing works
Gandhi, T. K., Sequist, T. D., Poon, E. G., et al. (2003) · PubMed · Cited by 35 · see citing works
Editorial: Doing Better with Critical Test Results
Bates, D. W., & Leape, L. L. (2005) · The Joint Commission Journal on Quality and Patient Safety · doi:10.1016/s1553-7250(05)31010-5 · Cited by 25 · see citing works
Introduction: Communicating Critical Test Results
Schiff, G. D. (2005) · The Joint Commission Journal on Quality and Patient Safety · doi:10.1016/s1553-7250(05)31009-9 · Cited by 19 · see citing works
Advanced alerting features: displaying new relevant data and retracting alerts.
Kuperman, G. J., Hiltz, F. L., & Teich, J. M. (1997) · Europe PMC (PubMed Central) · Cited by 12 · see citing works
Update: sentinel event statistics available online.
(2009) · PubMed · Cited by 5 · see citing works
Updated sentinel event statistics for midyear 2010.
(2010) · PubMed · Cited by 3 · see citing works
Flagged first, then by citations
3 of the 46 references couldn’t be matched to a known paper.
Research works printed but not linked: 2
- [1]Joint Commission on Accreditation of Healthcare Organizations Sentinel event statistics, March 31, 2006. Joint Commission on Accreditation of Healthcare Organizations 2005 Available at: URL: http://www.jointcommission.org/NR/rdonlyres/FA465646-5F5F-4543-AC8F-E8AF6571E372/0/root_cause_se.jpg 19343995
- [17]Wahls TL Haugen T Cram P Diagnostic errors associated with failure to follow up on abnormal results. Joint Commission Journal on Quality and Patient Safety/Joint Commission Resources 2007
Cited by
Sorted by citations
Holden, R. J., Carayon, P., Gurses, A. P., Hoonakker, P., Hundt, A. S., Ozok, A. A., & Rivera-Rodriguez, A. J. (2013) · Ergonomics · doi:10.1080/00140139.2013.838643 · Cited by 1,221 · see citing works
“…2008a, 2008b) Analysis of critical care work systems (Catchpole & McCulloch, 2010) Timeliness of follow-up of abnormal tests in outpatient settings (Singh et al., 2009) Workload and performance obstacles among ICU nurses (Carayon & Gurses, 2008; Gürses & Carayon, 2007; Holden, Scanlon, et al.,…”
BackgroundSittig, D. F., & Singh, H. (2010) · Quality and Safety in Health Care · doi:10.1136/qshc.2010.042085 · Cited by 897 · see citing works
“Safe and effective use of an EHR-based notification system involves many factors that are addressed by almost all dimensions of our model.”
BackgroundSelf-citationHuman factors systems approach to healthcare quality and patient safety
Carayon, P., Wetterneck, T. B., Rivera-Rodriguez, A. J., Hundt, A. S., Hoonakker, P., Holden, R., & Gürses, A. P. (2013) · Applied Ergonomics · doi:10.1016/j.apergo.2013.04.023 · Cited by 746 · see citing works
“Researchers have used the SEIPSmodel to study timeliness of follow-up of abnormal test results in outpatient settings (Singh et al., 2009), to examine the safety of EHR (Electronic Health Record) technology (Sittig and Singh, 2009), to evaluate ways of improving electronic communication and alerts (Hysong et al.”
MethodSingh, H., Meyer, A. N. D., & Thomas, E. J. (2014) · BMJ Quality & Safety · doi:10.1136/bmjqs-2013-002627 · Cited by 677 · see citing works
Self-citationThe incidence of diagnostic error in medicine
Graber, M. L. (2013) · BMJ Quality & Safety · doi:10.1136/bmjqs-2012-001615 · Cited by 626 · see citing works
Clinical Decision Support: a 25 Year Retrospective and a 25 Year Vision
Sittig, D. F., Wright, A. T., & Middleton, B. (2016) · Yearbook of Medical Informatics · doi:10.15265/iys-2016-s034 · Cited by 276 · see citing works
“There has been only modest growth in CDS however, because the focus has been on HIT adoption and use rather than on truly using it as a platform for care transformation [71].”
BackgroundSelf-citationFrequency and Types of Patient-Reported Errors in Electronic Health Record Ambulatory Care Notes
Bell, S. K., Delbanco, T., Elmore, J. G., Fitzgerald, P., Fossa, A. J., Harcourt, K., Leveille, S. G., Payne, T. H., Stametz, R. A., Walker, J., et al. (2020) · JAMA Network Open · doi:10.1001/jamanetworkopen.2020.5867 · Cited by 242 · see citing works
Failure to Follow-Up Test Results for Ambulatory Patients: A Systematic Review
Callen, J. L., Westbrook, J. I., Georgiou, A., & Li, J. (2012) · Journal of General Internal Medicine · doi:10.1007/s11606-011-1949-5 · Cited by 229 · see citing works
“Blurred lines of responsibility were evident in one study which evaluated the practice of dual-alert notification: this measure was adopted as a safeguard and entailed communicating the alert of an abnormal radiology result to two recipients, for example the hospital doctor who ordered the test and the primary care ph…”
BackgroundInfluentialInformation Overload and Missed Test Results in Electronic Health Record–Based Settings
Singh, H., Spitzmueller, C., Petersen, N. J., Sawhney, M. K., & Sittig, D. F. (2013) · JAMA Internal Medicine · doi:10.1001/2013.jamainternmed.61 · Cited by 209 · see citing works
“Off-label use is often recommended, but regulatory agencies have raised concerns about hypersensitivity reactions and neuropsychiatric adverse effects.(1,2) It is unclear how often and for which conditions modafinil is being prescribed in clinical practice.”
BackgroundSelf-citationElectronic health records and national patient-safety goals
Sittig, D. F., & Singh, H. (2012) · New England Journal of Medicine · doi:10.1056/nejmsb1205420 · Cited by 204 · see citing works
Self-citationDefining Health Information Technology–Related Errors
Sittig, D. F. (2011) · Archives of Internal Medicine · doi:10.1001/archinternmed.2011.327 · Cited by 167 · see citing works
Self-citationSingh, H., Thomas, E. J., Sittig, D. F., Wilson, L., Espadas, D., Khan, M. M., & Petersen, L. A. (2010) · The American Journal of Medicine · doi:10.1016/j.amjmed.2009.07.027 · Cited by 160 · see citing works
Self-citationZapka, J., Taplin, S. H., Anhang Price, R., Cranos, C., & Yabroff, R. (2010) · JNCI Monographs · doi:10.1093/jncimonographs/lgq009 · Cited by 145 · see citing works
“(61) when they found that even when health-care providers received and read abnormal results in an integrated medical record system, timely follow-up did not always occur.”
BackgroundLyratzopoulos, G., Vedsted, P., & Singh, H. (2015) · British Journal of Cancer · doi:10.1038/bjc.2015.47 · Cited by 154 · see citing works
“In England, work conducted by the (former) National Patient Safety Agency has highlighted factors implicated in diagnostic delays in cancer and recommended ‘routine monitoring of delayed diagnosis’ (NPSA, 2010).”
BackgroundMethodInfluentialSelf-citationMurphy, D. R., Reis, B., Sittig, D. F., & Singh, H. (2012) · The American Journal of Medicine · doi:10.1016/j.amjmed.2011.07.029 · Cited by 142 · see citing works
“We previously found that about 7% of abnormal laboratory and imaging test result alerts were missed in EHR systems.(8,9) Information overload from too any alerts might be a potential reason, given that practiioners must sort through a large number of alerts daily.”
BackgroundSelf-citationMazur, L. M., Mosaly, P. R., Moore, C., & Marks, L. (2019) · JAMA Network Open · doi:10.1001/jamanetworkopen.2019.1709 · Cited by 127 · see citing works
System-related interventions to reduce diagnostic errors: a narrative review
Singh, H., Graber, M. L., KISSAM, S. M., Sorensen, A. V., Lenfestey, N. F., Tant, E. M., Henriksen, K., & LaBresh, K. A. (2011) · BMJ Quality & Safety · doi:10.1136/bmjqs-2011-000150 · Cited by 134 · see citing works
“For instance, rapid prototyping techniques could be used to identify awkward and confusing interfaces, while testing the interventions in simulated or actual clinical settings might help identify unintended consequences.(62) 63 Better designs could help ensure that once an error occurs, it does not cascade through the…”
BackgroundSelf-citationSummarization of clinical information: A conceptual model
Feblowitz, J. C., Wright, A. T., Singh, H., Samal, L., & Sittig, D. F. (2011) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2011.03.008 · Cited by 124 · see citing works
Self-citationSingh, H., Hirani, K., Kadiyala, H., Rudomiotov, O., Giardina, T. D., Khan, M. M., & Wahls, T. L. (2010) · Journal of Clinical Oncology · doi:10.1200/jco.2009.25.6636 · Cited by 135 · see citing works
“However, for several reasons, clinicians may not always act on abnormal imaging results in a timely manner.(50) It is unlikely that this problem is unique to the VA.”
BackgroundSelf-citationElectronic Health Record-Based Surveillance of Diagnostic Errors in Primary Care
Singh, H., Giardina, T. D., Forjuoh, S. N., Reis, M. D., Kosmach, S., Khan, M. M., & Thomas, E. J. (2012) · BMJ Quality & Safety · doi:10.1136/bmjqs-2011-000304 · Cited by 131 · see citing works
Self-citationElectronic health record-based triggers to detect potential delays in cancer diagnosis
Murphy, D. R., Laxmisan, A., Reis, B., Thomas, E. J., Esquivel, A., Forjuoh, S. N., Parikh, R. C., Khan, M. M., & Singh, H. (2013) · BMJ Quality & Safety · doi:10.1136/bmjqs-2013-001874 · Cited by 125 · see citing works
Self-citationSingh, H. (2014) · The Joint Commission Journal on Quality and Patient Safety · doi:10.1016/s1553-7250(14)40012-6 · Cited by 108 · see citing works
Self-citationExploring situational awareness in diagnostic errors in primary care
Singh, H., Giardina, T. D., Petersen, L. A., Smith, M. W., Paul, L. W., Dismukes, K., Bhagwath, G., & Thomas, E. J. (2011) · BMJ Quality & Safety · doi:10.1136/bmjqs-2011-000310 · Cited by 103 · see citing works
“Retrieval of diagnostic information from several distributed sources(65) will become increasingly challenging as healthcare becomes more integrated through EHRs or health information exchanges.(66) 67 Thus, measuring and maximising various components of providers’ SA should be an essential consideration to reduce diag…”
BackgroundSelf-citationLegal, Ethical, and Financial Dilemmas in Electronic Health Record Adoption and Use
Sittig, D. F., & Singh, H. (2011) · Pediatrics · doi:10.1542/peds.2010-2184 · Cited by 100 · see citing works
Self-citationMurphy, D. R., Wu, L., Thomas, E. J., Forjuoh, S. N., Meyer, A. N. D., & Singh, H. (2015) · Journal of Clinical Oncology · doi:10.1200/jco.2015.61.1301 · Cited by 100 · see citing works
“These delays often result from a variety of cognitive and system factors that lead to lack of timely evaluation of red flag signs and symptoms or abnormal test results.(9-21) Despite increasing adoption of electronic health records (EHRs) and greater access to diagnostic information, delays persist(10,18,19,22-26) in…”
BackgroundSelf-citationCreating a Patient-Centered Imaging Service: Determining What Patients Want
Basu, P. A., Ruiz-Wibbelsmann, J. A., Spielman, S., Dalsem, V. F. V., Rosenberg, J., & Glazer, G. M. (2011) · American Journal of Roentgenology · doi:10.2214/ajr.10.5333 · Cited by 97 · see citing works
“In a study in an outpatient setting from November 2007 to June 2008, an electronic medical records system generated alerts for 1,196 abnormal images of 123,638 radiologic studies [14].”
BackgroundInfluentialMeeks, D. W., Takian, A., Sittig, D. F., Singh, H., & Barber, N. (2014) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2013-001762 · Cited by 91 · see citing works
Self-citationPatients Prefer Results From the Ordering Provider and Access to Their Radiology Reports
Cabarrus, M. C., Naeger, D. M., Rybkin, A. V., et al. (2015) · Journal of the American College of Radiology · doi:10.1016/j.jacr.2014.12.009 · Cited by 93 · see citing works
“Few studies have examined patients’ preferences, and those that have, have yielded conflicting results [8-13].”
BackgroundSingh, H., Spitzmueller, C., Petersen, N. J., Sawhney, M. K., Smith, M. W., Murphy, D. R., Espadas, D., Laxmisan, A., & Sittig, D. F. (2013) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2012-001267 · Cited by 92 · see citing works
Self-citationBeaber, E. F., Kim, J. J., Schapira, M. M., Tosteson, A. N. A., Zauber, A. G., Geiger, A. M., Kamineni, A., Weaver, D. L., & Tiro, J. A. (2015) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/djv120 · Cited by 88 · see citing works
Giardina, T. D., King, B. J., Ignaczak, A. P., Paull, D. E., Hoeksema, L. J., Mills, P. D., Neily, J. B., Hemphill, R. R., & Singh, H. (2013) · Health Affairs · doi:10.1377/hlthaff.2013.0130 · Cited by 85 · see citing works
Self-citationEight Recommendations for Policies for Communicating Abnormal Test Results
Singh, H., & Vij, M. S. (2010) · The Joint Commission Journal on Quality and Patient Safety · doi:10.1016/s1553-7250(10)36037-5 · Cited by 83 · see citing works
Self-citationThe patient portal and abnormal test results: An exploratory study of patient experiences
Giardina, T. D., Modi, V., Parrish, D. E., et al. (2015) · Patient Experience Journal · doi:10.35680/2372-0247.1055 · Cited by 74 · see citing works
Self-citationUnderstanding fragmentation of prostate cancer survivorship care
Skolarus, T. A., Zhang, Y., & Hollenbeck, B. K. (2011) · Cancer · doi:10.1002/cncr.26601 · Cited by 72 · see citing works
“Such PSA testing may have associated opportunity costs for providers who are ultimately responsible for the results and for patients subjected to additional visits and testing.(34) Perhaps more consolidation of care and better delineation of provider roles after initial treatment are Table 3.”
BackgroundSittig, D. F., Ash, J. S., & Singh, H. (2014) · PubMed · Cited by 72 · see citing works
Self-citationUnderstanding the management of electronic test result notifications in the outpatient setting
Hysong, S. J., Sawhney, M. K., Wilson, L., Sittig, D. F., Esquivel, A., Singh, S., & Singh, H. (2011) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-11-22 · Cited by 68 · see citing works
“For almost a decade, both study sites have used the Computerized Patient Record System (CPRS), the EHR system in use at VA facilities nationwide.”
BackgroundMethodInfluentialSelf-citationSociotechnical systems approach to healthcare quality and patient safety
Carayon, P. (2012) · Work · doi:10.3233/wor-2012-0091-3850 · Cited by 64 · see citing works
“Researchers have used the SEIPS model to study timeliness of follow-up of abnormal test results in outpatient settings [34], to examine the safety of EHR technology [35], to evaluate ways of improving electronic communication and alerts [25], and to assess work system barriers and facilitators to the provision of outp…”
MethodProvider management strategies of abnormal test result alerts: a cognitive task analysis
Hysong, S. J., Sawhney, M. K., Wilson, L., Sittig, D. F., Espadas, D., Davis, T., & Singh, H. (2010) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m3200 · Cited by 65 · see citing works
“For instance, we recently found that almost 8% of critical imaging results may not receive timely follow-up actions despite evidence of transmission of the alert to the ordering provider through a " View Alert " notification window in CPRS.”
BackgroundInfluentialSelf-citationMeasures to Improve Diagnostic Safety in Clinical Practice
Singh, H., Graber, M. L., & Hofer, T. P. (2016) · Journal of Patient Safety · doi:10.1097/pts.0000000000000338 · Cited by 66 · see citing works
“42 Measurement criteria are better defined for missed test results than other types of missed opportunities.(42,43)”
BackgroundSelf-citationLacson, R. C., Prevedello, L. M. S., Andriole, K. P., O’Connor, S. D., Roy, C. L., Gandhi, T. K., Dalal, A. K., Sato, L., & Khorasani, R. (2014) · American Journal of Roentgenology · doi:10.2214/ajr.14.13064 · Cited by 63 · see citing works
“No established guidelines exist for classifying imaging results as “critical,” unlike for laboratory tests where an outlier value beyond the range of acceptable values is deemed abnormal—making them easily amenable to automated alerts [30–32].”
BackgroundAnthony, S. G., Prevedello, L. M., Damiano, M. M., Gandhi, T. K., Doubilet, P. M., Seltzer, S. E., & Khorasani, R. (2011) · Radiology · doi:10.1148/radiol.11101396 · Cited by 61 · see citing works
Provider‐to‐provider electronic communication in the era of meaningful use: A review of the evidence
Walsh, C. G., Siegler, E. L., Cheston, E., O’Donnell, H. C., Collins, S., Stein, D. M., Vawdrey, D. K., & Stetson, P. D. (2013) · Journal of Hospital Medicine · doi:10.1002/jhm.2082 · Cited by 53 · see citing works
“Lack of acknowledgement of alerts impacted the results of every study, and in one of these studies, alerting two physicians, rather than just one, decreased the likelihood that the results would be followed up.(57) Providers did prefer e-mail to fax notification.”
ResultImproving Ambulatory Patient Safety
Wynia, M. K., & Classen, D. C. (2011) · JAMA · doi:10.1001/jama.2011.1820 · Cited by 58 · see citing works
“For example, lack of timely follow-up among outpatients with abnormal laboratory and imaging test results is a common error in the ambulatory arena, which leads to harms, and there are promising ways to track and improve performance in this regard.(7) A coordinated campaign would be needed, however, including the adop…”
BackgroundLanguage Barriers, Location of Care, and Delays in Follow-up of Abnormal Mammograms
Karliner, L., Ma, L., Hofmann, M., & Kerlikowske, K. M. (2011) · Medical Care · doi:10.1097/mlr.0b013e31822dcf2d · Cited by 60 · see citing works
“However, it will be important not to rely only on an electronic system as electronic reminders to physicians for follow-up of abnormal diagnostic tests have been shown to be inadequate for achieving consistent, timely follow-up.(44) In light of this inadequacy, and the documented importance of adequate verbal communic…”
BackgroundFollow-up Actions on Electronic Referral Communication in a Multispecialty Outpatient Setting
Singh, H., Esquivel, A., Sittig, D. F., Murphy, D., Kadiyala, H., Schiesser, R., Espadas, D., & Petersen, L. A. (2011) · Journal of General Internal Medicine · doi:10.1007/s11606-010-1501-z · Cited by 57 · see citing works
Self-citationAdvancing the research agenda for diagnostic error reduction
Zwaan, L., Schiff, G. D., & Singh, H. (2013) · BMJ Quality & Safety · doi:10.1136/bmjqs-2012-001624 · Cited by 56 · see citing works
Self-citationBell, S. K., Folcarelli, P. H., Fossa, A. J., Gerard, M., Harper, M. B., Leveille, S. G., Moore, C., Sands, K., Lee, B. S., Walker, J., et al. (2021) · PubMed · doi:10.1097/pts.0000000000000494 · Cited by 53 · see citing works
Reducing Delay in Diagnosis: Multistage Recommendation Tracking
Wandtke, B., & Gallagher, S. (2017) · American Journal of Roentgenology · doi:10.2214/ajr.17.18332 · Cited by 53 · see citing works
“Sending EHR-based alerts to both the primary care provider (PCP) and the ordering clinician decreased the likelihood of timely follow-up, a finding that was attributed to diffusion of responsibility [15].”
BackgroundVirani, S. S., Kennedy, K. F., Akeroyd, J. M., Morris, P. B., Bittner, V. A., Masoudi, F. A., Stone, N. J., Petersen, L. A., & Ballantyne, C. M. (2018) · Circulation: Cardiovascular Quality and Outcomes · doi:10.1161/circoutcomes.118.004652 · Cited by 53 · see citing works
Self-citationReducing Diagnostic Error Through Medical Home-Based Primary Care Reform
Singh, H., & Graber, M. L. (2010) · JAMA · doi:10.1001/jama.2010.1035 · Cited by 52 · see citing works
“For instance, the US Department of Veterans Affairs (VA) uses an advanced EHR and has a relatively unique integrated model of care that resembles PCMH, yet diagnostic errors persist in this system.(5,6) Whether medical homes can effectively improve safety will depend substantially on their ability to deliver timely an…”
Self-citationMurphy, D. R. (2012) · Archives of Internal Medicine · doi:10.1001/archinternmed.2011.740 · Cited by 52 · see citing works
Sarkar, U., Bonacum, D., Strull, W. M., Spitzmueller, C., Jin, N., López, A., Giardina, T. D., Meyer, A. N. D., & Singh, H. (2012) · BMJ Quality & Safety · doi:10.1136/bmjqs-2011-000541 · Cited by 53 · see citing works
“Other recent studies also demonstrate this to be a high-risk area despite improved information availability.(20) 21 For instance, studies reveal suboptimal follow-up of abnormal test results, even in the presence of electronic health record (EHR)-based clinician notification, suggesting that that missed or delayed dia…”
BackgroundSelf-citationCommunication breakdowns and diagnostic errors: a radiology perspective
Murphy, D. R., Singh, H., & Berlin, L. (2014) · Diagnosis · doi:10.1515/dx-2014-0035 · Cited by 52 · see citing works
Self-citationEbola US Patient Zero: lessons on misdiagnosis and effective use of electronic health records
Upadhyay, D. K., Sittig, D. F., & Singh, H. (2014) · Diagnosis · doi:10.1515/dx-2014-0064 · Cited by 50 · see citing works
“EHRs can lead to less verbal exchange [27], which is all the more needed and more effective when dealing with complex tasks and communicating critical information [28].”
BackgroundSelf-citationShould Patients Get Direct Access to Their Laboratory Test Results?
Giardina, T. D., & Singh, H. (2011) · JAMA · doi:10.1001/jama.2011.1797 · Cited by 47 · see citing works
Self-citationA Systems Approach to Identify Factors Influencing Adverse Drug Events in Nursing Homes
Al‐Jumaili, A. A., & Doucette, W. R. (2018) · Journal of the American Geriatrics Society · doi:10.1111/jgs.15389 · Cited by 38 · see citing works
“The SEIPS model also guided work in clinical discipline, workload obstacles for ICU nurses,(24) and workflow design and evaluation in outpatient settings.(25) In 2017, a review article was published summarizing the factors influencing medication safety in NHs and classified the literature according to the 5 domains of…”
MethodCallen, J. L., Giardina, T. D., Singh, H., Li, L., Paoloni, R., Georgiou, A., Runciman, W. B., & Westbrook, J. I. (2015) · Journal of Medical Internet Research · doi:10.2196/jmir.3721 · Cited by 43 · see citing works
“Radiology result follow-up remains a vulnerable area [15,56,57].”
BackgroundSelf-citationHow context affects electronic health record-based test result follow-up: a mixed-methods evaluation
Menon, S., Smith, M. W., Sittig, D. F., Petersen, N. J., Hysong, S. J., Espadas, D., Modi, V., & Singh, H. (2014) · BMJ Open · doi:10.1136/bmjopen-2014-005985 · Cited by 45 · see citing works
“Although some variation in test result follow-up can be attributed to individual providers,(12) system factors, such as organisational policies and procedures, are likely to play a substantial role.(10) 16 For example, in organisations using EHRs, the effectiveness of test result management may be influenced by techni…”
BackgroundSelf-citationMurphy, D. R., Thomas, E. J., Meyer, A. N. D., & Singh, H. (2015) · Radiology · doi:10.1148/radiol.2015142530 · Cited by 43 · see citing works
“Approximately 9% of patients with abnormal chest imaging findings suspicious for neoplasm fail to receive timely follow-up diagnostic evaluation (2,28).”
BackgroundSelf-citationTen Strategies to Improve Management of Abnormal Test Result Alerts in the Electronic Health Record
Singh, H., Wilson, L., Reis, B., Sawhney, M. K., Espadas, D., & Sittig, D. F. (2010) · Journal of Patient Safety · doi:10.1097/pts.0b013e3181ddf652 · Cited by 43 · see citing works
Self-citationWorkarounds and Test Results Follow-up in Electronic Health Record-Based Primary Care
Menon, S., Murphy, D. R., Singh, H., Meyer, A. N. D., & Sittig, D. F. (2016) · Applied Clinical Informatics · doi:10.4338/aci-2015-10-ra-0135 · Cited by 40 · see citing works
“Follow-up of abnormal test results is a high-risk clinical process, and its failure can lead to substantial patient safety implications and care delays [9].”
BackgroundSelf-citationBates, D., & Samal, L. (2018) · Health Services Research · doi:10.1111/1475-6773.12852 · Cited by 38 · see citing works
Hammer, M. M., Kapoor, N., Desai, S. P., et al. (2019) · American Journal of Roentgenology · doi:10.2214/ajr.18.20692 · Cited by 40 · see citing works
Patients as diagnostic collaborators: sharing visit notes to promote accuracy and safety
Blease, C., & Bell, S. K. (2019) · Diagnosis · doi:10.1515/dx-2018-0106 · Cited by 34 · see citing works
Computerized Triggers of Big Data to Detect Delays in Follow-up of Chest Imaging Results
Murphy, D. R., Meyer, A. N. D., Bhise, V., Russo, E. M., Sittig, D. F., Wei, L., Wu, L., & Singh, H. (2016) · CHEST Journal · doi:10.1016/j.chest.2016.05.001 · Cited by 39 · see citing works
Self-citationProcessing of discharge summaries in general practice: a retrospective record review
Spencer, R. A., Spencer, S. E. F., Rodgers, S., Campbell, S., & Avery, A. J. (2018) · British Journal of General Practice · doi:10.3399/bjgp18x697877 · Cited by 36 · see citing works
Dalal, A. K., Roy, C. L., Poon, E. G., Williams, D. H., Nolido, N. V., Yoon, C. S., Budris, J., Gandhi, T. K., Bates, D. W., & Schnipper, J. L. (2013) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2013-002030 · Cited by 40 · see citing works
Challenges in patient safety improvement research in the era of electronic health records
Russo, E. M., Sittig, D. F., Murphy, D. R., & Singh, H. (2016) · Healthcare · doi:10.1016/j.hjdsi.2016.06.005 · Cited by 39 · see citing works
Self-citationImproving Test Result Follow-up through Electronic Health Records Requires More than Just an Alert
Sittig, D. F., & Singh, H. (2012) · Journal of General Internal Medicine · doi:10.1007/s11606-012-2161-y · Cited by 37 · see citing works
“We previously identified ambiguity of responsibility for test result follow-up to be a key factor in failure to follow up abnormal results.(4) Several EHRs now use asynchronous alert notifications to transmit results, but providers often receive many other types of notifications in their electronic in-box.”
BackgroundSelf-citationGeorgiou, A., Li, J., Thomas, J., Dahm, M. R., & Westbrook, J. I. (2019) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocz032 · Cited by 40 · see citing works
“The authors noted that the procedures for management of results were not uniform, with only 55% of respondents reporting use of the electronic notification system for results management.38 Observational studies by Singh et al.32,34 and Bhise et al.57 reported a range of between 0.2% and 16.7% failure to follow up test…”
BackgroundInfluentialSloan, C. E., Chadalavada, S. C., Cook, T. S., Langlotz, C. P., Schnall, M. D., & Zafar, H. M. (2014) · Academic Radiology · doi:10.1016/j.acra.2014.07.006 · Cited by 39 · see citing works
“not documented, patients are less likely to receive follow-up care, even when physicians are aware of the abnormal results (16,17,20).”
BackgroundResultImpact of a national QI programme on reducing electronic health record notifications to clinicians
Shah, T., Patel-Teague, S., Kroupa, L., Meyer, A. N. D., & Singh, H. (2018) · BMJ Quality & Safety · doi:10.1136/bmjqs-2017-007447 · Cited by 36 · see citing works
Self-citationNaik, A. D., & Singh, H. (2010) · Medical Decision Making · doi:10.1177/0272989x10385846 · Cited by 35 · see citing works
“Although some degree of synchronous communication is always needed (such as communicating life-threatening findings), 17 frequent synchronous communication demands produce cognitive overload and excessive reliance on working, or short-term, memory resulting in work-related stress and anxiety as…”
BackgroundSelf-citationImproving outpatient safety through effective electronic communication: a study protocol
Hysong, S. J., Sawhney, M. K., Wilson, L., Sittig, D. F., Esquivel, A., Watford, M., Giardina, T. D., Espadas, D., & Singh, H. (2009) · Implementation Science · doi:10.1186/1748-5908-4-62 · Cited by 34 · see citing works
Self-citationHuman Factors and Ergonomics in Health Care and Patient Safety
Carayon, P. (2011) · CRC Press eBooks · doi:10.1201/b11219-8 · Cited by 36 · see citing works
Powell, L., Sittig, D. F., Chrouser, K. L., & Singh, H. (2020) · JAMA Network Open · doi:10.1001/jamanetworkopen.2020.6752 · Cited by 34 · see citing works
Self-citationDiagnostic errors: moving beyond ‘no respect’ and getting ready for prime time
Singh, H. (2013) · BMJ Quality & Safety · doi:10.1136/bmjqs-2013-002387 · Cited by 32 · see citing works
Self-citationLacson, R. C., O’Connor, S. D., Sahni, V. A., Roy, C. L., Dalal, A. K., Desai, S. P., & Khorasani, R. (2015) · BMJ Quality & Safety · doi:10.1136/bmjqs-2015-004276 · Cited by 36 · see citing works
Smith, M., Murphy, D., Laxmisan, A., Sittig, D., Reis, B., Esquivel, A., & Singh, H. (2013) · Applied Clinical Informatics · doi:10.4338/aci-2013-04-ra-0019 · Cited by 31 · see citing works
“In previous work involving 2,500 asynchronous alerts related to abnormal laboratory and imaging results, we found that 7% of laboratory and 8% of imaging result alerts lacked timely follow-up at 4 weeks [3, 4], even in cases with explicit acknowledgement of the notification by the recipient.”
BackgroundSelf-citationAmroze, A., Field, T. S., Fouayzi, H., Sundaresan, D., Burns, L., Garber, L., Sadasivam, R. S., Mazor, K. M., Gurwitz, J. H., & Cutrona, S. L. (2019) · JMIR Medical Informatics · doi:10.2196/12650 · Cited by 25 · see citing works
Murphy, D. R., Meyer, A. N. D., Vaghani, V., Russo, E., Sittig, D. F., Wei, L., Wu, L., & Singh, H. (2018) · Journal of the American College of Radiology · doi:10.1016/j.jacr.2017.10.001 · Cited by 30 · see citing works
Self-citationReleasing test results directly to patients: A multisite survey of physician perspectives
Giardina, T. D., Callen, J. L., Georgiou, A., Westbrook, J. I., Greisinger, A. J., Esquivel, A., Forjuoh, S. N., Parrish, D. E., & Singh, H. (2015) · Patient Education and Counseling · doi:10.1016/j.pec.2015.02.011 · Cited by 30 · see citing works
Self-citationKapoor, N., Lynch, E. A., Lacson, R. C., Flash, M. J. E., Guenette, J. P., Desai, S. P., Eappen, S., & Khorasani, R. (2022) · American Journal of Roentgenology · doi:10.2214/ajr.22.28378 · Cited by 28 · see citing works
Electronic health record–related safety concerns: A cross‐sectional survey
Menon, S., Singh, H., Meyer, A. N. D., Belmont, E., & Sittig, D. F. (2014) · Journal of Healthcare Risk Management · doi:10.1002/jhrm.21146 · Cited by 29 · see citing works
Self-citationTranslating Evidence into Practice Using a Systems Engineering Framework for Infection Prevention
Yanke, E. M., Carayon, P., & Safdar, N. (2014) · Infection Control and Hospital Epidemiology · doi:10.1086/677638 · Cited by 27 · see citing works
“It has been used to evaluate the timely follow-up of abnormal outpatient imaging results with an advanced, integrated electronic medical record (Veterans Health Administration’s Computerized Patient Record System).(26) Qualitatively, the SEIPS model has been employed to assess barriers to providing cognitive pharmaceu…”
MethodLack of Timely Follow-Up of Abnormal Imaging Results and Radiologists’ Recommendations
Al-Mutairi, A., Meyer, A. N. D., Chang, P. J., & Singh, H. (2015) · Journal of the American College of Radiology · doi:10.1016/j.jacr.2014.09.031 · Cited by 30 · see citing works
“Previous studies indicate that abnormal imaging results may not lead to timely follow-up even when they are transmitted to the ordering provider as electronic alerts [1-3].”
BackgroundMethodInfluentialSelf-citationWhy Test Results Are Still Getting “Lost” to Follow-up: a Qualitative Study of Implementation Gaps
Zimolzak, A. J., Shahid, U., Giardina, T. D., Memon, S. A., Mushtaq, U., Zubkoff, L., Murphy, D. R., Bradford, A. M., & Singh, H. (2021) · Journal of General Internal Medicine · doi:10.1007/s11606-021-06772-y · Cited by 27 · see citing works
Self-citationLaxmisan, A., Sittig, D. F., Pietz, K., Espadas, D., Krishnan, B., & Singh, H. (2012) · Medical Care · doi:10.1097/mlr.0b013e31825f6619 · Cited by 26 · see citing works
Self-citationO’Connor, S. D., Dalal, A. K., Sahni, V. A., Lacson, R. C., & Khorasani, R. (2015) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocv105 · Cited by 25 · see citing works
“The followup rate we observed is higher than in other studies of abnormal findings(15,16) and follow-up recommendations(17,18) in radiology reports, but it is consistent with other critical result alert systems.(8,7) Additionally, our findings confirm that results management applications work best when designed to sup…”
ResultElectronic Messaging System for Communicating Important, but Nonemergent, Abnormal Imaging Results
Eisenberg, R. L., Yamada, K., Yam, C.-S., Spirn, P. W., & Kruskal, J. B. (2010) · Radiology · doi:10.1148/radiol.10101015 · Cited by 26 · see citing works
Cutrona, S. L., Fouayzi, H., Burns, L., Sadasivam, R. S., Mazor, K. M., Gurwitz, J. H., Garber, L., Sundaresan, D., Houston, T. K., & Field, T. S. (2017) · Journal of General Internal Medicine · doi:10.1007/s11606-017-4146-3 · Cited by 24 · see citing works
Azqhandi, M. H. A., Shekari, M., & Ghalami‐Choobar, B. (2018) · Applied Organometallic Chemistry · doi:10.1002/aoc.4410 · Cited by 24 · see citing works
Omidi, M. H., Azqhandi, M. H. A., & Ghalami‐Choobar, B. (2018) · New Journal of Chemistry · doi:10.1039/c8nj03277g · Cited by 24 · see citing works
Shelver, J., Wendt, C. H., McClure, M., et al. (2017) · Journal of the American College of Radiology · doi:10.1016/j.jacr.2017.02.001 · Cited by 24 · see citing works
“Institutions across the country have approached this problem by developing interventions to reduce the loss-to-follow-up rates, with varying degrees of success [3,4,14-17].”
Edmonds, S. W., Wolinsky, F. D., Christensen, A. J., Lü, X., Jones, M. P., Roblin, D. W., Saag, K. G., & Cram, P. M. (2012) · Contemporary Clinical Trials · doi:10.1016/j.cct.2012.10.002 · Cited by 23 · see citing works
“Problems in virtually all aspects of the test result reporting and communication process have been documented and result in important test results being overlooked or missed [18-23].”
Hanley, O., Lotfi, A. S., Sanborn, T., et al. (2017) · Southern Medical Journal · doi:10.14423/smj.0000000000000741 · Cited by 23 · see citing works
“Although verbal communication is ideal and is associated with increased appropriate follow-up, it is not always possible and the electronic medical record remains the main tool of communication.(16) The major strengths of our study include thorough chart review using a validated abstraction form with high interrater a…”
Murphy, D. R., Satterly, T., Rogith, D., Sittig, D. F., & Singh, H. (2019) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2019.04.004 · Cited by 21 · see citing works
Self-citationDanforth, K. N., Hahn, E. E., Slezak, J. M., Chen, L. H., Li, B. H., Munoz‐Plaza, C. E., Luong, T. Q., Harrison, T. N., Mittman, B. S., Sim, J. J., et al. (2019) · American Journal of Kidney Diseases · doi:10.1053/j.ajkd.2019.05.003 · Cited by 24 · see citing works
Self-citationThe fate of radiology report recommendations at a pediatric medical center
Lee, B., Otero, H. J., & Whitehead, M. T. (2017) · Pediatric Radiology · doi:10.1007/s00247-017-3960-4 · Cited by 24 · see citing works
Assigning responsibility to close the loop on radiology test results
Kwan, J. L., & Singh, H. (2017) · Diagnosis · doi:10.1515/dx-2017-0019 · Cited by 23 · see citing works
“During the study, almost 20% of imaging-related alerts were not acknowledged, and almost 8% of abnormal imaging results lacked timely follow-up at 4 weeks [9].”
Self-citation
- Authors
- Singh, H., Thomas, E. J., Mani, S., Sittig, D., Arora, H., Espadas, D., Khan, M. M., & Petersen, L. A.
- Affiliations
- Michael E. DeBakey VA Medical Center (US)
- Memorial Hermann (US)
- Baylor College of Medicine (US)
- Venue
- Archives of Internal Medicine
- Publisher
- American Medical Association (AMA)
- Published
- 2009-09-28
- Volume
- 169
- Issue
- 17
- Pages
- 1578-86
- ISSN
- 0003-9926 (ISSN-L)
- PMC
- PMC2919821
- OpenAlex
- W2143111289
- Semantic Scholar
- e10c1783993bfba9a725ad6763f791cfa11a7c91
- Cited by
- 212
- References
- 46
- Citations by year
- 2026: 5 · 2025: 3 · 2024: 2 · 2023: 9 · 2022: 7 · 2021: 10 · 2020: 11 · 2019: 10 · 2018: 18 · 2017: 12 · 2016: 15 · 2015: 21 · 2014: 18 · 2013: 19 · 2012: 13
- FWCI
- 13.03
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Archives of Internal Medicine (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- Digital Commons-TMC (Texas Medical Center) (submittedVersion)
- Funders
- National Institutes of Health
- National Cancer Institute (K23CA125585)
- Topics
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiology practices and education
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Health Sciences › Medicine › Physiology › Clinical Laboratory Practices and Quality Control
- Keywords
- electronic health records, clinical decision support alerts, veterans' health care, trainee physicians, health care costs, quality of care, health care utilization
- MeSH terms
- Communication, Continuity of Patient Care, Diagnostic Imaging, Hospitals, Veterans, Humans, Outpatient Clinics, Hospital, Practice Patterns, Physicians', Radiology, Retrospective Studies, Time Factors, Medical Records Systems, Computerized, Reminder Systems, Process Assessment, Health Care
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-07.
Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-07.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
