Are Residents’ Decisions Influenced More by a Decision Aid or a Specialist’s Opinion? A Randomized Controlled Trial
Open accessAbstract
Background Physicians are reluctant to use decision aids despite their ability to improve care. A potential reason may be that physicians do not believe decision aid advice.Objective To determine whether internal medicine residents lend more credence to contradictory decision aid or human advice.Design Randomized controlled trial. Residents read a scenario of a patient with community-acquired pneumonia and were asked whether they would admit the patient to the intensive care unit or the floor. Residents were randomized to receive contrary advice from either a referenced decision aid or an anonymous pulmonologist. They were then asked, in light of this new information, where they would admit the patient.Participants One hundred eight internal medicine residents.Measurements The percentage of residents who changed their admission location and the change in confidence in the decision.Main results Residents were more likely to change their original admission location (OR 2.3, 95% CI 1.04 to 5.1, P = 0.04) and to reduce their confidence in the decision (adjusted difference between means -12.9%, 95% CI -3.0% to -22.8%, P = 0.011) in response to the referenced decision aid than to the anonymous pulmonologist. Confidence in their decision was more likely to change if they initially chose to admit the patient to the floor.Conclusions In a hypothetical case of community-acquired pneumonia, physicians were influenced more by contrary advice from a referenced decision aid than an anonymous specialist. Whether this holds for advice from a respected specialist or in actual practice remains to be studied.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Over-reliance or under-reliance on ai is wrong when decoupled from considerations on the relative informativeness of the ai.
“over-reliance or under-reliance on ai, regard a general attitude towards support systems, which is wrong when decoupled from considerations on the relative informativeness of the ai39,41,42.”
From Experimental evidence of effective human–AI collaboration in medical decision-making (2022) · cites this paper as [41]
The Discussion says disregarding advice solely on the basis of confidence may be ill-advised, and cautions that the management decision studied had no indisputably correct answer. This supports caution about judging reliance in isolation, but the paper does not establish the broader claim that both over- and under-reliance on AI are wrong unless assessed against the AI’s relative informativeness. (LLM, conf=0.78)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
R: A Language and Environment for Statistical Computing
R Core Team (2000) · doi:10.32614/r.manuals · Cited by 355,811 · see citing works
A Prediction Rule to Identify Low-Risk Patients with Community-Acquired Pneumonia
Fine, M. J., Auble, T. E., Yealy, D. M., Hanusa, B. H., Weissfeld, L. A., Singer, D. E., Coley, C. M., Marrie, T. J., & Kapoor, W. N. (1997) · New England Journal of Medicine · doi:10.1056/nejm199701233360402 · Cited by 4,560 · see citing works
Lim, W. S. (2003) · Thorax · doi:10.1136/thorax.58.5.377 · Cited by 3,066 · see citing works
Garg, A. X., Adhikari, N. K. J., McDonald, H., Rosas-Arellano, M. P., Devereaux, P. J., Beyene, J., Sam, J., & Haynes, R. B. (2005) · JAMA · doi:10.1001/jama.293.10.1223 · Cited by 2,936 · see citing works
Electronic health records in ambulatory care—a national survey of physicians
DesRoches, C. M., Campbell, E. G., Rao, S. R., Donelan, K., Ferris, T. G., Jha, A., Kaushal, R., Levy, D. E., Rosenbaum, S., Shields, A. E., et al. (2008) · New England Journal of Medicine · doi:10.1056/nejmsa0802005 · Cited by 1,111 · see citing works
Opinion Leaders vs Audit and Feedback to Implement Practice Guidelines
Lomas, J. (1991) · JAMA · doi:10.1001/jama.265.17.2202 · Cited by 741 · see citing works
Evaluating informatics applications – clinical decision support systems literature review
Kaplan, B. (2001) · International Journal of Medical Informatics · doi:10.1016/s1386-5056(01)00183-6 · Cited by 350 · see citing works
Diagnostic Yield of the Autopsy in a University Hospital and a Community Hospital
Landefeld, C. S., Chren, M., Myers, A. M., et al. (1988) · New England Journal of Medicine · doi:10.1056/nejm198805123181906 · Cited by 254 · see citing works
Lomas, J., Enkin, M., Anderson, G., Hannah, W. J., Vayda, E., & Singer, J. (1991) · PubMed · Cited by 223 · see citing works
Enhancement of Clinicians' Diagnostic Reasoning by Computer-Based Consultation
Friedman, C. P., Elstein, A. S., Wolf, F. M., Murphy, G. C., Franz, T. M., Heckerling, P. S., Fine, P. L., Miller, T. M., & Abraham, V. (1999) · JAMA · doi:10.1001/jama.282.19.1851 · Cited by 219 · see citing works
Enhanced Interpretation of Diagnostic Images
Getty, D. J., Pickett, R. M., Orsi, C. D., et al. (1988) · Investigative Radiology · doi:10.1097/00004424-198804000-00002 · Cited by 166 · see citing works
A survey of factors affecting clinician acceptance of clinical decision support
Sittig, D. F., Krall, M., Dykstra, R. H., Russell, A., & Chin, H. L. (2006) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-6-6 · Cited by 138 · see citing works
Patients Derogate Physicians Who Use a Computer-Assisted Diagnostic Aid
Arkes, H. R., Shaffer, V. A., & Medow, M. A. (2007) · Medical Decision Making · doi:10.1177/0272989x06297391 · Cited by 103 · see citing works
Self-citationDreiseitl, S., & Binder, M. (2005) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2004.07.007 · Cited by 102 · see citing works
Can computer aided teaching packages improve clinical care in patients with acute abdominal pain?
de Dombal, F. T., Dallos, V., & McADAM, W. A. F. (1991) · BMJ · doi:10.1136/bmj.302.6791.1495 · Cited by 68 · see citing works
Hemodynamic assessment in managing the critically iii: is physician confidence warranted?
Dawson, N. V., Connors, A. F., Speroff, T., Kemka, A., Shaw, P., & Arkes, H. R. (1993) · Medical Decision Making · doi:10.1177/0272989x9301300314 · Cited by 52 · see citing works
Self-citationApplying the acute ischemic heart disease predictive instrument.
Corey, G. A., & Merenstein, J. H. (1987) · PubMed · Cited by 49 · see citing works
Berner, E. S., Maisiak, R. S., Heuderbert, G. R., & Young, K. R. (2003) · PubMed · Cited by 37 · see citing works
Medical Information Management: Improving the Transfer of Research Results to Presurgical Evaluation
Chase, C. R., Vacek, P. M., Shinozaki, T., et al. (1983) · Medical Care · doi:10.1097/00005650-198304000-00004 · Cited by 28 · see citing works
A diagnostic support system in general practice: Is it feasible?
Ridderikhoff, J., & van Herk, E. (1997) · International Journal of Medical Informatics · doi:10.1016/s1386-5056(97)00022-1 · Cited by 22 · see citing works
Arkes, H. R., Shaffer, V. A., & Medow, M. A. (2008) · Medical Decision Making · doi:10.1177/0272989x07313280 · Cited by 18 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Experimental evidence of effective human–AI collaboration in medical decision-making
Reverberi, C., Rigon, T., Solari, A., Hassan, C., Cherubini, P., GI Genius CADx Study Group, Antonelli, G., Awadie, H., Bernhofer, S., Carballal, S., et al. (2022) · Scientific Reports · doi:10.1038/s41598-022-18751-2 · Cited by 235 · see citing works
What People Want From Their Professionals: Attitudes Toward Decision‐making Strategies
Eastwood, J., Snook, B., & Luther, K. (2012) · Journal of Behavioral Decision Making · doi:10.1002/bdm.741 · Cited by 97 · see citing works
Developing an intervention around referral and admissions to intensive care: a mixed-methods study
Bassford, C., Griffiths, F., Svantesson, M., Ryan, M., Krucien, N., Dale, J., Rees, S., Rees, K., Ignatowicz, A., Parsons, H., et al. (2019) · Health Services and Delivery Research · doi:10.3310/hsdr07390 · Cited by 27 · see citing works
- Authors
- Medow, M. A., Arkes, H. R., & Shaffer, V. A.
- Affiliations
- Boston University (US)
- The Ohio State University (US)
- Wichita State University (US)
- Corresponding author
- Medow, Mitchell A.
- Venue
- Journal of General Internal Medicine
- Published
- 2010-01-30
- Volume
- 25
- Issue
- 4
- Pages
- 316-320
- ISSN
- 0884-8734 (ISSN-L)
- 1525-1497
- PMC
- PMC2842548
- OpenAlex
- W2089253993
- Cited by
- 12
- References
- 21
- Citations by year
- 2022: 1 · 2021: 1 · 2020: 1 · 2019: 1 · 2017: 1 · 2015: 1 · 2014: 1 · 2012: 2
- FWCI
- 2.85
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Journal of General Internal Medicine (publishedVersion)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Science Foundation (0326468)
- Topics
- Health Sciences › Health Professions › General Health Professions › Patient-Provider Communication in Healthcare
- Health Sciences › Medicine › Family Practice › Clinical Reasoning and Diagnostic Skills
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Palliative Care and End-of-Life Issues
- Keywords
- decision aids, physician decision-making, internal medicine residents, community-acquired pneumonia, intensive care unit admission, expert opinion, physician confidence, clinical decision support
- MeSH terms
- Analysis of Variance, Attitude of Health Personnel, Choice Behavior, Humans, Internal Medicine, Internship and Residency, Medicine, Ohio, Pneumonia, Pulmonary Medicine, Confidence Intervals, Odds Ratio, Community-Acquired Infections, Decision Support Systems, Clinical
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
