Failure-to-Rescue: Comparing Definitions to Measure Quality of Care
Abstract
OBJECTIVES: Use of failure-to-rescue (FTR) as an indicator of hospital quality has increased over the past decade, but recent authors have used different sets of complications and deaths to define this measure. This study examines the reliability and validity of different FTR measures currently in use. RESEARCH DESIGN: We studied 3 definitions: (1) "original" FTR (using all deaths); (2) FTR-N, a "nursing sensitive" definition that uses only specific complications and deaths; and (3) FTR-A [another restricted definition of FTR used by Agency for Healthcare Research and Quality (AHRQ) for analyzing Healthcare Cost and Utilization Project (HCUP) data]. Each FTR measure was applied to 403,679 general surgical patients across 1567 hospitals reported in 1999-2000 Medicare MEDPAR data. RESULTS: Although FTR used all deaths, FTR-N and FTR-A definitions omitted 49% and 42% of deaths, respectively. Reliability was better for FTR than FTR-A or FTR-N (rho = 0.32 vs. 0.18 vs. 0.18, respectively). VALIDITY: Hospitals ranked by adjusted mortality were highly correlated with FTR (Kendall's tau = 0.83) and less correlated with FTR-A (tau = 0.43) and FTR-N (tau = 0.41). Adjusting for patient characteristics, all FTR measures showed strong associations with bed-to-nurse ratio, nursing mix, teaching status, and hospital size; however, hospital "high technology" was not as well associated with FTR-N. CONCLUSIONS: For general surgery, more limited definitions used by FTR-N and FTR-A omit over 40% of deaths, display less reliability, and may have more questionable validity than the original FTR measure. We encourage analysts to use the original FTR definition that uses all deaths when analyzing hospital quality of care.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Content validity and translation quality indices for all items used in this study were classified as 'good' or better (content validity index > 0.6).
“Content validity and translation quality indices for all items used in this study were classified as 'good' or better (content validity index > 0.6). 26”
From Nurses’ Shift Length and Overtime Working in 12 European Countries: The Association With Perceived Quality of Care and Patient Safety (2014) · cites this paper as [26]
Cites
Rozeboom, W. W., Lord, F. M., Novick, M. R., & Birnbaum, A. (1969) · American Educational Research Journal · doi:10.2307/1162101 · Cited by 8,012 · see citing works
Nonparametric Statistical Methods
(1999) · Technometrics · doi:10.1080/00401706.1999.10485954 · Cited by 6,117 · see citing works
Hospital Nurse Staffing and Patient Mortality, Nurse Burnout, and Job Dissatisfaction
Aiken, L. H. (2002) · JAMA · doi:10.1001/jama.288.16.1987 · Cited by 5,226 · see citing works
Nurse-Staffing Levels and the Quality of Care in Hospitals
Needleman, J., Buerhaus, P. I., Mattke, S., Stewart, M. T., & Zelevinsky, K. (2002) · New England Journal of Medicine · doi:10.1056/nejmsa012247 · Cited by 2,246 · see citing works
Educational Levels of Hospital Nurses and Surgical Patient Mortality
Aiken, L. H. (2003) · JAMA · doi:10.1001/jama.290.12.1617 · Cited by 1,726 · see citing works
Statistical theories of mental test scores
Lord, F. M., Novick, M. R., & Birnbaum, A. (1974) · Cited by 1,174 · see citing works
Hospital and Patient Characteristics Associated With Death After Surgery
Silber, J. H., Williams, S. V., Krakauer, H., & Schwartz, S. (1992) · Medical Care · doi:10.1097/00005650-199207000-00004 · Cited by 954 · see citing works
Self-citationThe society of thoracic surgeons: 30-day operative mortality and morbidity risk models
Shroyer, A. L., Coombs, L. P., Peterson, E. D., Eiken, M. C., DeLong, E. R., Chen, A., Ferguson, T. B., Grover, F. L., & Edwards, F. H. (2003) · The Annals of Thoracic Surgery · doi:10.1016/s0003-4975(03)00179-6 · Cited by 661 · see citing works
Nonparametric Statistical Methods
Ziegel, E. R., Hollander, M., & Wolfe, D. A. (1999) · Technometrics · doi:10.2307/1271371 · Cited by 610 · see citing works
Adverse Sedation Events in Pediatrics: A Critical Incident Analysis of Contributing Factors
Coté, C. J., Notterman, D. A., Karl, H. W., Weinberg, J. A., & McCloskey, C. A. (2000) · PEDIATRICS · doi:10.1542/peds.105.4.805 · Cited by 579 · see citing works
Hospital Characteristics and Mortality Rates
Hartz, A. J., Krakauer, H., Kuhn, E. M., Young, M., Jacobsen, S. J., Gay, G., Muenz, L., Katzoff, M., Bailey, R. C., & Rimm, A. A. (1989) · New England Journal of Medicine · doi:10.1056/nejm198912213212506 · Cited by 513 · see citing works
On The So-Called “Huber Sandwich Estimator” and “Robust Standard Errors”
Freedman, D. A. (2006) · The American Statistician · doi:10.1198/000313006x152207 · Cited by 498 · see citing works
Lang, T. A., Hodge, M. B., Olson, V., Romano, P. S., & Kravitz, R. L. (2004) · JONA The Journal of Nursing Administration · doi:10.1097/00005110-200407000-00005 · Cited by 429 · see citing works
Self-citationMeasuring hospital performance
Grosskopf, S., & Valdmanis, V. G. (1987) · Journal of Health Economics · doi:10.1016/0167-6296(87)90001-4 · Cited by 409 · see citing works
Clarke, S. P., & Aiken, L. H. (2003) · AJN American Journal of Nursing · doi:10.1097/00000446-200301000-00020 · Cited by 369 · see citing works
Silber, J. H. (1995) · JAMA · doi:10.1001/jama.1995.03530040045039 · Cited by 316 · see citing works
Self-citationSeverity-Adjusted Mortality and Length of Stay in Teaching and Nonteaching Hospitals
Rosenthal, G. E. (1997) · JAMA · doi:10.1001/jama.1997.03550060061037 · Cited by 290 · see citing works
A National Profile Of Patient Safety In U.S. Hospitals
Romano, P. S., Geppert, J., Davies, S. M., Miller, M. R., Elixhauser, A., & McDonald, K. M. (2003) · Health Affairs · doi:10.1377/hlthaff.22.2.154 · Cited by 267 · see citing works
Self-citationSilber, J. H., Rosenbaum, P. R., & Ross, R. N. (1995) · Journal of the American Statistical Association · doi:10.1080/01621459.1995.10476483 · Cited by 186 · see citing works
Self-citationAnesthesiologist Direction and Patient Outcomes
Silber, J. H., Kennedy, S. K., Even‐Shoshan, O., Chen, W., Koziol, L. F., Showan, A. M., & Longnecker, D. E. (2000) · Anesthesiology · doi:10.1097/00000542-200007000-00026 · Cited by 172 · see citing works
Self-citationUsing administrative data to screen hospitals for high complication rates.
Iezzoni, L. I., Daley, J., Heeren, T., Foley, S. M., Hughes, J. S., Fisher, E. S., Duncan, C. C., & Coffman, G. A. (1994) · PubMed · Cited by 168 · see citing works
Anesthesiologist Board Certification and Patient Outcomes
Silber, J. H., Kennedy, S. K., Even‐Shoshan, O., Chen, W., Mosher, R. E., Showan, A. M., & Longnecker, D. E. (2002) · Anesthesiology · doi:10.1097/00000542-200205000-00004 · Cited by 151 · see citing works
Self-citationEvaluating the Patient Safety Indicators
Rosen, A. K., Rivard, P. E., Zhao, S., Loveland, S. A., Tsilimingras, D., Christiansen, C. L., Elixhauser, A., & Romano, P. S. (2005) · Medical Care · doi:10.1097/01.mlr.0000173561.79742.fb · Cited by 114 · see citing works
Self-citationLongitudinal Analyses of Nurse Staffing and Patient Outcomes
Seago, J. A., Williamson, A., & Atwood, C. (2006) · JONA The Journal of Nursing Administration · doi:10.1097/00005110-200601000-00005 · Cited by 114 · see citing works
Nursing unit characteristics and patient outcomes.
Boyle, S. M. (2004) · PubMed · Cited by 102 · see citing works
Measuring outcomes of hospital care using multiple risk-adjusted indexes.
DesHarnais, S., McMahon, L. F., & Wroblewski, R. T. (1991) · PubMed Central · Cited by 97 · see citing works
Changes in Prognosis After the First Postoperative Complication
Silber, J. H., Rosenbaum, P. R., Trudeau, M. E., Chen, W., Zhang, X., Kelz, R. R., Mosher, R. E., & Even‐Shoshan, O. (2005) · Medical Care · doi:10.1097/00005650-200502000-00005 · Cited by 94 · see citing works
Self-citationNeonatal Hyperbilirubinemia and Risk of Autism Spectrum Disorders
Croen, L., Yoshida, C. K., Odouli, R., & Newman, T. B. (2005) · PEDIATRICS · doi:10.1542/peds.2004-1870 · Cited by 94 · see citing works
Sedman, A. B., Harris, J. M., Schulz, K. A., Schwalenstocker, E., Remus, D., Scanlon, M. C., & Bahl, V. (2004) · PEDIATRICS · doi:10.1542/peds.2004-1083 · Cited by 88 · see citing works
Measuring Hospital Performance
DesHarnais, S., McMahon, L. F., Wroblewski, R. T., et al. (1990) · Medical Care · doi:10.1097/00005650-199012000-00002 · Cited by 82 · see citing works
Hospital Nurse Staffing and Patient Mortality, Emotional Exhaustion, and Job Dissatisfaction
Halm, M. A., Peterson, M., KANDELS, M., Sabo, J. A., BLALOCK, M., BRADEN, R., GRYCZMAN, A., Krisko-Hagel, K. A., Larson, D., Lemay, D., et al. (2005) · Clinical Nurse Specialist · doi:10.1097/00002800-200509000-00007 · Cited by 79 · see citing works
A Spurious Correlation Between Hospital Mortality and Complication Rates
Silber, J. H., & Rosenbaum, P. R. (1997) · Medical Care · doi:10.1097/00005650-199710001-00011 · Cited by 76 · see citing works
Self-citationPredicted probabilities of hospital death as a measure of admission severity of illness.
Steen, P. M., Brewster, A. C., Bradbury, R. C., et al. (1993) · PubMed · Cited by 75 · see citing works
Silber, J. H., Rosenbaum, P. R., Williams, S. V., et al. (1997) · International Journal for Quality in Health Care · doi:10.1093/intqhc/9.3.193 · Cited by 67 · see citing works
Self-citationFailure to rescue: lessons from missed opportunities in care
Clarke, S. P. (2004) · Nursing Inquiry · doi:10.1111/j.1440-1800.2004.00210.x · Cited by 67 · see citing works
Variation in Chemotherapy Utilization in Ovarian Cancer: The Relative Contribution of Geography
Polsky, D. E., Armstrong, K., Randall, T. C., et al. (2006) · Health Services Research · doi:10.1111/j.1475-6773.2006.00596.x · Cited by 52 · see citing works
Simpson, K. R. (2005) · The Journal of Perinatal & Neonatal Nursing · doi:10.1097/00005237-200501000-00008 · Cited by 50 · see citing works
Postoperative Mortality and Pulmonary Complication Rankings
Arozullah, A. M., Henderson, W. G., Khuri, S. F., & Daley, J. (2003) · Medical Care · doi:10.1097/00005650-200308000-00011 · Cited by 43 · see citing works
McKee, M., Coles, J., & James, P. (1999) · Journal of Public Health · doi:10.1093/pubmed/21.4.453 · Cited by 37 · see citing works
Differentiating between pre-arrest and failure-to-rescue.
Ashcraft, A. S. (2004) · PubMed · Cited by 25 · see citing works
Long, K. A. (2004) · Nursing · doi:10.1097/00152193-200403000-00041 · Cited by 10 · see citing works
Failure to rescue in obstetrics.
Simpson, K. R. (2005) · PubMed · Cited by 6 · see citing works
Ashcraft, A. S. (2003) · AJN American Journal of Nursing · doi:10.1097/00000446-200305000-00006 · Cited by 2 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Griffiths, P., Dall’Ora, C., Simon, M., Ball, J., Lindqvist, R., Rafferty, A.-M., Schoonhoven, L., Tishelman, C., & Aiken, L. H. (2014) · Medical Care · doi:10.1097/mlr.0000000000000233 · Cited by 316 · see citing works
Nurse reported quality of care: a measure of hospital quality.
McHugh, M. D., & Stimpfel, A. W. (2012) · Research in Nursing & Health · doi:10.1002/nur.21503 · Cited by 231 · see citing works
Wells, C. I., Varghese, C., Boyle, L. J., McGuinness, M. J., Keane, C., O’Grady, G., Gurney, J., Koea, J., Harmston, C., & Bissett, I. P. (2023) · Annals of Surgery · doi:10.1097/sla.0000000000005650 · Cited by 28 · see citing works
The Benefits of Perioperative Screening for Sleep Apnea in Surgical Patients
Subramani, Y., Wong, J., Nagappa, M., & Chung, F. (2017) · Sleep Medicine Clinics · doi:10.1016/j.jsmc.2016.10.003 · Cited by 13 · see citing works
- Authors
- Silber, J. H., Romano, P. S., Rosen, A. K., Wang, Y., Even-Shoshan, O., & Volpp, K. G.
- Affiliations
- Boston University (US)
- University of California Davis Medical Center (US)
- United States Department of Veterans Affairs (US)
- Corresponding author
- Silber, Jeffrey H.
- Rosen, Amy K.
- Wang, Yanli
- Venue
- Medical Care
- Published
- 2007-10
- Volume
- 45
- Issue
- 10
- Pages
- 918-925
- ISSN
- 0025-7079 (ISSN-L)
- OpenAlex
- W2068198741
- Cited by
- 333
- References
- 43
- Citations by year
- 2026: 10 · 2025: 17 · 2024: 9 · 2023: 17 · 2022: 10 · 2021: 55 · 2020: 17 · 2019: 20 · 2018: 14 · 2017: 19 · 2016: 18 · 2015: 15 · 2014: 25 · 2013: 20 · 2012: 19
- FWCI
- 20.29
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Medical Care (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Heart, Lung, and Blood Institute (R01HL082637)
- Topics
- Health Sciences › Health Professions › Health Information Management › Healthcare Quality and Management
- Health Sciences › Health Professions › General Health Professions › Patient Satisfaction in Healthcare
- Health Sciences › Medicine › Emergency Medicine › Hospital Admissions and Outcomes
- Keywords
- failure to rescue, hospital quality indicators, general surgery, mortality, reliability statistics, validity, teaching hospitals, hospital size, Agency for Healthcare Research and Quality
- MeSH terms
- Medicare, Hospital Administration, Humans, Nursing Service, Hospital, Surgical Procedures, Operative, United States, Reproducibility of Results, United States Agency for Healthcare Research and Quality, Hospital Mortality, Treatment Failure, Quality Indicators, Health Care
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
