The DOI matches its Crossref record; 62 of the 62 references it cites were found in a registry or an index.
Development and Validation of a Tool to Identify Patients With Type 2 Diabetes at High Risk of Hypoglycemia-Related Emergency Department or Hospital Use
Open accessAbstract
Importance: Hypoglycemia-related emergency department (ED) or hospital use among patients with type 2 diabetes (T2D) is clinically significant and possibly preventable. Objective: To develop and validate a tool to categorize risk of hypoglycemic-related utilization in patients with T2D. Design, Setting, and Participants: Using recursive partitioning with a split-sample design, we created a classification tree based on potential predictors of hypoglycemia-related ED or hospital use. The resulting model was transcribed into a tool for practical application and tested in 1 internal and 2 fully independent, external samples. Development and internal testing was conducted in a split sample of 206 435 patients with T2D from Kaiser Permanente Northern California (KPNC), an integrated health care system. The tool was externally tested in 1 335 966 Veterans Health Administration and 14 972 Group Health Cooperative patients with T2D. Exposures: Based on a literature review, we identified 156 candidate predictor variables (prebaseline exposures) using data collected from electronic medical records. Main Outcomes and Measures: Hypoglycemia-related ED or hospital use during 12 months of follow-up. Results: The derivation sample (n = 165 148) had a mean (SD) age of 63.9 (13.0) years and included 78 576 (47.6%) women. The crude annual rate of at least 1 hypoglycemia-related ED or hospital encounter in the KPNC derivation sample was 0.49%. The resulting hypoglycemia risk stratification tool required 6 patient-specific inputs: number of prior episodes of hypoglycemia-related utilization, insulin use, sulfonylurea use, prior year ED use, chronic kidney disease stage, and age. We categorized the predicted 12-month risk of any hypoglycemia-related utilization as high (>5%), intermediate (1%-5%), or low (<1%). In the internal validation sample, 2.0%, 10.7%, and 87.3% were categorized as high, intermediate, and low risk, respectively, with observed 12-month hypoglycemia-related utilization rates of 6.7%, 1.4%, and 0.2%, respectively. There was good discrimination in the internal validation KPNC sample (C statistic = 0.83) and both external validation samples (Veterans Health Administration: C statistic = 0.81; Group Health Cooperative: C statistic = 0.79). Conclusions and Relevance: This hypoglycemia risk stratification tool categorizes the 12-month risk of hypoglycemia-related utilization in patients with T2D using only 6 inputs. This tool could facilitate targeted population management interventions, potentially reducing hypoglycemia risk and improving patient safety and quality of life.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Older adults can also be stratified for future risk for hypoglycemia with validated risk calculators (e.g., Kaiser Hypoglycemia Model) (34) and with consideration of hypoglycemia risk factors (Table 6.5).”
From 13. Older Adults: Standards of Care in Diabetes—2024 (2024) · cites this paper as [34]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Older adults can also be stratified for future risk for hypoglycemia with validated risk calculators (e.g., Kaiser Hypoglycemia Model for adults with type 2 diabetes) (40) and with consideration of hypoglycemia risk factors (Table 6.5).”
From 13. Older Adults: Standards of Care in Diabetes—2026 (2026) · cites this paper as [40]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Validated tools have been developed to estimate hypoglycemia risk using predominantly electronic health record data (101-103).”
From 6. Glycemic Goals, Hypoglycemia, and Hyperglycemic Crises: Standards of Care in Diabetes-2026. (2026) · cites this paper as [101]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(1990) · Choice Reviews Online · doi:10.5860/choice.27-3338 · Cited by 35,305
Classification and Regression Trees.
Ryzin, J. V., Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1986) · Journal of the American Statistical Association · doi:10.2307/2288003 · Cited by 21,030
Levey, A. S., Bosch, J. P., Lewis, J. B., Greene, T., Rogers, N., Roth, D., & for the Modification of Diet in Renal Disease Study Group* (1999) · Annals of Internal Medicine · doi:10.7326/0003-4819-130-6-199903160-00002 · Cited by 15,079
Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating
Steyerberg, E. W. (2019) · Statistics for Biology and Health · doi:10.1007/978-3-030-16399-0 · Cited by 3,281
Moons, K. G. M., de Groot, J. A. H., Bouwmeester, W., Vergouwe, Y., Mallett, S., Altman, D. G., Reitsma, J. B., & Collins, G. S. (2014) · PLoS Medicine · doi:10.1371/journal.pmed.1001744 · Cited by 2,018
Changes in diabetes-related complications in the United States, 1990-2010
Gregg, E. W., Li, Y., Wang, J., Rios Burrows, N., Ali, M. K., Rolka, D., Williams, D. E., & Geiss, L. (2014) · New England Journal of Medicine · doi:10.1056/nejmoa1310799 · Cited by 1,749
Emergency hospitalizations for adverse drug events in older Americans.
Budnitz, D., Lovegrove, M., Shehab, N., & Richards, C. (2011) · New England Journal of Medicine · doi:10.1056/nejmsa1103053 · Cited by 1,748
Fonarow, G. C. (2005) · JAMA · doi:10.1001/jama.293.5.572 · Cited by 1,617
Severe hypoglycemia and risks of vascular events and death
Zoungas, S., Patel, A., Chalmers, J., de Galan, B. E., Li, Q., Billot, L., Woodward, M., Ninomiya, T., Neal, B., MacMahon, S., et al. (2010) · New England Journal of Medicine · doi:10.1056/nejmoa1003795 · Cited by 1,471
Prognosis and prognostic research: validating a prognostic model
Altman, D. G., Vergouwe, Y., Royston, P., & Moons, K. G. M. (2009) · BMJ · doi:10.1136/bmj.b605 · Cited by 1,436
Seaquist, E. R., Anderson, J., Childs, B., Cryer, P., Dagogo-Jack, S., Fish, L., Heller, S. R., Rodriguez, H., Rosenzweig, J., & Vigersky, R. (2013) · Diabetes Care · doi:10.2337/dc12-2480 · Cited by 1,389
Prognosis and prognostic research: Developing a prognostic model
Royston, P., Moons, K. G. M., Altman, D. G., & Vergouwe, Y. (2009) · BMJ · doi:10.1136/bmj.b604 · Cited by 1,172
Assessing the Generalizability of Prognostic Information
Justice, A. C., Covinsky, K. E., & Berlin, J. A. (1999) · Annals of Internal Medicine · doi:10.7326/0003-4819-130-6-199903160-00016 · Cited by 1,160
Defining and Reporting Hypoglycemia in Diabetes
American Diabetes Association Workgroup on Hypoglycemia (2005) · Diabetes Care · doi:10.2337/diacare.28.5.1245 · Cited by 1,146
Hypoglycemic episodes and risk of dementia in older patients with type 2 diabetes mellitus
Whitmer, R. A. (2009) · JAMA · doi:10.1001/jama.2009.460 · Cited by 1,128
Prognosis and prognostic research: application and impact of prognostic models in clinical practice
Moons, K. G. M., Altman, D. G., Vergouwe, Y., & Royston, P. (2009) · BMJ · doi:10.1136/bmj.b606 · Cited by 934
Walter, L. C. (2001) · JAMA · doi:10.1001/jama.285.23.2987 · Cited by 673
Increased mortality of patients with diabetes reporting severe hypoglycemia
McCoy, R. G., Van Houten, H. K., Ziegenfuss, J. Y., Shah, N. D., Wermers, R. A., & Smith, S. A. (2012) · Diabetes Care · doi:10.2337/dc11-2054 · Cited by 539
Huang, E. S., Laiteerapong, N., Liu, J. Y., John, P. M., Moffet, H. H., & Karter, A. J. (2014) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2013.12956 · Cited by 501
Self-citationHypoglycaemia in Type 2 diabetes
Amiel, S. A., Dixon, T., Mann, R., & Jameson, K. (2008) · Diabetic Medicine · doi:10.1111/j.1464-5491.2007.02341.x · Cited by 472
Seaquist, E. R., Anderson, J., Childs, B., Cryer, P., Dagogo-Jack, S., Fish, L., Heller, S. R., Rodriguez, H., Rosenzweig, J., & Vigersky, R. (2013) · The Journal of Clinical Endocrinology & Metabolism · doi:10.1210/jc.2012-4127 · Cited by 412
Lipska, K. J., Ross, J. S., Wang, Y., Inzucchi, S. E., Minges, K. E., Karter, A. J., Huang, E. S., Desai, M. M., Gill, T. M., & Krumholz, H. M. (2014) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2014.1824 · Cited by 394
Self-citationGlycemic Control, Complications, and Death in Older Diabetic Patients
Huang, E. S., Liu, J. Y., Moffet, H. H., John, P. M., & Karter, A. J. (2011) · Diabetes Care · doi:10.2337/dc10-2377 · Cited by 347
Self-citationBarriers to Insulin Initiation
Karter, A. J., Subramanian, U., Saha, C., Crosson, J. C., Parker, M. M., Swain, B. E., Moffet, H. H., & Marrero, D. G. (2010) · Diabetes Care · doi:10.2337/dc09-1184 · Cited by 255
Self-citationHbA1c and Risk of Severe Hypoglycemia in Type 2 Diabetes
Lipska, K. J., Warton, E. M., Huang, E. S., Moffet, H. H., Inzucchi, S. E., Krumholz, H. M., & Karter, A. J. (2013) · Diabetes Care · doi:10.2337/dc13-0610 · Cited by 254
Self-citationValidation of ICD-9-CM coding algorithm for improved identification of hypoglycemia visits
Ginde, A. A., Blanc, P. G., Lieberman, R. M., & Camargo, C. A. (2008) · BMC Endocrine Disorders · doi:10.1186/1472-6823-8-4 · Cited by 234
Correlates of Quality of Life in Older Adults With Diabetes
Laiteerapong, N., Karter, A. J., Liu, J. Y., Moffet, H. H., Sudore, R., Schillinger, D., John, P. M., & Huang, E. S. (2011) · Diabetes Care · doi:10.2337/dc10-2424 · Cited by 227
Self-citationExhaustion Of Food Budgets At Month’s End And Hospital Admissions For Hypoglycemia
Seligman, H. K., Bolger, A. F., Guzman, D., López, A., & Bibbins-Domingo, K. B. (2014) · Health Affairs · doi:10.1377/hlthaff.2013.0096 · Cited by 200
Several methods to assess improvement in risk prediction models: Extension to survival analysis
Chambless, L. E., Cummiskey, C., & Cui, G. (2010) · Statistics in Medicine · doi:10.1002/sim.4026 · Cited by 194
Logistic Regression Diagnostics
Meurer, W. J., & Tolles, J. (2017) · JAMA · doi:10.1001/jama.2016.20441 · Cited by 186
Hypoglycemia as a manifestation of sepsis
Miller, S. I., Wallace, R. J., Musher, D. M., Septimus, E. J., Kohl, S., & BAUGHN, R. E. (1980) · The American Journal of Medicine · doi:10.1016/0002-9343(80)90250-8 · Cited by 177
Sarkar, U., Karter, A. J., Liu, J. Y., Moffet, H. H., Adler, N. E., & Schillinger, D. (2010) · Journal of General Internal Medicine · doi:10.1007/s11606-010-1389-7 · Cited by 173
Self-citationStahn, A., Pistrosch, F., Ganz, X., Teige, M., Koehler, C., Bornstein, S., & Hanefeld, M. (2014) · Diabetes Care · doi:10.2337/dc13-0600 · Cited by 173
Self‐reported non‐severe hypoglycaemic events in Europe
Östenson, C. G., Geelhoed-Duijvestijn, P. H. L. M., Lahtela, J. T., Weitgasser, R., Jensen, M. M., & Pedersen‐Bjergaard, U. (2013) · Diabetic Medicine · doi:10.1111/dme.12261 · Cited by 167
Assessing Potential Glycemic Overtreatment in Persons at Hypoglycemic Risk
Tseng, C., Soroka, O., Maney, M., Aron, D. C., & POGACH, L. M. (2013) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2013.12963 · Cited by 161
Intensive treatment and severe hypoglycemia among adults with type 2 diabetes
McCoy, R. G., Lipska, K. J., Yao, X., Ross, J. S., Montori, V. M., & Shah, N. D. (2016) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2016.2275 · Cited by 143
Self-citationMoffet, H. H., Adler, N. E., Schillinger, D., Ahmed, A. T., Laraia, B. A., Selby, J. V., Neugebauer, R. S., Liu, J. Y., Parker, M. M., Warton, M., et al. (2008) · International Journal of Epidemiology · doi:10.1093/ije/dyn040 · Cited by 133
Self-citationSignorovitch, J. E., Macaulay, D., Diener, M., Yan, Y., Wu, E. Q., Gruenberger, J. ‐., & Frier, B. M. (2013) · Diabetes, Obesity and Metabolism · doi:10.1111/dom.12031 · Cited by 125
Understanding the construct of fear of hypoglycemia in pediatric type 1 diabetes
Shepard, J. A., Vajda, K., Nyer, M., Clarke, W., & Gonder-Frederick, L. (2014) · Journal of Pediatric Psychology · doi:10.1093/jpepsy/jsu068 · Cited by 82
Basu (2017) · doi:10.1097/mlr.0000000000000728 · Cited by 77
Bonds, D. E., Miller, M. E., Dudl, J., Feinglos, M. N., Ismail‐Beigi, F., Malozowski, S., Seaquist, E. R., Simmons, D. L., & Sood, A. (2012) · BMC Endocrine Disorders · doi:10.1186/1472-6823-12-5 · Cited by 76
Berkowitz, S. A., Karter, A. J., Lyles, C. R., Liu, J. Y., Schillinger, D., Adler, N. E., Moffet, H. H., & Sarkar, U. (2014) · Journal of Health Care for the Poor and Underserved · doi:10.1353/hpu.2014.0106 · Cited by 67
Self-citationHypoglycemia Associated With Hospitalization and Adverse Events in Older People
Majumdar, S. R., Hemmelgarn, B. R., Lin, M., McBrien, K., Manns, B., & Tonelli, M. A. (2013) · Diabetes Care · doi:10.2337/dc13-0523 · Cited by 66
Frier, B. M., Jensen, M. M., & Chubb, B. (2015) · Diabetic Medicine · doi:10.1111/dme.12878 · Cited by 54
The social burden of hypoglycemia in the elderly
Nicolucci, A., Pintaudi, B., Rossi, M. C., Messina, R., Dotta, F., Frontoni, S., Caputo, S., & Lauro, R. (2015) · Acta Diabetologica · doi:10.1007/s00592-015-0717-0 · Cited by 50
Rondags, S. M. P. A., de Wit, M., Twisk, J. W., & Snoek, F. J. (2016) · Diabetes Care · doi:10.2337/dc16-1614 · Cited by 45
Balancing Hypoglycemia and Glycemic Control
POGACH, L. M., & Aron, D. C. (2010) · JAMA · doi:10.1001/jama.2010.655 · Cited by 41
Olsen, S. E., Bjørgaas, M. R., Åsvold, B. O., Sand, T., Stjern, M., Frier, B. M., & Nilsen, K. B. (2015) · Diabetes Care · doi:10.2337/dc15-1469 · Cited by 40
The Other Side of Quality Improvement in Diabetes for Seniors
POGACH, L. M., & Aron, D. C. (2012) · Archives of Internal Medicine · doi:10.1001/archinternmed.2012.4392 · Cited by 37
Senior, P. A., Bellin, M. D., Alejandro, R., Yankey, J. W., Clarke, W. R., Qidwai, J. C., Schwieger, T., Eggerman, T. L., Robien, M. A., & Rickels, M. R. (2015) · Diabetes Technology & Therapeutics · doi:10.1089/dia.2014.0289 · Cited by 33
Miller, D. R., & POGACH, L. M. (2008) · Journal of Diabetes Science and Technology · doi:10.1177/193229680800200105 · Cited by 30
Self-citationHypoglycemia as an indicator of good diabetes care:
Rodríguez‐Gutiérrez, R., Lipska, K. J., McCoy, R. G., Ospina, N. S., Ting, H. H., Montori, V. M., & Hypoglycemia as a Quality Measure in Diabetes Study Group (2016) · BMJ · doi:10.1136/bmj.i1084 · Cited by 30
Self-citationSo Much Insulin, So Much Hypoglycemia
Lee, S. J. (2014) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2013.13307 · Cited by 27
Elliott, D., Rankin, D., Jacques, R., Lawton, J., Emery, C., Campbell, M. J., Dixon, S., & Heller, S. R. (2014) · Diabetic Medicine · doi:10.1111/dme.12621 · Cited by 27
Rodríguez‐Gutiérrez, R., Ospina, N. S., McCoy, R. G., Lipska, K. J., Shah, N. D., Montori, V. M., & for the Hypoglycemia as a Quality Measure in Diabetes Study Group (2016) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2016.5046 · Cited by 25
Self-citationRondags, S. M. P. A., de Wit, M., & Snoek, F. J. (2015) · Diabetic Medicine · doi:10.1111/dme.12876 · Cited by 22
Hypoglycemia and Adverse Outcomes: Marker or Mediator?
Lipska, K. J., & Kosiborod, M. (2011) · Reviews in Cardiovascular Medicine · doi:10.3909/ricm0586 · Cited by 19
Self-citationEvidence for continuous glucose monitoring: sufficient for reimbursement?
Heinemann, L., & DeVries, J. H. (2013) · Diabetic Medicine · doi:10.1111/dme.12341 · Cited by 14
Rondags, S. M. P. A., de Wit, M., van Tulder, M. W., Diamant, M., & Snoek, F. J. (2015) · BMC Endocrine Disorders · doi:10.1186/s12902-015-0035-0 · Cited by 13
Meulstee, M., Whittemore, R., & Watts, S. A. (2015) · The Science of Diabetes Self-Management and Care · doi:10.1177/0145721715606805 · Cited by 8
Improving Safety of Diabetes Mellitus Management
Lipska, K. J. (2014) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2014.3290 · Cited by 5
Self-citationPHYTOPLASMAS IN POME FRUIT TREES: UPDATE OF THEIR PRESENCE AND THEIR VECTORS IN BELGIUM.
Peusens, G., Jonghe, K. D., De Roo, I., Steyer, S., Olivier, T., Fauche, F., Rys, F., Bylemans, D., & Beliën, T. (2015) · PubMed · Cited by 1
Flagged first, then by citations
Cited by
Sorted by citations
Heart Disease and Stroke Statistics—2021 Update: A Report From the American Heart Association
Virani, S. S., Alonso, A., Aparicio, H. J., Benjamin, E. J., Bittencourt, M. S., Callaway, C. W., Carson, A. P., Chamberlain, A. M., Cheng, S., Delling, F. N., et al. (2021) · Circulation · doi:10.1161/cir.0000000000000950 · Cited by 5,784
6. Glycemic Goals and Hypoglycemia: Standards of Care in Diabetes—2024
ElSayed, N. A., Aleppo, G., Bannuru, R. R., Bruemmer, D., Collins, B. S., Ekhlaspour, L., Hilliard, M. E., Johnson, E. L., Khunti, K., Lingvay, I., et al. (2023) · Diabetes Care · doi:10.2337/dc24-s006 · Cited by 574
6. Glycemic Goals and Hypoglycemia: Standards of Care in Diabetes—2025
American Diabetes Association Professional Practice Committee, ElSayed, N. A., McCoy, R. G., Aleppo, G., Balapattabi, K., Beverly, E. A., Briggs Early, K., Bruemmer, D., Echouffo-Tcheugui, J. B., Ekhlaspour, L., et al. (2025) · Diabetes Care · doi:10.2337/dc25-s006 · Cited by 440
11. Microvascular Complications and Foot Care: Standards of Medical Care in Diabetes—2019
American Diabetes Association (2018) · Diabetes Care · doi:10.2337/dc19-s011 · Cited by 416
11. Microvascular Complications and Foot Care: Standards of Medical Care in Diabetes—2021
American Diabetes Association (2021) · Diabetes Care · doi:10.2337/dc21-s011 · Cited by 395
Metadata
- Authors
- Karter, A. J., Warton, E. M., Lipska, K. J., Ralston, J. D., Moffet, H. H., Jackson, G. G., Huang, E. S., & Miller, D. R.
- Affiliations
- University of California, San Francisco (US)
- Kaiser Permanente (US)
- Yale University (US)
- Kaiser Permanente Washington Health Research Institute (US)
- University of Chicago (US)
- Veterans Health Administration (US)
- Corresponding author
- Karter, Andrew J.
- ORCID
- Venue
- JAMA Internal Medicine
- Publisher
- American Medical Association (AMA)
- Published
- 2017-10-01
- Volume
- 177
- Issue
- 10
- Pages
- 1461
- ISSN
- 2168-6106 (ISSN-L)
- PMC
- PMC5624849
- OpenAlex
- W2749343537
- Semantic Scholar
- 865993e54ca35e1a709ba058bb73942c855a57a4
- Cited by
- 150
- References
- 62
- Citations by year
- 2026: 7 · 2025: 18 · 2024: 21 · 2023: 18 · 2022: 16 · 2021: 17 · 2020: 19 · 2019: 17 · 2018: 14 · 2017: 3
- FWCI
- 11.47
- Type
- journal article
- Language
- en
- Open access
- green
- europepmc.org
- Locations
- JAMA Internal Medicine (publishedVersion)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Institutes of Health (UL1TR001863)
- National Institute on Aging (K23AG048359)
- National Institute of Diabetes and Digestive and Kidney Diseases (K24DK105340, R01DK103721, R01DK081796, P30DK092949, P30DK092924)
- National Center for Advancing Translational Sciences
- Topics
- Health Sciences › Medicine › Endocrinology, Diabetes and Metabolism › Diabetes Management and Research
- Health Sciences › Medicine › Endocrinology, Diabetes and Metabolism › Diabetes Treatment and Management
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Patient Safety and Medication Errors
- Keywords
- type 2 diabetes, hypoglycemia, hospital utilization, risk stratification tools, classification trees, insulin use, chronic kidney disease, risk prediction model, external validation, Kaiser Permanente, Veterans Health Administration, Group Health Cooperative, electronic health records, hypoglycemia prevention
- MeSH terms
- Diabetes Mellitus, Type 2, Emergency Service, Hospital, Female, Follow-Up Studies, Humans, Hypoglycemia, Hypoglycemic Agents, Male, Prospective Studies, Quality of Life, Risk Factors, United States, Incidence, Risk Assessment, Delivery of Health Care, Integrated, Electronic Health Records
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
