CoChat
Sign Up
Cited by 62Medical Coding and Health InformationDiabetes Management and ResearchDiabetes and associated disorders

Validation of a type 1 diabetes algorithm using electronic medical records and administrative healthcare data to study the population incidence and prevalence of type 1 diabetes in Ontario, Canada

Open access
Weisman, Alanna, Tu, Karen, Young, Jacqueline, Kumar, Matthew, Austin, Peter C, Jaakkimainen, Liisa, Lipscombe, Lorraine, Aronson, Ronnie +1 more·2020·BMJ Open Diabetes Research & Care
DOI 10.1136/bmjdrc-2020-001224PMC PMC7307536
Open this paper in CoChat — free accountOpen PDF

Abstract

Introduction: We aimed to develop algorithms distinguishing type 1 diabetes (T1D) from type 2 diabetes in adults ≥18 years old using primary care electronic medical record (EMRPC) and administrative healthcare data from Ontario, Canada, and to estimate T1D prevalence and incidence. Research design and methods: The reference population was a random sample of patients with diabetes in EMRPC whose charts were manually abstracted (n=5402). Algorithms were developed using classification trees, random forests, and rule-based methods, using electronic medical record (EMR) data, administrative data, or both. Algorithm performance was assessed in EMRPC. Administrative data algorithms were additionally evaluated using a diabetes clinic registry with endocrinologist-assigned diabetes type (n=29 371). Three algorithms were applied to the Ontario population to evaluate the minimum, moderate and maximum estimates of T1D prevalence and incidence rates between 2010 and 2017, and trends were analyzed using negative binomial regressions. Results: Of 5402 individuals with diabetes in EMRPC, 195 had T1D. Sensitivity, specificity, positive predictive value and negative predictive value for the best performing algorithms were 80.6% (75.9–87.2), 99.8% (99.7–100), 94.9% (92.3–98.7), and 99.3% (99.1–99.5) for EMR, 51.3% (44.0–58.5), 99.5% (99.3–99.7), 79.4% (71.2–86.1), and 98.2% (97.8–98.5) for administrative data, and 87.2% (81.7–91.5), 99.9% (99.7–100), 96.6% (92.7–98.7) and 99.5% (99.3–99.7) for combined EMR and administrative data. Administrative data algorithms had similar sensitivity and specificity in the diabetes clinic registry. Of 11 499 711 adults in Ontario in 2017, there were 24 789 (0.22%, minimum estimate) to 102 140 (0.89%, maximum estimate) with T1D. Between 2010 and 2017, the age-standardized and sex-standardized prevalence rates per 1000 person-years increased (minimum estimate 1.7 to 2.56, maximum estimate 7.48 to 9.86, p<0.0001). In contrast, incidence rates decreased (minimum estimate 0.1 to 0.04, maximum estimate 0.47 to 0.09, p<0.0001). Conclusions: Primary care EMR and administrative data algorithms performed well in identifying T1D and demonstrated increasing T1D prevalence in Ontario. These algorithms may permit the development of large, population-based cohort studies of T1D.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1136/bmjdrc-2020-001224

Cites

7 of 7 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Type 1 diabetes was identified using a previously validated algorithm defined by prescription of insulin without any other antihyperglycemic medications and age younger than 55 years old (sensitivity 72.8%, specificity 99.5%, positive predictive value 86%, negative predictive value 99%).

    “Type 1 diabetes was identified by a previously validated algorithm defined by prescription of insulin without any other antihyperglycemic medications and age younger than 55 years old (sensitivity 72.8%, specificity 99.5%, positive predictive value 86%, negative predictive value 99%).”

    From Impact of government-funded insulin pump programs on insulin pump use in Canada: a cross-sectional study using the National Diabetes Repository (2021) · cites this paper as [14]

    The Results (EMR data algorithms; page not available) describe validated algorithms using insulin and the absence of certain other antihyperglycemic medications. They do not describe the claimed rule requiring age younger than 55 or report the claimed set of performance figures; the best-performing EMR algorithm instead had 80.6% sensitivity, 99.8% specificity, 94.9% PPV, and 99.3% NPV. (LLM, conf=0.87)

    Experimental — accuracy not yet measured

  • In a population-based study using Ontario primary care EMR data, the proportion of adults with type 1 diabetes using insulin pumps was 32%.

    “In a population-based study using Ontario primary care EMR data, the proportion of adults with type 1 diabetes using insulin pumps was 32%.14”

    From Impact of government-funded insulin pump programs on insulin pump use in Canada: a cross-sectional study using the National Diabetes Repository (2021) · cites this paper as [14]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 33

  1. [1]Foster NC, Beck RW, Miller KM, et al State of type 1 diabetes management and outcomes from the T1D exchange in 2016-2018. Diabetes Technol Ther 2019; 21: 66– 72. 10.1089/dia.2018.0384 30657336 PMC7061293 · DOI · Paper page
  2. [2]Johnson JA, Pohar SL, Majumdar SR Health care use and costs in the decade after identification of type 1 and type 2 diabetes: a population-based study. Diabetes Care 2006; 29: 2403– 8. 10.2337/dc06-0735 17065675 · DOI
  3. [3]Diabetes Control and Complications Trial Research Group, Nathan DM, Genuth S, et al The effect of intensive treatment of diabetes on the development and progression of long-term complications in insulin-dependent diabetes mellitus. N Engl J Med 1993; 329: 977– 86. 10.1056/NEJM199309303291401 8366922 · DOI · Paper page
  4. [4]Saydah S, Imperatore G Emerging approaches in surveillance of type 1 diabetes. Curr Diab Rep 2018; 18: 61. 10.1007/s11892-018-1033-1 29995215 PMC11835205 · DOI
  5. [5]Patterson CC, Harjutsalo V, Rosenbauer J, et al Trends and cyclical variation in the incidence of childhood type 1 diabetes in 26 European centres in the 25 year period 1989-2013: a multicentre prospective registration study. Diabetologia 2019; 62: 408– 17. 10.1007/s00125-018-4763-3 30483858 · DOI
  6. [6]Dahlquist GG, Nyström L, Patterson CC, et al Incidence of type 1 diabetes in Sweden among individuals aged 0-34 years, 1983-2007: an analysis of time trends. Diabetes Care 2011; 34: 1754– 9. 10.2337/dc11-0056 21680725 PMC3142045 · DOI
  7. [7]Haynes A, Bulsara MK, Jones TW, et al Incidence of childhood onset type 1 diabetes in Western Australia from 1985 to 2016: evidence for a plateau. Pediatr Diabetes 2018; 19: 690– 2. 10.1111/pedi.12636 29314574 · DOI
  8. [9]Hux JE, Ivis F, Flintoft V, et al Diabetes in Ontario: determination of prevalence and incidence using a validated administrative data algorithm. Diabetes Care 2002; 25: 512– 6. 10.2337/diacare.25.3.512 11874939 · DOI
  9. [10]Banda JM, Seneviratne M, Hernandez-Boussard T, et al Advances in electronic phenotyping: from rule-based definitions to machine learning models. Annu Rev Biomed Data Sci 2018; 1: 53– 68. 10.1146/annurev-biodatasci-080917-013315 31218278 PMC6583807 · DOI
  10. [11]Lethebe BC, Williamson T, Garies S, et al Developing a case definition for type 1 diabetes mellitus in a primary care electronic medical record database: an exploratory study. CMAJ Open 2019; 7: E246– 51. 10.9778/cmajo.20180142 PMC6504632 31061005 · DOI
  11. [13]Schroeder EB, Donahoo WT, Goodrich GK, et al Validation of an algorithm for identifying type 1 diabetes in adults based on electronic health record data. Pharmacoepidemiol Drug Saf 2018; 27: 1053– 9. 10.1002/pds.4377 29292555 PMC6028322 · DOI
  12. [14]Sharma M, Petersen I, Nazareth I, et al An algorithm for identification and classification of individuals with type 1 and type 2 diabetes mellitus in a large primary care database. Clin Epidemiol 2016; 8: 373– 80. 10.2147/CLEP.S113415 27785102 PMC5066690 · DOI
  13. [15]Lo-Ciganic W, Zgibor JC, Ruppert K, et al Identifying type 1 and type 2 diabetic cases using administrative data: a tree-structured model. J Diabetes Sci Technol 2011; 5: 486– 93. 10.1177/193229681100500303 21722564 PMC3192615 · DOI
  14. [17]Vanderloo SE, Johnson JA, Reimer K, et al Validation of classification algorithms for childhood diabetes identified from administrative data. Pediatr Diabetes 2012; 13: 229– 34. 10.1111/j.1399-5448.2011.00795.x 21771232 · DOI
  15. [18]Zhong VW, Obeid JS, Craig JB, et al An efficient approach for surveillance of childhood diabetes by type derived from electronic health record data: the search for diabetes in youth study. J Am Med Inform Assoc 2016; 23: 1060– 7. 10.1093/jamia/ocv207 27107449 PMC5070516 · DOI
  16. [20]Ivers N, Pylypenko B, Tu K Identifying patients with ischemic heart disease in an electronic medical record. J Prim Care Community Health 2011; 2: 49– 53. 10.1177/2150131910382251 23804663 · DOI
  17. [21]Tu K, Manuel D, Lam K, et al Diabetics can be identified in an electronic medical record using laboratory tests and prescriptions. J Clin Epidemiol 2011; 64: 431– 5. 10.1016/j.jclinepi.2010.04.007 20638237 · DOI
  18. [22]Ivers NM, Tu K, Young J, et al Feedback gap: pragmatic, cluster-randomized trial of goal setting and action plans to increase the effectiveness of audit and feedback interventions in primary care. Implement Sci 2013; 8: 142. 10.1186/1748-5908-8-142 24341511 PMC3878579 · DOI
  19. [23]Jones SR, Carley S, Harrison M An introduction to power and sample size estimation. Emerg Med J 2003; 20: 453– 8. 10.1136/emj.20.5.453 12954688 PMC1726174 · DOI
  20. [24]Aronson R, Orzech N, Ye C, et al Specialist-Led diabetes registries and predictors of poor glycemic control in type 2 diabetes: insights into the functionally refractory patient from the LMC diabetes registry database. J Diabetes 2016; 8: 76– 85. 10.1111/1753-0407.12257 25565383 · DOI
  21. [25]Guttmann A, Nakhla M, Henderson M, et al Validation of a health administrative data algorithm for assessing the epidemiology of diabetes in Canadian children. Pediatr Diabetes 2010; 11: 122– 8. 10.1111/j.1399-5448.2009.00539.x 19500278 · DOI
  22. [27]Henrard S, Speybroeck N, Hermans C Classification and regression tree analysis vs. multivariable linear and logistic regression methods as statistical tools for studying haemophilia. Haemophilia 2015; 21: 715– 22. 10.1111/hae.12778 26248714 · DOI
  23. [28]Hahn GJ, Meeker WQ Statistical intervals. John Wiley & Sons, Inc, 1991.
  24. [30]Harrell FE, Lee KL, Mark DB Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med 1996; 15: 361– 87. 10.1002/(SICI)1097-0258(19960229)15:4&lt;361::AID-SIM168&gt;3.0.CO;2-4 8668867 · DOI
  25. [31]ALaM W Classification and regression by randomForest. R News 2002; 2: 18– 22.
  26. [33]Fralick M, Schneeweiss S, Patorno E Risk of diabetic ketoacidosis after initiation of an SGLT2 inhibitor. N Engl J Med 2017; 376: 2300– 2. 10.1056/NEJMc1701990 28591538 · DOI · Paper page
  27. [34]Mayer-Davis EJ, Lawrence JM, Dabelea D, et al Incidence trends of type 1 and type 2 diabetes among Youths, 2002-2012. N Engl J Med 2017; 376: 1419– 29. 10.1056/NEJMoa1610187 28402773 PMC5592722 · DOI · Paper page
  28. [35]Newhook LA, Grant M, Sloka S, et al Very high and increasing incidence of type 1 diabetes mellitus in Newfoundland and Labrador, Canada. Pediatr Diabetes 2008; 9: 62– 8. 10.1111/j.1399-5448.2007.00315.x 18221421 · DOI
  29. [36]Toth EL, Lee KC, Couch RM, et al High incidence of IDDM over 6 years in Edmonton, Alberta, Canada. Diabetes Care 1997; 20: 311– 3. 10.2337/diacare.20.3.311 9051379 · DOI
  30. [37]Blanchard JF, Dean H, Anderson K, et al Incidence and prevalence of diabetes in children aged 0-14 years in Manitoba, Canada, 1985-1993. Diabetes Care 1997; 20: 512– 5. 10.2337/diacare.20.4.512 9096971 · DOI
  31. [38]Legault L, Polychronakos C Annual incidence of type 1 diabetes in Québec between 1989-2000 in children. Clin Invest Med 2006; 29: 10– 13. 16553358
  32. [39]Fox DA, Islam N, Sutherland J, et al Type 1 diabetes incidence and prevalence trends in a cohort of Canadian children and youth. Pediatr Diabetes 2018; 19: 501– 5. 10.1111/pedi.12566 28857360 · DOI
  33. [42]Ostrauskas R, Žalinkevičius R, Jurgevičienė N, et al The incidence of type 1 diabetes mellitus among 15-34 years aged Lithuanian population: 18-year incidence study based on prospective databases. BMC Public Health 2011; 11: 813. 10.1186/1471-2458-11-813 22011463 PMC3210107 · DOI

Cited by

Sorted by citations

Authors
Weisman, A., Tu, K., Young, J., Kumar, M., Austin, P. C., Jaakkimainen, L., Lipscombe, L., Aronson, R., & Booth, G. L.
Affiliations
  • University of Toronto (CA)
  • University Health Network (CA)
  • Institute for Clinical Evaluative Sciences (CA)
  • LMC Diabetes & Endocrinology (Canada) (CA)
  • St. Michael's Hospital (CA)
Corresponding author
Weisman, Alanna
ORCID
Venue
BMJ Open Diabetes Research & Care
Publisher
BMJ
Published
2020-06
Volume
8
Issue
1
Pages
e001224
ISSN
2052-4897 (ISSN-L)
DOI
10.1136/bmjdrc-2020-001224
PMC
PMC7307536
OpenAlex
W3036804145
Cited by
62
References
7
Citations by year
2026: 8 · 2025: 4 · 2024: 16 · 2023: 16 · 2022: 6 · 2021: 12
FWCI
7.46
Type
journal article
Language
en
Open access
Licence
Locations
  • BMJ Open Diabetes Research & Care (publishedVersion, cc-by-nc)
  • PubMed Central (publishedVersion, cc-by-nc)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
  • Ontario SPOR Support Unit
  • Diabetes Action Canada
Topics
  • Health Sciences › Health Professions › Health Information Management › Medical Coding and Health Information
  • Health Sciences › Medicine › Endocrinology, Diabetes and Metabolism › Diabetes Management and Research
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Genetics › Diabetes and associated disorders
Keywords
type 1 diabetes, type 2 diabetes, electronic health records, administrative health data, diabetes prevalence, diabetes incidence, Ontario, diabetes trends, classification trees, random forest, rule-based algorithm, algorithm validation, primary care records
MeSH terms
Adolescent, Adult, Algorithms, Delivery of Health Care, Diabetes Mellitus, Type 1, Diabetes Mellitus, Type 2, Humans, Ontario, Incidence, Prevalence, Electronic Health Records

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review