Cited by 369Meta-analysis and systematic reviewsMedical Coding and Health InformationStatistical Methods in Epidemiology

The DOI matches its Crossref record; 33 of the 33 references it cites were found in a registry or an index.

Development and use of reporting guidelines for assessing the quality of validation studies of health administrative data

Benchimol, Eric I., Manuel, Douglas G., To, Teresa, Griffiths, Anne M., Rabeneck, Linda, Guttmann, Astrid·2011·Journal of Clinical Epidemiology

DOI 10.1016/j.jclinepi.2010.10.006

Open this paper in CoChat — free account

Abstract

Background and objectives Validation of health administrative data for identifying patients with different health states (diseases and conditions) is a research priority, but no guidelines exist for ensuring quality. We created reporting guidelines for studies validating administrative data identification algorithms and used them to assess the quality of reporting of validation studies in the literature.Methods Using Standards for Reporting of Diagnostic accuracy (STARD) criteria as a guide, we created a 40-item checklist of items with which identification accuracy studies should be reported. A systematic review identified studies that validated identification algorithms using administrative data. We used the checklist to assess the quality of reporting.Results In 271 included articles, goals and data sources were well reported but few reported four or more statistical estimates of accuracy (36.9%). In 65.9% of studies reporting positive predictive value (PPV)/negative predictive value (NPV), the prevalence of disease in the validation cohort was higher than in the administrative data, potentially falsely elevating predictive values. Subgroup accuracy (53.1%) and 95% confidence intervals for accuracy measures (35.8%) were also underreported.Conclusions The quality of studies validating health states in the administrative data varies, with significant deficits in reporting of markers of diagnostic accuracy, including the appropriate estimation of PPV and NPV. These omissions could lead to misclassification bias and incorrect estimation of incidence and health services utilization rates. Use of a reporting checklist, such as the one created for this study by modifying the STARD criteria, could improve the quality of reporting of validation studies, allowing for accurate application of algorithms, and interpretation of research using health administrative data.

Integrity signals

Registry

Crossref · 10.1016/j.jclinepi.2010.10.006

Cites

33 of 33 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement: Guidelines for Reporting Observational Studies
von Elm, E., Altman, D. G., Egger, M., Pocock, S. J., Gøtzsche, P. C., Vandenbroucke, J. P., & Initiative, F. T. S. (2007) · PLoS Medicine · doi:10.1371/journal.pmed.0040296 · Cited by 23,020
Towards complete and accurate reporting of studies of diagnostic accuracy: The STARD Initiative.
Bossuyt, P. M., Reitsma, J. B., Bruns, D. E., Gatsonis, C. A., Glasziou, P. P., Irwig, L. M., Lijmer, J. G., Moher, D., Rennie, D., de Vet, H. C. W., et al. (2003) · Annals of Internal Medicine · doi:10.7326/0003-4819-138-1-200301070-00010 · Cited by 3,219
Towards complete and accurate reporting of studies of diagnostic accuracy: the STARD initiative
Bossuyt, P. M., Reitsma, J. B., Bruns, D. E., Gatsonis, C. A., Glasziou, P. P., Irwig, L. M., Lijmer, J. G., Moher, D., Rennie, D., & de Vet, H. C. W. (2003) · BMJ · doi:10.1136/bmj.326.7379.41 · Cited by 3,215
Improving the Quality of Reporting of Randomized Controlled Trials: The CONSORT Statement
Begg, C. (1996) · JAMA · doi:10.1001/jama.1996.03540080059030 · Cited by 3,127
The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies
von Elm, E., Altman, D. G., Egger, M., Pocock, S. J., Gøtzsche, P. C., & Vandenbroucke, J. P. (2007) · Preventive Medicine · doi:10.1016/j.ypmed.2007.08.012 · Cited by 2,698
Statistics Notes: Diagnostic tests 2: predictive values
Altman, D. G., & Bland, J. M. (1994) · BMJ · doi:10.1136/bmj.309.6947.102 · Cited by 1,711
Diabetes in Ontario
Hux, J. E., Ivis, F. J., Flintoft, V. F., & Bica, A. (2002) · Diabetes Care · doi:10.2337/diacare.25.3.512 · Cited by 1,290
The CONSORT Statement
Cha, M., Eastwaad, S., Hartan, R., Maher, D., Olkin, I., Pitkin, R., Rennie, D., Schulz, K. F., Simel, D. L., & Straup, D. F. (1996) · Cited by 736
Epidemiology of Crohn's Disease and Ulcerative Colitis in a Central Canadian Province: A Population-based Study
Bernstein, C. N., Blanchard, J., Rawsthorne, P. E., & Wajda, A. (1999) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a009735 · Cited by 661
Medical Subject Headings
ROGERS, F. B. (1965) · Nature · doi:10.1038/205236a0 · Cited by 571

Show all 32 references

Sorted by citations

Items 201–300 of 369

Development and Evaluation of the Algorithm CErtaInty Tool (ACE-IT) to Assess Electronic Medical Record and Claims-based Algorithms’ Fit for Purpose for Safety Outcomes
Singh, S., Beyrer, J. K., Zhou, X., Swerdel, J. N., Harvey, R. A., Hornbuckle, K., Russo, L., Ghauri, K., Abi‐Elias, I. H., Cox, J. S., et al. (2022) · Drug Safety · doi:10.1007/s40264-022-01254-4 · Cited by 9
Assessing cardiovascular disease risk in women with a history of hypertensive disorders of pregnancy: A guidance paper for studies using administrative data
Johnston, A., Smith, G. N., Tanuseputro, P., Coutinho, T. A., & Edwards, J. D. (2024) · Paediatric and Perinatal Epidemiology · doi:10.1111/ppe.13043 · Cited by 8
Evaluation of reporting quality of cohort studies using real-world data based on RECORD: systematic review
Zhao, R., Zhang, W., Zhang, Z., He, C., Xu, R., Tang, X., & Wang, B. (2023) · BMC Medical Research Methodology · doi:10.1186/s12874-023-01960-2 · Cited by 8
Accuracy of International Classification of Diseases, 9th Revision, Clinical Modification codes for upper gastrointestinal complications varied by position and age: a validation study in a cohort of nonsteroidal anti‐inflammatory drugs users in Friuli Venezia Giulia, Italy
Pisa, F. E., Castellsagué, J., Drigo, D., Riera‐Guàrdia, N., Giangreco, M., Rosolen, V., Clagnan, E., Zanier, L., Perez‐Gutthann, S., & Barbone, F. (2013) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.3504 · Cited by 8
A systematic review of database validation studies among fertility populations
Bacal, V., Russo, M., Fell, D. B., Shapiro, H. M., WALKER, M. C., & Gaudet, L. (2019) · Human Reproduction Open · doi:10.1093/hropen/hoz010 · Cited by 8
Big data: Using databases and registries
Jacob‐Brassard, J., & de Mestral, C. (2022) · Seminars in Vascular Surgery · doi:10.1053/j.semvascsurg.2022.09.002 · Cited by 7
The German Quality Network Sepsis: Evaluation of a Quality Collaborative on Decreasing Sepsis-Related Mortality in a Controlled Interrupted Time Series Analysis
Schwarzkopf, D., Rüddel, H., Brinkmann, A., Fleischmann-Struzek, C., Friedrich, M. E., Glas, M., Gogoll, C., Gründling, M., Meybohm, P., Pletz, M. W., et al. (2022) · Frontiers in Medicine · doi:10.3389/fmed.2022.882340 · Cited by 9
Performance of French medico-administrative databases in epidemiology of infectious diseases: a scoping review
Tassi, M., Meur, N. L., Stéfic, K., & Grammatico‐Guillon, L. (2023) · Frontiers in Public Health · doi:10.3389/fpubh.2023.1161550 · Cited by 7
Validation and impact of algorithms for identifying variables in observational studies of routinely collected data
Wang, W., Liu, M., He, Q., Wang, M., Xu, J., Li, L., Li, G., He, L., Zou, K., & Sun, X. (2023) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2023.111232 · Cited by 7
The incidence and risk factors of persistent opioid use after surgery
Dreiling, J., Rose, N., Arnold, C., Baumbach, P., Fleischmann-Struzek, C., Kubulus, C., Komann, M., Marschall, U., Rittner, H. L., Volk, T. J., et al. (2024) · Deutsches Ärzteblatt international · doi:10.3238/arztebl.m2024.0200 · Cited by 7
Reliability and validity assessment of administrative databases in measuring the quality of rectal cancer management
Corbellini, C., Andreoni, B., Ansaloni, L., Sgroi, G., Martinotti, M., Scandroglio, I., Carzaniga, P., Longoni, M., Foschi, D., Dionigi, P., et al. (2017) · Tumori Journal · doi:10.5301/tj.5000708 · Cited by 7
Retrospective comparison of Australia’s Pharmaceutical Benefits Scheme claims data with prescription data in HER2-positive early breast cancer patients, 2008–2012
Harris, C. A., Daniels, B. J., Ward, R. L., & Pearson, S. (2017) · Public Health Research & Practice · doi:10.17061/phrp2751744 · Cited by 7
Validation study of German inpatient administrative health data for epidemiological surveillance and measurement of quality of care for sepsis: the OPTIMISE study protocol
Schwarzkopf, D., Fleischmann-Struzek, C., Schlattmann, P., Dorow, H., Ouart, D., Edel, A., Gonnert, F. A., Götz, J., Gründling, M., Heim, M. H., et al. (2020) · BMJ Open · doi:10.1136/bmjopen-2019-035763 · Cited by 7
Accuracy of Administrative Database Algorithms for Hospitalized Pneumonia in Adults: a Systematic Review
Corrales‐Medina, V., & van Walraven, C. (2021) · Journal of General Internal Medicine · doi:10.1007/s11606-020-06211-4 · Cited by 7
Development and validation of a case definition to identify hemophilia in administrative data
Alam, A. U., Karkhaneh, M., Wu, C. M., & Sun, H. (2021) · Thrombosis Research · doi:10.1016/j.thromres.2021.05.013 · Cited by 7
Validation of ICD-10 codes for the identification of paediatric out-of-hospital cardiac arrest patients
Gray, K., Cameron, S., McKenzie, K., Miller, M. J., Odoardi, N., & Tijssen, J. A. (2021) · Resuscitation · doi:10.1016/j.resuscitation.2021.12.017 · Cited by 7
Validity of the coding for appendicitis, appendectomy, and diagnostic laparoscopy in the Danish National Patient Registry
Fonnes, S., Erichsen, R., & Rosenberg, J. (2023) · Scandinavian Journal of Surgery · doi:10.1177/14574969221148078 · Cited by 7
Database Research for Pediatric Infectious Diseases
Kronman, M. P., Gerber, J. S., Newland, J. G., & Hersh, A. L. (2015) · Journal of the Pediatric Infectious Diseases Society · doi:10.1093/jpids/piv007 · Cited by 7
The Validity of Using Health Administrative Data To Identify the Involvement of Specialized Pediatric Palliative Care Teams in Children with Cancer in Ontario, Canada
Widger, K. A., Vadeboncoeur, C. M., Zelcer, S. M., Liu, Y., Kassam, A., Sutradhar, R., Rapoport, A., Nelson, K., Wolfe, J., Earle, C. C., et al. (2017) · Journal of Palliative Medicine · doi:10.1089/jpm.2017.0028 · Cited by 7
Needles in the haystack: Using open-text fields to identify persons with intellectual and developmental disabilities in administrative home care data
McKenzie, K., Martin, L., & Ouellette-Kuntz, H. M. J. (2017) · Research in Developmental Disabilities · doi:10.1016/j.ridd.2017.07.019 · Cited by 7
The agreement between chronic diseases reported by patients and derived from administrative data in patients undergoing joint arthroplasty
Podmore, B., Hutchings, A., Konan, S., & van der Meulen, J. (2019) · BMC Medical Research Methodology · doi:10.1186/s12874-019-0729-5 · Cited by 7
Internal deterministic record linkage using indirect identifiers for matching of same-patient hospital transfers and early readmissions after acute coronary syndrome in a nationwide hospital discharge database: a retrospective observational validation study
Rocha, A., Azevedo, L. F., Silva‐Cardoso, J., Allison, T. G., & Freitas, A. (2019) · BMJ Open · doi:10.1136/bmjopen-2019-033486 · Cited by 7
Identification and Validation of Hemophilia-Related Outcomes on Japanese Electronic Medical Record Database (Hemophilia-REAL V Study)
Fujiwara, T., Miyakoshi, C., Kanemitsu, T., Okumura, Y., & Tokumasu, H. (2021) · Journal of Blood Medicine · doi:10.2147/jbm.s313371 · Cited by 7
How to use administrative data for surgical outcomes in acute diverticulitis
Hong, M. K.-Y., Skandarajah, A. R., Faiz, O. D., & Hayes, I. (2016) · General Surgery · doi:10.18282/gs.v1i1.32 · Cited by 7
Comment garantir des données médicales de qualité dans les études cliniques, pharmaco-épidémiologiques et en pharmacovigilance ?
Alla, F., Rosilio, M., Funck‐Brentano, C., Barthélémy, P., Brisset, S., Cellier, D., Chassany, O., Demarez, J.-P., Diebolt, V., Francillon, A., et al. (2013) · Therapies · doi:10.2515/therapie/2013035 · Cited by 7
Validation of Insurance Billing Codes for Monitoring Antenatal Screening
Nethery, E., Hutcheon, J. A., Law, M. R., & Janssen, P. A. (2022) · Epidemiology · doi:10.1097/ede.0000000000001569 · Cited by 7
Validating a proxy for disease progression in metastatic cancer patients using prescribing and dispensing data
Joshi, V., Adelstein, B., Schaffer, A. L., Srasuebkul, P., Dobbins, T., Pearson, S., & the Elements of Cancer Care (EoCC) Investigators (2016) · Asia-Pacific Journal of Clinical Oncology · doi:10.1111/ajco.12602 · Cited by 6
Validation of acute exacerbation of chronic obstructive pulmonary disease (COPD) recording in electronic health records: a systematic review protocol
Stone, P., Sood, N., Feary, J. R., Roberts, C. M., & Quint, J. K. (2020) · BMJ Open · doi:10.1136/bmjopen-2019-032467 · Cited by 6
National Database Research in Head and Neck Reconstructive Surgery: A Call for Increased Transparency and Reproducibility
Bur, A. M., Villwock, M. R., Nallani, R., Gomez, E., Varvares, M. A., Villwock, J. A., Cannady, S. B., & Wax, M. K. (2020) · Otolaryngology · doi:10.1177/0194599820938044 · Cited by 6
Identification of complications requiring interventions after gastrointestinal cancer surgery from real‐world data: An external validation study
Kinoshita, H., Nishigori, T., Kunisawa, S., Hida, K., Hosogi, H., Inamoto, S., Hata, H., Matsusue, R., Imanaka, Y., Obama, K., et al. (2023) · Annals of Gastroenterological Surgery · doi:10.1002/ags3.12704 · Cited by 6
Model-based methods for case definitions from administrative health data: application to rheumatoid arthritis
Kroeker, K., Widdifield, J., Muthukumarana, S., Jiang, D., & Lix, L. M. (2017) · BMJ Open · doi:10.1136/bmjopen-2017-016173 · Cited by 6
A Validation Study: How Predictive Is a Diagnostic Coding Algorithm at Identifying Rheumatic Heart Disease in Western Australian Hospital Data?
Fitz-Gerald, J. A., Ongzalima, C. O., Ng, G. A., Greenland, M., Sanfilippo, F. M., Hung, J., & Katzenellenbogen, J. M. (2019) · Heart Lung and Circulation · doi:10.1016/j.hlc.2019.08.020 · Cited by 6
Validation of case definition algorithms for the ascertainment of congenital anomalies
de Escalante, Y. N., Abayomi, A., Langlois, S. F., Ye, X., Erickson, A. C., Ngo, H., Armour, R., Okamoto, R., Arbour, L. T., Bedard, T., et al. (2022) · Birth Defects Research · doi:10.1002/bdr2.2112 · Cited by 6
Development and validation of an administrative data algorithm to identify adults who have endoscopic sinus surgery for chronic rhinosinusitis
Macdonald, K. I., Kilty, S., & van Walraven, C. (2017) · Journal of Otolaryngology - Head and Neck Surgery · doi:10.1186/s40463-017-0216-0 · Cited by 6
Validity of algorithms for identifying mild traumatic brain injury in the French national emergency department database OSCOUR: a retrospective multicentre validation study protocol
Paget, L.-M., Forgeot, C., Lorton, F., Aghakhani, N., Raphaël, M., Gallay, A., & Beltzer, N. (2022) · BMJ Open · doi:10.1136/bmjopen-2021-059961 · Cited by 5
Assessment of the French National Health Insurance Information System as a tool for epidemiological surveillance of malaria
Delon, F., Mayet, A., Théllier, M., Kendjo, É., Michel, R., Ollivier, L., Châtellier, G., & Desjeux, G. (2016) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocw164 · Cited by 5
Accuracy of administrative database estimates of national surgical volume: Solid organ transplantation in the National Inpatient Sample
Fielding‐Singh, V., Grogan, T., & Neelankavil, J. P. (2021) · Clinical Transplantation · doi:10.1111/ctr.14441 · Cited by 5
Administrative database code accuracy did not vary notably with changes in disease prevalence
van Walraven, C., English, S. W., & Austin, P. C. (2016) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2016.05.009 · Cited by 5
Guidance of development, validation, and evaluation of algorithms for populating health status in observational studies of routinely collected data (DEVELOP-RCD)
Wang, W., Jin, Y.-H., Liu, M., He, Q., Xu, J., Wang, M., Li, G., Fu, B., Yan, S., Zou, K., et al. (2024) · Military Medical Research · doi:10.1186/s40779-024-00559-y · Cited by 5
Validating viral hepatitis B and C diagnosis codes: a retrospective analysis using Ontario’s health administrative data
Yasseen, A. S., Kwong, J. C., Kustra, R., Holder, L., Chung, H., MacDonald, L. M., Janjua, N. Z., Mazzulli, T., Feld, J. J., & Crowcroft, N. S. (2021) · Canadian Journal of Public Health · doi:10.17269/s41997-020-00435-x · Cited by 5
How can the Quality of Medical Data in Pharmacovigilance, Pharmacoepidemiology and Clinical Studies be Guaranteed?
Alla, F., Rosilio, M., Funck‐Brentano, C., Barthélémy, P., Brisset, S., Cellier, D., Chassany, O., Demarez, J.-P., Diebolt, V., Francillon, A., et al. (2013) · Therapies · doi:10.2515/therapie/2013040 · Cited by 5
Liver-FibroSTARD checklist and glossary: tools for standardized design and reporting of diagnostic accuracy studies of liver fibrosis tests
Guechot, J., Boursier, J., de Lédinghen, V., Poynard, T., Carrat, F., Leroy, V., Wong, G. L., Friedrich‐Rust, M., Fraquelli, M., Plebani, M., et al. (2015) · Clinical Chemistry and Laboratory Medicine (CCLM) · doi:10.1515/cclm-2015-0241 · Cited by 5
Epidemiology and Health Administrative Data
Benchimol, E. I. (2014) · Inflammatory Bowel Diseases · doi:10.1097/mib.0000000000000153 · Cited by 5
Self-citation
Pelvic floor disorders in women who consult primary care clinics: development and validation of case definitions using primary care electronic medical records
Ross, S., Fast, H., Garies, S., Slade, D., Jackson, D., Doraty, M., Miyagishima, R., Soós, B., Taylor, M., Williamson, T. S., et al. (2020) · CMAJ Open · doi:10.9778/cmajo.20190145 · Cited by 4
Trends in appropriateness of end-of-life care in people with cancer, COPD or with dementia measured with population-level quality indicators
Schreye, R. D., Deliëns, L., Annemans, L., Gielen, B., Smets, T., & Cohen, J. (2023) · PLoS ONE · doi:10.1371/journal.pone.0273997 · Cited by 4
Validation of ICD‐9‐CM codes for major genitourinary birth defects in Military Health System administrative data, 2006–2014
Romano, C. J., Magallon, S. M., Hall, C. B., Bukowinski, A. T., Gumbs, G. R., & Conlin, A. M. S. (2023) · Birth Defects Research · doi:10.1002/bdr2.2265 · Cited by 4
Validity of Physician Billing Claims to Identify Deceased Organ Donors in Large Healthcare Databases
Li, A. H., Kim, S. J., Rangrej, J., Scales, D. C., Shariff, S. Z., Redelmeier, D. A., Knoll, G. A., Young, A., & Garg, A. X. (2013) · PLoS ONE · doi:10.1371/journal.pone.0070825 · Cited by 4
Developing case‐finding algorithms for second events of oropharyngeal cancer using administrative data: A population‐based validation study
Xu, Y., Kong, S., Cheung, W. Y., Quan, M. L., Nakoneshny, S. C., & Dort, J. C. (2019) · Head & Neck · doi:10.1002/hed.25682 · Cited by 4
La déclaration RECORD-PE (Reporting of Studies Conducted Using Observational Routinely Collected Health Data Statement for Pharmacoepdemiology) : directives pour la communication des études realisées à partir de données de santé observationelles collectées en routine en pharmacoépidémiologie
Langan, S., Schmidt, S. A. J., Wing, K., Ehrenstein, V., Nicholls, S. G., Filion, K. B., Klungel, O. H., Petersen, I., Sørensen, H. T., Dixon, W. G., et al. (2019) · Canadian Medical Association Journal · doi:10.1503/cmaj.190347 · Cited by 4
Self-citation
Routinely collected data for population-based outcomes research
Welk, B. K. (2019) · Canadian Urological Association Journal · doi:10.5489/cuaj.6158 · Cited by 4
Automated approaches to support secondary study processes: a systematic review
Molléri, J. S., & Benitti, F. B. V. (2012) · doi:10.13140/2.1.3905.8886 · Cited by 4
The STROBE Extensions
Sharp, M. K., Hren, D., & Altman, D. G. (2018) · Epidemiology · doi:10.1097/ede.0000000000000899 · Cited by 4
Validity of electronic hospital discharge prescription records as a source of medication data for pharmacoepidemiological research
Fanning, L., Vo, L., Ilomäki, J., Bell, J. S., Elliott, R. A., & Darzins, P. J. (2018) · Therapeutic Advances in Drug Safety · doi:10.1177/2042098618776598 · Cited by 4
Practices, norms, and aspirations regarding the construction, validation, and reuse of code sets in the analysis of real-world dataPreprint
Gold, S., Lehmann, H. P., Schilling, L., & Lutters, W. G. (2021) · medRxiv · doi:10.1101/2021.10.14.21264917 · Cited by 4
Think sepsis, write sepsis, code sepsis – patient characteristics associated with sepsis (under-)coding in administrative health data
Thomas-Rüddel, D. O., Rose, N., Fleischmann-Struzek, C., Reinhart, K., Boden, B., Dorow, H., Edel, A., Gonnert, F. A., Götz, J., Gründling, M., et al. (2025) · Infection · doi:10.1007/s15010-025-02685-8 · Cited by 3
Validation of Diagnostic Code Definitions for Identifying Patient in Korean Health Insurance Claims Data: A Scoping Review
Kim, K.-H. (2025) · Journal of Korean Medical Science · doi:10.3346/jkms.2025.40.e343 · Cited by 3
Accuracy of routinely collected hospital administrative discharge data and death certificate ICD-10 diagnostic coding in progressive supranuclear palsy and corticobasal syndrome: a systematic review and validation study
Swallow, D. M. A., & Counsell, C. E. (2024) · Journal of Neurology · doi:10.1007/s00415-024-12280-w · Cited by 3
The use of healthcare services and disabling chronic pain: results from the cross-sectional population-based Andalusian Health Survey
Cáceres‐Matos, R., Gil‐García, E., Vázquez-Santiago, S., & Cabrera‐León, A. (2024) · European Journal of Public Health · doi:10.1093/eurpub/ckae079 · Cited by 3
Is pathology necessary to predict mortality among men with prostate-cancer?
Margel, D., Urbach, D. R., Lipscombe, L. L., Bell, C. M., Kulkarni, G. S., Baniel, J., Fleshner, N. E., & Austin, P. C. (2014) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-014-0114-6 · Cited by 3
Sensitivity and specificity of algorithms for the identification of nonspecific low back pain in medico-administrative databases
Ly, A., Sirois, C., & Dionne, C. E. (2023) · Pain · doi:10.1097/j.pain.0000000000002861 · Cited by 3
Algorithms Identifying Patients With Acute Exacerbation of Interstitial Pneumonia and Acute Interstitial Lung Diseases Developed Using Japanese Administrative Data
Anan, K., Kataoka, Y., Ichikado, K., Kawamura, K., Yasuda, Y. U., Hisanaga, J., Nitawaki, T., & Yamamoto, Y. (2024) · Cureus · doi:10.7759/cureus.53073 · Cited by 2
Bias and Confounding: Pharmacoepidemiological Study Using Administrative Database
Nojiri, S. (2015) · YAKUGAKU ZASSHI · doi:10.1248/yakushi.15-00006 · Cited by 3
Using Receiver Operating Characteristic (ROC) Analysis to Evaluate Information-Based Decision-Making
Hu, N. (2018) · Advances in logistics, operations, and management science book series · doi:10.4018/978-1-5225-7362-3.ch057 · Cited by 3
Validity of hospital discharge records to identify pregestational diabetes in an Italian population
Borsari, L., Malagoli, C., Ballotari, P., Girolamo, G. D., Bonora, K., Violi, F., Capelli, O., Rodolfi, R., Nicolini, F., & Vinceti, M. (2016) · Diabetes Research and Clinical Practice · doi:10.1016/j.diabres.2016.11.023 · Cited by 3
Construction of a Heart Failure Database Collating Administrative Claims Data and Electronic Medical Record Data to Evaluate Risk Factors for In-Hospital Death and Prolonged Hospitalization
Kodama, K., Sakamoto, T., Kubota, T., Takimura, H., Hongo, H., Chikashima, H., Shibasaki, Y., Yada, T., Node, K., Nakayama, T., et al. (2019) · Circulation Reports · doi:10.1253/circrep.cr-19-0051 · Cited by 3
Transferability of the early-stage ovarian malignancy (EOM) score: an external validation study that includes advanced-stage and metastatic ovarian cancer
Phinyo, P., Patumanond, J., Saenrungmuaeng, P., Chirdchim, W., Pipanmekaporn, T., Tantraworasin, A., Tongsong, T., & Tantipalakorn, C. (2021) · Archives of Gynecology and Obstetrics · doi:10.1007/s00404-020-05955-y · Cited by 3
Evaluation of the Quality of French Hospital Data for Perinatal Algorithms
Goueslard, K., Cottenet, J., Benzenine, É., Tubert‐Bitter, P., & Quantin, C. (2020) · Studies in health technology and informatics · doi:10.3233/shti200153 · Cited by 3
Association between maternal body mass index and hospital admissions for infection in offspring: longitudinal cohort study
Coathup, V., Carson, C., Ashdown, H. F., Santorelli, G., & QUIGLEY, M. A. (2025) · BMJ Medicine · doi:10.1136/bmjmed-2024-001050 · Cited by 2
Development and Validation of Case-Ascertainment Algorithms for Hypertensive Disorders of Pregnancy Using Longitudinal Electronic Health Records Data
Zhu, Y., Wang, E., Ngo, A. N., Greenberg, M., & FERRARA, A. (2025) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2025.112007 · Cited by 3
Validation of the ICD-10 Diagnoses of Early-Onset Neonatal Infection in the Danish National Patient Register from 2010 to 2018
Andersen, M. H., Haglund, A., Murra, M., Matthiesen, N. B., Nielsen, S. Y., & Henriksen, T. B. (2026) · Clinical Epidemiology · doi:10.2147/clep.s562634 · Cited by 2
Quality of cancer-related data from the Danish National Patient Registry (1994-2025) and the Danish Cancer Registry (2004-2025): a systematic review
Simoni, A. H., Hald, K., Overvad, T. F., Søgaard, M. I. V., & Ording, A. G. (2026) · European Journal of Epidemiology · doi:10.1007/s10654-025-01336-4 · Cited by 2
Validating exacerbations of asthma in electronic health records: a systematic review
Moore, E., Gassasse, Z., Sinha, I., Hawcutt, D. B., & Quint, J. K. (2026) · European Respiratory Review · doi:10.1183/16000617.0004-2026 · Cited by 3
Accuracy of administrative data in ascertaining health conditions: a systematic review
Campbell, A. C., Tyler, J., Shuttleworth, R. R., Pearce, L. A., Heerde, J. A., Borschmann, R., Sawyer, S. M., Preen, D. B., Kinner, S. A., & Calais‐Ferreira, L. (2026) · JAMIA Open · doi:10.1093/jamiaopen/ooag109 · Cited by 2
Validated methods to identify patients with asthma–COPD overlap in healthcare databases: a systematic review protocol
Amegadzie, J. E., Badejo, O. A., Gamble, J., Wright, M., Farrell, J., Jackson, B., Sultana, K., Hashmi, M., & Gao, Z. (2019) · BMJ Open · doi:10.1136/bmjopen-2018-024306 · Cited by 2
Population-Based Research Using Administrative Data to Evaluate Long-Term Outcomes in Burn Injury
Mason, S. A., Spiwak, R. P., & Logsetty, S. (2019) · Springer eBooks · doi:10.1007/978-3-030-18940-2_5 · Cited by 2
Validated algorithms for identifying timing of second event of oropharyngeal squamous cell carcinoma using r eal‐world data
Khair, S., Dort, J. C., Quan, M. L., Cheung, W. Y., Sauro, K. M., Nakoneshny, S. C., Popowich, B., Liu, P., Wu, G., & Xu, Y. (2022) · Head & Neck · doi:10.1002/hed.27109 · Cited by 2
Validation of ICD-10 codes for studying foreign body airway obstructions: A health administrative data cohort study
Dunne, C. L., Cirone, J., McRae, A. D., Blanchard, I. E., Holroyd-Leduc, J. M., & Sauro, K. M. (2023) · Resuscitation Plus · doi:10.1016/j.resplu.2023.100479 · Cited by 2
Development of a set of quality indicators in paediatric and perinatal care in Japan with a modified Delphi method
Shinjo, D., Ozawa, N., Nakadate, N., Kanamori, Y., Matsumoto, K., Noguchi, T., Ohtera, S., & Kato, H. (2023) · BMJ Paediatrics Open · doi:10.1136/bmjpo-2023-002209 · Cited by 2
Early- to mid-term outcome of a short, cementless, titanium, flat, tapered stem for primary total hip arthroplasty: an independent series
Bourget-Murray, J., Biniam, B., Bhullar, R. S., Kim, P., Gofton, W. T., Beaulé, P. E., & Grammatopoulos, G. A. (2023) · Hip International · doi:10.1177/11207000231216421 · Cited by 2
Identifying patients with diagnosed cirrhosis in administrative health databases: a validation study
Faisal, N., Lix, L. M., Walld, R., Singer, A. G., Renner, E. L., Singh, H., Kosowan, L., & Mahar, A. L. (2024) · Canadian Liver Journal · doi:10.3138/canlivj-2023-0013 · Cited by 2
External validation of the CARDOT score for predicting respiratory complications after thoracic surgery
Pipanmekaporn, T., Kitswat, P., Leurcharusmee, P., Runraksar, T., Bunchungmongkol, N., Khorana, J., Tantraworasin, A., Lapisatepun, P., & Saokaew, S. (2024) · BMC Anesthesiology · doi:10.1186/s12871-024-02685-5 · Cited by 2
Validating the recording of exacerbations of asthma in electronic health records: a systematic review protocol
Moore, E., Gassasse, Z., & Quint, J. K. (2024) · BMJ Open · doi:10.1136/bmjopen-2024-088849 · Cited by 2
Validation of algorithms to identify human immunodeficiency virus cases using administrative data in Manitoba
Anderson, A. J., Lix, L. M., Loeppky, C., Caeseele, P. G. V., Queenan, J. A., & Mahar, A. L. (2024) · Canadian Journal of Public Health · doi:10.17269/s41997-024-00951-0 · Cited by 2
Developing and Refining the Pregnancy Algorithm CErtaInty Tool ( ACE ‐ IT ) for Validating Pregnancy Outcomes
Singh, S., Huang, H., Andrade, S. E., Butler, L., Kim, S. H., Noan‐Lainé, M. L., Pinheiro, S. P., Shaughnessy, L., Desrochers, O., & Rodriguez‐Watson, C. (2025) · Clinical Pharmacology & Therapeutics · doi:10.1002/cpt.70036 · Cited by 2
Applying evidence to patient care: From population health to individual patient values
Jetté, N., Choi, H., & Wiebe, S. (2012) · Epilepsy & Behavior · doi:10.1016/j.yebeh.2012.08.017 · Cited by 2
Validation of health outcomes of interest in healthcare databases
Re, V. L. (2021) · Elsevier eBooks · doi:10.1016/b978-0-12-817663-4.00022-2 · Cited by 1
DM management in HIV patients: the adoption of population health management to transform the chronic management of HIV
Pacileo, G., Morando, V., Banks, H., Ferrara, L., Cattelan, A. M., Luzzati, R., Manfrin, V., & Tozzi, V. D. (2022) · European Journal of Public Health · doi:10.1093/eurpub/ckac120 · Cited by 3
Development and validation of a claims-based algorithm to identify patients with Neuromyelitis Optica Spectrum disorder
Patel, A. M., Exuzides, A., Yermilov, I., Dalglish, H., Gibbs, S. N., Reddy, S. R., Chang, E. Y., Paydar, C., Broder, M. S., COHAN, S. L., et al. (2024) · Journal of the Neurological Sciences · doi:10.1016/j.jns.2024.123110 · Cited by 2
Development and validation of a case-finding algorithm for neck and back pain in the Canadian Armed Forces using health administrative data
Thériault, F. L., Lu, D., & Hawes, R. A. (2019) · Journal of Military Veteran and Family Health · doi:10.3138/jmvfh.2018-0039 · Cited by 2
Predicting Individual-Level Probabilities of Dementia and Diabetes using Health Services Administrative Data
Chaudhry, M. (2015) · TSpace (University of Toronto) · Cited by 2
Creating and utilising the Wales Asthma Observatory to support health policy, health service planning and clinical research
Sallakh, M. A. A. (2018) · doi:10.23889/suthesis.48569 · Cited by 2
Development and validation of an algorithm of diagnostic and procedural codes for the identification of children hospitalized with a tracheostomy in Ontario, Canada
McKelvie, B., Pianosi, K., Chan, J., Tsampalieros, A. K., Benchimol, E. I., Macdonald, K. I., Strychowsky, J. E., Vaccani, J., & McNally, J. D. (2020) · Pediatric Pulmonology · doi:10.1002/ppul.24757 · Cited by 1
Self-citation
Mobilizing registry data for quality improvement: A convergent mixed-methods analysis and application to spinal cord injury
Krysa, J. A., Manhas, K. P., Sánchez, A. L., Casha, S., Burns, K. K., Charbonneau, R., Ho, C., & Papathanassoglou, E. D. E. (2023) · Frontiers in Rehabilitation Sciences · doi:10.3389/fresc.2023.899630 · Cited by 1
Validation of a primary care electronic medical records case definition for eczema: retrospective cross-sectional study
Stirton, H., Kosowan, L., Abrams, E. M., Protudjer, J. L. P., Queenan, J. A., & Singer, A. G. (2023) · Allergy Asthma and Clinical Immunology · doi:10.1186/s13223-023-00785-4 · Cited by 1
Identification of Mood Disorders in Self‐Reported Versus Health Administrative Data
Dohouin, I., Laberge, M., Lacasse, A., Poder, T. G., & TorSaDE Cohort working group (2024) · Brain and Behavior · doi:10.1002/brb3.70126 · Cited by 1
An algorithm for identifying causes of reoperations after orthopedic fracture surgery in health administrative data: a diagnostic accuracy study using the Danish National Patient Register
Jensen, S. S., Rønnegaard, A. B., Gundtoft, P. H., Kold, S., & Viberg, B. (2025) · Acta Orthopaedica · doi:10.2340/17453674.2024.42633 · Cited by 1
Reporting of validation studies: A systematic review and quantitative analysis of research publications in palliative care journals
Kumar, S. P., & Sisodia, V. (2013) · Saudi Journal for Health Sciences · doi:10.4103/2278-0521.127048 · Cited by 1
La déclaration RECORD (Reporting of Studies Conducted Using Observational Routinely Collected Health Data) : directives pour la communication des études réalisées à partir de données de santé collectées en routine
Benchimol, E. I., Smeeth, L. L., Guttmann, A., Harron, K. L., Moher, D., Petersen, I., Sørensen, H. T., Januel, J., von Elm, E. B., & Langan, S. (2019) · Canadian Medical Association Journal · doi:10.1503/cmaj.181309 · Cited by 2
Self-citation
Use of diagnostic likelihood ratio of outcome to evaluate misclassification bias in the planning of database studies
Ii, Y., Hiro, S., & Nakazuru, Y. (2022) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-022-01757-1 · Cited by 1
Validation of Algorithms to Identify Bone Metastases Using Administrative Claims Data in a Japanese Hospital
Hirano, T., Saito, N., Wakabayashi, R., & Kuwatsuru, R. (2023) · Drugs - Real World Outcomes · doi:10.1007/s40801-022-00347-x · Cited by 1

Authors: Benchimol, E. I., Manuel, D. G., To, T., Griffiths, A. M., Rabeneck, L., & Guttmann, A.
Affiliations: University of Ottawa (CA)
Statistics Canada (CA)
University of Toronto (CA)
Corresponding author: Benchimol, Eric I.
ORCID: Benchimol, Eric I.
Manuel, Douglas G.
To, Teresa
Griffiths, Anne M.
Rabeneck, Linda
Guttmann, Astrid
Venue: Journal of Clinical Epidemiology
Publisher: Elsevier BV
Published: 2011-08
Volume: 64
Issue: 8
Pages: 821-829
ISSN: 0895-4356 (ISSN-L)
DOI: 10.1016/j.jclinepi.2010.10.006
OpenAlex: W1979334594
Cited by: 368
References: 32
Citations by year: 2026: 20 · 2025: 13 · 2024: 25 · 2023: 29 · 2022: 21 · 2021: 30 · 2020: 28 · 2019: 40 · 2018: 27 · 2017: 27 · 2016: 27 · 2015: 21 · 2014: 25 · 2013: 16 · 2012: 14
FWCI: 2.92
Type: journal article
Language: en
Open access: closed
Licence: www.elsevier.com/tdm/userlicense/1.0/ (tdm)
www.elsevier.com/legal/tdmrep-license (tdm)
Locations: Journal of Clinical Epidemiology (publishedVersion)
PubMed (publishedVersion)
Funders: American College of Gastroenterology
Child and Family Research Institute
Public Health Agency of Canada
Canadian Child Health Clinician Scientist Program
Hospital for Sick Children
Institut pour la Recherche en Santé Publique
Canadian Institutes of Health Research
Topics: Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
Health Sciences › Health Professions › Health Information Management › Medical Coding and Health Information
Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
Keywords: administrative health data, data validation, identification algorithm, reporting guidelines, reporting quality, diagnostic accuracy, positive predictive value, negative predictive value, misclassification bias, disease prevalence, incidence estimation, health care utilization
MeSH terms: Algorithms, Forms and Records Control, Health Services Research, Health Status, Hospital Records, Humans, Quality Control, Guidelines as Topic, International Classification of Diseases, Validation Studies as Topic, Translational Research, Biomedical, Checklist

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Development and use of reporting guidelines for assessing the quality of validation studies of health administrative data

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks