Cited by 369Meta-analysis and systematic reviewsMedical Coding and Health InformationStatistical Methods in Epidemiology

The DOI matches its Crossref record; 33 of the 33 references it cites were found in a registry or an index.

Development and use of reporting guidelines for assessing the quality of validation studies of health administrative data

Benchimol, Eric I., Manuel, Douglas G., To, Teresa, Griffiths, Anne M., Rabeneck, Linda, Guttmann, Astrid·2011·Journal of Clinical Epidemiology

DOI 10.1016/j.jclinepi.2010.10.006

Open this paper in CoChat — free account

Abstract

Background and objectives Validation of health administrative data for identifying patients with different health states (diseases and conditions) is a research priority, but no guidelines exist for ensuring quality. We created reporting guidelines for studies validating administrative data identification algorithms and used them to assess the quality of reporting of validation studies in the literature.Methods Using Standards for Reporting of Diagnostic accuracy (STARD) criteria as a guide, we created a 40-item checklist of items with which identification accuracy studies should be reported. A systematic review identified studies that validated identification algorithms using administrative data. We used the checklist to assess the quality of reporting.Results In 271 included articles, goals and data sources were well reported but few reported four or more statistical estimates of accuracy (36.9%). In 65.9% of studies reporting positive predictive value (PPV)/negative predictive value (NPV), the prevalence of disease in the validation cohort was higher than in the administrative data, potentially falsely elevating predictive values. Subgroup accuracy (53.1%) and 95% confidence intervals for accuracy measures (35.8%) were also underreported.Conclusions The quality of studies validating health states in the administrative data varies, with significant deficits in reporting of markers of diagnostic accuracy, including the appropriate estimation of PPV and NPV. These omissions could lead to misclassification bias and incorrect estimation of incidence and health services utilization rates. Use of a reporting checklist, such as the one created for this study by modifying the STARD criteria, could improve the quality of reporting of validation studies, allowing for accurate application of algorithms, and interpretation of research using health administrative data.

Integrity signals

Registry

Crossref · 10.1016/j.jclinepi.2010.10.006

Cites

33 of 33 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement: Guidelines for Reporting Observational Studies
von Elm, E., Altman, D. G., Egger, M., Pocock, S. J., Gøtzsche, P. C., Vandenbroucke, J. P., & Initiative, F. T. S. (2007) · PLoS Medicine · doi:10.1371/journal.pmed.0040296 · Cited by 23,020
Towards complete and accurate reporting of studies of diagnostic accuracy: The STARD Initiative.
Bossuyt, P. M., Reitsma, J. B., Bruns, D. E., Gatsonis, C. A., Glasziou, P. P., Irwig, L. M., Lijmer, J. G., Moher, D., Rennie, D., de Vet, H. C. W., et al. (2003) · Annals of Internal Medicine · doi:10.7326/0003-4819-138-1-200301070-00010 · Cited by 3,219
Towards complete and accurate reporting of studies of diagnostic accuracy: the STARD initiative
Bossuyt, P. M., Reitsma, J. B., Bruns, D. E., Gatsonis, C. A., Glasziou, P. P., Irwig, L. M., Lijmer, J. G., Moher, D., Rennie, D., & de Vet, H. C. W. (2003) · BMJ · doi:10.1136/bmj.326.7379.41 · Cited by 3,215
Improving the Quality of Reporting of Randomized Controlled Trials: The CONSORT Statement
Begg, C. (1996) · JAMA · doi:10.1001/jama.1996.03540080059030 · Cited by 3,127
The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies
von Elm, E., Altman, D. G., Egger, M., Pocock, S. J., Gøtzsche, P. C., & Vandenbroucke, J. P. (2007) · Preventive Medicine · doi:10.1016/j.ypmed.2007.08.012 · Cited by 2,698
Statistics Notes: Diagnostic tests 2: predictive values
Altman, D. G., & Bland, J. M. (1994) · BMJ · doi:10.1136/bmj.309.6947.102 · Cited by 1,711
Diabetes in Ontario
Hux, J. E., Ivis, F. J., Flintoft, V. F., & Bica, A. (2002) · Diabetes Care · doi:10.2337/diacare.25.3.512 · Cited by 1,290
The CONSORT Statement
Cha, M., Eastwaad, S., Hartan, R., Maher, D., Olkin, I., Pitkin, R., Rennie, D., Schulz, K. F., Simel, D. L., & Straup, D. F. (1996) · Cited by 736
Epidemiology of Crohn's Disease and Ulcerative Colitis in a Central Canadian Province: A Population-based Study
Bernstein, C. N., Blanchard, J., Rawsthorne, P. E., & Wajda, A. (1999) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a009735 · Cited by 661
Medical Subject Headings
ROGERS, F. B. (1965) · Nature · doi:10.1038/205236a0 · Cited by 571

Show all 32 references

Sorted by citations

The Reporting of Studies Conducted Using Observational Routinely-Collected Health Data (RECORD) statement.
Benchimol, E. I., Smeeth, L., Guttmann, A., Harron, K., Moher, D., Petersen, I., Sørensen, H. T., von Elm, E., Langan, S. M., & RECORD Working Committee (2015) · PLOS Medicine · doi:10.1371/journal.pmed.1001885 · Cited by 5,749
Self-citation
A catalogue of reporting guidelines for health research
Simera, I., Moher, D., Hoey, J., Schulz, K. F., & Altman, D. G. (2010) · European Journal of Clinical Investigation · doi:10.1111/j.1365-2362.2009.02234.x · Cited by 1,047
The reporting of studies conducted using observational routinely collected health data statement for pharmacoepidemiology (RECORD-PE)
Langan, S., Schmidt, S. A. J., Wing, K., Ehrenstein, V., Nicholls, S. G., Filion, K. B., Klungel, O. H., Petersen, I., Sørensen, H. T., Dixon, W. G., et al. (2018) · BMJ · doi:10.1136/bmj.k3532 · Cited by 673
Self-citation
Population Density and Risk of Inflammatory Bowel Disease: A Prospective Population-Based Study in 13 Countries or Regions in Asia-Pacific
Ng, S. C., Kaplan, G. G., Tang, W., Banerjee, R., Adigopula, B., Underwood, F. E., Tanyingoh, D., Wei, S.-C., Lin, W.-C., Lin, H.-H., et al. (2019) · American Journal of Gastroenterology · doi:10.1038/s41395-018-0233-2 · Cited by 261
Can mental health diagnoses in administrative data be used for research? A systematic review of the accuracy of routinely collected diagnoses
Davis, K. A. S., Sudlow, C. L. M., & Hotopf, M. H. (2016) · BMC Psychiatry · doi:10.1186/s12888-016-0963-x · Cited by 264
Systematic review and assessment of validated case definitions for depression in administrative data
Fiest, K. M., Jetté, N., Quan, H., Smith, C., Metcalfe, A., Patten, S. B., & Beck, C. A. (2014) · BMC Psychiatry · doi:10.1186/s12888-014-0289-5 · Cited by 255
Relation of completeness of reporting of health research to journals' endorsement of reporting guidelines: systematic review
Stevens, A., Shamseer, L., Weinstein, E., Yazdi, F., Turner, L., Thielman, J., Altman, D. G., Hirst, A., Hoey, J., Palepu, A., et al. (2014) · BMJ · doi:10.1136/bmj.g3804 · Cited by 242
Ascertainment of chronic diseases using population health data: a comparison of health administrative data and patient self-report
Muggah, E., Graves, E., Bennett, C. L., & Manuel, D. G. (2013) · BMC Public Health · doi:10.1186/1471-2458-13-16 · Cited by 229
Self-citation
Validation of international algorithms to identify adults with inflammatory bowel disease in health administrative data from Ontario, Canada
Benchimol, E. I., Guttmann, A., Mack, D. R., Nguyen, G. C., Marshall, J. K., Gregor, J. C., Wong, J., Forster, A. J., & Manuel, D. G. (2014) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2014.02.019 · Cited by 224
Self-citation
The REporting of Studies Conducted Using Observational Routinely-Collected Health Data (RECORD) Statement: Methods for Arriving at Consensus and Developing Reporting Guidelines
Nicholls, S. G., Quach, P., von Elm, E., Guttmann, A., Moher, D., Petersen, I., Sørensen, H. T., Smeeth, L., Langan, S. M., & Benchimol, E. I. (2015) · PLOS ONE · doi:10.1371/journal.pone.0125620 · Cited by 217
Self-citation
Tradeoffs between accuracy measures for electronic health care data algorithms
Chubak, J., Pocobelli, G. S., & Weiss, N. S. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.09.002 · Cited by 221
Comorbidity in Multiple Sclerosis
Magyari, M., & Sørensen, P. S. (2020) · Frontiers in Neurology · doi:10.3389/fneur.2020.00851 · Cited by 211
Recommendations for optimal ICD codes to study neurologic conditions: A systematic review
St. Germaine-Smith, C., Metcalfe, A., Pringsheim, T., Roberts, J. I., Beck, C. A., Hemmelgarn, B. R., McChesney, J., Quan, H., & Jette, N. (2012) · Neurology · doi:10.1212/wnl.0b013e3182684707 · Cited by 206
Validity of administrative data in recording sepsis: a systematic review
Jolley, R., Sawka, K. J., Yergens, D., Quan, H., Jetté, N., & Doig, C. J. (2015) · Critical Care · doi:10.1186/s13054-015-0847-3 · Cited by 196
Reporting to Improve Reproducibility and Facilitate Validity Assessment for Healthcare Database Studies V1.0
Wang, S. V., Schneeweiß, S., Berger, M. L., Brown, J. S., de Vries, F., Douglas, I., Gagne, J. J., Gini, R., Klungel, O. H., Mullins, C. D., et al. (2017) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.4295 · Cited by 184
Validation of Case-Finding Algorithms Derived from Administrative Data for Identifying Adults Living with Human Immunodeficiency Virus Infection
Antoniou, T., Zagorski, B., Loutfy, M. R., Strıke, C., & Glazier, R. H. (2011) · PLoS ONE · doi:10.1371/journal.pone.0021748 · Cited by 179
Quebec Integrated Chronic Disease Surveillance System (QICDSS), an innovative approach
Blais, C., Jean, S., Sirois, C., Rochette, L., Plante, C., Larocque, I., Doucet, M., Ruel, G., Simard, M., Gamache, P., et al. (2014) · Chronic diseases and injuries in Canada · doi:10.24095/hpcdp.34.4.06 · Cited by 161
An administrative data validation study of the accuracy of algorithms for identifying rheumatoid arthritis: the influence of the reference standard on algorithm performance
Widdifield, J., Bombardier, C., Bernatsky, S., Paterson, J. M., Green, D., Young, J., Ivers, N. M., Butt, D. A., Jaakkimainen, R. L., Thorne, C., et al. (2014) · BMC Musculoskeletal Disorders · doi:10.1186/1471-2474-15-216 · Cited by 159
Accuracy of Canadian Health Administrative Databases in Identifying Patients With Rheumatoid Arthritis: A Validation Study Using the Medical Records of Rheumatologists
Widdifield, J., Bernatsky, S., Paterson, J. M., Tu, K., Ng, R., Thorne, C., Pope, J. E., & Bombardier, C. (2013) · Arthritis Care & Research · doi:10.1002/acr.22031 · Cited by 153
Accuracy of administrative data for surveillance of healthcare-associated infections: a systematic review
van Mourik, M. S. M., van Duijn, P. J., Moons, K. G. M., Bonten, M. J. M., & Lee, G. M. (2015) · BMJ Open · doi:10.1136/bmjopen-2015-008424 · Cited by 140
Administrative Data Algorithms to Identify Second Breast Cancer Events Following Early-Stage Invasive Breast Cancer
Chubak, J., Yu, O., Pocobelli, G. S., Lamerato, L. E., Webster, J., Prout, M. N., Yood, M. U., Barlow, W. E., & Buist, D. S. M. (2012) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/djs233 · Cited by 117
Validity of cancer diagnosis in the National Health Insurance database compared with the linked National Cancer Registry in Taiwan
Kao, W., Hong, J., See, L., Yu, H., Hsu, J., Chou, I., Chou, W., Chiou, M., Wang, C., & Kuo, C. (2017) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.4267 · Cited by 116
Case Definitions for Acute Myocardial Infarction in Administrative Databases and Their Impact on In‐Hospital Mortality Rates
Metcalfe, A., Neudam, A., Forde, S., Liu, M., Drösler, S. E., Quan, H., & Jetté, N. (2012) · Health Services Research · doi:10.1111/j.1475-6773.2012.01440.x · Cited by 114
Accuracy of Administrative Data for the Coding of Acute Stroke and TIAs
Hall, R., Mondor, L., Porter, J., Fang, J., & Kapral, M. K. (2016) · Canadian Journal of Neurological Sciences / Journal Canadien des Sciences Neurologiques · doi:10.1017/cjn.2016.278 · Cited by 113
Risk Factors for Periprosthetic Joint Infection Following Primary Total Hip Arthroplasty
a, T. M. A. C. (. (2019) · Journal of Bone and Joint Surgery · doi:10.2106/jbjs.19.00537 · Cited by 119
Untapped Potential of Observational Research to Inform Clinical Decision Making: American Society of Clinical Oncology Research Statement
Visvanathan, K., Levit, L. A., Raghavan, D., Hudis, C. A., Wong, S. L., Dueck, A. C., & Lyman, G. H. (2017) · Journal of Clinical Oncology · doi:10.1200/jco.2017.72.6414 · Cited by 112
Validity of International Classification of Diseases (ICD)-10 diagnosis codes for identification of acute heart failure hospitalization and heart failure with reduced versus preserved ejection fraction in a national Medicare sample
Bates, B. A., Akhabue, E., Nahass, M. M., Mukherjee, A., Hiltner, E., Rock, J., Wilton, B., Mittal, G., Visaria, A., Rua, M., et al. (2023) · Circulation: Cardiovascular Quality and Outcomes · doi:10.1161/circoutcomes.122.009078 · Cited by 118
Reporting to Improve Reproducibility and Facilitate Validity Assessment for Healthcare Database Studies V1.0
Wang, S. V., Schneeweiß, S., Berger, M. L., Brown, J. S., de Vries, F., Douglas, I., Gagne, J. J., Gini, R., Klungel, O. H., Mullins, C. D., et al. (2017) · Value in Health · doi:10.1016/j.jval.2017.08.3018 · Cited by 116
Validity of Hospital Discharge Diagnosis Codes for Stroke
Jones, S., Gottesman, R. F., Shahar, E., Wruck, L. M., & Rosamond, W. D. (2014) · Stroke · doi:10.1161/strokeaha.114.006316 · Cited by 99
Development and Validation of an Algorithm to Accurately Identify Atopic Eczema Patients in Primary Care Electronic Health Records from the UK
Abuabara, K., Magyari, A., Hoffstad, O., Jabbar‐Lopez, Z. K., Smeeth, L. L., Williams, H. C., Gelfand, J. M., Margolis, D. J., & Langan, S. (2017) · Journal of Investigative Dermatology · doi:10.1016/j.jid.2017.03.029 · Cited by 94
Das RECORD-Statement zum Berichten von Beobachtungsstudien, die routinemäßig gesammelte Gesundheitsdaten verwenden
Benchimol, E. I., Smeeth, L. L., Guttmann, A., Harron, K. L., Hemkens, L. G., Moher, D., Petersen, I., Sørensen, H. T., von Elm, E. B., & Langan, S. (2016) · Zeitschrift für Evidenz Fortbildung und Qualität im Gesundheitswesen · doi:10.1016/j.zefq.2016.07.010 · Cited by 98
Self-citation
Incidence of Hospitalization for Respiratory Syncytial Virus Infection amongst Children in Ontario, Canada: A Population-Based Study Using Validated Health Administrative Data
Pisesky, A., Benchimol, E. I., Wong, C. A., Hui, C. P., Crowe, M. L., Bélair, M., Pojsupap, S., Karnauchow, T., O’Hearn, K., Yasseen, A. S., et al. (2016) · PLoS ONE · doi:10.1371/journal.pone.0150416 · Cited by 91
Self-citation
Association of chronic pain with comorbidities and health care utilization: a retrospective cohort study using health administrative data
Foley, H. E., KNIGHT, J. C., Ploughman, M., Asghari, S., & Audas, R. P. (2021) · Pain · doi:10.1097/j.pain.0000000000002264 · Cited by 88
Key considerations when using health insurance claims data in advanced data analyses: an experience report
Konrad, R., Zhang, W., Bjarndóttir, M. V., & Proaño, R. A. (2019) · Health Systems · doi:10.1080/20476965.2019.1581433 · Cited by 86
A Validation Study of Administrative Data Algorithms to Identify Patients with Parkinsonism with Prevalence and Incidence Trends
Butt, D. A., Tu, K., Young, J., Green, D., Wang, M., Ivers, N. M., Jaakkimainen, L., Lam, R., & Guttman, M. (2014) · Neuroepidemiology · doi:10.1159/000365590 · Cited by 83
Studies using English administrative data (Hospital Episode Statistics) to assess health-care outcomes—systematic review and recommendations for reporting
Sinha, S., Peach, G., Poloniecki, J. D., THOMPSON, M. M., & Holt, P. (2012) · European Journal of Public Health · doi:10.1093/eurpub/cks046 · Cited by 81
Helping everyone do better: a call for validation studies of routinely recorded health data
Ehrenstein, V., Petersen, I., Smeeth, L. L., Jick, S. S., Benchimol, E. I., Ludvigsson, J. F., & Sørensen, H. T. (2016) · Clinical Epidemiology · doi:10.2147/clep.s104448 · Cited by 79
Self-citation
Development and validation of novel algorithms to identify patients with inflammatory bowel diseases in Israel: an epi-IIRN group study
Friedman, M. Y., Leventer‐Roberts, M., Rosenblum, J., Zigman, N., Goren, I., Mourad, V., Lederman, N., Cohen, N., Matz, E., Dushnitzky, D., et al. (2018) · Clinical Epidemiology · doi:10.2147/clep.s151339 · Cited by 76
Self-citation
Systematic Review of Validation Studies of the Use of Administrative Data to Identify Serious Infections
Barber, C. E. H., Lacaille, D. V., & Fortin, P. R. (2013) · Arthritis Care & Research · doi:10.1002/acr.21959 · Cited by 72
Recommendations for respiratory syncytial virus surveillance at the national level
Teirlinck, A. C., Broberg, E., Berg, A. S., Campbell, H. J., Reeves, R. M., Carnahan, A., Lina, B., Pakarna, G., Bøås, H., Nohynek, H., et al. (2021) · European Respiratory Journal · doi:10.1183/13993003.03766-2020 · Cited by 72
Systematic Review and Critical Appraisal of Validation Studies to Identify Rheumatic Diseases in Health Administrative Databases
Widdifield, J., Labrecque, J. A., Lix, L. M., Paterson, J. M., Bernatsky, S., Tu, K., Ivers, N. M., & Bombardier, C. (2013) · Arthritis Care & Research · doi:10.1002/acr.21993 · Cited by 66
Validation study of health administrative data algorithms to identify individuals experiencing homelessness and estimate population prevalence of homelessness in Ontario, Canada
Richard, L., Hwang, S. W., Forchuk, C., Nisenbaum, R. A., Clemens, K. K., Wiens, K., Booth, R. G., Azimaee, M., & Shariff, S. Z. (2019) · BMJ Open · doi:10.1136/bmjopen-2019-030221 · Cited by 65
Quality of administrative health databases in Canada: A scoping review
Hinds, A. M., Lix, L. M., Smith, M. J., Quan, H., & Sanmartin, C. (2016) · Canadian Journal of Public Health · doi:10.17269/cjph.107.5244 · Cited by 63
Using linked administrative and disease-specific databases to study end-of-life care on a population level
Maetens, A., Schreye, R. D., Faes, K., Houttekier, D., Deliëns, L., Gielen, B., Gendt, C. D., Lusyne, P., Annemans, L., & Cohen, J. (2016) · BMC Palliative Care · doi:10.1186/s12904-016-0159-7 · Cited by 59
Big Data Research in Neurosurgery: A Critical Look at this Popular New Study Design
Oravec, C. S., Motiwala, M., Reed, K., Kondziolka, D. S., Barker, F. G., Michael, L. M., & Klimo, P. (2017) · Neurosurgery · doi:10.1093/neuros/nyx328 · Cited by 59
Accuracy of administrative data for identification of patients with infective endocarditis
Tan, C., Hansen, M., Cohen, G. N., Boyle, K., Daneman, N., & Adhikari, N. K. J. (2016) · International Journal of Cardiology · doi:10.1016/j.ijcard.2016.09.030 · Cited by 58
Using data linkage to electronic patient records to assess the validity of selected mental health diagnoses in English Hospital Episode Statistics (HES)
Davis, K. A. S., Bashford, O., Jewell, A., Shetty, H., Stewart, R. J., Sudlow, C. L. M., & Hotopf, M. H. (2018) · PLoS ONE · doi:10.1371/journal.pone.0195002 · Cited by 58
A comparison of methods to correct for misclassification bias from administrative database diagnostic codes
van Walraven, C. (2017) · International Journal of Epidemiology · doi:10.1093/ije/dyx253 · Cited by 57
Application and Validation of Case-Finding Algorithms for Identifying Individuals with Human Immunodeficiency Virus from Administrative Data in British Columbia, Canada
Nosyk, B., Colley, G., Yip, B., Chan, K., Heath, K., Lima, V. D., Gilbert, M., Hogg, R. S., Harrigan, P. R., Montaner, J. S. G., et al. (2013) · PLoS ONE · doi:10.1371/journal.pone.0054416 · Cited by 55
Identification of Childhood-Onset Inflammatory Bowel Disease in Swedish Healthcare Registers: A Validation Study
Mouratidou, N., Malmborg, P., Järås, J., Sigurdsson, G. V., Sandström, O., Fagerberg, U. L., Bröms, G., Ludvigsson, J. F., & Olén, O. (2022) · Clinical Epidemiology · doi:10.2147/clep.s358031 · Cited by 57
Setting the RECORD straight: developing a guideline for the REporting of studies Conducted using Observational Routinely collected Data
Sørensen, H. T., Langan, S., Benchimol, E. I., Guttmann, Moher, D., Petersen, I., Smeeth, L. L., Stanley, & von Elm, E. B. (2013) · Clinical Epidemiology · doi:10.2147/clep.s36885 · Cited by 54
Self-citation
Routinely collected data: the importance of high-quality diagnostic coding to research
Nicholls, S. G., Langan, S., & Benchimol, E. I. (2017) · Canadian Medical Association Journal · doi:10.1503/cmaj.170807 · Cited by 54
Self-citation
The validity of dementia diagnoses in routinely collected electronic health records in the United Kingdom: A systematic review
McGuinness, L. A., Warren‐Gash, C., Moorhouse, L. R., & Thomas, S. L. (2019) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.4669 · Cited by 56
Validation of a type 1 diabetes algorithm using electronic medical records and administrative healthcare data to study the population incidence and prevalence of type 1 diabetes in Ontario, Canada
Weisman, A., Tu, K., Young, J., Kumar, M., Austin, P. C., Jaakkimainen, L., Lipscombe, L., Aronson, R., & Booth, G. L. (2020) · BMJ Open Diabetes Research & Care · doi:10.1136/bmjdrc-2020-001224 · Cited by 62
Administrative data identify sickle cell disease: A critical review of approaches in U.S. health services research
Grosse, S. D., Green, N., & Reeves, S. L. (2020) · Pediatric Blood & Cancer · doi:10.1002/pbc.28703 · Cited by 51
Core concepts in pharmacoepidemiology: Validation of health outcomes of interest within real‐world healthcare databases
Weinstein, E. J., Ritchey, M. E., & Re, V. L. (2022) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.5537 · Cited by 53
Merging of the National Cancer Institute–funded cooperative oncology group data with an administrative data source to develop a more effective platform for clinical trial analysis and comparative effectiveness research: a report from the Children's Oncology Group
Aplenc, R., Fisher, B. T., Huang, Y. S., Li, Y., Alonzo, T. A., Gerbing, R. B., Hall, M., Bertoch, D., Keren, R., Seif, A. E., et al. (2012) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.3241 · Cited by 49
Validation of healthcare administrative data for the diagnosis of epilepsy
Franchi, C., Giussani, G., Messina, P., Montesano, M., Romi, S., Nobili, A., Fortino, I., Bortolotti, A., Merlino, L. G., Beghi, E., et al. (2013) · Journal of Epidemiology & Community Health · doi:10.1136/jech-2013-202528 · Cited by 49
Applying Quality Indicators For Administrative Databases To Evaluate End-Of-Life Care For Cancer Patients In Belgium
Schreye, R. D., Smets, T., Annemans, L., Deliëns, L., Gielen, B., Gendt, C. D., & Cohen, J. (2017) · Health Affairs · doi:10.1377/hlthaff.2017.0199 · Cited by 49
Lessons Learned From the Design and Implementation of Myocardial Infarction Adjudication Tailored for HIV Clinical Cohorts
Crane, H. M., Heckbert, S. R., Drozd, D. R., Budoff, M. J., Delaney, J. A., Rodriguez, C. V., Paramsothy, P., Lober, W. B., Burkholder, G., Willig, J. H., et al. (2014) · American Journal of Epidemiology · doi:10.1093/aje/kwu010 · Cited by 48
A cross-sectional bibliometric study showed suboptimal journal endorsement rates of STROBE and its extensions
Sharp, M. K., Tokalić, R., Melis, G. G., Wager, E., Altman, D. G., & Hren, D. (2018) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2018.11.006 · Cited by 46
A data mining approach for grouping and analyzing trajectories of care using claim data: the example of breast cancer
Jay, N., Nuemi, G., Gadreau, M., & Quantin, C. (2013) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-13-130 · Cited by 53
Validation of algorithms in studies based on routinely collected health data: general principles
Ehrenstein, V., Hellfritzsch, M., Kahlert, J. A., Langan, S., Urushihara, H., Marinac‐Dabic, D., Lund, J. L., Sørensen, H. T., & Benchimol, E. I. (2024) · American Journal of Epidemiology · doi:10.1093/aje/kwae071 · Cited by 49
Self-citation
Establishment of an 11-Year Cohort of 8733 Pediatric Patients Hospitalized at United States Free-standing Children’s Hospitals With De Novo Acute Lymphoblastic Leukemia From Health Care Administrative Data
Fisher, B. T., Harris, T., Torp, K., Seif, A. E., Shah, A., Huang, Y.-S. V., Bailey, L. C., Kersun, L. S., Reilly, A. F., Rheingold, S. R., et al. (2012) · Medical Care · doi:10.1097/mlr.0b013e31824deff9 · Cited by 46
Development and validation of case-finding algorithms for recurrence of breast cancer using routinely collected administrative data
Xu, Y., Kong, S., Cheung, W. Y., Bouchard‐Fortier, A., Dort, J. C., Quan, H., Buie, E. M., McKinnon, G., & Quan, M. L. (2019) · BMC Cancer · doi:10.1186/s12885-019-5432-8 · Cited by 46
Diagnostic accuracy of administrative data algorithms in the diagnosis of osteoarthritis: a systematic review
Shrestha, S., Dave, A. J., Losina, E., & Katz, J. N. (2016) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-016-0319-y · Cited by 45
The Growing Need to Understand Very Early Onset Inflammatory Bowel Disease
Zheng, H. B., de la Morena, M. T., & Suskind, D. L. (2021) · Frontiers in Immunology · doi:10.3389/fimmu.2021.675186 · Cited by 44
Validation of the Hospital Episode Statistics Outpatient Dataset in England
Thorn, J. C., Turner, E. L., Hounsome, L. S., Walsh, E. I., Donovan, J., Verne, J., Neal, D. E., Hamdy, F. C., Martin, R. M., & Noble, S. (2015) · PharmacoEconomics · doi:10.1007/s40273-015-0326-3 · Cited by 43
A validation study of a new classification algorithm to identify rheumatoid arthritis using administrative health databases: case–control and cohort diagnostic accuracy studies. Results from the RECord linkage On Rheumatic Diseases study of the Italian Society for Rheumatology
Carrara, G., Scirè, C. A., Zambon, A., Cimmino, M. A., Cerra, C., Caprioli, M., Cagnotto, G., Nicotra, F., Arfè, A., Migliazza, S., et al. (2015) · BMJ Open · doi:10.1136/bmjopen-2014-006029 · Cited by 42
Validation of multiple sclerosis diagnoses in the Swedish National Patient Register
Murley, C., Friberg, E., Hillert, J. A., Alexanderson, K. A. E., & Yang, F. (2019) · European Journal of Epidemiology · doi:10.1007/s10654-019-00558-7 · Cited by 41
In Reply: Big Data Research in Neurosurgery: A Critical Look at this Popular New Study Design
Oravec, C. S., Motiwala, M., Reed, K., Kondziolka, D. S., Barker, F. G., Michael, L. M., & Klimo, P. (2018) · Neurosurgery · doi:10.1093/neuros/nyy102 · Cited by 41
Unicompartmental Knee Arthroplasty: Survivorship and Risk Factors for Revision
Ekhtiari, S., Bozzo, A., Madden, K., Winemaker, M., Adili, A. F., & Wood, T. J. (2021) · Journal of Bone and Joint Surgery · doi:10.2106/jbjs.21.00346 · Cited by 42
Call to RECORD: the need for complete reporting of research using routinely collected health data
Benchimol, E. I., Langan, S., & Guttmann, A. (2012) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2012.09.006 · Cited by 39
Self-citation
Accuracy of Algorithms to Identify Pulmonary Arterial Hypertension in Administrative Data
Gillmeyer, K. R., Lee, M.-M., Link, A. P., Klings, E. S., Rinne, S. T., & Wiener, R. S. (2018) · CHEST Journal · doi:10.1016/j.chest.2018.11.004 · Cited by 40
Performance of Administrative Algorithms to Identify Interstitial Lung Disease in Rheumatoid Arthritis
England, B. R., Roul, P., Mahajan, T., Singh, N., Yu, F., Sayles, H. R., Cannon, G. W., Sauer, B. C., Baker, J. F., Curtis, J. R., et al. (2019) · Arthritis Care & Research · doi:10.1002/acr.24043 · Cited by 39
Validity of algorithms for identifying five chronic conditions in MedicineInsight, an Australian national general practice database
Havard, A., Manski-Nankervis, J.-A. E., Thistlethwaite, J. E., Daniels, B. J., Myton, R., Tu, K., & Chidwick, K. (2021) · BMC Health Services Research · doi:10.1186/s12913-021-06593-z · Cited by 39
Accuracy of diagnosis codes to identify febrile young infants using administrative data
Aronson, P. L., Williams, D. J., Thurm, C. W., Tieder, J. S., Alpern, E. R., Nigrovic, L. E., Schondelmeyer, A. C., Balamuth, F., Myers, A., McCulloh, R. J., et al. (2015) · Journal of Hospital Medicine · doi:10.1002/jhm.2441 · Cited by 38
Is the Quebec provincial administrative database a valid source for research on chronic non‐cancer pain?
Lacasse, A., Ware, M. A., Dorais, M., Lanctôt, H., & Choinière, M. (2015) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.3820 · Cited by 37
Mortality in the German Pharmacoepidemiological Research Database (GePaRD) compared to national data in Germany: results from a validation study
Ohlmeier, C., Langner, I., Hillebrand, K., Schmedt, N., Mikolajczyk, R., Riedel, O., & Garbe, E. (2015) · BMC Public Health · doi:10.1186/s12889-015-1943-7 · Cited by 36
Understanding the biases to sepsis surveillance and quality assurance caused by inaccurate coding in administrative health data
Schwarzkopf, D., Rose, N., Fleischmann-Struzek, C., Boden, B., Dorow, H., Edel, A., Friedrich, M. E., Gonnert, F. A., Götz, J., Gründling, M., et al. (2023) · Infection · doi:10.1007/s15010-023-02091-y · Cited by 36
Accuracy of administrative databases in detecting primary breast cancer diagnoses: a systematic review
Abraha, I., Montedori, A., Serraino, D., Orso, M., Giovannini, G., Scotti, V., Granata, A., Cozzolino, F., Fusco, M., & Bidoli, E. (2018) · BMJ Open · doi:10.1136/bmjopen-2017-019264 · Cited by 35
Assembly of a cohort of children treated for acute myeloid leukemia at free‐standing children's hospitals in the United States using an administrative database
Kavčič, M., Fisher, B. T., Torp, K., Li, Y., Huang, Y.-S. V., Seif, A. E., Vujković, M., & Aplenc, R. (2012) · Pediatric Blood & Cancer · doi:10.1002/pbc.24402 · Cited by 35
Structured Approach for Evaluating Strategies for Cancer Ascertainment Using Large-Scale Electronic Health Record Data
Earles, A., Liu, L., Bustamante, R., Coke, P., Lynch, J. A., Messer, K. S., Martı́nez, M. E., Murphy, J. D., Williams, C. D., Fisher, D. A., et al. (2018) · JCO Clinical Cancer Informatics · doi:10.1200/cci.17.00072 · Cited by 35
Validated administrative data based ICD-10 algorithms for chronic conditions: A systematic review
Kuang, A., Xu, C., Southern, D. A., Sandhu, N., & Quan, H. (2024) · Journal of Epidemiology and Population Health · doi:10.1016/j.jeph.2024.202744 · Cited by 36
Consensus Statements for the Use of Administrative Health Data in Rheumatic Disease Research and Surveillance
Bernatsky, S., Lix, L. M., O’Donnell, S. C., Lacaille, D. V., & CANRAD Network (2012) · The Journal of Rheumatology · doi:10.3899/jrheum.120835 · Cited by 34
Validation of administrative health data for the pediatric population: a scoping review
Shiff, N. J., Jama, S., Boden, C., & Lix, L. M. (2014) · BMC Health Services Research · doi:10.1186/1472-6963-14-236 · Cited by 34
Validation study: evaluation of the metrological quality of French hospital data for perinatal algorithms
Goueslard, K., Cottenet, J., Benzenine, É., Tubert‐Bitter, P., & Quantin, C. (2020) · BMJ Open · doi:10.1136/bmjopen-2019-035218 · Cited by 32
Feasibility of using administrative data to compare hospital performance in the EU
Groene, O., Kristensen, S. R., Arah, O. A., Thompson, C. A., Bartels, P. D., Suñol, R., Klazinga, N., Klazinga, N., Kringos, D. S., Lombarts, K. M. J. M. H., et al. (2014) · International Journal for Quality in Health Care · doi:10.1093/intqhc/mzu015 · Cited by 32
Validation of administrative case ascertainment algorithms for chronic childhood arthritis in Manitoba, Canada
Shiff, N. J., Oen, K., Rabbani, R., & Lix, L. M. (2017) · Rheumatology International · doi:10.1007/s00296-017-3734-1 · Cited by 31
A primer on quantitative bias analysis with positive predictive values in research using electronic health data
Newcomer, S. R., Xu, S., Kulldorff, M., Daley, M. F., FIREMAN, B. H., & Glanz, J. M. (2019) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocz094 · Cited by 34
Validity of Diagnostic Algorithms for Cardiovascular Diseases in Japanese Health Insurance Claims
Kanaoka, K., Iwanaga, Y., Okada, K., Terasaki, S., Nishioka, Y., Nakai, M., Kamon, D., Myojin, T., Soeda, T., Noda, T., et al. (2023) · Circulation Journal · doi:10.1253/circj.cj-22-0566 · Cited by 32
Trends in bacterial sepsis incidence and mortality in France between 2015 and 2019 based on National Health Data System (Système National des données de Santé (SNDS)): a retrospective observational study
Pandolfi, F., Guillemot, D., Watier, L., & Brun‐Buisson, C. (2022) · BMJ Open · doi:10.1136/bmjopen-2021-058205 · Cited by 30
The RECORD reporting guidelines: meeting the methodological and ethical demands of transparency in research using routinely-collected health data
Nicholls, S. G., Langan, S., Sørensen, H. T., Petersen, I., & Benchimol, E. I. (2016) · Clinical Epidemiology · doi:10.2147/clep.s110528 · Cited by 29
Self-citation
Prevalence of multiple sclerosis in Sardinia: A systematic cross-sectional multi-source survey
Urru, S. A. M., Antonelli, A., Sechi, G. M., & MS Working Group (2019) · Multiple Sclerosis Journal · doi:10.1177/1352458519828600 · Cited by 29
Distributed data networks: a blueprint for Big Data sharing and healthcare analytics
Popovic, J. R. (2016) · Annals of the New York Academy of Sciences · doi:10.1111/nyas.13287 · Cited by 29
Identifying multiple myeloma patients using data from the French health insurance databases
Palmaro, A., Gauthier, M., Conte, C., Grosclaude, P., Despas, F., & Lapeyre‐Mestre, M. (2017) · Medicine · doi:10.1097/md.0000000000006189 · Cited by 28
Pediatric acquired demyelinating syndromes: a nationwide validation study of the Danish National Patient Register
Boesen, M. S., Magyari, M., Born, A. P., & Thygesen, L. C. (2018) · Clinical Epidemiology · doi:10.2147/clep.s156997 · Cited by 28
The validity of heart failure diagnoses in the Finnish Hospital Discharge Register
Vuori, M. A., Laukkanen, J. A., Pietilä, A., Havulinna, A. S., Kähönen, M. A. P., Salomaa, V. V., Niiranen, T., & for the FinnGen investigators (2019) · Scandinavian Journal of Public Health · doi:10.1177/1403494819847051 · Cited by 27
Validating a popular outpatient antibiotic database to reliably identify high prescribing physicians for patients 65 years of age and older
Schwartz, K. L., Chen, C., Langford, B. J., Brown, K. A., Daneman, N., Johnstone, J., Wu, J. H., Leung, V., & Garber, G. E. (2019) · PLoS ONE · doi:10.1371/journal.pone.0223097 · Cited by 27
A strategy for validation of variables derived from large-scale electronic health record data
Liu, L., Bustamante, R., Earles, A., Demb, J., Messer, K. S., & Gupta, S. N. (2021) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2021.103879 · Cited by 27

Authors: Benchimol, E. I., Manuel, D. G., To, T., Griffiths, A. M., Rabeneck, L., & Guttmann, A.
Affiliations: University of Ottawa (CA)
Statistics Canada (CA)
University of Toronto (CA)
Corresponding author: Benchimol, Eric I.
ORCID: Benchimol, Eric I.
Manuel, Douglas G.
To, Teresa
Griffiths, Anne M.
Rabeneck, Linda
Guttmann, Astrid
Venue: Journal of Clinical Epidemiology
Publisher: Elsevier BV
Published: 2011-08
Volume: 64
Issue: 8
Pages: 821-829
ISSN: 0895-4356 (ISSN-L)
DOI: 10.1016/j.jclinepi.2010.10.006
OpenAlex: W1979334594
Cited by: 368
References: 32
Citations by year: 2026: 20 · 2025: 13 · 2024: 25 · 2023: 29 · 2022: 21 · 2021: 30 · 2020: 28 · 2019: 40 · 2018: 27 · 2017: 27 · 2016: 27 · 2015: 21 · 2014: 25 · 2013: 16 · 2012: 14
FWCI: 2.92
Type: journal article
Language: en
Open access: closed
Licence: www.elsevier.com/tdm/userlicense/1.0/ (tdm)
www.elsevier.com/legal/tdmrep-license (tdm)
Locations: Journal of Clinical Epidemiology (publishedVersion)
PubMed (publishedVersion)
Funders: American College of Gastroenterology
Child and Family Research Institute
Public Health Agency of Canada
Canadian Child Health Clinician Scientist Program
Hospital for Sick Children
Institut pour la Recherche en Santé Publique
Canadian Institutes of Health Research
Topics: Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
Health Sciences › Health Professions › Health Information Management › Medical Coding and Health Information
Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
Keywords: administrative health data, data validation, identification algorithm, reporting guidelines, reporting quality, diagnostic accuracy, positive predictive value, negative predictive value, misclassification bias, disease prevalence, incidence estimation, health care utilization
MeSH terms: Algorithms, Forms and Records Control, Health Services Research, Health Status, Hospital Records, Humans, Quality Control, Guidelines as Topic, International Classification of Diseases, Validation Studies as Topic, Translational Research, Biomedical, Checklist

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Development and use of reporting guidelines for assessing the quality of validation studies of health administrative data

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks