The DOI matches its Crossref record; 164 of the 164 references it cites were found in a registry or an index.
A methodological review of how heterogeneity has been examined in systematic reviews of diagnostic test accuracy
Open accessAbstract
OBJECTIVES: To review how heterogeneity has been examined in systematic reviews of diagnostic test accuracy studies. DATA SOURCES: Centre for Reviews and Dissemination's Database of Abstracts of Reviews of Effects (DARE). REVIEW METHODS: Systematic reviews that evaluated a diagnostic or screening test by including studies that compared a test with a reference test were identified from DARE. Reviews for which structured abstracts had been written up to December 2002 were screened for inclusion. Data extraction was undertaken using standardised data extraction forms. RESULTS: A total of 189 systematic reviews met the inclusion criteria. The median number of studies included was 18. Meta-analyses have a higher number with a median of 22 studies compared with 11 for narrative reviews. Graphical plots to demonstrate the spread in study results were provided in 56% of meta-analyses; in 79% these were plots of sensitivity and specificity in the receiver operating characteristic (ROC) space. Statistical tests to identify heterogeneity were used in 32% of reviews: 41% of meta-analyses and 9% of reviews using narrative syntheses. The chi-squared test and Fisher's exact test to assess heterogeneity in individual aspects of test performance were the most common. In contrast, only 16% of meta-analyses used correlation coefficients to test for a threshold effect. A narrative synthesis was used in 30% of reviews. Of the meta-analyses, 52% carried out statistical pooling alone, 18% conducted only summary receiver operator characteristic (SROC) analyses and 30% used both methods of statistical synthesis. For those undertaking SROC analyses, the main differences between the models used were the weights chosen for the regression models, although in 42% of cases the use of, or choice of, weight was not provided. The proportion of reviews using statistical pooling alone has declined from 67% in 1995 to 42% in 2001, with a corresponding increase in the use of SROC methods, from 33% to 58%. However, two-thirds of those using SROC methods also carried out statistical pooling rather than presenting only SROC models. Reviews using SROC analyses also tended to present their results as some combination of sensitivity and specificity rather than using alternative, perhaps less clinically meaningful, means of data presentation such as diagnostic odds ratios. Three-quarters of meta-analyses attempted to investigate statistically possible sources of variation, using subgroup analysis or regression analysis. The impact of clinical or socio-demographic variables was investigated in 74% of these reviews and test- or threshold-related variables in 79%. At least one quality-related variable was investigated in 63% of reviews. Within this subset, the most commonly considered variables were the use of blinding, sample size, the reference test used and the avoidance of verification bias. CONCLUSIONS: The emphasis on pooling individual aspects of diagnostic test performance and the under-use of statistical tests and graphical approaches to identify heterogeneity perhaps reflect the uncertainty in the most appropriate methods to use and also greater familiarity with more traditional indices of test accuracy. This indicates the difficulty and complexity of carrying out such reviews. In these cases it is strongly suggested that meta-analyses are carried out with the involvement of a statistician familiar with the field. Further methodological work on the statistical methods available for combining diagnostic test accuracy studies is needed, as are sufficiently large, prospectively designed primary studies of diagnostic test accuracy comparing two or more tests for the same target disorder. Use of individual patient data meta-analysis in diagnostic test accuracy reviews should be explored to allow heterogeneity to be considered in more detail.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Measuring inconsistency in meta-analyses
Higgins, J. P. T., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003) · BMJ · doi:10.1136/bmj.327.7414.557 · Cited by 63,139
Self-citationMeta-analysis in clinical trials
DerSimonian, R., & Laird, N. (1986) · Controlled Clinical Trials · doi:10.1016/0197-2456(86)90046-2 · Cited by 39,552
Quantifying heterogeneity in a meta‐analysis
Higgins, J. P. T., & Thompson, S. G. (2002) · Statistics in Medicine · doi:10.1002/sim.1186 · Cited by 36,763
The meaning and use of the area under a receiver operating characteristic (ROC) curve.
Hanley, J. A., & McNeil, B. J. (1982) · Radiology · doi:10.1148/radiology.143.1.7063747 · Cited by 21,856
Evidence-Based Medicine: How to Practice and Teach EBM
Sackett, D. L. (2000) · Cited by 5,950
Deeks, J. J., Altman, D. G., & Bradburn, M. J. (2001) · Systematic Reviews in Health Care · doi:10.1002/9780470693926.ch15 · Cited by 2,163
Self-citationEmpirical evidence of design-related bias in studies of diagnostic tests.
Lijmer, J. G. (1999) · JAMA · doi:10.1001/jama.282.11.1061 · Cited by 1,861
Problems of Spectrum and Bias in Evaluating the Efficacy of Diagnostic Tests
Ransohoff, D. F., & Feinstein, A. R. (1978) · New England Journal of Medicine · doi:10.1056/nejm197810262991705 · Cited by 1,734
Serum cystatin C is superior to serum creatinine as a marker of kidney function: A meta-analysis
Dharnidharka, V. R., Kwon, C., & Stevens, G. (2002) · American Journal of Kidney Diseases · doi:10.1053/ajkd.2002.34487 · Cited by 1,663
Evidence-Based Medicine: How to Practice and Teach EBM
Satya-Murti, S. (2000) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.284.18.2382 · Cited by 1,643
Cited by
Systematic reviews: CRD's guidance for undertaking reviews in health care
Tacconelli, E. (2010) · The Lancet Infectious Diseases · doi:10.1016/s1473-3099(10)70065-7 · Cited by 3,245
Kahn (2012) · doi:10.1378/chest.11-2296 · Cited by 2,693
Prevention of VTE in Nonorthopedic Surgical Patients
Gould, M. K., Garcia, D. A., Wren, S. M., Karanicolas, P. J., Arcelus, J. I., Heit, J. A., & Samama, C. M. (2012) · Chest · doi:10.1378/chest.11-2297 · Cited by 2,131
Meta-DiSc: a software for meta-analysis of test accuracy data
Zamora, J., Abraira, V., Muriel, A., Khan, K., & Coomarasamy, A. (2006) · BMC Medical Research Methodology · doi:10.1186/1471-2288-6-31 · Cited by 1,883
Gu, P., Zhao, Y.-Z., Jiang, L.-Y., Zhang, W., Xin, Y., & Han, B.-H. (2009) · European Journal of Cancer · doi:10.1016/j.ejca.2008.11.043 · Cited by 620
Hegedus, E. J., Goode, A. P., Cook, C. E., Michener, L., Myer, C. A., Myer, D. M., & Wright, A. A. (2012) · British Journal of Sports Medicine · doi:10.1136/bjsports-2012-091066 · Cited by 538
Lee, J., Kim, K. W., Choi, S. H., Huh, J., & Park, S. H. (2015) · Korean Journal of Radiology · doi:10.3348/kjr.2015.16.6.1188 · Cited by 474
Antithrombotic and Thrombolytic Therapy for Ischemic Stroke
Lansberg, M. G., O'Donnell, M. J., Khatri, P., Lang, E. S., Nguyen-Huynh, M. N., Schwartz, N. E., Sonnenberg, F. A., Schulman, S., Vandvik, P. O., Spencer, F. A., et al. (2012) · Chest · doi:10.1378/chest.11-2302 · Cited by 455
Liu, S. S., & Wu, C. L. (2007) · Anesthesia & Analgesia · doi:10.1213/01.ane.0000255040.71600.41 · Cited by 435
Steinfort, D. P., Khor, Y. H., Manser, R. L., & Irving, L. B. (2011) · European Respiratory Journal · doi:10.1183/09031936.00075310 · Cited by 418
Janda, S., Shahidi, N., Gin, K., & Swiston, J. (2011) · Heart · doi:10.1136/hrt.2010.212084 · Cited by 375
The development of a quality appraisal tool for studies of diagnostic reliability (QAREL)
Lucas, N. P., Macaskill, P., Irwig, L., & Bogduk, N. (2010) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2009.10.002 · Cited by 374
Hodgkinson, J., Mant, J., Martin, U., Guo, B., Hobbs, F. D. R., Deeks, J. J., Heneghan, C., Roberts, N., & McManus, R. J. (2011) · BMJ · doi:10.1136/bmj.d3621 · Cited by 370
Schmidt, R. L., Hall, B. J., Wilson, A. R., & Layfield, L. J. (2011) · American Journal of Clinical Pathology · doi:10.1309/ajcpoie0cznat6sq · Cited by 362
Hewitt, C., Gilbody, S., Brealey, S., Paulden, M., Palmer, S., Mann, R., Green, J., Morrell, J., Barkham, M., Light, K., et al. (2009) · Health Technology Assessment · doi:10.3310/hta13360 · Cited by 330
Deep learning for caries detection: A systematic review
Mohammad-Rahimi, H., Motamedian, S. R., Rohban, M. H., Krois, J., Uribe, S. E., Mahmoudinia, E., Rokhshad, R., Nadimi, M., & Schwendicke, F. (2022) · Journal of Dentistry · doi:10.1016/j.jdent.2022.104115 · Cited by 313
van Agtmaal, M. J. M., Houben, A. J. H. M., Pouwer, F., Stehouwer, C. D. A., & Schram, M. T. (2017) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2017.0984 · Cited by 308
Boland, G. W. L., Dwamena, B. A., Jagtiani Sangwaiya, M., Goehler, A. G., Blake, M. A., Hahn, P. F., Scott, J. A., & Kalra, M. K. (2011) · Radiology · doi:10.1148/radiol.11100569 · Cited by 282
Diagnostic test accuracy: methods for systematic review and meta-analysis
Campbell, J. M., Klugar, M., Ding, S., Carmody, D. P., Hakonsen, S. J., Jadotte, Y. T., White, S., & Munn, Z. (2015) · International Journal of Evidence-Based Healthcare · doi:10.1097/xeb.0000000000000061 · Cited by 193
Empirical evidence of the importance of comparative studies of diagnostic test accuracy
Takwoingi, Y., Leeflang, M. M. G., & Deeks, J. J. (2013) · Annals of Internal Medicine · doi:10.7326/0003-4819-158-7-201304020-00006 · Cited by 187
Self-citationPrevalence of burnout syndrome in university students: a systematic review
Rosales-Ricardo, Y., Rizzo-Chunga, F., Mocha-Bonilla, J., & Ferreira, J. P. (2021) · Salud mental · doi:10.17711/sm.0185-3325.2021.013 · Cited by 121
Management of pelvic organ prolapse and quality of life: a systematic review and meta-analysis
Doaee, M., Moradi-Lakeh, M., Nourmohammadi, A., Razavi-Ratki, S. K., & Nojomi, M. (2014) · International Urogynecology Journal · doi:10.1007/s00192-013-2141-8 · Cited by 88
van Enst, W. A., Scholten, R. J. P. M., Whiting, P., Zwinderman, A. H., & Hooft, L. (2014) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2014.05.008 · Cited by 69
de Sousa, M. R., Morillo, C. A., Rabelo, F. T., Filho, A. M. N., & Ribeiro, A. L. P. (2008) · European Journal of Heart Failure · doi:10.1016/j.ejheart.2008.07.002 · Cited by 72
JOSEPH, S., SELVARAJ, J., MANI, I., KUMARAGURUPARI, T., SHANG, X., MUDGIL, P., RAVILLA, T., & HE, M. (2024) · American Journal of Ophthalmology · doi:10.1016/j.ajo.2024.02.012 · Cited by 69
Lefebvre, C., Glanville, J., Beale, S., Boachie, C., Duffy, S., Fraser, C., Harbour, J., McCool, R., & Smith, L. (2017) · Health Technology Assessment · doi:10.3310/hta21690 · Cited by 49
Yu, D., Sui, L., & Zhang, N. (2020) · Journal of Ultrasound in Medicine · doi:10.1002/jum.15123 · Cited by 33
Michelet, D., Julien-Marsollier, F., Vacher, T., Bellon, M., Skhiri, A., Bruneau, B., Fournier, J., Diallo, T., Luce, V., Brasher, C., et al. (2019) · Sleep Medicine · doi:10.1016/j.sleep.2018.09.027 · Cited by 22
Yu, D., Zhang, N., Sun, H., & Liu, M. (2023) · Journal of Clinical Ultrasound · doi:10.1002/jcu.23507 · Cited by 5
Metadata
- Authors
- Dinnes, J., Deeks, J., Kirby, J., & Roderick, P.
- Venue
- Health Technology Assessment
- Published
- 2005-03
- Volume
- 9
- Issue
- 12
- Pages
- 1-113, iii
- ISSN
- 1366-5278 (ISSN-L)
- 2046-4924
- OpenAlex
- W2005942667
- Cited by
- 438
- References
- 164
- FWCI
- 2.67
- Type
- journal article
- Language
- en
- Open access
- diamond
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.