The DOI matches its Crossref record; 175 of the 175 references it cites were found in a registry or an index.
Evaluation of diagnostic tests when there is no gold standard. A review of methods
Open accessAbstract
OBJECTIVE: To generate a classification of methods to evaluate medical tests when there is no gold standard. METHODS: Multiple search strategies were employed to obtain an overview of the different methods described in the literature, including searches of electronic databases, contacting experts for papers in personal archives, exploring databases from previous methodological projects and cross-checking of reference lists of useful papers already identified. RESULTS: All methods available were classified into four main groups. The first method group, impute or adjust for missing data on reference standard, needs careful attention to the pattern and fraction of missing values. The second group, correct imperfect reference standard, can be useful if there is reliable information about the degree of imperfection of the reference standard and about the correlation of the errors between the index test and the reference standard. The third group of methods, construct reference standard, have in common that they combine multiple test results to construct a reference standard outcome including deterministic predefined rules, consensus procedures and statistical modelling (latent class analysis). In the final group, validate index test results, the diagnostic test accuracy paradigm is abandoned and research examines, using a number of different methods, whether the results of an index test are meaningful in practice, for example by relating index test results to relevant other clinical characteristics and future clinical events. CONCLUSIONS: The majority of methods try to impute, adjust or construct a reference standard in an effort to obtain the familiar diagnostic accuracy statistics, such as sensitivity and specificity. In situations that deviate only marginally from the classical diagnostic accuracy paradigm, these are valuable methods. However, in situations where an acceptable reference standard does not exist, applying the concept of clinical test validation can provide a significant methodological advance. All methods summarised in this report need further development. Some methods, such as the construction of a reference standard using panel consensus methods and validation of tests outwith the accuracy paradigm, are particularly promising but are lacking in methodological research. These methods deserve particular attention in future research.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“We based our literature search on the principle of theoretical saturation [17, 18], with the aim of identifying all relevant recommendations, rather than all relevant articles.”
From Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations (2018) · cites this paper as [18]
In the citing paper
“Locating methodological papers in electronic databases was challenging: relevant articles may be indexed differently and there is no key word to search for this type of article [18].”
From Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations (2018) · cites this paper as [18]
In the citing paper
“However, our search was guided by the methodological principle of theoretical saturation [17, 18].”
From Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations (2018) · cites this paper as [18]
In the citing paper
“Once a set of methodological articles covering all relevant methodological aspects is identified, additional searches add little to the ideas already present in the initial set of articles.”
From Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations (2018) · cites this paper as [18]
In the citing paper
“Theoretical saturation suggests that identifying every article is not required in methodological research.”
From Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations (2018) · cites this paper as [18]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Cochrane Handbook for Systematic Reviews of Interventions: Cochrane Book Series
(2008) · doi:10.1002/9780470712184 · Cited by 25,048
Statistical Analysis With Missing Data
Lahiff, M., Little, R. J. A., & Rubin, D. B. (1989) · Journal of the American Statistical Association · doi:10.2307/2289883 · Cited by 17,385
International Organization for Standardization
(2011) · Springer eBooks · doi:10.1007/springerreference_16847 · Cited by 12,504
Construct validity in psychological tests.
Cronbach, L. J., & Meehl, P. E. (1955) · Psychological Bulletin · doi:10.1037/h0040957 · Cited by 11,195
Health Measurement Scales: A Practical Guide to Their Development and Use
Streiner, D. L., Norman, G. R., & Cairney, J. (1989) · Cited by 9,595
Multiple imputation after 18+ years
Rubin, D. B. (1996) · Journal of the American Statistical Association · doi:10.1080/01621459.1996.10476908 · Cited by 4,072
Schafer, J. L. (1999) · Statistical Methods in Medical Research · doi:10.1177/096228029900800102 · Cited by 3,949
Qualitative Research: Consensus methods for medical and health services research
Jones, J., & Hunter, D. (1995) · BMJ · doi:10.1136/bmj.311.7001.376 · Cited by 3,471
Multiple Imputation After 18+ Years
Rubin, D. B. (1996) · Journal of the American Statistical Association · doi:10.2307/2291635 · Cited by 2,686
Health Measurement Scales: A Practical Guide to Their Development and Use
Vogelzang, J. L. (2015) · Journal of Nutrition Education and Behavior · doi:10.1016/j.jneb.2015.03.007 · Cited by 2,393
Research works printed but not linked: 13
- [6]Lijmer JG, Mol BWJ, Bonsel GJ, Prins MH, Bossuyt PM. Strategies for the evaluation of diagnostic technologies. Evaluation of diagnostic tests: from accuracy to outcome (Thesis). Amsterdam; 2001. pp. 15-28.
- [19]Whiting P , Rutjes AW, Reitsma JB, Glas AS, Bossuyt PM, Kleijnen J. Sources of variation and bias in studies of diagnostic accuracy: a systematic review. Ann Intern Med 2004; 140 :189-202.
- [20]Rutjes AW, Reitsma JB, Irwig L, Bossuyt PM. Partial and differential bias in diagnostic accuracy studies. Sources of bias and variation in diagnostic accuracy studies (Thesis). Amsterdam; 2005. pp. 31-44.
- [27]Cochrane Handbook for Systematic Reviews of Interventions 4.2.5 [updated May 2005]. Chichester Wiley; 2006.
- [32]Diamond GA. Off Bayes: effect of verification bias on posterior probabilities calculated using Bayes' theorem. Med Decis Making 1992; 12 :22-31.
- [41]Zhou XH, Higgs RE. Assessing the relative accuracies of two screening tests in the presence of verification bias. Stat Med 2000; 19 :1697-705.
- [42]Molenberghs G, Goetghebeur EJ, Lipsitz SR, Kenward MG, Lesaffre E, Michiels B. Missing data perspectives of the fluvoxamine data set: a review. Stat Med 1999; 18 :2449-64.
- [63]Kline JA, Israel EG, Michelson EA, O'Neil BJ, Plewa MC, Portelli DC. Diagnostic accuracy of a bedside Ddimer assay and alveolar dead-space measurement for rapid exclusion of pulmonary embolism: a multicenter study. JAMA 2001; 285 :761-8.
- [68]Hadgu A. Bias in the evaluation of DNAamplification tests for detecting Chlamydia trachomatis. Stat Med 1997; 16 :1391-9.
- [87]Committee on Quality Management, Japan Society of Clinical Chemistry. Proposed standard for matrix reference materials for quantitative laboratory methods. JPN J Clin Chem 2003; 32 :180-5.
- [97]Bertrand P , Benichou J, Grenier P , Chastang C. Hui and Walter's latent-class reference-free approach may be more useful in assessing agreement than diagnostic performance. J Clin Epidemiol 2005; 58 :688-700.
- [103]Fletcher RH, Fletcher SW, Wagner EH. Abnormality. In Satterfield TS, editor. Clinical epidemiology: the essentials . Baltimore, MD: Williams and Wilkins; 1996. pp. 22-24.
- [107]Fryback DG, Thornbury JR. The efficacy of diagnostic imaging. Med Decis Making 1991; 11 : 88-94.
Cited by
Sorted by citations
Moons, K. G. M., Altman, D. G., Reitsma, J. B., Ioannidis, J. P. A., Macaskill, P., Steyerberg, E. W., Vickers, A. J., Ransohoff, D. F., & Collins, G. S. (2015) · Annals of Internal Medicine · doi:10.7326/m14-0698 · Cited by 5,147
Self-citationMoons, K. G. M., de Groot, J. A. H., Bouwmeester, W., Vergouwe, Y., Mallett, S., Altman, D. G., Reitsma, J. B., & Collins, G. S. (2014) · PLoS Medicine · doi:10.1371/journal.pmed.1001744 · Cited by 2,018
Self-citationMoons, K. G. M., Wolff, R. F., Riley, R. D., Whiting, P. F., Westwood, M., Collins, G. S., Reitsma, J. B., Kleijnen, J., & Mallett, S. (2019) · Annals of Internal Medicine · doi:10.7326/m18-1377 · Cited by 1,669
Self-citationHeilbronner, R. L., Sweet, J. J., Morgan, J. E., Larrabee, G. J., Millis, S. R., & Participants1, C. (2009) · The Clinical Neuropsychologist · doi:10.1080/13854040903155063 · Cited by 869
Proportion of hospital readmissions deemed avoidable: a systematic review
van Walraven, C., Bennett, C. L., Jennings, A., Austin, P. C., & Forster, A. J. (2011) · Canadian Medical Association Journal · doi:10.1503/cmaj.101860 · Cited by 732
European Respiratory Society guidelines for the diagnosis of primary ciliary dyskinesia
Lucas, J. S., Barbato, A., Collins, S. A., Goutaki, M., Behan, L., Caudri, D., Dell, S., Eber, E., Escudier, E., Hirst, R. A., et al. (2016) · European Respiratory Journal · doi:10.1183/13993003.01090-2016 · Cited by 706
Screening tests: a review with examples
Maxim, L. D., Niebo, R., & Utell, M. J. (2014) · Inhalation Toxicology · doi:10.3109/08958378.2014.955932 · Cited by 598
Degos, F., Perez, P., Roche, B., Mahmoudi, A., Asselineau, J., Voitot, H., & Bédossa, P. (2010) · Journal of Hepatology · doi:10.1016/j.jhep.2010.05.035 · Cited by 429
Mueller, M., D’Addario, M., Egger, M., Cevallos, M., Dekkers, O., Mugglin, C., & Scott, P. (2018) · BMC Medical Research Methodology · doi:10.1186/s12874-018-0495-9 · Cited by 400
Reitsma, J. B., Rutjes, A. W. S., Khan, K. S., Coomarasamy, A., & Bossuyt, P. M. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2009.02.005 · Cited by 405
Self-citationDiagnostic Accuracy of Five Serologic Tests for Strongyloides stercoralis Infection
Bisoffi, Z., Buonfrate, D., Sequi, M., Mejía, R., Cimino, R. O., Krolewiecki, A. J., Albonico, M., Gobbo, M., Bonafini, S., Angheben, A., et al. (2014) · PLoS neglected tropical diseases · doi:10.1371/journal.pntd.0002640 · Cited by 324
Boulware, D. R., Rolfes, M. A. R., Rajasingham, R., von Hohenberg, M., Qin, Z., Taseera, K., Schutz, C., Kwizera, R., Butler, E. K., Meintjes, G., et al. (2013) · Emerging infectious diseases · doi:10.3201/eid2001.130906 · Cited by 321
on behalf of the OPTIMA study group, van Randen, A., Laméris, W., van Es, H. W., van Heesewijk, H. P. M., van Ramshorst, B., ten Hove, W., Bouma, W. H., van Leeuwen, M. S., van Keulen, E. M., et al. (2011) · European Radiology · doi:10.1007/s00330-011-2087-5 · Cited by 297
Self-citationTaylor, S. A., Mallett, S., Bhatnagar, G., Baldwin‐Cleland, R., Bloom, S. L., Gupta, A., Hamlin, P. J., Hart, A. L., Higginson, A., Jacobs, I., et al. (2018) · The Lancet. Gastroenterology & hepatology · doi:10.1016/s2468-1253(18)30161-4 · Cited by 270
A Novel Host-Proteome Signature for Distinguishing between Acute Bacterial and Viral Infections
Oved, K., Cohen, A., Boico, O., Navon, R., Friedman, T., Etshtein, L., Kriger, O., Bamberger, E., Fonar, Y., Yacobov, R., et al. (2015) · PLoS ONE · doi:10.1371/journal.pone.0120012 · Cited by 239
Crellin, D., Harrison, D. M., Santamaria, N., & Babl, F. E. (2015) · Pain · doi:10.1097/j.pain.0000000000000305 · Cited by 252
Leeflang, M., Ang, C. W., Berkhout, J., Bijlmer, H. A., Bortel, W. V., Brandenburg, A. H., van Burgel, N. D., van Dam, A. P., Dessau, R. B., Fingerle, V., et al. (2016) · BMC Infectious Diseases · doi:10.1186/s12879-016-1468-4 · Cited by 243
Validity of the Manchester Triage System in emergency care: A prospective observational study
Zachariasse, J. M., Seiger, N., Rood, P. P. M., Alves, C. F., Freitas, P., Smit, F. J., Roukema, G. R., & Moll, H. A. (2017) · PLoS ONE · doi:10.1371/journal.pone.0170811 · Cited by 229
Molecular Diagnosis of T Cell-Mediated Rejection in Human Kidney Transplant Biopsies
REEVE, J. P., Sellarés, J., Mengel, M., Sis, B., Skene, A., Hidalgo, L. G., de Freitas, D. G., Famulski, K. S., & Halloran, P. F. (2013) · American Journal of Transplantation · doi:10.1111/ajt.12079 · Cited by 228
Performance of an Automated Polysomnography Scoring System Versus Computer-Assisted Manual Scoring
Malhotra, A., Younes, M. K., Kuna, S. T., Benca, R. M., Kushida, C. A., Walsh, J. A., Hanlon, A. L., Staley, B. A., Pack, A. I., & Pien, G. W. (2013) · SLEEP · doi:10.5665/sleep.2548 · Cited by 224
Fanti, S., Minozzi, S., Castellucci, P., Balduzzi, S., Herrmann, K., Krause, B. J., Oyen, W. J. G., & Chiti, A. (2015) · European Journal of Nuclear Medicine and Molecular Imaging · doi:10.1007/s00259-015-3202-7 · Cited by 225
Umemneku-Chikere, C. M., Wilson, K. J., Graziadio, S., Vale, L. D., & Allen, A. J. (2019) · PLoS ONE · doi:10.1371/journal.pone.0223832 · Cited by 229
Latent Class Models in Diagnostic Studies When There is No Reference Standard--A Systematic Review
van Smeden, M., Naaktgeboren, C. A., Reitsma, J. B., Moons, K. G. M., & de Groot, J. A. H. (2013) · American Journal of Epidemiology · doi:10.1093/aje/kwt286 · Cited by 217
Self-citationAccuracy of endoscopic and videofluoroscopic evaluations of swallowing for oropharyngeal dysphagia
Giraldo‐Cadavid, L. F., Leaño, L. R. L., Basantes, G. L., Bastidas, A., García, R., Ovalle, S., & Abondano‐Garavito, J. E. (2016) · The Laryngoscope · doi:10.1002/lary.26419 · Cited by 213
Validation of the Rome III Criteria for the Diagnosis of Irritable Bowel Syndrome in Secondary Care
Ford, A. C., Berčík, P., Morgan, D. G., Bolino, C., Pinto-Sánchez, M. I., & Moayyedi, P. (2013) · Gastroenterology · doi:10.1053/j.gastro.2013.08.048 · Cited by 204
van Houten, C. B., de Groot, J. A. H., Klein, A., Srugo, I., Chistyakov, I., de Waal, W. J., Meijssen, C. B., Avis, W., Wolfs, T. F. W., Shachor‐Meyouhas, Y., et al. (2016) · The Lancet Infectious Diseases · doi:10.1016/s1473-3099(16)30519-9 · Cited by 201
Who Said What: Modeling Individual Labelers Improves Classification
Guan, M. Y., Gulshan, V., Dai, A. M., & Hinton, G. E. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11756 · Cited by 43
Foody, G. M. (2023) · PLoS ONE · doi:10.1371/journal.pone.0291908 · Cited by 248
Halloran, P. F., Pereira, A. B., Chang, J. J., Matas, A. J., Picton, M. L., de Freitas, D., Bromberg, J. S., Serón, D., Sellarés, J., Einecke, G., et al. (2013) · American Journal of Transplantation · doi:10.1111/ajt.12465 · Cited by 186
Molecular assessment of disease states in kidney transplant biopsy samples
Halloran, P. F., Famulski, K. S., & REEVE, J. P. (2016) · Nature Reviews Nephrology · doi:10.1038/nrneph.2016.85 · Cited by 185
Poynard, T., Munteanu, M., Luckina, E., Perazzo, H., Ngo, Y., Royer, L., Fedchuk, L., Sattonnet, F., Pais, R., Lebray, P., et al. (2013) · Journal of Hepatology · doi:10.1016/j.jhep.2012.12.021 · Cited by 178
Validating Retinal Fundus Image Analysis Algorithms: Issues and a Proposal
Trucco, E., Ruggeri, A., Karnowski, T. P., Giancardo, L., Chaum, E., Hubschman, J., Al-Diri, B., Cheung, C. Y.-L., Wong, D. W. K., Abràmoff, M. D., et al. (2013) · Investigative Ophthalmology & Visual Science · doi:10.1167/iovs.12-10347 · Cited by 163
Brunt, E. M., Clouston, A. D., Goodman, Z. D., Guy, C. D., Kleiner, D. E., Lackner, C., TINIAKOS, D. G., Wee, A., Yeh, M. M., Leow, W. Q., et al. (2022) · Journal of Hepatology · doi:10.1016/j.jhep.2022.01.011 · Cited by 164
Bertens, L. C. M., Broekhuizen, B. D., Naaktgeboren, C. A., Rutten, F. H., Hoes, A. W., van Mourik, Y., Moons, K. G. M., & Reitsma, J. B. (2013) · PLoS Medicine · doi:10.1371/journal.pmed.1001531 · Cited by 160
Self-citationCauses of Persistent Dizziness in Elderly Patients in Primary Care
Maarsingh, O. R., Dros, J., Schellevis, F., van Weert, H. C. P. M., van der Windt, D., ter Riet, G., & van der Horst, H. E. (2010) · The Annals of Family Medicine · doi:10.1370/afm.1116 · Cited by 154
Buonfrate, D., Requena‐Méndez, A., Angheben, A., Cinquini, M., Cruciani, M., Fittipaldo, V. A., Giorli, G., Gobbi, F. G., Piubelli, C., & Bisoffi, Z. (2018) · PLoS neglected tropical diseases · doi:10.1371/journal.pntd.0006229 · Cited by 158
Serum ferritin thresholds for the diagnosis of iron deficiency in pregnancy: A systematic review
Daru, J., Allotey, J., Peña‐Rosas, J. P., & Khan, K. S. (2017) · Transfusion Medicine · doi:10.1111/tme.12408 · Cited by 138
Self-citationAccuracy of Five Serologic Tests for the Follow up of Strongyloides stercoralis Infection
Buonfrate, D., Sequi, M., Mejía, R., Cimino, R. O., Krolewiecki, A. J., Albonico, M., Degani, M., Tais, S., Angheben, A., Requena‐Méndez, A., et al. (2015) · PLoS neglected tropical diseases · doi:10.1371/journal.pntd.0003491 · Cited by 142
Performance of the PRO-C3 collagen neo-epitope biomarker in non-alcoholic fatty liver disease
Boyle, M., Tiniakos, D., Schattenberg, J. M., Ratziu, V., Bugianessi, E., Petta, S., Oliveira, C. P., Govaere, O., Younes, R., McPherson, S., et al. (2019) · JHEP Reports · doi:10.1016/j.jhepr.2019.06.004 · Cited by 140
Critical appraisal of artificial intelligence-based prediction models for cardiovascular disease
van Smeden, M., Heinze, G., CALSTER, B. V., Asselbergs, F. W., Vardas, P. E., Bruining, N., de Jaegere, P. P. T., Moore, J. H., Denaxas, S., Boulesteix, A., et al. (2022) · European Heart Journal · doi:10.1093/eurheartj/ehac238 · Cited by 155
Kühnisch, J., Söchtig, F., Pitchika, V., Laubender, R. P., Neuhaus, K. W., Lussi, A., & Hickel, R. (2015) · Clinical Oral Investigations · doi:10.1007/s00784-015-1559-4 · Cited by 138
Exhaled breath metabolomics as a noninvasive diagnostic tool for acute respiratory distress syndrome
Bos, L. D. J., Weda, H., Wang, Y., Knobel, H., Nijsen, T. M. E., Vink, T. J., Zwinderman, A. H., Sterk, P. J., & Schultz, M. J. (2014) · European Respiratory Journal · doi:10.1183/09031936.00005614 · Cited by 138
Taseera, K., Siedner, M. J., Klausner, J. D., Muzoora, C. K., & Boulware, D. R. (2013) · Clinical Infectious Diseases · doi:10.1093/cid/cit641 · Cited by 137
Blackledge, M., Collins, D. J., Tunariu, N., Orton, M. R., Padhani, A. R., Leach, M. O., & Koh, D. (2014) · PLoS ONE · doi:10.1371/journal.pone.0091779 · Cited by 131
Liu, F., Goh, G. B., TINIAKOS, D. G., Wee, A., Leow, W. Q., Zhao, J., Rao, H., Wang, X., Wang, Q., Wan, W., et al. (2019) · Hepatology · doi:10.1002/hep.30986 · Cited by 130
van Houten, C. B., Cohen, A., Engelhard, D., Hays, J. P., Karlsson, R., Moore, E. R. B., Fernández, D., Kreisberg, R., Collins, L. V., de Waal, W. J., et al. (2019) · European Journal of Clinical Microbiology & Infectious Diseases · doi:10.1007/s10096-018-03454-2 · Cited by 127
Quantifying the Added Value of a Diagnostic Test or Marker
Moons, K. G. M., de Groot, J. A. H., Linnet, K., Reitsma, J. B., & Bossuyt, P. M. M. (2012) · Clinical Chemistry · doi:10.1373/clinchem.2012.182550 · Cited by 152
Self-citationValidation of a Novel Assay to Distinguish Bacterial and Viral Infections
Srugo, I., Klein, A., Stein, M., Golan-Shany, O., Kerem, N. C., Chistyakov, I., Genizi, J., Glazer, O., Yaniv, L., German, A., et al. (2017) · PEDIATRICS · doi:10.1542/peds.2016-3453 · Cited by 122
Dark, P. M., Blackwood, B., Gates, S., McAuley, D. F., Perkins, G. D., McMullan, R., Wilson, C., Graham, D., Timms, K., & Warhurst, G. (2014) · Intensive Care Medicine · doi:10.1007/s00134-014-3553-8 · Cited by 120
Neuroimaging-Based Biomarkers in Psychiatry: Clinical Opportunities of a Paradigm Shift
Fu, C. H. Y., & Costafreda, S. G. (2013) · The Canadian Journal of Psychiatry · doi:10.1177/070674371305800904 · Cited by 122
Proposals for a Phased Evaluation of Medical Tests
Lijmer, J. G., Leeflang, M., & Bossuyt, P. M. (2009) · Medical Decision Making · doi:10.1177/0272989x09336144 · Cited by 118
Self-citationMeasurement error is often neglected in medical literature: a systematic review
Brakenhoff, T. B., Mitroiu, M., Keogh, R. H., Moons, K. G. M., Groenwold, R. H. H., & van Smeden, M. (2018) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2018.02.023 · Cited by 123
Estimation of diagnostic test accuracy without full verification: a review of latent class methods
Collins, J. P., & Huynh, M. (2014) · Statistics in Medicine · doi:10.1002/sim.6218 · Cited by 120
The Utility of Capsule Endoscopy in Patients With Suspected Crohn's Disease
Tukey, M. H., Pleskow, D. K., Legnani, P. E., Cheifetz, A. S., & Moss, A. C. (2009) · The American Journal of Gastroenterology · doi:10.1038/ajg.2009.404 · Cited by 113
Identification of Important Nodes in Directed Biological Networks: A Network Motif Approach
Wang, P., Lü, J., & Yu, X. (2014) · PLoS ONE · doi:10.1371/journal.pone.0106132 · Cited by 115
Incorporation bias in studies of diagnostic tests: how to avoid being biased about bias
Worster, A., & Carpenter, C. R. (2008) · Canadian Journal of Emergency Medicine · doi:10.1017/s1481803500009891 · Cited by 108
Discussion paper: what happened to the ‘bio’ in the bio-psycho-social model of low back pain?
Hancock, M. J., Maher, C. G., Laslett, M., Hay, E. M., & Koes, B. W. (2011) · European Spine Journal · doi:10.1007/s00586-011-1886-3 · Cited by 111
Value of composite reference standards in diagnostic research
Naaktgeboren, C. A., Bertens, L. C. M., Smeden, M. V., Groot, J. A. H. D., Moons, K. G. M., & Reitsma, J. B. (2013) · BMJ · doi:10.1136/bmj.f5605 · Cited by 105
Self-citationVerification problems in diagnostic accuracy studies: consequences and solutions
de Groot, J. A. H., Bossuyt, P. M., Reitsma, J. B., Rutjes, A. W., Dendukuri, N., Janssen, K. J. M., & Moons, K. G. M. (2011) · BMJ · doi:10.1136/bmj.d4770 · Cited by 103
Self-citationAssessment of the Predictive Validity of Etiologic Stroke Classification
Arsava, E. M., Helenius, J., Avery, R., Sorgun, M. H., Kim, G., Pontes‐Neto, O. M., Park, K., Rosand, J. M., Vangel, M. G., & Ay, H. (2017) · JAMA Neurology · doi:10.1001/jamaneurol.2016.5815 · Cited by 105
Freeman, K., Mistry, H., Tsertsvadze, A., Royle, P., McCarthy, N. D., Taylor‐Phillips, S., Manuel, R. J., & Mason, J. M. (2017) · Health Technology Assessment · doi:10.3310/hta21230 · Cited by 102
Small dense low-density lipoprotein: Analytical review
Kanonidou, C. (2021) · Clinica Chimica Acta · doi:10.1016/j.cca.2021.06.012 · Cited by 106
Price, S. J., Stapley, S., Shephard, E. A., Barraclough, K., & Hamilton, W. T. (2016) · BMJ Open · doi:10.1136/bmjopen-2016-011664 · Cited by 100
When Should a New Test Become the Current Reference Standard?
Glasziou, P., Irwig, L., & Deeks, J. J. (2008) · Annals of Internal Medicine · doi:10.7326/0003-4819-149-11-200812020-00009 · Cited by 97
Eustachian tube dysfunction: A diagnostic accuracy study and proposed diagnostic pathway
Smith, M. E., Takwoingi, Y., Deeks, J. J., Alper, C. M., Bance, M., Bhutta, M. F., Donnelly, N. P., Poe, D. S., & Tysome, J. R. (2018) · PLoS ONE · doi:10.1371/journal.pone.0206946 · Cited by 97
Ashkenazi‐Hoffnung, L., Oved, K., Navon, R., Friedman, T., Boico, O., Paz, M., Kronenfeld, G., Etshtein, L., Cohen, A., Gottlieb, T. M., et al. (2018) · European Journal of Clinical Microbiology & Infectious Diseases · doi:10.1007/s10096-018-3261-3 · Cited by 94
Bovine Respiratory Disease Diagnosis
Buczinski, S., & Pardon, B. (2020) · Veterinary Clinics of North America: Food Animal Practice · doi:10.1016/j.cvfa.2020.03.004 · Cited by 89
Poynard, T., de Lédinghen, V., Zarski, J., Stanciu, C., Munteanu, M., Vergniol, J., France, J., Trifan, A. V., Naour, G. L., Vaillant, J. C., et al. (2011) · Journal of Hepatology · doi:10.1016/j.jhep.2011.08.007 · Cited by 91
Ferraioli, G., Calcaterra, V., Lissandrin, R., Guazzotti, M., Maiocchi, L., Tinelli, C., Silvestri, A. D., Regalbuto, C., Pelizzo, G., Larizza, D., et al. (2017) · BMC Gastroenterology · doi:10.1186/s12876-017-0617-6 · Cited by 91
Abdeldaim, G. M. K., Strålin, K., Korsgaard, J., Blomberg, J., Welinder‐Olsson, C., & HERRMANN, B. F. (2010) · BMC Microbiology · doi:10.1186/1471-2180-10-310 · Cited by 89
Beltrame, A., Guerriero, M., Angheben, A., Gobbi, F. G., Requena‐Méndez, A., Zammarchi, L., Formenti, F., Perandin, F., Buonfrate, D., & Bisoffi, Z. (2017) · PLoS neglected tropical diseases · doi:10.1371/journal.pntd.0005593 · Cited by 89
Accuracy of High-Speed Video Analysis to Diagnose Primary Ciliary Dyskinesia
Rubbo, B., Shoemark, A., Jackson, C. L., Hirst, R. A., Thompson, J., Hayes, J., Frost, E., Copeland, F., Hogg, C. L., O’Callaghan, C., et al. (2019) · CHEST Journal · doi:10.1016/j.chest.2019.01.036 · Cited by 88
The estimation of diagnostic accuracy of tests for COVID-19: A scoping review
Axell‐House, D. B., Lavingia, R., Rafferty, M., Clark, E. H., Amirian, E. S., & Chiao, E. Y. (2020) · Journal of Infection · doi:10.1016/j.jinf.2020.08.043 · Cited by 83
Stengel, D., Ottersbach, C., Matthes, G., Weigeldt, M., Grundei, S., Rademacher, G., Tittel, A., Mutze, S., Ekkernkamp, A., Frank, M., et al. (2012) · Canadian Medical Association Journal · doi:10.1503/cmaj.111420 · Cited by 83
Kitsios, G. D., Fitch, A. C., Manatakis, D. V., Rapport, S. F., Li, K., Qin, S., Huwe, J., Zhang, Y., Doi, Y., Evankovich, J. W., et al. (2018) · Frontiers in Microbiology · doi:10.3389/fmicb.2018.01413 · Cited by 80
Clinical management of chronic mesenteric ischemia
van Dijk, L. J. D., van Noord, D., de Vries, A. C., Kolkman, J. J., Geelkerken, R. H., Verhagen, H. J. M., Moelker, A. D., & Bruno, M. J. (2018) · United European Gastroenterology Journal · doi:10.1177/2050640618817698 · Cited by 82
Chagas Disease Serological Test Performance in U.S. Blood Donor Specimens
Whitman, J. D., Bulman, C. A., Gunderson, E. L., Irish, A. M., Townsend, R. L., Stramer, S. L., Sakanari, J. A., & Bern, C. (2019) · Journal of Clinical Microbiology · doi:10.1128/jcm.01217-19 · Cited by 78
Jaworska, J. S. (2010) · ALTEX · doi:10.14573/altex.2010.4.231 · Cited by 78
Cota, G., de Sousa, M. R., Nogueira, B. M. D. F., Gomes, L. I., Oliveira, E., de Assis, T. S. M., de Mendonça, A. L. P., Pinto, B. F., Saliba, J. W., & Rabello, A. (2013) · American Journal of Tropical Medicine and Hygiene · doi:10.4269/ajtmh.13-0239 · Cited by 82
Dahlin, L. B., Zimmerman, M., Calcagni, M., Hundepool, C. A., van Alfen, N., & Chung, K. C. (2024) · Nature Reviews Disease Primers · doi:10.1038/s41572-024-00521-1 · Cited by 76
Zirkzee, E. J. M., Steup‐Beekman, G. M., van der Mast, R. C., Bollen, E. L. E. M., van der Wee, N. J. A., Baptist, E., Slee, T. M., Huisman, M. V., Middelkoop, H. A. M., Luyendijk, J., et al. (2012) · The Journal of Rheumatology · doi:10.3899/jrheum.120545 · Cited by 75
Taylor, S. A., Mallett, S., Beare, S., Bhatnagar, G., Blunt, D. M., Boavida, P., Bridgewater, J. A., Clarke, C. S., Duggan, M., Ellis, S., et al. (2019) · The Lancet. Gastroenterology & hepatology · doi:10.1016/s2468-1253(19)30056-1 · Cited by 75
Bosman, L. P., Cadrin‐Tourigny, J., Bourfiss, M., Ghasabeh, M. A., Sharma, A., Tichnell, C., Roudijk, R. W., Murray, B. A., Tandri, H. S., Khairy, P., et al. (2020) · EP Europace · doi:10.1093/europace/euaa039 · Cited by 72
Self-citationDiagnostic Test Accuracy in Childhood Pulmonary Tuberculosis: A Bayesian Latent Class Analysis
Schumacher, S. G., van Smeden, M., Dendukuri, N., Joseph, L., Nicol, M. P., Pai, M., & Zar, H. J. (2016) · American Journal of Epidemiology · doi:10.1093/aje/kww094 · Cited by 71
Questioning the “SPIN and SNOUT” rule in clinical testing
Baeyens, J., Serrien, B., Goossens, M., & Clijsen, R. (2019) · Archives of Physiotherapy · doi:10.1186/s40945-019-0056-5 · Cited by 71
Taylor, S. A., Mallett, S., Miles, A., Stephen Morris, A., Quinn, L., Clarke, C. S., Beare, S., Bridgewater, J. A., Goh, V., Janes, S. M., et al. (2019) · Health Technology Assessment · doi:10.3310/hta23660 · Cited by 70
Guidelines for Reporting Reliability and Agreement Studies (GRRAS)
Kottner, J., Gajewski, B. J., & Streiner, D. L. (2011) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2011.01.017 · Cited by 69
Review on the validity of self-report to assess work-related diseases
Lenderink, A. F., Zoer, I., van der Molen, H. F., Spreeuwers, D., Frings‐Dresen, M. H. W., & van Dijk, F. J. H. (2011) · International Archives of Occupational and Environmental Health · doi:10.1007/s00420-011-0662-3 · Cited by 68
Taylor, S. A., Mallett, S., Ball, S. T., Beare, S., Bhatnagar, G., Bhowmik, A., Boavida, P., Bridgewater, J. A., Clarke, C. S., Duggan, M., et al. (2019) · The Lancet Respiratory Medicine · doi:10.1016/s2213-2600(19)30090-6 · Cited by 68
van Oort, P. M. P., de Bruin, S., Weda, H., Knobel, H., Schultz, M. J., & Bos, L. D. J. (2017) · International Journal of Molecular Sciences · doi:10.3390/ijms18020449 · Cited by 67
Diagnostic accuracy of imaging modalities for suspected scaphoid fractures
Yin, Z.-G., Zhang, J.-B., Kan, S.-L., & Wang, X.-G. (2012) · Journal of Bone and Joint Surgery - British Volume · doi:10.1302/0301-620x.94b8.28998 · Cited by 66
Min, L. C., Tinetti, M. E., Langa, K. M., Ha, J., Alexander, N. B., & Hoffman, G. J. (2019) · JAMA Network Open · doi:10.1001/jamanetworkopen.2019.9679 · Cited by 66
Do pressure ulcer risk assessment scales improve clinical practice?
Kottner, J. (2010) · Journal of Multidisciplinary Healthcare · doi:10.2147/jmdh.s9286 · Cited by 64
Predicting Risk of Serious Bacterial Infections in Febrile Children in the Emergency Department
Irwin, A., Grant, A., Williams, R., Kolamunnage‐Dona, R., Drew, R. J., Paulus, S. C., Jeffers, G., Williams, K., Breen, R., Preston, J., et al. (2017) · PEDIATRICS · doi:10.1542/peds.2016-2853 · Cited by 65
Hooper, L., Ashton, K. E., Harvey, L. J., Décsi, T., & Fairweather‐Tait, S. J. (2009) · American Journal of Clinical Nutrition · doi:10.3945/ajcn.2009.27230a · Cited by 63
Course and prognosis of myasthenia gravis: a systematic review
Mao, Z. ‐., Mo, X. ‐., Qin, C., Lai, Y., & Hartman, T. O. (2010) · European Journal of Neurology · doi:10.1111/j.1468-1331.2010.03017.x · Cited by 63
Okonkwo, H., Bryant, R. A., Milne, J., Molyneaux, D. M., Sanders, J., Cunningham, G., Brangman, S. A., Eardley, W., Chan, G. K., Mayer, B., et al. (2020) · Wound Repair and Regeneration · doi:10.1111/wrr.12790 · Cited by 62
Pamfil, C. A. M., Fanouriakis, A., Damian, L. O., Rinzis, M., Sidiropoulos, P. I., Tsivgoulis, G. K., Rednic, S., Bertsias, G. K., & Boumpas, D. T. (2015) · Lara D. Veeken · doi:10.1093/rheumatology/keu482 · Cited by 61
Tidy, J. A., Brown, B., Healey, T. J., Daayana, S., Martin, M., Prendiville, W. J., & Kitchener, H. C. (2013) · BJOG An International Journal of Obstetrics & Gynaecology · doi:10.1111/1471-0528.12096 · Cited by 60
Meads, C., Auguste, P., Davenport, C. F., Małysiak, S., Sundar, S. S., Kowalska, M., Zapalska, A., GUEST, P. J., Thangaratinam, S., Martin‐Hirsch, P. L., et al. (2013) · Health Technology Assessment · doi:10.3310/hta17120 · Cited by 60
Self-citation
Metadata
- Authors
- Rutjes, A., Reitsma, J., Coomarasamy, A., Khan, K., & Bossuyt, P.
- Affiliations
- Amsterdam UMC Location University of Amsterdam (NL)
- University of Birmingham (GB)
- Venue
- Health Technology Assessment
- Published
- 2007-12
- Volume
- 11
- Issue
- 50
- Pages
- iii, ix-51
- ISSN
- 1366-5278 (ISSN-L)
- 2046-4924
- OpenAlex
- W2143712591
- Cited by
- 524
- References
- 175
- Citations by year
- 2026: 21 · 2025: 21 · 2024: 20 · 2023: 23 · 2022: 36 · 2021: 34 · 2020: 39 · 2019: 38 · 2018: 57 · 2017: 31 · 2016: 33 · 2015: 27 · 2014: 29 · 2013: 27 · 2012: 30
- FWCI
- 2.43
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Locations
- Health Technology Assessment (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Pure Amsterdam UMC (publishedVersion)
- IRIS UNIMORE (University of Modena and Reggio Emilia) (submittedVersion, cc-by)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- IRIS UNIMORE (University of Modena and Reggio Emilia) (submittedVersion)
- University of Birmingham Research Portal (University of Birmingham) (submittedVersion)
- Funders
- Health Technology Assessment programme (06/90/23)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
- Keywords
- diagnostic accuracy, absence of gold standard, reference materials, imperfect reference standard, latent class analysis, sensitivity and specificity, methodological review
- MeSH terms
- Humans, Reference Standards, Sensitivity and Specificity, Process Assessment, Health Care, Diagnostic Techniques and Procedures
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
