The DOI matches its Crossref record; 29 of the 31 references it cites were found in a registry or an index.
COSMIN Risk of Bias tool to assess the quality of studies on reliability or measurement error of outcome measurement instruments: a Delphi study
Open accessPublished version of a preprint
Abstract
Background: Scores on an outcome measurement instrument depend on the type and settings of the instrument used, how instructions are given to patients, how professionals administer and score the instrument, etc. The impact of all these sources of variation on scores can be assessed in studies on reliability and measurement error, if properly designed and analyzed. The aim of this study was to develop standards to assess the quality of studies on reliability and measurement error of clinician-reported outcome measurement instruments, performance-based outcome measurement instrument, and laboratory values. Methods: We conducted a 3-round Delphi study involving 52 panelists. Results: Consensus was reached on how a comprehensive research question can be deduced from the design of a reliability study to determine how the results of a study inform us about the quality of the outcome measurement instrument at issue. Consensus was reached on components of outcome measurement instruments, i.e. the potential sources of variation. Next, we reached consensus on standards on design requirements ( n = 5), standards on preferred statistical methods for reliability ( n = 3) and measurement error ( n = 2), and their ratings on a four-point scale. There was one term for a component and one rating of one standard on which no consensus was reached, and therefore required a decision by the steering committee. Conclusion: We developed a tool that enables researchers with and without thorough knowledge on measurement properties to assess the quality of a study on reliability and measurement error of outcome measurement instruments.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Intraclass correlations: Uses in assessing rater reliability.
Shrout, P. E., & Fleiss, J. L. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.2.420 · Cited by 24,532
Convergent and discriminant validation by the multitrait-multimethod matrix.
Campbell, D. T., & Fiske, D. W. (1959) · Psychological Bulletin · doi:10.1037/h0046016 · Cited by 17,116
Confirmatory factor analysis for applied research
Brown, T. A. (2007) · Choice Reviews Online · doi:10.5860/choice.44-2769 · Cited by 15,621
QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies
Whiting, P. F., Rutjes, A. W. S., Westwood, M. E., Mallett, S., Deeks, J. J., Reitsma, J. B., Leeflang, M. M. G., Sterne, J. A. C., Bossuyt, P. M. M., & the QUADAS-2 Group* (2011) · Annals of Internal Medicine · doi:10.7326/0003-4819-155-8-201110180-00009 · Cited by 14,199
Health Measurement Scales: A Practical Guide to Their Development and Use
Streiner, D. L., Norman, G. R., & Cairney, J. (1989) · Cited by 9,595
Forming inferences about some intraclass correlation coefficients.
McGraw, K. O., & Wong, S. P. (1996) · Psychological Methods · doi:10.1037/1082-989x.1.1.30 · Cited by 6,903
Mokkink, L. B., Terwee, C. B., Patrick, D. L., Alonso, J., Stratford, P. W., Knol, D. L., Bouter, L. M., & de Vet, H. C. W. (2010) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.02.006 · Cited by 4,465
Self-citationMokkink, L. B., Terwee, C. B., Patrick, D. L., Alonso, J., Stratford, P. W., Knol, D. L., Bouter, L. M., & de Vet, H. C. W. (2010) · Quality of Life Research · doi:10.1007/s11136-010-9606-8 · Cited by 4,260
Self-citationCOSMIN guideline for systematic reviews of patient-reported outcome measures
Prinsen, C. A. C., Mokkink, L. B., Bouter, L. M., Alonso, J., Patrick, D. L., de Vet, H. C. W., & Terwee, C. B. (2018) · Quality of Life Research · doi:10.1007/s11136-018-1798-3 · Cited by 3,345
Self-citationThe Delphi technique: myths and realities
Powell, C. (2003) · Journal of Advanced Nursing · doi:10.1046/j.1365-2648.2003.02537.x · Cited by 2,611
Research works printed but not linked: 2
- Shavelson RJ, Webb NM. Generalizability Theory. A primer. Measurement Methods for the Social Science, vol. 1. Newbury Park: Sage Publishing; 1991.
- CANCHILD Centre for disability research, I.o.a.h.s., McMaster University. Outcome measures rating form guideline. 2004[31/8/2017]; Available from: https://www.canchild.ca/system/tenon/assets/attachments/000/000/372/original/measrate.pdf?license=yes.
Cited by
Sorted by citations
Items 101–200 of 587
Katakis, P., Estrin, G. L., Wolstencroft, J., Sayani, S., Buckley, E., Mirzaei, V., Heys, M., & Skuse, D. (2023) · Review Journal of Autism and Developmental Disorders · doi:10.1007/s40489-023-00408-z · Cited by 14
“The quality of the included studies was assessed using the COnsensus-based Standards for the selection of health Measurement Instruments (COSMIN) Risk of Bias checklist, with adaptations for clinician-reported outcomes (Mokkink et al., 2018, 2020), where appropriate.”
Measuring Signs of Atopic Dermatitis in Clinical Practice
Jacobson, M. E., Leshem, Y. A., Apfelbacher, C., Spuls, P. I., Gerbens, L. A. A., Thomas, K. S., Williams, H. C., Katoh, N., Howells, L., Schmitt, J. M., et al. (2024) · JAMA Dermatology · doi:10.1001/jamadermatol.2024.1162 · Cited by 16
Willems, A., Houkes, C. M., Bierma‐Zeinstra, S., & Meuffels, D. E. (2022) · European Journal of Radiology · doi:10.1016/j.ejrad.2022.110511 · Cited by 13
Spasticity evaluation with the Amadeo Tyromotion device in patients with hemispheric stroke
Urrutia, R., Gutiérrez-Muto, A. M., Sanz-Morère, C. B., Gómez, A., Politi, A. M., Lunardini, F., Baccini, M., Cecchi, F., León, N., Oliviero, A., et al. (2023) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2023.1172770 · Cited by 13
“Other authors leave some time of rest between evaluations, to avoid this interference (Mokkink et al., 2020).”
BackgroundResultComparison between compressed sensing and segmented cine cardiac magnetic resonance: a meta-analysis
Craft, J., Li, Y., Nashta, N. F., & Weber, J. (2023) · BMC Cardiovascular Disorders · doi:10.1186/s12872-023-03426-1 · Cited by 13
“Study quality was independently evaluated by two authors (JC and JW) using the COSMIN Risk of Bias tool to assess the quality of studies on reliability and measurement error of outcome measurement instrument scale [17].”
MethodJung, A., Braun, T., Armijo-Olivo, S. L., Challoumas, D., & Luedtke, K. (2023) · Research Synthesis Methods · doi:10.1002/jrsm.1688 · Cited by 12
“30,33,34 In addition, panel-ists had the option to add open-ended comments to each proposal or question.”
BackgroundReliability, validity, and responsiveness of pinch strength assessment: a systematic review
Szekeres, M., Aspinall, D., Kulick, J., Sajid, A., Dabbagh, A., & MacDermid, J. C. (2024) · Disability and Rehabilitation · doi:10.1080/09638288.2024.2382907 · Cited by 14
“The results of each study were then rated by applying the updated criteria for measurement properties recommended in the COSMIN Guideline for Systematic Reviews of Patient-Reported Outcome Measures [13,14].”
Carrière, M. E., Mokkink, L. B., Pleat, J., Tyack, Z. F., Pijpe, A., de Vet, H. C. W., van Zuijlen, P. P. M., & The POSAS Group (2025) · Plastic & Reconstructive Surgery Global Open · doi:10.1097/gox.0000000000006416 · Cited by 13
Self-citationBerardi, A., Graziosi, G., Ferrazzano, G., Conti, L. C., Grasso, M. G., Tramontano, M., Conte, A., & Galeoto, G. (2022) · Healthcare · doi:10.3390/healthcare10060985 · Cited by 14
“Statistical Analysis Following the Consensus-Based Standards for the Selection of Health Status Measurement Instruments (COSMIN) checklist, internal consistency, construct validity, and cross-cultural validity of the scale were assessed [39].”
MethodInfluentialHansen, H., Beyer, N., Frølich, A., Godtfredsen, N. S., & Bieler, T. (2021) · Patient Related Outcome Measures · doi:10.2147/prom.s306352 · Cited by 14
Jacobson, M. E., Morimoto, R. Y., Leshem, Y. A., Howells, L., Williams, H. C., Grinich, E. E., Gerbens, L. A. A., Spuls, P. I., Schmitt, J., Staley, B., et al. (2024) · Journal of the European Academy of Dermatology and Venereology · doi:10.1111/jdv.20248 · Cited by 12
“14 For a ClinROM based on a formative model such as the EASI, whereby the components form the construct, there are six recommended measurement properties: content validity, construct validity, responsiveness, criterion validity, reliability and measurement error (Table 1).”
InfluentialKowalski, K. L., Lukacs, M. J., Mistry, J., Goodman, M., & Rushton, A. (2022) · BMJ Open · doi:10.1136/bmjopen-2022-060950 · Cited by 13
A meta-analysis of the criterion-related validity of Session-RPE scales in adolescent athletes
Liu, H., Yang, W., Liu, H., Bao, D., Cui, Y., Ho, I. M. K., & Li, Q. (2023) · BMC Sports Science Medicine and Rehabilitation · doi:10.1186/s13102-023-00712-5 · Cited by 12
“For analyses where I 2 was below 50%, a fixed effects model was applied and, if I 2 was above 50%, a random effects model was employed [34].”
MethodHidalgo‐Robles, Á., Paleg, G., & Livingstone, R. (2024) · Healthcare · doi:10.3390/healthcare12040493 · Cited by 15
Meikle, B., Simons, M., Mahoney, T., Reddan, T., Dai, B., Kimble, R. M., & Tyack, Z. F. (2024) · BMJ Open · doi:10.1136/bmjopen-2023-078361 · Cited by 12
Husain, S., Ayoub, N. P., & Hassmann, M. (2024) · Syariat · doi:10.35335/cfb3wk76 · Cited by 12
“Assessing quality and dependability also involves evaluating the pertinence and importance of the findings in relation to the study topic and aims(Mokkink et al., 2020; Sun et al., 2019; Terwee et al., 2018).”
Backgroundda Silva, R. S., da Silva, S. T., Cardoso, D. C. R., Quirino, M. A. F., Silva, M. H. A., Gomes, L. A., Fernandes, J. D., Oliveira, R., Fernandes, A. B. G. S., & Ribeiro, T. S. (2024) · Gait & Posture · doi:10.1016/j.gaitpost.2024.08.004 · Cited by 15
Liu, Y., Cui, N., Zhang, Y., Wang, X., Zhang, H., Chen, D., Sun, S., & Jin, J. (2021) · BMC Nursing · doi:10.1186/s12912-021-00651-x · Cited by 13
Reick, S., Burckhardt, M., Palm, R., & Kottner, J. (2023) · Nursing Open · doi:10.1002/nop2.1863 · Cited by 12
Lee, U., Lee, S., Kim, S.-A., Kim, Y., & Lee, S. (2025) · Frontiers in Bioengineering and Biotechnology · doi:10.3389/fbioe.2025.1570637 · Cited by 11
Lazrek, O., Karam, K. M., Bouché, P., Billaud, A., Pourchot, A., Godenèche, A., Freaud, O., Kany, J., Métais, P., Werthel, J.-D., et al. (2022) · Knee Surgery Sports Traumatology Arthroscopy · doi:10.1007/s00167-022-07290-y · Cited by 11
“This is a slightly shorter time than some studies in the literature [18]; however, this depends on the tool being assessed and validated [25].”
ResultShepherd, J., Hansjee, S., Divall, P., Raval, P., & Singh, H. (2023) · Shoulder & Elbow · doi:10.1177/17585732231195554 · Cited by 11
Carvalho, D. E. D., Mackey, S., To, D., Summers, A., Frey, M., Romme, K., Hogg‐Johnson, S., & Howarth, S. J. (2024) · Scientific Reports · doi:10.1038/s41598-024-52900-z · Cited by 12
“9 .”
BackgroundMethodResultInfluentialLoosli, S. V., Neumann, L. C., Wlasich, E., Prix, C., Koll, L., Weidinger, E., Vöglein, J., Wagemann, O., Danek, A., Nübling, G., et al. (2024) · Journal of Intellectual & Developmental Disability · doi:10.3109/13668250.2024.2317794 · Cited by 11
“…and reporting are based on the “ Consen-sus-based Standards for the Selection of Health Measurement Instruments (COSMIN) ” (Gagnier et al., 2021; Mokkink et al., 2020), the “ Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement ” (von Elm et al., 2008) and…”
Kowalski, K. L., Mistry, J., Beilin, A., Goodman, M., Lukacs, M. J., & Rushton, A. (2024) · PLoS ONE · doi:10.1371/journal.pone.0307004 · Cited by 12
A systematic review of post-stroke fatigue measurement scale based on COSMIN guidelines
Wu, L., & Jin, H. (2024) · Frontiers in Neurology · doi:10.3389/fneur.2024.1411472 · Cited by 11
Haug, T. R., Mišković, D., Ørntoft, M. W., Iversen, L. H., Johnsen, S. P., Valentin, J. B., Ruiz, M. G., Benz, S. R., Storli, K. E., Stearns, A. T., et al. (2022) · Colorectal Disease · doi:10.1111/codi.16317 · Cited by 11
“To evaluate the level of agreement between experts (inter- rater reliability) in each Delphi round, a mixed effect model was used to calculate the interclass correlation coefficient (ICC) in a fully crossed design [33].”
MethodSeizure severity assessment tools for adult epilepsy patients: A systematic review
Rusli, R. A., Makmor‐Bakry, M., Shah, N. M., & Jan, T. H. (2023) · Epilepsy & Behavior · doi:10.1016/j.yebeh.2023.109154 · Cited by 11
Lecuona, O. Z., Lin, C., Montag, C., Pontes, H. M., & Pakpour, A. H. (2024) · International Journal of Mental Health and Addiction · doi:10.1007/s11469-024-01320-1 · Cited by 11
“…of the IGDS9-SF has been synthesized by a systematic review (Poon et al., 2021) using the rigorous COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) method (Mokkink et al., 2020; Prinsen et al., 2018), the COSMIN method does not consider network analysis.”
Tomacheuski, R. M., Monteiro, B. P., Evangelista, M. C., Luna, S. P. L., & Steagall, P. V. M. (2021) · PLoS ONE · doi:10.1371/journal.pone.0251435 · Cited by 11
“The COSMIN methodology was specifically developed and validated for use in reviews of patient-reported outcome measures [41, 44, 45, 47].”
BackgroundMethodInfluentialOutcome reporting in therapeutic mammaplasty: a systematic review
Lee, A. E., Kwasnicki, R. M., Khan, H., Grant, Y., Chan, A., Fanshawe, A., & Leff, D. R. (2021) · BJS Open · doi:10.1093/bjsopen/zrab126 · Cited by 11
Grief Instruments in Children and Adolescents: A Systematic Review
Zhang, T., Krysinska, K. E., Alisic, E., & Andriessen, K. (2023) · OMEGA - Journal of Death and Dying · doi:10.1177/00302228231171188 · Cited by 10
“The COSMIN (COnsensus-based Standards for the selection of health Measurement In-struments) tool outlines the standards and psychometric variables used to assess the quality of studies and outcome measurement instruments (Mokkink et al., 2020).”
Dunne, L. A. M., Cole, M. H., Cormack, S. J., Howell, D. R., & Johnston, R. D. (2023) · Sports Medicine - Open · doi:10.1186/s40798-023-00625-0 · Cited by 10
Santos, O. P. D., Verloo, H., Joost, S., & Melly, P. (2023) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph20206963 · Cited by 11
“Finally, the COSMIN Risk of Bias tool will be used to assess the quality of studies’ reliability and measurement instruments’ measurement error of outcomes [40].”
BackgroundMethodClassification of short and long term mild traumatic brain injury using computerized eye tracking
Cade, A. E., & Turnbull, P. (2024) · Scientific Reports · doi:10.1038/s41598-024-63540-8 · Cited by 10
“51 Bland–Altman plots with mean differences and limits of agreement and two-way intraclass correlation coefficients (ICCs) for consistency were used to assess reliability of CEA tests 52 .”
MethodMeletani, S., Scataglini, S., Mandolini, M., Scalise, L., & Truijen, S. (2024) · Sensors · doi:10.3390/s24144669 · Cited by 10
“Computing the ICC [36] as a measurement for inter-trial and inter-subject reliability is recommended.”
MethodMargetts, J. L., Hazelton, M., Santangelo, P., Yorke, J., & Wilson, R. L. (2024) · International Journal of Mental Health Nursing · doi:10.1111/inm.13404 · Cited by 13
Dobie, K., Barr, C., George, S., Baker, N., Pankhurst, M., & van den Berg, M. E. L. (2025) · The Journal of Frailty & Aging · doi:10.1016/j.tjfa.2025.100033 · Cited by 10
Pometti, L. S., Piscitelli, D., Ugolini, A., Ferrarello, F., Notturni, F., Coppari, A., Caselli, S., Porta, F. L., Levin, M. F., & Pellicciari, L. (2025) · Neurorehabilitation and neural repair · doi:10.1177/15459683251327568 · Cited by 10
“20,24,26,27 The review included twenty-five studies involving 2009 participants in clinical and research settings, while the meta-analysis provided quantitative benchmarks (i.e., MDC and ES) for interpreting change scores.”
Mokkink, L. B., Herbelet, S., Terwee, C. B., & Boers, M. (2025) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2025.111836 · Cited by 10
Self-citationReliability of the Short Physical Performance Battery (SPPB): a systematic review with meta-analysis
Eusepi, D., Pellicciari, L., Ugolini, A., Graziani, L., Coppari, A., Carlizza, A., Caselli, S., Porta, F. L., Paci, M., Bari, M. D., et al. (2025) · European Geriatric Medicine · doi:10.1007/s41999-025-01277-x · Cited by 13
“The methodological quality of the included studies was evaluated using the COSMIN Risk of Bias checklist developed for studies on the reliability and/or measurement error of measurement instruments [19, 20].”
Made to measure—Selecting outcomes in aneurysmal subarachnoid hemorrhage research
Andersen, C., English, S. W., & Delaney, A. P. (2022) · Frontiers in Neurology · doi:10.3389/fneur.2022.1000454 · Cited by 10
Bomhof‐Roordink, H., Stiggelbout, A. M., Gärtner, F. R., Portielje, J. E. A., de Kroon, C. D., Peeters, K. C. M. J., Neelis, K. J., Dekker, J. W. T., van der Weijden, T., Pieterse, A. H., et al. (2021) · Patient Education and Counseling · doi:10.1016/j.pec.2021.08.034 · Cited by 9
Griefahn, A., Zalpour, C., & Luedtke, K. (2024) · Health and Technology · doi:10.1007/s12553-024-00827-w · Cited by 9
“The following assumptions were based on Delphi survey principles [27, 28]: An agreement rate of 67% indicated the presence of a contraindication, whereas an agreement rate of 75% for "no contraindication" indicated the absence of a contrain-dication.”
Aldhabi, R., Albadi, M., Kahraman, T., & Alsobhi, M. G. (2024) · Musculoskeletal Science and Practice · doi:10.1016/j.msksp.2024.103102 · Cited by 9
Garcia, P. A., Sampaio, R. X., de Moura, J. A., de Souza, P. F., da Costa, L. B., & Mendes, F. A. D. S. (2024) · Brazilian Journal of Physical Therapy · doi:10.1016/j.bjpt.2024.101104 · Cited by 9
van Silfhout, N. Y., van Muilekom, M. M., van Karnebeek, C., Haverman, L., & van Eeghen, A. M. (2024) · Orphanet Journal of Rare Diseases · doi:10.1186/s13023-024-03264-0 · Cited by 9
“In addition, the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) initiative usually use a consensus threshold of 67% in Delphi studies [51, 52].”
MethodMarichamy, M., Chockalingam, K., & Arunachalam, N. (2024) · Journal of Tribology · doi:10.1115/1.4066846 · Cited by 9
Wei, X., McKinlay, C. J. D., Harding, J. E., Wouldes, T. A., Rogers, J., Brown, G., & Franke, N. (2024) · Child Neuropsychology · doi:10.1080/09297049.2024.2447444 · Cited by 10
“The risk of bias for each study was assessed using the COSMIN-RoB tool (Supplementary Table S2) (Mokkink et al., 2020), which includes nine items rated on a four-point scale (very good, adequate, doubtful, inadequate).”
InfluentialUsing 3D and 4D digital human modeling in extended reality-based rehabilitation: a systematic review
Lu, M., Saeys, W., Maryam, M., Gjeleshi, I., Nazarahari, H., Truijen, S., & Scataglini, S. (2025) · Frontiers in Bioengineering and Biotechnology · doi:10.3389/fbioe.2025.1496168 · Cited by 11
Clinical rating scales for assessing pain in newborn infants
Färnqvist, K., Olsson, E., Garratt, A., Paraskevas, T., Soll, R. F., Bruschettini, M., & Persad, E. (2025) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000064.pub2 · Cited by 13
Wen, S., Cheng, D., Zhao, N., Chen, X., Lu, X., Li, Y., Liu, H., Gao, J., Hou, C., & Xu, R. (2025) · BMC Geriatrics · doi:10.1186/s12877-025-06030-4 · Cited by 10
“The 2021 update of the COS-MIN Risk of Bias Inventory expanded [23] its framework for assessing clinician-reported outcomes (ClinPOMs) and performance-based outcome measures (PerFOMs).”
Measurement of mentalizing: A systematic review and development of a construct validity framework
Hawksdale, H., Meng, B., Feigenbaum, J., Fonagy, P., & Nolte, T. (2025) · Clinical Psychology Review · doi:10.1016/j.cpr.2025.102657 · Cited by 11
Ultrasound measurements of pathological and physiological skin thickness: a scoping review protocol
Meikle, B., Kimble, R. M., & Tyack, Z. F. (2022) · BMJ Open · doi:10.1136/bmjopen-2021-056720 · Cited by 9
Fialho, H. R. F., Kamonseki, D. H., Jales, M. T. M., Gonçalves, G. V., Barbosa, G. M., & Calixtre, L. B. (2022) · Isokinetics and Exercise Science · doi:10.3233/ies-220018 · Cited by 9
“…to the declaration of Helsinki [31] for experiments involving humans and was designed and reported according to the COnsensus-based Standards for the selection of health status Measurement INstruments [32] and the Guidelines for Reporting Reliability and Agreement Studies [33], respectively.”
Smith, N. L., Smith, M. G., Gibson, N., Imms, C., Thornton, A., & Harvey, A. R. (2022) · Developmental Medicine & Child Neurology · doi:10.1111/dmcn.15410 · Cited by 9
“proxy measures).(13) The measurement properties of the identified tools were reported according to the criteria outlined in the COnsensus- based Standards for the selection of health Measurement INstruments (COSMIN) guidelines.”
MethodQuantifying Turning Tasks With Wearable Sensors: A Reliability Assessment
Weston, A. R., Antonellis, P., Fino, P. C., Hoppes, C. W., Lester, M., Weightman, M. M., Dibble, L. E., & King, L. A. (2023) · Physical Therapy · doi:10.1093/ptj/pzad134 · Cited by 9
Elsman, E. B. M., Mokkink, L. B., Abma, I. L., Aiyegbusi, O. L., Chiarotto, A., Haywood, K. L., Matvienko‐Sikar, K., Oosterveer, D. M., Pool, J. J. M., Swinkels-Meewisse, I., et al. (2024) · Quality of Life Research · doi:10.1007/s11136-024-03706-z · Cited by 11
“The COS-MIN risk of bias checklist [16] or tool [15] were specifically developed for this purpose and were used in 47 reviews.”
BackgroundMethodSelf-citationMadsø, K. G., Flo, E., Pachana, N. A., & Nordhus, I. H. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.742510 · Cited by 10
“Test–retest values may be used to calculate SDC for continuous scores (Prinsen et al., 2018; Mokkink et al., 2020).”
BackgroundMethodInfluentialde Brito, S. A. F., Scianni, A. A., Peniche, P. D. C., & Faria, C. D. C. D. M. (2022) · PLoS ONE · doi:10.1371/journal.pone.0265841 · Cited by 11
Aflalo, J., Quijoux, F., Truong, C., Bertin‐Hugault, F., & Ricard, D. (2022) · Journal of Personalized Medicine · doi:10.3390/jpm12081319 · Cited by 9
“The descriptions have been modified to be more specific to interventions using measurement instrument, following recommendations from the COSMIN risk of bias tool [32].”
MethodGraham, B., Smith, J. E., Nelmes, P., Squire, R., & Latour, J. M. (2023) · Healthcare · doi:10.3390/healthcare11050717 · Cited by 9
“From the outset, the development process for PREM-ED 65 is designed to ensure the final instrument meets the internationally accepted COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) guidelines [24].”
MethodGraham, B., Smith, J. E., Barham, F., & Latour, J. M. (2023) · Health Expectations · doi:10.1111/hex.13811 · Cited by 8
“These findings are reflected in a further systematic review of 88 PREMs which reported inconsistent adherence to established criteria for the selection of health instruments.(13,14) Recently, PREMs have been developed to capture older people's experience of hospital and community care, although no instrument specific…”
ResultWithin-individual variation of measured depression symptoms: A systematic review and meta-analysis
Gough, A., Marshall, T. P., Ferris, E., & Sitch, A. J. (2023) · Journal of Affective Disorders Reports · doi:10.1016/j.jadr.2023.100675 · Cited by 9
Li, Y., Ouyang, H., Gan, L., Peng, Y., Yao, J., & Chen, Y. (2025) · International Journal of Nursing Sciences · doi:10.1016/j.ijnss.2025.02.015 · Cited by 7
“The COSMIN Risk of Bias tool [15] was employed to evaluate reliability and measurement error, and the COSMIN Risk of Bias checklist [16] was used to assess various aspects of the instruments, including tool development, content validity, structural validity, internal consistency, cross-cultural…”
Lu, J. K., Wang, W., Goh, J., & Maier, A. B. (2025) · Mayo Clinic Proceedings Digital Health · doi:10.1016/j.mcpdig.2025.100202 · Cited by 8
“Some validation studies adhered to established guidelines, 13,39 ensuring reliable assessments of device performance.”
Khillan, A., Carter, L., Amor, D. J., Berryman, C. F., & Harvey, A. R. (2025) · Journal of Pain · doi:10.1016/j.jpain.2025.105388 · Cited by 8
Cooke, R., Rushton, A. B., Martín, J., Soundy, A., Herrington, L. C., & Heneghan, N. R. (2025) · BMJ Open Sport & Exercise Medicine · doi:10.1136/bmjsem-2024-002389 · Cited by 8
“21 Both tools use the ratings of ‘very good’, ‘adequate’, ‘doubtful’ or ‘inadequate’ quality, 21 26 with the overall quality rating for each measurement property within the study based on ‘the worst score counts principle’.”
InfluentialÖnal, B., Köse, N., Önal, Ş. N., & Zengin, H. Y. (2025) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.70141 · Cited by 8
“Our study was reported in accordance with the GRRAS and COSMIN checklists and its methodological quality was assessed [28, 29].”
Philip, S. S., Guzzetta, A., Gole, G. A., & Boyd, R. N. (2021) · Disability and Rehabilitation · doi:10.1080/09638288.2021.1990421 · Cited by 8
Ruffini, M., Berardi, A., Benvenuti, A., Beaudry‐Bellefeuille, I., Tofani, M., Galeoto, G., & Valente, D. (2022) · Children · doi:10.3390/children9071052 · Cited by 8
“Statistical Analysis Following the Consensus-Based Standards for the Selection of Health Status Measurement Instrument (COSMIN) checklist, the construct validity, internal consistency, and cross-cultural validity of the scale were assessed [27].”
MethodAbichandani, D., Ting, J. T. Y., Cancino, E. E., Althobaiti, S., & Falla, D. (2023) · Systematic Reviews · doi:10.1186/s13643-022-02162-5 · Cited by 12
Herren, D. B., Boeckstyns, M. E. H., Chung, K. C., Farnebo, S. J., Hagert, E. M., Tang, J. B., Verstreken, F., the FaiTh study group, & Marks, M. (2024) · Journal of Hand Surgery (European Volume) · doi:10.1177/17531934241227386 · Cited by 8
“As there is no formal definition of when consensus is achieved (Diamond et al., 2014), we decided to define consensus if 67% or more rated the treatment strategy as appropriate, which has also been used in other Delphi studies (Audig (cid:2) e and Schwyzer, 2019; Mokkink et al., 2020).”
Glover, P., Gray, H. G., Shanmugam, S., & McFadyen, A. K. (2021) · Journal of Interprofessional Care · doi:10.1080/13561820.2021.1902292 · Cited by 8
Verdú, M. M., Ferreira-Sánchez, M. D. R., Martín‐Casas, P., & Atín-Arratibel, M. D. L. Á. (2022) · Neurology Report · doi:10.1097/npt.0000000000000416 · Cited by 8
“Scores can be recorded as total (14-98), subscale (7-49), or individual item (1-7) values, where higher scores indicate better ability.”
BackgroundTakasaki, H., Kikkawa, K., Chiba, H., Handa, Y., Sesé, A., & Fernández‐Domínguez, J. C. (2021) · Progress in Rehabilitation Medicine · doi:10.2490/prm.20210034 · Cited by 8
Aveiro, B. D., Winsor, A., Davies, J., Nicholson, T. R., Pal, D. K., Richardson, M. P., & Pick, S. (2024) · Epilepsy & Behavior · doi:10.1016/j.yebeh.2024.109671 · Cited by 7
“The quality of each paper was assessed on the basis of the methods used to evaluate the PROM ’ s measurement properties, which included content validity, internal consistency, test – retest reliability, precision, and responsiveness, among others (see [20] for definitions).”
BackgroundMethodInfluentialWang, X., Rihari‐Thomas, J., Bail, K. S., Bala, N., & Traynor, V. (2024) · Journal of Advanced Nursing · doi:10.1111/jan.16358 · Cited by 7
“Utilizing measurement approaches with incomplete examination of psychometric properties can result in measurement in-accuracy, reduced confidence in the results and unreliable findings (Mokkink et al., 2020 ).”
Abe, H., Higashi, S., & Himuro, N. (2025) · Developmental Medicine & Child Neurology · doi:10.1111/dmcn.16465 · Cited by 7
“19 Specifically, we replaced the standard boxes for reliability and measurement error from the COSMIN Risk of Bias checklist with the COSMIN Risk of Bias tool designed for assessing the quality of studies on reliability and measurement error in performance-based outcome measures.”
Ndaa, M., Pandya, P., Swensson, J., Samala, N. R., Ajaz, S., Joshi, D., Ayoub, W. S., & Akisik, F. M. (2025) · Endocrine Practice · doi:10.1016/j.eprac.2025.09.205 · Cited by 7
The measurement properties reliability and measurement error explained – a COSMIN perspective
Mokkink, L. B., & Eekhout, I. E. (2025) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2025.112058 · Cited by 7
Self-citationWhat the mean absolute percentage error (MAPE) should adopt from Bland–Altman analyses
Warneke, K., Siegel, S. D., Afonso, J. R., & Wallot, S. (2025) · German Journal of Exercise and Sport Research · doi:10.1007/s12662-025-01084-3 · Cited by 11
Scoping review of exposure questionnaires and surveys in interstitial lung disease
Barnes, H., Elmrayed, S. A., Barber, C. M., Feary, J. R., Lee, C. T., Gandhi, S. A., Peters, C., Salisbury, M. L., & Johannson, K. A. (2024) · BMJ Open Respiratory Research · doi:10.1136/bmjresp-2023-002155 · Cited by 7
Morgan, A. R., Bégin, D., Heisz, J. J., Tang, A., Thabane, L., & Richardson, J. A. (2022) · Physical Therapy · doi:10.1093/ptj/pzac078 · Cited by 9
Penha, L. M. B., Pontes‐Silva, A., Santos‐de‐Araújo, A. D., Camargo, P. F., Pires, F. D. O., Gomes, C. A. F. D. P., Mostarda, C. T., Dibai, D. B., & Dibai‐Filho, A. V. (2023) · Journal of Chiropractic Medicine · doi:10.1016/j.jcm.2023.03.007 · Cited by 7
Ishigaki, T., Kubo, H., Yoshida, K., Shimizu, N., & Ogawa, T. (2024) · Frontiers in Rehabilitation Sciences · doi:10.3389/fresc.2024.1384369 · Cited by 7
van der Meer, H. A., Simanova, I., Kan, C. C., & Trujillo, J. P. (2021) · Journal of Autism and Developmental Disorders · doi:10.1007/s10803-021-05082-w · Cited by 7
McLaren, J., Fradera, A., & Cullen, B. (2024) · Neuropsychological Rehabilitation · doi:10.1080/09602011.2024.2357850 · Cited by 6
Piernicka, M., Labun, J., & Szumilewicz, A. (2025) · Journal of Clinical Medicine · doi:10.3390/jcm14134800 · Cited by 8
“The COSMIN Risk of Bias checklist was used to assess methodological quality and potential bias in the included NRCTs, following the recommendations of Mokkink et al. (2020) [26].”
InfluentialLi, S., Liu, S., Zhao, X., Chen, W., Fan, B., Wong, M. S., Li, J., & Jiang, M. (2025) · Archives of Physical Medicine and Rehabilitation · doi:10.1016/j.apmr.2025.08.009 · Cited by 7
Piscitelli, D., Buttram, A., Gibson, S., Hager, J., Thomas, B., & Solnik, S. (2023) · Journal of Biomechanics · doi:10.1016/j.jbiomech.2023.111902 · Cited by 6
Huang, J., Yang, J. A., Han, M., Xue, Z., Xu, M., Qi, H., Chen, J., Xue, C., & Wang, Y. (2024) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2024.104930 · Cited by 6
Antsygina, O., Rollo, S., McRae, N., Chaput, J., & Tremblay, M. S. (2024) · Sleep Medicine Reviews · doi:10.1016/j.smrv.2024.102023 · Cited by 7
Assessing the Content Validity of the Revised Health of the Nation Outcome Scales (HoNOS 2018)
Harris, M., Tapp, C., Arnautovska, U., Coombs, T., Dickson, R., James, M., Painter, J., Smith, M., Jury, A., Lai, J., et al. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph19169895 · Cited by 6
Klotz, T., Maddern, G. J., Caplash, Y., & Wagstaff, M. J. D. (2021) · JBI Evidence Synthesis · doi:10.11124/jbies-20-00468 · Cited by 6
Sheikhbahaeddinzadeh, E., Ashktorab, T., & Ebadi, A. (2022) · Journal of Education and Health Promotion · doi:10.4103/jehp.jehp_1779_21 · Cited by 6
“If article is not translated it is not necessary.[31] The two “CCMHN” and “Checklist of Clinical Competency in Mental Health Nursing Students” tools are suitable for the cultural context of Iran.”
BackgroundMethodInfluentialRamazanzadeh, N., Ghahramanian, A., Zamanzadeh, V., Onyeka, T. C., Valizadeh, L., & Ghaffarifar, S. (2023) · Nursing Open · doi:10.1002/nop2.1786 · Cited by 6
Jales, M. T. M., Barbosa, G. M., Gonçalves, G. V., Fialho, H. R. F., Calixtre, L. B., & Kamonseki, D. H. (2023) · Journal of Sport Rehabilitation · doi:10.1123/jsr.2022-0361 · Cited by 6
Leonardi, N. T., Kawakami, D. M. D. O., Hurst, J. R., Cruz, J., & Mendes, R. G. (2023) · European Respiratory Review · doi:10.1183/16000617.0013-2023 · Cited by 6
Systematic review: Validity, reliability, and diagnostic accuracy of the electrogustometer
Blijleven, E. E., Fuchten, D., Dullaart, M. J., Stokroos, R. J., Thomeer, H. G. X. M., & Wegner, I. (2023) · Laryngoscope Investigative Otolaryngology · doi:10.1002/lio2.1108 · Cited by 6
“Predefined criteria from the COnsensus-based Standards for the selection of health status Measurements INstruments (COSMIN) tool were used to assess the quality of the studies on reliability (Appendix S2).(28,29) Each box was scored on a four-point rating scale (i.”
Metadata
- Authors
- Mokkink, L. B., Boers, M., van der Vleuten, C. P. M., Bouter, L. M., Alonso, J., Patrick, D. L., de Vet, H. C. W., & Terwee, C. B.
- Affiliations
- Amsterdam Public Health (NL)
- Universitat Pompeu Fabra (ES)
- University of Washington (US)
- Corresponding author
- Mokkink, L. B.
- Venue
- BMC Medical Research Methodology
- Published
- 2020-12
- Volume
- 20
- Issue
- 1
- Pages
- 293-293
- ISSN
- 1471-2288 (ISSN-L)
- PMC
- PMC7712525
- OpenAlex
- W3107903990
- Cited by
- 588
- References
- 31
- Citations by year
- 2026: 106 · 2025: 121 · 2024: 149 · 2023: 101 · 2022: 72 · 2021: 37 · 2020: 2
- FWCI
- 32.32
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- BMC Medical Research Methodology (publishedVersion, cc-by)
- Research Square (acceptedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Research Publications (Maastricht University) (publishedVersion, cc-by)
- VU Research Portal (publishedVersion, other-oa)
- Pure Amsterdam UMC (submittedVersion, other-oa)
- Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Amsterdam UMC (VU Amsterdam) - Institutional Repository (submittedVersion)
- Repositori digital de la UPF (Universitat Pompeu Fabra) (submittedVersion, cc-by)
- Research Publications (Maastricht University) (submittedVersion, other-oa)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- Funders
- Nederlandse Organisatie voor Wetenschappelijk Onderzoek (91617098)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- COSMIN, risk of bias, reliability statistics, measurement error, outcome measurement instruments, clinician-reported outcomes, performance-based outcome measures, laboratory parameters, study quality assessment, Delphi consensus, measurement properties
- MeSH terms
- Delphi Technique, Humans, Reproducibility of Results, Bias, Consensus
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
