Cited by 588Reliability and Agreement in MeasurementDelphi Technique in ResearchMeta-analysis and systematic reviews

The DOI matches its Crossref record; 29 of the 31 references it cites were found in a registry or an index.

COSMIN Risk of Bias tool to assess the quality of studies on reliability or measurement error of outcome measurement instruments: a Delphi study

Open access

Mokkink, L. B., Boers, M., van der Vleuten, C. P. M., Bouter, L. M., Alonso, J., Patrick, D. L., de Vet, H. C. W., Terwee, C. B.·2020·BMC Medical Research Methodology

DOI 10.1186/s12874-020-01179-5PMC PMC7712525

Published version of a preprint

Open this paper in CoChat — free account Open PDF

Abstract

Background: Scores on an outcome measurement instrument depend on the type and settings of the instrument used, how instructions are given to patients, how professionals administer and score the instrument, etc. The impact of all these sources of variation on scores can be assessed in studies on reliability and measurement error, if properly designed and analyzed. The aim of this study was to develop standards to assess the quality of studies on reliability and measurement error of clinician-reported outcome measurement instruments, performance-based outcome measurement instrument, and laboratory values. Methods: We conducted a 3-round Delphi study involving 52 panelists. Results: Consensus was reached on how a comprehensive research question can be deduced from the design of a reliability study to determine how the results of a study inform us about the quality of the outcome measurement instrument at issue. Consensus was reached on components of outcome measurement instruments, i.e. the potential sources of variation. Next, we reached consensus on standards on design requirements ( n = 5), standards on preferred statistical methods for reliability ( n = 3) and measurement error ( n = 2), and their ratings on a four-point scale. There was one term for a component and one rating of one standard on which no consensus was reached, and therefore required a decision by the steering committee. Conclusion: We developed a tool that enables researchers with and without thorough knowledge on measurement properties to assess the quality of a study on reliability and measurement error of outcome measurement instruments.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.1186/s12874-020-01179-5

Cites

29 of 29 references verified0 citation errors · 0 not checkable

Statistics

Fig. 1 Content of the Delphi studyCC BY · PubMed Central

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Intraclass correlations: Uses in assessing rater reliability.
Shrout, P. E., & Fleiss, J. L. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.2.420 · Cited by 24,532
Convergent and discriminant validation by the multitrait-multimethod matrix.
Campbell, D. T., & Fiske, D. W. (1959) · Psychological Bulletin · doi:10.1037/h0046016 · Cited by 17,116
Confirmatory factor analysis for applied research
Brown, T. A. (2007) · Choice Reviews Online · doi:10.5860/choice.44-2769 · Cited by 15,621
QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies
Whiting, P. F., Rutjes, A. W. S., Westwood, M. E., Mallett, S., Deeks, J. J., Reitsma, J. B., Leeflang, M. M. G., Sterne, J. A. C., Bossuyt, P. M. M., & the QUADAS-2 Group* (2011) · Annals of Internal Medicine · doi:10.7326/0003-4819-155-8-201110180-00009 · Cited by 14,199
Health Measurement Scales: A Practical Guide to Their Development and Use
Streiner, D. L., Norman, G. R., & Cairney, J. (1989) · Cited by 9,595
Forming inferences about some intraclass correlation coefficients.
McGraw, K. O., & Wong, S. P. (1996) · Psychological Methods · doi:10.1037/1082-989x.1.1.30 · Cited by 6,903
The COSMIN study reached international consensus on taxonomy, terminology, and definitions of measurement properties for health-related patient-reported outcomes
Mokkink, L. B., Terwee, C. B., Patrick, D. L., Alonso, J., Stratford, P. W., Knol, D. L., Bouter, L. M., & de Vet, H. C. W. (2010) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.02.006 · Cited by 4,465
Self-citation
The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: an international Delphi study
Mokkink, L. B., Terwee, C. B., Patrick, D. L., Alonso, J., Stratford, P. W., Knol, D. L., Bouter, L. M., & de Vet, H. C. W. (2010) · Quality of Life Research · doi:10.1007/s11136-010-9606-8 · Cited by 4,260
Self-citation
COSMIN guideline for systematic reviews of patient-reported outcome measures
Prinsen, C. A. C., Mokkink, L. B., Bouter, L. M., Alonso, J., Patrick, D. L., de Vet, H. C. W., & Terwee, C. B. (2018) · Quality of Life Research · doi:10.1007/s11136-018-1798-3 · Cited by 3,345
Self-citation
The Delphi technique: myths and realities
Powell, C. (2003) · Journal of Advanced Nursing · doi:10.1046/j.1365-2648.2003.02537.x · Cited by 2,611

Show all 31 references

Research works printed but not linked: 2

Shavelson RJ, Webb NM. Generalizability Theory. A primer. Measurement Methods for the Social Science, vol. 1. Newbury Park: Sage Publishing; 1991.
CANCHILD Centre for disability research, I.o.a.h.s., McMaster University. Outcome measures rating form guideline. 2004[31/8/2017]; Available from: https://www.canchild.ca/system/tenon/assets/attachments/000/000/372/original/measrate.pdf?license=yes.

Sorted by citations

Items 401–500 of 587

Measurement properties of performance-based instruments to assess mental function during activity and participation in persons who have survived a stroke: A systematic review protocol
Kristensen, L. Q., van Tulder, M. W., Kristensen, H. K., Muren, M. A., Mokkink, L. B., & Oestergaard, L. G. (2022) · Scandinavian Journal of Occupational Therapy · doi:10.1080/11038128.2022.2053740 · Cited by 1
“This review will be conducted in accordance with the COnsensus-based Standards for the selection of health Measurements INstruments (COSMIN) methodology [23,24].”
BackgroundMethodInfluentialSelf-citation
Which objective itch‐assessment tools are applicable to patients with advanced cognitive impairments? A scoping review
Nitta, S., Maeda, T., Koudounas, S., Minematsu, T., Tobe, H., Weller, C. D., & Sanada, H. (2022) · International Journal of Older People Nursing · doi:10.1111/opn.12458 · Cited by 1
Fibrillar matrix, Echogenicity, Contour, Thickness, and Suture (FECTS) vs. Global: A comparison of 2 scales developed to assess ultrasound images post rotator cuff repair
Williams, T. G., Hackett, L. M., Lam, P. H., Stitz, D. J., Al‐Housni, H. S. A., Murrell, D. F., & Murrell, G. A. C. (2022) · Journal of Shoulder and Elbow Surgery · doi:10.1016/j.jse.2022.10.035 · Cited by 1
Categorization and recommendations for outcome measures for lower limb absence by an expert panel
Köhler, F., Halford, G. R. J., Lukin, M., Hafner, B. J., Boone, D. A., Desmond, D. M., Franchignoni, F. P., GAILEY, R. S., Hagberg, K., Major, M. J., et al. (2023) · Prosthetics and Orthotics International · doi:10.1097/pxr.0000000000000307 · Cited by 1
Is It Time to Go Beyond the Modified Ashworth Scale? Letter to the Editor on “Reliability of the Modified Ashworth Scale After Stroke for 13 Muscle Groups”
Piscitelli, D., Pellicciari, L., Vidmar, T., Goljar, N., & Puh, U. (2023) · Archives of Physical Medicine and Rehabilitation · doi:10.1016/j.apmr.2023.06.021 · Cited by 1
Protocol for a systematic review evaluating the measurement properties of mobility assessment scales used to evaluate patients in the emergency department.Preprint
Alves, L. D. D. S., Augusto, E. A. A., Chiarotto, A., Maher, C. G., Coombs, D., & Machado, G. C. (2023) · doi:10.31219/osf.io/w9tua · Cited by 1
Letter to the editor in response to the article of Sada, Abdullahi and Hassan (Hausa translation, cross-cultural adaptation, and assessment of psychometric properties of the motor activity log)
Ibrahim, A. A., & Kaka, B. (2021) · Disability and Rehabilitation · doi:10.1080/09638288.2020.1867911
“The calculation of standard error of measurement (SEM) and smallest detectable change (SDC) or limits of agreement are therefore recommended for pyschometric validation studies [3,5].”
Clinimetric Analysis of Outcome Measures for Airway Clearance in Adults with Cystic Fibrosis: A Systematic ReviewPreprint
Stanford, G. E., Jones, M., Charman, S. C., Bilton, D., Usmani, O. S., Davies, J. C., & Simmonds, N. J. (2021) · Research Square · doi:10.21203/rs.3.rs-585783/v1
Y Balance Test-aged: adaptação de instrumento e protocolo do YBT para mulheres idosas ativas estudo de confiabilidade
Lara, A. R. F. (2020) · doi:10.11606/d.100.2020.tde-12022021-112556
Measurement properties of remotely or self-administered physical performance measures to assess mobility: a systematic review protocol
Morgan, A. R., Bégin, D., Heisz, J. J., Tang, A., Thabane, L., & Richardson, J. A. (2021) · Physical Therapy Reviews · doi:10.1080/10833196.2021.1978779
Automatic market research of mobile health apps for the self-management of allergic rhinitis (Preprint)Preprint
Bousquet, J. (2021) · doi:10.2196/preprints.30081
A generic outcome assessment of mobility capacity in neurorehabilitation: measurement properties of the de Morton Mobility IndexPreprint
Braun, T., Marks, D., Thiel, C., & Grüneberg, C. (2021) · Research Square · doi:10.21203/rs.3.rs-343349/v1
Postal methods for monitoring HbA1c in diabetes mellitus: a protocol for systematic review
Colley, J., Dambha‐Miller, H., Stuart, B. L., Bartholomew, J., & Price, H. C. (2022) · BJGP Open · doi:10.3399/bjgpo.2021.0240
“Quality assessment The study will use ‘COnsensusbased Standards for the selection of health Measurement INstruments (COSMIN) Risk of Bias tool to assess the quality of studies on reliability and measurement error of outcome measurement instruments’ to assess the quality of selected studies.(30,31) The quality assessme…”
Method
Clinimetric properties of a postural assessment software in older people: reliability, standard error of measurement, and minimal detectable change
Barbosa, J. M. M., Santos, T. R. T., Santos, L. A. D. O., Cintra, T. M., Faccio, R. B. D. Á., & Pereira, L. S. M. (2022) · Geriatrics Gerontology and Aging · doi:10.53886/gga.e0220023
Reliability and validity of the East Asian children's version of mini-MACS in children with cerebral palsy
Hou, X., Qiu, H., Liu, L., Li, Y., He, L., Li, J., Tang, H., & Xu, K. (2022) · Frontiers in Rehabilitation Sciences · doi:10.3389/fresc.2022.997221
Developing guidelines for research institutions
Labib, K. (2023) · doi:10.5463/thesis.19
Systematic review of the measurement properties of functional performance tests and muscle strength tests in persons with anterior cruciate ligament or meniscal injuryPreprint
Berg, B., Whittaker, J. L., Øiestad, B. E., & Crossley, K. M. (2023) · doi:10.31219/osf.io/up3bk
Systematic review of the measurement properties of functional performance tests and muscle strength tests in persons with anterior cruciate ligament or meniscal injuryPreprint
Berg, B., Whittaker, J. L., Øiestad, B. E., & Crossley, K. M. (2023) · doi:10.31219/osf.io/752vu
Item distribution, internal consistency and structural validity of the German language Person-centred climate questionnaire - Staff version (PCQ-G-S): a cross-sectional studyPreprint
Wilfling, D., Mohler, R. A., Berg, A., Dörner, J., Bartmann, N., Klatt, T., Meyer, G., Halek, M., Köpke, S., & Dichter, M. (2023) · Research Square · doi:10.21203/rs.3.rs-2705323/v1
Reliability of offline inter-recti distance measurement on ultrasound images captured by novice examinersPreprint
Chmielewska, D. D., Cebula, M., Gnat, R., Rudek-Zeprzałka, M., Gruszczyńska, K., Baron, J., & Opala-Berdzik, A. (2023) · Research Square · doi:10.21203/rs.3.rs-2719788/v1
Cross-Cultural Adaptation and Psychometric Testing of the Urdu version of Copenhagen Neck Functional Disability Scale
Naz, S., Farooq, M. N., Iqbal, A., Naqvi, T. Z., & Rasul, S. M. F. U. (2023) · Pakistan Journal of Health Sciences · doi:10.54393/pjhs.v4i06.843
Development of Postgraduate Psychiatric Nursing Students' Clinical Competency Questionnaire.
Sheikhbahaeddinzadeh, E., Ashktorab, T., & Ebadi, A. (2023) · PubMed Central · doi:10.30476/jamp.2023.98879.1814
“Based on the COSMIN checklist, four domains of validity (face, content and construct validity), reliability (internal consistency, stability, SEM), responsiveness, and interpretability were evaluated for “PPNCCQ”.”
BackgroundMethodInfluential
Markerless Three-Dimensional Gait Analysis in Healthy Older Adults: Test-Retest Reliability, Standard Error of Measurement and Minimal Detectable ChangePreprint
Carvalho, A. N., Vanrenterghem, J., Cabral, S., Assunção, A., Fernandes, R. S., Veloso, A., & Moniz-Pereira, V. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4675743
Long-term outcome measures of atopic dermatitis gain importance: a validation study filling a research gap
von Kobyletzki, L. B., & Svensson, Å. (2024) · British Journal of Dermatology · doi:10.1093/bjd/ljae049
Modifying the Mobility Scale for Acute Stroke (MSAS) for All Stroke Phases (MSAllS): Measurement Properties and Clinical Application
Thilarajah, S., Low, A. Y., Lee, A. J., Ng, S., Seow, H. C., Choo, S., Bok, C. W., Silva, D. D., Chew, E. S., Chua, T. L., et al. (2024) · Archives of Physical Medicine and Rehabilitation · doi:10.1016/j.apmr.2024.02.722
Equilíbrio, quedas e risco de quedas na DPOC: revisão sistemática dos instrumentos de avaliação, propriedades de medida e utilidade clínica
Lamezon, A. C., Cavon, B. L., & Valderramas, S. (2024) · Revista Brasileira de Geriatria e Gerontologia · doi:10.1590/1981-22562024027.230189.pt
Psychometric Properties of Portable Devices to Assess Kinematic Post-Stroke Gait: A Systematic ReviewPreprint
da Silva, R. S., Silva, S., Cardoso, D. D. S., Quirino, M. A. F., Silva, M. H. A., Gomes, L., Fernandes, J. G., Oliveira, R., Fernandes, A. B. G. S., & Ribeiro, T. S. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4785726
Reliability and responsiveness of the Swedish short Hip‐RSI
Wörner, T., Sansone, M., Stålman, A., & Eek, F. (2024) · Journal of Experimental Orthopaedics · doi:10.1002/jeo2.12029
“This psychometric study was planned and reported with consideration of the COSMIN guidelines [10].”
Method
Cultural adaptation and validation in Italian of the Seated Postural Control Measure for Adults 2.0
Berardi, A., Simeon, R., Ricciardi, M., Tofani, M., Lucia, B., Aghilarre, F., Gagnon, B., Giuliani, S., & Galeoto, G. (2024) · Prosthetics and Orthotics International · doi:10.1097/pxr.0000000000000367
Retracted Article: Validity and Reliability of Sensor-based Measures of Lower Limb Range of Motion in Soccer Players: a Cross-sectional Study
Huber, S. D., & Alfuth, M. (2024) · Zeitschrift für Orthopädie und Unfallchirurgie · doi:10.1055/a-2331-1084
Retracted
Pain Sensitivity Questionnaire: Cross‐cultural adaptation and validation of the Greek version
Bilika, P., Aivatzidis, I., Kaloudis, K., Gkotzamanis, R., Ruscheweyh, R., & Kapreli, E. (2024) · Physiotherapy Research International · doi:10.1002/pri.2113
Evidence on psychometric properties of self-report questionnaires in evaluating blended learning in health sciences university students: research protocol for systematic review and meta-analysis
Dones, V. C., Dalusong, M. T., Manlapaz, D., Rojas, J. A. S., Ho, K. C. C., Reyes, J. J. R., Sangatanan, L. B., Narcelles, A. M. A., Ballesteros, M. B. B. P., Flores, R. K. S., et al. (2024) · BMJ Open · doi:10.1136/bmjopen-2023-075266
Development and content validation of a self-reported functional mobility assessment instrument for older adult patients
Lessa, E. A., Silva, C. F. E., Matos, L. R. R. D. S., & Trombini-Souza, F. (2024) · Revista Brasileira de Geriatria e Gerontologia · doi:10.1590/1981-22562024027.230252.en
“…to COSMIN guidelines 12,32 , the time interval between measurements should be sufficiently long to prevent memory bias regarding previous scores as they relate to intra-examiner reliability, yet sufficiently short so as to ensure patients do not present changes in the construct being measured 32 .”
BackgroundMethod
Objective assessment of diaper dermatitis
Janoušková, K., Resl, V., & Ratislavová, K. (2024) · Pediatrie pro praxi · doi:10.36290/ped.2024.065
Evaluation of the Psychometric Properties of the Musculoskeletal Health Questionnaire (MSK-HQ) in a Population of Kitesurfers: A Cross-Sectional Study
Tonti, M., Carlizza, A., & Galeoto, G. (2024) · Medicina · doi:10.3390/medicina60121995
Reliability Analysis of Proprioception Measurement Methods in Older Adults: A Systematic Review
Oh, J.-S., Lee, D.-Y., Hong, J.-H., Yu, J.-H., Nam, Y.-G., Kim, S.-G., & Kim, J.-S. (2024) · Journal of the Korean Society of Physical Medicine · doi:10.13066/kspm.2024.19.4.47
Reliability and Validity of Instruments Containing Reported Sleep Measures Among Children from Birth to <5 Years of Age: A Systematic Review
Antsygina, O. (2024) · doi:10.22215/etd/2024-16276
Avaliação da percepção auditiva da fala em crianças usuárias de dispositivos eletrônicos auxiliares de audição
de Jesus, K. D. C., Cidrônio, A. S., Fidêncio, V. L. D., & Vicente, L. C. (2024) · Distúrbios da Comunicação · doi:10.23925/2176-2724.2024v36i4e66741
The Eczema Area and Severity Index: An important update on validity and reliability
von Kobyletzki, L. B., & Svensson, Å. (2024) · Journal of the European Academy of Dermatology and Venereology · doi:10.1111/jdv.20415
Measurement of perceived healthcare discrimination and its implications on population health research: A systematic review protocolPreprint
Green, M. D., Harlow, A. B., Janek, S. E., Weinfert, K. P., Skalla, L. A., O'Brien, E. C., Brookhart, M. A., Návar, A. M., Muhigaba, P. B., Bentley‐Edwards, K. L., et al. (2024) · medRxiv · doi:10.1101/2024.12.18.24319227
Validity and Reliability of the Hungarian Version of the Pain Self-Efficacy Questionnaire Among Women with Endometriosis and Chronic Pelvic Pain
Kovács-Szabó, Z., Makai, A., Ács, P., & Hock, M. (2025) · Women s Health Reports · doi:10.1089/whr.2024.0109
“We asked 39 participants to ﬁ ll in the questionnaire 2 weeks after the initial testing according to the COnsenus-based Standards for the selection of health status Measurement INstruments (COSMIN) guideline and other relevant studies 31 – 33 (COSMIN checklist is in Supplementary Data S1).”
Measurement properties of the Trunk Impairment Scale-modified Norwegian version among people with mild to moderate multiple sclerosis
Dahl, S. S. H., Braaten, T., Arntzen, E. C., Verheyden, G., Fikke, H. K., & Normann, B. (2025) · Disability and Rehabilitation · doi:10.1080/09638288.2025.2454299
Psychometric properties of the Turkish version of the Fluid Intake Appraisal Inventory in adults receiving hemodialysis
Yangöz, Ş. T., Ceylan, H., Lindberg, M., & Özer, Z. (2025) · Therapeutic Apheresis and Dialysis · doi:10.1111/1744-9987.70003 · Cited by 1
Methods of measuring radiographic patellofemoral joint alignment and morphology: A scoping review
Hill, J. R., Oei, E. H. G., Crossley, K. M., Menz, H. B., Macri, E. M., Smith, M. D., Wyndow, N., Maclachlan, L., Ross, M. H., & Collins, N. J. (2025) · Osteoarthritis Imaging · doi:10.1016/j.ostima.2025.100262
“For studies that reported measurement properties, risk of bias was assessed using the COSMIN Risk of Bias tool, which was specifically designed to assess the quality of studies reporting the reliability and measurement error of outcome measurement instruments [29].”
Ensuring the Validity and Reliability of Research Instruments
(2025) · IGI Global eBooks · doi:10.4018/979-8-3693-7200-5.ch009
Impact of observation duration on behavioural pain assessment and intra-observer reliability in castrated piglets: A pilot study
Tomacheuski, R. M., Trindade, P. H. E., Merenda, V. R., Lopez-Soriano, M., & Pairis-Garcia, M. D. (2025) · Animal Welfare · doi:10.1017/awf.2025.22
Intra- and inter-rater reliability of anterior and posterior drawer tests for the assessment of people with shoulder instability
Aldon-Villegas, R., Chamorro-Moriana, G., Tarrida, P. L. L., & Benítez-Lugo, M.-L. (2025) · Clinical Rehabilitation · doi:10.1177/02692155251339380
The Lever Sign Test Demonstrates Limited Clinical Utility for Diagnosing Full-Thickness Anterior Cruciate Ligament Tears After a Traumatic Knee Injury
Norris, R. N., Price, A., Byrne, J., Pulford, S., van Melick, N., Maddox, T. W., Boswell, W., Kerin, C., & Oldershaw, R. A. (2025) · Orthopaedic Journal of Sports Medicine · doi:10.1177/23259671251334775
“To meet the COSMIN recommendations for an ‘‘excellent’’ sample size for reliability, a sample (cid:2) 100 is suggested 41 ; because this yielded the largest sample size, the sample was increased to a minimum of 100 participants.”
Properties of Measurements of Patient‐Reported Outcome Measures Physical Activity Assessment in Chronic Kidney Disease: A Systematic Review
Stella, T. C., Santos, I. S., Silva, G. A., Gomes, C. A. F. D. P., & Sampaio, L. M. M. (2025) · Physiotherapy Research International · doi:10.1002/pri.70076
Functional outcome measures for pediatric upper limb deficiencies with and without prostheses: A systematic review and appraisal
Cordray, H., Fiandeiro, M., Banala, M., Struble, S. L., Vaile, J. R., Pehnke, M., King, J. M., Shah, A. S., & Mendenhall, S. D. (2025) · Prosthetics and Orthotics International · doi:10.1097/pxr.0000000000000441
The Japanese Version of the Quality of Upper Extremity Skills Test
Katori, S., Himuro, N., Miura, M., Kitai, Y., Tanabe, R., & Ohnishi, H. (2025) · OTJR Occupational Therapy Journal of Research · doi:10.1177/15394492251349161
“Based on the COSMIN checklist (Mokkink et al., 2020), the target number for the reliability sample was 50, but this was not reached for two reasons: (a) the therapists could not find enough time to conduct the video evaluations (they were working on research data in their spare time from their…”
Enhancing Hand Dexterity Assessment Through Protocol Revision and Pilot Evaluation of the Virtual Eggs Test
Angelini, L., Piazzini, M., Guachi, R., Toso, F., Baccini, M., Cecchi, F., & Controzzi, M. (2025) · International Conference on Rehabilitation Robotics · doi:10.1109/icorr66766.2025.11063032
“We followed a two-step procedure according to the CONSMIN guidelines [26][27]: (1) evaluate the stability of patients; (2) compute the Intraclass Correlation Coefficient (ICC) with the 95% Confidence Interval (CI) if data were stable.”
Validation of the Test of Transversal Competencies (TECO-T) in Healthcare Students Form Sapienza University of Rome
Galeoto, G., Simeon, R., Valente, D., Padovani, A. M., Ruotolo, I., Amadio, E., Panuccio, F., & Berardi, A. (2025) · Communications in computer and information science · doi:10.1007/978-3-031-98003-9_20
Assessment of the psychometric properties of self-management measurement instruments for individuals with type 2 diabetes: A systematic review protocol
de Almeida, J. A. B., Monteiro, K. S., Santino, T. A., Florêncio, R. B., Fernandes, A. T. D. N. S. F., & Gualdi, L. P. (2025) · PLoS ONE · doi:10.1371/journal.pone.0330448
“The COSMIN Risk of Bias checklist, which includes 116 items, will be used to critically appraise the methodological quality of included studies [21,22].”
Landscape and Heterogeneity of Treatment Outcome Measures in Facioscapulohumeral Muscular Dystrophy: A Systematic Literature ReviewPreprint
Zhu, L., Eichinger, K. J., Barnieh, L., Beckerman, R., & Eldar‐Lissai, A. (2025) · medRxiv · doi:10.1101/2025.08.29.25334711
Measurement Properties and Muscular Demands of a Pneumatic Pain Meter: A Laboratory-Controlled Study
Summers, A., McGrath, A., Frey, M., Wong, A. Y. L., & Carvalho, D. D. (2025) · Measurement in Physical Education and Exercise Science · doi:10.1080/1091367x.2025.2567370
“To satisfy the COSMIN consensus criteria for investigating the criterion validity of performance-rated outcome measures (Mokkink et al., 2020), this laboratory-controlled cross-sectional study required 50 participants.”
Using Classical Test Theory to Determine the Psychometric Properties of the Italian Version of the Feeding/Swallowing Impact Survey
Crispiatico, V., Baffi, A., Buratti, M. A., Montali, L., & Speyer, R. (2025) · Journal of Clinical Medicine · doi:10.3390/jcm14217607
Interrater, test-retest, and intersession reliability of a test designed to measure tibialis posterior strength with a hand-held dynamometer
Bergdal, L., Svedmark, Å., Olofsson, L. B., & Tengman, E. (2025) · Journal of Bodywork and Movement Therapies · doi:10.1016/j.jbmt.2025.10.054
“The study was conducted in accordance with recommendations from the COSMIN Risk of Bias tool for studies investigating reliability (Mokkink et al., 2020).”
Cross-cultural adaptation of the Post-Intensive Care Syndrome Questionnaire to the Brazilian population: preliminary work
Rodrigues, G., Velloso, M., Leite, A. S., Barros, A. G. D. A., Marciano, I. A. V., Faria, I. D. C. B., Cândido, L. D. O., & Marinho, C. C. (2025) · Critical Care Science · doi:10.62675/2965-2774.20250104
Editorial: Advancing the science and practice of health care transition
BETZ, C. L. (2025) · Health Care Transitions · doi:10.1016/j.hctj.2025.100123
“17 It has been suggested that the intervention studies reported were initially developed as clinically-focused programs rather than developed for empirical purposes, which is an understandable development in an emerging field of practice and research.”
Reply to Wallace et al. Comment on “Panuccio et al. Quality of Assessment Tools for Aphasia: A Systematic Review. Brain Sci. 2025, 15, 271”
Panuccio, F., Rossi, G., Nuzzo, A. D., Ruotolo, I., Cianfriglia, G., Simeon, R., Sellitto, G., Berardi, A., & Galeoto, G. (2025) · Brain Sciences · doi:10.3390/brainsci15111234
Comment on Panuccio et al. Quality of Assessment Tools for Aphasia: A Systematic Review. Brain Sci. 2025, 15, 271
Wallace, S. J., Hilari, K., Willmes, K., Meinzer, M., Peñaloza, C., van Ewijk, L., Palmer, R., Zingelman, S., Hula, W. D., & Breitenstein, C. (2025) · Brain Sciences · doi:10.3390/brainsci15111233
Validity and Reliability of the Single Leg Bridge Test and Handheld Dynamometer for Assessing Knee Flexor Strength After Anterior Cruciate Ligament Reconstruction
Rodrigues, C. A. S., Albano, T. R., de Queiroz, J. H. M., de Souza, C. C. C., Lima, P. O. D. P., & Almeida, G. P. L. (2025) · Sports Health A Multidisciplinary Approach · doi:10.1177/19417381251387741
Development and Content Validity of the Pretrial Risks and Psychosocial Needs Assessment Questionnaire in Mexican Juvenile Offenders
Monroy-Vite, N., Fresán, A., Olvera, C. P. L., & Martínez-López, J. N. I. (2025) · Journal of Police and Criminal Psychology · doi:10.1007/s11896-025-09775-1
“First, their methodological quality is appraised using the COS-MIN Risk of Bias Checklist, using a four-point scale ranging from inadequate to very good (Mokkink et al. 2018, 2020; Terwee et al. 2018).”
Self-management measurement instruments specific to individuals with cardiovascular diseases: a systematic review protocol using COSMIN methodology
de Almeida, J. A. B., Monteiro, K. S., Santino, T. A., Florêncio, R. B., Fernandes, A. T. D. N. S. F., & Gualdi, L. P. (2025) · BMJ Open · doi:10.1136/bmjopen-2025-104523
Additional file 1 of A generic outcome assessment of mobility capacity in neurorehabilitation: measurement properties of the de Morton Mobility Index
Braun, T., Marks, D., Thiel, C., & Grüneberg, C. (2021) · Figshare · doi:10.6084/m9.figshare.15071213
Additional file 1 of Study protocol for developing, piloting and disseminating the PRISMA-COSMIN guideline: a new reporting guideline for systematic reviews of outcome measurement instruments
Elsman, E. B. M., Butcher, N. J., Mokkink, L. B., Terwee, C. B., Tricco, A., Gagnier, J. J., Aiyegbusi, O. L., Barnett, C., Smith, M., Moher, D., et al. (2022) · Figshare · doi:10.6084/m9.figshare.20064959
Self-citation
Additional file 1 of Psychometric properties of the ethical conflict in nursing questionnaire critical care version among Chinese nurses: a cross-sectional study
Liu, Y., Cui, N., Zhang, Y., Wang, X., Zhang, H., Chen, D., Sun, S., & Jingfen, J. (2021) · Figshare · doi:10.6084/m9.figshare.15071297
Additional file 1 of Translation and evaluation of the simplified Chinese version of the rating form of IBD patient concerns
Luo, J., Zhong, J., Li, H., Zhang, S., Zhang, L., Hou, J.-T., Ke, J., Li, H., Liu, F., & Chen, X.-L. (2022) · Figshare · doi:10.6084/m9.figshare.21192256
Additional file 5 of Guideline for reporting systematic reviews of outcome measurement instruments (OMIs): PRISMA-COSMIN for OMIs 2024
Elsman, E. B. M., Mokkink, L. B., Terwee, C. B., Beaton, D., Gagnier, J. J., Tricco, A. C., Baba, A., Butcher, N. J., Smith, M., Hofstetter, C., et al. (2024) · Figshare · doi:10.6084/m9.figshare.26740724
Self-citation
Additional file 1 of Measurement properties of cervical joint position error in people with and without neck pain: a systematic review and narrative synthesis
AlDahas, A., Heneghan, N. R., Althobaiti, S., Deane, J. A., Rushton, A. B., & Falla, D. (2024) · Figshare · doi:10.6084/m9.figshare.26665921
Additional file 6 of Measurement properties of cervical joint position error in people with and without neck pain: a systematic review and narrative synthesis
AlDahas, A., Heneghan, N. R., Althobaiti, S., Deane, J. A., Rushton, A. B., & Falla, D. (2024) · Figshare · doi:10.6084/m9.figshare.26665939
Additional file 1 of Guideline for reporting systematic reviews of outcome measurement instruments (OMIs): PRISMA-COSMIN for OMIs 2024
Elsman, E. B. M., Mokkink, L. B., Terwee, C. B., Beaton, D., Gagnier, J. J., Tricco, A. C., Baba, A., Butcher, N. J., Smith, M., Hofstetter, C., et al. (2024) · Figshare · doi:10.6084/m9.figshare.26740712
Self-citation
Additional file 2 of Guideline for reporting systematic reviews of outcome measurement instruments (OMIs): PRISMA-COSMIN for OMIs 2024
Elsman, E. B. M., Mokkink, L. B., Terwee, C. B., Beaton, D., Gagnier, J. J., Tricco, A. C., Baba, A., Butcher, N. J., Smith, M., Hofstetter, C., et al. (2024) · Figshare · doi:10.6084/m9.figshare.26740715
Self-citation
Additional file 2 of A meta-analysis of the criterion-related validity of Session-RPE scales in adolescent athletes
Liu, H., Yang, W., Liu, H., Bao, D., Cui, Y., Ho, I. M. K., & Li, Q. (2023) · Figshare · doi:10.6084/m9.figshare.23937241
Additional file 3 of A meta-analysis of the criterion-related validity of Session-RPE scales in adolescent athletes
Liu, H., Yang, W., Liu, H., Bao, D., Cui, Y., Ho, I. M. K., & Li, Q. (2023) · Figshare · doi:10.6084/m9.figshare.23937244
Additional file 1 of Cross-cultural adaption and inter-rater reliability of the Swedish version of the updated clinical frailty scale 2.0
Olsson, H., Åhlund, K., Alfredsson, J., Andersson, D., Boström, A.-M., Guidetti, S. G. E., Prytz, M., & Ekerstad, N. (2023) · Figshare · doi:10.6084/m9.figshare.24751382
Additional file 5 of Cross-cultural adaption and inter-rater reliability of the Swedish version of the updated clinical frailty scale 2.0
Olsson, H., Åhlund, K., Alfredsson, J., Andersson, D., Boström, A.-M., Guidetti, S. G. E., Prytz, M., & Ekerstad, N. (2024) · Figshare · doi:10.6084/m9.figshare.26649615
Additional file 3 of Guideline for reporting systematic reviews of outcome measurement instruments (OMIs): PRISMA-COSMIN for OMIs 2024
Elsman, E. B. M., Mokkink, L. B., Terwee, C. B., Beaton, D., Gagnier, J. J., Tricco, A. C., Baba, A., Butcher, N. J., Smith, M., Hofstetter, C., et al. (2024) · Figshare · doi:10.6084/m9.figshare.26740718
Self-citation
Additional file 4 of Guideline for reporting systematic reviews of outcome measurement instruments (OMIs): PRISMA-COSMIN for OMIs 2024
Elsman, E. B. M., Mokkink, L. B., Terwee, C. B., Beaton, D., Gagnier, J. J., Tricco, A. C., Baba, A., Butcher, N. J., Smith, M., Hofstetter, C., et al. (2024) · Figshare · doi:10.6084/m9.figshare.26740721
Self-citation
Additional file 1 of Item distribution, internal consistency and structural validity of the German language person-centred climate questionnaire - staff version (PCQ-G-S): a cross-sectional study
Wilfling, D., Mohler, R. A., Berg, A., Dörner, J., Bartmann, N., Klatt, T., Meyer, G., Halek, M., Köpke, S., & Dichter, M. N. (2024) · Figshare · doi:10.6084/m9.figshare.24993652
Additional file 1 of Reliability and validity of ultrasonography in evaluating the thickness, excursion, stiffness, and strain rate of respiratory muscles in non-hospitalized individuals: a systematic review
Zhou, E. F. M., Fu, S. N., Huang, C., Huang, X. P., & Wong, A. Y. L. (2024) · Figshare · doi:10.6084/m9.figshare.26648634
Additional file 3 of The psychometric properties of the Quality of Life in Neurological Disorders (Neuro-QoL) measurement system in neurorehabilitation populations: a systematic review
Ataman, R., Alhasani, R., Auneau-Enjalbert, L., Quigley, A., Michael, H. U., & Ahmed, S. (2024) · Figshare · doi:10.6084/m9.figshare.27058958
From Algorithm to AI
Saoud, S., Nassiri, N., & Mahani, Z. (2025) · Advances in educational technologies and instructional design book series · doi:10.4018/979-8-3693-8965-2.ch003
Additional file 5 of Measurement properties of cervical joint position error in people with and without neck pain: a systematic review and narrative synthesis
AlDahas, A., Heneghan, N. R., Althobaiti, S., Deane, J. A., Rushton, A. B., & Falla, D. (2024) · Figshare · doi:10.6084/m9.figshare.26665936
Additional file 3 of Cross-cultural adaption and inter-rater reliability of the Swedish version of the updated clinical frailty scale 2.0
Olsson, H., Åhlund, K., Alfredsson, J., Andersson, D., Boström, A.-M., Guidetti, S. G. E., Prytz, M., & Ekerstad, N. (2024) · Figshare · doi:10.6084/m9.figshare.26649609
Additional file 1 of Measurement properties of smartphone applications for the measurement of neck range of motion: a systematic review and meta analyses
Cancino, E. E., Rice, K., Abichandani, D., & Falla, D. (2022) · Figshare · doi:10.6084/m9.figshare.19159228
Additional file 2 of Measurement properties of cervical joint position error in people with and without neck pain: a systematic review and narrative synthesis
AlDahas, A., Heneghan, N. R., Althobaiti, S., Deane, J. A., Rushton, A. B., & Falla, D. (2024) · Figshare · doi:10.6084/m9.figshare.26665927
Additional file 2 of Measurement properties of smartphone applications for the measurement of neck range of motion: a systematic review and meta analyses
Cancino, E. E., Rice, K., Abichandani, D., & Falla, D. (2022) · Figshare · doi:10.6084/m9.figshare.19159231
Additional file 4 of Cross-cultural adaption and inter-rater reliability of the Swedish version of the updated clinical frailty scale 2.0
Olsson, H., Åhlund, K., Alfredsson, J., Andersson, D., Boström, A.-M., Guidetti, S. G. E., Prytz, M., & Ekerstad, N. (2024) · Figshare · doi:10.6084/m9.figshare.26649612
Additional file 3 of Measurement properties of cervical joint position error in people with and without neck pain: a systematic review and narrative synthesis
AlDahas, A., Heneghan, N. R., Althobaiti, S., Deane, J. A., Rushton, A. B., & Falla, D. (2024) · Figshare · doi:10.6084/m9.figshare.26665930
Additional file 1 of Measures of neck muscle strength and their measurement properties in adults with chronic neck pain—a systematic review
Abichandani, D., Ting, J. T. Y., Cancino, E. E., Althobaiti, S., & Falla, D. (2024) · Figshare · doi:10.6084/m9.figshare.26561500
Additional file 2 of A generic outcome assessment of mobility capacity in neurorehabilitation: measurement properties of the de Morton Mobility Index
Braun, T., Marks, D., Thiel, C., & Grüneberg, C. (2021) · Figshare · doi:10.6084/m9.figshare.15071216
Additional file 1 of The psychometric properties of the Quality of Life in Neurological Disorders (Neuro-QoL) measurement system in neurorehabilitation populations: a systematic review
Ataman, R., Alhasani, R., Auneau-Enjalbert, L., Quigley, A., Michael, H. U., & Ahmed, S. (2024) · Figshare · doi:10.6084/m9.figshare.27058952
Additional file 2 of The psychometric properties of the Quality of Life in Neurological Disorders (Neuro-QoL) measurement system in neurorehabilitation populations: a systematic review
Ataman, R., Alhasani, R., Auneau-Enjalbert, L., Quigley, A., Michael, H. U., & Ahmed, S. (2024) · Figshare · doi:10.6084/m9.figshare.27058955
Additional file 2 of Cross-cultural adaption and inter-rater reliability of the Swedish version of the updated clinical frailty scale 2.0
Olsson, H., Åhlund, K., Alfredsson, J., Andersson, D., Boström, A.-M., Guidetti, S. G. E., Prytz, M., & Ekerstad, N. (2024) · Figshare · doi:10.6084/m9.figshare.24751605
Additional file 1 of PROs for RARE: protocol for development of a core patient reported outcome set for individuals with genetic intellectual disability
van Silfhout, N. Y., van Muilekom, M. M., van Karnebeek, C., Haverman, L., & van Eeghen, A. M. (2024) · Figshare · doi:10.6084/m9.figshare.27123776
Additional file 4 of Measurement properties of cervical joint position error in people with and without neck pain: a systematic review and narrative synthesis
AlDahas, A., Heneghan, N. R., Althobaiti, S., Deane, J. A., Rushton, A. B., & Falla, D. (2024) · Figshare · doi:10.6084/m9.figshare.26665933
Additional file 1 of A meta-analysis of the criterion-related validity of Session-RPE scales in adolescent athletes
Liu, H., Yang, W., Liu, H., Bao, D., Cui, Y., Ho, I. M. K., & Li, Q. (2023) · Figshare · doi:10.6084/m9.figshare.23937238
Additional file 4 of The psychometric properties of the Quality of Life in Neurological Disorders (Neuro-QoL) measurement system in neurorehabilitation populations: a systematic review
Ataman, R., Alhasani, R., Auneau-Enjalbert, L., Quigley, A., Michael, H. U., & Ahmed, S. (2024) · Figshare · doi:10.6084/m9.figshare.27058961

Authors: Mokkink, L. B., Boers, M., van der Vleuten, C. P. M., Bouter, L. M., Alonso, J., Patrick, D. L., de Vet, H. C. W., & Terwee, C. B.
Affiliations: Amsterdam Public Health (NL)
Universitat Pompeu Fabra (ES)
University of Washington (US)
Corresponding author: Mokkink, L. B.
ORCID: Mokkink, L. B.
Boers, M.
Bouter, L. M.
Alonso, J.
Patrick, D. L.
Terwee, C. B.
Venue: BMC Medical Research Methodology
Publisher: Springer Science and Business Media LLC
Published: 2020-12
Volume: 20
Issue: 1
Pages: 293-293
ISSN: 1471-2288 (ISSN-L)
DOI: 10.1186/s12874-020-01179-5
PMC: PMC7712525
OpenAlex: W3107903990
Cited by: 588
References: 31
Citations by year: 2026: 106 · 2025: 121 · 2024: 149 · 2023: 101 · 2022: 72 · 2021: 37 · 2020: 2
FWCI: 32.32
Type: journal article
Language: en
Open access: gold
doi.org
Licence: cc-by
creativecommons.org/licenses/by/4.0/ (tdm)
creativecommons.org/licenses/by/4.0/ (vor, 2 d)
Locations: BMC Medical Research Methodology (publishedVersion, cc-by)
Research Square (acceptedVersion, cc-by)
PubMed Central (publishedVersion, cc-by)
PubMed (publishedVersion)
Research Publications (Maastricht University) (publishedVersion, cc-by)
VU Research Portal (publishedVersion, other-oa)
Pure Amsterdam UMC (submittedVersion, other-oa)
Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Amsterdam UMC (VU Amsterdam) - Institutional Repository (submittedVersion)
Repositori digital de la UPF (Universitat Pompeu Fabra) (submittedVersion, cc-by)
Research Publications (Maastricht University) (submittedVersion, other-oa)
Data Archiving and Networked Services (DANS) (submittedVersion)
Funders: Nederlandse Organisatie voor Wetenschappelijk Onderzoek (91617098)
Topics: Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
Social Sciences › Sociology and Political Science › Delphi Technique in Research
Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
Keywords: COSMIN, risk of bias, reliability statistics, measurement error, outcome measurement instruments, clinician-reported outcomes, performance-based outcome measures, laboratory parameters, study quality assessment, Delphi consensus, measurement properties
MeSH terms: Delphi Technique, Humans, Reproducibility of Results, Bias, Consensus

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

COSMIN Risk of Bias tool to assess the quality of studies on reliability or measurement error of outcome measurement instruments: a Delphi study

Abstract

Overview

Integrity signals

Figures

Verification

Claims the paper makes

Self-check

Citations

Cites

Cited by

Related

Metadata

Record

About these checks