COSMIN Risk of Bias tool to assess the quality of studies on reliability or measurement error of outcome measurement instruments: a Delphi study

Mokkink, L. B.; Boers, M.; van der Vleuten, C. P. M.; Bouter, L. M.; Alonso, J.; Patrick, D. L.; de Vet, H. C. W.; Terwee, C. B.

doi:10.1186/s12874-020-01179-5

Cited by 588Reliability and Agreement in MeasurementDelphi Technique in ResearchMeta-analysis and systematic reviews

COSMIN Risk of Bias tool to assess the quality of studies on reliability or measurement error of outcome measurement instruments: a Delphi study

Open access

Mokkink, L. B., Boers, M., van der Vleuten, C. P. M., Bouter, L. M., Alonso, J., Patrick, D. L., de Vet, H. C. W., Terwee, C. B.·2020·BMC Medical Research Methodology

DOI 10.1186/s12874-020-01179-5PMC PMC7712525

Published version of a preprint

Open this paper in CoChat — free account Open PDF

Abstract

Background: Scores on an outcome measurement instrument depend on the type and settings of the instrument used, how instructions are given to patients, how professionals administer and score the instrument, etc. The impact of all these sources of variation on scores can be assessed in studies on reliability and measurement error, if properly designed and analyzed. The aim of this study was to develop standards to assess the quality of studies on reliability and measurement error of clinician-reported outcome measurement instruments, performance-based outcome measurement instrument, and laboratory values. Methods: We conducted a 3-round Delphi study involving 52 panelists. Results: Consensus was reached on how a comprehensive research question can be deduced from the design of a reliability study to determine how the results of a study inform us about the quality of the outcome measurement instrument at issue. Consensus was reached on components of outcome measurement instruments, i.e. the potential sources of variation. Next, we reached consensus on standards on design requirements ( n = 5), standards on preferred statistical methods for reliability ( n = 3) and measurement error ( n = 2), and their ratings on a four-point scale. There was one term for a component and one rating of one standard on which no consensus was reached, and therefore required a decision by the steering committee. Conclusion: We developed a tool that enables researchers with and without thorough knowledge on measurement properties to assess the quality of a study on reliability and measurement error of outcome measurement instruments.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.1186/s12874-020-01179-5

Cites

29 of 29 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Intraclass correlations: Uses in assessing rater reliability.
Shrout, P. E., & Fleiss, J. L. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.2.420 · Cited by 23,433 · see citing works
Convergent and discriminant validation by the multitrait-multimethod matrix.
Campbell, D. T., & Fiske, D. W. (1959) · Psychological Bulletin · doi:10.1037/h0046016 · Cited by 17,116 · see citing works
Confirmatory factor analysis for applied research
Brown, T. A. (2007) · Choice Reviews Online · doi:10.5860/choice.44-2769 · Cited by 15,613 · see citing works
QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies
Whiting, P. F., Rutjes, A. W. S., Westwood, M. E., Mallett, S., Deeks, J. J., Reitsma, J. B., Leeflang, M. M. G., Sterne, J. A. C., Bossuyt, P. M. M., & the QUADAS-2 Group* (2011) · Annals of Internal Medicine · doi:10.7326/0003-4819-155-8-201110180-00009 · Cited by 14,199 · see citing works
Health Measurement Scales: A Practical Guide to Their Development and Use
Streiner, D. L., Norman, G. R., & Cairney, J. (1989) · Cited by 9,595 · see citing works
Forming inferences about some intraclass correlation coefficients.
McGraw, K. O., & Wong, S. P. (1996) · Psychological Methods · doi:10.1037/1082-989x.1.1.30 · Cited by 6,903 · see citing works
The COSMIN study reached international consensus on taxonomy, terminology, and definitions of measurement properties for health-related patient-reported outcomes
Mokkink, L. B., Terwee, C. B., Patrick, D. L., Alonso, J., Stratford, P. W., Knol, D. L., Bouter, L. M., & de Vet, H. C. W. (2010) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.02.006 · Cited by 4,465 · see citing works
Self-citation
The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: an international Delphi study
Mokkink, L. B., Terwee, C. B., Patrick, D. L., Alonso, J., Stratford, P. W., Knol, D. L., Bouter, L. M., & de Vet, H. C. W. (2010) · Quality of Life Research · doi:10.1007/s11136-010-9606-8 · Cited by 4,260 · see citing works
Self-citation
COSMIN guideline for systematic reviews of patient-reported outcome measures
Prinsen, C. A. C., Mokkink, L. B., Bouter, L. M., Alonso, J., Patrick, D. L., de Vet, H. C. W., & Terwee, C. B. (2018) · Quality of Life Research · doi:10.1007/s11136-018-1798-3 · Cited by 3,345 · see citing works
Self-citation
The Delphi technique: myths and realities
Powell, C. (2003) · Journal of Advanced Nursing · doi:10.1046/j.1365-2648.2003.02537.x · Cited by 2,611 · see citing works
COSMIN Risk of Bias checklist for systematic reviews of Patient-Reported Outcome Measures
Mokkink, L. B., de Vet, H. C. W., Prinsen, C. A. C., Patrick, D. L., Alonso, J., Bouter, L. M., & Terwee, C. B. (2018) · Quality of Life Research · doi:10.1007/s11136-017-1765-4 · Cited by 2,479 · see citing works
Self-citation
COSMIN methodology for evaluating the content validity of patient-reported outcome measures: A Delphi study
Terwee, C. B., Prinsen, C. A. C., Chiarotto, A., Westerman, M. J., Patrick, D. L., Alonso, J., Bouter, L. M., de Vet, H. C. W., & Mokkink, L. B. (2018) · Quality of Life Research · doi:10.1007/s11136-018-1829-0 · Cited by 2,348 · see citing works
Self-citation
Measurement in Medicine: A Practical Guide
de Vet, H. C. W., Terwee, C. B., Mokkink, L. B., & Knol, D. L. (2011) · doi:10.1017/cbo9780511996214 · Cited by 2,321 · see citing works
Self-citation
Guidelines for Reporting Reliability and Agreement Studies (GRRAS) were proposed
Kottner, J., Audigé, L., Brorson, S., Donner, A. P., Gajewski, B. J., Hróbjartsson, A., Roberts, C., Shoukri, M. M., & Streiner, D. L. (2010) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.03.002 · Cited by 2,239 · see citing works
A methodological framework for assessing health indices
Kirshner, B., & Guyatt, G. (1985) · Journal of Chronic Diseases · doi:10.1016/0021-9681(85)90005-0 · Cited by 1,538 · see citing works
Untitled work
(2011) · Amsterdam UMC (VU Amsterdam) - Institutional Repository · Cited by 1,209 · see citing works
Self-citation
Generalizability Theory
Brennan, R. L. (2001) · doi:10.1007/978-1-4757-3456-0 · Cited by 1,121 · see citing works
Guidelines for Reporting Reliability and Agreement Studies (GRRAS) were proposed
Kottner, J., Audigé, L., Brorson, S., Donner, A. P., Gajewski, B. J., Hróbjartsson, A., Roberts, C., Shoukri, M. M., & Streiner, D. L. (2011) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2011.01.016 · Cited by 997 · see citing works
The development of a quality appraisal tool for studies of diagnostic reliability (QAREL)
Lucas, N. P., Macaskill, P., Irwig, L., & Bogduk, N. (2010) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2009.10.002 · Cited by 376 · see citing works
Clinical Outcome Assessments: Conceptual Foundation—Report of the ISPOR Clinical Outcomes Assessment – Emerging Good Practices for Outcomes Research Task Force
Walton, M. K., Powers, J. H., Hobart, J. C., Patrick, D. L., Marquis, P., Vamvakas, S., Isaac, M., Molsen, E., Cano, S., & Burke, L. B. (2015) · Value in Health · doi:10.1016/j.jval.2015.08.006 · Cited by 310 · see citing works
Self-citation
Upper extremity function in multiple sclerosis: improving assessment sensitivity with box-and-block and nine-hole peg tests.
Goodkin, D. E., Hertsgaard, D., & Seminary, J. (1988) · PubMed · Cited by 263 · see citing works
Generalizability theory for the perplexed: A practical introduction and guide: AMEE Guide No. 68
Blöch, R., & Norman, G. R. (2012) · Medical Teacher · doi:10.3109/0142159x.2012.703791 · Cited by 248 · see citing works
Assessing test–retest reliability of patient-reported outcome measures using intraclass correlation coefficients: recommendations for selecting and documenting the analytical formula
Qin, S., Nelson, L. M., McLeod, L., Eremenco, S. L., & Coons, S. J. (2018) · Quality of Life Research · doi:10.1007/s11136-018-2076-0 · Cited by 241 · see citing works
Clinician-Reported Outcome Assessments of Treatment Benefit: Report of the ISPOR Clinical Outcome Assessment Emerging Good Practices Task Force
Powers, J. H., Patrick, D. L., Walton, M. K., Marquis, P., Cano, S., Hobart, J., Isaac, M., Vamvakas, S., Slagle, A., Molsen, E., et al. (2017) · Value in Health · doi:10.1016/j.jval.2016.11.005 · Cited by 191 · see citing works
Self-citation
How reliable are reliability studies of fracture classifications?A systematic review of their methodologies
Audigé, L., Bhandari, M., & Kellam, J. F. (2004) · Acta Orthopaedica Scandinavica · doi:10.1080/00016470412331294445 · Cited by 186 · see citing works
How to evaluate and improve the reliability of power Doppler ultrasonography for assessing enthesitis in spondylarthritis
D’Agostino, M. A., Aegerter, P., Jousse‐Joulin, S., Chary-Valckenaere, I., Lecoq, B., Gaudin, P., Brault, I., Schmitz, J., Dehaut, F.-X., Parc, J. M. L., et al. (2008) · Arthritis Care & Research · doi:10.1002/art.24369 · Cited by 98 · see citing works
Development, intra- and inter-rater reliability of the New Zealand Secretion Scale (NZSS)
Miles, A., & Hunting, A. (2018) · International Journal of Speech-Language Pathology · doi:10.1080/17549507.2018.1458901 · Cited by 40 · see citing works
The Cleft Aesthetic Rating Scale for 18-Year-Old Unilateral Cleft Lip and Palate Patients
Mulder, F. J., Mosmuller, D. G. M., de Vet, H. C. W., Mouës, C. M., Breugem, C. C., van der Molen, A. B. M., & Griot, J. P. W. D. (2016) · The Cleft Palate-Craniofacial Journal · doi:10.1597/16-123 · Cited by 22 · see citing works
The Impact of Ignoring Measurement Error when Estimating Sample Size for Epidemiologic Studies
Devine, O. J. (2003) · Evaluation & the Health Professions · doi:10.1177/0163278703255232 · Cited by 15 · see citing works

Flagged first, then by citations

2 of the 31 references couldn’t be matched to a known paper.

Research works printed but not linked: 1

[3]Brennan RL Generalizability Theory 2001 New York Springer

Sorted by citations

COSMIN guideline for systematic reviews of patient-reported outcome measures version 2.0
Mokkink, L. B., Elsman, E. B. M., & Terwee, C. B. (2024) · Quality of Life Research · doi:10.1007/s11136-024-03761-6 · Cited by 465 · see citing works
“…other groups in conducting their reviews [e.g. 18–20], as well as discussions in two Delphi studies in which we developed the COSMIN Risk of Bias tool for assessing reliability and measurement error [21], and the PRISMA-COSMIN for Outcome Measurement Instruments (OMIs) reporting guideline [22].”
InfluentialSelf-citation
A systematic review of AI literacy scales
Lintner, T. (2024) · npj Science of Learning · doi:10.1038/s41539-024-00264-4 · Cited by 292 · see citing works
“…based Standards for the selection of health Measurement INstruments (COSMIN) 52 – 54 for the self-report scales, and additionally the COSMIN Risk of Bias tool to assess the quality of studies on reliability or measurement error of outcome measurement instruments 55 for the performance-based scales.”
Accuracy, Validity, and Reliability of Markerless Camera-Based 3D Motion Capture Systems versus Marker-Based 3D Motion Capture Systems in Gait Analysis: A Systematic Review and Meta-Analysis
Scataglini, S., Abts, E., Bocxlaer, C. V., den Bussche, M. V., Meletani, S., & Truijen, S. (2024) · Sensors · doi:10.3390/s24113686 · Cited by 165 · see citing works
“Within the included studies, nineteen [31–36, 38”
Method
Using a distribution-based approach and systematic review methods to derive minimum clinically important differences
Watt, J. A., Veroniki, A. A., Tricco, A. C., & Straus, S. E. (2021) · BMC Medical Research Methodology · doi:10.1186/s12874-021-01228-7 · Cited by 163 · see citing works
The current use of ultrasound to measure skeletal muscle and its ability to predict clinical outcomes: a systematic review
Casey, P., Alasmar, M., McLaughlin, J., Ang, Y., McPhee, J., Heire, P., & Sultan, J. (2022) · Journal of Cachexia, Sarcopenia and Muscle · doi:10.1002/jcsm.13041 · Cited by 164 · see citing works
“A score was determined by taking the lowest rating of each criterion and defined to be poor, fair, good, or excellent.(21) Methodological quality was scored independently between reviewers, and agreement assessed using Cohen’s kappa coefficient where a score of >0.”
Method
RoBDEMAT: A risk of bias tool and guideline to support reporting of pre-clinical dental materials research and assessment of systematic reviews
Delgado, A. H. S., Sauro, S., Lima, A. F., Loguércio, A. D., Bona, Á. D., Mazzoni, A., Collares, F. M., Staxrud, F., Ferracane, J. L., Tsoi, J. K. H., et al. (2022) · Journal of Dentistry · doi:10.1016/j.jdent.2022.104350 · Cited by 146 · see citing works
Psychological, psychiatric, and behavioral sciences measurement scales: best practice guidelines for their development and validation
Stefana, A., Damiani, S., Granziol, U., Provenzani, U., Solmi, M., Youngstrom, E. A., & Fusar‐Poli, P. (2025) · Frontiers in Psychology · doi:10.3389/fpsyg.2024.1494261 · Cited by 197 · see citing works
OPTIKNEE 2022: consensus recommendations to optimise knee health after traumatic knee injury to prevent osteoarthritis
Whittaker, J. L., Culvenor, A. G., Juhl, C. B., Berg, B., Bricca, A., Filbay, S. R., Holm, P. M., Macri, E. M., Urhausen, A. P., Ardern, C. L., et al. (2022) · British Journal of Sports Medicine · doi:10.1136/bjsports-2022-106299 · Cited by 132 · see citing works
Measuring what matters in healthcare: a practical guide to psychometric principles and instrument development
Swan, K., Speyer, R., Scharitzer, M., Farneti, D., Brown, T., Woisard, V., & Cordier, R. (2023) · Frontiers in Psychology · doi:10.3389/fpsyg.2023.1225850 · Cited by 142 · see citing works
“Measurement (SEM), Smallest Detectable Change (SDC), Limits of Agreement (LoA) or Coefficient of Variation (CV) (Mokkink et al., 2020) Dichotomous/nominal/ordinal scores: the percentage”
BackgroundMethodInfluential
Guideline for reporting systematic reviews of outcome measurement instruments (OMIs): PRISMA-COSMIN for OMIs 2024
Elsman, E. B. M., Mokkink, L. B., Terwee, C. B., Beaton, D., Gagnier, J. J., Tricco, A. C., Baba, A., Butcher, N. J., Smith, M., Hofstetter, C., et al. (2024) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2024.111422 · Cited by 133 · see citing works
Self-citation
Guideline for reporting systematic reviews of outcome measurement instruments (OMIs): PRISMA-COSMIN for OMIs 2024
Elsman, E. B. M., Mokkink, L. B., Terwee, C. B., Beaton, D., Gagnier, J. J., Tricco, A. C., Baba, A., Butcher, N. J., Smith, M., Hofstetter, C., et al. (2024) · Quality of Life Research · doi:10.1007/s11136-024-03634-y · Cited by 96 · see citing works
“An OMI consists of a set of components and phases, i.e., ‘equipment’, ‘preparatory actions’, ‘collection of raw data’, ‘data processing and storage’, and ‘assignment of the score’ [57].”
InfluentialSelf-citation
Studies on Reliability and Measurement Error of Measurements in Medicine – From Design to Statistics Explained for Medical Researchers
Mokkink, L. B., Eekhout, I. E., Boers, M., van der Vleuten, C. P. M., & de Vet, H. C. W. (2023) · Patient Related Outcome Measures · doi:10.2147/prom.s398886 · Cited by 90 · see citing works
“Often in the clinical field, the object of measurement represents patients, and we use patients as the population of interest in this paper, but it can also be proxies, healthcare professionals, or body structures (eg, joints or lesions).(2) Figure 4 Nested inter-rater reliability design where the raters are nested in…”
InfluentialSelf-citation
Comparison of 3D scanning versus traditional methods of capturing foot and ankle morphology for the fabrication of orthoses: a systematic review
Farhan, M., Wang, J. Z., Bray, P., Burns, J., & Cheng, T. L. (2021) · Journal of Foot and Ankle Research · doi:10.1186/s13047-020-00442-8 · Cited by 86 · see citing works
“COSMIN is the gold standard for patient reported outcome measurement evaluation, however more recently it has been applied to clinician-rated assessments [17, 20, 21].”
Method
Measurement properties of grimace scales for pain assessment in nonhuman mammals: a systematic review
Evangelista, M. C., Monteiro, B. P., & Steagall, P. V. M. (2021) · Pain · doi:10.1097/j.pain.0000000000002474 · Cited by 79 · see citing works
Development and validation of combined symptom‐medication scores for allergic rhinitis*
Sousa‐Pinto, B., Azevedo, L. F., Jutel, M., Agache, I. O., Canonica, G. W., Czarlewski, W., Papadopoulos, N. G., Bergmann, K., Devillier, P., Laune, D., et al. (2021) · Allergy · doi:10.1111/all.15199 · Cited by 77 · see citing works
Tribology properties of additively manufactured Ti6Al4V alloy after heat treatment
Huang, J., Zhang, W., Fang, W., & Yi, Y. (2023) · Tribology International · doi:10.1016/j.triboint.2023.108485 · Cited by 74 · see citing works
Interobserver variability studies in diagnostic imaging: a methodological systematic review
Quinn, L., Tryposkiadis, K., Deeks, J. J., de Vet, H. C. W., Mallett, S., Mokkink, L. B., Takwoingi, Y., Taylor‐Phillips, S., & Sitch, A. J. (2023) · British Journal of Radiology · doi:10.1259/bjr.20220972 · Cited by 71 · see citing works
Self-citation
Study protocol for developing, piloting and disseminating the PRISMA-COSMIN guideline: a new reporting guideline for systematic reviews of outcome measurement instruments
Elsman, E. B. M., Butcher, N. J., Mokkink, L. B., Terwee, C. B., Tricco, A. C., Gagnier, J. J., Aiyegbusi, O. L., Barnett, C., Smith, M., Moher, D., et al. (2022) · Systematic Reviews · doi:10.1186/s13643-022-01994-5 · Cited by 71 · see citing works
“Based on previous experiences [45, 11], it is anticipated that at the most 50% of the invited persons will complete at least one round.”
BackgroundSelf-citation
Reliability of conditioned pain modulation in healthy individuals and chronic pain patients: a systematic review and meta-analysis
Nuwailati, R., Bobos, P., Drangsholt, M. T., & Curatolo, M. (2022) · Scandinavian Journal of Pain · doi:10.1515/sjpain-2021-0149 · Cited by 66 · see citing works
A Systematic Review of Diagnostic Accuracy and Clinical Applications of Wearable Movement Sensors for Knee Joint Rehabilitation
Prill, R., Walter, M., Królikowska, A., & Becker, R. (2021) · Sensors · doi:10.3390/s21248221 · Cited by 65 · see citing works
“A COSMIN Risk of Bias tool was used to examine the quality in a systematic and transparent manner [28].”
Checklists, risk of bias tools, and reporting guidelines for research in orthopedics, sports medicine, and rehabilitation
Prill, R., Królikowska, A., de Girolamo, L., Becker, R., & Karlsson, J. (2023) · Knee Surgery Sports Traumatology Arthroscopy · doi:10.1007/s00167-023-07442-8 · Cited by 65 · see citing works
A systematic review of maintenance following intensive therapy programs in chronic post-stroke aphasia: importance of individual response analysis
Menahemi-Falkov, M., Breitenstein, C., Pierce, J. E., Hill, A. J., O’Halloran, R., & Rose, M. L. (2021) · Disability and Rehabilitation · doi:10.1080/09638288.2021.1955303 · Cited by 61 · see citing works
“…the variability between two assessments into account when the SEm is used for calculating critical change scores such as the MDC 90 cut-off value [42], preferably the “absolute agreement” “single measure” variant of the intra-class correlation coefficient/ICC, the generalisability coefficient or…”
Validity of Estimating the Maximal Oxygen Consumption by Consumer Wearables: A Systematic Review with Meta-analysis and Expert Statement of the INTERLIVE Network
Molina‐García, P., Notbohm, H. L., Schumann, M., Argent, R., Hetherington‐Rauth, M., Stang, J. S., Bloch, W., Cheng, S., Ekelund, U., Sardinha, L. B., et al. (2022) · Sports Medicine · doi:10.1007/s40279-021-01639-y · Cited by 50 · see citing works
Measurement properties for muscle strength tests following anterior cruciate ligament and/or meniscus injury: What tests to use and where do we need to go? A systematic review with meta-analyses for the OPTIKNEE consensus
Urhausen, A. P., Berg, B., Øiestad, B. E., Whittaker, J. L., Culvenor, A. G., Crossley, K. M., Juhl, C. B., & Risberg, M. A. (2022) · British Journal of Sports Medicine · doi:10.1136/bjsports-2022-105498 · Cited by 45 · see citing works
The Reliability of the Microsoft Kinect and Ambulatory Sensor-Based Motion Tracking Devices to Measure Shoulder Range-of-Motion: A Systematic Review and Meta-Analysis
Beshara, P., Anderson, D., Pelletier, M. H., & Walsh, W. R. (2021) · Sensors · doi:10.3390/s21248186 · Cited by 39 · see citing works
“The authors acknowledge an appropriate time interval depends on the stability of the construct (COSMIN item one), and the target population [88].”
BackgroundMethod
Devices measuring transepidermal water loss: A systematic review of measurement properties
Klotz, T., Ibrahim, A., Maddern, G. J., Caplash, Y., & Wagstaff, M. J. D. (2022) · Skin Research and Technology · doi:10.1111/srt.13159 · Cited by 39 · see citing works
Less is more: reliability and measurement error for three versions of the Tampa Scale of Kinesiophobia (TSK-11, TSK-13, and TSK-17) in patients with high-impact chronic pain
Eiger, B., Errebo, M., Straszek, C. L., & Vægter, H. B. (2022) · Scandinavian Journal of Pain · doi:10.1515/sjpain-2021-0200 · Cited by 37 · see citing works
Italian Validation of the Pittsburgh Sleep Quality Index (PSQI) in a Population of Healthy Children: A Cross Sectional Study
Scialpi, A., Mignolli, E., Vito, C. D., Berardi, A., Tofani, M., Valente, D., & Galeoto, G. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph19159132 · Cited by 36 · see citing works
Operationalising treatment success in aphasia rehabilitation
Breitenstein, C., Hilari, K., Menahemi-Falkov, M., Rose, M. L., Wallace, S. J., Brady, M., Hillis, A. E., Kiran, S., Szaflarski, J. P., Tippett, D. C., et al. (2022) · Aphasiology · doi:10.1080/02687038.2021.2016594 · Cited by 37 · see citing works
“An alternate option is that parallel versions of a test may be administered for repeated assessments to disentangle changes in 'true' scores from measurement error (e.g., using bi-factor models; Mokkink, Boers, van der Vleuten, Bouter, et al., 2020, p. 31).”
BackgroundMethod
What tests should be used to assess functional performance in youth and young adults following anterior cruciate ligament or meniscal injury? A systematic review of measurement properties for the OPTIKNEE consensus
Berg, B., Urhausen, A. P., Øiestad, B. E., Whittaker, J. L., Culvenor, A. G., Roos, E. M., Crossley, K. M., Juhl, C. B., & Risberg, M. A. (2022) · British Journal of Sports Medicine · doi:10.1136/bjsports-2022-105510 · Cited by 40 · see citing works
Systematic Review: The Measurement Properties of the Children’s Depression Rating Scale−Revised in Adolescents With Major Depressive Disorder
Stallwood, E., Monsour, A., Rodrigues, C., Monga, S., Terwee, C. B., Offringa, M., & Butcher, N. J. (2020) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1016/j.jaac.2020.10.009 · Cited by 37 · see citing works
Self-citation
Important Topics for Fostering Research Integrity by Research Performing and Research Funding Organizations: A Delphi Consensus Study
Labib, K., Roje, R., Bouter, L. M., Widdershoven, G. A. M., Evans, N., Marušić, A., Mokkink, L. B., & Tijdink, J. K. (2021) · Science and Engineering Ethics · doi:10.1007/s11948-021-00322-9 · Cited by 36 · see citing works
“This explanation is supported by the fact that two recent Delphi studies using the same means of reporting their response rates had similar figures to ours (Haven et al., 2020; Mokkink et al., 2020).”
ResultSelf-citation
Guidelines for the development and validation of patient-reported outcome measures: a scoping review
Jung, A., Challoumas, D., Pagels, L., Armijo-Olivo, S. L., Braun, T., & Luedtke, K. (2024) · BMJ Evidence-Based Medicine · doi:10.1136/bmjebm-2023-112681 · Cited by 36 · see citing works
“21 As a result, many of their methods are considered the current best reference standards in instrument development and (evaluation of) quality of measurement properties.”
Definitions of non-response and response to biological therapy for severe asthma: a systematic review
Khaleva, E., Rattu, A., Brightling, C. E., Bush, A., BOURDIN, A. A., Bossios, A., Chung, K. F., Chaudhuri, R., Coleman, C., Djukanović, R., et al. (2023) · ERJ Open Research · doi:10.1183/23120541.00444-2022 · Cited by 33 · see citing works
Explanation & Elaboration document of the COSMIN Reporting Guideline 2.0 for studies on measurement properties of patient-reported outcome measures
de Arruda, G. T., Terwee, C. B., Elsman, E. B. M., Ávila, M. A., Gagnier, J. J., Mokkink, L. B., PROM Reporting Group, Dibai-Filho, A. V., Firth, A. D., Mehdipour, A., et al. (2025) · Quality of Life Research · doi:10.1007/s11136-025-03949-4 · Cited by 32 · see citing works
“In our experience, methods guidelines and reporting guidelines need to be updated regularly, for example, the COSMIN Risk of bias checklist [5, 12, 19, 20] and the COSMIN guideline on conducting systematic reviews [1, 5] have been updated over the years, as have the CONSORT statement [21, 22] and…”
Self-citation
Measurement properties of pain scoring instruments in farm animals: A systematic review using the COSMIN checklist
Tomacheuski, R. M., Monteiro, B. P., Evangelista, M. C., Luna, S. P. L., & Steagall, P. V. M. (2023) · PLoS ONE · doi:10.1371/journal.pone.0280830 · Cited by 32 · see citing works
“Adapted from the Consensus-based Standards for the Selection of Health Measurement Instruments (COSMIN) [38, 41, 45, 47].”
MethodInfluential
Assessment of Measures of Head and Neck Lymphedema Following Head and Neck Cancer Treatment: A Systematic Review
Starmer, H. M., Cherry, M. G., Patterson, J. M., Young, B., & Fleming, J. C. (2022) · Lymphatic Research and Biology · doi:10.1089/lrb.2021.0100 · Cited by 31 · see citing works
“…for each tool was evaluated by one reviewer (H.S.) using the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) risk of bias checklist for reliability studies 16 and the Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2) 17 for validation studies.”
The accuracy, validity and reliability of Theia3D markerless motion capture for studying the biomechanics of human movement: A systematic review
Varcin, F., & Boocock, M. (2025) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2025.103332 · Cited by 32 · see citing works
Elastography in the assessment of the Achilles tendon: a systematic review of measurement properties
Mifsud, T., Gatt, A., Micallef‐Stafrace, K., Chockalingam, N., & Padhiar, N. (2023) · Journal of Foot and Ankle Research · doi:10.1186/s13047-023-00623-1 · Cited by 29 · see citing works
A systematic review of the psychometric properties of physical performance tests for sarcopenia in community-dwelling older adults
van Exter, S. H., Koenders, N., van der Wees, P. J., & van den Berg, M. G. A. (2024) · Age and Ageing · doi:10.1093/ageing/afae113 · Cited by 33 · see citing works
“The COSMIN Risk of Bias tool [17] was used to critically appraise studies reporting on reliability and measurement error.”
Method
Improving domain definition and outcome instrument selection: Lessons learned for OMERACT from imaging
D’Agostino, M. A., Beaton, D., Maxwell, L. J., Cembalo, S. M., Hoens, A. M., Hofstetter, C., Zăbălan, C., Bird, P., Christensen, R., de Wit, M. P. T., et al. (2021) · Seminars in Arthritis and Rheumatism · doi:10.1016/j.semarthrit.2021.08.004 · Cited by 29 · see citing works
Self-citation
Connor-Davidson Resilience Scale: a systematic review psychometrics properties using the COSMIN
Nia, H. S., Sánchez‐Teruel, D., Froelicher, E. S., Hejazi, S. S., Hosseini, L., Fomani, F. K., Moshtagh, M., Mollaei, F., Goudarzian, A. H., & Babaei, A. (2024) · Annals of Medicine and Surgery · doi:10.1097/ms9.0000000000001968 · Cited by 28 · see citing works
“In other words, assessing the measurement error in each psychometric study can guide the researchers to decide about the level of trust they can place in the study ﬁ ndings and whether the parameters that where estimated are systematically not over-or underestimated [101] .”
Measurement properties of smartphone applications for the measurement of neck range of motion: a systematic review and meta analyses
Elgueta-Cancino, E., Rice, K., Abichandani, D., & Falla, D. (2022) · BMC Musculoskeletal Disorders · doi:10.1186/s12891-022-05066-6 · Cited by 26 · see citing works
“The Consensus-based Standards for the Selection of Health Measurement Instruments (COSMIN) initiative established a taxonomy of measurement properties which covers the following domains: reliability, validity, and responsiveness [16].”
BackgroundMethodInfluential
Measurement matters: A commentary on the state of the science on patient reported outcome measures ( PROMs ) in autism research
Schiltz, H. K., Williams, Z. J., Zheng, S., Kaplan‐Kahn, E. A., Morton, H. E., Rosenau, K. A., Nicolaidis, C., Sturm, A., Lord, C., & PROMnet, A. (2024) · Autism Research · doi:10.1002/aur.3114 · Cited by 28 · see citing works
“Notably, a number of tools such as the COSMIN extensions (Gagnier et al., 2021; Mokkink et al., 2016, 2018, 2020; Prinsen et al., 2016, 2018; Terwee et al., 2018) and the ConPsy checklist (Vitoratou et al., 2024) are also available to guide the development and evaluation of psychometric instruments…”
Method
Arriving at the empirically based conceptualization of restricted and repetitive behaviors: A systematic review and meta-analytic examination of factor analyses
Uljarević, M., Spackman, E. K., Whitehouse, A., Frazier, T. W., Billingham, W., Condron, P., Hardan, A. Y., & Leekam, S. (2023) · Clinical Psychology Review · doi:10.1016/j.cpr.2023.102286 · Cited by 27 · see citing works
Can Clinicians Trust Objective Measures of Hip Muscle Strength From Portable Dynamometers? A Systematic Review With Meta-analysis and Evidence Gap Map of 107 Studies of Reliability and Criterion Validity Using the COSMIN Methodology
Waiteman, M. C., Garcia, M. C., Briani, R. V., Norte, G. E., Glaviano, N. R., de Azevedo, F. M., & Bazett-Jones, D. M. (2023) · Journal of Orthopaedic and Sports Physical Therapy · doi:10.2519/jospt.2023.12045 · Cited by 26 · see citing works
Measurement properties of performance-based measures to assess physical function in chronic kidney disease: recommendations from a COSMIN systematic review
MacRae, J. M., Harasemiw, O., Lightfoot, C. J., Thompson, S., Wytsma-Fisher, K., Koufaki, P., Bohm, C. J., & Wilkinson, T. J. (2023) · Clinical Kidney Journal · doi:10.1093/ckj/sfad170 · Cited by 26 · see citing works
“…measurement property ( very good/adequate/doubtful/inadequate ) and reached consensus on each item in the checklist, as it applied to the individual study; the rankings derived from this then informed the overall RoB score using the COSMIN RoB tool [ 15 , 17 ] ( Supplementary Material 3 and 4 ) .”
BackgroundMethodInfluential
Guideline for reporting systematic reviews of outcome measurement instruments (OMIs): PRISMA-COSMIN for OMIs 2024
Elsman, E. B. M., Mokkink, L. B., Terwee, C. B., Beaton, D., Gagnier, J. J., Tricco, A. C., Baba, A., Butcher, N. J., Smith, M., Hofstetter, C., et al. (2024) · Journal of Patient-Reported Outcomes · doi:10.1186/s41687-024-00727-7 · Cited by 25 · see citing works
“An OMI consists of a set of components and phases, i.e., ‘equipment’, ‘preparatory actions’, ‘collection of raw data’, ‘data processing and storage’, and ‘assignment of the score’ [57].”
BackgroundMethodInfluentialSelf-citation
A Systematic Review of the Accuracy, Validity, and Reliability of Markerless Versus Marker Camera-Based 3D Motion Capture for Industrial Ergonomic Risk Analysis
Scataglini, S., Fontinovo, E., Khafaga, N., Khan, M. U., Khan, M. F., & Truijen, S. (2025) · Sensors · doi:10.3390/s25175513 · Cited by 25 · see citing works
“The Consensus-Based Standards for the Selection of Health Measurement Instruments (COSMIN) [26,27] was used for methodological quality appraisal.”
Influential
Measurement properties of upper extremity physical performance tests in athletes: a systematic review
Barbosa, G. M., Calixtre, L. B., Fialho, H. R. F., Locks, F., & Kamonseki, D. H. (2023) · Brazilian Journal of Physical Therapy · doi:10.1016/j.bjpt.2023.100575 · Cited by 27 · see citing works
“The methodological quality of inter-session SEM and the MDC 95 was very good, 9,10 and no evidence was established for measurement error because the MIC have not been de ﬁ ned.”
BackgroundMethodInfluential
A practical guide for selecting continuous monitoring wearable devices for community-dwelling adults
Lu, J. K., Wang, W., Goh, J., & Maier, A. B. (2024) · Heliyon · doi:10.1016/j.heliyon.2024.e33488 · Cited by 25 · see citing works
“The COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) Risk of Bias tool was integrated to ensure a procedure for evaluating technical performance and validation was included [13].”
BackgroundMethod
Interpreting Change in Disorders of Consciousness Using the Coma Recovery Scale-Revised
Weaver, J., Cogan, A. M., Kozlowski, A. J., Grady-Dominguez, P., O’Brien, K. A., Bodien, Y. G., Graham, J. E., Aichele, S., Ford, P., Kot, T., et al. (2024) · Journal of Neurotrauma · doi:10.1089/neu.2023.0567 · Cited by 24 · see citing works
Within-subject variation of HbA1c: A systematic review and meta-analysis
Gough, A., Sitch, A. J., Ferris, E., & Marshall, T. P. (2023) · PLoS ONE · doi:10.1371/journal.pone.0289085 · Cited by 22 · see citing works
Psychometric properties of the psychosocial impact of assistive devices scale (PIADS): A systematic review
Atigossou, O. L. G., Honado, A. S., Routhier, F., & Flamand, V. H. (2021) · Assistive Technology · doi:10.1080/10400435.2021.2010149 · Cited by 22 · see citing works
“The COnsensus-based Standards for the Selection of Health Measurement INstruments Risk of Bias checklist (COSMINRoB) was used to critically appraise the risk of bias of each psychometric data reported in each of the included studies (Mokkink et al., 2020, 2018).”
Method
Assessing anaerobic speed reserve: A systematic review on the validity and reliability of methods to determine maximal aerobic speed and maximal sprinting speed in running-based sports
Thron, M., Düking, P., Ruf, L., Härtel, S., Wöll, A., & Altmann, S. (2024) · PLoS ONE · doi:10.1371/journal.pone.0296866 · Cited by 22 · see citing works
Quality of measurement properties of medication adherence instruments in cardiovascular diseases and type 2 diabetes mellitus: a systematic review and meta-analysis
Oliveira, H. C., Neto, D. H., Carvalho, S. D. L., de Barros, R. D. C. L., Neves, M. L. D. S., Andrechuk, C. R. S., Alexandre, N. M. C., Ribeiro, P. A. B., & Rodrigues, R. C. M. (2023) · Systematic Reviews · doi:10.1186/s13643-023-02340-z · Cited by 24 · see citing works
Systematic Review and Meta-Analysis of the Relationship between Actual Exercise Intensity and Rating of Perceived Exertion in the Overweight and Obese Population
Yu, H., Sun, C., Sun, B., Chen, X., & Tan, Z. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph182412912 · Cited by 21 · see citing works
“For analyses where I2 was less than 50%, a fixed-effects model was used; if I2 was more than 50%, a random-effects model was employed [53].”
Influential
Automatic market research of mobile health apps for the self‐management of allergic rhinitis
Antó, A., Sousa‐Pinto, B., Czarlewski, W., Pfaar, O., Bosnic-Anticevich, S. Z., Klimek, L., Matricardi, P. M., Tripodi, S., Fonseca, J., Antó, J. M., et al. (2022) · Clinical & Experimental Allergy · doi:10.1111/cea.14135 · Cited by 21 · see citing works
The empty pelvis syndrome: a core data set from the PelvEx collaborative
Collaborative, P., Mirnezami, A. H., Drami, I., Glyn, T., Sutton, P. A., Tiernan, J. P., Behrenbruch, C., Guerra, G., Waters, P. S., Woodward, N., et al. (2024) · British journal of surgery · doi:10.1093/bjs/znae042 · Cited by 22 · see citing works
“Two studies were identified that calibrated the size of the empty pelvic cavity following PE to measure the size of appropriate breast implants for pelvic filling and reduction of dead space; however, these were not feasible and were inadequate using the COSMIN risk of bias tool 13,45,89”
Method
Guideline for reporting systematic reviews of outcome measurement instruments (OMIs): PRISMA-COSMIN for OMIs 2024
Elsman, E. B. M., Mokkink, L. B., Terwee, C. B., Beaton, D., Gagnier, J. J., Tricco, A. C., Baba, A., Butcher, N. J., Smith, M., Hofstetter, C., et al. (2024) · Health and Quality of Life Outcomes · doi:10.1186/s12955-024-02256-9 · Cited by 30 · see citing works
“The COSMIN Risk of Bias tool to assess the quality of studies on reliability or measurement error of OMIs can be used for any type of OMI [57]”
InfluentialSelf-citation
Measurement Techniques for Leg Length Discrepancy in Total Hip Arthroplasty: A Systematic Review of Reliability and Validity
Pettit, M. H., Kanavathy, S., McArthur, N., Weiss, O., & Khanduja, V. (2022) · The Journal of Arthroplasty · doi:10.1016/j.arth.2022.05.029 · Cited by 20 · see citing works
Psychometric properties of self-reported financial toxicity measures in cancer survivors: a systematic review
Zhu, Z., Xing, W., Wen, H., Sun, Y., So, W. K. W., Lizarondo, L. M., Peng, J. W., & Hu, Y. (2022) · BMJ Open · doi:10.1136/bmjopen-2021-057215 · Cited by 20 · see citing works
Identifying and appraising outcome measures for severe asthma: a systematic review
Rattu, A., Khaleva, E., Brightling, C. E., Dahlén, S., Bossios, A., Fleming, L., Chung, K. F., Melén, E., Djukanović, R., Chaudhuri, R., et al. (2022) · European Respiratory Journal · doi:10.1183/13993003.01231-2022 · Cited by 20 · see citing works
Within-subject variation of C-reactive protein and high-sensitivity C-reactive protein: A systematic review and meta-analysis
Gough, A., Sitch, A. J., Ferris, E., & Marshall, T. P. (2024) · PLoS ONE · doi:10.1371/journal.pone.0304961 · Cited by 22 · see citing works
Update on the psychometric properties for the Italian version of the Aachen Aphasia Test (IT-AAT)
Luzzatti, C., Bleser, R. D., Scola, I., Frustaci, M., & Willmes, K. (2022) · Aphasiology · doi:10.1080/02687038.2022.2037501 · Cited by 19 · see citing works
Psychometric properties of performance-based measures of physical function administered via telehealth among people with chronic conditions: A systematic review
Walsh, C. B., Cahalan, R., Hinman, R. S., & Sullivan, K. O. (2022) · PLoS ONE · doi:10.1371/journal.pone.0274349 · Cited by 19 · see citing works
“The methodological quality of each of the included studies was evaluated by two independent authors (CBW & KOS) using the COSMIN risk of bias checklist and scores were determined by consensus [34].”
Influential
Psychometric properties of functional postural control tests in children: A systematic review
Johnson, C., Hallemans, A., Goetschalckx, M., Meyns, P., Rameckers, E. A. A., Klingels, K., & Verbecque, E. (2023) · Annals of Physical and Rehabilitation Medicine · doi:10.1016/j.rehab.2022.101729 · Cited by 19 · see citing works
Knowledge user survey and Delphi process to inform development of a new risk of bias tool to assess systematic reviews with network meta-analysis (RoB NMA tool)
Lunny, C., Veroniki, A. A., Hutton, B., White, I. R., Higgins, J., Wright, J. M., Kim, J. Y., Thirugnanasampanthar, S. S., Siddiqui, S., Watt, J. A., et al. (2022) · BMJ Evidence-Based Medicine · doi:10.1136/bmjebm-2022-111944 · Cited by 18 · see citing works
A systematic review of validated assessments methods for head and neck lymphedema
Arends, C. R., Lindhout, J. E., van der Molen, L., Wilthagen, E. A., van den Brekel, M. W. M., & Stuiver, M. M. (2023) · European Archives of Oto-Rhino-Laryngology · doi:10.1007/s00405-023-07841-0 · Cited by 18 · see citing works
“For studies on clinician-rated instruments, we used the COSMIN Risk of Bias tool for assessing the quality of studies on reliability and measurement error of outcome measurement instruments [31].”
Method
Assessing parental emotion regulation in the context of parenting: A systematic review
Zhang, X.-T., Beatty, A. E., Abela, K. R., Melo, M. F., Kenny, M., Atkinson, L., & Gonzalez, A. (2023) · Developmental Review · doi:10.1016/j.dr.2023.101092 · Cited by 19 · see citing works
Measurement properties of cervical joint position error in people with and without neck pain: a systematic review and narrative synthesis
AlDahas, A., Heneghan, N. R., Althobaiti, S., Deane, J. A., Rushton, A. B., & Falla, D. (2024) · BMC Musculoskeletal Disorders · doi:10.1186/s12891-023-07111-4 · Cited by 18 · see citing works
“…reliability and measurement error of outcome measurement instruments as well as the COSMIN methodology for systematic reviews of Patient-Reported Outcome Measures (PROMs) [13, 14] and is reported in accordance with the Preferred Reporting Items for Systematic Review and Meta-Analysis (PRISMA) [15].”
MethodInfluential
The psychometric properties of the Quality of Life in Neurological Disorders (Neuro-QoL) measurement system in neurorehabilitation populations: a systematic review
Ataman, R., Alhasani, R., Auneau-Enjalbert, L., Quigley, A., Michael, H. U., & Ahmed, S. (2024) · Journal of Patient-Reported Outcomes · doi:10.1186/s41687-024-00743-7 · Cited by 17 · see citing works
“The overall rating of the methodological quality for a measurement property was based on the worst item rating [16, 24].”
Method
Reliability and Validity of Instrumented Timed Up and Go Test in Typical Adults and Elderly: A Systematic Review
Zhou, J., Yao, Q., Han, R., Bock, P. D., Vassard-Yu, G., Hallemans, A., & Laer, L. V. (2025) · Archives of Physical Medicine and Rehabilitation · doi:10.1016/j.apmr.2025.03.001 · Cited by 17 · see citing works
Software with artificial intelligence-derived algorithms for detecting and analysing lung nodules in CT scans: Systematic review and economic evaluation
Geppert, J., Auguste, P., Asgharzadeh, A., Ghiasvand, H., Patel, M., Brown, A., Jayakody, S., Helm, E., Todkill, D., Madan, J., et al. (2025) · Health Technology Assessment · doi:10.3310/jytw8921 · Cited by 16 · see citing works
Measurement Properties of the Mayo-Portland Adaptability Inventory (MPAI-4) and Related Measures: A Systematic Review
Ataman, R., Thomas, A., Roberge‐Dao, J., McKerral, M., Auger, C., Wittich, W., Talla, P. K., Boychuck, Z., & Ahmed, S. (2023) · Archives of Physical Medicine and Rehabilitation · doi:10.1016/j.apmr.2022.12.196 · Cited by 17 · see citing works
Measures of financial toxicity in cancer survivors: a systematic review
Thomy, L. B., Crichton, M., Jones, L. V., Yates, P. M., Hart, N. H., Collins, L. G., & Chan, R. J. (2024) · Supportive Care in Cancer · doi:10.1007/s00520-024-08601-4 · Cited by 17 · see citing works
“COSMIN criteria requires evaluation of relevance, comprehensive-ness, and comprehensibility in the assessment of content validity [15]].”
BackgroundInfluential
Clinically relevant estimation of minimal number of trials for the uncontrolled manifold analysis
Piscitelli, D., Buttram, A., Abernathy, K., Canelón, J., Knighten, D., & Solnik, S. (2024) · Journal of Biomechanics · doi:10.1016/j.jbiomech.2024.112195 · Cited by 17 · see citing works
Identifying spatial neglect – an updated systematic review of the psychometric properties of assessment tools in adults post-stroke
Williams, L. J., Loetscher, T., Hillier, S., Hreha, K. P., Jones, J., Bowen, A., & Kernot, J. (2024) · Neuropsychological Rehabilitation · doi:10.1080/09602011.2024.2346212 · Cited by 16 · see citing works
“This study is the first to report on the quality of the psychometric evidence for spatial neglect tools, as standards for evaluating the evidence such as COSMIN (Mokkink et al., 2020) have only recently been developed.”
Influential
The Moorong Self Efficacy Scale: translation, cultural adaptation, and validation in Italian; cross sectional study, in people with spinal cord injury
Márquez, M. A., Speroni, A., Galeoto, G., Ruotolo, I., Sellitto, G., Tofani, M., Gonzalez-Bernal, J. J., & Berardi, A. (2022) · Spinal Cord Series and Cases · doi:10.1038/s41394-022-00492-z · Cited by 16 · see citing works
Assessment of Manual Abilities Using the Box and Block Test in Children with Bilateral Cerebral Palsy
Zapata-Figueroa, V., & Ortiz‐Corredor, F. (2022) · Occupational Therapy International · doi:10.1155/2022/9980523 · Cited by 16 · see citing works
“From the perspective of the COSMIN standards for assessing the quality of studies on reliability and measurement error, the BBT is an example of a performance-based outcome measurement instrument (PerFOMs) in which a professional instructs a patient to comply with a specific manual task [30].”
Method
Quantification of breast lymphoedema following conservative breast cancer treatment: a systematic review
Fearn, N., Llanos, C., Dylke, E. S., Stuart, K. E., & Kilbreath, S. L. (2022) · Journal of Cancer Survivorship · doi:10.1007/s11764-022-01278-w · Cited by 16 · see citing works
Remote monitoring technologies for measuring cardiovascular functions in community-dwelling adults: a systematic review
Lu, J. K., Sijm, M., Janssens, G. E., Goh, J., & Maier, A. B. (2023) · GeroScience · doi:10.1007/s11357-023-00815-4 · Cited by 17 · see citing works
“The user manual of the COSMIN Risk of Bias tool was used to guide the reporting of validation outcomes [12].”
Method
Measures of attributes of locomotor capacity in older people: a systematic literature review following the COSMIN methodology
Honvo, G., Sabico, S., Veronese, N., Bruyère, O., Rizzoli, R., Thiyagarajan, J. A., Mikton, C. R., Diaz, T., Cooper, C. C., & Reginster, J. (2023) · Age and Ageing · doi:10.1093/ageing/afad139 · Cited by 16 · see citing works
Development, refinement, and validation of an equine musculoskeletal pain scale
Auer, U., Kelemen, Z., Vogl, C., von Ritgen, S., Haddad, R., Borda, L. T., Gabmaier, C., Breteler, J., & Jenner, F. (2024) · Frontiers in Pain Research · doi:10.3389/fpain.2023.1292299 · Cited by 16 · see citing works
The Long COVID Symptoms and Severity Score: Development, Validation, and Application
Ye, G., Zhu, Y., Bao, W., Zhou, H., Lai, J., Zhang, Y., Xie, J., Ma, Q., Luo, Z., Ma, S., et al. (2024) · Value in Health · doi:10.1016/j.jval.2024.04.009 · Cited by 16 · see citing works
Reproducibility of Microwave Breast Imaging: Analysis of Regular Scans of a Group of Volunteers
Wang, Z., Mojabi, P., Price, S., Lasemiimeni, Z., Besler, B. C., Bourqui, J., Deutscher, D., Besler, B. A., Shen, H., Docktor, B., et al. (2024) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2024.3512572 · Cited by 16 · see citing works
“The COSMIN Risk of Bias Tool was developed to assess the quality of studies addressing reliability and measurement variability [22], combining elements for an effective research question with 8-9 standards to assess the quality of a study.”
Evaluating a Muscle Ultrasound Education Program: Theoretical Knowledge, Hands-on Skills, Reliability, and Satisfaction of Critical Care Physiotherapists
González-Seguel, F., Pinto-Concha, J. J., Ríos-Castro, F., Silva-Gutiérrez, A., Camus-Molina, A., Mayer, K. P., & Parry, S. M. (2021) · Archives of Rehabilitation Research and Clinical Translation · doi:10.1016/j.arrct.2021.100142 · Cited by 16 · see citing works
Expert-Moderated Peer-to-Peer Online Support Group for People With Knee Osteoarthritis: Mixed Methods Randomized Controlled Pilot and Feasibility Study
Egerton, T., Lawford, B. J., Campbell, P. K., Plinsinga, M. L., Spiers, L., Mackenzie, D., Graham, B., Mills, K. A. G., Eyles, J. P., Knox, G., et al. (2021) · JMIR Formative Research · doi:10.2196/32627 · Cited by 16 · see citing works
Reply to the concerns raised by McKenna and Heaney about COSMIN
Mokkink, L. B., Terwee, C. B., de Vet, H. C. W., Bouter, L. M., Alonso, J., & Patrick, D. L. (2021) · Journal of Medical Economics · doi:10.1080/13696998.2021.1948231 · Cited by 16 · see citing works
Self-citation
The accuracy of Apple Watch measurements: a living systematic review and meta-analysis
Lambe, R., Baldwin, M., O’Grady, B., Schumann, M., Caulfield, B., & Doherty, C. (2026) · npj Digital Medicine · doi:10.1038/s41746-025-02238-1 · Cited by 17 · see citing works
A systematic review of the psychometric properties of Quebec user evaluation of satisfaction with assistive technology (QUEST)
Aledda, S., Galeoto, G., Fabbrini, G., Lucibello, L., Tofani, M., Conte, A., & Berardi, A. (2023) · Disability and Rehabilitation Assistive Technology · doi:10.1080/17483107.2022.2161648 · Cited by 15 · see citing works
“The review followed the 27-item Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) checklist [19] and COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) methodology for systematic reviews of patient-reported outcome measures (PROMs) [10].”
MethodInfluential
Reliability and validity of ultrasonography in evaluating the thickness, excursion, stiffness, and strain rate of respiratory muscles in non-hospitalized individuals: a systematic review
Zhou, E. F. M., Fu, S. N., Huang, C., Huang, X. P., & Wong, A. Y. L. (2023) · BMC Oral Health · doi:10.1186/s12903-023-03558-y · Cited by 15 · see citing works
“The methodological quality of the included studies was rated as “very good, adequate, doubtful, or inadequate” using the “worst-score counts” principle [31, 32].”
BackgroundMethod
Setting individualised goals for people living with dementia and their family carers: A systematic review of goal-setting outcome measures and their psychometric properties
Budgett, J., Sommerlad, A., Kupeli, N., Zabihi, S., Olsen, A., & Cooper, C. (2023) · Dementia · doi:10.1177/14713012231222309 · Cited by 15 · see citing works
“In this review, we follow the COnsensus-based Standards for the selection of health Measurement Instruments (COSMIN) guidelines (Mokkink et al., 2020), a comprehensive checklist designed to assess the psychometric properties and methodological quality of outcome measures.”
MethodInfluential
Reliability of Ultrasound Shear Wave Elastography for Evaluating Psoas Major and Quadratus Lumborum Stiffness: Gender and Physical Activity Effects
Zhou, E. F. M., Wong, A. Y. L., Zheng, Y., Lam, K. H. S., & Fu, S. N. (2024) · Ultrasound in Medicine & Biology · doi:10.1016/j.ultrasmedbio.2023.12.021 · Cited by 15 · see citing works
Clinimetric Properties of the Steep Ramp Test to Assess Cardiorespiratory Fitness, Its Underlying Physiological Responses, and Its Current Applications: A Scoping Review
Trul-Kreuze, I. A., Akkerman, M., Kieboom, E. A. M., Nieuwenhuis, M. K., Houdijk, H., & Bongers, B. C. (2024) · Archives of Physical Medicine and Rehabilitation · doi:10.1016/j.apmr.2024.02.717 · Cited by 15 · see citing works
Chronic pain interference assessment tools for children and adults who are unable to self‐report: A systematic review of psychometric properties
Smith, M. G., Farrar, L. C., Gibson, R. J., Russo, R. N., & Harvey, A. R. (2023) · Developmental Medicine & Child Neurology · doi:10.1111/dmcn.15535 · Cited by 15 · see citing works
Strengths and challenges of the COSMIN tools in outcome measures appraisal: A case example for speech–language therapy
Kwok, E. Y. L., Rosenbaum, P., Thomas‐Stonell, N., & Cunningham, B. J. (2021) · International Journal of Language & Communication Disorders · doi:10.1111/1460-6984.12603 · Cited by 14 · see citing works
Patient reported outcome measures assessing quality of life in patients with an intestinal stoma: A systematic review
Skibsted, C. V., Jensen, B. T., Juul, T., & Kristensen, H. Ø. (2022) · Colorectal Disease · doi:10.1111/codi.16202 · Cited by 14 · see citing works
“To determine the rating, the corresponding COSMIN Risk of Bias box was completed, with the lowest rating in the box deciding the overall rating in concordance with the user manual [30].”
Method
Inter-Rater Reliability of Delirium Screening of Infants in the Cardiac ICU: A Prospective, Observational Study
Cleveland, M. R., Baute, R., Clindaniel, C., Hertz, L., Pond, R., & Centers, G. I. (2023) · Pediatric Critical Care Medicine · doi:10.1097/pcc.0000000000003182 · Cited by 8 · see citing works
“Study procedures were designed following the Consensus-based Standards for the Selection of Health Measurement Instruments risk of bias tool (11) and reported following the Guidelines for Reporting Reliability and Agreement Studies (12).”
Method
Test-retest reliability of joint position sense in the elbow among healthy adults
Rider, J. V., & Valdes, K. A. (2023) · Journal of Hand Therapy · doi:10.1016/j.jht.2023.08.015 · Cited by 15 · see citing works

Additional file 1 of COSMIN Risk of Bias tool to assess the quality of studies on reliability or measurement error of outcome measurement instruments: a Delphi study
Mokkink, L. B., Boers, M., van der Vleuten, C. P. M., Bouter, L. M., Alonso, J., Patrick, D. L., de Vet, H. C. W., & Terwee, C. B. (2020) · Figshare · doi:10.6084/m9.figshare.13323854
Self-citation
COSMIN Delphi study on reliability and measurement errorPreprint
Mokkink, L. B. (2020) · OSF Preprints (OSF Preprints)
Self-citation
Risk of Bias tool to assess the quality of studies on reliability and measurement error of outcome measurement instruments
Mokkink, L. B., Boers, M., van der Vleuten, C. P. M., Bouter, L. M., Alonso, J., Patrick, D. L., de Vet, H. C. W., & Terwee, C. B. (2020) · Pure Amsterdam UMC
Self-citation
Mapping Tools for Risk of Bias Assessment in Reliability or Measurement Error Studies of Health Outcome Instruments: A Scoping Review ProtocolPreprint
Lopes, F. M., Silveira, L. C., De Sousa Utida, V. H., & Silva, L. T. (2024) · OSF Preprints (OSF Preprints) · doi:10.17605/osf.io/hm8ru
Studies on Reliability and Measurement Error of Measurements in Medicine – From Design to Statistics Explained for Medical Researchers
Mokkink, L. B., Eekhout, I. E., Boers, M., van der Vleuten, C. P. M., & de Vet, H. C. W. (2023) · Patient Related Outcome Measures · doi:10.2147/prom.s398886 · Cited by 90 · see citing works
Self-citation

Authors: Mokkink, L. B., Boers, M., van der Vleuten, C. P. M., Bouter, L. M., Alonso, J., Patrick, D. L., de Vet, H. C. W., & Terwee, C. B.
Affiliations: Amsterdam Public Health (NL)
Universitat Pompeu Fabra (ES)
University of Washington (US)
Corresponding author: Mokkink, L. B.
ORCID: Mokkink, L. B.
Boers, M.
Bouter, L. M.
Alonso, J.
Patrick, D. L.
Terwee, C. B.
Venue: BMC Medical Research Methodology
Publisher: Springer Science and Business Media LLC
Published: 2020-12
Volume: 20
Issue: 1
Pages: 293-293
ISSN: 1471-2288 (ISSN-L)
DOI: 10.1186/s12874-020-01179-5
PMC: PMC7712525
OpenAlex: W3107903990
Cited by: 588
References: 31
Citations by year: 2026: 106 · 2025: 121 · 2024: 149 · 2023: 101 · 2022: 72 · 2021: 37 · 2020: 2
FWCI: 32.32
Type: journal article
Language: en
Open access: gold
doi.org
Licence: cc-by
creativecommons.org/licenses/by/4.0/ (tdm)
creativecommons.org/licenses/by/4.0/ (vor, 2 d)
Locations: BMC Medical Research Methodology (publishedVersion, cc-by)
Research Square (acceptedVersion, cc-by)
PubMed Central (publishedVersion, cc-by)
PubMed (publishedVersion)
Research Publications (Maastricht University) (publishedVersion, cc-by)
VU Research Portal (publishedVersion, other-oa)
Pure Amsterdam UMC (submittedVersion, other-oa)
Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Amsterdam UMC (VU Amsterdam) - Institutional Repository (submittedVersion)
Repositori digital de la UPF (Universitat Pompeu Fabra) (submittedVersion, cc-by)
Research Publications (Maastricht University) (submittedVersion, other-oa)
Data Archiving and Networked Services (DANS) (submittedVersion)
Funders: Nederlandse Organisatie voor Wetenschappelijk Onderzoek (91617098)
Topics: Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
Social Sciences › Sociology and Political Science › Delphi Technique in Research
Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
Keywords: COSMIN, risk of bias, reliability statistics, measurement error, outcome measurement instruments, clinician-reported outcomes, performance-based outcome measures, laboratory parameters, study quality assessment, Delphi consensus, measurement properties
MeSH terms: Delphi Technique, Humans, Reproducibility of Results, Bias, Consensus

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

COSMIN Risk of Bias tool to assess the quality of studies on reliability or measurement error of outcome measurement instruments: a Delphi study

Abstract

Integrity signals

Claims the paper makes

Self-check

Cites

Cited by

Related

Record

About these checks