CoChat

Evaluation of the methodological quality of systematic reviews of health status measurement instruments

Open access

The DOI matches its Crossref record; 176 of the 178 references it cites were found in a registry or an index; 3 of the 37 sources its claims cite were checked for support.

Abstract

A systematic review of measurement properties of health-status instruments is a tool for evaluating the quality of instruments. Our aim was to appraise the quality of the review process, to describe how authors assess the methodological quality of primary studies of measurement properties, and to describe how authors evaluate results of the studies. Literature searches were performed in three databases. One hundred and forty-eight reviews were included. The purpose of included reviews was to identify health status instruments used in an evaluative application and to report on the measurement properties of these instruments. Two independent reviewers selected the articles and extracted the data. Reviews were often of low quality: 22% of the reviews used one database, the search strategy was often poorly described, and in many cases it was not reported whether article selection (75%) and data extraction (71%) was done by two independent reviewers. In 11 reviews the methodological quality of the primary studies was evaluated for all measurement properties, and of these 11 reviews only 7 evaluated the results. Methods to evaluate the quality of the primary studies and the results differed widely. The poor quality of reviews hampers evidence-based selection of instruments. Guidelines for conducting and reporting systematic reviews of measurement properties should be developed.

Overview

CoChat Summary

TL;DR

This paper appraises 148 systematic reviews of health status measurement instruments and finds that their methodological quality was often poor, especially for search reporting and the assessment of primary studies. The authors conclude that guidelines for conducting and reporting systematic reviews of measurement properties are needed.

Key findings

  • The search identified 7,779 records, of which 148 systematic reviews of measurement properties were included.
  • In 84% of reviews, the search strategy was described, but it was often limited; 22% used only one database. The databases most often used were PubMed (93%), PsycINFO (40%), CINAHL (39%), EMBASE (35%), and the Cochrane library (16%).
  • In 44% (65/148) of reviews, the methodological quality of the included studies was not assessed and the results were not appraised; in 56% (83/148), quality was (partly) assessed and some results were evaluated.
  • Among the 83 reviews that assessed quality/results, 53% (44/83) applied both standards and criteria of adequacy, 46% (38/83) applied only criteria of adequacy, and 1 review applied only standards. Only 11 reviews described and applied a complete set of fully reproducible standards, and of these 11, only 7 evaluated the results.

Methods

  • The authors searched PubMed (to March 2007), EMBASE (to March 2007), and PsycINFO (to June 2005), plus reference lists and authors’ own literature, to identify systematic reviews of measurement properties of health status instruments.
  • Included reviews had to claim to be systematic reviews, aim to identify all available health status measurement instruments for a specified population, concern instruments used in an evaluative/longitudinal context, and report on or evaluate measurement properties.
  • Two reviewers independently selected articles and extracted data on review characteristics, review-process quality, assessment of primary-study methodological quality, evaluation of results, and whether an overall conclusion was synthesized.

Limitations

  • The paper states that, because of the considerable variation in design, performance, and data presentation of the included reviews, some degree of judgement was unavoidable in appraising quality and describing standards.
  • The authors also note that they only used information from the published reviews and did not contact authors for additional information, so they may have slightly underrated review quality.
  • Evident from the text: the review was limited to English-language reviews and to reviews that performed at least one electronic database search, so relevant non-English or non-electronically searched reviews were excluded.

Model-written summary of a shortened version of the paper’s full text (it is longer than we read). Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1007/s11136-009-9451-9

Cites

176 of 176 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

3 Valid · 2 Not valid · 14 Not checked · 61 Own results and interpretations

41 more claims

Create a free account to see all 80 claims and to verify the 34 references still open.

See all claims — free account
  • Probably not a substantial claim. 21
  • Borderline: one of the tests for a substantial claim is unclear. 1
  • It may state two separate claims as one. 16

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 176 references

Research works printed but not linked: 1

  1. Higgins, J. P. T., & Green, S. (Eds.). (2006). Cochrane handbook for systematic reviews of interventions. Version 5.0.1 (update September 2008). The Cochrane Collaboration, 2008. Available from www.cochrane-handbook.org .

Cited by

See all 186

Metadata

Authors
Mokkink, L. B., Terwee, C. B., Stratford, P. W., Alonso, J., Patrick, D. L., Riphagen, I., Knol, D. L., Bouter, L. M., & de Vet, H. C. W.
Affiliations
  • Amsterdam Public Health (NL)
  • McMaster University (CA)
  • Municipal Institute for Medical Research (ES)
  • University of Washington (US)
  • Norwegian University of Science and Technology (NO)
Corresponding author
Mokkink, Lidwine B.
Venue
Quality of Life Research
Published
2009-02-24
Volume
18
Issue
3
Pages
313-333
ISSN
  • 0962-9343 (ISSN-L)
  • 1573-2649
OpenAlex
W1989226082
Cited by
186
References
178
Citations by year
2026: 2 · 2025: 6 · 2024: 17 · 2023: 9 · 2022: 13 · 2021: 5 · 2020: 10 · 2019: 10 · 2018: 24 · 2017: 21 · 2016: 16 · 2015: 11 · 2014: 12 · 2013: 7 · 2012: 8
FWCI
0.56
Type
journal article
Language
en
Open access
Licence
  • cc-by-nc
  • creativecommons.org/licenses/by-nc/2.0 (tdm)
Locations
  • Quality of Life Research (publishedVersion, cc-by-nc)
  • PubMed (publishedVersion)
  • VU Research Portal (publishedVersion, other-oa)
  • Pure Amsterdam UMC (submittedVersion)
  • Amsterdam UMC (VU Amsterdam) - Institutional Repository (submittedVersion)
  • Digital Academic REpository of VU University Amsterdam (Vrije Universiteit Amsterdam) (submittedVersion)
  • Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
Funders
Vrije Universiteit Amsterdam
Topics
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
  • Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Health Systems, Economic Evaluations, Quality of Life
  • Social Sciences › Sociology and Political Science › Delphi Technique in Research
Keywords
methodological quality, measurement properties, reporting quality, search strategies, inter-rater reliability, instrument selection, reporting guidelines
MeSH terms
Adolescent, Adult, Child, Child, Preschool, Evaluation Studies as Topic, Female, Health Status, Humans, Infant, Male, Middle Aged, Quality of Life, Surveys and Questionnaires, Young Adult

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Evaluation of the methodological quality of systematic reviews of health status measurement instruments. https://app.cochat.ai/papers/10.1007/s11136-009-9451-9
The paper
Mokkink, L. B., Terwee, C. B., Stratford, P. W., Alonso, J., Patrick, D. L., Riphagen, I., Knol, D. L., Bouter, L. M., & de Vet, H. C. W. (2009). Evaluation of the methodological quality of systematic reviews of health status measurement instruments. Quality of Life Research. https://doi.org/10.1007/s11136-009-9451-9

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.